Recent measurements - Continuation

POST {baseUrl}/v2/recent-datasource-measurements-continuation
Continues a previous request for recent measurements. Returns a list of recent measurements in time descending order.

Example
url: https://clarity-data-api.clarity.io/v2/recent-datasource-measurements-continuation
body:

{
    org: "myorg1234",
    continuationToken: "GH2J6...a long continuation token...A2EW1"
}

Continuations

A common access pattern is to routinely poll to get any new measurements since the last time you asked. It is awkward to accomplish this using startTime on your requests. Due to race conditions it is possible that you could miss a measurement, or get the same measurement twice. To make this easier, this endpoint accepts continuation tokens.

Your first call uses the recent-datasource-measurements-query endpoint and adds the additional parameter: replyWithContinuationToken: true. The response will include an x-clarity-continuation-token in the header.

On subsequent calls, this endpoint accepts that continuation token. The continuation response will pick up from the last call, returning newer measurements (if any exist) for the same datasources requested in the first call. An updated continuation token will be returned as well. A continuation token is good for 24 hours after it is issued.

Note: If the original call was for allDatasources in the org, then the continuations will include/exclude datasources as they are subscribed/unsubscribed.

Request Headers

parameter		description
x-api-key	string	The API key string.
Accept-Encoding	string	Encoding options, gzip is supported to compress data. Note: We strongly recommend including it in the headers to reduce response payload size. Example: Accept-Encoding: gzip

Request

body parameter		description
org	string required	Find the org ID by clicking on your user icon in Dashboard. Example org ID: "myorgVD43"
continuationToken	string	the token returned by the previous call

Response Headers

parameter		description
x-clarity-continuation-token	string	An updated continuation token

Response

see recent datasource measurement response formats

Example code

Polling for new measurements

The following sample Python code starts a continuation and then polls for new measurements, with each call picking up where the previous one left off. It requests json-long; for csv-wide read response.text instead of response.json() and read the continuation token from the header exactly the same way.

# polling for new measurements using continuation tokens

import requests
import os
import pprint
import time
from pathlib import Path


BASEURL = 'https://clarity-data-api.clarity.io'
HEADERS = {
    'Accept-Encoding': 'gzip',
    'x-api-key': os.environ.get('MY_CLARITY_API_KEY') # put your key in the environment or directly here
}
ORG_ID = '<your org id goes here>'  # you can find it in Dashboard under the menu item Organization>Resources

CONTINUATION_HEADER = 'x-clarity-continuation-token'
TOKEN_FILE = Path('clarity-continuation-token.txt')  # keep the newest token between runs
POLL_SECONDS = 300


def start_polling(org, output_frequency):
    # The first call is an ordinary recent measurements query that also asks for a token.
    # It returns the measurements already available; the token marks that point in time.
    url = BASEURL + "/v2/recent-datasource-measurements-query"
    request_body = {
        "org": org,
        "allDatasources": True,
        "outputFrequency": output_frequency,
        "format": "json-long",
        "replyWithContinuationToken": True,
    }
    response = requests.post(url, headers=HEADERS, json=request_body)
    response.raise_for_status()
    return response.json()["data"], response.headers[CONTINUATION_HEADER]


def poll_for_new_measurements(org, token):
    # Every later call goes to the continuation endpoint and picks up where the
    # previous call left off. Note there is no datasource list, outputFrequency or
    # format here -- the token remembers what the first call asked for.
    url = BASEURL + "/v2/recent-datasource-measurements-continuation"
    request_body = {
        "org": org,
        "continuationToken": token,
    }
    response = requests.post(url, headers=HEADERS, json=request_body)
    if response.status_code in (400, 404):
        return None, None  # token expired or not recognized -- time to start over
    response.raise_for_status()
    # Every response carries a NEW token. Always poll with the most recent one.
    return response.json()["data"], response.headers[CONTINUATION_HEADER]


def load_token():
    return TOKEN_FILE.read_text().strip() if TOKEN_FILE.exists() else None


token = load_token()

if token is None:
    measurements, token = start_polling(org=ORG_ID, output_frequency="minute")
    TOKEN_FILE.write_text(token)
    print(f"starting from {len(measurements)} measurements already available")

while True:
    time.sleep(POLL_SECONDS)

    measurements, token_after_poll = poll_for_new_measurements(org=ORG_ID, token=token)

    if token_after_poll is None:
        # A token is good for 24 hours after it is issued. Start a fresh one.
        measurements, token = start_polling(org=ORG_ID, output_frequency="minute")
        TOKEN_FILE.write_text(token)
        continue

    token = token_after_poll
    TOKEN_FILE.write_text(token)

    # An empty list is normal -- it means nothing new arrived since the last poll.
    if measurements:
        print(f"{len(measurements)} new measurements")
        pprint.pprint(measurements[:3])