Lens Documentation

API Reference 0.0.84

Captures

Below is a detailed explanation of the schema of the capture object, which is the resource of this API. Throughout this reference, required fields are bolded.

Captures Model

AttributeTypeDescription
created_atstring (date-time)The date/time when the capture was uploaded for processing. ISO 8601 format.
custom_dataobjectCustom data associated with the capture, such as an identifier to connect the capture to after processing. Any JSON-encodable values are allowed.
file_hashstringThe SHA-256, Base64-encoded hash of the file.
file_sizeintegerThe size of the file in bytes.
idstring (uuid)The ID of the uploaded capture.
label_detectionobjectThe labels detected in the photo. See Label Detection Object.
locationobjectThe reverse geocoded address of the verified GPS coordinates. See Location Object.
metadataobjectThe extracted metadata (EXIF, XMP, etc.). Only included if the metadata service is enabled.
{{group:tag}} mixed Each key is the metadata tag prepended with the group type, such as XMP:Make.
object_detectionobjectThe objects detected in the photo. See Object Detection Object.
processed_atstring (date-time)The date/time when the capture was processed. ISO 8601 format.
recaptureobjectWhether the photo was detected as a recapture. See Recapture Object.
reverse_image_searchobjectWhether a matching photo was found on the internet. See Reverse Image Search Object.
statusstringThe current processing state of the capture. Must be one of RECEIVING (Web SDK only), WAITING, PROCESSING, SUCCESS, or ERROR.
text_detectionobjectThe text detected in the photo. See Text Detection Object.
typestringThe type of capture. Must be one of PHOTO of VIDEO.
updated_atstring (date-time)The date/time when the capture was last updated. ISO 8601 format.
urlstring (uri)The URL of the original capture file. A token/signature is included in the query string that grants access for 10 hours. For a fresh token/signature with a new 10 hour expiration, simply re-fetch the capture.
verificationobjectThe extracted and verified details about the capture that are cryptographically signed into the file to prevent tampering. Contains multiple nested objects. See Verification Model below.
web_urlstring (uri)The URL of the capture file optimized for the web: 1000x1000px max, keeping aspect ratio intact. For video, this is a photo of the first frame. A token/signature is included in the query string that grants access for 10 hours. For a fresh token/signature with a new 10 hour expiration, simply re-fetch the capture.

Label Detection Object

The labels detected in the photo. Only included if the label_detection service is enabled.

AttributeTypeDescription
labelstringThe label
scorenumberA measure of confidence in the label actually being what the computer thinks it sees. The range is 0 to 1.

Location Object

The reverse geocoded address of the verified GPS coordinates, if specified. Only included if the location service is enabled.

AttributeTypeDescription
citystringThe city component of the address.
comparisonobjectThe results of calculating the distance between the verified GPS coordinates and another location, if enabled in the location service options.
addressstringThe original value specified in the location service options.
↳ distancenumberThe distance in meters between the verified GPS coordinates and the geocoded address. This does not take into account the accuracy of the GPS reading. null is returned if the address could not be geocoded to coordinates.
↳ distance_max_accuracynumberThe distance in meters adjusted higher to account for the accuracy of the GPS reading.
↳ distance_min_accuracynumberThe distance in meters adjusted lower to account for the accuracy of the GPS reading. The floor is 0 and does not go negative.
↳ distance_thresholdnumberThe original value specified in the location service options.
↳ distance_threshold_exceededbooleanWhether the distance_max_accuracy is greater than the distance_threshold. Only included if distance_threshold is set.
↳ …locationobjectThe same top-level location attributes such as city, state, etc., but for the comparison address.
countrystringThe ISO 3166-1 two-letter country code component of the address.
formatted_addressstringThe full address formatted appropriately for the locale.
latitudenumberThe verified GPS latitude (in decimal degrees) where the capture was taken.
longitudenumberThe verified GPS longitude (in decimal degrees) where the capture was taken.
postal_codestringThe postal code component of the address.
statestringThe state component of the address.
streetstringThe street component of the address.

Object Detection Object

The objects detected in the photo. Only included if the object_detection service is enabled.

AttributeTypeDescription
namestringThe name of the object.
scorenumberA measure of confidence in the object actually being what the computer thinks it sees. The range is 0 to 1.
bounding_polygonobjectThe bounding polygon where the object is located in the photo. Each object is a vertex representing a 2D point that’s relative to the original photo. For example, an x/y value of 0.5 is a point at the exact center of the original photo. Returns two attributes, x and y, both numbers, which represent the horizontal and vertical coordinates, respectively, relative to the original photo. The range for both is 0 to 1.

Recapture Object

Whether the photo was detected as a recapture. Only included if the recapture service is enabled.

AttributeTypeDescription
statusstringReturns one of the following values:
RECAPTURE the photo was detected as a recapture
PASS no recapture detected
WARNING a screen was detected
BLURRY blur_score <= 40
LOW_ENTROPY entropy_score <0.02
LOW_RESOLUTION (width * height <1000 * 1000)
SLOW_SHUTTERSPEED slower than 1/5
blur_scoreintegerA measure of the sharpness/blurriness of the photo using the variance of a Laplacian kernel. Typically in the range of 0 to 3,000, but there is no hard upper limit. A score <= 40 is considered blurry and sets the status to BLURRY.
entropy_scorenumberA measure of the entropy of the photo. A “flat” photo with no features (like a wall or ceiling without texture) has low entropy, while a photo with many features has high entropy. Typically in the range of 0 to 1.5, but there is no hard upper limit. A score < 0.02 is considered low entropy and sets the status to LOW_ENTROPY.
recapture_scorenumberA measure of the probability that the photo is a recapture. The range is 0 to 1. A score > 0.5 is considered a recapture and sets the status to RECAPTURE. If the score is borderline, but a screen was detected in the photo, then the status is set to WARNING.

Reverse Image Search Object

Whether a matching photo was found on the internet. Only included if the reverse_image_search service is enabled.

AttributeTypeDescription
statusstringreturns MATCH if a matching photo was found, otherwise NO_MATCH.
urlstrong (uri)If status is MATCH, returns URL of the matching photo, if NO_MATCH, returns null.

Text Detection Object

The text detected in the photo. Only included if the text_detection service is enabled.

AttributeTypeDescription
textstringThe string of text.
localenumberThe language code for the locale of the text, or an empty string if undetermined.
bounding_polygonobjectThe bounding polygon where the text is located in the photo. Each object is a vertex representing a 2D point that’s relative to the original photo. For example, an x/y value of 0.5 is a point at the exact center of the original photo. Returns two attributes, x and y, both numbers, which represent the horizontal and vertical coordinates, respectively, relative to the original photo. The range for both is 0 to 1.

Verification Model

The extracted and verified details about the capture that are cryptographically signed into the file to prevent tampering. Only included if the verification service is enabled.

AttributeTypeDescription
statusstringSUCCESS if the capture contains verified details that could be extracted, otherwise ERROR. Only the error object will be included if ERROR.
assertionsobjectThe assertions that are cryptographically signed into the file to prevent tampering. See Assertions Object.
certificateobjectThe details about the certificate and associated public/private key pair used to sign the assertions. See Certificate Object
producerobjectThe details about the software that produced the assertions.
name string The name and version of the software.
signatureobjectThe details about the signing of the assertions. See Signature Object
is_offlinebooleanWhether the capture was signed when the device had low or no connectivity and was unable to reach the TSA.
trusted_timestampobjectThe details about when the capture was signed, according to a trusted timestamp authority (TSA). See Trusted Timestamp Object
errorobjectThe error information if the status is ERROR. Contains
message string A helpful message explaining the error.

Assertions Object

The assertions that are cryptographically signed into the file to prevent tampering.

AttributeTypeDescription
date_timeobjectThe assertion details about the date/time of capture. Each contains:
status string The validation status of the date/time of capture. Must be one of VALID or INVALID.
status_reason string The status details if the status is INVALID.
timestamp string (date-time) The date/time of capture.
deviceobjectThe assertion details about the device. Contains:
status string The validation status of the device. Must be one of VALID or INVALID.
status_reason string The status details if the status is INVALID.
make string The make of the device.
model string The model of the device.
durationobjectThe assertion details about the duration of the video. Contains:
status string The validation status of the duration of the video. Must be one of VALID or INVALID.
status_reason string The status details if the status is INVALID.
seconds number The duration of the video in seconds. Includes milliseconds.
hashobjectThe assertion details about the hash of the capture “contents” (pixels, etc.).
status string The validation status of the hash of the capture “contents” (pixels, etc.). Must be one of VALID or INVALID.
status_reason string The status details if the status is INVALID.
locationobjectThe assertion details about the location. Contains:
status string The validation status of the location. Must be one of VALID or INVALID.
status_reason string The status details if the status is INVALID.
latitude number The GPS latitude (in decimal degrees) where the capture was taken.
longitude number The GPS longitude (in decimal degrees) where the capture was taken.
accuracy number The accuracy (in meters) of the GPS reading.
altitude number The GPS altitude (in meters) where the capture was taken.
timestamp string (date-time) The date/time of the GPS reading.
odometryobjectThe assertion details about the odometry/sensor readings. Contains:
status string The validation status of the odometry/sensor readings. Must be one of VALID or INVALID.
status_reason string The status details if the status is INVALID.
heading integer The direction (in degrees) that the camera was facing at time of capture. null if no odometry/sensor readings were recorded, or if the camera was facing straight up (ceiling) or down (floor).
thumbnailobjectThe assertion details about the thumbnail.
status string The validation status of the thumbnail. Must be one of VALID or INVALID.
status_reason string The status details if the status is INVALID.
url string (uri) The URL of the verified thumbnail file. A token/signature is included in the query string that grants access for 10 hours. For a fresh token/signature with a new 10 hour expiration, simply re-fetch the capture.

Certificate Object

The details about the certificate and associated public/private key pair used to sign the assertions.

AttributeTypeDescription
statusstringThe validation status of the certificate. Must be one of VALID or INVALID.
status_reasonstringThe status details if the status is INVALID.
issuer_namestringThe common name of the certificate’s issuer.
namestringThe common name from the certificate’s subject DN.
orgstringThe organization name from the certificate’s subject DN.
org_unitstringThe organizational unit’s name from the certificate’s subject DN.
valid_fromstring (date-time)The date/time from which the certificate is valid.
valid_tostring (date-time)The date/time until which the certificate is valid.

Signature Object

The details about the signing of the assertions.

AttributeTypeDescription
statusstringThe validation status of the signature. Must be one of VALID or INVALID.
status_reasonstringThe status details if the status is INVALID.
namestringThe name of the signer.

Trusted Timestamp Object

The details about when the capture was signed, according to a trusted timestamp authority (TSA).

AttributeTypeDescription
statusstringThe validation status of the timestamp. Must be one of VALID, OFFLINE, INVALID, or WARNING.
status_reasonstringThe status details if the status is OFFLINE, INVALID, or WARNING.
timestampstring (date-time)The date/time of signing. null if the status is OFFLINE, as the capture was signed when the device had low or no connectivity and was unable to reach the TSA.

Example Response for receiving status (Web SDK only)

{
  "created_at": "2022-01-28T17:47:10.950Z",
  "custom_data": {
    "session_id": 123
  },
  "file_hash": "cLBztEy/HJmLSPnn/BMdz2bisUJOVV4bbztQQiVy/rU=",
  "file_size": 6815727,
  "id": "8511d369-7646-4799-ac63-9f0503a06412",
  "processed_at": null,
  "status": "RECEIVING",
  "type": "PHOTO",
  "updated_at": "2022-01-28T17:47:10.950Z"
}

Example Response for waiting, processing, error statuses

{
  "created_at": "2022-01-28T17:47:10.950Z",
  "custom_data": {
    "session_id": 123
  },
  "file_hash": "cLBztEy/HJmLSPnn/BMdz2bisUJOVV4bbztQQiVy/rU=",
  "file_size": 6815727,
  "id": "8511d369-7646-4799-ac63-9f0503a06412",
  "processed_at": null,
  "status": "WAITING",
  "type": "PHOTO",
  "updated_at": "2022-01-28T17:47:10.950Z",
  "url": "https://s3.us-east-2.amazonaws.com/lens-captures-staging/8511d369-7646-4799-ac63-9f0503a06412.jpg?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Content-Sha256=UNSIGNED-PAYLOAD&X-Amz-Credential=ASIASOSZDTMLO4XDS6NZ%2F20220128%2Fus-east-2%2Fs3%2Faws4_request&X-Amz-Date=20220128T174711Z&X-Amz-Expires=36000&X-Amz-Security-Token=IQoJb3JpZ2luX2VjEK%2F%2F%2F%2F%2F%2F%2F%2F%2F%2F%2FwEaCXVzLWVhc3QtMiJIMEYCIQDMe6oh4zcrQ0084ueO2ybvmXCEj4bo5ss4c2VlyIsrwwIhAIUFG3yz758aW5RhygNlgoj9URYS8NU%2F7x2v%2BdhwUph0KoMECNj%2F%2F%2F%2F%2F%2F%2F%2F%2F%2FwEQAhoMMTY4NzY0MzQ5MjA2IgyZJVVfpMUXYsrsLNkq1wMxXV8M0SGmgj1QKMQ8ZhgI465PIvdsxl7mIGIFKFwvbWEfteME95s6w302LiIWWQt7kCs1QaNMNso4xAFEdWfz3Pz07n%2FO0MMUsqznsYLUe8eWzKw9e%2BR7QHrT5b%2BoECtF%2FF5cLAX4DdCd2UtpULtPQRL9wjEY2jMLfu8MeuWbUBNFICOhYwLjy8mVxM%2BunpBc0Vtz0DkoUS39R%2FLPPARMhDIxvkOtAMDsiAxSdSGXzfsSVFYQ0rtrJYmhY0MW3iXfjWbeDUEFziKo%2FF7TAPy8breTR5LXamf5lS04mhZ9jOGMOPkbfij9ZFzKP0ISj3NSmunEmWKxc2nEevvssfTuitYTk4WgtplNmIBtddyHBU3kq001YgcrJjt8uoT5RylN8BEVsABH55sM2oFN%2FjtC%2BGq6eadGdJL6kRSckH4%2BoqWqGhBO3d6s1VKmjJ2XxALhJRmKJhRwMiM%2FczlfKmQY4wdpfNYs9FSim6kAum5whTAG6rgHEGkFp1NhozJGgXoWdArJFR63yyzL9HLlrOu7pP5xAdi8VNl4pfTC%2FES5f9nEuri4NAfdbk7uTDljMdkXzJrEKfRBNPviK9gFJL%2BZ4MfXCZA%2B6qWQfclyaE9n6Pd8cSKntvkw9YLQjwY6pAHrfGSnAiNwAxShL7BBXkustbuMQcJYOvlZTCbmXs1PkLWzdhr3Os9tDz%2BED9cQrmWwj0D%2BQ2iFXqw9PH7UIF2vuUf9XYZDk9LrpLe%2B3JmUc5aOiLUAoR1YAU0PM%2ByXxyGBwO%2BmWnw0jJYx3MwBginL0vL7c8phC2G62JSZmpfeP7LvHzl73Z6MZEBqZi4%2FAKLB9LWRf6f1fws9STIZCfrjjbKP6w%3D%3D&X-Amz-Signature=f91af826a8bd0887c6bc7dfb4222cefa9f83deb6dec70af0eb0926da51c139b5&X-Amz-SignedHeaders=host&x-id=GetObject"
}

Example Response for success status

{
  "created_at": "2022-01-28T17:47:10.950Z",
  "custom_data": {
    "session_id": 123
  },
  "file_hash": "cLBztEy/HJmLSPnn/BMdz2bisUJOVV4bbztQQiVy/rU=",
  "file_size": 6815727,
  "id": "8511d369-7646-4799-ac63-9f0503a06412",
  "label_detection": [
    {
      "label": "Automotive parking light",
      "score": 0.9778186678886414
    },
    {
      "label": "Car",
      "score": 0.9718611836433411
    }
  ],
  "location": {
    "city": "San Diego",
    "country": "US",
    "formatted_address": "1205 Park Row, La Jolla, CA 92037, USA",
    "latitude": 32.8461131,
    "longitude": -117.2714113,
    "postal_code": "92037",
    "state": "CA",
    "street": "1205 Park Row"
  },
  "metadata": {
    "XMP:Make": "QUALCOMM",
    "XMP:Model": "RD8350 with Truepic Foresight Firmware v0.0.1",
    "...": "..."
  },
  "object_detection": [
    {
      "name": "Wheel",
      "score": 0.8580279350280762,
      "bounding_polygon": [
        { "x": 0.6226426959037781, "y": 0.26252779364585876 },
        { "x": 0.716256320476532, "y": 0.26252779364585876 },
        { "x": 0.716256320476532, "y": 0.40953460335731506 },
        { "x": 0.6226426959037781, "y": 0.40953460335731506 }
      ]
    },
    {
      "name": "Car",
      "score": 0.6619740724563599,
      "bounding_polygon": [
        { "x": 0.2538568675518036, "y": 0.133534237742424 },
        { "x": 0.3753494918346405, "y": 0.133534237742424 },
        { "x": 0.3753494918346405, "y": 0.5878967642784119 },
        { "x": 0.2538568675518036, "y": 0.5878967642784119 }
      ]
    }
  ],
  "processed_at": "2022-01-28T17:47:14.230Z",
  "recapture": {
    "status": "PASS",
    "blur_score": 76,
    "entropy_score": 0.17,
    "recapture_score": 0.002
  },
  "reverse_image_search": {
    "status": "MATCH",
    "url": "https://image.tmdb.org/t/p/original/7mGMvJZhT4hg4B7i5HyjYMjgaNM.jpg"
  },
  "status": "SUCCESS",
  "text_detection": [
    {
      "bounding_polygon": [
        { "x": 0.6011904761904762, "y": 0.43427579365079366 },
        { "x": 0.7060185185185185, "y": 0.4337797619047619 },
        { "x": 0.7060185185185185, "y": 0.45610119047619047 },
        { "x": 0.6011904761904762, "y": 0.4565972222222222 }
      ],
      "locale": "",
      "text": "Truepic"
    },
    {
      "bounding_polygon": [
        { "x": 0.23809523809523808, "y": 0.4885912698412698 },
        { "x": 0.2953042328042328, "y": 0.4885912698412698 },
        { "x": 0.2953042328042328, "y": 0.5037202380952381 },
        { "x": 0.23809523809523808, "y": 0.5037202380952381 }
      ],
      "locale": "",
      "text": "Nick"
    }
  ],
  "type": "PHOTO",
  "updated_at": "2022-01-28T17:47:14.230Z",
  "url": "https://s3.us-east-2.amazonaws.com/lens-captures-staging/8511d369-7646-4799-ac63-9f0503a06412.jpg?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Content-Sha256=UNSIGNED-PAYLOAD&X-Amz-Credential=ASIASOSZDTMLF6QGTMNF%2F20220128%2Fus-east-2%2Fs3%2Faws4_request&X-Amz-Date=20220128T174714Z&X-Amz-Expires=36000&X-Amz-Security-Token=IQoJb3JpZ2luX2VjELD%2F%2F%2F%2F%2F%2F%2F%2F%2F%2FwEaCXVzLWVhc3QtMiJIMEYCIQCVYsnhRsHh23pjf8sQQmeosTetmvqcLXbwI3Gqw19UBgIhALmAV8MPYgDhst2mjRJ9wdvIjhVbUNbt4TI9QEAwlWR%2BKoMECNn%2F%2F%2F%2F%2F%2F%2F%2F%2F%2FwEQAhoMMTY4NzY0MzQ5MjA2IgxFfdhho03DGMjQSlIq1wMUCuWE3lf8f%2FB0lEb1YbGJHoni62Oq7Awq8oYwyfNvjpd5t0%2BchiK7%2BsstmLmmS%2F8%2Bo2V7JWm%2Fp8%2BkbG%2Bzhv6mLxcgZz%2BsCbPJPwuxWalhVr6qfAZnjgNydtTZBxH9i7yOeI3O3qDnIBniMyHvMb599%2B000wPpfAqoQmqmH3Fk%2BaVTq3YzhCoH8mlszfc7EPG9tqQpKFLPe7l9bK787Y9FBtbCzyi9oneJJProLUJIDC97gTSWt%2FURDOL0EXN5vs6V7EwfxpefnYoSnEHiqdlwmGEr97u7F%2FijJNIAQnjkwd5PhLhDmlY1hlU6uwZKEzCIuDpDlvt1lUtsUp1pfNHFqvSZpTm5rotwn2sKbGUOCFGkMQMDQphFiP6KyY4QHqh%2F5sMB0UhTxhl7x7eOVhfEfkA5ku1iWB9U3pDXZypfXCMy9JBPtKzjfbMtt8B8c2Ruh280j9YBY7dzi8nsFnj5IhdCH4Y957ybT9B9y%2B%2BaFedtwMLiiOOr10IOccuUC3JmTbFfm21uJmaSNiigXW%2Bo4ac%2F6VFfAYzLx8%2F%2FTqVa0yt1BtIDWeL7MSiWbJ1894fdr1swTDeWXN5SoLzNlL8KtFMoXS%2BsTnnqc7GCv%2B26yxoAJTYH0O4wjJrQjwY6pAEcYVOF%2BiERQ1zoB7mleJOJsG6yBuedXMr2ZnIRcFxVLACf6xcukT%2Fb4%2BZW7I4zVvuUGtbBHUDW5F63yU5jQkoI8mWiQxvzqVhDVjOomvmNqX3cFBnrQq8aMwL9ByyFPY6eGnYsgTwZGgbSiv6mwVP4zBwIFHCArjkbNKfh%2F2n3BzDHVsY7gLG22NOcp1wQK6Kf7EyH3NvoyqyTrJFjx1%2FIuoJ1JQ%3D%3D&X-Amz-Signature=0b140e45d5a0af5d666ef1cc02fc4fce9ea5e542005710f4b778411d5387f279&X-Amz-SignedHeaders=host&x-id=GetObject",
  "verification": {
    "status": "SUCCESS",
    "assertions": {
      "date_time": {
        "status": "VALID",
        "timestamp": "2022-01-28T05:47:08Z"
      },
      "device": {
        "status": "VALID",
        "make": "Google",
        "model": "Pixel 5a"
      },
      "hash": {
        "status": "VALID"
      },
      "location": {
        "status": "VALID",
        "latitude": 32.8461131,
        "longitude": -117.2714113,
        "altitude": 123.8,
        "timestamp": "2022-01-28 17:47:02Z"
      },
      "odometry": {
        "status": "VALID",
        "heading": 43
      },
      "thumbnail": {
        "status": "VALID",
        "url": "https://s3.us-east-2.amazonaws.com/lens-captures-staging/8511d369-7646-4799-ac63-9f0503a06412_t.jpg?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Content-Sha256=UNSIGNED-PAYLOAD&X-Amz-Credential=ASIASOSZDTMLF6QGTMNF%2F20220128%2Fus-east-2%2Fs3%2Faws4_request&X-Amz-Date=20220128T174714Z&X-Amz-Expires=36000&X-Amz-Security-Token=IQoJb3JpZ2luX2VjELD%2F%2F%2F%2F%2F%2F%2F%2F%2F%2FwEaCXVzLWVhc3QtMiJIMEYCIQCVYsnhRsHh23pjf8sQQmeosTetmvqcLXbwI3Gqw19UBgIhALmAV8MPYgDhst2mjRJ9wdvIjhVbUNbt4TI9QEAwlWR%2BKoMECNn%2F%2F%2F%2F%2F%2F%2F%2F%2F%2FwEQAhoMMTY4NzY0MzQ5MjA2IgxFfdhho03DGMjQSlIq1wMUCuWE3lf8f%2FB0lEb1YbGJHoni62Oq7Awq8oYwyfNvjpd5t0%2BchiK7%2BsstmLmmS%2F8%2Bo2V7JWm%2Fp8%2BkbG%2Bzhv6mLxcgZz%2BsCbPJPwuxWalhVr6qfAZnjgNydtTZBxH9i7yOeI3O3qDnIBniMyHvMb599%2B000wPpfAqoQmqmH3Fk%2BaVTq3YzhCoH8mlszfc7EPG9tqQpKFLPe7l9bK787Y9FBtbCzyi9oneJJProLUJIDC97gTSWt%2FURDOL0EXN5vs6V7EwfxpefnYoSnEHiqdlwmGEr97u7F%2FijJNIAQnjkwd5PhLhDmlY1hlU6uwZKEzCIuDpDlvt1lUtsUp1pfNHFqvSZpTm5rotwn2sKbGUOCFGkMQMDQphFiP6KyY4QHqh%2F5sMB0UhTxhl7x7eOVhfEfkA5ku1iWB9U3pDXZypfXCMy9JBPtKzjfbMtt8B8c2Ruh280j9YBY7dzi8nsFnj5IhdCH4Y957ybT9B9y%2B%2BaFedtwMLiiOOr10IOccuUC3JmTbFfm21uJmaSNiigXW%2Bo4ac%2F6VFfAYzLx8%2F%2FTqVa0yt1BtIDWeL7MSiWbJ1894fdr1swTDeWXN5SoLzNlL8KtFMoXS%2BsTnnqc7GCv%2B26yxoAJTYH0O4wjJrQjwY6pAEcYVOF%2BiERQ1zoB7mleJOJsG6yBuedXMr2ZnIRcFxVLACf6xcukT%2Fb4%2BZW7I4zVvuUGtbBHUDW5F63yU5jQkoI8mWiQxvzqVhDVjOomvmNqX3cFBnrQq8aMwL9ByyFPY6eGnYsgTwZGgbSiv6mwVP4zBwIFHCArjkbNKfh%2F2n3BzDHVsY7gLG22NOcp1wQK6Kf7EyH3NvoyqyTrJFjx1%2FIuoJ1JQ%3D%3D&X-Amz-Signature=985473ff16f05414b3e267a6fabc1b2b58b724084a1624fe571fbe33c3f2a086&X-Amz-SignedHeaders=host&x-id=GetObject"
      }
    },
    "certificate": {
      "status": "VALID",
      "issuer_name": "AndroidClaimSigningCA",
      "name": "Truepic Lens SDK v0.1.0 in Test App v1.0.0",
      "org": "Test Org",
      "org_unit": "Test Org Unit",
      "valid_from": "2022-01-28T17:37:06Z",
      "valid_to": "2022-01-29T17:37:05Z"
    },
    "is_offline": false,
    "producer": {
      "name": "Truepic Lens SDK v0.1.0"
    },
    "signature": {
      "status": "VALID",
      "name": "Test Org Unit"
    },
    "trusted_timestamp": {
      "status": "VALID",
      "timestamp": "2022-01-28T17:47:09Z"
    }
  },
  "web_url": "https://s3.us-east-2.amazonaws.com/lens-captures-staging/8511d369-7646-4799-ac63-9f0503a06412_w.jpg?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Content-Sha256=UNSIGNED-PAYLOAD&X-Amz-Credential=ASIASOSZDTMLF6QGTMNF%2F20220128%2Fus-east-2%2Fs3%2Faws4_request&X-Amz-Date=20220128T174714Z&X-Amz-Expires=36000&X-Amz-Security-Token=IQoJb3JpZ2luX2VjELD%2F%2F%2F%2F%2F%2F%2F%2F%2F%2FwEaCXVzLWVhc3QtMiJIMEYCIQCVYsnhRsHh23pjf8sQQmeosTetmvqcLXbwI3Gqw19UBgIhALmAV8MPYgDhst2mjRJ9wdvIjhVbUNbt4TI9QEAwlWR%2BKoMECNn%2F%2F%2F%2F%2F%2F%2F%2F%2F%2FwEQAhoMMTY4NzY0MzQ5MjA2IgxFfdhho03DGMjQSlIq1wMUCuWE3lf8f%2FB0lEb1YbGJHoni62Oq7Awq8oYwyfNvjpd5t0%2BchiK7%2BsstmLmmS%2F8%2Bo2V7JWm%2Fp8%2BkbG%2Bzhv6mLxcgZz%2BsCbPJPwuxWalhVr6qfAZnjgNydtTZBxH9i7yOeI3O3qDnIBniMyHvMb599%2B000wPpfAqoQmqmH3Fk%2BaVTq3YzhCoH8mlszfc7EPG9tqQpKFLPe7l9bK787Y9FBtbCzyi9oneJJProLUJIDC97gTSWt%2FURDOL0EXN5vs6V7EwfxpefnYoSnEHiqdlwmGEr97u7F%2FijJNIAQnjkwd5PhLhDmlY1hlU6uwZKEzCIuDpDlvt1lUtsUp1pfNHFqvSZpTm5rotwn2sKbGUOCFGkMQMDQphFiP6KyY4QHqh%2F5sMB0UhTxhl7x7eOVhfEfkA5ku1iWB9U3pDXZypfXCMy9JBPtKzjfbMtt8B8c2Ruh280j9YBY7dzi8nsFnj5IhdCH4Y957ybT9B9y%2B%2BaFedtwMLiiOOr10IOccuUC3JmTbFfm21uJmaSNiigXW%2Bo4ac%2F6VFfAYzLx8%2F%2FTqVa0yt1BtIDWeL7MSiWbJ1894fdr1swTDeWXN5SoLzNlL8KtFMoXS%2BsTnnqc7GCv%2B26yxoAJTYH0O4wjJrQjwY6pAEcYVOF%2BiERQ1zoB7mleJOJsG6yBuedXMr2ZnIRcFxVLACf6xcukT%2Fb4%2BZW7I4zVvuUGtbBHUDW5F63yU5jQkoI8mWiQxvzqVhDVjOomvmNqX3cFBnrQq8aMwL9ByyFPY6eGnYsgTwZGgbSiv6mwVP4zBwIFHCArjkbNKfh%2F2n3BzDHVsY7gLG22NOcp1wQK6Kf7EyH3NvoyqyTrJFjx1%2FIuoJ1JQ%3D%3D&X-Amz-Signature=812c04ccf08635fa6e858911b5fb77270a7add3310e602df576cbf9229a9b2e3&X-Amz-SignedHeaders=host&x-id=GetObject"
}

Authentication

Lens uses API keys to authenticate with its API. Each org unit can have one or more API keys, and each one can be scoped to the exact permissions you desire. This allows you to create a separate key for each platform, environment, or use — whatever makes sense for your setup.

API keys should be treated as sensitive secrets and stored in a secure way outside of version control. If you suspect a key has been compromised, it can be temporarily disabled or completely revoked to immediately prevent further access.

At this time, API keys are managed internally by the Truepic team, so reach out to us if you’d like one created, disabled, or revoked.

Once you have an API key, it should be included in the authorization header of every request as a Bearer token.

Rate Limiting

The Lens API allows up to 120 requests per minute based on IP address. A number of headers are included with every response based on the IETF HTTPAPI Working Group’s Rate Limit header fields for HTTP draft:

HeaderTypeDescription
ratelimit-limitintegerThe current request rate limit per minute (120).
ratelimit-remainingintegerThe number of requests remaining before rate limiting is triggered.
ratelimit-resetintegerThe number of seconds before the rate limit window is reset.
retry-afterintegerThe same value as ratelimit-reset, but only included once rate limiting has been triggered.

These headers provide a real-time view into the current state of rate limiting, and can be used to programmatically build rate limiting into your integration.

When rate limiting has been triggered, a 429 Too Many Requests error is returned with the following payload:

{
  "error": {
    "message": "Rate limit exceeded, retry in 1 minute",
    "status": 429
  }
}

POST Upload Capture

https://lens-api.truepic.com/captures

Upload a capture to be processed.

Headers

Content-Type: multipart/form-data
Authorization: Bearer {{ api_key }}

Body

options object
JSON-encoded options to configure how the file is processed. Each org unit can have a default set of options configured, in which case this field only needs to be included to override the default. If no default is set, this field is required.

AttributeTypeDescriptionDefaultSupported Types
dedupbooleanWhether to prevent duplicate uploads (based on file hash). The existing record is returned if found.falsePhoto, Video
label_detectionbooleanWhether to detect labels in the photo.falsePhoto
locationboolean, objectWhether to reverse geocode the verified GPS coordinates to an address. Use an object to specify additional options.falsePhoto, Video
↳ comparisonobjectCalculate the distance between the verified GPS coordinates and another location, such as where you expect it to originate from.
↳↳ addressstringThe address to compare against. A partial, loosely-formatted address is acceptable, but may not yield as accurate of a result as a specific, well-formatted address.
↳↳ distance_thresholdnumberThe maximum distance to allow in meters between the two locations before flagging as exceeded. If not set, the distance is simply calculated.
metadatabooleanWhether to extract metadata (EXIF, XMP, etc.).falsePhoto, Video
object_detectionbooleanWhether to detect objects in the photo.falsePhoto
recapturebooleanWhether to check if the photo has been recaptured.falsePhoto
reverse_image_searchbooleanWhether to search the internet for a matching photo.falsePhoto
text_detectionbooleanWhether to detect text in the photo.falsePhoto
verificationbooleanWhether to extract and verify the cryptographically signed details about the capture.falsePhoto, Video

custom_data object
JSON-encoded custom data to associate with the capture, such as an identifier to connect the capture to after processing. Any JSON-encodable values are allowed.

file binary
The JPEG or MPEG-4 file to process. IMPORTANT: Due to the nature of multipart/form-data, be sure to add the file part last! There is currently a 100 MB per file limit for video uploads to the API.

Example Request

curl

curl --request POST \
  --url https://lens-api.truepic.com/captures \
  --header 'Authorization: Bearer ...' \
  --header 'Content-Type: multipart/form-data' \
  --form file=@capture.jpg \

Example Response

{
  "capture": {
    "id": "8511d369-7646-4799-ac63-9f0503a06412",
    "...": "..."
  }
}

GET List Captures

https://lens-api.truepic.com/captures

Get a list of uploaded captures.

Headers

Authorization: Bearer {{ api_key }}

Body

query query
QueryQL is used for filtering, sorting, and pagination on this route. See our QueryQL documentation on GitHub for general use information.

Filter: status [=, !=, in, not in] – Must be one of WAITING, PROCESSING, SUCCESS, or ERROR.
Sort: created_at – Defaults to desc.
Pagination: size – Defaults to 20 and is limited to a max of 100.

Example Request

curl

curl --request GET \
  --url https://lens-api.truepic.com/captures \
  --header 'Authorization: Bearer ...' \
   --data '"filter[status][\=]"==SUCCESS'

Example Response

{
  "captures": [
    {
      "id": "8511d369-7646-4799-ac63-9f0503a06412",
      "...": "..."
    },
    {
      "id": "dae05f02-2f5b-4ea6-9d09-781e4f99664a",
      "...": "..."
    }
  ]
}

GET Uploaded Capture

https://lens-api.truepic.com/captures/{id}

Get an uploaded capture.

Path Params

id string (uuid)
The ID of the uploaded capture.

Headers

Authorization: Bearer {{ api_key }}

Example Request

curl

curl --request GET \
  --url https://lens-api.truepic.com/captures/{{id}} \
  --header 'Authorization: Bearer ...' \

Example Response

{
  "capture": {
    "id": "8511d369-7646-4799-ac63-9f0503a06412",
    "...": "..."
  }
}

PUT Update Custom Data

https://lens-api.truepic.com/captures/{id}

Update an uploaded capture’s custom data.

Path Params

id string (uuid)
The ID of the uploaded capture.

Headers

Authorization: Bearer {{ api_key }}

Body

custom_data object
Custom data to associate with the capture, such as an identifier to connect the capture to after processing. Any JSON-encodable values are allowed.

Example Request

cURL

curl --request PUT \
  --url https://lens-api.truepic.com/captures/{{id}} \
  --header 'Authorization: Bearer ...' \
  --data-raw `{
    "custom_data": {
      "session_id": 456
    }

Example Response

{
  "capture": {
    "id": "8511d369-7646-4799-ac63-9f0503a06412",
    "...": "..."
  }
}

DELETE Remove Capture

https://lens-api.truepic.com/captures/{id}

Delete an uploaded capture and all associated files.

IMPORTANT: The request is placed in a queue and not processed instantly, so the data and associated files will still be accessible for a short period of time. Captures that are waiting to be processed (or currently being processed) may still be processed before deletion.

Path Params

id string (uuid)
The ID of the uploaded capture.

Headers

Authorization: Bearer {{ api_key }}

Example Request

curl

curl --request DELETE \
  --url https://lens-api.truepic.com/captures/{{id}} \
  --header 'Authorization: Bearer ...' \

Example Response

An empty object is returned to allow the response to be parsed as JSON, if necessary.

{}

Errors

All 4xx and 5xx errors from the application respond with a common payload structure. Errors from outside of the application – like hosting infrastructure – do not, but should be uncommon.

4xx-5xx application/json

  • error object – The error information.
    • message string – A helpful message explaining the error.
    • status integer – The HTTP status code of the response.
{
  "error": {
    "message": "File must be of type `image/jpeg`",
    "status": 422
  }
}

List of 4xx Errors

StatusDescription
400Invalid input in params, query string, body, etc.
401Missing or invalid API key for authentication.
403The API key does not have permission.
404The requested resource was not found.
406The request is not multipart/form-data.
413The request payload is too large.
422Invalid input in params, query string, body, etc.
429Rate limiting has been triggered.

Webhooks

Lens uses webhooks to notify you when a capture has been created, updated, processed, and deleted. Each org unit can have one or more webhooks configured, and each one can be scoped to the exact types you desire. This allows you to receive the results everywhere you need.

At this time, webhooks are managed internally by the Truepic team, so reach out to us if you’d like one created (or disabled/deleted)!

Request

Lens makes a POST request to each of your configured webhook URLs with the following payload structure:

application/json

  • type string – The unique identifier of the webhook type. Must be one of captures.notified (Web SDK only), captures.created, captures.updated, captures.processed, or captures.deleted.
  • data object – The payload specific to the webhook type. See examples below.

Response

Your server only needs to respond with a 2xx for Lens to consider it successfully delivered. No response body is necessary.

A 4xx or 5xx error response will cause Lens to retry the request later (up to 5 times).

Types

captures.notified

At this time, only uploads directly from the Web SDK trigger this type.

{
  "type": "captures.notified",
  "data": {
    "created_at": "2022-01-28T17:47:10.950Z",
    "custom_data": {
      "session_id": 123
    },
    "file_hash": "cLBztEy/HJmLSPnn/BMdz2bisUJOVV4bbztQQiVy/rU=",
    "file_size": 6815727,
    "id": "8511d369-7646-4799-ac63-9f0503a06412",
    "processed_at": null,
    "status": "RECEIVING",
    "type": "PHOTO",
    "updated_at": "2022-01-28T17:47:10.950Z"
  }
}

captures.created

{
  "type": "captures.created",
  "data": {
    "created_at": "2022-01-28T17:47:10.950Z",
    "custom_data": {
      "session_id": 123
    },
    "file_hash": "cLBztEy/HJmLSPnn/BMdz2bisUJOVV4bbztQQiVy/rU=",
    "file_size": 6815727,
    "id": "8511d369-7646-4799-ac63-9f0503a06412",
    "processed_at": null,
    "status": "WAITING",
    "type": "PHOTO",
    "updated_at": "2022-01-28T17:47:10.950Z",
    "url": "https://s3.us-east-2.amazonaws.com/lens-captures-staging/8511d369-7646-4799-ac63-9f0503a06412.jpg?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Content-Sha256=UNSIGNED-PAYLOAD&X-Amz-Credential=ASIASOSZDTMLO4XDS6NZ%2F20220128%2Fus-east-2%2Fs3%2Faws4_request&X-Amz-Date=20220128T174711Z&X-Amz-Expires=36000&X-Amz-Security-Token=IQoJb3JpZ2luX2VjEK%2F%2F%2F%2F%2F%2F%2F%2F%2F%2F%2FwEaCXVzLWVhc3QtMiJIMEYCIQDMe6oh4zcrQ0084ueO2ybvmXCEj4bo5ss4c2VlyIsrwwIhAIUFG3yz758aW5RhygNlgoj9URYS8NU%2F7x2v%2BdhwUph0KoMECNj%2F%2F%2F%2F%2F%2F%2F%2F%2F%2FwEQAhoMMTY4NzY0MzQ5MjA2IgyZJVVfpMUXYsrsLNkq1wMxXV8M0SGmgj1QKMQ8ZhgI465PIvdsxl7mIGIFKFwvbWEfteME95s6w302LiIWWQt7kCs1QaNMNso4xAFEdWfz3Pz07n%2FO0MMUsqznsYLUe8eWzKw9e%2BR7QHrT5b%2BoECtF%2FF5cLAX4DdCd2UtpULtPQRL9wjEY2jMLfu8MeuWbUBNFICOhYwLjy8mVxM%2BunpBc0Vtz0DkoUS39R%2FLPPARMhDIxvkOtAMDsiAxSdSGXzfsSVFYQ0rtrJYmhY0MW3iXfjWbeDUEFziKo%2FF7TAPy8breTR5LXamf5lS04mhZ9jOGMOPkbfij9ZFzKP0ISj3NSmunEmWKxc2nEevvssfTuitYTk4WgtplNmIBtddyHBU3kq001YgcrJjt8uoT5RylN8BEVsABH55sM2oFN%2FjtC%2BGq6eadGdJL6kRSckH4%2BoqWqGhBO3d6s1VKmjJ2XxALhJRmKJhRwMiM%2FczlfKmQY4wdpfNYs9FSim6kAum5whTAG6rgHEGkFp1NhozJGgXoWdArJFR63yyzL9HLlrOu7pP5xAdi8VNl4pfTC%2FES5f9nEuri4NAfdbk7uTDljMdkXzJrEKfRBNPviK9gFJL%2BZ4MfXCZA%2B6qWQfclyaE9n6Pd8cSKntvkw9YLQjwY6pAHrfGSnAiNwAxShL7BBXkustbuMQcJYOvlZTCbmXs1PkLWzdhr3Os9tDz%2BED9cQrmWwj0D%2BQ2iFXqw9PH7UIF2vuUf9XYZDk9LrpLe%2B3JmUc5aOiLUAoR1YAU0PM%2ByXxyGBwO%2BmWnw0jJYx3MwBginL0vL7c8phC2G62JSZmpfeP7LvHzl73Z6MZEBqZi4%2FAKLB9LWRf6f1fws9STIZCfrjjbKP6w%3D%3D&X-Amz-Signature=f91af826a8bd0887c6bc7dfb4222cefa9f83deb6dec70af0eb0926da51c139b5&X-Amz-SignedHeaders=host&x-id=GetObject"
  }
}

captures.updated

{
  "type": "captures.updated",
  "data": {
    "id": "8511d369-7646-4799-ac63-9f0503a06412",
    "...": "..."
  }
}

captures.processed

{
  "type": "captures.processed",
  "data": {
    "id": "8511d369-7646-4799-ac63-9f0503a06412",
    "...": "..."
  }
}
{
  "type": "captures.processed",
  "data": {
    "error": {
      "message": "File must be of type `image/jpeg`",
      "status": 422
    }
  }
}

captures.deleted

{
  "type": "captures.deleted",
  "data": {
    "id": "8511d369-7646-4799-ac63-9f0503a06412",
    "...": "..."
  }
}

Verifying Requests

Webhook requests can be verified to ensure they’re actually coming from Lens and not some potentially malicious bad actor. While this is completely optional, we recommend it for the increased security – and it’s not difficult to implement.

Each request from Lens includes a truepic-signature header that looks like this:

truepic-signature: t=1634066973,s=6FBEiVZ8EO79dk5XllfnG18b83ZvLt2kdxcE8FJ/BwU

As you can see, there’s a t value that’s the timestamp of when the request was sent, and an s value that’s the signature of the request.

Let’s walk through how to parse and verify this with examples in Node.js that should be easy enough to translate to the language/framework of your backend.

Parse the Header Value

To start, split the header value on the comma (,). This should leave you with two parts:

  • t=1634066973
  • s=6FBEiVZ8EO79dk5XllfnG18b83ZvLt2kdxcE8FJ/BwU

Next, split each part on the equals (=). This should leave you with two values for each part:

  • t
  • 1634066973
  • s
  • 6FBEiVZ8EO79dk5XllfnG18b83ZvLt2kdxcE8FJ/BwU

Here’s an example showing how to do that in Node.js with some validation along the way:

function parseSignatureHeader(header) {
  const error = new Error('Invalid truepic-signature header')

  if (!header?.length) {
    throw error
  }

  const [timestampParts, signatureParts] = header.split(',')

  if (!timestampParts?.length || !signatureParts?.length) {
    throw error
  }

  let [t, timestamp] = timestampParts.split('=')

  if (t !== 't' || !timestamp?.length) {
    throw error
  }

  timestamp = Number(timestamp)

  if (isNaN(timestamp)) {
    throw error
  }

  const [s, signature] = signatureParts.split('=')

  if (s !== 's' || !signature?.length) {
    throw error
  }

  return { timestamp, signature }
}

Verify the Timestamp

The timestamp (t) can be verified to ensure it’s a recent request and not a potentially delayed replay attack. Some leeway should be allowed in case the clocks on either end of the request aren’t quite in sync.

Here’s an example in Node.js with 5 minutes of leeway:

function verifyTimestamp({ timestamp, leewayMinutes = 5 }) {
  const diff = Math.abs(Date.now() - timestamp * 1000)
  const diffMinutes = Math.ceil(diff / (1000 * 60))

  return leewayMinutes >= diffMinutes
}

Verify the Signature

The signature (s) can be verified to ensure the recipient and payload haven’t been tampered with. This is done by rebuilding the signature – a HMAC digest using SHA-256 that’s Base64-encoded – with a secret that only Lens and you are privy to.

To create the message to sign, join your webhook’s URL together with the timestamp (t) and the raw request body using a comma (,). It’s important to use the raw request body (a string) before it’s parsed as JSON, as different languages/frameworks can parse/stringify JSON in subtly different ways, which could result in different signatures.

Next, sign the {{url}},{{timestamp}},{{body}} message with the secret and compare the Base64-encoded signatures with a constant-time algorithm to prevent timing attacks.

Here’s an example in Node.js:

import { createHmac, timingSafeEqual } from 'crypto'

function verifySignature({ url, timestamp, body, secret, signature }) {
  const comparisonSignature = createHmac('sha256', secret)

  comparisonSignature.update([url, timestamp, body].join(','))

  return timingSafeEqual(
    Buffer.from(comparisonSignature.digest('base64'), 'base64'),
    Buffer.from(signature, 'base64')
  )
}

Putting It All Together

Now that we have functions to parse and verify the truepic-signature header, let’s show the pieces working together. Here’s an example in Node.js with a few hard-coded values that should ultimately be pulled from elsewhere:

// These values should be stored in environment variables for security.
const URL = 'YOUR_WEBHOOK_URL'
const SECRET = 'YOUR_WEBHOOK_SECRET'

// These values should come from the request.
const header = 't=1634066973,s=6FBEiVZ8EO79dk5XllfnG18b83ZvLt2kdxcE8FJ/BwU'
const body =
  '{"type":"captures.processed","data":{"id":"8511d369-7646-4799-ac63-9f0503a06412","...":"..."}}'

const { timestamp, signature } = parseSignatureHeader(header)

const isTimestampVerified = verifyTimestamp({ timestamp })

if (!isTimestampVerified) {
  throw new Error('Invalid timestamp')
}

const isSignatureVerified = verifySignature({
  url: URL,
  timestamp,
  body,
  secret: SECRET,
  signature,
})

if (!isSignatureVerified) {
  throw new Error('Invalid signature')
}

// At this point, the webhook request is verified to be from Lens!

Code Copied

Glossary

Search Terms

  • API

  • C2PA

  • CAI

  • Controlled Capture

  • Entropy

  • EXIF

  • Geocoded

  • IDE

  • JPEG

  • JSON

  • Lens SDK

  • Provenance

  • Recapture

  • Reverse Image Search

  • XMP