Sync only the rows that changed
How to use the processDate argument of the Prescient GraphQL API so that a daily job copies new and updated rows and not the full history.
To keep a copy of your data current, set the processDate argument. The query then leaves out each row that was not created or updated on or after that date. Both modeledMetrics and reportedMetrics have this argument.
Why use processDate?
Section titled “Why use processDate?”A row can change after the first time you read it. A date range alone makes you read each row of the range again to find the changes. With processDate, the response has only the new rows and the changed rows, so the job reads less data.
The schema recommends processDate in each integration, and one pull of updates each day.
How do you set up a daily sync?
Section titled “How do you set up a daily sync?”-
Run a first pull with a date range that covers the history you need, and no
processDate. See Pull modeled metrics for a date range. -
Record the date on which the pull started.
-
In each later run, set
processDateto the date of the last run that finished. SetstartDateto the start of the history that you keep, because a changed row can have an oldreportedDate.Variables {"startDate": "2025-01-01","endDate": "2026-01-31","processDate": "2026-01-30","after": null} -
Follow the cursor until
hasNextPageisfalse. See Pagination. -
Write each row over the row in your copy that has the same key. Then record the date of this run.
Which fields identify a row?
Section titled “Which fields identify a row?”The API does not return a row ID, and the schema does not define a unique key for a row. Before you build the sync, choose the fields that identify a row in your copy and test them on your data.
- For
modeledMetrics, start withreportedDate,sourceChannelName,sourceCampaignId,target,metricName, andtargetChannelName. - For
reportedMetrics, start withreportedDate,channelName, andcampaignId.
Each row also has a processDate field, which is the date the row was created or last updated.
What should you know about the dates?
Section titled “What should you know about the dates?”processDateis a date, not a time. A row that changed on that date is in the response, so a run can return a row that the last run returned. Write over the row, and do not add it a second time.- If you leave out
startDate, the range starts 30 days before today. A changed row with an olderreportedDateis then not in the response.