Every Core dna site answers three kinds of API request under /cdna-api: REST for everything, GraphQL for reads that span several entity types, and web events for endpoints you define yourself as jobs. Which fits which task, one worked request for each, and what is true of all three. Jobs were called hooks until the module was renamed Orchestration; the API's routes still use the old name.
L1–L3
Unit 06 · Client code and imports
Bulk reads and failure handling
Client code and importsPage 24 of 25Updated
Settings that let a sync or an export read a whole catalogue without slowing a production site, an incremental read that pages by id, and what to do when a run fails halfway: which requests to retry, which to read before resending, and what to log.
Who this is for. The integrations engineer running a nightly sync, an export or a large import against a live site. Afterwards you can size and pace the requests so visitors do not notice the run, read only what changed since the last one, and recover from a failure by running again. It builds on The external-id pattern, which makes the second run safe.
What the platform does and does not limit
Pace your own API client: a job shares the site with every visitor, so size and space its requests yourself. What the API does limit is the size of a list:
Most lists have no default limit. A list without limit returns every match in one response.
Job lists and custom entity lists answer 100 records when no limit is sent, and refuse a limit above 500 with a 400, “The limit may not exceed 500”; the job runs list allows up to 1,000.
Settings that leave a production site unaffected
Setting
Value
Page size
100 to 250 records, with properties naming only the fields the job uses
Concurrency
One request at a time. Parallel pages save little and multiply the load
Pacing
A 250 ms pause between pages during business hours; none overnight
Paging
By id, with >id and orderBy=id:ASC, never by page number, so records added or removed during the run are neither skipped nor read twice
Incremental runs
Filter on >modified with the time the last successful run started, so a nightly job reads a night's changes, not the catalogue
Timeouts
30 seconds per request; a page that takes longer is too large, so halve it
These are starting points rather than platform limits. Watch the site's response times during the first runs and adjust.
An incremental read
Everything changed since a moment, a page at a time, paced. >modified takes an ISO 8601 time with its offset, as the API itself writes it (2026-10-19T00:00:00+11:00).
JavaScriptchanged-since.js
// Uses the coredna() helper from the external-id page's import.js
const pause = ms => new Promise(resolve => setTimeout(resolve, ms));
export async function* changedSince(path, since, properties) {
let lastId = 0;
for (;;) {
const page = await coredna('GET', path, { query: {
'>modified': since, // e.g. '2026-10-19T00:00:00+11:00'
'>id': lastId,
orderBy: 'id:ASC',
limit: 200,
retrieveUnpublished: 1,
properties, // '{ id, modified, externalId, name }'
} });
if (page.data.length === 0) return;
yield* page.data;
lastId = page.data[page.data.length - 1].id;
await pause(250);
}
}
// for await (const record of changedSince('pages/pages', lastRunStartedAt, '{ id, modified, name }')) { ... }
Store the time the run started as the next run's since, not the time it finished: a record changed while the run was reading is then read again next time instead of never. The paging how-to has the same loop for a one-off export.
Partial batches
An import of 5,000 records that fails at 3,200 has written 3,200. Because every write is an upsert keyed on externalId, the answer is to run it again from the start, not to work out where it stopped. Record the last source id completed only to report progress.
Retry what is safe to repeat
A timeout, a dropped connection, or a 502, 503 or 504 says nothing about whether a write happened. Retry those three times, waiting 2, 4 and 8 seconds, for requests that do no harm when repeated: reads, edits that set values, edits pinned to an originRevision, and deletes, where a 404 on the retry means the first one worked. Before repeating a create, look the record up by externalId. Treat a 500 on a write like a timeout: read before resending. Do not retry a 4xx unchanged: a 422 or a 403 fails the same way every time, except a 403 for an expired OAuth access token, which is worth one retry after refreshing it. Retries and writes that may have applied has what each kind of request does when it is sent twice, measured on a live site.
JavaScriptwith-retry.js
const RETRYABLE = new Set([502, 503, 504]);
const pause = ms => new Promise(resolve => setTimeout(resolve, ms));
// Only for requests that are safe to repeat: reads, value-setting edits, deletes.
export async function withRetry(send, { attempts = 4 } = {}) {
for (let attempt = 1; ; attempt++) {
try {
return await send();
} catch (error) {
const retryable = error.status === undefined || RETRYABLE.has(error.status); // no status: the connection failed
if (!retryable || attempt === attempts) throw error;
await pause(2000 * 2 ** (attempt - 1)); // 2, 4, 8 seconds
}
}
}
// await withRetry(() => coredna('PUT', 'custom-entities/event/12', { body }));
What to log
Identifiers, not payloads. For each failure, log the source id, the externalId, the status and the platform's message. Do not log request or response bodies verbatim: an import of people or orders carries names, emails, addresses and phone numbers, and a log file is copied, shipped and kept far longer than anyone intends.
A summary at the end. Created, updated, unchanged, failed. A run that reports 0 failed and 0 unchanged on its second night has stopped matching, and is creating duplicates.
The run's start time, saved only when the run succeeded, as the next run's since.