Every Core dna site answers three kinds of API request under /cdna-api: REST for everything, GraphQL for reads that span several entity types, and web events for endpoints you define yourself as jobs. Which fits which task, one worked request for each, and what is true of all three. Jobs were called hooks until the module was renamed Orchestration; the API's routes still use the old name.
L1–L3
Unit 06 · Client code and imports
The external-id pattern
Client code and importsPage 23 of 25Updated
Store the other system's identifier on every record you import, and look it up before every write, so that running an import twice leaves the site exactly as running it once did. Where externalId exists, the upsert, what to handle yourself, the test, and duplicates in the source.
Who this is for. The integrations engineer moving records into Core dna from another system: a product catalogue, a CRM, an events feed, an old site. Afterwards you can write an import that is safe to run again after any failure, and prove it. This is the pattern every integration on the portal depends on; the other guides that import data link here.
The pattern
When a record comes from another system, store that system's identifier for it on the Core dna record, in the field externalId. Before each write, ask the site whether it already holds a record with that externalId: if it does, update that record; if not, create one. That makes the import idempotent: running it twice leaves the site exactly as running it once did, so a run that fails halfway is fixed by running it again from the start.
Without the pattern there are only two ways to match a record: by name or slug, which editors change as a matter of course, or by remembering Core dna ids in the other system, which then has to store them. externalId belongs to the record and survives edits to its name, slug and content. Editors see it on the record's form in the admin, labelled External ID, and can change it there; tell them it is the import's key and not theirs to edit.
Which records have it
externalId is a text field of up to 190 characters, empty unless you set it, on:
Content: pages; books, their units and their pages; blogs and blog posts; content categories; layouts; custom entity records; files; forms; menus and menu elements.
Commerce: products, product variants and product categories; checkouts, orders and quotes; invoices, refunds and payment transactions; tax transactions; shipments; inventory batches and fulfilments.
People and mail: people, users and emails.
FAQ pages and questions, tags, global components and groups do not have it. For those, match on a field you control, such as a slug you derive from the source id.
It is written like any other field in a create or an edit, returned when properties names it, and filterable. A lookup for one id:
Is this ticket already on the site?
GET /cdna-api/pages/pages?externalId=ticket-4021&retrieveUnpublished=1&properties={id,externalId,name,published} HTTP/1.1
Host: hub.coredna.com
Authorization: Bearer YOUR_ACCESS_TOKEN
Accept: application/json
Send retrieveUnpublished=1 on every lookup. A list leaves unpublished records out without it, and an import that cannot see the unpublished copy it made last night makes another one tonight.
The upsert
For one record, on a server, with the credential read from the environment. The example imports events into a custom entity type with the identifier event, whose list is at custom-entities/events (Predicting an endpoint):
JavaScriptimport.js
// Runs on a server (Node 18 or later)
const SITE = process.env.COREDNA_URL;
const TOKEN = process.env.COREDNA_TOKEN; // an OAuth access token (an API key is the older option)
async function coredna(method, path, { query, body } = {}) {
const url = new URL(SITE + '/cdna-api/' + path);
for (const [k, v] of Object.entries(query ?? {})) url.searchParams.set(k, v);
const response = await fetch(url, {
method,
headers: { Authorization: 'Bearer ' + TOKEN, Accept: 'application/json', 'Content-Type': 'application/json' },
body: body === undefined ? undefined : JSON.stringify(body),
});
const data = await response.json().catch(() => ({}));
if (!response.ok) throw Object.assign(new Error(method + ' ' + path + ': ' + response.status + ' ' + (data.message ?? '')), { status: response.status });
return data;
}
// payloadFor(source) builds the body for your entity, as its API reference entry describes.
// For a versioned record, put the content in revisions: { name, areas, activeRevision: true }:
// on the create it is the first revision, on the edit a new live revision holding exactly
// what you sent. An edit without activeRevision is saved but not live.
export async function upsertEvent(source, payloadFor) {
const externalId = 'events-feed:' + source.id; // namespaced: say which system it came from
const found = await coredna('GET', 'custom-entities/events', {
query: { externalId, retrieveUnpublished: 1, properties: '{ id }' },
});
const body = { ...payloadFor(source), externalId };
if (found.totalCount === 0) return coredna('POST', 'custom-entities/event', { body });
if (found.totalCount === 1) return coredna('PUT', 'custom-entities/event/' + found.data[0].id, { body });
throw new Error(externalId + ' matches ' + found.totalCount + ' records: fix the duplicates first');
}
Enforce uniqueness yourself. The lookup before every write is the only guard against two records sharing an externalId; treat a count of more than one as an error to stop on, not a record to pick.
Serialise imports yourself. Run one import at a time, and make the scheduler skip a run while the last is still going, so two runs cannot both find nothing and both create.
It clears the field on a copy. A record duplicated in the admin or through a duplicate route starts with an empty externalId, so the copy is not mistaken for the original; the original keeps its link.
Namespace the id yourself. Prefix it with the source system (crm:, events-feed:) so two feeds that both number their records from 1 cannot collide.
A create that timed out may have happened
A POST that times out says nothing about whether the record was made. Never resend it blind: run the lookup again, and only create if it still finds nothing. The upsert above does exactly that on every run, which is why it is safe to repeat (Retries).
The test
Run the import twice against the same source and compare. The record count must not change, and every record must be identical apart from its modified date and its revision history. Automate it in the importer's own test suite, against a development site, before the first production run:
Count the target records, with limit=1 and the totalCount of the answer.
Run the import, then read every imported record with the fields the import writes.
Run it again, and read them again.
Assert the two counts are equal and the two reads are equal, field by field.
A second run that creates anything has a lookup that does not match: usually a namespace prefix added on one side only, or a lookup without retrieveUnpublished=1.
When the source has duplicates of its own
The same customer twice in a CRM, with different ids, becomes two people on the site, each correctly matched to its own source record. Decide the rule before the first run and put it in the importer: normalise the value you match on (a lower-cased email, a trimmed SKU), choose which source record wins, store the winner's id as externalId, and log the ids that were merged into it so the decision can be traced later.
The platform's own deduplication
Core dna has one built-in deduplication, and it is for a different job: the site data import, which copies configuration such as layouts and input types from one site or environment to another. Every record that import creates is given a UUID kept in a mapping table; the next import looks the UUID up first and, failing that, the record's unique fields, such as a slug. By default a record it finds is skipped; in overwrite mode it is updated. That is why moving layouts between environments twice does not double them (Moving configuration between sites).
None of that applies to records written through the API. For those, externalId is the key and the lookup is the check.