Datastores¶

Use Datastores for data your organisation keeps itself and an agent has to read or update: a list of customers, the settings of an integration, a small catalogue. A datastore is a collection of JSON records, each under a key of its own. You can set a schema every record must satisfy, and declare that a field in a record points at a record in another datastore.

A datastore is not a database. You find a record by its key, by the start of a key, by its metadata, or with a query that filters and sorts the records of one datastore on their fields. A query does not combine datastores and does not count or total. You reach what a datastore holds in three ways: here in the console, with a key through the API, and as a tool in an agent's turn.

You need the datastores capability: in the organisation's ceiling to see the section in the console, and on the key for the API and the tools.

Adding a datastore¶

Choose Add datastore. The drawer opens with four fields:

The name is fixed because other settings name a datastore by it: a key's policy and the references in another datastore's schema. If you want a different name, create a new datastore.

After Save the datastore is in the list, with No schema yet in the Latest schema version column. Click its name to open it. Its page has two tabs: Schema and Records.

Setting a schema¶

Without a schema, a datastore accepts any JSON object as a record. With one, Core checks every record before storing it and refuses a record that does not satisfy it.

On the Schema tab, fill in under New schema version:

Choose Save schema. Every save with a different schema or different references creates a new version; when everything equals the latest version, nothing happens. Schema versions lists every version, newest first. Open one to see its schema, its references and who created it. A version cannot be changed afterwards.

The form always starts empty, even when the datastore already has a schema. A new version replaces the whole schema and every reference. To change something, paste or build the complete schema again and add back the references that should stay. If you save the form empty, the new version validates nothing.

An example: this schema requires every record to have a name and email to be text.

{
  "type": "object",
  "properties": {
    "name": {"type": "string"},
    "email": {"type": "string"}
  },
  "required": ["name"]
}

Existing records and a new version¶

A new schema version does not recheck the records already stored. Each record remembers the version it was written under; the Schema version column of the records shows it. If old records must satisfy the new version, save them again.

What a schema may not contain¶

Core never fetches anything from another address. A $ref in your schema may therefore only point into the same schema (such as #/$defs/address). Core refuses a schema when you save it if it:

• has a $ref to another document or a URL;
• has an $id anywhere inside the schema;
• has a pattern that uses a backreference (\1) or a lookaround ((?=...));
• is not a valid JSON Schema;
• is larger than your deployment allows (64 KiB by default).

The message appears under the JSON Schema field.

References¶

A reference says that a field in a record holds the key of a record in another datastore. When a datastore orders has the reference Datastore customers and JSON Pointer /customer, the value of customer in every order record must be the key of an existing record in customers.

• A JSON Pointer starts with / and follows the path into the data: /customer, or /delivery/address for a nested field.
• The datastore you point at must already exist. A datastore may point at itself.
• A schema version has at most 32 references.
• A record where the field is empty or absent is not checked.
• When the field holds a key that does not exist, Core refuses the record, with the message under Data (JSON).

When the record a reference points at is deleted later, the reference stays exactly as it was written. The records list then shows No longer points anywhere in the References column, and the record itself shows the same label beside that reference.

Viewing records¶

The Records tab lists the records in key order, with for each record:

An open record shows at the top who it is for (Subject), the schema version it was written under, when it was created and updated, and by whom (By: the key or the administrator who wrote it). Below that are Data, Metadata and References, each reference with the key and the datastore it points at.

Every time you view records, Core records in the audit trail which records you were shown.

Filtering and sorting¶

Above the list are the controls that narrow and order it: one row for each filter, one row for the sort, and the buttons Add filter, Apply and Clear. They run a query; the rules for fields, types and operators are in that section.

1. Choose Add filter. A row appears with three controls, from left to right: the field, the operator and the value.
2. Pick the field. When the latest schema version declares fields, you pick from a list: Key, Created and Updated for the record's key and its two timestamps, followed by every top-level field of the schema. Otherwise you type the field's name; $key, $created_at and $updated_at stand for the key and the two timestamps.
3. Pick the operator, such as is, contains or is at least. The list holds only the operators the field's type takes; the operators table explains each.

4. Fill in the value. What you fill in depends on the field and the operator:

   Field or operator	What you fill in
   a text field	the text
   a number field	a number
   a date field	a date
   a date-time field, Created or Updated	a date and a time, in your own timezone
   a true-or-false field	Yes or No
   the operator is one of	the values, separated by commas
   the operator exists	Has a value or Has no value

5. To order the records, pick a field in the sort row, which starts at Key (default), and Ascending or Descending. Without a schema you type the field's name there; leave it empty for key order.

6. Choose Apply.

The list changes only when you choose Apply. With several filters, a record must meet all of them. A row with no field or no value is left out. To drop one filter, choose Remove filter (the bin at the end of its row) and Apply again. Clear removes every filter and the sort and shows all records in key order.

The applied filters and sort are part of the page's address, so a copied link or a bookmark opens the same selection.

When no record meets the filters, the list reads No record matches. When Core refuses a part of the query, such as a value that is not a number for a number field, its message appears at the control concerned. A query that takes too long is reported in place of the list; add a filter that narrows it, or sort by key.

Records of one person¶

A record belongs to one person when it was written in a request or turn for that person (see Build for people). Subject then shows that person's id. A record bound to no one person is Shared.

A key is unique per person. Anna and Bob can each have their own record preferences, next to a shared record preferences. The console shows all three, as three rows with the same key; open a row to see whose it is. The record's drawer also shows its Labels, one badge per value, which rules about Datastore rows read.

The console writes and deletes shared records only. Write record, Edit as JSON and Delete always act on the shared record under that key. For a record of one person, Edit as JSON therefore creates a new shared record with the same key, and Delete deletes the shared record or reports that there is none. The person's record stays as it was. Change or delete such a record through the API, with a request for that person.

Writing a record¶

Choose Write record. The New record drawer opens with:

When the key already exists as a shared record, that record is replaced whole; otherwise it is created. Core checks the data against the latest schema version and its references. When the record does not satisfy them, Data (JSON) says what is wrong, with each error's path in the data (such as data.email), at most ten errors at a time. A record is at most 256 KiB by default; your deployment may set another limit. A datastore that holds its maximum number of records accepts no record under a new key; see Limits.

To edit a record, open it and choose Edit as JSON. The key is then fixed; change Data (JSON) and Metadata (JSON) and choose Save. The record is replaced whole by what the fields hold.

Deleting a record¶

Choose Delete in the record's row menu and confirm. This cannot be undone. Records in other datastores that pointed at this record keep their reference, which then reads No longer points anywhere.

Editing or deleting a datastore¶

Edit in the list's row menu opens a drawer with Description, Metadata (JSON) and Retention. The name cannot be changed.

Delete in the row menu deletes the datastore with every schema version and record in it. This cannot be undone, and the confirmation asks you to type the datastore's name. When the latest schema version of another datastore still points at this one, Core refuses the deletion: change that other datastore's schema first so the reference goes away.

Querying records¶

A query returns the records of one datastore that meet every filter you give, in the order you ask for. The filters in the console, the query tool and the query route of the API run the same query under the same rules.

Fields¶

A filter or a sort names one field:

• A top-level field of the record's data, by its name: status for a record {"status": "open", "address": {"city": "Utrecht"}}. A field inside an object or a list cannot be named: address.city is read as a top-level field with exactly that name.
• $key, the record's key.
• $created_at and $updated_at, when the record was first and last written.

A name that starts with $ always means one of these three, and any other $ name is refused with 422. A field in your own data whose name starts with $ therefore cannot be queried.

Types¶

The datastore's latest schema version decides what a field is compared as:

A type written as a list with null beside one other type, such as ["integer", "null"], counts as that other type. $key is text; $created_at and $updated_at are moments.

In a datastore without a schema every field is text. A string is compared as its characters, and any other value the way JSON writes it. Text is ordered character by character, so a stored 100 comes before 9. Declare the field as a number in the schema to compare it as one.

The value in a filter must be readable as the field's type: a number for a number field, a string holding a date for a date field. Core refuses a value that is not with 422, naming the filter.

Stored records are never the reason a query fails. A record written before the schema declared the field may hold a value of another type, text where a number is expected for instance. Such a record matches no comparison on that field, and a sort places it with the records that have no value for the field. exists still counts it as a value: the record matches exists with true and not with false.

Operators¶

The three text operators ignore the difference between capitals and small letters and match the value literally: %, _ and * are ordinary characters, not wildcards.

A record without the field matches neither eq nor ne. To find the records that lack a field, use exists with false.

A query takes at most 20 filters, and a record must meet all of them. There is no or between filters. For several values of one field, use in; for alternatives across fields, run a query for each.

Sorting and paging¶

A query sorts on one field, ascending (asc) or descending (desc). Without a sort the records come in key order. Records with no value for the sort field come last in both directions, and records with the same value come in key order. A text field is sorted on its first 256 characters.

A query returns one page at a time. Through the API a page holds 20 records unless you set limit (1 to 100), and the answer carries next_cursor when there are more (Cursor pagination). For the next page, send the same filters and sort with that cursor. A cursor belongs to the sort it was issued under; with another sort Core refuses it with 422. Paging keeps its place when records are deleted between two pages, so you can read a page, delete what you handled and ask for the next one.

What a query does not do¶

• Combine datastores. A query reads one datastore. To follow a reference, read the record it points at by its key.
• Count, total or group. It answers records, never a number over them.
• Offer or between filters. Every filter must match.
• Reach into nested fields. Only top-level fields can be named. Store a value you want to filter on at the top level of the record.
• Filter on metadata. That is the records list of the API, with its metadata parameter.
• Select fields. Every matching record comes back whole.

An example¶

A datastore orders has a schema that declares status as a string, amount as a number and due as a string with the format date. This request asks for the open orders due before 1 November 2026, the largest amount first, two per page:

curl -s -X POST "$CORE_URL/datastores/ds_7Kp2.../records/query?limit=2" \
    -H "Authorization: Bearer $CORE_KEY" -H "Content-Type: application/json" \
    -d '{"filters": [{"field": "status", "op": "eq", "value": "open"},
                     {"field": "due", "op": "lt", "value": "2026-11-01"}],
         "sort": {"field": "amount", "direction": "desc"}}'

{
  "items": [
    {
      "object": "datastore.record",
      "datastore_id": "ds_7Kp2...",
      "key": "order-1042",
      "data": {"status": "open", "amount": 1250, "due": "2026-10-20"},
      "metadata": {},
      "schema_version": 2,
      "references": [],
      "subject": null,
      "created_by": "key_4Hd9...",
      "created_at": "2026-09-30T08:12:44Z",
      "updated_at": "2026-10-02T14:03:10Z"
    },
    {
      "object": "datastore.record",
      "datastore_id": "ds_7Kp2...",
      "key": "order-0977",
      "data": {"status": "open", "amount": 480.5, "due": "2026-10-28"},
      "metadata": {},
      "schema_version": 2,
      "references": [],
      "subject": null,
      "created_by": "key_4Hd9...",
      "created_at": "2026-09-12T10:40:02Z",
      "updated_at": "2026-09-12T10:40:02Z"
    }
  ],
  "next_cursor": "eyJ2IjoxLCJr..."
}

For the next two orders, send the same request with &cursor=eyJ2IjoxLCJr... added to the address. A next_cursor of null means there are no more.

Limits¶

The first limit is what keeps a query fast: a query reads all records of its datastore, so the number of records bounds its work. Data that grows without end, such as a log, does not belong in one datastore.

Using a datastore from an agent¶

Every datastore provides five tools a model can call in a turn:

<id> is the datastore's id, not its name. The tools are not in the Overview of Tools, which points to Datastores for them; in the Add tools picker each tool's description ends with the datastore's name in brackets. Give an agent a datastore by choosing the tools under Tools in the agent's form, or include them in a toolset, for instance with the pattern datastore__<id>__*. An agent that may only read gets only get, list and query.

The model does not see the datastore's name or description. Explain in the agent's instructions what the datastore holds and what its records look like.

The query tool tells the model which operators exist. When the datastore has a schema, it also names up to fifty of the declared fields with their types. A field is named there only when its name is a plain letter (a to z, A to Z) or an underscore, followed by at most 63 plain letters, digits, underscores, dots or hyphens. A field with another name, one with an accented letter or a currency sign for instance, can still be queried, so mention it in the instructions. When a query has more records than one call returns, the model gets a cursor to ask for the next ones.

A write by a tool runs the same checks as the console: schema, references and size. The model gets its answer at once, but the record is stored at the end of the turn. When something changes in the meantime that makes the write impossible after all (someone deletes the record it points at, for instance), only that write is skipped and recorded in the audit trail. The rest of the turn stands.

Every record the tools show carries its etag. The model can pass it to put or delete as if_match, and put also takes if_absent: true to create a record only. Core refuses such a write at once when the record changed since the model read it, and again at the end of the turn when someone else changed it in the meantime; then only that write is skipped, as above.

A turn for a person reads that person's own records and the shared ones, and writes that person's records. A turn for no one in particular reads and writes the shared records.

A key's access¶

What a key may do with datastores is set on Keys:

• The key needs the datastores capability. Without it, the routes answer 404 and a turn cannot use the tools.
• The rules about Datastores in the key's policy decide which datastores the key reaches, with name patterns such as customers-*. A datastore no rule permits is absent from lists, and its tools cannot be used.
• The rules about Datastore rows (Rules about a person) decide which records a person reaches. Under the default rules that is the person's own records and the shared ones.

The capability has two settings the console does not show: read_tools (the get, list and query tools) and write_tools (the put and delete tools). Both are on by default. A key with write_tools off can only read in a turn.

Through the API¶

A key with the datastores capability can do everything this page describes through the API as well, under /datastores: create datastores, write schema versions, and read, write and delete records. The records list also filters by the start of a key or by metadata, and a record can be created under a key Core chooses. A request for a person then reads that person's records and the shared ones, and writes and deletes that person's records.

POST /datastores/{datastore_id}/records/query runs a query: filters and sort in the body, limit and cursor in the address, and a page of records as the answer. The example shows one request and its answer.

The routes, their fields and every status are in the API reference.

Writing without overwriting someone else¶

Every record you read carries an etag, also sent as the ETag header of GET and PUT. Send it back in If-Match on the next PUT or DELETE:

curl -s -X PUT "$CORE_URL/datastores/ds_7Kp2.../records/cart-9" \
  -H "Authorization: Bearer $CORE_KEY" -H 'Content-Type: application/json' \
  -H 'If-Match: "dsr_01J....3"' \
  -d '{"data": {"items": ["TSHIRT-RED-M"]}}'

If someone wrote or deleted the record since you read it, Core answers 412 and writes nothing. Read the record again and decide on what it holds now. Send If-None-Match: * instead to create a record only while its key is free; a key that is taken is a 412 too.

Several writes at once¶

POST /datastores/batch takes up to a hundred operations and makes all of them or none. An operation is a put or a delete of a record, each optionally with if_match (and a put with if_absent), or a counter that takes the next number. A put can take its key from an earlier counter with key_from. Placing an order, for instance:

{"operations": [
  {"op": "counter", "datastore_id": "ds_orders", "counter": "orders"},
  {"op": "put", "datastore_id": "ds_orders", "key_from": {"operation": 0, "prefix": "O-"},
   "data": {"sku": "TSHIRT-RED-M", "quantity": 1}, "if_absent": true},
  {"op": "put", "datastore_id": "ds_stock", "key": "TSHIRT-RED-M",
   "data": {"left": 4}, "if_match": "\"dsr_01J....7\""}
]}

This numbers the order (O-1, O-2, ...), writes it and lowers the stock, or does nothing at all when the stock changed since you read it. Each operation is checked as its own route checks it. A refusal names the operation, as operations.2.data.left or a message that starts with Operation 2:. The answer lists each operation's key, etag and counter value.

Numbering¶

POST /datastores/{datastore_id}/counters/{name}/next answers the next number of a counter of the datastore: 1 the first time, then 2, 3 and so on. A request that fails gives its number back, so the numbers have no gaps. A counter's name follows the rules of a datastore name.

A unique field¶

A schema version may set unique to one top-level field, such as "unique": "sku". While that is the latest version, no two records hold the same value there: a second one is a 409 that names the field. It holds per owner: the shared records, and each person's own, may each hold a value once. The value must be a string or a whole number, and a record without the field is free. Setting unique on records that already share a value is refused with 409.

A function that decides every write¶

A datastore can name a function that sees every put and delete of its records before anything is stored, such as the rules of an order: a paid order may be shipped but never go back to the cart, and its total is always the sum of its lines. Set write_function when you create the datastore or with PATCH:

{"write_function": {"name": "order_rules", "version": null}}

version pins one published version; null runs the latest published version at each write. The function must be published and declare no reads and no writes, and the key that sets it needs tooling and a policy that lets it see and use function__<name>; otherwise the request is refused with 422. A function the key may not see answers the same 422 as one that does not exist. A key that only writes records needs no tooling. null for write_function removes it.

The function's arguments are the write and the record it replaces:

{"operation": "put", "datastore": "orders", "key": "O-1042",
 "data": {"status": "paid", "lines": [{"price": 5, "quantity": 2}]},
 "previous": {"data": {"status": "cart", "lines": [{"price": 5, "quantity": 2}], "total": 10},
              "etag": "\"dsr_01J….3\""}}

A delete has no data; previous is null for a new key. For a person's put, previous is that person's own record, never the shared one; for a delete it is the record the delete would take. key is null for a batch's put whose key a counter of the same batch makes. The function's own parameters validate this document; a document they reject is invalid_arguments. The function answers one of these:

When the function decides nothing, the answer is 422 with nothing written, refusal null and error_code saying how it ended:

The function runs on every write path: a single write, a batch, an agent's put and delete tools and a function's own writes. For a single write and a batch, a record that someone else changes while the function runs -- or a write function the datastore is given meanwhile -- makes Core run it again on what is stored now, up to three more times, and then answer 409. An agent's tools run the function when the agent calls them: a refusal reaches the model as the tool's result, quoting the reason, and a record that changes before the turn ends drops the write as conflict, without running the function again.

A batch writes one key of such a datastore once; a second write of it is 422 on operations.<i>.key. An agent's turn writes one such key once too; a second write is refused when the agent makes it. Expiry, retention and erasure remove records without asking the function.

Expiry and bookkeeping¶

record_ttl_days (1 to 3650) on a datastore, set when you create it or with PATCH, deletes its records once they have not changed for that many days, whatever the organisation's own retention window says. null turns it off. retention: "keep" keeps the records past the organisation's window; "window", the default, leaves them under it. retained_for: "bookkeeping" is for records you must keep, such as orders and invoices; it needs record_ttl_days. See Retention and erasure.

Announcing writes¶

Set emit_events: true when you create a datastore, or with PATCH, and every record written or deleted raises a datastore.record.written or datastore.record.deleted event: datastore_id, datastore (the name), key, revision and, on a write, created; never the record's data. Each operation of a batch raises its own event.

A workflow trigger takes one event_kind, so a trigger on datastore.record.written does not hear deletes; a second trigger on datastore.record.deleted does. The filter picks the datastore by name:

curl -s -X POST "$CORE_URL/workflow_triggers" \
    -H "Authorization: Bearer $CORE_KEY" -H "Content-Type: application/json" \
    -d '{"workflow": "fulfil-order",
         "event_kind": "datastore.record.written",
         "filter": {"datastore": "orders"}}'

A write that is refused and a batch that is rolled back raise no event. Neither do records that expire, leave with the organisation's retention window or are erased with a person, nor the records that go when you delete the datastore.

Retention and erasure¶

Records fall under the organisation's retention window. A record not changed for longer than that window is deleted; the Updated time counts, not the creation time. The datastores themselves and their schema versions stay. A datastore with its own record_ttl_days also deletes its records on that clock.

A datastore with retention: "keep" (Keep until deleted in the console) is passed by the organisation's window: use it for records you maintain rather than receive, such as a product catalogue, customer profiles or settings, which must not disappear because the organisation deletes its conversations after, say, thirty days. Its own record_ttl_days still applies, and erasing a person still deletes her records in it. Set it when you create the datastore or change it with PATCH; leaving the field out of a PATCH keeps it as it is.

When you erase a person (Erase a person), that person's records go too, in a datastore with retention: "keep" as well. Only a datastore with retained_for: "bookkeeping" holds on to them: those records stay, still marked as hers, until their record_ttl_days has passed, and the organisation's window does not delete them, whatever retention says. The erasure counts them as datastore_records_retained. Core keeps no list of erased people, so a later request with the same person's id reaches those records again. Such a datastore cannot be deleted (409) until you clear retained_for with PATCH; its records then become ordinary records again.

Empty lists and access limits¶

"No datastores yet" means no datastore has been created for this organisation yet. "No records yet." means the datastore is empty.

If you see datastores is off for this organisation, that capability is off in the organisation's ceiling. The administrator of the organisation above turns it on under Permitted capabilities on Settings; for an organisation with none above it, your operator does.

"Datastore not found" means the datastore was deleted, or your organisation has no access to it.