Datastores¶
A datastore is a named collection of JSON records in your organisation. Each record sits under a key
of its own, such as customer-1042 or settings. Your application reads and writes the records
through the API, an agent does the same with tools during a turn, and an administrator views and
edits them in the console. Think of a customer list, an integration's settings, the appointments an
assistant has made, or a small product catalogue.
What a datastore holds¶
| Part | What it is |
|---|---|
| Record | one JSON object under a key. Core sets the time it was created and last changed itself |
| Schema | optional: a JSON Schema every record must match. Each change is a new schema version; a record that does not match is refused |
| Reference | a field in a record that holds the key of a record in another datastore, such as customer in an order. Core checks on writing that the record exists |
| Owner | a record written for a person belongs to that person. A record without a person is shared |
You find a record by its key, by the start of a key, by its metadata, or with a query that filters and sorts one datastore's records on their fields.
When to use a datastore¶
Use a datastore for data your organisation keeps itself, with a shape you know, that an agent must not only read but also update. Three building blocks look alike:
| You have | Use |
|---|---|
| records with fixed fields an agent looks up and changes | a datastore |
| documents in which an agent searches for what fits a question | knowledge |
| loose facts an agent keeps between conversations | memory |
A datastore is not a database. A query does not combine datastores, does not count or sum, and reads all of the datastore's records. That is why a datastore holds at most 50,000 records by default. Data that grows without bound, such as a log, or questions that combine records of different kinds, belong in your own database. Your application can then give that data to an agent with a tool.
Per person¶
A request or turn that names a person (Build for
people) reads that person's records and the shared ones,
and writes records for that person. Anna and Bob can each have a record preferences of their own,
next to a shared record under the same key. When you erase a person, her records go with her,
except in a datastore you keep for your bookkeeping, such as invoices. You mark such a datastore
with retained_for: "bookkeeping", which needs a period of its own in record_ttl_days: its
records stay, still hers, until that period has passed. Core keeps no list of erased people, so a
later request with the same person's id reaches those kept records again. Such a datastore cannot
be deleted (409) until you clear retained_for. A request without a person reads and writes the
shared records.
Concurrent writes¶
Every record has an etag that changes with every write. Send it back in If-Match when you
write or delete the record, and Core refuses the write with 412 if someone changed the record
since you read it, so two people editing one cart never overwrite each other unseen. Send
If-None-Match: * to create a record only if its key is still free. An agent's put and delete
tools take the same etag as if_match.
To change several records at once, send up to a hundred writes in one batch to
POST /datastores/batch: Core applies all of them or none. A batch can also take the next number
from a counter and use it as a key, so placing an order can number the order, write it and lower
the stock in one step. A schema can also mark one field as unique, such as a SKU: no two records
of one owner -- the shared records, or one person's -- hold the same value there, and a second one
is refused with 409 naming the field. The value must be a string or a whole number.
A datastore can also announce its writes: switch on emit_events and every record written or
deleted raises a datastore.record.written or datastore.record.deleted event, which a
workflow trigger or a webhook can react to. The event names the
datastore and the key, never the record's contents (how to).
Rules a schema cannot state, such as which status an order may move to next, belong in a function the datastore names as its write function: it sees every write and the record it replaces, may complete the record, and may refuse the write with a reason of its own (How).
How it fits together¶
- A datastore belongs to one organisation and needs the
datastorescapability, in the organisation's ceiling and on the key. - Every datastore offers five tools to an agent:
get,list,query,putanddelete. Give an agent that may only read just the first three. - A key's policy decides which datastores it reaches, by name, and which records a person reaches.
- Records fall under the organisation's retention period, counted from their last change. A
datastore that keeps its records (
retention: "keep"), for records you maintain such as a catalogue or profiles, is passed by that period, and so is one kept for bookkeeping, whose records leave only by its own period. A datastore can also set a period of its own, such as a week for carts. Erasing a person takes her records from a datastore that keeps its records too, but not from one kept for bookkeeping. - A datastore cannot go into a publication. A solution that needs one creates it in every organisation where it is installed.
Read on¶
- Datastores: create a datastore, set a schema, write records, query, and give the tools to an agent.
- Build for people: naming people and rules per person.
- Knowledge and Memory: the two building blocks that look alike.