Knowledge through the API¶

The concept behind knowledge is explained in Knowledge.

On this page you make your organisation's documents searchable through the API, from collection to search tool: you store sources in a collection, create an index over that collection, then search the index directly or offer it to a model as a tool. The steps are in the order you carry them out.

Your key needs the knowledge capability for these endpoints. Uploading a file also needs files in some collections (step 1). Graphs have a page of their own: Graphs through the API.

Use the console¶

You can also manage collections, indexes, search tools and pipelines in the console’s Knowledge section. The API and console use the same resources. For example, an administrator can maintain documents in the console while your application searches them through the API.

Collections, indexes, pipelines and tools¶

Your key's policy can narrow which collections and indexes it reaches, by name. One it does not name is absent for that key: lists leave it out, a request naming it by id answers 404, and so does anything hanging off it -- a collection's sources, an index's stats, runs, search and its memory records. Creating or renaming one to a name the policy does not name answers 403, and nothing is stored. A search tool is no exception: the policy must permit the tool and every index, collection and graph its pipeline names (Which rules apply). See keys.

1. Create a collection and source¶

curl -s -X POST "$CORE_URL/knowledge/collections" \
    -H "Authorization: Bearer $CORE_KEY" -H "Content-Type: application/json" \
    -d '{"name": "handbook", "description": "Internal handbook"}'

curl -s -X POST "$CORE_URL/knowledge/collections/col_2Pf7.../sources" \
    -H "Authorization: Bearer $CORE_KEY" -H "Content-Type: application/json" \
    -d '{"kind": "text", "title": "Expenses policy",
         "text": "Claims are submitted within 30 days...",
         "metadata": {"department": "finance"}}'

A file goes in with one call. Post the file itself to .../sources/upload rather than uploading it first and naming the id:

curl -s -X POST "$CORE_URL/knowledge/collections/col_2Pf7.../sources/upload" \
    -H "Authorization: Bearer $CORE_KEY" \
    -F "file=@handbook.pdf" \
    -F 'metadata={"department": "finance"}'

title (the filename by default), retention and metadata -- a JSON object of string values, as a string, because a multipart form carries no types of its own -- are optional fields beside it. The bytes are stored once, where uploading to /files and then creating a file source from the id stores them twice and leaves an upload behind.

What the upload becomes follows the collection. In a key's own collection and in your organisation's uploads collection it is a file: your key needs the files capability, or the upload answers 403, and the name has to be one your key's file policy names (What a file id reaches). In every other collection it is a knowledge document, with no check on its name. A title or filename that holds a control character answers 422.

A large file goes in parts. A file larger than one upload may be, or one that has to survive a dropped connection, goes in through an upload session that ends as a source. Your key needs the knowledge capability, and your organisation's files must be kept in an object store:

curl -s -X POST "$CORE_URL/uploads" \
    -H "Authorization: Bearer $CORE_KEY" \
    -H "Content-Type: application/json" \
    -d '{"filename": "handbook.pdf", "media_type": "application/pdf", "size_bytes": 157286400,
         "target": {"kind": "source", "collection_id": "col_2Pf7...",
                    "title": "Handbook", "metadata": {"department": "finance"}}}'

The session is refused for what the one-call upload refuses, before any part is sent: a media type no pipeline reads (422), a collection your key does not reach (404), a linked copy (409). Send the parts, complete the session and poll it as Upload a large file in parts shows; once it is completed it carries the source_id. A source larger than the ordinary upload limit is stored and downloadable; an index still reads it in parts when it is text, a PDF, Word, PowerPoint or Excel up to 100 MiB (A large file is indexed in parts), and any other file that large is drawn into no index.

A collection's source routes show a key only the sources it holds the capability for: without artifacts no artifact is among a collection's sources, and without files no upload.

Choose one of these source types for the JSON body:

A file source copies the bytes into a blob of its own, so deleting either the original or the copy leaves the other alone. Copying an artifact is downloading it: it answers 403 when no rule permits your key artifact download. And a collection's name may not start with key_, which names the collection Core keeps for an API key.

Word files, slide decks, web pages and PDFs are read by the pipeline the index runs. The default pipeline reads all of them, so you only choose a pipeline when you want something other than the default. An older Word file (.doc, not .docx) is the exception: only a pipeline with the llama_parse step reads it, such as the ingest/llama-parse and ingest/rag-with-tables templates, and any other pipeline fails it with needs_parse. A spreadsheet arrives as tables instead: each worksheet becomes rows you can query by field, like a CSV.

Add metadata to filter sources during indexing and search.

Filtering the source list. GET .../sources takes folder, q (title contains, case-insensitive) and the repeatable status (an ingestion status of one of the indexes that drew the source in). folder matches whole path segments of the folder metadata, so policy returns policy and policy/2026 and never policy-old; add recursive=false for the folder's own sources only. An empty folder names the root: with recursive=false it returns the sources without a folder. For example:

curl -s "$CORE_URL/knowledge/collections/col_2Pf7.../sources?folder=policy&status=ready&status=failed" \
    -H "Authorization: Bearer $CORE_KEY"

Downloading a source. GET .../sources/{source_id}/content answers the bytes Core stores for a source, as an attachment named after its title: the file that was uploaded, a text source's text, or a records source's records as a JSON array.

curl -s "$CORE_URL/knowledge/collections/col_2Pf7.../sources/src_9Qd2.../content" \
    -H "Authorization: Bearer $CORE_KEY" -o handbook.pdf

Change a document afterwards¶

A source's title, metadata and retention are changed later with a PATCH. metadata replaces the whole map, as it does on a collection, so send every key you mean to keep — folder and the keys a connector wrote included:

curl -s -X PATCH "$CORE_URL/knowledge/collections/col_2Pf7.../sources/src_9Qd2..." \
    -H "Authorization: Bearer $CORE_KEY" -H "Content-Type: application/json" \
    -d '{"title": "Expenses policy 2026",
         "metadata": {"department": "finance", "folder": "policy/2026"}}'

Core stores the folder path normalised (/policy//2026/ becomes policy/2026), and a . or .. segment is a 422.

A new map is not a field update but a routing event: every index that selects this collection is re-evaluated against its own filter. A source that now matches is ingested there after all; one that no longer matches drops out. One that still matches only has the labels on its chunks rewritten, so nothing is processed again. A source's kind, shape, bytes and collection are fixed and are not on this body.

retention says whether your organisation's retention window covers this document: window lets it be deleted once it is older than the window your administrator set, keep keeps it until something deletes it. A collection you create defaults to keep and sets retention_default for the documents in it; your key's own collection -- where your uploads through /files and the files your tools produce land -- defaults to window. One document may always differ from its collection:

curl -s -X PATCH "$CORE_URL/knowledge/collections/col_2Pf7.../sources/src_9Qd2..." \
    -H "Authorization: Bearer $CORE_KEY" -H "Content-Type: application/json" \
    -d '{"retention": "keep"}'

DELETE .../sources/{source_id} puts a document in the trash: it is gone from every search and read at once, and its bytes are kept for 30 days unless your operator set another number. The collection's own listing shows its trash, and a restore puts a document back, ingested anew:

curl -s "$CORE_URL/knowledge/collections/col_2Pf7.../sources?trashed=true" \
    -H "Authorization: Bearer $CORE_KEY"
curl -s -X POST "$CORE_URL/knowledge/collections/col_2Pf7.../sources/src_9Qd2.../restore" \
    -H "Authorization: Bearer $CORE_KEY"

A trashed document carries deleted_at and purge_at. A second DELETE removes it for good. A synced document you delete stays in the trash even when its source changes it: the new version replaces it there under a new id, so look it up in the trash listing before you restore it. A key that also holds files reaches the same trash through /files (Delete, restore or purge a file).

Sync a collection from a system you have¶

Instead of creating every source yourself, give a collection a connector and Core fetches the documents. Each one lands as an ordinary source of that collection, so everything above — indexes, search, tools — works exactly as it does for a source you posted.

A collection carries one connector, and its kind is fixed once written: website (crawled through Firecrawl), sharepoint, gdrive, s3 and zendesk.

Firecrawl runs in the United States. If your key restricts where calls may run (policy.settings.regions_allow) to places that leave out us, a website connector's sync and test fail with region_denied and nothing is sent; name vendor:firecrawl in policy.settings.region_exceptions to allow it (your organisation's policy has to name it too when it restricts regions without admitting us). The same holds for the llama_parse step, whose service runs in the EU unless your operator configured another region: a key held elsewhere fails the source region_denied unless it names vendor:llamacloud. A connector or a source someone set up in the console, and a search run there, follow your organisation's own regions and exceptions the same way (Keep calls in your regions).

curl -s -X PUT "$CORE_URL/knowledge/collections/col_2Pf7.../connector" \
    -H "Authorization: Bearer $CORE_KEY" -H "Content-Type: application/json" \
    -d '{"config": {"kind": "sharepoint",
                    "site_url": "https://contoso.sharepoint.com/sites/Docs",
                    "tenant_id": "8f1c...", "client_id": "b3d2...",
                    "client_secret": "<the application secret>",
                    "exclude_path_regex": "^/Archive/"},
         "schedule_minutes": 1440}'

PUT writes the whole connector, so a secret you leave out is a secret you clear. PATCH changes part of it: send the settings you want and omit the secret to leave the stored one alone, or include it to rotate it. The read model has no credential field at all — has_credentials says only whether one is stored — and kind cannot be changed by a PATCH; replace the connector with a PUT.

include_path_regex and exclude_path_regex filter what is taken, and must be expressions RE2 accepts: a backreference or a lookaround is 422 with the field named. Images, video, archives and other binaries are always skipped on top of your own rule. schedule_minutes is minutes between scheduled syncs, or null for manual only.

A connector may carry schedule_cron instead of schedule_minutes: a five-field expression -- minute, hour, day-of-month, month, day-of-week -- read in schedule_timezone (an IANA zone, default UTC). The two are mutually exclusive: setting both is 422 naming both fields.

curl -s -X PATCH "$CORE_URL/knowledge/collections/col_2Pf7.../connector" \
    -H "Authorization: Bearer $CORE_KEY" -H "Content-Type: application/json" \
    -d '{"schedule_minutes": null,
         "schedule_cron": "0 6 * * 1-5", "schedule_timezone": "Europe/Amsterdam"}'

curl -s -X POST "$CORE_URL/knowledge/collections/col_2Pf7.../connector/test" \
    -H "Authorization: Bearer $CORE_KEY"

curl -s -X POST "$CORE_URL/knowledge/collections/col_2Pf7.../connector/sync" \
    -H "Authorization: Bearer $CORE_KEY"

curl -s "$CORE_URL/knowledge/collections/col_2Pf7.../connector/runs" \
    -H "Authorization: Bearer $CORE_KEY"

test reaches the system and lists at most one item, on the request path, storing nothing. sync answers 202 with a queued run, or 409 while one is already queued or running; POST .../connector/sync/cancel stops the active one between items. Each run row carries discovered, added, updated, removed, skipped and failed, updated as it goes, and a fixed error_code when it failed — one of auth_failed, not_found, rate_limited, egress_denied, too_many_items, provider_error, cancelled, deadline_exceeded, region_denied or tenant_unavailable (the organisation, or one above it, is suspended). A provider's own message is never returned. A document a sync could not index anyway is not stored: one larger than the index reads (by default 100 MiB for text and JSON, and 25 MiB for every other type, HTML pages and pictures included), or than the deployment's own sync ceiling (50 MiB by default), is counted in skipped, never in failed.

Deleting the connector (DELETE .../connector) leaves the sources it created in the collection. A connector configured on that collection afterwards inherits them: it updates the ones it also offers and deletes the ones it does not. Sources you created by hand carry no connector uri, so a sync never deletes them.

The console does not connect or disconnect separately from the collection: there the source is chosen when the collection is created, and the choice is fixed — see Collections .

2. Create an index¶

curl -s -X POST "$CORE_URL/knowledge/indexes" \
    -H "Authorization: Bearer $CORE_KEY" -H "Content-Type: application/json" \
    -d '{"name": "handbook", "collection_ids": [{"collection_id": "col_2Pf7..."}]}'

An index can include multiple collections. Each collection can also feed multiple indexes with different chunking settings. Add "metadata_filter": {"department": "finance"} to a collection link to select matching sources only.

Source processing continues after the request returns. Check GET /knowledge/indexes/{id}/sources for queued, processing, ready or failed; failures include a fixed error_code. Existing sources are not automatically processed when you first link their collection. Run POST /knowledge/indexes/{id}/reindex to include them and follow progress with GET /knowledge/indexes/{id}/runs.

A PDF the parsing service refuses is read from its text layer. When the llama_parse step sends a PDF to the parsing service and the service refuses it for a reason a retry does not fix, Core reads the PDF's own text layer instead. The source is ready and searchable, and its row in GET /knowledge/indexes/{id}/sources carries parse_fallback with the reason: credits_exhausted, api_key_rejected, api_key_missing or file_rejected; the field is null when the service read the document. Once the service works again, run a full reindex of the index to have the service read these documents; a file_rejected PDF is refused again. A document the service has already read for this index is never replaced by its text layer: a reindex during an outage fails for that document, and the chunks the service produced stay searchable. A timeout, a rate limit or a server error never falls back, and neither does a PDF without a readable text layer: the source fails with its error_code.

A large file is indexed in parts. A text file -- plain text, Markdown, CSV or JSON -- a PDF, and a Word, PowerPoint or Excel file, larger than 4 MiB and up to 100 MiB, is read and embedded one part at a time: a PDF a run of pages at a time, a Word file a run of paragraphs, a deck a run of slides, a workbook a run of rows, each table cut between two parts keeping its header row. Its chunks become searchable together once the last part is in; until then a search answers from what the index held before. While such a file is reindexed, a search can find both its old and its new chunks for the few minutes it takes to replace them, so results may hold duplicates in that window. A larger file, and a file of another type past 25 MiB, is stored and can be downloaded, but no index draws it in: it has no status in any index and no error_code. Only an index that draws its collection in later, through a reindex, lists it, as failed with too_large. A text, PDF, Word, PowerPoint or Excel file past 25 MiB in an index whose pipeline does more than load, parse, split, embed and store -- a summary, enrichment, tables, pictures, a graph or memory -- fails too_large_for_pipeline; so does a Word, PowerPoint or Excel file past 25 MiB in a pipeline without a parse step, which fails such a file up to 25 MiB with needs_parse. The default pipeline reads them all. A Word, PowerPoint or Excel file whose contents unpack to far more than the file itself fails too_large, and one that cannot be read parse_error. A knowledge graph reads a source whole, so it takes one up to 25 MiB and fails a larger one with too_large. These are the deployment's default limits; the operator can raise the 100 MiB.

Every answer that carries an index has reindex_required: true from the moment you change its model, pipeline or collection links until a full reindex has completed. Removing a collection or narrowing a metadata_filter takes effect on searches at once; the reindex removes what is left behind. A reindex is billed like any other use of the models. A run that was under way when you changed the index ends with status cancelled and error_code superseded; until Core has ended it, a new reindex answers 409.

3. Search the index¶

curl -s -X POST "$CORE_URL/knowledge/indexes/idx_8Bd3.../search" \
    -H "Authorization: Bearer $CORE_KEY" -H "Content-Type: application/json" \
    -d '{"query": "how long do I have to claim expenses", "top_k": 5,
         "metadata": {"department": "finance"}}'

Search combines keyword and semantic rankings, supporting both exact terms and related meanings. The metadata filter restricts results to sources with matching values.

Core embeds your question for the semantic ranking, and those embedding tokens are metered under your key (Usage). When your key or your organisation is at its monthly cap, the search answers 429 with Retry-After, unless the embedding model costs nothing.

A spreadsheet or a records file becomes one table per worksheet, body or table inside the document, each with its own inferred schema. GET /knowledge/collections/{id}/sources/{sid}/tables lists them; POST /knowledge/collections/{id}/sources/{sid}/query filters one of them by field, naming it in table_id -- a source with exactly one table lets you leave that out. An unknown field returns 422 with the field name.

A key whose policy requires an agent (settings.agents_required) cannot use these routes, nor POST /knowledge/retrieval-tools/{id}/query or POST /knowledge/pipelines/{id}/test: each answers 403 before anything is looked up, because it reads your knowledge outside a turn and so outside every agent. Its agents still search, inside their turns.

A policy whose rules narrow which models a key may call must permit the embedding model too: without that permit these routes answer 403 with The embedding model is not allowed for this key., and a turn runs without searching your knowledge.

Pictures found by words¶

An index can make pictures searchable: a PNG or JPEG you upload, and the images inside a PDF or a Word file. Choose the ingest/with-images pipeline when you create the index:

curl -s -X POST "$CORE_URL/knowledge/indexes" \
    -H "Authorization: Bearer $CORE_KEY" -H "Content-Type: application/json" \
    -d '{"name": "reports", "collection_ids": [{"collection_id": "col_2Pf7..."}],
         "ingest_pipeline": {"template": "ingest/with-images"}}'

The text is indexed as with the default pipeline, and each picture -- at most 50 per document, none larger than 20 MB -- is embedded as a picture. A text query then finds it: "a bar chart of revenue" can return the chart itself. The index needs an embedding model that reads images; where the deployment has none, creating a pipeline from this template answers 422, and a document ingested by an index whose model reads no images fails with image_embedding_unavailable. Creating or changing an index so that it runs this pipeline on an embedding model that reads no images answers 422. A picture you upload to a collection is stored whichever pipeline its indexes run; an index on a text pipeline marks it failed with unsupported_media_type. A scanned PDF with no text is stored for its pictures. Pictures in files a connector syncs are skipped, as on the platform; pictures inside the PDFs and Word files it brings in are read.

Some limits keep a document from asking for too much work: at most 200 pictures in one document are examined, a picture larger than 5000 by 5000 pixels is skipped, and a picture's own contents decide its type, whatever the file was called.

A picture comes back as a hit with empty text and an image field, never the bytes:

{"chunk_id": "chk_...", "source_id": "src_...", "text": "",
 "image": {"media_type": "image/png",
           "path": "/knowledge/indexes/idx_8Bd3.../chunks/chk_.../image"}}

Fetch the picture by putting path after the API's base URL, with the same key:

curl -s "$CORE_URL/knowledge/indexes/idx_8Bd3.../chunks/chk_.../image" \
    -H "Authorization: Bearer $CORE_KEY" -o chart.png

It answers 404 wherever the search could not have shown you the hit. If the document has changed since it was indexed so the picture is no longer where it was, it also answers 404 and asks you to reindex. A picture's text is not read by this pipeline; an index whose pipeline includes the llama_parse step reads the text in an uploaded image, as it does for a scanned page.

4. Enable model retrieval¶

A search tool is a retrieval pipeline with sources: it runs one version of a pipeline of your own, and it reaches exactly the sources that pipeline names -- an index, a collection. So you make the pipeline first, then register the tool on it, then offer the tool by name. What a tool may reach is yours to decide; the model only chooses the question.

Start from a template and fill in its index. A template's source blocks are empty, and a copy is only made once every one of them names something:

curl -s -X POST "$CORE_URL/knowledge/pipelines/from-template" \
    -H "Authorization: Bearer $CORE_KEY" -H "Content-Type: application/json" \
    -d '{"template": "retrieval/hybrid",
         "name": "handbook-search",
         "sources": {"index": "idx_8Bd3..."}}'

The answer is the new pipeline with its version 1. Register the tool on that version:

curl -s -X POST "$CORE_URL/knowledge/retrieval-tools" \
    -H "Authorization: Bearer $CORE_KEY" -H "Content-Type: application/json" \
    -d '{"name": "handbook",
         "description": "Search the internal handbook.",
         "pipeline": {"pipeline_id": "pip_...", "version": 1},
         "filter_keys": ["folder"]}'

{
  "model": "<a model id>",
  "tools": [{ "type": "tool", "name": "knowledge__handbook" }],
  "input": "How long do I have to claim expenses?"
}

A tool always pins a version of a pipeline of your own: naming a template on the tool itself answers 422, because a template reaches nothing until you fill it. A new version of the pipeline changes nothing until you PATCH the tool's pipeline to it. Every index and collection the version names is checked against your key when you write the version and again when you pin it. The tool carries no chat model -- each LLM step of its pipeline names its own, falling back to the turn's model. A tool name is lower case and at most 48 characters; the model sees knowledge__<name>.

Each key in filter_keys becomes one optional string argument on the tool, so the tool above is offered to the model as {"query", "top_k", "folder"}. The value a model passes is ANDed under the tool's own metadata_filter, so it narrows the question and can never widen it; a value that contradicts the tool's own filter answers no hits. With the echo provider you can drive that call yourself:

{
  "model": "<an echo model id>",
  "tools": [{ "type": "tool", "name": "knowledge__handbook" }],
  "input": "call:knowledge__handbook {\"query\": \"expenses\", \"folder\": \"hr\"}"
}

A tool naming no filter_keys is offered as {"query", "top_k"}. A tool on retrieval/tables also offers table_query: the model picks one of the tables the tool reaches -- each described by its document's title, the table's name and its columns -- and passes a query over it by field. An argument the tool never offered, a table it does not offer and a query the table's columns do not fit all come back to the model as "Tool arguments were not valid." or "Unknown table.", not as a failed turn.

One search tool you never register. Every turn of your key is also offered knowledge__uploads, a search over your key's own collection -- the files you uploaded and the files your tools produced. You do not name it in tools[] and there is nothing to create; a forbid rule of kind tool that names knowledge__uploads in your policy removes it, and so does turning index_uploads off in that key's files configuration, on POST /tenants/{id}/keys or PATCH /tenants/{id}/keys/{key_id} (the console has no field for it today).

Which rules apply when the model searches for a person. When a request names a person (Build for people), several rules meet in one search:

• Every index, collection and graph the tool's pipeline names is checked against the rules of kind knowledge_index, knowledge_collection and knowledge_graph, for that person and equally for a request that names nobody. If they refuse one of them, a request that offers the tool is refused with 404, as if the tool did not exist, whether it names the tool or reaches it through a toolset. The refusal is recorded in the audit trail.
• The hits are narrowed by the rules of kind knowledge_source, which read each source's labels, and for uploads and artifacts by those of kind stored_file. The rules over collections and indexes never remove single hits.

To keep a search tool away from some people altogether, label the search tool and write a rule of kind tool (Labels on everything). No rule over knowledge_collection hides your key's own collection, where its uploads go, from that key, and no rule over knowledge_index hides the index Core keeps over it from knowledge__uploads: that search is not checked against either kind, so the first point above does not apply to it. It searches your key's own collection: a PATCH that would change that index's labels or collection_ids is answered 409, from a key and from the console alike, unless it puts the index back to that one collection, unfiltered and without labels. To keep some people from it, write a forbid rule of kind tool that names knowledge__uploads. You can also label it with PUT /labels/tool/knowledge__uploads, from any key or from the console, and write the rule over that label; the label is your organisation's and holds for every key's uploads tool. The rules over knowledge_source and stored_file still narrow its hits. A request that reads or searches that index by id, on the /knowledge/indexes routes, is checked against the rules over knowledge_index like any other.

Custom retrieval pipelines¶

Build your own pipeline when no template is what you want, and name its sources as blocks: an index block for each index a search reads (a search needs at least one), a collection block for each collection whose documents a step reads whole. The tool's name and description tell the model when to use it:

curl -s -X POST "$CORE_URL/knowledge/pipelines/from-template" \
    -H "Authorization: Bearer $CORE_KEY" -H "Content-Type: application/json" \
    -d '{"template": "retrieval/hybrid-documents",
         "name": "handbook-whole",
         "sources": {"index": "idx_8Bd3...", "collection": "col_2Pf7..."}}'

Use POST /knowledge/retrieval-tools/{id}/query to test the tool outside a model turn. It returns the same search output, and takes the same table_query a model would pass. POST /knowledge/pipelines/{id}/test runs a version before any tool pins it. List built-in pipelines with GET /knowledge/pipeline-templates and available steps with GET /knowledge/pipeline-components. The component catalogue shows which model-dependent steps are configured. Selecting an unavailable component returns 422.

Create custom pipelines with POST /knowledge/pipelines. Like prompts, pipelines have immutable versions. A tool pins the version it uses.

GET /knowledge/pipelines/{id}/models lists every step of the latest version that calls a chat model: the step's node_id and node_type, the model it names, where that model runs (region, country) and whether your key's policy admits it (allowed). A step your policy refuses fails with policy_denied when it runs, so this is how to find it first. A step that names no model uses the turn's own and is listed with model: null. An ingest pipeline runs under the key that created the source, so for it allowed is the answer for sources your key creates. A pipeline shared with you sealed lists its models without step ids or types.

To keep a pipeline inside one folder or one label, give it a filter step: input -> filter -> semantic_search -> output, the step's metadata_filter a JSON object matched against a chunk's own metadata, such as {"folder": "hr"}. retrieval/filtered is that shape as a template. The step narrows every query the pipeline runs, including the variants a rewrite step added, and two steps compose with the later one winning a key they both name. A search step's own metadata_filter still wins over what the step set, and the tool's own filter still applies on top of both: asking for another value of a key it pins returns no results rather than a wider search.

Results and citations¶

Search results contain numbered passages and citations. Retrieved passages are a tool result: the model uses them when answering and they gain no instruction authority. The model is told they are data when your key holds trust_envelope.

The result's own citations list one entry per passage, in the order the passages are numbered: kind: "file", the chunk's id as ref -- or the document's own id, for a whole document a collection block read -- and the source's title. When the model then writes [n] into its answer, Core resolves that marker against the same numbering and hands it back as an annotation on the assistant's text, so you can mark the sentence rather than a list at the end. A second search in the same turn numbers on from where the first stopped ([4] after three passages), so the model never sees one number for two sources. A marker naming a passage no search returned is left as plain text (where an answer's sources are).

If retrieval fails during a turn, the tool returns a fixed message such as "Retrieval failed.", "Retrieval budget exhausted." or "Denied by policy.". The turn continues.

Read a cited passage¶

A citation names a passage by its chunk id: ref on a result's citations, file_id on a file_citation annotation, chunk_id on a search hit. To show the passage itself, read the chunk:

curl -s "$CORE_URL/knowledge/chunks/chk_..." -H "Authorization: Bearer $CORE_KEY"

{
  "id": "chk_...",
  "index_id": "idx_8Bd3...",
  "source_id": "src_...",
  "collection_id": "col_...",
  "source_title": "Expenses policy",
  "position": 4,
  "text": "Claims must be filed within 30 days ...",
  "truncated": false,
  "image": null
}

text is at most 32,768 characters, and truncated says whether the chunk was longer. A picture has empty text and an image link, as on a search hit. A citation that names a whole document rather than a chunk -- an id that does not start with chk_ -- is not a chunk and answers 404.

You read a chunk wherever you could have run the search that found it: through an index your key may search, from a source the person your request names may read. Anything else answers 404, the same as an id that never existed, so the route never tells you that a passage you may not see exists. A subject token reads it as its person, so a browser can open the passages its own answers cited.

Usage¶

Indexing records embedding tokens, 258 for each picture. Search records one retrieval per search step, query embedding tokens and model tokens for steps that call a model. Hybrid search counts two retrievals because it runs two rankings. See What Core meters.