Tools

Tools are how SID-1 takes actions. The API tools field only turns on tool-call parsing; its contents are ignored, so you pass a single placeholder stub. Each tool, its name and its behavior, is defined in the system prompt. You choose which tools to expose to fit your backend. Only report_helpful_ids is always required, because it ends the loop and returns the ranked document IDs.

How a tool is defined

The API tools field is a single placeholder stub that switches on tool-call parsing. SID-1 reads every tool name and what each tool does from the system prompt, not from this field.

tools = [{"type": "function", "function": {"name": "dummy"}}]

const tools: OpenAI.Chat.Completions.ChatCompletionTool[] = [
  { type: "function", function: { name: "dummy" } },
];

A recommended starting point

Every working integration needs at least one search tool and report_helpful_ids.

The search tool is how SID-1 gathers candidate documents. SID-1 works well with these two search shapes:

• search: semantic search over your corpus. Arguments: query (required), limit (optional, default 5, max 15).
• text_search: lexical (full-text) search over the same corpus. Same arguments as search.

Expose both when your backend supports both like we do in our example. SID-1 chooses which tools to call on each turn, and different retrieval modes help in different parts of a run. Semantic search is useful for exploratory queries, synonyms, and hypothetical answer-style queries. Lexical search is useful when exact words, names, codes, quoted phrases, or exclusions matter. SID-1 may call both in the same turn, for example a narrow keyword query alongside broader semantic queries.

search and text_search should run over the same documents and return the same shape; they differ only in retrieval mode. If your backend only supports one retrieval mode, expose that one and describe it clearly in the system prompt.

report_helpful_ids controls the loop. SID-1 calls it when it is done searching and ready to return the ranked document IDs. Your code should treat this call as the final output of the run.

Many datasets include long documents or chunks. Add read when your search results are often longer than about 100 tokens, or when a snippet is not enough to judge relevance. read fetches one full document by id after it appears in search results, which lets SID-1 inspect promising candidates without putting every full document into every search result. Cap read output at about 3000 tokens (roughly 12000 characters) so one long document cannot exhaust the context window.

Customize the tools

Adapt the set to your backend. You make every change in the system prompt; the API tools field stays a single placeholder. For a worked example, see running the tools on your backend.

• Describe one or both retrieval modes. If your backend only does lexical search, describe text_search and drop search; if it only does semantic search, do the reverse.
• Rename per corpus with a consistent convention: keep the algorithm in the name and prefix the corpus, such as documents_search and email_text_search. Keep the plain names when you expose a single corpus.
• Add filters as extra arguments, such as published_after and published_before (ISO 8601 strings), or plain metadata columns like author.
• Drop tools you do not need. Only report_helpful_ids is required.

Format tool results

Return each result as an XML <doc> block, newline-concatenated. This gives SID-1 stable document IDs to cite across turns. The example assumes your backend returns id (required), title, and content.

def format_docs_xml(docs: list[dict]) -> str:
    parts = []
    for doc in docs:
        title = doc.get("title", "")
        doc_id = doc.get("id", "")
        content = doc.get("content", "")
        parts.append(f'<doc id="{doc_id}" title="{title}">\n{content}\n</doc>')
    return "\n".join(parts)

function formatDocsXml(docs: Record<string, unknown>[]): string {
  return docs
    .map((doc) => `<doc id="${doc.id ?? ""}" title="${doc.title ?? ""}">\n${doc.content ?? ""}\n</doc>`)
    .join("\n");
}

id is required: SID-1 uses it to read documents with read and to return them with report_helpful_ids. The other fields are up to you, though the shape above works well. Append tool responses verbatim across turns. Do not summarize, rewrite, or re-rank them before putting them back into the message list.

ID rules

report_helpful_ids is the final output of the loop. SID-1 does not return a natural-language answer; it calls this tool with the document IDs it found most useful, most helpful first. Return that list from your code in the order SID-1 reported it.

Stable IDs are part of the contract. The id returned by search must be accepted by read and report_helpful_ids, and the same document keeps the same id for the whole run, because SID-1 uses IDs to reason across turns.

Use one ID namespace across the corpus. If your corpus has multiple object types, encode the type in the id, or use separate read tools such as read_email and read_thread, but only when they return different object types.

If your IDs are long (over about 36 characters), you can map them to shorter run-local IDs to save tokens and latency. Your code must keep the mapping stable for the whole run. Use this only in extreme cases.