Tool schemas

SID-2 reads what each tool does from its description in the API tools field, so demonstrate every tool you pass in tool_examples. The reference schemas below carry wording the model is trained against (the ranged-reference grammar, the seen markers, and the query-style guidance) so start from them verbatim and adapt only the parts that describe your backend.

Customize the tools

Adapt the set to your backend. Describe each tool in the API tools field and demonstrate it in tool_examples. Update the schema, description, example, and backend implementation together when you change a tool.

Expose one or both retrieval modes. If your backend only does lexical search, expose text_search and drop search; if it only does semantic search, do the reverse.
Add filters as extra arguments, such as published_after and published_before (ISO 8601 strings), or metadata fields like author. Describe the filters and query operators, and demonstrate them in tool_examples.
Drop tools you do not need. Only report_helpful_ids is required, and its schema must stay unchanged.

Multiple corpora

Prepend the corpus name to each search tool, keeping the retrieval mode in the name. For example, use email_text_search() for keyword search over email:

Tool	Corpus	Retrieval mode
`email_text_search`	Email	Keyword search
`email_search`	Email	Vector search
`documents_text_search`	Documents	Keyword search
`documents_search`	Documents	Vector search

Describe what each corpus contains in its tool description and provide an example for each tool in tool_examples. Route each call to the matching backend. Keep the plain search and text_search names for a single corpus.

Use one DocumentCache for the episode across all corpora. Database IDs must be unique across the cache: if two corpora can both return ID 123, register them as email:123 and documents:123. Keep the mapping back to the corpus and original database ID so you can resolve the reported IDs. A shared read tool can then serve documents from any corpus through the cache.

Search and read schemas

Cap read() at 10,000 characters of document text per call, including ranged reads. Enforce the cap in your handler before rendering; see Rendering reads.

Search tools return snippets from documents or chunks. read returns the full chunk or a requested character range.

The text_search description below assumes plain BM25. If your lexical backend parses phrase quoting, exclusions, or filters, describe those operators in the tool description and demonstrate them in tool_examples.

from sid.document_cache import SNIPPET_SIZE_DEFAULT

SNIPPET_SIZE_DESCRIPTION = (
    f"Words shown per result snippet (default {SNIPPET_SIZE_DEFAULT}). Must be a positive integer. "
    f"Pass a larger value to see more of each document."
)

SEARCH_TOOL = {
    "type": "function",
    "name": "search",
    "description": (
        "Semantic (vector) search over the dataset. Best for conceptual or paraphrased "
        "queries: a long, descriptive query of what you want beats a few keywords; use "
        "text_search when you know the distinctive words a match would contain. Result "
        "ids may be ranged references 'id#start:end' naming the exact character span of "
        "the document shown; stretches you have already been shown are replaced by "
        "'[seen: \"#start:end\"]' markers instead of repeating, and doc_length gives the "
        "document's total character count."
    ),
    "parameters": {
        "type": "object",
        "properties": {
            "query": {"type": "string", "description": "The query to search the dataset with"},
            "limit": {"type": "integer", "description": "The number of results to return"},
            "snippet_size": {"type": "integer", "description": SNIPPET_SIZE_DESCRIPTION},
        },
        "required": ["query"],
    },
}

TEXT_SEARCH_TOOL = {
    "type": "function",
    "name": "text_search",
    "description": (
        "Full-text keyword search over the dataset, ranked by BM25. Provide the terms you "
        "want matched as plain words separated by spaces: the distinctive content words a "
        "matching document would contain. Result ids may be ranged references 'id#start:end' naming the exact "
        "character span of the document shown; stretches you have already been shown are "
        "replaced by '[seen: \"#start:end\"]' markers instead of repeating, and doc_length "
        "gives the document's total character count."
    ),
    "parameters": {
        "type": "object",
        "properties": {
            "query": {
                "type": "string",
                "description": (
                    "Plain space-separated content words to match (BM25)."
                ),
            },
            "limit": {"type": "integer", "description": "The number of results to return"},
            "snippet_size": {"type": "integer", "description": SNIPPET_SIZE_DESCRIPTION},
        },
        "required": ["query"],
    },
}

READ_TOOL = {
    "type": "function",
    "name": "read",
    "description": "Search results show only short snippets. Read up to 10,000 characters of a document per call. A bare ID starts at character 0; a ranged reference starts at its requested offset. Use further ranged reads to continue.",
    "parameters": {
        "type": "object",
        "properties": {
            "id": {
                "type": "string",
                "description": (
                    "The id of the document to read. Also accepts a ranged reference "
                    "'id#start:end' (character offsets, e.g. '123456#800:1600') to read only "
                    "that part of the document's text; a range that runs past the end of the "
                    "document is trimmed to the part that overlaps it. doc_length on every "
                    "result tells you how far a document actually runs."
                ),
            },
        },
        "required": ["id"],
    },
}

import type { ResponseCreateParamsNonStreaming as RequestParams } from "openai/resources/responses/responses";

const SEARCH_TOOL = {
  "type": "function",
  "name": "search",
  "description": "Semantic (vector) search over the dataset. Best for conceptual or paraphrased queries: a long, descriptive query of what you want beats a few keywords; use text_search when you know the distinctive words a match would contain. Result ids may be ranged references 'id#start:end' naming the exact character span of the document shown; stretches you have already been shown are replaced by '[seen: \"#start:end\"]' markers instead of repeating, and doc_length gives the document's total character count.",
  "parameters": {
    "type": "object",
    "properties": {
      "query": {
        "type": "string",
        "description": "The query to search the dataset with"
      },
      "limit": {
        "type": "integer",
        "description": "The number of results to return"
      },
      "snippet_size": {
        "type": "integer",
        "description": "Words shown per result snippet (default 50). Must be a positive integer. Pass a larger value to see more of each document."
      }
    },
    "required": [
      "query"
    ]
  }
};

const TEXT_SEARCH_TOOL = {
  "type": "function",
  "name": "text_search",
  "description": "Full-text keyword search over the dataset, ranked by BM25. Provide the terms you want matched as plain words separated by spaces: the distinctive content words a matching document would contain. Result ids may be ranged references 'id#start:end' naming the exact character span of the document shown; stretches you have already been shown are replaced by '[seen: \"#start:end\"]' markers instead of repeating, and doc_length gives the document's total character count.",
  "parameters": {
    "type": "object",
    "properties": {
      "query": {
        "type": "string",
        "description": "Plain space-separated content words to match (BM25)."
      },
      "limit": {
        "type": "integer",
        "description": "The number of results to return"
      },
      "snippet_size": {
        "type": "integer",
        "description": "Words shown per result snippet (default 50). Must be a positive integer. Pass a larger value to see more of each document."
      }
    },
    "required": [
      "query"
    ]
  }
};

const READ_TOOL = {
  "type": "function",
  "name": "read",
  "description": "Search results show only short snippets. Read up to 10,000 characters of a document per call. A bare ID starts at character 0; a ranged reference starts at its requested offset. Use further ranged reads to continue.",
  "parameters": {
    "type": "object",
    "properties": {
      "id": {
        "type": "string",
        "description": "The id of the document to read. Also accepts a ranged reference 'id#start:end' (character offsets, e.g. '123456#800:1600') to read only that part of the document's text; a range that runs past the end of the document is trimmed to the part that overlaps it. doc_length on every result tells you how far a document actually runs."
      }
    },
    "required": [
      "id"
    ]
  }
};

// Include the unchanged REPORT_TOOL from the reference implementation.
const TOOLS = [SEARCH_TOOL, TEXT_SEARCH_TOOL, READ_TOOL, REPORT_TOOL] as RequestParams["tools"];

import sid "github.com/sidhq/sid-go"

var snippetSizeDescription = fmt.Sprintf(
	"Words shown per result snippet (default %d). Must be a positive integer. "+
		"Pass a larger value to see more of each document.",
	sid.SnippetSizeDefault,
)

var searchTool = map[string]any{
	"type": "function",
	"name": "search",
	"description": "Semantic (vector) search over the dataset. Best for conceptual or paraphrased " +
		"queries: a long, descriptive query of what you want beats a few keywords; use " +
		"text_search when you know the distinctive words a match would contain. Result " +
		"ids may be ranged references 'id#start:end' naming the exact character span of " +
		"the document shown; stretches you have already been shown are replaced by " +
		"'[seen: \"#start:end\"]' markers instead of repeating, and doc_length gives the " +
		"document's total character count.",
	"parameters": map[string]any{
		"type": "object",
		"properties": map[string]any{
			"query":        map[string]any{"type": "string", "description": "The query to search the dataset with"},
			"limit":        map[string]any{"type": "integer", "description": "The number of results to return"},
			"snippet_size": map[string]any{"type": "integer", "description": snippetSizeDescription},
		},
		"required": []string{"query"},
	},
}

var textSearchTool = map[string]any{
	"type": "function",
	"name": "text_search",
	"description": "Full-text keyword search over the dataset, ranked by BM25. Provide the terms you " +
		"want matched as plain words separated by spaces: the distinctive content words a " +
		"matching document would contain. Result ids may be ranged references 'id#start:end' naming the exact " +
		"character span of the document shown; stretches you have already been shown are " +
		"replaced by '[seen: \"#start:end\"]' markers instead of repeating, and doc_length " +
		"gives the document's total character count.",
	"parameters": map[string]any{
		"type": "object",
		"properties": map[string]any{
			"query":        map[string]any{"type": "string", "description": "Plain space-separated content words to match (BM25)."},
			"limit":        map[string]any{"type": "integer", "description": "The number of results to return"},
			"snippet_size": map[string]any{"type": "integer", "description": snippetSizeDescription},
		},
		"required": []string{"query"},
	},
}

var readTool = map[string]any{
	"type":        "function",
	"name":        "read",
	"description": "Search results show only short snippets. Read up to 10,000 characters of a document per call. " +
		"A bare ID starts at character 0; a ranged reference starts at its requested offset. Use further ranged reads to continue.",
	"parameters": map[string]any{
		"type": "object",
		"properties": map[string]any{
			"id": map[string]any{
				"type": "string",
				"description": "The id of the document to read. Also accepts a ranged reference " +
					"'id#start:end' (character offsets, e.g. '123456#800:1600') to read only " +
					"that part of the document's text; a range that runs past the end of the " +
					"document is trimmed to the part that overlaps it. doc_length on every " +
					"result tells you how far a document actually runs.",
			},
		},
		"required": []string{"id"},
	},
}

// Include the unchanged reportTool from the reference implementation.
var tools = []map[string]any{searchTool, textSearchTool, readTool, reportTool}

Report tool schema

The report tool, report_helpful_ids, is fixed and cannot be changed or modified. The model submits a list of document IDs it believes to be most relevant, terminating the agent loop. Include its schema below, unchanged from the reference implementation, alongside your search and read tools in tools.

# The fixed terminal tool: do not change its name, description or parameters.
REPORT_TOOL = {
    "type": "function",
    "name": "report_helpful_ids",
    "description": (
        "Report the most helpful document ids from the search results. This is required in "
        "the last step. Each id may be reported at most once; repeated ids are penalized. "
        "Ids may be whole documents ('123456') or ranged references ('123456#800:1600'); a "
        "ranged reference counts as its whole document, so there is no need to strip the "
        "'#start:end' part."
    ),
    "parameters": {
        "type": "object",
        "properties": {
            "ids": {
                "type": "array",
                "description": "The ids of the most helpful documents from the search results.",
                "items": {"type": "string"},
            },
        },
        "required": ["ids"],
    },
}

// The fixed terminal tool: do not change its name, description or parameters.
const REPORT_TOOL = {
  "type": "function",
  "name": "report_helpful_ids",
  "description": "Report the most helpful document ids from the search results. This is required in the last step. Each id may be reported at most once; repeated ids are penalized. Ids may be whole documents ('123456') or ranged references ('123456#800:1600'); a ranged reference counts as its whole document, so there is no need to strip the '#start:end' part.",
  "parameters": {
    "type": "object",
    "properties": {
      "ids": {
        "type": "array",
        "description": "The ids of the most helpful documents from the search results.",
        "items": {
          "type": "string"
        }
      }
    },
    "required": [
      "ids"
    ]
  }
};

// The fixed terminal tool: do not change its name, description or parameters.
var reportTool = map[string]any{
	"type": "function",
	"name": "report_helpful_ids",
	"description": "Report the most helpful document ids from the search results. This is required in " +
		"the last step. Each id may be reported at most once; repeated ids are penalized. " +
		"Ids may be whole documents ('123456') or ranged references ('123456#800:1600'); a " +
		"ranged reference counts as its whole document, so there is no need to strip the " +
		"'#start:end' part.",
	"parameters": map[string]any{
		"type": "object",
		"properties": map[string]any{
			"ids": map[string]any{
				"type":        "array",
				"description": "The ids of the most helpful documents from the search results.",
				"items":       map[string]any{"type": "string"},
			},
		},
		"required": []string{"ids"},
	},
}

Customize the tools

Multiple corpora

Search and read schemas

Report tool schema

Content