Skip to main content
graph never names a model in an agent, plan or tool. They name a role, such as chat or judge, and config.toml maps each role to a provider and a model.

​
Providers

A provider is a connection to a model API, written as [providers.<name>]: api_key = "${ANTHROPIC_API_KEY}" reads the key from the environment, so the file is safe to commit.

​
Roles

[models.<role>] maps a role to a provider and a model. Agent files, agent steps, prompt tools, builtin__infer and infer conditions select a role by name. graph’s own model calls use these roles: A role with no entry uses default, except decider. Put a cheaper model on the roles that run most often, such as judge and repair.

​
Custom roles

Any other name under [models] is a custom role, such as fast. A custom role never uses default, so a name that isn’t configured fails and lists the roles that are. A description on a role is what the plan drafter reads when it picks a model for a step, and it prefers the smallest one that fits. A role without one is still usable by name in files you write.

​
Fallbacks

fallbacks on any role, standard or custom, lists other models to try, in order, when the role’s provider is down. graph moves to the next one only after the provider’s own retries run out, and only on errors that look like an outage. A bad request fails right away. Each fallback names its own provider and model.

​
Decision models

A decision model answers a question about a piece of data with a probability instead of text, faster and cheaper than a chat model. Connect one with a systemone provider, which defaults to TypeSafe’s API, and map it to decider:
.graph/config.toml
Decision models run decide conditions, decision tools such as builtin__decide, and tool search when graph drafts a plan. Without one, tool search ranks tools by keyword. graph never swaps a chat model for a decision model or the reverse, and refuses a config that maps one to the other’s role.

​
What a run cost

graph ask, graph chat and graph plan run print a usage line on stderr after each run:
graph ships no prices. Add [pricing."<model>"] with input and output prices in US dollars per million tokens, or the line shows tokens only. See config.toml for every key.