> ## Documentation Index
> Fetch the complete documentation index at: https://restate-6d46e1dc-mintlify-a831f9d6.mintlify.site/llms.txt
> Use this file to discover all available pages before exploring further.
# Retries & Error Handling
> Implement robust error handling and retry strategies for reliable agents.
export const GitHubLink = ({url}) =>
;
};
Restate automatically retries failures of your agents until they succeed.
But LLM calls are costly, so you might want to configure retry behavior to fit your use case and to avoid retrying errors that cannot heal.
Restate distinguishes between two types of errors:
* **Transient errors**: Temporary issues like network failures or rate limits. Restate automatically retries these until they succeed or the retry policy is exhausted.
* **Terminal errors**: Permanent failures like invalid input or business rule violations. Restate does not retry these. The invocation fails permanently. You can catch these errors and handle them gracefully.
## Retrying LLM calls
LLM API calls can suffer from transient failures (rate limits, network issues, provider outages). Restate retries failed LLM calls so your agents recover automatically.
### Default behavior
The Vercel AI SDK and the Restate middleware each have their own retry layer, and they compose.
The Vercel AI SDK does the first layer of retries based on what is set for `maxRetries` on `generateText` (default: 2) . Once those are exhausted, the AI SDK throws an error.
Restate then takes over and retries the invocation. Each Restate retry replays the call, which goes through `maxRetries` Vercel AI SDK attempts again.
By default, Restate's retries follow the policy configured at the [service or handler level](/services/configuration#how-to-configure), or otherwise the [Restate server's default policy](/guides/error-handling#configure-restate-server-defaults).
Restate will go through a limited set of retries with exponential backoff (see [default policy](/references/server-config#param-default-retry-policy)), after which the invocation will be paused. This gives you time to fix the issue, and then [resume the invocation](/services/invocation/managing-invocations#resume).
By default, `DurableRunner.run` retries LLM calls according to the policy configured at the [service or handler level](/services/configuration#how-to-configure), or otherwise the [Restate server's default policy](/guides/error-handling#configure-restate-server-defaults).
Restate will go through a limited set of retries with exponential backoff (see [default policy](/references/server-config#param-default-retry-policy)), after which the invocation will be paused. This gives you time to fix the issue, and then [resume the invocation](/services/invocation/managing-invocations#resume).
By default, the `RestatePlugin` retries LLM calls according to the policy configured at the [service or handler level](/services/configuration#how-to-configure), or otherwise the [Restate server's default policy](/guides/error-handling#configure-restate-server-defaults).
Restate will go through a limited set of retries with exponential backoff (see [default policy](/references/server-config#param-default-retry-policy)), after which the invocation will be paused. This gives you time to fix the issue, and then [resume the invocation](/services/invocation/managing-invocations#resume).
By default, `RestateAgent` retries LLM calls according to the policy configured at the [service or handler level](/services/configuration#how-to-configure), or otherwise the [Restate server's default policy](/guides/error-handling#configure-restate-server-defaults).
Restate will go through a limited set of retries with exponential backoff (see [default policy](/references/server-config#param-default-retry-policy)), after which the invocation will be paused. This gives you time to fix the issue, and then [resume the invocation](/services/invocation/managing-invocations#resume).
When you wrap LLM calls in `ctx.run()`, Restate retries them according to the policy configured at the [service or handler level](/services/configuration#how-to-configure), or otherwise the [Restate server's default policy](/guides/error-handling#configure-restate-server-defaults).
Restate will go through a limited set of retries with exponential backoff (see [default policy](/references/server-config#param-default-retry-policy)), after which the invocation will be paused. This gives you time to fix the issue, and then [resume the invocation](/services/invocation/managing-invocations#resume).
When you wrap LLM calls in `ctx.run_typed()`, Restate retries them according to the policy configured at the [service or handler level](/services/configuration#how-to-configure), or otherwise the [Restate server's default policy](/guides/error-handling#configure-restate-server-defaults).
Restate will go through a limited set of retries with exponential backoff (see [default policy](/references/server-config#param-default-retry-policy)), after which the invocation will be paused. This gives you time to fix the issue, and then [resume the invocation](/services/invocation/managing-invocations#resume).
### Setting a retry policy
To set a separate retry policy for LLM calls, pass [`RunOptions`](https://restatedev.github.io/sdk-typescript/types/_restatedev_restate-sdk.RunOptions.html) to the `durableCalls` middleware:
```typescript errorhandling/fail-on-terminal-tool-agent.ts {"CODE_LOAD::https://raw.githubusercontent.com/restatedev/ai-examples/refs/heads/main/vercel-ai/tour-of-agents/src/errorhandling/fail-on-terminal-tool-agent.ts#max_attempts_example"} theme={null}
const model = wrapLanguageModel({
model: openai("gpt-5.4"),
middleware: durableCalls(ctx, { maxRetryAttempts: 3 }),
});
```
If you set a maximum number of retry attempts, Restate will still go through the AI SDK's `maxRetries` for each attempt, so the two limits multiply (e.g. `maxRetryAttempts`: 3 × `maxRetries`: 2 = up to 6 attempts).
Once Restate's retries are exhausted, the invocation fails with a `TerminalError` and won't be retried further. You can catch the Terminal Error in your handler and act accordingly.
To set a separate retry policy for LLM calls, pass [`RunOptions`](https://github.com/restatedev/sdk-python/blob/main/python/restate/context.py#L37) to `DurableRunner.run`:
```python error_handling.py {"CODE_LOAD::https://raw.githubusercontent.com/restatedev/ai-examples/refs/heads/main/openai-agents/tour-of-agents/app/error_handling.py#handle"} theme={null}
@agent_service.handler()
async def run(_ctx: restate.Context, req: WeatherPrompt) -> str:
try:
run_opts = RunOptions(
max_attempts=3, initial_retry_interval=timedelta(seconds=2)
)
result = await DurableRunner.run(agent, req.message, run_options=run_opts)
except restate.TerminalError as e:
# Handle terminal errors gracefully
return f"The agent couldn't complete the request: {e.message}"
return result.final_output
```
Once these retries are exhausted, the invocation fails with a `TerminalError` and won't be retried further. You can catch the Terminal Error in your handler and act accordingly.
To set a separate retry policy for LLM calls, pass [`RunOptions`](https://github.com/restatedev/sdk-python/blob/main/python/restate/context.py#L37) to the Restate plugin when activating it for your ADK App:
```python error_handling.py {"CODE_LOAD::https://raw.githubusercontent.com/restatedev/ai-examples/refs/heads/main/google-adk/tour-of-agents/app/error_handling.py#retries"} theme={null}
run_options = RunOptions(max_attempts=3, initial_retry_interval=timedelta(seconds=1))
app = App(
name=APP_NAME,
root_agent=agent,
plugins=[RestatePlugin(run_options=run_options)],
)
```
Once these retries are exhausted, the invocation fails with a `TerminalError` and won't be retried further. You can catch the Terminal Error in your handler and act accordingly.
To set a separate retry policy for LLM calls, pass [`RunOptions`](https://github.com/restatedev/sdk-python/blob/main/python/restate/context.py#L37) to `RestateAgent`:
```python error_handling.py {"CODE_LOAD::https://raw.githubusercontent.com/restatedev/ai-examples/refs/heads/main/pydantic-ai/tour-of-agents/app/error_handling.py#retries"} theme={null}
restate_agent = RestateAgent(
agent,
run_options=RunOptions(max_attempts=3, initial_retry_interval=timedelta(seconds=2)),
)
```
Once these retries are exhausted, the invocation fails with a `TerminalError` and won't be retried further. You can catch the Terminal Error in your handler and act accordingly.
Restate's `RestateMiddleware` lets you specify the retry behavior for LLM calls via `RunOptions`:
```python error_handling.py theme={null}
agent = create_agent(
model=init_chat_model("openai:gpt-4o-mini"),
tools=[get_weather],
middleware=[RestateMiddleware(run_options=RunOptions(max_attempts=3))],
)
```
By default, the middleware retries indefinitely with exponential backoff. Once Restate's retries are exhausted, the invocation fails with a `TerminalError` and won't be retried further.
To set a separate retry policy for LLM calls, pass [`RunOptions`](https://restatedev.github.io/sdk-typescript/types/_restatedev_restate-sdk.RunOptions.html) to `ctx.run()`:
```typescript theme={null}
// Retries up to 3 times with exponential backoff
const result = await ctx.run(
"LLM call",
async () => llmCall(messages, tools),
{ maxRetryAttempts: 3 }
);
```
Once these retries are exhausted, the invocation fails with a `TerminalError` and won't be retried further. You can catch the Terminal Error in your handler and act accordingly.
To set a separate retry policy for LLM calls, pass [`RunOptions`](https://github.com/restatedev/sdk-python/blob/main/python/restate/context.py#L37) to `ctx.run_typed()`:
```python theme={null}
# Retries up to 3 times with exponential backoff
result = await ctx.run_typed(
"LLM call",
llm_call,
RunOptions(max_attempts=3),
messages=messages,
tools=tools,
)
```
Once these retries are exhausted, the invocation fails with a `TerminalError` and won't be retried further. You can catch the Terminal Error in your handler and act accordingly.
## Tool execution errors
Restate makes tool execution resilient by retrying transient errors and propagating terminal ones.
### Transient errors
By default, the Vercel AI SDK converts any errors in tool executions into a message to the LLM, and the agent decides how to proceed. This is often desirable, as the LLM can decide to use a different tool or provide a fallback answer.
When you wrap external calls in Restate Context actions like `ctx.run`, Restate retries transient errors within the Context action before the result reaches the agent. This makes your tools resilient to network failures, database hiccups, and other temporary issues. For all operations that might suffer from transient errors, use Context actions:
```typescript {"CODE_LOAD::ts/src/tour/agents/inline-tool-errors.ts#here"} theme={null}
// Without ctx.run - error goes straight to agent
async function myTool() {
const result = await fetch("/api/data"); // Might fail due to network
// If this fails, agent gets the error immediately
}
// With ctx.run - Restate handles retries
async function myToolWithRestate(ctx: restate.Context) {
const result = await ctx.run("fetch-data", () => fetch("/api/data"));
// Network failures get retried automatically
// Only terminal errors reach the AI
}
```
Restate then retries the whole invocation according to the policy configured at the [service or handler level](/services/configuration#how-to-configure), or otherwise the [Restate server's default policy](/guides/error-handling#configure-restate-server-defaults).
Restate retries all transient errors to make your tools resilient to network failures, database hiccups, and other temporary issues.
By default, it uses the policy configured at the [service or handler level](/services/configuration#how-to-configure), or otherwise the [Restate server's default policy](/guides/error-handling#configure-restate-server-defaults).
Restate retries all transient errors to make your tools resilient to network failures, database hiccups, and other temporary issues.
By default, it uses the policy configured at the [service or handler level](/services/configuration#how-to-configure), or otherwise the [Restate server's default policy](/guides/error-handling#configure-restate-server-defaults).
Restate retries all transient errors to make your tools resilient to network failures, database hiccups, and other temporary issues.
By default, it uses the policy configured at the [service or handler level](/services/configuration#how-to-configure), or otherwise the [Restate server's default policy](/guides/error-handling#configure-restate-server-defaults).
Restate retries all transient errors to make your tools resilient to network failures, database hiccups, and other temporary issues.
By default, it uses the policy configured at the [service or handler level](/services/configuration#how-to-configure), or otherwise the [Restate server's default policy](/guides/error-handling#configure-restate-server-defaults).
Restate retries all transient errors to make your tools resilient to network failures, database hiccups, and other temporary issues.
By default, it uses the policy configured at the [service or handler level](/services/configuration#how-to-configure), or otherwise the [Restate server's default policy](/guides/error-handling#configure-restate-server-defaults).
### Setting a retry policy on run actions
If you do run actions in your tools, you can override the default retry policy by passing [`RunOptions`](https://restatedev.github.io/sdk-typescript/types/_restatedev_restate-sdk.RunOptions.html):
```ts {"CODE_LOAD::ts/src/ai/guides/errorhandling/error_handling.ts#retries"} theme={null}
const result = await ctx.run(
"fetch-data",
() => fetch("/api/data"),
{ maxRetryAttempts: 3 }
);
```
See [custom retry policies](/guides/error-handling#at-the-run-block-level) for more options. When retries are exhausted, the tool will fail with a Terminal Error.
If you do run actions in your tools, you can override the default retry policy by passing [`RunOptions`](https://github.com/restatedev/sdk-python/blob/main/python/restate/context.py#L37):
```python {"CODE_LOAD::python/src/ai/error_handling.py#retries"} theme={null}
result = await restate_context().run_typed(
"fetch data",
fetch_data,
RunOptions(max_attempts=3),
req=req,
)
```
See [custom retry policies](/guides/error-handling#at-the-run-block-level) for more options. When retries are exhausted, the tool will fail with a Terminal Error.
If you do run actions in your tools, you can override the default retry policy by passing [`RunOptions`](https://github.com/restatedev/sdk-python/blob/main/python/restate/context.py#L37):
```python {"CODE_LOAD::python/src/ai/error_handling.py#retries"} theme={null}
result = await restate_context().run_typed(
"fetch data",
fetch_data,
RunOptions(max_attempts=3),
req=req,
)
```
See [custom retry policies](/guides/error-handling#at-the-run-block-level) for more options. When retries are exhausted, the tool will fail with a Terminal Error.
If you do run actions in your tools, you can override the default retry policy by passing [`RunOptions`](https://github.com/restatedev/sdk-python/blob/main/python/restate/context.py#L37):
```python {"CODE_LOAD::python/src/ai/error_handling.py#retries"} theme={null}
result = await restate_context().run_typed(
"fetch data",
fetch_data,
RunOptions(max_attempts=3),
req=req,
)
```
See [custom retry policies](/guides/error-handling#at-the-run-block-level) for more options. When retries are exhausted, the tool will fail with a Terminal Error.
If you do `ctx.run` actions in your tools, you can override the default retry policy by passing [`RunOptions`](https://restatedev.github.io/sdk-typescript/types/_restatedev_restate-sdk.RunOptions.html):
```ts {"CODE_LOAD::ts/src/ai/guides/errorhandling/error_handling.ts#retries"} theme={null}
const result = await ctx.run(
"fetch-data",
() => fetch("/api/data"),
{ maxRetryAttempts: 3 }
);
```
See [custom retry policies](/guides/error-handling#at-the-run-block-level) for more options. When retries are exhausted, the tool will fail with a Terminal Error.
For `ctx.run_typed` actions specifically, you can override the default retry policy by passing [`RunOptions`](https://github.com/restatedev/sdk-python/blob/main/python/restate/context.py#L37):
```python {"CODE_LOAD::python/src/ai/error_handling.py#retries"} theme={null}
result = await restate_context().run_typed(
"fetch data",
fetch_data,
RunOptions(max_attempts=3),
req=req,
)
```
See [custom retry policies](/guides/error-handling#at-the-run-block-level) for more options. When retries are exhausted, the tool will fail with a Terminal Error.
### Terminal errors
For errors that should not be retried (invalid input, business rule violations, resource not found), use a `TerminalError` in your tool. Restate does not retry these:
```typescript {"CODE_LOAD::ts/src/tour/agents/terminal_error.ts#terminal_error"} theme={null}
throw new TerminalError("This tool is not allowed to run for this input.");
```
By default, Vercel AI converts the terminal error into a message to the LLM, and the agent decides how to proceed.
If you want to treat terminal tool errors as permanent failures and stop the agent instead, the Restate middleware provides two utilities:
**To fail the agent on terminal tool errors**, rethrow the error in `onStepFinish`:
```typescript errorhandling/fail-on-terminal-tool-agent.ts {"CODE_LOAD::https://raw.githubusercontent.com/restatedev/ai-examples/refs/heads/main/vercel-ai/tour-of-agents/src/errorhandling/fail-on-terminal-tool-agent.ts#option2"} theme={null}
const { text } = await generateText({
model,
tools: {
getWeather: tool({
description: "Get the current weather for a given city.",
inputSchema: z.object({ city: z.string() }),
execute: async ({ city }) => {
return await ctx.run("get weather", () => fetchWeather(city));
},
}),
},
stopWhen: [stepCountIs(5)],
onStepFinish: rethrowTerminalToolError,
system: "You are a helpful agent that provides weather updates.",
messages: [{ role: "user", content: prompt }],
});
```
To stop the agent on terminal tool errors and handle it after the agent finishes, you can use `hasTerminalToolError` in `stopWhen` and then inspect the steps for errors:
```typescript errorhandling/stop-on-terminal-tool-agent.ts {"CODE_LOAD::https://raw.githubusercontent.com/restatedev/ai-examples/refs/heads/main/vercel-ai/tour-of-agents/src/errorhandling/stop-on-terminal-tool-agent.ts#option3"} theme={null}
const { steps, text } = await generateText({
model,
tools: {
getWeather: tool({
description: "Get the current weather for a given city.",
inputSchema: z.object({ city: z.string() }),
execute: async ({ city }) => {
return await ctx.run("get weather", () => fetchWeather(city));
},
}),
},
stopWhen: [stepCountIs(5), hasTerminalToolError],
system: "You are a helpful agent that provides weather updates.",
messages: [{ role: "user", content: prompt }],
});
const terminalSteps = getTerminalToolSteps(steps);
if (terminalSteps.length > 0) {
// Do something with the terminal tool error steps
}
```
```python {"CODE_LOAD::python/src/ai/error_handling.py#terminal"} theme={null}
from restate import TerminalError
raise TerminalError("This tool is not allowed to run for this input.")
```
The Restate OpenAI integration raises terminal errors to your handler, where you can catch and handle them:
```python {"CODE_LOAD::https://raw.githubusercontent.com/restatedev/ai-examples/refs/heads/main/openai-agents/tour-of-agents/app/error_handling.py#handle"} theme={null}
@agent_service.handler()
async def run(_ctx: restate.Context, req: WeatherPrompt) -> str:
try:
run_opts = RunOptions(
max_attempts=3, initial_retry_interval=timedelta(seconds=2)
)
result = await DurableRunner.run(agent, req.message, run_options=run_opts)
except restate.TerminalError as e:
# Handle terminal errors gracefully
return f"The agent couldn't complete the request: {e.message}"
return result.final_output
```
The OpenAI Agent SDK also allows setting `failure_error_function` to `None`, which will rethrow any error in the agent execution as-is.
Also for example invalid LLM responses (e.g. tool call with invalid arguments or to a tool that doesn't exist).
The error will then lead to Restate retries. Since the error isn't transient, the invocation will be paused when the retries are exhausted, and will require manual intervention.
Therefore, we do not recommend using this setting and instead recommend handling these errors appropriately in your agent logic.
```python {"CODE_LOAD::python/src/ai/error_handling.py#terminal"} theme={null}
from restate import TerminalError
raise TerminalError("This tool is not allowed to run for this input.")
```
You can catch these terminal errors in your handler and handle them accordingly:
```python {"CODE_LOAD::https://raw.githubusercontent.com/restatedev/ai-examples/refs/heads/main/google-adk/tour-of-agents/app/error_handling.py#handle"} theme={null}
@agent_service.handler()
async def run(ctx: restate.ObjectContext, req: WeatherPrompt) -> str | None:
try:
events = runner.run_async(
user_id=ctx.key(),
session_id=req.session_id,
new_message=Content(role="user", parts=[Part.from_text(text=req.message)]),
)
return await parse_agent_response(events)
except TerminalError as e:
# Handle the error appropriately, e.g., log it or return a default response
return "Sorry, I'm unable to process your request at the moment."
```
```python {"CODE_LOAD::python/src/ai/error_handling.py#terminal"} theme={null}
from restate import TerminalError
raise TerminalError("This tool is not allowed to run for this input.")
```
You can catch these terminal errors in your handler and handle them accordingly:
```python {"CODE_LOAD::https://raw.githubusercontent.com/restatedev/ai-examples/refs/heads/main/pydantic-ai/tour-of-agents/app/error_handling.py#handle"} theme={null}
@agent_service.handler()
async def run(_ctx: restate.Context, req: WeatherPrompt) -> str:
try:
result = await restate_agent.run(req.message)
except TerminalError as e:
# Handle terminal errors gracefully
return f"The agent couldn't complete the request: {e.message}"
return result.output
```
When agent tools use Restate Context actions like `ctx.run`, Restate automatically retries transient errors in these operations. This makes your tools resilient to network failures, database hiccups, and other temporary issues. For all operations that might suffer from transient errors, use Context actions.
For example, wrapping a tool call in `restate_context().run_typed()` makes it durable with automatic retries:
```python error_handling.py {"CODE_LOAD::https://raw.githubusercontent.com/restatedev/ai-examples/refs/heads/main/langchain-python/tour-of-agents/app/error_handling.py#here"} theme={null}
@tool
async def get_weather(city: WeatherRequest) -> WeatherResponse:
"""Get the current weather for a given city."""
return await restate_context().run_typed(
"get weather", fetch_weather, RunOptions(max_attempts=3), req=city
)
```
For errors that should not be retried, raise a terminal error:
```python theme={null}
from restate import TerminalError
raise TerminalError("This tool is not allowed to run for this input.")
```
Restate retries tool executions until they succeed. Terminal errors propagate past LangChain's tool-error handling back to the service handler, where you can catch them:
```python error_handling.py {"CODE_LOAD::https://raw.githubusercontent.com/restatedev/ai-examples/refs/heads/main/langchain-python/tour-of-agents/app/error_handling.py#handle"} theme={null}
try:
result = await agent.ainvoke({"messages": req.message})
except restate.TerminalError as e:
return f"The agent couldn't complete the request: {e.message}"
```
```typescript {"CODE_LOAD::ts/src/tour/agents/terminal_error.ts#terminal_error"} theme={null}
throw new TerminalError("This tool is not allowed to run for this input.");
```
You can catch and handle terminal errors in your agent logic if needed.
```python {"CODE_LOAD::python/src/ai/error_handling.py#terminal"} theme={null}
from restate import TerminalError
raise TerminalError("This tool is not allowed to run for this input.")
```
You can catch and handle terminal errors in your agent logic if needed.
To learn more about error handling with Restate, consult the [error handling guide](/guides/error-handling).
## Combining with rollback
For multi-step agent workflows where steps have side effects (bookings, payments, emails), combine terminal errors with [compensation/rollback patterns](/ai/patterns/rollback) to undo completed work before finishing.