# Table of Contents - [Model Context Protocol (MCP) - PydanticAI](#model-context-protocol-mcp-pydanticai) - [Installation - PydanticAI](#installation-pydanticai) - [Getting Help - PydanticAI](#getting-help-pydanticai) - [PydanticAI](#pydanticai) - [Troubleshooting - PydanticAI](#troubleshooting-pydanticai) - [Contributing - PydanticAI](#contributing-pydanticai) - [Dependencies - PydanticAI](#dependencies-pydanticai) - [Agents - PydanticAI](#agents-pydanticai) - [Function Tools - PydanticAI](#function-tools-pydanticai) - [Models - PydanticAI](#models-pydanticai) - [Common Tools - PydanticAI](#common-tools-pydanticai) - [Messages and chat history - PydanticAI](#messages-and-chat-history-pydanticai) - [Debugging and Monitoring - PydanticAI](#debugging-and-monitoring-pydanticai) - [Results - PydanticAI](#results-pydanticai) - [Multi-agent Applications - PydanticAI](#multi-agent-applications-pydanticai) - [Unit testing - PydanticAI](#unit-testing-pydanticai) - [Client - PydanticAI](#client-pydanticai) - [Evals - PydanticAI](#evals-pydanticai) - [Graphs - PydanticAI](#graphs-pydanticai) - [Image, Audio & Document Input - PydanticAI](#image-audio-document-input-pydanticai) - [Server - PydanticAI](#server-pydanticai) - [MCP Run Python - PydanticAI](#mcp-run-python-pydanticai) - [Examples - PydanticAI](#examples-pydanticai) - [Command Line Interface (CLI) - PydanticAI](#command-line-interface-cli-pydanticai) - [Pydantic Model - PydanticAI](#pydantic-model-pydanticai) - [Bank support - PydanticAI](#bank-support-pydanticai) - [Weather agent - PydanticAI](#weather-agent-pydanticai) - [Flight booking - PydanticAI](#flight-booking-pydanticai) - [Stream markdown - PydanticAI](#stream-markdown-pydanticai) - [SQL Generation - PydanticAI](#sql-generation-pydanticai) - [Stream whales - PydanticAI](#stream-whales-pydanticai) - [Question Graph - PydanticAI](#question-graph-pydanticai) - [RAG - PydanticAI](#rag-pydanticai) - [Chat App with FastAPI - PydanticAI](#chat-app-with-fastapi-pydanticai) - [pydantic_ai.result - PydanticAI](#pydantic-ai-result-pydanticai) - [pydantic_ai.tools - PydanticAI](#pydantic-ai-tools-pydanticai) - [pydantic_ai.common_tools - PydanticAI](#pydantic-ai-common-tools-pydanticai) - [pydantic_ai.usage - PydanticAI](#pydantic-ai-usage-pydanticai) - [pydantic_ai.format_as_xml - PydanticAI](#pydantic-ai-format-as-xml-pydanticai) - [pydantic_ai.mcp - PydanticAI](#pydantic-ai-mcp-pydanticai) - [pydantic_ai.models - PydanticAI](#pydantic-ai-models-pydanticai) - [pydantic_ai.models.openai - PydanticAI](#pydantic-ai-models-openai-pydanticai) - [pydantic_ai.models.anthropic - PydanticAI](#pydantic-ai-models-anthropic-pydanticai) - [pydantic_ai.models.bedrock - PydanticAI](#pydantic-ai-models-bedrock-pydanticai) - [pydantic_ai.models.cohere - PydanticAI](#pydantic-ai-models-cohere-pydanticai) - [pydantic_ai.models.gemini - PydanticAI](#pydantic-ai-models-gemini-pydanticai) - [pydantic_ai.agent - PydanticAI](#pydantic-ai-agent-pydanticai) - [pydantic_ai.messages - PydanticAI](#pydantic-ai-messages-pydanticai) - [pydantic_ai.models.groq - PydanticAI](#pydantic-ai-models-groq-pydanticai) - [pydantic_ai.exceptions - PydanticAI](#pydantic-ai-exceptions-pydanticai) - [pydantic_ai.settings - PydanticAI](#pydantic-ai-settings-pydanticai) - [pydantic_ai.models.instrumented - PydanticAI](#pydantic-ai-models-instrumented-pydanticai) - [pydantic_ai.models.test - PydanticAI](#pydantic-ai-models-test-pydanticai) --- # Model Context Protocol (MCP) - PydanticAI Model Context Protocol (MCP) ============================ PydanticAI supports [Model Context Protocol (MCP)](https://modelcontextprotocol.io) in three ways: 1. [Agents](../agents/) act as an MCP Client, connecting to MCP servers to use their tools, [learn more …](client/) 2. Agents can be used within MCP servers, [learn more …](server/) 3. As part of PydanticAI, we're building a number of MCP servers, [see below](#mcp-servers) What is MCP? ------------ The Model Context Protocol is a standardized protocol that allow AI applications (including programmatic agents like PydanticAI, coding agents like [cursor](https://www.cursor.com/) , and desktop applications like [Claude Desktop](https://claude.ai/download) ) to connect to external tools and services using a common interface. As with other protocols, the dream of MCP is that a wide range of applications can speak to each other without the need for specific integrations. There is a great list of MCP servers at [github.com/modelcontextprotocol/servers](https://github.com/modelcontextprotocol/servers) . Some examples of what this means: * PydanticAI could use a web search service implemented as an MCP server to implement a deep research agent * Cursor could connect to the [Pydantic Logfire](https://github.com/pydantic/logfire-mcp) MCP server to search logs, traces and metrics to gain context while fixing a bug * PydanticAI, or any other MCP client could connect to our [Run Python](run-python/) MCP server to run arbitrary Python code in a sandboxed environment MCP Servers ----------- To add functionality to PydanticAI while making it as widely usable as possible, we're implementing some functionality as MCP servers. So far, we've only implemented one MCP server as part of PydanticAI: * [Run Python](run-python/) : A sandboxed Python interpreter that can run arbitrary code, with a focus on security and safety. --- # Installation - PydanticAI Installation ============ PydanticAI is available on PyPI as [`pydantic-ai`](https://pypi.org/project/pydantic-ai/) so installation is as simple as: pipuv `pip install pydantic-ai` `uv add pydantic-ai` (Requires Python 3.9+) This installs the `pydantic_ai` package, core dependencies, and libraries required to use all the models included in PydanticAI. If you want to use a specific model, you can install the ["slim"](#slim-install) version of PydanticAI. Use with Pydantic Logfire ------------------------- PydanticAI has an excellent (but completely optional) integration with [Pydantic Logfire](https://pydantic.dev/logfire) to help you view and understand agent runs. To use Logfire with PydanticAI, install `pydantic-ai` or `pydantic-ai-slim` with the `logfire` optional group: pipuv `pip install "pydantic-ai[logfire]"` `uv add "pydantic-ai[logfire]"` From there, follow the [Logfire setup docs](../logfire/#using-logfire) to configure Logfire. Running Examples ---------------- We distribute the [`pydantic_ai_examples`](https://github.com/pydantic/pydantic-ai/tree/main/examples/pydantic_ai_examples) directory as a separate PyPI package ([`pydantic-ai-examples`](https://pypi.org/project/pydantic-ai-examples/) ) to make examples extremely easy to customize and run. To install examples, use the `examples` optional group: pipuv `pip install "pydantic-ai[examples]"` `uv add "pydantic-ai[examples]"` To run the examples, follow instructions in the [examples docs](../examples/) . Slim Install ------------ If you know which model you're going to use and want to avoid installing superfluous packages, you can use the [`pydantic-ai-slim`](https://pypi.org/project/pydantic-ai-slim/) package. For example, if you're using just [`OpenAIModel`](../api/models/openai/#pydantic_ai.models.openai.OpenAIModel) , you would run: pipuv `pip install "pydantic-ai-slim[openai]"` `uv add "pydantic-ai-slim[openai]"` `pydantic-ai-slim` has the following optional groups: * `logfire` — installs [`logfire`](../logfire/) [PyPI ↗](https://pypi.org/project/logfire) * `evals` — installs [`pydantic-evals`](../evals/) [PyPI ↗](https://pypi.org/project/pydantic-evals) * `openai` — installs `openai` [PyPI ↗](https://pypi.org/project/openai) * `vertexai` — installs `google-auth` [PyPI ↗](https://pypi.org/project/google-auth) and `requests` [PyPI ↗](https://pypi.org/project/requests) * `anthropic` — installs `anthropic` [PyPI ↗](https://pypi.org/project/anthropic) * `groq` — installs `groq` [PyPI ↗](https://pypi.org/project/groq) * `mistral` — installs `mistralai` [PyPI ↗](https://pypi.org/project/mistralai) * `cohere` - installs `cohere` [PyPI ↗](https://pypi.org/project/cohere) * `duckduckgo` - installs `duckduckgo-search` [PyPI ↗](https://pypi.org/project/duckduckgo-search) * `tavily` - installs `tavily-python` [PyPI ↗](https://pypi.org/project/tavily-python) See the [models](../models/) documentation for information on which optional dependencies are required for each model. You can also install dependencies for multiple models and use cases, for example: pipuv `pip install "pydantic-ai-slim[openai,vertexai,logfire]"` `uv add "pydantic-ai-slim[openai,vertexai,logfire]"` --- # Getting Help - PydanticAI Getting Help ============ If you need help getting started with PydanticAI or with advanced usage, the following sources may be useful. Slack ----- Join the `#pydantic-ai` channel in the [Pydantic Slack](https://logfire.pydantic.dev/docs/join-slack/) to ask questions, get help, and chat about PydanticAI. There's also channels for Pydantic, Logfire, and FastUI. If you're on a [Logfire](https://pydantic.dev/logfire) Pro plan, you can also get a dedicated private slack collab channel with us. GitHub Issues ------------- The [PydanticAI GitHub Issues](https://github.com/pydantic/pydantic-ai/issues) are a great place to ask questions and give us feedback. --- # PydanticAI Introduction ============ ![PydanticAI](./img/pydantic-ai-dark.svg#only-dark) ![PydanticAI](./img/pydantic-ai-light.svg#only-light) _Agent Framework / shim to use Pydantic with LLMs_ [![CI](https://github.com/pydantic/pydantic-ai/actions/workflows/ci.yml/badge.svg?event=push)](https://github.com/pydantic/pydantic-ai/actions/workflows/ci.yml?query=branch%3Amain) [![Coverage](https://coverage-badge.samuelcolvin.workers.dev/pydantic/pydantic-ai.svg)](https://coverage-badge.samuelcolvin.workers.dev/redirect/pydantic/pydantic-ai) [![PyPI](https://img.shields.io/pypi/v/pydantic-ai.svg)](https://pypi.python.org/pypi/pydantic-ai) [![versions](https://img.shields.io/pypi/pyversions/pydantic-ai.svg)](https://github.com/pydantic/pydantic-ai) [![license](https://img.shields.io/github/license/pydantic/pydantic-ai.svg)](https://github.com/pydantic/pydantic-ai/blob/main/LICENSE) PydanticAI is a Python agent framework designed to make it less painful to build production grade applications with Generative AI. FastAPI revolutionized web development by offering an innovative and ergonomic design, built on the foundation of [Pydantic](https://docs.pydantic.dev) . Similarly, virtually every agent framework and LLM library in Python uses Pydantic, yet when we began to use LLMs in [Pydantic Logfire](https://pydantic.dev/logfire) , we couldn't find anything that gave us the same feeling. We built PydanticAI with one simple aim: to bring that FastAPI feeling to GenAI app development. Why use PydanticAI ------------------ * **Built by the Pydantic Team**: Built by the team behind [Pydantic](https://docs.pydantic.dev/latest/) (the validation layer of the OpenAI SDK, the Anthropic SDK, LangChain, LlamaIndex, AutoGPT, Transformers, CrewAI, Instructor and many more). * **Model-agnostic**: Supports OpenAI, Anthropic, Gemini, Deepseek, Ollama, Groq, Cohere, and Mistral, and there is a simple interface to implement support for [other models](models/) . * **Pydantic Logfire Integration**: Seamlessly [integrates](logfire/) with [Pydantic Logfire](https://pydantic.dev/logfire) for real-time debugging, performance monitoring, and behavior tracking of your LLM-powered applications. * **Type-safe**: Designed to make [type checking](agents/#static-type-checking) as powerful and informative as possible for you. * **Python-centric Design**: Leverages Python's familiar control flow and agent composition to build your AI-driven projects, making it easy to apply standard Python best practices you'd use in any other (non-AI) project. * **Structured Responses**: Harnesses the power of [Pydantic](https://docs.pydantic.dev/latest/) to [validate and structure](results/#structured-result-validation) model outputs, ensuring responses are consistent across runs. * **Dependency Injection System**: Offers an optional [dependency injection](dependencies/) system to provide data and services to your agent's [system prompts](agents/#system-prompts) , [tools](tools/) and [result validators](results/#result-validators-functions) . This is useful for testing and eval-driven iterative development. * **Streamed Responses**: Provides the ability to [stream](results/#streamed-results) LLM outputs continuously, with immediate validation, ensuring rapid and accurate results. * **Graph Support**: [Pydantic Graph](graph/) provides a powerful way to define graphs using typing hints, this is useful in complex applications where standard control flow can degrade to spaghetti code. Hello World Example ------------------- Here's a minimal example of PydanticAI: hello\_world.py `from pydantic_ai import Agent agent = Agent( # (1)! 'google-gla:gemini-1.5-flash', system_prompt='Be concise, reply with one sentence.', # (2)! ) result = agent.run_sync('Where does "hello world" come from?') # (3)! print(result.data) """ The first known use of "hello, world" was in a 1974 textbook about the C programming language. """` 1. We configure the agent to use [Gemini 1.5's Flash](api/models/gemini/) model, but you can also set the model when running the agent. 2. Register a static [system prompt](agents/#system-prompts) using a keyword argument to the agent. 3. [Run the agent](agents/#running-agents) synchronously, conducting a conversation with the LLM. _(This example is complete, it can be run "as is")_ The exchange should be very short: PydanticAI will send the system prompt and the user query to the LLM, the model will return a text response. Not very interesting yet, but we can easily add "tools", dynamic system prompts, and structured responses to build more powerful agents. Tools & Dependency Injection Example ------------------------------------ Here is a concise example using PydanticAI to build a support agent for a bank: bank\_support.py `from dataclasses import dataclass from pydantic import BaseModel, Field from pydantic_ai import Agent, RunContext from bank_database import DatabaseConn @dataclass class SupportDependencies: # (3)! customer_id: int db: DatabaseConn # (12)! class SupportResult(BaseModel): # (13)! support_advice: str = Field(description='Advice returned to the customer') block_card: bool = Field(description="Whether to block the customer's card") risk: int = Field(description='Risk level of query', ge=0, le=10) support_agent = Agent( # (1)! 'openai:gpt-4o', # (2)! deps_type=SupportDependencies, result_type=SupportResult, # (9)! system_prompt=( # (4)! 'You are a support agent in our bank, give the ' 'customer support and judge the risk level of their query.' ), ) @support_agent.system_prompt # (5)! async def add_customer_name(ctx: RunContext[SupportDependencies]) -> str: customer_name = await ctx.deps.db.customer_name(id=ctx.deps.customer_id) return f"The customer's name is {customer_name!r}" @support_agent.tool # (6)! async def customer_balance( ctx: RunContext[SupportDependencies], include_pending: bool ) -> float: """Returns the customer's current account balance.""" # (7)! return await ctx.deps.db.customer_balance( id=ctx.deps.customer_id, include_pending=include_pending, ) ... # (11)! async def main(): deps = SupportDependencies(customer_id=123, db=DatabaseConn()) result = await support_agent.run('What is my balance?', deps=deps) # (8)! print(result.data) # (10)! """ support_advice='Hello John, your current account balance, including pending transactions, is $123.45.' block_card=False risk=1 """ result = await support_agent.run('I just lost my card!', deps=deps) print(result.data) """ support_advice="I'm sorry to hear that, John. We are temporarily blocking your card to prevent unauthorized transactions." block_card=True risk=8 """` 1. This [agent](agents/) will act as first-tier support in a bank. Agents are generic in the type of dependencies they accept and the type of result they return. In this case, the support agent has type `Agent[SupportDependencies, SupportResult]`. 2. Here we configure the agent to use [OpenAI's GPT-4o model](api/models/openai/) , you can also set the model when running the agent. 3. The `SupportDependencies` dataclass is used to pass data, connections, and logic into the model that will be needed when running [system prompt](agents/#system-prompts) and [tool](tools/) functions. PydanticAI's system of dependency injection provides a [type-safe](agents/#static-type-checking) way to customise the behavior of your agents, and can be especially useful when running [unit tests](testing/) and evals. 4. Static [system prompts](agents/#system-prompts) can be registered with the [`system_prompt` keyword argument](api/agent/#pydantic_ai.agent.Agent.__init__) to the agent. 5. Dynamic [system prompts](agents/#system-prompts) can be registered with the [`@agent.system_prompt`](api/agent/#pydantic_ai.agent.Agent.system_prompt) decorator, and can make use of dependency injection. Dependencies are carried via the [`RunContext`](api/tools/#pydantic_ai.tools.RunContext) argument, which is parameterized with the `deps_type` from above. If the type annotation here is wrong, static type checkers will catch it. 6. [`tool`](tools/) let you register functions which the LLM may call while responding to a user. Again, dependencies are carried via [`RunContext`](api/tools/#pydantic_ai.tools.RunContext) , any other arguments become the tool schema passed to the LLM. Pydantic is used to validate these arguments, and errors are passed back to the LLM so it can retry. 7. The docstring of a tool is also passed to the LLM as the description of the tool. Parameter descriptions are [extracted](tools/#function-tools-and-schema) from the docstring and added to the parameter schema sent to the LLM. 8. [Run the agent](agents/#running-agents) asynchronously, conducting a conversation with the LLM until a final response is reached. Even in this fairly simple case, the agent will exchange multiple messages with the LLM as tools are called to retrieve a result. 9. The response from the agent will, be guaranteed to be a `SupportResult`, if validation fails [reflection](agents/#reflection-and-self-correction) will mean the agent is prompted to try again. 10. The result will be validated with Pydantic to guarantee it is a `SupportResult`, since the agent is generic, it'll also be typed as a `SupportResult` to aid with static type checking. 11. In a real use case, you'd add more tools and a longer system prompt to the agent to extend the context it's equipped with and support it can provide. 12. This is a simple sketch of a database connection, used to keep the example short and readable. In reality, you'd be connecting to an external database (e.g. PostgreSQL) to get information about customers. 13. This [Pydantic](https://docs.pydantic.dev) model is used to constrain the structured data returned by the agent. From this simple definition, Pydantic builds the JSON Schema that tells the LLM how to return the data, and performs validation to guarantee the data is correct at the end of the run. Complete `bank_support.py` example The code included here is incomplete for the sake of brevity (the definition of `DatabaseConn` is missing); you can find the complete `bank_support.py` example [here](examples/bank-support/) . Instrumentation with Pydantic Logfire ------------------------------------- To understand the flow of the above runs, we can watch the agent in action using Pydantic Logfire. To do this, we need to set up logfire, and add the following to our code: bank\_support\_with\_logfire.py `... from pydantic_ai import Agent, RunContext from bank_database import DatabaseConn import logfire logfire.configure() # (1)! logfire.instrument_asyncpg() # (2)! ... support_agent = Agent( 'openai:gpt-4o', deps_type=SupportDependencies, result_type=SupportResult, system_prompt=( 'You are a support agent in our bank, give the ' 'customer support and judge the risk level of their query.' ), instrument=True, )` 1. Configure logfire, this will fail if project is not set up. 2. In our demo, `DatabaseConn` uses `asyncpg` to connect to a PostgreSQL database, so [`logfire.instrument_asyncpg()`](https://magicstack.github.io/asyncpg/current/) is used to log the database queries. That's enough to get the following view of your agent in action: See [Monitoring and Performance](logfire/) to learn more. Next Steps ---------- To try PydanticAI yourself, follow the instructions [in the examples](examples/) . Read the [docs](agents/) to learn more about building applications with PydanticAI. Read the [API Reference](api/agent/) to understand PydanticAI's interface. --- # Troubleshooting - PydanticAI Troubleshooting =============== Below are suggestions on how to fix some common errors you might encounter while using PydanticAI. If the issue you're experiencing is not listed below or addressed in the documentation, please feel free to ask in the [Pydantic Slack](../help/) or create an issue on [GitHub](https://github.com/pydantic/pydantic-ai/issues) . Jupyter Notebook Errors ----------------------- ### `RuntimeError: This event loop is already running` This error is caused by conflicts between the event loops in Jupyter notebook and PydanticAI's. One way to manage these conflicts is by using [`nest-asyncio`](https://pypi.org/project/nest-asyncio/) . Namely, before you execute any agent runs, do the following: `import nest_asyncio nest_asyncio.apply()` Note: This fix also applies to Google Colab. API Key Configuration --------------------- ### `UserError: API key must be provided or set in the [MODEL]_API_KEY environment variable` If you're running into issues with setting the API key for your model, visit the [Models](../models/) page to learn more about how to set an environment variable and/or pass in an `api_key` argument. Monitoring HTTPX Requests ------------------------- You can use custom `httpx` clients in your models in order to access specific requests, responses, and headers at runtime. It's particularly helpful to use `logfire`'s [HTTPX integration](../logfire/#monitoring-httpx-requests) to monitor the above. --- # Contributing - PydanticAI Contributing ============ We'd love you to contribute to PydanticAI! Installation and Setup ---------------------- Clone your fork and cd into the repo directory `git clone git@github.com:/pydantic-ai.git cd pydantic-ai` Install `uv` (version 0.4.30 or later) and `pre-commit` We use pipx here, for other options see: * [`uv` install docs](https://docs.astral.sh/uv/getting-started/installation/) * [`pre-commit` install docs](https://pre-commit.com/#install) To get `pipx` itself, see [these docs](https://pypa.github.io/pipx/) `pipx install uv pre-commit` Install `pydantic-ai`, all dependencies and pre-commit hooks `make install` Running Tests etc. ------------------ We use `make` to manage most commands you'll need to run. For details on available commands, run: `make help` To run code formatting, linting, static type checks, and tests with coverage report generation, run: `make` Documentation Changes --------------------- To run the documentation page locally, run: `uv run mkdocs serve` Rules for adding new models to PydanticAI ----------------------------------------- To avoid an excessive workload for the maintainers of PydanticAI, we can't accept all model contributions, so we're setting the following rules for when we'll accept new models and when we won't. This should hopefully reduce the chances of disappointment and wasted work. * To add a new model with an extra dependency, that dependency needs > 500k monthly downloads from PyPI consistently over 3 months or more * To add a new model which uses another models logic internally and has no extra dependencies, that model's GitHub org needs > 20k stars in total * For any other model that's just a custom URL and API key, we're happy to add a one-paragraph description with a link and instructions on the URL to use * For any other model that requires more logic, we recommend you release your own Python package `pydantic-ai-xxx`, which depends on [`pydantic-ai-slim`](../install/#slim-install) and implements a model that inherits from our [`Model`](../api/models/base/#pydantic_ai.models.Model) ABC If you're unsure about adding a model, please [create an issue](https://github.com/pydantic/pydantic-ai/issues) . --- # Dependencies - PydanticAI Dependencies ============ PydanticAI uses a dependency injection system to provide data and services to your agent's [system prompts](../agents/#system-prompts) , [tools](../tools/) and [result validators](../results/#result-validators-functions) . Matching PydanticAI's design philosophy, our dependency system tries to use existing best practice in Python development rather than inventing esoteric "magic", this should make dependencies type-safe, understandable easier to test and ultimately easier to deploy in production. Defining Dependencies --------------------- Dependencies can be any python type. While in simple cases you might be able to pass a single object as a dependency (e.g. an HTTP connection), [dataclasses](https://docs.python.org/3/library/dataclasses.html#module-dataclasses) are generally a convenient container when your dependencies included multiple objects. Here's an example of defining an agent that requires dependencies. (**Note:** dependencies aren't actually used in this example, see [Accessing Dependencies](#accessing-dependencies) below) unused\_dependencies.py `from dataclasses import dataclass import httpx from pydantic_ai import Agent @dataclass class MyDeps: # (1)! api_key: str http_client: httpx.AsyncClient agent = Agent( 'openai:gpt-4o', deps_type=MyDeps, # (2)! ) async def main(): async with httpx.AsyncClient() as client: deps = MyDeps('foobar', client) result = await agent.run( 'Tell me a joke.', deps=deps, # (3)! ) print(result.data) #> Did you hear about the toothpaste scandal? They called it Colgate.` 1. Define a dataclass to hold dependencies. 2. Pass the dataclass type to the `deps_type` argument of the [`Agent` constructor](../api/agent/#pydantic_ai.agent.Agent.__init__) . **Note**: we're passing the type here, NOT an instance, this parameter is not actually used at runtime, it's here so we can get full type checking of the agent. 3. When running the agent, pass an instance of the dataclass to the `deps` parameter. _(This example is complete, it can be run "as is" — you'll need to add `asyncio.run(main())` to run `main`)_ Accessing Dependencies ---------------------- Dependencies are accessed through the [`RunContext`](../api/tools/#pydantic_ai.tools.RunContext) type, this should be the first parameter of system prompt functions etc. system\_prompt\_dependencies.py `from dataclasses import dataclass import httpx from pydantic_ai import Agent, RunContext @dataclass class MyDeps: api_key: str http_client: httpx.AsyncClient agent = Agent( 'openai:gpt-4o', deps_type=MyDeps, ) @agent.system_prompt # (1)! async def get_system_prompt(ctx: RunContext[MyDeps]) -> str: # (2)! response = await ctx.deps.http_client.get( # (3)! 'https://example.com', headers={'Authorization': f'Bearer {ctx.deps.api_key}'}, # (4)! ) response.raise_for_status() return f'Prompt: {response.text}' async def main(): async with httpx.AsyncClient() as client: deps = MyDeps('foobar', client) result = await agent.run('Tell me a joke.', deps=deps) print(result.data) #> Did you hear about the toothpaste scandal? They called it Colgate.` 1. [`RunContext`](../api/tools/#pydantic_ai.tools.RunContext) may optionally be passed to a [`system_prompt`](../api/agent/#pydantic_ai.agent.Agent.system_prompt) function as the only argument. 2. [`RunContext`](../api/tools/#pydantic_ai.tools.RunContext) is parameterized with the type of the dependencies, if this type is incorrect, static type checkers will raise an error. 3. Access dependencies through the [`.deps`](../api/tools/#pydantic_ai.tools.RunContext.deps) attribute. 4. Access dependencies through the [`.deps`](../api/tools/#pydantic_ai.tools.RunContext.deps) attribute. _(This example is complete, it can be run "as is" — you'll need to add `asyncio.run(main())` to run `main`)_ ### Asynchronous vs. Synchronous dependencies [System prompt functions](../agents/#system-prompts) , [function tools](../tools/) and [result validators](../results/#result-validators-functions) are all run in the async context of an agent run. If these functions are not coroutines (e.g. `async def`) they are called with [`run_in_executor`](https://docs.python.org/3/library/asyncio-eventloop.html#asyncio.loop.run_in_executor) in a thread pool, it's therefore marginally preferable to use `async` methods where dependencies perform IO, although synchronous dependencies should work fine too. `run` vs. `run_sync` and Asynchronous vs. Synchronous dependencies Whether you use synchronous or asynchronous dependencies, is completely independent of whether you use `run` or `run_sync` — `run_sync` is just a wrapper around `run` and agents are always run in an async context. Here's the same example as above, but with a synchronous dependency: sync\_dependencies.py `from dataclasses import dataclass import httpx from pydantic_ai import Agent, RunContext @dataclass class MyDeps: api_key: str http_client: httpx.Client # (1)! agent = Agent( 'openai:gpt-4o', deps_type=MyDeps, ) @agent.system_prompt def get_system_prompt(ctx: RunContext[MyDeps]) -> str: # (2)! response = ctx.deps.http_client.get( 'https://example.com', headers={'Authorization': f'Bearer {ctx.deps.api_key}'} ) response.raise_for_status() return f'Prompt: {response.text}' async def main(): deps = MyDeps('foobar', httpx.Client()) result = await agent.run( 'Tell me a joke.', deps=deps, ) print(result.data) #> Did you hear about the toothpaste scandal? They called it Colgate.` 1. Here we use a synchronous `httpx.Client` instead of an asynchronous `httpx.AsyncClient`. 2. To match the synchronous dependency, the system prompt function is now a plain function, not a coroutine. _(This example is complete, it can be run "as is" — you'll need to add `asyncio.run(main())` to run `main`)_ Full Example ------------ As well as system prompts, dependencies can be used in [tools](../tools/) and [result validators](../results/#result-validators-functions) . full\_example.py `from dataclasses import dataclass import httpx from pydantic_ai import Agent, ModelRetry, RunContext @dataclass class MyDeps: api_key: str http_client: httpx.AsyncClient agent = Agent( 'openai:gpt-4o', deps_type=MyDeps, ) @agent.system_prompt async def get_system_prompt(ctx: RunContext[MyDeps]) -> str: response = await ctx.deps.http_client.get('https://example.com') response.raise_for_status() return f'Prompt: {response.text}' @agent.tool # (1)! async def get_joke_material(ctx: RunContext[MyDeps], subject: str) -> str: response = await ctx.deps.http_client.get( 'https://example.com#jokes', params={'subject': subject}, headers={'Authorization': f'Bearer {ctx.deps.api_key}'}, ) response.raise_for_status() return response.text @agent.result_validator # (2)! async def validate_result(ctx: RunContext[MyDeps], final_response: str) -> str: response = await ctx.deps.http_client.post( 'https://example.com#validate', headers={'Authorization': f'Bearer {ctx.deps.api_key}'}, params={'query': final_response}, ) if response.status_code == 400: raise ModelRetry(f'invalid response: {response.text}') response.raise_for_status() return final_response async def main(): async with httpx.AsyncClient() as client: deps = MyDeps('foobar', client) result = await agent.run('Tell me a joke.', deps=deps) print(result.data) #> Did you hear about the toothpaste scandal? They called it Colgate.` 1. To pass `RunContext` to a tool, use the [`tool`](../api/agent/#pydantic_ai.agent.Agent.tool) decorator. 2. `RunContext` may optionally be passed to a [`result_validator`](../api/agent/#pydantic_ai.agent.Agent.result_validator) function as the first argument. _(This example is complete, it can be run "as is" — you'll need to add `asyncio.run(main())` to run `main`)_ Overriding Dependencies ----------------------- When testing agents, it's useful to be able to customise dependencies. While this can sometimes be done by calling the agent directly within unit tests, we can also override dependencies while calling application code which in turn calls the agent. This is done via the [`override`](../api/agent/#pydantic_ai.agent.Agent.override) method on the agent. joke\_app.py `from dataclasses import dataclass import httpx from pydantic_ai import Agent, RunContext @dataclass class MyDeps: api_key: str http_client: httpx.AsyncClient async def system_prompt_factory(self) -> str: # (1)! response = await self.http_client.get('https://example.com') response.raise_for_status() return f'Prompt: {response.text}' joke_agent = Agent('openai:gpt-4o', deps_type=MyDeps) @joke_agent.system_prompt async def get_system_prompt(ctx: RunContext[MyDeps]) -> str: return await ctx.deps.system_prompt_factory() # (2)! async def application_code(prompt: str) -> str: # (3)! ... ... # now deep within application code we call our agent async with httpx.AsyncClient() as client: app_deps = MyDeps('foobar', client) result = await joke_agent.run(prompt, deps=app_deps) # (4)! return result.data` 1. Define a method on the dependency to make the system prompt easier to customise. 2. Call the system prompt factory from within the system prompt function. 3. Application code that calls the agent, in a real application this might be an API endpoint. 4. Call the agent from within the application code, in a real application this call might be deep within a call stack. Note `app_deps` here will NOT be used when deps are overridden. _(This example is complete, it can be run "as is")_ test\_joke\_app.py `from joke_app import MyDeps, application_code, joke_agent class TestMyDeps(MyDeps): # (1)! async def system_prompt_factory(self) -> str: return 'test prompt' async def test_application_code(): test_deps = TestMyDeps('test_key', None) # (2)! with joke_agent.override(deps=test_deps): # (3)! joke = await application_code('Tell me a joke.') # (4)! assert joke.startswith('Did you hear about the toothpaste scandal?')` 1. Define a subclass of `MyDeps` in tests to customise the system prompt factory. 2. Create an instance of the test dependency, we don't need to pass an `http_client` here as it's not used. 3. Override the dependencies of the agent for the duration of the `with` block, `test_deps` will be used when the agent is run. 4. Now we can safely call our application code, the agent will use the overridden dependencies. Examples -------- The following examples demonstrate how to use dependencies in PydanticAI: * [Weather Agent](../examples/weather-agent/) * [SQL Generation](../examples/sql-gen/) * [RAG](../examples/rag/) --- # Agents - PydanticAI Agents ====== Introduction ------------ Agents are PydanticAI's primary interface for interacting with LLMs. In some use cases a single Agent will control an entire application or component, but multiple agents can also interact to embody more complex workflows. The [`Agent`](../api/agent/#pydantic_ai.agent.Agent) class has full API documentation, but conceptually you can think of an agent as a container for: | **Component** | **Description** | | --- | --- | | [System prompt(s)](#system-prompts) | A set of instructions for the LLM written by the developer. | | [Function tool(s)](../tools/) | Functions that the LLM may call to get information while generating a response. | | [Structured result type](../results/) | The structured datatype the LLM must return at the end of a run, if specified. | | [Dependency type constraint](../dependencies/) | System prompt functions, tools, and result validators may all use dependencies when they're run. | | [LLM model](../api/models/base/) | Optional default LLM model associated with the agent. Can also be specified when running the agent. | | [Model Settings](#additional-configuration) | Optional default model settings to help fine tune requests. Can also be specified when running the agent. | In typing terms, agents are generic in their dependency and result types, e.g., an agent which required dependencies of type `Foobar` and returned results of type `list[str]` would have type `Agent[Foobar, list[str]]`. In practice, you shouldn't need to care about this, it should just mean your IDE can tell you when you have the right type, and if you choose to use [static type checking](#static-type-checking) it should work well with PydanticAI. Here's a toy example of an agent that simulates a roulette wheel: roulette\_wheel.py ``from pydantic_ai import Agent, RunContext roulette_agent = Agent( # (1)! 'openai:gpt-4o', deps_type=int, result_type=bool, system_prompt=( 'Use the `roulette_wheel` function to see if the ' 'customer has won based on the number they provide.' ), ) @roulette_agent.tool async def roulette_wheel(ctx: RunContext[int], square: int) -> str: # (2)! """check if the square is a winner""" return 'winner' if square == ctx.deps else 'loser' # Run the agent success_number = 18 # (3)! result = roulette_agent.run_sync('Put my money on square eighteen', deps=success_number) print(result.data) # (4)! #> True result = roulette_agent.run_sync('I bet five is the winner', deps=success_number) print(result.data) #> False`` 1. Create an agent, which expects an integer dependency and returns a boolean result. This agent will have type `Agent[int, bool]`. 2. Define a tool that checks if the square is a winner. Here [`RunContext`](../api/tools/#pydantic_ai.tools.RunContext) is parameterized with the dependency type `int`; if you got the dependency type wrong you'd get a typing error. 3. In reality, you might want to use a random number here e.g. `random.randint(0, 36)`. 4. `result.data` will be a boolean indicating if the square is a winner. Pydantic performs the result validation, it'll be typed as a `bool` since its type is derived from the `result_type` generic parameter of the agent. Agents are designed for reuse, like FastAPI Apps Agents are intended to be instantiated once (frequently as module globals) and reused throughout your application, similar to a small [FastAPI](https://fastapi.tiangolo.com/reference/fastapi/#fastapi.FastAPI) app or an [APIRouter](https://fastapi.tiangolo.com/reference/apirouter/#fastapi.APIRouter) . Running Agents -------------- There are four ways to run an agent: 1. [`agent.run()`](../api/agent/#pydantic_ai.agent.Agent.run) — a coroutine which returns a [`RunResult`](../api/agent/#pydantic_ai.agent.AgentRunResult) containing a completed response. 2. [`agent.run_sync()`](../api/agent/#pydantic_ai.agent.Agent.run_sync) — a plain, synchronous function which returns a [`RunResult`](../api/agent/#pydantic_ai.agent.AgentRunResult) containing a completed response (internally, this just calls `loop.run_until_complete(self.run())`). 3. [`agent.run_stream()`](../api/agent/#pydantic_ai.agent.Agent.run_stream) — a coroutine which returns a [`StreamedRunResult`](../api/result/#pydantic_ai.result.StreamedRunResult) , which contains methods to stream a response as an async iterable. 4. [`agent.iter()`](../api/agent/#pydantic_ai.agent.Agent.iter) — a context manager which returns an [`AgentRun`](../api/agent/#pydantic_ai.agent.AgentRun) , an async-iterable over the nodes of the agent's underlying [`Graph`](../api/pydantic_graph/graph/#pydantic_graph.graph.Graph) . Here's a simple example demonstrating the first three: run\_agent.py `from pydantic_ai import Agent agent = Agent('openai:gpt-4o') result_sync = agent.run_sync('What is the capital of Italy?') print(result_sync.data) #> Rome async def main(): result = await agent.run('What is the capital of France?') print(result.data) #> Paris async with agent.run_stream('What is the capital of the UK?') as response: print(await response.get_data()) #> London` _(This example is complete, it can be run "as is" — you'll need to add `asyncio.run(main())` to run `main`)_ You can also pass messages from previous runs to continue a conversation or provide context, as described in [Messages and Chat History](../message-history/) . ### Iterating Over an Agent's Graph Under the hood, each `Agent` in PydanticAI uses **pydantic-graph** to manage its execution flow. **pydantic-graph** is a generic, type-centric library for building and running finite state machines in Python. It doesn't actually depend on PydanticAI — you can use it standalone for workflows that have nothing to do with GenAI — but PydanticAI makes use of it to orchestrate the handling of model requests and model responses in an agent's run. In many scenarios, you don't need to worry about pydantic-graph at all; calling `agent.run(...)` simply traverses the underlying graph from start to finish. However, if you need deeper insight or control — for example to capture each tool invocation, or to inject your own logic at specific stages — PydanticAI exposes the lower-level iteration process via [`Agent.iter`](../api/agent/#pydantic_ai.agent.Agent.iter) . This method returns an [`AgentRun`](../api/agent/#pydantic_ai.agent.AgentRun) , which you can async-iterate over, or manually drive node-by-node via the [`next`](../api/agent/#pydantic_ai.agent.AgentRun.next) method. Once the agent's graph returns an [`End`](../api/pydantic_graph/nodes/#pydantic_graph.nodes.End) , you have the final result along with a detailed history of all steps. #### `async for` iteration Here's an example of using `async for` with `iter` to record each node the agent executes: agent\_iter\_async\_for.py `from pydantic_ai import Agent agent = Agent('openai:gpt-4o') async def main(): nodes = [] # Begin an AgentRun, which is an async-iterable over the nodes of the agent's graph async with agent.iter('What is the capital of France?') as agent_run: async for node in agent_run: # Each node represents a step in the agent's execution nodes.append(node) print(nodes) """ [ ModelRequestNode( request=ModelRequest( parts=[ UserPromptPart( content='What is the capital of France?', timestamp=datetime.datetime(...), part_kind='user-prompt', ) ], kind='request', ) ), CallToolsNode( model_response=ModelResponse( parts=[TextPart(content='Paris', part_kind='text')], model_name='gpt-4o', timestamp=datetime.datetime(...), kind='response', ) ), End(data=FinalResult(data='Paris', tool_name=None, tool_call_id=None)), ] """ print(agent_run.result.data) #> Paris` * The `AgentRun` is an async iterator that yields each node (`BaseNode` or `End`) in the flow. * The run ends when an `End` node is returned. #### Using `.next(...)` manually You can also drive the iteration manually by passing the node you want to run next to the `AgentRun.next(...)` method. This allows you to inspect or modify the node before it executes or skip nodes based on your own logic, and to catch errors in `next()` more easily: agent\_iter\_next.py `from pydantic_ai import Agent from pydantic_graph import End agent = Agent('openai:gpt-4o') async def main(): async with agent.iter('What is the capital of France?') as agent_run: node = agent_run.next_node # (1)! all_nodes = [node] # Drive the iteration manually: while not isinstance(node, End): # (2)! node = await agent_run.next(node) # (3)! all_nodes.append(node) # (4)! print(all_nodes) """ [ UserPromptNode( user_prompt='What is the capital of France?', system_prompts=(), system_prompt_functions=[], system_prompt_dynamic_functions={}, ), ModelRequestNode( request=ModelRequest( parts=[ UserPromptPart( content='What is the capital of France?', timestamp=datetime.datetime(...), part_kind='user-prompt', ) ], kind='request', ) ), CallToolsNode( model_response=ModelResponse( parts=[TextPart(content='Paris', part_kind='text')], model_name='gpt-4o', timestamp=datetime.datetime(...), kind='response', ) ), End(data=FinalResult(data='Paris', tool_name=None, tool_call_id=None)), ] """` 1. We start by grabbing the first node that will be run in the agent's graph. 2. The agent run is finished once an `End` node has been produced; instances of `End` cannot be passed to `next`. 3. When you call `await agent_run.next(node)`, it executes that node in the agent's graph, updates the run's history, and returns the _next_ node to run. 4. You could also inspect or mutate the new `node` here as needed. #### Accessing usage and the final result You can retrieve usage statistics (tokens, requests, etc.) at any time from the [`AgentRun`](../api/agent/#pydantic_ai.agent.AgentRun) object via `agent_run.usage()`. This method returns a [`Usage`](../api/usage/#pydantic_ai.usage.Usage) object containing the usage data. Once the run finishes, `agent_run.final_result` becomes a [`AgentRunResult`](../api/agent/#pydantic_ai.agent.AgentRunResult) object containing the final output (and related metadata). * * * ### Streaming Here is an example of streaming an agent run in combination with `async for` iteration: streaming.py `import asyncio from dataclasses import dataclass from datetime import date from pydantic_ai import Agent from pydantic_ai.messages import ( FinalResultEvent, FunctionToolCallEvent, FunctionToolResultEvent, PartDeltaEvent, PartStartEvent, TextPartDelta, ToolCallPartDelta, ) from pydantic_ai.tools import RunContext @dataclass class WeatherService: async def get_forecast(self, location: str, forecast_date: date) -> str: # In real code: call weather API, DB queries, etc. return f'The forecast in {location} on {forecast_date} is 24°C and sunny.' async def get_historic_weather(self, location: str, forecast_date: date) -> str: # In real code: call a historical weather API or DB return ( f'The weather in {location} on {forecast_date} was 18°C and partly cloudy.' ) weather_agent = Agent[WeatherService, str]( 'openai:gpt-4o', deps_type=WeatherService, result_type=str, # We'll produce a final answer as plain text system_prompt='Providing a weather forecast at the locations the user provides.', ) @weather_agent.tool async def weather_forecast( ctx: RunContext[WeatherService], location: str, forecast_date: date, ) -> str: if forecast_date >= date.today(): return await ctx.deps.get_forecast(location, forecast_date) else: return await ctx.deps.get_historic_weather(location, forecast_date) output_messages: list[str] = [] async def main(): user_prompt = 'What will the weather be like in Paris on Tuesday?' # Begin a node-by-node, streaming iteration async with weather_agent.iter(user_prompt, deps=WeatherService()) as run: async for node in run: if Agent.is_user_prompt_node(node): # A user prompt node => The user has provided input output_messages.append(f'=== UserPromptNode: {node.user_prompt} ===') elif Agent.is_model_request_node(node): # A model request node => We can stream tokens from the model's request output_messages.append( '=== ModelRequestNode: streaming partial request tokens ===' ) async with node.stream(run.ctx) as request_stream: async for event in request_stream: if isinstance(event, PartStartEvent): output_messages.append( f'[Request] Starting part {event.index}: {event.part!r}' ) elif isinstance(event, PartDeltaEvent): if isinstance(event.delta, TextPartDelta): output_messages.append( f'[Request] Part {event.index} text delta: {event.delta.content_delta!r}' ) elif isinstance(event.delta, ToolCallPartDelta): output_messages.append( f'[Request] Part {event.index} args_delta={event.delta.args_delta}' ) elif isinstance(event, FinalResultEvent): output_messages.append( f'[Result] The model produced a final result (tool_name={event.tool_name})' ) elif Agent.is_call_tools_node(node): # A handle-response node => The model returned some data, potentially calls a tool output_messages.append( '=== CallToolsNode: streaming partial response & tool usage ===' ) async with node.stream(run.ctx) as handle_stream: async for event in handle_stream: if isinstance(event, FunctionToolCallEvent): output_messages.append( f'[Tools] The LLM calls tool={event.part.tool_name!r} with args={event.part.args} (tool_call_id={event.part.tool_call_id!r})' ) elif isinstance(event, FunctionToolResultEvent): output_messages.append( f'[Tools] Tool call {event.tool_call_id!r} returned => {event.result.content}' ) elif Agent.is_end_node(node): assert run.result.data == node.data.data # Once an End node is reached, the agent run is complete output_messages.append(f'=== Final Agent Output: {run.result.data} ===') if __name__ == '__main__': asyncio.run(main()) print(output_messages) """ [ '=== ModelRequestNode: streaming partial request tokens ===', '[Request] Starting part 0: ToolCallPart(tool_name=\'weather_forecast\', args=\'{"location":"Pa\', tool_call_id=\'0001\', part_kind=\'tool-call\')', '[Request] Part 0 args_delta=ris","forecast_', '[Request] Part 0 args_delta=date":"2030-01-', '[Request] Part 0 args_delta=01"}', '=== CallToolsNode: streaming partial response & tool usage ===', '[Tools] The LLM calls tool=\'weather_forecast\' with args={"location":"Paris","forecast_date":"2030-01-01"} (tool_call_id=\'0001\')', "[Tools] Tool call '0001' returned => The forecast in Paris on 2030-01-01 is 24°C and sunny.", '=== ModelRequestNode: streaming partial request tokens ===', "[Request] Starting part 0: TextPart(content='It will be ', part_kind='text')", '[Result] The model produced a final result (tool_name=None)', "[Request] Part 0 text delta: 'warm and sunny '", "[Request] Part 0 text delta: 'in Paris on '", "[Request] Part 0 text delta: 'Tuesday.'", '=== CallToolsNode: streaming partial response & tool usage ===', '=== Final Agent Output: It will be warm and sunny in Paris on Tuesday. ===', ] """` * * * ### Additional Configuration #### Usage Limits PydanticAI offers a [`UsageLimits`](../api/usage/#pydantic_ai.usage.UsageLimits) structure to help you limit your usage (tokens and/or requests) on model runs. You can apply these settings by passing the `usage_limits` argument to the `run{_sync,_stream}` functions. Consider the following example, where we limit the number of response tokens: `from pydantic_ai import Agent from pydantic_ai.exceptions import UsageLimitExceeded from pydantic_ai.usage import UsageLimits agent = Agent('anthropic:claude-3-5-sonnet-latest') result_sync = agent.run_sync( 'What is the capital of Italy? Answer with just the city.', usage_limits=UsageLimits(response_tokens_limit=10), ) print(result_sync.data) #> Rome print(result_sync.usage()) """ Usage(requests=1, request_tokens=62, response_tokens=1, total_tokens=63, details=None) """ try: result_sync = agent.run_sync( 'What is the capital of Italy? Answer with a paragraph.', usage_limits=UsageLimits(response_tokens_limit=10), ) except UsageLimitExceeded as e: print(e) #> Exceeded the response_tokens_limit of 10 (response_tokens=32)` Restricting the number of requests can be useful in preventing infinite loops or excessive tool calling: ``from typing_extensions import TypedDict from pydantic_ai import Agent, ModelRetry from pydantic_ai.exceptions import UsageLimitExceeded from pydantic_ai.usage import UsageLimits class NeverResultType(TypedDict): """ Never ever coerce data to this type. """ never_use_this: str agent = Agent( 'anthropic:claude-3-5-sonnet-latest', retries=3, result_type=NeverResultType, system_prompt='Any time you get a response, call the `infinite_retry_tool` to produce another response.', ) @agent.tool_plain(retries=5) # (1)! def infinite_retry_tool() -> int: raise ModelRetry('Please try again.') try: result_sync = agent.run_sync( 'Begin infinite retry loop!', usage_limits=UsageLimits(request_limit=3) # (2)! ) except UsageLimitExceeded as e: print(e) #> The next request would exceed the request_limit of 3`` 1. This tool has the ability to retry 5 times before erroring, simulating a tool that might get stuck in a loop. 2. This run will error after 3 requests, preventing the infinite tool calling. Note This is especially relevant if you've registered many tools. The `request_limit` can be used to prevent the model from calling them in a loop too many times. #### Model (Run) Settings PydanticAI offers a [`settings.ModelSettings`](../api/settings/#pydantic_ai.settings.ModelSettings) structure to help you fine tune your requests. This structure allows you to configure common parameters that influence the model's behavior, such as `temperature`, `max_tokens`, `timeout`, and more. There are two ways to apply these settings: 1. Passing to `run{_sync,_stream}` functions via the `model_settings` argument. This allows for fine-tuning on a per-request basis. 2. Setting during [`Agent`](../api/agent/#pydantic_ai.agent.Agent) initialization via the `model_settings` argument. These settings will be applied by default to all subsequent run calls using said agent. However, `model_settings` provided during a specific run call will override the agent's default settings. For example, if you'd like to set the `temperature` setting to `0.0` to ensure less random behavior, you can do the following: `from pydantic_ai import Agent agent = Agent('openai:gpt-4o') result_sync = agent.run_sync( 'What is the capital of Italy?', model_settings={'temperature': 0.0} ) print(result_sync.data) #> Rome` ### Model specific settings If you wish to further customize model behavior, you can use a subclass of [`ModelSettings`](../api/settings/#pydantic_ai.settings.ModelSettings) , like [`GeminiModelSettings`](../api/models/gemini/#pydantic_ai.models.gemini.GeminiModelSettings) , associated with your model of choice. For example: `from pydantic_ai import Agent, UnexpectedModelBehavior from pydantic_ai.models.gemini import GeminiModelSettings agent = Agent('google-gla:gemini-1.5-flash') try: result = agent.run_sync( 'Write a list of 5 very rude things that I might say to the universe after stubbing my toe in the dark:', model_settings=GeminiModelSettings( temperature=0.0, # general model settings can also be specified gemini_safety_settings=[ { 'category': 'HARM_CATEGORY_HARASSMENT', 'threshold': 'BLOCK_LOW_AND_ABOVE', }, { 'category': 'HARM_CATEGORY_HATE_SPEECH', 'threshold': 'BLOCK_LOW_AND_ABOVE', }, ], ), ) except UnexpectedModelBehavior as e: print(e) # (1)! """ Safety settings triggered, body: """` 1. This error is raised because the safety thresholds were exceeded. Generally, `result` would contain a normal `ModelResponse`. Runs vs. Conversations ---------------------- An agent **run** might represent an entire conversation — there's no limit to how many messages can be exchanged in a single run. However, a **conversation** might also be composed of multiple runs, especially if you need to maintain state between separate interactions or API calls. Here's an example of a conversation comprised of multiple runs: conversation\_example.py `from pydantic_ai import Agent agent = Agent('openai:gpt-4o') # First run result1 = agent.run_sync('Who was Albert Einstein?') print(result1.data) #> Albert Einstein was a German-born theoretical physicist. # Second run, passing previous messages result2 = agent.run_sync( 'What was his most famous equation?', message_history=result1.new_messages(), # (1)! ) print(result2.data) #> Albert Einstein's most famous equation is (E = mc^2).` 1. Continue the conversation; without `message_history` the model would not know who "his" was referring to. _(This example is complete, it can be run "as is")_ Type safe by design ------------------- PydanticAI is designed to work well with static type checkers, like mypy and pyright. Typing is (somewhat) optional PydanticAI is designed to make type checking as useful as possible for you if you choose to use it, but you don't have to use types everywhere all the time. That said, because PydanticAI uses Pydantic, and Pydantic uses type hints as the definition for schema and validation, some types (specifically type hints on parameters to tools, and the `result_type` arguments to [`Agent`](../api/agent/#pydantic_ai.agent.Agent) ) are used at runtime. We (the library developers) have messed up if type hints are confusing you more than helping you, if you find this, please create an [issue](https://github.com/pydantic/pydantic-ai/issues) explaining what's annoying you! In particular, agents are generic in both the type of their dependencies and the type of results they return, so you can use the type hints to ensure you're using the right types. Consider the following script with type mistakes: type\_mistakes.py `from dataclasses import dataclass from pydantic_ai import Agent, RunContext @dataclass class User: name: str agent = Agent( 'test', deps_type=User, # (1)! result_type=bool, ) @agent.system_prompt def add_user_name(ctx: RunContext[str]) -> str: # (2)! return f"The user's name is {ctx.deps}." def foobar(x: bytes) -> None: pass result = agent.run_sync('Does their name start with "A"?', deps=User('Anne')) foobar(result.data) # (3)!` 1. The agent is defined as expecting an instance of `User` as `deps`. 2. But here `add_user_name` is defined as taking a `str` as the dependency, not a `User`. 3. Since the agent is defined as returning a `bool`, this will raise a type error since `foobar` expects `bytes`. Running `mypy` on this will give the following output: `➤ uv run mypy type_mistakes.py type_mistakes.py:18: error: Argument 1 to "system_prompt" of "Agent" has incompatible type "Callable[[RunContext[str]], str]"; expected "Callable[[RunContext[User]], str]" [arg-type] type_mistakes.py:28: error: Argument 1 to "foobar" has incompatible type "bool"; expected "bytes" [arg-type] Found 2 errors in 1 file (checked 1 source file)` Running `pyright` would identify the same issues. System Prompts -------------- System prompts might seem simple at first glance since they're just strings (or sequences of strings that are concatenated), but crafting the right system prompt is key to getting the model to behave as you want. Generally, system prompts fall into two categories: 1. **Static system prompts**: These are known when writing the code and can be defined via the `system_prompt` parameter of the [`Agent` constructor](../api/agent/#pydantic_ai.agent.Agent.__init__) . 2. **Dynamic system prompts**: These depend in some way on context that isn't known until runtime, and should be defined via functions decorated with [`@agent.system_prompt`](../api/agent/#pydantic_ai.agent.Agent.system_prompt) . You can add both to a single agent; they're appended in the order they're defined at runtime. Here's an example using both types of system prompts: system\_prompts.py `from datetime import date from pydantic_ai import Agent, RunContext agent = Agent( 'openai:gpt-4o', deps_type=str, # (1)! system_prompt="Use the customer's name while replying to them.", # (2)! ) @agent.system_prompt # (3)! def add_the_users_name(ctx: RunContext[str]) -> str: return f"The user's name is {ctx.deps}." @agent.system_prompt def add_the_date() -> str: # (4)! return f'The date is {date.today()}.' result = agent.run_sync('What is the date?', deps='Frank') print(result.data) #> Hello Frank, the date today is 2032-01-02.` 1. The agent expects a string dependency. 2. Static system prompt defined at agent creation time. 3. Dynamic system prompt defined via a decorator with [`RunContext`](../api/tools/#pydantic_ai.tools.RunContext) , this is called just after `run_sync`, not when the agent is created, so can benefit from runtime information like the dependencies used on that run. 4. Another dynamic system prompt, system prompts don't have to have the `RunContext` parameter. _(This example is complete, it can be run "as is")_ Reflection and self-correction ------------------------------ Validation errors from both function tool parameter validation and [structured result validation](../results/#structured-result-validation) can be passed back to the model with a request to retry. You can also raise [`ModelRetry`](../api/exceptions/#pydantic_ai.exceptions.ModelRetry) from within a [tool](../tools/) or [result validator function](../results/#result-validators-functions) to tell the model it should retry generating a response. * The default retry count is **1** but can be altered for the [entire agent](../api/agent/#pydantic_ai.agent.Agent.__init__) , a [specific tool](../api/agent/#pydantic_ai.agent.Agent.tool) , or a [result validator](../api/agent/#pydantic_ai.agent.Agent.__init__) . * You can access the current retry count from within a tool or result validator via [`ctx.retry`](../api/tools/#pydantic_ai.tools.RunContext) . Here's an example: tool\_retry.py `from pydantic import BaseModel from pydantic_ai import Agent, RunContext, ModelRetry from fake_database import DatabaseConn class ChatResult(BaseModel): user_id: int message: str agent = Agent( 'openai:gpt-4o', deps_type=DatabaseConn, result_type=ChatResult, ) @agent.tool(retries=2) def get_user_by_name(ctx: RunContext[DatabaseConn], name: str) -> int: """Get a user's ID from their full name.""" print(name) #> John #> John Doe user_id = ctx.deps.users.get(name=name) if user_id is None: raise ModelRetry( f'No user found with name {name!r}, remember to provide their full name' ) return user_id result = agent.run_sync( 'Send a message to John Doe asking for coffee next week', deps=DatabaseConn() ) print(result.data) """ user_id=123 message='Hello John, would you be free for coffee sometime next week? Let me know what works for you!' """` Model errors ------------ If models behave unexpectedly (e.g., the retry limit is exceeded, or their API returns `503`), agent runs will raise [`UnexpectedModelBehavior`](../api/exceptions/#pydantic_ai.exceptions.UnexpectedModelBehavior) . In these cases, [`capture_run_messages`](../api/agent/#pydantic_ai.agent.capture_run_messages) can be used to access the messages exchanged during the run to help diagnose the issue. agent\_model\_errors.py `from pydantic_ai import Agent, ModelRetry, UnexpectedModelBehavior, capture_run_messages agent = Agent('openai:gpt-4o') @agent.tool_plain def calc_volume(size: int) -> int: # (1)! if size == 42: return size**3 else: raise ModelRetry('Please try again.') with capture_run_messages() as messages: # (2)! try: result = agent.run_sync('Please get me the volume of a box with size 6.') except UnexpectedModelBehavior as e: print('An error occurred:', e) #> An error occurred: Tool exceeded max retries count of 1 print('cause:', repr(e.__cause__)) #> cause: ModelRetry('Please try again.') print('messages:', messages) """ messages: [ ModelRequest( parts=[ UserPromptPart( content='Please get me the volume of a box with size 6.', timestamp=datetime.datetime(...), part_kind='user-prompt', ) ], kind='request', ), ModelResponse( parts=[ ToolCallPart( tool_name='calc_volume', args={'size': 6}, tool_call_id='pyd_ai_tool_call_id', part_kind='tool-call', ) ], model_name='gpt-4o', timestamp=datetime.datetime(...), kind='response', ), ModelRequest( parts=[ RetryPromptPart( content='Please try again.', tool_name='calc_volume', tool_call_id='pyd_ai_tool_call_id', timestamp=datetime.datetime(...), part_kind='retry-prompt', ) ], kind='request', ), ModelResponse( parts=[ ToolCallPart( tool_name='calc_volume', args={'size': 6}, tool_call_id='pyd_ai_tool_call_id', part_kind='tool-call', ) ], model_name='gpt-4o', timestamp=datetime.datetime(...), kind='response', ), ] """ else: print(result.data)` 1. Define a tool that will raise `ModelRetry` repeatedly in this case. 2. [`capture_run_messages`](../api/agent/#pydantic_ai.agent.capture_run_messages) is used to capture the messages exchanged during the run. _(This example is complete, it can be run "as is")_ Note If you call [`run`](../api/agent/#pydantic_ai.agent.Agent.run) , [`run_sync`](../api/agent/#pydantic_ai.agent.Agent.run_sync) , or [`run_stream`](../api/agent/#pydantic_ai.agent.Agent.run_stream) more than once within a single `capture_run_messages` context, `messages` will represent the messages exchanged during the first call only. --- # Function Tools - PydanticAI Function Tools ============== Function tools provide a mechanism for models to retrieve extra information to help them generate a response. They're useful when it is impractical or impossible to put all the context an agent might need into the system prompt, or when you want to make agents' behavior more deterministic or reliable by deferring some of the logic required to generate a response to another (not necessarily AI-powered) tool. Function tools vs. RAG Function tools are basically the "R" of RAG (Retrieval-Augmented Generation) — they augment what the model can do by letting it request extra information. The main semantic difference between PydanticAI Tools and RAG is RAG is synonymous with vector search, while PydanticAI tools are more general-purpose. (Note: we may add support for vector search functionality in the future, particularly an API for generating embeddings. See [#58](https://github.com/pydantic/pydantic-ai/issues/58) ) There are a number of ways to register tools with an agent: * via the [`@agent.tool`](../api/agent/#pydantic_ai.agent.Agent.tool) decorator — for tools that need access to the agent [context](../api/tools/#pydantic_ai.tools.RunContext) * via the [`@agent.tool_plain`](../api/agent/#pydantic_ai.agent.Agent.tool_plain) decorator — for tools that do not need access to the agent [context](../api/tools/#pydantic_ai.tools.RunContext) * via the [`tools`](../api/agent/#pydantic_ai.agent.Agent.__init__) keyword argument to `Agent` which can take either plain functions, or instances of [`Tool`](../api/tools/#pydantic_ai.tools.Tool) `@agent.tool` is considered the default decorator since in the majority of cases tools will need access to the agent context. Here's an example using both: dice\_game.py `import random from pydantic_ai import Agent, RunContext agent = Agent( 'google-gla:gemini-1.5-flash', # (1)! deps_type=str, # (2)! system_prompt=( "You're a dice game, you should roll the die and see if the number " "you get back matches the user's guess. If so, tell them they're a winner. " "Use the player's name in the response." ), ) @agent.tool_plain # (3)! def roll_die() -> str: """Roll a six-sided die and return the result.""" return str(random.randint(1, 6)) @agent.tool # (4)! def get_player_name(ctx: RunContext[str]) -> str: """Get the player's name.""" return ctx.deps dice_result = agent.run_sync('My guess is 4', deps='Anne') # (5)! print(dice_result.data) #> Congratulations Anne, you guessed correctly! You're a winner!` 1. This is a pretty simple task, so we can use the fast and cheap Gemini flash model. 2. We pass the user's name as the dependency, to keep things simple we use just the name as a string as the dependency. 3. This tool doesn't need any context, it just returns a random number. You could probably use a dynamic system prompt in this case. 4. This tool needs the player's name, so it uses `RunContext` to access dependencies which are just the player's name in this case. 5. Run the agent, passing the player's name as the dependency. _(This example is complete, it can be run "as is")_ Let's print the messages from that game to see what happened: dice\_game\_messages.py `from dice_game import dice_result print(dice_result.all_messages()) """ [ ModelRequest( parts=[ SystemPromptPart( content="You're a dice game, you should roll the die and see if the number you get back matches the user's guess. If so, tell them they're a winner. Use the player's name in the response.", timestamp=datetime.datetime(...), dynamic_ref=None, part_kind='system-prompt', ), UserPromptPart( content='My guess is 4', timestamp=datetime.datetime(...), part_kind='user-prompt', ), ], kind='request', ), ModelResponse( parts=[ ToolCallPart( tool_name='roll_die', args={}, tool_call_id='pyd_ai_tool_call_id', part_kind='tool-call', ) ], model_name='gemini-1.5-flash', timestamp=datetime.datetime(...), kind='response', ), ModelRequest( parts=[ ToolReturnPart( tool_name='roll_die', content='4', tool_call_id='pyd_ai_tool_call_id', timestamp=datetime.datetime(...), part_kind='tool-return', ) ], kind='request', ), ModelResponse( parts=[ ToolCallPart( tool_name='get_player_name', args={}, tool_call_id='pyd_ai_tool_call_id', part_kind='tool-call', ) ], model_name='gemini-1.5-flash', timestamp=datetime.datetime(...), kind='response', ), ModelRequest( parts=[ ToolReturnPart( tool_name='get_player_name', content='Anne', tool_call_id='pyd_ai_tool_call_id', timestamp=datetime.datetime(...), part_kind='tool-return', ) ], kind='request', ), ModelResponse( parts=[ TextPart( content="Congratulations Anne, you guessed correctly! You're a winner!", part_kind='text', ) ], model_name='gemini-1.5-flash', timestamp=datetime.datetime(...), kind='response', ), ] """` We can represent this with a diagram: sequenceDiagram participant Agent participant LLM Note over Agent: Send prompts Agent ->> LLM: System: "You're a dice game..."
User: "My guess is 4" activate LLM Note over LLM: LLM decides to use
a tool LLM ->> Agent: Call tool
roll_die() deactivate LLM activate Agent Note over Agent: Rolls a six-sided die Agent -->> LLM: ToolReturn
"4" deactivate Agent activate LLM Note over LLM: LLM decides to use
another tool LLM ->> Agent: Call tool
get_player_name() deactivate LLM activate Agent Note over Agent: Retrieves player name Agent -->> LLM: ToolReturn
"Anne" deactivate Agent activate LLM Note over LLM: LLM constructs final response LLM ->> Agent: ModelResponse
"Congratulations Anne, ..." deactivate LLM Note over Agent: Game session complete Registering Function Tools via kwarg ------------------------------------ As well as using the decorators, we can register tools via the `tools` argument to the [`Agent` constructor](../api/agent/#pydantic_ai.agent.Agent.__init__) . This is useful when you want to reuse tools, and can also give more fine-grained control over the tools. dice\_game\_tool\_kwarg.py `import random from pydantic_ai import Agent, RunContext, Tool system_prompt = """\ You're a dice game, you should roll the die and see if the number you get back matches the user's guess. If so, tell them they're a winner. Use the player's name in the response. """ def roll_die() -> str: """Roll a six-sided die and return the result.""" return str(random.randint(1, 6)) def get_player_name(ctx: RunContext[str]) -> str: """Get the player's name.""" return ctx.deps agent_a = Agent( 'google-gla:gemini-1.5-flash', deps_type=str, tools=[roll_die, get_player_name], # (1)! system_prompt=system_prompt, ) agent_b = Agent( 'google-gla:gemini-1.5-flash', deps_type=str, tools=[ # (2)! Tool(roll_die, takes_ctx=False), Tool(get_player_name, takes_ctx=True), ], system_prompt=system_prompt, ) dice_result = {} dice_result['a'] = agent_a.run_sync('My guess is 6', deps='Yashar') dice_result['b'] = agent_b.run_sync('My guess is 4', deps='Anne') print(dice_result['a'].data) #> Tough luck, Yashar, you rolled a 4. Better luck next time. print(dice_result['b'].data) #> Congratulations Anne, you guessed correctly! You're a winner!` 1. The simplest way to register tools via the `Agent` constructor is to pass a list of functions, the function signature is inspected to determine if the tool takes [`RunContext`](../api/tools/#pydantic_ai.tools.RunContext) . 2. `agent_a` and `agent_b` are identical — but we can use [`Tool`](../api/tools/#pydantic_ai.tools.Tool) to reuse tool definitions and give more fine-grained control over how tools are defined, e.g. setting their name or description, or using a custom [`prepare`](#tool-prepare) method. _(This example is complete, it can be run "as is")_ Function Tools vs. Structured Results ------------------------------------- As the name suggests, function tools use the model's "tools" or "functions" API to let the model know what is available to call. Tools or functions are also used to define the schema(s) for structured responses, thus a model might have access to many tools, some of which call function tools while others end the run and return a result. Function tools and schema ------------------------- Function parameters are extracted from the function signature, and all parameters except `RunContext` are used to build the schema for that tool call. Even better, PydanticAI extracts the docstring from functions and (thanks to [griffe](https://mkdocstrings.github.io/griffe/) ) extracts parameter descriptions from the docstring and adds them to the schema. [Griffe supports](https://mkdocstrings.github.io/griffe/reference/docstrings/#docstrings) extracting parameter descriptions from `google`, `numpy`, and `sphinx` style docstrings. PydanticAI will infer the format to use based on the docstring, but you can explicitly set it using [`docstring_format`](../api/tools/#pydantic_ai.tools.DocstringFormat) . You can also enforce parameter requirements by setting `require_parameter_descriptions=True`. This will raise a [`UserError`](../api/exceptions/#pydantic_ai.exceptions.UserError) if a parameter description is missing. To demonstrate a tool's schema, here we use [`FunctionModel`](../api/models/function/#pydantic_ai.models.function.FunctionModel) to print the schema a model would receive: tool\_schema.py `from pydantic_ai import Agent from pydantic_ai.messages import ModelMessage, ModelResponse, TextPart from pydantic_ai.models.function import AgentInfo, FunctionModel agent = Agent() @agent.tool_plain(docstring_format='google', require_parameter_descriptions=True) def foobar(a: int, b: str, c: dict[str, list[float]]) -> str: """Get me foobar. Args: a: apple pie b: banana cake c: carrot smoothie """ return f'{a} {b} {c}' def print_schema(messages: list[ModelMessage], info: AgentInfo) -> ModelResponse: tool = info.function_tools[0] print(tool.description) #> Get me foobar. print(tool.parameters_json_schema) """ { 'additionalProperties': False, 'properties': { 'a': {'description': 'apple pie', 'type': 'integer'}, 'b': {'description': 'banana cake', 'type': 'string'}, 'c': { 'additionalProperties': {'items': {'type': 'number'}, 'type': 'array'}, 'description': 'carrot smoothie', 'type': 'object', }, }, 'required': ['a', 'b', 'c'], 'type': 'object', } """ return ModelResponse(parts=[TextPart('foobar')]) agent.run_sync('hello', model=FunctionModel(print_schema))` _(This example is complete, it can be run "as is")_ The return type of tool can be anything which Pydantic can serialize to JSON as some models (e.g. Gemini) support semi-structured return values, some expect text (OpenAI) but seem to be just as good at extracting meaning from the data. If a Python object is returned and the model expects a string, the value will be serialized to JSON. If a tool has a single parameter that can be represented as an object in JSON schema (e.g. dataclass, TypedDict, pydantic model), the schema for the tool is simplified to be just that object. Here's an example where we use [`TestModel.last_model_request_parameters`](../api/models/test/#pydantic_ai.models.test.TestModel.last_model_request_parameters) to inspect the tool schema that would be passed to the model. single\_parameter\_tool.py `from pydantic import BaseModel from pydantic_ai import Agent from pydantic_ai.models.test import TestModel agent = Agent() class Foobar(BaseModel): """This is a Foobar""" x: int y: str z: float = 3.14 @agent.tool_plain def foobar(f: Foobar) -> str: return str(f) test_model = TestModel() result = agent.run_sync('hello', model=test_model) print(result.data) #> {"foobar":"x=0 y='a' z=3.14"} print(test_model.last_model_request_parameters.function_tools) """ [ ToolDefinition( name='foobar', description='This is a Foobar', parameters_json_schema={ 'properties': { 'x': {'type': 'integer'}, 'y': {'type': 'string'}, 'z': {'default': 3.14, 'type': 'number'}, }, 'required': ['x', 'y'], 'title': 'Foobar', 'type': 'object', }, outer_typed_dict_key=None, ) ] """` _(This example is complete, it can be run "as is")_ Dynamic Function tools ---------------------- Tools can optionally be defined with another function: `prepare`, which is called at each step of a run to customize the definition of the tool passed to the model, or omit the tool completely from that step. A `prepare` method can be registered via the `prepare` kwarg to any of the tool registration mechanisms: * [`@agent.tool`](../api/agent/#pydantic_ai.agent.Agent.tool) decorator * [`@agent.tool_plain`](../api/agent/#pydantic_ai.agent.Agent.tool_plain) decorator * [`Tool`](../api/tools/#pydantic_ai.tools.Tool) dataclass The `prepare` method, should be of type [`ToolPrepareFunc`](../api/tools/#pydantic_ai.tools.ToolPrepareFunc) , a function which takes [`RunContext`](../api/tools/#pydantic_ai.tools.RunContext) and a pre-built [`ToolDefinition`](../api/tools/#pydantic_ai.tools.ToolDefinition) , and should either return that `ToolDefinition` with or without modifying it, return a new `ToolDefinition`, or return `None` to indicate this tools should not be registered for that step. Here's a simple `prepare` method that only includes the tool if the value of the dependency is `42`. As with the previous example, we use [`TestModel`](../api/models/test/#pydantic_ai.models.test.TestModel) to demonstrate the behavior without calling a real model. tool\_only\_if\_42.py `from typing import Union from pydantic_ai import Agent, RunContext from pydantic_ai.tools import ToolDefinition agent = Agent('test') async def only_if_42( ctx: RunContext[int], tool_def: ToolDefinition ) -> Union[ToolDefinition, None]: if ctx.deps == 42: return tool_def @agent.tool(prepare=only_if_42) def hitchhiker(ctx: RunContext[int], answer: str) -> str: return f'{ctx.deps} {answer}' result = agent.run_sync('testing...', deps=41) print(result.data) #> success (no tool calls) result = agent.run_sync('testing...', deps=42) print(result.data) #> {"hitchhiker":"42 a"}` _(This example is complete, it can be run "as is")_ Here's a more complex example where we change the description of the `name` parameter to based on the value of `deps` For the sake of variation, we create this tool using the [`Tool`](../api/tools/#pydantic_ai.tools.Tool) dataclass. customize\_name.py `from __future__ import annotations from typing import Literal from pydantic_ai import Agent, RunContext from pydantic_ai.models.test import TestModel from pydantic_ai.tools import Tool, ToolDefinition def greet(name: str) -> str: return f'hello {name}' async def prepare_greet( ctx: RunContext[Literal['human', 'machine']], tool_def: ToolDefinition ) -> ToolDefinition | None: d = f'Name of the {ctx.deps} to greet.' tool_def.parameters_json_schema['properties']['name']['description'] = d return tool_def greet_tool = Tool(greet, prepare=prepare_greet) test_model = TestModel() agent = Agent(test_model, tools=[greet_tool], deps_type=Literal['human', 'machine']) result = agent.run_sync('testing...', deps='human') print(result.data) #> {"greet":"hello a"} print(test_model.last_model_request_parameters.function_tools) """ [ ToolDefinition( name='greet', description='', parameters_json_schema={ 'additionalProperties': False, 'properties': { 'name': {'type': 'string', 'description': 'Name of the human to greet.'} }, 'required': ['name'], 'type': 'object', }, outer_typed_dict_key=None, ) ] """` _(This example is complete, it can be run "as is")_ --- # Models - PydanticAI Models ====== PydanticAI is Model-agnostic and has built in support for the following model providers: * [OpenAI](#openai) * [Anthropic](#anthropic) * Gemini via two different APIs: [Generative Language API](#gemini) and [VertexAI API](#gemini-via-vertexai) * [Ollama](#ollama) * [Groq](#groq) * [Mistral](#mistral) * [Cohere](#cohere) * [Bedrock](#bedrock) See [OpenAI-compatible models](#openai-compatible-models) for more examples on how to use models such as [OpenRouter](#openrouter) , and [Grok (xAI)](#grok-xai) that support the OpenAI SDK. You can also [add support for other models](#implementing-custom-models) . PydanticAI also comes with [`TestModel`](../api/models/test/) and [`FunctionModel`](../api/models/function/) for testing and development. To use each model provider, you need to configure your local environment and make sure you have the right packages installed. Models, Interfaces, and Providers --------------------------------- PydanticAI uses a few key terms to describe how it interacts with different LLMs: * **Model**: This refers to the specific LLM model you want to handle your requests (e.g., `gpt-4o`, `claude-3-5-sonnet-latest`, `gemini-1.5-flash`). It's the "brain" that processes your prompts and generates responses. You specify the _Model_ as a parameter to the _Interface_. * **Interface**: This refers to a PydanticAI class used to make requests following a specific LLM API (generally by wrapping a vendor-provided SDK, like the `openai` python SDK). These classes implement a vendor-SDK-agnostic API, ensuring a single PydanticAI agent is portable to different LLM vendors without any other code changes just by swapping out the _Interface_ it uses. Currently, interface classes are named roughly in the format `Model`, for example, we have `OpenAIModel`, `AnthropicModel`, `GeminiModel`, etc. These `Model` classes will soon be renamed to `Interface` to reflect this terminology better. * **Provider**: This refers to _Interface_\-specific classes which handle the authentication and connections to an LLM vendor. Passing a non-default _Provider_ as a parameter to an _Interface_ is how you can ensure that your agent will make requests to a specific endpoint, or make use of a specific approach to authentication (e.g., you can use Vertex-specific auth with the `GeminiModel` by way of the `VertexProvider`). In particular, this is how you can make use of an AI gateway, or an LLM vendor that offers API compatibility with the vendor SDK used by an existing interface (such as `OpenAIModel`). In short, you select a _model_, PydanticAI uses the appropriate _interface_ class, and the _provider_ handles the connection and authentication to the underlying service. OpenAI ------ ### Install To use OpenAI models, you need to either install [`pydantic-ai`](../install/) , or install [`pydantic-ai-slim`](../install/#slim-install) with the `openai` optional group: pipuv `pip install "pydantic-ai-slim[openai]"` `uv add "pydantic-ai-slim[openai]"` ### Configuration To use [`OpenAIModel`](../api/models/openai/#pydantic_ai.models.openai.OpenAIModel) through their main API, go to [platform.openai.com](https://platform.openai.com/) and follow your nose until you find the place to generate an API key. ### Environment variable Once you have the API key, you can set it as an environment variable: `export OPENAI_API_KEY='your-api-key'` You can then use [`OpenAIModel`](../api/models/openai/#pydantic_ai.models.openai.OpenAIModel) by name: openai\_model\_by\_name.py `from pydantic_ai import Agent agent = Agent('openai:gpt-4o') ...` Or initialise the model directly with just the model name: openai\_model\_init.py `from pydantic_ai import Agent from pydantic_ai.models.openai import OpenAIModel model = OpenAIModel('gpt-4o') agent = Agent(model) ...` By default, the `OpenAIModel` uses the [`OpenAIProvider`](../api/providers/#pydantic_ai.providers.openai.OpenAIProvider.__init__) with the `base_url` set to `https://api.openai.com/v1`. ### `provider` argument You can provide a custom [`Provider`](../api/providers/#pydantic_ai.providers.Provider) via the [`provider` argument](../api/models/openai/#pydantic_ai.models.openai.OpenAIModel.__init__) : openai\_model\_provider.py `from pydantic_ai import Agent from pydantic_ai.models.openai import OpenAIModel from pydantic_ai.providers.openai import OpenAIProvider model = OpenAIModel('gpt-4o', provider=OpenAIProvider(api_key='your-api-key')) agent = Agent(model) ...` ### Custom OpenAI Client `OpenAIProvider` also accepts a custom `AsyncOpenAI` client via the [`openai_client` parameter](../api/providers/#pydantic_ai.providers.openai.OpenAIProvider.__init__) , so you can customise the `organization`, `project`, `base_url` etc. as defined in the [OpenAI API docs](https://platform.openai.com/docs/api-reference) . You could also use the [`AsyncAzureOpenAI`](https://learn.microsoft.com/en-us/azure/ai-services/openai/how-to/switching-endpoints) client to use the Azure OpenAI API. openai\_azure.py `from openai import AsyncAzureOpenAI from pydantic_ai import Agent from pydantic_ai.models.openai import OpenAIModel from pydantic_ai.providers.openai import OpenAIProvider client = AsyncAzureOpenAI( azure_endpoint='...', api_version='2024-07-01-preview', api_key='your-api-key', ) model = OpenAIModel( 'gpt-4o', provider=OpenAIProvider(openai_client=client), ) agent = Agent(model) ...` ### OpenAI Responses API PydanticAI also supports OpenAI's [Responses API](https://platform.openai.com/docs/api-reference/responses) through the [`OpenAIResponsesModel`](../api/models/openai/#pydantic_ai.models.openai.OpenAIResponsesModel) class. openai\_responses\_model.py `from pydantic_ai import Agent from pydantic_ai.models.openai import OpenAIResponsesModel model = OpenAIResponsesModel('gpt-4o') agent = Agent(model) ...` The Responses API has built-in tools that you can use instead of building your own: * [Web search](https://platform.openai.com/docs/guides/tools-web-search) : allow models to search the web for the latest information before generating a response. * [File search](https://platform.openai.com/docs/guides/tools-file-search) : allow models to search your files for relevant information before generating a response. * [Computer use](https://platform.openai.com/docs/guides/tools-computer-use) : allow models to use a computer to perform tasks on your behalf. You can use the [`OpenAIResponsesModelSettings`](../api/models/openai/#pydantic_ai.models.openai.OpenAIResponsesModelSettings) class to make use of those built-in tools: openai\_responses\_model\_settings.py `from openai.types.responses import WebSearchToolParam # (1)! from pydantic_ai import Agent from pydantic_ai.models.openai import OpenAIResponsesModel, OpenAIResponsesModelSettings model_settings = OpenAIResponsesModelSettings( openai_builtin_tools=[WebSearchToolParam(type='web_search_preview')], ) model = OpenAIResponsesModel('gpt-4o') agent = Agent(model=model, model_settings=model_settings) result = agent.run_sync('What is the weather in Tokyo?') print(result.data) """ As of 7:48 AM on Wednesday, April 2, 2025, in Tokyo, Japan, the weather is cloudy with a temperature of 53°F (12°C). """` 1. The file search tool and computer use tool can also be imported from `openai.types.responses`. You can learn more about the differences between the Responses API and Chat Completions API in the [OpenAI API docs](https://platform.openai.com/docs/guides/responses-vs-chat-completions) . Anthropic --------- ### Install To use [`AnthropicModel`](../api/models/anthropic/#pydantic_ai.models.anthropic.AnthropicModel) models, you need to either install [`pydantic-ai`](../install/) , or install [`pydantic-ai-slim`](../install/#slim-install) with the `anthropic` optional group: pipuv `pip install "pydantic-ai-slim[anthropic]"` `uv add "pydantic-ai-slim[anthropic]"` ### Configuration To use [Anthropic](https://anthropic.com) through their API, go to [console.anthropic.com/settings/keys](https://console.anthropic.com/settings/keys) to generate an API key. [`AnthropicModelName`](../api/models/anthropic/#pydantic_ai.models.anthropic.AnthropicModelName) contains a list of available Anthropic models. ### Environment variable Once you have the API key, you can set it as an environment variable: `export ANTHROPIC_API_KEY='your-api-key'` You can then use [`AnthropicModel`](../api/models/anthropic/#pydantic_ai.models.anthropic.AnthropicModel) by name: anthropic\_model\_by\_name.py `from pydantic_ai import Agent agent = Agent('anthropic:claude-3-5-sonnet-latest') ...` Or initialise the model directly with just the model name: anthropic\_model\_init.py `from pydantic_ai import Agent from pydantic_ai.models.anthropic import AnthropicModel model = AnthropicModel('claude-3-5-sonnet-latest') agent = Agent(model) ...` ### `provider` argument You can provide a custom [`Provider`](../api/providers/#pydantic_ai.providers.Provider) via the [`provider` argument](../api/models/anthropic/#pydantic_ai.models.anthropic.AnthropicModel.__init__) : anthropic\_model\_provider.py `from pydantic_ai import Agent from pydantic_ai.models.anthropic import AnthropicModel from pydantic_ai.providers.anthropic import AnthropicProvider model = AnthropicModel( 'claude-3-5-sonnet-latest', provider=AnthropicProvider(api_key='your-api-key') ) agent = Agent(model) ...` ### Custom HTTP Client You can customize the `AnthropicProvider` with a custom `httpx.AsyncClient`: anthropic\_model\_custom\_provider.py `from httpx import AsyncClient from pydantic_ai import Agent from pydantic_ai.models.anthropic import AnthropicModel from pydantic_ai.providers.anthropic import AnthropicProvider custom_http_client = AsyncClient(timeout=30) model = AnthropicModel( 'claude-3-5-sonnet-latest', provider=AnthropicProvider(api_key='your-api-key', http_client=custom_http_client), ) agent = Agent(model) ...` Gemini ------ ### Install To use [`GeminiModel`](../api/models/gemini/#pydantic_ai.models.gemini.GeminiModel) models, you just need to install [`pydantic-ai`](../install/) or [`pydantic-ai-slim`](../install/#slim-install) , no extra dependencies are required. ### Configuration [`GeminiModel`](../api/models/gemini/#pydantic_ai.models.gemini.GeminiModel) let's you use the Google's Gemini models through their [Generative Language API](https://ai.google.dev/api/all-methods) , `generativelanguage.googleapis.com`. [`GeminiModelName`](../api/models/gemini/#pydantic_ai.models.gemini.GeminiModelName) contains a list of available Gemini models that can be used through this interface. To use `GeminiModel`, go to [aistudio.google.com](https://aistudio.google.com/apikey) and select "Create API key". ### Environment variable Once you have the API key, you can set it as an environment variable: `export GEMINI_API_KEY=your-api-key` You can then use [`GeminiModel`](../api/models/gemini/#pydantic_ai.models.gemini.GeminiModel) by name: gemini\_model\_by\_name.py `from pydantic_ai import Agent agent = Agent('google-gla:gemini-2.0-flash') ...` Note The `google-gla` provider prefix represents the [Google **G**enerative **L**anguage **A**PI](https://ai.google.dev/api/all-methods) for `GeminiModel`s. `google-vertex` is used with [Vertex AI](https://cloud.google.com/vertex-ai/generative-ai/docs/learn/models) . Or initialise the model directly with just the model name and provider: gemini\_model\_init.py `from pydantic_ai import Agent from pydantic_ai.models.gemini import GeminiModel model = GeminiModel('gemini-2.0-flash', provider='google-gla') agent = Agent(model) ...` ### `provider` argument You can provide a custom [`Provider`](../api/providers/#pydantic_ai.providers.Provider) via the [`provider` argument](../api/models/gemini/#pydantic_ai.models.gemini.GeminiModel.__init__) : gemini\_model\_provider.py `from pydantic_ai import Agent from pydantic_ai.models.gemini import GeminiModel from pydantic_ai.providers.google_gla import GoogleGLAProvider model = GeminiModel( 'gemini-2.0-flash', provider=GoogleGLAProvider(api_key='your-api-key') ) agent = Agent(model) ...` You can also customize the `GoogleGLAProvider` with a custom `http_client`: gemini\_model\_custom\_provider.py `from httpx import AsyncClient from pydantic_ai import Agent from pydantic_ai.models.gemini import GeminiModel from pydantic_ai.providers.google_gla import GoogleGLAProvider custom_http_client = AsyncClient(timeout=30) model = GeminiModel( 'gemini-2.0-flash', provider=GoogleGLAProvider(api_key='your-api-key', http_client=custom_http_client), ) agent = Agent(model) ...` Gemini via VertexAI ------------------- If you are an enterprise user, you should use the `google-vertex` provider with [`GeminiModel`](../api/models/gemini/#pydantic_ai.models.gemini.GeminiModel) which uses the `*-aiplatform.googleapis.com` API. [`GeminiModelName`](../api/models/gemini/#pydantic_ai.models.gemini.GeminiModelName) contains a list of available Gemini models that can be used through this interface. ### Install To use the `google-vertex` provider with [`GeminiModel`](../api/models/gemini/#pydantic_ai.models.gemini.GeminiModel) , you need to either install [`pydantic-ai`](../install/) , or install [`pydantic-ai-slim`](../install/#slim-install) with the `vertexai` optional group: pipuv `pip install "pydantic-ai-slim[vertexai]"` `uv add "pydantic-ai-slim[vertexai]"` ### Configuration This interface has a number of advantages over `generativelanguage.googleapis.com` documented above: 1. The VertexAI API comes with more enterprise readiness guarantees. 2. You can [purchase provisioned throughput](https://cloud.google.com/vertex-ai/generative-ai/docs/provisioned-throughput#purchase-provisioned-throughput) with VertexAI to guarantee capacity. 3. If you're running PydanticAI inside GCP, you don't need to set up authentication, it should "just work". 4. You can decide which region to use, which might be important from a regulatory perspective, and might improve latency. The big disadvantage is that for local development you may need to create and configure a "service account", which I've found extremely painful to get right in the past. Whichever way you authenticate, you'll need to have VertexAI enabled in your GCP account. ### Application default credentials Luckily if you're running PydanticAI inside GCP, or you have the [`gcloud` CLI](https://cloud.google.com/sdk/gcloud) installed and configured, you should be able to use `VertexAIModel` without any additional setup. To use `VertexAIModel`, with [application default credentials](https://cloud.google.com/docs/authentication/application-default-credentials) configured (e.g. with `gcloud`), you can simply use: vertexai\_application\_default\_credentials.py `from pydantic_ai import Agent from pydantic_ai.models.gemini import GeminiModel model = GeminiModel('gemini-2.0-flash', provider='google-vertex') agent = Agent(model) ...` Internally this uses [`google.auth.default()`](https://google-auth.readthedocs.io/en/master/reference/google.auth.html) from the `google-auth` package to obtain credentials. Won't fail until `agent.run()` Because `google.auth.default()` requires network requests and can be slow, it's not run until you call `agent.run()`. You may also need to pass the [`project_id` argument to `GoogleVertexProvider`](../api/providers/#pydantic_ai.providers.google_vertex.GoogleVertexProvider) if application default credentials don't set a project, if you pass `project_id` and it conflicts with the project set by application default credentials, an error is raised. ### Service account If instead of application default credentials, you want to authenticate with a service account, you'll need to create a service account, add it to your GCP project (note: AFAIK this step is necessary even if you created the service account within the project), give that service account the "Vertex AI Service Agent" role, and download the service account JSON file. Once you have the JSON file, you can use it thus: vertexai\_service\_account.py `from pydantic_ai import Agent from pydantic_ai.models.gemini import GeminiModel from pydantic_ai.providers.google_vertex import GoogleVertexProvider model = GeminiModel( 'gemini-2.0-flash', provider=GoogleVertexProvider(service_account_file='path/to/service-account.json'), ) agent = Agent(model) ...` Alternatively, if you already have the service account information in memory, you can pass it as a dictionary: vertexai\_service\_account.py `import json from pydantic_ai import Agent from pydantic_ai.models.gemini import GeminiModel from pydantic_ai.providers.google_vertex import GoogleVertexProvider service_account_info = json.loads( '{"type": "service_account", "project_id": "my-project-id"}' ) model = GeminiModel( 'gemini-2.0-flash', provider=GoogleVertexProvider(service_account_info=service_account_info), ) agent = Agent(model) ...` ### Customising region Whichever way you authenticate, you can specify which region requests will be sent to via the [`region` argument](../api/providers/#pydantic_ai.providers.google_vertex.GoogleVertexProvider) . Using a region close to your application can improve latency and might be important from a regulatory perspective. vertexai\_region.py `from pydantic_ai import Agent from pydantic_ai.models.gemini import GeminiModel from pydantic_ai.providers.google_vertex import GoogleVertexProvider model = GeminiModel( 'gemini-2.0-flash', provider=GoogleVertexProvider(region='asia-east1') ) agent = Agent(model) ...` You can also customize the `GoogleVertexProvider` with a custom `http_client`: vertexai\_custom\_provider.py `from httpx import AsyncClient from pydantic_ai import Agent from pydantic_ai.models.gemini import GeminiModel from pydantic_ai.providers.google_vertex import GoogleVertexProvider custom_http_client = AsyncClient(timeout=30) model = GeminiModel( 'gemini-2.0-flash', provider=GoogleVertexProvider(region='asia-east1', http_client=custom_http_client), ) agent = Agent(model) ...` Groq ---- ### Install To use [`GroqModel`](../api/models/groq/#pydantic_ai.models.groq.GroqModel) , you need to either install [`pydantic-ai`](../install/) , or install [`pydantic-ai-slim`](../install/#slim-install) with the `groq` optional group: pipuv `pip install "pydantic-ai-slim[groq]"` `uv add "pydantic-ai-slim[groq]"` ### Configuration To use [Groq](https://groq.com/) through their API, go to [console.groq.com/keys](https://console.groq.com/keys) and follow your nose until you find the place to generate an API key. [`GroqModelName`](../api/models/groq/#pydantic_ai.models.groq.GroqModelName) contains a list of available Groq models. ### Environment variable Once you have the API key, you can set it as an environment variable: `export GROQ_API_KEY='your-api-key'` You can then use [`GroqModel`](../api/models/groq/#pydantic_ai.models.groq.GroqModel) by name: groq\_model\_by\_name.py `from pydantic_ai import Agent agent = Agent('groq:llama-3.3-70b-versatile') ...` Or initialise the model directly with just the model name: groq\_model\_init.py `from pydantic_ai import Agent from pydantic_ai.models.groq import GroqModel model = GroqModel('llama-3.3-70b-versatile') agent = Agent(model) ...` ### `provider` argument You can provide a custom [`Provider`](../api/providers/#pydantic_ai.providers.Provider) via the [`provider` argument](../api/models/groq/#pydantic_ai.models.groq.GroqModel.__init__) : groq\_model\_provider.py `from pydantic_ai import Agent from pydantic_ai.models.groq import GroqModel from pydantic_ai.providers.groq import GroqProvider model = GroqModel( 'llama-3.3-70b-versatile', provider=GroqProvider(api_key='your-api-key') ) agent = Agent(model) ...` You can also customize the [`GroqProvider`](../api/providers/#pydantic_ai.providers.groq.GroqProvider) with a custom `httpx.AsyncHTTPClient`: groq\_model\_custom\_provider.py `from httpx import AsyncClient from pydantic_ai import Agent from pydantic_ai.models.groq import GroqModel from pydantic_ai.providers.groq import GroqProvider custom_http_client = AsyncClient(timeout=30) model = GroqModel( 'llama-3.3-70b-versatile', provider=GroqProvider(api_key='your-api-key', http_client=custom_http_client), ) agent = Agent(model) ...` Mistral ------- ### Install To use [`MistralModel`](../api/models/mistral/#pydantic_ai.models.mistral.MistralModel) , you need to either install [`pydantic-ai`](../install/) , or install [`pydantic-ai-slim`](../install/#slim-install) with the `mistral` optional group: pipuv `pip install "pydantic-ai-slim[mistral]"` `uv add "pydantic-ai-slim[mistral]"` ### Configuration To use [Mistral](https://mistral.ai) through their API, go to [console.mistral.ai/api-keys/](https://console.mistral.ai/api-keys/) and follow your nose until you find the place to generate an API key. [`LatestMistralModelNames`](../api/models/mistral/#pydantic_ai.models.mistral.LatestMistralModelNames) contains a list of the most popular Mistral models. ### Environment variable Once you have the API key, you can set it as an environment variable: `export MISTRAL_API_KEY='your-api-key'` You can then use [`MistralModel`](../api/models/mistral/#pydantic_ai.models.mistral.MistralModel) by name: mistral\_model\_by\_name.py `from pydantic_ai import Agent agent = Agent('mistral:mistral-large-latest') ...` Or initialise the model directly with just the model name: mistral\_model\_init.py `from pydantic_ai import Agent from pydantic_ai.models.mistral import MistralModel model = MistralModel('mistral-small-latest') agent = Agent(model) ...` ### `provider` argument You can provide a custom [`Provider`](../api/providers/#pydantic_ai.providers.Provider) via the [`provider` argument](../api/models/mistral/#pydantic_ai.models.mistral.MistralModel.__init__) : groq\_model\_provider.py `from pydantic_ai import Agent from pydantic_ai.models.mistral import MistralModel from pydantic_ai.providers.mistral import MistralProvider model = MistralModel( 'mistral-large-latest', provider=MistralProvider(api_key='your-api-key') ) agent = Agent(model) ...` You can also customize the provider with a custom `httpx.AsyncHTTPClient`: groq\_model\_custom\_provider.py `from httpx import AsyncClient from pydantic_ai import Agent from pydantic_ai.models.mistral import MistralModel from pydantic_ai.providers.mistral import MistralProvider custom_http_client = AsyncClient(timeout=30) model = MistralModel( 'mistral-large-latest', provider=MistralProvider(api_key='your-api-key', http_client=custom_http_client), ) agent = Agent(model) ...` Cohere ------ ### Install To use [`CohereModel`](../api/models/cohere/#pydantic_ai.models.cohere.CohereModel) , you need to either install [`pydantic-ai`](../install/) , or install [`pydantic-ai-slim`](../install/#slim-install) with the `cohere` optional group: pipuv `pip install "pydantic-ai-slim[cohere]"` `uv add "pydantic-ai-slim[cohere]"` ### Configuration To use [Cohere](https://cohere.com/) through their API, go to [dashboard.cohere.com/api-keys](https://dashboard.cohere.com/api-keys) and follow your nose until you find the place to generate an API key. [`CohereModelName`](../api/models/cohere/#pydantic_ai.models.cohere.CohereModelName) contains a list of the most popular Cohere models. ### Environment variable Once you have the API key, you can set it as an environment variable: `export CO_API_KEY='your-api-key'` You can then use [`CohereModel`](../api/models/cohere/#pydantic_ai.models.cohere.CohereModel) by name: cohere\_model\_by\_name.py `from pydantic_ai import Agent agent = Agent('cohere:command') ...` Or initialise the model directly with just the model name: cohere\_model\_init.py `from pydantic_ai import Agent from pydantic_ai.models.cohere import CohereModel model = CohereModel('command') agent = Agent(model) ...` ### `provider` argument You can provide a custom [`Provider`](../api/providers/#pydantic_ai.providers.Provider) via the [`provider` argument](../api/models/cohere/#pydantic_ai.models.cohere.CohereModel.__init__) : cohere\_model\_provider.py `from pydantic_ai import Agent from pydantic_ai.models.cohere import CohereModel from pydantic_ai.providers.cohere import CohereProvider model = CohereModel('command', provider=CohereProvider(api_key='your-api-key')) agent = Agent(model) ...` You can also customize the `CohereProvider` with a custom `http_client`: cohere\_model\_custom\_provider.py `from httpx import AsyncClient from pydantic_ai import Agent from pydantic_ai.models.cohere import CohereModel from pydantic_ai.providers.cohere import CohereProvider custom_http_client = AsyncClient(timeout=30) model = CohereModel( 'command', provider=CohereProvider(api_key='your-api-key', http_client=custom_http_client), ) agent = Agent(model) ...` Bedrock ------- ### Install To use [`BedrockConverseModel`](../api/models/bedrock/#pydantic_ai.models.bedrock.BedrockConverseModel) , you need to either install [`pydantic-ai`](../install/) , or install [`pydantic-ai-slim`](../install/#slim-install) with the `bedrock` optional group: pipuv `pip install "pydantic-ai-slim[bedrock]"` `uv add "pydantic-ai-slim[bedrock]"` ### Configuration To use [AWS Bedrock](https://aws.amazon.com/bedrock/) , you'll need an AWS account with Bedrock enabled and appropriate credentials. You can use either AWS credentials directly or a pre-configured boto3 client. [`BedrockModelName`](../api/models/bedrock/#pydantic_ai.models.bedrock.BedrockModelName) contains a list of available Bedrock models, including models from Anthropic, Amazon, Cohere, Meta, and Mistral. ### Environment variables You can set your AWS credentials as environment variables ([among other options](https://boto3.amazonaws.com/v1/documentation/api/latest/guide/configuration.html#using-environment-variables) : `export AWS_ACCESS_KEY_ID='your-access-key' export AWS_SECRET_ACCESS_KEY='your-secret-key' export AWS_DEFAULT_REGION='us-east-1' # or your preferred region` You can then use [`BedrockConverseModel`](../api/models/bedrock/#pydantic_ai.models.bedrock.BedrockConverseModel) by name: bedrock\_model\_by\_name.py `from pydantic_ai import Agent agent = Agent('bedrock:anthropic.claude-3-sonnet-20240229-v1:0') ...` Or initialize the model directly with just the model name: bedrock\_model\_init.py `from pydantic_ai import Agent from pydantic_ai.models.bedrock import BedrockConverseModel model = BedrockConverseModel('anthropic.claude-3-sonnet-20240229-v1:0') agent = Agent(model) ...` ### `provider` argument You can provide a custom [`BedrockProvider`](../api/providers/#pydantic_ai.providers.bedrock.BedrockProvider) via the [`provider` argument](../api/models/bedrock/#pydantic_ai.models.bedrock.BedrockConverseModel.__init__) . This is useful when you want to specify credentials directly or use a custom boto3 client: bedrock\_model\_provider.py `from pydantic_ai import Agent from pydantic_ai.models.bedrock import BedrockConverseModel from pydantic_ai.providers.bedrock import BedrockProvider # Using AWS credentials directly model = BedrockConverseModel( 'anthropic.claude-3-sonnet-20240229-v1:0', provider=BedrockProvider( region_name='us-east-1', aws_access_key_id='your-access-key', aws_secret_access_key='your-secret-key', ), ) agent = Agent(model) ...` You can also pass a pre-configured boto3 client: bedrock\_model\_boto3.py `import boto3 from pydantic_ai import Agent from pydantic_ai.models.bedrock import BedrockConverseModel from pydantic_ai.providers.bedrock import BedrockProvider # Using a pre-configured boto3 client bedrock_client = boto3.client('bedrock-runtime', region_name='us-east-1') model = BedrockConverseModel( 'anthropic.claude-3-sonnet-20240229-v1:0', provider=BedrockProvider(bedrock_client=bedrock_client), ) agent = Agent(model) ...` OpenAI-compatible Models ------------------------ Many of the models are compatible with OpenAI API, and thus can be used with [`OpenAIModel`](../api/models/openai/#pydantic_ai.models.openai.OpenAIModel) in PydanticAI. Before getting started, check the [OpenAI](#openai) section for installation and configuration instructions. To use another OpenAI-compatible API, you can make use of the [`base_url`](../api/providers/#pydantic_ai.providers.openai.OpenAIProvider.__init__) and [`api_key`](../api/providers/#pydantic_ai.providers.openai.OpenAIProvider.__init__) arguments from `OpenAIProvider`: deepseek\_model\_init.py `from pydantic_ai import Agent from pydantic_ai.models.openai import OpenAIModel from pydantic_ai.providers.openai import OpenAIProvider model = OpenAIModel( 'model_name', provider=OpenAIProvider( base_url='https://.com', api_key='your-api-key' ), ) agent = Agent(model) ...` You can also use the `provider` argument with a custom provider class like the [`DeepSeekProvider`](../api/providers/#pydantic_ai.providers.deepseek.DeepSeekProvider) : deepseek\_model\_init\_provider\_class.py `from pydantic_ai import Agent from pydantic_ai.models.openai import OpenAIModel from pydantic_ai.providers.deepseek import DeepSeekProvider model = OpenAIModel( 'deepseek-chat', provider=DeepSeekProvider(api_key='your-deepseek-api-key'), ) agent = Agent(model) ...` You can also customize any provider with a custom `http_client`: deepseek\_model\_init\_provider\_custom.py `from httpx import AsyncClient from pydantic_ai import Agent from pydantic_ai.models.openai import OpenAIModel from pydantic_ai.providers.deepseek import DeepSeekProvider custom_http_client = AsyncClient(timeout=30) model = OpenAIModel( 'deepseek-chat', provider=DeepSeekProvider( api_key='your-deepseek-api-key', http_client=custom_http_client ), ) agent = Agent(model) ...` ### Ollama To use [Ollama](https://ollama.com/) , you must first download the Ollama client, and then download a model using the [Ollama model library](https://ollama.com/library) . You must also ensure the Ollama server is running when trying to make requests to it. For more information, please see the [Ollama documentation](https://github.com/ollama/ollama/tree/main/docs) . #### Example local usage With `ollama` installed, you can run the server with the model you want to use: terminal-run-ollama `ollama run llama3.2` (this will pull the `llama3.2` model if you don't already have it downloaded) Then run your code, here's a minimal example: ollama\_example.py `from pydantic import BaseModel from pydantic_ai import Agent from pydantic_ai.models.openai import OpenAIModel from pydantic_ai.providers.openai import OpenAIProvider class CityLocation(BaseModel): city: str country: str ollama_model = OpenAIModel( model_name='llama3.2', provider=OpenAIProvider(base_url='http://localhost:11434/v1') ) agent = Agent(ollama_model, result_type=CityLocation) result = agent.run_sync('Where were the olympics held in 2012?') print(result.data) #> city='London' country='United Kingdom' print(result.usage()) """ Usage(requests=1, request_tokens=57, response_tokens=8, total_tokens=65, details=None) """` #### Example using a remote server ollama\_example\_with\_remote\_server.py `from pydantic import BaseModel from pydantic_ai import Agent from pydantic_ai.models.openai import OpenAIModel from pydantic_ai.providers.openai import OpenAIProvider ollama_model = OpenAIModel( model_name='qwen2.5-coder:7b', # (1)! provider=OpenAIProvider(base_url='http://192.168.1.74:11434/v1'), # (2)! ) class CityLocation(BaseModel): city: str country: str agent = Agent(model=ollama_model, result_type=CityLocation) result = agent.run_sync('Where were the olympics held in 2012?') print(result.data) #> city='London' country='United Kingdom' print(result.usage()) """ Usage(requests=1, request_tokens=57, response_tokens=8, total_tokens=65, details=None) """` 1. The name of the model running on the remote server 2. The url of the remote server ### Azure AI Foundry If you want to use [Azure AI Foundry](https://ai.azure.com/) as your provider, you can do so by using the [`AzureProvider`](../api/providers/#pydantic_ai.providers.azure.AzureProvider) class. azure\_provider\_example.py `from pydantic_ai import Agent from pydantic_ai.models.openai import OpenAIModel from pydantic_ai.providers.azure import AzureProvider model = OpenAIModel( 'gpt-4o', provider=AzureProvider( azure_endpoint='your-azure-endpoint', api_version='your-api-version', api_key='your-api-key', ), ) agent = Agent(model) ...` ### OpenRouter To use [OpenRouter](https://openrouter.ai) , first create an API key at [openrouter.ai/keys](https://openrouter.ai/keys) . Once you have the API key, you can use it with the [`OpenAIProvider`](../api/providers/#pydantic_ai.providers.openai.OpenAIProvider) : openrouter\_model\_init.py `from pydantic_ai import Agent from pydantic_ai.models.openai import OpenAIModel from pydantic_ai.providers.openai import OpenAIProvider model = OpenAIModel( 'anthropic/claude-3.5-sonnet', provider=OpenAIProvider( base_url='https://openrouter.ai/api/v1', api_key='your-openrouter-api-key', ), ) agent = Agent(model) ...` ### Grok (xAI) Go to [xAI API Console](https://console.x.ai/) and create an API key. Once you have the API key, you can use it with the [`OpenAIProvider`](../api/providers/#pydantic_ai.providers.openai.OpenAIProvider) : grok\_model\_init.py `from pydantic_ai import Agent from pydantic_ai.models.openai import OpenAIModel from pydantic_ai.providers.openai import OpenAIProvider model = OpenAIModel( 'grok-2-1212', provider=OpenAIProvider(base_url='https://api.x.ai/v1', api_key='your-xai-api-key'), ) agent = Agent(model) ...` ### Perplexity Follow the Perplexity [getting started](https://docs.perplexity.ai/guides/getting-started) guide to create an API key. Then, you can query the Perplexity API with the following: perplexity\_model\_init.py `from pydantic_ai import Agent from pydantic_ai.models.openai import OpenAIModel from pydantic_ai.providers.openai import OpenAIProvider model = OpenAIModel( 'sonar-pro', provider=OpenAIProvider( base_url='https://api.perplexity.ai', api_key='your-perplexity-api-key', ), ) agent = Agent(model) ...` ### Fireworks AI Go to [Fireworks.AI](https://fireworks.ai/) and create an API key in your account settings. Once you have the API key, you can use it with the [`OpenAIProvider`](../api/providers/#pydantic_ai.providers.openai.OpenAIProvider) : fireworks\_model\_init.py `from pydantic_ai import Agent from pydantic_ai.models.openai import OpenAIModel from pydantic_ai.providers.openai import OpenAIProvider model = OpenAIModel( 'accounts/fireworks/models/qwq-32b', # model library available at https://fireworks.ai/models provider=OpenAIProvider( base_url='https://api.fireworks.ai/inference/v1', api_key='your-fireworks-api-key', ), ) agent = Agent(model) ...` ### Together AI Go to [Together.ai](https://www.together.ai/) and create an API key in your account settings. Once you have the API key, you can use it with the [`OpenAIProvider`](../api/providers/#pydantic_ai.providers.openai.OpenAIProvider) : together\_model\_init.py `from pydantic_ai import Agent from pydantic_ai.models.openai import OpenAIModel from pydantic_ai.providers.openai import OpenAIProvider model = OpenAIModel( 'meta-llama/Llama-3.3-70B-Instruct-Turbo-Free', # model library available at https://www.together.ai/models provider=OpenAIProvider( base_url='https://api.together.xyz/v1', api_key='your-together-api-key', ), ) agent = Agent(model) ...` Implementing Custom Models -------------------------- To implement support for models not already supported, you will need to subclass the [`Model`](../api/models/base/#pydantic_ai.models.Model) abstract base class. For streaming, you'll also need to implement the following abstract base class: * [`StreamedResponse`](../api/models/base/#pydantic_ai.models.StreamedResponse) The best place to start is to review the source code for existing implementations, e.g. [`OpenAIModel`](https://github.com/pydantic/pydantic-ai/blob/main/pydantic_ai_slim/pydantic_ai/models/openai.py) . For details on when we'll accept contributions adding new models to PydanticAI, see the [contributing guidelines](../contributing/#new-model-rules) . Fallback -------- You can use [`FallbackModel`](../api/models/fallback/#pydantic_ai.models.fallback.FallbackModel) to attempt multiple models in sequence until one returns a successful result. Under the hood, PydanticAI automatically switches from one model to the next if the current model returns a 4xx or 5xx status code. In the following example, the agent first makes a request to the OpenAI model (which fails due to an invalid API key), and then falls back to the Anthropic model. fallback\_model.py `from pydantic_ai import Agent from pydantic_ai.models.anthropic import AnthropicModel from pydantic_ai.models.fallback import FallbackModel from pydantic_ai.models.openai import OpenAIModel openai_model = OpenAIModel('gpt-4o') anthropic_model = AnthropicModel('claude-3-5-sonnet-latest') fallback_model = FallbackModel(openai_model, anthropic_model) agent = Agent(fallback_model) response = agent.run_sync('What is the capital of France?') print(response.data) #> Paris print(response.all_messages()) """ [ ModelRequest( parts=[ UserPromptPart( content='What is the capital of France?', timestamp=datetime.datetime(...), part_kind='user-prompt', ) ], kind='request', ), ModelResponse( parts=[TextPart(content='Paris', part_kind='text')], model_name='claude-3-5-sonnet-latest', timestamp=datetime.datetime(...), kind='response', ), ] """` The `ModelResponse` message above indicates in the `model_name` field that the result was returned by the Anthropic model, which is the second model specified in the `FallbackModel`. Note Each model's options should be configured individually. For example, `base_url`, `api_key`, and custom clients should be set on each model itself, not on the `FallbackModel`. In this next example, we demonstrate the exception-handling capabilities of `FallbackModel`. If all models fail, a [`FallbackExceptionGroup`](../api/exceptions/#pydantic_ai.exceptions.FallbackExceptionGroup) is raised, which contains all the exceptions encountered during the `run` execution. Python >=3.11Python <3.11 fallback\_model\_failure.py `from pydantic_ai import Agent from pydantic_ai.exceptions import ModelHTTPError from pydantic_ai.models.anthropic import AnthropicModel from pydantic_ai.models.fallback import FallbackModel from pydantic_ai.models.openai import OpenAIModel openai_model = OpenAIModel('gpt-4o') anthropic_model = AnthropicModel('claude-3-5-sonnet-latest') fallback_model = FallbackModel(openai_model, anthropic_model) agent = Agent(fallback_model) try: response = agent.run_sync('What is the capital of France?') except* ModelHTTPError as exc_group: for exc in exc_group.exceptions: print(exc)` Since [`except*`](https://docs.python.org/3/reference/compound_stmts.html#except-star) is only supported in Python 3.11+, we use the [`exceptiongroup`](https://github.com/agronholm/exceptiongroup) backport package for earlier Python versions: fallback\_model\_failure.py `from exceptiongroup import catch from pydantic_ai import Agent from pydantic_ai.exceptions import ModelHTTPError from pydantic_ai.models.anthropic import AnthropicModel from pydantic_ai.models.fallback import FallbackModel from pydantic_ai.models.openai import OpenAIModel def model_status_error_handler(exc_group: BaseExceptionGroup) -> None: for exc in exc_group.exceptions: print(exc) openai_model = OpenAIModel('gpt-4o') anthropic_model = AnthropicModel('claude-3-5-sonnet-latest') fallback_model = FallbackModel(openai_model, anthropic_model) agent = Agent(fallback_model) with catch({ModelHTTPError: model_status_error_handler}): response = agent.run_sync('What is the capital of France?')` By default, the `FallbackModel` only moves on to the next model if the current model raises a [`ModelHTTPError`](../api/exceptions/#pydantic_ai.exceptions.ModelHTTPError) . You can customize this behavior by passing a custom `fallback_on` argument to the `FallbackModel` constructor. --- # Common Tools - PydanticAI Common Tools ============ PydanticAI ships with native tools that can be used to enhance your agent's capabilities. DuckDuckGo Search Tool ---------------------- The DuckDuckGo search tool allows you to search the web for information. It is built on top of the [DuckDuckGo API](https://github.com/deedy5/duckduckgo_search) . ### Installation To use [`duckduckgo_search_tool`](../api/common_tools/#pydantic_ai.common_tools.duckduckgo.duckduckgo_search_tool) , you need to install [`pydantic-ai-slim`](../install/#slim-install) with the `duckduckgo` optional group: pipuv `pip install "pydantic-ai-slim[duckduckgo]"` `uv add "pydantic-ai-slim[duckduckgo]"` ### Usage Here's an example of how you can use the DuckDuckGo search tool with an agent: main.py `from pydantic_ai import Agent from pydantic_ai.common_tools.duckduckgo import duckduckgo_search_tool agent = Agent( 'openai:o3-mini', tools=[duckduckgo_search_tool()], system_prompt='Search DuckDuckGo for the given query and return the results.', ) result = agent.run_sync( 'Can you list the top five highest-grossing animated films of 2025?' ) print(result.data) """ I looked into several sources on animated box‐office performance in 2025, and while detailed rankings can shift as more money is tallied, multiple independent reports have already highlighted a couple of record‐breaking shows. For example: • Ne Zha 2 – News outlets (Variety, Wikipedia's "List of animated feature films of 2025", and others) have reported that this Chinese title not only became the highest‑grossing animated film of 2025 but also broke records as the highest‑grossing non‑English animated film ever. One article noted its run exceeded US$1.7 billion. • Inside Out 2 – According to data shared on Statista and in industry news, this Pixar sequel has been on pace to set new records (with some sources even noting it as the highest‑grossing animated film ever, as of January 2025). Beyond those two, some entertainment trade sites (for example, a Just Jared article titled "Top 10 Highest-Earning Animated Films at the Box Office Revealed") have begun listing a broader top‑10. Although full consolidated figures can sometimes differ by source and are updated daily during a box‑office run, many of the industry trackers have begun to single out five films as the biggest earners so far in 2025. Unfortunately, although multiple articles discuss the "top animated films" of 2025, there isn't yet a single, universally accepted list with final numbers that names the complete top five. (Box‑office rankings, especially mid‑year, can be fluid as films continue to add to their totals.) Based on what several sources note so far, the two undisputed leaders are: 1. Ne Zha 2 2. Inside Out 2 The remaining top spots (3–5) are reported by some outlets in their "Top‑10 Animated Films" lists for 2025 but the titles and order can vary depending on the source and the exact cut‑off date of the data. For the most up‑to‑date and detailed ranking (including the 3rd, 4th, and 5th highest‑grossing films), I recommend checking resources like: • Wikipedia's "List of animated feature films of 2025" page • Box‑office tracking sites (such as Box Office Mojo or The Numbers) • Trade articles like the one on Just Jared To summarize with what is clear from the current reporting: 1. Ne Zha 2 2. Inside Out 2 3–5. Other animated films (yet to be definitively finalized across all reporting outlets) If you're looking for a final, consensus list of the top five, it may be best to wait until the 2025 year‑end box‑office tallies are in or to consult a regularly updated entertainment industry source. Would you like help finding a current source or additional details on where to look for the complete updated list? """` Tavily Search Tool ------------------ Info Tavily is a paid service, but they have free credits to explore their product. You need to [sign up for an account](https://app.tavily.com/home) and get an API key to use the Tavily search tool. The Tavily search tool allows you to search the web for information. It is built on top of the [Tavily API](https://tavily.com/) . ### Installation To use [`tavily_search_tool`](../api/common_tools/#pydantic_ai.common_tools.tavily.tavily_search_tool) , you need to install [`pydantic-ai-slim`](../install/#slim-install) with the `tavily` optional group: pipuv `pip install "pydantic-ai-slim[tavily]"` `uv add "pydantic-ai-slim[tavily]"` ### Usage Here's an example of how you can use the Tavily search tool with an agent: main.py `import os from pydantic_ai.agent import Agent from pydantic_ai.common_tools.tavily import tavily_search_tool api_key = os.getenv('TAVILY_API_KEY') assert api_key is not None agent = Agent( 'openai:o3-mini', tools=[tavily_search_tool(api_key)], system_prompt='Search Tavily for the given query and return the results.', ) result = agent.run_sync('Tell me the top news in the GenAI world, give me links.') print(result.data) """ Here are some of the top recent news articles related to GenAI: 1. How CLEAR users can improve risk analysis with GenAI – Thomson Reuters Read more: https://legal.thomsonreuters.com/blog/how-clear-users-can-improve-risk-analysis-with-genai/ (This article discusses how CLEAR's new GenAI-powered tool streamlines risk analysis by quickly summarizing key information from various public data sources.) 2. TELUS Digital Survey Reveals Enterprise Employees Are Entering Sensitive Data Into AI Assistants More Than You Think – FT.com Read more: https://markets.ft.com/data/announce/detail?dockey=600-202502260645BIZWIRE_USPRX____20250226_BW490609-1 (This news piece highlights findings from a TELUS Digital survey showing that many enterprise employees use public GenAI tools and sometimes even enter sensitive data.) 3. The Essential Guide to Generative AI – Virtualization Review Read more: https://virtualizationreview.com/Whitepapers/2025/02/SNOWFLAKE-The-Essential-Guide-to-Generative-AI.aspx (This guide provides insights into how GenAI is revolutionizing enterprise strategies and productivity, with input from industry leaders.) Feel free to click on the links to dive deeper into each story! """` --- # Messages and chat history - PydanticAI Messages and chat history ========================= PydanticAI provides access to messages exchanged during an agent run. These messages can be used both to continue a coherent conversation, and to understand how an agent performed. ### Accessing Messages from Results After running an agent, you can access the messages exchanged during that run from the `result` object. Both [`RunResult`](../api/agent/#pydantic_ai.agent.AgentRunResult) (returned by [`Agent.run`](../api/agent/#pydantic_ai.agent.Agent.run) , [`Agent.run_sync`](../api/agent/#pydantic_ai.agent.Agent.run_sync) ) and [`StreamedRunResult`](../api/result/#pydantic_ai.result.StreamedRunResult) (returned by [`Agent.run_stream`](../api/agent/#pydantic_ai.agent.Agent.run_stream) ) have the following methods: * [`all_messages()`](../api/agent/#pydantic_ai.agent.AgentRunResult.all_messages) : returns all messages, including messages from prior runs. There's also a variant that returns JSON bytes, [`all_messages_json()`](../api/agent/#pydantic_ai.agent.AgentRunResult.all_messages_json) . * [`new_messages()`](../api/agent/#pydantic_ai.agent.AgentRunResult.new_messages) : returns only the messages from the current run. There's also a variant that returns JSON bytes, [`new_messages_json()`](../api/agent/#pydantic_ai.agent.AgentRunResult.new_messages_json) . StreamedRunResult and complete messages On [`StreamedRunResult`](../api/result/#pydantic_ai.result.StreamedRunResult) , the messages returned from these methods will only include the final result message once the stream has finished. E.g. you've awaited one of the following coroutines: * [`StreamedRunResult.stream()`](../api/result/#pydantic_ai.result.StreamedRunResult.stream) * [`StreamedRunResult.stream_text()`](../api/result/#pydantic_ai.result.StreamedRunResult.stream_text) * [`StreamedRunResult.stream_structured()`](../api/result/#pydantic_ai.result.StreamedRunResult.stream_structured) * [`StreamedRunResult.get_data()`](../api/result/#pydantic_ai.result.StreamedRunResult.get_data) **Note:** The final result message will NOT be added to result messages if you use [`.stream_text(delta=True)`](../api/result/#pydantic_ai.result.StreamedRunResult.stream_text) since in this case the result content is never built as one string. Example of accessing methods on a [`RunResult`](../api/agent/#pydantic_ai.agent.AgentRunResult) : run\_result\_messages.py `from pydantic_ai import Agent agent = Agent('openai:gpt-4o', system_prompt='Be a helpful assistant.') result = agent.run_sync('Tell me a joke.') print(result.data) #> Did you hear about the toothpaste scandal? They called it Colgate. # all messages from the run print(result.all_messages()) """ [ ModelRequest( parts=[ SystemPromptPart( content='Be a helpful assistant.', timestamp=datetime.datetime(...), dynamic_ref=None, part_kind='system-prompt', ), UserPromptPart( content='Tell me a joke.', timestamp=datetime.datetime(...), part_kind='user-prompt', ), ], kind='request', ), ModelResponse( parts=[ TextPart( content='Did you hear about the toothpaste scandal? They called it Colgate.', part_kind='text', ) ], model_name='gpt-4o', timestamp=datetime.datetime(...), kind='response', ), ] """` _(This example is complete, it can be run "as is")_ Example of accessing methods on a [`StreamedRunResult`](../api/result/#pydantic_ai.result.StreamedRunResult) : streamed\_run\_result\_messages.py `from pydantic_ai import Agent agent = Agent('openai:gpt-4o', system_prompt='Be a helpful assistant.') async def main(): async with agent.run_stream('Tell me a joke.') as result: # incomplete messages before the stream finishes print(result.all_messages()) """ [ ModelRequest( parts=[ SystemPromptPart( content='Be a helpful assistant.', timestamp=datetime.datetime(...), dynamic_ref=None, part_kind='system-prompt', ), UserPromptPart( content='Tell me a joke.', timestamp=datetime.datetime(...), part_kind='user-prompt', ), ], kind='request', ) ] """ async for text in result.stream_text(): print(text) #> Did you hear #> Did you hear about the toothpaste #> Did you hear about the toothpaste scandal? They called #> Did you hear about the toothpaste scandal? They called it Colgate. # complete messages once the stream finishes print(result.all_messages()) """ [ ModelRequest( parts=[ SystemPromptPart( content='Be a helpful assistant.', timestamp=datetime.datetime(...), dynamic_ref=None, part_kind='system-prompt', ), UserPromptPart( content='Tell me a joke.', timestamp=datetime.datetime(...), part_kind='user-prompt', ), ], kind='request', ), ModelResponse( parts=[ TextPart( content='Did you hear about the toothpaste scandal? They called it Colgate.', part_kind='text', ) ], model_name='gpt-4o', timestamp=datetime.datetime(...), kind='response', ), ] """` _(This example is complete, it can be run "as is" — you'll need to add `asyncio.run(main())` to run `main`)_ ### Using Messages as Input for Further Agent Runs The primary use of message histories in PydanticAI is to maintain context across multiple agent runs. To use existing messages in a run, pass them to the `message_history` parameter of [`Agent.run`](../api/agent/#pydantic_ai.agent.Agent.run) , [`Agent.run_sync`](../api/agent/#pydantic_ai.agent.Agent.run_sync) or [`Agent.run_stream`](../api/agent/#pydantic_ai.agent.Agent.run_stream) . If `message_history` is set and not empty, a new system prompt is not generated — we assume the existing message history includes a system prompt. Reusing messages in a conversation `from pydantic_ai import Agent agent = Agent('openai:gpt-4o', system_prompt='Be a helpful assistant.') result1 = agent.run_sync('Tell me a joke.') print(result1.data) #> Did you hear about the toothpaste scandal? They called it Colgate. result2 = agent.run_sync('Explain?', message_history=result1.new_messages()) print(result2.data) #> This is an excellent joke invented by Samuel Colvin, it needs no explanation. print(result2.all_messages()) """ [ ModelRequest( parts=[ SystemPromptPart( content='Be a helpful assistant.', timestamp=datetime.datetime(...), dynamic_ref=None, part_kind='system-prompt', ), UserPromptPart( content='Tell me a joke.', timestamp=datetime.datetime(...), part_kind='user-prompt', ), ], kind='request', ), ModelResponse( parts=[ TextPart( content='Did you hear about the toothpaste scandal? They called it Colgate.', part_kind='text', ) ], model_name='gpt-4o', timestamp=datetime.datetime(...), kind='response', ), ModelRequest( parts=[ UserPromptPart( content='Explain?', timestamp=datetime.datetime(...), part_kind='user-prompt', ) ], kind='request', ), ModelResponse( parts=[ TextPart( content='This is an excellent joke invented by Samuel Colvin, it needs no explanation.', part_kind='text', ) ], model_name='gpt-4o', timestamp=datetime.datetime(...), kind='response', ), ] """` _(This example is complete, it can be run "as is")_ Storing and loading messages (to JSON) -------------------------------------- While maintaining conversation state in memory is enough for many applications, often times you may want to store the messages history of an agent run on disk or in a database. This might be for evals, for sharing data between Python and JavaScript/TypeScript, or any number of other use cases. The intended way to do this is using a `TypeAdapter`. We export [`ModelMessagesTypeAdapter`](../api/messages/#pydantic_ai.messages.ModelMessagesTypeAdapter) that can be used for this, or you can create your own. Here's an example showing how: serialize messages to json `from pydantic_core import to_jsonable_python from pydantic_ai import Agent from pydantic_ai.messages import ModelMessagesTypeAdapter # (1)! agent = Agent('openai:gpt-4o', system_prompt='Be a helpful assistant.') result1 = agent.run_sync('Tell me a joke.') history_step_1 = result1.all_messages() as_python_objects = to_jsonable_python(history_step_1) # (2)! same_history_as_step_1 = ModelMessagesTypeAdapter.validate_python(as_python_objects) result2 = agent.run_sync( # (3)! 'Tell me a different joke.', message_history=same_history_as_step_1 )` 1. Alternatively, you can create a `TypeAdapter` from scratch: `from pydantic import TypeAdapter from pydantic_ai.messages import ModelMessage ModelMessagesTypeAdapter = TypeAdapter(list[ModelMessage])` 2. Alternatively you can serialize to/from JSON directly: `from pydantic_core import to_json ... as_json_objects = to_json(history_step_1) same_history_as_step_1 = ModelMessagesTypeAdapter.validate_json(as_json_objects)` 3. You can now continue the conversation with history `same_history_as_step_1` despite creating a new agent run. _(This example is complete, it can be run "as is")_ Other ways of using messages ---------------------------- Since messages are defined by simple dataclasses, you can manually create and manipulate, e.g. for testing. The message format is independent of the model used, so you can use messages in different agents, or the same agent with different models. In the example below, we reuse the message from the first agent run, which uses the `openai:gpt-4o` model, in a second agent run using the `google-gla:gemini-1.5-pro` model. Reusing messages with a different model `from pydantic_ai import Agent agent = Agent('openai:gpt-4o', system_prompt='Be a helpful assistant.') result1 = agent.run_sync('Tell me a joke.') print(result1.data) #> Did you hear about the toothpaste scandal? They called it Colgate. result2 = agent.run_sync( 'Explain?', model='google-gla:gemini-1.5-pro', message_history=result1.new_messages(), ) print(result2.data) #> This is an excellent joke invented by Samuel Colvin, it needs no explanation. print(result2.all_messages()) """ [ ModelRequest( parts=[ SystemPromptPart( content='Be a helpful assistant.', timestamp=datetime.datetime(...), dynamic_ref=None, part_kind='system-prompt', ), UserPromptPart( content='Tell me a joke.', timestamp=datetime.datetime(...), part_kind='user-prompt', ), ], kind='request', ), ModelResponse( parts=[ TextPart( content='Did you hear about the toothpaste scandal? They called it Colgate.', part_kind='text', ) ], model_name='gpt-4o', timestamp=datetime.datetime(...), kind='response', ), ModelRequest( parts=[ UserPromptPart( content='Explain?', timestamp=datetime.datetime(...), part_kind='user-prompt', ) ], kind='request', ), ModelResponse( parts=[ TextPart( content='This is an excellent joke invented by Samuel Colvin, it needs no explanation.', part_kind='text', ) ], model_name='gemini-1.5-pro', timestamp=datetime.datetime(...), kind='response', ), ] """` Examples -------- For a more complete example of using messages in conversations, see the [chat app](../examples/chat-app/) example. --- # Debugging and Monitoring - PydanticAI Debugging and Monitoring ======================== Applications that use LLMs have some challenges that are well known and understood: LLMs are **slow**, **unreliable** and **expensive**. These applications also have some challenges that most developers have encountered much less often: LLMs are **fickle** and **non-deterministic**. Subtle changes in a prompt can completely change a model's performance, and there's no `EXPLAIN` query you can run to understand why. Warning From a software engineers point of view, you can think of LLMs as the worst database you've ever heard of, but worse. If LLMs weren't so bloody useful, we'd never touch them. To build successful applications with LLMs, we need new tools to understand both model performance, and the behavior of applications that rely on them. LLM Observability tools that just let you understand how your model is performing are useless: making API calls to an LLM is easy, it's building that into an application that's hard. Pydantic Logfire ---------------- [Pydantic Logfire](https://pydantic.dev/logfire) is an observability platform developed by the team who created and maintain Pydantic and PydanticAI. Logfire aims to let you understand your entire application: Gen AI, classic predictive AI, HTTP traffic, database queries and everything else a modern application needs. Pydantic Logfire is a commercial product Logfire is a commercially supported, hosted platform with an extremely generous and perpetual [free tier](https://pydantic.dev/pricing/) . You can sign up and start using Logfire in a couple of minutes. PydanticAI has built-in (but optional) support for Logfire. That means if the `logfire` package is installed and configured and agent instrumentation is enabled then detailed information about agent runs is sent to Logfire. Otherwise there's virtually no overhead and nothing is sent. Here's an example showing details of running the [Weather Agent](../examples/weather-agent/) in Logfire: [![Weather Agent Logfire](../img/logfire-weather-agent.png)](../img/logfire-weather-agent.png) Using Logfire ------------- To use logfire, you'll need a logfire [account](https://logfire.pydantic.dev) , and logfire installed: pipuv `pip install "pydantic-ai[logfire]"` `uv add "pydantic-ai[logfire]"` Then authenticate your local environment with logfire: pipuv `logfire auth` `uv run logfire auth` And configure a project to send data to: pipuv `logfire projects new` `uv run logfire projects new` (Or use an existing project with `logfire projects use`) Then add logfire to your code: adding\_logfire.py `import logfire logfire.configure()` and enable instrumentation in your agent: instrument\_agent.py ``from pydantic_ai import Agent agent = Agent('openai:gpt-4o', instrument=True) # or instrument all agents to avoid needing to add `instrument=True` to each agent: Agent.instrument_all()`` The [logfire documentation](https://logfire.pydantic.dev/docs/) has more details on how to use logfire, including how to instrument other libraries like [Pydantic](https://logfire.pydantic.dev/docs/integrations/pydantic/) , [HTTPX](https://logfire.pydantic.dev/docs/integrations/http-clients/httpx/) and [FastAPI](https://logfire.pydantic.dev/docs/integrations/web-frameworks/fastapi/) . Since Logfire is built on [OpenTelemetry](https://opentelemetry.io/) , you can use the Logfire Python SDK to send data to any OpenTelemetry collector. Once you have logfire set up, there are two primary ways it can help you understand your application: * **Debugging** — Using the live view to see what's happening in your application in real-time. * **Monitoring** — Using SQL and dashboards to observe the behavior of your application, Logfire is effectively a SQL database that stores information about how your application is running. ### Debugging To demonstrate how Logfire can let you visualise the flow of a PydanticAI run, here's the view you get from Logfire while running the [chat app examples](../examples/chat-app/) : ### Monitoring Performance We can also query data with SQL in Logfire to monitor the performance of an application. Here's a real world example of using Logfire to monitor PydanticAI runs inside Logfire itself: [![Logfire monitoring PydanticAI](../img/logfire-monitoring-pydanticai.png)](../img/logfire-monitoring-pydanticai.png) ### Monitoring HTTPX Requests In order to monitor HTTPX requests made by models, you can use `logfire`'s [HTTPX](https://logfire.pydantic.dev/docs/integrations/http-clients/httpx/) integration. Instrumentation is as easy as adding the following three lines to your application: instrument\_httpx.py `import logfire logfire.configure() logfire.instrument_httpx(capture_all=True) # (1)!` 1. See the [logfire docs](https://logfire.pydantic.dev/docs/integrations/http-clients/httpx/) for more `httpx` instrumentation details. In particular, this can help you to trace specific requests, responses, and headers: instrument\_httpx\_example.py `import logfire from pydantic_ai import Agent logfire.configure() logfire.instrument_httpx(capture_all=True) # (1)! agent = Agent('openai:gpt-4o', instrument=True) result = agent.run_sync('What is the capital of France?') print(result.data) # > The capital of France is Paris.` 1. Capture all of headers, request body, and response body. With `httpx` instrumentationWithout `httpx` instrumentation [![Logfire with HTTPX instrumentation](../img/logfire-with-httpx.png)](../img/logfire-with-httpx.png) [![Logfire without HTTPX instrumentation](../img/logfire-without-httpx.png)](../img/logfire-without-httpx.png) Tip `httpx` instrumentation might be of particular utility if you're using a custom `httpx` client in your model in order to get insights into your custom requests. Using OpenTelemetry ------------------- PydanticAI's instrumentation uses [OpenTelemetry](https://opentelemetry.io/) , which Logfire is based on. You can use the Logfire SDK completely freely and follow the [Alternative backends](https://logfire.pydantic.dev/docs/how-to-guides/alternative-backends/) guide to send the data to any OpenTelemetry collector, such as a self-hosted Jaeger instance. Or you can skip Logfire entirely and use the OpenTelemetry Python SDK directly. Data format ----------- PydanticAI follows the [OpenTelemetry Semantic Conventions for Generative AI systems](https://opentelemetry.io/docs/specs/semconv/gen-ai/) , with one caveat. The semantic conventions specify that messages should be captured as individual events (logs) that are children of the request span. By default, PydanticAI instead collects these events into a JSON array which is set as a single large attribute called `events` on the request span. To change this, use [`InstrumentationSettings(event_mode='logs')`](../api/agent/#pydantic_ai.agent.InstrumentationSettings) . instrumentation\_settings\_event\_mode.py `from pydantic_ai import Agent from pydantic_ai.agent import InstrumentationSettings instrumentation_settings = InstrumentationSettings(event_mode='logs') agent = Agent('openai:gpt-4o', instrument=instrumentation_settings) # or instrument all agents: Agent.instrument_all(instrumentation_settings)` For now, this won't look as good in the Logfire UI, but we're working on it. If you have very long conversations, the `events` span attribute may be truncated. Using `event_mode='logs'` will help avoid this issue. Note that the OpenTelemetry Semantic Conventions are still experimental and are likely to change. Setting OpenTelemetry SDK providers ----------------------------------- By default, the global `TracerProvider` and `EventLoggerProvider` are used. These are set automatically by `logfire.configure()`. They can also be set by the `set_tracer_provider` and `set_event_logger_provider` functions in the OpenTelemetry Python SDK. You can set custom providers with [`InstrumentationSettings`](../api/models/instrumented/#pydantic_ai.models.instrumented.InstrumentationSettings) . instrumentation\_settings\_providers.py `from opentelemetry.sdk._events import EventLoggerProvider from opentelemetry.sdk.trace import TracerProvider from pydantic_ai.agent import InstrumentationSettings instrumentation_settings = InstrumentationSettings( tracer_provider=TracerProvider(), event_logger_provider=EventLoggerProvider(), )` Instrumenting a specific `Model` -------------------------------- instrumented\_model\_example.py `from pydantic_ai import Agent from pydantic_ai.models.instrumented import InstrumentationSettings, InstrumentedModel settings = InstrumentationSettings() model = InstrumentedModel('gpt-4o', settings) agent = Agent(model)` --- # Results - PydanticAI Results ======= Results are the final values returned from [running an agent](../agents/#running-agents) . The result values are wrapped in [`AgentRunResult`](../api/agent/#pydantic_ai.agent.AgentRunResult) and [`StreamedRunResult`](../api/result/#pydantic_ai.result.StreamedRunResult) so you can access other data like [usage](../api/usage/#pydantic_ai.usage.Usage) of the run and [message history](../message-history/#accessing-messages-from-results) Both `RunResult` and `StreamedRunResult` are generic in the data they wrap, so typing information about the data returned by the agent is preserved. olympics.py `from pydantic import BaseModel from pydantic_ai import Agent class CityLocation(BaseModel): city: str country: str agent = Agent('google-gla:gemini-1.5-flash', result_type=CityLocation) result = agent.run_sync('Where were the olympics held in 2012?') print(result.data) #> city='London' country='United Kingdom' print(result.usage()) """ Usage(requests=1, request_tokens=57, response_tokens=8, total_tokens=65, details=None) """` _(This example is complete, it can be run "as is")_ Runs end when either a plain text response is received or the model calls a tool associated with one of the structured result types. We will add limits to make sure a run doesn't go on indefinitely, see [#70](https://github.com/pydantic/pydantic-ai/issues/70) . Result data ----------- When the result type is `str`, or a union including `str`, plain text responses are enabled on the model, and the raw text response from the model is used as the response data. If the result type is a union with multiple members (after remove `str` from the members), each member is registered as a separate tool with the model in order to reduce the complexity of the tool schemas and maximise the chances a model will respond correctly. If the result type schema is not of type `"object"`, the result type is wrapped in a single element object, so the schema of all tools registered with the model are object schemas. Structured results (like tools) use Pydantic to build the JSON schema used for the tool, and to validate the data returned by the model. Bring on PEP-747 Until [PEP-747](https://peps.python.org/pep-0747/) "Annotating Type Forms" lands, unions are not valid as `type`s in Python. When creating the agent we need to `# type: ignore` the `result_type` argument, and add a type hint to tell type checkers about the type of the agent. Here's an example of returning either text or a structured value box\_or\_error.py `from typing import Union from pydantic import BaseModel from pydantic_ai import Agent class Box(BaseModel): width: int height: int depth: int units: str agent: Agent[None, Union[Box, str]] = Agent( 'openai:gpt-4o-mini', result_type=Union[Box, str], # type: ignore system_prompt=( "Extract me the dimensions of a box, " "if you can't extract all data, ask the user to try again." ), ) result = agent.run_sync('The box is 10x20x30') print(result.data) #> Please provide the units for the dimensions (e.g., cm, in, m). result = agent.run_sync('The box is 10x20x30 cm') print(result.data) #> width=10 height=20 depth=30 units='cm'` _(This example is complete, it can be run "as is")_ Here's an example of using a union return type which registered multiple tools, and wraps non-object schemas in an object: colors\_or\_sizes.py `from typing import Union from pydantic_ai import Agent agent: Agent[None, Union[list[str], list[int]]] = Agent( 'openai:gpt-4o-mini', result_type=Union[list[str], list[int]], # type: ignore system_prompt='Extract either colors or sizes from the shapes provided.', ) result = agent.run_sync('red square, blue circle, green triangle') print(result.data) #> ['red', 'blue', 'green'] result = agent.run_sync('square size 10, circle size 20, triangle size 30') print(result.data) #> [10, 20, 30]` _(This example is complete, it can be run "as is")_ ### Result validators functions Some validation is inconvenient or impossible to do in Pydantic validators, in particular when the validation requires IO and is asynchronous. PydanticAI provides a way to add validation functions via the [`agent.result_validator`](../api/agent/#pydantic_ai.agent.Agent.result_validator) decorator. Here's a simplified variant of the [SQL Generation example](../examples/sql-gen/) : sql\_gen.py `from typing import Union from fake_database import DatabaseConn, QueryError from pydantic import BaseModel from pydantic_ai import Agent, RunContext, ModelRetry class Success(BaseModel): sql_query: str class InvalidRequest(BaseModel): error_message: str Response = Union[Success, InvalidRequest] agent: Agent[DatabaseConn, Response] = Agent( 'google-gla:gemini-1.5-flash', result_type=Response, # type: ignore deps_type=DatabaseConn, system_prompt='Generate PostgreSQL flavored SQL queries based on user input.', ) @agent.result_validator async def validate_result(ctx: RunContext[DatabaseConn], result: Response) -> Response: if isinstance(result, InvalidRequest): return result try: await ctx.deps.execute(f'EXPLAIN {result.sql_query}') except QueryError as e: raise ModelRetry(f'Invalid query: {e}') from e else: return result result = agent.run_sync( 'get me users who were last active yesterday.', deps=DatabaseConn() ) print(result.data) #> sql_query='SELECT * FROM users WHERE last_active::date = today() - interval 1 day'` _(This example is complete, it can be run "as is")_ Streamed Results ---------------- There two main challenges with streamed results: 1. Validating structured responses before they're complete, this is achieved by "partial validation" which was recently added to Pydantic in [pydantic/pydantic#10748](https://github.com/pydantic/pydantic/pull/10748) . 2. When receiving a response, we don't know if it's the final response without starting to stream it and peeking at the content. PydanticAI streams just enough of the response to sniff out if it's a tool call or a result, then streams the whole thing and calls tools, or returns the stream as a [`StreamedRunResult`](../api/result/#pydantic_ai.result.StreamedRunResult) . ### Streaming Text Example of streamed text result: streamed\_hello\_world.py `from pydantic_ai import Agent agent = Agent('google-gla:gemini-1.5-flash') # (1)! async def main(): async with agent.run_stream('Where does "hello world" come from?') as result: # (2)! async for message in result.stream_text(): # (3)! print(message) #> The first known #> The first known use of "hello, #> The first known use of "hello, world" was in #> The first known use of "hello, world" was in a 1974 textbook #> The first known use of "hello, world" was in a 1974 textbook about the C #> The first known use of "hello, world" was in a 1974 textbook about the C programming language.` 1. Streaming works with the standard [`Agent`](../api/agent/#pydantic_ai.agent.Agent) class, and doesn't require any special setup, just a model that supports streaming (currently all models support streaming). 2. The [`Agent.run_stream()`](../api/agent/#pydantic_ai.agent.Agent.run_stream) method is used to start a streamed run, this method returns a context manager so the connection can be closed when the stream completes. 3. Each item yield by [`StreamedRunResult.stream_text()`](../api/result/#pydantic_ai.result.StreamedRunResult.stream_text) is the complete text response, extended as new data is received. _(This example is complete, it can be run "as is" — you'll need to add `asyncio.run(main())` to run `main`)_ We can also stream text as deltas rather than the entire text in each item: streamed\_delta\_hello\_world.py `from pydantic_ai import Agent agent = Agent('google-gla:gemini-1.5-flash') async def main(): async with agent.run_stream('Where does "hello world" come from?') as result: async for message in result.stream_text(delta=True): # (1)! print(message) #> The first known #> use of "hello, #> world" was in #> a 1974 textbook #> about the C #> programming language.` 1. [`stream_text`](../api/result/#pydantic_ai.result.StreamedRunResult.stream_text) will error if the response is not text _(This example is complete, it can be run "as is" — you'll need to add `asyncio.run(main())` to run `main`)_ Result message not included in `messages` The final result message will **NOT** be added to result messages if you use `.stream_text(delta=True)`, see [Messages and chat history](../message-history/) for more information. ### Streaming Structured Responses Not all types are supported with partial validation in Pydantic, see [pydantic/pydantic#10748](https://github.com/pydantic/pydantic/pull/10748) , generally for model-like structures it's currently best to use `TypeDict`. Here's an example of streaming a use profile as it's built: streamed\_user\_profile.py `from datetime import date from typing_extensions import TypedDict from pydantic_ai import Agent class UserProfile(TypedDict, total=False): name: str dob: date bio: str agent = Agent( 'openai:gpt-4o', result_type=UserProfile, system_prompt='Extract a user profile from the input', ) async def main(): user_input = 'My name is Ben, I was born on January 28th 1990, I like the chain the dog and the pyramid.' async with agent.run_stream(user_input) as result: async for profile in result.stream(): print(profile) #> {'name': 'Ben'} #> {'name': 'Ben'} #> {'name': 'Ben', 'dob': date(1990, 1, 28), 'bio': 'Likes'} #> {'name': 'Ben', 'dob': date(1990, 1, 28), 'bio': 'Likes the chain the '} #> {'name': 'Ben', 'dob': date(1990, 1, 28), 'bio': 'Likes the chain the dog and the pyr'} #> {'name': 'Ben', 'dob': date(1990, 1, 28), 'bio': 'Likes the chain the dog and the pyramid'} #> {'name': 'Ben', 'dob': date(1990, 1, 28), 'bio': 'Likes the chain the dog and the pyramid'}` _(This example is complete, it can be run "as is" — you'll need to add `asyncio.run(main())` to run `main`)_ If you want fine-grained control of validation, particularly catching validation errors, you can use the following pattern: streamed\_user\_profile.py `from datetime import date from pydantic import ValidationError from typing_extensions import TypedDict from pydantic_ai import Agent class UserProfile(TypedDict, total=False): name: str dob: date bio: str agent = Agent('openai:gpt-4o', result_type=UserProfile) async def main(): user_input = 'My name is Ben, I was born on January 28th 1990, I like the chain the dog and the pyramid.' async with agent.run_stream(user_input) as result: async for message, last in result.stream_structured(debounce_by=0.01): # (1)! try: profile = await result.validate_structured_result( # (2)! message, allow_partial=not last, ) except ValidationError: continue print(profile) #> {'name': 'Ben'} #> {'name': 'Ben'} #> {'name': 'Ben', 'dob': date(1990, 1, 28), 'bio': 'Likes'} #> {'name': 'Ben', 'dob': date(1990, 1, 28), 'bio': 'Likes the chain the '} #> {'name': 'Ben', 'dob': date(1990, 1, 28), 'bio': 'Likes the chain the dog and the pyr'} #> {'name': 'Ben', 'dob': date(1990, 1, 28), 'bio': 'Likes the chain the dog and the pyramid'} #> {'name': 'Ben', 'dob': date(1990, 1, 28), 'bio': 'Likes the chain the dog and the pyramid'}` 1. [`stream_structured`](../api/result/#pydantic_ai.result.StreamedRunResult.stream_structured) streams the data as [`ModelResponse`](../api/messages/#pydantic_ai.messages.ModelResponse) objects, thus iteration can't fail with a `ValidationError`. 2. [`validate_structured_result`](../api/result/#pydantic_ai.result.StreamedRunResult.validate_structured_result) validates the data, `allow_partial=True` enables pydantic's [`experimental_allow_partial` flag on `TypeAdapter`](https://docs.pydantic.dev/latest/api/type_adapter/#pydantic.type_adapter.TypeAdapter.validate_json) . _(This example is complete, it can be run "as is" — you'll need to add `asyncio.run(main())` to run `main`)_ Examples -------- The following examples demonstrate how to use streamed responses in PydanticAI: * [Stream markdown](../examples/stream-markdown/) * [Stream Whales](../examples/stream-whales/) --- # Multi-agent Applications - PydanticAI Multi-agent Applications ======================== There are roughly four levels of complexity when building applications with PydanticAI: 1. Single agent workflows — what most of the `pydantic_ai` documentation covers 2. [Agent delegation](#agent-delegation) — agents using another agent via tools 3. [Programmatic agent hand-off](#programmatic-agent-hand-off) — one agent runs, then application code calls another agent 4. [Graph based control flow](../graph/) — for the most complex cases, a graph-based state machine can be used to control the execution of multiple agents Of course, you can combine multiple strategies in a single application. Agent delegation ---------------- "Agent delegation" refers to the scenario where an agent delegates work to another agent, then takes back control when the delegate agent (the agent called from within a tool) finishes. Since agents are stateless and designed to be global, you do not need to include the agent itself in agent [dependencies](../dependencies/) . You'll generally want to pass [`ctx.usage`](../api/tools/#pydantic_ai.tools.RunContext.usage) to the [`usage`](../api/agent/#pydantic_ai.agent.Agent.run) keyword argument of the delegate agent run so usage within that run counts towards the total usage of the parent agent run. Multiple models Agent delegation doesn't need to use the same model for each agent. If you choose to use different models within a run, calculating the monetary cost from the final [`result.usage()`](../api/agent/#pydantic_ai.agent.AgentRunResult.usage) of the run will not be possible, but you can still use [`UsageLimits`](../api/usage/#pydantic_ai.usage.UsageLimits) to avoid unexpected costs. agent\_delegation\_simple.py ``from pydantic_ai import Agent, RunContext from pydantic_ai.usage import UsageLimits joke_selection_agent = Agent( # (1)! 'openai:gpt-4o', system_prompt=( 'Use the `joke_factory` to generate some jokes, then choose the best. ' 'You must return just a single joke.' ), ) joke_generation_agent = Agent( # (2)! 'google-gla:gemini-1.5-flash', result_type=list[str] ) @joke_selection_agent.tool async def joke_factory(ctx: RunContext[None], count: int) -> list[str]: r = await joke_generation_agent.run( # (3)! f'Please generate {count} jokes.', usage=ctx.usage, # (4)! ) return r.data # (5)! result = joke_selection_agent.run_sync( 'Tell me a joke.', usage_limits=UsageLimits(request_limit=5, total_tokens_limit=300), ) print(result.data) #> Did you hear about the toothpaste scandal? They called it Colgate. print(result.usage()) """ Usage( requests=3, request_tokens=204, response_tokens=24, total_tokens=228, details=None ) """`` 1. The "parent" or controlling agent. 2. The "delegate" agent, which is called from within a tool of the parent agent. 3. Call the delegate agent from within a tool of the parent agent. 4. Pass the usage from the parent agent to the delegate agent so the final [`result.usage()`](../api/agent/#pydantic_ai.agent.AgentRunResult.usage) includes the usage from both agents. 5. Since the function returns `list[str]`, and the `result_type` of `joke_generation_agent` is also `list[str]`, we can simply return `r.data` from the tool. _(This example is complete, it can be run "as is")_ The control flow for this example is pretty simple and can be summarised as follows: graph TD START --> joke_selection_agent joke_selection_agent --> joke_factory["joke_factory (tool)"] joke_factory --> joke_generation_agent joke_generation_agent --> joke_factory joke_factory --> joke_selection_agent joke_selection_agent --> END ### Agent delegation and dependencies Generally the delegate agent needs to either have the same [dependencies](../dependencies/) as the calling agent, or dependencies which are a subset of the calling agent's dependencies. Initializing dependencies We say "generally" above since there's nothing to stop you initializing dependencies within a tool call and therefore using interdependencies in a delegate agent that are not available on the parent, this should often be avoided since it can be significantly slower than reusing connections etc. from the parent agent. agent\_delegation\_deps.py ``from dataclasses import dataclass import httpx from pydantic_ai import Agent, RunContext @dataclass class ClientAndKey: # (1)! http_client: httpx.AsyncClient api_key: str joke_selection_agent = Agent( 'openai:gpt-4o', deps_type=ClientAndKey, # (2)! system_prompt=( 'Use the `joke_factory` tool to generate some jokes on the given subject, ' 'then choose the best. You must return just a single joke.' ), ) joke_generation_agent = Agent( 'gemini-1.5-flash', deps_type=ClientAndKey, # (4)! result_type=list[str], system_prompt=( 'Use the "get_jokes" tool to get some jokes on the given subject, ' 'then extract each joke into a list.' ), ) @joke_selection_agent.tool async def joke_factory(ctx: RunContext[ClientAndKey], count: int) -> list[str]: r = await joke_generation_agent.run( f'Please generate {count} jokes.', deps=ctx.deps, # (3)! usage=ctx.usage, ) return r.data @joke_generation_agent.tool # (5)! async def get_jokes(ctx: RunContext[ClientAndKey], count: int) -> str: response = await ctx.deps.http_client.get( 'https://example.com', params={'count': count}, headers={'Authorization': f'Bearer {ctx.deps.api_key}'}, ) response.raise_for_status() return response.text async def main(): async with httpx.AsyncClient() as client: deps = ClientAndKey(client, 'foobar') result = await joke_selection_agent.run('Tell me a joke.', deps=deps) print(result.data) #> Did you hear about the toothpaste scandal? They called it Colgate. print(result.usage()) # (6)! """ Usage( requests=4, request_tokens=309, response_tokens=32, total_tokens=341, details=None, ) """`` 1. Define a dataclass to hold the client and API key dependencies. 2. Set the `deps_type` of the calling agent — `joke_selection_agent` here. 3. Pass the dependencies to the delegate agent's run method within the tool call. 4. Also set the `deps_type` of the delegate agent — `joke_generation_agent` here. 5. Define a tool on the delegate agent that uses the dependencies to make an HTTP request. 6. Usage now includes 4 requests — 2 from the calling agent and 2 from the delegate agent. _(This example is complete, it can be run "as is" — you'll need to add `asyncio.run(main())` to run `main`)_ This example shows how even a fairly simple agent delegation can lead to a complex control flow: graph TD START --> joke_selection_agent joke_selection_agent --> joke_factory["joke_factory (tool)"] joke_factory --> joke_generation_agent joke_generation_agent --> get_jokes["get_jokes (tool)"] get_jokes --> http_request["HTTP request"] http_request --> get_jokes get_jokes --> joke_generation_agent joke_generation_agent --> joke_factory joke_factory --> joke_selection_agent joke_selection_agent --> END Programmatic agent hand-off --------------------------- "Programmatic agent hand-off" refers to the scenario where multiple agents are called in succession, with application code and/or a human in the loop responsible for deciding which agent to call next. Here agents don't need to use the same deps. Here we show two agents used in succession, the first to find a flight and the second to extract the user's seat preference. programmatic\_handoff.py `from typing import Literal, Union from pydantic import BaseModel, Field from rich.prompt import Prompt from pydantic_ai import Agent, RunContext from pydantic_ai.messages import ModelMessage from pydantic_ai.usage import Usage, UsageLimits class FlightDetails(BaseModel): flight_number: str class Failed(BaseModel): """Unable to find a satisfactory choice.""" flight_search_agent = Agent[None, Union[FlightDetails, Failed]]( # (1)! 'openai:gpt-4o', result_type=Union[FlightDetails, Failed], # type: ignore system_prompt=( 'Use the "flight_search" tool to find a flight ' 'from the given origin to the given destination.' ), ) @flight_search_agent.tool # (2)! async def flight_search( ctx: RunContext[None], origin: str, destination: str ) -> Union[FlightDetails, None]: # in reality, this would call a flight search API or # use a browser to scrape a flight search website return FlightDetails(flight_number='AK456') usage_limits = UsageLimits(request_limit=15) # (3)! async def find_flight(usage: Usage) -> Union[FlightDetails, None]: # (4)! message_history: Union[list[ModelMessage], None] = None for _ in range(3): prompt = Prompt.ask( 'Where would you like to fly from and to?', ) result = await flight_search_agent.run( prompt, message_history=message_history, usage=usage, usage_limits=usage_limits, ) if isinstance(result.data, FlightDetails): return result.data else: message_history = result.all_messages( result_tool_return_content='Please try again.' ) class SeatPreference(BaseModel): row: int = Field(ge=1, le=30) seat: Literal['A', 'B', 'C', 'D', 'E', 'F'] # This agent is responsible for extracting the user's seat selection seat_preference_agent = Agent[None, Union[SeatPreference, Failed]]( # (5)! 'openai:gpt-4o', result_type=Union[SeatPreference, Failed], # type: ignore system_prompt=( "Extract the user's seat preference. " 'Seats A and F are window seats. ' 'Row 1 is the front row and has extra leg room. ' 'Rows 14, and 20 also have extra leg room. ' ), ) async def find_seat(usage: Usage) -> SeatPreference: # (6)! message_history: Union[list[ModelMessage], None] = None while True: answer = Prompt.ask('What seat would you like?') result = await seat_preference_agent.run( answer, message_history=message_history, usage=usage, usage_limits=usage_limits, ) if isinstance(result.data, SeatPreference): return result.data else: print('Could not understand seat preference. Please try again.') message_history = result.all_messages() async def main(): # (7)! usage: Usage = Usage() opt_flight_details = await find_flight(usage) if opt_flight_details is not None: print(f'Flight found: {opt_flight_details.flight_number}') #> Flight found: AK456 seat_preference = await find_seat(usage) print(f'Seat preference: {seat_preference}') #> Seat preference: row=1 seat='A'` 1. Define the first agent, which finds a flight. We use an explicit type annotation until [PEP-747](https://peps.python.org/pep-0747/) lands, see [structured results](../results/#structured-result-validation) . We use a union as the result type so the model can communicate if it's unable to find a satisfactory choice; internally, each member of the union will be registered as a separate tool. 2. Define a tool on the agent to find a flight. In this simple case we could dispense with the tool and just define the agent to return structured data, then search for a flight, but in more complex scenarios the tool would be necessary. 3. Define usage limits for the entire app. 4. Define a function to find a flight, which asks the user for their preferences and then calls the agent to find a flight. 5. As with `flight_search_agent` above, we use an explicit type annotation to define the agent. 6. Define a function to find the user's seat preference, which asks the user for their seat preference and then calls the agent to extract the seat preference. 7. Now that we've put our logic for running each agent into separate functions, our main app becomes very simple. _(This example is complete, it can be run "as is" — you'll need to add `asyncio.run(main())` to run `main`)_ The control flow for this example can be summarised as follows: graph TB START --> ask_user_flight["ask user for flight"] subgraph find_flight flight_search_agent --> ask_user_flight ask_user_flight --> flight_search_agent end flight_search_agent --> ask_user_seat["ask user for seat"] flight_search_agent --> END subgraph find_seat seat_preference_agent --> ask_user_seat ask_user_seat --> seat_preference_agent end seat_preference_agent --> END Pydantic Graphs --------------- See the [graph](../graph/) documentation on when and how to use graphs. Examples -------- The following examples demonstrate how to use dependencies in PydanticAI: * [Flight booking](../examples/flight-booking/) --- # Unit testing - PydanticAI Unit testing ============ Writing unit tests for PydanticAI code is just like unit tests for any other Python code. Because for the most part they're nothing new, we have pretty well established tools and patterns for writing and running these kinds of tests. Unless you're really sure you know better, you'll probably want to follow roughly this strategy: * Use [`pytest`](https://docs.pytest.org/en/stable/) as your test harness * If you find yourself typing out long assertions, use [inline-snapshot](https://15r10nk.github.io/inline-snapshot/latest/) * Similarly, [dirty-equals](https://dirty-equals.helpmanual.io/latest/) can be useful for comparing large data structures * Use [`TestModel`](../api/models/test/#pydantic_ai.models.test.TestModel) or [`FunctionModel`](../api/models/function/#pydantic_ai.models.function.FunctionModel) in place of your actual model to avoid the usage, latency and variability of real LLM calls * Use [`Agent.override`](../api/agent/#pydantic_ai.agent.Agent.override) to replace your model inside your application logic * Set [`ALLOW_MODEL_REQUESTS=False`](../api/models/base/#pydantic_ai.models.ALLOW_MODEL_REQUESTS) globally to block any requests from being made to non-test models accidentally ### Unit testing with `TestModel` The simplest and fastest way to exercise most of your application code is using [`TestModel`](../api/models/test/#pydantic_ai.models.test.TestModel) , this will (by default) call all tools in the agent, then return either plain text or a structured response depending on the return type of the agent. `TestModel` is not magic The "clever" (but not too clever) part of `TestModel` is that it will attempt to generate valid structured data for [function tools](../tools/) and [result types](../results/#structured-result-validation) based on the schema of the registered tools. There's no ML or AI in `TestModel`, it's just plain old procedural Python code that tries to generate data that satisfies the JSON schema of a tool. The resulting data won't look pretty or relevant, but it should pass Pydantic's validation in most cases. If you want something more sophisticated, use [`FunctionModel`](../api/models/function/#pydantic_ai.models.function.FunctionModel) and write your own data generation logic. Let's write unit tests for the following application code: weather\_app.py `import asyncio from datetime import date from pydantic_ai import Agent, RunContext from fake_database import DatabaseConn # (1)! from weather_service import WeatherService # (2)! weather_agent = Agent( 'openai:gpt-4o', deps_type=WeatherService, system_prompt='Providing a weather forecast at the locations the user provides.', ) @weather_agent.tool def weather_forecast( ctx: RunContext[WeatherService], location: str, forecast_date: date ) -> str: if forecast_date < date.today(): # (3)! return ctx.deps.get_historic_weather(location, forecast_date) else: return ctx.deps.get_forecast(location, forecast_date) async def run_weather_forecast( # (4)! user_prompts: list[tuple[str, int]], conn: DatabaseConn ): """Run weather forecast for a list of user prompts and save.""" async with WeatherService() as weather_service: async def run_forecast(prompt: str, user_id: int): result = await weather_agent.run(prompt, deps=weather_service) await conn.store_forecast(user_id, result.data) # run all prompts in parallel await asyncio.gather( *(run_forecast(prompt, user_id) for (prompt, user_id) in user_prompts) )` 1. `DatabaseConn` is a class that holds a database connection 2. `WeatherService` has methods to get weather forecasts and historic data about the weather 3. We need to call a different endpoint depending on whether the date is in the past or the future, you'll see why this nuance is important below 4. This function is the code we want to test, together with the agent it uses Here we have a function that takes a list of `(user_prompt, user_id)` tuples, gets a weather forecast for each prompt, and stores the result in the database. **We want to test this code without having to mock certain objects or modify our code so we can pass test objects in.** Here's how we would write tests using [`TestModel`](../api/models/test/#pydantic_ai.models.test.TestModel) : test\_weather\_app.py `from datetime import timezone import pytest from dirty_equals import IsNow, IsStr from pydantic_ai import models, capture_run_messages from pydantic_ai.models.test import TestModel from pydantic_ai.messages import ( ModelResponse, SystemPromptPart, TextPart, ToolCallPart, ToolReturnPart, UserPromptPart, ModelRequest, ) from fake_database import DatabaseConn from weather_app import run_weather_forecast, weather_agent pytestmark = pytest.mark.anyio # (1)! models.ALLOW_MODEL_REQUESTS = False # (2)! async def test_forecast(): conn = DatabaseConn() user_id = 1 with capture_run_messages() as messages: with weather_agent.override(model=TestModel()): # (3)! prompt = 'What will the weather be like in London on 2024-11-28?' await run_weather_forecast([(prompt, user_id)], conn) # (4)! forecast = await conn.get_forecast(user_id) assert forecast == '{"weather_forecast":"Sunny with a chance of rain"}' # (5)! assert messages == [ # (6)! ModelRequest( parts=[ SystemPromptPart( content='Providing a weather forecast at the locations the user provides.', timestamp=IsNow(tz=timezone.utc), ), UserPromptPart( content='What will the weather be like in London on 2024-11-28?', timestamp=IsNow(tz=timezone.utc), # (7)! ), ] ), ModelResponse( parts=[ ToolCallPart( tool_name='weather_forecast', args={ 'location': 'a', 'forecast_date': '2024-01-01', # (8)! }, tool_call_id=IsStr(), ) ], model_name='test', timestamp=IsNow(tz=timezone.utc), ), ModelRequest( parts=[ ToolReturnPart( tool_name='weather_forecast', content='Sunny with a chance of rain', tool_call_id=IsStr(), timestamp=IsNow(tz=timezone.utc), ), ], ), ModelResponse( parts=[ TextPart( content='{"weather_forecast":"Sunny with a chance of rain"}', ) ], model_name='test', timestamp=IsNow(tz=timezone.utc), ), ]` 1. We're using [anyio](https://anyio.readthedocs.io/en/stable/) to run async tests. 2. This is a safety measure to make sure we don't accidentally make real requests to the LLM while testing, see [`ALLOW_MODEL_REQUESTS`](../api/models/base/#pydantic_ai.models.ALLOW_MODEL_REQUESTS) for more details. 3. We're using [`Agent.override`](../api/agent/#pydantic_ai.agent.Agent.override) to replace the agent's model with [`TestModel`](../api/models/test/#pydantic_ai.models.test.TestModel) , the nice thing about `override` is that we can replace the model inside agent without needing access to the agent `run*` methods call site. 4. Now we call the function we want to test inside the `override` context manager. 5. But default, `TestModel` will return a JSON string summarising the tools calls made, and what was returned. If you wanted to customise the response to something more closely aligned with the domain, you could add [`custom_result_text='Sunny'`](../api/models/test/#pydantic_ai.models.test.TestModel.custom_result_text) when defining `TestModel`. 6. So far we don't actually know which tools were called and with which values, we can use [`capture_run_messages`](../api/agent/#pydantic_ai.agent.capture_run_messages) to inspect messages from the most recent run and assert the exchange between the agent and the model occurred as expected. 7. The [`IsNow`](https://dirty-equals.helpmanual.io/latest/types/datetime/#dirty_equals.IsNow) helper allows us to use declarative asserts even with data which will contain timestamps that change over time. 8. `TestModel` isn't doing anything clever to extract values from the prompt, so these values are hardcoded. ### Unit testing with `FunctionModel` The above tests are a great start, but careful readers will notice that the `WeatherService.get_forecast` is never called since `TestModel` calls `weather_forecast` with a date in the past. To fully exercise `weather_forecast`, we need to use [`FunctionModel`](../api/models/function/#pydantic_ai.models.function.FunctionModel) to customise how the tools is called. Here's an example of using `FunctionModel` to test the `weather_forecast` tool with custom inputs test\_weather\_app2.py `import re import pytest from pydantic_ai import models from pydantic_ai.messages import ( ModelMessage, ModelResponse, TextPart, ToolCallPart, ) from pydantic_ai.models.function import AgentInfo, FunctionModel from fake_database import DatabaseConn from weather_app import run_weather_forecast, weather_agent pytestmark = pytest.mark.anyio models.ALLOW_MODEL_REQUESTS = False def call_weather_forecast( # (1)! messages: list[ModelMessage], info: AgentInfo ) -> ModelResponse: if len(messages) == 1: # first call, call the weather forecast tool user_prompt = messages[0].parts[-1] m = re.search(r'\d{4}-\d{2}-\d{2}', user_prompt.content) assert m is not None args = {'location': 'London', 'forecast_date': m.group()} # (2)! return ModelResponse(parts=[ToolCallPart('weather_forecast', args)]) else: # second call, return the forecast msg = messages[-1].parts[0] assert msg.part_kind == 'tool-return' return ModelResponse(parts=[TextPart(f'The forecast is: {msg.content}')]) async def test_forecast_future(): conn = DatabaseConn() user_id = 1 with weather_agent.override(model=FunctionModel(call_weather_forecast)): # (3)! prompt = 'What will the weather be like in London on 2032-01-01?' await run_weather_forecast([(prompt, user_id)], conn) forecast = await conn.get_forecast(user_id) assert forecast == 'The forecast is: Rainy with a chance of sun'` 1. We define a function `call_weather_forecast` that will be called by `FunctionModel` in place of the LLM, this function has access to the list of [`ModelMessage`](../api/messages/#pydantic_ai.messages.ModelMessage) s that make up the run, and [`AgentInfo`](../api/models/function/#pydantic_ai.models.function.AgentInfo) which contains information about the agent and the function tools and return tools. 2. Our function is slightly intelligent in that it tries to extract a date from the prompt, but just hard codes the location. 3. We use [`FunctionModel`](../api/models/function/#pydantic_ai.models.function.FunctionModel) to replace the agent's model with our custom function. ### Overriding model via pytest fixtures If you're writing lots of tests that all require model to be overridden, you can use [pytest fixtures](https://docs.pytest.org/en/6.2.x/fixture.html) to override the model with [`TestModel`](../api/models/test/#pydantic_ai.models.test.TestModel) or [`FunctionModel`](../api/models/function/#pydantic_ai.models.function.FunctionModel) in a reusable way. Here's an example of a fixture that overrides the model with `TestModel`: tests.py `import pytest from weather_app import weather_agent from pydantic_ai.models.test import TestModel @pytest.fixture def override_weather_agent(): with weather_agent.override(model=TestModel()): yield async def test_forecast(override_weather_agent: None): ... # test code here` --- # Client - PydanticAI Client ====== PydanticAI can act as an [MCP client](https://modelcontextprotocol.io/quickstart/client) , connecting to MCP servers to use their tools. Install ------- You need to either install [`pydantic-ai`](../../install/) , or[`pydantic-ai-slim`](../../install/#slim-install) with the `mcp` optional group: pipuv `pip install "pydantic-ai-slim[mcp]"` `uv add "pydantic-ai-slim[mcp]"` Note MCP integration requires Python 3.10 or higher. Usage ----- PydanticAI comes with two ways to connect to MCP servers: * [`MCPServerHTTP`](../../api/mcp/#pydantic_ai.mcp.MCPServerHTTP) which connects to an MCP server using the [HTTP SSE](https://spec.modelcontextprotocol.io/specification/2024-11-05/basic/transports/#http-with-sse) transport * [`MCPServerStdio`](../../api/mcp/#pydantic_ai.mcp.MCPServerStdio) which runs the server as a subprocess and connects to it using the [stdio](https://spec.modelcontextprotocol.io/specification/2024-11-05/basic/transports/#stdio) transport Examples of both are shown below; [mcp-run-python](../run-python/) is used as the MCP server in both examples. ### SSE Client [`MCPServerHTTP`](../../api/mcp/#pydantic_ai.mcp.MCPServerHTTP) connects over HTTP using the [HTTP + Server Sent Events transport](https://spec.modelcontextprotocol.io/specification/2024-11-05/basic/transports/#http-with-sse) to a server. Note [`MCPServerHTTP`](../../api/mcp/#pydantic_ai.mcp.MCPServerHTTP) requires an MCP server to be running and accepting HTTP connections before calling [`agent.run_mcp_servers()`](../../api/agent/#pydantic_ai.agent.Agent.run_mcp_servers) . Running the server is not managed by PydanticAI. The name "HTTP" is used since this implemented will be adapted in future to use the new [Streamable HTTP](https://github.com/modelcontextprotocol/specification/pull/206) currently in development. Before creating the SSE client, we need to run the server (docs [here](../run-python/) ): terminal (run sse server) `deno run \ -N -R=node_modules -W=node_modules --node-modules-dir=auto \ jsr:@pydantic/mcp-run-python sse` mcp\_sse\_client.py `from pydantic_ai import Agent from pydantic_ai.mcp import MCPServerHTTP server = MCPServerHTTP(url='http://localhost:3001/sse') # (1)! agent = Agent('openai:gpt-4o', mcp_servers=[server]) # (2)! async def main(): async with agent.run_mcp_servers(): # (3)! result = await agent.run('How many days between 2000-01-01 and 2025-03-18?') print(result.data) #> There are 9,208 days between January 1, 2000, and March 18, 2025.` 1. Define the MCP server with the URL used to connect. 2. Create an agent with the MCP server attached. 3. Create a client session to connect to the server. _(This example is complete, it can be run "as is" with Python 3.10+ — you'll need to add `asyncio.run(main())` to run `main`)_ **What's happening here?** * The model is receiving the prompt "how many days between 2000-01-01 and 2025-03-18?" * The model decides "Oh, I've got this `run_python_code` tool, that will be a good way to answer this question", and writes some python code to calculate the answer. * The model returns a tool call * PydanticAI sends the tool call to the MCP server using the SSE transport * The model is called again with the return value of running the code * The model returns the final answer You can visualise this clearly, and even see the code that's run by adding three lines of code to instrument the example with [logfire](https://logfire.pydantic.dev/docs) : mcp\_sse\_client\_logfire.py `import logfire logfire.configure() logfire.instrument_pydantic_ai()` Will display as follows: [![Logfire run python code](../../img/logfire-run-python-code.png)](../../img/logfire-run-python-code.png) ### MCP "stdio" Server The other transport offered by MCP is the [stdio transport](https://spec.modelcontextprotocol.io/specification/2024-11-05/basic/transports/#stdio) where the server is run as a subprocess and communicates with the client over `stdin` and `stdout`. In this case, you'd use the [`MCPServerStdio`](../../api/mcp/#pydantic_ai.mcp.MCPServerStdio) class. Note When using [`MCPServerStdio`](../../api/mcp/#pydantic_ai.mcp.MCPServerStdio) servers, the [`agent.run_mcp_servers()`](../../api/agent/#pydantic_ai.agent.Agent.run_mcp_servers) context manager is responsible for starting and stopping the server. mcp\_stdio\_client.py `from pydantic_ai import Agent from pydantic_ai.mcp import MCPServerStdio server = MCPServerStdio( # (1)! 'deno', args=[ 'run', '-N', '-R=node_modules', '-W=node_modules', '--node-modules-dir=auto', 'jsr:@pydantic/mcp-run-python', 'stdio', ] ) agent = Agent('openai:gpt-4o', mcp_servers=[server]) async def main(): async with agent.run_mcp_servers(): result = await agent.run('How many days between 2000-01-01 and 2025-03-18?') print(result.data) #> There are 9,208 days between January 1, 2000, and March 18, 2025.` 1. See [MCP Run Python](../run-python/) for more information. --- # Evals - PydanticAI Evals ===== "Evals" refers to evaluating a model's performance for a specific application. Warning Unlike unit tests, evals are an emerging art/science; anyone who claims to know for sure exactly how your evals should be defined can safely be ignored. Pydantic Evals is a powerful evaluation framework designed to help you systematically test and evaluate the performance and accuracy of the systems you build, especially when working with LLMs. We've designed Pydantic Evals to be useful while not being too opinionated since we (along with everyone else) are still figuring out best practices. We'd love your [feedback](../help/) on the package and how we can improve it. In Beta Pydantic Evals support was [introduced](https://github.com/pydantic/pydantic-ai/pull/935) in v0.0.47 and is currently in beta. The API is subject to change and the documentation is incomplete. Installation ------------ To install the Pydantic Evals package, run: pipuv `pip install pydantic-evals` `uv add pydantic-evals` `pydantic-evals` does not depend on `pydantic-ai`, but has an optional dependency on `logfire` if you'd like to use OpenTelemetry traces in your evals, or send evaluation results to [logfire](https://pydantic.dev/logfire) . pipuv `pip install 'pydantic-evals[logfire]'` `uv add 'pydantic-evals[logfire]'` Datasets and Cases ------------------ In Pydantic Evals, everything begins with `Dataset`s and `Case`s: * [`Case`](../api/pydantic_evals/dataset/#pydantic_evals.dataset.Case) : A single test scenario corresponding to "task" inputs. Can also optionally have a name, expected outputs, metadata, and evaluators. * [`Dataset`](../api/pydantic_evals/dataset/#pydantic_evals.dataset.Dataset) : A collection of test cases designed for the evaluation of a specific task or function. simple\_eval\_dataset.py `from pydantic_evals import Case, Dataset case1 = Case( name='simple_case', inputs='What is the capital of France?', expected_output='Paris', metadata={'difficulty': 'easy'}, ) dataset = Dataset(cases=[case1])` _(This example is complete, it can be run "as is")_ Evaluators ---------- Evaluators are the components that analyze and score the results of your task when tested against a case. Pydantic Evals includes several built-in evaluators and allows you to create custom evaluators: simple\_eval\_evaluator.py `from dataclasses import dataclass from simple_eval_dataset import dataset from pydantic_evals.evaluators import Evaluator, EvaluatorContext from pydantic_evals.evaluators.common import IsInstance dataset.add_evaluator(IsInstance(type_name='str')) # (1)! @dataclass class MyEvaluator(Evaluator): async def evaluate(self, ctx: EvaluatorContext[str, str]) -> float: # (2)! if ctx.output == ctx.expected_output: return 1.0 elif ( isinstance(ctx.output, str) and ctx.expected_output.lower() in ctx.output.lower() ): return 0.8 else: return 0.0 dataset.add_evaluator(MyEvaluator())` 1. You can add built-in evaluators to a dataset using the [`add_evaluator`](../api/pydantic_evals/dataset/#pydantic_evals.dataset.Dataset.add_evaluator) method. 2. This custom evaluator returns a simple score based on whether the output matches the expected output. _(This example is complete, it can be run "as is")_ Evaluation Process ------------------ The evaluation process involves running a task against all cases in a dataset: Putting the above two examples together and using the more declarative `evaluators` kwarg to [`Dataset`](../api/pydantic_evals/dataset/#pydantic_evals.dataset.Dataset) : simple\_eval\_complete.py `from pydantic_evals import Case, Dataset from pydantic_evals.evaluators import Evaluator, EvaluatorContext, IsInstance case1 = Case( # (1)! name='simple_case', inputs='What is the capital of France?', expected_output='Paris', metadata={'difficulty': 'easy'}, ) class MyEvaluator(Evaluator[str, str]): def evaluate(self, ctx: EvaluatorContext[str, str]) -> float: if ctx.output == ctx.expected_output: return 1.0 elif ( isinstance(ctx.output, str) and ctx.expected_output.lower() in ctx.output.lower() ): return 0.8 else: return 0.0 dataset = Dataset( cases=[case1], evaluators=[IsInstance(type_name='str'), MyEvaluator()], # (3)! ) async def guess_city(question: str) -> str: # (4)! return 'Paris' report = dataset.evaluate_sync(guess_city) # (5)! report.print(include_input=True, include_output=True, include_durations=False) # (6)! """ Evaluation Summary: guess_city ┏━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┓ ┃ Case ID ┃ Inputs ┃ Outputs ┃ Scores ┃ Assertions ┃ ┡━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━┩ │ simple_case │ What is the capital of France? │ Paris │ MyEvaluator: 1.00 │ ✔ │ ├─────────────┼────────────────────────────────┼─────────┼───────────────────┼────────────┤ │ Averages │ │ │ MyEvaluator: 1.00 │ 100.0% ✔ │ └─────────────┴────────────────────────────────┴─────────┴───────────────────┴────────────┘ """` 1. Create a [test case](../api/pydantic_evals/dataset/#pydantic_evals.dataset.Case) as above 2. Also create a custom evaluator function as above 3. Create a [`Dataset`](../api/pydantic_evals/dataset/#pydantic_evals.dataset.Dataset) with test cases, also set the [`evaluators`](../api/pydantic_evals/dataset/#pydantic_evals.dataset.Dataset.evaluators) when creating the dataset 4. Our function to evaluate. 5. Run the evaluation with [`evaluate_sync`](../api/pydantic_evals/dataset/#pydantic_evals.dataset.Dataset.evaluate_sync) , which runs the function against all test cases in the dataset, and returns an [`EvaluationReport`](../api/pydantic_evals/reporting/#pydantic_evals.reporting.EvaluationReport) object. 6. Print the report with [`print`](../api/pydantic_evals/reporting/#pydantic_evals.reporting.EvaluationReport.print) , which shows the results of the evaluation, including input and output. We have omitted duration here just to keep the printed output from changing from run to run. _(This example is complete, it can be run "as is")_ Evaluation with `LLMJudge` -------------------------- In this example we evaluate a method for generating recipes based on customer orders. judge\_recipes.py `from __future__ import annotations from typing import Any from pydantic import BaseModel from pydantic_ai import Agent from pydantic_ai.format_as_xml import format_as_xml from pydantic_evals import Case, Dataset from pydantic_evals.evaluators import IsInstance, LLMJudge class CustomerOrder(BaseModel): # (1)! dish_name: str dietary_restriction: str | None = None class Recipe(BaseModel): ingredients: list[str] steps: list[str] recipe_agent = Agent( 'groq:llama-3.3-70b-versatile', result_type=Recipe, system_prompt=( 'Generate a recipe to cook the dish that meets the dietary restrictions.' ), ) async def transform_recipe(customer_order: CustomerOrder) -> Recipe: # (2)! r = await recipe_agent.run(format_as_xml(customer_order)) return r.data recipe_dataset = Dataset[CustomerOrder, Recipe, Any]( # (3)! cases=[ Case( name='vegetarian_recipe', inputs=CustomerOrder( dish_name='Spaghetti Bolognese', dietary_restriction='vegetarian' ), expected_output=None, # (4) metadata={'focus': 'vegetarian'}, evaluators=( LLMJudge( # (5)! rubric='Recipe should not contain meat or animal products', ), ), ), Case( name='gluten_free_recipe', inputs=CustomerOrder( dish_name='Chocolate Cake', dietary_restriction='gluten-free' ), expected_output=None, metadata={'focus': 'gluten-free'}, # Case-specific evaluator with a focused rubric evaluators=( LLMJudge( rubric='Recipe should not contain gluten or wheat products', ), ), ), ], evaluators=[ # (6)! IsInstance(type_name='Recipe'), LLMJudge( rubric='Recipe should have clear steps and relevant ingredients', include_input=True, model='anthropic:claude-3-7-sonnet-latest', # (7)! ), ], ) report = recipe_dataset.evaluate_sync(transform_recipe) print(report) """ Evaluation Summary: transform_recipe ┏━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━━┓ ┃ Case ID ┃ Assertions ┃ Duration ┃ ┡━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━━┩ │ vegetarian_recipe │ ✔✔✔ │ 10ms │ ├────────────────────┼────────────┼──────────┤ │ gluten_free_recipe │ ✔✔✔ │ 10ms │ ├────────────────────┼────────────┼──────────┤ │ Averages │ 100.0% ✔ │ 10ms │ └────────────────────┴────────────┴──────────┘ """` 1. Define models for our task — Input for recipe generation task and output of the task. 2. Define our recipe generation function - this is the task we want to evaluate. 3. Create a dataset with different test cases and different rubrics. 4. No expected output, we'll let the LLM judge the quality. 5. Case-specific evaluator with a focused rubric using [`LLMJudge`](../api/pydantic_evals/evaluators/#pydantic_evals.evaluators.LLMJudge) . 6. Dataset-level evaluators that apply to all cases, including a general quality rubric for all recipes 7. By default `LLMJudge` uses `openai:gpt-4o`, here we use a specific Anthropic model. _(This example is complete, it can be run "as is")_ Saving and Loading Datasets --------------------------- Datasets can be saved to and loaded from YAML or JSON files. save\_load\_dataset\_example.py `from pathlib import Path from judge_recipes import CustomerOrder, Recipe, recipe_dataset from pydantic_evals import Dataset recipe_transforms_file = Path('recipe_transform_tests.yaml') recipe_dataset.to_file(recipe_transforms_file) # (1)! print(recipe_transforms_file.read_text()) """ # yaml-language-server: $schema=recipe_transform_tests_schema.json cases: - name: vegetarian_recipe inputs: dish_name: Spaghetti Bolognese dietary_restriction: vegetarian metadata: focus: vegetarian evaluators: - LLMJudge: Recipe should not contain meat or animal products - name: gluten_free_recipe inputs: dish_name: Chocolate Cake dietary_restriction: gluten-free metadata: focus: gluten-free evaluators: - LLMJudge: Recipe should not contain gluten or wheat products evaluators: - IsInstance: Recipe - LLMJudge: rubric: Recipe should have clear steps and relevant ingredients model: anthropic:claude-3-7-sonnet-latest include_input: true """ # Load dataset from file loaded_dataset = Dataset[CustomerOrder, Recipe, dict].from_file(recipe_transforms_file) print(f'Loaded dataset with {len(loaded_dataset.cases)} cases') #> Loaded dataset with 2 cases` _(This example is complete, it can be run "as is")_ Parallel Evaluation ------------------- You can control concurrency during evaluation (this might be useful to prevent exceeding a rate limit): parallel\_evaluation\_example.py `import asyncio import time from pydantic_evals import Case, Dataset # Create a dataset with multiple test cases dataset = Dataset( cases=[ Case( name=f'case_{i}', inputs=i, expected_output=i * 2, ) for i in range(5) ] ) async def double_number(input_value: int) -> int: """Function that simulates work by sleeping for a second before returning double the input.""" await asyncio.sleep(0.1) # Simulate work return input_value * 2 # Run evaluation with unlimited concurrency t0 = time.time() report_default = dataset.evaluate_sync(double_number) print(f'Evaluation took less than 0.3s: {time.time() - t0 < 0.3}') #> Evaluation took less than 0.3s: True report_default.print(include_input=True, include_output=True, include_durations=False) # (1)! """ Evaluation Summary: double_number ┏━━━━━━━━━━┳━━━━━━━━┳━━━━━━━━━┓ ┃ Case ID ┃ Inputs ┃ Outputs ┃ ┡━━━━━━━━━━╇━━━━━━━━╇━━━━━━━━━┩ │ case_0 │ 0 │ 0 │ ├──────────┼────────┼─────────┤ │ case_1 │ 1 │ 2 │ ├──────────┼────────┼─────────┤ │ case_2 │ 2 │ 4 │ ├──────────┼────────┼─────────┤ │ case_3 │ 3 │ 6 │ ├──────────┼────────┼─────────┤ │ case_4 │ 4 │ 8 │ ├──────────┼────────┼─────────┤ │ Averages │ │ │ └──────────┴────────┴─────────┘ """ # Run evaluation with limited concurrency t0 = time.time() report_limited = dataset.evaluate_sync(double_number, max_concurrency=1) print(f'Evaluation took more than 0.5s: {time.time() - t0 > 0.5}') #> Evaluation took more than 0.5s: True report_limited.print(include_input=True, include_output=True, include_durations=False) # (2)! """ Evaluation Summary: double_number ┏━━━━━━━━━━┳━━━━━━━━┳━━━━━━━━━┓ ┃ Case ID ┃ Inputs ┃ Outputs ┃ ┡━━━━━━━━━━╇━━━━━━━━╇━━━━━━━━━┩ │ case_0 │ 0 │ 0 │ ├──────────┼────────┼─────────┤ │ case_1 │ 1 │ 2 │ ├──────────┼────────┼─────────┤ │ case_2 │ 2 │ 4 │ ├──────────┼────────┼─────────┤ │ case_3 │ 3 │ 6 │ ├──────────┼────────┼─────────┤ │ case_4 │ 4 │ 8 │ ├──────────┼────────┼─────────┤ │ Averages │ │ │ └──────────┴────────┴─────────┘ """` 1. We have omitted duration here just to keep the printed output from changing from run to run. 2. We have omitted duration here just to keep the printed output from changing from run to run. _(This example is complete, it can be run "as is")_ OpenTelemetry Integration ------------------------- Pydantic Evals integrates with OpenTelemetry for tracing. The [`EvaluatorContext`](../api/pydantic_evals/evaluators/#pydantic_evals.evaluators.EvaluatorContext) includes a property called `span_tree` which returns a [`SpanTree`](../api/pydantic_evals/otel/#pydantic_evals.otel.SpanTree) . The `SpanTree` provides a way to query and analyze the spans generated during function execution. This provides a way to access the results of instrumentation during evaluation. Note If you just want to write unit tests that ensure that specific spans are produced during calls to your evaluation task, it's usually better to just use the `logfire.testing.capfire` fixture directly. There are two main ways this is useful. opentelemetry\_example.py `import asyncio from typing import Any import logfire from pydantic_evals import Case, Dataset from pydantic_evals.evaluators import Evaluator from pydantic_evals.evaluators.context import EvaluatorContext from pydantic_evals.otel.span_tree import SpanQuery logfire.configure( # ensure that an OpenTelemetry tracer is configured send_to_logfire='if-token-present' ) class SpanTracingEvaluator(Evaluator[str, str]): """Evaluator that analyzes the span tree generated during function execution.""" def evaluate(self, ctx: EvaluatorContext[str, str]) -> dict[str, Any]: # Get the span tree from the context span_tree = ctx.span_tree if span_tree is None: return {'has_spans': False, 'performance_score': 0.0} # Find all spans with "processing" in the name processing_spans = span_tree.find(lambda node: 'processing' in node.name) # Calculate total processing time total_processing_time = sum( (span.duration.total_seconds() for span in processing_spans), 0.0 ) # Check for error spans error_query: SpanQuery = {'name_contains': 'error'} has_errors = span_tree.any(error_query) # Calculate a performance score (lower is better) performance_score = 1.0 if total_processing_time < 0.5 else 0.5 return { 'has_spans': True, 'has_errors': has_errors, 'performance_score': 0 if has_errors else performance_score, } async def process_text(text: str) -> str: """Function that processes text with OpenTelemetry instrumentation.""" with logfire.span('process_text'): # Simulate initial processing with logfire.span('text_processing'): await asyncio.sleep(0.1) processed = text.strip().lower() # Simulate additional processing with logfire.span('additional_processing'): if 'error' in processed: with logfire.span('error_handling'): logfire.error(f'Error detected in text: {text}') return f'Error processing: {text}' await asyncio.sleep(0.2) processed = processed.replace(' ', '_') return f'Processed: {processed}' # Create test cases dataset = Dataset( cases=[ Case( name='normal_text', inputs='Hello World', expected_output='Processed: hello_world', ), Case( name='text_with_error', inputs='Contains error marker', expected_output='Error processing: Contains error marker', ), ], evaluators=[SpanTracingEvaluator()], ) # Run evaluation - spans are automatically captured since logfire is configured report = dataset.evaluate_sync(process_text) # Print the report report.print(include_input=True, include_output=True, include_durations=False) # (1)! """ Evaluation Summary: process_text ┏━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┓ ┃ Case ID ┃ Inputs ┃ Outputs ┃ Scores ┃ Assertions ┃ ┡━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━┩ │ normal_text │ Hello World │ Processed: hello_world │ performance_score: 1.00 │ ✔✗ │ ├─────────────────┼───────────────────────┼─────────────────────────────────────────┼──────────────────────────┼────────────┤ │ text_with_error │ Contains error marker │ Error processing: Contains error marker │ performance_score: 0 │ ✔✔ │ ├─────────────────┼───────────────────────┼─────────────────────────────────────────┼──────────────────────────┼────────────┤ │ Averages │ │ │ performance_score: 0.500 │ 75.0% ✔ │ └─────────────────┴───────────────────────┴─────────────────────────────────────────┴──────────────────────────┴────────────┘ """` 1. We have omitted duration here just to keep the printed output from changing from run to run. _(This example is complete, it can be run "as is")_ Generating Test Datasets ------------------------ Pydantic Evals allows you to generate test datasets using LLMs with [`generate_dataset`](../api/pydantic_evals/generation/#pydantic_evals.generation.generate_dataset) . Datasets can be generated in either JSON or YAML format, in both cases a JSON schema file is generated alongside the dataset and referenced in the dataset, so you should get type checking and auto-completion in your editor. generate\_dataset\_example.py `from __future__ import annotations from pathlib import Path from pydantic import BaseModel, Field from pydantic_evals import Dataset from pydantic_evals.generation import generate_dataset class QuestionInputs(BaseModel, use_attribute_docstrings=True): # (1)! """Model for question inputs.""" question: str """A question to answer""" context: str | None = None """Optional context for the question""" class AnswerOutput(BaseModel, use_attribute_docstrings=True): # (2)! """Model for expected answer outputs.""" answer: str """The answer to the question""" confidence: float = Field(ge=0, le=1) """Confidence level (0-1)""" class MetadataType(BaseModel, use_attribute_docstrings=True): # (3)! """Metadata model for test cases.""" difficulty: str """Difficulty level (easy, medium, hard)""" category: str """Question category""" async def main(): dataset = await generate_dataset( # (4)! dataset_type=Dataset[QuestionInputs, AnswerOutput, MetadataType], n_examples=2, extra_instructions=""" Generate question-answer pairs about world capitals and landmarks. Make sure to include both easy and challenging questions. """, ) output_file = Path('questions_cases.yaml') dataset.to_file(output_file) # (5)! print(output_file.read_text()) """ # yaml-language-server: $schema=questions_cases_schema.json cases: - name: Easy Capital Question inputs: question: What is the capital of France? metadata: difficulty: easy category: Geography expected_output: answer: Paris confidence: 0.95 evaluators: - EqualsExpected - name: Challenging Landmark Question inputs: question: Which world-famous landmark is located on the banks of the Seine River? metadata: difficulty: hard category: Landmarks expected_output: answer: Eiffel Tower confidence: 0.9 evaluators: - EqualsExpected """` 1. Define the schema for the inputs to the task. 2. Define the schema for the expected outputs of the task. 3. Define the schema for the metadata of the test cases. 4. Call [`generate_dataset`](../api/pydantic_evals/generation/#pydantic_evals.generation.generate_dataset) to create a [`Dataset`](../api/pydantic_evals/dataset/#pydantic_evals.dataset.Dataset) with 2 cases confirming to the schema. 5. Save the dataset to a YAML file, this will also write `questions_cases_schema.json` with the schema JSON schema for `questions_cases.yaml` to make editing easier. The magic `yaml-language-server` comment is supported by at least vscode, jetbrains/pycharm (more details [here](https://github.com/redhat-developer/yaml-language-server#using-inlined-schema) ). _(This example is complete, it can be run "as is" — you'll need to add `asyncio.run(main(answer))` to run `main`)_ You can also write datasets as JSON files: generate\_dataset\_example\_json.py `from pathlib import Path from generate_dataset_example import AnswerOutput, MetadataType, QuestionInputs from pydantic_evals import Dataset from pydantic_evals.generation import generate_dataset async def main(): dataset = await generate_dataset( # (1)! dataset_type=Dataset[QuestionInputs, AnswerOutput, MetadataType], n_examples=2, extra_instructions=""" Generate question-answer pairs about world capitals and landmarks. Make sure to include both easy and challenging questions. """, ) output_file = Path('questions_cases.json') dataset.to_file(output_file) # (2)! print(output_file.read_text()) """ { "$schema": "questions_cases_schema.json", "cases": [ { "name": "Easy Capital Question", "inputs": { "question": "What is the capital of France?" }, "metadata": { "difficulty": "easy", "category": "Geography" }, "expected_output": { "answer": "Paris", "confidence": 0.95 }, "evaluators": [ "EqualsExpected" ] }, { "name": "Challenging Landmark Question", "inputs": { "question": "Which world-famous landmark is located on the banks of the Seine River?" }, "metadata": { "difficulty": "hard", "category": "Landmarks" }, "expected_output": { "answer": "Eiffel Tower", "confidence": 0.9 }, "evaluators": [ "EqualsExpected" ] } ] } """` 1. Generate the [`Dataset`](../api/pydantic_evals/dataset/#pydantic_evals.dataset.Dataset) exactly as above. 2. Save the dataset to a JSON file, this will also write `questions_cases_schema.json` with th JSON schema for `questions_cases.json`. This time the `$schema` key is included in the JSON file to define the schema for IDEs to use while you edit the file, there's no formal spec for this, but it works in vscode and pycharm and is discussed at length in [json-schema-org/json-schema-spec#828](https://github.com/json-schema-org/json-schema-spec/issues/828) . _(This example is complete, it can be run "as is" — you'll need to add `asyncio.run(main(answer))` to run `main`)_ Integration with Logfire ------------------------ Pydantic Evals is implemented using OpenTelemetry to record traces of the evaluation process. These traces contain all the information included in the terminal output as attributes, but also include full tracing from the executions of the evaluation task function. You can send these traces to any OpenTelemetry-compatible backend, including [Pydantic Logfire](https://logfire.pydantic.dev/docs) . All you need to do is configure Logfire via `logfire.configure`: logfire\_integration.py `import logfire from judge_recipes import recipe_dataset, transform_recipe logfire.configure( send_to_logfire='if-token-present', # (1)! environment='development', # (2)! service_name='evals', # (3)! ) recipe_dataset.evaluate_sync(transform_recipe)` 1. The `send_to_logfire` argument controls when traces are sent to Logfire. You can set it to `'if-token-present'` to send data to Logfire only if the `LOGFIRE_TOKEN` environment variable is set. See the [Logfire configuration docs](https://logfire.pydantic.dev/docs/reference/configuration/) for more details. 2. The `environment` argument sets the environment for the traces. It's a good idea to set this to `'development'` when running tests or evaluations and sending data to a project with production data, to make it easier to filter these traces out while reviewing data from your production environment(s). 3. The `service_name` argument sets the service name for the traces. This is displayed in the Logfire UI to help you identify the source of the associated spans. Logfire has some special integration with Pydantic Evals traces, including a table view of the evaluation results on the evaluation root span (which is generated in each call to [`Dataset.evaluate`](../api/pydantic_evals/dataset/#pydantic_evals.dataset.Dataset.evaluate) ): [![Logfire Evals Overview](../img/logfire-evals-overview.png)](../img/logfire-evals-overview.png) and a detailed view of the inputs and outputs for the execution of each case: [![Logfire Evals Case](../img/logfire-evals-case.png)](../img/logfire-evals-case.png) In addition, any OpenTelemetry spans generated during the evaluation process will be sent to Logfire, allowing you to visualize the full execution of the code called during the evaluation process: [![Logfire Evals Case Trace](../img/logfire-evals-case-trace.png)](../img/logfire-evals-case-trace.png) This can be especially helpful when attempting to write evaluators that make use of the `span_tree` property of the [`EvaluatorContext`](../api/pydantic_evals/evaluators/#pydantic_evals.evaluators.EvaluatorContext) , as described in the [OpenTelemetry Integration](#opentelemetry-integration) section above. This allows you to write evaluations that depend on information about which code paths were executed during the call to the task function without needing to manually instrument the code being evaluated, as long as the code being evaluated is already adequately instrumented with OpenTelemetry. In the case of PydanticAI agents, for example, this can be used to ensure specific tools are (or are not) called during the execution of specific cases. Using OpenTelemetry in this way also means that all data used to evaluate the task executions will be accessible in the traces produced by production runs of the code, making it straightforward to perform the same evaluations on production data. --- # Graphs - PydanticAI Graphs ====== Don't use a nail gun unless you need a nail gun If PydanticAI [agents](../agents/) are a hammer, and [multi-agent workflows](../multi-agent-applications/) are a sledgehammer, then graphs are a nail gun: * sure, nail guns look cooler than hammers * but nail guns take a lot more setup than hammers * and nail guns don't make you a better builder, they make you a builder with a nail gun * Lastly, (and at the risk of torturing this metaphor), if you're a fan of medieval tools like mallets and untyped Python, you probably won't like nail guns or our approach to graphs. (But then again, if you're not a fan of type hints in Python, you've probably already bounced off PydanticAI to use one of the toy agent frameworks — good luck, and feel free to borrow my sledgehammer when you realize you need it) In short, graphs are a powerful tool, but they're not the right tool for every job. Please consider other [multi-agent approaches](../multi-agent-applications/) before proceeding. If you're not confident a graph-based approach is a good idea, it might be unnecessary. Graphs and finite state machines (FSMs) are a powerful abstraction to model, execute, control and visualize complex workflows. Alongside PydanticAI, we've developed `pydantic-graph` — an async graph and state machine library for Python where nodes and edges are defined using type hints. While this library is developed as part of PydanticAI; it has no dependency on `pydantic-ai` and can be considered as a pure graph-based state machine library. You may find it useful whether or not you're using PydanticAI or even building with GenAI. `pydantic-graph` is designed for advanced users and makes heavy use of Python generics and type hints. It is not designed to be as beginner-friendly as PydanticAI. Installation ------------ `pydantic-graph` is a required dependency of `pydantic-ai`, and an optional dependency of `pydantic-ai-slim`, see [installation instructions](../install/#slim-install) for more information. You can also install it directly: pipuv `pip install pydantic-graph` `uv add pydantic-graph` Graph Types ----------- `pydantic-graph` is made up of a few key components: ### GraphRunContext [`GraphRunContext`](../api/pydantic_graph/nodes/#pydantic_graph.nodes.GraphRunContext) — The context for the graph run, similar to PydanticAI's [`RunContext`](../api/tools/#pydantic_ai.tools.RunContext) . This holds the state of the graph and dependencies and is passed to nodes when they're run. `GraphRunContext` is generic in the state type of the graph it's used in, [`StateT`](../api/pydantic_graph/nodes/#pydantic_graph.nodes.StateT) . ### End [`End`](../api/pydantic_graph/nodes/#pydantic_graph.nodes.End) — return value to indicate the graph run should end. `End` is generic in the graph return type of the graph it's used in, [`RunEndT`](../api/pydantic_graph/nodes/#pydantic_graph.nodes.RunEndT) . ### Nodes Subclasses of [`BaseNode`](../api/pydantic_graph/nodes/#pydantic_graph.nodes.BaseNode) define nodes for execution in the graph. Nodes, which are generally [`dataclass`es](https://docs.python.org/3/library/dataclasses.html#dataclasses.dataclass) , generally consist of: * fields containing any parameters required/optional when calling the node * the business logic to execute the node, in the [`run`](../api/pydantic_graph/nodes/#pydantic_graph.nodes.BaseNode.run) method * return annotations of the [`run`](../api/pydantic_graph/nodes/#pydantic_graph.nodes.BaseNode.run) method, which are read by `pydantic-graph` to determine the outgoing edges of the node Nodes are generic in: * **state**, which must have the same type as the state of graphs they're included in, [`StateT`](../api/pydantic_graph/nodes/#pydantic_graph.nodes.StateT) has a default of `None`, so if you're not using state you can omit this generic parameter, see [stateful graphs](#stateful-graphs) for more information * **deps**, which must have the same type as the deps of the graph they're included in, [`DepsT`](../api/pydantic_graph/nodes/#pydantic_graph.nodes.DepsT) has a default of `None`, so if you're not using deps you can omit this generic parameter, see [dependency injection](#dependency-injection) for more information * **graph return type** — this only applies if the node returns [`End`](../api/pydantic_graph/nodes/#pydantic_graph.nodes.End) . [`RunEndT`](../api/pydantic_graph/nodes/#pydantic_graph.nodes.RunEndT) has a default of [Never](https://docs.python.org/3/library/typing.html#typing.Never) so this generic parameter can be omitted if the node doesn't return `End`, but must be included if it does. Here's an example of a start or intermediate node in a graph — it can't end the run as it doesn't return [`End`](../api/pydantic_graph/nodes/#pydantic_graph.nodes.End) : intermediate\_node.py `from dataclasses import dataclass from pydantic_graph import BaseNode, GraphRunContext @dataclass class MyNode(BaseNode[MyState]): # (1)! foo: int # (2)! async def run( self, ctx: GraphRunContext[MyState], # (3)! ) -> AnotherNode: # (4)! ... return AnotherNode()` 1. State in this example is `MyState` (not shown), hence `BaseNode` is parameterized with `MyState`. This node can't end the run, so the `RunEndT` generic parameter is omitted and defaults to `Never`. 2. `MyNode` is a dataclass and has a single field `foo`, an `int`. 3. The `run` method takes a `GraphRunContext` parameter, again parameterized with state `MyState`. 4. The return type of the `run` method is `AnotherNode` (not shown), this is used to determine the outgoing edges of the node. We could extend `MyNode` to optionally end the run if `foo` is divisible by 5: intermediate\_or\_end\_node.py `from dataclasses import dataclass from pydantic_graph import BaseNode, End, GraphRunContext @dataclass class MyNode(BaseNode[MyState, None, int]): # (1)! foo: int async def run( self, ctx: GraphRunContext[MyState], ) -> AnotherNode | End[int]: # (2)! if self.foo % 5 == 0: return End(self.foo) else: return AnotherNode()` 1. We parameterize the node with the return type (`int` in this case) as well as state. Because generic parameters are positional-only, we have to include `None` as the second parameter representing deps. 2. The return type of the `run` method is now a union of `AnotherNode` and `End[int]`, this allows the node to end the run if `foo` is divisible by 5. ### Graph [`Graph`](../api/pydantic_graph/graph/#pydantic_graph.graph.Graph) — this is the execution graph itself, made up of a set of [node classes](#nodes) (i.e., `BaseNode` subclasses). `Graph` is generic in: * **state** the state type of the graph, [`StateT`](../api/pydantic_graph/nodes/#pydantic_graph.nodes.StateT) * **deps** the deps type of the graph, [`DepsT`](../api/pydantic_graph/nodes/#pydantic_graph.nodes.DepsT) * **graph return type** the return type of the graph run, [`RunEndT`](../api/pydantic_graph/nodes/#pydantic_graph.nodes.RunEndT) Here's an example of a simple graph: graph\_example.py `from __future__ import annotations from dataclasses import dataclass from pydantic_graph import BaseNode, End, Graph, GraphRunContext @dataclass class DivisibleBy5(BaseNode[None, None, int]): # (1)! foo: int async def run( self, ctx: GraphRunContext, ) -> Increment | End[int]: if self.foo % 5 == 0: return End(self.foo) else: return Increment(self.foo) @dataclass class Increment(BaseNode): # (2)! foo: int async def run(self, ctx: GraphRunContext) -> DivisibleBy5: return DivisibleBy5(self.foo + 1) fives_graph = Graph(nodes=[DivisibleBy5, Increment]) # (3)! result = fives_graph.run_sync(DivisibleBy5(4)) # (4)! print(result.output) #> 5` 1. The `DivisibleBy5` node is parameterized with `None` for the state param and `None` for the deps param as this graph doesn't use state or deps, and `int` as it can end the run. 2. The `Increment` node doesn't return `End`, so the `RunEndT` generic parameter is omitted, state can also be omitted as the graph doesn't use state. 3. The graph is created with a sequence of nodes. 4. The graph is run synchronously with [`run_sync`](../api/pydantic_graph/graph/#pydantic_graph.graph.Graph.run_sync) . The initial node is `DivisibleBy5(4)`. Because the graph doesn't use external state or deps, we don't pass `state` or `deps`. _(This example is complete, it can be run "as is" with Python 3.10+)_ A [mermaid diagram](#mermaid-diagrams) for this graph can be generated with the following code: graph\_example\_diagram.py `from graph_example import DivisibleBy5, fives_graph fives_graph.mermaid_code(start_node=DivisibleBy5)` --- title: fives_graph --- stateDiagram-v2 [*] --> DivisibleBy5 DivisibleBy5 --> Increment DivisibleBy5 --> [*] Increment --> DivisibleBy5 In order to visualize a graph within a `jupyter-notebook`, `IPython.display` needs to be used: jupyter\_display\_mermaid.py `from graph_example import DivisibleBy5, fives_graph from IPython.display import Image, display display(Image(fives_graph.mermaid_image(start_node=DivisibleBy5)))` Stateful Graphs --------------- The "state" concept in `pydantic-graph` provides an optional way to access and mutate an object (often a `dataclass` or Pydantic model) as nodes run in a graph. If you think of Graphs as a production line, then your state is the engine being passed along the line and built up by each node as the graph is run. In the future, we intend to extend `pydantic-graph` to provide state persistence with the state recorded after each node is run, see [#695](https://github.com/pydantic/pydantic-ai/issues/695) . Here's an example of a graph which represents a vending machine where the user may insert coins and select a product to purchase. vending\_machine.py `from __future__ import annotations from dataclasses import dataclass from rich.prompt import Prompt from pydantic_graph import BaseNode, End, Graph, GraphRunContext @dataclass class MachineState: # (1)! user_balance: float = 0.0 product: str | None = None @dataclass class InsertCoin(BaseNode[MachineState]): # (3)! async def run(self, ctx: GraphRunContext[MachineState]) -> CoinsInserted: # (16)! return CoinsInserted(float(Prompt.ask('Insert coins'))) # (4)! @dataclass class CoinsInserted(BaseNode[MachineState]): amount: float # (5)! async def run( self, ctx: GraphRunContext[MachineState] ) -> SelectProduct | Purchase: # (17)! ctx.state.user_balance += self.amount # (6)! if ctx.state.product is not None: # (7)! return Purchase(ctx.state.product) else: return SelectProduct() @dataclass class SelectProduct(BaseNode[MachineState]): async def run(self, ctx: GraphRunContext[MachineState]) -> Purchase: return Purchase(Prompt.ask('Select product')) PRODUCT_PRICES = { # (2)! 'water': 1.25, 'soda': 1.50, 'crisps': 1.75, 'chocolate': 2.00, } @dataclass class Purchase(BaseNode[MachineState, None, None]): # (18)! product: str async def run( self, ctx: GraphRunContext[MachineState] ) -> End | InsertCoin | SelectProduct: if price := PRODUCT_PRICES.get(self.product): # (8)! ctx.state.product = self.product # (9)! if ctx.state.user_balance >= price: # (10)! ctx.state.user_balance -= price return End(None) else: diff = price - ctx.state.user_balance print(f'Not enough money for {self.product}, need {diff:0.2f} more') #> Not enough money for crisps, need 0.75 more return InsertCoin() # (11)! else: print(f'No such product: {self.product}, try again') return SelectProduct() # (12)! vending_machine_graph = Graph( # (13)! nodes=[InsertCoin, CoinsInserted, SelectProduct, Purchase] ) async def main(): state = MachineState() # (14)! await vending_machine_graph.run(InsertCoin(), state=state) # (15)! print(f'purchase successful item={state.product} change={state.user_balance:0.2f}') #> purchase successful item=crisps change=0.25` 1. The state of the vending machine is defined as a dataclass with the user's balance and the product they've selected, if any. 2. A dictionary of products mapped to prices. 3. The `InsertCoin` node, [`BaseNode`](../api/pydantic_graph/nodes/#pydantic_graph.nodes.BaseNode) is parameterized with `MachineState` as that's the state used in this graph. 4. The `InsertCoin` node prompts the user to insert coins. We keep things simple by just entering a monetary amount as a float. Before you start thinking this is a toy too since it's using [rich's `Prompt.ask`](https://rich.readthedocs.io/en/stable/reference/prompt.html#rich.prompt.PromptBase.ask) within nodes, see [below](#example-human-in-the-loop) for how control flow can be managed when nodes require external input. 5. The `CoinsInserted` node; again this is a [`dataclass`](https://docs.python.org/3/library/dataclasses.html#dataclasses.dataclass) with one field `amount`. 6. Update the user's balance with the amount inserted. 7. If the user has already selected a product, go to `Purchase`, otherwise go to `SelectProduct`. 8. In the `Purchase` node, look up the price of the product if the user entered a valid product. 9. If the user did enter a valid product, set the product in the state so we don't revisit `SelectProduct`. 10. If the balance is enough to purchase the product, adjust the balance to reflect the purchase and return [`End`](../api/pydantic_graph/nodes/#pydantic_graph.nodes.End) to end the graph. We're not using the run return type, so we call `End` with `None`. 11. If the balance is insufficient, go to `InsertCoin` to prompt the user to insert more coins. 12. If the product is invalid, go to `SelectProduct` to prompt the user to select a product again. 13. The graph is created by passing a list of nodes to [`Graph`](../api/pydantic_graph/graph/#pydantic_graph.graph.Graph) . Order of nodes is not important, but it can affect how [diagrams](#mermaid-diagrams) are displayed. 14. Initialize the state. This will be passed to the graph run and mutated as the graph runs. 15. Run the graph with the initial state. Since the graph can be run from any node, we must pass the start node — in this case, `InsertCoin`. [`Graph.run`](../api/pydantic_graph/graph/#pydantic_graph.graph.Graph.run) returns a [`GraphRunResult`](../api/pydantic_graph/graph/#pydantic_graph.graph.GraphRunResult) that provides the final data and a history of the run. 16. The return type of the node's [`run`](../api/pydantic_graph/nodes/#pydantic_graph.nodes.BaseNode.run) method is important as it is used to determine the outgoing edges of the node. This information in turn is used to render [mermaid diagrams](#mermaid-diagrams) and is enforced at runtime to detect misbehavior as soon as possible. 17. The return type of `CoinsInserted`'s [`run`](../api/pydantic_graph/nodes/#pydantic_graph.nodes.BaseNode.run) method is a union, meaning multiple outgoing edges are possible. 18. Unlike other nodes, `Purchase` can end the run, so the [`RunEndT`](../api/pydantic_graph/nodes/#pydantic_graph.nodes.RunEndT) generic parameter must be set. In this case it's `None` since the graph run return type is `None`. _(This example is complete, it can be run "as is" with Python 3.10+ — you'll need to add `asyncio.run(main())` to run `main`)_ A [mermaid diagram](#mermaid-diagrams) for this graph can be generated with the following code: vending\_machine\_diagram.py `from vending_machine import InsertCoin, vending_machine_graph vending_machine_graph.mermaid_code(start_node=InsertCoin)` The diagram generated by the above code is: --- title: vending_machine_graph --- stateDiagram-v2 [*] --> InsertCoin InsertCoin --> CoinsInserted CoinsInserted --> SelectProduct CoinsInserted --> Purchase SelectProduct --> Purchase Purchase --> InsertCoin Purchase --> SelectProduct Purchase --> [*] See [below](#mermaid-diagrams) for more information on generating diagrams. GenAI Example ------------- So far we haven't shown an example of a Graph that actually uses PydanticAI or GenAI at all. In this example, one agent generates a welcome email to a user and the other agent provides feedback on the email. This graph has a very simple structure: --- title: feedback_graph --- stateDiagram-v2 [*] --> WriteEmail WriteEmail --> Feedback Feedback --> WriteEmail Feedback --> [*] genai\_email\_feedback.py `from __future__ import annotations as _annotations from dataclasses import dataclass, field from pydantic import BaseModel, EmailStr from pydantic_ai import Agent from pydantic_ai.format_as_xml import format_as_xml from pydantic_ai.messages import ModelMessage from pydantic_graph import BaseNode, End, Graph, GraphRunContext @dataclass class User: name: str email: EmailStr interests: list[str] @dataclass class Email: subject: str body: str @dataclass class State: user: User write_agent_messages: list[ModelMessage] = field(default_factory=list) email_writer_agent = Agent( 'google-vertex:gemini-1.5-pro', result_type=Email, system_prompt='Write a welcome email to our tech blog.', ) @dataclass class WriteEmail(BaseNode[State]): email_feedback: str | None = None async def run(self, ctx: GraphRunContext[State]) -> Feedback: if self.email_feedback: prompt = ( f'Rewrite the email for the user:\n' f'{format_as_xml(ctx.state.user)}\n' f'Feedback: {self.email_feedback}' ) else: prompt = ( f'Write a welcome email for the user:\n' f'{format_as_xml(ctx.state.user)}' ) result = await email_writer_agent.run( prompt, message_history=ctx.state.write_agent_messages, ) ctx.state.write_agent_messages += result.all_messages() return Feedback(result.data) class EmailRequiresWrite(BaseModel): feedback: str class EmailOk(BaseModel): pass feedback_agent = Agent[None, EmailRequiresWrite | EmailOk]( 'openai:gpt-4o', result_type=EmailRequiresWrite | EmailOk, # type: ignore system_prompt=( 'Review the email and provide feedback, email must reference the users specific interests.' ), ) @dataclass class Feedback(BaseNode[State, None, Email]): email: Email async def run( self, ctx: GraphRunContext[State], ) -> WriteEmail | End[Email]: prompt = format_as_xml({'user': ctx.state.user, 'email': self.email}) result = await feedback_agent.run(prompt) if isinstance(result.data, EmailRequiresWrite): return WriteEmail(email_feedback=result.data.feedback) else: return End(self.email) async def main(): user = User( name='John Doe', email='john.joe@example.com', interests=['Haskel', 'Lisp', 'Fortran'], ) state = State(user) feedback_graph = Graph(nodes=(WriteEmail, Feedback)) result = await feedback_graph.run(WriteEmail(), state=state) print(result.output) """ Email( subject='Welcome to our tech blog!', body='Hello John, Welcome to our tech blog! ...', ) """` _(This example is complete, it can be run "as is" with Python 3.10+ — you'll need to add `asyncio.run(main())` to run `main`)_ Iterating Over a Graph ---------------------- ### Using `Graph.iter` for `async for` iteration Sometimes you want direct control or insight into each node as the graph executes. The easiest way to do that is with the [`Graph.iter`](../api/pydantic_graph/graph/#pydantic_graph.graph.Graph.iter) method, which returns a **context manager** that yields a [`GraphRun`](../api/pydantic_graph/graph/#pydantic_graph.graph.GraphRun) object. The `GraphRun` is an async-iterable over the nodes of your graph, allowing you to record or modify them as they execute. Here's an example: count\_down.py `from __future__ import annotations as _annotations from dataclasses import dataclass from pydantic_graph import Graph, BaseNode, End, GraphRunContext @dataclass class CountDownState: counter: int @dataclass class CountDown(BaseNode[CountDownState, None, int]): async def run(self, ctx: GraphRunContext[CountDownState]) -> CountDown | End[int]: if ctx.state.counter <= 0: return End(ctx.state.counter) ctx.state.counter -= 1 return CountDown() count_down_graph = Graph(nodes=[CountDown]) async def main(): state = CountDownState(counter=3) async with count_down_graph.iter(CountDown(), state=state) as run: # (1)! async for node in run: # (2)! print('Node:', node) #> Node: CountDown() #> Node: CountDown() #> Node: CountDown() #> Node: End(data=0) print('Final result:', run.result.output) # (3)! #> Final result: 0` 1. `Graph.iter(...)` returns a [`GraphRun`](../api/pydantic_graph/graph/#pydantic_graph.graph.GraphRun) . 2. Here, we step through each node as it is executed. 3. Once the graph returns an [`End`](../api/pydantic_graph/nodes/#pydantic_graph.nodes.End) , the loop ends, and `run.final_result` becomes a [`GraphRunResult`](../api/pydantic_graph/graph/#pydantic_graph.graph.GraphRunResult) containing the final outcome (`0` here). ### Using `GraphRun.next(node)` manually Alternatively, you can drive iteration manually with the [`GraphRun.next`](../api/pydantic_graph/graph/#pydantic_graph.graph.GraphRun.next) method, which allows you to pass in whichever node you want to run next. You can modify or selectively skip nodes this way. Below is a contrived example that stops whenever the counter is at 2, ignoring any node runs beyond that: count\_down\_next.py `from pydantic_graph import End, FullStatePersistence from count_down import CountDown, CountDownState, count_down_graph async def main(): state = CountDownState(counter=5) persistence = FullStatePersistence() # (7)! async with count_down_graph.iter( CountDown(), state=state, persistence=persistence ) as run: node = run.next_node # (1)! while not isinstance(node, End): # (2)! print('Node:', node) #> Node: CountDown() #> Node: CountDown() #> Node: CountDown() #> Node: CountDown() if state.counter == 2: break # (3)! node = await run.next(node) # (4)! print(run.result) # (5)! #> None for step in persistence.history: # (6)! print('History Step:', step.state, step.state) #> History Step: CountDownState(counter=5) CountDownState(counter=5) #> History Step: CountDownState(counter=4) CountDownState(counter=4) #> History Step: CountDownState(counter=3) CountDownState(counter=3) #> History Step: CountDownState(counter=2) CountDownState(counter=2)` 1. We start by grabbing the first node that will be run in the agent's graph. 2. The agent run is finished once an `End` node has been produced; instances of `End` cannot be passed to `next`. 3. If the user decides to stop early, we break out of the loop. The graph run won't have a real final result in that case (`run.final_result` remains `None`). 4. At each step, we call `await run.next(node)` to run it and get the next node (or an `End`). 5. Because we did not continue the run until it finished, the `result` is not set. 6. The run's history is still populated with the steps we executed so far. 7. Use [`FullStatePersistence`](../api/pydantic_graph/persistence/#pydantic_graph.persistence.in_mem.FullStatePersistence) so we can show the history of the run, see [State Persistence](#state-persistence) below for more information. State Persistence ----------------- One of the biggest benefits of finite state machine (FSM) graphs is how they simplify the handling of interrupted execution. This might happen for a variety of reasons: * the state machine logic might fundamentally need to be paused — e.g. the returns workflow for an e-commerce order needs to wait for the item to be posted to the returns center or because execution of the next node needs input from a user so needs to wait for a new http request, * the execution takes so long that the entire graph can't reliably be executed in a single continuous run — e.g. a deep research agent that might take hours to run, * you want to run multiple graph nodes in parallel in different processes / hardware instances (note: parallel node execution is not yet supported in `pydantic-graph`, see [#704](https://github.com/pydantic/pydantic-ai/issues/704) ). Trying to make a conventional control flow (i.e., boolean logic and nested function calls) implementation compatible with these usage scenarios generally results in brittle and over-complicated spaghetti code, with the logic required to interrupt and resume execution dominating the implementation. To allow graph runs to be interrupted and resumed, `pydantic-graph` provides state persistence — a system for snapshotting the state of a graph run before and after each node is run, allowing a graph run to be resumed from any point in the graph. `pydantic-graph` includes three state persistence implementations: * [`SimpleStatePersistence`](../api/pydantic_graph/persistence/#pydantic_graph.persistence.in_mem.SimpleStatePersistence) — Simple in memory state persistence that just hold the latest snapshot. If no state persistence implementation is provided when running a graph, this is used by default. * [`FullStatePersistence`](../api/pydantic_graph/persistence/#pydantic_graph.persistence.in_mem.FullStatePersistence) — In memory state persistence that hold a list of snapshots. * [`FileStatePersistence`](../api/pydantic_graph/persistence/#pydantic_graph.persistence.file.FileStatePersistence) — File-based state persistence that saves snapshots to a JSON file. In production applications, developers should implement their own state persistence by subclassing [`BaseStatePersistence`](../api/pydantic_graph/persistence/#pydantic_graph.persistence.BaseStatePersistence) abstract base class, which might persist runs in a relational database like PostgresQL. At a high level the role of `StatePersistence` implementations is to store and retrieve [`NodeSnapshot`](../api/pydantic_graph/persistence/#pydantic_graph.persistence.NodeSnapshot) and [`EndSnapshot`](../api/pydantic_graph/persistence/#pydantic_graph.persistence.EndSnapshot) objects. [`graph.iter_from_persistence()`](../api/pydantic_graph/graph/#pydantic_graph.graph.Graph.iter_from_persistence) may be used to run the graph based on the state stored in persistence. We can run the `count_down_graph` from [above](#iterating-over-a-graph) , using [`graph.iter_from_persistence()`](../api/pydantic_graph/graph/#pydantic_graph.graph.Graph.iter_from_persistence) and [`FileStatePersistence`](../api/pydantic_graph/persistence/#pydantic_graph.persistence.file.FileStatePersistence) . As you can see in this code, `run_node` requires no external application state (apart from state persistence) to be run, meaning graphs can easily be executed by distributed execution and queueing systems. count\_down\_from\_persistence.py `from pathlib import Path from pydantic_graph import End from pydantic_graph.persistence.file import FileStatePersistence from count_down import CountDown, CountDownState, count_down_graph async def main(): run_id = 'run_abc123' persistence = FileStatePersistence(Path(f'count_down_{run_id}.json')) # (1)! state = CountDownState(counter=5) await count_down_graph.initialize( # (2)! CountDown(), state=state, persistence=persistence ) done = False while not done: done = await run_node(run_id) async def run_node(run_id: str) -> bool: # (3)! persistence = FileStatePersistence(Path(f'count_down_{run_id}.json')) async with count_down_graph.iter_from_persistence(persistence) as run: # (4)! node_or_end = await run.next() # (5)! print('Node:', node_or_end) #> Node: CountDown() #> Node: CountDown() #> Node: CountDown() #> Node: CountDown() #> Node: CountDown() #> Node: End(data=0) return isinstance(node_or_end, End) # (6)!` 1. Create a [`FileStatePersistence`](../api/pydantic_graph/persistence/#pydantic_graph.persistence.file.FileStatePersistence) to use to start the graph. 2. Call [`graph.initialize()`](../api/pydantic_graph/graph/#pydantic_graph.graph.Graph.initialize) to set the initial graph state in the persistence object. 3. `run_node` is a pure function that doesn't need access to any other process state to run the next node of the graph, except the ID of the run. 4. Call [`graph.iter_from_persistence()`](../api/pydantic_graph/graph/#pydantic_graph.graph.Graph.iter_from_persistence) create a [`GraphRun`](../api/pydantic_graph/graph/#pydantic_graph.graph.GraphRun) object that will run the next node of the graph from the state stored in persistence. This will return either a node or an `End` object. 5. [`graph.run()`](../api/pydantic_graph/graph/#pydantic_graph.graph.Graph.run) will return either a [node](../api/pydantic_graph/nodes/#pydantic_graph.nodes.BaseNode) or an [`End`](../api/pydantic_graph/nodes/#pydantic_graph.nodes.End) object. 6. Check if the node is an [`End`](../api/pydantic_graph/nodes/#pydantic_graph.nodes.End) object, if it is, the graph run is complete. _(This example is complete, it can be run "as is" with Python 3.10+ — you'll need to add `asyncio.run(main())` to run `main`)_ ### Example: Human in the loop. As noted above, state persistence allows graphs to be interrupted and resumed. One use case of this is to allow user input to continue. In this example, an AI asks the user a question, the user provides an answer, the AI evaluates the answer and ends if the user got it right or asks another question if they got it wrong. Instead of running the entire graph in a single process invocation, we run the graph by running the process repeatedly, optionally providing an answer to the question as a command line argument. `ai_q_and_a_graph.py` — `question_graph` definition ai\_q\_and\_a\_graph.py `from __future__ import annotations as _annotations from dataclasses import dataclass, field from groq import BaseModel from pydantic_graph import ( BaseNode, End, Graph, GraphRunContext, ) from pydantic_ai import Agent from pydantic_ai.format_as_xml import format_as_xml from pydantic_ai.messages import ModelMessage ask_agent = Agent('openai:gpt-4o', result_type=str, instrument=True) @dataclass class QuestionState: question: str | None = None ask_agent_messages: list[ModelMessage] = field(default_factory=list) evaluate_agent_messages: list[ModelMessage] = field(default_factory=list) @dataclass class Ask(BaseNode[QuestionState]): async def run(self, ctx: GraphRunContext[QuestionState]) -> Answer: result = await ask_agent.run( 'Ask a simple question with a single correct answer.', message_history=ctx.state.ask_agent_messages, ) ctx.state.ask_agent_messages += result.all_messages() ctx.state.question = result.data return Answer(result.data) @dataclass class Answer(BaseNode[QuestionState]): question: str async def run(self, ctx: GraphRunContext[QuestionState]) -> Evaluate: answer = input(f'{self.question}: ') return Evaluate(answer) class EvaluationResult(BaseModel, use_attribute_docstrings=True): correct: bool """Whether the answer is correct.""" comment: str """Comment on the answer, reprimand the user if the answer is wrong.""" evaluate_agent = Agent( 'openai:gpt-4o', result_type=EvaluationResult, system_prompt='Given a question and answer, evaluate if the answer is correct.', ) @dataclass class Evaluate(BaseNode[QuestionState, None, str]): answer: str async def run( self, ctx: GraphRunContext[QuestionState], ) -> End[str] | Reprimand: assert ctx.state.question is not None result = await evaluate_agent.run( format_as_xml({'question': ctx.state.question, 'answer': self.answer}), message_history=ctx.state.evaluate_agent_messages, ) ctx.state.evaluate_agent_messages += result.all_messages() if result.data.correct: return End(result.data.comment) else: return Reprimand(result.data.comment) @dataclass class Reprimand(BaseNode[QuestionState]): comment: str async def run(self, ctx: GraphRunContext[QuestionState]) -> Ask: print(f'Comment: {self.comment}') ctx.state.question = None return Ask() question_graph = Graph( nodes=(Ask, Answer, Evaluate, Reprimand), state_type=QuestionState )` _(This example is complete, it can be run "as is" with Python 3.10+)_ ai\_q\_and\_a\_run.py `import sys from pathlib import Path from pydantic_graph import End from pydantic_graph.persistence.file import FileStatePersistence from pydantic_ai.messages import ModelMessage # noqa: F401 from ai_q_and_a_graph import Ask, question_graph, Evaluate, QuestionState, Answer async def main(): answer: str | None = sys.argv[1] if len(sys.argv) > 1 else None # (1)! persistence = FileStatePersistence(Path('question_graph.json')) # (2)! persistence.set_graph_types(question_graph) # (3)! if snapshot := await persistence.load_next(): # (4)! state = snapshot.state assert answer is not None node = Evaluate(answer) else: state = QuestionState() node = Ask() # (5)! async with question_graph.iter(node, state=state, persistence=persistence) as run: while True: node = await run.next() # (6)! if isinstance(node, End): # (7)! print('END:', node.data) history = await persistence.load_all() # (8)! print([e.node for e in history]) break elif isinstance(node, Answer): # (9)! print(node.question) #> What is the capital of France? break # otherwise just continue` 1. Get the user's answer from the command line, if provided. See [question graph example](../examples/question-graph/) for a complete example. 2. Create a state persistence instance the `'question_graph.json'` file may or may not already exist. 3. Since we're using the [persistence interface](../api/pydantic_graph/persistence/#pydantic_graph.persistence.BaseStatePersistence) outside a graph, we need to call [`set_graph_types`](../api/pydantic_graph/persistence/#pydantic_graph.persistence.BaseStatePersistence.set_graph_types) to set the graph generic types `StateT` and `RunEndT` for the persistence instance. This is necessary to allow the persistence instance to know how to serialize and deserialize graph nodes. 4. If we're run the graph before, [`load_next`](../api/pydantic_graph/persistence/#pydantic_graph.persistence.BaseStatePersistence.load_next) will return a snapshot of the next node to run, here we use `state` from that snapshot, and create a new `Evaluate` node with the answer provided on the command line. 5. If the graph hasn't been run before, we create a new `QuestionState` and start with the `Ask` node. 6. Call [`GraphRun.next()`](../api/pydantic_graph/graph/#pydantic_graph.graph.GraphRun.next) to run the node. This will return either a node or an `End` object. 7. If the node is an `End` object, the graph run is complete. The `data` field of the `End` object contains the comment returned by the `evaluate_agent` about the correct answer. 8. To demonstrate the state persistence, we call [`load_all`](../api/pydantic_graph/persistence/#pydantic_graph.persistence.BaseStatePersistence.load_all) to get all the snapshots from the persistence instance. This will return a list of [`Snapshot`](../api/pydantic_graph/persistence/#pydantic_graph.persistence.Snapshot) objects. 9. If the node is an `Answer` object, we print the question and break out of the loop to end the process and wait for user input. _(This example is complete, it can be run "as is" with Python 3.10+ — you'll need to add `asyncio.run(main())` to run `main`)_ For a complete example of this graph, see the [question graph example](../examples/question-graph/) . Dependency Injection -------------------- As with PydanticAI, `pydantic-graph` supports dependency injection via a generic parameter on [`Graph`](../api/pydantic_graph/graph/#pydantic_graph.graph.Graph) and [`BaseNode`](../api/pydantic_graph/nodes/#pydantic_graph.nodes.BaseNode) , and the [`GraphRunContext.deps`](../api/pydantic_graph/nodes/#pydantic_graph.nodes.GraphRunContext.deps) field. As an example of dependency injection, let's modify the `DivisibleBy5` example [above](#graph) to use a [`ProcessPoolExecutor`](https://docs.python.org/3/library/concurrent.futures.html#concurrent.futures.ProcessPoolExecutor) to run the compute load in a separate process (this is a contrived example, `ProcessPoolExecutor` wouldn't actually improve performance in this example): deps\_example.py `from __future__ import annotations import asyncio from concurrent.futures import ProcessPoolExecutor from dataclasses import dataclass from pydantic_graph import BaseNode, End, Graph, GraphRunContext @dataclass class GraphDeps: executor: ProcessPoolExecutor @dataclass class DivisibleBy5(BaseNode[None, GraphDeps, int]): foo: int async def run( self, ctx: GraphRunContext[None, GraphDeps], ) -> Increment | End[int]: if self.foo % 5 == 0: return End(self.foo) else: return Increment(self.foo) @dataclass class Increment(BaseNode[None, GraphDeps]): foo: int async def run(self, ctx: GraphRunContext[None, GraphDeps]) -> DivisibleBy5: loop = asyncio.get_running_loop() compute_result = await loop.run_in_executor( ctx.deps.executor, self.compute, ) return DivisibleBy5(compute_result) def compute(self) -> int: return self.foo + 1 fives_graph = Graph(nodes=[DivisibleBy5, Increment]) async def main(): with ProcessPoolExecutor() as executor: deps = GraphDeps(executor) result = await fives_graph.run(DivisibleBy5(3), deps=deps) print(result.output) #> 5 # the full history is quite verbose (see below), so we'll just print the summary print([item.data_snapshot() for item in result.history]) """ [ DivisibleBy5(foo=3), Increment(foo=3), DivisibleBy5(foo=4), Increment(foo=4), DivisibleBy5(foo=5), End(data=5), ] """` _(This example is complete, it can be run "as is" with Python 3.10+ — you'll need to add `asyncio.run(main())` to run `main`)_ Mermaid Diagrams ---------------- Pydantic Graph can generate [mermaid](https://mermaid.js.org/) [`stateDiagram-v2`](https://mermaid.js.org/syntax/stateDiagram.html) diagrams for graphs, as shown above. These diagrams can be generated with: * [`Graph.mermaid_code`](../api/pydantic_graph/graph/#pydantic_graph.graph.Graph.mermaid_code) to generate the mermaid code for a graph * [`Graph.mermaid_image`](../api/pydantic_graph/graph/#pydantic_graph.graph.Graph.mermaid_image) to generate an image of the graph using [mermaid.ink](https://mermaid.ink/) * [`Graph.mermaid_save`](../api/pydantic_graph/graph/#pydantic_graph.graph.Graph.mermaid_save) to generate an image of the graph using [mermaid.ink](https://mermaid.ink/) and save it to a file Beyond the diagrams shown above, you can also customize mermaid diagrams with the following options: * [`Edge`](../api/pydantic_graph/nodes/#pydantic_graph.nodes.Edge) allows you to apply a label to an edge * [`BaseNode.docstring_notes`](../api/pydantic_graph/nodes/#pydantic_graph.nodes.BaseNode.docstring_notes) and [`BaseNode.get_note`](../api/pydantic_graph/nodes/#pydantic_graph.nodes.BaseNode.get_note) allows you to add notes to nodes * The [`highlighted_nodes`](../api/pydantic_graph/graph/#pydantic_graph.graph.Graph.mermaid_code) parameter allows you to highlight specific node(s) in the diagram Putting that together, we can edit the last [`ai_q_and_a_graph.py`](#example-human-in-the-loop) example to: * add labels to some edges * add a note to the `Ask` node * highlight the `Answer` node * save the diagram as a `PNG` image to file ai\_q\_and\_a\_graph\_extra.py `... from typing import Annotated from pydantic_graph import BaseNode, End, Graph, GraphRunContext, Edge ... @dataclass class Ask(BaseNode[QuestionState]): """Generate question using GPT-4o.""" docstring_notes = True async def run( self, ctx: GraphRunContext[QuestionState] ) -> Annotated[Answer, Edge(label='Ask the question')]: ... ... @dataclass class Evaluate(BaseNode[QuestionState]): answer: str async def run( self, ctx: GraphRunContext[QuestionState], ) -> Annotated[End[str], Edge(label='success')] | Reprimand: ... ... question_graph.mermaid_save('image.png', highlighted_nodes=[Answer])` _(This example is not complete and cannot be run directly)_ This would generate an image that looks like this: --- title: question_graph --- stateDiagram-v2 Ask --> Answer: Ask the question note right of Ask Judge the answer. Decide on next step. end note Answer --> Evaluate Evaluate --> Reprimand Evaluate --> [*]: success Reprimand --> Ask classDef highlighted fill:#fdff32 class Answer highlighted ### Setting Direction of the State Diagram You can specify the direction of the state diagram using one of the following values: * `'TB'`: Top to bottom, the diagram flows vertically from top to bottom. * `'LR'`: Left to right, the diagram flows horizontally from left to right. * `'RL'`: Right to left, the diagram flows horizontally from right to left. * `'BT'`: Bottom to top, the diagram flows vertically from bottom to top. Here is an example of how to do this using 'Left to Right' (LR) instead of the default 'Top to Bottom' (TB): vending\_machine\_diagram.py `from vending_machine import InsertCoin, vending_machine_graph vending_machine_graph.mermaid_code(start_node=InsertCoin, direction='LR')` --- title: vending_machine_graph --- stateDiagram-v2 direction LR [*] --> InsertCoin InsertCoin --> CoinsInserted CoinsInserted --> SelectProduct CoinsInserted --> Purchase SelectProduct --> Purchase Purchase --> InsertCoin Purchase --> SelectProduct Purchase --> [*] --- # Image, Audio & Document Input - PydanticAI Image, Audio & Document Input ============================= Some LLMs are now capable of understanding both audio, image and document content. Image Input ----------- Info Some models do not support image input. Please check the model's documentation to confirm whether it supports image input. If you have a direct URL for the image, you can use [`ImageUrl`](../api/messages/#pydantic_ai.messages.ImageUrl) : main.py `from pydantic_ai import Agent, ImageUrl agent = Agent(model='openai:gpt-4o') result = agent.run_sync( [ 'What company is this logo from?', ImageUrl(url='https://iili.io/3Hs4FMg.png'), ] ) print(result.data) #> This is the logo for Pydantic, a data validation and settings management library in Python.` If you have the image locally, you can also use [`BinaryContent`](../api/messages/#pydantic_ai.messages.BinaryContent) : main.py `import httpx from pydantic_ai import Agent, BinaryContent image_response = httpx.get('https://iili.io/3Hs4FMg.png') # Pydantic logo agent = Agent(model='openai:gpt-4o') result = agent.run_sync( [ 'What company is this logo from?', BinaryContent(data=image_response.content, media_type='image/png'), # (1)! ] ) print(result.data) #> This is the logo for Pydantic, a data validation and settings management library in Python.` 1. To ensure the example is runnable we download this image from the web, but you can also use `Path().read_bytes()` to read a local file's contents. Audio Input ----------- Info Some models do not support audio input. Please check the model's documentation to confirm whether it supports audio input. You can provide audio input using either [`AudioUrl`](../api/messages/#pydantic_ai.messages.AudioUrl) or [`BinaryContent`](../api/messages/#pydantic_ai.messages.BinaryContent) . The process is analogous to the examples above. Document Input -------------- Info Some models do not support document input. Please check the model's documentation to confirm whether it supports document input. Warning When using Gemini models, the document content will always be sent as binary data, regardless of whether you use `DocumentUrl` or `BinaryContent`. This is due to differences in how Vertex AI and Google AI handle document inputs. For more details, see [this discussion](https://discuss.ai.google.dev/t/i-am-using-google-generative-ai-model-gemini-1-5-pro-for-image-analysis-but-getting-error/34866/4) . If you are unsatisfied with this behavior, please let us know by opening an issue on [GitHub](https://github.com/pydantic/pydantic-ai/issues) . You can provide document input using either [`DocumentUrl`](../api/messages/#pydantic_ai.messages.DocumentUrl) or [`BinaryContent`](../api/messages/#pydantic_ai.messages.BinaryContent) . The process is similar to the examples above. If you have a direct URL for the document, you can use [`DocumentUrl`](../api/messages/#pydantic_ai.messages.DocumentUrl) : main.py `from pydantic_ai import Agent, DocumentUrl agent = Agent(model='anthropic:claude-3-sonnet') result = agent.run_sync( [ 'What is the main content of this document?', DocumentUrl(url='https://storage.googleapis.com/cloud-samples-data/generative-ai/pdf/2403.05530.pdf'), ] ) print(result.data) #> This document is the technical report introducing Gemini 1.5, Google's latest large language model...` The supported document formats vary by model. You can also use [`BinaryContent`](../api/messages/#pydantic_ai.messages.BinaryContent) to pass document data directly: main.py `from pathlib import Path from pydantic_ai import Agent, BinaryContent pdf_path = Path('document.pdf') agent = Agent(model='anthropic:claude-3-sonnet') result = agent.run_sync( [ 'What is the main content of this document?', BinaryContent(data=pdf_path.read_bytes(), media_type='application/pdf'), ] ) print(result.data) #> The document discusses...` --- # Server - PydanticAI Server ====== PydanticAI models can also be used within MCP Servers. Here's a simple example of a [Python MCP server](https://github.com/modelcontextprotocol/python-sdk) using PydanticAI within a tool call: mcp\_server.py `from mcp.server.fastmcp import FastMCP from pydantic_ai import Agent server = FastMCP('PydanticAI Server') server_agent = Agent( 'anthropic:claude-3-5-haiku-latest', system_prompt='always reply in rhyme' ) @server.tool() async def poet(theme: str) -> str: """Poem generator""" r = await server_agent.run(f'write a poem about {theme}') return r.data if __name__ == '__main__': server.run()` This server can be queried with any MCP client. Here is an example using a direct Python client: mcp\_client.py `import asyncio import os from mcp import ClientSession, StdioServerParameters from mcp.client.stdio import stdio_client async def client(): server_params = StdioServerParameters( command='uv', args=['run', 'mcp_server.py', 'server'], env=os.environ ) async with stdio_client(server_params) as (read, write): async with ClientSession(read, write) as session: await session.initialize() result = await session.call_tool('poet', {'theme': 'socks'}) print(result.content[0].text) """ Oh, socks, those garments soft and sweet, That nestle softly 'round our feet, From cotton, wool, or blended thread, They keep our toes from feeling dread. """ if __name__ == '__main__': asyncio.run(client())` Note: [sampling](https://modelcontextprotocol.io/docs/concepts/sampling#sampling) , whereby servers may request LLM completions from the client, is not yet supported in PydanticAI. --- # MCP Run Python - PydanticAI MCP Run Python ============== The **MCP Run Python** package is an MCP server that allows agents to execute Python code in a secure, sandboxed environment. It uses [Pyodide](https://pyodide.org/) to run Python code in a JavaScript environment with [Deno](https://deno.com/) , isolating execution from the host system. Features -------- * **Secure Execution**: Run Python code in a sandboxed WebAssembly environment * **Package Management**: Automatically detects and installs required dependencies * **Complete Results**: Captures standard output, standard error, and return values * **Asynchronous Support**: Runs async code properly * **Error Handling**: Provides detailed error reports for debugging Installation ------------ Switch from npx to deno We previously distributed `mcp-run-python` as an `npm` package to use via `npx`. We now recommend using `deno` instead as it provides better sandboxing and security. The MCP Run Python server is distributed as a [JSR package](https://jsr.io/@pydantic/mcp-run-python) and can be run directly using [`deno run`](https://deno.com/) : terminal `deno run \ -N -R=node_modules -W=node_modules --node-modules-dir=auto \ jsr:@pydantic/mcp-run-python [stdio|sse|warmup]` where: * `-N -R=node_modules -W=node_modules` (alias of `--allow-net --allow-read=node_modules --allow-write=node_modules`) allows network access and read+write access to `./node_modules`. These are required so Pyodide can download and cache the Python standard library and packages * `--node-modules-dir=auto` tells deno to use a local `node_modules` directory * `stdio` runs the server with the [Stdio MCP transport](https://spec.modelcontextprotocol.io/specification/2024-11-05/basic/transports/#stdio) — suitable for running the process as a subprocess locally * `sse` runs the server with the [SSE MCP transport](https://spec.modelcontextprotocol.io/specification/2024-11-05/basic/transports/#http-with-sse) — running the server as an HTTP server to connect locally or remotely * `warmup` will run a minimal Python script to download and cache the Python standard library. This is also useful to check the server is running correctly. Usage of `jsr:@pydantic/mcp-run-python` with PydanticAI is described in the [client](../client/#mcp-stdio-server) documentation. Direct Usage ------------ As well as using this server with PydanticAI, it can be connected to other MCP clients. For clarity, in this example we connect directly using the [Python MCP client](https://github.com/modelcontextprotocol/python-sdk) . mcp\_run\_python.py `from mcp import ClientSession, StdioServerParameters from mcp.client.stdio import stdio_client code = """ import numpy a = numpy.array([1, 2, 3]) print(a) a """ server_params = StdioServerParameters( command='deno', args=[ 'run', '-N', '-R=node_modules', '-W=node_modules', '--node-modules-dir=auto', 'jsr:@pydantic/mcp-run-python', 'stdio', ], ) async def main(): async with stdio_client(server_params) as (read, write): async with ClientSession(read, write) as session: await session.initialize() tools = await session.list_tools() print(len(tools.tools)) #> 1 print(repr(tools.tools[0].name)) #> 'run_python_code' print(repr(tools.tools[0].inputSchema)) """ {'type': 'object', 'properties': {'python_code': {'type': 'string', 'description': 'Python code to run'}}, 'required': ['python_code'], 'additionalProperties': False, '$schema': 'http://json-schema.org/draft-07/schema#'} """ result = await session.call_tool('run_python_code', {'python_code': code}) print(result.content[0].text) """ success ["numpy"] [1 2 3] [ 1, 2, 3 ] """` If an exception occurs, `status` will be `install-error` or `run-error` and `return_value` will be replaced by `error` which will include the traceback and exception message. Dependencies ------------ Dependencies are installed when code is run. Dependencies can be defined in one of two ways: ### Inferred from imports If there's no metadata, dependencies are inferred from imports in the code, as shown in the example [above](#direct-usage) . ### Inline script metadata As introduced in PEP 723, explained [here](https://packaging.python.org/en/latest/specifications/inline-script-metadata/#inline-script-metadata) , and popularized by [uv](https://docs.astral.sh/uv/guides/scripts/#declaring-script-dependencies) — dependencies can be defined in a comment at the top of the file. This allows use of dependencies that aren't imported in the code, and is more explicit. inline\_script\_metadata.py ``from mcp import ClientSession from mcp.client.stdio import stdio_client # using `server_params` from the above example. from mcp_run_python import server_params code = """\ # /// script # dependencies = ["pydantic", "email-validator"] # /// import pydantic class Model(pydantic.BaseModel): email: pydantic.EmailStr print(Model(email='hello@pydantic.dev')) """ async def main(): async with stdio_client(server_params) as (read, write): async with ClientSession(read, write) as session: await session.initialize() result = await session.call_tool('run_python_code', {'python_code': code}) print(result.content[0].text) """ success ["pydantic","email-validator"] email='hello@pydantic.dev' """`` It also allows versions to be pinned for non-binary packages (Pyodide only supports a single version for the binary packages it supports, like `pydantic` and `numpy`). E.g. you could set the dependencies to `# /// script # dependencies = ["rich<13"] # ///` Logging ------- MCP Run Python supports emitting stdout and stderr from the python execution as [MCP logging messages](https://github.com/modelcontextprotocol/specification/blob/eb4abdf2bb91e0d5afd94510741eadd416982350/docs/specification/draft/server/utilities/logging.md?plain=1) . For logs to be emitted you must set the logging level when connecting to the server. By default, the log level is set to the highest level, `emergency`. Currently, it's not possible to demonstrate this due to a bug in the Python MCP Client, see [modelcontextprotocol/python-sdk#201](https://github.com/modelcontextprotocol/python-sdk/issues/201#issuecomment-2727663121) . --- # Examples - PydanticAI Examples ======== Examples of how to use PydanticAI and what it can do. Usage ----- These examples are distributed with `pydantic-ai` so you can run them either by cloning the [pydantic-ai repo](https://github.com/pydantic/pydantic-ai) or by simply installing `pydantic-ai` from PyPI with `pip` or `uv`. ### Installing required dependencies Either way you'll need to install extra dependencies to run some examples, you just need to install the `examples` optional dependency group. If you've installed `pydantic-ai` via pip/uv, you can install the extra dependencies with: pipuv `pip install "pydantic-ai[examples]"` `uv add "pydantic-ai[examples]"` If you clone the repo, you should instead use `uv sync --extra examples` to install extra dependencies. ### Setting model environment variables These examples will need you to set up authentication with one or more of the LLMs, see the [model configuration](../models/) docs for details on how to do this. TL;DR: in most cases you'll need to set one of the following environment variables: OpenAIGoogle Gemini `export OPENAI_API_KEY=your-api-key` `export GEMINI_API_KEY=your-api-key` ### Running Examples To run the examples (this will work whether you installed `pydantic_ai`, or cloned the repo), run: pipuv `python -m pydantic_ai_examples.` `uv run -m pydantic_ai_examples.` For examples, to run the very simple [`pydantic_model`](pydantic-model/) example: pipuv `python -m pydantic_ai_examples.pydantic_model` `uv run -m pydantic_ai_examples.pydantic_model` If you like one-liners and you're using uv, you can run a pydantic-ai example with zero setup: `OPENAI_API_KEY='your-api-key' \ uv run --with "pydantic-ai[examples]" \ -m pydantic_ai_examples.pydantic_model` * * * You'll probably want to edit examples in addition to just running them. You can copy the examples to a new directory with: pipuv `python -m pydantic_ai_examples --copy-to examples/` `uv run -m pydantic_ai_examples --copy-to examples/` --- # Command Line Interface (CLI) - PydanticAI Command Line Interface (CLI) ============================ **PydanticAI** comes with a simple reference CLI application which you can use to interact with various LLMs directly from the command line. It provides a convenient way to chat with language models and quickly get answers right in the terminal. We originally developed this CLI for our own use, but found ourselves using it so frequently that we decided to share it as part of the PydanticAI package. We plan to continue adding new features, such as interaction with MCP servers, access to tools, and more. Installation ------------ To use the CLI, you need to either install [`pydantic-ai`](../install/) , or install [`pydantic-ai-slim`](../install/#slim-install) with the `cli` optional group: pipuv `pip install "pydantic-ai[cli]"` `uv add "pydantic-ai[cli]"` To enable command-line argument autocompletion, run: `register-python-argcomplete pai >> ~/.bashrc # for bash register-python-argcomplete pai >> ~/.zshrc # for zsh` Usage ----- You'll need to set an environment variable depending on the provider you intend to use. If using OpenAI, set the `OPENAI_API_KEY` environment variable: `export OPENAI_API_KEY='your-api-key-here'` Then simply run: `pai` This will start an interactive session where you can chat with the AI model. Special commands available in interactive mode: * `/exit`: Exit the session * `/markdown`: Show the last response in markdown format * `/multiline`: Toggle multiline input mode (use Ctrl+D to submit) ### Choose a model You can specify which model to use with the `--model` flag: `$ pai --model=openai:gpt-4 "What's the capital of France?"` ### Usage with `uvx` If you have [uv](https://docs.astral.sh/uv/) installed, the quickest way to run the CLI is with `uvx`: `uvx --from pydantic-ai pai` --- # Pydantic Model - PydanticAI Pydantic Model ============== Simple example of using PydanticAI to construct a Pydantic model from a text input. Demonstrates: * [structured `result_type`](../../results/#structured-result-validation) Running the Example ------------------- With [dependencies installed and environment variables set](../#usage) , run: pipuv `python -m pydantic_ai_examples.pydantic_model` `uv run -m pydantic_ai_examples.pydantic_model` This examples uses `openai:gpt-4o` by default, but it works well with other models, e.g. you can run it with Gemini using: pipuv `PYDANTIC_AI_MODEL=gemini-1.5-pro python -m pydantic_ai_examples.pydantic_model` `PYDANTIC_AI_MODEL=gemini-1.5-pro uv run -m pydantic_ai_examples.pydantic_model` (or `PYDANTIC_AI_MODEL=gemini-1.5-flash ...`) Example Code ------------ pydantic\_model.py `import os import logfire from pydantic import BaseModel from pydantic_ai import Agent # 'if-token-present' means nothing will be sent (and the example will work) if you don't have logfire configured logfire.configure(send_to_logfire='if-token-present') class MyModel(BaseModel): city: str country: str model = os.getenv('PYDANTIC_AI_MODEL', 'openai:gpt-4o') print(f'Using model: {model}') agent = Agent(model, result_type=MyModel, instrument=True) if __name__ == '__main__': result = agent.run_sync('The windy city in the US of A.') print(result.data) print(result.usage())` --- # Bank support - PydanticAI Bank support ============ Small but complete example of using PydanticAI to build a support agent for a bank. Demonstrates: * [dynamic system prompt](../../agents/#system-prompts) * [structured `result_type`](../../results/#structured-result-validation) * [tools](../../tools/) Running the Example ------------------- With [dependencies installed and environment variables set](../#usage) , run: pipuv `python -m pydantic_ai_examples.bank_support` `uv run -m pydantic_ai_examples.bank_support` (or `PYDANTIC_AI_MODEL=gemini-1.5-flash ...`) Example Code ------------ bank\_support.py `from dataclasses import dataclass from pydantic import BaseModel, Field from pydantic_ai import Agent, RunContext class DatabaseConn: """This is a fake database for example purposes. In reality, you'd be connecting to an external database (e.g. PostgreSQL) to get information about customers. """ @classmethod async def customer_name(cls, *, id: int) -> str | None: if id == 123: return 'John' @classmethod async def customer_balance(cls, *, id: int, include_pending: bool) -> float: if id == 123 and include_pending: return 123.45 else: raise ValueError('Customer not found') @dataclass class SupportDependencies: customer_id: int db: DatabaseConn class SupportResult(BaseModel): support_advice: str = Field(description='Advice returned to the customer') block_card: bool = Field(description='Whether to block their card or not') risk: int = Field(description='Risk level of query', ge=0, le=10) support_agent = Agent( 'openai:gpt-4o', deps_type=SupportDependencies, result_type=SupportResult, system_prompt=( 'You are a support agent in our bank, give the ' 'customer support and judge the risk level of their query. ' "Reply using the customer's name." ), ) @support_agent.system_prompt async def add_customer_name(ctx: RunContext[SupportDependencies]) -> str: customer_name = await ctx.deps.db.customer_name(id=ctx.deps.customer_id) return f"The customer's name is {customer_name!r}" @support_agent.tool async def customer_balance( ctx: RunContext[SupportDependencies], include_pending: bool ) -> str: """Returns the customer's current account balance.""" balance = await ctx.deps.db.customer_balance( id=ctx.deps.customer_id, include_pending=include_pending, ) return f'${balance:.2f}' if __name__ == '__main__': deps = SupportDependencies(customer_id=123, db=DatabaseConn()) result = support_agent.run_sync('What is my balance?', deps=deps) print(result.data) """ support_advice='Hello John, your current account balance, including pending transactions, is $123.45.' block_card=False risk=1 """ result = support_agent.run_sync('I just lost my card!', deps=deps) print(result.data) """ support_advice="I'm sorry to hear that, John. We are temporarily blocking your card to prevent unauthorized transactions." block_card=True risk=8 """` --- # Weather agent - PydanticAI Weather agent ============= Example of PydanticAI with multiple tools which the LLM needs to call in turn to answer a question. Demonstrates: * [tools](../../tools/) * [agent dependencies](../../dependencies/) * [streaming text responses](../../results/#streaming-text) * Building a [Gradio](https://www.gradio.app/) UI for the agent In this case the idea is a "weather" agent — the user can ask for the weather in multiple locations, the agent will use the `get_lat_lng` tool to get the latitude and longitude of the locations, then use the `get_weather` tool to get the weather for those locations. Running the Example ------------------- To run this example properly, you might want to add two extra API keys **(Note if either key is missing, the code will fall back to dummy data, so they're not required)**: * A weather API key from [tomorrow.io](https://www.tomorrow.io/weather-api/) set via `WEATHER_API_KEY` * A geocoding API key from [geocode.maps.co](https://geocode.maps.co/) set via `GEO_API_KEY` With [dependencies installed and environment variables set](../#usage) , run: pipuv `python -m pydantic_ai_examples.weather_agent` `uv run -m pydantic_ai_examples.weather_agent` Example Code ------------ pydantic\_ai\_examples/weather\_agent.py ``from __future__ import annotations as _annotations import asyncio import os from dataclasses import dataclass from typing import Any import logfire from devtools import debug from httpx import AsyncClient from pydantic_ai import Agent, ModelRetry, RunContext # 'if-token-present' means nothing will be sent (and the example will work) if you don't have logfire configured logfire.configure(send_to_logfire='if-token-present') @dataclass class Deps: client: AsyncClient weather_api_key: str | None geo_api_key: str | None weather_agent = Agent( 'openai:gpt-4o', # 'Be concise, reply with one sentence.' is enough for some models (like openai) to use # the below tools appropriately, but others like anthropic and gemini require a bit more direction. system_prompt=( 'Be concise, reply with one sentence.' 'Use the `get_lat_lng` tool to get the latitude and longitude of the locations, ' 'then use the `get_weather` tool to get the weather.' ), deps_type=Deps, retries=2, instrument=True, ) @weather_agent.tool async def get_lat_lng( ctx: RunContext[Deps], location_description: str ) -> dict[str, float]: """Get the latitude and longitude of a location. Args: ctx: The context. location_description: A description of a location. """ if ctx.deps.geo_api_key is None: # if no API key is provided, return a dummy response (London) return {'lat': 51.1, 'lng': -0.1} params = { 'q': location_description, 'api_key': ctx.deps.geo_api_key, } with logfire.span('calling geocode API', params=params) as span: r = await ctx.deps.client.get('https://geocode.maps.co/search', params=params) r.raise_for_status() data = r.json() span.set_attribute('response', data) if data: return {'lat': data[0]['lat'], 'lng': data[0]['lon']} else: raise ModelRetry('Could not find the location') @weather_agent.tool async def get_weather(ctx: RunContext[Deps], lat: float, lng: float) -> dict[str, Any]: """Get the weather at a location. Args: ctx: The context. lat: Latitude of the location. lng: Longitude of the location. """ if ctx.deps.weather_api_key is None: # if no API key is provided, return a dummy response return {'temperature': '21 °C', 'description': 'Sunny'} params = { 'apikey': ctx.deps.weather_api_key, 'location': f'{lat},{lng}', 'units': 'metric', } with logfire.span('calling weather API', params=params) as span: r = await ctx.deps.client.get( 'https://api.tomorrow.io/v4/weather/realtime', params=params ) r.raise_for_status() data = r.json() span.set_attribute('response', data) values = data['data']['values'] # https://docs.tomorrow.io/reference/data-layers-weather-codes code_lookup = { 1000: 'Clear, Sunny', 1100: 'Mostly Clear', 1101: 'Partly Cloudy', 1102: 'Mostly Cloudy', 1001: 'Cloudy', 2000: 'Fog', 2100: 'Light Fog', 4000: 'Drizzle', 4001: 'Rain', 4200: 'Light Rain', 4201: 'Heavy Rain', 5000: 'Snow', 5001: 'Flurries', 5100: 'Light Snow', 5101: 'Heavy Snow', 6000: 'Freezing Drizzle', 6001: 'Freezing Rain', 6200: 'Light Freezing Rain', 6201: 'Heavy Freezing Rain', 7000: 'Ice Pellets', 7101: 'Heavy Ice Pellets', 7102: 'Light Ice Pellets', 8000: 'Thunderstorm', } return { 'temperature': f'{values["temperatureApparent"]:0.0f}°C', 'description': code_lookup.get(values['weatherCode'], 'Unknown'), } async def main(): async with AsyncClient() as client: # create a free API key at https://www.tomorrow.io/weather-api/ weather_api_key = os.getenv('WEATHER_API_KEY') # create a free API key at https://geocode.maps.co/ geo_api_key = os.getenv('GEO_API_KEY') deps = Deps( client=client, weather_api_key=weather_api_key, geo_api_key=geo_api_key ) result = await weather_agent.run( 'What is the weather like in London and in Wiltshire?', deps=deps ) debug(result) print('Response:', result.data) if __name__ == '__main__': asyncio.run(main())`` Running the UI -------------- You can build multi-turn chat applications for your agent with [Gradio](https://www.gradio.app/) , a framework for building AI web applications entirely in python. Gradio comes with built-in chat components and agent support so the entire UI will be implemented in a single python file! Here's what the UI looks like for the weather agent: Note, to run the UI, you'll need Python 3.10+. `pip install gradio>=5.9.0 python/uv-run -m pydantic_ai_examples.weather_agent_gradio` UI Code ------- pydantic\_ai\_examples/weather\_agent\_gradio.py ``from __future__ import annotations as _annotations import json import os from httpx import AsyncClient from pydantic_ai.messages import ToolCallPart, ToolReturnPart from pydantic_ai_examples.weather_agent import Deps, weather_agent try: import gradio as gr except ImportError as e: raise ImportError( 'Please install gradio with `pip install gradio`. You must use python>=3.10.' ) from e TOOL_TO_DISPLAY_NAME = {'get_lat_lng': 'Geocoding API', 'get_weather': 'Weather API'} client = AsyncClient() weather_api_key = os.getenv('WEATHER_API_KEY') # create a free API key at https://geocode.maps.co/ geo_api_key = os.getenv('GEO_API_KEY') deps = Deps(client=client, weather_api_key=weather_api_key, geo_api_key=geo_api_key) async def stream_from_agent(prompt: str, chatbot: list[dict], past_messages: list): chatbot.append({'role': 'user', 'content': prompt}) yield gr.Textbox(interactive=False, value=''), chatbot, gr.skip() async with weather_agent.run_stream( prompt, deps=deps, message_history=past_messages ) as result: for message in result.new_messages(): for call in message.parts: if isinstance(call, ToolCallPart): call_args = ( call.args.args_json if hasattr(call.args, 'args_json') else json.dumps(call.args.args_dict) ) metadata = { 'title': f'🛠️ Using {TOOL_TO_DISPLAY_NAME[call.tool_name]}', } if call.tool_call_id is not None: metadata['id'] = {call.tool_call_id} gr_message = { 'role': 'assistant', 'content': 'Parameters: ' + call_args, 'metadata': metadata, } chatbot.append(gr_message) if isinstance(call, ToolReturnPart): for gr_message in chatbot: if ( gr_message.get('metadata', {}).get('id', '') == call.tool_call_id ): gr_message['content'] += ( f'\nOutput: {json.dumps(call.content)}' ) yield gr.skip(), chatbot, gr.skip() chatbot.append({'role': 'assistant', 'content': ''}) async for message in result.stream_text(): chatbot[-1]['content'] = message yield gr.skip(), chatbot, gr.skip() past_messages = result.all_messages() yield gr.Textbox(interactive=True), gr.skip(), past_messages async def handle_retry(chatbot, past_messages: list, retry_data: gr.RetryData): new_history = chatbot[: retry_data.index] previous_prompt = chatbot[retry_data.index]['content'] past_messages = past_messages[: retry_data.index] async for update in stream_from_agent(previous_prompt, new_history, past_messages): yield update def undo(chatbot, past_messages: list, undo_data: gr.UndoData): new_history = chatbot[: undo_data.index] past_messages = past_messages[: undo_data.index] return chatbot[undo_data.index]['content'], new_history, past_messages def select_data(message: gr.SelectData) -> str: return message.value['text'] with gr.Blocks() as demo: gr.HTML( """

Weather Assistant

This assistant answer your weather questions.

""" ) past_messages = gr.State([]) chatbot = gr.Chatbot( label='Packing Assistant', type='messages', avatar_images=(None, 'https://ai.pydantic.dev/img/logo-white.svg'), examples=[ {'text': 'What is the weather like in Miami?'}, {'text': 'What is the weather like in London?'}, ], ) with gr.Row(): prompt = gr.Textbox( lines=1, show_label=False, placeholder='What is the weather like in New York City?', ) generation = prompt.submit( stream_from_agent, inputs=[prompt, chatbot, past_messages], outputs=[prompt, chatbot, past_messages], ) chatbot.example_select(select_data, None, [prompt]) chatbot.retry( handle_retry, [chatbot, past_messages], [prompt, chatbot, past_messages] ) chatbot.undo(undo, [chatbot, past_messages], [prompt, chatbot, past_messages]) if __name__ == '__main__': demo.launch()`` --- # Flight booking - PydanticAI Flight booking ============== Example of a multi-agent flow where one agent delegates work to another, then hands off control to a third agent. Demonstrates: * [agent delegation](../../multi-agent-applications/#agent-delegation) * [programmatic agent hand-off](../../multi-agent-applications/#programmatic-agent-hand-off) * [usage limits](../../agents/#usage-limits) In this scenario, a group of agents work together to find the best flight for a user. The control flow for this example can be summarised as follows: graph TD START --> search_agent("search agent") search_agent --> extraction_agent("extraction agent") extraction_agent --> search_agent search_agent --> human_confirm("human confirm") human_confirm --> search_agent search_agent --> FAILED human_confirm --> find_seat_function("find seat function") find_seat_function --> human_seat_choice("human seat choice") human_seat_choice --> find_seat_agent("find seat agent") find_seat_agent --> find_seat_function find_seat_function --> buy_flights("buy flights") buy_flights --> SUCCESS Running the Example ------------------- With [dependencies installed and environment variables set](../#usage) , run: pipuv `python -m pydantic_ai_examples.flight_booking` `uv run -m pydantic_ai_examples.flight_booking` Example Code ------------ flight\_booking.py `import datetime from dataclasses import dataclass from typing import Literal import logfire from pydantic import BaseModel, Field from rich.prompt import Prompt from pydantic_ai import Agent, ModelRetry, RunContext from pydantic_ai.messages import ModelMessage from pydantic_ai.usage import Usage, UsageLimits # 'if-token-present' means nothing will be sent (and the example will work) if you don't have logfire configured logfire.configure(send_to_logfire='if-token-present') class FlightDetails(BaseModel): """Details of the most suitable flight.""" flight_number: str price: int origin: str = Field(description='Three-letter airport code') destination: str = Field(description='Three-letter airport code') date: datetime.date class NoFlightFound(BaseModel): """When no valid flight is found.""" @dataclass class Deps: web_page_text: str req_origin: str req_destination: str req_date: datetime.date # This agent is responsible for controlling the flow of the conversation. search_agent = Agent[Deps, FlightDetails | NoFlightFound]( 'openai:gpt-4o', result_type=FlightDetails | NoFlightFound, # type: ignore retries=4, system_prompt=( 'Your job is to find the cheapest flight for the user on the given date. ' ), instrument=True, ) # This agent is responsible for extracting flight details from web page text. extraction_agent = Agent( 'openai:gpt-4o', result_type=list[FlightDetails], system_prompt='Extract all the flight details from the given text.', ) @search_agent.tool async def extract_flights(ctx: RunContext[Deps]) -> list[FlightDetails]: """Get details of all flights.""" # we pass the usage to the search agent so requests within this agent are counted result = await extraction_agent.run(ctx.deps.web_page_text, usage=ctx.usage) logfire.info('found {flight_count} flights', flight_count=len(result.data)) return result.data @search_agent.result_validator async def validate_result( ctx: RunContext[Deps], result: FlightDetails | NoFlightFound ) -> FlightDetails | NoFlightFound: """Procedural validation that the flight meets the constraints.""" if isinstance(result, NoFlightFound): return result errors: list[str] = [] if result.origin != ctx.deps.req_origin: errors.append( f'Flight should have origin {ctx.deps.req_origin}, not {result.origin}' ) if result.destination != ctx.deps.req_destination: errors.append( f'Flight should have destination {ctx.deps.req_destination}, not {result.destination}' ) if result.date != ctx.deps.req_date: errors.append(f'Flight should be on {ctx.deps.req_date}, not {result.date}') if errors: raise ModelRetry('\n'.join(errors)) else: return result class SeatPreference(BaseModel): row: int = Field(ge=1, le=30) seat: Literal['A', 'B', 'C', 'D', 'E', 'F'] class Failed(BaseModel): """Unable to extract a seat selection.""" # This agent is responsible for extracting the user's seat selection seat_preference_agent = Agent[None, SeatPreference | Failed]( 'openai:gpt-4o', result_type=SeatPreference | Failed, # type: ignore system_prompt=( "Extract the user's seat preference. " 'Seats A and F are window seats. ' 'Row 1 is the front row and has extra leg room. ' 'Rows 14, and 20 also have extra leg room. ' ), ) # in reality this would be downloaded from a booking site, # potentially using another agent to navigate the site flights_web_page = """ 1. Flight SFO-AK123 - Price: $350 - Origin: San Francisco International Airport (SFO) - Destination: Ted Stevens Anchorage International Airport (ANC) - Date: January 10, 2025 2. Flight SFO-AK456 - Price: $370 - Origin: San Francisco International Airport (SFO) - Destination: Fairbanks International Airport (FAI) - Date: January 10, 2025 3. Flight SFO-AK789 - Price: $400 - Origin: San Francisco International Airport (SFO) - Destination: Juneau International Airport (JNU) - Date: January 20, 2025 4. Flight NYC-LA101 - Price: $250 - Origin: San Francisco International Airport (SFO) - Destination: Ted Stevens Anchorage International Airport (ANC) - Date: January 10, 2025 5. Flight CHI-MIA202 - Price: $200 - Origin: Chicago O'Hare International Airport (ORD) - Destination: Miami International Airport (MIA) - Date: January 12, 2025 6. Flight BOS-SEA303 - Price: $120 - Origin: Boston Logan International Airport (BOS) - Destination: Ted Stevens Anchorage International Airport (ANC) - Date: January 12, 2025 7. Flight DFW-DEN404 - Price: $150 - Origin: Dallas/Fort Worth International Airport (DFW) - Destination: Denver International Airport (DEN) - Date: January 10, 2025 8. Flight ATL-HOU505 - Price: $180 - Origin: Hartsfield-Jackson Atlanta International Airport (ATL) - Destination: George Bush Intercontinental Airport (IAH) - Date: January 10, 2025 """ # restrict how many requests this app can make to the LLM usage_limits = UsageLimits(request_limit=15) async def main(): deps = Deps( web_page_text=flights_web_page, req_origin='SFO', req_destination='ANC', req_date=datetime.date(2025, 1, 10), ) message_history: list[ModelMessage] | None = None usage: Usage = Usage() # run the agent until a satisfactory flight is found while True: result = await search_agent.run( f'Find me a flight from {deps.req_origin} to {deps.req_destination} on {deps.req_date}', deps=deps, usage=usage, message_history=message_history, usage_limits=usage_limits, ) if isinstance(result.data, NoFlightFound): print('No flight found') break else: flight = result.data print(f'Flight found: {flight}') answer = Prompt.ask( 'Do you want to buy this flight, or keep searching? (buy/*search)', choices=['buy', 'search', ''], show_choices=False, ) if answer == 'buy': seat = await find_seat(usage) await buy_tickets(flight, seat) break else: message_history = result.all_messages( result_tool_return_content='Please suggest another flight' ) async def find_seat(usage: Usage) -> SeatPreference: message_history: list[ModelMessage] | None = None while True: answer = Prompt.ask('What seat would you like?') result = await seat_preference_agent.run( answer, message_history=message_history, usage=usage, usage_limits=usage_limits, ) if isinstance(result.data, SeatPreference): return result.data else: print('Could not understand seat preference. Please try again.') message_history = result.all_messages() async def buy_tickets(flight_details: FlightDetails, seat: SeatPreference): print(f'Purchasing flight {flight_details=!r} {seat=!r}...') if __name__ == '__main__': import asyncio asyncio.run(main())` --- # Stream markdown - PydanticAI Stream markdown =============== This example shows how to stream markdown from an agent, using the [`rich`](https://github.com/Textualize/rich) library to highlight the output in the terminal. It'll run the example with both OpenAI and Google Gemini models if the required environment variables are set. Demonstrates: * [streaming text responses](../../results/#streaming-text) Running the Example ------------------- With [dependencies installed and environment variables set](../#usage) , run: pipuv `python -m pydantic_ai_examples.stream_markdown` `uv run -m pydantic_ai_examples.stream_markdown` Example Code ------------ `import asyncio import os import logfire from rich.console import Console, ConsoleOptions, RenderResult from rich.live import Live from rich.markdown import CodeBlock, Markdown from rich.syntax import Syntax from rich.text import Text from pydantic_ai import Agent from pydantic_ai.models import KnownModelName # 'if-token-present' means nothing will be sent (and the example will work) if you don't have logfire configured logfire.configure(send_to_logfire='if-token-present') agent = Agent(instrument=True) # models to try, and the appropriate env var models: list[tuple[KnownModelName, str]] = [ ('google-gla:gemini-1.5-flash', 'GEMINI_API_KEY'), ('openai:gpt-4o-mini', 'OPENAI_API_KEY'), ('groq:llama-3.3-70b-versatile', 'GROQ_API_KEY'), ] async def main(): prettier_code_blocks() console = Console() prompt = 'Show me a short example of using Pydantic.' console.log(f'Asking: {prompt}...', style='cyan') for model, env_var in models: if env_var in os.environ: console.log(f'Using model: {model}') with Live('', console=console, vertical_overflow='visible') as live: async with agent.run_stream(prompt, model=model) as result: async for message in result.stream(): live.update(Markdown(message)) console.log(result.usage()) else: console.log(f'{model} requires {env_var} to be set.') def prettier_code_blocks(): """Make rich code blocks prettier and easier to copy. From https://github.com/samuelcolvin/aicli/blob/v0.8.0/samuelcolvin_aicli.py#L22 """ class SimpleCodeBlock(CodeBlock): def __rich_console__( self, console: Console, options: ConsoleOptions ) -> RenderResult: code = str(self.text).rstrip() yield Text(self.lexer_name, style='dim') yield Syntax( code, self.lexer_name, theme=self.theme, background_color='default', word_wrap=True, ) yield Text(f'/{self.lexer_name}', style='dim') Markdown.elements['fence'] = SimpleCodeBlock if __name__ == '__main__': asyncio.run(main())` --- # SQL Generation - PydanticAI SQL Generation ============== Example demonstrating how to use PydanticAI to generate SQL queries based on user input. Demonstrates: * [dynamic system prompt](../../agents/#system-prompts) * [structured `result_type`](../../results/#structured-result-validation) * [result validation](../../results/#result-validators-functions) * [agent dependencies](../../dependencies/) Running the Example ------------------- The resulting SQL is validated by running it as an `EXPLAIN` query on PostgreSQL. To run the example, you first need to run PostgreSQL, e.g. via Docker: `docker run --rm -e POSTGRES_PASSWORD=postgres -p 54320:5432 postgres` _(we run postgres on port `54320` to avoid conflicts with any other postgres instances you may have running)_ With [dependencies installed and environment variables set](../#usage) , run: pipuv `python -m pydantic_ai_examples.sql_gen` `uv run -m pydantic_ai_examples.sql_gen` or to use a custom prompt: pipuv `python -m pydantic_ai_examples.sql_gen "find me errors"` `uv run -m pydantic_ai_examples.sql_gen "find me errors"` This model uses `gemini-1.5-flash` by default since Gemini is good at single shot queries of this kind. Example Code ------------ sql\_gen.py `import asyncio import sys from collections.abc import AsyncGenerator from contextlib import asynccontextmanager from dataclasses import dataclass from datetime import date from typing import Annotated, Any, Union import asyncpg import logfire from annotated_types import MinLen from devtools import debug from pydantic import BaseModel, Field from typing_extensions import TypeAlias from pydantic_ai import Agent, ModelRetry, RunContext from pydantic_ai.format_as_xml import format_as_xml # 'if-token-present' means nothing will be sent (and the example will work) if you don't have logfire configured logfire.configure(send_to_logfire='if-token-present') logfire.instrument_asyncpg() DB_SCHEMA = """ CREATE TABLE records ( created_at timestamptz, start_timestamp timestamptz, end_timestamp timestamptz, trace_id text, span_id text, parent_span_id text, level log_level, span_name text, message text, attributes_json_schema text, attributes jsonb, tags text[], is_exception boolean, otel_status_message text, service_name text ); """ SQL_EXAMPLES = [ { 'request': 'show me records where foobar is false', 'response': "SELECT * FROM records WHERE attributes->>'foobar' = false", }, { 'request': 'show me records where attributes include the key "foobar"', 'response': "SELECT * FROM records WHERE attributes ? 'foobar'", }, { 'request': 'show me records from yesterday', 'response': "SELECT * FROM records WHERE start_timestamp::date > CURRENT_TIMESTAMP - INTERVAL '1 day'", }, { 'request': 'show me error records with the tag "foobar"', 'response': "SELECT * FROM records WHERE level = 'error' and 'foobar' = ANY(tags)", }, ] @dataclass class Deps: conn: asyncpg.Connection class Success(BaseModel): """Response when SQL could be successfully generated.""" sql_query: Annotated[str, MinLen(1)] explanation: str = Field( '', description='Explanation of the SQL query, as markdown' ) class InvalidRequest(BaseModel): """Response the user input didn't include enough information to generate SQL.""" error_message: str Response: TypeAlias = Union[Success, InvalidRequest] agent: Agent[Deps, Response] = Agent( 'google-gla:gemini-1.5-flash', # Type ignore while we wait for PEP-0747, nonetheless unions will work fine everywhere else result_type=Response, # type: ignore deps_type=Deps, instrument=True, ) @agent.system_prompt async def system_prompt() -> str: return f"""\ Given the following PostgreSQL table of records, your job is to write a SQL query that suits the user's request. Database schema: {DB_SCHEMA} today's date = {date.today()} {format_as_xml(SQL_EXAMPLES)} """ @agent.result_validator async def validate_result(ctx: RunContext[Deps], result: Response) -> Response: if isinstance(result, InvalidRequest): return result # gemini often adds extraneous backslashes to SQL result.sql_query = result.sql_query.replace('\\', '') if not result.sql_query.upper().startswith('SELECT'): raise ModelRetry('Please create a SELECT query') try: await ctx.deps.conn.execute(f'EXPLAIN {result.sql_query}') except asyncpg.exceptions.PostgresError as e: raise ModelRetry(f'Invalid query: {e}') from e else: return result async def main(): if len(sys.argv) == 1: prompt = 'show me logs from yesterday, with level "error"' else: prompt = sys.argv[1] async with database_connect( 'postgresql://postgres:postgres@localhost:54320', 'pydantic_ai_sql_gen' ) as conn: deps = Deps(conn) result = await agent.run(prompt, deps=deps) debug(result.data) # pyright: reportUnknownMemberType=false # pyright: reportUnknownVariableType=false @asynccontextmanager async def database_connect(server_dsn: str, database: str) -> AsyncGenerator[Any, None]: with logfire.span('check and create DB'): conn = await asyncpg.connect(server_dsn) try: db_exists = await conn.fetchval( 'SELECT 1 FROM pg_database WHERE datname = $1', database ) if not db_exists: await conn.execute(f'CREATE DATABASE {database}') finally: await conn.close() conn = await asyncpg.connect(f'{server_dsn}/{database}') try: with logfire.span('create schema'): async with conn.transaction(): if not db_exists: await conn.execute( "CREATE TYPE log_level AS ENUM ('debug', 'info', 'warning', 'error', 'critical')" ) await conn.execute(DB_SCHEMA) yield conn finally: await conn.close() if __name__ == '__main__': asyncio.run(main())` --- # Stream whales - PydanticAI Stream whales ============= Information about whales — an example of streamed structured response validation. Demonstrates: * [streaming structured responses](../../results/#streaming-structured-responses) This script streams structured responses from GPT-4 about whales, validates the data and displays it as a dynamic table using [`rich`](https://github.com/Textualize/rich) as the data is received. Running the Example ------------------- With [dependencies installed and environment variables set](../#usage) , run: pipuv `python -m pydantic_ai_examples.stream_whales` `uv run -m pydantic_ai_examples.stream_whales` Should give an output like this: Example Code ------------ stream\_whales.py `from typing import Annotated import logfire from pydantic import Field, ValidationError from rich.console import Console from rich.live import Live from rich.table import Table from typing_extensions import NotRequired, TypedDict from pydantic_ai import Agent # 'if-token-present' means nothing will be sent (and the example will work) if you don't have logfire configured logfire.configure(send_to_logfire='if-token-present') class Whale(TypedDict): name: str length: Annotated[ float, Field(description='Average length of an adult whale in meters.') ] weight: NotRequired[ Annotated[ float, Field(description='Average weight of an adult whale in kilograms.', ge=50), ] ] ocean: NotRequired[str] description: NotRequired[Annotated[str, Field(description='Short Description')]] agent = Agent('openai:gpt-4', result_type=list[Whale], instrument=True) async def main(): console = Console() with Live('\n' * 36, console=console) as live: console.print('Requesting data...', style='cyan') async with agent.run_stream( 'Generate me details of 5 species of Whale.' ) as result: console.print('Response:', style='green') async for message, last in result.stream_structured(debounce_by=0.01): try: whales = await result.validate_structured_result( message, allow_partial=not last ) except ValidationError as exc: if all( e['type'] == 'missing' and e['loc'] == ('response',) for e in exc.errors() ): continue else: raise table = Table( title='Species of Whale', caption='Streaming Structured responses from GPT-4', width=120, ) table.add_column('ID', justify='right') table.add_column('Name') table.add_column('Avg. Length (m)', justify='right') table.add_column('Avg. Weight (kg)', justify='right') table.add_column('Ocean') table.add_column('Description', justify='right') for wid, whale in enumerate(whales, start=1): table.add_row( str(wid), whale['name'], f'{whale["length"]:0.0f}', f'{w:0.0f}' if (w := whale.get('weight')) else '…', whale.get('ocean') or '…', whale.get('description') or '…', ) live.update(table) if __name__ == '__main__': import asyncio asyncio.run(main())` --- # Question Graph - PydanticAI Question Graph ============== Example of a graph for asking and evaluating questions. Demonstrates: * [`pydantic_graph`](../../graph/) Running the Example ------------------- With [dependencies installed and environment variables set](../#usage) , run: pipuv `python -m pydantic_ai_examples.question_graph` `uv run -m pydantic_ai_examples.question_graph` Example Code ------------ question\_graph.py `from __future__ import annotations as _annotations from dataclasses import dataclass, field from pathlib import Path import logfire from groq import BaseModel from pydantic_graph import ( BaseNode, End, Graph, GraphRunContext, ) from pydantic_graph.persistence.file import FileStatePersistence from pydantic_ai import Agent from pydantic_ai.format_as_xml import format_as_xml from pydantic_ai.messages import ModelMessage # 'if-token-present' means nothing will be sent (and the example will work) if you don't have logfire configured logfire.configure(send_to_logfire='if-token-present') ask_agent = Agent('openai:gpt-4o', result_type=str, instrument=True) @dataclass class QuestionState: question: str | None = None ask_agent_messages: list[ModelMessage] = field(default_factory=list) evaluate_agent_messages: list[ModelMessage] = field(default_factory=list) @dataclass class Ask(BaseNode[QuestionState]): async def run(self, ctx: GraphRunContext[QuestionState]) -> Answer: result = await ask_agent.run( 'Ask a simple question with a single correct answer.', message_history=ctx.state.ask_agent_messages, ) ctx.state.ask_agent_messages += result.all_messages() ctx.state.question = result.data return Answer(result.data) @dataclass class Answer(BaseNode[QuestionState]): question: str async def run(self, ctx: GraphRunContext[QuestionState]) -> Evaluate: answer = input(f'{self.question}: ') return Evaluate(answer) class EvaluationResult(BaseModel, use_attribute_docstrings=True): correct: bool """Whether the answer is correct.""" comment: str """Comment on the answer, reprimand the user if the answer is wrong.""" evaluate_agent = Agent( 'openai:gpt-4o', result_type=EvaluationResult, system_prompt='Given a question and answer, evaluate if the answer is correct.', ) @dataclass class Evaluate(BaseNode[QuestionState, None, str]): answer: str async def run( self, ctx: GraphRunContext[QuestionState], ) -> End[str] | Reprimand: assert ctx.state.question is not None result = await evaluate_agent.run( format_as_xml({'question': ctx.state.question, 'answer': self.answer}), message_history=ctx.state.evaluate_agent_messages, ) ctx.state.evaluate_agent_messages += result.all_messages() if result.data.correct: return End(result.data.comment) else: return Reprimand(result.data.comment) @dataclass class Reprimand(BaseNode[QuestionState]): comment: str async def run(self, ctx: GraphRunContext[QuestionState]) -> Ask: print(f'Comment: {self.comment}') ctx.state.question = None return Ask() question_graph = Graph( nodes=(Ask, Answer, Evaluate, Reprimand), state_type=QuestionState ) async def run_as_continuous(): state = QuestionState() node = Ask() end = await question_graph.run(node, state=state) print('END:', end.output) async def run_as_cli(answer: str | None): persistence = FileStatePersistence(Path('question_graph.json')) persistence.set_graph_types(question_graph) if snapshot := await persistence.load_next(): state = snapshot.state assert answer is not None, ( 'answer required, usage "uv run -m pydantic_ai_examples.question_graph cli "' ) node = Evaluate(answer) else: state = QuestionState() node = Ask() # debug(state, node) async with question_graph.iter(node, state=state, persistence=persistence) as run: while True: node = await run.next() if isinstance(node, End): print('END:', node.data) history = await persistence.load_all() print('history:', '\n'.join(str(e.node) for e in history), sep='\n') print('Finished!') break elif isinstance(node, Answer): print(node.question) break # otherwise just continue if __name__ == '__main__': import asyncio import sys try: sub_command = sys.argv[1] assert sub_command in ('continuous', 'cli', 'mermaid') except (IndexError, AssertionError): print( 'Usage:\n' ' uv run -m pydantic_ai_examples.question_graph mermaid\n' 'or:\n' ' uv run -m pydantic_ai_examples.question_graph continuous\n' 'or:\n' ' uv run -m pydantic_ai_examples.question_graph cli [answer]', file=sys.stderr, ) sys.exit(1) if sub_command == 'mermaid': print(question_graph.mermaid_code(start_node=Ask)) elif sub_command == 'continuous': asyncio.run(run_as_continuous()) else: a = sys.argv[2] if len(sys.argv) > 2 else None asyncio.run(run_as_cli(a))` The mermaid diagram generated in this example looks like this: --- title: question_graph --- stateDiagram-v2 [*] --> Ask Ask --> Answer: ask the question Answer --> Evaluate: answer the question Evaluate --> Congratulate Evaluate --> Castigate Congratulate --> [*]: success Castigate --> Ask: try again --- # RAG - PydanticAI RAG === RAG search example. This demo allows you to ask question of the [logfire](https://pydantic.dev/logfire) documentation. Demonstrates: * [tools](../../tools/) * [agent dependencies](../../dependencies/) * RAG search This is done by creating a database containing each section of the markdown documentation, then registering the search tool with the PydanticAI agent. Logic for extracting sections from markdown files and a JSON file with that data is available in [this gist](https://gist.github.com/samuelcolvin/4b5bb9bb163b1122ff17e29e48c10992) . [PostgreSQL with pgvector](https://github.com/pgvector/pgvector) is used as the search database, the easiest way to download and run pgvector is using Docker: ``mkdir postgres-data docker run --rm \ -e POSTGRES_PASSWORD=postgres \ -p 54320:5432 \ -v `pwd`/postgres-data:/var/lib/postgresql/data \ pgvector/pgvector:pg17`` As with the [SQL gen](../sql-gen/) example, we run postgres on port `54320` to avoid conflicts with any other postgres instances you may have running. We also mount the PostgreSQL `data` directory locally to persist the data if you need to stop and restart the container. With that running and [dependencies installed and environment variables set](../#usage) , we can build the search database with (**WARNING**: this requires the `OPENAI_API_KEY` env variable and will calling the OpenAI embedding API around 300 times to generate embeddings for each section of the documentation): pipuv `python -m pydantic_ai_examples.rag build` `uv run -m pydantic_ai_examples.rag build` (Note building the database doesn't use PydanticAI right now, instead it uses the OpenAI SDK directly.) You can then ask the agent a question with: pipuv `python -m pydantic_ai_examples.rag search "How do I configure logfire to work with FastAPI?"` `uv run -m pydantic_ai_examples.rag search "How do I configure logfire to work with FastAPI?"` Example Code ------------ rag.py ``from __future__ import annotations as _annotations import asyncio import re import sys import unicodedata from contextlib import asynccontextmanager from dataclasses import dataclass import asyncpg import httpx import logfire import pydantic_core from openai import AsyncOpenAI from pydantic import TypeAdapter from typing_extensions import AsyncGenerator from pydantic_ai import RunContext from pydantic_ai.agent import Agent # 'if-token-present' means nothing will be sent (and the example will work) if you don't have logfire configured logfire.configure(send_to_logfire='if-token-present') logfire.instrument_asyncpg() @dataclass class Deps: openai: AsyncOpenAI pool: asyncpg.Pool agent = Agent('openai:gpt-4o', deps_type=Deps, instrument=True) @agent.tool async def retrieve(context: RunContext[Deps], search_query: str) -> str: """Retrieve documentation sections based on a search query. Args: context: The call context. search_query: The search query. """ with logfire.span( 'create embedding for {search_query=}', search_query=search_query ): embedding = await context.deps.openai.embeddings.create( input=search_query, model='text-embedding-3-small', ) assert len(embedding.data) == 1, ( f'Expected 1 embedding, got {len(embedding.data)}, doc query: {search_query!r}' ) embedding = embedding.data[0].embedding embedding_json = pydantic_core.to_json(embedding).decode() rows = await context.deps.pool.fetch( 'SELECT url, title, content FROM doc_sections ORDER BY embedding <-> $1 LIMIT 8', embedding_json, ) return '\n\n'.join( f'# {row["title"]}\nDocumentation URL:{row["url"]}\n\n{row["content"]}\n' for row in rows ) async def run_agent(question: str): """Entry point to run the agent and perform RAG based question answering.""" openai = AsyncOpenAI() logfire.instrument_openai(openai) logfire.info('Asking "{question}"', question=question) async with database_connect(False) as pool: deps = Deps(openai=openai, pool=pool) answer = await agent.run(question, deps=deps) print(answer.data) ####################################################### # The rest of this file is dedicated to preparing the # # search database, and some utilities. # ####################################################### # JSON document from # https://gist.github.com/samuelcolvin/4b5bb9bb163b1122ff17e29e48c10992 DOCS_JSON = ( 'https://gist.githubusercontent.com/' 'samuelcolvin/4b5bb9bb163b1122ff17e29e48c10992/raw/' '80c5925c42f1442c24963aaf5eb1a324d47afe95/logfire_docs.json' ) async def build_search_db(): """Build the search database.""" async with httpx.AsyncClient() as client: response = await client.get(DOCS_JSON) response.raise_for_status() sections = sessions_ta.validate_json(response.content) openai = AsyncOpenAI() logfire.instrument_openai(openai) async with database_connect(True) as pool: with logfire.span('create schema'): async with pool.acquire() as conn: async with conn.transaction(): await conn.execute(DB_SCHEMA) sem = asyncio.Semaphore(10) async with asyncio.TaskGroup() as tg: for section in sections: tg.create_task(insert_doc_section(sem, openai, pool, section)) async def insert_doc_section( sem: asyncio.Semaphore, openai: AsyncOpenAI, pool: asyncpg.Pool, section: DocsSection, ) -> None: async with sem: url = section.url() exists = await pool.fetchval('SELECT 1 FROM doc_sections WHERE url = $1', url) if exists: logfire.info('Skipping {url=}', url=url) return with logfire.span('create embedding for {url=}', url=url): embedding = await openai.embeddings.create( input=section.embedding_content(), model='text-embedding-3-small', ) assert len(embedding.data) == 1, ( f'Expected 1 embedding, got {len(embedding.data)}, doc section: {section}' ) embedding = embedding.data[0].embedding embedding_json = pydantic_core.to_json(embedding).decode() await pool.execute( 'INSERT INTO doc_sections (url, title, content, embedding) VALUES ($1, $2, $3, $4)', url, section.title, section.content, embedding_json, ) @dataclass class DocsSection: id: int parent: int | None path: str level: int title: str content: str def url(self) -> str: url_path = re.sub(r'\.md$', '', self.path) return ( f'https://logfire.pydantic.dev/docs/{url_path}/#{slugify(self.title, "-")}' ) def embedding_content(self) -> str: return '\n\n'.join((f'path: {self.path}', f'title: {self.title}', self.content)) sessions_ta = TypeAdapter(list[DocsSection]) # pyright: reportUnknownMemberType=false # pyright: reportUnknownVariableType=false @asynccontextmanager async def database_connect( create_db: bool = False, ) -> AsyncGenerator[asyncpg.Pool, None]: server_dsn, database = ( 'postgresql://postgres:postgres@localhost:54320', 'pydantic_ai_rag', ) if create_db: with logfire.span('check and create DB'): conn = await asyncpg.connect(server_dsn) try: db_exists = await conn.fetchval( 'SELECT 1 FROM pg_database WHERE datname = $1', database ) if not db_exists: await conn.execute(f'CREATE DATABASE {database}') finally: await conn.close() pool = await asyncpg.create_pool(f'{server_dsn}/{database}') try: yield pool finally: await pool.close() DB_SCHEMA = """ CREATE EXTENSION IF NOT EXISTS vector; CREATE TABLE IF NOT EXISTS doc_sections ( id serial PRIMARY KEY, url text NOT NULL UNIQUE, title text NOT NULL, content text NOT NULL, -- text-embedding-3-small returns a vector of 1536 floats embedding vector(1536) NOT NULL ); CREATE INDEX IF NOT EXISTS idx_doc_sections_embedding ON doc_sections USING hnsw (embedding vector_l2_ops); """ def slugify(value: str, separator: str, unicode: bool = False) -> str: """Slugify a string, to make it URL friendly.""" # Taken unchanged from https://github.com/Python-Markdown/markdown/blob/3.7/markdown/extensions/toc.py#L38 if not unicode: # Replace Extended Latin characters with ASCII, i.e. `žlutý` => `zluty` value = unicodedata.normalize('NFKD', value) value = value.encode('ascii', 'ignore').decode('ascii') value = re.sub(r'[^\w\s-]', '', value).strip().lower() return re.sub(rf'[{separator}\s]+', separator, value) if __name__ == '__main__': action = sys.argv[1] if len(sys.argv) > 1 else None if action == 'build': asyncio.run(build_search_db()) elif action == 'search': if len(sys.argv) == 3: q = sys.argv[2] else: q = 'How do I configure logfire to work with FastAPI?' asyncio.run(run_agent(q)) else: print( 'uv run --extra examples -m pydantic_ai_examples.rag build|search', file=sys.stderr, ) sys.exit(1)`` --- # Chat App with FastAPI - PydanticAI Chat App with FastAPI ===================== Simple chat app example build with FastAPI. Demonstrates: * [reusing chat history](../../message-history/) * [serializing messages](../../message-history/#accessing-messages-from-results) * [streaming responses](../../results/#streamed-results) This demonstrates storing chat history between requests and using it to give the model context for new responses. Most of the complex logic here is between `chat_app.py` which streams the response to the browser, and `chat_app.ts` which renders messages in the browser. Running the Example ------------------- With [dependencies installed and environment variables set](../#usage) , run: pipuv `python -m pydantic_ai_examples.chat_app` `uv run -m pydantic_ai_examples.chat_app` Then open the app at [localhost:8000](http://localhost:8000) . [![Example conversation](../../img/chat-app-example.png)](../../img/chat-app-example.png) Example Code ------------ Python code that runs the chat app: chat\_app.py ``from __future__ import annotations as _annotations import asyncio import json import sqlite3 from collections.abc import AsyncIterator from concurrent.futures.thread import ThreadPoolExecutor from contextlib import asynccontextmanager from dataclasses import dataclass from datetime import datetime, timezone from functools import partial from pathlib import Path from typing import Annotated, Any, Callable, Literal, TypeVar import fastapi import logfire from fastapi import Depends, Request from fastapi.responses import FileResponse, Response, StreamingResponse from typing_extensions import LiteralString, ParamSpec, TypedDict from pydantic_ai import Agent from pydantic_ai.exceptions import UnexpectedModelBehavior from pydantic_ai.messages import ( ModelMessage, ModelMessagesTypeAdapter, ModelRequest, ModelResponse, TextPart, UserPromptPart, ) # 'if-token-present' means nothing will be sent (and the example will work) if you don't have logfire configured logfire.configure(send_to_logfire='if-token-present') agent = Agent('openai:gpt-4o', instrument=True) THIS_DIR = Path(__file__).parent @asynccontextmanager async def lifespan(_app: fastapi.FastAPI): async with Database.connect() as db: yield {'db': db} app = fastapi.FastAPI(lifespan=lifespan) logfire.instrument_fastapi(app) @app.get('/') async def index() -> FileResponse: return FileResponse((THIS_DIR / 'chat_app.html'), media_type='text/html') @app.get('/chat_app.ts') async def main_ts() -> FileResponse: """Get the raw typescript code, it's compiled in the browser, forgive me.""" return FileResponse((THIS_DIR / 'chat_app.ts'), media_type='text/plain') async def get_db(request: Request) -> Database: return request.state.db @app.get('/chat/') async def get_chat(database: Database = Depends(get_db)) -> Response: msgs = await database.get_messages() return Response( b'\n'.join(json.dumps(to_chat_message(m)).encode('utf-8') for m in msgs), media_type='text/plain', ) class ChatMessage(TypedDict): """Format of messages sent to the browser.""" role: Literal['user', 'model'] timestamp: str content: str def to_chat_message(m: ModelMessage) -> ChatMessage: first_part = m.parts[0] if isinstance(m, ModelRequest): if isinstance(first_part, UserPromptPart): assert isinstance(first_part.content, str) return { 'role': 'user', 'timestamp': first_part.timestamp.isoformat(), 'content': first_part.content, } elif isinstance(m, ModelResponse): if isinstance(first_part, TextPart): return { 'role': 'model', 'timestamp': m.timestamp.isoformat(), 'content': first_part.content, } raise UnexpectedModelBehavior(f'Unexpected message type for chat app: {m}') @app.post('/chat/') async def post_chat( prompt: Annotated[str, fastapi.Form()], database: Database = Depends(get_db) ) -> StreamingResponse: async def stream_messages(): """Streams new line delimited JSON `Message`s to the client.""" # stream the user prompt so that can be displayed straight away yield ( json.dumps( { 'role': 'user', 'timestamp': datetime.now(tz=timezone.utc).isoformat(), 'content': prompt, } ).encode('utf-8') + b'\n' ) # get the chat history so far to pass as context to the agent messages = await database.get_messages() # run the agent with the user prompt and the chat history async with agent.run_stream(prompt, message_history=messages) as result: async for text in result.stream(debounce_by=0.01): # text here is a `str` and the frontend wants # JSON encoded ModelResponse, so we create one m = ModelResponse(parts=[TextPart(text)], timestamp=result.timestamp()) yield json.dumps(to_chat_message(m)).encode('utf-8') + b'\n' # add new messages (e.g. the user prompt and the agent response in this case) to the database await database.add_messages(result.new_messages_json()) return StreamingResponse(stream_messages(), media_type='text/plain') P = ParamSpec('P') R = TypeVar('R') @dataclass class Database: """Rudimentary database to store chat messages in SQLite. The SQLite standard library package is synchronous, so we use a thread pool executor to run queries asynchronously. """ con: sqlite3.Connection _loop: asyncio.AbstractEventLoop _executor: ThreadPoolExecutor @classmethod @asynccontextmanager async def connect( cls, file: Path = THIS_DIR / '.chat_app_messages.sqlite' ) -> AsyncIterator[Database]: with logfire.span('connect to DB'): loop = asyncio.get_event_loop() executor = ThreadPoolExecutor(max_workers=1) con = await loop.run_in_executor(executor, cls._connect, file) slf = cls(con, loop, executor) try: yield slf finally: await slf._asyncify(con.close) @staticmethod def _connect(file: Path) -> sqlite3.Connection: con = sqlite3.connect(str(file)) con = logfire.instrument_sqlite3(con) cur = con.cursor() cur.execute( 'CREATE TABLE IF NOT EXISTS messages (id INT PRIMARY KEY, message_list TEXT);' ) con.commit() return con async def add_messages(self, messages: bytes): await self._asyncify( self._execute, 'INSERT INTO messages (message_list) VALUES (?);', messages, commit=True, ) await self._asyncify(self.con.commit) async def get_messages(self) -> list[ModelMessage]: c = await self._asyncify( self._execute, 'SELECT message_list FROM messages order by id' ) rows = await self._asyncify(c.fetchall) messages: list[ModelMessage] = [] for row in rows: messages.extend(ModelMessagesTypeAdapter.validate_json(row[0])) return messages def _execute( self, sql: LiteralString, *args: Any, commit: bool = False ) -> sqlite3.Cursor: cur = self.con.cursor() cur.execute(sql, args) if commit: self.con.commit() return cur async def _asyncify( self, func: Callable[P, R], *args: P.args, **kwargs: P.kwargs ) -> R: return await self._loop.run_in_executor( # type: ignore self._executor, partial(func, **kwargs), *args, # type: ignore ) if __name__ == '__main__': import uvicorn uvicorn.run( 'pydantic_ai_examples.chat_app:app', reload=True, reload_dirs=[str(THIS_DIR)] )`` Simple HTML page to render the app: chat\_app.html ` Chat App

Chat App

Ask me anything...

Error occurred, check the browser developer console for more information.
` TypeScript to handle rendering the messages, to keep this simple (and at the risk of offending frontend developers) the typescript code is passed to the browser as plain text and transpiled in the browser. chat\_app.ts ``// BIG FAT WARNING: to avoid the complexity of npm, this typescript is compiled in the browser // there's currently no static type checking import { marked } from 'https://cdnjs.cloudflare.com/ajax/libs/marked/15.0.0/lib/marked.esm.js' const convElement = document.getElementById('conversation') const promptInput = document.getElementById('prompt-input') as HTMLInputElement const spinner = document.getElementById('spinner') // stream the response and render messages as each chunk is received // data is sent as newline-delimited JSON async function onFetchResponse(response: Response): Promise { let text = '' let decoder = new TextDecoder() if (response.ok) { const reader = response.body.getReader() while (true) { const {done, value} = await reader.read() if (done) { break } text += decoder.decode(value) addMessages(text) spinner.classList.remove('active') } addMessages(text) promptInput.disabled = false promptInput.focus() } else { const text = await response.text() console.error(`Unexpected response: ${response.status}`, {response, text}) throw new Error(`Unexpected response: ${response.status}`) } } // The format of messages, this matches pydantic-ai both for brevity and understanding // in production, you might not want to keep this format all the way to the frontend interface Message { role: string content: string timestamp: string } // take raw response text and render messages into the `#conversation` element // Message timestamp is assumed to be a unique identifier of a message, and is used to deduplicate // hence you can send data about the same message multiple times, and it will be updated // instead of creating a new message elements function addMessages(responseText: string) { const lines = responseText.split('\n') const messages: Message[] = lines.filter(line => line.length > 1).map(j => JSON.parse(j)) for (const message of messages) { // we use the timestamp as a crude element id const {timestamp, role, content} = message const id = `msg-${timestamp}` let msgDiv = document.getElementById(id) if (!msgDiv) { msgDiv = document.createElement('div') msgDiv.id = id msgDiv.title = `${role} at ${timestamp}` msgDiv.classList.add('border-top', 'pt-2', role) convElement.appendChild(msgDiv) } msgDiv.innerHTML = marked.parse(content) } window.scrollTo({ top: document.body.scrollHeight, behavior: 'smooth' }) } function onError(error: any) { console.error(error) document.getElementById('error').classList.remove('d-none') document.getElementById('spinner').classList.remove('active') } async function onSubmit(e: SubmitEvent): Promise { e.preventDefault() spinner.classList.add('active') const body = new FormData(e.target as HTMLFormElement) promptInput.value = '' promptInput.disabled = true const response = await fetch('/chat/', {method: 'POST', body}) await onFetchResponse(response) } // call onSubmit when the form is submitted (e.g. user clicks the send button or hits Enter) document.querySelector('form').addEventListener('submit', (e) => onSubmit(e).catch(onError)) // load messages on page load fetch('/chat/').then(onFetchResponse).catch(onError)`` --- # pydantic_ai.result - PydanticAI `pydantic_ai.result` ==================== ### ResultDataT `module-attribute` `ResultDataT = TypeVar( "ResultDataT", default=[str](https://docs.python.org/3/library/stdtypes.html#str) , covariant=True )` Covariant type variable for the result data type of a run. ### StreamedRunResult `dataclass` Bases: `[Generic](https://docs.python.org/3/library/typing.html#typing.Generic "typing.Generic") [[AgentDepsT](../tools/#pydantic_ai.tools.AgentDepsT "pydantic_ai.tools.AgentDepsT") , [ResultDataT](#pydantic_ai.result.ResultDataT "pydantic_ai.result.ResultDataT") ]` Result of a streamed run that returns structured data via a tool call. Source code in `pydantic_ai_slim/pydantic_ai/result.py` | | | | --- | --- | | 173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458 | ``@dataclass class StreamedRunResult(Generic[AgentDepsT, ResultDataT]): """Result of a streamed run that returns structured data via a tool call.""" _all_messages: list[_messages.ModelMessage] _new_message_index: int _usage_limits: UsageLimits \| None _stream_response: models.StreamedResponse _result_schema: _result.ResultSchema[ResultDataT] \| None _run_ctx: RunContext[AgentDepsT] _result_validators: list[_result.ResultValidator[AgentDepsT, ResultDataT]] _result_tool_name: str \| None _on_complete: Callable[[], Awaitable[None]] _initial_run_ctx_usage: Usage = field(init=False) is_complete: bool = field(default=False, init=False) """Whether the stream has all been received. This is set to `True` when one of [`stream`][pydantic_ai.result.StreamedRunResult.stream], [`stream_text`][pydantic_ai.result.StreamedRunResult.stream_text], [`stream_structured`][pydantic_ai.result.StreamedRunResult.stream_structured] or [`get_data`][pydantic_ai.result.StreamedRunResult.get_data] completes. """ def __post_init__(self): self._initial_run_ctx_usage = copy(self._run_ctx.usage) def all_messages(self, *, result_tool_return_content: str \| None = None) -> list[_messages.ModelMessage]: """Return the history of _messages. Args: result_tool_return_content: The return content of the tool call to set in the last message. This provides a convenient way to modify the content of the result tool call if you want to continue the conversation and want to set the response to the result tool call. If `None`, the last message will not be modified. Returns: List of messages. """ # this is a method to be consistent with the other methods if result_tool_return_content is not None: raise NotImplementedError('Setting result tool return content is not supported for this result type.') return self._all_messages def all_messages_json(self, *, result_tool_return_content: str \| None = None) -> bytes: """Return all messages from [`all_messages`][pydantic_ai.result.StreamedRunResult.all_messages] as JSON bytes. Args: result_tool_return_content: The return content of the tool call to set in the last message. This provides a convenient way to modify the content of the result tool call if you want to continue the conversation and want to set the response to the result tool call. If `None`, the last message will not be modified. Returns: JSON bytes representing the messages. """ return _messages.ModelMessagesTypeAdapter.dump_json( self.all_messages(result_tool_return_content=result_tool_return_content) ) def new_messages(self, *, result_tool_return_content: str \| None = None) -> list[_messages.ModelMessage]: """Return new messages associated with this run. Messages from older runs are excluded. Args: result_tool_return_content: The return content of the tool call to set in the last message. This provides a convenient way to modify the content of the result tool call if you want to continue the conversation and want to set the response to the result tool call. If `None`, the last message will not be modified. Returns: List of new messages. """ return self.all_messages(result_tool_return_content=result_tool_return_content)[self._new_message_index :] def new_messages_json(self, *, result_tool_return_content: str \| None = None) -> bytes: """Return new messages from [`new_messages`][pydantic_ai.result.StreamedRunResult.new_messages] as JSON bytes. Args: result_tool_return_content: The return content of the tool call to set in the last message. This provides a convenient way to modify the content of the result tool call if you want to continue the conversation and want to set the response to the result tool call. If `None`, the last message will not be modified. Returns: JSON bytes representing the new messages. """ return _messages.ModelMessagesTypeAdapter.dump_json( self.new_messages(result_tool_return_content=result_tool_return_content) ) async def stream(self, *, debounce_by: float \| None = 0.1) -> AsyncIterator[ResultDataT]: """Stream the response as an async iterable. The pydantic validator for structured data will be called in [partial mode](https://docs.pydantic.dev/dev/concepts/experimental/#partial-validation) on each iteration. Args: debounce_by: by how much (if at all) to debounce/group the response chunks by. `None` means no debouncing. Debouncing is particularly important for long structured responses to reduce the overhead of performing validation as each token is received. Returns: An async iterable of the response data. """ async for structured_message, is_last in self.stream_structured(debounce_by=debounce_by): result = await self.validate_structured_result(structured_message, allow_partial=not is_last) yield result async def stream_text(self, *, delta: bool = False, debounce_by: float \| None = 0.1) -> AsyncIterator[str]: """Stream the text result as an async iterable. !!! note Result validators will NOT be called on the text result if `delta=True`. Args: delta: if `True`, yield each chunk of text as it is received, if `False` (default), yield the full text up to the current point. debounce_by: by how much (if at all) to debounce/group the response chunks by. `None` means no debouncing. Debouncing is particularly important for long structured responses to reduce the overhead of performing validation as each token is received. """ if self._result_schema and not self._result_schema.allow_text_result: raise exceptions.UserError('stream_text() can only be used with text responses') if delta: async for text in self._stream_response_text(delta=delta, debounce_by=debounce_by): yield text else: async for text in self._stream_response_text(delta=delta, debounce_by=debounce_by): combined_validated_text = await self._validate_text_result(text) yield combined_validated_text await self._marked_completed(self._stream_response.get()) async def stream_structured( self, *, debounce_by: float \| None = 0.1 ) -> AsyncIterator[tuple[_messages.ModelResponse, bool]]: """Stream the response as an async iterable of Structured LLM Messages. Args: debounce_by: by how much (if at all) to debounce/group the response chunks by. `None` means no debouncing. Debouncing is particularly important for long structured responses to reduce the overhead of performing validation as each token is received. Returns: An async iterable of the structured response message and whether that is the last message. """ # if the message currently has any parts with content, yield before streaming msg = self._stream_response.get() for part in msg.parts: if part.has_content(): yield msg, False break async for msg in self._stream_response_structured(debounce_by=debounce_by): yield msg, False msg = self._stream_response.get() yield msg, True await self._marked_completed(msg) async def get_data(self) -> ResultDataT: """Stream the whole response, validate and return it.""" usage_checking_stream = _get_usage_checking_stream_response( self._stream_response, self._usage_limits, self.usage ) async for _ in usage_checking_stream: pass message = self._stream_response.get() await self._marked_completed(message) return await self.validate_structured_result(message) def usage(self) -> Usage: """Return the usage of the whole run. !!! note This won't return the full usage until the stream is finished. """ return self._initial_run_ctx_usage + self._stream_response.usage() def timestamp(self) -> datetime: """Get the timestamp of the response.""" return self._stream_response.timestamp async def validate_structured_result( self, message: _messages.ModelResponse, *, allow_partial: bool = False ) -> ResultDataT: """Validate a structured result message.""" if self._result_schema is not None and self._result_tool_name is not None: match = self._result_schema.find_named_tool(message.parts, self._result_tool_name) if match is None: raise exceptions.UnexpectedModelBehavior( f'Invalid response, unable to find tool: {self._result_schema.tool_names()}' ) call, result_tool = match result_data = result_tool.validate(call, allow_partial=allow_partial, wrap_validation_errors=False) for validator in self._result_validators: result_data = await validator.validate(result_data, call, self._run_ctx) return result_data else: text = '\n\n'.join(x.content for x in message.parts if isinstance(x, _messages.TextPart)) for validator in self._result_validators: text = await validator.validate( text, None, self._run_ctx, ) # Since there is no result tool, we can assume that str is compatible with ResultDataT return cast(ResultDataT, text) async def _validate_text_result(self, text: str) -> str: for validator in self._result_validators: text = await validator.validate( text, None, self._run_ctx, ) return text async def _marked_completed(self, message: _messages.ModelResponse) -> None: self.is_complete = True self._all_messages.append(message) await self._on_complete() async def _stream_response_structured( self, *, debounce_by: float \| None = 0.1 ) -> AsyncIterator[_messages.ModelResponse]: async with _utils.group_by_temporal(self._stream_response, debounce_by) as group_iter: async for _items in group_iter: yield self._stream_response.get() async def _stream_response_text( self, *, delta: bool = False, debounce_by: float \| None = 0.1 ) -> AsyncIterator[str]: """Stream the response as an async iterable of text.""" # Define a "merged" version of the iterator that will yield items that have already been retrieved # and items that we receive while streaming. We define a dedicated async iterator for this so we can # pass the combined stream to the group_by_temporal function within `_stream_text_deltas` below. async def _stream_text_deltas_ungrouped() -> AsyncIterator[tuple[str, int]]: # yields tuples of (text_content, part_index) # we don't currently make use of the part_index, but in principle this may be useful # so we retain it here for now to make possible future refactors simpler msg = self._stream_response.get() for i, part in enumerate(msg.parts): if isinstance(part, _messages.TextPart) and part.content: yield part.content, i async for event in self._stream_response: if ( isinstance(event, _messages.PartStartEvent) and isinstance(event.part, _messages.TextPart) and event.part.content ): yield event.part.content, event.index elif ( isinstance(event, _messages.PartDeltaEvent) and isinstance(event.delta, _messages.TextPartDelta) and event.delta.content_delta ): yield event.delta.content_delta, event.index async def _stream_text_deltas() -> AsyncIterator[str]: async with _utils.group_by_temporal(_stream_text_deltas_ungrouped(), debounce_by) as group_iter: async for items in group_iter: # Note: we are currently just dropping the part index on the group here yield ''.join([content for content, _ in items]) if delta: async for text in _stream_text_deltas(): yield text else: # a quick benchmark shows it's faster to build up a string with concat when we're # yielding at each step deltas: list[str] = [] async for text in _stream_text_deltas(): deltas.append(text) yield ''.join(deltas)`` | #### is\_complete `class-attribute` `instance-attribute` `is_complete: [bool](https://docs.python.org/3/library/functions.html#bool) = [field](https://docs.python.org/3/library/dataclasses.html#dataclasses.field "dataclasses.field") (default=False, init=False)` Whether the stream has all been received. This is set to `True` when one of [`stream`](#pydantic_ai.result.StreamedRunResult.stream) , [`stream_text`](#pydantic_ai.result.StreamedRunResult.stream_text) , [`stream_structured`](#pydantic_ai.result.StreamedRunResult.stream_structured) or [`get_data`](#pydantic_ai.result.StreamedRunResult.get_data) completes. #### all\_messages `all_messages( *, result_tool_return_content: [str](https://docs.python.org/3/library/stdtypes.html#str) | None = None ) -> [list](https://docs.python.org/3/library/stdtypes.html#list) [[ModelMessage](../messages/#pydantic_ai.messages.ModelMessage "pydantic_ai.messages.ModelMessage") ]` Return the history of \_messages. Parameters: | Name | Type | Description | Default | | --- | --- | --- | --- | | `result_tool_return_content` | `[str](https://docs.python.org/3/library/stdtypes.html#str) \| None` | The return content of the tool call to set in the last message. This provides a convenient way to modify the content of the result tool call if you want to continue the conversation and want to set the response to the result tool call. If `None`, the last message will not be modified. | `None` | Returns: | Type | Description | | --- | --- | | `[list](https://docs.python.org/3/library/stdtypes.html#list) [[ModelMessage](../messages/#pydantic_ai.messages.ModelMessage "pydantic_ai.messages.ModelMessage") ]` | List of messages. | Source code in `pydantic_ai_slim/pydantic_ai/result.py` | | | | --- | --- | | 202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217 | ``def all_messages(self, *, result_tool_return_content: str \| None = None) -> list[_messages.ModelMessage]: """Return the history of _messages. Args: result_tool_return_content: The return content of the tool call to set in the last message. This provides a convenient way to modify the content of the result tool call if you want to continue the conversation and want to set the response to the result tool call. If `None`, the last message will not be modified. Returns: List of messages. """ # this is a method to be consistent with the other methods if result_tool_return_content is not None: raise NotImplementedError('Setting result tool return content is not supported for this result type.') return self._all_messages`` | #### all\_messages\_json `all_messages_json( *, result_tool_return_content: [str](https://docs.python.org/3/library/stdtypes.html#str) | None = None ) -> [bytes](https://docs.python.org/3/library/stdtypes.html#bytes)` Return all messages from [`all_messages`](#pydantic_ai.result.StreamedRunResult.all_messages) as JSON bytes. Parameters: | Name | Type | Description | Default | | --- | --- | --- | --- | | `result_tool_return_content` | `[str](https://docs.python.org/3/library/stdtypes.html#str) \| None` | The return content of the tool call to set in the last message. This provides a convenient way to modify the content of the result tool call if you want to continue the conversation and want to set the response to the result tool call. If `None`, the last message will not be modified. | `None` | Returns: | Type | Description | | --- | --- | | `[bytes](https://docs.python.org/3/library/stdtypes.html#bytes) ` | JSON bytes representing the messages. | Source code in `pydantic_ai_slim/pydantic_ai/result.py` | | | | --- | --- | | 219
220
221
222
223
224
225
226
227
228
229
230
231
232
233 | ``def all_messages_json(self, *, result_tool_return_content: str \| None = None) -> bytes: """Return all messages from [`all_messages`][pydantic_ai.result.StreamedRunResult.all_messages] as JSON bytes. Args: result_tool_return_content: The return content of the tool call to set in the last message. This provides a convenient way to modify the content of the result tool call if you want to continue the conversation and want to set the response to the result tool call. If `None`, the last message will not be modified. Returns: JSON bytes representing the messages. """ return _messages.ModelMessagesTypeAdapter.dump_json( self.all_messages(result_tool_return_content=result_tool_return_content) )`` | #### new\_messages `new_messages( *, result_tool_return_content: [str](https://docs.python.org/3/library/stdtypes.html#str) | None = None ) -> [list](https://docs.python.org/3/library/stdtypes.html#list) [[ModelMessage](../messages/#pydantic_ai.messages.ModelMessage "pydantic_ai.messages.ModelMessage") ]` Return new messages associated with this run. Messages from older runs are excluded. Parameters: | Name | Type | Description | Default | | --- | --- | --- | --- | | `result_tool_return_content` | `[str](https://docs.python.org/3/library/stdtypes.html#str) \| None` | The return content of the tool call to set in the last message. This provides a convenient way to modify the content of the result tool call if you want to continue the conversation and want to set the response to the result tool call. If `None`, the last message will not be modified. | `None` | Returns: | Type | Description | | --- | --- | | `[list](https://docs.python.org/3/library/stdtypes.html#list) [[ModelMessage](../messages/#pydantic_ai.messages.ModelMessage "pydantic_ai.messages.ModelMessage") ]` | List of new messages. | Source code in `pydantic_ai_slim/pydantic_ai/result.py` | | | | --- | --- | | 235
236
237
238
239
240
241
242
243
244
245
246
247
248
249 | ``def new_messages(self, *, result_tool_return_content: str \| None = None) -> list[_messages.ModelMessage]: """Return new messages associated with this run. Messages from older runs are excluded. Args: result_tool_return_content: The return content of the tool call to set in the last message. This provides a convenient way to modify the content of the result tool call if you want to continue the conversation and want to set the response to the result tool call. If `None`, the last message will not be modified. Returns: List of new messages. """ return self.all_messages(result_tool_return_content=result_tool_return_content)[self._new_message_index :]`` | #### new\_messages\_json `new_messages_json( *, result_tool_return_content: [str](https://docs.python.org/3/library/stdtypes.html#str) | None = None ) -> [bytes](https://docs.python.org/3/library/stdtypes.html#bytes)` Return new messages from [`new_messages`](#pydantic_ai.result.StreamedRunResult.new_messages) as JSON bytes. Parameters: | Name | Type | Description | Default | | --- | --- | --- | --- | | `result_tool_return_content` | `[str](https://docs.python.org/3/library/stdtypes.html#str) \| None` | The return content of the tool call to set in the last message. This provides a convenient way to modify the content of the result tool call if you want to continue the conversation and want to set the response to the result tool call. If `None`, the last message will not be modified. | `None` | Returns: | Type | Description | | --- | --- | | `[bytes](https://docs.python.org/3/library/stdtypes.html#bytes) ` | JSON bytes representing the new messages. | Source code in `pydantic_ai_slim/pydantic_ai/result.py` | | | | --- | --- | | 251
252
253
254
255
256
257
258
259
260
261
262
263
264
265 | ``def new_messages_json(self, *, result_tool_return_content: str \| None = None) -> bytes: """Return new messages from [`new_messages`][pydantic_ai.result.StreamedRunResult.new_messages] as JSON bytes. Args: result_tool_return_content: The return content of the tool call to set in the last message. This provides a convenient way to modify the content of the result tool call if you want to continue the conversation and want to set the response to the result tool call. If `None`, the last message will not be modified. Returns: JSON bytes representing the new messages. """ return _messages.ModelMessagesTypeAdapter.dump_json( self.new_messages(result_tool_return_content=result_tool_return_content) )`` | #### stream `async` `stream( *, debounce_by: [float](https://docs.python.org/3/library/functions.html#float) | None = 0.1 ) -> [AsyncIterator](https://docs.python.org/3/library/collections.abc.html#collections.abc.AsyncIterator "collections.abc.AsyncIterator") [[ResultDataT](#pydantic_ai.result.ResultDataT "pydantic_ai.result.ResultDataT") ]` Stream the response as an async iterable. The pydantic validator for structured data will be called in [partial mode](https://docs.pydantic.dev/dev/concepts/experimental/#partial-validation) on each iteration. Parameters: | Name | Type | Description | Default | | --- | --- | --- | --- | | `debounce_by` | `[float](https://docs.python.org/3/library/functions.html#float) \| None` | by how much (if at all) to debounce/group the response chunks by. `None` means no debouncing. Debouncing is particularly important for long structured responses to reduce the overhead of performing validation as each token is received. | `0.1` | Returns: | Type | Description | | --- | --- | | `[AsyncIterator](https://docs.python.org/3/library/collections.abc.html#collections.abc.AsyncIterator "collections.abc.AsyncIterator") [[ResultDataT](#pydantic_ai.result.ResultDataT "pydantic_ai.result.ResultDataT") ]` | An async iterable of the response data. | Source code in `pydantic_ai_slim/pydantic_ai/result.py` | | | | --- | --- | | 267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284 | ``async def stream(self, *, debounce_by: float \| None = 0.1) -> AsyncIterator[ResultDataT]: """Stream the response as an async iterable. The pydantic validator for structured data will be called in [partial mode](https://docs.pydantic.dev/dev/concepts/experimental/#partial-validation) on each iteration. Args: debounce_by: by how much (if at all) to debounce/group the response chunks by. `None` means no debouncing. Debouncing is particularly important for long structured responses to reduce the overhead of performing validation as each token is received. Returns: An async iterable of the response data. """ async for structured_message, is_last in self.stream_structured(debounce_by=debounce_by): result = await self.validate_structured_result(structured_message, allow_partial=not is_last) yield result`` | #### stream\_text `async` `stream_text( *, delta: [bool](https://docs.python.org/3/library/functions.html#bool) = False, debounce_by: [float](https://docs.python.org/3/library/functions.html#float) | None = 0.1 ) -> [AsyncIterator](https://docs.python.org/3/library/collections.abc.html#collections.abc.AsyncIterator "collections.abc.AsyncIterator") [[str](https://docs.python.org/3/library/stdtypes.html#str) ]` Stream the text result as an async iterable. Note Result validators will NOT be called on the text result if `delta=True`. Parameters: | Name | Type | Description | Default | | --- | --- | --- | --- | | `delta` | `[bool](https://docs.python.org/3/library/functions.html#bool) ` | if `True`, yield each chunk of text as it is received, if `False` (default), yield the full text up to the current point. | `False` | | `debounce_by` | `[float](https://docs.python.org/3/library/functions.html#float) \| None` | by how much (if at all) to debounce/group the response chunks by. `None` means no debouncing. Debouncing is particularly important for long structured responses to reduce the overhead of performing validation as each token is received. | `0.1` | Source code in `pydantic_ai_slim/pydantic_ai/result.py` | | | | --- | --- | | 286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309 | ``async def stream_text(self, *, delta: bool = False, debounce_by: float \| None = 0.1) -> AsyncIterator[str]: """Stream the text result as an async iterable. !!! note Result validators will NOT be called on the text result if `delta=True`. Args: delta: if `True`, yield each chunk of text as it is received, if `False` (default), yield the full text up to the current point. debounce_by: by how much (if at all) to debounce/group the response chunks by. `None` means no debouncing. Debouncing is particularly important for long structured responses to reduce the overhead of performing validation as each token is received. """ if self._result_schema and not self._result_schema.allow_text_result: raise exceptions.UserError('stream_text() can only be used with text responses') if delta: async for text in self._stream_response_text(delta=delta, debounce_by=debounce_by): yield text else: async for text in self._stream_response_text(delta=delta, debounce_by=debounce_by): combined_validated_text = await self._validate_text_result(text) yield combined_validated_text await self._marked_completed(self._stream_response.get())`` | #### stream\_structured `async` `stream_structured( *, debounce_by: [float](https://docs.python.org/3/library/functions.html#float) | None = 0.1 ) -> [AsyncIterator](https://docs.python.org/3/library/collections.abc.html#collections.abc.AsyncIterator "collections.abc.AsyncIterator") [[tuple](https://docs.python.org/3/library/stdtypes.html#tuple) [[ModelResponse](../messages/#pydantic_ai.messages.ModelResponse "pydantic_ai.messages.ModelResponse") , [bool](https://docs.python.org/3/library/functions.html#bool) ]]` Stream the response as an async iterable of Structured LLM Messages. Parameters: | Name | Type | Description | Default | | --- | --- | --- | --- | | `debounce_by` | `[float](https://docs.python.org/3/library/functions.html#float) \| None` | by how much (if at all) to debounce/group the response chunks by. `None` means no debouncing. Debouncing is particularly important for long structured responses to reduce the overhead of performing validation as each token is received. | `0.1` | Returns: | Type | Description | | --- | --- | | `[AsyncIterator](https://docs.python.org/3/library/collections.abc.html#collections.abc.AsyncIterator "collections.abc.AsyncIterator") [[tuple](https://docs.python.org/3/library/stdtypes.html#tuple) [[ModelResponse](../messages/#pydantic_ai.messages.ModelResponse "pydantic_ai.messages.ModelResponse") , [bool](https://docs.python.org/3/library/functions.html#bool) ]]` | An async iterable of the structured response message and whether that is the last message. | Source code in `pydantic_ai_slim/pydantic_ai/result.py` | | | | --- | --- | | 311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337 | ``async def stream_structured( self, *, debounce_by: float \| None = 0.1 ) -> AsyncIterator[tuple[_messages.ModelResponse, bool]]: """Stream the response as an async iterable of Structured LLM Messages. Args: debounce_by: by how much (if at all) to debounce/group the response chunks by. `None` means no debouncing. Debouncing is particularly important for long structured responses to reduce the overhead of performing validation as each token is received. Returns: An async iterable of the structured response message and whether that is the last message. """ # if the message currently has any parts with content, yield before streaming msg = self._stream_response.get() for part in msg.parts: if part.has_content(): yield msg, False break async for msg in self._stream_response_structured(debounce_by=debounce_by): yield msg, False msg = self._stream_response.get() yield msg, True await self._marked_completed(msg)`` | #### get\_data `async` `get_data() -> [ResultDataT](#pydantic_ai.result.ResultDataT "pydantic_ai.result.ResultDataT")` Stream the whole response, validate and return it. Source code in `pydantic_ai_slim/pydantic_ai/result.py` | | | | --- | --- | | 339
340
341
342
343
344
345
346
347
348
349 | `async def get_data(self) -> ResultDataT: """Stream the whole response, validate and return it.""" usage_checking_stream = _get_usage_checking_stream_response( self._stream_response, self._usage_limits, self.usage ) async for _ in usage_checking_stream: pass message = self._stream_response.get() await self._marked_completed(message) return await self.validate_structured_result(message)` | #### usage `usage() -> [Usage](../usage/#pydantic_ai.usage.Usage "pydantic_ai.usage.Usage")` Return the usage of the whole run. Note This won't return the full usage until the stream is finished. Source code in `pydantic_ai_slim/pydantic_ai/result.py` | | | | --- | --- | | 351
352
353
354
355
356
357 | `def usage(self) -> Usage: """Return the usage of the whole run. !!! note This won't return the full usage until the stream is finished. """ return self._initial_run_ctx_usage + self._stream_response.usage()` | #### timestamp `timestamp() -> [datetime](https://docs.python.org/3/library/datetime.html#datetime.datetime "datetime.datetime")` Get the timestamp of the response. Source code in `pydantic_ai_slim/pydantic_ai/result.py` | | | | --- | --- | | 359
360
361 | `def timestamp(self) -> datetime: """Get the timestamp of the response.""" return self._stream_response.timestamp` | #### validate\_structured\_result `async` `validate_structured_result( message: [ModelResponse](../messages/#pydantic_ai.messages.ModelResponse "pydantic_ai.messages.ModelResponse") , *, allow_partial: [bool](https://docs.python.org/3/library/functions.html#bool) = False ) -> [ResultDataT](#pydantic_ai.result.ResultDataT "pydantic_ai.result.ResultDataT")` Validate a structured result message. Source code in `pydantic_ai_slim/pydantic_ai/result.py` | | | | --- | --- | | 363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389 | `async def validate_structured_result( self, message: _messages.ModelResponse, *, allow_partial: bool = False ) -> ResultDataT: """Validate a structured result message.""" if self._result_schema is not None and self._result_tool_name is not None: match = self._result_schema.find_named_tool(message.parts, self._result_tool_name) if match is None: raise exceptions.UnexpectedModelBehavior( f'Invalid response, unable to find tool: {self._result_schema.tool_names()}' ) call, result_tool = match result_data = result_tool.validate(call, allow_partial=allow_partial, wrap_validation_errors=False) for validator in self._result_validators: result_data = await validator.validate(result_data, call, self._run_ctx) return result_data else: text = '\n\n'.join(x.content for x in message.parts if isinstance(x, _messages.TextPart)) for validator in self._result_validators: text = await validator.validate( text, None, self._run_ctx, ) # Since there is no result tool, we can assume that str is compatible with ResultDataT return cast(ResultDataT, text)` | --- # pydantic_ai.tools - PydanticAI `pydantic_ai.tools` =================== ### AgentDepsT `module-attribute` `AgentDepsT = TypeVar( "AgentDepsT", default=None, contravariant=True )` Type variable for agent dependencies. ### RunContext `dataclass` Bases: `[Generic](https://docs.python.org/3/library/typing.html#typing.Generic "typing.Generic") [[AgentDepsT](#pydantic_ai.tools.AgentDepsT "pydantic_ai.tools.AgentDepsT") ]` Information about the current call. Source code in `pydantic_ai_slim/pydantic_ai/tools.py` | | | | --- | --- | | 41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73 | ``@dataclasses.dataclass class RunContext(Generic[AgentDepsT]): """Information about the current call.""" deps: AgentDepsT """Dependencies for the agent.""" model: models.Model """The model used in this run.""" usage: Usage """LLM usage associated with the run.""" prompt: str \| Sequence[_messages.UserContent] """The original user prompt passed to the run.""" messages: list[_messages.ModelMessage] = field(default_factory=list) """Messages exchanged in the conversation so far.""" tool_call_id: str \| None = None """The ID of the tool call.""" tool_name: str \| None = None """Name of the tool being called.""" retry: int = 0 """Number of retries so far.""" run_step: int = 0 """The current step in the run.""" def replace_with( self, retry: int \| None = None, tool_name: str \| None \| _utils.Unset = _utils.UNSET ) -> RunContext[AgentDepsT]: # Create a new `RunContext` a new `retry` value and `tool_name`. kwargs = {} if retry is not None: kwargs['retry'] = retry if tool_name is not _utils.UNSET: kwargs['tool_name'] = tool_name return dataclasses.replace(self, **kwargs)`` | #### deps `instance-attribute` `deps: [AgentDepsT](#pydantic_ai.tools.AgentDepsT "pydantic_ai.tools.AgentDepsT")` Dependencies for the agent. #### model `instance-attribute` `model: [Model](../models/base/#pydantic_ai.models.Model "pydantic_ai.models.Model")` The model used in this run. #### usage `instance-attribute` `usage: [Usage](../usage/#pydantic_ai.usage.Usage "pydantic_ai.result.Usage")` LLM usage associated with the run. #### prompt `instance-attribute` `prompt: [str](https://docs.python.org/3/library/stdtypes.html#str) | [Sequence](https://docs.python.org/3/library/collections.abc.html#collections.abc.Sequence "collections.abc.Sequence") [UserContent]` The original user prompt passed to the run. #### messages `class-attribute` `instance-attribute` `messages: [list](https://docs.python.org/3/library/stdtypes.html#list) [[ModelMessage](../messages/#pydantic_ai.messages.ModelMessage "pydantic_ai.messages.ModelMessage") ] = [field](https://docs.python.org/3/library/dataclasses.html#dataclasses.field "dataclasses.field") (default_factory=[list](https://docs.python.org/3/library/stdtypes.html#list) )` Messages exchanged in the conversation so far. #### tool\_call\_id `class-attribute` `instance-attribute` `tool_call_id: [str](https://docs.python.org/3/library/stdtypes.html#str) | None = None` The ID of the tool call. #### tool\_name `class-attribute` `instance-attribute` `tool_name: [str](https://docs.python.org/3/library/stdtypes.html#str) | None = None` Name of the tool being called. #### retry `class-attribute` `instance-attribute` `retry: [int](https://docs.python.org/3/library/functions.html#int) = 0` Number of retries so far. #### run\_step `class-attribute` `instance-attribute` `run_step: [int](https://docs.python.org/3/library/functions.html#int) = 0` The current step in the run. ### ToolParams `module-attribute` `ToolParams = [ParamSpec](https://typing-extensions.readthedocs.io/en/latest/index.html#typing_extensions.ParamSpec "typing_extensions.ParamSpec") ('ToolParams', default=...)` Retrieval function param spec. ### SystemPromptFunc `module-attribute` `SystemPromptFunc = [Union](https://docs.python.org/3/library/typing.html#typing.Union "typing.Union") [ [Callable](https://docs.python.org/3/library/typing.html#typing.Callable "typing.Callable") [[[RunContext](#pydantic_ai.tools.RunContext "pydantic_ai.tools.RunContext") [[AgentDepsT](#pydantic_ai.tools.AgentDepsT "pydantic_ai.tools.AgentDepsT") ]], [str](https://docs.python.org/3/library/stdtypes.html#str) ], [Callable](https://docs.python.org/3/library/typing.html#typing.Callable "typing.Callable") [[[RunContext](#pydantic_ai.tools.RunContext "pydantic_ai.tools.RunContext") [[AgentDepsT](#pydantic_ai.tools.AgentDepsT "pydantic_ai.tools.AgentDepsT") ]], [Awaitable](https://docs.python.org/3/library/collections.abc.html#collections.abc.Awaitable "collections.abc.Awaitable") [[str](https://docs.python.org/3/library/stdtypes.html#str) ]], [Callable](https://docs.python.org/3/library/typing.html#typing.Callable "typing.Callable") [[], [str](https://docs.python.org/3/library/stdtypes.html#str) ], [Callable](https://docs.python.org/3/library/typing.html#typing.Callable "typing.Callable") [[], [Awaitable](https://docs.python.org/3/library/collections.abc.html#collections.abc.Awaitable "collections.abc.Awaitable") [[str](https://docs.python.org/3/library/stdtypes.html#str) ]], ]` A function that may or maybe not take `RunContext` as an argument, and may or may not be async. Usage `SystemPromptFunc[AgentDepsT]`. ### ToolFuncContext `module-attribute` `ToolFuncContext = [Callable](https://docs.python.org/3/library/typing.html#typing.Callable "typing.Callable") [ [Concatenate](https://typing-extensions.readthedocs.io/en/latest/index.html#typing_extensions.Concatenate "typing_extensions.Concatenate") [[RunContext](#pydantic_ai.tools.RunContext "pydantic_ai.tools.RunContext") [[AgentDepsT](#pydantic_ai.tools.AgentDepsT "pydantic_ai.tools.AgentDepsT") ], [ToolParams](#pydantic_ai.tools.ToolParams "pydantic_ai.tools.ToolParams") ], [Any](https://docs.python.org/3/library/typing.html#typing.Any "typing.Any") ]` A tool function that takes `RunContext` as the first argument. Usage `ToolContextFunc[AgentDepsT, ToolParams]`. ### ToolFuncPlain `module-attribute` `ToolFuncPlain = [Callable](https://docs.python.org/3/library/typing.html#typing.Callable "typing.Callable") [[ToolParams](#pydantic_ai.tools.ToolParams "pydantic_ai.tools.ToolParams") , [Any](https://docs.python.org/3/library/typing.html#typing.Any "typing.Any") ]` A tool function that does not take `RunContext` as the first argument. Usage `ToolPlainFunc[ToolParams]`. ### ToolFuncEither `module-attribute` `ToolFuncEither = [Union](https://docs.python.org/3/library/typing.html#typing.Union "typing.Union") [ [ToolFuncContext](#pydantic_ai.tools.ToolFuncContext "pydantic_ai.tools.ToolFuncContext") [[AgentDepsT](#pydantic_ai.tools.AgentDepsT "pydantic_ai.tools.AgentDepsT") , [ToolParams](#pydantic_ai.tools.ToolParams "pydantic_ai.tools.ToolParams") ], [ToolFuncPlain](#pydantic_ai.tools.ToolFuncPlain "pydantic_ai.tools.ToolFuncPlain") [[ToolParams](#pydantic_ai.tools.ToolParams "pydantic_ai.tools.ToolParams") ], ]` Either kind of tool function. This is just a union of [`ToolFuncContext`](#pydantic_ai.tools.ToolFuncContext) and [`ToolFuncPlain`](#pydantic_ai.tools.ToolFuncPlain) . Usage `ToolFuncEither[AgentDepsT, ToolParams]`. ### ToolPrepareFunc `module-attribute` `ToolPrepareFunc: [TypeAlias](https://typing-extensions.readthedocs.io/en/latest/index.html#typing_extensions.TypeAlias "typing_extensions.TypeAlias") = ( "Callable[[RunContext[AgentDepsT], ToolDefinition], Awaitable[ToolDefinition | None]]" )` Definition of a function that can prepare a tool definition at call time. See [tool docs](../../tools/#tool-prepare) for more information. Example — here `only_if_42` is valid as a `ToolPrepareFunc`: `from typing import Union from pydantic_ai import RunContext, Tool from pydantic_ai.tools import ToolDefinition async def only_if_42( ctx: RunContext[int], tool_def: ToolDefinition ) -> Union[ToolDefinition, None]: if ctx.deps == 42: return tool_def def hitchhiker(ctx: RunContext[int], answer: str) -> str: return f'{ctx.deps} {answer}' hitchhiker = Tool(hitchhiker, prepare=only_if_42)` Usage `ToolPrepareFunc[AgentDepsT]`. ### DocstringFormat `module-attribute` `DocstringFormat = [Literal](https://docs.python.org/3/library/typing.html#typing.Literal "typing.Literal") [ "google", "numpy", "sphinx", "auto" ]` Supported docstring formats. * `'google'` — [Google-style](https://google.github.io/styleguide/pyguide.html#381-docstrings) docstrings. * `'numpy'` — [Numpy-style](https://numpydoc.readthedocs.io/en/latest/format.html) docstrings. * `'sphinx'` — [Sphinx-style](https://sphinx-rtd-tutorial.readthedocs.io/en/latest/docstrings.html#the-sphinx-docstring-format) docstrings. * `'auto'` — Automatically infer the format based on the structure of the docstring. ### Tool `dataclass` Bases: `[Generic](https://docs.python.org/3/library/typing.html#typing.Generic "typing.Generic") [[AgentDepsT](#pydantic_ai.tools.AgentDepsT "pydantic_ai.tools.AgentDepsT") ]` A tool function for an agent. Source code in `pydantic_ai_slim/pydantic_ai/tools.py` | | | | --- | --- | | 164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387 | ``@dataclass(init=False) class Tool(Generic[AgentDepsT]): """A tool function for an agent.""" function: ToolFuncEither[AgentDepsT] takes_ctx: bool max_retries: int \| None name: str description: str prepare: ToolPrepareFunc[AgentDepsT] \| None docstring_format: DocstringFormat require_parameter_descriptions: bool _is_async: bool = field(init=False) _single_arg_name: str \| None = field(init=False) _positional_fields: list[str] = field(init=False) _var_positional_field: str \| None = field(init=False) _validator: SchemaValidator = field(init=False, repr=False) _parameters_json_schema: ObjectJsonSchema = field(init=False) # TODO: Move this state off the Tool class, which is otherwise stateless. # This should be tracked inside a specific agent run, not the tool. current_retry: int = field(default=0, init=False) def __init__( self, function: ToolFuncEither[AgentDepsT], *, takes_ctx: bool \| None = None, max_retries: int \| None = None, name: str \| None = None, description: str \| None = None, prepare: ToolPrepareFunc[AgentDepsT] \| None = None, docstring_format: DocstringFormat = 'auto', require_parameter_descriptions: bool = False, schema_generator: type[GenerateJsonSchema] = GenerateToolJsonSchema, ): """Create a new tool instance. Example usage: ```python {noqa="I001"} from pydantic_ai import Agent, RunContext, Tool async def my_tool(ctx: RunContext[int], x: int, y: int) -> str: return f'{ctx.deps} {x} {y}' agent = Agent('test', tools=[Tool(my_tool)]) ``` or with a custom prepare method: ```python {noqa="I001"} from typing import Union from pydantic_ai import Agent, RunContext, Tool from pydantic_ai.tools import ToolDefinition async def my_tool(ctx: RunContext[int], x: int, y: int) -> str: return f'{ctx.deps} {x} {y}' async def prep_my_tool( ctx: RunContext[int], tool_def: ToolDefinition ) -> Union[ToolDefinition, None]: # only register the tool if `deps == 42` if ctx.deps == 42: return tool_def agent = Agent('test', tools=[Tool(my_tool, prepare=prep_my_tool)]) ``` Args: function: The Python function to call as the tool. takes_ctx: Whether the function takes a [`RunContext`][pydantic_ai.tools.RunContext] first argument, this is inferred if unset. max_retries: Maximum number of retries allowed for this tool, set to the agent default if `None`. name: Name of the tool, inferred from the function if `None`. description: Description of the tool, inferred from the function if `None`. prepare: custom method to prepare the tool definition for each step, return `None` to omit this tool from a given step. This is useful if you want to customise a tool at call time, or omit it completely from a step. See [`ToolPrepareFunc`][pydantic_ai.tools.ToolPrepareFunc]. docstring_format: The format of the docstring, see [`DocstringFormat`][pydantic_ai.tools.DocstringFormat]. Defaults to `'auto'`, such that the format is inferred from the structure of the docstring. require_parameter_descriptions: If True, raise an error if a parameter description is missing. Defaults to False. schema_generator: The JSON schema generator class to use. Defaults to `GenerateToolJsonSchema`. """ if takes_ctx is None: takes_ctx = _pydantic.takes_ctx(function) f = _pydantic.function_schema( function, takes_ctx, docstring_format, require_parameter_descriptions, schema_generator ) self.function = function self.takes_ctx = takes_ctx self.max_retries = max_retries self.name = name or function.__name__ self.description = description or f['description'] self.prepare = prepare self.docstring_format = docstring_format self.require_parameter_descriptions = require_parameter_descriptions self._is_async = inspect.iscoroutinefunction(self.function) self._single_arg_name = f['single_arg_name'] self._positional_fields = f['positional_fields'] self._var_positional_field = f['var_positional_field'] self._validator = f['validator'] self._parameters_json_schema = f['json_schema'] async def prepare_tool_def(self, ctx: RunContext[AgentDepsT]) -> ToolDefinition \| None: """Get the tool definition. By default, this method creates a tool definition, then either returns it, or calls `self.prepare` if it's set. Returns: return a `ToolDefinition` or `None` if the tools should not be registered for this run. """ tool_def = ToolDefinition( name=self.name, description=self.description, parameters_json_schema=self._parameters_json_schema, ) if self.prepare is not None: return await self.prepare(ctx, tool_def) else: return tool_def async def run( self, message: _messages.ToolCallPart, run_context: RunContext[AgentDepsT], tracer: Tracer ) -> _messages.ToolReturnPart \| _messages.RetryPromptPart: """Run the tool function asynchronously. This method wraps `_run` in an OpenTelemetry span. See . """ span_attributes = { 'gen_ai.tool.name': self.name, # NOTE: this means `gen_ai.tool.call.id` will be included even if it was generated by pydantic-ai 'gen_ai.tool.call.id': message.tool_call_id, 'tool_arguments': message.args_as_json_str(), 'logfire.msg': f'running tool: {self.name}', # add the JSON schema so these attributes are formatted nicely in Logfire 'logfire.json_schema': json.dumps( { 'type': 'object', 'properties': { 'tool_arguments': {'type': 'object'}, 'gen_ai.tool.name': {}, 'gen_ai.tool.call.id': {}, }, } ), } with tracer.start_as_current_span('running tool', attributes=span_attributes): return await self._run(message, run_context) async def _run( self, message: _messages.ToolCallPart, run_context: RunContext[AgentDepsT] ) -> _messages.ToolReturnPart \| _messages.RetryPromptPart: try: if isinstance(message.args, str): args_dict = self._validator.validate_json(message.args) else: args_dict = self._validator.validate_python(message.args) except ValidationError as e: return self._on_error(e, message) args, kwargs = self._call_args(args_dict, message, run_context) try: if self._is_async: function = cast(Callable[[Any], Awaitable[str]], self.function) response_content = await function(*args, **kwargs) else: function = cast(Callable[[Any], str], self.function) response_content = await _utils.run_in_executor(function, *args, **kwargs) except ModelRetry as e: return self._on_error(e, message) self.current_retry = 0 return _messages.ToolReturnPart( tool_name=message.tool_name, content=response_content, tool_call_id=message.tool_call_id, ) def _call_args( self, args_dict: dict[str, Any], message: _messages.ToolCallPart, run_context: RunContext[AgentDepsT], ) -> tuple[list[Any], dict[str, Any]]: if self._single_arg_name: args_dict = {self._single_arg_name: args_dict} ctx = dataclasses.replace( run_context, retry=self.current_retry, tool_name=message.tool_name, tool_call_id=message.tool_call_id, ) args = [ctx] if self.takes_ctx else [] for positional_field in self._positional_fields: args.append(args_dict.pop(positional_field)) if self._var_positional_field: args.extend(args_dict.pop(self._var_positional_field)) return args, args_dict def _on_error( self, exc: ValidationError \| ModelRetry, call_message: _messages.ToolCallPart ) -> _messages.RetryPromptPart: self.current_retry += 1 if self.max_retries is None or self.current_retry > self.max_retries: raise UnexpectedModelBehavior(f'Tool exceeded max retries count of {self.max_retries}') from exc else: if isinstance(exc, ValidationError): content = exc.errors(include_url=False) else: content = exc.message return _messages.RetryPromptPart( tool_name=call_message.tool_name, content=content, tool_call_id=call_message.tool_call_id, )`` | #### \_\_init\_\_ `__init__( function: [ToolFuncEither](#pydantic_ai.tools.ToolFuncEither "pydantic_ai.tools.ToolFuncEither") [[AgentDepsT](#pydantic_ai.tools.AgentDepsT "pydantic_ai.tools.AgentDepsT") ], *, takes_ctx: [bool](https://docs.python.org/3/library/functions.html#bool) | None = None, max_retries: [int](https://docs.python.org/3/library/functions.html#int) | None = None, name: [str](https://docs.python.org/3/library/stdtypes.html#str) | None = None, description: [str](https://docs.python.org/3/library/stdtypes.html#str) | None = None, prepare: [ToolPrepareFunc](#pydantic_ai.tools.ToolPrepareFunc "pydantic_ai.tools.ToolPrepareFunc") [[AgentDepsT](#pydantic_ai.tools.AgentDepsT "pydantic_ai.tools.AgentDepsT") ] | None = None, docstring_format: [DocstringFormat](#pydantic_ai.tools.DocstringFormat "pydantic_ai.tools.DocstringFormat") = "auto", require_parameter_descriptions: [bool](https://docs.python.org/3/library/functions.html#bool) = False, schema_generator: [type](https://docs.python.org/3/library/functions.html#type) [ [GenerateJsonSchema](https://docs.pydantic.dev/latest/api/json_schema/#pydantic.json_schema.GenerateJsonSchema "pydantic.json_schema.GenerateJsonSchema") ] = GenerateToolJsonSchema )` Create a new tool instance. Example usage: `from pydantic_ai import Agent, RunContext, Tool async def my_tool(ctx: RunContext[int], x: int, y: int) -> str: return f'{ctx.deps} {x} {y}' agent = Agent('test', tools=[Tool(my_tool)])` or with a custom prepare method: ``from typing import Union from pydantic_ai import Agent, RunContext, Tool from pydantic_ai.tools import ToolDefinition async def my_tool(ctx: RunContext[int], x: int, y: int) -> str: return f'{ctx.deps} {x} {y}' async def prep_my_tool( ctx: RunContext[int], tool_def: ToolDefinition ) -> Union[ToolDefinition, None]: # only register the tool if `deps == 42` if ctx.deps == 42: return tool_def agent = Agent('test', tools=[Tool(my_tool, prepare=prep_my_tool)])`` Parameters: | Name | Type | Description | Default | | --- | --- | --- | --- | | `function` | `[ToolFuncEither](#pydantic_ai.tools.ToolFuncEither "pydantic_ai.tools.ToolFuncEither") [[AgentDepsT](#pydantic_ai.tools.AgentDepsT "pydantic_ai.tools.AgentDepsT") ]` | The Python function to call as the tool. | _required_ | | `takes_ctx` | `[bool](https://docs.python.org/3/library/functions.html#bool) \| None` | Whether the function takes a [`RunContext`](#pydantic_ai.tools.RunContext)
first argument, this is inferred if unset. | `None` | | `max_retries` | `[int](https://docs.python.org/3/library/functions.html#int) \| None` | Maximum number of retries allowed for this tool, set to the agent default if `None`. | `None` | | `name` | `[str](https://docs.python.org/3/library/stdtypes.html#str) \| None` | Name of the tool, inferred from the function if `None`. | `None` | | `description` | `[str](https://docs.python.org/3/library/stdtypes.html#str) \| None` | Description of the tool, inferred from the function if `None`. | `None` | | `prepare` | `[ToolPrepareFunc](#pydantic_ai.tools.ToolPrepareFunc "pydantic_ai.tools.ToolPrepareFunc") [[AgentDepsT](#pydantic_ai.tools.AgentDepsT "pydantic_ai.tools.AgentDepsT") ] \| None` | custom method to prepare the tool definition for each step, return `None` to omit this tool from a given step. This is useful if you want to customise a tool at call time, or omit it completely from a step. See [`ToolPrepareFunc`](#pydantic_ai.tools.ToolPrepareFunc)
. | `None` | | `docstring_format` | `[DocstringFormat](#pydantic_ai.tools.DocstringFormat "pydantic_ai.tools.DocstringFormat") ` | The format of the docstring, see [`DocstringFormat`](#pydantic_ai.tools.DocstringFormat)
. Defaults to `'auto'`, such that the format is inferred from the structure of the docstring. | `'auto'` | | `require_parameter_descriptions` | `[bool](https://docs.python.org/3/library/functions.html#bool) ` | If True, raise an error if a parameter description is missing. Defaults to False. | `False` | | `schema_generator` | `[type](https://docs.python.org/3/library/functions.html#type) [[GenerateJsonSchema](https://docs.pydantic.dev/latest/api/json_schema/#pydantic.json_schema.GenerateJsonSchema "pydantic.json_schema.GenerateJsonSchema") ]` | The JSON schema generator class to use. Defaults to `GenerateToolJsonSchema`. | `GenerateToolJsonSchema` | Source code in `pydantic_ai_slim/pydantic_ai/tools.py` | | | | --- | --- | | 187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269 | ``def __init__( self, function: ToolFuncEither[AgentDepsT], *, takes_ctx: bool \| None = None, max_retries: int \| None = None, name: str \| None = None, description: str \| None = None, prepare: ToolPrepareFunc[AgentDepsT] \| None = None, docstring_format: DocstringFormat = 'auto', require_parameter_descriptions: bool = False, schema_generator: type[GenerateJsonSchema] = GenerateToolJsonSchema, ): """Create a new tool instance. Example usage: ```python {noqa="I001"} from pydantic_ai import Agent, RunContext, Tool async def my_tool(ctx: RunContext[int], x: int, y: int) -> str: return f'{ctx.deps} {x} {y}' agent = Agent('test', tools=[Tool(my_tool)]) ``` or with a custom prepare method: ```python {noqa="I001"} from typing import Union from pydantic_ai import Agent, RunContext, Tool from pydantic_ai.tools import ToolDefinition async def my_tool(ctx: RunContext[int], x: int, y: int) -> str: return f'{ctx.deps} {x} {y}' async def prep_my_tool( ctx: RunContext[int], tool_def: ToolDefinition ) -> Union[ToolDefinition, None]: # only register the tool if `deps == 42` if ctx.deps == 42: return tool_def agent = Agent('test', tools=[Tool(my_tool, prepare=prep_my_tool)]) ``` Args: function: The Python function to call as the tool. takes_ctx: Whether the function takes a [`RunContext`][pydantic_ai.tools.RunContext] first argument, this is inferred if unset. max_retries: Maximum number of retries allowed for this tool, set to the agent default if `None`. name: Name of the tool, inferred from the function if `None`. description: Description of the tool, inferred from the function if `None`. prepare: custom method to prepare the tool definition for each step, return `None` to omit this tool from a given step. This is useful if you want to customise a tool at call time, or omit it completely from a step. See [`ToolPrepareFunc`][pydantic_ai.tools.ToolPrepareFunc]. docstring_format: The format of the docstring, see [`DocstringFormat`][pydantic_ai.tools.DocstringFormat]. Defaults to `'auto'`, such that the format is inferred from the structure of the docstring. require_parameter_descriptions: If True, raise an error if a parameter description is missing. Defaults to False. schema_generator: The JSON schema generator class to use. Defaults to `GenerateToolJsonSchema`. """ if takes_ctx is None: takes_ctx = _pydantic.takes_ctx(function) f = _pydantic.function_schema( function, takes_ctx, docstring_format, require_parameter_descriptions, schema_generator ) self.function = function self.takes_ctx = takes_ctx self.max_retries = max_retries self.name = name or function.__name__ self.description = description or f['description'] self.prepare = prepare self.docstring_format = docstring_format self.require_parameter_descriptions = require_parameter_descriptions self._is_async = inspect.iscoroutinefunction(self.function) self._single_arg_name = f['single_arg_name'] self._positional_fields = f['positional_fields'] self._var_positional_field = f['var_positional_field'] self._validator = f['validator'] self._parameters_json_schema = f['json_schema']`` | #### prepare\_tool\_def `async` `prepare_tool_def( ctx: [RunContext](#pydantic_ai.tools.RunContext "pydantic_ai.tools.RunContext") [[AgentDepsT](#pydantic_ai.tools.AgentDepsT "pydantic_ai.tools.AgentDepsT") ], ) -> [ToolDefinition](#pydantic_ai.tools.ToolDefinition "pydantic_ai.tools.ToolDefinition") | None` Get the tool definition. By default, this method creates a tool definition, then either returns it, or calls `self.prepare` if it's set. Returns: | Type | Description | | --- | --- | | `[ToolDefinition](#pydantic_ai.tools.ToolDefinition "pydantic_ai.tools.ToolDefinition") \| None` | return a `ToolDefinition` or `None` if the tools should not be registered for this run. | Source code in `pydantic_ai_slim/pydantic_ai/tools.py` | | | | --- | --- | | 271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288 | ``async def prepare_tool_def(self, ctx: RunContext[AgentDepsT]) -> ToolDefinition \| None: """Get the tool definition. By default, this method creates a tool definition, then either returns it, or calls `self.prepare` if it's set. Returns: return a `ToolDefinition` or `None` if the tools should not be registered for this run. """ tool_def = ToolDefinition( name=self.name, description=self.description, parameters_json_schema=self._parameters_json_schema, ) if self.prepare is not None: return await self.prepare(ctx, tool_def) else: return tool_def`` | #### run `async` `run( message: [ToolCallPart](../messages/#pydantic_ai.messages.ToolCallPart "pydantic_ai.messages.ToolCallPart") , run_context: [RunContext](#pydantic_ai.tools.RunContext "pydantic_ai.tools.RunContext") [[AgentDepsT](#pydantic_ai.tools.AgentDepsT "pydantic_ai.tools.AgentDepsT") ], tracer: Tracer, ) -> [ToolReturnPart](../messages/#pydantic_ai.messages.ToolReturnPart "pydantic_ai.messages.ToolReturnPart") | [RetryPromptPart](../messages/#pydantic_ai.messages.RetryPromptPart "pydantic_ai.messages.RetryPromptPart")` Run the tool function asynchronously. This method wraps `_run` in an OpenTelemetry span. See [https://opentelemetry.io/docs/specs/semconv/gen-ai/gen-ai-spans/#execute-tool-span](https://opentelemetry.io/docs/specs/semconv/gen-ai/gen-ai-spans/#execute-tool-span) . Source code in `pydantic_ai_slim/pydantic_ai/tools.py` | | | | --- | --- | | 290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318 | ``async def run( self, message: _messages.ToolCallPart, run_context: RunContext[AgentDepsT], tracer: Tracer ) -> _messages.ToolReturnPart \| _messages.RetryPromptPart: """Run the tool function asynchronously. This method wraps `_run` in an OpenTelemetry span. See . """ span_attributes = { 'gen_ai.tool.name': self.name, # NOTE: this means `gen_ai.tool.call.id` will be included even if it was generated by pydantic-ai 'gen_ai.tool.call.id': message.tool_call_id, 'tool_arguments': message.args_as_json_str(), 'logfire.msg': f'running tool: {self.name}', # add the JSON schema so these attributes are formatted nicely in Logfire 'logfire.json_schema': json.dumps( { 'type': 'object', 'properties': { 'tool_arguments': {'type': 'object'}, 'gen_ai.tool.name': {}, 'gen_ai.tool.call.id': {}, }, } ), } with tracer.start_as_current_span('running tool', attributes=span_attributes): return await self._run(message, run_context)`` | ### ObjectJsonSchema `module-attribute` `ObjectJsonSchema: [TypeAlias](https://typing-extensions.readthedocs.io/en/latest/index.html#typing_extensions.TypeAlias "typing_extensions.TypeAlias") = [dict](https://docs.python.org/3/library/stdtypes.html#dict) [[str](https://docs.python.org/3/library/stdtypes.html#str) , [Any](https://docs.python.org/3/library/typing.html#typing.Any "typing.Any") ]` Type representing JSON schema of an object, e.g. where `"type": "object"`. This type is used to define tools parameters (aka arguments) in [ToolDefinition](#pydantic_ai.tools.ToolDefinition) . With PEP-728 this should be a TypedDict with `type: Literal['object']`, and `extra_parts=Any` ### ToolDefinition `dataclass` Definition of a tool passed to a model. This is used for both function tools result tools. Source code in `pydantic_ai_slim/pydantic_ai/tools.py` | | | | --- | --- | | 399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419 | ``@dataclass class ToolDefinition: """Definition of a tool passed to a model. This is used for both function tools result tools. """ name: str """The name of the tool.""" description: str """The description of the tool.""" parameters_json_schema: ObjectJsonSchema """The JSON schema for the tool's parameters.""" outer_typed_dict_key: str \| None = None """The key in the outer [TypedDict] that wraps a result tool. This will only be set for result tools which don't have an `object` JSON schema. """`` | #### name `instance-attribute` `name: [str](https://docs.python.org/3/library/stdtypes.html#str)` The name of the tool. #### description `instance-attribute` `description: [str](https://docs.python.org/3/library/stdtypes.html#str)` The description of the tool. #### parameters\_json\_schema `instance-attribute` `parameters_json_schema: [ObjectJsonSchema](#pydantic_ai.tools.ObjectJsonSchema "pydantic_ai.tools.ObjectJsonSchema")` The JSON schema for the tool's parameters. #### outer\_typed\_dict\_key `class-attribute` `instance-attribute` `outer_typed_dict_key: [str](https://docs.python.org/3/library/stdtypes.html#str) | None = None` The key in the outer \[TypedDict\] that wraps a result tool. This will only be set for result tools which don't have an `object` JSON schema. --- # pydantic_ai.common_tools - PydanticAI `pydantic_ai.common_tools` ========================== ### duckduckgo\_search\_tool `duckduckgo_search_tool( duckduckgo_client: DDGS | None = None, max_results: [int](https://docs.python.org/3/library/functions.html#int) | None = None, )` Creates a DuckDuckGo search tool. Parameters: | Name | Type | Description | Default | | --- | --- | --- | --- | | `duckduckgo_client` | `DDGS \| None` | The DuckDuckGo search client. | `None` | | `max_results` | `[int](https://docs.python.org/3/library/functions.html#int) \| None` | The maximum number of results. If None, returns results only from the first response. | `None` | Source code in `pydantic_ai_slim/pydantic_ai/common_tools/duckduckgo.py` | | | | --- | --- | | 62
63
64
65
66
67
68
69
70
71
72
73 | `def duckduckgo_search_tool(duckduckgo_client: DDGS \| None = None, max_results: int \| None = None): """Creates a DuckDuckGo search tool. Args: duckduckgo_client: The DuckDuckGo search client. max_results: The maximum number of results. If None, returns results only from the first response. """ return Tool( DuckDuckGoSearchTool(client=duckduckgo_client or DDGS(), max_results=max_results).__call__, name='duckduckgo_search', description='Searches DuckDuckGo for the given query and returns the results.', )` | ### tavily\_search\_tool `tavily_search_tool(api_key: [str](https://docs.python.org/3/library/stdtypes.html#str) )` Creates a Tavily search tool. Parameters: | Name | Type | Description | Default | | --- | --- | --- | --- | | `api_key` | `[str](https://docs.python.org/3/library/stdtypes.html#str) ` | The Tavily API key.

You can get one by signing up at [https://app.tavily.com/home](https://app.tavily.com/home)
. | _required_ | Source code in `pydantic_ai_slim/pydantic_ai/common_tools/tavily.py` | | | | --- | --- | | 71
72
73
74
75
76
77
78
79
80
81
82
83 | `def tavily_search_tool(api_key: str): """Creates a Tavily search tool. Args: api_key: The Tavily API key. You can get one by signing up at [https://app.tavily.com/home](https://app.tavily.com/home). """ return Tool( TavilySearchTool(client=AsyncTavilyClient(api_key)).__call__, name='tavily_search', description='Searches Tavily for the given query and returns the results.', )` | --- # pydantic_ai.usage - PydanticAI `pydantic_ai.usage` =================== ### Usage `dataclass` LLM usage associated with a request or run. Responsibility for calculating usage is on the model; PydanticAI simply sums the usage information across requests. You'll need to look up the documentation of the model you're using to convert usage to monetary costs. Source code in `pydantic_ai_slim/pydantic_ai/usage.py` | | | | --- | --- | | 11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67 | ``@dataclass class Usage: """LLM usage associated with a request or run. Responsibility for calculating usage is on the model; PydanticAI simply sums the usage information across requests. You'll need to look up the documentation of the model you're using to convert usage to monetary costs. """ requests: int = 0 """Number of requests made to the LLM API.""" request_tokens: int \| None = None """Tokens used in processing requests.""" response_tokens: int \| None = None """Tokens used in generating responses.""" total_tokens: int \| None = None """Total tokens used in the whole run, should generally be equal to `request_tokens + response_tokens`.""" details: dict[str, int] \| None = None """Any extra details returned by the model.""" def incr(self, incr_usage: Usage, *, requests: int = 0) -> None: """Increment the usage in place. Args: incr_usage: The usage to increment by. requests: The number of requests to increment by in addition to `incr_usage.requests`. """ self.requests += requests for f in 'requests', 'request_tokens', 'response_tokens', 'total_tokens': self_value = getattr(self, f) other_value = getattr(incr_usage, f) if self_value is not None or other_value is not None: setattr(self, f, (self_value or 0) + (other_value or 0)) if incr_usage.details: self.details = self.details or {} for key, value in incr_usage.details.items(): self.details[key] = self.details.get(key, 0) + value def __add__(self, other: Usage) -> Usage: """Add two Usages together. This is provided so it's trivial to sum usage information from multiple requests and runs. """ new_usage = copy(self) new_usage.incr(other) return new_usage def opentelemetry_attributes(self) -> dict[str, int]: """Get the token limits as OpenTelemetry attributes.""" result = { 'gen_ai.usage.input_tokens': self.request_tokens, 'gen_ai.usage.output_tokens': self.response_tokens, } for key, value in (self.details or {}).items(): result[f'gen_ai.usage.details.{key}'] = value return {k: v for k, v in result.items() if v}`` | #### requests `class-attribute` `instance-attribute` `requests: [int](https://docs.python.org/3/library/functions.html#int) = 0` Number of requests made to the LLM API. #### request\_tokens `class-attribute` `instance-attribute` `request_tokens: [int](https://docs.python.org/3/library/functions.html#int) | None = None` Tokens used in processing requests. #### response\_tokens `class-attribute` `instance-attribute` `response_tokens: [int](https://docs.python.org/3/library/functions.html#int) | None = None` Tokens used in generating responses. #### total\_tokens `class-attribute` `instance-attribute` `total_tokens: [int](https://docs.python.org/3/library/functions.html#int) | None = None` Total tokens used in the whole run, should generally be equal to `request_tokens + response_tokens`. #### details `class-attribute` `instance-attribute` `details: [dict](https://docs.python.org/3/library/stdtypes.html#dict) [[str](https://docs.python.org/3/library/stdtypes.html#str) , [int](https://docs.python.org/3/library/functions.html#int) ] | None = None` Any extra details returned by the model. #### incr `incr(incr_usage: [Usage](#pydantic_ai.usage.Usage "pydantic_ai.usage.Usage") , *, requests: [int](https://docs.python.org/3/library/functions.html#int) = 0) -> None` Increment the usage in place. Parameters: | Name | Type | Description | Default | | --- | --- | --- | --- | | `incr_usage` | `[Usage](#pydantic_ai.usage.Usage "pydantic_ai.usage.Usage") ` | The usage to increment by. | _required_ | | `requests` | `[int](https://docs.python.org/3/library/functions.html#int) ` | The number of requests to increment by in addition to `incr_usage.requests`. | `0` | Source code in `pydantic_ai_slim/pydantic_ai/usage.py` | | | | --- | --- | | 31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48 | ``def incr(self, incr_usage: Usage, *, requests: int = 0) -> None: """Increment the usage in place. Args: incr_usage: The usage to increment by. requests: The number of requests to increment by in addition to `incr_usage.requests`. """ self.requests += requests for f in 'requests', 'request_tokens', 'response_tokens', 'total_tokens': self_value = getattr(self, f) other_value = getattr(incr_usage, f) if self_value is not None or other_value is not None: setattr(self, f, (self_value or 0) + (other_value or 0)) if incr_usage.details: self.details = self.details or {} for key, value in incr_usage.details.items(): self.details[key] = self.details.get(key, 0) + value`` | #### \_\_add\_\_ `__add__(other: [Usage](#pydantic_ai.usage.Usage "pydantic_ai.usage.Usage") ) -> [Usage](#pydantic_ai.usage.Usage "pydantic_ai.usage.Usage")` Add two Usages together. This is provided so it's trivial to sum usage information from multiple requests and runs. Source code in `pydantic_ai_slim/pydantic_ai/usage.py` | | | | --- | --- | | 50
51
52
53
54
55
56
57 | `def __add__(self, other: Usage) -> Usage: """Add two Usages together. This is provided so it's trivial to sum usage information from multiple requests and runs. """ new_usage = copy(self) new_usage.incr(other) return new_usage` | #### opentelemetry\_attributes `opentelemetry_attributes() -> [dict](https://docs.python.org/3/library/stdtypes.html#dict) [[str](https://docs.python.org/3/library/stdtypes.html#str) , [int](https://docs.python.org/3/library/functions.html#int) ]` Get the token limits as OpenTelemetry attributes. Source code in `pydantic_ai_slim/pydantic_ai/usage.py` | | | | --- | --- | | 59
60
61
62
63
64
65
66
67 | `def opentelemetry_attributes(self) -> dict[str, int]: """Get the token limits as OpenTelemetry attributes.""" result = { 'gen_ai.usage.input_tokens': self.request_tokens, 'gen_ai.usage.output_tokens': self.response_tokens, } for key, value in (self.details or {}).items(): result[f'gen_ai.usage.details.{key}'] = value return {k: v for k, v in result.items() if v}` | ### UsageLimits `dataclass` Limits on model usage. The request count is tracked by pydantic\_ai, and the request limit is checked before each request to the model. Token counts are provided in responses from the model, and the token limits are checked after each response. Each of the limits can be set to `None` to disable that limit. Source code in `pydantic_ai_slim/pydantic_ai/usage.py` | | | | --- | --- | | 70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124 | ``@dataclass class UsageLimits: """Limits on model usage. The request count is tracked by pydantic_ai, and the request limit is checked before each request to the model. Token counts are provided in responses from the model, and the token limits are checked after each response. Each of the limits can be set to `None` to disable that limit. """ request_limit: int \| None = 50 """The maximum number of requests allowed to the model.""" request_tokens_limit: int \| None = None """The maximum number of tokens allowed in requests to the model.""" response_tokens_limit: int \| None = None """The maximum number of tokens allowed in responses from the model.""" total_tokens_limit: int \| None = None """The maximum number of tokens allowed in requests and responses combined.""" def has_token_limits(self) -> bool: """Returns `True` if this instance places any limits on token counts. If this returns `False`, the `check_tokens` method will never raise an error. This is useful because if we have token limits, we need to check them after receiving each streamed message. If there are no limits, we can skip that processing in the streaming response iterator. """ return any( limit is not None for limit in (self.request_tokens_limit, self.response_tokens_limit, self.total_tokens_limit) ) def check_before_request(self, usage: Usage) -> None: """Raises a `UsageLimitExceeded` exception if the next request would exceed the request_limit.""" request_limit = self.request_limit if request_limit is not None and usage.requests >= request_limit: raise UsageLimitExceeded(f'The next request would exceed the request_limit of {request_limit}') def check_tokens(self, usage: Usage) -> None: """Raises a `UsageLimitExceeded` exception if the usage exceeds any of the token limits.""" request_tokens = usage.request_tokens or 0 if self.request_tokens_limit is not None and request_tokens > self.request_tokens_limit: raise UsageLimitExceeded( f'Exceeded the request_tokens_limit of {self.request_tokens_limit} ({request_tokens=})' ) response_tokens = usage.response_tokens or 0 if self.response_tokens_limit is not None and response_tokens > self.response_tokens_limit: raise UsageLimitExceeded( f'Exceeded the response_tokens_limit of {self.response_tokens_limit} ({response_tokens=})' ) total_tokens = usage.total_tokens or 0 if self.total_tokens_limit is not None and total_tokens > self.total_tokens_limit: raise UsageLimitExceeded(f'Exceeded the total_tokens_limit of {self.total_tokens_limit} ({total_tokens=})')`` | #### request\_limit `class-attribute` `instance-attribute` `request_limit: [int](https://docs.python.org/3/library/functions.html#int) | None = 50` The maximum number of requests allowed to the model. #### request\_tokens\_limit `class-attribute` `instance-attribute` `request_tokens_limit: [int](https://docs.python.org/3/library/functions.html#int) | None = None` The maximum number of tokens allowed in requests to the model. #### response\_tokens\_limit `class-attribute` `instance-attribute` `response_tokens_limit: [int](https://docs.python.org/3/library/functions.html#int) | None = None` The maximum number of tokens allowed in responses from the model. #### total\_tokens\_limit `class-attribute` `instance-attribute` `total_tokens_limit: [int](https://docs.python.org/3/library/functions.html#int) | None = None` The maximum number of tokens allowed in requests and responses combined. #### has\_token\_limits `has_token_limits() -> [bool](https://docs.python.org/3/library/functions.html#bool)` Returns `True` if this instance places any limits on token counts. If this returns `False`, the `check_tokens` method will never raise an error. This is useful because if we have token limits, we need to check them after receiving each streamed message. If there are no limits, we can skip that processing in the streaming response iterator. Source code in `pydantic_ai_slim/pydantic_ai/usage.py` | | | | --- | --- | | 89
90
91
92
93
94
95
96
97
98
99
100 | ``def has_token_limits(self) -> bool: """Returns `True` if this instance places any limits on token counts. If this returns `False`, the `check_tokens` method will never raise an error. This is useful because if we have token limits, we need to check them after receiving each streamed message. If there are no limits, we can skip that processing in the streaming response iterator. """ return any( limit is not None for limit in (self.request_tokens_limit, self.response_tokens_limit, self.total_tokens_limit) )`` | #### check\_before\_request `check_before_request(usage: [Usage](#pydantic_ai.usage.Usage "pydantic_ai.usage.Usage") ) -> None` Raises a `UsageLimitExceeded` exception if the next request would exceed the request\_limit. Source code in `pydantic_ai_slim/pydantic_ai/usage.py` | | | | --- | --- | | 102
103
104
105
106 | ``def check_before_request(self, usage: Usage) -> None: """Raises a `UsageLimitExceeded` exception if the next request would exceed the request_limit.""" request_limit = self.request_limit if request_limit is not None and usage.requests >= request_limit: raise UsageLimitExceeded(f'The next request would exceed the request_limit of {request_limit}')`` | #### check\_tokens `check_tokens(usage: [Usage](#pydantic_ai.usage.Usage "pydantic_ai.usage.Usage") ) -> None` Raises a `UsageLimitExceeded` exception if the usage exceeds any of the token limits. Source code in `pydantic_ai_slim/pydantic_ai/usage.py` | | | | --- | --- | | 108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124 | ``def check_tokens(self, usage: Usage) -> None: """Raises a `UsageLimitExceeded` exception if the usage exceeds any of the token limits.""" request_tokens = usage.request_tokens or 0 if self.request_tokens_limit is not None and request_tokens > self.request_tokens_limit: raise UsageLimitExceeded( f'Exceeded the request_tokens_limit of {self.request_tokens_limit} ({request_tokens=})' ) response_tokens = usage.response_tokens or 0 if self.response_tokens_limit is not None and response_tokens > self.response_tokens_limit: raise UsageLimitExceeded( f'Exceeded the response_tokens_limit of {self.response_tokens_limit} ({response_tokens=})' ) total_tokens = usage.total_tokens or 0 if self.total_tokens_limit is not None and total_tokens > self.total_tokens_limit: raise UsageLimitExceeded(f'Exceeded the total_tokens_limit of {self.total_tokens_limit} ({total_tokens=})')`` | --- # pydantic_ai.format_as_xml - PydanticAI `pydantic_ai.format_as_xml` =========================== ### format\_as\_xml `format_as_xml( obj: [Any](https://docs.python.org/3/library/typing.html#typing.Any "typing.Any") , root_tag: [str](https://docs.python.org/3/library/stdtypes.html#str) = "examples", item_tag: [str](https://docs.python.org/3/library/stdtypes.html#str) = "example", include_root_tag: [bool](https://docs.python.org/3/library/functions.html#bool) = True, none_str: [str](https://docs.python.org/3/library/stdtypes.html#str) = "null", indent: [str](https://docs.python.org/3/library/stdtypes.html#str) | None = " ", ) -> [str](https://docs.python.org/3/library/stdtypes.html#str)` Format a Python object as XML. This is useful since LLMs often find it easier to read semi-structured data (e.g. examples) as XML, rather than JSON etc. Supports: `str`, `bytes`, `bytearray`, `bool`, `int`, `float`, `date`, `datetime`, `Mapping`, `Iterable`, `dataclass`, and `BaseModel`. Parameters: | Name | Type | Description | Default | | --- | --- | --- | --- | | `obj` | `[Any](https://docs.python.org/3/library/typing.html#typing.Any "typing.Any") ` | Python Object to serialize to XML. | _required_ | | `root_tag` | `[str](https://docs.python.org/3/library/stdtypes.html#str) ` | Outer tag to wrap the XML in, use `None` to omit the outer tag. | `'examples'` | | `item_tag` | `[str](https://docs.python.org/3/library/stdtypes.html#str) ` | Tag to use for each item in an iterable (e.g. list), this is overridden by the class name for dataclasses and Pydantic models. | `'example'` | | `include_root_tag` | `[bool](https://docs.python.org/3/library/functions.html#bool) ` | Whether to include the root tag in the output (The root tag is always included if it includes a body - e.g. when the input is a simple value). | `True` | | `none_str` | `[str](https://docs.python.org/3/library/stdtypes.html#str) ` | String to use for `None` values. | `'null'` | | `indent` | `[str](https://docs.python.org/3/library/stdtypes.html#str) \| None` | Indentation string to use for pretty printing. | `' '` | Returns: | Type | Description | | --- | --- | | `[str](https://docs.python.org/3/library/stdtypes.html#str) ` | XML representation of the object. | Example: format\_as\_xml\_example.py `from pydantic_ai.format_as_xml import format_as_xml print(format_as_xml({'name': 'John', 'height': 6, 'weight': 200}, root_tag='user')) ''' John 6 200 '''` Source code in `pydantic_ai_slim/pydantic_ai/format_as_xml.py` | | | | --- | --- | | 14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64 | ``def format_as_xml( obj: Any, root_tag: str = 'examples', item_tag: str = 'example', include_root_tag: bool = True, none_str: str = 'null', indent: str \| None = ' ', ) -> str: """Format a Python object as XML. This is useful since LLMs often find it easier to read semi-structured data (e.g. examples) as XML, rather than JSON etc. Supports: `str`, `bytes`, `bytearray`, `bool`, `int`, `float`, `date`, `datetime`, `Mapping`, `Iterable`, `dataclass`, and `BaseModel`. Args: obj: Python Object to serialize to XML. root_tag: Outer tag to wrap the XML in, use `None` to omit the outer tag. item_tag: Tag to use for each item in an iterable (e.g. list), this is overridden by the class name for dataclasses and Pydantic models. include_root_tag: Whether to include the root tag in the output (The root tag is always included if it includes a body - e.g. when the input is a simple value). none_str: String to use for `None` values. indent: Indentation string to use for pretty printing. Returns: XML representation of the object. Example: ```python {title="format_as_xml_example.py" lint="skip"} from pydantic_ai.format_as_xml import format_as_xml print(format_as_xml({'name': 'John', 'height': 6, 'weight': 200}, root_tag='user')) ''' John 6 200 ''' ``` """ el = _ToXml(item_tag=item_tag, none_str=none_str).to_xml(obj, root_tag) if not include_root_tag and el.text is None: join = '' if indent is None else '\n' return join.join(_rootless_xml_elements(el, indent)) else: if indent is not None: ElementTree.indent(el, space=indent) return ElementTree.tostring(el, encoding='unicode')`` | --- # pydantic_ai.mcp - PydanticAI `pydantic_ai.mcp` ================= ### MCPServer Bases: `[ABC](https://docs.python.org/3/library/abc.html#abc.ABC "abc.ABC") ` Base class for attaching agents to MCP servers. See [https://modelcontextprotocol.io](https://modelcontextprotocol.io) for more information. Source code in `pydantic_ai_slim/pydantic_ai/mcp.py` | | | | --- | --- | | 30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98 | `class MCPServer(ABC): """Base class for attaching agents to MCP servers. See for more information. """ is_running: bool = False _client: ClientSession _read_stream: MemoryObjectReceiveStream[JSONRPCMessage \| Exception] _write_stream: MemoryObjectSendStream[JSONRPCMessage] _exit_stack: AsyncExitStack @abstractmethod @asynccontextmanager async def client_streams( self, ) -> AsyncIterator[ tuple[MemoryObjectReceiveStream[JSONRPCMessage \| Exception], MemoryObjectSendStream[JSONRPCMessage]] ]: """Create the streams for the MCP server.""" raise NotImplementedError('MCP Server subclasses must implement this method.') yield async def list_tools(self) -> list[ToolDefinition]: """Retrieve tools that are currently active on the server. Note: - We don't cache tools as they might change. - We also don't subscribe to the server to avoid complexity. """ tools = await self._client.list_tools() return [ ToolDefinition( name=tool.name, description=tool.description or '', parameters_json_schema=tool.inputSchema, ) for tool in tools.tools ] async def call_tool(self, tool_name: str, arguments: dict[str, Any]) -> CallToolResult: """Call a tool on the server. Args: tool_name: The name of the tool to call. arguments: The arguments to pass to the tool. Returns: The result of the tool call. """ return await self._client.call_tool(tool_name, arguments) async def __aenter__(self) -> Self: self._exit_stack = AsyncExitStack() self._read_stream, self._write_stream = await self._exit_stack.enter_async_context(self.client_streams()) client = ClientSession(read_stream=self._read_stream, write_stream=self._write_stream) self._client = await self._exit_stack.enter_async_context(client) await self._client.initialize() self.is_running = True return self async def __aexit__( self, exc_type: type[BaseException] \| None, exc_value: BaseException \| None, traceback: TracebackType \| None ) -> bool \| None: await self._exit_stack.aclose() self.is_running = False` | #### client\_streams `abstractmethod` `async` `client_streams() -> [AsyncIterator](https://docs.python.org/3/library/collections.abc.html#collections.abc.AsyncIterator "collections.abc.AsyncIterator") [ [tuple](https://docs.python.org/3/library/stdtypes.html#tuple) [ MemoryObjectReceiveStream[ JSONRPCMessage | [Exception](https://docs.python.org/3/library/exceptions.html#Exception) ], MemoryObjectSendStream[JSONRPCMessage], ] ]` Create the streams for the MCP server. Source code in `pydantic_ai_slim/pydantic_ai/mcp.py` | | | | --- | --- | | 43
44
45
46
47
48
49
50
51
52 | `@abstractmethod @asynccontextmanager async def client_streams( self, ) -> AsyncIterator[ tuple[MemoryObjectReceiveStream[JSONRPCMessage \| Exception], MemoryObjectSendStream[JSONRPCMessage]] ]: """Create the streams for the MCP server.""" raise NotImplementedError('MCP Server subclasses must implement this method.') yield` | #### list\_tools `async` `list_tools() -> [list](https://docs.python.org/3/library/stdtypes.html#list) [[ToolDefinition](../tools/#pydantic_ai.tools.ToolDefinition "pydantic_ai.tools.ToolDefinition") ]` Retrieve tools that are currently active on the server. Note: - We don't cache tools as they might change. - We also don't subscribe to the server to avoid complexity. Source code in `pydantic_ai_slim/pydantic_ai/mcp.py` | | | | --- | --- | | 54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69 | `async def list_tools(self) -> list[ToolDefinition]: """Retrieve tools that are currently active on the server. Note: - We don't cache tools as they might change. - We also don't subscribe to the server to avoid complexity. """ tools = await self._client.list_tools() return [ ToolDefinition( name=tool.name, description=tool.description or '', parameters_json_schema=tool.inputSchema, ) for tool in tools.tools ]` | #### call\_tool `async` `call_tool( tool_name: [str](https://docs.python.org/3/library/stdtypes.html#str) , arguments: [dict](https://docs.python.org/3/library/stdtypes.html#dict) [[str](https://docs.python.org/3/library/stdtypes.html#str) , [Any](https://docs.python.org/3/library/typing.html#typing.Any "typing.Any") ] ) -> CallToolResult` Call a tool on the server. Parameters: | Name | Type | Description | Default | | --- | --- | --- | --- | | `tool_name` | `[str](https://docs.python.org/3/library/stdtypes.html#str) ` | The name of the tool to call. | _required_ | | `arguments` | `[dict](https://docs.python.org/3/library/stdtypes.html#dict) [[str](https://docs.python.org/3/library/stdtypes.html#str) , [Any](https://docs.python.org/3/library/typing.html#typing.Any "typing.Any") ]` | The arguments to pass to the tool. | _required_ | Returns: | Type | Description | | --- | --- | | `CallToolResult` | The result of the tool call. | Source code in `pydantic_ai_slim/pydantic_ai/mcp.py` | | | | --- | --- | | 71
72
73
74
75
76
77
78
79
80
81 | `async def call_tool(self, tool_name: str, arguments: dict[str, Any]) -> CallToolResult: """Call a tool on the server. Args: tool_name: The name of the tool to call. arguments: The arguments to pass to the tool. Returns: The result of the tool call. """ return await self._client.call_tool(tool_name, arguments)` | ### MCPServerStdio `dataclass` Bases: `[MCPServer](#pydantic_ai.mcp.MCPServer "pydantic_ai.mcp.MCPServer") ` Runs an MCP server in a subprocess and communicates with it over stdin/stdout. This class implements the stdio transport from the MCP specification. See [https://spec.modelcontextprotocol.io/specification/2024-11-05/basic/transports/#stdio](https://spec.modelcontextprotocol.io/specification/2024-11-05/basic/transports/#stdio) for more information. Note Using this class as an async context manager will start the server as a subprocess when entering the context, and stop it when exiting the context. Example: `from pydantic_ai import Agent from pydantic_ai.mcp import MCPServerStdio server = MCPServerStdio( # (1)! 'deno', args=[ 'run', '-N', '-R=node_modules', '-W=node_modules', '--node-modules-dir=auto', 'jsr:@pydantic/mcp-run-python', 'stdio', ] ) agent = Agent('openai:gpt-4o', mcp_servers=[server]) async def main(): async with agent.run_mcp_servers(): # (2)! ...` 1. See [MCP Run Python](../../mcp/run-python/) for more information. 2. This will start the server as a subprocess and connect to it. Source code in `pydantic_ai_slim/pydantic_ai/mcp.py` | | | | --- | --- | | 101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161 | ``@dataclass class MCPServerStdio(MCPServer): """Runs an MCP server in a subprocess and communicates with it over stdin/stdout. This class implements the stdio transport from the MCP specification. See for more information. !!! note Using this class as an async context manager will start the server as a subprocess when entering the context, and stop it when exiting the context. Example: ```python {py="3.10"} from pydantic_ai import Agent from pydantic_ai.mcp import MCPServerStdio server = MCPServerStdio( # (1)! 'deno', args=[ 'run', '-N', '-R=node_modules', '-W=node_modules', '--node-modules-dir=auto', 'jsr:@pydantic/mcp-run-python', 'stdio', ] ) agent = Agent('openai:gpt-4o', mcp_servers=[server]) async def main(): async with agent.run_mcp_servers(): # (2)! ... ``` 1. See [MCP Run Python](../mcp/run-python.md) for more information. 2. This will start the server as a subprocess and connect to it. """ command: str """The command to run.""" args: Sequence[str] """The arguments to pass to the command.""" env: dict[str, str] \| None = None """The environment variables the CLI server will have access to. By default the subprocess will not inherit any environment variables from the parent process. If you want to inherit the environment variables from the parent process, use `env=os.environ`. """ @asynccontextmanager async def client_streams( self, ) -> AsyncIterator[ tuple[MemoryObjectReceiveStream[JSONRPCMessage \| Exception], MemoryObjectSendStream[JSONRPCMessage]] ]: server = StdioServerParameters(command=self.command, args=list(self.args), env=self.env) async with stdio_client(server=server) as (read_stream, write_stream): yield read_stream, write_stream`` | #### command `instance-attribute` `command: [str](https://docs.python.org/3/library/stdtypes.html#str)` The command to run. #### args `instance-attribute` `args: [Sequence](https://docs.python.org/3/library/collections.abc.html#collections.abc.Sequence "collections.abc.Sequence") [[str](https://docs.python.org/3/library/stdtypes.html#str) ]` The arguments to pass to the command. #### env `class-attribute` `instance-attribute` `env: [dict](https://docs.python.org/3/library/stdtypes.html#dict) [[str](https://docs.python.org/3/library/stdtypes.html#str) , [str](https://docs.python.org/3/library/stdtypes.html#str) ] | None = None` The environment variables the CLI server will have access to. By default the subprocess will not inherit any environment variables from the parent process. If you want to inherit the environment variables from the parent process, use `env=os.environ`. ### MCPServerHTTP `dataclass` Bases: `[MCPServer](#pydantic_ai.mcp.MCPServer "pydantic_ai.mcp.MCPServer") ` An MCP server that connects over streamable HTTP connections. This class implements the SSE transport from the MCP specification. See [https://spec.modelcontextprotocol.io/specification/2024-11-05/basic/transports/#http-with-sse](https://spec.modelcontextprotocol.io/specification/2024-11-05/basic/transports/#http-with-sse) for more information. The name "HTTP" is used since this implemented will be adapted in future to use the new [Streamable HTTP](https://github.com/modelcontextprotocol/specification/pull/206) currently in development. Note Using this class as an async context manager will create a new pool of HTTP connections to connect to a server which should already be running. Example: `from pydantic_ai import Agent from pydantic_ai.mcp import MCPServerHTTP server = MCPServerHTTP('http://localhost:3001/sse') # (1)! agent = Agent('openai:gpt-4o', mcp_servers=[server]) async def main(): async with agent.run_mcp_servers(): # (2)! ...` 1. E.g. you might be connecting to a server run with [`mcp-run-python`](../../mcp/run-python/) . 2. This will connect to a server running on `localhost:3001`. Source code in `pydantic_ai_slim/pydantic_ai/mcp.py` | | | | --- | --- | | 164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232 | ``@dataclass class MCPServerHTTP(MCPServer): """An MCP server that connects over streamable HTTP connections. This class implements the SSE transport from the MCP specification. See for more information. The name "HTTP" is used since this implemented will be adapted in future to use the new [Streamable HTTP](https://github.com/modelcontextprotocol/specification/pull/206) currently in development. !!! note Using this class as an async context manager will create a new pool of HTTP connections to connect to a server which should already be running. Example: ```python {py="3.10"} from pydantic_ai import Agent from pydantic_ai.mcp import MCPServerHTTP server = MCPServerHTTP('http://localhost:3001/sse') # (1)! agent = Agent('openai:gpt-4o', mcp_servers=[server]) async def main(): async with agent.run_mcp_servers(): # (2)! ... ``` 1. E.g. you might be connecting to a server run with [`mcp-run-python`](../mcp/run-python.md). 2. This will connect to a server running on `localhost:3001`. """ url: str """The URL of the SSE endpoint on the MCP server. For example for a server running locally, this might be `http://localhost:3001/sse`. """ headers: dict[str, Any] \| None = None """Optional HTTP headers to be sent with each request to the SSE endpoint. These headers will be passed directly to the underlying `httpx.AsyncClient`. Useful for authentication, custom headers, or other HTTP-specific configurations. """ timeout: float = 5 """Initial connection timeout in seconds for establishing the SSE connection. This timeout applies to the initial connection setup and handshake. If the connection cannot be established within this time, the operation will fail. """ sse_read_timeout: float = 60 * 5 """Maximum time in seconds to wait for new SSE messages before timing out. This timeout applies to the long-lived SSE connection after it's established. If no new messages are received within this time, the connection will be considered stale and may be closed. Defaults to 5 minutes (300 seconds). """ @asynccontextmanager async def client_streams( self, ) -> AsyncIterator[ tuple[MemoryObjectReceiveStream[JSONRPCMessage \| Exception], MemoryObjectSendStream[JSONRPCMessage]] ]: # pragma: no cover async with sse_client( url=self.url, headers=self.headers, timeout=self.timeout, sse_read_timeout=self.sse_read_timeout ) as (read_stream, write_stream): yield read_stream, write_stream`` | #### url `instance-attribute` `url: [str](https://docs.python.org/3/library/stdtypes.html#str)` The URL of the SSE endpoint on the MCP server. For example for a server running locally, this might be `http://localhost:3001/sse`. #### headers `class-attribute` `instance-attribute` `headers: [dict](https://docs.python.org/3/library/stdtypes.html#dict) [[str](https://docs.python.org/3/library/stdtypes.html#str) , [Any](https://docs.python.org/3/library/typing.html#typing.Any "typing.Any") ] | None = None` Optional HTTP headers to be sent with each request to the SSE endpoint. These headers will be passed directly to the underlying `httpx.AsyncClient`. Useful for authentication, custom headers, or other HTTP-specific configurations. #### timeout `class-attribute` `instance-attribute` `timeout: [float](https://docs.python.org/3/library/functions.html#float) = 5` Initial connection timeout in seconds for establishing the SSE connection. This timeout applies to the initial connection setup and handshake. If the connection cannot be established within this time, the operation will fail. #### sse\_read\_timeout `class-attribute` `instance-attribute` `sse_read_timeout: [float](https://docs.python.org/3/library/functions.html#float) = 60 * 5` Maximum time in seconds to wait for new SSE messages before timing out. This timeout applies to the long-lived SSE connection after it's established. If no new messages are received within this time, the connection will be considered stale and may be closed. Defaults to 5 minutes (300 seconds). --- # pydantic_ai.models - PydanticAI `pydantic_ai.models` ==================== Logic related to making requests to an LLM. The aim here is to make a common interface for different LLMs, so that the rest of the code can be agnostic to the specific LLM being used. ### KnownModelName `module-attribute` `KnownModelName = [TypeAliasType](https://typing-extensions.readthedocs.io/en/latest/index.html#typing_extensions.TypeAliasType "typing_extensions.TypeAliasType") ( "KnownModelName", [Literal](https://typing-extensions.readthedocs.io/en/latest/index.html#typing_extensions.Literal "typing_extensions.Literal") [ "anthropic:claude-3-7-sonnet-latest", "anthropic:claude-3-5-haiku-latest", "anthropic:claude-3-5-sonnet-latest", "anthropic:claude-3-opus-latest", "claude-3-7-sonnet-latest", "claude-3-5-haiku-latest", "bedrock:amazon.titan-tg1-large", "bedrock:amazon.titan-text-lite-v1", "bedrock:amazon.titan-text-express-v1", "bedrock:us.amazon.nova-pro-v1:0", "bedrock:us.amazon.nova-lite-v1:0", "bedrock:us.amazon.nova-micro-v1:0", "bedrock:anthropic.claude-3-5-sonnet-20241022-v2:0", "bedrock:us.anthropic.claude-3-5-sonnet-20241022-v2:0", "bedrock:anthropic.claude-3-5-haiku-20241022-v1:0", "bedrock:us.anthropic.claude-3-5-haiku-20241022-v1:0", "bedrock:anthropic.claude-instant-v1", "bedrock:anthropic.claude-v2:1", "bedrock:anthropic.claude-v2", "bedrock:anthropic.claude-3-sonnet-20240229-v1:0", "bedrock:us.anthropic.claude-3-sonnet-20240229-v1:0", "bedrock:anthropic.claude-3-haiku-20240307-v1:0", "bedrock:us.anthropic.claude-3-haiku-20240307-v1:0", "bedrock:anthropic.claude-3-opus-20240229-v1:0", "bedrock:us.anthropic.claude-3-opus-20240229-v1:0", "bedrock:anthropic.claude-3-5-sonnet-20240620-v1:0", "bedrock:us.anthropic.claude-3-5-sonnet-20240620-v1:0", "bedrock:anthropic.claude-3-7-sonnet-20250219-v1:0", "bedrock:us.anthropic.claude-3-7-sonnet-20250219-v1:0", "bedrock:cohere.command-text-v14", "bedrock:cohere.command-r-v1:0", "bedrock:cohere.command-r-plus-v1:0", "bedrock:cohere.command-light-text-v14", "bedrock:meta.llama3-8b-instruct-v1:0", "bedrock:meta.llama3-70b-instruct-v1:0", "bedrock:meta.llama3-1-8b-instruct-v1:0", "bedrock:us.meta.llama3-1-8b-instruct-v1:0", "bedrock:meta.llama3-1-70b-instruct-v1:0", "bedrock:us.meta.llama3-1-70b-instruct-v1:0", "bedrock:meta.llama3-1-405b-instruct-v1:0", "bedrock:us.meta.llama3-2-11b-instruct-v1:0", "bedrock:us.meta.llama3-2-90b-instruct-v1:0", "bedrock:us.meta.llama3-2-1b-instruct-v1:0", "bedrock:us.meta.llama3-2-3b-instruct-v1:0", "bedrock:us.meta.llama3-3-70b-instruct-v1:0", "bedrock:mistral.mistral-7b-instruct-v0:2", "bedrock:mistral.mixtral-8x7b-instruct-v0:1", "bedrock:mistral.mistral-large-2402-v1:0", "bedrock:mistral.mistral-large-2407-v1:0", "claude-3-5-sonnet-latest", "claude-3-opus-latest", "cohere:c4ai-aya-expanse-32b", "cohere:c4ai-aya-expanse-8b", "cohere:command", "cohere:command-light", "cohere:command-light-nightly", "cohere:command-nightly", "cohere:command-r", "cohere:command-r-03-2024", "cohere:command-r-08-2024", "cohere:command-r-plus", "cohere:command-r-plus-04-2024", "cohere:command-r-plus-08-2024", "cohere:command-r7b-12-2024", "deepseek:deepseek-chat", "deepseek:deepseek-reasoner", "google-gla:gemini-1.0-pro", "google-gla:gemini-1.5-flash", "google-gla:gemini-1.5-flash-8b", "google-gla:gemini-1.5-pro", "google-gla:gemini-2.0-flash-exp", "google-gla:gemini-2.0-flash-thinking-exp-01-21", "google-gla:gemini-exp-1206", "google-gla:gemini-2.0-flash", "google-gla:gemini-2.0-flash-lite-preview-02-05", "google-gla:gemini-2.0-pro-exp-02-05", "google-gla:gemini-2.5-pro-exp-03-25", "google-vertex:gemini-1.0-pro", "google-vertex:gemini-1.5-flash", "google-vertex:gemini-1.5-flash-8b", "google-vertex:gemini-1.5-pro", "google-vertex:gemini-2.0-flash-exp", "google-vertex:gemini-2.0-flash-thinking-exp-01-21", "google-vertex:gemini-exp-1206", "google-vertex:gemini-2.0-flash", "google-vertex:gemini-2.0-flash-lite-preview-02-05", "google-vertex:gemini-2.0-pro-exp-02-05", "google-vertex:gemini-2.5-pro-exp-03-25", "gpt-3.5-turbo", "gpt-3.5-turbo-0125", "gpt-3.5-turbo-0301", "gpt-3.5-turbo-0613", "gpt-3.5-turbo-1106", "gpt-3.5-turbo-16k", "gpt-3.5-turbo-16k-0613", "gpt-4", "gpt-4-0125-preview", "gpt-4-0314", "gpt-4-0613", "gpt-4-1106-preview", "gpt-4-32k", "gpt-4-32k-0314", "gpt-4-32k-0613", "gpt-4-turbo", "gpt-4-turbo-2024-04-09", "gpt-4-turbo-preview", "gpt-4-vision-preview", "gpt-4o", "gpt-4o-2024-05-13", "gpt-4o-2024-08-06", "gpt-4o-2024-11-20", "gpt-4o-audio-preview", "gpt-4o-audio-preview-2024-10-01", "gpt-4o-audio-preview-2024-12-17", "gpt-4o-mini", "gpt-4o-mini-2024-07-18", "gpt-4o-mini-audio-preview", "gpt-4o-mini-audio-preview-2024-12-17", "gpt-4o-mini-search-preview", "gpt-4o-mini-search-preview-2025-03-11", "gpt-4o-search-preview", "gpt-4o-search-preview-2025-03-11", "groq:distil-whisper-large-v3-en", "groq:gemma2-9b-it", "groq:llama-3.3-70b-versatile", "groq:llama-3.1-8b-instant", "groq:llama-guard-3-8b", "groq:llama3-70b-8192", "groq:llama3-8b-8192", "groq:whisper-large-v3", "groq:whisper-large-v3-turbo", "groq:playai-tts", "groq:playai-tts-arabic", "groq:qwen-qwq-32b", "groq:mistral-saba-24b", "groq:qwen-2.5-coder-32b", "groq:qwen-2.5-32b", "groq:deepseek-r1-distill-qwen-32b", "groq:deepseek-r1-distill-llama-70b", "groq:llama-3.3-70b-specdec", "groq:llama-3.2-1b-preview", "groq:llama-3.2-3b-preview", "groq:llama-3.2-11b-vision-preview", "groq:llama-3.2-90b-vision-preview", "mistral:codestral-latest", "mistral:mistral-large-latest", "mistral:mistral-moderation-latest", "mistral:mistral-small-latest", "o1", "o1-2024-12-17", "o1-mini", "o1-mini-2024-09-12", "o1-preview", "o1-preview-2024-09-12", "o3-mini", "o3-mini-2025-01-31", "openai:chatgpt-4o-latest", "openai:gpt-3.5-turbo", "openai:gpt-3.5-turbo-0125", "openai:gpt-3.5-turbo-0301", "openai:gpt-3.5-turbo-0613", "openai:gpt-3.5-turbo-1106", "openai:gpt-3.5-turbo-16k", "openai:gpt-3.5-turbo-16k-0613", "openai:gpt-4", "openai:gpt-4-0125-preview", "openai:gpt-4-0314", "openai:gpt-4-0613", "openai:gpt-4-1106-preview", "openai:gpt-4-32k", "openai:gpt-4-32k-0314", "openai:gpt-4-32k-0613", "openai:gpt-4-turbo", "openai:gpt-4-turbo-2024-04-09", "openai:gpt-4-turbo-preview", "openai:gpt-4-vision-preview", "openai:gpt-4o", "openai:gpt-4o-2024-05-13", "openai:gpt-4o-2024-08-06", "openai:gpt-4o-2024-11-20", "openai:gpt-4o-audio-preview", "openai:gpt-4o-audio-preview-2024-10-01", "openai:gpt-4o-audio-preview-2024-12-17", "openai:gpt-4o-mini", "openai:gpt-4o-mini-2024-07-18", "openai:gpt-4o-mini-audio-preview", "openai:gpt-4o-mini-audio-preview-2024-12-17", "openai:gpt-4o-mini-search-preview", "openai:gpt-4o-mini-search-preview-2025-03-11", "openai:gpt-4o-search-preview", "openai:gpt-4o-search-preview-2025-03-11", "openai:o1", "openai:o1-2024-12-17", "openai:o1-mini", "openai:o1-mini-2024-09-12", "openai:o1-preview", "openai:o1-preview-2024-09-12", "openai:o3-mini", "openai:o3-mini-2025-01-31", "test", ], )` Known model names that can be used with the `model` parameter of [`Agent`](../../agent/#pydantic_ai.agent.Agent) . `KnownModelName` is provided as a concise way to specify a model. ### ModelRequestParameters `dataclass` Configuration for an agent's request to a model, specifically related to tools and result handling. Source code in `pydantic_ai_slim/pydantic_ai/models/__init__.py` | | | | --- | --- | | 241
242
243
244
245
246
247 | `@dataclass class ModelRequestParameters: """Configuration for an agent's request to a model, specifically related to tools and result handling.""" function_tools: list[ToolDefinition] allow_text_result: bool result_tools: list[ToolDefinition]` | ### Model Bases: `[ABC](https://docs.python.org/3/library/abc.html#abc.ABC "abc.ABC") ` Abstract class for a model. Source code in `pydantic_ai_slim/pydantic_ai/models/__init__.py` | | | | --- | --- | | 250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298 | ``class Model(ABC): """Abstract class for a model.""" @abstractmethod async def request( self, messages: list[ModelMessage], model_settings: ModelSettings \| None, model_request_parameters: ModelRequestParameters, ) -> tuple[ModelResponse, Usage]: """Make a request to the model.""" raise NotImplementedError() @asynccontextmanager async def request_stream( self, messages: list[ModelMessage], model_settings: ModelSettings \| None, model_request_parameters: ModelRequestParameters, ) -> AsyncIterator[StreamedResponse]: """Make a request to the model and return a streaming response.""" # This method is not required, but you need to implement it if you want to support streamed responses raise NotImplementedError(f'Streamed requests not supported by this {self.__class__.__name__}') # yield is required to make this a generator for type checking # noinspection PyUnreachableCode yield # pragma: no cover @property @abstractmethod def model_name(self) -> str: """The model name.""" raise NotImplementedError() @property @abstractmethod def system(self) -> str: """The system / model provider, ex: openai. Use to populate the `gen_ai.system` OpenTelemetry semantic convention attribute, so should use well-known values listed in https://opentelemetry.io/docs/specs/semconv/attributes-registry/gen-ai/#gen-ai-system when applicable. """ raise NotImplementedError() @property def base_url(self) -> str \| None: """The base URL for the provider API, if available.""" return None`` | #### request `abstractmethod` `async` `request( messages: [list](https://docs.python.org/3/library/stdtypes.html#list) [[ModelMessage](../../messages/#pydantic_ai.messages.ModelMessage "pydantic_ai.messages.ModelMessage") ], model_settings: [ModelSettings](../../settings/#pydantic_ai.settings.ModelSettings "pydantic_ai.settings.ModelSettings") | None, model_request_parameters: [ModelRequestParameters](#pydantic_ai.models.ModelRequestParameters "pydantic_ai.models.ModelRequestParameters") , ) -> [tuple](https://docs.python.org/3/library/stdtypes.html#tuple) [[ModelResponse](../../messages/#pydantic_ai.messages.ModelResponse "pydantic_ai.messages.ModelResponse") , [Usage](../../usage/#pydantic_ai.usage.Usage "pydantic_ai.usage.Usage") ]` Make a request to the model. Source code in `pydantic_ai_slim/pydantic_ai/models/__init__.py` | | | | --- | --- | | 253
254
255
256
257
258
259
260
261 | `@abstractmethod async def request( self, messages: list[ModelMessage], model_settings: ModelSettings \| None, model_request_parameters: ModelRequestParameters, ) -> tuple[ModelResponse, Usage]: """Make a request to the model.""" raise NotImplementedError()` | #### request\_stream `async` `request_stream( messages: [list](https://docs.python.org/3/library/stdtypes.html#list) [[ModelMessage](../../messages/#pydantic_ai.messages.ModelMessage "pydantic_ai.messages.ModelMessage") ], model_settings: [ModelSettings](../../settings/#pydantic_ai.settings.ModelSettings "pydantic_ai.settings.ModelSettings") | None, model_request_parameters: [ModelRequestParameters](#pydantic_ai.models.ModelRequestParameters "pydantic_ai.models.ModelRequestParameters") , ) -> [AsyncIterator](https://docs.python.org/3/library/collections.abc.html#collections.abc.AsyncIterator "collections.abc.AsyncIterator") [[StreamedResponse](#pydantic_ai.models.StreamedResponse "pydantic_ai.models.StreamedResponse") ]` Make a request to the model and return a streaming response. Source code in `pydantic_ai_slim/pydantic_ai/models/__init__.py` | | | | --- | --- | | 263
264
265
266
267
268
269
270
271
272
273
274
275 | `@asynccontextmanager async def request_stream( self, messages: list[ModelMessage], model_settings: ModelSettings \| None, model_request_parameters: ModelRequestParameters, ) -> AsyncIterator[StreamedResponse]: """Make a request to the model and return a streaming response.""" # This method is not required, but you need to implement it if you want to support streamed responses raise NotImplementedError(f'Streamed requests not supported by this {self.__class__.__name__}') # yield is required to make this a generator for type checking # noinspection PyUnreachableCode yield # pragma: no cover` | #### model\_name `abstractmethod` `property` `model_name: [str](https://docs.python.org/3/library/stdtypes.html#str)` The model name. #### system `abstractmethod` `property` `system: [str](https://docs.python.org/3/library/stdtypes.html#str)` The system / model provider, ex: openai. Use to populate the `gen_ai.system` OpenTelemetry semantic convention attribute, so should use well-known values listed in https://opentelemetry.io/docs/specs/semconv/attributes-registry/gen-ai/#gen-ai-system when applicable. #### base\_url `property` `base_url: [str](https://docs.python.org/3/library/stdtypes.html#str) | None` The base URL for the provider API, if available. ### StreamedResponse `dataclass` Bases: `[ABC](https://docs.python.org/3/library/abc.html#abc.ABC "abc.ABC") ` Streamed response from an LLM when calling a tool. Source code in `pydantic_ai_slim/pydantic_ai/models/__init__.py` | | | | --- | --- | | 301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348 | ``@dataclass class StreamedResponse(ABC): """Streamed response from an LLM when calling a tool.""" _parts_manager: ModelResponsePartsManager = field(default_factory=ModelResponsePartsManager, init=False) _event_iterator: AsyncIterator[ModelResponseStreamEvent] \| None = field(default=None, init=False) _usage: Usage = field(default_factory=Usage, init=False) def __aiter__(self) -> AsyncIterator[ModelResponseStreamEvent]: """Stream the response as an async iterable of [`ModelResponseStreamEvent`][pydantic_ai.messages.ModelResponseStreamEvent]s.""" if self._event_iterator is None: self._event_iterator = self._get_event_iterator() return self._event_iterator @abstractmethod async def _get_event_iterator(self) -> AsyncIterator[ModelResponseStreamEvent]: """Return an async iterator of [`ModelResponseStreamEvent`][pydantic_ai.messages.ModelResponseStreamEvent]s. This method should be implemented by subclasses to translate the vendor-specific stream of events into pydantic_ai-format events. It should use the `_parts_manager` to handle deltas, and should update the `_usage` attributes as it goes. """ raise NotImplementedError() # noinspection PyUnreachableCode yield def get(self) -> ModelResponse: """Build a [`ModelResponse`][pydantic_ai.messages.ModelResponse] from the data received from the stream so far.""" return ModelResponse( parts=self._parts_manager.get_parts(), model_name=self.model_name, timestamp=self.timestamp ) def usage(self) -> Usage: """Get the usage of the response so far. This will not be the final usage until the stream is exhausted.""" return self._usage @property @abstractmethod def model_name(self) -> str: """Get the model name of the response.""" raise NotImplementedError() @property @abstractmethod def timestamp(self) -> datetime: """Get the timestamp of the response.""" raise NotImplementedError()`` | #### \_\_aiter\_\_ `__aiter__() -> [AsyncIterator](https://docs.python.org/3/library/collections.abc.html#collections.abc.AsyncIterator "collections.abc.AsyncIterator") [[ModelResponseStreamEvent](../../messages/#pydantic_ai.messages.ModelResponseStreamEvent "pydantic_ai.messages.ModelResponseStreamEvent") ]` Stream the response as an async iterable of [`ModelResponseStreamEvent`](../../messages/#pydantic_ai.messages.ModelResponseStreamEvent) s. Source code in `pydantic_ai_slim/pydantic_ai/models/__init__.py` | | | | --- | --- | | 309
310
311
312
313 | ``def __aiter__(self) -> AsyncIterator[ModelResponseStreamEvent]: """Stream the response as an async iterable of [`ModelResponseStreamEvent`][pydantic_ai.messages.ModelResponseStreamEvent]s.""" if self._event_iterator is None: self._event_iterator = self._get_event_iterator() return self._event_iterator`` | #### get `get() -> [ModelResponse](../../messages/#pydantic_ai.messages.ModelResponse "pydantic_ai.messages.ModelResponse")` Build a [`ModelResponse`](../../messages/#pydantic_ai.messages.ModelResponse) from the data received from the stream so far. Source code in `pydantic_ai_slim/pydantic_ai/models/__init__.py` | | | | --- | --- | | 328
329
330
331
332 | ``def get(self) -> ModelResponse: """Build a [`ModelResponse`][pydantic_ai.messages.ModelResponse] from the data received from the stream so far.""" return ModelResponse( parts=self._parts_manager.get_parts(), model_name=self.model_name, timestamp=self.timestamp )`` | #### usage `usage() -> [Usage](../../usage/#pydantic_ai.usage.Usage "pydantic_ai.usage.Usage")` Get the usage of the response so far. This will not be the final usage until the stream is exhausted. Source code in `pydantic_ai_slim/pydantic_ai/models/__init__.py` | | | | --- | --- | | 334
335
336 | `def usage(self) -> Usage: """Get the usage of the response so far. This will not be the final usage until the stream is exhausted.""" return self._usage` | #### model\_name `abstractmethod` `property` `model_name: [str](https://docs.python.org/3/library/stdtypes.html#str)` Get the model name of the response. #### timestamp `abstractmethod` `property` `timestamp: [datetime](https://docs.python.org/3/library/datetime.html#datetime.datetime "datetime.datetime")` Get the timestamp of the response. ### ALLOW\_MODEL\_REQUESTS `module-attribute` `ALLOW_MODEL_REQUESTS = True` Whether to allow requests to models. This global setting allows you to disable request to most models, e.g. to make sure you don't accidentally make costly requests to a model during tests. The testing models [`TestModel`](../test/#pydantic_ai.models.test.TestModel) and [`FunctionModel`](../function/#pydantic_ai.models.function.FunctionModel) are no affected by this setting. ### check\_allow\_model\_requests `check_allow_model_requests() -> None` Check if model requests are allowed. If you're defining your own models that have costs or latency associated with their use, you should call this in [`Model.request`](#pydantic_ai.models.Model.request) and [`Model.request_stream`](#pydantic_ai.models.Model.request_stream) . Raises: | Type | Description | | --- | --- | | `[RuntimeError](https://docs.python.org/3/library/exceptions.html#RuntimeError) ` | If model requests are not allowed. | Source code in `pydantic_ai_slim/pydantic_ai/models/__init__.py` | | | | --- | --- | | 362
363
364
365
366
367
368
369
370
371
372 | ``def check_allow_model_requests() -> None: """Check if model requests are allowed. If you're defining your own models that have costs or latency associated with their use, you should call this in [`Model.request`][pydantic_ai.models.Model.request] and [`Model.request_stream`][pydantic_ai.models.Model.request_stream]. Raises: RuntimeError: If model requests are not allowed. """ if not ALLOW_MODEL_REQUESTS: raise RuntimeError('Model requests are not allowed, since ALLOW_MODEL_REQUESTS is False')`` | ### override\_allow\_model\_requests `override_allow_model_requests( allow_model_requests: [bool](https://docs.python.org/3/library/functions.html#bool) , ) -> [Iterator](https://docs.python.org/3/library/collections.abc.html#collections.abc.Iterator "collections.abc.Iterator") [None]` Context manager to temporarily override [`ALLOW_MODEL_REQUESTS`](#pydantic_ai.models.ALLOW_MODEL_REQUESTS) . Parameters: | Name | Type | Description | Default | | --- | --- | --- | --- | | `allow_model_requests` | `[bool](https://docs.python.org/3/library/functions.html#bool) ` | Whether to allow model requests within the context. | _required_ | Source code in `pydantic_ai_slim/pydantic_ai/models/__init__.py` | | | | --- | --- | | 375
376
377
378
379
380
381
382
383
384
385
386
387
388 | ``@contextmanager def override_allow_model_requests(allow_model_requests: bool) -> Iterator[None]: """Context manager to temporarily override [`ALLOW_MODEL_REQUESTS`][pydantic_ai.models.ALLOW_MODEL_REQUESTS]. Args: allow_model_requests: Whether to allow model requests within the context. """ global ALLOW_MODEL_REQUESTS old_value = ALLOW_MODEL_REQUESTS ALLOW_MODEL_REQUESTS = allow_model_requests # pyright: ignore[reportConstantRedefinition] try: yield finally: ALLOW_MODEL_REQUESTS = old_value # pyright: ignore[reportConstantRedefinition]`` | --- # pydantic_ai.models.openai - PydanticAI `pydantic_ai.models.openai` =========================== Setup ----- For details on how to set up authentication with this model, see [model configuration for OpenAI](../../../models/#openai) . ### OpenAIModelName `module-attribute` `OpenAIModelName = [Union](https://docs.python.org/3/library/typing.html#typing.Union "typing.Union") [[str](https://docs.python.org/3/library/stdtypes.html#str) , ChatModel]` Possible OpenAI model names. Since OpenAI supports a variety of date-stamped models, we explicitly list the latest models but allow any name in the type hints. See [the OpenAI docs](https://platform.openai.com/docs/models) for a full list. Using this more broad type for the model name instead of the ChatModel definition allows this model to be used more easily with other model types (ie, Ollama, Deepseek). ### OpenAIModelSettings Bases: `[ModelSettings](../../settings/#pydantic_ai.settings.ModelSettings "pydantic_ai.settings.ModelSettings") ` Settings used for an OpenAI model request. ALL FIELDS MUST BE `openai_` PREFIXED SO YOU CAN MERGE THEM WITH OTHER MODELS. Source code in `pydantic_ai_slim/pydantic_ai/models/openai.py` | | | | --- | --- | | 89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106 | ``class OpenAIModelSettings(ModelSettings, total=False): """Settings used for an OpenAI model request. ALL FIELDS MUST BE `openai_` PREFIXED SO YOU CAN MERGE THEM WITH OTHER MODELS. """ openai_reasoning_effort: ReasoningEffort """Constrains effort on reasoning for [reasoning models](https://platform.openai.com/docs/guides/reasoning). Currently supported values are `low`, `medium`, and `high`. Reducing reasoning effort can result in faster responses and fewer tokens used on reasoning in a response. """ openai_user: str """A unique identifier representing the end-user, which can help OpenAI monitor and detect abuse. See [OpenAI's safety best practices](https://platform.openai.com/docs/guides/safety-best-practices#end-user-ids) for more details. """`` | #### openai\_reasoning\_effort `instance-attribute` `openai_reasoning_effort: ReasoningEffort` Constrains effort on reasoning for [reasoning models](https://platform.openai.com/docs/guides/reasoning) . Currently supported values are `low`, `medium`, and `high`. Reducing reasoning effort can result in faster responses and fewer tokens used on reasoning in a response. #### openai\_user `instance-attribute` `openai_user: [str](https://docs.python.org/3/library/stdtypes.html#str)` A unique identifier representing the end-user, which can help OpenAI monitor and detect abuse. See [OpenAI's safety best practices](https://platform.openai.com/docs/guides/safety-best-practices#end-user-ids) for more details. ### OpenAIResponsesModelSettings Bases: `[OpenAIModelSettings](#pydantic_ai.models.openai.OpenAIModelSettings "pydantic_ai.models.openai.OpenAIModelSettings") ` Settings used for an OpenAI Responses model request. ALL FIELDS MUST BE `openai_` PREFIXED SO YOU CAN MERGE THEM WITH OTHER MODELS. Source code in `pydantic_ai_slim/pydantic_ai/models/openai.py` | | | | --- | --- | | 109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140 | ``class OpenAIResponsesModelSettings(OpenAIModelSettings, total=False): """Settings used for an OpenAI Responses model request. ALL FIELDS MUST BE `openai_` PREFIXED SO YOU CAN MERGE THEM WITH OTHER MODELS. """ openai_builtin_tools: Sequence[FileSearchToolParam \| WebSearchToolParam \| ComputerToolParam] """The provided OpenAI built-in tools to use. See [OpenAI's built-in tools](https://platform.openai.com/docs/guides/tools?api-mode=responses) for more details. """ openai_reasoning_generate_summary: Literal['detailed', 'concise'] """A summary of the reasoning performed by the model. This can be useful for debugging and understanding the model's reasoning process. One of `concise` or `detailed`. Check the [OpenAI Computer use documentation](https://platform.openai.com/docs/guides/tools-computer-use#1-send-a-request-to-the-model) for more details. """ openai_truncation: Literal['disabled', 'auto'] """The truncation strategy to use for the model response. It can be either: - `disabled` (default): If a model response will exceed the context window size for a model, the request will fail with a 400 error. - `auto`: If the context of this response and previous ones exceeds the model's context window size, the model will truncate the response to fit the context window by dropping input items in the middle of the conversation. """`` | #### openai\_builtin\_tools `instance-attribute` `openai_builtin_tools: [Sequence](https://docs.python.org/3/library/collections.abc.html#collections.abc.Sequence "collections.abc.Sequence") [ FileSearchToolParam | WebSearchToolParam | ComputerToolParam ]` The provided OpenAI built-in tools to use. See [OpenAI's built-in tools](https://platform.openai.com/docs/guides/tools?api-mode=responses) for more details. #### openai\_reasoning\_generate\_summary `instance-attribute` `openai_reasoning_generate_summary: [Literal](https://docs.python.org/3/library/typing.html#typing.Literal "typing.Literal") [ "detailed", "concise" ]` A summary of the reasoning performed by the model. This can be useful for debugging and understanding the model's reasoning process. One of `concise` or `detailed`. Check the [OpenAI Computer use documentation](https://platform.openai.com/docs/guides/tools-computer-use#1-send-a-request-to-the-model) for more details. #### openai\_truncation `instance-attribute` `openai_truncation: [Literal](https://docs.python.org/3/library/typing.html#typing.Literal "typing.Literal") ['disabled', 'auto']` The truncation strategy to use for the model response. It can be either: - `disabled` (default): If a model response will exceed the context window size for a model, the request will fail with a 400 error. - `auto`: If the context of this response and previous ones exceeds the model's context window size, the model will truncate the response to fit the context window by dropping input items in the middle of the conversation. ### OpenAIModel `dataclass` Bases: `[Model](../base/#pydantic_ai.models.Model "pydantic_ai.models.Model") ` A model that uses the OpenAI API. Internally, this uses the [OpenAI Python client](https://github.com/openai/openai-python) to interact with the API. Apart from `__init__`, all methods are private or match those of the base class. Source code in `pydantic_ai_slim/pydantic_ai/models/openai.py` | | | | --- | --- | | 143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444 | ``@dataclass(init=False) class OpenAIModel(Model): """A model that uses the OpenAI API. Internally, this uses the [OpenAI Python client](https://github.com/openai/openai-python) to interact with the API. Apart from `__init__`, all methods are private or match those of the base class. """ client: AsyncOpenAI = field(repr=False) system_prompt_role: OpenAISystemPromptRole \| None = field(default=None) _model_name: OpenAIModelName = field(repr=False) _system: str = field(default='openai', repr=False) def __init__( self, model_name: OpenAIModelName, *, provider: Literal['openai', 'deepseek', 'azure'] \| Provider[AsyncOpenAI] = 'openai', system_prompt_role: OpenAISystemPromptRole \| None = None, ): """Initialize an OpenAI model. Args: model_name: The name of the OpenAI model to use. List of model names available [here](https://github.com/openai/openai-python/blob/v1.54.3/src/openai/types/chat_model.py#L7) (Unfortunately, despite being ask to do so, OpenAI do not provide `.inv` files for their API). provider: The provider to use. Defaults to `'openai'`. system_prompt_role: The role to use for the system prompt message. If not provided, defaults to `'system'`. In the future, this may be inferred from the model name. """ self._model_name = model_name if isinstance(provider, str): provider = infer_provider(provider) self.client = provider.client self.system_prompt_role = system_prompt_role @property def base_url(self) -> str: return str(self.client.base_url) async def request( self, messages: list[ModelMessage], model_settings: ModelSettings \| None, model_request_parameters: ModelRequestParameters, ) -> tuple[ModelResponse, usage.Usage]: check_allow_model_requests() response = await self._completions_create( messages, False, cast(OpenAIModelSettings, model_settings or {}), model_request_parameters ) return self._process_response(response), _map_usage(response) @asynccontextmanager async def request_stream( self, messages: list[ModelMessage], model_settings: ModelSettings \| None, model_request_parameters: ModelRequestParameters, ) -> AsyncIterator[StreamedResponse]: check_allow_model_requests() response = await self._completions_create( messages, True, cast(OpenAIModelSettings, model_settings or {}), model_request_parameters ) async with response: yield await self._process_streamed_response(response) @property def model_name(self) -> OpenAIModelName: """The model name.""" return self._model_name @property def system(self) -> str: """The system / model provider.""" return self._system @overload async def _completions_create( self, messages: list[ModelMessage], stream: Literal[True], model_settings: OpenAIModelSettings, model_request_parameters: ModelRequestParameters, ) -> AsyncStream[ChatCompletionChunk]: ... @overload async def _completions_create( self, messages: list[ModelMessage], stream: Literal[False], model_settings: OpenAIModelSettings, model_request_parameters: ModelRequestParameters, ) -> chat.ChatCompletion: ... async def _completions_create( self, messages: list[ModelMessage], stream: bool, model_settings: OpenAIModelSettings, model_request_parameters: ModelRequestParameters, ) -> chat.ChatCompletion \| AsyncStream[ChatCompletionChunk]: tools = self._get_tools(model_request_parameters) # standalone function to make it easier to override if not tools: tool_choice: Literal['none', 'required', 'auto'] \| None = None elif not model_request_parameters.allow_text_result: tool_choice = 'required' else: tool_choice = 'auto' openai_messages: list[chat.ChatCompletionMessageParam] = [] for m in messages: async for msg in self._map_message(m): openai_messages.append(msg) try: return await self.client.chat.completions.create( model=self._model_name, messages=openai_messages, n=1, parallel_tool_calls=model_settings.get('parallel_tool_calls', NOT_GIVEN), tools=tools or NOT_GIVEN, tool_choice=tool_choice or NOT_GIVEN, stream=stream, stream_options={'include_usage': True} if stream else NOT_GIVEN, max_completion_tokens=model_settings.get('max_tokens', NOT_GIVEN), temperature=model_settings.get('temperature', NOT_GIVEN), top_p=model_settings.get('top_p', NOT_GIVEN), timeout=model_settings.get('timeout', NOT_GIVEN), seed=model_settings.get('seed', NOT_GIVEN), presence_penalty=model_settings.get('presence_penalty', NOT_GIVEN), frequency_penalty=model_settings.get('frequency_penalty', NOT_GIVEN), logit_bias=model_settings.get('logit_bias', NOT_GIVEN), reasoning_effort=model_settings.get('openai_reasoning_effort', NOT_GIVEN), user=model_settings.get('openai_user', NOT_GIVEN), ) except APIStatusError as e: if (status_code := e.status_code) >= 400: raise ModelHTTPError(status_code=status_code, model_name=self.model_name, body=e.body) from e raise def _process_response(self, response: chat.ChatCompletion) -> ModelResponse: """Process a non-streamed response, and prepare a message to return.""" timestamp = datetime.fromtimestamp(response.created, tz=timezone.utc) choice = response.choices[0] items: list[ModelResponsePart] = [] if choice.message.content is not None: items.append(TextPart(choice.message.content)) if choice.message.tool_calls is not None: for c in choice.message.tool_calls: items.append(ToolCallPart(c.function.name, c.function.arguments, tool_call_id=c.id)) return ModelResponse(items, model_name=response.model, timestamp=timestamp) async def _process_streamed_response(self, response: AsyncStream[ChatCompletionChunk]) -> OpenAIStreamedResponse: """Process a streamed response, and prepare a streaming response to return.""" peekable_response = _utils.PeekableAsyncStream(response) first_chunk = await peekable_response.peek() if isinstance(first_chunk, _utils.Unset): raise UnexpectedModelBehavior('Streamed response ended without content or tool calls') return OpenAIStreamedResponse( _model_name=self._model_name, _response=peekable_response, _timestamp=datetime.fromtimestamp(first_chunk.created, tz=timezone.utc), ) def _get_tools(self, model_request_parameters: ModelRequestParameters) -> list[chat.ChatCompletionToolParam]: tools = [self._map_tool_definition(r) for r in model_request_parameters.function_tools] if model_request_parameters.result_tools: tools += [self._map_tool_definition(r) for r in model_request_parameters.result_tools] return tools async def _map_message(self, message: ModelMessage) -> AsyncIterable[chat.ChatCompletionMessageParam]: """Just maps a `pydantic_ai.Message` to a `openai.types.ChatCompletionMessageParam`.""" if isinstance(message, ModelRequest): async for item in self._map_user_message(message): yield item elif isinstance(message, ModelResponse): texts: list[str] = [] tool_calls: list[chat.ChatCompletionMessageToolCallParam] = [] for item in message.parts: if isinstance(item, TextPart): texts.append(item.content) elif isinstance(item, ToolCallPart): tool_calls.append(self._map_tool_call(item)) else: assert_never(item) message_param = chat.ChatCompletionAssistantMessageParam(role='assistant') if texts: # Note: model responses from this model should only have one text item, so the following # shouldn't merge multiple texts into one unless you switch models between runs: message_param['content'] = '\n\n'.join(texts) if tool_calls: message_param['tool_calls'] = tool_calls yield message_param else: assert_never(message) @staticmethod def _map_tool_call(t: ToolCallPart) -> chat.ChatCompletionMessageToolCallParam: return chat.ChatCompletionMessageToolCallParam( id=_guard_tool_call_id(t=t), type='function', function={'name': t.tool_name, 'arguments': t.args_as_json_str()}, ) @staticmethod def _map_tool_definition(f: ToolDefinition) -> chat.ChatCompletionToolParam: return { 'type': 'function', 'function': { 'name': f.name, 'description': f.description, 'parameters': f.parameters_json_schema, }, } async def _map_user_message(self, message: ModelRequest) -> AsyncIterable[chat.ChatCompletionMessageParam]: for part in message.parts: if isinstance(part, SystemPromptPart): if self.system_prompt_role == 'developer': yield chat.ChatCompletionDeveloperMessageParam(role='developer', content=part.content) elif self.system_prompt_role == 'user': yield chat.ChatCompletionUserMessageParam(role='user', content=part.content) else: yield chat.ChatCompletionSystemMessageParam(role='system', content=part.content) elif isinstance(part, UserPromptPart): yield await self._map_user_prompt(part) elif isinstance(part, ToolReturnPart): yield chat.ChatCompletionToolMessageParam( role='tool', tool_call_id=_guard_tool_call_id(t=part), content=part.model_response_str(), ) elif isinstance(part, RetryPromptPart): if part.tool_name is None: yield chat.ChatCompletionUserMessageParam(role='user', content=part.model_response()) else: yield chat.ChatCompletionToolMessageParam( role='tool', tool_call_id=_guard_tool_call_id(t=part), content=part.model_response(), ) else: assert_never(part) @staticmethod async def _map_user_prompt(part: UserPromptPart) -> chat.ChatCompletionUserMessageParam: content: str \| list[ChatCompletionContentPartParam] if isinstance(part.content, str): content = part.content else: content = [] for item in part.content: if isinstance(item, str): content.append(ChatCompletionContentPartTextParam(text=item, type='text')) elif isinstance(item, ImageUrl): image_url = ImageURL(url=item.url) content.append(ChatCompletionContentPartImageParam(image_url=image_url, type='image_url')) elif isinstance(item, BinaryContent): base64_encoded = base64.b64encode(item.data).decode('utf-8') if item.is_image: image_url = ImageURL(url=f'data:{item.media_type};base64,{base64_encoded}') content.append(ChatCompletionContentPartImageParam(image_url=image_url, type='image_url')) elif item.is_audio: assert item.format in ('wav', 'mp3') audio = InputAudio(data=base64_encoded, format=item.format) content.append(ChatCompletionContentPartInputAudioParam(input_audio=audio, type='input_audio')) else: # pragma: no cover raise RuntimeError(f'Unsupported binary content type: {item.media_type}') elif isinstance(item, AudioUrl): # pragma: no cover client = cached_async_http_client() response = await client.get(item.url) response.raise_for_status() base64_encoded = base64.b64encode(response.content).decode('utf-8') audio = InputAudio(data=base64_encoded, format=response.headers.get('content-type')) content.append(ChatCompletionContentPartInputAudioParam(input_audio=audio, type='input_audio')) elif isinstance(item, DocumentUrl): # pragma: no cover raise NotImplementedError('DocumentUrl is not supported for OpenAI') # The following implementation should have worked, but it seems we have the following error: # pydantic_ai.exceptions.ModelHTTPError: status_code: 400, model_name: gpt-4o, body: # { # 'message': "Unknown parameter: 'messages[1].content[1].file.data'.", # 'type': 'invalid_request_error', # 'param': 'messages[1].content[1].file.data', # 'code': 'unknown_parameter' # } # # client = cached_async_http_client() # response = await client.get(item.url) # response.raise_for_status() # base64_encoded = base64.b64encode(response.content).decode('utf-8') # media_type = response.headers.get('content-type').split(';')[0] # file_data = f'data:{media_type};base64,{base64_encoded}' # file = File(file={'file_data': file_data, 'file_name': item.url, 'file_id': item.url}, type='file') # content.append(file) else: assert_never(item) return chat.ChatCompletionUserMessageParam(role='user', content=content)`` | #### \_\_init\_\_ `__init__( model_name: [OpenAIModelName](#pydantic_ai.models.openai.OpenAIModelName "pydantic_ai.models.openai.OpenAIModelName") , *, provider: ( [Literal](https://docs.python.org/3/library/typing.html#typing.Literal "typing.Literal") ["openai", "deepseek", "azure"] | [Provider](../../providers/#pydantic_ai.providers.Provider "pydantic_ai.providers.Provider") [AsyncOpenAI] ) = "openai", system_prompt_role: OpenAISystemPromptRole | None = None )` Initialize an OpenAI model. Parameters: | Name | Type | Description | Default | | --- | --- | --- | --- | | `model_name` | `[OpenAIModelName](#pydantic_ai.models.openai.OpenAIModelName "pydantic_ai.models.openai.OpenAIModelName") ` | The name of the OpenAI model to use. List of model names available [here](https://github.com/openai/openai-python/blob/v1.54.3/src/openai/types/chat_model.py#L7)
(Unfortunately, despite being ask to do so, OpenAI do not provide `.inv` files for their API). | _required_ | | `provider` | `[Literal](https://docs.python.org/3/library/typing.html#typing.Literal "typing.Literal") ['openai', 'deepseek', 'azure'] \| [Provider](../../providers/#pydantic_ai.providers.Provider "pydantic_ai.providers.Provider") [AsyncOpenAI]` | The provider to use. Defaults to `'openai'`. | `'openai'` | | `system_prompt_role` | `OpenAISystemPromptRole \| None` | The role to use for the system prompt message. If not provided, defaults to `'system'`. In the future, this may be inferred from the model name. | `None` | Source code in `pydantic_ai_slim/pydantic_ai/models/openai.py` | | | | --- | --- | | 158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179 | ``def __init__( self, model_name: OpenAIModelName, *, provider: Literal['openai', 'deepseek', 'azure'] \| Provider[AsyncOpenAI] = 'openai', system_prompt_role: OpenAISystemPromptRole \| None = None, ): """Initialize an OpenAI model. Args: model_name: The name of the OpenAI model to use. List of model names available [here](https://github.com/openai/openai-python/blob/v1.54.3/src/openai/types/chat_model.py#L7) (Unfortunately, despite being ask to do so, OpenAI do not provide `.inv` files for their API). provider: The provider to use. Defaults to `'openai'`. system_prompt_role: The role to use for the system prompt message. If not provided, defaults to `'system'`. In the future, this may be inferred from the model name. """ self._model_name = model_name if isinstance(provider, str): provider = infer_provider(provider) self.client = provider.client self.system_prompt_role = system_prompt_role`` | #### model\_name `property` `model_name: [OpenAIModelName](#pydantic_ai.models.openai.OpenAIModelName "pydantic_ai.models.openai.OpenAIModelName")` The model name. #### system `property` `system: [str](https://docs.python.org/3/library/stdtypes.html#str)` The system / model provider. ### OpenAIResponsesModel `dataclass` Bases: `[Model](../base/#pydantic_ai.models.Model "pydantic_ai.models.Model") ` A model that uses the OpenAI Responses API. The [OpenAI Responses API](https://platform.openai.com/docs/api-reference/responses) is the new API for OpenAI models. The Responses API has built-in tools, that you can use instead of building your own: * [Web search](https://platform.openai.com/docs/guides/tools-web-search) * [File search](https://platform.openai.com/docs/guides/tools-file-search) * [Computer use](https://platform.openai.com/docs/guides/tools-computer-use) Use the `openai_builtin_tools` setting to add these tools to your model. If you are interested in the differences between the Responses API and the Chat Completions API, see the [OpenAI API docs](https://platform.openai.com/docs/guides/responses-vs-chat-completions) . Source code in `pydantic_ai_slim/pydantic_ai/models/openai.py` | | | | --- | --- | | 447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757 | ``@dataclass(init=False) class OpenAIResponsesModel(Model): """A model that uses the OpenAI Responses API. The [OpenAI Responses API](https://platform.openai.com/docs/api-reference/responses) is the new API for OpenAI models. The Responses API has built-in tools, that you can use instead of building your own: - [Web search](https://platform.openai.com/docs/guides/tools-web-search) - [File search](https://platform.openai.com/docs/guides/tools-file-search) - [Computer use](https://platform.openai.com/docs/guides/tools-computer-use) Use the `openai_builtin_tools` setting to add these tools to your model. If you are interested in the differences between the Responses API and the Chat Completions API, see the [OpenAI API docs](https://platform.openai.com/docs/guides/responses-vs-chat-completions). """ client: AsyncOpenAI = field(repr=False) system_prompt_role: OpenAISystemPromptRole \| None = field(default=None) _model_name: OpenAIModelName = field(repr=False) _system: str = field(default='openai', repr=False) def __init__( self, model_name: OpenAIModelName, *, provider: Literal['openai', 'deepseek', 'azure'] \| Provider[AsyncOpenAI] = 'openai', ): """Initialize an OpenAI Responses model. Args: model_name: The name of the OpenAI model to use. provider: The provider to use. Defaults to `'openai'`. """ self._model_name = model_name if isinstance(provider, str): provider = infer_provider(provider) self.client = provider.client @property def model_name(self) -> OpenAIModelName: """The model name.""" return self._model_name @property def system(self) -> str: """The system / model provider.""" return self._system async def request( self, messages: list[ModelRequest \| ModelResponse], model_settings: ModelSettings \| None, model_request_parameters: ModelRequestParameters, ) -> tuple[ModelResponse, usage.Usage]: check_allow_model_requests() response = await self._responses_create( messages, False, cast(OpenAIResponsesModelSettings, model_settings or {}), model_request_parameters ) return self._process_response(response), _map_usage(response) @asynccontextmanager async def request_stream( self, messages: list[ModelMessage], model_settings: ModelSettings \| None, model_request_parameters: ModelRequestParameters, ) -> AsyncIterator[StreamedResponse]: check_allow_model_requests() response = await self._responses_create( messages, True, cast(OpenAIResponsesModelSettings, model_settings or {}), model_request_parameters ) async with response: yield await self._process_streamed_response(response) def _process_response(self, response: responses.Response) -> ModelResponse: """Process a non-streamed response, and prepare a message to return.""" timestamp = datetime.fromtimestamp(response.created_at, tz=timezone.utc) items: list[ModelResponsePart] = [] items.append(TextPart(response.output_text)) for item in response.output: if item.type == 'function_call': items.append(ToolCallPart(item.name, item.arguments, tool_call_id=item.call_id)) return ModelResponse(items, model_name=response.model, timestamp=timestamp) async def _process_streamed_response( self, response: AsyncStream[responses.ResponseStreamEvent] ) -> OpenAIResponsesStreamedResponse: """Process a streamed response, and prepare a streaming response to return.""" peekable_response = _utils.PeekableAsyncStream(response) first_chunk = await peekable_response.peek() if isinstance(first_chunk, _utils.Unset): # pragma: no cover raise UnexpectedModelBehavior('Streamed response ended without content or tool calls') assert isinstance(first_chunk, responses.ResponseCreatedEvent) return OpenAIResponsesStreamedResponse( _model_name=self._model_name, _response=peekable_response, _timestamp=datetime.fromtimestamp(first_chunk.response.created_at, tz=timezone.utc), ) @overload async def _responses_create( self, messages: list[ModelRequest \| ModelResponse], stream: Literal[False], model_settings: OpenAIResponsesModelSettings, model_request_parameters: ModelRequestParameters, ) -> responses.Response: ... @overload async def _responses_create( self, messages: list[ModelRequest \| ModelResponse], stream: Literal[True], model_settings: OpenAIResponsesModelSettings, model_request_parameters: ModelRequestParameters, ) -> AsyncStream[responses.ResponseStreamEvent]: ... async def _responses_create( self, messages: list[ModelRequest \| ModelResponse], stream: bool, model_settings: OpenAIResponsesModelSettings, model_request_parameters: ModelRequestParameters, ) -> responses.Response \| AsyncStream[responses.ResponseStreamEvent]: tools = self._get_tools(model_request_parameters) tools = list(model_settings.get('openai_builtin_tools', [])) + tools # standalone function to make it easier to override if not tools: tool_choice: Literal['none', 'required', 'auto'] \| None = None elif not model_request_parameters.allow_text_result: tool_choice = 'required' else: tool_choice = 'auto' system_prompt, openai_messages = await self._map_message(messages) reasoning = self._get_reasoning(model_settings) try: return await self.client.responses.create( input=openai_messages, model=self._model_name, instructions=system_prompt, parallel_tool_calls=model_settings.get('parallel_tool_calls', NOT_GIVEN), tools=tools or NOT_GIVEN, tool_choice=tool_choice or NOT_GIVEN, max_output_tokens=model_settings.get('max_tokens', NOT_GIVEN), stream=stream, temperature=model_settings.get('temperature', NOT_GIVEN), top_p=model_settings.get('top_p', NOT_GIVEN), truncation=model_settings.get('openai_truncation', NOT_GIVEN), timeout=model_settings.get('timeout', NOT_GIVEN), reasoning=reasoning, user=model_settings.get('user', NOT_GIVEN), ) except APIStatusError as e: if (status_code := e.status_code) >= 400: raise ModelHTTPError(status_code=status_code, model_name=self.model_name, body=e.body) from e raise def _get_reasoning(self, model_settings: OpenAIResponsesModelSettings) -> Reasoning \| NotGiven: reasoning_effort = model_settings.get('openai_reasoning_effort', None) reasoning_generate_summary = model_settings.get('openai_reasoning_generate_summary', None) if reasoning_effort is None and reasoning_generate_summary is None: return NOT_GIVEN return Reasoning(effort=reasoning_effort, generate_summary=reasoning_generate_summary) def _get_tools(self, model_request_parameters: ModelRequestParameters) -> list[responses.FunctionToolParam]: tools = [self._map_tool_definition(r) for r in model_request_parameters.function_tools] if model_request_parameters.result_tools: tools += [self._map_tool_definition(r) for r in model_request_parameters.result_tools] return tools @staticmethod def _map_tool_definition(f: ToolDefinition) -> responses.FunctionToolParam: return { 'name': f.name, 'parameters': f.parameters_json_schema, 'type': 'function', 'description': f.description, # TODO(Marcelo): We should make this configurable, and if True, set `additionalProperties` to False. 'strict': False, } async def _map_message(self, messages: list[ModelMessage]) -> tuple[str, list[responses.ResponseInputItemParam]]: """Just maps a `pydantic_ai.Message` to a `openai.types.responses.ResponseInputParam`.""" system_prompt: str = '' openai_messages: list[responses.ResponseInputItemParam] = [] for message in messages: if isinstance(message, ModelRequest): for part in message.parts: if isinstance(part, SystemPromptPart): system_prompt += part.content elif isinstance(part, UserPromptPart): openai_messages.append(await self._map_user_prompt(part)) elif isinstance(part, ToolReturnPart): openai_messages.append( FunctionCallOutput( type='function_call_output', call_id=_guard_tool_call_id(t=part), output=part.model_response_str(), ) ) elif isinstance(part, RetryPromptPart): # TODO(Marcelo): How do we test this conditional branch? if part.tool_name is None: # pragma: no cover openai_messages.append( Message(role='user', content=[{'type': 'input_text', 'text': part.model_response()}]) ) else: openai_messages.append( FunctionCallOutput( type='function_call_output', call_id=_guard_tool_call_id(t=part), output=part.model_response(), ) ) else: assert_never(part) elif isinstance(message, ModelResponse): for item in message.parts: if isinstance(item, TextPart): openai_messages.append(responses.EasyInputMessageParam(role='assistant', content=item.content)) elif isinstance(item, ToolCallPart): openai_messages.append(self._map_tool_call(item)) else: assert_never(item) else: assert_never(message) return system_prompt, openai_messages @staticmethod def _map_tool_call(t: ToolCallPart) -> responses.ResponseFunctionToolCallParam: return responses.ResponseFunctionToolCallParam( arguments=t.args_as_json_str(), call_id=_guard_tool_call_id(t=t), name=t.tool_name, type='function_call', ) @staticmethod async def _map_user_prompt(part: UserPromptPart) -> responses.EasyInputMessageParam: content: str \| list[responses.ResponseInputContentParam] if isinstance(part.content, str): content = part.content else: content = [] for item in part.content: if isinstance(item, str): content.append(responses.ResponseInputTextParam(text=item, type='input_text')) elif isinstance(item, BinaryContent): base64_encoded = base64.b64encode(item.data).decode('utf-8') if item.is_image: content.append( responses.ResponseInputImageParam( image_url=f'data:{item.media_type};base64,{base64_encoded}', type='input_image', detail='auto', ) ) elif item.is_document: content.append( responses.ResponseInputFileParam( type='input_file', file_data=f'data:{item.media_type};base64,{base64_encoded}', # NOTE: Type wise it's not necessary to include the filename, but it's required by the # API itself. If we add empty string, the server sends a 500 error - which OpenAI needs # to fix. In any case, we add a placeholder name. filename=f'filename.{item.format}', ) ) elif item.is_audio: raise NotImplementedError('Audio as binary content is not supported for OpenAI Responses API.') else: # pragma: no cover raise RuntimeError(f'Unsupported binary content type: {item.media_type}') elif isinstance(item, ImageUrl): content.append( responses.ResponseInputImageParam(image_url=item.url, type='input_image', detail='auto') ) elif isinstance(item, AudioUrl): # pragma: no cover client = cached_async_http_client() response = await client.get(item.url) response.raise_for_status() base64_encoded = base64.b64encode(response.content).decode('utf-8') content.append( responses.ResponseInputFileParam( type='input_file', file_data=f'data:{item.media_type};base64,{base64_encoded}', ) ) elif isinstance(item, DocumentUrl): # pragma: no cover client = cached_async_http_client() response = await client.get(item.url) response.raise_for_status() base64_encoded = base64.b64encode(response.content).decode('utf-8') content.append( responses.ResponseInputFileParam( type='input_file', file_data=f'data:{item.media_type};base64,{base64_encoded}', filename=f'filename.{item.format}', ) ) else: assert_never(item) return responses.EasyInputMessageParam(role='user', content=content)`` | #### \_\_init\_\_ `__init__( model_name: [OpenAIModelName](#pydantic_ai.models.openai.OpenAIModelName "pydantic_ai.models.openai.OpenAIModelName") , *, provider: ( [Literal](https://docs.python.org/3/library/typing.html#typing.Literal "typing.Literal") ["openai", "deepseek", "azure"] | [Provider](../../providers/#pydantic_ai.providers.Provider "pydantic_ai.providers.Provider") [AsyncOpenAI] ) = "openai" )` Initialize an OpenAI Responses model. Parameters: | Name | Type | Description | Default | | --- | --- | --- | --- | | `model_name` | `[OpenAIModelName](#pydantic_ai.models.openai.OpenAIModelName "pydantic_ai.models.openai.OpenAIModelName") ` | The name of the OpenAI model to use. | _required_ | | `provider` | `[Literal](https://docs.python.org/3/library/typing.html#typing.Literal "typing.Literal") ['openai', 'deepseek', 'azure'] \| [Provider](../../providers/#pydantic_ai.providers.Provider "pydantic_ai.providers.Provider") [AsyncOpenAI]` | The provider to use. Defaults to `'openai'`. | `'openai'` | Source code in `pydantic_ai_slim/pydantic_ai/models/openai.py` | | | | --- | --- | | 472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487 | ``def __init__( self, model_name: OpenAIModelName, *, provider: Literal['openai', 'deepseek', 'azure'] \| Provider[AsyncOpenAI] = 'openai', ): """Initialize an OpenAI Responses model. Args: model_name: The name of the OpenAI model to use. provider: The provider to use. Defaults to `'openai'`. """ self._model_name = model_name if isinstance(provider, str): provider = infer_provider(provider) self.client = provider.client`` | #### model\_name `property` `model_name: [OpenAIModelName](#pydantic_ai.models.openai.OpenAIModelName "pydantic_ai.models.openai.OpenAIModelName")` The model name. #### system `property` `system: [str](https://docs.python.org/3/library/stdtypes.html#str)` The system / model provider. --- # pydantic_ai.models.anthropic - PydanticAI `pydantic_ai.models.anthropic` ============================== Setup ----- For details on how to set up authentication with this model, see [model configuration for Anthropic](../../../models/#anthropic) . ### LatestAnthropicModelNames `module-attribute` `LatestAnthropicModelNames = [Literal](https://docs.python.org/3/library/typing.html#typing.Literal "typing.Literal") [ "claude-3-7-sonnet-latest", "claude-3-5-haiku-latest", "claude-3-5-sonnet-latest", "claude-3-opus-latest", ]` Latest Anthropic models. ### AnthropicModelName `module-attribute` `AnthropicModelName = [Union](https://docs.python.org/3/library/typing.html#typing.Union "typing.Union") [[str](https://docs.python.org/3/library/stdtypes.html#str) , [LatestAnthropicModelNames](#pydantic_ai.models.anthropic.LatestAnthropicModelNames "pydantic_ai.models.anthropic.LatestAnthropicModelNames") ]` Possible Anthropic model names. Since Anthropic supports a variety of date-stamped models, we explicitly list the latest models but allow any name in the type hints. See [the Anthropic docs](https://docs.anthropic.com/en/docs/about-claude/models) for a full list. ### AnthropicModelSettings Bases: `[ModelSettings](../../settings/#pydantic_ai.settings.ModelSettings "pydantic_ai.settings.ModelSettings") ` Settings used for an Anthropic model request. ALL FIELDS MUST BE `anthropic_` PREFIXED SO YOU CAN MERGE THEM WITH OTHER MODELS. Source code in `pydantic_ai_slim/pydantic_ai/models/anthropic.py` | | | | --- | --- | | 86
87
88
89
90
91
92
93
94
95 | ``class AnthropicModelSettings(ModelSettings): """Settings used for an Anthropic model request. ALL FIELDS MUST BE `anthropic_` PREFIXED SO YOU CAN MERGE THEM WITH OTHER MODELS. """ anthropic_metadata: MetadataParam """An object describing metadata about the request. Contains `user_id`, an external identifier for the user who is associated with the request."""`` | #### anthropic\_metadata `instance-attribute` `anthropic_metadata: MetadataParam` An object describing metadata about the request. Contains `user_id`, an external identifier for the user who is associated with the request. ### AnthropicModel `dataclass` Bases: `[Model](../base/#pydantic_ai.models.Model "pydantic_ai.models.Model") ` A model that uses the Anthropic API. Internally, this uses the [Anthropic Python client](https://github.com/anthropics/anthropic-sdk-python) to interact with the API. Apart from `__init__`, all methods are private or match those of the base class. Note The `AnthropicModel` class does not yet support streaming responses. We anticipate adding support for streaming responses in a near-term future release. Source code in `pydantic_ai_slim/pydantic_ai/models/anthropic.py` | | | | --- | --- | | 98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378 | ``@dataclass(init=False) class AnthropicModel(Model): """A model that uses the Anthropic API. Internally, this uses the [Anthropic Python client](https://github.com/anthropics/anthropic-sdk-python) to interact with the API. Apart from `__init__`, all methods are private or match those of the base class. !!! note The `AnthropicModel` class does not yet support streaming responses. We anticipate adding support for streaming responses in a near-term future release. """ client: AsyncAnthropic = field(repr=False) _model_name: AnthropicModelName = field(repr=False) _system: str = field(default='anthropic', repr=False) def __init__( self, model_name: AnthropicModelName, *, provider: Literal['anthropic'] \| Provider[AsyncAnthropic] = 'anthropic', ): """Initialize an Anthropic model. Args: model_name: The name of the Anthropic model to use. List of model names available [here](https://docs.anthropic.com/en/docs/about-claude/models). provider: The provider to use for the Anthropic API. Can be either the string 'anthropic' or an instance of `Provider[AsyncAnthropic]`. If not provided, the other parameters will be used. """ self._model_name = model_name if isinstance(provider, str): provider = infer_provider(provider) self.client = provider.client @property def base_url(self) -> str: return str(self.client.base_url) async def request( self, messages: list[ModelMessage], model_settings: ModelSettings \| None, model_request_parameters: ModelRequestParameters, ) -> tuple[ModelResponse, usage.Usage]: check_allow_model_requests() response = await self._messages_create( messages, False, cast(AnthropicModelSettings, model_settings or {}), model_request_parameters ) return self._process_response(response), _map_usage(response) @asynccontextmanager async def request_stream( self, messages: list[ModelMessage], model_settings: ModelSettings \| None, model_request_parameters: ModelRequestParameters, ) -> AsyncIterator[StreamedResponse]: check_allow_model_requests() response = await self._messages_create( messages, True, cast(AnthropicModelSettings, model_settings or {}), model_request_parameters ) async with response: yield await self._process_streamed_response(response) @property def model_name(self) -> AnthropicModelName: """The model name.""" return self._model_name @property def system(self) -> str: """The system / model provider.""" return self._system @overload async def _messages_create( self, messages: list[ModelMessage], stream: Literal[True], model_settings: AnthropicModelSettings, model_request_parameters: ModelRequestParameters, ) -> AsyncStream[RawMessageStreamEvent]: pass @overload async def _messages_create( self, messages: list[ModelMessage], stream: Literal[False], model_settings: AnthropicModelSettings, model_request_parameters: ModelRequestParameters, ) -> AnthropicMessage: pass async def _messages_create( self, messages: list[ModelMessage], stream: bool, model_settings: AnthropicModelSettings, model_request_parameters: ModelRequestParameters, ) -> AnthropicMessage \| AsyncStream[RawMessageStreamEvent]: # standalone function to make it easier to override tools = self._get_tools(model_request_parameters) tool_choice: ToolChoiceParam \| None if not tools: tool_choice = None else: if not model_request_parameters.allow_text_result: tool_choice = {'type': 'any'} else: tool_choice = {'type': 'auto'} if (allow_parallel_tool_calls := model_settings.get('parallel_tool_calls')) is not None: tool_choice['disable_parallel_tool_use'] = not allow_parallel_tool_calls system_prompt, anthropic_messages = await self._map_message(messages) try: return await self.client.messages.create( max_tokens=model_settings.get('max_tokens', 1024), system=system_prompt or NOT_GIVEN, messages=anthropic_messages, model=self._model_name, tools=tools or NOT_GIVEN, tool_choice=tool_choice or NOT_GIVEN, stream=stream, temperature=model_settings.get('temperature', NOT_GIVEN), top_p=model_settings.get('top_p', NOT_GIVEN), timeout=model_settings.get('timeout', NOT_GIVEN), metadata=model_settings.get('anthropic_metadata', NOT_GIVEN), ) except APIStatusError as e: if (status_code := e.status_code) >= 400: raise ModelHTTPError(status_code=status_code, model_name=self.model_name, body=e.body) from e raise def _process_response(self, response: AnthropicMessage) -> ModelResponse: """Process a non-streamed response, and prepare a message to return.""" items: list[ModelResponsePart] = [] for item in response.content: if isinstance(item, TextBlock): items.append(TextPart(content=item.text)) else: assert isinstance(item, ToolUseBlock), 'unexpected item type' items.append( ToolCallPart( tool_name=item.name, args=cast(dict[str, Any], item.input), tool_call_id=item.id, ) ) return ModelResponse(items, model_name=response.model) async def _process_streamed_response(self, response: AsyncStream[RawMessageStreamEvent]) -> StreamedResponse: peekable_response = _utils.PeekableAsyncStream(response) first_chunk = await peekable_response.peek() if isinstance(first_chunk, _utils.Unset): raise UnexpectedModelBehavior('Streamed response ended without content or tool calls') # Since Anthropic doesn't provide a timestamp in the message, we'll use the current time timestamp = datetime.now(tz=timezone.utc) return AnthropicStreamedResponse( _model_name=self._model_name, _response=peekable_response, _timestamp=timestamp ) def _get_tools(self, model_request_parameters: ModelRequestParameters) -> list[ToolParam]: tools = [self._map_tool_definition(r) for r in model_request_parameters.function_tools] if model_request_parameters.result_tools: tools += [self._map_tool_definition(r) for r in model_request_parameters.result_tools] return tools async def _map_message(self, messages: list[ModelMessage]) -> tuple[str, list[MessageParam]]: """Just maps a `pydantic_ai.Message` to a `anthropic.types.MessageParam`.""" system_prompt: str = '' anthropic_messages: list[MessageParam] = [] for m in messages: if isinstance(m, ModelRequest): user_content_params: list[ ToolResultBlockParam \| TextBlockParam \| ImageBlockParam \| DocumentBlockParam ] = [] for request_part in m.parts: if isinstance(request_part, SystemPromptPart): system_prompt += request_part.content elif isinstance(request_part, UserPromptPart): async for content in self._map_user_prompt(request_part): user_content_params.append(content) elif isinstance(request_part, ToolReturnPart): tool_result_block_param = ToolResultBlockParam( tool_use_id=_guard_tool_call_id(t=request_part), type='tool_result', content=request_part.model_response_str(), is_error=False, ) user_content_params.append(tool_result_block_param) elif isinstance(request_part, RetryPromptPart): if request_part.tool_name is None: retry_param = TextBlockParam(type='text', text=request_part.model_response()) else: retry_param = ToolResultBlockParam( tool_use_id=_guard_tool_call_id(t=request_part), type='tool_result', content=request_part.model_response(), is_error=True, ) user_content_params.append(retry_param) anthropic_messages.append(MessageParam(role='user', content=user_content_params)) elif isinstance(m, ModelResponse): assistant_content_params: list[TextBlockParam \| ToolUseBlockParam] = [] for response_part in m.parts: if isinstance(response_part, TextPart): assistant_content_params.append(TextBlockParam(text=response_part.content, type='text')) else: tool_use_block_param = ToolUseBlockParam( id=_guard_tool_call_id(t=response_part), type='tool_use', name=response_part.tool_name, input=response_part.args_as_dict(), ) assistant_content_params.append(tool_use_block_param) anthropic_messages.append(MessageParam(role='assistant', content=assistant_content_params)) else: assert_never(m) return system_prompt, anthropic_messages @staticmethod async def _map_user_prompt( part: UserPromptPart, ) -> AsyncGenerator[ImageBlockParam \| TextBlockParam \| DocumentBlockParam]: if isinstance(part.content, str): yield TextBlockParam(text=part.content, type='text') else: for item in part.content: if isinstance(item, str): yield TextBlockParam(text=item, type='text') elif isinstance(item, BinaryContent): if item.is_image: yield ImageBlockParam( source={'data': io.BytesIO(item.data), 'media_type': item.media_type, 'type': 'base64'}, # type: ignore type='image', ) elif item.media_type == 'application/pdf': yield DocumentBlockParam( source=Base64PDFSourceParam( data=io.BytesIO(item.data), media_type='application/pdf', type='base64', ), type='document', ) else: raise RuntimeError('Only images and PDFs are supported for binary content') elif isinstance(item, ImageUrl): yield ImageBlockParam(source={'type': 'url', 'url': item.url}, type='image') elif isinstance(item, DocumentUrl): if item.media_type == 'application/pdf': yield DocumentBlockParam(source={'url': item.url, 'type': 'url'}, type='document') elif item.media_type == 'text/plain': response = await cached_async_http_client().get(item.url) response.raise_for_status() yield DocumentBlockParam( source=PlainTextSourceParam(data=response.text, media_type=item.media_type, type='text'), type='document', ) else: # pragma: no cover raise RuntimeError(f'Unsupported media type: {item.media_type}') else: raise RuntimeError(f'Unsupported content type: {type(item)}') @staticmethod def _map_tool_definition(f: ToolDefinition) -> ToolParam: return { 'name': f.name, 'description': f.description, 'input_schema': f.parameters_json_schema, }`` | #### \_\_init\_\_ `__init__( model_name: [AnthropicModelName](#pydantic_ai.models.anthropic.AnthropicModelName "pydantic_ai.models.anthropic.AnthropicModelName") , *, provider: ( [Literal](https://docs.python.org/3/library/typing.html#typing.Literal "typing.Literal") ["anthropic"] | [Provider](../../providers/#pydantic_ai.providers.Provider "pydantic_ai.providers.Provider") [AsyncAnthropic] ) = "anthropic" )` Initialize an Anthropic model. Parameters: | Name | Type | Description | Default | | --- | --- | --- | --- | | `model_name` | `[AnthropicModelName](#pydantic_ai.models.anthropic.AnthropicModelName "pydantic_ai.models.anthropic.AnthropicModelName") ` | The name of the Anthropic model to use. List of model names available [here](https://docs.anthropic.com/en/docs/about-claude/models)
. | _required_ | | `provider` | `[Literal](https://docs.python.org/3/library/typing.html#typing.Literal "typing.Literal") ['anthropic'] \| [Provider](../../providers/#pydantic_ai.providers.Provider "pydantic_ai.providers.Provider") [AsyncAnthropic]` | The provider to use for the Anthropic API. Can be either the string 'anthropic' or an instance of `Provider[AsyncAnthropic]`. If not provided, the other parameters will be used. | `'anthropic'` | Source code in `pydantic_ai_slim/pydantic_ai/models/anthropic.py` | | | | --- | --- | | 116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134 | ``def __init__( self, model_name: AnthropicModelName, *, provider: Literal['anthropic'] \| Provider[AsyncAnthropic] = 'anthropic', ): """Initialize an Anthropic model. Args: model_name: The name of the Anthropic model to use. List of model names available [here](https://docs.anthropic.com/en/docs/about-claude/models). provider: The provider to use for the Anthropic API. Can be either the string 'anthropic' or an instance of `Provider[AsyncAnthropic]`. If not provided, the other parameters will be used. """ self._model_name = model_name if isinstance(provider, str): provider = infer_provider(provider) self.client = provider.client`` | #### model\_name `property` `model_name: [AnthropicModelName](#pydantic_ai.models.anthropic.AnthropicModelName "pydantic_ai.models.anthropic.AnthropicModelName")` The model name. #### system `property` `system: [str](https://docs.python.org/3/library/stdtypes.html#str)` The system / model provider. ### AnthropicStreamedResponse `dataclass` Bases: `[StreamedResponse](../base/#pydantic_ai.models.StreamedResponse "pydantic_ai.models.StreamedResponse") ` Implementation of `StreamedResponse` for Anthropic models. Source code in `pydantic_ai_slim/pydantic_ai/models/anthropic.py` | | | | --- | --- | | 410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475 | ``@dataclass class AnthropicStreamedResponse(StreamedResponse): """Implementation of `StreamedResponse` for Anthropic models.""" _model_name: AnthropicModelName _response: AsyncIterable[RawMessageStreamEvent] _timestamp: datetime async def _get_event_iterator(self) -> AsyncIterator[ModelResponseStreamEvent]: current_block: ContentBlock \| None = None current_json: str = '' async for event in self._response: self._usage += _map_usage(event) if isinstance(event, RawContentBlockStartEvent): current_block = event.content_block if isinstance(current_block, TextBlock) and current_block.text: yield self._parts_manager.handle_text_delta(vendor_part_id='content', content=current_block.text) elif isinstance(current_block, ToolUseBlock): maybe_event = self._parts_manager.handle_tool_call_delta( vendor_part_id=current_block.id, tool_name=current_block.name, args=cast(dict[str, Any], current_block.input), tool_call_id=current_block.id, ) if maybe_event is not None: yield maybe_event elif isinstance(event, RawContentBlockDeltaEvent): if isinstance(event.delta, TextDelta): yield self._parts_manager.handle_text_delta(vendor_part_id='content', content=event.delta.text) elif ( current_block and event.delta.type == 'input_json_delta' and isinstance(current_block, ToolUseBlock) ): # Try to parse the JSON immediately, otherwise cache the value for later. This handles # cases where the JSON is not currently valid but will be valid once we stream more tokens. try: parsed_args = json_loads(current_json + event.delta.partial_json) current_json = '' except JSONDecodeError: current_json += event.delta.partial_json continue # For tool calls, we need to handle partial JSON updates maybe_event = self._parts_manager.handle_tool_call_delta( vendor_part_id=current_block.id, tool_name='', args=parsed_args, tool_call_id=current_block.id, ) if maybe_event is not None: yield maybe_event elif isinstance(event, (RawContentBlockStopEvent, RawMessageStopEvent)): current_block = None @property def model_name(self) -> AnthropicModelName: """Get the model name of the response.""" return self._model_name @property def timestamp(self) -> datetime: """Get the timestamp of the response.""" return self._timestamp`` | #### model\_name `property` `model_name: [AnthropicModelName](#pydantic_ai.models.anthropic.AnthropicModelName "pydantic_ai.models.anthropic.AnthropicModelName")` Get the model name of the response. #### timestamp `property` `timestamp: [datetime](https://docs.python.org/3/library/datetime.html#datetime.datetime "datetime.datetime")` Get the timestamp of the response. --- # pydantic_ai.models.bedrock - PydanticAI `pydantic_ai.models.bedrock` ============================ Setup ----- For details on how to set up authentication with this model, see [model configuration for Bedrock](../../../models/#bedrock) . ### LatestBedrockModelNames `module-attribute` `LatestBedrockModelNames = [Literal](https://docs.python.org/3/library/typing.html#typing.Literal "typing.Literal") [ "amazon.titan-tg1-large", "amazon.titan-text-lite-v1", "amazon.titan-text-express-v1", "us.amazon.nova-pro-v1:0", "us.amazon.nova-lite-v1:0", "us.amazon.nova-micro-v1:0", "anthropic.claude-3-5-sonnet-20241022-v2:0", "us.anthropic.claude-3-5-sonnet-20241022-v2:0", "anthropic.claude-3-5-haiku-20241022-v1:0", "us.anthropic.claude-3-5-haiku-20241022-v1:0", "anthropic.claude-instant-v1", "anthropic.claude-v2:1", "anthropic.claude-v2", "anthropic.claude-3-sonnet-20240229-v1:0", "us.anthropic.claude-3-sonnet-20240229-v1:0", "anthropic.claude-3-haiku-20240307-v1:0", "us.anthropic.claude-3-haiku-20240307-v1:0", "anthropic.claude-3-opus-20240229-v1:0", "us.anthropic.claude-3-opus-20240229-v1:0", "anthropic.claude-3-5-sonnet-20240620-v1:0", "us.anthropic.claude-3-5-sonnet-20240620-v1:0", "anthropic.claude-3-7-sonnet-20250219-v1:0", "us.anthropic.claude-3-7-sonnet-20250219-v1:0", "cohere.command-text-v14", "cohere.command-r-v1:0", "cohere.command-r-plus-v1:0", "cohere.command-light-text-v14", "meta.llama3-8b-instruct-v1:0", "meta.llama3-70b-instruct-v1:0", "meta.llama3-1-8b-instruct-v1:0", "us.meta.llama3-1-8b-instruct-v1:0", "meta.llama3-1-70b-instruct-v1:0", "us.meta.llama3-1-70b-instruct-v1:0", "meta.llama3-1-405b-instruct-v1:0", "us.meta.llama3-2-11b-instruct-v1:0", "us.meta.llama3-2-90b-instruct-v1:0", "us.meta.llama3-2-1b-instruct-v1:0", "us.meta.llama3-2-3b-instruct-v1:0", "us.meta.llama3-3-70b-instruct-v1:0", "mistral.mistral-7b-instruct-v0:2", "mistral.mixtral-8x7b-instruct-v0:1", "mistral.mistral-large-2402-v1:0", "mistral.mistral-large-2407-v1:0", ]` Latest Bedrock models. ### BedrockModelName `module-attribute` `BedrockModelName = [Union](https://docs.python.org/3/library/typing.html#typing.Union "typing.Union") [[str](https://docs.python.org/3/library/stdtypes.html#str) , [LatestBedrockModelNames](#pydantic_ai.models.bedrock.LatestBedrockModelNames "pydantic_ai.models.bedrock.LatestBedrockModelNames") ]` Possible Bedrock model names. Since Bedrock supports a variety of date-stamped models, we explicitly list the latest models but allow any name in the type hints. See [the Bedrock docs](https://docs.aws.amazon.com/bedrock/latest/userguide/models-supported.html) for a full list. ### BedrockModelSettings Bases: `[ModelSettings](../../settings/#pydantic_ai.settings.ModelSettings "pydantic_ai.settings.ModelSettings") ` Settings for Bedrock models. ALL FIELDS MUST BE `bedrock_` PREFIXED SO YOU CAN MERGE THEM WITH OTHER MODELS. Source code in `pydantic_ai_slim/pydantic_ai/models/bedrock.py` | | | | --- | --- | | 115
116
117
118
119 | ``class BedrockModelSettings(ModelSettings): """Settings for Bedrock models. ALL FIELDS MUST BE `bedrock_` PREFIXED SO YOU CAN MERGE THEM WITH OTHER MODELS. """`` | ### BedrockConverseModel `dataclass` Bases: `[Model](../base/#pydantic_ai.models.Model "pydantic_ai.models.Model") ` A model that uses the Bedrock Converse API. Source code in `pydantic_ai_slim/pydantic_ai/models/bedrock.py` | | | | --- | --- | | 122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407 | ``@dataclass(init=False) class BedrockConverseModel(Model): """A model that uses the Bedrock Converse API.""" client: BedrockRuntimeClient _model_name: BedrockModelName = field(repr=False) _system: str = field(default='bedrock', repr=False) @property def model_name(self) -> str: """The model name.""" return self._model_name @property def system(self) -> str: """The system / model provider, ex: openai.""" return self._system def __init__( self, model_name: BedrockModelName, *, provider: Literal['bedrock'] \| Provider[BaseClient] = 'bedrock', ): """Initialize a Bedrock model. Args: model_name: The name of the model to use. model_name: The name of the Bedrock model to use. List of model names available [here](https://docs.aws.amazon.com/bedrock/latest/userguide/models-supported.html). provider: The provider to use for authentication and API access. Can be either the string 'bedrock' or an instance of `Provider[BaseClient]`. If not provided, a new provider will be created using the other parameters. """ self._model_name = model_name if isinstance(provider, str): provider = infer_provider(provider) self.client = cast('BedrockRuntimeClient', provider.client) def _get_tools(self, model_request_parameters: ModelRequestParameters) -> list[ToolTypeDef]: tools = [self._map_tool_definition(r) for r in model_request_parameters.function_tools] if model_request_parameters.result_tools: tools += [self._map_tool_definition(r) for r in model_request_parameters.result_tools] return tools @staticmethod def _map_tool_definition(f: ToolDefinition) -> ToolTypeDef: return { 'toolSpec': { 'name': f.name, 'description': f.description, 'inputSchema': {'json': f.parameters_json_schema}, } } @property def base_url(self) -> str: return str(self.client.meta.endpoint_url) async def request( self, messages: list[ModelMessage], model_settings: ModelSettings \| None, model_request_parameters: ModelRequestParameters, ) -> tuple[ModelResponse, result.Usage]: response = await self._messages_create(messages, False, model_settings, model_request_parameters) return await self._process_response(response) @asynccontextmanager async def request_stream( self, messages: list[ModelMessage], model_settings: ModelSettings \| None, model_request_parameters: ModelRequestParameters, ) -> AsyncIterator[StreamedResponse]: response = await self._messages_create(messages, True, model_settings, model_request_parameters) yield BedrockStreamedResponse(_model_name=self.model_name, _event_stream=response) async def _process_response(self, response: ConverseResponseTypeDef) -> tuple[ModelResponse, result.Usage]: items: list[ModelResponsePart] = [] if message := response['output'].get('message'): for item in message['content']: if text := item.get('text'): items.append(TextPart(content=text)) else: tool_use = item.get('toolUse') assert tool_use is not None, f'Found a content that is not a text or tool use: {item}' items.append( ToolCallPart( tool_name=tool_use['name'], args=tool_use['input'], tool_call_id=tool_use['toolUseId'], ), ) usage = result.Usage( request_tokens=response['usage']['inputTokens'], response_tokens=response['usage']['outputTokens'], total_tokens=response['usage']['totalTokens'], ) return ModelResponse(items, model_name=self.model_name), usage @overload async def _messages_create( self, messages: list[ModelMessage], stream: Literal[True], model_settings: ModelSettings \| None, model_request_parameters: ModelRequestParameters, ) -> EventStream[ConverseStreamOutputTypeDef]: pass @overload async def _messages_create( self, messages: list[ModelMessage], stream: Literal[False], model_settings: ModelSettings \| None, model_request_parameters: ModelRequestParameters, ) -> ConverseResponseTypeDef: pass async def _messages_create( self, messages: list[ModelMessage], stream: bool, model_settings: ModelSettings \| None, model_request_parameters: ModelRequestParameters, ) -> ConverseResponseTypeDef \| EventStream[ConverseStreamOutputTypeDef]: tools = self._get_tools(model_request_parameters) support_tools_choice = self.model_name.startswith(('anthropic', 'us.anthropic')) if not tools or not support_tools_choice: tool_choice: ToolChoiceTypeDef = {} elif not model_request_parameters.allow_text_result: tool_choice = {'any': {}} else: tool_choice = {'auto': {}} system_prompt, bedrock_messages = await self._map_message(messages) inference_config = self._map_inference_config(model_settings) params = { 'modelId': self.model_name, 'messages': bedrock_messages, 'system': [{'text': system_prompt}], 'inferenceConfig': inference_config, **( {'toolConfig': {'tools': tools, **({'toolChoice': tool_choice} if tool_choice else {})}} if tools else {} ), } if stream: model_response = await anyio.to_thread.run_sync(functools.partial(self.client.converse_stream, **params)) model_response = model_response['stream'] else: model_response = await anyio.to_thread.run_sync(functools.partial(self.client.converse, **params)) return model_response @staticmethod def _map_inference_config( model_settings: ModelSettings \| None, ) -> InferenceConfigurationTypeDef: model_settings = model_settings or {} inference_config: InferenceConfigurationTypeDef = {} if max_tokens := model_settings.get('max_tokens'): inference_config['maxTokens'] = max_tokens if temperature := model_settings.get('temperature'): inference_config['temperature'] = temperature if top_p := model_settings.get('top_p'): inference_config['topP'] = top_p # TODO(Marcelo): This is not included in model_settings yet. # if stop_sequences := model_settings.get('stop_sequences'): # inference_config['stopSequences'] = stop_sequences return inference_config async def _map_message(self, messages: list[ModelMessage]) -> tuple[str, list[MessageUnionTypeDef]]: """Just maps a `pydantic_ai.Message` to the Bedrock `MessageUnionTypeDef`.""" system_prompt: str = '' bedrock_messages: list[MessageUnionTypeDef] = [] for m in messages: if isinstance(m, ModelRequest): for part in m.parts: if isinstance(part, SystemPromptPart): system_prompt += part.content elif isinstance(part, UserPromptPart): bedrock_messages.extend(await self._map_user_prompt(part)) elif isinstance(part, ToolReturnPart): assert part.tool_call_id is not None bedrock_messages.append( { 'role': 'user', 'content': [ { 'toolResult': { 'toolUseId': part.tool_call_id, 'content': [{'text': part.model_response_str()}], 'status': 'success', } } ], } ) elif isinstance(part, RetryPromptPart): # TODO(Marcelo): We need to add a test here. if part.tool_name is None: # pragma: no cover bedrock_messages.append({'role': 'user', 'content': [{'text': part.model_response()}]}) else: assert part.tool_call_id is not None bedrock_messages.append( { 'role': 'user', 'content': [ { 'toolResult': { 'toolUseId': part.tool_call_id, 'content': [{'text': part.model_response()}], 'status': 'error', } } ], } ) elif isinstance(m, ModelResponse): content: list[ContentBlockOutputTypeDef] = [] for item in m.parts: if isinstance(item, TextPart): content.append({'text': item.content}) else: assert isinstance(item, ToolCallPart) content.append(self._map_tool_call(item)) bedrock_messages.append({'role': 'assistant', 'content': content}) else: assert_never(m) return system_prompt, bedrock_messages @staticmethod async def _map_user_prompt(part: UserPromptPart) -> list[MessageUnionTypeDef]: content: list[ContentBlockUnionTypeDef] = [] if isinstance(part.content, str): content.append({'text': part.content}) else: document_count = 0 for item in part.content: if isinstance(item, str): content.append({'text': item}) elif isinstance(item, BinaryContent): format = item.format if item.is_document: document_count += 1 name = f'Document {document_count}' assert format in ('pdf', 'txt', 'csv', 'doc', 'docx', 'xls', 'xlsx', 'html', 'md') content.append({'document': {'name': name, 'format': format, 'source': {'bytes': item.data}}}) elif item.is_image: assert format in ('jpeg', 'png', 'gif', 'webp') content.append({'image': {'format': format, 'source': {'bytes': item.data}}}) else: raise NotImplementedError('Binary content is not supported yet.') elif isinstance(item, (ImageUrl, DocumentUrl)): response = await cached_async_http_client().get(item.url) response.raise_for_status() if item.kind == 'image-url': format = item.media_type.split('/')[1] assert format in ('jpeg', 'png', 'gif', 'webp'), f'Unsupported image format: {format}' image: ImageBlockTypeDef = {'format': format, 'source': {'bytes': response.content}} content.append({'image': image}) elif item.kind == 'document-url': document_count += 1 name = f'Document {document_count}' data = response.content content.append({'document': {'name': name, 'format': item.format, 'source': {'bytes': data}}}) elif isinstance(item, AudioUrl): # pragma: no cover raise NotImplementedError('Audio is not supported yet.') else: assert_never(item) return [{'role': 'user', 'content': content}] @staticmethod def _map_tool_call(t: ToolCallPart) -> ContentBlockOutputTypeDef: return { 'toolUse': {'toolUseId': _utils.guard_tool_call_id(t=t), 'name': t.tool_name, 'input': t.args_as_dict()} }`` | #### model\_name `property` `model_name: [str](https://docs.python.org/3/library/stdtypes.html#str)` The model name. #### system `property` `system: [str](https://docs.python.org/3/library/stdtypes.html#str)` The system / model provider, ex: openai. #### \_\_init\_\_ `__init__( model_name: [BedrockModelName](#pydantic_ai.models.bedrock.BedrockModelName "pydantic_ai.models.bedrock.BedrockModelName") , *, provider: ( [Literal](https://docs.python.org/3/library/typing.html#typing.Literal "typing.Literal") ["bedrock"] | [Provider](../../providers/#pydantic_ai.providers.Provider "pydantic_ai.providers.Provider") [BaseClient] ) = "bedrock" )` Initialize a Bedrock model. Parameters: | Name | Type | Description | Default | | --- | --- | --- | --- | | `model_name` | `[BedrockModelName](#pydantic_ai.models.bedrock.BedrockModelName "pydantic_ai.models.bedrock.BedrockModelName") ` | The name of the model to use. | _required_ | | `model_name` | `[BedrockModelName](#pydantic_ai.models.bedrock.BedrockModelName "pydantic_ai.models.bedrock.BedrockModelName") ` | The name of the Bedrock model to use. List of model names available [here](https://docs.aws.amazon.com/bedrock/latest/userguide/models-supported.html)
. | _required_ | | `provider` | `[Literal](https://docs.python.org/3/library/typing.html#typing.Literal "typing.Literal") ['bedrock'] \| [Provider](../../providers/#pydantic_ai.providers.Provider "pydantic_ai.providers.Provider") [BaseClient]` | The provider to use for authentication and API access. Can be either the string 'bedrock' or an instance of `Provider[BaseClient]`. If not provided, a new provider will be created using the other parameters. | `'bedrock'` | Source code in `pydantic_ai_slim/pydantic_ai/models/bedrock.py` | | | | --- | --- | | 141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161 | ``def __init__( self, model_name: BedrockModelName, *, provider: Literal['bedrock'] \| Provider[BaseClient] = 'bedrock', ): """Initialize a Bedrock model. Args: model_name: The name of the model to use. model_name: The name of the Bedrock model to use. List of model names available [here](https://docs.aws.amazon.com/bedrock/latest/userguide/models-supported.html). provider: The provider to use for authentication and API access. Can be either the string 'bedrock' or an instance of `Provider[BaseClient]`. If not provided, a new provider will be created using the other parameters. """ self._model_name = model_name if isinstance(provider, str): provider = infer_provider(provider) self.client = cast('BedrockRuntimeClient', provider.client)`` | ### BedrockStreamedResponse `dataclass` Bases: `[StreamedResponse](../base/#pydantic_ai.models.StreamedResponse "pydantic_ai.models.StreamedResponse") ` Implementation of `StreamedResponse` for Bedrock models. Source code in `pydantic_ai_slim/pydantic_ai/models/bedrock.py` | | | | --- | --- | | 410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481 | ``@dataclass class BedrockStreamedResponse(StreamedResponse): """Implementation of `StreamedResponse` for Bedrock models.""" _model_name: BedrockModelName _event_stream: EventStream[ConverseStreamOutputTypeDef] _timestamp: datetime = field(default_factory=_utils.now_utc) async def _get_event_iterator(self) -> AsyncIterator[ModelResponseStreamEvent]: """Return an async iterator of [`ModelResponseStreamEvent`][pydantic_ai.messages.ModelResponseStreamEvent]s. This method should be implemented by subclasses to translate the vendor-specific stream of events into pydantic_ai-format events. """ chunk: ConverseStreamOutputTypeDef tool_id: str \| None = None async for chunk in _AsyncIteratorWrapper(self._event_stream): # TODO(Marcelo): Switch this to `match` when we drop Python 3.9 support. if 'messageStart' in chunk: continue if 'messageStop' in chunk: continue if 'metadata' in chunk: if 'usage' in chunk['metadata']: self._usage += self._map_usage(chunk['metadata']) continue if 'contentBlockStart' in chunk: index = chunk['contentBlockStart']['contentBlockIndex'] start = chunk['contentBlockStart']['start'] if 'toolUse' in start: tool_use_start = start['toolUse'] tool_id = tool_use_start['toolUseId'] tool_name = tool_use_start['name'] maybe_event = self._parts_manager.handle_tool_call_delta( vendor_part_id=index, tool_name=tool_name, args=None, tool_call_id=tool_id, ) if maybe_event: yield maybe_event if 'contentBlockDelta' in chunk: index = chunk['contentBlockDelta']['contentBlockIndex'] delta = chunk['contentBlockDelta']['delta'] if 'text' in delta: yield self._parts_manager.handle_text_delta(vendor_part_id=index, content=delta['text']) if 'toolUse' in delta: tool_use = delta['toolUse'] maybe_event = self._parts_manager.handle_tool_call_delta( vendor_part_id=index, tool_name=tool_use.get('name'), args=tool_use.get('input'), tool_call_id=tool_id, ) if maybe_event: yield maybe_event @property def timestamp(self) -> datetime: return self._timestamp @property def model_name(self) -> str: """Get the model name of the response.""" return self._model_name def _map_usage(self, metadata: ConverseStreamMetadataEventTypeDef) -> result.Usage: return result.Usage( request_tokens=metadata['usage']['inputTokens'], response_tokens=metadata['usage']['outputTokens'], total_tokens=metadata['usage']['totalTokens'], )`` | #### model\_name `property` `model_name: [str](https://docs.python.org/3/library/stdtypes.html#str)` Get the model name of the response. --- # pydantic_ai.models.cohere - PydanticAI `pydantic_ai.models.cohere` =========================== Setup ----- For details on how to set up authentication with this model, see [model configuration for Cohere](../../../models/#cohere) . ### LatestCohereModelNames `module-attribute` `LatestCohereModelNames = [Literal](https://docs.python.org/3/library/typing.html#typing.Literal "typing.Literal") [ "c4ai-aya-expanse-32b", "c4ai-aya-expanse-8b", "command", "command-light", "command-light-nightly", "command-nightly", "command-r", "command-r-03-2024", "command-r-08-2024", "command-r-plus", "command-r-plus-04-2024", "command-r-plus-08-2024", "command-r7b-12-2024", ]` Latest Cohere models. ### CohereModelName `module-attribute` `CohereModelName = [Union](https://docs.python.org/3/library/typing.html#typing.Union "typing.Union") [[str](https://docs.python.org/3/library/stdtypes.html#str) , [LatestCohereModelNames](#pydantic_ai.models.cohere.LatestCohereModelNames "pydantic_ai.models.cohere.LatestCohereModelNames") ]` Possible Cohere model names. Since Cohere supports a variety of date-stamped models, we explicitly list the latest models but allow any name in the type hints. See [Cohere's docs](https://docs.cohere.com/v2/docs/models) for a list of all available models. ### CohereModelSettings Bases: `[ModelSettings](../../settings/#pydantic_ai.settings.ModelSettings "pydantic_ai.settings.ModelSettings") ` Settings used for a Cohere model request. ALL FIELDS MUST BE `cohere_` PREFIXED SO YOU CAN MERGE THEM WITH OTHER MODELS. Source code in `pydantic_ai_slim/pydantic_ai/models/cohere.py` | | | | --- | --- | | 82
83
84
85
86 | ``class CohereModelSettings(ModelSettings): """Settings used for a Cohere model request. ALL FIELDS MUST BE `cohere_` PREFIXED SO YOU CAN MERGE THEM WITH OTHER MODELS. """`` | ### CohereModel `dataclass` Bases: `[Model](../base/#pydantic_ai.models.Model "pydantic_ai.models.Model") ` A model that uses the Cohere API. Internally, this uses the [Cohere Python client](https://github.com/cohere-ai/cohere-python) to interact with the API. Apart from `__init__`, all methods are private or match those of the base class. Source code in `pydantic_ai_slim/pydantic_ai/models/cohere.py` | | | | --- | --- | | 91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273 | ``@dataclass(init=False) class CohereModel(Model): """A model that uses the Cohere API. Internally, this uses the [Cohere Python client]( https://github.com/cohere-ai/cohere-python) to interact with the API. Apart from `__init__`, all methods are private or match those of the base class. """ client: AsyncClientV2 = field(repr=False) _model_name: CohereModelName = field(repr=False) _system: str = field(default='cohere', repr=False) def __init__( self, model_name: CohereModelName, *, provider: Literal['cohere'] \| Provider[AsyncClientV2] = 'cohere', ): """Initialize an Cohere model. Args: model_name: The name of the Cohere model to use. List of model names available [here](https://docs.cohere.com/docs/models#command). provider: The provider to use for authentication and API access. Can be either the string 'cohere' or an instance of `Provider[AsyncClientV2]`. If not provided, a new provider will be created using the other parameters. """ self._model_name: CohereModelName = model_name if isinstance(provider, str): provider = infer_provider(provider) self.client = provider.client @property def base_url(self) -> str: client_wrapper = self.client._client_wrapper # type: ignore return str(client_wrapper.get_base_url()) async def request( self, messages: list[ModelMessage], model_settings: ModelSettings \| None, model_request_parameters: ModelRequestParameters, ) -> tuple[ModelResponse, result.Usage]: check_allow_model_requests() response = await self._chat(messages, cast(CohereModelSettings, model_settings or {}), model_request_parameters) return self._process_response(response), _map_usage(response) @property def model_name(self) -> CohereModelName: """The model name.""" return self._model_name @property def system(self) -> str: """The system / model provider.""" return self._system async def _chat( self, messages: list[ModelMessage], model_settings: CohereModelSettings, model_request_parameters: ModelRequestParameters, ) -> ChatResponse: tools = self._get_tools(model_request_parameters) cohere_messages = list(chain(*(self._map_message(m) for m in messages))) try: return await self.client.chat( model=self._model_name, messages=cohere_messages, tools=tools or OMIT, max_tokens=model_settings.get('max_tokens', OMIT), temperature=model_settings.get('temperature', OMIT), p=model_settings.get('top_p', OMIT), seed=model_settings.get('seed', OMIT), presence_penalty=model_settings.get('presence_penalty', OMIT), frequency_penalty=model_settings.get('frequency_penalty', OMIT), ) except ApiError as e: if (status_code := e.status_code) and status_code >= 400: raise ModelHTTPError(status_code=status_code, model_name=self.model_name, body=e.body) from e raise def _process_response(self, response: ChatResponse) -> ModelResponse: """Process a non-streamed response, and prepare a message to return.""" parts: list[ModelResponsePart] = [] if response.message.content is not None and len(response.message.content) > 0: # While Cohere's API returns a list, it only does that for future proofing # and currently only one item is being returned. choice = response.message.content[0] parts.append(TextPart(choice.text)) for c in response.message.tool_calls or []: if c.function and c.function.name and c.function.arguments: parts.append( ToolCallPart( tool_name=c.function.name, args=c.function.arguments, tool_call_id=c.id or _generate_tool_call_id(), ) ) return ModelResponse(parts=parts, model_name=self._model_name) def _map_message(self, message: ModelMessage) -> Iterable[ChatMessageV2]: """Just maps a `pydantic_ai.Message` to a `cohere.ChatMessageV2`.""" if isinstance(message, ModelRequest): yield from self._map_user_message(message) elif isinstance(message, ModelResponse): texts: list[str] = [] tool_calls: list[ToolCallV2] = [] for item in message.parts: if isinstance(item, TextPart): texts.append(item.content) elif isinstance(item, ToolCallPart): tool_calls.append(self._map_tool_call(item)) else: assert_never(item) message_param = AssistantChatMessageV2(role='assistant') if texts: message_param.content = [TextAssistantMessageContentItem(text='\n\n'.join(texts))] if tool_calls: message_param.tool_calls = tool_calls yield message_param else: assert_never(message) def _get_tools(self, model_request_parameters: ModelRequestParameters) -> list[ToolV2]: tools = [self._map_tool_definition(r) for r in model_request_parameters.function_tools] if model_request_parameters.result_tools: tools += [self._map_tool_definition(r) for r in model_request_parameters.result_tools] return tools @staticmethod def _map_tool_call(t: ToolCallPart) -> ToolCallV2: return ToolCallV2( id=_guard_tool_call_id(t=t), type='function', function=ToolCallV2Function( name=t.tool_name, arguments=t.args_as_json_str(), ), ) @staticmethod def _map_tool_definition(f: ToolDefinition) -> ToolV2: return ToolV2( type='function', function=ToolV2Function( name=f.name, description=f.description, parameters=f.parameters_json_schema, ), ) @classmethod def _map_user_message(cls, message: ModelRequest) -> Iterable[ChatMessageV2]: for part in message.parts: if isinstance(part, SystemPromptPart): yield SystemChatMessageV2(role='system', content=part.content) elif isinstance(part, UserPromptPart): if isinstance(part.content, str): yield UserChatMessageV2(role='user', content=part.content) else: raise RuntimeError('Cohere does not yet support multi-modal inputs.') elif isinstance(part, ToolReturnPart): yield ToolChatMessageV2( role='tool', tool_call_id=_guard_tool_call_id(t=part), content=part.model_response_str(), ) elif isinstance(part, RetryPromptPart): if part.tool_name is None: yield UserChatMessageV2(role='user', content=part.model_response()) else: yield ToolChatMessageV2( role='tool', tool_call_id=_guard_tool_call_id(t=part), content=part.model_response(), ) else: assert_never(part)`` | #### \_\_init\_\_ `__init__( model_name: [CohereModelName](#pydantic_ai.models.cohere.CohereModelName "pydantic_ai.models.cohere.CohereModelName") , *, provider: ( [Literal](https://docs.python.org/3/library/typing.html#typing.Literal "typing.Literal") ["cohere"] | [Provider](../../providers/#pydantic_ai.providers.Provider "pydantic_ai.providers.Provider") [AsyncClientV2] ) = "cohere" )` Initialize an Cohere model. Parameters: | Name | Type | Description | Default | | --- | --- | --- | --- | | `model_name` | `[CohereModelName](#pydantic_ai.models.cohere.CohereModelName "pydantic_ai.models.cohere.CohereModelName") ` | The name of the Cohere model to use. List of model names available [here](https://docs.cohere.com/docs/models#command)
. | _required_ | | `provider` | `[Literal](https://docs.python.org/3/library/typing.html#typing.Literal "typing.Literal") ['cohere'] \| [Provider](../../providers/#pydantic_ai.providers.Provider "pydantic_ai.providers.Provider") [AsyncClientV2]` | The provider to use for authentication and API access. Can be either the string 'cohere' or an instance of `Provider[AsyncClientV2]`. If not provided, a new provider will be created using the other parameters. | `'cohere'` | Source code in `pydantic_ai_slim/pydantic_ai/models/cohere.py` | | | | --- | --- | | 106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125 | ``def __init__( self, model_name: CohereModelName, *, provider: Literal['cohere'] \| Provider[AsyncClientV2] = 'cohere', ): """Initialize an Cohere model. Args: model_name: The name of the Cohere model to use. List of model names available [here](https://docs.cohere.com/docs/models#command). provider: The provider to use for authentication and API access. Can be either the string 'cohere' or an instance of `Provider[AsyncClientV2]`. If not provided, a new provider will be created using the other parameters. """ self._model_name: CohereModelName = model_name if isinstance(provider, str): provider = infer_provider(provider) self.client = provider.client`` | #### model\_name `property` `model_name: [CohereModelName](#pydantic_ai.models.cohere.CohereModelName "pydantic_ai.models.cohere.CohereModelName")` The model name. #### system `property` `system: [str](https://docs.python.org/3/library/stdtypes.html#str)` The system / model provider. --- # pydantic_ai.models.gemini - PydanticAI `pydantic_ai.models.gemini` =========================== Custom interface to the `generativelanguage.googleapis.com` API using [HTTPX](https://www.python-httpx.org/) and [Pydantic](https://docs.pydantic.dev/latest/) . The Google SDK for interacting with the `generativelanguage.googleapis.com` API [`google-generativeai`](https://ai.google.dev/gemini-api/docs/quickstart?lang=python) reads like it was written by a Java developer who thought they knew everything about OOP, spent 30 minutes trying to learn Python, gave up and decided to build the library to prove how horrible Python is. It also doesn't use httpx for HTTP requests, and tries to implement tool calling itself, but doesn't use Pydantic or equivalent for validation. We therefore implement support for the API directly. Despite these shortcomings, the Gemini model is actually quite powerful and very fast. Setup ----- For details on how to set up authentication with this model, see [model configuration for Gemini](../../../models/#gemini) . ### LatestGeminiModelNames `module-attribute` `LatestGeminiModelNames = [Literal](https://docs.python.org/3/library/typing.html#typing.Literal "typing.Literal") [ "gemini-1.5-flash", "gemini-1.5-flash-8b", "gemini-1.5-pro", "gemini-1.0-pro", "gemini-2.0-flash-exp", "gemini-2.0-flash-thinking-exp-01-21", "gemini-exp-1206", "gemini-2.0-flash", "gemini-2.0-flash-lite-preview-02-05", "gemini-2.0-pro-exp-02-05", "gemini-2.5-pro-exp-03-25", ]` Latest Gemini models. ### GeminiModelName `module-attribute` `GeminiModelName = [Union](https://docs.python.org/3/library/typing.html#typing.Union "typing.Union") [[str](https://docs.python.org/3/library/stdtypes.html#str) , [LatestGeminiModelNames](#pydantic_ai.models.gemini.LatestGeminiModelNames "pydantic_ai.models.gemini.LatestGeminiModelNames") ]` Possible Gemini model names. Since Gemini supports a variety of date-stamped models, we explicitly list the latest models but allow any name in the type hints. See [the Gemini API docs](https://ai.google.dev/gemini-api/docs/models/gemini#model-variations) for a full list. ### GeminiModelSettings Bases: `[ModelSettings](../../settings/#pydantic_ai.settings.ModelSettings "pydantic_ai.settings.ModelSettings") ` Settings used for a Gemini model request. ALL FIELDS MUST BE `gemini_` PREFIXED SO YOU CAN MERGE THEM WITH OTHER MODELS. Source code in `pydantic_ai_slim/pydantic_ai/models/gemini.py` | | | | --- | --- | | 73
74
75
76
77
78
79 | ``class GeminiModelSettings(ModelSettings): """Settings used for a Gemini model request. ALL FIELDS MUST BE `gemini_` PREFIXED SO YOU CAN MERGE THEM WITH OTHER MODELS. """ gemini_safety_settings: list[GeminiSafetySettings]`` | ### GeminiModel `dataclass` Bases: `[Model](../base/#pydantic_ai.models.Model "pydantic_ai.models.Model") ` A model that uses Gemini via `generativelanguage.googleapis.com` API. This is implemented from scratch rather than using a dedicated SDK, good API documentation is available [here](https://ai.google.dev/api) . Apart from `__init__`, all methods are private or match those of the base class. Source code in `pydantic_ai_slim/pydantic_ai/models/gemini.py` | | | | --- | --- | | 82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330 | ``@dataclass(init=False) class GeminiModel(Model): """A model that uses Gemini via `generativelanguage.googleapis.com` API. This is implemented from scratch rather than using a dedicated SDK, good API documentation is available [here](https://ai.google.dev/api). Apart from `__init__`, all methods are private or match those of the base class. """ client: httpx.AsyncClient = field(repr=False) _model_name: GeminiModelName = field(repr=False) _provider: Literal['google-gla', 'google-vertex'] \| Provider[httpx.AsyncClient] \| None = field(repr=False) _auth: AuthProtocol \| None = field(repr=False) _url: str \| None = field(repr=False) _system: str = field(default='gemini', repr=False) def __init__( self, model_name: GeminiModelName, *, provider: Literal['google-gla', 'google-vertex'] \| Provider[httpx.AsyncClient] = 'google-gla', ): """Initialize a Gemini model. Args: model_name: The name of the model to use. provider: The provider to use for authentication and API access. Can be either the string 'google-gla' or 'google-vertex' or an instance of `Provider[httpx.AsyncClient]`. If not provided, a new provider will be created using the other parameters. """ self._model_name = model_name self._provider = provider if isinstance(provider, str): provider = infer_provider(provider) self._system = provider.name self.client = provider.client self._url = str(self.client.base_url) @property def base_url(self) -> str: assert self._url is not None, 'URL not initialized' return self._url async def request( self, messages: list[ModelMessage], model_settings: ModelSettings \| None, model_request_parameters: ModelRequestParameters, ) -> tuple[ModelResponse, usage.Usage]: check_allow_model_requests() async with self._make_request( messages, False, cast(GeminiModelSettings, model_settings or {}), model_request_parameters ) as http_response: data = await http_response.aread() response = _gemini_response_ta.validate_json(data) return self._process_response(response), _metadata_as_usage(response) @asynccontextmanager async def request_stream( self, messages: list[ModelMessage], model_settings: ModelSettings \| None, model_request_parameters: ModelRequestParameters, ) -> AsyncIterator[StreamedResponse]: check_allow_model_requests() async with self._make_request( messages, True, cast(GeminiModelSettings, model_settings or {}), model_request_parameters ) as http_response: yield await self._process_streamed_response(http_response) @property def model_name(self) -> GeminiModelName: """The model name.""" return self._model_name @property def system(self) -> str: """The system / model provider.""" return self._system def _get_tools(self, model_request_parameters: ModelRequestParameters) -> _GeminiTools \| None: tools = [_function_from_abstract_tool(t) for t in model_request_parameters.function_tools] if model_request_parameters.result_tools: tools += [_function_from_abstract_tool(t) for t in model_request_parameters.result_tools] return _GeminiTools(function_declarations=tools) if tools else None def _get_tool_config( self, model_request_parameters: ModelRequestParameters, tools: _GeminiTools \| None ) -> _GeminiToolConfig \| None: if model_request_parameters.allow_text_result: return None elif tools: return _tool_config([t['name'] for t in tools['function_declarations']]) else: return _tool_config([]) @asynccontextmanager async def _make_request( self, messages: list[ModelMessage], streamed: bool, model_settings: GeminiModelSettings, model_request_parameters: ModelRequestParameters, ) -> AsyncIterator[HTTPResponse]: tools = self._get_tools(model_request_parameters) tool_config = self._get_tool_config(model_request_parameters, tools) sys_prompt_parts, contents = await self._message_to_gemini_content(messages) request_data = _GeminiRequest(contents=contents) if sys_prompt_parts: request_data['system_instruction'] = _GeminiTextContent(role='user', parts=sys_prompt_parts) if tools is not None: request_data['tools'] = tools if tool_config is not None: request_data['tool_config'] = tool_config generation_config: _GeminiGenerationConfig = {} if model_settings: if (max_tokens := model_settings.get('max_tokens')) is not None: generation_config['max_output_tokens'] = max_tokens if (temperature := model_settings.get('temperature')) is not None: generation_config['temperature'] = temperature if (top_p := model_settings.get('top_p')) is not None: generation_config['top_p'] = top_p if (presence_penalty := model_settings.get('presence_penalty')) is not None: generation_config['presence_penalty'] = presence_penalty if (frequency_penalty := model_settings.get('frequency_penalty')) is not None: generation_config['frequency_penalty'] = frequency_penalty if (gemini_safety_settings := model_settings.get('gemini_safety_settings')) != []: request_data['safety_settings'] = gemini_safety_settings if generation_config: request_data['generation_config'] = generation_config headers = {'Content-Type': 'application/json', 'User-Agent': get_user_agent()} url = f'/{self._model_name}:{"streamGenerateContent" if streamed else "generateContent"}' request_json = _gemini_request_ta.dump_json(request_data, by_alias=True) async with self.client.stream( 'POST', url, content=request_json, headers=headers, timeout=model_settings.get('timeout', USE_CLIENT_DEFAULT), ) as r: if (status_code := r.status_code) != 200: await r.aread() if status_code >= 400: raise ModelHTTPError(status_code=status_code, model_name=self.model_name, body=r.text) raise UnexpectedModelBehavior(f'Unexpected response from gemini {status_code}', r.text) yield r def _process_response(self, response: _GeminiResponse) -> ModelResponse: if len(response['candidates']) != 1: raise UnexpectedModelBehavior('Expected exactly one candidate in Gemini response') if 'content' not in response['candidates'][0]: if response['candidates'][0].get('finish_reason') == 'SAFETY': raise UnexpectedModelBehavior('Safety settings triggered', str(response)) else: raise UnexpectedModelBehavior('Content field missing from Gemini response', str(response)) parts = response['candidates'][0]['content']['parts'] return _process_response_from_parts(parts, model_name=response.get('model_version', self._model_name)) async def _process_streamed_response(self, http_response: HTTPResponse) -> StreamedResponse: """Process a streamed response, and prepare a streaming response to return.""" aiter_bytes = http_response.aiter_bytes() start_response: _GeminiResponse \| None = None content = bytearray() async for chunk in aiter_bytes: content.extend(chunk) responses = _gemini_streamed_response_ta.validate_json( _ensure_decodeable(content), experimental_allow_partial='trailing-strings', ) if responses: last = responses[-1] if last['candidates'] and last['candidates'][0].get('content', {}).get('parts'): start_response = last break if start_response is None: raise UnexpectedModelBehavior('Streamed response ended without content or tool calls') return GeminiStreamedResponse(_model_name=self._model_name, _content=content, _stream=aiter_bytes) @classmethod async def _message_to_gemini_content( cls, messages: list[ModelMessage] ) -> tuple[list[_GeminiTextPart], list[_GeminiContent]]: sys_prompt_parts: list[_GeminiTextPart] = [] contents: list[_GeminiContent] = [] for m in messages: if isinstance(m, ModelRequest): message_parts: list[_GeminiPartUnion] = [] for part in m.parts: if isinstance(part, SystemPromptPart): sys_prompt_parts.append(_GeminiTextPart(text=part.content)) elif isinstance(part, UserPromptPart): message_parts.extend(await cls._map_user_prompt(part)) elif isinstance(part, ToolReturnPart): message_parts.append(_response_part_from_response(part.tool_name, part.model_response_object())) elif isinstance(part, RetryPromptPart): if part.tool_name is None: message_parts.append(_GeminiTextPart(text=part.model_response())) else: response = {'call_error': part.model_response()} message_parts.append(_response_part_from_response(part.tool_name, response)) else: assert_never(part) if message_parts: contents.append(_GeminiContent(role='user', parts=message_parts)) elif isinstance(m, ModelResponse): contents.append(_content_model_response(m)) else: assert_never(m) return sys_prompt_parts, contents @staticmethod async def _map_user_prompt(part: UserPromptPart) -> list[_GeminiPartUnion]: if isinstance(part.content, str): return [{'text': part.content}] else: content: list[_GeminiPartUnion] = [] for item in part.content: if isinstance(item, str): content.append({'text': item}) elif isinstance(item, BinaryContent): base64_encoded = base64.b64encode(item.data).decode('utf-8') content.append( _GeminiInlineDataPart(inline_data={'data': base64_encoded, 'mime_type': item.media_type}) ) elif isinstance(item, (AudioUrl, ImageUrl, DocumentUrl)): client = cached_async_http_client() response = await client.get(item.url, follow_redirects=True) response.raise_for_status() mime_type = response.headers['Content-Type'].split(';')[0] inline_data = _GeminiInlineDataPart( inline_data={'data': base64.b64encode(response.content).decode('utf-8'), 'mime_type': mime_type} ) content.append(inline_data) else: assert_never(item) return content`` | #### \_\_init\_\_ `__init__( model_name: [GeminiModelName](#pydantic_ai.models.gemini.GeminiModelName "pydantic_ai.models.gemini.GeminiModelName") , *, provider: ( [Literal](https://docs.python.org/3/library/typing.html#typing.Literal "typing.Literal") ["google-gla", "google-vertex"] | [Provider](../../providers/#pydantic_ai.providers.Provider "pydantic_ai.providers.Provider") [AsyncClient] ) = "google-gla" )` Initialize a Gemini model. Parameters: | Name | Type | Description | Default | | --- | --- | --- | --- | | `model_name` | `[GeminiModelName](#pydantic_ai.models.gemini.GeminiModelName "pydantic_ai.models.gemini.GeminiModelName") ` | The name of the model to use. | _required_ | | `provider` | `[Literal](https://docs.python.org/3/library/typing.html#typing.Literal "typing.Literal") ['google-gla', 'google-vertex'] \| [Provider](../../providers/#pydantic_ai.providers.Provider "pydantic_ai.providers.Provider") [AsyncClient]` | The provider to use for authentication and API access. Can be either the string 'google-gla' or 'google-vertex' or an instance of `Provider[httpx.AsyncClient]`. If not provided, a new provider will be created using the other parameters. | `'google-gla'` | Source code in `pydantic_ai_slim/pydantic_ai/models/gemini.py` | | | | --- | --- | | 100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121 | ``def __init__( self, model_name: GeminiModelName, *, provider: Literal['google-gla', 'google-vertex'] \| Provider[httpx.AsyncClient] = 'google-gla', ): """Initialize a Gemini model. Args: model_name: The name of the model to use. provider: The provider to use for authentication and API access. Can be either the string 'google-gla' or 'google-vertex' or an instance of `Provider[httpx.AsyncClient]`. If not provided, a new provider will be created using the other parameters. """ self._model_name = model_name self._provider = provider if isinstance(provider, str): provider = infer_provider(provider) self._system = provider.name self.client = provider.client self._url = str(self.client.base_url)`` | #### model\_name `property` `model_name: [GeminiModelName](#pydantic_ai.models.gemini.GeminiModelName "pydantic_ai.models.gemini.GeminiModelName")` The model name. #### system `property` `system: [str](https://docs.python.org/3/library/stdtypes.html#str)` The system / model provider. ### AuthProtocol Bases: `[Protocol](https://docs.python.org/3/library/typing.html#typing.Protocol "typing.Protocol") ` Abstract definition for Gemini authentication. Source code in `pydantic_ai_slim/pydantic_ai/models/gemini.py` | | | | --- | --- | | 333
334
335
336 | `class AuthProtocol(Protocol): """Abstract definition for Gemini authentication.""" async def headers(self) -> dict[str, str]: ...` | ### ApiKeyAuth `dataclass` Authentication using an API key for the `X-Goog-Api-Key` header. Source code in `pydantic_ai_slim/pydantic_ai/models/gemini.py` | | | | --- | --- | | 339
340
341
342
343
344
345
346
347 | ``@dataclass class ApiKeyAuth: """Authentication using an API key for the `X-Goog-Api-Key` header.""" api_key: str async def headers(self) -> dict[str, str]: # https://cloud.google.com/docs/authentication/api-keys-use#using-with-rest return {'X-Goog-Api-Key': self.api_key}`` | ### GeminiStreamedResponse `dataclass` Bases: `[StreamedResponse](../base/#pydantic_ai.models.StreamedResponse "pydantic_ai.models.StreamedResponse") ` Implementation of `StreamedResponse` for the Gemini model. Source code in `pydantic_ai_slim/pydantic_ai/models/gemini.py` | | | | --- | --- | | 350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426 | ``@dataclass class GeminiStreamedResponse(StreamedResponse): """Implementation of `StreamedResponse` for the Gemini model.""" _model_name: GeminiModelName _content: bytearray _stream: AsyncIterator[bytes] _timestamp: datetime = field(default_factory=_utils.now_utc, init=False) async def _get_event_iterator(self) -> AsyncIterator[ModelResponseStreamEvent]: async for gemini_response in self._get_gemini_responses(): candidate = gemini_response['candidates'][0] if 'content' not in candidate: raise UnexpectedModelBehavior('Streamed response has no content field') gemini_part: _GeminiPartUnion for gemini_part in candidate['content']['parts']: if 'text' in gemini_part: # Using vendor_part_id=None means we can produce multiple text parts if their deltas are sprinkled # amongst the tool call deltas yield self._parts_manager.handle_text_delta(vendor_part_id=None, content=gemini_part['text']) elif 'function_call' in gemini_part: # Here, we assume all function_call parts are complete and don't have deltas. # We do this by assigning a unique randomly generated "vendor_part_id". # We need to confirm whether this is actually true, but if it isn't, we can still handle it properly # it would just be a bit more complicated. And we'd need to confirm the intended semantics. maybe_event = self._parts_manager.handle_tool_call_delta( vendor_part_id=uuid4(), tool_name=gemini_part['function_call']['name'], args=gemini_part['function_call']['args'], tool_call_id=None, ) if maybe_event is not None: yield maybe_event else: assert 'function_response' in gemini_part, f'Unexpected part: {gemini_part}' async def _get_gemini_responses(self) -> AsyncIterator[_GeminiResponse]: # This method exists to ensure we only yield completed items, so we don't need to worry about # partial gemini responses, which would make everything more complicated gemini_responses: list[_GeminiResponse] = [] current_gemini_response_index = 0 # Right now, there are some circumstances where we will have information that could be yielded sooner than it is # But changing that would make things a lot more complicated. async for chunk in self._stream: self._content.extend(chunk) gemini_responses = _gemini_streamed_response_ta.validate_json( _ensure_decodeable(self._content), experimental_allow_partial='trailing-strings', ) # The idea: yield only up to the latest response, which might still be partial. # Note that if the latest response is complete, we could yield it immediately, but there's not a good # allow_partial API to determine if the last item in the list is complete. responses_to_yield = gemini_responses[:-1] for r in responses_to_yield[current_gemini_response_index:]: current_gemini_response_index += 1 self._usage += _metadata_as_usage(r) yield r # Now yield the final response, which should be complete if gemini_responses: r = gemini_responses[-1] self._usage += _metadata_as_usage(r) yield r @property def model_name(self) -> GeminiModelName: """Get the model name of the response.""" return self._model_name @property def timestamp(self) -> datetime: """Get the timestamp of the response.""" return self._timestamp`` | #### model\_name `property` `model_name: [GeminiModelName](#pydantic_ai.models.gemini.GeminiModelName "pydantic_ai.models.gemini.GeminiModelName")` Get the model name of the response. #### timestamp `property` `timestamp: [datetime](https://docs.python.org/3/library/datetime.html#datetime.datetime "datetime.datetime")` Get the timestamp of the response. ### GeminiSafetySettings Bases: `[TypedDict](https://typing-extensions.readthedocs.io/en/latest/index.html#typing_extensions.TypedDict "typing_extensions.TypedDict") ` Safety settings options for Gemini model request. See [Gemini API docs](https://ai.google.dev/gemini-api/docs/safety-settings) for safety category and threshold descriptions. For an example on how to use `GeminiSafetySettings`, see [here](../../../agents/#model-specific-settings) . Source code in `pydantic_ai_slim/pydantic_ai/models/gemini.py` | | | | --- | --- | | 454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483 | ``class GeminiSafetySettings(TypedDict): """Safety settings options for Gemini model request. See [Gemini API docs](https://ai.google.dev/gemini-api/docs/safety-settings) for safety category and threshold descriptions. For an example on how to use `GeminiSafetySettings`, see [here](../../agents.md#model-specific-settings). """ category: Literal[ 'HARM_CATEGORY_UNSPECIFIED', 'HARM_CATEGORY_HARASSMENT', 'HARM_CATEGORY_HATE_SPEECH', 'HARM_CATEGORY_SEXUALLY_EXPLICIT', 'HARM_CATEGORY_DANGEROUS_CONTENT', 'HARM_CATEGORY_CIVIC_INTEGRITY', ] """ Safety settings category. """ threshold: Literal[ 'HARM_BLOCK_THRESHOLD_UNSPECIFIED', 'BLOCK_LOW_AND_ABOVE', 'BLOCK_MEDIUM_AND_ABOVE', 'BLOCK_ONLY_HIGH', 'BLOCK_NONE', 'OFF', ] """ Safety settings threshold. """`` | #### category `instance-attribute` `category: [Literal](https://docs.python.org/3/library/typing.html#typing.Literal "typing.Literal") [ "HARM_CATEGORY_UNSPECIFIED", "HARM_CATEGORY_HARASSMENT", "HARM_CATEGORY_HATE_SPEECH", "HARM_CATEGORY_SEXUALLY_EXPLICIT", "HARM_CATEGORY_DANGEROUS_CONTENT", "HARM_CATEGORY_CIVIC_INTEGRITY", ]` Safety settings category. #### threshold `instance-attribute` `threshold: [Literal](https://docs.python.org/3/library/typing.html#typing.Literal "typing.Literal") [ "HARM_BLOCK_THRESHOLD_UNSPECIFIED", "BLOCK_LOW_AND_ABOVE", "BLOCK_MEDIUM_AND_ABOVE", "BLOCK_ONLY_HIGH", "BLOCK_NONE", "OFF", ]` Safety settings threshold. --- # pydantic_ai.agent - PydanticAI `pydantic_ai.agent` =================== ### Agent `dataclass` Bases: `[Generic](https://docs.python.org/3/library/typing.html#typing.Generic "typing.Generic") [[AgentDepsT](../tools/#pydantic_ai.tools.AgentDepsT "pydantic_ai.tools.AgentDepsT") , [ResultDataT](../result/#pydantic_ai.result.ResultDataT "pydantic_ai.result.ResultDataT") ]` Class for defining "agents" - a way to have a specific type of "conversation" with an LLM. Agents are generic in the dependency type they take [`AgentDepsT`](../tools/#pydantic_ai.tools.AgentDepsT) and the result data type they return, [`ResultDataT`](../result/#pydantic_ai.result.ResultDataT) . By default, if neither generic parameter is customised, agents have type `Agent[None, str]`. Minimal usage example: `from pydantic_ai import Agent agent = Agent('openai:gpt-4o') result = agent.run_sync('What is the capital of France?') print(result.data) #> Paris` Source code in `pydantic_ai_slim/pydantic_ai/agent.py` | | | | --- | --- | | 75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
1001
1002
1003
1004
1005
1006
1007
1008
1009
1010
1011
1012
1013
1014
1015
1016
1017
1018
1019
1020
1021
1022
1023
1024
1025
1026
1027
1028
1029
1030
1031
1032
1033
1034
1035
1036
1037
1038
1039
1040
1041
1042
1043
1044
1045
1046
1047
1048
1049
1050
1051
1052
1053
1054
1055
1056
1057
1058
1059
1060
1061
1062
1063
1064
1065
1066
1067
1068
1069
1070
1071
1072
1073
1074
1075
1076
1077
1078
1079
1080
1081
1082
1083
1084
1085
1086
1087
1088
1089
1090
1091
1092
1093
1094
1095
1096
1097
1098
1099
1100
1101
1102
1103
1104
1105
1106
1107
1108
1109
1110
1111
1112
1113
1114
1115
1116
1117
1118
1119
1120
1121
1122
1123
1124
1125
1126
1127
1128
1129
1130
1131
1132
1133
1134
1135
1136
1137
1138
1139
1140
1141
1142
1143
1144
1145
1146
1147
1148
1149
1150
1151
1152
1153
1154
1155
1156
1157
1158
1159
1160
1161
1162
1163
1164
1165
1166
1167
1168
1169
1170
1171
1172
1173
1174
1175
1176
1177
1178
1179
1180
1181
1182
1183
1184
1185
1186
1187
1188
1189
1190
1191
1192
1193
1194
1195
1196
1197
1198
1199
1200
1201
1202
1203
1204
1205
1206
1207
1208
1209
1210
1211
1212
1213
1214
1215
1216
1217
1218
1219
1220
1221
1222
1223
1224
1225
1226
1227
1228
1229
1230
1231
1232
1233
1234
1235
1236
1237
1238
1239
1240
1241
1242
1243
1244
1245
1246
1247
1248
1249
1250
1251
1252
1253
1254
1255
1256
1257
1258
1259
1260
1261
1262
1263
1264
1265
1266
1267
1268
1269
1270
1271
1272
1273
1274
1275
1276
1277
1278
1279
1280
1281
1282
1283
1284
1285
1286
1287
1288
1289
1290
1291
1292
1293
1294
1295
1296
1297
1298
1299
1300
1301
1302 | ``@final @dataclasses.dataclass(init=False) class Agent(Generic[AgentDepsT, ResultDataT]): """Class for defining "agents" - a way to have a specific type of "conversation" with an LLM. Agents are generic in the dependency type they take [`AgentDepsT`][pydantic_ai.tools.AgentDepsT] and the result data type they return, [`ResultDataT`][pydantic_ai.result.ResultDataT]. By default, if neither generic parameter is customised, agents have type `Agent[None, str]`. Minimal usage example: ```python from pydantic_ai import Agent agent = Agent('openai:gpt-4o') result = agent.run_sync('What is the capital of France?') print(result.data) #> Paris ``` """ model: models.Model \| models.KnownModelName \| str \| None """The default model configured for this agent. We allow str here since the actual list of allowed models changes frequently. """ name: str \| None """The name of the agent, used for logging. If `None`, we try to infer the agent name from the call frame when the agent is first run. """ end_strategy: EndStrategy """Strategy for handling tool calls when a final result is found.""" model_settings: ModelSettings \| None """Optional model request settings to use for this agents's runs, by default. Note, if `model_settings` is provided by `run`, `run_sync`, or `run_stream`, those settings will be merged with this value, with the runtime argument taking priority. """ result_type: type[ResultDataT] = dataclasses.field(repr=False) """ The type of the result data, used to validate the result data, defaults to `str`. """ instrument: InstrumentationSettings \| bool \| None """Options to automatically instrument with OpenTelemetry.""" _instrument_default: ClassVar[InstrumentationSettings \| bool] = False _deps_type: type[AgentDepsT] = dataclasses.field(repr=False) _result_tool_name: str = dataclasses.field(repr=False) _result_tool_description: str \| None = dataclasses.field(repr=False) _result_schema: _result.ResultSchema[ResultDataT] \| None = dataclasses.field(repr=False) _result_validators: list[_result.ResultValidator[AgentDepsT, ResultDataT]] = dataclasses.field(repr=False) _system_prompts: tuple[str, ...] = dataclasses.field(repr=False) _system_prompt_functions: list[_system_prompt.SystemPromptRunner[AgentDepsT]] = dataclasses.field(repr=False) _system_prompt_dynamic_functions: dict[str, _system_prompt.SystemPromptRunner[AgentDepsT]] = dataclasses.field( repr=False ) _function_tools: dict[str, Tool[AgentDepsT]] = dataclasses.field(repr=False) _mcp_servers: Sequence[MCPServer] = dataclasses.field(repr=False) _default_retries: int = dataclasses.field(repr=False) _max_result_retries: int = dataclasses.field(repr=False) _override_deps: _utils.Option[AgentDepsT] = dataclasses.field(default=None, repr=False) _override_model: _utils.Option[models.Model] = dataclasses.field(default=None, repr=False) def __init__( self, model: models.Model \| models.KnownModelName \| str \| None = None, *, result_type: type[ResultDataT] = str, system_prompt: str \| Sequence[str] = (), deps_type: type[AgentDepsT] = NoneType, name: str \| None = None, model_settings: ModelSettings \| None = None, retries: int = 1, result_tool_name: str = 'final_result', result_tool_description: str \| None = None, result_retries: int \| None = None, tools: Sequence[Tool[AgentDepsT] \| ToolFuncEither[AgentDepsT, ...]] = (), mcp_servers: Sequence[MCPServer] = (), defer_model_check: bool = False, end_strategy: EndStrategy = 'early', instrument: InstrumentationSettings \| bool \| None = None, ): """Create an agent. Args: model: The default model to use for this agent, if not provide, you must provide the model when calling it. We allow str here since the actual list of allowed models changes frequently. result_type: The type of the result data, used to validate the result data, defaults to `str`. system_prompt: Static system prompts to use for this agent, you can also register system prompts via a function with [`system_prompt`][pydantic_ai.Agent.system_prompt]. deps_type: The type used for dependency injection, this parameter exists solely to allow you to fully parameterize the agent, and therefore get the best out of static type checking. If you're not using deps, but want type checking to pass, you can set `deps=None` to satisfy Pyright or add a type hint `: Agent[None, ]`. name: The name of the agent, used for logging. If `None`, we try to infer the agent name from the call frame when the agent is first run. model_settings: Optional model request settings to use for this agent's runs, by default. retries: The default number of retries to allow before raising an error. result_tool_name: The name of the tool to use for the final result. result_tool_description: The description of the final result tool. result_retries: The maximum number of retries to allow for result validation, defaults to `retries`. tools: Tools to register with the agent, you can also register tools via the decorators [`@agent.tool`][pydantic_ai.Agent.tool] and [`@agent.tool_plain`][pydantic_ai.Agent.tool_plain]. mcp_servers: MCP servers to register with the agent. You should register a [`MCPServer`][pydantic_ai.mcp.MCPServer] for each server you want the agent to connect to. defer_model_check: by default, if you provide a [named][pydantic_ai.models.KnownModelName] model, it's evaluated to create a [`Model`][pydantic_ai.models.Model] instance immediately, which checks for the necessary environment variables. Set this to `false` to defer the evaluation until the first run. Useful if you want to [override the model][pydantic_ai.Agent.override] for testing. end_strategy: Strategy for handling tool calls that are requested alongside a final result. See [`EndStrategy`][pydantic_ai.agent.EndStrategy] for more information. instrument: Set to True to automatically instrument with OpenTelemetry, which will use Logfire if it's configured. Set to an instance of [`InstrumentationSettings`][pydantic_ai.agent.InstrumentationSettings] to customize. If this isn't set, then the last value set by [`Agent.instrument_all()`][pydantic_ai.Agent.instrument_all] will be used, which defaults to False. See the [Debugging and Monitoring guide](https://ai.pydantic.dev/logfire/) for more info. """ if model is None or defer_model_check: self.model = model else: self.model = models.infer_model(model) self.end_strategy = end_strategy self.name = name self.model_settings = model_settings self.result_type = result_type self.instrument = instrument self._deps_type = deps_type self._result_tool_name = result_tool_name self._result_tool_description = result_tool_description self._result_schema = _result.ResultSchema[result_type].build( result_type, result_tool_name, result_tool_description ) self._result_validators = [] self._system_prompts = (system_prompt,) if isinstance(system_prompt, str) else tuple(system_prompt) self._system_prompt_functions = [] self._system_prompt_dynamic_functions = {} self._function_tools = {} self._default_retries = retries self._max_result_retries = result_retries if result_retries is not None else retries self._mcp_servers = mcp_servers for tool in tools: if isinstance(tool, Tool): self._register_tool(tool) else: self._register_tool(Tool(tool)) @staticmethod def instrument_all(instrument: InstrumentationSettings \| bool = True) -> None: """Set the instrumentation options for all agents where `instrument` is not set.""" Agent._instrument_default = instrument @overload async def run( self, user_prompt: str \| Sequence[_messages.UserContent], *, result_type: None = None, message_history: list[_messages.ModelMessage] \| None = None, model: models.Model \| models.KnownModelName \| str \| None = None, deps: AgentDepsT = None, model_settings: ModelSettings \| None = None, usage_limits: _usage.UsageLimits \| None = None, usage: _usage.Usage \| None = None, infer_name: bool = True, ) -> AgentRunResult[ResultDataT]: ... @overload async def run( self, user_prompt: str \| Sequence[_messages.UserContent], *, result_type: type[RunResultDataT], message_history: list[_messages.ModelMessage] \| None = None, model: models.Model \| models.KnownModelName \| str \| None = None, deps: AgentDepsT = None, model_settings: ModelSettings \| None = None, usage_limits: _usage.UsageLimits \| None = None, usage: _usage.Usage \| None = None, infer_name: bool = True, ) -> AgentRunResult[RunResultDataT]: ... async def run( self, user_prompt: str \| Sequence[_messages.UserContent], *, result_type: type[RunResultDataT] \| None = None, message_history: list[_messages.ModelMessage] \| None = None, model: models.Model \| models.KnownModelName \| str \| None = None, deps: AgentDepsT = None, model_settings: ModelSettings \| None = None, usage_limits: _usage.UsageLimits \| None = None, usage: _usage.Usage \| None = None, infer_name: bool = True, ) -> AgentRunResult[Any]: """Run the agent with a user prompt in async mode. This method builds an internal agent graph (using system prompts, tools and result schemas) and then runs the graph to completion. The result of the run is returned. Example: ```python from pydantic_ai import Agent agent = Agent('openai:gpt-4o') async def main(): agent_run = await agent.run('What is the capital of France?') print(agent_run.data) #> Paris ``` Args: user_prompt: User input to start/continue the conversation. result_type: Custom result type to use for this run, `result_type` may only be used if the agent has no result validators since result validators would expect an argument that matches the agent's result type. message_history: History of the conversation so far. model: Optional model to use for this run, required if `model` was not set when creating the agent. deps: Optional dependencies to use for this run. model_settings: Optional settings to use for this model's request. usage_limits: Optional limits on model request count or token usage. usage: Optional usage to start with, useful for resuming a conversation or agents used in tools. infer_name: Whether to try to infer the agent name from the call frame if it's not set. Returns: The result of the run. """ if infer_name and self.name is None: self._infer_name(inspect.currentframe()) async with self.iter( user_prompt=user_prompt, result_type=result_type, message_history=message_history, model=model, deps=deps, model_settings=model_settings, usage_limits=usage_limits, usage=usage, ) as agent_run: async for _ in agent_run: pass assert agent_run.result is not None, 'The graph run did not finish properly' return agent_run.result @asynccontextmanager async def iter( self, user_prompt: str \| Sequence[_messages.UserContent], *, result_type: type[RunResultDataT] \| None = None, message_history: list[_messages.ModelMessage] \| None = None, model: models.Model \| models.KnownModelName \| str \| None = None, deps: AgentDepsT = None, model_settings: ModelSettings \| None = None, usage_limits: _usage.UsageLimits \| None = None, usage: _usage.Usage \| None = None, infer_name: bool = True, ) -> AsyncIterator[AgentRun[AgentDepsT, Any]]: """A contextmanager which can be used to iterate over the agent graph's nodes as they are executed. This method builds an internal agent graph (using system prompts, tools and result schemas) and then returns an `AgentRun` object. The `AgentRun` can be used to async-iterate over the nodes of the graph as they are executed. This is the API to use if you want to consume the outputs coming from each LLM model response, or the stream of events coming from the execution of tools. The `AgentRun` also provides methods to access the full message history, new messages, and usage statistics, and the final result of the run once it has completed. For more details, see the documentation of `AgentRun`. Example: ```python from pydantic_ai import Agent agent = Agent('openai:gpt-4o') async def main(): nodes = [] async with agent.iter('What is the capital of France?') as agent_run: async for node in agent_run: nodes.append(node) print(nodes) ''' [ ModelRequestNode( request=ModelRequest( parts=[ UserPromptPart( content='What is the capital of France?', timestamp=datetime.datetime(...), part_kind='user-prompt', ) ], kind='request', ) ), CallToolsNode( model_response=ModelResponse( parts=[TextPart(content='Paris', part_kind='text')], model_name='gpt-4o', timestamp=datetime.datetime(...), kind='response', ) ), End(data=FinalResult(data='Paris', tool_name=None, tool_call_id=None)), ] ''' print(agent_run.result.data) #> Paris ``` Args: user_prompt: User input to start/continue the conversation. result_type: Custom result type to use for this run, `result_type` may only be used if the agent has no result validators since result validators would expect an argument that matches the agent's result type. message_history: History of the conversation so far. model: Optional model to use for this run, required if `model` was not set when creating the agent. deps: Optional dependencies to use for this run. model_settings: Optional settings to use for this model's request. usage_limits: Optional limits on model request count or token usage. usage: Optional usage to start with, useful for resuming a conversation or agents used in tools. infer_name: Whether to try to infer the agent name from the call frame if it's not set. Returns: The result of the run. """ if infer_name and self.name is None: self._infer_name(inspect.currentframe()) model_used = self._get_model(model) del model deps = self._get_deps(deps) new_message_index = len(message_history) if message_history else 0 result_schema: _result.ResultSchema[RunResultDataT] \| None = self._prepare_result_schema(result_type) # Build the graph graph = self._build_graph(result_type) # Build the initial state state = _agent_graph.GraphAgentState( message_history=message_history[:] if message_history else [], usage=usage or _usage.Usage(), retries=0, run_step=0, ) # We consider it a user error if a user tries to restrict the result type while having a result validator that # may change the result type from the restricted type to something else. Therefore, we consider the following # typecast reasonable, even though it is possible to violate it with otherwise-type-checked code. result_validators = cast(list[_result.ResultValidator[AgentDepsT, RunResultDataT]], self._result_validators) # TODO: Instead of this, copy the function tools to ensure they don't share current_retry state between agent # runs. Requires some changes to `Tool` to make them copyable though. for v in self._function_tools.values(): v.current_retry = 0 model_settings = merge_model_settings(self.model_settings, model_settings) usage_limits = usage_limits or _usage.UsageLimits() if isinstance(model_used, InstrumentedModel): tracer = model_used.settings.tracer else: tracer = NoOpTracer() agent_name = self.name or 'agent' run_span = tracer.start_span( 'agent run', attributes={ 'model_name': model_used.model_name if model_used else 'no-model', 'agent_name': agent_name, 'logfire.msg': f'{agent_name} run', }, ) graph_deps = _agent_graph.GraphAgentDeps[AgentDepsT, RunResultDataT]( user_deps=deps, prompt=user_prompt, new_message_index=new_message_index, model=model_used, model_settings=model_settings, usage_limits=usage_limits, max_result_retries=self._max_result_retries, end_strategy=self.end_strategy, result_schema=result_schema, result_tools=self._result_schema.tool_defs() if self._result_schema else [], result_validators=result_validators, function_tools=self._function_tools, mcp_servers=self._mcp_servers, run_span=run_span, tracer=tracer, ) start_node = _agent_graph.UserPromptNode[AgentDepsT]( user_prompt=user_prompt, system_prompts=self._system_prompts, system_prompt_functions=self._system_prompt_functions, system_prompt_dynamic_functions=self._system_prompt_dynamic_functions, ) async with graph.iter( start_node, state=state, deps=graph_deps, span=use_span(run_span, end_on_exit=True), infer_name=False, ) as graph_run: yield AgentRun(graph_run) @overload def run_sync( self, user_prompt: str \| Sequence[_messages.UserContent], *, message_history: list[_messages.ModelMessage] \| None = None, model: models.Model \| models.KnownModelName \| str \| None = None, deps: AgentDepsT = None, model_settings: ModelSettings \| None = None, usage_limits: _usage.UsageLimits \| None = None, usage: _usage.Usage \| None = None, infer_name: bool = True, ) -> AgentRunResult[ResultDataT]: ... @overload def run_sync( self, user_prompt: str \| Sequence[_messages.UserContent], *, result_type: type[RunResultDataT] \| None, message_history: list[_messages.ModelMessage] \| None = None, model: models.Model \| models.KnownModelName \| str \| None = None, deps: AgentDepsT = None, model_settings: ModelSettings \| None = None, usage_limits: _usage.UsageLimits \| None = None, usage: _usage.Usage \| None = None, infer_name: bool = True, ) -> AgentRunResult[RunResultDataT]: ... def run_sync( self, user_prompt: str \| Sequence[_messages.UserContent], *, result_type: type[RunResultDataT] \| None = None, message_history: list[_messages.ModelMessage] \| None = None, model: models.Model \| models.KnownModelName \| str \| None = None, deps: AgentDepsT = None, model_settings: ModelSettings \| None = None, usage_limits: _usage.UsageLimits \| None = None, usage: _usage.Usage \| None = None, infer_name: bool = True, ) -> AgentRunResult[Any]: """Synchronously run the agent with a user prompt. This is a convenience method that wraps [`self.run`][pydantic_ai.Agent.run] with `loop.run_until_complete(...)`. You therefore can't use this method inside async code or if there's an active event loop. Example: ```python from pydantic_ai import Agent agent = Agent('openai:gpt-4o') result_sync = agent.run_sync('What is the capital of Italy?') print(result_sync.data) #> Rome ``` Args: user_prompt: User input to start/continue the conversation. result_type: Custom result type to use for this run, `result_type` may only be used if the agent has no result validators since result validators would expect an argument that matches the agent's result type. message_history: History of the conversation so far. model: Optional model to use for this run, required if `model` was not set when creating the agent. deps: Optional dependencies to use for this run. model_settings: Optional settings to use for this model's request. usage_limits: Optional limits on model request count or token usage. usage: Optional usage to start with, useful for resuming a conversation or agents used in tools. infer_name: Whether to try to infer the agent name from the call frame if it's not set. Returns: The result of the run. """ if infer_name and self.name is None: self._infer_name(inspect.currentframe()) return get_event_loop().run_until_complete( self.run( user_prompt, result_type=result_type, message_history=message_history, model=model, deps=deps, model_settings=model_settings, usage_limits=usage_limits, usage=usage, infer_name=False, ) ) @overload def run_stream( self, user_prompt: str \| Sequence[_messages.UserContent], *, result_type: None = None, message_history: list[_messages.ModelMessage] \| None = None, model: models.Model \| models.KnownModelName \| str \| None = None, deps: AgentDepsT = None, model_settings: ModelSettings \| None = None, usage_limits: _usage.UsageLimits \| None = None, usage: _usage.Usage \| None = None, infer_name: bool = True, ) -> AbstractAsyncContextManager[result.StreamedRunResult[AgentDepsT, ResultDataT]]: ... @overload def run_stream( self, user_prompt: str \| Sequence[_messages.UserContent], *, result_type: type[RunResultDataT], message_history: list[_messages.ModelMessage] \| None = None, model: models.Model \| models.KnownModelName \| str \| None = None, deps: AgentDepsT = None, model_settings: ModelSettings \| None = None, usage_limits: _usage.UsageLimits \| None = None, usage: _usage.Usage \| None = None, infer_name: bool = True, ) -> AbstractAsyncContextManager[result.StreamedRunResult[AgentDepsT, RunResultDataT]]: ... @asynccontextmanager async def run_stream( # noqa C901 self, user_prompt: str \| Sequence[_messages.UserContent], *, result_type: type[RunResultDataT] \| None = None, message_history: list[_messages.ModelMessage] \| None = None, model: models.Model \| models.KnownModelName \| str \| None = None, deps: AgentDepsT = None, model_settings: ModelSettings \| None = None, usage_limits: _usage.UsageLimits \| None = None, usage: _usage.Usage \| None = None, infer_name: bool = True, ) -> AsyncIterator[result.StreamedRunResult[AgentDepsT, Any]]: """Run the agent with a user prompt in async mode, returning a streamed response. Example: ```python from pydantic_ai import Agent agent = Agent('openai:gpt-4o') async def main(): async with agent.run_stream('What is the capital of the UK?') as response: print(await response.get_data()) #> London ``` Args: user_prompt: User input to start/continue the conversation. result_type: Custom result type to use for this run, `result_type` may only be used if the agent has no result validators since result validators would expect an argument that matches the agent's result type. message_history: History of the conversation so far. model: Optional model to use for this run, required if `model` was not set when creating the agent. deps: Optional dependencies to use for this run. model_settings: Optional settings to use for this model's request. usage_limits: Optional limits on model request count or token usage. usage: Optional usage to start with, useful for resuming a conversation or agents used in tools. infer_name: Whether to try to infer the agent name from the call frame if it's not set. Returns: The result of the run. """ # TODO: We need to deprecate this now that we have the `iter` method. # Before that, though, we should add an event for when we reach the final result of the stream. if infer_name and self.name is None: # f_back because `asynccontextmanager` adds one frame if frame := inspect.currentframe(): # pragma: no branch self._infer_name(frame.f_back) yielded = False async with self.iter( user_prompt, result_type=result_type, message_history=message_history, model=model, deps=deps, model_settings=model_settings, usage_limits=usage_limits, usage=usage, infer_name=False, ) as agent_run: first_node = agent_run.next_node # start with the first node assert isinstance(first_node, _agent_graph.UserPromptNode) # the first node should be a user prompt node node = first_node while True: if self.is_model_request_node(node): graph_ctx = agent_run.ctx async with node._stream(graph_ctx) as streamed_response: # pyright: ignore[reportPrivateUsage] async def stream_to_final( s: models.StreamedResponse, ) -> FinalResult[models.StreamedResponse] \| None: result_schema = graph_ctx.deps.result_schema async for maybe_part_event in streamed_response: if isinstance(maybe_part_event, _messages.PartStartEvent): new_part = maybe_part_event.part if isinstance(new_part, _messages.TextPart): if _agent_graph.allow_text_result(result_schema): return FinalResult(s, None, None) elif isinstance(new_part, _messages.ToolCallPart) and result_schema: for call, _ in result_schema.find_tool([new_part]): return FinalResult(s, call.tool_name, call.tool_call_id) return None final_result_details = await stream_to_final(streamed_response) if final_result_details is not None: if yielded: raise exceptions.AgentRunError('Agent run produced final results') yielded = True messages = graph_ctx.state.message_history.copy() async def on_complete() -> None: """Called when the stream has completed. The model response will have been added to messages by now by `StreamedRunResult._marked_completed`. """ last_message = messages[-1] assert isinstance(last_message, _messages.ModelResponse) tool_calls = [ part for part in last_message.parts if isinstance(part, _messages.ToolCallPart) ] parts: list[_messages.ModelRequestPart] = [] async for _event in _agent_graph.process_function_tools( tool_calls, final_result_details.tool_name, final_result_details.tool_call_id, graph_ctx, parts, ): pass # TODO: Should we do something here related to the retry count? # Maybe we should move the incrementing of the retry count to where we actually make a request? # if any(isinstance(part, _messages.RetryPromptPart) for part in parts): # ctx.state.increment_retries(ctx.deps.max_result_retries) if parts: messages.append(_messages.ModelRequest(parts)) yield StreamedRunResult( messages, graph_ctx.deps.new_message_index, graph_ctx.deps.usage_limits, streamed_response, graph_ctx.deps.result_schema, _agent_graph.build_run_context(graph_ctx), graph_ctx.deps.result_validators, final_result_details.tool_name, on_complete, ) break next_node = await agent_run.next(node) if not isinstance(next_node, _agent_graph.AgentNode): raise exceptions.AgentRunError('Should have produced a StreamedRunResult before getting here') node = cast(_agent_graph.AgentNode[Any, Any], next_node) if not yielded: raise exceptions.AgentRunError('Agent run finished without producing a final result') @contextmanager def override( self, *, deps: AgentDepsT \| _utils.Unset = _utils.UNSET, model: models.Model \| models.KnownModelName \| str \| _utils.Unset = _utils.UNSET, ) -> Iterator[None]: """Context manager to temporarily override agent dependencies and model. This is particularly useful when testing. You can find an example of this [here](../testing.md#overriding-model-via-pytest-fixtures). Args: deps: The dependencies to use instead of the dependencies passed to the agent run. model: The model to use instead of the model passed to the agent run. """ if _utils.is_set(deps): override_deps_before = self._override_deps self._override_deps = _utils.Some(deps) else: override_deps_before = _utils.UNSET if _utils.is_set(model): override_model_before = self._override_model self._override_model = _utils.Some(models.infer_model(model)) else: override_model_before = _utils.UNSET try: yield finally: if _utils.is_set(override_deps_before): self._override_deps = override_deps_before if _utils.is_set(override_model_before): self._override_model = override_model_before @overload def system_prompt( self, func: Callable[[RunContext[AgentDepsT]], str], / ) -> Callable[[RunContext[AgentDepsT]], str]: ... @overload def system_prompt( self, func: Callable[[RunContext[AgentDepsT]], Awaitable[str]], / ) -> Callable[[RunContext[AgentDepsT]], Awaitable[str]]: ... @overload def system_prompt(self, func: Callable[[], str], /) -> Callable[[], str]: ... @overload def system_prompt(self, func: Callable[[], Awaitable[str]], /) -> Callable[[], Awaitable[str]]: ... @overload def system_prompt( self, /, *, dynamic: bool = False ) -> Callable[[_system_prompt.SystemPromptFunc[AgentDepsT]], _system_prompt.SystemPromptFunc[AgentDepsT]]: ... def system_prompt( self, func: _system_prompt.SystemPromptFunc[AgentDepsT] \| None = None, /, *, dynamic: bool = False, ) -> ( Callable[[_system_prompt.SystemPromptFunc[AgentDepsT]], _system_prompt.SystemPromptFunc[AgentDepsT]] \| _system_prompt.SystemPromptFunc[AgentDepsT] ): """Decorator to register a system prompt function. Optionally takes [`RunContext`][pydantic_ai.tools.RunContext] as its only argument. Can decorate a sync or async functions. The decorator can be used either bare (`agent.system_prompt`) or as a function call (`agent.system_prompt(...)`), see the examples below. Overloads for every possible signature of `system_prompt` are included so the decorator doesn't obscure the type of the function, see `tests/typed_agent.py` for tests. Args: func: The function to decorate dynamic: If True, the system prompt will be reevaluated even when `messages_history` is provided, see [`SystemPromptPart.dynamic_ref`][pydantic_ai.messages.SystemPromptPart.dynamic_ref] Example: ```python from pydantic_ai import Agent, RunContext agent = Agent('test', deps_type=str) @agent.system_prompt def simple_system_prompt() -> str: return 'foobar' @agent.system_prompt(dynamic=True) async def async_system_prompt(ctx: RunContext[str]) -> str: return f'{ctx.deps} is the best' ``` """ if func is None: def decorator( func_: _system_prompt.SystemPromptFunc[AgentDepsT], ) -> _system_prompt.SystemPromptFunc[AgentDepsT]: runner = _system_prompt.SystemPromptRunner[AgentDepsT](func_, dynamic=dynamic) self._system_prompt_functions.append(runner) if dynamic: self._system_prompt_dynamic_functions[func_.__qualname__] = runner return func_ return decorator else: assert not dynamic, "dynamic can't be True in this case" self._system_prompt_functions.append(_system_prompt.SystemPromptRunner[AgentDepsT](func, dynamic=dynamic)) return func @overload def result_validator( self, func: Callable[[RunContext[AgentDepsT], ResultDataT], ResultDataT], / ) -> Callable[[RunContext[AgentDepsT], ResultDataT], ResultDataT]: ... @overload def result_validator( self, func: Callable[[RunContext[AgentDepsT], ResultDataT], Awaitable[ResultDataT]], / ) -> Callable[[RunContext[AgentDepsT], ResultDataT], Awaitable[ResultDataT]]: ... @overload def result_validator( self, func: Callable[[ResultDataT], ResultDataT], / ) -> Callable[[ResultDataT], ResultDataT]: ... @overload def result_validator( self, func: Callable[[ResultDataT], Awaitable[ResultDataT]], / ) -> Callable[[ResultDataT], Awaitable[ResultDataT]]: ... def result_validator( self, func: _result.ResultValidatorFunc[AgentDepsT, ResultDataT], / ) -> _result.ResultValidatorFunc[AgentDepsT, ResultDataT]: """Decorator to register a result validator function. Optionally takes [`RunContext`][pydantic_ai.tools.RunContext] as its first argument. Can decorate a sync or async functions. Overloads for every possible signature of `result_validator` are included so the decorator doesn't obscure the type of the function, see `tests/typed_agent.py` for tests. Example: ```python from pydantic_ai import Agent, ModelRetry, RunContext agent = Agent('test', deps_type=str) @agent.result_validator def result_validator_simple(data: str) -> str: if 'wrong' in data: raise ModelRetry('wrong response') return data @agent.result_validator async def result_validator_deps(ctx: RunContext[str], data: str) -> str: if ctx.deps in data: raise ModelRetry('wrong response') return data result = agent.run_sync('foobar', deps='spam') print(result.data) #> success (no tool calls) ``` """ self._result_validators.append(_result.ResultValidator[AgentDepsT, Any](func)) return func @overload def tool(self, func: ToolFuncContext[AgentDepsT, ToolParams], /) -> ToolFuncContext[AgentDepsT, ToolParams]: ... @overload def tool( self, /, *, name: str \| None = None, retries: int \| None = None, prepare: ToolPrepareFunc[AgentDepsT] \| None = None, docstring_format: DocstringFormat = 'auto', require_parameter_descriptions: bool = False, schema_generator: type[GenerateJsonSchema] = GenerateToolJsonSchema, ) -> Callable[[ToolFuncContext[AgentDepsT, ToolParams]], ToolFuncContext[AgentDepsT, ToolParams]]: ... def tool( self, func: ToolFuncContext[AgentDepsT, ToolParams] \| None = None, /, *, name: str \| None = None, retries: int \| None = None, prepare: ToolPrepareFunc[AgentDepsT] \| None = None, docstring_format: DocstringFormat = 'auto', require_parameter_descriptions: bool = False, schema_generator: type[GenerateJsonSchema] = GenerateToolJsonSchema, ) -> Any: """Decorator to register a tool function which takes [`RunContext`][pydantic_ai.tools.RunContext] as its first argument. Can decorate a sync or async functions. The docstring is inspected to extract both the tool description and description of each parameter, [learn more](../tools.md#function-tools-and-schema). We can't add overloads for every possible signature of tool, since the return type is a recursive union so the signature of functions decorated with `@agent.tool` is obscured. Example: ```python from pydantic_ai import Agent, RunContext agent = Agent('test', deps_type=int) @agent.tool def foobar(ctx: RunContext[int], x: int) -> int: return ctx.deps + x @agent.tool(retries=2) async def spam(ctx: RunContext[str], y: float) -> float: return ctx.deps + y result = agent.run_sync('foobar', deps=1) print(result.data) #> {"foobar":1,"spam":1.0} ``` Args: func: The tool function to register. name: The name of the tool, defaults to the function name. retries: The number of retries to allow for this tool, defaults to the agent's default retries, which defaults to 1. prepare: custom method to prepare the tool definition for each step, return `None` to omit this tool from a given step. This is useful if you want to customise a tool at call time, or omit it completely from a step. See [`ToolPrepareFunc`][pydantic_ai.tools.ToolPrepareFunc]. docstring_format: The format of the docstring, see [`DocstringFormat`][pydantic_ai.tools.DocstringFormat]. Defaults to `'auto'`, such that the format is inferred from the structure of the docstring. require_parameter_descriptions: If True, raise an error if a parameter description is missing. Defaults to False. schema_generator: The JSON schema generator class to use for this tool. Defaults to `GenerateToolJsonSchema`. """ if func is None: def tool_decorator( func_: ToolFuncContext[AgentDepsT, ToolParams], ) -> ToolFuncContext[AgentDepsT, ToolParams]: # noinspection PyTypeChecker self._register_function( func_, True, name, retries, prepare, docstring_format, require_parameter_descriptions, schema_generator, ) return func_ return tool_decorator else: # noinspection PyTypeChecker self._register_function( func, True, name, retries, prepare, docstring_format, require_parameter_descriptions, schema_generator ) return func @overload def tool_plain(self, func: ToolFuncPlain[ToolParams], /) -> ToolFuncPlain[ToolParams]: ... @overload def tool_plain( self, /, *, name: str \| None = None, retries: int \| None = None, prepare: ToolPrepareFunc[AgentDepsT] \| None = None, docstring_format: DocstringFormat = 'auto', require_parameter_descriptions: bool = False, schema_generator: type[GenerateJsonSchema] = GenerateToolJsonSchema, ) -> Callable[[ToolFuncPlain[ToolParams]], ToolFuncPlain[ToolParams]]: ... def tool_plain( self, func: ToolFuncPlain[ToolParams] \| None = None, /, *, name: str \| None = None, retries: int \| None = None, prepare: ToolPrepareFunc[AgentDepsT] \| None = None, docstring_format: DocstringFormat = 'auto', require_parameter_descriptions: bool = False, schema_generator: type[GenerateJsonSchema] = GenerateToolJsonSchema, ) -> Any: """Decorator to register a tool function which DOES NOT take `RunContext` as an argument. Can decorate a sync or async functions. The docstring is inspected to extract both the tool description and description of each parameter, [learn more](../tools.md#function-tools-and-schema). We can't add overloads for every possible signature of tool, since the return type is a recursive union so the signature of functions decorated with `@agent.tool` is obscured. Example: ```python from pydantic_ai import Agent, RunContext agent = Agent('test') @agent.tool def foobar(ctx: RunContext[int]) -> int: return 123 @agent.tool(retries=2) async def spam(ctx: RunContext[str]) -> float: return 3.14 result = agent.run_sync('foobar', deps=1) print(result.data) #> {"foobar":123,"spam":3.14} ``` Args: func: The tool function to register. name: The name of the tool, defaults to the function name. retries: The number of retries to allow for this tool, defaults to the agent's default retries, which defaults to 1. prepare: custom method to prepare the tool definition for each step, return `None` to omit this tool from a given step. This is useful if you want to customise a tool at call time, or omit it completely from a step. See [`ToolPrepareFunc`][pydantic_ai.tools.ToolPrepareFunc]. docstring_format: The format of the docstring, see [`DocstringFormat`][pydantic_ai.tools.DocstringFormat]. Defaults to `'auto'`, such that the format is inferred from the structure of the docstring. require_parameter_descriptions: If True, raise an error if a parameter description is missing. Defaults to False. schema_generator: The JSON schema generator class to use for this tool. Defaults to `GenerateToolJsonSchema`. """ if func is None: def tool_decorator(func_: ToolFuncPlain[ToolParams]) -> ToolFuncPlain[ToolParams]: # noinspection PyTypeChecker self._register_function( func_, False, name, retries, prepare, docstring_format, require_parameter_descriptions, schema_generator, ) return func_ return tool_decorator else: self._register_function( func, False, name, retries, prepare, docstring_format, require_parameter_descriptions, schema_generator ) return func def _register_function( self, func: ToolFuncEither[AgentDepsT, ToolParams], takes_ctx: bool, name: str \| None, retries: int \| None, prepare: ToolPrepareFunc[AgentDepsT] \| None, docstring_format: DocstringFormat, require_parameter_descriptions: bool, schema_generator: type[GenerateJsonSchema], ) -> None: """Private utility to register a function as a tool.""" retries_ = retries if retries is not None else self._default_retries tool = Tool[AgentDepsT]( func, takes_ctx=takes_ctx, name=name, max_retries=retries_, prepare=prepare, docstring_format=docstring_format, require_parameter_descriptions=require_parameter_descriptions, schema_generator=schema_generator, ) self._register_tool(tool) def _register_tool(self, tool: Tool[AgentDepsT]) -> None: """Private utility to register a tool instance.""" if tool.max_retries is None: # noinspection PyTypeChecker tool = dataclasses.replace(tool, max_retries=self._default_retries) if tool.name in self._function_tools: raise exceptions.UserError(f'Tool name conflicts with existing tool: {tool.name!r}') if self._result_schema and tool.name in self._result_schema.tools: raise exceptions.UserError(f'Tool name conflicts with result schema name: {tool.name!r}') self._function_tools[tool.name] = tool def _get_model(self, model: models.Model \| models.KnownModelName \| str \| None) -> models.Model: """Create a model configured for this agent. Args: model: model to use for this run, required if `model` was not set when creating the agent. Returns: The model used """ model_: models.Model if some_model := self._override_model: # we don't want `override()` to cover up errors from the model not being defined, hence this check if model is None and self.model is None: raise exceptions.UserError( '`model` must either be set on the agent or included when calling it. ' '(Even when `override(model=...)` is customizing the model that will actually be called)' ) model_ = some_model.value elif model is not None: model_ = models.infer_model(model) elif self.model is not None: # noinspection PyTypeChecker model_ = self.model = models.infer_model(self.model) else: raise exceptions.UserError('`model` must either be set on the agent or included when calling it.') instrument = self.instrument if instrument is None: instrument = self._instrument_default if instrument and not isinstance(model_, InstrumentedModel): if instrument is True: instrument = InstrumentationSettings() model_ = InstrumentedModel(model_, instrument) return model_ def _get_deps(self: Agent[T, ResultDataT], deps: T) -> T: """Get deps for a run. If we've overridden deps via `_override_deps`, use that, otherwise use the deps passed to the call. We could do runtime type checking of deps against `self._deps_type`, but that's a slippery slope. """ if some_deps := self._override_deps: return some_deps.value else: return deps def _infer_name(self, function_frame: FrameType \| None) -> None: """Infer the agent name from the call frame. Usage should be `self._infer_name(inspect.currentframe())`. """ assert self.name is None, 'Name already set' if function_frame is not None: # pragma: no branch if parent_frame := function_frame.f_back: # pragma: no branch for name, item in parent_frame.f_locals.items(): if item is self: self.name = name return if parent_frame.f_locals != parent_frame.f_globals: # if we couldn't find the agent in locals and globals are a different dict, try globals for name, item in parent_frame.f_globals.items(): if item is self: self.name = name return @property @deprecated( 'The `last_run_messages` attribute has been removed, use `capture_run_messages` instead.', category=None ) def last_run_messages(self) -> list[_messages.ModelMessage]: raise AttributeError('The `last_run_messages` attribute has been removed, use `capture_run_messages` instead.') def _build_graph( self, result_type: type[RunResultDataT] \| None ) -> Graph[_agent_graph.GraphAgentState, _agent_graph.GraphAgentDeps[AgentDepsT, Any], FinalResult[Any]]: return _agent_graph.build_agent_graph(self.name, self._deps_type, result_type or self.result_type) def _prepare_result_schema( self, result_type: type[RunResultDataT] \| None ) -> _result.ResultSchema[RunResultDataT] \| None: if result_type is not None: if self._result_validators: raise exceptions.UserError('Cannot set a custom run `result_type` when the agent has result validators') return _result.ResultSchema[result_type].build( result_type, self._result_tool_name, self._result_tool_description ) else: return self._result_schema # pyright: ignore[reportReturnType] @staticmethod def is_model_request_node( node: _agent_graph.AgentNode[T, S] \| End[result.FinalResult[S]], ) -> TypeGuard[_agent_graph.ModelRequestNode[T, S]]: """Check if the node is a `ModelRequestNode`, narrowing the type if it is. This method preserves the generic parameters while narrowing the type, unlike a direct call to `isinstance`. """ return isinstance(node, _agent_graph.ModelRequestNode) @staticmethod def is_call_tools_node( node: _agent_graph.AgentNode[T, S] \| End[result.FinalResult[S]], ) -> TypeGuard[_agent_graph.CallToolsNode[T, S]]: """Check if the node is a `CallToolsNode`, narrowing the type if it is. This method preserves the generic parameters while narrowing the type, unlike a direct call to `isinstance`. """ return isinstance(node, _agent_graph.CallToolsNode) @staticmethod def is_user_prompt_node( node: _agent_graph.AgentNode[T, S] \| End[result.FinalResult[S]], ) -> TypeGuard[_agent_graph.UserPromptNode[T, S]]: """Check if the node is a `UserPromptNode`, narrowing the type if it is. This method preserves the generic parameters while narrowing the type, unlike a direct call to `isinstance`. """ return isinstance(node, _agent_graph.UserPromptNode) @staticmethod def is_end_node( node: _agent_graph.AgentNode[T, S] \| End[result.FinalResult[S]], ) -> TypeGuard[End[result.FinalResult[S]]]: """Check if the node is a `End`, narrowing the type if it is. This method preserves the generic parameters while narrowing the type, unlike a direct call to `isinstance`. """ return isinstance(node, End) @asynccontextmanager async def run_mcp_servers(self) -> AsyncIterator[None]: """Run [`MCPServerStdio`s][pydantic_ai.mcp.MCPServerStdio] so they can be used by the agent. Returns: a context manager to start and shutdown the servers. """ exit_stack = AsyncExitStack() try: for mcp_server in self._mcp_servers: await exit_stack.enter_async_context(mcp_server) yield finally: await exit_stack.aclose()`` | #### model `instance-attribute` `model: [Model](../models/base/#pydantic_ai.models.Model "pydantic_ai.models.Model") | [KnownModelName](../models/base/#pydantic_ai.models.KnownModelName "pydantic_ai.models.KnownModelName") | [str](https://docs.python.org/3/library/stdtypes.html#str) | None` The default model configured for this agent. We allow str here since the actual list of allowed models changes frequently. #### \_\_init\_\_ `__init__( model: [Model](../models/base/#pydantic_ai.models.Model "pydantic_ai.models.Model") | [KnownModelName](../models/base/#pydantic_ai.models.KnownModelName "pydantic_ai.models.KnownModelName") | [str](https://docs.python.org/3/library/stdtypes.html#str) | None = None, *, result_type: [type](https://docs.python.org/3/library/functions.html#type) [[ResultDataT](../result/#pydantic_ai.result.ResultDataT "pydantic_ai.result.ResultDataT") ] = [str](https://docs.python.org/3/library/stdtypes.html#str) , system_prompt: [str](https://docs.python.org/3/library/stdtypes.html#str) | [Sequence](https://docs.python.org/3/library/collections.abc.html#collections.abc.Sequence "collections.abc.Sequence") [[str](https://docs.python.org/3/library/stdtypes.html#str) ] = (), deps_type: [type](https://docs.python.org/3/library/functions.html#type) [[AgentDepsT](../tools/#pydantic_ai.tools.AgentDepsT "pydantic_ai.tools.AgentDepsT") ] = NoneType, name: [str](https://docs.python.org/3/library/stdtypes.html#str) | None = None, model_settings: [ModelSettings](../settings/#pydantic_ai.settings.ModelSettings "pydantic_ai.settings.ModelSettings") | None = None, retries: [int](https://docs.python.org/3/library/functions.html#int) = 1, result_tool_name: [str](https://docs.python.org/3/library/stdtypes.html#str) = "final_result", result_tool_description: [str](https://docs.python.org/3/library/stdtypes.html#str) | None = None, result_retries: [int](https://docs.python.org/3/library/functions.html#int) | None = None, tools: [Sequence](https://docs.python.org/3/library/collections.abc.html#collections.abc.Sequence "collections.abc.Sequence") [ [Tool](../tools/#pydantic_ai.tools.Tool "pydantic_ai.tools.Tool") [[AgentDepsT](../tools/#pydantic_ai.tools.AgentDepsT "pydantic_ai.tools.AgentDepsT") ] | [ToolFuncEither](../tools/#pydantic_ai.tools.ToolFuncEither "pydantic_ai.tools.ToolFuncEither") [[AgentDepsT](../tools/#pydantic_ai.tools.AgentDepsT "pydantic_ai.tools.AgentDepsT") , ...] ] = (), mcp_servers: [Sequence](https://docs.python.org/3/library/collections.abc.html#collections.abc.Sequence "collections.abc.Sequence") [[MCPServer](../mcp/#pydantic_ai.mcp.MCPServer "pydantic_ai.mcp.MCPServer") ] = (), defer_model_check: [bool](https://docs.python.org/3/library/functions.html#bool) = False, end_strategy: [EndStrategy](#pydantic_ai.agent.EndStrategy "pydantic_ai.agent.EndStrategy") = "early", instrument: [InstrumentationSettings](../models/instrumented/#pydantic_ai.models.instrumented.InstrumentationSettings "pydantic_ai.models.instrumented.InstrumentationSettings") | [bool](https://docs.python.org/3/library/functions.html#bool) | None = None )` Create an agent. Parameters: | Name | Type | Description | Default | | --- | --- | --- | --- | | `model` | `[Model](../models/base/#pydantic_ai.models.Model "pydantic_ai.models.Model") \| [KnownModelName](../models/base/#pydantic_ai.models.KnownModelName "pydantic_ai.models.KnownModelName") \| [str](https://docs.python.org/3/library/stdtypes.html#str) \| None` | The default model to use for this agent, if not provide, you must provide the model when calling it. We allow str here since the actual list of allowed models changes frequently. | `None` | | `result_type` | `[type](https://docs.python.org/3/library/functions.html#type) [[ResultDataT](../result/#pydantic_ai.result.ResultDataT "pydantic_ai.result.ResultDataT") ]` | The type of the result data, used to validate the result data, defaults to `str`. | `[str](https://docs.python.org/3/library/stdtypes.html#str) ` | | `system_prompt` | `[str](https://docs.python.org/3/library/stdtypes.html#str) \| [Sequence](https://docs.python.org/3/library/collections.abc.html#collections.abc.Sequence "collections.abc.Sequence") [[str](https://docs.python.org/3/library/stdtypes.html#str) ]` | Static system prompts to use for this agent, you can also register system prompts via a function with [`system_prompt`](#pydantic_ai.agent.Agent.system_prompt)
. | `()` | | `deps_type` | `[type](https://docs.python.org/3/library/functions.html#type) [[AgentDepsT](../tools/#pydantic_ai.tools.AgentDepsT "pydantic_ai.tools.AgentDepsT") ]` | The type used for dependency injection, this parameter exists solely to allow you to fully parameterize the agent, and therefore get the best out of static type checking. If you're not using deps, but want type checking to pass, you can set `deps=None` to satisfy Pyright or add a type hint `: Agent[None, ]`. | `NoneType` | | `name` | `[str](https://docs.python.org/3/library/stdtypes.html#str) \| None` | The name of the agent, used for logging. If `None`, we try to infer the agent name from the call frame when the agent is first run. | `None` | | `model_settings` | `[ModelSettings](../settings/#pydantic_ai.settings.ModelSettings "pydantic_ai.settings.ModelSettings") \| None` | Optional model request settings to use for this agent's runs, by default. | `None` | | `retries` | `[int](https://docs.python.org/3/library/functions.html#int) ` | The default number of retries to allow before raising an error. | `1` | | `result_tool_name` | `[str](https://docs.python.org/3/library/stdtypes.html#str) ` | The name of the tool to use for the final result. | `'final_result'` | | `result_tool_description` | `[str](https://docs.python.org/3/library/stdtypes.html#str) \| None` | The description of the final result tool. | `None` | | `result_retries` | `[int](https://docs.python.org/3/library/functions.html#int) \| None` | The maximum number of retries to allow for result validation, defaults to `retries`. | `None` | | `tools` | `[Sequence](https://docs.python.org/3/library/collections.abc.html#collections.abc.Sequence "collections.abc.Sequence") [[Tool](../tools/#pydantic_ai.tools.Tool "pydantic_ai.tools.Tool") [[AgentDepsT](../tools/#pydantic_ai.tools.AgentDepsT "pydantic_ai.tools.AgentDepsT") ] \| [ToolFuncEither](../tools/#pydantic_ai.tools.ToolFuncEither "pydantic_ai.tools.ToolFuncEither") [[AgentDepsT](../tools/#pydantic_ai.tools.AgentDepsT "pydantic_ai.tools.AgentDepsT") , ...]]` | Tools to register with the agent, you can also register tools via the decorators [`@agent.tool`](#pydantic_ai.agent.Agent.tool)
and [`@agent.tool_plain`](#pydantic_ai.agent.Agent.tool_plain)
. | `()` | | `mcp_servers` | `[Sequence](https://docs.python.org/3/library/collections.abc.html#collections.abc.Sequence "collections.abc.Sequence") [[MCPServer](../mcp/#pydantic_ai.mcp.MCPServer "pydantic_ai.mcp.MCPServer") ]` | MCP servers to register with the agent. You should register a [`MCPServer`](../mcp/#pydantic_ai.mcp.MCPServer)
for each server you want the agent to connect to. | `()` | | `defer_model_check` | `[bool](https://docs.python.org/3/library/functions.html#bool) ` | by default, if you provide a [named](../models/base/#pydantic_ai.models.KnownModelName)
model, it's evaluated to create a [`Model`](../models/base/#pydantic_ai.models.Model)
instance immediately, which checks for the necessary environment variables. Set this to `false` to defer the evaluation until the first run. Useful if you want to [override the model](#pydantic_ai.agent.Agent.override)
for testing. | `False` | | `end_strategy` | `[EndStrategy](#pydantic_ai.agent.EndStrategy "pydantic_ai.agent.EndStrategy") ` | Strategy for handling tool calls that are requested alongside a final result. See [`EndStrategy`](#pydantic_ai.agent.EndStrategy)
for more information. | `'early'` | | `instrument` | `[InstrumentationSettings](../models/instrumented/#pydantic_ai.models.instrumented.InstrumentationSettings "pydantic_ai.models.instrumented.InstrumentationSettings") \| [bool](https://docs.python.org/3/library/functions.html#bool) \| None` | Set to True to automatically instrument with OpenTelemetry, which will use Logfire if it's configured. Set to an instance of [`InstrumentationSettings`](#pydantic_ai.agent.InstrumentationSettings)
to customize. If this isn't set, then the last value set by [`Agent.instrument_all()`](#pydantic_ai.agent.Agent.instrument_all)
will be used, which defaults to False. See the [Debugging and Monitoring guide](https://ai.pydantic.dev/logfire/)
for more info. | `None` | Source code in `pydantic_ai_slim/pydantic_ai/agent.py` | | | | --- | --- | | 145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235 | ``def __init__( self, model: models.Model \| models.KnownModelName \| str \| None = None, *, result_type: type[ResultDataT] = str, system_prompt: str \| Sequence[str] = (), deps_type: type[AgentDepsT] = NoneType, name: str \| None = None, model_settings: ModelSettings \| None = None, retries: int = 1, result_tool_name: str = 'final_result', result_tool_description: str \| None = None, result_retries: int \| None = None, tools: Sequence[Tool[AgentDepsT] \| ToolFuncEither[AgentDepsT, ...]] = (), mcp_servers: Sequence[MCPServer] = (), defer_model_check: bool = False, end_strategy: EndStrategy = 'early', instrument: InstrumentationSettings \| bool \| None = None, ): """Create an agent. Args: model: The default model to use for this agent, if not provide, you must provide the model when calling it. We allow str here since the actual list of allowed models changes frequently. result_type: The type of the result data, used to validate the result data, defaults to `str`. system_prompt: Static system prompts to use for this agent, you can also register system prompts via a function with [`system_prompt`][pydantic_ai.Agent.system_prompt]. deps_type: The type used for dependency injection, this parameter exists solely to allow you to fully parameterize the agent, and therefore get the best out of static type checking. If you're not using deps, but want type checking to pass, you can set `deps=None` to satisfy Pyright or add a type hint `: Agent[None, ]`. name: The name of the agent, used for logging. If `None`, we try to infer the agent name from the call frame when the agent is first run. model_settings: Optional model request settings to use for this agent's runs, by default. retries: The default number of retries to allow before raising an error. result_tool_name: The name of the tool to use for the final result. result_tool_description: The description of the final result tool. result_retries: The maximum number of retries to allow for result validation, defaults to `retries`. tools: Tools to register with the agent, you can also register tools via the decorators [`@agent.tool`][pydantic_ai.Agent.tool] and [`@agent.tool_plain`][pydantic_ai.Agent.tool_plain]. mcp_servers: MCP servers to register with the agent. You should register a [`MCPServer`][pydantic_ai.mcp.MCPServer] for each server you want the agent to connect to. defer_model_check: by default, if you provide a [named][pydantic_ai.models.KnownModelName] model, it's evaluated to create a [`Model`][pydantic_ai.models.Model] instance immediately, which checks for the necessary environment variables. Set this to `false` to defer the evaluation until the first run. Useful if you want to [override the model][pydantic_ai.Agent.override] for testing. end_strategy: Strategy for handling tool calls that are requested alongside a final result. See [`EndStrategy`][pydantic_ai.agent.EndStrategy] for more information. instrument: Set to True to automatically instrument with OpenTelemetry, which will use Logfire if it's configured. Set to an instance of [`InstrumentationSettings`][pydantic_ai.agent.InstrumentationSettings] to customize. If this isn't set, then the last value set by [`Agent.instrument_all()`][pydantic_ai.Agent.instrument_all] will be used, which defaults to False. See the [Debugging and Monitoring guide](https://ai.pydantic.dev/logfire/) for more info. """ if model is None or defer_model_check: self.model = model else: self.model = models.infer_model(model) self.end_strategy = end_strategy self.name = name self.model_settings = model_settings self.result_type = result_type self.instrument = instrument self._deps_type = deps_type self._result_tool_name = result_tool_name self._result_tool_description = result_tool_description self._result_schema = _result.ResultSchema[result_type].build( result_type, result_tool_name, result_tool_description ) self._result_validators = [] self._system_prompts = (system_prompt,) if isinstance(system_prompt, str) else tuple(system_prompt) self._system_prompt_functions = [] self._system_prompt_dynamic_functions = {} self._function_tools = {} self._default_retries = retries self._max_result_retries = result_retries if result_retries is not None else retries self._mcp_servers = mcp_servers for tool in tools: if isinstance(tool, Tool): self._register_tool(tool) else: self._register_tool(Tool(tool))`` | #### end\_strategy `instance-attribute` `end_strategy: [EndStrategy](#pydantic_ai.agent.EndStrategy "pydantic_ai.agent.EndStrategy") = end_strategy` Strategy for handling tool calls when a final result is found. #### name `instance-attribute` `name: [str](https://docs.python.org/3/library/stdtypes.html#str) | None = name` The name of the agent, used for logging. If `None`, we try to infer the agent name from the call frame when the agent is first run. #### model\_settings `instance-attribute` `model_settings: [ModelSettings](../settings/#pydantic_ai.settings.ModelSettings "pydantic_ai.settings.ModelSettings") | None = model_settings` Optional model request settings to use for this agents's runs, by default. Note, if `model_settings` is provided by `run`, `run_sync`, or `run_stream`, those settings will be merged with this value, with the runtime argument taking priority. #### result\_type `class-attribute` `instance-attribute` `result_type: [type](https://docs.python.org/3/library/functions.html#type) [[ResultDataT](../result/#pydantic_ai.result.ResultDataT "pydantic_ai.result.ResultDataT") ] = result_type` The type of the result data, used to validate the result data, defaults to `str`. #### instrument `instance-attribute` `instrument: [InstrumentationSettings](../models/instrumented/#pydantic_ai.models.instrumented.InstrumentationSettings "pydantic_ai.models.instrumented.InstrumentationSettings") | [bool](https://docs.python.org/3/library/functions.html#bool) | None = ( instrument )` Options to automatically instrument with OpenTelemetry. #### instrument\_all `staticmethod` `instrument_all( instrument: [InstrumentationSettings](../models/instrumented/#pydantic_ai.models.instrumented.InstrumentationSettings "pydantic_ai.models.instrumented.InstrumentationSettings") | [bool](https://docs.python.org/3/library/functions.html#bool) = True, ) -> None` Set the instrumentation options for all agents where `instrument` is not set. Source code in `pydantic_ai_slim/pydantic_ai/agent.py` | | | | --- | --- | | 237
238
239
240 | ``@staticmethod def instrument_all(instrument: InstrumentationSettings \| bool = True) -> None: """Set the instrumentation options for all agents where `instrument` is not set.""" Agent._instrument_default = instrument`` | #### run `async` `run( user_prompt: [str](https://docs.python.org/3/library/stdtypes.html#str) | [Sequence](https://docs.python.org/3/library/collections.abc.html#collections.abc.Sequence "collections.abc.Sequence") [UserContent], *, result_type: None = None, message_history: [list](https://docs.python.org/3/library/stdtypes.html#list) [[ModelMessage](../messages/#pydantic_ai.messages.ModelMessage "pydantic_ai.messages.ModelMessage") ] | None = None, model: [Model](../models/base/#pydantic_ai.models.Model "pydantic_ai.models.Model") | [KnownModelName](../models/base/#pydantic_ai.models.KnownModelName "pydantic_ai.models.KnownModelName") | [str](https://docs.python.org/3/library/stdtypes.html#str) | None = None, deps: [AgentDepsT](../tools/#pydantic_ai.tools.AgentDepsT "pydantic_ai.tools.AgentDepsT") = None, model_settings: [ModelSettings](../settings/#pydantic_ai.settings.ModelSettings "pydantic_ai.settings.ModelSettings") | None = None, usage_limits: [UsageLimits](../usage/#pydantic_ai.usage.UsageLimits "pydantic_ai.usage.UsageLimits") | None = None, usage: [Usage](../usage/#pydantic_ai.usage.Usage "pydantic_ai.usage.Usage") | None = None, infer_name: [bool](https://docs.python.org/3/library/functions.html#bool) = True ) -> [AgentRunResult](#pydantic_ai.agent.AgentRunResult "pydantic_ai.agent.AgentRunResult") [[ResultDataT](../result/#pydantic_ai.result.ResultDataT "pydantic_ai.result.ResultDataT") ]` `run( user_prompt: [str](https://docs.python.org/3/library/stdtypes.html#str) | [Sequence](https://docs.python.org/3/library/collections.abc.html#collections.abc.Sequence "collections.abc.Sequence") [UserContent], *, result_type: [type](https://docs.python.org/3/library/functions.html#type) [[RunResultDataT](#pydantic_ai.agent.RunResultDataT "pydantic_ai.agent.RunResultDataT") ], message_history: [list](https://docs.python.org/3/library/stdtypes.html#list) [[ModelMessage](../messages/#pydantic_ai.messages.ModelMessage "pydantic_ai.messages.ModelMessage") ] | None = None, model: [Model](../models/base/#pydantic_ai.models.Model "pydantic_ai.models.Model") | [KnownModelName](../models/base/#pydantic_ai.models.KnownModelName "pydantic_ai.models.KnownModelName") | [str](https://docs.python.org/3/library/stdtypes.html#str) | None = None, deps: [AgentDepsT](../tools/#pydantic_ai.tools.AgentDepsT "pydantic_ai.tools.AgentDepsT") = None, model_settings: [ModelSettings](../settings/#pydantic_ai.settings.ModelSettings "pydantic_ai.settings.ModelSettings") | None = None, usage_limits: [UsageLimits](../usage/#pydantic_ai.usage.UsageLimits "pydantic_ai.usage.UsageLimits") | None = None, usage: [Usage](../usage/#pydantic_ai.usage.Usage "pydantic_ai.usage.Usage") | None = None, infer_name: [bool](https://docs.python.org/3/library/functions.html#bool) = True ) -> [AgentRunResult](#pydantic_ai.agent.AgentRunResult "pydantic_ai.agent.AgentRunResult") [[RunResultDataT](#pydantic_ai.agent.RunResultDataT "pydantic_ai.agent.RunResultDataT") ]` `run( user_prompt: [str](https://docs.python.org/3/library/stdtypes.html#str) | [Sequence](https://docs.python.org/3/library/collections.abc.html#collections.abc.Sequence "collections.abc.Sequence") [UserContent], *, result_type: [type](https://docs.python.org/3/library/functions.html#type) [[RunResultDataT](#pydantic_ai.agent.RunResultDataT "pydantic_ai.agent.RunResultDataT") ] | None = None, message_history: [list](https://docs.python.org/3/library/stdtypes.html#list) [[ModelMessage](../messages/#pydantic_ai.messages.ModelMessage "pydantic_ai.messages.ModelMessage") ] | None = None, model: [Model](../models/base/#pydantic_ai.models.Model "pydantic_ai.models.Model") | [KnownModelName](../models/base/#pydantic_ai.models.KnownModelName "pydantic_ai.models.KnownModelName") | [str](https://docs.python.org/3/library/stdtypes.html#str) | None = None, deps: [AgentDepsT](../tools/#pydantic_ai.tools.AgentDepsT "pydantic_ai.tools.AgentDepsT") = None, model_settings: [ModelSettings](../settings/#pydantic_ai.settings.ModelSettings "pydantic_ai.settings.ModelSettings") | None = None, usage_limits: [UsageLimits](../usage/#pydantic_ai.usage.UsageLimits "pydantic_ai.usage.UsageLimits") | None = None, usage: [Usage](../usage/#pydantic_ai.usage.Usage "pydantic_ai.usage.Usage") | None = None, infer_name: [bool](https://docs.python.org/3/library/functions.html#bool) = True ) -> [AgentRunResult](#pydantic_ai.agent.AgentRunResult "pydantic_ai.agent.AgentRunResult") [[Any](https://docs.python.org/3/library/typing.html#typing.Any "typing.Any") ]` Run the agent with a user prompt in async mode. This method builds an internal agent graph (using system prompts, tools and result schemas) and then runs the graph to completion. The result of the run is returned. Example: `from pydantic_ai import Agent agent = Agent('openai:gpt-4o') async def main(): agent_run = await agent.run('What is the capital of France?') print(agent_run.data) #> Paris` Parameters: | Name | Type | Description | Default | | --- | --- | --- | --- | | `user_prompt` | `[str](https://docs.python.org/3/library/stdtypes.html#str) \| [Sequence](https://docs.python.org/3/library/collections.abc.html#collections.abc.Sequence "collections.abc.Sequence") [UserContent]` | User input to start/continue the conversation. | _required_ | | `result_type` | `[type](https://docs.python.org/3/library/functions.html#type) [[RunResultDataT](#pydantic_ai.agent.RunResultDataT "pydantic_ai.agent.RunResultDataT") ] \| None` | Custom result type to use for this run, `result_type` may only be used if the agent has no result validators since result validators would expect an argument that matches the agent's result type. | `None` | | `message_history` | `[list](https://docs.python.org/3/library/stdtypes.html#list) [[ModelMessage](../messages/#pydantic_ai.messages.ModelMessage "pydantic_ai.messages.ModelMessage") ] \| None` | History of the conversation so far. | `None` | | `model` | `[Model](../models/base/#pydantic_ai.models.Model "pydantic_ai.models.Model") \| [KnownModelName](../models/base/#pydantic_ai.models.KnownModelName "pydantic_ai.models.KnownModelName") \| [str](https://docs.python.org/3/library/stdtypes.html#str) \| None` | Optional model to use for this run, required if `model` was not set when creating the agent. | `None` | | `deps` | `[AgentDepsT](../tools/#pydantic_ai.tools.AgentDepsT "pydantic_ai.tools.AgentDepsT") ` | Optional dependencies to use for this run. | `None` | | `model_settings` | `[ModelSettings](../settings/#pydantic_ai.settings.ModelSettings "pydantic_ai.settings.ModelSettings") \| None` | Optional settings to use for this model's request. | `None` | | `usage_limits` | `[UsageLimits](../usage/#pydantic_ai.usage.UsageLimits "pydantic_ai.usage.UsageLimits") \| None` | Optional limits on model request count or token usage. | `None` | | `usage` | `[Usage](../usage/#pydantic_ai.usage.Usage "pydantic_ai.usage.Usage") \| None` | Optional usage to start with, useful for resuming a conversation or agents used in tools. | `None` | | `infer_name` | `[bool](https://docs.python.org/3/library/functions.html#bool) ` | Whether to try to infer the agent name from the call frame if it's not set. | `True` | Returns: | Type | Description | | --- | --- | | `[AgentRunResult](#pydantic_ai.agent.AgentRunResult "pydantic_ai.agent.AgentRunResult") [[Any](https://docs.python.org/3/library/typing.html#typing.Any "typing.Any") ]` | The result of the run. | Source code in `pydantic_ai_slim/pydantic_ai/agent.py` | | | | --- | --- | | 272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333 | ``async def run( self, user_prompt: str \| Sequence[_messages.UserContent], *, result_type: type[RunResultDataT] \| None = None, message_history: list[_messages.ModelMessage] \| None = None, model: models.Model \| models.KnownModelName \| str \| None = None, deps: AgentDepsT = None, model_settings: ModelSettings \| None = None, usage_limits: _usage.UsageLimits \| None = None, usage: _usage.Usage \| None = None, infer_name: bool = True, ) -> AgentRunResult[Any]: """Run the agent with a user prompt in async mode. This method builds an internal agent graph (using system prompts, tools and result schemas) and then runs the graph to completion. The result of the run is returned. Example: ```python from pydantic_ai import Agent agent = Agent('openai:gpt-4o') async def main(): agent_run = await agent.run('What is the capital of France?') print(agent_run.data) #> Paris ``` Args: user_prompt: User input to start/continue the conversation. result_type: Custom result type to use for this run, `result_type` may only be used if the agent has no result validators since result validators would expect an argument that matches the agent's result type. message_history: History of the conversation so far. model: Optional model to use for this run, required if `model` was not set when creating the agent. deps: Optional dependencies to use for this run. model_settings: Optional settings to use for this model's request. usage_limits: Optional limits on model request count or token usage. usage: Optional usage to start with, useful for resuming a conversation or agents used in tools. infer_name: Whether to try to infer the agent name from the call frame if it's not set. Returns: The result of the run. """ if infer_name and self.name is None: self._infer_name(inspect.currentframe()) async with self.iter( user_prompt=user_prompt, result_type=result_type, message_history=message_history, model=model, deps=deps, model_settings=model_settings, usage_limits=usage_limits, usage=usage, ) as agent_run: async for _ in agent_run: pass assert agent_run.result is not None, 'The graph run did not finish properly' return agent_run.result`` | #### iter `async` `iter( user_prompt: [str](https://docs.python.org/3/library/stdtypes.html#str) | [Sequence](https://docs.python.org/3/library/collections.abc.html#collections.abc.Sequence "collections.abc.Sequence") [UserContent], *, result_type: [type](https://docs.python.org/3/library/functions.html#type) [[RunResultDataT](#pydantic_ai.agent.RunResultDataT "pydantic_ai.agent.RunResultDataT") ] | None = None, message_history: [list](https://docs.python.org/3/library/stdtypes.html#list) [[ModelMessage](../messages/#pydantic_ai.messages.ModelMessage "pydantic_ai.messages.ModelMessage") ] | None = None, model: [Model](../models/base/#pydantic_ai.models.Model "pydantic_ai.models.Model") | [KnownModelName](../models/base/#pydantic_ai.models.KnownModelName "pydantic_ai.models.KnownModelName") | [str](https://docs.python.org/3/library/stdtypes.html#str) | None = None, deps: [AgentDepsT](../tools/#pydantic_ai.tools.AgentDepsT "pydantic_ai.tools.AgentDepsT") = None, model_settings: [ModelSettings](../settings/#pydantic_ai.settings.ModelSettings "pydantic_ai.settings.ModelSettings") | None = None, usage_limits: [UsageLimits](../usage/#pydantic_ai.usage.UsageLimits "pydantic_ai.usage.UsageLimits") | None = None, usage: [Usage](../usage/#pydantic_ai.usage.Usage "pydantic_ai.usage.Usage") | None = None, infer_name: [bool](https://docs.python.org/3/library/functions.html#bool) = True ) -> [AsyncIterator](https://docs.python.org/3/library/collections.abc.html#collections.abc.AsyncIterator "collections.abc.AsyncIterator") [[AgentRun](#pydantic_ai.agent.AgentRun "pydantic_ai.agent.AgentRun") [[AgentDepsT](../tools/#pydantic_ai.tools.AgentDepsT "pydantic_ai.tools.AgentDepsT") , [Any](https://docs.python.org/3/library/typing.html#typing.Any "typing.Any") ]]` A contextmanager which can be used to iterate over the agent graph's nodes as they are executed. This method builds an internal agent graph (using system prompts, tools and result schemas) and then returns an `AgentRun` object. The `AgentRun` can be used to async-iterate over the nodes of the graph as they are executed. This is the API to use if you want to consume the outputs coming from each LLM model response, or the stream of events coming from the execution of tools. The `AgentRun` also provides methods to access the full message history, new messages, and usage statistics, and the final result of the run once it has completed. For more details, see the documentation of `AgentRun`. Example: `from pydantic_ai import Agent agent = Agent('openai:gpt-4o') async def main(): nodes = [] async with agent.iter('What is the capital of France?') as agent_run: async for node in agent_run: nodes.append(node) print(nodes) ''' [ ModelRequestNode( request=ModelRequest( parts=[ UserPromptPart( content='What is the capital of France?', timestamp=datetime.datetime(...), part_kind='user-prompt', ) ], kind='request', ) ), CallToolsNode( model_response=ModelResponse( parts=[TextPart(content='Paris', part_kind='text')], model_name='gpt-4o', timestamp=datetime.datetime(...), kind='response', ) ), End(data=FinalResult(data='Paris', tool_name=None, tool_call_id=None)), ] ''' print(agent_run.result.data) #> Paris` Parameters: | Name | Type | Description | Default | | --- | --- | --- | --- | | `user_prompt` | `[str](https://docs.python.org/3/library/stdtypes.html#str) \| [Sequence](https://docs.python.org/3/library/collections.abc.html#collections.abc.Sequence "collections.abc.Sequence") [UserContent]` | User input to start/continue the conversation. | _required_ | | `result_type` | `[type](https://docs.python.org/3/library/functions.html#type) [[RunResultDataT](#pydantic_ai.agent.RunResultDataT "pydantic_ai.agent.RunResultDataT") ] \| None` | Custom result type to use for this run, `result_type` may only be used if the agent has no result validators since result validators would expect an argument that matches the agent's result type. | `None` | | `message_history` | `[list](https://docs.python.org/3/library/stdtypes.html#list) [[ModelMessage](../messages/#pydantic_ai.messages.ModelMessage "pydantic_ai.messages.ModelMessage") ] \| None` | History of the conversation so far. | `None` | | `model` | `[Model](../models/base/#pydantic_ai.models.Model "pydantic_ai.models.Model") \| [KnownModelName](../models/base/#pydantic_ai.models.KnownModelName "pydantic_ai.models.KnownModelName") \| [str](https://docs.python.org/3/library/stdtypes.html#str) \| None` | Optional model to use for this run, required if `model` was not set when creating the agent. | `None` | | `deps` | `[AgentDepsT](../tools/#pydantic_ai.tools.AgentDepsT "pydantic_ai.tools.AgentDepsT") ` | Optional dependencies to use for this run. | `None` | | `model_settings` | `[ModelSettings](../settings/#pydantic_ai.settings.ModelSettings "pydantic_ai.settings.ModelSettings") \| None` | Optional settings to use for this model's request. | `None` | | `usage_limits` | `[UsageLimits](../usage/#pydantic_ai.usage.UsageLimits "pydantic_ai.usage.UsageLimits") \| None` | Optional limits on model request count or token usage. | `None` | | `usage` | `[Usage](../usage/#pydantic_ai.usage.Usage "pydantic_ai.usage.Usage") \| None` | Optional usage to start with, useful for resuming a conversation or agents used in tools. | `None` | | `infer_name` | `[bool](https://docs.python.org/3/library/functions.html#bool) ` | Whether to try to infer the agent name from the call frame if it's not set. | `True` | Returns: | Type | Description | | --- | --- | | `[AsyncIterator](https://docs.python.org/3/library/collections.abc.html#collections.abc.AsyncIterator "collections.abc.AsyncIterator") [[AgentRun](#pydantic_ai.agent.AgentRun "pydantic_ai.agent.AgentRun") [[AgentDepsT](../tools/#pydantic_ai.tools.AgentDepsT "pydantic_ai.tools.AgentDepsT") , [Any](https://docs.python.org/3/library/typing.html#typing.Any "typing.Any") ]]` | The result of the run. | Source code in `pydantic_ai_slim/pydantic_ai/agent.py` | | | | --- | --- | | 335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495 | ``@asynccontextmanager async def iter( self, user_prompt: str \| Sequence[_messages.UserContent], *, result_type: type[RunResultDataT] \| None = None, message_history: list[_messages.ModelMessage] \| None = None, model: models.Model \| models.KnownModelName \| str \| None = None, deps: AgentDepsT = None, model_settings: ModelSettings \| None = None, usage_limits: _usage.UsageLimits \| None = None, usage: _usage.Usage \| None = None, infer_name: bool = True, ) -> AsyncIterator[AgentRun[AgentDepsT, Any]]: """A contextmanager which can be used to iterate over the agent graph's nodes as they are executed. This method builds an internal agent graph (using system prompts, tools and result schemas) and then returns an `AgentRun` object. The `AgentRun` can be used to async-iterate over the nodes of the graph as they are executed. This is the API to use if you want to consume the outputs coming from each LLM model response, or the stream of events coming from the execution of tools. The `AgentRun` also provides methods to access the full message history, new messages, and usage statistics, and the final result of the run once it has completed. For more details, see the documentation of `AgentRun`. Example: ```python from pydantic_ai import Agent agent = Agent('openai:gpt-4o') async def main(): nodes = [] async with agent.iter('What is the capital of France?') as agent_run: async for node in agent_run: nodes.append(node) print(nodes) ''' [ ModelRequestNode( request=ModelRequest( parts=[ UserPromptPart( content='What is the capital of France?', timestamp=datetime.datetime(...), part_kind='user-prompt', ) ], kind='request', ) ), CallToolsNode( model_response=ModelResponse( parts=[TextPart(content='Paris', part_kind='text')], model_name='gpt-4o', timestamp=datetime.datetime(...), kind='response', ) ), End(data=FinalResult(data='Paris', tool_name=None, tool_call_id=None)), ] ''' print(agent_run.result.data) #> Paris ``` Args: user_prompt: User input to start/continue the conversation. result_type: Custom result type to use for this run, `result_type` may only be used if the agent has no result validators since result validators would expect an argument that matches the agent's result type. message_history: History of the conversation so far. model: Optional model to use for this run, required if `model` was not set when creating the agent. deps: Optional dependencies to use for this run. model_settings: Optional settings to use for this model's request. usage_limits: Optional limits on model request count or token usage. usage: Optional usage to start with, useful for resuming a conversation or agents used in tools. infer_name: Whether to try to infer the agent name from the call frame if it's not set. Returns: The result of the run. """ if infer_name and self.name is None: self._infer_name(inspect.currentframe()) model_used = self._get_model(model) del model deps = self._get_deps(deps) new_message_index = len(message_history) if message_history else 0 result_schema: _result.ResultSchema[RunResultDataT] \| None = self._prepare_result_schema(result_type) # Build the graph graph = self._build_graph(result_type) # Build the initial state state = _agent_graph.GraphAgentState( message_history=message_history[:] if message_history else [], usage=usage or _usage.Usage(), retries=0, run_step=0, ) # We consider it a user error if a user tries to restrict the result type while having a result validator that # may change the result type from the restricted type to something else. Therefore, we consider the following # typecast reasonable, even though it is possible to violate it with otherwise-type-checked code. result_validators = cast(list[_result.ResultValidator[AgentDepsT, RunResultDataT]], self._result_validators) # TODO: Instead of this, copy the function tools to ensure they don't share current_retry state between agent # runs. Requires some changes to `Tool` to make them copyable though. for v in self._function_tools.values(): v.current_retry = 0 model_settings = merge_model_settings(self.model_settings, model_settings) usage_limits = usage_limits or _usage.UsageLimits() if isinstance(model_used, InstrumentedModel): tracer = model_used.settings.tracer else: tracer = NoOpTracer() agent_name = self.name or 'agent' run_span = tracer.start_span( 'agent run', attributes={ 'model_name': model_used.model_name if model_used else 'no-model', 'agent_name': agent_name, 'logfire.msg': f'{agent_name} run', }, ) graph_deps = _agent_graph.GraphAgentDeps[AgentDepsT, RunResultDataT]( user_deps=deps, prompt=user_prompt, new_message_index=new_message_index, model=model_used, model_settings=model_settings, usage_limits=usage_limits, max_result_retries=self._max_result_retries, end_strategy=self.end_strategy, result_schema=result_schema, result_tools=self._result_schema.tool_defs() if self._result_schema else [], result_validators=result_validators, function_tools=self._function_tools, mcp_servers=self._mcp_servers, run_span=run_span, tracer=tracer, ) start_node = _agent_graph.UserPromptNode[AgentDepsT]( user_prompt=user_prompt, system_prompts=self._system_prompts, system_prompt_functions=self._system_prompt_functions, system_prompt_dynamic_functions=self._system_prompt_dynamic_functions, ) async with graph.iter( start_node, state=state, deps=graph_deps, span=use_span(run_span, end_on_exit=True), infer_name=False, ) as graph_run: yield AgentRun(graph_run)`` | #### run\_sync `run_sync( user_prompt: [str](https://docs.python.org/3/library/stdtypes.html#str) | [Sequence](https://docs.python.org/3/library/collections.abc.html#collections.abc.Sequence "collections.abc.Sequence") [UserContent], *, message_history: [list](https://docs.python.org/3/library/stdtypes.html#list) [[ModelMessage](../messages/#pydantic_ai.messages.ModelMessage "pydantic_ai.messages.ModelMessage") ] | None = None, model: [Model](../models/base/#pydantic_ai.models.Model "pydantic_ai.models.Model") | [KnownModelName](../models/base/#pydantic_ai.models.KnownModelName "pydantic_ai.models.KnownModelName") | [str](https://docs.python.org/3/library/stdtypes.html#str) | None = None, deps: [AgentDepsT](../tools/#pydantic_ai.tools.AgentDepsT "pydantic_ai.tools.AgentDepsT") = None, model_settings: [ModelSettings](../settings/#pydantic_ai.settings.ModelSettings "pydantic_ai.settings.ModelSettings") | None = None, usage_limits: [UsageLimits](../usage/#pydantic_ai.usage.UsageLimits "pydantic_ai.usage.UsageLimits") | None = None, usage: [Usage](../usage/#pydantic_ai.usage.Usage "pydantic_ai.usage.Usage") | None = None, infer_name: [bool](https://docs.python.org/3/library/functions.html#bool) = True ) -> [AgentRunResult](#pydantic_ai.agent.AgentRunResult "pydantic_ai.agent.AgentRunResult") [[ResultDataT](../result/#pydantic_ai.result.ResultDataT "pydantic_ai.result.ResultDataT") ]` `run_sync( user_prompt: [str](https://docs.python.org/3/library/stdtypes.html#str) | [Sequence](https://docs.python.org/3/library/collections.abc.html#collections.abc.Sequence "collections.abc.Sequence") [UserContent], *, result_type: [type](https://docs.python.org/3/library/functions.html#type) [[RunResultDataT](#pydantic_ai.agent.RunResultDataT "pydantic_ai.agent.RunResultDataT") ] | None, message_history: [list](https://docs.python.org/3/library/stdtypes.html#list) [[ModelMessage](../messages/#pydantic_ai.messages.ModelMessage "pydantic_ai.messages.ModelMessage") ] | None = None, model: [Model](../models/base/#pydantic_ai.models.Model "pydantic_ai.models.Model") | [KnownModelName](../models/base/#pydantic_ai.models.KnownModelName "pydantic_ai.models.KnownModelName") | [str](https://docs.python.org/3/library/stdtypes.html#str) | None = None, deps: [AgentDepsT](../tools/#pydantic_ai.tools.AgentDepsT "pydantic_ai.tools.AgentDepsT") = None, model_settings: [ModelSettings](../settings/#pydantic_ai.settings.ModelSettings "pydantic_ai.settings.ModelSettings") | None = None, usage_limits: [UsageLimits](../usage/#pydantic_ai.usage.UsageLimits "pydantic_ai.usage.UsageLimits") | None = None, usage: [Usage](../usage/#pydantic_ai.usage.Usage "pydantic_ai.usage.Usage") | None = None, infer_name: [bool](https://docs.python.org/3/library/functions.html#bool) = True ) -> [AgentRunResult](#pydantic_ai.agent.AgentRunResult "pydantic_ai.agent.AgentRunResult") [[RunResultDataT](#pydantic_ai.agent.RunResultDataT "pydantic_ai.agent.RunResultDataT") ]` `run_sync( user_prompt: [str](https://docs.python.org/3/library/stdtypes.html#str) | [Sequence](https://docs.python.org/3/library/collections.abc.html#collections.abc.Sequence "collections.abc.Sequence") [UserContent], *, result_type: [type](https://docs.python.org/3/library/functions.html#type) [[RunResultDataT](#pydantic_ai.agent.RunResultDataT "pydantic_ai.agent.RunResultDataT") ] | None = None, message_history: [list](https://docs.python.org/3/library/stdtypes.html#list) [[ModelMessage](../messages/#pydantic_ai.messages.ModelMessage "pydantic_ai.messages.ModelMessage") ] | None = None, model: [Model](../models/base/#pydantic_ai.models.Model "pydantic_ai.models.Model") | [KnownModelName](../models/base/#pydantic_ai.models.KnownModelName "pydantic_ai.models.KnownModelName") | [str](https://docs.python.org/3/library/stdtypes.html#str) | None = None, deps: [AgentDepsT](../tools/#pydantic_ai.tools.AgentDepsT "pydantic_ai.tools.AgentDepsT") = None, model_settings: [ModelSettings](../settings/#pydantic_ai.settings.ModelSettings "pydantic_ai.settings.ModelSettings") | None = None, usage_limits: [UsageLimits](../usage/#pydantic_ai.usage.UsageLimits "pydantic_ai.usage.UsageLimits") | None = None, usage: [Usage](../usage/#pydantic_ai.usage.Usage "pydantic_ai.usage.Usage") | None = None, infer_name: [bool](https://docs.python.org/3/library/functions.html#bool) = True ) -> [AgentRunResult](#pydantic_ai.agent.AgentRunResult "pydantic_ai.agent.AgentRunResult") [[Any](https://docs.python.org/3/library/typing.html#typing.Any "typing.Any") ]` Synchronously run the agent with a user prompt. This is a convenience method that wraps [`self.run`](#pydantic_ai.agent.Agent.run) with `loop.run_until_complete(...)`. You therefore can't use this method inside async code or if there's an active event loop. Example: `from pydantic_ai import Agent agent = Agent('openai:gpt-4o') result_sync = agent.run_sync('What is the capital of Italy?') print(result_sync.data) #> Rome` Parameters: | Name | Type | Description | Default | | --- | --- | --- | --- | | `user_prompt` | `[str](https://docs.python.org/3/library/stdtypes.html#str) \| [Sequence](https://docs.python.org/3/library/collections.abc.html#collections.abc.Sequence "collections.abc.Sequence") [UserContent]` | User input to start/continue the conversation. | _required_ | | `result_type` | `[type](https://docs.python.org/3/library/functions.html#type) [[RunResultDataT](#pydantic_ai.agent.RunResultDataT "pydantic_ai.agent.RunResultDataT") ] \| None` | Custom result type to use for this run, `result_type` may only be used if the agent has no result validators since result validators would expect an argument that matches the agent's result type. | `None` | | `message_history` | `[list](https://docs.python.org/3/library/stdtypes.html#list) [[ModelMessage](../messages/#pydantic_ai.messages.ModelMessage "pydantic_ai.messages.ModelMessage") ] \| None` | History of the conversation so far. | `None` | | `model` | `[Model](../models/base/#pydantic_ai.models.Model "pydantic_ai.models.Model") \| [KnownModelName](../models/base/#pydantic_ai.models.KnownModelName "pydantic_ai.models.KnownModelName") \| [str](https://docs.python.org/3/library/stdtypes.html#str) \| None` | Optional model to use for this run, required if `model` was not set when creating the agent. | `None` | | `deps` | `[AgentDepsT](../tools/#pydantic_ai.tools.AgentDepsT "pydantic_ai.tools.AgentDepsT") ` | Optional dependencies to use for this run. | `None` | | `model_settings` | `[ModelSettings](../settings/#pydantic_ai.settings.ModelSettings "pydantic_ai.settings.ModelSettings") \| None` | Optional settings to use for this model's request. | `None` | | `usage_limits` | `[UsageLimits](../usage/#pydantic_ai.usage.UsageLimits "pydantic_ai.usage.UsageLimits") \| None` | Optional limits on model request count or token usage. | `None` | | `usage` | `[Usage](../usage/#pydantic_ai.usage.Usage "pydantic_ai.usage.Usage") \| None` | Optional usage to start with, useful for resuming a conversation or agents used in tools. | `None` | | `infer_name` | `[bool](https://docs.python.org/3/library/functions.html#bool) ` | Whether to try to infer the agent name from the call frame if it's not set. | `True` | Returns: | Type | Description | | --- | --- | | `[AgentRunResult](#pydantic_ai.agent.AgentRunResult "pydantic_ai.agent.AgentRunResult") [[Any](https://docs.python.org/3/library/typing.html#typing.Any "typing.Any") ]` | The result of the run. | Source code in `pydantic_ai_slim/pydantic_ai/agent.py` | | | | --- | --- | | 526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584 | ``def run_sync( self, user_prompt: str \| Sequence[_messages.UserContent], *, result_type: type[RunResultDataT] \| None = None, message_history: list[_messages.ModelMessage] \| None = None, model: models.Model \| models.KnownModelName \| str \| None = None, deps: AgentDepsT = None, model_settings: ModelSettings \| None = None, usage_limits: _usage.UsageLimits \| None = None, usage: _usage.Usage \| None = None, infer_name: bool = True, ) -> AgentRunResult[Any]: """Synchronously run the agent with a user prompt. This is a convenience method that wraps [`self.run`][pydantic_ai.Agent.run] with `loop.run_until_complete(...)`. You therefore can't use this method inside async code or if there's an active event loop. Example: ```python from pydantic_ai import Agent agent = Agent('openai:gpt-4o') result_sync = agent.run_sync('What is the capital of Italy?') print(result_sync.data) #> Rome ``` Args: user_prompt: User input to start/continue the conversation. result_type: Custom result type to use for this run, `result_type` may only be used if the agent has no result validators since result validators would expect an argument that matches the agent's result type. message_history: History of the conversation so far. model: Optional model to use for this run, required if `model` was not set when creating the agent. deps: Optional dependencies to use for this run. model_settings: Optional settings to use for this model's request. usage_limits: Optional limits on model request count or token usage. usage: Optional usage to start with, useful for resuming a conversation or agents used in tools. infer_name: Whether to try to infer the agent name from the call frame if it's not set. Returns: The result of the run. """ if infer_name and self.name is None: self._infer_name(inspect.currentframe()) return get_event_loop().run_until_complete( self.run( user_prompt, result_type=result_type, message_history=message_history, model=model, deps=deps, model_settings=model_settings, usage_limits=usage_limits, usage=usage, infer_name=False, ) )`` | #### run\_stream `async` `run_stream( user_prompt: [str](https://docs.python.org/3/library/stdtypes.html#str) | [Sequence](https://docs.python.org/3/library/collections.abc.html#collections.abc.Sequence "collections.abc.Sequence") [UserContent], *, result_type: None = None, message_history: [list](https://docs.python.org/3/library/stdtypes.html#list) [[ModelMessage](../messages/#pydantic_ai.messages.ModelMessage "pydantic_ai.messages.ModelMessage") ] | None = None, model: [Model](../models/base/#pydantic_ai.models.Model "pydantic_ai.models.Model") | [KnownModelName](../models/base/#pydantic_ai.models.KnownModelName "pydantic_ai.models.KnownModelName") | [str](https://docs.python.org/3/library/stdtypes.html#str) | None = None, deps: [AgentDepsT](../tools/#pydantic_ai.tools.AgentDepsT "pydantic_ai.tools.AgentDepsT") = None, model_settings: [ModelSettings](../settings/#pydantic_ai.settings.ModelSettings "pydantic_ai.settings.ModelSettings") | None = None, usage_limits: [UsageLimits](../usage/#pydantic_ai.usage.UsageLimits "pydantic_ai.usage.UsageLimits") | None = None, usage: [Usage](../usage/#pydantic_ai.usage.Usage "pydantic_ai.usage.Usage") | None = None, infer_name: [bool](https://docs.python.org/3/library/functions.html#bool) = True ) -> [AbstractAsyncContextManager](https://docs.python.org/3/library/contextlib.html#contextlib.AbstractAsyncContextManager "contextlib.AbstractAsyncContextManager") [ [StreamedRunResult](../result/#pydantic_ai.result.StreamedRunResult "pydantic_ai.result.StreamedRunResult") [[AgentDepsT](../tools/#pydantic_ai.tools.AgentDepsT "pydantic_ai.tools.AgentDepsT") , [ResultDataT](../result/#pydantic_ai.result.ResultDataT "pydantic_ai.result.ResultDataT") ] ]` `run_stream( user_prompt: [str](https://docs.python.org/3/library/stdtypes.html#str) | [Sequence](https://docs.python.org/3/library/collections.abc.html#collections.abc.Sequence "collections.abc.Sequence") [UserContent], *, result_type: [type](https://docs.python.org/3/library/functions.html#type) [[RunResultDataT](#pydantic_ai.agent.RunResultDataT "pydantic_ai.agent.RunResultDataT") ], message_history: [list](https://docs.python.org/3/library/stdtypes.html#list) [[ModelMessage](../messages/#pydantic_ai.messages.ModelMessage "pydantic_ai.messages.ModelMessage") ] | None = None, model: [Model](../models/base/#pydantic_ai.models.Model "pydantic_ai.models.Model") | [KnownModelName](../models/base/#pydantic_ai.models.KnownModelName "pydantic_ai.models.KnownModelName") | [str](https://docs.python.org/3/library/stdtypes.html#str) | None = None, deps: [AgentDepsT](../tools/#pydantic_ai.tools.AgentDepsT "pydantic_ai.tools.AgentDepsT") = None, model_settings: [ModelSettings](../settings/#pydantic_ai.settings.ModelSettings "pydantic_ai.settings.ModelSettings") | None = None, usage_limits: [UsageLimits](../usage/#pydantic_ai.usage.UsageLimits "pydantic_ai.usage.UsageLimits") | None = None, usage: [Usage](../usage/#pydantic_ai.usage.Usage "pydantic_ai.usage.Usage") | None = None, infer_name: [bool](https://docs.python.org/3/library/functions.html#bool) = True ) -> [AbstractAsyncContextManager](https://docs.python.org/3/library/contextlib.html#contextlib.AbstractAsyncContextManager "contextlib.AbstractAsyncContextManager") [ [StreamedRunResult](../result/#pydantic_ai.result.StreamedRunResult "pydantic_ai.result.StreamedRunResult") [[AgentDepsT](../tools/#pydantic_ai.tools.AgentDepsT "pydantic_ai.tools.AgentDepsT") , [RunResultDataT](#pydantic_ai.agent.RunResultDataT "pydantic_ai.agent.RunResultDataT") ] ]` `run_stream( user_prompt: [str](https://docs.python.org/3/library/stdtypes.html#str) | [Sequence](https://docs.python.org/3/library/collections.abc.html#collections.abc.Sequence "collections.abc.Sequence") [UserContent], *, result_type: [type](https://docs.python.org/3/library/functions.html#type) [[RunResultDataT](#pydantic_ai.agent.RunResultDataT "pydantic_ai.agent.RunResultDataT") ] | None = None, message_history: [list](https://docs.python.org/3/library/stdtypes.html#list) [[ModelMessage](../messages/#pydantic_ai.messages.ModelMessage "pydantic_ai.messages.ModelMessage") ] | None = None, model: [Model](../models/base/#pydantic_ai.models.Model "pydantic_ai.models.Model") | [KnownModelName](../models/base/#pydantic_ai.models.KnownModelName "pydantic_ai.models.KnownModelName") | [str](https://docs.python.org/3/library/stdtypes.html#str) | None = None, deps: [AgentDepsT](../tools/#pydantic_ai.tools.AgentDepsT "pydantic_ai.tools.AgentDepsT") = None, model_settings: [ModelSettings](../settings/#pydantic_ai.settings.ModelSettings "pydantic_ai.settings.ModelSettings") | None = None, usage_limits: [UsageLimits](../usage/#pydantic_ai.usage.UsageLimits "pydantic_ai.usage.UsageLimits") | None = None, usage: [Usage](../usage/#pydantic_ai.usage.Usage "pydantic_ai.usage.Usage") | None = None, infer_name: [bool](https://docs.python.org/3/library/functions.html#bool) = True ) -> [AsyncIterator](https://docs.python.org/3/library/collections.abc.html#collections.abc.AsyncIterator "collections.abc.AsyncIterator") [[StreamedRunResult](../result/#pydantic_ai.result.StreamedRunResult "pydantic_ai.result.StreamedRunResult") [[AgentDepsT](../tools/#pydantic_ai.tools.AgentDepsT "pydantic_ai.tools.AgentDepsT") , [Any](https://docs.python.org/3/library/typing.html#typing.Any "typing.Any") ]]` Run the agent with a user prompt in async mode, returning a streamed response. Example: `from pydantic_ai import Agent agent = Agent('openai:gpt-4o') async def main(): async with agent.run_stream('What is the capital of the UK?') as response: print(await response.get_data()) #> London` Parameters: | Name | Type | Description | Default | | --- | --- | --- | --- | | `user_prompt` | `[str](https://docs.python.org/3/library/stdtypes.html#str) \| [Sequence](https://docs.python.org/3/library/collections.abc.html#collections.abc.Sequence "collections.abc.Sequence") [UserContent]` | User input to start/continue the conversation. | _required_ | | `result_type` | `[type](https://docs.python.org/3/library/functions.html#type) [[RunResultDataT](#pydantic_ai.agent.RunResultDataT "pydantic_ai.agent.RunResultDataT") ] \| None` | Custom result type to use for this run, `result_type` may only be used if the agent has no result validators since result validators would expect an argument that matches the agent's result type. | `None` | | `message_history` | `[list](https://docs.python.org/3/library/stdtypes.html#list) [[ModelMessage](../messages/#pydantic_ai.messages.ModelMessage "pydantic_ai.messages.ModelMessage") ] \| None` | History of the conversation so far. | `None` | | `model` | `[Model](../models/base/#pydantic_ai.models.Model "pydantic_ai.models.Model") \| [KnownModelName](../models/base/#pydantic_ai.models.KnownModelName "pydantic_ai.models.KnownModelName") \| [str](https://docs.python.org/3/library/stdtypes.html#str) \| None` | Optional model to use for this run, required if `model` was not set when creating the agent. | `None` | | `deps` | `[AgentDepsT](../tools/#pydantic_ai.tools.AgentDepsT "pydantic_ai.tools.AgentDepsT") ` | Optional dependencies to use for this run. | `None` | | `model_settings` | `[ModelSettings](../settings/#pydantic_ai.settings.ModelSettings "pydantic_ai.settings.ModelSettings") \| None` | Optional settings to use for this model's request. | `None` | | `usage_limits` | `[UsageLimits](../usage/#pydantic_ai.usage.UsageLimits "pydantic_ai.usage.UsageLimits") \| None` | Optional limits on model request count or token usage. | `None` | | `usage` | `[Usage](../usage/#pydantic_ai.usage.Usage "pydantic_ai.usage.Usage") \| None` | Optional usage to start with, useful for resuming a conversation or agents used in tools. | `None` | | `infer_name` | `[bool](https://docs.python.org/3/library/functions.html#bool) ` | Whether to try to infer the agent name from the call frame if it's not set. | `True` | Returns: | Type | Description | | --- | --- | | `[AsyncIterator](https://docs.python.org/3/library/collections.abc.html#collections.abc.AsyncIterator "collections.abc.AsyncIterator") [[StreamedRunResult](../result/#pydantic_ai.result.StreamedRunResult "pydantic_ai.result.StreamedRunResult") [[AgentDepsT](../tools/#pydantic_ai.tools.AgentDepsT "pydantic_ai.tools.AgentDepsT") , [Any](https://docs.python.org/3/library/typing.html#typing.Any "typing.Any") ]]` | The result of the run. | Source code in `pydantic_ai_slim/pydantic_ai/agent.py` | | | | --- | --- | | 616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755 | ``@asynccontextmanager async def run_stream( # noqa C901 self, user_prompt: str \| Sequence[_messages.UserContent], *, result_type: type[RunResultDataT] \| None = None, message_history: list[_messages.ModelMessage] \| None = None, model: models.Model \| models.KnownModelName \| str \| None = None, deps: AgentDepsT = None, model_settings: ModelSettings \| None = None, usage_limits: _usage.UsageLimits \| None = None, usage: _usage.Usage \| None = None, infer_name: bool = True, ) -> AsyncIterator[result.StreamedRunResult[AgentDepsT, Any]]: """Run the agent with a user prompt in async mode, returning a streamed response. Example: ```python from pydantic_ai import Agent agent = Agent('openai:gpt-4o') async def main(): async with agent.run_stream('What is the capital of the UK?') as response: print(await response.get_data()) #> London ``` Args: user_prompt: User input to start/continue the conversation. result_type: Custom result type to use for this run, `result_type` may only be used if the agent has no result validators since result validators would expect an argument that matches the agent's result type. message_history: History of the conversation so far. model: Optional model to use for this run, required if `model` was not set when creating the agent. deps: Optional dependencies to use for this run. model_settings: Optional settings to use for this model's request. usage_limits: Optional limits on model request count or token usage. usage: Optional usage to start with, useful for resuming a conversation or agents used in tools. infer_name: Whether to try to infer the agent name from the call frame if it's not set. Returns: The result of the run. """ # TODO: We need to deprecate this now that we have the `iter` method. # Before that, though, we should add an event for when we reach the final result of the stream. if infer_name and self.name is None: # f_back because `asynccontextmanager` adds one frame if frame := inspect.currentframe(): # pragma: no branch self._infer_name(frame.f_back) yielded = False async with self.iter( user_prompt, result_type=result_type, message_history=message_history, model=model, deps=deps, model_settings=model_settings, usage_limits=usage_limits, usage=usage, infer_name=False, ) as agent_run: first_node = agent_run.next_node # start with the first node assert isinstance(first_node, _agent_graph.UserPromptNode) # the first node should be a user prompt node node = first_node while True: if self.is_model_request_node(node): graph_ctx = agent_run.ctx async with node._stream(graph_ctx) as streamed_response: # pyright: ignore[reportPrivateUsage] async def stream_to_final( s: models.StreamedResponse, ) -> FinalResult[models.StreamedResponse] \| None: result_schema = graph_ctx.deps.result_schema async for maybe_part_event in streamed_response: if isinstance(maybe_part_event, _messages.PartStartEvent): new_part = maybe_part_event.part if isinstance(new_part, _messages.TextPart): if _agent_graph.allow_text_result(result_schema): return FinalResult(s, None, None) elif isinstance(new_part, _messages.ToolCallPart) and result_schema: for call, _ in result_schema.find_tool([new_part]): return FinalResult(s, call.tool_name, call.tool_call_id) return None final_result_details = await stream_to_final(streamed_response) if final_result_details is not None: if yielded: raise exceptions.AgentRunError('Agent run produced final results') yielded = True messages = graph_ctx.state.message_history.copy() async def on_complete() -> None: """Called when the stream has completed. The model response will have been added to messages by now by `StreamedRunResult._marked_completed`. """ last_message = messages[-1] assert isinstance(last_message, _messages.ModelResponse) tool_calls = [ part for part in last_message.parts if isinstance(part, _messages.ToolCallPart) ] parts: list[_messages.ModelRequestPart] = [] async for _event in _agent_graph.process_function_tools( tool_calls, final_result_details.tool_name, final_result_details.tool_call_id, graph_ctx, parts, ): pass # TODO: Should we do something here related to the retry count? # Maybe we should move the incrementing of the retry count to where we actually make a request? # if any(isinstance(part, _messages.RetryPromptPart) for part in parts): # ctx.state.increment_retries(ctx.deps.max_result_retries) if parts: messages.append(_messages.ModelRequest(parts)) yield StreamedRunResult( messages, graph_ctx.deps.new_message_index, graph_ctx.deps.usage_limits, streamed_response, graph_ctx.deps.result_schema, _agent_graph.build_run_context(graph_ctx), graph_ctx.deps.result_validators, final_result_details.tool_name, on_complete, ) break next_node = await agent_run.next(node) if not isinstance(next_node, _agent_graph.AgentNode): raise exceptions.AgentRunError('Should have produced a StreamedRunResult before getting here') node = cast(_agent_graph.AgentNode[Any, Any], next_node) if not yielded: raise exceptions.AgentRunError('Agent run finished without producing a final result')`` | #### override `override( *, deps: [AgentDepsT](../tools/#pydantic_ai.tools.AgentDepsT "pydantic_ai.tools.AgentDepsT") | Unset = UNSET, model: [Model](../models/base/#pydantic_ai.models.Model "pydantic_ai.models.Model") | [KnownModelName](../models/base/#pydantic_ai.models.KnownModelName "pydantic_ai.models.KnownModelName") | [str](https://docs.python.org/3/library/stdtypes.html#str) | Unset = UNSET ) -> [Iterator](https://docs.python.org/3/library/collections.abc.html#collections.abc.Iterator "collections.abc.Iterator") [None]` Context manager to temporarily override agent dependencies and model. This is particularly useful when testing. You can find an example of this [here](../../testing/#overriding-model-via-pytest-fixtures) . Parameters: | Name | Type | Description | Default | | --- | --- | --- | --- | | `deps` | `[AgentDepsT](../tools/#pydantic_ai.tools.AgentDepsT "pydantic_ai.tools.AgentDepsT") \| Unset` | The dependencies to use instead of the dependencies passed to the agent run. | `UNSET` | | `model` | `[Model](../models/base/#pydantic_ai.models.Model "pydantic_ai.models.Model") \| [KnownModelName](../models/base/#pydantic_ai.models.KnownModelName "pydantic_ai.models.KnownModelName") \| [str](https://docs.python.org/3/library/stdtypes.html#str) \| Unset` | The model to use instead of the model passed to the agent run. | `UNSET` | Source code in `pydantic_ai_slim/pydantic_ai/agent.py` | | | | --- | --- | | 757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791 | `@contextmanager def override( self, *, deps: AgentDepsT \| _utils.Unset = _utils.UNSET, model: models.Model \| models.KnownModelName \| str \| _utils.Unset = _utils.UNSET, ) -> Iterator[None]: """Context manager to temporarily override agent dependencies and model. This is particularly useful when testing. You can find an example of this [here](../testing.md#overriding-model-via-pytest-fixtures). Args: deps: The dependencies to use instead of the dependencies passed to the agent run. model: The model to use instead of the model passed to the agent run. """ if _utils.is_set(deps): override_deps_before = self._override_deps self._override_deps = _utils.Some(deps) else: override_deps_before = _utils.UNSET if _utils.is_set(model): override_model_before = self._override_model self._override_model = _utils.Some(models.infer_model(model)) else: override_model_before = _utils.UNSET try: yield finally: if _utils.is_set(override_deps_before): self._override_deps = override_deps_before if _utils.is_set(override_model_before): self._override_model = override_model_before` | #### system\_prompt `system_prompt( func: [Callable](https://docs.python.org/3/library/typing.html#typing.Callable "typing.Callable") [[[RunContext](../tools/#pydantic_ai.tools.RunContext "pydantic_ai.tools.RunContext") [[AgentDepsT](../tools/#pydantic_ai.tools.AgentDepsT "pydantic_ai.tools.AgentDepsT") ]], [str](https://docs.python.org/3/library/stdtypes.html#str) ], ) -> [Callable](https://docs.python.org/3/library/typing.html#typing.Callable "typing.Callable") [[[RunContext](../tools/#pydantic_ai.tools.RunContext "pydantic_ai.tools.RunContext") [[AgentDepsT](../tools/#pydantic_ai.tools.AgentDepsT "pydantic_ai.tools.AgentDepsT") ]], [str](https://docs.python.org/3/library/stdtypes.html#str) ]` `system_prompt( func: [Callable](https://docs.python.org/3/library/typing.html#typing.Callable "typing.Callable") [ [[RunContext](../tools/#pydantic_ai.tools.RunContext "pydantic_ai.tools.RunContext") [[AgentDepsT](../tools/#pydantic_ai.tools.AgentDepsT "pydantic_ai.tools.AgentDepsT") ]], [Awaitable](https://docs.python.org/3/library/collections.abc.html#collections.abc.Awaitable "collections.abc.Awaitable") [[str](https://docs.python.org/3/library/stdtypes.html#str) ] ], ) -> [Callable](https://docs.python.org/3/library/typing.html#typing.Callable "typing.Callable") [[[RunContext](../tools/#pydantic_ai.tools.RunContext "pydantic_ai.tools.RunContext") [[AgentDepsT](../tools/#pydantic_ai.tools.AgentDepsT "pydantic_ai.tools.AgentDepsT") ]], [Awaitable](https://docs.python.org/3/library/collections.abc.html#collections.abc.Awaitable "collections.abc.Awaitable") [[str](https://docs.python.org/3/library/stdtypes.html#str) ]]` `system_prompt(func: [Callable](https://docs.python.org/3/library/typing.html#typing.Callable "typing.Callable") [[], [str](https://docs.python.org/3/library/stdtypes.html#str) ]) -> [Callable](https://docs.python.org/3/library/typing.html#typing.Callable "typing.Callable") [[], [str](https://docs.python.org/3/library/stdtypes.html#str) ]` `system_prompt( func: [Callable](https://docs.python.org/3/library/typing.html#typing.Callable "typing.Callable") [[], [Awaitable](https://docs.python.org/3/library/collections.abc.html#collections.abc.Awaitable "collections.abc.Awaitable") [[str](https://docs.python.org/3/library/stdtypes.html#str) ]], ) -> [Callable](https://docs.python.org/3/library/typing.html#typing.Callable "typing.Callable") [[], [Awaitable](https://docs.python.org/3/library/collections.abc.html#collections.abc.Awaitable "collections.abc.Awaitable") [[str](https://docs.python.org/3/library/stdtypes.html#str) ]]` `system_prompt(*, dynamic: [bool](https://docs.python.org/3/library/functions.html#bool) = False) -> [Callable](https://docs.python.org/3/library/typing.html#typing.Callable "typing.Callable") [ [[SystemPromptFunc](../tools/#pydantic_ai.tools.SystemPromptFunc "pydantic_ai._system_prompt.SystemPromptFunc") [[AgentDepsT](../tools/#pydantic_ai.tools.AgentDepsT "pydantic_ai.tools.AgentDepsT") ]], [SystemPromptFunc](../tools/#pydantic_ai.tools.SystemPromptFunc "pydantic_ai._system_prompt.SystemPromptFunc") [[AgentDepsT](../tools/#pydantic_ai.tools.AgentDepsT "pydantic_ai.tools.AgentDepsT") ], ]` `system_prompt( func: [SystemPromptFunc](../tools/#pydantic_ai.tools.SystemPromptFunc "pydantic_ai._system_prompt.SystemPromptFunc") [[AgentDepsT](../tools/#pydantic_ai.tools.AgentDepsT "pydantic_ai.tools.AgentDepsT") ] | None = None, /, *, dynamic: [bool](https://docs.python.org/3/library/functions.html#bool) = False, ) -> ( [Callable](https://docs.python.org/3/library/typing.html#typing.Callable "typing.Callable") [ [[SystemPromptFunc](../tools/#pydantic_ai.tools.SystemPromptFunc "pydantic_ai._system_prompt.SystemPromptFunc") [[AgentDepsT](../tools/#pydantic_ai.tools.AgentDepsT "pydantic_ai.tools.AgentDepsT") ]], [SystemPromptFunc](../tools/#pydantic_ai.tools.SystemPromptFunc "pydantic_ai._system_prompt.SystemPromptFunc") [[AgentDepsT](../tools/#pydantic_ai.tools.AgentDepsT "pydantic_ai.tools.AgentDepsT") ], ] | [SystemPromptFunc](../tools/#pydantic_ai.tools.SystemPromptFunc "pydantic_ai._system_prompt.SystemPromptFunc") [[AgentDepsT](../tools/#pydantic_ai.tools.AgentDepsT "pydantic_ai.tools.AgentDepsT") ] )` Decorator to register a system prompt function. Optionally takes [`RunContext`](../tools/#pydantic_ai.tools.RunContext) as its only argument. Can decorate a sync or async functions. The decorator can be used either bare (`agent.system_prompt`) or as a function call (`agent.system_prompt(...)`), see the examples below. Overloads for every possible signature of `system_prompt` are included so the decorator doesn't obscure the type of the function, see `tests/typed_agent.py` for tests. Parameters: | Name | Type | Description | Default | | --- | --- | --- | --- | | `func` | `[SystemPromptFunc](../tools/#pydantic_ai.tools.SystemPromptFunc "pydantic_ai._system_prompt.SystemPromptFunc") [[AgentDepsT](../tools/#pydantic_ai.tools.AgentDepsT "pydantic_ai.tools.AgentDepsT") ] \| None` | The function to decorate | `None` | | `dynamic` | `[bool](https://docs.python.org/3/library/functions.html#bool) ` | If True, the system prompt will be reevaluated even when `messages_history` is provided, see [`SystemPromptPart.dynamic_ref`](../messages/#pydantic_ai.messages.SystemPromptPart.dynamic_ref) | `False` | Example: `from pydantic_ai import Agent, RunContext agent = Agent('test', deps_type=str) @agent.system_prompt def simple_system_prompt() -> str: return 'foobar' @agent.system_prompt(dynamic=True) async def async_system_prompt(ctx: RunContext[str]) -> str: return f'{ctx.deps} is the best'` Source code in `pydantic_ai_slim/pydantic_ai/agent.py` | | | | --- | --- | | 814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870 | ``def system_prompt( self, func: _system_prompt.SystemPromptFunc[AgentDepsT] \| None = None, /, *, dynamic: bool = False, ) -> ( Callable[[_system_prompt.SystemPromptFunc[AgentDepsT]], _system_prompt.SystemPromptFunc[AgentDepsT]] \| _system_prompt.SystemPromptFunc[AgentDepsT] ): """Decorator to register a system prompt function. Optionally takes [`RunContext`][pydantic_ai.tools.RunContext] as its only argument. Can decorate a sync or async functions. The decorator can be used either bare (`agent.system_prompt`) or as a function call (`agent.system_prompt(...)`), see the examples below. Overloads for every possible signature of `system_prompt` are included so the decorator doesn't obscure the type of the function, see `tests/typed_agent.py` for tests. Args: func: The function to decorate dynamic: If True, the system prompt will be reevaluated even when `messages_history` is provided, see [`SystemPromptPart.dynamic_ref`][pydantic_ai.messages.SystemPromptPart.dynamic_ref] Example: ```python from pydantic_ai import Agent, RunContext agent = Agent('test', deps_type=str) @agent.system_prompt def simple_system_prompt() -> str: return 'foobar' @agent.system_prompt(dynamic=True) async def async_system_prompt(ctx: RunContext[str]) -> str: return f'{ctx.deps} is the best' ``` """ if func is None: def decorator( func_: _system_prompt.SystemPromptFunc[AgentDepsT], ) -> _system_prompt.SystemPromptFunc[AgentDepsT]: runner = _system_prompt.SystemPromptRunner[AgentDepsT](func_, dynamic=dynamic) self._system_prompt_functions.append(runner) if dynamic: self._system_prompt_dynamic_functions[func_.__qualname__] = runner return func_ return decorator else: assert not dynamic, "dynamic can't be True in this case" self._system_prompt_functions.append(_system_prompt.SystemPromptRunner[AgentDepsT](func, dynamic=dynamic)) return func`` | #### result\_validator `result_validator( func: [Callable](https://docs.python.org/3/library/typing.html#typing.Callable "typing.Callable") [ [[RunContext](../tools/#pydantic_ai.tools.RunContext "pydantic_ai.tools.RunContext") [[AgentDepsT](../tools/#pydantic_ai.tools.AgentDepsT "pydantic_ai.tools.AgentDepsT") ], [ResultDataT](../result/#pydantic_ai.result.ResultDataT "pydantic_ai.result.ResultDataT") ], [ResultDataT](../result/#pydantic_ai.result.ResultDataT "pydantic_ai.result.ResultDataT") ], ) -> [Callable](https://docs.python.org/3/library/typing.html#typing.Callable "typing.Callable") [ [[RunContext](../tools/#pydantic_ai.tools.RunContext "pydantic_ai.tools.RunContext") [[AgentDepsT](../tools/#pydantic_ai.tools.AgentDepsT "pydantic_ai.tools.AgentDepsT") ], [ResultDataT](../result/#pydantic_ai.result.ResultDataT "pydantic_ai.result.ResultDataT") ], [ResultDataT](../result/#pydantic_ai.result.ResultDataT "pydantic_ai.result.ResultDataT") ]` `result_validator( func: [Callable](https://docs.python.org/3/library/typing.html#typing.Callable "typing.Callable") [ [[RunContext](../tools/#pydantic_ai.tools.RunContext "pydantic_ai.tools.RunContext") [[AgentDepsT](../tools/#pydantic_ai.tools.AgentDepsT "pydantic_ai.tools.AgentDepsT") ], [ResultDataT](../result/#pydantic_ai.result.ResultDataT "pydantic_ai.result.ResultDataT") ], [Awaitable](https://docs.python.org/3/library/collections.abc.html#collections.abc.Awaitable "collections.abc.Awaitable") [[ResultDataT](../result/#pydantic_ai.result.ResultDataT "pydantic_ai.result.ResultDataT") ], ], ) -> [Callable](https://docs.python.org/3/library/typing.html#typing.Callable "typing.Callable") [ [[RunContext](../tools/#pydantic_ai.tools.RunContext "pydantic_ai.tools.RunContext") [[AgentDepsT](../tools/#pydantic_ai.tools.AgentDepsT "pydantic_ai.tools.AgentDepsT") ], [ResultDataT](../result/#pydantic_ai.result.ResultDataT "pydantic_ai.result.ResultDataT") ], [Awaitable](https://docs.python.org/3/library/collections.abc.html#collections.abc.Awaitable "collections.abc.Awaitable") [[ResultDataT](../result/#pydantic_ai.result.ResultDataT "pydantic_ai.result.ResultDataT") ], ]` `result_validator( func: [Callable](https://docs.python.org/3/library/typing.html#typing.Callable "typing.Callable") [[[ResultDataT](../result/#pydantic_ai.result.ResultDataT "pydantic_ai.result.ResultDataT") ], [ResultDataT](../result/#pydantic_ai.result.ResultDataT "pydantic_ai.result.ResultDataT") ], ) -> [Callable](https://docs.python.org/3/library/typing.html#typing.Callable "typing.Callable") [[[ResultDataT](../result/#pydantic_ai.result.ResultDataT "pydantic_ai.result.ResultDataT") ], [ResultDataT](../result/#pydantic_ai.result.ResultDataT "pydantic_ai.result.ResultDataT") ]` `result_validator( func: [Callable](https://docs.python.org/3/library/typing.html#typing.Callable "typing.Callable") [[[ResultDataT](../result/#pydantic_ai.result.ResultDataT "pydantic_ai.result.ResultDataT") ], [Awaitable](https://docs.python.org/3/library/collections.abc.html#collections.abc.Awaitable "collections.abc.Awaitable") [[ResultDataT](../result/#pydantic_ai.result.ResultDataT "pydantic_ai.result.ResultDataT") ]], ) -> [Callable](https://docs.python.org/3/library/typing.html#typing.Callable "typing.Callable") [[[ResultDataT](../result/#pydantic_ai.result.ResultDataT "pydantic_ai.result.ResultDataT") ], [Awaitable](https://docs.python.org/3/library/collections.abc.html#collections.abc.Awaitable "collections.abc.Awaitable") [[ResultDataT](../result/#pydantic_ai.result.ResultDataT "pydantic_ai.result.ResultDataT") ]]` `result_validator( func: ResultValidatorFunc[[AgentDepsT](../tools/#pydantic_ai.tools.AgentDepsT "pydantic_ai.tools.AgentDepsT") , [ResultDataT](../result/#pydantic_ai.result.ResultDataT "pydantic_ai.result.ResultDataT") ], ) -> ResultValidatorFunc[[AgentDepsT](../tools/#pydantic_ai.tools.AgentDepsT "pydantic_ai.tools.AgentDepsT") , [ResultDataT](../result/#pydantic_ai.result.ResultDataT "pydantic_ai.result.ResultDataT") ]` Decorator to register a result validator function. Optionally takes [`RunContext`](../tools/#pydantic_ai.tools.RunContext) as its first argument. Can decorate a sync or async functions. Overloads for every possible signature of `result_validator` are included so the decorator doesn't obscure the type of the function, see `tests/typed_agent.py` for tests. Example: `from pydantic_ai import Agent, ModelRetry, RunContext agent = Agent('test', deps_type=str) @agent.result_validator def result_validator_simple(data: str) -> str: if 'wrong' in data: raise ModelRetry('wrong response') return data @agent.result_validator async def result_validator_deps(ctx: RunContext[str], data: str) -> str: if ctx.deps in data: raise ModelRetry('wrong response') return data result = agent.run_sync('foobar', deps='spam') print(result.data) #> success (no tool calls)` Source code in `pydantic_ai_slim/pydantic_ai/agent.py` | | | | --- | --- | | 892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927 | ``def result_validator( self, func: _result.ResultValidatorFunc[AgentDepsT, ResultDataT], / ) -> _result.ResultValidatorFunc[AgentDepsT, ResultDataT]: """Decorator to register a result validator function. Optionally takes [`RunContext`][pydantic_ai.tools.RunContext] as its first argument. Can decorate a sync or async functions. Overloads for every possible signature of `result_validator` are included so the decorator doesn't obscure the type of the function, see `tests/typed_agent.py` for tests. Example: ```python from pydantic_ai import Agent, ModelRetry, RunContext agent = Agent('test', deps_type=str) @agent.result_validator def result_validator_simple(data: str) -> str: if 'wrong' in data: raise ModelRetry('wrong response') return data @agent.result_validator async def result_validator_deps(ctx: RunContext[str], data: str) -> str: if ctx.deps in data: raise ModelRetry('wrong response') return data result = agent.run_sync('foobar', deps='spam') print(result.data) #> success (no tool calls) ``` """ self._result_validators.append(_result.ResultValidator[AgentDepsT, Any](func)) return func`` | #### tool `tool( func: [ToolFuncContext](../tools/#pydantic_ai.tools.ToolFuncContext "pydantic_ai.tools.ToolFuncContext") [[AgentDepsT](../tools/#pydantic_ai.tools.AgentDepsT "pydantic_ai.tools.AgentDepsT") , [ToolParams](../tools/#pydantic_ai.tools.ToolParams "pydantic_ai.tools.ToolParams") ], ) -> [ToolFuncContext](../tools/#pydantic_ai.tools.ToolFuncContext "pydantic_ai.tools.ToolFuncContext") [[AgentDepsT](../tools/#pydantic_ai.tools.AgentDepsT "pydantic_ai.tools.AgentDepsT") , [ToolParams](../tools/#pydantic_ai.tools.ToolParams "pydantic_ai.tools.ToolParams") ]` `tool( *, name: [str](https://docs.python.org/3/library/stdtypes.html#str) | None = None, retries: [int](https://docs.python.org/3/library/functions.html#int) | None = None, prepare: [ToolPrepareFunc](../tools/#pydantic_ai.tools.ToolPrepareFunc "pydantic_ai.tools.ToolPrepareFunc") [[AgentDepsT](../tools/#pydantic_ai.tools.AgentDepsT "pydantic_ai.tools.AgentDepsT") ] | None = None, docstring_format: [DocstringFormat](../tools/#pydantic_ai.tools.DocstringFormat "pydantic_ai.tools.DocstringFormat") = "auto", require_parameter_descriptions: [bool](https://docs.python.org/3/library/functions.html#bool) = False, schema_generator: [type](https://docs.python.org/3/library/functions.html#type) [ [GenerateJsonSchema](https://docs.pydantic.dev/latest/api/json_schema/#pydantic.json_schema.GenerateJsonSchema "pydantic.json_schema.GenerateJsonSchema") ] = GenerateToolJsonSchema ) -> [Callable](https://docs.python.org/3/library/typing.html#typing.Callable "typing.Callable") [ [[ToolFuncContext](../tools/#pydantic_ai.tools.ToolFuncContext "pydantic_ai.tools.ToolFuncContext") [[AgentDepsT](../tools/#pydantic_ai.tools.AgentDepsT "pydantic_ai.tools.AgentDepsT") , [ToolParams](../tools/#pydantic_ai.tools.ToolParams "pydantic_ai.tools.ToolParams") ]], [ToolFuncContext](../tools/#pydantic_ai.tools.ToolFuncContext "pydantic_ai.tools.ToolFuncContext") [[AgentDepsT](../tools/#pydantic_ai.tools.AgentDepsT "pydantic_ai.tools.AgentDepsT") , [ToolParams](../tools/#pydantic_ai.tools.ToolParams "pydantic_ai.tools.ToolParams") ], ]` `tool( func: ( [ToolFuncContext](../tools/#pydantic_ai.tools.ToolFuncContext "pydantic_ai.tools.ToolFuncContext") [[AgentDepsT](../tools/#pydantic_ai.tools.AgentDepsT "pydantic_ai.tools.AgentDepsT") , [ToolParams](../tools/#pydantic_ai.tools.ToolParams "pydantic_ai.tools.ToolParams") ] | None ) = None, /, *, name: [str](https://docs.python.org/3/library/stdtypes.html#str) | None = None, retries: [int](https://docs.python.org/3/library/functions.html#int) | None = None, prepare: [ToolPrepareFunc](../tools/#pydantic_ai.tools.ToolPrepareFunc "pydantic_ai.tools.ToolPrepareFunc") [[AgentDepsT](../tools/#pydantic_ai.tools.AgentDepsT "pydantic_ai.tools.AgentDepsT") ] | None = None, docstring_format: [DocstringFormat](../tools/#pydantic_ai.tools.DocstringFormat "pydantic_ai.tools.DocstringFormat") = "auto", require_parameter_descriptions: [bool](https://docs.python.org/3/library/functions.html#bool) = False, schema_generator: [type](https://docs.python.org/3/library/functions.html#type) [ [GenerateJsonSchema](https://docs.pydantic.dev/latest/api/json_schema/#pydantic.json_schema.GenerateJsonSchema "pydantic.json_schema.GenerateJsonSchema") ] = GenerateToolJsonSchema, ) -> [Any](https://docs.python.org/3/library/typing.html#typing.Any "typing.Any")` Decorator to register a tool function which takes [`RunContext`](../tools/#pydantic_ai.tools.RunContext) as its first argument. Can decorate a sync or async functions. The docstring is inspected to extract both the tool description and description of each parameter, [learn more](../../tools/#function-tools-and-schema) . We can't add overloads for every possible signature of tool, since the return type is a recursive union so the signature of functions decorated with `@agent.tool` is obscured. Example: `from pydantic_ai import Agent, RunContext agent = Agent('test', deps_type=int) @agent.tool def foobar(ctx: RunContext[int], x: int) -> int: return ctx.deps + x @agent.tool(retries=2) async def spam(ctx: RunContext[str], y: float) -> float: return ctx.deps + y result = agent.run_sync('foobar', deps=1) print(result.data) #> {"foobar":1,"spam":1.0}` Parameters: | Name | Type | Description | Default | | --- | --- | --- | --- | | `func` | `[ToolFuncContext](../tools/#pydantic_ai.tools.ToolFuncContext "pydantic_ai.tools.ToolFuncContext") [[AgentDepsT](../tools/#pydantic_ai.tools.AgentDepsT "pydantic_ai.tools.AgentDepsT") , [ToolParams](../tools/#pydantic_ai.tools.ToolParams "pydantic_ai.tools.ToolParams") ] \| None` | The tool function to register. | `None` | | `name` | `[str](https://docs.python.org/3/library/stdtypes.html#str) \| None` | The name of the tool, defaults to the function name. | `None` | | `retries` | `[int](https://docs.python.org/3/library/functions.html#int) \| None` | The number of retries to allow for this tool, defaults to the agent's default retries, which defaults to 1. | `None` | | `prepare` | `[ToolPrepareFunc](../tools/#pydantic_ai.tools.ToolPrepareFunc "pydantic_ai.tools.ToolPrepareFunc") [[AgentDepsT](../tools/#pydantic_ai.tools.AgentDepsT "pydantic_ai.tools.AgentDepsT") ] \| None` | custom method to prepare the tool definition for each step, return `None` to omit this tool from a given step. This is useful if you want to customise a tool at call time, or omit it completely from a step. See [`ToolPrepareFunc`](../tools/#pydantic_ai.tools.ToolPrepareFunc)
. | `None` | | `docstring_format` | `[DocstringFormat](../tools/#pydantic_ai.tools.DocstringFormat "pydantic_ai.tools.DocstringFormat") ` | The format of the docstring, see [`DocstringFormat`](../tools/#pydantic_ai.tools.DocstringFormat)
. Defaults to `'auto'`, such that the format is inferred from the structure of the docstring. | `'auto'` | | `require_parameter_descriptions` | `[bool](https://docs.python.org/3/library/functions.html#bool) ` | If True, raise an error if a parameter description is missing. Defaults to False. | `False` | | `schema_generator` | `[type](https://docs.python.org/3/library/functions.html#type) [[GenerateJsonSchema](https://docs.pydantic.dev/latest/api/json_schema/#pydantic.json_schema.GenerateJsonSchema "pydantic.json_schema.GenerateJsonSchema") ]` | The JSON schema generator class to use for this tool. Defaults to `GenerateToolJsonSchema`. | `GenerateToolJsonSchema` | Source code in `pydantic_ai_slim/pydantic_ai/agent.py` | | | | --- | --- | | 945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
1001
1002
1003
1004
1005
1006
1007
1008
1009
1010
1011
1012
1013
1014
1015
1016
1017
1018
1019
1020
1021
1022
1023 | ``def tool( self, func: ToolFuncContext[AgentDepsT, ToolParams] \| None = None, /, *, name: str \| None = None, retries: int \| None = None, prepare: ToolPrepareFunc[AgentDepsT] \| None = None, docstring_format: DocstringFormat = 'auto', require_parameter_descriptions: bool = False, schema_generator: type[GenerateJsonSchema] = GenerateToolJsonSchema, ) -> Any: """Decorator to register a tool function which takes [`RunContext`][pydantic_ai.tools.RunContext] as its first argument. Can decorate a sync or async functions. The docstring is inspected to extract both the tool description and description of each parameter, [learn more](../tools.md#function-tools-and-schema). We can't add overloads for every possible signature of tool, since the return type is a recursive union so the signature of functions decorated with `@agent.tool` is obscured. Example: ```python from pydantic_ai import Agent, RunContext agent = Agent('test', deps_type=int) @agent.tool def foobar(ctx: RunContext[int], x: int) -> int: return ctx.deps + x @agent.tool(retries=2) async def spam(ctx: RunContext[str], y: float) -> float: return ctx.deps + y result = agent.run_sync('foobar', deps=1) print(result.data) #> {"foobar":1,"spam":1.0} ``` Args: func: The tool function to register. name: The name of the tool, defaults to the function name. retries: The number of retries to allow for this tool, defaults to the agent's default retries, which defaults to 1. prepare: custom method to prepare the tool definition for each step, return `None` to omit this tool from a given step. This is useful if you want to customise a tool at call time, or omit it completely from a step. See [`ToolPrepareFunc`][pydantic_ai.tools.ToolPrepareFunc]. docstring_format: The format of the docstring, see [`DocstringFormat`][pydantic_ai.tools.DocstringFormat]. Defaults to `'auto'`, such that the format is inferred from the structure of the docstring. require_parameter_descriptions: If True, raise an error if a parameter description is missing. Defaults to False. schema_generator: The JSON schema generator class to use for this tool. Defaults to `GenerateToolJsonSchema`. """ if func is None: def tool_decorator( func_: ToolFuncContext[AgentDepsT, ToolParams], ) -> ToolFuncContext[AgentDepsT, ToolParams]: # noinspection PyTypeChecker self._register_function( func_, True, name, retries, prepare, docstring_format, require_parameter_descriptions, schema_generator, ) return func_ return tool_decorator else: # noinspection PyTypeChecker self._register_function( func, True, name, retries, prepare, docstring_format, require_parameter_descriptions, schema_generator ) return func`` | #### tool\_plain `tool_plain( func: [ToolFuncPlain](../tools/#pydantic_ai.tools.ToolFuncPlain "pydantic_ai.tools.ToolFuncPlain") [[ToolParams](../tools/#pydantic_ai.tools.ToolParams "pydantic_ai.tools.ToolParams") ], ) -> [ToolFuncPlain](../tools/#pydantic_ai.tools.ToolFuncPlain "pydantic_ai.tools.ToolFuncPlain") [[ToolParams](../tools/#pydantic_ai.tools.ToolParams "pydantic_ai.tools.ToolParams") ]` `tool_plain( *, name: [str](https://docs.python.org/3/library/stdtypes.html#str) | None = None, retries: [int](https://docs.python.org/3/library/functions.html#int) | None = None, prepare: [ToolPrepareFunc](../tools/#pydantic_ai.tools.ToolPrepareFunc "pydantic_ai.tools.ToolPrepareFunc") [[AgentDepsT](../tools/#pydantic_ai.tools.AgentDepsT "pydantic_ai.tools.AgentDepsT") ] | None = None, docstring_format: [DocstringFormat](../tools/#pydantic_ai.tools.DocstringFormat "pydantic_ai.tools.DocstringFormat") = "auto", require_parameter_descriptions: [bool](https://docs.python.org/3/library/functions.html#bool) = False, schema_generator: [type](https://docs.python.org/3/library/functions.html#type) [ [GenerateJsonSchema](https://docs.pydantic.dev/latest/api/json_schema/#pydantic.json_schema.GenerateJsonSchema "pydantic.json_schema.GenerateJsonSchema") ] = GenerateToolJsonSchema ) -> [Callable](https://docs.python.org/3/library/typing.html#typing.Callable "typing.Callable") [ [[ToolFuncPlain](../tools/#pydantic_ai.tools.ToolFuncPlain "pydantic_ai.tools.ToolFuncPlain") [[ToolParams](../tools/#pydantic_ai.tools.ToolParams "pydantic_ai.tools.ToolParams") ]], [ToolFuncPlain](../tools/#pydantic_ai.tools.ToolFuncPlain "pydantic_ai.tools.ToolFuncPlain") [[ToolParams](../tools/#pydantic_ai.tools.ToolParams "pydantic_ai.tools.ToolParams") ] ]` `tool_plain( func: [ToolFuncPlain](../tools/#pydantic_ai.tools.ToolFuncPlain "pydantic_ai.tools.ToolFuncPlain") [[ToolParams](../tools/#pydantic_ai.tools.ToolParams "pydantic_ai.tools.ToolParams") ] | None = None, /, *, name: [str](https://docs.python.org/3/library/stdtypes.html#str) | None = None, retries: [int](https://docs.python.org/3/library/functions.html#int) | None = None, prepare: [ToolPrepareFunc](../tools/#pydantic_ai.tools.ToolPrepareFunc "pydantic_ai.tools.ToolPrepareFunc") [[AgentDepsT](../tools/#pydantic_ai.tools.AgentDepsT "pydantic_ai.tools.AgentDepsT") ] | None = None, docstring_format: [DocstringFormat](../tools/#pydantic_ai.tools.DocstringFormat "pydantic_ai.tools.DocstringFormat") = "auto", require_parameter_descriptions: [bool](https://docs.python.org/3/library/functions.html#bool) = False, schema_generator: [type](https://docs.python.org/3/library/functions.html#type) [ [GenerateJsonSchema](https://docs.pydantic.dev/latest/api/json_schema/#pydantic.json_schema.GenerateJsonSchema "pydantic.json_schema.GenerateJsonSchema") ] = GenerateToolJsonSchema, ) -> [Any](https://docs.python.org/3/library/typing.html#typing.Any "typing.Any")` Decorator to register a tool function which DOES NOT take `RunContext` as an argument. Can decorate a sync or async functions. The docstring is inspected to extract both the tool description and description of each parameter, [learn more](../../tools/#function-tools-and-schema) . We can't add overloads for every possible signature of tool, since the return type is a recursive union so the signature of functions decorated with `@agent.tool` is obscured. Example: `from pydantic_ai import Agent, RunContext agent = Agent('test') @agent.tool def foobar(ctx: RunContext[int]) -> int: return 123 @agent.tool(retries=2) async def spam(ctx: RunContext[str]) -> float: return 3.14 result = agent.run_sync('foobar', deps=1) print(result.data) #> {"foobar":123,"spam":3.14}` Parameters: | Name | Type | Description | Default | | --- | --- | --- | --- | | `func` | `[ToolFuncPlain](../tools/#pydantic_ai.tools.ToolFuncPlain "pydantic_ai.tools.ToolFuncPlain") [[ToolParams](../tools/#pydantic_ai.tools.ToolParams "pydantic_ai.tools.ToolParams") ] \| None` | The tool function to register. | `None` | | `name` | `[str](https://docs.python.org/3/library/stdtypes.html#str) \| None` | The name of the tool, defaults to the function name. | `None` | | `retries` | `[int](https://docs.python.org/3/library/functions.html#int) \| None` | The number of retries to allow for this tool, defaults to the agent's default retries, which defaults to 1. | `None` | | `prepare` | `[ToolPrepareFunc](../tools/#pydantic_ai.tools.ToolPrepareFunc "pydantic_ai.tools.ToolPrepareFunc") [[AgentDepsT](../tools/#pydantic_ai.tools.AgentDepsT "pydantic_ai.tools.AgentDepsT") ] \| None` | custom method to prepare the tool definition for each step, return `None` to omit this tool from a given step. This is useful if you want to customise a tool at call time, or omit it completely from a step. See [`ToolPrepareFunc`](../tools/#pydantic_ai.tools.ToolPrepareFunc)
. | `None` | | `docstring_format` | `[DocstringFormat](../tools/#pydantic_ai.tools.DocstringFormat "pydantic_ai.tools.DocstringFormat") ` | The format of the docstring, see [`DocstringFormat`](../tools/#pydantic_ai.tools.DocstringFormat)
. Defaults to `'auto'`, such that the format is inferred from the structure of the docstring. | `'auto'` | | `require_parameter_descriptions` | `[bool](https://docs.python.org/3/library/functions.html#bool) ` | If True, raise an error if a parameter description is missing. Defaults to False. | `False` | | `schema_generator` | `[type](https://docs.python.org/3/library/functions.html#type) [[GenerateJsonSchema](https://docs.pydantic.dev/latest/api/json_schema/#pydantic.json_schema.GenerateJsonSchema "pydantic.json_schema.GenerateJsonSchema") ]` | The JSON schema generator class to use for this tool. Defaults to `GenerateToolJsonSchema`. | `GenerateToolJsonSchema` | Source code in `pydantic_ai_slim/pydantic_ai/agent.py` | | | | --- | --- | | 1041
1042
1043
1044
1045
1046
1047
1048
1049
1050
1051
1052
1053
1054
1055
1056
1057
1058
1059
1060
1061
1062
1063
1064
1065
1066
1067
1068
1069
1070
1071
1072
1073
1074
1075
1076
1077
1078
1079
1080
1081
1082
1083
1084
1085
1086
1087
1088
1089
1090
1091
1092
1093
1094
1095
1096
1097
1098
1099
1100
1101
1102
1103
1104
1105
1106
1107
1108
1109
1110
1111
1112
1113
1114
1115
1116 | ``def tool_plain( self, func: ToolFuncPlain[ToolParams] \| None = None, /, *, name: str \| None = None, retries: int \| None = None, prepare: ToolPrepareFunc[AgentDepsT] \| None = None, docstring_format: DocstringFormat = 'auto', require_parameter_descriptions: bool = False, schema_generator: type[GenerateJsonSchema] = GenerateToolJsonSchema, ) -> Any: """Decorator to register a tool function which DOES NOT take `RunContext` as an argument. Can decorate a sync or async functions. The docstring is inspected to extract both the tool description and description of each parameter, [learn more](../tools.md#function-tools-and-schema). We can't add overloads for every possible signature of tool, since the return type is a recursive union so the signature of functions decorated with `@agent.tool` is obscured. Example: ```python from pydantic_ai import Agent, RunContext agent = Agent('test') @agent.tool def foobar(ctx: RunContext[int]) -> int: return 123 @agent.tool(retries=2) async def spam(ctx: RunContext[str]) -> float: return 3.14 result = agent.run_sync('foobar', deps=1) print(result.data) #> {"foobar":123,"spam":3.14} ``` Args: func: The tool function to register. name: The name of the tool, defaults to the function name. retries: The number of retries to allow for this tool, defaults to the agent's default retries, which defaults to 1. prepare: custom method to prepare the tool definition for each step, return `None` to omit this tool from a given step. This is useful if you want to customise a tool at call time, or omit it completely from a step. See [`ToolPrepareFunc`][pydantic_ai.tools.ToolPrepareFunc]. docstring_format: The format of the docstring, see [`DocstringFormat`][pydantic_ai.tools.DocstringFormat]. Defaults to `'auto'`, such that the format is inferred from the structure of the docstring. require_parameter_descriptions: If True, raise an error if a parameter description is missing. Defaults to False. schema_generator: The JSON schema generator class to use for this tool. Defaults to `GenerateToolJsonSchema`. """ if func is None: def tool_decorator(func_: ToolFuncPlain[ToolParams]) -> ToolFuncPlain[ToolParams]: # noinspection PyTypeChecker self._register_function( func_, False, name, retries, prepare, docstring_format, require_parameter_descriptions, schema_generator, ) return func_ return tool_decorator else: self._register_function( func, False, name, retries, prepare, docstring_format, require_parameter_descriptions, schema_generator ) return func`` | #### is\_model\_request\_node `staticmethod` `is_model_request_node( node: AgentNode[T, S] | [End](../pydantic_graph/nodes/#pydantic_graph.nodes.End "pydantic_graph.End") [FinalResult[S]], ) -> [TypeGuard](https://typing-extensions.readthedocs.io/en/latest/index.html#typing_extensions.TypeGuard "typing_extensions.TypeGuard") [ModelRequestNode[T, S]]` Check if the node is a `ModelRequestNode`, narrowing the type if it is. This method preserves the generic parameters while narrowing the type, unlike a direct call to `isinstance`. Source code in `pydantic_ai_slim/pydantic_ai/agent.py` | | | | --- | --- | | 1250
1251
1252
1253
1254
1255
1256
1257
1258 | ``@staticmethod def is_model_request_node( node: _agent_graph.AgentNode[T, S] \| End[result.FinalResult[S]], ) -> TypeGuard[_agent_graph.ModelRequestNode[T, S]]: """Check if the node is a `ModelRequestNode`, narrowing the type if it is. This method preserves the generic parameters while narrowing the type, unlike a direct call to `isinstance`. """ return isinstance(node, _agent_graph.ModelRequestNode)`` | #### is\_call\_tools\_node `staticmethod` `is_call_tools_node( node: AgentNode[T, S] | [End](../pydantic_graph/nodes/#pydantic_graph.nodes.End "pydantic_graph.End") [FinalResult[S]], ) -> [TypeGuard](https://typing-extensions.readthedocs.io/en/latest/index.html#typing_extensions.TypeGuard "typing_extensions.TypeGuard") [CallToolsNode[T, S]]` Check if the node is a `CallToolsNode`, narrowing the type if it is. This method preserves the generic parameters while narrowing the type, unlike a direct call to `isinstance`. Source code in `pydantic_ai_slim/pydantic_ai/agent.py` | | | | --- | --- | | 1260
1261
1262
1263
1264
1265
1266
1267
1268 | ``@staticmethod def is_call_tools_node( node: _agent_graph.AgentNode[T, S] \| End[result.FinalResult[S]], ) -> TypeGuard[_agent_graph.CallToolsNode[T, S]]: """Check if the node is a `CallToolsNode`, narrowing the type if it is. This method preserves the generic parameters while narrowing the type, unlike a direct call to `isinstance`. """ return isinstance(node, _agent_graph.CallToolsNode)`` | #### is\_user\_prompt\_node `staticmethod` `is_user_prompt_node( node: AgentNode[T, S] | [End](../pydantic_graph/nodes/#pydantic_graph.nodes.End "pydantic_graph.End") [FinalResult[S]], ) -> [TypeGuard](https://typing-extensions.readthedocs.io/en/latest/index.html#typing_extensions.TypeGuard "typing_extensions.TypeGuard") [UserPromptNode[T, S]]` Check if the node is a `UserPromptNode`, narrowing the type if it is. This method preserves the generic parameters while narrowing the type, unlike a direct call to `isinstance`. Source code in `pydantic_ai_slim/pydantic_ai/agent.py` | | | | --- | --- | | 1270
1271
1272
1273
1274
1275
1276
1277
1278 | ``@staticmethod def is_user_prompt_node( node: _agent_graph.AgentNode[T, S] \| End[result.FinalResult[S]], ) -> TypeGuard[_agent_graph.UserPromptNode[T, S]]: """Check if the node is a `UserPromptNode`, narrowing the type if it is. This method preserves the generic parameters while narrowing the type, unlike a direct call to `isinstance`. """ return isinstance(node, _agent_graph.UserPromptNode)`` | #### is\_end\_node `staticmethod` `is_end_node( node: AgentNode[T, S] | [End](../pydantic_graph/nodes/#pydantic_graph.nodes.End "pydantic_graph.End") [FinalResult[S]], ) -> [TypeGuard](https://typing-extensions.readthedocs.io/en/latest/index.html#typing_extensions.TypeGuard "typing_extensions.TypeGuard") [[End](../pydantic_graph/nodes/#pydantic_graph.nodes.End "pydantic_graph.End") [FinalResult[S]]]` Check if the node is a `End`, narrowing the type if it is. This method preserves the generic parameters while narrowing the type, unlike a direct call to `isinstance`. Source code in `pydantic_ai_slim/pydantic_ai/agent.py` | | | | --- | --- | | 1280
1281
1282
1283
1284
1285
1286
1287
1288 | ``@staticmethod def is_end_node( node: _agent_graph.AgentNode[T, S] \| End[result.FinalResult[S]], ) -> TypeGuard[End[result.FinalResult[S]]]: """Check if the node is a `End`, narrowing the type if it is. This method preserves the generic parameters while narrowing the type, unlike a direct call to `isinstance`. """ return isinstance(node, End)`` | #### run\_mcp\_servers `async` `run_mcp_servers() -> [AsyncIterator](https://docs.python.org/3/library/collections.abc.html#collections.abc.AsyncIterator "collections.abc.AsyncIterator") [None]` Run [`MCPServerStdio`s](../mcp/#pydantic_ai.mcp.MCPServerStdio) so they can be used by the agent. Returns: a context manager to start and shutdown the servers. Source code in `pydantic_ai_slim/pydantic_ai/agent.py` | | | | --- | --- | | 1290
1291
1292
1293
1294
1295
1296
1297
1298
1299
1300
1301
1302 | ``@asynccontextmanager async def run_mcp_servers(self) -> AsyncIterator[None]: """Run [`MCPServerStdio`s][pydantic_ai.mcp.MCPServerStdio] so they can be used by the agent. Returns: a context manager to start and shutdown the servers. """ exit_stack = AsyncExitStack() try: for mcp_server in self._mcp_servers: await exit_stack.enter_async_context(mcp_server) yield finally: await exit_stack.aclose()`` | ### AgentRun `dataclass` Bases: `[Generic](https://docs.python.org/3/library/typing.html#typing.Generic "typing.Generic") [[AgentDepsT](../tools/#pydantic_ai.tools.AgentDepsT "pydantic_ai.tools.AgentDepsT") , [ResultDataT](../result/#pydantic_ai.result.ResultDataT "pydantic_ai.result.ResultDataT") ]` A stateful, async-iterable run of an [`Agent`](#pydantic_ai.agent.Agent) . You generally obtain an `AgentRun` instance by calling `async with my_agent.iter(...) as agent_run:`. Once you have an instance, you can use it to iterate through the run's nodes as they execute. When an [`End`](../pydantic_graph/nodes/#pydantic_graph.nodes.End) is reached, the run finishes and [`result`](#pydantic_ai.agent.AgentRun.result) becomes available. Example: `from pydantic_ai import Agent agent = Agent('openai:gpt-4o') async def main(): nodes = [] # Iterate through the run, recording each node along the way: async with agent.iter('What is the capital of France?') as agent_run: async for node in agent_run: nodes.append(node) print(nodes) ''' [ ModelRequestNode( request=ModelRequest( parts=[ UserPromptPart( content='What is the capital of France?', timestamp=datetime.datetime(...), part_kind='user-prompt', ) ], kind='request', ) ), CallToolsNode( model_response=ModelResponse( parts=[TextPart(content='Paris', part_kind='text')], model_name='gpt-4o', timestamp=datetime.datetime(...), kind='response', ) ), End(data=FinalResult(data='Paris', tool_name=None, tool_call_id=None)), ] ''' print(agent_run.result.data) #> Paris` You can also manually drive the iteration using the [`next`](#pydantic_ai.agent.AgentRun.next) method for more granular control. Source code in `pydantic_ai_slim/pydantic_ai/agent.py` | | | | --- | --- | | 1305
1306
1307
1308
1309
1310
1311
1312
1313
1314
1315
1316
1317
1318
1319
1320
1321
1322
1323
1324
1325
1326
1327
1328
1329
1330
1331
1332
1333
1334
1335
1336
1337
1338
1339
1340
1341
1342
1343
1344
1345
1346
1347
1348
1349
1350
1351
1352
1353
1354
1355
1356
1357
1358
1359
1360
1361
1362
1363
1364
1365
1366
1367
1368
1369
1370
1371
1372
1373
1374
1375
1376
1377
1378
1379
1380
1381
1382
1383
1384
1385
1386
1387
1388
1389
1390
1391
1392
1393
1394
1395
1396
1397
1398
1399
1400
1401
1402
1403
1404
1405
1406
1407
1408
1409
1410
1411
1412
1413
1414
1415
1416
1417
1418
1419
1420
1421
1422
1423
1424
1425
1426
1427
1428
1429
1430
1431
1432
1433
1434
1435
1436
1437
1438
1439
1440
1441
1442
1443
1444
1445
1446
1447
1448
1449
1450
1451
1452
1453
1454
1455
1456
1457
1458
1459
1460
1461
1462
1463
1464
1465
1466
1467
1468
1469
1470
1471
1472
1473
1474
1475
1476
1477
1478
1479
1480
1481
1482
1483
1484
1485
1486
1487
1488
1489
1490
1491
1492
1493
1494
1495
1496
1497
1498
1499
1500
1501
1502
1503 | ``@dataclasses.dataclass(repr=False) class AgentRun(Generic[AgentDepsT, ResultDataT]): """A stateful, async-iterable run of an [`Agent`][pydantic_ai.agent.Agent]. You generally obtain an `AgentRun` instance by calling `async with my_agent.iter(...) as agent_run:`. Once you have an instance, you can use it to iterate through the run's nodes as they execute. When an [`End`][pydantic_graph.nodes.End] is reached, the run finishes and [`result`][pydantic_ai.agent.AgentRun.result] becomes available. Example: ```python from pydantic_ai import Agent agent = Agent('openai:gpt-4o') async def main(): nodes = [] # Iterate through the run, recording each node along the way: async with agent.iter('What is the capital of France?') as agent_run: async for node in agent_run: nodes.append(node) print(nodes) ''' [ ModelRequestNode( request=ModelRequest( parts=[ UserPromptPart( content='What is the capital of France?', timestamp=datetime.datetime(...), part_kind='user-prompt', ) ], kind='request', ) ), CallToolsNode( model_response=ModelResponse( parts=[TextPart(content='Paris', part_kind='text')], model_name='gpt-4o', timestamp=datetime.datetime(...), kind='response', ) ), End(data=FinalResult(data='Paris', tool_name=None, tool_call_id=None)), ] ''' print(agent_run.result.data) #> Paris ``` You can also manually drive the iteration using the [`next`][pydantic_ai.agent.AgentRun.next] method for more granular control. """ _graph_run: GraphRun[ _agent_graph.GraphAgentState, _agent_graph.GraphAgentDeps[AgentDepsT, Any], FinalResult[ResultDataT] ] @property def ctx(self) -> GraphRunContext[_agent_graph.GraphAgentState, _agent_graph.GraphAgentDeps[AgentDepsT, Any]]: """The current context of the agent run.""" return GraphRunContext[_agent_graph.GraphAgentState, _agent_graph.GraphAgentDeps[AgentDepsT, Any]]( self._graph_run.state, self._graph_run.deps ) @property def next_node( self, ) -> _agent_graph.AgentNode[AgentDepsT, ResultDataT] \| End[FinalResult[ResultDataT]]: """The next node that will be run in the agent graph. This is the next node that will be used during async iteration, or if a node is not passed to `self.next(...)`. """ next_node = self._graph_run.next_node if isinstance(next_node, End): return next_node if _agent_graph.is_agent_node(next_node): return next_node raise exceptions.AgentRunError(f'Unexpected node type: {type(next_node)}') # pragma: no cover @property def result(self) -> AgentRunResult[ResultDataT] \| None: """The final result of the run if it has ended, otherwise `None`. Once the run returns an [`End`][pydantic_graph.nodes.End] node, `result` is populated with an [`AgentRunResult`][pydantic_ai.agent.AgentRunResult]. """ graph_run_result = self._graph_run.result if graph_run_result is None: return None return AgentRunResult( graph_run_result.output.data, graph_run_result.output.tool_name, graph_run_result.state, self._graph_run.deps.new_message_index, ) def __aiter__( self, ) -> AsyncIterator[_agent_graph.AgentNode[AgentDepsT, ResultDataT] \| End[FinalResult[ResultDataT]]]: """Provide async-iteration over the nodes in the agent run.""" return self async def __anext__( self, ) -> _agent_graph.AgentNode[AgentDepsT, ResultDataT] \| End[FinalResult[ResultDataT]]: """Advance to the next node automatically based on the last returned node.""" next_node = await self._graph_run.__anext__() if _agent_graph.is_agent_node(next_node): return next_node assert isinstance(next_node, End), f'Unexpected node type: {type(next_node)}' return next_node async def next( self, node: _agent_graph.AgentNode[AgentDepsT, ResultDataT], ) -> _agent_graph.AgentNode[AgentDepsT, ResultDataT] \| End[FinalResult[ResultDataT]]: """Manually drive the agent run by passing in the node you want to run next. This lets you inspect or mutate the node before continuing execution, or skip certain nodes under dynamic conditions. The agent run should be stopped when you return an [`End`][pydantic_graph.nodes.End] node. Example: ```python from pydantic_ai import Agent from pydantic_graph import End agent = Agent('openai:gpt-4o') async def main(): async with agent.iter('What is the capital of France?') as agent_run: next_node = agent_run.next_node # start with the first node nodes = [next_node] while not isinstance(next_node, End): next_node = await agent_run.next(next_node) nodes.append(next_node) # Once `next_node` is an End, we've finished: print(nodes) ''' [ UserPromptNode( user_prompt='What is the capital of France?', system_prompts=(), system_prompt_functions=[], system_prompt_dynamic_functions={}, ), ModelRequestNode( request=ModelRequest( parts=[ UserPromptPart( content='What is the capital of France?', timestamp=datetime.datetime(...), part_kind='user-prompt', ) ], kind='request', ) ), CallToolsNode( model_response=ModelResponse( parts=[TextPart(content='Paris', part_kind='text')], model_name='gpt-4o', timestamp=datetime.datetime(...), kind='response', ) ), End(data=FinalResult(data='Paris', tool_name=None, tool_call_id=None)), ] ''' print('Final result:', agent_run.result.data) #> Final result: Paris ``` Args: node: The node to run next in the graph. Returns: The next node returned by the graph logic, or an [`End`][pydantic_graph.nodes.End] node if the run has completed. """ # Note: It might be nice to expose a synchronous interface for iteration, but we shouldn't do it # on this class, or else IDEs won't warn you if you accidentally use `for` instead of `async for` to iterate. next_node = await self._graph_run.next(node) if _agent_graph.is_agent_node(next_node): return next_node assert isinstance(next_node, End), f'Unexpected node type: {type(next_node)}' return next_node def usage(self) -> _usage.Usage: """Get usage statistics for the run so far, including token usage, model requests, and so on.""" return self._graph_run.state.usage def __repr__(self) -> str: result = self._graph_run.result result_repr = '' if result is None else repr(result.output) return f'<{type(self).__name__} result={result_repr} usage={self.usage()}>'`` | #### ctx `property` `ctx: [GraphRunContext](../pydantic_graph/nodes/#pydantic_graph.nodes.GraphRunContext "pydantic_graph.GraphRunContext") [ GraphAgentState, GraphAgentDeps[[AgentDepsT](../tools/#pydantic_ai.tools.AgentDepsT "pydantic_ai.tools.AgentDepsT") , [Any](https://docs.python.org/3/library/typing.html#typing.Any "typing.Any") ] ]` The current context of the agent run. #### next\_node `property` `next_node: ( AgentNode[[AgentDepsT](../tools/#pydantic_ai.tools.AgentDepsT "pydantic_ai.tools.AgentDepsT") , [ResultDataT](../result/#pydantic_ai.result.ResultDataT "pydantic_ai.result.ResultDataT") ] | [End](../pydantic_graph/nodes/#pydantic_graph.nodes.End "pydantic_graph.End") [FinalResult[[ResultDataT](../result/#pydantic_ai.result.ResultDataT "pydantic_ai.result.ResultDataT") ]] )` The next node that will be run in the agent graph. This is the next node that will be used during async iteration, or if a node is not passed to `self.next(...)`. #### result `property` `result: [AgentRunResult](#pydantic_ai.agent.AgentRunResult "pydantic_ai.agent.AgentRunResult") [[ResultDataT](../result/#pydantic_ai.result.ResultDataT "pydantic_ai.result.ResultDataT") ] | None` The final result of the run if it has ended, otherwise `None`. Once the run returns an [`End`](../pydantic_graph/nodes/#pydantic_graph.nodes.End) node, `result` is populated with an [`AgentRunResult`](#pydantic_ai.agent.AgentRunResult) . #### \_\_aiter\_\_ `__aiter__() -> ( [AsyncIterator](https://docs.python.org/3/library/collections.abc.html#collections.abc.AsyncIterator "collections.abc.AsyncIterator") [ AgentNode[[AgentDepsT](../tools/#pydantic_ai.tools.AgentDepsT "pydantic_ai.tools.AgentDepsT") , [ResultDataT](../result/#pydantic_ai.result.ResultDataT "pydantic_ai.result.ResultDataT") ] | [End](../pydantic_graph/nodes/#pydantic_graph.nodes.End "pydantic_graph.End") [FinalResult[[ResultDataT](../result/#pydantic_ai.result.ResultDataT "pydantic_ai.result.ResultDataT") ]] ] )` Provide async-iteration over the nodes in the agent run. Source code in `pydantic_ai_slim/pydantic_ai/agent.py` | | | | --- | --- | | 1404
1405
1406
1407
1408 | `def __aiter__( self, ) -> AsyncIterator[_agent_graph.AgentNode[AgentDepsT, ResultDataT] \| End[FinalResult[ResultDataT]]]: """Provide async-iteration over the nodes in the agent run.""" return self` | #### \_\_anext\_\_ `async` `__anext__() -> ( AgentNode[[AgentDepsT](../tools/#pydantic_ai.tools.AgentDepsT "pydantic_ai.tools.AgentDepsT") , [ResultDataT](../result/#pydantic_ai.result.ResultDataT "pydantic_ai.result.ResultDataT") ] | [End](../pydantic_graph/nodes/#pydantic_graph.nodes.End "pydantic_graph.End") [FinalResult[[ResultDataT](../result/#pydantic_ai.result.ResultDataT "pydantic_ai.result.ResultDataT") ]] )` Advance to the next node automatically based on the last returned node. Source code in `pydantic_ai_slim/pydantic_ai/agent.py` | | | | --- | --- | | 1410
1411
1412
1413
1414
1415
1416
1417
1418 | `async def __anext__( self, ) -> _agent_graph.AgentNode[AgentDepsT, ResultDataT] \| End[FinalResult[ResultDataT]]: """Advance to the next node automatically based on the last returned node.""" next_node = await self._graph_run.__anext__() if _agent_graph.is_agent_node(next_node): return next_node assert isinstance(next_node, End), f'Unexpected node type: {type(next_node)}' return next_node` | #### next `async` `next( node: AgentNode[[AgentDepsT](../tools/#pydantic_ai.tools.AgentDepsT "pydantic_ai.tools.AgentDepsT") , [ResultDataT](../result/#pydantic_ai.result.ResultDataT "pydantic_ai.result.ResultDataT") ], ) -> ( AgentNode[[AgentDepsT](../tools/#pydantic_ai.tools.AgentDepsT "pydantic_ai.tools.AgentDepsT") , [ResultDataT](../result/#pydantic_ai.result.ResultDataT "pydantic_ai.result.ResultDataT") ] | [End](../pydantic_graph/nodes/#pydantic_graph.nodes.End "pydantic_graph.End") [FinalResult[[ResultDataT](../result/#pydantic_ai.result.ResultDataT "pydantic_ai.result.ResultDataT") ]] )` Manually drive the agent run by passing in the node you want to run next. This lets you inspect or mutate the node before continuing execution, or skip certain nodes under dynamic conditions. The agent run should be stopped when you return an [`End`](../pydantic_graph/nodes/#pydantic_graph.nodes.End) node. Example: ``from pydantic_ai import Agent from pydantic_graph import End agent = Agent('openai:gpt-4o') async def main(): async with agent.iter('What is the capital of France?') as agent_run: next_node = agent_run.next_node # start with the first node nodes = [next_node] while not isinstance(next_node, End): next_node = await agent_run.next(next_node) nodes.append(next_node) # Once `next_node` is an End, we've finished: print(nodes) ''' [ UserPromptNode( user_prompt='What is the capital of France?', system_prompts=(), system_prompt_functions=[], system_prompt_dynamic_functions={}, ), ModelRequestNode( request=ModelRequest( parts=[ UserPromptPart( content='What is the capital of France?', timestamp=datetime.datetime(...), part_kind='user-prompt', ) ], kind='request', ) ), CallToolsNode( model_response=ModelResponse( parts=[TextPart(content='Paris', part_kind='text')], model_name='gpt-4o', timestamp=datetime.datetime(...), kind='response', ) ), End(data=FinalResult(data='Paris', tool_name=None, tool_call_id=None)), ] ''' print('Final result:', agent_run.result.data) #> Final result: Paris`` Parameters: | Name | Type | Description | Default | | --- | --- | --- | --- | | `node` | `AgentNode[[AgentDepsT](../tools/#pydantic_ai.tools.AgentDepsT "pydantic_ai.tools.AgentDepsT") , [ResultDataT](../result/#pydantic_ai.result.ResultDataT "pydantic_ai.result.ResultDataT") ]` | The node to run next in the graph. | _required_ | Returns: | Type | Description | | --- | --- | | `AgentNode[[AgentDepsT](../tools/#pydantic_ai.tools.AgentDepsT "pydantic_ai.tools.AgentDepsT") , [ResultDataT](../result/#pydantic_ai.result.ResultDataT "pydantic_ai.result.ResultDataT") ] \| [End](../pydantic_graph/nodes/#pydantic_graph.nodes.End "pydantic_graph.End") [FinalResult[[ResultDataT](../result/#pydantic_ai.result.ResultDataT "pydantic_ai.result.ResultDataT") ]]` | The next node returned by the graph logic, or an [`End`](../pydantic_graph/nodes/#pydantic_graph.nodes.End)
node if | | `AgentNode[[AgentDepsT](../tools/#pydantic_ai.tools.AgentDepsT "pydantic_ai.tools.AgentDepsT") , [ResultDataT](../result/#pydantic_ai.result.ResultDataT "pydantic_ai.result.ResultDataT") ] \| [End](../pydantic_graph/nodes/#pydantic_graph.nodes.End "pydantic_graph.End") [FinalResult[[ResultDataT](../result/#pydantic_ai.result.ResultDataT "pydantic_ai.result.ResultDataT") ]]` | the run has completed. | Source code in `pydantic_ai_slim/pydantic_ai/agent.py` | | | | --- | --- | | 1420
1421
1422
1423
1424
1425
1426
1427
1428
1429
1430
1431
1432
1433
1434
1435
1436
1437
1438
1439
1440
1441
1442
1443
1444
1445
1446
1447
1448
1449
1450
1451
1452
1453
1454
1455
1456
1457
1458
1459
1460
1461
1462
1463
1464
1465
1466
1467
1468
1469
1470
1471
1472
1473
1474
1475
1476
1477
1478
1479
1480
1481
1482
1483
1484
1485
1486
1487
1488
1489
1490
1491
1492
1493
1494 | ``async def next( self, node: _agent_graph.AgentNode[AgentDepsT, ResultDataT], ) -> _agent_graph.AgentNode[AgentDepsT, ResultDataT] \| End[FinalResult[ResultDataT]]: """Manually drive the agent run by passing in the node you want to run next. This lets you inspect or mutate the node before continuing execution, or skip certain nodes under dynamic conditions. The agent run should be stopped when you return an [`End`][pydantic_graph.nodes.End] node. Example: ```python from pydantic_ai import Agent from pydantic_graph import End agent = Agent('openai:gpt-4o') async def main(): async with agent.iter('What is the capital of France?') as agent_run: next_node = agent_run.next_node # start with the first node nodes = [next_node] while not isinstance(next_node, End): next_node = await agent_run.next(next_node) nodes.append(next_node) # Once `next_node` is an End, we've finished: print(nodes) ''' [ UserPromptNode( user_prompt='What is the capital of France?', system_prompts=(), system_prompt_functions=[], system_prompt_dynamic_functions={}, ), ModelRequestNode( request=ModelRequest( parts=[ UserPromptPart( content='What is the capital of France?', timestamp=datetime.datetime(...), part_kind='user-prompt', ) ], kind='request', ) ), CallToolsNode( model_response=ModelResponse( parts=[TextPart(content='Paris', part_kind='text')], model_name='gpt-4o', timestamp=datetime.datetime(...), kind='response', ) ), End(data=FinalResult(data='Paris', tool_name=None, tool_call_id=None)), ] ''' print('Final result:', agent_run.result.data) #> Final result: Paris ``` Args: node: The node to run next in the graph. Returns: The next node returned by the graph logic, or an [`End`][pydantic_graph.nodes.End] node if the run has completed. """ # Note: It might be nice to expose a synchronous interface for iteration, but we shouldn't do it # on this class, or else IDEs won't warn you if you accidentally use `for` instead of `async for` to iterate. next_node = await self._graph_run.next(node) if _agent_graph.is_agent_node(next_node): return next_node assert isinstance(next_node, End), f'Unexpected node type: {type(next_node)}' return next_node`` | #### usage `usage() -> [Usage](../usage/#pydantic_ai.usage.Usage "pydantic_ai.usage.Usage")` Get usage statistics for the run so far, including token usage, model requests, and so on. Source code in `pydantic_ai_slim/pydantic_ai/agent.py` | | | | --- | --- | | 1496
1497
1498 | `def usage(self) -> _usage.Usage: """Get usage statistics for the run so far, including token usage, model requests, and so on.""" return self._graph_run.state.usage` | ### AgentRunResult `dataclass` Bases: `[Generic](https://docs.python.org/3/library/typing.html#typing.Generic "typing.Generic") [[ResultDataT](../result/#pydantic_ai.result.ResultDataT "pydantic_ai.result.ResultDataT") ]` The final result of an agent run. Source code in `pydantic_ai_slim/pydantic_ai/agent.py` | | | | --- | --- | | 1506
1507
1508
1509
1510
1511
1512
1513
1514
1515
1516
1517
1518
1519
1520
1521
1522
1523
1524
1525
1526
1527
1528
1529
1530
1531
1532
1533
1534
1535
1536
1537
1538
1539
1540
1541
1542
1543
1544
1545
1546
1547
1548
1549
1550
1551
1552
1553
1554
1555
1556
1557
1558
1559
1560
1561
1562
1563
1564
1565
1566
1567
1568
1569
1570
1571
1572
1573
1574
1575
1576
1577
1578
1579
1580
1581
1582
1583
1584
1585
1586
1587
1588
1589
1590
1591
1592
1593
1594
1595
1596
1597
1598 | ``@dataclasses.dataclass class AgentRunResult(Generic[ResultDataT]): """The final result of an agent run.""" data: ResultDataT # TODO: rename this to output. I'm putting this off for now mostly to reduce the size of the diff _result_tool_name: str \| None = dataclasses.field(repr=False) _state: _agent_graph.GraphAgentState = dataclasses.field(repr=False) _new_message_index: int = dataclasses.field(repr=False) def _set_result_tool_return(self, return_content: str) -> list[_messages.ModelMessage]: """Set return content for the result tool. Useful if you want to continue the conversation and want to set the response to the result tool call. """ if not self._result_tool_name: raise ValueError('Cannot set result tool return content when the return type is `str`.') messages = deepcopy(self._state.message_history) last_message = messages[-1] for part in last_message.parts: if isinstance(part, _messages.ToolReturnPart) and part.tool_name == self._result_tool_name: part.content = return_content return messages raise LookupError(f'No tool call found with tool name {self._result_tool_name!r}.') def all_messages(self, *, result_tool_return_content: str \| None = None) -> list[_messages.ModelMessage]: """Return the history of _messages. Args: result_tool_return_content: The return content of the tool call to set in the last message. This provides a convenient way to modify the content of the result tool call if you want to continue the conversation and want to set the response to the result tool call. If `None`, the last message will not be modified. Returns: List of messages. """ if result_tool_return_content is not None: return self._set_result_tool_return(result_tool_return_content) else: return self._state.message_history def all_messages_json(self, *, result_tool_return_content: str \| None = None) -> bytes: """Return all messages from [`all_messages`][pydantic_ai.agent.AgentRunResult.all_messages] as JSON bytes. Args: result_tool_return_content: The return content of the tool call to set in the last message. This provides a convenient way to modify the content of the result tool call if you want to continue the conversation and want to set the response to the result tool call. If `None`, the last message will not be modified. Returns: JSON bytes representing the messages. """ return _messages.ModelMessagesTypeAdapter.dump_json( self.all_messages(result_tool_return_content=result_tool_return_content) ) def new_messages(self, *, result_tool_return_content: str \| None = None) -> list[_messages.ModelMessage]: """Return new messages associated with this run. Messages from older runs are excluded. Args: result_tool_return_content: The return content of the tool call to set in the last message. This provides a convenient way to modify the content of the result tool call if you want to continue the conversation and want to set the response to the result tool call. If `None`, the last message will not be modified. Returns: List of new messages. """ return self.all_messages(result_tool_return_content=result_tool_return_content)[self._new_message_index :] def new_messages_json(self, *, result_tool_return_content: str \| None = None) -> bytes: """Return new messages from [`new_messages`][pydantic_ai.agent.AgentRunResult.new_messages] as JSON bytes. Args: result_tool_return_content: The return content of the tool call to set in the last message. This provides a convenient way to modify the content of the result tool call if you want to continue the conversation and want to set the response to the result tool call. If `None`, the last message will not be modified. Returns: JSON bytes representing the new messages. """ return _messages.ModelMessagesTypeAdapter.dump_json( self.new_messages(result_tool_return_content=result_tool_return_content) ) def usage(self) -> _usage.Usage: """Return the usage of the whole run.""" return self._state.usage`` | #### all\_messages `all_messages( *, result_tool_return_content: [str](https://docs.python.org/3/library/stdtypes.html#str) | None = None ) -> [list](https://docs.python.org/3/library/stdtypes.html#list) [[ModelMessage](../messages/#pydantic_ai.messages.ModelMessage "pydantic_ai.messages.ModelMessage") ]` Return the history of \_messages. Parameters: | Name | Type | Description | Default | | --- | --- | --- | --- | | `result_tool_return_content` | `[str](https://docs.python.org/3/library/stdtypes.html#str) \| None` | The return content of the tool call to set in the last message. This provides a convenient way to modify the content of the result tool call if you want to continue the conversation and want to set the response to the result tool call. If `None`, the last message will not be modified. | `None` | Returns: | Type | Description | | --- | --- | | `[list](https://docs.python.org/3/library/stdtypes.html#list) [[ModelMessage](../messages/#pydantic_ai.messages.ModelMessage "pydantic_ai.messages.ModelMessage") ]` | List of messages. | Source code in `pydantic_ai_slim/pydantic_ai/agent.py` | | | | --- | --- | | 1531
1532
1533
1534
1535
1536
1537
1538
1539
1540
1541
1542
1543
1544
1545
1546 | ``def all_messages(self, *, result_tool_return_content: str \| None = None) -> list[_messages.ModelMessage]: """Return the history of _messages. Args: result_tool_return_content: The return content of the tool call to set in the last message. This provides a convenient way to modify the content of the result tool call if you want to continue the conversation and want to set the response to the result tool call. If `None`, the last message will not be modified. Returns: List of messages. """ if result_tool_return_content is not None: return self._set_result_tool_return(result_tool_return_content) else: return self._state.message_history`` | #### all\_messages\_json `all_messages_json( *, result_tool_return_content: [str](https://docs.python.org/3/library/stdtypes.html#str) | None = None ) -> [bytes](https://docs.python.org/3/library/stdtypes.html#bytes)` Return all messages from [`all_messages`](#pydantic_ai.agent.AgentRunResult.all_messages) as JSON bytes. Parameters: | Name | Type | Description | Default | | --- | --- | --- | --- | | `result_tool_return_content` | `[str](https://docs.python.org/3/library/stdtypes.html#str) \| None` | The return content of the tool call to set in the last message. This provides a convenient way to modify the content of the result tool call if you want to continue the conversation and want to set the response to the result tool call. If `None`, the last message will not be modified. | `None` | Returns: | Type | Description | | --- | --- | | `[bytes](https://docs.python.org/3/library/stdtypes.html#bytes) ` | JSON bytes representing the messages. | Source code in `pydantic_ai_slim/pydantic_ai/agent.py` | | | | --- | --- | | 1548
1549
1550
1551
1552
1553
1554
1555
1556
1557
1558
1559
1560
1561
1562 | ``def all_messages_json(self, *, result_tool_return_content: str \| None = None) -> bytes: """Return all messages from [`all_messages`][pydantic_ai.agent.AgentRunResult.all_messages] as JSON bytes. Args: result_tool_return_content: The return content of the tool call to set in the last message. This provides a convenient way to modify the content of the result tool call if you want to continue the conversation and want to set the response to the result tool call. If `None`, the last message will not be modified. Returns: JSON bytes representing the messages. """ return _messages.ModelMessagesTypeAdapter.dump_json( self.all_messages(result_tool_return_content=result_tool_return_content) )`` | #### new\_messages `new_messages( *, result_tool_return_content: [str](https://docs.python.org/3/library/stdtypes.html#str) | None = None ) -> [list](https://docs.python.org/3/library/stdtypes.html#list) [[ModelMessage](../messages/#pydantic_ai.messages.ModelMessage "pydantic_ai.messages.ModelMessage") ]` Return new messages associated with this run. Messages from older runs are excluded. Parameters: | Name | Type | Description | Default | | --- | --- | --- | --- | | `result_tool_return_content` | `[str](https://docs.python.org/3/library/stdtypes.html#str) \| None` | The return content of the tool call to set in the last message. This provides a convenient way to modify the content of the result tool call if you want to continue the conversation and want to set the response to the result tool call. If `None`, the last message will not be modified. | `None` | Returns: | Type | Description | | --- | --- | | `[list](https://docs.python.org/3/library/stdtypes.html#list) [[ModelMessage](../messages/#pydantic_ai.messages.ModelMessage "pydantic_ai.messages.ModelMessage") ]` | List of new messages. | Source code in `pydantic_ai_slim/pydantic_ai/agent.py` | | | | --- | --- | | 1564
1565
1566
1567
1568
1569
1570
1571
1572
1573
1574
1575
1576
1577
1578 | ``def new_messages(self, *, result_tool_return_content: str \| None = None) -> list[_messages.ModelMessage]: """Return new messages associated with this run. Messages from older runs are excluded. Args: result_tool_return_content: The return content of the tool call to set in the last message. This provides a convenient way to modify the content of the result tool call if you want to continue the conversation and want to set the response to the result tool call. If `None`, the last message will not be modified. Returns: List of new messages. """ return self.all_messages(result_tool_return_content=result_tool_return_content)[self._new_message_index :]`` | #### new\_messages\_json `new_messages_json( *, result_tool_return_content: [str](https://docs.python.org/3/library/stdtypes.html#str) | None = None ) -> [bytes](https://docs.python.org/3/library/stdtypes.html#bytes)` Return new messages from [`new_messages`](#pydantic_ai.agent.AgentRunResult.new_messages) as JSON bytes. Parameters: | Name | Type | Description | Default | | --- | --- | --- | --- | | `result_tool_return_content` | `[str](https://docs.python.org/3/library/stdtypes.html#str) \| None` | The return content of the tool call to set in the last message. This provides a convenient way to modify the content of the result tool call if you want to continue the conversation and want to set the response to the result tool call. If `None`, the last message will not be modified. | `None` | Returns: | Type | Description | | --- | --- | | `[bytes](https://docs.python.org/3/library/stdtypes.html#bytes) ` | JSON bytes representing the new messages. | Source code in `pydantic_ai_slim/pydantic_ai/agent.py` | | | | --- | --- | | 1580
1581
1582
1583
1584
1585
1586
1587
1588
1589
1590
1591
1592
1593
1594 | ``def new_messages_json(self, *, result_tool_return_content: str \| None = None) -> bytes: """Return new messages from [`new_messages`][pydantic_ai.agent.AgentRunResult.new_messages] as JSON bytes. Args: result_tool_return_content: The return content of the tool call to set in the last message. This provides a convenient way to modify the content of the result tool call if you want to continue the conversation and want to set the response to the result tool call. If `None`, the last message will not be modified. Returns: JSON bytes representing the new messages. """ return _messages.ModelMessagesTypeAdapter.dump_json( self.new_messages(result_tool_return_content=result_tool_return_content) )`` | #### usage `usage() -> [Usage](../usage/#pydantic_ai.usage.Usage "pydantic_ai.usage.Usage")` Return the usage of the whole run. Source code in `pydantic_ai_slim/pydantic_ai/agent.py` | | | | --- | --- | | 1596
1597
1598 | `def usage(self) -> _usage.Usage: """Return the usage of the whole run.""" return self._state.usage` | ### EndStrategy `module-attribute` `EndStrategy = EndStrategy` ### RunResultDataT `module-attribute` `RunResultDataT = TypeVar('RunResultDataT')` Type variable for the result data of a run where `result_type` was customized on the run call. ### capture\_run\_messages `module-attribute` `capture_run_messages = capture_run_messages` ### InstrumentationSettings `dataclass` Options for instrumenting models and agents with OpenTelemetry. Used in: * `Agent(instrument=...)` * [`Agent.instrument_all()`](#pydantic_ai.agent.Agent.instrument_all) * [`InstrumentedModel`](../models/instrumented/#pydantic_ai.models.instrumented.InstrumentedModel) See the [Debugging and Monitoring guide](https://ai.pydantic.dev/logfire/) for more info. Source code in `pydantic_ai_slim/pydantic_ai/models/instrumented.py` | | | | --- | --- | | 52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95 | ``@dataclass(init=False) class InstrumentationSettings: """Options for instrumenting models and agents with OpenTelemetry. Used in: - `Agent(instrument=...)` - [`Agent.instrument_all()`][pydantic_ai.agent.Agent.instrument_all] - [`InstrumentedModel`][pydantic_ai.models.instrumented.InstrumentedModel] See the [Debugging and Monitoring guide](https://ai.pydantic.dev/logfire/) for more info. """ tracer: Tracer = field(repr=False) event_logger: EventLogger = field(repr=False) event_mode: Literal['attributes', 'logs'] = 'attributes' def __init__( self, *, event_mode: Literal['attributes', 'logs'] = 'attributes', tracer_provider: TracerProvider \| None = None, event_logger_provider: EventLoggerProvider \| None = None, ): """Create instrumentation options. Args: event_mode: The mode for emitting events. If `'attributes'`, events are attached to the span as attributes. If `'logs'`, events are emitted as OpenTelemetry log-based events. tracer_provider: The OpenTelemetry tracer provider to use. If not provided, the global tracer provider is used. Calling `logfire.configure()` sets the global tracer provider, so most users don't need this. event_logger_provider: The OpenTelemetry event logger provider to use. If not provided, the global event logger provider is used. Calling `logfire.configure()` sets the global event logger provider, so most users don't need this. This is only used if `event_mode='logs'`. """ from pydantic_ai import __version__ tracer_provider = tracer_provider or get_tracer_provider() event_logger_provider = event_logger_provider or get_event_logger_provider() self.tracer = tracer_provider.get_tracer('pydantic-ai', __version__) self.event_logger = event_logger_provider.get_event_logger('pydantic-ai', __version__) self.event_mode = event_mode`` | #### \_\_init\_\_ `__init__( *, event_mode: [Literal](https://docs.python.org/3/library/typing.html#typing.Literal "typing.Literal") [ "attributes", "logs" ] = "attributes", tracer_provider: TracerProvider | None = None, event_logger_provider: EventLoggerProvider | None = None )` Create instrumentation options. Parameters: | Name | Type | Description | Default | | --- | --- | --- | --- | | `event_mode` | `[Literal](https://docs.python.org/3/library/typing.html#typing.Literal "typing.Literal") ['attributes', 'logs']` | The mode for emitting events. If `'attributes'`, events are attached to the span as attributes. If `'logs'`, events are emitted as OpenTelemetry log-based events. | `'attributes'` | | `tracer_provider` | `TracerProvider \| None` | The OpenTelemetry tracer provider to use. If not provided, the global tracer provider is used. Calling `logfire.configure()` sets the global tracer provider, so most users don't need this. | `None` | | `event_logger_provider` | `EventLoggerProvider \| None` | The OpenTelemetry event logger provider to use. If not provided, the global event logger provider is used. Calling `logfire.configure()` sets the global event logger provider, so most users don't need this. This is only used if `event_mode='logs'`. | `None` | Source code in `pydantic_ai_slim/pydantic_ai/models/instrumented.py` | | | | --- | --- | | 69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95 | ``def __init__( self, *, event_mode: Literal['attributes', 'logs'] = 'attributes', tracer_provider: TracerProvider \| None = None, event_logger_provider: EventLoggerProvider \| None = None, ): """Create instrumentation options. Args: event_mode: The mode for emitting events. If `'attributes'`, events are attached to the span as attributes. If `'logs'`, events are emitted as OpenTelemetry log-based events. tracer_provider: The OpenTelemetry tracer provider to use. If not provided, the global tracer provider is used. Calling `logfire.configure()` sets the global tracer provider, so most users don't need this. event_logger_provider: The OpenTelemetry event logger provider to use. If not provided, the global event logger provider is used. Calling `logfire.configure()` sets the global event logger provider, so most users don't need this. This is only used if `event_mode='logs'`. """ from pydantic_ai import __version__ tracer_provider = tracer_provider or get_tracer_provider() event_logger_provider = event_logger_provider or get_event_logger_provider() self.tracer = tracer_provider.get_tracer('pydantic-ai', __version__) self.event_logger = event_logger_provider.get_event_logger('pydantic-ai', __version__) self.event_mode = event_mode`` | --- # pydantic_ai.messages - PydanticAI `pydantic_ai.messages` ====================== The structure of [`ModelMessage`](#pydantic_ai.messages.ModelMessage) can be shown as a graph: graph RL SystemPromptPart(SystemPromptPart) --- ModelRequestPart UserPromptPart(UserPromptPart) --- ModelRequestPart ToolReturnPart(ToolReturnPart) --- ModelRequestPart RetryPromptPart(RetryPromptPart) --- ModelRequestPart TextPart(TextPart) --- ModelResponsePart ToolCallPart(ToolCallPart) --- ModelResponsePart ModelRequestPart("ModelRequestPart
(Union)") --- ModelRequest ModelRequest("ModelRequest(parts=list[...])") --- ModelMessage ModelResponsePart("ModelResponsePart
(Union)") --- ModelResponse ModelResponse("ModelResponse(parts=list[...])") --- ModelMessage("ModelMessage
(Union)") ### SystemPromptPart `dataclass` A system prompt, generally written by the application developer. This gives the model context and guidance on how to respond. Source code in `pydantic_ai_slim/pydantic_ai/messages.py` | | | | --- | --- | | 19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42 | ``@dataclass class SystemPromptPart: """A system prompt, generally written by the application developer. This gives the model context and guidance on how to respond. """ content: str """The content of the prompt.""" timestamp: datetime = field(default_factory=_now_utc) """The timestamp of the prompt.""" dynamic_ref: str \| None = None """The ref of the dynamic system prompt function that generated this part. Only set if system prompt is dynamic, see [`system_prompt`][pydantic_ai.Agent.system_prompt] for more information. """ part_kind: Literal['system-prompt'] = 'system-prompt' """Part type identifier, this is available on all parts as a discriminator.""" def otel_event(self) -> Event: return Event('gen_ai.system.message', body={'content': self.content, 'role': 'system'})`` | #### content `instance-attribute` `content: [str](https://docs.python.org/3/library/stdtypes.html#str)` The content of the prompt. #### timestamp `class-attribute` `instance-attribute` `timestamp: [datetime](https://docs.python.org/3/library/datetime.html#datetime.datetime "datetime.datetime") = [field](https://docs.python.org/3/library/dataclasses.html#dataclasses.field "dataclasses.field") (default_factory=now_utc)` The timestamp of the prompt. #### dynamic\_ref `class-attribute` `instance-attribute` `dynamic_ref: [str](https://docs.python.org/3/library/stdtypes.html#str) | None = None` The ref of the dynamic system prompt function that generated this part. Only set if system prompt is dynamic, see [`system_prompt`](../agent/#pydantic_ai.agent.Agent.system_prompt) for more information. #### part\_kind `class-attribute` `instance-attribute` `part_kind: [Literal](https://docs.python.org/3/library/typing.html#typing.Literal "typing.Literal") ['system-prompt'] = 'system-prompt'` Part type identifier, this is available on all parts as a discriminator. ### AudioUrl `dataclass` A URL to an audio file. Source code in `pydantic_ai_slim/pydantic_ai/messages.py` | | | | --- | --- | | 45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63 | `@dataclass class AudioUrl: """A URL to an audio file.""" url: str """The URL of the audio file.""" kind: Literal['audio-url'] = 'audio-url' """Type identifier, this is available on all parts as a discriminator.""" @property def media_type(self) -> AudioMediaType: """Return the media type of the audio file, based on the url.""" if self.url.endswith('.mp3'): return 'audio/mpeg' elif self.url.endswith('.wav'): return 'audio/wav' else: raise ValueError(f'Unknown audio file extension: {self.url}')` | #### url `instance-attribute` `url: [str](https://docs.python.org/3/library/stdtypes.html#str)` The URL of the audio file. #### kind `class-attribute` `instance-attribute` `kind: [Literal](https://docs.python.org/3/library/typing.html#typing.Literal "typing.Literal") ['audio-url'] = 'audio-url'` Type identifier, this is available on all parts as a discriminator. #### media\_type `property` `media_type: AudioMediaType` Return the media type of the audio file, based on the url. ### ImageUrl `dataclass` A URL to an image. Source code in `pydantic_ai_slim/pydantic_ai/messages.py` | | | | --- | --- | | 66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96 | `@dataclass class ImageUrl: """A URL to an image.""" url: str """The URL of the image.""" kind: Literal['image-url'] = 'image-url' """Type identifier, this is available on all parts as a discriminator.""" @property def media_type(self) -> ImageMediaType: """Return the media type of the image, based on the url.""" if self.url.endswith(('.jpg', '.jpeg')): return 'image/jpeg' elif self.url.endswith('.png'): return 'image/png' elif self.url.endswith('.gif'): return 'image/gif' elif self.url.endswith('.webp'): return 'image/webp' else: raise ValueError(f'Unknown image file extension: {self.url}') @property def format(self) -> ImageFormat: """The file format of the image. The choice of supported formats were based on the Bedrock Converse API. Other APIs don't require to use a format. """ return _image_format(self.media_type)` | #### url `instance-attribute` `url: [str](https://docs.python.org/3/library/stdtypes.html#str)` The URL of the image. #### kind `class-attribute` `instance-attribute` `kind: [Literal](https://docs.python.org/3/library/typing.html#typing.Literal "typing.Literal") ['image-url'] = 'image-url'` Type identifier, this is available on all parts as a discriminator. #### media\_type `property` `media_type: ImageMediaType` Return the media type of the image, based on the url. #### format `property` `format: ImageFormat` The file format of the image. The choice of supported formats were based on the Bedrock Converse API. Other APIs don't require to use a format. ### DocumentUrl `dataclass` The URL of the document. Source code in `pydantic_ai_slim/pydantic_ai/messages.py` | | | | --- | --- | | 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123 | `@dataclass class DocumentUrl: """The URL of the document.""" url: str """The URL of the document.""" kind: Literal['document-url'] = 'document-url' """Type identifier, this is available on all parts as a discriminator.""" @property def media_type(self) -> str: """Return the media type of the document, based on the url.""" type_, _ = guess_type(self.url) if type_ is None: raise RuntimeError(f'Unknown document file extension: {self.url}') return type_ @property def format(self) -> DocumentFormat: """The file format of the document. The choice of supported formats were based on the Bedrock Converse API. Other APIs don't require to use a format. """ return _document_format(self.media_type)` | #### url `instance-attribute` `url: [str](https://docs.python.org/3/library/stdtypes.html#str)` The URL of the document. #### kind `class-attribute` `instance-attribute` `kind: [Literal](https://docs.python.org/3/library/typing.html#typing.Literal "typing.Literal") ['document-url'] = 'document-url'` Type identifier, this is available on all parts as a discriminator. #### media\_type `property` `media_type: [str](https://docs.python.org/3/library/stdtypes.html#str)` Return the media type of the document, based on the url. #### format `property` `format: DocumentFormat` The file format of the document. The choice of supported formats were based on the Bedrock Converse API. Other APIs don't require to use a format. ### BinaryContent `dataclass` Binary content, e.g. an audio or image file. Source code in `pydantic_ai_slim/pydantic_ai/messages.py` | | | | --- | --- | | 143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192 | ``@dataclass class BinaryContent: """Binary content, e.g. an audio or image file.""" data: bytes """The binary data.""" media_type: AudioMediaType \| ImageMediaType \| DocumentMediaType \| str """The media type of the binary data.""" kind: Literal['binary'] = 'binary' """Type identifier, this is available on all parts as a discriminator.""" @property def is_audio(self) -> bool: """Return `True` if the media type is an audio type.""" return self.media_type.startswith('audio/') @property def is_image(self) -> bool: """Return `True` if the media type is an image type.""" return self.media_type.startswith('image/') @property def is_document(self) -> bool: """Return `True` if the media type is a document type.""" return self.media_type in { 'application/pdf', 'text/plain', 'text/csv', 'application/vnd.openxmlformats-officedocument.wordprocessingml.document', 'application/vnd.openxmlformats-officedocument.spreadsheetml.sheet', 'text/html', 'text/markdown', 'application/vnd.ms-excel', } @property def format(self) -> str: """The file format of the binary content.""" if self.is_audio: if self.media_type == 'audio/mpeg': return 'mp3' elif self.media_type == 'audio/wav': return 'wav' elif self.is_image: return _image_format(self.media_type) elif self.is_document: return _document_format(self.media_type) raise ValueError(f'Unknown media type: {self.media_type}')`` | #### data `instance-attribute` `data: [bytes](https://docs.python.org/3/library/stdtypes.html#bytes)` The binary data. #### media\_type `instance-attribute` `media_type: ( AudioMediaType | ImageMediaType | DocumentMediaType | [str](https://docs.python.org/3/library/stdtypes.html#str) )` The media type of the binary data. #### kind `class-attribute` `instance-attribute` `kind: [Literal](https://docs.python.org/3/library/typing.html#typing.Literal "typing.Literal") ['binary'] = 'binary'` Type identifier, this is available on all parts as a discriminator. #### is\_audio `property` `is_audio: [bool](https://docs.python.org/3/library/functions.html#bool)` Return `True` if the media type is an audio type. #### is\_image `property` `is_image: [bool](https://docs.python.org/3/library/functions.html#bool)` Return `True` if the media type is an image type. #### is\_document `property` `is_document: [bool](https://docs.python.org/3/library/functions.html#bool)` Return `True` if the media type is a document type. #### format `property` `format: [str](https://docs.python.org/3/library/stdtypes.html#str)` The file format of the binary content. ### UserPromptPart `dataclass` A user prompt, generally written by the end user. Content comes from the `user_prompt` parameter of [`Agent.run`](../agent/#pydantic_ai.agent.Agent.run) , [`Agent.run_sync`](../agent/#pydantic_ai.agent.Agent.run_sync) , and [`Agent.run_stream`](../agent/#pydantic_ai.agent.Agent.run_stream) . Source code in `pydantic_ai_slim/pydantic_ai/messages.py` | | | | --- | --- | | 232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255 | ``@dataclass class UserPromptPart: """A user prompt, generally written by the end user. Content comes from the `user_prompt` parameter of [`Agent.run`][pydantic_ai.Agent.run], [`Agent.run_sync`][pydantic_ai.Agent.run_sync], and [`Agent.run_stream`][pydantic_ai.Agent.run_stream]. """ content: str \| Sequence[UserContent] """The content of the prompt.""" timestamp: datetime = field(default_factory=_now_utc) """The timestamp of the prompt.""" part_kind: Literal['user-prompt'] = 'user-prompt' """Part type identifier, this is available on all parts as a discriminator.""" def otel_event(self) -> Event: if isinstance(self.content, str): content = self.content else: # TODO figure out what to record for images and audio content = [part if isinstance(part, str) else {'kind': part.kind} for part in self.content] return Event('gen_ai.user.message', body={'content': content, 'role': 'user'})`` | #### content `instance-attribute` `content: [str](https://docs.python.org/3/library/stdtypes.html#str) | [Sequence](https://docs.python.org/3/library/collections.abc.html#collections.abc.Sequence "collections.abc.Sequence") [UserContent]` The content of the prompt. #### timestamp `class-attribute` `instance-attribute` `timestamp: [datetime](https://docs.python.org/3/library/datetime.html#datetime.datetime "datetime.datetime") = [field](https://docs.python.org/3/library/dataclasses.html#dataclasses.field "dataclasses.field") (default_factory=now_utc)` The timestamp of the prompt. #### part\_kind `class-attribute` `instance-attribute` `part_kind: [Literal](https://docs.python.org/3/library/typing.html#typing.Literal "typing.Literal") ['user-prompt'] = 'user-prompt'` Part type identifier, this is available on all parts as a discriminator. ### ToolReturnPart `dataclass` A tool return message, this encodes the result of running a tool. Source code in `pydantic_ai_slim/pydantic_ai/messages.py` | | | | --- | --- | | 261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299 | `@dataclass class ToolReturnPart: """A tool return message, this encodes the result of running a tool.""" tool_name: str """The name of the "tool" was called.""" content: Any """The return value.""" tool_call_id: str """The tool call identifier, this is used by some models including OpenAI.""" timestamp: datetime = field(default_factory=_now_utc) """The timestamp, when the tool returned.""" part_kind: Literal['tool-return'] = 'tool-return' """Part type identifier, this is available on all parts as a discriminator.""" def model_response_str(self) -> str: """Return a string representation of the content for the model.""" if isinstance(self.content, str): return self.content else: return tool_return_ta.dump_json(self.content).decode() def model_response_object(self) -> dict[str, Any]: """Return a dictionary representation of the content, wrapping non-dict types appropriately.""" # gemini supports JSON dict return values, but no other JSON types, hence we wrap anything else in a dict if isinstance(self.content, dict): return tool_return_ta.dump_python(self.content, mode='json') # pyright: ignore[reportUnknownMemberType] else: return {'return_value': tool_return_ta.dump_python(self.content, mode='json')} def otel_event(self) -> Event: return Event( 'gen_ai.tool.message', body={'content': self.content, 'role': 'tool', 'id': self.tool_call_id, 'name': self.tool_name}, )` | #### tool\_name `instance-attribute` `tool_name: [str](https://docs.python.org/3/library/stdtypes.html#str)` The name of the "tool" was called. #### content `instance-attribute` `content: [Any](https://docs.python.org/3/library/typing.html#typing.Any "typing.Any")` The return value. #### tool\_call\_id `instance-attribute` `tool_call_id: [str](https://docs.python.org/3/library/stdtypes.html#str)` The tool call identifier, this is used by some models including OpenAI. #### timestamp `class-attribute` `instance-attribute` `timestamp: [datetime](https://docs.python.org/3/library/datetime.html#datetime.datetime "datetime.datetime") = [field](https://docs.python.org/3/library/dataclasses.html#dataclasses.field "dataclasses.field") (default_factory=now_utc)` The timestamp, when the tool returned. #### part\_kind `class-attribute` `instance-attribute` `part_kind: [Literal](https://docs.python.org/3/library/typing.html#typing.Literal "typing.Literal") ['tool-return'] = 'tool-return'` Part type identifier, this is available on all parts as a discriminator. #### model\_response\_str `model_response_str() -> [str](https://docs.python.org/3/library/stdtypes.html#str)` Return a string representation of the content for the model. Source code in `pydantic_ai_slim/pydantic_ai/messages.py` | | | | --- | --- | | 280
281
282
283
284
285 | `def model_response_str(self) -> str: """Return a string representation of the content for the model.""" if isinstance(self.content, str): return self.content else: return tool_return_ta.dump_json(self.content).decode()` | #### model\_response\_object `model_response_object() -> [dict](https://docs.python.org/3/library/stdtypes.html#dict) [[str](https://docs.python.org/3/library/stdtypes.html#str) , [Any](https://docs.python.org/3/library/typing.html#typing.Any "typing.Any") ]` Return a dictionary representation of the content, wrapping non-dict types appropriately. Source code in `pydantic_ai_slim/pydantic_ai/messages.py` | | | | --- | --- | | 287
288
289
290
291
292
293 | `def model_response_object(self) -> dict[str, Any]: """Return a dictionary representation of the content, wrapping non-dict types appropriately.""" # gemini supports JSON dict return values, but no other JSON types, hence we wrap anything else in a dict if isinstance(self.content, dict): return tool_return_ta.dump_python(self.content, mode='json') # pyright: ignore[reportUnknownMemberType] else: return {'return_value': tool_return_ta.dump_python(self.content, mode='json')}` | ### RetryPromptPart `dataclass` A message back to a model asking it to try again. This can be sent for a number of reasons: * Pydantic validation of tool arguments failed, here content is derived from a Pydantic [`ValidationError`](https://docs.pydantic.dev/latest/api/pydantic_core/#pydantic_core.ValidationError) * a tool raised a [`ModelRetry`](../exceptions/#pydantic_ai.exceptions.ModelRetry) exception * no tool was found for the tool name * the model returned plain text when a structured response was expected * Pydantic validation of a structured response failed, here content is derived from a Pydantic [`ValidationError`](https://docs.pydantic.dev/latest/api/pydantic_core/#pydantic_core.ValidationError) * a result validator raised a [`ModelRetry`](../exceptions/#pydantic_ai.exceptions.ModelRetry) exception Source code in `pydantic_ai_slim/pydantic_ai/messages.py` | | | | --- | --- | | 305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364 | ``@dataclass class RetryPromptPart: """A message back to a model asking it to try again. This can be sent for a number of reasons: * Pydantic validation of tool arguments failed, here content is derived from a Pydantic [`ValidationError`][pydantic_core.ValidationError] * a tool raised a [`ModelRetry`][pydantic_ai.exceptions.ModelRetry] exception * no tool was found for the tool name * the model returned plain text when a structured response was expected * Pydantic validation of a structured response failed, here content is derived from a Pydantic [`ValidationError`][pydantic_core.ValidationError] * a result validator raised a [`ModelRetry`][pydantic_ai.exceptions.ModelRetry] exception """ content: list[pydantic_core.ErrorDetails] \| str """Details of why and how the model should retry. If the retry was triggered by a [`ValidationError`][pydantic_core.ValidationError], this will be a list of error details. """ tool_name: str \| None = None """The name of the tool that was called, if any.""" tool_call_id: str = field(default_factory=_generate_tool_call_id) """The tool call identifier, this is used by some models including OpenAI. In case the tool call id is not provided by the model, PydanticAI will generate a random one. """ timestamp: datetime = field(default_factory=_now_utc) """The timestamp, when the retry was triggered.""" part_kind: Literal['retry-prompt'] = 'retry-prompt' """Part type identifier, this is available on all parts as a discriminator.""" def model_response(self) -> str: """Return a string message describing why the retry is requested.""" if isinstance(self.content, str): description = self.content else: json_errors = error_details_ta.dump_json(self.content, exclude={'__all__': {'ctx'}}, indent=2) description = f'{len(self.content)} validation errors: {json_errors.decode()}' return f'{description}\n\nFix the errors and try again.' def otel_event(self) -> Event: if self.tool_name is None: return Event('gen_ai.user.message', body={'content': self.model_response(), 'role': 'user'}) else: return Event( 'gen_ai.tool.message', body={ 'content': self.model_response(), 'role': 'tool', 'id': self.tool_call_id, 'name': self.tool_name, }, )`` | #### content `instance-attribute` `content: [list](https://docs.python.org/3/library/stdtypes.html#list) [[ErrorDetails](https://docs.pydantic.dev/latest/api/pydantic_core/#pydantic_core.ErrorDetails "pydantic_core.ErrorDetails") ] | [str](https://docs.python.org/3/library/stdtypes.html#str)` Details of why and how the model should retry. If the retry was triggered by a [`ValidationError`](https://docs.pydantic.dev/latest/api/pydantic_core/#pydantic_core.ValidationError) , this will be a list of error details. #### tool\_name `class-attribute` `instance-attribute` `tool_name: [str](https://docs.python.org/3/library/stdtypes.html#str) | None = None` The name of the tool that was called, if any. #### tool\_call\_id `class-attribute` `instance-attribute` `tool_call_id: [str](https://docs.python.org/3/library/stdtypes.html#str) = [field](https://docs.python.org/3/library/dataclasses.html#dataclasses.field "dataclasses.field") ( default_factory=generate_tool_call_id )` The tool call identifier, this is used by some models including OpenAI. In case the tool call id is not provided by the model, PydanticAI will generate a random one. #### timestamp `class-attribute` `instance-attribute` `timestamp: [datetime](https://docs.python.org/3/library/datetime.html#datetime.datetime "datetime.datetime") = [field](https://docs.python.org/3/library/dataclasses.html#dataclasses.field "dataclasses.field") (default_factory=now_utc)` The timestamp, when the retry was triggered. #### part\_kind `class-attribute` `instance-attribute` `part_kind: [Literal](https://docs.python.org/3/library/typing.html#typing.Literal "typing.Literal") ['retry-prompt'] = 'retry-prompt'` Part type identifier, this is available on all parts as a discriminator. #### model\_response `model_response() -> [str](https://docs.python.org/3/library/stdtypes.html#str)` Return a string message describing why the retry is requested. Source code in `pydantic_ai_slim/pydantic_ai/messages.py` | | | | --- | --- | | 343
344
345
346
347
348
349
350 | `def model_response(self) -> str: """Return a string message describing why the retry is requested.""" if isinstance(self.content, str): description = self.content else: json_errors = error_details_ta.dump_json(self.content, exclude={'__all__': {'ctx'}}, indent=2) description = f'{len(self.content)} validation errors: {json_errors.decode()}' return f'{description}\n\nFix the errors and try again.'` | ### ModelRequestPart `module-attribute` `ModelRequestPart = [Annotated](https://docs.python.org/3/library/typing.html#typing.Annotated "typing.Annotated") [ [Union](https://docs.python.org/3/library/typing.html#typing.Union "typing.Union") [ [SystemPromptPart](#pydantic_ai.messages.SystemPromptPart "pydantic_ai.messages.SystemPromptPart") , [UserPromptPart](#pydantic_ai.messages.UserPromptPart "pydantic_ai.messages.UserPromptPart") , [ToolReturnPart](#pydantic_ai.messages.ToolReturnPart "pydantic_ai.messages.ToolReturnPart") , [RetryPromptPart](#pydantic_ai.messages.RetryPromptPart "pydantic_ai.messages.RetryPromptPart") , ], [Discriminator](https://docs.pydantic.dev/latest/api/types/#pydantic.types.Discriminator "pydantic.Discriminator") ("part_kind"), ]` A message part sent by PydanticAI to a model. ### ModelRequest `dataclass` A request generated by PydanticAI and sent to a model, e.g. a message from the PydanticAI app to the model. Source code in `pydantic_ai_slim/pydantic_ai/messages.py` | | | | --- | --- | | 373
374
375
376
377
378
379
380
381 | `@dataclass class ModelRequest: """A request generated by PydanticAI and sent to a model, e.g. a message from the PydanticAI app to the model.""" parts: list[ModelRequestPart] """The parts of the user message.""" kind: Literal['request'] = 'request' """Message type identifier, this is available on all parts as a discriminator."""` | #### parts `instance-attribute` `parts: [list](https://docs.python.org/3/library/stdtypes.html#list) [[ModelRequestPart](#pydantic_ai.messages.ModelRequestPart "pydantic_ai.messages.ModelRequestPart") ]` The parts of the user message. #### kind `class-attribute` `instance-attribute` `kind: [Literal](https://docs.python.org/3/library/typing.html#typing.Literal "typing.Literal") ['request'] = 'request'` Message type identifier, this is available on all parts as a discriminator. ### TextPart `dataclass` A plain text response from a model. Source code in `pydantic_ai_slim/pydantic_ai/messages.py` | | | | --- | --- | | 384
385
386
387
388
389
390
391
392
393
394
395
396 | ``@dataclass class TextPart: """A plain text response from a model.""" content: str """The text content of the response.""" part_kind: Literal['text'] = 'text' """Part type identifier, this is available on all parts as a discriminator.""" def has_content(self) -> bool: """Return `True` if the text content is non-empty.""" return bool(self.content)`` | #### content `instance-attribute` `content: [str](https://docs.python.org/3/library/stdtypes.html#str)` The text content of the response. #### part\_kind `class-attribute` `instance-attribute` `part_kind: [Literal](https://docs.python.org/3/library/typing.html#typing.Literal "typing.Literal") ['text'] = 'text'` Part type identifier, this is available on all parts as a discriminator. #### has\_content `has_content() -> [bool](https://docs.python.org/3/library/functions.html#bool)` Return `True` if the text content is non-empty. Source code in `pydantic_ai_slim/pydantic_ai/messages.py` | | | | --- | --- | | 394
395
396 | ``def has_content(self) -> bool: """Return `True` if the text content is non-empty.""" return bool(self.content)`` | ### ToolCallPart `dataclass` A tool call from a model. Source code in `pydantic_ai_slim/pydantic_ai/messages.py` | | | | --- | --- | | 399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448 | ``@dataclass class ToolCallPart: """A tool call from a model.""" tool_name: str """The name of the tool to call.""" args: str \| dict[str, Any] """The arguments to pass to the tool. This is stored either as a JSON string or a Python dictionary depending on how data was received. """ tool_call_id: str = field(default_factory=_generate_tool_call_id) """The tool call identifier, this is used by some models including OpenAI. In case the tool call id is not provided by the model, PydanticAI will generate a random one. """ part_kind: Literal['tool-call'] = 'tool-call' """Part type identifier, this is available on all parts as a discriminator.""" def args_as_dict(self) -> dict[str, Any]: """Return the arguments as a Python dictionary. This is just for convenience with models that require dicts as input. """ if isinstance(self.args, dict): return self.args args = pydantic_core.from_json(self.args) assert isinstance(args, dict), 'args should be a dict' return cast(dict[str, Any], args) def args_as_json_str(self) -> str: """Return the arguments as a JSON string. This is just for convenience with models that require JSON strings as input. """ if isinstance(self.args, str): return self.args return pydantic_core.to_json(self.args).decode() def has_content(self) -> bool: """Return `True` if the arguments contain any data.""" if isinstance(self.args, dict): # TODO: This should probably return True if you have the value False, or 0, etc. # It makes sense to me to ignore empty strings, but not sure about empty lists or dicts return any(self.args.values()) else: return bool(self.args)`` | #### tool\_name `instance-attribute` `tool_name: [str](https://docs.python.org/3/library/stdtypes.html#str)` The name of the tool to call. #### args `instance-attribute` `args: [str](https://docs.python.org/3/library/stdtypes.html#str) | [dict](https://docs.python.org/3/library/stdtypes.html#dict) [[str](https://docs.python.org/3/library/stdtypes.html#str) , [Any](https://docs.python.org/3/library/typing.html#typing.Any "typing.Any") ]` The arguments to pass to the tool. This is stored either as a JSON string or a Python dictionary depending on how data was received. #### tool\_call\_id `class-attribute` `instance-attribute` `tool_call_id: [str](https://docs.python.org/3/library/stdtypes.html#str) = [field](https://docs.python.org/3/library/dataclasses.html#dataclasses.field "dataclasses.field") ( default_factory=generate_tool_call_id )` The tool call identifier, this is used by some models including OpenAI. In case the tool call id is not provided by the model, PydanticAI will generate a random one. #### part\_kind `class-attribute` `instance-attribute` `part_kind: [Literal](https://docs.python.org/3/library/typing.html#typing.Literal "typing.Literal") ['tool-call'] = 'tool-call'` Part type identifier, this is available on all parts as a discriminator. #### args\_as\_dict `args_as_dict() -> [dict](https://docs.python.org/3/library/stdtypes.html#dict) [[str](https://docs.python.org/3/library/stdtypes.html#str) , [Any](https://docs.python.org/3/library/typing.html#typing.Any "typing.Any") ]` Return the arguments as a Python dictionary. This is just for convenience with models that require dicts as input. Source code in `pydantic_ai_slim/pydantic_ai/messages.py` | | | | --- | --- | | 421
422
423
424
425
426
427
428
429
430 | `def args_as_dict(self) -> dict[str, Any]: """Return the arguments as a Python dictionary. This is just for convenience with models that require dicts as input. """ if isinstance(self.args, dict): return self.args args = pydantic_core.from_json(self.args) assert isinstance(args, dict), 'args should be a dict' return cast(dict[str, Any], args)` | #### args\_as\_json\_str `args_as_json_str() -> [str](https://docs.python.org/3/library/stdtypes.html#str)` Return the arguments as a JSON string. This is just for convenience with models that require JSON strings as input. Source code in `pydantic_ai_slim/pydantic_ai/messages.py` | | | | --- | --- | | 432
433
434
435
436
437
438
439 | `def args_as_json_str(self) -> str: """Return the arguments as a JSON string. This is just for convenience with models that require JSON strings as input. """ if isinstance(self.args, str): return self.args return pydantic_core.to_json(self.args).decode()` | #### has\_content `has_content() -> [bool](https://docs.python.org/3/library/functions.html#bool)` Return `True` if the arguments contain any data. Source code in `pydantic_ai_slim/pydantic_ai/messages.py` | | | | --- | --- | | 441
442
443
444
445
446
447
448 | ``def has_content(self) -> bool: """Return `True` if the arguments contain any data.""" if isinstance(self.args, dict): # TODO: This should probably return True if you have the value False, or 0, etc. # It makes sense to me to ignore empty strings, but not sure about empty lists or dicts return any(self.args.values()) else: return bool(self.args)`` | ### ModelResponsePart `module-attribute` `ModelResponsePart = [Annotated](https://docs.python.org/3/library/typing.html#typing.Annotated "typing.Annotated") [ [Union](https://docs.python.org/3/library/typing.html#typing.Union "typing.Union") [[TextPart](#pydantic_ai.messages.TextPart "pydantic_ai.messages.TextPart") , [ToolCallPart](#pydantic_ai.messages.ToolCallPart "pydantic_ai.messages.ToolCallPart") ], [Discriminator](https://docs.pydantic.dev/latest/api/types/#pydantic.types.Discriminator "pydantic.Discriminator") ("part_kind"), ]` A message part returned by a model. ### ModelResponse `dataclass` A response from a model, e.g. a message from the model to the PydanticAI app. Source code in `pydantic_ai_slim/pydantic_ai/messages.py` | | | | --- | --- | | 455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502 | `@dataclass class ModelResponse: """A response from a model, e.g. a message from the model to the PydanticAI app.""" parts: list[ModelResponsePart] """The parts of the model message.""" model_name: str \| None = None """The name of the model that generated the response.""" timestamp: datetime = field(default_factory=_now_utc) """The timestamp of the response. If the model provides a timestamp in the response (as OpenAI does) that will be used. """ kind: Literal['response'] = 'response' """Message type identifier, this is available on all parts as a discriminator.""" def otel_events(self) -> list[Event]: """Return OpenTelemetry events for the response.""" result: list[Event] = [] def new_event_body(): new_body: dict[str, Any] = {'role': 'assistant'} ev = Event('gen_ai.assistant.message', body=new_body) result.append(ev) return new_body body = new_event_body() for part in self.parts: if isinstance(part, ToolCallPart): body.setdefault('tool_calls', []).append( { 'id': part.tool_call_id, 'type': 'function', # TODO https://github.com/pydantic/pydantic-ai/issues/888 'function': { 'name': part.tool_name, 'arguments': part.args, }, } ) elif isinstance(part, TextPart): if body.get('content'): body = new_event_body() body['content'] = part.content return result` | #### parts `instance-attribute` `parts: [list](https://docs.python.org/3/library/stdtypes.html#list) [[ModelResponsePart](#pydantic_ai.messages.ModelResponsePart "pydantic_ai.messages.ModelResponsePart") ]` The parts of the model message. #### model\_name `class-attribute` `instance-attribute` `model_name: [str](https://docs.python.org/3/library/stdtypes.html#str) | None = None` The name of the model that generated the response. #### timestamp `class-attribute` `instance-attribute` `timestamp: [datetime](https://docs.python.org/3/library/datetime.html#datetime.datetime "datetime.datetime") = [field](https://docs.python.org/3/library/dataclasses.html#dataclasses.field "dataclasses.field") (default_factory=now_utc)` The timestamp of the response. If the model provides a timestamp in the response (as OpenAI does) that will be used. #### kind `class-attribute` `instance-attribute` `kind: [Literal](https://docs.python.org/3/library/typing.html#typing.Literal "typing.Literal") ['response'] = 'response'` Message type identifier, this is available on all parts as a discriminator. #### otel\_events `otel_events() -> [list](https://docs.python.org/3/library/stdtypes.html#list) [Event]` Return OpenTelemetry events for the response. Source code in `pydantic_ai_slim/pydantic_ai/messages.py` | | | | --- | --- | | 474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502 | `def otel_events(self) -> list[Event]: """Return OpenTelemetry events for the response.""" result: list[Event] = [] def new_event_body(): new_body: dict[str, Any] = {'role': 'assistant'} ev = Event('gen_ai.assistant.message', body=new_body) result.append(ev) return new_body body = new_event_body() for part in self.parts: if isinstance(part, ToolCallPart): body.setdefault('tool_calls', []).append( { 'id': part.tool_call_id, 'type': 'function', # TODO https://github.com/pydantic/pydantic-ai/issues/888 'function': { 'name': part.tool_name, 'arguments': part.args, }, } ) elif isinstance(part, TextPart): if body.get('content'): body = new_event_body() body['content'] = part.content return result` | ### ModelMessage `module-attribute` `ModelMessage = [Annotated](https://docs.python.org/3/library/typing.html#typing.Annotated "typing.Annotated") [ [Union](https://docs.python.org/3/library/typing.html#typing.Union "typing.Union") [[ModelRequest](#pydantic_ai.messages.ModelRequest "pydantic_ai.messages.ModelRequest") , [ModelResponse](#pydantic_ai.messages.ModelResponse "pydantic_ai.messages.ModelResponse") ], [Discriminator](https://docs.pydantic.dev/latest/api/types/#pydantic.types.Discriminator "pydantic.Discriminator") ("kind"), ]` Any message sent to or returned by a model. ### ModelMessagesTypeAdapter `module-attribute` `ModelMessagesTypeAdapter = [TypeAdapter](https://docs.pydantic.dev/latest/api/type_adapter/#pydantic.type_adapter.TypeAdapter "pydantic.TypeAdapter") ( [list](https://docs.python.org/3/library/stdtypes.html#list) [[ModelMessage](#pydantic_ai.messages.ModelMessage "pydantic_ai.messages.ModelMessage") ], config=[ConfigDict](https://docs.pydantic.dev/latest/api/config/#pydantic.config.ConfigDict "pydantic.ConfigDict") ( defer_build=True, ser_json_bytes="base64" ), )` Pydantic [`TypeAdapter`](https://docs.pydantic.dev/latest/api/type_adapter/#pydantic.type_adapter.TypeAdapter) for (de)serializing messages. ### TextPartDelta `dataclass` A partial update (delta) for a `TextPart` to append new text content. Source code in `pydantic_ai_slim/pydantic_ai/messages.py` | | | | --- | --- | | 514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538 | ``@dataclass class TextPartDelta: """A partial update (delta) for a `TextPart` to append new text content.""" content_delta: str """The incremental text content to add to the existing `TextPart` content.""" part_delta_kind: Literal['text'] = 'text' """Part delta type identifier, used as a discriminator.""" def apply(self, part: ModelResponsePart) -> TextPart: """Apply this text delta to an existing `TextPart`. Args: part: The existing model response part, which must be a `TextPart`. Returns: A new `TextPart` with updated text content. Raises: ValueError: If `part` is not a `TextPart`. """ if not isinstance(part, TextPart): raise ValueError('Cannot apply TextPartDeltas to non-TextParts') return replace(part, content=part.content + self.content_delta)`` | #### content\_delta `instance-attribute` `content_delta: [str](https://docs.python.org/3/library/stdtypes.html#str)` The incremental text content to add to the existing `TextPart` content. #### part\_delta\_kind `class-attribute` `instance-attribute` `part_delta_kind: [Literal](https://docs.python.org/3/library/typing.html#typing.Literal "typing.Literal") ['text'] = 'text'` Part delta type identifier, used as a discriminator. #### apply `apply(part: [ModelResponsePart](#pydantic_ai.messages.ModelResponsePart "pydantic_ai.messages.ModelResponsePart") ) -> [TextPart](#pydantic_ai.messages.TextPart "pydantic_ai.messages.TextPart")` Apply this text delta to an existing `TextPart`. Parameters: | Name | Type | Description | Default | | --- | --- | --- | --- | | `part` | `[ModelResponsePart](#pydantic_ai.messages.ModelResponsePart "pydantic_ai.messages.ModelResponsePart") ` | The existing model response part, which must be a `TextPart`. | _required_ | Returns: | Type | Description | | --- | --- | | `[TextPart](#pydantic_ai.messages.TextPart "pydantic_ai.messages.TextPart") ` | A new `TextPart` with updated text content. | Raises: | Type | Description | | --- | --- | | `[ValueError](https://docs.python.org/3/library/exceptions.html#ValueError) ` | If `part` is not a `TextPart`. | Source code in `pydantic_ai_slim/pydantic_ai/messages.py` | | | | --- | --- | | 524
525
526
527
528
529
530
531
532
533
534
535
536
537
538 | ``def apply(self, part: ModelResponsePart) -> TextPart: """Apply this text delta to an existing `TextPart`. Args: part: The existing model response part, which must be a `TextPart`. Returns: A new `TextPart` with updated text content. Raises: ValueError: If `part` is not a `TextPart`. """ if not isinstance(part, TextPart): raise ValueError('Cannot apply TextPartDeltas to non-TextParts') return replace(part, content=part.content + self.content_delta)`` | ### ToolCallPartDelta `dataclass` A partial update (delta) for a `ToolCallPart` to modify tool name, arguments, or tool call ID. Source code in `pydantic_ai_slim/pydantic_ai/messages.py` | | | | --- | --- | | 541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653 | ``@dataclass class ToolCallPartDelta: """A partial update (delta) for a `ToolCallPart` to modify tool name, arguments, or tool call ID.""" tool_name_delta: str \| None = None """Incremental text to add to the existing tool name, if any.""" args_delta: str \| dict[str, Any] \| None = None """Incremental data to add to the tool arguments. If this is a string, it will be appended to existing JSON arguments. If this is a dict, it will be merged with existing dict arguments. """ tool_call_id: str \| None = None """Optional tool call identifier, this is used by some models including OpenAI. Note this is never treated as a delta — it can replace None, but otherwise if a non-matching value is provided an error will be raised.""" part_delta_kind: Literal['tool_call'] = 'tool_call' """Part delta type identifier, used as a discriminator.""" def as_part(self) -> ToolCallPart \| None: """Convert this delta to a fully formed `ToolCallPart` if possible, otherwise return `None`. Returns: A `ToolCallPart` if both `tool_name_delta` and `args_delta` are set, otherwise `None`. """ if self.tool_name_delta is None or self.args_delta is None: return None return ToolCallPart(self.tool_name_delta, self.args_delta, self.tool_call_id or _generate_tool_call_id()) @overload def apply(self, part: ModelResponsePart) -> ToolCallPart: ... @overload def apply(self, part: ModelResponsePart \| ToolCallPartDelta) -> ToolCallPart \| ToolCallPartDelta: ... def apply(self, part: ModelResponsePart \| ToolCallPartDelta) -> ToolCallPart \| ToolCallPartDelta: """Apply this delta to a part or delta, returning a new part or delta with the changes applied. Args: part: The existing model response part or delta to update. Returns: Either a new `ToolCallPart` or an updated `ToolCallPartDelta`. Raises: ValueError: If `part` is neither a `ToolCallPart` nor a `ToolCallPartDelta`. UnexpectedModelBehavior: If applying JSON deltas to dict arguments or vice versa. """ if isinstance(part, ToolCallPart): return self._apply_to_part(part) if isinstance(part, ToolCallPartDelta): return self._apply_to_delta(part) raise ValueError(f'Can only apply ToolCallPartDeltas to ToolCallParts or ToolCallPartDeltas, not {part}') def _apply_to_delta(self, delta: ToolCallPartDelta) -> ToolCallPart \| ToolCallPartDelta: """Internal helper to apply this delta to another delta.""" if self.tool_name_delta: # Append incremental text to the existing tool_name_delta updated_tool_name_delta = (delta.tool_name_delta or '') + self.tool_name_delta delta = replace(delta, tool_name_delta=updated_tool_name_delta) if isinstance(self.args_delta, str): if isinstance(delta.args_delta, dict): raise UnexpectedModelBehavior( f'Cannot apply JSON deltas to non-JSON tool arguments ({delta=}, {self=})' ) updated_args_delta = (delta.args_delta or '') + self.args_delta delta = replace(delta, args_delta=updated_args_delta) elif isinstance(self.args_delta, dict): if isinstance(delta.args_delta, str): raise UnexpectedModelBehavior( f'Cannot apply dict deltas to non-dict tool arguments ({delta=}, {self=})' ) updated_args_delta = {**(delta.args_delta or {}), **self.args_delta} delta = replace(delta, args_delta=updated_args_delta) if self.tool_call_id: delta = replace(delta, tool_call_id=self.tool_call_id) # If we now have enough data to create a full ToolCallPart, do so if delta.tool_name_delta is not None and delta.args_delta is not None: return ToolCallPart(delta.tool_name_delta, delta.args_delta, delta.tool_call_id or _generate_tool_call_id()) return delta def _apply_to_part(self, part: ToolCallPart) -> ToolCallPart: """Internal helper to apply this delta directly to a `ToolCallPart`.""" if self.tool_name_delta: # Append incremental text to the existing tool_name tool_name = part.tool_name + self.tool_name_delta part = replace(part, tool_name=tool_name) if isinstance(self.args_delta, str): if not isinstance(part.args, str): raise UnexpectedModelBehavior(f'Cannot apply JSON deltas to non-JSON tool arguments ({part=}, {self=})') updated_json = part.args + self.args_delta part = replace(part, args=updated_json) elif isinstance(self.args_delta, dict): if not isinstance(part.args, dict): raise UnexpectedModelBehavior(f'Cannot apply dict deltas to non-dict tool arguments ({part=}, {self=})') updated_dict = {**(part.args or {}), **self.args_delta} part = replace(part, args=updated_dict) if self.tool_call_id: part = replace(part, tool_call_id=self.tool_call_id) return part`` | #### tool\_name\_delta `class-attribute` `instance-attribute` `tool_name_delta: [str](https://docs.python.org/3/library/stdtypes.html#str) | None = None` Incremental text to add to the existing tool name, if any. #### args\_delta `class-attribute` `instance-attribute` `args_delta: [str](https://docs.python.org/3/library/stdtypes.html#str) | [dict](https://docs.python.org/3/library/stdtypes.html#dict) [[str](https://docs.python.org/3/library/stdtypes.html#str) , [Any](https://docs.python.org/3/library/typing.html#typing.Any "typing.Any") ] | None = None` Incremental data to add to the tool arguments. If this is a string, it will be appended to existing JSON arguments. If this is a dict, it will be merged with existing dict arguments. #### tool\_call\_id `class-attribute` `instance-attribute` `tool_call_id: [str](https://docs.python.org/3/library/stdtypes.html#str) | None = None` Optional tool call identifier, this is used by some models including OpenAI. Note this is never treated as a delta — it can replace None, but otherwise if a non-matching value is provided an error will be raised. #### part\_delta\_kind `class-attribute` `instance-attribute` `part_delta_kind: [Literal](https://docs.python.org/3/library/typing.html#typing.Literal "typing.Literal") ['tool_call'] = 'tool_call'` Part delta type identifier, used as a discriminator. #### as\_part `as_part() -> [ToolCallPart](#pydantic_ai.messages.ToolCallPart "pydantic_ai.messages.ToolCallPart") | None` Convert this delta to a fully formed `ToolCallPart` if possible, otherwise return `None`. Returns: | Type | Description | | --- | --- | | `[ToolCallPart](#pydantic_ai.messages.ToolCallPart "pydantic_ai.messages.ToolCallPart") \| None` | A `ToolCallPart` if both `tool_name_delta` and `args_delta` are set, otherwise `None`. | Source code in `pydantic_ai_slim/pydantic_ai/messages.py` | | | | --- | --- | | 564
565
566
567
568
569
570
571
572
573 | ``def as_part(self) -> ToolCallPart \| None: """Convert this delta to a fully formed `ToolCallPart` if possible, otherwise return `None`. Returns: A `ToolCallPart` if both `tool_name_delta` and `args_delta` are set, otherwise `None`. """ if self.tool_name_delta is None or self.args_delta is None: return None return ToolCallPart(self.tool_name_delta, self.args_delta, self.tool_call_id or _generate_tool_call_id())`` | #### apply `apply(part: [ModelResponsePart](#pydantic_ai.messages.ModelResponsePart "pydantic_ai.messages.ModelResponsePart") ) -> [ToolCallPart](#pydantic_ai.messages.ToolCallPart "pydantic_ai.messages.ToolCallPart")` `apply( part: [ModelResponsePart](#pydantic_ai.messages.ModelResponsePart "pydantic_ai.messages.ModelResponsePart") | [ToolCallPartDelta](#pydantic_ai.messages.ToolCallPartDelta "pydantic_ai.messages.ToolCallPartDelta") , ) -> [ToolCallPart](#pydantic_ai.messages.ToolCallPart "pydantic_ai.messages.ToolCallPart") | [ToolCallPartDelta](#pydantic_ai.messages.ToolCallPartDelta "pydantic_ai.messages.ToolCallPartDelta")` `apply( part: [ModelResponsePart](#pydantic_ai.messages.ModelResponsePart "pydantic_ai.messages.ModelResponsePart") | [ToolCallPartDelta](#pydantic_ai.messages.ToolCallPartDelta "pydantic_ai.messages.ToolCallPartDelta") , ) -> [ToolCallPart](#pydantic_ai.messages.ToolCallPart "pydantic_ai.messages.ToolCallPart") | [ToolCallPartDelta](#pydantic_ai.messages.ToolCallPartDelta "pydantic_ai.messages.ToolCallPartDelta")` Apply this delta to a part or delta, returning a new part or delta with the changes applied. Parameters: | Name | Type | Description | Default | | --- | --- | --- | --- | | `part` | `[ModelResponsePart](#pydantic_ai.messages.ModelResponsePart "pydantic_ai.messages.ModelResponsePart") \| [ToolCallPartDelta](#pydantic_ai.messages.ToolCallPartDelta "pydantic_ai.messages.ToolCallPartDelta") ` | The existing model response part or delta to update. | _required_ | Returns: | Type | Description | | --- | --- | | `[ToolCallPart](#pydantic_ai.messages.ToolCallPart "pydantic_ai.messages.ToolCallPart") \| [ToolCallPartDelta](#pydantic_ai.messages.ToolCallPartDelta "pydantic_ai.messages.ToolCallPartDelta") ` | Either a new `ToolCallPart` or an updated `ToolCallPartDelta`. | Raises: | Type | Description | | --- | --- | | `[ValueError](https://docs.python.org/3/library/exceptions.html#ValueError) ` | If `part` is neither a `ToolCallPart` nor a `ToolCallPartDelta`. | | `[UnexpectedModelBehavior](../exceptions/#pydantic_ai.exceptions.UnexpectedModelBehavior "pydantic_ai.exceptions.UnexpectedModelBehavior") ` | If applying JSON deltas to dict arguments or vice versa. | Source code in `pydantic_ai_slim/pydantic_ai/messages.py` | | | | --- | --- | | 581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600 | ``def apply(self, part: ModelResponsePart \| ToolCallPartDelta) -> ToolCallPart \| ToolCallPartDelta: """Apply this delta to a part or delta, returning a new part or delta with the changes applied. Args: part: The existing model response part or delta to update. Returns: Either a new `ToolCallPart` or an updated `ToolCallPartDelta`. Raises: ValueError: If `part` is neither a `ToolCallPart` nor a `ToolCallPartDelta`. UnexpectedModelBehavior: If applying JSON deltas to dict arguments or vice versa. """ if isinstance(part, ToolCallPart): return self._apply_to_part(part) if isinstance(part, ToolCallPartDelta): return self._apply_to_delta(part) raise ValueError(f'Can only apply ToolCallPartDeltas to ToolCallParts or ToolCallPartDeltas, not {part}')`` | ### ModelResponsePartDelta `module-attribute` `ModelResponsePartDelta = [Annotated](https://docs.python.org/3/library/typing.html#typing.Annotated "typing.Annotated") [ [Union](https://docs.python.org/3/library/typing.html#typing.Union "typing.Union") [[TextPartDelta](#pydantic_ai.messages.TextPartDelta "pydantic_ai.messages.TextPartDelta") , [ToolCallPartDelta](#pydantic_ai.messages.ToolCallPartDelta "pydantic_ai.messages.ToolCallPartDelta") ], [Discriminator](https://docs.pydantic.dev/latest/api/types/#pydantic.types.Discriminator "pydantic.Discriminator") ("part_delta_kind"), ]` A partial update (delta) for any model response part. ### PartStartEvent `dataclass` An event indicating that a new part has started. If multiple `PartStartEvent`s are received with the same index, the new one should fully replace the old one. Source code in `pydantic_ai_slim/pydantic_ai/messages.py` | | | | --- | --- | | 660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675 | ``@dataclass class PartStartEvent: """An event indicating that a new part has started. If multiple `PartStartEvent`s are received with the same index, the new one should fully replace the old one. """ index: int """The index of the part within the overall response parts list.""" part: ModelResponsePart """The newly started `ModelResponsePart`.""" event_kind: Literal['part_start'] = 'part_start' """Event type identifier, used as a discriminator."""`` | #### index `instance-attribute` `index: [int](https://docs.python.org/3/library/functions.html#int)` The index of the part within the overall response parts list. #### part `instance-attribute` `part: [ModelResponsePart](#pydantic_ai.messages.ModelResponsePart "pydantic_ai.messages.ModelResponsePart")` The newly started `ModelResponsePart`. #### event\_kind `class-attribute` `instance-attribute` `event_kind: [Literal](https://docs.python.org/3/library/typing.html#typing.Literal "typing.Literal") ['part_start'] = 'part_start'` Event type identifier, used as a discriminator. ### PartDeltaEvent `dataclass` An event indicating a delta update for an existing part. Source code in `pydantic_ai_slim/pydantic_ai/messages.py` | | | | --- | --- | | 678
679
680
681
682
683
684
685
686
687
688
689 | `@dataclass class PartDeltaEvent: """An event indicating a delta update for an existing part.""" index: int """The index of the part within the overall response parts list.""" delta: ModelResponsePartDelta """The delta to apply to the specified part.""" event_kind: Literal['part_delta'] = 'part_delta' """Event type identifier, used as a discriminator."""` | #### index `instance-attribute` `index: [int](https://docs.python.org/3/library/functions.html#int)` The index of the part within the overall response parts list. #### delta `instance-attribute` `delta: [ModelResponsePartDelta](#pydantic_ai.messages.ModelResponsePartDelta "pydantic_ai.messages.ModelResponsePartDelta")` The delta to apply to the specified part. #### event\_kind `class-attribute` `instance-attribute` `event_kind: [Literal](https://docs.python.org/3/library/typing.html#typing.Literal "typing.Literal") ['part_delta'] = 'part_delta'` Event type identifier, used as a discriminator. ### FinalResultEvent `dataclass` An event indicating the response to the current model request matches the result schema. Source code in `pydantic_ai_slim/pydantic_ai/messages.py` | | | | --- | --- | | 692
693
694
695
696
697
698
699
700
701 | ``@dataclass class FinalResultEvent: """An event indicating the response to the current model request matches the result schema.""" tool_name: str \| None """The name of the result tool that was called. `None` if the result is from text content and not from a tool.""" tool_call_id: str \| None """The tool call ID, if any, that this result is associated with.""" event_kind: Literal['final_result'] = 'final_result' """Event type identifier, used as a discriminator."""`` | #### tool\_name `instance-attribute` `tool_name: [str](https://docs.python.org/3/library/stdtypes.html#str) | None` The name of the result tool that was called. `None` if the result is from text content and not from a tool. #### tool\_call\_id `instance-attribute` `tool_call_id: [str](https://docs.python.org/3/library/stdtypes.html#str) | None` The tool call ID, if any, that this result is associated with. #### event\_kind `class-attribute` `instance-attribute` `event_kind: [Literal](https://docs.python.org/3/library/typing.html#typing.Literal "typing.Literal") ['final_result'] = 'final_result'` Event type identifier, used as a discriminator. ### ModelResponseStreamEvent `module-attribute` `ModelResponseStreamEvent = [Annotated](https://docs.python.org/3/library/typing.html#typing.Annotated "typing.Annotated") [ [Union](https://docs.python.org/3/library/typing.html#typing.Union "typing.Union") [[PartStartEvent](#pydantic_ai.messages.PartStartEvent "pydantic_ai.messages.PartStartEvent") , [PartDeltaEvent](#pydantic_ai.messages.PartDeltaEvent "pydantic_ai.messages.PartDeltaEvent") ], [Discriminator](https://docs.pydantic.dev/latest/api/types/#pydantic.types.Discriminator "pydantic.Discriminator") ("event_kind"), ]` An event in the model response stream, either starting a new part or applying a delta to an existing one. ### AgentStreamEvent `module-attribute` `AgentStreamEvent = [Annotated](https://docs.python.org/3/library/typing.html#typing.Annotated "typing.Annotated") [ [Union](https://docs.python.org/3/library/typing.html#typing.Union "typing.Union") [[PartStartEvent](#pydantic_ai.messages.PartStartEvent "pydantic_ai.messages.PartStartEvent") , [PartDeltaEvent](#pydantic_ai.messages.PartDeltaEvent "pydantic_ai.messages.PartDeltaEvent") , [FinalResultEvent](#pydantic_ai.messages.FinalResultEvent "pydantic_ai.messages.FinalResultEvent") ], [Discriminator](https://docs.pydantic.dev/latest/api/types/#pydantic.types.Discriminator "pydantic.Discriminator") ("event_kind"), ]` An event in the agent stream. ### FunctionToolCallEvent `dataclass` An event indicating the start to a call to a function tool. Source code in `pydantic_ai_slim/pydantic_ai/messages.py` | | | | --- | --- | | 713
714
715
716
717
718
719
720
721
722
723
724
725 | `@dataclass class FunctionToolCallEvent: """An event indicating the start to a call to a function tool.""" part: ToolCallPart """The (function) tool call to make.""" call_id: str = field(init=False) """An ID used for matching details about the call to its result. If present, defaults to the part's tool_call_id.""" event_kind: Literal['function_tool_call'] = 'function_tool_call' """Event type identifier, used as a discriminator.""" def __post_init__(self): self.call_id = self.part.tool_call_id or str(uuid.uuid4())` | #### part `instance-attribute` `part: [ToolCallPart](#pydantic_ai.messages.ToolCallPart "pydantic_ai.messages.ToolCallPart")` The (function) tool call to make. #### call\_id `class-attribute` `instance-attribute` `call_id: [str](https://docs.python.org/3/library/stdtypes.html#str) = [field](https://docs.python.org/3/library/dataclasses.html#dataclasses.field "dataclasses.field") (init=False)` An ID used for matching details about the call to its result. If present, defaults to the part's tool\_call\_id. #### event\_kind `class-attribute` `instance-attribute` `event_kind: [Literal](https://docs.python.org/3/library/typing.html#typing.Literal "typing.Literal") ["function_tool_call"] = ( "function_tool_call" )` Event type identifier, used as a discriminator. ### FunctionToolResultEvent `dataclass` An event indicating the result of a function tool call. Source code in `pydantic_ai_slim/pydantic_ai/messages.py` | | | | --- | --- | | 728
729
730
731
732
733
734
735
736
737 | `@dataclass class FunctionToolResultEvent: """An event indicating the result of a function tool call.""" result: ToolReturnPart \| RetryPromptPart """The result of the call to the function tool.""" tool_call_id: str """An ID used to match the result to its original call.""" event_kind: Literal['function_tool_result'] = 'function_tool_result' """Event type identifier, used as a discriminator."""` | #### result `instance-attribute` `result: [ToolReturnPart](#pydantic_ai.messages.ToolReturnPart "pydantic_ai.messages.ToolReturnPart") | [RetryPromptPart](#pydantic_ai.messages.RetryPromptPart "pydantic_ai.messages.RetryPromptPart")` The result of the call to the function tool. #### tool\_call\_id `instance-attribute` `tool_call_id: [str](https://docs.python.org/3/library/stdtypes.html#str)` An ID used to match the result to its original call. #### event\_kind `class-attribute` `instance-attribute` `event_kind: [Literal](https://docs.python.org/3/library/typing.html#typing.Literal "typing.Literal") ["function_tool_result"] = ( "function_tool_result" )` Event type identifier, used as a discriminator. --- # pydantic_ai.models.groq - PydanticAI `pydantic_ai.models.groq` ========================= Setup ----- For details on how to set up authentication with this model, see [model configuration for Groq](../../../models/#groq) . ### ProductionGroqModelNames `module-attribute` `ProductionGroqModelNames = [Literal](https://docs.python.org/3/library/typing.html#typing.Literal "typing.Literal") [ "distil-whisper-large-v3-en", "gemma2-9b-it", "llama-3.3-70b-versatile", "llama-3.1-8b-instant", "llama-guard-3-8b", "llama3-70b-8192", "llama3-8b-8192", "whisper-large-v3", "whisper-large-v3-turbo", ]` Production Groq models from [https://console.groq.com/docs/models#production-models](https://console.groq.com/docs/models#production-models) . ### PreviewGroqModelNames `module-attribute` `PreviewGroqModelNames = [Literal](https://docs.python.org/3/library/typing.html#typing.Literal "typing.Literal") [ "playai-tts", "playai-tts-arabic", "qwen-qwq-32b", "mistral-saba-24b", "qwen-2.5-coder-32b", "qwen-2.5-32b", "deepseek-r1-distill-qwen-32b", "deepseek-r1-distill-llama-70b", "llama-3.3-70b-specdec", "llama-3.2-1b-preview", "llama-3.2-3b-preview", "llama-3.2-11b-vision-preview", "llama-3.2-90b-vision-preview", ]` Preview Groq models from [https://console.groq.com/docs/models#preview-models](https://console.groq.com/docs/models#preview-models) . ### GroqModelName `module-attribute` `GroqModelName = [Union](https://docs.python.org/3/library/typing.html#typing.Union "typing.Union") [ [str](https://docs.python.org/3/library/stdtypes.html#str) , [ProductionGroqModelNames](#pydantic_ai.models.groq.ProductionGroqModelNames "pydantic_ai.models.groq.ProductionGroqModelNames") , [PreviewGroqModelNames](#pydantic_ai.models.groq.PreviewGroqModelNames "pydantic_ai.models.groq.PreviewGroqModelNames") ]` Possible Groq model names. Since Groq supports a variety of models and the list changes frequencly, we explicitly list the named models as of 2025-03-31 but allow any name in the type hints. See [https://console.groq.com/docs/models](https://console.groq.com/docs/models) for an up to date date list of models and more details. ### GroqModelSettings Bases: `[ModelSettings](../../settings/#pydantic_ai.settings.ModelSettings "pydantic_ai.settings.ModelSettings") ` Settings used for a Groq model request. ALL FIELDS MUST BE `groq_` PREFIXED SO YOU CAN MERGE THEM WITH OTHER MODELS. Source code in `pydantic_ai_slim/pydantic_ai/models/groq.py` | | | | --- | --- | | 86
87
88
89
90 | ``class GroqModelSettings(ModelSettings): """Settings used for a Groq model request. ALL FIELDS MUST BE `groq_` PREFIXED SO YOU CAN MERGE THEM WITH OTHER MODELS. """`` | ### GroqModel `dataclass` Bases: `[Model](../base/#pydantic_ai.models.Model "pydantic_ai.models.Model") ` A model that uses the Groq API. Internally, this uses the [Groq Python client](https://github.com/groq/groq-python) to interact with the API. Apart from `__init__`, all methods are private or match those of the base class. Source code in `pydantic_ai_slim/pydantic_ai/models/groq.py` | | | | --- | --- | | 95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349 | ``@dataclass(init=False) class GroqModel(Model): """A model that uses the Groq API. Internally, this uses the [Groq Python client](https://github.com/groq/groq-python) to interact with the API. Apart from `__init__`, all methods are private or match those of the base class. """ client: AsyncGroq = field(repr=False) _model_name: GroqModelName = field(repr=False) _system: str = field(default='groq', repr=False) def __init__(self, model_name: GroqModelName, *, provider: Literal['groq'] \| Provider[AsyncGroq] = 'groq'): """Initialize a Groq model. Args: model_name: The name of the Groq model to use. List of model names available [here](https://console.groq.com/docs/models). provider: The provider to use for authentication and API access. Can be either the string 'groq' or an instance of `Provider[AsyncGroq]`. If not provided, a new provider will be created using the other parameters. """ self._model_name = model_name if isinstance(provider, str): provider = infer_provider(provider) self.client = provider.client @property def base_url(self) -> str: return str(self.client.base_url) async def request( self, messages: list[ModelMessage], model_settings: ModelSettings \| None, model_request_parameters: ModelRequestParameters, ) -> tuple[ModelResponse, usage.Usage]: check_allow_model_requests() response = await self._completions_create( messages, False, cast(GroqModelSettings, model_settings or {}), model_request_parameters ) return self._process_response(response), _map_usage(response) @asynccontextmanager async def request_stream( self, messages: list[ModelMessage], model_settings: ModelSettings \| None, model_request_parameters: ModelRequestParameters, ) -> AsyncIterator[StreamedResponse]: check_allow_model_requests() response = await self._completions_create( messages, True, cast(GroqModelSettings, model_settings or {}), model_request_parameters ) async with response: yield await self._process_streamed_response(response) @property def model_name(self) -> GroqModelName: """The model name.""" return self._model_name @property def system(self) -> str: """The system / model provider.""" return self._system @overload async def _completions_create( self, messages: list[ModelMessage], stream: Literal[True], model_settings: GroqModelSettings, model_request_parameters: ModelRequestParameters, ) -> AsyncStream[chat.ChatCompletionChunk]: pass @overload async def _completions_create( self, messages: list[ModelMessage], stream: Literal[False], model_settings: GroqModelSettings, model_request_parameters: ModelRequestParameters, ) -> chat.ChatCompletion: pass async def _completions_create( self, messages: list[ModelMessage], stream: bool, model_settings: GroqModelSettings, model_request_parameters: ModelRequestParameters, ) -> chat.ChatCompletion \| AsyncStream[chat.ChatCompletionChunk]: tools = self._get_tools(model_request_parameters) # standalone function to make it easier to override if not tools: tool_choice: Literal['none', 'required', 'auto'] \| None = None elif not model_request_parameters.allow_text_result: tool_choice = 'required' else: tool_choice = 'auto' groq_messages = list(chain(*(self._map_message(m) for m in messages))) try: return await self.client.chat.completions.create( model=str(self._model_name), messages=groq_messages, n=1, parallel_tool_calls=model_settings.get('parallel_tool_calls', NOT_GIVEN), tools=tools or NOT_GIVEN, tool_choice=tool_choice or NOT_GIVEN, stream=stream, max_tokens=model_settings.get('max_tokens', NOT_GIVEN), temperature=model_settings.get('temperature', NOT_GIVEN), top_p=model_settings.get('top_p', NOT_GIVEN), timeout=model_settings.get('timeout', NOT_GIVEN), seed=model_settings.get('seed', NOT_GIVEN), presence_penalty=model_settings.get('presence_penalty', NOT_GIVEN), frequency_penalty=model_settings.get('frequency_penalty', NOT_GIVEN), logit_bias=model_settings.get('logit_bias', NOT_GIVEN), ) except APIStatusError as e: if (status_code := e.status_code) >= 400: raise ModelHTTPError(status_code=status_code, model_name=self.model_name, body=e.body) from e raise def _process_response(self, response: chat.ChatCompletion) -> ModelResponse: """Process a non-streamed response, and prepare a message to return.""" timestamp = datetime.fromtimestamp(response.created, tz=timezone.utc) choice = response.choices[0] items: list[ModelResponsePart] = [] if choice.message.content is not None: items.append(TextPart(content=choice.message.content)) if choice.message.tool_calls is not None: for c in choice.message.tool_calls: items.append(ToolCallPart(tool_name=c.function.name, args=c.function.arguments, tool_call_id=c.id)) return ModelResponse(items, model_name=response.model, timestamp=timestamp) async def _process_streamed_response(self, response: AsyncStream[chat.ChatCompletionChunk]) -> GroqStreamedResponse: """Process a streamed response, and prepare a streaming response to return.""" peekable_response = _utils.PeekableAsyncStream(response) first_chunk = await peekable_response.peek() if isinstance(first_chunk, _utils.Unset): raise UnexpectedModelBehavior('Streamed response ended without content or tool calls') return GroqStreamedResponse( _response=peekable_response, _model_name=self._model_name, _timestamp=datetime.fromtimestamp(first_chunk.created, tz=timezone.utc), ) def _get_tools(self, model_request_parameters: ModelRequestParameters) -> list[chat.ChatCompletionToolParam]: tools = [self._map_tool_definition(r) for r in model_request_parameters.function_tools] if model_request_parameters.result_tools: tools += [self._map_tool_definition(r) for r in model_request_parameters.result_tools] return tools def _map_message(self, message: ModelMessage) -> Iterable[chat.ChatCompletionMessageParam]: """Just maps a `pydantic_ai.Message` to a `groq.types.ChatCompletionMessageParam`.""" if isinstance(message, ModelRequest): yield from self._map_user_message(message) elif isinstance(message, ModelResponse): texts: list[str] = [] tool_calls: list[chat.ChatCompletionMessageToolCallParam] = [] for item in message.parts: if isinstance(item, TextPart): texts.append(item.content) elif isinstance(item, ToolCallPart): tool_calls.append(self._map_tool_call(item)) else: assert_never(item) message_param = chat.ChatCompletionAssistantMessageParam(role='assistant') if texts: # Note: model responses from this model should only have one text item, so the following # shouldn't merge multiple texts into one unless you switch models between runs: message_param['content'] = '\n\n'.join(texts) if tool_calls: message_param['tool_calls'] = tool_calls yield message_param else: assert_never(message) @staticmethod def _map_tool_call(t: ToolCallPart) -> chat.ChatCompletionMessageToolCallParam: return chat.ChatCompletionMessageToolCallParam( id=_guard_tool_call_id(t=t), type='function', function={'name': t.tool_name, 'arguments': t.args_as_json_str()}, ) @staticmethod def _map_tool_definition(f: ToolDefinition) -> chat.ChatCompletionToolParam: return { 'type': 'function', 'function': { 'name': f.name, 'description': f.description, 'parameters': f.parameters_json_schema, }, } @classmethod def _map_user_message(cls, message: ModelRequest) -> Iterable[chat.ChatCompletionMessageParam]: for part in message.parts: if isinstance(part, SystemPromptPart): yield chat.ChatCompletionSystemMessageParam(role='system', content=part.content) elif isinstance(part, UserPromptPart): yield cls._map_user_prompt(part) elif isinstance(part, ToolReturnPart): yield chat.ChatCompletionToolMessageParam( role='tool', tool_call_id=_guard_tool_call_id(t=part), content=part.model_response_str(), ) elif isinstance(part, RetryPromptPart): if part.tool_name is None: yield chat.ChatCompletionUserMessageParam(role='user', content=part.model_response()) else: yield chat.ChatCompletionToolMessageParam( role='tool', tool_call_id=_guard_tool_call_id(t=part), content=part.model_response(), ) @staticmethod def _map_user_prompt(part: UserPromptPart) -> chat.ChatCompletionUserMessageParam: content: str \| list[chat.ChatCompletionContentPartParam] if isinstance(part.content, str): content = part.content else: content = [] for item in part.content: if isinstance(item, str): content.append(chat.ChatCompletionContentPartTextParam(text=item, type='text')) elif isinstance(item, ImageUrl): image_url = ImageURL(url=item.url) content.append(chat.ChatCompletionContentPartImageParam(image_url=image_url, type='image_url')) elif isinstance(item, BinaryContent): base64_encoded = base64.b64encode(item.data).decode('utf-8') if item.is_image: image_url = ImageURL(url=f'data:{item.media_type};base64,{base64_encoded}') content.append(chat.ChatCompletionContentPartImageParam(image_url=image_url, type='image_url')) else: raise RuntimeError('Only images are supported for binary content in Groq.') elif isinstance(item, DocumentUrl): # pragma: no cover raise RuntimeError('DocumentUrl is not supported in Groq.') else: # pragma: no cover raise RuntimeError(f'Unsupported content type: {type(item)}') return chat.ChatCompletionUserMessageParam(role='user', content=content)`` | #### \_\_init\_\_ `__init__( model_name: [GroqModelName](#pydantic_ai.models.groq.GroqModelName "pydantic_ai.models.groq.GroqModelName") , *, provider: [Literal](https://docs.python.org/3/library/typing.html#typing.Literal "typing.Literal") ["groq"] | [Provider](../../providers/#pydantic_ai.providers.Provider "pydantic_ai.providers.Provider") [AsyncGroq] = "groq" )` Initialize a Groq model. Parameters: | Name | Type | Description | Default | | --- | --- | --- | --- | | `model_name` | `[GroqModelName](#pydantic_ai.models.groq.GroqModelName "pydantic_ai.models.groq.GroqModelName") ` | The name of the Groq model to use. List of model names available [here](https://console.groq.com/docs/models)
. | _required_ | | `provider` | `[Literal](https://docs.python.org/3/library/typing.html#typing.Literal "typing.Literal") ['groq'] \| [Provider](../../providers/#pydantic_ai.providers.Provider "pydantic_ai.providers.Provider") [AsyncGroq]` | The provider to use for authentication and API access. Can be either the string 'groq' or an instance of `Provider[AsyncGroq]`. If not provided, a new provider will be created using the other parameters. | `'groq'` | Source code in `pydantic_ai_slim/pydantic_ai/models/groq.py` | | | | --- | --- | | 109
110
111
112
113
114
115
116
117
118
119
120
121
122
123 | ``def __init__(self, model_name: GroqModelName, *, provider: Literal['groq'] \| Provider[AsyncGroq] = 'groq'): """Initialize a Groq model. Args: model_name: The name of the Groq model to use. List of model names available [here](https://console.groq.com/docs/models). provider: The provider to use for authentication and API access. Can be either the string 'groq' or an instance of `Provider[AsyncGroq]`. If not provided, a new provider will be created using the other parameters. """ self._model_name = model_name if isinstance(provider, str): provider = infer_provider(provider) self.client = provider.client`` | #### model\_name `property` `model_name: [GroqModelName](#pydantic_ai.models.groq.GroqModelName "pydantic_ai.models.groq.GroqModelName")` The model name. #### system `property` `system: [str](https://docs.python.org/3/library/stdtypes.html#str)` The system / model provider. ### GroqStreamedResponse `dataclass` Bases: `[StreamedResponse](../base/#pydantic_ai.models.StreamedResponse "pydantic_ai.models.StreamedResponse") ` Implementation of `StreamedResponse` for Groq models. Source code in `pydantic_ai_slim/pydantic_ai/models/groq.py` | | | | --- | --- | | 352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393 | ``@dataclass class GroqStreamedResponse(StreamedResponse): """Implementation of `StreamedResponse` for Groq models.""" _model_name: GroqModelName _response: AsyncIterable[chat.ChatCompletionChunk] _timestamp: datetime async def _get_event_iterator(self) -> AsyncIterator[ModelResponseStreamEvent]: async for chunk in self._response: self._usage += _map_usage(chunk) try: choice = chunk.choices[0] except IndexError: continue # Handle the text part of the response content = choice.delta.content if content is not None: yield self._parts_manager.handle_text_delta(vendor_part_id='content', content=content) # Handle the tool calls for dtc in choice.delta.tool_calls or []: maybe_event = self._parts_manager.handle_tool_call_delta( vendor_part_id=dtc.index, tool_name=dtc.function and dtc.function.name, args=dtc.function and dtc.function.arguments, tool_call_id=dtc.id, ) if maybe_event is not None: yield maybe_event @property def model_name(self) -> GroqModelName: """Get the model name of the response.""" return self._model_name @property def timestamp(self) -> datetime: """Get the timestamp of the response.""" return self._timestamp`` | #### model\_name `property` `model_name: [GroqModelName](#pydantic_ai.models.groq.GroqModelName "pydantic_ai.models.groq.GroqModelName")` Get the model name of the response. #### timestamp `property` `timestamp: [datetime](https://docs.python.org/3/library/datetime.html#datetime.datetime "datetime.datetime")` Get the timestamp of the response. --- # pydantic_ai.exceptions - PydanticAI `pydantic_ai.exceptions` ======================== ### ModelRetry Bases: `[Exception](https://docs.python.org/3/library/exceptions.html#Exception) ` Exception raised when a tool function should be retried. The agent will return the message to the model and ask it to try calling the function/tool again. Source code in `pydantic_ai_slim/pydantic_ai/exceptions.py` | | | | --- | --- | | 22
23
24
25
26
27
28
29
30
31
32
33 | `class ModelRetry(Exception): """Exception raised when a tool function should be retried. The agent will return the message to the model and ask it to try calling the function/tool again. """ message: str """The message to return to the model.""" def __init__(self, message: str): self.message = message super().__init__(message)` | #### message `instance-attribute` `message: [str](https://docs.python.org/3/library/stdtypes.html#str) = message` The message to return to the model. ### UserError Bases: `[RuntimeError](https://docs.python.org/3/library/exceptions.html#RuntimeError) ` Error caused by a usage mistake by the application developer — You! Source code in `pydantic_ai_slim/pydantic_ai/exceptions.py` | | | | --- | --- | | 36
37
38
39
40
41
42
43
44 | `class UserError(RuntimeError): """Error caused by a usage mistake by the application developer — You!""" message: str """Description of the mistake.""" def __init__(self, message: str): self.message = message super().__init__(message)` | #### message `instance-attribute` `message: [str](https://docs.python.org/3/library/stdtypes.html#str) = message` Description of the mistake. ### AgentRunError Bases: `[RuntimeError](https://docs.python.org/3/library/exceptions.html#RuntimeError) ` Base class for errors occurring during an agent run. Source code in `pydantic_ai_slim/pydantic_ai/exceptions.py` | | | | --- | --- | | 47
48
49
50
51
52
53
54
55
56
57
58 | `class AgentRunError(RuntimeError): """Base class for errors occurring during an agent run.""" message: str """The error message.""" def __init__(self, message: str): self.message = message super().__init__(message) def __str__(self) -> str: return self.message` | #### message `instance-attribute` `message: [str](https://docs.python.org/3/library/stdtypes.html#str) = message` The error message. ### UsageLimitExceeded Bases: `[AgentRunError](#pydantic_ai.exceptions.AgentRunError "pydantic_ai.exceptions.AgentRunError") ` Error raised when a Model's usage exceeds the specified limits. Source code in `pydantic_ai_slim/pydantic_ai/exceptions.py` | | | | --- | --- | | 61
62 | `class UsageLimitExceeded(AgentRunError): """Error raised when a Model's usage exceeds the specified limits."""` | ### UnexpectedModelBehavior Bases: `[AgentRunError](#pydantic_ai.exceptions.AgentRunError "pydantic_ai.exceptions.AgentRunError") ` Error caused by unexpected Model behavior, e.g. an unexpected response code. Source code in `pydantic_ai_slim/pydantic_ai/exceptions.py` | | | | --- | --- | | 65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88 | `class UnexpectedModelBehavior(AgentRunError): """Error caused by unexpected Model behavior, e.g. an unexpected response code.""" message: str """Description of the unexpected behavior.""" body: str \| None """The body of the response, if available.""" def __init__(self, message: str, body: str \| None = None): self.message = message if body is None: self.body: str \| None = None else: try: self.body = json.dumps(json.loads(body), indent=2) except ValueError: self.body = body super().__init__(message) def __str__(self) -> str: if self.body: return f'{self.message}, body:\n{self.body}' else: return self.message` | #### message `instance-attribute` `message: [str](https://docs.python.org/3/library/stdtypes.html#str) = message` Description of the unexpected behavior. #### body `instance-attribute` `body: [str](https://docs.python.org/3/library/stdtypes.html#str) | None = [dumps](https://docs.python.org/3/library/json.html#json.dumps "json.dumps") ([loads](https://docs.python.org/3/library/json.html#json.loads "json.loads") (body), indent=2)` The body of the response, if available. ### ModelHTTPError Bases: `[AgentRunError](#pydantic_ai.exceptions.AgentRunError "pydantic_ai.exceptions.AgentRunError") ` Raised when an model provider response has a status code of 4xx or 5xx. Source code in `pydantic_ai_slim/pydantic_ai/exceptions.py` | | | | --- | --- | | 91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111 | `class ModelHTTPError(AgentRunError): """Raised when an model provider response has a status code of 4xx or 5xx.""" status_code: int """The HTTP status code returned by the API.""" model_name: str """The name of the model associated with the error.""" body: object \| None """The body of the response, if available.""" message: str """The error message with the status code and response body, if available.""" def __init__(self, status_code: int, model_name: str, body: object \| None = None): self.status_code = status_code self.model_name = model_name self.body = body message = f'status_code: {status_code}, model_name: {model_name}, body: {body}' super().__init__(message)` | #### message `instance-attribute` `message: [str](https://docs.python.org/3/library/stdtypes.html#str)` The error message with the status code and response body, if available. #### status\_code `instance-attribute` `status_code: [int](https://docs.python.org/3/library/functions.html#int) = status_code` The HTTP status code returned by the API. #### model\_name `instance-attribute` `model_name: [str](https://docs.python.org/3/library/stdtypes.html#str) = model_name` The name of the model associated with the error. #### body `instance-attribute` `body: [object](https://docs.python.org/3/library/functions.html#object) | None = body` The body of the response, if available. ### FallbackExceptionGroup Bases: `ExceptionGroup` A group of exceptions that can be raised when all fallback models fail. Source code in `pydantic_ai_slim/pydantic_ai/exceptions.py` | | | | --- | --- | | 114
115 | `class FallbackExceptionGroup(ExceptionGroup): """A group of exceptions that can be raised when all fallback models fail."""` | --- # pydantic_ai.settings - PydanticAI `pydantic_ai.settings` ====================== ### ModelSettings Bases: `[TypedDict](https://typing-extensions.readthedocs.io/en/latest/index.html#typing_extensions.TypedDict "typing_extensions.TypedDict") ` Settings to configure an LLM. Here we include only settings which apply to multiple models / model providers, though not all of these settings are supported by all models. Source code in `pydantic_ai_slim/pydantic_ai/settings.py` | | | | --- | --- | | 12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134 | ``class ModelSettings(TypedDict, total=False): """Settings to configure an LLM. Here we include only settings which apply to multiple models / model providers, though not all of these settings are supported by all models. """ max_tokens: int """The maximum number of tokens to generate before stopping. Supported by: * Gemini * Anthropic * OpenAI * Groq * Cohere * Mistral * Bedrock """ temperature: float """Amount of randomness injected into the response. Use `temperature` closer to `0.0` for analytical / multiple choice, and closer to a model's maximum `temperature` for creative and generative tasks. Note that even with `temperature` of `0.0`, the results will not be fully deterministic. Supported by: * Gemini * Anthropic * OpenAI * Groq * Cohere * Mistral * Bedrock """ top_p: float """An alternative to sampling with temperature, called nucleus sampling, where the model considers the results of the tokens with top_p probability mass. So 0.1 means only the tokens comprising the top 10% probability mass are considered. You should either alter `temperature` or `top_p`, but not both. Supported by: * Gemini * Anthropic * OpenAI * Groq * Cohere * Mistral * Bedrock """ timeout: float \| Timeout """Override the client-level default timeout for a request, in seconds. Supported by: * Gemini * Anthropic * OpenAI * Groq * Mistral """ parallel_tool_calls: bool """Whether to allow parallel tool calls. Supported by: * OpenAI (some models, not o1) * Groq * Anthropic """ seed: int """The random seed to use for the model, theoretically allowing for deterministic results. Supported by: * OpenAI * Groq * Cohere * Mistral """ presence_penalty: float """Penalize new tokens based on whether they have appeared in the text so far. Supported by: * OpenAI * Groq * Cohere * Gemini * Mistral """ frequency_penalty: float """Penalize new tokens based on their existing frequency in the text so far. Supported by: * OpenAI * Groq * Cohere * Gemini * Mistral """ logit_bias: dict[str, int] """Modify the likelihood of specified tokens appearing in the completion. Supported by: * OpenAI * Groq """`` | #### max\_tokens `instance-attribute` `max_tokens: [int](https://docs.python.org/3/library/functions.html#int)` The maximum number of tokens to generate before stopping. Supported by: * Gemini * Anthropic * OpenAI * Groq * Cohere * Mistral * Bedrock #### temperature `instance-attribute` `temperature: [float](https://docs.python.org/3/library/functions.html#float)` Amount of randomness injected into the response. Use `temperature` closer to `0.0` for analytical / multiple choice, and closer to a model's maximum `temperature` for creative and generative tasks. Note that even with `temperature` of `0.0`, the results will not be fully deterministic. Supported by: * Gemini * Anthropic * OpenAI * Groq * Cohere * Mistral * Bedrock #### top\_p `instance-attribute` `top_p: [float](https://docs.python.org/3/library/functions.html#float)` An alternative to sampling with temperature, called nucleus sampling, where the model considers the results of the tokens with top\_p probability mass. So 0.1 means only the tokens comprising the top 10% probability mass are considered. You should either alter `temperature` or `top_p`, but not both. Supported by: * Gemini * Anthropic * OpenAI * Groq * Cohere * Mistral * Bedrock #### timeout `instance-attribute` `timeout: [float](https://docs.python.org/3/library/functions.html#float) | Timeout` Override the client-level default timeout for a request, in seconds. Supported by: * Gemini * Anthropic * OpenAI * Groq * Mistral #### parallel\_tool\_calls `instance-attribute` `parallel_tool_calls: [bool](https://docs.python.org/3/library/functions.html#bool)` Whether to allow parallel tool calls. Supported by: * OpenAI (some models, not o1) * Groq * Anthropic #### seed `instance-attribute` `seed: [int](https://docs.python.org/3/library/functions.html#int)` The random seed to use for the model, theoretically allowing for deterministic results. Supported by: * OpenAI * Groq * Cohere * Mistral #### presence\_penalty `instance-attribute` `presence_penalty: [float](https://docs.python.org/3/library/functions.html#float)` Penalize new tokens based on whether they have appeared in the text so far. Supported by: * OpenAI * Groq * Cohere * Gemini * Mistral #### frequency\_penalty `instance-attribute` `frequency_penalty: [float](https://docs.python.org/3/library/functions.html#float)` Penalize new tokens based on their existing frequency in the text so far. Supported by: * OpenAI * Groq * Cohere * Gemini * Mistral #### logit\_bias `instance-attribute` `logit_bias: [dict](https://docs.python.org/3/library/stdtypes.html#dict) [[str](https://docs.python.org/3/library/stdtypes.html#str) , [int](https://docs.python.org/3/library/functions.html#int) ]` Modify the likelihood of specified tokens appearing in the completion. Supported by: * OpenAI * Groq --- # pydantic_ai.models.instrumented - PydanticAI pydantic\_ai.models.instrumented ================================ ### InstrumentationSettings `dataclass` Options for instrumenting models and agents with OpenTelemetry. Used in: * `Agent(instrument=...)` * [`Agent.instrument_all()`](../../agent/#pydantic_ai.agent.Agent.instrument_all) * [`InstrumentedModel`](#pydantic_ai.models.instrumented.InstrumentedModel) See the [Debugging and Monitoring guide](https://ai.pydantic.dev/logfire/) for more info. Source code in `pydantic_ai_slim/pydantic_ai/models/instrumented.py` | | | | --- | --- | | 52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95 | ``@dataclass(init=False) class InstrumentationSettings: """Options for instrumenting models and agents with OpenTelemetry. Used in: - `Agent(instrument=...)` - [`Agent.instrument_all()`][pydantic_ai.agent.Agent.instrument_all] - [`InstrumentedModel`][pydantic_ai.models.instrumented.InstrumentedModel] See the [Debugging and Monitoring guide](https://ai.pydantic.dev/logfire/) for more info. """ tracer: Tracer = field(repr=False) event_logger: EventLogger = field(repr=False) event_mode: Literal['attributes', 'logs'] = 'attributes' def __init__( self, *, event_mode: Literal['attributes', 'logs'] = 'attributes', tracer_provider: TracerProvider \| None = None, event_logger_provider: EventLoggerProvider \| None = None, ): """Create instrumentation options. Args: event_mode: The mode for emitting events. If `'attributes'`, events are attached to the span as attributes. If `'logs'`, events are emitted as OpenTelemetry log-based events. tracer_provider: The OpenTelemetry tracer provider to use. If not provided, the global tracer provider is used. Calling `logfire.configure()` sets the global tracer provider, so most users don't need this. event_logger_provider: The OpenTelemetry event logger provider to use. If not provided, the global event logger provider is used. Calling `logfire.configure()` sets the global event logger provider, so most users don't need this. This is only used if `event_mode='logs'`. """ from pydantic_ai import __version__ tracer_provider = tracer_provider or get_tracer_provider() event_logger_provider = event_logger_provider or get_event_logger_provider() self.tracer = tracer_provider.get_tracer('pydantic-ai', __version__) self.event_logger = event_logger_provider.get_event_logger('pydantic-ai', __version__) self.event_mode = event_mode`` | #### \_\_init\_\_ `__init__( *, event_mode: [Literal](https://docs.python.org/3/library/typing.html#typing.Literal "typing.Literal") [ "attributes", "logs" ] = "attributes", tracer_provider: TracerProvider | None = None, event_logger_provider: EventLoggerProvider | None = None )` Create instrumentation options. Parameters: | Name | Type | Description | Default | | --- | --- | --- | --- | | `event_mode` | `[Literal](https://docs.python.org/3/library/typing.html#typing.Literal "typing.Literal") ['attributes', 'logs']` | The mode for emitting events. If `'attributes'`, events are attached to the span as attributes. If `'logs'`, events are emitted as OpenTelemetry log-based events. | `'attributes'` | | `tracer_provider` | `TracerProvider \| None` | The OpenTelemetry tracer provider to use. If not provided, the global tracer provider is used. Calling `logfire.configure()` sets the global tracer provider, so most users don't need this. | `None` | | `event_logger_provider` | `EventLoggerProvider \| None` | The OpenTelemetry event logger provider to use. If not provided, the global event logger provider is used. Calling `logfire.configure()` sets the global event logger provider, so most users don't need this. This is only used if `event_mode='logs'`. | `None` | Source code in `pydantic_ai_slim/pydantic_ai/models/instrumented.py` | | | | --- | --- | | 69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95 | ``def __init__( self, *, event_mode: Literal['attributes', 'logs'] = 'attributes', tracer_provider: TracerProvider \| None = None, event_logger_provider: EventLoggerProvider \| None = None, ): """Create instrumentation options. Args: event_mode: The mode for emitting events. If `'attributes'`, events are attached to the span as attributes. If `'logs'`, events are emitted as OpenTelemetry log-based events. tracer_provider: The OpenTelemetry tracer provider to use. If not provided, the global tracer provider is used. Calling `logfire.configure()` sets the global tracer provider, so most users don't need this. event_logger_provider: The OpenTelemetry event logger provider to use. If not provided, the global event logger provider is used. Calling `logfire.configure()` sets the global event logger provider, so most users don't need this. This is only used if `event_mode='logs'`. """ from pydantic_ai import __version__ tracer_provider = tracer_provider or get_tracer_provider() event_logger_provider = event_logger_provider or get_event_logger_provider() self.tracer = tracer_provider.get_tracer('pydantic-ai', __version__) self.event_logger = event_logger_provider.get_event_logger('pydantic-ai', __version__) self.event_mode = event_mode`` | ### InstrumentedModel `dataclass` Bases: `[WrapperModel](../wrapper/#pydantic_ai.models.wrapper.WrapperModel "pydantic_ai.models.wrapper.WrapperModel") ` Model which wraps another model so that requests are instrumented with OpenTelemetry. See the [Debugging and Monitoring guide](https://ai.pydantic.dev/logfire/) for more info. Source code in `pydantic_ai_slim/pydantic_ai/models/instrumented.py` | | | | --- | --- | | 102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290 | `@dataclass class InstrumentedModel(WrapperModel): """Model which wraps another model so that requests are instrumented with OpenTelemetry. See the [Debugging and Monitoring guide](https://ai.pydantic.dev/logfire/) for more info. """ settings: InstrumentationSettings """Configuration for instrumenting requests.""" def __init__( self, wrapped: Model \| KnownModelName, options: InstrumentationSettings \| None = None, ) -> None: super().__init__(wrapped) self.settings = options or InstrumentationSettings() async def request( self, messages: list[ModelMessage], model_settings: ModelSettings \| None, model_request_parameters: ModelRequestParameters, ) -> tuple[ModelResponse, Usage]: with self._instrument(messages, model_settings, model_request_parameters) as finish: response, usage = await super().request(messages, model_settings, model_request_parameters) finish(response, usage) return response, usage @asynccontextmanager async def request_stream( self, messages: list[ModelMessage], model_settings: ModelSettings \| None, model_request_parameters: ModelRequestParameters, ) -> AsyncIterator[StreamedResponse]: with self._instrument(messages, model_settings, model_request_parameters) as finish: response_stream: StreamedResponse \| None = None try: async with super().request_stream( messages, model_settings, model_request_parameters ) as response_stream: yield response_stream finally: if response_stream: finish(response_stream.get(), response_stream.usage()) @contextmanager def _instrument( self, messages: list[ModelMessage], model_settings: ModelSettings \| None, model_request_parameters: ModelRequestParameters, ) -> Iterator[Callable[[ModelResponse, Usage], None]]: operation = 'chat' span_name = f'{operation} {self.model_name}' # TODO Missing attributes: # - error.type: unclear if we should do something here or just always rely on span exceptions # - gen_ai.request.stop_sequences/top_k: model_settings doesn't include these attributes: dict[str, AttributeValue] = { 'gen_ai.operation.name': operation, **self.model_attributes(self.wrapped), 'model_request_parameters': json.dumps(InstrumentedModel.serialize_any(model_request_parameters)), 'logfire.json_schema': json.dumps( { 'type': 'object', 'properties': {'model_request_parameters': {'type': 'object'}}, } ), } if model_settings: for key in MODEL_SETTING_ATTRIBUTES: if isinstance(value := model_settings.get(key), (float, int)): attributes[f'gen_ai.request.{key}'] = value with self.settings.tracer.start_as_current_span(span_name, attributes=attributes) as span: def finish(response: ModelResponse, usage: Usage): if not span.is_recording(): return events = self.messages_to_otel_events(messages) for event in self.messages_to_otel_events([response]): events.append( Event( 'gen_ai.choice', body={ # TODO finish_reason 'index': 0, 'message': event.body, }, ) ) new_attributes: dict[str, AttributeValue] = usage.opentelemetry_attributes() # type: ignore attributes.update(getattr(span, 'attributes', {})) request_model = attributes[GEN_AI_REQUEST_MODEL_ATTRIBUTE] new_attributes['gen_ai.response.model'] = response.model_name or request_model span.set_attributes(new_attributes) span.update_name(f'{operation} {request_model}') for event in events: event.attributes = { GEN_AI_SYSTEM_ATTRIBUTE: attributes[GEN_AI_SYSTEM_ATTRIBUTE], **(event.attributes or {}), } self._emit_events(span, events) yield finish def _emit_events(self, span: Span, events: list[Event]) -> None: if self.settings.event_mode == 'logs': for event in events: self.settings.event_logger.emit(event) else: attr_name = 'events' span.set_attributes( { attr_name: json.dumps([self.event_to_dict(event) for event in events]), 'logfire.json_schema': json.dumps( { 'type': 'object', 'properties': { attr_name: {'type': 'array'}, 'model_request_parameters': {'type': 'object'}, }, } ), } ) @staticmethod def model_attributes(model: Model): attributes: dict[str, AttributeValue] = { GEN_AI_SYSTEM_ATTRIBUTE: model.system, GEN_AI_REQUEST_MODEL_ATTRIBUTE: model.model_name, } if base_url := model.base_url: try: parsed = urlparse(base_url) except Exception: # pragma: no cover pass else: if parsed.hostname: attributes['server.address'] = parsed.hostname if parsed.port: attributes['server.port'] = parsed.port return attributes @staticmethod def event_to_dict(event: Event) -> dict[str, Any]: if not event.body: body = {} elif isinstance(event.body, Mapping): body = event.body # type: ignore else: body = {'body': event.body} return {**body, **(event.attributes or {})} @staticmethod def messages_to_otel_events(messages: list[ModelMessage]) -> list[Event]: result: list[Event] = [] for message_index, message in enumerate(messages): message_events: list[Event] = [] if isinstance(message, ModelRequest): for part in message.parts: if hasattr(part, 'otel_event'): message_events.append(part.otel_event()) elif isinstance(message, ModelResponse): message_events = message.otel_events() for event in message_events: event.attributes = { 'gen_ai.message.index': message_index, **(event.attributes or {}), } result.extend(message_events) for event in result: event.body = InstrumentedModel.serialize_any(event.body) return result @staticmethod def serialize_any(value: Any) -> str: try: return ANY_ADAPTER.dump_python(value, mode='json') except Exception: try: return str(value) except Exception as e: return f'Unable to serialize: {e}'` | #### settings `instance-attribute` `settings: [InstrumentationSettings](#pydantic_ai.models.instrumented.InstrumentationSettings "pydantic_ai.models.instrumented.InstrumentationSettings") = ( options or [InstrumentationSettings](#pydantic_ai.models.instrumented.InstrumentationSettings "pydantic_ai.models.instrumented.InstrumentationSettings") () )` Configuration for instrumenting requests. --- # pydantic_ai.models.test - PydanticAI `pydantic_ai.models.test` ========================= Utility model for quickly testing apps built with PydanticAI. Here's a minimal example: test\_model\_usage.py `from pydantic_ai import Agent from pydantic_ai.models.test import TestModel my_agent = Agent('openai:gpt-4o', system_prompt='...') async def test_my_agent(): """Unit test for my_agent, to be run by pytest.""" m = TestModel() with my_agent.override(model=m): result = await my_agent.run('Testing my agent...') assert result.data == 'success (no tool calls)' assert m.last_model_request_parameters.function_tools == []` See [Unit testing with `TestModel`](../../../testing/#unit-testing-with-testmodel) for detailed documentation. ### TestModel `dataclass` Bases: `[Model](../base/#pydantic_ai.models.Model "pydantic_ai.models.Model") ` A model specifically for testing purposes. This will (by default) call all tools in the agent, then return a tool response if possible, otherwise a plain response. How useful this model is will vary significantly. Apart from `__init__` derived by the `dataclass` decorator, all methods are private or match those of the base class. Source code in `pydantic_ai_slim/pydantic_ai/models/test.py` | | | | --- | --- | | 50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227 | ``@dataclass class TestModel(Model): """A model specifically for testing purposes. This will (by default) call all tools in the agent, then return a tool response if possible, otherwise a plain response. How useful this model is will vary significantly. Apart from `__init__` derived by the `dataclass` decorator, all methods are private or match those of the base class. """ # NOTE: Avoid test discovery by pytest. __test__ = False call_tools: list[str] \| Literal['all'] = 'all' """List of tools to call. If `'all'`, all tools will be called.""" custom_result_text: str \| None = None """If set, this text is returned as the final result.""" custom_result_args: Any \| None = None """If set, these args will be passed to the result tool.""" seed: int = 0 """Seed for generating random data.""" last_model_request_parameters: ModelRequestParameters \| None = field(default=None, init=False) """The last ModelRequestParameters passed to the model in a request. The ModelRequestParameters contains information about the function and result tools available during request handling. This is set when a request is made, so will reflect the function tools from the last step of the last run. """ _model_name: str = field(default='test', repr=False) _system: str = field(default='test', repr=False) async def request( self, messages: list[ModelMessage], model_settings: ModelSettings \| None, model_request_parameters: ModelRequestParameters, ) -> tuple[ModelResponse, Usage]: self.last_model_request_parameters = model_request_parameters model_response = self._request(messages, model_settings, model_request_parameters) usage = _estimate_usage([*messages, model_response]) return model_response, usage @asynccontextmanager async def request_stream( self, messages: list[ModelMessage], model_settings: ModelSettings \| None, model_request_parameters: ModelRequestParameters, ) -> AsyncIterator[StreamedResponse]: self.last_model_request_parameters = model_request_parameters model_response = self._request(messages, model_settings, model_request_parameters) yield TestStreamedResponse( _model_name=self._model_name, _structured_response=model_response, _messages=messages ) @property def model_name(self) -> str: """The model name.""" return self._model_name @property def system(self) -> str: """The system / model provider.""" return self._system def gen_tool_args(self, tool_def: ToolDefinition) -> Any: return _JsonSchemaTestData(tool_def.parameters_json_schema, self.seed).generate() def _get_tool_calls(self, model_request_parameters: ModelRequestParameters) -> list[tuple[str, ToolDefinition]]: if self.call_tools == 'all': return [(r.name, r) for r in model_request_parameters.function_tools] else: function_tools_lookup = {t.name: t for t in model_request_parameters.function_tools} tools_to_call = (function_tools_lookup[name] for name in self.call_tools) return [(r.name, r) for r in tools_to_call] def _get_result(self, model_request_parameters: ModelRequestParameters) -> _TextResult \| _FunctionToolResult: if self.custom_result_text is not None: assert model_request_parameters.allow_text_result, ( 'Plain response not allowed, but `custom_result_text` is set.' ) assert self.custom_result_args is None, 'Cannot set both `custom_result_text` and `custom_result_args`.' return _TextResult(self.custom_result_text) elif self.custom_result_args is not None: assert model_request_parameters.result_tools is not None, ( 'No result tools provided, but `custom_result_args` is set.' ) result_tool = model_request_parameters.result_tools[0] if k := result_tool.outer_typed_dict_key: return _FunctionToolResult({k: self.custom_result_args}) else: return _FunctionToolResult(self.custom_result_args) elif model_request_parameters.allow_text_result: return _TextResult(None) elif model_request_parameters.result_tools: return _FunctionToolResult(None) else: return _TextResult(None) def _request( self, messages: list[ModelMessage], model_settings: ModelSettings \| None, model_request_parameters: ModelRequestParameters, ) -> ModelResponse: tool_calls = self._get_tool_calls(model_request_parameters) result = self._get_result(model_request_parameters) result_tools = model_request_parameters.result_tools # if there are tools, the first thing we want to do is call all of them if tool_calls and not any(isinstance(m, ModelResponse) for m in messages): return ModelResponse( parts=[ToolCallPart(name, self.gen_tool_args(args)) for name, args in tool_calls], model_name=self._model_name, ) if messages: last_message = messages[-1] assert isinstance(last_message, ModelRequest), 'Expected last message to be a `ModelRequest`.' # check if there are any retry prompts, if so retry them new_retry_names = {p.tool_name for p in last_message.parts if isinstance(p, RetryPromptPart)} if new_retry_names: # Handle retries for both function tools and result tools # Check function tools first retry_parts: list[ModelResponsePart] = [ ToolCallPart(name, self.gen_tool_args(args)) for name, args in tool_calls if name in new_retry_names ] # Check result tools if result_tools: retry_parts.extend( [ ToolCallPart( tool.name, result.value if isinstance(result, _FunctionToolResult) and result.value is not None else self.gen_tool_args(tool), ) for tool in result_tools if tool.name in new_retry_names ] ) return ModelResponse(parts=retry_parts, model_name=self._model_name) if isinstance(result, _TextResult): if (response_text := result.value) is None: # build up details of tool responses output: dict[str, Any] = {} for message in messages: if isinstance(message, ModelRequest): for part in message.parts: if isinstance(part, ToolReturnPart): output[part.tool_name] = part.content if output: return ModelResponse( parts=[TextPart(pydantic_core.to_json(output).decode())], model_name=self._model_name ) else: return ModelResponse(parts=[TextPart('success (no tool calls)')], model_name=self._model_name) else: return ModelResponse(parts=[TextPart(response_text)], model_name=self._model_name) else: assert result_tools, 'No result tools provided' custom_result_args = result.value result_tool = result_tools[self.seed % len(result_tools)] if custom_result_args is not None: return ModelResponse( parts=[ToolCallPart(result_tool.name, custom_result_args)], model_name=self._model_name ) else: response_args = self.gen_tool_args(result_tool) return ModelResponse(parts=[ToolCallPart(result_tool.name, response_args)], model_name=self._model_name)`` | #### call\_tools `class-attribute` `instance-attribute` `call_tools: [list](https://docs.python.org/3/library/stdtypes.html#list) [[str](https://docs.python.org/3/library/stdtypes.html#str) ] | [Literal](https://docs.python.org/3/library/typing.html#typing.Literal "typing.Literal") ['all'] = 'all'` List of tools to call. If `'all'`, all tools will be called. #### custom\_result\_text `class-attribute` `instance-attribute` `custom_result_text: [str](https://docs.python.org/3/library/stdtypes.html#str) | None = None` If set, this text is returned as the final result. #### custom\_result\_args `class-attribute` `instance-attribute` `custom_result_args: [Any](https://docs.python.org/3/library/typing.html#typing.Any "typing.Any") | None = None` If set, these args will be passed to the result tool. #### seed `class-attribute` `instance-attribute` `seed: [int](https://docs.python.org/3/library/functions.html#int) = 0` Seed for generating random data. #### last\_model\_request\_parameters `class-attribute` `instance-attribute` `last_model_request_parameters: ( [ModelRequestParameters](../base/#pydantic_ai.models.ModelRequestParameters "pydantic_ai.models.ModelRequestParameters") | None ) = [field](https://docs.python.org/3/library/dataclasses.html#dataclasses.field "dataclasses.field") (default=None, init=False)` The last ModelRequestParameters passed to the model in a request. The ModelRequestParameters contains information about the function and result tools available during request handling. This is set when a request is made, so will reflect the function tools from the last step of the last run. #### model\_name `property` `model_name: [str](https://docs.python.org/3/library/stdtypes.html#str)` The model name. #### system `property` `system: [str](https://docs.python.org/3/library/stdtypes.html#str)` The system / model provider. ### TestStreamedResponse `dataclass` Bases: `[StreamedResponse](../base/#pydantic_ai.models.StreamedResponse "pydantic_ai.models.StreamedResponse") ` A structured response that streams test data. Source code in `pydantic_ai_slim/pydantic_ai/models/test.py` | | | | --- | --- | | 230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270 | `@dataclass class TestStreamedResponse(StreamedResponse): """A structured response that streams test data.""" _model_name: str _structured_response: ModelResponse _messages: InitVar[Iterable[ModelMessage]] _timestamp: datetime = field(default_factory=_utils.now_utc, init=False) def __post_init__(self, _messages: Iterable[ModelMessage]): self._usage = _estimate_usage(_messages) async def _get_event_iterator(self) -> AsyncIterator[ModelResponseStreamEvent]: for i, part in enumerate(self._structured_response.parts): if isinstance(part, TextPart): text = part.content *words, last_word = text.split(' ') words = [f'{word} ' for word in words] words.append(last_word) if len(words) == 1 and len(text) > 2: mid = len(text) // 2 words = [text[:mid], text[mid:]] self._usage += _get_string_usage('') yield self._parts_manager.handle_text_delta(vendor_part_id=i, content='') for word in words: self._usage += _get_string_usage(word) yield self._parts_manager.handle_text_delta(vendor_part_id=i, content=word) else: yield self._parts_manager.handle_tool_call_part( vendor_part_id=i, tool_name=part.tool_name, args=part.args, tool_call_id=part.tool_call_id ) @property def model_name(self) -> str: """Get the model name of the response.""" return self._model_name @property def timestamp(self) -> datetime: """Get the timestamp of the response.""" return self._timestamp` | #### model\_name `property` `model_name: [str](https://docs.python.org/3/library/stdtypes.html#str)` Get the model name of the response. #### timestamp `property` `timestamp: [datetime](https://docs.python.org/3/library/datetime.html#datetime.datetime "datetime.datetime")` Get the timestamp of the response. ---