# Table of Contents - [Agent-lightning](#agent-lightning) - [Agent-lightning](#agent-lightning) - [Overview - Agent-lightning](#overview-agent-lightning) - [APO - Agent-lightning](#apo-agent-lightning) - [Changelog - Agent-lightning](#changelog-agent-lightning) - [Examples Catalog - Agent-lightning](#examples-catalog-agent-lightning) - [Train the First Agent - Agent-lightning](#train-the-first-agent-agent-lightning) - [VERL - Agent-lightning](#verl-agent-lightning) - [Contributing Guide - Agent-lightning](#contributing-guide-agent-lightning) - [Agent - Agent-lightning](#agent-agent-lightning) - [Maintainer Guide - Agent-lightning](#maintainer-guide-agent-lightning) - [Serving LLM - Agent-lightning](#serving-llm-agent-lightning) - [Algorithm - Agent-lightning](#algorithm-agent-lightning) - [Internal - Agent-lightning](#internal-agent-lightning) - [Runner - Agent-lightning](#runner-agent-lightning) - [Semantic Conventions - Agent-lightning](#semantic-conventions-agent-lightning) - [Trainer - Agent-lightning](#trainer-agent-lightning) - [Utilities - Agent-lightning](#utilities-agent-lightning) - [Debugging - Agent-lightning](#debugging-agent-lightning) - [Use Emitters - Agent-lightning](#use-emitters-agent-lightning) - [Write Agents - Agent-lightning](#write-agents-agent-lightning) - [SFT with Unsloth - Agent-lightning](#sft-with-unsloth-agent-lightning) - [Store - Agent-lightning](#store-agent-lightning) - [Command Line - Agent-lightning](#command-line-agent-lightning) - [Bird's Eye View - Agent-lightning](#bird-s-eye-view-agent-lightning) - [Understanding Store - Agent-lightning](#understanding-store-agent-lightning) - [Train SQL Agent with RL - Agent-lightning](#train-sql-agent-with-rl-agent-lightning) - [Write the First Algorithm - Agent-lightning](#write-the-first-algorithm-agent-lightning) - [Instrumentation - Agent-lightning](#instrumentation-agent-lightning) - [RESTful - Agent-lightning](#restful-agent-lightning) - [Installation - Agent-lightning](#installation-agent-lightning) - [Parallelize - Agent-lightning](#parallelize-agent-lightning) - [Types - Agent-lightning](#types-agent-lightning) - [Work with Traces - Agent-lightning](#work-with-traces-agent-lightning) - [Agent Lightning](#agent-lightning) - [Installation - Agent-lightning](#installation-agent-lightning) - [Examples Catalog - Agent-lightning](#examples-catalog-agent-lightning) - [Train the First Agent - Agent-lightning](#train-the-first-agent-agent-lightning) - [Write the First Algorithm - Agent-lightning](#write-the-first-algorithm-agent-lightning) - [Write Agents - Agent-lightning](#write-agents-agent-lightning) - [Overview - Agent-lightning](#overview-agent-lightning) - [SFT with Unsloth - Agent-lightning](#sft-with-unsloth-agent-lightning) - [Train SQL Agent with RL - Agent-lightning](#train-sql-agent-with-rl-agent-lightning) - [Work with Traces - Agent-lightning](#work-with-traces-agent-lightning) - [Use Emitters - Agent-lightning](#use-emitters-agent-lightning) - [Command Line - Agent-lightning](#command-line-agent-lightning) - [Debugging - Agent-lightning](#debugging-agent-lightning) - [Serving LLM - Agent-lightning](#serving-llm-agent-lightning) - [APO - Agent-lightning](#apo-agent-lightning) - [RESTful - Agent-lightning](#restful-agent-lightning) - [Instrumentation - Agent-lightning](#instrumentation-agent-lightning) - [VERL - Agent-lightning](#verl-agent-lightning) - [Semantic Conventions - Agent-lightning](#semantic-conventions-agent-lightning) - [Parallelize - Agent-lightning](#parallelize-agent-lightning) - [Maintainer Guide - Agent-lightning](#maintainer-guide-agent-lightning) - [Bird's Eye View - Agent-lightning](#bird-s-eye-view-agent-lightning) - [Understanding Store - Agent-lightning](#understanding-store-agent-lightning) - [Agent - Agent-lightning](#agent-agent-lightning) - [Algorithm - Agent-lightning](#algorithm-agent-lightning) - [Contributing Guide - Agent-lightning](#contributing-guide-agent-lightning) - [Runner - Agent-lightning](#runner-agent-lightning) - [Trainer - Agent-lightning](#trainer-agent-lightning) - [Utilities - Agent-lightning](#utilities-agent-lightning) - [Types - Agent-lightning](#types-agent-lightning) - [Internal - Agent-lightning](#internal-agent-lightning) - [Changelog - Agent-lightning](#changelog-agent-lightning) - [Store - Agent-lightning](#store-agent-lightning) - [Agent-lightning](#agent-lightning) - [Agent-lightning](#agent-lightning) - [Agent-lightning](#agent-lightning) - [Agent-lightning](#agent-lightning) - [RESTful - Agent-lightning](#restful-agent-lightning) - [Semantic Conventions - Agent-lightning](#semantic-conventions-agent-lightning) - [VERL - Agent-lightning](#verl-agent-lightning) - [Debugging - Agent-lightning](#debugging-agent-lightning) - [Installation - Agent-lightning](#installation-agent-lightning) - [Train the First Agent - Agent-lightning](#train-the-first-agent-agent-lightning) - [Write the First Algorithm - Agent-lightning](#write-the-first-algorithm-agent-lightning) - [Installation - Agent-lightning](#installation-agent-lightning) - [Installation - Agent-lightning](#installation-agent-lightning) - [Installation - Agent Lightning](#installation-agent-lightning) - [Installation - Agent-lightning](#installation-agent-lightning) - [Examples Catalog - Agent-lightning](#examples-catalog-agent-lightning) - [Train the First Agent - Agent-lightning](#train-the-first-agent-agent-lightning) - [Train the First Agent - Agent-lightning](#train-the-first-agent-agent-lightning) - [Getting Started - Agent Lightning](#getting-started-agent-lightning) - [Train the First Agent - Agent-lightning](#train-the-first-agent-agent-lightning) - [Train SQL Agent - Agent Lightning](#train-sql-agent-agent-lightning) - [SFT with Unsloth - Agent-lightning](#sft-with-unsloth-agent-lightning) - [Write the First Algorithm - Agent-lightning](#write-the-first-algorithm-agent-lightning) - [Write the First Algorithm - Agent-lightning](#write-the-first-algorithm-agent-lightning) - [Overview - Agent-lightning](#overview-agent-lightning) - [Write the First Algorithm - Agent-lightning](#write-the-first-algorithm-agent-lightning) - [Server-Client Architecture - Agent Lightning](#server-client-architecture-agent-lightning) - [Parallelize - Agent-lightning](#parallelize-agent-lightning) - [Overview - Agent-lightning](#overview-agent-lightning) - [APO - Agent-lightning](#apo-agent-lightning) - [APO - Agent-lightning](#apo-agent-lightning) - [APO - Agent-lightning](#apo-agent-lightning) - [VERL - Agent-lightning](#verl-agent-lightning) - [VERL - Agent-lightning](#verl-agent-lightning) - [VERL - Agent-lightning](#verl-agent-lightning) - [Understanding Store - Agent-lightning](#understanding-store-agent-lightning) - [Serving LLM - Agent-lightning](#serving-llm-agent-lightning) - [Understanding Store - Agent-lightning](#understanding-store-agent-lightning) - [Serving LLM - Agent-lightning](#serving-llm-agent-lightning) - [Bird's Eye View - Agent-lightning](#bird-s-eye-view-agent-lightning) - [Serving LLM - Agent-lightning](#serving-llm-agent-lightning) - [Bird's Eye View - Agent-lightning](#bird-s-eye-view-agent-lightning) - [Understanding Store - Agent-lightning](#understanding-store-agent-lightning) - [Command Line - Agent-lightning](#command-line-agent-lightning) - [Command Line - Agent-lightning](#command-line-agent-lightning) - [Bird's Eye View - Agent-lightning](#bird-s-eye-view-agent-lightning) - [Bird's Eye View - Agent-lightning](#bird-s-eye-view-agent-lightning) - [Command Line - Agent-lightning](#command-line-agent-lightning) - [Instrumentation - Agent-lightning](#instrumentation-agent-lightning) - [Agent - Agent-lightning](#agent-agent-lightning) - [Instrumentation - Agent-lightning](#instrumentation-agent-lightning) - [Instrumentation - Agent-lightning](#instrumentation-agent-lightning) - [Agent - Agent-lightning](#agent-agent-lightning) - [Agent - Agent-lightning](#agent-agent-lightning) - [Understanding Store - Agent-lightning](#understanding-store-agent-lightning) - [Serving LLM - Agent-lightning](#serving-llm-agent-lightning) - [Overview - Agent-lightning](#overview-agent-lightning) --- # Agent-lightning [Skip to content](https://microsoft.github.io/agent-lightning/stable/#agent-lightning) ⚠️ You are viewing development documentation. For stable documentation, please visit [https://microsoft.github.io/agent-lightning/stable/](https://microsoft.github.io/agent-lightning/stable/) Agent Lightning[¶](https://microsoft.github.io/agent-lightning/stable/#agent-lightning "Permanent link") ========================================================================================================= Agent Lightning is the absolute trainer to light up AI agents. [Join our Discord community](https://discord.gg/RYk7CdvDR7) to connect with other users and contributors. Features[¶](https://microsoft.github.io/agent-lightning/stable/#features "Permanent link") ------------------------------------------------------------------------------------------- * Turn your agent into an optimizable beast with **ZERO CODE CHANGE** (almost)! 💤 * Build with **ANY** agent framework (LangChain, OpenAI Agent SDK, AutoGen, CrewAI, Microsoft Agent Framework...); or even WITHOUT agent framework (Python OpenAI). You name it! 🤖 * **Selectively** optimize one or more agents in a multi-agent system. 🎯 * Embraces **Algorithms** like Reinforcement Learning, Automatic Prompt Optimization, Supervised Fine-tuning and more. 🤗 How to Read this Documentation[¶](https://microsoft.github.io/agent-lightning/stable/#how-to-read-this-documentation "Permanent link") --------------------------------------------------------------------------------------------------------------------------------------- This documentation is organized into the following parts: * [Installation](https://microsoft.github.io/agent-lightning/stable/tutorials/installation/) - Get started with Agent Lightning * How-to Recipes (e.g., [Train SQL Agent with RL](https://microsoft.github.io/agent-lightning/stable/how-to/train-sql-agent/) ) - Practical examples of training agents and customizing algorithms. * Learning More (e.g., [Debugging](https://microsoft.github.io/agent-lightning/stable/tutorials/debug/) ) - Guides on specific topics like debugging or parallelization. * Algorithm Zoo (e.g., [APO](https://microsoft.github.io/agent-lightning/stable/algorithm-zoo/apo/) ) - References for built-in algorithms. * Deep Dive (e.g., [Bird's Eye View](https://microsoft.github.io/agent-lightning/stable/deep-dive/birds-eye-view/) ) - For a deeper understanding of what Agent-lightning is doing under the hood. * API References (e.g., [Agent](https://microsoft.github.io/agent-lightning/stable/reference/agent/) ) - References for the Agent-lightning Python API. Resources[¶](https://microsoft.github.io/agent-lightning/stable/#resources "Permanent link") --------------------------------------------------------------------------------------------- * 11/4/2025 [Tuning ANY AI agent with Tinker ✕ Agent-lightning](https://medium.com/@yugez/tuning-any-ai-agent-with-tinker-agent-lightning-part-1-1d8c9a397f0e) Medium. See also [Part 2](https://medium.com/@yugez/tuning-any-ai-agent-with-tinker-agent-lightning-part-2-332c5437f0dc) . * 10/22/2025 [No More Retokenization Drift: Returning Token IDs via the OpenAI Compatible API Matters in Agent RL](https://blog.vllm.ai/2025/10/22/agent-lightning.html) vLLM blog. See also [Zhihu writeup](https://zhuanlan.zhihu.com/p/1965067274642785725) . * 8/11/2025 [Training AI Agents to Write and Self-correct SQL with Reinforcement Learning](https://medium.com/@yugez/training-ai-agents-to-write-and-self-correct-sql-with-reinforcement-learning-571ed31281ad) Medium. * 8/5/2025 [Agent Lightning: Train ANY AI Agents with Reinforcement Learning](https://arxiv.org/abs/2508.03680) arXiv paper. * 7/26/2025 [We discovered an approach to train any AI agent with RL, with (almost) zero code changes.](https://www.reddit.com/r/LocalLLaMA/comments/1m9m670/we_discovered_an_approach_to_train_any_ai_agent/) Reddit. * 6/6/2025 [Agent Lightning - Microsoft Research](https://www.microsoft.com/en-us/research/project/agent-lightning/) Project page. Community Projects[¶](https://microsoft.github.io/agent-lightning/stable/#community-projects "Permanent link") --------------------------------------------------------------------------------------------------------------- * [DeepWerewolf](https://github.com/af-74413592/DeepWerewolf) — A case study of agent RL training for the Chinese Werewolf game built with AgentScope and Agent Lightning. * [AgentFlow](https://agentflow.stanford.edu/) — A modular multi-agent framework that combines planner, executor, verifier, and generator agents with the Flow-GRPO algorithm to tackle long-horizon, sparse-reward tasks. * [Youtu-Agent](https://github.com/TencentCloudADP/Youtu-agent) — Youtu-Agent lets you build and train your agent with ease. Built with [a modified branch](https://github.com/microsoft/agent-lightning/tree/contrib/youtu-agent-lightning) of Agent Lightning, Youtu-Agent has verified up to 128 GPUs RL training on maths/code and search capabilities with steady convergence. Also check [the recipe](https://github.com/TencentCloudADP/youtu-agent/tree/rl/agl) and their blog [_Stop Wrestling with Your Agent RL: How Youtu-Agent Achieved Stable, 128-GPU Scaling Without Breaking a Sweat_](https://spotted-coconut-df8.notion.site/Stop-Wrestling-with-Your-Agent-RL-How-Youtu-Agent-Achieved-Stable-128-GPU-Scaling-Without-Breaking-2ca5e8f089ba80539a98c582b65e0233) . Citation[¶](https://microsoft.github.io/agent-lightning/stable/#citation "Permanent link") ------------------------------------------------------------------------------------------- If you find Agent Lightning useful in your research or projects, please cite our paper: `[](https://microsoft.github.io/agent-lightning/stable/#__codelineno-0-1) @misc{luo2025agentlightningtrainai, [](https://microsoft.github.io/agent-lightning/stable/#__codelineno-0-2) title={Agent Lightning: Train ANY AI Agents with Reinforcement Learning}, [](https://microsoft.github.io/agent-lightning/stable/#__codelineno-0-3) author={Xufang Luo and Yuge Zhang and Zhiyuan He and Zilong Wang and Siyun Zhao and Dongsheng Li and Luna K. Qiu and Yuqing Yang}, [](https://microsoft.github.io/agent-lightning/stable/#__codelineno-0-4) year={2025}, [](https://microsoft.github.io/agent-lightning/stable/#__codelineno-0-5) eprint={2508.03680}, [](https://microsoft.github.io/agent-lightning/stable/#__codelineno-0-6) archivePrefix={arXiv}, [](https://microsoft.github.io/agent-lightning/stable/#__codelineno-0-7) primaryClass={cs.AI}, [](https://microsoft.github.io/agent-lightning/stable/#__codelineno-0-8) url={https://arxiv.org/abs/2508.03680}, [](https://microsoft.github.io/agent-lightning/stable/#__codelineno-0-9) }` License[¶](https://microsoft.github.io/agent-lightning/stable/#license "Permanent link") ----------------------------------------------------------------------------------------- See the [LICENSE](https://github.com/microsoft/agent-lightning/blob/main/LICENSE) file for details. Back to top --- # Agent-lightning [Skip to content](https://microsoft.github.io/agent-lightning/latest/#agent-lightning) ⚠️ You are viewing development documentation. For stable documentation, please visit [https://microsoft.github.io/agent-lightning/stable/](https://microsoft.github.io/agent-lightning/stable/) Agent Lightning[¶](https://microsoft.github.io/agent-lightning/latest/#agent-lightning "Permanent link") ========================================================================================================= Agent Lightning is the absolute trainer to light up AI agents. [Join our Discord community](https://discord.gg/RYk7CdvDR7) to connect with other users and contributors. Features[¶](https://microsoft.github.io/agent-lightning/latest/#features "Permanent link") ------------------------------------------------------------------------------------------- * Turn your agent into an optimizable beast with **ZERO CODE CHANGE** (almost)! 💤 * Build with **ANY** agent framework (LangChain, OpenAI Agent SDK, AutoGen, CrewAI, Microsoft Agent Framework...); or even WITHOUT agent framework (Python OpenAI). You name it! 🤖 * **Selectively** optimize one or more agents in a multi-agent system. 🎯 * Embraces **Algorithms** like Reinforcement Learning, Automatic Prompt Optimization, Supervised Fine-tuning and more. 🤗 How to Read this Documentation[¶](https://microsoft.github.io/agent-lightning/latest/#how-to-read-this-documentation "Permanent link") --------------------------------------------------------------------------------------------------------------------------------------- This documentation is organized into the following parts: * [Installation](https://microsoft.github.io/agent-lightning/latest/tutorials/installation/) - Get started with Agent Lightning * How-to Recipes (e.g., [Train SQL Agent with RL](https://microsoft.github.io/agent-lightning/latest/how-to/train-sql-agent/) ) - Practical examples of training agents and customizing algorithms. * Learning More (e.g., [Debugging](https://microsoft.github.io/agent-lightning/latest/tutorials/debug/) ) - Guides on specific topics like debugging or parallelization. * Algorithm Zoo (e.g., [APO](https://microsoft.github.io/agent-lightning/latest/algorithm-zoo/apo/) ) - References for built-in algorithms. * Deep Dive (e.g., [Bird's Eye View](https://microsoft.github.io/agent-lightning/latest/deep-dive/birds-eye-view/) ) - For a deeper understanding of what Agent-lightning is doing under the hood. * API References (e.g., [Agent](https://microsoft.github.io/agent-lightning/latest/reference/agent/) ) - References for the Agent-lightning Python API. Resources[¶](https://microsoft.github.io/agent-lightning/latest/#resources "Permanent link") --------------------------------------------------------------------------------------------- * 11/4/2025 [Tuning ANY AI agent with Tinker ✕ Agent-lightning](https://medium.com/@yugez/tuning-any-ai-agent-with-tinker-agent-lightning-part-1-1d8c9a397f0e) Medium. See also [Part 2](https://medium.com/@yugez/tuning-any-ai-agent-with-tinker-agent-lightning-part-2-332c5437f0dc) . * 10/22/2025 [No More Retokenization Drift: Returning Token IDs via the OpenAI Compatible API Matters in Agent RL](https://blog.vllm.ai/2025/10/22/agent-lightning.html) vLLM blog. See also [Zhihu writeup](https://zhuanlan.zhihu.com/p/1965067274642785725) . * 8/11/2025 [Training AI Agents to Write and Self-correct SQL with Reinforcement Learning](https://medium.com/@yugez/training-ai-agents-to-write-and-self-correct-sql-with-reinforcement-learning-571ed31281ad) Medium. * 8/5/2025 [Agent Lightning: Train ANY AI Agents with Reinforcement Learning](https://arxiv.org/abs/2508.03680) arXiv paper. * 7/26/2025 [We discovered an approach to train any AI agent with RL, with (almost) zero code changes.](https://www.reddit.com/r/LocalLLaMA/comments/1m9m670/we_discovered_an_approach_to_train_any_ai_agent/) Reddit. * 6/6/2025 [Agent Lightning - Microsoft Research](https://www.microsoft.com/en-us/research/project/agent-lightning/) Project page. Community Projects[¶](https://microsoft.github.io/agent-lightning/latest/#community-projects "Permanent link") --------------------------------------------------------------------------------------------------------------- * [DeepWerewolf](https://github.com/af-74413592/DeepWerewolf) — A case study of agent RL training for the Chinese Werewolf game built with AgentScope and Agent Lightning. * [AgentFlow](https://agentflow.stanford.edu/) — A modular multi-agent framework that combines planner, executor, verifier, and generator agents with the Flow-GRPO algorithm to tackle long-horizon, sparse-reward tasks. * [Youtu-Agent](https://github.com/TencentCloudADP/Youtu-agent) — Youtu-Agent lets you build and train your agent with ease. Built with [a modified branch](https://github.com/microsoft/agent-lightning/tree/contrib/youtu-agent-lightning) of Agent Lightning, Youtu-Agent has verified up to 128 GPUs RL training on maths/code and search capabilities with steady convergence. Also check [the recipe](https://github.com/TencentCloudADP/youtu-agent/tree/rl/agl) and their blog [_Stop Wrestling with Your Agent RL: How Youtu-Agent Achieved Stable, 128-GPU Scaling Without Breaking a Sweat_](https://spotted-coconut-df8.notion.site/Stop-Wrestling-with-Your-Agent-RL-How-Youtu-Agent-Achieved-Stable-128-GPU-Scaling-Without-Breaking-2ca5e8f089ba80539a98c582b65e0233) . Citation[¶](https://microsoft.github.io/agent-lightning/latest/#citation "Permanent link") ------------------------------------------------------------------------------------------- If you find Agent Lightning useful in your research or projects, please cite our paper: `[](https://microsoft.github.io/agent-lightning/latest/#__codelineno-0-1) @misc{luo2025agentlightningtrainai, [](https://microsoft.github.io/agent-lightning/latest/#__codelineno-0-2) title={Agent Lightning: Train ANY AI Agents with Reinforcement Learning}, [](https://microsoft.github.io/agent-lightning/latest/#__codelineno-0-3) author={Xufang Luo and Yuge Zhang and Zhiyuan He and Zilong Wang and Siyun Zhao and Dongsheng Li and Luna K. Qiu and Yuqing Yang}, [](https://microsoft.github.io/agent-lightning/latest/#__codelineno-0-4) year={2025}, [](https://microsoft.github.io/agent-lightning/latest/#__codelineno-0-5) eprint={2508.03680}, [](https://microsoft.github.io/agent-lightning/latest/#__codelineno-0-6) archivePrefix={arXiv}, [](https://microsoft.github.io/agent-lightning/latest/#__codelineno-0-7) primaryClass={cs.AI}, [](https://microsoft.github.io/agent-lightning/latest/#__codelineno-0-8) url={https://arxiv.org/abs/2508.03680}, [](https://microsoft.github.io/agent-lightning/latest/#__codelineno-0-9) }` License[¶](https://microsoft.github.io/agent-lightning/latest/#license "Permanent link") ----------------------------------------------------------------------------------------- See the [LICENSE](https://github.com/microsoft/agent-lightning/blob/main/LICENSE) file for details. Back to top --- # Overview - Agent-lightning [Skip to content](https://microsoft.github.io/agent-lightning/latest/algorithm-zoo/#algorithm-zoo) ⚠️ You are viewing development documentation. For stable documentation, please visit [https://microsoft.github.io/agent-lightning/stable/](https://microsoft.github.io/agent-lightning/stable/) Algorithm Zoo[¶](https://microsoft.github.io/agent-lightning/latest/algorithm-zoo/#algorithm-zoo "Permanent link") =================================================================================================================== AgentLightning includes several popular and frequently requested algorithms in its built-in library, allowing agent developers to use them directly. These algorithms are designed to be compatible with most agent scenarios. For customizing algorithms, see [Algorithm-side References](https://microsoft.github.io/agent-lightning/latest/reference/algorithm/) . | Algorithm | Optimizing Resources | Description | | --- | --- | --- | | [APO](https://microsoft.github.io/agent-lightning/latest/algorithm-zoo/apo/) | `{: [PromptTemplate][agentlightning.PromptTemplate]}` | Automatic Prompt Optimization (APO) algorithm using textual gradients and beam search. | | [VERL](https://microsoft.github.io/agent-lightning/latest/algorithm-zoo/verl/) | `{"main_llm": [LLM][agentlightning.LLM]}` | Reinforcement Learning with [VERL framework](https://github.com/volcengine/verl)
. | Back to top --- # APO - Agent-lightning [Skip to content](https://microsoft.github.io/agent-lightning/latest/algorithm-zoo/apo/#apo) ⚠️ You are viewing development documentation. For stable documentation, please visit [https://microsoft.github.io/agent-lightning/stable/](https://microsoft.github.io/agent-lightning/stable/) APO[¶](https://microsoft.github.io/agent-lightning/latest/algorithm-zoo/apo/#apo "Permanent link") =================================================================================================== Shortcut You can use the shortcut `agl.APO(...)` to create an APO instance. `[](https://microsoft.github.io/agent-lightning/latest/algorithm-zoo/apo/#__codelineno-0-1) import agentlightning as agl [](https://microsoft.github.io/agent-lightning/latest/algorithm-zoo/apo/#__codelineno-0-2) [](https://microsoft.github.io/agent-lightning/latest/algorithm-zoo/apo/#__codelineno-0-3) agl.APO(...)` Installation[¶](https://microsoft.github.io/agent-lightning/latest/algorithm-zoo/apo/#installation "Permanent link") --------------------------------------------------------------------------------------------------------------------- `[](https://microsoft.github.io/agent-lightning/latest/algorithm-zoo/apo/#__codelineno-1-1) pip install agentlightning[apo]` Scope of Current Implementation[¶](https://microsoft.github.io/agent-lightning/latest/algorithm-zoo/apo/#scope-of-current-implementation "Permanent link") ----------------------------------------------------------------------------------------------------------------------------------------------------------- APO is currently scoped to optimize a single prompt template. Optimizing multiple prompt templates is not supported yet. There is however no restriction on the number of variable placeholders in the prompt template (can range from zero to many). It's possible that invalid prompts are created during the optimization process. It is up to the agent developer to ensure that the prompt template is valid for the agent's task. Initial Prompt[¶](https://microsoft.github.io/agent-lightning/latest/algorithm-zoo/apo/#initial-prompt "Permanent link") ------------------------------------------------------------------------------------------------------------------------- APO expects the initial prompt to be provided in the `initial_resources` dictionary. This can be done in two approaches: 1. Pass to the [Trainer](https://microsoft.github.io/agent-lightning/latest/reference/trainer/#agentlightning.Trainer " agentlightning.Trainer") constructor: `[](https://microsoft.github.io/agent-lightning/latest/algorithm-zoo/apo/#__codelineno-2-1) trainer = agl.Trainer( [](https://microsoft.github.io/agent-lightning/latest/algorithm-zoo/apo/#__codelineno-2-2) algorithm=agl.APO(...), [](https://microsoft.github.io/agent-lightning/latest/algorithm-zoo/apo/#__codelineno-2-3) initial_resources={"main_prompt": agl.PromptTemplate(template="You are a helpful assistant.", engine="f-string")}, [](https://microsoft.github.io/agent-lightning/latest/algorithm-zoo/apo/#__codelineno-2-4) )` 1. Pass to the `[APO][agentlightning.algorithm.apo.APO].set_initial_resources()` method: `[](https://microsoft.github.io/agent-lightning/latest/algorithm-zoo/apo/#__codelineno-3-1) algo = agl.APO(...) [](https://microsoft.github.io/agent-lightning/latest/algorithm-zoo/apo/#__codelineno-3-2) algo.set_initial_resources( [](https://microsoft.github.io/agent-lightning/latest/algorithm-zoo/apo/#__codelineno-3-3) {"this_is_also_valid_key": agl.PromptTemplate(template="You are a helpful assistant.", engine="f-string")} [](https://microsoft.github.io/agent-lightning/latest/algorithm-zoo/apo/#__codelineno-3-4) )` The resource key can be arbitrary, which is used to identify the prompt template in [class-based implementations](https://microsoft.github.io/agent-lightning/latest/tutorials/write-agents/) when you have multiple resources. When the key changes, the agent developer needs to update the key in the `rollout` method. Tutorials Using APO[¶](https://microsoft.github.io/agent-lightning/latest/algorithm-zoo/apo/#tutorials-using-apo "Permanent link") ----------------------------------------------------------------------------------------------------------------------------------- * [Train the First Agent with APO](https://microsoft.github.io/agent-lightning/latest/how-to/train-first-agent/) - A step-by-step guide to training your first agent using APO. References[¶](https://microsoft.github.io/agent-lightning/latest/algorithm-zoo/apo/#references "Permanent link") ----------------------------------------------------------------------------------------------------------------- ### `agentlightning.algorithm.apo` [¶](https://microsoft.github.io/agent-lightning/latest/algorithm-zoo/apo/#agentlightning.algorithm.apo "Permanent link") #### `APO` [¶](https://microsoft.github.io/agent-lightning/latest/algorithm-zoo/apo/#agentlightning.algorithm.apo.APO "Permanent link") Bases: `[Algorithm](https://microsoft.github.io/agent-lightning/latest/reference/algorithm/#agentlightning.Algorithm " agentlightning.Algorithm (agentlightning.algorithm.base.Algorithm)") `, `Generic[T_task]` Automatic Prompt Optimization (APO) algorithm using textual gradients and beam search. APO is an iterative prompt optimization algorithm that uses LLM-generated textual gradients to improve prompts through a beam search process. It evaluates prompts on rollouts, computes critiques based on the results, and applies edits to generate improved prompts. The algorithm operates in rounds, where each round: 1. Samples parent prompts from the current beam 2. Generates new prompts by computing textual gradients and applying edits 3. Evaluates all candidates on a validation set 4. Selects the top-k prompts for the next round Based on the ideas from: * [ProTeGi](https://aclanthology.org/2023.emnlp-main.494.pdf) * [TextGrad](https://github.com/zou-group/textgrad) ##### `__init__(async_openai_client, *, gradient_model='gpt-5-mini', apply_edit_model='gpt-4.1-mini', diversity_temperature=1.0, gradient_batch_size=4, val_batch_size=16, beam_width=4, branch_factor=4, beam_rounds=3, rollout_batch_timeout=3600.0, run_initial_validation=True, gradient_prompt_files=None, apply_edit_prompt_files=None, _poml_trace=False)` [¶](https://microsoft.github.io/agent-lightning/latest/algorithm-zoo/apo/#agentlightning.algorithm.apo.APO.__init__ "Permanent link") Initialize the APO algorithm with configuration parameters. Parameters: * **`async_openai_client`** (`AsyncOpenAI`) – AsyncOpenAI client for making LLM API calls. * **`gradient_model`** (`str`, default: `'gpt-5-mini'` ) – Model name for computing textual gradients (critiques). * **`apply_edit_model`** (`str`, default: `'gpt-4.1-mini'` ) – Model name for applying edits based on critiques. * **`diversity_temperature`** (`float`, default: `1.0` ) – Temperature parameter for LLM calls to control diversity. * **`gradient_batch_size`** (`int`, default: `4` ) – Number of rollout results to sample for gradient computation. * **`val_batch_size`** (`int`, default: `16` ) – Number of validation examples to use for evaluation. * **`beam_width`** (`int`, default: `4` ) – Number of top-scoring prompts to keep in the beam at each round. * **`branch_factor`** (`int`, default: `4` ) – Number of new prompt candidates to generate from each parent prompt by applying textual gradient edits. This controls the expansion of the search tree. * **`beam_rounds`** (`int`, default: `3` ) – Number of beam search rounds to perform. * **`rollout_batch_timeout`** (`float`, default: `3600.0` ) – Maximum time in seconds to wait for rollout batch completion. * **`run_initial_validation`** (`bool`, default: `True` ) – If True, runs validation on the seed prompt before starting optimization to establish a baseline score. Defaults to True. * **`gradient_prompt_files`** (`Optional[List[Path]]`, default: `None` ) – Prompt templates used to compute textual gradients (critiques). * **`apply_edit_prompt_files`** (`Optional[List[Path]]`, default: `None` ) – Prompt templates used to apply edits based on critiques. ##### `compute_textual_gradient(current_prompt, rollout_results, *, prefix=None)` `async` [¶](https://microsoft.github.io/agent-lightning/latest/algorithm-zoo/apo/#agentlightning.algorithm.apo.APO.compute_textual_gradient "Permanent link") Compute a textual gradient (critique) for the current prompt based on rollout results. This method samples rollout results, sends them to an LLM along with the current prompt, and generates a critique describing how the prompt could be improved. Parameters: * **`current_prompt`** (`VersionedPromptTemplate`) – The prompt template to critique. * **`rollout_results`** (`List[RolloutResultForAPO]`) – List of rollout results containing spans, messages, and rewards. Returns: * `Optional[str]` – A textual critique generated by the LLM, or None if generation fails. ##### `evaluate_prompt_on_batch(prompt, resource_name, dataset, mode, *, prefix=None)` `async` [¶](https://microsoft.github.io/agent-lightning/latest/algorithm-zoo/apo/#agentlightning.algorithm.apo.APO.evaluate_prompt_on_batch "Permanent link") Evaluate a prompt on a batch of tasks by running rollouts and computing average reward. This method: 1. Adds the prompt as a named resource to the store 2. Enqueues rollouts for each task in the dataset 3. Waits for rollouts to complete (with timeout) 4. Computes and returns the average reward Parameters: * **`prompt`** (`VersionedPromptTemplate`) – The prompt template string to evaluate. * **`resource_name`** (`str`) – The name to register the prompt under in the store. * **`dataset`** (`Sequence[T_task]`) – Sequence of tasks to evaluate the prompt on. * **`mode`** (`[RolloutMode](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.RolloutMode " agentlightning.RolloutMode = Literal['train', 'val', 'test'] module-attribute (agentlightning.types.RolloutMode)") `) – Rollout mode ("train" or "val") for logging/tracking. Returns: * `List[RolloutResultForAPO]` – A tuple of (rollout\_results, average\_reward) where rollout\_results contains * `float` – detailed information for each rollout and average\_reward is the mean final reward. ##### `get_adapter()` [¶](https://microsoft.github.io/agent-lightning/latest/algorithm-zoo/apo/#agentlightning.algorithm.apo.APO.get_adapter "Permanent link") Get the adapter for converting spans to messages. Returns: * `[TraceToMessages](https://microsoft.github.io/agent-lightning/latest/reference/algorithm/#agentlightning.TraceToMessages " agentlightning.TraceToMessages (agentlightning.adapter.messages.TraceToMessages)") ` – The TraceToMessages instance for this algorithm. Raises: * `ValueError` – If the adapter is not a TraceToMessages. ##### `get_best_prompt()` [¶](https://microsoft.github.io/agent-lightning/latest/algorithm-zoo/apo/#agentlightning.algorithm.apo.APO.get_best_prompt "Permanent link") Retrieve the best prompt discovered during optimization. Returns: * `[PromptTemplate](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.PromptTemplate " agentlightning.PromptTemplate (agentlightning.types.PromptTemplate)") ` – The prompt template with the highest validation score found so far. Raises: * `ValueError` – If no best prompt has been found yet (run() not called). ##### `get_rollout_results(store, rollout, *, prefix=None)` `async` [¶](https://microsoft.github.io/agent-lightning/latest/algorithm-zoo/apo/#agentlightning.algorithm.apo.APO.get_rollout_results "Permanent link") Convert completed rollouts to APO-compatible result format. Fetches spans for each rollout, adapts them to messages, and packages them with rewards and status information for gradient computation. Parameters: * **`rollout`** (`List[[Rollout](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.Rollout " agentlightning.Rollout (agentlightning.types.Rollout)") ]`) – List of completed rollout metadata. Returns: * `List[RolloutResultForAPO]` – List of rollout results formatted for APO processing. ##### `get_seed_prompt_template()` [¶](https://microsoft.github.io/agent-lightning/latest/algorithm-zoo/apo/#agentlightning.algorithm.apo.APO.get_seed_prompt_template "Permanent link") Extract the initial prompt template from the algorithm's resources. Returns: * `Tuple[str, [PromptTemplate](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.PromptTemplate " agentlightning.PromptTemplate (agentlightning.types.PromptTemplate)") ]` – A tuple of (resource\_name, prompt\_template) representing the seed prompt. Raises: * `ValueError` – If initial\_resources is not set or no PromptTemplate is found. ##### `run(store, llm_proxy, train_dataset=None, val_dataset=None)` `async` [¶](https://microsoft.github.io/agent-lightning/latest/algorithm-zoo/apo/#agentlightning.algorithm.apo.APO.run "Permanent link") Execute the APO algorithm to optimize prompts through beam search with textual gradients. The algorithm performs iterative prompt optimization over multiple rounds: * Each round: samples parent prompts, generates new candidates via textual gradients, evaluates all candidates on validation data, and keeps the top performers * Tracks the historically best prompt across all rounds * Uses different training data samples for each gradient computation to ensure diversity Parameters: * **`train_dataset`** (`Optional[[Dataset](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.Dataset " agentlightning.Dataset (agentlightning.types.Dataset)") [T_task]]`, default: `None` ) – Dataset of tasks for computing textual gradients. Required. * **`val_dataset`** (`Optional[[Dataset](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.Dataset " agentlightning.Dataset (agentlightning.types.Dataset)") [T_task]]`, default: `None` ) – Dataset of tasks for evaluating and selecting prompts. Required. Raises: * `ValueError` – If train\_dataset or val\_dataset is None, or if resources are not set. ##### `textual_gradient_and_apply_edit(current_prompt, rollout, *, prefix=None)` `async` [¶](https://microsoft.github.io/agent-lightning/latest/algorithm-zoo/apo/#agentlightning.algorithm.apo.APO.textual_gradient_and_apply_edit "Permanent link") Generate an improved prompt by computing a textual gradient and applying an edit. This is the main optimization step that: 1. Computes a critique (textual gradient) based on rollout performance 2. Uses another LLM to apply the critique and generate an improved prompt Parameters: * **`current_prompt`** (`VersionedPromptTemplate`) – The current prompt template to improve. * **`rollout`** (`List[RolloutResultForAPO]`) – List of rollout results to base the critique on. Returns: * `Optional[str]` – The improved prompt text, or the original prompt if gradient computation fails. Back to top --- # Changelog - Agent-lightning [Skip to content](https://microsoft.github.io/agent-lightning/latest/changelog/#changelog) ⚠️ You are viewing development documentation. For stable documentation, please visit [https://microsoft.github.io/agent-lightning/stable/](https://microsoft.github.io/agent-lightning/stable/) Changelog[¶](https://microsoft.github.io/agent-lightning/latest/changelog/#changelog "Permanent link") ======================================================================================================= Agent-lightning v0.3.0 (12/24/2025)[¶](https://microsoft.github.io/agent-lightning/latest/changelog/#agent-lightning-v030-12242025 "Permanent link") ----------------------------------------------------------------------------------------------------------------------------------------------------- Agent-lightning v0.3.0 is a major release that introduces several new features and bug fixes. The release is a collaborative effort between Agent-lightning core teams and the community. Thanks to all the contributors who made this release possible. ### Highlights[¶](https://microsoft.github.io/agent-lightning/latest/changelog/#highlights "Permanent link") * **Tinker integration**: Support Tinker as an alternative backend for Reinforcement Learning (#226 #245 #264 #269 #327). See [example code](https://github.com/microsoft/agent-lightning/tree/v0.3.0/examples/tinker) , [blog 1](https://medium.com/@yugez/tuning-any-ai-agent-with-tinker-agent-lightning-part-1-1d8c9a397f0e) and [blog 2](https://medium.com/@yugez/tuning-any-ai-agent-with-tinker-agent-lightning-part-2-332c5437f0dc) . * **Azure OpenAI integration**: Support Azure OpenAI as a backend for LLM inference and supervised fine-tuning (#256 #327). [Example code](https://github.com/microsoft/agent-lightning/tree/v0.3.0/examples/azure) . * **MongoDB-based Lightning Store** is added as an alternative backend for Lightning Store (#323). [Documentation](https://microsoft.github.io/agent-lightning/0.3.0/tutorials/parallelize/#parallelizing-lightningstore) . * **Contrib package**: Add contrib package for community projects. Search-R1 is integrated as a contrib recipe. More coming. (#239 #396 #410 #412 #417). * **RESTful API**: Stabilize and document RESTful API for Lightning Store (#241 #275). [Documentation](https://microsoft.github.io/agent-lightning/0.3.0/reference/restful/) . * **OTel Semantic Conventions** that are specifically designed for Agent-optimization areas (#340). [Documentation](https://microsoft.github.io/agent-lightning/0.3.0/reference/semconv/) . * _\[Preview\]_ **Agent-lightning Dashboard** is now available (#288 #289 #291 #296 #371 #375). It's the official web application for inspecting and debugging Agent-lightning experiments. See details [here](https://microsoft.github.io/agent-lightning/0.3.0/tutorials/debug/) . * _\[Preview\]_ **Multi-modality example** featuring VERL and a LangGraph agent on ChartQA dataset (#379). [Example code](https://github.com/microsoft/agent-lightning/tree/v0.3.0/examples/chartqa) . * _\[Preview\]_ Integrate **Claude Code** as a LitAgent and support training on SWE-Bench (#332 #346 #348). [Example code](https://github.com/microsoft/agent-lightning/tree/v0.3.0/examples/claude_code) . * _\[Preview\]_ **Weave tracer** as a substitute for AgentOps tracer (#277 #411 #420 #423). [Documentation](https://microsoft.github.io/agent-lightning/0.3.0/tutorials/traces/#weave-tracer-experimental) . * _\[Preview\]_ **Trajectory Level Aggregation** for more efficient training with VERL. See [blog](https://agent-lightning.github.io/posts/trajectory_level_aggregation/) and [documentation](https://microsoft.github.io/agent-lightning/0.3.0/algorithm-zoo/verl/) . ### Store Benchmark[¶](https://microsoft.github.io/agent-lightning/latest/changelog/#store-benchmark "Permanent link") In this release, the Lightning Store core was redesigned for significantly greater efficiency and scalability (#315 #318 #328 #342 #344 #356 #380 #388 #418 #421). The benchmark results below demonstrate the impact: with large numbers of concurrent runners, v0.3.0 delivers up to a 15x increase in throughput compared to v0.2.2. | Throughput (#rollout/sec) | v0.2.2 | v0.3.0 (in-memory) | v0.3.0 (Mongo) | | --- | --- | --- | --- | | Minimal (batch, #runner=32, #turns=6) | 8.73 | 9.06 | 8.71 | | Medium (batch, #runners=100, #turns=10) | 12.03 | 23.26 | 32.79 | | Mid-high (batch, #runners=300, #turns=6) | 10.61 | 24.42 | 40.24 | | Large (batch, #runners=1000, #turns=3) | 3.36 | 14.60 | 50.05 | | Long queue (queue, #runners=256, #turns=4) | 7.42 | 30.86 | 57.01 | | Heavy trace (queue, #runners=512, #turns=20) | 5.93 | 13.28 | 29.41 | _Notes:_ 1. Benchmarks were run on a single Standard\_D32as\_v4 Azure VM (Large and heavy trace tests used Standard\_D64ads\_v5), executed via GitHub Actions. 2. Two algorithm patterns are evaluated: the batch pattern submits a group of rollouts and waits for all to finish before starting the next group, while the queue pattern maintains a set number of in-flight rollouts, submitting new ones as soon as capacity frees up. Configuration details are available [here](https://github.com/microsoft/agent-lightning/blob/v0.3.0/.github/workflows/benchmark.yml) . 3. The number of turns is directly proportional to the number of spans each rollout generates. ### Maintenance and Bug fixes[¶](https://microsoft.github.io/agent-lightning/latest/changelog/#maintenance-and-bug-fixes "Permanent link") #### Core (Store, Interfaces, etc.)[¶](https://microsoft.github.io/agent-lightning/latest/changelog/#core-store-interfaces-etc "Permanent link") * Add Trainer port option for client-server strategies (#198) * Fix store port conflict handling (#227) * Unified PythonServerLauncher (#286 #292 #303) * Make health timeout configurable (#305) * Refactor logging (#306) * Support OTLP in LightningStore (#313) * Centralized metrics helper (#368) * Fix redundant cancel tracebacks on Ctrl+C (#370) #### Proxy, Adapters and Algorithms[¶](https://microsoft.github.io/agent-lightning/latest/changelog/#proxy-adapters-and-algorithms "Permanent link") * Fix training metrics before and after processing in VERL (#145) * Forward streaming requests for Anthropic and OpenAI APIs (as non-streaming requests) (#299) * Check traces with reward for VERL (#317) * Patch LiteLLM root span (#341) * Handle ref\_in\_actor flag for LoRA compatibility (#386) * Support `with_llm_proxy` and `with_store` in algorithms (#398) * Support image URL export in TracerTraceToTriplets (#400) * Fix match\_rewards assign\_to elements in TraceTree (#403) * Support customizing trainer and daemon in VERL (#407) #### Runners, Tracers and Agents[¶](https://microsoft.github.io/agent-lightning/latest/changelog/#runners-tracers-and-agents "Permanent link") * Refactor tracer initialization (#321) * Fix OpenAI Agents 0.6 compatibility (#322) * `emit_operation`, `emit_annotation`, tags and links (#359) * Sunset HTTP tracer (#402) #### Examples[¶](https://microsoft.github.io/agent-lightning/latest/changelog/#examples "Permanent link") * Fix typos in train-first-agent.md (#263) * Fix room\_selector example which always runs the first task (#270) * Fix typo in SQL agent example (#285) * Add the README and script files for training SQL agent on NPU (#272) * Examples Catalog and Refine Contribution Guide (#331) * Upgrade LangChain to 1.x (#364) * Update RAG example to Agent-lightning v0.2.x (#349) #### Miscellaneous[¶](https://microsoft.github.io/agent-lightning/latest/changelog/#miscellaneous "Permanent link") * DeepWiki Badge (#263) * Add AGENTS.md (#374) ### New Contributors[¶](https://microsoft.github.io/agent-lightning/latest/changelog/#new-contributors "Permanent link") Warm welcome to our first-time contributors: @cptnm3, @TerryChan, @genji970, @zxgx, @xiaochulaoban, @lspinheiro, @Kwanghoon-Choi, @Vasuk12, @totoluo, @jinghuan-Chen 🎉 **Full Changelog**: https://github.com/microsoft/agent-lightning/compare/v0.2.0...v0.3.0 * * * Agent-lightning v0.2.2 (11/12/2025)[¶](https://microsoft.github.io/agent-lightning/latest/changelog/#agent-lightning-v022-11122025 "Permanent link") ----------------------------------------------------------------------------------------------------------------------------------------------------- Agent-lightning v0.2.2 is a stabilization release for v0.2.1. It introduces several bug fixes. * Fix compatibility issues with VERL 0.6.0. * Fix model name for pre-downloaded models in VERL. * Fix preparing status transition on rollout when creating attempts. * Fix OpenAI Agents SDK compatibility issues. **Full Changelog**: https://github.com/microsoft/agent-lightning/compare/v0.2.1...v0.2.2 * * * Agent-lightning v0.2.1 (10/30/2025)[¶](https://microsoft.github.io/agent-lightning/latest/changelog/#agent-lightning-v021-10302025 "Permanent link") ----------------------------------------------------------------------------------------------------------------------------------------------------- Agent-lightning v0.2.1 is a stabilization release for v0.2.0. It introduces several bug fixes and new features, plus a number of unlisted CI improvements. ### Bug fixes[¶](https://microsoft.github.io/agent-lightning/latest/changelog/#bug-fixes "Permanent link") * Fix LiteLLM issues when restarting the proxy multiple times in the same process (#174 #206) * Fix LiteLLM model name selection when multiple servers use the same model (#197) * Fix store port conflict handling (#227) ### New Features[¶](https://microsoft.github.io/agent-lightning/latest/changelog/#new-features "Permanent link") * Add trainer port option for client-server strategies (#198) ### Documentation[¶](https://microsoft.github.io/agent-lightning/latest/changelog/#documentation "Permanent link") * Add tutorial for launching workers on separate machines (#213) * Add link to VERL framework (#210) * Add link to vLLM blog (#215) * Fix a couple of typos and avoid emacs backup files (#237) ### New Contributors[¶](https://microsoft.github.io/agent-lightning/latest/changelog/#new-contributors_1 "Permanent link") A warm welcome to our first-time contributors: @scott-vsi, @ddsfda99, @jeis4wpi 🎉 **Full Changelog**: https://github.com/microsoft/agent-lightning/compare/v0.2.0...v0.2.1 * * * Agent-lightning v0.2.0 (10/22/2025)[¶](https://microsoft.github.io/agent-lightning/latest/changelog/#agent-lightning-v020-10222025 "Permanent link") ----------------------------------------------------------------------------------------------------------------------------------------------------- Agent-Lightning v0.2.0 introduces major framework improvements, new execution strategies, expanded documentation, and enhanced reliability across the agent training and deployment workflow. This release includes **78 pull requests** since v0.1.2. ### Core Enhancements[¶](https://microsoft.github.io/agent-lightning/latest/changelog/#core-enhancements "Permanent link") * **Lightning Store**: Added unified interface and implementation for Agent-lightning's core storage. * **Emitter**: Emitting any objects as spans to the store. * **Adapter** and **Tracer**: Adapting to OpenAI-like messages, and OpenTelemetry dummy tracer. * **LLM Proxy**: Added LLM Proxy as the first-class citizen in Agent-lightning. * **Agent Runner**: New version providing a more modular and robust runner design. * **Embedded Algorithms**: Algorithms are now embedded directly into trainers for simplicity. * **New Execution Strategies**: Introduced _Client-Server_ and _Shared Memory_ execution models. * **Trainer Updates**: Integrated v0.2 interfaces and FastAlgorithm validation. ### Documentation & Examples[¶](https://microsoft.github.io/agent-lightning/latest/changelog/#documentation-examples "Permanent link") * Revamped documentation with new guides for **agent creation**, **training**, **debugging**, and **store concepts**. * Improved quickstart tutorials, clarified installation and new deep-dive articles. * Added and updated examples: _SQL Agent_, _Calc-X_, _Local SFT_, _Search-R1_, and _APO algorithm_. ### Developer Experience[¶](https://microsoft.github.io/agent-lightning/latest/changelog/#developer-experience "Permanent link") * Migrated build and CI pipelines to **1ES**, split workflows and aggregate badges for clarity. * Adopted **uv** as the dependency manager. * Added GPU-based pytest workflows for full test coverage. * Enhanced debugging UX, pre-commit configs, and linting (Pyright fixes, import sorting). ### Ecosystem & Integrations[¶](https://microsoft.github.io/agent-lightning/latest/changelog/#ecosystem-integrations "Permanent link") * Added support for agents built with [**Agent-framework**](https://github.com/microsoft/agent-framework) . * Added new community listings: [_DeepWerewolf_](https://github.com/af-74413592/DeepWerewolf) and [_AgentFlow_](https://agentflow.stanford.edu/) . ### New Contributors[¶](https://microsoft.github.io/agent-lightning/latest/changelog/#new-contributors_2 "Permanent link") A warm welcome to our first-time contributors: @hzy46, @lunaqiu, @syeehyn, @linhx1999, @SiyunZhao, and @acured 🎉 **Full changelog:** [v0.1.2 → v0.2.0](https://github.com/microsoft/agent-lightning/compare/v0.1.2...v0.2.0) * * * Agent-lightning v0.1.2 (08/12/2025)[¶](https://microsoft.github.io/agent-lightning/latest/changelog/#agent-lightning-v012-08122025 "Permanent link") ----------------------------------------------------------------------------------------------------------------------------------------------------- ### What's Changed[¶](https://microsoft.github.io/agent-lightning/latest/changelog/#whats-changed "Permanent link") * Add basic documentation in https://github.com/microsoft/agent-lightning/pull/33 * RAG example by @wizardlancet in https://github.com/microsoft/agent-lightning/pull/21 ### New Contributors[¶](https://microsoft.github.io/agent-lightning/latest/changelog/#new-contributors_3 "Permanent link") * @wizardlancet made their first contribution in https://github.com/microsoft/agent-lightning/pull/21 **Full Changelog**: https://github.com/microsoft/agent-lightning/compare/v0.1.1...v0.1.2 * * * Agent-lightning v0.1.1 (08/06/2025)[¶](https://microsoft.github.io/agent-lightning/latest/changelog/#agent-lightning-v011-08062025 "Permanent link") ----------------------------------------------------------------------------------------------------------------------------------------------------- ### What's Changed[¶](https://microsoft.github.io/agent-lightning/latest/changelog/#whats-changed_1 "Permanent link") * Disable HTTP tracer tests and bump to 0.1.1 in https://github.com/microsoft/agent-lightning/pull/26 * Fix trainer bugs in v0.1 in https://github.com/microsoft/agent-lightning/pull/24 **Full Changelog**: https://github.com/microsoft/agent-lightning/compare/v0.1...v0.1.1 * * * Agent-lightning v0.1.0 (08/04/2025)[¶](https://microsoft.github.io/agent-lightning/latest/changelog/#agent-lightning-v010-08042025 "Permanent link") ----------------------------------------------------------------------------------------------------------------------------------------------------- The first release of Agent-lightning! * Turn your agent into an optimizable beast with **ZERO CODE CHANGE** (almost)! 💤 * Build with **ANY** agent framework (LangChain, OpenAI Agent SDK, AutoGen, CrewAI, ...); or even WITHOUT agent framework (Python OpenAI). You name it! 🤖 * **Selectively** optimize one or more agents in a multi-agent system. 🎯 * Embraces Reinforcement Learning, Automatic Prompt Optimization and more **algorithms**. 🤗 Install via `pip install agentlightning`. Back to top --- # Examples Catalog - Agent-lightning [Skip to content](https://microsoft.github.io/agent-lightning/latest/how-to/examples-catalog/#examples-catalog) ⚠️ You are viewing development documentation. For stable documentation, please visit [https://microsoft.github.io/agent-lightning/stable/](https://microsoft.github.io/agent-lightning/stable/) Examples Catalog[¶](https://microsoft.github.io/agent-lightning/latest/how-to/examples-catalog/#examples-catalog "Permanent link") =================================================================================================================================== Want to Contribute? We welcome contributions to the examples catalog! Please refer to the [Contributing](https://microsoft.github.io/agent-lightning/latest/community/contributing/) guide for more details. * **APO room selector** * * * Prompt-optimize a room-booking agent with the built-in APO algorithm, then contrast it with the write-your-own algorithm and debugging workflows in the tutorials. Pairs well with the [Train the First Agent how-to](https://github.com/microsoft/agent-lightning/tree/c746af2f76bebcae56007a59c223cdace411c89b/docs/how-to/train-first-agent.md) and the [Write the First Algorithm guide](https://github.com/microsoft/agent-lightning/tree/c746af2f76bebcae56007a59c223cdace411c89b/docs/how-to/write-first-algorithm.md) . [Browse source](https://github.com/microsoft/agent-lightning/tree/c746af2f76bebcae56007a59c223cdace411c89b/examples/apo) * **Azure OpenAI SFT** * * * Run a supervised fine-tuning loop against Azure OpenAI: roll out the capital-lookup agent, turn traces into JSONL, launch fine-tunes, and redeploy the resulting checkpoints through Azure CLI. [Browse source](https://github.com/microsoft/agent-lightning/tree/c746af2f76bebcae56007a59c223cdace411c89b/examples/azure) * **Calc-X VERL math** * * * VERL-based reinforcement learning setup for a math-reasoning agent that uses AutoGen plus an MCP calculator tool to solve Calc-X problems end to end. [Browse source](https://github.com/microsoft/agent-lightning/tree/c746af2f76bebcae56007a59c223cdace411c89b/examples/calc_x) * **ChartQA vision-language RL** * * * LangGraph-powered workflow for answering chart questions end to end: rollout the multi-modality agent with GPT or vLLM, and train with VERL/GRPO plus self-refinement loops. [Browse source](https://github.com/microsoft/agent-lightning/tree/c746af2f76bebcae56007a59c223cdace411c89b/examples/chartqa) * **Claude Code SWE-bench** * * * Instrumented driver that runs Anthropic's Claude Code workflow on SWE-bench instances while streaming traces through Agent-lightning—supports hosted vLLM, official Anthropic, or any OpenAI-compatible backend and emits datasets for downstream tuning. [Browse source](https://github.com/microsoft/agent-lightning/tree/c746af2f76bebcae56007a59c223cdace411c89b/examples/claude_code) * **Minimal building blocks** * * * Bite-sized scripts that isolate Agent-lightning primitives (e.g., LightningStore usage, LLM proxying, minimal vLLM host) so you can study each part before composing larger workflows. [Browse source](https://github.com/microsoft/agent-lightning/tree/c746af2f76bebcae56007a59c223cdace411c89b/examples/minimal) * **RAG (MuSiQue)** * * * Retrieval-Augmented Generation pipeline that preps a Wikipedia retriever via MCP and trains a MuSiQue QA agent with GRPO. Documented for historical reference (verified on Agent-lightning v0.1.x). [Browse source](https://github.com/microsoft/agent-lightning/tree/c746af2f76bebcae56007a59c223cdace411c89b/examples/rag) * **Spider SQL agent** * * * LangGraph-powered text-to-SQL workflow for the Spider benchmark, combining LangChain tooling with Agent-lightning rollouts; follow along with the [how-to for training SQL agents](https://github.com/microsoft/agent-lightning/tree/c746af2f76bebcae56007a59c223cdace411c89b/docs/how-to/train-sql-agent.md) . [Browse source](https://github.com/microsoft/agent-lightning/tree/c746af2f76bebcae56007a59c223cdace411c89b/examples/spider) * **Tinker integration** * * * Adapter package ([`agl_tinker`](https://github.com/microsoft/agent-lightning/tree/c746af2f76bebcae56007a59c223cdace411c89b/examples/tinker/agl_tinker) ) with Tinker plus sample CrewAI/OpenAI agents that feed Agent-lightning traces into Tinker’s reinforcement-learning backend for both toy and 20-Questions-style workflows. [Browse source](https://github.com/microsoft/agent-lightning/tree/c746af2f76bebcae56007a59c223cdace411c89b/examples/tinker) * **Unsloth SFT** * * * Supervised fine-tuning loop that ranks math-agent rollouts, fine-tunes with Unsloth’s 4-bit LoRA stack, and mirrors the [Fine-tune with Unsloth recipe](https://github.com/microsoft/agent-lightning/tree/c746af2f76bebcae56007a59c223cdace411c89b/docs/how-to/unsloth-sft.md) . [Browse source](https://github.com/microsoft/agent-lightning/tree/c746af2f76bebcae56007a59c223cdace411c89b/examples/unsloth) Back to top --- # Train the First Agent - Agent-lightning [Skip to content](https://microsoft.github.io/agent-lightning/latest/how-to/train-first-agent/#train-the-first-agent-with-agent-lightning) ⚠️ You are viewing development documentation. For stable documentation, please visit [https://microsoft.github.io/agent-lightning/stable/](https://microsoft.github.io/agent-lightning/stable/) Train the First Agent with Agent-lightning[¶](https://microsoft.github.io/agent-lightning/latest/how-to/train-first-agent/#train-the-first-agent-with-agent-lightning "Permanent link") ======================================================================================================================================================================================== Welcome! This tutorial is your first step into making AI agents smarter using the **Agent-lightning** framework. We'll show you how to take a simple agent and automatically improve its performance through a process called [**Automatic Prompt Optimization (APO)**](https://microsoft.github.io/agent-lightning/latest/algorithm-zoo/apo/) . The main goal of Agent-lightning is to provide a structured way to **train your agents**. Just like you train a machine learning model on data, you can train an agent on a task dataset. This could involve using Reinforcement Learning (RL) to teach it new behaviors or, as we'll do today, optimizing its prompts to make it more accurate and reliable. Tip You can open the sample code [room\_selector\_apo.py](https://github.com/microsoft/agent-lightning/tree/c746af2f76bebcae56007a59c223cdace411c89b/examples/apo/room_selector_apo.py) and [room\_selector.py](https://github.com/microsoft/agent-lightning/tree/c746af2f76bebcae56007a59c223cdace411c89b/examples/apo/room_selector.py) as you go through this tutorial. Our Example: The Room Selector Agent[¶](https://microsoft.github.io/agent-lightning/latest/how-to/train-first-agent/#our-example-the-room-selector-agent "Permanent link") --------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Today, we'll work with an agent whose job is to book a meeting room. It's a common but tricky task with multiple constraints. Here's how the agent works: * **Input:** It receives a task with specific requirements, like "`Find a room for 4 people at 10:00 AM with a whiteboard.`" * **Action:** The agent uses a Large Language Model (LLM) to understand the request. It can also use tools, which are pre-defined functions it can call, to get more information, such as checking room availability in an external database. * **Output:** Its final decision is the ID of the best room it found, like "`A103`". * **Reward:** After the agent makes its choice, a separate "grader" function scores its performance on a scale of 0 to 1. This score is called its **reward**. A perfect choice gets a 1.0, while a wrong one gets a 0.0. The agent's logic is sound, but its performance heavily depends on its initial prompt. A poorly worded prompt will confuse the LLM, leading to bad decisions. Our goal is to use Agent-lightning to find the best possible prompt automatically. A Closer Look at the Agent's Logic Modern LLMs can do more than just generate text; they can decide to call functions you provide. This is often called tool use or function calling. Our agent uses this capability to make informed decisions. If you're new to this concept, you can read more about it in [OpenAI's documentation](https://platform.openai.com/docs/guides/function-calling) . Here is a sketch of the agent's logic, adhering closely to the OpenAI API: `[](https://microsoft.github.io/agent-lightning/latest/how-to/train-first-agent/#__codelineno-0-1) # Pseudo-code for the Room Selector agent [](https://microsoft.github.io/agent-lightning/latest/how-to/train-first-agent/#__codelineno-0-2) [](https://microsoft.github.io/agent-lightning/latest/how-to/train-first-agent/#__codelineno-0-3) import openai [](https://microsoft.github.io/agent-lightning/latest/how-to/train-first-agent/#__codelineno-0-4) import json [](https://microsoft.github.io/agent-lightning/latest/how-to/train-first-agent/#__codelineno-0-5) [](https://microsoft.github.io/agent-lightning/latest/how-to/train-first-agent/#__codelineno-0-6) def room_selector_agent(task, prompt): [](https://microsoft.github.io/agent-lightning/latest/how-to/train-first-agent/#__codelineno-0-7) client = openai.OpenAI() [](https://microsoft.github.io/agent-lightning/latest/how-to/train-first-agent/#__codelineno-0-8) messages = [{"role": "user", "content": prompt.format(**task)}] [](https://microsoft.github.io/agent-lightning/latest/how-to/train-first-agent/#__codelineno-0-9) tools = [ ... ] # Tool definition for the LLM [](https://microsoft.github.io/agent-lightning/latest/how-to/train-first-agent/#__codelineno-0-10) [](https://microsoft.github.io/agent-lightning/latest/how-to/train-first-agent/#__codelineno-0-11) # 1. First LLM call to decide if a tool is needed. [](https://microsoft.github.io/agent-lightning/latest/how-to/train-first-agent/#__codelineno-0-12) response = client.chat.completions.create( [](https://microsoft.github.io/agent-lightning/latest/how-to/train-first-agent/#__codelineno-0-13) model="gpt-5-mini", [](https://microsoft.github.io/agent-lightning/latest/how-to/train-first-agent/#__codelineno-0-14) messages=messages, [](https://microsoft.github.io/agent-lightning/latest/how-to/train-first-agent/#__codelineno-0-15) tools=tools, [](https://microsoft.github.io/agent-lightning/latest/how-to/train-first-agent/#__codelineno-0-16) tool_choice="auto", [](https://microsoft.github.io/agent-lightning/latest/how-to/train-first-agent/#__codelineno-0-17) ) [](https://microsoft.github.io/agent-lightning/latest/how-to/train-first-agent/#__codelineno-0-18) response_message = response.choices[0].message [](https://microsoft.github.io/agent-lightning/latest/how-to/train-first-agent/#__codelineno-0-19) tool_calls = response_message.tool_calls [](https://microsoft.github.io/agent-lightning/latest/how-to/train-first-agent/#__codelineno-0-20) [](https://microsoft.github.io/agent-lightning/latest/how-to/train-first-agent/#__codelineno-0-21) # 2. Check if the LLM wants to use a tool. [](https://microsoft.github.io/agent-lightning/latest/how-to/train-first-agent/#__codelineno-0-22) if tool_calls: [](https://microsoft.github.io/agent-lightning/latest/how-to/train-first-agent/#__codelineno-0-23) messages.append(response_message) # Append assistant's reply [](https://microsoft.github.io/agent-lightning/latest/how-to/train-first-agent/#__codelineno-0-24) [](https://microsoft.github.io/agent-lightning/latest/how-to/train-first-agent/#__codelineno-0-25) # 3. Execute the tool and get the real-world data. [](https://microsoft.github.io/agent-lightning/latest/how-to/train-first-agent/#__codelineno-0-26) for tool_call in tool_calls: [](https://microsoft.github.io/agent-lightning/latest/how-to/train-first-agent/#__codelineno-0-27) function_name = tool_call.function.name [](https://microsoft.github.io/agent-lightning/latest/how-to/train-first-agent/#__codelineno-0-28) if function_name == "get_rooms_and_availability": [](https://microsoft.github.io/agent-lightning/latest/how-to/train-first-agent/#__codelineno-0-29) function_args = json.loads(tool_call.function.arguments) [](https://microsoft.github.io/agent-lightning/latest/how-to/train-first-agent/#__codelineno-0-30) # Query the local room database [](https://microsoft.github.io/agent-lightning/latest/how-to/train-first-agent/#__codelineno-0-31) function_response = get_rooms_and_availability( [](https://microsoft.github.io/agent-lightning/latest/how-to/train-first-agent/#__codelineno-0-32) date=function_args.get("date"), [](https://microsoft.github.io/agent-lightning/latest/how-to/train-first-agent/#__codelineno-0-33) time_str=function_args.get("time"), [](https://microsoft.github.io/agent-lightning/latest/how-to/train-first-agent/#__codelineno-0-34) duration_min=function_args.get("duration_min"), [](https://microsoft.github.io/agent-lightning/latest/how-to/train-first-agent/#__codelineno-0-35) ) [](https://microsoft.github.io/agent-lightning/latest/how-to/train-first-agent/#__codelineno-0-36) messages.append({ [](https://microsoft.github.io/agent-lightning/latest/how-to/train-first-agent/#__codelineno-0-37) "tool_call_id": tool_call.id, [](https://microsoft.github.io/agent-lightning/latest/how-to/train-first-agent/#__codelineno-0-38) "role": "tool", [](https://microsoft.github.io/agent-lightning/latest/how-to/train-first-agent/#__codelineno-0-39) "name": function_name, [](https://microsoft.github.io/agent-lightning/latest/how-to/train-first-agent/#__codelineno-0-40) "content": json.dumps(function_response), [](https://microsoft.github.io/agent-lightning/latest/how-to/train-first-agent/#__codelineno-0-41) }) [](https://microsoft.github.io/agent-lightning/latest/how-to/train-first-agent/#__codelineno-0-42) [](https://microsoft.github.io/agent-lightning/latest/how-to/train-first-agent/#__codelineno-0-43) # 4. Second LLM call with the tool's output to get a final choice. [](https://microsoft.github.io/agent-lightning/latest/how-to/train-first-agent/#__codelineno-0-44) second_response = client.chat.completions.create( [](https://microsoft.github.io/agent-lightning/latest/how-to/train-first-agent/#__codelineno-0-45) model="gpt-5-mini", [](https://microsoft.github.io/agent-lightning/latest/how-to/train-first-agent/#__codelineno-0-46) messages=messages, [](https://microsoft.github.io/agent-lightning/latest/how-to/train-first-agent/#__codelineno-0-47) ) [](https://microsoft.github.io/agent-lightning/latest/how-to/train-first-agent/#__codelineno-0-48) final_choice = second_response.choices[0].message.content [](https://microsoft.github.io/agent-lightning/latest/how-to/train-first-agent/#__codelineno-0-49) else: [](https://microsoft.github.io/agent-lightning/latest/how-to/train-first-agent/#__codelineno-0-50) final_choice = response_message.content [](https://microsoft.github.io/agent-lightning/latest/how-to/train-first-agent/#__codelineno-0-51) [](https://microsoft.github.io/agent-lightning/latest/how-to/train-first-agent/#__codelineno-0-52) # 5. Grade the final choice to get a reward. [](https://microsoft.github.io/agent-lightning/latest/how-to/train-first-agent/#__codelineno-0-53) reward = grade_the_choice(final_choice, task["expected_choice"]) [](https://microsoft.github.io/agent-lightning/latest/how-to/train-first-agent/#__codelineno-0-54) return reward` In Agent-lightning, you wrap this logic in a Python function marked with the [`@rollout`](https://microsoft.github.io/agent-lightning/latest/reference/agent/#agentlightning.rollout " agentlightning.rollout(func)") decorator, so that the agent can be managed and tuned by Agent-lightning's runner and trainer. The `prompt_template` that the APO algorithm tunes is passed in as an argument: `[](https://microsoft.github.io/agent-lightning/latest/how-to/train-first-agent/#__codelineno-1-1) import agentlightning as agl [](https://microsoft.github.io/agent-lightning/latest/how-to/train-first-agent/#__codelineno-1-2) [](https://microsoft.github.io/agent-lightning/latest/how-to/train-first-agent/#__codelineno-1-3) @agl.rollout [](https://microsoft.github.io/agent-lightning/latest/how-to/train-first-agent/#__codelineno-1-4) def room_selector(task: RoomSelectionTask, prompt_template: agl.PromptTemplate) -> float: [](https://microsoft.github.io/agent-lightning/latest/how-to/train-first-agent/#__codelineno-1-5) # ... agent logic using the prompt_template ... [](https://microsoft.github.io/agent-lightning/latest/how-to/train-first-agent/#__codelineno-1-6) [](https://microsoft.github.io/agent-lightning/latest/how-to/train-first-agent/#__codelineno-1-7) # The final reward is determined by a grader function [](https://microsoft.github.io/agent-lightning/latest/how-to/train-first-agent/#__codelineno-1-8) reward = room_selection_grader(client, final_message, task["expected_choice"]) [](https://microsoft.github.io/agent-lightning/latest/how-to/train-first-agent/#__codelineno-1-9) return reward` Core Concepts: Tasks, Rollouts, Spans, and Prompt Templates[¶](https://microsoft.github.io/agent-lightning/latest/how-to/train-first-agent/#core-concepts-tasks-rollouts-spans-and-prompt-templates "Permanent link") ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- To understand how Agent-lightning works, you need to know these key terms. ### Task[¶](https://microsoft.github.io/agent-lightning/latest/how-to/train-first-agent/#task "Permanent link") A task is a specific input or problem statement given to the agent. It defines what the agent needs to accomplish. Analogy: Task If the agent is a chef, a task is the recipe request: "Bake a chocolate cake." ### Rollout[¶](https://microsoft.github.io/agent-lightning/latest/how-to/train-first-agent/#rollout "Permanent link") A rollout is a single, complete execution of an agent attempting to solve a given **task**. It's the entire story from receiving the task to producing a final result and receiving a reward. A rollout captures a full trace of the agent's execution. Analogy: Rollout A rollout is one full attempt by the chef to bake the chocolate cake, from gathering ingredients to the final taste test. ### Span[¶](https://microsoft.github.io/agent-lightning/latest/how-to/train-first-agent/#span "Permanent link") A span represents a single unit of work or an operation within a **rollout**. Spans are the building blocks of a trace. They have a start and end time and contain details about the specific operation, like an LLM call, a tool execution, or a reward calculation. For a more precise definition, see the [OpenTelemetry documentation](https://opentelemetry.io/docs/concepts/signals/traces/) . Analogy: Span If the rollout is "baking a cake," a span could be "preheating the oven," "mixing flour and sugar," or "adding frosting." Each is a distinct step or unit of work. The picture below from [ADK](https://google.github.io/adk-docs/observability/cloud-trace/) shows a typical rollout, where each rectangle in the waterfall visualizes a span. As can be seen in the visualization, spans can be sequential, parallel or nested among each other. In other frameworks, the terminology might be slightly different. Agent-lightning follows the terminologies used by OpenTelemetry to avoid confusion. ![AgentOps Waterfall Visualization](https://microsoft.github.io/agent-lightning/latest/assets/agentops-waterfall-visualization.jpg) ### Prompt Template[¶](https://microsoft.github.io/agent-lightning/latest/how-to/train-first-agent/#prompt-template "Permanent link") A prompt template is a reusable instruction for the agent, often containing placeholders that can be filled in with specific details from a task. It is a key **"resource"** that the algorithm learns and improves over time. Analogy: Resource (Prompt Template) If the task is the recipe request, the prompt template is the master recipe card that the chef follows. The algorithm's job is to edit this recipe card to make the instructions clearer and the final dish better. The Training Loop: How the Magic Happens[¶](https://microsoft.github.io/agent-lightning/latest/how-to/train-first-agent/#the-training-loop-how-the-magic-happens "Permanent link") ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Training in Agent-lightning revolves around a clear, managed loop, orchestrated by the **Trainer**. The diagram below illustrates this core interaction: ![Loop of Tasks and Spans](https://microsoft.github.io/agent-lightning/latest/assets/tasks-spans-loop.svg) **The Loop Explained:** * **Algorithm to Agent (via Trainer):** The **Algorithm** (the "brain") creates an improved **Prompt Template** and selects **Tasks**. The Trainer then sends both to the Agent. * **Agent to Algorithm (via Trainer):** For each task it receives, the Agent uses the provided prompt template to perform a Rollout, executing its logic and potentially using tools. During this rollout, the runner that runs the agent captures Spans that detail every step. The agent also calculates a Reward for its performance on the task. These spans and rewards are then sent back to the Algorithm via the Trainer. * **Algorithm Learning:** The Algorithm then analyzes these spans and rewards to learn how to improve the agent's behavior, for example, by generating a better prompt. This improved prompt is then used in the next iteration of tasks. This cycle continues, allowing the agent to continuously learn and get better at solving tasks. Note In the next tutorial, we will see that the "via Trainer" here is not accurate. It's actually via the runner and store. ### The Algorithm[¶](https://microsoft.github.io/agent-lightning/latest/how-to/train-first-agent/#the-algorithm "Permanent link") The algorithm is the smart part of the system that drives the improvement. In this tutorial, we use [**APO**](https://microsoft.github.io/agent-lightning/latest/algorithm-zoo/apo/#agentlightning.algorithm.apo.APO " APO") (Automatic Prompt Optimization). It works in a few steps: 1. **Evaluate:** The algorithm first asks for rollouts to be run using the current prompt template to see how well it performs. 2. **Critique:** It then looks at the detailed spans from those rollouts. Using a powerful LLM (`gpt-5-mini`), it generates a "textual gradient", which is a natural language critique of the prompt. For example: "The prompt is ambiguous about how to handle tie-breakers for equally good rooms." 3. **Rewrite:** Finally, it gives the critique and the original prompt to another LLM (`gpt-4.1-mini`) and asks it to apply the edits, generating a new, improved prompt template. This cycle repeats, with each round producing a slightly better prompt. To use it, you simply initialize the APO class with your desired hyperparameters. `[](https://microsoft.github.io/agent-lightning/latest/how-to/train-first-agent/#__codelineno-2-1) # In the main training script: run_apo.py [](https://microsoft.github.io/agent-lightning/latest/how-to/train-first-agent/#__codelineno-2-2) from openai import AsyncOpenAI [](https://microsoft.github.io/agent-lightning/latest/how-to/train-first-agent/#__codelineno-2-3) [](https://microsoft.github.io/agent-lightning/latest/how-to/train-first-agent/#__codelineno-2-4) openai = AsyncOpenAI() [](https://microsoft.github.io/agent-lightning/latest/how-to/train-first-agent/#__codelineno-2-5) algo = agl.APO(openai)` Tip Make sure you have `OPENAI_API_KEY` set in your environment variables. ### The Trainer[¶](https://microsoft.github.io/agent-lightning/latest/how-to/train-first-agent/#the-trainer "Permanent link") The Trainer is the central component you'll interact with. It connects everything and manages the entire workflow by running the loop described above. You configure the Trainer, providing the algorithm, the number of parallel runners, and the initial prompt. A single call to [`trainer.fit()`](https://microsoft.github.io/agent-lightning/latest/reference/trainer/#agentlightning.Trainer.fit " fit(agent, train_dataset=None, *, val_dataset=None)") kicks off the entire process! ``[](https://microsoft.github.io/agent-lightning/latest/how-to/train-first-agent/#__codelineno-3-1) # 1. Configure the Trainer with the algorithm and initial prompt [](https://microsoft.github.io/agent-lightning/latest/how-to/train-first-agent/#__codelineno-3-2) trainer = agl.Trainer( [](https://microsoft.github.io/agent-lightning/latest/how-to/train-first-agent/#__codelineno-3-3) algorithm=algo, [](https://microsoft.github.io/agent-lightning/latest/how-to/train-first-agent/#__codelineno-3-4) n_runners=8, # Run 8 agents in parallel to try out the prompts [](https://microsoft.github.io/agent-lightning/latest/how-to/train-first-agent/#__codelineno-3-5) initial_resources={ [](https://microsoft.github.io/agent-lightning/latest/how-to/train-first-agent/#__codelineno-3-6) # The initial prompt template to be tuned [](https://microsoft.github.io/agent-lightning/latest/how-to/train-first-agent/#__codelineno-3-7) "prompt_template": prompt_template_baseline() [](https://microsoft.github.io/agent-lightning/latest/how-to/train-first-agent/#__codelineno-3-8) }, [](https://microsoft.github.io/agent-lightning/latest/how-to/train-first-agent/#__codelineno-3-9) # This is used to convert the span data into a message format consumable by APO algorithm [](https://microsoft.github.io/agent-lightning/latest/how-to/train-first-agent/#__codelineno-3-10) adapter=agl.TraceToMessages(), [](https://microsoft.github.io/agent-lightning/latest/how-to/train-first-agent/#__codelineno-3-11) ) [](https://microsoft.github.io/agent-lightning/latest/how-to/train-first-agent/#__codelineno-3-12) [](https://microsoft.github.io/agent-lightning/latest/how-to/train-first-agent/#__codelineno-3-13) # 2. Load datasets: They can be list of task objects consumable by `room_selector`. [](https://microsoft.github.io/agent-lightning/latest/how-to/train-first-agent/#__codelineno-3-14) dataset_train, dataset_val = ... [](https://microsoft.github.io/agent-lightning/latest/how-to/train-first-agent/#__codelineno-3-15) [](https://microsoft.github.io/agent-lightning/latest/how-to/train-first-agent/#__codelineno-3-16) # 3. Start the training process! [](https://microsoft.github.io/agent-lightning/latest/how-to/train-first-agent/#__codelineno-3-17) trainer.fit( [](https://microsoft.github.io/agent-lightning/latest/how-to/train-first-agent/#__codelineno-3-18) agent=room_selector, [](https://microsoft.github.io/agent-lightning/latest/how-to/train-first-agent/#__codelineno-3-19) train_dataset=dataset_train, [](https://microsoft.github.io/agent-lightning/latest/how-to/train-first-agent/#__codelineno-3-20) val_dataset=dataset_val [](https://microsoft.github.io/agent-lightning/latest/how-to/train-first-agent/#__codelineno-3-21) )`` Tip [`TraceToMessages`](https://microsoft.github.io/agent-lightning/latest/reference/algorithm/#agentlightning.TraceToMessages " agentlightning.TraceToMessages") is a convenience adapter that converts spans into OpenAI chat messages. It requires `openai >= 1.100.0` to be installed. Training Results[¶](https://microsoft.github.io/agent-lightning/latest/how-to/train-first-agent/#training-results "Permanent link") ------------------------------------------------------------------------------------------------------------------------------------ The APO algorithm successfully improved the agent's performance. We ran the example with the following hyper-parameters: * `val_batch_size` = 10 * `gradient_batch_size` = 4 * `beam_width` = 2 * `branch_factor` = 2 * `beam_rounds` = 2 The validation accuracy on the 29 samples of datasets steadily increase from 0.569 (baseline) to **0.721** (after round 2). The tuning takes around 10 minutes with 8 runners. We ran twice, and the results are shown in the chart below. This demonstrates how Agent-lightning can efficiently and automatically enhance your agent's capabilities with just a few lines of code. Back to top --- # VERL - Agent-lightning [Skip to content](https://microsoft.github.io/agent-lightning/latest/algorithm-zoo/verl/#verl) ⚠️ You are viewing development documentation. For stable documentation, please visit [https://microsoft.github.io/agent-lightning/stable/](https://microsoft.github.io/agent-lightning/stable/) VERL[¶](https://microsoft.github.io/agent-lightning/latest/algorithm-zoo/verl/#verl "Permanent link") ====================================================================================================== Shortcut You can use the shortcut `agl.VERL(...)` to create a VERL instance. `[](https://microsoft.github.io/agent-lightning/latest/algorithm-zoo/verl/#__codelineno-0-1) import agentlightning as agl [](https://microsoft.github.io/agent-lightning/latest/algorithm-zoo/verl/#__codelineno-0-2) [](https://microsoft.github.io/agent-lightning/latest/algorithm-zoo/verl/#__codelineno-0-3) agl.VERL(...)` Installation[¶](https://microsoft.github.io/agent-lightning/latest/algorithm-zoo/verl/#installation "Permanent link") ---------------------------------------------------------------------------------------------------------------------- `[](https://microsoft.github.io/agent-lightning/latest/algorithm-zoo/verl/#__codelineno-1-1) pip install agentlightning[verl]` Warning To avoid various compatibility issues, follow the steps in the [installation guide](https://microsoft.github.io/agent-lightning/latest/tutorials/installation/) to set up VERL and its dependencies. Installing VERL directly with `pip install agentlightning[verl]` can cause issues unless you already have a compatible version of PyTorch installed. Notes for Readers [VERL](https://microsoft.github.io/agent-lightning/latest/algorithm-zoo/verl/#agentlightning.algorithm.verl.VERL " VERL") in this article refers to a wrapper, provided by Agent-lightning, of the [VERL framework](https://github.com/volcengine/verl) . It's a subclass of [agentlightning.Algorithm](https://microsoft.github.io/agent-lightning/latest/reference/algorithm/#agentlightning.Algorithm " agentlightning.Algorithm") . To differentiate it from the VERL framework, all references to the VERL framework shall use the term "VERL framework", and all references to the Agent-lightning wrapper shall be highlighted with a link. Resources[¶](https://microsoft.github.io/agent-lightning/latest/algorithm-zoo/verl/#resources "Permanent link") ---------------------------------------------------------------------------------------------------------------- [VERL](https://microsoft.github.io/agent-lightning/latest/algorithm-zoo/verl/#agentlightning.algorithm.verl.VERL " VERL") expects no initial resources. The first LLM endpoint is directly deployed from the VERL configuration (`.actor_rollout_ref.model.path`). The resource key is always `main_llm`. [VERL](https://microsoft.github.io/agent-lightning/latest/algorithm-zoo/verl/#agentlightning.algorithm.verl.VERL " VERL") currently does not support optimizing multiple [LLM](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.LLM " agentlightning.LLM") s together. Note The resource type created by [VERL](https://microsoft.github.io/agent-lightning/latest/algorithm-zoo/verl/#agentlightning.algorithm.verl.VERL " VERL") is actually a [ProxyLLM](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.ProxyLLM " agentlightning.ProxyLLM") , a subclass of the [LLM](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.LLM " agentlightning.LLM") type. This object contains a **URL template** provided by [VERL](https://microsoft.github.io/agent-lightning/latest/algorithm-zoo/verl/#agentlightning.algorithm.verl.VERL " VERL") , with placeholders for rollout and attempt IDs. When a rollout begins on the agent side, the framework uses the current `rollout_id` and `attempt_id` to format this template, generating a final, unique endpoint URL. This URL points to [VERL](https://microsoft.github.io/agent-lightning/latest/algorithm-zoo/verl/#agentlightning.algorithm.verl.VERL " VERL") 's internal proxy, allowing it to intercept and log all traffic for that specific attempt, for tracing and load balancing purposes. For agents created with the `@rollout` decorator, this resolution of the template is handled automatically ("auto-stripped"). Class-based agents will need to manually resolve the [`ProxyLLM`](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.ProxyLLM " agentlightning.ProxyLLM") using the rollout context. `[](https://microsoft.github.io/agent-lightning/latest/algorithm-zoo/verl/#__codelineno-2-1) proxy_llm = resources["main_llm"] [](https://microsoft.github.io/agent-lightning/latest/algorithm-zoo/verl/#__codelineno-2-2) proxy_llm.get_base_url(rollout.rollout_id, rollout.attempt.attempt_id)` Customization[¶](https://microsoft.github.io/agent-lightning/latest/algorithm-zoo/verl/#customization "Permanent link") ------------------------------------------------------------------------------------------------------------------------ Internally, [VERL](https://microsoft.github.io/agent-lightning/latest/algorithm-zoo/verl/#agentlightning.algorithm.verl.VERL " VERL") decomposes each agent execution into prompt–response pairs via the [Adapter](https://microsoft.github.io/agent-lightning/latest/reference/algorithm/#agentlightning.Adapter " agentlightning.Adapter") and associates them with their corresponding reward signals as [Triplet](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.Triplet " agentlightning.Triplet") objects. The final scalar reward, derived from the last triplet in the trajectory, is propagated to all preceding triplets following the [identical assignment strategy](https://arxiv.org/abs/2508.03680) . This ensures that each triplet receives an identical reward signal and can be independently optimized as a valid RLHF trajectory within the VERL framework. At present, [VERL](https://microsoft.github.io/agent-lightning/latest/algorithm-zoo/verl/#agentlightning.algorithm.verl.VERL " VERL") does not expose fine-grained control over its reward propagation or credit assignment mechanisms. Users requiring customized reward shaping or trajectory decomposition are advised to clone and modify the [VERL](https://microsoft.github.io/agent-lightning/latest/algorithm-zoo/verl/#agentlightning.algorithm.verl.VERL " VERL") source implementation directly. Tutorials Using VERL[¶](https://microsoft.github.io/agent-lightning/latest/algorithm-zoo/verl/#tutorials-using-verl "Permanent link") -------------------------------------------------------------------------------------------------------------------------------------- * [Train SQL Agent with RL](https://microsoft.github.io/agent-lightning/latest/how-to/train-sql-agent/) - A practical example of training a SQL agent using VERL. References - Entrypoint[¶](https://microsoft.github.io/agent-lightning/latest/algorithm-zoo/verl/#references-entrypoint "Permanent link") ------------------------------------------------------------------------------------------------------------------------------------------ ### `agentlightning.algorithm.verl` [¶](https://microsoft.github.io/agent-lightning/latest/algorithm-zoo/verl/#agentlightning.algorithm.verl "Permanent link") #### `VERL` [¶](https://microsoft.github.io/agent-lightning/latest/algorithm-zoo/verl/#agentlightning.algorithm.verl.VERL "Permanent link") Bases: `[Algorithm](https://microsoft.github.io/agent-lightning/latest/reference/algorithm/#agentlightning.Algorithm " agentlightning.Algorithm (agentlightning.algorithm.base.Algorithm)") ` VERL-powered algorithm that delegates training to the VERL PPO runner. Warning Advanced customisation currently requires copying the VERL source and modifying it directly. Native hooks for overriding training behaviour will land in a future release. Parameters: * **`config`** (`dict[str, Any]`) – Dictionary mirroring the overrides passed to the VERL CLI. The overrides are merged with VERL's packaged defaults via Hydra before launching training. * **`trainer_cls`** (`Optional[Type[[AgentLightningTrainer](https://microsoft.github.io/agent-lightning/latest/algorithm-zoo/verl/#agentlightning.verl.AgentLightningTrainer " AgentLightningTrainer (agentlightning.verl.trainer.AgentLightningTrainer)") ]]`, default: `None` ) – Optional override for the trainer class. Experimental. * **`daemon_cls`** (`Optional[Type[[AgentModeDaemon](https://microsoft.github.io/agent-lightning/latest/algorithm-zoo/verl/#agentlightning.verl.AgentModeDaemon " AgentModeDaemon (agentlightning.verl.daemon.AgentModeDaemon)") ]]`, default: `None` ) – Optional override for the daemon class. Experimental. Trajectory aggregation (experimental) Trajectory-level aggregation merges an entire multi-turn rollout into a single, masked training sample so GPU time is spent once per trajectory rather than N times per turn. Enable it via: `[](https://microsoft.github.io/agent-lightning/latest/algorithm-zoo/verl/#__codelineno-0-1) config["agentlightning"]["trace_aggregator"] = { [](https://microsoft.github.io/agent-lightning/latest/algorithm-zoo/verl/#__codelineno-0-2) "level": "trajectory", [](https://microsoft.github.io/agent-lightning/latest/algorithm-zoo/verl/#__codelineno-0-3) "trajectory_max_prompt_length": 4096, [](https://microsoft.github.io/agent-lightning/latest/algorithm-zoo/verl/#__codelineno-0-4) "trajectory_max_response_length": 34384, [](https://microsoft.github.io/agent-lightning/latest/algorithm-zoo/verl/#__codelineno-0-5) }` Keep conversations structured (message lists rather than manual string concatenation) so prefix matching can stitch traces. `trajectory_max_prompt_length` should be set to the maximum length of the prompt for the first turn, and `trajectory_max_response_length` should be set to the maximum cumulative length of agent responses in the full trajectory. Toggle `debug=True` plus `mismatch_log_dir` when you need to inspect retokenization or chat-template mismatches. See [this blog post](https://agent-lightning.github.io/posts/trajectory_level_aggregation/) for more details. Examples: `[](https://microsoft.github.io/agent-lightning/latest/algorithm-zoo/verl/#__codelineno-0-1) from agentlightning.algorithm.verl import VERL [](https://microsoft.github.io/agent-lightning/latest/algorithm-zoo/verl/#__codelineno-0-2) [](https://microsoft.github.io/agent-lightning/latest/algorithm-zoo/verl/#__codelineno-0-3) algorithm = VERL( [](https://microsoft.github.io/agent-lightning/latest/algorithm-zoo/verl/#__codelineno-0-4) config={ [](https://microsoft.github.io/agent-lightning/latest/algorithm-zoo/verl/#__codelineno-0-5) "algorithm": { [](https://microsoft.github.io/agent-lightning/latest/algorithm-zoo/verl/#__codelineno-0-6) "adv_estimator": "grpo", [](https://microsoft.github.io/agent-lightning/latest/algorithm-zoo/verl/#__codelineno-0-7) "use_kl_in_reward": False, [](https://microsoft.github.io/agent-lightning/latest/algorithm-zoo/verl/#__codelineno-0-8) }, [](https://microsoft.github.io/agent-lightning/latest/algorithm-zoo/verl/#__codelineno-0-9) "data": { [](https://microsoft.github.io/agent-lightning/latest/algorithm-zoo/verl/#__codelineno-0-10) "train_batch_size": 32, [](https://microsoft.github.io/agent-lightning/latest/algorithm-zoo/verl/#__codelineno-0-11) "max_prompt_length": 4096, [](https://microsoft.github.io/agent-lightning/latest/algorithm-zoo/verl/#__codelineno-0-12) "max_response_length": 2048, [](https://microsoft.github.io/agent-lightning/latest/algorithm-zoo/verl/#__codelineno-0-13) }, [](https://microsoft.github.io/agent-lightning/latest/algorithm-zoo/verl/#__codelineno-0-14) "actor_rollout_ref": { [](https://microsoft.github.io/agent-lightning/latest/algorithm-zoo/verl/#__codelineno-0-15) "rollout": { [](https://microsoft.github.io/agent-lightning/latest/algorithm-zoo/verl/#__codelineno-0-16) "tensor_model_parallel_size": 1, [](https://microsoft.github.io/agent-lightning/latest/algorithm-zoo/verl/#__codelineno-0-17) "n": 4, [](https://microsoft.github.io/agent-lightning/latest/algorithm-zoo/verl/#__codelineno-0-18) "log_prob_micro_batch_size_per_gpu": 4, [](https://microsoft.github.io/agent-lightning/latest/algorithm-zoo/verl/#__codelineno-0-19) "multi_turn": {"format": "hermes"}, [](https://microsoft.github.io/agent-lightning/latest/algorithm-zoo/verl/#__codelineno-0-20) "name": "vllm", [](https://microsoft.github.io/agent-lightning/latest/algorithm-zoo/verl/#__codelineno-0-21) "gpu_memory_utilization": 0.6, [](https://microsoft.github.io/agent-lightning/latest/algorithm-zoo/verl/#__codelineno-0-22) }, [](https://microsoft.github.io/agent-lightning/latest/algorithm-zoo/verl/#__codelineno-0-23) "actor": { [](https://microsoft.github.io/agent-lightning/latest/algorithm-zoo/verl/#__codelineno-0-24) "ppo_mini_batch_size": 32, [](https://microsoft.github.io/agent-lightning/latest/algorithm-zoo/verl/#__codelineno-0-25) "ppo_micro_batch_size_per_gpu": 4, [](https://microsoft.github.io/agent-lightning/latest/algorithm-zoo/verl/#__codelineno-0-26) "optim": {"lr": 1e-6}, [](https://microsoft.github.io/agent-lightning/latest/algorithm-zoo/verl/#__codelineno-0-27) "use_kl_loss": False, [](https://microsoft.github.io/agent-lightning/latest/algorithm-zoo/verl/#__codelineno-0-28) "kl_loss_coef": 0.0, [](https://microsoft.github.io/agent-lightning/latest/algorithm-zoo/verl/#__codelineno-0-29) "entropy_coeff": 0, [](https://microsoft.github.io/agent-lightning/latest/algorithm-zoo/verl/#__codelineno-0-30) "clip_ratio_low": 0.2, [](https://microsoft.github.io/agent-lightning/latest/algorithm-zoo/verl/#__codelineno-0-31) "clip_ratio_high": 0.3, [](https://microsoft.github.io/agent-lightning/latest/algorithm-zoo/verl/#__codelineno-0-32) "fsdp_config": { [](https://microsoft.github.io/agent-lightning/latest/algorithm-zoo/verl/#__codelineno-0-33) "param_offload": True, [](https://microsoft.github.io/agent-lightning/latest/algorithm-zoo/verl/#__codelineno-0-34) "optimizer_offload": True, [](https://microsoft.github.io/agent-lightning/latest/algorithm-zoo/verl/#__codelineno-0-35) }, [](https://microsoft.github.io/agent-lightning/latest/algorithm-zoo/verl/#__codelineno-0-36) }, [](https://microsoft.github.io/agent-lightning/latest/algorithm-zoo/verl/#__codelineno-0-37) "ref": { [](https://microsoft.github.io/agent-lightning/latest/algorithm-zoo/verl/#__codelineno-0-38) "log_prob_micro_batch_size_per_gpu": 8, [](https://microsoft.github.io/agent-lightning/latest/algorithm-zoo/verl/#__codelineno-0-39) "fsdp_config": {"param_offload": True}, [](https://microsoft.github.io/agent-lightning/latest/algorithm-zoo/verl/#__codelineno-0-40) }, [](https://microsoft.github.io/agent-lightning/latest/algorithm-zoo/verl/#__codelineno-0-41) "model": { [](https://microsoft.github.io/agent-lightning/latest/algorithm-zoo/verl/#__codelineno-0-42) "path": "Qwen/Qwen2.5-1.5B-Instruct", [](https://microsoft.github.io/agent-lightning/latest/algorithm-zoo/verl/#__codelineno-0-43) "use_remove_padding": True, [](https://microsoft.github.io/agent-lightning/latest/algorithm-zoo/verl/#__codelineno-0-44) "enable_gradient_checkpointing": True, [](https://microsoft.github.io/agent-lightning/latest/algorithm-zoo/verl/#__codelineno-0-45) }, [](https://microsoft.github.io/agent-lightning/latest/algorithm-zoo/verl/#__codelineno-0-46) }, [](https://microsoft.github.io/agent-lightning/latest/algorithm-zoo/verl/#__codelineno-0-47) "trainer": { [](https://microsoft.github.io/agent-lightning/latest/algorithm-zoo/verl/#__codelineno-0-48) "n_gpus_per_node": 1, [](https://microsoft.github.io/agent-lightning/latest/algorithm-zoo/verl/#__codelineno-0-49) "val_before_train": True, [](https://microsoft.github.io/agent-lightning/latest/algorithm-zoo/verl/#__codelineno-0-50) "critic_warmup": 0, [](https://microsoft.github.io/agent-lightning/latest/algorithm-zoo/verl/#__codelineno-0-51) "logger": ["console", "wandb"], [](https://microsoft.github.io/agent-lightning/latest/algorithm-zoo/verl/#__codelineno-0-52) "project_name": "AgentLightning", [](https://microsoft.github.io/agent-lightning/latest/algorithm-zoo/verl/#__codelineno-0-53) "experiment_name": "calc_x", [](https://microsoft.github.io/agent-lightning/latest/algorithm-zoo/verl/#__codelineno-0-54) "nnodes": 1, [](https://microsoft.github.io/agent-lightning/latest/algorithm-zoo/verl/#__codelineno-0-55) "save_freq": 64, [](https://microsoft.github.io/agent-lightning/latest/algorithm-zoo/verl/#__codelineno-0-56) "test_freq": 32, [](https://microsoft.github.io/agent-lightning/latest/algorithm-zoo/verl/#__codelineno-0-57) "total_epochs": 2, [](https://microsoft.github.io/agent-lightning/latest/algorithm-zoo/verl/#__codelineno-0-58) }, [](https://microsoft.github.io/agent-lightning/latest/algorithm-zoo/verl/#__codelineno-0-59) } [](https://microsoft.github.io/agent-lightning/latest/algorithm-zoo/verl/#__codelineno-0-60) ) [](https://microsoft.github.io/agent-lightning/latest/algorithm-zoo/verl/#__codelineno-0-61) trainer.fit(algorithm, train_dataset=my_train_dataset)` ##### `get_client()` [¶](https://microsoft.github.io/agent-lightning/latest/algorithm-zoo/verl/#agentlightning.algorithm.verl.VERL.get_client "Permanent link") Create a client bound to the VERL-managed Agent Lightning server. Deprecated Since v0.2. ##### `run(train_dataset=None, val_dataset=None)` [¶](https://microsoft.github.io/agent-lightning/latest/algorithm-zoo/verl/#agentlightning.algorithm.verl.VERL.run "Permanent link") Launch the VERL PPO entrypoint with the configured runtime context. Parameters: * **`train_dataset`** (`Optional[[Dataset](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.Dataset " agentlightning.Dataset (agentlightning.types.Dataset)") [Any]]`, default: `None` ) – Optional dataset forwarded to VERL for training. * **`val_dataset`** (`Optional[[Dataset](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.Dataset " agentlightning.Dataset (agentlightning.types.Dataset)") [Any]]`, default: `None` ) – Optional dataset forwarded to VERL for evaluation. Raises: * `ValueError` – If required dependencies such as the store, LLM proxy, or adapter have been garbage-collected when using the V1 execution mode. References - Implementation[¶](https://microsoft.github.io/agent-lightning/latest/algorithm-zoo/verl/#references-implementation "Permanent link") -------------------------------------------------------------------------------------------------------------------------------------------------- ### `agentlightning.verl` [¶](https://microsoft.github.io/agent-lightning/latest/algorithm-zoo/verl/#agentlightning.verl "Permanent link") This package contains a _hacky_ integration of VERL with Agent Lightning. #### `AgentLightningTrainer` [¶](https://microsoft.github.io/agent-lightning/latest/algorithm-zoo/verl/#agentlightning.verl.AgentLightningTrainer "Permanent link") Bases: `RayPPOTrainer` Specialized PPO trainer for agent-based reinforcement learning. This trainer is designed specifically for scenarios where the model interacts with external environments, tools, or APIs through an AgentLightningServer. It simplifies the training loop by removing the complex conditional logic present in the original RayPPOTrainer and focusing on the agent mode workflow. Key differences from RayPPOTrainer: 1. Uses AgentModeDaemon for server communication 2. Simplified data flow without pop/union operations 3. Direct batch processing through agent daemon 4. Streamlined validation using agent\_mode validation #### `AgentModeDaemon` [¶](https://microsoft.github.io/agent-lightning/latest/algorithm-zoo/verl/#agentlightning.verl.AgentModeDaemon "Permanent link") AgentModeDaemon using the AgentLightningServer SDK. This class manages the server lifecycle, task queueing, and results retrieval, while also running a proxy server for LLM requests. It maintains the original interface for compatibility with the RayPPOTrainer. ##### `clear_data_and_server()` [¶](https://microsoft.github.io/agent-lightning/latest/algorithm-zoo/verl/#agentlightning.verl.AgentModeDaemon.clear_data_and_server "Permanent link") Resets the internal state of the daemon for the next run. ##### `get_test_metrics()` [¶](https://microsoft.github.io/agent-lightning/latest/algorithm-zoo/verl/#agentlightning.verl.AgentModeDaemon.get_test_metrics "Permanent link") Calculates and returns metrics for a validation run. ##### `get_train_data_batch(max_prompt_length, max_response_length, device, global_steps)` [¶](https://microsoft.github.io/agent-lightning/latest/algorithm-zoo/verl/#agentlightning.verl.AgentModeDaemon.get_train_data_batch "Permanent link") Processes completed rollouts to generate a training data batch. This function reconstructs the logic from the original AgentModeDaemon, using data retrieved from the new server architecture. It handles padding, truncation, and tensor creation for the PPO training loop. ##### `run_until_all_finished(verbose=True)` [¶](https://microsoft.github.io/agent-lightning/latest/algorithm-zoo/verl/#agentlightning.verl.AgentModeDaemon.run_until_all_finished "Permanent link") Synchronously waits for all queued tasks to be completed and reported. ##### `set_up_data_and_server(data, server_addresses, is_train=True)` [¶](https://microsoft.github.io/agent-lightning/latest/algorithm-zoo/verl/#agentlightning.verl.AgentModeDaemon.set_up_data_and_server "Permanent link") Synchronous wrapper for setting up data and server resources. ##### `start()` [¶](https://microsoft.github.io/agent-lightning/latest/algorithm-zoo/verl/#agentlightning.verl.AgentModeDaemon.start "Permanent link") Starts the main AgentLightningServer and the proxy server. #### `get_left_padded_ids_and_attention_mask(ids, max_length, pad_token_id)` [¶](https://microsoft.github.io/agent-lightning/latest/algorithm-zoo/verl/#agentlightning.verl.get_left_padded_ids_and_attention_mask "Permanent link") Left-pad (or truncate) a sequence of token IDs to a fixed length, and build the corresponding attention mask. Parameters: * **`ids`** (`List[int]`) – the original list of token IDs. * **`max_length`** (`int`) – desired total length after padding/truncation. * **`pad_token_id`** (`int`) – ID to use for padding. Returns: * **`padded_ids`** ( `any` ) – list of length == max\_length. * **`attention_mask`** ( `any` ) – list of same length: 1 for non-pad tokens, 0 for pads. #### `get_right_padded_ids_and_attention_mask(ids, max_length, pad_token_id)` [¶](https://microsoft.github.io/agent-lightning/latest/algorithm-zoo/verl/#agentlightning.verl.get_right_padded_ids_and_attention_mask "Permanent link") Right-pad (or truncate) a sequence of token IDs to a fixed length, and build the corresponding attention mask. Parameters: * **`ids`** (`List[int]`) – the original list of token IDs. * **`max_length`** (`int`) – desired total length after padding/truncation. * **`pad_token_id`** (`int`) – ID to use for padding. Returns: * **`padded_ids`** ( `any` ) – list of length == max\_length. * **`attention_mask`** ( `any` ) – list of same length: 1 for non-pad tokens, 0 for pads. Back to top --- # Contributing Guide - Agent-lightning [Skip to content](https://microsoft.github.io/agent-lightning/latest/community/contributing/#contributing-guide) ⚠️ You are viewing development documentation. For stable documentation, please visit [https://microsoft.github.io/agent-lightning/stable/](https://microsoft.github.io/agent-lightning/stable/) Contributing Guide[¶](https://microsoft.github.io/agent-lightning/latest/community/contributing/#contributing-guide "Permanent link") ====================================================================================================================================== Agent Lightning gets better every time someone files a clear bug, polishes docs, improves tests, or lands a new feature. This guide collects the expectations, checklists, and tips that help you go from “I have an idea” to “my pull request just merged.” Before You Start[¶](https://microsoft.github.io/agent-lightning/latest/community/contributing/#before-you-start "Permanent link") ---------------------------------------------------------------------------------------------------------------------------------- Agent-lightning is built by a small Microsoft Research team with limited reviewer hours and GPU budget. For any sizeable change (new algorithm, example, or API surface) please first discuss scope with us in [Discord](https://discord.gg/RYk7CdvDR7) . Early alignment keeps your effort from being blocked late in the process. Where You Can Help[¶](https://microsoft.github.io/agent-lightning/latest/community/contributing/#where-you-can-help "Permanent link") -------------------------------------------------------------------------------------------------------------------------------------- Pick a lane, or combine several. Just keep the discussion-first principle in mind for anything non-trivial. ### Documentation Improvements[¶](https://microsoft.github.io/agent-lightning/latest/community/contributing/#documentation-improvements "Permanent link") Documentation improvements are the easiest way to get started. You can find more about how to write good documentations and organize documentations in the following sections. Here are some general contribution points we can think of: * Tighten language, fix typos, clarify confusing sections, or add missing links. Fresh eyes catch docs gaps best. * Organize content using the directories listed below so readers can actually find it. * Avoid duplicate prose, unrelated “how-to” guides, or translations (we cannot maintain them today). Changes that are usually rejected * Copy/pasting existing docs with shallow edits. * Adding a `how-to` guide that is not tied to a new example. * Adding doc translations to other languages (no capacity to review/maintain yet). ### Bug Fixes[¶](https://microsoft.github.io/agent-lightning/latest/community/contributing/#bug-fixes "Permanent link") Bug fixes are the fastest way to get familiar with the codebase. To get started, you can: * Browse the ["help wanted"](https://github.com/microsoft/agent-lightning/labels/help%20wanted) and ["bug"](https://github.com/microsoft/agent-lightning/labels/bug) labels; drop a comment before you start so we can mark it as taken. * For fresh bugs, open an issue with reproduction steps, logs, and expected behavior before submitting a fix. * Keep each pull request focused, ideally avoiding breaking API changes. Larger refactors should be discussed via RFC or maintainer sync. ### New Examples[¶](https://microsoft.github.io/agent-lightning/latest/community/contributing/#new-examples "Permanent link") Examples must be curated so that we can maintain them. We generally merge only those that meet at least one (ideally several) of these criteria: * Demonstrates an agent framework or workflow that is materially different from what already exists. ([LangChain](https://www.langchain.com/) vs. [LlamaIndex](https://www.llamaindex.ai/) is not different enough; [LangChain](https://www.langchain.com/) vs. [n8n](https://n8n.io/) or [Vercel AI SDK](https://ai-sdk.dev/) is, because they either have different orchestration paradigms or differ in programming languages.) * Shows measurable performance gains on a **real-world** problem with a **real-world** dataset, such as tuning a search agent with Google Search API or improving a coding agent’s (e.g., Claude Code) SWE-Bench score. * Integrates a new algorithm, training backend, or serving stack (see “New Algorithms” below). * Validates scenarios that are rarely tested, such as multi-modality agents or long-lived memory/workflow agents. Bonus points for examples that: * Ship CI or self-test coverage so we know they still work as the core evolves. **Otherwise, we would have to mark the example as unmaintained because we won't be able to test the examples manually before each release.** * Include a [`docs/how-to/`](https://github.com/microsoft/agent-lightning/tree/c746af2f76bebcae56007a59c223cdace411c89b/docs/how-to/) guide (or a detailed README if no how-to exists) without duplicating content in multiple places. * Favor simple, dependency-light code over heavy abstractions. * Ship a README that documents smoke-test instructions and includes an "Included Files" section summarizing every file and its role; keep the runnable module self-contained with a module-level docstring explaining CLI usage, plus targeted docstrings or inline comments for educational functions/classes. Please discuss first Examples tend to be the most time-consuming contributions for both you and reviewers. Sync with us on Discord or through an issue before diving into a new one. ### Fresh Implementations of Core Modules[¶](https://microsoft.github.io/agent-lightning/latest/community/contributing/#fresh-implementations-of-core-modules "Permanent link") If you are looking to extend [`Runner`](https://microsoft.github.io/agent-lightning/latest/reference/runner/#agentlightning.Runner " agentlightning.Runner") , [`Tracer`](https://microsoft.github.io/agent-lightning/latest/reference/runner/#agentlightning.Tracer " agentlightning.Tracer") , [`Adapter`](https://microsoft.github.io/agent-lightning/latest/reference/algorithm/#agentlightning.Adapter " agentlightning.Adapter") , [`LightningStore`](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.LightningStore " agentlightning.LightningStore") , or another core interface, here are the steps: 1. File an issue or proposal first. 2. Explain which interface you are extending, why existing implementations are insufficient, and how you intend to test compatibility with the rest of the stack (unit tests, documentation updates, example refreshes, etc.). 3. Any API changes must be reviewed up front. DO NOT begin coding large changes before the discussion lands! ### New Algorithms[¶](https://microsoft.github.io/agent-lightning/latest/community/contributing/#new-algorithms "Permanent link") If you are integrating a new training/serving backend, check whether it already lives in the [Algorithm Zoo](https://microsoft.github.io/agent-lightning/latest/algorithm-zoo/) or is covered in the [Examples Catalog](https://microsoft.github.io/agent-lightning/latest/how-to/examples-catalog/) . We especially welcome: * Currently unsupported or under-tested algorithms such as Supervised Fine-tuning (SFT), Direct Policy Optimization (DPO), or Monte Carlo Tree Search (MCTS). * Tuning [Resource](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.Resource " agentlightning.Resource") s that are not supported yet, such as workflows or memory. * Expansions of supported stacks, e.g., adding multi-modality to APO or multi-agent prompt tuning. * Reinforcement-learning integrations beyond our current stack of [VERL](https://github.com/volcengine/verl) , [vLLM](https://vllm.ai/) , [Azure OpenAI](https://azure.microsoft.com/en-us/products/ai-foundry/models/openai) , and [Tinker](https://tinker-docs.thinkingmachines.ai/) . Contributions using [SGLang](https://github.com/sgl-project/sglang) , [TRL](https://github.com/huggingface/trl) , [SkyRL](https://github.com/NovaSky-AI/SkyRL) , [RLinf](https://github.com/RLinf/RLinf) , [litgpt](https://github.com/Lightning-AI/litgpt) , or similar are welcome. Most brand-new algorithms ultimately land as “new examples,” so read that section too. Post an issue or design doc to scope the work, reuse existing utilities, and avoid duplicating efforts. Mature, battle-tested examples graduate into the [Algorithm Zoo](https://microsoft.github.io/agent-lightning/latest/algorithm-zoo/) . ### Ecosystem Projects[¶](https://microsoft.github.io/agent-lightning/latest/community/contributing/#ecosystem-projects "Permanent link") Have a project that builds on Agent-lightning but does not belong in the main repo? Fork it or depend on it externally, then let us know. We can showcase notable projects in [Community Projects](https://microsoft.github.io/agent-lightning/latest/) and the main [README](https://github.com/microsoft/agent-lightning/tree/c746af2f76bebcae56007a59c223cdace411c89b/README.md) . ### Agent-lightning Contrib[¶](https://microsoft.github.io/agent-lightning/latest/community/contributing/#agent-lightning-contrib "Permanent link") [`contrib/`](https://github.com/microsoft/agent-lightning/tree/c746af2f76bebcae56007a59c223cdace411c89b/contrib) is where work-in-progress or third-party integrations, and curated recipes live before they are hardened enough for the core runtime tree. Think of it as an incubator: additions should remain easy to consume, clearly owned, and scoped so downstream users can vendor them with minimal risk. The following types of contributions are welcome in the contrib area: * **Recipes** that assemble multiple Agent Lightning components for a narrow task (`contrib/recipes//`). Each recipe must be self-contained, include running instructions and result reports. * **Runtime extensions** that would bloat the primary `agentlightning/` namespace (`contrib/agentlightning/contrib//`). These should mirror the published wheel layout so that `import agentlightning.contrib.` works out of the box. * **Supporting scripts and assets** (`contrib/scripts/`) that automate dataset downloads, environment preparation, or benchmarks required by contrib modules. If you are unsure where a contribution should live, start a thread in Discord or open an issue before writing code. The [contrib README](https://github.com/microsoft/agent-lightning/tree/c746af2f76bebcae56007a59c223cdace411c89b/contrib/README.md) also lists the directory expectations. A quick checklist for contributions to be accepted: 1. **Document everything.** Include configuration steps, environment variables, and sample commands so contributors can reproduce the results without guesswork. Pin to a specific version of Agent-lightning and other dependencies to avoid unexpected changes if you don't want to update the recipe frequently. 2. **Keep quality predictable.** Match the repo’s style guide, apply exhaustive type hints, and run `uv run --no-sync pyright` plus targeted `pytest` suites for any Python module you touch. 3. **Ship reproducibility artifacts.** Store only scripts or instructions for downloading datasets, weights, or binaries. Never upload large artifacts or credentials directly. 4. **Update ownership.** Add `CODEOWNERS` entries when new directories appear so maintainers know who can review follow-up fixes. Contrib entries do not need the same maturity level as core code, but they must still meet the baseline above. Submissions that lack documentation, hide ownership, or depend on untracked assets are typically rejected until those gaps are resolved. ### Other Contribution Ideas[¶](https://microsoft.github.io/agent-lightning/latest/community/contributing/#other-contribution-ideas "Permanent link") * **Tests.** Add or improve cases in [`tests/`](https://github.com/microsoft/agent-lightning/tree/c746af2f76bebcae56007a59c223cdace411c89b/tests) (unit, integration, or end-to-end). * **Benchmarks.** Expand [`tests/benchmark`](https://github.com/microsoft/agent-lightning/tree/c746af2f76bebcae56007a59c223cdace411c89b/tests/benchmark) to stress large-scale training or rollouts. * **Issue triage.** Reproduce bugs, confirm whether they reproduce on `main`, or suggest short-term mitigations so maintainers can prioritize. Contribution Workflow[¶](https://microsoft.github.io/agent-lightning/latest/community/contributing/#contribution-workflow "Permanent link") -------------------------------------------------------------------------------------------------------------------------------------------- The steps below keep changes reviewable and CI-friendly. Follow them in order; rerun the relevant pieces if you revisit a branch later. ### 1\. Prepare Your Environment[¶](https://microsoft.github.io/agent-lightning/latest/community/contributing/#1-prepare-your-environment "Permanent link") Minimum tooling: * **Python** 3.10+ (3.12 recommended). * **uv** for dependency and virtual-environment management. Install it using the [official uv docs](https://docs.astral.sh/uv/getting-started/installation/) . * **Git** configured with your GitHub credentials. Clone your fork and point `upstream` at the official repo: `[](https://microsoft.github.io/agent-lightning/latest/community/contributing/#__codelineno-0-1) git clone git@github.com:/agent-lightning.git [](https://microsoft.github.io/agent-lightning/latest/community/contributing/#__codelineno-0-2) cd agent-lightning [](https://microsoft.github.io/agent-lightning/latest/community/contributing/#__codelineno-0-3) git remote add upstream https://github.com/microsoft/agent-lightning.git` Install the default development stack: `[](https://microsoft.github.io/agent-lightning/latest/community/contributing/#__codelineno-1-1) uv sync --group dev` Need GPU extras or specific optional dependencies? Lock them in with one command: `[](https://microsoft.github.io/agent-lightning/latest/community/contributing/#__codelineno-2-1) uv sync --frozen \ [](https://microsoft.github.io/agent-lightning/latest/community/contributing/#__codelineno-2-2) --extra apo \ [](https://microsoft.github.io/agent-lightning/latest/community/contributing/#__codelineno-2-3) --extra verl \ [](https://microsoft.github.io/agent-lightning/latest/community/contributing/#__codelineno-2-4) --group dev \ [](https://microsoft.github.io/agent-lightning/latest/community/contributing/#__codelineno-2-5) --group torch-cpu \ [](https://microsoft.github.io/agent-lightning/latest/community/contributing/#__codelineno-2-6) --group torch-stable \ [](https://microsoft.github.io/agent-lightning/latest/community/contributing/#__codelineno-2-7) --group agents \ [](https://microsoft.github.io/agent-lightning/latest/community/contributing/#__codelineno-2-8) --no-default-groups` After `uv sync`, run commands via `uv run ...` (add `--no-sync` once the environment is locked) or activate `.venv/`. ### 2\. Install and Run Pre-commit[¶](https://microsoft.github.io/agent-lightning/latest/community/contributing/#2-install-and-run-pre-commit "Permanent link") Formatting and linting are enforced through [pre-commit](https://pre-commit.com/) . Install once, then run before each push: `[](https://microsoft.github.io/agent-lightning/latest/community/contributing/#__codelineno-3-1) uv run --no-sync pre-commit install [](https://microsoft.github.io/agent-lightning/latest/community/contributing/#__codelineno-3-2) uv run --no-sync pre-commit run --all-files --show-diff-on-failure --color=always` Once installed, the hooks run automatically on every `git commit`. Running the pre-commit hooks locally keeps CI green and diffs manageable. ### 3\. Branch from Fresh `main` and Code[¶](https://microsoft.github.io/agent-lightning/latest/community/contributing/#3-branch-from-fresh-main-and-code "Permanent link") Start all work from the latest upstream state: `[](https://microsoft.github.io/agent-lightning/latest/community/contributing/#__codelineno-4-1) git fetch upstream [](https://microsoft.github.io/agent-lightning/latest/community/contributing/#__codelineno-4-2) git checkout main [](https://microsoft.github.io/agent-lightning/latest/community/contributing/#__codelineno-4-3) git merge upstream/main` Branch naming convention: * `feature/` for new features. * `fix/` for bug fixes. * `docs/` for documentation-only updates. * `chore/` for tooling or maintenance. Use lowercase with hyphens, e.g., `feature/async-runner-hooks`. Where should docs or examples live? Many new contributors get confused about what to put in the `docs/how-to/` directory and what to put in the `examples/` directory (particularly README files). Here is a quick reference you can refer to: | Location | Description | | --- | --- | | `docs/algorithm-zoo/` | Documentation for **built-in algorithms** shipped with Agent-lightning. | | `docs/how-to/` | Step-by-step **how-to guides**, usually tied to an example in `examples/`. | | `docs/tutorials/` | Conceptual walkthroughs for components or workflows. See [debugging](https://microsoft.github.io/agent-lightning/latest/tutorials/debug/)
or [parallelization](https://microsoft.github.io/agent-lightning/latest/tutorials/parallelize/)
for examples. | | `docs/deep-dive/` | Advanced explanations and in-depth concepts. | | `examples//README.md` | Example-specific README. If any related how-to if that exists, link to it avoid duplicating the same instructions twice; write only brief instructions on how to install and run the example. Otherwise, you can make the README more detailed and self-explanatory. | Remember to register new docs in [`mkdocs.yml`](https://github.com/microsoft/agent-lightning/tree/c746af2f76bebcae56007a59c223cdace411c89b/mkdocs.yml) , add examples to [examples/README](https://github.com/microsoft/agent-lightning/tree/c746af2f76bebcae56007a59c223cdace411c89b/examples/README.md) , and update the [Examples Catalog](https://microsoft.github.io/agent-lightning/latest/how-to/examples-catalog/) . Before you start coding, bring the shared coding conventions with you: * Target `requires-python >= 3.10`, four-space indentation, ~120-character lines (docstrings may run longer), and formatter-owned diffs (Black + isort with the `black` profile). * Use `snake_case` for modules, functions, and variables; `PascalCase` for classes and React components; lowercase hyphenation for CLI flags, branch names, and TypeScript filenames. * Maintain exhaustive type hints (pyright enforces them), write succinct Google-style docstrings (with `[][]` cross-references). * Prefer dataclasses or Pydantic models from `agentlightning.types`. * Log via `logging.getLogger(__name__)` with targeted DEBUG/INFO/WARNING/ERROR calls—especially for long multi-step functions or broad `try/except` blocks. ### 4\. Test and Validate[¶](https://microsoft.github.io/agent-lightning/latest/community/contributing/#4-test-and-validate "Permanent link") Most contributions require automated checks. Once `uv sync` locks dependencies, prefix commands with `uv run --no-sync ...` so they share the same environment as CI. **Full test suite** `[](https://microsoft.github.io/agent-lightning/latest/community/contributing/#__codelineno-5-1) uv run --no-sync pytest -v` **Targeted tests** `[](https://microsoft.github.io/agent-lightning/latest/community/contributing/#__codelineno-6-1) uv run --no-sync pytest tests/path/to/test_file.py -k test_name` **Optional/gated tests:** GPU-specific suites or API-dependent tests run automatically when the required hardware or environment variables (such as `OPENAI_API_KEY`) are present. **Static analysis:** `[](https://microsoft.github.io/agent-lightning/latest/community/contributing/#__codelineno-7-1) uv run --no-sync pyright` If you have touched code under `examples/`, you should run the example-specific smoke tests. Each directory includes a README with example-specific smoke tests—run those too. Build documentation when needed Keep API references under [docs/reference](https://github.com/microsoft/agent-lightning/tree/c746af2f76bebcae56007a59c223cdace411c89b/docs/reference/) up to date. Doc-only changes should still build cleanly: `[](https://microsoft.github.io/agent-lightning/latest/community/contributing/#__codelineno-8-1) uv run --no-sync mkdocs serve --strict # live reload [](https://microsoft.github.io/agent-lightning/latest/community/contributing/#__codelineno-8-2) uv run --no-sync mkdocs build --strict # CI-equivalent` `--strict` elevates warnings to errors so you catch issues before CI. Before opening a PR, double-check the basics: * Run `uv lock` if you changed dependencies. * Run `uv run --no-sync pre-commit run --all-files --show-diff-on-failure` (hooks installed via `pre-commit install` run automatically on `git commit`, but rerun them if you amended history). * Execute the relevant commands from the test list above. * Validate each affected example via its README instructions. ### 5\. Open a Pull Request[¶](https://microsoft.github.io/agent-lightning/latest/community/contributing/#5-open-a-pull-request "Permanent link") 1. Push your branch: `[](https://microsoft.github.io/agent-lightning/latest/community/contributing/#__codelineno-9-1) git push origin ` 2. Open a PR against `microsoft/agent-lightning:main`. 3. Fill out the template with a concise summary, the commands/tests you ran, and linked issues (use `Fixes #123` syntax to auto-close). 4. Include screenshots or logs if they clarify behavior. 5. Address review feedback promptly. Follow-up tweaks work best as focused commits; `git commit --fixup` is handy for reviewer-suggested edits. Thanks for contributing! every improvement strengthens the Agent Lightning community! Back to top --- # Agent - Agent-lightning [Skip to content](https://microsoft.github.io/agent-lightning/latest/reference/agent/#agent-developer-apis) ⚠️ You are viewing development documentation. For stable documentation, please visit [https://microsoft.github.io/agent-lightning/stable/](https://microsoft.github.io/agent-lightning/stable/) Agent Developer APIs[¶](https://microsoft.github.io/agent-lightning/latest/reference/agent/#agent-developer-apis "Permanent link") =================================================================================================================================== Agent Decorators[¶](https://microsoft.github.io/agent-lightning/latest/reference/agent/#agent-decorators "Permanent link") --------------------------------------------------------------------------------------------------------------------------- Tip These are convenient helpers for creating agents from functions. First-time users are recommended to use these decorators to create agents. ### `agentlightning.rollout(func)` [¶](https://microsoft.github.io/agent-lightning/latest/reference/agent/#agentlightning.rollout "Permanent link") Create a [`FunctionalLitAgent`](https://microsoft.github.io/agent-lightning/latest/reference/internal/#agentlightning.litagent.decorator.FunctionalLitAgent " agentlightning.litagent.decorator.FunctionalLitAgent") from an arbitrary rollout function. This function inspects the provided callable and creates the appropriate agent type based on its signature. It supports both LLM-based and prompt-template-based agents. The returned agent instance is callable, preserving the original function's behavior and type hints. See [`llm_rollout`](https://microsoft.github.io/agent-lightning/latest/reference/internal/#agentlightning.litagent.decorator.llm_rollout " agentlightning.litagent.decorator.llm_rollout(func=None, *, strip_proxy=True)") and [`prompt_rollout`](https://microsoft.github.io/agent-lightning/latest/reference/internal/#agentlightning.litagent.decorator.prompt_rollout " agentlightning.litagent.decorator.prompt_rollout(func=None)") for more details. Parameters: * **`func`** (`Union[LlmRolloutFunc[T], PromptRolloutFunc[T], Callable[..., Any]]`) – Callable that implements the rollout. Supported signatures: * `[async ](task, llm[, rollout])` for LLM-based agents * `[async ](task, prompt_template[, rollout])` for prompt-template-based agents The supported output types of `func` is same as the return type of [`rollout`](https://microsoft.github.io/agent-lightning/latest/reference/agent/#agentlightning.LitAgent.rollout " rollout(task, resources, rollout)") . Returns: * `[FunctionalLitAgent](https://microsoft.github.io/agent-lightning/latest/reference/internal/#agentlightning.litagent.decorator.FunctionalLitAgent " agentlightning.litagent.decorator.FunctionalLitAgent") [T]` – [`FunctionalLitAgent`](https://microsoft.github.io/agent-lightning/latest/reference/internal/#agentlightning.litagent.decorator.FunctionalLitAgent " agentlightning.litagent.decorator.FunctionalLitAgent") that * `[FunctionalLitAgent](https://microsoft.github.io/agent-lightning/latest/reference/internal/#agentlightning.litagent.decorator.FunctionalLitAgent " agentlightning.litagent.decorator.FunctionalLitAgent") [T]` – wraps the supplied function. Examples: `[](https://microsoft.github.io/agent-lightning/latest/reference/agent/#__codelineno-0-1) # LLM-based agent [](https://microsoft.github.io/agent-lightning/latest/reference/agent/#__codelineno-0-2) @rollout [](https://microsoft.github.io/agent-lightning/latest/reference/agent/#__codelineno-0-3) def my_llm_agent(task, llm): [](https://microsoft.github.io/agent-lightning/latest/reference/agent/#__codelineno-0-4) client = OpenAI(base_url=llm.endpoint) [](https://microsoft.github.io/agent-lightning/latest/reference/agent/#__codelineno-0-5) response = client.chat.completions.create( [](https://microsoft.github.io/agent-lightning/latest/reference/agent/#__codelineno-0-6) model=llm.model, [](https://microsoft.github.io/agent-lightning/latest/reference/agent/#__codelineno-0-7) messages=[{"role": "user", "content": task.input}], [](https://microsoft.github.io/agent-lightning/latest/reference/agent/#__codelineno-0-8) ) [](https://microsoft.github.io/agent-lightning/latest/reference/agent/#__codelineno-0-9) return response [](https://microsoft.github.io/agent-lightning/latest/reference/agent/#__codelineno-0-10) [](https://microsoft.github.io/agent-lightning/latest/reference/agent/#__codelineno-0-11) # Prompt-template-based agent [](https://microsoft.github.io/agent-lightning/latest/reference/agent/#__codelineno-0-12) @rollout [](https://microsoft.github.io/agent-lightning/latest/reference/agent/#__codelineno-0-13) def my_prompt_agent(task, prompt_template): [](https://microsoft.github.io/agent-lightning/latest/reference/agent/#__codelineno-0-14) messages = prompt_template.format(task=task.input) [](https://microsoft.github.io/agent-lightning/latest/reference/agent/#__codelineno-0-15) # ... perform rollout with the formatted prompt [](https://microsoft.github.io/agent-lightning/latest/reference/agent/#__codelineno-0-16) return response [](https://microsoft.github.io/agent-lightning/latest/reference/agent/#__codelineno-0-17) [](https://microsoft.github.io/agent-lightning/latest/reference/agent/#__codelineno-0-18) # Function is still callable with original behavior [](https://microsoft.github.io/agent-lightning/latest/reference/agent/#__codelineno-0-19) result = my_llm_agent(task, llm) [](https://microsoft.github.io/agent-lightning/latest/reference/agent/#__codelineno-0-20) [](https://microsoft.github.io/agent-lightning/latest/reference/agent/#__codelineno-0-21) # Agent methods are also available [](https://microsoft.github.io/agent-lightning/latest/reference/agent/#__codelineno-0-22) result = my_llm_agent.rollout(task, resources, rollout)` Raises: * `NotImplementedError` – If the function signature doesn't match any known patterns. Warning The following two decorators are implementations of [`agentlightning.rollout`](https://microsoft.github.io/agent-lightning/latest/reference/agent/#agentlightning.rollout " agentlightning.rollout(func)") . They are not recommended for new users. ### `agentlightning.llm_rollout(func=None, *, strip_proxy=True)` [¶](https://microsoft.github.io/agent-lightning/latest/reference/agent/#agentlightning.llm_rollout "Permanent link") `[](https://microsoft.github.io/agent-lightning/latest/reference/agent/#__codelineno-0-1) llm_rollout( [](https://microsoft.github.io/agent-lightning/latest/reference/agent/#__codelineno-0-2) func: LlmRolloutFunc[T], [](https://microsoft.github.io/agent-lightning/latest/reference/agent/#__codelineno-0-3) ) -> FunctionalLitAgent[T]` `[](https://microsoft.github.io/agent-lightning/latest/reference/agent/#__codelineno-0-1) llm_rollout( [](https://microsoft.github.io/agent-lightning/latest/reference/agent/#__codelineno-0-2) *, strip_proxy: bool = True [](https://microsoft.github.io/agent-lightning/latest/reference/agent/#__codelineno-0-3) ) -> Callable[[LlmRolloutFunc[T]], FunctionalLitAgent[T]]` Create a [`FunctionalLitAgent`](https://microsoft.github.io/agent-lightning/latest/reference/internal/#agentlightning.litagent.decorator.FunctionalLitAgent " agentlightning.litagent.decorator.FunctionalLitAgent") for LLM-based rollouts. Parameters: * **`func`** (`LlmRolloutFunc[T] | None`, default: `None` ) – Callable defining the agent's behaviour. Supported signatures include: * `(task, llm) -> result` * `(task, llm, rollout) -> result` * `async (task, llm) -> result` * `async (task, llm, rollout) -> result` * **`strip_proxy`** (`bool`, default: `True` ) – When `True`, convert proxy resources into concrete [`LLM`](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.LLM " agentlightning.LLM") instances before calling the function. Defaults to `True`. Returns: * `[FunctionalLitAgent](https://microsoft.github.io/agent-lightning/latest/reference/internal/#agentlightning.litagent.decorator.FunctionalLitAgent " agentlightning.litagent.decorator.FunctionalLitAgent") [T] | Callable[[LlmRolloutFunc[T]], [FunctionalLitAgent](https://microsoft.github.io/agent-lightning/latest/reference/internal/#agentlightning.litagent.decorator.FunctionalLitAgent " agentlightning.litagent.decorator.FunctionalLitAgent") [T]]` – [`FunctionalLitAgent`](https://microsoft.github.io/agent-lightning/latest/reference/internal/#agentlightning.litagent.decorator.FunctionalLitAgent " agentlightning.litagent.decorator.FunctionalLitAgent") that * `[FunctionalLitAgent](https://microsoft.github.io/agent-lightning/latest/reference/internal/#agentlightning.litagent.decorator.FunctionalLitAgent " agentlightning.litagent.decorator.FunctionalLitAgent") [T] | Callable[[LlmRolloutFunc[T]], [FunctionalLitAgent](https://microsoft.github.io/agent-lightning/latest/reference/internal/#agentlightning.litagent.decorator.FunctionalLitAgent " agentlightning.litagent.decorator.FunctionalLitAgent") [T]]` – wraps the supplied function. Examples: `[](https://microsoft.github.io/agent-lightning/latest/reference/agent/#__codelineno-0-1) @llm_rollout [](https://microsoft.github.io/agent-lightning/latest/reference/agent/#__codelineno-0-2) def my_agent(task, llm): [](https://microsoft.github.io/agent-lightning/latest/reference/agent/#__codelineno-0-3) return llm.endpoint [](https://microsoft.github.io/agent-lightning/latest/reference/agent/#__codelineno-0-4) [](https://microsoft.github.io/agent-lightning/latest/reference/agent/#__codelineno-0-5) @llm_rollout(strip_proxy=False) [](https://microsoft.github.io/agent-lightning/latest/reference/agent/#__codelineno-0-6) def my_agent_no_strip(task, llm): [](https://microsoft.github.io/agent-lightning/latest/reference/agent/#__codelineno-0-7) return llm.model [](https://microsoft.github.io/agent-lightning/latest/reference/agent/#__codelineno-0-8) [](https://microsoft.github.io/agent-lightning/latest/reference/agent/#__codelineno-0-9) result = my_agent(task, llm) [](https://microsoft.github.io/agent-lightning/latest/reference/agent/#__codelineno-0-10) result = my_agent.rollout(task, resources, rollout)` ### `agentlightning.prompt_rollout(func=None)` [¶](https://microsoft.github.io/agent-lightning/latest/reference/agent/#agentlightning.prompt_rollout "Permanent link") `[](https://microsoft.github.io/agent-lightning/latest/reference/agent/#__codelineno-0-1) prompt_rollout( [](https://microsoft.github.io/agent-lightning/latest/reference/agent/#__codelineno-0-2) func: PromptRolloutFunc[T], [](https://microsoft.github.io/agent-lightning/latest/reference/agent/#__codelineno-0-3) ) -> FunctionalLitAgent[T]` `[](https://microsoft.github.io/agent-lightning/latest/reference/agent/#__codelineno-0-1) prompt_rollout() -> ( [](https://microsoft.github.io/agent-lightning/latest/reference/agent/#__codelineno-0-2) Callable[[PromptRolloutFunc[T]], FunctionalLitAgent[T]] [](https://microsoft.github.io/agent-lightning/latest/reference/agent/#__codelineno-0-3) )` Create a [`FunctionalLitAgent`](https://microsoft.github.io/agent-lightning/latest/reference/internal/#agentlightning.litagent.decorator.FunctionalLitAgent " agentlightning.litagent.decorator.FunctionalLitAgent") for prompt-based rollouts. This decorator is designed for agents that work with tunable prompt templates. It enables a workflow where algorithms manage and optimize the prompt template, while agents consume the template to perform rollouts. This is particularly useful for prompt optimization scenarios. Parameters: * **`func`** (`PromptRolloutFunc[T] | None`, default: `None` ) – Callable defining the agent's behavior. Supported signatures include: * `(task, prompt_template) -> result` * `(task, prompt_template, rollout) -> result` * `async (task, prompt_template) -> result` * `async (task, prompt_template, rollout) -> result` Returns: * `[FunctionalLitAgent](https://microsoft.github.io/agent-lightning/latest/reference/internal/#agentlightning.litagent.decorator.FunctionalLitAgent " agentlightning.litagent.decorator.FunctionalLitAgent") [T] | Callable[[PromptRolloutFunc[T]], [FunctionalLitAgent](https://microsoft.github.io/agent-lightning/latest/reference/internal/#agentlightning.litagent.decorator.FunctionalLitAgent " agentlightning.litagent.decorator.FunctionalLitAgent") [T]]` – [`FunctionalLitAgent`](https://microsoft.github.io/agent-lightning/latest/reference/internal/#agentlightning.litagent.decorator.FunctionalLitAgent " agentlightning.litagent.decorator.FunctionalLitAgent") that * `[FunctionalLitAgent](https://microsoft.github.io/agent-lightning/latest/reference/internal/#agentlightning.litagent.decorator.FunctionalLitAgent " agentlightning.litagent.decorator.FunctionalLitAgent") [T] | Callable[[PromptRolloutFunc[T]], [FunctionalLitAgent](https://microsoft.github.io/agent-lightning/latest/reference/internal/#agentlightning.litagent.decorator.FunctionalLitAgent " agentlightning.litagent.decorator.FunctionalLitAgent") [T]]` – wraps the supplied function. Examples: `[](https://microsoft.github.io/agent-lightning/latest/reference/agent/#__codelineno-0-1) @prompt_rollout [](https://microsoft.github.io/agent-lightning/latest/reference/agent/#__codelineno-0-2) def my_agent(task, prompt_template): [](https://microsoft.github.io/agent-lightning/latest/reference/agent/#__codelineno-0-3) messages = prompt_template.format(task=task.input) [](https://microsoft.github.io/agent-lightning/latest/reference/agent/#__codelineno-0-4) return messages [](https://microsoft.github.io/agent-lightning/latest/reference/agent/#__codelineno-0-5) [](https://microsoft.github.io/agent-lightning/latest/reference/agent/#__codelineno-0-6) result = my_agent(task, prompt_template) [](https://microsoft.github.io/agent-lightning/latest/reference/agent/#__codelineno-0-7) result = my_agent.rollout(task, resources, rollout)` Class-based Agents[¶](https://microsoft.github.io/agent-lightning/latest/reference/agent/#class-based-agents "Permanent link") ------------------------------------------------------------------------------------------------------------------------------- ### `agentlightning.LitAgent` [¶](https://microsoft.github.io/agent-lightning/latest/reference/agent/#agentlightning.LitAgent "Permanent link") Bases: `Generic[T]` Base class for implementing agent rollouts. Subclasses override the rollout methods to process tasks while the trainer and runner infrastructure manages orchestration, tracing, and persistence. #### `runner` `property` [¶](https://microsoft.github.io/agent-lightning/latest/reference/agent/#agentlightning.LitAgent.runner "Permanent link") Return the runner responsible for executing rollouts. #### `tracer` `property` [¶](https://microsoft.github.io/agent-lightning/latest/reference/agent/#agentlightning.LitAgent.tracer "Permanent link") Return the tracer configured for this agent. #### `trainer` `property` [¶](https://microsoft.github.io/agent-lightning/latest/reference/agent/#agentlightning.LitAgent.trainer "Permanent link") Return the trainer associated with this agent. #### `__init__(*, trained_agents=None)` [¶](https://microsoft.github.io/agent-lightning/latest/reference/agent/#agentlightning.LitAgent.__init__ "Permanent link") Initialize the agent instance. Parameters: * **`trained_agents`** (`Optional[str]`, default: `None` ) – Optional identifier used by legacy tooling to mark trained agents. Deprecated The `trained_agents` flag is deprecated. Configure `agent_match` in the adapter layer instead. See [`TracerTraceToTriplet`](https://microsoft.github.io/agent-lightning/latest/reference/algorithm/#agentlightning.TracerTraceToTriplet " agentlightning.TracerTraceToTriplet") for more details. #### `get_runner()` [¶](https://microsoft.github.io/agent-lightning/latest/reference/agent/#agentlightning.LitAgent.get_runner "Permanent link") Return the runner responsible for executing rollouts. #### `get_tracer()` [¶](https://microsoft.github.io/agent-lightning/latest/reference/agent/#agentlightning.LitAgent.get_tracer "Permanent link") Return the tracer configured for this agent. #### `get_trainer()` [¶](https://microsoft.github.io/agent-lightning/latest/reference/agent/#agentlightning.LitAgent.get_trainer "Permanent link") Return the trainer associated with this agent. #### `is_async()` [¶](https://microsoft.github.io/agent-lightning/latest/reference/agent/#agentlightning.LitAgent.is_async "Permanent link") Return `True` when the agent overrides any asynchronous rollout methods. Override this method for customized async detection logic. #### `on_rollout_end(task, rollout, runner, tracer)` [¶](https://microsoft.github.io/agent-lightning/latest/reference/agent/#agentlightning.LitAgent.on_rollout_end "Permanent link") Hook invoked after a rollout completes. Subclasses can override this method for cleanup or additional logging. The default implementation is a no-op. Parameters: * **`task`** (`[Task](https://microsoft.github.io/agent-lightning/latest/reference/internal/#agentlightning.Task " agentlightning.Task (agentlightning.types.Task)") `) – [`Task`](https://microsoft.github.io/agent-lightning/latest/reference/internal/#agentlightning.Task " agentlightning.Task") that was processed. * **`rollout`** (`[Rollout](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.Rollout " agentlightning.Rollout (agentlightning.types.Rollout)") `) – Resulting [`Rollout`](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.Rollout " agentlightning.Rollout") . * **`runner`** (`[Runner](https://microsoft.github.io/agent-lightning/latest/reference/runner/#agentlightning.Runner " agentlightning.Runner (agentlightning.runner.Runner)") [T]`) – [`Runner`](https://microsoft.github.io/agent-lightning/latest/reference/runner/#agentlightning.Runner " agentlightning.Runner") managing the rollout. * **`tracer`** (`[Tracer](https://microsoft.github.io/agent-lightning/latest/reference/runner/#agentlightning.Tracer " agentlightning.Tracer (agentlightning.tracer.Tracer)") `) – [`Tracer`](https://microsoft.github.io/agent-lightning/latest/reference/runner/#agentlightning.Tracer " agentlightning.Tracer") associated with the runner. Deprecated Override [`Hook.on_rollout_end`](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.Hook.on_rollout_end " on_rollout_end(*, agent, runner, rollout, spans) async ") instead of this method when extending agents. #### `on_rollout_start(task, runner, tracer)` [¶](https://microsoft.github.io/agent-lightning/latest/reference/agent/#agentlightning.LitAgent.on_rollout_start "Permanent link") Hook invoked immediately before a rollout begins. Subclasses can override this method to implement custom logic such as logging, metric collection, or resource setup. The default implementation is a no-op. Parameters: * **`task`** (`[Task](https://microsoft.github.io/agent-lightning/latest/reference/internal/#agentlightning.Task " agentlightning.Task (agentlightning.types.Task)") `) – [`Task`](https://microsoft.github.io/agent-lightning/latest/reference/internal/#agentlightning.Task " agentlightning.Task") that will be processed. * **`runner`** (`[Runner](https://microsoft.github.io/agent-lightning/latest/reference/runner/#agentlightning.Runner " agentlightning.Runner (agentlightning.runner.Runner)") [T]`) – [`Runner`](https://microsoft.github.io/agent-lightning/latest/reference/runner/#agentlightning.Runner " agentlightning.Runner") managing the rollout. * **`tracer`** (`[Tracer](https://microsoft.github.io/agent-lightning/latest/reference/runner/#agentlightning.Tracer " agentlightning.Tracer (agentlightning.tracer.Tracer)") `) – [`Tracer`](https://microsoft.github.io/agent-lightning/latest/reference/runner/#agentlightning.Tracer " agentlightning.Tracer") associated with the runner. Deprecated Override [`Hook.on_rollout_start`](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.Hook.on_rollout_start " on_rollout_start(*, agent, runner, rollout) async ") instead of this method when extending agents. #### `rollout(task, resources, rollout)` [¶](https://microsoft.github.io/agent-lightning/latest/reference/agent/#agentlightning.LitAgent.rollout "Permanent link") Execute a rollout synchronously. If you don't wish to implement both training rollout and validation rollout separately, you can just implement `rollout` which will work for both. Parameters: * **`task`** (`T`) – Task payload provided by the scheduler. * **`resources`** (`[NamedResources](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.NamedResources " agentlightning.NamedResources = Dict[str, ResourceUnion] module-attribute (agentlightning.types.NamedResources)") `) – Mapping of named resources (for example LLMs or prompt templates). * **`rollout`** (`[Rollout](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.Rollout " agentlightning.Rollout (agentlightning.types.Rollout)") `) – Rollout metadata. Avoid mutating this object directly unless a subclass needs to override defaults. Returns: * `[RolloutRawResult](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.RolloutRawResult " agentlightning.RolloutRawResult = Union[None, float, List[ReadableSpan], List[Span], List[SpanCoreFields]] module-attribute (agentlightning.types.RolloutRawResult)") ` – One of the following values: * `[RolloutRawResult](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.RolloutRawResult " agentlightning.RolloutRawResult = Union[None, float, List[ReadableSpan], List[Span], List[SpanCoreFields]] module-attribute (agentlightning.types.RolloutRawResult)") ` – * `None` when tracing is handled by the runner. * `[RolloutRawResult](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.RolloutRawResult " agentlightning.RolloutRawResult = Union[None, float, List[ReadableSpan], List[Span], List[SpanCoreFields]] module-attribute (agentlightning.types.RolloutRawResult)") ` – * `float` representing the final reward. * `[RolloutRawResult](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.RolloutRawResult " agentlightning.RolloutRawResult = Union[None, float, List[ReadableSpan], List[Span], List[SpanCoreFields]] module-attribute (agentlightning.types.RolloutRawResult)") ` – * `List[ReadableSpan]` with OpenTelemetry spans. * `[RolloutRawResult](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.RolloutRawResult " agentlightning.RolloutRawResult = Union[None, float, List[ReadableSpan], List[Span], List[SpanCoreFields]] module-attribute (agentlightning.types.RolloutRawResult)") ` – * `List[Span]` with Agent Lightning spans. * `[RolloutRawResult](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.RolloutRawResult " agentlightning.RolloutRawResult = Union[None, float, List[ReadableSpan], List[Span], List[SpanCoreFields]] module-attribute (agentlightning.types.RolloutRawResult)") ` – * `List[SpanCoreFields]` with Agent Lightning spans. #### `rollout_async(task, resources, rollout)` `async` [¶](https://microsoft.github.io/agent-lightning/latest/reference/agent/#agentlightning.LitAgent.rollout_async "Permanent link") Execute a rollout asynchronously. Parameters: * **`task`** (`T`) – Task payload provided by the scheduler. * **`resources`** (`[NamedResources](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.NamedResources " agentlightning.NamedResources = Dict[str, ResourceUnion] module-attribute (agentlightning.types.NamedResources)") `) – Mapping of named resources (for example LLMs or prompt templates). * **`rollout`** (`[Rollout](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.Rollout " agentlightning.Rollout (agentlightning.types.Rollout)") `) – Rollout metadata. Avoid mutating this object directly unless a subclass needs to override defaults. Returns: * `[RolloutRawResult](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.RolloutRawResult " agentlightning.RolloutRawResult = Union[None, float, List[ReadableSpan], List[Span], List[SpanCoreFields]] module-attribute (agentlightning.types.RolloutRawResult)") ` – Same possible return values as * `[RolloutRawResult](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.RolloutRawResult " agentlightning.RolloutRawResult = Union[None, float, List[ReadableSpan], List[Span], List[SpanCoreFields]] module-attribute (agentlightning.types.RolloutRawResult)") ` – [`rollout`](https://microsoft.github.io/agent-lightning/latest/reference/agent/#agentlightning.LitAgent.rollout " rollout(task, resources, rollout)") . #### `set_runner(runner)` [¶](https://microsoft.github.io/agent-lightning/latest/reference/agent/#agentlightning.LitAgent.set_runner "Permanent link") Attach the runner responsible for executing rollouts. Parameters: * **`runner`** (`[Runner](https://microsoft.github.io/agent-lightning/latest/reference/runner/#agentlightning.Runner " agentlightning.Runner (agentlightning.runner.Runner)") [T]`) – [`Runner`](https://microsoft.github.io/agent-lightning/latest/reference/runner/#agentlightning.Runner " agentlightning.Runner") coordinating execution. #### `set_trainer(trainer)` [¶](https://microsoft.github.io/agent-lightning/latest/reference/agent/#agentlightning.LitAgent.set_trainer "Permanent link") Attach the trainer responsible for orchestration. Parameters: * **`trainer`** (`[Trainer](https://microsoft.github.io/agent-lightning/latest/reference/trainer/#agentlightning.Trainer " agentlightning.Trainer (agentlightning.trainer.Trainer)") `) – [`Trainer`](https://microsoft.github.io/agent-lightning/latest/reference/trainer/#agentlightning.Trainer " agentlightning.Trainer") that manages the agent. #### `training_rollout(task, resources, rollout)` [¶](https://microsoft.github.io/agent-lightning/latest/reference/agent/#agentlightning.LitAgent.training_rollout "Permanent link") Process a single training task synchronously. By default, this method delegates to [`rollout`](https://microsoft.github.io/agent-lightning/latest/reference/agent/#agentlightning.LitAgent.rollout " rollout(task, resources, rollout)") . #### `training_rollout_async(task, resources, rollout)` `async` [¶](https://microsoft.github.io/agent-lightning/latest/reference/agent/#agentlightning.LitAgent.training_rollout_async "Permanent link") Process a single training task asynchronously. By default, this method delegates to [`rollout_async`](https://microsoft.github.io/agent-lightning/latest/reference/agent/#agentlightning.LitAgent.rollout_async " rollout_async(task, resources, rollout) async ") . #### `validation_rollout(task, resources, rollout)` [¶](https://microsoft.github.io/agent-lightning/latest/reference/agent/#agentlightning.LitAgent.validation_rollout "Permanent link") Process a single validation task synchronously. Override this method when validation should differ from training. The default implementation delegates to [`training_rollout`](https://microsoft.github.io/agent-lightning/latest/reference/agent/#agentlightning.LitAgent.training_rollout " training_rollout(task, resources, rollout)") . #### `validation_rollout_async(task, resources, rollout)` `async` [¶](https://microsoft.github.io/agent-lightning/latest/reference/agent/#agentlightning.LitAgent.validation_rollout_async "Permanent link") Process a single validation task asynchronously. Override this method when validation should differ from training. The default implementation delegates to [`training_rollout_async`](https://microsoft.github.io/agent-lightning/latest/reference/agent/#agentlightning.LitAgent.training_rollout_async " training_rollout_async(task, resources, rollout) async ") . Emitter[¶](https://microsoft.github.io/agent-lightning/latest/reference/agent/#emitter "Permanent link") --------------------------------------------------------------------------------------------------------- ### `agentlightning.operation(fn=None, *, propagate=True, name=None, **additional_attributes)` [¶](https://microsoft.github.io/agent-lightning/latest/reference/agent/#agentlightning.operation "Permanent link") `[](https://microsoft.github.io/agent-lightning/latest/reference/agent/#__codelineno-0-1) operation( [](https://microsoft.github.io/agent-lightning/latest/reference/agent/#__codelineno-0-2) fn: _FnType, [](https://microsoft.github.io/agent-lightning/latest/reference/agent/#__codelineno-0-3) *, [](https://microsoft.github.io/agent-lightning/latest/reference/agent/#__codelineno-0-4) propagate: bool = True, [](https://microsoft.github.io/agent-lightning/latest/reference/agent/#__codelineno-0-5) name: Optional[str] = None, [](https://microsoft.github.io/agent-lightning/latest/reference/agent/#__codelineno-0-6) **additional_attributes: Any [](https://microsoft.github.io/agent-lightning/latest/reference/agent/#__codelineno-0-7) ) -> _FnType` `[](https://microsoft.github.io/agent-lightning/latest/reference/agent/#__codelineno-0-1) operation( [](https://microsoft.github.io/agent-lightning/latest/reference/agent/#__codelineno-0-2) *, [](https://microsoft.github.io/agent-lightning/latest/reference/agent/#__codelineno-0-3) propagate: bool = True, [](https://microsoft.github.io/agent-lightning/latest/reference/agent/#__codelineno-0-4) name: Optional[str] = None, [](https://microsoft.github.io/agent-lightning/latest/reference/agent/#__codelineno-0-5) **additional_attributes: Any [](https://microsoft.github.io/agent-lightning/latest/reference/agent/#__codelineno-0-6) ) -> OperationContext` `[](https://microsoft.github.io/agent-lightning/latest/reference/agent/#__codelineno-0-1) operation( [](https://microsoft.github.io/agent-lightning/latest/reference/agent/#__codelineno-0-2) fn: _FnType, [](https://microsoft.github.io/agent-lightning/latest/reference/agent/#__codelineno-0-3) *, [](https://microsoft.github.io/agent-lightning/latest/reference/agent/#__codelineno-0-4) name: Optional[str] = None, [](https://microsoft.github.io/agent-lightning/latest/reference/agent/#__codelineno-0-5) **additional_attributes: Any [](https://microsoft.github.io/agent-lightning/latest/reference/agent/#__codelineno-0-6) ) -> _FnType` `[](https://microsoft.github.io/agent-lightning/latest/reference/agent/#__codelineno-0-1) operation( [](https://microsoft.github.io/agent-lightning/latest/reference/agent/#__codelineno-0-2) *, [](https://microsoft.github.io/agent-lightning/latest/reference/agent/#__codelineno-0-3) name: Optional[str] = None, [](https://microsoft.github.io/agent-lightning/latest/reference/agent/#__codelineno-0-4) **additional_attributes: Any [](https://microsoft.github.io/agent-lightning/latest/reference/agent/#__codelineno-0-5) ) -> OperationContext` `[](https://microsoft.github.io/agent-lightning/latest/reference/agent/#__codelineno-0-1) operation( [](https://microsoft.github.io/agent-lightning/latest/reference/agent/#__codelineno-0-2) fn: _FnType, **additional_attributes: Any [](https://microsoft.github.io/agent-lightning/latest/reference/agent/#__codelineno-0-3) ) -> _FnType` `[](https://microsoft.github.io/agent-lightning/latest/reference/agent/#__codelineno-0-1) operation(**additional_attributes: Any) -> OperationContext` Entry point for tracking operations. This helper can be used either as a decorator or as a context manager. The span name is fixed to [`AGL_OPERATION`](https://microsoft.github.io/agent-lightning/latest/reference/semconv/#agentlightning.semconv.AGL_OPERATION " AGL_OPERATION = 'agentlightning.operation' module-attribute ") ; custom span names are not supported. Any keyword arguments are recorded as span attributes. Usage as a decorator: `[](https://microsoft.github.io/agent-lightning/latest/reference/agent/#__codelineno-0-1) @operation [](https://microsoft.github.io/agent-lightning/latest/reference/agent/#__codelineno-0-2) def func(...): [](https://microsoft.github.io/agent-lightning/latest/reference/agent/#__codelineno-0-3) ... [](https://microsoft.github.io/agent-lightning/latest/reference/agent/#__codelineno-0-4) [](https://microsoft.github.io/agent-lightning/latest/reference/agent/#__codelineno-0-5) @operation(category="compute") [](https://microsoft.github.io/agent-lightning/latest/reference/agent/#__codelineno-0-6) def func(...): [](https://microsoft.github.io/agent-lightning/latest/reference/agent/#__codelineno-0-7) ...` Usage as a context manager: `[](https://microsoft.github.io/agent-lightning/latest/reference/agent/#__codelineno-1-1) with operation(user_id=123) as op: [](https://microsoft.github.io/agent-lightning/latest/reference/agent/#__codelineno-1-2) op.set_input(data=data) [](https://microsoft.github.io/agent-lightning/latest/reference/agent/#__codelineno-1-3) # ... do work ... [](https://microsoft.github.io/agent-lightning/latest/reference/agent/#__codelineno-1-4) op.set_output(result)` Parameters: * **`fn`** (`Optional[_FnType]`, default: `None` ) – When used as `@operation`, this is the wrapped function. When used as `operation(**attrs)`, this should be omitted (or left as `None`) and only keyword attributes are provided. * **`propagate`** (`bool`, default: `True` ) – Whether spans should use the active span processor. When False, spans will stay local and not be exported. * **`name`** (`Optional[str]`, default: `None` ) – Optional alias that populates [`LightningSpanAttributes.OPERATION_NAME`](https://microsoft.github.io/agent-lightning/latest/reference/semconv/#agentlightning.semconv.LightningSpanAttributes.OPERATION_NAME " OPERATION_NAME = 'agentlightning.operation.name' class-attribute instance-attribute ") when `additional_attributes` does not already define it. * **`**additional_attributes`** (`Any`, default: `{}` ) – Additional span attributes to attach at creation time. Returns: * `Union[_FnType, [OperationContext](https://microsoft.github.io/agent-lightning/latest/reference/internal/#agentlightning.emitter.annotation.OperationContext " agentlightning.emitter.annotation.OperationContext") ]` – Either a wrapped callable (when used as a decorator) or an * `Union[_FnType, [OperationContext](https://microsoft.github.io/agent-lightning/latest/reference/internal/#agentlightning.emitter.annotation.OperationContext " agentlightning.emitter.annotation.OperationContext") ]` – [`OperationContext`](https://microsoft.github.io/agent-lightning/latest/reference/internal/#agentlightning.emitter.annotation.OperationContext " agentlightning.emitter.annotation.OperationContext") * `Union[_FnType, [OperationContext](https://microsoft.github.io/agent-lightning/latest/reference/internal/#agentlightning.emitter.annotation.OperationContext " agentlightning.emitter.annotation.OperationContext") ]` – (when used as a context manager factory). ### `agentlightning.emit_annotation(annotation, propagate=True)` [¶](https://microsoft.github.io/agent-lightning/latest/reference/agent/#agentlightning.emit_annotation "Permanent link") Emit a new annotation span. This is the underlying implementation of [`emit_reward`](https://microsoft.github.io/agent-lightning/latest/reference/agent/#agentlightning.emit_reward " agentlightning.emit_reward(reward, *, primary_key=None, attributes=None, propagate=True)") . Annotation spans are used to annotate a specific event or a part of rollout. See [semconv](https://microsoft.github.io/agent-lightning/latest/reference/semconv/#agentlightning.semconv " agentlightning.semconv") for conventional annotation keys in Agent-lightning. If annotations contain nested dicts, they will be flattened before emitting. Complex objects will lead to emitting failures. Parameters: * **`annotation`** (`Dict[str, Any]`) – Dictionary containing annotation key-value pairs. Representatives are rewards, tags, and metadata. * **`propagate`** (`bool`, default: `True` ) – Whether to propagate the span to tracers automatically. ### `agentlightning.emit_reward(reward, *, primary_key=None, attributes=None, propagate=True)` [¶](https://microsoft.github.io/agent-lightning/latest/reference/agent/#agentlightning.emit_reward "Permanent link") Emit a reward value as an OpenTelemetry span. Examples: Emit a single-dimensional reward: `[](https://microsoft.github.io/agent-lightning/latest/reference/agent/#__codelineno-0-1) >>> emit_reward(1.0)` Emit multi-dimensional rewards: `[](https://microsoft.github.io/agent-lightning/latest/reference/agent/#__codelineno-0-1) >>> emit_reward({"task_completion": 1.0, "efficiency": 0.8}, primary_key="task_completion")` Emit a reward with additional attributes (for example linking to another response span): `[](https://microsoft.github.io/agent-lightning/latest/reference/agent/#__codelineno-0-1) >>> from agentlightning.utils.otel import make_link_attributes [](https://microsoft.github.io/agent-lightning/latest/reference/agent/#__codelineno-0-2) >>> emit_reward(0.5, attributes=make_link_attributes({"gen_ai.response.id": "response-123"}))` Or adding tags onto the reward span: `[](https://microsoft.github.io/agent-lightning/latest/reference/agent/#__codelineno-0-1) >>> from agentlightning.utils.otel import make_tag_attributes [](https://microsoft.github.io/agent-lightning/latest/reference/agent/#__codelineno-0-2) >>> emit_reward(0.7, attributes=make_tag_attributes(["fast", "reliable"]))` Parameters: * **`reward`** (`float | Dict[str, Any]`) – Numeric reward to record. Integers and booleans are converted to floating point numbers for consistency. Use a dictionary to represent a multi-dimensional reward. * **`attributes`** (`Dict[str, Any] | None`, default: `None` ) – Other optional span attributes. * **`propagate`** (`bool`, default: `True` ) – Whether to propagate the span to exporters automatically. Returns: * `[SpanCoreFields](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.SpanCoreFields " agentlightning.SpanCoreFields (agentlightning.types.SpanCoreFields)") ` – Span core fields capturing the recorded reward. ### `agentlightning.emit_message(message, attributes=None, propagate=True)` [¶](https://microsoft.github.io/agent-lightning/latest/reference/agent/#agentlightning.emit_message "Permanent link") Emit a textual message as an OpenTelemetry span. Commonly used for sending debugging and logging messages. Parameters: * **`message`** (`str`) – Human readable message to attach as a span attribute. * **`attributes`** (`Optional[Dict[str, Any]]`, default: `None` ) – Additional attributes to attach to the message span. * **`propagate`** (`bool`, default: `True` ) – Whether to propagate the span to exporters automatically. Note OpenTelemetry distinguishes between logs and spans. Emitting the message as a span keeps all Agent Lightning telemetry in a single data store for analysis. ### `agentlightning.emit_object(object, attributes=None, propagate=True)` [¶](https://microsoft.github.io/agent-lightning/latest/reference/agent/#agentlightning.emit_object "Permanent link") Emit an object's serialized representation as an OpenTelemetry span. Parameters: * **`object`** (`Any`) – Data structure to encode as JSON and attach to the span payload. * **`attributes`** (`Optional[Dict[str, Any]]`, default: `None` ) – Additional attributes to attach to the object span. * **`propagate`** (`bool`, default: `True` ) – Whether to propagate the span to exporters automatically. Note The payload must be JSON serializable. Non-serializable objects will lead to a RuntimeError. ### `agentlightning.emit_exception(exception, attributes=None, propagate=True)` [¶](https://microsoft.github.io/agent-lightning/latest/reference/agent/#agentlightning.emit_exception "Permanent link") Record an exception with OpenTelemetry metadata. Classic OpenTelemetry records exceptions in a dedicated logging service. We simplify the model and use trace spans to record exceptions as well. Parameters: * **`exception`** (`BaseException`) – Raised exception instance to serialize into telemetry attributes. * **`attributes`** (`Optional[Dict[str, Any]]`, default: `None` ) – Additional attributes to attach to the exception span. * **`propagate`** (`bool`, default: `True` ) – Whether to propagate the span to exporters automatically. Note The helper validates its input. If a non-exception value is provided, a TypeError is raised to indicate a programming mistake. Emitter Helpers[¶](https://microsoft.github.io/agent-lightning/latest/reference/agent/#emitter-helpers "Permanent link") ------------------------------------------------------------------------------------------------------------------------- ### `agentlightning.get_message_value(span)` [¶](https://microsoft.github.io/agent-lightning/latest/reference/agent/#agentlightning.get_message_value "Permanent link") Extract the message string from a message span. Parameters: * **`span`** (`[SpanLike](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.SpanLike " agentlightning.SpanLike = Union[ReadableSpan, Span] module-attribute (agentlightning.types.SpanLike)") `) – Span-like object to extract the message from. ### `agentlightning.get_object_value(span)` [¶](https://microsoft.github.io/agent-lightning/latest/reference/agent/#agentlightning.get_object_value "Permanent link") Extract the object payload from an object span. Parameters: * **`span`** (`[SpanLike](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.SpanLike " agentlightning.SpanLike = Union[ReadableSpan, Span] module-attribute (agentlightning.types.SpanLike)") `) – Span object produced by Agent Lightning emitters. ### `agentlightning.find_final_reward(spans)` [¶](https://microsoft.github.io/agent-lightning/latest/reference/agent/#agentlightning.find_final_reward "Permanent link") Return the last reward value present in the provided spans. Parameters: * **`spans`** (`Sequence[[SpanLike](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.SpanLike " agentlightning.SpanLike = Union[ReadableSpan, Span] module-attribute (agentlightning.types.SpanLike)") ]`) – Sequence containing [`ReadableSpan`](https://opentelemetry.io/docs/concepts/signals/traces/) objects or mocked span-like values. Returns: * `Optional[float]` – Reward value from the latest reward span, or `None` when none are found. ### `agentlightning.find_reward_spans(spans)` [¶](https://microsoft.github.io/agent-lightning/latest/reference/agent/#agentlightning.find_reward_spans "Permanent link") Return all reward spans in the provided sequence. Parameters: * **`spans`** (`Sequence[[SpanLike](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.SpanLike " agentlightning.SpanLike = Union[ReadableSpan, Span] module-attribute (agentlightning.types.SpanLike)") ]`) – Sequence containing [`ReadableSpan`](https://opentelemetry.io/docs/concepts/signals/traces/) objects or mocked span-like values. Returns: * `List[[SpanLike](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.SpanLike " agentlightning.SpanLike = Union[ReadableSpan, Span] module-attribute (agentlightning.types.SpanLike)") ]` – List of spans that could be parsed as rewards. ### `agentlightning.get_reward_value(span)` [¶](https://microsoft.github.io/agent-lightning/latest/reference/agent/#agentlightning.get_reward_value "Permanent link") Extract the reward value from a span, if available. Parameters: * **`span`** (`[SpanLike](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.SpanLike " agentlightning.SpanLike = Union[ReadableSpan, Span] module-attribute (agentlightning.types.SpanLike)") `) – Span object produced by AgentOps or Agent Lightning emitters. Returns: * `Optional[float]` – The primary reward encoded in the span or `None` when the span does not represent a reward. ### `agentlightning.get_rewards_from_span(span)` [¶](https://microsoft.github.io/agent-lightning/latest/reference/agent/#agentlightning.get_rewards_from_span "Permanent link") Extract the reward as a list from a span, if available. Parameters: * **`span`** (`[SpanLike](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.SpanLike " agentlightning.SpanLike = Union[ReadableSpan, Span] module-attribute (agentlightning.types.SpanLike)") `) – Span object produced by AgentOps or Agent Lightning emitters. Returns: * `List[[RewardPydanticModel](https://microsoft.github.io/agent-lightning/latest/reference/semconv/#agentlightning.semconv.RewardPydanticModel " RewardPydanticModel (agentlightning.semconv.RewardPydanticModel)") ]` – A list of reward dimensions encoded in the span or an empty list when the span does not represent a reward. ### `agentlightning.is_reward_span(span)` [¶](https://microsoft.github.io/agent-lightning/latest/reference/agent/#agentlightning.is_reward_span "Permanent link") Return `True` when the provided span encodes a reward value. Back to top --- # Maintainer Guide - Agent-lightning [Skip to content](https://microsoft.github.io/agent-lightning/latest/community/maintainers/#maintainer-guide) ⚠️ You are viewing development documentation. For stable documentation, please visit [https://microsoft.github.io/agent-lightning/stable/](https://microsoft.github.io/agent-lightning/stable/) Maintainer Guide[¶](https://microsoft.github.io/agent-lightning/latest/community/maintainers/#maintainer-guide "Permanent link") ================================================================================================================================= This guide describes the day-to-day responsibilities for Agent Lightning maintainers—how to bump versions, run release ceremonies, interact with CI, and backport fixes safely. Release Workflow[¶](https://microsoft.github.io/agent-lightning/latest/community/maintainers/#release-workflow "Permanent link") --------------------------------------------------------------------------------------------------------------------------------- Follow this checklist throughout each release cycle. ### Immediately After Shipping[¶](https://microsoft.github.io/agent-lightning/latest/community/maintainers/#immediately-after-shipping "Permanent link") Agent Lightning uses a **bump-first** strategy. As soon as a release is published: 1. Update version metadata: * `pyproject.toml`: bump the `version`. * `agentlightning/__init__.py`: update `__version__` if it exists. * `uv.lock`: refresh the lock file after the bump. 2. Refresh dependency pins as needed: `[](https://microsoft.github.io/agent-lightning/latest/community/maintainers/#__codelineno-0-1) uv lock --upgrade` 3. For a new minor or major release, create a stable branch from `main`: `[](https://microsoft.github.io/agent-lightning/latest/community/maintainers/#__codelineno-1-1) git checkout main [](https://microsoft.github.io/agent-lightning/latest/community/maintainers/#__codelineno-1-2) git pull upstream main [](https://microsoft.github.io/agent-lightning/latest/community/maintainers/#__codelineno-1-3) git checkout -b stable/v2.0.x # adjust to the new series [](https://microsoft.github.io/agent-lightning/latest/community/maintainers/#__codelineno-1-4) git push upstream stable/v2.0.x` All future changes to the stable branch must land via pull requests. ### Preparing the Next Release[¶](https://microsoft.github.io/agent-lightning/latest/community/maintainers/#preparing-the-next-release "Permanent link") When it is time to publish the next version: 1. **Draft release notes** in `docs/changelog.md`, collecting every notable change since the previous tag. 2. **Open a release PR** targeting `main` (for minor/major) or the relevant stable branch (for patch releases). Use the title `[Release] vX.Y.Z`. 3. **Run extended CI** by labeling the PR with `ci-all` and commenting `/ci`. Investigate and resolve any failures. 4. **Merge the release PR** once notes are final and CI is green. 5. **Tag the release** from the branch you just merged into: `[](https://microsoft.github.io/agent-lightning/latest/community/maintainers/#__codelineno-2-1) git checkout main # minor/major releases [](https://microsoft.github.io/agent-lightning/latest/community/maintainers/#__codelineno-2-2) git checkout stable/vX.Y.Z # patch releases [](https://microsoft.github.io/agent-lightning/latest/community/maintainers/#__codelineno-2-3) [](https://microsoft.github.io/agent-lightning/latest/community/maintainers/#__codelineno-2-4) git pull [](https://microsoft.github.io/agent-lightning/latest/community/maintainers/#__codelineno-2-5) git tag vX.Y.Z -m "Release vX.Y.Z" [](https://microsoft.github.io/agent-lightning/latest/community/maintainers/#__codelineno-2-6) git push upstream vX.Y.Z` Pushing the tag publishes to PyPI and deploys the documentation. 6. **Publish the GitHub release** using the drafted notes, and confirm the docs site and PyPI listing reflect the new version. Working with CI Labels and `/ci`[¶](https://microsoft.github.io/agent-lightning/latest/community/maintainers/#working-with-ci-labels-and-ci "Permanent link") -------------------------------------------------------------------------------------------------------------------------------------------------------------- GPU suites and example end-to-end runs are opt-in. To trigger them on a pull request: 1. Apply the appropriate labels before issuing the command: * `ci-all` for every repository-dispatch workflow. * `ci-gpu` for GPU integration tests (`tests-full.yml`). * `ci-apo`, `ci-calc-x`, `ci-spider`, `ci-unsloth`, `ci-compat` for the individual example pipelines. 2. Comment `/ci` on the PR. The `issue-comment` workflow acknowledges the request and tracks job results inline. 3. Remove the labels once you have the signal to avoid accidental re-runs. Use `/ci` whenever changes touch shared infrastructure, dependencies, or training loops that require coverage beyond the default PR checks. Note `/ci` always executes the workflow definitions on the current `main` branch, then checks out the PR diff. If you need to test workflow modifications, push the changes to a branch in the upstream repo and run: `[](https://microsoft.github.io/agent-lightning/latest/community/maintainers/#__codelineno-3-1) gh workflow run examples-xxx.yml --ref your-branch-name` Backporting Pull Requests[¶](https://microsoft.github.io/agent-lightning/latest/community/maintainers/#backporting-pull-requests "Permanent link") --------------------------------------------------------------------------------------------------------------------------------------------------- Supported stable branches rely on automated backports: 1. Identify the target branch (for example `stable/v0.2.x`). 2. Before merging the original PR into `main`, add the matching `stable/` label (e.g. `stable/v0.2.x`). 3. The `backport.yml` workflow opens a follow-up PR named `backport//` authored by `agent-lightning-bot`. 4. Review the generated PR, ensure CI is green, and merge into the stable branch. 5. Resolve conflicts by pushing manual fixes to the backport branch and re-running `/ci` if required. Keep stable branches healthy by cherry-picking only critical fixes and ensuring documentation and example metadata stay aligned with each release line. Back to top --- # Serving LLM - Agent-lightning [Skip to content](https://microsoft.github.io/agent-lightning/latest/deep-dive/serving-llm/#serving-llms-under-agent-lightning) ⚠️ You are viewing development documentation. For stable documentation, please visit [https://microsoft.github.io/agent-lightning/stable/](https://microsoft.github.io/agent-lightning/stable/) Serving LLMs under Agent-lightning[¶](https://microsoft.github.io/agent-lightning/latest/deep-dive/serving-llm/#serving-llms-under-agent-lightning "Permanent link") ===================================================================================================================================================================== Agent-lightning focuses on data, learning signals, and control flow — **not** on running model inference. This deep dive explains how to **serve** a model alongside Agent-lightning so runners can call it reliably, how the **LLM Proxy** fits into the loop, and why **token IDs** matter if you care about correctness in training and evaluation. General background on LLM serving[¶](https://microsoft.github.io/agent-lightning/latest/deep-dive/serving-llm/#general-background-on-llm-serving "Permanent link") ------------------------------------------------------------------------------------------------------------------------------------------------------------------- [](https://microsoft.github.io/agent-lightning/latest/deep-dive/serving-llm/) Serving a model is essential if you want to train it, especially when you use the model’s own generations as training data. We’ll briefly review the general background to ensure all readers are aligned. Modern LLM servers solve a difficult scheduling problem: keeping GPUs fully utilized while handling prompts of different lengths, streaming tokens as they arrive, and fitting large KV caches into limited memory. Techniques like [**continuous batching**](https://www.anyscale.com/blog/continuous-batching-llm-inference) and [**paged attention**](https://arxiv.org/abs/2309.06180) address these challenges. Continuous batching interleaves decoding across requests to reuse weights efficiently; with careful memory planning, it achieves major throughput gains without increasing latency. PagedAttention reduces KV-cache fragmentation so batching remains effective as sequences grow. See [vLLM’s PagedAttention paper](https://arxiv.org/abs/2309.06180) and [industry analyses](https://www.baseten.co/blog/continuous-vs-dynamic-batching-for-ai-inference/) for details. Balancing inference correctness and efficiency is difficult — a [recent blog](https://thinkingmachines.ai/blog/defeating-nondeterminism-in-llm-inference/) from Thinking Machines Labs highlights how inference nondeterminism ultimately affects training. Beyond scheduling, servers expose an HTTP API, often **OpenAI-compatible** (`/v1/chat/completions` and `/v1/responses`), which is itself a complex stack. In addition to text prompts and chat messages, the API defines many parameters and response fields such as [tool calls](https://platform.openai.com/docs/guides/function-calling) , [structured output](https://platform.openai.com/docs/guides/structured-outputs) , and [multimodal support](https://platform.openai.com/docs/guides/images-vision) . Much effort has been put into implementing all these parameters for many frameworks. Popular engines like **vLLM** and [**SGLang**](https://github.com/sgl-project/sglang) ship with OpenAI-compatible frontends so you can reuse existing client code. [Ollama](https://ollama.com/blog/openai-compatibility) and [llama.cpp](https://llama-cpp-python.readthedocs.io/en/latest/server/) provide similar capabilities. However, because models differ internally, each framework interprets and implements the API slightly differently. Even with identical requests, the tokens passed to the model can vary substantially across frameworks. What Agent-lightning expects from a served LLM[¶](https://microsoft.github.io/agent-lightning/latest/deep-dive/serving-llm/#what-agent-lightning-expects-from-a-served-llm "Permanent link") --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Most of the issues above either have workarounds or remain open research problems. Keep them in mind, but the key question is: what does Agent-lightning expect from a served LLM? The answer includes at least two things: * An OpenAI-compatible **Chat Completions** or **Responses** endpoint the agent can call during rollouts. * Optional training and debugging signals: **logprobs**, **usage**, and ideally **token IDs**. (OpenAI’s public API exposes usage and logprobs, but **not** token IDs — more on [why IDs matter](https://microsoft.github.io/agent-lightning/latest/deep-dive/serving-llm/#token-ids-matter "Token IDs and why they matter") later.) Launching a serving framework[¶](https://microsoft.github.io/agent-lightning/latest/deep-dive/serving-llm/#launching-a-serving-framework "Permanent link") ----------------------------------------------------------------------------------------------------------------------------------------------------------- For many algorithms, you’ll start an engine (e.g., **vLLM** or **SGLang**) before rollouts, then shut it down afterward to free GPU memory. Most frameworks provide a one-line “serve” command to launch the OpenAI-compatible server. You can use those to bring up `/v1/chat/completions` with your checkpoint, ensuring streaming and any required tool-calling features are enabled. A working example is shown in [Unsloth SFT](https://microsoft.github.io/agent-lightning/latest/how-to/unsloth-sft/) . Weight updates — which occur after each training step — are trickier. Some frameworks like [vLLM](https://vllm.ai/) support hot-updating model weights, but it’s usually simpler and more reliable to restart the engine to load new weights. For medium-sized tasks (hundreds of rollouts taking 10+ minutes), the restart overhead (under 30 seconds) is typically negligible. If you’re using Agent-lightning’s [**VERL**](https://microsoft.github.io/agent-lightning/latest/algorithm-zoo/verl/#agentlightning.algorithm.verl.VERL " VERL") integration, the algorithm can **manage the server automatically**. The [VERL framework](https://github.com/volcengine/verl) intelligently allocates compute resources and wraps vLLM/SGLang behind an `AsyncLLMServer` abstraction. You can directly use this as the LLM endpoint for agents. Since VERL can spawn multiple vLLM replicas, using [`LLMProxy`](https://microsoft.github.io/agent-lightning/latest/reference/algorithm/#agentlightning.LLMProxy " agentlightning.LLMProxy") to manage them adds an additional safety layer. A full sequence diagram of how [VERL](https://microsoft.github.io/agent-lightning/latest/algorithm-zoo/verl/#agentlightning.algorithm.verl.VERL " VERL") interacts with the LLM server and proxy is available [here](https://microsoft.github.io/agent-lightning/latest/deep-dive/birds-eye-view/#birds-eye-view-verl-example "Putting It All Together: A Reinforcement Learning Example (VERL)") . LLM Proxy[¶](https://microsoft.github.io/agent-lightning/latest/deep-dive/serving-llm/#llm-proxy "Permanent link") ------------------------------------------------------------------------------------------------------------------- The **LLM Proxy** is a utility class in Agent-lightning, built on [LiteLLM](https://docs.litellm.ai/) , that sits between runners and your backend engine(s) or server(s). In Agent-lightning it acts as a single URL registered as a [`Resource`](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.Resource " agentlightning.Resource") in the store, offering three key benefits: 1. **Unified endpoint & hot-swaps.** You can redirect traffic between OpenAI, Anthropic, local vLLM/SGLang, or canary checkpoints without modifying agent code — simply repoint the proxy. 2. **First-class tracing.** The proxy emits **OpenTelemetry** spans for every call and sends them to the [`LightningStore`](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.LightningStore " agentlightning.LightningStore") . It includes rollout and attempt identifiers in request headers so spans are correctly attributed. Sequence numbers are allocated monotonically via the store to [prevent clock-skew issues](https://microsoft.github.io/agent-lightning/latest/tutorials/traces/#distributed-tracing "LLM Proxy") and allow reliable reconstruction of execution trees. 3. **Token IDs.** The proxy can return prompt and response token IDs along with the model output. More details are available in the [next section](https://microsoft.github.io/agent-lightning/latest/deep-dive/serving-llm/#token-ids-matter "Token IDs and why they matter") . Operationally, running the proxy alongside the algorithm works best: the algorithm registers the backend (e.g., the vLLM URL) via [`LLMProxy.update_model_list`](https://microsoft.github.io/agent-lightning/latest/reference/algorithm/#agentlightning.LLMProxy.update_model_list " update_model_list(model_list)") , publishes the proxy URL as a resource via [`LightningStore.add_resources`](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.LightningStore.add_resources " add_resources(resources) async ") , and runners simply use that URL during rollouts. This mirrors many production client–server setups. Token IDs and why they matter[¶](https://microsoft.github.io/agent-lightning/latest/deep-dive/serving-llm/#token-ids-and-why-they-matter "Permanent link") ----------------------------------------------------------------------------------------------------------------------------------------------------------- [](https://microsoft.github.io/agent-lightning/latest/deep-dive/serving-llm/) This section explains how Agent-lightning handles and uses token IDs — a subtle but important detail for training stability and accuracy. Most agents interact with LLMs via **Chat Completion APIs**, exchanging chat messages. There are two main approaches to collecting training data from such agents. Note Tokenization here refers to the process of converting **Chat Messages** into **Token IDs**. Detokenization is the reverse process of converting **Token IDs** back to **Chat Messages**. Normally, the tokenizer is published along with the pretrained model, which includes a vocabulary, special tokens, and a chat template to dealing with chat messages. **1\. Retokenizing chat messages.** In this approach, you store chat messages as text and let training algorithms **retokenize** them later, as done in many SFT workflows (e.g., [HuggingFace SFT](https://huggingface.co/docs/trl/sft_trainer) ). In practice, we’ve found this method unstable and less accurate. The chart below compares training results. The retokenization approach is run twice. All settings are the same except for the retokenization approach. This instability has three causes. Firstly, chat template used in different frameworks could be slightly different. For example, one single LLaMA model can work with multiple chat templates (multiple in [vLLM](https://github.com/vllm-project/vllm/tree/1d165d6d859d3c50720f0c07209db2363c4fd33b/examples) and one in [HuggingFace](https://huggingface.co/meta-llama) ). It's possible that the chat template used in detokenization is different from the one used in tokenization (this is actually an implementation bug). Secondly, a word might be generated as two tokens (e.g., `H + AVING`) but later retokenized as `HAV + ING`. The text looks identical, but the token IDs differ from what the model originally produced. Thirdly, a generated tool call text like `{ "name": ... }` is parsed by tool call parser into an object that is required by chat completion API. Later, the object is rendered back to `{ "name": ... }` and retokenized again, tool call parsing and re-rendering might cause changes in whitespace and formatting. In some situations, JSON errors may even be auto-corrected by the tool call parser — masking the model’s true generation errors and preventing them from being trained away. * * * **2\. Saving token IDs directly.** The alternative is to save the token IDs generated by the model, as done in RL setups like [Tinker](https://thinkingmachines.ai/tinker/) . This requires a training pipeline that treats tokens as first-class entities, meaning agents must communicate with the inference engine at the token level. However, most agents — especially those built with frameworks like LangChain — rely on OpenAI-compatible APIs and can’t tokenize or detokenize themselves. As mentioned [earlier](https://microsoft.github.io/agent-lightning/latest/deep-dive/serving-llm/#general-llm-serving-background "General background on LLM serving") , implementing this layer manually is complex and error-prone. Some frameworks implement custom solutions (e.g., [VERL Agent Loop](https://github.com/volcengine/verl/blob/4da0d3d3188072772cb2ec817b3d6cf4a463821f/recipe/langgraph_agent/chat_model.py) , [Tinker Renderer](https://github.com/thinking-machines-lab/tinker-cookbook/blob/34a6588d7055040c259985d98e71c0140b389ba7/tinker_cookbook/renderers.py) ), while others leave it to users (e.g., [SkyRL Search-R1](https://novasky-ai.notion.site/skyrl-searchr1) ). * * * A better solution is to use an **OpenAI-compatible API that returns token IDs directly.** This lets agents continue using familiar APIs while capturing token IDs via [tracing](https://microsoft.github.io/agent-lightning/latest/tutorials/traces/) for training. The limitation, of course, is that the serving framework must actually support this capability. When Agent-lightning was first released, we implemented an [instrumented vLLM server](https://github.com/microsoft/agent-lightning/blob/v0.1/agentlightning/instrumentation/vllm.py) that monkey-patched vLLM’s OpenAI server to return token IDs. Since then, the Agent-lightning and vLLM teams have collaborated to add this feature directly to [vLLM core](https://github.com/vllm-project/vllm/pull/22587) . Starting with **vLLM v0.10.2**, the OpenAI-compatible API includes a [`return_token_ids` parameter](https://docs.vllm.ai/en/v0.10.2/serving/openai_compatible_server.html#api-reference) , allowing token IDs to be requested alongside chat messages. SGLang has tracked [similar feature requests](https://github.com/sgl-project/sglang/issues/2634) , though its OpenAI-compatible layer doesn’t yet support them. In short, when using vLLM v0.10.2 or newer, [`LLMProxy`](https://microsoft.github.io/agent-lightning/latest/reference/algorithm/#agentlightning.LLMProxy " agentlightning.LLMProxy") automatically adds `return_token_ids` to each request so the engine includes token IDs in its response. For older vLLM versions, you still need the instrumented version (via `agl vllm` CLI command). Finally, if you only save token IDs in spans, it will have its own limitations — if you train one model using spans from another model with a different tokenizer, incompatibilities can arise. In practice, though, spans in Agent-lightning always store both chat messages and token IDs (actually the full request and response objects), allowing you to fall back to retokenization when necessary. Back to top --- # Algorithm - Agent-lightning [Skip to content](https://microsoft.github.io/agent-lightning/latest/reference/algorithm/#algorithm-side-references) ⚠️ You are viewing development documentation. For stable documentation, please visit [https://microsoft.github.io/agent-lightning/stable/](https://microsoft.github.io/agent-lightning/stable/) Algorithm ========= Algorithm-side References[¶](https://microsoft.github.io/agent-lightning/latest/reference/algorithm/#algorithm-side-references "Permanent link") ------------------------------------------------------------------------------------------------------------------------------------------------- Note This reference covers APIs that are designed to be used at "Algorithm Side". For built-in algorithms, see [Algorithm Zoo](https://microsoft.github.io/agent-lightning/latest/algorithm-zoo/) . Base Class and Decorators[¶](https://microsoft.github.io/agent-lightning/latest/reference/algorithm/#base-class-and-decorators "Permanent link") ------------------------------------------------------------------------------------------------------------------------------------------------- ### `agentlightning.Algorithm` [¶](https://microsoft.github.io/agent-lightning/latest/reference/algorithm/#agentlightning.Algorithm "Permanent link") Algorithm is the strategy, or tuner to train the agent. #### `get_adapter()` [¶](https://microsoft.github.io/agent-lightning/latest/reference/algorithm/#agentlightning.Algorithm.get_adapter "Permanent link") Retrieve the adapter for this algorithm to communicate with the runners. #### `get_client()` [¶](https://microsoft.github.io/agent-lightning/latest/reference/algorithm/#agentlightning.Algorithm.get_client "Permanent link") Get the client to communicate with the algorithm. If the algorithm does not require a server-client communication, it can also create a mock client that never communicates with itself. Deprecated and will be removed in a future version. Returns: * `[AgentLightningClient](https://microsoft.github.io/agent-lightning/latest/reference/internal/#agentlightning.client.AgentLightningClient " agentlightning.client.AgentLightningClient") ` – The AgentLightningClient instance associated with this algorithm. #### `get_initial_resources()` [¶](https://microsoft.github.io/agent-lightning/latest/reference/algorithm/#agentlightning.Algorithm.get_initial_resources "Permanent link") Get the initial resources for this algorithm. #### `get_llm_proxy()` [¶](https://microsoft.github.io/agent-lightning/latest/reference/algorithm/#agentlightning.Algorithm.get_llm_proxy "Permanent link") Retrieve the configured LLM proxy instance, if one has been set. Returns: * `Optional[[LLMProxy](https://microsoft.github.io/agent-lightning/latest/reference/algorithm/#agentlightning.LLMProxy " agentlightning.LLMProxy (agentlightning.llm_proxy.LLMProxy)") ]` – The active LLMProxy instance or None when not configured. #### `get_store()` [¶](https://microsoft.github.io/agent-lightning/latest/reference/algorithm/#agentlightning.Algorithm.get_store "Permanent link") Retrieve the store for this algorithm to communicate with the runners. #### `get_trainer()` [¶](https://microsoft.github.io/agent-lightning/latest/reference/algorithm/#agentlightning.Algorithm.get_trainer "Permanent link") Get the trainer for this algorithm. Returns: * `[Trainer](https://microsoft.github.io/agent-lightning/latest/reference/trainer/#agentlightning.Trainer " agentlightning.Trainer (agentlightning.trainer.Trainer)") ` – The Trainer instance associated with this agent. #### `is_async()` [¶](https://microsoft.github.io/agent-lightning/latest/reference/algorithm/#agentlightning.Algorithm.is_async "Permanent link") Return True if the algorithm is asynchronous. #### `run(train_dataset=None, val_dataset=None)` [¶](https://microsoft.github.io/agent-lightning/latest/reference/algorithm/#agentlightning.Algorithm.run "Permanent link") Subclasses should implement this method to implement the algorithm. Parameters: * **`train_dataset`** (`Optional[[Dataset](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.Dataset " agentlightning.Dataset (agentlightning.types.Dataset)") [Any]]`, default: `None` ) – The dataset to train on. Not all algorithms require a training dataset. * **`val_dataset`** (`Optional[[Dataset](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.Dataset " agentlightning.Dataset (agentlightning.types.Dataset)") [Any]]`, default: `None` ) – The dataset to validate on. Not all algorithms require a validation dataset. Returns: * `Union[None, Awaitable[None]]` – Algorithm should refrain from returning anything. It should just run the algorithm. #### `set_adapter(adapter)` [¶](https://microsoft.github.io/agent-lightning/latest/reference/algorithm/#agentlightning.Algorithm.set_adapter "Permanent link") Set the adapter for this algorithm to collect and convert traces. #### `set_initial_resources(resources)` [¶](https://microsoft.github.io/agent-lightning/latest/reference/algorithm/#agentlightning.Algorithm.set_initial_resources "Permanent link") Set the initial resources for this algorithm. #### `set_llm_proxy(llm_proxy)` [¶](https://microsoft.github.io/agent-lightning/latest/reference/algorithm/#agentlightning.Algorithm.set_llm_proxy "Permanent link") Set the LLM proxy for this algorithm to reuse when available. Parameters: * **`llm_proxy`** (`[LLMProxy](https://microsoft.github.io/agent-lightning/latest/reference/algorithm/#agentlightning.LLMProxy " agentlightning.LLMProxy (agentlightning.llm_proxy.LLMProxy)") | None`) – The LLMProxy instance configured by the trainer, if any. #### `set_store(store)` [¶](https://microsoft.github.io/agent-lightning/latest/reference/algorithm/#agentlightning.Algorithm.set_store "Permanent link") Set the store for this algorithm to communicate with the runners. Store is set directly instead of using weakref because its copy is meant to be maintained throughout the algorithm's lifecycle. #### `set_trainer(trainer)` [¶](https://microsoft.github.io/agent-lightning/latest/reference/algorithm/#agentlightning.Algorithm.set_trainer "Permanent link") Set the trainer for this algorithm. Parameters: * **`trainer`** (`[Trainer](https://microsoft.github.io/agent-lightning/latest/reference/trainer/#agentlightning.Trainer " agentlightning.Trainer (agentlightning.trainer.Trainer)") `) – The Trainer instance that will handle training and validation. ### `agentlightning.algo(func)` [¶](https://microsoft.github.io/agent-lightning/latest/reference/algorithm/#agentlightning.algo "Permanent link") `[](https://microsoft.github.io/agent-lightning/latest/reference/algorithm/#__codelineno-0-1) algo( [](https://microsoft.github.io/agent-lightning/latest/reference/algorithm/#__codelineno-0-2) func: AlgorithmFuncAsync, [](https://microsoft.github.io/agent-lightning/latest/reference/algorithm/#__codelineno-0-3) ) -> FunctionalAlgorithm[Literal[True]]` `[](https://microsoft.github.io/agent-lightning/latest/reference/algorithm/#__codelineno-0-1) algo( [](https://microsoft.github.io/agent-lightning/latest/reference/algorithm/#__codelineno-0-2) func: AlgorithmFuncAsyncFallback, [](https://microsoft.github.io/agent-lightning/latest/reference/algorithm/#__codelineno-0-3) ) -> FunctionalAlgorithm[Any]` `[](https://microsoft.github.io/agent-lightning/latest/reference/algorithm/#__codelineno-0-1) algo( [](https://microsoft.github.io/agent-lightning/latest/reference/algorithm/#__codelineno-0-2) func: AlgorithmFuncSync, [](https://microsoft.github.io/agent-lightning/latest/reference/algorithm/#__codelineno-0-3) ) -> FunctionalAlgorithm[Literal[False]]` `[](https://microsoft.github.io/agent-lightning/latest/reference/algorithm/#__codelineno-0-1) algo( [](https://microsoft.github.io/agent-lightning/latest/reference/algorithm/#__codelineno-0-2) func: AlgorithmFuncSyncFallback, [](https://microsoft.github.io/agent-lightning/latest/reference/algorithm/#__codelineno-0-3) ) -> FunctionalAlgorithm[Any]` Convert a callable into a [`FunctionalAlgorithm`](https://microsoft.github.io/agent-lightning/latest/reference/internal/#agentlightning.algorithm.decorator.FunctionalAlgorithm " agentlightning.algorithm.decorator.FunctionalAlgorithm") . The decorator inspects the callable signature to decide which dependencies to inject at runtime, enabling concise algorithm definitions that still leverage the full training runtime. Parameters: * **`func`** (`Union[AlgorithmFuncSync, AlgorithmFuncAsync, AlgorithmFuncSyncFallback, AlgorithmFuncAsyncFallback]`) – Function implementing the algorithm logic. May be synchronous or asynchronous. The function can expect all of, or a subset of the following parameters: * `store`: [`LightningStore`](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.LightningStore " agentlightning.LightningStore") , * `train_dataset`: [`Dataset`](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.Dataset " agentlightning.Dataset") , * `val_dataset`: [`Dataset`](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.Dataset " agentlightning.Dataset") , * `llm_proxy`: [`LLMProxy`](https://microsoft.github.io/agent-lightning/latest/reference/algorithm/#agentlightning.LLMProxy " agentlightning.LLMProxy") , * `adapter`: [`TraceAdapter`](https://microsoft.github.io/agent-lightning/latest/reference/algorithm/#agentlightning.TraceAdapter " agentlightning.TraceAdapter") , * `initial_resources`: [`NamedResources`](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.NamedResources " agentlightning.NamedResources = Dict[str, ResourceUnion] module-attribute ") , If the function does not expect a parameter, the wrapper will not inject it into the call. Using `*args` and `**kwargs` will not work and no parameters will be injected. Returns: * `Union[[FunctionalAlgorithm](https://microsoft.github.io/agent-lightning/latest/reference/internal/#agentlightning.algorithm.decorator.FunctionalAlgorithm " agentlightning.algorithm.decorator.FunctionalAlgorithm") [Literal[False]], [FunctionalAlgorithm](https://microsoft.github.io/agent-lightning/latest/reference/internal/#agentlightning.algorithm.decorator.FunctionalAlgorithm " agentlightning.algorithm.decorator.FunctionalAlgorithm") [Literal[True]]]` – FunctionalAlgorithm that proxies the callable while exposing the * `Union[[FunctionalAlgorithm](https://microsoft.github.io/agent-lightning/latest/reference/internal/#agentlightning.algorithm.decorator.FunctionalAlgorithm " agentlightning.algorithm.decorator.FunctionalAlgorithm") [Literal[False]], [FunctionalAlgorithm](https://microsoft.github.io/agent-lightning/latest/reference/internal/#agentlightning.algorithm.decorator.FunctionalAlgorithm " agentlightning.algorithm.decorator.FunctionalAlgorithm") [Literal[True]]]` – `Algorithm` interface. Examples: `[](https://microsoft.github.io/agent-lightning/latest/reference/algorithm/#__codelineno-0-1) from agentlightning.algorithm.decorator import algo [](https://microsoft.github.io/agent-lightning/latest/reference/algorithm/#__codelineno-0-2) [](https://microsoft.github.io/agent-lightning/latest/reference/algorithm/#__codelineno-0-3) @algo [](https://microsoft.github.io/agent-lightning/latest/reference/algorithm/#__codelineno-0-4) def batching_algorithm(*, store, train_dataset, val_dataset): [](https://microsoft.github.io/agent-lightning/latest/reference/algorithm/#__codelineno-0-5) for sample in train_dataset: [](https://microsoft.github.io/agent-lightning/latest/reference/algorithm/#__codelineno-0-6) store.enqueue_rollout(input=sample, mode="train") [](https://microsoft.github.io/agent-lightning/latest/reference/algorithm/#__codelineno-0-7) [](https://microsoft.github.io/agent-lightning/latest/reference/algorithm/#__codelineno-0-8) @algo [](https://microsoft.github.io/agent-lightning/latest/reference/algorithm/#__codelineno-0-9) async def async_algorithm(*, store, train_dataset=None, val_dataset=None): [](https://microsoft.github.io/agent-lightning/latest/reference/algorithm/#__codelineno-0-10) await store.enqueue_rollout(input={"prompt": "hello"}, mode="train")` Fast Algorithms (for Debugging)[¶](https://microsoft.github.io/agent-lightning/latest/reference/algorithm/#fast-algorithms-for-debugging "Permanent link") ----------------------------------------------------------------------------------------------------------------------------------------------------------- ### `agentlightning.FastAlgorithm` [¶](https://microsoft.github.io/agent-lightning/latest/reference/algorithm/#agentlightning.FastAlgorithm "Permanent link") Bases: `[Algorithm](https://microsoft.github.io/agent-lightning/latest/reference/algorithm/#agentlightning.Algorithm " agentlightning.Algorithm (agentlightning.algorithm.base.Algorithm)") ` Base class for lightweight algorithms optimised for developer workflows. Fast algorithms prioritise short feedback loops so an agent developer can run small-scale experiments without waiting for long-running training jobs to finish. ### `agentlightning.Baseline` [¶](https://microsoft.github.io/agent-lightning/latest/reference/algorithm/#agentlightning.Baseline "Permanent link") Bases: `[FastAlgorithm](https://microsoft.github.io/agent-lightning/latest/reference/algorithm/#agentlightning.FastAlgorithm " agentlightning.FastAlgorithm (agentlightning.algorithm.fast.FastAlgorithm)") ` Reference implementation that streams the full dataset through the rollout queue. The baseline algorithm batches task submissions, waits for each rollout to finish, and logs every collected span and reward. It is primarily useful as a smoke test for the platform plumbing rather than a performant trainer. The baseline algorithm will auto-start a LLM proxy if one is provided and not yet started. Parameters: * **`n_epochs`** (`int`, default: `1` ) – Number of dataset passes to execute for both the train and val splits during developer experiments. * **`train_split`** (`float`, default: `0.5` ) – Fraction of the concatenated dataset to treat as training data. Must be strictly between 0 and 1. * **`polling_interval`** (`float`, default: `5.0` ) – Interval, in seconds, to poll the store for queue depth and rollout completion. * **`max_queue_length`** (`int`, default: `4` ) – Number of rollouts allowed to wait in the queue before throttling additional submissions. * **`span_verbosity`** (`Literal['keys', 'key_values', 'none']`, default: `'keys'` ) – Level of detail to include when logging span metadata. Raises: * `ValueError` – If `train_split` falls outside the `(0, 1)` interval. Examples: `[](https://microsoft.github.io/agent-lightning/latest/reference/algorithm/#__codelineno-0-1) from agentlightning.algorithm.fast import Baseline [](https://microsoft.github.io/agent-lightning/latest/reference/algorithm/#__codelineno-0-2) [](https://microsoft.github.io/agent-lightning/latest/reference/algorithm/#__codelineno-0-3) algorithm = Baseline(n_epochs=2, train_split=0.8, span_verbosity="key_values") [](https://microsoft.github.io/agent-lightning/latest/reference/algorithm/#__codelineno-0-4) trainer.fit(algorithm, train_dataset=my_train, val_dataset=my_val)` #### `run(store, llm_proxy, train_dataset=None, val_dataset=None)` `async` [¶](https://microsoft.github.io/agent-lightning/latest/reference/algorithm/#agentlightning.Baseline.run "Permanent link") Execute the baseline loop across the provided datasets. Adapter[¶](https://microsoft.github.io/agent-lightning/latest/reference/algorithm/#adapter "Permanent link") ------------------------------------------------------------------------------------------------------------- ### `agentlightning.Adapter` [¶](https://microsoft.github.io/agent-lightning/latest/reference/algorithm/#agentlightning.Adapter "Permanent link") Bases: `Generic[T_from, T_to]` Base class for synchronous adapters that convert data from one format to another. The class defines a minimal protocol so that adapters can be treated like callables while still allowing subclasses to supply the concrete transformation logic. Note Subclasses must override [`adapt()`](https://microsoft.github.io/agent-lightning/latest/reference/algorithm/#agentlightning.Adapter.adapt " adapt(source)") to provide the actual conversion. Type Variables: `T_from: Source data type supplied to the adapter. T_to: Target data type produced by the adapter.` Examples: `[](https://microsoft.github.io/agent-lightning/latest/reference/algorithm/#__codelineno-0-1) >>> class IntToStrAdapter(Adapter[int, str]): [](https://microsoft.github.io/agent-lightning/latest/reference/algorithm/#__codelineno-0-2) ... def adapt(self, source: int) -> str: [](https://microsoft.github.io/agent-lightning/latest/reference/algorithm/#__codelineno-0-3) ... return str(source) [](https://microsoft.github.io/agent-lightning/latest/reference/algorithm/#__codelineno-0-4) ... [](https://microsoft.github.io/agent-lightning/latest/reference/algorithm/#__codelineno-0-5) >>> adapter = IntToStrAdapter() [](https://microsoft.github.io/agent-lightning/latest/reference/algorithm/#__codelineno-0-6) >>> adapter(42) [](https://microsoft.github.io/agent-lightning/latest/reference/algorithm/#__codelineno-0-7) '42'` #### `__call__(source)` [¶](https://microsoft.github.io/agent-lightning/latest/reference/algorithm/#agentlightning.Adapter.__call__ "Permanent link") Convert the data to the target format. This method delegates to [`adapt()`](https://microsoft.github.io/agent-lightning/latest/reference/algorithm/#agentlightning.Adapter.adapt " adapt(source)") so that an instance of [`Adapter`](https://microsoft.github.io/agent-lightning/latest/reference/algorithm/#agentlightning.Adapter " agentlightning.Adapter") can be used like a standard function. Parameters: * **`source`** (`T_from`) – Input data in the source format. Returns: * `T_to` – Data converted to the target format. #### `adapt(source)` [¶](https://microsoft.github.io/agent-lightning/latest/reference/algorithm/#agentlightning.Adapter.adapt "Permanent link") Convert the data to the target format. Subclasses must override this method with the concrete transformation logic. The base implementation raises `NotImplementedError` to make the requirement explicit. Parameters: * **`source`** (`T_from`) – Input data in the source format. Returns: * `T_to` – Data converted to the target format. ### `agentlightning.TraceAdapter` [¶](https://microsoft.github.io/agent-lightning/latest/reference/algorithm/#agentlightning.TraceAdapter "Permanent link") Bases: `[Adapter](https://microsoft.github.io/agent-lightning/latest/reference/algorithm/#agentlightning.Adapter " agentlightning.Adapter (agentlightning.adapter.base.Adapter)") [Sequence[[Span](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.Span " agentlightning.Span (agentlightning.types.Span)") ], T_to]`, `Generic[T_to]` Base class for adapters that convert trace spans into other formats. This class specializes [`Adapter`](https://microsoft.github.io/agent-lightning/latest/reference/algorithm/#agentlightning.Adapter " agentlightning.Adapter") for working with [`Span`](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.Span " agentlightning.Span") instances emitted by Agent Lightning instrumentation. Subclasses receive entire trace slices and return a format suited for the downstream consumer, for example reinforcement learning training data or observability metrics. ### `agentlightning.OtelTraceAdapter` [¶](https://microsoft.github.io/agent-lightning/latest/reference/algorithm/#agentlightning.OtelTraceAdapter "Permanent link") Bases: `[Adapter](https://microsoft.github.io/agent-lightning/latest/reference/algorithm/#agentlightning.Adapter " agentlightning.Adapter (agentlightning.adapter.base.Adapter)") [Sequence[ReadableSpan], T_to]`, `Generic[T_to]` Base class for adapters that convert OpenTelemetry trace spans into other formats. This specialization of [`Adapter`](https://microsoft.github.io/agent-lightning/latest/reference/algorithm/#agentlightning.Adapter " agentlightning.Adapter") expects a list of `opentelemetry.sdk.trace.ReadableSpan` instances and produces any target format, such as reinforcement learning trajectories, structured logs, or analytics-ready payloads. Examples: `[](https://microsoft.github.io/agent-lightning/latest/reference/algorithm/#__codelineno-0-1) >>> class TraceToDictAdapter(OtelTraceAdapter[dict]): [](https://microsoft.github.io/agent-lightning/latest/reference/algorithm/#__codelineno-0-2) ... def adapt(self, spans: List[ReadableSpan]) -> dict: [](https://microsoft.github.io/agent-lightning/latest/reference/algorithm/#__codelineno-0-3) ... return {"count": len(spans)} [](https://microsoft.github.io/agent-lightning/latest/reference/algorithm/#__codelineno-0-4) ... [](https://microsoft.github.io/agent-lightning/latest/reference/algorithm/#__codelineno-0-5) >>> adapter = TraceToDictAdapter() [](https://microsoft.github.io/agent-lightning/latest/reference/algorithm/#__codelineno-0-6) >>> adapter([span1, span2]) [](https://microsoft.github.io/agent-lightning/latest/reference/algorithm/#__codelineno-0-7) {'count': 2}` ### `agentlightning.TraceToTripletBase` [¶](https://microsoft.github.io/agent-lightning/latest/reference/algorithm/#agentlightning.TraceToTripletBase "Permanent link") Bases: `[TraceAdapter](https://microsoft.github.io/agent-lightning/latest/reference/algorithm/#agentlightning.TraceAdapter " agentlightning.TraceAdapter (agentlightning.adapter.base.TraceAdapter)") [List[[Triplet](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.Triplet " agentlightning.Triplet (agentlightning.types.Triplet)") ]]` Base class for adapters that emit [`Triplet`](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.Triplet " agentlightning.Triplet") trajectories. ### `agentlightning.TracerTraceToTriplet` [¶](https://microsoft.github.io/agent-lightning/latest/reference/algorithm/#agentlightning.TracerTraceToTriplet "Permanent link") Bases: `[TraceToTripletBase](https://microsoft.github.io/agent-lightning/latest/reference/algorithm/#agentlightning.TraceToTripletBase " agentlightning.TraceToTripletBase (agentlightning.adapter.triplet.TraceToTripletBase)") ` Convert tracer-emitted spans into triplet trajectories. Attributes: * **`repair_hierarchy`** – When `True`, repair the span tree using [`TraceTree.repair_hierarchy()`](https://microsoft.github.io/agent-lightning/latest/reference/internal/#agentlightning.adapter.triplet.TraceTree.repair_hierarchy " repair_hierarchy()") before matching calls and rewards. * **`llm_call_match`** – Regular expression pattern that selects LLM call span names. * **`agent_match`** – Optional regular expression pattern for agent span names. When omitted, spans from any agent are considered. * **`exclude_llm_call_in_reward`** – When `True`, ignore matches under reward spans while searching for rewards. * **`reward_match`** – Strategy used to associate rewards with LLM calls. #### `adapt(source)` [¶](https://microsoft.github.io/agent-lightning/latest/reference/algorithm/#agentlightning.TracerTraceToTriplet.adapt "Permanent link") Convert tracer spans into [`Triplet`](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.Triplet " agentlightning.Triplet") trajectories. Parameters: * **`source`** (`Union[Sequence[[Span](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.Span " agentlightning.Span (agentlightning.types.Span)") ], Sequence[ReadableSpan]]`) – Agent Lightning spans or raw OpenTelemetry spans that form a trace. Returns: * `List[[Triplet](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.Triplet " agentlightning.Triplet (agentlightning.types.Triplet)") ]` – Ordered list of trajectory transitions with prompt, response, and reward information. #### `visualize(source, /, filename='trace_tree', interested_span_match=None)` [¶](https://microsoft.github.io/agent-lightning/latest/reference/algorithm/#agentlightning.TracerTraceToTriplet.visualize "Permanent link") Visualize the trace tree built from the supplied spans. Parameters: * **`source`** (`Union[List[[Span](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.Span " agentlightning.Span (agentlightning.types.Span)") ], List[ReadableSpan]]`) – Collection of Agent Lightning [`Span`](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.Span " agentlightning.Span") objects or raw `opentelemetry.sdk.trace.ReadableSpan` instances. * **`filename`** (`str`, default: `'trace_tree'` ) – Base filename for the generated image; `.png` is appended automatically. * **`interested_span_match`** (`str | None`, default: `None` ) – Optional regular expression used to highlight a subset of spans. Returns: * `[TraceTree](https://microsoft.github.io/agent-lightning/latest/reference/internal/#agentlightning.adapter.triplet.TraceTree " agentlightning.adapter.triplet.TraceTree") ` – The [`TraceTree`](https://microsoft.github.io/agent-lightning/latest/reference/internal/#agentlightning.adapter.triplet.TraceTree " agentlightning.adapter.triplet.TraceTree") built from the provided * `[TraceTree](https://microsoft.github.io/agent-lightning/latest/reference/internal/#agentlightning.adapter.triplet.TraceTree " agentlightning.adapter.triplet.TraceTree") ` – spans. ### `agentlightning.LlmProxyTraceToTriplet` [¶](https://microsoft.github.io/agent-lightning/latest/reference/algorithm/#agentlightning.LlmProxyTraceToTriplet "Permanent link") Bases: `[TraceToTripletBase](https://microsoft.github.io/agent-lightning/latest/reference/algorithm/#agentlightning.TraceToTripletBase " agentlightning.TraceToTripletBase (agentlightning.adapter.triplet.TraceToTripletBase)") ` Convert telemetry emitted by the LLM Proxy into triplet trajectories. Warning This adapter is experimental and might be merged with [`TracerTraceToTriplet`](https://microsoft.github.io/agent-lightning/latest/reference/algorithm/#agentlightning.TracerTraceToTriplet " agentlightning.TracerTraceToTriplet") in the future. Danger Do not rely on timestamps when using this adapter. Proxy spans can originate on different machines with unsynchronised clocks, so `sequence_id` is treated as the sole source of ordering. Strategy: 1. Sort spans by `(sequence_id, start_time)` for deterministic processing. 2. Extract token identifiers from `litellm_request` or `raw_gen_ai_request` spans. 3. Extract rewards from spans exposing AgentOps-style payloads or explicit reward spans. 4. Match each reward to the most recent unmatched LLM call whose sequence is smaller. #### `adapt(source)` [¶](https://microsoft.github.io/agent-lightning/latest/reference/algorithm/#agentlightning.LlmProxyTraceToTriplet.adapt "Permanent link") Convert LLM Proxy spans into [`Triplet`](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.Triplet " agentlightning.Triplet") trajectories. Parameters: * **`source`** (`Sequence[[Span](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.Span " agentlightning.Span (agentlightning.types.Span)") ]`) – Spans emitted by the LLM Proxy containing prompt, response, and reward data. Returns: * `List[[Triplet](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.Triplet " agentlightning.Triplet (agentlightning.types.Triplet)") ]` – Ordered trajectory transitions matched purely by `sequence_id`. ### `agentlightning.TraceToMessages` [¶](https://microsoft.github.io/agent-lightning/latest/reference/algorithm/#agentlightning.TraceToMessages "Permanent link") Bases: `[TraceAdapter](https://microsoft.github.io/agent-lightning/latest/reference/algorithm/#agentlightning.TraceAdapter " agentlightning.TraceAdapter (agentlightning.adapter.base.TraceAdapter)") [List[[OpenAIMessages](https://microsoft.github.io/agent-lightning/latest/reference/internal/#agentlightning.adapter.messages.OpenAIMessages " agentlightning.adapter.messages.OpenAIMessages") ]]` Convert trace spans into OpenAI-compatible conversation messages. The adapter reconstructs prompts, completions, tool calls, and function definitions from `gen_ai.*` span attributes. The resulting objects match the JSONL structure expected by the OpenAI fine-tuning pipeline. Warning The adapter assumes all spans share a common trace and that tool call spans are direct children of the associated completion span. #### `adapt(source)` [¶](https://microsoft.github.io/agent-lightning/latest/reference/algorithm/#agentlightning.TraceToMessages.adapt "Permanent link") Transform trace spans into OpenAI chat payloads. Parameters: * **`source`** (`Sequence[[Span](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.Span " agentlightning.Span (agentlightning.types.Span)") ]`) – Spans containing `gen_ai.*` attributes emitted by the tracing pipeline. Returns: * `List[[OpenAIMessages](https://microsoft.github.io/agent-lightning/latest/reference/internal/#agentlightning.adapter.messages.OpenAIMessages " agentlightning.adapter.messages.OpenAIMessages") ]` – A list of [`OpenAIMessages`](https://microsoft.github.io/agent-lightning/latest/reference/internal/#agentlightning.adapter.messages.OpenAIMessages " agentlightning.adapter.messages.OpenAIMessages") entries that * `List[[OpenAIMessages](https://microsoft.github.io/agent-lightning/latest/reference/internal/#agentlightning.adapter.messages.OpenAIMessages " agentlightning.adapter.messages.OpenAIMessages") ]` – capture prompts, completions, tools, and metadata. #### `get_tool_calls(completion, all_spans)` [¶](https://microsoft.github.io/agent-lightning/latest/reference/algorithm/#agentlightning.TraceToMessages.get_tool_calls "Permanent link") Yield tool call payloads for a completion span. Parameters: * **`completion`** (`[Span](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.Span " agentlightning.Span (agentlightning.types.Span)") `) – The completion span whose descendants should be inspected. * **`all_spans`** (`Sequence[[Span](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.Span " agentlightning.Span (agentlightning.types.Span)") ]`) – The complete span list belonging to the trace. Yields: * `Iterable[Dict[str, Any]]` – Dictionaries describing tool calls with identifiers, names, and arguments. Raises: * `ValueError` – If a candidate tool span cannot be converted into a dictionary. LLM Proxy[¶](https://microsoft.github.io/agent-lightning/latest/reference/algorithm/#llm-proxy "Permanent link") ----------------------------------------------------------------------------------------------------------------- ### `agentlightning.LLMProxy` [¶](https://microsoft.github.io/agent-lightning/latest/reference/algorithm/#agentlightning.LLMProxy "Permanent link") Host a LiteLLM OpenAI-compatible proxy bound to a LightningStore. The proxy: * Serves an OpenAI-compatible API via uvicorn. * Adds rollout/attempt routing and headers via middleware. * Registers OTEL export and token-id callbacks. * Writes a LiteLLM worker config file with `model_list` and settings. Lifecycle: * [`start()`](https://microsoft.github.io/agent-lightning/latest/reference/algorithm/#agentlightning.LLMProxy.start " start() async ") writes config, starts uvicorn server in a thread, and waits until ready. * [`stop()`](https://microsoft.github.io/agent-lightning/latest/reference/algorithm/#agentlightning.LLMProxy.stop " stop() async ") tears down the server and removes the temp config file. * [`restart()`](https://microsoft.github.io/agent-lightning/latest/reference/algorithm/#agentlightning.LLMProxy.restart " restart(*, _port=None) async ") convenience wrapper to stop then start. Note As the LLM Proxy sets up an OpenTelemetry tracer, it's recommended to run it in a different process from the main runner (i.e., tracer from agents). See `launch_mode` for how to change that. Warning By default (or when "stream\_conversion" middleware is enabled), the LLM Proxy will convert OpenAI and Anthropic requests with `stream=True` to a non-streaming request before going through the LiteLLM proxy. This is because the OpenTelemetry tracer provided by LiteLLM is buggy with streaming responses. You can disable this by removing the "stream\_conversion" middleware. In that case, you might lose some tracing information like token IDs. Danger Do not run LLM proxy in the same process as the main runner. It's easy to cause conflicts in the tracer provider with tracers like [`AgentOpsTracer`](https://microsoft.github.io/agent-lightning/latest/reference/runner/#agentlightning.AgentOpsTracer " agentlightning.AgentOpsTracer") . Parameters: * **`port`** (`int | None`, default: `None` ) – TCP port to bind. Will bind to a random port if not provided. * **`model_list`** (`List[[ModelConfig](https://microsoft.github.io/agent-lightning/latest/reference/internal/#agentlightning.llm_proxy.ModelConfig " agentlightning.llm_proxy.ModelConfig") ] | None`, default: `None` ) – LiteLLM `model_list` entries. * **`store`** (`Optional[[LightningStore](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.LightningStore " agentlightning.LightningStore (agentlightning.store.base.LightningStore)") ]`, default: `None` ) – LightningStore used for span sequence and persistence. * **`host`** (`str | None`, default: `None` ) – Publicly reachable host used in resource endpoints. See `host` of `launcher_args` for more details. * **`litellm_config`** (`Dict[str, Any] | None`, default: `None` ) – Extra LiteLLM proxy config merged with `model_list`. * **`num_retries`** (`int`, default: `0` ) – Default LiteLLM retry count injected into `litellm_settings`. * **`num_workers`** (`int`, default: `1` ) – Number of workers to run in the server. Only applicable for "mp" launch mode. Ignored if launcher\_args is provided. When `num_workers > 1`, the server will be run using [gunicorn](https://gunicorn.org/) . * **`launch_mode`** (`[LaunchMode](https://microsoft.github.io/agent-lightning/latest/reference/utilities/#agentlightning.utils.server_launcher.LaunchMode " agentlightning.utils.server_launcher.LaunchMode = Literal['asyncio', 'thread', 'mp'] module-attribute (agentlightning.utils.server_launcher.LaunchMode)") `, default: `'mp'` ) – Launch mode for the server. Defaults to "mp". Cannot be used together with launcher\_args. Ignored if launcher\_args is provided. It's recommended to use `launch_mode="mp"` to launch the proxy, which will launch the server in a separate process. `launch_mode="thread"` can also be used if used in caution. It will launch the server in a separate thread. `launch_mode="asyncio"` launches the server in the current thread as an asyncio task. It is NOT recommended because it often causes hanging requests. Only use it if you know what you are doing. * **`launcher_args`** (`[PythonServerLauncherArgs](https://microsoft.github.io/agent-lightning/latest/reference/utilities/#agentlightning.utils.server_launcher.PythonServerLauncherArgs " agentlightning.utils.server_launcher.PythonServerLauncherArgs dataclass ") | None`, default: `None` ) – Arguments for the server launcher. If this is provided, host, port, and launch\_mode will be ignored. Cannot be used together with port, host, and launch\_mode. * **`middlewares`** (`Sequence[Union[Type[BaseHTTPMiddleware], str]] | None`, default: `None` ) – List of FastAPI middleware classes or strings to register. You can specify the class aliases or classes that have been imported. If not provided, the default middlewares (RolloutAttemptMiddleware and StreamConversionMiddleware) will be used. Available middleware aliases are: "rollout\_attempt", "stream\_conversion", "message\_inspection". Middlewares are the **first layer** of request processing. They are applied to all requests before the LiteLLM proxy. * **`callbacks`** (`Sequence[Union[Type[CustomLogger], str]] | None`, default: `None` ) – List of LiteLLM callback classes or strings to register. You can specify the class aliases or classes that have been imported. If not provided, the default callbacks (AddReturnTokenIds and LightningOpenTelemetry) will be used. Available callback aliases are: "return\_token\_ids", "opentelemetry", "logprobs". #### `as_resource(rollout_id=None, attempt_id=None, model=None, sampling_parameters=None)` [¶](https://microsoft.github.io/agent-lightning/latest/reference/algorithm/#agentlightning.LLMProxy.as_resource "Permanent link") Create an `LLM` resource pointing at this proxy with rollout context. The returned endpoint is `http://{host}:{port}/rollout/{rollout_id}/attempt/{attempt_id}` Parameters: * **`rollout_id`** (`str | None`, default: `None` ) – Rollout identifier used for span attribution. If None, will instantiate a ProxyLLM resource. * **`attempt_id`** (`str | None`, default: `None` ) – Attempt identifier used for span attribution. If None, will instantiate a ProxyLLM resource. * **`model`** (`str | None`, default: `None` ) – Logical model name to use. If omitted and exactly one model is configured or all models have the same name, that model is used. * **`sampling_parameters`** (`Dict[str, Any] | None`, default: `None` ) – Optional default sampling parameters. Returns: * **`LLM`** ( `[LLM](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.LLM " agentlightning.LLM (agentlightning.types.LLM)") ` ) – Configured resource ready for OpenAI-compatible calls. Raises: * `ValueError` – If `model` is omitted and zero or multiple models are configured. #### `get_store()` [¶](https://microsoft.github.io/agent-lightning/latest/reference/algorithm/#agentlightning.LLMProxy.get_store "Permanent link") Get the store used by the proxy. Returns: * `Optional[[LightningStore](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.LightningStore " agentlightning.LightningStore (agentlightning.store.base.LightningStore)") ]` – The store used by the proxy. #### `initialize()` [¶](https://microsoft.github.io/agent-lightning/latest/reference/algorithm/#agentlightning.LLMProxy.initialize "Permanent link") Initialize global middleware and LiteLLM callbacks. Installs: * A FastAPI middleware that rewrites /rollout/{rid}/attempt/{aid}/... paths, injects rollout/attempt/sequence headers, and forwards downstream. * LiteLLM callbacks for token ids and OpenTelemetry export. The middleware can only be installed once because once the FastAPI app has started, the middleware cannot be changed any more. This function does not start any server. It only wires global hooks. #### `is_running()` [¶](https://microsoft.github.io/agent-lightning/latest/reference/algorithm/#agentlightning.LLMProxy.is_running "Permanent link") Return whether the uvicorn server is active. Returns: * **`bool`** ( `bool` ) – True if server was started and did not signal exit. #### `restart(*, _port=None)` `async` [¶](https://microsoft.github.io/agent-lightning/latest/reference/algorithm/#agentlightning.LLMProxy.restart "Permanent link") Restart the proxy if running, else start it. Convenience wrapper calling `stop()` followed by `start()`. #### `set_store(store)` [¶](https://microsoft.github.io/agent-lightning/latest/reference/algorithm/#agentlightning.LLMProxy.set_store "Permanent link") Set the store for the proxy. Parameters: * **`store`** (`[LightningStore](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.LightningStore " agentlightning.LightningStore (agentlightning.store.base.LightningStore)") `) – The store to use for the proxy. #### `start()` `async` [¶](https://microsoft.github.io/agent-lightning/latest/reference/algorithm/#agentlightning.LLMProxy.start "Permanent link") Start the proxy server thread and initialize global wiring. Side effects: * Sets the module-level global store for middleware/exporter access. * Calls `initialize()` once to register middleware and callbacks. * Writes a temporary YAML config consumed by LiteLLM worker. * Launches uvicorn in a daemon thread and waits for readiness. #### `stop()` `async` [¶](https://microsoft.github.io/agent-lightning/latest/reference/algorithm/#agentlightning.LLMProxy.stop "Permanent link") Stop the proxy server and clean up temporary artifacts. This is a best-effort graceful shutdown with a bounded join timeout. #### `update_model_list(model_list)` [¶](https://microsoft.github.io/agent-lightning/latest/reference/algorithm/#agentlightning.LLMProxy.update_model_list "Permanent link") Replace the in-memory model list. Parameters: * **`model_list`** (`List[[ModelConfig](https://microsoft.github.io/agent-lightning/latest/reference/internal/#agentlightning.llm_proxy.ModelConfig " agentlightning.llm_proxy.ModelConfig") ]`) – New list of model entries. Back to top --- # Internal - Agent-lightning [Skip to content](https://microsoft.github.io/agent-lightning/latest/reference/internal/#internal-api-references) ⚠️ You are viewing development documentation. For stable documentation, please visit [https://microsoft.github.io/agent-lightning/stable/](https://microsoft.github.io/agent-lightning/stable/) Internal API References[¶](https://microsoft.github.io/agent-lightning/latest/reference/internal/#internal-api-references "Permanent link") ============================================================================================================================================ Danger The following APIs should be used with extra caution because they are very likely to change in the future. Algorithms and Adapters[¶](https://microsoft.github.io/agent-lightning/latest/reference/internal/#algorithms-and-adapters "Permanent link") -------------------------------------------------------------------------------------------------------------------------------------------- ### `agentlightning.adapter.messages.OpenAIMessages` [¶](https://microsoft.github.io/agent-lightning/latest/reference/internal/#agentlightning.adapter.messages.OpenAIMessages "Permanent link") Bases: `TypedDict` OpenAI-style chat messages with optional tool definitions. Attributes: * **`messages`** (`List[ChatCompletionMessageParam]`) – Ordered chat messages that describe the conversation. * **`tools`** (`Optional[List[ChatCompletionFunctionToolParam]]`) – Tool specifications available to the assistant, if any. ### `agentlightning.adapter.triplet.TraceTree` [¶](https://microsoft.github.io/agent-lightning/latest/reference/internal/#agentlightning.adapter.triplet.TraceTree "Permanent link") Tree representation of a trace span and its descendants. Attributes: * **`id`** – Unique identifier for the span node. * **`span`** – [`Span`](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.Span " agentlightning.Span") backing this node. * **`children`** – Child nodes connected to the current span. #### `agent_name()` [¶](https://microsoft.github.io/agent-lightning/latest/reference/internal/#agentlightning.adapter.triplet.TraceTree.agent_name "Permanent link") Return the agent name associated with the span, if any. Returns: * `Optional[str]` – Agent name extracted from known attributes, otherwise `None`. #### `extract_prompt_image_urls(prompt_raw_content)` [¶](https://microsoft.github.io/agent-lightning/latest/reference/internal/#agentlightning.adapter.triplet.TraceTree.extract_prompt_image_urls "Permanent link") Extract image URLs from the span attributes, in order of appearance. Parameters: * **`prompt_raw_content`** (`Any`) – The raw content of the prompt, which can be in one of several formats: * List\[dict\]: A list of message entries, each being a dict with at least a "content" key. * Dict\[str, Any\]: A dictionary, often with numeric string keys (e.g., `{"0": {...}, "1": {...}}`), where each value is a message entry. If the dict does not have numeric keys, it is treated as a single message entry. #### `find_llm_calls(*, llm_call_match, agent_match, within_matching_subtree=None, within_reward=None, within_llm_call=None, existing_llm_call_response_ids=None)` [¶](https://microsoft.github.io/agent-lightning/latest/reference/internal/#agentlightning.adapter.triplet.TraceTree.find_llm_calls "Permanent link") Find LLM call spans matching the supplied filters. Parameters: * **`llm_call_match`** (`str`) – Regular expression used to match span names that qualify as LLM calls. * **`agent_match`** (`Optional[str]`) – Optional regular expression that must match the enclosing agent span name. * **`within_matching_subtree`** (`str | None`, default: `None` ) – Marker propagated through recursive calls to record matching agents. * **`within_reward`** (`Optional[bool]`, default: `None` ) – When `True`, suppresses LLM matches under reward spans. * **`within_llm_call`** (`Optional[bool]`, default: `None` ) – When `True`, prevents duplicate matches for nested LLM calls. * **`existing_llm_call_response_ids`** (`Optional[set[str]]`, default: `None` ) – Known response identifiers used to deduplicate spans. Returns: * `List[Tuple['TraceTree', str]]` – A list of tuples pairing the matching node with the agent subtree label that triggered the * `List[Tuple['TraceTree', str]]` – match. #### `from_spans(spans)` `classmethod` [¶](https://microsoft.github.io/agent-lightning/latest/reference/internal/#agentlightning.adapter.triplet.TraceTree.from_spans "Permanent link") Construct a tree from a flat list of spans. Parameters: * **`spans`** (`List[[Span](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.Span " agentlightning.Span (agentlightning.types.Span)") ]`) – Spans that collectively form a single trace segment. Returns: * `'TraceTree'` – A [`TraceTree`](https://microsoft.github.io/agent-lightning/latest/reference/internal/#agentlightning.adapter.triplet.TraceTree " agentlightning.adapter.triplet.TraceTree") rooted at either the * `'TraceTree'` – discovered root span or a synthetic root when multiple roots are present. Raises: * `ValueError` – If the span list is empty or no root span can be inferred. #### `is_reward_span()` [¶](https://microsoft.github.io/agent-lightning/latest/reference/internal/#agentlightning.adapter.triplet.TraceTree.is_reward_span "Permanent link") Return whether the span explicitly encodes a reward. Returns: * `bool` – `True` when the span payload describes a reward, otherwise `False`. #### `match_rewards(reward_match, llm_calls)` [¶](https://microsoft.github.io/agent-lightning/latest/reference/internal/#agentlightning.adapter.triplet.TraceTree.match_rewards "Permanent link") Assign rewards to previously matched LLM calls. Parameters: * **`reward_match`** (`str`) – Strategy identifier from [`RewardMatchPolicy`](https://microsoft.github.io/agent-lightning/latest/reference/internal/#agentlightning.adapter.triplet.RewardMatchPolicy " agentlightning.adapter.triplet.RewardMatchPolicy") . * **`llm_calls`** (`List['TraceTree']`) – Trace nodes representing LLM call spans. Returns: * `dict[str, Optional[float]]` – Mapping from span identifier to reward value or `None` when no reward is available. #### `maybe_reward_dict()` [¶](https://microsoft.github.io/agent-lightning/latest/reference/internal/#agentlightning.adapter.triplet.TraceTree.maybe_reward_dict "Permanent link") Return a reward payload if the span encodes one. Returns: * `dict[str, Any]` – Dictionary containing reward metadata, or an empty dictionary when no reward is found. #### `names_tuple()` [¶](https://microsoft.github.io/agent-lightning/latest/reference/internal/#agentlightning.adapter.triplet.TraceTree.names_tuple "Permanent link") Return the span name alongside nested child names. Returns: * `str` – A tuple of the current span name and a list of tuples for each child containing the * `List[Any]` – child name and its descendants. #### `repair_hierarchy()` [¶](https://microsoft.github.io/agent-lightning/latest/reference/internal/#agentlightning.adapter.triplet.TraceTree.repair_hierarchy "Permanent link") Repair missing parent-child relationships introduced by mixed tracing systems. Some agent frameworks emit spans via multiple subsystems, which can cause LLM completion spans to float directly under the root span instead of being nested under the correct agent. The method re-parents those spans to the closest ancestor that fully envelopes the child in time. If we don't, when we want to select the LLM completion span with agent as filter. We will never get the correct span underneath. #### `span_to_triplet(span, agent_name)` [¶](https://microsoft.github.io/agent-lightning/latest/reference/internal/#agentlightning.adapter.triplet.TraceTree.span_to_triplet "Permanent link") Convert a span to a triplet. Subclass can override this method to add more fields to the triplet, such as chat messages and tool calls. #### `to_json()` [¶](https://microsoft.github.io/agent-lightning/latest/reference/internal/#agentlightning.adapter.triplet.TraceTree.to_json "Permanent link") Convert the tree node into a JSON-serialisable structure. #### `to_trajectory(llm_call_match='openai\\.chat\\.completion', agent_match=None, exclude_llm_call_in_reward=True, dedup_llm_call=True, reward_match=RewardMatchPolicy.FIRST_OCCURRENCE, final_reward=None, _skip_empty_token_spans=False)` [¶](https://microsoft.github.io/agent-lightning/latest/reference/internal/#agentlightning.adapter.triplet.TraceTree.to_trajectory "Permanent link") Convert the trace tree into a trajectory of [`Triplet`](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.Triplet " agentlightning.Triplet") items. Parameters: * **`llm_call_match`** (`str`, default: `'openai\\.chat\\.completion'` ) – Regular expression for LLM call span names. * **`agent_match`** (`Optional[str]`, default: `None` ) – Optional regular expression for agent span names. * **`exclude_llm_call_in_reward`** (`bool`, default: `True` ) – When `True`, prevents searching for rewards under the LLM call subtree. * **`dedup_llm_call`** (`bool`, default: `True` ) – When `True`, deduplicates spans using the LLM response identifier. * **`reward_match`** (`[RewardMatchPolicy](https://microsoft.github.io/agent-lightning/latest/reference/internal/#agentlightning.adapter.triplet.RewardMatchPolicy " agentlightning.adapter.triplet.RewardMatchPolicy") `, default: `[FIRST_OCCURRENCE](https://microsoft.github.io/agent-lightning/latest/reference/internal/#agentlightning.adapter.triplet.RewardMatchPolicy.FIRST_OCCURRENCE " FIRST_OCCURRENCE = 'first_occurrence' class-attribute instance-attribute (agentlightning.adapter.triplet.RewardMatchPolicy.FIRST_OCCURRENCE)") ` ) – Reward matching policy used to associate reward spans with LLM calls. * **`final_reward`** (`Optional[float]`, default: `None` ) – Optional reward appended to the final transition when provided. Returns: * `List[[Triplet](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.Triplet " agentlightning.Triplet (agentlightning.types.Triplet)") ]` – A list of [`Triplet`](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.Triplet " agentlightning.Triplet") objects ordered by call sequence. #### `traverse()` [¶](https://microsoft.github.io/agent-lightning/latest/reference/internal/#agentlightning.adapter.triplet.TraceTree.traverse "Permanent link") Traverse the tree depth first and return every node. #### `visualize(filename, interested_span_match=None)` [¶](https://microsoft.github.io/agent-lightning/latest/reference/internal/#agentlightning.adapter.triplet.TraceTree.visualize "Permanent link") Render the trace tree with Graphviz for debugging purposes. Parameters: * **`filename`** (`str`) – Base filename for the generated `.png` diagram. * **`interested_span_match`** (`str | None`, default: `None` ) – Optional regular expression used to keep only matching spans (and their ancestors) in the output. Note The method requires the optional `graphviz` dependency to be available in the runtime environment. ### `agentlightning.adapter.triplet.Transition` [¶](https://microsoft.github.io/agent-lightning/latest/reference/internal/#agentlightning.adapter.triplet.Transition "Permanent link") Bases: `BaseModel` A single transition within a reinforcement learning trajectory. Attributes: * **`state`** (`List[int]`) – Token identifiers describing the model input state. * **`action`** (`List[int]`) – Token identifiers representing the model output. * **`response_id`** (`Optional[str]`) – Identifier of the LLM response used to deduplicate spans. * **`agent_name`** (`str`) – Human-readable agent name captured from the trace. * **`reward`** (`Optional[float]`) – Scalar reward associated with the transition, if available. ### `agentlightning.adapter.triplet.RewardMatchPolicy` [¶](https://microsoft.github.io/agent-lightning/latest/reference/internal/#agentlightning.adapter.triplet.RewardMatchPolicy "Permanent link") Bases: `str`, `Enum` Strategies for matching rewards to LLM call spans. Note Each reward span must expose a payload shaped like `{"type": "reward", "value": |None}` as described in `reward.py`. #### `FIRST_OCCURRENCE = 'first_occurrence'` `class-attribute` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/latest/reference/internal/#agentlightning.adapter.triplet.RewardMatchPolicy.FIRST_OCCURRENCE "Permanent link") Use the first reward encountered in chronological order after the current LLM call match. #### `FIRST_SIBLING = 'first_sibling'` `class-attribute` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/latest/reference/internal/#agentlightning.adapter.triplet.RewardMatchPolicy.FIRST_SIBLING "Permanent link") Use the first sibling in the current trace subtree as the reward unless another LLM call match is found. ### `agentlightning.algorithm.decorator.FunctionalAlgorithm` [¶](https://microsoft.github.io/agent-lightning/latest/reference/internal/#agentlightning.algorithm.decorator.FunctionalAlgorithm "Permanent link") Bases: `[Algorithm](https://microsoft.github.io/agent-lightning/latest/reference/algorithm/#agentlightning.Algorithm " agentlightning.Algorithm (agentlightning.algorithm.base.Algorithm)") `, `Generic[AF]` An algorithm wrapper built from a callable implementation. Functional algorithms let you provide an ordinary function instead of subclassing [`Algorithm`](https://microsoft.github.io/agent-lightning/latest/reference/algorithm/#agentlightning.Algorithm " agentlightning.Algorithm") . The wrapper inspects the callable signature to supply optional dependencies such as the store, adapter, and LLM proxy. #### `__init__(algorithm_func)` [¶](https://microsoft.github.io/agent-lightning/latest/reference/internal/#agentlightning.algorithm.decorator.FunctionalAlgorithm.__init__ "Permanent link") `[](https://microsoft.github.io/agent-lightning/latest/reference/internal/#__codelineno-0-1) __init__(algorithm_func: AlgorithmFuncSyncLike) -> None` `[](https://microsoft.github.io/agent-lightning/latest/reference/internal/#__codelineno-0-1) __init__(algorithm_func: AlgorithmFuncAsyncLike) -> None` Wrap a function that implements algorithm behaviour. Parameters: * **`algorithm_func`** (`Union[AlgorithmFuncSyncLike, AlgorithmFuncAsyncLike]`) – Sync or async callable implementing the algorithm contract. Arguments are detected automatically based on the function signature. #### `run(train_dataset=None, val_dataset=None)` [¶](https://microsoft.github.io/agent-lightning/latest/reference/internal/#agentlightning.algorithm.decorator.FunctionalAlgorithm.run "Permanent link") `[](https://microsoft.github.io/agent-lightning/latest/reference/internal/#__codelineno-0-1) run( [](https://microsoft.github.io/agent-lightning/latest/reference/internal/#__codelineno-0-2) train_dataset: Optional[Dataset[Any]] = None, [](https://microsoft.github.io/agent-lightning/latest/reference/internal/#__codelineno-0-3) val_dataset: Optional[Dataset[Any]] = None, [](https://microsoft.github.io/agent-lightning/latest/reference/internal/#__codelineno-0-4) ) -> None` `[](https://microsoft.github.io/agent-lightning/latest/reference/internal/#__codelineno-0-1) run( [](https://microsoft.github.io/agent-lightning/latest/reference/internal/#__codelineno-0-2) train_dataset: Optional[Dataset[Any]] = None, [](https://microsoft.github.io/agent-lightning/latest/reference/internal/#__codelineno-0-3) val_dataset: Optional[Dataset[Any]] = None, [](https://microsoft.github.io/agent-lightning/latest/reference/internal/#__codelineno-0-4) ) -> Awaitable[None]` Execute the wrapped function with injected dependencies. Parameters: * **`train_dataset`** (`Optional[[Dataset](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.Dataset " agentlightning.Dataset (agentlightning.types.Dataset)") [Any]]`, default: `None` ) – Optional training dataset passed through when the callable declares a `train_dataset` parameter. * **`val_dataset`** (`Optional[[Dataset](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.Dataset " agentlightning.Dataset (agentlightning.types.Dataset)") [Any]]`, default: `None` ) – Optional validation dataset passed through when the callable declares a `val_dataset` parameter. Returns: * `Union[None, Awaitable[None]]` – None for sync callables or an awaitable when the callable is async. Raises: * `TypeError` – If a dataset is provided but the function signature does not accept the corresponding argument. LitAgent[¶](https://microsoft.github.io/agent-lightning/latest/reference/internal/#litagent "Permanent link") -------------------------------------------------------------------------------------------------------------- ### `agentlightning.litagent.decorator.FunctionalLitAgent` [¶](https://microsoft.github.io/agent-lightning/latest/reference/internal/#agentlightning.litagent.decorator.FunctionalLitAgent "Permanent link") Bases: `[LitAgent](https://microsoft.github.io/agent-lightning/latest/reference/agent/#agentlightning.LitAgent " agentlightning.LitAgent (agentlightning.litagent.litagent.LitAgent)") [T]` Adapter that turns plain rollout functions into [`LitAgent`](https://microsoft.github.io/agent-lightning/latest/reference/agent/#agentlightning.LitAgent " agentlightning.LitAgent") instances. The helper inspects the wrapped function to determine which resources to inject, allowing both synchronous and asynchronous callables to participate in the training loop without writing a dedicated subclass. #### `__call__(*args, **kwargs)` [¶](https://microsoft.github.io/agent-lightning/latest/reference/internal/#agentlightning.litagent.decorator.FunctionalLitAgent.__call__ "Permanent link") Make the agent instance callable, preserving the original function behavior. #### `__init__(rollout_func, *, strip_proxy=True)` [¶](https://microsoft.github.io/agent-lightning/latest/reference/internal/#agentlightning.litagent.decorator.FunctionalLitAgent.__init__ "Permanent link") Initialize the wrapper around a rollout function. Parameters: * **`rollout_func`** (`FunctionalLitAgentFunc[T]`) – Callable that implements the rollout. It may be synchronous or asynchronous and can optionally receive a [`Rollout`](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.Rollout " agentlightning.Rollout") alongside resources such as `llm` or `prompt_template`. * **`strip_proxy`** (`bool`, default: `True` ) – When `True`, convert [`ProxyLLM`](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.ProxyLLM " agentlightning.ProxyLLM") inputs into [`LLM`](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.LLM " agentlightning.LLM") instances before calling the rollout function. Defaults to `True`. #### `rollout(task, resources, rollout)` [¶](https://microsoft.github.io/agent-lightning/latest/reference/internal/#agentlightning.litagent.decorator.FunctionalLitAgent.rollout "Permanent link") Execute a synchronous rollout using the wrapped function. Parameters: * **`task`** (`T`) – Task input data. * **`resources`** (`[NamedResources](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.NamedResources " agentlightning.NamedResources = Dict[str, ResourceUnion] module-attribute (agentlightning.types.NamedResources)") `) – Mapping of named resources available to the agent. * **`rollout`** (`[Rollout](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.Rollout " agentlightning.Rollout (agentlightning.types.Rollout)") `) – Rollout metadata provided by the runtime. Returns: * `[RolloutRawResult](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.RolloutRawResult " agentlightning.RolloutRawResult = Union[None, float, List[ReadableSpan], List[Span], List[SpanCoreFields]] module-attribute (agentlightning.types.RolloutRawResult)") ` – Result produced by the wrapped rollout function. Raises: * `RuntimeError` – If the wrapped function is asynchronous. #### `rollout_async(task, resources, rollout)` `async` [¶](https://microsoft.github.io/agent-lightning/latest/reference/internal/#agentlightning.litagent.decorator.FunctionalLitAgent.rollout_async "Permanent link") Execute an asynchronous rollout using the wrapped function. Parameters: * **`task`** (`T`) – Task input data. * **`resources`** (`[NamedResources](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.NamedResources " agentlightning.NamedResources = Dict[str, ResourceUnion] module-attribute (agentlightning.types.NamedResources)") `) – Mapping of named resources available to the agent. * **`rollout`** (`[Rollout](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.Rollout " agentlightning.Rollout (agentlightning.types.Rollout)") `) – Rollout metadata provided by the runtime. Returns: * `[RolloutRawResult](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.RolloutRawResult " agentlightning.RolloutRawResult = Union[None, float, List[ReadableSpan], List[Span], List[SpanCoreFields]] module-attribute (agentlightning.types.RolloutRawResult)") ` – Result produced by the wrapped rollout coroutine. Raises: * `RuntimeError` – If the wrapped function is synchronous. ### `agentlightning.litagent.decorator.llm_rollout(func=None, *, strip_proxy=True)` [¶](https://microsoft.github.io/agent-lightning/latest/reference/internal/#agentlightning.litagent.decorator.llm_rollout "Permanent link") `[](https://microsoft.github.io/agent-lightning/latest/reference/internal/#__codelineno-0-1) llm_rollout( [](https://microsoft.github.io/agent-lightning/latest/reference/internal/#__codelineno-0-2) func: LlmRolloutFunc[T], [](https://microsoft.github.io/agent-lightning/latest/reference/internal/#__codelineno-0-3) ) -> FunctionalLitAgent[T]` `[](https://microsoft.github.io/agent-lightning/latest/reference/internal/#__codelineno-0-1) llm_rollout( [](https://microsoft.github.io/agent-lightning/latest/reference/internal/#__codelineno-0-2) *, strip_proxy: bool = True [](https://microsoft.github.io/agent-lightning/latest/reference/internal/#__codelineno-0-3) ) -> Callable[[LlmRolloutFunc[T]], FunctionalLitAgent[T]]` Create a [`FunctionalLitAgent`](https://microsoft.github.io/agent-lightning/latest/reference/internal/#agentlightning.litagent.decorator.FunctionalLitAgent " agentlightning.litagent.decorator.FunctionalLitAgent") for LLM-based rollouts. Parameters: * **`func`** (`LlmRolloutFunc[T] | None`, default: `None` ) – Callable defining the agent's behaviour. Supported signatures include: * `(task, llm) -> result` * `(task, llm, rollout) -> result` * `async (task, llm) -> result` * `async (task, llm, rollout) -> result` * **`strip_proxy`** (`bool`, default: `True` ) – When `True`, convert proxy resources into concrete [`LLM`](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.LLM " agentlightning.LLM") instances before calling the function. Defaults to `True`. Returns: * `[FunctionalLitAgent](https://microsoft.github.io/agent-lightning/latest/reference/internal/#agentlightning.litagent.decorator.FunctionalLitAgent " agentlightning.litagent.decorator.FunctionalLitAgent") [T] | Callable[[LlmRolloutFunc[T]], [FunctionalLitAgent](https://microsoft.github.io/agent-lightning/latest/reference/internal/#agentlightning.litagent.decorator.FunctionalLitAgent " agentlightning.litagent.decorator.FunctionalLitAgent") [T]]` – [`FunctionalLitAgent`](https://microsoft.github.io/agent-lightning/latest/reference/internal/#agentlightning.litagent.decorator.FunctionalLitAgent " agentlightning.litagent.decorator.FunctionalLitAgent") that * `[FunctionalLitAgent](https://microsoft.github.io/agent-lightning/latest/reference/internal/#agentlightning.litagent.decorator.FunctionalLitAgent " agentlightning.litagent.decorator.FunctionalLitAgent") [T] | Callable[[LlmRolloutFunc[T]], [FunctionalLitAgent](https://microsoft.github.io/agent-lightning/latest/reference/internal/#agentlightning.litagent.decorator.FunctionalLitAgent " agentlightning.litagent.decorator.FunctionalLitAgent") [T]]` – wraps the supplied function. Examples: `[](https://microsoft.github.io/agent-lightning/latest/reference/internal/#__codelineno-0-1) @llm_rollout [](https://microsoft.github.io/agent-lightning/latest/reference/internal/#__codelineno-0-2) def my_agent(task, llm): [](https://microsoft.github.io/agent-lightning/latest/reference/internal/#__codelineno-0-3) return llm.endpoint [](https://microsoft.github.io/agent-lightning/latest/reference/internal/#__codelineno-0-4) [](https://microsoft.github.io/agent-lightning/latest/reference/internal/#__codelineno-0-5) @llm_rollout(strip_proxy=False) [](https://microsoft.github.io/agent-lightning/latest/reference/internal/#__codelineno-0-6) def my_agent_no_strip(task, llm): [](https://microsoft.github.io/agent-lightning/latest/reference/internal/#__codelineno-0-7) return llm.model [](https://microsoft.github.io/agent-lightning/latest/reference/internal/#__codelineno-0-8) [](https://microsoft.github.io/agent-lightning/latest/reference/internal/#__codelineno-0-9) result = my_agent(task, llm) [](https://microsoft.github.io/agent-lightning/latest/reference/internal/#__codelineno-0-10) result = my_agent.rollout(task, resources, rollout)` ### `agentlightning.litagent.decorator.prompt_rollout(func=None)` [¶](https://microsoft.github.io/agent-lightning/latest/reference/internal/#agentlightning.litagent.decorator.prompt_rollout "Permanent link") `[](https://microsoft.github.io/agent-lightning/latest/reference/internal/#__codelineno-0-1) prompt_rollout( [](https://microsoft.github.io/agent-lightning/latest/reference/internal/#__codelineno-0-2) func: PromptRolloutFunc[T], [](https://microsoft.github.io/agent-lightning/latest/reference/internal/#__codelineno-0-3) ) -> FunctionalLitAgent[T]` `[](https://microsoft.github.io/agent-lightning/latest/reference/internal/#__codelineno-0-1) prompt_rollout() -> ( [](https://microsoft.github.io/agent-lightning/latest/reference/internal/#__codelineno-0-2) Callable[[PromptRolloutFunc[T]], FunctionalLitAgent[T]] [](https://microsoft.github.io/agent-lightning/latest/reference/internal/#__codelineno-0-3) )` Create a [`FunctionalLitAgent`](https://microsoft.github.io/agent-lightning/latest/reference/internal/#agentlightning.litagent.decorator.FunctionalLitAgent " agentlightning.litagent.decorator.FunctionalLitAgent") for prompt-based rollouts. This decorator is designed for agents that work with tunable prompt templates. It enables a workflow where algorithms manage and optimize the prompt template, while agents consume the template to perform rollouts. This is particularly useful for prompt optimization scenarios. Parameters: * **`func`** (`PromptRolloutFunc[T] | None`, default: `None` ) – Callable defining the agent's behavior. Supported signatures include: * `(task, prompt_template) -> result` * `(task, prompt_template, rollout) -> result` * `async (task, prompt_template) -> result` * `async (task, prompt_template, rollout) -> result` Returns: * `[FunctionalLitAgent](https://microsoft.github.io/agent-lightning/latest/reference/internal/#agentlightning.litagent.decorator.FunctionalLitAgent " agentlightning.litagent.decorator.FunctionalLitAgent") [T] | Callable[[PromptRolloutFunc[T]], [FunctionalLitAgent](https://microsoft.github.io/agent-lightning/latest/reference/internal/#agentlightning.litagent.decorator.FunctionalLitAgent " agentlightning.litagent.decorator.FunctionalLitAgent") [T]]` – [`FunctionalLitAgent`](https://microsoft.github.io/agent-lightning/latest/reference/internal/#agentlightning.litagent.decorator.FunctionalLitAgent " agentlightning.litagent.decorator.FunctionalLitAgent") that * `[FunctionalLitAgent](https://microsoft.github.io/agent-lightning/latest/reference/internal/#agentlightning.litagent.decorator.FunctionalLitAgent " agentlightning.litagent.decorator.FunctionalLitAgent") [T] | Callable[[PromptRolloutFunc[T]], [FunctionalLitAgent](https://microsoft.github.io/agent-lightning/latest/reference/internal/#agentlightning.litagent.decorator.FunctionalLitAgent " agentlightning.litagent.decorator.FunctionalLitAgent") [T]]` – wraps the supplied function. Examples: `[](https://microsoft.github.io/agent-lightning/latest/reference/internal/#__codelineno-0-1) @prompt_rollout [](https://microsoft.github.io/agent-lightning/latest/reference/internal/#__codelineno-0-2) def my_agent(task, prompt_template): [](https://microsoft.github.io/agent-lightning/latest/reference/internal/#__codelineno-0-3) messages = prompt_template.format(task=task.input) [](https://microsoft.github.io/agent-lightning/latest/reference/internal/#__codelineno-0-4) return messages [](https://microsoft.github.io/agent-lightning/latest/reference/internal/#__codelineno-0-5) [](https://microsoft.github.io/agent-lightning/latest/reference/internal/#__codelineno-0-6) result = my_agent(task, prompt_template) [](https://microsoft.github.io/agent-lightning/latest/reference/internal/#__codelineno-0-7) result = my_agent.rollout(task, resources, rollout)` ### `agentlightning.emitter.annotation.OperationContext` [¶](https://microsoft.github.io/agent-lightning/latest/reference/internal/#agentlightning.emitter.annotation.OperationContext "Permanent link") Context manager and decorator for tracing operations. This class manages a tracer-backed span for a logical unit of work. It can be used either: * As a decorator, in which case inputs and outputs are inferred automatically from the wrapped function's signature. * As a context manager, in which case inputs and outputs can be recorded explicitly via [`set_input`](https://microsoft.github.io/agent-lightning/latest/reference/internal/#agentlightning.emitter.annotation.OperationContext.set_input " set_input(*args, **kwargs)") and [`set_output`](https://microsoft.github.io/agent-lightning/latest/reference/internal/#agentlightning.emitter.annotation.OperationContext.set_output " set_output(output)") . Attributes: * **`name`** – Human-readable span name. * **`initial_attributes`** – Attributes applied when the span is created. * **`tracer`** – Tracer implementation used to create spans. #### `__call__(fn)` [¶](https://microsoft.github.io/agent-lightning/latest/reference/internal/#agentlightning.emitter.annotation.OperationContext.__call__ "Permanent link") Wrap a callable so its execution is traced in a span. When used as a decorator, a new span is created for each call to the wrapped function. The bound arguments are recorded as input attributes, the return value is recorded as an output attribute, and any exception is recorded and marks the span as an error. Parameters: * **`fn`** (`_FnType`) – The function or coroutine function to wrap. Returns: * `_FnType` – The wrapped callable. #### `__enter__()` [¶](https://microsoft.github.io/agent-lightning/latest/reference/internal/#agentlightning.emitter.annotation.OperationContext.__enter__ "Permanent link") Enter the context manager and start a new span. Returns: * `[OperationContext](https://microsoft.github.io/agent-lightning/latest/reference/internal/#agentlightning.emitter.annotation.OperationContext " agentlightning.emitter.annotation.OperationContext") ` – The current :class:`OperationContext` instance with an active span. #### `__exit__(exc_type, exc_val, exc_tb)` [¶](https://microsoft.github.io/agent-lightning/latest/reference/internal/#agentlightning.emitter.annotation.OperationContext.__exit__ "Permanent link") Exit the context manager and finish the span. #### `__init__(name, attributes, propagate=True)` [¶](https://microsoft.github.io/agent-lightning/latest/reference/internal/#agentlightning.emitter.annotation.OperationContext.__init__ "Permanent link") Initialize a new operation context. Parameters: * **`name`** (`str`) – Human-readable name of the span. * **`attributes`** (`Dict[str, Any]`) – Initial attributes attached to the span. Values are JSON-serialized where necessary. * **`propagate`** (`bool`, default: `True` ) – Whether the span should be sent to active exporters. #### `set_input(*args, **kwargs)` [¶](https://microsoft.github.io/agent-lightning/latest/reference/internal/#agentlightning.emitter.annotation.OperationContext.set_input "Permanent link") Record input arguments on the current span. Positional arguments are stored under the `input.args.` attributes, and keyword arguments are stored under `input.` attributes. This is intended for use inside a `with operation(...) as op` block. Parameters: * **`*args`** (`Any`, default: `()` ) – Positional arguments to record. * **`**kwargs`** (`Any`, default: `{}` ) – Keyword arguments to record. #### `set_output(output)` [¶](https://microsoft.github.io/agent-lightning/latest/reference/internal/#agentlightning.emitter.annotation.OperationContext.set_output "Permanent link") Record the output value on the current span. This is intended for use inside a `with operation(...) as op` block. Parameters: * **`output`** (`Any`) – The output value to record. #### `span()` [¶](https://microsoft.github.io/agent-lightning/latest/reference/internal/#agentlightning.emitter.annotation.OperationContext.span "Permanent link") Get the span that was created by this context manager. LLM Proxy[¶](https://microsoft.github.io/agent-lightning/latest/reference/internal/#llm-proxy "Permanent link") ---------------------------------------------------------------------------------------------------------------- ### `agentlightning.llm_proxy.ModelConfig` [¶](https://microsoft.github.io/agent-lightning/latest/reference/internal/#agentlightning.llm_proxy.ModelConfig "Permanent link") Bases: `TypedDict` LiteLLM model registration entry. This mirrors the items in LiteLLM's `model_list` section. Attributes: * **`model_name`** (`str`) – Logical model name exposed by the proxy. * **`litellm_params`** (`Dict[str, Any]`) – Parameters passed to LiteLLM for this model (e.g., backend model id, api\_base, additional options). ### `agentlightning.llm_proxy.LightningSpanExporter` [¶](https://microsoft.github.io/agent-lightning/latest/reference/internal/#agentlightning.llm_proxy.LightningSpanExporter "Permanent link") Bases: `SpanExporter` Buffered OTEL span exporter with subtree flushing and training-store sink. Design: * Spans are buffered until a root span's entire subtree is available. * A private event loop on a daemon thread runs async flush logic. * Rollout/attempt/sequence metadata is reconstructed by merging headers from any span within a subtree. Thread-safety: * Buffer access is protected by a re-entrant lock. * Export is synchronous to the caller yet schedules an async flush on the internal loop, then waits for completion. #### `export(spans)` [¶](https://microsoft.github.io/agent-lightning/latest/reference/internal/#agentlightning.llm_proxy.LightningSpanExporter.export "Permanent link") Export spans via buffered subtree flush. Appends spans to the internal buffer, then triggers an async flush on the private event loop. Blocks until that flush completes. Parameters: * **`spans`** (`Sequence[ReadableSpan]`) – Sequence of spans to export. Returns: * **`SpanExportResult`** ( `SpanExportResult` ) – SUCCESS on flush success, else FAILURE. #### `shutdown()` [¶](https://microsoft.github.io/agent-lightning/latest/reference/internal/#agentlightning.llm_proxy.LightningSpanExporter.shutdown "Permanent link") Shut down the exporter event loop. Safe to call at process exit. ### `agentlightning.llm_proxy.LightningOpenTelemetry` [¶](https://microsoft.github.io/agent-lightning/latest/reference/internal/#agentlightning.llm_proxy.LightningOpenTelemetry "Permanent link") Bases: `OpenTelemetry` OpenTelemetry integration that exports spans to the Lightning store. Responsibilities: * Ensures each request is annotated with a per-attempt sequence id so spans are ordered deterministically even with clock skew across nodes. * Uses [`LightningSpanExporter`](https://microsoft.github.io/agent-lightning/latest/reference/internal/#agentlightning.llm_proxy.LightningSpanExporter " agentlightning.llm_proxy.LightningSpanExporter") to persist spans for analytics and training. #### `async_pre_call_deployment_hook(kwargs, call_type=None)` `async` [¶](https://microsoft.github.io/agent-lightning/latest/reference/internal/#agentlightning.llm_proxy.LightningOpenTelemetry.async_pre_call_deployment_hook "Permanent link") The root span is sometimes missing (e.g., when Anthropic endpoint is used). It is created in an auth module in LiteLLM. If it's missing, we create it here. ### `agentlightning.llm_proxy.AddReturnTokenIds` [¶](https://microsoft.github.io/agent-lightning/latest/reference/internal/#agentlightning.llm_proxy.AddReturnTokenIds "Permanent link") Bases: `CustomLogger` LiteLLM logger hook to request token ids from vLLM. This mutates the outgoing request payload to include `return_token_ids=True` for backends that support token id return (e.g., vLLM). See also [vLLM PR #22587](https://github.com/vllm-project/vllm/pull/22587) #### `async_pre_call_hook(*args, **kwargs)` `async` [¶](https://microsoft.github.io/agent-lightning/latest/reference/internal/#agentlightning.llm_proxy.AddReturnTokenIds.async_pre_call_hook "Permanent link") Async pre-call hook to adjust request payload. Parameters: * **`args`** (`Any`, default: `()` ) – Positional args from LiteLLM. * **`kwargs`** (`Any`, default: `{}` ) – Keyword args from LiteLLM. Returns: * `Optional[Union[Exception, str, Dict[str, Any]]]` – Either an updated payload dict or an Exception to short-circuit. ### `agentlightning.llm_proxy.StreamConversionMiddleware` [¶](https://microsoft.github.io/agent-lightning/latest/reference/internal/#agentlightning.llm_proxy.StreamConversionMiddleware "Permanent link") Bases: `BaseHTTPMiddleware` Middleware to convert streaming responses to non-streaming responses. Useful for backend that only supports non-streaming responses. LiteLLM's OpenTelemetry is also buggy with streaming responses. The conversion will hopefully bypass the bug. #### `anthropic_stream_generator(original_response)` `async` [¶](https://microsoft.github.io/agent-lightning/latest/reference/internal/#agentlightning.llm_proxy.StreamConversionMiddleware.anthropic_stream_generator "Permanent link") Generate Anthropic SSE-formatted chunks from complete content blocks This is a dirty hack for Anthropic-style streaming from non-streaming response. The sse format is subject to change based on Anthropic's implementation. If so, try to use `MessageInspectionMiddleware` to inspect the update and fix accordingly. #### `openai_stream_generator(response_json)` `async` [¶](https://microsoft.github.io/agent-lightning/latest/reference/internal/#agentlightning.llm_proxy.StreamConversionMiddleware.openai_stream_generator "Permanent link") Convert a _complete_ OpenAI chat.completions choice into a stream of OpenAI-compatible SSE chunks. This emits: * an initial delta with the role ("assistant"), * a sequence of deltas for message.content (split into small chunks), * deltas for any tool\_calls (including id/name and chunked arguments), * a terminal chunk with finish\_reason, * and finally the literal '\[DONE\]'. Notes: * We only handle a _single_ choice (index 0 typically). * We purposefully don't attempt to stream logprobs. * Chunking strategy is simple and conservative to avoid splitting multi-byte characters: we slice on spaces where possible, then fall back to fixed-size substrings. ### `agentlightning.llm_proxy.MessageInspectionMiddleware` [¶](https://microsoft.github.io/agent-lightning/latest/reference/internal/#agentlightning.llm_proxy.MessageInspectionMiddleware "Permanent link") Bases: `BaseHTTPMiddleware` Middleware to inspect the request and response bodies. It's for debugging purposes. Add it via "message\_inspection" middleware alias. ### `agentlightning.llm_proxy.RolloutAttemptMiddleware` [¶](https://microsoft.github.io/agent-lightning/latest/reference/internal/#agentlightning.llm_proxy.RolloutAttemptMiddleware "Permanent link") Bases: `BaseHTTPMiddleware` Rewrites /rollout/{rid}/attempt/{aid}/... -> /... and injects x-rollout-id, x-attempt-id, x-sequence-id headers. LLMProxy can update store later without rebuilding middleware. Store[¶](https://microsoft.github.io/agent-lightning/latest/reference/internal/#store "Permanent link") -------------------------------------------------------------------------------------------------------- ### `agentlightning.store.base.UNSET = _UnsetType()` `module-attribute` [¶](https://microsoft.github.io/agent-lightning/latest/reference/internal/#agentlightning.store.base.UNSET "Permanent link") ### `agentlightning.store.utils.rollout_status_from_attempt(attempt, config)` `async` [¶](https://microsoft.github.io/agent-lightning/latest/reference/internal/#agentlightning.store.utils.rollout_status_from_attempt "Permanent link") Propagate the status of an attempt to the rollout. Returns: * `[RolloutStatus](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.RolloutStatus " agentlightning.RolloutStatus = Literal['queuing', 'preparing', 'running', 'failed', 'succeeded', 'cancelled', 'requeuing'] module-attribute (agentlightning.types.RolloutStatus)") ` – The status of the rollout from the perspective of the attempt. ### `agentlightning.store.utils.scan_unhealthy_rollouts(rollouts)` `async` [¶](https://microsoft.github.io/agent-lightning/latest/reference/internal/#agentlightning.store.utils.scan_unhealthy_rollouts "Permanent link") Perform health check on all running rollouts in the store. This method should be called periodically to: 1. Check for unresponsive attempts (no heartbeat or spans for a while) 2. Check for timed-out rollouts (running too long since start\_time) This operation is completely unlocked. The caller is responsible for locking the store. Parameters: * **`rollouts`** (`List[[AttemptedRollout](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.AttemptedRollout " agentlightning.AttemptedRollout (agentlightning.types.AttemptedRollout)") ]`) – The list of running rollouts to check. Returns: * `Dict[Tuple[str, str], [AttemptStatus](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.AttemptStatus " agentlightning.AttemptStatus = Literal['preparing', 'running', 'failed', 'succeeded', 'unresponsive', 'timeout'] module-attribute (agentlightning.types.AttemptStatus)") ]` – A dictionary of updates to the rollouts. Tracing and OpenTelemetry[¶](https://microsoft.github.io/agent-lightning/latest/reference/internal/#tracing-and-opentelemetry "Permanent link") ------------------------------------------------------------------------------------------------------------------------------------------------ ### `agentlightning.tracer.otel.LightningSpanProcessor` [¶](https://microsoft.github.io/agent-lightning/latest/reference/internal/#agentlightning.tracer.otel.LightningSpanProcessor "Permanent link") Bases: `SpanProcessor` Span processor that subclasses OpenTelemetry's `SpanProcessor` and adds support to dump traces to a [`LightningStore`](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.LightningStore " agentlightning.LightningStore") . It serves two purposes: 1. Records all the spans in a local buffer. 2. Submits the spans to the event loop to be added to the store. #### `attempt_id` `property` [¶](https://microsoft.github.io/agent-lightning/latest/reference/internal/#agentlightning.tracer.otel.LightningSpanProcessor.attempt_id "Permanent link") The attempt ID to submit the spans to. #### `disable_store_submission` `property` `writable` [¶](https://microsoft.github.io/agent-lightning/latest/reference/internal/#agentlightning.tracer.otel.LightningSpanProcessor.disable_store_submission "Permanent link") Whether to disable submitting spans to the store. #### `rollout_id` `property` [¶](https://microsoft.github.io/agent-lightning/latest/reference/internal/#agentlightning.tracer.otel.LightningSpanProcessor.rollout_id "Permanent link") The rollout ID to submit the spans to. #### `store` `property` [¶](https://microsoft.github.io/agent-lightning/latest/reference/internal/#agentlightning.tracer.otel.LightningSpanProcessor.store "Permanent link") The store to submit the spans to. #### `on_end(span)` [¶](https://microsoft.github.io/agent-lightning/latest/reference/internal/#agentlightning.tracer.otel.LightningSpanProcessor.on_end "Permanent link") Process a span when it ends. Parameters: * **`span`** (`ReadableSpan`) – The span that has ended. #### `spans()` [¶](https://microsoft.github.io/agent-lightning/latest/reference/internal/#agentlightning.tracer.otel.LightningSpanProcessor.spans "Permanent link") Get the list of spans collected by this processor. This is useful for debugging and testing purposes. Returns: * `List[[Span](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.Span " agentlightning.Span (agentlightning.types.Span)") ]` – List of [`Span`](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.Span " agentlightning.Span") objects collected during tracing. Deprecated APIs[¶](https://microsoft.github.io/agent-lightning/latest/reference/internal/#deprecated-apis "Permanent link") ---------------------------------------------------------------------------------------------------------------------------- ### `agentlightning.emitter.reward.reward(fn)` [¶](https://microsoft.github.io/agent-lightning/latest/reference/internal/#agentlightning.emitter.reward.reward "Permanent link") Decorate a reward function so its outputs are tracked as spans. The decorator integrates with AgentOps when it is available and falls back to the built-in telemetry otherwise. Both synchronous and asynchronous functions are supported transparently. Deprecated This decorator is deprecated. Use [`emit_reward`](https://microsoft.github.io/agent-lightning/latest/reference/agent/#agentlightning.emit_reward " agentlightning.emit_reward(reward, *, primary_key=None, attributes=None, propagate=True)") instead. Parameters: * **`fn`** (`_FnType`) – Callable that produces a numeric reward. Returns: * `_FnType` – Wrapped callable that preserves the original signature. ### `agentlightning.server.AgentLightningServer` [¶](https://microsoft.github.io/agent-lightning/latest/reference/internal/#agentlightning.server.AgentLightningServer "Permanent link") High-level controller for the legacy Agent Lightning FastAPI server. The controller orchestrates server start-up, task queueing, resource updates, and retrieval of client rollouts. It is primarily used by existing systems that still rely on the HTTP-based workflow. Deprecated [`AgentLightningServer`](https://microsoft.github.io/agent-lightning/latest/reference/internal/#agentlightning.server.AgentLightningServer " agentlightning.server.AgentLightningServer") is part of the legacy client/server stack. Prefer the store-based runtime for new integrations. #### `__init__(host='127.0.0.1', port=8000, task_timeout_seconds=300.0)` [¶](https://microsoft.github.io/agent-lightning/latest/reference/internal/#agentlightning.server.AgentLightningServer.__init__ "Permanent link") Initialize the controller. Parameters: * **`host`** (`str`, default: `'127.0.0.1'` ) – Hostname or IP address to bind the HTTP server to. * **`port`** (`int`, default: `8000` ) – TCP port exposed by the server. * **`task_timeout_seconds`** (`float`, default: `300.0` ) – Seconds before a claimed task is considered stale and re-queued. #### `get_completed_rollout(rollout_id)` `async` [¶](https://microsoft.github.io/agent-lightning/latest/reference/internal/#agentlightning.server.AgentLightningServer.get_completed_rollout "Permanent link") Retrieve a specific completed rollout by identifier. #### `poll_completed_rollout(rollout_id, timeout=None)` `async` [¶](https://microsoft.github.io/agent-lightning/latest/reference/internal/#agentlightning.server.AgentLightningServer.poll_completed_rollout "Permanent link") Poll for a completed rollout until it becomes available or a timeout expires. Parameters: * **`rollout_id`** (`str`) – Identifier of the rollout to wait for. * **`timeout`** (`Optional[float]`, default: `None` ) – Maximum number of seconds to wait. `None` waits indefinitely. Returns: * `Optional[[RolloutLegacy](https://microsoft.github.io/agent-lightning/latest/reference/internal/#agentlightning.RolloutLegacy " agentlightning.RolloutLegacy (agentlightning.types.RolloutLegacy)") ]` – Retrieved rollout, or `None` when the timeout is reached without success. #### `queue_task(sample, mode=None, resources_id=None, metadata=None)` `async` [¶](https://microsoft.github.io/agent-lightning/latest/reference/internal/#agentlightning.server.AgentLightningServer.queue_task "Permanent link") Add a task to the queue for a client to process. #### `retrieve_completed_rollouts()` `async` [¶](https://microsoft.github.io/agent-lightning/latest/reference/internal/#agentlightning.server.AgentLightningServer.retrieve_completed_rollouts "Permanent link") Return every completed rollout and clear the internal buffer. #### `run_forever()` `async` [¶](https://microsoft.github.io/agent-lightning/latest/reference/internal/#agentlightning.server.AgentLightningServer.run_forever "Permanent link") Run the server indefinitely until `stop()` is invoked. #### `start()` `async` [¶](https://microsoft.github.io/agent-lightning/latest/reference/internal/#agentlightning.server.AgentLightningServer.start "Permanent link") Start the FastAPI server in the background. #### `stop()` `async` [¶](https://microsoft.github.io/agent-lightning/latest/reference/internal/#agentlightning.server.AgentLightningServer.stop "Permanent link") Stop the FastAPI server and wait for a graceful shutdown. #### `update_resources(resources)` `async` [¶](https://microsoft.github.io/agent-lightning/latest/reference/internal/#agentlightning.server.AgentLightningServer.update_resources "Permanent link") Publish a new resource bundle and return its generated identifier. ### `agentlightning.server.ServerDataStore` [¶](https://microsoft.github.io/agent-lightning/latest/reference/internal/#agentlightning.server.ServerDataStore "Permanent link") Async-safe container for in-memory server state. The store tracks queued tasks, claimed tasks, uploaded rollouts, and the currently published resources. All interactions are guarded by asyncio locks so that the FastAPI handlers can safely run in parallel. Deprecated [`ServerDataStore`](https://microsoft.github.io/agent-lightning/latest/reference/internal/#agentlightning.server.ServerDataStore " agentlightning.server.ServerDataStore") is part of the legacy client/server stack. Use [`LightningStore`](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.LightningStore " agentlightning.LightningStore") instead. #### `add_task(sample, mode=None, resources_id=None, metadata=None)` `async` [¶](https://microsoft.github.io/agent-lightning/latest/reference/internal/#agentlightning.server.ServerDataStore.add_task "Permanent link") Enqueue a new task and return the generated rollout identifier. Parameters: * **`sample`** (`Any`) – Payload that describes the task input. * **`mode`** (`Literal['train', 'val', 'test'] | None`, default: `None` ) – Phase in which the sample should be executed (`"train"`, `"val"`, or `"test"`). * **`resources_id`** (`str | None`, default: `None` ) – Identifier of a resource bundle that the executor should load before running the task. * **`metadata`** (`Dict[str, Any] | None`, default: `None` ) – Optional metadata forwarded to the executor. Returns: * `str` – Unique rollout identifier assigned to the task. #### `get_latest_resources()` `async` [¶](https://microsoft.github.io/agent-lightning/latest/reference/internal/#agentlightning.server.ServerDataStore.get_latest_resources "Permanent link") Return the most recent resource bundle, if one exists. #### `get_next_task()` `async` [¶](https://microsoft.github.io/agent-lightning/latest/reference/internal/#agentlightning.server.ServerDataStore.get_next_task "Permanent link") Retrieve the next task from the queue without blocking. Returns: * `Optional[[Task](https://microsoft.github.io/agent-lightning/latest/reference/internal/#agentlightning.Task " agentlightning.Task (agentlightning.types.Task)") ]` – Next [`Task`](https://microsoft.github.io/agent-lightning/latest/reference/internal/#agentlightning.Task " agentlightning.Task") ready to execute, or `None` * `Optional[[Task](https://microsoft.github.io/agent-lightning/latest/reference/internal/#agentlightning.Task " agentlightning.Task (agentlightning.types.Task)") ]` – when the queue is empty. #### `get_processing_tasks()` [¶](https://microsoft.github.io/agent-lightning/latest/reference/internal/#agentlightning.server.ServerDataStore.get_processing_tasks "Permanent link") Return a copy of currently processing tasks for timeout checking. #### `get_resources_by_id(resources_id)` `async` [¶](https://microsoft.github.io/agent-lightning/latest/reference/internal/#agentlightning.server.ServerDataStore.get_resources_by_id "Permanent link") Retrieve a specific resource bundle by identifier. Parameters: * **`resources_id`** (`str`) – Identifier that was previously published to the store. Returns: * `Optional[[ResourcesUpdate](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.ResourcesUpdate " agentlightning.ResourcesUpdate (agentlightning.types.ResourcesUpdate)") ]` – Matching [`ResourcesUpdate`](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.ResourcesUpdate " agentlightning.ResourcesUpdate") * `Optional[[ResourcesUpdate](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.ResourcesUpdate " agentlightning.ResourcesUpdate (agentlightning.types.ResourcesUpdate)") ]` – instance, or `None` when the identifier is unknown. #### `requeue_task(task)` `async` [¶](https://microsoft.github.io/agent-lightning/latest/reference/internal/#agentlightning.server.ServerDataStore.requeue_task "Permanent link") Requeue a task that timed out while being processed. #### `retrieve_completed_rollouts()` `async` [¶](https://microsoft.github.io/agent-lightning/latest/reference/internal/#agentlightning.server.ServerDataStore.retrieve_completed_rollouts "Permanent link") Return all completed rollouts and clear the internal buffer. #### `retrieve_rollout(rollout_id)` `async` [¶](https://microsoft.github.io/agent-lightning/latest/reference/internal/#agentlightning.server.ServerDataStore.retrieve_rollout "Permanent link") Retrieve and remove a stored rollout by identifier. Parameters: * **`rollout_id`** (`str`) – Identifier of the rollout to fetch. Returns: * `Optional[[RolloutLegacy](https://microsoft.github.io/agent-lightning/latest/reference/internal/#agentlightning.RolloutLegacy " agentlightning.RolloutLegacy (agentlightning.types.RolloutLegacy)") ]` – Stored [`RolloutLegacy`](https://microsoft.github.io/agent-lightning/latest/reference/internal/#agentlightning.RolloutLegacy " agentlightning.RolloutLegacy") , or `None` * `Optional[[RolloutLegacy](https://microsoft.github.io/agent-lightning/latest/reference/internal/#agentlightning.RolloutLegacy " agentlightning.RolloutLegacy (agentlightning.types.RolloutLegacy)") ]` – when the identifier is unknown. #### `store_rollout(rollout)` `async` [¶](https://microsoft.github.io/agent-lightning/latest/reference/internal/#agentlightning.server.ServerDataStore.store_rollout "Permanent link") Persist a completed rollout for later inspection. Parameters: * **`rollout`** (`[RolloutLegacy](https://microsoft.github.io/agent-lightning/latest/reference/internal/#agentlightning.RolloutLegacy " agentlightning.RolloutLegacy (agentlightning.types.RolloutLegacy)") `) – Rollout returned by a client. #### `update_resources(update)` `async` [¶](https://microsoft.github.io/agent-lightning/latest/reference/internal/#agentlightning.server.ServerDataStore.update_resources "Permanent link") Persist a new resource bundle and mark it as the latest version. Parameters: * **`update`** (`[ResourcesUpdate](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.ResourcesUpdate " agentlightning.ResourcesUpdate (agentlightning.types.ResourcesUpdate)") `) – Resource payload received from a client. ### `agentlightning.client.AgentLightningClient` [¶](https://microsoft.github.io/agent-lightning/latest/reference/internal/#agentlightning.client.AgentLightningClient "Permanent link") Client wrapper for the legacy version-aware Agent Lightning server. The client exposes synchronous and asynchronous helpers for polling tasks, retrieving resource bundles, and submitting rollouts. It also maintains a simple in-memory cache keyed by the server-provided resource identifier to avoid redundant network requests. Deprecated [`AgentLightningClient`](https://microsoft.github.io/agent-lightning/latest/reference/internal/#agentlightning.client.AgentLightningClient " agentlightning.client.AgentLightningClient") is part of the legacy client/server stack. New code should rely on the store-based APIs implemented in `agentlightning.store`. Attributes: * **`endpoint`** – Base URL of the Agent Lightning server. * **`poll_interval`** – Delay in seconds between polling attempts when no task is available. * **`timeout`** – Timeout in seconds applied to HTTP requests. * **`task_count`** – Number of tasks claimed during the lifetime of this client. #### `__init__(endpoint, poll_interval=5.0, timeout=10.0)` [¶](https://microsoft.github.io/agent-lightning/latest/reference/internal/#agentlightning.client.AgentLightningClient.__init__ "Permanent link") Initialize the client. Parameters: * **`endpoint`** (`str`) – Root URL of the Agent Lightning server. * **`poll_interval`** (`float`, default: `5.0` ) – Seconds to wait between polling attempts. * **`timeout`** (`float`, default: `10.0` ) – Seconds before a request to the server is considered timed out. #### `get_latest_resources()` [¶](https://microsoft.github.io/agent-lightning/latest/reference/internal/#agentlightning.client.AgentLightningClient.get_latest_resources "Permanent link") Fetch the most recent resource bundle advertised by the server. Returns: * `Optional[[ResourcesUpdate](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.ResourcesUpdate " agentlightning.ResourcesUpdate (agentlightning.types.ResourcesUpdate)") ]` – [`ResourcesUpdate`](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.ResourcesUpdate " agentlightning.ResourcesUpdate") for the * `Optional[[ResourcesUpdate](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.ResourcesUpdate " agentlightning.ResourcesUpdate (agentlightning.types.ResourcesUpdate)") ]` – newest version, or `None` when unavailable. #### `get_latest_resources_async()` `async` [¶](https://microsoft.github.io/agent-lightning/latest/reference/internal/#agentlightning.client.AgentLightningClient.get_latest_resources_async "Permanent link") Fetch the most recent resource bundle advertised by the server. Returns: * `Optional[[ResourcesUpdate](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.ResourcesUpdate " agentlightning.ResourcesUpdate (agentlightning.types.ResourcesUpdate)") ]` – [`ResourcesUpdate`](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.ResourcesUpdate " agentlightning.ResourcesUpdate") for the * `Optional[[ResourcesUpdate](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.ResourcesUpdate " agentlightning.ResourcesUpdate (agentlightning.types.ResourcesUpdate)") ]` – newest version, or `None` when unavailable. #### `get_resources_by_id(resource_id)` [¶](https://microsoft.github.io/agent-lightning/latest/reference/internal/#agentlightning.client.AgentLightningClient.get_resources_by_id "Permanent link") Fetch a specific resource bundle by identifier. Parameters: * **`resource_id`** (`str`) – Identifier sourced from the task metadata. Returns: * `Optional[[ResourcesUpdate](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.ResourcesUpdate " agentlightning.ResourcesUpdate (agentlightning.types.ResourcesUpdate)") ]` – Cached or freshly downloaded * `Optional[[ResourcesUpdate](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.ResourcesUpdate " agentlightning.ResourcesUpdate (agentlightning.types.ResourcesUpdate)") ]` – [`ResourcesUpdate`](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.ResourcesUpdate " agentlightning.ResourcesUpdate") , or * `Optional[[ResourcesUpdate](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.ResourcesUpdate " agentlightning.ResourcesUpdate (agentlightning.types.ResourcesUpdate)") ]` – `None` when the server returns an error. #### `get_resources_by_id_async(resource_id)` `async` [¶](https://microsoft.github.io/agent-lightning/latest/reference/internal/#agentlightning.client.AgentLightningClient.get_resources_by_id_async "Permanent link") Fetch a specific resource bundle by identifier. Parameters: * **`resource_id`** (`str`) – Identifier sourced from the task metadata. Returns: * `Optional[[ResourcesUpdate](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.ResourcesUpdate " agentlightning.ResourcesUpdate (agentlightning.types.ResourcesUpdate)") ]` – Cached or freshly downloaded * `Optional[[ResourcesUpdate](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.ResourcesUpdate " agentlightning.ResourcesUpdate (agentlightning.types.ResourcesUpdate)") ]` – [`ResourcesUpdate`](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.ResourcesUpdate " agentlightning.ResourcesUpdate") , or * `Optional[[ResourcesUpdate](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.ResourcesUpdate " agentlightning.ResourcesUpdate (agentlightning.types.ResourcesUpdate)") ]` – `None` when the server returns an error. #### `poll_next_task()` [¶](https://microsoft.github.io/agent-lightning/latest/reference/internal/#agentlightning.client.AgentLightningClient.poll_next_task "Permanent link") Poll the server synchronously until a task becomes available. Returns: * `Optional[[Task](https://microsoft.github.io/agent-lightning/latest/reference/internal/#agentlightning.Task " agentlightning.Task (agentlightning.types.Task)") ]` – The next [`Task`](https://microsoft.github.io/agent-lightning/latest/reference/internal/#agentlightning.Task " agentlightning.Task") available for execution, or * `Optional[[Task](https://microsoft.github.io/agent-lightning/latest/reference/internal/#agentlightning.Task " agentlightning.Task (agentlightning.types.Task)") ]` – `None` if polling fails. #### `poll_next_task_async()` `async` [¶](https://microsoft.github.io/agent-lightning/latest/reference/internal/#agentlightning.client.AgentLightningClient.poll_next_task_async "Permanent link") Poll the server asynchronously until a task becomes available. Returns: * `Optional[[Task](https://microsoft.github.io/agent-lightning/latest/reference/internal/#agentlightning.Task " agentlightning.Task (agentlightning.types.Task)") ]` – The next [`Task`](https://microsoft.github.io/agent-lightning/latest/reference/internal/#agentlightning.Task " agentlightning.Task") exposed by the server, * `Optional[[Task](https://microsoft.github.io/agent-lightning/latest/reference/internal/#agentlightning.Task " agentlightning.Task (agentlightning.types.Task)") ]` – or `None` if polling fails. #### `post_rollout(rollout)` [¶](https://microsoft.github.io/agent-lightning/latest/reference/internal/#agentlightning.client.AgentLightningClient.post_rollout "Permanent link") Submit a completed rollout back to the server. Parameters: * **`rollout`** (`[RolloutLegacy](https://microsoft.github.io/agent-lightning/latest/reference/internal/#agentlightning.RolloutLegacy " agentlightning.RolloutLegacy (agentlightning.types.RolloutLegacy)") `) – Legacy rollout payload produced by the executor. Returns: * `Optional[Dict[str, Any]]` – Parsed JSON response returned by the server, or `None` when the request fails. #### `post_rollout_async(rollout)` `async` [¶](https://microsoft.github.io/agent-lightning/latest/reference/internal/#agentlightning.client.AgentLightningClient.post_rollout_async "Permanent link") Submit a completed rollout back to the server. Parameters: * **`rollout`** (`[RolloutLegacy](https://microsoft.github.io/agent-lightning/latest/reference/internal/#agentlightning.RolloutLegacy " agentlightning.RolloutLegacy (agentlightning.types.RolloutLegacy)") `) – Legacy rollout payload produced by the executor. Returns: * `Optional[Dict[str, Any]]` – Parsed JSON response returned by the server, or `None` when the request fails. ### `agentlightning.client.DevTaskLoader` [¶](https://microsoft.github.io/agent-lightning/latest/reference/internal/#agentlightning.client.DevTaskLoader "Permanent link") Bases: `[AgentLightningClient](https://microsoft.github.io/agent-lightning/latest/reference/internal/#agentlightning.client.AgentLightningClient " agentlightning.client.AgentLightningClient") ` In-memory task loader used for development and integration tests. The loader mimics the behavior of the legacy HTTP server by storing tasks and resources locally. Polling methods simply iterate over the provided collection, allowing rapid iteration without provisioning any external infrastructure. Deprecated [`DevTaskLoader`](https://microsoft.github.io/agent-lightning/latest/reference/internal/#agentlightning.client.DevTaskLoader " agentlightning.client.DevTaskLoader") is a compatibility shim. Prefer [`Trainer.dev`](https://microsoft.github.io/agent-lightning/latest/reference/trainer/#agentlightning.Trainer.dev " dev(agent, train_dataset=None, *, val_dataset=None)") for new code. #### `rollouts` `property` [¶](https://microsoft.github.io/agent-lightning/latest/reference/internal/#agentlightning.client.DevTaskLoader.rollouts "Permanent link") Return the rollouts posted back to the loader during development runs. #### `__init__(tasks, resources, **kwargs)` [¶](https://microsoft.github.io/agent-lightning/latest/reference/internal/#agentlightning.client.DevTaskLoader.__init__ "Permanent link") Initialize the loader with predefined tasks and resources. Parameters: * **`tasks`** (`Union[List[[TaskInput](https://microsoft.github.io/agent-lightning/latest/reference/internal/#agentlightning.TaskInput " agentlightning.TaskInput = Any module-attribute (agentlightning.types.TaskInput)") ], List[[Task](https://microsoft.github.io/agent-lightning/latest/reference/internal/#agentlightning.Task " agentlightning.Task (agentlightning.types.Task)") ]]`) – Sequence of task inputs or preconstructed tasks that will be served in order. * **`resources`** (`Union[[NamedResources](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.NamedResources " agentlightning.NamedResources = Dict[str, ResourceUnion] module-attribute (agentlightning.types.NamedResources)") , [ResourcesUpdate](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.ResourcesUpdate " agentlightning.ResourcesUpdate (agentlightning.types.ResourcesUpdate)") ]`) – Static resources returned for any `resources_id` query. * **`**kwargs`** (`Any`, default: `{}` ) – Additional keyword arguments forwarded to the parent client. Raises: * `ValueError` – If no tasks are provided or both [`Task`](https://microsoft.github.io/agent-lightning/latest/reference/internal/#agentlightning.Task " agentlightning.Task") and [`TaskInput`](https://microsoft.github.io/agent-lightning/latest/reference/internal/#agentlightning.TaskInput " agentlightning.TaskInput = Any module-attribute ") instances are mixed. #### `poll_next_task()` [¶](https://microsoft.github.io/agent-lightning/latest/reference/internal/#agentlightning.client.DevTaskLoader.poll_next_task "Permanent link") Return the next task from the local queue. If [`TaskInput`](https://microsoft.github.io/agent-lightning/latest/reference/internal/#agentlightning.TaskInput " agentlightning.TaskInput = Any module-attribute ") instances were provided, they are converted into [`Task`](https://microsoft.github.io/agent-lightning/latest/reference/internal/#agentlightning.Task " agentlightning.Task") objects on the fly. Otherwise, the preconstructed tasks are returned in sequence. Returns: * `Optional[[Task](https://microsoft.github.io/agent-lightning/latest/reference/internal/#agentlightning.Task " agentlightning.Task (agentlightning.types.Task)") ]` – Next task to execute. ### `agentlightning.Task` [¶](https://microsoft.github.io/agent-lightning/latest/reference/internal/#agentlightning.Task "Permanent link") Bases: `BaseModel` Rollout request served to client agents. Deprecated The legacy HTTP client/server stack still uses this model. Prefer [`LightningStore`](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.LightningStore " agentlightning.LightningStore") APIs for new workflows. ### `agentlightning.TaskInput = Any` `module-attribute` [¶](https://microsoft.github.io/agent-lightning/latest/reference/internal/#agentlightning.TaskInput "Permanent link") Task input type. Accepts arbitrary payloads. ### `agentlightning.TaskIfAny` [¶](https://microsoft.github.io/agent-lightning/latest/reference/internal/#agentlightning.TaskIfAny "Permanent link") Bases: `BaseModel` A task or indication that no task is available. Deprecated Use [`LightningStore`](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.LightningStore " agentlightning.LightningStore") APIs for new workflows. #### `is_available` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/latest/reference/internal/#agentlightning.TaskIfAny.is_available "Permanent link") Indication that a task is available. ### `agentlightning.RolloutRawResultLegacy = Union[None, float, List[Triplet], List[Dict[str, Any]], List[ReadableSpan], RolloutLegacy]` `module-attribute` [¶](https://microsoft.github.io/agent-lightning/latest/reference/internal/#agentlightning.RolloutRawResultLegacy "Permanent link") Legacy rollout result type. Deprecated Use [`RolloutRawResult`](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.RolloutRawResult " agentlightning.RolloutRawResult = Union[None, float, List[ReadableSpan], List[Span], List[SpanCoreFields]] module-attribute ") instead. ### `agentlightning.RolloutLegacy` [¶](https://microsoft.github.io/agent-lightning/latest/reference/internal/#agentlightning.RolloutLegacy "Permanent link") Bases: `BaseModel` Legacy reporting payload exchanged with the deprecated HTTP server. Deprecated Use [`Rollout`](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.Rollout " agentlightning.Rollout") instead. Back to top --- # Runner - Agent-lightning [Skip to content](https://microsoft.github.io/agent-lightning/latest/reference/runner/#runner-side-references) ⚠️ You are viewing development documentation. For stable documentation, please visit [https://microsoft.github.io/agent-lightning/stable/](https://microsoft.github.io/agent-lightning/stable/) Runner-side References[¶](https://microsoft.github.io/agent-lightning/latest/reference/runner/#runner-side-references "Permanent link") ======================================================================================================================================== Note This reference covers APIs that are designed to be used at "Runner Side". Runners[¶](https://microsoft.github.io/agent-lightning/latest/reference/runner/#runners "Permanent link") ---------------------------------------------------------------------------------------------------------- ### `agentlightning.LitAgentRunner` [¶](https://microsoft.github.io/agent-lightning/latest/reference/runner/#agentlightning.LitAgentRunner "Permanent link") Bases: `[Runner](https://microsoft.github.io/agent-lightning/latest/reference/runner/#agentlightning.Runner " agentlightning.Runner (agentlightning.runner.base.Runner)") [T_task]` Execute [`LitAgent`](https://microsoft.github.io/agent-lightning/latest/reference/agent/#agentlightning.LitAgent " agentlightning.LitAgent") tasks with tracing support. This runner manages the complete lifecycle of agent rollout execution, including task polling, resource management, tracing, and hooks. It supports both continuous iteration over tasks from the store and single-step execution. Attributes: * **`worker_id`** (`Optional[int]`) – Identifier for the active worker process, if any. #### `tracer` `property` [¶](https://microsoft.github.io/agent-lightning/latest/reference/runner/#agentlightning.LitAgentRunner.tracer "Permanent link") Get the tracer instance. Returns: * `[Tracer](https://microsoft.github.io/agent-lightning/latest/reference/runner/#agentlightning.Tracer " agentlightning.Tracer (agentlightning.tracer.base.Tracer)") ` – The Tracer instance used by this runner. #### `__init__(tracer, max_rollouts=None, poll_interval=5.0, heartbeat_interval=10.0, interval_jitter=0.5, heartbeat_launch_mode='thread', heartbeat_include_gpu=False)` [¶](https://microsoft.github.io/agent-lightning/latest/reference/runner/#agentlightning.LitAgentRunner.__init__ "Permanent link") Initialize the agent runner. Parameters: * **`tracer`** (`[Tracer](https://microsoft.github.io/agent-lightning/latest/reference/runner/#agentlightning.Tracer " agentlightning.Tracer (agentlightning.tracer.base.Tracer)") `) – [`Tracer`](https://microsoft.github.io/agent-lightning/latest/reference/runner/#agentlightning.Tracer " agentlightning.Tracer") used for rollout spans. * **`max_rollouts`** (`Optional[int]`, default: `None` ) – Optional cap on iterations processed by [`iter`](https://microsoft.github.io/agent-lightning/latest/reference/runner/#agentlightning.LitAgentRunner.iter " iter(*, event=None) async ") . * **`poll_interval`** (`float`, default: `5.0` ) – Seconds to wait between store polls when no work is available. * **`heartbeat_interval`** (`float`, default: `10.0` ) – Seconds to wait between sending heartbeats to the store. * **`interval_jitter`** (`float`, default: `0.5` ) – Jitter factor for the poll interval. The actual interval will be between poll\_interval - interval\_jitter and poll\_interval + interval\_jitter. This is to avoid the overload caused by the synchronization of the runners. * **`heartbeat_launch_mode`** (`Literal['asyncio', 'thread']`, default: `'thread'` ) – Launch mode for the heartbeat loop. Can be "asyncio" or "thread". "thread" is the default and recommended mode as it prevents blocking the event loop under load. Use "asyncio" for simpler deployments with low worker counts. * **`heartbeat_include_gpu`** (`bool`, default: `False` ) – Whether to include GPU stats in heartbeat snapshots. Querying GPU stats can be slow under load, so this is disabled by default. #### `get_agent()` [¶](https://microsoft.github.io/agent-lightning/latest/reference/runner/#agentlightning.LitAgentRunner.get_agent "Permanent link") Get the agent instance. Returns: * `[LitAgent](https://microsoft.github.io/agent-lightning/latest/reference/agent/#agentlightning.LitAgent " agentlightning.LitAgent (agentlightning.litagent.LitAgent)") [T_task]` – The LitAgent instance managed by this runner. Raises: * `ValueError` – If the agent has not been initialized via [`init`](https://microsoft.github.io/agent-lightning/latest/reference/runner/#agentlightning.LitAgentRunner.init " init(agent, *, hooks=None, **kwargs)") . #### `get_store()` [¶](https://microsoft.github.io/agent-lightning/latest/reference/runner/#agentlightning.LitAgentRunner.get_store "Permanent link") Get the store instance. Returns: * `[LightningStore](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.LightningStore " agentlightning.LightningStore (agentlightning.store.base.LightningStore)") ` – The LightningStore instance for this worker. Raises: * `ValueError` – If the store has not been initialized via [`init_worker`](https://microsoft.github.io/agent-lightning/latest/reference/runner/#agentlightning.LitAgentRunner.init_worker " init_worker(worker_id, store, **kwargs)") . #### `get_worker_id()` [¶](https://microsoft.github.io/agent-lightning/latest/reference/runner/#agentlightning.LitAgentRunner.get_worker_id "Permanent link") Get the formatted worker ID string. Returns: * `str` – A formatted string like "Worker-0" if initialized, or "Worker-Unknown" * `str` – if the worker ID has not been set. #### `init(agent, *, hooks=None, **kwargs)` [¶](https://microsoft.github.io/agent-lightning/latest/reference/runner/#agentlightning.LitAgentRunner.init "Permanent link") Initialize the runner with the agent. This sets up the agent-runner relationship, registers hooks, and initializes the tracer. Parameters: * **`agent`** (`[LitAgent](https://microsoft.github.io/agent-lightning/latest/reference/agent/#agentlightning.LitAgent " agentlightning.LitAgent (agentlightning.litagent.LitAgent)") [T_task]`) – [`LitAgent`](https://microsoft.github.io/agent-lightning/latest/reference/agent/#agentlightning.LitAgent " agentlightning.LitAgent") instance executed by the runner. * **`hooks`** (`Optional[Sequence[[Hook](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.Hook " agentlightning.Hook (agentlightning.types.Hook)") ]]`, default: `None` ) – Optional sequence of [`Hook`](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.Hook " agentlightning.Hook") callbacks invoked around tracing and rollout boundaries. * **`**kwargs`** (`Any`, default: `{}` ) – Additional initialization arguments (currently unused). #### `init_worker(worker_id, store, **kwargs)` [¶](https://microsoft.github.io/agent-lightning/latest/reference/runner/#agentlightning.LitAgentRunner.init_worker "Permanent link") Initialize the runner for each worker with worker\_id and store. This method is called once per worker in a distributed setup to provide the worker with its ID and store connection. Parameters: * **`worker_id`** (`int`) – Unique identifier for this worker process. * **`store`** (`[LightningStore](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.LightningStore " agentlightning.LightningStore (agentlightning.store.base.LightningStore)") `) – [`LightningStore`](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.LightningStore " agentlightning.LightningStore") used for task coordination and persistence. * **`**kwargs`** (`Any`, default: `{}` ) – Additional worker-specific initialization arguments (currently unused). #### `iter(*, event=None)` `async` [¶](https://microsoft.github.io/agent-lightning/latest/reference/runner/#agentlightning.LitAgentRunner.iter "Permanent link") Run the runner, continuously iterating over tasks in the store. This method polls the store for new rollouts and executes them until: * The event is set (if provided) * The max\_rollouts limit is reached (if configured) * No more tasks are available All exceptions during rollout execution are caught and logged but not propagated, allowing the runner to continue processing subsequent tasks. Parameters: * **`event`** (`Optional[[ExecutionEvent](https://microsoft.github.io/agent-lightning/latest/reference/trainer/#agentlightning.ExecutionEvent " agentlightning.ExecutionEvent (agentlightning.execution.events.ExecutionEvent)") ]`, default: `None` ) – Optional ExecutionEvent object to signal the runner to stop. The runner will check this event periodically and stop gracefully when set. #### `step(input, *, resources=None, mode=None, event=None)` `async` [¶](https://microsoft.github.io/agent-lightning/latest/reference/runner/#agentlightning.LitAgentRunner.step "Permanent link") Execute a single task directly, bypassing the task queue. This method creates a new rollout for the given input and executes it immediately. Unlike [`iter()`](https://microsoft.github.io/agent-lightning/latest/reference/runner/#agentlightning.LitAgentRunner.iter " iter(*, event=None) async ") , exceptions are propagated to the caller. Parameters: * **`input`** (`T_task`) – The task input to be processed by the agent. * **`resources`** (`Optional[[NamedResources](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.NamedResources " agentlightning.NamedResources = Dict[str, ResourceUnion] module-attribute (agentlightning.types.NamedResources)") ]`, default: `None` ) – Optional named resources to be used for this specific task. If provided, a new resources entry will be created in the store. If not provided, the latest resources from the store will be used. * **`mode`** (`Optional[[RolloutMode](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.RolloutMode " agentlightning.RolloutMode = Literal['train', 'val', 'test'] module-attribute (agentlightning.types.RolloutMode)") ]`, default: `None` ) – Optional rollout mode ("train" or "validation"). If not provided, the agent's default mode will be used. * **`event`** (`Optional[[ExecutionEvent](https://microsoft.github.io/agent-lightning/latest/reference/trainer/#agentlightning.ExecutionEvent " agentlightning.ExecutionEvent (agentlightning.execution.events.ExecutionEvent)") ]`, default: `None` ) – Optional ExecutionEvent object to signal interruption (currently unused but included for interface consistency). Returns: * `[Rollout](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.Rollout " agentlightning.Rollout (agentlightning.types.Rollout)") ` – The completed rollout. Raises: * `Exception` – Any exception that occurs during rollout execution will be re-raised to the caller. #### `teardown(*args, **kwargs)` [¶](https://microsoft.github.io/agent-lightning/latest/reference/runner/#agentlightning.LitAgentRunner.teardown "Permanent link") Teardown the runner and clean up all resources. This method resets all internal state including the agent, store, hooks, and worker ID, and calls the tracer's teardown method. Parameters: * **`*args`** (`Any`, default: `()` ) – Additional teardown arguments (currently unused). * **`**kwargs`** (`Any`, default: `{}` ) – Additional teardown keyword arguments (currently unused). #### `teardown_worker(worker_id, *args, **kwargs)` [¶](https://microsoft.github.io/agent-lightning/latest/reference/runner/#agentlightning.LitAgentRunner.teardown_worker "Permanent link") Teardown the runner for a specific worker. This method cleans up worker-specific resources and resets the worker ID. Parameters: * **`worker_id`** (`int`) – Unique identifier of the worker being torn down. * **`*args`** (`Any`, default: `()` ) – Additional teardown arguments (currently unused). * **`**kwargs`** (`Any`, default: `{}` ) – Additional teardown keyword arguments (currently unused). ### `agentlightning.Runner` [¶](https://microsoft.github.io/agent-lightning/latest/reference/runner/#agentlightning.Runner "Permanent link") Bases: `[ParallelWorkerBase](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.ParallelWorkerBase " agentlightning.ParallelWorkerBase (agentlightning.types.ParallelWorkerBase)") `, `Generic[T_task]` Abstract base class for long-running agent executors. Runner implementations coordinate [`LitAgent`](https://microsoft.github.io/agent-lightning/latest/reference/agent/#agentlightning.LitAgent " agentlightning.LitAgent") instances, acquire work from a [`LightningStore`](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.LightningStore " agentlightning.LightningStore") , and emit [`Rollout`](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.Rollout " agentlightning.Rollout") objects. Subclasses decide how to schedule work (polling, streaming, etc.) while this base class provides a minimal lifecycle contract. #### `init(agent, **kwargs)` [¶](https://microsoft.github.io/agent-lightning/latest/reference/runner/#agentlightning.Runner.init "Permanent link") Prepare the runner to execute tasks for `agent`. This method is called only once during the setup for all workers, not for each worker. Parameters: * **`agent`** (`[LitAgent](https://microsoft.github.io/agent-lightning/latest/reference/agent/#agentlightning.LitAgent " agentlightning.LitAgent (agentlightning.litagent.LitAgent)") [T_task]`) – Agent instance providing task-specific logic. * **`**kwargs`** (`Any`, default: `{}` ) – Optional runner-specific configuration. Raises: * `NotImplementedError` – Subclasses must supply the initialization routine. #### `init_worker(worker_id, store, **kwargs)` [¶](https://microsoft.github.io/agent-lightning/latest/reference/runner/#agentlightning.Runner.init_worker "Permanent link") Configure worker-local state before processing tasks. This method is called for **each** worker during the setup. Parameters: * **`worker_id`** (`int`) – Unique identifier for this worker process or thread. * **`store`** (`[LightningStore](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.LightningStore " agentlightning.LightningStore (agentlightning.store.base.LightningStore)") `) – Shared [`LightningStore`](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.LightningStore " agentlightning.LightningStore") backing task coordination. * **`**kwargs`** (`Any`, default: `{}` ) – Optional worker-specific configuration. Raises: * `NotImplementedError` – Subclasses must prepare per-worker resources. #### `iter(*, event=None)` `async` [¶](https://microsoft.github.io/agent-lightning/latest/reference/runner/#agentlightning.Runner.iter "Permanent link") Run the runner, continuously iterating over tasks in the store. This method runs in a loop, polling the store for new tasks and executing them until interrupted by the event or when no more tasks are available. Parameters: * **`event`** (`Optional[[ExecutionEvent](https://microsoft.github.io/agent-lightning/latest/reference/trainer/#agentlightning.ExecutionEvent " agentlightning.ExecutionEvent (agentlightning.execution.events.ExecutionEvent)") ]`, default: `None` ) – Cooperative stop signal. When set, the runner should complete the current unit of work and exit the loop. Raises: * `NotImplementedError` – Subclasses provide the iteration behavior. #### `run(*args, **kwargs)` [¶](https://microsoft.github.io/agent-lightning/latest/reference/runner/#agentlightning.Runner.run "Permanent link") Deprecated synchronous entry point. Use [`iter()`](https://microsoft.github.io/agent-lightning/latest/reference/runner/#agentlightning.Runner.iter " iter(*, event=None) async ") or [`step()`](https://microsoft.github.io/agent-lightning/latest/reference/runner/#agentlightning.Runner.step " step(input, *, resources=None, mode=None, event=None) async ") instead. Raises: * `RuntimeError` – Always raised to direct callers to [iter()](https://microsoft.github.io/agent-lightning/latest/reference/runner/#agentlightning.Runner.iter " iter(*, event=None) async ") or [step()](https://microsoft.github.io/agent-lightning/latest/reference/runner/#agentlightning.Runner.step " step(input, *, resources=None, mode=None, event=None) async ") . #### `run_context(*, agent, store, hooks=None, worker_id=None)` [¶](https://microsoft.github.io/agent-lightning/latest/reference/runner/#agentlightning.Runner.run_context "Permanent link") Initialize and tear down a runner within a simple context manager. The helper is primarily intended for debugging runner implementations outside of a full [`Trainer`](https://microsoft.github.io/agent-lightning/latest/reference/trainer/#agentlightning.Trainer " agentlightning.Trainer") stack. Parameters: * **`agent`** (`[LitAgent](https://microsoft.github.io/agent-lightning/latest/reference/agent/#agentlightning.LitAgent " agentlightning.LitAgent (agentlightning.litagent.LitAgent)") [T_task]`) – Agent executed by this runner. * **`store`** (`[LightningStore](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.LightningStore " agentlightning.LightningStore (agentlightning.store.base.LightningStore)") `) – Backing [`LightningStore`](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.LightningStore " agentlightning.LightningStore") . If you don't have one, you can easily create one with [`InMemoryLightningStore`](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.InMemoryLightningStore " agentlightning.InMemoryLightningStore") . * **`hooks`** (`Optional[Sequence[[Hook](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.Hook " agentlightning.Hook (agentlightning.types.Hook)") ]]`, default: `None` ) – Optional sequence of hooks recognised by the runner. Not all runners support hooks. * **`worker_id`** (`Optional[int]`, default: `None` ) – Override the worker identifier used during setup. Defaults to `0`. #### `step(input, *, resources=None, mode=None, event=None)` `async` [¶](https://microsoft.github.io/agent-lightning/latest/reference/runner/#agentlightning.Runner.step "Permanent link") Execute a single task with the given input. This method provides fine-grained control for executing individual tasks directly, bypassing the store's task queue. Parameters: * **`input`** (`T_task`) – Task payload consumed by the agent. * **`resources`** (`Optional[[NamedResources](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.NamedResources " agentlightning.NamedResources = Dict[str, ResourceUnion] module-attribute (agentlightning.types.NamedResources)") ]`, default: `None` ) – Optional named resources scoped to this invocation. * **`mode`** (`Optional[[RolloutMode](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.RolloutMode " agentlightning.RolloutMode = Literal['train', 'val', 'test'] module-attribute (agentlightning.types.RolloutMode)") ]`, default: `None` ) – Optional rollout mode such as `"train"` or `"eval"`. * **`event`** (`Optional[[ExecutionEvent](https://microsoft.github.io/agent-lightning/latest/reference/trainer/#agentlightning.ExecutionEvent " agentlightning.ExecutionEvent (agentlightning.execution.events.ExecutionEvent)") ]`, default: `None` ) – Cooperative stop signal for long-running tasks. Returns: * `[Rollout](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.Rollout " agentlightning.Rollout (agentlightning.types.Rollout)") ` – Completed rollout produced by the agent. Raises: * `NotImplementedError` – Subclasses provide the execution behavior. #### `teardown(*args, **kwargs)` [¶](https://microsoft.github.io/agent-lightning/latest/reference/runner/#agentlightning.Runner.teardown "Permanent link") Release resources acquired during [`init()`](https://microsoft.github.io/agent-lightning/latest/reference/runner/#agentlightning.Runner.init " init(agent, **kwargs)") . Raises: * `NotImplementedError` – Subclasses must implement the shutdown routine. #### `teardown_worker(worker_id, *args, **kwargs)` [¶](https://microsoft.github.io/agent-lightning/latest/reference/runner/#agentlightning.Runner.teardown_worker "Permanent link") Release per-worker resources allocated by [`init_worker()`](https://microsoft.github.io/agent-lightning/latest/reference/runner/#agentlightning.Runner.init_worker " init_worker(worker_id, store, **kwargs)") . Parameters: * **`worker_id`** (`int`) – Identifier of the worker being torn down. Raises: * `NotImplementedError` – Subclasses must implement the shutdown routine. Tracer[¶](https://microsoft.github.io/agent-lightning/latest/reference/runner/#tracer "Permanent link") -------------------------------------------------------------------------------------------------------- ### `agentlightning.AgentOpsTracer` [¶](https://microsoft.github.io/agent-lightning/latest/reference/runner/#agentlightning.AgentOpsTracer "Permanent link") Bases: `[OtelTracer](https://microsoft.github.io/agent-lightning/latest/reference/runner/#agentlightning.OtelTracer " agentlightning.OtelTracer (agentlightning.tracer.otel.OtelTracer)") ` Traces agent execution using AgentOps. This tracer provides functionality to capture execution details using the AgentOps library. It manages the AgentOps client initialization, server setup, and integration with the OpenTelemetry tracing ecosystem. Attributes: * **`agentops_managed`** – Whether to automatically manage `agentops`. When set to true, tracer calls `agentops.init()` automatically and launches an agentops endpoint locally. If not, you are responsible for calling and using it before using the tracer. * **`instrument_managed`** – Whether to automatically manage instrumentation. When set to false, you will manage the instrumentation yourself and the tracer might not work as expected. * **`daemon`** – Whether the AgentOps server runs as a daemon process. Only applicable if `agentops_managed` is True. #### `get_langchain_handler(tags=None)` [¶](https://microsoft.github.io/agent-lightning/latest/reference/runner/#agentlightning.AgentOpsTracer.get_langchain_handler "Permanent link") Get the Langchain callback handler for integrating with Langchain. Parameters: * **`tags`** (`List[str] | None`, default: `None` ) – Optional list of tags to apply to the Langchain callback handler. Returns: * `LangchainCallbackHandler` – An instance of the Langchain callback handler. #### `trace_context(name=None, *, store=None, rollout_id=None, attempt_id=None)` `async` [¶](https://microsoft.github.io/agent-lightning/latest/reference/runner/#agentlightning.AgentOpsTracer.trace_context "Permanent link") Starts a new tracing context. This should be used as a context manager. Parameters: * **`name`** (`Optional[str]`, default: `None` ) – Optional name for the tracing context. * **`store`** (`Optional[[LightningStore](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.LightningStore " agentlightning.LightningStore (agentlightning.store.base.LightningStore)") ]`, default: `None` ) – Optional store to add the spans to. * **`rollout_id`** (`Optional[str]`, default: `None` ) – Optional rollout ID to add the spans to. * **`attempt_id`** (`Optional[str]`, default: `None` ) – Optional attempt ID to add the spans to. Yields: * `AsyncGenerator[Tracer, None]` – The OpenTelemetry tracer instance to collect spans. ### `agentlightning.OtelTracer` [¶](https://microsoft.github.io/agent-lightning/latest/reference/runner/#agentlightning.OtelTracer "Permanent link") Bases: `[Tracer](https://microsoft.github.io/agent-lightning/latest/reference/runner/#agentlightning.Tracer " agentlightning.Tracer (agentlightning.tracer.base.Tracer)") ` Tracer that provides a basic OpenTelemetry tracer provider. You should be able to collect agent-lightning signals like rewards with this tracer, but no other function instrumentations like `openai.chat.completion`. #### `get_last_trace()` [¶](https://microsoft.github.io/agent-lightning/latest/reference/runner/#agentlightning.OtelTracer.get_last_trace "Permanent link") Retrieves the raw list of captured spans from the most recent trace. Returns: * `List[[Span](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.Span " agentlightning.Span (agentlightning.types.Span)") ]` – A list of [`Span`](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.Span " agentlightning.Span") objects captured during the most recent trace. #### `trace_context(name=None, *, store=None, rollout_id=None, attempt_id=None)` `async` [¶](https://microsoft.github.io/agent-lightning/latest/reference/runner/#agentlightning.OtelTracer.trace_context "Permanent link") Starts a new tracing context. This should be used as a context manager. Parameters: * **`name`** (`Optional[str]`, default: `None` ) – Optional name for the tracing context. * **`store`** (`Optional[[LightningStore](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.LightningStore " agentlightning.LightningStore (agentlightning.store.base.LightningStore)") ]`, default: `None` ) – Optional store to add the spans to. * **`rollout_id`** (`Optional[str]`, default: `None` ) – Optional rollout ID to add the spans to. * **`attempt_id`** (`Optional[str]`, default: `None` ) – Optional attempt ID to add the spans to. Yields: * `AsyncGenerator[Tracer, None]` – The OpenTelemetry tracer instance to collect spans. ### `agentlightning.Tracer` [¶](https://microsoft.github.io/agent-lightning/latest/reference/runner/#agentlightning.Tracer "Permanent link") Bases: `[ParallelWorkerBase](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.ParallelWorkerBase " agentlightning.ParallelWorkerBase (agentlightning.types.ParallelWorkerBase)") ` An abstract base class for tracers. This class defines a standard interface for tracing code execution, capturing the resulting spans, and providing them for analysis. It is designed to be backend-agnostic, allowing for different implementations (e.g., for AgentOps, OpenTelemetry, Docker, etc.). The primary interaction pattern is through the [`trace_context`](https://microsoft.github.io/agent-lightning/latest/reference/runner/#agentlightning.Tracer.trace_context " trace_context(name=None, *, store=None, rollout_id=None, attempt_id=None)") context manager, which ensures that traces are properly started and captured, even in the case of exceptions. A typical workflow: `[](https://microsoft.github.io/agent-lightning/latest/reference/runner/#__codelineno-0-1) tracer = YourTracerImplementation() [](https://microsoft.github.io/agent-lightning/latest/reference/runner/#__codelineno-0-2) [](https://microsoft.github.io/agent-lightning/latest/reference/runner/#__codelineno-0-3) try: [](https://microsoft.github.io/agent-lightning/latest/reference/runner/#__codelineno-0-4) async with tracer.trace_context(name="my_traced_task"): [](https://microsoft.github.io/agent-lightning/latest/reference/runner/#__codelineno-0-5) # ... code to be traced ... [](https://microsoft.github.io/agent-lightning/latest/reference/runner/#__codelineno-0-6) await run_my_agent_logic() [](https://microsoft.github.io/agent-lightning/latest/reference/runner/#__codelineno-0-7) except Exception as e: [](https://microsoft.github.io/agent-lightning/latest/reference/runner/#__codelineno-0-8) print(f"An error occurred: {e}") [](https://microsoft.github.io/agent-lightning/latest/reference/runner/#__codelineno-0-9) [](https://microsoft.github.io/agent-lightning/latest/reference/runner/#__codelineno-0-10) # Retrieve the trace data after the context block [](https://microsoft.github.io/agent-lightning/latest/reference/runner/#__codelineno-0-11) spans: list[ReadableSpan] = tracer.get_last_trace() [](https://microsoft.github.io/agent-lightning/latest/reference/runner/#__codelineno-0-12) [](https://microsoft.github.io/agent-lightning/latest/reference/runner/#__codelineno-0-13) # Process the trace data [](https://microsoft.github.io/agent-lightning/latest/reference/runner/#__codelineno-0-14) if trace_tree: [](https://microsoft.github.io/agent-lightning/latest/reference/runner/#__codelineno-0-15) rl_triplets = TracerTraceToTriplet().adapt(spans) [](https://microsoft.github.io/agent-lightning/latest/reference/runner/#__codelineno-0-16) # ... do something with the triplets` #### `create_span(name, attributes=None, timestamp=None, status=None)` [¶](https://microsoft.github.io/agent-lightning/latest/reference/runner/#agentlightning.Tracer.create_span "Permanent link") Notify the tracer that a span should be created here. It uses a fire-and-forget approach and doesn't wait for the span to be created. Parameters: * **`name`** (`str`) – The name of the span. * **`attributes`** (`Optional[[Attributes](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.Attributes " agentlightning.Attributes = Dict[str, AttributeValue] module-attribute (agentlightning.types.Attributes)") ]`, default: `None` ) – The attributes of the span. * **`timestamp`** (`Optional[float]`, default: `None` ) – The timestamp of the span. * **`status`** (`Optional[[TraceStatus](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.TraceStatus " agentlightning.TraceStatus (agentlightning.types.TraceStatus)") ]`, default: `None` ) – The status of the span. Returns: * `[SpanCoreFields](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.SpanCoreFields " agentlightning.SpanCoreFields (agentlightning.types.SpanCoreFields)") ` – The core fields of the span. #### `get_langchain_handler()` [¶](https://microsoft.github.io/agent-lightning/latest/reference/runner/#agentlightning.Tracer.get_langchain_handler "Permanent link") Get a handler to install in langchain agent callback. Agents are expected to use this handler in their agents to enable tracing. #### `get_last_trace()` [¶](https://microsoft.github.io/agent-lightning/latest/reference/runner/#agentlightning.Tracer.get_last_trace "Permanent link") Retrieves the raw list of captured spans from the most recent trace. Returns: * `List[[Span](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.Span " agentlightning.Span (agentlightning.types.Span)") ]` – A list of [`Span`](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.Span " agentlightning.Span") objects collected during the last trace. #### `init_worker(worker_id, store=None)` [¶](https://microsoft.github.io/agent-lightning/latest/reference/runner/#agentlightning.Tracer.init_worker "Permanent link") Initialize the tracer for a worker. Parameters: * **`worker_id`** (`int`) – The ID of the worker. * **`store`** (`Optional[[LightningStore](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.LightningStore " agentlightning.LightningStore (agentlightning.store.base.LightningStore)") ]`, default: `None` ) – The store to add the spans to. If it's provided, traces will be added to the store when tracing. #### `lifespan(store=None)` [¶](https://microsoft.github.io/agent-lightning/latest/reference/runner/#agentlightning.Tracer.lifespan "Permanent link") A context manager to manage the lifespan of the tracer. This can be used to set up and tear down any necessary resources for the tracer, useful for debugging purposes. Parameters: * **`store`** (`Optional[[LightningStore](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.LightningStore " agentlightning.LightningStore (agentlightning.store.base.LightningStore)") ]`, default: `None` ) – The store to add the spans to. If it's provided, traces will be added to the store when tracing. #### `operation_context(name, attributes=None, start_time=None, end_time=None)` [¶](https://microsoft.github.io/agent-lightning/latest/reference/runner/#agentlightning.Tracer.operation_context "Permanent link") Start to record an operation to a span. Parameters: * **`name`** (`str`) – The name of the operation. * **`attributes`** (`Optional[[Attributes](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.Attributes " agentlightning.Attributes = Dict[str, AttributeValue] module-attribute (agentlightning.types.Attributes)") ]`, default: `None` ) – The attributes of the operation. * **`start_time`** (`Optional[float]`, default: `None` ) – The start time of the operation. * **`end_time`** (`Optional[float]`, default: `None` ) – The end time of the operation. Returns: * `ContextManager[[SpanRecordingContext](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.SpanRecordingContext " agentlightning.SpanRecordingContext (agentlightning.types.SpanRecordingContext)") ]` – A [`SpanRecordingContext`](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.SpanRecordingContext " agentlightning.SpanRecordingContext") for recording the operation on the span. #### `trace_context(name=None, *, store=None, rollout_id=None, attempt_id=None)` [¶](https://microsoft.github.io/agent-lightning/latest/reference/runner/#agentlightning.Tracer.trace_context "Permanent link") Starts a new tracing context. This should be used as a context manager. The implementation should handle the setup and teardown of the tracing for the enclosed code block. It must ensure that any spans generated within the `with` block are collected and made available via [`get_last_trace`](https://microsoft.github.io/agent-lightning/latest/reference/runner/#agentlightning.Tracer.get_last_trace " get_last_trace()") . Parameters: * **`name`** (`Optional[str]`, default: `None` ) – The name for the root span of this trace context. * **`store`** (`Optional[[LightningStore](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.LightningStore " agentlightning.LightningStore (agentlightning.store.base.LightningStore)") ]`, default: `None` ) – The store to add the spans to. Deprecated in favor of passing store to init\_worker(). * **`rollout_id`** (`Optional[str]`, default: `None` ) – The rollout ID to add the spans to. * **`attempt_id`** (`Optional[str]`, default: `None` ) – The attempt ID to add the spans to. #### `trace_run(func, *args, **kwargs)` [¶](https://microsoft.github.io/agent-lightning/latest/reference/runner/#agentlightning.Tracer.trace_run "Permanent link") A convenience wrapper to trace the execution of a single synchronous function. Deprecated in favor of customizing Runners. Parameters: * **`func`** (`Callable[..., Any]`) – The synchronous function to execute and trace. * **`*args`** (`Any`, default: `()` ) – Positional arguments to pass to the function. * **`**kwargs`** (`Any`, default: `{}` ) – Keyword arguments to pass to the function. Returns: * `Any` – The return value of the function. #### `trace_run_async(func, *args, **kwargs)` `async` [¶](https://microsoft.github.io/agent-lightning/latest/reference/runner/#agentlightning.Tracer.trace_run_async "Permanent link") A convenience wrapper to trace the execution of a single asynchronous function. Deprecated in favor of customizing Runners. Parameters: * **`func`** (`Callable[..., Awaitable[Any]]`) – The asynchronous function to execute and trace. * **`*args`** (`Any`, default: `()` ) – Positional arguments to pass to the function. * **`**kwargs`** (`Any`, default: `{}` ) – Keyword arguments to pass to the function. Returns: * `Any` – The return value of the function. ### `agentlightning.tracer.weave.WeaveTracer` [¶](https://microsoft.github.io/agent-lightning/latest/reference/runner/#agentlightning.tracer.weave.WeaveTracer "Permanent link") Bases: `[Tracer](https://microsoft.github.io/agent-lightning/latest/reference/runner/#agentlightning.Tracer " agentlightning.Tracer (agentlightning.tracer.base.Tracer)") ` Tracer implementation using Weave for telemetry and trace logging. This replaces AgentOpsTracer with a Weave-based manual trace context. It tracks: * Function/method calls * Input/Output data * Exceptions and logs them to Weave Cloud (W&B backend) or optionally bypasses the network for testing. #### `__init__(*, project_name=None, weave_user_settings=None, instrument_managed=True)` [¶](https://microsoft.github.io/agent-lightning/latest/reference/runner/#agentlightning.tracer.weave.WeaveTracer.__init__ "Permanent link") Initialize a WeaveTracer instance. Parameters: * **`project_name`** (`str | None`, default: `None` ) – Optional project name for Weave; defaults to the current module name. * **`weave_user_settings`** (`UserSettings | None`, default: `None` ) – Optional UserSettings for Weave. * **`instrument_managed`** (`bool`, default: `True` ) – Whether to patch the Weave/W&B integration to bypass actual network calls for testing. #### `complete_call_handler(call)` `async` [¶](https://microsoft.github.io/agent-lightning/latest/reference/runner/#agentlightning.tracer.weave.WeaveTracer.complete_call_handler "Permanent link") Handler called when a Weave Call finishes. Converts the call (including nested children) into spans and stores them in LightningStore. #### `convert_call_to_span(call, rollout_id=None, attempt_id=None, sequence_id=None)` `async` [¶](https://microsoft.github.io/agent-lightning/latest/reference/runner/#agentlightning.tracer.weave.WeaveTracer.convert_call_to_span "Permanent link") Convert a Weave Call (with nested children) into a Agent-lightning Span. `rollout_id` and `attempt_id` are required to attach the spans to the store. Parameters: * **`call`** (`CallSchema`) – The Weave Call object. * **`rollout_id`** (`Optional[str]`, default: `None` ) – Optional rollout ID to attach to spans. * **`attempt_id`** (`Optional[str]`, default: `None` ) – Optional attempt ID to attach to spans. * **`sequence_id`** (`Optional[int]`, default: `None` ) – Optional sequence ID to attach to spans. Returns: * `[Span](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.Span " agentlightning.Span (agentlightning.types.Span)") ` – List of converted spans. #### `init_worker(worker_id, store=None)` [¶](https://microsoft.github.io/agent-lightning/latest/reference/runner/#agentlightning.tracer.weave.WeaveTracer.init_worker "Permanent link") Initialize the tracer for a worker thread/process. Parameters: * **`worker_id`** (`int`) – Identifier of the worker. * **`store`** (`Optional[[LightningStore](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.LightningStore " agentlightning.LightningStore (agentlightning.store.base.LightningStore)") ]`, default: `None` ) – Optional LightningStore for storing spans. #### `partial_call_handler(request_content)` `async` [¶](https://microsoft.github.io/agent-lightning/latest/reference/runner/#agentlightning.tracer.weave.WeaveTracer.partial_call_handler "Permanent link") Handler called when a Weave Call starts. Parameters: * **`request_content`** (`Dict[str, Any]`) – The partial Weave Call object. Returns: * `int` – The sequence ID for the call. #### `teardown_worker(worker_id)` [¶](https://microsoft.github.io/agent-lightning/latest/reference/runner/#agentlightning.tracer.weave.WeaveTracer.teardown_worker "Permanent link") Clean up tracer resources for the worker. Parameters: * **`worker_id`** (`int`) – Identifier of the worker. #### `trace_context(name=None, *, rollout_id=None, attempt_id=None, **kwargs)` `async` [¶](https://microsoft.github.io/agent-lightning/latest/reference/runner/#agentlightning.tracer.weave.WeaveTracer.trace_context "Permanent link") Asynchronous implementation of the tracing context. Parameters: * **`name`** (`Optional[str]`, default: `None` ) – Optional operation name. * **`rollout_id`** (`Optional[str]`, default: `None` ) – Optional rollout ID. * **`attempt_id`** (`Optional[str]`, default: `None` ) – Optional attempt ID. Raises: * `ValueError` – If store, rollout\_id, and attempt\_id are inconsistently provided. * `RuntimeError` – If Weave is not installed or client is uninitialized. ### `agentlightning.DummyTracer` [¶](https://microsoft.github.io/agent-lightning/latest/reference/runner/#agentlightning.DummyTracer "Permanent link") Bases: `[Tracer](https://microsoft.github.io/agent-lightning/latest/reference/runner/#agentlightning.Tracer " agentlightning.Tracer (agentlightning.tracer.base.Tracer)") ` A dummy tracer that does not trace anything, but it is compatible with the emitter API. It doesn't rely on any backend tracer, and also doesn't use any stores. ### `agentlightning.set_active_tracer(tracer)` [¶](https://microsoft.github.io/agent-lightning/latest/reference/runner/#agentlightning.set_active_tracer "Permanent link") Set the active tracer for the current process. Parameters: * **`tracer`** (`[Tracer](https://microsoft.github.io/agent-lightning/latest/reference/runner/#agentlightning.Tracer " agentlightning.Tracer (agentlightning.tracer.base.Tracer)") `) – The tracer to set as active. ### `agentlightning.get_active_tracer()` [¶](https://microsoft.github.io/agent-lightning/latest/reference/runner/#agentlightning.get_active_tracer "Permanent link") Get the active tracer for the current process. Returns: * `Optional[[Tracer](https://microsoft.github.io/agent-lightning/latest/reference/runner/#agentlightning.Tracer " agentlightning.Tracer (agentlightning.tracer.base.Tracer)") ]` – The active tracer, or None if no tracer is active. ### `agentlightning.clear_active_tracer()` [¶](https://microsoft.github.io/agent-lightning/latest/reference/runner/#agentlightning.clear_active_tracer "Permanent link") Clear the active tracer for the current process. ### `agentlightning.tracer.weave.WeaveTracer` [¶](https://microsoft.github.io/agent-lightning/latest/reference/runner/#agentlightning.tracer.weave.WeaveTracer "Permanent link") Bases: `[Tracer](https://microsoft.github.io/agent-lightning/latest/reference/runner/#agentlightning.Tracer " agentlightning.Tracer (agentlightning.tracer.base.Tracer)") ` Tracer implementation using Weave for telemetry and trace logging. This replaces AgentOpsTracer with a Weave-based manual trace context. It tracks: * Function/method calls * Input/Output data * Exceptions and logs them to Weave Cloud (W&B backend) or optionally bypasses the network for testing. #### `__init__(*, project_name=None, weave_user_settings=None, instrument_managed=True)` [¶](https://microsoft.github.io/agent-lightning/latest/reference/runner/#agentlightning.tracer.weave.WeaveTracer.__init__ "Permanent link") Initialize a WeaveTracer instance. Parameters: * **`project_name`** (`str | None`, default: `None` ) – Optional project name for Weave; defaults to the current module name. * **`weave_user_settings`** (`UserSettings | None`, default: `None` ) – Optional UserSettings for Weave. * **`instrument_managed`** (`bool`, default: `True` ) – Whether to patch the Weave/W&B integration to bypass actual network calls for testing. #### `complete_call_handler(call)` `async` [¶](https://microsoft.github.io/agent-lightning/latest/reference/runner/#agentlightning.tracer.weave.WeaveTracer.complete_call_handler "Permanent link") Handler called when a Weave Call finishes. Converts the call (including nested children) into spans and stores them in LightningStore. #### `convert_call_to_span(call, rollout_id=None, attempt_id=None, sequence_id=None)` `async` [¶](https://microsoft.github.io/agent-lightning/latest/reference/runner/#agentlightning.tracer.weave.WeaveTracer.convert_call_to_span "Permanent link") Convert a Weave Call (with nested children) into a Agent-lightning Span. `rollout_id` and `attempt_id` are required to attach the spans to the store. Parameters: * **`call`** (`CallSchema`) – The Weave Call object. * **`rollout_id`** (`Optional[str]`, default: `None` ) – Optional rollout ID to attach to spans. * **`attempt_id`** (`Optional[str]`, default: `None` ) – Optional attempt ID to attach to spans. * **`sequence_id`** (`Optional[int]`, default: `None` ) – Optional sequence ID to attach to spans. Returns: * `[Span](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.Span " agentlightning.Span (agentlightning.types.Span)") ` – List of converted spans. #### `init_worker(worker_id, store=None)` [¶](https://microsoft.github.io/agent-lightning/latest/reference/runner/#agentlightning.tracer.weave.WeaveTracer.init_worker "Permanent link") Initialize the tracer for a worker thread/process. Parameters: * **`worker_id`** (`int`) – Identifier of the worker. * **`store`** (`Optional[[LightningStore](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.LightningStore " agentlightning.LightningStore (agentlightning.store.base.LightningStore)") ]`, default: `None` ) – Optional LightningStore for storing spans. #### `partial_call_handler(request_content)` `async` [¶](https://microsoft.github.io/agent-lightning/latest/reference/runner/#agentlightning.tracer.weave.WeaveTracer.partial_call_handler "Permanent link") Handler called when a Weave Call starts. Parameters: * **`request_content`** (`Dict[str, Any]`) – The partial Weave Call object. Returns: * `int` – The sequence ID for the call. #### `teardown_worker(worker_id)` [¶](https://microsoft.github.io/agent-lightning/latest/reference/runner/#agentlightning.tracer.weave.WeaveTracer.teardown_worker "Permanent link") Clean up tracer resources for the worker. Parameters: * **`worker_id`** (`int`) – Identifier of the worker. #### `trace_context(name=None, *, rollout_id=None, attempt_id=None, **kwargs)` `async` [¶](https://microsoft.github.io/agent-lightning/latest/reference/runner/#agentlightning.tracer.weave.WeaveTracer.trace_context "Permanent link") Asynchronous implementation of the tracing context. Parameters: * **`name`** (`Optional[str]`, default: `None` ) – Optional operation name. * **`rollout_id`** (`Optional[str]`, default: `None` ) – Optional rollout ID. * **`attempt_id`** (`Optional[str]`, default: `None` ) – Optional attempt ID. Raises: * `ValueError` – If store, rollout\_id, and attempt\_id are inconsistently provided. * `RuntimeError` – If Weave is not installed or client is uninitialized. Back to top --- # Semantic Conventions - Agent-lightning [Skip to content](https://microsoft.github.io/agent-lightning/latest/reference/semconv/#semantic-conventions) ⚠️ You are viewing development documentation. For stable documentation, please visit [https://microsoft.github.io/agent-lightning/stable/](https://microsoft.github.io/agent-lightning/stable/) Semantic Conventions[¶](https://microsoft.github.io/agent-lightning/latest/reference/semconv/#semantic-conventions "Permanent link") ===================================================================================================================================== ### `agentlightning.semconv` [¶](https://microsoft.github.io/agent-lightning/latest/reference/semconv/#agentlightning.semconv "Permanent link") Semantic conventions for Agent-lightning spans. Conventions in this file are added on demand. We generally DO NOT add new semantic conventions unless it's absolutely needed for certain algorithms or scenarios. #### `AGL_ANNOTATION = 'agentlightning.annotation'` `module-attribute` [¶](https://microsoft.github.io/agent-lightning/latest/reference/semconv/#agentlightning.semconv.AGL_ANNOTATION "Permanent link") Agent-lightning's standard span name for annotations. Annotations are minimal span units for rewards, tags, and metadatas. They are used to "annotate" a specific event or a part of rollout. #### `AGL_EXCEPTION = 'agentlightning.exception'` `module-attribute` [¶](https://microsoft.github.io/agent-lightning/latest/reference/semconv/#agentlightning.semconv.AGL_EXCEPTION "Permanent link") Agent-lightning's standard span name for exceptions. Used by the exception emitter to record exception details. #### `AGL_MESSAGE = 'agentlightning.message'` `module-attribute` [¶](https://microsoft.github.io/agent-lightning/latest/reference/semconv/#agentlightning.semconv.AGL_MESSAGE "Permanent link") Agent-lightning's standard span name for messages and logs. #### `AGL_OBJECT = 'agentlightning.object'` `module-attribute` [¶](https://microsoft.github.io/agent-lightning/latest/reference/semconv/#agentlightning.semconv.AGL_OBJECT "Permanent link") Agent-lightning's standard span name for customized objects. #### `AGL_OPERATION = 'agentlightning.operation'` `module-attribute` [¶](https://microsoft.github.io/agent-lightning/latest/reference/semconv/#agentlightning.semconv.AGL_OPERATION "Permanent link") Agent-lightning's standard span name for functions. Wrap function or code-blocks as operations. #### `AGL_REWARD = 'agentlightning.reward'` `module-attribute` [¶](https://microsoft.github.io/agent-lightning/latest/reference/semconv/#agentlightning.semconv.AGL_REWARD "Permanent link") Agent-lightning's standard span name for reward operations. #### `AGL_VIRTUAL = 'agentlightning.virtual'` `module-attribute` [¶](https://microsoft.github.io/agent-lightning/latest/reference/semconv/#agentlightning.semconv.AGL_VIRTUAL "Permanent link") Agent-lightning's standard span name for virtual operations. Mostly used in adapter when needing to represent the root or intermediate operations. #### `LightningResourceAttributes` [¶](https://microsoft.github.io/agent-lightning/latest/reference/semconv/#agentlightning.semconv.LightningResourceAttributes "Permanent link") Bases: `Enum` Resource attribute names used in Agent-lightning spans. ##### `ATTEMPT_ID = 'agentlightning.attempt_id'` `class-attribute` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/latest/reference/semconv/#agentlightning.semconv.LightningResourceAttributes.ATTEMPT_ID "Permanent link") Resource name for attempt ID in Agent-lightning spans. ##### `ROLLOUT_ID = 'agentlightning.rollout_id'` `class-attribute` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/latest/reference/semconv/#agentlightning.semconv.LightningResourceAttributes.ROLLOUT_ID "Permanent link") Resource name for rollout ID in Agent-lightning spans. ##### `SPAN_SEQUENCE_ID = 'agentlightning.span_sequence_id'` `class-attribute` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/latest/reference/semconv/#agentlightning.semconv.LightningResourceAttributes.SPAN_SEQUENCE_ID "Permanent link") Resource name for span sequence ID in Agent-lightning spans. ##### `TRACER_NAME = 'agentlightning.tracer.name'` `class-attribute` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/latest/reference/semconv/#agentlightning.semconv.LightningResourceAttributes.TRACER_NAME "Permanent link") Which tracer is used to create this span. #### `LightningSpanAttributes` [¶](https://microsoft.github.io/agent-lightning/latest/reference/semconv/#agentlightning.semconv.LightningSpanAttributes "Permanent link") Bases: `Enum` Attribute names that commonly appear in Agent-lightning spans. Exception types can't be found here because they are defined in OpenTelemetry's official semantic conventions. ##### `LINK = 'agentlightning.link'` `class-attribute` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/latest/reference/semconv/#agentlightning.semconv.LightningSpanAttributes.LINK "Permanent link") Attribute name for linking the current span to another span or other objects like requests/responses. ##### `MESSAGE_BODY = 'agentlightning.message.body'` `class-attribute` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/latest/reference/semconv/#agentlightning.semconv.LightningSpanAttributes.MESSAGE_BODY "Permanent link") Attribute name for message text in message spans. ##### `OBJECT_JSON = 'agentlightning.object.json'` `class-attribute` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/latest/reference/semconv/#agentlightning.semconv.LightningSpanAttributes.OBJECT_JSON "Permanent link") Attribute name for object serialized value (JSON) in object spans. ##### `OBJECT_LITERAL = 'agentlightning.object.literal'` `class-attribute` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/latest/reference/semconv/#agentlightning.semconv.LightningSpanAttributes.OBJECT_LITERAL "Permanent link") Attribute name for object literal value in object spans (for str, int, bool, ...). ##### `OBJECT_TYPE = 'agentlightning.object.type'` `class-attribute` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/latest/reference/semconv/#agentlightning.semconv.LightningSpanAttributes.OBJECT_TYPE "Permanent link") Attribute name for object type (full qualified name) in object spans. I think builtin types like str, int, bool, list, dict are self-explanatory and should also be qualified to use here. ##### `OPERATION_INPUT = 'agentlightning.operation.input'` `class-attribute` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/latest/reference/semconv/#agentlightning.semconv.LightningSpanAttributes.OPERATION_INPUT "Permanent link") Attribute name for operation input in operation spans. ##### `OPERATION_NAME = 'agentlightning.operation.name'` `class-attribute` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/latest/reference/semconv/#agentlightning.semconv.LightningSpanAttributes.OPERATION_NAME "Permanent link") Attribute name for operation name in operation spans, normally the function name. ##### `OPERATION_OUTPUT = 'agentlightning.operation.output'` `class-attribute` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/latest/reference/semconv/#agentlightning.semconv.LightningSpanAttributes.OPERATION_OUTPUT "Permanent link") Attribute name for operation output in operation spans. ##### `REWARD = 'agentlightning.reward'` `class-attribute` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/latest/reference/semconv/#agentlightning.semconv.LightningSpanAttributes.REWARD "Permanent link") Attribute prefix for rewards-related data in reward spans. It should be used as a prefix. For example, "agentlightning.reward.0.value" can be used to track a specific metric. See [RewardAttributes](https://microsoft.github.io/agent-lightning/latest/reference/semconv/#agentlightning.semconv.RewardAttributes " RewardAttributes") . ##### `TAG = 'agentlightning.tag'` `class-attribute` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/latest/reference/semconv/#agentlightning.semconv.LightningSpanAttributes.TAG "Permanent link") Attribute name for tagging spans with customized strings. #### `LinkAttributes` [¶](https://microsoft.github.io/agent-lightning/latest/reference/semconv/#agentlightning.semconv.LinkAttributes "Permanent link") Bases: `Enum` Standard link types used in Agent-lightning spans. The link is more powerful than [OpenTelemetry link](https://opentelemetry.io/docs/specs/otel/trace/api/#link) in that it supports linking to a queryset of spans. It can even link to span object that hasn't been emitted yet. ##### `KEY_MATCH = 'key_match'` `class-attribute` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/latest/reference/semconv/#agentlightning.semconv.LinkAttributes.KEY_MATCH "Permanent link") Linking to spans with matching attribute keys. `trace_id` and `span_id` are reserved and will be used to link to specific spans directly. For example, it can be `gen_ai.response.id` if intended to be link to a chat completion response span. Or it can be `span_id` to link to a specific span by its ID. ##### `VALUE_MATCH = 'value_match'` `class-attribute` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/latest/reference/semconv/#agentlightning.semconv.LinkAttributes.VALUE_MATCH "Permanent link") Linking to spans with corresponding attribute values on those keys. #### `LinkPydanticModel` [¶](https://microsoft.github.io/agent-lightning/latest/reference/semconv/#agentlightning.semconv.LinkPydanticModel "Permanent link") Bases: `BaseModel` A stricter implementation of LinkAttributes used in otel helpers. ##### `key_match` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/latest/reference/semconv/#agentlightning.semconv.LinkPydanticModel.key_match "Permanent link") The attribute key to match on the target spans. ##### `value_match` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/latest/reference/semconv/#agentlightning.semconv.LinkPydanticModel.value_match "Permanent link") The attribute value to match on the target spans. #### `RewardAttributes` [¶](https://microsoft.github.io/agent-lightning/latest/reference/semconv/#agentlightning.semconv.RewardAttributes "Permanent link") Bases: `Enum` Multi-dimensional reward attributes will look like: `[](https://microsoft.github.io/agent-lightning/latest/reference/semconv/#__codelineno-0-1) {"agentlightning.reward.0.name": "efficiency", "agentlightning.reward.0.value": 0.75}` The first reward in the reward list will automatically be the primary reward. If the reward list has greater than 1, it shall be a multi-dimensional case. ##### `REWARD_NAME = 'name'` `class-attribute` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/latest/reference/semconv/#agentlightning.semconv.RewardAttributes.REWARD_NAME "Permanent link") Key for each dimension in multi-dimensional reward spans. ##### `REWARD_VALUE = 'value'` `class-attribute` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/latest/reference/semconv/#agentlightning.semconv.RewardAttributes.REWARD_VALUE "Permanent link") Value for each dimension in multi-dimensional reward spans. #### `RewardPydanticModel` [¶](https://microsoft.github.io/agent-lightning/latest/reference/semconv/#agentlightning.semconv.RewardPydanticModel "Permanent link") Bases: `BaseModel` A stricter implementation of RewardAttributes used in otel helpers. ##### `name` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/latest/reference/semconv/#agentlightning.semconv.RewardPydanticModel.name "Permanent link") Name of the reward dimension. ##### `value` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/latest/reference/semconv/#agentlightning.semconv.RewardPydanticModel.value "Permanent link") Value of the reward dimension. Back to top --- # Trainer - Agent-lightning [Skip to content](https://microsoft.github.io/agent-lightning/latest/reference/trainer/#agent-lightning-trainer) ⚠️ You are viewing development documentation. For stable documentation, please visit [https://microsoft.github.io/agent-lightning/stable/](https://microsoft.github.io/agent-lightning/stable/) Agent-lightning Trainer[¶](https://microsoft.github.io/agent-lightning/latest/reference/trainer/#agent-lightning-trainer "Permanent link") =========================================================================================================================================== ### `agentlightning.Trainer` [¶](https://microsoft.github.io/agent-lightning/latest/reference/trainer/#agentlightning.Trainer "Permanent link") Bases: `TrainerLegacy` High-level orchestration layer that wires Algorithm <-> Runner <-> Store. A [`Trainer`](https://microsoft.github.io/agent-lightning/latest/reference/trainer/#agentlightning.Trainer " agentlightning.Trainer") packages the moving parts of Agent-Lightning's training loop into a single entry point: * **Algorithm lifecycle:** Instantiates or accepts an [`Algorithm`](https://microsoft.github.io/agent-lightning/latest/reference/algorithm/#agentlightning.Algorithm " agentlightning.Algorithm") , attaches the current [`LightningStore`](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.LightningStore " agentlightning.LightningStore") , adapter, and initial resources, then executes the algorithm role inside the configured execution strategy. * **Runner fleet:** Spawns one or more [`Runner`](https://microsoft.github.io/agent-lightning/latest/reference/runner/#agentlightning.Runner " agentlightning.Runner") instances (defaulting to [`LitAgentRunner`](https://microsoft.github.io/agent-lightning/latest/reference/runner/#agentlightning.LitAgentRunner " agentlightning.LitAgentRunner") ) that hydrate a [`LitAgent`](https://microsoft.github.io/agent-lightning/latest/reference/agent/#agentlightning.LitAgent " agentlightning.LitAgent") , claim rollouts, stream spans, and respect graceful termination signals from the execution strategy. * **Execution strategy:** Delegates process management to an [`ExecutionStrategy`](https://microsoft.github.io/agent-lightning/latest/reference/trainer/#agentlightning.ExecutionStrategy " agentlightning.ExecutionStrategy") (shared memory, client/server, etc.), so advanced users can swap orchestration backends without changing trainer code. * **Telemetry plumbing:** Ensures tracers, adapters, and optional [`LLMProxy`](https://microsoft.github.io/agent-lightning/latest/reference/algorithm/#agentlightning.LLMProxy " agentlightning.LLMProxy") are wired into both algorithm and runners so telemetry flows back into the store. The trainer exposes two convenience entry points: [`fit()`](https://microsoft.github.io/agent-lightning/latest/reference/trainer/#agentlightning.Trainer.fit " fit(agent, train_dataset=None, *, val_dataset=None)") for full training and [`dev()`](https://microsoft.github.io/agent-lightning/latest/reference/trainer/#agentlightning.Trainer.dev " dev(agent, train_dataset=None, *, val_dataset=None)") for fast, reproducible dry-runs. See the [Train the First Agent](https://microsoft.github.io/agent-lightning/latest/how-to/train-first-agent/) and [Write the First Algorithm](https://microsoft.github.io/agent-lightning/latest/how-to/write-first-algorithm/) tutorials for the broader context. #### `adapter = self._make_adapter(adapter_spec)` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/latest/reference/trainer/#agentlightning.Trainer.adapter "Permanent link") An instance of [`TraceAdapter`](https://microsoft.github.io/agent-lightning/latest/reference/algorithm/#agentlightning.TraceAdapter " agentlightning.TraceAdapter") to export data consumble by algorithms from traces. #### `algorithm = self._make_algorithm(algorithm)` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/latest/reference/trainer/#agentlightning.Trainer.algorithm "Permanent link") An instance of [`Algorithm`](https://microsoft.github.io/agent-lightning/latest/reference/algorithm/#agentlightning.Algorithm " agentlightning.Algorithm") to use for training. #### `daemon = daemon` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/latest/reference/trainer/#agentlightning.Trainer.daemon "Permanent link") Whether worker processes should be daemons. Daemon processes are terminated automatically when the main process exits. Deprecated. Only have effect with `fit_v0`. #### `hooks = self._normalize_hooks(hooks)` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/latest/reference/trainer/#agentlightning.Trainer.hooks "Permanent link") A sequence of [`Hook`](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.Hook " agentlightning.Hook") instances to be called at various lifecycle stages (e.g., `on_trace_start`, `on_trace_end`, `on_rollout_start`, `on_rollout_end`). #### `initial_resources = initial_resources` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/latest/reference/trainer/#agentlightning.Trainer.initial_resources "Permanent link") An instance of [`NamedResources`](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.NamedResources " agentlightning.NamedResources = Dict[str, ResourceUnion] module-attribute ") to use for bootstrapping the fit/dev process. The resources will be handed over to the algorithm. Note that not all algorithms support seeding resources. #### `llm_proxy = self._make_llm_proxy(llm_proxy, store=(self.store))` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/latest/reference/trainer/#agentlightning.Trainer.llm_proxy "Permanent link") An instance of [`LLMProxy`](https://microsoft.github.io/agent-lightning/latest/reference/algorithm/#agentlightning.LLMProxy " agentlightning.LLMProxy") to use for intercepting the LLM calls. If not provided, algorithm may create one on its own. #### `max_rollouts = max_rollouts` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/latest/reference/trainer/#agentlightning.Trainer.max_rollouts "Permanent link") Maximum number of rollouts to process per runner. If None, workers run until no more rollouts are available. #### `max_tasks = max_tasks if max_tasks is not None else max_rollouts` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/latest/reference/trainer/#agentlightning.Trainer.max_tasks "Permanent link") Maximum number of tasks to process per runner. Deprecated in favor of `max_rollouts`. #### `n_runners = n_runners` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/latest/reference/trainer/#agentlightning.Trainer.n_runners "Permanent link") Number of agent runners to run in parallel. #### `n_workers = n_runners` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/latest/reference/trainer/#agentlightning.Trainer.n_workers "Permanent link") Number of agent workers to run in parallel. Deprecated in favor of `n_runners`. #### `port = port` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/latest/reference/trainer/#agentlightning.Trainer.port "Permanent link") Port forwarded to [`ClientServerExecutionStrategy`](https://microsoft.github.io/agent-lightning/latest/reference/trainer/#agentlightning.ClientServerExecutionStrategy " agentlightning.ClientServerExecutionStrategy") . #### `runner = self._make_runner(runner)` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/latest/reference/trainer/#agentlightning.Trainer.runner "Permanent link") An instance of [`Runner`](https://microsoft.github.io/agent-lightning/latest/reference/runner/#agentlightning.Runner " agentlightning.Runner") to use for running the agent. #### `store = self._make_store(store, self.strategy)` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/latest/reference/trainer/#agentlightning.Trainer.store "Permanent link") An instance of [`LightningStore`](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.LightningStore " agentlightning.LightningStore") to use for storing tasks and traces. #### `strategy = self._make_strategy(strategy, n_runners=(self.n_runners), port=port)` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/latest/reference/trainer/#agentlightning.Trainer.strategy "Permanent link") An instance of [`ExecutionStrategy`](https://microsoft.github.io/agent-lightning/latest/reference/trainer/#agentlightning.ExecutionStrategy " agentlightning.ExecutionStrategy") to use for spawning the algorithm and runners. #### `tracer = self._make_tracer(tracer)` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/latest/reference/trainer/#agentlightning.Trainer.tracer "Permanent link") A tracer instance, or a string pointing to the class full name or a dictionary with a 'type' key that specifies the class full name and other initialization parameters. If None, a default [`AgentOpsTracer`](https://microsoft.github.io/agent-lightning/latest/reference/runner/#agentlightning.AgentOpsTracer " agentlightning.AgentOpsTracer") will be created with the current settings. #### `triplet_exporter = self.adapter` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/latest/reference/trainer/#agentlightning.Trainer.triplet_exporter "Permanent link") An instance of [`TracerTraceToTriplet`](https://microsoft.github.io/agent-lightning/latest/reference/algorithm/#agentlightning.TracerTraceToTriplet " agentlightning.TracerTraceToTriplet") to export triplets from traces, or a dictionary with the initialization parameters for the exporter. Deprecated. Use [`adapter`](https://microsoft.github.io/agent-lightning/latest/reference/trainer/#agentlightning.Trainer.adapter " adapter = self._make_adapter(adapter_spec) instance-attribute ") instead. #### `__init__(*, dev=False, n_runners=None, max_rollouts=None, initial_resources=None, tracer=None, adapter=None, store=None, runner=None, strategy=None, port=None, algorithm=None, llm_proxy=None, n_workers=None, max_tasks=None, daemon=True, triplet_exporter=None, hooks=None)` [¶](https://microsoft.github.io/agent-lightning/latest/reference/trainer/#agentlightning.Trainer.__init__ "Permanent link") Configure the trainer and resolve user-provided component specifications. Each keyword accepts either a concrete instance, a class, a callable factory, a registry string, or a lightweight configuration dictionary (see [`build_component()`](https://microsoft.github.io/agent-lightning/latest/reference/trainer/#agentlightning.build_component " agentlightning.build_component(spec, *, expected_type, spec_name, default_factory=None, allow_none=False, optional_defaults=None, dict_requires_type=True, dict_default_cls=None, type_error_fmt=None, invalid_spec_error_fmt=None, registry=None)") ). When `port` is provided it is forwarded to [`ClientServerExecutionStrategy`](https://microsoft.github.io/agent-lightning/latest/reference/trainer/#agentlightning.ClientServerExecutionStrategy " agentlightning.ClientServerExecutionStrategy") instances constructed (or supplied) for the trainer. #### `dev(agent, train_dataset=None, *, val_dataset=None)` [¶](https://microsoft.github.io/agent-lightning/latest/reference/trainer/#agentlightning.Trainer.dev "Permanent link") Exercise the infrastructure using a fast, synchronous algorithm. [`Trainer.dev`](https://microsoft.github.io/agent-lightning/latest/reference/trainer/#agentlightning.Trainer.dev " dev(agent, train_dataset=None, *, val_dataset=None)") mirrors [`fit()`](https://microsoft.github.io/agent-lightning/latest/reference/trainer/#agentlightning.Trainer.fit " fit(agent, train_dataset=None, *, val_dataset=None)") but insists on an [`Algorithm`](https://microsoft.github.io/agent-lightning/latest/reference/algorithm/#agentlightning.Algorithm " agentlightning.Algorithm") subtype that also derives from [`FastAlgorithm`](https://microsoft.github.io/agent-lightning/latest/reference/algorithm/#agentlightning.FastAlgorithm " agentlightning.FastAlgorithm") . This keeps the loop responsive for debugging while still touching the same store, runners, hooks, and tracer plumbing. If no algorithm is provided, a default [`Baseline`](https://microsoft.github.io/agent-lightning/latest/reference/algorithm/#agentlightning.Baseline " agentlightning.Baseline") algorithm will be used. Parameters: * **`agent`** (`[LitAgent](https://microsoft.github.io/agent-lightning/latest/reference/agent/#agentlightning.LitAgent " agentlightning.LitAgent (agentlightning.litagent.LitAgent)") [T_co]`) – [`LitAgent`](https://microsoft.github.io/agent-lightning/latest/reference/agent/#agentlightning.LitAgent " agentlightning.LitAgent") implementation to execute. * **`train_dataset`** (`Optional[[Dataset](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.Dataset " agentlightning.Dataset (agentlightning.types.Dataset)") [T_co]]`, default: `None` ) – Optional iterable passed to the algorithm. * **`val_dataset`** (`Optional[[Dataset](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.Dataset " agentlightning.Dataset (agentlightning.types.Dataset)") [T_co]]`, default: `None` ) – Optional iterable passed to the algorithm. Raises: * `TypeError` – If the configured algorithm does not inherit from `FastAlgorithm`. #### `fit(agent, train_dataset=None, *, val_dataset=None)` [¶](https://microsoft.github.io/agent-lightning/latest/reference/trainer/#agentlightning.Trainer.fit "Permanent link") Execute the full algorithm/runner training loop. [`Trainer.fit`](https://microsoft.github.io/agent-lightning/latest/reference/trainer/#agentlightning.Trainer.fit " fit(agent, train_dataset=None, *, val_dataset=None)") packages the algorithm and runner bundles, then hands them to the active [`ExecutionStrategy`](https://microsoft.github.io/agent-lightning/latest/reference/trainer/#agentlightning.ExecutionStrategy " agentlightning.ExecutionStrategy") . The strategy rarely returns until: * The algorithm exhausts the dataset(s) and stops enqueuing rollouts. * `max_rollouts` causes individual runners to exit. * An exception or interrupt cancels the shared [`ExecutionEvent`](https://microsoft.github.io/agent-lightning/latest/reference/trainer/#agentlightning.ExecutionEvent " agentlightning.ExecutionEvent") . Parameters: * **`agent`** (`[LitAgent](https://microsoft.github.io/agent-lightning/latest/reference/agent/#agentlightning.LitAgent " agentlightning.LitAgent (agentlightning.litagent.LitAgent)") [T_co]`) – [`LitAgent`](https://microsoft.github.io/agent-lightning/latest/reference/agent/#agentlightning.LitAgent " agentlightning.LitAgent") implementation executed by runners. * **`train_dataset`** (`Optional[[Dataset](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.Dataset " agentlightning.Dataset (agentlightning.types.Dataset)") [T_co]]`, default: `None` ) – Optional iterable of rollout inputs consumed by the algorithm. * **`val_dataset`** (`Optional[[Dataset](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.Dataset " agentlightning.Dataset (agentlightning.types.Dataset)") [T_co]]`, default: `None` ) – Optional iterable consumed by validation passes. ### `agentlightning.build_component(spec, *, expected_type, spec_name, default_factory=None, allow_none=False, optional_defaults=None, dict_requires_type=True, dict_default_cls=None, type_error_fmt=None, invalid_spec_error_fmt=None, registry=None)` [¶](https://microsoft.github.io/agent-lightning/latest/reference/trainer/#agentlightning.build_component "Permanent link") `[](https://microsoft.github.io/agent-lightning/latest/reference/trainer/#__codelineno-0-1) build_component( [](https://microsoft.github.io/agent-lightning/latest/reference/trainer/#__codelineno-0-2) spec: Union[ [](https://microsoft.github.io/agent-lightning/latest/reference/trainer/#__codelineno-0-3) T, [](https://microsoft.github.io/agent-lightning/latest/reference/trainer/#__codelineno-0-4) str, [](https://microsoft.github.io/agent-lightning/latest/reference/trainer/#__codelineno-0-5) Dict[str, Any], [](https://microsoft.github.io/agent-lightning/latest/reference/trainer/#__codelineno-0-6) type[T], [](https://microsoft.github.io/agent-lightning/latest/reference/trainer/#__codelineno-0-7) Callable[[], T], [](https://microsoft.github.io/agent-lightning/latest/reference/trainer/#__codelineno-0-8) None, [](https://microsoft.github.io/agent-lightning/latest/reference/trainer/#__codelineno-0-9) ], [](https://microsoft.github.io/agent-lightning/latest/reference/trainer/#__codelineno-0-10) *, [](https://microsoft.github.io/agent-lightning/latest/reference/trainer/#__codelineno-0-11) expected_type: type[T], [](https://microsoft.github.io/agent-lightning/latest/reference/trainer/#__codelineno-0-12) spec_name: str, [](https://microsoft.github.io/agent-lightning/latest/reference/trainer/#__codelineno-0-13) default_factory: Callable[[], T], [](https://microsoft.github.io/agent-lightning/latest/reference/trainer/#__codelineno-0-14) allow_none: bool = ..., [](https://microsoft.github.io/agent-lightning/latest/reference/trainer/#__codelineno-0-15) optional_defaults: Optional[OptionalDefaults] = ..., [](https://microsoft.github.io/agent-lightning/latest/reference/trainer/#__codelineno-0-16) dict_requires_type: bool = ..., [](https://microsoft.github.io/agent-lightning/latest/reference/trainer/#__codelineno-0-17) dict_default_cls: type[T] | None = ..., [](https://microsoft.github.io/agent-lightning/latest/reference/trainer/#__codelineno-0-18) type_error_fmt: str | None = ..., [](https://microsoft.github.io/agent-lightning/latest/reference/trainer/#__codelineno-0-19) invalid_spec_error_fmt: str | None = ..., [](https://microsoft.github.io/agent-lightning/latest/reference/trainer/#__codelineno-0-20) registry: Optional[Dict[str, str]] = ... [](https://microsoft.github.io/agent-lightning/latest/reference/trainer/#__codelineno-0-21) ) -> T` `[](https://microsoft.github.io/agent-lightning/latest/reference/trainer/#__codelineno-0-1) build_component( [](https://microsoft.github.io/agent-lightning/latest/reference/trainer/#__codelineno-0-2) spec: Union[ [](https://microsoft.github.io/agent-lightning/latest/reference/trainer/#__codelineno-0-3) T, [](https://microsoft.github.io/agent-lightning/latest/reference/trainer/#__codelineno-0-4) str, [](https://microsoft.github.io/agent-lightning/latest/reference/trainer/#__codelineno-0-5) Dict[str, Any], [](https://microsoft.github.io/agent-lightning/latest/reference/trainer/#__codelineno-0-6) type[T], [](https://microsoft.github.io/agent-lightning/latest/reference/trainer/#__codelineno-0-7) Callable[[], T], [](https://microsoft.github.io/agent-lightning/latest/reference/trainer/#__codelineno-0-8) None, [](https://microsoft.github.io/agent-lightning/latest/reference/trainer/#__codelineno-0-9) ], [](https://microsoft.github.io/agent-lightning/latest/reference/trainer/#__codelineno-0-10) *, [](https://microsoft.github.io/agent-lightning/latest/reference/trainer/#__codelineno-0-11) expected_type: type[T], [](https://microsoft.github.io/agent-lightning/latest/reference/trainer/#__codelineno-0-12) spec_name: str, [](https://microsoft.github.io/agent-lightning/latest/reference/trainer/#__codelineno-0-13) default_factory: None = ..., [](https://microsoft.github.io/agent-lightning/latest/reference/trainer/#__codelineno-0-14) allow_none: bool, [](https://microsoft.github.io/agent-lightning/latest/reference/trainer/#__codelineno-0-15) optional_defaults: Optional[OptionalDefaults] = ..., [](https://microsoft.github.io/agent-lightning/latest/reference/trainer/#__codelineno-0-16) dict_requires_type: bool = ..., [](https://microsoft.github.io/agent-lightning/latest/reference/trainer/#__codelineno-0-17) dict_default_cls: type[T] | None = ..., [](https://microsoft.github.io/agent-lightning/latest/reference/trainer/#__codelineno-0-18) type_error_fmt: str | None = ..., [](https://microsoft.github.io/agent-lightning/latest/reference/trainer/#__codelineno-0-19) invalid_spec_error_fmt: str | None = ..., [](https://microsoft.github.io/agent-lightning/latest/reference/trainer/#__codelineno-0-20) registry: Optional[Dict[str, str]] = ... [](https://microsoft.github.io/agent-lightning/latest/reference/trainer/#__codelineno-0-21) ) -> T | None` `[](https://microsoft.github.io/agent-lightning/latest/reference/trainer/#__codelineno-0-1) build_component( [](https://microsoft.github.io/agent-lightning/latest/reference/trainer/#__codelineno-0-2) spec: Union[ [](https://microsoft.github.io/agent-lightning/latest/reference/trainer/#__codelineno-0-3) T, [](https://microsoft.github.io/agent-lightning/latest/reference/trainer/#__codelineno-0-4) str, [](https://microsoft.github.io/agent-lightning/latest/reference/trainer/#__codelineno-0-5) Dict[str, Any], [](https://microsoft.github.io/agent-lightning/latest/reference/trainer/#__codelineno-0-6) type[T], [](https://microsoft.github.io/agent-lightning/latest/reference/trainer/#__codelineno-0-7) Callable[[], T], [](https://microsoft.github.io/agent-lightning/latest/reference/trainer/#__codelineno-0-8) None, [](https://microsoft.github.io/agent-lightning/latest/reference/trainer/#__codelineno-0-9) ], [](https://microsoft.github.io/agent-lightning/latest/reference/trainer/#__codelineno-0-10) *, [](https://microsoft.github.io/agent-lightning/latest/reference/trainer/#__codelineno-0-11) expected_type: type[T], [](https://microsoft.github.io/agent-lightning/latest/reference/trainer/#__codelineno-0-12) spec_name: str, [](https://microsoft.github.io/agent-lightning/latest/reference/trainer/#__codelineno-0-13) default_factory: None = ..., [](https://microsoft.github.io/agent-lightning/latest/reference/trainer/#__codelineno-0-14) allow_none: bool = ..., [](https://microsoft.github.io/agent-lightning/latest/reference/trainer/#__codelineno-0-15) optional_defaults: Optional[OptionalDefaults] = ..., [](https://microsoft.github.io/agent-lightning/latest/reference/trainer/#__codelineno-0-16) dict_requires_type: bool = ..., [](https://microsoft.github.io/agent-lightning/latest/reference/trainer/#__codelineno-0-17) dict_default_cls: type[T] | None = ..., [](https://microsoft.github.io/agent-lightning/latest/reference/trainer/#__codelineno-0-18) type_error_fmt: str | None = ..., [](https://microsoft.github.io/agent-lightning/latest/reference/trainer/#__codelineno-0-19) invalid_spec_error_fmt: str | None = ..., [](https://microsoft.github.io/agent-lightning/latest/reference/trainer/#__codelineno-0-20) registry: Optional[Dict[str, str]] = ... [](https://microsoft.github.io/agent-lightning/latest/reference/trainer/#__codelineno-0-21) ) -> T | None` Build and return a component instance from a flexible specification. This function provides a flexible way to create component instances from various input formats including direct instances, class types, factory functions, import paths, or configuration dictionaries. Parameters: * **`spec`** (`Union[T, str, Dict[str, Any], type[T], Callable[[], T], None]`) – The component specification. Can be: - An instance of expected\_type (returned as-is) - A string import path (e.g., 'module.Class') or registry key - A dict with 'type' key (import path or registry key) and constructor kwargs - A class type (will be instantiated) - A factory function (will be called) - None (uses default\_factory or returns None if allow\_none=True) * **`expected_type`** (`type[T]`) – The type that the resulting instance must be or inherit from. * **`spec_name`** (`str`) – Descriptive name for the spec, used in error messages. * **`default_factory`** (`Callable[[], T] | None`, default: `None` ) – Optional factory function called when spec is None. * **`allow_none`** (`bool`, default: `False` ) – If True, allows None to be returned when spec is None and no default\_factory is provided. * **`optional_defaults`** (`Optional[OptionalDefaults]`, default: `None` ) – Dict mapping parameter names to default values or factory functions that will be injected if the constructor accepts them. * **`dict_requires_type`** (`bool`, default: `True` ) – If True, dict specs must include a 'type' key. * **`dict_default_cls`** (`type[T] | None`, default: `None` ) – Default class to use for dict specs without a 'type' key (only used when dict\_requires\_type=False). * **`type_error_fmt`** (`str | None`, default: `None` ) – Custom format string for type validation errors. Should include {type\_name} and {expected\_type} placeholders. * **`invalid_spec_error_fmt`** (`str | None`, default: `None` ) – Custom format string for invalid spec type errors. Should include {actual\_type} and {expected\_type} placeholders. * **`registry`** (`Optional[Dict[str, str]]`, default: `None` ) – Optional mapping of short names to fully qualified import paths. When provided, string specs or dict 'type'/'name' entries are first resolved through this registry before attempting to import. Returns: * `T | None` – An instance of expected\_type, or None if allow\_none=True and spec is None * `T | None` – without a default\_factory. Raises: * `TypeError` – If the instantiated object is not an instance of expected\_type. * `ValueError` – If spec is None and neither default\_factory nor allow\_none is set, or if spec type is invalid, or if dict spec is invalid. Examples: `[](https://microsoft.github.io/agent-lightning/latest/reference/trainer/#__codelineno-0-1) >>> # Direct instance [](https://microsoft.github.io/agent-lightning/latest/reference/trainer/#__codelineno-0-2) >>> optimizer = build_component(AdamW(), expected_type=Optimizer, spec_name='optimizer') [](https://microsoft.github.io/agent-lightning/latest/reference/trainer/#__codelineno-0-3) >>> [](https://microsoft.github.io/agent-lightning/latest/reference/trainer/#__codelineno-0-4) >>> # String import path [](https://microsoft.github.io/agent-lightning/latest/reference/trainer/#__codelineno-0-5) >>> optimizer = build_component('torch.optim.AdamW', expected_type=Optimizer, spec_name='optimizer') [](https://microsoft.github.io/agent-lightning/latest/reference/trainer/#__codelineno-0-6) >>> [](https://microsoft.github.io/agent-lightning/latest/reference/trainer/#__codelineno-0-7) >>> # Dict with type and kwargs [](https://microsoft.github.io/agent-lightning/latest/reference/trainer/#__codelineno-0-8) >>> spec = {'type': 'torch.optim.AdamW', 'lr': 0.001} [](https://microsoft.github.io/agent-lightning/latest/reference/trainer/#__codelineno-0-9) >>> optimizer = build_component(spec, expected_type=Optimizer, spec_name='optimizer') [](https://microsoft.github.io/agent-lightning/latest/reference/trainer/#__codelineno-0-10) >>> [](https://microsoft.github.io/agent-lightning/latest/reference/trainer/#__codelineno-0-11) >>> # Class type [](https://microsoft.github.io/agent-lightning/latest/reference/trainer/#__codelineno-0-12) >>> optimizer = build_component(AdamW, expected_type=Optimizer, spec_name='optimizer') [](https://microsoft.github.io/agent-lightning/latest/reference/trainer/#__codelineno-0-13) >>> [](https://microsoft.github.io/agent-lightning/latest/reference/trainer/#__codelineno-0-14) >>> # Factory function [](https://microsoft.github.io/agent-lightning/latest/reference/trainer/#__codelineno-0-15) >>> optimizer = build_component(lambda: AdamW(lr=0.001), expected_type=Optimizer, [](https://microsoft.github.io/agent-lightning/latest/reference/trainer/#__codelineno-0-16) ... spec_name='optimizer')` Execution Strategy[¶](https://microsoft.github.io/agent-lightning/latest/reference/trainer/#execution-strategy "Permanent link") --------------------------------------------------------------------------------------------------------------------------------- ### `agentlightning.ExecutionStrategy` [¶](https://microsoft.github.io/agent-lightning/latest/reference/trainer/#agentlightning.ExecutionStrategy "Permanent link") Coordinate algorithm and runner bundles within a single process abstraction. Strategies decide how many worker bundles to launch, whether to communicate through shared memory or an HTTP boundary, and how to react to shutdown signals. They intentionally avoid inspecting the bundle internals; instead, each bundle remains responsible for its own scheduling semantics. Note Implementations must honor the [execute()](https://microsoft.github.io/agent-lightning/latest/reference/trainer/#agentlightning.ExecutionStrategy.execute " execute(algorithm, runner, store)") contract by propagating `KeyboardInterrupt` and ensuring resources are released when an error occurs on either side of the algorithm/runner pair. #### `execute(algorithm, runner, store)` [¶](https://microsoft.github.io/agent-lightning/latest/reference/trainer/#agentlightning.ExecutionStrategy.execute "Permanent link") Run the provided bundles using the configured orchestration model. Parameters: * **`algorithm`** (`AlgorithmBundle`) – Callable bundle responsible for algorithm execution. * **`runner`** (`RunnerBundle`) – Callable bundle for runner workers. * **`store`** (`[LightningStore](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.LightningStore " agentlightning.LightningStore (agentlightning.store.base.LightningStore)") `) – Concrete [`LightningStore`](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.LightningStore " agentlightning.LightningStore") shared across bundles. Raises: * `NotImplementedError` – Subclasses must provide the orchestration implementation. ### `agentlightning.ClientServerExecutionStrategy` [¶](https://microsoft.github.io/agent-lightning/latest/reference/trainer/#agentlightning.ClientServerExecutionStrategy "Permanent link") Bases: `[ExecutionStrategy](https://microsoft.github.io/agent-lightning/latest/reference/trainer/#agentlightning.ExecutionStrategy " agentlightning.ExecutionStrategy (agentlightning.execution.base.ExecutionStrategy)") ` Run algorithm and runner bundles as separate processes over HTTP. Execution Roles: * `"algorithm"`: Start [`LightningStoreServer`](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.LightningStoreServer " agentlightning.LightningStoreServer") in-process and execute the algorithm bundle against it. * `"runner"`: Connect to an existing server with [`LightningStoreClient`](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.LightningStoreClient " agentlightning.LightningStoreClient") and run the runner bundle locally (spawning multiple processes when requested). * `"both"`: Spawn runner processes first, then execute the algorithm and server on the same machine. This mode orchestrates the full loop locally. When `role == "both"` you may choose which side runs on the main process via `main_process`. The runner-on-main option is limited to `n_runners == 1` because each additional runner requires its own event loop and process. Warning When `main_process == "runner"` the algorithm and HTTP server execute in a child process. Store mutations remain isolated inside that process, so the original store instance passed to [execute()](https://microsoft.github.io/agent-lightning/latest/reference/trainer/#agentlightning.ExecutionStrategy.execute " execute(algorithm, runner, store)") is not updated. Abort Model (four-step escalation): 1. Cooperative stop. Every bundle receives a shared [`MultiprocessingEvent`](https://microsoft.github.io/agent-lightning/latest/reference/trainer/#agentlightning.MultiprocessingEvent " agentlightning.MultiprocessingEvent") (`stop_evt`). Any failure flips the event so peers can exit cleanly. Ctrl+C on the main process also sets the flag. 2. KeyboardInterrupt synthesis. Remaining subprocesses receive `SIGINT` to trigger `KeyboardInterrupt` handlers. 3. Termination. Stubborn processes are asked to `terminate()` (`SIGTERM` on POSIX). 4. Kill. As a last resort `kill()` is invoked (`SIGKILL` on POSIX). This mirrors the semantics implemented in [`SharedMemoryExecutionStrategy`](https://microsoft.github.io/agent-lightning/latest/reference/trainer/#agentlightning.SharedMemoryExecutionStrategy " agentlightning.SharedMemoryExecutionStrategy") but adapts them to multiple processes and the HTTP client/server boundary. #### `__init__(role=None, server_host=None, server_port=None, n_runners=1, graceful_timeout=10.0, terminate_timeout=10.0, main_process='algorithm', managed_store=None, allowed_exit_codes=(0, -15))` [¶](https://microsoft.github.io/agent-lightning/latest/reference/trainer/#agentlightning.ClientServerExecutionStrategy.__init__ "Permanent link") Configure the strategy. Parameters: * **`role`** (`Literal['algorithm', 'runner', 'both'] | None`, default: `None` ) – Which side(s) to run in this process. When omitted, the `AGL_CURRENT_ROLE` environment variable is used. * **`server_host`** (`str | None`, default: `None` ) – Interface the HTTP server binds to when running the algorithm bundle locally. Defaults to `AGL_SERVER_HOST` or `"localhost"` if unset. * **`server_port`** (`int | None`, default: `None` ) – Port for the HTTP server in "algorithm"/"both" modes. Defaults to `AGL_SERVER_PORT` or `4747` if unset. * **`n_runners`** (`int`, default: `1` ) – Number of runner processes to spawn in "runner"/"both". * **`graceful_timeout`** (`float`, default: `10.0` ) – How long to wait (seconds) after setting the stop event before escalating to signals. * **`terminate_timeout`** (`float`, default: `10.0` ) – How long to wait between escalation steps beyond the cooperative phase (re-used for SIGINT, terminate, and kill). * **`main_process`** (`Literal['algorithm', 'runner']`, default: `'algorithm'` ) – Which bundle runs on the main process when `role == "both"`. `"runner"` requires `n_runners == 1` and is primarily intended for debugging. * **`managed_store`** (`bool | None`, default: `None` ) – When `True` (default) the strategy constructs LightningStore client/server wrappers automatically. When `False` the provided `store` is passed directly to the bundles, allowing callers to manage store wrappers manually. * **`allowed_exit_codes`** (`Iterable[int]`, default: `(0, -15)` ) – Allowed exit codes for subprocesses. By default, runner can exit gracefully with code 0 or terminated by SIGTERM (-15). ### `agentlightning.SharedMemoryExecutionStrategy` [¶](https://microsoft.github.io/agent-lightning/latest/reference/trainer/#agentlightning.SharedMemoryExecutionStrategy "Permanent link") Bases: `[ExecutionStrategy](https://microsoft.github.io/agent-lightning/latest/reference/trainer/#agentlightning.ExecutionStrategy " agentlightning.ExecutionStrategy (agentlightning.execution.base.ExecutionStrategy)") ` Execute bundles in a single process with cooperative worker threads. Stop Model: * All bundles share one [`ThreadingEvent`](https://microsoft.github.io/agent-lightning/latest/reference/trainer/#agentlightning.ThreadingEvent " agentlightning.ThreadingEvent") named `stop_evt`. * Only the main thread receives `KeyboardInterrupt`. When Ctrl+C occurs we set `stop_evt`. * Any exception raised inside a bundle sets `stop_evt` so other threads can unwind cooperatively. * Once the bundle running on the main thread exits successfully the treatment depends on `main_thread`: * `"algorithm"`: the runners are asked to stop by setting `stop_evt`. * `"runner"`: the algorithm keeps running until it exits naturally. * Background threads are marked as daemons. We join them briefly and log any stragglers before shutting down. Note Signals other than `SIGINT` (such as `SIGTERM`) are not intercepted; Python's default behavior for those signals is preserved. Events[¶](https://microsoft.github.io/agent-lightning/latest/reference/trainer/#events "Permanent link") --------------------------------------------------------------------------------------------------------- ### `agentlightning.ExecutionEvent` [¶](https://microsoft.github.io/agent-lightning/latest/reference/trainer/#agentlightning.ExecutionEvent "Permanent link") Bases: `Protocol` Protocol capturing the cooperative stop contract shared by strategies. Implementations mirror the API of `threading.Event` and `multiprocessing.Event` so the rest of the execution layer can remain agnostic to the underlying concurrency primitive. Methods: `set: Signal cancellation. The call must be idempotent. clear: Reset the event to the unsignaled state. is_set: Return ``True`` when cancellation has been requested. wait: Block until the event is signaled or an optional timeout elapses.` ### `agentlightning.ThreadingEvent` [¶](https://microsoft.github.io/agent-lightning/latest/reference/trainer/#agentlightning.ThreadingEvent "Permanent link") Thread-safe implementation of [`ExecutionEvent`](https://microsoft.github.io/agent-lightning/latest/reference/trainer/#agentlightning.ExecutionEvent " agentlightning.ExecutionEvent") . ### `agentlightning.MultiprocessingEvent` [¶](https://microsoft.github.io/agent-lightning/latest/reference/trainer/#agentlightning.MultiprocessingEvent "Permanent link") Process-safe implementation of [`ExecutionEvent`](https://microsoft.github.io/agent-lightning/latest/reference/trainer/#agentlightning.ExecutionEvent " agentlightning.ExecutionEvent") . CLI Builder[¶](https://microsoft.github.io/agent-lightning/latest/reference/trainer/#cli-builder "Permanent link") ------------------------------------------------------------------------------------------------------------------- ### `agentlightning.lightning_cli(*classes)` [¶](https://microsoft.github.io/agent-lightning/latest/reference/trainer/#agentlightning.lightning_cli "Permanent link") `[](https://microsoft.github.io/agent-lightning/latest/reference/trainer/#__codelineno-0-1) lightning_cli(cls1: Type[_C1]) -> _C1` `[](https://microsoft.github.io/agent-lightning/latest/reference/trainer/#__codelineno-0-1) lightning_cli( [](https://microsoft.github.io/agent-lightning/latest/reference/trainer/#__codelineno-0-2) cls1: Type[_C1], cls2: Type[_C2] [](https://microsoft.github.io/agent-lightning/latest/reference/trainer/#__codelineno-0-3) ) -> Tuple[_C1, _C2]` `[](https://microsoft.github.io/agent-lightning/latest/reference/trainer/#__codelineno-0-1) lightning_cli( [](https://microsoft.github.io/agent-lightning/latest/reference/trainer/#__codelineno-0-2) cls1: Type[_C1], cls2: Type[_C2], cls3: Type[_C3] [](https://microsoft.github.io/agent-lightning/latest/reference/trainer/#__codelineno-0-3) ) -> Tuple[_C1, _C2, _C3]` `[](https://microsoft.github.io/agent-lightning/latest/reference/trainer/#__codelineno-0-1) lightning_cli( [](https://microsoft.github.io/agent-lightning/latest/reference/trainer/#__codelineno-0-2) cls1: Type[_C1], [](https://microsoft.github.io/agent-lightning/latest/reference/trainer/#__codelineno-0-3) cls2: Type[_C2], [](https://microsoft.github.io/agent-lightning/latest/reference/trainer/#__codelineno-0-4) cls3: Type[_C3], [](https://microsoft.github.io/agent-lightning/latest/reference/trainer/#__codelineno-0-5) cls4: Type[_C4], [](https://microsoft.github.io/agent-lightning/latest/reference/trainer/#__codelineno-0-6) ) -> Tuple[_C1, _C2, _C3, _C4]` `[](https://microsoft.github.io/agent-lightning/latest/reference/trainer/#__codelineno-0-1) lightning_cli( [](https://microsoft.github.io/agent-lightning/latest/reference/trainer/#__codelineno-0-2) *classes: Type[CliConfigurable], [](https://microsoft.github.io/agent-lightning/latest/reference/trainer/#__codelineno-0-3) ) -> Tuple[CliConfigurable, ...]` Parses command-line arguments to configure and instantiate provided CliConfigurable classes. Parameters: * **`*classes`** (`Type[CliConfigurable]`, default: `()` ) – One or more classes that inherit from CliConfigurable. Each class's **init** parameters will be exposed as command-line arguments. Returns: * `CliConfigurable | Tuple[CliConfigurable, ...]` – A tuple of instantiated objects, corresponding to the input classes in order. Logging[¶](https://microsoft.github.io/agent-lightning/latest/reference/trainer/#logging "Permanent link") ----------------------------------------------------------------------------------------------------------- ### `agentlightning.configure_logger(level=logging.INFO, name='agentlightning')` [¶](https://microsoft.github.io/agent-lightning/latest/reference/trainer/#agentlightning.configure_logger "Permanent link") Create or reset a namespaced logger with a consistent console format. This helper clears any previously attached handlers before binding a single `StreamHandler` that writes to standard output. The resulting logger does not propagate to the root logger, preventing duplicate log emission when applications compose multiple logging configurations. Danger This function is deprecated in favor of [`setup_logging`](https://microsoft.github.io/agent-lightning/latest/reference/trainer/#agentlightning.setup_logging " agentlightning.setup_logging(level='INFO', *, console=True, color=True, propagate=False, disable_existing_loggers=False, capture_warnings=False, submodule_levels=None, extra_handlers=None, formatter=None, apply_to=None, files=None)") . Parameters: * **`level`** (`int`, default: `INFO` ) – Logging level applied both to the logger and the installed handler. Defaults to `logging.INFO`. * **`name`** (`str`, default: `'agentlightning'` ) – Dotted path for the logger instance. Defaults to `"agentlightning"`. Returns: * `Logger` – Configured logger instance ready for immediate use. Examples: `[](https://microsoft.github.io/agent-lightning/latest/reference/trainer/#__codelineno-0-1) from agentlightning import configure_logger [](https://microsoft.github.io/agent-lightning/latest/reference/trainer/#__codelineno-0-2) [](https://microsoft.github.io/agent-lightning/latest/reference/trainer/#__codelineno-0-3) logger = configure_logger(level=logging.INFO) [](https://microsoft.github.io/agent-lightning/latest/reference/trainer/#__codelineno-0-4) logger.info("agent-lightning is ready!")` ### `agentlightning.setup_module_logging(level='INFO', *, name='agentlightning', console=True, color=True, propagate=False, disable_existing_loggers=False)` [¶](https://microsoft.github.io/agent-lightning/latest/reference/trainer/#agentlightning.setup_module_logging "Permanent link") Initializes and returns the base logger for `agentlightning`. This function constructs and applies a `dictConfig` configuration for the logger hierarchy rooted at `name`. It supports either rich console formatting (via `RichHandler`) or plain text formatting, based on the `color` argument. Unlike [`setup_logging`](https://microsoft.github.io/agent-lightning/latest/reference/trainer/#agentlightning.setup_logging " agentlightning.setup_logging(level='INFO', *, console=True, color=True, propagate=False, disable_existing_loggers=False, capture_warnings=False, submodule_levels=None, extra_handlers=None, formatter=None, apply_to=None, files=None)") , this function configures only a single logger namespace and does not attach extra handlers or submodule levels. It is primarily used internally by [`setup_logging`](https://microsoft.github.io/agent-lightning/latest/reference/trainer/#agentlightning.setup_logging " agentlightning.setup_logging(level='INFO', *, console=True, color=True, propagate=False, disable_existing_loggers=False, capture_warnings=False, submodule_levels=None, extra_handlers=None, formatter=None, apply_to=None, files=None)") but is also suitable for direct integration in custom logging workflows. ### `agentlightning.setup_logging(level='INFO', *, console=True, color=True, propagate=False, disable_existing_loggers=False, capture_warnings=False, submodule_levels=None, extra_handlers=None, formatter=None, apply_to=None, files=None)` [¶](https://microsoft.github.io/agent-lightning/latest/reference/trainer/#agentlightning.setup_logging "Permanent link") Configures logging for the `agentlightning` logger hierarchy. This function provides a one-stop setup utility for configuring the `agentlightning` root logger and optionally its submodules or external loggers. It supports console logging, colored rich output, per-submodule log levels, and optional handler/formatter injection. The setup is intentionally isolated: it does not modify the global root logger or loggers belonging to other libraries unless explicitly directed via `apply_to`. Parameters: * **`level`** (`int | str`, default: `'INFO'` ) – Logging level for the base `agentlightning` logger. Accepts either an integer (e.g., `logging.DEBUG`) or a string level name (e.g., `"INFO"`). Defaults to `"INFO"`. * **`console`** (`bool`, default: `True` ) – Whether to attach a console handler to the logger. Defaults to `True`. * **`color`** (`bool | Dict[str, Any]`, default: `True` ) – Enables rich-formatted output using `RichHandler` when `True` or a configuration dict. If `False`, a plain text formatter is used instead. Defaults to `True`. * **`propagate`** (`bool`, default: `False` ) – Whether `agentlightning` logs should propagate to ancestor loggers. Defaults to `False`. * **`disable_existing_loggers`** (`bool`, default: `False` ) – Passed to `logging.config.dictConfig`. If `True`, disables all existing configured loggers before applying this configuration. Defaults to `False`. * **`capture_warnings`** (`bool`, default: `False` ) – If `True`, redirects Python `warnings` emitted via the `warnings` module into the logging system. Defaults to `False`. * **`submodule_levels`** (`Optional[dict[str, int | str]]`, default: `None` ) – Mapping of submodule logger names to logging levels. If a specified submodule level is more verbose than the base level, a warning is emitted. * **`extra_handlers`** (`Optional[list[Handler]]`, default: `None` ) – A list of user-provided handlers to attach to the `agentlightning` logger. Handlers are added idempotently; duplicates are not reattached. * **`formatter`** (`Optional[Formatter]`, default: `None` ) – A formatter to apply to any handler under `agentlightning` that does not already have one assigned. Useful for customizing output without overwriting formatters on custom handlers. * **`apply_to`** (`Optional[list[str]]`, default: `None` ) – A list of additional logger names to configure identically to `agentlightning` base logger. Their handlers are replaced with copies of the base handlers, and propagation is disabled to avoid duplicate log emission. * **`files`** (`Optional[str | dict[str, str]]`, default: `None` ) – If a string, attach a FileHandler to the base `agentlightning` logger. If a dict, for each `(logger_name, filename)` pair, attach a FileHandler directly to that logger. Each file handler should use the logger's effective level at creation. Notes * On Windows, this function forces UTF-8 mode in the console to prevent issues with rich output or special characters. * Submodule loggers can generate records below the handler's emission threshold. Whether such records appear depends on both the logger's level and the handler's level. * `apply_to` loggers inherit the same handlers but do not propagate upward, yielding isolated, consistent behavior. Examples: Basic setup: `[](https://microsoft.github.io/agent-lightning/latest/reference/trainer/#__codelineno-0-1) >>> setup()` Enabling debug mode with no color: `[](https://microsoft.github.io/agent-lightning/latest/reference/trainer/#__codelineno-0-1) >>> setup(level="DEBUG", color=False)` Overriding specific submodule levels: `[](https://microsoft.github.io/agent-lightning/latest/reference/trainer/#__codelineno-0-1) >>> setup(submodule_levels={"agentlightning.io": "DEBUG"})` Attaching an additional file handler: `[](https://microsoft.github.io/agent-lightning/latest/reference/trainer/#__codelineno-0-1) >>> fh = logging.FileHandler("app.log") [](https://microsoft.github.io/agent-lightning/latest/reference/trainer/#__codelineno-0-2) >>> setup(extra_handlers=[fh])` Back to top --- # Utilities - Agent-lightning [Skip to content](https://microsoft.github.io/agent-lightning/latest/reference/utilities/#utility-references) ⚠️ You are viewing development documentation. For stable documentation, please visit [https://microsoft.github.io/agent-lightning/stable/](https://microsoft.github.io/agent-lightning/stable/) Utility References[¶](https://microsoft.github.io/agent-lightning/latest/reference/utilities/#utility-references "Permanent link") =================================================================================================================================== ID[¶](https://microsoft.github.io/agent-lightning/latest/reference/utilities/#id "Permanent link") --------------------------------------------------------------------------------------------------- ### `agentlightning.utils.id.generate_id(length)` [¶](https://microsoft.github.io/agent-lightning/latest/reference/utilities/#agentlightning.utils.id.generate_id "Permanent link") Generate a random hexadecimal ID of the given length. Parameters: * **`length`** (`int`) – The desired length of the generated ID. Must be a positive integer. Returns: * `str` – A random hexadecimal ID string of the given length. Raises: * `ValueError` – If length is not a positive integer. Metrics[¶](https://microsoft.github.io/agent-lightning/latest/reference/utilities/#metrics "Permanent link") ------------------------------------------------------------------------------------------------------------- ### `agentlightning.utils.metrics.MetricsBackend` [¶](https://microsoft.github.io/agent-lightning/latest/reference/utilities/#agentlightning.utils.metrics.MetricsBackend "Permanent link") Abstract base class for metrics backends. #### `has_prometheus()` [¶](https://microsoft.github.io/agent-lightning/latest/reference/utilities/#agentlightning.utils.metrics.MetricsBackend.has_prometheus "Permanent link") Check if the backend has prometheus support. #### `inc_counter(name, amount=1.0, labels=None)` `async` [¶](https://microsoft.github.io/agent-lightning/latest/reference/utilities/#agentlightning.utils.metrics.MetricsBackend.inc_counter "Permanent link") Increments a registered counter. Parameters: * **`name`** (`str`) – Metric name (must be registered as a counter). * **`amount`** (`float`, default: `1.0` ) – Increment amount. * **`labels`** (`Optional[LabelDict]`, default: `None` ) – Label values. Raises: * `ValueError` – If the metric is not registered, has the wrong type, or label keys do not match the registered label names. #### `observe_histogram(name, value, labels=None)` `async` [¶](https://microsoft.github.io/agent-lightning/latest/reference/utilities/#agentlightning.utils.metrics.MetricsBackend.observe_histogram "Permanent link") Records an observation for a registered histogram. Parameters: * **`name`** (`str`) – Metric name (must be registered as a histogram). * **`value`** (`float`) – Observed value. * **`labels`** (`Optional[LabelDict]`, default: `None` ) – Label values. Raises: * `ValueError` – If the metric is not registered, has the wrong type, or label keys do not match the registered label names. #### `register_counter(name, label_names=None, group_level=None)` [¶](https://microsoft.github.io/agent-lightning/latest/reference/utilities/#agentlightning.utils.metrics.MetricsBackend.register_counter "Permanent link") Registers a counter metric. Parameters: * **`name`** (`str`) – Metric name. * **`label_names`** (`Optional[Sequence[str]]`, default: `None` ) – List of label names. Order determines the truncation priority for group-level logging. * **`group_level`** (`Optional[int]`, default: `None` ) – Optional per-metric grouping depth for backends that support label grouping (Console). Global backend settings take precedence when provided. Raises: * `ValueError` – If the metric is already registered with a different type or label set. #### `register_histogram(name, label_names=None, buckets=None, group_level=None)` [¶](https://microsoft.github.io/agent-lightning/latest/reference/utilities/#agentlightning.utils.metrics.MetricsBackend.register_histogram "Permanent link") Registers a histogram metric. Parameters: * **`name`** (`str`) – Metric name. * **`label_names`** (`Optional[Sequence[str]]`, default: `None` ) – List of label names. Order determines the truncation priority for group-level logging. * **`buckets`** (`Optional[Sequence[float]]`, default: `None` ) – Bucket boundaries (exclusive upper bounds). If None, the backend may choose defaults. * **`group_level`** (`Optional[int]`, default: `None` ) – Optional per-metric grouping depth for backends that support label grouping (Console). Global backend settings take precedence when provided. Raises: * `ValueError` – If the metric is already registered with a different type or label set. ### `agentlightning.utils.metrics.ConsoleMetricsBackend` [¶](https://microsoft.github.io/agent-lightning/latest/reference/utilities/#agentlightning.utils.metrics.ConsoleMetricsBackend "Permanent link") Bases: `[MetricsBackend](https://microsoft.github.io/agent-lightning/latest/reference/utilities/#agentlightning.utils.metrics.MetricsBackend " agentlightning.utils.metrics.MetricsBackend") ` Console backend with sliding-window aggregations and label grouping. This backend: * Requires explicit metric registration. * Stores timestamped events per (metric\_name, labels) key. * Computes rate and percentiles (P50, P95, P99) over a sliding time window. * Uses a single global logging decision: when logging is triggered, it logs all metric groups, not just the one being updated. Rate is always per second. Label grouping: When logging, label dictionaries are truncated to the first `group_level` label pairs (following the registered label order) and metrics with identical truncated labels are aggregated together. For example: `[](https://microsoft.github.io/agent-lightning/latest/reference/utilities/#__codelineno-0-1) labels = {"method": "GET", "path": "/", "status": "200"} [](https://microsoft.github.io/agent-lightning/latest/reference/utilities/#__codelineno-0-2) group_level = 2 # aggregated labels {"method": "GET", "path": "/"}` If `group_level` is None or < 1, all label combinations for a metric are merged into a single log entry (equivalent to grouping by zero labels). Individual counters or histograms can set their own `group_level` during registration; those values apply only when the backend-level `group_level` is unset, allowing selective overrides. Thread-safety: Runtime updates and snapshotting use two aiologic locks: one for mutating shared state and another that serializes the global logging decision/snapshot capture so other tasks can continue writing. Metric registration happens during initialization, so it is intentionally left lock-free; this assumption is documented here to avoid blocking writes unnecessarily. #### `__init__(window_seconds=60.0, log_interval_seconds=10.0, group_level=None)` [¶](https://microsoft.github.io/agent-lightning/latest/reference/utilities/#agentlightning.utils.metrics.ConsoleMetricsBackend.__init__ "Permanent link") Initializes ConsoleMetricsBackend. Parameters: * **`window_seconds`** (`Optional[float]`, default: `60.0` ) – Sliding window size (in seconds) used when computing rate and percentiles. If None, all in-memory events are used. * **`log_interval_seconds`** (`float`, default: `10.0` ) – Minimum time (in seconds) between log bursts. When the interval elapses, the next metric event triggers a snapshot and logging of all metrics. * **`group_level`** (`Optional[int]`, default: `None` ) – Label grouping depth. When logging, only the first `group_level` labels (following registered order) are retained and metric events sharing those labels are aggregated. If None or < 1, all label combinations collapse into a single group per metric. #### `inc_counter(name, amount=1.0, labels=None)` `async` [¶](https://microsoft.github.io/agent-lightning/latest/reference/utilities/#agentlightning.utils.metrics.ConsoleMetricsBackend.inc_counter "Permanent link") Increments a registered counter metric. See base class for behavior and error conditions. #### `observe_histogram(name, value, labels=None)` `async` [¶](https://microsoft.github.io/agent-lightning/latest/reference/utilities/#agentlightning.utils.metrics.ConsoleMetricsBackend.observe_histogram "Permanent link") Records an observation for a registered histogram metric. See base class for behavior and error conditions. #### `register_counter(name, label_names=None, group_level=None)` [¶](https://microsoft.github.io/agent-lightning/latest/reference/utilities/#agentlightning.utils.metrics.ConsoleMetricsBackend.register_counter "Permanent link") Registers a counter metric. See base class for argument documentation. #### `register_histogram(name, label_names=None, buckets=None, group_level=None)` [¶](https://microsoft.github.io/agent-lightning/latest/reference/utilities/#agentlightning.utils.metrics.ConsoleMetricsBackend.register_histogram "Permanent link") Registers a histogram metric. See base class for argument documentation. ### `agentlightning.utils.metrics.PrometheusMetricsBackend` [¶](https://microsoft.github.io/agent-lightning/latest/reference/utilities/#agentlightning.utils.metrics.PrometheusMetricsBackend "Permanent link") Bases: `[MetricsBackend](https://microsoft.github.io/agent-lightning/latest/reference/utilities/#agentlightning.utils.metrics.MetricsBackend " agentlightning.utils.metrics.MetricsBackend") ` Metrics backend that forwards events to prometheus\_client. All metrics must be registered before use. This backend does not compute any aggregations; it only updates Prometheus metrics. Thread-safety: Registration is protected by a lock. Metric updates assume metrics are registered during initialization and then remain stable. Due to the nature of Prometheus, this backend is only suitable for recording high-volume metrics. Low-volume metrics might be lost if the event has only appeared once. #### `__init__()` [¶](https://microsoft.github.io/agent-lightning/latest/reference/utilities/#agentlightning.utils.metrics.PrometheusMetricsBackend.__init__ "Permanent link") Initializes PrometheusMetricsBackend. Raises: * `ImportError` – If prometheus\_client is not installed. #### `has_prometheus()` [¶](https://microsoft.github.io/agent-lightning/latest/reference/utilities/#agentlightning.utils.metrics.PrometheusMetricsBackend.has_prometheus "Permanent link") Check if the backend has prometheus support. #### `inc_counter(name, amount=1.0, labels=None)` `async` [¶](https://microsoft.github.io/agent-lightning/latest/reference/utilities/#agentlightning.utils.metrics.PrometheusMetricsBackend.inc_counter "Permanent link") Increments a registered Prometheus counter. #### `observe_histogram(name, value, labels=None)` `async` [¶](https://microsoft.github.io/agent-lightning/latest/reference/utilities/#agentlightning.utils.metrics.PrometheusMetricsBackend.observe_histogram "Permanent link") Records an observation for a registered Prometheus histogram. #### `register_counter(name, label_names=None, group_level=None)` [¶](https://microsoft.github.io/agent-lightning/latest/reference/utilities/#agentlightning.utils.metrics.PrometheusMetricsBackend.register_counter "Permanent link") Registers a Prometheus counter metric. #### `register_histogram(name, label_names=None, buckets=None, group_level=None)` [¶](https://microsoft.github.io/agent-lightning/latest/reference/utilities/#agentlightning.utils.metrics.PrometheusMetricsBackend.register_histogram "Permanent link") Registers a Prometheus histogram metric. ### `agentlightning.utils.metrics.MultiMetricsBackend` [¶](https://microsoft.github.io/agent-lightning/latest/reference/utilities/#agentlightning.utils.metrics.MultiMetricsBackend "Permanent link") Bases: `[MetricsBackend](https://microsoft.github.io/agent-lightning/latest/reference/utilities/#agentlightning.utils.metrics.MetricsBackend " agentlightning.utils.metrics.MetricsBackend") ` Metrics backend that forwards calls to multiple underlying backends. #### `__init__(backends)` [¶](https://microsoft.github.io/agent-lightning/latest/reference/utilities/#agentlightning.utils.metrics.MultiMetricsBackend.__init__ "Permanent link") Initializes MultiMetricsBackend. Parameters: * **`backends`** (`Sequence[[MetricsBackend](https://microsoft.github.io/agent-lightning/latest/reference/utilities/#agentlightning.utils.metrics.MetricsBackend " agentlightning.utils.metrics.MetricsBackend") ]`) – Sequence of underlying backends. Raises: * `ValueError` – If no backends are provided. #### `has_prometheus()` [¶](https://microsoft.github.io/agent-lightning/latest/reference/utilities/#agentlightning.utils.metrics.MultiMetricsBackend.has_prometheus "Permanent link") Check if the backend has prometheus support. #### `inc_counter(name, amount=1.0, labels=None)` `async` [¶](https://microsoft.github.io/agent-lightning/latest/reference/utilities/#agentlightning.utils.metrics.MultiMetricsBackend.inc_counter "Permanent link") Increments a counter metric in all underlying backends. #### `observe_histogram(name, value, labels=None)` `async` [¶](https://microsoft.github.io/agent-lightning/latest/reference/utilities/#agentlightning.utils.metrics.MultiMetricsBackend.observe_histogram "Permanent link") Records a histogram observation in all underlying backends. #### `register_counter(name, label_names=None, group_level=None)` [¶](https://microsoft.github.io/agent-lightning/latest/reference/utilities/#agentlightning.utils.metrics.MultiMetricsBackend.register_counter "Permanent link") Registers a counter metric in all underlying backends. #### `register_histogram(name, label_names=None, buckets=None, group_level=None)` [¶](https://microsoft.github.io/agent-lightning/latest/reference/utilities/#agentlightning.utils.metrics.MultiMetricsBackend.register_histogram "Permanent link") Registers a histogram metric in all underlying backends. ### `agentlightning.utils.metrics.setup_multiprocess_prometheus()` [¶](https://microsoft.github.io/agent-lightning/latest/reference/utilities/#agentlightning.utils.metrics.setup_multiprocess_prometheus "Permanent link") Set up prometheus multiprocessing directory if not already configured. ### `agentlightning.utils.metrics.get_prometheus_registry()` [¶](https://microsoft.github.io/agent-lightning/latest/reference/utilities/#agentlightning.utils.metrics.get_prometheus_registry "Permanent link") Get the appropriate prometheus registry based on multiprocessing configuration. ### `agentlightning.utils.metrics.shutdown_metrics(server=None, worker=None, *args, **kwargs)` [¶](https://microsoft.github.io/agent-lightning/latest/reference/utilities/#agentlightning.utils.metrics.shutdown_metrics "Permanent link") Shutdown prometheus metrics. Server Launcher[¶](https://microsoft.github.io/agent-lightning/latest/reference/utilities/#server-launcher "Permanent link") ----------------------------------------------------------------------------------------------------------------------------- ### `agentlightning.utils.server_launcher.PythonServerLauncher` [¶](https://microsoft.github.io/agent-lightning/latest/reference/utilities/#agentlightning.utils.server_launcher.PythonServerLauncher "Permanent link") Unified launcher for FastAPI, using uvicorn or gunicorn per mode/worker count. See [`PythonServerLauncherArgs`](https://microsoft.github.io/agent-lightning/latest/reference/utilities/#agentlightning.utils.server_launcher.PythonServerLauncherArgs " agentlightning.utils.server_launcher.PythonServerLauncherArgs dataclass ") for configuration options. Parameters: * **`app`** (`FastAPI`) – The FastAPI app to launch. * **`args`** (`[PythonServerLauncherArgs](https://microsoft.github.io/agent-lightning/latest/reference/utilities/#agentlightning.utils.server_launcher.PythonServerLauncherArgs " agentlightning.utils.server_launcher.PythonServerLauncherArgs dataclass ") `) – The configuration for the server. * **`serve_context`** (`Optional[AsyncContextManager[Any]]`, default: `None` ) – An optional context manager to apply around the server startup. #### `access_endpoint` `property` [¶](https://microsoft.github.io/agent-lightning/latest/reference/utilities/#agentlightning.utils.server_launcher.PythonServerLauncher.access_endpoint "Permanent link") Return a loopback-friendly URL so health checks succeed even when binding to 0.0.0.0. #### `endpoint` `property` [¶](https://microsoft.github.io/agent-lightning/latest/reference/utilities/#agentlightning.utils.server_launcher.PythonServerLauncher.endpoint "Permanent link") Return the externally advertised host:port pair regardless of accessibility. #### `health_url` `property` [¶](https://microsoft.github.io/agent-lightning/latest/reference/utilities/#agentlightning.utils.server_launcher.PythonServerLauncher.health_url "Permanent link") Build the absolute health-check endpoint from args, if one is configured. #### `__getstate__()` [¶](https://microsoft.github.io/agent-lightning/latest/reference/utilities/#agentlightning.utils.server_launcher.PythonServerLauncher.__getstate__ "Permanent link") Control pickling to prevent server state from being sent to subprocesses. #### `__init__(app, args, serve_context=None)` [¶](https://microsoft.github.io/agent-lightning/latest/reference/utilities/#agentlightning.utils.server_launcher.PythonServerLauncher.__init__ "Permanent link") Initialize the launcher with the FastAPI app, configuration, and optional serve context. #### `is_running()` [¶](https://microsoft.github.io/agent-lightning/latest/reference/utilities/#agentlightning.utils.server_launcher.PythonServerLauncher.is_running "Permanent link") Return True if the server has been started and not yet stopped. #### `reload()` `async` [¶](https://microsoft.github.io/agent-lightning/latest/reference/utilities/#agentlightning.utils.server_launcher.PythonServerLauncher.reload "Permanent link") Restart the server by stopping it if necessary and invoking start again. #### `run_forever()` `async` [¶](https://microsoft.github.io/agent-lightning/latest/reference/utilities/#agentlightning.utils.server_launcher.PythonServerLauncher.run_forever "Permanent link") Start the server and block the caller until it exits, respecting the configured mode. #### `start()` `async` [¶](https://microsoft.github.io/agent-lightning/latest/reference/utilities/#agentlightning.utils.server_launcher.PythonServerLauncher.start "Permanent link") Starts the server according to launch\_mode and n\_workers. #### `stop()` `async` [¶](https://microsoft.github.io/agent-lightning/latest/reference/utilities/#agentlightning.utils.server_launcher.PythonServerLauncher.stop "Permanent link") Stop the server using the inverse of whatever launch mode was used to start it. ### `agentlightning.utils.server_launcher.PythonServerLauncherArgs` `dataclass` [¶](https://microsoft.github.io/agent-lightning/latest/reference/utilities/#agentlightning.utils.server_launcher.PythonServerLauncherArgs "Permanent link") #### `access_host = None` `class-attribute` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/latest/reference/utilities/#agentlightning.utils.server_launcher.PythonServerLauncherArgs.access_host "Permanent link") The hostname or IP address to advertise to the client. If not provided, the server will use the default outbound IPv4 address for this machine. #### `access_log = False` `class-attribute` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/latest/reference/utilities/#agentlightning.utils.server_launcher.PythonServerLauncherArgs.access_log "Permanent link") Whether to turn on access logs. #### `healthcheck_url = None` `class-attribute` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/latest/reference/utilities/#agentlightning.utils.server_launcher.PythonServerLauncherArgs.healthcheck_url "Permanent link") The health check URL to use. If not provided, the server will not be checked for healthiness after starting. #### `host = None` `class-attribute` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/latest/reference/utilities/#agentlightning.utils.server_launcher.PythonServerLauncherArgs.host "Permanent link") The hostname or IP address to bind the server to. #### `kill_unhealthy_server = True` `class-attribute` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/latest/reference/utilities/#agentlightning.utils.server_launcher.PythonServerLauncherArgs.kill_unhealthy_server "Permanent link") Whether to kill the server if it is not healthy after startup. This setting is ignored when `launch_mode` is not `asyncio`. #### `launch_mode = 'asyncio'` `class-attribute` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/latest/reference/utilities/#agentlightning.utils.server_launcher.PythonServerLauncherArgs.launch_mode "Permanent link") The launch mode. `asyncio` is the default mode to runs the server in the current thread. `thread` runs the server in a separate thread. `mp` runs the server in a separate process. #### `log_level = logging.INFO` `class-attribute` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/latest/reference/utilities/#agentlightning.utils.server_launcher.PythonServerLauncherArgs.log_level "Permanent link") The log level to use. #### `n_workers = 1` `class-attribute` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/latest/reference/utilities/#agentlightning.utils.server_launcher.PythonServerLauncherArgs.n_workers "Permanent link") The number of workers to run in the server. Only applicable for `mp` mode. When `n_workers > 1`, the server will be run using Gunicorn. #### `port = None` `class-attribute` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/latest/reference/utilities/#agentlightning.utils.server_launcher.PythonServerLauncherArgs.port "Permanent link") The TCP port to listen on. If not provided, the server will use a random available port. #### `process_join_timeout = 10.0` `class-attribute` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/latest/reference/utilities/#agentlightning.utils.server_launcher.PythonServerLauncherArgs.process_join_timeout "Permanent link") The timeout to wait for the process to join. #### `startup_timeout = 60.0` `class-attribute` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/latest/reference/utilities/#agentlightning.utils.server_launcher.PythonServerLauncherArgs.startup_timeout "Permanent link") The timeout to wait for the server to start up. #### `thread_join_timeout = 10.0` `class-attribute` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/latest/reference/utilities/#agentlightning.utils.server_launcher.PythonServerLauncherArgs.thread_join_timeout "Permanent link") The timeout to wait for the thread to join. #### `timeout_keep_alive = 30` `class-attribute` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/latest/reference/utilities/#agentlightning.utils.server_launcher.PythonServerLauncherArgs.timeout_keep_alive "Permanent link") The timeout to keep the connection alive. ### `agentlightning.utils.server_launcher.LaunchMode = Literal['asyncio', 'thread', 'mp']` `module-attribute` [¶](https://microsoft.github.io/agent-lightning/latest/reference/utilities/#agentlightning.utils.server_launcher.LaunchMode "Permanent link") The launch mode for the server. OpenTelemetry[¶](https://microsoft.github.io/agent-lightning/latest/reference/utilities/#opentelemetry "Permanent link") ------------------------------------------------------------------------------------------------------------------------- ### `agentlightning.utils.otel.full_qualified_name(obj)` [¶](https://microsoft.github.io/agent-lightning/latest/reference/utilities/#agentlightning.utils.otel.full_qualified_name "Permanent link") ### `agentlightning.utils.otel.get_tracer_provider(inspect=True)` [¶](https://microsoft.github.io/agent-lightning/latest/reference/utilities/#agentlightning.utils.otel.get_tracer_provider "Permanent link") Get the OpenTelemetry tracer provider configured for Agent Lightning. Parameters: * **`inspect`** (`bool`, default: `True` ) – Whether to inspect the tracer provider and log its configuration. When it's on, make sure you also set the logger level to DEBUG to see the logs. ### `agentlightning.utils.otel.get_tracer(use_active_span_processor=True)` [¶](https://microsoft.github.io/agent-lightning/latest/reference/utilities/#agentlightning.utils.otel.get_tracer "Permanent link") Resolve the OpenTelemetry tracer configured for Agent Lightning. Parameters: * **`use_active_span_processor`** (`bool`, default: `True` ) – Whether to use the active span processor. Returns: * `Tracer` – OpenTelemetry tracer tagged with the `agentlightning` instrumentation name. Raises: * `RuntimeError` – If OpenTelemetry was not initialized before calling this helper. ### `agentlightning.utils.otel.make_tag_attributes(tags)` [¶](https://microsoft.github.io/agent-lightning/latest/reference/utilities/#agentlightning.utils.otel.make_tag_attributes "Permanent link") Convert a list of tags into flattened attributes for span tagging. There is no syntax enforced for tags, they are just strings. For example: `[](https://microsoft.github.io/agent-lightning/latest/reference/utilities/#__codelineno-0-1) ["gen_ai.model:gpt-4", "reward.extrinsic"]` ### `agentlightning.utils.otel.extract_tags_from_attributes(attributes)` [¶](https://microsoft.github.io/agent-lightning/latest/reference/utilities/#agentlightning.utils.otel.extract_tags_from_attributes "Permanent link") Extract tag attributes from flattened span attributes. Parameters: * **`attributes`** (`Dict[str, Any]`) – A dictionary of flattened span attributes. ### `agentlightning.utils.otel.make_link_attributes(links)` [¶](https://microsoft.github.io/agent-lightning/latest/reference/utilities/#agentlightning.utils.otel.make_link_attributes "Permanent link") Convert a dictionary of links into flattened attributes for span linking. Links example: `[](https://microsoft.github.io/agent-lightning/latest/reference/utilities/#__codelineno-0-1) { [](https://microsoft.github.io/agent-lightning/latest/reference/utilities/#__codelineno-0-2) "gen_ai.response.id": "response-123", [](https://microsoft.github.io/agent-lightning/latest/reference/utilities/#__codelineno-0-3) "span_id": "abcd-efgh-ijkl", [](https://microsoft.github.io/agent-lightning/latest/reference/utilities/#__codelineno-0-4) }` ### `agentlightning.utils.otel.query_linked_spans(spans, links)` [¶](https://microsoft.github.io/agent-lightning/latest/reference/utilities/#agentlightning.utils.otel.query_linked_spans "Permanent link") Query spans that are linked by the given link attributes. Parameters: * **`spans`** (`Sequence[T_SpanLike]`) – A sequence of spans to search. * **`links`** (`List[[LinkPydanticModel](https://microsoft.github.io/agent-lightning/latest/reference/semconv/#agentlightning.semconv.LinkPydanticModel " LinkPydanticModel (agentlightning.semconv.LinkPydanticModel)") ]`) – A list of link attributes to match. Returns: * `List[T_SpanLike]` – A list of spans that match the given link attributes. ### `agentlightning.utils.otel.extract_links_from_attributes(attributes)` [¶](https://microsoft.github.io/agent-lightning/latest/reference/utilities/#agentlightning.utils.otel.extract_links_from_attributes "Permanent link") Extract link attributes from flattened span attributes. Parameters: * **`attributes`** (`Dict[str, Any]`) – A dictionary of flattened span attributes. ### `agentlightning.utils.otel.filter_attributes(attributes, prefix)` [¶](https://microsoft.github.io/agent-lightning/latest/reference/utilities/#agentlightning.utils.otel.filter_attributes "Permanent link") Filter attributes that start with the given prefix. The attribute must start with `prefix.` or be exactly `prefix` to be included. Parameters: * **`attributes`** (`Dict[str, Any]`) – A dictionary of span attributes. * **`prefix`** (`str`) – The prefix to filter by. Returns: * `Dict[str, Any]` – A dictionary of attributes that start with the given prefix. ### `agentlightning.utils.otel.filter_and_unflatten_attributes(attributes, prefix)` [¶](https://microsoft.github.io/agent-lightning/latest/reference/utilities/#agentlightning.utils.otel.filter_and_unflatten_attributes "Permanent link") Filter attributes that start with the given prefix and unflatten them. The prefix will be removed during unflattening. Parameters: * **`attributes`** (`Dict[str, Any]`) – A dictionary of span attributes. * **`prefix`** (`str`) – The prefix to filter by. Returns: * `Union[Dict[str, Any], List[Any]]` – A nested dictionary or list of attributes that start with the given prefix. ### `agentlightning.utils.otel.flatten_attributes(nested_data, *, expand_leaf_lists=False)` [¶](https://microsoft.github.io/agent-lightning/latest/reference/utilities/#agentlightning.utils.otel.flatten_attributes "Permanent link") Flatten a nested dictionary or list into a flat dictionary with dotted keys. This function recursively traverses dictionaries and lists, producing a flat key-value mapping where nested paths are represented via dot-separated keys. Lists are indexed numerically. Example: `>>> flatten_attributes({"a": {"b": 1, "c": [2, 3]}}, expand_leaf_lists=True) {"a.b": 1, "a.c.0": 2, "a.c.1": 3}` Parameters: * **`nested_data`** (`Union[Dict[str, Any], List[Any]]`) – A nested structure composed of dictionaries, lists, or primitive values. * **`expand_leaf_lists`** (`bool`, default: `False` ) – Whether to expand lists composed only of primitive values. When `False` (the default), lists of str/int/float/bool are treated as leaf values and stored without enumerating their indices. Returns: * `Dict[str, Any]` – A flat dictionary mapping dotted-string paths to primitive values. ### `agentlightning.utils.otel.unflatten_attributes(flat_data)` [¶](https://microsoft.github.io/agent-lightning/latest/reference/utilities/#agentlightning.utils.otel.unflatten_attributes "Permanent link") Reconstruct a nested dictionary/list structure from a flat dictionary. Keys are dot-separated paths. Segments that are digit strings will only become list indices if _all_ keys in that dict form a consecutive 0..n-1 range. Otherwise they remain dict keys. Example: `>>> unflatten_attributes({"a.b": 1, "a.c.0": 2, "a.c.1": 3}) {"a": {"b": 1, "c": [2, 3]}}` Parameters: * **`flat_data`** (`Dict[str, Any]`) – A dictionary whose keys are dot-separated paths and whose values are primitive data elements. Returns: * `Union[Dict[str, Any], List[Any]]` – A nested dictionary (and lists where appropriate) corresponding to * `Union[Dict[str, Any], List[Any]]` – the flattened structure. ### `agentlightning.utils.otel.sanitize_attribute_value(object, force=True)` [¶](https://microsoft.github.io/agent-lightning/latest/reference/utilities/#agentlightning.utils.otel.sanitize_attribute_value "Permanent link") Sanitize an attribute value to be a valid OpenTelemetry attribute value. ### `agentlightning.utils.otel.sanitize_attributes(attributes, force=True)` [¶](https://microsoft.github.io/agent-lightning/latest/reference/utilities/#agentlightning.utils.otel.sanitize_attributes "Permanent link") Sanitize a dictionary of attributes to be a valid OpenTelemetry attributes. Parameters: * **`attributes`** (`Dict[str, Any]`) – A dictionary of attributes to sanitize. * **`force`** (`bool`, default: `True` ) – Whether to force sanitization even when the value is not JSON serializable. ### `agentlightning.utils.otel.sanitize_list_attribute_sanity(maybe_list)` [¶](https://microsoft.github.io/agent-lightning/latest/reference/utilities/#agentlightning.utils.otel.sanitize_list_attribute_sanity "Permanent link") Try to sanitize a list of attributes to be a valid OpenTelemetry attribute value. Raise error if the list contains multiple types of primitive values. ### `agentlightning.utils.otel.check_attributes_sanity(attributes)` [¶](https://microsoft.github.io/agent-lightning/latest/reference/utilities/#agentlightning.utils.otel.check_attributes_sanity "Permanent link") Check if a dictionary of attributes is a valid OpenTelemetry attributes. ### `agentlightning.utils.otel.format_exception_attributes(exception)` [¶](https://microsoft.github.io/agent-lightning/latest/reference/utilities/#agentlightning.utils.otel.format_exception_attributes "Permanent link") Format an exception into a dictionary of attributes. OTLP[¶](https://microsoft.github.io/agent-lightning/latest/reference/utilities/#otlp "Permanent link") ------------------------------------------------------------------------------------------------------- ### `agentlightning.utils.otlp.handle_otlp_export(request, request_message_cls, response_message_cls, message_callback, signal_name)` `async` [¶](https://microsoft.github.io/agent-lightning/latest/reference/utilities/#agentlightning.utils.otlp.handle_otlp_export "Permanent link") Generic handler for /v1/traces, /v1/metrics, /v1/logs. Convert the OTLP Protobuf request to a JSON-like object. ### `agentlightning.utils.otlp.spans_from_proto(request, sequence_id_bulk_issuer)` `async` [¶](https://microsoft.github.io/agent-lightning/latest/reference/utilities/#agentlightning.utils.otlp.spans_from_proto "Permanent link") Parse an OTLP proto payload into List\[Span\]. A store is needed here for generating a sequence ID for each span. System Snapshot[¶](https://microsoft.github.io/agent-lightning/latest/reference/utilities/#system-snapshot "Permanent link") ----------------------------------------------------------------------------------------------------------------------------- ### `agentlightning.utils.system_snapshot.system_snapshot(include_gpu=False)` [¶](https://microsoft.github.io/agent-lightning/latest/reference/utilities/#agentlightning.utils.system_snapshot.system_snapshot "Permanent link") Capture a snapshot of the system's hardware and software information. Parameters: * **`include_gpu`** (`bool`, default: `False` ) – Whether to include GPU information. Returns: * `Dict[str, Any]` – A dictionary containing the system's hardware and software information. Back to top --- # Debugging - Agent-lightning [Skip to content](https://microsoft.github.io/agent-lightning/latest/tutorials/debug/#debugging-and-troubleshooting) ⚠️ You are viewing development documentation. For stable documentation, please visit [https://microsoft.github.io/agent-lightning/stable/](https://microsoft.github.io/agent-lightning/stable/) Debugging and Troubleshooting[¶](https://microsoft.github.io/agent-lightning/latest/tutorials/debug/#debugging-and-troubleshooting "Permanent link") ===================================================================================================================================================== When you train your own agent with Agent-lightning, most failures surface because the agent logic is brittle or simply incorrect. Debugging becomes easier when you peel back the stack: start by driving the rollout logic on its own, dry-run the trainer loop, and only then bring the full algorithm and runner topology online. The [`examples/apo/apo_debug.py`](https://github.com/microsoft/agent-lightning/tree/c746af2f76bebcae56007a59c223cdace411c89b/examples/apo/apo_debug.py) script demonstrates these techniques; this guide expands on each approach and helps you decide when to reach for them. Debugging with Dashboard[¶](https://microsoft.github.io/agent-lightning/latest/tutorials/debug/#debugging-with-dashboard "Permanent link") ------------------------------------------------------------------------------------------------------------------------------------------- When you launch an experiment with [`Trainer.fit`](https://microsoft.github.io/agent-lightning/latest/reference/trainer/#agentlightning.Trainer.fit " fit(agent, train_dataset=None, *, val_dataset=None)") or start an isolated store via [`agl store`](https://microsoft.github.io/agent-lightning/latest/reference/cli/) , the terminal prints a message similar to: `[](https://microsoft.github.io/agent-lightning/latest/tutorials/debug/#__codelineno-0-1) INFO Agent-lightning dashboard will be available at http://192.168.0.107:4747` Visit that URL, and you will see the Agent-lightning dashboard: ![Dashboard](https://microsoft.github.io/agent-lightning/latest/assets/dashboard-page-rollouts.png) The dashboard surfaces everything stored inside [the store](https://microsoft.github.io/agent-lightning/latest/deep-dive/store/) . Because the store mediates interactions between algorithms and runners, inspecting it often reveals which side is causing issues such as stale rollouts, unresponsive workers, or empty traces. For example, the VERL algorithm may receive no token IDs and emit `cannot reshape tensor of 0 elements into shape [1, 0, -1, 128] because the unspecified dimension size -1 can be any value and is ambiguous` ([Issue #50](https://github.com/microsoft/agent-lightning/issues/50) , [Issue #76](https://github.com/microsoft/agent-lightning/issues/76) ). Several scenarios can produce that error: the runner might not produce trace spans at all, it might produce spans without token IDs, or the IDs may be present but formatted incorrectly. Inspecting the dashboard traces helps you pinpoint which condition applies. ![Dashboard Traces Page](https://microsoft.github.io/agent-lightning/latest/assets/dashboard-page-traces.png) By checking whether the trace span is empty and whether token IDs appear in the span attributes, you can narrow the issue to either the runner (agent) side or the algorithm side. Then apply the techniques below to debug the faulty component. Debug-level Logging[¶](https://microsoft.github.io/agent-lightning/latest/tutorials/debug/#debug-level-logging "Permanent link") --------------------------------------------------------------------------------------------------------------------------------- Starting from v0.3, detailed signals such as store server access logs, runner lifecycle logs, and span payloads only appear when the log level is `DEBUG` so the default output stays readable. Enable debug-level logging by adding the following snippet near the top of your script: `[](https://microsoft.github.io/agent-lightning/latest/tutorials/debug/#__codelineno-1-1) import agentlightning as agl [](https://microsoft.github.io/agent-lightning/latest/tutorials/debug/#__codelineno-1-2) [](https://microsoft.github.io/agent-lightning/latest/tutorials/debug/#__codelineno-1-3) agl.setup_logging("DEBUG")` Set the log level on every process if your setup involves multiple workers. For example, when [running stores in isolation](https://microsoft.github.io/agent-lightning/latest/tutorials/debug/#debug-with-external-store "Debug the Algorithm-Runner Boundary") , configure the store process explicitly: `[](https://microsoft.github.io/agent-lightning/latest/tutorials/debug/#__codelineno-2-1) agl store --port 4747 --log-level DEBUG` Using [`Runner`](https://microsoft.github.io/agent-lightning/latest/reference/runner/#agentlightning.Runner " agentlightning.Runner") in Isolation[¶](https://microsoft.github.io/agent-lightning/latest/tutorials/debug/#using-runner-in-isolation "Permanent link") ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- [`Runner`](https://microsoft.github.io/agent-lightning/latest/reference/runner/#agentlightning.Runner " agentlightning.Runner") is a long-lived worker that wraps your [`LitAgent`](https://microsoft.github.io/agent-lightning/latest/reference/agent/#agentlightning.LitAgent " agentlightning.LitAgent") , coordinates tracing, and talks to the [`LightningStore`](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.LightningStore " agentlightning.LightningStore") . In typical training flows the trainer manages runners for you, but being able to spin one up manually is invaluable while debugging. If you define rollout logic with [`@rollout`](https://microsoft.github.io/agent-lightning/latest/reference/agent/#agentlightning.rollout " agentlightning.rollout(func)") or implement a [`LitAgent`](https://microsoft.github.io/agent-lightning/latest/reference/agent/#agentlightning.LitAgent " agentlightning.LitAgent") directly, you will get a [`LitAgent`](https://microsoft.github.io/agent-lightning/latest/reference/agent/#agentlightning.LitAgent " agentlightning.LitAgent") instance and you should be able to execute it with [`LitAgentRunner`](https://microsoft.github.io/agent-lightning/latest/reference/runner/#agentlightning.LitAgentRunner " agentlightning.LitAgentRunner") , which is a subclass of [`Runner`](https://microsoft.github.io/agent-lightning/latest/reference/runner/#agentlightning.Runner " agentlightning.Runner") . The runner needs but does not instantiate a [`Tracer`](https://microsoft.github.io/agent-lightning/latest/reference/runner/#agentlightning.Tracer " agentlightning.Tracer") , so supply one yourself. See [Working with Traces](https://microsoft.github.io/agent-lightning/latest/tutorials/traces/) for a walkthrough of tracer options. [`Runner.run_context`](https://microsoft.github.io/agent-lightning/latest/reference/runner/#agentlightning.Runner.run_context " run_context(*, agent, store, hooks=None, worker_id=None)") prepares the runner to execute a particular agent. Besides the agent and tracer you must provide a store that will collect spans and rollouts. [`InMemoryLightningStore`](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.InMemoryLightningStore " agentlightning.InMemoryLightningStore") keeps everything in-process, which is perfect for debugging sessions. `[](https://microsoft.github.io/agent-lightning/latest/tutorials/debug/#__codelineno-3-1) import agentlightning as agl [](https://microsoft.github.io/agent-lightning/latest/tutorials/debug/#__codelineno-3-2) [](https://microsoft.github.io/agent-lightning/latest/tutorials/debug/#__codelineno-3-3) tracer = agl.OtelTracer() [](https://microsoft.github.io/agent-lightning/latest/tutorials/debug/#__codelineno-3-4) runner = agl.LitAgentRunner(tracer) [](https://microsoft.github.io/agent-lightning/latest/tutorials/debug/#__codelineno-3-5) store = agl.InMemoryLightningStore() [](https://microsoft.github.io/agent-lightning/latest/tutorials/debug/#__codelineno-3-6) [](https://microsoft.github.io/agent-lightning/latest/tutorials/debug/#__codelineno-3-7) with runner.run_context(agent=apo_rollout, store=store): [](https://microsoft.github.io/agent-lightning/latest/tutorials/debug/#__codelineno-3-8) ...` Inside the [`run_context`](https://microsoft.github.io/agent-lightning/latest/reference/runner/#agentlightning.Runner.run_context " run_context(*, agent, store, hooks=None, worker_id=None)") block you can call [`runner.step(...)`](https://microsoft.github.io/agent-lightning/latest/reference/runner/#agentlightning.Runner.step " step(input, *, resources=None, mode=None, event=None) async ") to execute a single rollout. The payload includes the task input and any [`NamedResources`](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.NamedResources " agentlightning.NamedResources = Dict[str, ResourceUnion] module-attribute ") the agent expects. Read [introduction to Resources](https://microsoft.github.io/agent-lightning/latest/how-to/write-first-algorithm/#introduction-to-resources "Resources: The Tunable Assets") and [NamedResources](https://microsoft.github.io/agent-lightning/latest/tutorials/write-agents/#introduction-to-named-resources "Class-based Agents") for more details. For example, if your agent references a [`PromptTemplate`](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.PromptTemplate " agentlightning.PromptTemplate") , pass it through the `resources` argument: `[](https://microsoft.github.io/agent-lightning/latest/tutorials/debug/#__codelineno-4-1) with runner.run_context(agent=apo_rollout, store=store): [](https://microsoft.github.io/agent-lightning/latest/tutorials/debug/#__codelineno-4-2) resource = agl.PromptTemplate(template="You are a helpful assistant. {any_question}", engine="f-string") [](https://microsoft.github.io/agent-lightning/latest/tutorials/debug/#__codelineno-4-3) rollout = await runner.step( [](https://microsoft.github.io/agent-lightning/latest/tutorials/debug/#__codelineno-4-4) "Explain why the sky appears blue using principles of light scattering in 100 words.", [](https://microsoft.github.io/agent-lightning/latest/tutorials/debug/#__codelineno-4-5) resources={"main_prompt": resource}, [](https://microsoft.github.io/agent-lightning/latest/tutorials/debug/#__codelineno-4-6) )` You can do as many things as you want within the [`Runner.run_context`](https://microsoft.github.io/agent-lightning/latest/reference/runner/#agentlightning.Runner.run_context " run_context(*, agent, store, hooks=None, worker_id=None)") block. After the rollout finishes you can query the store to inspect what happened: `[](https://microsoft.github.io/agent-lightning/latest/tutorials/debug/#__codelineno-5-1) print(await store.query_rollouts()) [](https://microsoft.github.io/agent-lightning/latest/tutorials/debug/#__codelineno-5-2) print(await store.query_spans(rollout.rollout_id))` Example output (with a reward span captured): `[](https://microsoft.github.io/agent-lightning/latest/tutorials/debug/#__codelineno-6-1) [Rollout(rollout_id='ro-519769241af8', input='Explain why the sky appears blue using principles of light scattering in 100 words.', start_time=1760706315.6996238, ..., status='succeeded')] [](https://microsoft.github.io/agent-lightning/latest/tutorials/debug/#__codelineno-6-2) [Span(rollout_id='ro-519769241af8', attempt_id='at-a6b62caf', sequence_id=1, ..., name='agentlightning.annotation', attributes={'agentlightning.reward.0.value': 0.95}, ...)]` Swap in an [`AgentOpsTracer`](https://microsoft.github.io/agent-lightning/latest/reference/runner/#agentlightning.AgentOpsTracer " agentlightning.AgentOpsTracer") instead of [`OtelTracer`](https://microsoft.github.io/agent-lightning/latest/reference/runner/#agentlightning.OtelTracer " agentlightning.OtelTracer") to see the underlying LLM spans alongside reward information: `[](https://microsoft.github.io/agent-lightning/latest/tutorials/debug/#__codelineno-7-1) [ [](https://microsoft.github.io/agent-lightning/latest/tutorials/debug/#__codelineno-7-2) Span(rollout_id='ro-519769241af8', attempt_id='at-a6b62caf', sequence_id=1, ..., name='openai.chat.completion', attributes={..., 'gen_ai.prompt.0.role': 'user', 'gen_ai.prompt.0.content': 'You are a helpful assistant. Explain why the sky appears blue using principles of light scattering in 100 words.', ...}), [](https://microsoft.github.io/agent-lightning/latest/tutorials/debug/#__codelineno-7-3) Span(rollout_id='ro-519769241af8', attempt_id='at-a6b62caf', sequence_id=2, ..., name='openai.chat.completion', attributes={..., 'gen_ai.prompt.0.role': 'user', 'gen_ai.prompt.0.content': 'Evaluate how well the output fulfills the task...', ...}), [](https://microsoft.github.io/agent-lightning/latest/tutorials/debug/#__codelineno-7-4) Span(rollout_id='ro-519769241af8', attempt_id='at-a6b62caf', sequence_id=3, ..., name='agentlightning.annotation', attributes={'agentlightning.reward.0.value': 0.95}, ...) [](https://microsoft.github.io/agent-lightning/latest/tutorials/debug/#__codelineno-7-5) ]` Tip Spans too difficult to read? Try using [`Adapter`](https://microsoft.github.io/agent-lightning/latest/reference/algorithm/#agentlightning.Adapter " agentlightning.Adapter") to convert them into a [more readable format](https://microsoft.github.io/agent-lightning/latest/tutorials/traces/) . [`Runner.step`](https://microsoft.github.io/agent-lightning/latest/reference/runner/#agentlightning.Runner.step " step(input, *, resources=None, mode=None, event=None) async ") executes a full rollout even though it is named "step". The companion method [`Runner.iter`](https://microsoft.github.io/agent-lightning/latest/reference/runner/#agentlightning.Runner.iter " iter(*, event=None) async ") executes multiple "steps" by continuously pulling new rollout inputs from the store until a stop event is set. Use `iter` once you are confident the single-step path works and you have another worker [`enqueue_rollout`](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.LightningStore.enqueue_rollout " enqueue_rollout(input, mode=None, resources_id=None, config=None, metadata=None) async ") to the store. Tip You can also call [`Runner.step`](https://microsoft.github.io/agent-lightning/latest/reference/runner/#agentlightning.Runner.step " step(input, *, resources=None, mode=None, event=None) async ") to inject ad-hoc rollouts into a running store being used by another algorithm, so that the rollouts can be consumed by the algorithms. This is very recently known as the paradigm of ["online RL"](https://cursor.com/blog/tab-rl) . At the moment, no algorithm in the [algorithm zoo](https://microsoft.github.io/agent-lightning/latest/algorithm-zoo/) consumes externally generated rollouts, but the data flow is available there if you need it. Debug with LLM Proxy[¶](https://microsoft.github.io/agent-lightning/latest/tutorials/debug/#debug-with-llm-proxy "Permanent link") ----------------------------------------------------------------------------------------------------------------------------------- If you are dealing with LLM optimization like Reinforcement Learning, we generally recommend using an online stable LLM service for your debugging purposes, like `openai/gpt-4.1-nano`. After the debugging is done, you can switch to a local training endpoint. However, if you want to use a local LLM features like [getting the token IDs](https://microsoft.github.io/agent-lightning/latest/deep-dive/serving-llm/) , you can also manually start a local vLLM server by: `[](https://microsoft.github.io/agent-lightning/latest/tutorials/debug/#__codelineno-8-1) vllm serve Qwen/Qwen2.5-0.5B-Instruct --port 8080` Then start the LLM proxy via the following script: `[](https://microsoft.github.io/agent-lightning/latest/tutorials/debug/#__codelineno-9-1) import asyncio [](https://microsoft.github.io/agent-lightning/latest/tutorials/debug/#__codelineno-9-2) import aiohttp [](https://microsoft.github.io/agent-lightning/latest/tutorials/debug/#__codelineno-9-3) import agentlightning as agl [](https://microsoft.github.io/agent-lightning/latest/tutorials/debug/#__codelineno-9-4) [](https://microsoft.github.io/agent-lightning/latest/tutorials/debug/#__codelineno-9-5) async def serve_llm_proxy(): [](https://microsoft.github.io/agent-lightning/latest/tutorials/debug/#__codelineno-9-6) store = agl.InMemoryLightningStore() [](https://microsoft.github.io/agent-lightning/latest/tutorials/debug/#__codelineno-9-7) store_server = agl.LightningStoreServer(store, "127.0.0.1", 8081) [](https://microsoft.github.io/agent-lightning/latest/tutorials/debug/#__codelineno-9-8) await store_server.start() [](https://microsoft.github.io/agent-lightning/latest/tutorials/debug/#__codelineno-9-9) [](https://microsoft.github.io/agent-lightning/latest/tutorials/debug/#__codelineno-9-10) llm_proxy = agl.LLMProxy( [](https://microsoft.github.io/agent-lightning/latest/tutorials/debug/#__codelineno-9-11) port=8082, [](https://microsoft.github.io/agent-lightning/latest/tutorials/debug/#__codelineno-9-12) model_list=[ [](https://microsoft.github.io/agent-lightning/latest/tutorials/debug/#__codelineno-9-13) { [](https://microsoft.github.io/agent-lightning/latest/tutorials/debug/#__codelineno-9-14) "model_name": "Qwen/Qwen2.5-0.5B-Instruct", [](https://microsoft.github.io/agent-lightning/latest/tutorials/debug/#__codelineno-9-15) "litellm_params": { [](https://microsoft.github.io/agent-lightning/latest/tutorials/debug/#__codelineno-9-16) "model": "hosted_vllm/Qwen/Qwen2.5-0.5B-Instruct", [](https://microsoft.github.io/agent-lightning/latest/tutorials/debug/#__codelineno-9-17) "api_base": "http://localhost:8080/v1", [](https://microsoft.github.io/agent-lightning/latest/tutorials/debug/#__codelineno-9-18) }, [](https://microsoft.github.io/agent-lightning/latest/tutorials/debug/#__codelineno-9-19) } [](https://microsoft.github.io/agent-lightning/latest/tutorials/debug/#__codelineno-9-20) ], [](https://microsoft.github.io/agent-lightning/latest/tutorials/debug/#__codelineno-9-21) store=store_server, [](https://microsoft.github.io/agent-lightning/latest/tutorials/debug/#__codelineno-9-22) ) [](https://microsoft.github.io/agent-lightning/latest/tutorials/debug/#__codelineno-9-23) [](https://microsoft.github.io/agent-lightning/latest/tutorials/debug/#__codelineno-9-24) await llm_proxy.start() [](https://microsoft.github.io/agent-lightning/latest/tutorials/debug/#__codelineno-9-25) await asyncio.sleep(1000000)` Test the served LLM proxy with a client like: `[](https://microsoft.github.io/agent-lightning/latest/tutorials/debug/#__codelineno-10-1) async def test_llm_proxy(): [](https://microsoft.github.io/agent-lightning/latest/tutorials/debug/#__codelineno-10-2) async with aiohttp.ClientSession() as session: [](https://microsoft.github.io/agent-lightning/latest/tutorials/debug/#__codelineno-10-3) async with session.post("http://localhost:8082/v1/chat/completions", json={ [](https://microsoft.github.io/agent-lightning/latest/tutorials/debug/#__codelineno-10-4) "model": "Qwen/Qwen2.5-0.5B-Instruct", [](https://microsoft.github.io/agent-lightning/latest/tutorials/debug/#__codelineno-10-5) "messages": [{"role": "user", "content": "Hello, world!"}], [](https://microsoft.github.io/agent-lightning/latest/tutorials/debug/#__codelineno-10-6) }) as response: [](https://microsoft.github.io/agent-lightning/latest/tutorials/debug/#__codelineno-10-7) print(await response.json())` You can now use the LLM proxy by specifying environment variables: `[](https://microsoft.github.io/agent-lightning/latest/tutorials/debug/#__codelineno-11-1) export OPENAI_API_BASE=http://localhost:8081/v1 [](https://microsoft.github.io/agent-lightning/latest/tutorials/debug/#__codelineno-11-2) export OPENAI_API_KEY=dummy` You might see warnings about `Missing or invalid rollout_id, attempt_id, or sequence_id` in the LLM proxy logs. This is fine because you don't have a rollout and attempt yet when you are debugging. When you started the training, the algorithm will create the rollouts for you and the warnings will go away. Hook into Runner's Lifecycle[¶](https://microsoft.github.io/agent-lightning/latest/tutorials/debug/#hook-into-runners-lifecycle "Permanent link") -------------------------------------------------------------------------------------------------------------------------------------------------- [`Runner.run_context`](https://microsoft.github.io/agent-lightning/latest/reference/runner/#agentlightning.Runner.run_context " run_context(*, agent, store, hooks=None, worker_id=None)") accepts a `hooks` argument so you can observe or augment lifecycle events without editing your agent. Hooks subclass [`Hook`](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.Hook " agentlightning.Hook") and can respond to four asynchronous callbacks: [`on_trace_start`](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.Hook.on_trace_start " on_trace_start(*, agent, runner, tracer, rollout) async ") , [`on_rollout_start`](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.Hook.on_rollout_start " on_rollout_start(*, agent, runner, rollout) async ") , [`on_rollout_end`](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.Hook.on_rollout_end " on_rollout_end(*, agent, runner, rollout, spans) async ") , and [`on_trace_end`](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.Hook.on_trace_end " on_trace_end(*, agent, runner, tracer, rollout) async ") . This is useful for: * Capturing raw OpenTelemetry spans before they hit the store and before the [`LitAgentRunner`](https://microsoft.github.io/agent-lightning/latest/reference/runner/#agentlightning.LitAgentRunner " agentlightning.LitAgentRunner") do postprocessing on the rollout * Inspecting the tracer instance after they are activated * Logging rollout inputs before they are processed by the agent The `hook` mode in [`examples/apo/apo_debug.py`](https://github.com/microsoft/agent-lightning/tree/c746af2f76bebcae56007a59c223cdace411c89b/examples/apo/apo_debug.py) prints every span collected during a rollout: `[](https://microsoft.github.io/agent-lightning/latest/tutorials/debug/#__codelineno-12-1) import agentlightning as agl [](https://microsoft.github.io/agent-lightning/latest/tutorials/debug/#__codelineno-12-2) [](https://microsoft.github.io/agent-lightning/latest/tutorials/debug/#__codelineno-12-3) # ... Same as previous example [](https://microsoft.github.io/agent-lightning/latest/tutorials/debug/#__codelineno-12-4) [](https://microsoft.github.io/agent-lightning/latest/tutorials/debug/#__codelineno-12-5) class DebugHook(agl.Hook): [](https://microsoft.github.io/agent-lightning/latest/tutorials/debug/#__codelineno-12-6) async def on_trace_end(self, *, agent, runner, tracer, rollout): [](https://microsoft.github.io/agent-lightning/latest/tutorials/debug/#__codelineno-12-7) trace = tracer.get_last_trace() [](https://microsoft.github.io/agent-lightning/latest/tutorials/debug/#__codelineno-12-8) print("Trace spans collected during the rollout:") [](https://microsoft.github.io/agent-lightning/latest/tutorials/debug/#__codelineno-12-9) for span in trace: [](https://microsoft.github.io/agent-lightning/latest/tutorials/debug/#__codelineno-12-10) print(f"- {span.name} (status: {span.status}):\n {span.attributes}") [](https://microsoft.github.io/agent-lightning/latest/tutorials/debug/#__codelineno-12-11) [](https://microsoft.github.io/agent-lightning/latest/tutorials/debug/#__codelineno-12-12) with runner.run_context( [](https://microsoft.github.io/agent-lightning/latest/tutorials/debug/#__codelineno-12-13) agent=apo_rollout, [](https://microsoft.github.io/agent-lightning/latest/tutorials/debug/#__codelineno-12-14) store=store, [](https://microsoft.github.io/agent-lightning/latest/tutorials/debug/#__codelineno-12-15) hooks=[DebugHook()], [](https://microsoft.github.io/agent-lightning/latest/tutorials/debug/#__codelineno-12-16) ): [](https://microsoft.github.io/agent-lightning/latest/tutorials/debug/#__codelineno-12-17) await runner.step( [](https://microsoft.github.io/agent-lightning/latest/tutorials/debug/#__codelineno-12-18) "Explain why the sky appears blue using principles of light scattering in 100 words.", [](https://microsoft.github.io/agent-lightning/latest/tutorials/debug/#__codelineno-12-19) resources={"main_prompt": resource}, [](https://microsoft.github.io/agent-lightning/latest/tutorials/debug/#__codelineno-12-20) )` Because hooks run inside the runner process you can also attach debuggers or breakpoints directly in the callback implementations. Note For a better understanding of where hooks are called, we show a pseudo code of Runner's working flow below: `[](https://microsoft.github.io/agent-lightning/latest/tutorials/debug/#__codelineno-13-1) resources = await store.get_latest_resources() [](https://microsoft.github.io/agent-lightning/latest/tutorials/debug/#__codelineno-13-2) rollout = ... [](https://microsoft.github.io/agent-lightning/latest/tutorials/debug/#__codelineno-13-3) try: [](https://microsoft.github.io/agent-lightning/latest/tutorials/debug/#__codelineno-13-4) # <-- on_rollout_start [](https://microsoft.github.io/agent-lightning/latest/tutorials/debug/#__codelineno-13-5) with tracer.trace_context(...): [](https://microsoft.github.io/agent-lightning/latest/tutorials/debug/#__codelineno-13-6) # <--- on_trace_start [](https://microsoft.github.io/agent-lightning/latest/tutorials/debug/#__codelineno-13-7) result = await agent.rollout(...) [](https://microsoft.github.io/agent-lightning/latest/tutorials/debug/#__codelineno-13-8) # <--- on_trace_end [](https://microsoft.github.io/agent-lightning/latest/tutorials/debug/#__codelineno-13-9) post_process_result(result) [](https://microsoft.github.io/agent-lightning/latest/tutorials/debug/#__codelineno-13-10) except Exception: [](https://microsoft.github.io/agent-lightning/latest/tutorials/debug/#__codelineno-13-11) # <-- on_rollout_end [](https://microsoft.github.io/agent-lightning/latest/tutorials/debug/#__codelineno-13-12) await store.update_attempt(status=...)` Dry-Run the Trainer Loop[¶](https://microsoft.github.io/agent-lightning/latest/tutorials/debug/#dry-run-the-trainer-loop "Permanent link") ------------------------------------------------------------------------------------------------------------------------------------------- Once single rollouts behave, switch to the trainer’s dry-run mode. [`Trainer.dev`](https://microsoft.github.io/agent-lightning/latest/reference/trainer/#agentlightning.Trainer.dev " dev(agent, train_dataset=None, *, val_dataset=None)") spins up a lightweight fast algorithm — [`agentlightning.Baseline`](https://microsoft.github.io/agent-lightning/latest/reference/algorithm/#agentlightning.Baseline " agentlightning.Baseline") by default — so you can exercise the same infrastructure as [`Trainer.fit`](https://microsoft.github.io/agent-lightning/latest/reference/trainer/#agentlightning.Trainer.fit " fit(agent, train_dataset=None, *, val_dataset=None)") without standing up complex stacks like RL or SFT. Warning When you enable multiple runners via `n_runners`, the trainer may execute them in separate worker processes. Attaching a debugger such as `pdb` is only practical when `n_runners=1`, and even then the runner might not live in the main process. `[](https://microsoft.github.io/agent-lightning/latest/tutorials/debug/#__codelineno-14-1) import agentlightning as agl [](https://microsoft.github.io/agent-lightning/latest/tutorials/debug/#__codelineno-14-2) [](https://microsoft.github.io/agent-lightning/latest/tutorials/debug/#__codelineno-14-3) dataset: agl.Dataset[str] = [ [](https://microsoft.github.io/agent-lightning/latest/tutorials/debug/#__codelineno-14-4) "Explain why the sky appears blue using principles of light scattering in 100 words.", [](https://microsoft.github.io/agent-lightning/latest/tutorials/debug/#__codelineno-14-5) "What's the capital of France?", [](https://microsoft.github.io/agent-lightning/latest/tutorials/debug/#__codelineno-14-6) ] [](https://microsoft.github.io/agent-lightning/latest/tutorials/debug/#__codelineno-14-7) resource = agl.PromptTemplate(template="You are a helpful assistant. {any_question}", engine="f-string") [](https://microsoft.github.io/agent-lightning/latest/tutorials/debug/#__codelineno-14-8) [](https://microsoft.github.io/agent-lightning/latest/tutorials/debug/#__codelineno-14-9) trainer = agl.Trainer( [](https://microsoft.github.io/agent-lightning/latest/tutorials/debug/#__codelineno-14-10) n_runners=1, [](https://microsoft.github.io/agent-lightning/latest/tutorials/debug/#__codelineno-14-11) initial_resources={"main_prompt": resource}, [](https://microsoft.github.io/agent-lightning/latest/tutorials/debug/#__codelineno-14-12) ) [](https://microsoft.github.io/agent-lightning/latest/tutorials/debug/#__codelineno-14-13) trainer.dev(apo_rollout, dataset)` Just like [`Runner.run_context`](https://microsoft.github.io/agent-lightning/latest/reference/runner/#agentlightning.Runner.run_context " run_context(*, agent, store, hooks=None, worker_id=None)") , [`Trainer.dev`](https://microsoft.github.io/agent-lightning/latest/reference/trainer/#agentlightning.Trainer.dev " dev(agent, train_dataset=None, *, val_dataset=None)") requires the [`NamedResources`](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.NamedResources " agentlightning.NamedResources = Dict[str, ResourceUnion] module-attribute ") your agent expects. The key difference is that resources are attached to the trainer rather than the runner. [`Trainer.dev`](https://microsoft.github.io/agent-lightning/latest/reference/trainer/#agentlightning.Trainer.dev " dev(agent, train_dataset=None, *, val_dataset=None)") uses an almost switchable interface from [`Trainer.fit`](https://microsoft.github.io/agent-lightning/latest/reference/trainer/#agentlightning.Trainer.fit " fit(agent, train_dataset=None, *, val_dataset=None)") . It also needs a dataset to iterate over, similar to [`fit`](https://microsoft.github.io/agent-lightning/latest/reference/trainer/#agentlightning.Trainer.fit " fit(agent, train_dataset=None, *, val_dataset=None)") . Under the hood [`dev`](https://microsoft.github.io/agent-lightning/latest/reference/trainer/#agentlightning.Trainer.dev " dev(agent, train_dataset=None, *, val_dataset=None)") uses the same implementation as [`fit`](https://microsoft.github.io/agent-lightning/latest/reference/trainer/#agentlightning.Trainer.fit " fit(agent, train_dataset=None, *, val_dataset=None)") , which means you can spin up multiple runners, observe scheduler behavior, and validate how algorithms adapt rollouts. The default [`Baseline`](https://microsoft.github.io/agent-lightning/latest/reference/algorithm/#agentlightning.Baseline " agentlightning.Baseline") logs detailed traces so you can see each rollout as the algorithm perceives it: `[](https://microsoft.github.io/agent-lightning/latest/tutorials/debug/#__codelineno-15-1) 21:20:30 Initial resources set: {'main_prompt': PromptTemplate(resource_type='prompt_template', template='You are a helpful assistant. {any_question}', engine='f-string')} [](https://microsoft.github.io/agent-lightning/latest/tutorials/debug/#__codelineno-15-2) 21:20:30 Proceeding epoch 1/1. [](https://microsoft.github.io/agent-lightning/latest/tutorials/debug/#__codelineno-15-3) 21:20:30 Enqueued rollout ro-302fb202bd85 in train mode with sample: Explain why the sky appears blue using principles of light scattering in 100 words. [](https://microsoft.github.io/agent-lightning/latest/tutorials/debug/#__codelineno-15-4) 21:20:30 Enqueued rollout ro-e65a3ffaa540 in train mode with sample: What's the capital of France? [](https://microsoft.github.io/agent-lightning/latest/tutorials/debug/#__codelineno-15-5) 21:20:30 Waiting for 2 harvest tasks to complete... [](https://microsoft.github.io/agent-lightning/latest/tutorials/debug/#__codelineno-15-6) 21:20:30 [Rollout ro-302fb202bd85] Status is initialized to queuing. [](https://microsoft.github.io/agent-lightning/latest/tutorials/debug/#__codelineno-15-7) 21:20:30 [Rollout ro-e65a3ffaa540] Status is initialized to queuing. [](https://microsoft.github.io/agent-lightning/latest/tutorials/debug/#__codelineno-15-8) 21:20:35 [Rollout ro-302fb202bd85] Finished with status succeeded in 3.80 seconds. [](https://microsoft.github.io/agent-lightning/latest/tutorials/debug/#__codelineno-15-9) 21:20:35 [Rollout ro-302fb202bd85 | Attempt 1] ID: at-f84ad21c. Status: succeeded. Worker: Worker-0 [](https://microsoft.github.io/agent-lightning/latest/tutorials/debug/#__codelineno-15-10) 21:20:35 [Rollout ro-302fb202bd85 | Attempt at-f84ad21c | Span 3a286a856af6bea8] #1 (openai.chat.completion) ... 1.95 seconds. Attribute keys: ['gen_ai.request.type', 'gen_ai.system', ...] [](https://microsoft.github.io/agent-lightning/latest/tutorials/debug/#__codelineno-15-11) 21:20:35 [Rollout ro-302fb202bd85 | Attempt at-f84ad21c | Span e2f44b775e058dd6] #2 (openai.chat.completion) ... 1.24 seconds. Attribute keys: ['gen_ai.request.type', 'gen_ai.system', ...] [](https://microsoft.github.io/agent-lightning/latest/tutorials/debug/#__codelineno-15-12) 21:20:35 [Rollout ro-302fb202bd85 | Attempt at-f84ad21c | Span 45ee3c94fa1070ec] #3 (agentlightning.annotation) ... 0.00 seconds. Attribute keys: ['agentlightning.reward.0.value'] [](https://microsoft.github.io/agent-lightning/latest/tutorials/debug/#__codelineno-15-13) 21:20:35 [Rollout ro-302fb202bd85] Adapted data: [Triplet(prompt={'token_ids': []}, response={'token_ids': []}, reward=None, metadata={'response_id': '...', 'agent_name': ''}), Triplet(prompt={'token_ids': []}, response={'token_ids': []}, reward=0.95, metadata={'response_id': '...', 'agent_name': ''})] [](https://microsoft.github.io/agent-lightning/latest/tutorials/debug/#__codelineno-15-14) 21:20:35 Finished 1 rollouts. [](https://microsoft.github.io/agent-lightning/latest/tutorials/debug/#__codelineno-15-15) 21:20:35 [Rollout ro-e65a3ffaa540] Status changed to preparing. [](https://microsoft.github.io/agent-lightning/latest/tutorials/debug/#__codelineno-15-16) 21:20:40 [Rollout ro-e65a3ffaa540] Finished with status succeeded in 6.39 seconds. [](https://microsoft.github.io/agent-lightning/latest/tutorials/debug/#__codelineno-15-17) 21:20:40 [Rollout ro-e65a3ffaa540 | Attempt 1] ID: at-eaefa5d4. Status: succeeded. Worker: Worker-0 [](https://microsoft.github.io/agent-lightning/latest/tutorials/debug/#__codelineno-15-18) 21:20:40 [Rollout ro-e65a3ffaa540 | Attempt at-eaefa5d4 | Span 901dd6acc0f50147] #1 (openai.chat.completion) ... 1.30 seconds. Attribute keys: ['gen_ai.request.type', 'gen_ai.system', ...] [](https://microsoft.github.io/agent-lightning/latest/tutorials/debug/#__codelineno-15-19) 21:20:40 [Rollout ro-e65a3ffaa540 | Attempt at-eaefa5d4 | Span 52e0aa63e02be611] #2 (openai.chat.completion) ... 1.26 seconds. Attribute keys: ['gen_ai.request.type', 'gen_ai.system', ...] [](https://microsoft.github.io/agent-lightning/latest/tutorials/debug/#__codelineno-15-20) 21:20:40 [Rollout ro-e65a3ffaa540 | Attempt at-eaefa5d4 | Span 6c452de193fbffd3] #3 (agentlightning.annotation) ... 0.00 seconds. Attribute keys: ['agentlightning.reward.0.value'] [](https://microsoft.github.io/agent-lightning/latest/tutorials/debug/#__codelineno-15-21) 21:20:40 [Rollout ro-e65a3ffaa540] Adapted data: [Triplet(prompt={'token_ids': []}, response={'token_ids': []}, reward=None, metadata={'response_id': '...', 'agent_name': ''}), Triplet(prompt={'token_ids': []}, response={'token_ids': []}, reward=1.0, metadata={'response_id': '...', 'agent_name': ''})] [](https://microsoft.github.io/agent-lightning/latest/tutorials/debug/#__codelineno-15-22) 21:20:40 Finished 2 rollouts.` The only limitation is that resources remain static and components like [`LLMProxy`](https://microsoft.github.io/agent-lightning/latest/reference/algorithm/#agentlightning.LLMProxy " agentlightning.LLMProxy") are not wired in. For richer dry runs you can subclass [`FastAlgorithm`](https://microsoft.github.io/agent-lightning/latest/reference/algorithm/#agentlightning.FastAlgorithm " agentlightning.FastAlgorithm") and override the pieces you care about. Debug the Algorithm-Runner Boundary[¶](https://microsoft.github.io/agent-lightning/latest/tutorials/debug/#debug-the-algorithm-runner-boundary "Permanent link") ----------------------------------------------------------------------------------------------------------------------------------------------------------------- [](https://microsoft.github.io/agent-lightning/latest/tutorials/debug/) Debugging algorithms in Agent-Lightning is often more challenging than debugging agents. Algorithms are typically **stateful** and depend on several moving parts — runners, stores, and trainers — which makes it difficult to isolate and inspect their behavior. Even mocking an agent to cooperate with an algorithm can be costly and error-prone. To simplify this, Agent-Lightning provides a way to run algorithms in isolation so you can attach a debugger and inspect internal state without interference from other components. By default, [`Trainer.fit`](https://microsoft.github.io/agent-lightning/latest/reference/trainer/#agentlightning.Trainer.fit " fit(agent, train_dataset=None, *, val_dataset=None)") runs the algorithm in the main process and thread, but its logs are interleaved with those from the store and runners, making it hard to follow what’s happening inside the algorithm itself. In [_Write Your First Algorithm_](https://microsoft.github.io/agent-lightning/latest/how-to/write-first-algorithm/) , we covered how to stand up a store, algorithm, and runner in isolation for your own implementations. This section extends that approach to cover two common questions: 1. How can I run built-in or class-based algorithms (inheriting from [`Algorithm`](https://microsoft.github.io/agent-lightning/latest/reference/algorithm/#agentlightning.Algorithm " agentlightning.Algorithm") ) in isolation? 2. How can I still use [`Trainer`](https://microsoft.github.io/agent-lightning/latest/reference/trainer/#agentlightning.Trainer " agentlightning.Trainer") features like `n_runners`, `adapter`, or `llm_proxy` while debugging? The solution is to keep using a [`Trainer`](https://microsoft.github.io/agent-lightning/latest/reference/trainer/#agentlightning.Trainer " agentlightning.Trainer") instance but **manage the store yourself**, running the algorithm and runner roles separately. This approach mirrors the internal process orchestration of [`Trainer.fit`](https://microsoft.github.io/agent-lightning/latest/reference/trainer/#agentlightning.Trainer.fit " fit(agent, train_dataset=None, *, val_dataset=None)") , but with more visibility and control. Below, we show a step-by-step guide to achieve this with the [`calc_agent` example](https://github.com/microsoft/agent-lightning/tree/c746af2f76bebcae56007a59c223cdace411c89b/examples/calc_x/train_calc_agent.py) . **1\. Launch the store manually.** In a separate terminal, start the store: `[](https://microsoft.github.io/agent-lightning/latest/tutorials/debug/#__codelineno-16-1) agl store --port 4747` Add `--log-level DEBUG` to the command to see the detailed logs. Then, in your training script, create a [`LightningStoreClient`](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.LightningStoreClient " agentlightning.LightningStoreClient") and pass it to the trainer: `[](https://microsoft.github.io/agent-lightning/latest/tutorials/debug/#__codelineno-17-1) client = agl.LightningStoreClient("http://localhost:4747") [](https://microsoft.github.io/agent-lightning/latest/tutorials/debug/#__codelineno-17-2) trainer = agl.Trainer(store=client, ...)` Set the environment variable `AGL_MANAGED_STORE=0` so the trainer doesn't attempt to manage the store automatically. **2\. Start the runner and algorithm processes separately.** Each process should run the same training script, but with different environment variables specifying the current role. This setup faithfully mirrors how [`Trainer.fit`](https://microsoft.github.io/agent-lightning/latest/reference/trainer/#agentlightning.Trainer.fit " fit(agent, train_dataset=None, *, val_dataset=None)") orchestrates these components behind the scenes. `[](https://microsoft.github.io/agent-lightning/latest/tutorials/debug/#__codelineno-18-1) # Terminal 2 – Runner process [](https://microsoft.github.io/agent-lightning/latest/tutorials/debug/#__codelineno-18-2) AGL_MANAGED_STORE=0 AGL_CURRENT_ROLE=runner \ [](https://microsoft.github.io/agent-lightning/latest/tutorials/debug/#__codelineno-18-3) python train_calc_agent.py --external-store-address http://localhost:4747 --val-file data/test_mini.parquet [](https://microsoft.github.io/agent-lightning/latest/tutorials/debug/#__codelineno-18-4) [](https://microsoft.github.io/agent-lightning/latest/tutorials/debug/#__codelineno-18-5) # Terminal 3 – Algorithm process [](https://microsoft.github.io/agent-lightning/latest/tutorials/debug/#__codelineno-18-6) AGL_MANAGED_STORE=0 AGL_CURRENT_ROLE=algorithm \ [](https://microsoft.github.io/agent-lightning/latest/tutorials/debug/#__codelineno-18-7) python train_calc_agent.py --external-store-address http://localhost:4747 --val-file data/test_mini.parquet` **3\. Reuse your existing trainer configuration.** You can continue using the same datasets, adapters, and proxies as usual. Because the store is now external, you can: * Attach debuggers to either the algorithm or runner process * Add fine-grained logging or tracing * Simulate partial failures or latency in individual components This setup provides a faithful reproduction of the algorithm–runner interaction while keeping the store visible for inspection. Once you’ve resolved the issue, simply set `AGL_MANAGED_STORE=1` (or omit it) to return to the standard managed training workflow. Back to top --- # Use Emitters - Agent-lightning [Skip to content](https://microsoft.github.io/agent-lightning/latest/tutorials/emitter/#using-emitters) ⚠️ You are viewing development documentation. For stable documentation, please visit [https://microsoft.github.io/agent-lightning/stable/](https://microsoft.github.io/agent-lightning/stable/) Using Emitters[¶](https://microsoft.github.io/agent-lightning/latest/tutorials/emitter/#using-emitters "Permanent link") ========================================================================================================================= [](https://microsoft.github.io/agent-lightning/latest/tutorials/emitter/) While returning a single float for the final reward is sufficient for many algorithm-agent combinations, some advanced scenarios require richer feedback. For instance, an algorithm might learn more effectively if it receives intermediate rewards throughout a multi-step task, or if the agent needs to emit additional spans for debugging or analysis. Agent-lightning provides an **emitter** module for recording custom spans inside your agent logic. Just as [Tracer](https://microsoft.github.io/agent-lightning/latest/reference/runner/#agentlightning.Tracer " agentlightning.Tracer") automatically instruments common operations (for example, LLM calls), each emitter helper sends a [Span](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.Span " agentlightning.Span") that captures Agent-lightning-specific work so downstream algorithms can query it later. See [Working with Traces](https://microsoft.github.io/agent-lightning/latest/tutorials/traces/) for more details. For multi-step routines such as function calls, tools, or adapters, wrap code with [`operation`](https://microsoft.github.io/agent-lightning/latest/reference/agent/#agentlightning.operation " agentlightning.operation(fn=None, *, propagate=True, name=None, **additional_attributes)") — either as a decorator or a context manager — to capture inputs, outputs, and metadata on a dedicated [`operation`](https://microsoft.github.io/agent-lightning/latest/reference/agent/#agentlightning.operation " agentlightning.operation(fn=None, *, propagate=True, name=None, **additional_attributes)") span. This makes it easier to correlate downstream annotations (like rewards or messages) with the higher-level work that produced them. You can find the emitter functions in [`agentlightning.emitter`](https://microsoft.github.io/agent-lightning/latest/reference/agent/) . Emitting Rewards, Messages, and More[¶](https://microsoft.github.io/agent-lightning/latest/tutorials/emitter/#emitting-rewards-messages-and-more "Permanent link") ------------------------------------------------------------------------------------------------------------------------------------------------------------------- Here are the primary emitter functions: * [`emit_reward(value: float)`](https://microsoft.github.io/agent-lightning/latest/reference/agent/#agentlightning.emit_reward " agentlightning.emit_reward(reward, *, primary_key=None, attributes=None, propagate=True)") : Records an intermediate/final reward, which is a convenient wrapper of [`emit_annotation`](https://microsoft.github.io/agent-lightning/latest/reference/agent/#agentlightning.emit_annotation " agentlightning.emit_annotation(annotation, propagate=True)") . * [`emit_annotation(attributes: Dict[str, Any])`](https://microsoft.github.io/agent-lightning/latest/reference/agent/#agentlightning.emit_annotation " agentlightning.emit_annotation(annotation, propagate=True)") : Records arbitrary metadata as a span. * [`emit_message(message: str)`](https://microsoft.github.io/agent-lightning/latest/reference/agent/#agentlightning.emit_message " agentlightning.emit_message(message, attributes=None, propagate=True)") : Records a simple log message as a span. * [`emit_exception(exception: BaseException)`](https://microsoft.github.io/agent-lightning/latest/reference/agent/#agentlightning.emit_exception " agentlightning.emit_exception(exception, attributes=None, propagate=True)") : Records a Python exception, including its type, message, and stack trace. * [`emit_object(obj: Any)`](https://microsoft.github.io/agent-lightning/latest/reference/agent/#agentlightning.emit_object " agentlightning.emit_object(object, attributes=None, propagate=True)") : Records any JSON-serializable object, perfect for structured data. Let's first see an example of an agent using these emitters to provide detailed feedback. `[](https://microsoft.github.io/agent-lightning/latest/tutorials/emitter/#__codelineno-0-1) import agentlightning as agl [](https://microsoft.github.io/agent-lightning/latest/tutorials/emitter/#__codelineno-0-2) [](https://microsoft.github.io/agent-lightning/latest/tutorials/emitter/#__codelineno-0-3) @agl.rollout [](https://microsoft.github.io/agent-lightning/latest/tutorials/emitter/#__codelineno-0-4) def multi_step_agent(task: dict, prompt_template: PromptTemplate) -> float: [](https://microsoft.github.io/agent-lightning/latest/tutorials/emitter/#__codelineno-0-5) try: [](https://microsoft.github.io/agent-lightning/latest/tutorials/emitter/#__codelineno-0-6) # Step 1: Initial planning [](https://microsoft.github.io/agent-lightning/latest/tutorials/emitter/#__codelineno-0-7) agl.emit_message("Starting planning phase.") [](https://microsoft.github.io/agent-lightning/latest/tutorials/emitter/#__codelineno-0-8) plan = generate_plan(task, prompt_template) [](https://microsoft.github.io/agent-lightning/latest/tutorials/emitter/#__codelineno-0-9) agl.emit_object({"plan_steps": len(plan), "first_step": plan[0]}) [](https://microsoft.github.io/agent-lightning/latest/tutorials/emitter/#__codelineno-0-10) [](https://microsoft.github.io/agent-lightning/latest/tutorials/emitter/#__codelineno-0-11) # Award a small reward for a valid plan [](https://microsoft.github.io/agent-lightning/latest/tutorials/emitter/#__codelineno-0-12) plan_reward = grade_plan(plan) [](https://microsoft.github.io/agent-lightning/latest/tutorials/emitter/#__codelineno-0-13) agl.emit_reward(plan_reward) [](https://microsoft.github.io/agent-lightning/latest/tutorials/emitter/#__codelineno-0-14) [](https://microsoft.github.io/agent-lightning/latest/tutorials/emitter/#__codelineno-0-15) # Step 2: Execute the plan [](https://microsoft.github.io/agent-lightning/latest/tutorials/emitter/#__codelineno-0-16) agl.emit_message(f"Executing {len(plan)}-step plan.") [](https://microsoft.github.io/agent-lightning/latest/tutorials/emitter/#__codelineno-0-17) execution_result = execute_plan(plan) [](https://microsoft.github.io/agent-lightning/latest/tutorials/emitter/#__codelineno-0-18) [](https://microsoft.github.io/agent-lightning/latest/tutorials/emitter/#__codelineno-0-19) # Step 3: Final evaluation [](https://microsoft.github.io/agent-lightning/latest/tutorials/emitter/#__codelineno-0-20) final_reward = custom_grade_final_result(execution_result, task["expected_output"]) [](https://microsoft.github.io/agent-lightning/latest/tutorials/emitter/#__codelineno-0-21) [](https://microsoft.github.io/agent-lightning/latest/tutorials/emitter/#__codelineno-0-22) # The return value is treated as the final reward for the rollout [](https://microsoft.github.io/agent-lightning/latest/tutorials/emitter/#__codelineno-0-23) return final_reward [](https://microsoft.github.io/agent-lightning/latest/tutorials/emitter/#__codelineno-0-24) [](https://microsoft.github.io/agent-lightning/latest/tutorials/emitter/#__codelineno-0-25) except ValueError as e: [](https://microsoft.github.io/agent-lightning/latest/tutorials/emitter/#__codelineno-0-26) # Record the specific error and return a failure reward [](https://microsoft.github.io/agent-lightning/latest/tutorials/emitter/#__codelineno-0-27) agl.emit_exception(e) [](https://microsoft.github.io/agent-lightning/latest/tutorials/emitter/#__codelineno-0-28) return 0.0` Each helper accepts nested `attributes` (or keyword arguments for [`operation`](https://microsoft.github.io/agent-lightning/latest/reference/agent/#agentlightning.operation " agentlightning.operation(fn=None, *, propagate=True, name=None, **additional_attributes)") ) and automatically flattens/sanitizes them into dotted OpenTelemetry keys. This means you can pass ordinary dictionaries/lists without pre-processing and still get consistent attribute names such as `meta.any_attribute` across all emitter operations. Agent-lightning does not restrict the attributes you supply, but it is best to consult [OpenTelemetry's semantic conventions](https://opentelemetry.io/docs/specs/semconv/) for recommended names. Agent-lightning also defines [specific semconv](https://microsoft.github.io/agent-lightning/latest/reference/semconv/) for its own use cases. The pattern looks like this: `[](https://microsoft.github.io/agent-lightning/latest/tutorials/emitter/#__codelineno-1-1) from opentelemetry.semconv.attributes import server_attributes [](https://microsoft.github.io/agent-lightning/latest/tutorials/emitter/#__codelineno-1-2) from agentlightning import emit_object [](https://microsoft.github.io/agent-lightning/latest/tutorials/emitter/#__codelineno-1-3) [](https://microsoft.github.io/agent-lightning/latest/tutorials/emitter/#__codelineno-1-4) emit_object({ [](https://microsoft.github.io/agent-lightning/latest/tutorials/emitter/#__codelineno-1-5) "name": "John Doe", [](https://microsoft.github.io/agent-lightning/latest/tutorials/emitter/#__codelineno-1-6) "age": 30, [](https://microsoft.github.io/agent-lightning/latest/tutorials/emitter/#__codelineno-1-7) "email": "john.doe@example.com", [](https://microsoft.github.io/agent-lightning/latest/tutorials/emitter/#__codelineno-1-8) }, attributes={ [](https://microsoft.github.io/agent-lightning/latest/tutorials/emitter/#__codelineno-1-9) server_attributes.SERVER_ADDRESS: "127.0.0.1", [](https://microsoft.github.io/agent-lightning/latest/tutorials/emitter/#__codelineno-1-10) server_attributes.SERVER_PORT: 8080, [](https://microsoft.github.io/agent-lightning/latest/tutorials/emitter/#__codelineno-1-11) })` Running the above code sends the following span to the backend if you have a tracer active: `[](https://microsoft.github.io/agent-lightning/latest/tutorials/emitter/#__codelineno-2-1) Span( [](https://microsoft.github.io/agent-lightning/latest/tutorials/emitter/#__codelineno-2-2) name='agentlightning.object', [](https://microsoft.github.io/agent-lightning/latest/tutorials/emitter/#__codelineno-2-3) attributes={ [](https://microsoft.github.io/agent-lightning/latest/tutorials/emitter/#__codelineno-2-4) 'agentlightning.object.type': 'dict', [](https://microsoft.github.io/agent-lightning/latest/tutorials/emitter/#__codelineno-2-5) 'agentlightning.object.json': '{"name": "John Doe", "age": 30, "email": "john.doe@example.com"}', [](https://microsoft.github.io/agent-lightning/latest/tutorials/emitter/#__codelineno-2-6) 'server.address': '127.0.0.1', [](https://microsoft.github.io/agent-lightning/latest/tutorials/emitter/#__codelineno-2-7) 'server.port': 8080 [](https://microsoft.github.io/agent-lightning/latest/tutorials/emitter/#__codelineno-2-8) } [](https://microsoft.github.io/agent-lightning/latest/tutorials/emitter/#__codelineno-2-9) )` Tip If you don't have a tracer active, the above code will raise the following error: `[](https://microsoft.github.io/agent-lightning/latest/tutorials/emitter/#__codelineno-3-1) RuntimeError: No active tracer found. Cannot emit object span.` By default, emitter helpers delegate to the active tracer to create and export spans (specifically via [`Tracer.create_span`](https://microsoft.github.io/agent-lightning/latest/reference/runner/#agentlightning.Tracer.create_span " create_span(name, attributes=None, timestamp=None, status=None)") ). If you want to emit spans without an active tracer, set `propagate=False` to keep the span local — a useful option for offline tests. The default `True` streams spans through the active tracer/exporters. When working with [agentlightning.semconv](https://microsoft.github.io/agent-lightning/latest/reference/semconv/) , you typically use utilities such as [`make_tag_attributes`](https://microsoft.github.io/agent-lightning/latest/reference/utilities/#agentlightning.utils.otel.make_tag_attributes " agentlightning.utils.otel.make_tag_attributes(tags)") and [`make_link_attributes`](https://microsoft.github.io/agent-lightning/latest/reference/utilities/#agentlightning.utils.otel.make_link_attributes " agentlightning.utils.otel.make_link_attributes(links)") to build the attributes dictionary. For example: `[](https://microsoft.github.io/agent-lightning/latest/tutorials/emitter/#__codelineno-4-1) from agentlightning.utils.otel import make_tag_attributes [](https://microsoft.github.io/agent-lightning/latest/tutorials/emitter/#__codelineno-4-2) [](https://microsoft.github.io/agent-lightning/latest/tutorials/emitter/#__codelineno-4-3) emit_annotation(make_tag_attributes(["tool", "calculator", "fast", "good"]))` The above code will send a span with the following attributes to the backend: `[](https://microsoft.github.io/agent-lightning/latest/tutorials/emitter/#__codelineno-5-1) { [](https://microsoft.github.io/agent-lightning/latest/tutorials/emitter/#__codelineno-5-2) "agentlightning.tag.0": "tool", [](https://microsoft.github.io/agent-lightning/latest/tutorials/emitter/#__codelineno-5-3) "agentlightning.tag.1": "calculator", [](https://microsoft.github.io/agent-lightning/latest/tutorials/emitter/#__codelineno-5-4) "agentlightning.tag.2": "fast", [](https://microsoft.github.io/agent-lightning/latest/tutorials/emitter/#__codelineno-5-5) "agentlightning.tag.3": "good" [](https://microsoft.github.io/agent-lightning/latest/tutorials/emitter/#__codelineno-5-6) }` A counterpart utility function [`extract_tags_from_attributes`](https://microsoft.github.io/agent-lightning/latest/reference/utilities/#agentlightning.utils.otel.extract_tags_from_attributes " agentlightning.utils.otel.extract_tags_from_attributes(attributes)") is also available to extract the tags from the attributes dictionary. Operations[¶](https://microsoft.github.io/agent-lightning/latest/tutorials/emitter/#operations "Permanent link") ----------------------------------------------------------------------------------------------------------------- The [`operation`](https://microsoft.github.io/agent-lightning/latest/reference/agent/#agentlightning.operation " agentlightning.operation(fn=None, *, propagate=True, name=None, **additional_attributes)") helper tracks logical units of work within your agent, capturing inputs, outputs, timing, and success/failure status. Unlike point-in-time emitters, operations create a span representing a time interval. Use operations for tool calls, multi-step workflows, debugging, and performance monitoring. [`operation`](https://microsoft.github.io/agent-lightning/latest/reference/agent/#agentlightning.operation " agentlightning.operation(fn=None, *, propagate=True, name=None, **additional_attributes)") works as either a decorator or a context manager. The decorator automatically captures function arguments as inputs and the return value as output: `[](https://microsoft.github.io/agent-lightning/latest/tutorials/emitter/#__codelineno-6-1) import agentlightning as agl [](https://microsoft.github.io/agent-lightning/latest/tutorials/emitter/#__codelineno-6-2) [](https://microsoft.github.io/agent-lightning/latest/tutorials/emitter/#__codelineno-6-3) @agl.operation [](https://microsoft.github.io/agent-lightning/latest/tutorials/emitter/#__codelineno-6-4) def search_documents(query: str, max_results: int = 10) -> list[dict]: [](https://microsoft.github.io/agent-lightning/latest/tutorials/emitter/#__codelineno-6-5) results = perform_search(query, max_results) [](https://microsoft.github.io/agent-lightning/latest/tutorials/emitter/#__codelineno-6-6) return results [](https://microsoft.github.io/agent-lightning/latest/tutorials/emitter/#__codelineno-6-7) [](https://microsoft.github.io/agent-lightning/latest/tutorials/emitter/#__codelineno-6-8) @agl.operation(category="tool", priority="high") [](https://microsoft.github.io/agent-lightning/latest/tutorials/emitter/#__codelineno-6-9) def execute_calculation(expression: str) -> float: [](https://microsoft.github.io/agent-lightning/latest/tutorials/emitter/#__codelineno-6-10) return eval_safely(expression)` The example above emits a span with `{"category": "tool", "priority": "high"}` attributes. It also records the function input and output via [OPERATION\_INPUT](https://microsoft.github.io/agent-lightning/latest/reference/semconv/#agentlightning.semconv.LightningSpanAttributes.OPERATION_INPUT " OPERATION_INPUT = 'agentlightning.operation.input' class-attribute instance-attribute ") and [OPERATION\_OUTPUT](https://microsoft.github.io/agent-lightning/latest/reference/semconv/#agentlightning.semconv.LightningSpanAttributes.OPERATION_OUTPUT " OPERATION_OUTPUT = 'agentlightning.operation.output' class-attribute instance-attribute ") . It works with async functions too: `[](https://microsoft.github.io/agent-lightning/latest/tutorials/emitter/#__codelineno-7-1) @agl.operation [](https://microsoft.github.io/agent-lightning/latest/tutorials/emitter/#__codelineno-7-2) async def async_api_call(endpoint: str, payload: dict) -> dict: [](https://microsoft.github.io/agent-lightning/latest/tutorials/emitter/#__codelineno-7-3) response = await http_client.post(endpoint, json=payload) [](https://microsoft.github.io/agent-lightning/latest/tutorials/emitter/#__codelineno-7-4) return response.json()` Override the operation name if needed: `[](https://microsoft.github.io/agent-lightning/latest/tutorials/emitter/#__codelineno-8-1) @agl.operation(name="custom-name") [](https://microsoft.github.io/agent-lightning/latest/tutorials/emitter/#__codelineno-8-2) def any_weird_name_i_dont_want(): [](https://microsoft.github.io/agent-lightning/latest/tutorials/emitter/#__codelineno-8-3) pass` For more control, [`operation`](https://microsoft.github.io/agent-lightning/latest/reference/agent/#agentlightning.operation " agentlightning.operation(fn=None, *, propagate=True, name=None, **additional_attributes)") can also be used as a context manager to explicitly record inputs and outputs: `[](https://microsoft.github.io/agent-lightning/latest/tutorials/emitter/#__codelineno-9-1) with agl.operation(tool_name="web_search") as op: [](https://microsoft.github.io/agent-lightning/latest/tutorials/emitter/#__codelineno-9-2) op.set_input(query="latest AI research", filters={"date": "2024"}) [](https://microsoft.github.io/agent-lightning/latest/tutorials/emitter/#__codelineno-9-3) results = search_web("latest AI research", {"date": "2024"}) [](https://microsoft.github.io/agent-lightning/latest/tutorials/emitter/#__codelineno-9-4) op.set_output({"result_count": len(results), "top_result": results[0]})` The `propagate=False` flag also applies to [`operation`](https://microsoft.github.io/agent-lightning/latest/reference/agent/#agentlightning.operation " agentlightning.operation(fn=None, *, propagate=True, name=None, **additional_attributes)") when you want to keep operations local without requiring an active tracer: `[](https://microsoft.github.io/agent-lightning/latest/tutorials/emitter/#__codelineno-10-1) @agl.operation(propagate=False) [](https://microsoft.github.io/agent-lightning/latest/tutorials/emitter/#__codelineno-10-2) def local_test(): [](https://microsoft.github.io/agent-lightning/latest/tutorials/emitter/#__codelineno-10-3) return "Not sent to backend"` Linking to Other Spans[¶](https://microsoft.github.io/agent-lightning/latest/tutorials/emitter/#linking-to-other-spans "Permanent link") ----------------------------------------------------------------------------------------------------------------------------------------- Sometimes a span should explicitly point back to another span that produced the input it is working on (for example, linking a reward annotation to the [`agentlightning.operation`](https://microsoft.github.io/agent-lightning/latest/reference/agent/#agentlightning.operation " agentlightning.operation(fn=None, *, propagate=True, name=None, **additional_attributes)") span that generated a response). Agent-lightning encodes these relationships through flattened link attributes. The helper [`make_link_attributes`](https://microsoft.github.io/agent-lightning/latest/reference/utilities/#agentlightning.utils.otel.make_link_attributes " agentlightning.utils.otel.make_link_attributes(links)") converts a dictionary of keys such as `trace_id`, `span_id`, or any custom attribute into the `"agentlightning.link.*"` ([LightningSpanAttributes.LINK](https://microsoft.github.io/agent-lightning/latest/reference/semconv/#agentlightning.semconv.LightningSpanAttributes.LINK " LINK = 'agentlightning.link' class-attribute instance-attribute ") ) fields expected by the backend. Later, [`query_linked_spans`](https://microsoft.github.io/agent-lightning/latest/reference/utilities/#agentlightning.utils.otel.query_linked_spans " agentlightning.utils.otel.query_linked_spans(spans, links)") can recover the original span(s) from those link descriptors. `[](https://microsoft.github.io/agent-lightning/latest/tutorials/emitter/#__codelineno-11-1) import opentelemetry.trace as trace_api [](https://microsoft.github.io/agent-lightning/latest/tutorials/emitter/#__codelineno-11-2) from agentlightning import emit_annotation, operation [](https://microsoft.github.io/agent-lightning/latest/tutorials/emitter/#__codelineno-11-3) from agentlightning.utils.otel import make_link_attributes, make_tag_attributes [](https://microsoft.github.io/agent-lightning/latest/tutorials/emitter/#__codelineno-11-4) [](https://microsoft.github.io/agent-lightning/latest/tutorials/emitter/#__codelineno-11-5) with operation(conversation_id="chat-42") as op: [](https://microsoft.github.io/agent-lightning/latest/tutorials/emitter/#__codelineno-11-6) # ... perform the work ... [](https://microsoft.github.io/agent-lightning/latest/tutorials/emitter/#__codelineno-11-7) link_attrs = make_link_attributes({ [](https://microsoft.github.io/agent-lightning/latest/tutorials/emitter/#__codelineno-11-8) "conversation_id": "chat-42", [](https://microsoft.github.io/agent-lightning/latest/tutorials/emitter/#__codelineno-11-9) }) [](https://microsoft.github.io/agent-lightning/latest/tutorials/emitter/#__codelineno-11-10) [](https://microsoft.github.io/agent-lightning/latest/tutorials/emitter/#__codelineno-11-11) emit_annotation( [](https://microsoft.github.io/agent-lightning/latest/tutorials/emitter/#__codelineno-11-12) { [](https://microsoft.github.io/agent-lightning/latest/tutorials/emitter/#__codelineno-11-13) **link_attrs, [](https://microsoft.github.io/agent-lightning/latest/tutorials/emitter/#__codelineno-11-14) **make_tag_attributes(["reward", "good"]), [](https://microsoft.github.io/agent-lightning/latest/tutorials/emitter/#__codelineno-11-15) } [](https://microsoft.github.io/agent-lightning/latest/tutorials/emitter/#__codelineno-11-16) )` When analyzing in adapters, pass the extracted link models to [`query_linked_spans`](https://microsoft.github.io/agent-lightning/latest/reference/utilities/#agentlightning.utils.otel.query_linked_spans " agentlightning.utils.otel.query_linked_spans(spans, links)") to retrieve the matching span(s): `[](https://microsoft.github.io/agent-lightning/latest/tutorials/emitter/#__codelineno-12-1) from agentlightning.utils.otel import extract_links_from_attributes, query_linked_spans [](https://microsoft.github.io/agent-lightning/latest/tutorials/emitter/#__codelineno-12-2) [](https://microsoft.github.io/agent-lightning/latest/tutorials/emitter/#__codelineno-12-3) annotation_span = ... # Span from your trace store [](https://microsoft.github.io/agent-lightning/latest/tutorials/emitter/#__codelineno-12-4) operation_spans = [...] # list of spans you want to search [](https://microsoft.github.io/agent-lightning/latest/tutorials/emitter/#__codelineno-12-5) [](https://microsoft.github.io/agent-lightning/latest/tutorials/emitter/#__codelineno-12-6) link_models = extract_links_from_attributes(annotation_span.attributes) [](https://microsoft.github.io/agent-lightning/latest/tutorials/emitter/#__codelineno-12-7) matches = query_linked_spans(operation_spans, link_models) [](https://microsoft.github.io/agent-lightning/latest/tutorials/emitter/#__codelineno-12-8) assert matches # Contains the original operation span` Correlating Rewards with LLM Requests [Tracer](https://microsoft.github.io/agent-lightning/latest/tutorials/traces/) instruments each request/response as its own span. You can link to the [`gen_ai.response.id`](https://opentelemetry.io/docs/specs/semconv/gen-ai/gen-ai-events/) attribute, which comes from the LLM response ID. `[](https://microsoft.github.io/agent-lightning/latest/tutorials/emitter/#__codelineno-13-1) from agentlightning import emit_reward [](https://microsoft.github.io/agent-lightning/latest/tutorials/emitter/#__codelineno-13-2) from agentlightning.utils.otel import make_link_attributes [](https://microsoft.github.io/agent-lightning/latest/tutorials/emitter/#__codelineno-13-3) [](https://microsoft.github.io/agent-lightning/latest/tutorials/emitter/#__codelineno-13-4) result = call_llm(prompt) [](https://microsoft.github.io/agent-lightning/latest/tutorials/emitter/#__codelineno-13-5) reward_links = make_link_attributes({"gen_ai.response.id": result.id}) [](https://microsoft.github.io/agent-lightning/latest/tutorials/emitter/#__codelineno-13-6) emit_reward(0.9, attributes=reward_links)` Later, use the same `gen_ai.response.id` key inside `query_linked_spans` to find the reward(s) that reference that specific LLM request span. Back to top --- # Write Agents - Agent-lightning [Skip to content](https://microsoft.github.io/agent-lightning/latest/tutorials/write-agents/#writing-agents) ⚠️ You are viewing development documentation. For stable documentation, please visit [https://microsoft.github.io/agent-lightning/stable/](https://microsoft.github.io/agent-lightning/stable/) Writing Agents[¶](https://microsoft.github.io/agent-lightning/latest/tutorials/write-agents/#writing-agents "Permanent link") ============================================================================================================================== This tutorial will focus on the heart of the system: the agent itself, guiding you through the different ways to define an agent's logic in Agent-lightning. The basic requirements for any agent are: 1. It must accept a single **task** as input. 2. It must accept a set of tunable **resources** (like a [PromptTemplate](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.PromptTemplate " agentlightning.PromptTemplate") or [LLM](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.LLM " agentlightning.LLM") ). 3. It must **emit** trace span data so that algorithms can understand its behavior and learn from it. _The simplest way to do this is by returning a final reward._ In practice, please also bear in mind that tasks, resources, and spans have extra requirements, in order to make it _trainable_ within Agent-lightning: 1. You will need a training dataset containing a set of tasks, of the same type that your agent expects as input. 2. The tunable resources are related to the algorithm. For example, the APO algorithm we've seen tunes a [PromptTemplate](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.PromptTemplate " agentlightning.PromptTemplate") . Other algorithms might tune model weights or other configurations. 3. The type of spans an algorithm can use varies. Almost all algorithms support a single, final reward span at the end of a rollout. However, not all algorithms support rewards emitted mid-rollout, let alone other kinds of spans like exceptions or log messages. This tutorial will show you how to write an agent that can handle various tasks and resources and emit all kinds of spans. However, you should understand that agents and algorithms are often co-designed. Supporting new types of resources or spans in an algorithm is often much more complex than just adding them to an agent. [`@rollout`](https://microsoft.github.io/agent-lightning/latest/reference/agent/#agentlightning.rollout " agentlightning.rollout(func)") Decorator[¶](https://microsoft.github.io/agent-lightning/latest/tutorials/write-agents/#rollout-decorator "Permanent link") --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- The simplest way to create an agent is by writing a standard Python function and marking it with the [@rollout](https://microsoft.github.io/agent-lightning/latest/reference/agent/#agentlightning.rollout " agentlightning.rollout(func)") decorator. This approach is perfect for agents with straightforward logic that doesn't require complex state management. Agent-lightning automatically inspects your function's signature and injects the required resources. For example, if your function has a parameter named `prompt_template`, Agent-lightning will find the [PromptTemplate](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.PromptTemplate " agentlightning.PromptTemplate") resource for the current rollout and pass it in. Let's revisit the `room_selector` agent from the first tutorial: `[](https://microsoft.github.io/agent-lightning/latest/tutorials/write-agents/#__codelineno-0-1) from typing import TypedDict [](https://microsoft.github.io/agent-lightning/latest/tutorials/write-agents/#__codelineno-0-2) from agentlightning import PromptTemplate, rollout [](https://microsoft.github.io/agent-lightning/latest/tutorials/write-agents/#__codelineno-0-3) [](https://microsoft.github.io/agent-lightning/latest/tutorials/write-agents/#__codelineno-0-4) # Define a data structure for the task input [](https://microsoft.github.io/agent-lightning/latest/tutorials/write-agents/#__codelineno-0-5) class RoomSelectionTask(TypedDict): [](https://microsoft.github.io/agent-lightning/latest/tutorials/write-agents/#__codelineno-0-6) # ... fields for the task ... [](https://microsoft.github.io/agent-lightning/latest/tutorials/write-agents/#__codelineno-0-7) pass [](https://microsoft.github.io/agent-lightning/latest/tutorials/write-agents/#__codelineno-0-8) [](https://microsoft.github.io/agent-lightning/latest/tutorials/write-agents/#__codelineno-0-9) @rollout [](https://microsoft.github.io/agent-lightning/latest/tutorials/write-agents/#__codelineno-0-10) def room_selector(task: RoomSelectionTask, prompt_template: PromptTemplate) -> float: [](https://microsoft.github.io/agent-lightning/latest/tutorials/write-agents/#__codelineno-0-11) # 1. Use the injected prompt_template to format the input for the LLM [](https://microsoft.github.io/agent-lightning/latest/tutorials/write-agents/#__codelineno-0-12) prompt = prompt_template.format(**task) [](https://microsoft.github.io/agent-lightning/latest/tutorials/write-agents/#__codelineno-0-13) [](https://microsoft.github.io/agent-lightning/latest/tutorials/write-agents/#__codelineno-0-14) # 2. Execute the agent's logic (e.g., call an LLM, use tools) [](https://microsoft.github.io/agent-lightning/latest/tutorials/write-agents/#__codelineno-0-15) # ... [](https://microsoft.github.io/agent-lightning/latest/tutorials/write-agents/#__codelineno-0-16) [](https://microsoft.github.io/agent-lightning/latest/tutorials/write-agents/#__codelineno-0-17) # 3. Grade the final choice to get a reward [](https://microsoft.github.io/agent-lightning/latest/tutorials/write-agents/#__codelineno-0-18) reward = room_selection_grader(final_message, task["expected_choice"]) [](https://microsoft.github.io/agent-lightning/latest/tutorials/write-agents/#__codelineno-0-19) [](https://microsoft.github.io/agent-lightning/latest/tutorials/write-agents/#__codelineno-0-20) # 4. Return the final reward as a float [](https://microsoft.github.io/agent-lightning/latest/tutorials/write-agents/#__codelineno-0-21) return reward` When you train this agent, the dataset is expected to be a list of `RoomSelectionTask` objects: `[](https://microsoft.github.io/agent-lightning/latest/tutorials/write-agents/#__codelineno-1-1) from agentlightning import Dataset, Trainer [](https://microsoft.github.io/agent-lightning/latest/tutorials/write-agents/#__codelineno-1-2) [](https://microsoft.github.io/agent-lightning/latest/tutorials/write-agents/#__codelineno-1-3) dataset: Dataset[RoomSelectionTask] = [ [](https://microsoft.github.io/agent-lightning/latest/tutorials/write-agents/#__codelineno-1-4) RoomSelectionTask(date="2025-10-15", time="10:00", duration_min=60, attendees=10), [](https://microsoft.github.io/agent-lightning/latest/tutorials/write-agents/#__codelineno-1-5) RoomSelectionTask(date="2025-10-16", time="10:00", duration_min=60, attendees=10), [](https://microsoft.github.io/agent-lightning/latest/tutorials/write-agents/#__codelineno-1-6) ] [](https://microsoft.github.io/agent-lightning/latest/tutorials/write-agents/#__codelineno-1-7) [](https://microsoft.github.io/agent-lightning/latest/tutorials/write-agents/#__codelineno-1-8) Trainer().fit(agent=room_selector, train_dataset=dataset)` Behind the scenes, the [`@rollout`](https://microsoft.github.io/agent-lightning/latest/reference/agent/#agentlightning.rollout " agentlightning.rollout(func)") decorator wraps your function in a `FunctionalLitAgent` object, which is a subclass of [LitAgent](https://microsoft.github.io/agent-lightning/latest/reference/agent/#agentlightning.LitAgent " agentlightning.LitAgent") introduced below, making it compatible with the [Trainer](https://microsoft.github.io/agent-lightning/latest/reference/trainer/#agentlightning.Trainer " agentlightning.Trainer") and [Runner](https://microsoft.github.io/agent-lightning/latest/reference/runner/#agentlightning.Runner " agentlightning.Runner") . It supports parameters like `task`, `prompt_template`, `llm`, and `rollout`, giving you flexible access to the execution context. Here is another example with more advanced usage with `llm` and `rollout` as parameters. The `llm` parameter gives you an OpenAI-compatible LLM endpoint to interact with, which can be tuned under the hood by algorithms. The `rollout` parameter gives you the full [Rollout](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.Rollout " agentlightning.Rollout") object, which contains the rollout ID, rollout mode (training or validation), etc. ``[](https://microsoft.github.io/agent-lightning/latest/tutorials/write-agents/#__codelineno-2-1) from openai import OpenAI [](https://microsoft.github.io/agent-lightning/latest/tutorials/write-agents/#__codelineno-2-2) from agentlightning import LLM, Rollout [](https://microsoft.github.io/agent-lightning/latest/tutorials/write-agents/#__codelineno-2-3) [](https://microsoft.github.io/agent-lightning/latest/tutorials/write-agents/#__codelineno-2-4) class FlightBookingTask(TypedDict): [](https://microsoft.github.io/agent-lightning/latest/tutorials/write-agents/#__codelineno-2-5) request: str [](https://microsoft.github.io/agent-lightning/latest/tutorials/write-agents/#__codelineno-2-6) expected_booking: dict [](https://microsoft.github.io/agent-lightning/latest/tutorials/write-agents/#__codelineno-2-7) [](https://microsoft.github.io/agent-lightning/latest/tutorials/write-agents/#__codelineno-2-8) @rollout [](https://microsoft.github.io/agent-lightning/latest/tutorials/write-agents/#__codelineno-2-9) def flight_assistant(task: FlightBookingTask, llm: LLM, rollout: Rollout) -> float: [](https://microsoft.github.io/agent-lightning/latest/tutorials/write-agents/#__codelineno-2-10) print(f"Rollout ID: {rollout.rollout_id}") [](https://microsoft.github.io/agent-lightning/latest/tutorials/write-agents/#__codelineno-2-11) print(f"Rollout Mode: {rollout.mode}") [](https://microsoft.github.io/agent-lightning/latest/tutorials/write-agents/#__codelineno-2-12) [](https://microsoft.github.io/agent-lightning/latest/tutorials/write-agents/#__codelineno-2-13) # Use the tuned LLM resource to create an OpenAI client [](https://microsoft.github.io/agent-lightning/latest/tutorials/write-agents/#__codelineno-2-14) client = OpenAI( [](https://microsoft.github.io/agent-lightning/latest/tutorials/write-agents/#__codelineno-2-15) # This endpoint could be a proxy to a proxy to a proxy ... [](https://microsoft.github.io/agent-lightning/latest/tutorials/write-agents/#__codelineno-2-16) # It could be different every time `flight_assistant` is called [](https://microsoft.github.io/agent-lightning/latest/tutorials/write-agents/#__codelineno-2-17) # But it should be OpenAI-API compatible [](https://microsoft.github.io/agent-lightning/latest/tutorials/write-agents/#__codelineno-2-18) base_url=llm.endpoint, [](https://microsoft.github.io/agent-lightning/latest/tutorials/write-agents/#__codelineno-2-19) [](https://microsoft.github.io/agent-lightning/latest/tutorials/write-agents/#__codelineno-2-20) # Use a dummy key if not provided [](https://microsoft.github.io/agent-lightning/latest/tutorials/write-agents/#__codelineno-2-21) # Usually this does not matter because the training LLM is often not guarded by an API key [](https://microsoft.github.io/agent-lightning/latest/tutorials/write-agents/#__codelineno-2-22) # But you can use `or os.environ["OPENAI_API_KEY"]` to make the function compatible with 3rd-party LLMs [](https://microsoft.github.io/agent-lightning/latest/tutorials/write-agents/#__codelineno-2-23) api_key=llm.api_key or "dummy-key", [](https://microsoft.github.io/agent-lightning/latest/tutorials/write-agents/#__codelineno-2-24) ) [](https://microsoft.github.io/agent-lightning/latest/tutorials/write-agents/#__codelineno-2-25) [](https://microsoft.github.io/agent-lightning/latest/tutorials/write-agents/#__codelineno-2-26) # Make an API call with the specified model [](https://microsoft.github.io/agent-lightning/latest/tutorials/write-agents/#__codelineno-2-27) response = client.chat.completions.create( [](https://microsoft.github.io/agent-lightning/latest/tutorials/write-agents/#__codelineno-2-28) model=llm.model, [](https://microsoft.github.io/agent-lightning/latest/tutorials/write-agents/#__codelineno-2-29) messages=[{"role": "user", "content": task["request"]}], [](https://microsoft.github.io/agent-lightning/latest/tutorials/write-agents/#__codelineno-2-30) ) [](https://microsoft.github.io/agent-lightning/latest/tutorials/write-agents/#__codelineno-2-31) # Whether the API supports features like streaming, tool calls, etc. depends on [](https://microsoft.github.io/agent-lightning/latest/tutorials/write-agents/#__codelineno-2-32) # the endpoint that algorithms are serving to you. [](https://microsoft.github.io/agent-lightning/latest/tutorials/write-agents/#__codelineno-2-33) final_message = response.choices[0].message.content [](https://microsoft.github.io/agent-lightning/latest/tutorials/write-agents/#__codelineno-2-34) [](https://microsoft.github.io/agent-lightning/latest/tutorials/write-agents/#__codelineno-2-35) # Grade the result and return a reward [](https://microsoft.github.io/agent-lightning/latest/tutorials/write-agents/#__codelineno-2-36) reward = grade_flight_booking(final_message, task["expected_booking"]) [](https://microsoft.github.io/agent-lightning/latest/tutorials/write-agents/#__codelineno-2-37) return reward`` Return Values from Agents[¶](https://microsoft.github.io/agent-lightning/latest/tutorials/write-agents/#return-values-from-agents "Permanent link") ---------------------------------------------------------------------------------------------------------------------------------------------------- The value your agent function returns (i.e., the return value of the function decorated by [`@rollout`](https://microsoft.github.io/agent-lightning/latest/reference/agent/#agentlightning.rollout " agentlightning.rollout(func)") ) is crucial, as it's the primary way to report the outcome of a rollout. Agent-lightning supports several return types to accommodate different scenarios, from simple rewards to detailed, custom traces. * **`float`**: This is the simplest and most common return type. The `float` is treated as the **final reward** for the entire rollout. Agent-lightning automatically creates a final reward span based on this value. * **`None`**: Returning `None` tells the runner that trace collection is being handled entirely by the [Tracer](https://microsoft.github.io/agent-lightning/latest/reference/runner/#agentlightning.Tracer " agentlightning.Tracer") through auto-instrumentation (e.g., via AgentOps). In this case, the runner will simply retrieve the spans that the tracer has already captured. Emitting the Final Reward When returning `None`, you must still ensure a final reward is logged. You can do this by using the [`emit_reward`](https://microsoft.github.io/agent-lightning/latest/reference/agent/#agentlightning.emit_reward " agentlightning.emit_reward(reward, *, primary_key=None, attributes=None, propagate=True)") function (covered in the [Use Emitters](https://microsoft.github.io/agent-lightning/latest/tutorials/emitter/) documentation). Wrapping your reward calculation function with the `@reward` decorator is NOT the recommended approach any more. * **`list[ReadableSpan]`**, **`list[SpanCoreFields]`**, or **`list[Span]`**: For advanced use cases, you can manually construct and return a complete list of all spans for the rollout. This gives you full control over the trace data. You can return either a list of OpenTelemetry `ReadableSpan` objects or Agent-lightning's native `Span` objects. For most users, returning a **`float`** for simple agents or returning **`None`** and using the emitter for more complex ones are the recommended approaches. Class-based Agents[¶](https://microsoft.github.io/agent-lightning/latest/tutorials/write-agents/#class-based-agents "Permanent link") -------------------------------------------------------------------------------------------------------------------------------------- For more complex agents that require state, helper methods, or distinct logic for training versus validation, you can create a class that inherits from [`LitAgent`](https://microsoft.github.io/agent-lightning/latest/reference/agent/#agentlightning.LitAgent " agentlightning.LitAgent") . This object-oriented approach provides more structure and control over the agent's lifecycle. To create a class-based agent, you subclass [agentlightning.LitAgent](https://microsoft.github.io/agent-lightning/latest/reference/agent/#agentlightning.LitAgent " agentlightning.LitAgent") and implement its [`rollout`](https://microsoft.github.io/agent-lightning/latest/reference/agent/#agentlightning.LitAgent.rollout " rollout(task, resources, rollout)") method. [](https://microsoft.github.io/agent-lightning/latest/tutorials/write-agents/) Here's how the `room_selector` could be implemented as a class. The rollout method has a slightly different signature than the function-based agent, mainly in how it handles the resources. Putting it simply, algorithms do not just send a [PromptTemplate](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.PromptTemplate " agentlightning.PromptTemplate") to the agents, they instead send [NamedResources](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.NamedResources " agentlightning.NamedResources = Dict[str, ResourceUnion] module-attribute ") , which is a mapping from resource key to [Resource](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.Resource " agentlightning.Resource") . This design is to allow for more advanced features like multi-resource tuning. With [`@rollout`](https://microsoft.github.io/agent-lightning/latest/reference/agent/#agentlightning.rollout " agentlightning.rollout(func)") decorator, the resource with correctly matched type will be automatically injected into the rollout method. However, when you use a class-based agent, you need to manually access the resource from the `resources` dictionary. Built-in algorithms listed their resource key naming conventions [here](https://microsoft.github.io/agent-lightning/latest/algorithm-zoo/) . `[](https://microsoft.github.io/agent-lightning/latest/tutorials/write-agents/#__codelineno-3-1) import agentlightning as agl [](https://microsoft.github.io/agent-lightning/latest/tutorials/write-agents/#__codelineno-3-2) [](https://microsoft.github.io/agent-lightning/latest/tutorials/write-agents/#__codelineno-3-3) class RoomSelectorAgent(agl.LitAgent[RoomSelectionTask]): [](https://microsoft.github.io/agent-lightning/latest/tutorials/write-agents/#__codelineno-3-4) def rollout(self, task: RoomSelectionTask, resources: agl.NamedResources, rollout: agl.Rollout) -> float: [](https://microsoft.github.io/agent-lightning/latest/tutorials/write-agents/#__codelineno-3-5) # 1. Access the prompt_template from the resources dictionary [](https://microsoft.github.io/agent-lightning/latest/tutorials/write-agents/#__codelineno-3-6) prompt_template = resources["prompt_template"] [](https://microsoft.github.io/agent-lightning/latest/tutorials/write-agents/#__codelineno-3-7) [](https://microsoft.github.io/agent-lightning/latest/tutorials/write-agents/#__codelineno-3-8) # 2. Execute the agent's logic [](https://microsoft.github.io/agent-lightning/latest/tutorials/write-agents/#__codelineno-3-9) prompt = prompt_template.format(**task) [](https://microsoft.github.io/agent-lightning/latest/tutorials/write-agents/#__codelineno-3-10) # ... [](https://microsoft.github.io/agent-lightning/latest/tutorials/write-agents/#__codelineno-3-11) [](https://microsoft.github.io/agent-lightning/latest/tutorials/write-agents/#__codelineno-3-12) # 3. Grade the final choice [](https://microsoft.github.io/agent-lightning/latest/tutorials/write-agents/#__codelineno-3-13) reward = room_selection_grader(final_message, task["expected_choice"]) [](https://microsoft.github.io/agent-lightning/latest/tutorials/write-agents/#__codelineno-3-14) [](https://microsoft.github.io/agent-lightning/latest/tutorials/write-agents/#__codelineno-3-15) # 4. Return the final reward [](https://microsoft.github.io/agent-lightning/latest/tutorials/write-agents/#__codelineno-3-16) return reward [](https://microsoft.github.io/agent-lightning/latest/tutorials/write-agents/#__codelineno-3-17) [](https://microsoft.github.io/agent-lightning/latest/tutorials/write-agents/#__codelineno-3-18) # To use it with the trainer: [](https://microsoft.github.io/agent-lightning/latest/tutorials/write-agents/#__codelineno-3-19) # agent = RoomSelectorAgent() [](https://microsoft.github.io/agent-lightning/latest/tutorials/write-agents/#__codelineno-3-20) # trainer.fit(agent=agent, ...)` The [`LitAgent`](https://microsoft.github.io/agent-lightning/latest/reference/agent/#agentlightning.LitAgent " agentlightning.LitAgent") class provides several methods you can override for more fine-grained control: * [`rollout()`](https://microsoft.github.io/agent-lightning/latest/reference/agent/#agentlightning.LitAgent.rollout " rollout(task, resources, rollout)") : The primary method for the agent's logic. It's called for both training and validation by default. * [`training_rollout()`](https://microsoft.github.io/agent-lightning/latest/reference/agent/#agentlightning.LitAgent.training_rollout " training_rollout(task, resources, rollout)") / [`validation_rollout()`](https://microsoft.github.io/agent-lightning/latest/reference/agent/#agentlightning.LitAgent.validation_rollout " validation_rollout(task, resources, rollout)") : Implement these if you need different behavior during training (e.g., with exploration) and validation (e.g., with deterministic choices). * [`rollout_async()`](https://microsoft.github.io/agent-lightning/latest/reference/agent/#agentlightning.LitAgent.rollout_async " rollout_async(task, resources, rollout) async ") / [`training_rollout_async()`](https://microsoft.github.io/agent-lightning/latest/reference/agent/#agentlightning.LitAgent.training_rollout_async " training_rollout_async(task, resources, rollout) async ") / [`validation_rollout_async()`](https://microsoft.github.io/agent-lightning/latest/reference/agent/#agentlightning.LitAgent.validation_rollout_async " validation_rollout_async(task, resources, rollout) async ") : Implement the asynchronous versions of these methods if your agent uses `asyncio`. Note Rollout is always executed in an asynchronous context no matter whether the agent is asynchronous or synchronous. If your synchronous agent contains some `asyncio.run()` calls, it might raise an error that there is already an event loop running. To avoid blocking the event loop, it's recommended to offload the inner async operations to a separate thread. Here is a sample code: `[](https://microsoft.github.io/agent-lightning/latest/tutorials/write-agents/#__codelineno-4-1) import asyncio [](https://microsoft.github.io/agent-lightning/latest/tutorials/write-agents/#__codelineno-4-2) import queue [](https://microsoft.github.io/agent-lightning/latest/tutorials/write-agents/#__codelineno-4-3) import threading [](https://microsoft.github.io/agent-lightning/latest/tutorials/write-agents/#__codelineno-4-4) [](https://microsoft.github.io/agent-lightning/latest/tutorials/write-agents/#__codelineno-4-5) def run_sync_ephemeral(coro) -> Any: [](https://microsoft.github.io/agent-lightning/latest/tutorials/write-agents/#__codelineno-4-6) """ [](https://microsoft.github.io/agent-lightning/latest/tutorials/write-agents/#__codelineno-4-7) Run an async coroutine from sync code. [](https://microsoft.github.io/agent-lightning/latest/tutorials/write-agents/#__codelineno-4-8) - If no loop in this thread: use asyncio.run() directly. [](https://microsoft.github.io/agent-lightning/latest/tutorials/write-agents/#__codelineno-4-9) - If already in an event loop: spawn a worker thread that calls asyncio.run() [](https://microsoft.github.io/agent-lightning/latest/tutorials/write-agents/#__codelineno-4-10) (which creates and closes a brand-new event loop per call). [](https://microsoft.github.io/agent-lightning/latest/tutorials/write-agents/#__codelineno-4-11) """ [](https://microsoft.github.io/agent-lightning/latest/tutorials/write-agents/#__codelineno-4-12) try: [](https://microsoft.github.io/agent-lightning/latest/tutorials/write-agents/#__codelineno-4-13) asyncio.get_running_loop() [](https://microsoft.github.io/agent-lightning/latest/tutorials/write-agents/#__codelineno-4-14) except RuntimeError: [](https://microsoft.github.io/agent-lightning/latest/tutorials/write-agents/#__codelineno-4-15) # No running loop in this thread; safe to use asyncio.run [](https://microsoft.github.io/agent-lightning/latest/tutorials/write-agents/#__codelineno-4-16) return asyncio.run(coro) [](https://microsoft.github.io/agent-lightning/latest/tutorials/write-agents/#__codelineno-4-17) [](https://microsoft.github.io/agent-lightning/latest/tutorials/write-agents/#__codelineno-4-18) # Already in a running loop -> execute in a worker thread [](https://microsoft.github.io/agent-lightning/latest/tutorials/write-agents/#__codelineno-4-19) q = queue.Queue[Any]() [](https://microsoft.github.io/agent-lightning/latest/tutorials/write-agents/#__codelineno-4-20) [](https://microsoft.github.io/agent-lightning/latest/tutorials/write-agents/#__codelineno-4-21) def worker(): [](https://microsoft.github.io/agent-lightning/latest/tutorials/write-agents/#__codelineno-4-22) try: [](https://microsoft.github.io/agent-lightning/latest/tutorials/write-agents/#__codelineno-4-23) result = asyncio.run(coro) # creates & closes its own loop [](https://microsoft.github.io/agent-lightning/latest/tutorials/write-agents/#__codelineno-4-24) q.put((True, result)) [](https://microsoft.github.io/agent-lightning/latest/tutorials/write-agents/#__codelineno-4-25) except BaseException as e: [](https://microsoft.github.io/agent-lightning/latest/tutorials/write-agents/#__codelineno-4-26) q.put((False, e)) [](https://microsoft.github.io/agent-lightning/latest/tutorials/write-agents/#__codelineno-4-27) [](https://microsoft.github.io/agent-lightning/latest/tutorials/write-agents/#__codelineno-4-28) t = threading.Thread(target=worker, daemon=True) [](https://microsoft.github.io/agent-lightning/latest/tutorials/write-agents/#__codelineno-4-29) t.start() [](https://microsoft.github.io/agent-lightning/latest/tutorials/write-agents/#__codelineno-4-30) ok, payload = q.get() [](https://microsoft.github.io/agent-lightning/latest/tutorials/write-agents/#__codelineno-4-31) t.join() [](https://microsoft.github.io/agent-lightning/latest/tutorials/write-agents/#__codelineno-4-32) if ok: [](https://microsoft.github.io/agent-lightning/latest/tutorials/write-agents/#__codelineno-4-33) return payload [](https://microsoft.github.io/agent-lightning/latest/tutorials/write-agents/#__codelineno-4-34) raise payload` Back to top --- # SFT with Unsloth - Agent-lightning [Skip to content](https://microsoft.github.io/agent-lightning/latest/how-to/unsloth-sft/#fine-tune-with-unsloth-sft) ⚠️ You are viewing development documentation. For stable documentation, please visit [https://microsoft.github.io/agent-lightning/stable/](https://microsoft.github.io/agent-lightning/stable/) Fine-tune with Unsloth SFT[¶](https://microsoft.github.io/agent-lightning/latest/how-to/unsloth-sft/#fine-tune-with-unsloth-sft "Permanent link") ================================================================================================================================================== Prerequisites Please make sure you have read [Write the First Algorithm](https://microsoft.github.io/agent-lightning/latest/how-to/write-first-algorithm/) . Although that recipe is based on a simple prompt tuning algorithm, it introduces the core concepts of Agent-lightning and you should be familiar with them before proceeding. This recipe builds on [Write the First Algorithm](https://microsoft.github.io/agent-lightning/latest/how-to/write-first-algorithm/) . Instead of iterating on a prompt, we will fine-tune a large language model with [Unsloth](https://docs.unsloth.ai/) 's SFT Trainer and keep the whole loop inside Agent-lightning. The new pieces you will meet are the **LLM proxy**, the **trace-to-triplet adapter**, a [vLLM](https://github.com/vllm-project/vllm) inference endpoint, and an agent implemented with the [OpenAI Agents SDK](https://openai.github.io/openai-agents-python/) . The full sample code is available in the [`examples/unsloth`](https://github.com/microsoft/agent-lightning/tree/c746af2f76bebcae56007a59c223cdace411c89b/examples/unsloth) folder. Warning You need a GPU that can host the Unsloth base model and run vLLM. The sample defaults to `unsloth/Qwen3-4B-Instruct-2507`, which requires at least 16GB of GPU memory under 4-bit quantization. The Data and Serving Loop[¶](https://microsoft.github.io/agent-lightning/latest/how-to/unsloth-sft/#the-data-and-serving-loop "Permanent link") ------------------------------------------------------------------------------------------------------------------------------------------------ To tune a large language model in Supervised Fine-Tuning (SFT), we commonly need a dataset with input/output samples. For example, the [TRL SFT Trainer](https://huggingface.co/docs/trl/sft_trainer) expects a dataset with samples like the following: `[](https://microsoft.github.io/agent-lightning/latest/how-to/unsloth-sft/#__codelineno-0-1) {"messages": [{"role": "user", "content": "What color is the sky?"}, [](https://microsoft.github.io/agent-lightning/latest/how-to/unsloth-sft/#__codelineno-0-2) {"role": "assistant", "content": "It is blue."}]}` With supervised fine-tuning, the LLM learns to generate the "assistant" response as close as possible to the completion in the dataset. Typically, the dataset used in SFT should be a curated set of samples. The samples can be either hand-written by humans, or generated by a more powerful model, which is known as [data distillation](https://docs.nvidia.com/nemo-framework/user-guide/24.12/modelalignment/knowledge-distillation.html) . However, in this recipe, we use a different setup that relies on samples generated by the model itself. We use the reward emitted by the agent to select the top-performing samples. Overall, the flow of the algorithm is an iteration of the following steps: 1. Serve the current checkpoint (with vLLM). 2. Publish the vLLM endpoint through the LLM proxy and let runners roll out some tasks with the current model. 3. Collect the traces from the rollouts and transform the highest-rewarded ones into a dataset that is acceptable for Unsloth SFT Trainer. 4. Launch Unsloth to fine-tune on the dataset and save a new checkpoint. You will find the full source code of this iteration in `sft_one_iter` in [sft\_algorithm.py](https://github.com/microsoft/agent-lightning/tree/c746af2f76bebcae56007a59c223cdace411c89b/examples/unsloth/sft_algorithm.py) . We will elaborate on each part below. ### Serving the Model with vLLM and Proxy[¶](https://microsoft.github.io/agent-lightning/latest/how-to/unsloth-sft/#serving-the-model-with-vllm-and-proxy "Permanent link") Most modern agents do not use the model directly; instead, they use an API like the OpenAI chat completions API to interact with the model. Therefore, we need a vLLM-based inference server launched before rollouts. The serving code looks like the following. See the `vllm_server` function in [sft\_algorithm.py](https://github.com/microsoft/agent-lightning/tree/c746af2f76bebcae56007a59c223cdace411c89b/examples/unsloth/sft_algorithm.py) if you want to see a more robust version. `[](https://microsoft.github.io/agent-lightning/latest/how-to/unsloth-sft/#__codelineno-1-1) from openai import OpenAI [](https://microsoft.github.io/agent-lightning/latest/how-to/unsloth-sft/#__codelineno-1-2) [](https://microsoft.github.io/agent-lightning/latest/how-to/unsloth-sft/#__codelineno-1-3) vllm_process = subprocess.Popen([ [](https://microsoft.github.io/agent-lightning/latest/how-to/unsloth-sft/#__codelineno-1-4) "vllm", "serve", model_path, "--port", str(port), [](https://microsoft.github.io/agent-lightning/latest/how-to/unsloth-sft/#__codelineno-1-5) "--enable-auto-tool-choice", "--tool-call-parser", "hermes" [](https://microsoft.github.io/agent-lightning/latest/how-to/unsloth-sft/#__codelineno-1-6) ]) [](https://microsoft.github.io/agent-lightning/latest/how-to/unsloth-sft/#__codelineno-1-7) [](https://microsoft.github.io/agent-lightning/latest/how-to/unsloth-sft/#__codelineno-1-8) # Wait for the server to be ready [](https://microsoft.github.io/agent-lightning/latest/how-to/unsloth-sft/#__codelineno-1-9) url = f"http://localhost:{port}/health" [](https://microsoft.github.io/agent-lightning/latest/how-to/unsloth-sft/#__codelineno-1-10) start = time.time() [](https://microsoft.github.io/agent-lightning/latest/how-to/unsloth-sft/#__codelineno-1-11) client = httpx.Client() [](https://microsoft.github.io/agent-lightning/latest/how-to/unsloth-sft/#__codelineno-1-12) [](https://microsoft.github.io/agent-lightning/latest/how-to/unsloth-sft/#__codelineno-1-13) while True: [](https://microsoft.github.io/agent-lightning/latest/how-to/unsloth-sft/#__codelineno-1-14) if client.get(url).status_code == 200: [](https://microsoft.github.io/agent-lightning/latest/how-to/unsloth-sft/#__codelineno-1-15) break [](https://microsoft.github.io/agent-lightning/latest/how-to/unsloth-sft/#__codelineno-1-16) [](https://microsoft.github.io/agent-lightning/latest/how-to/unsloth-sft/#__codelineno-1-17) server_address = f"http://localhost:{port}/v1" [](https://microsoft.github.io/agent-lightning/latest/how-to/unsloth-sft/#__codelineno-1-18) [](https://microsoft.github.io/agent-lightning/latest/how-to/unsloth-sft/#__codelineno-1-19) # Try using the vLLM server [](https://microsoft.github.io/agent-lightning/latest/how-to/unsloth-sft/#__codelineno-1-20) openai = OpenAI(base_url=server_address) [](https://microsoft.github.io/agent-lightning/latest/how-to/unsloth-sft/#__codelineno-1-21) ...` In this recipe, we do not expose the server address directly to the agent runners, because we want to install a "middleware" to collect the prompts and responses of all the requests. In general, it's up to you to decide whether to hide the vLLM server behind a proxy or not. The "middleware" here is [`LLMProxy`](https://microsoft.github.io/agent-lightning/latest/reference/algorithm/#agentlightning.LLMProxy " agentlightning.LLMProxy") , which is an independent [LiteLLM](https://docs.litellm.ai/) server that forwards the requests to the vLLM server. It also exposes an OpenAI-compatible API that the runners can target without caring about where the model lives. The benefits of using the proxy are: 1. **Traces:** The proxy automatically logs the prompts and responses of all the requests into the store. 2. **Token IDs:** The proxy augments the requests so that the vLLM server can return the prompt and response token IDs (see more details in [Serving LLM](https://microsoft.github.io/agent-lightning/latest/deep-dive/serving-llm/) ). The [`LLMProxy`](https://microsoft.github.io/agent-lightning/latest/reference/algorithm/#agentlightning.LLMProxy " agentlightning.LLMProxy") accepts a list of model configurations, in the same syntax as LiteLLM's [`model_list`](https://docs.litellm.ai/docs/proxy/configs) . Include a `hosted_vllm/` prefix to the models to activate LiteLLM's [vLLM integration](https://docs.litellm.ai/docs/providers/vllm) . `[](https://microsoft.github.io/agent-lightning/latest/how-to/unsloth-sft/#__codelineno-2-1) import agentlightning as agl [](https://microsoft.github.io/agent-lightning/latest/how-to/unsloth-sft/#__codelineno-2-2) [](https://microsoft.github.io/agent-lightning/latest/how-to/unsloth-sft/#__codelineno-2-3) llm_proxy = agl.LLMProxy(port=port, store=store) [](https://microsoft.github.io/agent-lightning/latest/how-to/unsloth-sft/#__codelineno-2-4) model_list = [ [](https://microsoft.github.io/agent-lightning/latest/how-to/unsloth-sft/#__codelineno-2-5) { [](https://microsoft.github.io/agent-lightning/latest/how-to/unsloth-sft/#__codelineno-2-6) "model_name": "Qwen3-4B-Instruct", [](https://microsoft.github.io/agent-lightning/latest/how-to/unsloth-sft/#__codelineno-2-7) "litellm_params": {"model": f"hosted_vllm/{model_path}", "api_base": server_address}, [](https://microsoft.github.io/agent-lightning/latest/how-to/unsloth-sft/#__codelineno-2-8) } [](https://microsoft.github.io/agent-lightning/latest/how-to/unsloth-sft/#__codelineno-2-9) ] [](https://microsoft.github.io/agent-lightning/latest/how-to/unsloth-sft/#__codelineno-2-10) llm_proxy.update_model_list(model_list) [](https://microsoft.github.io/agent-lightning/latest/how-to/unsloth-sft/#__codelineno-2-11) # If the proxy is not running, it will start automatically. [](https://microsoft.github.io/agent-lightning/latest/how-to/unsloth-sft/#__codelineno-2-12) await llm_proxy.restart() [](https://microsoft.github.io/agent-lightning/latest/how-to/unsloth-sft/#__codelineno-2-13) # Add the proxy as a resource to the store so that the runners can access it via URL. [](https://microsoft.github.io/agent-lightning/latest/how-to/unsloth-sft/#__codelineno-2-14) resource_update = await store.add_resources({"main_llm": llm_proxy.as_resource()})` ### Spawn Rollout and Collect Spans[¶](https://microsoft.github.io/agent-lightning/latest/how-to/unsloth-sft/#spawn-rollout-and-collect-spans "Permanent link") Once the proxy is registered as a resource, the algorithm schedules work for the rollout runners. Each problem from a training dataset becomes a rollout with the proxy baked into its resources: `[](https://microsoft.github.io/agent-lightning/latest/how-to/unsloth-sft/#__codelineno-3-1) rollouts: list[Rollout] = [] [](https://microsoft.github.io/agent-lightning/latest/how-to/unsloth-sft/#__codelineno-3-2) for sample in train_dataset: [](https://microsoft.github.io/agent-lightning/latest/how-to/unsloth-sft/#__codelineno-3-3) rollouts.append( [](https://microsoft.github.io/agent-lightning/latest/how-to/unsloth-sft/#__codelineno-3-4) await store.enqueue_rollout( [](https://microsoft.github.io/agent-lightning/latest/how-to/unsloth-sft/#__codelineno-3-5) input=sample, [](https://microsoft.github.io/agent-lightning/latest/how-to/unsloth-sft/#__codelineno-3-6) mode="train", [](https://microsoft.github.io/agent-lightning/latest/how-to/unsloth-sft/#__codelineno-3-7) resources_id=resources_update.resources_id, [](https://microsoft.github.io/agent-lightning/latest/how-to/unsloth-sft/#__codelineno-3-8) ) [](https://microsoft.github.io/agent-lightning/latest/how-to/unsloth-sft/#__codelineno-3-9) )` `resources_id` ties every rollout to the `main_llm` proxy resource we just uploaded. The runners on the other side poll the store ([`LitAgentRunner.iter()`](https://microsoft.github.io/agent-lightning/latest/reference/runner/#agentlightning.LitAgentRunner.iter " iter(*, event=None) async ") ) and execute the agent for each rollout. On the algorithm side we wait for completions with a non-blocking polling loop: `[](https://microsoft.github.io/agent-lightning/latest/how-to/unsloth-sft/#__codelineno-4-1) completed_rollouts: list[Rollout] = [] [](https://microsoft.github.io/agent-lightning/latest/how-to/unsloth-sft/#__codelineno-4-2) while True: [](https://microsoft.github.io/agent-lightning/latest/how-to/unsloth-sft/#__codelineno-4-3) completed_rollouts = await store.wait_for_rollouts( [](https://microsoft.github.io/agent-lightning/latest/how-to/unsloth-sft/#__codelineno-4-4) rollout_ids=[r.rollout_id for r in rollouts], [](https://microsoft.github.io/agent-lightning/latest/how-to/unsloth-sft/#__codelineno-4-5) timeout=0.0, [](https://microsoft.github.io/agent-lightning/latest/how-to/unsloth-sft/#__codelineno-4-6) ) [](https://microsoft.github.io/agent-lightning/latest/how-to/unsloth-sft/#__codelineno-4-7) if len(completed_rollouts) == len(rollouts): [](https://microsoft.github.io/agent-lightning/latest/how-to/unsloth-sft/#__codelineno-4-8) break [](https://microsoft.github.io/agent-lightning/latest/how-to/unsloth-sft/#__codelineno-4-9) await asyncio.sleep(5.0)` Note The `timeout=0.0` is needed here because this example uses a [`LightningStoreClient`](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.LightningStoreClient " agentlightning.LightningStoreClient") , and `wait_for_rollouts` establishes an HTTP connection to that store. Currently, only non-blocking wait requests are supported, which avoids holding the store connection open. Once the rollouts complete, we terminate the vLLM server to free up GPU memory. `[](https://microsoft.github.io/agent-lightning/latest/how-to/unsloth-sft/#__codelineno-5-1) vllm_process.terminate() [](https://microsoft.github.io/agent-lightning/latest/how-to/unsloth-sft/#__codelineno-5-2) vllm_process.join(timeout=10.0)` ### Adapt the Spans to HuggingFace Dataset[¶](https://microsoft.github.io/agent-lightning/latest/how-to/unsloth-sft/#adapt-the-spans-to-huggingface-dataset "Permanent link") [`LlmProxyTraceToTriplet`](https://microsoft.github.io/agent-lightning/latest/reference/algorithm/#agentlightning.LlmProxyTraceToTriplet " agentlightning.LlmProxyTraceToTriplet") converts the proxy’s spans (which might be dozens to hundreds per rollout) into [`Triplet`](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.Triplet " agentlightning.Triplet") objects that contain prompt/response token IDs plus an optional reward. The adapter may return multiple triplets per rollout (one per chat-completion call). To bias training toward successful reasoning chains the algorithm walks the triplets in reverse order, keeps the most recent reward, and turns each prompt/response pair into Hugging Face dataset rows: `[](https://microsoft.github.io/agent-lightning/latest/how-to/unsloth-sft/#__codelineno-6-1) all_triplets = [] [](https://microsoft.github.io/agent-lightning/latest/how-to/unsloth-sft/#__codelineno-6-2) data_adapter = agl.LlmProxyTraceToTriplet() [](https://microsoft.github.io/agent-lightning/latest/how-to/unsloth-sft/#__codelineno-6-3) [](https://microsoft.github.io/agent-lightning/latest/how-to/unsloth-sft/#__codelineno-6-4) for rollout in completed_rollouts: [](https://microsoft.github.io/agent-lightning/latest/how-to/unsloth-sft/#__codelineno-6-5) spans = await store.query_spans(rollout.rollout_id, "latest") [](https://microsoft.github.io/agent-lightning/latest/how-to/unsloth-sft/#__codelineno-6-6) triplets = data_adapter.adapt(spans) [](https://microsoft.github.io/agent-lightning/latest/how-to/unsloth-sft/#__codelineno-6-7) [](https://microsoft.github.io/agent-lightning/latest/how-to/unsloth-sft/#__codelineno-6-8) recent_reward = None [](https://microsoft.github.io/agent-lightning/latest/how-to/unsloth-sft/#__codelineno-6-9) for triplet in reversed(triplets): [](https://microsoft.github.io/agent-lightning/latest/how-to/unsloth-sft/#__codelineno-6-10) if triplet.reward is not None: [](https://microsoft.github.io/agent-lightning/latest/how-to/unsloth-sft/#__codelineno-6-11) recent_reward = triplet.reward [](https://microsoft.github.io/agent-lightning/latest/how-to/unsloth-sft/#__codelineno-6-12) if recent_reward is None: [](https://microsoft.github.io/agent-lightning/latest/how-to/unsloth-sft/#__codelineno-6-13) continue [](https://microsoft.github.io/agent-lightning/latest/how-to/unsloth-sft/#__codelineno-6-14) [](https://microsoft.github.io/agent-lightning/latest/how-to/unsloth-sft/#__codelineno-6-15) input_ids = triplet.prompt["token_ids"] + triplet.response["token_ids"] [](https://microsoft.github.io/agent-lightning/latest/how-to/unsloth-sft/#__codelineno-6-16) # We don't train on prompt tokens, so they are masked out by setting to -100. [](https://microsoft.github.io/agent-lightning/latest/how-to/unsloth-sft/#__codelineno-6-17) labels = [-100] * len(triplet.prompt["token_ids"]) + triplet.response["token_ids"] [](https://microsoft.github.io/agent-lightning/latest/how-to/unsloth-sft/#__codelineno-6-18) # This matches the dataset format required by the Unsloth SFT trainer. [](https://microsoft.github.io/agent-lightning/latest/how-to/unsloth-sft/#__codelineno-6-19) all_triplets.append( [](https://microsoft.github.io/agent-lightning/latest/how-to/unsloth-sft/#__codelineno-6-20) { [](https://microsoft.github.io/agent-lightning/latest/how-to/unsloth-sft/#__codelineno-6-21) "input_ids": input_ids, [](https://microsoft.github.io/agent-lightning/latest/how-to/unsloth-sft/#__codelineno-6-22) "attention_mask": [1] * len(input_ids), [](https://microsoft.github.io/agent-lightning/latest/how-to/unsloth-sft/#__codelineno-6-23) "labels": labels, [](https://microsoft.github.io/agent-lightning/latest/how-to/unsloth-sft/#__codelineno-6-24) "reward": recent_reward, [](https://microsoft.github.io/agent-lightning/latest/how-to/unsloth-sft/#__codelineno-6-25) } [](https://microsoft.github.io/agent-lightning/latest/how-to/unsloth-sft/#__codelineno-6-26) )` Note You might notice that the dataset format used here differs from the format described in the **SFT Trainer** documentation. According to the documentation, dataset samples should be provided as plain text strings or message objects. As a matter of fact, this example leverages some [undocumented behavior](https://github.com/huggingface/trl/blob/e0eec055b412c48ad754149c475a87a8fca34fb4/trl/trainer/sft_trainer.py#L887) in the SFT Trainer implementation. When the dataset already includes a `"input_ids"` column, the Trainer automatically marks it as `is_processed` and skips the internal tokenization step. Since we already have spans with token IDs generated by the [`LLMProxy`](https://microsoft.github.io/agent-lightning/latest/reference/algorithm/#agentlightning.LLMProxy " agentlightning.LLMProxy") , providing them directly avoids unnecessary [**re-tokenization** and related complications](https://microsoft.github.io/agent-lightning/latest/deep-dive/serving-llm/) . This approach will both save processing time and increase consistency between training and inference. After aggregating every rollout we shuffle, sort by reward, and keep the top fraction (e.g., 50%) before shuffling again. The resulting list feeds directly into `datasets.Dataset.from_list`, which is the format Unsloth’s SFT trainer expects. `[](https://microsoft.github.io/agent-lightning/latest/how-to/unsloth-sft/#__codelineno-7-1) from datasets import Dataset as HuggingFaceDataset [](https://microsoft.github.io/agent-lightning/latest/how-to/unsloth-sft/#__codelineno-7-2) [](https://microsoft.github.io/agent-lightning/latest/how-to/unsloth-sft/#__codelineno-7-3) random.shuffle(all_triplets) [](https://microsoft.github.io/agent-lightning/latest/how-to/unsloth-sft/#__codelineno-7-4) all_triplets.sort(key=lambda x: x["reward"], reverse=True) [](https://microsoft.github.io/agent-lightning/latest/how-to/unsloth-sft/#__codelineno-7-5) sliced_triplets = all_triplets[: max(1, int(len(all_triplets) * triplet_fraction))] [](https://microsoft.github.io/agent-lightning/latest/how-to/unsloth-sft/#__codelineno-7-6) # Shuffle the sliced triplets again [](https://microsoft.github.io/agent-lightning/latest/how-to/unsloth-sft/#__codelineno-7-7) random.shuffle(sliced_triplets) [](https://microsoft.github.io/agent-lightning/latest/how-to/unsloth-sft/#__codelineno-7-8) [](https://microsoft.github.io/agent-lightning/latest/how-to/unsloth-sft/#__codelineno-7-9) sft_dataset = HuggingFaceDataset.from_list(sliced_triplets)` ### Launch Unsloth Training[¶](https://microsoft.github.io/agent-lightning/latest/how-to/unsloth-sft/#launch-unsloth-training "Permanent link") The heavy lifting happens in [`trl.SFTTrainer`](https://huggingface.co/docs/trl/sft_trainer) (see [unsloth\_helper.py](https://github.com/microsoft/agent-lightning/tree/c746af2f76bebcae56007a59c223cdace411c89b/examples/unsloth/unsloth_helper.py) on how it's used). We launch it in a fresh process created with `multiprocessing.get_context("spawn")` so CUDA memory is reliably reclaimed when training ends. Launching it in the same process will also work for the first iteration, but we found that the memory won't be freed properly for subsequent vLLM serving. `[](https://microsoft.github.io/agent-lightning/latest/how-to/unsloth-sft/#__codelineno-8-1) context = multiprocessing.get_context("spawn") [](https://microsoft.github.io/agent-lightning/latest/how-to/unsloth-sft/#__codelineno-8-2) unsloth_process = context.Process( [](https://microsoft.github.io/agent-lightning/latest/how-to/unsloth-sft/#__codelineno-8-3) target=unsloth_training, [](https://microsoft.github.io/agent-lightning/latest/how-to/unsloth-sft/#__codelineno-8-4) args=(model_path, sft_dataset, next_model_path), [](https://microsoft.github.io/agent-lightning/latest/how-to/unsloth-sft/#__codelineno-8-5) daemon=True, [](https://microsoft.github.io/agent-lightning/latest/how-to/unsloth-sft/#__codelineno-8-6) ) [](https://microsoft.github.io/agent-lightning/latest/how-to/unsloth-sft/#__codelineno-8-7) unsloth_process.start() [](https://microsoft.github.io/agent-lightning/latest/how-to/unsloth-sft/#__codelineno-8-8) unsloth_process.join(timeout=600.0)` Inside the `unsloth_training` subprocess, Unsloth loads the previous checkpoint in 4-bit, applies LoRA adapters, and forwards the Hugging Face dataset to [`trl.SFTTrainer`](https://huggingface.co/docs/trl/sft_trainer) with the configuration defined in [`SFTConfig`](https://huggingface.co/docs/trl/sft_trainer#trl.SFTConfig) (batch size, accumulation steps, learning rate, etc.). The merged 16-bit weights are saved under `models/version_` so the next iteration can immediately serve them with vLLM. `[](https://microsoft.github.io/agent-lightning/latest/how-to/unsloth-sft/#__codelineno-9-1) from unsloth import FastLanguageModel [](https://microsoft.github.io/agent-lightning/latest/how-to/unsloth-sft/#__codelineno-9-2) # TRL is patched by unsloth. [](https://microsoft.github.io/agent-lightning/latest/how-to/unsloth-sft/#__codelineno-9-3) from trl import SFTConfig, SFTTrainer [](https://microsoft.github.io/agent-lightning/latest/how-to/unsloth-sft/#__codelineno-9-4) [](https://microsoft.github.io/agent-lightning/latest/how-to/unsloth-sft/#__codelineno-9-5) model, tokenizer = FastLanguageModel.from_pretrained( [](https://microsoft.github.io/agent-lightning/latest/how-to/unsloth-sft/#__codelineno-9-6) model_name=model_path, [](https://microsoft.github.io/agent-lightning/latest/how-to/unsloth-sft/#__codelineno-9-7) load_in_4bit=True, # 4 bit quantization to reduce memory [](https://microsoft.github.io/agent-lightning/latest/how-to/unsloth-sft/#__codelineno-9-8) ) [](https://microsoft.github.io/agent-lightning/latest/how-to/unsloth-sft/#__codelineno-9-9) [](https://microsoft.github.io/agent-lightning/latest/how-to/unsloth-sft/#__codelineno-9-10) # Config the model to use LoRA [](https://microsoft.github.io/agent-lightning/latest/how-to/unsloth-sft/#__codelineno-9-11) model = FastLanguageModel.get_peft_model( [](https://microsoft.github.io/agent-lightning/latest/how-to/unsloth-sft/#__codelineno-9-12) model, [](https://microsoft.github.io/agent-lightning/latest/how-to/unsloth-sft/#__codelineno-9-13) r=32, [](https://microsoft.github.io/agent-lightning/latest/how-to/unsloth-sft/#__codelineno-9-14) ... [](https://microsoft.github.io/agent-lightning/latest/how-to/unsloth-sft/#__codelineno-9-15) ) [](https://microsoft.github.io/agent-lightning/latest/how-to/unsloth-sft/#__codelineno-9-16) [](https://microsoft.github.io/agent-lightning/latest/how-to/unsloth-sft/#__codelineno-9-17) trainer = SFTTrainer( [](https://microsoft.github.io/agent-lightning/latest/how-to/unsloth-sft/#__codelineno-9-18) model=model, [](https://microsoft.github.io/agent-lightning/latest/how-to/unsloth-sft/#__codelineno-9-19) tokenizer=tokenizer, [](https://microsoft.github.io/agent-lightning/latest/how-to/unsloth-sft/#__codelineno-9-20) train_dataset=sft_dataset, [](https://microsoft.github.io/agent-lightning/latest/how-to/unsloth-sft/#__codelineno-9-21) ... [](https://microsoft.github.io/agent-lightning/latest/how-to/unsloth-sft/#__codelineno-9-22) ) [](https://microsoft.github.io/agent-lightning/latest/how-to/unsloth-sft/#__codelineno-9-23) [](https://microsoft.github.io/agent-lightning/latest/how-to/unsloth-sft/#__codelineno-9-24) # This is the heaviest step. [](https://microsoft.github.io/agent-lightning/latest/how-to/unsloth-sft/#__codelineno-9-25) trainer_stats = trainer.train() [](https://microsoft.github.io/agent-lightning/latest/how-to/unsloth-sft/#__codelineno-9-26) [](https://microsoft.github.io/agent-lightning/latest/how-to/unsloth-sft/#__codelineno-9-27) # Save in 16-bit for vLLM inference later [](https://microsoft.github.io/agent-lightning/latest/how-to/unsloth-sft/#__codelineno-9-28) model.save_pretrained_merged(next_model_path, tokenizer, save_method="merged_16bit")` Math Agent: OpenAI Agents SDK with MCP[¶](https://microsoft.github.io/agent-lightning/latest/how-to/unsloth-sft/#math-agent-openai-agents-sdk-with-mcp "Permanent link") ------------------------------------------------------------------------------------------------------------------------------------------------------------------------- We build an agent with the [OpenAI Agents SDK](https://openai.github.io/openai-agents-python/) to wire a calculator MCP tool and an OpenAI-compatible chat completion model together. The agent aims to solve a math problem and returns a reward indicating whether the answer is correct or not. The runner injects the `LLM` resource supplied by the algorithm side: `[](https://microsoft.github.io/agent-lightning/latest/how-to/unsloth-sft/#__codelineno-10-1) import os [](https://microsoft.github.io/agent-lightning/latest/how-to/unsloth-sft/#__codelineno-10-2) from typing import TypedDict [](https://microsoft.github.io/agent-lightning/latest/how-to/unsloth-sft/#__codelineno-10-3) [](https://microsoft.github.io/agent-lightning/latest/how-to/unsloth-sft/#__codelineno-10-4) import agentlightning as agl [](https://microsoft.github.io/agent-lightning/latest/how-to/unsloth-sft/#__codelineno-10-5) from agents import Agent, ModelSettings, OpenAIChatCompletionsModel, Runner as OpenAIRunner [](https://microsoft.github.io/agent-lightning/latest/how-to/unsloth-sft/#__codelineno-10-6) from agents.mcp import MCPServerStdio [](https://microsoft.github.io/agent-lightning/latest/how-to/unsloth-sft/#__codelineno-10-7) from openai import AsyncOpenAI [](https://microsoft.github.io/agent-lightning/latest/how-to/unsloth-sft/#__codelineno-10-8) [](https://microsoft.github.io/agent-lightning/latest/how-to/unsloth-sft/#__codelineno-10-9) class GsmProblem(TypedDict): [](https://microsoft.github.io/agent-lightning/latest/how-to/unsloth-sft/#__codelineno-10-10) input: str [](https://microsoft.github.io/agent-lightning/latest/how-to/unsloth-sft/#__codelineno-10-11) target: float [](https://microsoft.github.io/agent-lightning/latest/how-to/unsloth-sft/#__codelineno-10-12) [](https://microsoft.github.io/agent-lightning/latest/how-to/unsloth-sft/#__codelineno-10-13) def compute_reward(result: str, target: float) -> float: [](https://microsoft.github.io/agent-lightning/latest/how-to/unsloth-sft/#__codelineno-10-14) ... [](https://microsoft.github.io/agent-lightning/latest/how-to/unsloth-sft/#__codelineno-10-15) [](https://microsoft.github.io/agent-lightning/latest/how-to/unsloth-sft/#__codelineno-10-16) @agl.rollout [](https://microsoft.github.io/agent-lightning/latest/how-to/unsloth-sft/#__codelineno-10-17) async def math_agent(task: GsmProblem, llm: agl.LLM) -> float: [](https://microsoft.github.io/agent-lightning/latest/how-to/unsloth-sft/#__codelineno-10-18) async with MCPServerStdio( [](https://microsoft.github.io/agent-lightning/latest/how-to/unsloth-sft/#__codelineno-10-19) name="Calculator via uvx", [](https://microsoft.github.io/agent-lightning/latest/how-to/unsloth-sft/#__codelineno-10-20) params={"command": "uvx", "args": ["mcp-server-calculator"]}, [](https://microsoft.github.io/agent-lightning/latest/how-to/unsloth-sft/#__codelineno-10-21) ) as server: [](https://microsoft.github.io/agent-lightning/latest/how-to/unsloth-sft/#__codelineno-10-22) agent = Agent( [](https://microsoft.github.io/agent-lightning/latest/how-to/unsloth-sft/#__codelineno-10-23) name="Assistant", [](https://microsoft.github.io/agent-lightning/latest/how-to/unsloth-sft/#__codelineno-10-24) instructions=( [](https://microsoft.github.io/agent-lightning/latest/how-to/unsloth-sft/#__codelineno-10-25) "Use the calculator tool for every question. " [](https://microsoft.github.io/agent-lightning/latest/how-to/unsloth-sft/#__codelineno-10-26) "Return only the numeric answer wrapped like ### ###." [](https://microsoft.github.io/agent-lightning/latest/how-to/unsloth-sft/#__codelineno-10-27) ), [](https://microsoft.github.io/agent-lightning/latest/how-to/unsloth-sft/#__codelineno-10-28) mcp_servers=[server], [](https://microsoft.github.io/agent-lightning/latest/how-to/unsloth-sft/#__codelineno-10-29) model=OpenAIChatCompletionsModel( [](https://microsoft.github.io/agent-lightning/latest/how-to/unsloth-sft/#__codelineno-10-30) model=llm.model, [](https://microsoft.github.io/agent-lightning/latest/how-to/unsloth-sft/#__codelineno-10-31) openai_client=AsyncOpenAI( [](https://microsoft.github.io/agent-lightning/latest/how-to/unsloth-sft/#__codelineno-10-32) base_url=llm.endpoint, [](https://microsoft.github.io/agent-lightning/latest/how-to/unsloth-sft/#__codelineno-10-33) api_key=llm.api_key or "dummy", [](https://microsoft.github.io/agent-lightning/latest/how-to/unsloth-sft/#__codelineno-10-34) ), [](https://microsoft.github.io/agent-lightning/latest/how-to/unsloth-sft/#__codelineno-10-35) ), [](https://microsoft.github.io/agent-lightning/latest/how-to/unsloth-sft/#__codelineno-10-36) model_settings=ModelSettings( [](https://microsoft.github.io/agent-lightning/latest/how-to/unsloth-sft/#__codelineno-10-37) temperature=llm.sampling_parameters.get("temperature", 0.0), [](https://microsoft.github.io/agent-lightning/latest/how-to/unsloth-sft/#__codelineno-10-38) ), [](https://microsoft.github.io/agent-lightning/latest/how-to/unsloth-sft/#__codelineno-10-39) ) [](https://microsoft.github.io/agent-lightning/latest/how-to/unsloth-sft/#__codelineno-10-40) result = await OpenAIRunner.run(agent, task["input"]) [](https://microsoft.github.io/agent-lightning/latest/how-to/unsloth-sft/#__codelineno-10-41) return compute_reward(result.final_output, task["target"])` Tip You can test the agent with a dry run: `[](https://microsoft.github.io/agent-lightning/latest/how-to/unsloth-sft/#__codelineno-11-1) import asyncio [](https://microsoft.github.io/agent-lightning/latest/how-to/unsloth-sft/#__codelineno-11-2) [](https://microsoft.github.io/agent-lightning/latest/how-to/unsloth-sft/#__codelineno-11-3) llm = agl.LLM( [](https://microsoft.github.io/agent-lightning/latest/how-to/unsloth-sft/#__codelineno-11-4) endpoint=os.environ["OPENAI_BASE_URL"], [](https://microsoft.github.io/agent-lightning/latest/how-to/unsloth-sft/#__codelineno-11-5) api_key=os.environ["OPENAI_API_KEY"], [](https://microsoft.github.io/agent-lightning/latest/how-to/unsloth-sft/#__codelineno-11-6) model="gpt-4.1-mini", [](https://microsoft.github.io/agent-lightning/latest/how-to/unsloth-sft/#__codelineno-11-7) ) [](https://microsoft.github.io/agent-lightning/latest/how-to/unsloth-sft/#__codelineno-11-8) asyncio.run(math_agent({"input": "What is 1 + 1?", "target": 2.0}, llm))` Run this Recipe[¶](https://microsoft.github.io/agent-lightning/latest/how-to/unsloth-sft/#run-this-recipe "Permanent link") ---------------------------------------------------------------------------------------------------------------------------- The full runnable script for this recipe resides in [`examples/unsloth`](https://github.com/microsoft/agent-lightning/tree/c746af2f76bebcae56007a59c223cdace411c89b/examples/unsloth) folder. Before running this example, install `unsloth`, `vllm`, and the other libraries used in the examples (the project uses CUDA tooling, TRL, rich, datasets, etc.). We tested with `unsloth==2025.10.1`. `unsloth==2025.10.2` and `2025.10.3` are not working because of an [issue](https://github.com/unslothai/unsloth/issues/3451) we have been investigating with the unsloth team. It's recommended to download the base model before running the example, such that the first iteration and subsequent iterations can both load from local checkpoints. `[](https://microsoft.github.io/agent-lightning/latest/how-to/unsloth-sft/#__codelineno-12-1) hf download unsloth/Qwen3-4B-Instruct-2507 --local-dir models/version_0` The repository already contains `examples/unsloth/data_gsmhard.jsonl` (which is a very small subset of the [GSM-hard math dataset](https://huggingface.co/datasets/reasoning-machines/gsm-hard) for demonstration purposes). ### Run Manually[¶](https://microsoft.github.io/agent-lightning/latest/how-to/unsloth-sft/#run-manually "Permanent link") Similar to the [Write the First Algorithm](https://microsoft.github.io/agent-lightning/latest/how-to/write-first-algorithm/) recipe, you can open three terminals and start each component in parallel. `[](https://microsoft.github.io/agent-lightning/latest/how-to/unsloth-sft/#__codelineno-13-1) agl store --port 4747 [](https://microsoft.github.io/agent-lightning/latest/how-to/unsloth-sft/#__codelineno-13-2) python examples/unsloth/sft_rollout_runners.py [](https://microsoft.github.io/agent-lightning/latest/how-to/unsloth-sft/#__codelineno-13-3) python examples/unsloth/sft_algorithm.py` In this case, [`sft_rollout_runners.py`](https://github.com/microsoft/agent-lightning/tree/c746af2f76bebcae56007a59c223cdace411c89b/examples/unsloth/sft_rollout_runners.py) is a simple spawner implemented in Python that spawns 4 runners in parallel. The runners all connect to the same store server executing in another terminal. `[](https://microsoft.github.io/agent-lightning/latest/how-to/unsloth-sft/#__codelineno-14-1) import agentlightning as agl [](https://microsoft.github.io/agent-lightning/latest/how-to/unsloth-sft/#__codelineno-14-2) [](https://microsoft.github.io/agent-lightning/latest/how-to/unsloth-sft/#__codelineno-14-3) def run_rollout(store: agl.LightningStore, worker_id: int) -> None: [](https://microsoft.github.io/agent-lightning/latest/how-to/unsloth-sft/#__codelineno-14-4) # Since the server side has already used LiteLLM proxy to collect traces, [](https://microsoft.github.io/agent-lightning/latest/how-to/unsloth-sft/#__codelineno-14-5) # a simple OtelTracer to collect the rewards is enough. [](https://microsoft.github.io/agent-lightning/latest/how-to/unsloth-sft/#__codelineno-14-6) tracer = agl.OtelTracer() [](https://microsoft.github.io/agent-lightning/latest/how-to/unsloth-sft/#__codelineno-14-7) [](https://microsoft.github.io/agent-lightning/latest/how-to/unsloth-sft/#__codelineno-14-8) runner = agl.LitAgentRunner(tracer=tracer) [](https://microsoft.github.io/agent-lightning/latest/how-to/unsloth-sft/#__codelineno-14-9) [](https://microsoft.github.io/agent-lightning/latest/how-to/unsloth-sft/#__codelineno-14-10) with runner.run_context(agent=math_agent, store=store, worker_id=worker_id): [](https://microsoft.github.io/agent-lightning/latest/how-to/unsloth-sft/#__codelineno-14-11) asyncio.run(runner.iter()) [](https://microsoft.github.io/agent-lightning/latest/how-to/unsloth-sft/#__codelineno-14-12) [](https://microsoft.github.io/agent-lightning/latest/how-to/unsloth-sft/#__codelineno-14-13) [](https://microsoft.github.io/agent-lightning/latest/how-to/unsloth-sft/#__codelineno-14-14) def spawn_runners(store: agl.LightningStore, n_runners: int) -> None: [](https://microsoft.github.io/agent-lightning/latest/how-to/unsloth-sft/#__codelineno-14-15) runners = [ [](https://microsoft.github.io/agent-lightning/latest/how-to/unsloth-sft/#__codelineno-14-16) multiprocessing.Process(target=run_rollout, args=(store, worker_id)) [](https://microsoft.github.io/agent-lightning/latest/how-to/unsloth-sft/#__codelineno-14-17) for worker_id in range(n_runners) [](https://microsoft.github.io/agent-lightning/latest/how-to/unsloth-sft/#__codelineno-14-18) ] [](https://microsoft.github.io/agent-lightning/latest/how-to/unsloth-sft/#__codelineno-14-19) for runner in runners: [](https://microsoft.github.io/agent-lightning/latest/how-to/unsloth-sft/#__codelineno-14-20) runner.start() [](https://microsoft.github.io/agent-lightning/latest/how-to/unsloth-sft/#__codelineno-14-21) [](https://microsoft.github.io/agent-lightning/latest/how-to/unsloth-sft/#__codelineno-14-22) for runner in runners: [](https://microsoft.github.io/agent-lightning/latest/how-to/unsloth-sft/#__codelineno-14-23) runner.join() [](https://microsoft.github.io/agent-lightning/latest/how-to/unsloth-sft/#__codelineno-14-24) [](https://microsoft.github.io/agent-lightning/latest/how-to/unsloth-sft/#__codelineno-14-25) [](https://microsoft.github.io/agent-lightning/latest/how-to/unsloth-sft/#__codelineno-14-26) store = agl.LightningStoreClient("http://localhost:4747") [](https://microsoft.github.io/agent-lightning/latest/how-to/unsloth-sft/#__codelineno-14-27) spawn_runners(store=store, n_runners=4)` Tip Try to swap [`OtelTracer`](https://microsoft.github.io/agent-lightning/latest/reference/runner/#agentlightning.OtelTracer " agentlightning.OtelTracer") in the runners with other tracers like [`AgentOpsTracer`](https://microsoft.github.io/agent-lightning/latest/reference/runner/#agentlightning.AgentOpsTracer " agentlightning.AgentOpsTracer") . Try to use a different adapter at the algorithm side such as [`TracerTraceToTriplet`](https://microsoft.github.io/agent-lightning/latest/reference/algorithm/#agentlightning.TracerTraceToTriplet " agentlightning.TracerTraceToTriplet") to see what happens. ### Run Everything with Trainer[¶](https://microsoft.github.io/agent-lightning/latest/how-to/unsloth-sft/#run-everything-with-trainer "Permanent link") We also show how to wrap everything into a single script using [`Trainer`](https://microsoft.github.io/agent-lightning/latest/reference/trainer/#agentlightning.Trainer " agentlightning.Trainer") . [`sft_allinone.py`](https://github.com/microsoft/agent-lightning/tree/c746af2f76bebcae56007a59c223cdace411c89b/examples/unsloth/sft_allinone.py) wires the same components together, replacing the manual management of runners above. `[](https://microsoft.github.io/agent-lightning/latest/how-to/unsloth-sft/#__codelineno-15-1) class UnslothSupervisedFinetuning(agl.Algorithm): [](https://microsoft.github.io/agent-lightning/latest/how-to/unsloth-sft/#__codelineno-15-2) [](https://microsoft.github.io/agent-lightning/latest/how-to/unsloth-sft/#__codelineno-15-3) async def run( [](https://microsoft.github.io/agent-lightning/latest/how-to/unsloth-sft/#__codelineno-15-4) self, [](https://microsoft.github.io/agent-lightning/latest/how-to/unsloth-sft/#__codelineno-15-5) train_dataset: Optional[Dataset[GsmProblem]] = None, [](https://microsoft.github.io/agent-lightning/latest/how-to/unsloth-sft/#__codelineno-15-6) val_dataset: Optional[Dataset[GsmProblem]] = None, [](https://microsoft.github.io/agent-lightning/latest/how-to/unsloth-sft/#__codelineno-15-7) ): [](https://microsoft.github.io/agent-lightning/latest/how-to/unsloth-sft/#__codelineno-15-8) # Use the store, llm_proxy, and adapter from the trainer [](https://microsoft.github.io/agent-lightning/latest/how-to/unsloth-sft/#__codelineno-15-9) store = self.get_store() [](https://microsoft.github.io/agent-lightning/latest/how-to/unsloth-sft/#__codelineno-15-10) llm_proxy = self.get_llm_proxy() [](https://microsoft.github.io/agent-lightning/latest/how-to/unsloth-sft/#__codelineno-15-11) data_adapter = self.get_adapter() [](https://microsoft.github.io/agent-lightning/latest/how-to/unsloth-sft/#__codelineno-15-12) [](https://microsoft.github.io/agent-lightning/latest/how-to/unsloth-sft/#__codelineno-15-13) for iteration in range(self.max_iterations): [](https://microsoft.github.io/agent-lightning/latest/how-to/unsloth-sft/#__codelineno-15-14) ... # Same logic as sft_algorithm.py [](https://microsoft.github.io/agent-lightning/latest/how-to/unsloth-sft/#__codelineno-15-15) [](https://microsoft.github.io/agent-lightning/latest/how-to/unsloth-sft/#__codelineno-15-16) algo = UnslothSupervisedFinetuning( [](https://microsoft.github.io/agent-lightning/latest/how-to/unsloth-sft/#__codelineno-15-17) max_iterations=2, [](https://microsoft.github.io/agent-lightning/latest/how-to/unsloth-sft/#__codelineno-15-18) vllm_port=12316, [](https://microsoft.github.io/agent-lightning/latest/how-to/unsloth-sft/#__codelineno-15-19) train_triplet_fraction=0.5, [](https://microsoft.github.io/agent-lightning/latest/how-to/unsloth-sft/#__codelineno-15-20) initial_model_path="models/version_0", [](https://microsoft.github.io/agent-lightning/latest/how-to/unsloth-sft/#__codelineno-15-21) ) [](https://microsoft.github.io/agent-lightning/latest/how-to/unsloth-sft/#__codelineno-15-22) [](https://microsoft.github.io/agent-lightning/latest/how-to/unsloth-sft/#__codelineno-15-23) # The LLM proxy can be created before Trainer [](https://microsoft.github.io/agent-lightning/latest/how-to/unsloth-sft/#__codelineno-15-24) trainer = Trainer( [](https://microsoft.github.io/agent-lightning/latest/how-to/unsloth-sft/#__codelineno-15-25) n_runners=4, [](https://microsoft.github.io/agent-lightning/latest/how-to/unsloth-sft/#__codelineno-15-26) algorithm=algo, [](https://microsoft.github.io/agent-lightning/latest/how-to/unsloth-sft/#__codelineno-15-27) llm_proxy=LLMProxy(port=12358), [](https://microsoft.github.io/agent-lightning/latest/how-to/unsloth-sft/#__codelineno-15-28) ) [](https://microsoft.github.io/agent-lightning/latest/how-to/unsloth-sft/#__codelineno-15-29) [](https://microsoft.github.io/agent-lightning/latest/how-to/unsloth-sft/#__codelineno-15-30) trainer.fit(math_agent, load_math_dataset())` You might wonder where the initialization of [`Adapter`](https://microsoft.github.io/agent-lightning/latest/reference/algorithm/#agentlightning.Adapter " agentlightning.Adapter") happens in this code. It turns out that [`TracerTraceToTriplet`](https://microsoft.github.io/agent-lightning/latest/reference/algorithm/#agentlightning.TracerTraceToTriplet " agentlightning.TracerTraceToTriplet") is the default adapter in [`Trainer`](https://microsoft.github.io/agent-lightning/latest/reference/trainer/#agentlightning.Trainer " agentlightning.Trainer") , so we don't need to create one manually. Now you can run the example with: `[](https://microsoft.github.io/agent-lightning/latest/how-to/unsloth-sft/#__codelineno-16-1) python examples/unsloth/sft_allinone.py` It starts an [`InMemoryLighningStore`](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.InMemoryLightningStore " agentlightning.InMemoryLightningStore") for you, launches four worker processes, iterates the SFT loop, and prints the final checkpoint path when done. Adjust `max_iterations`, `train_triplet_fraction`, `n_runners`, or the proxy port to match your hardware or training goals. If you already run an external store or proxy you can also pass those objects into [`Trainer`](https://microsoft.github.io/agent-lightning/latest/reference/trainer/#agentlightning.Trainer " agentlightning.Trainer") instead of relying on the [Trainer-managed defaults](https://microsoft.github.io/agent-lightning/latest/tutorials/debug/#debug-with-external-store "Debug the Algorithm-Runner Boundary") . Info As a future plan, we might graduate this example into a more powerful SFT algorithm bundled into [Algorithm Zoo](https://microsoft.github.io/agent-lightning/latest/algorithm-zoo/) . Currently, this `UnslothSupervisedFinetuning` is still for demo purposes. Back to top --- # Store - Agent-lightning [Skip to content](https://microsoft.github.io/agent-lightning/latest/reference/store/#store-references) ⚠️ You are viewing development documentation. For stable documentation, please visit [https://microsoft.github.io/agent-lightning/stable/](https://microsoft.github.io/agent-lightning/stable/) Store References[¶](https://microsoft.github.io/agent-lightning/latest/reference/store/#store-references "Permanent link") =========================================================================================================================== ### `agentlightning.LightningStore` [¶](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.LightningStore "Permanent link") Contract for the persistent control-plane that coordinates training rollouts. A `LightningStore` mediates every interaction between algorithms and runners: * **Rollout lifecycle:** accept new rollouts, queue them for execution, create attempts, and drive the rollout status machine (`"queuing"` → `"preparing"` → `"running"` → `{"succeeded","failed","cancelled"}` or `"requeuing"` when a retry is justified). * **Attempt tracking:** record each execution attempt, including progress heartbeats, retry sequencing, and terminal states such as `"timeout"` or `"unresponsive"`. * **Span ingest:** capture structured telemetry emitted by runners (either as native [`Span`](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.Span " agentlightning.Span") objects or as `opentelemetry.sdk.trace.ReadableSpan` instances) so that algorithms can reconstruct trajectories and rewards. * **Resource versioning:** manage immutable snapshots of named resources (prompt templates, model checkpoints, proxy endpoints, …) and expose a single "latest" snapshot that runners can fetch just after claiming work. Implementations must provide thread-safe/async-safe semantics: each coroutine should appear atomic to callers even when multiple algorithms or runners call the API concurrently. Unless stated otherwise, missing identifiers should result in a `ValueError`. #### `capabilities` `property` [¶](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.LightningStore.capabilities "Permanent link") Return the capabilities of the store. #### `add_many_spans(spans)` `async` [¶](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.LightningStore.add_many_spans "Permanent link") Persist a sequence of pre-constructed spans emitted during rollout execution. Implementations can simply delegate to [`add_span()`](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.LightningStore.add_span " add_span(span) async ") for each span. However, if the store supports bulk insertion, it can implement this method to improve performance. #### `add_otel_span(rollout_id, attempt_id, readable_span, sequence_id=None)` `async` [¶](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.LightningStore.add_otel_span "Permanent link") Convert and persist an OpenTelemetry span for a particular attempt. Implementations must transform the `readable_span` into a [`Span`](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.Span " agentlightning.Span") (typically via [`Span.from_opentelemetry()`](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.Span.from_opentelemetry " from_opentelemetry(src, rollout_id, attempt_id, sequence_id) classmethod ") ), assign a strictly increasing `sequence_id` when one is not provided, and persist it using the same semantics as [`add_span()`](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.LightningStore.add_span " add_span(span) async ") . Parameters: * **`rollout_id`** (`str`) – Identifier of the rollout that produced the span. * **`attempt_id`** (`str`) – Attempt identifier the span belongs to. * **`readable_span`** (`ReadableSpan`) – OpenTelemetry span in SDK form. * **`sequence_id`** (`int | None`, default: `None` ) – Optional explicit ordering hint. When omitted, call [`get_next_span_sequence_id()`](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.LightningStore.get_next_span_sequence_id " get_next_span_sequence_id(rollout_id, attempt_id) async ") automatically. Returns: * `Optional[[Span](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.Span " agentlightning.Span (agentlightning.types.Span)") ]` – The stored span record. Return `None` if the span was not added due to a duplicate. Raises: * `NotImplementedError` – Subclasses must implement span persistence. * `ValueError` – Implementations must raise when the rollout or attempt is unknown. #### `add_resources(resources)` `async` [¶](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.LightningStore.add_resources "Permanent link") Persist a new immutable snapshot of named resources and mark it as latest. Implementations must assign a fresh `resources_id` and ensure subsequent calls to [`get_latest_resources()`](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.LightningStore.get_latest_resources " get_latest_resources() async ") return the snapshot produced here. Parameters: * **`resources`** (`[NamedResources](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.NamedResources " agentlightning.NamedResources = Dict[str, ResourceUnion] module-attribute (agentlightning.types.NamedResources)") `) – Mapping of resource names to their serialized payloads. Returns: * `[ResourcesUpdate](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.ResourcesUpdate " agentlightning.ResourcesUpdate (agentlightning.types.ResourcesUpdate)") ` – The stored [`ResourcesUpdate`](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.ResourcesUpdate " agentlightning.ResourcesUpdate") including its generated id. Raises: * `NotImplementedError` – Subclasses must implement resource persistence. #### `add_span(span)` `async` [¶](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.LightningStore.add_span "Permanent link") Persist a pre-constructed span emitted during rollout execution. The provided [`Span`](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.Span " agentlightning.Span") must already contain the `rollout_id`, `attempt_id`, and `sequence_id`. Implementations must: * Verify that both rollout and attempt exist. * Ensure span ordering remains strictly increasing per attempt (rejecting or keeping duplicates). * Treat the span arrival as a heartbeat: update the attempt's `last_heartbeat_time` and transition both attempt and rollout to `"running"` if they were still `"preparing"` or `"requeuing"`. Parameters: * **`span`** (`[Span](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.Span " agentlightning.Span (agentlightning.types.Span)") `) – Fully populated span to persist. Returns: * `Optional[[Span](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.Span " agentlightning.Span (agentlightning.types.Span)") ]` – The stored span record (implementations may return a copy). * `Optional[[Span](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.Span " agentlightning.Span (agentlightning.types.Span)") ]` – Return `None` if the span was not added due to a duplicate. Raises: * `NotImplementedError` – Subclasses must implement span persistence. * `ValueError` – Implementations must raise when the referenced rollout or attempt is missing. #### `dequeue_many_rollouts(*, limit=1, worker_id=None)` `async` [¶](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.LightningStore.dequeue_many_rollouts "Permanent link") Claim up to `limit` queued rollouts without blocking. The implementation can repeatedly invokes [`dequeue_rollout()`](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.LightningStore.dequeue_rollout " dequeue_rollout(worker_id=None) async ") until reaching the requested limit or the queue is empty. Subclasses can override it to fetch multiple rollouts atomically. Parameters: * **`limit`** (`int`, default: `1` ) – Maximum number of rollouts to claim. Non-positive values return an empty list. * **`worker_id`** (`Optional[str]`, default: `None` ) – Optional worker identifier passed through to each dequeue call. Returns: * `Sequence[[AttemptedRollout](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.AttemptedRollout " agentlightning.AttemptedRollout (agentlightning.types.AttemptedRollout)") ]` – Attempted rollouts claimed in FIFO order. May contain fewer than `limit` entries * `Sequence[[AttemptedRollout](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.AttemptedRollout " agentlightning.AttemptedRollout (agentlightning.types.AttemptedRollout)") ]` – when the queue is exhausted. #### `dequeue_rollout(worker_id=None)` `async` [¶](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.LightningStore.dequeue_rollout "Permanent link") Claim the oldest queued rollout and transition it to `preparing`. This function do not block. Retrieval must be FIFO across rollouts that remain in `queuing` or `requeuing` state. When a rollout is claimed, implementations must: * Transition its status to `"preparing"`. * Create a new attempt with `status="preparing"` and `sequence_id` equal to the number of attempts already registered for the rollout plus one. * Return an [`AttemptedRollout`](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.AttemptedRollout " agentlightning.AttemptedRollout") snapshot so the runner knows both rollout metadata and the attempt identifier. * Optionally refresh the caller's [`Worker`](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.Worker " agentlightning.Worker") telemetry (e.g., `last_dequeue_time`) when `worker_id` is provided. Parameters: * **`worker_id`** (`Optional[str]`, default: `None` ) – Optional worker identifier to associate the claimed attempt with. Returns: * `Optional[[AttemptedRollout](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.AttemptedRollout " agentlightning.AttemptedRollout (agentlightning.types.AttemptedRollout)") ]` – The next attempt to execute, or `None` when no eligible rollouts are queued. Raises: * `NotImplementedError` – Subclasses must implement queue retrieval. #### `enqueue_many_rollouts(rollouts)` `async` [¶](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.LightningStore.enqueue_many_rollouts "Permanent link") Persist multiple rollouts in `queuing` state. The implementation can delegate to [`enqueue_rollout()`](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.LightningStore.enqueue_rollout " enqueue_rollout(input, mode=None, resources_id=None, config=None, metadata=None) async ") per request and preserves the input ordering. Subclasses can override to provide more efficient bulk enqueue semantics. Parameters: * **`rollouts`** (`Sequence[[EnqueueRolloutRequest](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.EnqueueRolloutRequest " agentlightning.EnqueueRolloutRequest (agentlightning.types.EnqueueRolloutRequest)") ]`) – Rollout submission payloads mirroring [`enqueue_rollout()`](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.LightningStore.enqueue_rollout " enqueue_rollout(input, mode=None, resources_id=None, config=None, metadata=None) async ") 's parameters. Each entry requires `input` and can optionally include other fields. Returns: * `Sequence[[Rollout](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.Rollout " agentlightning.Rollout (agentlightning.types.Rollout)") ]` – Rollouts enqueued in the same order as `rollouts`. #### `enqueue_rollout(input, mode=None, resources_id=None, config=None, metadata=None)` `async` [¶](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.LightningStore.enqueue_rollout "Permanent link") Persist a rollout in `queuing` state so runners can claim it later. Note Different from [`start_rollout()`](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.LightningStore.start_rollout " start_rollout(input, mode=None, resources_id=None, config=None, metadata=None, worker_id=None) async ") , this method is called when the caller only wants to submit work for later scheduling. Implementations must generate a unique `rollout_id`, stamp `start_time` with the current time, default `config` to a fresh [`RolloutConfig`](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.RolloutConfig " agentlightning.RolloutConfig") , and insert the rollout at the tail of the scheduling queue. No attempt is created yet. Parameters: * **`input`** (`[TaskInput](https://microsoft.github.io/agent-lightning/latest/reference/internal/#agentlightning.TaskInput " agentlightning.TaskInput = Any module-attribute (agentlightning.types.TaskInput)") `) – Arbitrary task payload supplied by an algorithm. * **`mode`** (`Literal['train', 'val', 'test'] | None`, default: `None` ) – Optional semantic mode indicator (`"train"`, `"val"`, `"test"`). * **`resources_id`** (`str | None`, default: `None` ) – Resource snapshot used when a runner eventually executes the rollout. * **`config`** (`[RolloutConfig](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.RolloutConfig " agentlightning.RolloutConfig (agentlightning.types.RolloutConfig)") | None`, default: `None` ) – Fine-grained retry/timeout parameters to persist with the rollout. * **`metadata`** (`Dict[str, Any] | None`, default: `None` ) – Free-form metadata stored verbatim with the rollout record. Returns: * `[Rollout](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.Rollout " agentlightning.Rollout (agentlightning.types.Rollout)") ` – The stored [`Rollout`](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.Rollout " agentlightning.Rollout") in `queuing` status. Raises: * `NotImplementedError` – Subclasses must persist the rollout. * `ValueError` – Implementations should raise when `resources_id` does not exist. #### `get_latest_attempt(rollout_id)` `async` [¶](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.LightningStore.get_latest_attempt "Permanent link") Fetch the attempt with the highest `sequence_id` for `rollout_id`. Parameters: * **`rollout_id`** (`str`) – Identifier to inspect. Returns: * `Optional[[Attempt](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.Attempt " agentlightning.Attempt (agentlightning.types.Attempt)") ]` – The most recent attempt or `None` when no attempts exist yet. Raises: * `NotImplementedError` – Subclasses must implement retrieval. * `ValueError` – Implementations must raise when the rollout does not exist. #### `get_latest_resources()` `async` [¶](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.LightningStore.get_latest_resources "Permanent link") Fetch the latest resource snapshot marked as the global default. Returns: * `Optional[[ResourcesUpdate](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.ResourcesUpdate " agentlightning.ResourcesUpdate (agentlightning.types.ResourcesUpdate)") ]` – The current latest [`ResourcesUpdate`](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.ResourcesUpdate " agentlightning.ResourcesUpdate") , or `None` when * `Optional[[ResourcesUpdate](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.ResourcesUpdate " agentlightning.ResourcesUpdate (agentlightning.types.ResourcesUpdate)") ]` – no resources have been registered yet. Raises: * `NotImplementedError` – Subclasses must implement retrieval. #### `get_many_span_sequence_ids(rollout_attempt_ids)` `async` [¶](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.LightningStore.get_many_span_sequence_ids "Permanent link") Bulk allocate the next strictly increasing sequence number used to order spans. Implementations may delegate to [`get_next_span_sequence_id()`](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.LightningStore.get_next_span_sequence_id " get_next_span_sequence_id(rollout_id, attempt_id) async ") for each rollout and attempt. Parameters: * **`rollout_attempt_ids`** (`Sequence[Tuple[str, str]]`) – List of tuples of rollout and attempt identifiers. Returns: * `Sequence[int]` – List of sequence numbers. #### `get_next_span_sequence_id(rollout_id, attempt_id)` `async` [¶](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.LightningStore.get_next_span_sequence_id "Permanent link") Allocate the next strictly increasing sequence number used to order spans. Implementations must retain counters so repeated calls return `1, 2, ...` without gaps unless spans were explicitly inserted with a custom `sequence_id`. The counter may be scoped per rollout or per attempt, but the sequence must be strictly increasing for spans emitted by the specified attempt so traces remain totally ordered. See [Distributed Tracing](https://microsoft.github.io/agent-lightning/latest/tutorials/traces/#distributed-tracing "LLM Proxy") for detailed motivations. Parameters: * **`rollout_id`** (`str`) – Identifier of the rollout emitting spans. * **`attempt_id`** (`str`) – Attempt identifier for the upcoming span. Returns: * `int` – The next integer sequence identifier, unique within the attempt. Raises: * `NotImplementedError` – Subclasses must provide the allocator. * `ValueError` – Implementations must raise when the rollout or attempt does not exist. #### `get_resources_by_id(resources_id)` `async` [¶](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.LightningStore.get_resources_by_id "Permanent link") Return a specific named resource snapshot by identifier. Parameters: * **`resources_id`** (`str`) – Identifier of the snapshot. Returns: * `Optional[[ResourcesUpdate](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.ResourcesUpdate " agentlightning.ResourcesUpdate (agentlightning.types.ResourcesUpdate)") ]` – The stored [`ResourcesUpdate`](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.ResourcesUpdate " agentlightning.ResourcesUpdate") , or `None` when missing. Raises: * `NotImplementedError` – Subclasses must implement retrieval. #### `get_rollout_by_id(rollout_id)` `async` [¶](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.LightningStore.get_rollout_by_id "Permanent link") Fetch a rollout by identifier without mutating its state. Parameters: * **`rollout_id`** (`str`) – Identifier to retrieve. Returns: * `Optional[[Rollout](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.Rollout " agentlightning.Rollout (agentlightning.types.Rollout)") ]` – The rollout when found, otherwise `None`. Raises: * `NotImplementedError` – Subclasses must implement retrieval. #### `get_worker_by_id(worker_id)` `async` [¶](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.LightningStore.get_worker_by_id "Permanent link") Retrieve a single worker by identifier. Parameters: * **`worker_id`** (`str`) – Identifier of the worker. Returns: * `Optional[[Worker](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.Worker " agentlightning.Worker (agentlightning.types.Worker)") ]` – The worker record if it exists, otherwise `None`. Raises: * `NotImplementedError` – Subclasses must implement lookup semantics. #### `otlp_traces_endpoint()` [¶](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.LightningStore.otlp_traces_endpoint "Permanent link") Return the OTLP/HTTP traces endpoint of the store. The traces can have rollout ID and attempt ID (and optionally sequence ID) saved in the "resource" of the spans. The store, if it supports OTLP, should be able to receive the traces and save them via [`add_span`](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.LightningStore.add_span " add_span(span) async ") or [`add_otel_span`](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.LightningStore.add_otel_span " add_otel_span(rollout_id, attempt_id, readable_span, sequence_id=None) async ") . The endpoint should be compatible with [OTLP HTTP protocol](https://opentelemetry.io/docs/specs/otlp/) . It's not necessarily compatible with OTLP gRPC protocol. The returned endpoint will usually ends with `/v1/traces`. #### `query_attempts(rollout_id, *, sort_by='sequence_id', sort_order='asc', limit=-1, offset=0)` `async` [¶](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.LightningStore.query_attempts "Permanent link") Return every attempt ever created for `rollout_id` in ascending sequence order. The parameters allow callers to re-order or paginate the attempts so that large retry histories can be streamed lazily. Parameters: * **`rollout_id`** (`str`) – Identifier of the rollout being inspected. * **`sort_by`** (`Optional[str]`, default: `'sequence_id'` ) – Field to sort by. Must be a numeric or string field of [`Attempt`](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.Attempt " agentlightning.Attempt") . Defaults to `sequence_id` (oldest first). * **`sort_order`** (`Literal['asc', 'desc']`, default: `'asc'` ) – Order to sort by. * **`limit`** (`int`, default: `-1` ) – Limit on the number of results. `-1` for unlimited. * **`offset`** (`int`, default: `0` ) – Offset into the results. Returns: * `Sequence[[Attempt](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.Attempt " agentlightning.Attempt (agentlightning.types.Attempt)") ]` – Sequence of Attempts. Returns an empty sequence when none exist. * `Sequence[[Attempt](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.Attempt " agentlightning.Attempt (agentlightning.types.Attempt)") ]` – The return value is not guaranteed to be a list. Raises: * `NotImplementedError` – Subclasses must implement the query. * `ValueError` – Implementations must raise when the rollout does not exist. #### `query_resources(*, resources_id=None, resources_id_contains=None, sort_by=None, sort_order='asc', limit=-1, offset=0)` `async` [¶](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.LightningStore.query_resources "Permanent link") List every stored resource snapshot in insertion order. Supports lightweight filtering, sorting, and pagination for embedding in dashboards. Parameters: * **`resources_id`** (`Optional[str]`, default: `None` ) – Optional identifier of the resources to include. * **`resources_id_contains`** (`Optional[str]`, default: `None` ) – Optional substring match for resources identifiers. * **`sort_by`** (`Optional[str]`, default: `None` ) – Optional field to sort by (must be numeric or string on [`ResourcesUpdate`](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.ResourcesUpdate " agentlightning.ResourcesUpdate") ). * **`sort_order`** (`Literal['asc', 'desc']`, default: `'asc'` ) – Order to sort by. * **`limit`** (`int`, default: `-1` ) – Limit on the number of results. `-1` for unlimited. * **`offset`** (`int`, default: `0` ) – Offset into the results. Returns: * `Sequence[[ResourcesUpdate](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.ResourcesUpdate " agentlightning.ResourcesUpdate (agentlightning.types.ResourcesUpdate)") ]` – [`ResourcesUpdate`](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.ResourcesUpdate " agentlightning.ResourcesUpdate") objects. * `Sequence[[ResourcesUpdate](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.ResourcesUpdate " agentlightning.ResourcesUpdate (agentlightning.types.ResourcesUpdate)") ]` – By default, resources are sorted in a deterministic but undefined order. * `Sequence[[ResourcesUpdate](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.ResourcesUpdate " agentlightning.ResourcesUpdate (agentlightning.types.ResourcesUpdate)") ]` – The return value is not guaranteed to be a list. Raises: * `NotImplementedError` – Subclasses must implement retrieval. #### `query_rollouts(*, status_in=None, rollout_id_in=None, rollout_id_contains=None, filter_logic='and', sort_by=None, sort_order='asc', limit=-1, offset=0, status=None, rollout_ids=None)` `async` [¶](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.LightningStore.query_rollouts "Permanent link") Retrieve rollouts filtered by status and/or explicit identifiers. This interface supports structured filtering, sorting, and pagination so callers can build simple dashboards without copying data out of the store. The legacy parameters `status` and `rollout_ids` remain valid and are treated as aliases for `status_in` and `rollout_id_in` respectively—when both the new and deprecated parameters are supplied the new parameters take precedence. Parameters: * **`status_in`** (`Optional[Sequence[[RolloutStatus](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.RolloutStatus " agentlightning.RolloutStatus = Literal['queuing', 'preparing', 'running', 'failed', 'succeeded', 'cancelled', 'requeuing'] module-attribute (agentlightning.types.RolloutStatus)") ]]`, default: `None` ) – Optional whitelist of [`RolloutStatus`](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.RolloutStatus " agentlightning.RolloutStatus = Literal['queuing', 'preparing', 'running', 'failed', 'succeeded', 'cancelled', 'requeuing'] module-attribute ") values. * **`rollout_id_in`** (`Optional[Sequence[str]]`, default: `None` ) – Optional whitelist of rollout identifiers to include. * **`rollout_id_contains`** (`Optional[str]`, default: `None` ) – Optional substring match for rollout identifiers. * **`filter_logic`** (`Literal['and', 'or']`, default: `'and'` ) – Logical operator to combine filters. * **`sort_by`** (`Optional[str]`, default: `None` ) – Optional field to sort by. Must reference a numeric or string field on [`Rollout`](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.Rollout " agentlightning.Rollout") . * **`sort_order`** (`Literal['asc', 'desc']`, default: `'asc'` ) – Direction to sort when `sort_by` is provided. * **`limit`** (`int`, default: `-1` ) – Maximum number of rows to return. Use `-1` for "no limit". * **`offset`** (`int`, default: `0` ) – Number of rows to skip before returning results. * **`status`** (`Optional[Sequence[[RolloutStatus](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.RolloutStatus " agentlightning.RolloutStatus = Literal['queuing', 'preparing', 'running', 'failed', 'succeeded', 'cancelled', 'requeuing'] module-attribute (agentlightning.types.RolloutStatus)") ]]`, default: `None` ) – Deprecated field. Use `status_in` instead. * **`rollout_ids`** (`Optional[Sequence[str]]`, default: `None` ) – Deprecated field. Use `rollout_id_in` instead. Returns: * `Sequence[[Rollout](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.Rollout " agentlightning.Rollout (agentlightning.types.Rollout)") ]` – A sequence of matching rollouts (or [`AttemptedRollout`](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.AttemptedRollout " agentlightning.AttemptedRollout") * `Sequence[[Rollout](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.Rollout " agentlightning.Rollout (agentlightning.types.Rollout)") ]` – when attempts exist). Ordering is deterministic when `sort_by` is set. * `Sequence[[Rollout](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.Rollout " agentlightning.Rollout (agentlightning.types.Rollout)") ]` – The return value is not guaranteed to be a list. Raises: * `NotImplementedError` – Subclasses must implement the query. #### `query_spans(rollout_id, attempt_id=None, *, trace_id=None, trace_id_contains=None, span_id=None, span_id_contains=None, parent_id=None, parent_id_contains=None, name=None, name_contains=None, filter_logic='and', limit=-1, offset=0, sort_by='sequence_id', sort_order='asc')` `async` [¶](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.LightningStore.query_spans "Permanent link") Return the stored spans for a rollout, optionally scoped to one attempt. Supports a handful of filters that cover the most common debugging scenarios (matching `trace_id`/`span_id`/`parent_id` or substring matches on the span name). `attempt_id="latest"` acts as a convenience that resolves the most recent attempt before evaluating filters. When `attempt_id=None`, spans across every attempt are eligible. By default results are sorted by `sequence_id` (oldest first). Implementations may raise a `RuntimeError` when spans were evicted or expired. Parameters: * **`rollout_id`** (`str`) – Identifier of the rollout being inspected. * **`attempt_id`** (`str | Literal['latest'] | None`, default: `None` ) – Attempt identifier to filter by. Pass `"latest"` to retrieve only the most recent attempt, or `None` to return all spans across attempts. * **`trace_id`** (`Optional[str]`, default: `None` ) – Optional trace ID to filter by. * **`trace_id_contains`** (`Optional[str]`, default: `None` ) – Optional substring match for trace IDs. * **`span_id`** (`Optional[str]`, default: `None` ) – Optional span ID to filter by. * **`span_id_contains`** (`Optional[str]`, default: `None` ) – Optional substring match for span IDs. * **`parent_id`** (`Optional[str]`, default: `None` ) – Optional parent span ID to filter by. * **`parent_id_contains`** (`Optional[str]`, default: `None` ) – Optional substring match for parent span IDs. * **`name`** (`Optional[str]`, default: `None` ) – Optional span name to filter by. * **`name_contains`** (`Optional[str]`, default: `None` ) – Optional substring match for span names. * **`filter_logic`** (`Literal['and', 'or']`, default: `'and'` ) – Logical operator to combine the optional filters above. The `rollout_id` argument is always applied with AND semantics. * **`limit`** (`int`, default: `-1` ) – Limit on the number of results. `-1` for unlimited. * **`offset`** (`int`, default: `0` ) – Offset into the results. * **`sort_by`** (`Optional[str]`, default: `'sequence_id'` ) – Field to sort by. Must be a numeric or string field of [`Span`](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.Span " agentlightning.Span") . * **`sort_order`** (`Literal['asc', 'desc']`, default: `'asc'` ) – Order to sort by. Returns: * `Sequence[[Span](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.Span " agentlightning.Span (agentlightning.types.Span)") ]` – An ordered list of spans (possibly empty). * `Sequence[[Span](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.Span " agentlightning.Span (agentlightning.types.Span)") ]` – The return value is not guaranteed to be a list. Raises: * `NotImplementedError` – Subclasses must implement the query. * `ValueError` – Implementations must raise when the rollout or attempt is unknown. #### `query_workers(*, status_in=None, worker_id_contains=None, filter_logic='and', sort_by=None, sort_order='asc', limit=-1, offset=0)` `async` [¶](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.LightningStore.query_workers "Permanent link") Query all workers in the system. Parameters: * **`status_in`** (`Optional[Sequence[[WorkerStatus](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.WorkerStatus " agentlightning.WorkerStatus = Literal['idle', 'busy', 'unknown'] module-attribute (agentlightning.types.WorkerStatus)") ]]`, default: `None` ) – Optional whitelist of [`WorkerStatus`](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.WorkerStatus " agentlightning.WorkerStatus = Literal['idle', 'busy', 'unknown'] module-attribute ") values. * **`worker_id_contains`** (`Optional[str]`, default: `None` ) – Optional substring match for worker identifiers. * **`filter_logic`** (`Literal['and', 'or']`, default: `'and'` ) – Logical operator to combine the optional filters above. * **`sort_by`** (`Optional[str]`, default: `None` ) – Field to sort by. Must be a numeric or string field of [`Worker`](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.Worker " agentlightning.Worker") . * **`sort_order`** (`Literal['asc', 'desc']`, default: `'asc'` ) – Order to sort by. * **`limit`** (`int`, default: `-1` ) – Limit on the number of results. `-1` for unlimited. * **`offset`** (`int`, default: `0` ) – Offset into the results. Returns: * `Sequence[[Worker](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.Worker " agentlightning.Worker (agentlightning.types.Worker)") ]` – Sequence of Workers. Returns an empty sequence when none exist. * `Sequence[[Worker](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.Worker " agentlightning.Worker (agentlightning.types.Worker)") ]` – The return value is not guaranteed to be a list. #### `start_attempt(rollout_id, worker_id=None)` `async` [¶](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.LightningStore.start_attempt "Permanent link") Create a manual retry attempt for an existing rollout. This is typically invoked by runners that wish to retry outside of the normal queue flow (for example in an online RL setup). Implementations must validate that the rollout exists, allocate a fresh `attempt_id`, increment the `sequence_id` monotonically, stamp the new attempt with `status="preparing"`, and return an up-to-date [`AttemptedRollout`](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.AttemptedRollout " agentlightning.AttemptedRollout") . Parameters: * **`rollout_id`** (`str`) – Unique identifier of the rollout receiving a new attempt. * **`worker_id`** (`Optional[str]`, default: `None` ) – Optional worker identifier to associate the new attempt with. Returns: * `[AttemptedRollout](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.AttemptedRollout " agentlightning.AttemptedRollout (agentlightning.types.AttemptedRollout)") ` – The rollout paired with its newly-created attempt. Raises: * `NotImplementedError` – Subclasses must implement attempt creation. * `ValueError` – Implementations must raise when `rollout_id` is unknown. #### `start_rollout(input, mode=None, resources_id=None, config=None, metadata=None, worker_id=None)` `async` [¶](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.LightningStore.start_rollout "Permanent link") Register a rollout and immediately create its first attempt. Note Use [`enqueue_rollout()`](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.LightningStore.enqueue_rollout " enqueue_rollout(input, mode=None, resources_id=None, config=None, metadata=None) async ") when the caller only wants to submit work for later scheduling. The rollout must be persisted with `status="preparing"` and an initial attempt with `sequence_id == 1` so the caller can begin execution without visiting the public queue. Implementations are expected to: 1. Generate a unique `rollout_id` and `attempt_id`. 2. Record `start_time` for both rollout and attempt based on the current clock. 3. Copy `config` and `metadata` so later mutations do not leak shared references. 4. Resolve `resources_id` to the latest resource snapshot when `None` is supplied. Parameters: * **`input`** (`[TaskInput](https://microsoft.github.io/agent-lightning/latest/reference/internal/#agentlightning.TaskInput " agentlightning.TaskInput = Any module-attribute (agentlightning.types.TaskInput)") `) – Arbitrary task payload supplied by an algorithm. * **`mode`** (`[RolloutMode](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.RolloutMode " agentlightning.RolloutMode = Literal['train', 'val', 'test'] module-attribute (agentlightning.types.RolloutMode)") | None`, default: `None` ) – Optional semantic mode for downstream analytics (`"train"`, `"val"`, `"test"`). * **`resources_id`** (`str | None`, default: `None` ) – Concrete resource snapshot to execute against; defaults to the latest stored snapshot. * **`config`** (`[RolloutConfig](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.RolloutConfig " agentlightning.RolloutConfig (agentlightning.types.RolloutConfig)") | None`, default: `None` ) – Rollout retry/timeout policy. Should default to a fresh [`RolloutConfig`](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.RolloutConfig " agentlightning.RolloutConfig") . * **`metadata`** (`Dict[str, Any] | None`, default: `None` ) – Free-form metadata persisted verbatim with the rollout. * **`worker_id`** (`str | None`, default: `None` ) – Optional worker identifier to associate the new attempt with. Returns: * `[AttemptedRollout](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.AttemptedRollout " agentlightning.AttemptedRollout (agentlightning.types.AttemptedRollout)") ` – The fully-populated [`AttemptedRollout`](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.AttemptedRollout " agentlightning.AttemptedRollout") including * `[AttemptedRollout](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.AttemptedRollout " agentlightning.AttemptedRollout (agentlightning.types.AttemptedRollout)") ` – the just-created attempt. Raises: * `NotImplementedError` – Subclasses must provide durable storage for the rollout. * `ValueError` – Implementations should raise when `resources_id` does not exist. #### `statistics()` `async` [¶](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.LightningStore.statistics "Permanent link") Return the statistics of the store. #### `update_attempt(rollout_id, attempt_id, status=UNSET, worker_id=UNSET, last_heartbeat_time=UNSET, metadata=UNSET)` `async` [¶](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.LightningStore.update_attempt "Permanent link") Update attempt bookkeeping such as status, worker ownership, and heartbeats. When `attempt_id` is `"latest"` the update must target the attempt with the highest `sequence_id`; otherwise it must target the specific attempt. Implementations should propagate status changes to the rollout (for example via [`rollout_status_from_attempt()`](https://microsoft.github.io/agent-lightning/latest/reference/internal/#agentlightning.store.utils.rollout_status_from_attempt " agentlightning.store.utils.rollout_status_from_attempt(attempt, config) async ") ) once the latest attempt transitions to a terminal state. Similar to [`update_rollout()`](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.LightningStore.update_rollout " update_rollout(rollout_id, input=UNSET, mode=UNSET, resources_id=UNSET, status=UNSET, config=UNSET, metadata=UNSET) async ") , parameters also default to the sentinel [`UNSET`](https://microsoft.github.io/agent-lightning/latest/reference/internal/#agentlightning.store.base.UNSET " agentlightning.store.base.UNSET = _UnsetType() module-attribute ") . If `worker_id` is present, the worker status will be updated following the rules: 1. If attempt status is "succeeded" or "failed", the corresponding worker status will be set to "idle". 2. If attempt status is "unresponsive" or "timeout", the corresponding worker status will be set to "unknown". 3. Otherwise, the worker status will be set to "busy". Parameters: * **`rollout_id`** (`str`) – Identifier of the rollout whose attempt will be updated. * **`attempt_id`** (`str | Literal['latest']`) – Attempt identifier or `"latest"` as a convenience. * **`status`** (`[AttemptStatus](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.AttemptStatus " agentlightning.AttemptStatus = Literal['preparing', 'running', 'failed', 'succeeded', 'unresponsive', 'timeout'] module-attribute (agentlightning.types.AttemptStatus)") | Unset`, default: `[UNSET](https://microsoft.github.io/agent-lightning/latest/reference/internal/#agentlightning.store.base.UNSET " agentlightning.store.base.UNSET = _UnsetType() module-attribute (agentlightning.store.base.UNSET)") ` ) – Replacement attempt status. Terminal statuses must set `end_time`. * **`worker_id`** (`str | Unset`, default: `[UNSET](https://microsoft.github.io/agent-lightning/latest/reference/internal/#agentlightning.store.base.UNSET " agentlightning.store.base.UNSET = _UnsetType() module-attribute (agentlightning.store.base.UNSET)") ` ) – Identifier for the worker currently processing the attempt. * **`last_heartbeat_time`** (`float | Unset`, default: `[UNSET](https://microsoft.github.io/agent-lightning/latest/reference/internal/#agentlightning.store.base.UNSET " agentlightning.store.base.UNSET = _UnsetType() module-attribute (agentlightning.store.base.UNSET)") ` ) – Wall-clock timestamp (seconds) of the latest heartbeat/span. * **`metadata`** (`Optional[Dict[str, Any]] | Unset`, default: `[UNSET](https://microsoft.github.io/agent-lightning/latest/reference/internal/#agentlightning.store.base.UNSET " agentlightning.store.base.UNSET = _UnsetType() module-attribute (agentlightning.store.base.UNSET)") ` ) – Replacement metadata dictionary. Returns: * `[Attempt](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.Attempt " agentlightning.Attempt (agentlightning.types.Attempt)") ` – The updated attempt record. Raises: * `NotImplementedError` – Subclasses must implement mutation logic. * `ValueError` – Implementations must raise when the rollout or attempt is unknown. #### `update_resources(resources_id, resources)` `async` [¶](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.LightningStore.update_resources "Permanent link") Overwrite or extend an existing resource snapshot and mark it as latest. This API is typically used by algorithms that maintain mutable resources (e.g., model checkpoints) under a stable identifier. Parameters: * **`resources_id`** (`str`) – Identifier of the snapshot to replace. * **`resources`** (`[NamedResources](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.NamedResources " agentlightning.NamedResources = Dict[str, ResourceUnion] module-attribute (agentlightning.types.NamedResources)") `) – Updated mapping of resource names to payloads. Returns: * `[ResourcesUpdate](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.ResourcesUpdate " agentlightning.ResourcesUpdate (agentlightning.types.ResourcesUpdate)") ` – The persisted [`ResourcesUpdate`](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.ResourcesUpdate " agentlightning.ResourcesUpdate") . Raises: * `NotImplementedError` – Subclasses must implement resource persistence. * `ValueError` – Implementations must raise when `resources_id` does not exist. #### `update_rollout(rollout_id, input=UNSET, mode=UNSET, resources_id=UNSET, status=UNSET, config=UNSET, metadata=UNSET)` `async` [¶](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.LightningStore.update_rollout "Permanent link") Update rollout metadata and, when provided, drive status transitions. Parameters default to the sentinel [`UNSET`](https://microsoft.github.io/agent-lightning/latest/reference/internal/#agentlightning.store.base.UNSET " agentlightning.store.base.UNSET = _UnsetType() module-attribute ") to distinguish omitted fields from explicit `None` assignments. Implementations must: * Validate the rollout exists before mutating it. * Replace each property when a concrete value (including `None`) is supplied. * When the status switches into a terminal state, set `end_time` and signal any waiters. * When the status re-enters a queueing state, ensure the rollout is enqueued exactly once. Parameters: * **`rollout_id`** (`str`) – Identifier of the rollout to update. * **`input`** (`[TaskInput](https://microsoft.github.io/agent-lightning/latest/reference/internal/#agentlightning.TaskInput " agentlightning.TaskInput = Any module-attribute (agentlightning.types.TaskInput)") | Unset`, default: `[UNSET](https://microsoft.github.io/agent-lightning/latest/reference/internal/#agentlightning.store.base.UNSET " agentlightning.store.base.UNSET = _UnsetType() module-attribute (agentlightning.store.base.UNSET)") ` ) – Replacement task payload; pass `None` to explicitly clear the input. * **`mode`** (`Optional[Literal['train', 'val', 'test']] | Unset`, default: `[UNSET](https://microsoft.github.io/agent-lightning/latest/reference/internal/#agentlightning.store.base.UNSET " agentlightning.store.base.UNSET = _UnsetType() module-attribute (agentlightning.store.base.UNSET)") ` ) – Replacement rollout mode. * **`resources_id`** (`Optional[str] | Unset`, default: `[UNSET](https://microsoft.github.io/agent-lightning/latest/reference/internal/#agentlightning.store.base.UNSET " agentlightning.store.base.UNSET = _UnsetType() module-attribute (agentlightning.store.base.UNSET)") ` ) – Replacement resources snapshot reference. * **`status`** (`[RolloutStatus](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.RolloutStatus " agentlightning.RolloutStatus = Literal['queuing', 'preparing', 'running', 'failed', 'succeeded', 'cancelled', 'requeuing'] module-attribute (agentlightning.types.RolloutStatus)") | Unset`, default: `[UNSET](https://microsoft.github.io/agent-lightning/latest/reference/internal/#agentlightning.store.base.UNSET " agentlightning.store.base.UNSET = _UnsetType() module-attribute (agentlightning.store.base.UNSET)") ` ) – Target rollout status. * **`config`** (`[RolloutConfig](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.RolloutConfig " agentlightning.RolloutConfig (agentlightning.types.RolloutConfig)") | Unset`, default: `[UNSET](https://microsoft.github.io/agent-lightning/latest/reference/internal/#agentlightning.store.base.UNSET " agentlightning.store.base.UNSET = _UnsetType() module-attribute (agentlightning.store.base.UNSET)") ` ) – Replacement retry/timeout configuration. * **`metadata`** (`Optional[Dict[str, Any]] | Unset`, default: `[UNSET](https://microsoft.github.io/agent-lightning/latest/reference/internal/#agentlightning.store.base.UNSET " agentlightning.store.base.UNSET = _UnsetType() module-attribute (agentlightning.store.base.UNSET)") ` ) – Replacement metadata dictionary. Returns: * `[Rollout](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.Rollout " agentlightning.Rollout (agentlightning.types.Rollout)") ` – The updated rollout record. Raises: * `NotImplementedError` – Subclasses must implement mutation logic. * `ValueError` – Implementations must raise when the rollout is unknown or the update is invalid. #### `update_worker(worker_id, heartbeat_stats=UNSET)` `async` [¶](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.LightningStore.update_worker "Permanent link") Record a heartbeat for `worker_id` and refresh telemetry. Implementations must treat this API as heartbeat-only: it should snapshot the latest stats when provided, stamp `last_heartbeat_time` with the current wall clock, and rely on other store mutations (`dequeue_rollout`, `update_attempt`, etc.) to drive the worker's busy/idle status, assignment, and activity timestamps. Parameters: * **`worker_id`** (`str`) – Identifier of the worker to update. * **`heartbeat_stats`** (`Dict[str, Any] | Unset`, default: `[UNSET](https://microsoft.github.io/agent-lightning/latest/reference/internal/#agentlightning.store.base.UNSET " agentlightning.store.base.UNSET = _UnsetType() module-attribute (agentlightning.store.base.UNSET)") ` ) – Replacement worker heartbeat statistics (non-null when provided). #### `wait_for_rollouts(*, rollout_ids, timeout=None)` `async` [¶](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.LightningStore.wait_for_rollouts "Permanent link") Block until the targeted rollouts reach a terminal status or the timeout expires. Terminal statuses are `"succeeded"`, `"failed"`, and `"cancelled"`. When the timeout elapses, implementations should return the subset of rollouts that are already terminal and omit the rest. Warning It's dangerous and might be event-loop blocking to call this function with a long timeout. It's a good idea to poll for the method to check if new completed rollouts can coming. Be careful in implementing the sleep logic to avoid busy-waiting. Parameters: * **`rollout_ids`** (`List[str]`) – Identifiers of rollouts to watch. * **`timeout`** (`Optional[float]`, default: `None` ) – Maximum time in seconds to wait. `None` waits indefinitely. Returns: * `List[[Rollout](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.Rollout " agentlightning.Rollout (agentlightning.types.Rollout)") ]` – Rollouts that finished before the deadline, in arbitrary order. Raises: * `NotImplementedError` – Subclasses must implement waiting semantics. * `ValueError` – Implementations must raise when a rollout identifier is unknown. ### `agentlightning.LightningStoreCapabilities` [¶](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.LightningStoreCapabilities "Permanent link") Bases: `TypedDict` Capability of a LightningStore implementation. All keys are optional and false by default. #### `async_safe` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.LightningStoreCapabilities.async_safe "Permanent link") Whether the store is async-safe. #### `otlp_traces` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.LightningStoreCapabilities.otlp_traces "Permanent link") Whether the store supports OTLP/HTTP traces. #### `thread_safe` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.LightningStoreCapabilities.thread_safe "Permanent link") Whether the store is thread-safe. #### `zero_copy` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.LightningStoreCapabilities.zero_copy "Permanent link") Whether the store has only one copy across all threads/processes. Store Implementations[¶](https://microsoft.github.io/agent-lightning/latest/reference/store/#store-implementations "Permanent link") ------------------------------------------------------------------------------------------------------------------------------------- ### `agentlightning.InMemoryLightningStore` [¶](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.InMemoryLightningStore "Permanent link") Bases: `[CollectionBasedLightningStore](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.CollectionBasedLightningStore " agentlightning.CollectionBasedLightningStore (agentlightning.store.collection_based.CollectionBasedLightningStore)") [[InMemoryLightningCollections](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.store.collection.InMemoryLightningCollections " agentlightning.store.collection.InMemoryLightningCollections") ]` In-memory implementation of LightningStore using Python data structures. Thread-safe and async-compatible but data is not persistent. Parameters: * **`thread_safe`** (`bool`, default: `False` ) – Whether the store is thread-safe. * **`eviction_memory_threshold`** (`float | int | None`, default: `None` ) – The threshold for evicting spans in bytes. By default, it's 70% of the total VRAM available. * **`safe_memory_threshold`** (`float | int | None`, default: `None` ) – The threshold for safe memory usage in bytes. By default, it's 80% of the eviction threshold. * **`span_size_estimator`** (`Callable[[[Span](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.Span " agentlightning.Span (agentlightning.types.Span)") ], int] | None`, default: `None` ) – A function to estimate the size of a span in bytes. By default, it's a simple size estimator that uses sys.getsizeof. * **`tracker`** (`[MetricsBackend](https://microsoft.github.io/agent-lightning/latest/reference/utilities/#agentlightning.utils.metrics.MetricsBackend " agentlightning.utils.metrics.MetricsBackend") | None`, default: `None` ) – The metrics tracker to use. * **`scan_debounce_seconds`** (`float`, default: `10.0` ) – The debounce time for the scan for unhealthy rollouts. Set to 0 to disable debouncing. #### `capabilities` `property` [¶](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.InMemoryLightningStore.capabilities "Permanent link") Return the capabilities of the store. #### `statistics()` `async` [¶](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.InMemoryLightningStore.statistics "Permanent link") Return the statistics of the store. #### `wait_for_rollout(rollout_id, timeout=None)` `async` [¶](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.InMemoryLightningStore.wait_for_rollout "Permanent link") Wait for a specific rollout to complete with a timeout. ### `agentlightning.store.mongo.MongoLightningStore` [¶](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.store.mongo.MongoLightningStore "Permanent link") Bases: `[CollectionBasedLightningStore](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.CollectionBasedLightningStore " agentlightning.CollectionBasedLightningStore (agentlightning.store.collection_based.CollectionBasedLightningStore)") [[MongoLightningCollections](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.store.collection.mongo.MongoLightningCollections " agentlightning.store.collection.mongo.MongoLightningCollections") ]` MongoDB implementation of LightningStore using MongoDB collections. Data is persistent and can be shared between multiple processes. Parameters: * **`mongo_uri`** (`str`, default: `'mongodb://localhost:27017/?replicaSet=rs0'` ) – MongoDB connection string (defaults to local replica set). * **`mongo_client_kwargs`** (`Mapping[str, Any] | None`, default: `None` ) – Extra keyword arguments forwarded to `AsyncMongoClient`. * **`database_name`** (`str | None`, default: `None` ) – The MongoDB database name. Defaults to `agentlightning`. * **`partition_id`** (`str | None`, default: `None` ) – The partition id. Useful when sharing the database among multiple Agent-lightning trainers. * **`tracker`** (`[MetricsBackend](https://microsoft.github.io/agent-lightning/latest/reference/utilities/#agentlightning.utils.metrics.MetricsBackend " agentlightning.utils.metrics.MetricsBackend") | None`, default: `None` ) – The metrics tracker to use. * **`scan_debounce_seconds`** (`float`, default: `10.0` ) – The debounce time for the scan for unhealthy rollouts. Set to 0 to disable debouncing. #### `capabilities` `property` [¶](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.store.mongo.MongoLightningStore.capabilities "Permanent link") Return the capabilities of the store. #### `close()` `async` [¶](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.store.mongo.MongoLightningStore.close "Permanent link") Close the store by closing the client pool. #### `wait_for_rollouts(*, rollout_ids, timeout=None)` `async` [¶](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.store.mongo.MongoLightningStore.wait_for_rollouts "Permanent link") Wait for specified rollouts to complete with a timeout. Concurrently wait for all rollouts to complete with a timeout. ### `agentlightning.CollectionBasedLightningStore` [¶](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.CollectionBasedLightningStore "Permanent link") Bases: `[LightningStore](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.LightningStore " agentlightning.LightningStore (agentlightning.store.base.LightningStore)") `, `Generic[T_collections]` It's the standard implementation of LightningStore that uses collections to store data. If the store implementation is to use the store's default behavior, it's recommended to inherit from this class and override the methods if needed. Bring your own collection implementation by using a different `collections` argument. The methods in this class should generally not call each other, especially those that are locked. Parameters: * **`collections`** (`T_collections`) – The collections to use for storage. * **`read_snapshot`** (`bool`, default: `False` ) – Make sure read operations are atomic. If set to true, all read operations like `query_rollouts` will have better consistency. It may use an isolated snapshot that supports repeatable reads. * **`tracker`** (`[MetricsBackend](https://microsoft.github.io/agent-lightning/latest/reference/utilities/#agentlightning.utils.metrics.MetricsBackend " agentlightning.utils.metrics.MetricsBackend") | None`, default: `None` ) – Enable metrics tracking. * **`scan_debounce_seconds`** (`float`, default: `10.0` ) – The debounce time for the scan for unhealthy rollouts. Set to 0 to disable debouncing. The debounce is a non-perfect traffic control. It's isolated for each store instance if there are multiple worker replicas. #### `capabilities` `property` [¶](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.CollectionBasedLightningStore.capabilities "Permanent link") Return the capabilities of the store. This store supports no capability. The capability depends on the underlying collections. #### `add_many_spans(spans)` `async` [¶](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.CollectionBasedLightningStore.add_many_spans "Permanent link") Persist a sequence of pre-converted spans. See [`LightningStore.add_many_spans()`](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.LightningStore.add_many_spans " add_many_spans(spans) async ") for semantics. #### `add_otel_span(rollout_id, attempt_id, readable_span, sequence_id=None)` `async` [¶](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.CollectionBasedLightningStore.add_otel_span "Permanent link") Add an opentelemetry span to the store. See [`LightningStore.add_otel_span()`](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.LightningStore.add_otel_span " add_otel_span(rollout_id, attempt_id, readable_span, sequence_id=None) async ") for semantics. #### `add_resources(resources)` `async` [¶](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.CollectionBasedLightningStore.add_resources "Permanent link") Stores a new version of named resources and sets it as the latest. See [`LightningStore.add_resources()`](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.LightningStore.add_resources " add_resources(resources) async ") for semantics. #### `add_span(span)` `async` [¶](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.CollectionBasedLightningStore.add_span "Permanent link") Persist a pre-converted span. See [`LightningStore.add_span()`](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.LightningStore.add_span " add_span(span) async ") for semantics. #### `dequeue_many_rollouts(*, limit=1, worker_id=None)` `async` [¶](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.CollectionBasedLightningStore.dequeue_many_rollouts "Permanent link") Retrieves up to `limit` tasks from the queue without blocking. #### `dequeue_rollout(worker_id=None)` `async` [¶](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.CollectionBasedLightningStore.dequeue_rollout "Permanent link") Retrieves the next task from the queue without blocking. Returns `None` if the queue is empty. Will set the rollout status to preparing and create a new attempt. See [`LightningStore.dequeue_rollout()`](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.LightningStore.dequeue_rollout " dequeue_rollout(worker_id=None) async ") for semantics. #### `enqueue_many_rollouts(rollouts)` `async` [¶](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.CollectionBasedLightningStore.enqueue_many_rollouts "Permanent link") Adds many rollouts in a batch. #### `enqueue_rollout(input, mode=None, resources_id=None, config=None, metadata=None)` `async` [¶](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.CollectionBasedLightningStore.enqueue_rollout "Permanent link") Adds a new task to the queue with specific metadata and returns the rollout. See [`LightningStore.enqueue_rollout()`](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.LightningStore.enqueue_rollout " enqueue_rollout(input, mode=None, resources_id=None, config=None, metadata=None) async ") for semantics. #### `get_latest_attempt(rollout_id)` `async` [¶](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.CollectionBasedLightningStore.get_latest_attempt "Permanent link") Retrieves the latest attempt for a given rollout ID. See [`LightningStore.get_latest_attempt()`](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.LightningStore.get_latest_attempt " get_latest_attempt(rollout_id) async ") for semantics. #### `get_latest_resources()` `async` [¶](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.CollectionBasedLightningStore.get_latest_resources "Permanent link") Retrieves the latest version of named resources. See [`LightningStore.get_latest_resources()`](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.LightningStore.get_latest_resources " get_latest_resources() async ") for semantics. #### `get_many_span_sequence_ids(rollout_attempt_ids)` `async` [¶](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.CollectionBasedLightningStore.get_many_span_sequence_ids "Permanent link") Get the next span sequence IDs for a given list of rollout and attempt identifiers. #### `get_next_span_sequence_id(rollout_id, attempt_id)` `async` [¶](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.CollectionBasedLightningStore.get_next_span_sequence_id "Permanent link") Get the next span sequence ID for a given rollout and attempt. The number is strictly increasing for each rollout. The store will not issue the same sequence ID twice. See [`LightningStore.get_next_span_sequence_id()`](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.LightningStore.get_next_span_sequence_id " get_next_span_sequence_id(rollout_id, attempt_id) async ") for semantics. #### `get_resources_by_id(resources_id)` `async` [¶](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.CollectionBasedLightningStore.get_resources_by_id "Permanent link") Retrieves a specific version of named resources by its ID. See [`LightningStore.get_resources_by_id()`](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.LightningStore.get_resources_by_id " get_resources_by_id(resources_id) async ") for semantics. #### `get_rollout_by_id(rollout_id)` `async` [¶](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.CollectionBasedLightningStore.get_rollout_by_id "Permanent link") Retrieves a specific rollout by its ID. See [`LightningStore.get_rollout_by_id()`](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.LightningStore.get_rollout_by_id " get_rollout_by_id(rollout_id) async ") for semantics. If the rollout has been attempted, the latest attempt will also be returned. #### `query_attempts(rollout_id, *, sort_by='sequence_id', sort_order='asc', limit=-1, offset=0)` `async` [¶](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.CollectionBasedLightningStore.query_attempts "Permanent link") Retrieve attempts for a rollout with optional ordering/pagination. #### `query_resources(*, resources_id=None, resources_id_contains=None, sort_by=None, sort_order='asc', limit=-1, offset=0)` `async` [¶](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.CollectionBasedLightningStore.query_resources "Permanent link") Return every stored resource snapshot in insertion order. #### `query_rollouts(*, status_in=None, rollout_id_in=None, rollout_id_contains=None, filter_logic='and', sort_by=None, sort_order='asc', limit=-1, offset=0, status=None, rollout_ids=None)` `async` [¶](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.CollectionBasedLightningStore.query_rollouts "Permanent link") Retrieve rollouts with filtering and pagination. See [`LightningStore.query_rollouts()`](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.LightningStore.query_rollouts " query_rollouts(*, status_in=None, rollout_id_in=None, rollout_id_contains=None, filter_logic='and', sort_by=None, sort_order='asc', limit=-1, offset=0, status=None, rollout_ids=None) async ") for semantics. #### `query_spans(rollout_id, attempt_id=None, *, trace_id=None, trace_id_contains=None, span_id=None, span_id_contains=None, parent_id=None, parent_id_contains=None, name=None, name_contains=None, filter_logic='and', limit=-1, offset=0, sort_by='sequence_id', sort_order='asc')` `async` [¶](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.CollectionBasedLightningStore.query_spans "Permanent link") Query and retrieve spans associated with a specific rollout ID. Returns an empty list if no spans are found. See [`LightningStore.query_spans()`](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.LightningStore.query_spans " query_spans(rollout_id, attempt_id=None, *, trace_id=None, trace_id_contains=None, span_id=None, span_id_contains=None, parent_id=None, parent_id_contains=None, name=None, name_contains=None, filter_logic='and', limit=-1, offset=0, sort_by='sequence_id', sort_order='asc') async ") for semantics. #### `query_workers(*, status_in=None, worker_id_contains=None, filter_logic='and', sort_by=None, sort_order='asc', limit=-1, offset=0)` `async` [¶](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.CollectionBasedLightningStore.query_workers "Permanent link") Return the current snapshot of all workers. #### `start_attempt(rollout_id, worker_id=None)` `async` [¶](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.CollectionBasedLightningStore.start_attempt "Permanent link") Creates a new attempt for a given rollout ID and return the attempt details. See [`LightningStore.start_attempt()`](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.LightningStore.start_attempt " start_attempt(rollout_id, worker_id=None) async ") for semantics. #### `start_rollout(input, mode=None, resources_id=None, config=None, metadata=None, worker_id=None)` `async` [¶](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.CollectionBasedLightningStore.start_rollout "Permanent link") Notify the store that I'm about to run a rollout. See [`LightningStore.start_rollout()`](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.LightningStore.start_rollout " start_rollout(input, mode=None, resources_id=None, config=None, metadata=None, worker_id=None) async ") for semantics. #### `statistics()` `async` [¶](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.CollectionBasedLightningStore.statistics "Permanent link") Return the statistics of the store. #### `update_attempt(rollout_id, attempt_id, status=UNSET, worker_id=UNSET, last_heartbeat_time=UNSET, metadata=UNSET)` `async` [¶](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.CollectionBasedLightningStore.update_attempt "Permanent link") Update a specific or latest attempt for a given rollout. See [`LightningStore.update_attempt()`](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.LightningStore.update_attempt " update_attempt(rollout_id, attempt_id, status=UNSET, worker_id=UNSET, last_heartbeat_time=UNSET, metadata=UNSET) async ") for semantics. #### `update_resources(collections, resources_id, resources)` `async` [¶](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.CollectionBasedLightningStore.update_resources "Permanent link") Safely stores a new version of named resources and sets it as the latest. See [`LightningStore.update_resources()`](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.LightningStore.update_resources " update_resources(resources_id, resources) async ") for semantics. #### `update_rollout(rollout_id, input=UNSET, mode=UNSET, resources_id=UNSET, status=UNSET, config=UNSET, metadata=UNSET)` `async` [¶](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.CollectionBasedLightningStore.update_rollout "Permanent link") Update the rollout status and related metadata. See [`LightningStore.update_rollout()`](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.LightningStore.update_rollout " update_rollout(rollout_id, input=UNSET, mode=UNSET, resources_id=UNSET, status=UNSET, config=UNSET, metadata=UNSET) async ") for semantics. #### `update_worker(worker_id, heartbeat_stats=UNSET)` `async` [¶](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.CollectionBasedLightningStore.update_worker "Permanent link") Create or update a worker entry. #### `wait_for_rollout(rollout_id, timeout=None)` `async` [¶](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.CollectionBasedLightningStore.wait_for_rollout "Permanent link") Wait for a specific rollout to complete with a timeout. Subclass may use advanced mechanisms like events to accelerate this. Returns the completed rollout, or None if timeout is reached. #### `wait_for_rollouts(*, rollout_ids, timeout=None)` `async` [¶](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.CollectionBasedLightningStore.wait_for_rollouts "Permanent link") Wait for specified rollouts to complete with a timeout. Returns the completed rollouts, potentially incomplete if timeout is reached. This method does not change the state of the store. See [`LightningStore.wait_for_rollouts()`](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.LightningStore.wait_for_rollouts " wait_for_rollouts(*, rollout_ids, timeout=None) async ") for semantics. Client-Server and Thread-safe Wrappers[¶](https://microsoft.github.io/agent-lightning/latest/reference/store/#client-server-and-thread-safe-wrappers "Permanent link") ----------------------------------------------------------------------------------------------------------------------------------------------------------------------- ### `agentlightning.LightningStoreServer` [¶](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.LightningStoreServer "Permanent link") Bases: `[LightningStore](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.LightningStore " agentlightning.LightningStore (agentlightning.store.base.LightningStore)") ` Server wrapper that exposes a LightningStore via HTTP API. Delegates all operations to an underlying store implementation. Healthcheck and watchdog relies on the underlying store. `agl store` is a convenient CLI to start a store server. When the server is executed in a subprocess, the store will discover itself having a different PID and automatically delegate to an HTTP client instead of using the local store. This ensures one single copy of the store will be shared across all processes. This server exporting OTLP-compatible traces via the `/v1/traces` endpoint. Parameters: * **`store`** (`[LightningStore](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.LightningStore " agentlightning.LightningStore (agentlightning.store.base.LightningStore)") `) – The underlying store to delegate operations to. * **`host`** (`str | None`, default: `None` ) – The hostname or IP address to bind the server to. * **`port`** (`int | None`, default: `None` ) – The TCP port to listen on. * **`cors_allow_origins`** (`Sequence[str] | str | None`, default: `None` ) – A list of CORS origins to allow. Use '\*' to allow all origins. * **`launch_mode`** (`[LaunchMode](https://microsoft.github.io/agent-lightning/latest/reference/utilities/#agentlightning.utils.server_launcher.LaunchMode " agentlightning.utils.server_launcher.LaunchMode = Literal['asyncio', 'thread', 'mp'] module-attribute (agentlightning.utils.server_launcher.LaunchMode)") `, default: `'thread'` ) – The launch mode to use for the server. Defaults to "thread", which runs the server in a separate thread. * **`launcher_args`** (`[PythonServerLauncherArgs](https://microsoft.github.io/agent-lightning/latest/reference/utilities/#agentlightning.utils.server_launcher.PythonServerLauncherArgs " agentlightning.utils.server_launcher.PythonServerLauncherArgs dataclass ") | None`, default: `None` ) – The arguments to use for the server launcher. It's not allowed to set `host`, `port`, `launch_mode` together with `launcher_args`. * **`n_workers`** (`int`, default: `1` ) – The number of workers to run in the server. Only applicable for `mp` launch mode. * **`tracker`** (`[MetricsBackend](https://microsoft.github.io/agent-lightning/latest/reference/utilities/#agentlightning.utils.metrics.MetricsBackend " agentlightning.utils.metrics.MetricsBackend") | None`, default: `None` ) – The metrics tracker to use for the server. #### `capabilities` `property` [¶](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.LightningStoreServer.capabilities "Permanent link") Return the capabilities of the store. #### `endpoint` `property` [¶](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.LightningStoreServer.endpoint "Permanent link") Endpoint is the address that the client will use to connect to the server. #### `__getstate__()` [¶](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.LightningStoreServer.__getstate__ "Permanent link") Control pickling to prevent server state from being sent to subprocesses. When LightningStoreServer is pickled (e.g., passed to a subprocess), we only serialize the underlying store and connection details. The client instance and process-awareness state are excluded as they should not be transferred between processes. The subprocess should create its own server instance if needed. #### `__setstate__(state)` [¶](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.LightningStoreServer.__setstate__ "Permanent link") Restore from pickle by reconstructing only the essential attributes. Note: This creates a new server instance without FastAPI/uvicorn initialized. Call **init**() pattern or create a new LightningStoreServer if you need a fully functional server in the subprocess. The unpickled server will also have no app and store attributes, this is to make sure there is only one copy of the server in the whole system. #### `otlp_traces_endpoint()` [¶](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.LightningStoreServer.otlp_traces_endpoint "Permanent link") Return the OTLP/HTTP traces endpoint of the store. #### `run_forever()` `async` [¶](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.LightningStoreServer.run_forever "Permanent link") Runs the FastAPI server indefinitely. #### `start()` `async` [¶](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.LightningStoreServer.start "Permanent link") Starts the FastAPI server in the background. You need to call this method in the same process as the server was created in. #### `stop()` `async` [¶](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.LightningStoreServer.stop "Permanent link") Gracefully stops the running FastAPI server. You need to call this method in the same process as the server was created in. ### `agentlightning.LightningStoreClient` [¶](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.LightningStoreClient "Permanent link") Bases: `[LightningStore](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.LightningStore " agentlightning.LightningStore (agentlightning.store.base.LightningStore)") ` HTTP client that talks to a remote LightningStoreServer. Parameters: * **`server_address`** (`str`) – The address of the LightningStoreServer to connect to. * **`retry_delays`** (`Sequence[float]`, default: `(1.0, 2.0, 5.0)` ) – Backoff schedule (seconds) used when the initial request fails for a non-application reason. Each entry is a retry attempt. Setting to an empty sequence to disable retries. * **`health_retry_delays`** (`Sequence[float]`, default: `(0.1, 0.2, 0.5)` ) – Delays between /health probes while waiting for the server to come back. Setting to an empty sequence to disable health checks. * **`request_timeout`** (`float`, default: `30.0` ) – Timeout (seconds) for each request. * **`connection_timeout`** (`float`, default: `5.0` ) – Timeout (seconds) for establishing connection. #### `capabilities` `property` [¶](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.LightningStoreClient.capabilities "Permanent link") Return the capabilities of the store. #### `__getstate__()` [¶](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.LightningStoreClient.__getstate__ "Permanent link") When LightningStoreClient is pickled (e.g., passed to a subprocess), we only serialize the server address and retry configurations. The ClientSessions are excluded as they should not be transferred between processes. #### `__setstate__(state)` [¶](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.LightningStoreClient.__setstate__ "Permanent link") Restore from pickle by reconstructing only the essential attributes. Replicating `__init__` logic to create another client instance in the subprocess. #### `close()` `async` [¶](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.LightningStoreClient.close "Permanent link") Close the HTTP session. #### `dequeue_rollout(worker_id=None)` `async` [¶](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.LightningStoreClient.dequeue_rollout "Permanent link") Dequeue a rollout from the server queue. Returns: * `Optional[[AttemptedRollout](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.AttemptedRollout " agentlightning.AttemptedRollout (agentlightning.types.AttemptedRollout)") ]` – AttemptedRollout if a rollout is available, None if queue is empty. Note This method does NOT retry on failures. If any exception occurs (network error, server error, etc.), it logs the error and returns None immediately. #### `get_latest_attempt(rollout_id)` `async` [¶](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.LightningStoreClient.get_latest_attempt "Permanent link") Get the latest attempt for a rollout. Parameters: * **`rollout_id`** (`str`) – ID of the rollout to query. Returns: * `Optional[[Attempt](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.Attempt " agentlightning.Attempt (agentlightning.types.Attempt)") ]` – Attempt if found, None if not found or if all retries are exhausted. Note This method retries on transient failures (network errors, 5xx status codes). If all retries fail, it logs the error and returns None instead of raising an exception. #### `get_latest_resources()` `async` [¶](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.LightningStoreClient.get_latest_resources "Permanent link") Get the latest resources. Returns: * `Optional[[ResourcesUpdate](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.ResourcesUpdate " agentlightning.ResourcesUpdate (agentlightning.types.ResourcesUpdate)") ]` – ResourcesUpdate if found, None if not found or if all retries are exhausted. Note This method retries on transient failures (network errors, 5xx status codes). If all retries fail, it logs the error and returns None instead of raising an exception. #### `get_resources_by_id(resources_id)` `async` [¶](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.LightningStoreClient.get_resources_by_id "Permanent link") Get resources by their ID. Parameters: * **`resources_id`** (`str`) – ID of the resources to retrieve. Returns: * `Optional[[ResourcesUpdate](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.ResourcesUpdate " agentlightning.ResourcesUpdate (agentlightning.types.ResourcesUpdate)") ]` – ResourcesUpdate if found, None if not found or if all retries are exhausted. Note This method retries on transient failures (network errors, 5xx status codes). If all retries fail, it logs the error and returns None instead of raising an exception. #### `get_rollout_by_id(rollout_id)` `async` [¶](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.LightningStoreClient.get_rollout_by_id "Permanent link") Get a rollout by its ID. Parameters: * **`rollout_id`** (`str`) – ID of the rollout to retrieve. Returns: * `Optional[[Rollout](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.Rollout " agentlightning.Rollout (agentlightning.types.Rollout)") ]` – Rollout if found, None if not found or if all retries are exhausted. Note This method retries on transient failures (network errors, 5xx status codes). If all retries fail, it logs the error and returns None instead of raising an exception. #### `otlp_traces_endpoint()` [¶](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.LightningStoreClient.otlp_traces_endpoint "Permanent link") Return the OTLP/HTTP traces endpoint of the store. #### `query_resources(*, resources_id=None, resources_id_contains=None, sort_by=None, sort_order='asc', limit=-1, offset=0)` `async` [¶](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.LightningStoreClient.query_resources "Permanent link") List all resource snapshots stored on the server. #### `wait_for_rollouts(*, rollout_ids, timeout=None)` `async` [¶](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.LightningStoreClient.wait_for_rollouts "Permanent link") Wait for rollouts to complete. Parameters: * **`rollout_ids`** (`List[str]`) – List of rollout IDs to wait for. * **`timeout`** (`Optional[float]`, default: `None` ) – Timeout in seconds. If not None, the method will raise a ValueError if the timeout is greater than 0.1 seconds. Returns: * `List[[Rollout](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.Rollout " agentlightning.Rollout (agentlightning.types.Rollout)") ]` – List of rollouts that are completed. ### `agentlightning.LightningStoreThreaded` [¶](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.LightningStoreThreaded "Permanent link") Bases: `[LightningStore](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.LightningStore " agentlightning.LightningStore (agentlightning.store.base.LightningStore)") ` Facade that delegates all store operations to a underlying store instance. The operations are guaranteed to be thread-safe. Make sure the threaded stores are instantiated before initializing the threads. #### `capabilities` `property` [¶](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.LightningStoreThreaded.capabilities "Permanent link") Return the capabilities of the store. #### `statistics()` `async` [¶](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.LightningStoreThreaded.statistics "Permanent link") Return the statistics of the store. Collections and Collection Implementations[¶](https://microsoft.github.io/agent-lightning/latest/reference/store/#collections-and-collection-implementations "Permanent link") ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- ### `agentlightning.store.collection.AtomicMode = Literal['r', 'w', 'rw']` `module-attribute` [¶](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.store.collection.AtomicMode "Permanent link") What is expected within the atomic context. Can be "read", "write", or "read-write". ### `agentlightning.store.collection.AtomicLabels = Literal['rollouts', 'attempts', 'spans', 'resources', 'workers', 'rollout_queue', 'span_sequence_ids', 'generic']` `module-attribute` [¶](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.store.collection.AtomicLabels "Permanent link") Labels for atomic operations. These labels are used to identify the collections that are affected by the atomic operation. The `generic` label is used to identify atomic operations that are not associated with any specific collection. ### `agentlightning.store.collection.Collection` [¶](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.store.collection.Collection "Permanent link") Bases: `TrackedCollection`, `Generic[T]` Standard collection interface. Behaves like a list of items. Supporting addition, updating, and deletion of items. #### `delete(items)` `async` [¶](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.store.collection.Collection.delete "Permanent link") Delete the given items from the collection. Parameters: * **`items`** (`Sequence[T]`) – The items to delete from the collection. Raises: * `ValueError` – If the items with the primary keys to be deleted do not exist. #### `get(filter=None, sort=None)` `async` [¶](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.store.collection.Collection.get "Permanent link") Get the first item that matches the given filters. Parameters: * **`filter`** (`Optional[[FilterOptions](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.FilterOptions " agentlightning.FilterOptions = Mapping[Union[str, Literal['_aggregate', '_must']], Union[FilterField, Literal['and', 'or'], Mapping[str, FilterField]]] module-attribute (agentlightning.types.FilterOptions)") ]`, default: `None` ) – The filters to apply to the collection. See [`FilterOptions`](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.FilterOptions " agentlightning.FilterOptions = Mapping[Union[str, Literal['_aggregate', '_must']], Union[FilterField, Literal['and', 'or'], Mapping[str, FilterField]]] module-attribute ") . * **`sort`** (`Optional[[SortOptions](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.SortOptions " agentlightning.SortOptions (agentlightning.types.SortOptions)") ]`, default: `None` ) – Sort options. See [`SortOptions`](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.SortOptions " agentlightning.SortOptions") . Returns: * `Optional[T]` – The first item that matches the given filters, or None if no item matches. #### `insert(items)` `async` [¶](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.store.collection.Collection.insert "Permanent link") Add the given items to the collection. Raises: * `ValueError` – If an item with the same primary key already exists. #### `item_type()` [¶](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.store.collection.Collection.item_type "Permanent link") Get the type of the items in the collection. #### `primary_keys()` [¶](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.store.collection.Collection.primary_keys "Permanent link") Get the primary keys of the collection. #### `query(filter=None, sort=None, limit=-1, offset=0)` `async` [¶](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.store.collection.Collection.query "Permanent link") Query the collection with the given filters, sort order, and pagination. Parameters: * **`filter`** (`Optional[[FilterOptions](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.FilterOptions " agentlightning.FilterOptions = Mapping[Union[str, Literal['_aggregate', '_must']], Union[FilterField, Literal['and', 'or'], Mapping[str, FilterField]]] module-attribute (agentlightning.types.FilterOptions)") ]`, default: `None` ) – The filters to apply to the collection. See [`FilterOptions`](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.FilterOptions " agentlightning.FilterOptions = Mapping[Union[str, Literal['_aggregate', '_must']], Union[FilterField, Literal['and', 'or'], Mapping[str, FilterField]]] module-attribute ") . * **`sort`** (`Optional[[SortOptions](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.SortOptions " agentlightning.SortOptions (agentlightning.types.SortOptions)") ]`, default: `None` ) – The options for sorting the collection. See [`SortOptions`](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.SortOptions " agentlightning.SortOptions") . The field must exist in the model. If field might contain null values, in which case the behavior is undefined (i.e., depending on the implementation). * **`limit`** (`int`, default: `-1` ) – Max number of items to return. Use -1 for "no limit". * **`offset`** (`int`, default: `0` ) – Number of items to skip from the start of the _matching_ items. Returns: * `[PaginatedResult](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.PaginatedResult " agentlightning.PaginatedResult (agentlightning.types.PaginatedResult)") [T]` – PaginatedResult with items, limit, offset, and total matched items. #### `size()` `async` [¶](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.store.collection.Collection.size "Permanent link") Get the number of items in the collection. #### `update(items, update_fields=None)` `async` [¶](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.store.collection.Collection.update "Permanent link") Update the given items in the collection. Parameters: * **`items`** (`Sequence[T]`) – The items to update in the collection. * **`update_fields`** (`Sequence[str] | None`, default: `None` ) – The fields to update. If not provided, all fields in the type will be updated. Only applicable if the item type is a Pydantic BaseModel. Raises: * `ValueError` – If an item with the primary keys does not exist. Returns: * `Sequence[T]` – The items that were updated. #### `upsert(items, update_fields=None)` `async` [¶](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.store.collection.Collection.upsert "Permanent link") Upsert the given items into the collection. If the items with the same primary keys already exist, they will be updated. Otherwise, they will be inserted. The operation has three semantics configurable via `update_fields`: * `update_or_insert` via `collection.upsert(items, update_fields=["status", "updated_at"])`. If the item with the same primary keys already exists, only the specified fields will be updated. Otherwise, the item will be inserted. * `get_or_insert` via `collection.upsert(items, update_fields=[])`. If the item with the same primary keys already exists, the item will be left unchanged. Otherwise, the item will be inserted. * `replace_ish` via `collection.upsert(items)`. If the item with the same primary keys already exists, all fields from the item will be set. Otherwise, the item will be inserted. Returns: * `Sequence[T]` – The items that were upserted. ### `agentlightning.store.collection.Queue` [¶](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.store.collection.Queue "Permanent link") Bases: `TrackedCollection`, `Generic[T]` Behaves like a deque. Supporting appending items to the end and popping items from the front. #### `dequeue(limit=1)` `async` [¶](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.store.collection.Queue.dequeue "Permanent link") Pop the given number of items from the front of the queue. Parameters: * **`limit`** (`int`, default: `1` ) – The number of items to pop from the front of the queue. Returns: * `Sequence[T]` – The items that were popped from the front of the queue. * `Sequence[T]` – If there are less than `limit` items in the queue, the remaining items will be returned. #### `enqueue(items)` `async` [¶](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.store.collection.Queue.enqueue "Permanent link") Append the given items to the end of the queue. Parameters: * **`items`** (`Sequence[T]`) – The items to append to the end of the queue. Returns: * `Sequence[T]` – The items that were appended to the end of the queue. #### `has(item)` `async` [¶](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.store.collection.Queue.has "Permanent link") Check if the given item is in the queue. #### `item_type()` [¶](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.store.collection.Queue.item_type "Permanent link") Get the type of the items in the queue. #### `peek(limit=1)` `async` [¶](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.store.collection.Queue.peek "Permanent link") Peek the given number of items from the front of the queue. Parameters: * **`limit`** (`int`, default: `1` ) – The number of items to peek from the front of the queue. Returns: * `Sequence[T]` – The items that were peeked from the front of the queue. * `Sequence[T]` – If there are less than `limit` items in the queue, the remaining items will be returned. #### `size()` `async` [¶](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.store.collection.Queue.size "Permanent link") Get the number of items in the queue. ### `agentlightning.store.collection.KeyValue` [¶](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.store.collection.KeyValue "Permanent link") Bases: `TrackedCollection`, `Generic[K, V]` Behaves like a dictionary. Supporting addition, updating, and deletion of items. #### `chmax(key, value)` `async` [¶](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.store.collection.KeyValue.chmax "Permanent link") Set the value for the given key to the maximum of the current and new value. Raises: * `TypeError` – If the existing value or `value` is not numeric. #### `get(key, default=None)` `async` [¶](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.store.collection.KeyValue.get "Permanent link") Get the value for the given key, or the default value if the key is not found. #### `has(key)` `async` [¶](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.store.collection.KeyValue.has "Permanent link") Check if the given key is in the dictionary. #### `inc(key, amount)` `async` [¶](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.store.collection.KeyValue.inc "Permanent link") Increase the numeric value for the given key by `amount` and return the new value. Raises: * `TypeError` – If the existing value or `amount` is not numeric. #### `pop(key, default=None)` `async` [¶](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.store.collection.KeyValue.pop "Permanent link") Pop the value for the given key, or the default value if the key is not found. #### `set(key, value)` `async` [¶](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.store.collection.KeyValue.set "Permanent link") Set the value for the given key. #### `size()` `async` [¶](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.store.collection.KeyValue.size "Permanent link") Get the number of items in the dictionary. ### `agentlightning.store.collection.LightningCollections` [¶](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.store.collection.LightningCollections "Permanent link") Bases: `TrackedCollection` Collections of rollouts, attempts, spans, resources, and workers. [LightningStore](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.LightningStore " agentlightning.LightningStore") implementations can use this as a storage base to implement the store API. #### `attempts` `property` [¶](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.store.collection.LightningCollections.attempts "Permanent link") Collections of attempts. #### `resources` `property` [¶](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.store.collection.LightningCollections.resources "Permanent link") Collections of resources. #### `rollout_queue` `property` [¶](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.store.collection.LightningCollections.rollout_queue "Permanent link") Queue of rollouts (tasks). #### `rollouts` `property` [¶](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.store.collection.LightningCollections.rollouts "Permanent link") Collections of rollouts. #### `span_sequence_ids` `property` [¶](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.store.collection.LightningCollections.span_sequence_ids "Permanent link") Dictionary (counter) of span sequence IDs. #### `spans` `property` [¶](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.store.collection.LightningCollections.spans "Permanent link") Collections of spans. #### `workers` `property` [¶](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.store.collection.LightningCollections.workers "Permanent link") Collections of workers. #### `atomic(*, mode='rw', snapshot=False, commit=False, labels=None, **kwargs)` [¶](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.store.collection.LightningCollections.atomic "Permanent link") Perform a atomic operation on the collections. Subclass may use args and kwargs to support multiple levels of atomicity. The arguments can be seen as tags. They only imply the behavior of the operation, not the implementation. Parameters: * **`mode`** (`[AtomicMode](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.store.collection.AtomicMode " agentlightning.store.collection.AtomicMode = Literal['r', 'w', 'rw'] module-attribute (agentlightning.store.collection.base.AtomicMode)") `, default: `'rw'` ) – The mode of atomicity. See [`AtomicMode`](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.store.collection.AtomicMode " agentlightning.store.collection.AtomicMode = Literal['r', 'w', 'rw'] module-attribute ") . * **`snapshot`** (`bool`, default: `False` ) – Enable read snapshot for repeatable reads. Data consistency is guaranteed. The real behavior is implementation-dependent. * **`commit`** (`bool`, default: `False` ) – Enable commitment for write operations. Unsuccessful operations will be rolled back depending on the implementation. Recommend to use [`execute()`](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.store.collection.LightningCollections.execute " execute(callback, *, mode='rw', snapshot=False, commit=False, labels=None, **kwargs) async ") for this level to enable automatic retries. Remember that the real behavior is implementation-dependent. * **`labels`** (`Optional[Sequence[[AtomicLabels](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.store.collection.AtomicLabels " agentlightning.store.collection.AtomicLabels = Literal['rollouts', 'attempts', 'spans', 'resources', 'workers', 'rollout_queue', 'span_sequence_ids', 'generic'] module-attribute (agentlightning.store.collection.base.AtomicLabels)") ]]`, default: `None` ) – Labels to add to the atomic operation (commonly used as lock names or collection names). * **`**kwargs`** (`Any`, default: `{}` ) – Keyword arguments to pass to the operation. #### `execute(callback, *, mode='rw', snapshot=False, commit=False, labels=None, **kwargs)` `async` [¶](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.store.collection.LightningCollections.execute "Permanent link") Execute the given callback within an atomic operation. Retry on transient errors is implied. See [`atomic()`](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.store.collection.LightningCollections.atomic " atomic(*, mode='rw', snapshot=False, commit=False, labels=None, **kwargs)") for more details. ### `agentlightning.store.collection.ListBasedCollection` [¶](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.store.collection.ListBasedCollection "Permanent link") Bases: `[Collection](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.store.collection.Collection " agentlightning.store.collection.Collection (agentlightning.store.collection.base.Collection)") [T]` In-memory implementation of Collection using a nested dict for O(1) primary-key lookup. The internal structure is: `{ pk1_value: { pk2_value: { ... pkN_value: item } } }` where the nesting depth equals the number of primary keys. Sorting behavior: 1. If no sort\_by is provided, the items are returned in the order of insertion. 2. If sort\_by is provided, the items are sorted by the value of the sort\_by field. 3. If the sort\_by field is a timestamp, the null values are treated as infinity. 4. If the sort\_by field is not a timestamp, the null values are treated as empty string if the field is str-like, 0 if the field is int-like, 0.0 if the field is float-like. #### `delete(items)` `async` [¶](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.store.collection.ListBasedCollection.delete "Permanent link") Delete the given items. Raises: * `ValueError` – If any item with the given primary keys does not exist. #### `get(filter=None, sort=None)` `async` [¶](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.store.collection.ListBasedCollection.get "Permanent link") Return the first (or best-sorted) item that matches the given filters, or None. #### `insert(items)` `async` [¶](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.store.collection.ListBasedCollection.insert "Permanent link") Insert the given items. Raises: * `DuplicatedPrimaryKeyError` – If any item with the same primary keys already exists. #### `item_type()` [¶](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.store.collection.ListBasedCollection.item_type "Permanent link") Return the Pydantic model type of items stored in this collection. #### `primary_keys()` [¶](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.store.collection.ListBasedCollection.primary_keys "Permanent link") Return the primary key field names for this collection. #### `query(filter=None, sort=None, limit=-1, offset=0)` `async` [¶](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.store.collection.ListBasedCollection.query "Permanent link") Query the collection with filters, sort order, and pagination. Parameters: * **`filter`** (`Optional[[FilterOptions](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.FilterOptions " agentlightning.FilterOptions = Mapping[Union[str, Literal['_aggregate', '_must']], Union[FilterField, Literal['and', 'or'], Mapping[str, FilterField]]] module-attribute (agentlightning.types.FilterOptions)") ]`, default: `None` ) – Mapping of field name to operator dict along with the optional `_aggregate` logic. * **`sort`** (`Optional[[SortOptions](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.SortOptions " agentlightning.SortOptions (agentlightning.types.SortOptions)") ]`, default: `None` ) – Options describing which field to sort by and in which order. * **`limit`** (`int`, default: `-1` ) – Max number of items to return. Use -1 for "no limit". * **`offset`** (`int`, default: `0` ) – Number of items to skip from the start of the _matching_ items. #### `size()` `async` [¶](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.store.collection.ListBasedCollection.size "Permanent link") Return the number of items stored in the collection. #### `update(items, update_fields=None)` `async` [¶](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.store.collection.ListBasedCollection.update "Permanent link") Update the given items. Raises: * `ValueError` – If any item with the given primary keys does not exist. #### `upsert(items, update_fields=None)` `async` [¶](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.store.collection.ListBasedCollection.upsert "Permanent link") Upsert the given items (insert if missing, otherwise update). ### `agentlightning.store.collection.DequeBasedQueue` [¶](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.store.collection.DequeBasedQueue "Permanent link") Bases: `[Queue](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.store.collection.Queue " agentlightning.store.collection.Queue (agentlightning.store.collection.base.Queue)") [T]` Queue implementation backed by collections.deque. Provides O(1) amortized enqueue (append) and dequeue (popleft). ### `agentlightning.store.collection.DictBasedKeyValue` [¶](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.store.collection.DictBasedKeyValue "Permanent link") Bases: `[KeyValue](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.store.collection.KeyValue " agentlightning.store.collection.KeyValue (agentlightning.store.collection.base.KeyValue)") [K, V]` KeyValue implementation backed by a plain dictionary. ### `agentlightning.store.collection.InMemoryLightningCollections` [¶](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.store.collection.InMemoryLightningCollections "Permanent link") Bases: `[LightningCollections](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.store.collection.LightningCollections " agentlightning.store.collection.LightningCollections (agentlightning.store.collection.base.LightningCollections)") ` In-memory implementation of LightningCollections using Python data structures. Serves as the storage base for [`InMemoryLightningStore`](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.InMemoryLightningStore " agentlightning.InMemoryLightningStore") . #### `atomic(*, mode='rw', snapshot=False, labels=None, **kwargs)` `async` [¶](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.store.collection.InMemoryLightningCollections.atomic "Permanent link") In-memory collections apply a lock outside. It doesn't need to manipulate the collections inside. Skip the locking if mode is "r" and snapshot is False. This collection implementation does NOT support rollback / commit. #### `evict_spans_for_rollout(rollout_id)` `async` [¶](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.store.collection.InMemoryLightningCollections.evict_spans_for_rollout "Permanent link") Evict all spans for a given rollout ID. Uses private API for efficiency. ### `agentlightning.store.collection.mongo.MongoBasedCollection` [¶](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.store.collection.mongo.MongoBasedCollection "Permanent link") Bases: `[Collection](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.store.collection.Collection " agentlightning.store.collection.Collection (agentlightning.store.collection.base.Collection)") [T_model]` Mongo-based implementation of Collection. Parameters: * **`client_pool`** (`[MongoClientPool](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.store.collection.mongo.MongoClientPool " agentlightning.store.collection.mongo.MongoClientPool") [Mapping[str, Any]]`) – The pool of MongoDB clients. * **`database_name`** (`str`) – The name of the database. * **`collection_name`** (`str`) – The name of the collection. * **`partition_id`** (`str`) – The partition ID. Used to partition the collection into multiple collections. * **`primary_keys`** (`Sequence[str]`) – The primary keys of the collection. * **`item_type`** (`Type[T_model]`) – The type of the items in the collection. * **`extra_indexes`** (`Sequence[Sequence[str]]`, default: `[]` ) – The extra indexes to create on the collection. * **`tracker`** (`[MetricsBackend](https://microsoft.github.io/agent-lightning/latest/reference/utilities/#agentlightning.utils.metrics.MetricsBackend " agentlightning.utils.metrics.MetricsBackend") | None`, default: `None` ) – The metrics tracker to use. #### `ensure_collection()` `async` [¶](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.store.collection.mongo.MongoBasedCollection.ensure_collection "Permanent link") Ensure the backing MongoDB collection exists (and optionally its indexes). This method is idempotent and safe to call multiple times. It will also create a unique index across the configured primary key fields. #### `insert(items)` `async` [¶](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.store.collection.mongo.MongoBasedCollection.insert "Permanent link") Insert items into the collection. The implementation does NOT do checks for duplicate primary keys, neither within the same insert call nor across different insert calls. It relies on the database to enforce uniqueness via indexes. #### `primary_keys()` [¶](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.store.collection.mongo.MongoBasedCollection.primary_keys "Permanent link") Return the primary key field names for this collection. #### `query(filter=None, sort=None, limit=-1, offset=0)` `async` [¶](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.store.collection.mongo.MongoBasedCollection.query "Permanent link") Mongo-based implementation of Collection.query. The handling of null-values in sorting is different from memory-based implementation. In MongoDB, null values are treated as less than non-null values. #### `with_session(session)` [¶](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.store.collection.mongo.MongoBasedCollection.with_session "Permanent link") Create a new collection with the same configuration but a new session. ### `agentlightning.store.collection.mongo.MongoBasedQueue` [¶](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.store.collection.mongo.MongoBasedQueue "Permanent link") Bases: `[Queue](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.store.collection.Queue " agentlightning.store.collection.Queue (agentlightning.store.collection.base.Queue)") [T_generic]`, `Generic[T_generic]` Mongo-based implementation of Queue backed by a MongoDB collection. Items are stored append-only; dequeue marks items as consumed instead of deleting them. #### `__init__(client_pool, database_name, collection_name, partition_id, item_type, tracker=None)` [¶](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.store.collection.mongo.MongoBasedQueue.__init__ "Permanent link") Parameters: * **`client_pool`** (`[MongoClientPool](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.store.collection.mongo.MongoClientPool " agentlightning.store.collection.mongo.MongoClientPool") [Mapping[str, Any]]`) – The pool of MongoDB clients. * **`database_name`** (`str`) – The name of the database. * **`collection_name`** (`str`) – The name of the collection backing the queue. * **`partition_id`** (`str`) – Partition identifier; allows multiple logical queues in one collection. * **`item_type`** (`Type[T_generic]`) – The Python type of queue items (primitive or BaseModel subclass). #### `ensure_collection()` `async` [¶](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.store.collection.mongo.MongoBasedQueue.ensure_collection "Permanent link") Ensure the backing collection exists. If it already exists, it returns the existing collection. ### `agentlightning.store.collection.mongo.MongoBasedKeyValue` [¶](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.store.collection.mongo.MongoBasedKeyValue "Permanent link") Bases: `[KeyValue](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.store.collection.KeyValue " agentlightning.store.collection.KeyValue (agentlightning.store.collection.base.KeyValue)") [K, V]`, `Generic[K, V]` Mongo-based implementation of KeyValue. #### `__init__(client_pool, database_name, collection_name, partition_id, key_type, value_type, tracker=None)` [¶](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.store.collection.mongo.MongoBasedKeyValue.__init__ "Permanent link") Parameters: * **`client_pool`** (`[MongoClientPool](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.store.collection.mongo.MongoClientPool " agentlightning.store.collection.mongo.MongoClientPool") [Mapping[str, Any]]`) – The pool of MongoDB clients. * **`database_name`** (`str`) – The name of the database. * **`collection_name`** (`str`) – The name of the collection backing the key-value store. * **`partition_id`** (`str`) – Partition identifier; allows multiple logical maps in one collection. * **`key_type`** (`Type[K]`) – The Python type of keys (primitive or BaseModel). * **`value_type`** (`Type[V]`) – The Python type of values (primitive or BaseModel). * **`tracker`** (`[MetricsBackend](https://microsoft.github.io/agent-lightning/latest/reference/utilities/#agentlightning.utils.metrics.MetricsBackend " agentlightning.utils.metrics.MetricsBackend") | None`, default: `None` ) – The metrics tracker to use. #### `ensure_collection(*, create_indexes=True)` `async` [¶](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.store.collection.mongo.MongoBasedKeyValue.ensure_collection "Permanent link") Ensure the backing collection exists (and optionally its indexes). ### `agentlightning.store.collection.mongo.MongoClientPool` [¶](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.store.collection.mongo.MongoClientPool "Permanent link") Bases: `Generic[T_mapping]` A pool of MongoDB clients, each bound to a specific event loop. The pool lazily creates `AsyncMongoClient` instances per event loop using the provided connection parameters, ensuring we never try to reuse a client across loops. #### `close()` `async` [¶](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.store.collection.mongo.MongoClientPool.close "Permanent link") Close all clients currently tracked by the pool. ### `agentlightning.store.collection.mongo.MongoLightningCollections` [¶](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.store.collection.mongo.MongoLightningCollections "Permanent link") Bases: `[LightningCollections](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.store.collection.LightningCollections " agentlightning.store.collection.LightningCollections (agentlightning.store.collection.base.LightningCollections)") ` Mongo implementation of LightningCollections using MongoDB collections. Serves as the storage base for [`MongoLightningStore`](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.store.mongo.MongoLightningStore " agentlightning.store.mongo.MongoLightningStore") . #### `atomic(mode='rw', snapshot=False, commit=False, labels=None, *args, **kwargs)` `async` [¶](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.store.collection.mongo.MongoLightningCollections.atomic "Permanent link") Perform a atomic operation on the collections. #### `execute(callback, *, mode='rw', snapshot=False, commit=False, labels=None, **kwargs)` `async` [¶](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.store.collection.mongo.MongoLightningCollections.execute "Permanent link") Execute the given callback within an atomic operation, and with retries on transient errors. Back to top --- # Command Line - Agent-lightning [Skip to content](https://microsoft.github.io/agent-lightning/latest/reference/cli/#command-line-interface) ⚠️ You are viewing development documentation. For stable documentation, please visit [https://microsoft.github.io/agent-lightning/stable/](https://microsoft.github.io/agent-lightning/stable/) Command Line Interface[¶](https://microsoft.github.io/agent-lightning/latest/reference/cli/#command-line-interface "Permanent link") ===================================================================================================================================== Warning This document is a work in progress and might not be updated with the latest changes. Try to use `agl -h` to get the latest help message. Tip Agent-lightning also provides utilities to help you build your own CLI for [LitAgent](https://microsoft.github.io/agent-lightning/latest/reference/agent/#agentlightning.LitAgent " agentlightning.LitAgent") and [Trainer](https://microsoft.github.io/agent-lightning/latest/reference/trainer/#agentlightning.Trainer " agentlightning.Trainer") . See [Trainer](https://microsoft.github.io/agent-lightning/latest/reference/trainer/) for references. agl[¶](https://microsoft.github.io/agent-lightning/latest/reference/cli/#agl "Permanent link") ----------------------------------------------------------------------------------------------- `[](https://microsoft.github.io/agent-lightning/latest/reference/cli/#__codelineno-0-1) usage: agl [-h] {vllm,store,prometheus,agentops} [](https://microsoft.github.io/agent-lightning/latest/reference/cli/#__codelineno-0-2) [](https://microsoft.github.io/agent-lightning/latest/reference/cli/#__codelineno-0-3) Agent Lightning CLI entry point. [](https://microsoft.github.io/agent-lightning/latest/reference/cli/#__codelineno-0-4) [](https://microsoft.github.io/agent-lightning/latest/reference/cli/#__codelineno-0-5) Available subcommands: [](https://microsoft.github.io/agent-lightning/latest/reference/cli/#__codelineno-0-6) vllm Run the vLLM CLI with Agent Lightning instrumentation. [](https://microsoft.github.io/agent-lightning/latest/reference/cli/#__codelineno-0-7) store Run a LightningStore server. [](https://microsoft.github.io/agent-lightning/latest/reference/cli/#__codelineno-0-8) prometheus Serve Prometheus metrics from the multiprocess registry. [](https://microsoft.github.io/agent-lightning/latest/reference/cli/#__codelineno-0-9) agentops Start the AgentOps server manager. [](https://microsoft.github.io/agent-lightning/latest/reference/cli/#__codelineno-0-10) [](https://microsoft.github.io/agent-lightning/latest/reference/cli/#__codelineno-0-11) positional arguments: [](https://microsoft.github.io/agent-lightning/latest/reference/cli/#__codelineno-0-12) {vllm,store,prometheus,agentops} [](https://microsoft.github.io/agent-lightning/latest/reference/cli/#__codelineno-0-13) Subcommand to run. [](https://microsoft.github.io/agent-lightning/latest/reference/cli/#__codelineno-0-14) [](https://microsoft.github.io/agent-lightning/latest/reference/cli/#__codelineno-0-15) options: [](https://microsoft.github.io/agent-lightning/latest/reference/cli/#__codelineno-0-16) -h, --help show this help message and exit` agl vllm[¶](https://microsoft.github.io/agent-lightning/latest/reference/cli/#agl-vllm "Permanent link") --------------------------------------------------------------------------------------------------------- Agent-lightning's instrumented vLLM CLI. `[](https://microsoft.github.io/agent-lightning/latest/reference/cli/#__codelineno-1-1) usage: agl vllm [-h] [-v] {chat,complete,serve,bench,collect-env,run-batch} ... [](https://microsoft.github.io/agent-lightning/latest/reference/cli/#__codelineno-1-2) [](https://microsoft.github.io/agent-lightning/latest/reference/cli/#__codelineno-1-3) vLLM CLI [](https://microsoft.github.io/agent-lightning/latest/reference/cli/#__codelineno-1-4) [](https://microsoft.github.io/agent-lightning/latest/reference/cli/#__codelineno-1-5) positional arguments: [](https://microsoft.github.io/agent-lightning/latest/reference/cli/#__codelineno-1-6) {chat,complete,serve,bench,collect-env,run-batch} [](https://microsoft.github.io/agent-lightning/latest/reference/cli/#__codelineno-1-7) chat Generate chat completions via the running API server. [](https://microsoft.github.io/agent-lightning/latest/reference/cli/#__codelineno-1-8) complete Generate text completions based on the given prompt via the running API server. [](https://microsoft.github.io/agent-lightning/latest/reference/cli/#__codelineno-1-9) collect-env Start collecting environment information. [](https://microsoft.github.io/agent-lightning/latest/reference/cli/#__codelineno-1-10) run-batch Run batch prompts and write results to file. [](https://microsoft.github.io/agent-lightning/latest/reference/cli/#__codelineno-1-11) [](https://microsoft.github.io/agent-lightning/latest/reference/cli/#__codelineno-1-12) options: [](https://microsoft.github.io/agent-lightning/latest/reference/cli/#__codelineno-1-13) -h, --help show this help message and exit [](https://microsoft.github.io/agent-lightning/latest/reference/cli/#__codelineno-1-14) -v, --version show program's version number and exit [](https://microsoft.github.io/agent-lightning/latest/reference/cli/#__codelineno-1-15) [](https://microsoft.github.io/agent-lightning/latest/reference/cli/#__codelineno-1-16) For full list: vllm [subcommand] --help=all [](https://microsoft.github.io/agent-lightning/latest/reference/cli/#__codelineno-1-17) For a section: vllm [subcommand] --help=ModelConfig (case-insensitive) [](https://microsoft.github.io/agent-lightning/latest/reference/cli/#__codelineno-1-18) For a flag: vllm [subcommand] --help=max-model-len (_ or - accepted) [](https://microsoft.github.io/agent-lightning/latest/reference/cli/#__codelineno-1-19) Documentation: https://docs.vllm.ai` agl store[¶](https://microsoft.github.io/agent-lightning/latest/reference/cli/#agl-store "Permanent link") ----------------------------------------------------------------------------------------------------------- Agent-lightning's LightningStore CLI. Use it to start an independent LightningStore server. Currently the store data are stored in memory and will be lost when the server is stopped. ``[](https://microsoft.github.io/agent-lightning/latest/reference/cli/#__codelineno-2-1) usage: agl store [-h] [--host HOST] [--port PORT] [--cors-origin CORS_ORIGINS] [--log-level {DEBUG,INFO,WARNING,ERROR}] [--tracker {prometheus,console} [{prometheus,console} ...]] [--n-workers N_WORKERS] [--backend {memory,mongo}] [](https://microsoft.github.io/agent-lightning/latest/reference/cli/#__codelineno-2-2) [--mongo-uri MONGO_URI] [](https://microsoft.github.io/agent-lightning/latest/reference/cli/#__codelineno-2-3) [](https://microsoft.github.io/agent-lightning/latest/reference/cli/#__codelineno-2-4) Run a LightningStore server [](https://microsoft.github.io/agent-lightning/latest/reference/cli/#__codelineno-2-5) [](https://microsoft.github.io/agent-lightning/latest/reference/cli/#__codelineno-2-6) options: [](https://microsoft.github.io/agent-lightning/latest/reference/cli/#__codelineno-2-7) -h, --help show this help message and exit [](https://microsoft.github.io/agent-lightning/latest/reference/cli/#__codelineno-2-8) --host HOST Host to bind the server to [](https://microsoft.github.io/agent-lightning/latest/reference/cli/#__codelineno-2-9) --port PORT Port to run the server on [](https://microsoft.github.io/agent-lightning/latest/reference/cli/#__codelineno-2-10) --cors-origin CORS_ORIGINS [](https://microsoft.github.io/agent-lightning/latest/reference/cli/#__codelineno-2-11) Allowed CORS origin. Repeat for multiple origins. Use '*' to allow all origins. [](https://microsoft.github.io/agent-lightning/latest/reference/cli/#__codelineno-2-12) --log-level {DEBUG,INFO,WARNING,ERROR} [](https://microsoft.github.io/agent-lightning/latest/reference/cli/#__codelineno-2-13) Configure the logging level for the store. [](https://microsoft.github.io/agent-lightning/latest/reference/cli/#__codelineno-2-14) --tracker {prometheus,console} [{prometheus,console} ...] [](https://microsoft.github.io/agent-lightning/latest/reference/cli/#__codelineno-2-15) Enable metrics tracking. Repeat for multiple trackers. [](https://microsoft.github.io/agent-lightning/latest/reference/cli/#__codelineno-2-16) --n-workers N_WORKERS [](https://microsoft.github.io/agent-lightning/latest/reference/cli/#__codelineno-2-17) Number of workers to run in the server. When it's greater than 1, the server will be run using `mp` launch mode. Only applicable for zero-copy stores such as MongoDB backend. [](https://microsoft.github.io/agent-lightning/latest/reference/cli/#__codelineno-2-18) --backend {memory,mongo} [](https://microsoft.github.io/agent-lightning/latest/reference/cli/#__codelineno-2-19) Backend to use for the store. [](https://microsoft.github.io/agent-lightning/latest/reference/cli/#__codelineno-2-20) --mongo-uri MONGO_URI [](https://microsoft.github.io/agent-lightning/latest/reference/cli/#__codelineno-2-21) MongoDB URI to use for the store. Applicable only if --backend is 'mongo'.`` Tip After launching the store via CLI, you can tell the [`Trainer`](https://microsoft.github.io/agent-lightning/latest/reference/trainer/#agentlightning.Trainer " agentlightning.Trainer") to use the store by passing the store address to the trainer. `[](https://microsoft.github.io/agent-lightning/latest/reference/cli/#__codelineno-3-1) store_client = agl.LightningStoreClient("http://localhost:4747") [](https://microsoft.github.io/agent-lightning/latest/reference/cli/#__codelineno-3-2) trainer = agl.Trainer(store=store_client, ...)` See [using external store](https://microsoft.github.io/agent-lightning/latest/tutorials/debug/#debug-with-external-store "Debug the Algorithm-Runner Boundary") for more details. agl prometheus[¶](https://microsoft.github.io/agent-lightning/latest/reference/cli/#agl-prometheus "Permanent link") --------------------------------------------------------------------------------------------------------------------- Expose the Prometheus multiprocess registry on a dedicated FastAPI server. This is useful when the main LightningStore service is under heavy load; exporters can scrape this auxiliary endpoint instead. `[](https://microsoft.github.io/agent-lightning/latest/reference/cli/#__codelineno-4-1) usage: agl prometheus [-h] [--host HOST] [--port PORT] [--metrics-path METRICS_PATH] [--log-level {DEBUG,INFO,WARNING,ERROR}] [--access-log] [](https://microsoft.github.io/agent-lightning/latest/reference/cli/#__codelineno-4-2) [](https://microsoft.github.io/agent-lightning/latest/reference/cli/#__codelineno-4-3) Serve Prometheus metrics outside the LightningStore server. [](https://microsoft.github.io/agent-lightning/latest/reference/cli/#__codelineno-4-4) [](https://microsoft.github.io/agent-lightning/latest/reference/cli/#__codelineno-4-5) options: [](https://microsoft.github.io/agent-lightning/latest/reference/cli/#__codelineno-4-6) -h, --help show this help message and exit [](https://microsoft.github.io/agent-lightning/latest/reference/cli/#__codelineno-4-7) --host HOST Host to bind the metrics server to. [](https://microsoft.github.io/agent-lightning/latest/reference/cli/#__codelineno-4-8) --port PORT Port to expose the Prometheus metrics on. [](https://microsoft.github.io/agent-lightning/latest/reference/cli/#__codelineno-4-9) --metrics-path METRICS_PATH [](https://microsoft.github.io/agent-lightning/latest/reference/cli/#__codelineno-4-10) HTTP path used to expose metrics. Must start with '/' and not be the root path. [](https://microsoft.github.io/agent-lightning/latest/reference/cli/#__codelineno-4-11) --log-level {DEBUG,INFO,WARNING,ERROR} [](https://microsoft.github.io/agent-lightning/latest/reference/cli/#__codelineno-4-12) Configure the logging level for the metrics server. [](https://microsoft.github.io/agent-lightning/latest/reference/cli/#__codelineno-4-13) --access-log Enable uvicorn access logs. Disabled by default to reduce noise.` Back to top --- # Bird's Eye View - Agent-lightning [Skip to content](https://microsoft.github.io/agent-lightning/latest/deep-dive/birds-eye-view/#the-birds-eye-view-of-agent-lightning) ⚠️ You are viewing development documentation. For stable documentation, please visit [https://microsoft.github.io/agent-lightning/stable/](https://microsoft.github.io/agent-lightning/stable/) The Bird's Eye View of Agent-lightning[¶](https://microsoft.github.io/agent-lightning/latest/deep-dive/birds-eye-view/#the-birds-eye-view-of-agent-lightning "Permanent link") =============================================================================================================================================================================== High Volume of Information Ahead This article provides an in-depth exploration of the Agent-lightning architecture. It is not intended as a beginner’s guide or usage tutorial. This article summarizes how Agent-lightning (as of v0.2) wires the [Algorithm](https://microsoft.github.io/agent-lightning/latest/reference/algorithm/#agentlightning.Algorithm " agentlightning.Algorithm") , [Runner](https://microsoft.github.io/agent-lightning/latest/reference/runner/#agentlightning.Runner " agentlightning.Runner") , and [LightningStore](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.LightningStore " agentlightning.LightningStore") loop together and shows where auxiliary components (the [Tracer](https://microsoft.github.io/agent-lightning/latest/reference/runner/#agentlightning.Tracer " agentlightning.Tracer") , [Adapter](https://microsoft.github.io/agent-lightning/latest/reference/algorithm/#agentlightning.Adapter " agentlightning.Adapter") , and [LLM Proxy](https://microsoft.github.io/agent-lightning/latest/reference/algorithm/#agentlightning.LLMProxy " agentlightning.LLMProxy") ) plug into the loop. Each section provides a diagram for a different perspective of the system. Algorithm ↔ Runner ↔ Store data flow[¶](https://microsoft.github.io/agent-lightning/latest/deep-dive/birds-eye-view/#algorithm-runner-store-data-flow "Permanent link") ------------------------------------------------------------------------------------------------------------------------------------------------------------------------ At its heart, Agent-lightning is built on three main components that work in a coordinated loop: * **[Algorithm](https://microsoft.github.io/agent-lightning/latest/reference/algorithm/#agentlightning.Algorithm " agentlightning.Algorithm") :** The "brain" of the system. It decides what tasks to run, learns from the results, and updates resources (like AI models or prompts). * **[Runner](https://microsoft.github.io/agent-lightning/latest/reference/runner/#agentlightning.Runner " agentlightning.Runner") :** The "worker" of the system. It executes tasks assigned by the algorithm, runs the agent, and records the results. * **[LightningStore](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.LightningStore " agentlightning.LightningStore") :** The central "database" and message queue. It acts as the single source of truth, storing tasks, results, and resources, and enabling communication between the Algorithm and Runner. The typical data flow in a training loop is as follows: The **[Algorithm](https://microsoft.github.io/agent-lightning/latest/reference/algorithm/#agentlightning.Algorithm " agentlightning.Algorithm") ** enqueues tasks (called **[Rollouts](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.Rollout " agentlightning.Rollout") **) into the **[LightningStore](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.LightningStore " agentlightning.LightningStore") **. A **[Runner](https://microsoft.github.io/agent-lightning/latest/reference/runner/#agentlightning.Runner " agentlightning.Runner") ** then dequeues a task, executes it, and streams the results (called **[Spans](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.Span " agentlightning.Span") **) back to the store. Once the task is complete, the algorithm can query the new data from the store to learn and update its resources. The diagram below shows this fundamental interaction in a simple, non-parallel setup. sequenceDiagram autonumber participant Algo as Algorithm participant Store as LightningStore participant Runner participant Agent loop Over the dataset Algo-->>Store: add_resources + enqueue_rollout Store-->>Runner: dequeue_rollout → AttemptedRollout Store-->>Runner: get_latest_resources Runner-->>Store: update_attempt("running", worker_id) Runner->>Agent: rollout + resources Agent->>Runner: reward / spans Runner-->>Store: add_span or add_otel_span Runner-->>Store: update_attempt("finished", status) Store-->>Algo: query_rollouts + spans Algo-->>Algo: Update resources (optional) end Solid lines represent direct calls, while dashed lines are asynchronous or long-running operations. ### Key Terminology[¶](https://microsoft.github.io/agent-lightning/latest/deep-dive/birds-eye-view/#key-terminology "Permanent link") We define the following terms, which may be helpful for understanding the diagram above. * **[Resources](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.Resource " agentlightning.Resource") :** A collection of assets to be tuned or trained. Agents perform rollouts against resources and collect span data. Algorithms use those data to update the resources. In RL training, the resources are a tunable model. In prompt tuning, the resources are prompt templates. * **[Rollout](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.Rollout " agentlightning.Rollout") :** A unit of work that an agent performs against a resource. A rollout (noun) can be incomplete, in which case it is also known as a **task**, **sample**, or **job** (these terms are used interchangeably). The agent executes its own defined workflow against the rollout — the process is also called "to rollout" (verb). After execution, the rollout (noun) is considered _complete_. * **[Attempt](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.Attempt " agentlightning.Attempt") :** A single execution of a rollout. One rollout can have multiple attempts in case of failures or timeouts. * **[Span](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.Span " agentlightning.Span") :** During the rollout, the agent can generate multiple spans (also known as "traces" or "events"). The recorded spans are collected in the store, which is crucial for understanding agent behavior and optimizing agents. * **[Reward](https://microsoft.github.io/agent-lightning/latest/reference/agent/#agentlightning.emit_reward " agentlightning.emit_reward(reward, *, primary_key=None, attributes=None, propagate=True)") :** A special span that is defined as a number judging the quality of the rollout during some period of the rollout. * **[Dataset](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.Dataset " agentlightning.Dataset") :** A collection of incomplete rollouts (i.e., tasks) for the agent to process. The dual datasets (train, val) serve as the initial input for the algorithm to enqueue the first batch of rollouts. ### Store[¶](https://microsoft.github.io/agent-lightning/latest/deep-dive/birds-eye-view/#store "Permanent link") As discussed previously, the [LightningStore](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.LightningStore " agentlightning.LightningStore") is the central hub for all data in Agent-lightning. The store exposes a set of APIs for algorithms and runners to interact with the data; the most important ones are: `[](https://microsoft.github.io/agent-lightning/latest/deep-dive/birds-eye-view/#__codelineno-0-1) from agentlightning.types import AttemptedRollout, ResourcesUpdate, Span, TaskInput [](https://microsoft.github.io/agent-lightning/latest/deep-dive/birds-eye-view/#__codelineno-0-2) [](https://microsoft.github.io/agent-lightning/latest/deep-dive/birds-eye-view/#__codelineno-0-3) class LightningStore: [](https://microsoft.github.io/agent-lightning/latest/deep-dive/birds-eye-view/#__codelineno-0-4) [](https://microsoft.github.io/agent-lightning/latest/deep-dive/birds-eye-view/#__codelineno-0-5) async def enqueue_rollout(self, input: TaskInput, ...) -> Rollout: ... [](https://microsoft.github.io/agent-lightning/latest/deep-dive/birds-eye-view/#__codelineno-0-6) [](https://microsoft.github.io/agent-lightning/latest/deep-dive/birds-eye-view/#__codelineno-0-7) async def dequeue_rollout(self) -> AttemptedRollout | None: ... [](https://microsoft.github.io/agent-lightning/latest/deep-dive/birds-eye-view/#__codelineno-0-8) [](https://microsoft.github.io/agent-lightning/latest/deep-dive/birds-eye-view/#__codelineno-0-9) async def add_span(self, span: Span) -> Span: ... [](https://microsoft.github.io/agent-lightning/latest/deep-dive/birds-eye-view/#__codelineno-0-10) [](https://microsoft.github.io/agent-lightning/latest/deep-dive/birds-eye-view/#__codelineno-0-11) async def get_latest_resources(self) -> Optional[ResourcesUpdate]: ... [](https://microsoft.github.io/agent-lightning/latest/deep-dive/birds-eye-view/#__codelineno-0-12) [](https://microsoft.github.io/agent-lightning/latest/deep-dive/birds-eye-view/#__codelineno-0-13) async def wait_for_rollouts(self, rollout_ids: List[str], ...): ... [](https://microsoft.github.io/agent-lightning/latest/deep-dive/birds-eye-view/#__codelineno-0-14) [](https://microsoft.github.io/agent-lightning/latest/deep-dive/birds-eye-view/#__codelineno-0-15) async def query_spans(self, rollout_id: str, ...): ... [](https://microsoft.github.io/agent-lightning/latest/deep-dive/birds-eye-view/#__codelineno-0-16) [](https://microsoft.github.io/agent-lightning/latest/deep-dive/birds-eye-view/#__codelineno-0-17) async def update_attempt(self, rollout_id: str, attempt_id: str, status: str, ...): ... [](https://microsoft.github.io/agent-lightning/latest/deep-dive/birds-eye-view/#__codelineno-0-18) [](https://microsoft.github.io/agent-lightning/latest/deep-dive/birds-eye-view/#__codelineno-0-19) ...` These interfaces operate on [`AttemptedRollout`](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.AttemptedRollout " agentlightning.AttemptedRollout") , [`ResourcesUpdate`](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.ResourcesUpdate " agentlightning.ResourcesUpdate") , [`Span`](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.Span " agentlightning.Span") , and [`TaskInput`](https://microsoft.github.io/agent-lightning/latest/reference/internal/#agentlightning.TaskInput " agentlightning.TaskInput = Any module-attribute ") instances from `agentlightning.types`. As the APIs show, the store essentially provides a queue for rollouts and storage for resources, spans, and attempts. Developers should implement the store carefully to ensure data integrity and consistency, especially when multiple runners work in parallel across multiple attempts. The store is designed to be extensible. Users can implement their own store by inheriting from [`LightningStore`](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.LightningStore " agentlightning.LightningStore") and overriding methods. Agent-lightning provides a few reference implementations, such as [`InMemoryLightningStore`](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.InMemoryLightningStore " agentlightning.InMemoryLightningStore") (default) and `SqliteLightningStore` (under construction). When parallelized, the store may need special wrappers to ensure thread/process safety or delegate computation to a store in another process or machine. Supporting Components in the Loop[¶](https://microsoft.github.io/agent-lightning/latest/deep-dive/birds-eye-view/#supporting-components-in-the-loop "Permanent link") ---------------------------------------------------------------------------------------------------------------------------------------------------------------------- While the core loop is simple, Agent-lightning provides several components to make development easier and more powerful. ### Tracer[¶](https://microsoft.github.io/agent-lightning/latest/deep-dive/birds-eye-view/#tracer "Permanent link") The [`Tracer`](https://microsoft.github.io/agent-lightning/latest/reference/runner/#agentlightning.Tracer " agentlightning.Tracer") is a component within the [`Runner`](https://microsoft.github.io/agent-lightning/latest/reference/runner/#agentlightning.Runner " agentlightning.Runner") that records detailed spans (events) during an agent's execution and sends them to the [`LightningStore`](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.LightningStore " agentlightning.LightningStore") . Instead of requiring the agent to manually log every span, the tracer automatically instruments key methods (e.g., LLM calls) and captures their inputs, outputs, and metadata. This provides a detailed log of the agent's behavior with minimal effort. sequenceDiagram autonumber participant Store participant Runner participant Tracer participant Agent Note over Runner,Tracer: Runner manages tracer as member Tracer->>Agent: Apply instrumentation loop Until no more rollouts Store-->>Runner: dequeue_rollout → AttemptedRollout Store-->>Runner: get_latest_resources Runner->>Agent: training_rollout / validation_rollout loop For each finished span Agent-->>Tracer: openai.chat.completion invoked
agent.execute invoked
... Agent->>Tracer: emit intermediate reward Tracer-->>Store: add_otel_span(rollout_id, attempt_id, span) end Agent->>Runner: final reward + extra spans (if any) Runner-->>Store: add_span(rollout_id, attempt_id, span) Runner-->>Store: update_attempt(status) end Tracer->>Agent: Unapply instrumentation The above diagram shows the overall data flow between store, tracer and agent. In realistic, it's a bit more complicated than that. Spans are not emitted actively by the agent; they are intercepted by the tracer by hooking and instrumenting key methods used in the agents. The tracer uses a callback (called exporter) to monitor events and log to the store. Before a rollout starts, the runner enters a [`trace_context`](https://microsoft.github.io/agent-lightning/latest/reference/runner/#agentlightning.Tracer.trace_context " trace_context(name=None, *, store=None, rollout_id=None, attempt_id=None)") before invoking the agent, wiring store identifiers into the tracer. Each span completion streams back to the store through `LightningSpanProcessor`, so the agent’s instrumentation lands in [`add_otel_span`](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.LightningStore.add_otel_span " add_otel_span(rollout_id, attempt_id, readable_span, sequence_id=None) async ") . If the agent’s rollout method returns a numeric reward, the runner emits one more OpenTelemetry span before finalizing the attempt. ### Hooks[¶](https://microsoft.github.io/agent-lightning/latest/deep-dive/birds-eye-view/#hooks "Permanent link") [`Hook`](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.Hook " agentlightning.Hook") implementations are user-defined callback functions that allow you to augment a [`Runner`](https://microsoft.github.io/agent-lightning/latest/reference/runner/#agentlightning.Runner " agentlightning.Runner") 's behavior at specific points in its lifecycle. You can use hooks to add custom logging, set up resources before a rollout begins, or tear them down after it ends. Hooks can be triggered at four key moments: `on_rollout_start`, `on_trace_start`, `on_trace_end`, and `on_rollout_end`. Users should pay special attention to the difference between `on_trace_end` and `on_rollout_end`. The former is called right before the tracer exits the trace context, while the latter is called after the runner processes the final leftover rewards and spans, and finalizes the attempt in the store. sequenceDiagram autonumber participant Store participant Hooks participant Runner participant Tracer participant Agent Note over Runner,Hooks: Runner manages hooks as member loop Until no more rollouts Store-->>Runner: dequeue_rollout → AttemptedRollout Store-->>Runner: get_latest_resources Runner->>Hooks: on_rollout_start(agent, runner, rollout) Runner->>Agent: training_rollout / validation_rollout Tracer->>Agent: enter_trace_context activate Tracer Runner->>Hooks: on_trace_start(agent, runner, tracer, rollout) Note over Runner,Agent: Agent rollout omitted Runner->>Hooks: on_trace_end(agent, runner, tracer, rollout) Tracer->>Agent: exit_trace_context deactivate Tracer Agent->>Runner: final reward + extra spans (if any) Runner-->>Store: add_span(rollout_id, attempt_id, span) Runner->>Hooks: on_rollout_end(agent, runner, rollout, status) end ### Adapter[¶](https://microsoft.github.io/agent-lightning/latest/deep-dive/birds-eye-view/#adapter "Permanent link") The [`Adapter`](https://microsoft.github.io/agent-lightning/latest/reference/algorithm/#agentlightning.Adapter " agentlightning.Adapter") is a component used by the [`Algorithm`](https://microsoft.github.io/agent-lightning/latest/reference/algorithm/#agentlightning.Algorithm " agentlightning.Algorithm") to transform raw data from the [`LightningStore`](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.LightningStore " agentlightning.LightningStore") into a format suitable for learning. Runners stream raw spans into the store during execution. Later, the algorithm queries these spans and uses an adapter to convert them into structured data, like training examples for a reinforcement learning model. For instance, the [`TracerTraceToTriplet`](https://microsoft.github.io/agent-lightning/latest/reference/algorithm/#agentlightning.TracerTraceToTriplet " agentlightning.TracerTraceToTriplet") processes OpenTelemetry spans to create `(prompt, response, reward)` triplets, which are the fundamental data structure for many RL fine-tuning algorithms. flowchart LR Runner -- (1) add_otel_span --> Store Store -- (2) query_spans --> Algorithm Algorithm -- (3) spans --> Adapter Adapter -- (4) transformed data --> Algorithm ### LLM Proxy[¶](https://microsoft.github.io/agent-lightning/latest/deep-dive/birds-eye-view/#llm-proxy "Permanent link") The [`LLMProxy`](https://microsoft.github.io/agent-lightning/latest/reference/algorithm/#agentlightning.LLMProxy " agentlightning.LLMProxy") is an optional bridge component that sits between an agent and the algorithms' resources. It acts as a centralized endpoint for all LLM calls. Usually the proxy URL is added to the store as a special resource, so that the [`Runner`](https://microsoft.github.io/agent-lightning/latest/reference/runner/#agentlightning.Runner " agentlightning.Runner") can fetch it along with other resources when dequeuing a rollout. During rollouts, the runner invokes the proxy's HTTP endpoint instead of calling a model backend directly. This design offers several benefits: 1. **Instrumentation:** It automatically captures detailed traces of LLM interactions (prompts, responses, metadata) and sends them to the store, complementing the tracer, especially when the agent's code is hard to instrument directly. 2. **Backend Abstraction:** It provides a unified interface for various LLM backends (OpenAI, Anthropic, local models) and can add features like retry logic, rate limiting, and caching. 3. **Resource Management:** The algorithm can dynamically update which LLM the agent uses (e.g., swapping to a newly fine-tuned model) by simply swapping the backend model the proxy is using, without interrupting the agent's code. The benefits above seem to be all discussed within the context of model fine-tuning. As a matter of fact, the proxy can be useful for prompt tuning as well. The algorithm can register one of the following two types of endpoints into the proxy: 1. **Endpoint served by the algorithm:** If the algorithm is internally updating the LLM weights (e.g., RL), it can launch an LLM inference engine (i.e., a model server) and register the endpoint URL with the proxy. The proxy then forwards all LLM calls to that endpoint. 2. **Third-party LLM endpoint:** If the algorithm is not updating the LLM weights (e.g., prompt tuning), it can register a third-party LLM endpoint into the proxy. We show a diagram below that illustrates how the proxy fits into the overall data flow. sequenceDiagram autonumber participant Algo as Algorithm participant LLMProxy as LLM Proxy participant Store participant Runner participant Agent Note over Algo,LLMProxy: Algorithm manages LLMProxy as member loop Over the Dataset Algo->>Algo: Launch LLM Inference Engine
(optional) Algo->>LLMProxy: Register Inference Engine
(optional) Algo-->>Store: enqueue_rollout LLMProxy->>Store: Proxy URL added as Resource Store-->>Runner: dequeue_rollout → AttemptedRollout Store-->>Runner: get_latest_resources Runner->>Agent: rollout + resources
(LLM Proxy URL as resource) loop Defined by Agent Agent-->>LLMProxy: LLM calls activate LLMProxy LLMProxy-->>Store: add_span or add_otel_span LLMProxy-->>Agent: LLM responses deactivate LLMProxy Agent-->>Runner: rewards Runner-->>Store: add_span or add_otel_span end Runner-->>Store: update_attempt("finished", status) Store-->>Algo: query_rollouts + spans Algo-->>Algo: Update LLM Weights
(optional) end In this diagram, the store receives spans from both the proxy and the runner. We will see a problem later with parallelism where the proxy and runner are in different machines, and spans need to obtain a special counter from the store to ensure the ordering of spans. ### Trainer[¶](https://microsoft.github.io/agent-lightning/latest/deep-dive/birds-eye-view/#trainer "Permanent link") The [Trainer](https://microsoft.github.io/agent-lightning/latest/reference/trainer/#agentlightning.Trainer " agentlightning.Trainer") is the high-level orchestrator that initializes and connects all major components -- [Algorithm](https://microsoft.github.io/agent-lightning/latest/reference/algorithm/#agentlightning.Algorithm " agentlightning.Algorithm") , [Runner](https://microsoft.github.io/agent-lightning/latest/reference/runner/#agentlightning.Runner " agentlightning.Runner") , [LightningStore](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.LightningStore " agentlightning.LightningStore") , [Tracer](https://microsoft.github.io/agent-lightning/latest/reference/runner/#agentlightning.Tracer " agentlightning.Tracer") , [Adapter](https://microsoft.github.io/agent-lightning/latest/reference/algorithm/#agentlightning.Adapter " agentlightning.Adapter") , [LLM Proxy](https://microsoft.github.io/agent-lightning/latest/reference/algorithm/#agentlightning.LLMProxy " agentlightning.LLMProxy") , and [Hook](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.Hook " agentlightning.Hook") . The components can have a lifecycle as long as the trainer. The trainer manages their lifecycles and handles dependency injection, ensuring that every part of the system operates within a consistent and shared environment. Below, we demonstrate how the components relate to each other and their roles. We first clarify the roles and relationships shown in the diagram: 1. **Owns:** components that the trainer constructs and manages directly (e.g., runner, tracer). 2. **Injects:** components passed into others as dependencies. 3. **References:** weak links for coordination without ownership. 4. **Uses:** components that are temporarily interacted with. For example, the [LightningStore](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.LightningStore " agentlightning.LightningStore") is injected into the [Algorithm](https://microsoft.github.io/agent-lightning/latest/reference/algorithm/#agentlightning.Algorithm " agentlightning.Algorithm") and [Runner](https://microsoft.github.io/agent-lightning/latest/reference/runner/#agentlightning.Runner " agentlightning.Runner") . The [Tracer](https://microsoft.github.io/agent-lightning/latest/reference/runner/#agentlightning.Tracer " agentlightning.Tracer") and [LitAgent](https://microsoft.github.io/agent-lightning/latest/reference/agent/#agentlightning.LitAgent " agentlightning.LitAgent") are injected into the runner. The [Adapter](https://microsoft.github.io/agent-lightning/latest/reference/algorithm/#agentlightning.Adapter " agentlightning.Adapter") and [LLM Proxy](https://microsoft.github.io/agent-lightning/latest/reference/algorithm/#agentlightning.LLMProxy " agentlightning.LLMProxy") are injected into the algorithm. The store is further injected into the tracer, adapter and LLM proxy by the runner and algorithm respectively. flowchart TD %% === Left side: Algorithm domain === subgraph L["Algorithm Side"] Algorithm["Algorithm
(no default)"] Adapter["Adapter
(TracerTraceToTriplet*)"] LLMProxy["LLM Proxy
(no default)"] Algorithm -.injects.-> Adapter Algorithm -.injects.-> LLMProxy end linkStyle 0,1 stroke:#896978,stroke-width:2px; %% === Middle: Core trainer and store === subgraph M["Core"] Trainer["Trainer"] Store["LightningStore
(InMemory* default)"] Trainer --has--> Algorithm Trainer --has--> Store Trainer --has--> Adapter Trainer --has--> LLMProxy end linkStyle 2,3,4,5 stroke:#839791,stroke-width:2px; %% === Right side: Runner side === subgraph R["Runner Side"] Runner["Runner
(LitAgentRunner* default)"] Tracer["Tracer
(AgentOpsTracer*)"] Hooks["Hooks (empty default)"] Agent["Agent
(LitAgent*)"] Runner -.injects.-> Tracer Runner -.injects.-> Store Runner -.injects.-> Agent Runner -.injects.-> Hooks Tracer -.injects.-> Store Hooks -.uses.-> Runner Hooks -.uses.-> Agent Hooks -.uses.-> Tracer end linkStyle 6,7,8,9,10 stroke:#896978,stroke-width:2px; linkStyle 11,12,13 stroke:#7a89c2,stroke-width:2px; %% === Cross-section connections === Trainer --has--> Runner Trainer --has--> Tracer Trainer --has--> Hooks Trainer --uses--> Agent Algorithm -.injects.-> Store LLMProxy -.injects.-> Store Agent -.references.-> Trainer Runner -.references.-> Trainer Algorithm -.references.-> Trainer linkStyle 14,15,16 stroke:#839791,stroke-width:2px; linkStyle 17,20,21,22 stroke:#7a89c2,stroke-width:2px; linkStyle 18,19 stroke:#896978,stroke-width:2px; style L fill:none; style M fill:none; style R fill:none; Putting It All Together: A Reinforcement Learning Example (VERL)[¶](https://microsoft.github.io/agent-lightning/latest/deep-dive/birds-eye-view/#putting-it-all-together-a-reinforcement-learning-example-verl "Permanent link") --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- [](https://microsoft.github.io/agent-lightning/latest/deep-dive/birds-eye-view/) VERL shows how an algorithm consumes the shared infrastructure. For historical reasons, code lives in `agentlightning.algorithm.verl` and `agentlightning.verl`. The latter is legacy and reuses terms like `Trainer` in confusing ways. The former is a thin wrapper that conforms to the new algorithm interface. Future versions will merge the two. Reinforcement learning aims to learn a policy that takes actions in states to maximize expected reward. For agents, the policy is usually a language model. Inputs are prompts (state). Outputs are generated text (action). A numeric score judges quality (reward). The `(state, action, reward)` **triplet** is the basic learning unit. In Agent-lightning, the environment is implicit in the agent’s workflow, which orchestrates one or more LLM calls and often self-judges using rules or additional model calls. During a rollout, the agent emits spans that contain everything needed for RL training, including LLM call traces and numeric judge/reward signals. The "algorithm", on the other hand, have more responsibilities. 1. Providing a language model deployment that is currently learning and improving for the agent to interact with; 2. Preparing the tasks that the agents will perform; 3. Querying the spans generated, extracting triplets, and converting them into a format that the underlying RL library can consume; 4. Updating the language model based on the learning signals. In the VERL integration, the algorithm launches a chat completion endpoint using `vLLM` and wraps training with `FSDP` for distributed optimization. It enqueues tasks from the dataset. After rollouts finish, it queries spans and converts them to triplets with `TracerTraceToTriplet`. VERL’s native training loop then consumes these triplets to update model weights. The workflow can be summarized in the following diagram. sequenceDiagram autonumber participant vLLM as vLLM Chat
Completion Endpoint participant FSDP as FSDP / Megatron
Weights Optimizer participant Algo as Algorithm
Main Controller
(Main Process) participant Adapter as TracerTraceToTriplet participant LLMProxy as LLM Proxy participant Store as LightningStore participant Runner as Runner + Agent Note over Algo,LLMProxy: LLMProxy and Adapter are injected by Trainer as member Note over vLLM,Algo: Algorithm creates and owns vLLM and FSDP loop Over the Dataset in Batches Algo->>vLLM: Create Chat Completion Endpoint activate vLLM vLLM->>LLMProxy: Registered as Backend Endpoint LLMProxy->>Store: Proxy URL added as Resource par Over data samples in the batch Algo-->>Store: enqueue_rollout Store-->>Runner: Dequeue Rollout +
Resources (i.e., URL) loop One Rollout Attempt Runner-->>LLMProxy: LLM calls LLMProxy-->>vLLM: Forwarded LLM calls vLLM-->>LLMProxy: LLM responses LLMProxy-->>Store: add_span / add_otel_span LLMProxy-->>Runner: Forwarded LLM responses Runner-->>Store: add_span / add_otel_span
(by tracer, including rewards) end Runner-->>Store: update_attempt("finished", status) end Algo-->>Store: Poll for completed rollouts + spans Algo->>vLLM: Chat Completion Endpoint Sleeps deactivate vLLM Algo->>Adapter: adapt(spans) Adapter->>FSDP: Triplets (state, action, reward) activate FSDP FSDP-->>Algo: Updated LLM weights deactivate FSDP end **Notes:** 1. There are interactions between different components injected into or owned by algorithms in the diagram, such as the output of the adapter feeding into the FSDP optimizer. This is for simplicity of illustration and slightly different from the actual implementation, where it's the algorithm main controller that orchestrates the data flow between components. 2. **On mapping to VERL.** VERL uses a classic RLHF setup where each action is a single token, the state is the full conversation history up to that token, and reward is given at the end. This is very different from our setup where each action is actually a chunk of text, although they are both called RL! Therefore, after the adapter produces triplets, the algorithm converts each `(state, action, reward)` into a VERL trajectory (`DataProto`) with keys like `input_ids`, `position_ids`, `attention_mask`, and `token_level_scores`. That conversion happens after triplet generation and is not shown in the diagram. Execution Strategies and Parallelism[¶](https://microsoft.github.io/agent-lightning/latest/deep-dive/birds-eye-view/#execution-strategies-and-parallelism "Permanent link") ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Readers might have observed from the diagram above that there is absolutely no communication between (1) runner and agents and (2) algorithm. The only overlap of them is the [Trainer](https://microsoft.github.io/agent-lightning/latest/reference/trainer/#agentlightning.Trainer " agentlightning.Trainer") and [LightningStore](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.LightningStore " agentlightning.LightningStore") . This observation is very clear with the diagram within the trainer section. This design allows us to flexibly scale the runner and algorithm independently, which is crucial for large-scale training. Agent-lightning packages two executable bundles: a runner bundle ([Runner](https://microsoft.github.io/agent-lightning/latest/reference/runner/#agentlightning.Runner " agentlightning.Runner") , [Tracer](https://microsoft.github.io/agent-lightning/latest/reference/runner/#agentlightning.Tracer " agentlightning.Tracer") , [Hook](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.Hook " agentlightning.Hook") , [LitAgent](https://microsoft.github.io/agent-lightning/latest/reference/agent/#agentlightning.LitAgent " agentlightning.LitAgent") ) and an algorithm bundle ([Algorithm](https://microsoft.github.io/agent-lightning/latest/reference/algorithm/#agentlightning.Algorithm " agentlightning.Algorithm") , [Adapter](https://microsoft.github.io/agent-lightning/latest/reference/algorithm/#agentlightning.Adapter " agentlightning.Adapter") , [LLM Proxy](https://microsoft.github.io/agent-lightning/latest/reference/algorithm/#agentlightning.LLMProxy " agentlightning.LLMProxy") ). Both share the [LightningStore](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.LightningStore " agentlightning.LightningStore") . The trainer initializes and connects the bundles. graph TD subgraph Runner_Side["Runner Bundle"] direction LR R[Runner] --- T[Tracer] --- H[Hooks] --- A1[Agent] end subgraph Algorithm_Side["Algorithm Bundle"] direction LR ALG[Algorithm] --- AD[Adapter] --- LLM[LLM Proxy] end S[(Store)] TR[Trainer] Runner_Side <--> S Algorithm_Side <--> S TR --> Runner_Side TR --> Algorithm_Side linkStyle 0,1,2,3,4 opacity:0; An [execution strategy](https://microsoft.github.io/agent-lightning/latest/reference/trainer/#agentlightning.ExecutionStrategy " agentlightning.ExecutionStrategy") , defined and owned by the trainer, governs how algorithm and runner bundles are placed, connected, scaled, and aborted. It serves four primary purposes. Execution strategies first determine **bundle placement** — whether the two bundles run in the same thread, process, machine, or across separate machines. They also define **store management**, wrapping the store and specifying how data is shared between bundles. In terms of **scalability**, the strategy can replicate the runner bundle across multiple threads, processes, or machines to expand throughput on the runner side. The algorithm side remains single-process due to the complexity of parallelization. Mature frameworks such as _DeepSpeed_ and _Megatron_ already support distributed model training, so scaling of the algorithm bundle is delegated to those implementations. **Abort handling** is another core responsibility. Aborts may be triggered by normal exits, failures in either bundle, or user interrupts. The trainer must include cancellation interfaces for the bundles so that bundles can be cleanly aborted. When the algorithm bundle exits normally, the strategy signals the runner bundle to terminate. If the runner exits first, no signal is sent to the algorithm, as it may still be processing completed rollouts. In cases of failure or user interruption, the strategy signals both bundles to abort; if a bundle fails to respond, the strategy should attempt a forceful termination. Agent-lightning currently provides two execution strategies: **shared-memory** and **client-server**, described in the following sections. ### Shared-memory Strategy[¶](https://microsoft.github.io/agent-lightning/latest/deep-dive/birds-eye-view/#shared-memory-strategy "Permanent link") [`SharedMemoryExecutionStrategy`](https://microsoft.github.io/agent-lightning/latest/reference/trainer/#agentlightning.SharedMemoryExecutionStrategy " agentlightning.SharedMemoryExecutionStrategy") runs algorithm and runner bundles as threads in one process. The strategy wraps the store with [`LightningStoreThreaded`](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.LightningStoreThreaded " agentlightning.LightningStoreThreaded") , which guards calls with a lock for safe concurrency. This is good for lightweight debugging because components share one Python heap and avoid serialization. It is not suitable for heavy RL training or compute-intensive agents. flowchart TB subgraph MainProcess direction TB subgraph AlgorithmThread [Thread 0] Algorithm[Algorithm bundle] end subgraph RunnerThread1 [Thread 1] Runner1[Runner bundle #1] end subgraph RunnerThread2 [Thread 2] Runner2[Runner bundle #2] end subgraph RunnerThread3 [Thread 3] RunnerN[Runner bundle #N] end LightningStoreFacade[LightningStoreThreaded] BaseStore[Underlying LightningStore] end Algorithm -- async calls --> LightningStoreFacade Runner1 -- async calls --> LightningStoreFacade Runner2 -- async calls --> LightningStoreFacade RunnerN -- async calls --> LightningStoreFacade LightningStoreFacade -->|thread-safe delegates| BaseStore You can configure which role runs on the main thread. If the main thread runs the algorithm, it is able to spawn multiple runner threads. If it runs a runner, `n_runners` must be 1 and the runner lives on the main thread. ### Client-server Strategy[¶](https://microsoft.github.io/agent-lightning/latest/deep-dive/birds-eye-view/#client-server-strategy "Permanent link") [](https://microsoft.github.io/agent-lightning/latest/deep-dive/birds-eye-view/) [`ClientServerExecutionStrategy`](https://microsoft.github.io/agent-lightning/latest/reference/trainer/#agentlightning.ClientServerExecutionStrategy " agentlightning.ClientServerExecutionStrategy") splits concerns across processes. The algorithm bundle starts a [`LightningStoreServer`](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.LightningStoreServer " agentlightning.LightningStoreServer") (HTTP API) that wraps the underlying store. Runners connect via [`LightningStoreClient`](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.LightningStoreClient " agentlightning.LightningStoreClient") to call the same interface over REST. The server embeds a client to support algorithm-launched subprocesses (e.g., an LLM proxy worker) that need to talk back to the algorithm’s process through the same API. Currently this design introduces an extra wrapper in the Server side (as shown in the diagram), which helps debugging and improves fault tolerance. We might revisit this design in the future and enforce the client to be the only way to communicate with the store. flowchart TD subgraph Algorithm Process Group subgraph StoreServer[LightningStoreServer] StoreHttpClient[HTTP Client] StoreHttpServer[HTTP Server] StoreWrapper[LightningStore Wrapper] StoreHttpClient -- HTTP --> StoreHttpServer end subgraph Algorithm Bundle Algorithm[Algorithm Main Process] subgraph Another subprocess LLMProxy[LLM Proxy] end end LLMProxy -- async calls --> StoreHttpClient Algorithm -- async calls --> StoreWrapper end subgraph RunnerSide ["Runner Side"] subgraph Runner Process 1 Runner1[Runner bundle #1] Runner1 -- async calls --> LightningStoreClient1 LightningStoreClient1[LightningStoreClient] end subgraph Runner Process 2 Runner2[Runner bundle #2] Runner2 -- async calls --> LightningStoreClient2 LightningStoreClient2[LightningStoreClient] end subgraph Runner Process N RunnerN[Runner bundle #N] RunnerN -- async calls --> LightningStoreClientN LightningStoreClientN[LightningStoreClient] end end LocalStore[Underlying LightningStore] StoreHttpServer -->|delegates| StoreWrapper StoreWrapper -->|delegates| LocalStore LightningStoreClient1 -- HTTP --> StoreHttpServer LightningStoreClient2 -- HTTP --> StoreHttpServer LightningStoreClientN -- HTTP --> StoreHttpServer style RunnerSide fill:none; Online/Continuous Learning[¶](https://microsoft.github.io/agent-lightning/latest/deep-dive/birds-eye-view/#onlinecontinuous-learning "Permanent link") ------------------------------------------------------------------------------------------------------------------------------------------------------- Continuous learning keeps the algorithm loop running while runners report tasks and spans opportunistically. Key differences from batch mode: 1. The algorithm does not enqueue rollouts from a fixed dataset. Runners report tasks/rollouts and spans spontaneously. 2. The algorithm can wait for rollouts with a expected set of rollout IDs, but more often polls for new rollouts and spans or waits for a count to arrive. 3. The [`Runner`](https://microsoft.github.io/agent-lightning/latest/reference/runner/#agentlightning.Runner " agentlightning.Runner") processes one rollout at a time via [`step(task)`](https://microsoft.github.io/agent-lightning/latest/reference/runner/#agentlightning.Runner.step " step(input, *, resources=None, mode=None, event=None) async ") instead of exhausting a task queue. It notifies the store when starting a rollout so the store records it. 4. A user or higher-level loop controls which resources the next step uses and when to retry. [Spans](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.Span " agentlightning.Span") , [Adapter](https://microsoft.github.io/agent-lightning/latest/reference/algorithm/#agentlightning.Adapter " agentlightning.Adapter") implementations, and the [LLM Proxy](https://microsoft.github.io/agent-lightning/latest/reference/algorithm/#agentlightning.LLMProxy " agentlightning.LLMProxy") work the same way. sequenceDiagram autonumber actor User participant Runner participant Agent participant Store as LightningStore participant Algorithm Note over Algorithm: Algorithm is long-running and loops continuously loop Continuous Learning Loop activate User opt Decide what to do next User-->>Store: get_resources_by_id Store-->>User: Resources User-->>User: Prepare input for next step end User->>Runner: step(input, resources) activate Runner Runner-->>Store: Notify: start_rollout(input) Runner->>Agent: rollout(input, resources) Agent-->>Runner: add_span / reward spans Runner-->>Store: add_span or add_otel_span Runner-->>Store: update_attempt(status="finished") deactivate Runner deactivate User Algorithm->>Store: poll for new rollouts and spans opt If there is enough new data Store-->>Algorithm: new spans Algorithm->>Algorithm: adapt spans → learning signal Algorithm->>Store: update_resources end end Back to top --- # Understanding Store - Agent-lightning [Skip to content](https://microsoft.github.io/agent-lightning/latest/deep-dive/store/#understanding-store) ⚠️ You are viewing development documentation. For stable documentation, please visit [https://microsoft.github.io/agent-lightning/stable/](https://microsoft.github.io/agent-lightning/stable/) Understanding Store[¶](https://microsoft.github.io/agent-lightning/latest/deep-dive/store/#understanding-store "Permanent link") ================================================================================================================================= The **[`LightningStore`](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.LightningStore " agentlightning.LightningStore") ** is the central coordination point for Agent-lightning. It holds the task queue, rollouts, attempts, spans, and versioned resources, and exposes a small API both Runners and Algorithms use to communicate. This document explains what's in the store, how statuses transition, how spans are recorded, and the concurrency model (threads & processes). What's in the Store?[¶](https://microsoft.github.io/agent-lightning/latest/deep-dive/store/#whats-in-the-store "Permanent link") --------------------------------------------------------------------------------------------------------------------------------- ![Store Architecture](https://microsoft.github.io/agent-lightning/latest/assets/store-api-visualized.svg) At a high level: * **Task Queue** – [`enqueue_rollout`](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.LightningStore.enqueue_rollout " enqueue_rollout(input, mode=None, resources_id=None, config=None, metadata=None) async ") adds work; workers poll with [`dequeue_rollout`](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.LightningStore.dequeue_rollout " dequeue_rollout(worker_id=None) async ") . When a rollout is dequeued, it automatically creates a new attempt associated with itself. * **Rollouts** – A rollout is one unit of work. It has input, metadata, links to resources, and a lifecycle (`queuing → preparing → running → ...`). Valid [RolloutStatus](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.RolloutStatus " agentlightning.RolloutStatus = Literal['queuing', 'preparing', 'running', 'failed', 'succeeded', 'cancelled', 'requeuing'] module-attribute ") are **`queuing`**, `preparing`, `running`, `succeeded`, `failed`, **`requeuing`**, **`cancelled`**. For algorithms and runners, the rollout can be seen as a whole, without worrying about the internal attempts. * **Attempts** – Each rollout can have multiple executions (retries). Attempts track [`status`](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.Attempt.status " status = 'preparing' class-attribute instance-attribute ") , [`start_time`](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.Attempt.start_time " start_time instance-attribute ") , [`end_time`](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.Attempt.end_time " end_time = None class-attribute instance-attribute ") , [`last_heartbeat_time`](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.Attempt.last_heartbeat_time " last_heartbeat_time = None class-attribute instance-attribute ") and link to spans. Valid [AttemptStatus](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.AttemptStatus " agentlightning.AttemptStatus = Literal['preparing', 'running', 'failed', 'succeeded', 'unresponsive', 'timeout'] module-attribute ") are `preparing`, `running`, `succeeded`, `failed`, `requeuing`, `cancelled`. * **Spans** – Structured trace events produced by the Tracer during an attempt. Spans are ordered by a **monotonic sequence id** per `(rollout_id, attempt_id)`. * **Resources** – Versioned, named bundles (e.g., prompt templates) referenced by rollouts. * **Workers** – Metadata about runner instances: heartbeat timestamps, current assignment, and status. Rollout and Task share the same surface in practice: [`Rollout.input`](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.Rollout " agentlightning.Rollout") is the task input. The queue stores rollouts that are not yet running; [Runners](https://microsoft.github.io/agent-lightning/latest/reference/runner/#agentlightning.Runner " agentlightning.Runner") dequeue them and update the same rollout's status as work progresses. Before we look at status transitions, it helps to keep in mind that rollouts are the "outside view," while attempts are the "inside view." Attempts are what actually run; rollouts summarize the latest attempt plus a small set of control actions like queueing and cancellation. Attempt Status Transitions[¶](https://microsoft.github.io/agent-lightning/latest/deep-dive/store/#attempt-status-transitions "Permanent link") ----------------------------------------------------------------------------------------------------------------------------------------------- The status model is intentionally small and operationally clear. stateDiagram-v2 direction LR [*] --> preparing: Runner calls dequeue_rollout()
or start_rollout()
or start_attempt() preparing --> running: Runner calls
add_[otel_]span()
for the first time state c_runner <> state c_watch <> preparing --> c_runner: Runner calls
update_attempt(...) running --> c_runner: Runner calls
update_attempt(...) running --> c_watch: Watchdog checks preparing --> c_watch: Watchdog checks state "Client-set outcome" as Client { direction TB succeeded failed } state "Watchdog / policy" as Watch { direction TB timeout unresponsive } c_runner --> succeeded: update_attempt(status=succeeded) c_runner --> failed: update_attempt(status=failed) c_watch --> timeout: now - start_time > timeout_seconds c_watch --> unresponsive: now - last_heartbeat > unresponsive_seconds unresponsive --> running: Runner calls
add_[otel_]span() Each attempt begins in **preparing**, created either when a rollout is dequeued or explicitly started. It transitions to **running** the first time a span is recorded. From there, a few clear rules govern how it can change: * When the runner explicitly marks completion, the attempt becomes **succeeded** or **failed** (when the runner catches exception thrown out by the agent). * When the watchdog detects that the total elapsed time since start exceeds the configured limit, it marks the attempt as **timeout**. * If heartbeats stop arriving for too long, the watchdog marks it **unresponsive**. * A new span from the runner can immediately revive an **unresponsive** attempt back to **running**. What's a Watchdog? The watchdog enforces timing and liveness rules defined by each rollout’s [`RolloutConfig`](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.RolloutConfig " agentlightning.RolloutConfig") . It’s not a separate thread or service, but a function periodically invoked (e.g., before store mutations) to keep attempts healthy and consistent. This simple model allows the system to distinguish between normal termination, abnormal stalling, and recoverable interruption without additional state flags. Worker Telemetry[¶](https://microsoft.github.io/agent-lightning/latest/deep-dive/store/#worker-telemetry "Permanent link") --------------------------------------------------------------------------------------------------------------------------- Workers track runner-level activity timestamps (`last_heartbeat_time`, `last_dequeue_time`, `last_busy_time`, `last_idle_time`) plus their current rollout assignment. Those fields are now derived automatically: * [`dequeue_rollout(worker_id=...)`](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.LightningStore.dequeue_rollout " dequeue_rollout(worker_id=None) async ") records which worker polled the queue and refreshes `last_dequeue_time`. * [`update_attempt(..., worker_id=...)`](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.LightningStore.update_attempt " update_attempt(rollout_id, attempt_id, status=UNSET, worker_id=UNSET, last_heartbeat_time=UNSET, metadata=UNSET) async ") drives the worker status machine. Assigning an attempt marks the worker **busy** and stamps `last_busy_time`; finishing with `status in {"succeeded","failed"}` switches to **idle**, while watchdog transitions such as `timeout`/`unresponsive` make the worker **unknown** and clear `current_rollout_id` / `current_attempt_id`. * [`update_worker(...)`](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.LightningStore.update_worker " update_worker(worker_id, heartbeat_stats=UNSET) async ") is reserved for heartbeats. It snapshots optional `heartbeat_stats` and always updates `last_heartbeat_time`. Because every transition flows through these APIs, worker status is derived automatically from rollout execution and heartbeats. Note, however, that calling `update_worker` with a new `worker_id` will create a new worker record with status "unknown" if one does not exist. Thus, while manual status changes are not allowed, new worker records can be created externally via heartbeats. Rollout Transition Map[¶](https://microsoft.github.io/agent-lightning/latest/deep-dive/store/#rollout-transition-map "Permanent link") --------------------------------------------------------------------------------------------------------------------------------------- Rollout status is an **aggregated view** of its latest attempt’s status, with additional transitions for queueing and explicit cancellation. A rollout’s retry behavior is controlled by [`Rollout.config`](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.Rollout " agentlightning.Rollout") (a [`RolloutConfig`](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.RolloutConfig " agentlightning.RolloutConfig") ). The key fields are: * [`timeout_seconds`](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.RolloutConfig.timeout_seconds " timeout_seconds = None class-attribute instance-attribute ") – maximum wall-clock time for an attempt before it is marked `timeout`. * [`unresponsive_seconds`](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.RolloutConfig.unresponsive_seconds " unresponsive_seconds = None class-attribute instance-attribute ") – maximum silence between heartbeats before an attempt is marked `unresponsive`. * [`max_attempts`](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.RolloutConfig.max_attempts " max_attempts = Field(default=1, ge=1) class-attribute instance-attribute ") – total number of attempts allowed for the rollout (including the first). * [`retry_condition`](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.RolloutConfig.retry_condition " retry_condition = Field(default_factory=(cast(Callable[[], List[AttemptStatus]], list))) class-attribute instance-attribute ") – which attempt terminal statuses should trigger a retry (e.g., `["failed", "timeout", "unresponsive"]`). **How it plays out:** The runner works on attempt `k`. If the attempt ends in a status that is listed in `retry_condition`, and `k < max_attempts`, the rollout moves to **requeuing** and the store creates attempt `k+1`. Otherwise, the rollout becomes **failed** (or **succeeded** if the runner marked it so). `timeout_seconds` and `unresponsive_seconds` are enforced by the watchdog and feed into the same decision flow. A minimal example of how to use `RolloutConfig`: `[](https://microsoft.github.io/agent-lightning/latest/deep-dive/store/#__codelineno-0-1) from agentlightning import RolloutConfig [](https://microsoft.github.io/agent-lightning/latest/deep-dive/store/#__codelineno-0-2) [](https://microsoft.github.io/agent-lightning/latest/deep-dive/store/#__codelineno-0-3) # Retry on explicit failures or timeouts, up to 3 attempts in total. [](https://microsoft.github.io/agent-lightning/latest/deep-dive/store/#__codelineno-0-4) cfg = RolloutConfig( [](https://microsoft.github.io/agent-lightning/latest/deep-dive/store/#__codelineno-0-5) timeout_seconds=600, [](https://microsoft.github.io/agent-lightning/latest/deep-dive/store/#__codelineno-0-6) unresponsive_seconds=120, [](https://microsoft.github.io/agent-lightning/latest/deep-dive/store/#__codelineno-0-7) max_attempts=3, [](https://microsoft.github.io/agent-lightning/latest/deep-dive/store/#__codelineno-0-8) retry_condition=["failed", "timeout"] [](https://microsoft.github.io/agent-lightning/latest/deep-dive/store/#__codelineno-0-9) ) [](https://microsoft.github.io/agent-lightning/latest/deep-dive/store/#__codelineno-0-10) [](https://microsoft.github.io/agent-lightning/latest/deep-dive/store/#__codelineno-0-11) # When creating/enqueuing a rollout, attach this config. [](https://microsoft.github.io/agent-lightning/latest/deep-dive/store/#__codelineno-0-12) # The store will propagate attempt outcomes according to cfg. [](https://microsoft.github.io/agent-lightning/latest/deep-dive/store/#__codelineno-0-13) rollout = await store.enqueue_rollout(input, config=cfg)` | Latest attempt status | Rollout transition | Notes / guards | | --- | --- | --- | | N/A | `queuing` | Created by `enqueue_rollout()`. | | `preparing` | `queuing/requeuing` → `preparing` | Typically `dequeue_rollout()` or `start_rollout()`/`start_attempt()` creates a new attempt. | | `running` | `preparing/queuing/requeuing` → `running` | First `add_[otel_]span()` flips the attempt to `running`; rollout follows via `rollout_status_from_attempt`. | | `succeeded` | `*` → `succeeded` | Terminal. Rollout `end_time` set. | | `failed` / `timeout` / `unresponsive` | `*` → `requeuing` | **Only if** `status ∈ retry_condition ∧ sequence_id < max_attempts`. | | `failed` / `timeout` / `unresponsive` | `*` → `failed` | Otherwise (no retries left or retries disabled). | | `*` | `*` → `cancelled` | Explicitly set by `update_rollout(status=cancelled)`. | Why aggregation? In code, we use `rollout_status_from_attempt()` which actively updates the rollout based on the latest attempt. Reading the table above is usually easier than reverse-engineering the propagation logic in the code: think of the rollout’s transitions as _callbacks_ on attempt state changes, plus queue/cancel paths. Spans[¶](https://microsoft.github.io/agent-lightning/latest/deep-dive/store/#spans "Permanent link") ----------------------------------------------------------------------------------------------------- Every traceable operation in a rollout is stored as a [Span](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.Span " agentlightning.Span") . Spans not only capture fine-grained instrumentation but also act as periodic heartbeats that demonstrate liveness. The first span marks activation; each subsequent one both extends the trace and refreshes the attempt’s [`last_heartbeat_time`](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.Attempt.last_heartbeat_time " last_heartbeat_time = None class-attribute instance-attribute ") . If no span arrives within the configured [`unresponsive_seconds`](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.RolloutConfig.unresponsive_seconds " unresponsive_seconds = None class-attribute instance-attribute ") , the watchdog downgrades the attempt to **unresponsive** until activity resumes. Spans are indexed by `(rollout_id, attempt_id, sequence_id)` where the sequence is monotonic. Tracing analysis tools like [Adapter](https://microsoft.github.io/agent-lightning/latest/reference/algorithm/#agentlightning.Adapter " agentlightning.Adapter") usually rely on "time order" to reconstruct the trace. However, in a distributed system, the recorded start time and end time of a span are not necessarily in the right order when they aggregated into a central store. Therefore, we enforce every span creation to retrieve a monotonically increasing [`sequence_id`](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.Span.sequence_id " sequence_id instance-attribute ") from the store before adding the span. Note In practice, one `sequence_id` can be used to create multiple spans. In that case, the orders between the multiple spans are determined by the order of `start_time` and `end_time` of the spans. ### OpenTelemetry conversion[¶](https://microsoft.github.io/agent-lightning/latest/deep-dive/store/#opentelemetry-conversion "Permanent link") Runners often produce [OpenTelemetry `ReadableSpan`](https://opentelemetry.io/docs/concepts/signals/traces/#attributes) objects directly. The store normalizes them into [`Span`](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.Span " agentlightning.Span") as follows: 1. The runner first requests [`get_next_span_sequence_id`](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.LightningStore.get_next_span_sequence_id " get_next_span_sequence_id(rollout_id, attempt_id) async ") via `sequence_id = await store.get_next_span_sequence_id(rollout_id, attempt_id)`. This guarantees ordering within the attempt regardless of clock skew. 2. `trace_id`, `span_id`, `parent_id`, `name`, `status`, timestamps, attributes, events, links, and resource are copied from the OTEL span. Timestamps are auto-normalized to seconds (nanoseconds are converted). 3. OTEL `SpanContext` and parent context are preserved so downstream tools can correlate traces across systems. 4. Any additional serializable fields present on the `ReadableSpan` are retained in the stored span (after safe JSON serialization), which keeps the representation forward-compatible. Programmatically this is encapsulated by [`Span.from_opentelemetry(readable_span, rollout_id, attempt_id, sequence_id)`](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.Span.from_opentelemetry " from_opentelemetry(src, rollout_id, attempt_id, sequence_id) classmethod ") ; [`store.add_otel_span(...)`](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.LightningStore.add_otel_span " add_otel_span(rollout_id, attempt_id, readable_span, sequence_id=None) async ") simply wraps the fetch-then-add flow. The end result is a store span that is stable to sort, merge, and query, while still preserving the richness of the original OTEL payload. Tip [`add_span`](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.LightningStore.add_span " add_span(span) async ") or [`add_otel_span`](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.LightningStore.add_otel_span " add_otel_span(rollout_id, attempt_id, readable_span, sequence_id=None) async ") both appends a span _and_ acts as a heartbeat that can revive `unresponsive` → `running`. ### OTLP Compatibility[¶](https://microsoft.github.io/agent-lightning/latest/deep-dive/store/#otlp-compatibility "Permanent link") Some of the LightningStore implementations support exporting traces via the [OTLP/HTTP specification](https://opentelemetry.io/docs/specs/otlp/) . For example, [`LightningStoreServer`](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.LightningStoreServer " agentlightning.LightningStoreServer") exposes `/v1/traces` endpoint, it implements the binary Protobuf variant defined by the spec, including the required `Content-Type: application/x-protobuf`, optional `Content-Encoding: gzip`, and status responses encoded as `google.rpc.Status`. Agent-lightning helps parsing `ExportTraceServiceRequest` messages, validate identifiers, normalize resource metadata, and allocate sequence numbers so store implementations only need to persist [`Span`](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.Span " agentlightning.Span") objects in order. Because the interface speaks standard OTLP, any OpenTelemetry-compatible SDK or collector can emit spans directly to a LightningStore OTLP endpoint without custom shims. The server responds according to the OTLP contract (status code, encoding, and error payloads), which keeps Agent-lightning interoperable with existing observability tooling. This compatibility serves as a strong complement to the OpenTelemetry conversion discussed above. Check whether the store supports OTLP traces via the [`capabilities["otlp_traces"]`](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.LightningStore.capabilities " capabilities property ") property. Implementation Overview[¶](https://microsoft.github.io/agent-lightning/latest/deep-dive/store/#implementation-overview "Permanent link") ----------------------------------------------------------------------------------------------------------------------------------------- The `agentlightning.store` module is organized into two distinct layers plus optional wrappers: classDiagram direction TB class LightningStore { <> +enqueue_rollout() +dequeue_rollout() +update_attempt() +add_span() +query_rollouts() ... } class LightningCollections { <> +rollouts: Collection +attempts: Collection +spans: Collection +resources: Collection +workers: Collection +rollout_queue: Queue +span_sequence_ids: KeyValue +atomic() } class CollectionBasedLightningStore~T~ { +collections: T -healthcheck_before() -tracked() } class InMemoryLightningStore class MongoLightningStore class InMemoryLightningCollections class MongoLightningCollections class LightningStoreServer { +store: LightningStore +start() +stop() } class LightningStoreClient { +server_address: str } class LightningStoreThreaded { +store: LightningStore } LightningStore <|-- CollectionBasedLightningStore LightningStore <|-- LightningStoreServer LightningStore <|-- LightningStoreClient LightningStore <|-- LightningStoreThreaded CollectionBasedLightningStore <|-- InMemoryLightningStore CollectionBasedLightningStore <|-- MongoLightningStore LightningCollections <|-- InMemoryLightningCollections LightningCollections <|-- MongoLightningCollections InMemoryLightningStore ..> InMemoryLightningCollections : uses MongoLightningStore ..> MongoLightningCollections : uses LightningStoreServer o-- LightningStore : wraps LightningStoreThreaded o-- LightningStore : wraps 1. **Collections Layer** – Low-level storage primitives ([`LightningCollections`](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.store.collection.LightningCollections " agentlightning.store.collection.LightningCollections") ) providing CRUD operations via [`Collection`](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.store.collection.Collection " agentlightning.store.collection.Collection") , [`Queue`](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.store.collection.Queue " agentlightning.store.collection.Queue") , and [`KeyValue`](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.store.collection.KeyValue " agentlightning.store.collection.KeyValue") interfaces. Each backend (in-memory, MongoDB) implements these primitives. 2. **Store Layer** – All [`LightningStore`](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.LightningStore " agentlightning.LightningStore") implementations must inherit from [`LightningStore`](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.LightningStore " agentlightning.LightningStore") and override the methods to implement the storage logic. [`CollectionBasedLightningStore`](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.CollectionBasedLightningStore " agentlightning.CollectionBasedLightningStore") builds on collections to implement the full [`LightningStore`](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.LightningStore " agentlightning.LightningStore") API, including business logic like status transitions, watchdog health checks, and retry policies. 3. **Wrappers** – Cross-cutting concerns live in thin wrappers: * [`LightningStoreThreaded`](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.LightningStoreThreaded " agentlightning.LightningStoreThreaded") adds mutex-based thread safety. * [`LightningStoreServer`](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.LightningStoreServer " agentlightning.LightningStoreServer") / [`LightningStoreClient`](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.LightningStoreClient " agentlightning.LightningStoreClient") enable multi-process access over HTTP. Collections[¶](https://microsoft.github.io/agent-lightning/latest/deep-dive/store/#collections "Permanent link") ----------------------------------------------------------------------------------------------------------------- The collections layer provides storage primitives that [`CollectionBasedLightningStore`](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.CollectionBasedLightningStore " agentlightning.CollectionBasedLightningStore") builds upon. This separation keeps business logic (status transitions, watchdog, retries) in the store layer while allowing different backends to focus purely on persistence. The off-the-shelf implementations are [`InMemoryLightningCollections`](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.store.collection.InMemoryLightningCollections " agentlightning.store.collection.InMemoryLightningCollections") and [`MongoLightningCollections`](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.store.collection.mongo.MongoLightningCollections " agentlightning.store.collection.mongo.MongoLightningCollections") , which are the underlying collections for [`InMemoryLightningStore`](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.InMemoryLightningStore " agentlightning.InMemoryLightningStore") and [`MongoLightningStore`](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.store.mongo.MongoLightningStore " agentlightning.store.mongo.MongoLightningStore") , respectively. ### Collection Primitives[¶](https://microsoft.github.io/agent-lightning/latest/deep-dive/store/#collection-primitives "Permanent link") [`LightningCollections`](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.store.collection.LightningCollections " agentlightning.store.collection.LightningCollections") bundles three primitive types: | Primitive | Purpose | Methods | | --- | --- | --- | | [`Collection[T]`](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.store.collection.Collection " agentlightning.store.collection.Collection") | Indexed storage with primary keys | [`query()`](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.store.collection.Collection.query " query(filter=None, sort=None, limit=-1, offset=0)


async
")
, [`get()`](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.store.collection.Collection.get " get(filter=None, sort=None)


async
")
, [`insert()`](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.store.collection.Collection.insert " insert(items)


async
")
, [`update()`](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.store.collection.Collection.update " update(items, update_fields=None)


async
")
, [`upsert()`](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.store.collection.Collection.upsert " upsert(items, update_fields=None)


async
")
, [`delete()`](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.store.collection.Collection.delete " delete(items)


async
") | | [`Queue[T]`](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.store.collection.Queue " agentlightning.store.collection.Queue") | FIFO queue for task scheduling | [`enqueue()`](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.store.collection.Queue.enqueue " enqueue(items)


async
")
, [`dequeue()`](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.store.collection.Queue.dequeue " dequeue(limit=1)


async
")
, [`peek()`](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.store.collection.Queue.peek " peek(limit=1)


async
")
, [`size()`](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.store.collection.Queue.size " size()


async
") | | [`KeyValue[K, V]`](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.store.collection.KeyValue " agentlightning.store.collection.KeyValue") | Simple key-value store | [`get()`](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.store.collection.KeyValue.get " get(key, default=None)


async
")
, [`set()`](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.store.collection.KeyValue.set " set(key, value)


async
")
, [`inc()`](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.store.collection.KeyValue.inc " inc(key, amount)


async
")
, [`chmax()`](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.store.collection.KeyValue.chmax " chmax(key, value)


async
")
, [`pop()`](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.store.collection.KeyValue.pop " pop(key, default=None)


async
") | Every [`LightningCollections`](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.store.collection.LightningCollections " agentlightning.store.collection.LightningCollections") instance exposes these named collections: * `rollouts` – [`Collection[Rollout]`](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.store.collection.Collection " agentlightning.store.collection.Collection") keyed by `rollout_id` * `attempts` – [`Collection[Attempt]`](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.store.collection.Collection " agentlightning.store.collection.Collection") keyed by `(rollout_id, attempt_id)` * `spans` – [`Collection[Span]`](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.store.collection.Collection " agentlightning.store.collection.Collection") keyed by `(rollout_id, attempt_id, span_id)` * `resources` – [`Collection[ResourcesUpdate]`](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.store.collection.Collection " agentlightning.store.collection.Collection") keyed by `resources_id` * `workers` – [`Collection[Worker]`](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.store.collection.Collection " agentlightning.store.collection.Collection") keyed by `worker_id` * `rollout_queue` – [`Queue[str]`](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.store.collection.Queue " agentlightning.store.collection.Queue") holding rollout IDs awaiting execution * `span_sequence_ids` – [`KeyValue[str, int]`](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.store.collection.KeyValue " agentlightning.store.collection.KeyValue") tracking monotonic sequence counters ### Atomic Operations[¶](https://microsoft.github.io/agent-lightning/latest/deep-dive/store/#atomic-operations "Permanent link") Collections support atomic operations through the [`atomic()`](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.store.collection.LightningCollections.atomic " atomic(*, mode='rw', snapshot=False, commit=False, labels=None, **kwargs)") context manager: `[](https://microsoft.github.io/agent-lightning/latest/deep-dive/store/#__codelineno-1-1) async with collections.atomic(mode="rw", labels=["rollouts", "attempts"]) as ctx: [](https://microsoft.github.io/agent-lightning/latest/deep-dive/store/#__codelineno-1-2) rollout = await ctx.rollouts.get(filter={"rollout_id": {"exact": rollout_id}}) [](https://microsoft.github.io/agent-lightning/latest/deep-dive/store/#__codelineno-1-3) # modify and update within the same transaction [](https://microsoft.github.io/agent-lightning/latest/deep-dive/store/#__codelineno-1-4) await ctx.rollouts.update([updated_rollout])` The arguments passed to [`atomic()`](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.store.collection.LightningCollections.atomic " atomic(*, mode='rw', snapshot=False, commit=False, labels=None, **kwargs)") are quite arbitrary and flexible. Different implementations may have different interpretations of the arguments. For example, to [`InMemoryLightningCollections`](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.store.collection.InMemoryLightningCollections " agentlightning.store.collection.InMemoryLightningCollections") , the `mode` parameter controls locking behavior (`"r"` for read-only, `"rw"` for read-write), while `labels` specifies which collections to lock. Acquiring locks in sorted order prevents deadlocks when multiple operations run concurrently. ### Implementing a Custom Backend[¶](https://microsoft.github.io/agent-lightning/latest/deep-dive/store/#implementing-a-custom-backend "Permanent link") To add a new storage backend, implement [`LightningCollections`](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.store.collection.LightningCollections " agentlightning.store.collection.LightningCollections") : `[](https://microsoft.github.io/agent-lightning/latest/deep-dive/store/#__codelineno-2-1) from agentlightning.store.collection import LightningCollections, Collection, Queue, KeyValue [](https://microsoft.github.io/agent-lightning/latest/deep-dive/store/#__codelineno-2-2) [](https://microsoft.github.io/agent-lightning/latest/deep-dive/store/#__codelineno-2-3) class MyLightningCollections(LightningCollections): [](https://microsoft.github.io/agent-lightning/latest/deep-dive/store/#__codelineno-2-4) @property [](https://microsoft.github.io/agent-lightning/latest/deep-dive/store/#__codelineno-2-5) def rollouts(self) -> Collection[Rollout]: [](https://microsoft.github.io/agent-lightning/latest/deep-dive/store/#__codelineno-2-6) return self._rollouts # your implementation [](https://microsoft.github.io/agent-lightning/latest/deep-dive/store/#__codelineno-2-7) [](https://microsoft.github.io/agent-lightning/latest/deep-dive/store/#__codelineno-2-8) @property [](https://microsoft.github.io/agent-lightning/latest/deep-dive/store/#__codelineno-2-9) def rollout_queue(self) -> Queue[str]: [](https://microsoft.github.io/agent-lightning/latest/deep-dive/store/#__codelineno-2-10) return self._queue # your implementation [](https://microsoft.github.io/agent-lightning/latest/deep-dive/store/#__codelineno-2-11) [](https://microsoft.github.io/agent-lightning/latest/deep-dive/store/#__codelineno-2-12) # ... implement remaining properties [](https://microsoft.github.io/agent-lightning/latest/deep-dive/store/#__codelineno-2-13) [](https://microsoft.github.io/agent-lightning/latest/deep-dive/store/#__codelineno-2-14) async def atomic(self, *, mode, snapshot=False, labels=None, **kwargs): [](https://microsoft.github.io/agent-lightning/latest/deep-dive/store/#__codelineno-2-15) # provide transaction / locking semantics [](https://microsoft.github.io/agent-lightning/latest/deep-dive/store/#__codelineno-2-16) ...` Then instantiate your store: `[](https://microsoft.github.io/agent-lightning/latest/deep-dive/store/#__codelineno-3-1) from agentlightning.store.collection_based import CollectionBasedLightningStore [](https://microsoft.github.io/agent-lightning/latest/deep-dive/store/#__codelineno-3-2) [](https://microsoft.github.io/agent-lightning/latest/deep-dive/store/#__codelineno-3-3) store = CollectionBasedLightningStore(collections=MyLightningCollections())` The store layer handles all business logic; your collections just need to provide correct CRUD semantics. Collection-based Store Implementations[¶](https://microsoft.github.io/agent-lightning/latest/deep-dive/store/#collection-based-store-implementations "Permanent link") ----------------------------------------------------------------------------------------------------------------------------------------------------------------------- Agent-lightning ships with two collection-based store implementations: ### InMemoryLightningStore[¶](https://microsoft.github.io/agent-lightning/latest/deep-dive/store/#inmemorylightningstore "Permanent link") [`InMemoryLightningStore`](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.InMemoryLightningStore " agentlightning.InMemoryLightningStore") uses [`InMemoryLightningCollections`](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.store.collection.InMemoryLightningCollections " agentlightning.store.collection.InMemoryLightningCollections") backed by Python data structures. It supports **fast startup** with zero external dependencies—ideal for local development, CI, and unit tests. It also provides two lock modes, configurable between `"asyncio"` (single-thread, multiple coroutines) and `"thread"` (multi-threaded via [aiologic](https://github.com/x42005e1f/aiologic) ). [`InMemoryLightningCollections`](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.store.collection.InMemoryLightningCollections " agentlightning.store.collection.InMemoryLightningCollections") use nested dictionaries for O(1) primary-key lookup and `deque` for the task queue. ### MongoLightningStore[¶](https://microsoft.github.io/agent-lightning/latest/deep-dive/store/#mongolightningstore "Permanent link") [`MongoLightningStore`](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.store.mongo.MongoLightningStore " agentlightning.store.mongo.MongoLightningStore") uses [`MongoLightningCollections`](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.store.collection.mongo.MongoLightningCollections " agentlightning.store.collection.mongo.MongoLightningCollections") backed by MongoDB. It supports **persistent storage** suitable for production deployments and **multi-process safe** via database-level atomicity. It also supports **partition support** via `partition_id` for running multiple trainers against the same database. `[](https://microsoft.github.io/agent-lightning/latest/deep-dive/store/#__codelineno-4-1) from agentlightning.store.mongo import MongoLightningStore [](https://microsoft.github.io/agent-lightning/latest/deep-dive/store/#__codelineno-4-2) [](https://microsoft.github.io/agent-lightning/latest/deep-dive/store/#__codelineno-4-3) store = MongoLightningStore( [](https://microsoft.github.io/agent-lightning/latest/deep-dive/store/#__codelineno-4-4) mongo_uri="mongodb://localhost:27017/?replicaSet=rs0", [](https://microsoft.github.io/agent-lightning/latest/deep-dive/store/#__codelineno-4-5) database_name="agentlightning", [](https://microsoft.github.io/agent-lightning/latest/deep-dive/store/#__codelineno-4-6) partition_id="trainer-1", # optional: isolate data per trainer [](https://microsoft.github.io/agent-lightning/latest/deep-dive/store/#__codelineno-4-7) )` Note [`MongoLightningStore`](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.store.mongo.MongoLightningStore " agentlightning.store.mongo.MongoLightningStore") requires the `mongo` optional dependency. Install with `pip install agentlightning[mongo]`. ### Capabilities[¶](https://microsoft.github.io/agent-lightning/latest/deep-dive/store/#capabilities "Permanent link") [](https://microsoft.github.io/agent-lightning/latest/deep-dive/store/) Different stores have different capabilities. Check the [`capabilities`](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.LightningStore.capabilities " capabilities property ") property to understand what a store supports: | Capability | Description | InMemory | Mongo | Server | Client | | --- | --- | --- | --- | --- | --- | | `thread_safe` | Safe for concurrent access from multiple threads | configurable | ✓ | ✓ | ✓ | | `async_safe` | Safe for concurrent access from multiple coroutines | ✓ | ✓ | ✓ | ✓ | | `zero_copy` | Can be shared across processes without serialization | ✗ | ✓ | ✓ | ✓ | | `otlp_traces` | Exposes an OTLP-compatible `/v1/traces` endpoint | ✗ | ✗ | ✓ | ✓ | Thread Safety[¶](https://microsoft.github.io/agent-lightning/latest/deep-dive/store/#thread-safety "Permanent link") --------------------------------------------------------------------------------------------------------------------- Thread safety can be achieved at different layers: **At the collections layer**: [`InMemoryLightningCollections`](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.store.collection.InMemoryLightningCollections " agentlightning.store.collection.InMemoryLightningCollections") accepts a `lock_type` parameter: * `"asyncio"` – Uses per-event-loop `asyncio.Lock` for single-threaded, multi-coroutine scenarios. * `"thread"` – Uses `aiologic.Lock` for true multi-threaded access. **At the store layer**: [`LightningStoreThreaded`](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.LightningStoreThreaded " agentlightning.LightningStoreThreaded") wraps any [`LightningStore`](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.LightningStore " agentlightning.LightningStore") to add mutex-based thread safety: * Methods like [`start_rollout`](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.LightningStore.start_rollout " start_rollout(input, mode=None, resources_id=None, config=None, metadata=None, worker_id=None) async ") , [`enqueue_rollout`](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.LightningStore.enqueue_rollout " enqueue_rollout(input, mode=None, resources_id=None, config=None, metadata=None) async ") , [`update_attempt`](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.LightningStore.update_attempt " update_attempt(rollout_id, attempt_id, status=UNSET, worker_id=UNSET, last_heartbeat_time=UNSET, metadata=UNSET) async ") , [`add_span`](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.LightningStore.add_span " add_span(span) async ") , etc. are guarded by a lock. * Non-mutating, potentially blocking calls remain pass-through by design (e.g., [`wait_for_rollouts`](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.LightningStore.wait_for_rollouts " wait_for_rollouts(*, rollout_ids, timeout=None) async ") ), as they don't modify shared state and should not hold the lock for long periods. Database-based stores like [`MongoLightningStore`](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.store.mongo.MongoLightningStore " agentlightning.store.mongo.MongoLightningStore") are inherently thread-safe through database atomicity guarantees. Process Safety and Client-server Store[¶](https://microsoft.github.io/agent-lightning/latest/deep-dive/store/#process-safety-and-client-server-store "Permanent link") ----------------------------------------------------------------------------------------------------------------------------------------------------------------------- **[`LightningStoreServer`](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.LightningStoreServer " agentlightning.LightningStoreServer") ** wraps another underlying store and runs a FastAPI app to expose the store API over HTTP. [`LightningStoreClient`](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.LightningStoreClient " agentlightning.LightningStoreClient") is a small [`LightningStore`](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.LightningStore " agentlightning.LightningStore") implementation that talks to the HTTP API. Warning The server HTTP API is not considered a stable API at this moment. Users are encouraged to use the [`LightningStoreClient`](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.LightningStoreClient " agentlightning.LightningStoreClient") to communicate with the server as a stable interface. The server tracks the creator PID. In the owner process it delegates directly to the in-memory store; in other processes it lazily constructs a [`LightningStoreClient`](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.LightningStoreClient " agentlightning.LightningStoreClient") to talk to the HTTP API. This prevents accidental cross-process mutation of the wrong memory image. When the server is pickled (e.g., via `multiprocessing`), only the minimal fields are serialized, but **NOT** the FastAPI/uvicorn objects. Subprocesses won’t accidentally carry live server state. Forked subprocess should also use [`LightningStoreClient`](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.LightningStoreClient " agentlightning.LightningStoreClient") to communicate with the server in the main process. On the client side, the client retries network/5xx failures using a small backoff, and probes `/v1/agl/health` between attempts. Application exceptions inside the server are wrapped as HTTP 400 with a traceback—these are **not retried**. The client also maintains a **per-event-loop** `aiohttp.ClientSession` map so that tracer callbacks (often on separate loops/threads) don’t hang by reusing a session from another loop. Minimal lifecycle: `[](https://microsoft.github.io/agent-lightning/latest/deep-dive/store/#__codelineno-5-1) import agentlightning as agl [](https://microsoft.github.io/agent-lightning/latest/deep-dive/store/#__codelineno-5-2) [](https://microsoft.github.io/agent-lightning/latest/deep-dive/store/#__codelineno-5-3) # Server (owner process) [](https://microsoft.github.io/agent-lightning/latest/deep-dive/store/#__codelineno-5-4) in_memory_store = agl.InMemoryLightningStore() [](https://microsoft.github.io/agent-lightning/latest/deep-dive/store/#__codelineno-5-5) server = agl.LightningStoreServer(store=in_memory_store, host="0.0.0.0", port=4747) [](https://microsoft.github.io/agent-lightning/latest/deep-dive/store/#__codelineno-5-6) await server.start() # starts uvicorn in a daemon thread and waits for /health [](https://microsoft.github.io/agent-lightning/latest/deep-dive/store/#__codelineno-5-7) # or keep your own event loop and stop via await server.stop() [](https://microsoft.github.io/agent-lightning/latest/deep-dive/store/#__codelineno-5-8) # await server.run_forever() [](https://microsoft.github.io/agent-lightning/latest/deep-dive/store/#__codelineno-5-9) [](https://microsoft.github.io/agent-lightning/latest/deep-dive/store/#__codelineno-5-10) # Client (same or different process) [](https://microsoft.github.io/agent-lightning/latest/deep-dive/store/#__codelineno-5-11) client = agl.LightningStoreClient("http://localhost:4747") [](https://microsoft.github.io/agent-lightning/latest/deep-dive/store/#__codelineno-5-12) [](https://microsoft.github.io/agent-lightning/latest/deep-dive/store/#__codelineno-5-13) print(await client.query_rollouts(status_in=["queuing"])) [](https://microsoft.github.io/agent-lightning/latest/deep-dive/store/#__codelineno-5-14) [](https://microsoft.github.io/agent-lightning/latest/deep-dive/store/#__codelineno-5-15) await client.close() [](https://microsoft.github.io/agent-lightning/latest/deep-dive/store/#__codelineno-5-16) await server.stop()` Another approach is to use a dedicated command line to start a long running server process, possibly sharable across multiple processes. In the main process, you can always use [`LightningStoreClient`](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.LightningStoreClient " agentlightning.LightningStoreClient") to communicate with the server. `[](https://microsoft.github.io/agent-lightning/latest/deep-dive/store/#__codelineno-6-1) agl store --port 4747` Note [`LightningStoreClient.wait_for_rollouts`](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.LightningStoreClient.wait_for_rollouts " wait_for_rollouts(*, rollout_ids, timeout=None) async ") intentionally enforces a tiny timeout (≤ 0.1s) to avoid blocking event loops. Poll with short timeouts or compose with `asyncio.wait_for` at a higher layer. Back to top --- # Train SQL Agent with RL - Agent-lightning [Skip to content](https://microsoft.github.io/agent-lightning/latest/how-to/train-sql-agent/#train-sql-agent-with-agent-lightning-and-verl) ⚠️ You are viewing development documentation. For stable documentation, please visit [https://microsoft.github.io/agent-lightning/stable/](https://microsoft.github.io/agent-lightning/stable/) Train SQL Agent with Agent-lightning and VERL[¶](https://microsoft.github.io/agent-lightning/latest/how-to/train-sql-agent/#train-sql-agent-with-agent-lightning-and-verl "Permanent link") ============================================================================================================================================================================================ This walkthrough builds upon the **Agent-lightning SQL Agent** example and explains how the system components integrate: a **LangGraph-based SQL agent** wrapped as a [`LitAgent`](https://microsoft.github.io/agent-lightning/latest/reference/agent/#agentlightning.LitAgent " agentlightning.LitAgent") , the **[`VERL`](https://microsoft.github.io/agent-lightning/latest/algorithm-zoo/verl/#agentlightning.algorithm.verl.VERL " VERL") reinforcement learning (RL) algorithm**, and the **[`Trainer`](https://microsoft.github.io/agent-lightning/latest/reference/trainer/#agentlightning.Trainer " agentlightning.Trainer") **, which coordinates both training and debugging. The command-line interface in [`examples/spider/train_sql_agent.py`](https://github.com/microsoft/agent-lightning/tree/c746af2f76bebcae56007a59c223cdace411c89b/examples/spider/train_sql_agent.py) provides a complete runnable example. However, this document focuses on understanding the underlying architecture so you can effectively adapt the workflow to your own agents. SQL Agent Architecture[¶](https://microsoft.github.io/agent-lightning/latest/how-to/train-sql-agent/#sql-agent-architecture "Permanent link") ---------------------------------------------------------------------------------------------------------------------------------------------- Agent-lightning integrates seamlessly with various orchestration frameworks, including [Agent Framework](https://github.com/microsoft/agent-framework) , [AutoGen](https://github.com/microsoft/autogen) , [CrewAI](https://www.crewai.com/) , [LangGraph](https://github.com/langchain-ai/langgraph) , and the [OpenAI Agents SDK](https://github.com/openai/openai-agents-python) . It can also interoperate with custom Python logic. In this example, **LangGraph** defines a cyclic workflow that mirrors an analyst’s iterative SQL development process. The following graph (rendered directly from [`sql_agent.py`](https://github.com/microsoft/agent-lightning/tree/c746af2f76bebcae56007a59c223cdace411c89b/examples/spider/sql_agent.py) ) illustrates how the agent drafts, executes, critiques, and refines queries until a satisfactory result is achieved. --- config: flowchart: curve: linear --- graph LR; __start__([

__start__

]):::first write_query(write_query) execute_query(execute_query) check_query(check_query) rewrite_query(rewrite_query) __end__([

__end__

]):::last __start__ --> write_query; check_query -.-> __end__; check_query -.-> rewrite_query; execute_query --> check_query; rewrite_query --> execute_query; write_query --> execute_query; classDef default fill:#f2f2f2,line-height:1.2 classDef first fill-opacity:0 classDef last fill:#cccccc Note The workflow proceeds through the following stages: 1. **write\_query** – Generates an initial SQL query from the user’s question and the database schema. 2. **execute\_query** – Executes the generated query against the target database. 3. **check\_query** – Evaluates the query and its results (or errors) using a specialized prompt (`CHECK_QUERY_PROMPT`) to detect issues. 4. **rewrite\_query** – If issues are identified, the agent rewrites the query using feedback from the previous step and re-enters the loop. 5. **END** – The cycle terminates when the query is validated or the maximum iteration count (`max_turns`) is reached. Each _turn_ consists of one full loop through the `write_query`, `execute_query`, `check_query`, and (if applicable) `rewrite_query` stages. In this tutorial, **reinforcement learning (RL)** is used to optimize the `write_query` and `rewrite_query` stages. While the `check_query` step shares the same underlying LLM weights, its trace data is not used for learning. To keep the design modular and maintainable, it is recommended to define the LangGraph-based SQL Agent in a separate file and expose it via a builder function such as: `[](https://microsoft.github.io/agent-lightning/latest/how-to/train-sql-agent/#__codelineno-0-1) def build_langgraph_sql_agent( [](https://microsoft.github.io/agent-lightning/latest/how-to/train-sql-agent/#__codelineno-0-2) database_path: str, [](https://microsoft.github.io/agent-lightning/latest/how-to/train-sql-agent/#__codelineno-0-3) openai_base_url: str, [](https://microsoft.github.io/agent-lightning/latest/how-to/train-sql-agent/#__codelineno-0-4) model: str, [](https://microsoft.github.io/agent-lightning/latest/how-to/train-sql-agent/#__codelineno-0-5) sampling_parameters: Dict[str, Any], [](https://microsoft.github.io/agent-lightning/latest/how-to/train-sql-agent/#__codelineno-0-6) max_turns: int, [](https://microsoft.github.io/agent-lightning/latest/how-to/train-sql-agent/#__codelineno-0-7) truncate_length: int [](https://microsoft.github.io/agent-lightning/latest/how-to/train-sql-agent/#__codelineno-0-8) ): [](https://microsoft.github.io/agent-lightning/latest/how-to/train-sql-agent/#__codelineno-0-9) builder = StateGraph(State) [](https://microsoft.github.io/agent-lightning/latest/how-to/train-sql-agent/#__codelineno-0-10) builder.add_node(write_query) [](https://microsoft.github.io/agent-lightning/latest/how-to/train-sql-agent/#__codelineno-0-11) ... [](https://microsoft.github.io/agent-lightning/latest/how-to/train-sql-agent/#__codelineno-0-12) [](https://microsoft.github.io/agent-lightning/latest/how-to/train-sql-agent/#__codelineno-0-13) builder.add_edge(START, "write_query") [](https://microsoft.github.io/agent-lightning/latest/how-to/train-sql-agent/#__codelineno-0-14) ... [](https://microsoft.github.io/agent-lightning/latest/how-to/train-sql-agent/#__codelineno-0-15) [](https://microsoft.github.io/agent-lightning/latest/how-to/train-sql-agent/#__codelineno-0-16) return builder.compile().graph()` This approach isolates your LangGraph logic from Agent-lightning version changes, improving both readability and debuggability. Bridging LangGraph and Agent-lightning[¶](https://microsoft.github.io/agent-lightning/latest/how-to/train-sql-agent/#bridging-langgraph-and-agent-lightning "Permanent link") ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ Tip Keep [`sql_agent.py`](https://github.com/microsoft/agent-lightning/tree/c746af2f76bebcae56007a59c223cdace411c89b/examples/spider/sql_agent.py) open on the side while reading this section. This will help you understand how the code snippets shown here work in practice. The **`LitSQLAgent`** class defined in [`sql_agent.py`](https://github.com/microsoft/agent-lightning/tree/c746af2f76bebcae56007a59c223cdace411c89b/examples/spider/sql_agent.py) acts as the bridge. It subclasses [`agl.LitAgent`](https://microsoft.github.io/agent-lightning/latest/reference/agent/#agentlightning.LitAgent " agentlightning.LitAgent") , allowing the runner to provision shared resources (e.g., [LLMs](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.LLM " agentlightning.LLM") ) for each rollout. Below is a simplified illustration of the key logic (note: this is conceptual pseudocode; the actual implementation includes dataset-specific details): `[](https://microsoft.github.io/agent-lightning/latest/how-to/train-sql-agent/#__codelineno-1-1) class LitSQLAgent(agl.LitAgent[Dict[str, Any]]): [](https://microsoft.github.io/agent-lightning/latest/how-to/train-sql-agent/#__codelineno-1-2) [](https://microsoft.github.io/agent-lightning/latest/how-to/train-sql-agent/#__codelineno-1-3) def __init__(self, max_turns: int, truncate_length: int): [](https://microsoft.github.io/agent-lightning/latest/how-to/train-sql-agent/#__codelineno-1-4) # Every turn here refers to a full cycle of write/exe/check/rewrite [](https://microsoft.github.io/agent-lightning/latest/how-to/train-sql-agent/#__codelineno-1-5) self.max_turns = max_turns [](https://microsoft.github.io/agent-lightning/latest/how-to/train-sql-agent/#__codelineno-1-6) self.truncate_length = truncate_length [](https://microsoft.github.io/agent-lightning/latest/how-to/train-sql-agent/#__codelineno-1-7) [](https://microsoft.github.io/agent-lightning/latest/how-to/train-sql-agent/#__codelineno-1-8) def rollout( [](https://microsoft.github.io/agent-lightning/latest/how-to/train-sql-agent/#__codelineno-1-9) self, [](https://microsoft.github.io/agent-lightning/latest/how-to/train-sql-agent/#__codelineno-1-10) task: Dict[str, Any], [](https://microsoft.github.io/agent-lightning/latest/how-to/train-sql-agent/#__codelineno-1-11) resources: agl.NamedResources, [](https://microsoft.github.io/agent-lightning/latest/how-to/train-sql-agent/#__codelineno-1-12) rollout: agl.Rollout [](https://microsoft.github.io/agent-lightning/latest/how-to/train-sql-agent/#__codelineno-1-13) ) -> float | None: [](https://microsoft.github.io/agent-lightning/latest/how-to/train-sql-agent/#__codelineno-1-14) llm: agl.LLM = resources["main_llm"] [](https://microsoft.github.io/agent-lightning/latest/how-to/train-sql-agent/#__codelineno-1-15) agent = build_langgraph_sql_agent( [](https://microsoft.github.io/agent-lightning/latest/how-to/train-sql-agent/#__codelineno-1-16) database_path="sqlite:///" + task["db_id"], [](https://microsoft.github.io/agent-lightning/latest/how-to/train-sql-agent/#__codelineno-1-17) max_turns=self.max_turns, [](https://microsoft.github.io/agent-lightning/latest/how-to/train-sql-agent/#__codelineno-1-18) truncate_length=self.truncate_length, [](https://microsoft.github.io/agent-lightning/latest/how-to/train-sql-agent/#__codelineno-1-19) openai_base_url=llm.get_base_url(rollout.rollout_id, rollout.attempt.attempt_id), [](https://microsoft.github.io/agent-lightning/latest/how-to/train-sql-agent/#__codelineno-1-20) model=llm.model, [](https://microsoft.github.io/agent-lightning/latest/how-to/train-sql-agent/#__codelineno-1-21) sampling_parameters=llm.sampling_parameters, [](https://microsoft.github.io/agent-lightning/latest/how-to/train-sql-agent/#__codelineno-1-22) ) [](https://microsoft.github.io/agent-lightning/latest/how-to/train-sql-agent/#__codelineno-1-23) result = agent.invoke({"question": question}, { [](https://microsoft.github.io/agent-lightning/latest/how-to/train-sql-agent/#__codelineno-1-24) "callbacks": [self.tracer.get_langchain_handler()], [](https://microsoft.github.io/agent-lightning/latest/how-to/train-sql-agent/#__codelineno-1-25) "recursion_limit": 100, [](https://microsoft.github.io/agent-lightning/latest/how-to/train-sql-agent/#__codelineno-1-26) }) [](https://microsoft.github.io/agent-lightning/latest/how-to/train-sql-agent/#__codelineno-1-27) reward = evaluate_query(result["query"], ground_truth, db_path, raise_on_error=False) [](https://microsoft.github.io/agent-lightning/latest/how-to/train-sql-agent/#__codelineno-1-28) return reward` The `LitSQLAgent` serves as a lightweight wrapper around the LangGraph agent, providing the correct interface for the [`rollout`](https://microsoft.github.io/agent-lightning/latest/reference/agent/#agentlightning.LitAgent.rollout " rollout(task, resources, rollout)") method. It constructs the LangGraph agent, invokes it, and returns the evaluation result as a reward signal. The `"main_llm"` resource key is a convention between the agent and [VERL](https://microsoft.github.io/agent-lightning/latest/algorithm-zoo/verl/#agentlightning.algorithm.verl.VERL " VERL") . It is used to inject an OpenAI-compatible endpoint from the [VERL](https://microsoft.github.io/agent-lightning/latest/algorithm-zoo/verl/#agentlightning.algorithm.verl.VERL " VERL") algorithm during rollout. Two approaches are supported to use this [agentlightning.LLM](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.LLM " agentlightning.LLM") resource: 1. **Direct access** – Use [`llm.endpoint`](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.LLM.endpoint " endpoint instance-attribute ") for a simple integration (identical to the v0.1 example). 2. **Context-aware access** – Use [`get_base_url`](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.ProxyLLM.get_base_url " get_base_url(rollout_id, attempt_id)") with [`rollout.rollout_id`](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.Rollout.rollout_id " rollout_id instance-attribute ") and [`rollout.attempt.attempt_id`](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.Attempt.attempt_id " attempt_id instance-attribute ") . This approach enables per-caller trace attribution, improving trace collection per rollout or attempt when runner-side tracers are unavailable. For details, see [Working with Traces](https://microsoft.github.io/agent-lightning/latest/tutorials/traces/) . Reward Signal and Evaluation[¶](https://microsoft.github.io/agent-lightning/latest/how-to/train-sql-agent/#reward-signal-and-evaluation "Permanent link") ---------------------------------------------------------------------------------------------------------------------------------------------------------- The `evaluate_query` function provides the reward mechanism for RL training. In agent training, obtaining a consistent and meaningful reward signal is often challenging. Fortunately, this is simplified when using the [**Spider dataset**](https://yale-lily.github.io/spider) . The dataset includes ~8k samples containing natural-language questions, database schemas, and ground-truth SQL queries. Using the [**Spider evaluator**](https://github.com/taoyds/test-suite-sql-eval) , the agent's generated query is executed and compared to the ground-truth query on the target database. The two queries are considered equivalent if they produce identical execution results. Attention The ground-truth queries must **never** be exposed to the agent during training to prevent data leakage. In this setup, the reward is returned directly from the [`rollout`](https://microsoft.github.io/agent-lightning/latest/reference/agent/#agentlightning.LitAgent.rollout " rollout(task, resources, rollout)") method, enabling the runner to forward it back to the RL algorithm. Warning Avoid using [`emit_reward`](https://microsoft.github.io/agent-lightning/latest/reference/agent/#agentlightning.emit_reward " agentlightning.emit_reward(reward, *, primary_key=None, attributes=None, propagate=True)") in conjunction with returning a reward value. Doing both will cause the algorithm to receive duplicate reward signals, leading to inconsistent training behavior. Configuring VERL for Reinforcement Learning[¶](https://microsoft.github.io/agent-lightning/latest/how-to/train-sql-agent/#configuring-verl-for-reinforcement-learning "Permanent link") ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- View [`examples/spider/train_sql_agent.py`](https://github.com/microsoft/agent-lightning/tree/c746af2f76bebcae56007a59c223cdace411c89b/examples/spider/train_sql_agent.py) for a full reinforcement learning configuration, which is a plain Python dictionary. It mirrors (and actually _is_) the [shell arguments](https://verl.readthedocs.io/en/latest/index.html) used to launch training in the VERL framework but is easier to tweak programmatically: `[](https://microsoft.github.io/agent-lightning/latest/how-to/train-sql-agent/#__codelineno-2-1) verl_config: Dict[str, Any] = { [](https://microsoft.github.io/agent-lightning/latest/how-to/train-sql-agent/#__codelineno-2-2) "algorithm": {"adv_estimator": "grpo", "use_kl_in_reward": False}, [](https://microsoft.github.io/agent-lightning/latest/how-to/train-sql-agent/#__codelineno-2-3) "data": { [](https://microsoft.github.io/agent-lightning/latest/how-to/train-sql-agent/#__codelineno-2-4) # train_files and val_files are no longer needed here [](https://microsoft.github.io/agent-lightning/latest/how-to/train-sql-agent/#__codelineno-2-5) # because data are read in agl.Trainer [](https://microsoft.github.io/agent-lightning/latest/how-to/train-sql-agent/#__codelineno-2-6) ..., [](https://microsoft.github.io/agent-lightning/latest/how-to/train-sql-agent/#__codelineno-2-7) # Controls how many tasks are pooled per step [](https://microsoft.github.io/agent-lightning/latest/how-to/train-sql-agent/#__codelineno-2-8) # (multiplied by actor_rollout_ref.rollout.n) [](https://microsoft.github.io/agent-lightning/latest/how-to/train-sql-agent/#__codelineno-2-9) "train_batch_size": 32, [](https://microsoft.github.io/agent-lightning/latest/how-to/train-sql-agent/#__codelineno-2-10) # Prompt and responses larger than these lengths are truncated [](https://microsoft.github.io/agent-lightning/latest/how-to/train-sql-agent/#__codelineno-2-11) "max_prompt_length": 4096, [](https://microsoft.github.io/agent-lightning/latest/how-to/train-sql-agent/#__codelineno-2-12) "max_response_length": 2048, [](https://microsoft.github.io/agent-lightning/latest/how-to/train-sql-agent/#__codelineno-2-13) }, [](https://microsoft.github.io/agent-lightning/latest/how-to/train-sql-agent/#__codelineno-2-14) "actor_rollout_ref": { [](https://microsoft.github.io/agent-lightning/latest/how-to/train-sql-agent/#__codelineno-2-15) "rollout": { [](https://microsoft.github.io/agent-lightning/latest/how-to/train-sql-agent/#__codelineno-2-16) # Only vLLM is supported currently [](https://microsoft.github.io/agent-lightning/latest/how-to/train-sql-agent/#__codelineno-2-17) "name": "vllm", [](https://microsoft.github.io/agent-lightning/latest/how-to/train-sql-agent/#__codelineno-2-18) # Equals to group size of GRPO [](https://microsoft.github.io/agent-lightning/latest/how-to/train-sql-agent/#__codelineno-2-19) "n": 4, [](https://microsoft.github.io/agent-lightning/latest/how-to/train-sql-agent/#__codelineno-2-20) # Used to enable tool call parser in vLLM [](https://microsoft.github.io/agent-lightning/latest/how-to/train-sql-agent/#__codelineno-2-21) "multi_turn": {"format": "hermes"}, [](https://microsoft.github.io/agent-lightning/latest/how-to/train-sql-agent/#__codelineno-2-22) ... [](https://microsoft.github.io/agent-lightning/latest/how-to/train-sql-agent/#__codelineno-2-23) }, [](https://microsoft.github.io/agent-lightning/latest/how-to/train-sql-agent/#__codelineno-2-24) "actor": {"ppo_mini_batch_size": 32, "optim": {"lr": 1e-6}, ...}, [](https://microsoft.github.io/agent-lightning/latest/how-to/train-sql-agent/#__codelineno-2-25) "model": { [](https://microsoft.github.io/agent-lightning/latest/how-to/train-sql-agent/#__codelineno-2-26) # Config your preferred LLM here [](https://microsoft.github.io/agent-lightning/latest/how-to/train-sql-agent/#__codelineno-2-27) "path": "Qwen/Qwen2.5-Coder-1.5B-Instruct", [](https://microsoft.github.io/agent-lightning/latest/how-to/train-sql-agent/#__codelineno-2-28) ... [](https://microsoft.github.io/agent-lightning/latest/how-to/train-sql-agent/#__codelineno-2-29) }, [](https://microsoft.github.io/agent-lightning/latest/how-to/train-sql-agent/#__codelineno-2-30) }, [](https://microsoft.github.io/agent-lightning/latest/how-to/train-sql-agent/#__codelineno-2-31) "trainer": { [](https://microsoft.github.io/agent-lightning/latest/how-to/train-sql-agent/#__codelineno-2-32) "n_gpus_per_node": 1, [](https://microsoft.github.io/agent-lightning/latest/how-to/train-sql-agent/#__codelineno-2-33) # Validation once before training starts [](https://microsoft.github.io/agent-lightning/latest/how-to/train-sql-agent/#__codelineno-2-34) "val_before_train": True, [](https://microsoft.github.io/agent-lightning/latest/how-to/train-sql-agent/#__codelineno-2-35) # Validation every N training steps [](https://microsoft.github.io/agent-lightning/latest/how-to/train-sql-agent/#__codelineno-2-36) "test_freq": 32, [](https://microsoft.github.io/agent-lightning/latest/how-to/train-sql-agent/#__codelineno-2-37) # Save checkpoints every N training steps [](https://microsoft.github.io/agent-lightning/latest/how-to/train-sql-agent/#__codelineno-2-38) "save_freq": 64, [](https://microsoft.github.io/agent-lightning/latest/how-to/train-sql-agent/#__codelineno-2-39) # Go through the train dataset this many times [](https://microsoft.github.io/agent-lightning/latest/how-to/train-sql-agent/#__codelineno-2-40) "total_epochs": 2 [](https://microsoft.github.io/agent-lightning/latest/how-to/train-sql-agent/#__codelineno-2-41) }, [](https://microsoft.github.io/agent-lightning/latest/how-to/train-sql-agent/#__codelineno-2-42) }` This is equivalent to the following CLI invocation: `[](https://microsoft.github.io/agent-lightning/latest/how-to/train-sql-agent/#__codelineno-3-1) python3 -m verl.trainer.main_ppo \ [](https://microsoft.github.io/agent-lightning/latest/how-to/train-sql-agent/#__codelineno-3-2) algorithm.adv_estimator=grpo \ [](https://microsoft.github.io/agent-lightning/latest/how-to/train-sql-agent/#__codelineno-3-3) algorithm.use_kl_in_reward=False \ [](https://microsoft.github.io/agent-lightning/latest/how-to/train-sql-agent/#__codelineno-3-4) data.train_batch_size=32 \ [](https://microsoft.github.io/agent-lightning/latest/how-to/train-sql-agent/#__codelineno-3-5) data.max_prompt_length=4096 \ [](https://microsoft.github.io/agent-lightning/latest/how-to/train-sql-agent/#__codelineno-3-6) data.max_response_length=2048 \ [](https://microsoft.github.io/agent-lightning/latest/how-to/train-sql-agent/#__codelineno-3-7) actor_rollout_ref.rollout.name=vllm \ [](https://microsoft.github.io/agent-lightning/latest/how-to/train-sql-agent/#__codelineno-3-8) actor_rollout_ref.rollout.n=4 \ [](https://microsoft.github.io/agent-lightning/latest/how-to/train-sql-agent/#__codelineno-3-9) actor_rollout_ref.rollout.multi_turn.format=hermes \ [](https://microsoft.github.io/agent-lightning/latest/how-to/train-sql-agent/#__codelineno-3-10) actor_rollout_ref.actor.ppo_mini_batch_size=32 \ [](https://microsoft.github.io/agent-lightning/latest/how-to/train-sql-agent/#__codelineno-3-11) actor_rollout_ref.actor.optim.lr=1e-6 \ [](https://microsoft.github.io/agent-lightning/latest/how-to/train-sql-agent/#__codelineno-3-12) actor_rollout_ref.model.path=Qwen/Qwen2.5-Coder-1.5B-Instruct \ [](https://microsoft.github.io/agent-lightning/latest/how-to/train-sql-agent/#__codelineno-3-13) trainer.n_gpus_per_node=1 \ [](https://microsoft.github.io/agent-lightning/latest/how-to/train-sql-agent/#__codelineno-3-14) trainer.val_before_train=True \ [](https://microsoft.github.io/agent-lightning/latest/how-to/train-sql-agent/#__codelineno-3-15) trainer.test_freq=32 \ [](https://microsoft.github.io/agent-lightning/latest/how-to/train-sql-agent/#__codelineno-3-16) trainer.save_freq=64 \ [](https://microsoft.github.io/agent-lightning/latest/how-to/train-sql-agent/#__codelineno-3-17) trainer.total_epochs=2` Warning We used to provide a CLI called `python -m agentlightning.verl` to launch training in v0.1. This is no longer the recommended approach. Instead, use [`agl.Trainer`](https://microsoft.github.io/agent-lightning/latest/reference/trainer/#agentlightning.Trainer " agentlightning.Trainer") to run VERL and agent runners together, or follow the [debugging tutorial](https://microsoft.github.io/agent-lightning/latest/tutorials/debug/) if you want an isolated experience similar to v0.1. Orchestrating Training with [`Trainer`](https://microsoft.github.io/agent-lightning/latest/reference/trainer/#agentlightning.Trainer " agentlightning.Trainer") [¶](https://microsoft.github.io/agent-lightning/latest/how-to/train-sql-agent/#orchestrating-training-with-trainer "Permanent link") ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- [`Trainer`](https://microsoft.github.io/agent-lightning/latest/reference/trainer/#agentlightning.Trainer " agentlightning.Trainer") is the high-level orchestrator that integrates the agent, algorithm, dataset, and distributed runners. The key benefits of using the [`Trainer`](https://microsoft.github.io/agent-lightning/latest/reference/trainer/#agentlightning.Trainer " agentlightning.Trainer") are: 1. It allows you to launch everything with a single line of code: `trainer.fit(...)`. 2. It exposes configuration options such as `n_runners` to control parallelism and `adapter` to define how algorithms interpret the trace data produced by the agent. An example usage is shown below: `[](https://microsoft.github.io/agent-lightning/latest/how-to/train-sql-agent/#__codelineno-4-1) import agentlightning as agl [](https://microsoft.github.io/agent-lightning/latest/how-to/train-sql-agent/#__codelineno-4-2) [](https://microsoft.github.io/agent-lightning/latest/how-to/train-sql-agent/#__codelineno-4-3) agent = LitSQLAgent() [](https://microsoft.github.io/agent-lightning/latest/how-to/train-sql-agent/#__codelineno-4-4) algorithm = agl.VERL(verl_config) [](https://microsoft.github.io/agent-lightning/latest/how-to/train-sql-agent/#__codelineno-4-5) trainer = agl.Trainer( [](https://microsoft.github.io/agent-lightning/latest/how-to/train-sql-agent/#__codelineno-4-6) n_runners=10, [](https://microsoft.github.io/agent-lightning/latest/how-to/train-sql-agent/#__codelineno-4-7) algorithm=algorithm, [](https://microsoft.github.io/agent-lightning/latest/how-to/train-sql-agent/#__codelineno-4-8) adapter={"agent_match": active_agent}, [](https://microsoft.github.io/agent-lightning/latest/how-to/train-sql-agent/#__codelineno-4-9) ) [](https://microsoft.github.io/agent-lightning/latest/how-to/train-sql-agent/#__codelineno-4-10) train_data = pd.read_parquet("data/train_spider.parquet").to_dict("records") [](https://microsoft.github.io/agent-lightning/latest/how-to/train-sql-agent/#__codelineno-4-11) val_data = pd.read_parquet("data/test_dev_500.parquet").to_dict("records") [](https://microsoft.github.io/agent-lightning/latest/how-to/train-sql-agent/#__codelineno-4-12) trainer.fit(agent, train_dataset=train_data, val_dataset=val_data)` First, `agl.VERL(verl_config)` launches the [`VERL`](https://microsoft.github.io/agent-lightning/latest/algorithm-zoo/verl/#agentlightning.algorithm.verl.VERL " VERL") algorithm and its OpenAI-compatible proxy. The `train_data` and `val_data` are passed into [`VERL`](https://microsoft.github.io/agent-lightning/latest/algorithm-zoo/verl/#agentlightning.algorithm.verl.VERL " VERL") , which enqueues tasks to a centralized task queue managed by the [`LightningStore`](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.LightningStore " agentlightning.LightningStore") , accessible to all runners. When [`Trainer.fit`](https://microsoft.github.io/agent-lightning/latest/reference/trainer/#agentlightning.Trainer.fit " fit(agent, train_dataset=None, *, val_dataset=None)") is called, it launches 10 concurrent runners (as specified by `n_runners=10`). Each runner pulls tasks from the centralized task queue, executes the agent’s [`rollout`](https://microsoft.github.io/agent-lightning/latest/reference/agent/#agentlightning.LitAgent.rollout " rollout(task, resources, rollout)") method, collects traces, and returns rewards to VERL for training. The [`Adapter`](https://microsoft.github.io/agent-lightning/latest/reference/algorithm/#agentlightning.Adapter " agentlightning.Adapter") , as discussed earlier, is used at the algorithm side, and receives the traces emitted by the agent and runners. The `agent_match` parameter ensures [`VERL`](https://microsoft.github.io/agent-lightning/latest/algorithm-zoo/verl/#agentlightning.algorithm.verl.VERL " VERL") only ingests spans from the specific agent you want to optimize. In the example above, there are at least three agents—`write_query`, `rewrite_query`, and `check_query`. By setting `agent_match` to a regex like `"write"`, both `write_query` and `rewrite_query` agents are optimized simultaneously. You can also set it to `"write|check"` or `None` to include all agents if desired. Dry-Run the Pipeline with [`Trainer.dev`](https://microsoft.github.io/agent-lightning/latest/reference/trainer/#agentlightning.Trainer.dev " dev(agent, train_dataset=None, *, val_dataset=None)") [¶](https://microsoft.github.io/agent-lightning/latest/how-to/train-sql-agent/#dry-run-the-pipeline-with-trainerdev "Permanent link") ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Before committing hours of GPU time, you can **dry-run** the agent with [`Trainer.dev()`](https://microsoft.github.io/agent-lightning/latest/reference/trainer/#agentlightning.Trainer.dev " dev(agent, train_dataset=None, *, val_dataset=None)") . This method swaps in the lightweight [`Baseline`](https://microsoft.github.io/agent-lightning/latest/reference/algorithm/#agentlightning.Baseline " agentlightning.Baseline") algorithm, enqueues up to ten tasks, and prints every span emitted by the agent. Because it uses the same runner stack as full training, it’s ideal for verifying database connections and LangGraph control flow. To begin, the agent needs a valid OpenAI-compatible endpoint since VERL is not active in this mode. You can use OpenAI’s official API or your own local LLM endpoint. Wrap it as follows: `[](https://microsoft.github.io/agent-lightning/latest/how-to/train-sql-agent/#__codelineno-5-1) trainer = agl.Trainer( [](https://microsoft.github.io/agent-lightning/latest/how-to/train-sql-agent/#__codelineno-5-2) n_workers=1, [](https://microsoft.github.io/agent-lightning/latest/how-to/train-sql-agent/#__codelineno-5-3) initial_resources={ [](https://microsoft.github.io/agent-lightning/latest/how-to/train-sql-agent/#__codelineno-5-4) "main_llm": agl.LLM( [](https://microsoft.github.io/agent-lightning/latest/how-to/train-sql-agent/#__codelineno-5-5) endpoint=os.environ["OPENAI_API_BASE"], [](https://microsoft.github.io/agent-lightning/latest/how-to/train-sql-agent/#__codelineno-5-6) model="gpt-4.1-nano", [](https://microsoft.github.io/agent-lightning/latest/how-to/train-sql-agent/#__codelineno-5-7) sampling_parameters={"temperature": 0.7}, [](https://microsoft.github.io/agent-lightning/latest/how-to/train-sql-agent/#__codelineno-5-8) ) [](https://microsoft.github.io/agent-lightning/latest/how-to/train-sql-agent/#__codelineno-5-9) }, [](https://microsoft.github.io/agent-lightning/latest/how-to/train-sql-agent/#__codelineno-5-10) )` Then, call [`trainer.dev(...)`](https://microsoft.github.io/agent-lightning/latest/reference/trainer/#agentlightning.Trainer.dev " dev(agent, train_dataset=None, *, val_dataset=None)") with a small number of tasks: `[](https://microsoft.github.io/agent-lightning/latest/how-to/train-sql-agent/#__codelineno-6-1) dev_data = pd.read_parquet("data/test_dev_500.parquet").to_dict("records")[:10] [](https://microsoft.github.io/agent-lightning/latest/how-to/train-sql-agent/#__codelineno-6-2) trainer.dev(agent, dev_dataset=dev_data)` Run this in a Python session or adapt your script to include a `--dev` flag. Once the spans appear healthy and the rewards are non-zero, switch back to [`trainer.fit(...)`](https://microsoft.github.io/agent-lightning/latest/reference/trainer/#agentlightning.Trainer.fit " fit(agent, train_dataset=None, *, val_dataset=None)") for full RL training. See the [debugging tutorial](https://microsoft.github.io/agent-lightning/latest/tutorials/debug/) for more tips on how to debug the agent. Running the Sample Code[¶](https://microsoft.github.io/agent-lightning/latest/how-to/train-sql-agent/#running-the-sample-code "Permanent link") ------------------------------------------------------------------------------------------------------------------------------------------------ The following tutorial explains how to run the complete example in [`examples/spider`](https://github.com/microsoft/agent-lightning/tree/c746af2f76bebcae56007a59c223cdace411c89b/examples/spider) . ### Dataset[¶](https://microsoft.github.io/agent-lightning/latest/how-to/train-sql-agent/#dataset "Permanent link") The trainer expects three Parquet files inside `examples/spider/data`: `train_spider.parquet`, `test_dev_500.parquet`, and `test_dev.parquet`. Download the curated dataset bundle provided with the repository: `[](https://microsoft.github.io/agent-lightning/latest/how-to/train-sql-agent/#__codelineno-7-1) cd examples/spider [](https://microsoft.github.io/agent-lightning/latest/how-to/train-sql-agent/#__codelineno-7-2) pip install gdown # included in the 'experiment' optional dependency [](https://microsoft.github.io/agent-lightning/latest/how-to/train-sql-agent/#__codelineno-7-3) gdown --fuzzy https://drive.google.com/file/d/1oi9J1jZP9TyM35L85CL3qeGWl2jqlnL6/view [](https://microsoft.github.io/agent-lightning/latest/how-to/train-sql-agent/#__codelineno-7-4) unzip -q spider-data.zip -d data [](https://microsoft.github.io/agent-lightning/latest/how-to/train-sql-agent/#__codelineno-7-5) rm spider-data.zip` If you prefer to generate the files yourself, download [Spider 1.0](https://yale-lily.github.io/spider) and run: `[](https://microsoft.github.io/agent-lightning/latest/how-to/train-sql-agent/#__codelineno-8-1) python spider_eval/convert_dataset.py` Set `VERL_SPIDER_DATA_DIR` if you store the dataset outside the default `data` directory. ### Dependencies[¶](https://microsoft.github.io/agent-lightning/latest/how-to/train-sql-agent/#dependencies "Permanent link") Create a clean virtual environment, activate it, and install Agent-lightning with the VERL extras required by [this tutorial](https://microsoft.github.io/agent-lightning/latest/tutorials/installation/) . Install LangChain-related dependencies as needed. For full training profiles, plan to use a GPU with at least **40 GB** of memory. ### Launch Training[¶](https://microsoft.github.io/agent-lightning/latest/how-to/train-sql-agent/#launch-training "Permanent link") From [`examples/spider`](https://github.com/microsoft/agent-lightning/tree/c746af2f76bebcae56007a59c223cdace411c89b/examples/spider) , run one of the helper scripts depending on your model preference: `[](https://microsoft.github.io/agent-lightning/latest/how-to/train-sql-agent/#__codelineno-9-1) python train_sql_agent.py qwen # Default Qwen-2.5-Coder-1.5B run [](https://microsoft.github.io/agent-lightning/latest/how-to/train-sql-agent/#__codelineno-9-2) python train_sql_agent.py llama # LLaMA-3.2-1B with llama3_json tool parser` The script instantiates `LitSQLAgent` and launches [`trainer.fit`](https://microsoft.github.io/agent-lightning/latest/reference/trainer/#agentlightning.Trainer.fit " fit(agent, train_dataset=None, *, val_dataset=None)") . Provide `--active-agent my_agent_variant` if you only want to train one of the agents in the graph. For the LLaMA profile, export an `HF_TOKEN` before running so VERL can download the model weights. Troubleshooting If you have got some Ray worker errors on either `WANDB_API_KEY` not set, or `HF_TOKEN` not set, or data not found, please try to restart the Ray cluster with the helper script: [scripts/restart\_ray.sh](https://github.com/microsoft/agent-lightning/tree/c746af2f76bebcae56007a59c223cdace411c89b/scripts/restart_ray.sh) , which essentially stops the ray cluster if any, and starts a new one: `[](https://microsoft.github.io/agent-lightning/latest/how-to/train-sql-agent/#__codelineno-10-1) env RAY_DEBUG=legacy HYDRA_FULL_ERROR=1 VLLM_USE_V1=1 ray start --head --dashboard-host=0.0.0.0` Launching Training with NPUs The example also supports running with **Huawei Ascend NPUs**. This feature is contributed by [Teams from Huawei](https://github.com/microsoft/agent-lightning/pull/272) . To use it, resort to the function `config_train_npu` in the script. **Hardware Supported:** Atlas 200T A2 Box16, Atlas 900 A2 PODc, Atlas 800T A3. At least **a single 40GB NPU** is required to run the **Qwen2.5-Coder-1.5B-Instruct** model. **Environment Setup:** Python 3.11.13, CANN 8.2.RC1, torch 2.7.1+cpu, torch\_npu 2.7.1.dev20250724. For basic environment preparation, please refer to this [document](https://gitcode.com/Ascend/pytorch) . Before installing dependencies, configure the following pip mirrors: `[](https://microsoft.github.io/agent-lightning/latest/how-to/train-sql-agent/#__codelineno-11-1) pip config set global.index-url http://repo.huaweicloud.com/repository/pypi/simple [](https://microsoft.github.io/agent-lightning/latest/how-to/train-sql-agent/#__codelineno-11-2) pip config set global.extra-index-url "https://download.pytorch.org/whl/cpu/ https://mirrors.huaweicloud.com/ascend/repos/pypi"` Then install vLLM, vLLM-Ascend and VERL: `[](https://microsoft.github.io/agent-lightning/latest/how-to/train-sql-agent/#__codelineno-12-1) pip install vllm==0.10.0 --trusted-host repo.huaweicloud.com [](https://microsoft.github.io/agent-lightning/latest/how-to/train-sql-agent/#__codelineno-12-2) pip install vllm-Ascend==0.10.0rc1 --trusted-host repo.huaweicloud.com [](https://microsoft.github.io/agent-lightning/latest/how-to/train-sql-agent/#__codelineno-12-3) pip install verl==0.5.0` To ensure the VERL framework runs correctly on NPU, add the following lines to `verl/utils/vllm_utils.py`: `[](https://microsoft.github.io/agent-lightning/latest/how-to/train-sql-agent/#__codelineno-13-1) from vllm_ascend.patch import platform [](https://microsoft.github.io/agent-lightning/latest/how-to/train-sql-agent/#__codelineno-13-2) from vllm_ascend.patch import worker` See the following reference for more details: [https://github.com/vllm-project/vllm-ascend/issues/1776](https://github.com/vllm-project/vllm-ascend/issues/1776) . After the above dependencies have been installed, from [`examples/spider`](https://github.com/microsoft/agent-lightning/tree/c746af2f76bebcae56007a59c223cdace411c89b/examples/spider) run the following script command: `[](https://microsoft.github.io/agent-lightning/latest/how-to/train-sql-agent/#__codelineno-14-1) python train_sql_agent.py npu` ### Debugging the Agent without VERL[¶](https://microsoft.github.io/agent-lightning/latest/how-to/train-sql-agent/#debugging-the-agent-without-verl "Permanent link") [`sql_agent.py`](https://github.com/microsoft/agent-lightning/tree/c746af2f76bebcae56007a59c223cdace411c89b/examples/spider/sql_agent.py) also provides a `debug_sql_agent()` helper to run the LangGraph workflow directly against a local or hosted OpenAI-compatible endpoint before using VERL. Set the following environment variables, then execute the file: `[](https://microsoft.github.io/agent-lightning/latest/how-to/train-sql-agent/#__codelineno-15-1) export OPENAI_API_BASE= [](https://microsoft.github.io/agent-lightning/latest/how-to/train-sql-agent/#__codelineno-15-2) export OPENAI_API_KEY= [](https://microsoft.github.io/agent-lightning/latest/how-to/train-sql-agent/#__codelineno-15-3) cd examples/spider [](https://microsoft.github.io/agent-lightning/latest/how-to/train-sql-agent/#__codelineno-15-4) python sql_agent.py` This allows you to verify that the workflow and prompts behave as expected before reinforcement learning is introduced. ### Evaluation[¶](https://microsoft.github.io/agent-lightning/latest/how-to/train-sql-agent/#evaluation "Permanent link") The following results were obtained by running `python train_sql_agent.py qwen` on a single 80 GB GPU. Training completes in approximately **12 hours**. The training curves below are smoothed by aggregating every 16 steps for better visualization. Additional evaluation results were collected with a legacy version — Agent-lightning v0.1.1, `verl==0.5.0`, and `vllm==0.10.0`. You can find them in this write-up: [Training AI Agents to Write and Self-Correct SQL with Reinforcement Learning](https://medium.com/@yugez/training-ai-agents-to-write-and-self-correct-sql-with-reinforcement-learning-571ed31281ad) Back to top --- # Write the First Algorithm - Agent-lightning [Skip to content](https://microsoft.github.io/agent-lightning/latest/how-to/write-first-algorithm/#write-the-first-algorithm-with-agent-lightning) ⚠️ You are viewing development documentation. For stable documentation, please visit [https://microsoft.github.io/agent-lightning/stable/](https://microsoft.github.io/agent-lightning/stable/) Write the First Algorithm with Agent-lightning[¶](https://microsoft.github.io/agent-lightning/latest/how-to/write-first-algorithm/#write-the-first-algorithm-with-agent-lightning "Permanent link") ==================================================================================================================================================================================================== In the [first tutorial](https://microsoft.github.io/agent-lightning/latest/how-to/train-first-agent/) , "Train the First Agent," we introduced the [Trainer](https://microsoft.github.io/agent-lightning/latest/reference/trainer/#agentlightning.Trainer " agentlightning.Trainer") and showed how to use a pre-built algorithm like **Automatic Prompt Optimization (APO)** to improve an agent's performance. The [Trainer](https://microsoft.github.io/agent-lightning/latest/reference/trainer/#agentlightning.Trainer " agentlightning.Trainer") handled all the complex interactions, letting us focus on the agent's logic. Now, we'll go a step deeper. What if you have a unique training idea that doesn't fit a standard algorithm? This tutorial will show you how to write your own custom algorithm from scratch. We'll build a simple algorithm that systematically tests a list of prompt templates and identifies the one with the highest reward. By the end, you'll understand the core mechanics of how the **Algorithm**, **Runner**, and a new component, the **Store**, work together to create the powerful training loop at the heart of Agent-lightning. Tip This tutorial helps you build a basic understanding of how to interact with Agent-lightning's core components. It's recommended that all users customizing algorithms should read this tutorial, even for those who are not planning to do prompt optimization. Core Concepts for Training[¶](https://microsoft.github.io/agent-lightning/latest/how-to/write-first-algorithm/#core-concepts-for-training "Permanent link") ------------------------------------------------------------------------------------------------------------------------------------------------------------ Before diving into the [LightningStore](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.LightningStore " agentlightning.LightningStore") , let's define two key concepts that are central to any training process in Agent-lightning: **Resources** and the **Tracer**. ### Resources: The Tunable Assets[¶](https://microsoft.github.io/agent-lightning/latest/how-to/write-first-algorithm/#resources-the-tunable-assets "Permanent link") [](https://microsoft.github.io/agent-lightning/latest/how-to/write-first-algorithm/) **Resources** are the assets your algorithm is trying to improve. Think of them as the "recipe" an agent uses to perform its task. This recipe can be: * A **prompt template** that guides an LLM. * The **weights** of a machine learning model. * Any other configuration or data your agent needs. The algorithm's job is to run experiments and iteratively update these resources to find the best-performing version. ### Tracer: The Data Collector[¶](https://microsoft.github.io/agent-lightning/latest/how-to/write-first-algorithm/#tracer-the-data-collector "Permanent link") How does the algorithm know if a change was an improvement? It needs data. This is where the **Tracer** comes in. The Tracer automatically **instruments** (aka modifies / patches) the agent's code. This means it watches for important events, like an LLM call, a tool being used, or **reward signals**, and records a detailed log of what happened. Each of these logs is called a **Span** (which has already been introduced in the [last tutorial](https://microsoft.github.io/agent-lightning/latest/how-to/train-first-agent/) ). A collection of spans from a single task execution gives the algorithm a complete, step-by-step trace of the agent's behavior, which is essential for learning and making improvements. Our default tracer is built on the AgentOps SDK to support instrumenting code written in various Agent/non-agent frameworks. The Central Hub: The LightningStore[¶](https://microsoft.github.io/agent-lightning/latest/how-to/write-first-algorithm/#the-central-hub-the-lightningstore "Permanent link") ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Now, where do all these resources, tasks, and spans live? They are all managed by the **LightningStore**. The LightningStore acts as the central database and message queue for the entire system. It's the single source of truth that decouples the Algorithm from the Runners. Note In the [last tutorial](https://microsoft.github.io/agent-lightning/latest/how-to/train-first-agent/) we simplified the training loop, saying the Algorithm and Agent communicate "via the Trainer." That's true at a high level, but the component that makes it all possible is actually the **LightningStore**. * The **Algorithm** connects to the Store to `enqueue_rollout` (tasks) and `update_resources` (like new prompt templates). It also queries the Store to retrieve the resulting spans and rewards from completed rollouts. * The **Runners** connect to the Store to `dequeue_rollout` (polling for available tasks). After executing a task, they use the `Tracer` to write the resulting spans and status updates back to the Store. This architecture is key to Agent-lightning's scalability. Since the Algorithm and Runners only talk to the Store, they can run in different processes or even on different machines. ![Store Architecture](https://microsoft.github.io/agent-lightning/latest/assets/store-api-visualized.svg) A Mental Model of What the Store Contains The [LightningStore](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.LightningStore " agentlightning.LightningStore") isn't just a simple database; it's an organized system for managing the entire training lifecycle. Here's what it keeps track of: * **Task Queue**: A queue of pending **Rollouts** waiting for a Runner to pick them up, interactable via `enqueue_rollout` and `dequeue_rollout`. * **Rollouts**: The record of a single task. A rollout contains metadata about the task and tracks all **Attempts** to complete it, interactable via `query_rollouts` and `wait_for_rollouts`. * **Attempts**: A single execution of a rollout. If an attempt fails (e.g., due to a network error), the Store can automatically schedules a retry if it's configured. Each attempt is linked to its parent rollout and contains the status and timing information. The rollout status is [synced](https://microsoft.github.io/agent-lightning/latest/deep-dive/store/) with its children's status. **For beginners, you can assume each rollout has only one attempt unless you have explicitly configure the retry.** * **Spans**: The detailed, structured logs generated by the `Tracer` during an attempt. Each span is linked to its parent attempt and rollout. * **Resources**: A versioned collection of the assets (like prompt templates) that the algorithm creates. Each rollout is linked to the specific version of the resources it should use. Building a Custom Algorithm[¶](https://microsoft.github.io/agent-lightning/latest/how-to/write-first-algorithm/#building-a-custom-algorithm "Permanent link") -------------------------------------------------------------------------------------------------------------------------------------------------------------- Let's build an algorithm that finds the best system prompt from a predefined list. The logic is straightforward: 1. Start with a list of candidate prompt templates. 2. For each template, create a "resource" bundle in the Store. 3. Enqueue a rollout (a task), telling the Runner to use this specific resource. 4. Wait for a Runner to pick up the task and complete it. 5. Query the Store to get the final reward from the rollout's spans. 6. After testing all templates, compare the rewards and declare the best one. We can implement this as a simple Python function that interacts directly with the [LightningStore](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.LightningStore " agentlightning.LightningStore") . `[](https://microsoft.github.io/agent-lightning/latest/how-to/write-first-algorithm/#__codelineno-0-1) async def find_best_prompt(store, prompts_to_test, task_input): [](https://microsoft.github.io/agent-lightning/latest/how-to/write-first-algorithm/#__codelineno-0-2) """A simple algorithm to find the best prompt from a list.""" [](https://microsoft.github.io/agent-lightning/latest/how-to/write-first-algorithm/#__codelineno-0-3) results = [] [](https://microsoft.github.io/agent-lightning/latest/how-to/write-first-algorithm/#__codelineno-0-4) [](https://microsoft.github.io/agent-lightning/latest/how-to/write-first-algorithm/#__codelineno-0-5) # Iterate through each prompt to test it [](https://microsoft.github.io/agent-lightning/latest/how-to/write-first-algorithm/#__codelineno-0-6) for prompt in prompts_to_test: [](https://microsoft.github.io/agent-lightning/latest/how-to/write-first-algorithm/#__codelineno-0-7) print(f"[Algo] Updating prompt template to: '{prompt}'") [](https://microsoft.github.io/agent-lightning/latest/how-to/write-first-algorithm/#__codelineno-0-8) [](https://microsoft.github.io/agent-lightning/latest/how-to/write-first-algorithm/#__codelineno-0-9) # 1. Update the resources in the store with the new prompt [](https://microsoft.github.io/agent-lightning/latest/how-to/write-first-algorithm/#__codelineno-0-10) resources_update = await store.add_resources( [](https://microsoft.github.io/agent-lightning/latest/how-to/write-first-algorithm/#__codelineno-0-11) resources={"prompt_template": prompt} [](https://microsoft.github.io/agent-lightning/latest/how-to/write-first-algorithm/#__codelineno-0-12) ) [](https://microsoft.github.io/agent-lightning/latest/how-to/write-first-algorithm/#__codelineno-0-13) [](https://microsoft.github.io/agent-lightning/latest/how-to/write-first-algorithm/#__codelineno-0-14) # 2. Enqueue a rollout task for a runner to execute [](https://microsoft.github.io/agent-lightning/latest/how-to/write-first-algorithm/#__codelineno-0-15) print("[Algo] Queuing task for clients...") [](https://microsoft.github.io/agent-lightning/latest/how-to/write-first-algorithm/#__codelineno-0-16) rollout = await store.enqueue_rollout( [](https://microsoft.github.io/agent-lightning/latest/how-to/write-first-algorithm/#__codelineno-0-17) input=task_input, [](https://microsoft.github.io/agent-lightning/latest/how-to/write-first-algorithm/#__codelineno-0-18) resources_id=resources_update.resources_id, [](https://microsoft.github.io/agent-lightning/latest/how-to/write-first-algorithm/#__codelineno-0-19) ) [](https://microsoft.github.io/agent-lightning/latest/how-to/write-first-algorithm/#__codelineno-0-20) print(f"[Algo] Task '{rollout.rollout_id}' is now available for clients.") [](https://microsoft.github.io/agent-lightning/latest/how-to/write-first-algorithm/#__codelineno-0-21) [](https://microsoft.github.io/agent-lightning/latest/how-to/write-first-algorithm/#__codelineno-0-22) # 3. Wait for the rollout to be completed by a runner [](https://microsoft.github.io/agent-lightning/latest/how-to/write-first-algorithm/#__codelineno-0-23) await store.wait_for_rollouts([rollout.rollout_id]) [](https://microsoft.github.io/agent-lightning/latest/how-to/write-first-algorithm/#__codelineno-0-24) [](https://microsoft.github.io/agent-lightning/latest/how-to/write-first-algorithm/#__codelineno-0-25) # 4. Query the completed rollout and its spans [](https://microsoft.github.io/agent-lightning/latest/how-to/write-first-algorithm/#__codelineno-0-26) completed_rollout = await store.get_rollout_by_id(rollout.rollout_id) [](https://microsoft.github.io/agent-lightning/latest/how-to/write-first-algorithm/#__codelineno-0-27) print(f"[Algo] Received Result: {completed_rollout.model_dump_json(indent=None)}") [](https://microsoft.github.io/agent-lightning/latest/how-to/write-first-algorithm/#__codelineno-0-28) [](https://microsoft.github.io/agent-lightning/latest/how-to/write-first-algorithm/#__codelineno-0-29) spans = await store.query_spans(rollout.rollout_id) [](https://microsoft.github.io/agent-lightning/latest/how-to/write-first-algorithm/#__codelineno-0-30) # We expect at least two spans: one for the LLM call and one for the final reward [](https://microsoft.github.io/agent-lightning/latest/how-to/write-first-algorithm/#__codelineno-0-31) print(f"[Algo] Queried Spans:\n - " + "\n - ".join(str(span) for span in spans)) [](https://microsoft.github.io/agent-lightning/latest/how-to/write-first-algorithm/#__codelineno-0-32) # find_final_reward is a helper function to extract the reward span [](https://microsoft.github.io/agent-lightning/latest/how-to/write-first-algorithm/#__codelineno-0-33) final_reward = find_final_reward(spans) [](https://microsoft.github.io/agent-lightning/latest/how-to/write-first-algorithm/#__codelineno-0-34) print(f"[Algo] Final reward: {final_reward}\n") [](https://microsoft.github.io/agent-lightning/latest/how-to/write-first-algorithm/#__codelineno-0-35) [](https://microsoft.github.io/agent-lightning/latest/how-to/write-first-algorithm/#__codelineno-0-36) results.append((prompt, final_reward)) [](https://microsoft.github.io/agent-lightning/latest/how-to/write-first-algorithm/#__codelineno-0-37) [](https://microsoft.github.io/agent-lightning/latest/how-to/write-first-algorithm/#__codelineno-0-38) # 5. Find and print the best prompt based on the collected rewards [](https://microsoft.github.io/agent-lightning/latest/how-to/write-first-algorithm/#__codelineno-0-39) print(f"[Algo] All prompts and their rewards: {results}") [](https://microsoft.github.io/agent-lightning/latest/how-to/write-first-algorithm/#__codelineno-0-40) best_prompt, best_reward = max(results, key=lambda item: item[1]) [](https://microsoft.github.io/agent-lightning/latest/how-to/write-first-algorithm/#__codelineno-0-41) print(f"[Algo] Best prompt found: '{best_prompt}' with reward {best_reward}")` Asynchronous Operations You'll notice the `async` and `await` keywords. Agent-lightning is built on asyncio to handle concurrent operations efficiently. All interactions with the store are asynchronous network calls, so they must be awaited. The Agent and Runner[¶](https://microsoft.github.io/agent-lightning/latest/how-to/write-first-algorithm/#the-agent-and-runner "Permanent link") ------------------------------------------------------------------------------------------------------------------------------------------------ Our algorithm needs an **agent** to execute the tasks and a **runner** to manage the process. The runner is a long-lived worker process. Its job is simple: 1. Connect to the [LightningStore](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.LightningStore " agentlightning.LightningStore") via a [LightningStoreClient](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.LightningStoreClient " agentlightning.LightningStoreClient") . 2. Enter a loop, constantly asking the [LightningStore](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.LightningStore " agentlightning.LightningStore") for new tasks (`dequeue_rollout`). 3. When it gets a task, it runs the `simple_agent` function. 4. Crucially, the runner wraps the agent execution with a **Tracer**. The tracer automatically captures all the important events (like the LLM call and the final reward) as spans and sends them back to the [LightningStore](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.LightningStore " agentlightning.LightningStore") . `[](https://microsoft.github.io/agent-lightning/latest/how-to/write-first-algorithm/#__codelineno-1-1) # Connecting to Store [](https://microsoft.github.io/agent-lightning/latest/how-to/write-first-algorithm/#__codelineno-1-2) store = agl.LightningStoreClient("http://localhost:4747") # or some other address [](https://microsoft.github.io/agent-lightning/latest/how-to/write-first-algorithm/#__codelineno-1-3) runner = LitAgentRunner[str](tracer=AgentOpsTracer()) [](https://microsoft.github.io/agent-lightning/latest/how-to/write-first-algorithm/#__codelineno-1-4) with runner.run_context(agent=simple_agent, store=store): # <-- where the wrapping and instrumentation happens [](https://microsoft.github.io/agent-lightning/latest/how-to/write-first-algorithm/#__codelineno-1-5) await runner.iter() # polling for new tasks forever` For this example, the agent's job is to take the prompt from the resources, use it to ask an LLM a question, and return a score. `[](https://microsoft.github.io/agent-lightning/latest/how-to/write-first-algorithm/#__codelineno-2-1) def simple_agent(task: str, prompt_template: PromptTemplate) -> float: [](https://microsoft.github.io/agent-lightning/latest/how-to/write-first-algorithm/#__codelineno-2-2) """An agent that answers a question and gets judged by an LLM.""" [](https://microsoft.github.io/agent-lightning/latest/how-to/write-first-algorithm/#__codelineno-2-3) client = OpenAI() [](https://microsoft.github.io/agent-lightning/latest/how-to/write-first-algorithm/#__codelineno-2-4) [](https://microsoft.github.io/agent-lightning/latest/how-to/write-first-algorithm/#__codelineno-2-5) # Generate a response using the provided prompt template [](https://microsoft.github.io/agent-lightning/latest/how-to/write-first-algorithm/#__codelineno-2-6) prompt = prompt_template.format(any_question=task) [](https://microsoft.github.io/agent-lightning/latest/how-to/write-first-algorithm/#__codelineno-2-7) response = client.chat.completions.create( [](https://microsoft.github.io/agent-lightning/latest/how-to/write-first-algorithm/#__codelineno-2-8) model="gpt-4.1-nano", messages=[{"role": "user", "content": prompt}] [](https://microsoft.github.io/agent-lightning/latest/how-to/write-first-algorithm/#__codelineno-2-9) ) [](https://microsoft.github.io/agent-lightning/latest/how-to/write-first-algorithm/#__codelineno-2-10) llm_output = response.choices[0].message.content [](https://microsoft.github.io/agent-lightning/latest/how-to/write-first-algorithm/#__codelineno-2-11) print(f"[Rollout] LLM returned: {llm_output}") [](https://microsoft.github.io/agent-lightning/latest/how-to/write-first-algorithm/#__codelineno-2-12) [](https://microsoft.github.io/agent-lightning/latest/how-to/write-first-algorithm/#__codelineno-2-13) # This llm_output and the final score are automatically logged as spans by the Tracer [](https://microsoft.github.io/agent-lightning/latest/how-to/write-first-algorithm/#__codelineno-2-14) score = random.uniform(0, 1) # Replace with actual scoring logic if needed [](https://microsoft.github.io/agent-lightning/latest/how-to/write-first-algorithm/#__codelineno-2-15) return score` Running the Example[¶](https://microsoft.github.io/agent-lightning/latest/how-to/write-first-algorithm/#running-the-example "Permanent link") ---------------------------------------------------------------------------------------------------------------------------------------------- To see everything in action, you'll need three separate terminal windows. Tip If you want to follow along, you can find the complete code for this example in the [apo\_custom\_algorithm.py](https://github.com/microsoft/agent-lightning/tree/c746af2f76bebcae56007a59c223cdace411c89b/examples/apo/apo_custom_algorithm.py) file. **1\. Start the Store:** In the first terminal, start the LightningStore server. This component will wait for connections from the algorithm and the runner. The store will be listening on port `4747` ⚡ by default. `[](https://microsoft.github.io/agent-lightning/latest/how-to/write-first-algorithm/#__codelineno-3-1) agl store` **2\. Start the Runner:** In the second terminal, start the runner process. It will connect to the store and wait for tasks. The code to start the runner looks like the following: `[](https://microsoft.github.io/agent-lightning/latest/how-to/write-first-algorithm/#__codelineno-4-1) export OPENAI_API_KEY=sk-... # Your OpenAI API key [](https://microsoft.github.io/agent-lightning/latest/how-to/write-first-algorithm/#__codelineno-4-2) python apo_custom_algorithm.py runner` You will see output indicating the runner has started and is waiting for rollouts. `[](https://microsoft.github.io/agent-lightning/latest/how-to/write-first-algorithm/#__codelineno-5-1) 2025-10-14 22:23:41,339 [INFO] ... [Worker 0] Setting up tracer... [](https://microsoft.github.io/agent-lightning/latest/how-to/write-first-algorithm/#__codelineno-5-2) 2025-10-14 22:23:41,343 [INFO] ... [Worker 0] Instrumentation applied. [](https://microsoft.github.io/agent-lightning/latest/how-to/write-first-algorithm/#__codelineno-5-3) 2025-10-14 22:23:41,494 [INFO] ... [Worker 0] AgentOps client initialized. [](https://microsoft.github.io/agent-lightning/latest/how-to/write-first-algorithm/#__codelineno-5-4) 2025-10-14 22:23:41,494 [INFO] ... [Worker 0] Started async rollouts (max: unlimited).` **3\. Start the Algorithm:** In the third terminal, run the algorithm. This will kick off the entire process. For example, we run the algorithm code shown above with the following parameters: `[](https://microsoft.github.io/agent-lightning/latest/how-to/write-first-algorithm/#__codelineno-6-1) prompts_to_test = [ [](https://microsoft.github.io/agent-lightning/latest/how-to/write-first-algorithm/#__codelineno-6-2) "You are a helpful assistant. {any_question}", [](https://microsoft.github.io/agent-lightning/latest/how-to/write-first-algorithm/#__codelineno-6-3) "You are a knowledgeable AI. {any_question}", [](https://microsoft.github.io/agent-lightning/latest/how-to/write-first-algorithm/#__codelineno-6-4) "You are a friendly chatbot. {any_question}", [](https://microsoft.github.io/agent-lightning/latest/how-to/write-first-algorithm/#__codelineno-6-5) ] [](https://microsoft.github.io/agent-lightning/latest/how-to/write-first-algorithm/#__codelineno-6-6) task_input = "Why is the sky blue?" [](https://microsoft.github.io/agent-lightning/latest/how-to/write-first-algorithm/#__codelineno-6-7) store = agl.LightningStoreClient("http://localhost:4747") [](https://microsoft.github.io/agent-lightning/latest/how-to/write-first-algorithm/#__codelineno-6-8) find_best_prompt(store, prompts_to_test, task_input)` Or you can simply use our pre-written script to try out: `[](https://microsoft.github.io/agent-lightning/latest/how-to/write-first-algorithm/#__codelineno-7-1) python apo_custom_algorithm.py algo` ### Understanding the Output[¶](https://microsoft.github.io/agent-lightning/latest/how-to/write-first-algorithm/#understanding-the-output "Permanent link") As the algorithm runs, you'll see logs appear across all three terminals, showing the components interacting in real-time. **Algorithm Output:** The algorithm terminal shows the main control flow: updating prompts, queuing tasks, and receiving the final results. You can also see the raw span data it retrieves from the store. `[](https://microsoft.github.io/agent-lightning/latest/how-to/write-first-algorithm/#__codelineno-8-1) [Algo] Updating prompt template to: 'You are a helpful assistant. {any_question}' [](https://microsoft.github.io/agent-lightning/latest/how-to/write-first-algorithm/#__codelineno-8-2) [Algo] Queuing task for clients... [](https://microsoft.github.io/agent-lightning/latest/how-to/write-first-algorithm/#__codelineno-8-3) [Algo] Task 'ro-1d18988581cd' is now available for clients. [](https://microsoft.github.io/agent-lightning/latest/how-to/write-first-algorithm/#__codelineno-8-4) [Algo] Received Result: rollout_id='ro-1d18988581cd' ... status='succeeded' ... [](https://microsoft.github.io/agent-lightning/latest/how-to/write-first-algorithm/#__codelineno-8-5) [Algo] Queried Spans: [](https://microsoft.github.io/agent-lightning/latest/how-to/write-first-algorithm/#__codelineno-8-6) - Span(name='openai.chat.completion', attributes={'gen_ai.prompt.0.content': 'You are a helpful assistant...', 'gen_ai.completion.0.content': 'The sky appears blue...'}) [](https://microsoft.github.io/agent-lightning/latest/how-to/write-first-algorithm/#__codelineno-8-7) - Span(name='reward', attributes={'value': 0.95}) [](https://microsoft.github.io/agent-lightning/latest/how-to/write-first-algorithm/#__codelineno-8-8) [Algo] Final reward: 0.95 [](https://microsoft.github.io/agent-lightning/latest/how-to/write-first-algorithm/#__codelineno-8-9) [](https://microsoft.github.io/agent-lightning/latest/how-to/write-first-algorithm/#__codelineno-8-10) [Algo] Updating prompt template to: 'You are a knowledgeable AI. {any_question}' [](https://microsoft.github.io/agent-lightning/latest/how-to/write-first-algorithm/#__codelineno-8-11) ... [](https://microsoft.github.io/agent-lightning/latest/how-to/write-first-algorithm/#__codelineno-8-12) [Algo] Final reward: 0.95 [](https://microsoft.github.io/agent-lightning/latest/how-to/write-first-algorithm/#__codelineno-8-13) [](https://microsoft.github.io/agent-lightning/latest/how-to/write-first-algorithm/#__codelineno-8-14) [Algo] Updating prompt template to: 'You are a friendly chatbot. {any_question}' [](https://microsoft.github.io/agent-lightning/latest/how-to/write-first-algorithm/#__codelineno-8-15) ... [](https://microsoft.github.io/agent-lightning/latest/how-to/write-first-algorithm/#__codelineno-8-16) [Algo] Final reward: 1.0 [](https://microsoft.github.io/agent-lightning/latest/how-to/write-first-algorithm/#__codelineno-8-17) [](https://microsoft.github.io/agent-lightning/latest/how-to/write-first-algorithm/#__codelineno-8-18) [Algo] All prompts and their rewards: [('You are a helpful assistant. {any_question}', 0.95), ('You are a knowledgeable AI. {any_question}', 0.95), ('You are a friendly chatbot. {any_question}', 1.0)] [](https://microsoft.github.io/agent-lightning/latest/how-to/write-first-algorithm/#__codelineno-8-19) [Algo] Best prompt found: 'You are a friendly chatbot. {any_question}' with reward 1.0` **Runner Output:** The runner terminal shows it picking up each task, executing the agent logic, and reporting the completion. `[](https://microsoft.github.io/agent-lightning/latest/how-to/write-first-algorithm/#__codelineno-9-1) [Rollout] LLM returned: The sky appears blue due to Rayleigh scattering... [](https://microsoft.github.io/agent-lightning/latest/how-to/write-first-algorithm/#__codelineno-9-2) 2025-10-14 22:25:50,803 [INFO] ... [Worker 0 | Rollout ro-a9f54ac19af5] Completed in 4.24s. ... [](https://microsoft.github.io/agent-lightning/latest/how-to/write-first-algorithm/#__codelineno-9-3) [](https://microsoft.github.io/agent-lightning/latest/how-to/write-first-algorithm/#__codelineno-9-4) [Rollout] LLM returned: The sky looks blue because of a process called Rayleigh scattering... [](https://microsoft.github.io/agent-lightning/latest/how-to/write-first-algorithm/#__codelineno-9-5) 2025-10-14 22:25:59,863 [INFO] ... [Worker 0 | Rollout ro-c67eaa9016b6] Completed in 4.06s. ...` **Store Server Output:** The store terminal shows a detailed log of every interaction, confirming its role as the central hub. You can see requests to enqueue and dequeue rollouts, add spans, and update statuses. `[](https://microsoft.github.io/agent-lightning/latest/how-to/write-first-algorithm/#__codelineno-10-1) ... "POST /enqueue_rollout HTTP/1.1" 200 ... [](https://microsoft.github.io/agent-lightning/latest/how-to/write-first-algorithm/#__codelineno-10-2) ... "GET /dequeue_rollout HTTP/1.1" 200 ... [](https://microsoft.github.io/agent-lightning/latest/how-to/write-first-algorithm/#__codelineno-10-3) ... "POST /add_span HTTP/1.1" 200 ... [](https://microsoft.github.io/agent-lightning/latest/how-to/write-first-algorithm/#__codelineno-10-4) ... "POST /update_attempt HTTP/1.1" 200 ... [](https://microsoft.github.io/agent-lightning/latest/how-to/write-first-algorithm/#__codelineno-10-5) ... "POST /wait_for_rollouts HTTP/1.1" 200 ... [](https://microsoft.github.io/agent-lightning/latest/how-to/write-first-algorithm/#__codelineno-10-6) ... "GET /query_spans/ro-c67eaa9016b6 HTTP/1.1" 200 ...` So Where is Trainer? You might be wondering why the [last tutorial](https://microsoft.github.io/agent-lightning/latest/how-to/train-first-agent/) focused on the [Trainer](https://microsoft.github.io/agent-lightning/latest/reference/trainer/#agentlightning.Trainer " agentlightning.Trainer") class, but we haven't used it here. Think of the [Trainer](https://microsoft.github.io/agent-lightning/latest/reference/trainer/#agentlightning.Trainer " agentlightning.Trainer") as a convenient wrapper that manages the entire training process for you. It's perfect when you want to apply a pre-built algorithm to your agent without worrying about the underlying mechanics. The [Trainer](https://microsoft.github.io/agent-lightning/latest/reference/trainer/#agentlightning.Trainer " agentlightning.Trainer") handles starting the [LightningStore](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.LightningStore " agentlightning.LightningStore") , coordinating the [Runners](https://microsoft.github.io/agent-lightning/latest/reference/runner/#agentlightning.Runner " agentlightning.Runner") , managing their lifecycles, and handling errors. In this tutorial, however, our goal is to _build a new algorithm_. To do that, we need to interact directly with the core components: the [Store](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.LightningStore " agentlightning.LightningStore") , the [Runner](https://microsoft.github.io/agent-lightning/latest/reference/runner/#agentlightning.Runner " agentlightning.Runner") , and the algorithm logic itself. Running them separately gives you more control and clearer, isolated logs, which is ideal for development and debugging. Once your custom algorithm is mature, you can package it to comply with our standard interface ([@algo](https://microsoft.github.io/agent-lightning/latest/reference/algorithm/#agentlightning.algo " agentlightning.algo(func)") or [Algorithm](https://microsoft.github.io/agent-lightning/latest/reference/algorithm/#agentlightning.Algorithm " agentlightning.Algorithm") ). This allows you to use it with the [Trainer](https://microsoft.github.io/agent-lightning/latest/reference/trainer/#agentlightning.Trainer " agentlightning.Trainer") again, getting all the benefits of automated lifecycle management while using your own custom logic. A sample code doing this is available in [apo\_custom\_algorithm\_trainer.py](https://github.com/microsoft/agent-lightning/tree/c746af2f76bebcae56007a59c223cdace411c89b/examples/apo/apo_custom_algorithm_trainer.py) . Back to top --- # Instrumentation - Agent-lightning [Skip to content](https://microsoft.github.io/agent-lightning/latest/reference/instrumentation/#instrumentation-api) ⚠️ You are viewing development documentation. For stable documentation, please visit [https://microsoft.github.io/agent-lightning/stable/](https://microsoft.github.io/agent-lightning/stable/) Instrumentation API[¶](https://microsoft.github.io/agent-lightning/latest/reference/instrumentation/#instrumentation-api "Permanent link") =========================================================================================================================================== ### `agentlightning.instrumentation.instrument_all()` [¶](https://microsoft.github.io/agent-lightning/latest/reference/instrumentation/#agentlightning.instrumentation.instrument_all "Permanent link") Instrument all the instrumentation libraries. ### `agentlightning.instrumentation.uninstrument_all()` [¶](https://microsoft.github.io/agent-lightning/latest/reference/instrumentation/#agentlightning.instrumentation.uninstrument_all "Permanent link") Uninstrument all the instrumentation libraries. AgentOps LangChain[¶](https://microsoft.github.io/agent-lightning/latest/reference/instrumentation/#agentops-langchain "Permanent link") ----------------------------------------------------------------------------------------------------------------------------------------- ### `agentlightning.instrumentation.agentops_langchain` [¶](https://microsoft.github.io/agent-lightning/latest/reference/instrumentation/#agentlightning.instrumentation.agentops_langchain "Permanent link") #### `instrument_agentops_langchain()` [¶](https://microsoft.github.io/agent-lightning/latest/reference/instrumentation/#agentlightning.instrumentation.agentops_langchain.instrument_agentops_langchain "Permanent link") Bypass AgentOp's native support for Langchain. #### `uninstrument_agentops_langchain()` [¶](https://microsoft.github.io/agent-lightning/latest/reference/instrumentation/#agentlightning.instrumentation.agentops_langchain.uninstrument_agentops_langchain "Permanent link") Restore AgentOp's native support for Langchain. AgentOps[¶](https://microsoft.github.io/agent-lightning/latest/reference/instrumentation/#agentops "Permanent link") --------------------------------------------------------------------------------------------------------------------- ### `agentlightning.instrumentation.agentops` [¶](https://microsoft.github.io/agent-lightning/latest/reference/instrumentation/#agentlightning.instrumentation.agentops "Permanent link") #### `BypassableAuthenticatedOTLPExporter` [¶](https://microsoft.github.io/agent-lightning/latest/reference/instrumentation/#agentlightning.instrumentation.agentops.BypassableAuthenticatedOTLPExporter "Permanent link") Bases: `LightningStoreOTLPExporter`, `AuthenticatedOTLPExporter` AuthenticatedOTLPExporter with switchable service control. When `_agentops_service_enabled` is False, skip export and return success. #### `BypassableOTLPMetricExporter` [¶](https://microsoft.github.io/agent-lightning/latest/reference/instrumentation/#agentlightning.instrumentation.agentops.BypassableOTLPMetricExporter "Permanent link") Bases: `OTLPMetricExporter` OTLPMetricExporter with switchable service control. When `_agentops_service_enabled` is False, skip export and return success. #### `BypassableOTLPSpanExporter` [¶](https://microsoft.github.io/agent-lightning/latest/reference/instrumentation/#agentlightning.instrumentation.agentops.BypassableOTLPSpanExporter "Permanent link") Bases: `LightningStoreOTLPExporter` OTLPSpanExporter with switchable service control. When `_agentops_service_enabled` is False, skip export and return success. This is used instead of BypassableAuthenticatedOTLPExporter on legacy AgentOps versions. #### `BypassableV3Client` [¶](https://microsoft.github.io/agent-lightning/latest/reference/instrumentation/#agentlightning.instrumentation.agentops.BypassableV3Client "Permanent link") Bases: `V3Client` V3Client with toggleable authentication calls. Returns dummy auth response when `_agentops_service_enabled` is False. #### `BypassableV4Client` [¶](https://microsoft.github.io/agent-lightning/latest/reference/instrumentation/#agentlightning.instrumentation.agentops.BypassableV4Client "Permanent link") Bases: `V4Client` V4Client with toggleable post requests. Returns dummy response when `_agentops_service_enabled` is False. #### `enable_agentops_service(enabled=True)` [¶](https://microsoft.github.io/agent-lightning/latest/reference/instrumentation/#agentlightning.instrumentation.agentops.enable_agentops_service "Permanent link") Enable or disable communication with the AgentOps service. By default, AgentOps exporters and clients will run in local mode and will NOT attempt to communicate with the remote AgentOps service. Parameters: * **`enabled`** (`bool`, default: `True` ) – If True, enable all AgentOps exporters and clients. All exporters and clients will operate in normal mode and send data to the [AgentOps service](https://www.agentops.ai/) . #### `instrument_agentops()` [¶](https://microsoft.github.io/agent-lightning/latest/reference/instrumentation/#agentlightning.instrumentation.agentops.instrument_agentops "Permanent link") Instrument agentops to capture token IDs. Automatically detects and uses the appropriate patching method based on the installed agentops version. #### `uninstrument_agentops()` [¶](https://microsoft.github.io/agent-lightning/latest/reference/instrumentation/#agentlightning.instrumentation.agentops.uninstrument_agentops "Permanent link") Uninstrument agentops to stop capturing token IDs. LiteLLM[¶](https://microsoft.github.io/agent-lightning/latest/reference/instrumentation/#litellm "Permanent link") ------------------------------------------------------------------------------------------------------------------- ### `agentlightning.instrumentation.litellm` [¶](https://microsoft.github.io/agent-lightning/latest/reference/instrumentation/#agentlightning.instrumentation.litellm "Permanent link") LiteLLM instrumentations. It's unclear whether or not this file is useful. It seems that LiteLLM owns its own telemetry from their own entrance [Related documentation](https://docs.litellm.ai/docs/observability/agentops_integration) . #### `instrument_litellm()` [¶](https://microsoft.github.io/agent-lightning/latest/reference/instrumentation/#agentlightning.instrumentation.litellm.instrument_litellm "Permanent link") Instrument litellm to capture token IDs. #### `uninstrument_litellm()` [¶](https://microsoft.github.io/agent-lightning/latest/reference/instrumentation/#agentlightning.instrumentation.litellm.uninstrument_litellm "Permanent link") Uninstrument litellm to stop capturing token IDs. vLLM[¶](https://microsoft.github.io/agent-lightning/latest/reference/instrumentation/#vllm "Permanent link") ------------------------------------------------------------------------------------------------------------- ### `agentlightning.instrumentation.vllm` [¶](https://microsoft.github.io/agent-lightning/latest/reference/instrumentation/#agentlightning.instrumentation.vllm "Permanent link") #### `instrument_vllm()` [¶](https://microsoft.github.io/agent-lightning/latest/reference/instrumentation/#agentlightning.instrumentation.vllm.instrument_vllm "Permanent link") Instrument vLLM to capture token IDs generated by engine. This instrumentation has been merged to upstream vLLM since v0.10.2. #### `uninstrument_vllm()` [¶](https://microsoft.github.io/agent-lightning/latest/reference/instrumentation/#agentlightning.instrumentation.vllm.uninstrument_vllm "Permanent link") Uninstrument vLLM to stop capturing token IDs generated by engine. Back to top --- # RESTful - Agent-lightning [Skip to content](https://microsoft.github.io/agent-lightning/latest/reference/restful/#restful-api-references) ⚠️ You are viewing development documentation. For stable documentation, please visit [https://microsoft.github.io/agent-lightning/stable/](https://microsoft.github.io/agent-lightning/stable/) RESTful API References[¶](https://microsoft.github.io/agent-lightning/latest/reference/restful/#restful-api-references "Permanent link") ========================================================================================================================================= Note Shown in the following is the RESTful API for Lightning Store. Back to top --- # Installation - Agent-lightning [Skip to content](https://microsoft.github.io/agent-lightning/latest/tutorials/installation/#installation-guide) ⚠️ You are viewing development documentation. For stable documentation, please visit [https://microsoft.github.io/agent-lightning/stable/](https://microsoft.github.io/agent-lightning/stable/) Installation Guide[¶](https://microsoft.github.io/agent-lightning/latest/tutorials/installation/#installation-guide "Permanent link") ====================================================================================================================================== This guide explains how to install **Agent-Lightning**. You can install it from **PyPI** (the Python Package Index) for general use or directly from the **source code** if you plan to contribute or need fine-grained control over dependencies. Platform and Hardware Requirements Agent-Lightning is officially supported on **Linux distributions** (Ubuntu 22.04 or later is recommended). At the moment **macOS and Windows** (outside of WSL2) are not supported. The Python runtime must be **Python 3.10 or newer**. We recommend using the latest patch release of Python 3.10, 3.11, or 3.12 to pick up performance and security updates. A **GPU is optional**—you only need CUDA-capable hardware if you plan to fine-tune model weights or run GPU-accelerated workloads. CPU-only environments are fully supported for evaluation and inference. Installing from PyPI[¶](https://microsoft.github.io/agent-lightning/latest/tutorials/installation/#installing-from-pypi "Permanent link") ------------------------------------------------------------------------------------------------------------------------------------------ The easiest way to get started is by installing Agent-Lightning directly from PyPI. This ensures you get the latest **stable release** of the package, tested for compatibility and reliability. ### Install the Stable Release[¶](https://microsoft.github.io/agent-lightning/latest/tutorials/installation/#install-the-stable-release "Permanent link") Run the following command in your terminal: `[](https://microsoft.github.io/agent-lightning/latest/tutorials/installation/#__codelineno-0-1) pip install --upgrade agentlightning` This installs or upgrades Agent-Lightning to the newest stable version. Tip If you intend to use **Agent-Lightning** with [**VERL**](https://microsoft.github.io/agent-lightning/latest/algorithm-zoo/verl/) or run any of its **example scripts**, you’ll need to install some additional dependencies. See the sections on [Algorithm-specific installation](https://microsoft.github.io/agent-lightning/latest/tutorials/installation/#algorithm-specific-installation) and [Example-specific installation](https://microsoft.github.io/agent-lightning/latest/tutorials/installation/#example-specific-installation) for details. ### Install the Nightly Build (Latest Features)[¶](https://microsoft.github.io/agent-lightning/latest/tutorials/installation/#install-the-nightly-build-latest-features "Permanent link") Agent-Lightning also publishes **nightly builds**, which contain the latest experimental features and improvements from the main branch. These are available via **Test PyPI**. `[](https://microsoft.github.io/agent-lightning/latest/tutorials/installation/#__codelineno-1-1) pip install --upgrade --index-url https://test.pypi.org/simple/ --extra-index-url https://pypi.org/simple/ --pre agentlightning` Warning The nightly builds are cutting-edge but may include unstable or untested changes. Use them **at your own risk**, especially in production environments. Algorithm-specific Installation[¶](https://microsoft.github.io/agent-lightning/latest/tutorials/installation/#algorithm-specific-installation "Permanent link") ---------------------------------------------------------------------------------------------------------------------------------------------------------------- Agent-Lightning supports multiple learning algorithms. Some of them like [APO](https://microsoft.github.io/agent-lightning/latest/algorithm-zoo/apo/) or [VERL](https://microsoft.github.io/agent-lightning/latest/algorithm-zoo/verl/) require extra dependencies. You can install them automatically using **optional extras** or manually if you prefer finer control. ### Installing APO[¶](https://microsoft.github.io/agent-lightning/latest/tutorials/installation/#installing-apo "Permanent link") [APO](https://microsoft.github.io/agent-lightning/latest/algorithm-zoo/apo/) is an algorithm module that depends on libraries such as [POML](https://github.com/microsoft/POML) . You can install Agent-Lightning with APO support by running: `[](https://microsoft.github.io/agent-lightning/latest/tutorials/installation/#__codelineno-2-1) pip install agentlightning[apo]` Warning APO also depends on the [OpenAI Python SDK](https://github.com/openai/openai-python) , version **2.0 or newer**. Ensure your SDK version is up to date to avoid compatibility issues. ### Installing VERL[¶](https://microsoft.github.io/agent-lightning/latest/tutorials/installation/#installing-verl "Permanent link") [VERL](https://microsoft.github.io/agent-lightning/latest/algorithm-zoo/verl/) integrates with libraries like **PyTorch**, **vLLM**, and **VERL framework**. Although you _can_ install all dependencies automatically, we recommend doing it manually to avoid version conflicts. `[](https://microsoft.github.io/agent-lightning/latest/tutorials/installation/#__codelineno-3-1) pip install agentlightning[verl]` Recommended Manual Setup (More Stable) Automated installation may cause issues if you don’t have a compatible **PyTorch** or **CUDA** version preinstalled. For a more stable setup, install dependencies step-by-step: `[](https://microsoft.github.io/agent-lightning/latest/tutorials/installation/#__codelineno-4-1) pip install torch==2.8.0 torchvision==0.23.0 --index-url https://download.pytorch.org/whl/cu128 [](https://microsoft.github.io/agent-lightning/latest/tutorials/installation/#__codelineno-4-2) pip install flash-attn --no-build-isolation [](https://microsoft.github.io/agent-lightning/latest/tutorials/installation/#__codelineno-4-3) pip install vllm==0.10.2 [](https://microsoft.github.io/agent-lightning/latest/tutorials/installation/#__codelineno-4-4) pip install verl==0.5.0` This approach ensures compatibility with CUDA 12.8 and minimizes dependency conflicts. Example-specific Installation[¶](https://microsoft.github.io/agent-lightning/latest/tutorials/installation/#example-specific-installation "Permanent link") ------------------------------------------------------------------------------------------------------------------------------------------------------------ Each example in the `examples/` directory may have its own additional dependencies. Please refer to the **README** file of each example for detailed setup instructions: [See Example READMEs](https://github.com/microsoft/agent-lightning/tree/c746af2f76bebcae56007a59c223cdace411c89b/examples) . Installing from Source (for Developers and Contributors)[¶](https://microsoft.github.io/agent-lightning/latest/tutorials/installation/#installing-from-source-for-developers-and-contributors "Permanent link") ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- If you plan to contribute to Agent-Lightning or prefer to work with the latest development code, install it directly from the **source repository**. ### Why Install from Source?[¶](https://microsoft.github.io/agent-lightning/latest/tutorials/installation/#why-install-from-source "Permanent link") * You want to **modify or contribute** to the project. * You prefer an **isolated development environment**. * You want to test unreleased features or fix bugs locally. ### Using `uv` for Dependency Management[¶](https://microsoft.github.io/agent-lightning/latest/tutorials/installation/#using-uv-for-dependency-management "Permanent link") Starting with version **0.2**, Agent-Lightning uses [`uv`](https://docs.astral.sh/uv/) as its **default dependency manager**. `uv` is a fast and safe alternative to `pip` that: * Installs packages **in seconds** (instead of minutes), * Prevents **dependency conflicts**, * Supports **grouped dependencies** for optional features. Before proceeding, make sure `uv` is installed. ### Minimal Developer Installation[¶](https://microsoft.github.io/agent-lightning/latest/tutorials/installation/#minimal-developer-installation "Permanent link") `[](https://microsoft.github.io/agent-lightning/latest/tutorials/installation/#__codelineno-5-1) git clone https://github.com/microsoft/agent-lightning [](https://microsoft.github.io/agent-lightning/latest/tutorials/installation/#__codelineno-5-2) cd agent-lightning [](https://microsoft.github.io/agent-lightning/latest/tutorials/installation/#__codelineno-5-3) uv sync --group dev` This command sets up a clean development environment with only the essential dependencies. ### Installing All Extras (CPU or GPU)[¶](https://microsoft.github.io/agent-lightning/latest/tutorials/installation/#installing-all-extras-cpu-or-gpu "Permanent link") `uv sync` can also handle algorithm-specific and example-specific dependencies in one step. For a CPU-only machine: `[](https://microsoft.github.io/agent-lightning/latest/tutorials/installation/#__codelineno-6-1) uv sync --frozen \ [](https://microsoft.github.io/agent-lightning/latest/tutorials/installation/#__codelineno-6-2) --extra apo \ [](https://microsoft.github.io/agent-lightning/latest/tutorials/installation/#__codelineno-6-3) --extra verl \ [](https://microsoft.github.io/agent-lightning/latest/tutorials/installation/#__codelineno-6-4) --group dev \ [](https://microsoft.github.io/agent-lightning/latest/tutorials/installation/#__codelineno-6-5) --group torch-cpu \ [](https://microsoft.github.io/agent-lightning/latest/tutorials/installation/#__codelineno-6-6) --group torch-stable \ [](https://microsoft.github.io/agent-lightning/latest/tutorials/installation/#__codelineno-6-7) --group trl \ [](https://microsoft.github.io/agent-lightning/latest/tutorials/installation/#__codelineno-6-8) --group agents \ [](https://microsoft.github.io/agent-lightning/latest/tutorials/installation/#__codelineno-6-9) --no-default-groups` For a GPU-equipped machine that is CUDA 12.8 compatible: `[](https://microsoft.github.io/agent-lightning/latest/tutorials/installation/#__codelineno-7-1) uv sync --frozen \ [](https://microsoft.github.io/agent-lightning/latest/tutorials/installation/#__codelineno-7-2) --extra apo \ [](https://microsoft.github.io/agent-lightning/latest/tutorials/installation/#__codelineno-7-3) --extra verl \ [](https://microsoft.github.io/agent-lightning/latest/tutorials/installation/#__codelineno-7-4) --group dev \ [](https://microsoft.github.io/agent-lightning/latest/tutorials/installation/#__codelineno-7-5) --group torch-gpu-stable \ [](https://microsoft.github.io/agent-lightning/latest/tutorials/installation/#__codelineno-7-6) --group trl \ [](https://microsoft.github.io/agent-lightning/latest/tutorials/installation/#__codelineno-7-7) --group agents \ [](https://microsoft.github.io/agent-lightning/latest/tutorials/installation/#__codelineno-7-8) --no-default-groups` Read more about Agent-lightning managed dependency groups [here](https://github.com/microsoft/agent-lightning/tree/c746af2f76bebcae56007a59c223cdace411c89b/pyproject.toml) . ### Building the Dashboard[¶](https://microsoft.github.io/agent-lightning/latest/tutorials/installation/#building-the-dashboard "Permanent link") The Agent-Lightning dashboard is built using [Vite](https://vite.dev/) . To build the dashboard, run the following command: `[](https://microsoft.github.io/agent-lightning/latest/tutorials/installation/#__codelineno-8-1) cd dashboard [](https://microsoft.github.io/agent-lightning/latest/tutorials/installation/#__codelineno-8-2) npm ci [](https://microsoft.github.io/agent-lightning/latest/tutorials/installation/#__codelineno-8-3) npm run build` Some HTML and JavaScript assets will be generated in the `agentlightning/dashboard` directory. ### Activating Your Environment[¶](https://microsoft.github.io/agent-lightning/latest/tutorials/installation/#activating-your-environment "Permanent link") After syncing dependencies, `uv` automatically creates a virtual environment inside the `.venv/` directory. You can use it in two ways: `[](https://microsoft.github.io/agent-lightning/latest/tutorials/installation/#__codelineno-9-1) # Option 1: Prefix commands with uv run [](https://microsoft.github.io/agent-lightning/latest/tutorials/installation/#__codelineno-9-2) uv run python your_script.py [](https://microsoft.github.io/agent-lightning/latest/tutorials/installation/#__codelineno-9-3) [](https://microsoft.github.io/agent-lightning/latest/tutorials/installation/#__codelineno-9-4) # Option 2: Activate the virtual environment [](https://microsoft.github.io/agent-lightning/latest/tutorials/installation/#__codelineno-9-5) source .venv/bin/activate [](https://microsoft.github.io/agent-lightning/latest/tutorials/installation/#__codelineno-9-6) python your_script.py` Before Contributing Agent-Lightning enforces code style and linting rules via **pre-commit hooks**. Installing them early prevents many avoidable formatting issues. `[](https://microsoft.github.io/agent-lightning/latest/tutorials/installation/#__codelineno-10-1) uv run pre-commit install [](https://microsoft.github.io/agent-lightning/latest/tutorials/installation/#__codelineno-10-2) uv run pre-commit run --all-files --show-diff-on-failure --color=always` Back to top --- # Parallelize - Agent-lightning [Skip to content](https://microsoft.github.io/agent-lightning/latest/tutorials/parallelize/#scaling-out-agent-lightning) ⚠️ You are viewing development documentation. For stable documentation, please visit [https://microsoft.github.io/agent-lightning/stable/](https://microsoft.github.io/agent-lightning/stable/) Scaling out Agent-lightning[¶](https://microsoft.github.io/agent-lightning/latest/tutorials/parallelize/#scaling-out-agent-lightning "Permanent link") ======================================================================================================================================================= Agent-lightning splits training into an **algorithm bundle** and a **runner bundle** that exchange work through the [`LightningStore`](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.LightningStore " agentlightning.LightningStore") . This tutorial shows how to increase rollout throughput, place bundles across processes or machines, and keep the algorithm side scalable with external frameworks. Parallelizing Rollouts with [`Trainer`](https://microsoft.github.io/agent-lightning/latest/reference/trainer/#agentlightning.Trainer " agentlightning.Trainer") [¶](https://microsoft.github.io/agent-lightning/latest/tutorials/parallelize/#parallelizing-rollouts-with-trainer "Permanent link") --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Before we dive into the details of the bundles and execution strategies, let's first revisit how to parallelize rollouts with [`Trainer`](https://microsoft.github.io/agent-lightning/latest/reference/trainer/#agentlightning.Trainer " agentlightning.Trainer") . [`Trainer`](https://microsoft.github.io/agent-lightning/latest/reference/trainer/#agentlightning.Trainer " agentlightning.Trainer") is the quickest way to dial up parallelism. Even when `n_runners = 1`, calling [`Trainer.fit`](https://microsoft.github.io/agent-lightning/latest/reference/trainer/#agentlightning.Trainer.fit " fit(agent, train_dataset=None, *, val_dataset=None)") runs the algorithm and runners in parallel. The algorithm enqueues rollouts; runners dequeue them and execute your [`LitAgent`](https://microsoft.github.io/agent-lightning/latest/reference/agent/#agentlightning.LitAgent " agentlightning.LitAgent") , and the algorithm collects spans via its [`Adapter`](https://microsoft.github.io/agent-lightning/latest/reference/algorithm/#agentlightning.Adapter " agentlightning.Adapter") before scheduling the next batch. Note One of the most important features of [`Trainer`](https://microsoft.github.io/agent-lightning/latest/reference/trainer/#agentlightning.Trainer " agentlightning.Trainer") is the ability to abort things gracefully. For example, if you press `Ctrl+C` in the terminal, the algorithm will abort and the runners will stop executing. If the algorithm crashes, the runners will also stop executing. Increase throughput by setting `n_runners` when constructing the trainer. The following example comes from [train\_calc\_agent.py](https://github.com/microsoft/agent-lightning/tree/c746af2f76bebcae56007a59c223cdace411c89b/examples/calc_x/train_calc_agent.py) . Since backend LLMs usually use techniques like [continuous batching](https://docs.vllm.ai/en/latest/) to increase throughput, you do not have to worry about overwhelming the backend with too many requests. `[](https://microsoft.github.io/agent-lightning/latest/tutorials/parallelize/#__codelineno-0-1) import agentlightning as agl [](https://microsoft.github.io/agent-lightning/latest/tutorials/parallelize/#__codelineno-0-2) from datasets import Dataset as HFDataset [](https://microsoft.github.io/agent-lightning/latest/tutorials/parallelize/#__codelineno-0-3) from calc_agent import calc_agent [](https://microsoft.github.io/agent-lightning/latest/tutorials/parallelize/#__codelineno-0-4) [](https://microsoft.github.io/agent-lightning/latest/tutorials/parallelize/#__codelineno-0-5) train_dataset = HFDataset.from_parquet("data/train.parquet").to_list() [](https://microsoft.github.io/agent-lightning/latest/tutorials/parallelize/#__codelineno-0-6) val_dataset = HFDataset.from_parquet("data/test.parquet").to_list() [](https://microsoft.github.io/agent-lightning/latest/tutorials/parallelize/#__codelineno-0-7) [](https://microsoft.github.io/agent-lightning/latest/tutorials/parallelize/#__codelineno-0-8) algorithm = agl.VERL(verl_config) [](https://microsoft.github.io/agent-lightning/latest/tutorials/parallelize/#__codelineno-0-9) [](https://microsoft.github.io/agent-lightning/latest/tutorials/parallelize/#__codelineno-0-10) trainer = agl.Trainer( [](https://microsoft.github.io/agent-lightning/latest/tutorials/parallelize/#__codelineno-0-11) algorithm=algorithm, [](https://microsoft.github.io/agent-lightning/latest/tutorials/parallelize/#__codelineno-0-12) n_runners=8, # launch eight rollout workers [](https://microsoft.github.io/agent-lightning/latest/tutorials/parallelize/#__codelineno-0-13) tracer=agl.OtelTracer(), [](https://microsoft.github.io/agent-lightning/latest/tutorials/parallelize/#__codelineno-0-14) adapter=agl.LlmProxyTraceToTriplet(), [](https://microsoft.github.io/agent-lightning/latest/tutorials/parallelize/#__codelineno-0-15) ) [](https://microsoft.github.io/agent-lightning/latest/tutorials/parallelize/#__codelineno-0-16) [](https://microsoft.github.io/agent-lightning/latest/tutorials/parallelize/#__codelineno-0-17) trainer.fit(calc_agent, train_dataset=train_dataset, val_dataset=val_dataset)` In [`Trainer`](https://microsoft.github.io/agent-lightning/latest/reference/trainer/#agentlightning.Trainer " agentlightning.Trainer") , there are multiple other initialization parameters that you can use to customize the training process. For example, you can use `max_rollouts` to keep smoke tests short. Pass a concrete [`LightningStore`](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.LightningStore " agentlightning.LightningStore") instance when you need persistence or want to share the queue across multiple scripts. Tip Before scaling out, run [`Trainer.dev()`](https://microsoft.github.io/agent-lightning/latest/reference/trainer/#agentlightning.Trainer.dev " dev(agent, train_dataset=None, *, val_dataset=None)") with `n_runners=1` to verify the rollout logic and spans without burning GPU hours. Bundles and Execution Strategies[¶](https://microsoft.github.io/agent-lightning/latest/tutorials/parallelize/#bundles-and-execution-strategies "Permanent link") ----------------------------------------------------------------------------------------------------------------------------------------------------------------- When [`Trainer`](https://microsoft.github.io/agent-lightning/latest/reference/trainer/#agentlightning.Trainer " agentlightning.Trainer") starts, it packages its configuration into two callable **bundles**: ![Illustration of bundles and execution strategies](https://microsoft.github.io/agent-lightning/latest/assets/execution-bundles.svg) The **algorithm bundle** wraps your [`Algorithm`](https://microsoft.github.io/agent-lightning/latest/reference/algorithm/#agentlightning.Algorithm " agentlightning.Algorithm") , adapter, and any LLM proxy into a single callable that can be aborted via a signal event. `[](https://microsoft.github.io/agent-lightning/latest/tutorials/parallelize/#__codelineno-1-1) async def algorithm_bundle(store: LightningStore, event: ExecutionEvent) -> None: [](https://microsoft.github.io/agent-lightning/latest/tutorials/parallelize/#__codelineno-1-2) ...` The **runner bundle** wraps the [`Runner`](https://microsoft.github.io/agent-lightning/latest/reference/runner/#agentlightning.Runner " agentlightning.Runner") , tracer, hooks, and agent into a single callable that can be aborted via a signal event. Unlike the algorithm bundle, the runner bundle is expected to be replicated. `[](https://microsoft.github.io/agent-lightning/latest/tutorials/parallelize/#__codelineno-2-1) async def runner_bundle(store: LightningStore, worker_id: int, event: ExecutionEvent) -> None: [](https://microsoft.github.io/agent-lightning/latest/tutorials/parallelize/#__codelineno-2-2) ...` An **execution strategy** then decides where those bundles are placed (threads vs processes vs multiple machines), how many runner replicas to launch, and how lifecycle events such as shutdown are coordinated. By default, the trainer builds an [`InMemoryLightningStore`](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.InMemoryLightningStore " agentlightning.InMemoryLightningStore") if you do not provide one. Because that store has no locking or cross-process transport, the execution strategy is the component that wraps it in thread-safe or HTTP-safe facades ([`LightningStoreThreaded`](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.LightningStoreThreaded " agentlightning.LightningStoreThreaded") , [`LightningStoreServer`](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.LightningStoreServer " agentlightning.LightningStoreServer") ) before handing it to bundles. For a deeper look at these facades, see [Understanding the Store](https://microsoft.github.io/agent-lightning/latest/deep-dive/store/) and [Birds' Eye View](https://microsoft.github.io/agent-lightning/latest/deep-dive/birds-eye-view/) . Agent-lightning provides two built-in execution strategies: [`SharedMemoryExecutionStrategy`](https://microsoft.github.io/agent-lightning/latest/reference/trainer/#agentlightning.SharedMemoryExecutionStrategy " agentlightning.SharedMemoryExecutionStrategy") and [`ClientServerExecutionStrategy`](https://microsoft.github.io/agent-lightning/latest/reference/trainer/#agentlightning.ClientServerExecutionStrategy " agentlightning.ClientServerExecutionStrategy") . You can pass a string alias, a configuration dictionary, or a pre-built strategy instance: ``[](https://microsoft.github.io/agent-lightning/latest/tutorials/parallelize/#__codelineno-3-1) import agentlightning as agl [](https://microsoft.github.io/agent-lightning/latest/tutorials/parallelize/#__codelineno-3-2) [](https://microsoft.github.io/agent-lightning/latest/tutorials/parallelize/#__codelineno-3-3) algorithm = agl.Baseline() [](https://microsoft.github.io/agent-lightning/latest/tutorials/parallelize/#__codelineno-3-4) [](https://microsoft.github.io/agent-lightning/latest/tutorials/parallelize/#__codelineno-3-5) # Short alias for the shared-memory strategy. [](https://microsoft.github.io/agent-lightning/latest/tutorials/parallelize/#__codelineno-3-6) # Because the runner lives on the main thread in this mode, [](https://microsoft.github.io/agent-lightning/latest/tutorials/parallelize/#__codelineno-3-7) # n_runners must be 1 unless you move the algorithm to the main thread. [](https://microsoft.github.io/agent-lightning/latest/tutorials/parallelize/#__codelineno-3-8) trainer = agl.Trainer(algorithm=algorithm, n_runners=1, strategy="shm") [](https://microsoft.github.io/agent-lightning/latest/tutorials/parallelize/#__codelineno-3-9) [](https://microsoft.github.io/agent-lightning/latest/tutorials/parallelize/#__codelineno-3-10) # Dict with overrides; keep the algorithm on the main thread so multiple runner threads can spawn. [](https://microsoft.github.io/agent-lightning/latest/tutorials/parallelize/#__codelineno-3-11) # Specifying `n_runners` inside strategy is equivalent to passing `n_runners` to the trainer. [](https://microsoft.github.io/agent-lightning/latest/tutorials/parallelize/#__codelineno-3-12) trainer = agl.Trainer( [](https://microsoft.github.io/agent-lightning/latest/tutorials/parallelize/#__codelineno-3-13) algorithm=algorithm, [](https://microsoft.github.io/agent-lightning/latest/tutorials/parallelize/#__codelineno-3-14) strategy={ [](https://microsoft.github.io/agent-lightning/latest/tutorials/parallelize/#__codelineno-3-15) "type": "shm", [](https://microsoft.github.io/agent-lightning/latest/tutorials/parallelize/#__codelineno-3-16) "n_runners": 8, [](https://microsoft.github.io/agent-lightning/latest/tutorials/parallelize/#__codelineno-3-17) "main_thread": "algorithm", [](https://microsoft.github.io/agent-lightning/latest/tutorials/parallelize/#__codelineno-3-18) }, [](https://microsoft.github.io/agent-lightning/latest/tutorials/parallelize/#__codelineno-3-19) ) [](https://microsoft.github.io/agent-lightning/latest/tutorials/parallelize/#__codelineno-3-20) [](https://microsoft.github.io/agent-lightning/latest/tutorials/parallelize/#__codelineno-3-21) # Pass an existing strategy instance – Trainer respects the strategy's own `n_runners`. [](https://microsoft.github.io/agent-lightning/latest/tutorials/parallelize/#__codelineno-3-22) strategy = agl.SharedMemoryExecutionStrategy(main_thread="algorithm", n_runners=4) [](https://microsoft.github.io/agent-lightning/latest/tutorials/parallelize/#__codelineno-3-23) trainer = agl.Trainer(algorithm=algorithm, strategy=strategy)`` If you omit the strategy, the trainer defaults to `ClientServerExecutionStrategy(n_runners=trainer.n_runners)`. You can still re-specify the client-server strategy through aliases or configuration to tweak ports and other settings: `[](https://microsoft.github.io/agent-lightning/latest/tutorials/parallelize/#__codelineno-4-1) trainer = agl.Trainer( [](https://microsoft.github.io/agent-lightning/latest/tutorials/parallelize/#__codelineno-4-2) algorithm=algorithm, [](https://microsoft.github.io/agent-lightning/latest/tutorials/parallelize/#__codelineno-4-3) n_runners=8, [](https://microsoft.github.io/agent-lightning/latest/tutorials/parallelize/#__codelineno-4-4) strategy={"type": "cs", "server_port": 9999}, [](https://microsoft.github.io/agent-lightning/latest/tutorials/parallelize/#__codelineno-4-5) )` Environment variables give you another layer of control. For example: `[](https://microsoft.github.io/agent-lightning/latest/tutorials/parallelize/#__codelineno-5-1) import os [](https://microsoft.github.io/agent-lightning/latest/tutorials/parallelize/#__codelineno-5-2) [](https://microsoft.github.io/agent-lightning/latest/tutorials/parallelize/#__codelineno-5-3) os.environ["AGL_SERVER_PORT"] = "10000" [](https://microsoft.github.io/agent-lightning/latest/tutorials/parallelize/#__codelineno-5-4) os.environ["AGL_CURRENT_ROLE"] = "algorithm" [](https://microsoft.github.io/agent-lightning/latest/tutorials/parallelize/#__codelineno-5-5) os.environ["AGL_MANAGED_STORE"] = "0" [](https://microsoft.github.io/agent-lightning/latest/tutorials/parallelize/#__codelineno-5-6) [](https://microsoft.github.io/agent-lightning/latest/tutorials/parallelize/#__codelineno-5-7) trainer = agl.Trainer(algorithm=algorithm, n_runners=8, strategy="cs")` The resulting [`ClientServerExecutionStrategy`](https://microsoft.github.io/agent-lightning/latest/reference/trainer/#agentlightning.ClientServerExecutionStrategy " agentlightning.ClientServerExecutionStrategy") picks up the port, role, and managed-store flag from the environment. Tip The same configuration patterns apply to other trainer components. For example, `[](https://microsoft.github.io/agent-lightning/latest/tutorials/parallelize/#__codelineno-6-1) trainer = agl.Trainer(algorithm=algorithm, tracer=agl.OtelTracer())` wires in a custom tracer, while `[](https://microsoft.github.io/agent-lightning/latest/tutorials/parallelize/#__codelineno-7-1) trainer = agl.Trainer(algorithm=algorithm, adapter="agentlightning.adapter.TraceToMessages")` swaps in a different adapter. Passing a dict lets you tweak the init parameters of defaults without naming the class explicitly: `[](https://microsoft.github.io/agent-lightning/latest/tutorials/parallelize/#__codelineno-8-1) trainer = agl.Trainer( [](https://microsoft.github.io/agent-lightning/latest/tutorials/parallelize/#__codelineno-8-2) algorithm=algorithm, [](https://microsoft.github.io/agent-lightning/latest/tutorials/parallelize/#__codelineno-8-3) adapter={"agent_match": "plan_agent", "repair_hierarchy": False}, [](https://microsoft.github.io/agent-lightning/latest/tutorials/parallelize/#__codelineno-8-4) )` The next sections walk through the two built-in strategies and how they affect placement and store access. Client-server Architecture[¶](https://microsoft.github.io/agent-lightning/latest/tutorials/parallelize/#client-server-architecture "Permanent link") ----------------------------------------------------------------------------------------------------------------------------------------------------- The default [`ClientServerExecutionStrategy`](https://microsoft.github.io/agent-lightning/latest/reference/trainer/#agentlightning.ClientServerExecutionStrategy " agentlightning.ClientServerExecutionStrategy") starts a [`LightningStoreServer`](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.LightningStoreServer " agentlightning.LightningStoreServer") alongside the algorithm and spawns runner processes that talk to it through [`LightningStoreClient`](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.LightningStoreClient " agentlightning.LightningStoreClient") . All runners share the HTTP endpoint, so the queue and spans stay consistent across processes or machines. If you simply instantiate [`Trainer`](https://microsoft.github.io/agent-lightning/latest/reference/trainer/#agentlightning.Trainer " agentlightning.Trainer") (as above), it will send the algorithm bundle and runner bundle to [`ClientServerExecutionStrategy`](https://microsoft.github.io/agent-lightning/latest/reference/trainer/#agentlightning.ClientServerExecutionStrategy " agentlightning.ClientServerExecutionStrategy") , which will then: 1. Launch \\(N+1\\) processes: \\(N\\) runner processes and 1 algorithm process (one of them could live in the main process). 2. The algorithm process will take the store received from [`Trainer`](https://microsoft.github.io/agent-lightning/latest/reference/trainer/#agentlightning.Trainer " agentlightning.Trainer") , wrap it in a [`LightningStoreServer`](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.LightningStoreServer " agentlightning.LightningStoreServer") , and start serving it over HTTP. 3. The runner processes discard the store and create a new store, which is a client that connects to the algorithm process through [`LightningStoreClient`](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.LightningStoreClient " agentlightning.LightningStoreClient") , and start executing the runner bundle. 4. The strategy automatically escalates shutdown (cooperative stop → `SIGINT` → `terminate()` → `kill()`) so long-running runners do not linger. You can override server placement or ports, and whether to automatically wrap the store, through constructor arguments or environment variables: `[](https://microsoft.github.io/agent-lightning/latest/tutorials/parallelize/#__codelineno-9-1) trainer = agl.Trainer( [](https://microsoft.github.io/agent-lightning/latest/tutorials/parallelize/#__codelineno-9-2) algorithm=algorithm, [](https://microsoft.github.io/agent-lightning/latest/tutorials/parallelize/#__codelineno-9-3) n_runners=1, [](https://microsoft.github.io/agent-lightning/latest/tutorials/parallelize/#__codelineno-9-4) strategy={ [](https://microsoft.github.io/agent-lightning/latest/tutorials/parallelize/#__codelineno-9-5) "type": "cs", [](https://microsoft.github.io/agent-lightning/latest/tutorials/parallelize/#__codelineno-9-6) "server_host": "0.0.0.0", [](https://microsoft.github.io/agent-lightning/latest/tutorials/parallelize/#__codelineno-9-7) "server_port": 9999, [](https://microsoft.github.io/agent-lightning/latest/tutorials/parallelize/#__codelineno-9-8) "main_process": "runner", [](https://microsoft.github.io/agent-lightning/latest/tutorials/parallelize/#__codelineno-9-9) }, [](https://microsoft.github.io/agent-lightning/latest/tutorials/parallelize/#__codelineno-9-10) )` Set `AGL_SERVER_HOST` and `AGL_SERVER_PORT` if you prefer environment-based configuration. You can also use `AGL_MANAGED_STORE` if you do not want the execution strategy to wrap the store for you. An example is shown in [Debugging with External Store](https://microsoft.github.io/agent-lightning/latest/tutorials/debug/#debug-with-external-store "Debug the Algorithm-Runner Boundary") . Algorithms sometimes require heterogeneous computation resources, such as GPU accelerators, while runners sometimes require a specific environment to run because many agent frameworks are fragile in their dependencies. A role-based launch pattern helps you place the algorithm on a dedicated machine with more GPU memory, while runners can live on another machine with more flexible dependencies. This is possible via `AGL_CURRENT_ROLE="algorithm"` or `AGL_CURRENT_ROLE="runner"` environment variables. When running on different machines, you also need to set `AGL_SERVER_HOST` and `AGL_SERVER_PORT` to the IP address and port of the algorithm machine. You might recognize that this convention is very similar to `MASTER_ADDR` and `MASTER_PORT` in [PyTorch distributed training](https://docs.pytorch.org/docs/stable/notes/ddp.html) . ### Launching Algorithm and Runner Roles on Separate Machines[¶](https://microsoft.github.io/agent-lightning/latest/tutorials/parallelize/#launching-algorithm-and-runner-roles-on-separate-machines "Permanent link") When you want to stretch the algorithm onto a GPU-rich machine and keep rollout workers close to the data source (or on machines with a more permissive dependency stack), launch the same training script in different terminals with role-specific environment variables. The client–server strategy will route each process to the right side of the queue as long as they share the same `AGL_SERVER_HOST`/`AGL_SERVER_PORT` pair. **1\. Pick an address and port for the store.** Decide which machine will host the algorithm. Choose a TCP port that can be reached by the runner machines (for example, open it in your firewall configuration). In this example we will use `10.0.0.4:4747`. **2\. Start the algorithm process.** On the machine that should run the algorithm, expose the store by binding to all network interfaces and mark the role as `algorithm`. `[](https://microsoft.github.io/agent-lightning/latest/tutorials/parallelize/#__codelineno-10-1) export AGL_SERVER_HOST=0.0.0.0 [](https://microsoft.github.io/agent-lightning/latest/tutorials/parallelize/#__codelineno-10-2) export AGL_SERVER_PORT=4747 [](https://microsoft.github.io/agent-lightning/latest/tutorials/parallelize/#__codelineno-10-3) export AGL_CURRENT_ROLE=algorithm [](https://microsoft.github.io/agent-lightning/latest/tutorials/parallelize/#__codelineno-10-4) [](https://microsoft.github.io/agent-lightning/latest/tutorials/parallelize/#__codelineno-10-5) python train_calc_agent.py` Leaving `AGL_MANAGED_STORE` unset (or setting it to `1`) lets the strategy create the [`LightningStoreServer`](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.LightningStoreServer " agentlightning.LightningStoreServer") for you. Otherwise, you can use the method in the previous section to create a store on your own. **3\. Start rollout workers on remote machines.** Every runner machine should point to the algorithm host and declare itself as the `runner` role. You can start multiple processes per machine or repeat the command on additional hosts. `[](https://microsoft.github.io/agent-lightning/latest/tutorials/parallelize/#__codelineno-11-1) export AGL_SERVER_HOST=10.0.0.4 [](https://microsoft.github.io/agent-lightning/latest/tutorials/parallelize/#__codelineno-11-2) export AGL_SERVER_PORT=4747 [](https://microsoft.github.io/agent-lightning/latest/tutorials/parallelize/#__codelineno-11-3) export AGL_CURRENT_ROLE=runner [](https://microsoft.github.io/agent-lightning/latest/tutorials/parallelize/#__codelineno-11-4) python train_calc_agent.py --n-runners 4` The runner process automatically connects via [`LightningStoreClient`](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.LightningStoreClient " agentlightning.LightningStoreClient") . Adjust `--n-runners` to spawn the desired number of worker processes on that machine. **4\. Scale out as needed.** Repeat step 3 on as many machines as you need. When you are done, stop the algorithm process. However, since the runners are on different machines, the strategy WILL NOT send a cooperative stop signal to the connected runners. So you need to kill the runners on your own. This role-based launch mirrors what [`Trainer.fit`](https://microsoft.github.io/agent-lightning/latest/reference/trainer/#agentlightning.Trainer.fit " fit(agent, train_dataset=None, *, val_dataset=None)") does inside a single machine while letting you spread work across a fleet. Because every process shares the same training script, you keep a single source of truth for dataset loading, adapters, and tracers, but you can tune compute resources independently for the algorithm and rollout workers. ### Shared-memory Strategy[¶](https://microsoft.github.io/agent-lightning/latest/tutorials/parallelize/#shared-memory-strategy "Permanent link") [`SharedMemoryExecutionStrategy`](https://microsoft.github.io/agent-lightning/latest/reference/trainer/#agentlightning.SharedMemoryExecutionStrategy " agentlightning.SharedMemoryExecutionStrategy") keeps everything inside one process. The runner runs on the main thread (by default) while the algorithm lives on a Python thread guarded by [`LightningStoreThreaded`](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.LightningStoreThreaded " agentlightning.LightningStoreThreaded") . Use it when you want easier debugging with shared breakpoints and no serialization overhead, or minimal startup time for unit tests. It's not a good choice for many algorithms that require heavy model training because [`LightningStoreThreaded`](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.LightningStoreThreaded " agentlightning.LightningStoreThreaded") does not work for multiprocessing. Using it with multiprocessing algorithms will lead to undefined behavior. Sample configuration: `[](https://microsoft.github.io/agent-lightning/latest/tutorials/parallelize/#__codelineno-12-1) trainer = agl.Trainer( [](https://microsoft.github.io/agent-lightning/latest/tutorials/parallelize/#__codelineno-12-2) algorithm=algorithm, [](https://microsoft.github.io/agent-lightning/latest/tutorials/parallelize/#__codelineno-12-3) strategy="shm", [](https://microsoft.github.io/agent-lightning/latest/tutorials/parallelize/#__codelineno-12-4) )` You can further customize the init parameters of [`SharedMemoryExecutionStrategy`](https://microsoft.github.io/agent-lightning/latest/reference/trainer/#agentlightning.SharedMemoryExecutionStrategy " agentlightning.SharedMemoryExecutionStrategy") . With `main_thread="runner"`, the runner occupies the main thread and `n_runners` must be `1`. The strategy respects `AGL_MANAGED_STORE`; set it to `0` to opt out of the `LightningStoreThreaded` wrapper. Parallelizing Algorithms[¶](https://microsoft.github.io/agent-lightning/latest/tutorials/parallelize/#parallelizing-algorithms "Permanent link") ------------------------------------------------------------------------------------------------------------------------------------------------- Runner parallelism scales rollout throughput, but the algorithm loop remains a single-process loop inside the execution strategy. We understand that many algorithms have parallelization built in, but that's outside the parallelization scope of Agent-lightning. Agent-lightning strives to make algorithms’ own parallelization work well under our execution strategies. The biggest challenge turns out to come from the store. For example, [`VERL`](https://microsoft.github.io/agent-lightning/latest/algorithm-zoo/verl/#agentlightning.algorithm.verl.VERL " VERL") uses [Ray](https://www.ray.io/) and launches [FSDP](https://docs.pytorch.org/tutorials/intermediate/FSDP_tutorial.html) and [vLLM](https://vllm.ai/) components internally. [`ClientServerExecutionStrategy`](https://microsoft.github.io/agent-lightning/latest/reference/trainer/#agentlightning.ClientServerExecutionStrategy " agentlightning.ClientServerExecutionStrategy") has to make sure that the server is not simultaneously serving in multiple processes or Ray workers, and that there is only one single authoritative source of truth for all subprocesses to connect to. Subprocesses connect to the store via a small [`LightningStoreClient`](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.LightningStoreClient " agentlightning.LightningStoreClient") bundled within [`LightningStoreServer`](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.LightningStoreServer " agentlightning.LightningStoreServer") . Note The [birds' eye view](https://microsoft.github.io/agent-lightning/latest/deep-dive/birds-eye-view/#birds-eye-view-client-server-strategy "Client-server Strategy") illustrates how adapters, proxies, and stores interact when the algorithm spawns additional workers. Use that diagram as a checklist when introducing new distributed components. Parallelizing [`LightningStore`](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.LightningStore " agentlightning.LightningStore") [¶](https://microsoft.github.io/agent-lightning/latest/tutorials/parallelize/#parallelizing-lightningstore "Permanent link") ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- By default, Agent-lightning persists rollouts and spans in an in-memory store. [`Trainer.fit`](https://microsoft.github.io/agent-lightning/latest/reference/trainer/#agentlightning.Trainer.fit " fit(agent, train_dataset=None, *, val_dataset=None)") spins it up automatically, or you can launch it yourself via the [`agl store` command](https://microsoft.github.io/agent-lightning/latest/reference/cli/) . [`InMemoryLightningStore`](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.InMemoryLightningStore " agentlightning.InMemoryLightningStore") keeps all state inside the current process, which makes local iteration fast but introduces two production constraints: 1. Spans are evicted once the process crosses its memory cap, so long runs risk data loss unless the host has abundant RAM. 2. Although the store is well optimized via asynchronous programming, the store lives in a single process and remains bound by the GIL, preventing it from saturating multi-core machines. General note for all server-client stores If your algorithm and runners communicate through HTTP protocol (which should be the default for 99% of the cases), you need to ensure the file limit is sufficiently large to avoid the "Too many open files" error. You can set the file limit by running the following command: `[](https://microsoft.github.io/agent-lightning/latest/tutorials/parallelize/#__codelineno-13-1) ulimit -n 100000` For resilient runs, switch to a persistent backend such as [`MongoLightningStore`](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.store.mongo.MongoLightningStore " agentlightning.store.mongo.MongoLightningStore") , which writes data to MongoDB instead of local RAM. Agent-lightning relies on [pymongo](https://pymongo.readthedocs.io/en/stable/) to interact with MongoDB, which can be installed via: `[](https://microsoft.github.io/agent-lightning/latest/tutorials/parallelize/#__codelineno-14-1) pip install agentlightning[mongo]` To use the MongoDB store, you need to pass the MongoDB URI to the store constructor. The URI should be in the format of `mongodb://:/?replicaSet=`. `[](https://microsoft.github.io/agent-lightning/latest/tutorials/parallelize/#__codelineno-15-1) from agentlightning.store.mongo import MongoLightningStore [](https://microsoft.github.io/agent-lightning/latest/tutorials/parallelize/#__codelineno-15-2) [](https://microsoft.github.io/agent-lightning/latest/tutorials/parallelize/#__codelineno-15-3) trainer = agl.Trainer( [](https://microsoft.github.io/agent-lightning/latest/tutorials/parallelize/#__codelineno-15-4) algorithm=algorithm, [](https://microsoft.github.io/agent-lightning/latest/tutorials/parallelize/#__codelineno-15-5) store=MongoLightningStore(mongo_uri="mongodb://localhost:27017/?replicaSet=rs0"), [](https://microsoft.github.io/agent-lightning/latest/tutorials/parallelize/#__codelineno-15-6) )` Setting up MongoDB MongoDB is a popular document-oriented database. Before running Agent-lightning with [`MongoLightningStore`](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.store.mongo.MongoLightningStore " agentlightning.store.mongo.MongoLightningStore") , make sure that you've already had a MongoDB instance running. Setting up can be conveniently done via Docker Compose via [compose.mongo.yml](https://github.com/microsoft/agent-lightning/tree/c746af2f76bebcae56007a59c223cdace411c89b/docker/compose.mongo.yml) . Unless targeting serious production use, we recommend creating the data folders and setting them to `777` permission to avoid permission issues. `[](https://microsoft.github.io/agent-lightning/latest/tutorials/parallelize/#__codelineno-16-1) mkdir -p data/mongo-host [](https://microsoft.github.io/agent-lightning/latest/tutorials/parallelize/#__codelineno-16-2) chmod 777 data/mongo-host [](https://microsoft.github.io/agent-lightning/latest/tutorials/parallelize/#__codelineno-16-3) docker compose -f compose.mongo.yml up -d` Alternatively, you can also install MongoDB manually following the [official documentation](https://www.mongodb.com/docs/manual/installation/) . If you installed MongoDB manually, an important note is that you need to ensure that the MongoDB instance has enabled replica set feature, since Agent-lightning uses the transactional operations internally. The simplest approach is to use the following script (executed in the MongoDB shell) to initialize the replica set: `[](https://microsoft.github.io/agent-lightning/latest/tutorials/parallelize/#__codelineno-17-1) rs.initiate({ [](https://microsoft.github.io/agent-lightning/latest/tutorials/parallelize/#__codelineno-17-2) _id: "rs0", [](https://microsoft.github.io/agent-lightning/latest/tutorials/parallelize/#__codelineno-17-3) members: [{ _id: 0, host: "localhost:27017" }], [](https://microsoft.github.io/agent-lightning/latest/tutorials/parallelize/#__codelineno-17-4) });` To scale out further, launch the store server via [`agl store --backend mongo`](https://microsoft.github.io/agent-lightning/latest/reference/cli/) (see [Debugging with External Store](https://microsoft.github.io/agent-lightning/latest/tutorials/debug/#debug-with-external-store "Debug the Algorithm-Runner Boundary") ). The CLI accepts `--n-workers`, which starts the server under `gunicorn` with multiple worker processes so concurrent runners can push and pull at higher throughput. This option applies only to persistent backends; an in-memory store, on the other hand, cannot be sharded across workers because its state lives inside one process. Note The `--n-workers` here is the number of worker processes for the store server, NOT related to the number of rollout runners. Increasing Throughput of LLM Proxy[¶](https://microsoft.github.io/agent-lightning/latest/tutorials/parallelize/#increasing-throughput-of-llm-proxy "Permanent link") --------------------------------------------------------------------------------------------------------------------------------------------------------------------- Agent-lightning includes an optional [`LLMProxy`](https://microsoft.github.io/agent-lightning/latest/reference/algorithm/#agentlightning.LLMProxy " agentlightning.LLMProxy") that wraps [LiteLLM](https://docs.litellm.ai/) to provide a unified OpenAI-compatible endpoint for your agents. When rollout throughput increases, the proxy can become a bottleneck. You can scale it out using the same pattern as the store server. To increase proxy throughput, pass `num_workers` when constructing the proxy: `[](https://microsoft.github.io/agent-lightning/latest/tutorials/parallelize/#__codelineno-18-1) import agentlightning as agl [](https://microsoft.github.io/agent-lightning/latest/tutorials/parallelize/#__codelineno-18-2) [](https://microsoft.github.io/agent-lightning/latest/tutorials/parallelize/#__codelineno-18-3) proxy = agl.LLMProxy( [](https://microsoft.github.io/agent-lightning/latest/tutorials/parallelize/#__codelineno-18-4) port=4000, [](https://microsoft.github.io/agent-lightning/latest/tutorials/parallelize/#__codelineno-18-5) launch_mode="mp", # multiprocessing mode [](https://microsoft.github.io/agent-lightning/latest/tutorials/parallelize/#__codelineno-18-6) num_workers=4, # four gunicorn workers handle concurrent requests [](https://microsoft.github.io/agent-lightning/latest/tutorials/parallelize/#__codelineno-18-7) )` You can also configure the proxy through [`Trainer`](https://microsoft.github.io/agent-lightning/latest/reference/trainer/#agentlightning.Trainer " agentlightning.Trainer") : `[](https://microsoft.github.io/agent-lightning/latest/tutorials/parallelize/#__codelineno-19-1) trainer = agl.Trainer( [](https://microsoft.github.io/agent-lightning/latest/tutorials/parallelize/#__codelineno-19-2) algorithm=algorithm, [](https://microsoft.github.io/agent-lightning/latest/tutorials/parallelize/#__codelineno-19-3) n_runners=8, # The runners here is the rollout runners, not related to LLM proxy replicas [](https://microsoft.github.io/agent-lightning/latest/tutorials/parallelize/#__codelineno-19-4) llm_proxy={"port": 4000, "num_workers": 4}, # launch mode is actually mp by default [](https://microsoft.github.io/agent-lightning/latest/tutorials/parallelize/#__codelineno-19-5) )` When `num_workers > 1`, the launcher starts gunicorn with the specified number of worker processes. Each worker runs its own event loop, allowing the proxy to handle many concurrent LLM requests without being blocked by Python's GIL. Tip When using `mp` launch mode, [`LLMProxy`](https://microsoft.github.io/agent-lightning/latest/reference/algorithm/#agentlightning.LLMProxy " agentlightning.LLMProxy") will start the server in a separate process. To make sure the proxy is still accessing the same store as the main process, you need to set the store to be [zero-copy compatible](https://microsoft.github.io/agent-lightning/latest/deep-dive/store/#store-capabilities "Capabilities") , which means, either the store is a native zero-copy store like [`MongoLightningStore`](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.store.mongo.MongoLightningStore " agentlightning.store.mongo.MongoLightningStore") or the store is wrapped via [`LightningStoreServer`](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.LightningStoreServer " agentlightning.LightningStoreServer") or [`LightningStoreClient`](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.LightningStoreClient " agentlightning.LightningStoreClient") . Shared Server Infrastructure Both [`LightningStoreServer`](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.LightningStoreServer " agentlightning.LightningStoreServer") and [`LLMProxy`](https://microsoft.github.io/agent-lightning/latest/reference/algorithm/#agentlightning.LLMProxy " agentlightning.LLMProxy") rely on a common utility called [`PythonServerLauncherArgs`](https://microsoft.github.io/agent-lightning/latest/reference/utilities/#agentlightning.utils.server_launcher.PythonServerLauncherArgs " agentlightning.utils.server_launcher.PythonServerLauncherArgs dataclass ") . This dataclass captures the settings needed to launch a FastAPI application: `[](https://microsoft.github.io/agent-lightning/latest/tutorials/parallelize/#__codelineno-20-1) from agentlightning.utils import PythonServerLauncherArgs [](https://microsoft.github.io/agent-lightning/latest/tutorials/parallelize/#__codelineno-20-2) [](https://microsoft.github.io/agent-lightning/latest/tutorials/parallelize/#__codelineno-20-3) args = PythonServerLauncherArgs( [](https://microsoft.github.io/agent-lightning/latest/tutorials/parallelize/#__codelineno-20-4) port=8000, [](https://microsoft.github.io/agent-lightning/latest/tutorials/parallelize/#__codelineno-20-5) host="0.0.0.0", [](https://microsoft.github.io/agent-lightning/latest/tutorials/parallelize/#__codelineno-20-6) n_workers=4, # spawn 4 gunicorn workers [](https://microsoft.github.io/agent-lightning/latest/tutorials/parallelize/#__codelineno-20-7) launch_mode="thread", # or "mp" for multiprocessing, "asyncio" for in-loop [](https://microsoft.github.io/agent-lightning/latest/tutorials/parallelize/#__codelineno-20-8) )` Under the hood, [`PythonServerLauncher`](https://microsoft.github.io/agent-lightning/latest/reference/utilities/#agentlightning.utils.server_launcher.PythonServerLauncher " agentlightning.utils.server_launcher.PythonServerLauncher") reads these arguments and chooses between uvicorn (single worker) and gunicorn (multiple workers) automatically. Back to top --- # Types - Agent-lightning [Skip to content](https://microsoft.github.io/agent-lightning/latest/reference/types/#type-references) ⚠️ You are viewing development documentation. For stable documentation, please visit [https://microsoft.github.io/agent-lightning/stable/](https://microsoft.github.io/agent-lightning/stable/) Type References[¶](https://microsoft.github.io/agent-lightning/latest/reference/types/#type-references "Permanent link") ========================================================================================================================= Core Types[¶](https://microsoft.github.io/agent-lightning/latest/reference/types/#core-types "Permanent link") --------------------------------------------------------------------------------------------------------------- ### `agentlightning.Triplet` [¶](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.Triplet "Permanent link") Bases: `BaseModel` Single interaction turn captured during reinforcement learning. ### `agentlightning.RolloutRawResult = Union[None, float, List[ReadableSpan], List[Span], List[SpanCoreFields]]` `module-attribute` [¶](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.RolloutRawResult "Permanent link") Rollout result type. Possible return values of [`rollout`](https://microsoft.github.io/agent-lightning/latest/reference/agent/#agentlightning.LitAgent.rollout " rollout(task, resources, rollout)") . ### `agentlightning.RolloutMode = Literal['train', 'val', 'test']` `module-attribute` [¶](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.RolloutMode "Permanent link") Possible rollout modes. ### `agentlightning.GenericResponse` [¶](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.GenericResponse "Permanent link") Bases: `BaseModel` Generic server response used by compatibility endpoints. Deprecated This response is no longer used by the new [`LightningStore`](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.LightningStore " agentlightning.LightningStore") APIs. Attributes: * **`status`** (`str`) – Status string describing the result of the request. * **`message`** (`Optional[str]`) – Optional human readable explanation. * **`data`** (`Optional[Dict[str, Any]]`) – Arbitrary payload serialized as JSON. ### `agentlightning.ParallelWorkerBase` [¶](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.ParallelWorkerBase "Permanent link") Base class for workloads executed across multiple worker processes. The lifecycle is orchestrated by the main process: * [`init()`](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.ParallelWorkerBase.init " init(*args, **kwargs)") prepares shared state. * Each worker calls [`init_worker()`](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.ParallelWorkerBase.init_worker " init_worker(worker_id, *args, **kwargs)") during start-up. * [`run()`](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.ParallelWorkerBase.run " run(*args, **kwargs)") performs the parallel workload. * Workers call [`teardown_worker()`](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.ParallelWorkerBase.teardown_worker " teardown_worker(worker_id, *args, **kwargs)") before exiting. * The main process finalizes through [`teardown()`](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.ParallelWorkerBase.teardown " teardown(*args, **kwargs)") . Subclasses must implement [`run()`](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.ParallelWorkerBase.run " run(*args, **kwargs)") and can override other lifecycle hooks. #### `__init__()` [¶](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.ParallelWorkerBase.__init__ "Permanent link") Initialize the base class. This method can be overridden by subclasses. #### `init(*args, **kwargs)` [¶](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.ParallelWorkerBase.init "Permanent link") Initialize before spawning the workers. This method can be overridden by subclasses. #### `init_worker(worker_id, *args, **kwargs)` [¶](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.ParallelWorkerBase.init_worker "Permanent link") Initialize the worker. This method can be overridden by subclasses. #### `run(*args, **kwargs)` [¶](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.ParallelWorkerBase.run "Permanent link") Run the workload. This method can be overridden by subclasses. #### `teardown(*args, **kwargs)` [¶](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.ParallelWorkerBase.teardown "Permanent link") Teardown after the workers have exited. This method can be overridden by subclasses. #### `teardown_worker(worker_id, *args, **kwargs)` [¶](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.ParallelWorkerBase.teardown_worker "Permanent link") Teardown the worker. This method can be overridden by subclasses. ### `agentlightning.Dataset` [¶](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.Dataset "Permanent link") Bases: `Protocol`, `Generic[T_co]` The general interface for a dataset. It's currently implemented as a protocol, having a similar interface to `torch.utils.data.Dataset`. You don't have to inherit from this class; you can use a simple list if you want to. ### `agentlightning.AttemptStatus = Literal['preparing', 'running', 'failed', 'succeeded', 'unresponsive', 'timeout']` `module-attribute` [¶](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.AttemptStatus "Permanent link") The status of an attempt. ### `agentlightning.RolloutStatus = Literal['queuing', 'preparing', 'running', 'failed', 'succeeded', 'cancelled', 'requeuing']` `module-attribute` [¶](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.RolloutStatus "Permanent link") The status of a rollout. ### `agentlightning.RolloutConfig` [¶](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.RolloutConfig "Permanent link") Bases: `BaseModel` Configuration controlling rollout retries and timeouts. #### `max_attempts = Field(default=1, ge=1)` `class-attribute` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.RolloutConfig.max_attempts "Permanent link") The maximum number of attempts for the rollout, including the first attempt. #### `retry_condition = Field(default_factory=(cast(Callable[[], List[AttemptStatus]], list)))` `class-attribute` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.RolloutConfig.retry_condition "Permanent link") The list of statuses that should trigger a retry. #### `timeout_seconds = None` `class-attribute` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.RolloutConfig.timeout_seconds "Permanent link") The timeout for the rollout, in seconds. None indicates no timeout. #### `unresponsive_seconds = None` `class-attribute` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.RolloutConfig.unresponsive_seconds "Permanent link") The unresponsive timeout for the rollout, in seconds. None indicates no unresponsive timeout. ### `agentlightning.Rollout` [¶](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.Rollout "Permanent link") Bases: `BaseModel` #### `config = Field(default_factory=RolloutConfig)` `class-attribute` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.Rollout.config "Permanent link") Retry and timeout configuration associated with the rollout. #### `end_time = None` `class-attribute` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.Rollout.end_time "Permanent link") Timestamp when the rollout ended. #### `input` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.Rollout.input "Permanent link") Task input used to generate the rollout. #### `metadata = None` `class-attribute` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.Rollout.metadata "Permanent link") Additional metadata attached to the rollout. #### `mode = None` `class-attribute` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.Rollout.mode "Permanent link") Execution mode such as `"train"`, `"val"` or `"test"`. See [`RolloutMode`](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.RolloutMode " agentlightning.RolloutMode = Literal['train', 'val', 'test'] module-attribute ") . #### `resources_id = None` `class-attribute` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.Rollout.resources_id "Permanent link") Identifier of the resources required to execute the rollout. #### `rollout_id` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.Rollout.rollout_id "Permanent link") Unique identifier for the rollout. #### `start_time` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.Rollout.start_time "Permanent link") Timestamp when the rollout started. #### `status = 'queuing'` `class-attribute` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.Rollout.status "Permanent link") Latest status emitted by the controller. ### `agentlightning.EnqueueRolloutRequest` [¶](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.EnqueueRolloutRequest "Permanent link") Bases: `BaseModel` Payload describing a rollout to be queued via [`enqueue_rollout`](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.LightningStore.enqueue_rollout " enqueue_rollout(input, mode=None, resources_id=None, config=None, metadata=None) async ") . A subset of fields from [`Rollout`](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.Rollout " agentlightning.Rollout") used for queuing new rollouts. #### `config = None` `class-attribute` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.EnqueueRolloutRequest.config "Permanent link") Retry and timeout configuration associated with the rollout. #### `input` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.EnqueueRolloutRequest.input "Permanent link") Task input used to generate the rollout. #### `metadata = None` `class-attribute` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.EnqueueRolloutRequest.metadata "Permanent link") Additional metadata attached to the rollout. #### `mode = None` `class-attribute` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.EnqueueRolloutRequest.mode "Permanent link") Execution mode such as `"train"`, `"val"` or `"test"`. See [`RolloutMode`](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.RolloutMode " agentlightning.RolloutMode = Literal['train', 'val', 'test'] module-attribute ") . #### `resources_id = None` `class-attribute` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.EnqueueRolloutRequest.resources_id "Permanent link") Identifier of the resources required to execute the rollout. ### `agentlightning.Attempt` [¶](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.Attempt "Permanent link") Bases: `BaseModel` Execution attempt for a rollout, including metadata for retries. #### `attempt_id` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.Attempt.attempt_id "Permanent link") The universal id for current attempt. #### `end_time = None` `class-attribute` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.Attempt.end_time "Permanent link") The time when the attempt has ended. #### `last_heartbeat_time = None` `class-attribute` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.Attempt.last_heartbeat_time "Permanent link") The last time when the worker has reported progress (i.e., a span). #### `metadata = None` `class-attribute` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.Attempt.metadata "Permanent link") A bucket for any other relevant information. #### `rollout_id` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.Attempt.rollout_id "Permanent link") The rollout which this attempt belongs to. #### `sequence_id` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.Attempt.sequence_id "Permanent link") The sequence number of the attempt, starting from 1. #### `start_time` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.Attempt.start_time "Permanent link") The time when the attempt has started. #### `status = 'preparing'` `class-attribute` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.Attempt.status "Permanent link") The status of the attempt. #### `worker_id = None` `class-attribute` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.Attempt.worker_id "Permanent link") The rollout worker which is executing this attempt. ### `agentlightning.AttemptedRollout` [¶](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.AttemptedRollout "Permanent link") Bases: `[Rollout](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.Rollout " agentlightning.Rollout (agentlightning.types.core.Rollout)") ` Rollout paired with the currently active attempt. #### `attempt` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.AttemptedRollout.attempt "Permanent link") The attempt that is currently processing the rollout. ### `agentlightning.Worker` [¶](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.Worker "Permanent link") Bases: `BaseModel` Worker information. This is actually the same as Runner info. #### `current_attempt_id = None` `class-attribute` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.Worker.current_attempt_id "Permanent link") The ID of the current attempt that the worker is processing. #### `current_rollout_id = None` `class-attribute` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.Worker.current_rollout_id "Permanent link") The ID of the current rollout that the worker is processing. #### `heartbeat_stats = None` `class-attribute` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.Worker.heartbeat_stats "Permanent link") Statistics about the worker's heartbeat. #### `last_busy_time = None` `class-attribute` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.Worker.last_busy_time "Permanent link") The last time when the worker has started an attempt and became busy. #### `last_dequeue_time = None` `class-attribute` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.Worker.last_dequeue_time "Permanent link") The last time when the worker has tried to dequeue a rollout. #### `last_heartbeat_time = None` `class-attribute` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.Worker.last_heartbeat_time "Permanent link") The last time when the worker has reported the stats. #### `last_idle_time = None` `class-attribute` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.Worker.last_idle_time "Permanent link") The last time when the worker has triggered the end of an attempt and became idle. #### `status = 'unknown'` `class-attribute` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.Worker.status "Permanent link") The status of the worker. #### `worker_id` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.Worker.worker_id "Permanent link") The ID of the worker. ### `agentlightning.WorkerStatus = Literal['idle', 'busy', 'unknown']` `module-attribute` [¶](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.WorkerStatus "Permanent link") ### `agentlightning.Hook` [¶](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.Hook "Permanent link") Bases: `[ParallelWorkerBase](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.ParallelWorkerBase " agentlightning.ParallelWorkerBase (agentlightning.types.core.ParallelWorkerBase)") ` Base class for defining hooks in the agent runner's lifecycle. #### `on_rollout_end(*, agent, runner, rollout, spans)` `async` [¶](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.Hook.on_rollout_end "Permanent link") Hook called after a rollout _attempt_ completes. Parameters: * **`agent`** (`[LitAgent](https://microsoft.github.io/agent-lightning/latest/reference/agent/#agentlightning.LitAgent " agentlightning.LitAgent (agentlightning.litagent.LitAgent)") [Any]`) – The [`LitAgent`](https://microsoft.github.io/agent-lightning/latest/reference/agent/#agentlightning.LitAgent " agentlightning.LitAgent") instance associated with the runner. * **`runner`** (`[Runner](https://microsoft.github.io/agent-lightning/latest/reference/runner/#agentlightning.Runner " agentlightning.Runner (agentlightning.runner.base.Runner)") [Any]`) – The [`Runner`](https://microsoft.github.io/agent-lightning/latest/reference/runner/#agentlightning.Runner " agentlightning.Runner") managing the rollout. * **`rollout`** (`[Rollout](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.Rollout " agentlightning.Rollout (agentlightning.types.core.Rollout)") `) – The [`Rollout`](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.Rollout " agentlightning.Rollout") object that has been processed. * **`spans`** (`Union[List[ReadableSpan], List[[Span](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.Span " agentlightning.Span (agentlightning.types.tracer.Span)") ]]`) – The spans that have been added to the store. Subclasses can override this method for cleanup or additional logging. By default, this is a no-op. #### `on_rollout_start(*, agent, runner, rollout)` `async` [¶](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.Hook.on_rollout_start "Permanent link") Hook called immediately before a rollout _attempt_ begins. Parameters: * **`agent`** (`[LitAgent](https://microsoft.github.io/agent-lightning/latest/reference/agent/#agentlightning.LitAgent " agentlightning.LitAgent (agentlightning.litagent.LitAgent)") [Any]`) – The [`LitAgent`](https://microsoft.github.io/agent-lightning/latest/reference/agent/#agentlightning.LitAgent " agentlightning.LitAgent") instance associated with the runner. * **`runner`** (`[Runner](https://microsoft.github.io/agent-lightning/latest/reference/runner/#agentlightning.Runner " agentlightning.Runner (agentlightning.runner.base.Runner)") [Any]`) – The [`Runner`](https://microsoft.github.io/agent-lightning/latest/reference/runner/#agentlightning.Runner " agentlightning.Runner") managing the rollout. * **`rollout`** (`[Rollout](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.Rollout " agentlightning.Rollout (agentlightning.types.core.Rollout)") `) – The [`Rollout`](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.Rollout " agentlightning.Rollout") object that will be processed. Subclasses can override this method to implement custom logic such as logging, metric collection, or resource setup. By default, this is a no-op. #### `on_trace_end(*, agent, runner, tracer, rollout)` `async` [¶](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.Hook.on_trace_end "Permanent link") Hook called immediately after the rollout completes but before the tracer exits the trace context. Parameters: * **`agent`** (`[LitAgent](https://microsoft.github.io/agent-lightning/latest/reference/agent/#agentlightning.LitAgent " agentlightning.LitAgent (agentlightning.litagent.LitAgent)") [Any]`) – The [`LitAgent`](https://microsoft.github.io/agent-lightning/latest/reference/agent/#agentlightning.LitAgent " agentlightning.LitAgent") instance associated with the runner. * **`runner`** (`[Runner](https://microsoft.github.io/agent-lightning/latest/reference/runner/#agentlightning.Runner " agentlightning.Runner (agentlightning.runner.base.Runner)") [Any]`) – The [`Runner`](https://microsoft.github.io/agent-lightning/latest/reference/runner/#agentlightning.Runner " agentlightning.Runner") managing the rollout. * **`tracer`** (`[Tracer](https://microsoft.github.io/agent-lightning/latest/reference/runner/#agentlightning.Tracer " agentlightning.Tracer (agentlightning.tracer.base.Tracer)") `) – The [`Tracer`](https://microsoft.github.io/agent-lightning/latest/reference/runner/#agentlightning.Tracer " agentlightning.Tracer") instance associated with the runner. * **`rollout`** (`[Rollout](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.Rollout " agentlightning.Rollout (agentlightning.types.core.Rollout)") `) – The [`Rollout`](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.Rollout " agentlightning.Rollout") object that has been processed. Subclasses can override this method to implement custom logic such as logging, metric collection, or resource cleanup. By default, this is a no-op. #### `on_trace_start(*, agent, runner, tracer, rollout)` `async` [¶](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.Hook.on_trace_start "Permanent link") Hook called immediately after the tracer enters the trace context but before the rollout begins. Parameters: * **`agent`** (`[LitAgent](https://microsoft.github.io/agent-lightning/latest/reference/agent/#agentlightning.LitAgent " agentlightning.LitAgent (agentlightning.litagent.LitAgent)") [Any]`) – The [`LitAgent`](https://microsoft.github.io/agent-lightning/latest/reference/agent/#agentlightning.LitAgent " agentlightning.LitAgent") instance associated with the runner. * **`runner`** (`[Runner](https://microsoft.github.io/agent-lightning/latest/reference/runner/#agentlightning.Runner " agentlightning.Runner (agentlightning.runner.base.Runner)") [Any]`) – The [`Runner`](https://microsoft.github.io/agent-lightning/latest/reference/runner/#agentlightning.Runner " agentlightning.Runner") managing the rollout. * **`tracer`** (`[Tracer](https://microsoft.github.io/agent-lightning/latest/reference/runner/#agentlightning.Tracer " agentlightning.Tracer (agentlightning.tracer.base.Tracer)") `) – The [`Tracer`](https://microsoft.github.io/agent-lightning/latest/reference/runner/#agentlightning.Tracer " agentlightning.Tracer") instance associated with the runner. * **`rollout`** (`[Rollout](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.Rollout " agentlightning.Rollout (agentlightning.types.core.Rollout)") `) – The [`Rollout`](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.Rollout " agentlightning.Rollout") object that will be processed. Subclasses can override this method to implement custom logic such as logging, metric collection, or resource setup. By default, this is a no-op. ### `agentlightning.PaginatedResult` [¶](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.PaginatedResult "Permanent link") Bases: `BaseModel`, `Sequence[T_item]` Result of a paginated query. Behaves like a sequence, but also carries pagination metadata (limit, offset, total). #### `items` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.PaginatedResult.items "Permanent link") Items in the result. #### `limit` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.PaginatedResult.limit "Permanent link") Limit of the result. #### `offset` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.PaginatedResult.offset "Permanent link") Offset of the result. #### `total` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.PaginatedResult.total "Permanent link") Total number of items in the collection. ### `agentlightning.FilterOptions = Mapping[Union[str, Literal['_aggregate', '_must']], Union[FilterField, Literal['and', 'or'], Mapping[str, FilterField]]]` `module-attribute` [¶](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.FilterOptions "Permanent link") A mapping of field name -> operator dict. Each operator dict can contain: * "exact": value for exact equality. * "within": iterable of allowed values. * "contains": substring to search for in string fields. The filter can also have a special field called "\_aggregate" that can be used to specify the logic to combine the results of the filters: * "and": all conditions must match. This is the default value if not specified. * "or": at least one condition must match. All conditions within a field and between different fields are stored in a unified pool and combined using `_aggregate`. The filter can also have a special group called "\_must", which is a mapping of filters that must all match, no matter whether the aggregate logic is "and" or "or". Example: `[](https://microsoft.github.io/agent-lightning/latest/reference/types/#__codelineno-0-1) { [](https://microsoft.github.io/agent-lightning/latest/reference/types/#__codelineno-0-2) "_aggregate": "or", [](https://microsoft.github.io/agent-lightning/latest/reference/types/#__codelineno-0-3) "_must": { [](https://microsoft.github.io/agent-lightning/latest/reference/types/#__codelineno-0-4) "city": {"exact": "New York"}, [](https://microsoft.github.io/agent-lightning/latest/reference/types/#__codelineno-0-5) "timezone": {"within": ["America/New_York", "America/Los_Angeles"]}, [](https://microsoft.github.io/agent-lightning/latest/reference/types/#__codelineno-0-6) }, [](https://microsoft.github.io/agent-lightning/latest/reference/types/#__codelineno-0-7) "status": {"exact": "active"}, [](https://microsoft.github.io/agent-lightning/latest/reference/types/#__codelineno-0-8) "id": {"within": [1, 2, 3]}, [](https://microsoft.github.io/agent-lightning/latest/reference/types/#__codelineno-0-9) "name": {"contains": "foo"}, [](https://microsoft.github.io/agent-lightning/latest/reference/types/#__codelineno-0-10) }` ### `agentlightning.SortOptions` [¶](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.SortOptions "Permanent link") Bases: `TypedDict` Options for sorting the collection. #### `name` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.SortOptions.name "Permanent link") The name of the field to sort by. #### `order` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.SortOptions.order "Permanent link") The order to sort by. ### `agentlightning.FilterField` [¶](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.FilterField "Permanent link") Bases: `TypedDict` An operator dict for a single field. Resources[¶](https://microsoft.github.io/agent-lightning/latest/reference/types/#resources "Permanent link") ------------------------------------------------------------------------------------------------------------- ### `agentlightning.Resource` [¶](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.Resource "Permanent link") Bases: `BaseModel` Base class for tunable resources distributed to executors. #### `resource_type` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.Resource.resource_type "Permanent link") Alias of the resource type. ### `agentlightning.LLM` [¶](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.LLM "Permanent link") Bases: `[Resource](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.Resource " agentlightning.Resource (agentlightning.types.resources.Resource)") ` Resource that identifies an LLM endpoint and its configuration. #### `api_key = None` `class-attribute` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.LLM.api_key "Permanent link") Optional secret used to authenticate requests. #### `endpoint` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.LLM.endpoint "Permanent link") The URL of the LLM API endpoint. #### `model` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.LLM.model "Permanent link") The identifier for the model to be used (e.g., 'gpt-4o'). #### `sampling_parameters = Field(default_factory=dict)` `class-attribute` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.LLM.sampling_parameters "Permanent link") A dictionary of hyperparameters for model inference, such as temperature, top\_p, etc. #### `get_base_url(*args, **kwargs)` [¶](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.LLM.get_base_url "Permanent link") Return the base URL consumed by OpenAI-compatible clients. Users are encouraged to use `get_base_url(rollout_id, attempt_id)` to get the LLM endpoint instead of accessing `.endpoint` directly. ### `agentlightning.ProxyLLM` [¶](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.ProxyLLM "Permanent link") Bases: `[LLM](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.LLM " agentlightning.LLM (agentlightning.types.resources.LLM)") ` LLM resource that rewrites endpoints through [`LLMProxy`](https://microsoft.github.io/agent-lightning/latest/reference/algorithm/#agentlightning.LLMProxy " agentlightning.LLMProxy") . The proxy injects rollout- and attempt-specific routing information into the endpoint so that downstream services can attribute requests correctly. #### `__getattribute__(name)` [¶](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.ProxyLLM.__getattribute__ "Permanent link") Emit a warning when `endpoint` is accessed directly after initialization. #### `get_base_url(rollout_id, attempt_id)` [¶](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.ProxyLLM.get_base_url "Permanent link") Return the routed endpoint for a specific rollout/attempt pair. Parameters: * **`rollout_id`** (`Optional[str]`) – Identifier of the rollout making the request. * **`attempt_id`** (`Optional[str]`) – Identifier of the attempt within that rollout. Returns: * `str` – Fully qualified endpoint including rollout metadata. Raises: * `ValueError` – If exactly one of `rollout_id` or `attempt_id` is provided. #### `model_post_init(__context)` [¶](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.ProxyLLM.model_post_init "Permanent link") Mark initialization as complete after Pydantic finishes setup. #### `with_attempted_rollout(rollout)` [¶](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.ProxyLLM.with_attempted_rollout "Permanent link") Bake rollout metadata into a concrete [`LLM`](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.LLM " agentlightning.LLM") instance. ### `agentlightning.PromptTemplate` [¶](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.PromptTemplate "Permanent link") Bases: `[Resource](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.Resource " agentlightning.Resource (agentlightning.types.resources.Resource)") ` Resource describing a reusable prompt template. #### `engine` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.PromptTemplate.engine "Permanent link") The templating engine to use for rendering the prompt. #### `template` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.PromptTemplate.template "Permanent link") The template string. The format depends on the engine. #### `format(**kwargs)` [¶](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.PromptTemplate.format "Permanent link") Format the prompt using keyword arguments. Warning Only the `f-string` engine is supported for now. ### `agentlightning.ResourceUnion = Annotated[Union[LLM, ProxyLLM, PromptTemplate], Field(discriminator='resource_type')]` `module-attribute` [¶](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.ResourceUnion "Permanent link") ### `agentlightning.NamedResources = Dict[str, ResourceUnion]` `module-attribute` [¶](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.NamedResources "Permanent link") Mapping from resource names to their configured instances. Examples: `[](https://microsoft.github.io/agent-lightning/latest/reference/types/#__codelineno-0-1) resources: NamedResources = { [](https://microsoft.github.io/agent-lightning/latest/reference/types/#__codelineno-0-2) "main_llm": LLM( [](https://microsoft.github.io/agent-lightning/latest/reference/types/#__codelineno-0-3) endpoint="http://localhost:8080", [](https://microsoft.github.io/agent-lightning/latest/reference/types/#__codelineno-0-4) model="llama3", [](https://microsoft.github.io/agent-lightning/latest/reference/types/#__codelineno-0-5) sampling_parameters={"temperature": 0.7, "max_tokens": 100}, [](https://microsoft.github.io/agent-lightning/latest/reference/types/#__codelineno-0-6) ), [](https://microsoft.github.io/agent-lightning/latest/reference/types/#__codelineno-0-7) "system_prompt": PromptTemplate( [](https://microsoft.github.io/agent-lightning/latest/reference/types/#__codelineno-0-8) template="You are a helpful assistant.", [](https://microsoft.github.io/agent-lightning/latest/reference/types/#__codelineno-0-9) engine="f-string", [](https://microsoft.github.io/agent-lightning/latest/reference/types/#__codelineno-0-10) ), [](https://microsoft.github.io/agent-lightning/latest/reference/types/#__codelineno-0-11) }` ### `agentlightning.ResourcesUpdate` [¶](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.ResourcesUpdate "Permanent link") Bases: `BaseModel` Update payload broadcast to clients when resources change. #### `create_time` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.ResourcesUpdate.create_time "Permanent link") Timestamp of the creation time of the resources. #### `resources` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.ResourcesUpdate.resources "Permanent link") Mapping of resource names to their definitions. #### `resources_id` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.ResourcesUpdate.resources_id "Permanent link") Identifier used to version the resources. #### `update_time` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.ResourcesUpdate.update_time "Permanent link") Timestamp of the last update time of the resources. #### `version` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.ResourcesUpdate.version "Permanent link") Version of the resources. Traces[¶](https://microsoft.github.io/agent-lightning/latest/reference/types/#traces "Permanent link") ------------------------------------------------------------------------------------------------------- ### `agentlightning.AttributeValue = Union[str, bool, int, float, Sequence[str], Sequence[bool], Sequence[int], Sequence[float]]` `module-attribute` [¶](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.AttributeValue "Permanent link") Possible values for OpenTelemetry attributes. ### `agentlightning.Attributes = Dict[str, AttributeValue]` `module-attribute` [¶](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.Attributes "Permanent link") Mapping from attribute names to their values. Same as OpenTelemetry `Attributes` type. ### `agentlightning.TraceState = Dict[str, str]` `module-attribute` [¶](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.TraceState "Permanent link") Mapping from trace state key to its value. Same as OpenTelemetry `TraceState` type. ### `agentlightning.SpanContext` [¶](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.SpanContext "Permanent link") Bases: `BaseModel` Pydantic representation of `opentelemetry.trace.SpanContext` values. #### `is_remote` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.SpanContext.is_remote "Permanent link") Whether the span is remote. #### `span_id` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.SpanContext.span_id "Permanent link") The span ID of the span. #### `trace_id` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.SpanContext.trace_id "Permanent link") The trace ID of the span. #### `trace_state` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.SpanContext.trace_state "Permanent link") Mapping from trace state key to its value. #### `from_opentelemetry(src)` `classmethod` [¶](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.SpanContext.from_opentelemetry "Permanent link") Construct a [`SpanContext`](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.SpanContext " agentlightning.SpanContext") from OpenTelemetry data. ### `agentlightning.TraceStatus` [¶](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.TraceStatus "Permanent link") Bases: `BaseModel` Serializable variant of `opentelemetry.trace.Status`. #### `description = None` `class-attribute` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.TraceStatus.description "Permanent link") The description of the span. Same as OpenTelemetry `Status.description` type. #### `status_code` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.TraceStatus.status_code "Permanent link") The status code of the span. Same as OpenTelemetry `Status.status_code` type. #### `from_opentelemetry(src)` `classmethod` [¶](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.TraceStatus.from_opentelemetry "Permanent link") Create a [`TraceStatus`](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.TraceStatus " agentlightning.TraceStatus") from OpenTelemetry metadata. ### `agentlightning.Event` [¶](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.Event "Permanent link") Bases: `BaseModel` Serializable representation of OpenTelemetry `Event` values. #### `attributes` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.Event.attributes "Permanent link") Mapping from attribute names to their values. Same as OpenTelemetry `Attributes` type. #### `name` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.Event.name "Permanent link") The name of the event. #### `timestamp = None` `class-attribute` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.Event.timestamp "Permanent link") The timestamp of the event. Same as OpenTelemetry `Event.timestamp` type. #### `from_opentelemetry(src)` `classmethod` [¶](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.Event.from_opentelemetry "Permanent link") Create an [`Event`](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.Event " agentlightning.Event") from an OpenTelemetry event. ### `agentlightning.Link` [¶](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.Link "Permanent link") Bases: `BaseModel` Serializable representation of OpenTelemetry `Link` values. #### `attributes = None` `class-attribute` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.Link.attributes "Permanent link") Optional attributes. #### `context` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.Link.context "Permanent link") The context of the link. #### `from_opentelemetry(src)` `classmethod` [¶](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.Link.from_opentelemetry "Permanent link") Create a [`Link`](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.Link " agentlightning.Link") from an OpenTelemetry link. ### `agentlightning.Resource` [¶](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.Resource "Permanent link") Bases: `BaseModel` Base class for tunable resources distributed to executors. #### `resource_type` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.Resource.resource_type "Permanent link") Alias of the resource type. ### `agentlightning.Span` [¶](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.Span "Permanent link") Bases: `BaseModel` Agent Lightning's canonical span model used for persistence and analytics. The model captures the most relevant fields from `opentelemetry.sdk.trace.ReadableSpan` instances while preserving unmodeled attributes in Pydantic `BaseModel`'s extra storage. This keeps the serialized format stable even as upstream OpenTelemetry types evolve. #### `attempt_id` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.Span.attempt_id "Permanent link") The attempt which this span belongs to. #### `attributes` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.Span.attributes "Permanent link") The attributes of the span. See [OpenTelemetry docs](https://opentelemetry.io/docs/concepts/signals/traces/) . #### `context` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.Span.context "Permanent link") The context of the span. See [OpenTelemetry docs](https://opentelemetry.io/docs/concepts/signals/traces/) . #### `end_time` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.Span.end_time "Permanent link") The end time of the span. See [OpenTelemetry docs](https://opentelemetry.io/docs/concepts/signals/traces/) . #### `events` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.Span.events "Permanent link") The events of the span. See [OpenTelemetry docs](https://opentelemetry.io/docs/concepts/signals/traces/) . #### `links` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.Span.links "Permanent link") The links of the span. See [OpenTelemetry docs](https://opentelemetry.io/docs/concepts/signals/traces/) . #### `name` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.Span.name "Permanent link") The name of the span. See [OpenTelemetry docs](https://opentelemetry.io/docs/concepts/signals/traces/) . #### `parent` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.Span.parent "Permanent link") The parent context of the span. See [OpenTelemetry docs](https://opentelemetry.io/docs/concepts/signals/traces/) . #### `parent_id` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.Span.parent_id "Permanent link") The parent span ID of the span. #### `resource` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.Span.resource "Permanent link") The resource of the span. See [OpenTelemetry docs](https://opentelemetry.io/docs/concepts/signals/traces/) . #### `rollout_id` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.Span.rollout_id "Permanent link") The rollout which this span belongs to. #### `sequence_id` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.Span.sequence_id "Permanent link") The ID to make spans ordered within a single attempt. #### `span_id` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.Span.span_id "Permanent link") The span ID of the span. This ID comes from the OpenTelemetry span ID generator. #### `start_time` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.Span.start_time "Permanent link") The start time of the span. See [OpenTelemetry docs](https://opentelemetry.io/docs/concepts/signals/traces/) . #### `status` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.Span.status "Permanent link") The status of the span. See [OpenTelemetry docs](https://opentelemetry.io/docs/concepts/signals/traces/) . #### `trace_id` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.Span.trace_id "Permanent link") The trace ID of the span. One rollout/attempt can have multiple traces. This ID comes from the OpenTelemetry trace ID generator. #### `from_attributes(*, attributes, rollout_id=None, attempt_id=None, sequence_id=None, name=None, trace_id=None, span_id=None, parent_id=None, start_time=None, end_time=None, resource=None, status=None)` `classmethod` [¶](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.Span.from_attributes "Permanent link") Build a synthetic span from raw attributes. Different from the [`from_opentelemetry`](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.Span.from_opentelemetry " from_opentelemetry(src, rollout_id, attempt_id, sequence_id) classmethod ") method, all parameters other than `attributes` are optional and will be generated if not provided. Parameters: * **`attributes`** (`[Attributes](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.Attributes " agentlightning.Attributes = Dict[str, AttributeValue] module-attribute (agentlightning.types.tracer.Attributes)") `) – Span attributes to persist. * **`rollout_id`** (`Optional[str]`, default: `None` ) – Optional rollout identifier associated with the span. * **`attempt_id`** (`Optional[str]`, default: `None` ) – Optional attempt identifier associated with the span. * **`sequence_id`** (`Optional[int]`, default: `None` ) – Optional sequence number to preserve ordering. * **`name`** (`Optional[str]`, default: `None` ) – Optional human-readable span name. * **`trace_id`** (`Optional[str]`, default: `None` ) – Custom trace identifier. When omitted, a random identifier is generated. * **`span_id`** (`Optional[str]`, default: `None` ) – Custom span identifier. When omitted, a random identifier is generated. * **`parent_id`** (`Optional[str]`, default: `None` ) – Optional parent span identifier. * **`start_time`** (`Optional[float]`, default: `None` ) – Span start timestamp in seconds. * **`end_time`** (`Optional[float]`, default: `None` ) – Span end timestamp in seconds. * **`resource`** (`Optional[OtelResource]`, default: `None` ) – Explicit resource information to attach to the span. * **`status`** (`Optional[[TraceStatus](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.TraceStatus " agentlightning.TraceStatus (agentlightning.types.tracer.TraceStatus)") ]`, default: `None` ) – Optional status of the span. Returns: * `'Span'` – [`Span`](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.Span " agentlightning.Span") populated with the provided attributes. #### `from_core_fields(core, *, rollout_id=None, attempt_id=None, sequence_id=None)` `classmethod` [¶](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.Span.from_core_fields "Permanent link") Build a span from a core span. Parameters: * **`core`** (`[SpanCoreFields](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.SpanCoreFields " agentlightning.SpanCoreFields (agentlightning.types.tracer.SpanCoreFields)") `) – Core span to build from. * **`rollout_id`** (`Optional[str]`, default: `None` ) – Optional rollout identifier associated with the span. * **`attempt_id`** (`Optional[str]`, default: `None` ) – Optional attempt identifier associated with the span. * **`sequence_id`** (`Optional[int]`, default: `None` ) – Optional sequence number to preserve ordering. Returns: * `[Span](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.Span " agentlightning.Span (agentlightning.types.tracer.Span)") ` – [`Span`](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.Span " agentlightning.Span") populated with the provided attributes. #### `from_opentelemetry(src, rollout_id, attempt_id, sequence_id)` `classmethod` [¶](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.Span.from_opentelemetry "Permanent link") Convert an OpenTelemetry span into the Agent Lightning data model. Parameters: * **`src`** (`ReadableSpan`) – Span captured by OpenTelemetry. * **`rollout_id`** (`str`) – Identifier for the rollout that produced the span. * **`attempt_id`** (`str`) – Identifier of the attempt within the rollout. * **`sequence_id`** (`int`) – Monotonically increasing identifier assigned to the span. Returns: * `'Span'` – Parsed [`Span`](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.Span " agentlightning.Span") instance suitable for persistence. ### `agentlightning.SpanAttributeNames` [¶](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.SpanAttributeNames "Permanent link") Bases: `str`, `Enum` Canonical attribute names written by Agent Lightning emitters. Deprecated in favor of [semconv](https://microsoft.github.io/agent-lightning/latest/reference/semconv/#agentlightning.semconv " agentlightning.semconv") . #### `MESSAGE = 'message'` `class-attribute` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.SpanAttributeNames.MESSAGE "Permanent link") The name of the message attribute. #### `OBJECT = 'object'` `class-attribute` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.SpanAttributeNames.OBJECT "Permanent link") The name of the object attribute. ### `agentlightning.SpanLike = Union[ReadableSpan, Span]` `module-attribute` [¶](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.SpanLike "Permanent link") Union type of OpenTelemetry `ReadableSpan` and Agent-lightning [`Span`](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.Span " agentlightning.Span") . ### `agentlightning.SpanCoreFields` [¶](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.SpanCoreFields "Permanent link") Bases: `BaseModel` Core fields of a span. Used by span creators who don't care about the full span model. If the spans are managed by some OTel tracer provider, it's not advised to create spans via this path. #### `attributes` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.SpanCoreFields.attributes "Permanent link") The attributes of the span. #### `end_time` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.SpanCoreFields.end_time "Permanent link") The end time of the span. #### `name` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.SpanCoreFields.name "Permanent link") The name of the span. #### `start_time` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.SpanCoreFields.start_time "Permanent link") The start time of the span. #### `status` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.SpanCoreFields.status "Permanent link") The status of the span. ### `agentlightning.SpanRecordingContext` [¶](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.SpanRecordingContext "Permanent link") Bases: `Protocol` Context for recording operations on a span. It doesn't have to finalize the span; the caller will do it. #### `get_recorded_span()` [¶](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.SpanRecordingContext.get_recorded_span "Permanent link") Get the recording of the span. #### `record_attributes(attributes)` [¶](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.SpanRecordingContext.record_attributes "Permanent link") Record attributes on the span. #### `record_exception(exception)` [¶](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.SpanRecordingContext.record_exception "Permanent link") Record an exception on the span. #### `record_status(status_code, description=None)` [¶](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.SpanRecordingContext.record_status "Permanent link") Record the status of the span. Environment Variables[¶](https://microsoft.github.io/agent-lightning/latest/reference/types/#environment-variables "Permanent link") ------------------------------------------------------------------------------------------------------------------------------------- ### `agentlightning.LightningEnvVar` [¶](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.LightningEnvVar "Permanent link") Bases: `Enum` Environment variables for Agent Lightning. #### `AGL_CURRENT_ROLE = 'AGL_CURRENT_ROLE'` `class-attribute` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.LightningEnvVar.AGL_CURRENT_ROLE "Permanent link") Which side(s) to run in this process. Used in [`ClientServerExecutionStrategy`](https://microsoft.github.io/agent-lightning/latest/reference/trainer/#agentlightning.ClientServerExecutionStrategy " agentlightning.ClientServerExecutionStrategy") . #### `AGL_EMITTER_DEBUG = 'AGL_EMITTER_DEBUG'` `class-attribute` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.LightningEnvVar.AGL_EMITTER_DEBUG "Permanent link") Enable debug logging for the emitter. #### `AGL_MANAGED_STORE = 'AGL_MANAGED_STORE'` `class-attribute` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.LightningEnvVar.AGL_MANAGED_STORE "Permanent link") If yes, the [`ExecutionStrategy`](https://microsoft.github.io/agent-lightning/latest/reference/trainer/#agentlightning.ExecutionStrategy " agentlightning.ExecutionStrategy") constructs LightningStore wrappers automatically. When `False` the provided `store` is passed directly to the bundles, allowing callers to manage store wrappers manually. #### `AGL_SERVER_HOST = 'AGL_SERVER_HOST'` `class-attribute` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.LightningEnvVar.AGL_SERVER_HOST "Permanent link") Interface the [`LightningStoreServer`](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.LightningStoreServer " agentlightning.LightningStoreServer") binds to when running the algorithm bundle locally. #### `AGL_SERVER_PORT = 'AGL_SERVER_PORT'` `class-attribute` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.LightningEnvVar.AGL_SERVER_PORT "Permanent link") Port the [`LightningStoreServer`](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.LightningStoreServer " agentlightning.LightningStoreServer") listens to. ### `agentlightning.resolve_bool_env_var(env_var, override=None, fallback=None)` [¶](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.resolve_bool_env_var "Permanent link") `[](https://microsoft.github.io/agent-lightning/latest/reference/types/#__codelineno-0-1) resolve_bool_env_var( [](https://microsoft.github.io/agent-lightning/latest/reference/types/#__codelineno-0-2) env_var: LightningEnvVar, override: bool, fallback: bool [](https://microsoft.github.io/agent-lightning/latest/reference/types/#__codelineno-0-3) ) -> bool` `[](https://microsoft.github.io/agent-lightning/latest/reference/types/#__codelineno-0-1) resolve_bool_env_var( [](https://microsoft.github.io/agent-lightning/latest/reference/types/#__codelineno-0-2) env_var: LightningEnvVar, *, fallback: bool [](https://microsoft.github.io/agent-lightning/latest/reference/types/#__codelineno-0-3) ) -> bool` `[](https://microsoft.github.io/agent-lightning/latest/reference/types/#__codelineno-0-1) resolve_bool_env_var( [](https://microsoft.github.io/agent-lightning/latest/reference/types/#__codelineno-0-2) env_var: LightningEnvVar, [](https://microsoft.github.io/agent-lightning/latest/reference/types/#__codelineno-0-3) override: bool | None = None, [](https://microsoft.github.io/agent-lightning/latest/reference/types/#__codelineno-0-4) fallback: bool | None = None, [](https://microsoft.github.io/agent-lightning/latest/reference/types/#__codelineno-0-5) ) -> bool | None` Resolve a boolean environment variable. Parameters: * **`env_var`** (`[LightningEnvVar](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.LightningEnvVar " agentlightning.LightningEnvVar (agentlightning.env_var.LightningEnvVar)") `) – The environment variable to resolve. * **`override`** (`bool | None`, default: `None` ) – Optional override supplied by the caller. * **`fallback`** (`bool | None`, default: `None` ) – Default value if the environment variable is not set. ### `agentlightning.resolve_int_env_var(env_var, override=None, fallback=None)` [¶](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.resolve_int_env_var "Permanent link") `[](https://microsoft.github.io/agent-lightning/latest/reference/types/#__codelineno-0-1) resolve_int_env_var( [](https://microsoft.github.io/agent-lightning/latest/reference/types/#__codelineno-0-2) env_var: LightningEnvVar, override: int, fallback: int [](https://microsoft.github.io/agent-lightning/latest/reference/types/#__codelineno-0-3) ) -> int` `[](https://microsoft.github.io/agent-lightning/latest/reference/types/#__codelineno-0-1) resolve_int_env_var( [](https://microsoft.github.io/agent-lightning/latest/reference/types/#__codelineno-0-2) env_var: LightningEnvVar, *, fallback: int [](https://microsoft.github.io/agent-lightning/latest/reference/types/#__codelineno-0-3) ) -> int` `[](https://microsoft.github.io/agent-lightning/latest/reference/types/#__codelineno-0-1) resolve_int_env_var( [](https://microsoft.github.io/agent-lightning/latest/reference/types/#__codelineno-0-2) env_var: LightningEnvVar, [](https://microsoft.github.io/agent-lightning/latest/reference/types/#__codelineno-0-3) override: int | None = None, [](https://microsoft.github.io/agent-lightning/latest/reference/types/#__codelineno-0-4) fallback: int | None = None, [](https://microsoft.github.io/agent-lightning/latest/reference/types/#__codelineno-0-5) ) -> int | None` Resolve an integer environment variable. Parameters: * **`env_var`** (`[LightningEnvVar](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.LightningEnvVar " agentlightning.LightningEnvVar (agentlightning.env_var.LightningEnvVar)") `) – The environment variable to resolve. * **`override`** (`int | None`, default: `None` ) – Optional override supplied by the caller. * **`fallback`** (`int | None`, default: `None` ) – Default value if the environment variable is not set. ### `agentlightning.resolve_str_env_var(env_var, override=None, fallback=None)` [¶](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.resolve_str_env_var "Permanent link") `[](https://microsoft.github.io/agent-lightning/latest/reference/types/#__codelineno-0-1) resolve_str_env_var( [](https://microsoft.github.io/agent-lightning/latest/reference/types/#__codelineno-0-2) env_var: LightningEnvVar, override: str, fallback: str [](https://microsoft.github.io/agent-lightning/latest/reference/types/#__codelineno-0-3) ) -> str` `[](https://microsoft.github.io/agent-lightning/latest/reference/types/#__codelineno-0-1) resolve_str_env_var( [](https://microsoft.github.io/agent-lightning/latest/reference/types/#__codelineno-0-2) env_var: LightningEnvVar, *, fallback: str [](https://microsoft.github.io/agent-lightning/latest/reference/types/#__codelineno-0-3) ) -> str` `[](https://microsoft.github.io/agent-lightning/latest/reference/types/#__codelineno-0-1) resolve_str_env_var( [](https://microsoft.github.io/agent-lightning/latest/reference/types/#__codelineno-0-2) env_var: LightningEnvVar, [](https://microsoft.github.io/agent-lightning/latest/reference/types/#__codelineno-0-3) override: str | None = None, [](https://microsoft.github.io/agent-lightning/latest/reference/types/#__codelineno-0-4) fallback: str | None = None, [](https://microsoft.github.io/agent-lightning/latest/reference/types/#__codelineno-0-5) ) -> str | None` Resolve a string environment variable. Parameters: * **`env_var`** (`[LightningEnvVar](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.LightningEnvVar " agentlightning.LightningEnvVar (agentlightning.env_var.LightningEnvVar)") `) – The environment variable to resolve. * **`override`** (`str | None`, default: `None` ) – Optional override supplied by the caller. * **`fallback`** (`str | None`, default: `None` ) – Default value if the environment variable is not set. Back to top --- # Work with Traces - Agent-lightning [Skip to content](https://microsoft.github.io/agent-lightning/latest/tutorials/traces/#working-with-traces) ⚠️ You are viewing development documentation. For stable documentation, please visit [https://microsoft.github.io/agent-lightning/stable/](https://microsoft.github.io/agent-lightning/stable/) Working with Traces[¶](https://microsoft.github.io/agent-lightning/latest/tutorials/traces/#working-with-traces "Permanent link") ================================================================================================================================== Tracing is the secret capability that lets Agent-lightning train almost any agent without rewriting its core logic. The idea was born in observability tooling inside LLMOps workflows and, in Agent-lightning, evolved into a first-class primitive inside the learning loop. Beyond helping you understand what happened inside a rollout, traces provide reward spans and other learning signals that power reinforcement learning and fine-tuning algorithms. ![OpenTelemetry spans](https://microsoft.github.io/agent-lightning/latest/assets/opentelemetry-trace.jpg) Agent-lightning stores every recorded operation as a [`Span`](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.Span " agentlightning.Span") inside a [`LightningStore`](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.LightningStore " agentlightning.LightningStore") . The naming comes from [OpenTelemetry spans](https://opentelemetry.io/docs/concepts/signals/traces/) , shown in the screenshot above. A span can represent an LLM call, a tool invocation, a graph edge, an explicit reward emission, or an arbitrary Python code block. Spans form a tree where parent spans describe higher-level steps and children record the detailed work. The sections below walk through how spans are produced and how to interpret them once they reach the store. Writing Spans[¶](https://microsoft.github.io/agent-lightning/latest/tutorials/traces/#writing-spans "Permanent link") ---------------------------------------------------------------------------------------------------------------------- Most [`Runner`](https://microsoft.github.io/agent-lightning/latest/reference/runner/#agentlightning.Runner " agentlightning.Runner") implementations wire a [`Tracer`](https://microsoft.github.io/agent-lightning/latest/reference/runner/#agentlightning.Tracer " agentlightning.Tracer") into the agent’s lifecycle. The tracer is responsible for installing instrumentation, buffering OpenTelemetry spans, and committing them to the [`LightningStore`](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.LightningStore " agentlightning.LightningStore") . When a runner executes a rollout, it allocates a store-backed tracing context: `[](https://microsoft.github.io/agent-lightning/latest/tutorials/traces/#__codelineno-0-1) async with tracer.trace_context( [](https://microsoft.github.io/agent-lightning/latest/tutorials/traces/#__codelineno-0-2) name="my-rollout", [](https://microsoft.github.io/agent-lightning/latest/tutorials/traces/#__codelineno-0-3) store=store, [](https://microsoft.github.io/agent-lightning/latest/tutorials/traces/#__codelineno-0-4) rollout_id=rollout.rollout_id, [](https://microsoft.github.io/agent-lightning/latest/tutorials/traces/#__codelineno-0-5) attempt_id=attempt.attempt_id, [](https://microsoft.github.io/agent-lightning/latest/tutorials/traces/#__codelineno-0-6) ): [](https://microsoft.github.io/agent-lightning/latest/tutorials/traces/#__codelineno-0-7) await run_agent_logic()` The context manager then requests sequence numbers from the store, converts OpenTelemetry spans into [`Span`](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.Span " agentlightning.Span") objects, and persists them in the middle or at the end of the attempt, depending on the tracer implementation. Agent-lightning ships two tracers out of the box; both rely on [OpenTelemetry Traces](https://opentelemetry.io/docs/concepts/signals/traces/) and ignore metrics or logs. What's instrumentation? In simple terms, _instrumentation_ means adding "patches" or hooks inside your code so you can observe what it’s doing while it runs. Think of it like putting flight recorders in an airplane — instrumentation records key actions, inputs, outputs, and timings without changing how the code behaves. In Agent-lightning tracers, this instrumentation automatically creates spans (small, structured records of work) that show what each part of an agent did, how long it took, and how different steps connect together. ### AgentOps Tracer[¶](https://microsoft.github.io/agent-lightning/latest/tutorials/traces/#agentops-tracer "Permanent link") [`AgentOpsTracer`](https://microsoft.github.io/agent-lightning/latest/reference/runner/#agentlightning.AgentOpsTracer " agentlightning.AgentOpsTracer") will be the default tracer when [`Trainer`](https://microsoft.github.io/agent-lightning/latest/reference/trainer/#agentlightning.Trainer " agentlightning.Trainer") is used but no tracer is explicitly specified. It bootstraps the [AgentOps SDK](https://www.agentops.ai/) locally, installs the supplied instrumentation hooks (LangChain, LangGraph, LiteLLM, FastAPI, and others) provided by the [AgentOps Python SDK](https://github.com/AgentOps-AI/agentops) , and forwards everything through a local OpenTelemetry [`TracerProvider`](https://opentelemetry.io/docs/specs/otel/trace/api/) . [`AgentOpsTracer`](https://microsoft.github.io/agent-lightning/latest/reference/runner/#agentlightning.AgentOpsTracer " agentlightning.AgentOpsTracer") never calls the hosted AgentOps service; instead, it attaches a `LightningSpanProcessor` implemented by the Agent-lightning team so that spans are captured and shipped straight into the store. Because it shares the AgentOps instrumentation surface, any framework supported by AgentOps automatically gains tracing in Agent-lightning. We layer additional hooks on top of AgentOps to capture features that the SDK misses today: 1. Certain providers emit extra metadata — for example, [token IDs returned by vLLM](https://microsoft.github.io/agent-lightning/latest/deep-dive/serving-llm/) — that are not recorded by the stock SDK. We augment those spans with the missing payloads. 2. AgentOps constructs parent-child relationships on a best-effort basis, but mixed instrumentation (for example, OpenAI Agent SDK alongside direct OpenAI Chat Completion calls) can leave segments disconnected. Our implementation (actually implemented in the [`TracerTraceToTriplet`](https://microsoft.github.io/agent-lightning/latest/reference/algorithm/#agentlightning.TracerTraceToTriplet " agentlightning.TracerTraceToTriplet") adapter) repairs those relationships when the hierarchy can be inferred from rollout context. 3. Some versions of downstream frameworks simply do not emit spans for critical events (LangGraph node entrances are a common example). The tracer installs lightweight shims so those spans appear consistently. If a vendor integration behaves unexpectedly, users are encouraged to combine the tracer with [Hooks](https://microsoft.github.io/agent-lightning/latest/tutorials/debug/) to inspect the raw spans or diagnostics, and/or implement a specialized tracer for the framework in question. ### OpenTelemetry Tracer[¶](https://microsoft.github.io/agent-lightning/latest/tutorials/traces/#opentelemetry-tracer "Permanent link") [`OtelTracer`](https://microsoft.github.io/agent-lightning/latest/reference/runner/#agentlightning.OtelTracer " agentlightning.OtelTracer") is a minimal implementation that initializes a vanilla [`TracerProvider`](https://opentelemetry.io/docs/specs/otel/trace/api/) and gives you direct control over span creation through the standard `opentelemetry.trace` API. Use it when you already have explicit instrumentation in your agent, when the AgentOps SDK does not support your framework, or when you want to emit custom spans from business logic. Note [Microsoft Agent Framework](https://github.com/microsoft/agent-framework) is a typical example with built-in OpenTelemetry support. Once you set `OBSERVABILITY_SETTINGS.enable_otel = True`, the framework will automatically emit OpenTelemetry spans, and [`OtelTracer`](https://microsoft.github.io/agent-lightning/latest/reference/runner/#agentlightning.OtelTracer " agentlightning.OtelTracer") will be able to capture them. No extra instrumentation is needed. Inside your agent you can call `opentelemetry.trace.get_trace_provider().get_tracer("my-agent")` and use that tracer to [create spans](https://opentelemetry.io/docs/languages/python/cookbook/) exactly as you would in any OpenTelemetry application. The Lightning span processor attached by [`OtelTracer`](https://microsoft.github.io/agent-lightning/latest/reference/runner/#agentlightning.OtelTracer " agentlightning.OtelTracer") guarantees that every span is sequenced, converted, and written to the store. The same applies for emitted rewards ([`emit_reward`](https://microsoft.github.io/agent-lightning/latest/reference/agent/#agentlightning.emit_reward " agentlightning.emit_reward(reward, *, primary_key=None, attributes=None, propagate=True)") ) and other emitter signals, which are just a special case of manually-created spans. ### Weave Tracer (Experimental)[¶](https://microsoft.github.io/agent-lightning/latest/tutorials/traces/#weave-tracer-experimental "Permanent link") [`WeaveTracer`](https://microsoft.github.io/agent-lightning/latest/reference/runner/#agentlightning.tracer.weave.WeaveTracer " agentlightning.tracer.weave.WeaveTracer") is an experimental tracer that integrates with the [Weave Python SDK](https://docs.wandb.ai/weave) . Use it as a substitute for [`AgentOpsTracer`](https://microsoft.github.io/agent-lightning/latest/reference/runner/#agentlightning.AgentOpsTracer " agentlightning.AgentOpsTracer") when the AgentOps SDK does not fit your environment. The Weave SDK instruments LLM calls and agent libraries directly. Unlike [`AgentOpsTracer`](https://microsoft.github.io/agent-lightning/latest/reference/runner/#agentlightning.AgentOpsTracer " agentlightning.AgentOpsTracer") , Weave does not rely on OpenTelemetry to export spans; it routes everything through a dedicated Weave Trace Server. Agent-lightning implements a custom Weave Trace Server so every call captured by the Weave SDK can be persisted to the [`LightningStore`](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.LightningStore " agentlightning.LightningStore") . Warning [`WeaveTracer`](https://microsoft.github.io/agent-lightning/latest/reference/runner/#agentlightning.tracer.weave.WeaveTracer " agentlightning.tracer.weave.WeaveTracer") remains experimental and has not been tested as thoroughly as [`AgentOpsTracer`](https://microsoft.github.io/agent-lightning/latest/reference/runner/#agentlightning.AgentOpsTracer " agentlightning.AgentOpsTracer") . It may conflict with libraries that ship OpenTelemetry instrumentation by default (for example, LiteLLM-based LLM proxies). Use the tracer with caution and report any issues to the Agent-lightning team. ### LLM Proxy[¶](https://microsoft.github.io/agent-lightning/latest/tutorials/traces/#llm-proxy "Permanent link") Sometimes the runner can’t observe the agent directly — because it’s in another language or running remotely. [`LLMProxy`](https://microsoft.github.io/agent-lightning/latest/reference/algorithm/#agentlightning.LLMProxy " agentlightning.LLMProxy") bridges that gap by instrumenting the server side of LLM calls. It wraps [LiteLLM](https://docs.litellm.ai/) and adds middleware that accepts prefixed routes like `/rollout/{rid}/attempt/{aid}/v1/chat/completions`. Before forwarding, the middleware rewrites the path to `/v1/chat/completions`, fetches a monotonic `sequence_id` from the `LightningStore`, injects `x-rollout-id`, `x-attempt-id`, and `x-sequence-id` into the request headers, and then forwards the request to the backend LLM endpoint. LiteLLM produces OpenTelemetry spans for the request/response. A custom `LightningSpanExporter` reads the rollout/attempt/sequence identifiers from the recorded request headers and persists each span to the store. Because the `sequence_id` is allocated at the start of the request, traces stay in strict order even across machines with skewed clocks or asynchronous responses. sequenceDiagram participant Agent participant Proxy as LLM Proxy participant Backend as LLM Backend participant Store as LightningStore Agent->>Proxy: POST /rollout/{rid}/attempt/{aid}/v1/chat/completions Proxy->>Store: get_next_span_sequence_id(rid, aid) Store-->>Proxy: sequence_id Proxy->>Backend: Forward /v1/chat/completions
(headers: rid, aid, sid) Backend-->>Proxy: Response (tokens, usage, token_ids) Proxy->>Store: Export OTEL spans (rid, aid, sequence_id) Proxy-->>Agent: OpenAI-compatible response [`LLMProxy`](https://microsoft.github.io/agent-lightning/latest/reference/algorithm/#agentlightning.LLMProxy " agentlightning.LLMProxy") actually provides more functionalities than just the middleware for tracing. Read [Serving LLM](https://microsoft.github.io/agent-lightning/latest/deep-dive/serving-llm/) for more details. [](https://microsoft.github.io/agent-lightning/latest/tutorials/traces/) Distributed Tracing Agent-lightning enforces deterministic span ordering by assigning a monotonic [`sequence_id`](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.Span.sequence_id " sequence_id instance-attribute ") to every span within an attempt. Before calling [`LightningStore.add_span`](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.LightningStore.add_span " add_span(span) async ") or [`LightningStore.add_otel_span`](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.LightningStore.add_otel_span " add_otel_span(rollout_id, attempt_id, readable_span, sequence_id=None) async ") , tracers are expected to call [`LightningStore.get_next_span_sequence_id`](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.LightningStore.get_next_span_sequence_id " get_next_span_sequence_id(rollout_id, attempt_id) async ") to get the next sequence id. This removes clock skew and merges spans produced on different machines or threads. If you implement a custom tracer or exporter, make sure you do this (or respect the one provided in headers by components such as [`LLMProxy`](https://microsoft.github.io/agent-lightning/latest/reference/algorithm/#agentlightning.LLMProxy " agentlightning.LLMProxy") ); otherwise, adapters will struggle to properly reconstruct the execution tree. ### Custom Tracer[¶](https://microsoft.github.io/agent-lightning/latest/tutorials/traces/#custom-tracer "Permanent link") If none of the built-in tracers fit your environment, the first option to consider is to [return the spans](https://microsoft.github.io/agent-lightning/latest/tutorials/write-agents/) directly from your agent implementation. If that's not possible, or you want to support multiple agents in a unified effort, you can implement your own tracer by subclassing [`Tracer`](https://microsoft.github.io/agent-lightning/latest/reference/runner/#agentlightning.Tracer " agentlightning.Tracer") . Custom tracers must implement at least [`trace_context`](https://microsoft.github.io/agent-lightning/latest/reference/runner/#agentlightning.Tracer.trace_context " trace_context(name=None, *, store=None, rollout_id=None, attempt_id=None)") . The [`trace_context`](https://microsoft.github.io/agent-lightning/latest/reference/runner/#agentlightning.Tracer.trace_context " trace_context(name=None, *, store=None, rollout_id=None, attempt_id=None)") coroutine should install or activate whatever instrumentation you need, then yield a span processor that ultimately adds spans to the store. You can reuse the `LightningSpanProcessor` if you produce OpenTelemetry `ReadableSpan` objects, or call [`LightningStore.add_span`](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.LightningStore.add_span " add_span(span) async ") directly if you generate [`Span`](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.Span " agentlightning.Span") instances yourself. Advanced tracers often run auxiliary services (for example, starting a telemetry daemon or attaching to a container runtime) inside `init_worker` and tear them down in `teardown_worker`. The [`ParallelWorkerBase`](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.ParallelWorkerBase " agentlightning.ParallelWorkerBase") lifecycle that `Tracer` inherits from ensures those hooks are executed in every runner subprocess. Reading Traces[¶](https://microsoft.github.io/agent-lightning/latest/tutorials/traces/#reading-traces "Permanent link") ------------------------------------------------------------------------------------------------------------------------ Generally, there are two approaches to reading traces. When you only need a quick look, [`Tracer.get_last_trace`](https://microsoft.github.io/agent-lightning/latest/reference/runner/#agentlightning.Tracer.get_last_trace " get_last_trace()") returns the raw OpenTelemetry spans captured most recently. For historical analysis, use the [`LightningStore.query_spans`](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.LightningStore.query_spans " query_spans(rollout_id, attempt_id=None, *, trace_id=None, trace_id_contains=None, span_id=None, span_id_contains=None, parent_id=None, parent_id_contains=None, name=None, name_contains=None, filter_logic='and', limit=-1, offset=0, sort_by='sequence_id', sort_order='asc') async ") API, which yields normalized [`Span`](https://microsoft.github.io/agent-lightning/latest/reference/types/#agentlightning.Span " agentlightning.Span") objects keyed by rollout ID and attempt ID. Combine those queries with [`LightningStore.query_rollouts`](https://microsoft.github.io/agent-lightning/latest/reference/store/#agentlightning.LightningStore.query_rollouts " query_rollouts(*, status_in=None, rollout_id_in=None, rollout_id_contains=None, filter_logic='and', sort_by=None, sort_order='asc', limit=-1, offset=0, status=None, rollout_ids=None) async ") to align spans with rollout status, retries, and timing information. Spans arrive asynchronously, originate from different processes, and form hierarchies rather than simple lists. The attributes of each span are tedious and unfriendly to human readers. This combination makes raw traces time-consuming to inspect, especially when you only care about specific signals such as rewards, LLM prompts, responses, or tool outputs. Understanding how the store exposes traces and how adapters reshape them will save hours when debugging or training. Why traces can be difficult to read? The trace tree for a single rollout typically mixes multiple abstraction layers: a planner span may contain several LLM spans, each of which contains tool execution spans that can themselves trigger nested agent invocations. There are also instrumentations at different levels. For example, when a request delegates to another library (e.g., from LangChain to OpenAI), two libraries might emit spans for the same request. At the top level, there could be concurrently running agents that may flush spans slightly out of order. Sorting by `sequence_id` restores the chronological view, but interpreting the tree requires additional context about parent-child relationships and rollout metadata. ### Adapter[¶](https://microsoft.github.io/agent-lightning/latest/tutorials/traces/#adapter "Permanent link") [Adapters](https://microsoft.github.io/agent-lightning/latest/reference/algorithm/#agentlightning.Adapter " agentlightning.Adapter") transform lists of spans into higher-level data structures that training algorithms can consume directly. Agent-lightning provides several adapters out of the box: * [`TracerTraceToTriplet`](https://microsoft.github.io/agent-lightning/latest/reference/algorithm/#agentlightning.TracerTraceToTriplet " agentlightning.TracerTraceToTriplet") converts spans into `(prompt, response, reward)` triplets, which power reinforcement-learning algorithms such as [VERL](https://microsoft.github.io/agent-lightning/latest/algorithm-zoo/verl/) and connect trace data to gradient updates. * [`TraceToMessages`](https://microsoft.github.io/agent-lightning/latest/reference/algorithm/#agentlightning.TraceToMessages " agentlightning.TraceToMessages") rewrites spans into OpenAI chat message JSON suitable for supervised fine-tuning or evaluation harnesses. * [`LlmProxyTraceToTriplet`](https://microsoft.github.io/agent-lightning/latest/reference/algorithm/#agentlightning.LlmProxyTraceToTriplet " agentlightning.LlmProxyTraceToTriplet") mirrors [`TracerTraceToTriplet`](https://microsoft.github.io/agent-lightning/latest/reference/algorithm/#agentlightning.TracerTraceToTriplet " agentlightning.TracerTraceToTriplet") but understands spans emitted by [LLMProxy](https://microsoft.github.io/agent-lightning/latest/reference/algorithm/#agentlightning.LLMProxy " agentlightning.LLMProxy") . It is experimental and might be merged with [`TracerTraceToTriplet`](https://microsoft.github.io/agent-lightning/latest/reference/algorithm/#agentlightning.TracerTraceToTriplet " agentlightning.TracerTraceToTriplet") in the future. Adapters are regular Python callable instances, so you can plug them into [`Trainer`](https://microsoft.github.io/agent-lightning/latest/reference/trainer/#agentlightning.Trainer " agentlightning.Trainer") via the `adapter` argument, or call them manually during exploration. When used in [`Trainer`](https://microsoft.github.io/agent-lightning/latest/reference/trainer/#agentlightning.Trainer " agentlightning.Trainer") , adapters are bundled into the [`Algorithm`](https://microsoft.github.io/agent-lightning/latest/reference/algorithm/#agentlightning.Algorithm " agentlightning.Algorithm") before the algorithm runs, through the [`Algorithm.set_adapter`](https://microsoft.github.io/agent-lightning/latest/reference/algorithm/#agentlightning.Algorithm.set_adapter " set_adapter(adapter)") method. You can also customize an [`Adapter`](https://microsoft.github.io/agent-lightning/latest/reference/algorithm/#agentlightning.Adapter " agentlightning.Adapter") by extending the implementations above or subclassing the base class. If you need a bespoke format, subclass [`TraceAdapter`](https://microsoft.github.io/agent-lightning/latest/reference/algorithm/#agentlightning.TraceAdapter " agentlightning.TraceAdapter") (for store spans) or [`OtelTraceAdapter`](https://microsoft.github.io/agent-lightning/latest/reference/algorithm/#agentlightning.OtelTraceAdapter " agentlightning.OtelTraceAdapter") (for raw OpenTelemetry spans) and implement `adapt` (these two classes can usually share the same implementation). ### Reading Rewards[¶](https://microsoft.github.io/agent-lightning/latest/tutorials/traces/#reading-rewards "Permanent link") Rewards are recorded as dedicated spans named [`agentlightning.annotation`](https://microsoft.github.io/agent-lightning/latest/reference/semconv/#agentlightning.semconv.AGL_ANNOTATION " AGL_ANNOTATION = 'agentlightning.annotation' module-attribute ") . Emitting a reward through [`emit_reward`](https://microsoft.github.io/agent-lightning/latest/reference/agent/#agentlightning.emit_reward " agentlightning.emit_reward(reward, *, primary_key=None, attributes=None, propagate=True)") or [`emit_annotation`](https://microsoft.github.io/agent-lightning/latest/reference/agent/#agentlightning.emit_annotation " agentlightning.emit_annotation(annotation, propagate=True)") ensures the value is stored in the span’s `attributes`. To audit rewards, fetch spans from the store and use the helper utilities in [`agentlightning.emitter`](https://microsoft.github.io/agent-lightning/latest/reference/agent/) : `[](https://microsoft.github.io/agent-lightning/latest/tutorials/traces/#__codelineno-1-1) from agentlightning.emitter import find_final_reward [](https://microsoft.github.io/agent-lightning/latest/tutorials/traces/#__codelineno-1-2) [](https://microsoft.github.io/agent-lightning/latest/tutorials/traces/#__codelineno-1-3) spans = await store.query_spans(rollout_id) [](https://microsoft.github.io/agent-lightning/latest/tutorials/traces/#__codelineno-1-4) reward = find_final_reward(spans) [](https://microsoft.github.io/agent-lightning/latest/tutorials/traces/#__codelineno-1-5) print(f"Final reward: {reward}")` [`find_reward_spans`](https://microsoft.github.io/agent-lightning/latest/reference/agent/#agentlightning.find_reward_spans " agentlightning.find_reward_spans(spans)") returns every reward span so you can visualize intermediate shaping signals, while [`find_final_reward`](https://microsoft.github.io/agent-lightning/latest/reference/agent/#agentlightning.find_final_reward " agentlightning.find_final_reward(spans)") extracts the last non-null reward per attempt. While these helpers are convenient, they may not help you fully understand the chronological or hierarchical relationships between reward spans and other spans. Using an [`Adapter`](https://microsoft.github.io/agent-lightning/latest/reference/algorithm/#agentlightning.Adapter " agentlightning.Adapter") — especially the same one used in the algorithm you’re working with — remains the recommended way to inspect your generated spans. Back to top --- # Agent Lightning [Skip to content](https://microsoft.github.io/agent-lightning/0.1.2/#agent-lightning) Agent Lightning[¶](https://microsoft.github.io/agent-lightning/0.1.2/#agent-lightning "Permanent link") ======================================================================================================== Agent Lightning is the absolute trainer to light up AI agents. Features[¶](https://microsoft.github.io/agent-lightning/0.1.2/#features "Permanent link") ------------------------------------------------------------------------------------------ * Turn your agent into an optimizable beast with **ZERO CODE CHANGE** (almost)! 💤 * Build with **ANY** agent framework (LangChain, OpenAI Agent SDK, AutoGen, CrewAI, ...); or even WITHOUT agent framework (Python OpenAI). You name it! 🤖 * **Selectively** optimize one or more agents in a multi-agent system. 🎯 * Embraces Reinforcement Learning, Automatic Prompt Optimization and more **algorithms**. 🤗 Quick Links[¶](https://microsoft.github.io/agent-lightning/0.1.2/#quick-links "Permanent link") ------------------------------------------------------------------------------------------------ * [Installation](https://microsoft.github.io/agent-lightning/0.1.2/quickstart/installation/) - Get started with Agent Lightning * [Quickstart](https://microsoft.github.io/agent-lightning/0.1.2/quickstart/getting-started/) - Learn the fundamentals of Agent Lightning * [Train SQL Agent with RL](https://microsoft.github.io/agent-lightning/0.1.2/how-to/train-sql-agent/) - A practical example of training a SQL agent * [API Reference](https://microsoft.github.io/agent-lightning/0.1.2/reference/core/) - Complete API documentation Resources[¶](https://microsoft.github.io/agent-lightning/0.1.2/#resources "Permanent link") -------------------------------------------------------------------------------------------- * 8/11/2025 [Training AI Agents to Write and Self-correct SQL with Reinforcement Learning](https://medium.com/@yugez/training-ai-agents-to-write-and-self-correct-sql-with-reinforcement-learning-571ed31281ad) Medium. * 8/5/2025 [Agent Lightning: Train ANY AI Agents with Reinforcement Learning](https://arxiv.org/abs/2508.03680) arXiv paper. * 7/26/2025 [We discovered an approach to train any AI agent with RL, with (almost) zero code changes.](https://www.reddit.com/r/LocalLLaMA/comments/1m9m670/we_discovered_an_approach_to_train_any_ai_agent/) Reddit. * 6/6/2025 [Agent Lightning - Microsoft Research](https://www.microsoft.com/en-us/research/project/agent-lightning/) Project page. Citation[¶](https://microsoft.github.io/agent-lightning/0.1.2/#citation "Permanent link") ------------------------------------------------------------------------------------------ If you find Agent Lightning useful in your research or projects, please cite our paper: `[](https://microsoft.github.io/agent-lightning/0.1.2/#__codelineno-0-1) @misc{luo2025agentlightningtrainai, [](https://microsoft.github.io/agent-lightning/0.1.2/#__codelineno-0-2) title={Agent Lightning: Train ANY AI Agents with Reinforcement Learning}, [](https://microsoft.github.io/agent-lightning/0.1.2/#__codelineno-0-3) author={Xufang Luo and Yuge Zhang and Zhiyuan He and Zilong Wang and Siyun Zhao and Dongsheng Li and Luna K. Qiu and Yuqing Yang}, [](https://microsoft.github.io/agent-lightning/0.1.2/#__codelineno-0-4) year={2025}, [](https://microsoft.github.io/agent-lightning/0.1.2/#__codelineno-0-5) eprint={2508.03680}, [](https://microsoft.github.io/agent-lightning/0.1.2/#__codelineno-0-6) archivePrefix={arXiv}, [](https://microsoft.github.io/agent-lightning/0.1.2/#__codelineno-0-7) primaryClass={cs.AI}, [](https://microsoft.github.io/agent-lightning/0.1.2/#__codelineno-0-8) url={https://arxiv.org/abs/2508.03680}, [](https://microsoft.github.io/agent-lightning/0.1.2/#__codelineno-0-9) }` License[¶](https://microsoft.github.io/agent-lightning/0.1.2/#license "Permanent link") ---------------------------------------------------------------------------------------- See the [LICENSE](https://github.com/microsoft/agent-lightning/blob/main/LICENSE) file for details. Back to top --- # Installation - Agent-lightning [Skip to content](https://microsoft.github.io/agent-lightning/stable/tutorials/installation/#installation-guide) ⚠️ You are viewing development documentation. For stable documentation, please visit [https://microsoft.github.io/agent-lightning/stable/](https://microsoft.github.io/agent-lightning/stable/) Installation Guide[¶](https://microsoft.github.io/agent-lightning/stable/tutorials/installation/#installation-guide "Permanent link") ====================================================================================================================================== This guide explains how to install **Agent-Lightning**. You can install it from **PyPI** (the Python Package Index) for general use or directly from the **source code** if you plan to contribute or need fine-grained control over dependencies. Platform and Hardware Requirements Agent-Lightning is officially supported on **Linux distributions** (Ubuntu 22.04 or later is recommended). At the moment **macOS and Windows** (outside of WSL2) are not supported. The Python runtime must be **Python 3.10 or newer**. We recommend using the latest patch release of Python 3.10, 3.11, or 3.12 to pick up performance and security updates. A **GPU is optional**—you only need CUDA-capable hardware if you plan to fine-tune model weights or run GPU-accelerated workloads. CPU-only environments are fully supported for evaluation and inference. Installing from PyPI[¶](https://microsoft.github.io/agent-lightning/stable/tutorials/installation/#installing-from-pypi "Permanent link") ------------------------------------------------------------------------------------------------------------------------------------------ The easiest way to get started is by installing Agent-Lightning directly from PyPI. This ensures you get the latest **stable release** of the package, tested for compatibility and reliability. ### Install the Stable Release[¶](https://microsoft.github.io/agent-lightning/stable/tutorials/installation/#install-the-stable-release "Permanent link") Run the following command in your terminal: `[](https://microsoft.github.io/agent-lightning/stable/tutorials/installation/#__codelineno-0-1) pip install --upgrade agentlightning` This installs or upgrades Agent-Lightning to the newest stable version. Tip If you intend to use **Agent-Lightning** with [**VERL**](https://microsoft.github.io/agent-lightning/stable/algorithm-zoo/verl/) or run any of its **example scripts**, you’ll need to install some additional dependencies. See the sections on [Algorithm-specific installation](https://microsoft.github.io/agent-lightning/stable/tutorials/installation/#algorithm-specific-installation) and [Example-specific installation](https://microsoft.github.io/agent-lightning/stable/tutorials/installation/#example-specific-installation) for details. ### Install the Nightly Build (Latest Features)[¶](https://microsoft.github.io/agent-lightning/stable/tutorials/installation/#install-the-nightly-build-latest-features "Permanent link") Agent-Lightning also publishes **nightly builds**, which contain the latest experimental features and improvements from the main branch. These are available via **Test PyPI**. `[](https://microsoft.github.io/agent-lightning/stable/tutorials/installation/#__codelineno-1-1) pip install --upgrade --index-url https://test.pypi.org/simple/ --extra-index-url https://pypi.org/simple/ --pre agentlightning` Warning The nightly builds are cutting-edge but may include unstable or untested changes. Use them **at your own risk**, especially in production environments. Algorithm-specific Installation[¶](https://microsoft.github.io/agent-lightning/stable/tutorials/installation/#algorithm-specific-installation "Permanent link") ---------------------------------------------------------------------------------------------------------------------------------------------------------------- Agent-Lightning supports multiple learning algorithms. Some of them like [APO](https://microsoft.github.io/agent-lightning/stable/algorithm-zoo/apo/) or [VERL](https://microsoft.github.io/agent-lightning/stable/algorithm-zoo/verl/) require extra dependencies. You can install them automatically using **optional extras** or manually if you prefer finer control. ### Installing APO[¶](https://microsoft.github.io/agent-lightning/stable/tutorials/installation/#installing-apo "Permanent link") [APO](https://microsoft.github.io/agent-lightning/stable/algorithm-zoo/apo/) is an algorithm module that depends on libraries such as [POML](https://github.com/microsoft/POML) . You can install Agent-Lightning with APO support by running: `[](https://microsoft.github.io/agent-lightning/stable/tutorials/installation/#__codelineno-2-1) pip install agentlightning[apo]` Warning APO also depends on the [OpenAI Python SDK](https://github.com/openai/openai-python) , version **2.0 or newer**. Ensure your SDK version is up to date to avoid compatibility issues. ### Installing VERL[¶](https://microsoft.github.io/agent-lightning/stable/tutorials/installation/#installing-verl "Permanent link") [VERL](https://microsoft.github.io/agent-lightning/stable/algorithm-zoo/verl/) integrates with libraries like **PyTorch**, **vLLM**, and **VERL framework**. Although you _can_ install all dependencies automatically, we recommend doing it manually to avoid version conflicts. `[](https://microsoft.github.io/agent-lightning/stable/tutorials/installation/#__codelineno-3-1) pip install agentlightning[verl]` Recommended Manual Setup (More Stable) Automated installation may cause issues if you don’t have a compatible **PyTorch** or **CUDA** version preinstalled. For a more stable setup, install dependencies step-by-step: `[](https://microsoft.github.io/agent-lightning/stable/tutorials/installation/#__codelineno-4-1) pip install torch==2.8.0 torchvision==0.23.0 --index-url https://download.pytorch.org/whl/cu128 [](https://microsoft.github.io/agent-lightning/stable/tutorials/installation/#__codelineno-4-2) pip install flash-attn --no-build-isolation [](https://microsoft.github.io/agent-lightning/stable/tutorials/installation/#__codelineno-4-3) pip install vllm==0.10.2 [](https://microsoft.github.io/agent-lightning/stable/tutorials/installation/#__codelineno-4-4) pip install verl==0.5.0` This approach ensures compatibility with CUDA 12.8 and minimizes dependency conflicts. Example-specific Installation[¶](https://microsoft.github.io/agent-lightning/stable/tutorials/installation/#example-specific-installation "Permanent link") ------------------------------------------------------------------------------------------------------------------------------------------------------------ Each example in the `examples/` directory may have its own additional dependencies. Please refer to the **README** file of each example for detailed setup instructions: [See Example READMEs](https://github.com/microsoft/agent-lightning/tree/3b5d733861cf313fc09821a23240bbdf3cb2ee5b/examples) . Installing from Source (for Developers and Contributors)[¶](https://microsoft.github.io/agent-lightning/stable/tutorials/installation/#installing-from-source-for-developers-and-contributors "Permanent link") ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- If you plan to contribute to Agent-Lightning or prefer to work with the latest development code, install it directly from the **source repository**. ### Why Install from Source?[¶](https://microsoft.github.io/agent-lightning/stable/tutorials/installation/#why-install-from-source "Permanent link") * You want to **modify or contribute** to the project. * You prefer an **isolated development environment**. * You want to test unreleased features or fix bugs locally. ### Using `uv` for Dependency Management[¶](https://microsoft.github.io/agent-lightning/stable/tutorials/installation/#using-uv-for-dependency-management "Permanent link") Starting with version **0.2**, Agent-Lightning uses [`uv`](https://docs.astral.sh/uv/) as its **default dependency manager**. `uv` is a fast and safe alternative to `pip` that: * Installs packages **in seconds** (instead of minutes), * Prevents **dependency conflicts**, * Supports **grouped dependencies** for optional features. Before proceeding, make sure `uv` is installed. ### Minimal Developer Installation[¶](https://microsoft.github.io/agent-lightning/stable/tutorials/installation/#minimal-developer-installation "Permanent link") `[](https://microsoft.github.io/agent-lightning/stable/tutorials/installation/#__codelineno-5-1) git clone https://github.com/microsoft/agent-lightning [](https://microsoft.github.io/agent-lightning/stable/tutorials/installation/#__codelineno-5-2) cd agent-lightning [](https://microsoft.github.io/agent-lightning/stable/tutorials/installation/#__codelineno-5-3) uv sync --group dev` This command sets up a clean development environment with only the essential dependencies. ### Installing All Extras (CPU or GPU)[¶](https://microsoft.github.io/agent-lightning/stable/tutorials/installation/#installing-all-extras-cpu-or-gpu "Permanent link") `uv sync` can also handle algorithm-specific and example-specific dependencies in one step. For a CPU-only machine: `[](https://microsoft.github.io/agent-lightning/stable/tutorials/installation/#__codelineno-6-1) uv sync --frozen \ [](https://microsoft.github.io/agent-lightning/stable/tutorials/installation/#__codelineno-6-2) --extra apo \ [](https://microsoft.github.io/agent-lightning/stable/tutorials/installation/#__codelineno-6-3) --extra verl \ [](https://microsoft.github.io/agent-lightning/stable/tutorials/installation/#__codelineno-6-4) --group dev \ [](https://microsoft.github.io/agent-lightning/stable/tutorials/installation/#__codelineno-6-5) --group torch-cpu \ [](https://microsoft.github.io/agent-lightning/stable/tutorials/installation/#__codelineno-6-6) --group torch-stable \ [](https://microsoft.github.io/agent-lightning/stable/tutorials/installation/#__codelineno-6-7) --group trl \ [](https://microsoft.github.io/agent-lightning/stable/tutorials/installation/#__codelineno-6-8) --group agents \ [](https://microsoft.github.io/agent-lightning/stable/tutorials/installation/#__codelineno-6-9) --no-default-groups` For a GPU-equipped machine that is CUDA 12.8 compatible: `[](https://microsoft.github.io/agent-lightning/stable/tutorials/installation/#__codelineno-7-1) uv sync --frozen \ [](https://microsoft.github.io/agent-lightning/stable/tutorials/installation/#__codelineno-7-2) --extra apo \ [](https://microsoft.github.io/agent-lightning/stable/tutorials/installation/#__codelineno-7-3) --extra verl \ [](https://microsoft.github.io/agent-lightning/stable/tutorials/installation/#__codelineno-7-4) --group dev \ [](https://microsoft.github.io/agent-lightning/stable/tutorials/installation/#__codelineno-7-5) --group torch-gpu-stable \ [](https://microsoft.github.io/agent-lightning/stable/tutorials/installation/#__codelineno-7-6) --group trl \ [](https://microsoft.github.io/agent-lightning/stable/tutorials/installation/#__codelineno-7-7) --group agents \ [](https://microsoft.github.io/agent-lightning/stable/tutorials/installation/#__codelineno-7-8) --no-default-groups` Read more about Agent-lightning managed dependency groups [here](https://github.com/microsoft/agent-lightning/tree/3b5d733861cf313fc09821a23240bbdf3cb2ee5b/pyproject.toml) . ### Building the Dashboard[¶](https://microsoft.github.io/agent-lightning/stable/tutorials/installation/#building-the-dashboard "Permanent link") The Agent-Lightning dashboard is built using [Vite](https://vite.dev/) . To build the dashboard, run the following command: `[](https://microsoft.github.io/agent-lightning/stable/tutorials/installation/#__codelineno-8-1) cd dashboard [](https://microsoft.github.io/agent-lightning/stable/tutorials/installation/#__codelineno-8-2) npm ci [](https://microsoft.github.io/agent-lightning/stable/tutorials/installation/#__codelineno-8-3) npm run build` Some HTML and JavaScript assets will be generated in the `agentlightning/dashboard` directory. ### Activating Your Environment[¶](https://microsoft.github.io/agent-lightning/stable/tutorials/installation/#activating-your-environment "Permanent link") After syncing dependencies, `uv` automatically creates a virtual environment inside the `.venv/` directory. You can use it in two ways: `[](https://microsoft.github.io/agent-lightning/stable/tutorials/installation/#__codelineno-9-1) # Option 1: Prefix commands with uv run [](https://microsoft.github.io/agent-lightning/stable/tutorials/installation/#__codelineno-9-2) uv run python your_script.py [](https://microsoft.github.io/agent-lightning/stable/tutorials/installation/#__codelineno-9-3) [](https://microsoft.github.io/agent-lightning/stable/tutorials/installation/#__codelineno-9-4) # Option 2: Activate the virtual environment [](https://microsoft.github.io/agent-lightning/stable/tutorials/installation/#__codelineno-9-5) source .venv/bin/activate [](https://microsoft.github.io/agent-lightning/stable/tutorials/installation/#__codelineno-9-6) python your_script.py` Before Contributing Agent-Lightning enforces code style and linting rules via **pre-commit hooks**. Installing them early prevents many avoidable formatting issues. `[](https://microsoft.github.io/agent-lightning/stable/tutorials/installation/#__codelineno-10-1) uv run pre-commit install [](https://microsoft.github.io/agent-lightning/stable/tutorials/installation/#__codelineno-10-2) uv run pre-commit run --all-files --show-diff-on-failure --color=always` Back to top --- # Examples Catalog - Agent-lightning [Skip to content](https://microsoft.github.io/agent-lightning/stable/how-to/examples-catalog/#examples-catalog) ⚠️ You are viewing development documentation. For stable documentation, please visit [https://microsoft.github.io/agent-lightning/stable/](https://microsoft.github.io/agent-lightning/stable/) Examples Catalog[¶](https://microsoft.github.io/agent-lightning/stable/how-to/examples-catalog/#examples-catalog "Permanent link") =================================================================================================================================== Want to Contribute? We welcome contributions to the examples catalog! Please refer to the [Contributing](https://microsoft.github.io/agent-lightning/stable/community/contributing/) guide for more details. * **APO room selector** * * * Prompt-optimize a room-booking agent with the built-in APO algorithm, then contrast it with the write-your-own algorithm and debugging workflows in the tutorials. Pairs well with the [Train the First Agent how-to](https://github.com/microsoft/agent-lightning/tree/3b5d733861cf313fc09821a23240bbdf3cb2ee5b/docs/how-to/train-first-agent.md) and the [Write the First Algorithm guide](https://github.com/microsoft/agent-lightning/tree/3b5d733861cf313fc09821a23240bbdf3cb2ee5b/docs/how-to/write-first-algorithm.md) . [Browse source](https://github.com/microsoft/agent-lightning/tree/3b5d733861cf313fc09821a23240bbdf3cb2ee5b/examples/apo) * **Azure OpenAI SFT** * * * Run a supervised fine-tuning loop against Azure OpenAI: roll out the capital-lookup agent, turn traces into JSONL, launch fine-tunes, and redeploy the resulting checkpoints through Azure CLI. [Browse source](https://github.com/microsoft/agent-lightning/tree/3b5d733861cf313fc09821a23240bbdf3cb2ee5b/examples/azure) * **Calc-X VERL math** * * * VERL-based reinforcement learning setup for a math-reasoning agent that uses AutoGen plus an MCP calculator tool to solve Calc-X problems end to end. [Browse source](https://github.com/microsoft/agent-lightning/tree/3b5d733861cf313fc09821a23240bbdf3cb2ee5b/examples/calc_x) * **ChartQA vision-language RL** * * * LangGraph-powered workflow for answering chart questions end to end: rollout the multi-modality agent with GPT or vLLM, and train with VERL/GRPO plus self-refinement loops. [Browse source](https://github.com/microsoft/agent-lightning/tree/3b5d733861cf313fc09821a23240bbdf3cb2ee5b/examples/chartqa) * **Claude Code SWE-bench** * * * Instrumented driver that runs Anthropic's Claude Code workflow on SWE-bench instances while streaming traces through Agent-lightning—supports hosted vLLM, official Anthropic, or any OpenAI-compatible backend and emits datasets for downstream tuning. [Browse source](https://github.com/microsoft/agent-lightning/tree/3b5d733861cf313fc09821a23240bbdf3cb2ee5b/examples/claude_code) * **Minimal building blocks** * * * Bite-sized scripts that isolate Agent-lightning primitives (e.g., LightningStore usage, LLM proxying, minimal vLLM host) so you can study each part before composing larger workflows. [Browse source](https://github.com/microsoft/agent-lightning/tree/3b5d733861cf313fc09821a23240bbdf3cb2ee5b/examples/minimal) * **RAG (MuSiQue)** * * * Retrieval-Augmented Generation pipeline that preps a Wikipedia retriever via MCP and trains a MuSiQue QA agent with GRPO. Documented for historical reference (verified on Agent-lightning v0.1.x). [Browse source](https://github.com/microsoft/agent-lightning/tree/3b5d733861cf313fc09821a23240bbdf3cb2ee5b/examples/rag) * **Spider SQL agent** * * * LangGraph-powered text-to-SQL workflow for the Spider benchmark, combining LangChain tooling with Agent-lightning rollouts; follow along with the [how-to for training SQL agents](https://github.com/microsoft/agent-lightning/tree/3b5d733861cf313fc09821a23240bbdf3cb2ee5b/docs/how-to/train-sql-agent.md) . [Browse source](https://github.com/microsoft/agent-lightning/tree/3b5d733861cf313fc09821a23240bbdf3cb2ee5b/examples/spider) * **Tinker integration** * * * Adapter package ([`agl_tinker`](https://github.com/microsoft/agent-lightning/tree/3b5d733861cf313fc09821a23240bbdf3cb2ee5b/examples/tinker/agl_tinker) ) with Tinker plus sample CrewAI/OpenAI agents that feed Agent-lightning traces into Tinker’s reinforcement-learning backend for both toy and 20-Questions-style workflows. [Browse source](https://github.com/microsoft/agent-lightning/tree/3b5d733861cf313fc09821a23240bbdf3cb2ee5b/examples/tinker) * **Unsloth SFT** * * * Supervised fine-tuning loop that ranks math-agent rollouts, fine-tunes with Unsloth’s 4-bit LoRA stack, and mirrors the [Fine-tune with Unsloth recipe](https://github.com/microsoft/agent-lightning/tree/3b5d733861cf313fc09821a23240bbdf3cb2ee5b/docs/how-to/unsloth-sft.md) . [Browse source](https://github.com/microsoft/agent-lightning/tree/3b5d733861cf313fc09821a23240bbdf3cb2ee5b/examples/unsloth) Back to top --- # Train the First Agent - Agent-lightning [Skip to content](https://microsoft.github.io/agent-lightning/stable/how-to/train-first-agent/#train-the-first-agent-with-agent-lightning) ⚠️ You are viewing development documentation. For stable documentation, please visit [https://microsoft.github.io/agent-lightning/stable/](https://microsoft.github.io/agent-lightning/stable/) Train the First Agent with Agent-lightning[¶](https://microsoft.github.io/agent-lightning/stable/how-to/train-first-agent/#train-the-first-agent-with-agent-lightning "Permanent link") ======================================================================================================================================================================================== Welcome! This tutorial is your first step into making AI agents smarter using the **Agent-lightning** framework. We'll show you how to take a simple agent and automatically improve its performance through a process called [**Automatic Prompt Optimization (APO)**](https://microsoft.github.io/agent-lightning/stable/algorithm-zoo/apo/) . The main goal of Agent-lightning is to provide a structured way to **train your agents**. Just like you train a machine learning model on data, you can train an agent on a task dataset. This could involve using Reinforcement Learning (RL) to teach it new behaviors or, as we'll do today, optimizing its prompts to make it more accurate and reliable. Tip You can open the sample code [room\_selector\_apo.py](https://github.com/microsoft/agent-lightning/tree/3b5d733861cf313fc09821a23240bbdf3cb2ee5b/examples/apo/room_selector_apo.py) and [room\_selector.py](https://github.com/microsoft/agent-lightning/tree/3b5d733861cf313fc09821a23240bbdf3cb2ee5b/examples/apo/room_selector.py) as you go through this tutorial. Our Example: The Room Selector Agent[¶](https://microsoft.github.io/agent-lightning/stable/how-to/train-first-agent/#our-example-the-room-selector-agent "Permanent link") --------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Today, we'll work with an agent whose job is to book a meeting room. It's a common but tricky task with multiple constraints. Here's how the agent works: * **Input:** It receives a task with specific requirements, like "`Find a room for 4 people at 10:00 AM with a whiteboard.`" * **Action:** The agent uses a Large Language Model (LLM) to understand the request. It can also use tools, which are pre-defined functions it can call, to get more information, such as checking room availability in an external database. * **Output:** Its final decision is the ID of the best room it found, like "`A103`". * **Reward:** After the agent makes its choice, a separate "grader" function scores its performance on a scale of 0 to 1. This score is called its **reward**. A perfect choice gets a 1.0, while a wrong one gets a 0.0. The agent's logic is sound, but its performance heavily depends on its initial prompt. A poorly worded prompt will confuse the LLM, leading to bad decisions. Our goal is to use Agent-lightning to find the best possible prompt automatically. A Closer Look at the Agent's Logic Modern LLMs can do more than just generate text; they can decide to call functions you provide. This is often called tool use or function calling. Our agent uses this capability to make informed decisions. If you're new to this concept, you can read more about it in [OpenAI's documentation](https://platform.openai.com/docs/guides/function-calling) . Here is a sketch of the agent's logic, adhering closely to the OpenAI API: `[](https://microsoft.github.io/agent-lightning/stable/how-to/train-first-agent/#__codelineno-0-1) # Pseudo-code for the Room Selector agent [](https://microsoft.github.io/agent-lightning/stable/how-to/train-first-agent/#__codelineno-0-2) [](https://microsoft.github.io/agent-lightning/stable/how-to/train-first-agent/#__codelineno-0-3) import openai [](https://microsoft.github.io/agent-lightning/stable/how-to/train-first-agent/#__codelineno-0-4) import json [](https://microsoft.github.io/agent-lightning/stable/how-to/train-first-agent/#__codelineno-0-5) [](https://microsoft.github.io/agent-lightning/stable/how-to/train-first-agent/#__codelineno-0-6) def room_selector_agent(task, prompt): [](https://microsoft.github.io/agent-lightning/stable/how-to/train-first-agent/#__codelineno-0-7) client = openai.OpenAI() [](https://microsoft.github.io/agent-lightning/stable/how-to/train-first-agent/#__codelineno-0-8) messages = [{"role": "user", "content": prompt.format(**task)}] [](https://microsoft.github.io/agent-lightning/stable/how-to/train-first-agent/#__codelineno-0-9) tools = [ ... ] # Tool definition for the LLM [](https://microsoft.github.io/agent-lightning/stable/how-to/train-first-agent/#__codelineno-0-10) [](https://microsoft.github.io/agent-lightning/stable/how-to/train-first-agent/#__codelineno-0-11) # 1. First LLM call to decide if a tool is needed. [](https://microsoft.github.io/agent-lightning/stable/how-to/train-first-agent/#__codelineno-0-12) response = client.chat.completions.create( [](https://microsoft.github.io/agent-lightning/stable/how-to/train-first-agent/#__codelineno-0-13) model="gpt-5-mini", [](https://microsoft.github.io/agent-lightning/stable/how-to/train-first-agent/#__codelineno-0-14) messages=messages, [](https://microsoft.github.io/agent-lightning/stable/how-to/train-first-agent/#__codelineno-0-15) tools=tools, [](https://microsoft.github.io/agent-lightning/stable/how-to/train-first-agent/#__codelineno-0-16) tool_choice="auto", [](https://microsoft.github.io/agent-lightning/stable/how-to/train-first-agent/#__codelineno-0-17) ) [](https://microsoft.github.io/agent-lightning/stable/how-to/train-first-agent/#__codelineno-0-18) response_message = response.choices[0].message [](https://microsoft.github.io/agent-lightning/stable/how-to/train-first-agent/#__codelineno-0-19) tool_calls = response_message.tool_calls [](https://microsoft.github.io/agent-lightning/stable/how-to/train-first-agent/#__codelineno-0-20) [](https://microsoft.github.io/agent-lightning/stable/how-to/train-first-agent/#__codelineno-0-21) # 2. Check if the LLM wants to use a tool. [](https://microsoft.github.io/agent-lightning/stable/how-to/train-first-agent/#__codelineno-0-22) if tool_calls: [](https://microsoft.github.io/agent-lightning/stable/how-to/train-first-agent/#__codelineno-0-23) messages.append(response_message) # Append assistant's reply [](https://microsoft.github.io/agent-lightning/stable/how-to/train-first-agent/#__codelineno-0-24) [](https://microsoft.github.io/agent-lightning/stable/how-to/train-first-agent/#__codelineno-0-25) # 3. Execute the tool and get the real-world data. [](https://microsoft.github.io/agent-lightning/stable/how-to/train-first-agent/#__codelineno-0-26) for tool_call in tool_calls: [](https://microsoft.github.io/agent-lightning/stable/how-to/train-first-agent/#__codelineno-0-27) function_name = tool_call.function.name [](https://microsoft.github.io/agent-lightning/stable/how-to/train-first-agent/#__codelineno-0-28) if function_name == "get_rooms_and_availability": [](https://microsoft.github.io/agent-lightning/stable/how-to/train-first-agent/#__codelineno-0-29) function_args = json.loads(tool_call.function.arguments) [](https://microsoft.github.io/agent-lightning/stable/how-to/train-first-agent/#__codelineno-0-30) # Query the local room database [](https://microsoft.github.io/agent-lightning/stable/how-to/train-first-agent/#__codelineno-0-31) function_response = get_rooms_and_availability( [](https://microsoft.github.io/agent-lightning/stable/how-to/train-first-agent/#__codelineno-0-32) date=function_args.get("date"), [](https://microsoft.github.io/agent-lightning/stable/how-to/train-first-agent/#__codelineno-0-33) time_str=function_args.get("time"), [](https://microsoft.github.io/agent-lightning/stable/how-to/train-first-agent/#__codelineno-0-34) duration_min=function_args.get("duration_min"), [](https://microsoft.github.io/agent-lightning/stable/how-to/train-first-agent/#__codelineno-0-35) ) [](https://microsoft.github.io/agent-lightning/stable/how-to/train-first-agent/#__codelineno-0-36) messages.append({ [](https://microsoft.github.io/agent-lightning/stable/how-to/train-first-agent/#__codelineno-0-37) "tool_call_id": tool_call.id, [](https://microsoft.github.io/agent-lightning/stable/how-to/train-first-agent/#__codelineno-0-38) "role": "tool", [](https://microsoft.github.io/agent-lightning/stable/how-to/train-first-agent/#__codelineno-0-39) "name": function_name, [](https://microsoft.github.io/agent-lightning/stable/how-to/train-first-agent/#__codelineno-0-40) "content": json.dumps(function_response), [](https://microsoft.github.io/agent-lightning/stable/how-to/train-first-agent/#__codelineno-0-41) }) [](https://microsoft.github.io/agent-lightning/stable/how-to/train-first-agent/#__codelineno-0-42) [](https://microsoft.github.io/agent-lightning/stable/how-to/train-first-agent/#__codelineno-0-43) # 4. Second LLM call with the tool's output to get a final choice. [](https://microsoft.github.io/agent-lightning/stable/how-to/train-first-agent/#__codelineno-0-44) second_response = client.chat.completions.create( [](https://microsoft.github.io/agent-lightning/stable/how-to/train-first-agent/#__codelineno-0-45) model="gpt-5-mini", [](https://microsoft.github.io/agent-lightning/stable/how-to/train-first-agent/#__codelineno-0-46) messages=messages, [](https://microsoft.github.io/agent-lightning/stable/how-to/train-first-agent/#__codelineno-0-47) ) [](https://microsoft.github.io/agent-lightning/stable/how-to/train-first-agent/#__codelineno-0-48) final_choice = second_response.choices[0].message.content [](https://microsoft.github.io/agent-lightning/stable/how-to/train-first-agent/#__codelineno-0-49) else: [](https://microsoft.github.io/agent-lightning/stable/how-to/train-first-agent/#__codelineno-0-50) final_choice = response_message.content [](https://microsoft.github.io/agent-lightning/stable/how-to/train-first-agent/#__codelineno-0-51) [](https://microsoft.github.io/agent-lightning/stable/how-to/train-first-agent/#__codelineno-0-52) # 5. Grade the final choice to get a reward. [](https://microsoft.github.io/agent-lightning/stable/how-to/train-first-agent/#__codelineno-0-53) reward = grade_the_choice(final_choice, task["expected_choice"]) [](https://microsoft.github.io/agent-lightning/stable/how-to/train-first-agent/#__codelineno-0-54) return reward` In Agent-lightning, you wrap this logic in a Python function marked with the [`@rollout`](https://microsoft.github.io/agent-lightning/stable/reference/agent/#agentlightning.rollout " agentlightning.rollout(func)") decorator, so that the agent can be managed and tuned by Agent-lightning's runner and trainer. The `prompt_template` that the APO algorithm tunes is passed in as an argument: `[](https://microsoft.github.io/agent-lightning/stable/how-to/train-first-agent/#__codelineno-1-1) import agentlightning as agl [](https://microsoft.github.io/agent-lightning/stable/how-to/train-first-agent/#__codelineno-1-2) [](https://microsoft.github.io/agent-lightning/stable/how-to/train-first-agent/#__codelineno-1-3) @agl.rollout [](https://microsoft.github.io/agent-lightning/stable/how-to/train-first-agent/#__codelineno-1-4) def room_selector(task: RoomSelectionTask, prompt_template: agl.PromptTemplate) -> float: [](https://microsoft.github.io/agent-lightning/stable/how-to/train-first-agent/#__codelineno-1-5) # ... agent logic using the prompt_template ... [](https://microsoft.github.io/agent-lightning/stable/how-to/train-first-agent/#__codelineno-1-6) [](https://microsoft.github.io/agent-lightning/stable/how-to/train-first-agent/#__codelineno-1-7) # The final reward is determined by a grader function [](https://microsoft.github.io/agent-lightning/stable/how-to/train-first-agent/#__codelineno-1-8) reward = room_selection_grader(client, final_message, task["expected_choice"]) [](https://microsoft.github.io/agent-lightning/stable/how-to/train-first-agent/#__codelineno-1-9) return reward` Core Concepts: Tasks, Rollouts, Spans, and Prompt Templates[¶](https://microsoft.github.io/agent-lightning/stable/how-to/train-first-agent/#core-concepts-tasks-rollouts-spans-and-prompt-templates "Permanent link") ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- To understand how Agent-lightning works, you need to know these key terms. ### Task[¶](https://microsoft.github.io/agent-lightning/stable/how-to/train-first-agent/#task "Permanent link") A task is a specific input or problem statement given to the agent. It defines what the agent needs to accomplish. Analogy: Task If the agent is a chef, a task is the recipe request: "Bake a chocolate cake." ### Rollout[¶](https://microsoft.github.io/agent-lightning/stable/how-to/train-first-agent/#rollout "Permanent link") A rollout is a single, complete execution of an agent attempting to solve a given **task**. It's the entire story from receiving the task to producing a final result and receiving a reward. A rollout captures a full trace of the agent's execution. Analogy: Rollout A rollout is one full attempt by the chef to bake the chocolate cake, from gathering ingredients to the final taste test. ### Span[¶](https://microsoft.github.io/agent-lightning/stable/how-to/train-first-agent/#span "Permanent link") A span represents a single unit of work or an operation within a **rollout**. Spans are the building blocks of a trace. They have a start and end time and contain details about the specific operation, like an LLM call, a tool execution, or a reward calculation. For a more precise definition, see the [OpenTelemetry documentation](https://opentelemetry.io/docs/concepts/signals/traces/) . Analogy: Span If the rollout is "baking a cake," a span could be "preheating the oven," "mixing flour and sugar," or "adding frosting." Each is a distinct step or unit of work. The picture below from [ADK](https://google.github.io/adk-docs/observability/cloud-trace/) shows a typical rollout, where each rectangle in the waterfall visualizes a span. As can be seen in the visualization, spans can be sequential, parallel or nested among each other. In other frameworks, the terminology might be slightly different. Agent-lightning follows the terminologies used by OpenTelemetry to avoid confusion. ![AgentOps Waterfall Visualization](https://microsoft.github.io/agent-lightning/stable/assets/agentops-waterfall-visualization.jpg) ### Prompt Template[¶](https://microsoft.github.io/agent-lightning/stable/how-to/train-first-agent/#prompt-template "Permanent link") A prompt template is a reusable instruction for the agent, often containing placeholders that can be filled in with specific details from a task. It is a key **"resource"** that the algorithm learns and improves over time. Analogy: Resource (Prompt Template) If the task is the recipe request, the prompt template is the master recipe card that the chef follows. The algorithm's job is to edit this recipe card to make the instructions clearer and the final dish better. The Training Loop: How the Magic Happens[¶](https://microsoft.github.io/agent-lightning/stable/how-to/train-first-agent/#the-training-loop-how-the-magic-happens "Permanent link") ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Training in Agent-lightning revolves around a clear, managed loop, orchestrated by the **Trainer**. The diagram below illustrates this core interaction: ![Loop of Tasks and Spans](https://microsoft.github.io/agent-lightning/stable/assets/tasks-spans-loop.svg) **The Loop Explained:** * **Algorithm to Agent (via Trainer):** The **Algorithm** (the "brain") creates an improved **Prompt Template** and selects **Tasks**. The Trainer then sends both to the Agent. * **Agent to Algorithm (via Trainer):** For each task it receives, the Agent uses the provided prompt template to perform a Rollout, executing its logic and potentially using tools. During this rollout, the runner that runs the agent captures Spans that detail every step. The agent also calculates a Reward for its performance on the task. These spans and rewards are then sent back to the Algorithm via the Trainer. * **Algorithm Learning:** The Algorithm then analyzes these spans and rewards to learn how to improve the agent's behavior, for example, by generating a better prompt. This improved prompt is then used in the next iteration of tasks. This cycle continues, allowing the agent to continuously learn and get better at solving tasks. Note In the next tutorial, we will see that the "via Trainer" here is not accurate. It's actually via the runner and store. ### The Algorithm[¶](https://microsoft.github.io/agent-lightning/stable/how-to/train-first-agent/#the-algorithm "Permanent link") The algorithm is the smart part of the system that drives the improvement. In this tutorial, we use [**APO**](https://microsoft.github.io/agent-lightning/stable/algorithm-zoo/apo/#agentlightning.algorithm.apo.APO " APO") (Automatic Prompt Optimization). It works in a few steps: 1. **Evaluate:** The algorithm first asks for rollouts to be run using the current prompt template to see how well it performs. 2. **Critique:** It then looks at the detailed spans from those rollouts. Using a powerful LLM (`gpt-5-mini`), it generates a "textual gradient", which is a natural language critique of the prompt. For example: "The prompt is ambiguous about how to handle tie-breakers for equally good rooms." 3. **Rewrite:** Finally, it gives the critique and the original prompt to another LLM (`gpt-4.1-mini`) and asks it to apply the edits, generating a new, improved prompt template. This cycle repeats, with each round producing a slightly better prompt. To use it, you simply initialize the APO class with your desired hyperparameters. `[](https://microsoft.github.io/agent-lightning/stable/how-to/train-first-agent/#__codelineno-2-1) # In the main training script: run_apo.py [](https://microsoft.github.io/agent-lightning/stable/how-to/train-first-agent/#__codelineno-2-2) from openai import AsyncOpenAI [](https://microsoft.github.io/agent-lightning/stable/how-to/train-first-agent/#__codelineno-2-3) [](https://microsoft.github.io/agent-lightning/stable/how-to/train-first-agent/#__codelineno-2-4) openai = AsyncOpenAI() [](https://microsoft.github.io/agent-lightning/stable/how-to/train-first-agent/#__codelineno-2-5) algo = agl.APO(openai)` Tip Make sure you have `OPENAI_API_KEY` set in your environment variables. ### The Trainer[¶](https://microsoft.github.io/agent-lightning/stable/how-to/train-first-agent/#the-trainer "Permanent link") The Trainer is the central component you'll interact with. It connects everything and manages the entire workflow by running the loop described above. You configure the Trainer, providing the algorithm, the number of parallel runners, and the initial prompt. A single call to [`trainer.fit()`](https://microsoft.github.io/agent-lightning/stable/reference/trainer/#agentlightning.Trainer.fit " fit(agent, train_dataset=None, *, val_dataset=None)") kicks off the entire process! ``[](https://microsoft.github.io/agent-lightning/stable/how-to/train-first-agent/#__codelineno-3-1) # 1. Configure the Trainer with the algorithm and initial prompt [](https://microsoft.github.io/agent-lightning/stable/how-to/train-first-agent/#__codelineno-3-2) trainer = agl.Trainer( [](https://microsoft.github.io/agent-lightning/stable/how-to/train-first-agent/#__codelineno-3-3) algorithm=algo, [](https://microsoft.github.io/agent-lightning/stable/how-to/train-first-agent/#__codelineno-3-4) n_runners=8, # Run 8 agents in parallel to try out the prompts [](https://microsoft.github.io/agent-lightning/stable/how-to/train-first-agent/#__codelineno-3-5) initial_resources={ [](https://microsoft.github.io/agent-lightning/stable/how-to/train-first-agent/#__codelineno-3-6) # The initial prompt template to be tuned [](https://microsoft.github.io/agent-lightning/stable/how-to/train-first-agent/#__codelineno-3-7) "prompt_template": prompt_template_baseline() [](https://microsoft.github.io/agent-lightning/stable/how-to/train-first-agent/#__codelineno-3-8) }, [](https://microsoft.github.io/agent-lightning/stable/how-to/train-first-agent/#__codelineno-3-9) # This is used to convert the span data into a message format consumable by APO algorithm [](https://microsoft.github.io/agent-lightning/stable/how-to/train-first-agent/#__codelineno-3-10) adapter=agl.TraceToMessages(), [](https://microsoft.github.io/agent-lightning/stable/how-to/train-first-agent/#__codelineno-3-11) ) [](https://microsoft.github.io/agent-lightning/stable/how-to/train-first-agent/#__codelineno-3-12) [](https://microsoft.github.io/agent-lightning/stable/how-to/train-first-agent/#__codelineno-3-13) # 2. Load datasets: They can be list of task objects consumable by `room_selector`. [](https://microsoft.github.io/agent-lightning/stable/how-to/train-first-agent/#__codelineno-3-14) dataset_train, dataset_val = ... [](https://microsoft.github.io/agent-lightning/stable/how-to/train-first-agent/#__codelineno-3-15) [](https://microsoft.github.io/agent-lightning/stable/how-to/train-first-agent/#__codelineno-3-16) # 3. Start the training process! [](https://microsoft.github.io/agent-lightning/stable/how-to/train-first-agent/#__codelineno-3-17) trainer.fit( [](https://microsoft.github.io/agent-lightning/stable/how-to/train-first-agent/#__codelineno-3-18) agent=room_selector, [](https://microsoft.github.io/agent-lightning/stable/how-to/train-first-agent/#__codelineno-3-19) train_dataset=dataset_train, [](https://microsoft.github.io/agent-lightning/stable/how-to/train-first-agent/#__codelineno-3-20) val_dataset=dataset_val [](https://microsoft.github.io/agent-lightning/stable/how-to/train-first-agent/#__codelineno-3-21) )`` Tip [`TraceToMessages`](https://microsoft.github.io/agent-lightning/stable/reference/algorithm/#agentlightning.TraceToMessages " agentlightning.TraceToMessages") is a convenience adapter that converts spans into OpenAI chat messages. It requires `openai >= 1.100.0` to be installed. Training Results[¶](https://microsoft.github.io/agent-lightning/stable/how-to/train-first-agent/#training-results "Permanent link") ------------------------------------------------------------------------------------------------------------------------------------ The APO algorithm successfully improved the agent's performance. We ran the example with the following hyper-parameters: * `val_batch_size` = 10 * `gradient_batch_size` = 4 * `beam_width` = 2 * `branch_factor` = 2 * `beam_rounds` = 2 The validation accuracy on the 29 samples of datasets steadily increase from 0.569 (baseline) to **0.721** (after round 2). The tuning takes around 10 minutes with 8 runners. We ran twice, and the results are shown in the chart below. This demonstrates how Agent-lightning can efficiently and automatically enhance your agent's capabilities with just a few lines of code. Back to top --- # Write the First Algorithm - Agent-lightning [Skip to content](https://microsoft.github.io/agent-lightning/stable/how-to/write-first-algorithm/#write-the-first-algorithm-with-agent-lightning) ⚠️ You are viewing development documentation. For stable documentation, please visit [https://microsoft.github.io/agent-lightning/stable/](https://microsoft.github.io/agent-lightning/stable/) Write the First Algorithm with Agent-lightning[¶](https://microsoft.github.io/agent-lightning/stable/how-to/write-first-algorithm/#write-the-first-algorithm-with-agent-lightning "Permanent link") ==================================================================================================================================================================================================== In the [first tutorial](https://microsoft.github.io/agent-lightning/stable/how-to/train-first-agent/) , "Train the First Agent," we introduced the [Trainer](https://microsoft.github.io/agent-lightning/stable/reference/trainer/#agentlightning.Trainer " agentlightning.Trainer") and showed how to use a pre-built algorithm like **Automatic Prompt Optimization (APO)** to improve an agent's performance. The [Trainer](https://microsoft.github.io/agent-lightning/stable/reference/trainer/#agentlightning.Trainer " agentlightning.Trainer") handled all the complex interactions, letting us focus on the agent's logic. Now, we'll go a step deeper. What if you have a unique training idea that doesn't fit a standard algorithm? This tutorial will show you how to write your own custom algorithm from scratch. We'll build a simple algorithm that systematically tests a list of prompt templates and identifies the one with the highest reward. By the end, you'll understand the core mechanics of how the **Algorithm**, **Runner**, and a new component, the **Store**, work together to create the powerful training loop at the heart of Agent-lightning. Tip This tutorial helps you build a basic understanding of how to interact with Agent-lightning's core components. It's recommended that all users customizing algorithms should read this tutorial, even for those who are not planning to do prompt optimization. Core Concepts for Training[¶](https://microsoft.github.io/agent-lightning/stable/how-to/write-first-algorithm/#core-concepts-for-training "Permanent link") ------------------------------------------------------------------------------------------------------------------------------------------------------------ Before diving into the [LightningStore](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.LightningStore " agentlightning.LightningStore") , let's define two key concepts that are central to any training process in Agent-lightning: **Resources** and the **Tracer**. ### Resources: The Tunable Assets[¶](https://microsoft.github.io/agent-lightning/stable/how-to/write-first-algorithm/#resources-the-tunable-assets "Permanent link") [](https://microsoft.github.io/agent-lightning/stable/how-to/write-first-algorithm/) **Resources** are the assets your algorithm is trying to improve. Think of them as the "recipe" an agent uses to perform its task. This recipe can be: * A **prompt template** that guides an LLM. * The **weights** of a machine learning model. * Any other configuration or data your agent needs. The algorithm's job is to run experiments and iteratively update these resources to find the best-performing version. ### Tracer: The Data Collector[¶](https://microsoft.github.io/agent-lightning/stable/how-to/write-first-algorithm/#tracer-the-data-collector "Permanent link") How does the algorithm know if a change was an improvement? It needs data. This is where the **Tracer** comes in. The Tracer automatically **instruments** (aka modifies / patches) the agent's code. This means it watches for important events, like an LLM call, a tool being used, or **reward signals**, and records a detailed log of what happened. Each of these logs is called a **Span** (which has already been introduced in the [last tutorial](https://microsoft.github.io/agent-lightning/stable/how-to/train-first-agent/) ). A collection of spans from a single task execution gives the algorithm a complete, step-by-step trace of the agent's behavior, which is essential for learning and making improvements. Our default tracer is built on the AgentOps SDK to support instrumenting code written in various Agent/non-agent frameworks. The Central Hub: The LightningStore[¶](https://microsoft.github.io/agent-lightning/stable/how-to/write-first-algorithm/#the-central-hub-the-lightningstore "Permanent link") ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Now, where do all these resources, tasks, and spans live? They are all managed by the **LightningStore**. The LightningStore acts as the central database and message queue for the entire system. It's the single source of truth that decouples the Algorithm from the Runners. Note In the [last tutorial](https://microsoft.github.io/agent-lightning/stable/how-to/train-first-agent/) we simplified the training loop, saying the Algorithm and Agent communicate "via the Trainer." That's true at a high level, but the component that makes it all possible is actually the **LightningStore**. * The **Algorithm** connects to the Store to `enqueue_rollout` (tasks) and `update_resources` (like new prompt templates). It also queries the Store to retrieve the resulting spans and rewards from completed rollouts. * The **Runners** connect to the Store to `dequeue_rollout` (polling for available tasks). After executing a task, they use the `Tracer` to write the resulting spans and status updates back to the Store. This architecture is key to Agent-lightning's scalability. Since the Algorithm and Runners only talk to the Store, they can run in different processes or even on different machines. ![Store Architecture](https://microsoft.github.io/agent-lightning/stable/assets/store-api-visualized.svg) A Mental Model of What the Store Contains The [LightningStore](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.LightningStore " agentlightning.LightningStore") isn't just a simple database; it's an organized system for managing the entire training lifecycle. Here's what it keeps track of: * **Task Queue**: A queue of pending **Rollouts** waiting for a Runner to pick them up, interactable via `enqueue_rollout` and `dequeue_rollout`. * **Rollouts**: The record of a single task. A rollout contains metadata about the task and tracks all **Attempts** to complete it, interactable via `query_rollouts` and `wait_for_rollouts`. * **Attempts**: A single execution of a rollout. If an attempt fails (e.g., due to a network error), the Store can automatically schedules a retry if it's configured. Each attempt is linked to its parent rollout and contains the status and timing information. The rollout status is [synced](https://microsoft.github.io/agent-lightning/stable/deep-dive/store/) with its children's status. **For beginners, you can assume each rollout has only one attempt unless you have explicitly configure the retry.** * **Spans**: The detailed, structured logs generated by the `Tracer` during an attempt. Each span is linked to its parent attempt and rollout. * **Resources**: A versioned collection of the assets (like prompt templates) that the algorithm creates. Each rollout is linked to the specific version of the resources it should use. Building a Custom Algorithm[¶](https://microsoft.github.io/agent-lightning/stable/how-to/write-first-algorithm/#building-a-custom-algorithm "Permanent link") -------------------------------------------------------------------------------------------------------------------------------------------------------------- Let's build an algorithm that finds the best system prompt from a predefined list. The logic is straightforward: 1. Start with a list of candidate prompt templates. 2. For each template, create a "resource" bundle in the Store. 3. Enqueue a rollout (a task), telling the Runner to use this specific resource. 4. Wait for a Runner to pick up the task and complete it. 5. Query the Store to get the final reward from the rollout's spans. 6. After testing all templates, compare the rewards and declare the best one. We can implement this as a simple Python function that interacts directly with the [LightningStore](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.LightningStore " agentlightning.LightningStore") . `[](https://microsoft.github.io/agent-lightning/stable/how-to/write-first-algorithm/#__codelineno-0-1) async def find_best_prompt(store, prompts_to_test, task_input): [](https://microsoft.github.io/agent-lightning/stable/how-to/write-first-algorithm/#__codelineno-0-2) """A simple algorithm to find the best prompt from a list.""" [](https://microsoft.github.io/agent-lightning/stable/how-to/write-first-algorithm/#__codelineno-0-3) results = [] [](https://microsoft.github.io/agent-lightning/stable/how-to/write-first-algorithm/#__codelineno-0-4) [](https://microsoft.github.io/agent-lightning/stable/how-to/write-first-algorithm/#__codelineno-0-5) # Iterate through each prompt to test it [](https://microsoft.github.io/agent-lightning/stable/how-to/write-first-algorithm/#__codelineno-0-6) for prompt in prompts_to_test: [](https://microsoft.github.io/agent-lightning/stable/how-to/write-first-algorithm/#__codelineno-0-7) print(f"[Algo] Updating prompt template to: '{prompt}'") [](https://microsoft.github.io/agent-lightning/stable/how-to/write-first-algorithm/#__codelineno-0-8) [](https://microsoft.github.io/agent-lightning/stable/how-to/write-first-algorithm/#__codelineno-0-9) # 1. Update the resources in the store with the new prompt [](https://microsoft.github.io/agent-lightning/stable/how-to/write-first-algorithm/#__codelineno-0-10) resources_update = await store.add_resources( [](https://microsoft.github.io/agent-lightning/stable/how-to/write-first-algorithm/#__codelineno-0-11) resources={"prompt_template": prompt} [](https://microsoft.github.io/agent-lightning/stable/how-to/write-first-algorithm/#__codelineno-0-12) ) [](https://microsoft.github.io/agent-lightning/stable/how-to/write-first-algorithm/#__codelineno-0-13) [](https://microsoft.github.io/agent-lightning/stable/how-to/write-first-algorithm/#__codelineno-0-14) # 2. Enqueue a rollout task for a runner to execute [](https://microsoft.github.io/agent-lightning/stable/how-to/write-first-algorithm/#__codelineno-0-15) print("[Algo] Queuing task for clients...") [](https://microsoft.github.io/agent-lightning/stable/how-to/write-first-algorithm/#__codelineno-0-16) rollout = await store.enqueue_rollout( [](https://microsoft.github.io/agent-lightning/stable/how-to/write-first-algorithm/#__codelineno-0-17) input=task_input, [](https://microsoft.github.io/agent-lightning/stable/how-to/write-first-algorithm/#__codelineno-0-18) resources_id=resources_update.resources_id, [](https://microsoft.github.io/agent-lightning/stable/how-to/write-first-algorithm/#__codelineno-0-19) ) [](https://microsoft.github.io/agent-lightning/stable/how-to/write-first-algorithm/#__codelineno-0-20) print(f"[Algo] Task '{rollout.rollout_id}' is now available for clients.") [](https://microsoft.github.io/agent-lightning/stable/how-to/write-first-algorithm/#__codelineno-0-21) [](https://microsoft.github.io/agent-lightning/stable/how-to/write-first-algorithm/#__codelineno-0-22) # 3. Wait for the rollout to be completed by a runner [](https://microsoft.github.io/agent-lightning/stable/how-to/write-first-algorithm/#__codelineno-0-23) await store.wait_for_rollouts([rollout.rollout_id]) [](https://microsoft.github.io/agent-lightning/stable/how-to/write-first-algorithm/#__codelineno-0-24) [](https://microsoft.github.io/agent-lightning/stable/how-to/write-first-algorithm/#__codelineno-0-25) # 4. Query the completed rollout and its spans [](https://microsoft.github.io/agent-lightning/stable/how-to/write-first-algorithm/#__codelineno-0-26) completed_rollout = await store.get_rollout_by_id(rollout.rollout_id) [](https://microsoft.github.io/agent-lightning/stable/how-to/write-first-algorithm/#__codelineno-0-27) print(f"[Algo] Received Result: {completed_rollout.model_dump_json(indent=None)}") [](https://microsoft.github.io/agent-lightning/stable/how-to/write-first-algorithm/#__codelineno-0-28) [](https://microsoft.github.io/agent-lightning/stable/how-to/write-first-algorithm/#__codelineno-0-29) spans = await store.query_spans(rollout.rollout_id) [](https://microsoft.github.io/agent-lightning/stable/how-to/write-first-algorithm/#__codelineno-0-30) # We expect at least two spans: one for the LLM call and one for the final reward [](https://microsoft.github.io/agent-lightning/stable/how-to/write-first-algorithm/#__codelineno-0-31) print(f"[Algo] Queried Spans:\n - " + "\n - ".join(str(span) for span in spans)) [](https://microsoft.github.io/agent-lightning/stable/how-to/write-first-algorithm/#__codelineno-0-32) # find_final_reward is a helper function to extract the reward span [](https://microsoft.github.io/agent-lightning/stable/how-to/write-first-algorithm/#__codelineno-0-33) final_reward = find_final_reward(spans) [](https://microsoft.github.io/agent-lightning/stable/how-to/write-first-algorithm/#__codelineno-0-34) print(f"[Algo] Final reward: {final_reward}\n") [](https://microsoft.github.io/agent-lightning/stable/how-to/write-first-algorithm/#__codelineno-0-35) [](https://microsoft.github.io/agent-lightning/stable/how-to/write-first-algorithm/#__codelineno-0-36) results.append((prompt, final_reward)) [](https://microsoft.github.io/agent-lightning/stable/how-to/write-first-algorithm/#__codelineno-0-37) [](https://microsoft.github.io/agent-lightning/stable/how-to/write-first-algorithm/#__codelineno-0-38) # 5. Find and print the best prompt based on the collected rewards [](https://microsoft.github.io/agent-lightning/stable/how-to/write-first-algorithm/#__codelineno-0-39) print(f"[Algo] All prompts and their rewards: {results}") [](https://microsoft.github.io/agent-lightning/stable/how-to/write-first-algorithm/#__codelineno-0-40) best_prompt, best_reward = max(results, key=lambda item: item[1]) [](https://microsoft.github.io/agent-lightning/stable/how-to/write-first-algorithm/#__codelineno-0-41) print(f"[Algo] Best prompt found: '{best_prompt}' with reward {best_reward}")` Asynchronous Operations You'll notice the `async` and `await` keywords. Agent-lightning is built on asyncio to handle concurrent operations efficiently. All interactions with the store are asynchronous network calls, so they must be awaited. The Agent and Runner[¶](https://microsoft.github.io/agent-lightning/stable/how-to/write-first-algorithm/#the-agent-and-runner "Permanent link") ------------------------------------------------------------------------------------------------------------------------------------------------ Our algorithm needs an **agent** to execute the tasks and a **runner** to manage the process. The runner is a long-lived worker process. Its job is simple: 1. Connect to the [LightningStore](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.LightningStore " agentlightning.LightningStore") via a [LightningStoreClient](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.LightningStoreClient " agentlightning.LightningStoreClient") . 2. Enter a loop, constantly asking the [LightningStore](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.LightningStore " agentlightning.LightningStore") for new tasks (`dequeue_rollout`). 3. When it gets a task, it runs the `simple_agent` function. 4. Crucially, the runner wraps the agent execution with a **Tracer**. The tracer automatically captures all the important events (like the LLM call and the final reward) as spans and sends them back to the [LightningStore](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.LightningStore " agentlightning.LightningStore") . `[](https://microsoft.github.io/agent-lightning/stable/how-to/write-first-algorithm/#__codelineno-1-1) # Connecting to Store [](https://microsoft.github.io/agent-lightning/stable/how-to/write-first-algorithm/#__codelineno-1-2) store = agl.LightningStoreClient("http://localhost:4747") # or some other address [](https://microsoft.github.io/agent-lightning/stable/how-to/write-first-algorithm/#__codelineno-1-3) runner = LitAgentRunner[str](tracer=AgentOpsTracer()) [](https://microsoft.github.io/agent-lightning/stable/how-to/write-first-algorithm/#__codelineno-1-4) with runner.run_context(agent=simple_agent, store=store): # <-- where the wrapping and instrumentation happens [](https://microsoft.github.io/agent-lightning/stable/how-to/write-first-algorithm/#__codelineno-1-5) await runner.iter() # polling for new tasks forever` For this example, the agent's job is to take the prompt from the resources, use it to ask an LLM a question, and return a score. `[](https://microsoft.github.io/agent-lightning/stable/how-to/write-first-algorithm/#__codelineno-2-1) def simple_agent(task: str, prompt_template: PromptTemplate) -> float: [](https://microsoft.github.io/agent-lightning/stable/how-to/write-first-algorithm/#__codelineno-2-2) """An agent that answers a question and gets judged by an LLM.""" [](https://microsoft.github.io/agent-lightning/stable/how-to/write-first-algorithm/#__codelineno-2-3) client = OpenAI() [](https://microsoft.github.io/agent-lightning/stable/how-to/write-first-algorithm/#__codelineno-2-4) [](https://microsoft.github.io/agent-lightning/stable/how-to/write-first-algorithm/#__codelineno-2-5) # Generate a response using the provided prompt template [](https://microsoft.github.io/agent-lightning/stable/how-to/write-first-algorithm/#__codelineno-2-6) prompt = prompt_template.format(any_question=task) [](https://microsoft.github.io/agent-lightning/stable/how-to/write-first-algorithm/#__codelineno-2-7) response = client.chat.completions.create( [](https://microsoft.github.io/agent-lightning/stable/how-to/write-first-algorithm/#__codelineno-2-8) model="gpt-4.1-nano", messages=[{"role": "user", "content": prompt}] [](https://microsoft.github.io/agent-lightning/stable/how-to/write-first-algorithm/#__codelineno-2-9) ) [](https://microsoft.github.io/agent-lightning/stable/how-to/write-first-algorithm/#__codelineno-2-10) llm_output = response.choices[0].message.content [](https://microsoft.github.io/agent-lightning/stable/how-to/write-first-algorithm/#__codelineno-2-11) print(f"[Rollout] LLM returned: {llm_output}") [](https://microsoft.github.io/agent-lightning/stable/how-to/write-first-algorithm/#__codelineno-2-12) [](https://microsoft.github.io/agent-lightning/stable/how-to/write-first-algorithm/#__codelineno-2-13) # This llm_output and the final score are automatically logged as spans by the Tracer [](https://microsoft.github.io/agent-lightning/stable/how-to/write-first-algorithm/#__codelineno-2-14) score = random.uniform(0, 1) # Replace with actual scoring logic if needed [](https://microsoft.github.io/agent-lightning/stable/how-to/write-first-algorithm/#__codelineno-2-15) return score` Running the Example[¶](https://microsoft.github.io/agent-lightning/stable/how-to/write-first-algorithm/#running-the-example "Permanent link") ---------------------------------------------------------------------------------------------------------------------------------------------- To see everything in action, you'll need three separate terminal windows. Tip If you want to follow along, you can find the complete code for this example in the [apo\_custom\_algorithm.py](https://github.com/microsoft/agent-lightning/tree/3b5d733861cf313fc09821a23240bbdf3cb2ee5b/examples/apo/apo_custom_algorithm.py) file. **1\. Start the Store:** In the first terminal, start the LightningStore server. This component will wait for connections from the algorithm and the runner. The store will be listening on port `4747` ⚡ by default. `[](https://microsoft.github.io/agent-lightning/stable/how-to/write-first-algorithm/#__codelineno-3-1) agl store` **2\. Start the Runner:** In the second terminal, start the runner process. It will connect to the store and wait for tasks. The code to start the runner looks like the following: `[](https://microsoft.github.io/agent-lightning/stable/how-to/write-first-algorithm/#__codelineno-4-1) export OPENAI_API_KEY=sk-... # Your OpenAI API key [](https://microsoft.github.io/agent-lightning/stable/how-to/write-first-algorithm/#__codelineno-4-2) python apo_custom_algorithm.py runner` You will see output indicating the runner has started and is waiting for rollouts. `[](https://microsoft.github.io/agent-lightning/stable/how-to/write-first-algorithm/#__codelineno-5-1) 2025-10-14 22:23:41,339 [INFO] ... [Worker 0] Setting up tracer... [](https://microsoft.github.io/agent-lightning/stable/how-to/write-first-algorithm/#__codelineno-5-2) 2025-10-14 22:23:41,343 [INFO] ... [Worker 0] Instrumentation applied. [](https://microsoft.github.io/agent-lightning/stable/how-to/write-first-algorithm/#__codelineno-5-3) 2025-10-14 22:23:41,494 [INFO] ... [Worker 0] AgentOps client initialized. [](https://microsoft.github.io/agent-lightning/stable/how-to/write-first-algorithm/#__codelineno-5-4) 2025-10-14 22:23:41,494 [INFO] ... [Worker 0] Started async rollouts (max: unlimited).` **3\. Start the Algorithm:** In the third terminal, run the algorithm. This will kick off the entire process. For example, we run the algorithm code shown above with the following parameters: `[](https://microsoft.github.io/agent-lightning/stable/how-to/write-first-algorithm/#__codelineno-6-1) prompts_to_test = [ [](https://microsoft.github.io/agent-lightning/stable/how-to/write-first-algorithm/#__codelineno-6-2) "You are a helpful assistant. {any_question}", [](https://microsoft.github.io/agent-lightning/stable/how-to/write-first-algorithm/#__codelineno-6-3) "You are a knowledgeable AI. {any_question}", [](https://microsoft.github.io/agent-lightning/stable/how-to/write-first-algorithm/#__codelineno-6-4) "You are a friendly chatbot. {any_question}", [](https://microsoft.github.io/agent-lightning/stable/how-to/write-first-algorithm/#__codelineno-6-5) ] [](https://microsoft.github.io/agent-lightning/stable/how-to/write-first-algorithm/#__codelineno-6-6) task_input = "Why is the sky blue?" [](https://microsoft.github.io/agent-lightning/stable/how-to/write-first-algorithm/#__codelineno-6-7) store = agl.LightningStoreClient("http://localhost:4747") [](https://microsoft.github.io/agent-lightning/stable/how-to/write-first-algorithm/#__codelineno-6-8) find_best_prompt(store, prompts_to_test, task_input)` Or you can simply use our pre-written script to try out: `[](https://microsoft.github.io/agent-lightning/stable/how-to/write-first-algorithm/#__codelineno-7-1) python apo_custom_algorithm.py algo` ### Understanding the Output[¶](https://microsoft.github.io/agent-lightning/stable/how-to/write-first-algorithm/#understanding-the-output "Permanent link") As the algorithm runs, you'll see logs appear across all three terminals, showing the components interacting in real-time. **Algorithm Output:** The algorithm terminal shows the main control flow: updating prompts, queuing tasks, and receiving the final results. You can also see the raw span data it retrieves from the store. `[](https://microsoft.github.io/agent-lightning/stable/how-to/write-first-algorithm/#__codelineno-8-1) [Algo] Updating prompt template to: 'You are a helpful assistant. {any_question}' [](https://microsoft.github.io/agent-lightning/stable/how-to/write-first-algorithm/#__codelineno-8-2) [Algo] Queuing task for clients... [](https://microsoft.github.io/agent-lightning/stable/how-to/write-first-algorithm/#__codelineno-8-3) [Algo] Task 'ro-1d18988581cd' is now available for clients. [](https://microsoft.github.io/agent-lightning/stable/how-to/write-first-algorithm/#__codelineno-8-4) [Algo] Received Result: rollout_id='ro-1d18988581cd' ... status='succeeded' ... [](https://microsoft.github.io/agent-lightning/stable/how-to/write-first-algorithm/#__codelineno-8-5) [Algo] Queried Spans: [](https://microsoft.github.io/agent-lightning/stable/how-to/write-first-algorithm/#__codelineno-8-6) - Span(name='openai.chat.completion', attributes={'gen_ai.prompt.0.content': 'You are a helpful assistant...', 'gen_ai.completion.0.content': 'The sky appears blue...'}) [](https://microsoft.github.io/agent-lightning/stable/how-to/write-first-algorithm/#__codelineno-8-7) - Span(name='reward', attributes={'value': 0.95}) [](https://microsoft.github.io/agent-lightning/stable/how-to/write-first-algorithm/#__codelineno-8-8) [Algo] Final reward: 0.95 [](https://microsoft.github.io/agent-lightning/stable/how-to/write-first-algorithm/#__codelineno-8-9) [](https://microsoft.github.io/agent-lightning/stable/how-to/write-first-algorithm/#__codelineno-8-10) [Algo] Updating prompt template to: 'You are a knowledgeable AI. {any_question}' [](https://microsoft.github.io/agent-lightning/stable/how-to/write-first-algorithm/#__codelineno-8-11) ... [](https://microsoft.github.io/agent-lightning/stable/how-to/write-first-algorithm/#__codelineno-8-12) [Algo] Final reward: 0.95 [](https://microsoft.github.io/agent-lightning/stable/how-to/write-first-algorithm/#__codelineno-8-13) [](https://microsoft.github.io/agent-lightning/stable/how-to/write-first-algorithm/#__codelineno-8-14) [Algo] Updating prompt template to: 'You are a friendly chatbot. {any_question}' [](https://microsoft.github.io/agent-lightning/stable/how-to/write-first-algorithm/#__codelineno-8-15) ... [](https://microsoft.github.io/agent-lightning/stable/how-to/write-first-algorithm/#__codelineno-8-16) [Algo] Final reward: 1.0 [](https://microsoft.github.io/agent-lightning/stable/how-to/write-first-algorithm/#__codelineno-8-17) [](https://microsoft.github.io/agent-lightning/stable/how-to/write-first-algorithm/#__codelineno-8-18) [Algo] All prompts and their rewards: [('You are a helpful assistant. {any_question}', 0.95), ('You are a knowledgeable AI. {any_question}', 0.95), ('You are a friendly chatbot. {any_question}', 1.0)] [](https://microsoft.github.io/agent-lightning/stable/how-to/write-first-algorithm/#__codelineno-8-19) [Algo] Best prompt found: 'You are a friendly chatbot. {any_question}' with reward 1.0` **Runner Output:** The runner terminal shows it picking up each task, executing the agent logic, and reporting the completion. `[](https://microsoft.github.io/agent-lightning/stable/how-to/write-first-algorithm/#__codelineno-9-1) [Rollout] LLM returned: The sky appears blue due to Rayleigh scattering... [](https://microsoft.github.io/agent-lightning/stable/how-to/write-first-algorithm/#__codelineno-9-2) 2025-10-14 22:25:50,803 [INFO] ... [Worker 0 | Rollout ro-a9f54ac19af5] Completed in 4.24s. ... [](https://microsoft.github.io/agent-lightning/stable/how-to/write-first-algorithm/#__codelineno-9-3) [](https://microsoft.github.io/agent-lightning/stable/how-to/write-first-algorithm/#__codelineno-9-4) [Rollout] LLM returned: The sky looks blue because of a process called Rayleigh scattering... [](https://microsoft.github.io/agent-lightning/stable/how-to/write-first-algorithm/#__codelineno-9-5) 2025-10-14 22:25:59,863 [INFO] ... [Worker 0 | Rollout ro-c67eaa9016b6] Completed in 4.06s. ...` **Store Server Output:** The store terminal shows a detailed log of every interaction, confirming its role as the central hub. You can see requests to enqueue and dequeue rollouts, add spans, and update statuses. `[](https://microsoft.github.io/agent-lightning/stable/how-to/write-first-algorithm/#__codelineno-10-1) ... "POST /enqueue_rollout HTTP/1.1" 200 ... [](https://microsoft.github.io/agent-lightning/stable/how-to/write-first-algorithm/#__codelineno-10-2) ... "GET /dequeue_rollout HTTP/1.1" 200 ... [](https://microsoft.github.io/agent-lightning/stable/how-to/write-first-algorithm/#__codelineno-10-3) ... "POST /add_span HTTP/1.1" 200 ... [](https://microsoft.github.io/agent-lightning/stable/how-to/write-first-algorithm/#__codelineno-10-4) ... "POST /update_attempt HTTP/1.1" 200 ... [](https://microsoft.github.io/agent-lightning/stable/how-to/write-first-algorithm/#__codelineno-10-5) ... "POST /wait_for_rollouts HTTP/1.1" 200 ... [](https://microsoft.github.io/agent-lightning/stable/how-to/write-first-algorithm/#__codelineno-10-6) ... "GET /query_spans/ro-c67eaa9016b6 HTTP/1.1" 200 ...` So Where is Trainer? You might be wondering why the [last tutorial](https://microsoft.github.io/agent-lightning/stable/how-to/train-first-agent/) focused on the [Trainer](https://microsoft.github.io/agent-lightning/stable/reference/trainer/#agentlightning.Trainer " agentlightning.Trainer") class, but we haven't used it here. Think of the [Trainer](https://microsoft.github.io/agent-lightning/stable/reference/trainer/#agentlightning.Trainer " agentlightning.Trainer") as a convenient wrapper that manages the entire training process for you. It's perfect when you want to apply a pre-built algorithm to your agent without worrying about the underlying mechanics. The [Trainer](https://microsoft.github.io/agent-lightning/stable/reference/trainer/#agentlightning.Trainer " agentlightning.Trainer") handles starting the [LightningStore](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.LightningStore " agentlightning.LightningStore") , coordinating the [Runners](https://microsoft.github.io/agent-lightning/stable/reference/runner/#agentlightning.Runner " agentlightning.Runner") , managing their lifecycles, and handling errors. In this tutorial, however, our goal is to _build a new algorithm_. To do that, we need to interact directly with the core components: the [Store](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.LightningStore " agentlightning.LightningStore") , the [Runner](https://microsoft.github.io/agent-lightning/stable/reference/runner/#agentlightning.Runner " agentlightning.Runner") , and the algorithm logic itself. Running them separately gives you more control and clearer, isolated logs, which is ideal for development and debugging. Once your custom algorithm is mature, you can package it to comply with our standard interface ([@algo](https://microsoft.github.io/agent-lightning/stable/reference/algorithm/#agentlightning.algo " agentlightning.algo(func)") or [Algorithm](https://microsoft.github.io/agent-lightning/stable/reference/algorithm/#agentlightning.Algorithm " agentlightning.Algorithm") ). This allows you to use it with the [Trainer](https://microsoft.github.io/agent-lightning/stable/reference/trainer/#agentlightning.Trainer " agentlightning.Trainer") again, getting all the benefits of automated lifecycle management while using your own custom logic. A sample code doing this is available in [apo\_custom\_algorithm\_trainer.py](https://github.com/microsoft/agent-lightning/tree/3b5d733861cf313fc09821a23240bbdf3cb2ee5b/examples/apo/apo_custom_algorithm_trainer.py) . Back to top --- # Write Agents - Agent-lightning [Skip to content](https://microsoft.github.io/agent-lightning/stable/tutorials/write-agents/#writing-agents) ⚠️ You are viewing development documentation. For stable documentation, please visit [https://microsoft.github.io/agent-lightning/stable/](https://microsoft.github.io/agent-lightning/stable/) Writing Agents[¶](https://microsoft.github.io/agent-lightning/stable/tutorials/write-agents/#writing-agents "Permanent link") ============================================================================================================================== This tutorial will focus on the heart of the system: the agent itself, guiding you through the different ways to define an agent's logic in Agent-lightning. The basic requirements for any agent are: 1. It must accept a single **task** as input. 2. It must accept a set of tunable **resources** (like a [PromptTemplate](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.PromptTemplate " agentlightning.PromptTemplate") or [LLM](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.LLM " agentlightning.LLM") ). 3. It must **emit** trace span data so that algorithms can understand its behavior and learn from it. _The simplest way to do this is by returning a final reward._ In practice, please also bear in mind that tasks, resources, and spans have extra requirements, in order to make it _trainable_ within Agent-lightning: 1. You will need a training dataset containing a set of tasks, of the same type that your agent expects as input. 2. The tunable resources are related to the algorithm. For example, the APO algorithm we've seen tunes a [PromptTemplate](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.PromptTemplate " agentlightning.PromptTemplate") . Other algorithms might tune model weights or other configurations. 3. The type of spans an algorithm can use varies. Almost all algorithms support a single, final reward span at the end of a rollout. However, not all algorithms support rewards emitted mid-rollout, let alone other kinds of spans like exceptions or log messages. This tutorial will show you how to write an agent that can handle various tasks and resources and emit all kinds of spans. However, you should understand that agents and algorithms are often co-designed. Supporting new types of resources or spans in an algorithm is often much more complex than just adding them to an agent. [`@rollout`](https://microsoft.github.io/agent-lightning/stable/reference/agent/#agentlightning.rollout " agentlightning.rollout(func)") Decorator[¶](https://microsoft.github.io/agent-lightning/stable/tutorials/write-agents/#rollout-decorator "Permanent link") --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- The simplest way to create an agent is by writing a standard Python function and marking it with the [@rollout](https://microsoft.github.io/agent-lightning/stable/reference/agent/#agentlightning.rollout " agentlightning.rollout(func)") decorator. This approach is perfect for agents with straightforward logic that doesn't require complex state management. Agent-lightning automatically inspects your function's signature and injects the required resources. For example, if your function has a parameter named `prompt_template`, Agent-lightning will find the [PromptTemplate](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.PromptTemplate " agentlightning.PromptTemplate") resource for the current rollout and pass it in. Let's revisit the `room_selector` agent from the first tutorial: `[](https://microsoft.github.io/agent-lightning/stable/tutorials/write-agents/#__codelineno-0-1) from typing import TypedDict [](https://microsoft.github.io/agent-lightning/stable/tutorials/write-agents/#__codelineno-0-2) from agentlightning import PromptTemplate, rollout [](https://microsoft.github.io/agent-lightning/stable/tutorials/write-agents/#__codelineno-0-3) [](https://microsoft.github.io/agent-lightning/stable/tutorials/write-agents/#__codelineno-0-4) # Define a data structure for the task input [](https://microsoft.github.io/agent-lightning/stable/tutorials/write-agents/#__codelineno-0-5) class RoomSelectionTask(TypedDict): [](https://microsoft.github.io/agent-lightning/stable/tutorials/write-agents/#__codelineno-0-6) # ... fields for the task ... [](https://microsoft.github.io/agent-lightning/stable/tutorials/write-agents/#__codelineno-0-7) pass [](https://microsoft.github.io/agent-lightning/stable/tutorials/write-agents/#__codelineno-0-8) [](https://microsoft.github.io/agent-lightning/stable/tutorials/write-agents/#__codelineno-0-9) @rollout [](https://microsoft.github.io/agent-lightning/stable/tutorials/write-agents/#__codelineno-0-10) def room_selector(task: RoomSelectionTask, prompt_template: PromptTemplate) -> float: [](https://microsoft.github.io/agent-lightning/stable/tutorials/write-agents/#__codelineno-0-11) # 1. Use the injected prompt_template to format the input for the LLM [](https://microsoft.github.io/agent-lightning/stable/tutorials/write-agents/#__codelineno-0-12) prompt = prompt_template.format(**task) [](https://microsoft.github.io/agent-lightning/stable/tutorials/write-agents/#__codelineno-0-13) [](https://microsoft.github.io/agent-lightning/stable/tutorials/write-agents/#__codelineno-0-14) # 2. Execute the agent's logic (e.g., call an LLM, use tools) [](https://microsoft.github.io/agent-lightning/stable/tutorials/write-agents/#__codelineno-0-15) # ... [](https://microsoft.github.io/agent-lightning/stable/tutorials/write-agents/#__codelineno-0-16) [](https://microsoft.github.io/agent-lightning/stable/tutorials/write-agents/#__codelineno-0-17) # 3. Grade the final choice to get a reward [](https://microsoft.github.io/agent-lightning/stable/tutorials/write-agents/#__codelineno-0-18) reward = room_selection_grader(final_message, task["expected_choice"]) [](https://microsoft.github.io/agent-lightning/stable/tutorials/write-agents/#__codelineno-0-19) [](https://microsoft.github.io/agent-lightning/stable/tutorials/write-agents/#__codelineno-0-20) # 4. Return the final reward as a float [](https://microsoft.github.io/agent-lightning/stable/tutorials/write-agents/#__codelineno-0-21) return reward` When you train this agent, the dataset is expected to be a list of `RoomSelectionTask` objects: `[](https://microsoft.github.io/agent-lightning/stable/tutorials/write-agents/#__codelineno-1-1) from agentlightning import Dataset, Trainer [](https://microsoft.github.io/agent-lightning/stable/tutorials/write-agents/#__codelineno-1-2) [](https://microsoft.github.io/agent-lightning/stable/tutorials/write-agents/#__codelineno-1-3) dataset: Dataset[RoomSelectionTask] = [ [](https://microsoft.github.io/agent-lightning/stable/tutorials/write-agents/#__codelineno-1-4) RoomSelectionTask(date="2025-10-15", time="10:00", duration_min=60, attendees=10), [](https://microsoft.github.io/agent-lightning/stable/tutorials/write-agents/#__codelineno-1-5) RoomSelectionTask(date="2025-10-16", time="10:00", duration_min=60, attendees=10), [](https://microsoft.github.io/agent-lightning/stable/tutorials/write-agents/#__codelineno-1-6) ] [](https://microsoft.github.io/agent-lightning/stable/tutorials/write-agents/#__codelineno-1-7) [](https://microsoft.github.io/agent-lightning/stable/tutorials/write-agents/#__codelineno-1-8) Trainer().fit(agent=room_selector, train_dataset=dataset)` Behind the scenes, the [`@rollout`](https://microsoft.github.io/agent-lightning/stable/reference/agent/#agentlightning.rollout " agentlightning.rollout(func)") decorator wraps your function in a `FunctionalLitAgent` object, which is a subclass of [LitAgent](https://microsoft.github.io/agent-lightning/stable/reference/agent/#agentlightning.LitAgent " agentlightning.LitAgent") introduced below, making it compatible with the [Trainer](https://microsoft.github.io/agent-lightning/stable/reference/trainer/#agentlightning.Trainer " agentlightning.Trainer") and [Runner](https://microsoft.github.io/agent-lightning/stable/reference/runner/#agentlightning.Runner " agentlightning.Runner") . It supports parameters like `task`, `prompt_template`, `llm`, and `rollout`, giving you flexible access to the execution context. Here is another example with more advanced usage with `llm` and `rollout` as parameters. The `llm` parameter gives you an OpenAI-compatible LLM endpoint to interact with, which can be tuned under the hood by algorithms. The `rollout` parameter gives you the full [Rollout](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.Rollout " agentlightning.Rollout") object, which contains the rollout ID, rollout mode (training or validation), etc. ``[](https://microsoft.github.io/agent-lightning/stable/tutorials/write-agents/#__codelineno-2-1) from openai import OpenAI [](https://microsoft.github.io/agent-lightning/stable/tutorials/write-agents/#__codelineno-2-2) from agentlightning import LLM, Rollout [](https://microsoft.github.io/agent-lightning/stable/tutorials/write-agents/#__codelineno-2-3) [](https://microsoft.github.io/agent-lightning/stable/tutorials/write-agents/#__codelineno-2-4) class FlightBookingTask(TypedDict): [](https://microsoft.github.io/agent-lightning/stable/tutorials/write-agents/#__codelineno-2-5) request: str [](https://microsoft.github.io/agent-lightning/stable/tutorials/write-agents/#__codelineno-2-6) expected_booking: dict [](https://microsoft.github.io/agent-lightning/stable/tutorials/write-agents/#__codelineno-2-7) [](https://microsoft.github.io/agent-lightning/stable/tutorials/write-agents/#__codelineno-2-8) @rollout [](https://microsoft.github.io/agent-lightning/stable/tutorials/write-agents/#__codelineno-2-9) def flight_assistant(task: FlightBookingTask, llm: LLM, rollout: Rollout) -> float: [](https://microsoft.github.io/agent-lightning/stable/tutorials/write-agents/#__codelineno-2-10) print(f"Rollout ID: {rollout.rollout_id}") [](https://microsoft.github.io/agent-lightning/stable/tutorials/write-agents/#__codelineno-2-11) print(f"Rollout Mode: {rollout.mode}") [](https://microsoft.github.io/agent-lightning/stable/tutorials/write-agents/#__codelineno-2-12) [](https://microsoft.github.io/agent-lightning/stable/tutorials/write-agents/#__codelineno-2-13) # Use the tuned LLM resource to create an OpenAI client [](https://microsoft.github.io/agent-lightning/stable/tutorials/write-agents/#__codelineno-2-14) client = OpenAI( [](https://microsoft.github.io/agent-lightning/stable/tutorials/write-agents/#__codelineno-2-15) # This endpoint could be a proxy to a proxy to a proxy ... [](https://microsoft.github.io/agent-lightning/stable/tutorials/write-agents/#__codelineno-2-16) # It could be different every time `flight_assistant` is called [](https://microsoft.github.io/agent-lightning/stable/tutorials/write-agents/#__codelineno-2-17) # But it should be OpenAI-API compatible [](https://microsoft.github.io/agent-lightning/stable/tutorials/write-agents/#__codelineno-2-18) base_url=llm.endpoint, [](https://microsoft.github.io/agent-lightning/stable/tutorials/write-agents/#__codelineno-2-19) [](https://microsoft.github.io/agent-lightning/stable/tutorials/write-agents/#__codelineno-2-20) # Use a dummy key if not provided [](https://microsoft.github.io/agent-lightning/stable/tutorials/write-agents/#__codelineno-2-21) # Usually this does not matter because the training LLM is often not guarded by an API key [](https://microsoft.github.io/agent-lightning/stable/tutorials/write-agents/#__codelineno-2-22) # But you can use `or os.environ["OPENAI_API_KEY"]` to make the function compatible with 3rd-party LLMs [](https://microsoft.github.io/agent-lightning/stable/tutorials/write-agents/#__codelineno-2-23) api_key=llm.api_key or "dummy-key", [](https://microsoft.github.io/agent-lightning/stable/tutorials/write-agents/#__codelineno-2-24) ) [](https://microsoft.github.io/agent-lightning/stable/tutorials/write-agents/#__codelineno-2-25) [](https://microsoft.github.io/agent-lightning/stable/tutorials/write-agents/#__codelineno-2-26) # Make an API call with the specified model [](https://microsoft.github.io/agent-lightning/stable/tutorials/write-agents/#__codelineno-2-27) response = client.chat.completions.create( [](https://microsoft.github.io/agent-lightning/stable/tutorials/write-agents/#__codelineno-2-28) model=llm.model, [](https://microsoft.github.io/agent-lightning/stable/tutorials/write-agents/#__codelineno-2-29) messages=[{"role": "user", "content": task["request"]}], [](https://microsoft.github.io/agent-lightning/stable/tutorials/write-agents/#__codelineno-2-30) ) [](https://microsoft.github.io/agent-lightning/stable/tutorials/write-agents/#__codelineno-2-31) # Whether the API supports features like streaming, tool calls, etc. depends on [](https://microsoft.github.io/agent-lightning/stable/tutorials/write-agents/#__codelineno-2-32) # the endpoint that algorithms are serving to you. [](https://microsoft.github.io/agent-lightning/stable/tutorials/write-agents/#__codelineno-2-33) final_message = response.choices[0].message.content [](https://microsoft.github.io/agent-lightning/stable/tutorials/write-agents/#__codelineno-2-34) [](https://microsoft.github.io/agent-lightning/stable/tutorials/write-agents/#__codelineno-2-35) # Grade the result and return a reward [](https://microsoft.github.io/agent-lightning/stable/tutorials/write-agents/#__codelineno-2-36) reward = grade_flight_booking(final_message, task["expected_booking"]) [](https://microsoft.github.io/agent-lightning/stable/tutorials/write-agents/#__codelineno-2-37) return reward`` Return Values from Agents[¶](https://microsoft.github.io/agent-lightning/stable/tutorials/write-agents/#return-values-from-agents "Permanent link") ---------------------------------------------------------------------------------------------------------------------------------------------------- The value your agent function returns (i.e., the return value of the function decorated by [`@rollout`](https://microsoft.github.io/agent-lightning/stable/reference/agent/#agentlightning.rollout " agentlightning.rollout(func)") ) is crucial, as it's the primary way to report the outcome of a rollout. Agent-lightning supports several return types to accommodate different scenarios, from simple rewards to detailed, custom traces. * **`float`**: This is the simplest and most common return type. The `float` is treated as the **final reward** for the entire rollout. Agent-lightning automatically creates a final reward span based on this value. * **`None`**: Returning `None` tells the runner that trace collection is being handled entirely by the [Tracer](https://microsoft.github.io/agent-lightning/stable/reference/runner/#agentlightning.Tracer " agentlightning.Tracer") through auto-instrumentation (e.g., via AgentOps). In this case, the runner will simply retrieve the spans that the tracer has already captured. Emitting the Final Reward When returning `None`, you must still ensure a final reward is logged. You can do this by using the [`emit_reward`](https://microsoft.github.io/agent-lightning/stable/reference/agent/#agentlightning.emit_reward " agentlightning.emit_reward(reward, *, primary_key=None, attributes=None, propagate=True)") function (covered in the [Use Emitters](https://microsoft.github.io/agent-lightning/stable/tutorials/emitter/) documentation). Wrapping your reward calculation function with the `@reward` decorator is NOT the recommended approach any more. * **`list[ReadableSpan]`**, **`list[SpanCoreFields]`**, or **`list[Span]`**: For advanced use cases, you can manually construct and return a complete list of all spans for the rollout. This gives you full control over the trace data. You can return either a list of OpenTelemetry `ReadableSpan` objects or Agent-lightning's native `Span` objects. For most users, returning a **`float`** for simple agents or returning **`None`** and using the emitter for more complex ones are the recommended approaches. Class-based Agents[¶](https://microsoft.github.io/agent-lightning/stable/tutorials/write-agents/#class-based-agents "Permanent link") -------------------------------------------------------------------------------------------------------------------------------------- For more complex agents that require state, helper methods, or distinct logic for training versus validation, you can create a class that inherits from [`LitAgent`](https://microsoft.github.io/agent-lightning/stable/reference/agent/#agentlightning.LitAgent " agentlightning.LitAgent") . This object-oriented approach provides more structure and control over the agent's lifecycle. To create a class-based agent, you subclass [agentlightning.LitAgent](https://microsoft.github.io/agent-lightning/stable/reference/agent/#agentlightning.LitAgent " agentlightning.LitAgent") and implement its [`rollout`](https://microsoft.github.io/agent-lightning/stable/reference/agent/#agentlightning.LitAgent.rollout " rollout(task, resources, rollout)") method. [](https://microsoft.github.io/agent-lightning/stable/tutorials/write-agents/) Here's how the `room_selector` could be implemented as a class. The rollout method has a slightly different signature than the function-based agent, mainly in how it handles the resources. Putting it simply, algorithms do not just send a [PromptTemplate](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.PromptTemplate " agentlightning.PromptTemplate") to the agents, they instead send [NamedResources](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.NamedResources " agentlightning.NamedResources = Dict[str, ResourceUnion] module-attribute ") , which is a mapping from resource key to [Resource](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.Resource " agentlightning.Resource") . This design is to allow for more advanced features like multi-resource tuning. With [`@rollout`](https://microsoft.github.io/agent-lightning/stable/reference/agent/#agentlightning.rollout " agentlightning.rollout(func)") decorator, the resource with correctly matched type will be automatically injected into the rollout method. However, when you use a class-based agent, you need to manually access the resource from the `resources` dictionary. Built-in algorithms listed their resource key naming conventions [here](https://microsoft.github.io/agent-lightning/stable/algorithm-zoo/) . `[](https://microsoft.github.io/agent-lightning/stable/tutorials/write-agents/#__codelineno-3-1) import agentlightning as agl [](https://microsoft.github.io/agent-lightning/stable/tutorials/write-agents/#__codelineno-3-2) [](https://microsoft.github.io/agent-lightning/stable/tutorials/write-agents/#__codelineno-3-3) class RoomSelectorAgent(agl.LitAgent[RoomSelectionTask]): [](https://microsoft.github.io/agent-lightning/stable/tutorials/write-agents/#__codelineno-3-4) def rollout(self, task: RoomSelectionTask, resources: agl.NamedResources, rollout: agl.Rollout) -> float: [](https://microsoft.github.io/agent-lightning/stable/tutorials/write-agents/#__codelineno-3-5) # 1. Access the prompt_template from the resources dictionary [](https://microsoft.github.io/agent-lightning/stable/tutorials/write-agents/#__codelineno-3-6) prompt_template = resources["prompt_template"] [](https://microsoft.github.io/agent-lightning/stable/tutorials/write-agents/#__codelineno-3-7) [](https://microsoft.github.io/agent-lightning/stable/tutorials/write-agents/#__codelineno-3-8) # 2. Execute the agent's logic [](https://microsoft.github.io/agent-lightning/stable/tutorials/write-agents/#__codelineno-3-9) prompt = prompt_template.format(**task) [](https://microsoft.github.io/agent-lightning/stable/tutorials/write-agents/#__codelineno-3-10) # ... [](https://microsoft.github.io/agent-lightning/stable/tutorials/write-agents/#__codelineno-3-11) [](https://microsoft.github.io/agent-lightning/stable/tutorials/write-agents/#__codelineno-3-12) # 3. Grade the final choice [](https://microsoft.github.io/agent-lightning/stable/tutorials/write-agents/#__codelineno-3-13) reward = room_selection_grader(final_message, task["expected_choice"]) [](https://microsoft.github.io/agent-lightning/stable/tutorials/write-agents/#__codelineno-3-14) [](https://microsoft.github.io/agent-lightning/stable/tutorials/write-agents/#__codelineno-3-15) # 4. Return the final reward [](https://microsoft.github.io/agent-lightning/stable/tutorials/write-agents/#__codelineno-3-16) return reward [](https://microsoft.github.io/agent-lightning/stable/tutorials/write-agents/#__codelineno-3-17) [](https://microsoft.github.io/agent-lightning/stable/tutorials/write-agents/#__codelineno-3-18) # To use it with the trainer: [](https://microsoft.github.io/agent-lightning/stable/tutorials/write-agents/#__codelineno-3-19) # agent = RoomSelectorAgent() [](https://microsoft.github.io/agent-lightning/stable/tutorials/write-agents/#__codelineno-3-20) # trainer.fit(agent=agent, ...)` The [`LitAgent`](https://microsoft.github.io/agent-lightning/stable/reference/agent/#agentlightning.LitAgent " agentlightning.LitAgent") class provides several methods you can override for more fine-grained control: * [`rollout()`](https://microsoft.github.io/agent-lightning/stable/reference/agent/#agentlightning.LitAgent.rollout " rollout(task, resources, rollout)") : The primary method for the agent's logic. It's called for both training and validation by default. * [`training_rollout()`](https://microsoft.github.io/agent-lightning/stable/reference/agent/#agentlightning.LitAgent.training_rollout " training_rollout(task, resources, rollout)") / [`validation_rollout()`](https://microsoft.github.io/agent-lightning/stable/reference/agent/#agentlightning.LitAgent.validation_rollout " validation_rollout(task, resources, rollout)") : Implement these if you need different behavior during training (e.g., with exploration) and validation (e.g., with deterministic choices). * [`rollout_async()`](https://microsoft.github.io/agent-lightning/stable/reference/agent/#agentlightning.LitAgent.rollout_async " rollout_async(task, resources, rollout) async ") / [`training_rollout_async()`](https://microsoft.github.io/agent-lightning/stable/reference/agent/#agentlightning.LitAgent.training_rollout_async " training_rollout_async(task, resources, rollout) async ") / [`validation_rollout_async()`](https://microsoft.github.io/agent-lightning/stable/reference/agent/#agentlightning.LitAgent.validation_rollout_async " validation_rollout_async(task, resources, rollout) async ") : Implement the asynchronous versions of these methods if your agent uses `asyncio`. Note Rollout is always executed in an asynchronous context no matter whether the agent is asynchronous or synchronous. If your synchronous agent contains some `asyncio.run()` calls, it might raise an error that there is already an event loop running. To avoid blocking the event loop, it's recommended to offload the inner async operations to a separate thread. Here is a sample code: `[](https://microsoft.github.io/agent-lightning/stable/tutorials/write-agents/#__codelineno-4-1) import asyncio [](https://microsoft.github.io/agent-lightning/stable/tutorials/write-agents/#__codelineno-4-2) import queue [](https://microsoft.github.io/agent-lightning/stable/tutorials/write-agents/#__codelineno-4-3) import threading [](https://microsoft.github.io/agent-lightning/stable/tutorials/write-agents/#__codelineno-4-4) [](https://microsoft.github.io/agent-lightning/stable/tutorials/write-agents/#__codelineno-4-5) def run_sync_ephemeral(coro) -> Any: [](https://microsoft.github.io/agent-lightning/stable/tutorials/write-agents/#__codelineno-4-6) """ [](https://microsoft.github.io/agent-lightning/stable/tutorials/write-agents/#__codelineno-4-7) Run an async coroutine from sync code. [](https://microsoft.github.io/agent-lightning/stable/tutorials/write-agents/#__codelineno-4-8) - If no loop in this thread: use asyncio.run() directly. [](https://microsoft.github.io/agent-lightning/stable/tutorials/write-agents/#__codelineno-4-9) - If already in an event loop: spawn a worker thread that calls asyncio.run() [](https://microsoft.github.io/agent-lightning/stable/tutorials/write-agents/#__codelineno-4-10) (which creates and closes a brand-new event loop per call). [](https://microsoft.github.io/agent-lightning/stable/tutorials/write-agents/#__codelineno-4-11) """ [](https://microsoft.github.io/agent-lightning/stable/tutorials/write-agents/#__codelineno-4-12) try: [](https://microsoft.github.io/agent-lightning/stable/tutorials/write-agents/#__codelineno-4-13) asyncio.get_running_loop() [](https://microsoft.github.io/agent-lightning/stable/tutorials/write-agents/#__codelineno-4-14) except RuntimeError: [](https://microsoft.github.io/agent-lightning/stable/tutorials/write-agents/#__codelineno-4-15) # No running loop in this thread; safe to use asyncio.run [](https://microsoft.github.io/agent-lightning/stable/tutorials/write-agents/#__codelineno-4-16) return asyncio.run(coro) [](https://microsoft.github.io/agent-lightning/stable/tutorials/write-agents/#__codelineno-4-17) [](https://microsoft.github.io/agent-lightning/stable/tutorials/write-agents/#__codelineno-4-18) # Already in a running loop -> execute in a worker thread [](https://microsoft.github.io/agent-lightning/stable/tutorials/write-agents/#__codelineno-4-19) q = queue.Queue[Any]() [](https://microsoft.github.io/agent-lightning/stable/tutorials/write-agents/#__codelineno-4-20) [](https://microsoft.github.io/agent-lightning/stable/tutorials/write-agents/#__codelineno-4-21) def worker(): [](https://microsoft.github.io/agent-lightning/stable/tutorials/write-agents/#__codelineno-4-22) try: [](https://microsoft.github.io/agent-lightning/stable/tutorials/write-agents/#__codelineno-4-23) result = asyncio.run(coro) # creates & closes its own loop [](https://microsoft.github.io/agent-lightning/stable/tutorials/write-agents/#__codelineno-4-24) q.put((True, result)) [](https://microsoft.github.io/agent-lightning/stable/tutorials/write-agents/#__codelineno-4-25) except BaseException as e: [](https://microsoft.github.io/agent-lightning/stable/tutorials/write-agents/#__codelineno-4-26) q.put((False, e)) [](https://microsoft.github.io/agent-lightning/stable/tutorials/write-agents/#__codelineno-4-27) [](https://microsoft.github.io/agent-lightning/stable/tutorials/write-agents/#__codelineno-4-28) t = threading.Thread(target=worker, daemon=True) [](https://microsoft.github.io/agent-lightning/stable/tutorials/write-agents/#__codelineno-4-29) t.start() [](https://microsoft.github.io/agent-lightning/stable/tutorials/write-agents/#__codelineno-4-30) ok, payload = q.get() [](https://microsoft.github.io/agent-lightning/stable/tutorials/write-agents/#__codelineno-4-31) t.join() [](https://microsoft.github.io/agent-lightning/stable/tutorials/write-agents/#__codelineno-4-32) if ok: [](https://microsoft.github.io/agent-lightning/stable/tutorials/write-agents/#__codelineno-4-33) return payload [](https://microsoft.github.io/agent-lightning/stable/tutorials/write-agents/#__codelineno-4-34) raise payload` Back to top --- # Overview - Agent-lightning [Skip to content](https://microsoft.github.io/agent-lightning/stable/algorithm-zoo/#algorithm-zoo) ⚠️ You are viewing development documentation. For stable documentation, please visit [https://microsoft.github.io/agent-lightning/stable/](https://microsoft.github.io/agent-lightning/stable/) Algorithm Zoo[¶](https://microsoft.github.io/agent-lightning/stable/algorithm-zoo/#algorithm-zoo "Permanent link") =================================================================================================================== AgentLightning includes several popular and frequently requested algorithms in its built-in library, allowing agent developers to use them directly. These algorithms are designed to be compatible with most agent scenarios. For customizing algorithms, see [Algorithm-side References](https://microsoft.github.io/agent-lightning/stable/reference/algorithm/) . | Algorithm | Optimizing Resources | Description | | --- | --- | --- | | [APO](https://microsoft.github.io/agent-lightning/stable/algorithm-zoo/apo/) | `{: [PromptTemplate][agentlightning.PromptTemplate]}` | Automatic Prompt Optimization (APO) algorithm using textual gradients and beam search. | | [VERL](https://microsoft.github.io/agent-lightning/stable/algorithm-zoo/verl/) | `{"main_llm": [LLM][agentlightning.LLM]}` | Reinforcement Learning with [VERL framework](https://github.com/volcengine/verl)
. | Back to top --- # SFT with Unsloth - Agent-lightning [Skip to content](https://microsoft.github.io/agent-lightning/stable/how-to/unsloth-sft/#fine-tune-with-unsloth-sft) ⚠️ You are viewing development documentation. For stable documentation, please visit [https://microsoft.github.io/agent-lightning/stable/](https://microsoft.github.io/agent-lightning/stable/) Fine-tune with Unsloth SFT[¶](https://microsoft.github.io/agent-lightning/stable/how-to/unsloth-sft/#fine-tune-with-unsloth-sft "Permanent link") ================================================================================================================================================== Prerequisites Please make sure you have read [Write the First Algorithm](https://microsoft.github.io/agent-lightning/stable/how-to/write-first-algorithm/) . Although that recipe is based on a simple prompt tuning algorithm, it introduces the core concepts of Agent-lightning and you should be familiar with them before proceeding. This recipe builds on [Write the First Algorithm](https://microsoft.github.io/agent-lightning/stable/how-to/write-first-algorithm/) . Instead of iterating on a prompt, we will fine-tune a large language model with [Unsloth](https://docs.unsloth.ai/) 's SFT Trainer and keep the whole loop inside Agent-lightning. The new pieces you will meet are the **LLM proxy**, the **trace-to-triplet adapter**, a [vLLM](https://github.com/vllm-project/vllm) inference endpoint, and an agent implemented with the [OpenAI Agents SDK](https://openai.github.io/openai-agents-python/) . The full sample code is available in the [`examples/unsloth`](https://github.com/microsoft/agent-lightning/tree/3b5d733861cf313fc09821a23240bbdf3cb2ee5b/examples/unsloth) folder. Warning You need a GPU that can host the Unsloth base model and run vLLM. The sample defaults to `unsloth/Qwen3-4B-Instruct-2507`, which requires at least 16GB of GPU memory under 4-bit quantization. The Data and Serving Loop[¶](https://microsoft.github.io/agent-lightning/stable/how-to/unsloth-sft/#the-data-and-serving-loop "Permanent link") ------------------------------------------------------------------------------------------------------------------------------------------------ To tune a large language model in Supervised Fine-Tuning (SFT), we commonly need a dataset with input/output samples. For example, the [TRL SFT Trainer](https://huggingface.co/docs/trl/sft_trainer) expects a dataset with samples like the following: `[](https://microsoft.github.io/agent-lightning/stable/how-to/unsloth-sft/#__codelineno-0-1) {"messages": [{"role": "user", "content": "What color is the sky?"}, [](https://microsoft.github.io/agent-lightning/stable/how-to/unsloth-sft/#__codelineno-0-2) {"role": "assistant", "content": "It is blue."}]}` With supervised fine-tuning, the LLM learns to generate the "assistant" response as close as possible to the completion in the dataset. Typically, the dataset used in SFT should be a curated set of samples. The samples can be either hand-written by humans, or generated by a more powerful model, which is known as [data distillation](https://docs.nvidia.com/nemo-framework/user-guide/24.12/modelalignment/knowledge-distillation.html) . However, in this recipe, we use a different setup that relies on samples generated by the model itself. We use the reward emitted by the agent to select the top-performing samples. Overall, the flow of the algorithm is an iteration of the following steps: 1. Serve the current checkpoint (with vLLM). 2. Publish the vLLM endpoint through the LLM proxy and let runners roll out some tasks with the current model. 3. Collect the traces from the rollouts and transform the highest-rewarded ones into a dataset that is acceptable for Unsloth SFT Trainer. 4. Launch Unsloth to fine-tune on the dataset and save a new checkpoint. You will find the full source code of this iteration in `sft_one_iter` in [sft\_algorithm.py](https://github.com/microsoft/agent-lightning/tree/3b5d733861cf313fc09821a23240bbdf3cb2ee5b/examples/unsloth/sft_algorithm.py) . We will elaborate on each part below. ### Serving the Model with vLLM and Proxy[¶](https://microsoft.github.io/agent-lightning/stable/how-to/unsloth-sft/#serving-the-model-with-vllm-and-proxy "Permanent link") Most modern agents do not use the model directly; instead, they use an API like the OpenAI chat completions API to interact with the model. Therefore, we need a vLLM-based inference server launched before rollouts. The serving code looks like the following. See the `vllm_server` function in [sft\_algorithm.py](https://github.com/microsoft/agent-lightning/tree/3b5d733861cf313fc09821a23240bbdf3cb2ee5b/examples/unsloth/sft_algorithm.py) if you want to see a more robust version. `[](https://microsoft.github.io/agent-lightning/stable/how-to/unsloth-sft/#__codelineno-1-1) from openai import OpenAI [](https://microsoft.github.io/agent-lightning/stable/how-to/unsloth-sft/#__codelineno-1-2) [](https://microsoft.github.io/agent-lightning/stable/how-to/unsloth-sft/#__codelineno-1-3) vllm_process = subprocess.Popen([ [](https://microsoft.github.io/agent-lightning/stable/how-to/unsloth-sft/#__codelineno-1-4) "vllm", "serve", model_path, "--port", str(port), [](https://microsoft.github.io/agent-lightning/stable/how-to/unsloth-sft/#__codelineno-1-5) "--enable-auto-tool-choice", "--tool-call-parser", "hermes" [](https://microsoft.github.io/agent-lightning/stable/how-to/unsloth-sft/#__codelineno-1-6) ]) [](https://microsoft.github.io/agent-lightning/stable/how-to/unsloth-sft/#__codelineno-1-7) [](https://microsoft.github.io/agent-lightning/stable/how-to/unsloth-sft/#__codelineno-1-8) # Wait for the server to be ready [](https://microsoft.github.io/agent-lightning/stable/how-to/unsloth-sft/#__codelineno-1-9) url = f"http://localhost:{port}/health" [](https://microsoft.github.io/agent-lightning/stable/how-to/unsloth-sft/#__codelineno-1-10) start = time.time() [](https://microsoft.github.io/agent-lightning/stable/how-to/unsloth-sft/#__codelineno-1-11) client = httpx.Client() [](https://microsoft.github.io/agent-lightning/stable/how-to/unsloth-sft/#__codelineno-1-12) [](https://microsoft.github.io/agent-lightning/stable/how-to/unsloth-sft/#__codelineno-1-13) while True: [](https://microsoft.github.io/agent-lightning/stable/how-to/unsloth-sft/#__codelineno-1-14) if client.get(url).status_code == 200: [](https://microsoft.github.io/agent-lightning/stable/how-to/unsloth-sft/#__codelineno-1-15) break [](https://microsoft.github.io/agent-lightning/stable/how-to/unsloth-sft/#__codelineno-1-16) [](https://microsoft.github.io/agent-lightning/stable/how-to/unsloth-sft/#__codelineno-1-17) server_address = f"http://localhost:{port}/v1" [](https://microsoft.github.io/agent-lightning/stable/how-to/unsloth-sft/#__codelineno-1-18) [](https://microsoft.github.io/agent-lightning/stable/how-to/unsloth-sft/#__codelineno-1-19) # Try using the vLLM server [](https://microsoft.github.io/agent-lightning/stable/how-to/unsloth-sft/#__codelineno-1-20) openai = OpenAI(base_url=server_address) [](https://microsoft.github.io/agent-lightning/stable/how-to/unsloth-sft/#__codelineno-1-21) ...` In this recipe, we do not expose the server address directly to the agent runners, because we want to install a "middleware" to collect the prompts and responses of all the requests. In general, it's up to you to decide whether to hide the vLLM server behind a proxy or not. The "middleware" here is [`LLMProxy`](https://microsoft.github.io/agent-lightning/stable/reference/algorithm/#agentlightning.LLMProxy " agentlightning.LLMProxy") , which is an independent [LiteLLM](https://docs.litellm.ai/) server that forwards the requests to the vLLM server. It also exposes an OpenAI-compatible API that the runners can target without caring about where the model lives. The benefits of using the proxy are: 1. **Traces:** The proxy automatically logs the prompts and responses of all the requests into the store. 2. **Token IDs:** The proxy augments the requests so that the vLLM server can return the prompt and response token IDs (see more details in [Serving LLM](https://microsoft.github.io/agent-lightning/stable/deep-dive/serving-llm/) ). The [`LLMProxy`](https://microsoft.github.io/agent-lightning/stable/reference/algorithm/#agentlightning.LLMProxy " agentlightning.LLMProxy") accepts a list of model configurations, in the same syntax as LiteLLM's [`model_list`](https://docs.litellm.ai/docs/proxy/configs) . Include a `hosted_vllm/` prefix to the models to activate LiteLLM's [vLLM integration](https://docs.litellm.ai/docs/providers/vllm) . `[](https://microsoft.github.io/agent-lightning/stable/how-to/unsloth-sft/#__codelineno-2-1) import agentlightning as agl [](https://microsoft.github.io/agent-lightning/stable/how-to/unsloth-sft/#__codelineno-2-2) [](https://microsoft.github.io/agent-lightning/stable/how-to/unsloth-sft/#__codelineno-2-3) llm_proxy = agl.LLMProxy(port=port, store=store) [](https://microsoft.github.io/agent-lightning/stable/how-to/unsloth-sft/#__codelineno-2-4) model_list = [ [](https://microsoft.github.io/agent-lightning/stable/how-to/unsloth-sft/#__codelineno-2-5) { [](https://microsoft.github.io/agent-lightning/stable/how-to/unsloth-sft/#__codelineno-2-6) "model_name": "Qwen3-4B-Instruct", [](https://microsoft.github.io/agent-lightning/stable/how-to/unsloth-sft/#__codelineno-2-7) "litellm_params": {"model": f"hosted_vllm/{model_path}", "api_base": server_address}, [](https://microsoft.github.io/agent-lightning/stable/how-to/unsloth-sft/#__codelineno-2-8) } [](https://microsoft.github.io/agent-lightning/stable/how-to/unsloth-sft/#__codelineno-2-9) ] [](https://microsoft.github.io/agent-lightning/stable/how-to/unsloth-sft/#__codelineno-2-10) llm_proxy.update_model_list(model_list) [](https://microsoft.github.io/agent-lightning/stable/how-to/unsloth-sft/#__codelineno-2-11) # If the proxy is not running, it will start automatically. [](https://microsoft.github.io/agent-lightning/stable/how-to/unsloth-sft/#__codelineno-2-12) await llm_proxy.restart() [](https://microsoft.github.io/agent-lightning/stable/how-to/unsloth-sft/#__codelineno-2-13) # Add the proxy as a resource to the store so that the runners can access it via URL. [](https://microsoft.github.io/agent-lightning/stable/how-to/unsloth-sft/#__codelineno-2-14) resource_update = await store.add_resources({"main_llm": llm_proxy.as_resource()})` ### Spawn Rollout and Collect Spans[¶](https://microsoft.github.io/agent-lightning/stable/how-to/unsloth-sft/#spawn-rollout-and-collect-spans "Permanent link") Once the proxy is registered as a resource, the algorithm schedules work for the rollout runners. Each problem from a training dataset becomes a rollout with the proxy baked into its resources: `[](https://microsoft.github.io/agent-lightning/stable/how-to/unsloth-sft/#__codelineno-3-1) rollouts: list[Rollout] = [] [](https://microsoft.github.io/agent-lightning/stable/how-to/unsloth-sft/#__codelineno-3-2) for sample in train_dataset: [](https://microsoft.github.io/agent-lightning/stable/how-to/unsloth-sft/#__codelineno-3-3) rollouts.append( [](https://microsoft.github.io/agent-lightning/stable/how-to/unsloth-sft/#__codelineno-3-4) await store.enqueue_rollout( [](https://microsoft.github.io/agent-lightning/stable/how-to/unsloth-sft/#__codelineno-3-5) input=sample, [](https://microsoft.github.io/agent-lightning/stable/how-to/unsloth-sft/#__codelineno-3-6) mode="train", [](https://microsoft.github.io/agent-lightning/stable/how-to/unsloth-sft/#__codelineno-3-7) resources_id=resources_update.resources_id, [](https://microsoft.github.io/agent-lightning/stable/how-to/unsloth-sft/#__codelineno-3-8) ) [](https://microsoft.github.io/agent-lightning/stable/how-to/unsloth-sft/#__codelineno-3-9) )` `resources_id` ties every rollout to the `main_llm` proxy resource we just uploaded. The runners on the other side poll the store ([`LitAgentRunner.iter()`](https://microsoft.github.io/agent-lightning/stable/reference/runner/#agentlightning.LitAgentRunner.iter " iter(*, event=None) async ") ) and execute the agent for each rollout. On the algorithm side we wait for completions with a non-blocking polling loop: `[](https://microsoft.github.io/agent-lightning/stable/how-to/unsloth-sft/#__codelineno-4-1) completed_rollouts: list[Rollout] = [] [](https://microsoft.github.io/agent-lightning/stable/how-to/unsloth-sft/#__codelineno-4-2) while True: [](https://microsoft.github.io/agent-lightning/stable/how-to/unsloth-sft/#__codelineno-4-3) completed_rollouts = await store.wait_for_rollouts( [](https://microsoft.github.io/agent-lightning/stable/how-to/unsloth-sft/#__codelineno-4-4) rollout_ids=[r.rollout_id for r in rollouts], [](https://microsoft.github.io/agent-lightning/stable/how-to/unsloth-sft/#__codelineno-4-5) timeout=0.0, [](https://microsoft.github.io/agent-lightning/stable/how-to/unsloth-sft/#__codelineno-4-6) ) [](https://microsoft.github.io/agent-lightning/stable/how-to/unsloth-sft/#__codelineno-4-7) if len(completed_rollouts) == len(rollouts): [](https://microsoft.github.io/agent-lightning/stable/how-to/unsloth-sft/#__codelineno-4-8) break [](https://microsoft.github.io/agent-lightning/stable/how-to/unsloth-sft/#__codelineno-4-9) await asyncio.sleep(5.0)` Note The `timeout=0.0` is needed here because this example uses a [`LightningStoreClient`](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.LightningStoreClient " agentlightning.LightningStoreClient") , and `wait_for_rollouts` establishes an HTTP connection to that store. Currently, only non-blocking wait requests are supported, which avoids holding the store connection open. Once the rollouts complete, we terminate the vLLM server to free up GPU memory. `[](https://microsoft.github.io/agent-lightning/stable/how-to/unsloth-sft/#__codelineno-5-1) vllm_process.terminate() [](https://microsoft.github.io/agent-lightning/stable/how-to/unsloth-sft/#__codelineno-5-2) vllm_process.join(timeout=10.0)` ### Adapt the Spans to HuggingFace Dataset[¶](https://microsoft.github.io/agent-lightning/stable/how-to/unsloth-sft/#adapt-the-spans-to-huggingface-dataset "Permanent link") [`LlmProxyTraceToTriplet`](https://microsoft.github.io/agent-lightning/stable/reference/algorithm/#agentlightning.LlmProxyTraceToTriplet " agentlightning.LlmProxyTraceToTriplet") converts the proxy’s spans (which might be dozens to hundreds per rollout) into [`Triplet`](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.Triplet " agentlightning.Triplet") objects that contain prompt/response token IDs plus an optional reward. The adapter may return multiple triplets per rollout (one per chat-completion call). To bias training toward successful reasoning chains the algorithm walks the triplets in reverse order, keeps the most recent reward, and turns each prompt/response pair into Hugging Face dataset rows: `[](https://microsoft.github.io/agent-lightning/stable/how-to/unsloth-sft/#__codelineno-6-1) all_triplets = [] [](https://microsoft.github.io/agent-lightning/stable/how-to/unsloth-sft/#__codelineno-6-2) data_adapter = agl.LlmProxyTraceToTriplet() [](https://microsoft.github.io/agent-lightning/stable/how-to/unsloth-sft/#__codelineno-6-3) [](https://microsoft.github.io/agent-lightning/stable/how-to/unsloth-sft/#__codelineno-6-4) for rollout in completed_rollouts: [](https://microsoft.github.io/agent-lightning/stable/how-to/unsloth-sft/#__codelineno-6-5) spans = await store.query_spans(rollout.rollout_id, "latest") [](https://microsoft.github.io/agent-lightning/stable/how-to/unsloth-sft/#__codelineno-6-6) triplets = data_adapter.adapt(spans) [](https://microsoft.github.io/agent-lightning/stable/how-to/unsloth-sft/#__codelineno-6-7) [](https://microsoft.github.io/agent-lightning/stable/how-to/unsloth-sft/#__codelineno-6-8) recent_reward = None [](https://microsoft.github.io/agent-lightning/stable/how-to/unsloth-sft/#__codelineno-6-9) for triplet in reversed(triplets): [](https://microsoft.github.io/agent-lightning/stable/how-to/unsloth-sft/#__codelineno-6-10) if triplet.reward is not None: [](https://microsoft.github.io/agent-lightning/stable/how-to/unsloth-sft/#__codelineno-6-11) recent_reward = triplet.reward [](https://microsoft.github.io/agent-lightning/stable/how-to/unsloth-sft/#__codelineno-6-12) if recent_reward is None: [](https://microsoft.github.io/agent-lightning/stable/how-to/unsloth-sft/#__codelineno-6-13) continue [](https://microsoft.github.io/agent-lightning/stable/how-to/unsloth-sft/#__codelineno-6-14) [](https://microsoft.github.io/agent-lightning/stable/how-to/unsloth-sft/#__codelineno-6-15) input_ids = triplet.prompt["token_ids"] + triplet.response["token_ids"] [](https://microsoft.github.io/agent-lightning/stable/how-to/unsloth-sft/#__codelineno-6-16) # We don't train on prompt tokens, so they are masked out by setting to -100. [](https://microsoft.github.io/agent-lightning/stable/how-to/unsloth-sft/#__codelineno-6-17) labels = [-100] * len(triplet.prompt["token_ids"]) + triplet.response["token_ids"] [](https://microsoft.github.io/agent-lightning/stable/how-to/unsloth-sft/#__codelineno-6-18) # This matches the dataset format required by the Unsloth SFT trainer. [](https://microsoft.github.io/agent-lightning/stable/how-to/unsloth-sft/#__codelineno-6-19) all_triplets.append( [](https://microsoft.github.io/agent-lightning/stable/how-to/unsloth-sft/#__codelineno-6-20) { [](https://microsoft.github.io/agent-lightning/stable/how-to/unsloth-sft/#__codelineno-6-21) "input_ids": input_ids, [](https://microsoft.github.io/agent-lightning/stable/how-to/unsloth-sft/#__codelineno-6-22) "attention_mask": [1] * len(input_ids), [](https://microsoft.github.io/agent-lightning/stable/how-to/unsloth-sft/#__codelineno-6-23) "labels": labels, [](https://microsoft.github.io/agent-lightning/stable/how-to/unsloth-sft/#__codelineno-6-24) "reward": recent_reward, [](https://microsoft.github.io/agent-lightning/stable/how-to/unsloth-sft/#__codelineno-6-25) } [](https://microsoft.github.io/agent-lightning/stable/how-to/unsloth-sft/#__codelineno-6-26) )` Note You might notice that the dataset format used here differs from the format described in the **SFT Trainer** documentation. According to the documentation, dataset samples should be provided as plain text strings or message objects. As a matter of fact, this example leverages some [undocumented behavior](https://github.com/huggingface/trl/blob/e0eec055b412c48ad754149c475a87a8fca34fb4/trl/trainer/sft_trainer.py#L887) in the SFT Trainer implementation. When the dataset already includes a `"input_ids"` column, the Trainer automatically marks it as `is_processed` and skips the internal tokenization step. Since we already have spans with token IDs generated by the [`LLMProxy`](https://microsoft.github.io/agent-lightning/stable/reference/algorithm/#agentlightning.LLMProxy " agentlightning.LLMProxy") , providing them directly avoids unnecessary [**re-tokenization** and related complications](https://microsoft.github.io/agent-lightning/stable/deep-dive/serving-llm/) . This approach will both save processing time and increase consistency between training and inference. After aggregating every rollout we shuffle, sort by reward, and keep the top fraction (e.g., 50%) before shuffling again. The resulting list feeds directly into `datasets.Dataset.from_list`, which is the format Unsloth’s SFT trainer expects. `[](https://microsoft.github.io/agent-lightning/stable/how-to/unsloth-sft/#__codelineno-7-1) from datasets import Dataset as HuggingFaceDataset [](https://microsoft.github.io/agent-lightning/stable/how-to/unsloth-sft/#__codelineno-7-2) [](https://microsoft.github.io/agent-lightning/stable/how-to/unsloth-sft/#__codelineno-7-3) random.shuffle(all_triplets) [](https://microsoft.github.io/agent-lightning/stable/how-to/unsloth-sft/#__codelineno-7-4) all_triplets.sort(key=lambda x: x["reward"], reverse=True) [](https://microsoft.github.io/agent-lightning/stable/how-to/unsloth-sft/#__codelineno-7-5) sliced_triplets = all_triplets[: max(1, int(len(all_triplets) * triplet_fraction))] [](https://microsoft.github.io/agent-lightning/stable/how-to/unsloth-sft/#__codelineno-7-6) # Shuffle the sliced triplets again [](https://microsoft.github.io/agent-lightning/stable/how-to/unsloth-sft/#__codelineno-7-7) random.shuffle(sliced_triplets) [](https://microsoft.github.io/agent-lightning/stable/how-to/unsloth-sft/#__codelineno-7-8) [](https://microsoft.github.io/agent-lightning/stable/how-to/unsloth-sft/#__codelineno-7-9) sft_dataset = HuggingFaceDataset.from_list(sliced_triplets)` ### Launch Unsloth Training[¶](https://microsoft.github.io/agent-lightning/stable/how-to/unsloth-sft/#launch-unsloth-training "Permanent link") The heavy lifting happens in [`trl.SFTTrainer`](https://huggingface.co/docs/trl/sft_trainer) (see [unsloth\_helper.py](https://github.com/microsoft/agent-lightning/tree/3b5d733861cf313fc09821a23240bbdf3cb2ee5b/examples/unsloth/unsloth_helper.py) on how it's used). We launch it in a fresh process created with `multiprocessing.get_context("spawn")` so CUDA memory is reliably reclaimed when training ends. Launching it in the same process will also work for the first iteration, but we found that the memory won't be freed properly for subsequent vLLM serving. `[](https://microsoft.github.io/agent-lightning/stable/how-to/unsloth-sft/#__codelineno-8-1) context = multiprocessing.get_context("spawn") [](https://microsoft.github.io/agent-lightning/stable/how-to/unsloth-sft/#__codelineno-8-2) unsloth_process = context.Process( [](https://microsoft.github.io/agent-lightning/stable/how-to/unsloth-sft/#__codelineno-8-3) target=unsloth_training, [](https://microsoft.github.io/agent-lightning/stable/how-to/unsloth-sft/#__codelineno-8-4) args=(model_path, sft_dataset, next_model_path), [](https://microsoft.github.io/agent-lightning/stable/how-to/unsloth-sft/#__codelineno-8-5) daemon=True, [](https://microsoft.github.io/agent-lightning/stable/how-to/unsloth-sft/#__codelineno-8-6) ) [](https://microsoft.github.io/agent-lightning/stable/how-to/unsloth-sft/#__codelineno-8-7) unsloth_process.start() [](https://microsoft.github.io/agent-lightning/stable/how-to/unsloth-sft/#__codelineno-8-8) unsloth_process.join(timeout=600.0)` Inside the `unsloth_training` subprocess, Unsloth loads the previous checkpoint in 4-bit, applies LoRA adapters, and forwards the Hugging Face dataset to [`trl.SFTTrainer`](https://huggingface.co/docs/trl/sft_trainer) with the configuration defined in [`SFTConfig`](https://huggingface.co/docs/trl/sft_trainer#trl.SFTConfig) (batch size, accumulation steps, learning rate, etc.). The merged 16-bit weights are saved under `models/version_` so the next iteration can immediately serve them with vLLM. `[](https://microsoft.github.io/agent-lightning/stable/how-to/unsloth-sft/#__codelineno-9-1) from unsloth import FastLanguageModel [](https://microsoft.github.io/agent-lightning/stable/how-to/unsloth-sft/#__codelineno-9-2) # TRL is patched by unsloth. [](https://microsoft.github.io/agent-lightning/stable/how-to/unsloth-sft/#__codelineno-9-3) from trl import SFTConfig, SFTTrainer [](https://microsoft.github.io/agent-lightning/stable/how-to/unsloth-sft/#__codelineno-9-4) [](https://microsoft.github.io/agent-lightning/stable/how-to/unsloth-sft/#__codelineno-9-5) model, tokenizer = FastLanguageModel.from_pretrained( [](https://microsoft.github.io/agent-lightning/stable/how-to/unsloth-sft/#__codelineno-9-6) model_name=model_path, [](https://microsoft.github.io/agent-lightning/stable/how-to/unsloth-sft/#__codelineno-9-7) load_in_4bit=True, # 4 bit quantization to reduce memory [](https://microsoft.github.io/agent-lightning/stable/how-to/unsloth-sft/#__codelineno-9-8) ) [](https://microsoft.github.io/agent-lightning/stable/how-to/unsloth-sft/#__codelineno-9-9) [](https://microsoft.github.io/agent-lightning/stable/how-to/unsloth-sft/#__codelineno-9-10) # Config the model to use LoRA [](https://microsoft.github.io/agent-lightning/stable/how-to/unsloth-sft/#__codelineno-9-11) model = FastLanguageModel.get_peft_model( [](https://microsoft.github.io/agent-lightning/stable/how-to/unsloth-sft/#__codelineno-9-12) model, [](https://microsoft.github.io/agent-lightning/stable/how-to/unsloth-sft/#__codelineno-9-13) r=32, [](https://microsoft.github.io/agent-lightning/stable/how-to/unsloth-sft/#__codelineno-9-14) ... [](https://microsoft.github.io/agent-lightning/stable/how-to/unsloth-sft/#__codelineno-9-15) ) [](https://microsoft.github.io/agent-lightning/stable/how-to/unsloth-sft/#__codelineno-9-16) [](https://microsoft.github.io/agent-lightning/stable/how-to/unsloth-sft/#__codelineno-9-17) trainer = SFTTrainer( [](https://microsoft.github.io/agent-lightning/stable/how-to/unsloth-sft/#__codelineno-9-18) model=model, [](https://microsoft.github.io/agent-lightning/stable/how-to/unsloth-sft/#__codelineno-9-19) tokenizer=tokenizer, [](https://microsoft.github.io/agent-lightning/stable/how-to/unsloth-sft/#__codelineno-9-20) train_dataset=sft_dataset, [](https://microsoft.github.io/agent-lightning/stable/how-to/unsloth-sft/#__codelineno-9-21) ... [](https://microsoft.github.io/agent-lightning/stable/how-to/unsloth-sft/#__codelineno-9-22) ) [](https://microsoft.github.io/agent-lightning/stable/how-to/unsloth-sft/#__codelineno-9-23) [](https://microsoft.github.io/agent-lightning/stable/how-to/unsloth-sft/#__codelineno-9-24) # This is the heaviest step. [](https://microsoft.github.io/agent-lightning/stable/how-to/unsloth-sft/#__codelineno-9-25) trainer_stats = trainer.train() [](https://microsoft.github.io/agent-lightning/stable/how-to/unsloth-sft/#__codelineno-9-26) [](https://microsoft.github.io/agent-lightning/stable/how-to/unsloth-sft/#__codelineno-9-27) # Save in 16-bit for vLLM inference later [](https://microsoft.github.io/agent-lightning/stable/how-to/unsloth-sft/#__codelineno-9-28) model.save_pretrained_merged(next_model_path, tokenizer, save_method="merged_16bit")` Math Agent: OpenAI Agents SDK with MCP[¶](https://microsoft.github.io/agent-lightning/stable/how-to/unsloth-sft/#math-agent-openai-agents-sdk-with-mcp "Permanent link") ------------------------------------------------------------------------------------------------------------------------------------------------------------------------- We build an agent with the [OpenAI Agents SDK](https://openai.github.io/openai-agents-python/) to wire a calculator MCP tool and an OpenAI-compatible chat completion model together. The agent aims to solve a math problem and returns a reward indicating whether the answer is correct or not. The runner injects the `LLM` resource supplied by the algorithm side: `[](https://microsoft.github.io/agent-lightning/stable/how-to/unsloth-sft/#__codelineno-10-1) import os [](https://microsoft.github.io/agent-lightning/stable/how-to/unsloth-sft/#__codelineno-10-2) from typing import TypedDict [](https://microsoft.github.io/agent-lightning/stable/how-to/unsloth-sft/#__codelineno-10-3) [](https://microsoft.github.io/agent-lightning/stable/how-to/unsloth-sft/#__codelineno-10-4) import agentlightning as agl [](https://microsoft.github.io/agent-lightning/stable/how-to/unsloth-sft/#__codelineno-10-5) from agents import Agent, ModelSettings, OpenAIChatCompletionsModel, Runner as OpenAIRunner [](https://microsoft.github.io/agent-lightning/stable/how-to/unsloth-sft/#__codelineno-10-6) from agents.mcp import MCPServerStdio [](https://microsoft.github.io/agent-lightning/stable/how-to/unsloth-sft/#__codelineno-10-7) from openai import AsyncOpenAI [](https://microsoft.github.io/agent-lightning/stable/how-to/unsloth-sft/#__codelineno-10-8) [](https://microsoft.github.io/agent-lightning/stable/how-to/unsloth-sft/#__codelineno-10-9) class GsmProblem(TypedDict): [](https://microsoft.github.io/agent-lightning/stable/how-to/unsloth-sft/#__codelineno-10-10) input: str [](https://microsoft.github.io/agent-lightning/stable/how-to/unsloth-sft/#__codelineno-10-11) target: float [](https://microsoft.github.io/agent-lightning/stable/how-to/unsloth-sft/#__codelineno-10-12) [](https://microsoft.github.io/agent-lightning/stable/how-to/unsloth-sft/#__codelineno-10-13) def compute_reward(result: str, target: float) -> float: [](https://microsoft.github.io/agent-lightning/stable/how-to/unsloth-sft/#__codelineno-10-14) ... [](https://microsoft.github.io/agent-lightning/stable/how-to/unsloth-sft/#__codelineno-10-15) [](https://microsoft.github.io/agent-lightning/stable/how-to/unsloth-sft/#__codelineno-10-16) @agl.rollout [](https://microsoft.github.io/agent-lightning/stable/how-to/unsloth-sft/#__codelineno-10-17) async def math_agent(task: GsmProblem, llm: agl.LLM) -> float: [](https://microsoft.github.io/agent-lightning/stable/how-to/unsloth-sft/#__codelineno-10-18) async with MCPServerStdio( [](https://microsoft.github.io/agent-lightning/stable/how-to/unsloth-sft/#__codelineno-10-19) name="Calculator via uvx", [](https://microsoft.github.io/agent-lightning/stable/how-to/unsloth-sft/#__codelineno-10-20) params={"command": "uvx", "args": ["mcp-server-calculator"]}, [](https://microsoft.github.io/agent-lightning/stable/how-to/unsloth-sft/#__codelineno-10-21) ) as server: [](https://microsoft.github.io/agent-lightning/stable/how-to/unsloth-sft/#__codelineno-10-22) agent = Agent( [](https://microsoft.github.io/agent-lightning/stable/how-to/unsloth-sft/#__codelineno-10-23) name="Assistant", [](https://microsoft.github.io/agent-lightning/stable/how-to/unsloth-sft/#__codelineno-10-24) instructions=( [](https://microsoft.github.io/agent-lightning/stable/how-to/unsloth-sft/#__codelineno-10-25) "Use the calculator tool for every question. " [](https://microsoft.github.io/agent-lightning/stable/how-to/unsloth-sft/#__codelineno-10-26) "Return only the numeric answer wrapped like ### ###." [](https://microsoft.github.io/agent-lightning/stable/how-to/unsloth-sft/#__codelineno-10-27) ), [](https://microsoft.github.io/agent-lightning/stable/how-to/unsloth-sft/#__codelineno-10-28) mcp_servers=[server], [](https://microsoft.github.io/agent-lightning/stable/how-to/unsloth-sft/#__codelineno-10-29) model=OpenAIChatCompletionsModel( [](https://microsoft.github.io/agent-lightning/stable/how-to/unsloth-sft/#__codelineno-10-30) model=llm.model, [](https://microsoft.github.io/agent-lightning/stable/how-to/unsloth-sft/#__codelineno-10-31) openai_client=AsyncOpenAI( [](https://microsoft.github.io/agent-lightning/stable/how-to/unsloth-sft/#__codelineno-10-32) base_url=llm.endpoint, [](https://microsoft.github.io/agent-lightning/stable/how-to/unsloth-sft/#__codelineno-10-33) api_key=llm.api_key or "dummy", [](https://microsoft.github.io/agent-lightning/stable/how-to/unsloth-sft/#__codelineno-10-34) ), [](https://microsoft.github.io/agent-lightning/stable/how-to/unsloth-sft/#__codelineno-10-35) ), [](https://microsoft.github.io/agent-lightning/stable/how-to/unsloth-sft/#__codelineno-10-36) model_settings=ModelSettings( [](https://microsoft.github.io/agent-lightning/stable/how-to/unsloth-sft/#__codelineno-10-37) temperature=llm.sampling_parameters.get("temperature", 0.0), [](https://microsoft.github.io/agent-lightning/stable/how-to/unsloth-sft/#__codelineno-10-38) ), [](https://microsoft.github.io/agent-lightning/stable/how-to/unsloth-sft/#__codelineno-10-39) ) [](https://microsoft.github.io/agent-lightning/stable/how-to/unsloth-sft/#__codelineno-10-40) result = await OpenAIRunner.run(agent, task["input"]) [](https://microsoft.github.io/agent-lightning/stable/how-to/unsloth-sft/#__codelineno-10-41) return compute_reward(result.final_output, task["target"])` Tip You can test the agent with a dry run: `[](https://microsoft.github.io/agent-lightning/stable/how-to/unsloth-sft/#__codelineno-11-1) import asyncio [](https://microsoft.github.io/agent-lightning/stable/how-to/unsloth-sft/#__codelineno-11-2) [](https://microsoft.github.io/agent-lightning/stable/how-to/unsloth-sft/#__codelineno-11-3) llm = agl.LLM( [](https://microsoft.github.io/agent-lightning/stable/how-to/unsloth-sft/#__codelineno-11-4) endpoint=os.environ["OPENAI_BASE_URL"], [](https://microsoft.github.io/agent-lightning/stable/how-to/unsloth-sft/#__codelineno-11-5) api_key=os.environ["OPENAI_API_KEY"], [](https://microsoft.github.io/agent-lightning/stable/how-to/unsloth-sft/#__codelineno-11-6) model="gpt-4.1-mini", [](https://microsoft.github.io/agent-lightning/stable/how-to/unsloth-sft/#__codelineno-11-7) ) [](https://microsoft.github.io/agent-lightning/stable/how-to/unsloth-sft/#__codelineno-11-8) asyncio.run(math_agent({"input": "What is 1 + 1?", "target": 2.0}, llm))` Run this Recipe[¶](https://microsoft.github.io/agent-lightning/stable/how-to/unsloth-sft/#run-this-recipe "Permanent link") ---------------------------------------------------------------------------------------------------------------------------- The full runnable script for this recipe resides in [`examples/unsloth`](https://github.com/microsoft/agent-lightning/tree/3b5d733861cf313fc09821a23240bbdf3cb2ee5b/examples/unsloth) folder. Before running this example, install `unsloth`, `vllm`, and the other libraries used in the examples (the project uses CUDA tooling, TRL, rich, datasets, etc.). We tested with `unsloth==2025.10.1`. `unsloth==2025.10.2` and `2025.10.3` are not working because of an [issue](https://github.com/unslothai/unsloth/issues/3451) we have been investigating with the unsloth team. It's recommended to download the base model before running the example, such that the first iteration and subsequent iterations can both load from local checkpoints. `[](https://microsoft.github.io/agent-lightning/stable/how-to/unsloth-sft/#__codelineno-12-1) hf download unsloth/Qwen3-4B-Instruct-2507 --local-dir models/version_0` The repository already contains `examples/unsloth/data_gsmhard.jsonl` (which is a very small subset of the [GSM-hard math dataset](https://huggingface.co/datasets/reasoning-machines/gsm-hard) for demonstration purposes). ### Run Manually[¶](https://microsoft.github.io/agent-lightning/stable/how-to/unsloth-sft/#run-manually "Permanent link") Similar to the [Write the First Algorithm](https://microsoft.github.io/agent-lightning/stable/how-to/write-first-algorithm/) recipe, you can open three terminals and start each component in parallel. `[](https://microsoft.github.io/agent-lightning/stable/how-to/unsloth-sft/#__codelineno-13-1) agl store --port 4747 [](https://microsoft.github.io/agent-lightning/stable/how-to/unsloth-sft/#__codelineno-13-2) python examples/unsloth/sft_rollout_runners.py [](https://microsoft.github.io/agent-lightning/stable/how-to/unsloth-sft/#__codelineno-13-3) python examples/unsloth/sft_algorithm.py` In this case, [`sft_rollout_runners.py`](https://github.com/microsoft/agent-lightning/tree/3b5d733861cf313fc09821a23240bbdf3cb2ee5b/examples/unsloth/sft_rollout_runners.py) is a simple spawner implemented in Python that spawns 4 runners in parallel. The runners all connect to the same store server executing in another terminal. `[](https://microsoft.github.io/agent-lightning/stable/how-to/unsloth-sft/#__codelineno-14-1) import agentlightning as agl [](https://microsoft.github.io/agent-lightning/stable/how-to/unsloth-sft/#__codelineno-14-2) [](https://microsoft.github.io/agent-lightning/stable/how-to/unsloth-sft/#__codelineno-14-3) def run_rollout(store: agl.LightningStore, worker_id: int) -> None: [](https://microsoft.github.io/agent-lightning/stable/how-to/unsloth-sft/#__codelineno-14-4) # Since the server side has already used LiteLLM proxy to collect traces, [](https://microsoft.github.io/agent-lightning/stable/how-to/unsloth-sft/#__codelineno-14-5) # a simple OtelTracer to collect the rewards is enough. [](https://microsoft.github.io/agent-lightning/stable/how-to/unsloth-sft/#__codelineno-14-6) tracer = agl.OtelTracer() [](https://microsoft.github.io/agent-lightning/stable/how-to/unsloth-sft/#__codelineno-14-7) [](https://microsoft.github.io/agent-lightning/stable/how-to/unsloth-sft/#__codelineno-14-8) runner = agl.LitAgentRunner(tracer=tracer) [](https://microsoft.github.io/agent-lightning/stable/how-to/unsloth-sft/#__codelineno-14-9) [](https://microsoft.github.io/agent-lightning/stable/how-to/unsloth-sft/#__codelineno-14-10) with runner.run_context(agent=math_agent, store=store, worker_id=worker_id): [](https://microsoft.github.io/agent-lightning/stable/how-to/unsloth-sft/#__codelineno-14-11) asyncio.run(runner.iter()) [](https://microsoft.github.io/agent-lightning/stable/how-to/unsloth-sft/#__codelineno-14-12) [](https://microsoft.github.io/agent-lightning/stable/how-to/unsloth-sft/#__codelineno-14-13) [](https://microsoft.github.io/agent-lightning/stable/how-to/unsloth-sft/#__codelineno-14-14) def spawn_runners(store: agl.LightningStore, n_runners: int) -> None: [](https://microsoft.github.io/agent-lightning/stable/how-to/unsloth-sft/#__codelineno-14-15) runners = [ [](https://microsoft.github.io/agent-lightning/stable/how-to/unsloth-sft/#__codelineno-14-16) multiprocessing.Process(target=run_rollout, args=(store, worker_id)) [](https://microsoft.github.io/agent-lightning/stable/how-to/unsloth-sft/#__codelineno-14-17) for worker_id in range(n_runners) [](https://microsoft.github.io/agent-lightning/stable/how-to/unsloth-sft/#__codelineno-14-18) ] [](https://microsoft.github.io/agent-lightning/stable/how-to/unsloth-sft/#__codelineno-14-19) for runner in runners: [](https://microsoft.github.io/agent-lightning/stable/how-to/unsloth-sft/#__codelineno-14-20) runner.start() [](https://microsoft.github.io/agent-lightning/stable/how-to/unsloth-sft/#__codelineno-14-21) [](https://microsoft.github.io/agent-lightning/stable/how-to/unsloth-sft/#__codelineno-14-22) for runner in runners: [](https://microsoft.github.io/agent-lightning/stable/how-to/unsloth-sft/#__codelineno-14-23) runner.join() [](https://microsoft.github.io/agent-lightning/stable/how-to/unsloth-sft/#__codelineno-14-24) [](https://microsoft.github.io/agent-lightning/stable/how-to/unsloth-sft/#__codelineno-14-25) [](https://microsoft.github.io/agent-lightning/stable/how-to/unsloth-sft/#__codelineno-14-26) store = agl.LightningStoreClient("http://localhost:4747") [](https://microsoft.github.io/agent-lightning/stable/how-to/unsloth-sft/#__codelineno-14-27) spawn_runners(store=store, n_runners=4)` Tip Try to swap [`OtelTracer`](https://microsoft.github.io/agent-lightning/stable/reference/runner/#agentlightning.OtelTracer " agentlightning.OtelTracer") in the runners with other tracers like [`AgentOpsTracer`](https://microsoft.github.io/agent-lightning/stable/reference/runner/#agentlightning.AgentOpsTracer " agentlightning.AgentOpsTracer") . Try to use a different adapter at the algorithm side such as [`TracerTraceToTriplet`](https://microsoft.github.io/agent-lightning/stable/reference/algorithm/#agentlightning.TracerTraceToTriplet " agentlightning.TracerTraceToTriplet") to see what happens. ### Run Everything with Trainer[¶](https://microsoft.github.io/agent-lightning/stable/how-to/unsloth-sft/#run-everything-with-trainer "Permanent link") We also show how to wrap everything into a single script using [`Trainer`](https://microsoft.github.io/agent-lightning/stable/reference/trainer/#agentlightning.Trainer " agentlightning.Trainer") . [`sft_allinone.py`](https://github.com/microsoft/agent-lightning/tree/3b5d733861cf313fc09821a23240bbdf3cb2ee5b/examples/unsloth/sft_allinone.py) wires the same components together, replacing the manual management of runners above. `[](https://microsoft.github.io/agent-lightning/stable/how-to/unsloth-sft/#__codelineno-15-1) class UnslothSupervisedFinetuning(agl.Algorithm): [](https://microsoft.github.io/agent-lightning/stable/how-to/unsloth-sft/#__codelineno-15-2) [](https://microsoft.github.io/agent-lightning/stable/how-to/unsloth-sft/#__codelineno-15-3) async def run( [](https://microsoft.github.io/agent-lightning/stable/how-to/unsloth-sft/#__codelineno-15-4) self, [](https://microsoft.github.io/agent-lightning/stable/how-to/unsloth-sft/#__codelineno-15-5) train_dataset: Optional[Dataset[GsmProblem]] = None, [](https://microsoft.github.io/agent-lightning/stable/how-to/unsloth-sft/#__codelineno-15-6) val_dataset: Optional[Dataset[GsmProblem]] = None, [](https://microsoft.github.io/agent-lightning/stable/how-to/unsloth-sft/#__codelineno-15-7) ): [](https://microsoft.github.io/agent-lightning/stable/how-to/unsloth-sft/#__codelineno-15-8) # Use the store, llm_proxy, and adapter from the trainer [](https://microsoft.github.io/agent-lightning/stable/how-to/unsloth-sft/#__codelineno-15-9) store = self.get_store() [](https://microsoft.github.io/agent-lightning/stable/how-to/unsloth-sft/#__codelineno-15-10) llm_proxy = self.get_llm_proxy() [](https://microsoft.github.io/agent-lightning/stable/how-to/unsloth-sft/#__codelineno-15-11) data_adapter = self.get_adapter() [](https://microsoft.github.io/agent-lightning/stable/how-to/unsloth-sft/#__codelineno-15-12) [](https://microsoft.github.io/agent-lightning/stable/how-to/unsloth-sft/#__codelineno-15-13) for iteration in range(self.max_iterations): [](https://microsoft.github.io/agent-lightning/stable/how-to/unsloth-sft/#__codelineno-15-14) ... # Same logic as sft_algorithm.py [](https://microsoft.github.io/agent-lightning/stable/how-to/unsloth-sft/#__codelineno-15-15) [](https://microsoft.github.io/agent-lightning/stable/how-to/unsloth-sft/#__codelineno-15-16) algo = UnslothSupervisedFinetuning( [](https://microsoft.github.io/agent-lightning/stable/how-to/unsloth-sft/#__codelineno-15-17) max_iterations=2, [](https://microsoft.github.io/agent-lightning/stable/how-to/unsloth-sft/#__codelineno-15-18) vllm_port=12316, [](https://microsoft.github.io/agent-lightning/stable/how-to/unsloth-sft/#__codelineno-15-19) train_triplet_fraction=0.5, [](https://microsoft.github.io/agent-lightning/stable/how-to/unsloth-sft/#__codelineno-15-20) initial_model_path="models/version_0", [](https://microsoft.github.io/agent-lightning/stable/how-to/unsloth-sft/#__codelineno-15-21) ) [](https://microsoft.github.io/agent-lightning/stable/how-to/unsloth-sft/#__codelineno-15-22) [](https://microsoft.github.io/agent-lightning/stable/how-to/unsloth-sft/#__codelineno-15-23) # The LLM proxy can be created before Trainer [](https://microsoft.github.io/agent-lightning/stable/how-to/unsloth-sft/#__codelineno-15-24) trainer = Trainer( [](https://microsoft.github.io/agent-lightning/stable/how-to/unsloth-sft/#__codelineno-15-25) n_runners=4, [](https://microsoft.github.io/agent-lightning/stable/how-to/unsloth-sft/#__codelineno-15-26) algorithm=algo, [](https://microsoft.github.io/agent-lightning/stable/how-to/unsloth-sft/#__codelineno-15-27) llm_proxy=LLMProxy(port=12358), [](https://microsoft.github.io/agent-lightning/stable/how-to/unsloth-sft/#__codelineno-15-28) ) [](https://microsoft.github.io/agent-lightning/stable/how-to/unsloth-sft/#__codelineno-15-29) [](https://microsoft.github.io/agent-lightning/stable/how-to/unsloth-sft/#__codelineno-15-30) trainer.fit(math_agent, load_math_dataset())` You might wonder where the initialization of [`Adapter`](https://microsoft.github.io/agent-lightning/stable/reference/algorithm/#agentlightning.Adapter " agentlightning.Adapter") happens in this code. It turns out that [`TracerTraceToTriplet`](https://microsoft.github.io/agent-lightning/stable/reference/algorithm/#agentlightning.TracerTraceToTriplet " agentlightning.TracerTraceToTriplet") is the default adapter in [`Trainer`](https://microsoft.github.io/agent-lightning/stable/reference/trainer/#agentlightning.Trainer " agentlightning.Trainer") , so we don't need to create one manually. Now you can run the example with: `[](https://microsoft.github.io/agent-lightning/stable/how-to/unsloth-sft/#__codelineno-16-1) python examples/unsloth/sft_allinone.py` It starts an [`InMemoryLighningStore`](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.InMemoryLightningStore " agentlightning.InMemoryLightningStore") for you, launches four worker processes, iterates the SFT loop, and prints the final checkpoint path when done. Adjust `max_iterations`, `train_triplet_fraction`, `n_runners`, or the proxy port to match your hardware or training goals. If you already run an external store or proxy you can also pass those objects into [`Trainer`](https://microsoft.github.io/agent-lightning/stable/reference/trainer/#agentlightning.Trainer " agentlightning.Trainer") instead of relying on the [Trainer-managed defaults](https://microsoft.github.io/agent-lightning/stable/tutorials/debug/#debug-with-external-store "Debug the Algorithm-Runner Boundary") . Info As a future plan, we might graduate this example into a more powerful SFT algorithm bundled into [Algorithm Zoo](https://microsoft.github.io/agent-lightning/stable/algorithm-zoo/) . Currently, this `UnslothSupervisedFinetuning` is still for demo purposes. Back to top --- # Train SQL Agent with RL - Agent-lightning [Skip to content](https://microsoft.github.io/agent-lightning/stable/how-to/train-sql-agent/#train-sql-agent-with-agent-lightning-and-verl) ⚠️ You are viewing development documentation. For stable documentation, please visit [https://microsoft.github.io/agent-lightning/stable/](https://microsoft.github.io/agent-lightning/stable/) Train SQL Agent with Agent-lightning and VERL[¶](https://microsoft.github.io/agent-lightning/stable/how-to/train-sql-agent/#train-sql-agent-with-agent-lightning-and-verl "Permanent link") ============================================================================================================================================================================================ This walkthrough builds upon the **Agent-lightning SQL Agent** example and explains how the system components integrate: a **LangGraph-based SQL agent** wrapped as a [`LitAgent`](https://microsoft.github.io/agent-lightning/stable/reference/agent/#agentlightning.LitAgent " agentlightning.LitAgent") , the **[`VERL`](https://microsoft.github.io/agent-lightning/stable/algorithm-zoo/verl/#agentlightning.algorithm.verl.VERL " VERL") reinforcement learning (RL) algorithm**, and the **[`Trainer`](https://microsoft.github.io/agent-lightning/stable/reference/trainer/#agentlightning.Trainer " agentlightning.Trainer") **, which coordinates both training and debugging. The command-line interface in [`examples/spider/train_sql_agent.py`](https://github.com/microsoft/agent-lightning/tree/3b5d733861cf313fc09821a23240bbdf3cb2ee5b/examples/spider/train_sql_agent.py) provides a complete runnable example. However, this document focuses on understanding the underlying architecture so you can effectively adapt the workflow to your own agents. SQL Agent Architecture[¶](https://microsoft.github.io/agent-lightning/stable/how-to/train-sql-agent/#sql-agent-architecture "Permanent link") ---------------------------------------------------------------------------------------------------------------------------------------------- Agent-lightning integrates seamlessly with various orchestration frameworks, including [Agent Framework](https://github.com/microsoft/agent-framework) , [AutoGen](https://github.com/microsoft/autogen) , [CrewAI](https://www.crewai.com/) , [LangGraph](https://github.com/langchain-ai/langgraph) , and the [OpenAI Agents SDK](https://github.com/openai/openai-agents-python) . It can also interoperate with custom Python logic. In this example, **LangGraph** defines a cyclic workflow that mirrors an analyst’s iterative SQL development process. The following graph (rendered directly from [`sql_agent.py`](https://github.com/microsoft/agent-lightning/tree/3b5d733861cf313fc09821a23240bbdf3cb2ee5b/examples/spider/sql_agent.py) ) illustrates how the agent drafts, executes, critiques, and refines queries until a satisfactory result is achieved. --- config: flowchart: curve: linear --- graph LR; __start__([

__start__

]):::first write_query(write_query) execute_query(execute_query) check_query(check_query) rewrite_query(rewrite_query) __end__([

__end__

]):::last __start__ --> write_query; check_query -.-> __end__; check_query -.-> rewrite_query; execute_query --> check_query; rewrite_query --> execute_query; write_query --> execute_query; classDef default fill:#f2f2f2,line-height:1.2 classDef first fill-opacity:0 classDef last fill:#cccccc Note The workflow proceeds through the following stages: 1. **write\_query** – Generates an initial SQL query from the user’s question and the database schema. 2. **execute\_query** – Executes the generated query against the target database. 3. **check\_query** – Evaluates the query and its results (or errors) using a specialized prompt (`CHECK_QUERY_PROMPT`) to detect issues. 4. **rewrite\_query** – If issues are identified, the agent rewrites the query using feedback from the previous step and re-enters the loop. 5. **END** – The cycle terminates when the query is validated or the maximum iteration count (`max_turns`) is reached. Each _turn_ consists of one full loop through the `write_query`, `execute_query`, `check_query`, and (if applicable) `rewrite_query` stages. In this tutorial, **reinforcement learning (RL)** is used to optimize the `write_query` and `rewrite_query` stages. While the `check_query` step shares the same underlying LLM weights, its trace data is not used for learning. To keep the design modular and maintainable, it is recommended to define the LangGraph-based SQL Agent in a separate file and expose it via a builder function such as: `[](https://microsoft.github.io/agent-lightning/stable/how-to/train-sql-agent/#__codelineno-0-1) def build_langgraph_sql_agent( [](https://microsoft.github.io/agent-lightning/stable/how-to/train-sql-agent/#__codelineno-0-2) database_path: str, [](https://microsoft.github.io/agent-lightning/stable/how-to/train-sql-agent/#__codelineno-0-3) openai_base_url: str, [](https://microsoft.github.io/agent-lightning/stable/how-to/train-sql-agent/#__codelineno-0-4) model: str, [](https://microsoft.github.io/agent-lightning/stable/how-to/train-sql-agent/#__codelineno-0-5) sampling_parameters: Dict[str, Any], [](https://microsoft.github.io/agent-lightning/stable/how-to/train-sql-agent/#__codelineno-0-6) max_turns: int, [](https://microsoft.github.io/agent-lightning/stable/how-to/train-sql-agent/#__codelineno-0-7) truncate_length: int [](https://microsoft.github.io/agent-lightning/stable/how-to/train-sql-agent/#__codelineno-0-8) ): [](https://microsoft.github.io/agent-lightning/stable/how-to/train-sql-agent/#__codelineno-0-9) builder = StateGraph(State) [](https://microsoft.github.io/agent-lightning/stable/how-to/train-sql-agent/#__codelineno-0-10) builder.add_node(write_query) [](https://microsoft.github.io/agent-lightning/stable/how-to/train-sql-agent/#__codelineno-0-11) ... [](https://microsoft.github.io/agent-lightning/stable/how-to/train-sql-agent/#__codelineno-0-12) [](https://microsoft.github.io/agent-lightning/stable/how-to/train-sql-agent/#__codelineno-0-13) builder.add_edge(START, "write_query") [](https://microsoft.github.io/agent-lightning/stable/how-to/train-sql-agent/#__codelineno-0-14) ... [](https://microsoft.github.io/agent-lightning/stable/how-to/train-sql-agent/#__codelineno-0-15) [](https://microsoft.github.io/agent-lightning/stable/how-to/train-sql-agent/#__codelineno-0-16) return builder.compile().graph()` This approach isolates your LangGraph logic from Agent-lightning version changes, improving both readability and debuggability. Bridging LangGraph and Agent-lightning[¶](https://microsoft.github.io/agent-lightning/stable/how-to/train-sql-agent/#bridging-langgraph-and-agent-lightning "Permanent link") ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ Tip Keep [`sql_agent.py`](https://github.com/microsoft/agent-lightning/tree/3b5d733861cf313fc09821a23240bbdf3cb2ee5b/examples/spider/sql_agent.py) open on the side while reading this section. This will help you understand how the code snippets shown here work in practice. The **`LitSQLAgent`** class defined in [`sql_agent.py`](https://github.com/microsoft/agent-lightning/tree/3b5d733861cf313fc09821a23240bbdf3cb2ee5b/examples/spider/sql_agent.py) acts as the bridge. It subclasses [`agl.LitAgent`](https://microsoft.github.io/agent-lightning/stable/reference/agent/#agentlightning.LitAgent " agentlightning.LitAgent") , allowing the runner to provision shared resources (e.g., [LLMs](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.LLM " agentlightning.LLM") ) for each rollout. Below is a simplified illustration of the key logic (note: this is conceptual pseudocode; the actual implementation includes dataset-specific details): `[](https://microsoft.github.io/agent-lightning/stable/how-to/train-sql-agent/#__codelineno-1-1) class LitSQLAgent(agl.LitAgent[Dict[str, Any]]): [](https://microsoft.github.io/agent-lightning/stable/how-to/train-sql-agent/#__codelineno-1-2) [](https://microsoft.github.io/agent-lightning/stable/how-to/train-sql-agent/#__codelineno-1-3) def __init__(self, max_turns: int, truncate_length: int): [](https://microsoft.github.io/agent-lightning/stable/how-to/train-sql-agent/#__codelineno-1-4) # Every turn here refers to a full cycle of write/exe/check/rewrite [](https://microsoft.github.io/agent-lightning/stable/how-to/train-sql-agent/#__codelineno-1-5) self.max_turns = max_turns [](https://microsoft.github.io/agent-lightning/stable/how-to/train-sql-agent/#__codelineno-1-6) self.truncate_length = truncate_length [](https://microsoft.github.io/agent-lightning/stable/how-to/train-sql-agent/#__codelineno-1-7) [](https://microsoft.github.io/agent-lightning/stable/how-to/train-sql-agent/#__codelineno-1-8) def rollout( [](https://microsoft.github.io/agent-lightning/stable/how-to/train-sql-agent/#__codelineno-1-9) self, [](https://microsoft.github.io/agent-lightning/stable/how-to/train-sql-agent/#__codelineno-1-10) task: Dict[str, Any], [](https://microsoft.github.io/agent-lightning/stable/how-to/train-sql-agent/#__codelineno-1-11) resources: agl.NamedResources, [](https://microsoft.github.io/agent-lightning/stable/how-to/train-sql-agent/#__codelineno-1-12) rollout: agl.Rollout [](https://microsoft.github.io/agent-lightning/stable/how-to/train-sql-agent/#__codelineno-1-13) ) -> float | None: [](https://microsoft.github.io/agent-lightning/stable/how-to/train-sql-agent/#__codelineno-1-14) llm: agl.LLM = resources["main_llm"] [](https://microsoft.github.io/agent-lightning/stable/how-to/train-sql-agent/#__codelineno-1-15) agent = build_langgraph_sql_agent( [](https://microsoft.github.io/agent-lightning/stable/how-to/train-sql-agent/#__codelineno-1-16) database_path="sqlite:///" + task["db_id"], [](https://microsoft.github.io/agent-lightning/stable/how-to/train-sql-agent/#__codelineno-1-17) max_turns=self.max_turns, [](https://microsoft.github.io/agent-lightning/stable/how-to/train-sql-agent/#__codelineno-1-18) truncate_length=self.truncate_length, [](https://microsoft.github.io/agent-lightning/stable/how-to/train-sql-agent/#__codelineno-1-19) openai_base_url=llm.get_base_url(rollout.rollout_id, rollout.attempt.attempt_id), [](https://microsoft.github.io/agent-lightning/stable/how-to/train-sql-agent/#__codelineno-1-20) model=llm.model, [](https://microsoft.github.io/agent-lightning/stable/how-to/train-sql-agent/#__codelineno-1-21) sampling_parameters=llm.sampling_parameters, [](https://microsoft.github.io/agent-lightning/stable/how-to/train-sql-agent/#__codelineno-1-22) ) [](https://microsoft.github.io/agent-lightning/stable/how-to/train-sql-agent/#__codelineno-1-23) result = agent.invoke({"question": question}, { [](https://microsoft.github.io/agent-lightning/stable/how-to/train-sql-agent/#__codelineno-1-24) "callbacks": [self.tracer.get_langchain_handler()], [](https://microsoft.github.io/agent-lightning/stable/how-to/train-sql-agent/#__codelineno-1-25) "recursion_limit": 100, [](https://microsoft.github.io/agent-lightning/stable/how-to/train-sql-agent/#__codelineno-1-26) }) [](https://microsoft.github.io/agent-lightning/stable/how-to/train-sql-agent/#__codelineno-1-27) reward = evaluate_query(result["query"], ground_truth, db_path, raise_on_error=False) [](https://microsoft.github.io/agent-lightning/stable/how-to/train-sql-agent/#__codelineno-1-28) return reward` The `LitSQLAgent` serves as a lightweight wrapper around the LangGraph agent, providing the correct interface for the [`rollout`](https://microsoft.github.io/agent-lightning/stable/reference/agent/#agentlightning.LitAgent.rollout " rollout(task, resources, rollout)") method. It constructs the LangGraph agent, invokes it, and returns the evaluation result as a reward signal. The `"main_llm"` resource key is a convention between the agent and [VERL](https://microsoft.github.io/agent-lightning/stable/algorithm-zoo/verl/#agentlightning.algorithm.verl.VERL " VERL") . It is used to inject an OpenAI-compatible endpoint from the [VERL](https://microsoft.github.io/agent-lightning/stable/algorithm-zoo/verl/#agentlightning.algorithm.verl.VERL " VERL") algorithm during rollout. Two approaches are supported to use this [agentlightning.LLM](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.LLM " agentlightning.LLM") resource: 1. **Direct access** – Use [`llm.endpoint`](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.LLM.endpoint " endpoint instance-attribute ") for a simple integration (identical to the v0.1 example). 2. **Context-aware access** – Use [`get_base_url`](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.ProxyLLM.get_base_url " get_base_url(rollout_id, attempt_id)") with [`rollout.rollout_id`](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.Rollout.rollout_id " rollout_id instance-attribute ") and [`rollout.attempt.attempt_id`](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.Attempt.attempt_id " attempt_id instance-attribute ") . This approach enables per-caller trace attribution, improving trace collection per rollout or attempt when runner-side tracers are unavailable. For details, see [Working with Traces](https://microsoft.github.io/agent-lightning/stable/tutorials/traces/) . Reward Signal and Evaluation[¶](https://microsoft.github.io/agent-lightning/stable/how-to/train-sql-agent/#reward-signal-and-evaluation "Permanent link") ---------------------------------------------------------------------------------------------------------------------------------------------------------- The `evaluate_query` function provides the reward mechanism for RL training. In agent training, obtaining a consistent and meaningful reward signal is often challenging. Fortunately, this is simplified when using the [**Spider dataset**](https://yale-lily.github.io/spider) . The dataset includes ~8k samples containing natural-language questions, database schemas, and ground-truth SQL queries. Using the [**Spider evaluator**](https://github.com/taoyds/test-suite-sql-eval) , the agent's generated query is executed and compared to the ground-truth query on the target database. The two queries are considered equivalent if they produce identical execution results. Attention The ground-truth queries must **never** be exposed to the agent during training to prevent data leakage. In this setup, the reward is returned directly from the [`rollout`](https://microsoft.github.io/agent-lightning/stable/reference/agent/#agentlightning.LitAgent.rollout " rollout(task, resources, rollout)") method, enabling the runner to forward it back to the RL algorithm. Warning Avoid using [`emit_reward`](https://microsoft.github.io/agent-lightning/stable/reference/agent/#agentlightning.emit_reward " agentlightning.emit_reward(reward, *, primary_key=None, attributes=None, propagate=True)") in conjunction with returning a reward value. Doing both will cause the algorithm to receive duplicate reward signals, leading to inconsistent training behavior. Configuring VERL for Reinforcement Learning[¶](https://microsoft.github.io/agent-lightning/stable/how-to/train-sql-agent/#configuring-verl-for-reinforcement-learning "Permanent link") ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- View [`examples/spider/train_sql_agent.py`](https://github.com/microsoft/agent-lightning/tree/3b5d733861cf313fc09821a23240bbdf3cb2ee5b/examples/spider/train_sql_agent.py) for a full reinforcement learning configuration, which is a plain Python dictionary. It mirrors (and actually _is_) the [shell arguments](https://verl.readthedocs.io/en/latest/index.html) used to launch training in the VERL framework but is easier to tweak programmatically: `[](https://microsoft.github.io/agent-lightning/stable/how-to/train-sql-agent/#__codelineno-2-1) verl_config: Dict[str, Any] = { [](https://microsoft.github.io/agent-lightning/stable/how-to/train-sql-agent/#__codelineno-2-2) "algorithm": {"adv_estimator": "grpo", "use_kl_in_reward": False}, [](https://microsoft.github.io/agent-lightning/stable/how-to/train-sql-agent/#__codelineno-2-3) "data": { [](https://microsoft.github.io/agent-lightning/stable/how-to/train-sql-agent/#__codelineno-2-4) # train_files and val_files are no longer needed here [](https://microsoft.github.io/agent-lightning/stable/how-to/train-sql-agent/#__codelineno-2-5) # because data are read in agl.Trainer [](https://microsoft.github.io/agent-lightning/stable/how-to/train-sql-agent/#__codelineno-2-6) ..., [](https://microsoft.github.io/agent-lightning/stable/how-to/train-sql-agent/#__codelineno-2-7) # Controls how many tasks are pooled per step [](https://microsoft.github.io/agent-lightning/stable/how-to/train-sql-agent/#__codelineno-2-8) # (multiplied by actor_rollout_ref.rollout.n) [](https://microsoft.github.io/agent-lightning/stable/how-to/train-sql-agent/#__codelineno-2-9) "train_batch_size": 32, [](https://microsoft.github.io/agent-lightning/stable/how-to/train-sql-agent/#__codelineno-2-10) # Prompt and responses larger than these lengths are truncated [](https://microsoft.github.io/agent-lightning/stable/how-to/train-sql-agent/#__codelineno-2-11) "max_prompt_length": 4096, [](https://microsoft.github.io/agent-lightning/stable/how-to/train-sql-agent/#__codelineno-2-12) "max_response_length": 2048, [](https://microsoft.github.io/agent-lightning/stable/how-to/train-sql-agent/#__codelineno-2-13) }, [](https://microsoft.github.io/agent-lightning/stable/how-to/train-sql-agent/#__codelineno-2-14) "actor_rollout_ref": { [](https://microsoft.github.io/agent-lightning/stable/how-to/train-sql-agent/#__codelineno-2-15) "rollout": { [](https://microsoft.github.io/agent-lightning/stable/how-to/train-sql-agent/#__codelineno-2-16) # Only vLLM is supported currently [](https://microsoft.github.io/agent-lightning/stable/how-to/train-sql-agent/#__codelineno-2-17) "name": "vllm", [](https://microsoft.github.io/agent-lightning/stable/how-to/train-sql-agent/#__codelineno-2-18) # Equals to group size of GRPO [](https://microsoft.github.io/agent-lightning/stable/how-to/train-sql-agent/#__codelineno-2-19) "n": 4, [](https://microsoft.github.io/agent-lightning/stable/how-to/train-sql-agent/#__codelineno-2-20) # Used to enable tool call parser in vLLM [](https://microsoft.github.io/agent-lightning/stable/how-to/train-sql-agent/#__codelineno-2-21) "multi_turn": {"format": "hermes"}, [](https://microsoft.github.io/agent-lightning/stable/how-to/train-sql-agent/#__codelineno-2-22) ... [](https://microsoft.github.io/agent-lightning/stable/how-to/train-sql-agent/#__codelineno-2-23) }, [](https://microsoft.github.io/agent-lightning/stable/how-to/train-sql-agent/#__codelineno-2-24) "actor": {"ppo_mini_batch_size": 32, "optim": {"lr": 1e-6}, ...}, [](https://microsoft.github.io/agent-lightning/stable/how-to/train-sql-agent/#__codelineno-2-25) "model": { [](https://microsoft.github.io/agent-lightning/stable/how-to/train-sql-agent/#__codelineno-2-26) # Config your preferred LLM here [](https://microsoft.github.io/agent-lightning/stable/how-to/train-sql-agent/#__codelineno-2-27) "path": "Qwen/Qwen2.5-Coder-1.5B-Instruct", [](https://microsoft.github.io/agent-lightning/stable/how-to/train-sql-agent/#__codelineno-2-28) ... [](https://microsoft.github.io/agent-lightning/stable/how-to/train-sql-agent/#__codelineno-2-29) }, [](https://microsoft.github.io/agent-lightning/stable/how-to/train-sql-agent/#__codelineno-2-30) }, [](https://microsoft.github.io/agent-lightning/stable/how-to/train-sql-agent/#__codelineno-2-31) "trainer": { [](https://microsoft.github.io/agent-lightning/stable/how-to/train-sql-agent/#__codelineno-2-32) "n_gpus_per_node": 1, [](https://microsoft.github.io/agent-lightning/stable/how-to/train-sql-agent/#__codelineno-2-33) # Validation once before training starts [](https://microsoft.github.io/agent-lightning/stable/how-to/train-sql-agent/#__codelineno-2-34) "val_before_train": True, [](https://microsoft.github.io/agent-lightning/stable/how-to/train-sql-agent/#__codelineno-2-35) # Validation every N training steps [](https://microsoft.github.io/agent-lightning/stable/how-to/train-sql-agent/#__codelineno-2-36) "test_freq": 32, [](https://microsoft.github.io/agent-lightning/stable/how-to/train-sql-agent/#__codelineno-2-37) # Save checkpoints every N training steps [](https://microsoft.github.io/agent-lightning/stable/how-to/train-sql-agent/#__codelineno-2-38) "save_freq": 64, [](https://microsoft.github.io/agent-lightning/stable/how-to/train-sql-agent/#__codelineno-2-39) # Go through the train dataset this many times [](https://microsoft.github.io/agent-lightning/stable/how-to/train-sql-agent/#__codelineno-2-40) "total_epochs": 2 [](https://microsoft.github.io/agent-lightning/stable/how-to/train-sql-agent/#__codelineno-2-41) }, [](https://microsoft.github.io/agent-lightning/stable/how-to/train-sql-agent/#__codelineno-2-42) }` This is equivalent to the following CLI invocation: `[](https://microsoft.github.io/agent-lightning/stable/how-to/train-sql-agent/#__codelineno-3-1) python3 -m verl.trainer.main_ppo \ [](https://microsoft.github.io/agent-lightning/stable/how-to/train-sql-agent/#__codelineno-3-2) algorithm.adv_estimator=grpo \ [](https://microsoft.github.io/agent-lightning/stable/how-to/train-sql-agent/#__codelineno-3-3) algorithm.use_kl_in_reward=False \ [](https://microsoft.github.io/agent-lightning/stable/how-to/train-sql-agent/#__codelineno-3-4) data.train_batch_size=32 \ [](https://microsoft.github.io/agent-lightning/stable/how-to/train-sql-agent/#__codelineno-3-5) data.max_prompt_length=4096 \ [](https://microsoft.github.io/agent-lightning/stable/how-to/train-sql-agent/#__codelineno-3-6) data.max_response_length=2048 \ [](https://microsoft.github.io/agent-lightning/stable/how-to/train-sql-agent/#__codelineno-3-7) actor_rollout_ref.rollout.name=vllm \ [](https://microsoft.github.io/agent-lightning/stable/how-to/train-sql-agent/#__codelineno-3-8) actor_rollout_ref.rollout.n=4 \ [](https://microsoft.github.io/agent-lightning/stable/how-to/train-sql-agent/#__codelineno-3-9) actor_rollout_ref.rollout.multi_turn.format=hermes \ [](https://microsoft.github.io/agent-lightning/stable/how-to/train-sql-agent/#__codelineno-3-10) actor_rollout_ref.actor.ppo_mini_batch_size=32 \ [](https://microsoft.github.io/agent-lightning/stable/how-to/train-sql-agent/#__codelineno-3-11) actor_rollout_ref.actor.optim.lr=1e-6 \ [](https://microsoft.github.io/agent-lightning/stable/how-to/train-sql-agent/#__codelineno-3-12) actor_rollout_ref.model.path=Qwen/Qwen2.5-Coder-1.5B-Instruct \ [](https://microsoft.github.io/agent-lightning/stable/how-to/train-sql-agent/#__codelineno-3-13) trainer.n_gpus_per_node=1 \ [](https://microsoft.github.io/agent-lightning/stable/how-to/train-sql-agent/#__codelineno-3-14) trainer.val_before_train=True \ [](https://microsoft.github.io/agent-lightning/stable/how-to/train-sql-agent/#__codelineno-3-15) trainer.test_freq=32 \ [](https://microsoft.github.io/agent-lightning/stable/how-to/train-sql-agent/#__codelineno-3-16) trainer.save_freq=64 \ [](https://microsoft.github.io/agent-lightning/stable/how-to/train-sql-agent/#__codelineno-3-17) trainer.total_epochs=2` Warning We used to provide a CLI called `python -m agentlightning.verl` to launch training in v0.1. This is no longer the recommended approach. Instead, use [`agl.Trainer`](https://microsoft.github.io/agent-lightning/stable/reference/trainer/#agentlightning.Trainer " agentlightning.Trainer") to run VERL and agent runners together, or follow the [debugging tutorial](https://microsoft.github.io/agent-lightning/stable/tutorials/debug/) if you want an isolated experience similar to v0.1. Orchestrating Training with [`Trainer`](https://microsoft.github.io/agent-lightning/stable/reference/trainer/#agentlightning.Trainer " agentlightning.Trainer") [¶](https://microsoft.github.io/agent-lightning/stable/how-to/train-sql-agent/#orchestrating-training-with-trainer "Permanent link") ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- [`Trainer`](https://microsoft.github.io/agent-lightning/stable/reference/trainer/#agentlightning.Trainer " agentlightning.Trainer") is the high-level orchestrator that integrates the agent, algorithm, dataset, and distributed runners. The key benefits of using the [`Trainer`](https://microsoft.github.io/agent-lightning/stable/reference/trainer/#agentlightning.Trainer " agentlightning.Trainer") are: 1. It allows you to launch everything with a single line of code: `trainer.fit(...)`. 2. It exposes configuration options such as `n_runners` to control parallelism and `adapter` to define how algorithms interpret the trace data produced by the agent. An example usage is shown below: `[](https://microsoft.github.io/agent-lightning/stable/how-to/train-sql-agent/#__codelineno-4-1) import agentlightning as agl [](https://microsoft.github.io/agent-lightning/stable/how-to/train-sql-agent/#__codelineno-4-2) [](https://microsoft.github.io/agent-lightning/stable/how-to/train-sql-agent/#__codelineno-4-3) agent = LitSQLAgent() [](https://microsoft.github.io/agent-lightning/stable/how-to/train-sql-agent/#__codelineno-4-4) algorithm = agl.VERL(verl_config) [](https://microsoft.github.io/agent-lightning/stable/how-to/train-sql-agent/#__codelineno-4-5) trainer = agl.Trainer( [](https://microsoft.github.io/agent-lightning/stable/how-to/train-sql-agent/#__codelineno-4-6) n_runners=10, [](https://microsoft.github.io/agent-lightning/stable/how-to/train-sql-agent/#__codelineno-4-7) algorithm=algorithm, [](https://microsoft.github.io/agent-lightning/stable/how-to/train-sql-agent/#__codelineno-4-8) adapter={"agent_match": active_agent}, [](https://microsoft.github.io/agent-lightning/stable/how-to/train-sql-agent/#__codelineno-4-9) ) [](https://microsoft.github.io/agent-lightning/stable/how-to/train-sql-agent/#__codelineno-4-10) train_data = pd.read_parquet("data/train_spider.parquet").to_dict("records") [](https://microsoft.github.io/agent-lightning/stable/how-to/train-sql-agent/#__codelineno-4-11) val_data = pd.read_parquet("data/test_dev_500.parquet").to_dict("records") [](https://microsoft.github.io/agent-lightning/stable/how-to/train-sql-agent/#__codelineno-4-12) trainer.fit(agent, train_dataset=train_data, val_dataset=val_data)` First, `agl.VERL(verl_config)` launches the [`VERL`](https://microsoft.github.io/agent-lightning/stable/algorithm-zoo/verl/#agentlightning.algorithm.verl.VERL " VERL") algorithm and its OpenAI-compatible proxy. The `train_data` and `val_data` are passed into [`VERL`](https://microsoft.github.io/agent-lightning/stable/algorithm-zoo/verl/#agentlightning.algorithm.verl.VERL " VERL") , which enqueues tasks to a centralized task queue managed by the [`LightningStore`](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.LightningStore " agentlightning.LightningStore") , accessible to all runners. When [`Trainer.fit`](https://microsoft.github.io/agent-lightning/stable/reference/trainer/#agentlightning.Trainer.fit " fit(agent, train_dataset=None, *, val_dataset=None)") is called, it launches 10 concurrent runners (as specified by `n_runners=10`). Each runner pulls tasks from the centralized task queue, executes the agent’s [`rollout`](https://microsoft.github.io/agent-lightning/stable/reference/agent/#agentlightning.LitAgent.rollout " rollout(task, resources, rollout)") method, collects traces, and returns rewards to VERL for training. The [`Adapter`](https://microsoft.github.io/agent-lightning/stable/reference/algorithm/#agentlightning.Adapter " agentlightning.Adapter") , as discussed earlier, is used at the algorithm side, and receives the traces emitted by the agent and runners. The `agent_match` parameter ensures [`VERL`](https://microsoft.github.io/agent-lightning/stable/algorithm-zoo/verl/#agentlightning.algorithm.verl.VERL " VERL") only ingests spans from the specific agent you want to optimize. In the example above, there are at least three agents—`write_query`, `rewrite_query`, and `check_query`. By setting `agent_match` to a regex like `"write"`, both `write_query` and `rewrite_query` agents are optimized simultaneously. You can also set it to `"write|check"` or `None` to include all agents if desired. Dry-Run the Pipeline with [`Trainer.dev`](https://microsoft.github.io/agent-lightning/stable/reference/trainer/#agentlightning.Trainer.dev " dev(agent, train_dataset=None, *, val_dataset=None)") [¶](https://microsoft.github.io/agent-lightning/stable/how-to/train-sql-agent/#dry-run-the-pipeline-with-trainerdev "Permanent link") ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Before committing hours of GPU time, you can **dry-run** the agent with [`Trainer.dev()`](https://microsoft.github.io/agent-lightning/stable/reference/trainer/#agentlightning.Trainer.dev " dev(agent, train_dataset=None, *, val_dataset=None)") . This method swaps in the lightweight [`Baseline`](https://microsoft.github.io/agent-lightning/stable/reference/algorithm/#agentlightning.Baseline " agentlightning.Baseline") algorithm, enqueues up to ten tasks, and prints every span emitted by the agent. Because it uses the same runner stack as full training, it’s ideal for verifying database connections and LangGraph control flow. To begin, the agent needs a valid OpenAI-compatible endpoint since VERL is not active in this mode. You can use OpenAI’s official API or your own local LLM endpoint. Wrap it as follows: `[](https://microsoft.github.io/agent-lightning/stable/how-to/train-sql-agent/#__codelineno-5-1) trainer = agl.Trainer( [](https://microsoft.github.io/agent-lightning/stable/how-to/train-sql-agent/#__codelineno-5-2) n_workers=1, [](https://microsoft.github.io/agent-lightning/stable/how-to/train-sql-agent/#__codelineno-5-3) initial_resources={ [](https://microsoft.github.io/agent-lightning/stable/how-to/train-sql-agent/#__codelineno-5-4) "main_llm": agl.LLM( [](https://microsoft.github.io/agent-lightning/stable/how-to/train-sql-agent/#__codelineno-5-5) endpoint=os.environ["OPENAI_API_BASE"], [](https://microsoft.github.io/agent-lightning/stable/how-to/train-sql-agent/#__codelineno-5-6) model="gpt-4.1-nano", [](https://microsoft.github.io/agent-lightning/stable/how-to/train-sql-agent/#__codelineno-5-7) sampling_parameters={"temperature": 0.7}, [](https://microsoft.github.io/agent-lightning/stable/how-to/train-sql-agent/#__codelineno-5-8) ) [](https://microsoft.github.io/agent-lightning/stable/how-to/train-sql-agent/#__codelineno-5-9) }, [](https://microsoft.github.io/agent-lightning/stable/how-to/train-sql-agent/#__codelineno-5-10) )` Then, call [`trainer.dev(...)`](https://microsoft.github.io/agent-lightning/stable/reference/trainer/#agentlightning.Trainer.dev " dev(agent, train_dataset=None, *, val_dataset=None)") with a small number of tasks: `[](https://microsoft.github.io/agent-lightning/stable/how-to/train-sql-agent/#__codelineno-6-1) dev_data = pd.read_parquet("data/test_dev_500.parquet").to_dict("records")[:10] [](https://microsoft.github.io/agent-lightning/stable/how-to/train-sql-agent/#__codelineno-6-2) trainer.dev(agent, dev_dataset=dev_data)` Run this in a Python session or adapt your script to include a `--dev` flag. Once the spans appear healthy and the rewards are non-zero, switch back to [`trainer.fit(...)`](https://microsoft.github.io/agent-lightning/stable/reference/trainer/#agentlightning.Trainer.fit " fit(agent, train_dataset=None, *, val_dataset=None)") for full RL training. See the [debugging tutorial](https://microsoft.github.io/agent-lightning/stable/tutorials/debug/) for more tips on how to debug the agent. Running the Sample Code[¶](https://microsoft.github.io/agent-lightning/stable/how-to/train-sql-agent/#running-the-sample-code "Permanent link") ------------------------------------------------------------------------------------------------------------------------------------------------ The following tutorial explains how to run the complete example in [`examples/spider`](https://github.com/microsoft/agent-lightning/tree/3b5d733861cf313fc09821a23240bbdf3cb2ee5b/examples/spider) . ### Dataset[¶](https://microsoft.github.io/agent-lightning/stable/how-to/train-sql-agent/#dataset "Permanent link") The trainer expects three Parquet files inside `examples/spider/data`: `train_spider.parquet`, `test_dev_500.parquet`, and `test_dev.parquet`. Download the curated dataset bundle provided with the repository: `[](https://microsoft.github.io/agent-lightning/stable/how-to/train-sql-agent/#__codelineno-7-1) cd examples/spider [](https://microsoft.github.io/agent-lightning/stable/how-to/train-sql-agent/#__codelineno-7-2) pip install gdown # included in the 'experiment' optional dependency [](https://microsoft.github.io/agent-lightning/stable/how-to/train-sql-agent/#__codelineno-7-3) gdown --fuzzy https://drive.google.com/file/d/1oi9J1jZP9TyM35L85CL3qeGWl2jqlnL6/view [](https://microsoft.github.io/agent-lightning/stable/how-to/train-sql-agent/#__codelineno-7-4) unzip -q spider-data.zip -d data [](https://microsoft.github.io/agent-lightning/stable/how-to/train-sql-agent/#__codelineno-7-5) rm spider-data.zip` If you prefer to generate the files yourself, download [Spider 1.0](https://yale-lily.github.io/spider) and run: `[](https://microsoft.github.io/agent-lightning/stable/how-to/train-sql-agent/#__codelineno-8-1) python spider_eval/convert_dataset.py` Set `VERL_SPIDER_DATA_DIR` if you store the dataset outside the default `data` directory. ### Dependencies[¶](https://microsoft.github.io/agent-lightning/stable/how-to/train-sql-agent/#dependencies "Permanent link") Create a clean virtual environment, activate it, and install Agent-lightning with the VERL extras required by [this tutorial](https://microsoft.github.io/agent-lightning/stable/tutorials/installation/) . Install LangChain-related dependencies as needed. For full training profiles, plan to use a GPU with at least **40 GB** of memory. ### Launch Training[¶](https://microsoft.github.io/agent-lightning/stable/how-to/train-sql-agent/#launch-training "Permanent link") From [`examples/spider`](https://github.com/microsoft/agent-lightning/tree/3b5d733861cf313fc09821a23240bbdf3cb2ee5b/examples/spider) , run one of the helper scripts depending on your model preference: `[](https://microsoft.github.io/agent-lightning/stable/how-to/train-sql-agent/#__codelineno-9-1) python train_sql_agent.py qwen # Default Qwen-2.5-Coder-1.5B run [](https://microsoft.github.io/agent-lightning/stable/how-to/train-sql-agent/#__codelineno-9-2) python train_sql_agent.py llama # LLaMA-3.2-1B with llama3_json tool parser` The script instantiates `LitSQLAgent` and launches [`trainer.fit`](https://microsoft.github.io/agent-lightning/stable/reference/trainer/#agentlightning.Trainer.fit " fit(agent, train_dataset=None, *, val_dataset=None)") . Provide `--active-agent my_agent_variant` if you only want to train one of the agents in the graph. For the LLaMA profile, export an `HF_TOKEN` before running so VERL can download the model weights. Troubleshooting If you have got some Ray worker errors on either `WANDB_API_KEY` not set, or `HF_TOKEN` not set, or data not found, please try to restart the Ray cluster with the helper script: [scripts/restart\_ray.sh](https://github.com/microsoft/agent-lightning/tree/3b5d733861cf313fc09821a23240bbdf3cb2ee5b/scripts/restart_ray.sh) , which essentially stops the ray cluster if any, and starts a new one: `[](https://microsoft.github.io/agent-lightning/stable/how-to/train-sql-agent/#__codelineno-10-1) env RAY_DEBUG=legacy HYDRA_FULL_ERROR=1 VLLM_USE_V1=1 ray start --head --dashboard-host=0.0.0.0` Launching Training with NPUs The example also supports running with **Huawei Ascend NPUs**. This feature is contributed by [Teams from Huawei](https://github.com/microsoft/agent-lightning/pull/272) . To use it, resort to the function `config_train_npu` in the script. **Hardware Supported:** Atlas 200T A2 Box16, Atlas 900 A2 PODc, Atlas 800T A3. At least **a single 40GB NPU** is required to run the **Qwen2.5-Coder-1.5B-Instruct** model. **Environment Setup:** Python 3.11.13, CANN 8.2.RC1, torch 2.7.1+cpu, torch\_npu 2.7.1.dev20250724. For basic environment preparation, please refer to this [document](https://gitcode.com/Ascend/pytorch) . Before installing dependencies, configure the following pip mirrors: `[](https://microsoft.github.io/agent-lightning/stable/how-to/train-sql-agent/#__codelineno-11-1) pip config set global.index-url http://repo.huaweicloud.com/repository/pypi/simple [](https://microsoft.github.io/agent-lightning/stable/how-to/train-sql-agent/#__codelineno-11-2) pip config set global.extra-index-url "https://download.pytorch.org/whl/cpu/ https://mirrors.huaweicloud.com/ascend/repos/pypi"` Then install vLLM, vLLM-Ascend and VERL: `[](https://microsoft.github.io/agent-lightning/stable/how-to/train-sql-agent/#__codelineno-12-1) pip install vllm==0.10.0 --trusted-host repo.huaweicloud.com [](https://microsoft.github.io/agent-lightning/stable/how-to/train-sql-agent/#__codelineno-12-2) pip install vllm-Ascend==0.10.0rc1 --trusted-host repo.huaweicloud.com [](https://microsoft.github.io/agent-lightning/stable/how-to/train-sql-agent/#__codelineno-12-3) pip install verl==0.5.0` To ensure the VERL framework runs correctly on NPU, add the following lines to `verl/utils/vllm_utils.py`: `[](https://microsoft.github.io/agent-lightning/stable/how-to/train-sql-agent/#__codelineno-13-1) from vllm_ascend.patch import platform [](https://microsoft.github.io/agent-lightning/stable/how-to/train-sql-agent/#__codelineno-13-2) from vllm_ascend.patch import worker` See the following reference for more details: [https://github.com/vllm-project/vllm-ascend/issues/1776](https://github.com/vllm-project/vllm-ascend/issues/1776) . After the above dependencies have been installed, from [`examples/spider`](https://github.com/microsoft/agent-lightning/tree/3b5d733861cf313fc09821a23240bbdf3cb2ee5b/examples/spider) run the following script command: `[](https://microsoft.github.io/agent-lightning/stable/how-to/train-sql-agent/#__codelineno-14-1) python train_sql_agent.py npu` ### Debugging the Agent without VERL[¶](https://microsoft.github.io/agent-lightning/stable/how-to/train-sql-agent/#debugging-the-agent-without-verl "Permanent link") [`sql_agent.py`](https://github.com/microsoft/agent-lightning/tree/3b5d733861cf313fc09821a23240bbdf3cb2ee5b/examples/spider/sql_agent.py) also provides a `debug_sql_agent()` helper to run the LangGraph workflow directly against a local or hosted OpenAI-compatible endpoint before using VERL. Set the following environment variables, then execute the file: `[](https://microsoft.github.io/agent-lightning/stable/how-to/train-sql-agent/#__codelineno-15-1) export OPENAI_API_BASE= [](https://microsoft.github.io/agent-lightning/stable/how-to/train-sql-agent/#__codelineno-15-2) export OPENAI_API_KEY= [](https://microsoft.github.io/agent-lightning/stable/how-to/train-sql-agent/#__codelineno-15-3) cd examples/spider [](https://microsoft.github.io/agent-lightning/stable/how-to/train-sql-agent/#__codelineno-15-4) python sql_agent.py` This allows you to verify that the workflow and prompts behave as expected before reinforcement learning is introduced. ### Evaluation[¶](https://microsoft.github.io/agent-lightning/stable/how-to/train-sql-agent/#evaluation "Permanent link") The following results were obtained by running `python train_sql_agent.py qwen` on a single 80 GB GPU. Training completes in approximately **12 hours**. The training curves below are smoothed by aggregating every 16 steps for better visualization. Additional evaluation results were collected with a legacy version — Agent-lightning v0.1.1, `verl==0.5.0`, and `vllm==0.10.0`. You can find them in this write-up: [Training AI Agents to Write and Self-Correct SQL with Reinforcement Learning](https://medium.com/@yugez/training-ai-agents-to-write-and-self-correct-sql-with-reinforcement-learning-571ed31281ad) Back to top --- # Work with Traces - Agent-lightning [Skip to content](https://microsoft.github.io/agent-lightning/stable/tutorials/traces/#working-with-traces) ⚠️ You are viewing development documentation. For stable documentation, please visit [https://microsoft.github.io/agent-lightning/stable/](https://microsoft.github.io/agent-lightning/stable/) Working with Traces[¶](https://microsoft.github.io/agent-lightning/stable/tutorials/traces/#working-with-traces "Permanent link") ================================================================================================================================== Tracing is the secret capability that lets Agent-lightning train almost any agent without rewriting its core logic. The idea was born in observability tooling inside LLMOps workflows and, in Agent-lightning, evolved into a first-class primitive inside the learning loop. Beyond helping you understand what happened inside a rollout, traces provide reward spans and other learning signals that power reinforcement learning and fine-tuning algorithms. ![OpenTelemetry spans](https://microsoft.github.io/agent-lightning/stable/assets/opentelemetry-trace.jpg) Agent-lightning stores every recorded operation as a [`Span`](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.Span " agentlightning.Span") inside a [`LightningStore`](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.LightningStore " agentlightning.LightningStore") . The naming comes from [OpenTelemetry spans](https://opentelemetry.io/docs/concepts/signals/traces/) , shown in the screenshot above. A span can represent an LLM call, a tool invocation, a graph edge, an explicit reward emission, or an arbitrary Python code block. Spans form a tree where parent spans describe higher-level steps and children record the detailed work. The sections below walk through how spans are produced and how to interpret them once they reach the store. Writing Spans[¶](https://microsoft.github.io/agent-lightning/stable/tutorials/traces/#writing-spans "Permanent link") ---------------------------------------------------------------------------------------------------------------------- Most [`Runner`](https://microsoft.github.io/agent-lightning/stable/reference/runner/#agentlightning.Runner " agentlightning.Runner") implementations wire a [`Tracer`](https://microsoft.github.io/agent-lightning/stable/reference/runner/#agentlightning.Tracer " agentlightning.Tracer") into the agent’s lifecycle. The tracer is responsible for installing instrumentation, buffering OpenTelemetry spans, and committing them to the [`LightningStore`](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.LightningStore " agentlightning.LightningStore") . When a runner executes a rollout, it allocates a store-backed tracing context: `[](https://microsoft.github.io/agent-lightning/stable/tutorials/traces/#__codelineno-0-1) async with tracer.trace_context( [](https://microsoft.github.io/agent-lightning/stable/tutorials/traces/#__codelineno-0-2) name="my-rollout", [](https://microsoft.github.io/agent-lightning/stable/tutorials/traces/#__codelineno-0-3) store=store, [](https://microsoft.github.io/agent-lightning/stable/tutorials/traces/#__codelineno-0-4) rollout_id=rollout.rollout_id, [](https://microsoft.github.io/agent-lightning/stable/tutorials/traces/#__codelineno-0-5) attempt_id=attempt.attempt_id, [](https://microsoft.github.io/agent-lightning/stable/tutorials/traces/#__codelineno-0-6) ): [](https://microsoft.github.io/agent-lightning/stable/tutorials/traces/#__codelineno-0-7) await run_agent_logic()` The context manager then requests sequence numbers from the store, converts OpenTelemetry spans into [`Span`](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.Span " agentlightning.Span") objects, and persists them in the middle or at the end of the attempt, depending on the tracer implementation. Agent-lightning ships two tracers out of the box; both rely on [OpenTelemetry Traces](https://opentelemetry.io/docs/concepts/signals/traces/) and ignore metrics or logs. What's instrumentation? In simple terms, _instrumentation_ means adding "patches" or hooks inside your code so you can observe what it’s doing while it runs. Think of it like putting flight recorders in an airplane — instrumentation records key actions, inputs, outputs, and timings without changing how the code behaves. In Agent-lightning tracers, this instrumentation automatically creates spans (small, structured records of work) that show what each part of an agent did, how long it took, and how different steps connect together. ### AgentOps Tracer[¶](https://microsoft.github.io/agent-lightning/stable/tutorials/traces/#agentops-tracer "Permanent link") [`AgentOpsTracer`](https://microsoft.github.io/agent-lightning/stable/reference/runner/#agentlightning.AgentOpsTracer " agentlightning.AgentOpsTracer") will be the default tracer when [`Trainer`](https://microsoft.github.io/agent-lightning/stable/reference/trainer/#agentlightning.Trainer " agentlightning.Trainer") is used but no tracer is explicitly specified. It bootstraps the [AgentOps SDK](https://www.agentops.ai/) locally, installs the supplied instrumentation hooks (LangChain, LangGraph, LiteLLM, FastAPI, and others) provided by the [AgentOps Python SDK](https://github.com/AgentOps-AI/agentops) , and forwards everything through a local OpenTelemetry [`TracerProvider`](https://opentelemetry.io/docs/specs/otel/trace/api/) . [`AgentOpsTracer`](https://microsoft.github.io/agent-lightning/stable/reference/runner/#agentlightning.AgentOpsTracer " agentlightning.AgentOpsTracer") never calls the hosted AgentOps service; instead, it attaches a `LightningSpanProcessor` implemented by the Agent-lightning team so that spans are captured and shipped straight into the store. Because it shares the AgentOps instrumentation surface, any framework supported by AgentOps automatically gains tracing in Agent-lightning. We layer additional hooks on top of AgentOps to capture features that the SDK misses today: 1. Certain providers emit extra metadata — for example, [token IDs returned by vLLM](https://microsoft.github.io/agent-lightning/stable/deep-dive/serving-llm/) — that are not recorded by the stock SDK. We augment those spans with the missing payloads. 2. AgentOps constructs parent-child relationships on a best-effort basis, but mixed instrumentation (for example, OpenAI Agent SDK alongside direct OpenAI Chat Completion calls) can leave segments disconnected. Our implementation (actually implemented in the [`TracerTraceToTriplet`](https://microsoft.github.io/agent-lightning/stable/reference/algorithm/#agentlightning.TracerTraceToTriplet " agentlightning.TracerTraceToTriplet") adapter) repairs those relationships when the hierarchy can be inferred from rollout context. 3. Some versions of downstream frameworks simply do not emit spans for critical events (LangGraph node entrances are a common example). The tracer installs lightweight shims so those spans appear consistently. If a vendor integration behaves unexpectedly, users are encouraged to combine the tracer with [Hooks](https://microsoft.github.io/agent-lightning/stable/tutorials/debug/) to inspect the raw spans or diagnostics, and/or implement a specialized tracer for the framework in question. ### OpenTelemetry Tracer[¶](https://microsoft.github.io/agent-lightning/stable/tutorials/traces/#opentelemetry-tracer "Permanent link") [`OtelTracer`](https://microsoft.github.io/agent-lightning/stable/reference/runner/#agentlightning.OtelTracer " agentlightning.OtelTracer") is a minimal implementation that initializes a vanilla [`TracerProvider`](https://opentelemetry.io/docs/specs/otel/trace/api/) and gives you direct control over span creation through the standard `opentelemetry.trace` API. Use it when you already have explicit instrumentation in your agent, when the AgentOps SDK does not support your framework, or when you want to emit custom spans from business logic. Note [Microsoft Agent Framework](https://github.com/microsoft/agent-framework) is a typical example with built-in OpenTelemetry support. Once you set `OBSERVABILITY_SETTINGS.enable_otel = True`, the framework will automatically emit OpenTelemetry spans, and [`OtelTracer`](https://microsoft.github.io/agent-lightning/stable/reference/runner/#agentlightning.OtelTracer " agentlightning.OtelTracer") will be able to capture them. No extra instrumentation is needed. Inside your agent you can call `opentelemetry.trace.get_trace_provider().get_tracer("my-agent")` and use that tracer to [create spans](https://opentelemetry.io/docs/languages/python/cookbook/) exactly as you would in any OpenTelemetry application. The Lightning span processor attached by [`OtelTracer`](https://microsoft.github.io/agent-lightning/stable/reference/runner/#agentlightning.OtelTracer " agentlightning.OtelTracer") guarantees that every span is sequenced, converted, and written to the store. The same applies for emitted rewards ([`emit_reward`](https://microsoft.github.io/agent-lightning/stable/reference/agent/#agentlightning.emit_reward " agentlightning.emit_reward(reward, *, primary_key=None, attributes=None, propagate=True)") ) and other emitter signals, which are just a special case of manually-created spans. ### Weave Tracer (Experimental)[¶](https://microsoft.github.io/agent-lightning/stable/tutorials/traces/#weave-tracer-experimental "Permanent link") [`WeaveTracer`](https://microsoft.github.io/agent-lightning/stable/reference/runner/#agentlightning.tracer.weave.WeaveTracer " agentlightning.tracer.weave.WeaveTracer") is an experimental tracer that integrates with the [Weave Python SDK](https://docs.wandb.ai/weave) . Use it as a substitute for [`AgentOpsTracer`](https://microsoft.github.io/agent-lightning/stable/reference/runner/#agentlightning.AgentOpsTracer " agentlightning.AgentOpsTracer") when the AgentOps SDK does not fit your environment. The Weave SDK instruments LLM calls and agent libraries directly. Unlike [`AgentOpsTracer`](https://microsoft.github.io/agent-lightning/stable/reference/runner/#agentlightning.AgentOpsTracer " agentlightning.AgentOpsTracer") , Weave does not rely on OpenTelemetry to export spans; it routes everything through a dedicated Weave Trace Server. Agent-lightning implements a custom Weave Trace Server so every call captured by the Weave SDK can be persisted to the [`LightningStore`](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.LightningStore " agentlightning.LightningStore") . Warning [`WeaveTracer`](https://microsoft.github.io/agent-lightning/stable/reference/runner/#agentlightning.tracer.weave.WeaveTracer " agentlightning.tracer.weave.WeaveTracer") remains experimental and has not been tested as thoroughly as [`AgentOpsTracer`](https://microsoft.github.io/agent-lightning/stable/reference/runner/#agentlightning.AgentOpsTracer " agentlightning.AgentOpsTracer") . It may conflict with libraries that ship OpenTelemetry instrumentation by default (for example, LiteLLM-based LLM proxies). Use the tracer with caution and report any issues to the Agent-lightning team. ### LLM Proxy[¶](https://microsoft.github.io/agent-lightning/stable/tutorials/traces/#llm-proxy "Permanent link") Sometimes the runner can’t observe the agent directly — because it’s in another language or running remotely. [`LLMProxy`](https://microsoft.github.io/agent-lightning/stable/reference/algorithm/#agentlightning.LLMProxy " agentlightning.LLMProxy") bridges that gap by instrumenting the server side of LLM calls. It wraps [LiteLLM](https://docs.litellm.ai/) and adds middleware that accepts prefixed routes like `/rollout/{rid}/attempt/{aid}/v1/chat/completions`. Before forwarding, the middleware rewrites the path to `/v1/chat/completions`, fetches a monotonic `sequence_id` from the `LightningStore`, injects `x-rollout-id`, `x-attempt-id`, and `x-sequence-id` into the request headers, and then forwards the request to the backend LLM endpoint. LiteLLM produces OpenTelemetry spans for the request/response. A custom `LightningSpanExporter` reads the rollout/attempt/sequence identifiers from the recorded request headers and persists each span to the store. Because the `sequence_id` is allocated at the start of the request, traces stay in strict order even across machines with skewed clocks or asynchronous responses. sequenceDiagram participant Agent participant Proxy as LLM Proxy participant Backend as LLM Backend participant Store as LightningStore Agent->>Proxy: POST /rollout/{rid}/attempt/{aid}/v1/chat/completions Proxy->>Store: get_next_span_sequence_id(rid, aid) Store-->>Proxy: sequence_id Proxy->>Backend: Forward /v1/chat/completions
(headers: rid, aid, sid) Backend-->>Proxy: Response (tokens, usage, token_ids) Proxy->>Store: Export OTEL spans (rid, aid, sequence_id) Proxy-->>Agent: OpenAI-compatible response [`LLMProxy`](https://microsoft.github.io/agent-lightning/stable/reference/algorithm/#agentlightning.LLMProxy " agentlightning.LLMProxy") actually provides more functionalities than just the middleware for tracing. Read [Serving LLM](https://microsoft.github.io/agent-lightning/stable/deep-dive/serving-llm/) for more details. [](https://microsoft.github.io/agent-lightning/stable/tutorials/traces/) Distributed Tracing Agent-lightning enforces deterministic span ordering by assigning a monotonic [`sequence_id`](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.Span.sequence_id " sequence_id instance-attribute ") to every span within an attempt. Before calling [`LightningStore.add_span`](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.LightningStore.add_span " add_span(span) async ") or [`LightningStore.add_otel_span`](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.LightningStore.add_otel_span " add_otel_span(rollout_id, attempt_id, readable_span, sequence_id=None) async ") , tracers are expected to call [`LightningStore.get_next_span_sequence_id`](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.LightningStore.get_next_span_sequence_id " get_next_span_sequence_id(rollout_id, attempt_id) async ") to get the next sequence id. This removes clock skew and merges spans produced on different machines or threads. If you implement a custom tracer or exporter, make sure you do this (or respect the one provided in headers by components such as [`LLMProxy`](https://microsoft.github.io/agent-lightning/stable/reference/algorithm/#agentlightning.LLMProxy " agentlightning.LLMProxy") ); otherwise, adapters will struggle to properly reconstruct the execution tree. ### Custom Tracer[¶](https://microsoft.github.io/agent-lightning/stable/tutorials/traces/#custom-tracer "Permanent link") If none of the built-in tracers fit your environment, the first option to consider is to [return the spans](https://microsoft.github.io/agent-lightning/stable/tutorials/write-agents/) directly from your agent implementation. If that's not possible, or you want to support multiple agents in a unified effort, you can implement your own tracer by subclassing [`Tracer`](https://microsoft.github.io/agent-lightning/stable/reference/runner/#agentlightning.Tracer " agentlightning.Tracer") . Custom tracers must implement at least [`trace_context`](https://microsoft.github.io/agent-lightning/stable/reference/runner/#agentlightning.Tracer.trace_context " trace_context(name=None, *, store=None, rollout_id=None, attempt_id=None)") . The [`trace_context`](https://microsoft.github.io/agent-lightning/stable/reference/runner/#agentlightning.Tracer.trace_context " trace_context(name=None, *, store=None, rollout_id=None, attempt_id=None)") coroutine should install or activate whatever instrumentation you need, then yield a span processor that ultimately adds spans to the store. You can reuse the `LightningSpanProcessor` if you produce OpenTelemetry `ReadableSpan` objects, or call [`LightningStore.add_span`](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.LightningStore.add_span " add_span(span) async ") directly if you generate [`Span`](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.Span " agentlightning.Span") instances yourself. Advanced tracers often run auxiliary services (for example, starting a telemetry daemon or attaching to a container runtime) inside `init_worker` and tear them down in `teardown_worker`. The [`ParallelWorkerBase`](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.ParallelWorkerBase " agentlightning.ParallelWorkerBase") lifecycle that `Tracer` inherits from ensures those hooks are executed in every runner subprocess. Reading Traces[¶](https://microsoft.github.io/agent-lightning/stable/tutorials/traces/#reading-traces "Permanent link") ------------------------------------------------------------------------------------------------------------------------ Generally, there are two approaches to reading traces. When you only need a quick look, [`Tracer.get_last_trace`](https://microsoft.github.io/agent-lightning/stable/reference/runner/#agentlightning.Tracer.get_last_trace " get_last_trace()") returns the raw OpenTelemetry spans captured most recently. For historical analysis, use the [`LightningStore.query_spans`](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.LightningStore.query_spans " query_spans(rollout_id, attempt_id=None, *, trace_id=None, trace_id_contains=None, span_id=None, span_id_contains=None, parent_id=None, parent_id_contains=None, name=None, name_contains=None, filter_logic='and', limit=-1, offset=0, sort_by='sequence_id', sort_order='asc') async ") API, which yields normalized [`Span`](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.Span " agentlightning.Span") objects keyed by rollout ID and attempt ID. Combine those queries with [`LightningStore.query_rollouts`](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.LightningStore.query_rollouts " query_rollouts(*, status_in=None, rollout_id_in=None, rollout_id_contains=None, filter_logic='and', sort_by=None, sort_order='asc', limit=-1, offset=0, status=None, rollout_ids=None) async ") to align spans with rollout status, retries, and timing information. Spans arrive asynchronously, originate from different processes, and form hierarchies rather than simple lists. The attributes of each span are tedious and unfriendly to human readers. This combination makes raw traces time-consuming to inspect, especially when you only care about specific signals such as rewards, LLM prompts, responses, or tool outputs. Understanding how the store exposes traces and how adapters reshape them will save hours when debugging or training. Why traces can be difficult to read? The trace tree for a single rollout typically mixes multiple abstraction layers: a planner span may contain several LLM spans, each of which contains tool execution spans that can themselves trigger nested agent invocations. There are also instrumentations at different levels. For example, when a request delegates to another library (e.g., from LangChain to OpenAI), two libraries might emit spans for the same request. At the top level, there could be concurrently running agents that may flush spans slightly out of order. Sorting by `sequence_id` restores the chronological view, but interpreting the tree requires additional context about parent-child relationships and rollout metadata. ### Adapter[¶](https://microsoft.github.io/agent-lightning/stable/tutorials/traces/#adapter "Permanent link") [Adapters](https://microsoft.github.io/agent-lightning/stable/reference/algorithm/#agentlightning.Adapter " agentlightning.Adapter") transform lists of spans into higher-level data structures that training algorithms can consume directly. Agent-lightning provides several adapters out of the box: * [`TracerTraceToTriplet`](https://microsoft.github.io/agent-lightning/stable/reference/algorithm/#agentlightning.TracerTraceToTriplet " agentlightning.TracerTraceToTriplet") converts spans into `(prompt, response, reward)` triplets, which power reinforcement-learning algorithms such as [VERL](https://microsoft.github.io/agent-lightning/stable/algorithm-zoo/verl/) and connect trace data to gradient updates. * [`TraceToMessages`](https://microsoft.github.io/agent-lightning/stable/reference/algorithm/#agentlightning.TraceToMessages " agentlightning.TraceToMessages") rewrites spans into OpenAI chat message JSON suitable for supervised fine-tuning or evaluation harnesses. * [`LlmProxyTraceToTriplet`](https://microsoft.github.io/agent-lightning/stable/reference/algorithm/#agentlightning.LlmProxyTraceToTriplet " agentlightning.LlmProxyTraceToTriplet") mirrors [`TracerTraceToTriplet`](https://microsoft.github.io/agent-lightning/stable/reference/algorithm/#agentlightning.TracerTraceToTriplet " agentlightning.TracerTraceToTriplet") but understands spans emitted by [LLMProxy](https://microsoft.github.io/agent-lightning/stable/reference/algorithm/#agentlightning.LLMProxy " agentlightning.LLMProxy") . It is experimental and might be merged with [`TracerTraceToTriplet`](https://microsoft.github.io/agent-lightning/stable/reference/algorithm/#agentlightning.TracerTraceToTriplet " agentlightning.TracerTraceToTriplet") in the future. Adapters are regular Python callable instances, so you can plug them into [`Trainer`](https://microsoft.github.io/agent-lightning/stable/reference/trainer/#agentlightning.Trainer " agentlightning.Trainer") via the `adapter` argument, or call them manually during exploration. When used in [`Trainer`](https://microsoft.github.io/agent-lightning/stable/reference/trainer/#agentlightning.Trainer " agentlightning.Trainer") , adapters are bundled into the [`Algorithm`](https://microsoft.github.io/agent-lightning/stable/reference/algorithm/#agentlightning.Algorithm " agentlightning.Algorithm") before the algorithm runs, through the [`Algorithm.set_adapter`](https://microsoft.github.io/agent-lightning/stable/reference/algorithm/#agentlightning.Algorithm.set_adapter " set_adapter(adapter)") method. You can also customize an [`Adapter`](https://microsoft.github.io/agent-lightning/stable/reference/algorithm/#agentlightning.Adapter " agentlightning.Adapter") by extending the implementations above or subclassing the base class. If you need a bespoke format, subclass [`TraceAdapter`](https://microsoft.github.io/agent-lightning/stable/reference/algorithm/#agentlightning.TraceAdapter " agentlightning.TraceAdapter") (for store spans) or [`OtelTraceAdapter`](https://microsoft.github.io/agent-lightning/stable/reference/algorithm/#agentlightning.OtelTraceAdapter " agentlightning.OtelTraceAdapter") (for raw OpenTelemetry spans) and implement `adapt` (these two classes can usually share the same implementation). ### Reading Rewards[¶](https://microsoft.github.io/agent-lightning/stable/tutorials/traces/#reading-rewards "Permanent link") Rewards are recorded as dedicated spans named [`agentlightning.annotation`](https://microsoft.github.io/agent-lightning/stable/reference/semconv/#agentlightning.semconv.AGL_ANNOTATION " AGL_ANNOTATION = 'agentlightning.annotation' module-attribute ") . Emitting a reward through [`emit_reward`](https://microsoft.github.io/agent-lightning/stable/reference/agent/#agentlightning.emit_reward " agentlightning.emit_reward(reward, *, primary_key=None, attributes=None, propagate=True)") or [`emit_annotation`](https://microsoft.github.io/agent-lightning/stable/reference/agent/#agentlightning.emit_annotation " agentlightning.emit_annotation(annotation, propagate=True)") ensures the value is stored in the span’s `attributes`. To audit rewards, fetch spans from the store and use the helper utilities in [`agentlightning.emitter`](https://microsoft.github.io/agent-lightning/stable/reference/agent/) : `[](https://microsoft.github.io/agent-lightning/stable/tutorials/traces/#__codelineno-1-1) from agentlightning.emitter import find_final_reward [](https://microsoft.github.io/agent-lightning/stable/tutorials/traces/#__codelineno-1-2) [](https://microsoft.github.io/agent-lightning/stable/tutorials/traces/#__codelineno-1-3) spans = await store.query_spans(rollout_id) [](https://microsoft.github.io/agent-lightning/stable/tutorials/traces/#__codelineno-1-4) reward = find_final_reward(spans) [](https://microsoft.github.io/agent-lightning/stable/tutorials/traces/#__codelineno-1-5) print(f"Final reward: {reward}")` [`find_reward_spans`](https://microsoft.github.io/agent-lightning/stable/reference/agent/#agentlightning.find_reward_spans " agentlightning.find_reward_spans(spans)") returns every reward span so you can visualize intermediate shaping signals, while [`find_final_reward`](https://microsoft.github.io/agent-lightning/stable/reference/agent/#agentlightning.find_final_reward " agentlightning.find_final_reward(spans)") extracts the last non-null reward per attempt. While these helpers are convenient, they may not help you fully understand the chronological or hierarchical relationships between reward spans and other spans. Using an [`Adapter`](https://microsoft.github.io/agent-lightning/stable/reference/algorithm/#agentlightning.Adapter " agentlightning.Adapter") — especially the same one used in the algorithm you’re working with — remains the recommended way to inspect your generated spans. Back to top --- # Use Emitters - Agent-lightning [Skip to content](https://microsoft.github.io/agent-lightning/stable/tutorials/emitter/#using-emitters) ⚠️ You are viewing development documentation. For stable documentation, please visit [https://microsoft.github.io/agent-lightning/stable/](https://microsoft.github.io/agent-lightning/stable/) Using Emitters[¶](https://microsoft.github.io/agent-lightning/stable/tutorials/emitter/#using-emitters "Permanent link") ========================================================================================================================= [](https://microsoft.github.io/agent-lightning/stable/tutorials/emitter/) While returning a single float for the final reward is sufficient for many algorithm-agent combinations, some advanced scenarios require richer feedback. For instance, an algorithm might learn more effectively if it receives intermediate rewards throughout a multi-step task, or if the agent needs to emit additional spans for debugging or analysis. Agent-lightning provides an **emitter** module for recording custom spans inside your agent logic. Just as [Tracer](https://microsoft.github.io/agent-lightning/stable/reference/runner/#agentlightning.Tracer " agentlightning.Tracer") automatically instruments common operations (for example, LLM calls), each emitter helper sends a [Span](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.Span " agentlightning.Span") that captures Agent-lightning-specific work so downstream algorithms can query it later. See [Working with Traces](https://microsoft.github.io/agent-lightning/stable/tutorials/traces/) for more details. For multi-step routines such as function calls, tools, or adapters, wrap code with [`operation`](https://microsoft.github.io/agent-lightning/stable/reference/agent/#agentlightning.operation " agentlightning.operation(fn=None, *, propagate=True, name=None, **additional_attributes)") — either as a decorator or a context manager — to capture inputs, outputs, and metadata on a dedicated [`operation`](https://microsoft.github.io/agent-lightning/stable/reference/agent/#agentlightning.operation " agentlightning.operation(fn=None, *, propagate=True, name=None, **additional_attributes)") span. This makes it easier to correlate downstream annotations (like rewards or messages) with the higher-level work that produced them. You can find the emitter functions in [`agentlightning.emitter`](https://microsoft.github.io/agent-lightning/stable/reference/agent/) . Emitting Rewards, Messages, and More[¶](https://microsoft.github.io/agent-lightning/stable/tutorials/emitter/#emitting-rewards-messages-and-more "Permanent link") ------------------------------------------------------------------------------------------------------------------------------------------------------------------- Here are the primary emitter functions: * [`emit_reward(value: float)`](https://microsoft.github.io/agent-lightning/stable/reference/agent/#agentlightning.emit_reward " agentlightning.emit_reward(reward, *, primary_key=None, attributes=None, propagate=True)") : Records an intermediate/final reward, which is a convenient wrapper of [`emit_annotation`](https://microsoft.github.io/agent-lightning/stable/reference/agent/#agentlightning.emit_annotation " agentlightning.emit_annotation(annotation, propagate=True)") . * [`emit_annotation(attributes: Dict[str, Any])`](https://microsoft.github.io/agent-lightning/stable/reference/agent/#agentlightning.emit_annotation " agentlightning.emit_annotation(annotation, propagate=True)") : Records arbitrary metadata as a span. * [`emit_message(message: str)`](https://microsoft.github.io/agent-lightning/stable/reference/agent/#agentlightning.emit_message " agentlightning.emit_message(message, attributes=None, propagate=True)") : Records a simple log message as a span. * [`emit_exception(exception: BaseException)`](https://microsoft.github.io/agent-lightning/stable/reference/agent/#agentlightning.emit_exception " agentlightning.emit_exception(exception, attributes=None, propagate=True)") : Records a Python exception, including its type, message, and stack trace. * [`emit_object(obj: Any)`](https://microsoft.github.io/agent-lightning/stable/reference/agent/#agentlightning.emit_object " agentlightning.emit_object(object, attributes=None, propagate=True)") : Records any JSON-serializable object, perfect for structured data. Let's first see an example of an agent using these emitters to provide detailed feedback. `[](https://microsoft.github.io/agent-lightning/stable/tutorials/emitter/#__codelineno-0-1) import agentlightning as agl [](https://microsoft.github.io/agent-lightning/stable/tutorials/emitter/#__codelineno-0-2) [](https://microsoft.github.io/agent-lightning/stable/tutorials/emitter/#__codelineno-0-3) @agl.rollout [](https://microsoft.github.io/agent-lightning/stable/tutorials/emitter/#__codelineno-0-4) def multi_step_agent(task: dict, prompt_template: PromptTemplate) -> float: [](https://microsoft.github.io/agent-lightning/stable/tutorials/emitter/#__codelineno-0-5) try: [](https://microsoft.github.io/agent-lightning/stable/tutorials/emitter/#__codelineno-0-6) # Step 1: Initial planning [](https://microsoft.github.io/agent-lightning/stable/tutorials/emitter/#__codelineno-0-7) agl.emit_message("Starting planning phase.") [](https://microsoft.github.io/agent-lightning/stable/tutorials/emitter/#__codelineno-0-8) plan = generate_plan(task, prompt_template) [](https://microsoft.github.io/agent-lightning/stable/tutorials/emitter/#__codelineno-0-9) agl.emit_object({"plan_steps": len(plan), "first_step": plan[0]}) [](https://microsoft.github.io/agent-lightning/stable/tutorials/emitter/#__codelineno-0-10) [](https://microsoft.github.io/agent-lightning/stable/tutorials/emitter/#__codelineno-0-11) # Award a small reward for a valid plan [](https://microsoft.github.io/agent-lightning/stable/tutorials/emitter/#__codelineno-0-12) plan_reward = grade_plan(plan) [](https://microsoft.github.io/agent-lightning/stable/tutorials/emitter/#__codelineno-0-13) agl.emit_reward(plan_reward) [](https://microsoft.github.io/agent-lightning/stable/tutorials/emitter/#__codelineno-0-14) [](https://microsoft.github.io/agent-lightning/stable/tutorials/emitter/#__codelineno-0-15) # Step 2: Execute the plan [](https://microsoft.github.io/agent-lightning/stable/tutorials/emitter/#__codelineno-0-16) agl.emit_message(f"Executing {len(plan)}-step plan.") [](https://microsoft.github.io/agent-lightning/stable/tutorials/emitter/#__codelineno-0-17) execution_result = execute_plan(plan) [](https://microsoft.github.io/agent-lightning/stable/tutorials/emitter/#__codelineno-0-18) [](https://microsoft.github.io/agent-lightning/stable/tutorials/emitter/#__codelineno-0-19) # Step 3: Final evaluation [](https://microsoft.github.io/agent-lightning/stable/tutorials/emitter/#__codelineno-0-20) final_reward = custom_grade_final_result(execution_result, task["expected_output"]) [](https://microsoft.github.io/agent-lightning/stable/tutorials/emitter/#__codelineno-0-21) [](https://microsoft.github.io/agent-lightning/stable/tutorials/emitter/#__codelineno-0-22) # The return value is treated as the final reward for the rollout [](https://microsoft.github.io/agent-lightning/stable/tutorials/emitter/#__codelineno-0-23) return final_reward [](https://microsoft.github.io/agent-lightning/stable/tutorials/emitter/#__codelineno-0-24) [](https://microsoft.github.io/agent-lightning/stable/tutorials/emitter/#__codelineno-0-25) except ValueError as e: [](https://microsoft.github.io/agent-lightning/stable/tutorials/emitter/#__codelineno-0-26) # Record the specific error and return a failure reward [](https://microsoft.github.io/agent-lightning/stable/tutorials/emitter/#__codelineno-0-27) agl.emit_exception(e) [](https://microsoft.github.io/agent-lightning/stable/tutorials/emitter/#__codelineno-0-28) return 0.0` Each helper accepts nested `attributes` (or keyword arguments for [`operation`](https://microsoft.github.io/agent-lightning/stable/reference/agent/#agentlightning.operation " agentlightning.operation(fn=None, *, propagate=True, name=None, **additional_attributes)") ) and automatically flattens/sanitizes them into dotted OpenTelemetry keys. This means you can pass ordinary dictionaries/lists without pre-processing and still get consistent attribute names such as `meta.any_attribute` across all emitter operations. Agent-lightning does not restrict the attributes you supply, but it is best to consult [OpenTelemetry's semantic conventions](https://opentelemetry.io/docs/specs/semconv/) for recommended names. Agent-lightning also defines [specific semconv](https://microsoft.github.io/agent-lightning/stable/reference/semconv/) for its own use cases. The pattern looks like this: `[](https://microsoft.github.io/agent-lightning/stable/tutorials/emitter/#__codelineno-1-1) from opentelemetry.semconv.attributes import server_attributes [](https://microsoft.github.io/agent-lightning/stable/tutorials/emitter/#__codelineno-1-2) from agentlightning import emit_object [](https://microsoft.github.io/agent-lightning/stable/tutorials/emitter/#__codelineno-1-3) [](https://microsoft.github.io/agent-lightning/stable/tutorials/emitter/#__codelineno-1-4) emit_object({ [](https://microsoft.github.io/agent-lightning/stable/tutorials/emitter/#__codelineno-1-5) "name": "John Doe", [](https://microsoft.github.io/agent-lightning/stable/tutorials/emitter/#__codelineno-1-6) "age": 30, [](https://microsoft.github.io/agent-lightning/stable/tutorials/emitter/#__codelineno-1-7) "email": "john.doe@example.com", [](https://microsoft.github.io/agent-lightning/stable/tutorials/emitter/#__codelineno-1-8) }, attributes={ [](https://microsoft.github.io/agent-lightning/stable/tutorials/emitter/#__codelineno-1-9) server_attributes.SERVER_ADDRESS: "127.0.0.1", [](https://microsoft.github.io/agent-lightning/stable/tutorials/emitter/#__codelineno-1-10) server_attributes.SERVER_PORT: 8080, [](https://microsoft.github.io/agent-lightning/stable/tutorials/emitter/#__codelineno-1-11) })` Running the above code sends the following span to the backend if you have a tracer active: `[](https://microsoft.github.io/agent-lightning/stable/tutorials/emitter/#__codelineno-2-1) Span( [](https://microsoft.github.io/agent-lightning/stable/tutorials/emitter/#__codelineno-2-2) name='agentlightning.object', [](https://microsoft.github.io/agent-lightning/stable/tutorials/emitter/#__codelineno-2-3) attributes={ [](https://microsoft.github.io/agent-lightning/stable/tutorials/emitter/#__codelineno-2-4) 'agentlightning.object.type': 'dict', [](https://microsoft.github.io/agent-lightning/stable/tutorials/emitter/#__codelineno-2-5) 'agentlightning.object.json': '{"name": "John Doe", "age": 30, "email": "john.doe@example.com"}', [](https://microsoft.github.io/agent-lightning/stable/tutorials/emitter/#__codelineno-2-6) 'server.address': '127.0.0.1', [](https://microsoft.github.io/agent-lightning/stable/tutorials/emitter/#__codelineno-2-7) 'server.port': 8080 [](https://microsoft.github.io/agent-lightning/stable/tutorials/emitter/#__codelineno-2-8) } [](https://microsoft.github.io/agent-lightning/stable/tutorials/emitter/#__codelineno-2-9) )` Tip If you don't have a tracer active, the above code will raise the following error: `[](https://microsoft.github.io/agent-lightning/stable/tutorials/emitter/#__codelineno-3-1) RuntimeError: No active tracer found. Cannot emit object span.` By default, emitter helpers delegate to the active tracer to create and export spans (specifically via [`Tracer.create_span`](https://microsoft.github.io/agent-lightning/stable/reference/runner/#agentlightning.Tracer.create_span " create_span(name, attributes=None, timestamp=None, status=None)") ). If you want to emit spans without an active tracer, set `propagate=False` to keep the span local — a useful option for offline tests. The default `True` streams spans through the active tracer/exporters. When working with [agentlightning.semconv](https://microsoft.github.io/agent-lightning/stable/reference/semconv/) , you typically use utilities such as [`make_tag_attributes`](https://microsoft.github.io/agent-lightning/stable/reference/utilities/#agentlightning.utils.otel.make_tag_attributes " agentlightning.utils.otel.make_tag_attributes(tags)") and [`make_link_attributes`](https://microsoft.github.io/agent-lightning/stable/reference/utilities/#agentlightning.utils.otel.make_link_attributes " agentlightning.utils.otel.make_link_attributes(links)") to build the attributes dictionary. For example: `[](https://microsoft.github.io/agent-lightning/stable/tutorials/emitter/#__codelineno-4-1) from agentlightning.utils.otel import make_tag_attributes [](https://microsoft.github.io/agent-lightning/stable/tutorials/emitter/#__codelineno-4-2) [](https://microsoft.github.io/agent-lightning/stable/tutorials/emitter/#__codelineno-4-3) emit_annotation(make_tag_attributes(["tool", "calculator", "fast", "good"]))` The above code will send a span with the following attributes to the backend: `[](https://microsoft.github.io/agent-lightning/stable/tutorials/emitter/#__codelineno-5-1) { [](https://microsoft.github.io/agent-lightning/stable/tutorials/emitter/#__codelineno-5-2) "agentlightning.tag.0": "tool", [](https://microsoft.github.io/agent-lightning/stable/tutorials/emitter/#__codelineno-5-3) "agentlightning.tag.1": "calculator", [](https://microsoft.github.io/agent-lightning/stable/tutorials/emitter/#__codelineno-5-4) "agentlightning.tag.2": "fast", [](https://microsoft.github.io/agent-lightning/stable/tutorials/emitter/#__codelineno-5-5) "agentlightning.tag.3": "good" [](https://microsoft.github.io/agent-lightning/stable/tutorials/emitter/#__codelineno-5-6) }` A counterpart utility function [`extract_tags_from_attributes`](https://microsoft.github.io/agent-lightning/stable/reference/utilities/#agentlightning.utils.otel.extract_tags_from_attributes " agentlightning.utils.otel.extract_tags_from_attributes(attributes)") is also available to extract the tags from the attributes dictionary. Operations[¶](https://microsoft.github.io/agent-lightning/stable/tutorials/emitter/#operations "Permanent link") ----------------------------------------------------------------------------------------------------------------- The [`operation`](https://microsoft.github.io/agent-lightning/stable/reference/agent/#agentlightning.operation " agentlightning.operation(fn=None, *, propagate=True, name=None, **additional_attributes)") helper tracks logical units of work within your agent, capturing inputs, outputs, timing, and success/failure status. Unlike point-in-time emitters, operations create a span representing a time interval. Use operations for tool calls, multi-step workflows, debugging, and performance monitoring. [`operation`](https://microsoft.github.io/agent-lightning/stable/reference/agent/#agentlightning.operation " agentlightning.operation(fn=None, *, propagate=True, name=None, **additional_attributes)") works as either a decorator or a context manager. The decorator automatically captures function arguments as inputs and the return value as output: `[](https://microsoft.github.io/agent-lightning/stable/tutorials/emitter/#__codelineno-6-1) import agentlightning as agl [](https://microsoft.github.io/agent-lightning/stable/tutorials/emitter/#__codelineno-6-2) [](https://microsoft.github.io/agent-lightning/stable/tutorials/emitter/#__codelineno-6-3) @agl.operation [](https://microsoft.github.io/agent-lightning/stable/tutorials/emitter/#__codelineno-6-4) def search_documents(query: str, max_results: int = 10) -> list[dict]: [](https://microsoft.github.io/agent-lightning/stable/tutorials/emitter/#__codelineno-6-5) results = perform_search(query, max_results) [](https://microsoft.github.io/agent-lightning/stable/tutorials/emitter/#__codelineno-6-6) return results [](https://microsoft.github.io/agent-lightning/stable/tutorials/emitter/#__codelineno-6-7) [](https://microsoft.github.io/agent-lightning/stable/tutorials/emitter/#__codelineno-6-8) @agl.operation(category="tool", priority="high") [](https://microsoft.github.io/agent-lightning/stable/tutorials/emitter/#__codelineno-6-9) def execute_calculation(expression: str) -> float: [](https://microsoft.github.io/agent-lightning/stable/tutorials/emitter/#__codelineno-6-10) return eval_safely(expression)` The example above emits a span with `{"category": "tool", "priority": "high"}` attributes. It also records the function input and output via [OPERATION\_INPUT](https://microsoft.github.io/agent-lightning/stable/reference/semconv/#agentlightning.semconv.LightningSpanAttributes.OPERATION_INPUT " OPERATION_INPUT = 'agentlightning.operation.input' class-attribute instance-attribute ") and [OPERATION\_OUTPUT](https://microsoft.github.io/agent-lightning/stable/reference/semconv/#agentlightning.semconv.LightningSpanAttributes.OPERATION_OUTPUT " OPERATION_OUTPUT = 'agentlightning.operation.output' class-attribute instance-attribute ") . It works with async functions too: `[](https://microsoft.github.io/agent-lightning/stable/tutorials/emitter/#__codelineno-7-1) @agl.operation [](https://microsoft.github.io/agent-lightning/stable/tutorials/emitter/#__codelineno-7-2) async def async_api_call(endpoint: str, payload: dict) -> dict: [](https://microsoft.github.io/agent-lightning/stable/tutorials/emitter/#__codelineno-7-3) response = await http_client.post(endpoint, json=payload) [](https://microsoft.github.io/agent-lightning/stable/tutorials/emitter/#__codelineno-7-4) return response.json()` Override the operation name if needed: `[](https://microsoft.github.io/agent-lightning/stable/tutorials/emitter/#__codelineno-8-1) @agl.operation(name="custom-name") [](https://microsoft.github.io/agent-lightning/stable/tutorials/emitter/#__codelineno-8-2) def any_weird_name_i_dont_want(): [](https://microsoft.github.io/agent-lightning/stable/tutorials/emitter/#__codelineno-8-3) pass` For more control, [`operation`](https://microsoft.github.io/agent-lightning/stable/reference/agent/#agentlightning.operation " agentlightning.operation(fn=None, *, propagate=True, name=None, **additional_attributes)") can also be used as a context manager to explicitly record inputs and outputs: `[](https://microsoft.github.io/agent-lightning/stable/tutorials/emitter/#__codelineno-9-1) with agl.operation(tool_name="web_search") as op: [](https://microsoft.github.io/agent-lightning/stable/tutorials/emitter/#__codelineno-9-2) op.set_input(query="latest AI research", filters={"date": "2024"}) [](https://microsoft.github.io/agent-lightning/stable/tutorials/emitter/#__codelineno-9-3) results = search_web("latest AI research", {"date": "2024"}) [](https://microsoft.github.io/agent-lightning/stable/tutorials/emitter/#__codelineno-9-4) op.set_output({"result_count": len(results), "top_result": results[0]})` The `propagate=False` flag also applies to [`operation`](https://microsoft.github.io/agent-lightning/stable/reference/agent/#agentlightning.operation " agentlightning.operation(fn=None, *, propagate=True, name=None, **additional_attributes)") when you want to keep operations local without requiring an active tracer: `[](https://microsoft.github.io/agent-lightning/stable/tutorials/emitter/#__codelineno-10-1) @agl.operation(propagate=False) [](https://microsoft.github.io/agent-lightning/stable/tutorials/emitter/#__codelineno-10-2) def local_test(): [](https://microsoft.github.io/agent-lightning/stable/tutorials/emitter/#__codelineno-10-3) return "Not sent to backend"` Linking to Other Spans[¶](https://microsoft.github.io/agent-lightning/stable/tutorials/emitter/#linking-to-other-spans "Permanent link") ----------------------------------------------------------------------------------------------------------------------------------------- Sometimes a span should explicitly point back to another span that produced the input it is working on (for example, linking a reward annotation to the [`agentlightning.operation`](https://microsoft.github.io/agent-lightning/stable/reference/agent/#agentlightning.operation " agentlightning.operation(fn=None, *, propagate=True, name=None, **additional_attributes)") span that generated a response). Agent-lightning encodes these relationships through flattened link attributes. The helper [`make_link_attributes`](https://microsoft.github.io/agent-lightning/stable/reference/utilities/#agentlightning.utils.otel.make_link_attributes " agentlightning.utils.otel.make_link_attributes(links)") converts a dictionary of keys such as `trace_id`, `span_id`, or any custom attribute into the `"agentlightning.link.*"` ([LightningSpanAttributes.LINK](https://microsoft.github.io/agent-lightning/stable/reference/semconv/#agentlightning.semconv.LightningSpanAttributes.LINK " LINK = 'agentlightning.link' class-attribute instance-attribute ") ) fields expected by the backend. Later, [`query_linked_spans`](https://microsoft.github.io/agent-lightning/stable/reference/utilities/#agentlightning.utils.otel.query_linked_spans " agentlightning.utils.otel.query_linked_spans(spans, links)") can recover the original span(s) from those link descriptors. `[](https://microsoft.github.io/agent-lightning/stable/tutorials/emitter/#__codelineno-11-1) import opentelemetry.trace as trace_api [](https://microsoft.github.io/agent-lightning/stable/tutorials/emitter/#__codelineno-11-2) from agentlightning import emit_annotation, operation [](https://microsoft.github.io/agent-lightning/stable/tutorials/emitter/#__codelineno-11-3) from agentlightning.utils.otel import make_link_attributes, make_tag_attributes [](https://microsoft.github.io/agent-lightning/stable/tutorials/emitter/#__codelineno-11-4) [](https://microsoft.github.io/agent-lightning/stable/tutorials/emitter/#__codelineno-11-5) with operation(conversation_id="chat-42") as op: [](https://microsoft.github.io/agent-lightning/stable/tutorials/emitter/#__codelineno-11-6) # ... perform the work ... [](https://microsoft.github.io/agent-lightning/stable/tutorials/emitter/#__codelineno-11-7) link_attrs = make_link_attributes({ [](https://microsoft.github.io/agent-lightning/stable/tutorials/emitter/#__codelineno-11-8) "conversation_id": "chat-42", [](https://microsoft.github.io/agent-lightning/stable/tutorials/emitter/#__codelineno-11-9) }) [](https://microsoft.github.io/agent-lightning/stable/tutorials/emitter/#__codelineno-11-10) [](https://microsoft.github.io/agent-lightning/stable/tutorials/emitter/#__codelineno-11-11) emit_annotation( [](https://microsoft.github.io/agent-lightning/stable/tutorials/emitter/#__codelineno-11-12) { [](https://microsoft.github.io/agent-lightning/stable/tutorials/emitter/#__codelineno-11-13) **link_attrs, [](https://microsoft.github.io/agent-lightning/stable/tutorials/emitter/#__codelineno-11-14) **make_tag_attributes(["reward", "good"]), [](https://microsoft.github.io/agent-lightning/stable/tutorials/emitter/#__codelineno-11-15) } [](https://microsoft.github.io/agent-lightning/stable/tutorials/emitter/#__codelineno-11-16) )` When analyzing in adapters, pass the extracted link models to [`query_linked_spans`](https://microsoft.github.io/agent-lightning/stable/reference/utilities/#agentlightning.utils.otel.query_linked_spans " agentlightning.utils.otel.query_linked_spans(spans, links)") to retrieve the matching span(s): `[](https://microsoft.github.io/agent-lightning/stable/tutorials/emitter/#__codelineno-12-1) from agentlightning.utils.otel import extract_links_from_attributes, query_linked_spans [](https://microsoft.github.io/agent-lightning/stable/tutorials/emitter/#__codelineno-12-2) [](https://microsoft.github.io/agent-lightning/stable/tutorials/emitter/#__codelineno-12-3) annotation_span = ... # Span from your trace store [](https://microsoft.github.io/agent-lightning/stable/tutorials/emitter/#__codelineno-12-4) operation_spans = [...] # list of spans you want to search [](https://microsoft.github.io/agent-lightning/stable/tutorials/emitter/#__codelineno-12-5) [](https://microsoft.github.io/agent-lightning/stable/tutorials/emitter/#__codelineno-12-6) link_models = extract_links_from_attributes(annotation_span.attributes) [](https://microsoft.github.io/agent-lightning/stable/tutorials/emitter/#__codelineno-12-7) matches = query_linked_spans(operation_spans, link_models) [](https://microsoft.github.io/agent-lightning/stable/tutorials/emitter/#__codelineno-12-8) assert matches # Contains the original operation span` Correlating Rewards with LLM Requests [Tracer](https://microsoft.github.io/agent-lightning/stable/tutorials/traces/) instruments each request/response as its own span. You can link to the [`gen_ai.response.id`](https://opentelemetry.io/docs/specs/semconv/gen-ai/gen-ai-events/) attribute, which comes from the LLM response ID. `[](https://microsoft.github.io/agent-lightning/stable/tutorials/emitter/#__codelineno-13-1) from agentlightning import emit_reward [](https://microsoft.github.io/agent-lightning/stable/tutorials/emitter/#__codelineno-13-2) from agentlightning.utils.otel import make_link_attributes [](https://microsoft.github.io/agent-lightning/stable/tutorials/emitter/#__codelineno-13-3) [](https://microsoft.github.io/agent-lightning/stable/tutorials/emitter/#__codelineno-13-4) result = call_llm(prompt) [](https://microsoft.github.io/agent-lightning/stable/tutorials/emitter/#__codelineno-13-5) reward_links = make_link_attributes({"gen_ai.response.id": result.id}) [](https://microsoft.github.io/agent-lightning/stable/tutorials/emitter/#__codelineno-13-6) emit_reward(0.9, attributes=reward_links)` Later, use the same `gen_ai.response.id` key inside `query_linked_spans` to find the reward(s) that reference that specific LLM request span. Back to top --- # Command Line - Agent-lightning [Skip to content](https://microsoft.github.io/agent-lightning/stable/reference/cli/#command-line-interface) ⚠️ You are viewing development documentation. For stable documentation, please visit [https://microsoft.github.io/agent-lightning/stable/](https://microsoft.github.io/agent-lightning/stable/) Command Line Interface[¶](https://microsoft.github.io/agent-lightning/stable/reference/cli/#command-line-interface "Permanent link") ===================================================================================================================================== Warning This document is a work in progress and might not be updated with the latest changes. Try to use `agl -h` to get the latest help message. Tip Agent-lightning also provides utilities to help you build your own CLI for [LitAgent](https://microsoft.github.io/agent-lightning/stable/reference/agent/#agentlightning.LitAgent " agentlightning.LitAgent") and [Trainer](https://microsoft.github.io/agent-lightning/stable/reference/trainer/#agentlightning.Trainer " agentlightning.Trainer") . See [Trainer](https://microsoft.github.io/agent-lightning/stable/reference/trainer/) for references. agl[¶](https://microsoft.github.io/agent-lightning/stable/reference/cli/#agl "Permanent link") ----------------------------------------------------------------------------------------------- `[](https://microsoft.github.io/agent-lightning/stable/reference/cli/#__codelineno-0-1) usage: agl [-h] {vllm,store,prometheus,agentops} [](https://microsoft.github.io/agent-lightning/stable/reference/cli/#__codelineno-0-2) [](https://microsoft.github.io/agent-lightning/stable/reference/cli/#__codelineno-0-3) Agent Lightning CLI entry point. [](https://microsoft.github.io/agent-lightning/stable/reference/cli/#__codelineno-0-4) [](https://microsoft.github.io/agent-lightning/stable/reference/cli/#__codelineno-0-5) Available subcommands: [](https://microsoft.github.io/agent-lightning/stable/reference/cli/#__codelineno-0-6) vllm Run the vLLM CLI with Agent Lightning instrumentation. [](https://microsoft.github.io/agent-lightning/stable/reference/cli/#__codelineno-0-7) store Run a LightningStore server. [](https://microsoft.github.io/agent-lightning/stable/reference/cli/#__codelineno-0-8) prometheus Serve Prometheus metrics from the multiprocess registry. [](https://microsoft.github.io/agent-lightning/stable/reference/cli/#__codelineno-0-9) agentops Start the AgentOps server manager. [](https://microsoft.github.io/agent-lightning/stable/reference/cli/#__codelineno-0-10) [](https://microsoft.github.io/agent-lightning/stable/reference/cli/#__codelineno-0-11) positional arguments: [](https://microsoft.github.io/agent-lightning/stable/reference/cli/#__codelineno-0-12) {vllm,store,prometheus,agentops} [](https://microsoft.github.io/agent-lightning/stable/reference/cli/#__codelineno-0-13) Subcommand to run. [](https://microsoft.github.io/agent-lightning/stable/reference/cli/#__codelineno-0-14) [](https://microsoft.github.io/agent-lightning/stable/reference/cli/#__codelineno-0-15) options: [](https://microsoft.github.io/agent-lightning/stable/reference/cli/#__codelineno-0-16) -h, --help show this help message and exit` agl vllm[¶](https://microsoft.github.io/agent-lightning/stable/reference/cli/#agl-vllm "Permanent link") --------------------------------------------------------------------------------------------------------- Agent-lightning's instrumented vLLM CLI. `[](https://microsoft.github.io/agent-lightning/stable/reference/cli/#__codelineno-1-1) usage: agl vllm [-h] [-v] {chat,complete,serve,bench,collect-env,run-batch} ... [](https://microsoft.github.io/agent-lightning/stable/reference/cli/#__codelineno-1-2) [](https://microsoft.github.io/agent-lightning/stable/reference/cli/#__codelineno-1-3) vLLM CLI [](https://microsoft.github.io/agent-lightning/stable/reference/cli/#__codelineno-1-4) [](https://microsoft.github.io/agent-lightning/stable/reference/cli/#__codelineno-1-5) positional arguments: [](https://microsoft.github.io/agent-lightning/stable/reference/cli/#__codelineno-1-6) {chat,complete,serve,bench,collect-env,run-batch} [](https://microsoft.github.io/agent-lightning/stable/reference/cli/#__codelineno-1-7) chat Generate chat completions via the running API server. [](https://microsoft.github.io/agent-lightning/stable/reference/cli/#__codelineno-1-8) complete Generate text completions based on the given prompt via the running API server. [](https://microsoft.github.io/agent-lightning/stable/reference/cli/#__codelineno-1-9) collect-env Start collecting environment information. [](https://microsoft.github.io/agent-lightning/stable/reference/cli/#__codelineno-1-10) run-batch Run batch prompts and write results to file. [](https://microsoft.github.io/agent-lightning/stable/reference/cli/#__codelineno-1-11) [](https://microsoft.github.io/agent-lightning/stable/reference/cli/#__codelineno-1-12) options: [](https://microsoft.github.io/agent-lightning/stable/reference/cli/#__codelineno-1-13) -h, --help show this help message and exit [](https://microsoft.github.io/agent-lightning/stable/reference/cli/#__codelineno-1-14) -v, --version show program's version number and exit [](https://microsoft.github.io/agent-lightning/stable/reference/cli/#__codelineno-1-15) [](https://microsoft.github.io/agent-lightning/stable/reference/cli/#__codelineno-1-16) For full list: vllm [subcommand] --help=all [](https://microsoft.github.io/agent-lightning/stable/reference/cli/#__codelineno-1-17) For a section: vllm [subcommand] --help=ModelConfig (case-insensitive) [](https://microsoft.github.io/agent-lightning/stable/reference/cli/#__codelineno-1-18) For a flag: vllm [subcommand] --help=max-model-len (_ or - accepted) [](https://microsoft.github.io/agent-lightning/stable/reference/cli/#__codelineno-1-19) Documentation: https://docs.vllm.ai` agl store[¶](https://microsoft.github.io/agent-lightning/stable/reference/cli/#agl-store "Permanent link") ----------------------------------------------------------------------------------------------------------- Agent-lightning's LightningStore CLI. Use it to start an independent LightningStore server. Currently the store data are stored in memory and will be lost when the server is stopped. ``[](https://microsoft.github.io/agent-lightning/stable/reference/cli/#__codelineno-2-1) usage: agl store [-h] [--host HOST] [--port PORT] [--cors-origin CORS_ORIGINS] [--log-level {DEBUG,INFO,WARNING,ERROR}] [--tracker {prometheus,console} [{prometheus,console} ...]] [--n-workers N_WORKERS] [--backend {memory,mongo}] [](https://microsoft.github.io/agent-lightning/stable/reference/cli/#__codelineno-2-2) [--mongo-uri MONGO_URI] [](https://microsoft.github.io/agent-lightning/stable/reference/cli/#__codelineno-2-3) [](https://microsoft.github.io/agent-lightning/stable/reference/cli/#__codelineno-2-4) Run a LightningStore server [](https://microsoft.github.io/agent-lightning/stable/reference/cli/#__codelineno-2-5) [](https://microsoft.github.io/agent-lightning/stable/reference/cli/#__codelineno-2-6) options: [](https://microsoft.github.io/agent-lightning/stable/reference/cli/#__codelineno-2-7) -h, --help show this help message and exit [](https://microsoft.github.io/agent-lightning/stable/reference/cli/#__codelineno-2-8) --host HOST Host to bind the server to [](https://microsoft.github.io/agent-lightning/stable/reference/cli/#__codelineno-2-9) --port PORT Port to run the server on [](https://microsoft.github.io/agent-lightning/stable/reference/cli/#__codelineno-2-10) --cors-origin CORS_ORIGINS [](https://microsoft.github.io/agent-lightning/stable/reference/cli/#__codelineno-2-11) Allowed CORS origin. Repeat for multiple origins. Use '*' to allow all origins. [](https://microsoft.github.io/agent-lightning/stable/reference/cli/#__codelineno-2-12) --log-level {DEBUG,INFO,WARNING,ERROR} [](https://microsoft.github.io/agent-lightning/stable/reference/cli/#__codelineno-2-13) Configure the logging level for the store. [](https://microsoft.github.io/agent-lightning/stable/reference/cli/#__codelineno-2-14) --tracker {prometheus,console} [{prometheus,console} ...] [](https://microsoft.github.io/agent-lightning/stable/reference/cli/#__codelineno-2-15) Enable metrics tracking. Repeat for multiple trackers. [](https://microsoft.github.io/agent-lightning/stable/reference/cli/#__codelineno-2-16) --n-workers N_WORKERS [](https://microsoft.github.io/agent-lightning/stable/reference/cli/#__codelineno-2-17) Number of workers to run in the server. When it's greater than 1, the server will be run using `mp` launch mode. Only applicable for zero-copy stores such as MongoDB backend. [](https://microsoft.github.io/agent-lightning/stable/reference/cli/#__codelineno-2-18) --backend {memory,mongo} [](https://microsoft.github.io/agent-lightning/stable/reference/cli/#__codelineno-2-19) Backend to use for the store. [](https://microsoft.github.io/agent-lightning/stable/reference/cli/#__codelineno-2-20) --mongo-uri MONGO_URI [](https://microsoft.github.io/agent-lightning/stable/reference/cli/#__codelineno-2-21) MongoDB URI to use for the store. Applicable only if --backend is 'mongo'.`` Tip After launching the store via CLI, you can tell the [`Trainer`](https://microsoft.github.io/agent-lightning/stable/reference/trainer/#agentlightning.Trainer " agentlightning.Trainer") to use the store by passing the store address to the trainer. `[](https://microsoft.github.io/agent-lightning/stable/reference/cli/#__codelineno-3-1) store_client = agl.LightningStoreClient("http://localhost:4747") [](https://microsoft.github.io/agent-lightning/stable/reference/cli/#__codelineno-3-2) trainer = agl.Trainer(store=store_client, ...)` See [using external store](https://microsoft.github.io/agent-lightning/stable/tutorials/debug/#debug-with-external-store "Debug the Algorithm-Runner Boundary") for more details. agl prometheus[¶](https://microsoft.github.io/agent-lightning/stable/reference/cli/#agl-prometheus "Permanent link") --------------------------------------------------------------------------------------------------------------------- Expose the Prometheus multiprocess registry on a dedicated FastAPI server. This is useful when the main LightningStore service is under heavy load; exporters can scrape this auxiliary endpoint instead. `[](https://microsoft.github.io/agent-lightning/stable/reference/cli/#__codelineno-4-1) usage: agl prometheus [-h] [--host HOST] [--port PORT] [--metrics-path METRICS_PATH] [--log-level {DEBUG,INFO,WARNING,ERROR}] [--access-log] [](https://microsoft.github.io/agent-lightning/stable/reference/cli/#__codelineno-4-2) [](https://microsoft.github.io/agent-lightning/stable/reference/cli/#__codelineno-4-3) Serve Prometheus metrics outside the LightningStore server. [](https://microsoft.github.io/agent-lightning/stable/reference/cli/#__codelineno-4-4) [](https://microsoft.github.io/agent-lightning/stable/reference/cli/#__codelineno-4-5) options: [](https://microsoft.github.io/agent-lightning/stable/reference/cli/#__codelineno-4-6) -h, --help show this help message and exit [](https://microsoft.github.io/agent-lightning/stable/reference/cli/#__codelineno-4-7) --host HOST Host to bind the metrics server to. [](https://microsoft.github.io/agent-lightning/stable/reference/cli/#__codelineno-4-8) --port PORT Port to expose the Prometheus metrics on. [](https://microsoft.github.io/agent-lightning/stable/reference/cli/#__codelineno-4-9) --metrics-path METRICS_PATH [](https://microsoft.github.io/agent-lightning/stable/reference/cli/#__codelineno-4-10) HTTP path used to expose metrics. Must start with '/' and not be the root path. [](https://microsoft.github.io/agent-lightning/stable/reference/cli/#__codelineno-4-11) --log-level {DEBUG,INFO,WARNING,ERROR} [](https://microsoft.github.io/agent-lightning/stable/reference/cli/#__codelineno-4-12) Configure the logging level for the metrics server. [](https://microsoft.github.io/agent-lightning/stable/reference/cli/#__codelineno-4-13) --access-log Enable uvicorn access logs. Disabled by default to reduce noise.` Back to top --- # Debugging - Agent-lightning [Skip to content](https://microsoft.github.io/agent-lightning/stable/tutorials/debug/#debugging-and-troubleshooting) ⚠️ You are viewing development documentation. For stable documentation, please visit [https://microsoft.github.io/agent-lightning/stable/](https://microsoft.github.io/agent-lightning/stable/) Debugging and Troubleshooting[¶](https://microsoft.github.io/agent-lightning/stable/tutorials/debug/#debugging-and-troubleshooting "Permanent link") ===================================================================================================================================================== When you train your own agent with Agent-lightning, most failures surface because the agent logic is brittle or simply incorrect. Debugging becomes easier when you peel back the stack: start by driving the rollout logic on its own, dry-run the trainer loop, and only then bring the full algorithm and runner topology online. The [`examples/apo/apo_debug.py`](https://github.com/microsoft/agent-lightning/tree/3b5d733861cf313fc09821a23240bbdf3cb2ee5b/examples/apo/apo_debug.py) script demonstrates these techniques; this guide expands on each approach and helps you decide when to reach for them. Debugging with Dashboard[¶](https://microsoft.github.io/agent-lightning/stable/tutorials/debug/#debugging-with-dashboard "Permanent link") ------------------------------------------------------------------------------------------------------------------------------------------- When you launch an experiment with [`Trainer.fit`](https://microsoft.github.io/agent-lightning/stable/reference/trainer/#agentlightning.Trainer.fit " fit(agent, train_dataset=None, *, val_dataset=None)") or start an isolated store via [`agl store`](https://microsoft.github.io/agent-lightning/stable/reference/cli/) , the terminal prints a message similar to: `[](https://microsoft.github.io/agent-lightning/stable/tutorials/debug/#__codelineno-0-1) INFO Agent-lightning dashboard will be available at http://192.168.0.107:4747` Visit that URL, and you will see the Agent-lightning dashboard: ![Dashboard](https://microsoft.github.io/agent-lightning/stable/assets/dashboard-page-rollouts.png) The dashboard surfaces everything stored inside [the store](https://microsoft.github.io/agent-lightning/stable/deep-dive/store/) . Because the store mediates interactions between algorithms and runners, inspecting it often reveals which side is causing issues such as stale rollouts, unresponsive workers, or empty traces. For example, the VERL algorithm may receive no token IDs and emit `cannot reshape tensor of 0 elements into shape [1, 0, -1, 128] because the unspecified dimension size -1 can be any value and is ambiguous` ([Issue #50](https://github.com/microsoft/agent-lightning/issues/50) , [Issue #76](https://github.com/microsoft/agent-lightning/issues/76) ). Several scenarios can produce that error: the runner might not produce trace spans at all, it might produce spans without token IDs, or the IDs may be present but formatted incorrectly. Inspecting the dashboard traces helps you pinpoint which condition applies. ![Dashboard Traces Page](https://microsoft.github.io/agent-lightning/stable/assets/dashboard-page-traces.png) By checking whether the trace span is empty and whether token IDs appear in the span attributes, you can narrow the issue to either the runner (agent) side or the algorithm side. Then apply the techniques below to debug the faulty component. Debug-level Logging[¶](https://microsoft.github.io/agent-lightning/stable/tutorials/debug/#debug-level-logging "Permanent link") --------------------------------------------------------------------------------------------------------------------------------- Starting from v0.3, detailed signals such as store server access logs, runner lifecycle logs, and span payloads only appear when the log level is `DEBUG` so the default output stays readable. Enable debug-level logging by adding the following snippet near the top of your script: `[](https://microsoft.github.io/agent-lightning/stable/tutorials/debug/#__codelineno-1-1) import agentlightning as agl [](https://microsoft.github.io/agent-lightning/stable/tutorials/debug/#__codelineno-1-2) [](https://microsoft.github.io/agent-lightning/stable/tutorials/debug/#__codelineno-1-3) agl.setup_logging("DEBUG")` Set the log level on every process if your setup involves multiple workers. For example, when [running stores in isolation](https://microsoft.github.io/agent-lightning/stable/tutorials/debug/#debug-with-external-store "Debug the Algorithm-Runner Boundary") , configure the store process explicitly: `[](https://microsoft.github.io/agent-lightning/stable/tutorials/debug/#__codelineno-2-1) agl store --port 4747 --log-level DEBUG` Using [`Runner`](https://microsoft.github.io/agent-lightning/stable/reference/runner/#agentlightning.Runner " agentlightning.Runner") in Isolation[¶](https://microsoft.github.io/agent-lightning/stable/tutorials/debug/#using-runner-in-isolation "Permanent link") ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- [`Runner`](https://microsoft.github.io/agent-lightning/stable/reference/runner/#agentlightning.Runner " agentlightning.Runner") is a long-lived worker that wraps your [`LitAgent`](https://microsoft.github.io/agent-lightning/stable/reference/agent/#agentlightning.LitAgent " agentlightning.LitAgent") , coordinates tracing, and talks to the [`LightningStore`](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.LightningStore " agentlightning.LightningStore") . In typical training flows the trainer manages runners for you, but being able to spin one up manually is invaluable while debugging. If you define rollout logic with [`@rollout`](https://microsoft.github.io/agent-lightning/stable/reference/agent/#agentlightning.rollout " agentlightning.rollout(func)") or implement a [`LitAgent`](https://microsoft.github.io/agent-lightning/stable/reference/agent/#agentlightning.LitAgent " agentlightning.LitAgent") directly, you will get a [`LitAgent`](https://microsoft.github.io/agent-lightning/stable/reference/agent/#agentlightning.LitAgent " agentlightning.LitAgent") instance and you should be able to execute it with [`LitAgentRunner`](https://microsoft.github.io/agent-lightning/stable/reference/runner/#agentlightning.LitAgentRunner " agentlightning.LitAgentRunner") , which is a subclass of [`Runner`](https://microsoft.github.io/agent-lightning/stable/reference/runner/#agentlightning.Runner " agentlightning.Runner") . The runner needs but does not instantiate a [`Tracer`](https://microsoft.github.io/agent-lightning/stable/reference/runner/#agentlightning.Tracer " agentlightning.Tracer") , so supply one yourself. See [Working with Traces](https://microsoft.github.io/agent-lightning/stable/tutorials/traces/) for a walkthrough of tracer options. [`Runner.run_context`](https://microsoft.github.io/agent-lightning/stable/reference/runner/#agentlightning.Runner.run_context " run_context(*, agent, store, hooks=None, worker_id=None)") prepares the runner to execute a particular agent. Besides the agent and tracer you must provide a store that will collect spans and rollouts. [`InMemoryLightningStore`](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.InMemoryLightningStore " agentlightning.InMemoryLightningStore") keeps everything in-process, which is perfect for debugging sessions. `[](https://microsoft.github.io/agent-lightning/stable/tutorials/debug/#__codelineno-3-1) import agentlightning as agl [](https://microsoft.github.io/agent-lightning/stable/tutorials/debug/#__codelineno-3-2) [](https://microsoft.github.io/agent-lightning/stable/tutorials/debug/#__codelineno-3-3) tracer = agl.OtelTracer() [](https://microsoft.github.io/agent-lightning/stable/tutorials/debug/#__codelineno-3-4) runner = agl.LitAgentRunner(tracer) [](https://microsoft.github.io/agent-lightning/stable/tutorials/debug/#__codelineno-3-5) store = agl.InMemoryLightningStore() [](https://microsoft.github.io/agent-lightning/stable/tutorials/debug/#__codelineno-3-6) [](https://microsoft.github.io/agent-lightning/stable/tutorials/debug/#__codelineno-3-7) with runner.run_context(agent=apo_rollout, store=store): [](https://microsoft.github.io/agent-lightning/stable/tutorials/debug/#__codelineno-3-8) ...` Inside the [`run_context`](https://microsoft.github.io/agent-lightning/stable/reference/runner/#agentlightning.Runner.run_context " run_context(*, agent, store, hooks=None, worker_id=None)") block you can call [`runner.step(...)`](https://microsoft.github.io/agent-lightning/stable/reference/runner/#agentlightning.Runner.step " step(input, *, resources=None, mode=None, event=None) async ") to execute a single rollout. The payload includes the task input and any [`NamedResources`](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.NamedResources " agentlightning.NamedResources = Dict[str, ResourceUnion] module-attribute ") the agent expects. Read [introduction to Resources](https://microsoft.github.io/agent-lightning/stable/how-to/write-first-algorithm/#introduction-to-resources "Resources: The Tunable Assets") and [NamedResources](https://microsoft.github.io/agent-lightning/stable/tutorials/write-agents/#introduction-to-named-resources "Class-based Agents") for more details. For example, if your agent references a [`PromptTemplate`](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.PromptTemplate " agentlightning.PromptTemplate") , pass it through the `resources` argument: `[](https://microsoft.github.io/agent-lightning/stable/tutorials/debug/#__codelineno-4-1) with runner.run_context(agent=apo_rollout, store=store): [](https://microsoft.github.io/agent-lightning/stable/tutorials/debug/#__codelineno-4-2) resource = agl.PromptTemplate(template="You are a helpful assistant. {any_question}", engine="f-string") [](https://microsoft.github.io/agent-lightning/stable/tutorials/debug/#__codelineno-4-3) rollout = await runner.step( [](https://microsoft.github.io/agent-lightning/stable/tutorials/debug/#__codelineno-4-4) "Explain why the sky appears blue using principles of light scattering in 100 words.", [](https://microsoft.github.io/agent-lightning/stable/tutorials/debug/#__codelineno-4-5) resources={"main_prompt": resource}, [](https://microsoft.github.io/agent-lightning/stable/tutorials/debug/#__codelineno-4-6) )` You can do as many things as you want within the [`Runner.run_context`](https://microsoft.github.io/agent-lightning/stable/reference/runner/#agentlightning.Runner.run_context " run_context(*, agent, store, hooks=None, worker_id=None)") block. After the rollout finishes you can query the store to inspect what happened: `[](https://microsoft.github.io/agent-lightning/stable/tutorials/debug/#__codelineno-5-1) print(await store.query_rollouts()) [](https://microsoft.github.io/agent-lightning/stable/tutorials/debug/#__codelineno-5-2) print(await store.query_spans(rollout.rollout_id))` Example output (with a reward span captured): `[](https://microsoft.github.io/agent-lightning/stable/tutorials/debug/#__codelineno-6-1) [Rollout(rollout_id='ro-519769241af8', input='Explain why the sky appears blue using principles of light scattering in 100 words.', start_time=1760706315.6996238, ..., status='succeeded')] [](https://microsoft.github.io/agent-lightning/stable/tutorials/debug/#__codelineno-6-2) [Span(rollout_id='ro-519769241af8', attempt_id='at-a6b62caf', sequence_id=1, ..., name='agentlightning.annotation', attributes={'agentlightning.reward.0.value': 0.95}, ...)]` Swap in an [`AgentOpsTracer`](https://microsoft.github.io/agent-lightning/stable/reference/runner/#agentlightning.AgentOpsTracer " agentlightning.AgentOpsTracer") instead of [`OtelTracer`](https://microsoft.github.io/agent-lightning/stable/reference/runner/#agentlightning.OtelTracer " agentlightning.OtelTracer") to see the underlying LLM spans alongside reward information: `[](https://microsoft.github.io/agent-lightning/stable/tutorials/debug/#__codelineno-7-1) [ [](https://microsoft.github.io/agent-lightning/stable/tutorials/debug/#__codelineno-7-2) Span(rollout_id='ro-519769241af8', attempt_id='at-a6b62caf', sequence_id=1, ..., name='openai.chat.completion', attributes={..., 'gen_ai.prompt.0.role': 'user', 'gen_ai.prompt.0.content': 'You are a helpful assistant. Explain why the sky appears blue using principles of light scattering in 100 words.', ...}), [](https://microsoft.github.io/agent-lightning/stable/tutorials/debug/#__codelineno-7-3) Span(rollout_id='ro-519769241af8', attempt_id='at-a6b62caf', sequence_id=2, ..., name='openai.chat.completion', attributes={..., 'gen_ai.prompt.0.role': 'user', 'gen_ai.prompt.0.content': 'Evaluate how well the output fulfills the task...', ...}), [](https://microsoft.github.io/agent-lightning/stable/tutorials/debug/#__codelineno-7-4) Span(rollout_id='ro-519769241af8', attempt_id='at-a6b62caf', sequence_id=3, ..., name='agentlightning.annotation', attributes={'agentlightning.reward.0.value': 0.95}, ...) [](https://microsoft.github.io/agent-lightning/stable/tutorials/debug/#__codelineno-7-5) ]` Tip Spans too difficult to read? Try using [`Adapter`](https://microsoft.github.io/agent-lightning/stable/reference/algorithm/#agentlightning.Adapter " agentlightning.Adapter") to convert them into a [more readable format](https://microsoft.github.io/agent-lightning/stable/tutorials/traces/) . [`Runner.step`](https://microsoft.github.io/agent-lightning/stable/reference/runner/#agentlightning.Runner.step " step(input, *, resources=None, mode=None, event=None) async ") executes a full rollout even though it is named "step". The companion method [`Runner.iter`](https://microsoft.github.io/agent-lightning/stable/reference/runner/#agentlightning.Runner.iter " iter(*, event=None) async ") executes multiple "steps" by continuously pulling new rollout inputs from the store until a stop event is set. Use `iter` once you are confident the single-step path works and you have another worker [`enqueue_rollout`](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.LightningStore.enqueue_rollout " enqueue_rollout(input, mode=None, resources_id=None, config=None, metadata=None) async ") to the store. Tip You can also call [`Runner.step`](https://microsoft.github.io/agent-lightning/stable/reference/runner/#agentlightning.Runner.step " step(input, *, resources=None, mode=None, event=None) async ") to inject ad-hoc rollouts into a running store being used by another algorithm, so that the rollouts can be consumed by the algorithms. This is very recently known as the paradigm of ["online RL"](https://cursor.com/blog/tab-rl) . At the moment, no algorithm in the [algorithm zoo](https://microsoft.github.io/agent-lightning/stable/algorithm-zoo/) consumes externally generated rollouts, but the data flow is available there if you need it. Debug with LLM Proxy[¶](https://microsoft.github.io/agent-lightning/stable/tutorials/debug/#debug-with-llm-proxy "Permanent link") ----------------------------------------------------------------------------------------------------------------------------------- If you are dealing with LLM optimization like Reinforcement Learning, we generally recommend using an online stable LLM service for your debugging purposes, like `openai/gpt-4.1-nano`. After the debugging is done, you can switch to a local training endpoint. However, if you want to use a local LLM features like [getting the token IDs](https://microsoft.github.io/agent-lightning/stable/deep-dive/serving-llm/) , you can also manually start a local vLLM server by: `[](https://microsoft.github.io/agent-lightning/stable/tutorials/debug/#__codelineno-8-1) vllm serve Qwen/Qwen2.5-0.5B-Instruct --port 8080` Then start the LLM proxy via the following script: `[](https://microsoft.github.io/agent-lightning/stable/tutorials/debug/#__codelineno-9-1) import asyncio [](https://microsoft.github.io/agent-lightning/stable/tutorials/debug/#__codelineno-9-2) import aiohttp [](https://microsoft.github.io/agent-lightning/stable/tutorials/debug/#__codelineno-9-3) import agentlightning as agl [](https://microsoft.github.io/agent-lightning/stable/tutorials/debug/#__codelineno-9-4) [](https://microsoft.github.io/agent-lightning/stable/tutorials/debug/#__codelineno-9-5) async def serve_llm_proxy(): [](https://microsoft.github.io/agent-lightning/stable/tutorials/debug/#__codelineno-9-6) store = agl.InMemoryLightningStore() [](https://microsoft.github.io/agent-lightning/stable/tutorials/debug/#__codelineno-9-7) store_server = agl.LightningStoreServer(store, "127.0.0.1", 8081) [](https://microsoft.github.io/agent-lightning/stable/tutorials/debug/#__codelineno-9-8) await store_server.start() [](https://microsoft.github.io/agent-lightning/stable/tutorials/debug/#__codelineno-9-9) [](https://microsoft.github.io/agent-lightning/stable/tutorials/debug/#__codelineno-9-10) llm_proxy = agl.LLMProxy( [](https://microsoft.github.io/agent-lightning/stable/tutorials/debug/#__codelineno-9-11) port=8082, [](https://microsoft.github.io/agent-lightning/stable/tutorials/debug/#__codelineno-9-12) model_list=[ [](https://microsoft.github.io/agent-lightning/stable/tutorials/debug/#__codelineno-9-13) { [](https://microsoft.github.io/agent-lightning/stable/tutorials/debug/#__codelineno-9-14) "model_name": "Qwen/Qwen2.5-0.5B-Instruct", [](https://microsoft.github.io/agent-lightning/stable/tutorials/debug/#__codelineno-9-15) "litellm_params": { [](https://microsoft.github.io/agent-lightning/stable/tutorials/debug/#__codelineno-9-16) "model": "hosted_vllm/Qwen/Qwen2.5-0.5B-Instruct", [](https://microsoft.github.io/agent-lightning/stable/tutorials/debug/#__codelineno-9-17) "api_base": "http://localhost:8080/v1", [](https://microsoft.github.io/agent-lightning/stable/tutorials/debug/#__codelineno-9-18) }, [](https://microsoft.github.io/agent-lightning/stable/tutorials/debug/#__codelineno-9-19) } [](https://microsoft.github.io/agent-lightning/stable/tutorials/debug/#__codelineno-9-20) ], [](https://microsoft.github.io/agent-lightning/stable/tutorials/debug/#__codelineno-9-21) store=store_server, [](https://microsoft.github.io/agent-lightning/stable/tutorials/debug/#__codelineno-9-22) ) [](https://microsoft.github.io/agent-lightning/stable/tutorials/debug/#__codelineno-9-23) [](https://microsoft.github.io/agent-lightning/stable/tutorials/debug/#__codelineno-9-24) await llm_proxy.start() [](https://microsoft.github.io/agent-lightning/stable/tutorials/debug/#__codelineno-9-25) await asyncio.sleep(1000000)` Test the served LLM proxy with a client like: `[](https://microsoft.github.io/agent-lightning/stable/tutorials/debug/#__codelineno-10-1) async def test_llm_proxy(): [](https://microsoft.github.io/agent-lightning/stable/tutorials/debug/#__codelineno-10-2) async with aiohttp.ClientSession() as session: [](https://microsoft.github.io/agent-lightning/stable/tutorials/debug/#__codelineno-10-3) async with session.post("http://localhost:8082/v1/chat/completions", json={ [](https://microsoft.github.io/agent-lightning/stable/tutorials/debug/#__codelineno-10-4) "model": "Qwen/Qwen2.5-0.5B-Instruct", [](https://microsoft.github.io/agent-lightning/stable/tutorials/debug/#__codelineno-10-5) "messages": [{"role": "user", "content": "Hello, world!"}], [](https://microsoft.github.io/agent-lightning/stable/tutorials/debug/#__codelineno-10-6) }) as response: [](https://microsoft.github.io/agent-lightning/stable/tutorials/debug/#__codelineno-10-7) print(await response.json())` You can now use the LLM proxy by specifying environment variables: `[](https://microsoft.github.io/agent-lightning/stable/tutorials/debug/#__codelineno-11-1) export OPENAI_API_BASE=http://localhost:8081/v1 [](https://microsoft.github.io/agent-lightning/stable/tutorials/debug/#__codelineno-11-2) export OPENAI_API_KEY=dummy` You might see warnings about `Missing or invalid rollout_id, attempt_id, or sequence_id` in the LLM proxy logs. This is fine because you don't have a rollout and attempt yet when you are debugging. When you started the training, the algorithm will create the rollouts for you and the warnings will go away. Hook into Runner's Lifecycle[¶](https://microsoft.github.io/agent-lightning/stable/tutorials/debug/#hook-into-runners-lifecycle "Permanent link") -------------------------------------------------------------------------------------------------------------------------------------------------- [`Runner.run_context`](https://microsoft.github.io/agent-lightning/stable/reference/runner/#agentlightning.Runner.run_context " run_context(*, agent, store, hooks=None, worker_id=None)") accepts a `hooks` argument so you can observe or augment lifecycle events without editing your agent. Hooks subclass [`Hook`](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.Hook " agentlightning.Hook") and can respond to four asynchronous callbacks: [`on_trace_start`](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.Hook.on_trace_start " on_trace_start(*, agent, runner, tracer, rollout) async ") , [`on_rollout_start`](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.Hook.on_rollout_start " on_rollout_start(*, agent, runner, rollout) async ") , [`on_rollout_end`](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.Hook.on_rollout_end " on_rollout_end(*, agent, runner, rollout, spans) async ") , and [`on_trace_end`](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.Hook.on_trace_end " on_trace_end(*, agent, runner, tracer, rollout) async ") . This is useful for: * Capturing raw OpenTelemetry spans before they hit the store and before the [`LitAgentRunner`](https://microsoft.github.io/agent-lightning/stable/reference/runner/#agentlightning.LitAgentRunner " agentlightning.LitAgentRunner") do postprocessing on the rollout * Inspecting the tracer instance after they are activated * Logging rollout inputs before they are processed by the agent The `hook` mode in [`examples/apo/apo_debug.py`](https://github.com/microsoft/agent-lightning/tree/3b5d733861cf313fc09821a23240bbdf3cb2ee5b/examples/apo/apo_debug.py) prints every span collected during a rollout: `[](https://microsoft.github.io/agent-lightning/stable/tutorials/debug/#__codelineno-12-1) import agentlightning as agl [](https://microsoft.github.io/agent-lightning/stable/tutorials/debug/#__codelineno-12-2) [](https://microsoft.github.io/agent-lightning/stable/tutorials/debug/#__codelineno-12-3) # ... Same as previous example [](https://microsoft.github.io/agent-lightning/stable/tutorials/debug/#__codelineno-12-4) [](https://microsoft.github.io/agent-lightning/stable/tutorials/debug/#__codelineno-12-5) class DebugHook(agl.Hook): [](https://microsoft.github.io/agent-lightning/stable/tutorials/debug/#__codelineno-12-6) async def on_trace_end(self, *, agent, runner, tracer, rollout): [](https://microsoft.github.io/agent-lightning/stable/tutorials/debug/#__codelineno-12-7) trace = tracer.get_last_trace() [](https://microsoft.github.io/agent-lightning/stable/tutorials/debug/#__codelineno-12-8) print("Trace spans collected during the rollout:") [](https://microsoft.github.io/agent-lightning/stable/tutorials/debug/#__codelineno-12-9) for span in trace: [](https://microsoft.github.io/agent-lightning/stable/tutorials/debug/#__codelineno-12-10) print(f"- {span.name} (status: {span.status}):\n {span.attributes}") [](https://microsoft.github.io/agent-lightning/stable/tutorials/debug/#__codelineno-12-11) [](https://microsoft.github.io/agent-lightning/stable/tutorials/debug/#__codelineno-12-12) with runner.run_context( [](https://microsoft.github.io/agent-lightning/stable/tutorials/debug/#__codelineno-12-13) agent=apo_rollout, [](https://microsoft.github.io/agent-lightning/stable/tutorials/debug/#__codelineno-12-14) store=store, [](https://microsoft.github.io/agent-lightning/stable/tutorials/debug/#__codelineno-12-15) hooks=[DebugHook()], [](https://microsoft.github.io/agent-lightning/stable/tutorials/debug/#__codelineno-12-16) ): [](https://microsoft.github.io/agent-lightning/stable/tutorials/debug/#__codelineno-12-17) await runner.step( [](https://microsoft.github.io/agent-lightning/stable/tutorials/debug/#__codelineno-12-18) "Explain why the sky appears blue using principles of light scattering in 100 words.", [](https://microsoft.github.io/agent-lightning/stable/tutorials/debug/#__codelineno-12-19) resources={"main_prompt": resource}, [](https://microsoft.github.io/agent-lightning/stable/tutorials/debug/#__codelineno-12-20) )` Because hooks run inside the runner process you can also attach debuggers or breakpoints directly in the callback implementations. Note For a better understanding of where hooks are called, we show a pseudo code of Runner's working flow below: `[](https://microsoft.github.io/agent-lightning/stable/tutorials/debug/#__codelineno-13-1) resources = await store.get_latest_resources() [](https://microsoft.github.io/agent-lightning/stable/tutorials/debug/#__codelineno-13-2) rollout = ... [](https://microsoft.github.io/agent-lightning/stable/tutorials/debug/#__codelineno-13-3) try: [](https://microsoft.github.io/agent-lightning/stable/tutorials/debug/#__codelineno-13-4) # <-- on_rollout_start [](https://microsoft.github.io/agent-lightning/stable/tutorials/debug/#__codelineno-13-5) with tracer.trace_context(...): [](https://microsoft.github.io/agent-lightning/stable/tutorials/debug/#__codelineno-13-6) # <--- on_trace_start [](https://microsoft.github.io/agent-lightning/stable/tutorials/debug/#__codelineno-13-7) result = await agent.rollout(...) [](https://microsoft.github.io/agent-lightning/stable/tutorials/debug/#__codelineno-13-8) # <--- on_trace_end [](https://microsoft.github.io/agent-lightning/stable/tutorials/debug/#__codelineno-13-9) post_process_result(result) [](https://microsoft.github.io/agent-lightning/stable/tutorials/debug/#__codelineno-13-10) except Exception: [](https://microsoft.github.io/agent-lightning/stable/tutorials/debug/#__codelineno-13-11) # <-- on_rollout_end [](https://microsoft.github.io/agent-lightning/stable/tutorials/debug/#__codelineno-13-12) await store.update_attempt(status=...)` Dry-Run the Trainer Loop[¶](https://microsoft.github.io/agent-lightning/stable/tutorials/debug/#dry-run-the-trainer-loop "Permanent link") ------------------------------------------------------------------------------------------------------------------------------------------- Once single rollouts behave, switch to the trainer’s dry-run mode. [`Trainer.dev`](https://microsoft.github.io/agent-lightning/stable/reference/trainer/#agentlightning.Trainer.dev " dev(agent, train_dataset=None, *, val_dataset=None)") spins up a lightweight fast algorithm — [`agentlightning.Baseline`](https://microsoft.github.io/agent-lightning/stable/reference/algorithm/#agentlightning.Baseline " agentlightning.Baseline") by default — so you can exercise the same infrastructure as [`Trainer.fit`](https://microsoft.github.io/agent-lightning/stable/reference/trainer/#agentlightning.Trainer.fit " fit(agent, train_dataset=None, *, val_dataset=None)") without standing up complex stacks like RL or SFT. Warning When you enable multiple runners via `n_runners`, the trainer may execute them in separate worker processes. Attaching a debugger such as `pdb` is only practical when `n_runners=1`, and even then the runner might not live in the main process. `[](https://microsoft.github.io/agent-lightning/stable/tutorials/debug/#__codelineno-14-1) import agentlightning as agl [](https://microsoft.github.io/agent-lightning/stable/tutorials/debug/#__codelineno-14-2) [](https://microsoft.github.io/agent-lightning/stable/tutorials/debug/#__codelineno-14-3) dataset: agl.Dataset[str] = [ [](https://microsoft.github.io/agent-lightning/stable/tutorials/debug/#__codelineno-14-4) "Explain why the sky appears blue using principles of light scattering in 100 words.", [](https://microsoft.github.io/agent-lightning/stable/tutorials/debug/#__codelineno-14-5) "What's the capital of France?", [](https://microsoft.github.io/agent-lightning/stable/tutorials/debug/#__codelineno-14-6) ] [](https://microsoft.github.io/agent-lightning/stable/tutorials/debug/#__codelineno-14-7) resource = agl.PromptTemplate(template="You are a helpful assistant. {any_question}", engine="f-string") [](https://microsoft.github.io/agent-lightning/stable/tutorials/debug/#__codelineno-14-8) [](https://microsoft.github.io/agent-lightning/stable/tutorials/debug/#__codelineno-14-9) trainer = agl.Trainer( [](https://microsoft.github.io/agent-lightning/stable/tutorials/debug/#__codelineno-14-10) n_runners=1, [](https://microsoft.github.io/agent-lightning/stable/tutorials/debug/#__codelineno-14-11) initial_resources={"main_prompt": resource}, [](https://microsoft.github.io/agent-lightning/stable/tutorials/debug/#__codelineno-14-12) ) [](https://microsoft.github.io/agent-lightning/stable/tutorials/debug/#__codelineno-14-13) trainer.dev(apo_rollout, dataset)` Just like [`Runner.run_context`](https://microsoft.github.io/agent-lightning/stable/reference/runner/#agentlightning.Runner.run_context " run_context(*, agent, store, hooks=None, worker_id=None)") , [`Trainer.dev`](https://microsoft.github.io/agent-lightning/stable/reference/trainer/#agentlightning.Trainer.dev " dev(agent, train_dataset=None, *, val_dataset=None)") requires the [`NamedResources`](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.NamedResources " agentlightning.NamedResources = Dict[str, ResourceUnion] module-attribute ") your agent expects. The key difference is that resources are attached to the trainer rather than the runner. [`Trainer.dev`](https://microsoft.github.io/agent-lightning/stable/reference/trainer/#agentlightning.Trainer.dev " dev(agent, train_dataset=None, *, val_dataset=None)") uses an almost switchable interface from [`Trainer.fit`](https://microsoft.github.io/agent-lightning/stable/reference/trainer/#agentlightning.Trainer.fit " fit(agent, train_dataset=None, *, val_dataset=None)") . It also needs a dataset to iterate over, similar to [`fit`](https://microsoft.github.io/agent-lightning/stable/reference/trainer/#agentlightning.Trainer.fit " fit(agent, train_dataset=None, *, val_dataset=None)") . Under the hood [`dev`](https://microsoft.github.io/agent-lightning/stable/reference/trainer/#agentlightning.Trainer.dev " dev(agent, train_dataset=None, *, val_dataset=None)") uses the same implementation as [`fit`](https://microsoft.github.io/agent-lightning/stable/reference/trainer/#agentlightning.Trainer.fit " fit(agent, train_dataset=None, *, val_dataset=None)") , which means you can spin up multiple runners, observe scheduler behavior, and validate how algorithms adapt rollouts. The default [`Baseline`](https://microsoft.github.io/agent-lightning/stable/reference/algorithm/#agentlightning.Baseline " agentlightning.Baseline") logs detailed traces so you can see each rollout as the algorithm perceives it: `[](https://microsoft.github.io/agent-lightning/stable/tutorials/debug/#__codelineno-15-1) 21:20:30 Initial resources set: {'main_prompt': PromptTemplate(resource_type='prompt_template', template='You are a helpful assistant. {any_question}', engine='f-string')} [](https://microsoft.github.io/agent-lightning/stable/tutorials/debug/#__codelineno-15-2) 21:20:30 Proceeding epoch 1/1. [](https://microsoft.github.io/agent-lightning/stable/tutorials/debug/#__codelineno-15-3) 21:20:30 Enqueued rollout ro-302fb202bd85 in train mode with sample: Explain why the sky appears blue using principles of light scattering in 100 words. [](https://microsoft.github.io/agent-lightning/stable/tutorials/debug/#__codelineno-15-4) 21:20:30 Enqueued rollout ro-e65a3ffaa540 in train mode with sample: What's the capital of France? [](https://microsoft.github.io/agent-lightning/stable/tutorials/debug/#__codelineno-15-5) 21:20:30 Waiting for 2 harvest tasks to complete... [](https://microsoft.github.io/agent-lightning/stable/tutorials/debug/#__codelineno-15-6) 21:20:30 [Rollout ro-302fb202bd85] Status is initialized to queuing. [](https://microsoft.github.io/agent-lightning/stable/tutorials/debug/#__codelineno-15-7) 21:20:30 [Rollout ro-e65a3ffaa540] Status is initialized to queuing. [](https://microsoft.github.io/agent-lightning/stable/tutorials/debug/#__codelineno-15-8) 21:20:35 [Rollout ro-302fb202bd85] Finished with status succeeded in 3.80 seconds. [](https://microsoft.github.io/agent-lightning/stable/tutorials/debug/#__codelineno-15-9) 21:20:35 [Rollout ro-302fb202bd85 | Attempt 1] ID: at-f84ad21c. Status: succeeded. Worker: Worker-0 [](https://microsoft.github.io/agent-lightning/stable/tutorials/debug/#__codelineno-15-10) 21:20:35 [Rollout ro-302fb202bd85 | Attempt at-f84ad21c | Span 3a286a856af6bea8] #1 (openai.chat.completion) ... 1.95 seconds. Attribute keys: ['gen_ai.request.type', 'gen_ai.system', ...] [](https://microsoft.github.io/agent-lightning/stable/tutorials/debug/#__codelineno-15-11) 21:20:35 [Rollout ro-302fb202bd85 | Attempt at-f84ad21c | Span e2f44b775e058dd6] #2 (openai.chat.completion) ... 1.24 seconds. Attribute keys: ['gen_ai.request.type', 'gen_ai.system', ...] [](https://microsoft.github.io/agent-lightning/stable/tutorials/debug/#__codelineno-15-12) 21:20:35 [Rollout ro-302fb202bd85 | Attempt at-f84ad21c | Span 45ee3c94fa1070ec] #3 (agentlightning.annotation) ... 0.00 seconds. Attribute keys: ['agentlightning.reward.0.value'] [](https://microsoft.github.io/agent-lightning/stable/tutorials/debug/#__codelineno-15-13) 21:20:35 [Rollout ro-302fb202bd85] Adapted data: [Triplet(prompt={'token_ids': []}, response={'token_ids': []}, reward=None, metadata={'response_id': '...', 'agent_name': ''}), Triplet(prompt={'token_ids': []}, response={'token_ids': []}, reward=0.95, metadata={'response_id': '...', 'agent_name': ''})] [](https://microsoft.github.io/agent-lightning/stable/tutorials/debug/#__codelineno-15-14) 21:20:35 Finished 1 rollouts. [](https://microsoft.github.io/agent-lightning/stable/tutorials/debug/#__codelineno-15-15) 21:20:35 [Rollout ro-e65a3ffaa540] Status changed to preparing. [](https://microsoft.github.io/agent-lightning/stable/tutorials/debug/#__codelineno-15-16) 21:20:40 [Rollout ro-e65a3ffaa540] Finished with status succeeded in 6.39 seconds. [](https://microsoft.github.io/agent-lightning/stable/tutorials/debug/#__codelineno-15-17) 21:20:40 [Rollout ro-e65a3ffaa540 | Attempt 1] ID: at-eaefa5d4. Status: succeeded. Worker: Worker-0 [](https://microsoft.github.io/agent-lightning/stable/tutorials/debug/#__codelineno-15-18) 21:20:40 [Rollout ro-e65a3ffaa540 | Attempt at-eaefa5d4 | Span 901dd6acc0f50147] #1 (openai.chat.completion) ... 1.30 seconds. Attribute keys: ['gen_ai.request.type', 'gen_ai.system', ...] [](https://microsoft.github.io/agent-lightning/stable/tutorials/debug/#__codelineno-15-19) 21:20:40 [Rollout ro-e65a3ffaa540 | Attempt at-eaefa5d4 | Span 52e0aa63e02be611] #2 (openai.chat.completion) ... 1.26 seconds. Attribute keys: ['gen_ai.request.type', 'gen_ai.system', ...] [](https://microsoft.github.io/agent-lightning/stable/tutorials/debug/#__codelineno-15-20) 21:20:40 [Rollout ro-e65a3ffaa540 | Attempt at-eaefa5d4 | Span 6c452de193fbffd3] #3 (agentlightning.annotation) ... 0.00 seconds. Attribute keys: ['agentlightning.reward.0.value'] [](https://microsoft.github.io/agent-lightning/stable/tutorials/debug/#__codelineno-15-21) 21:20:40 [Rollout ro-e65a3ffaa540] Adapted data: [Triplet(prompt={'token_ids': []}, response={'token_ids': []}, reward=None, metadata={'response_id': '...', 'agent_name': ''}), Triplet(prompt={'token_ids': []}, response={'token_ids': []}, reward=1.0, metadata={'response_id': '...', 'agent_name': ''})] [](https://microsoft.github.io/agent-lightning/stable/tutorials/debug/#__codelineno-15-22) 21:20:40 Finished 2 rollouts.` The only limitation is that resources remain static and components like [`LLMProxy`](https://microsoft.github.io/agent-lightning/stable/reference/algorithm/#agentlightning.LLMProxy " agentlightning.LLMProxy") are not wired in. For richer dry runs you can subclass [`FastAlgorithm`](https://microsoft.github.io/agent-lightning/stable/reference/algorithm/#agentlightning.FastAlgorithm " agentlightning.FastAlgorithm") and override the pieces you care about. Debug the Algorithm-Runner Boundary[¶](https://microsoft.github.io/agent-lightning/stable/tutorials/debug/#debug-the-algorithm-runner-boundary "Permanent link") ----------------------------------------------------------------------------------------------------------------------------------------------------------------- [](https://microsoft.github.io/agent-lightning/stable/tutorials/debug/) Debugging algorithms in Agent-Lightning is often more challenging than debugging agents. Algorithms are typically **stateful** and depend on several moving parts — runners, stores, and trainers — which makes it difficult to isolate and inspect their behavior. Even mocking an agent to cooperate with an algorithm can be costly and error-prone. To simplify this, Agent-Lightning provides a way to run algorithms in isolation so you can attach a debugger and inspect internal state without interference from other components. By default, [`Trainer.fit`](https://microsoft.github.io/agent-lightning/stable/reference/trainer/#agentlightning.Trainer.fit " fit(agent, train_dataset=None, *, val_dataset=None)") runs the algorithm in the main process and thread, but its logs are interleaved with those from the store and runners, making it hard to follow what’s happening inside the algorithm itself. In [_Write Your First Algorithm_](https://microsoft.github.io/agent-lightning/stable/how-to/write-first-algorithm/) , we covered how to stand up a store, algorithm, and runner in isolation for your own implementations. This section extends that approach to cover two common questions: 1. How can I run built-in or class-based algorithms (inheriting from [`Algorithm`](https://microsoft.github.io/agent-lightning/stable/reference/algorithm/#agentlightning.Algorithm " agentlightning.Algorithm") ) in isolation? 2. How can I still use [`Trainer`](https://microsoft.github.io/agent-lightning/stable/reference/trainer/#agentlightning.Trainer " agentlightning.Trainer") features like `n_runners`, `adapter`, or `llm_proxy` while debugging? The solution is to keep using a [`Trainer`](https://microsoft.github.io/agent-lightning/stable/reference/trainer/#agentlightning.Trainer " agentlightning.Trainer") instance but **manage the store yourself**, running the algorithm and runner roles separately. This approach mirrors the internal process orchestration of [`Trainer.fit`](https://microsoft.github.io/agent-lightning/stable/reference/trainer/#agentlightning.Trainer.fit " fit(agent, train_dataset=None, *, val_dataset=None)") , but with more visibility and control. Below, we show a step-by-step guide to achieve this with the [`calc_agent` example](https://github.com/microsoft/agent-lightning/tree/3b5d733861cf313fc09821a23240bbdf3cb2ee5b/examples/calc_x/train_calc_agent.py) . **1\. Launch the store manually.** In a separate terminal, start the store: `[](https://microsoft.github.io/agent-lightning/stable/tutorials/debug/#__codelineno-16-1) agl store --port 4747` Add `--log-level DEBUG` to the command to see the detailed logs. Then, in your training script, create a [`LightningStoreClient`](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.LightningStoreClient " agentlightning.LightningStoreClient") and pass it to the trainer: `[](https://microsoft.github.io/agent-lightning/stable/tutorials/debug/#__codelineno-17-1) client = agl.LightningStoreClient("http://localhost:4747") [](https://microsoft.github.io/agent-lightning/stable/tutorials/debug/#__codelineno-17-2) trainer = agl.Trainer(store=client, ...)` Set the environment variable `AGL_MANAGED_STORE=0` so the trainer doesn't attempt to manage the store automatically. **2\. Start the runner and algorithm processes separately.** Each process should run the same training script, but with different environment variables specifying the current role. This setup faithfully mirrors how [`Trainer.fit`](https://microsoft.github.io/agent-lightning/stable/reference/trainer/#agentlightning.Trainer.fit " fit(agent, train_dataset=None, *, val_dataset=None)") orchestrates these components behind the scenes. `[](https://microsoft.github.io/agent-lightning/stable/tutorials/debug/#__codelineno-18-1) # Terminal 2 – Runner process [](https://microsoft.github.io/agent-lightning/stable/tutorials/debug/#__codelineno-18-2) AGL_MANAGED_STORE=0 AGL_CURRENT_ROLE=runner \ [](https://microsoft.github.io/agent-lightning/stable/tutorials/debug/#__codelineno-18-3) python train_calc_agent.py --external-store-address http://localhost:4747 --val-file data/test_mini.parquet [](https://microsoft.github.io/agent-lightning/stable/tutorials/debug/#__codelineno-18-4) [](https://microsoft.github.io/agent-lightning/stable/tutorials/debug/#__codelineno-18-5) # Terminal 3 – Algorithm process [](https://microsoft.github.io/agent-lightning/stable/tutorials/debug/#__codelineno-18-6) AGL_MANAGED_STORE=0 AGL_CURRENT_ROLE=algorithm \ [](https://microsoft.github.io/agent-lightning/stable/tutorials/debug/#__codelineno-18-7) python train_calc_agent.py --external-store-address http://localhost:4747 --val-file data/test_mini.parquet` **3\. Reuse your existing trainer configuration.** You can continue using the same datasets, adapters, and proxies as usual. Because the store is now external, you can: * Attach debuggers to either the algorithm or runner process * Add fine-grained logging or tracing * Simulate partial failures or latency in individual components This setup provides a faithful reproduction of the algorithm–runner interaction while keeping the store visible for inspection. Once you’ve resolved the issue, simply set `AGL_MANAGED_STORE=1` (or omit it) to return to the standard managed training workflow. Back to top --- # Serving LLM - Agent-lightning [Skip to content](https://microsoft.github.io/agent-lightning/stable/deep-dive/serving-llm/#serving-llms-under-agent-lightning) ⚠️ You are viewing development documentation. For stable documentation, please visit [https://microsoft.github.io/agent-lightning/stable/](https://microsoft.github.io/agent-lightning/stable/) Serving LLMs under Agent-lightning[¶](https://microsoft.github.io/agent-lightning/stable/deep-dive/serving-llm/#serving-llms-under-agent-lightning "Permanent link") ===================================================================================================================================================================== Agent-lightning focuses on data, learning signals, and control flow — **not** on running model inference. This deep dive explains how to **serve** a model alongside Agent-lightning so runners can call it reliably, how the **LLM Proxy** fits into the loop, and why **token IDs** matter if you care about correctness in training and evaluation. General background on LLM serving[¶](https://microsoft.github.io/agent-lightning/stable/deep-dive/serving-llm/#general-background-on-llm-serving "Permanent link") ------------------------------------------------------------------------------------------------------------------------------------------------------------------- [](https://microsoft.github.io/agent-lightning/stable/deep-dive/serving-llm/) Serving a model is essential if you want to train it, especially when you use the model’s own generations as training data. We’ll briefly review the general background to ensure all readers are aligned. Modern LLM servers solve a difficult scheduling problem: keeping GPUs fully utilized while handling prompts of different lengths, streaming tokens as they arrive, and fitting large KV caches into limited memory. Techniques like [**continuous batching**](https://www.anyscale.com/blog/continuous-batching-llm-inference) and [**paged attention**](https://arxiv.org/abs/2309.06180) address these challenges. Continuous batching interleaves decoding across requests to reuse weights efficiently; with careful memory planning, it achieves major throughput gains without increasing latency. PagedAttention reduces KV-cache fragmentation so batching remains effective as sequences grow. See [vLLM’s PagedAttention paper](https://arxiv.org/abs/2309.06180) and [industry analyses](https://www.baseten.co/blog/continuous-vs-dynamic-batching-for-ai-inference/) for details. Balancing inference correctness and efficiency is difficult — a [recent blog](https://thinkingmachines.ai/blog/defeating-nondeterminism-in-llm-inference/) from Thinking Machines Labs highlights how inference nondeterminism ultimately affects training. Beyond scheduling, servers expose an HTTP API, often **OpenAI-compatible** (`/v1/chat/completions` and `/v1/responses`), which is itself a complex stack. In addition to text prompts and chat messages, the API defines many parameters and response fields such as [tool calls](https://platform.openai.com/docs/guides/function-calling) , [structured output](https://platform.openai.com/docs/guides/structured-outputs) , and [multimodal support](https://platform.openai.com/docs/guides/images-vision) . Much effort has been put into implementing all these parameters for many frameworks. Popular engines like **vLLM** and [**SGLang**](https://github.com/sgl-project/sglang) ship with OpenAI-compatible frontends so you can reuse existing client code. [Ollama](https://ollama.com/blog/openai-compatibility) and [llama.cpp](https://llama-cpp-python.readthedocs.io/en/latest/server/) provide similar capabilities. However, because models differ internally, each framework interprets and implements the API slightly differently. Even with identical requests, the tokens passed to the model can vary substantially across frameworks. What Agent-lightning expects from a served LLM[¶](https://microsoft.github.io/agent-lightning/stable/deep-dive/serving-llm/#what-agent-lightning-expects-from-a-served-llm "Permanent link") --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Most of the issues above either have workarounds or remain open research problems. Keep them in mind, but the key question is: what does Agent-lightning expect from a served LLM? The answer includes at least two things: * An OpenAI-compatible **Chat Completions** or **Responses** endpoint the agent can call during rollouts. * Optional training and debugging signals: **logprobs**, **usage**, and ideally **token IDs**. (OpenAI’s public API exposes usage and logprobs, but **not** token IDs — more on [why IDs matter](https://microsoft.github.io/agent-lightning/stable/deep-dive/serving-llm/#token-ids-matter "Token IDs and why they matter") later.) Launching a serving framework[¶](https://microsoft.github.io/agent-lightning/stable/deep-dive/serving-llm/#launching-a-serving-framework "Permanent link") ----------------------------------------------------------------------------------------------------------------------------------------------------------- For many algorithms, you’ll start an engine (e.g., **vLLM** or **SGLang**) before rollouts, then shut it down afterward to free GPU memory. Most frameworks provide a one-line “serve” command to launch the OpenAI-compatible server. You can use those to bring up `/v1/chat/completions` with your checkpoint, ensuring streaming and any required tool-calling features are enabled. A working example is shown in [Unsloth SFT](https://microsoft.github.io/agent-lightning/stable/how-to/unsloth-sft/) . Weight updates — which occur after each training step — are trickier. Some frameworks like [vLLM](https://vllm.ai/) support hot-updating model weights, but it’s usually simpler and more reliable to restart the engine to load new weights. For medium-sized tasks (hundreds of rollouts taking 10+ minutes), the restart overhead (under 30 seconds) is typically negligible. If you’re using Agent-lightning’s [**VERL**](https://microsoft.github.io/agent-lightning/stable/algorithm-zoo/verl/#agentlightning.algorithm.verl.VERL " VERL") integration, the algorithm can **manage the server automatically**. The [VERL framework](https://github.com/volcengine/verl) intelligently allocates compute resources and wraps vLLM/SGLang behind an `AsyncLLMServer` abstraction. You can directly use this as the LLM endpoint for agents. Since VERL can spawn multiple vLLM replicas, using [`LLMProxy`](https://microsoft.github.io/agent-lightning/stable/reference/algorithm/#agentlightning.LLMProxy " agentlightning.LLMProxy") to manage them adds an additional safety layer. A full sequence diagram of how [VERL](https://microsoft.github.io/agent-lightning/stable/algorithm-zoo/verl/#agentlightning.algorithm.verl.VERL " VERL") interacts with the LLM server and proxy is available [here](https://microsoft.github.io/agent-lightning/stable/deep-dive/birds-eye-view/#birds-eye-view-verl-example "Putting It All Together: A Reinforcement Learning Example (VERL)") . LLM Proxy[¶](https://microsoft.github.io/agent-lightning/stable/deep-dive/serving-llm/#llm-proxy "Permanent link") ------------------------------------------------------------------------------------------------------------------- The **LLM Proxy** is a utility class in Agent-lightning, built on [LiteLLM](https://docs.litellm.ai/) , that sits between runners and your backend engine(s) or server(s). In Agent-lightning it acts as a single URL registered as a [`Resource`](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.Resource " agentlightning.Resource") in the store, offering three key benefits: 1. **Unified endpoint & hot-swaps.** You can redirect traffic between OpenAI, Anthropic, local vLLM/SGLang, or canary checkpoints without modifying agent code — simply repoint the proxy. 2. **First-class tracing.** The proxy emits **OpenTelemetry** spans for every call and sends them to the [`LightningStore`](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.LightningStore " agentlightning.LightningStore") . It includes rollout and attempt identifiers in request headers so spans are correctly attributed. Sequence numbers are allocated monotonically via the store to [prevent clock-skew issues](https://microsoft.github.io/agent-lightning/stable/tutorials/traces/#distributed-tracing "LLM Proxy") and allow reliable reconstruction of execution trees. 3. **Token IDs.** The proxy can return prompt and response token IDs along with the model output. More details are available in the [next section](https://microsoft.github.io/agent-lightning/stable/deep-dive/serving-llm/#token-ids-matter "Token IDs and why they matter") . Operationally, running the proxy alongside the algorithm works best: the algorithm registers the backend (e.g., the vLLM URL) via [`LLMProxy.update_model_list`](https://microsoft.github.io/agent-lightning/stable/reference/algorithm/#agentlightning.LLMProxy.update_model_list " update_model_list(model_list)") , publishes the proxy URL as a resource via [`LightningStore.add_resources`](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.LightningStore.add_resources " add_resources(resources) async ") , and runners simply use that URL during rollouts. This mirrors many production client–server setups. Token IDs and why they matter[¶](https://microsoft.github.io/agent-lightning/stable/deep-dive/serving-llm/#token-ids-and-why-they-matter "Permanent link") ----------------------------------------------------------------------------------------------------------------------------------------------------------- [](https://microsoft.github.io/agent-lightning/stable/deep-dive/serving-llm/) This section explains how Agent-lightning handles and uses token IDs — a subtle but important detail for training stability and accuracy. Most agents interact with LLMs via **Chat Completion APIs**, exchanging chat messages. There are two main approaches to collecting training data from such agents. Note Tokenization here refers to the process of converting **Chat Messages** into **Token IDs**. Detokenization is the reverse process of converting **Token IDs** back to **Chat Messages**. Normally, the tokenizer is published along with the pretrained model, which includes a vocabulary, special tokens, and a chat template to dealing with chat messages. **1\. Retokenizing chat messages.** In this approach, you store chat messages as text and let training algorithms **retokenize** them later, as done in many SFT workflows (e.g., [HuggingFace SFT](https://huggingface.co/docs/trl/sft_trainer) ). In practice, we’ve found this method unstable and less accurate. The chart below compares training results. The retokenization approach is run twice. All settings are the same except for the retokenization approach. This instability has three causes. Firstly, chat template used in different frameworks could be slightly different. For example, one single LLaMA model can work with multiple chat templates (multiple in [vLLM](https://github.com/vllm-project/vllm/tree/1d165d6d859d3c50720f0c07209db2363c4fd33b/examples) and one in [HuggingFace](https://huggingface.co/meta-llama) ). It's possible that the chat template used in detokenization is different from the one used in tokenization (this is actually an implementation bug). Secondly, a word might be generated as two tokens (e.g., `H + AVING`) but later retokenized as `HAV + ING`. The text looks identical, but the token IDs differ from what the model originally produced. Thirdly, a generated tool call text like `{ "name": ... }` is parsed by tool call parser into an object that is required by chat completion API. Later, the object is rendered back to `{ "name": ... }` and retokenized again, tool call parsing and re-rendering might cause changes in whitespace and formatting. In some situations, JSON errors may even be auto-corrected by the tool call parser — masking the model’s true generation errors and preventing them from being trained away. * * * **2\. Saving token IDs directly.** The alternative is to save the token IDs generated by the model, as done in RL setups like [Tinker](https://thinkingmachines.ai/tinker/) . This requires a training pipeline that treats tokens as first-class entities, meaning agents must communicate with the inference engine at the token level. However, most agents — especially those built with frameworks like LangChain — rely on OpenAI-compatible APIs and can’t tokenize or detokenize themselves. As mentioned [earlier](https://microsoft.github.io/agent-lightning/stable/deep-dive/serving-llm/#general-llm-serving-background "General background on LLM serving") , implementing this layer manually is complex and error-prone. Some frameworks implement custom solutions (e.g., [VERL Agent Loop](https://github.com/volcengine/verl/blob/4da0d3d3188072772cb2ec817b3d6cf4a463821f/recipe/langgraph_agent/chat_model.py) , [Tinker Renderer](https://github.com/thinking-machines-lab/tinker-cookbook/blob/34a6588d7055040c259985d98e71c0140b389ba7/tinker_cookbook/renderers.py) ), while others leave it to users (e.g., [SkyRL Search-R1](https://novasky-ai.notion.site/skyrl-searchr1) ). * * * A better solution is to use an **OpenAI-compatible API that returns token IDs directly.** This lets agents continue using familiar APIs while capturing token IDs via [tracing](https://microsoft.github.io/agent-lightning/stable/tutorials/traces/) for training. The limitation, of course, is that the serving framework must actually support this capability. When Agent-lightning was first released, we implemented an [instrumented vLLM server](https://github.com/microsoft/agent-lightning/blob/v0.1/agentlightning/instrumentation/vllm.py) that monkey-patched vLLM’s OpenAI server to return token IDs. Since then, the Agent-lightning and vLLM teams have collaborated to add this feature directly to [vLLM core](https://github.com/vllm-project/vllm/pull/22587) . Starting with **vLLM v0.10.2**, the OpenAI-compatible API includes a [`return_token_ids` parameter](https://docs.vllm.ai/en/v0.10.2/serving/openai_compatible_server.html#api-reference) , allowing token IDs to be requested alongside chat messages. SGLang has tracked [similar feature requests](https://github.com/sgl-project/sglang/issues/2634) , though its OpenAI-compatible layer doesn’t yet support them. In short, when using vLLM v0.10.2 or newer, [`LLMProxy`](https://microsoft.github.io/agent-lightning/stable/reference/algorithm/#agentlightning.LLMProxy " agentlightning.LLMProxy") automatically adds `return_token_ids` to each request so the engine includes token IDs in its response. For older vLLM versions, you still need the instrumented version (via `agl vllm` CLI command). Finally, if you only save token IDs in spans, it will have its own limitations — if you train one model using spans from another model with a different tokenizer, incompatibilities can arise. In practice, though, spans in Agent-lightning always store both chat messages and token IDs (actually the full request and response objects), allowing you to fall back to retokenization when necessary. Back to top --- # APO - Agent-lightning [Skip to content](https://microsoft.github.io/agent-lightning/stable/algorithm-zoo/apo/#apo) ⚠️ You are viewing development documentation. For stable documentation, please visit [https://microsoft.github.io/agent-lightning/stable/](https://microsoft.github.io/agent-lightning/stable/) APO[¶](https://microsoft.github.io/agent-lightning/stable/algorithm-zoo/apo/#apo "Permanent link") =================================================================================================== Shortcut You can use the shortcut `agl.APO(...)` to create an APO instance. `[](https://microsoft.github.io/agent-lightning/stable/algorithm-zoo/apo/#__codelineno-0-1) import agentlightning as agl [](https://microsoft.github.io/agent-lightning/stable/algorithm-zoo/apo/#__codelineno-0-2) [](https://microsoft.github.io/agent-lightning/stable/algorithm-zoo/apo/#__codelineno-0-3) agl.APO(...)` Installation[¶](https://microsoft.github.io/agent-lightning/stable/algorithm-zoo/apo/#installation "Permanent link") --------------------------------------------------------------------------------------------------------------------- `[](https://microsoft.github.io/agent-lightning/stable/algorithm-zoo/apo/#__codelineno-1-1) pip install agentlightning[apo]` Scope of Current Implementation[¶](https://microsoft.github.io/agent-lightning/stable/algorithm-zoo/apo/#scope-of-current-implementation "Permanent link") ----------------------------------------------------------------------------------------------------------------------------------------------------------- APO is currently scoped to optimize a single prompt template. Optimizing multiple prompt templates is not supported yet. There is however no restriction on the number of variable placeholders in the prompt template (can range from zero to many). It's possible that invalid prompts are created during the optimization process. It is up to the agent developer to ensure that the prompt template is valid for the agent's task. Initial Prompt[¶](https://microsoft.github.io/agent-lightning/stable/algorithm-zoo/apo/#initial-prompt "Permanent link") ------------------------------------------------------------------------------------------------------------------------- APO expects the initial prompt to be provided in the `initial_resources` dictionary. This can be done in two approaches: 1. Pass to the [Trainer](https://microsoft.github.io/agent-lightning/stable/reference/trainer/#agentlightning.Trainer " agentlightning.Trainer") constructor: `[](https://microsoft.github.io/agent-lightning/stable/algorithm-zoo/apo/#__codelineno-2-1) trainer = agl.Trainer( [](https://microsoft.github.io/agent-lightning/stable/algorithm-zoo/apo/#__codelineno-2-2) algorithm=agl.APO(...), [](https://microsoft.github.io/agent-lightning/stable/algorithm-zoo/apo/#__codelineno-2-3) initial_resources={"main_prompt": agl.PromptTemplate(template="You are a helpful assistant.", engine="f-string")}, [](https://microsoft.github.io/agent-lightning/stable/algorithm-zoo/apo/#__codelineno-2-4) )` 1. Pass to the `[APO][agentlightning.algorithm.apo.APO].set_initial_resources()` method: `[](https://microsoft.github.io/agent-lightning/stable/algorithm-zoo/apo/#__codelineno-3-1) algo = agl.APO(...) [](https://microsoft.github.io/agent-lightning/stable/algorithm-zoo/apo/#__codelineno-3-2) algo.set_initial_resources( [](https://microsoft.github.io/agent-lightning/stable/algorithm-zoo/apo/#__codelineno-3-3) {"this_is_also_valid_key": agl.PromptTemplate(template="You are a helpful assistant.", engine="f-string")} [](https://microsoft.github.io/agent-lightning/stable/algorithm-zoo/apo/#__codelineno-3-4) )` The resource key can be arbitrary, which is used to identify the prompt template in [class-based implementations](https://microsoft.github.io/agent-lightning/stable/tutorials/write-agents/) when you have multiple resources. When the key changes, the agent developer needs to update the key in the `rollout` method. Tutorials Using APO[¶](https://microsoft.github.io/agent-lightning/stable/algorithm-zoo/apo/#tutorials-using-apo "Permanent link") ----------------------------------------------------------------------------------------------------------------------------------- * [Train the First Agent with APO](https://microsoft.github.io/agent-lightning/stable/how-to/train-first-agent/) - A step-by-step guide to training your first agent using APO. References[¶](https://microsoft.github.io/agent-lightning/stable/algorithm-zoo/apo/#references "Permanent link") ----------------------------------------------------------------------------------------------------------------- ### `agentlightning.algorithm.apo` [¶](https://microsoft.github.io/agent-lightning/stable/algorithm-zoo/apo/#agentlightning.algorithm.apo "Permanent link") #### `APO` [¶](https://microsoft.github.io/agent-lightning/stable/algorithm-zoo/apo/#agentlightning.algorithm.apo.APO "Permanent link") Bases: `[Algorithm](https://microsoft.github.io/agent-lightning/stable/reference/algorithm/#agentlightning.Algorithm " agentlightning.Algorithm (agentlightning.algorithm.base.Algorithm)") `, `Generic[T_task]` Automatic Prompt Optimization (APO) algorithm using textual gradients and beam search. APO is an iterative prompt optimization algorithm that uses LLM-generated textual gradients to improve prompts through a beam search process. It evaluates prompts on rollouts, computes critiques based on the results, and applies edits to generate improved prompts. The algorithm operates in rounds, where each round: 1. Samples parent prompts from the current beam 2. Generates new prompts by computing textual gradients and applying edits 3. Evaluates all candidates on a validation set 4. Selects the top-k prompts for the next round Based on the ideas from: * [ProTeGi](https://aclanthology.org/2023.emnlp-main.494.pdf) * [TextGrad](https://github.com/zou-group/textgrad) ##### `__init__(async_openai_client, *, gradient_model='gpt-5-mini', apply_edit_model='gpt-4.1-mini', diversity_temperature=1.0, gradient_batch_size=4, val_batch_size=16, beam_width=4, branch_factor=4, beam_rounds=3, rollout_batch_timeout=3600.0, run_initial_validation=True, _poml_trace=False)` [¶](https://microsoft.github.io/agent-lightning/stable/algorithm-zoo/apo/#agentlightning.algorithm.apo.APO.__init__ "Permanent link") Initialize the APO algorithm with configuration parameters. Parameters: * **`async_openai_client`** (`AsyncOpenAI`) – AsyncOpenAI client for making LLM API calls. * **`gradient_model`** (`str`, default: `'gpt-5-mini'` ) – Model name for computing textual gradients (critiques). * **`apply_edit_model`** (`str`, default: `'gpt-4.1-mini'` ) – Model name for applying edits based on critiques. * **`diversity_temperature`** (`float`, default: `1.0` ) – Temperature parameter for LLM calls to control diversity. * **`gradient_batch_size`** (`int`, default: `4` ) – Number of rollout results to sample for gradient computation. * **`val_batch_size`** (`int`, default: `16` ) – Number of validation examples to use for evaluation. * **`beam_width`** (`int`, default: `4` ) – Number of top-scoring prompts to keep in the beam at each round. * **`branch_factor`** (`int`, default: `4` ) – Number of new prompt candidates to generate from each parent prompt by applying textual gradient edits. This controls the expansion of the search tree. * **`beam_rounds`** (`int`, default: `3` ) – Number of beam search rounds to perform. * **`rollout_batch_timeout`** (`float`, default: `3600.0` ) – Maximum time in seconds to wait for rollout batch completion. * **`run_initial_validation`** (`bool`, default: `True` ) – If True, runs validation on the seed prompt before starting optimization to establish a baseline score. Defaults to True. ##### `compute_textual_gradient(current_prompt, rollout_results, *, prefix=None)` `async` [¶](https://microsoft.github.io/agent-lightning/stable/algorithm-zoo/apo/#agentlightning.algorithm.apo.APO.compute_textual_gradient "Permanent link") Compute a textual gradient (critique) for the current prompt based on rollout results. This method samples rollout results, sends them to an LLM along with the current prompt, and generates a critique describing how the prompt could be improved. Parameters: * **`current_prompt`** (`VersionedPromptTemplate`) – The prompt template to critique. * **`rollout_results`** (`List[RolloutResultForAPO]`) – List of rollout results containing spans, messages, and rewards. Returns: * `Optional[str]` – A textual critique generated by the LLM, or None if generation fails. ##### `evaluate_prompt_on_batch(prompt, resource_name, dataset, mode, *, prefix=None)` `async` [¶](https://microsoft.github.io/agent-lightning/stable/algorithm-zoo/apo/#agentlightning.algorithm.apo.APO.evaluate_prompt_on_batch "Permanent link") Evaluate a prompt on a batch of tasks by running rollouts and computing average reward. This method: 1. Adds the prompt as a named resource to the store 2. Enqueues rollouts for each task in the dataset 3. Waits for rollouts to complete (with timeout) 4. Computes and returns the average reward Parameters: * **`prompt`** (`VersionedPromptTemplate`) – The prompt template string to evaluate. * **`resource_name`** (`str`) – The name to register the prompt under in the store. * **`dataset`** (`Sequence[T_task]`) – Sequence of tasks to evaluate the prompt on. * **`mode`** (`[RolloutMode](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.RolloutMode " agentlightning.RolloutMode = Literal['train', 'val', 'test'] module-attribute (agentlightning.types.RolloutMode)") `) – Rollout mode ("train" or "val") for logging/tracking. Returns: * `List[RolloutResultForAPO]` – A tuple of (rollout\_results, average\_reward) where rollout\_results contains * `float` – detailed information for each rollout and average\_reward is the mean final reward. ##### `get_adapter()` [¶](https://microsoft.github.io/agent-lightning/stable/algorithm-zoo/apo/#agentlightning.algorithm.apo.APO.get_adapter "Permanent link") Get the adapter for converting spans to messages. Returns: * `[TraceToMessages](https://microsoft.github.io/agent-lightning/stable/reference/algorithm/#agentlightning.TraceToMessages " agentlightning.TraceToMessages (agentlightning.adapter.messages.TraceToMessages)") ` – The TraceToMessages instance for this algorithm. Raises: * `ValueError` – If the adapter is not a TraceToMessages. ##### `get_best_prompt()` [¶](https://microsoft.github.io/agent-lightning/stable/algorithm-zoo/apo/#agentlightning.algorithm.apo.APO.get_best_prompt "Permanent link") Retrieve the best prompt discovered during optimization. Returns: * `[PromptTemplate](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.PromptTemplate " agentlightning.PromptTemplate (agentlightning.types.PromptTemplate)") ` – The prompt template with the highest validation score found so far. Raises: * `ValueError` – If no best prompt has been found yet (run() not called). ##### `get_rollout_results(store, rollout, *, prefix=None)` `async` [¶](https://microsoft.github.io/agent-lightning/stable/algorithm-zoo/apo/#agentlightning.algorithm.apo.APO.get_rollout_results "Permanent link") Convert completed rollouts to APO-compatible result format. Fetches spans for each rollout, adapts them to messages, and packages them with rewards and status information for gradient computation. Parameters: * **`rollout`** (`List[[Rollout](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.Rollout " agentlightning.Rollout (agentlightning.types.Rollout)") ]`) – List of completed rollout metadata. Returns: * `List[RolloutResultForAPO]` – List of rollout results formatted for APO processing. ##### `get_seed_prompt_template()` [¶](https://microsoft.github.io/agent-lightning/stable/algorithm-zoo/apo/#agentlightning.algorithm.apo.APO.get_seed_prompt_template "Permanent link") Extract the initial prompt template from the algorithm's resources. Returns: * `Tuple[str, [PromptTemplate](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.PromptTemplate " agentlightning.PromptTemplate (agentlightning.types.PromptTemplate)") ]` – A tuple of (resource\_name, prompt\_template) representing the seed prompt. Raises: * `ValueError` – If initial\_resources is not set or no PromptTemplate is found. ##### `run(store, llm_proxy, train_dataset=None, val_dataset=None)` `async` [¶](https://microsoft.github.io/agent-lightning/stable/algorithm-zoo/apo/#agentlightning.algorithm.apo.APO.run "Permanent link") Execute the APO algorithm to optimize prompts through beam search with textual gradients. The algorithm performs iterative prompt optimization over multiple rounds: * Each round: samples parent prompts, generates new candidates via textual gradients, evaluates all candidates on validation data, and keeps the top performers * Tracks the historically best prompt across all rounds * Uses different training data samples for each gradient computation to ensure diversity Parameters: * **`train_dataset`** (`Optional[[Dataset](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.Dataset " agentlightning.Dataset (agentlightning.types.Dataset)") [T_task]]`, default: `None` ) – Dataset of tasks for computing textual gradients. Required. * **`val_dataset`** (`Optional[[Dataset](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.Dataset " agentlightning.Dataset (agentlightning.types.Dataset)") [T_task]]`, default: `None` ) – Dataset of tasks for evaluating and selecting prompts. Required. Raises: * `ValueError` – If train\_dataset or val\_dataset is None, or if resources are not set. ##### `textual_gradient_and_apply_edit(current_prompt, rollout, *, prefix=None)` `async` [¶](https://microsoft.github.io/agent-lightning/stable/algorithm-zoo/apo/#agentlightning.algorithm.apo.APO.textual_gradient_and_apply_edit "Permanent link") Generate an improved prompt by computing a textual gradient and applying an edit. This is the main optimization step that: 1. Computes a critique (textual gradient) based on rollout performance 2. Uses another LLM to apply the critique and generate an improved prompt Parameters: * **`current_prompt`** (`VersionedPromptTemplate`) – The current prompt template to improve. * **`rollout`** (`List[RolloutResultForAPO]`) – List of rollout results to base the critique on. Returns: * `Optional[str]` – The improved prompt text, or the original prompt if gradient computation fails. Back to top --- # RESTful - Agent-lightning [Skip to content](https://microsoft.github.io/agent-lightning/stable/reference/restful/#restful-api-references) ⚠️ You are viewing development documentation. For stable documentation, please visit [https://microsoft.github.io/agent-lightning/stable/](https://microsoft.github.io/agent-lightning/stable/) RESTful API References[¶](https://microsoft.github.io/agent-lightning/stable/reference/restful/#restful-api-references "Permanent link") ========================================================================================================================================= Note Shown in the following is the RESTful API for Lightning Store. Back to top --- # Instrumentation - Agent-lightning [Skip to content](https://microsoft.github.io/agent-lightning/stable/reference/instrumentation/#instrumentation-api) ⚠️ You are viewing development documentation. For stable documentation, please visit [https://microsoft.github.io/agent-lightning/stable/](https://microsoft.github.io/agent-lightning/stable/) Instrumentation API[¶](https://microsoft.github.io/agent-lightning/stable/reference/instrumentation/#instrumentation-api "Permanent link") =========================================================================================================================================== ### `agentlightning.instrumentation.instrument_all()` [¶](https://microsoft.github.io/agent-lightning/stable/reference/instrumentation/#agentlightning.instrumentation.instrument_all "Permanent link") Instrument all the instrumentation libraries. ### `agentlightning.instrumentation.uninstrument_all()` [¶](https://microsoft.github.io/agent-lightning/stable/reference/instrumentation/#agentlightning.instrumentation.uninstrument_all "Permanent link") Uninstrument all the instrumentation libraries. AgentOps LangChain[¶](https://microsoft.github.io/agent-lightning/stable/reference/instrumentation/#agentops-langchain "Permanent link") ----------------------------------------------------------------------------------------------------------------------------------------- ### `agentlightning.instrumentation.agentops_langchain` [¶](https://microsoft.github.io/agent-lightning/stable/reference/instrumentation/#agentlightning.instrumentation.agentops_langchain "Permanent link") #### `instrument_agentops_langchain()` [¶](https://microsoft.github.io/agent-lightning/stable/reference/instrumentation/#agentlightning.instrumentation.agentops_langchain.instrument_agentops_langchain "Permanent link") Bypass AgentOp's native support for Langchain. #### `uninstrument_agentops_langchain()` [¶](https://microsoft.github.io/agent-lightning/stable/reference/instrumentation/#agentlightning.instrumentation.agentops_langchain.uninstrument_agentops_langchain "Permanent link") Restore AgentOp's native support for Langchain. AgentOps[¶](https://microsoft.github.io/agent-lightning/stable/reference/instrumentation/#agentops "Permanent link") --------------------------------------------------------------------------------------------------------------------- ### `agentlightning.instrumentation.agentops` [¶](https://microsoft.github.io/agent-lightning/stable/reference/instrumentation/#agentlightning.instrumentation.agentops "Permanent link") #### `BypassableAuthenticatedOTLPExporter` [¶](https://microsoft.github.io/agent-lightning/stable/reference/instrumentation/#agentlightning.instrumentation.agentops.BypassableAuthenticatedOTLPExporter "Permanent link") Bases: `LightningStoreOTLPExporter`, `AuthenticatedOTLPExporter` AuthenticatedOTLPExporter with switchable service control. When `_agentops_service_enabled` is False, skip export and return success. #### `BypassableOTLPMetricExporter` [¶](https://microsoft.github.io/agent-lightning/stable/reference/instrumentation/#agentlightning.instrumentation.agentops.BypassableOTLPMetricExporter "Permanent link") Bases: `OTLPMetricExporter` OTLPMetricExporter with switchable service control. When `_agentops_service_enabled` is False, skip export and return success. #### `BypassableOTLPSpanExporter` [¶](https://microsoft.github.io/agent-lightning/stable/reference/instrumentation/#agentlightning.instrumentation.agentops.BypassableOTLPSpanExporter "Permanent link") Bases: `LightningStoreOTLPExporter` OTLPSpanExporter with switchable service control. When `_agentops_service_enabled` is False, skip export and return success. This is used instead of BypassableAuthenticatedOTLPExporter on legacy AgentOps versions. #### `BypassableV3Client` [¶](https://microsoft.github.io/agent-lightning/stable/reference/instrumentation/#agentlightning.instrumentation.agentops.BypassableV3Client "Permanent link") Bases: `V3Client` V3Client with toggleable authentication calls. Returns dummy auth response when `_agentops_service_enabled` is False. #### `BypassableV4Client` [¶](https://microsoft.github.io/agent-lightning/stable/reference/instrumentation/#agentlightning.instrumentation.agentops.BypassableV4Client "Permanent link") Bases: `V4Client` V4Client with toggleable post requests. Returns dummy response when `_agentops_service_enabled` is False. #### `enable_agentops_service(enabled=True)` [¶](https://microsoft.github.io/agent-lightning/stable/reference/instrumentation/#agentlightning.instrumentation.agentops.enable_agentops_service "Permanent link") Enable or disable communication with the AgentOps service. By default, AgentOps exporters and clients will run in local mode and will NOT attempt to communicate with the remote AgentOps service. Parameters: * **`enabled`** (`bool`, default: `True` ) – If True, enable all AgentOps exporters and clients. All exporters and clients will operate in normal mode and send data to the [AgentOps service](https://www.agentops.ai/) . #### `instrument_agentops()` [¶](https://microsoft.github.io/agent-lightning/stable/reference/instrumentation/#agentlightning.instrumentation.agentops.instrument_agentops "Permanent link") Instrument agentops to capture token IDs. Automatically detects and uses the appropriate patching method based on the installed agentops version. #### `uninstrument_agentops()` [¶](https://microsoft.github.io/agent-lightning/stable/reference/instrumentation/#agentlightning.instrumentation.agentops.uninstrument_agentops "Permanent link") Uninstrument agentops to stop capturing token IDs. LiteLLM[¶](https://microsoft.github.io/agent-lightning/stable/reference/instrumentation/#litellm "Permanent link") ------------------------------------------------------------------------------------------------------------------- ### `agentlightning.instrumentation.litellm` [¶](https://microsoft.github.io/agent-lightning/stable/reference/instrumentation/#agentlightning.instrumentation.litellm "Permanent link") LiteLLM instrumentations. It's unclear whether or not this file is useful. It seems that LiteLLM owns its own telemetry from their own entrance [Related documentation](https://docs.litellm.ai/docs/observability/agentops_integration) . #### `instrument_litellm()` [¶](https://microsoft.github.io/agent-lightning/stable/reference/instrumentation/#agentlightning.instrumentation.litellm.instrument_litellm "Permanent link") Instrument litellm to capture token IDs. #### `uninstrument_litellm()` [¶](https://microsoft.github.io/agent-lightning/stable/reference/instrumentation/#agentlightning.instrumentation.litellm.uninstrument_litellm "Permanent link") Uninstrument litellm to stop capturing token IDs. vLLM[¶](https://microsoft.github.io/agent-lightning/stable/reference/instrumentation/#vllm "Permanent link") ------------------------------------------------------------------------------------------------------------- ### `agentlightning.instrumentation.vllm` [¶](https://microsoft.github.io/agent-lightning/stable/reference/instrumentation/#agentlightning.instrumentation.vllm "Permanent link") #### `instrument_vllm()` [¶](https://microsoft.github.io/agent-lightning/stable/reference/instrumentation/#agentlightning.instrumentation.vllm.instrument_vllm "Permanent link") Instrument vLLM to capture token IDs generated by engine. This instrumentation has been merged to upstream vLLM since v0.10.2. #### `uninstrument_vllm()` [¶](https://microsoft.github.io/agent-lightning/stable/reference/instrumentation/#agentlightning.instrumentation.vllm.uninstrument_vllm "Permanent link") Uninstrument vLLM to stop capturing token IDs generated by engine. Back to top --- # VERL - Agent-lightning [Skip to content](https://microsoft.github.io/agent-lightning/stable/algorithm-zoo/verl/#verl) ⚠️ You are viewing development documentation. For stable documentation, please visit [https://microsoft.github.io/agent-lightning/stable/](https://microsoft.github.io/agent-lightning/stable/) VERL[¶](https://microsoft.github.io/agent-lightning/stable/algorithm-zoo/verl/#verl "Permanent link") ====================================================================================================== Shortcut You can use the shortcut `agl.VERL(...)` to create a VERL instance. `[](https://microsoft.github.io/agent-lightning/stable/algorithm-zoo/verl/#__codelineno-0-1) import agentlightning as agl [](https://microsoft.github.io/agent-lightning/stable/algorithm-zoo/verl/#__codelineno-0-2) [](https://microsoft.github.io/agent-lightning/stable/algorithm-zoo/verl/#__codelineno-0-3) agl.VERL(...)` Installation[¶](https://microsoft.github.io/agent-lightning/stable/algorithm-zoo/verl/#installation "Permanent link") ---------------------------------------------------------------------------------------------------------------------- `[](https://microsoft.github.io/agent-lightning/stable/algorithm-zoo/verl/#__codelineno-1-1) pip install agentlightning[verl]` Warning To avoid various compatibility issues, follow the steps in the [installation guide](https://microsoft.github.io/agent-lightning/stable/tutorials/installation/) to set up VERL and its dependencies. Installing VERL directly with `pip install agentlightning[verl]` can cause issues unless you already have a compatible version of PyTorch installed. Notes for Readers [VERL](https://microsoft.github.io/agent-lightning/stable/algorithm-zoo/verl/#agentlightning.algorithm.verl.VERL " VERL") in this article refers to a wrapper, provided by Agent-lightning, of the [VERL framework](https://github.com/volcengine/verl) . It's a subclass of [agentlightning.Algorithm](https://microsoft.github.io/agent-lightning/stable/reference/algorithm/#agentlightning.Algorithm " agentlightning.Algorithm") . To differentiate it from the VERL framework, all references to the VERL framework shall use the term "VERL framework", and all references to the Agent-lightning wrapper shall be highlighted with a link. Resources[¶](https://microsoft.github.io/agent-lightning/stable/algorithm-zoo/verl/#resources "Permanent link") ---------------------------------------------------------------------------------------------------------------- [VERL](https://microsoft.github.io/agent-lightning/stable/algorithm-zoo/verl/#agentlightning.algorithm.verl.VERL " VERL") expects no initial resources. The first LLM endpoint is directly deployed from the VERL configuration (`.actor_rollout_ref.model.path`). The resource key is always `main_llm`. [VERL](https://microsoft.github.io/agent-lightning/stable/algorithm-zoo/verl/#agentlightning.algorithm.verl.VERL " VERL") currently does not support optimizing multiple [LLM](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.LLM " agentlightning.LLM") s together. Note The resource type created by [VERL](https://microsoft.github.io/agent-lightning/stable/algorithm-zoo/verl/#agentlightning.algorithm.verl.VERL " VERL") is actually a [ProxyLLM](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.ProxyLLM " agentlightning.ProxyLLM") , a subclass of the [LLM](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.LLM " agentlightning.LLM") type. This object contains a **URL template** provided by [VERL](https://microsoft.github.io/agent-lightning/stable/algorithm-zoo/verl/#agentlightning.algorithm.verl.VERL " VERL") , with placeholders for rollout and attempt IDs. When a rollout begins on the agent side, the framework uses the current `rollout_id` and `attempt_id` to format this template, generating a final, unique endpoint URL. This URL points to [VERL](https://microsoft.github.io/agent-lightning/stable/algorithm-zoo/verl/#agentlightning.algorithm.verl.VERL " VERL") 's internal proxy, allowing it to intercept and log all traffic for that specific attempt, for tracing and load balancing purposes. For agents created with the `@rollout` decorator, this resolution of the template is handled automatically ("auto-stripped"). Class-based agents will need to manually resolve the [`ProxyLLM`](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.ProxyLLM " agentlightning.ProxyLLM") using the rollout context. `[](https://microsoft.github.io/agent-lightning/stable/algorithm-zoo/verl/#__codelineno-2-1) proxy_llm = resources["main_llm"] [](https://microsoft.github.io/agent-lightning/stable/algorithm-zoo/verl/#__codelineno-2-2) proxy_llm.get_base_url(rollout.rollout_id, rollout.attempt.attempt_id)` Customization[¶](https://microsoft.github.io/agent-lightning/stable/algorithm-zoo/verl/#customization "Permanent link") ------------------------------------------------------------------------------------------------------------------------ Internally, [VERL](https://microsoft.github.io/agent-lightning/stable/algorithm-zoo/verl/#agentlightning.algorithm.verl.VERL " VERL") decomposes each agent execution into prompt–response pairs via the [Adapter](https://microsoft.github.io/agent-lightning/stable/reference/algorithm/#agentlightning.Adapter " agentlightning.Adapter") and associates them with their corresponding reward signals as [Triplet](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.Triplet " agentlightning.Triplet") objects. The final scalar reward, derived from the last triplet in the trajectory, is propagated to all preceding triplets following the [identical assignment strategy](https://arxiv.org/abs/2508.03680) . This ensures that each triplet receives an identical reward signal and can be independently optimized as a valid RLHF trajectory within the VERL framework. At present, [VERL](https://microsoft.github.io/agent-lightning/stable/algorithm-zoo/verl/#agentlightning.algorithm.verl.VERL " VERL") does not expose fine-grained control over its reward propagation or credit assignment mechanisms. Users requiring customized reward shaping or trajectory decomposition are advised to clone and modify the [VERL](https://microsoft.github.io/agent-lightning/stable/algorithm-zoo/verl/#agentlightning.algorithm.verl.VERL " VERL") source implementation directly. Tutorials Using VERL[¶](https://microsoft.github.io/agent-lightning/stable/algorithm-zoo/verl/#tutorials-using-verl "Permanent link") -------------------------------------------------------------------------------------------------------------------------------------- * [Train SQL Agent with RL](https://microsoft.github.io/agent-lightning/stable/how-to/train-sql-agent/) - A practical example of training a SQL agent using VERL. References - Entrypoint[¶](https://microsoft.github.io/agent-lightning/stable/algorithm-zoo/verl/#references-entrypoint "Permanent link") ------------------------------------------------------------------------------------------------------------------------------------------ ### `agentlightning.algorithm.verl` [¶](https://microsoft.github.io/agent-lightning/stable/algorithm-zoo/verl/#agentlightning.algorithm.verl "Permanent link") #### `VERL` [¶](https://microsoft.github.io/agent-lightning/stable/algorithm-zoo/verl/#agentlightning.algorithm.verl.VERL "Permanent link") Bases: `[Algorithm](https://microsoft.github.io/agent-lightning/stable/reference/algorithm/#agentlightning.Algorithm " agentlightning.Algorithm (agentlightning.algorithm.base.Algorithm)") ` VERL-powered algorithm that delegates training to the VERL PPO runner. Warning Advanced customisation currently requires copying the VERL source and modifying it directly. Native hooks for overriding training behaviour will land in a future release. Parameters: * **`config`** (`dict[str, Any]`) – Dictionary mirroring the overrides passed to the VERL CLI. The overrides are merged with VERL's packaged defaults via Hydra before launching training. * **`trainer_cls`** (`Optional[Type[[AgentLightningTrainer](https://microsoft.github.io/agent-lightning/stable/algorithm-zoo/verl/#agentlightning.verl.AgentLightningTrainer " AgentLightningTrainer (agentlightning.verl.trainer.AgentLightningTrainer)") ]]`, default: `None` ) – Optional override for the trainer class. Experimental. * **`daemon_cls`** (`Optional[Type[[AgentModeDaemon](https://microsoft.github.io/agent-lightning/stable/algorithm-zoo/verl/#agentlightning.verl.AgentModeDaemon " AgentModeDaemon (agentlightning.verl.daemon.AgentModeDaemon)") ]]`, default: `None` ) – Optional override for the daemon class. Experimental. Trajectory aggregation (experimental) Trajectory-level aggregation merges an entire multi-turn rollout into a single, masked training sample so GPU time is spent once per trajectory rather than N times per turn. Enable it via: `[](https://microsoft.github.io/agent-lightning/stable/algorithm-zoo/verl/#__codelineno-0-1) config["agentlightning"]["trace_aggregator"] = { [](https://microsoft.github.io/agent-lightning/stable/algorithm-zoo/verl/#__codelineno-0-2) "level": "trajectory", [](https://microsoft.github.io/agent-lightning/stable/algorithm-zoo/verl/#__codelineno-0-3) "trajectory_max_prompt_length": 4096, [](https://microsoft.github.io/agent-lightning/stable/algorithm-zoo/verl/#__codelineno-0-4) "trajectory_max_response_length": 34384, [](https://microsoft.github.io/agent-lightning/stable/algorithm-zoo/verl/#__codelineno-0-5) }` Keep conversations structured (message lists rather than manual string concatenation) so prefix matching can stitch traces. `trajectory_max_prompt_length` should be set to the maximum length of the prompt for the first turn, and `trajectory_max_response_length` should be set to the maximum cumulative length of agent responses in the full trajectory. Toggle `debug=True` plus `mismatch_log_dir` when you need to inspect retokenization or chat-template mismatches. See [this blog post](https://agent-lightning.github.io/posts/trajectory_level_aggregation/) for more details. Examples: `[](https://microsoft.github.io/agent-lightning/stable/algorithm-zoo/verl/#__codelineno-0-1) from agentlightning.algorithm.verl import VERL [](https://microsoft.github.io/agent-lightning/stable/algorithm-zoo/verl/#__codelineno-0-2) [](https://microsoft.github.io/agent-lightning/stable/algorithm-zoo/verl/#__codelineno-0-3) algorithm = VERL( [](https://microsoft.github.io/agent-lightning/stable/algorithm-zoo/verl/#__codelineno-0-4) config={ [](https://microsoft.github.io/agent-lightning/stable/algorithm-zoo/verl/#__codelineno-0-5) "algorithm": { [](https://microsoft.github.io/agent-lightning/stable/algorithm-zoo/verl/#__codelineno-0-6) "adv_estimator": "grpo", [](https://microsoft.github.io/agent-lightning/stable/algorithm-zoo/verl/#__codelineno-0-7) "use_kl_in_reward": False, [](https://microsoft.github.io/agent-lightning/stable/algorithm-zoo/verl/#__codelineno-0-8) }, [](https://microsoft.github.io/agent-lightning/stable/algorithm-zoo/verl/#__codelineno-0-9) "data": { [](https://microsoft.github.io/agent-lightning/stable/algorithm-zoo/verl/#__codelineno-0-10) "train_batch_size": 32, [](https://microsoft.github.io/agent-lightning/stable/algorithm-zoo/verl/#__codelineno-0-11) "max_prompt_length": 4096, [](https://microsoft.github.io/agent-lightning/stable/algorithm-zoo/verl/#__codelineno-0-12) "max_response_length": 2048, [](https://microsoft.github.io/agent-lightning/stable/algorithm-zoo/verl/#__codelineno-0-13) }, [](https://microsoft.github.io/agent-lightning/stable/algorithm-zoo/verl/#__codelineno-0-14) "actor_rollout_ref": { [](https://microsoft.github.io/agent-lightning/stable/algorithm-zoo/verl/#__codelineno-0-15) "rollout": { [](https://microsoft.github.io/agent-lightning/stable/algorithm-zoo/verl/#__codelineno-0-16) "tensor_model_parallel_size": 1, [](https://microsoft.github.io/agent-lightning/stable/algorithm-zoo/verl/#__codelineno-0-17) "n": 4, [](https://microsoft.github.io/agent-lightning/stable/algorithm-zoo/verl/#__codelineno-0-18) "log_prob_micro_batch_size_per_gpu": 4, [](https://microsoft.github.io/agent-lightning/stable/algorithm-zoo/verl/#__codelineno-0-19) "multi_turn": {"format": "hermes"}, [](https://microsoft.github.io/agent-lightning/stable/algorithm-zoo/verl/#__codelineno-0-20) "name": "vllm", [](https://microsoft.github.io/agent-lightning/stable/algorithm-zoo/verl/#__codelineno-0-21) "gpu_memory_utilization": 0.6, [](https://microsoft.github.io/agent-lightning/stable/algorithm-zoo/verl/#__codelineno-0-22) }, [](https://microsoft.github.io/agent-lightning/stable/algorithm-zoo/verl/#__codelineno-0-23) "actor": { [](https://microsoft.github.io/agent-lightning/stable/algorithm-zoo/verl/#__codelineno-0-24) "ppo_mini_batch_size": 32, [](https://microsoft.github.io/agent-lightning/stable/algorithm-zoo/verl/#__codelineno-0-25) "ppo_micro_batch_size_per_gpu": 4, [](https://microsoft.github.io/agent-lightning/stable/algorithm-zoo/verl/#__codelineno-0-26) "optim": {"lr": 1e-6}, [](https://microsoft.github.io/agent-lightning/stable/algorithm-zoo/verl/#__codelineno-0-27) "use_kl_loss": False, [](https://microsoft.github.io/agent-lightning/stable/algorithm-zoo/verl/#__codelineno-0-28) "kl_loss_coef": 0.0, [](https://microsoft.github.io/agent-lightning/stable/algorithm-zoo/verl/#__codelineno-0-29) "entropy_coeff": 0, [](https://microsoft.github.io/agent-lightning/stable/algorithm-zoo/verl/#__codelineno-0-30) "clip_ratio_low": 0.2, [](https://microsoft.github.io/agent-lightning/stable/algorithm-zoo/verl/#__codelineno-0-31) "clip_ratio_high": 0.3, [](https://microsoft.github.io/agent-lightning/stable/algorithm-zoo/verl/#__codelineno-0-32) "fsdp_config": { [](https://microsoft.github.io/agent-lightning/stable/algorithm-zoo/verl/#__codelineno-0-33) "param_offload": True, [](https://microsoft.github.io/agent-lightning/stable/algorithm-zoo/verl/#__codelineno-0-34) "optimizer_offload": True, [](https://microsoft.github.io/agent-lightning/stable/algorithm-zoo/verl/#__codelineno-0-35) }, [](https://microsoft.github.io/agent-lightning/stable/algorithm-zoo/verl/#__codelineno-0-36) }, [](https://microsoft.github.io/agent-lightning/stable/algorithm-zoo/verl/#__codelineno-0-37) "ref": { [](https://microsoft.github.io/agent-lightning/stable/algorithm-zoo/verl/#__codelineno-0-38) "log_prob_micro_batch_size_per_gpu": 8, [](https://microsoft.github.io/agent-lightning/stable/algorithm-zoo/verl/#__codelineno-0-39) "fsdp_config": {"param_offload": True}, [](https://microsoft.github.io/agent-lightning/stable/algorithm-zoo/verl/#__codelineno-0-40) }, [](https://microsoft.github.io/agent-lightning/stable/algorithm-zoo/verl/#__codelineno-0-41) "model": { [](https://microsoft.github.io/agent-lightning/stable/algorithm-zoo/verl/#__codelineno-0-42) "path": "Qwen/Qwen2.5-1.5B-Instruct", [](https://microsoft.github.io/agent-lightning/stable/algorithm-zoo/verl/#__codelineno-0-43) "use_remove_padding": True, [](https://microsoft.github.io/agent-lightning/stable/algorithm-zoo/verl/#__codelineno-0-44) "enable_gradient_checkpointing": True, [](https://microsoft.github.io/agent-lightning/stable/algorithm-zoo/verl/#__codelineno-0-45) }, [](https://microsoft.github.io/agent-lightning/stable/algorithm-zoo/verl/#__codelineno-0-46) }, [](https://microsoft.github.io/agent-lightning/stable/algorithm-zoo/verl/#__codelineno-0-47) "trainer": { [](https://microsoft.github.io/agent-lightning/stable/algorithm-zoo/verl/#__codelineno-0-48) "n_gpus_per_node": 1, [](https://microsoft.github.io/agent-lightning/stable/algorithm-zoo/verl/#__codelineno-0-49) "val_before_train": True, [](https://microsoft.github.io/agent-lightning/stable/algorithm-zoo/verl/#__codelineno-0-50) "critic_warmup": 0, [](https://microsoft.github.io/agent-lightning/stable/algorithm-zoo/verl/#__codelineno-0-51) "logger": ["console", "wandb"], [](https://microsoft.github.io/agent-lightning/stable/algorithm-zoo/verl/#__codelineno-0-52) "project_name": "AgentLightning", [](https://microsoft.github.io/agent-lightning/stable/algorithm-zoo/verl/#__codelineno-0-53) "experiment_name": "calc_x", [](https://microsoft.github.io/agent-lightning/stable/algorithm-zoo/verl/#__codelineno-0-54) "nnodes": 1, [](https://microsoft.github.io/agent-lightning/stable/algorithm-zoo/verl/#__codelineno-0-55) "save_freq": 64, [](https://microsoft.github.io/agent-lightning/stable/algorithm-zoo/verl/#__codelineno-0-56) "test_freq": 32, [](https://microsoft.github.io/agent-lightning/stable/algorithm-zoo/verl/#__codelineno-0-57) "total_epochs": 2, [](https://microsoft.github.io/agent-lightning/stable/algorithm-zoo/verl/#__codelineno-0-58) }, [](https://microsoft.github.io/agent-lightning/stable/algorithm-zoo/verl/#__codelineno-0-59) } [](https://microsoft.github.io/agent-lightning/stable/algorithm-zoo/verl/#__codelineno-0-60) ) [](https://microsoft.github.io/agent-lightning/stable/algorithm-zoo/verl/#__codelineno-0-61) trainer.fit(algorithm, train_dataset=my_train_dataset)` ##### `get_client()` [¶](https://microsoft.github.io/agent-lightning/stable/algorithm-zoo/verl/#agentlightning.algorithm.verl.VERL.get_client "Permanent link") Create a client bound to the VERL-managed Agent Lightning server. Deprecated Since v0.2. ##### `run(train_dataset=None, val_dataset=None)` [¶](https://microsoft.github.io/agent-lightning/stable/algorithm-zoo/verl/#agentlightning.algorithm.verl.VERL.run "Permanent link") Launch the VERL PPO entrypoint with the configured runtime context. Parameters: * **`train_dataset`** (`Optional[[Dataset](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.Dataset " agentlightning.Dataset (agentlightning.types.Dataset)") [Any]]`, default: `None` ) – Optional dataset forwarded to VERL for training. * **`val_dataset`** (`Optional[[Dataset](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.Dataset " agentlightning.Dataset (agentlightning.types.Dataset)") [Any]]`, default: `None` ) – Optional dataset forwarded to VERL for evaluation. Raises: * `ValueError` – If required dependencies such as the store, LLM proxy, or adapter have been garbage-collected when using the V1 execution mode. References - Implementation[¶](https://microsoft.github.io/agent-lightning/stable/algorithm-zoo/verl/#references-implementation "Permanent link") -------------------------------------------------------------------------------------------------------------------------------------------------- ### `agentlightning.verl` [¶](https://microsoft.github.io/agent-lightning/stable/algorithm-zoo/verl/#agentlightning.verl "Permanent link") This package contains a _hacky_ integration of VERL with Agent Lightning. #### `AgentLightningTrainer` [¶](https://microsoft.github.io/agent-lightning/stable/algorithm-zoo/verl/#agentlightning.verl.AgentLightningTrainer "Permanent link") Bases: `RayPPOTrainer` Specialized PPO trainer for agent-based reinforcement learning. This trainer is designed specifically for scenarios where the model interacts with external environments, tools, or APIs through an AgentLightningServer. It simplifies the training loop by removing the complex conditional logic present in the original RayPPOTrainer and focusing on the agent mode workflow. Key differences from RayPPOTrainer: 1. Uses AgentModeDaemon for server communication 2. Simplified data flow without pop/union operations 3. Direct batch processing through agent daemon 4. Streamlined validation using agent\_mode validation #### `AgentModeDaemon` [¶](https://microsoft.github.io/agent-lightning/stable/algorithm-zoo/verl/#agentlightning.verl.AgentModeDaemon "Permanent link") AgentModeDaemon using the AgentLightningServer SDK. This class manages the server lifecycle, task queueing, and results retrieval, while also running a proxy server for LLM requests. It maintains the original interface for compatibility with the RayPPOTrainer. ##### `clear_data_and_server()` [¶](https://microsoft.github.io/agent-lightning/stable/algorithm-zoo/verl/#agentlightning.verl.AgentModeDaemon.clear_data_and_server "Permanent link") Resets the internal state of the daemon for the next run. ##### `get_test_metrics()` [¶](https://microsoft.github.io/agent-lightning/stable/algorithm-zoo/verl/#agentlightning.verl.AgentModeDaemon.get_test_metrics "Permanent link") Calculates and returns metrics for a validation run. ##### `get_train_data_batch(max_prompt_length, max_response_length, device, global_steps)` [¶](https://microsoft.github.io/agent-lightning/stable/algorithm-zoo/verl/#agentlightning.verl.AgentModeDaemon.get_train_data_batch "Permanent link") Processes completed rollouts to generate a training data batch. This function reconstructs the logic from the original AgentModeDaemon, using data retrieved from the new server architecture. It handles padding, truncation, and tensor creation for the PPO training loop. ##### `run_until_all_finished(verbose=True)` [¶](https://microsoft.github.io/agent-lightning/stable/algorithm-zoo/verl/#agentlightning.verl.AgentModeDaemon.run_until_all_finished "Permanent link") Synchronously waits for all queued tasks to be completed and reported. ##### `set_up_data_and_server(data, server_addresses, is_train=True)` [¶](https://microsoft.github.io/agent-lightning/stable/algorithm-zoo/verl/#agentlightning.verl.AgentModeDaemon.set_up_data_and_server "Permanent link") Synchronous wrapper for setting up data and server resources. ##### `start()` [¶](https://microsoft.github.io/agent-lightning/stable/algorithm-zoo/verl/#agentlightning.verl.AgentModeDaemon.start "Permanent link") Starts the main AgentLightningServer and the proxy server. #### `get_left_padded_ids_and_attention_mask(ids, max_length, pad_token_id)` [¶](https://microsoft.github.io/agent-lightning/stable/algorithm-zoo/verl/#agentlightning.verl.get_left_padded_ids_and_attention_mask "Permanent link") Left-pad (or truncate) a sequence of token IDs to a fixed length, and build the corresponding attention mask. Parameters: * **`ids`** (`List[int]`) – the original list of token IDs. * **`max_length`** (`int`) – desired total length after padding/truncation. * **`pad_token_id`** (`int`) – ID to use for padding. Returns: * **`padded_ids`** ( `any` ) – list of length == max\_length. * **`attention_mask`** ( `any` ) – list of same length: 1 for non-pad tokens, 0 for pads. #### `get_right_padded_ids_and_attention_mask(ids, max_length, pad_token_id)` [¶](https://microsoft.github.io/agent-lightning/stable/algorithm-zoo/verl/#agentlightning.verl.get_right_padded_ids_and_attention_mask "Permanent link") Right-pad (or truncate) a sequence of token IDs to a fixed length, and build the corresponding attention mask. Parameters: * **`ids`** (`List[int]`) – the original list of token IDs. * **`max_length`** (`int`) – desired total length after padding/truncation. * **`pad_token_id`** (`int`) – ID to use for padding. Returns: * **`padded_ids`** ( `any` ) – list of length == max\_length. * **`attention_mask`** ( `any` ) – list of same length: 1 for non-pad tokens, 0 for pads. Back to top --- # Semantic Conventions - Agent-lightning [Skip to content](https://microsoft.github.io/agent-lightning/stable/reference/semconv/#semantic-conventions) ⚠️ You are viewing development documentation. For stable documentation, please visit [https://microsoft.github.io/agent-lightning/stable/](https://microsoft.github.io/agent-lightning/stable/) Semantic Conventions[¶](https://microsoft.github.io/agent-lightning/stable/reference/semconv/#semantic-conventions "Permanent link") ===================================================================================================================================== ### `agentlightning.semconv` [¶](https://microsoft.github.io/agent-lightning/stable/reference/semconv/#agentlightning.semconv "Permanent link") Semantic conventions for Agent-lightning spans. Conventions in this file are added on demand. We generally DO NOT add new semantic conventions unless it's absolutely needed for certain algorithms or scenarios. #### `AGL_ANNOTATION = 'agentlightning.annotation'` `module-attribute` [¶](https://microsoft.github.io/agent-lightning/stable/reference/semconv/#agentlightning.semconv.AGL_ANNOTATION "Permanent link") Agent-lightning's standard span name for annotations. Annotations are minimal span units for rewards, tags, and metadatas. They are used to "annotate" a specific event or a part of rollout. #### `AGL_EXCEPTION = 'agentlightning.exception'` `module-attribute` [¶](https://microsoft.github.io/agent-lightning/stable/reference/semconv/#agentlightning.semconv.AGL_EXCEPTION "Permanent link") Agent-lightning's standard span name for exceptions. Used by the exception emitter to record exception details. #### `AGL_MESSAGE = 'agentlightning.message'` `module-attribute` [¶](https://microsoft.github.io/agent-lightning/stable/reference/semconv/#agentlightning.semconv.AGL_MESSAGE "Permanent link") Agent-lightning's standard span name for messages and logs. #### `AGL_OBJECT = 'agentlightning.object'` `module-attribute` [¶](https://microsoft.github.io/agent-lightning/stable/reference/semconv/#agentlightning.semconv.AGL_OBJECT "Permanent link") Agent-lightning's standard span name for customized objects. #### `AGL_OPERATION = 'agentlightning.operation'` `module-attribute` [¶](https://microsoft.github.io/agent-lightning/stable/reference/semconv/#agentlightning.semconv.AGL_OPERATION "Permanent link") Agent-lightning's standard span name for functions. Wrap function or code-blocks as operations. #### `AGL_REWARD = 'agentlightning.reward'` `module-attribute` [¶](https://microsoft.github.io/agent-lightning/stable/reference/semconv/#agentlightning.semconv.AGL_REWARD "Permanent link") Agent-lightning's standard span name for reward operations. #### `AGL_VIRTUAL = 'agentlightning.virtual'` `module-attribute` [¶](https://microsoft.github.io/agent-lightning/stable/reference/semconv/#agentlightning.semconv.AGL_VIRTUAL "Permanent link") Agent-lightning's standard span name for virtual operations. Mostly used in adapter when needing to represent the root or intermediate operations. #### `LightningResourceAttributes` [¶](https://microsoft.github.io/agent-lightning/stable/reference/semconv/#agentlightning.semconv.LightningResourceAttributes "Permanent link") Bases: `Enum` Resource attribute names used in Agent-lightning spans. ##### `ATTEMPT_ID = 'agentlightning.attempt_id'` `class-attribute` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/stable/reference/semconv/#agentlightning.semconv.LightningResourceAttributes.ATTEMPT_ID "Permanent link") Resource name for attempt ID in Agent-lightning spans. ##### `ROLLOUT_ID = 'agentlightning.rollout_id'` `class-attribute` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/stable/reference/semconv/#agentlightning.semconv.LightningResourceAttributes.ROLLOUT_ID "Permanent link") Resource name for rollout ID in Agent-lightning spans. ##### `SPAN_SEQUENCE_ID = 'agentlightning.span_sequence_id'` `class-attribute` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/stable/reference/semconv/#agentlightning.semconv.LightningResourceAttributes.SPAN_SEQUENCE_ID "Permanent link") Resource name for span sequence ID in Agent-lightning spans. ##### `TRACER_NAME = 'agentlightning.tracer.name'` `class-attribute` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/stable/reference/semconv/#agentlightning.semconv.LightningResourceAttributes.TRACER_NAME "Permanent link") Which tracer is used to create this span. #### `LightningSpanAttributes` [¶](https://microsoft.github.io/agent-lightning/stable/reference/semconv/#agentlightning.semconv.LightningSpanAttributes "Permanent link") Bases: `Enum` Attribute names that commonly appear in Agent-lightning spans. Exception types can't be found here because they are defined in OpenTelemetry's official semantic conventions. ##### `LINK = 'agentlightning.link'` `class-attribute` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/stable/reference/semconv/#agentlightning.semconv.LightningSpanAttributes.LINK "Permanent link") Attribute name for linking the current span to another span or other objects like requests/responses. ##### `MESSAGE_BODY = 'agentlightning.message.body'` `class-attribute` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/stable/reference/semconv/#agentlightning.semconv.LightningSpanAttributes.MESSAGE_BODY "Permanent link") Attribute name for message text in message spans. ##### `OBJECT_JSON = 'agentlightning.object.json'` `class-attribute` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/stable/reference/semconv/#agentlightning.semconv.LightningSpanAttributes.OBJECT_JSON "Permanent link") Attribute name for object serialized value (JSON) in object spans. ##### `OBJECT_LITERAL = 'agentlightning.object.literal'` `class-attribute` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/stable/reference/semconv/#agentlightning.semconv.LightningSpanAttributes.OBJECT_LITERAL "Permanent link") Attribute name for object literal value in object spans (for str, int, bool, ...). ##### `OBJECT_TYPE = 'agentlightning.object.type'` `class-attribute` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/stable/reference/semconv/#agentlightning.semconv.LightningSpanAttributes.OBJECT_TYPE "Permanent link") Attribute name for object type (full qualified name) in object spans. I think builtin types like str, int, bool, list, dict are self-explanatory and should also be qualified to use here. ##### `OPERATION_INPUT = 'agentlightning.operation.input'` `class-attribute` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/stable/reference/semconv/#agentlightning.semconv.LightningSpanAttributes.OPERATION_INPUT "Permanent link") Attribute name for operation input in operation spans. ##### `OPERATION_NAME = 'agentlightning.operation.name'` `class-attribute` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/stable/reference/semconv/#agentlightning.semconv.LightningSpanAttributes.OPERATION_NAME "Permanent link") Attribute name for operation name in operation spans, normally the function name. ##### `OPERATION_OUTPUT = 'agentlightning.operation.output'` `class-attribute` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/stable/reference/semconv/#agentlightning.semconv.LightningSpanAttributes.OPERATION_OUTPUT "Permanent link") Attribute name for operation output in operation spans. ##### `REWARD = 'agentlightning.reward'` `class-attribute` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/stable/reference/semconv/#agentlightning.semconv.LightningSpanAttributes.REWARD "Permanent link") Attribute prefix for rewards-related data in reward spans. It should be used as a prefix. For example, "agentlightning.reward.0.value" can be used to track a specific metric. See [RewardAttributes](https://microsoft.github.io/agent-lightning/stable/reference/semconv/#agentlightning.semconv.RewardAttributes " RewardAttributes") . ##### `TAG = 'agentlightning.tag'` `class-attribute` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/stable/reference/semconv/#agentlightning.semconv.LightningSpanAttributes.TAG "Permanent link") Attribute name for tagging spans with customized strings. #### `LinkAttributes` [¶](https://microsoft.github.io/agent-lightning/stable/reference/semconv/#agentlightning.semconv.LinkAttributes "Permanent link") Bases: `Enum` Standard link types used in Agent-lightning spans. The link is more powerful than [OpenTelemetry link](https://opentelemetry.io/docs/specs/otel/trace/api/#link) in that it supports linking to a queryset of spans. It can even link to span object that hasn't been emitted yet. ##### `KEY_MATCH = 'key_match'` `class-attribute` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/stable/reference/semconv/#agentlightning.semconv.LinkAttributes.KEY_MATCH "Permanent link") Linking to spans with matching attribute keys. `trace_id` and `span_id` are reserved and will be used to link to specific spans directly. For example, it can be `gen_ai.response.id` if intended to be link to a chat completion response span. Or it can be `span_id` to link to a specific span by its ID. ##### `VALUE_MATCH = 'value_match'` `class-attribute` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/stable/reference/semconv/#agentlightning.semconv.LinkAttributes.VALUE_MATCH "Permanent link") Linking to spans with corresponding attribute values on those keys. #### `LinkPydanticModel` [¶](https://microsoft.github.io/agent-lightning/stable/reference/semconv/#agentlightning.semconv.LinkPydanticModel "Permanent link") Bases: `BaseModel` A stricter implementation of LinkAttributes used in otel helpers. ##### `key_match` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/stable/reference/semconv/#agentlightning.semconv.LinkPydanticModel.key_match "Permanent link") The attribute key to match on the target spans. ##### `value_match` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/stable/reference/semconv/#agentlightning.semconv.LinkPydanticModel.value_match "Permanent link") The attribute value to match on the target spans. #### `RewardAttributes` [¶](https://microsoft.github.io/agent-lightning/stable/reference/semconv/#agentlightning.semconv.RewardAttributes "Permanent link") Bases: `Enum` Multi-dimensional reward attributes will look like: `[](https://microsoft.github.io/agent-lightning/stable/reference/semconv/#__codelineno-0-1) {"agentlightning.reward.0.name": "efficiency", "agentlightning.reward.0.value": 0.75}` The first reward in the reward list will automatically be the primary reward. If the reward list has greater than 1, it shall be a multi-dimensional case. ##### `REWARD_NAME = 'name'` `class-attribute` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/stable/reference/semconv/#agentlightning.semconv.RewardAttributes.REWARD_NAME "Permanent link") Key for each dimension in multi-dimensional reward spans. ##### `REWARD_VALUE = 'value'` `class-attribute` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/stable/reference/semconv/#agentlightning.semconv.RewardAttributes.REWARD_VALUE "Permanent link") Value for each dimension in multi-dimensional reward spans. #### `RewardPydanticModel` [¶](https://microsoft.github.io/agent-lightning/stable/reference/semconv/#agentlightning.semconv.RewardPydanticModel "Permanent link") Bases: `BaseModel` A stricter implementation of RewardAttributes used in otel helpers. ##### `name` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/stable/reference/semconv/#agentlightning.semconv.RewardPydanticModel.name "Permanent link") Name of the reward dimension. ##### `value` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/stable/reference/semconv/#agentlightning.semconv.RewardPydanticModel.value "Permanent link") Value of the reward dimension. Back to top --- # Parallelize - Agent-lightning [Skip to content](https://microsoft.github.io/agent-lightning/stable/tutorials/parallelize/#scaling-out-agent-lightning) ⚠️ You are viewing development documentation. For stable documentation, please visit [https://microsoft.github.io/agent-lightning/stable/](https://microsoft.github.io/agent-lightning/stable/) Scaling out Agent-lightning[¶](https://microsoft.github.io/agent-lightning/stable/tutorials/parallelize/#scaling-out-agent-lightning "Permanent link") ======================================================================================================================================================= Agent-lightning splits training into an **algorithm bundle** and a **runner bundle** that exchange work through the [`LightningStore`](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.LightningStore " agentlightning.LightningStore") . This tutorial shows how to increase rollout throughput, place bundles across processes or machines, and keep the algorithm side scalable with external frameworks. Parallelizing Rollouts with [`Trainer`](https://microsoft.github.io/agent-lightning/stable/reference/trainer/#agentlightning.Trainer " agentlightning.Trainer") [¶](https://microsoft.github.io/agent-lightning/stable/tutorials/parallelize/#parallelizing-rollouts-with-trainer "Permanent link") --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Before we dive into the details of the bundles and execution strategies, let's first revisit how to parallelize rollouts with [`Trainer`](https://microsoft.github.io/agent-lightning/stable/reference/trainer/#agentlightning.Trainer " agentlightning.Trainer") . [`Trainer`](https://microsoft.github.io/agent-lightning/stable/reference/trainer/#agentlightning.Trainer " agentlightning.Trainer") is the quickest way to dial up parallelism. Even when `n_runners = 1`, calling [`Trainer.fit`](https://microsoft.github.io/agent-lightning/stable/reference/trainer/#agentlightning.Trainer.fit " fit(agent, train_dataset=None, *, val_dataset=None)") runs the algorithm and runners in parallel. The algorithm enqueues rollouts; runners dequeue them and execute your [`LitAgent`](https://microsoft.github.io/agent-lightning/stable/reference/agent/#agentlightning.LitAgent " agentlightning.LitAgent") , and the algorithm collects spans via its [`Adapter`](https://microsoft.github.io/agent-lightning/stable/reference/algorithm/#agentlightning.Adapter " agentlightning.Adapter") before scheduling the next batch. Note One of the most important features of [`Trainer`](https://microsoft.github.io/agent-lightning/stable/reference/trainer/#agentlightning.Trainer " agentlightning.Trainer") is the ability to abort things gracefully. For example, if you press `Ctrl+C` in the terminal, the algorithm will abort and the runners will stop executing. If the algorithm crashes, the runners will also stop executing. Increase throughput by setting `n_runners` when constructing the trainer. The following example comes from [train\_calc\_agent.py](https://github.com/microsoft/agent-lightning/tree/3b5d733861cf313fc09821a23240bbdf3cb2ee5b/examples/calc_x/train_calc_agent.py) . Since backend LLMs usually use techniques like [continuous batching](https://docs.vllm.ai/en/latest/) to increase throughput, you do not have to worry about overwhelming the backend with too many requests. `[](https://microsoft.github.io/agent-lightning/stable/tutorials/parallelize/#__codelineno-0-1) import agentlightning as agl [](https://microsoft.github.io/agent-lightning/stable/tutorials/parallelize/#__codelineno-0-2) from datasets import Dataset as HFDataset [](https://microsoft.github.io/agent-lightning/stable/tutorials/parallelize/#__codelineno-0-3) from calc_agent import calc_agent [](https://microsoft.github.io/agent-lightning/stable/tutorials/parallelize/#__codelineno-0-4) [](https://microsoft.github.io/agent-lightning/stable/tutorials/parallelize/#__codelineno-0-5) train_dataset = HFDataset.from_parquet("data/train.parquet").to_list() [](https://microsoft.github.io/agent-lightning/stable/tutorials/parallelize/#__codelineno-0-6) val_dataset = HFDataset.from_parquet("data/test.parquet").to_list() [](https://microsoft.github.io/agent-lightning/stable/tutorials/parallelize/#__codelineno-0-7) [](https://microsoft.github.io/agent-lightning/stable/tutorials/parallelize/#__codelineno-0-8) algorithm = agl.VERL(verl_config) [](https://microsoft.github.io/agent-lightning/stable/tutorials/parallelize/#__codelineno-0-9) [](https://microsoft.github.io/agent-lightning/stable/tutorials/parallelize/#__codelineno-0-10) trainer = agl.Trainer( [](https://microsoft.github.io/agent-lightning/stable/tutorials/parallelize/#__codelineno-0-11) algorithm=algorithm, [](https://microsoft.github.io/agent-lightning/stable/tutorials/parallelize/#__codelineno-0-12) n_runners=8, # launch eight rollout workers [](https://microsoft.github.io/agent-lightning/stable/tutorials/parallelize/#__codelineno-0-13) tracer=agl.OtelTracer(), [](https://microsoft.github.io/agent-lightning/stable/tutorials/parallelize/#__codelineno-0-14) adapter=agl.LlmProxyTraceToTriplet(), [](https://microsoft.github.io/agent-lightning/stable/tutorials/parallelize/#__codelineno-0-15) ) [](https://microsoft.github.io/agent-lightning/stable/tutorials/parallelize/#__codelineno-0-16) [](https://microsoft.github.io/agent-lightning/stable/tutorials/parallelize/#__codelineno-0-17) trainer.fit(calc_agent, train_dataset=train_dataset, val_dataset=val_dataset)` In [`Trainer`](https://microsoft.github.io/agent-lightning/stable/reference/trainer/#agentlightning.Trainer " agentlightning.Trainer") , there are multiple other initialization parameters that you can use to customize the training process. For example, you can use `max_rollouts` to keep smoke tests short. Pass a concrete [`LightningStore`](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.LightningStore " agentlightning.LightningStore") instance when you need persistence or want to share the queue across multiple scripts. Tip Before scaling out, run [`Trainer.dev()`](https://microsoft.github.io/agent-lightning/stable/reference/trainer/#agentlightning.Trainer.dev " dev(agent, train_dataset=None, *, val_dataset=None)") with `n_runners=1` to verify the rollout logic and spans without burning GPU hours. Bundles and Execution Strategies[¶](https://microsoft.github.io/agent-lightning/stable/tutorials/parallelize/#bundles-and-execution-strategies "Permanent link") ----------------------------------------------------------------------------------------------------------------------------------------------------------------- When [`Trainer`](https://microsoft.github.io/agent-lightning/stable/reference/trainer/#agentlightning.Trainer " agentlightning.Trainer") starts, it packages its configuration into two callable **bundles**: ![Illustration of bundles and execution strategies](https://microsoft.github.io/agent-lightning/stable/assets/execution-bundles.svg) The **algorithm bundle** wraps your [`Algorithm`](https://microsoft.github.io/agent-lightning/stable/reference/algorithm/#agentlightning.Algorithm " agentlightning.Algorithm") , adapter, and any LLM proxy into a single callable that can be aborted via a signal event. `[](https://microsoft.github.io/agent-lightning/stable/tutorials/parallelize/#__codelineno-1-1) async def algorithm_bundle(store: LightningStore, event: ExecutionEvent) -> None: [](https://microsoft.github.io/agent-lightning/stable/tutorials/parallelize/#__codelineno-1-2) ...` The **runner bundle** wraps the [`Runner`](https://microsoft.github.io/agent-lightning/stable/reference/runner/#agentlightning.Runner " agentlightning.Runner") , tracer, hooks, and agent into a single callable that can be aborted via a signal event. Unlike the algorithm bundle, the runner bundle is expected to be replicated. `[](https://microsoft.github.io/agent-lightning/stable/tutorials/parallelize/#__codelineno-2-1) async def runner_bundle(store: LightningStore, worker_id: int, event: ExecutionEvent) -> None: [](https://microsoft.github.io/agent-lightning/stable/tutorials/parallelize/#__codelineno-2-2) ...` An **execution strategy** then decides where those bundles are placed (threads vs processes vs multiple machines), how many runner replicas to launch, and how lifecycle events such as shutdown are coordinated. By default, the trainer builds an [`InMemoryLightningStore`](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.InMemoryLightningStore " agentlightning.InMemoryLightningStore") if you do not provide one. Because that store has no locking or cross-process transport, the execution strategy is the component that wraps it in thread-safe or HTTP-safe facades ([`LightningStoreThreaded`](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.LightningStoreThreaded " agentlightning.LightningStoreThreaded") , [`LightningStoreServer`](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.LightningStoreServer " agentlightning.LightningStoreServer") ) before handing it to bundles. For a deeper look at these facades, see [Understanding the Store](https://microsoft.github.io/agent-lightning/stable/deep-dive/store/) and [Birds' Eye View](https://microsoft.github.io/agent-lightning/stable/deep-dive/birds-eye-view/) . Agent-lightning provides two built-in execution strategies: [`SharedMemoryExecutionStrategy`](https://microsoft.github.io/agent-lightning/stable/reference/trainer/#agentlightning.SharedMemoryExecutionStrategy " agentlightning.SharedMemoryExecutionStrategy") and [`ClientServerExecutionStrategy`](https://microsoft.github.io/agent-lightning/stable/reference/trainer/#agentlightning.ClientServerExecutionStrategy " agentlightning.ClientServerExecutionStrategy") . You can pass a string alias, a configuration dictionary, or a pre-built strategy instance: ``[](https://microsoft.github.io/agent-lightning/stable/tutorials/parallelize/#__codelineno-3-1) import agentlightning as agl [](https://microsoft.github.io/agent-lightning/stable/tutorials/parallelize/#__codelineno-3-2) [](https://microsoft.github.io/agent-lightning/stable/tutorials/parallelize/#__codelineno-3-3) algorithm = agl.Baseline() [](https://microsoft.github.io/agent-lightning/stable/tutorials/parallelize/#__codelineno-3-4) [](https://microsoft.github.io/agent-lightning/stable/tutorials/parallelize/#__codelineno-3-5) # Short alias for the shared-memory strategy. [](https://microsoft.github.io/agent-lightning/stable/tutorials/parallelize/#__codelineno-3-6) # Because the runner lives on the main thread in this mode, [](https://microsoft.github.io/agent-lightning/stable/tutorials/parallelize/#__codelineno-3-7) # n_runners must be 1 unless you move the algorithm to the main thread. [](https://microsoft.github.io/agent-lightning/stable/tutorials/parallelize/#__codelineno-3-8) trainer = agl.Trainer(algorithm=algorithm, n_runners=1, strategy="shm") [](https://microsoft.github.io/agent-lightning/stable/tutorials/parallelize/#__codelineno-3-9) [](https://microsoft.github.io/agent-lightning/stable/tutorials/parallelize/#__codelineno-3-10) # Dict with overrides; keep the algorithm on the main thread so multiple runner threads can spawn. [](https://microsoft.github.io/agent-lightning/stable/tutorials/parallelize/#__codelineno-3-11) # Specifying `n_runners` inside strategy is equivalent to passing `n_runners` to the trainer. [](https://microsoft.github.io/agent-lightning/stable/tutorials/parallelize/#__codelineno-3-12) trainer = agl.Trainer( [](https://microsoft.github.io/agent-lightning/stable/tutorials/parallelize/#__codelineno-3-13) algorithm=algorithm, [](https://microsoft.github.io/agent-lightning/stable/tutorials/parallelize/#__codelineno-3-14) strategy={ [](https://microsoft.github.io/agent-lightning/stable/tutorials/parallelize/#__codelineno-3-15) "type": "shm", [](https://microsoft.github.io/agent-lightning/stable/tutorials/parallelize/#__codelineno-3-16) "n_runners": 8, [](https://microsoft.github.io/agent-lightning/stable/tutorials/parallelize/#__codelineno-3-17) "main_thread": "algorithm", [](https://microsoft.github.io/agent-lightning/stable/tutorials/parallelize/#__codelineno-3-18) }, [](https://microsoft.github.io/agent-lightning/stable/tutorials/parallelize/#__codelineno-3-19) ) [](https://microsoft.github.io/agent-lightning/stable/tutorials/parallelize/#__codelineno-3-20) [](https://microsoft.github.io/agent-lightning/stable/tutorials/parallelize/#__codelineno-3-21) # Pass an existing strategy instance – Trainer respects the strategy's own `n_runners`. [](https://microsoft.github.io/agent-lightning/stable/tutorials/parallelize/#__codelineno-3-22) strategy = agl.SharedMemoryExecutionStrategy(main_thread="algorithm", n_runners=4) [](https://microsoft.github.io/agent-lightning/stable/tutorials/parallelize/#__codelineno-3-23) trainer = agl.Trainer(algorithm=algorithm, strategy=strategy)`` If you omit the strategy, the trainer defaults to `ClientServerExecutionStrategy(n_runners=trainer.n_runners)`. You can still re-specify the client-server strategy through aliases or configuration to tweak ports and other settings: `[](https://microsoft.github.io/agent-lightning/stable/tutorials/parallelize/#__codelineno-4-1) trainer = agl.Trainer( [](https://microsoft.github.io/agent-lightning/stable/tutorials/parallelize/#__codelineno-4-2) algorithm=algorithm, [](https://microsoft.github.io/agent-lightning/stable/tutorials/parallelize/#__codelineno-4-3) n_runners=8, [](https://microsoft.github.io/agent-lightning/stable/tutorials/parallelize/#__codelineno-4-4) strategy={"type": "cs", "server_port": 9999}, [](https://microsoft.github.io/agent-lightning/stable/tutorials/parallelize/#__codelineno-4-5) )` Environment variables give you another layer of control. For example: `[](https://microsoft.github.io/agent-lightning/stable/tutorials/parallelize/#__codelineno-5-1) import os [](https://microsoft.github.io/agent-lightning/stable/tutorials/parallelize/#__codelineno-5-2) [](https://microsoft.github.io/agent-lightning/stable/tutorials/parallelize/#__codelineno-5-3) os.environ["AGL_SERVER_PORT"] = "10000" [](https://microsoft.github.io/agent-lightning/stable/tutorials/parallelize/#__codelineno-5-4) os.environ["AGL_CURRENT_ROLE"] = "algorithm" [](https://microsoft.github.io/agent-lightning/stable/tutorials/parallelize/#__codelineno-5-5) os.environ["AGL_MANAGED_STORE"] = "0" [](https://microsoft.github.io/agent-lightning/stable/tutorials/parallelize/#__codelineno-5-6) [](https://microsoft.github.io/agent-lightning/stable/tutorials/parallelize/#__codelineno-5-7) trainer = agl.Trainer(algorithm=algorithm, n_runners=8, strategy="cs")` The resulting [`ClientServerExecutionStrategy`](https://microsoft.github.io/agent-lightning/stable/reference/trainer/#agentlightning.ClientServerExecutionStrategy " agentlightning.ClientServerExecutionStrategy") picks up the port, role, and managed-store flag from the environment. Tip The same configuration patterns apply to other trainer components. For example, `[](https://microsoft.github.io/agent-lightning/stable/tutorials/parallelize/#__codelineno-6-1) trainer = agl.Trainer(algorithm=algorithm, tracer=agl.OtelTracer())` wires in a custom tracer, while `[](https://microsoft.github.io/agent-lightning/stable/tutorials/parallelize/#__codelineno-7-1) trainer = agl.Trainer(algorithm=algorithm, adapter="agentlightning.adapter.TraceToMessages")` swaps in a different adapter. Passing a dict lets you tweak the init parameters of defaults without naming the class explicitly: `[](https://microsoft.github.io/agent-lightning/stable/tutorials/parallelize/#__codelineno-8-1) trainer = agl.Trainer( [](https://microsoft.github.io/agent-lightning/stable/tutorials/parallelize/#__codelineno-8-2) algorithm=algorithm, [](https://microsoft.github.io/agent-lightning/stable/tutorials/parallelize/#__codelineno-8-3) adapter={"agent_match": "plan_agent", "repair_hierarchy": False}, [](https://microsoft.github.io/agent-lightning/stable/tutorials/parallelize/#__codelineno-8-4) )` The next sections walk through the two built-in strategies and how they affect placement and store access. Client-server Architecture[¶](https://microsoft.github.io/agent-lightning/stable/tutorials/parallelize/#client-server-architecture "Permanent link") ----------------------------------------------------------------------------------------------------------------------------------------------------- The default [`ClientServerExecutionStrategy`](https://microsoft.github.io/agent-lightning/stable/reference/trainer/#agentlightning.ClientServerExecutionStrategy " agentlightning.ClientServerExecutionStrategy") starts a [`LightningStoreServer`](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.LightningStoreServer " agentlightning.LightningStoreServer") alongside the algorithm and spawns runner processes that talk to it through [`LightningStoreClient`](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.LightningStoreClient " agentlightning.LightningStoreClient") . All runners share the HTTP endpoint, so the queue and spans stay consistent across processes or machines. If you simply instantiate [`Trainer`](https://microsoft.github.io/agent-lightning/stable/reference/trainer/#agentlightning.Trainer " agentlightning.Trainer") (as above), it will send the algorithm bundle and runner bundle to [`ClientServerExecutionStrategy`](https://microsoft.github.io/agent-lightning/stable/reference/trainer/#agentlightning.ClientServerExecutionStrategy " agentlightning.ClientServerExecutionStrategy") , which will then: 1. Launch \\(N+1\\) processes: \\(N\\) runner processes and 1 algorithm process (one of them could live in the main process). 2. The algorithm process will take the store received from [`Trainer`](https://microsoft.github.io/agent-lightning/stable/reference/trainer/#agentlightning.Trainer " agentlightning.Trainer") , wrap it in a [`LightningStoreServer`](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.LightningStoreServer " agentlightning.LightningStoreServer") , and start serving it over HTTP. 3. The runner processes discard the store and create a new store, which is a client that connects to the algorithm process through [`LightningStoreClient`](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.LightningStoreClient " agentlightning.LightningStoreClient") , and start executing the runner bundle. 4. The strategy automatically escalates shutdown (cooperative stop → `SIGINT` → `terminate()` → `kill()`) so long-running runners do not linger. You can override server placement or ports, and whether to automatically wrap the store, through constructor arguments or environment variables: `[](https://microsoft.github.io/agent-lightning/stable/tutorials/parallelize/#__codelineno-9-1) trainer = agl.Trainer( [](https://microsoft.github.io/agent-lightning/stable/tutorials/parallelize/#__codelineno-9-2) algorithm=algorithm, [](https://microsoft.github.io/agent-lightning/stable/tutorials/parallelize/#__codelineno-9-3) n_runners=1, [](https://microsoft.github.io/agent-lightning/stable/tutorials/parallelize/#__codelineno-9-4) strategy={ [](https://microsoft.github.io/agent-lightning/stable/tutorials/parallelize/#__codelineno-9-5) "type": "cs", [](https://microsoft.github.io/agent-lightning/stable/tutorials/parallelize/#__codelineno-9-6) "server_host": "0.0.0.0", [](https://microsoft.github.io/agent-lightning/stable/tutorials/parallelize/#__codelineno-9-7) "server_port": 9999, [](https://microsoft.github.io/agent-lightning/stable/tutorials/parallelize/#__codelineno-9-8) "main_process": "runner", [](https://microsoft.github.io/agent-lightning/stable/tutorials/parallelize/#__codelineno-9-9) }, [](https://microsoft.github.io/agent-lightning/stable/tutorials/parallelize/#__codelineno-9-10) )` Set `AGL_SERVER_HOST` and `AGL_SERVER_PORT` if you prefer environment-based configuration. You can also use `AGL_MANAGED_STORE` if you do not want the execution strategy to wrap the store for you. An example is shown in [Debugging with External Store](https://microsoft.github.io/agent-lightning/stable/tutorials/debug/#debug-with-external-store "Debug the Algorithm-Runner Boundary") . Algorithms sometimes require heterogeneous computation resources, such as GPU accelerators, while runners sometimes require a specific environment to run because many agent frameworks are fragile in their dependencies. A role-based launch pattern helps you place the algorithm on a dedicated machine with more GPU memory, while runners can live on another machine with more flexible dependencies. This is possible via `AGL_CURRENT_ROLE="algorithm"` or `AGL_CURRENT_ROLE="runner"` environment variables. When running on different machines, you also need to set `AGL_SERVER_HOST` and `AGL_SERVER_PORT` to the IP address and port of the algorithm machine. You might recognize that this convention is very similar to `MASTER_ADDR` and `MASTER_PORT` in [PyTorch distributed training](https://docs.pytorch.org/docs/stable/notes/ddp.html) . ### Launching Algorithm and Runner Roles on Separate Machines[¶](https://microsoft.github.io/agent-lightning/stable/tutorials/parallelize/#launching-algorithm-and-runner-roles-on-separate-machines "Permanent link") When you want to stretch the algorithm onto a GPU-rich machine and keep rollout workers close to the data source (or on machines with a more permissive dependency stack), launch the same training script in different terminals with role-specific environment variables. The client–server strategy will route each process to the right side of the queue as long as they share the same `AGL_SERVER_HOST`/`AGL_SERVER_PORT` pair. **1\. Pick an address and port for the store.** Decide which machine will host the algorithm. Choose a TCP port that can be reached by the runner machines (for example, open it in your firewall configuration). In this example we will use `10.0.0.4:4747`. **2\. Start the algorithm process.** On the machine that should run the algorithm, expose the store by binding to all network interfaces and mark the role as `algorithm`. `[](https://microsoft.github.io/agent-lightning/stable/tutorials/parallelize/#__codelineno-10-1) export AGL_SERVER_HOST=0.0.0.0 [](https://microsoft.github.io/agent-lightning/stable/tutorials/parallelize/#__codelineno-10-2) export AGL_SERVER_PORT=4747 [](https://microsoft.github.io/agent-lightning/stable/tutorials/parallelize/#__codelineno-10-3) export AGL_CURRENT_ROLE=algorithm [](https://microsoft.github.io/agent-lightning/stable/tutorials/parallelize/#__codelineno-10-4) [](https://microsoft.github.io/agent-lightning/stable/tutorials/parallelize/#__codelineno-10-5) python train_calc_agent.py` Leaving `AGL_MANAGED_STORE` unset (or setting it to `1`) lets the strategy create the [`LightningStoreServer`](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.LightningStoreServer " agentlightning.LightningStoreServer") for you. Otherwise, you can use the method in the previous section to create a store on your own. **3\. Start rollout workers on remote machines.** Every runner machine should point to the algorithm host and declare itself as the `runner` role. You can start multiple processes per machine or repeat the command on additional hosts. `[](https://microsoft.github.io/agent-lightning/stable/tutorials/parallelize/#__codelineno-11-1) export AGL_SERVER_HOST=10.0.0.4 [](https://microsoft.github.io/agent-lightning/stable/tutorials/parallelize/#__codelineno-11-2) export AGL_SERVER_PORT=4747 [](https://microsoft.github.io/agent-lightning/stable/tutorials/parallelize/#__codelineno-11-3) export AGL_CURRENT_ROLE=runner [](https://microsoft.github.io/agent-lightning/stable/tutorials/parallelize/#__codelineno-11-4) python train_calc_agent.py --n-runners 4` The runner process automatically connects via [`LightningStoreClient`](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.LightningStoreClient " agentlightning.LightningStoreClient") . Adjust `--n-runners` to spawn the desired number of worker processes on that machine. **4\. Scale out as needed.** Repeat step 3 on as many machines as you need. When you are done, stop the algorithm process. However, since the runners are on different machines, the strategy WILL NOT send a cooperative stop signal to the connected runners. So you need to kill the runners on your own. This role-based launch mirrors what [`Trainer.fit`](https://microsoft.github.io/agent-lightning/stable/reference/trainer/#agentlightning.Trainer.fit " fit(agent, train_dataset=None, *, val_dataset=None)") does inside a single machine while letting you spread work across a fleet. Because every process shares the same training script, you keep a single source of truth for dataset loading, adapters, and tracers, but you can tune compute resources independently for the algorithm and rollout workers. ### Shared-memory Strategy[¶](https://microsoft.github.io/agent-lightning/stable/tutorials/parallelize/#shared-memory-strategy "Permanent link") [`SharedMemoryExecutionStrategy`](https://microsoft.github.io/agent-lightning/stable/reference/trainer/#agentlightning.SharedMemoryExecutionStrategy " agentlightning.SharedMemoryExecutionStrategy") keeps everything inside one process. The runner runs on the main thread (by default) while the algorithm lives on a Python thread guarded by [`LightningStoreThreaded`](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.LightningStoreThreaded " agentlightning.LightningStoreThreaded") . Use it when you want easier debugging with shared breakpoints and no serialization overhead, or minimal startup time for unit tests. It's not a good choice for many algorithms that require heavy model training because [`LightningStoreThreaded`](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.LightningStoreThreaded " agentlightning.LightningStoreThreaded") does not work for multiprocessing. Using it with multiprocessing algorithms will lead to undefined behavior. Sample configuration: `[](https://microsoft.github.io/agent-lightning/stable/tutorials/parallelize/#__codelineno-12-1) trainer = agl.Trainer( [](https://microsoft.github.io/agent-lightning/stable/tutorials/parallelize/#__codelineno-12-2) algorithm=algorithm, [](https://microsoft.github.io/agent-lightning/stable/tutorials/parallelize/#__codelineno-12-3) strategy="shm", [](https://microsoft.github.io/agent-lightning/stable/tutorials/parallelize/#__codelineno-12-4) )` You can further customize the init parameters of [`SharedMemoryExecutionStrategy`](https://microsoft.github.io/agent-lightning/stable/reference/trainer/#agentlightning.SharedMemoryExecutionStrategy " agentlightning.SharedMemoryExecutionStrategy") . With `main_thread="runner"`, the runner occupies the main thread and `n_runners` must be `1`. The strategy respects `AGL_MANAGED_STORE`; set it to `0` to opt out of the `LightningStoreThreaded` wrapper. Parallelizing Algorithms[¶](https://microsoft.github.io/agent-lightning/stable/tutorials/parallelize/#parallelizing-algorithms "Permanent link") ------------------------------------------------------------------------------------------------------------------------------------------------- Runner parallelism scales rollout throughput, but the algorithm loop remains a single-process loop inside the execution strategy. We understand that many algorithms have parallelization built in, but that's outside the parallelization scope of Agent-lightning. Agent-lightning strives to make algorithms’ own parallelization work well under our execution strategies. The biggest challenge turns out to come from the store. For example, [`VERL`](https://microsoft.github.io/agent-lightning/stable/algorithm-zoo/verl/#agentlightning.algorithm.verl.VERL " VERL") uses [Ray](https://www.ray.io/) and launches [FSDP](https://docs.pytorch.org/tutorials/intermediate/FSDP_tutorial.html) and [vLLM](https://vllm.ai/) components internally. [`ClientServerExecutionStrategy`](https://microsoft.github.io/agent-lightning/stable/reference/trainer/#agentlightning.ClientServerExecutionStrategy " agentlightning.ClientServerExecutionStrategy") has to make sure that the server is not simultaneously serving in multiple processes or Ray workers, and that there is only one single authoritative source of truth for all subprocesses to connect to. Subprocesses connect to the store via a small [`LightningStoreClient`](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.LightningStoreClient " agentlightning.LightningStoreClient") bundled within [`LightningStoreServer`](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.LightningStoreServer " agentlightning.LightningStoreServer") . Note The [birds' eye view](https://microsoft.github.io/agent-lightning/stable/deep-dive/birds-eye-view/#birds-eye-view-client-server-strategy "Client-server Strategy") illustrates how adapters, proxies, and stores interact when the algorithm spawns additional workers. Use that diagram as a checklist when introducing new distributed components. Parallelizing [`LightningStore`](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.LightningStore " agentlightning.LightningStore") [¶](https://microsoft.github.io/agent-lightning/stable/tutorials/parallelize/#parallelizing-lightningstore "Permanent link") ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- By default, Agent-lightning persists rollouts and spans in an in-memory store. [`Trainer.fit`](https://microsoft.github.io/agent-lightning/stable/reference/trainer/#agentlightning.Trainer.fit " fit(agent, train_dataset=None, *, val_dataset=None)") spins it up automatically, or you can launch it yourself via the [`agl store` command](https://microsoft.github.io/agent-lightning/stable/reference/cli/) . [`InMemoryLightningStore`](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.InMemoryLightningStore " agentlightning.InMemoryLightningStore") keeps all state inside the current process, which makes local iteration fast but introduces two production constraints: 1. Spans are evicted once the process crosses its memory cap, so long runs risk data loss unless the host has abundant RAM. 2. Although the store is well optimized via asynchronous programming, the store lives in a single process and remains bound by the GIL, preventing it from saturating multi-core machines. General note for all server-client stores If your algorithm and runners communicate through HTTP protocol (which should be the default for 99% of the cases), you need to ensure the file limit is sufficiently large to avoid the "Too many open files" error. You can set the file limit by running the following command: `[](https://microsoft.github.io/agent-lightning/stable/tutorials/parallelize/#__codelineno-13-1) ulimit -n 100000` For resilient runs, switch to a persistent backend such as [`MongoLightningStore`](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.store.mongo.MongoLightningStore " agentlightning.store.mongo.MongoLightningStore") , which writes data to MongoDB instead of local RAM. Agent-lightning relies on [pymongo](https://pymongo.readthedocs.io/en/stable/) to interact with MongoDB, which can be installed via: `[](https://microsoft.github.io/agent-lightning/stable/tutorials/parallelize/#__codelineno-14-1) pip install agentlightning[mongo]` To use the MongoDB store, you need to pass the MongoDB URI to the store constructor. The URI should be in the format of `mongodb://:/?replicaSet=`. `[](https://microsoft.github.io/agent-lightning/stable/tutorials/parallelize/#__codelineno-15-1) from agentlightning.store.mongo import MongoLightningStore [](https://microsoft.github.io/agent-lightning/stable/tutorials/parallelize/#__codelineno-15-2) [](https://microsoft.github.io/agent-lightning/stable/tutorials/parallelize/#__codelineno-15-3) trainer = agl.Trainer( [](https://microsoft.github.io/agent-lightning/stable/tutorials/parallelize/#__codelineno-15-4) algorithm=algorithm, [](https://microsoft.github.io/agent-lightning/stable/tutorials/parallelize/#__codelineno-15-5) store=MongoLightningStore(mongo_uri="mongodb://localhost:27017/?replicaSet=rs0"), [](https://microsoft.github.io/agent-lightning/stable/tutorials/parallelize/#__codelineno-15-6) )` Setting up MongoDB MongoDB is a popular document-oriented database. Before running Agent-lightning with [`MongoLightningStore`](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.store.mongo.MongoLightningStore " agentlightning.store.mongo.MongoLightningStore") , make sure that you've already had a MongoDB instance running. Setting up can be conveniently done via Docker Compose via [compose.mongo.yml](https://github.com/microsoft/agent-lightning/tree/3b5d733861cf313fc09821a23240bbdf3cb2ee5b/docker/compose.mongo.yml) . Unless targeting serious production use, we recommend creating the data folders and setting them to `777` permission to avoid permission issues. `[](https://microsoft.github.io/agent-lightning/stable/tutorials/parallelize/#__codelineno-16-1) mkdir -p data/mongo-host [](https://microsoft.github.io/agent-lightning/stable/tutorials/parallelize/#__codelineno-16-2) chmod 777 data/mongo-host [](https://microsoft.github.io/agent-lightning/stable/tutorials/parallelize/#__codelineno-16-3) docker compose -f compose.mongo.yml up -d` Alternatively, you can also install MongoDB manually following the [official documentation](https://www.mongodb.com/docs/manual/installation/) . If you installed MongoDB manually, an important note is that you need to ensure that the MongoDB instance has enabled replica set feature, since Agent-lightning uses the transactional operations internally. The simplest approach is to use the following script (executed in the MongoDB shell) to initialize the replica set: `[](https://microsoft.github.io/agent-lightning/stable/tutorials/parallelize/#__codelineno-17-1) rs.initiate({ [](https://microsoft.github.io/agent-lightning/stable/tutorials/parallelize/#__codelineno-17-2) _id: "rs0", [](https://microsoft.github.io/agent-lightning/stable/tutorials/parallelize/#__codelineno-17-3) members: [{ _id: 0, host: "localhost:27017" }], [](https://microsoft.github.io/agent-lightning/stable/tutorials/parallelize/#__codelineno-17-4) });` To scale out further, launch the store server via [`agl store --backend mongo`](https://microsoft.github.io/agent-lightning/stable/reference/cli/) (see [Debugging with External Store](https://microsoft.github.io/agent-lightning/stable/tutorials/debug/#debug-with-external-store "Debug the Algorithm-Runner Boundary") ). The CLI accepts `--n-workers`, which starts the server under `gunicorn` with multiple worker processes so concurrent runners can push and pull at higher throughput. This option applies only to persistent backends; an in-memory store, on the other hand, cannot be sharded across workers because its state lives inside one process. Note The `--n-workers` here is the number of worker processes for the store server, NOT related to the number of rollout runners. Increasing Throughput of LLM Proxy[¶](https://microsoft.github.io/agent-lightning/stable/tutorials/parallelize/#increasing-throughput-of-llm-proxy "Permanent link") --------------------------------------------------------------------------------------------------------------------------------------------------------------------- Agent-lightning includes an optional [`LLMProxy`](https://microsoft.github.io/agent-lightning/stable/reference/algorithm/#agentlightning.LLMProxy " agentlightning.LLMProxy") that wraps [LiteLLM](https://docs.litellm.ai/) to provide a unified OpenAI-compatible endpoint for your agents. When rollout throughput increases, the proxy can become a bottleneck. You can scale it out using the same pattern as the store server. To increase proxy throughput, pass `num_workers` when constructing the proxy: `[](https://microsoft.github.io/agent-lightning/stable/tutorials/parallelize/#__codelineno-18-1) import agentlightning as agl [](https://microsoft.github.io/agent-lightning/stable/tutorials/parallelize/#__codelineno-18-2) [](https://microsoft.github.io/agent-lightning/stable/tutorials/parallelize/#__codelineno-18-3) proxy = agl.LLMProxy( [](https://microsoft.github.io/agent-lightning/stable/tutorials/parallelize/#__codelineno-18-4) port=4000, [](https://microsoft.github.io/agent-lightning/stable/tutorials/parallelize/#__codelineno-18-5) launch_mode="mp", # multiprocessing mode [](https://microsoft.github.io/agent-lightning/stable/tutorials/parallelize/#__codelineno-18-6) num_workers=4, # four gunicorn workers handle concurrent requests [](https://microsoft.github.io/agent-lightning/stable/tutorials/parallelize/#__codelineno-18-7) )` You can also configure the proxy through [`Trainer`](https://microsoft.github.io/agent-lightning/stable/reference/trainer/#agentlightning.Trainer " agentlightning.Trainer") : `[](https://microsoft.github.io/agent-lightning/stable/tutorials/parallelize/#__codelineno-19-1) trainer = agl.Trainer( [](https://microsoft.github.io/agent-lightning/stable/tutorials/parallelize/#__codelineno-19-2) algorithm=algorithm, [](https://microsoft.github.io/agent-lightning/stable/tutorials/parallelize/#__codelineno-19-3) n_runners=8, # The runners here is the rollout runners, not related to LLM proxy replicas [](https://microsoft.github.io/agent-lightning/stable/tutorials/parallelize/#__codelineno-19-4) llm_proxy={"port": 4000, "num_workers": 4}, # launch mode is actually mp by default [](https://microsoft.github.io/agent-lightning/stable/tutorials/parallelize/#__codelineno-19-5) )` When `num_workers > 1`, the launcher starts gunicorn with the specified number of worker processes. Each worker runs its own event loop, allowing the proxy to handle many concurrent LLM requests without being blocked by Python's GIL. Tip When using `mp` launch mode, [`LLMProxy`](https://microsoft.github.io/agent-lightning/stable/reference/algorithm/#agentlightning.LLMProxy " agentlightning.LLMProxy") will start the server in a separate process. To make sure the proxy is still accessing the same store as the main process, you need to set the store to be [zero-copy compatible](https://microsoft.github.io/agent-lightning/stable/deep-dive/store/#store-capabilities "Capabilities") , which means, either the store is a native zero-copy store like [`MongoLightningStore`](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.store.mongo.MongoLightningStore " agentlightning.store.mongo.MongoLightningStore") or the store is wrapped via [`LightningStoreServer`](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.LightningStoreServer " agentlightning.LightningStoreServer") or [`LightningStoreClient`](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.LightningStoreClient " agentlightning.LightningStoreClient") . Shared Server Infrastructure Both [`LightningStoreServer`](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.LightningStoreServer " agentlightning.LightningStoreServer") and [`LLMProxy`](https://microsoft.github.io/agent-lightning/stable/reference/algorithm/#agentlightning.LLMProxy " agentlightning.LLMProxy") rely on a common utility called [`PythonServerLauncherArgs`](https://microsoft.github.io/agent-lightning/stable/reference/utilities/#agentlightning.utils.server_launcher.PythonServerLauncherArgs " agentlightning.utils.server_launcher.PythonServerLauncherArgs dataclass ") . This dataclass captures the settings needed to launch a FastAPI application: `[](https://microsoft.github.io/agent-lightning/stable/tutorials/parallelize/#__codelineno-20-1) from agentlightning.utils import PythonServerLauncherArgs [](https://microsoft.github.io/agent-lightning/stable/tutorials/parallelize/#__codelineno-20-2) [](https://microsoft.github.io/agent-lightning/stable/tutorials/parallelize/#__codelineno-20-3) args = PythonServerLauncherArgs( [](https://microsoft.github.io/agent-lightning/stable/tutorials/parallelize/#__codelineno-20-4) port=8000, [](https://microsoft.github.io/agent-lightning/stable/tutorials/parallelize/#__codelineno-20-5) host="0.0.0.0", [](https://microsoft.github.io/agent-lightning/stable/tutorials/parallelize/#__codelineno-20-6) n_workers=4, # spawn 4 gunicorn workers [](https://microsoft.github.io/agent-lightning/stable/tutorials/parallelize/#__codelineno-20-7) launch_mode="thread", # or "mp" for multiprocessing, "asyncio" for in-loop [](https://microsoft.github.io/agent-lightning/stable/tutorials/parallelize/#__codelineno-20-8) )` Under the hood, [`PythonServerLauncher`](https://microsoft.github.io/agent-lightning/stable/reference/utilities/#agentlightning.utils.server_launcher.PythonServerLauncher " agentlightning.utils.server_launcher.PythonServerLauncher") reads these arguments and chooses between uvicorn (single worker) and gunicorn (multiple workers) automatically. Back to top --- # Maintainer Guide - Agent-lightning [Skip to content](https://microsoft.github.io/agent-lightning/stable/community/maintainers/#maintainer-guide) ⚠️ You are viewing development documentation. For stable documentation, please visit [https://microsoft.github.io/agent-lightning/stable/](https://microsoft.github.io/agent-lightning/stable/) Maintainer Guide[¶](https://microsoft.github.io/agent-lightning/stable/community/maintainers/#maintainer-guide "Permanent link") ================================================================================================================================= This guide describes the day-to-day responsibilities for Agent Lightning maintainers—how to bump versions, run release ceremonies, interact with CI, and backport fixes safely. Release Workflow[¶](https://microsoft.github.io/agent-lightning/stable/community/maintainers/#release-workflow "Permanent link") --------------------------------------------------------------------------------------------------------------------------------- Follow this checklist throughout each release cycle. ### Immediately After Shipping[¶](https://microsoft.github.io/agent-lightning/stable/community/maintainers/#immediately-after-shipping "Permanent link") Agent Lightning uses a **bump-first** strategy. As soon as a release is published: 1. Update version metadata: * `pyproject.toml`: bump the `version`. * `agentlightning/__init__.py`: update `__version__` if it exists. * `uv.lock`: refresh the lock file after the bump. 2. Refresh dependency pins as needed: `[](https://microsoft.github.io/agent-lightning/stable/community/maintainers/#__codelineno-0-1) uv lock --upgrade` 3. For a new minor or major release, create a stable branch from `main`: `[](https://microsoft.github.io/agent-lightning/stable/community/maintainers/#__codelineno-1-1) git checkout main [](https://microsoft.github.io/agent-lightning/stable/community/maintainers/#__codelineno-1-2) git pull upstream main [](https://microsoft.github.io/agent-lightning/stable/community/maintainers/#__codelineno-1-3) git checkout -b stable/v2.0.x # adjust to the new series [](https://microsoft.github.io/agent-lightning/stable/community/maintainers/#__codelineno-1-4) git push upstream stable/v2.0.x` All future changes to the stable branch must land via pull requests. ### Preparing the Next Release[¶](https://microsoft.github.io/agent-lightning/stable/community/maintainers/#preparing-the-next-release "Permanent link") When it is time to publish the next version: 1. **Draft release notes** in `docs/changelog.md`, collecting every notable change since the previous tag. 2. **Open a release PR** targeting `main` (for minor/major) or the relevant stable branch (for patch releases). Use the title `[Release] vX.Y.Z`. 3. **Run extended CI** by labeling the PR with `ci-all` and commenting `/ci`. Investigate and resolve any failures. 4. **Merge the release PR** once notes are final and CI is green. 5. **Tag the release** from the branch you just merged into: `[](https://microsoft.github.io/agent-lightning/stable/community/maintainers/#__codelineno-2-1) git checkout main # minor/major releases [](https://microsoft.github.io/agent-lightning/stable/community/maintainers/#__codelineno-2-2) git checkout stable/vX.Y.Z # patch releases [](https://microsoft.github.io/agent-lightning/stable/community/maintainers/#__codelineno-2-3) [](https://microsoft.github.io/agent-lightning/stable/community/maintainers/#__codelineno-2-4) git pull [](https://microsoft.github.io/agent-lightning/stable/community/maintainers/#__codelineno-2-5) git tag vX.Y.Z -m "Release vX.Y.Z" [](https://microsoft.github.io/agent-lightning/stable/community/maintainers/#__codelineno-2-6) git push upstream vX.Y.Z` Pushing the tag publishes to PyPI and deploys the documentation. 6. **Publish the GitHub release** using the drafted notes, and confirm the docs site and PyPI listing reflect the new version. Working with CI Labels and `/ci`[¶](https://microsoft.github.io/agent-lightning/stable/community/maintainers/#working-with-ci-labels-and-ci "Permanent link") -------------------------------------------------------------------------------------------------------------------------------------------------------------- GPU suites and example end-to-end runs are opt-in. To trigger them on a pull request: 1. Apply the appropriate labels before issuing the command: * `ci-all` for every repository-dispatch workflow. * `ci-gpu` for GPU integration tests (`tests-full.yml`). * `ci-apo`, `ci-calc-x`, `ci-spider`, `ci-unsloth`, `ci-compat` for the individual example pipelines. 2. Comment `/ci` on the PR. The `issue-comment` workflow acknowledges the request and tracks job results inline. 3. Remove the labels once you have the signal to avoid accidental re-runs. Use `/ci` whenever changes touch shared infrastructure, dependencies, or training loops that require coverage beyond the default PR checks. Note `/ci` always executes the workflow definitions on the current `main` branch, then checks out the PR diff. If you need to test workflow modifications, push the changes to a branch in the upstream repo and run: `[](https://microsoft.github.io/agent-lightning/stable/community/maintainers/#__codelineno-3-1) gh workflow run examples-xxx.yml --ref your-branch-name` Backporting Pull Requests[¶](https://microsoft.github.io/agent-lightning/stable/community/maintainers/#backporting-pull-requests "Permanent link") --------------------------------------------------------------------------------------------------------------------------------------------------- Supported stable branches rely on automated backports: 1. Identify the target branch (for example `stable/v0.2.x`). 2. Before merging the original PR into `main`, add the matching `stable/` label (e.g. `stable/v0.2.x`). 3. The `backport.yml` workflow opens a follow-up PR named `backport//` authored by `agent-lightning-bot`. 4. Review the generated PR, ensure CI is green, and merge into the stable branch. 5. Resolve conflicts by pushing manual fixes to the backport branch and re-running `/ci` if required. Keep stable branches healthy by cherry-picking only critical fixes and ensuring documentation and example metadata stay aligned with each release line. Back to top --- # Bird's Eye View - Agent-lightning [Skip to content](https://microsoft.github.io/agent-lightning/stable/deep-dive/birds-eye-view/#the-birds-eye-view-of-agent-lightning) ⚠️ You are viewing development documentation. For stable documentation, please visit [https://microsoft.github.io/agent-lightning/stable/](https://microsoft.github.io/agent-lightning/stable/) The Bird's Eye View of Agent-lightning[¶](https://microsoft.github.io/agent-lightning/stable/deep-dive/birds-eye-view/#the-birds-eye-view-of-agent-lightning "Permanent link") =============================================================================================================================================================================== High Volume of Information Ahead This article provides an in-depth exploration of the Agent-lightning architecture. It is not intended as a beginner’s guide or usage tutorial. This article summarizes how Agent-lightning (as of v0.2) wires the [Algorithm](https://microsoft.github.io/agent-lightning/stable/reference/algorithm/#agentlightning.Algorithm " agentlightning.Algorithm") , [Runner](https://microsoft.github.io/agent-lightning/stable/reference/runner/#agentlightning.Runner " agentlightning.Runner") , and [LightningStore](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.LightningStore " agentlightning.LightningStore") loop together and shows where auxiliary components (the [Tracer](https://microsoft.github.io/agent-lightning/stable/reference/runner/#agentlightning.Tracer " agentlightning.Tracer") , [Adapter](https://microsoft.github.io/agent-lightning/stable/reference/algorithm/#agentlightning.Adapter " agentlightning.Adapter") , and [LLM Proxy](https://microsoft.github.io/agent-lightning/stable/reference/algorithm/#agentlightning.LLMProxy " agentlightning.LLMProxy") ) plug into the loop. Each section provides a diagram for a different perspective of the system. Algorithm ↔ Runner ↔ Store data flow[¶](https://microsoft.github.io/agent-lightning/stable/deep-dive/birds-eye-view/#algorithm-runner-store-data-flow "Permanent link") ------------------------------------------------------------------------------------------------------------------------------------------------------------------------ At its heart, Agent-lightning is built on three main components that work in a coordinated loop: * **[Algorithm](https://microsoft.github.io/agent-lightning/stable/reference/algorithm/#agentlightning.Algorithm " agentlightning.Algorithm") :** The "brain" of the system. It decides what tasks to run, learns from the results, and updates resources (like AI models or prompts). * **[Runner](https://microsoft.github.io/agent-lightning/stable/reference/runner/#agentlightning.Runner " agentlightning.Runner") :** The "worker" of the system. It executes tasks assigned by the algorithm, runs the agent, and records the results. * **[LightningStore](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.LightningStore " agentlightning.LightningStore") :** The central "database" and message queue. It acts as the single source of truth, storing tasks, results, and resources, and enabling communication between the Algorithm and Runner. The typical data flow in a training loop is as follows: The **[Algorithm](https://microsoft.github.io/agent-lightning/stable/reference/algorithm/#agentlightning.Algorithm " agentlightning.Algorithm") ** enqueues tasks (called **[Rollouts](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.Rollout " agentlightning.Rollout") **) into the **[LightningStore](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.LightningStore " agentlightning.LightningStore") **. A **[Runner](https://microsoft.github.io/agent-lightning/stable/reference/runner/#agentlightning.Runner " agentlightning.Runner") ** then dequeues a task, executes it, and streams the results (called **[Spans](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.Span " agentlightning.Span") **) back to the store. Once the task is complete, the algorithm can query the new data from the store to learn and update its resources. The diagram below shows this fundamental interaction in a simple, non-parallel setup. sequenceDiagram autonumber participant Algo as Algorithm participant Store as LightningStore participant Runner participant Agent loop Over the dataset Algo-->>Store: add_resources + enqueue_rollout Store-->>Runner: dequeue_rollout → AttemptedRollout Store-->>Runner: get_latest_resources Runner-->>Store: update_attempt("running", worker_id) Runner->>Agent: rollout + resources Agent->>Runner: reward / spans Runner-->>Store: add_span or add_otel_span Runner-->>Store: update_attempt("finished", status) Store-->>Algo: query_rollouts + spans Algo-->>Algo: Update resources (optional) end Solid lines represent direct calls, while dashed lines are asynchronous or long-running operations. ### Key Terminology[¶](https://microsoft.github.io/agent-lightning/stable/deep-dive/birds-eye-view/#key-terminology "Permanent link") We define the following terms, which may be helpful for understanding the diagram above. * **[Resources](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.Resource " agentlightning.Resource") :** A collection of assets to be tuned or trained. Agents perform rollouts against resources and collect span data. Algorithms use those data to update the resources. In RL training, the resources are a tunable model. In prompt tuning, the resources are prompt templates. * **[Rollout](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.Rollout " agentlightning.Rollout") :** A unit of work that an agent performs against a resource. A rollout (noun) can be incomplete, in which case it is also known as a **task**, **sample**, or **job** (these terms are used interchangeably). The agent executes its own defined workflow against the rollout — the process is also called "to rollout" (verb). After execution, the rollout (noun) is considered _complete_. * **[Attempt](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.Attempt " agentlightning.Attempt") :** A single execution of a rollout. One rollout can have multiple attempts in case of failures or timeouts. * **[Span](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.Span " agentlightning.Span") :** During the rollout, the agent can generate multiple spans (also known as "traces" or "events"). The recorded spans are collected in the store, which is crucial for understanding agent behavior and optimizing agents. * **[Reward](https://microsoft.github.io/agent-lightning/stable/reference/agent/#agentlightning.emit_reward " agentlightning.emit_reward(reward, *, primary_key=None, attributes=None, propagate=True)") :** A special span that is defined as a number judging the quality of the rollout during some period of the rollout. * **[Dataset](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.Dataset " agentlightning.Dataset") :** A collection of incomplete rollouts (i.e., tasks) for the agent to process. The dual datasets (train, val) serve as the initial input for the algorithm to enqueue the first batch of rollouts. ### Store[¶](https://microsoft.github.io/agent-lightning/stable/deep-dive/birds-eye-view/#store "Permanent link") As discussed previously, the [LightningStore](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.LightningStore " agentlightning.LightningStore") is the central hub for all data in Agent-lightning. The store exposes a set of APIs for algorithms and runners to interact with the data; the most important ones are: `[](https://microsoft.github.io/agent-lightning/stable/deep-dive/birds-eye-view/#__codelineno-0-1) from agentlightning.types import AttemptedRollout, ResourcesUpdate, Span, TaskInput [](https://microsoft.github.io/agent-lightning/stable/deep-dive/birds-eye-view/#__codelineno-0-2) [](https://microsoft.github.io/agent-lightning/stable/deep-dive/birds-eye-view/#__codelineno-0-3) class LightningStore: [](https://microsoft.github.io/agent-lightning/stable/deep-dive/birds-eye-view/#__codelineno-0-4) [](https://microsoft.github.io/agent-lightning/stable/deep-dive/birds-eye-view/#__codelineno-0-5) async def enqueue_rollout(self, input: TaskInput, ...) -> Rollout: ... [](https://microsoft.github.io/agent-lightning/stable/deep-dive/birds-eye-view/#__codelineno-0-6) [](https://microsoft.github.io/agent-lightning/stable/deep-dive/birds-eye-view/#__codelineno-0-7) async def dequeue_rollout(self) -> AttemptedRollout | None: ... [](https://microsoft.github.io/agent-lightning/stable/deep-dive/birds-eye-view/#__codelineno-0-8) [](https://microsoft.github.io/agent-lightning/stable/deep-dive/birds-eye-view/#__codelineno-0-9) async def add_span(self, span: Span) -> Span: ... [](https://microsoft.github.io/agent-lightning/stable/deep-dive/birds-eye-view/#__codelineno-0-10) [](https://microsoft.github.io/agent-lightning/stable/deep-dive/birds-eye-view/#__codelineno-0-11) async def get_latest_resources(self) -> Optional[ResourcesUpdate]: ... [](https://microsoft.github.io/agent-lightning/stable/deep-dive/birds-eye-view/#__codelineno-0-12) [](https://microsoft.github.io/agent-lightning/stable/deep-dive/birds-eye-view/#__codelineno-0-13) async def wait_for_rollouts(self, rollout_ids: List[str], ...): ... [](https://microsoft.github.io/agent-lightning/stable/deep-dive/birds-eye-view/#__codelineno-0-14) [](https://microsoft.github.io/agent-lightning/stable/deep-dive/birds-eye-view/#__codelineno-0-15) async def query_spans(self, rollout_id: str, ...): ... [](https://microsoft.github.io/agent-lightning/stable/deep-dive/birds-eye-view/#__codelineno-0-16) [](https://microsoft.github.io/agent-lightning/stable/deep-dive/birds-eye-view/#__codelineno-0-17) async def update_attempt(self, rollout_id: str, attempt_id: str, status: str, ...): ... [](https://microsoft.github.io/agent-lightning/stable/deep-dive/birds-eye-view/#__codelineno-0-18) [](https://microsoft.github.io/agent-lightning/stable/deep-dive/birds-eye-view/#__codelineno-0-19) ...` These interfaces operate on [`AttemptedRollout`](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.AttemptedRollout " agentlightning.AttemptedRollout") , [`ResourcesUpdate`](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.ResourcesUpdate " agentlightning.ResourcesUpdate") , [`Span`](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.Span " agentlightning.Span") , and [`TaskInput`](https://microsoft.github.io/agent-lightning/stable/reference/internal/#agentlightning.TaskInput " agentlightning.TaskInput = Any module-attribute ") instances from `agentlightning.types`. As the APIs show, the store essentially provides a queue for rollouts and storage for resources, spans, and attempts. Developers should implement the store carefully to ensure data integrity and consistency, especially when multiple runners work in parallel across multiple attempts. The store is designed to be extensible. Users can implement their own store by inheriting from [`LightningStore`](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.LightningStore " agentlightning.LightningStore") and overriding methods. Agent-lightning provides a few reference implementations, such as [`InMemoryLightningStore`](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.InMemoryLightningStore " agentlightning.InMemoryLightningStore") (default) and `SqliteLightningStore` (under construction). When parallelized, the store may need special wrappers to ensure thread/process safety or delegate computation to a store in another process or machine. Supporting Components in the Loop[¶](https://microsoft.github.io/agent-lightning/stable/deep-dive/birds-eye-view/#supporting-components-in-the-loop "Permanent link") ---------------------------------------------------------------------------------------------------------------------------------------------------------------------- While the core loop is simple, Agent-lightning provides several components to make development easier and more powerful. ### Tracer[¶](https://microsoft.github.io/agent-lightning/stable/deep-dive/birds-eye-view/#tracer "Permanent link") The [`Tracer`](https://microsoft.github.io/agent-lightning/stable/reference/runner/#agentlightning.Tracer " agentlightning.Tracer") is a component within the [`Runner`](https://microsoft.github.io/agent-lightning/stable/reference/runner/#agentlightning.Runner " agentlightning.Runner") that records detailed spans (events) during an agent's execution and sends them to the [`LightningStore`](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.LightningStore " agentlightning.LightningStore") . Instead of requiring the agent to manually log every span, the tracer automatically instruments key methods (e.g., LLM calls) and captures their inputs, outputs, and metadata. This provides a detailed log of the agent's behavior with minimal effort. sequenceDiagram autonumber participant Store participant Runner participant Tracer participant Agent Note over Runner,Tracer: Runner manages tracer as member Tracer->>Agent: Apply instrumentation loop Until no more rollouts Store-->>Runner: dequeue_rollout → AttemptedRollout Store-->>Runner: get_latest_resources Runner->>Agent: training_rollout / validation_rollout loop For each finished span Agent-->>Tracer: openai.chat.completion invoked
agent.execute invoked
... Agent->>Tracer: emit intermediate reward Tracer-->>Store: add_otel_span(rollout_id, attempt_id, span) end Agent->>Runner: final reward + extra spans (if any) Runner-->>Store: add_span(rollout_id, attempt_id, span) Runner-->>Store: update_attempt(status) end Tracer->>Agent: Unapply instrumentation The above diagram shows the overall data flow between store, tracer and agent. In realistic, it's a bit more complicated than that. Spans are not emitted actively by the agent; they are intercepted by the tracer by hooking and instrumenting key methods used in the agents. The tracer uses a callback (called exporter) to monitor events and log to the store. Before a rollout starts, the runner enters a [`trace_context`](https://microsoft.github.io/agent-lightning/stable/reference/runner/#agentlightning.Tracer.trace_context " trace_context(name=None, *, store=None, rollout_id=None, attempt_id=None)") before invoking the agent, wiring store identifiers into the tracer. Each span completion streams back to the store through `LightningSpanProcessor`, so the agent’s instrumentation lands in [`add_otel_span`](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.LightningStore.add_otel_span " add_otel_span(rollout_id, attempt_id, readable_span, sequence_id=None) async ") . If the agent’s rollout method returns a numeric reward, the runner emits one more OpenTelemetry span before finalizing the attempt. ### Hooks[¶](https://microsoft.github.io/agent-lightning/stable/deep-dive/birds-eye-view/#hooks "Permanent link") [`Hook`](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.Hook " agentlightning.Hook") implementations are user-defined callback functions that allow you to augment a [`Runner`](https://microsoft.github.io/agent-lightning/stable/reference/runner/#agentlightning.Runner " agentlightning.Runner") 's behavior at specific points in its lifecycle. You can use hooks to add custom logging, set up resources before a rollout begins, or tear them down after it ends. Hooks can be triggered at four key moments: `on_rollout_start`, `on_trace_start`, `on_trace_end`, and `on_rollout_end`. Users should pay special attention to the difference between `on_trace_end` and `on_rollout_end`. The former is called right before the tracer exits the trace context, while the latter is called after the runner processes the final leftover rewards and spans, and finalizes the attempt in the store. sequenceDiagram autonumber participant Store participant Hooks participant Runner participant Tracer participant Agent Note over Runner,Hooks: Runner manages hooks as member loop Until no more rollouts Store-->>Runner: dequeue_rollout → AttemptedRollout Store-->>Runner: get_latest_resources Runner->>Hooks: on_rollout_start(agent, runner, rollout) Runner->>Agent: training_rollout / validation_rollout Tracer->>Agent: enter_trace_context activate Tracer Runner->>Hooks: on_trace_start(agent, runner, tracer, rollout) Note over Runner,Agent: Agent rollout omitted Runner->>Hooks: on_trace_end(agent, runner, tracer, rollout) Tracer->>Agent: exit_trace_context deactivate Tracer Agent->>Runner: final reward + extra spans (if any) Runner-->>Store: add_span(rollout_id, attempt_id, span) Runner->>Hooks: on_rollout_end(agent, runner, rollout, status) end ### Adapter[¶](https://microsoft.github.io/agent-lightning/stable/deep-dive/birds-eye-view/#adapter "Permanent link") The [`Adapter`](https://microsoft.github.io/agent-lightning/stable/reference/algorithm/#agentlightning.Adapter " agentlightning.Adapter") is a component used by the [`Algorithm`](https://microsoft.github.io/agent-lightning/stable/reference/algorithm/#agentlightning.Algorithm " agentlightning.Algorithm") to transform raw data from the [`LightningStore`](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.LightningStore " agentlightning.LightningStore") into a format suitable for learning. Runners stream raw spans into the store during execution. Later, the algorithm queries these spans and uses an adapter to convert them into structured data, like training examples for a reinforcement learning model. For instance, the [`TracerTraceToTriplet`](https://microsoft.github.io/agent-lightning/stable/reference/algorithm/#agentlightning.TracerTraceToTriplet " agentlightning.TracerTraceToTriplet") processes OpenTelemetry spans to create `(prompt, response, reward)` triplets, which are the fundamental data structure for many RL fine-tuning algorithms. flowchart LR Runner -- (1) add_otel_span --> Store Store -- (2) query_spans --> Algorithm Algorithm -- (3) spans --> Adapter Adapter -- (4) transformed data --> Algorithm ### LLM Proxy[¶](https://microsoft.github.io/agent-lightning/stable/deep-dive/birds-eye-view/#llm-proxy "Permanent link") The [`LLMProxy`](https://microsoft.github.io/agent-lightning/stable/reference/algorithm/#agentlightning.LLMProxy " agentlightning.LLMProxy") is an optional bridge component that sits between an agent and the algorithms' resources. It acts as a centralized endpoint for all LLM calls. Usually the proxy URL is added to the store as a special resource, so that the [`Runner`](https://microsoft.github.io/agent-lightning/stable/reference/runner/#agentlightning.Runner " agentlightning.Runner") can fetch it along with other resources when dequeuing a rollout. During rollouts, the runner invokes the proxy's HTTP endpoint instead of calling a model backend directly. This design offers several benefits: 1. **Instrumentation:** It automatically captures detailed traces of LLM interactions (prompts, responses, metadata) and sends them to the store, complementing the tracer, especially when the agent's code is hard to instrument directly. 2. **Backend Abstraction:** It provides a unified interface for various LLM backends (OpenAI, Anthropic, local models) and can add features like retry logic, rate limiting, and caching. 3. **Resource Management:** The algorithm can dynamically update which LLM the agent uses (e.g., swapping to a newly fine-tuned model) by simply swapping the backend model the proxy is using, without interrupting the agent's code. The benefits above seem to be all discussed within the context of model fine-tuning. As a matter of fact, the proxy can be useful for prompt tuning as well. The algorithm can register one of the following two types of endpoints into the proxy: 1. **Endpoint served by the algorithm:** If the algorithm is internally updating the LLM weights (e.g., RL), it can launch an LLM inference engine (i.e., a model server) and register the endpoint URL with the proxy. The proxy then forwards all LLM calls to that endpoint. 2. **Third-party LLM endpoint:** If the algorithm is not updating the LLM weights (e.g., prompt tuning), it can register a third-party LLM endpoint into the proxy. We show a diagram below that illustrates how the proxy fits into the overall data flow. sequenceDiagram autonumber participant Algo as Algorithm participant LLMProxy as LLM Proxy participant Store participant Runner participant Agent Note over Algo,LLMProxy: Algorithm manages LLMProxy as member loop Over the Dataset Algo->>Algo: Launch LLM Inference Engine
(optional) Algo->>LLMProxy: Register Inference Engine
(optional) Algo-->>Store: enqueue_rollout LLMProxy->>Store: Proxy URL added as Resource Store-->>Runner: dequeue_rollout → AttemptedRollout Store-->>Runner: get_latest_resources Runner->>Agent: rollout + resources
(LLM Proxy URL as resource) loop Defined by Agent Agent-->>LLMProxy: LLM calls activate LLMProxy LLMProxy-->>Store: add_span or add_otel_span LLMProxy-->>Agent: LLM responses deactivate LLMProxy Agent-->>Runner: rewards Runner-->>Store: add_span or add_otel_span end Runner-->>Store: update_attempt("finished", status) Store-->>Algo: query_rollouts + spans Algo-->>Algo: Update LLM Weights
(optional) end In this diagram, the store receives spans from both the proxy and the runner. We will see a problem later with parallelism where the proxy and runner are in different machines, and spans need to obtain a special counter from the store to ensure the ordering of spans. ### Trainer[¶](https://microsoft.github.io/agent-lightning/stable/deep-dive/birds-eye-view/#trainer "Permanent link") The [Trainer](https://microsoft.github.io/agent-lightning/stable/reference/trainer/#agentlightning.Trainer " agentlightning.Trainer") is the high-level orchestrator that initializes and connects all major components -- [Algorithm](https://microsoft.github.io/agent-lightning/stable/reference/algorithm/#agentlightning.Algorithm " agentlightning.Algorithm") , [Runner](https://microsoft.github.io/agent-lightning/stable/reference/runner/#agentlightning.Runner " agentlightning.Runner") , [LightningStore](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.LightningStore " agentlightning.LightningStore") , [Tracer](https://microsoft.github.io/agent-lightning/stable/reference/runner/#agentlightning.Tracer " agentlightning.Tracer") , [Adapter](https://microsoft.github.io/agent-lightning/stable/reference/algorithm/#agentlightning.Adapter " agentlightning.Adapter") , [LLM Proxy](https://microsoft.github.io/agent-lightning/stable/reference/algorithm/#agentlightning.LLMProxy " agentlightning.LLMProxy") , and [Hook](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.Hook " agentlightning.Hook") . The components can have a lifecycle as long as the trainer. The trainer manages their lifecycles and handles dependency injection, ensuring that every part of the system operates within a consistent and shared environment. Below, we demonstrate how the components relate to each other and their roles. We first clarify the roles and relationships shown in the diagram: 1. **Owns:** components that the trainer constructs and manages directly (e.g., runner, tracer). 2. **Injects:** components passed into others as dependencies. 3. **References:** weak links for coordination without ownership. 4. **Uses:** components that are temporarily interacted with. For example, the [LightningStore](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.LightningStore " agentlightning.LightningStore") is injected into the [Algorithm](https://microsoft.github.io/agent-lightning/stable/reference/algorithm/#agentlightning.Algorithm " agentlightning.Algorithm") and [Runner](https://microsoft.github.io/agent-lightning/stable/reference/runner/#agentlightning.Runner " agentlightning.Runner") . The [Tracer](https://microsoft.github.io/agent-lightning/stable/reference/runner/#agentlightning.Tracer " agentlightning.Tracer") and [LitAgent](https://microsoft.github.io/agent-lightning/stable/reference/agent/#agentlightning.LitAgent " agentlightning.LitAgent") are injected into the runner. The [Adapter](https://microsoft.github.io/agent-lightning/stable/reference/algorithm/#agentlightning.Adapter " agentlightning.Adapter") and [LLM Proxy](https://microsoft.github.io/agent-lightning/stable/reference/algorithm/#agentlightning.LLMProxy " agentlightning.LLMProxy") are injected into the algorithm. The store is further injected into the tracer, adapter and LLM proxy by the runner and algorithm respectively. flowchart TD %% === Left side: Algorithm domain === subgraph L["Algorithm Side"] Algorithm["Algorithm
(no default)"] Adapter["Adapter
(TracerTraceToTriplet*)"] LLMProxy["LLM Proxy
(no default)"] Algorithm -.injects.-> Adapter Algorithm -.injects.-> LLMProxy end linkStyle 0,1 stroke:#896978,stroke-width:2px; %% === Middle: Core trainer and store === subgraph M["Core"] Trainer["Trainer"] Store["LightningStore
(InMemory* default)"] Trainer --has--> Algorithm Trainer --has--> Store Trainer --has--> Adapter Trainer --has--> LLMProxy end linkStyle 2,3,4,5 stroke:#839791,stroke-width:2px; %% === Right side: Runner side === subgraph R["Runner Side"] Runner["Runner
(LitAgentRunner* default)"] Tracer["Tracer
(AgentOpsTracer*)"] Hooks["Hooks (empty default)"] Agent["Agent
(LitAgent*)"] Runner -.injects.-> Tracer Runner -.injects.-> Store Runner -.injects.-> Agent Runner -.injects.-> Hooks Tracer -.injects.-> Store Hooks -.uses.-> Runner Hooks -.uses.-> Agent Hooks -.uses.-> Tracer end linkStyle 6,7,8,9,10 stroke:#896978,stroke-width:2px; linkStyle 11,12,13 stroke:#7a89c2,stroke-width:2px; %% === Cross-section connections === Trainer --has--> Runner Trainer --has--> Tracer Trainer --has--> Hooks Trainer --uses--> Agent Algorithm -.injects.-> Store LLMProxy -.injects.-> Store Agent -.references.-> Trainer Runner -.references.-> Trainer Algorithm -.references.-> Trainer linkStyle 14,15,16 stroke:#839791,stroke-width:2px; linkStyle 17,20,21,22 stroke:#7a89c2,stroke-width:2px; linkStyle 18,19 stroke:#896978,stroke-width:2px; style L fill:none; style M fill:none; style R fill:none; Putting It All Together: A Reinforcement Learning Example (VERL)[¶](https://microsoft.github.io/agent-lightning/stable/deep-dive/birds-eye-view/#putting-it-all-together-a-reinforcement-learning-example-verl "Permanent link") --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- [](https://microsoft.github.io/agent-lightning/stable/deep-dive/birds-eye-view/) VERL shows how an algorithm consumes the shared infrastructure. For historical reasons, code lives in `agentlightning.algorithm.verl` and `agentlightning.verl`. The latter is legacy and reuses terms like `Trainer` in confusing ways. The former is a thin wrapper that conforms to the new algorithm interface. Future versions will merge the two. Reinforcement learning aims to learn a policy that takes actions in states to maximize expected reward. For agents, the policy is usually a language model. Inputs are prompts (state). Outputs are generated text (action). A numeric score judges quality (reward). The `(state, action, reward)` **triplet** is the basic learning unit. In Agent-lightning, the environment is implicit in the agent’s workflow, which orchestrates one or more LLM calls and often self-judges using rules or additional model calls. During a rollout, the agent emits spans that contain everything needed for RL training, including LLM call traces and numeric judge/reward signals. The "algorithm", on the other hand, have more responsibilities. 1. Providing a language model deployment that is currently learning and improving for the agent to interact with; 2. Preparing the tasks that the agents will perform; 3. Querying the spans generated, extracting triplets, and converting them into a format that the underlying RL library can consume; 4. Updating the language model based on the learning signals. In the VERL integration, the algorithm launches a chat completion endpoint using `vLLM` and wraps training with `FSDP` for distributed optimization. It enqueues tasks from the dataset. After rollouts finish, it queries spans and converts them to triplets with `TracerTraceToTriplet`. VERL’s native training loop then consumes these triplets to update model weights. The workflow can be summarized in the following diagram. sequenceDiagram autonumber participant vLLM as vLLM Chat
Completion Endpoint participant FSDP as FSDP / Megatron
Weights Optimizer participant Algo as Algorithm
Main Controller
(Main Process) participant Adapter as TracerTraceToTriplet participant LLMProxy as LLM Proxy participant Store as LightningStore participant Runner as Runner + Agent Note over Algo,LLMProxy: LLMProxy and Adapter are injected by Trainer as member Note over vLLM,Algo: Algorithm creates and owns vLLM and FSDP loop Over the Dataset in Batches Algo->>vLLM: Create Chat Completion Endpoint activate vLLM vLLM->>LLMProxy: Registered as Backend Endpoint LLMProxy->>Store: Proxy URL added as Resource par Over data samples in the batch Algo-->>Store: enqueue_rollout Store-->>Runner: Dequeue Rollout +
Resources (i.e., URL) loop One Rollout Attempt Runner-->>LLMProxy: LLM calls LLMProxy-->>vLLM: Forwarded LLM calls vLLM-->>LLMProxy: LLM responses LLMProxy-->>Store: add_span / add_otel_span LLMProxy-->>Runner: Forwarded LLM responses Runner-->>Store: add_span / add_otel_span
(by tracer, including rewards) end Runner-->>Store: update_attempt("finished", status) end Algo-->>Store: Poll for completed rollouts + spans Algo->>vLLM: Chat Completion Endpoint Sleeps deactivate vLLM Algo->>Adapter: adapt(spans) Adapter->>FSDP: Triplets (state, action, reward) activate FSDP FSDP-->>Algo: Updated LLM weights deactivate FSDP end **Notes:** 1. There are interactions between different components injected into or owned by algorithms in the diagram, such as the output of the adapter feeding into the FSDP optimizer. This is for simplicity of illustration and slightly different from the actual implementation, where it's the algorithm main controller that orchestrates the data flow between components. 2. **On mapping to VERL.** VERL uses a classic RLHF setup where each action is a single token, the state is the full conversation history up to that token, and reward is given at the end. This is very different from our setup where each action is actually a chunk of text, although they are both called RL! Therefore, after the adapter produces triplets, the algorithm converts each `(state, action, reward)` into a VERL trajectory (`DataProto`) with keys like `input_ids`, `position_ids`, `attention_mask`, and `token_level_scores`. That conversion happens after triplet generation and is not shown in the diagram. Execution Strategies and Parallelism[¶](https://microsoft.github.io/agent-lightning/stable/deep-dive/birds-eye-view/#execution-strategies-and-parallelism "Permanent link") ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Readers might have observed from the diagram above that there is absolutely no communication between (1) runner and agents and (2) algorithm. The only overlap of them is the [Trainer](https://microsoft.github.io/agent-lightning/stable/reference/trainer/#agentlightning.Trainer " agentlightning.Trainer") and [LightningStore](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.LightningStore " agentlightning.LightningStore") . This observation is very clear with the diagram within the trainer section. This design allows us to flexibly scale the runner and algorithm independently, which is crucial for large-scale training. Agent-lightning packages two executable bundles: a runner bundle ([Runner](https://microsoft.github.io/agent-lightning/stable/reference/runner/#agentlightning.Runner " agentlightning.Runner") , [Tracer](https://microsoft.github.io/agent-lightning/stable/reference/runner/#agentlightning.Tracer " agentlightning.Tracer") , [Hook](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.Hook " agentlightning.Hook") , [LitAgent](https://microsoft.github.io/agent-lightning/stable/reference/agent/#agentlightning.LitAgent " agentlightning.LitAgent") ) and an algorithm bundle ([Algorithm](https://microsoft.github.io/agent-lightning/stable/reference/algorithm/#agentlightning.Algorithm " agentlightning.Algorithm") , [Adapter](https://microsoft.github.io/agent-lightning/stable/reference/algorithm/#agentlightning.Adapter " agentlightning.Adapter") , [LLM Proxy](https://microsoft.github.io/agent-lightning/stable/reference/algorithm/#agentlightning.LLMProxy " agentlightning.LLMProxy") ). Both share the [LightningStore](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.LightningStore " agentlightning.LightningStore") . The trainer initializes and connects the bundles. graph TD subgraph Runner_Side["Runner Bundle"] direction LR R[Runner] --- T[Tracer] --- H[Hooks] --- A1[Agent] end subgraph Algorithm_Side["Algorithm Bundle"] direction LR ALG[Algorithm] --- AD[Adapter] --- LLM[LLM Proxy] end S[(Store)] TR[Trainer] Runner_Side <--> S Algorithm_Side <--> S TR --> Runner_Side TR --> Algorithm_Side linkStyle 0,1,2,3,4 opacity:0; An [execution strategy](https://microsoft.github.io/agent-lightning/stable/reference/trainer/#agentlightning.ExecutionStrategy " agentlightning.ExecutionStrategy") , defined and owned by the trainer, governs how algorithm and runner bundles are placed, connected, scaled, and aborted. It serves four primary purposes. Execution strategies first determine **bundle placement** — whether the two bundles run in the same thread, process, machine, or across separate machines. They also define **store management**, wrapping the store and specifying how data is shared between bundles. In terms of **scalability**, the strategy can replicate the runner bundle across multiple threads, processes, or machines to expand throughput on the runner side. The algorithm side remains single-process due to the complexity of parallelization. Mature frameworks such as _DeepSpeed_ and _Megatron_ already support distributed model training, so scaling of the algorithm bundle is delegated to those implementations. **Abort handling** is another core responsibility. Aborts may be triggered by normal exits, failures in either bundle, or user interrupts. The trainer must include cancellation interfaces for the bundles so that bundles can be cleanly aborted. When the algorithm bundle exits normally, the strategy signals the runner bundle to terminate. If the runner exits first, no signal is sent to the algorithm, as it may still be processing completed rollouts. In cases of failure or user interruption, the strategy signals both bundles to abort; if a bundle fails to respond, the strategy should attempt a forceful termination. Agent-lightning currently provides two execution strategies: **shared-memory** and **client-server**, described in the following sections. ### Shared-memory Strategy[¶](https://microsoft.github.io/agent-lightning/stable/deep-dive/birds-eye-view/#shared-memory-strategy "Permanent link") [`SharedMemoryExecutionStrategy`](https://microsoft.github.io/agent-lightning/stable/reference/trainer/#agentlightning.SharedMemoryExecutionStrategy " agentlightning.SharedMemoryExecutionStrategy") runs algorithm and runner bundles as threads in one process. The strategy wraps the store with [`LightningStoreThreaded`](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.LightningStoreThreaded " agentlightning.LightningStoreThreaded") , which guards calls with a lock for safe concurrency. This is good for lightweight debugging because components share one Python heap and avoid serialization. It is not suitable for heavy RL training or compute-intensive agents. flowchart TB subgraph MainProcess direction TB subgraph AlgorithmThread [Thread 0] Algorithm[Algorithm bundle] end subgraph RunnerThread1 [Thread 1] Runner1[Runner bundle #1] end subgraph RunnerThread2 [Thread 2] Runner2[Runner bundle #2] end subgraph RunnerThread3 [Thread 3] RunnerN[Runner bundle #N] end LightningStoreFacade[LightningStoreThreaded] BaseStore[Underlying LightningStore] end Algorithm -- async calls --> LightningStoreFacade Runner1 -- async calls --> LightningStoreFacade Runner2 -- async calls --> LightningStoreFacade RunnerN -- async calls --> LightningStoreFacade LightningStoreFacade -->|thread-safe delegates| BaseStore You can configure which role runs on the main thread. If the main thread runs the algorithm, it is able to spawn multiple runner threads. If it runs a runner, `n_runners` must be 1 and the runner lives on the main thread. ### Client-server Strategy[¶](https://microsoft.github.io/agent-lightning/stable/deep-dive/birds-eye-view/#client-server-strategy "Permanent link") [](https://microsoft.github.io/agent-lightning/stable/deep-dive/birds-eye-view/) [`ClientServerExecutionStrategy`](https://microsoft.github.io/agent-lightning/stable/reference/trainer/#agentlightning.ClientServerExecutionStrategy " agentlightning.ClientServerExecutionStrategy") splits concerns across processes. The algorithm bundle starts a [`LightningStoreServer`](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.LightningStoreServer " agentlightning.LightningStoreServer") (HTTP API) that wraps the underlying store. Runners connect via [`LightningStoreClient`](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.LightningStoreClient " agentlightning.LightningStoreClient") to call the same interface over REST. The server embeds a client to support algorithm-launched subprocesses (e.g., an LLM proxy worker) that need to talk back to the algorithm’s process through the same API. Currently this design introduces an extra wrapper in the Server side (as shown in the diagram), which helps debugging and improves fault tolerance. We might revisit this design in the future and enforce the client to be the only way to communicate with the store. flowchart TD subgraph Algorithm Process Group subgraph StoreServer[LightningStoreServer] StoreHttpClient[HTTP Client] StoreHttpServer[HTTP Server] StoreWrapper[LightningStore Wrapper] StoreHttpClient -- HTTP --> StoreHttpServer end subgraph Algorithm Bundle Algorithm[Algorithm Main Process] subgraph Another subprocess LLMProxy[LLM Proxy] end end LLMProxy -- async calls --> StoreHttpClient Algorithm -- async calls --> StoreWrapper end subgraph RunnerSide ["Runner Side"] subgraph Runner Process 1 Runner1[Runner bundle #1] Runner1 -- async calls --> LightningStoreClient1 LightningStoreClient1[LightningStoreClient] end subgraph Runner Process 2 Runner2[Runner bundle #2] Runner2 -- async calls --> LightningStoreClient2 LightningStoreClient2[LightningStoreClient] end subgraph Runner Process N RunnerN[Runner bundle #N] RunnerN -- async calls --> LightningStoreClientN LightningStoreClientN[LightningStoreClient] end end LocalStore[Underlying LightningStore] StoreHttpServer -->|delegates| StoreWrapper StoreWrapper -->|delegates| LocalStore LightningStoreClient1 -- HTTP --> StoreHttpServer LightningStoreClient2 -- HTTP --> StoreHttpServer LightningStoreClientN -- HTTP --> StoreHttpServer style RunnerSide fill:none; Online/Continuous Learning[¶](https://microsoft.github.io/agent-lightning/stable/deep-dive/birds-eye-view/#onlinecontinuous-learning "Permanent link") ------------------------------------------------------------------------------------------------------------------------------------------------------- Continuous learning keeps the algorithm loop running while runners report tasks and spans opportunistically. Key differences from batch mode: 1. The algorithm does not enqueue rollouts from a fixed dataset. Runners report tasks/rollouts and spans spontaneously. 2. The algorithm can wait for rollouts with a expected set of rollout IDs, but more often polls for new rollouts and spans or waits for a count to arrive. 3. The [`Runner`](https://microsoft.github.io/agent-lightning/stable/reference/runner/#agentlightning.Runner " agentlightning.Runner") processes one rollout at a time via [`step(task)`](https://microsoft.github.io/agent-lightning/stable/reference/runner/#agentlightning.Runner.step " step(input, *, resources=None, mode=None, event=None) async ") instead of exhausting a task queue. It notifies the store when starting a rollout so the store records it. 4. A user or higher-level loop controls which resources the next step uses and when to retry. [Spans](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.Span " agentlightning.Span") , [Adapter](https://microsoft.github.io/agent-lightning/stable/reference/algorithm/#agentlightning.Adapter " agentlightning.Adapter") implementations, and the [LLM Proxy](https://microsoft.github.io/agent-lightning/stable/reference/algorithm/#agentlightning.LLMProxy " agentlightning.LLMProxy") work the same way. sequenceDiagram autonumber actor User participant Runner participant Agent participant Store as LightningStore participant Algorithm Note over Algorithm: Algorithm is long-running and loops continuously loop Continuous Learning Loop activate User opt Decide what to do next User-->>Store: get_resources_by_id Store-->>User: Resources User-->>User: Prepare input for next step end User->>Runner: step(input, resources) activate Runner Runner-->>Store: Notify: start_rollout(input) Runner->>Agent: rollout(input, resources) Agent-->>Runner: add_span / reward spans Runner-->>Store: add_span or add_otel_span Runner-->>Store: update_attempt(status="finished") deactivate Runner deactivate User Algorithm->>Store: poll for new rollouts and spans opt If there is enough new data Store-->>Algorithm: new spans Algorithm->>Algorithm: adapt spans → learning signal Algorithm->>Store: update_resources end end Back to top --- # Understanding Store - Agent-lightning [Skip to content](https://microsoft.github.io/agent-lightning/stable/deep-dive/store/#understanding-store) ⚠️ You are viewing development documentation. For stable documentation, please visit [https://microsoft.github.io/agent-lightning/stable/](https://microsoft.github.io/agent-lightning/stable/) Understanding Store[¶](https://microsoft.github.io/agent-lightning/stable/deep-dive/store/#understanding-store "Permanent link") ================================================================================================================================= The **[`LightningStore`](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.LightningStore " agentlightning.LightningStore") ** is the central coordination point for Agent-lightning. It holds the task queue, rollouts, attempts, spans, and versioned resources, and exposes a small API both Runners and Algorithms use to communicate. This document explains what's in the store, how statuses transition, how spans are recorded, and the concurrency model (threads & processes). What's in the Store?[¶](https://microsoft.github.io/agent-lightning/stable/deep-dive/store/#whats-in-the-store "Permanent link") --------------------------------------------------------------------------------------------------------------------------------- ![Store Architecture](https://microsoft.github.io/agent-lightning/stable/assets/store-api-visualized.svg) At a high level: * **Task Queue** – [`enqueue_rollout`](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.LightningStore.enqueue_rollout " enqueue_rollout(input, mode=None, resources_id=None, config=None, metadata=None) async ") adds work; workers poll with [`dequeue_rollout`](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.LightningStore.dequeue_rollout " dequeue_rollout(worker_id=None) async ") . When a rollout is dequeued, it automatically creates a new attempt associated with itself. * **Rollouts** – A rollout is one unit of work. It has input, metadata, links to resources, and a lifecycle (`queuing → preparing → running → ...`). Valid [RolloutStatus](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.RolloutStatus " agentlightning.RolloutStatus = Literal['queuing', 'preparing', 'running', 'failed', 'succeeded', 'cancelled', 'requeuing'] module-attribute ") are **`queuing`**, `preparing`, `running`, `succeeded`, `failed`, **`requeuing`**, **`cancelled`**. For algorithms and runners, the rollout can be seen as a whole, without worrying about the internal attempts. * **Attempts** – Each rollout can have multiple executions (retries). Attempts track [`status`](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.Attempt.status " status = 'preparing' class-attribute instance-attribute ") , [`start_time`](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.Attempt.start_time " start_time instance-attribute ") , [`end_time`](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.Attempt.end_time " end_time = None class-attribute instance-attribute ") , [`last_heartbeat_time`](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.Attempt.last_heartbeat_time " last_heartbeat_time = None class-attribute instance-attribute ") and link to spans. Valid [AttemptStatus](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.AttemptStatus " agentlightning.AttemptStatus = Literal['preparing', 'running', 'failed', 'succeeded', 'unresponsive', 'timeout'] module-attribute ") are `preparing`, `running`, `succeeded`, `failed`, `requeuing`, `cancelled`. * **Spans** – Structured trace events produced by the Tracer during an attempt. Spans are ordered by a **monotonic sequence id** per `(rollout_id, attempt_id)`. * **Resources** – Versioned, named bundles (e.g., prompt templates) referenced by rollouts. * **Workers** – Metadata about runner instances: heartbeat timestamps, current assignment, and status. Rollout and Task share the same surface in practice: [`Rollout.input`](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.Rollout " agentlightning.Rollout") is the task input. The queue stores rollouts that are not yet running; [Runners](https://microsoft.github.io/agent-lightning/stable/reference/runner/#agentlightning.Runner " agentlightning.Runner") dequeue them and update the same rollout's status as work progresses. Before we look at status transitions, it helps to keep in mind that rollouts are the "outside view," while attempts are the "inside view." Attempts are what actually run; rollouts summarize the latest attempt plus a small set of control actions like queueing and cancellation. Attempt Status Transitions[¶](https://microsoft.github.io/agent-lightning/stable/deep-dive/store/#attempt-status-transitions "Permanent link") ----------------------------------------------------------------------------------------------------------------------------------------------- The status model is intentionally small and operationally clear. stateDiagram-v2 direction LR [*] --> preparing: Runner calls dequeue_rollout()
or start_rollout()
or start_attempt() preparing --> running: Runner calls
add_[otel_]span()
for the first time state c_runner <> state c_watch <> preparing --> c_runner: Runner calls
update_attempt(...) running --> c_runner: Runner calls
update_attempt(...) running --> c_watch: Watchdog checks preparing --> c_watch: Watchdog checks state "Client-set outcome" as Client { direction TB succeeded failed } state "Watchdog / policy" as Watch { direction TB timeout unresponsive } c_runner --> succeeded: update_attempt(status=succeeded) c_runner --> failed: update_attempt(status=failed) c_watch --> timeout: now - start_time > timeout_seconds c_watch --> unresponsive: now - last_heartbeat > unresponsive_seconds unresponsive --> running: Runner calls
add_[otel_]span() Each attempt begins in **preparing**, created either when a rollout is dequeued or explicitly started. It transitions to **running** the first time a span is recorded. From there, a few clear rules govern how it can change: * When the runner explicitly marks completion, the attempt becomes **succeeded** or **failed** (when the runner catches exception thrown out by the agent). * When the watchdog detects that the total elapsed time since start exceeds the configured limit, it marks the attempt as **timeout**. * If heartbeats stop arriving for too long, the watchdog marks it **unresponsive**. * A new span from the runner can immediately revive an **unresponsive** attempt back to **running**. What's a Watchdog? The watchdog enforces timing and liveness rules defined by each rollout’s [`RolloutConfig`](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.RolloutConfig " agentlightning.RolloutConfig") . It’s not a separate thread or service, but a function periodically invoked (e.g., before store mutations) to keep attempts healthy and consistent. This simple model allows the system to distinguish between normal termination, abnormal stalling, and recoverable interruption without additional state flags. Worker Telemetry[¶](https://microsoft.github.io/agent-lightning/stable/deep-dive/store/#worker-telemetry "Permanent link") --------------------------------------------------------------------------------------------------------------------------- Workers track runner-level activity timestamps (`last_heartbeat_time`, `last_dequeue_time`, `last_busy_time`, `last_idle_time`) plus their current rollout assignment. Those fields are now derived automatically: * [`dequeue_rollout(worker_id=...)`](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.LightningStore.dequeue_rollout " dequeue_rollout(worker_id=None) async ") records which worker polled the queue and refreshes `last_dequeue_time`. * [`update_attempt(..., worker_id=...)`](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.LightningStore.update_attempt " update_attempt(rollout_id, attempt_id, status=UNSET, worker_id=UNSET, last_heartbeat_time=UNSET, metadata=UNSET) async ") drives the worker status machine. Assigning an attempt marks the worker **busy** and stamps `last_busy_time`; finishing with `status in {"succeeded","failed"}` switches to **idle**, while watchdog transitions such as `timeout`/`unresponsive` make the worker **unknown** and clear `current_rollout_id` / `current_attempt_id`. * [`update_worker(...)`](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.LightningStore.update_worker " update_worker(worker_id, heartbeat_stats=UNSET) async ") is reserved for heartbeats. It snapshots optional `heartbeat_stats` and always updates `last_heartbeat_time`. Because every transition flows through these APIs, worker status is derived automatically from rollout execution and heartbeats. Note, however, that calling `update_worker` with a new `worker_id` will create a new worker record with status "unknown" if one does not exist. Thus, while manual status changes are not allowed, new worker records can be created externally via heartbeats. Rollout Transition Map[¶](https://microsoft.github.io/agent-lightning/stable/deep-dive/store/#rollout-transition-map "Permanent link") --------------------------------------------------------------------------------------------------------------------------------------- Rollout status is an **aggregated view** of its latest attempt’s status, with additional transitions for queueing and explicit cancellation. A rollout’s retry behavior is controlled by [`Rollout.config`](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.Rollout " agentlightning.Rollout") (a [`RolloutConfig`](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.RolloutConfig " agentlightning.RolloutConfig") ). The key fields are: * [`timeout_seconds`](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.RolloutConfig.timeout_seconds " timeout_seconds = None class-attribute instance-attribute ") – maximum wall-clock time for an attempt before it is marked `timeout`. * [`unresponsive_seconds`](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.RolloutConfig.unresponsive_seconds " unresponsive_seconds = None class-attribute instance-attribute ") – maximum silence between heartbeats before an attempt is marked `unresponsive`. * [`max_attempts`](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.RolloutConfig.max_attempts " max_attempts = Field(default=1, ge=1) class-attribute instance-attribute ") – total number of attempts allowed for the rollout (including the first). * [`retry_condition`](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.RolloutConfig.retry_condition " retry_condition = Field(default_factory=(cast(Callable[[], List[AttemptStatus]], list))) class-attribute instance-attribute ") – which attempt terminal statuses should trigger a retry (e.g., `["failed", "timeout", "unresponsive"]`). **How it plays out:** The runner works on attempt `k`. If the attempt ends in a status that is listed in `retry_condition`, and `k < max_attempts`, the rollout moves to **requeuing** and the store creates attempt `k+1`. Otherwise, the rollout becomes **failed** (or **succeeded** if the runner marked it so). `timeout_seconds` and `unresponsive_seconds` are enforced by the watchdog and feed into the same decision flow. A minimal example of how to use `RolloutConfig`: `[](https://microsoft.github.io/agent-lightning/stable/deep-dive/store/#__codelineno-0-1) from agentlightning import RolloutConfig [](https://microsoft.github.io/agent-lightning/stable/deep-dive/store/#__codelineno-0-2) [](https://microsoft.github.io/agent-lightning/stable/deep-dive/store/#__codelineno-0-3) # Retry on explicit failures or timeouts, up to 3 attempts in total. [](https://microsoft.github.io/agent-lightning/stable/deep-dive/store/#__codelineno-0-4) cfg = RolloutConfig( [](https://microsoft.github.io/agent-lightning/stable/deep-dive/store/#__codelineno-0-5) timeout_seconds=600, [](https://microsoft.github.io/agent-lightning/stable/deep-dive/store/#__codelineno-0-6) unresponsive_seconds=120, [](https://microsoft.github.io/agent-lightning/stable/deep-dive/store/#__codelineno-0-7) max_attempts=3, [](https://microsoft.github.io/agent-lightning/stable/deep-dive/store/#__codelineno-0-8) retry_condition=["failed", "timeout"] [](https://microsoft.github.io/agent-lightning/stable/deep-dive/store/#__codelineno-0-9) ) [](https://microsoft.github.io/agent-lightning/stable/deep-dive/store/#__codelineno-0-10) [](https://microsoft.github.io/agent-lightning/stable/deep-dive/store/#__codelineno-0-11) # When creating/enqueuing a rollout, attach this config. [](https://microsoft.github.io/agent-lightning/stable/deep-dive/store/#__codelineno-0-12) # The store will propagate attempt outcomes according to cfg. [](https://microsoft.github.io/agent-lightning/stable/deep-dive/store/#__codelineno-0-13) rollout = await store.enqueue_rollout(input, config=cfg)` | Latest attempt status | Rollout transition | Notes / guards | | --- | --- | --- | | N/A | `queuing` | Created by `enqueue_rollout()`. | | `preparing` | `queuing/requeuing` → `preparing` | Typically `dequeue_rollout()` or `start_rollout()`/`start_attempt()` creates a new attempt. | | `running` | `preparing/queuing/requeuing` → `running` | First `add_[otel_]span()` flips the attempt to `running`; rollout follows via `rollout_status_from_attempt`. | | `succeeded` | `*` → `succeeded` | Terminal. Rollout `end_time` set. | | `failed` / `timeout` / `unresponsive` | `*` → `requeuing` | **Only if** `status ∈ retry_condition ∧ sequence_id < max_attempts`. | | `failed` / `timeout` / `unresponsive` | `*` → `failed` | Otherwise (no retries left or retries disabled). | | `*` | `*` → `cancelled` | Explicitly set by `update_rollout(status=cancelled)`. | Why aggregation? In code, we use `rollout_status_from_attempt()` which actively updates the rollout based on the latest attempt. Reading the table above is usually easier than reverse-engineering the propagation logic in the code: think of the rollout’s transitions as _callbacks_ on attempt state changes, plus queue/cancel paths. Spans[¶](https://microsoft.github.io/agent-lightning/stable/deep-dive/store/#spans "Permanent link") ----------------------------------------------------------------------------------------------------- Every traceable operation in a rollout is stored as a [Span](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.Span " agentlightning.Span") . Spans not only capture fine-grained instrumentation but also act as periodic heartbeats that demonstrate liveness. The first span marks activation; each subsequent one both extends the trace and refreshes the attempt’s [`last_heartbeat_time`](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.Attempt.last_heartbeat_time " last_heartbeat_time = None class-attribute instance-attribute ") . If no span arrives within the configured [`unresponsive_seconds`](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.RolloutConfig.unresponsive_seconds " unresponsive_seconds = None class-attribute instance-attribute ") , the watchdog downgrades the attempt to **unresponsive** until activity resumes. Spans are indexed by `(rollout_id, attempt_id, sequence_id)` where the sequence is monotonic. Tracing analysis tools like [Adapter](https://microsoft.github.io/agent-lightning/stable/reference/algorithm/#agentlightning.Adapter " agentlightning.Adapter") usually rely on "time order" to reconstruct the trace. However, in a distributed system, the recorded start time and end time of a span are not necessarily in the right order when they aggregated into a central store. Therefore, we enforce every span creation to retrieve a monotonically increasing [`sequence_id`](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.Span.sequence_id " sequence_id instance-attribute ") from the store before adding the span. Note In practice, one `sequence_id` can be used to create multiple spans. In that case, the orders between the multiple spans are determined by the order of `start_time` and `end_time` of the spans. ### OpenTelemetry conversion[¶](https://microsoft.github.io/agent-lightning/stable/deep-dive/store/#opentelemetry-conversion "Permanent link") Runners often produce [OpenTelemetry `ReadableSpan`](https://opentelemetry.io/docs/concepts/signals/traces/#attributes) objects directly. The store normalizes them into [`Span`](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.Span " agentlightning.Span") as follows: 1. The runner first requests [`get_next_span_sequence_id`](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.LightningStore.get_next_span_sequence_id " get_next_span_sequence_id(rollout_id, attempt_id) async ") via `sequence_id = await store.get_next_span_sequence_id(rollout_id, attempt_id)`. This guarantees ordering within the attempt regardless of clock skew. 2. `trace_id`, `span_id`, `parent_id`, `name`, `status`, timestamps, attributes, events, links, and resource are copied from the OTEL span. Timestamps are auto-normalized to seconds (nanoseconds are converted). 3. OTEL `SpanContext` and parent context are preserved so downstream tools can correlate traces across systems. 4. Any additional serializable fields present on the `ReadableSpan` are retained in the stored span (after safe JSON serialization), which keeps the representation forward-compatible. Programmatically this is encapsulated by [`Span.from_opentelemetry(readable_span, rollout_id, attempt_id, sequence_id)`](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.Span.from_opentelemetry " from_opentelemetry(src, rollout_id, attempt_id, sequence_id) classmethod ") ; [`store.add_otel_span(...)`](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.LightningStore.add_otel_span " add_otel_span(rollout_id, attempt_id, readable_span, sequence_id=None) async ") simply wraps the fetch-then-add flow. The end result is a store span that is stable to sort, merge, and query, while still preserving the richness of the original OTEL payload. Tip [`add_span`](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.LightningStore.add_span " add_span(span) async ") or [`add_otel_span`](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.LightningStore.add_otel_span " add_otel_span(rollout_id, attempt_id, readable_span, sequence_id=None) async ") both appends a span _and_ acts as a heartbeat that can revive `unresponsive` → `running`. ### OTLP Compatibility[¶](https://microsoft.github.io/agent-lightning/stable/deep-dive/store/#otlp-compatibility "Permanent link") Some of the LightningStore implementations support exporting traces via the [OTLP/HTTP specification](https://opentelemetry.io/docs/specs/otlp/) . For example, [`LightningStoreServer`](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.LightningStoreServer " agentlightning.LightningStoreServer") exposes `/v1/traces` endpoint, it implements the binary Protobuf variant defined by the spec, including the required `Content-Type: application/x-protobuf`, optional `Content-Encoding: gzip`, and status responses encoded as `google.rpc.Status`. Agent-lightning helps parsing `ExportTraceServiceRequest` messages, validate identifiers, normalize resource metadata, and allocate sequence numbers so store implementations only need to persist [`Span`](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.Span " agentlightning.Span") objects in order. Because the interface speaks standard OTLP, any OpenTelemetry-compatible SDK or collector can emit spans directly to a LightningStore OTLP endpoint without custom shims. The server responds according to the OTLP contract (status code, encoding, and error payloads), which keeps Agent-lightning interoperable with existing observability tooling. This compatibility serves as a strong complement to the OpenTelemetry conversion discussed above. Check whether the store supports OTLP traces via the [`capabilities["otlp_traces"]`](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.LightningStore.capabilities " capabilities property ") property. Implementation Overview[¶](https://microsoft.github.io/agent-lightning/stable/deep-dive/store/#implementation-overview "Permanent link") ----------------------------------------------------------------------------------------------------------------------------------------- The `agentlightning.store` module is organized into two distinct layers plus optional wrappers: classDiagram direction TB class LightningStore { <> +enqueue_rollout() +dequeue_rollout() +update_attempt() +add_span() +query_rollouts() ... } class LightningCollections { <> +rollouts: Collection +attempts: Collection +spans: Collection +resources: Collection +workers: Collection +rollout_queue: Queue +span_sequence_ids: KeyValue +atomic() } class CollectionBasedLightningStore~T~ { +collections: T -healthcheck_before() -tracked() } class InMemoryLightningStore class MongoLightningStore class InMemoryLightningCollections class MongoLightningCollections class LightningStoreServer { +store: LightningStore +start() +stop() } class LightningStoreClient { +server_address: str } class LightningStoreThreaded { +store: LightningStore } LightningStore <|-- CollectionBasedLightningStore LightningStore <|-- LightningStoreServer LightningStore <|-- LightningStoreClient LightningStore <|-- LightningStoreThreaded CollectionBasedLightningStore <|-- InMemoryLightningStore CollectionBasedLightningStore <|-- MongoLightningStore LightningCollections <|-- InMemoryLightningCollections LightningCollections <|-- MongoLightningCollections InMemoryLightningStore ..> InMemoryLightningCollections : uses MongoLightningStore ..> MongoLightningCollections : uses LightningStoreServer o-- LightningStore : wraps LightningStoreThreaded o-- LightningStore : wraps 1. **Collections Layer** – Low-level storage primitives ([`LightningCollections`](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.store.collection.LightningCollections " agentlightning.store.collection.LightningCollections") ) providing CRUD operations via [`Collection`](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.store.collection.Collection " agentlightning.store.collection.Collection") , [`Queue`](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.store.collection.Queue " agentlightning.store.collection.Queue") , and [`KeyValue`](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.store.collection.KeyValue " agentlightning.store.collection.KeyValue") interfaces. Each backend (in-memory, MongoDB) implements these primitives. 2. **Store Layer** – All [`LightningStore`](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.LightningStore " agentlightning.LightningStore") implementations must inherit from [`LightningStore`](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.LightningStore " agentlightning.LightningStore") and override the methods to implement the storage logic. [`CollectionBasedLightningStore`](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.CollectionBasedLightningStore " agentlightning.CollectionBasedLightningStore") builds on collections to implement the full [`LightningStore`](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.LightningStore " agentlightning.LightningStore") API, including business logic like status transitions, watchdog health checks, and retry policies. 3. **Wrappers** – Cross-cutting concerns live in thin wrappers: * [`LightningStoreThreaded`](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.LightningStoreThreaded " agentlightning.LightningStoreThreaded") adds mutex-based thread safety. * [`LightningStoreServer`](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.LightningStoreServer " agentlightning.LightningStoreServer") / [`LightningStoreClient`](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.LightningStoreClient " agentlightning.LightningStoreClient") enable multi-process access over HTTP. Collections[¶](https://microsoft.github.io/agent-lightning/stable/deep-dive/store/#collections "Permanent link") ----------------------------------------------------------------------------------------------------------------- The collections layer provides storage primitives that [`CollectionBasedLightningStore`](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.CollectionBasedLightningStore " agentlightning.CollectionBasedLightningStore") builds upon. This separation keeps business logic (status transitions, watchdog, retries) in the store layer while allowing different backends to focus purely on persistence. The off-the-shelf implementations are [`InMemoryLightningCollections`](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.store.collection.InMemoryLightningCollections " agentlightning.store.collection.InMemoryLightningCollections") and [`MongoLightningCollections`](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.store.collection.mongo.MongoLightningCollections " agentlightning.store.collection.mongo.MongoLightningCollections") , which are the underlying collections for [`InMemoryLightningStore`](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.InMemoryLightningStore " agentlightning.InMemoryLightningStore") and [`MongoLightningStore`](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.store.mongo.MongoLightningStore " agentlightning.store.mongo.MongoLightningStore") , respectively. ### Collection Primitives[¶](https://microsoft.github.io/agent-lightning/stable/deep-dive/store/#collection-primitives "Permanent link") [`LightningCollections`](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.store.collection.LightningCollections " agentlightning.store.collection.LightningCollections") bundles three primitive types: | Primitive | Purpose | Methods | | --- | --- | --- | | [`Collection[T]`](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.store.collection.Collection " agentlightning.store.collection.Collection") | Indexed storage with primary keys | [`query()`](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.store.collection.Collection.query " query(filter=None, sort=None, limit=-1, offset=0)


async
")
, [`get()`](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.store.collection.Collection.get " get(filter=None, sort=None)


async
")
, [`insert()`](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.store.collection.Collection.insert " insert(items)


async
")
, [`update()`](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.store.collection.Collection.update " update(items, update_fields=None)


async
")
, [`upsert()`](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.store.collection.Collection.upsert " upsert(items, update_fields=None)


async
")
, [`delete()`](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.store.collection.Collection.delete " delete(items)


async
") | | [`Queue[T]`](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.store.collection.Queue " agentlightning.store.collection.Queue") | FIFO queue for task scheduling | [`enqueue()`](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.store.collection.Queue.enqueue " enqueue(items)


async
")
, [`dequeue()`](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.store.collection.Queue.dequeue " dequeue(limit=1)


async
")
, [`peek()`](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.store.collection.Queue.peek " peek(limit=1)


async
")
, [`size()`](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.store.collection.Queue.size " size()


async
") | | [`KeyValue[K, V]`](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.store.collection.KeyValue " agentlightning.store.collection.KeyValue") | Simple key-value store | [`get()`](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.store.collection.KeyValue.get " get(key, default=None)


async
")
, [`set()`](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.store.collection.KeyValue.set " set(key, value)


async
")
, [`inc()`](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.store.collection.KeyValue.inc " inc(key, amount)


async
")
, [`chmax()`](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.store.collection.KeyValue.chmax " chmax(key, value)


async
")
, [`pop()`](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.store.collection.KeyValue.pop " pop(key, default=None)


async
") | Every [`LightningCollections`](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.store.collection.LightningCollections " agentlightning.store.collection.LightningCollections") instance exposes these named collections: * `rollouts` – [`Collection[Rollout]`](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.store.collection.Collection " agentlightning.store.collection.Collection") keyed by `rollout_id` * `attempts` – [`Collection[Attempt]`](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.store.collection.Collection " agentlightning.store.collection.Collection") keyed by `(rollout_id, attempt_id)` * `spans` – [`Collection[Span]`](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.store.collection.Collection " agentlightning.store.collection.Collection") keyed by `(rollout_id, attempt_id, span_id)` * `resources` – [`Collection[ResourcesUpdate]`](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.store.collection.Collection " agentlightning.store.collection.Collection") keyed by `resources_id` * `workers` – [`Collection[Worker]`](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.store.collection.Collection " agentlightning.store.collection.Collection") keyed by `worker_id` * `rollout_queue` – [`Queue[str]`](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.store.collection.Queue " agentlightning.store.collection.Queue") holding rollout IDs awaiting execution * `span_sequence_ids` – [`KeyValue[str, int]`](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.store.collection.KeyValue " agentlightning.store.collection.KeyValue") tracking monotonic sequence counters ### Atomic Operations[¶](https://microsoft.github.io/agent-lightning/stable/deep-dive/store/#atomic-operations "Permanent link") Collections support atomic operations through the [`atomic()`](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.store.collection.LightningCollections.atomic " atomic(*, mode='rw', snapshot=False, commit=False, labels=None, **kwargs)") context manager: `[](https://microsoft.github.io/agent-lightning/stable/deep-dive/store/#__codelineno-1-1) async with collections.atomic(mode="rw", labels=["rollouts", "attempts"]) as ctx: [](https://microsoft.github.io/agent-lightning/stable/deep-dive/store/#__codelineno-1-2) rollout = await ctx.rollouts.get(filter={"rollout_id": {"exact": rollout_id}}) [](https://microsoft.github.io/agent-lightning/stable/deep-dive/store/#__codelineno-1-3) # modify and update within the same transaction [](https://microsoft.github.io/agent-lightning/stable/deep-dive/store/#__codelineno-1-4) await ctx.rollouts.update([updated_rollout])` The arguments passed to [`atomic()`](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.store.collection.LightningCollections.atomic " atomic(*, mode='rw', snapshot=False, commit=False, labels=None, **kwargs)") are quite arbitrary and flexible. Different implementations may have different interpretations of the arguments. For example, to [`InMemoryLightningCollections`](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.store.collection.InMemoryLightningCollections " agentlightning.store.collection.InMemoryLightningCollections") , the `mode` parameter controls locking behavior (`"r"` for read-only, `"rw"` for read-write), while `labels` specifies which collections to lock. Acquiring locks in sorted order prevents deadlocks when multiple operations run concurrently. ### Implementing a Custom Backend[¶](https://microsoft.github.io/agent-lightning/stable/deep-dive/store/#implementing-a-custom-backend "Permanent link") To add a new storage backend, implement [`LightningCollections`](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.store.collection.LightningCollections " agentlightning.store.collection.LightningCollections") : `[](https://microsoft.github.io/agent-lightning/stable/deep-dive/store/#__codelineno-2-1) from agentlightning.store.collection import LightningCollections, Collection, Queue, KeyValue [](https://microsoft.github.io/agent-lightning/stable/deep-dive/store/#__codelineno-2-2) [](https://microsoft.github.io/agent-lightning/stable/deep-dive/store/#__codelineno-2-3) class MyLightningCollections(LightningCollections): [](https://microsoft.github.io/agent-lightning/stable/deep-dive/store/#__codelineno-2-4) @property [](https://microsoft.github.io/agent-lightning/stable/deep-dive/store/#__codelineno-2-5) def rollouts(self) -> Collection[Rollout]: [](https://microsoft.github.io/agent-lightning/stable/deep-dive/store/#__codelineno-2-6) return self._rollouts # your implementation [](https://microsoft.github.io/agent-lightning/stable/deep-dive/store/#__codelineno-2-7) [](https://microsoft.github.io/agent-lightning/stable/deep-dive/store/#__codelineno-2-8) @property [](https://microsoft.github.io/agent-lightning/stable/deep-dive/store/#__codelineno-2-9) def rollout_queue(self) -> Queue[str]: [](https://microsoft.github.io/agent-lightning/stable/deep-dive/store/#__codelineno-2-10) return self._queue # your implementation [](https://microsoft.github.io/agent-lightning/stable/deep-dive/store/#__codelineno-2-11) [](https://microsoft.github.io/agent-lightning/stable/deep-dive/store/#__codelineno-2-12) # ... implement remaining properties [](https://microsoft.github.io/agent-lightning/stable/deep-dive/store/#__codelineno-2-13) [](https://microsoft.github.io/agent-lightning/stable/deep-dive/store/#__codelineno-2-14) async def atomic(self, *, mode, snapshot=False, labels=None, **kwargs): [](https://microsoft.github.io/agent-lightning/stable/deep-dive/store/#__codelineno-2-15) # provide transaction / locking semantics [](https://microsoft.github.io/agent-lightning/stable/deep-dive/store/#__codelineno-2-16) ...` Then instantiate your store: `[](https://microsoft.github.io/agent-lightning/stable/deep-dive/store/#__codelineno-3-1) from agentlightning.store.collection_based import CollectionBasedLightningStore [](https://microsoft.github.io/agent-lightning/stable/deep-dive/store/#__codelineno-3-2) [](https://microsoft.github.io/agent-lightning/stable/deep-dive/store/#__codelineno-3-3) store = CollectionBasedLightningStore(collections=MyLightningCollections())` The store layer handles all business logic; your collections just need to provide correct CRUD semantics. Collection-based Store Implementations[¶](https://microsoft.github.io/agent-lightning/stable/deep-dive/store/#collection-based-store-implementations "Permanent link") ----------------------------------------------------------------------------------------------------------------------------------------------------------------------- Agent-lightning ships with two collection-based store implementations: ### InMemoryLightningStore[¶](https://microsoft.github.io/agent-lightning/stable/deep-dive/store/#inmemorylightningstore "Permanent link") [`InMemoryLightningStore`](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.InMemoryLightningStore " agentlightning.InMemoryLightningStore") uses [`InMemoryLightningCollections`](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.store.collection.InMemoryLightningCollections " agentlightning.store.collection.InMemoryLightningCollections") backed by Python data structures. It supports **fast startup** with zero external dependencies—ideal for local development, CI, and unit tests. It also provides two lock modes, configurable between `"asyncio"` (single-thread, multiple coroutines) and `"thread"` (multi-threaded via [aiologic](https://github.com/x42005e1f/aiologic) ). [`InMemoryLightningCollections`](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.store.collection.InMemoryLightningCollections " agentlightning.store.collection.InMemoryLightningCollections") use nested dictionaries for O(1) primary-key lookup and `deque` for the task queue. ### MongoLightningStore[¶](https://microsoft.github.io/agent-lightning/stable/deep-dive/store/#mongolightningstore "Permanent link") [`MongoLightningStore`](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.store.mongo.MongoLightningStore " agentlightning.store.mongo.MongoLightningStore") uses [`MongoLightningCollections`](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.store.collection.mongo.MongoLightningCollections " agentlightning.store.collection.mongo.MongoLightningCollections") backed by MongoDB. It supports **persistent storage** suitable for production deployments and **multi-process safe** via database-level atomicity. It also supports **partition support** via `partition_id` for running multiple trainers against the same database. `[](https://microsoft.github.io/agent-lightning/stable/deep-dive/store/#__codelineno-4-1) from agentlightning.store.mongo import MongoLightningStore [](https://microsoft.github.io/agent-lightning/stable/deep-dive/store/#__codelineno-4-2) [](https://microsoft.github.io/agent-lightning/stable/deep-dive/store/#__codelineno-4-3) store = MongoLightningStore( [](https://microsoft.github.io/agent-lightning/stable/deep-dive/store/#__codelineno-4-4) mongo_uri="mongodb://localhost:27017/?replicaSet=rs0", [](https://microsoft.github.io/agent-lightning/stable/deep-dive/store/#__codelineno-4-5) database_name="agentlightning", [](https://microsoft.github.io/agent-lightning/stable/deep-dive/store/#__codelineno-4-6) partition_id="trainer-1", # optional: isolate data per trainer [](https://microsoft.github.io/agent-lightning/stable/deep-dive/store/#__codelineno-4-7) )` Note [`MongoLightningStore`](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.store.mongo.MongoLightningStore " agentlightning.store.mongo.MongoLightningStore") requires the `mongo` optional dependency. Install with `pip install agentlightning[mongo]`. ### Capabilities[¶](https://microsoft.github.io/agent-lightning/stable/deep-dive/store/#capabilities "Permanent link") [](https://microsoft.github.io/agent-lightning/stable/deep-dive/store/) Different stores have different capabilities. Check the [`capabilities`](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.LightningStore.capabilities " capabilities property ") property to understand what a store supports: | Capability | Description | InMemory | Mongo | Server | Client | | --- | --- | --- | --- | --- | --- | | `thread_safe` | Safe for concurrent access from multiple threads | configurable | ✓ | ✓ | ✓ | | `async_safe` | Safe for concurrent access from multiple coroutines | ✓ | ✓ | ✓ | ✓ | | `zero_copy` | Can be shared across processes without serialization | ✗ | ✓ | ✓ | ✓ | | `otlp_traces` | Exposes an OTLP-compatible `/v1/traces` endpoint | ✗ | ✗ | ✓ | ✓ | Thread Safety[¶](https://microsoft.github.io/agent-lightning/stable/deep-dive/store/#thread-safety "Permanent link") --------------------------------------------------------------------------------------------------------------------- Thread safety can be achieved at different layers: **At the collections layer**: [`InMemoryLightningCollections`](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.store.collection.InMemoryLightningCollections " agentlightning.store.collection.InMemoryLightningCollections") accepts a `lock_type` parameter: * `"asyncio"` – Uses per-event-loop `asyncio.Lock` for single-threaded, multi-coroutine scenarios. * `"thread"` – Uses `aiologic.Lock` for true multi-threaded access. **At the store layer**: [`LightningStoreThreaded`](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.LightningStoreThreaded " agentlightning.LightningStoreThreaded") wraps any [`LightningStore`](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.LightningStore " agentlightning.LightningStore") to add mutex-based thread safety: * Methods like [`start_rollout`](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.LightningStore.start_rollout " start_rollout(input, mode=None, resources_id=None, config=None, metadata=None, worker_id=None) async ") , [`enqueue_rollout`](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.LightningStore.enqueue_rollout " enqueue_rollout(input, mode=None, resources_id=None, config=None, metadata=None) async ") , [`update_attempt`](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.LightningStore.update_attempt " update_attempt(rollout_id, attempt_id, status=UNSET, worker_id=UNSET, last_heartbeat_time=UNSET, metadata=UNSET) async ") , [`add_span`](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.LightningStore.add_span " add_span(span) async ") , etc. are guarded by a lock. * Non-mutating, potentially blocking calls remain pass-through by design (e.g., [`wait_for_rollouts`](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.LightningStore.wait_for_rollouts " wait_for_rollouts(*, rollout_ids, timeout=None) async ") ), as they don't modify shared state and should not hold the lock for long periods. Database-based stores like [`MongoLightningStore`](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.store.mongo.MongoLightningStore " agentlightning.store.mongo.MongoLightningStore") are inherently thread-safe through database atomicity guarantees. Process Safety and Client-server Store[¶](https://microsoft.github.io/agent-lightning/stable/deep-dive/store/#process-safety-and-client-server-store "Permanent link") ----------------------------------------------------------------------------------------------------------------------------------------------------------------------- **[`LightningStoreServer`](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.LightningStoreServer " agentlightning.LightningStoreServer") ** wraps another underlying store and runs a FastAPI app to expose the store API over HTTP. [`LightningStoreClient`](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.LightningStoreClient " agentlightning.LightningStoreClient") is a small [`LightningStore`](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.LightningStore " agentlightning.LightningStore") implementation that talks to the HTTP API. Warning The server HTTP API is not considered a stable API at this moment. Users are encouraged to use the [`LightningStoreClient`](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.LightningStoreClient " agentlightning.LightningStoreClient") to communicate with the server as a stable interface. The server tracks the creator PID. In the owner process it delegates directly to the in-memory store; in other processes it lazily constructs a [`LightningStoreClient`](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.LightningStoreClient " agentlightning.LightningStoreClient") to talk to the HTTP API. This prevents accidental cross-process mutation of the wrong memory image. When the server is pickled (e.g., via `multiprocessing`), only the minimal fields are serialized, but **NOT** the FastAPI/uvicorn objects. Subprocesses won’t accidentally carry live server state. Forked subprocess should also use [`LightningStoreClient`](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.LightningStoreClient " agentlightning.LightningStoreClient") to communicate with the server in the main process. On the client side, the client retries network/5xx failures using a small backoff, and probes `/v1/agl/health` between attempts. Application exceptions inside the server are wrapped as HTTP 400 with a traceback—these are **not retried**. The client also maintains a **per-event-loop** `aiohttp.ClientSession` map so that tracer callbacks (often on separate loops/threads) don’t hang by reusing a session from another loop. Minimal lifecycle: `[](https://microsoft.github.io/agent-lightning/stable/deep-dive/store/#__codelineno-5-1) import agentlightning as agl [](https://microsoft.github.io/agent-lightning/stable/deep-dive/store/#__codelineno-5-2) [](https://microsoft.github.io/agent-lightning/stable/deep-dive/store/#__codelineno-5-3) # Server (owner process) [](https://microsoft.github.io/agent-lightning/stable/deep-dive/store/#__codelineno-5-4) in_memory_store = agl.InMemoryLightningStore() [](https://microsoft.github.io/agent-lightning/stable/deep-dive/store/#__codelineno-5-5) server = agl.LightningStoreServer(store=in_memory_store, host="0.0.0.0", port=4747) [](https://microsoft.github.io/agent-lightning/stable/deep-dive/store/#__codelineno-5-6) await server.start() # starts uvicorn in a daemon thread and waits for /health [](https://microsoft.github.io/agent-lightning/stable/deep-dive/store/#__codelineno-5-7) # or keep your own event loop and stop via await server.stop() [](https://microsoft.github.io/agent-lightning/stable/deep-dive/store/#__codelineno-5-8) # await server.run_forever() [](https://microsoft.github.io/agent-lightning/stable/deep-dive/store/#__codelineno-5-9) [](https://microsoft.github.io/agent-lightning/stable/deep-dive/store/#__codelineno-5-10) # Client (same or different process) [](https://microsoft.github.io/agent-lightning/stable/deep-dive/store/#__codelineno-5-11) client = agl.LightningStoreClient("http://localhost:4747") [](https://microsoft.github.io/agent-lightning/stable/deep-dive/store/#__codelineno-5-12) [](https://microsoft.github.io/agent-lightning/stable/deep-dive/store/#__codelineno-5-13) print(await client.query_rollouts(status_in=["queuing"])) [](https://microsoft.github.io/agent-lightning/stable/deep-dive/store/#__codelineno-5-14) [](https://microsoft.github.io/agent-lightning/stable/deep-dive/store/#__codelineno-5-15) await client.close() [](https://microsoft.github.io/agent-lightning/stable/deep-dive/store/#__codelineno-5-16) await server.stop()` Another approach is to use a dedicated command line to start a long running server process, possibly sharable across multiple processes. In the main process, you can always use [`LightningStoreClient`](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.LightningStoreClient " agentlightning.LightningStoreClient") to communicate with the server. `[](https://microsoft.github.io/agent-lightning/stable/deep-dive/store/#__codelineno-6-1) agl store --port 4747` Note [`LightningStoreClient.wait_for_rollouts`](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.LightningStoreClient.wait_for_rollouts " wait_for_rollouts(*, rollout_ids, timeout=None) async ") intentionally enforces a tiny timeout (≤ 0.1s) to avoid blocking event loops. Poll with short timeouts or compose with `asyncio.wait_for` at a higher layer. Back to top --- # Agent - Agent-lightning [Skip to content](https://microsoft.github.io/agent-lightning/stable/reference/agent/#agent-developer-apis) ⚠️ You are viewing development documentation. For stable documentation, please visit [https://microsoft.github.io/agent-lightning/stable/](https://microsoft.github.io/agent-lightning/stable/) Agent Developer APIs[¶](https://microsoft.github.io/agent-lightning/stable/reference/agent/#agent-developer-apis "Permanent link") =================================================================================================================================== Agent Decorators[¶](https://microsoft.github.io/agent-lightning/stable/reference/agent/#agent-decorators "Permanent link") --------------------------------------------------------------------------------------------------------------------------- Tip These are convenient helpers for creating agents from functions. First-time users are recommended to use these decorators to create agents. ### `agentlightning.rollout(func)` [¶](https://microsoft.github.io/agent-lightning/stable/reference/agent/#agentlightning.rollout "Permanent link") Create a [`FunctionalLitAgent`](https://microsoft.github.io/agent-lightning/stable/reference/internal/#agentlightning.litagent.decorator.FunctionalLitAgent " agentlightning.litagent.decorator.FunctionalLitAgent") from an arbitrary rollout function. This function inspects the provided callable and creates the appropriate agent type based on its signature. It supports both LLM-based and prompt-template-based agents. The returned agent instance is callable, preserving the original function's behavior and type hints. See [`llm_rollout`](https://microsoft.github.io/agent-lightning/stable/reference/internal/#agentlightning.litagent.decorator.llm_rollout " agentlightning.litagent.decorator.llm_rollout(func=None, *, strip_proxy=True)") and [`prompt_rollout`](https://microsoft.github.io/agent-lightning/stable/reference/internal/#agentlightning.litagent.decorator.prompt_rollout " agentlightning.litagent.decorator.prompt_rollout(func=None)") for more details. Parameters: * **`func`** (`Union[LlmRolloutFunc[T], PromptRolloutFunc[T], Callable[..., Any]]`) – Callable that implements the rollout. Supported signatures: * `[async ](task, llm[, rollout])` for LLM-based agents * `[async ](task, prompt_template[, rollout])` for prompt-template-based agents The supported output types of `func` is same as the return type of [`rollout`](https://microsoft.github.io/agent-lightning/stable/reference/agent/#agentlightning.LitAgent.rollout " rollout(task, resources, rollout)") . Returns: * `[FunctionalLitAgent](https://microsoft.github.io/agent-lightning/stable/reference/internal/#agentlightning.litagent.decorator.FunctionalLitAgent " agentlightning.litagent.decorator.FunctionalLitAgent") [T]` – [`FunctionalLitAgent`](https://microsoft.github.io/agent-lightning/stable/reference/internal/#agentlightning.litagent.decorator.FunctionalLitAgent " agentlightning.litagent.decorator.FunctionalLitAgent") that * `[FunctionalLitAgent](https://microsoft.github.io/agent-lightning/stable/reference/internal/#agentlightning.litagent.decorator.FunctionalLitAgent " agentlightning.litagent.decorator.FunctionalLitAgent") [T]` – wraps the supplied function. Examples: `[](https://microsoft.github.io/agent-lightning/stable/reference/agent/#__codelineno-0-1) # LLM-based agent [](https://microsoft.github.io/agent-lightning/stable/reference/agent/#__codelineno-0-2) @rollout [](https://microsoft.github.io/agent-lightning/stable/reference/agent/#__codelineno-0-3) def my_llm_agent(task, llm): [](https://microsoft.github.io/agent-lightning/stable/reference/agent/#__codelineno-0-4) client = OpenAI(base_url=llm.endpoint) [](https://microsoft.github.io/agent-lightning/stable/reference/agent/#__codelineno-0-5) response = client.chat.completions.create( [](https://microsoft.github.io/agent-lightning/stable/reference/agent/#__codelineno-0-6) model=llm.model, [](https://microsoft.github.io/agent-lightning/stable/reference/agent/#__codelineno-0-7) messages=[{"role": "user", "content": task.input}], [](https://microsoft.github.io/agent-lightning/stable/reference/agent/#__codelineno-0-8) ) [](https://microsoft.github.io/agent-lightning/stable/reference/agent/#__codelineno-0-9) return response [](https://microsoft.github.io/agent-lightning/stable/reference/agent/#__codelineno-0-10) [](https://microsoft.github.io/agent-lightning/stable/reference/agent/#__codelineno-0-11) # Prompt-template-based agent [](https://microsoft.github.io/agent-lightning/stable/reference/agent/#__codelineno-0-12) @rollout [](https://microsoft.github.io/agent-lightning/stable/reference/agent/#__codelineno-0-13) def my_prompt_agent(task, prompt_template): [](https://microsoft.github.io/agent-lightning/stable/reference/agent/#__codelineno-0-14) messages = prompt_template.format(task=task.input) [](https://microsoft.github.io/agent-lightning/stable/reference/agent/#__codelineno-0-15) # ... perform rollout with the formatted prompt [](https://microsoft.github.io/agent-lightning/stable/reference/agent/#__codelineno-0-16) return response [](https://microsoft.github.io/agent-lightning/stable/reference/agent/#__codelineno-0-17) [](https://microsoft.github.io/agent-lightning/stable/reference/agent/#__codelineno-0-18) # Function is still callable with original behavior [](https://microsoft.github.io/agent-lightning/stable/reference/agent/#__codelineno-0-19) result = my_llm_agent(task, llm) [](https://microsoft.github.io/agent-lightning/stable/reference/agent/#__codelineno-0-20) [](https://microsoft.github.io/agent-lightning/stable/reference/agent/#__codelineno-0-21) # Agent methods are also available [](https://microsoft.github.io/agent-lightning/stable/reference/agent/#__codelineno-0-22) result = my_llm_agent.rollout(task, resources, rollout)` Raises: * `NotImplementedError` – If the function signature doesn't match any known patterns. Warning The following two decorators are implementations of [`agentlightning.rollout`](https://microsoft.github.io/agent-lightning/stable/reference/agent/#agentlightning.rollout " agentlightning.rollout(func)") . They are not recommended for new users. ### `agentlightning.llm_rollout(func=None, *, strip_proxy=True)` [¶](https://microsoft.github.io/agent-lightning/stable/reference/agent/#agentlightning.llm_rollout "Permanent link") `[](https://microsoft.github.io/agent-lightning/stable/reference/agent/#__codelineno-0-1) llm_rollout( [](https://microsoft.github.io/agent-lightning/stable/reference/agent/#__codelineno-0-2) func: LlmRolloutFunc[T], [](https://microsoft.github.io/agent-lightning/stable/reference/agent/#__codelineno-0-3) ) -> FunctionalLitAgent[T]` `[](https://microsoft.github.io/agent-lightning/stable/reference/agent/#__codelineno-0-1) llm_rollout( [](https://microsoft.github.io/agent-lightning/stable/reference/agent/#__codelineno-0-2) *, strip_proxy: bool = True [](https://microsoft.github.io/agent-lightning/stable/reference/agent/#__codelineno-0-3) ) -> Callable[[LlmRolloutFunc[T]], FunctionalLitAgent[T]]` Create a [`FunctionalLitAgent`](https://microsoft.github.io/agent-lightning/stable/reference/internal/#agentlightning.litagent.decorator.FunctionalLitAgent " agentlightning.litagent.decorator.FunctionalLitAgent") for LLM-based rollouts. Parameters: * **`func`** (`LlmRolloutFunc[T] | None`, default: `None` ) – Callable defining the agent's behaviour. Supported signatures include: * `(task, llm) -> result` * `(task, llm, rollout) -> result` * `async (task, llm) -> result` * `async (task, llm, rollout) -> result` * **`strip_proxy`** (`bool`, default: `True` ) – When `True`, convert proxy resources into concrete [`LLM`](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.LLM " agentlightning.LLM") instances before calling the function. Defaults to `True`. Returns: * `[FunctionalLitAgent](https://microsoft.github.io/agent-lightning/stable/reference/internal/#agentlightning.litagent.decorator.FunctionalLitAgent " agentlightning.litagent.decorator.FunctionalLitAgent") [T] | Callable[[LlmRolloutFunc[T]], [FunctionalLitAgent](https://microsoft.github.io/agent-lightning/stable/reference/internal/#agentlightning.litagent.decorator.FunctionalLitAgent " agentlightning.litagent.decorator.FunctionalLitAgent") [T]]` – [`FunctionalLitAgent`](https://microsoft.github.io/agent-lightning/stable/reference/internal/#agentlightning.litagent.decorator.FunctionalLitAgent " agentlightning.litagent.decorator.FunctionalLitAgent") that * `[FunctionalLitAgent](https://microsoft.github.io/agent-lightning/stable/reference/internal/#agentlightning.litagent.decorator.FunctionalLitAgent " agentlightning.litagent.decorator.FunctionalLitAgent") [T] | Callable[[LlmRolloutFunc[T]], [FunctionalLitAgent](https://microsoft.github.io/agent-lightning/stable/reference/internal/#agentlightning.litagent.decorator.FunctionalLitAgent " agentlightning.litagent.decorator.FunctionalLitAgent") [T]]` – wraps the supplied function. Examples: `[](https://microsoft.github.io/agent-lightning/stable/reference/agent/#__codelineno-0-1) @llm_rollout [](https://microsoft.github.io/agent-lightning/stable/reference/agent/#__codelineno-0-2) def my_agent(task, llm): [](https://microsoft.github.io/agent-lightning/stable/reference/agent/#__codelineno-0-3) return llm.endpoint [](https://microsoft.github.io/agent-lightning/stable/reference/agent/#__codelineno-0-4) [](https://microsoft.github.io/agent-lightning/stable/reference/agent/#__codelineno-0-5) @llm_rollout(strip_proxy=False) [](https://microsoft.github.io/agent-lightning/stable/reference/agent/#__codelineno-0-6) def my_agent_no_strip(task, llm): [](https://microsoft.github.io/agent-lightning/stable/reference/agent/#__codelineno-0-7) return llm.model [](https://microsoft.github.io/agent-lightning/stable/reference/agent/#__codelineno-0-8) [](https://microsoft.github.io/agent-lightning/stable/reference/agent/#__codelineno-0-9) result = my_agent(task, llm) [](https://microsoft.github.io/agent-lightning/stable/reference/agent/#__codelineno-0-10) result = my_agent.rollout(task, resources, rollout)` ### `agentlightning.prompt_rollout(func=None)` [¶](https://microsoft.github.io/agent-lightning/stable/reference/agent/#agentlightning.prompt_rollout "Permanent link") `[](https://microsoft.github.io/agent-lightning/stable/reference/agent/#__codelineno-0-1) prompt_rollout( [](https://microsoft.github.io/agent-lightning/stable/reference/agent/#__codelineno-0-2) func: PromptRolloutFunc[T], [](https://microsoft.github.io/agent-lightning/stable/reference/agent/#__codelineno-0-3) ) -> FunctionalLitAgent[T]` `[](https://microsoft.github.io/agent-lightning/stable/reference/agent/#__codelineno-0-1) prompt_rollout() -> ( [](https://microsoft.github.io/agent-lightning/stable/reference/agent/#__codelineno-0-2) Callable[[PromptRolloutFunc[T]], FunctionalLitAgent[T]] [](https://microsoft.github.io/agent-lightning/stable/reference/agent/#__codelineno-0-3) )` Create a [`FunctionalLitAgent`](https://microsoft.github.io/agent-lightning/stable/reference/internal/#agentlightning.litagent.decorator.FunctionalLitAgent " agentlightning.litagent.decorator.FunctionalLitAgent") for prompt-based rollouts. This decorator is designed for agents that work with tunable prompt templates. It enables a workflow where algorithms manage and optimize the prompt template, while agents consume the template to perform rollouts. This is particularly useful for prompt optimization scenarios. Parameters: * **`func`** (`PromptRolloutFunc[T] | None`, default: `None` ) – Callable defining the agent's behavior. Supported signatures include: * `(task, prompt_template) -> result` * `(task, prompt_template, rollout) -> result` * `async (task, prompt_template) -> result` * `async (task, prompt_template, rollout) -> result` Returns: * `[FunctionalLitAgent](https://microsoft.github.io/agent-lightning/stable/reference/internal/#agentlightning.litagent.decorator.FunctionalLitAgent " agentlightning.litagent.decorator.FunctionalLitAgent") [T] | Callable[[PromptRolloutFunc[T]], [FunctionalLitAgent](https://microsoft.github.io/agent-lightning/stable/reference/internal/#agentlightning.litagent.decorator.FunctionalLitAgent " agentlightning.litagent.decorator.FunctionalLitAgent") [T]]` – [`FunctionalLitAgent`](https://microsoft.github.io/agent-lightning/stable/reference/internal/#agentlightning.litagent.decorator.FunctionalLitAgent " agentlightning.litagent.decorator.FunctionalLitAgent") that * `[FunctionalLitAgent](https://microsoft.github.io/agent-lightning/stable/reference/internal/#agentlightning.litagent.decorator.FunctionalLitAgent " agentlightning.litagent.decorator.FunctionalLitAgent") [T] | Callable[[PromptRolloutFunc[T]], [FunctionalLitAgent](https://microsoft.github.io/agent-lightning/stable/reference/internal/#agentlightning.litagent.decorator.FunctionalLitAgent " agentlightning.litagent.decorator.FunctionalLitAgent") [T]]` – wraps the supplied function. Examples: `[](https://microsoft.github.io/agent-lightning/stable/reference/agent/#__codelineno-0-1) @prompt_rollout [](https://microsoft.github.io/agent-lightning/stable/reference/agent/#__codelineno-0-2) def my_agent(task, prompt_template): [](https://microsoft.github.io/agent-lightning/stable/reference/agent/#__codelineno-0-3) messages = prompt_template.format(task=task.input) [](https://microsoft.github.io/agent-lightning/stable/reference/agent/#__codelineno-0-4) return messages [](https://microsoft.github.io/agent-lightning/stable/reference/agent/#__codelineno-0-5) [](https://microsoft.github.io/agent-lightning/stable/reference/agent/#__codelineno-0-6) result = my_agent(task, prompt_template) [](https://microsoft.github.io/agent-lightning/stable/reference/agent/#__codelineno-0-7) result = my_agent.rollout(task, resources, rollout)` Class-based Agents[¶](https://microsoft.github.io/agent-lightning/stable/reference/agent/#class-based-agents "Permanent link") ------------------------------------------------------------------------------------------------------------------------------- ### `agentlightning.LitAgent` [¶](https://microsoft.github.io/agent-lightning/stable/reference/agent/#agentlightning.LitAgent "Permanent link") Bases: `Generic[T]` Base class for implementing agent rollouts. Subclasses override the rollout methods to process tasks while the trainer and runner infrastructure manages orchestration, tracing, and persistence. #### `runner` `property` [¶](https://microsoft.github.io/agent-lightning/stable/reference/agent/#agentlightning.LitAgent.runner "Permanent link") Return the runner responsible for executing rollouts. #### `tracer` `property` [¶](https://microsoft.github.io/agent-lightning/stable/reference/agent/#agentlightning.LitAgent.tracer "Permanent link") Return the tracer configured for this agent. #### `trainer` `property` [¶](https://microsoft.github.io/agent-lightning/stable/reference/agent/#agentlightning.LitAgent.trainer "Permanent link") Return the trainer associated with this agent. #### `__init__(*, trained_agents=None)` [¶](https://microsoft.github.io/agent-lightning/stable/reference/agent/#agentlightning.LitAgent.__init__ "Permanent link") Initialize the agent instance. Parameters: * **`trained_agents`** (`Optional[str]`, default: `None` ) – Optional identifier used by legacy tooling to mark trained agents. Deprecated The `trained_agents` flag is deprecated. Configure `agent_match` in the adapter layer instead. See [`TracerTraceToTriplet`](https://microsoft.github.io/agent-lightning/stable/reference/algorithm/#agentlightning.TracerTraceToTriplet " agentlightning.TracerTraceToTriplet") for more details. #### `get_runner()` [¶](https://microsoft.github.io/agent-lightning/stable/reference/agent/#agentlightning.LitAgent.get_runner "Permanent link") Return the runner responsible for executing rollouts. #### `get_tracer()` [¶](https://microsoft.github.io/agent-lightning/stable/reference/agent/#agentlightning.LitAgent.get_tracer "Permanent link") Return the tracer configured for this agent. #### `get_trainer()` [¶](https://microsoft.github.io/agent-lightning/stable/reference/agent/#agentlightning.LitAgent.get_trainer "Permanent link") Return the trainer associated with this agent. #### `is_async()` [¶](https://microsoft.github.io/agent-lightning/stable/reference/agent/#agentlightning.LitAgent.is_async "Permanent link") Return `True` when the agent overrides any asynchronous rollout methods. Override this method for customized async detection logic. #### `on_rollout_end(task, rollout, runner, tracer)` [¶](https://microsoft.github.io/agent-lightning/stable/reference/agent/#agentlightning.LitAgent.on_rollout_end "Permanent link") Hook invoked after a rollout completes. Subclasses can override this method for cleanup or additional logging. The default implementation is a no-op. Parameters: * **`task`** (`[Task](https://microsoft.github.io/agent-lightning/stable/reference/internal/#agentlightning.Task " agentlightning.Task (agentlightning.types.Task)") `) – [`Task`](https://microsoft.github.io/agent-lightning/stable/reference/internal/#agentlightning.Task " agentlightning.Task") that was processed. * **`rollout`** (`[Rollout](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.Rollout " agentlightning.Rollout (agentlightning.types.Rollout)") `) – Resulting [`Rollout`](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.Rollout " agentlightning.Rollout") . * **`runner`** (`[Runner](https://microsoft.github.io/agent-lightning/stable/reference/runner/#agentlightning.Runner " agentlightning.Runner (agentlightning.runner.Runner)") [T]`) – [`Runner`](https://microsoft.github.io/agent-lightning/stable/reference/runner/#agentlightning.Runner " agentlightning.Runner") managing the rollout. * **`tracer`** (`[Tracer](https://microsoft.github.io/agent-lightning/stable/reference/runner/#agentlightning.Tracer " agentlightning.Tracer (agentlightning.tracer.Tracer)") `) – [`Tracer`](https://microsoft.github.io/agent-lightning/stable/reference/runner/#agentlightning.Tracer " agentlightning.Tracer") associated with the runner. Deprecated Override [`Hook.on_rollout_end`](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.Hook.on_rollout_end " on_rollout_end(*, agent, runner, rollout, spans) async ") instead of this method when extending agents. #### `on_rollout_start(task, runner, tracer)` [¶](https://microsoft.github.io/agent-lightning/stable/reference/agent/#agentlightning.LitAgent.on_rollout_start "Permanent link") Hook invoked immediately before a rollout begins. Subclasses can override this method to implement custom logic such as logging, metric collection, or resource setup. The default implementation is a no-op. Parameters: * **`task`** (`[Task](https://microsoft.github.io/agent-lightning/stable/reference/internal/#agentlightning.Task " agentlightning.Task (agentlightning.types.Task)") `) – [`Task`](https://microsoft.github.io/agent-lightning/stable/reference/internal/#agentlightning.Task " agentlightning.Task") that will be processed. * **`runner`** (`[Runner](https://microsoft.github.io/agent-lightning/stable/reference/runner/#agentlightning.Runner " agentlightning.Runner (agentlightning.runner.Runner)") [T]`) – [`Runner`](https://microsoft.github.io/agent-lightning/stable/reference/runner/#agentlightning.Runner " agentlightning.Runner") managing the rollout. * **`tracer`** (`[Tracer](https://microsoft.github.io/agent-lightning/stable/reference/runner/#agentlightning.Tracer " agentlightning.Tracer (agentlightning.tracer.Tracer)") `) – [`Tracer`](https://microsoft.github.io/agent-lightning/stable/reference/runner/#agentlightning.Tracer " agentlightning.Tracer") associated with the runner. Deprecated Override [`Hook.on_rollout_start`](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.Hook.on_rollout_start " on_rollout_start(*, agent, runner, rollout) async ") instead of this method when extending agents. #### `rollout(task, resources, rollout)` [¶](https://microsoft.github.io/agent-lightning/stable/reference/agent/#agentlightning.LitAgent.rollout "Permanent link") Execute a rollout synchronously. If you don't wish to implement both training rollout and validation rollout separately, you can just implement `rollout` which will work for both. Parameters: * **`task`** (`T`) – Task payload provided by the scheduler. * **`resources`** (`[NamedResources](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.NamedResources " agentlightning.NamedResources = Dict[str, ResourceUnion] module-attribute (agentlightning.types.NamedResources)") `) – Mapping of named resources (for example LLMs or prompt templates). * **`rollout`** (`[Rollout](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.Rollout " agentlightning.Rollout (agentlightning.types.Rollout)") `) – Rollout metadata. Avoid mutating this object directly unless a subclass needs to override defaults. Returns: * `[RolloutRawResult](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.RolloutRawResult " agentlightning.RolloutRawResult = Union[None, float, List[ReadableSpan], List[Span], List[SpanCoreFields]] module-attribute (agentlightning.types.RolloutRawResult)") ` – One of the following values: * `[RolloutRawResult](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.RolloutRawResult " agentlightning.RolloutRawResult = Union[None, float, List[ReadableSpan], List[Span], List[SpanCoreFields]] module-attribute (agentlightning.types.RolloutRawResult)") ` – * `None` when tracing is handled by the runner. * `[RolloutRawResult](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.RolloutRawResult " agentlightning.RolloutRawResult = Union[None, float, List[ReadableSpan], List[Span], List[SpanCoreFields]] module-attribute (agentlightning.types.RolloutRawResult)") ` – * `float` representing the final reward. * `[RolloutRawResult](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.RolloutRawResult " agentlightning.RolloutRawResult = Union[None, float, List[ReadableSpan], List[Span], List[SpanCoreFields]] module-attribute (agentlightning.types.RolloutRawResult)") ` – * `List[ReadableSpan]` with OpenTelemetry spans. * `[RolloutRawResult](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.RolloutRawResult " agentlightning.RolloutRawResult = Union[None, float, List[ReadableSpan], List[Span], List[SpanCoreFields]] module-attribute (agentlightning.types.RolloutRawResult)") ` – * `List[Span]` with Agent Lightning spans. * `[RolloutRawResult](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.RolloutRawResult " agentlightning.RolloutRawResult = Union[None, float, List[ReadableSpan], List[Span], List[SpanCoreFields]] module-attribute (agentlightning.types.RolloutRawResult)") ` – * `List[SpanCoreFields]` with Agent Lightning spans. #### `rollout_async(task, resources, rollout)` `async` [¶](https://microsoft.github.io/agent-lightning/stable/reference/agent/#agentlightning.LitAgent.rollout_async "Permanent link") Execute a rollout asynchronously. Parameters: * **`task`** (`T`) – Task payload provided by the scheduler. * **`resources`** (`[NamedResources](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.NamedResources " agentlightning.NamedResources = Dict[str, ResourceUnion] module-attribute (agentlightning.types.NamedResources)") `) – Mapping of named resources (for example LLMs or prompt templates). * **`rollout`** (`[Rollout](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.Rollout " agentlightning.Rollout (agentlightning.types.Rollout)") `) – Rollout metadata. Avoid mutating this object directly unless a subclass needs to override defaults. Returns: * `[RolloutRawResult](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.RolloutRawResult " agentlightning.RolloutRawResult = Union[None, float, List[ReadableSpan], List[Span], List[SpanCoreFields]] module-attribute (agentlightning.types.RolloutRawResult)") ` – Same possible return values as * `[RolloutRawResult](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.RolloutRawResult " agentlightning.RolloutRawResult = Union[None, float, List[ReadableSpan], List[Span], List[SpanCoreFields]] module-attribute (agentlightning.types.RolloutRawResult)") ` – [`rollout`](https://microsoft.github.io/agent-lightning/stable/reference/agent/#agentlightning.LitAgent.rollout " rollout(task, resources, rollout)") . #### `set_runner(runner)` [¶](https://microsoft.github.io/agent-lightning/stable/reference/agent/#agentlightning.LitAgent.set_runner "Permanent link") Attach the runner responsible for executing rollouts. Parameters: * **`runner`** (`[Runner](https://microsoft.github.io/agent-lightning/stable/reference/runner/#agentlightning.Runner " agentlightning.Runner (agentlightning.runner.Runner)") [T]`) – [`Runner`](https://microsoft.github.io/agent-lightning/stable/reference/runner/#agentlightning.Runner " agentlightning.Runner") coordinating execution. #### `set_trainer(trainer)` [¶](https://microsoft.github.io/agent-lightning/stable/reference/agent/#agentlightning.LitAgent.set_trainer "Permanent link") Attach the trainer responsible for orchestration. Parameters: * **`trainer`** (`[Trainer](https://microsoft.github.io/agent-lightning/stable/reference/trainer/#agentlightning.Trainer " agentlightning.Trainer (agentlightning.trainer.Trainer)") `) – [`Trainer`](https://microsoft.github.io/agent-lightning/stable/reference/trainer/#agentlightning.Trainer " agentlightning.Trainer") that manages the agent. #### `training_rollout(task, resources, rollout)` [¶](https://microsoft.github.io/agent-lightning/stable/reference/agent/#agentlightning.LitAgent.training_rollout "Permanent link") Process a single training task synchronously. By default, this method delegates to [`rollout`](https://microsoft.github.io/agent-lightning/stable/reference/agent/#agentlightning.LitAgent.rollout " rollout(task, resources, rollout)") . #### `training_rollout_async(task, resources, rollout)` `async` [¶](https://microsoft.github.io/agent-lightning/stable/reference/agent/#agentlightning.LitAgent.training_rollout_async "Permanent link") Process a single training task asynchronously. By default, this method delegates to [`rollout_async`](https://microsoft.github.io/agent-lightning/stable/reference/agent/#agentlightning.LitAgent.rollout_async " rollout_async(task, resources, rollout) async ") . #### `validation_rollout(task, resources, rollout)` [¶](https://microsoft.github.io/agent-lightning/stable/reference/agent/#agentlightning.LitAgent.validation_rollout "Permanent link") Process a single validation task synchronously. Override this method when validation should differ from training. The default implementation delegates to [`training_rollout`](https://microsoft.github.io/agent-lightning/stable/reference/agent/#agentlightning.LitAgent.training_rollout " training_rollout(task, resources, rollout)") . #### `validation_rollout_async(task, resources, rollout)` `async` [¶](https://microsoft.github.io/agent-lightning/stable/reference/agent/#agentlightning.LitAgent.validation_rollout_async "Permanent link") Process a single validation task asynchronously. Override this method when validation should differ from training. The default implementation delegates to [`training_rollout_async`](https://microsoft.github.io/agent-lightning/stable/reference/agent/#agentlightning.LitAgent.training_rollout_async " training_rollout_async(task, resources, rollout) async ") . Emitter[¶](https://microsoft.github.io/agent-lightning/stable/reference/agent/#emitter "Permanent link") --------------------------------------------------------------------------------------------------------- ### `agentlightning.operation(fn=None, *, propagate=True, name=None, **additional_attributes)` [¶](https://microsoft.github.io/agent-lightning/stable/reference/agent/#agentlightning.operation "Permanent link") `[](https://microsoft.github.io/agent-lightning/stable/reference/agent/#__codelineno-0-1) operation( [](https://microsoft.github.io/agent-lightning/stable/reference/agent/#__codelineno-0-2) fn: _FnType, [](https://microsoft.github.io/agent-lightning/stable/reference/agent/#__codelineno-0-3) *, [](https://microsoft.github.io/agent-lightning/stable/reference/agent/#__codelineno-0-4) propagate: bool = True, [](https://microsoft.github.io/agent-lightning/stable/reference/agent/#__codelineno-0-5) name: Optional[str] = None, [](https://microsoft.github.io/agent-lightning/stable/reference/agent/#__codelineno-0-6) **additional_attributes: Any [](https://microsoft.github.io/agent-lightning/stable/reference/agent/#__codelineno-0-7) ) -> _FnType` `[](https://microsoft.github.io/agent-lightning/stable/reference/agent/#__codelineno-0-1) operation( [](https://microsoft.github.io/agent-lightning/stable/reference/agent/#__codelineno-0-2) *, [](https://microsoft.github.io/agent-lightning/stable/reference/agent/#__codelineno-0-3) propagate: bool = True, [](https://microsoft.github.io/agent-lightning/stable/reference/agent/#__codelineno-0-4) name: Optional[str] = None, [](https://microsoft.github.io/agent-lightning/stable/reference/agent/#__codelineno-0-5) **additional_attributes: Any [](https://microsoft.github.io/agent-lightning/stable/reference/agent/#__codelineno-0-6) ) -> OperationContext` `[](https://microsoft.github.io/agent-lightning/stable/reference/agent/#__codelineno-0-1) operation( [](https://microsoft.github.io/agent-lightning/stable/reference/agent/#__codelineno-0-2) fn: _FnType, [](https://microsoft.github.io/agent-lightning/stable/reference/agent/#__codelineno-0-3) *, [](https://microsoft.github.io/agent-lightning/stable/reference/agent/#__codelineno-0-4) name: Optional[str] = None, [](https://microsoft.github.io/agent-lightning/stable/reference/agent/#__codelineno-0-5) **additional_attributes: Any [](https://microsoft.github.io/agent-lightning/stable/reference/agent/#__codelineno-0-6) ) -> _FnType` `[](https://microsoft.github.io/agent-lightning/stable/reference/agent/#__codelineno-0-1) operation( [](https://microsoft.github.io/agent-lightning/stable/reference/agent/#__codelineno-0-2) *, [](https://microsoft.github.io/agent-lightning/stable/reference/agent/#__codelineno-0-3) name: Optional[str] = None, [](https://microsoft.github.io/agent-lightning/stable/reference/agent/#__codelineno-0-4) **additional_attributes: Any [](https://microsoft.github.io/agent-lightning/stable/reference/agent/#__codelineno-0-5) ) -> OperationContext` `[](https://microsoft.github.io/agent-lightning/stable/reference/agent/#__codelineno-0-1) operation( [](https://microsoft.github.io/agent-lightning/stable/reference/agent/#__codelineno-0-2) fn: _FnType, **additional_attributes: Any [](https://microsoft.github.io/agent-lightning/stable/reference/agent/#__codelineno-0-3) ) -> _FnType` `[](https://microsoft.github.io/agent-lightning/stable/reference/agent/#__codelineno-0-1) operation(**additional_attributes: Any) -> OperationContext` Entry point for tracking operations. This helper can be used either as a decorator or as a context manager. The span name is fixed to [`AGL_OPERATION`](https://microsoft.github.io/agent-lightning/stable/reference/semconv/#agentlightning.semconv.AGL_OPERATION " AGL_OPERATION = 'agentlightning.operation' module-attribute ") ; custom span names are not supported. Any keyword arguments are recorded as span attributes. Usage as a decorator: `[](https://microsoft.github.io/agent-lightning/stable/reference/agent/#__codelineno-0-1) @operation [](https://microsoft.github.io/agent-lightning/stable/reference/agent/#__codelineno-0-2) def func(...): [](https://microsoft.github.io/agent-lightning/stable/reference/agent/#__codelineno-0-3) ... [](https://microsoft.github.io/agent-lightning/stable/reference/agent/#__codelineno-0-4) [](https://microsoft.github.io/agent-lightning/stable/reference/agent/#__codelineno-0-5) @operation(category="compute") [](https://microsoft.github.io/agent-lightning/stable/reference/agent/#__codelineno-0-6) def func(...): [](https://microsoft.github.io/agent-lightning/stable/reference/agent/#__codelineno-0-7) ...` Usage as a context manager: `[](https://microsoft.github.io/agent-lightning/stable/reference/agent/#__codelineno-1-1) with operation(user_id=123) as op: [](https://microsoft.github.io/agent-lightning/stable/reference/agent/#__codelineno-1-2) op.set_input(data=data) [](https://microsoft.github.io/agent-lightning/stable/reference/agent/#__codelineno-1-3) # ... do work ... [](https://microsoft.github.io/agent-lightning/stable/reference/agent/#__codelineno-1-4) op.set_output(result)` Parameters: * **`fn`** (`Optional[_FnType]`, default: `None` ) – When used as `@operation`, this is the wrapped function. When used as `operation(**attrs)`, this should be omitted (or left as `None`) and only keyword attributes are provided. * **`propagate`** (`bool`, default: `True` ) – Whether spans should use the active span processor. When False, spans will stay local and not be exported. * **`name`** (`Optional[str]`, default: `None` ) – Optional alias that populates [`LightningSpanAttributes.OPERATION_NAME`](https://microsoft.github.io/agent-lightning/stable/reference/semconv/#agentlightning.semconv.LightningSpanAttributes.OPERATION_NAME " OPERATION_NAME = 'agentlightning.operation.name' class-attribute instance-attribute ") when `additional_attributes` does not already define it. * **`**additional_attributes`** (`Any`, default: `{}` ) – Additional span attributes to attach at creation time. Returns: * `Union[_FnType, [OperationContext](https://microsoft.github.io/agent-lightning/stable/reference/internal/#agentlightning.emitter.annotation.OperationContext " agentlightning.emitter.annotation.OperationContext") ]` – Either a wrapped callable (when used as a decorator) or an * `Union[_FnType, [OperationContext](https://microsoft.github.io/agent-lightning/stable/reference/internal/#agentlightning.emitter.annotation.OperationContext " agentlightning.emitter.annotation.OperationContext") ]` – [`OperationContext`](https://microsoft.github.io/agent-lightning/stable/reference/internal/#agentlightning.emitter.annotation.OperationContext " agentlightning.emitter.annotation.OperationContext") * `Union[_FnType, [OperationContext](https://microsoft.github.io/agent-lightning/stable/reference/internal/#agentlightning.emitter.annotation.OperationContext " agentlightning.emitter.annotation.OperationContext") ]` – (when used as a context manager factory). ### `agentlightning.emit_annotation(annotation, propagate=True)` [¶](https://microsoft.github.io/agent-lightning/stable/reference/agent/#agentlightning.emit_annotation "Permanent link") Emit a new annotation span. This is the underlying implementation of [`emit_reward`](https://microsoft.github.io/agent-lightning/stable/reference/agent/#agentlightning.emit_reward " agentlightning.emit_reward(reward, *, primary_key=None, attributes=None, propagate=True)") . Annotation spans are used to annotate a specific event or a part of rollout. See [semconv](https://microsoft.github.io/agent-lightning/stable/reference/semconv/#agentlightning.semconv " agentlightning.semconv") for conventional annotation keys in Agent-lightning. If annotations contain nested dicts, they will be flattened before emitting. Complex objects will lead to emitting failures. Parameters: * **`annotation`** (`Dict[str, Any]`) – Dictionary containing annotation key-value pairs. Representatives are rewards, tags, and metadata. * **`propagate`** (`bool`, default: `True` ) – Whether to propagate the span to tracers automatically. ### `agentlightning.emit_reward(reward, *, primary_key=None, attributes=None, propagate=True)` [¶](https://microsoft.github.io/agent-lightning/stable/reference/agent/#agentlightning.emit_reward "Permanent link") Emit a reward value as an OpenTelemetry span. Examples: Emit a single-dimensional reward: `[](https://microsoft.github.io/agent-lightning/stable/reference/agent/#__codelineno-0-1) >>> emit_reward(1.0)` Emit multi-dimensional rewards: `[](https://microsoft.github.io/agent-lightning/stable/reference/agent/#__codelineno-0-1) >>> emit_reward({"task_completion": 1.0, "efficiency": 0.8}, primary_key="task_completion")` Emit a reward with additional attributes (for example linking to another response span): `[](https://microsoft.github.io/agent-lightning/stable/reference/agent/#__codelineno-0-1) >>> from agentlightning.utils.otel import make_link_attributes [](https://microsoft.github.io/agent-lightning/stable/reference/agent/#__codelineno-0-2) >>> emit_reward(0.5, attributes=make_link_attributes({"gen_ai.response.id": "response-123"}))` Or adding tags onto the reward span: `[](https://microsoft.github.io/agent-lightning/stable/reference/agent/#__codelineno-0-1) >>> from agentlightning.utils.otel import make_tag_attributes [](https://microsoft.github.io/agent-lightning/stable/reference/agent/#__codelineno-0-2) >>> emit_reward(0.7, attributes=make_tag_attributes(["fast", "reliable"]))` Parameters: * **`reward`** (`float | Dict[str, Any]`) – Numeric reward to record. Integers and booleans are converted to floating point numbers for consistency. Use a dictionary to represent a multi-dimensional reward. * **`attributes`** (`Dict[str, Any] | None`, default: `None` ) – Other optional span attributes. * **`propagate`** (`bool`, default: `True` ) – Whether to propagate the span to exporters automatically. Returns: * `[SpanCoreFields](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.SpanCoreFields " agentlightning.SpanCoreFields (agentlightning.types.SpanCoreFields)") ` – Span core fields capturing the recorded reward. ### `agentlightning.emit_message(message, attributes=None, propagate=True)` [¶](https://microsoft.github.io/agent-lightning/stable/reference/agent/#agentlightning.emit_message "Permanent link") Emit a textual message as an OpenTelemetry span. Commonly used for sending debugging and logging messages. Parameters: * **`message`** (`str`) – Human readable message to attach as a span attribute. * **`attributes`** (`Optional[Dict[str, Any]]`, default: `None` ) – Additional attributes to attach to the message span. * **`propagate`** (`bool`, default: `True` ) – Whether to propagate the span to exporters automatically. Note OpenTelemetry distinguishes between logs and spans. Emitting the message as a span keeps all Agent Lightning telemetry in a single data store for analysis. ### `agentlightning.emit_object(object, attributes=None, propagate=True)` [¶](https://microsoft.github.io/agent-lightning/stable/reference/agent/#agentlightning.emit_object "Permanent link") Emit an object's serialized representation as an OpenTelemetry span. Parameters: * **`object`** (`Any`) – Data structure to encode as JSON and attach to the span payload. * **`attributes`** (`Optional[Dict[str, Any]]`, default: `None` ) – Additional attributes to attach to the object span. * **`propagate`** (`bool`, default: `True` ) – Whether to propagate the span to exporters automatically. Note The payload must be JSON serializable. Non-serializable objects will lead to a RuntimeError. ### `agentlightning.emit_exception(exception, attributes=None, propagate=True)` [¶](https://microsoft.github.io/agent-lightning/stable/reference/agent/#agentlightning.emit_exception "Permanent link") Record an exception with OpenTelemetry metadata. Classic OpenTelemetry records exceptions in a dedicated logging service. We simplify the model and use trace spans to record exceptions as well. Parameters: * **`exception`** (`BaseException`) – Raised exception instance to serialize into telemetry attributes. * **`attributes`** (`Optional[Dict[str, Any]]`, default: `None` ) – Additional attributes to attach to the exception span. * **`propagate`** (`bool`, default: `True` ) – Whether to propagate the span to exporters automatically. Note The helper validates its input. If a non-exception value is provided, a TypeError is raised to indicate a programming mistake. Emitter Helpers[¶](https://microsoft.github.io/agent-lightning/stable/reference/agent/#emitter-helpers "Permanent link") ------------------------------------------------------------------------------------------------------------------------- ### `agentlightning.get_message_value(span)` [¶](https://microsoft.github.io/agent-lightning/stable/reference/agent/#agentlightning.get_message_value "Permanent link") Extract the message string from a message span. Parameters: * **`span`** (`[SpanLike](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.SpanLike " agentlightning.SpanLike = Union[ReadableSpan, Span] module-attribute (agentlightning.types.SpanLike)") `) – Span-like object to extract the message from. ### `agentlightning.get_object_value(span)` [¶](https://microsoft.github.io/agent-lightning/stable/reference/agent/#agentlightning.get_object_value "Permanent link") Extract the object payload from an object span. Parameters: * **`span`** (`[SpanLike](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.SpanLike " agentlightning.SpanLike = Union[ReadableSpan, Span] module-attribute (agentlightning.types.SpanLike)") `) – Span object produced by Agent Lightning emitters. ### `agentlightning.find_final_reward(spans)` [¶](https://microsoft.github.io/agent-lightning/stable/reference/agent/#agentlightning.find_final_reward "Permanent link") Return the last reward value present in the provided spans. Parameters: * **`spans`** (`Sequence[[SpanLike](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.SpanLike " agentlightning.SpanLike = Union[ReadableSpan, Span] module-attribute (agentlightning.types.SpanLike)") ]`) – Sequence containing [`ReadableSpan`](https://opentelemetry.io/docs/concepts/signals/traces/) objects or mocked span-like values. Returns: * `Optional[float]` – Reward value from the latest reward span, or `None` when none are found. ### `agentlightning.find_reward_spans(spans)` [¶](https://microsoft.github.io/agent-lightning/stable/reference/agent/#agentlightning.find_reward_spans "Permanent link") Return all reward spans in the provided sequence. Parameters: * **`spans`** (`Sequence[[SpanLike](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.SpanLike " agentlightning.SpanLike = Union[ReadableSpan, Span] module-attribute (agentlightning.types.SpanLike)") ]`) – Sequence containing [`ReadableSpan`](https://opentelemetry.io/docs/concepts/signals/traces/) objects or mocked span-like values. Returns: * `List[[SpanLike](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.SpanLike " agentlightning.SpanLike = Union[ReadableSpan, Span] module-attribute (agentlightning.types.SpanLike)") ]` – List of spans that could be parsed as rewards. ### `agentlightning.get_reward_value(span)` [¶](https://microsoft.github.io/agent-lightning/stable/reference/agent/#agentlightning.get_reward_value "Permanent link") Extract the reward value from a span, if available. Parameters: * **`span`** (`[SpanLike](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.SpanLike " agentlightning.SpanLike = Union[ReadableSpan, Span] module-attribute (agentlightning.types.SpanLike)") `) – Span object produced by AgentOps or Agent Lightning emitters. Returns: * `Optional[float]` – The primary reward encoded in the span or `None` when the span does not represent a reward. ### `agentlightning.get_rewards_from_span(span)` [¶](https://microsoft.github.io/agent-lightning/stable/reference/agent/#agentlightning.get_rewards_from_span "Permanent link") Extract the reward as a list from a span, if available. Parameters: * **`span`** (`[SpanLike](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.SpanLike " agentlightning.SpanLike = Union[ReadableSpan, Span] module-attribute (agentlightning.types.SpanLike)") `) – Span object produced by AgentOps or Agent Lightning emitters. Returns: * `List[[RewardPydanticModel](https://microsoft.github.io/agent-lightning/stable/reference/semconv/#agentlightning.semconv.RewardPydanticModel " RewardPydanticModel (agentlightning.semconv.RewardPydanticModel)") ]` – A list of reward dimensions encoded in the span or an empty list when the span does not represent a reward. ### `agentlightning.is_reward_span(span)` [¶](https://microsoft.github.io/agent-lightning/stable/reference/agent/#agentlightning.is_reward_span "Permanent link") Return `True` when the provided span encodes a reward value. Back to top --- # Algorithm - Agent-lightning [Skip to content](https://microsoft.github.io/agent-lightning/stable/reference/algorithm/#algorithm-side-references) ⚠️ You are viewing development documentation. For stable documentation, please visit [https://microsoft.github.io/agent-lightning/stable/](https://microsoft.github.io/agent-lightning/stable/) Algorithm ========= Algorithm-side References[¶](https://microsoft.github.io/agent-lightning/stable/reference/algorithm/#algorithm-side-references "Permanent link") ------------------------------------------------------------------------------------------------------------------------------------------------- Note This reference covers APIs that are designed to be used at "Algorithm Side". For built-in algorithms, see [Algorithm Zoo](https://microsoft.github.io/agent-lightning/stable/algorithm-zoo/) . Base Class and Decorators[¶](https://microsoft.github.io/agent-lightning/stable/reference/algorithm/#base-class-and-decorators "Permanent link") ------------------------------------------------------------------------------------------------------------------------------------------------- ### `agentlightning.Algorithm` [¶](https://microsoft.github.io/agent-lightning/stable/reference/algorithm/#agentlightning.Algorithm "Permanent link") Algorithm is the strategy, or tuner to train the agent. #### `get_adapter()` [¶](https://microsoft.github.io/agent-lightning/stable/reference/algorithm/#agentlightning.Algorithm.get_adapter "Permanent link") Retrieve the adapter for this algorithm to communicate with the runners. #### `get_client()` [¶](https://microsoft.github.io/agent-lightning/stable/reference/algorithm/#agentlightning.Algorithm.get_client "Permanent link") Get the client to communicate with the algorithm. If the algorithm does not require a server-client communication, it can also create a mock client that never communicates with itself. Deprecated and will be removed in a future version. Returns: * `[AgentLightningClient](https://microsoft.github.io/agent-lightning/stable/reference/internal/#agentlightning.client.AgentLightningClient " agentlightning.client.AgentLightningClient") ` – The AgentLightningClient instance associated with this algorithm. #### `get_initial_resources()` [¶](https://microsoft.github.io/agent-lightning/stable/reference/algorithm/#agentlightning.Algorithm.get_initial_resources "Permanent link") Get the initial resources for this algorithm. #### `get_llm_proxy()` [¶](https://microsoft.github.io/agent-lightning/stable/reference/algorithm/#agentlightning.Algorithm.get_llm_proxy "Permanent link") Retrieve the configured LLM proxy instance, if one has been set. Returns: * `Optional[[LLMProxy](https://microsoft.github.io/agent-lightning/stable/reference/algorithm/#agentlightning.LLMProxy " agentlightning.LLMProxy (agentlightning.llm_proxy.LLMProxy)") ]` – The active LLMProxy instance or None when not configured. #### `get_store()` [¶](https://microsoft.github.io/agent-lightning/stable/reference/algorithm/#agentlightning.Algorithm.get_store "Permanent link") Retrieve the store for this algorithm to communicate with the runners. #### `get_trainer()` [¶](https://microsoft.github.io/agent-lightning/stable/reference/algorithm/#agentlightning.Algorithm.get_trainer "Permanent link") Get the trainer for this algorithm. Returns: * `[Trainer](https://microsoft.github.io/agent-lightning/stable/reference/trainer/#agentlightning.Trainer " agentlightning.Trainer (agentlightning.trainer.Trainer)") ` – The Trainer instance associated with this agent. #### `is_async()` [¶](https://microsoft.github.io/agent-lightning/stable/reference/algorithm/#agentlightning.Algorithm.is_async "Permanent link") Return True if the algorithm is asynchronous. #### `run(train_dataset=None, val_dataset=None)` [¶](https://microsoft.github.io/agent-lightning/stable/reference/algorithm/#agentlightning.Algorithm.run "Permanent link") Subclasses should implement this method to implement the algorithm. Parameters: * **`train_dataset`** (`Optional[[Dataset](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.Dataset " agentlightning.Dataset (agentlightning.types.Dataset)") [Any]]`, default: `None` ) – The dataset to train on. Not all algorithms require a training dataset. * **`val_dataset`** (`Optional[[Dataset](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.Dataset " agentlightning.Dataset (agentlightning.types.Dataset)") [Any]]`, default: `None` ) – The dataset to validate on. Not all algorithms require a validation dataset. Returns: * `Union[None, Awaitable[None]]` – Algorithm should refrain from returning anything. It should just run the algorithm. #### `set_adapter(adapter)` [¶](https://microsoft.github.io/agent-lightning/stable/reference/algorithm/#agentlightning.Algorithm.set_adapter "Permanent link") Set the adapter for this algorithm to collect and convert traces. #### `set_initial_resources(resources)` [¶](https://microsoft.github.io/agent-lightning/stable/reference/algorithm/#agentlightning.Algorithm.set_initial_resources "Permanent link") Set the initial resources for this algorithm. #### `set_llm_proxy(llm_proxy)` [¶](https://microsoft.github.io/agent-lightning/stable/reference/algorithm/#agentlightning.Algorithm.set_llm_proxy "Permanent link") Set the LLM proxy for this algorithm to reuse when available. Parameters: * **`llm_proxy`** (`[LLMProxy](https://microsoft.github.io/agent-lightning/stable/reference/algorithm/#agentlightning.LLMProxy " agentlightning.LLMProxy (agentlightning.llm_proxy.LLMProxy)") | None`) – The LLMProxy instance configured by the trainer, if any. #### `set_store(store)` [¶](https://microsoft.github.io/agent-lightning/stable/reference/algorithm/#agentlightning.Algorithm.set_store "Permanent link") Set the store for this algorithm to communicate with the runners. Store is set directly instead of using weakref because its copy is meant to be maintained throughout the algorithm's lifecycle. #### `set_trainer(trainer)` [¶](https://microsoft.github.io/agent-lightning/stable/reference/algorithm/#agentlightning.Algorithm.set_trainer "Permanent link") Set the trainer for this algorithm. Parameters: * **`trainer`** (`[Trainer](https://microsoft.github.io/agent-lightning/stable/reference/trainer/#agentlightning.Trainer " agentlightning.Trainer (agentlightning.trainer.Trainer)") `) – The Trainer instance that will handle training and validation. ### `agentlightning.algo(func)` [¶](https://microsoft.github.io/agent-lightning/stable/reference/algorithm/#agentlightning.algo "Permanent link") `[](https://microsoft.github.io/agent-lightning/stable/reference/algorithm/#__codelineno-0-1) algo( [](https://microsoft.github.io/agent-lightning/stable/reference/algorithm/#__codelineno-0-2) func: AlgorithmFuncAsync, [](https://microsoft.github.io/agent-lightning/stable/reference/algorithm/#__codelineno-0-3) ) -> FunctionalAlgorithm[Literal[True]]` `[](https://microsoft.github.io/agent-lightning/stable/reference/algorithm/#__codelineno-0-1) algo( [](https://microsoft.github.io/agent-lightning/stable/reference/algorithm/#__codelineno-0-2) func: AlgorithmFuncAsyncFallback, [](https://microsoft.github.io/agent-lightning/stable/reference/algorithm/#__codelineno-0-3) ) -> FunctionalAlgorithm[Any]` `[](https://microsoft.github.io/agent-lightning/stable/reference/algorithm/#__codelineno-0-1) algo( [](https://microsoft.github.io/agent-lightning/stable/reference/algorithm/#__codelineno-0-2) func: AlgorithmFuncSync, [](https://microsoft.github.io/agent-lightning/stable/reference/algorithm/#__codelineno-0-3) ) -> FunctionalAlgorithm[Literal[False]]` `[](https://microsoft.github.io/agent-lightning/stable/reference/algorithm/#__codelineno-0-1) algo( [](https://microsoft.github.io/agent-lightning/stable/reference/algorithm/#__codelineno-0-2) func: AlgorithmFuncSyncFallback, [](https://microsoft.github.io/agent-lightning/stable/reference/algorithm/#__codelineno-0-3) ) -> FunctionalAlgorithm[Any]` Convert a callable into a [`FunctionalAlgorithm`](https://microsoft.github.io/agent-lightning/stable/reference/internal/#agentlightning.algorithm.decorator.FunctionalAlgorithm " agentlightning.algorithm.decorator.FunctionalAlgorithm") . The decorator inspects the callable signature to decide which dependencies to inject at runtime, enabling concise algorithm definitions that still leverage the full training runtime. Parameters: * **`func`** (`Union[AlgorithmFuncSync, AlgorithmFuncAsync, AlgorithmFuncSyncFallback, AlgorithmFuncAsyncFallback]`) – Function implementing the algorithm logic. May be synchronous or asynchronous. The function can expect all of, or a subset of the following parameters: * `store`: [`LightningStore`](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.LightningStore " agentlightning.LightningStore") , * `train_dataset`: [`Dataset`](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.Dataset " agentlightning.Dataset") , * `val_dataset`: [`Dataset`](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.Dataset " agentlightning.Dataset") , * `llm_proxy`: [`LLMProxy`](https://microsoft.github.io/agent-lightning/stable/reference/algorithm/#agentlightning.LLMProxy " agentlightning.LLMProxy") , * `adapter`: [`TraceAdapter`](https://microsoft.github.io/agent-lightning/stable/reference/algorithm/#agentlightning.TraceAdapter " agentlightning.TraceAdapter") , * `initial_resources`: [`NamedResources`](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.NamedResources " agentlightning.NamedResources = Dict[str, ResourceUnion] module-attribute ") , If the function does not expect a parameter, the wrapper will not inject it into the call. Using `*args` and `**kwargs` will not work and no parameters will be injected. Returns: * `Union[[FunctionalAlgorithm](https://microsoft.github.io/agent-lightning/stable/reference/internal/#agentlightning.algorithm.decorator.FunctionalAlgorithm " agentlightning.algorithm.decorator.FunctionalAlgorithm") [Literal[False]], [FunctionalAlgorithm](https://microsoft.github.io/agent-lightning/stable/reference/internal/#agentlightning.algorithm.decorator.FunctionalAlgorithm " agentlightning.algorithm.decorator.FunctionalAlgorithm") [Literal[True]]]` – FunctionalAlgorithm that proxies the callable while exposing the * `Union[[FunctionalAlgorithm](https://microsoft.github.io/agent-lightning/stable/reference/internal/#agentlightning.algorithm.decorator.FunctionalAlgorithm " agentlightning.algorithm.decorator.FunctionalAlgorithm") [Literal[False]], [FunctionalAlgorithm](https://microsoft.github.io/agent-lightning/stable/reference/internal/#agentlightning.algorithm.decorator.FunctionalAlgorithm " agentlightning.algorithm.decorator.FunctionalAlgorithm") [Literal[True]]]` – `Algorithm` interface. Examples: `[](https://microsoft.github.io/agent-lightning/stable/reference/algorithm/#__codelineno-0-1) from agentlightning.algorithm.decorator import algo [](https://microsoft.github.io/agent-lightning/stable/reference/algorithm/#__codelineno-0-2) [](https://microsoft.github.io/agent-lightning/stable/reference/algorithm/#__codelineno-0-3) @algo [](https://microsoft.github.io/agent-lightning/stable/reference/algorithm/#__codelineno-0-4) def batching_algorithm(*, store, train_dataset, val_dataset): [](https://microsoft.github.io/agent-lightning/stable/reference/algorithm/#__codelineno-0-5) for sample in train_dataset: [](https://microsoft.github.io/agent-lightning/stable/reference/algorithm/#__codelineno-0-6) store.enqueue_rollout(input=sample, mode="train") [](https://microsoft.github.io/agent-lightning/stable/reference/algorithm/#__codelineno-0-7) [](https://microsoft.github.io/agent-lightning/stable/reference/algorithm/#__codelineno-0-8) @algo [](https://microsoft.github.io/agent-lightning/stable/reference/algorithm/#__codelineno-0-9) async def async_algorithm(*, store, train_dataset=None, val_dataset=None): [](https://microsoft.github.io/agent-lightning/stable/reference/algorithm/#__codelineno-0-10) await store.enqueue_rollout(input={"prompt": "hello"}, mode="train")` Fast Algorithms (for Debugging)[¶](https://microsoft.github.io/agent-lightning/stable/reference/algorithm/#fast-algorithms-for-debugging "Permanent link") ----------------------------------------------------------------------------------------------------------------------------------------------------------- ### `agentlightning.FastAlgorithm` [¶](https://microsoft.github.io/agent-lightning/stable/reference/algorithm/#agentlightning.FastAlgorithm "Permanent link") Bases: `[Algorithm](https://microsoft.github.io/agent-lightning/stable/reference/algorithm/#agentlightning.Algorithm " agentlightning.Algorithm (agentlightning.algorithm.base.Algorithm)") ` Base class for lightweight algorithms optimised for developer workflows. Fast algorithms prioritise short feedback loops so an agent developer can run small-scale experiments without waiting for long-running training jobs to finish. ### `agentlightning.Baseline` [¶](https://microsoft.github.io/agent-lightning/stable/reference/algorithm/#agentlightning.Baseline "Permanent link") Bases: `[FastAlgorithm](https://microsoft.github.io/agent-lightning/stable/reference/algorithm/#agentlightning.FastAlgorithm " agentlightning.FastAlgorithm (agentlightning.algorithm.fast.FastAlgorithm)") ` Reference implementation that streams the full dataset through the rollout queue. The baseline algorithm batches task submissions, waits for each rollout to finish, and logs every collected span and reward. It is primarily useful as a smoke test for the platform plumbing rather than a performant trainer. The baseline algorithm will auto-start a LLM proxy if one is provided and not yet started. Parameters: * **`n_epochs`** (`int`, default: `1` ) – Number of dataset passes to execute for both the train and val splits during developer experiments. * **`train_split`** (`float`, default: `0.5` ) – Fraction of the concatenated dataset to treat as training data. Must be strictly between 0 and 1. * **`polling_interval`** (`float`, default: `5.0` ) – Interval, in seconds, to poll the store for queue depth and rollout completion. * **`max_queue_length`** (`int`, default: `4` ) – Number of rollouts allowed to wait in the queue before throttling additional submissions. * **`span_verbosity`** (`Literal['keys', 'key_values', 'none']`, default: `'keys'` ) – Level of detail to include when logging span metadata. Raises: * `ValueError` – If `train_split` falls outside the `(0, 1)` interval. Examples: `[](https://microsoft.github.io/agent-lightning/stable/reference/algorithm/#__codelineno-0-1) from agentlightning.algorithm.fast import Baseline [](https://microsoft.github.io/agent-lightning/stable/reference/algorithm/#__codelineno-0-2) [](https://microsoft.github.io/agent-lightning/stable/reference/algorithm/#__codelineno-0-3) algorithm = Baseline(n_epochs=2, train_split=0.8, span_verbosity="key_values") [](https://microsoft.github.io/agent-lightning/stable/reference/algorithm/#__codelineno-0-4) trainer.fit(algorithm, train_dataset=my_train, val_dataset=my_val)` #### `run(store, llm_proxy, train_dataset=None, val_dataset=None)` `async` [¶](https://microsoft.github.io/agent-lightning/stable/reference/algorithm/#agentlightning.Baseline.run "Permanent link") Execute the baseline loop across the provided datasets. Adapter[¶](https://microsoft.github.io/agent-lightning/stable/reference/algorithm/#adapter "Permanent link") ------------------------------------------------------------------------------------------------------------- ### `agentlightning.Adapter` [¶](https://microsoft.github.io/agent-lightning/stable/reference/algorithm/#agentlightning.Adapter "Permanent link") Bases: `Generic[T_from, T_to]` Base class for synchronous adapters that convert data from one format to another. The class defines a minimal protocol so that adapters can be treated like callables while still allowing subclasses to supply the concrete transformation logic. Note Subclasses must override [`adapt()`](https://microsoft.github.io/agent-lightning/stable/reference/algorithm/#agentlightning.Adapter.adapt " adapt(source)") to provide the actual conversion. Type Variables: `T_from: Source data type supplied to the adapter. T_to: Target data type produced by the adapter.` Examples: `[](https://microsoft.github.io/agent-lightning/stable/reference/algorithm/#__codelineno-0-1) >>> class IntToStrAdapter(Adapter[int, str]): [](https://microsoft.github.io/agent-lightning/stable/reference/algorithm/#__codelineno-0-2) ... def adapt(self, source: int) -> str: [](https://microsoft.github.io/agent-lightning/stable/reference/algorithm/#__codelineno-0-3) ... return str(source) [](https://microsoft.github.io/agent-lightning/stable/reference/algorithm/#__codelineno-0-4) ... [](https://microsoft.github.io/agent-lightning/stable/reference/algorithm/#__codelineno-0-5) >>> adapter = IntToStrAdapter() [](https://microsoft.github.io/agent-lightning/stable/reference/algorithm/#__codelineno-0-6) >>> adapter(42) [](https://microsoft.github.io/agent-lightning/stable/reference/algorithm/#__codelineno-0-7) '42'` #### `__call__(source)` [¶](https://microsoft.github.io/agent-lightning/stable/reference/algorithm/#agentlightning.Adapter.__call__ "Permanent link") Convert the data to the target format. This method delegates to [`adapt()`](https://microsoft.github.io/agent-lightning/stable/reference/algorithm/#agentlightning.Adapter.adapt " adapt(source)") so that an instance of [`Adapter`](https://microsoft.github.io/agent-lightning/stable/reference/algorithm/#agentlightning.Adapter " agentlightning.Adapter") can be used like a standard function. Parameters: * **`source`** (`T_from`) – Input data in the source format. Returns: * `T_to` – Data converted to the target format. #### `adapt(source)` [¶](https://microsoft.github.io/agent-lightning/stable/reference/algorithm/#agentlightning.Adapter.adapt "Permanent link") Convert the data to the target format. Subclasses must override this method with the concrete transformation logic. The base implementation raises `NotImplementedError` to make the requirement explicit. Parameters: * **`source`** (`T_from`) – Input data in the source format. Returns: * `T_to` – Data converted to the target format. ### `agentlightning.TraceAdapter` [¶](https://microsoft.github.io/agent-lightning/stable/reference/algorithm/#agentlightning.TraceAdapter "Permanent link") Bases: `[Adapter](https://microsoft.github.io/agent-lightning/stable/reference/algorithm/#agentlightning.Adapter " agentlightning.Adapter (agentlightning.adapter.base.Adapter)") [Sequence[[Span](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.Span " agentlightning.Span (agentlightning.types.Span)") ], T_to]`, `Generic[T_to]` Base class for adapters that convert trace spans into other formats. This class specializes [`Adapter`](https://microsoft.github.io/agent-lightning/stable/reference/algorithm/#agentlightning.Adapter " agentlightning.Adapter") for working with [`Span`](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.Span " agentlightning.Span") instances emitted by Agent Lightning instrumentation. Subclasses receive entire trace slices and return a format suited for the downstream consumer, for example reinforcement learning training data or observability metrics. ### `agentlightning.OtelTraceAdapter` [¶](https://microsoft.github.io/agent-lightning/stable/reference/algorithm/#agentlightning.OtelTraceAdapter "Permanent link") Bases: `[Adapter](https://microsoft.github.io/agent-lightning/stable/reference/algorithm/#agentlightning.Adapter " agentlightning.Adapter (agentlightning.adapter.base.Adapter)") [Sequence[ReadableSpan], T_to]`, `Generic[T_to]` Base class for adapters that convert OpenTelemetry trace spans into other formats. This specialization of [`Adapter`](https://microsoft.github.io/agent-lightning/stable/reference/algorithm/#agentlightning.Adapter " agentlightning.Adapter") expects a list of `opentelemetry.sdk.trace.ReadableSpan` instances and produces any target format, such as reinforcement learning trajectories, structured logs, or analytics-ready payloads. Examples: `[](https://microsoft.github.io/agent-lightning/stable/reference/algorithm/#__codelineno-0-1) >>> class TraceToDictAdapter(OtelTraceAdapter[dict]): [](https://microsoft.github.io/agent-lightning/stable/reference/algorithm/#__codelineno-0-2) ... def adapt(self, spans: List[ReadableSpan]) -> dict: [](https://microsoft.github.io/agent-lightning/stable/reference/algorithm/#__codelineno-0-3) ... return {"count": len(spans)} [](https://microsoft.github.io/agent-lightning/stable/reference/algorithm/#__codelineno-0-4) ... [](https://microsoft.github.io/agent-lightning/stable/reference/algorithm/#__codelineno-0-5) >>> adapter = TraceToDictAdapter() [](https://microsoft.github.io/agent-lightning/stable/reference/algorithm/#__codelineno-0-6) >>> adapter([span1, span2]) [](https://microsoft.github.io/agent-lightning/stable/reference/algorithm/#__codelineno-0-7) {'count': 2}` ### `agentlightning.TraceToTripletBase` [¶](https://microsoft.github.io/agent-lightning/stable/reference/algorithm/#agentlightning.TraceToTripletBase "Permanent link") Bases: `[TraceAdapter](https://microsoft.github.io/agent-lightning/stable/reference/algorithm/#agentlightning.TraceAdapter " agentlightning.TraceAdapter (agentlightning.adapter.base.TraceAdapter)") [List[[Triplet](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.Triplet " agentlightning.Triplet (agentlightning.types.Triplet)") ]]` Base class for adapters that emit [`Triplet`](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.Triplet " agentlightning.Triplet") trajectories. ### `agentlightning.TracerTraceToTriplet` [¶](https://microsoft.github.io/agent-lightning/stable/reference/algorithm/#agentlightning.TracerTraceToTriplet "Permanent link") Bases: `[TraceToTripletBase](https://microsoft.github.io/agent-lightning/stable/reference/algorithm/#agentlightning.TraceToTripletBase " agentlightning.TraceToTripletBase (agentlightning.adapter.triplet.TraceToTripletBase)") ` Convert tracer-emitted spans into triplet trajectories. Attributes: * **`repair_hierarchy`** – When `True`, repair the span tree using [`TraceTree.repair_hierarchy()`](https://microsoft.github.io/agent-lightning/stable/reference/internal/#agentlightning.adapter.triplet.TraceTree.repair_hierarchy " repair_hierarchy()") before matching calls and rewards. * **`llm_call_match`** – Regular expression pattern that selects LLM call span names. * **`agent_match`** – Optional regular expression pattern for agent span names. When omitted, spans from any agent are considered. * **`exclude_llm_call_in_reward`** – When `True`, ignore matches under reward spans while searching for rewards. * **`reward_match`** – Strategy used to associate rewards with LLM calls. #### `adapt(source)` [¶](https://microsoft.github.io/agent-lightning/stable/reference/algorithm/#agentlightning.TracerTraceToTriplet.adapt "Permanent link") Convert tracer spans into [`Triplet`](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.Triplet " agentlightning.Triplet") trajectories. Parameters: * **`source`** (`Union[Sequence[[Span](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.Span " agentlightning.Span (agentlightning.types.Span)") ], Sequence[ReadableSpan]]`) – Agent Lightning spans or raw OpenTelemetry spans that form a trace. Returns: * `List[[Triplet](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.Triplet " agentlightning.Triplet (agentlightning.types.Triplet)") ]` – Ordered list of trajectory transitions with prompt, response, and reward information. #### `visualize(source, /, filename='trace_tree', interested_span_match=None)` [¶](https://microsoft.github.io/agent-lightning/stable/reference/algorithm/#agentlightning.TracerTraceToTriplet.visualize "Permanent link") Visualize the trace tree built from the supplied spans. Parameters: * **`source`** (`Union[List[[Span](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.Span " agentlightning.Span (agentlightning.types.Span)") ], List[ReadableSpan]]`) – Collection of Agent Lightning [`Span`](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.Span " agentlightning.Span") objects or raw `opentelemetry.sdk.trace.ReadableSpan` instances. * **`filename`** (`str`, default: `'trace_tree'` ) – Base filename for the generated image; `.png` is appended automatically. * **`interested_span_match`** (`str | None`, default: `None` ) – Optional regular expression used to highlight a subset of spans. Returns: * `[TraceTree](https://microsoft.github.io/agent-lightning/stable/reference/internal/#agentlightning.adapter.triplet.TraceTree " agentlightning.adapter.triplet.TraceTree") ` – The [`TraceTree`](https://microsoft.github.io/agent-lightning/stable/reference/internal/#agentlightning.adapter.triplet.TraceTree " agentlightning.adapter.triplet.TraceTree") built from the provided * `[TraceTree](https://microsoft.github.io/agent-lightning/stable/reference/internal/#agentlightning.adapter.triplet.TraceTree " agentlightning.adapter.triplet.TraceTree") ` – spans. ### `agentlightning.LlmProxyTraceToTriplet` [¶](https://microsoft.github.io/agent-lightning/stable/reference/algorithm/#agentlightning.LlmProxyTraceToTriplet "Permanent link") Bases: `[TraceToTripletBase](https://microsoft.github.io/agent-lightning/stable/reference/algorithm/#agentlightning.TraceToTripletBase " agentlightning.TraceToTripletBase (agentlightning.adapter.triplet.TraceToTripletBase)") ` Convert telemetry emitted by the LLM Proxy into triplet trajectories. Warning This adapter is experimental and might be merged with [`TracerTraceToTriplet`](https://microsoft.github.io/agent-lightning/stable/reference/algorithm/#agentlightning.TracerTraceToTriplet " agentlightning.TracerTraceToTriplet") in the future. Danger Do not rely on timestamps when using this adapter. Proxy spans can originate on different machines with unsynchronised clocks, so `sequence_id` is treated as the sole source of ordering. Strategy: 1. Sort spans by `(sequence_id, start_time)` for deterministic processing. 2. Extract token identifiers from `litellm_request` or `raw_gen_ai_request` spans. 3. Extract rewards from spans exposing AgentOps-style payloads or explicit reward spans. 4. Match each reward to the most recent unmatched LLM call whose sequence is smaller. #### `adapt(source)` [¶](https://microsoft.github.io/agent-lightning/stable/reference/algorithm/#agentlightning.LlmProxyTraceToTriplet.adapt "Permanent link") Convert LLM Proxy spans into [`Triplet`](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.Triplet " agentlightning.Triplet") trajectories. Parameters: * **`source`** (`Sequence[[Span](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.Span " agentlightning.Span (agentlightning.types.Span)") ]`) – Spans emitted by the LLM Proxy containing prompt, response, and reward data. Returns: * `List[[Triplet](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.Triplet " agentlightning.Triplet (agentlightning.types.Triplet)") ]` – Ordered trajectory transitions matched purely by `sequence_id`. ### `agentlightning.TraceToMessages` [¶](https://microsoft.github.io/agent-lightning/stable/reference/algorithm/#agentlightning.TraceToMessages "Permanent link") Bases: `[TraceAdapter](https://microsoft.github.io/agent-lightning/stable/reference/algorithm/#agentlightning.TraceAdapter " agentlightning.TraceAdapter (agentlightning.adapter.base.TraceAdapter)") [List[[OpenAIMessages](https://microsoft.github.io/agent-lightning/stable/reference/internal/#agentlightning.adapter.messages.OpenAIMessages " agentlightning.adapter.messages.OpenAIMessages") ]]` Convert trace spans into OpenAI-compatible conversation messages. The adapter reconstructs prompts, completions, tool calls, and function definitions from `gen_ai.*` span attributes. The resulting objects match the JSONL structure expected by the OpenAI fine-tuning pipeline. Warning The adapter assumes all spans share a common trace and that tool call spans are direct children of the associated completion span. #### `adapt(source)` [¶](https://microsoft.github.io/agent-lightning/stable/reference/algorithm/#agentlightning.TraceToMessages.adapt "Permanent link") Transform trace spans into OpenAI chat payloads. Parameters: * **`source`** (`Sequence[[Span](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.Span " agentlightning.Span (agentlightning.types.Span)") ]`) – Spans containing `gen_ai.*` attributes emitted by the tracing pipeline. Returns: * `List[[OpenAIMessages](https://microsoft.github.io/agent-lightning/stable/reference/internal/#agentlightning.adapter.messages.OpenAIMessages " agentlightning.adapter.messages.OpenAIMessages") ]` – A list of [`OpenAIMessages`](https://microsoft.github.io/agent-lightning/stable/reference/internal/#agentlightning.adapter.messages.OpenAIMessages " agentlightning.adapter.messages.OpenAIMessages") entries that * `List[[OpenAIMessages](https://microsoft.github.io/agent-lightning/stable/reference/internal/#agentlightning.adapter.messages.OpenAIMessages " agentlightning.adapter.messages.OpenAIMessages") ]` – capture prompts, completions, tools, and metadata. #### `get_tool_calls(completion, all_spans)` [¶](https://microsoft.github.io/agent-lightning/stable/reference/algorithm/#agentlightning.TraceToMessages.get_tool_calls "Permanent link") Yield tool call payloads for a completion span. Parameters: * **`completion`** (`[Span](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.Span " agentlightning.Span (agentlightning.types.Span)") `) – The completion span whose descendants should be inspected. * **`all_spans`** (`Sequence[[Span](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.Span " agentlightning.Span (agentlightning.types.Span)") ]`) – The complete span list belonging to the trace. Yields: * `Iterable[Dict[str, Any]]` – Dictionaries describing tool calls with identifiers, names, and arguments. Raises: * `ValueError` – If a candidate tool span cannot be converted into a dictionary. LLM Proxy[¶](https://microsoft.github.io/agent-lightning/stable/reference/algorithm/#llm-proxy "Permanent link") ----------------------------------------------------------------------------------------------------------------- ### `agentlightning.LLMProxy` [¶](https://microsoft.github.io/agent-lightning/stable/reference/algorithm/#agentlightning.LLMProxy "Permanent link") Host a LiteLLM OpenAI-compatible proxy bound to a LightningStore. The proxy: * Serves an OpenAI-compatible API via uvicorn. * Adds rollout/attempt routing and headers via middleware. * Registers OTEL export and token-id callbacks. * Writes a LiteLLM worker config file with `model_list` and settings. Lifecycle: * [`start()`](https://microsoft.github.io/agent-lightning/stable/reference/algorithm/#agentlightning.LLMProxy.start " start() async ") writes config, starts uvicorn server in a thread, and waits until ready. * [`stop()`](https://microsoft.github.io/agent-lightning/stable/reference/algorithm/#agentlightning.LLMProxy.stop " stop() async ") tears down the server and removes the temp config file. * [`restart()`](https://microsoft.github.io/agent-lightning/stable/reference/algorithm/#agentlightning.LLMProxy.restart " restart(*, _port=None) async ") convenience wrapper to stop then start. Note As the LLM Proxy sets up an OpenTelemetry tracer, it's recommended to run it in a different process from the main runner (i.e., tracer from agents). See `launch_mode` for how to change that. Warning By default (or when "stream\_conversion" middleware is enabled), the LLM Proxy will convert OpenAI and Anthropic requests with `stream=True` to a non-streaming request before going through the LiteLLM proxy. This is because the OpenTelemetry tracer provided by LiteLLM is buggy with streaming responses. You can disable this by removing the "stream\_conversion" middleware. In that case, you might lose some tracing information like token IDs. Danger Do not run LLM proxy in the same process as the main runner. It's easy to cause conflicts in the tracer provider with tracers like [`AgentOpsTracer`](https://microsoft.github.io/agent-lightning/stable/reference/runner/#agentlightning.AgentOpsTracer " agentlightning.AgentOpsTracer") . Parameters: * **`port`** (`int | None`, default: `None` ) – TCP port to bind. Will bind to a random port if not provided. * **`model_list`** (`List[[ModelConfig](https://microsoft.github.io/agent-lightning/stable/reference/internal/#agentlightning.llm_proxy.ModelConfig " agentlightning.llm_proxy.ModelConfig") ] | None`, default: `None` ) – LiteLLM `model_list` entries. * **`store`** (`Optional[[LightningStore](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.LightningStore " agentlightning.LightningStore (agentlightning.store.base.LightningStore)") ]`, default: `None` ) – LightningStore used for span sequence and persistence. * **`host`** (`str | None`, default: `None` ) – Publicly reachable host used in resource endpoints. See `host` of `launcher_args` for more details. * **`litellm_config`** (`Dict[str, Any] | None`, default: `None` ) – Extra LiteLLM proxy config merged with `model_list`. * **`num_retries`** (`int`, default: `0` ) – Default LiteLLM retry count injected into `litellm_settings`. * **`num_workers`** (`int`, default: `1` ) – Number of workers to run in the server. Only applicable for "mp" launch mode. Ignored if launcher\_args is provided. When `num_workers > 1`, the server will be run using [gunicorn](https://gunicorn.org/) . * **`launch_mode`** (`[LaunchMode](https://microsoft.github.io/agent-lightning/stable/reference/utilities/#agentlightning.utils.server_launcher.LaunchMode " agentlightning.utils.server_launcher.LaunchMode = Literal['asyncio', 'thread', 'mp'] module-attribute (agentlightning.utils.server_launcher.LaunchMode)") `, default: `'mp'` ) – Launch mode for the server. Defaults to "mp". Cannot be used together with launcher\_args. Ignored if launcher\_args is provided. It's recommended to use `launch_mode="mp"` to launch the proxy, which will launch the server in a separate process. `launch_mode="thread"` can also be used if used in caution. It will launch the server in a separate thread. `launch_mode="asyncio"` launches the server in the current thread as an asyncio task. It is NOT recommended because it often causes hanging requests. Only use it if you know what you are doing. * **`launcher_args`** (`[PythonServerLauncherArgs](https://microsoft.github.io/agent-lightning/stable/reference/utilities/#agentlightning.utils.server_launcher.PythonServerLauncherArgs " agentlightning.utils.server_launcher.PythonServerLauncherArgs dataclass ") | None`, default: `None` ) – Arguments for the server launcher. If this is provided, host, port, and launch\_mode will be ignored. Cannot be used together with port, host, and launch\_mode. * **`middlewares`** (`Sequence[Union[Type[BaseHTTPMiddleware], str]] | None`, default: `None` ) – List of FastAPI middleware classes or strings to register. You can specify the class aliases or classes that have been imported. If not provided, the default middlewares (RolloutAttemptMiddleware and StreamConversionMiddleware) will be used. Available middleware aliases are: "rollout\_attempt", "stream\_conversion", "message\_inspection". Middlewares are the **first layer** of request processing. They are applied to all requests before the LiteLLM proxy. * **`callbacks`** (`Sequence[Union[Type[CustomLogger], str]] | None`, default: `None` ) – List of LiteLLM callback classes or strings to register. You can specify the class aliases or classes that have been imported. If not provided, the default callbacks (AddReturnTokenIds and LightningOpenTelemetry) will be used. Available callback aliases are: "return\_token\_ids", "opentelemetry", "logprobs". #### `as_resource(rollout_id=None, attempt_id=None, model=None, sampling_parameters=None)` [¶](https://microsoft.github.io/agent-lightning/stable/reference/algorithm/#agentlightning.LLMProxy.as_resource "Permanent link") Create an `LLM` resource pointing at this proxy with rollout context. The returned endpoint is `http://{host}:{port}/rollout/{rollout_id}/attempt/{attempt_id}` Parameters: * **`rollout_id`** (`str | None`, default: `None` ) – Rollout identifier used for span attribution. If None, will instantiate a ProxyLLM resource. * **`attempt_id`** (`str | None`, default: `None` ) – Attempt identifier used for span attribution. If None, will instantiate a ProxyLLM resource. * **`model`** (`str | None`, default: `None` ) – Logical model name to use. If omitted and exactly one model is configured or all models have the same name, that model is used. * **`sampling_parameters`** (`Dict[str, Any] | None`, default: `None` ) – Optional default sampling parameters. Returns: * **`LLM`** ( `[LLM](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.LLM " agentlightning.LLM (agentlightning.types.LLM)") ` ) – Configured resource ready for OpenAI-compatible calls. Raises: * `ValueError` – If `model` is omitted and zero or multiple models are configured. #### `get_store()` [¶](https://microsoft.github.io/agent-lightning/stable/reference/algorithm/#agentlightning.LLMProxy.get_store "Permanent link") Get the store used by the proxy. Returns: * `Optional[[LightningStore](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.LightningStore " agentlightning.LightningStore (agentlightning.store.base.LightningStore)") ]` – The store used by the proxy. #### `initialize()` [¶](https://microsoft.github.io/agent-lightning/stable/reference/algorithm/#agentlightning.LLMProxy.initialize "Permanent link") Initialize global middleware and LiteLLM callbacks. Installs: * A FastAPI middleware that rewrites /rollout/{rid}/attempt/{aid}/... paths, injects rollout/attempt/sequence headers, and forwards downstream. * LiteLLM callbacks for token ids and OpenTelemetry export. The middleware can only be installed once because once the FastAPI app has started, the middleware cannot be changed any more. This function does not start any server. It only wires global hooks. #### `is_running()` [¶](https://microsoft.github.io/agent-lightning/stable/reference/algorithm/#agentlightning.LLMProxy.is_running "Permanent link") Return whether the uvicorn server is active. Returns: * **`bool`** ( `bool` ) – True if server was started and did not signal exit. #### `restart(*, _port=None)` `async` [¶](https://microsoft.github.io/agent-lightning/stable/reference/algorithm/#agentlightning.LLMProxy.restart "Permanent link") Restart the proxy if running, else start it. Convenience wrapper calling `stop()` followed by `start()`. #### `set_store(store)` [¶](https://microsoft.github.io/agent-lightning/stable/reference/algorithm/#agentlightning.LLMProxy.set_store "Permanent link") Set the store for the proxy. Parameters: * **`store`** (`[LightningStore](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.LightningStore " agentlightning.LightningStore (agentlightning.store.base.LightningStore)") `) – The store to use for the proxy. #### `start()` `async` [¶](https://microsoft.github.io/agent-lightning/stable/reference/algorithm/#agentlightning.LLMProxy.start "Permanent link") Start the proxy server thread and initialize global wiring. Side effects: * Sets the module-level global store for middleware/exporter access. * Calls `initialize()` once to register middleware and callbacks. * Writes a temporary YAML config consumed by LiteLLM worker. * Launches uvicorn in a daemon thread and waits for readiness. #### `stop()` `async` [¶](https://microsoft.github.io/agent-lightning/stable/reference/algorithm/#agentlightning.LLMProxy.stop "Permanent link") Stop the proxy server and clean up temporary artifacts. This is a best-effort graceful shutdown with a bounded join timeout. #### `update_model_list(model_list)` [¶](https://microsoft.github.io/agent-lightning/stable/reference/algorithm/#agentlightning.LLMProxy.update_model_list "Permanent link") Replace the in-memory model list. Parameters: * **`model_list`** (`List[[ModelConfig](https://microsoft.github.io/agent-lightning/stable/reference/internal/#agentlightning.llm_proxy.ModelConfig " agentlightning.llm_proxy.ModelConfig") ]`) – New list of model entries. Back to top --- # Contributing Guide - Agent-lightning [Skip to content](https://microsoft.github.io/agent-lightning/stable/community/contributing/#contributing-guide) ⚠️ You are viewing development documentation. For stable documentation, please visit [https://microsoft.github.io/agent-lightning/stable/](https://microsoft.github.io/agent-lightning/stable/) Contributing Guide[¶](https://microsoft.github.io/agent-lightning/stable/community/contributing/#contributing-guide "Permanent link") ====================================================================================================================================== Agent Lightning gets better every time someone files a clear bug, polishes docs, improves tests, or lands a new feature. This guide collects the expectations, checklists, and tips that help you go from “I have an idea” to “my pull request just merged.” Before You Start[¶](https://microsoft.github.io/agent-lightning/stable/community/contributing/#before-you-start "Permanent link") ---------------------------------------------------------------------------------------------------------------------------------- Agent-lightning is built by a small Microsoft Research team with limited reviewer hours and GPU budget. For any sizeable change (new algorithm, example, or API surface) please first discuss scope with us in [Discord](https://discord.gg/RYk7CdvDR7) . Early alignment keeps your effort from being blocked late in the process. Where You Can Help[¶](https://microsoft.github.io/agent-lightning/stable/community/contributing/#where-you-can-help "Permanent link") -------------------------------------------------------------------------------------------------------------------------------------- Pick a lane, or combine several. Just keep the discussion-first principle in mind for anything non-trivial. ### Documentation Improvements[¶](https://microsoft.github.io/agent-lightning/stable/community/contributing/#documentation-improvements "Permanent link") Documentation improvements are the easiest way to get started. You can find more about how to write good documentations and organize documentations in the following sections. Here are some general contribution points we can think of: * Tighten language, fix typos, clarify confusing sections, or add missing links. Fresh eyes catch docs gaps best. * Organize content using the directories listed below so readers can actually find it. * Avoid duplicate prose, unrelated “how-to” guides, or translations (we cannot maintain them today). Changes that are usually rejected * Copy/pasting existing docs with shallow edits. * Adding a `how-to` guide that is not tied to a new example. * Adding doc translations to other languages (no capacity to review/maintain yet). ### Bug Fixes[¶](https://microsoft.github.io/agent-lightning/stable/community/contributing/#bug-fixes "Permanent link") Bug fixes are the fastest way to get familiar with the codebase. To get started, you can: * Browse the ["help wanted"](https://github.com/microsoft/agent-lightning/labels/help%20wanted) and ["bug"](https://github.com/microsoft/agent-lightning/labels/bug) labels; drop a comment before you start so we can mark it as taken. * For fresh bugs, open an issue with reproduction steps, logs, and expected behavior before submitting a fix. * Keep each pull request focused, ideally avoiding breaking API changes. Larger refactors should be discussed via RFC or maintainer sync. ### New Examples[¶](https://microsoft.github.io/agent-lightning/stable/community/contributing/#new-examples "Permanent link") Examples must be curated so that we can maintain them. We generally merge only those that meet at least one (ideally several) of these criteria: * Demonstrates an agent framework or workflow that is materially different from what already exists. ([LangChain](https://www.langchain.com/) vs. [LlamaIndex](https://www.llamaindex.ai/) is not different enough; [LangChain](https://www.langchain.com/) vs. [n8n](https://n8n.io/) or [Vercel AI SDK](https://ai-sdk.dev/) is, because they either have different orchestration paradigms or differ in programming languages.) * Shows measurable performance gains on a **real-world** problem with a **real-world** dataset, such as tuning a search agent with Google Search API or improving a coding agent’s (e.g., Claude Code) SWE-Bench score. * Integrates a new algorithm, training backend, or serving stack (see “New Algorithms” below). * Validates scenarios that are rarely tested, such as multi-modality agents or long-lived memory/workflow agents. Bonus points for examples that: * Ship CI or self-test coverage so we know they still work as the core evolves. **Otherwise, we would have to mark the example as unmaintained because we won't be able to test the examples manually before each release.** * Include a [`docs/how-to/`](https://github.com/microsoft/agent-lightning/tree/3b5d733861cf313fc09821a23240bbdf3cb2ee5b/docs/how-to/) guide (or a detailed README if no how-to exists) without duplicating content in multiple places. * Favor simple, dependency-light code over heavy abstractions. * Ship a README that documents smoke-test instructions and includes an "Included Files" section summarizing every file and its role; keep the runnable module self-contained with a module-level docstring explaining CLI usage, plus targeted docstrings or inline comments for educational functions/classes. Please discuss first Examples tend to be the most time-consuming contributions for both you and reviewers. Sync with us on Discord or through an issue before diving into a new one. ### Fresh Implementations of Core Modules[¶](https://microsoft.github.io/agent-lightning/stable/community/contributing/#fresh-implementations-of-core-modules "Permanent link") If you are looking to extend [`Runner`](https://microsoft.github.io/agent-lightning/stable/reference/runner/#agentlightning.Runner " agentlightning.Runner") , [`Tracer`](https://microsoft.github.io/agent-lightning/stable/reference/runner/#agentlightning.Tracer " agentlightning.Tracer") , [`Adapter`](https://microsoft.github.io/agent-lightning/stable/reference/algorithm/#agentlightning.Adapter " agentlightning.Adapter") , [`LightningStore`](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.LightningStore " agentlightning.LightningStore") , or another core interface, here are the steps: 1. File an issue or proposal first. 2. Explain which interface you are extending, why existing implementations are insufficient, and how you intend to test compatibility with the rest of the stack (unit tests, documentation updates, example refreshes, etc.). 3. Any API changes must be reviewed up front. DO NOT begin coding large changes before the discussion lands! ### New Algorithms[¶](https://microsoft.github.io/agent-lightning/stable/community/contributing/#new-algorithms "Permanent link") If you are integrating a new training/serving backend, check whether it already lives in the [Algorithm Zoo](https://microsoft.github.io/agent-lightning/stable/algorithm-zoo/) or is covered in the [Examples Catalog](https://microsoft.github.io/agent-lightning/stable/how-to/examples-catalog/) . We especially welcome: * Currently unsupported or under-tested algorithms such as Supervised Fine-tuning (SFT), Direct Policy Optimization (DPO), or Monte Carlo Tree Search (MCTS). * Tuning [Resource](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.Resource " agentlightning.Resource") s that are not supported yet, such as workflows or memory. * Expansions of supported stacks, e.g., adding multi-modality to APO or multi-agent prompt tuning. * Reinforcement-learning integrations beyond our current stack of [VERL](https://github.com/volcengine/verl) , [vLLM](https://vllm.ai/) , [Azure OpenAI](https://azure.microsoft.com/en-us/products/ai-foundry/models/openai) , and [Tinker](https://tinker-docs.thinkingmachines.ai/) . Contributions using [SGLang](https://github.com/sgl-project/sglang) , [TRL](https://github.com/huggingface/trl) , [SkyRL](https://github.com/NovaSky-AI/SkyRL) , [RLinf](https://github.com/RLinf/RLinf) , [litgpt](https://github.com/Lightning-AI/litgpt) , or similar are welcome. Most brand-new algorithms ultimately land as “new examples,” so read that section too. Post an issue or design doc to scope the work, reuse existing utilities, and avoid duplicating efforts. Mature, battle-tested examples graduate into the [Algorithm Zoo](https://microsoft.github.io/agent-lightning/stable/algorithm-zoo/) . ### Ecosystem Projects[¶](https://microsoft.github.io/agent-lightning/stable/community/contributing/#ecosystem-projects "Permanent link") Have a project that builds on Agent-lightning but does not belong in the main repo? Fork it or depend on it externally, then let us know. We can showcase notable projects in [Community Projects](https://microsoft.github.io/agent-lightning/stable/) and the main [README](https://github.com/microsoft/agent-lightning/tree/3b5d733861cf313fc09821a23240bbdf3cb2ee5b/README.md) . ### Agent-lightning Contrib[¶](https://microsoft.github.io/agent-lightning/stable/community/contributing/#agent-lightning-contrib "Permanent link") [`contrib/`](https://github.com/microsoft/agent-lightning/tree/3b5d733861cf313fc09821a23240bbdf3cb2ee5b/contrib) is where work-in-progress or third-party integrations, and curated recipes live before they are hardened enough for the core runtime tree. Think of it as an incubator: additions should remain easy to consume, clearly owned, and scoped so downstream users can vendor them with minimal risk. The following types of contributions are welcome in the contrib area: * **Recipes** that assemble multiple Agent Lightning components for a narrow task (`contrib/recipes//`). Each recipe must be self-contained, include running instructions and result reports. * **Runtime extensions** that would bloat the primary `agentlightning/` namespace (`contrib/agentlightning/contrib//`). These should mirror the published wheel layout so that `import agentlightning.contrib.` works out of the box. * **Supporting scripts and assets** (`contrib/scripts/`) that automate dataset downloads, environment preparation, or benchmarks required by contrib modules. If you are unsure where a contribution should live, start a thread in Discord or open an issue before writing code. The [contrib README](https://github.com/microsoft/agent-lightning/tree/3b5d733861cf313fc09821a23240bbdf3cb2ee5b/contrib/README.md) also lists the directory expectations. A quick checklist for contributions to be accepted: 1. **Document everything.** Include configuration steps, environment variables, and sample commands so contributors can reproduce the results without guesswork. Pin to a specific version of Agent-lightning and other dependencies to avoid unexpected changes if you don't want to update the recipe frequently. 2. **Keep quality predictable.** Match the repo’s style guide, apply exhaustive type hints, and run `uv run --no-sync pyright` plus targeted `pytest` suites for any Python module you touch. 3. **Ship reproducibility artifacts.** Store only scripts or instructions for downloading datasets, weights, or binaries. Never upload large artifacts or credentials directly. 4. **Update ownership.** Add `CODEOWNERS` entries when new directories appear so maintainers know who can review follow-up fixes. Contrib entries do not need the same maturity level as core code, but they must still meet the baseline above. Submissions that lack documentation, hide ownership, or depend on untracked assets are typically rejected until those gaps are resolved. ### Other Contribution Ideas[¶](https://microsoft.github.io/agent-lightning/stable/community/contributing/#other-contribution-ideas "Permanent link") * **Tests.** Add or improve cases in [`tests/`](https://github.com/microsoft/agent-lightning/tree/3b5d733861cf313fc09821a23240bbdf3cb2ee5b/tests) (unit, integration, or end-to-end). * **Benchmarks.** Expand [`tests/benchmark`](https://github.com/microsoft/agent-lightning/tree/3b5d733861cf313fc09821a23240bbdf3cb2ee5b/tests/benchmark) to stress large-scale training or rollouts. * **Issue triage.** Reproduce bugs, confirm whether they reproduce on `main`, or suggest short-term mitigations so maintainers can prioritize. Contribution Workflow[¶](https://microsoft.github.io/agent-lightning/stable/community/contributing/#contribution-workflow "Permanent link") -------------------------------------------------------------------------------------------------------------------------------------------- The steps below keep changes reviewable and CI-friendly. Follow them in order; rerun the relevant pieces if you revisit a branch later. ### 1\. Prepare Your Environment[¶](https://microsoft.github.io/agent-lightning/stable/community/contributing/#1-prepare-your-environment "Permanent link") Minimum tooling: * **Python** 3.10+ (3.12 recommended). * **uv** for dependency and virtual-environment management. Install it using the [official uv docs](https://docs.astral.sh/uv/getting-started/installation/) . * **Git** configured with your GitHub credentials. Clone your fork and point `upstream` at the official repo: `[](https://microsoft.github.io/agent-lightning/stable/community/contributing/#__codelineno-0-1) git clone git@github.com:/agent-lightning.git [](https://microsoft.github.io/agent-lightning/stable/community/contributing/#__codelineno-0-2) cd agent-lightning [](https://microsoft.github.io/agent-lightning/stable/community/contributing/#__codelineno-0-3) git remote add upstream https://github.com/microsoft/agent-lightning.git` Install the default development stack: `[](https://microsoft.github.io/agent-lightning/stable/community/contributing/#__codelineno-1-1) uv sync --group dev` Need GPU extras or specific optional dependencies? Lock them in with one command: `[](https://microsoft.github.io/agent-lightning/stable/community/contributing/#__codelineno-2-1) uv sync --frozen \ [](https://microsoft.github.io/agent-lightning/stable/community/contributing/#__codelineno-2-2) --extra apo \ [](https://microsoft.github.io/agent-lightning/stable/community/contributing/#__codelineno-2-3) --extra verl \ [](https://microsoft.github.io/agent-lightning/stable/community/contributing/#__codelineno-2-4) --group dev \ [](https://microsoft.github.io/agent-lightning/stable/community/contributing/#__codelineno-2-5) --group torch-cpu \ [](https://microsoft.github.io/agent-lightning/stable/community/contributing/#__codelineno-2-6) --group torch-stable \ [](https://microsoft.github.io/agent-lightning/stable/community/contributing/#__codelineno-2-7) --group agents \ [](https://microsoft.github.io/agent-lightning/stable/community/contributing/#__codelineno-2-8) --no-default-groups` After `uv sync`, run commands via `uv run ...` (add `--no-sync` once the environment is locked) or activate `.venv/`. ### 2\. Install and Run Pre-commit[¶](https://microsoft.github.io/agent-lightning/stable/community/contributing/#2-install-and-run-pre-commit "Permanent link") Formatting and linting are enforced through [pre-commit](https://pre-commit.com/) . Install once, then run before each push: `[](https://microsoft.github.io/agent-lightning/stable/community/contributing/#__codelineno-3-1) uv run --no-sync pre-commit install [](https://microsoft.github.io/agent-lightning/stable/community/contributing/#__codelineno-3-2) uv run --no-sync pre-commit run --all-files --show-diff-on-failure --color=always` Once installed, the hooks run automatically on every `git commit`. Running the pre-commit hooks locally keeps CI green and diffs manageable. ### 3\. Branch from Fresh `main` and Code[¶](https://microsoft.github.io/agent-lightning/stable/community/contributing/#3-branch-from-fresh-main-and-code "Permanent link") Start all work from the latest upstream state: `[](https://microsoft.github.io/agent-lightning/stable/community/contributing/#__codelineno-4-1) git fetch upstream [](https://microsoft.github.io/agent-lightning/stable/community/contributing/#__codelineno-4-2) git checkout main [](https://microsoft.github.io/agent-lightning/stable/community/contributing/#__codelineno-4-3) git merge upstream/main` Branch naming convention: * `feature/` for new features. * `fix/` for bug fixes. * `docs/` for documentation-only updates. * `chore/` for tooling or maintenance. Use lowercase with hyphens, e.g., `feature/async-runner-hooks`. Where should docs or examples live? Many new contributors get confused about what to put in the `docs/how-to/` directory and what to put in the `examples/` directory (particularly README files). Here is a quick reference you can refer to: | Location | Description | | --- | --- | | `docs/algorithm-zoo/` | Documentation for **built-in algorithms** shipped with Agent-lightning. | | `docs/how-to/` | Step-by-step **how-to guides**, usually tied to an example in `examples/`. | | `docs/tutorials/` | Conceptual walkthroughs for components or workflows. See [debugging](https://microsoft.github.io/agent-lightning/stable/tutorials/debug/)
or [parallelization](https://microsoft.github.io/agent-lightning/stable/tutorials/parallelize/)
for examples. | | `docs/deep-dive/` | Advanced explanations and in-depth concepts. | | `examples//README.md` | Example-specific README. If any related how-to if that exists, link to it avoid duplicating the same instructions twice; write only brief instructions on how to install and run the example. Otherwise, you can make the README more detailed and self-explanatory. | Remember to register new docs in [`mkdocs.yml`](https://github.com/microsoft/agent-lightning/tree/3b5d733861cf313fc09821a23240bbdf3cb2ee5b/mkdocs.yml) , add examples to [examples/README](https://github.com/microsoft/agent-lightning/tree/3b5d733861cf313fc09821a23240bbdf3cb2ee5b/examples/README.md) , and update the [Examples Catalog](https://microsoft.github.io/agent-lightning/stable/how-to/examples-catalog/) . Before you start coding, bring the shared coding conventions with you: * Target `requires-python >= 3.10`, four-space indentation, ~120-character lines (docstrings may run longer), and formatter-owned diffs (Black + isort with the `black` profile). * Use `snake_case` for modules, functions, and variables; `PascalCase` for classes and React components; lowercase hyphenation for CLI flags, branch names, and TypeScript filenames. * Maintain exhaustive type hints (pyright enforces them), write succinct Google-style docstrings (with `[][]` cross-references). * Prefer dataclasses or Pydantic models from `agentlightning.types`. * Log via `logging.getLogger(__name__)` with targeted DEBUG/INFO/WARNING/ERROR calls—especially for long multi-step functions or broad `try/except` blocks. ### 4\. Test and Validate[¶](https://microsoft.github.io/agent-lightning/stable/community/contributing/#4-test-and-validate "Permanent link") Most contributions require automated checks. Once `uv sync` locks dependencies, prefix commands with `uv run --no-sync ...` so they share the same environment as CI. **Full test suite** `[](https://microsoft.github.io/agent-lightning/stable/community/contributing/#__codelineno-5-1) uv run --no-sync pytest -v` **Targeted tests** `[](https://microsoft.github.io/agent-lightning/stable/community/contributing/#__codelineno-6-1) uv run --no-sync pytest tests/path/to/test_file.py -k test_name` **Optional/gated tests:** GPU-specific suites or API-dependent tests run automatically when the required hardware or environment variables (such as `OPENAI_API_KEY`) are present. **Static analysis:** `[](https://microsoft.github.io/agent-lightning/stable/community/contributing/#__codelineno-7-1) uv run --no-sync pyright` If you have touched code under `examples/`, you should run the example-specific smoke tests. Each directory includes a README with example-specific smoke tests—run those too. Build documentation when needed Keep API references under [docs/reference](https://github.com/microsoft/agent-lightning/tree/3b5d733861cf313fc09821a23240bbdf3cb2ee5b/docs/reference/) up to date. Doc-only changes should still build cleanly: `[](https://microsoft.github.io/agent-lightning/stable/community/contributing/#__codelineno-8-1) uv run --no-sync mkdocs serve --strict # live reload [](https://microsoft.github.io/agent-lightning/stable/community/contributing/#__codelineno-8-2) uv run --no-sync mkdocs build --strict # CI-equivalent` `--strict` elevates warnings to errors so you catch issues before CI. Before opening a PR, double-check the basics: * Run `uv lock` if you changed dependencies. * Run `uv run --no-sync pre-commit run --all-files --show-diff-on-failure` (hooks installed via `pre-commit install` run automatically on `git commit`, but rerun them if you amended history). * Execute the relevant commands from the test list above. * Validate each affected example via its README instructions. ### 5\. Open a Pull Request[¶](https://microsoft.github.io/agent-lightning/stable/community/contributing/#5-open-a-pull-request "Permanent link") 1. Push your branch: `[](https://microsoft.github.io/agent-lightning/stable/community/contributing/#__codelineno-9-1) git push origin ` 2. Open a PR against `microsoft/agent-lightning:main`. 3. Fill out the template with a concise summary, the commands/tests you ran, and linked issues (use `Fixes #123` syntax to auto-close). 4. Include screenshots or logs if they clarify behavior. 5. Address review feedback promptly. Follow-up tweaks work best as focused commits; `git commit --fixup` is handy for reviewer-suggested edits. Thanks for contributing! every improvement strengthens the Agent Lightning community! Back to top --- # Runner - Agent-lightning [Skip to content](https://microsoft.github.io/agent-lightning/stable/reference/runner/#runner-side-references) ⚠️ You are viewing development documentation. For stable documentation, please visit [https://microsoft.github.io/agent-lightning/stable/](https://microsoft.github.io/agent-lightning/stable/) Runner-side References[¶](https://microsoft.github.io/agent-lightning/stable/reference/runner/#runner-side-references "Permanent link") ======================================================================================================================================== Note This reference covers APIs that are designed to be used at "Runner Side". Runners[¶](https://microsoft.github.io/agent-lightning/stable/reference/runner/#runners "Permanent link") ---------------------------------------------------------------------------------------------------------- ### `agentlightning.LitAgentRunner` [¶](https://microsoft.github.io/agent-lightning/stable/reference/runner/#agentlightning.LitAgentRunner "Permanent link") Bases: `[Runner](https://microsoft.github.io/agent-lightning/stable/reference/runner/#agentlightning.Runner " agentlightning.Runner (agentlightning.runner.base.Runner)") [T_task]` Execute [`LitAgent`](https://microsoft.github.io/agent-lightning/stable/reference/agent/#agentlightning.LitAgent " agentlightning.LitAgent") tasks with tracing support. This runner manages the complete lifecycle of agent rollout execution, including task polling, resource management, tracing, and hooks. It supports both continuous iteration over tasks from the store and single-step execution. Attributes: * **`worker_id`** (`Optional[int]`) – Identifier for the active worker process, if any. #### `tracer` `property` [¶](https://microsoft.github.io/agent-lightning/stable/reference/runner/#agentlightning.LitAgentRunner.tracer "Permanent link") Get the tracer instance. Returns: * `[Tracer](https://microsoft.github.io/agent-lightning/stable/reference/runner/#agentlightning.Tracer " agentlightning.Tracer (agentlightning.tracer.base.Tracer)") ` – The Tracer instance used by this runner. #### `__init__(tracer, max_rollouts=None, poll_interval=5.0, heartbeat_interval=10.0, interval_jitter=0.5, heartbeat_launch_mode='thread', heartbeat_include_gpu=False)` [¶](https://microsoft.github.io/agent-lightning/stable/reference/runner/#agentlightning.LitAgentRunner.__init__ "Permanent link") Initialize the agent runner. Parameters: * **`tracer`** (`[Tracer](https://microsoft.github.io/agent-lightning/stable/reference/runner/#agentlightning.Tracer " agentlightning.Tracer (agentlightning.tracer.base.Tracer)") `) – [`Tracer`](https://microsoft.github.io/agent-lightning/stable/reference/runner/#agentlightning.Tracer " agentlightning.Tracer") used for rollout spans. * **`max_rollouts`** (`Optional[int]`, default: `None` ) – Optional cap on iterations processed by [`iter`](https://microsoft.github.io/agent-lightning/stable/reference/runner/#agentlightning.LitAgentRunner.iter " iter(*, event=None) async ") . * **`poll_interval`** (`float`, default: `5.0` ) – Seconds to wait between store polls when no work is available. * **`heartbeat_interval`** (`float`, default: `10.0` ) – Seconds to wait between sending heartbeats to the store. * **`interval_jitter`** (`float`, default: `0.5` ) – Jitter factor for the poll interval. The actual interval will be between poll\_interval - interval\_jitter and poll\_interval + interval\_jitter. This is to avoid the overload caused by the synchronization of the runners. * **`heartbeat_launch_mode`** (`Literal['asyncio', 'thread']`, default: `'thread'` ) – Launch mode for the heartbeat loop. Can be "asyncio" or "thread". "thread" is the default and recommended mode as it prevents blocking the event loop under load. Use "asyncio" for simpler deployments with low worker counts. * **`heartbeat_include_gpu`** (`bool`, default: `False` ) – Whether to include GPU stats in heartbeat snapshots. Querying GPU stats can be slow under load, so this is disabled by default. #### `get_agent()` [¶](https://microsoft.github.io/agent-lightning/stable/reference/runner/#agentlightning.LitAgentRunner.get_agent "Permanent link") Get the agent instance. Returns: * `[LitAgent](https://microsoft.github.io/agent-lightning/stable/reference/agent/#agentlightning.LitAgent " agentlightning.LitAgent (agentlightning.litagent.LitAgent)") [T_task]` – The LitAgent instance managed by this runner. Raises: * `ValueError` – If the agent has not been initialized via [`init`](https://microsoft.github.io/agent-lightning/stable/reference/runner/#agentlightning.LitAgentRunner.init " init(agent, *, hooks=None, **kwargs)") . #### `get_store()` [¶](https://microsoft.github.io/agent-lightning/stable/reference/runner/#agentlightning.LitAgentRunner.get_store "Permanent link") Get the store instance. Returns: * `[LightningStore](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.LightningStore " agentlightning.LightningStore (agentlightning.store.base.LightningStore)") ` – The LightningStore instance for this worker. Raises: * `ValueError` – If the store has not been initialized via [`init_worker`](https://microsoft.github.io/agent-lightning/stable/reference/runner/#agentlightning.LitAgentRunner.init_worker " init_worker(worker_id, store, **kwargs)") . #### `get_worker_id()` [¶](https://microsoft.github.io/agent-lightning/stable/reference/runner/#agentlightning.LitAgentRunner.get_worker_id "Permanent link") Get the formatted worker ID string. Returns: * `str` – A formatted string like "Worker-0" if initialized, or "Worker-Unknown" * `str` – if the worker ID has not been set. #### `init(agent, *, hooks=None, **kwargs)` [¶](https://microsoft.github.io/agent-lightning/stable/reference/runner/#agentlightning.LitAgentRunner.init "Permanent link") Initialize the runner with the agent. This sets up the agent-runner relationship, registers hooks, and initializes the tracer. Parameters: * **`agent`** (`[LitAgent](https://microsoft.github.io/agent-lightning/stable/reference/agent/#agentlightning.LitAgent " agentlightning.LitAgent (agentlightning.litagent.LitAgent)") [T_task]`) – [`LitAgent`](https://microsoft.github.io/agent-lightning/stable/reference/agent/#agentlightning.LitAgent " agentlightning.LitAgent") instance executed by the runner. * **`hooks`** (`Optional[Sequence[[Hook](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.Hook " agentlightning.Hook (agentlightning.types.Hook)") ]]`, default: `None` ) – Optional sequence of [`Hook`](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.Hook " agentlightning.Hook") callbacks invoked around tracing and rollout boundaries. * **`**kwargs`** (`Any`, default: `{}` ) – Additional initialization arguments (currently unused). #### `init_worker(worker_id, store, **kwargs)` [¶](https://microsoft.github.io/agent-lightning/stable/reference/runner/#agentlightning.LitAgentRunner.init_worker "Permanent link") Initialize the runner for each worker with worker\_id and store. This method is called once per worker in a distributed setup to provide the worker with its ID and store connection. Parameters: * **`worker_id`** (`int`) – Unique identifier for this worker process. * **`store`** (`[LightningStore](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.LightningStore " agentlightning.LightningStore (agentlightning.store.base.LightningStore)") `) – [`LightningStore`](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.LightningStore " agentlightning.LightningStore") used for task coordination and persistence. * **`**kwargs`** (`Any`, default: `{}` ) – Additional worker-specific initialization arguments (currently unused). #### `iter(*, event=None)` `async` [¶](https://microsoft.github.io/agent-lightning/stable/reference/runner/#agentlightning.LitAgentRunner.iter "Permanent link") Run the runner, continuously iterating over tasks in the store. This method polls the store for new rollouts and executes them until: * The event is set (if provided) * The max\_rollouts limit is reached (if configured) * No more tasks are available All exceptions during rollout execution are caught and logged but not propagated, allowing the runner to continue processing subsequent tasks. Parameters: * **`event`** (`Optional[[ExecutionEvent](https://microsoft.github.io/agent-lightning/stable/reference/trainer/#agentlightning.ExecutionEvent " agentlightning.ExecutionEvent (agentlightning.execution.events.ExecutionEvent)") ]`, default: `None` ) – Optional ExecutionEvent object to signal the runner to stop. The runner will check this event periodically and stop gracefully when set. #### `step(input, *, resources=None, mode=None, event=None)` `async` [¶](https://microsoft.github.io/agent-lightning/stable/reference/runner/#agentlightning.LitAgentRunner.step "Permanent link") Execute a single task directly, bypassing the task queue. This method creates a new rollout for the given input and executes it immediately. Unlike [`iter()`](https://microsoft.github.io/agent-lightning/stable/reference/runner/#agentlightning.LitAgentRunner.iter " iter(*, event=None) async ") , exceptions are propagated to the caller. Parameters: * **`input`** (`T_task`) – The task input to be processed by the agent. * **`resources`** (`Optional[[NamedResources](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.NamedResources " agentlightning.NamedResources = Dict[str, ResourceUnion] module-attribute (agentlightning.types.NamedResources)") ]`, default: `None` ) – Optional named resources to be used for this specific task. If provided, a new resources entry will be created in the store. If not provided, the latest resources from the store will be used. * **`mode`** (`Optional[[RolloutMode](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.RolloutMode " agentlightning.RolloutMode = Literal['train', 'val', 'test'] module-attribute (agentlightning.types.RolloutMode)") ]`, default: `None` ) – Optional rollout mode ("train" or "validation"). If not provided, the agent's default mode will be used. * **`event`** (`Optional[[ExecutionEvent](https://microsoft.github.io/agent-lightning/stable/reference/trainer/#agentlightning.ExecutionEvent " agentlightning.ExecutionEvent (agentlightning.execution.events.ExecutionEvent)") ]`, default: `None` ) – Optional ExecutionEvent object to signal interruption (currently unused but included for interface consistency). Returns: * `[Rollout](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.Rollout " agentlightning.Rollout (agentlightning.types.Rollout)") ` – The completed rollout. Raises: * `Exception` – Any exception that occurs during rollout execution will be re-raised to the caller. #### `teardown(*args, **kwargs)` [¶](https://microsoft.github.io/agent-lightning/stable/reference/runner/#agentlightning.LitAgentRunner.teardown "Permanent link") Teardown the runner and clean up all resources. This method resets all internal state including the agent, store, hooks, and worker ID, and calls the tracer's teardown method. Parameters: * **`*args`** (`Any`, default: `()` ) – Additional teardown arguments (currently unused). * **`**kwargs`** (`Any`, default: `{}` ) – Additional teardown keyword arguments (currently unused). #### `teardown_worker(worker_id, *args, **kwargs)` [¶](https://microsoft.github.io/agent-lightning/stable/reference/runner/#agentlightning.LitAgentRunner.teardown_worker "Permanent link") Teardown the runner for a specific worker. This method cleans up worker-specific resources and resets the worker ID. Parameters: * **`worker_id`** (`int`) – Unique identifier of the worker being torn down. * **`*args`** (`Any`, default: `()` ) – Additional teardown arguments (currently unused). * **`**kwargs`** (`Any`, default: `{}` ) – Additional teardown keyword arguments (currently unused). ### `agentlightning.Runner` [¶](https://microsoft.github.io/agent-lightning/stable/reference/runner/#agentlightning.Runner "Permanent link") Bases: `[ParallelWorkerBase](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.ParallelWorkerBase " agentlightning.ParallelWorkerBase (agentlightning.types.ParallelWorkerBase)") `, `Generic[T_task]` Abstract base class for long-running agent executors. Runner implementations coordinate [`LitAgent`](https://microsoft.github.io/agent-lightning/stable/reference/agent/#agentlightning.LitAgent " agentlightning.LitAgent") instances, acquire work from a [`LightningStore`](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.LightningStore " agentlightning.LightningStore") , and emit [`Rollout`](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.Rollout " agentlightning.Rollout") objects. Subclasses decide how to schedule work (polling, streaming, etc.) while this base class provides a minimal lifecycle contract. #### `init(agent, **kwargs)` [¶](https://microsoft.github.io/agent-lightning/stable/reference/runner/#agentlightning.Runner.init "Permanent link") Prepare the runner to execute tasks for `agent`. This method is called only once during the setup for all workers, not for each worker. Parameters: * **`agent`** (`[LitAgent](https://microsoft.github.io/agent-lightning/stable/reference/agent/#agentlightning.LitAgent " agentlightning.LitAgent (agentlightning.litagent.LitAgent)") [T_task]`) – Agent instance providing task-specific logic. * **`**kwargs`** (`Any`, default: `{}` ) – Optional runner-specific configuration. Raises: * `NotImplementedError` – Subclasses must supply the initialization routine. #### `init_worker(worker_id, store, **kwargs)` [¶](https://microsoft.github.io/agent-lightning/stable/reference/runner/#agentlightning.Runner.init_worker "Permanent link") Configure worker-local state before processing tasks. This method is called for **each** worker during the setup. Parameters: * **`worker_id`** (`int`) – Unique identifier for this worker process or thread. * **`store`** (`[LightningStore](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.LightningStore " agentlightning.LightningStore (agentlightning.store.base.LightningStore)") `) – Shared [`LightningStore`](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.LightningStore " agentlightning.LightningStore") backing task coordination. * **`**kwargs`** (`Any`, default: `{}` ) – Optional worker-specific configuration. Raises: * `NotImplementedError` – Subclasses must prepare per-worker resources. #### `iter(*, event=None)` `async` [¶](https://microsoft.github.io/agent-lightning/stable/reference/runner/#agentlightning.Runner.iter "Permanent link") Run the runner, continuously iterating over tasks in the store. This method runs in a loop, polling the store for new tasks and executing them until interrupted by the event or when no more tasks are available. Parameters: * **`event`** (`Optional[[ExecutionEvent](https://microsoft.github.io/agent-lightning/stable/reference/trainer/#agentlightning.ExecutionEvent " agentlightning.ExecutionEvent (agentlightning.execution.events.ExecutionEvent)") ]`, default: `None` ) – Cooperative stop signal. When set, the runner should complete the current unit of work and exit the loop. Raises: * `NotImplementedError` – Subclasses provide the iteration behavior. #### `run(*args, **kwargs)` [¶](https://microsoft.github.io/agent-lightning/stable/reference/runner/#agentlightning.Runner.run "Permanent link") Deprecated synchronous entry point. Use [`iter()`](https://microsoft.github.io/agent-lightning/stable/reference/runner/#agentlightning.Runner.iter " iter(*, event=None) async ") or [`step()`](https://microsoft.github.io/agent-lightning/stable/reference/runner/#agentlightning.Runner.step " step(input, *, resources=None, mode=None, event=None) async ") instead. Raises: * `RuntimeError` – Always raised to direct callers to [iter()](https://microsoft.github.io/agent-lightning/stable/reference/runner/#agentlightning.Runner.iter " iter(*, event=None) async ") or [step()](https://microsoft.github.io/agent-lightning/stable/reference/runner/#agentlightning.Runner.step " step(input, *, resources=None, mode=None, event=None) async ") . #### `run_context(*, agent, store, hooks=None, worker_id=None)` [¶](https://microsoft.github.io/agent-lightning/stable/reference/runner/#agentlightning.Runner.run_context "Permanent link") Initialize and tear down a runner within a simple context manager. The helper is primarily intended for debugging runner implementations outside of a full [`Trainer`](https://microsoft.github.io/agent-lightning/stable/reference/trainer/#agentlightning.Trainer " agentlightning.Trainer") stack. Parameters: * **`agent`** (`[LitAgent](https://microsoft.github.io/agent-lightning/stable/reference/agent/#agentlightning.LitAgent " agentlightning.LitAgent (agentlightning.litagent.LitAgent)") [T_task]`) – Agent executed by this runner. * **`store`** (`[LightningStore](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.LightningStore " agentlightning.LightningStore (agentlightning.store.base.LightningStore)") `) – Backing [`LightningStore`](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.LightningStore " agentlightning.LightningStore") . If you don't have one, you can easily create one with [`InMemoryLightningStore`](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.InMemoryLightningStore " agentlightning.InMemoryLightningStore") . * **`hooks`** (`Optional[Sequence[[Hook](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.Hook " agentlightning.Hook (agentlightning.types.Hook)") ]]`, default: `None` ) – Optional sequence of hooks recognised by the runner. Not all runners support hooks. * **`worker_id`** (`Optional[int]`, default: `None` ) – Override the worker identifier used during setup. Defaults to `0`. #### `step(input, *, resources=None, mode=None, event=None)` `async` [¶](https://microsoft.github.io/agent-lightning/stable/reference/runner/#agentlightning.Runner.step "Permanent link") Execute a single task with the given input. This method provides fine-grained control for executing individual tasks directly, bypassing the store's task queue. Parameters: * **`input`** (`T_task`) – Task payload consumed by the agent. * **`resources`** (`Optional[[NamedResources](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.NamedResources " agentlightning.NamedResources = Dict[str, ResourceUnion] module-attribute (agentlightning.types.NamedResources)") ]`, default: `None` ) – Optional named resources scoped to this invocation. * **`mode`** (`Optional[[RolloutMode](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.RolloutMode " agentlightning.RolloutMode = Literal['train', 'val', 'test'] module-attribute (agentlightning.types.RolloutMode)") ]`, default: `None` ) – Optional rollout mode such as `"train"` or `"eval"`. * **`event`** (`Optional[[ExecutionEvent](https://microsoft.github.io/agent-lightning/stable/reference/trainer/#agentlightning.ExecutionEvent " agentlightning.ExecutionEvent (agentlightning.execution.events.ExecutionEvent)") ]`, default: `None` ) – Cooperative stop signal for long-running tasks. Returns: * `[Rollout](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.Rollout " agentlightning.Rollout (agentlightning.types.Rollout)") ` – Completed rollout produced by the agent. Raises: * `NotImplementedError` – Subclasses provide the execution behavior. #### `teardown(*args, **kwargs)` [¶](https://microsoft.github.io/agent-lightning/stable/reference/runner/#agentlightning.Runner.teardown "Permanent link") Release resources acquired during [`init()`](https://microsoft.github.io/agent-lightning/stable/reference/runner/#agentlightning.Runner.init " init(agent, **kwargs)") . Raises: * `NotImplementedError` – Subclasses must implement the shutdown routine. #### `teardown_worker(worker_id, *args, **kwargs)` [¶](https://microsoft.github.io/agent-lightning/stable/reference/runner/#agentlightning.Runner.teardown_worker "Permanent link") Release per-worker resources allocated by [`init_worker()`](https://microsoft.github.io/agent-lightning/stable/reference/runner/#agentlightning.Runner.init_worker " init_worker(worker_id, store, **kwargs)") . Parameters: * **`worker_id`** (`int`) – Identifier of the worker being torn down. Raises: * `NotImplementedError` – Subclasses must implement the shutdown routine. Tracer[¶](https://microsoft.github.io/agent-lightning/stable/reference/runner/#tracer "Permanent link") -------------------------------------------------------------------------------------------------------- ### `agentlightning.AgentOpsTracer` [¶](https://microsoft.github.io/agent-lightning/stable/reference/runner/#agentlightning.AgentOpsTracer "Permanent link") Bases: `[OtelTracer](https://microsoft.github.io/agent-lightning/stable/reference/runner/#agentlightning.OtelTracer " agentlightning.OtelTracer (agentlightning.tracer.otel.OtelTracer)") ` Traces agent execution using AgentOps. This tracer provides functionality to capture execution details using the AgentOps library. It manages the AgentOps client initialization, server setup, and integration with the OpenTelemetry tracing ecosystem. Attributes: * **`agentops_managed`** – Whether to automatically manage `agentops`. When set to true, tracer calls `agentops.init()` automatically and launches an agentops endpoint locally. If not, you are responsible for calling and using it before using the tracer. * **`instrument_managed`** – Whether to automatically manage instrumentation. When set to false, you will manage the instrumentation yourself and the tracer might not work as expected. * **`daemon`** – Whether the AgentOps server runs as a daemon process. Only applicable if `agentops_managed` is True. #### `get_langchain_handler(tags=None)` [¶](https://microsoft.github.io/agent-lightning/stable/reference/runner/#agentlightning.AgentOpsTracer.get_langchain_handler "Permanent link") Get the Langchain callback handler for integrating with Langchain. Parameters: * **`tags`** (`List[str] | None`, default: `None` ) – Optional list of tags to apply to the Langchain callback handler. Returns: * `LangchainCallbackHandler` – An instance of the Langchain callback handler. #### `trace_context(name=None, *, store=None, rollout_id=None, attempt_id=None)` `async` [¶](https://microsoft.github.io/agent-lightning/stable/reference/runner/#agentlightning.AgentOpsTracer.trace_context "Permanent link") Starts a new tracing context. This should be used as a context manager. Parameters: * **`name`** (`Optional[str]`, default: `None` ) – Optional name for the tracing context. * **`store`** (`Optional[[LightningStore](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.LightningStore " agentlightning.LightningStore (agentlightning.store.base.LightningStore)") ]`, default: `None` ) – Optional store to add the spans to. * **`rollout_id`** (`Optional[str]`, default: `None` ) – Optional rollout ID to add the spans to. * **`attempt_id`** (`Optional[str]`, default: `None` ) – Optional attempt ID to add the spans to. Yields: * `AsyncGenerator[Tracer, None]` – The OpenTelemetry tracer instance to collect spans. ### `agentlightning.OtelTracer` [¶](https://microsoft.github.io/agent-lightning/stable/reference/runner/#agentlightning.OtelTracer "Permanent link") Bases: `[Tracer](https://microsoft.github.io/agent-lightning/stable/reference/runner/#agentlightning.Tracer " agentlightning.Tracer (agentlightning.tracer.base.Tracer)") ` Tracer that provides a basic OpenTelemetry tracer provider. You should be able to collect agent-lightning signals like rewards with this tracer, but no other function instrumentations like `openai.chat.completion`. #### `get_last_trace()` [¶](https://microsoft.github.io/agent-lightning/stable/reference/runner/#agentlightning.OtelTracer.get_last_trace "Permanent link") Retrieves the raw list of captured spans from the most recent trace. Returns: * `List[[Span](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.Span " agentlightning.Span (agentlightning.types.Span)") ]` – A list of [`Span`](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.Span " agentlightning.Span") objects captured during the most recent trace. #### `trace_context(name=None, *, store=None, rollout_id=None, attempt_id=None)` `async` [¶](https://microsoft.github.io/agent-lightning/stable/reference/runner/#agentlightning.OtelTracer.trace_context "Permanent link") Starts a new tracing context. This should be used as a context manager. Parameters: * **`name`** (`Optional[str]`, default: `None` ) – Optional name for the tracing context. * **`store`** (`Optional[[LightningStore](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.LightningStore " agentlightning.LightningStore (agentlightning.store.base.LightningStore)") ]`, default: `None` ) – Optional store to add the spans to. * **`rollout_id`** (`Optional[str]`, default: `None` ) – Optional rollout ID to add the spans to. * **`attempt_id`** (`Optional[str]`, default: `None` ) – Optional attempt ID to add the spans to. Yields: * `AsyncGenerator[Tracer, None]` – The OpenTelemetry tracer instance to collect spans. ### `agentlightning.Tracer` [¶](https://microsoft.github.io/agent-lightning/stable/reference/runner/#agentlightning.Tracer "Permanent link") Bases: `[ParallelWorkerBase](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.ParallelWorkerBase " agentlightning.ParallelWorkerBase (agentlightning.types.ParallelWorkerBase)") ` An abstract base class for tracers. This class defines a standard interface for tracing code execution, capturing the resulting spans, and providing them for analysis. It is designed to be backend-agnostic, allowing for different implementations (e.g., for AgentOps, OpenTelemetry, Docker, etc.). The primary interaction pattern is through the [`trace_context`](https://microsoft.github.io/agent-lightning/stable/reference/runner/#agentlightning.Tracer.trace_context " trace_context(name=None, *, store=None, rollout_id=None, attempt_id=None)") context manager, which ensures that traces are properly started and captured, even in the case of exceptions. A typical workflow: `[](https://microsoft.github.io/agent-lightning/stable/reference/runner/#__codelineno-0-1) tracer = YourTracerImplementation() [](https://microsoft.github.io/agent-lightning/stable/reference/runner/#__codelineno-0-2) [](https://microsoft.github.io/agent-lightning/stable/reference/runner/#__codelineno-0-3) try: [](https://microsoft.github.io/agent-lightning/stable/reference/runner/#__codelineno-0-4) async with tracer.trace_context(name="my_traced_task"): [](https://microsoft.github.io/agent-lightning/stable/reference/runner/#__codelineno-0-5) # ... code to be traced ... [](https://microsoft.github.io/agent-lightning/stable/reference/runner/#__codelineno-0-6) await run_my_agent_logic() [](https://microsoft.github.io/agent-lightning/stable/reference/runner/#__codelineno-0-7) except Exception as e: [](https://microsoft.github.io/agent-lightning/stable/reference/runner/#__codelineno-0-8) print(f"An error occurred: {e}") [](https://microsoft.github.io/agent-lightning/stable/reference/runner/#__codelineno-0-9) [](https://microsoft.github.io/agent-lightning/stable/reference/runner/#__codelineno-0-10) # Retrieve the trace data after the context block [](https://microsoft.github.io/agent-lightning/stable/reference/runner/#__codelineno-0-11) spans: list[ReadableSpan] = tracer.get_last_trace() [](https://microsoft.github.io/agent-lightning/stable/reference/runner/#__codelineno-0-12) [](https://microsoft.github.io/agent-lightning/stable/reference/runner/#__codelineno-0-13) # Process the trace data [](https://microsoft.github.io/agent-lightning/stable/reference/runner/#__codelineno-0-14) if trace_tree: [](https://microsoft.github.io/agent-lightning/stable/reference/runner/#__codelineno-0-15) rl_triplets = TracerTraceToTriplet().adapt(spans) [](https://microsoft.github.io/agent-lightning/stable/reference/runner/#__codelineno-0-16) # ... do something with the triplets` #### `create_span(name, attributes=None, timestamp=None, status=None)` [¶](https://microsoft.github.io/agent-lightning/stable/reference/runner/#agentlightning.Tracer.create_span "Permanent link") Notify the tracer that a span should be created here. It uses a fire-and-forget approach and doesn't wait for the span to be created. Parameters: * **`name`** (`str`) – The name of the span. * **`attributes`** (`Optional[[Attributes](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.Attributes " agentlightning.Attributes = Dict[str, AttributeValue] module-attribute (agentlightning.types.Attributes)") ]`, default: `None` ) – The attributes of the span. * **`timestamp`** (`Optional[float]`, default: `None` ) – The timestamp of the span. * **`status`** (`Optional[[TraceStatus](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.TraceStatus " agentlightning.TraceStatus (agentlightning.types.TraceStatus)") ]`, default: `None` ) – The status of the span. Returns: * `[SpanCoreFields](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.SpanCoreFields " agentlightning.SpanCoreFields (agentlightning.types.SpanCoreFields)") ` – The core fields of the span. #### `get_langchain_handler()` [¶](https://microsoft.github.io/agent-lightning/stable/reference/runner/#agentlightning.Tracer.get_langchain_handler "Permanent link") Get a handler to install in langchain agent callback. Agents are expected to use this handler in their agents to enable tracing. #### `get_last_trace()` [¶](https://microsoft.github.io/agent-lightning/stable/reference/runner/#agentlightning.Tracer.get_last_trace "Permanent link") Retrieves the raw list of captured spans from the most recent trace. Returns: * `List[[Span](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.Span " agentlightning.Span (agentlightning.types.Span)") ]` – A list of [`Span`](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.Span " agentlightning.Span") objects collected during the last trace. #### `init_worker(worker_id, store=None)` [¶](https://microsoft.github.io/agent-lightning/stable/reference/runner/#agentlightning.Tracer.init_worker "Permanent link") Initialize the tracer for a worker. Parameters: * **`worker_id`** (`int`) – The ID of the worker. * **`store`** (`Optional[[LightningStore](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.LightningStore " agentlightning.LightningStore (agentlightning.store.base.LightningStore)") ]`, default: `None` ) – The store to add the spans to. If it's provided, traces will be added to the store when tracing. #### `lifespan(store=None)` [¶](https://microsoft.github.io/agent-lightning/stable/reference/runner/#agentlightning.Tracer.lifespan "Permanent link") A context manager to manage the lifespan of the tracer. This can be used to set up and tear down any necessary resources for the tracer, useful for debugging purposes. Parameters: * **`store`** (`Optional[[LightningStore](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.LightningStore " agentlightning.LightningStore (agentlightning.store.base.LightningStore)") ]`, default: `None` ) – The store to add the spans to. If it's provided, traces will be added to the store when tracing. #### `operation_context(name, attributes=None, start_time=None, end_time=None)` [¶](https://microsoft.github.io/agent-lightning/stable/reference/runner/#agentlightning.Tracer.operation_context "Permanent link") Start to record an operation to a span. Parameters: * **`name`** (`str`) – The name of the operation. * **`attributes`** (`Optional[[Attributes](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.Attributes " agentlightning.Attributes = Dict[str, AttributeValue] module-attribute (agentlightning.types.Attributes)") ]`, default: `None` ) – The attributes of the operation. * **`start_time`** (`Optional[float]`, default: `None` ) – The start time of the operation. * **`end_time`** (`Optional[float]`, default: `None` ) – The end time of the operation. Returns: * `ContextManager[[SpanRecordingContext](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.SpanRecordingContext " agentlightning.SpanRecordingContext (agentlightning.types.SpanRecordingContext)") ]` – A [`SpanRecordingContext`](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.SpanRecordingContext " agentlightning.SpanRecordingContext") for recording the operation on the span. #### `trace_context(name=None, *, store=None, rollout_id=None, attempt_id=None)` [¶](https://microsoft.github.io/agent-lightning/stable/reference/runner/#agentlightning.Tracer.trace_context "Permanent link") Starts a new tracing context. This should be used as a context manager. The implementation should handle the setup and teardown of the tracing for the enclosed code block. It must ensure that any spans generated within the `with` block are collected and made available via [`get_last_trace`](https://microsoft.github.io/agent-lightning/stable/reference/runner/#agentlightning.Tracer.get_last_trace " get_last_trace()") . Parameters: * **`name`** (`Optional[str]`, default: `None` ) – The name for the root span of this trace context. * **`store`** (`Optional[[LightningStore](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.LightningStore " agentlightning.LightningStore (agentlightning.store.base.LightningStore)") ]`, default: `None` ) – The store to add the spans to. Deprecated in favor of passing store to init\_worker(). * **`rollout_id`** (`Optional[str]`, default: `None` ) – The rollout ID to add the spans to. * **`attempt_id`** (`Optional[str]`, default: `None` ) – The attempt ID to add the spans to. #### `trace_run(func, *args, **kwargs)` [¶](https://microsoft.github.io/agent-lightning/stable/reference/runner/#agentlightning.Tracer.trace_run "Permanent link") A convenience wrapper to trace the execution of a single synchronous function. Deprecated in favor of customizing Runners. Parameters: * **`func`** (`Callable[..., Any]`) – The synchronous function to execute and trace. * **`*args`** (`Any`, default: `()` ) – Positional arguments to pass to the function. * **`**kwargs`** (`Any`, default: `{}` ) – Keyword arguments to pass to the function. Returns: * `Any` – The return value of the function. #### `trace_run_async(func, *args, **kwargs)` `async` [¶](https://microsoft.github.io/agent-lightning/stable/reference/runner/#agentlightning.Tracer.trace_run_async "Permanent link") A convenience wrapper to trace the execution of a single asynchronous function. Deprecated in favor of customizing Runners. Parameters: * **`func`** (`Callable[..., Awaitable[Any]]`) – The asynchronous function to execute and trace. * **`*args`** (`Any`, default: `()` ) – Positional arguments to pass to the function. * **`**kwargs`** (`Any`, default: `{}` ) – Keyword arguments to pass to the function. Returns: * `Any` – The return value of the function. ### `agentlightning.tracer.weave.WeaveTracer` [¶](https://microsoft.github.io/agent-lightning/stable/reference/runner/#agentlightning.tracer.weave.WeaveTracer "Permanent link") Bases: `[Tracer](https://microsoft.github.io/agent-lightning/stable/reference/runner/#agentlightning.Tracer " agentlightning.Tracer (agentlightning.tracer.base.Tracer)") ` Tracer implementation using Weave for telemetry and trace logging. This replaces AgentOpsTracer with a Weave-based manual trace context. It tracks: * Function/method calls * Input/Output data * Exceptions and logs them to Weave Cloud (W&B backend) or optionally bypasses the network for testing. #### `__init__(*, project_name=None, weave_user_settings=None, instrument_managed=True)` [¶](https://microsoft.github.io/agent-lightning/stable/reference/runner/#agentlightning.tracer.weave.WeaveTracer.__init__ "Permanent link") Initialize a WeaveTracer instance. Parameters: * **`project_name`** (`str | None`, default: `None` ) – Optional project name for Weave; defaults to the current module name. * **`weave_user_settings`** (`UserSettings | None`, default: `None` ) – Optional UserSettings for Weave. * **`instrument_managed`** (`bool`, default: `True` ) – Whether to patch the Weave/W&B integration to bypass actual network calls for testing. #### `complete_call_handler(call)` `async` [¶](https://microsoft.github.io/agent-lightning/stable/reference/runner/#agentlightning.tracer.weave.WeaveTracer.complete_call_handler "Permanent link") Handler called when a Weave Call finishes. Converts the call (including nested children) into spans and stores them in LightningStore. #### `convert_call_to_span(call, rollout_id=None, attempt_id=None, sequence_id=None)` `async` [¶](https://microsoft.github.io/agent-lightning/stable/reference/runner/#agentlightning.tracer.weave.WeaveTracer.convert_call_to_span "Permanent link") Convert a Weave Call (with nested children) into a Agent-lightning Span. `rollout_id` and `attempt_id` are required to attach the spans to the store. Parameters: * **`call`** (`CallSchema`) – The Weave Call object. * **`rollout_id`** (`Optional[str]`, default: `None` ) – Optional rollout ID to attach to spans. * **`attempt_id`** (`Optional[str]`, default: `None` ) – Optional attempt ID to attach to spans. * **`sequence_id`** (`Optional[int]`, default: `None` ) – Optional sequence ID to attach to spans. Returns: * `[Span](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.Span " agentlightning.Span (agentlightning.types.Span)") ` – List of converted spans. #### `init_worker(worker_id, store=None)` [¶](https://microsoft.github.io/agent-lightning/stable/reference/runner/#agentlightning.tracer.weave.WeaveTracer.init_worker "Permanent link") Initialize the tracer for a worker thread/process. Parameters: * **`worker_id`** (`int`) – Identifier of the worker. * **`store`** (`Optional[[LightningStore](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.LightningStore " agentlightning.LightningStore (agentlightning.store.base.LightningStore)") ]`, default: `None` ) – Optional LightningStore for storing spans. #### `partial_call_handler(request_content)` `async` [¶](https://microsoft.github.io/agent-lightning/stable/reference/runner/#agentlightning.tracer.weave.WeaveTracer.partial_call_handler "Permanent link") Handler called when a Weave Call starts. Parameters: * **`request_content`** (`Dict[str, Any]`) – The partial Weave Call object. Returns: * `int` – The sequence ID for the call. #### `teardown_worker(worker_id)` [¶](https://microsoft.github.io/agent-lightning/stable/reference/runner/#agentlightning.tracer.weave.WeaveTracer.teardown_worker "Permanent link") Clean up tracer resources for the worker. Parameters: * **`worker_id`** (`int`) – Identifier of the worker. #### `trace_context(name=None, *, rollout_id=None, attempt_id=None, **kwargs)` `async` [¶](https://microsoft.github.io/agent-lightning/stable/reference/runner/#agentlightning.tracer.weave.WeaveTracer.trace_context "Permanent link") Asynchronous implementation of the tracing context. Parameters: * **`name`** (`Optional[str]`, default: `None` ) – Optional operation name. * **`rollout_id`** (`Optional[str]`, default: `None` ) – Optional rollout ID. * **`attempt_id`** (`Optional[str]`, default: `None` ) – Optional attempt ID. Raises: * `ValueError` – If store, rollout\_id, and attempt\_id are inconsistently provided. * `RuntimeError` – If Weave is not installed or client is uninitialized. ### `agentlightning.DummyTracer` [¶](https://microsoft.github.io/agent-lightning/stable/reference/runner/#agentlightning.DummyTracer "Permanent link") Bases: `[Tracer](https://microsoft.github.io/agent-lightning/stable/reference/runner/#agentlightning.Tracer " agentlightning.Tracer (agentlightning.tracer.base.Tracer)") ` A dummy tracer that does not trace anything, but it is compatible with the emitter API. It doesn't rely on any backend tracer, and also doesn't use any stores. ### `agentlightning.set_active_tracer(tracer)` [¶](https://microsoft.github.io/agent-lightning/stable/reference/runner/#agentlightning.set_active_tracer "Permanent link") Set the active tracer for the current process. Parameters: * **`tracer`** (`[Tracer](https://microsoft.github.io/agent-lightning/stable/reference/runner/#agentlightning.Tracer " agentlightning.Tracer (agentlightning.tracer.base.Tracer)") `) – The tracer to set as active. ### `agentlightning.get_active_tracer()` [¶](https://microsoft.github.io/agent-lightning/stable/reference/runner/#agentlightning.get_active_tracer "Permanent link") Get the active tracer for the current process. Returns: * `Optional[[Tracer](https://microsoft.github.io/agent-lightning/stable/reference/runner/#agentlightning.Tracer " agentlightning.Tracer (agentlightning.tracer.base.Tracer)") ]` – The active tracer, or None if no tracer is active. ### `agentlightning.clear_active_tracer()` [¶](https://microsoft.github.io/agent-lightning/stable/reference/runner/#agentlightning.clear_active_tracer "Permanent link") Clear the active tracer for the current process. ### `agentlightning.tracer.weave.WeaveTracer` [¶](https://microsoft.github.io/agent-lightning/stable/reference/runner/#agentlightning.tracer.weave.WeaveTracer "Permanent link") Bases: `[Tracer](https://microsoft.github.io/agent-lightning/stable/reference/runner/#agentlightning.Tracer " agentlightning.Tracer (agentlightning.tracer.base.Tracer)") ` Tracer implementation using Weave for telemetry and trace logging. This replaces AgentOpsTracer with a Weave-based manual trace context. It tracks: * Function/method calls * Input/Output data * Exceptions and logs them to Weave Cloud (W&B backend) or optionally bypasses the network for testing. #### `__init__(*, project_name=None, weave_user_settings=None, instrument_managed=True)` [¶](https://microsoft.github.io/agent-lightning/stable/reference/runner/#agentlightning.tracer.weave.WeaveTracer.__init__ "Permanent link") Initialize a WeaveTracer instance. Parameters: * **`project_name`** (`str | None`, default: `None` ) – Optional project name for Weave; defaults to the current module name. * **`weave_user_settings`** (`UserSettings | None`, default: `None` ) – Optional UserSettings for Weave. * **`instrument_managed`** (`bool`, default: `True` ) – Whether to patch the Weave/W&B integration to bypass actual network calls for testing. #### `complete_call_handler(call)` `async` [¶](https://microsoft.github.io/agent-lightning/stable/reference/runner/#agentlightning.tracer.weave.WeaveTracer.complete_call_handler "Permanent link") Handler called when a Weave Call finishes. Converts the call (including nested children) into spans and stores them in LightningStore. #### `convert_call_to_span(call, rollout_id=None, attempt_id=None, sequence_id=None)` `async` [¶](https://microsoft.github.io/agent-lightning/stable/reference/runner/#agentlightning.tracer.weave.WeaveTracer.convert_call_to_span "Permanent link") Convert a Weave Call (with nested children) into a Agent-lightning Span. `rollout_id` and `attempt_id` are required to attach the spans to the store. Parameters: * **`call`** (`CallSchema`) – The Weave Call object. * **`rollout_id`** (`Optional[str]`, default: `None` ) – Optional rollout ID to attach to spans. * **`attempt_id`** (`Optional[str]`, default: `None` ) – Optional attempt ID to attach to spans. * **`sequence_id`** (`Optional[int]`, default: `None` ) – Optional sequence ID to attach to spans. Returns: * `[Span](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.Span " agentlightning.Span (agentlightning.types.Span)") ` – List of converted spans. #### `init_worker(worker_id, store=None)` [¶](https://microsoft.github.io/agent-lightning/stable/reference/runner/#agentlightning.tracer.weave.WeaveTracer.init_worker "Permanent link") Initialize the tracer for a worker thread/process. Parameters: * **`worker_id`** (`int`) – Identifier of the worker. * **`store`** (`Optional[[LightningStore](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.LightningStore " agentlightning.LightningStore (agentlightning.store.base.LightningStore)") ]`, default: `None` ) – Optional LightningStore for storing spans. #### `partial_call_handler(request_content)` `async` [¶](https://microsoft.github.io/agent-lightning/stable/reference/runner/#agentlightning.tracer.weave.WeaveTracer.partial_call_handler "Permanent link") Handler called when a Weave Call starts. Parameters: * **`request_content`** (`Dict[str, Any]`) – The partial Weave Call object. Returns: * `int` – The sequence ID for the call. #### `teardown_worker(worker_id)` [¶](https://microsoft.github.io/agent-lightning/stable/reference/runner/#agentlightning.tracer.weave.WeaveTracer.teardown_worker "Permanent link") Clean up tracer resources for the worker. Parameters: * **`worker_id`** (`int`) – Identifier of the worker. #### `trace_context(name=None, *, rollout_id=None, attempt_id=None, **kwargs)` `async` [¶](https://microsoft.github.io/agent-lightning/stable/reference/runner/#agentlightning.tracer.weave.WeaveTracer.trace_context "Permanent link") Asynchronous implementation of the tracing context. Parameters: * **`name`** (`Optional[str]`, default: `None` ) – Optional operation name. * **`rollout_id`** (`Optional[str]`, default: `None` ) – Optional rollout ID. * **`attempt_id`** (`Optional[str]`, default: `None` ) – Optional attempt ID. Raises: * `ValueError` – If store, rollout\_id, and attempt\_id are inconsistently provided. * `RuntimeError` – If Weave is not installed or client is uninitialized. Back to top --- # Trainer - Agent-lightning [Skip to content](https://microsoft.github.io/agent-lightning/stable/reference/trainer/#agent-lightning-trainer) ⚠️ You are viewing development documentation. For stable documentation, please visit [https://microsoft.github.io/agent-lightning/stable/](https://microsoft.github.io/agent-lightning/stable/) Agent-lightning Trainer[¶](https://microsoft.github.io/agent-lightning/stable/reference/trainer/#agent-lightning-trainer "Permanent link") =========================================================================================================================================== ### `agentlightning.Trainer` [¶](https://microsoft.github.io/agent-lightning/stable/reference/trainer/#agentlightning.Trainer "Permanent link") Bases: `TrainerLegacy` High-level orchestration layer that wires Algorithm <-> Runner <-> Store. A [`Trainer`](https://microsoft.github.io/agent-lightning/stable/reference/trainer/#agentlightning.Trainer " agentlightning.Trainer") packages the moving parts of Agent-Lightning's training loop into a single entry point: * **Algorithm lifecycle:** Instantiates or accepts an [`Algorithm`](https://microsoft.github.io/agent-lightning/stable/reference/algorithm/#agentlightning.Algorithm " agentlightning.Algorithm") , attaches the current [`LightningStore`](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.LightningStore " agentlightning.LightningStore") , adapter, and initial resources, then executes the algorithm role inside the configured execution strategy. * **Runner fleet:** Spawns one or more [`Runner`](https://microsoft.github.io/agent-lightning/stable/reference/runner/#agentlightning.Runner " agentlightning.Runner") instances (defaulting to [`LitAgentRunner`](https://microsoft.github.io/agent-lightning/stable/reference/runner/#agentlightning.LitAgentRunner " agentlightning.LitAgentRunner") ) that hydrate a [`LitAgent`](https://microsoft.github.io/agent-lightning/stable/reference/agent/#agentlightning.LitAgent " agentlightning.LitAgent") , claim rollouts, stream spans, and respect graceful termination signals from the execution strategy. * **Execution strategy:** Delegates process management to an [`ExecutionStrategy`](https://microsoft.github.io/agent-lightning/stable/reference/trainer/#agentlightning.ExecutionStrategy " agentlightning.ExecutionStrategy") (shared memory, client/server, etc.), so advanced users can swap orchestration backends without changing trainer code. * **Telemetry plumbing:** Ensures tracers, adapters, and optional [`LLMProxy`](https://microsoft.github.io/agent-lightning/stable/reference/algorithm/#agentlightning.LLMProxy " agentlightning.LLMProxy") are wired into both algorithm and runners so telemetry flows back into the store. The trainer exposes two convenience entry points: [`fit()`](https://microsoft.github.io/agent-lightning/stable/reference/trainer/#agentlightning.Trainer.fit " fit(agent, train_dataset=None, *, val_dataset=None)") for full training and [`dev()`](https://microsoft.github.io/agent-lightning/stable/reference/trainer/#agentlightning.Trainer.dev " dev(agent, train_dataset=None, *, val_dataset=None)") for fast, reproducible dry-runs. See the [Train the First Agent](https://microsoft.github.io/agent-lightning/stable/how-to/train-first-agent/) and [Write the First Algorithm](https://microsoft.github.io/agent-lightning/stable/how-to/write-first-algorithm/) tutorials for the broader context. #### `adapter = self._make_adapter(adapter_spec)` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/stable/reference/trainer/#agentlightning.Trainer.adapter "Permanent link") An instance of [`TraceAdapter`](https://microsoft.github.io/agent-lightning/stable/reference/algorithm/#agentlightning.TraceAdapter " agentlightning.TraceAdapter") to export data consumble by algorithms from traces. #### `algorithm = self._make_algorithm(algorithm)` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/stable/reference/trainer/#agentlightning.Trainer.algorithm "Permanent link") An instance of [`Algorithm`](https://microsoft.github.io/agent-lightning/stable/reference/algorithm/#agentlightning.Algorithm " agentlightning.Algorithm") to use for training. #### `daemon = daemon` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/stable/reference/trainer/#agentlightning.Trainer.daemon "Permanent link") Whether worker processes should be daemons. Daemon processes are terminated automatically when the main process exits. Deprecated. Only have effect with `fit_v0`. #### `hooks = self._normalize_hooks(hooks)` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/stable/reference/trainer/#agentlightning.Trainer.hooks "Permanent link") A sequence of [`Hook`](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.Hook " agentlightning.Hook") instances to be called at various lifecycle stages (e.g., `on_trace_start`, `on_trace_end`, `on_rollout_start`, `on_rollout_end`). #### `initial_resources = initial_resources` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/stable/reference/trainer/#agentlightning.Trainer.initial_resources "Permanent link") An instance of [`NamedResources`](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.NamedResources " agentlightning.NamedResources = Dict[str, ResourceUnion] module-attribute ") to use for bootstrapping the fit/dev process. The resources will be handed over to the algorithm. Note that not all algorithms support seeding resources. #### `llm_proxy = self._make_llm_proxy(llm_proxy, store=(self.store))` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/stable/reference/trainer/#agentlightning.Trainer.llm_proxy "Permanent link") An instance of [`LLMProxy`](https://microsoft.github.io/agent-lightning/stable/reference/algorithm/#agentlightning.LLMProxy " agentlightning.LLMProxy") to use for intercepting the LLM calls. If not provided, algorithm may create one on its own. #### `max_rollouts = max_rollouts` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/stable/reference/trainer/#agentlightning.Trainer.max_rollouts "Permanent link") Maximum number of rollouts to process per runner. If None, workers run until no more rollouts are available. #### `max_tasks = max_tasks if max_tasks is not None else max_rollouts` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/stable/reference/trainer/#agentlightning.Trainer.max_tasks "Permanent link") Maximum number of tasks to process per runner. Deprecated in favor of `max_rollouts`. #### `n_runners = n_runners` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/stable/reference/trainer/#agentlightning.Trainer.n_runners "Permanent link") Number of agent runners to run in parallel. #### `n_workers = n_runners` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/stable/reference/trainer/#agentlightning.Trainer.n_workers "Permanent link") Number of agent workers to run in parallel. Deprecated in favor of `n_runners`. #### `port = port` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/stable/reference/trainer/#agentlightning.Trainer.port "Permanent link") Port forwarded to [`ClientServerExecutionStrategy`](https://microsoft.github.io/agent-lightning/stable/reference/trainer/#agentlightning.ClientServerExecutionStrategy " agentlightning.ClientServerExecutionStrategy") . #### `runner = self._make_runner(runner)` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/stable/reference/trainer/#agentlightning.Trainer.runner "Permanent link") An instance of [`Runner`](https://microsoft.github.io/agent-lightning/stable/reference/runner/#agentlightning.Runner " agentlightning.Runner") to use for running the agent. #### `store = self._make_store(store, self.strategy)` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/stable/reference/trainer/#agentlightning.Trainer.store "Permanent link") An instance of [`LightningStore`](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.LightningStore " agentlightning.LightningStore") to use for storing tasks and traces. #### `strategy = self._make_strategy(strategy, n_runners=(self.n_runners), port=port)` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/stable/reference/trainer/#agentlightning.Trainer.strategy "Permanent link") An instance of [`ExecutionStrategy`](https://microsoft.github.io/agent-lightning/stable/reference/trainer/#agentlightning.ExecutionStrategy " agentlightning.ExecutionStrategy") to use for spawning the algorithm and runners. #### `tracer = self._make_tracer(tracer)` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/stable/reference/trainer/#agentlightning.Trainer.tracer "Permanent link") A tracer instance, or a string pointing to the class full name or a dictionary with a 'type' key that specifies the class full name and other initialization parameters. If None, a default [`AgentOpsTracer`](https://microsoft.github.io/agent-lightning/stable/reference/runner/#agentlightning.AgentOpsTracer " agentlightning.AgentOpsTracer") will be created with the current settings. #### `triplet_exporter = self.adapter` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/stable/reference/trainer/#agentlightning.Trainer.triplet_exporter "Permanent link") An instance of [`TracerTraceToTriplet`](https://microsoft.github.io/agent-lightning/stable/reference/algorithm/#agentlightning.TracerTraceToTriplet " agentlightning.TracerTraceToTriplet") to export triplets from traces, or a dictionary with the initialization parameters for the exporter. Deprecated. Use [`adapter`](https://microsoft.github.io/agent-lightning/stable/reference/trainer/#agentlightning.Trainer.adapter " adapter = self._make_adapter(adapter_spec) instance-attribute ") instead. #### `__init__(*, dev=False, n_runners=None, max_rollouts=None, initial_resources=None, tracer=None, adapter=None, store=None, runner=None, strategy=None, port=None, algorithm=None, llm_proxy=None, n_workers=None, max_tasks=None, daemon=True, triplet_exporter=None, hooks=None)` [¶](https://microsoft.github.io/agent-lightning/stable/reference/trainer/#agentlightning.Trainer.__init__ "Permanent link") Configure the trainer and resolve user-provided component specifications. Each keyword accepts either a concrete instance, a class, a callable factory, a registry string, or a lightweight configuration dictionary (see [`build_component()`](https://microsoft.github.io/agent-lightning/stable/reference/trainer/#agentlightning.build_component " agentlightning.build_component(spec, *, expected_type, spec_name, default_factory=None, allow_none=False, optional_defaults=None, dict_requires_type=True, dict_default_cls=None, type_error_fmt=None, invalid_spec_error_fmt=None, registry=None)") ). When `port` is provided it is forwarded to [`ClientServerExecutionStrategy`](https://microsoft.github.io/agent-lightning/stable/reference/trainer/#agentlightning.ClientServerExecutionStrategy " agentlightning.ClientServerExecutionStrategy") instances constructed (or supplied) for the trainer. #### `dev(agent, train_dataset=None, *, val_dataset=None)` [¶](https://microsoft.github.io/agent-lightning/stable/reference/trainer/#agentlightning.Trainer.dev "Permanent link") Exercise the infrastructure using a fast, synchronous algorithm. [`Trainer.dev`](https://microsoft.github.io/agent-lightning/stable/reference/trainer/#agentlightning.Trainer.dev " dev(agent, train_dataset=None, *, val_dataset=None)") mirrors [`fit()`](https://microsoft.github.io/agent-lightning/stable/reference/trainer/#agentlightning.Trainer.fit " fit(agent, train_dataset=None, *, val_dataset=None)") but insists on an [`Algorithm`](https://microsoft.github.io/agent-lightning/stable/reference/algorithm/#agentlightning.Algorithm " agentlightning.Algorithm") subtype that also derives from [`FastAlgorithm`](https://microsoft.github.io/agent-lightning/stable/reference/algorithm/#agentlightning.FastAlgorithm " agentlightning.FastAlgorithm") . This keeps the loop responsive for debugging while still touching the same store, runners, hooks, and tracer plumbing. If no algorithm is provided, a default [`Baseline`](https://microsoft.github.io/agent-lightning/stable/reference/algorithm/#agentlightning.Baseline " agentlightning.Baseline") algorithm will be used. Parameters: * **`agent`** (`[LitAgent](https://microsoft.github.io/agent-lightning/stable/reference/agent/#agentlightning.LitAgent " agentlightning.LitAgent (agentlightning.litagent.LitAgent)") [T_co]`) – [`LitAgent`](https://microsoft.github.io/agent-lightning/stable/reference/agent/#agentlightning.LitAgent " agentlightning.LitAgent") implementation to execute. * **`train_dataset`** (`Optional[[Dataset](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.Dataset " agentlightning.Dataset (agentlightning.types.Dataset)") [T_co]]`, default: `None` ) – Optional iterable passed to the algorithm. * **`val_dataset`** (`Optional[[Dataset](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.Dataset " agentlightning.Dataset (agentlightning.types.Dataset)") [T_co]]`, default: `None` ) – Optional iterable passed to the algorithm. Raises: * `TypeError` – If the configured algorithm does not inherit from `FastAlgorithm`. #### `fit(agent, train_dataset=None, *, val_dataset=None)` [¶](https://microsoft.github.io/agent-lightning/stable/reference/trainer/#agentlightning.Trainer.fit "Permanent link") Execute the full algorithm/runner training loop. [`Trainer.fit`](https://microsoft.github.io/agent-lightning/stable/reference/trainer/#agentlightning.Trainer.fit " fit(agent, train_dataset=None, *, val_dataset=None)") packages the algorithm and runner bundles, then hands them to the active [`ExecutionStrategy`](https://microsoft.github.io/agent-lightning/stable/reference/trainer/#agentlightning.ExecutionStrategy " agentlightning.ExecutionStrategy") . The strategy rarely returns until: * The algorithm exhausts the dataset(s) and stops enqueuing rollouts. * `max_rollouts` causes individual runners to exit. * An exception or interrupt cancels the shared [`ExecutionEvent`](https://microsoft.github.io/agent-lightning/stable/reference/trainer/#agentlightning.ExecutionEvent " agentlightning.ExecutionEvent") . Parameters: * **`agent`** (`[LitAgent](https://microsoft.github.io/agent-lightning/stable/reference/agent/#agentlightning.LitAgent " agentlightning.LitAgent (agentlightning.litagent.LitAgent)") [T_co]`) – [`LitAgent`](https://microsoft.github.io/agent-lightning/stable/reference/agent/#agentlightning.LitAgent " agentlightning.LitAgent") implementation executed by runners. * **`train_dataset`** (`Optional[[Dataset](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.Dataset " agentlightning.Dataset (agentlightning.types.Dataset)") [T_co]]`, default: `None` ) – Optional iterable of rollout inputs consumed by the algorithm. * **`val_dataset`** (`Optional[[Dataset](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.Dataset " agentlightning.Dataset (agentlightning.types.Dataset)") [T_co]]`, default: `None` ) – Optional iterable consumed by validation passes. ### `agentlightning.build_component(spec, *, expected_type, spec_name, default_factory=None, allow_none=False, optional_defaults=None, dict_requires_type=True, dict_default_cls=None, type_error_fmt=None, invalid_spec_error_fmt=None, registry=None)` [¶](https://microsoft.github.io/agent-lightning/stable/reference/trainer/#agentlightning.build_component "Permanent link") `[](https://microsoft.github.io/agent-lightning/stable/reference/trainer/#__codelineno-0-1) build_component( [](https://microsoft.github.io/agent-lightning/stable/reference/trainer/#__codelineno-0-2) spec: Union[ [](https://microsoft.github.io/agent-lightning/stable/reference/trainer/#__codelineno-0-3) T, [](https://microsoft.github.io/agent-lightning/stable/reference/trainer/#__codelineno-0-4) str, [](https://microsoft.github.io/agent-lightning/stable/reference/trainer/#__codelineno-0-5) Dict[str, Any], [](https://microsoft.github.io/agent-lightning/stable/reference/trainer/#__codelineno-0-6) type[T], [](https://microsoft.github.io/agent-lightning/stable/reference/trainer/#__codelineno-0-7) Callable[[], T], [](https://microsoft.github.io/agent-lightning/stable/reference/trainer/#__codelineno-0-8) None, [](https://microsoft.github.io/agent-lightning/stable/reference/trainer/#__codelineno-0-9) ], [](https://microsoft.github.io/agent-lightning/stable/reference/trainer/#__codelineno-0-10) *, [](https://microsoft.github.io/agent-lightning/stable/reference/trainer/#__codelineno-0-11) expected_type: type[T], [](https://microsoft.github.io/agent-lightning/stable/reference/trainer/#__codelineno-0-12) spec_name: str, [](https://microsoft.github.io/agent-lightning/stable/reference/trainer/#__codelineno-0-13) default_factory: Callable[[], T], [](https://microsoft.github.io/agent-lightning/stable/reference/trainer/#__codelineno-0-14) allow_none: bool = ..., [](https://microsoft.github.io/agent-lightning/stable/reference/trainer/#__codelineno-0-15) optional_defaults: Optional[OptionalDefaults] = ..., [](https://microsoft.github.io/agent-lightning/stable/reference/trainer/#__codelineno-0-16) dict_requires_type: bool = ..., [](https://microsoft.github.io/agent-lightning/stable/reference/trainer/#__codelineno-0-17) dict_default_cls: type[T] | None = ..., [](https://microsoft.github.io/agent-lightning/stable/reference/trainer/#__codelineno-0-18) type_error_fmt: str | None = ..., [](https://microsoft.github.io/agent-lightning/stable/reference/trainer/#__codelineno-0-19) invalid_spec_error_fmt: str | None = ..., [](https://microsoft.github.io/agent-lightning/stable/reference/trainer/#__codelineno-0-20) registry: Optional[Dict[str, str]] = ... [](https://microsoft.github.io/agent-lightning/stable/reference/trainer/#__codelineno-0-21) ) -> T` `[](https://microsoft.github.io/agent-lightning/stable/reference/trainer/#__codelineno-0-1) build_component( [](https://microsoft.github.io/agent-lightning/stable/reference/trainer/#__codelineno-0-2) spec: Union[ [](https://microsoft.github.io/agent-lightning/stable/reference/trainer/#__codelineno-0-3) T, [](https://microsoft.github.io/agent-lightning/stable/reference/trainer/#__codelineno-0-4) str, [](https://microsoft.github.io/agent-lightning/stable/reference/trainer/#__codelineno-0-5) Dict[str, Any], [](https://microsoft.github.io/agent-lightning/stable/reference/trainer/#__codelineno-0-6) type[T], [](https://microsoft.github.io/agent-lightning/stable/reference/trainer/#__codelineno-0-7) Callable[[], T], [](https://microsoft.github.io/agent-lightning/stable/reference/trainer/#__codelineno-0-8) None, [](https://microsoft.github.io/agent-lightning/stable/reference/trainer/#__codelineno-0-9) ], [](https://microsoft.github.io/agent-lightning/stable/reference/trainer/#__codelineno-0-10) *, [](https://microsoft.github.io/agent-lightning/stable/reference/trainer/#__codelineno-0-11) expected_type: type[T], [](https://microsoft.github.io/agent-lightning/stable/reference/trainer/#__codelineno-0-12) spec_name: str, [](https://microsoft.github.io/agent-lightning/stable/reference/trainer/#__codelineno-0-13) default_factory: None = ..., [](https://microsoft.github.io/agent-lightning/stable/reference/trainer/#__codelineno-0-14) allow_none: bool, [](https://microsoft.github.io/agent-lightning/stable/reference/trainer/#__codelineno-0-15) optional_defaults: Optional[OptionalDefaults] = ..., [](https://microsoft.github.io/agent-lightning/stable/reference/trainer/#__codelineno-0-16) dict_requires_type: bool = ..., [](https://microsoft.github.io/agent-lightning/stable/reference/trainer/#__codelineno-0-17) dict_default_cls: type[T] | None = ..., [](https://microsoft.github.io/agent-lightning/stable/reference/trainer/#__codelineno-0-18) type_error_fmt: str | None = ..., [](https://microsoft.github.io/agent-lightning/stable/reference/trainer/#__codelineno-0-19) invalid_spec_error_fmt: str | None = ..., [](https://microsoft.github.io/agent-lightning/stable/reference/trainer/#__codelineno-0-20) registry: Optional[Dict[str, str]] = ... [](https://microsoft.github.io/agent-lightning/stable/reference/trainer/#__codelineno-0-21) ) -> T | None` `[](https://microsoft.github.io/agent-lightning/stable/reference/trainer/#__codelineno-0-1) build_component( [](https://microsoft.github.io/agent-lightning/stable/reference/trainer/#__codelineno-0-2) spec: Union[ [](https://microsoft.github.io/agent-lightning/stable/reference/trainer/#__codelineno-0-3) T, [](https://microsoft.github.io/agent-lightning/stable/reference/trainer/#__codelineno-0-4) str, [](https://microsoft.github.io/agent-lightning/stable/reference/trainer/#__codelineno-0-5) Dict[str, Any], [](https://microsoft.github.io/agent-lightning/stable/reference/trainer/#__codelineno-0-6) type[T], [](https://microsoft.github.io/agent-lightning/stable/reference/trainer/#__codelineno-0-7) Callable[[], T], [](https://microsoft.github.io/agent-lightning/stable/reference/trainer/#__codelineno-0-8) None, [](https://microsoft.github.io/agent-lightning/stable/reference/trainer/#__codelineno-0-9) ], [](https://microsoft.github.io/agent-lightning/stable/reference/trainer/#__codelineno-0-10) *, [](https://microsoft.github.io/agent-lightning/stable/reference/trainer/#__codelineno-0-11) expected_type: type[T], [](https://microsoft.github.io/agent-lightning/stable/reference/trainer/#__codelineno-0-12) spec_name: str, [](https://microsoft.github.io/agent-lightning/stable/reference/trainer/#__codelineno-0-13) default_factory: None = ..., [](https://microsoft.github.io/agent-lightning/stable/reference/trainer/#__codelineno-0-14) allow_none: bool = ..., [](https://microsoft.github.io/agent-lightning/stable/reference/trainer/#__codelineno-0-15) optional_defaults: Optional[OptionalDefaults] = ..., [](https://microsoft.github.io/agent-lightning/stable/reference/trainer/#__codelineno-0-16) dict_requires_type: bool = ..., [](https://microsoft.github.io/agent-lightning/stable/reference/trainer/#__codelineno-0-17) dict_default_cls: type[T] | None = ..., [](https://microsoft.github.io/agent-lightning/stable/reference/trainer/#__codelineno-0-18) type_error_fmt: str | None = ..., [](https://microsoft.github.io/agent-lightning/stable/reference/trainer/#__codelineno-0-19) invalid_spec_error_fmt: str | None = ..., [](https://microsoft.github.io/agent-lightning/stable/reference/trainer/#__codelineno-0-20) registry: Optional[Dict[str, str]] = ... [](https://microsoft.github.io/agent-lightning/stable/reference/trainer/#__codelineno-0-21) ) -> T | None` Build and return a component instance from a flexible specification. This function provides a flexible way to create component instances from various input formats including direct instances, class types, factory functions, import paths, or configuration dictionaries. Parameters: * **`spec`** (`Union[T, str, Dict[str, Any], type[T], Callable[[], T], None]`) – The component specification. Can be: - An instance of expected\_type (returned as-is) - A string import path (e.g., 'module.Class') or registry key - A dict with 'type' key (import path or registry key) and constructor kwargs - A class type (will be instantiated) - A factory function (will be called) - None (uses default\_factory or returns None if allow\_none=True) * **`expected_type`** (`type[T]`) – The type that the resulting instance must be or inherit from. * **`spec_name`** (`str`) – Descriptive name for the spec, used in error messages. * **`default_factory`** (`Callable[[], T] | None`, default: `None` ) – Optional factory function called when spec is None. * **`allow_none`** (`bool`, default: `False` ) – If True, allows None to be returned when spec is None and no default\_factory is provided. * **`optional_defaults`** (`Optional[OptionalDefaults]`, default: `None` ) – Dict mapping parameter names to default values or factory functions that will be injected if the constructor accepts them. * **`dict_requires_type`** (`bool`, default: `True` ) – If True, dict specs must include a 'type' key. * **`dict_default_cls`** (`type[T] | None`, default: `None` ) – Default class to use for dict specs without a 'type' key (only used when dict\_requires\_type=False). * **`type_error_fmt`** (`str | None`, default: `None` ) – Custom format string for type validation errors. Should include {type\_name} and {expected\_type} placeholders. * **`invalid_spec_error_fmt`** (`str | None`, default: `None` ) – Custom format string for invalid spec type errors. Should include {actual\_type} and {expected\_type} placeholders. * **`registry`** (`Optional[Dict[str, str]]`, default: `None` ) – Optional mapping of short names to fully qualified import paths. When provided, string specs or dict 'type'/'name' entries are first resolved through this registry before attempting to import. Returns: * `T | None` – An instance of expected\_type, or None if allow\_none=True and spec is None * `T | None` – without a default\_factory. Raises: * `TypeError` – If the instantiated object is not an instance of expected\_type. * `ValueError` – If spec is None and neither default\_factory nor allow\_none is set, or if spec type is invalid, or if dict spec is invalid. Examples: `[](https://microsoft.github.io/agent-lightning/stable/reference/trainer/#__codelineno-0-1) >>> # Direct instance [](https://microsoft.github.io/agent-lightning/stable/reference/trainer/#__codelineno-0-2) >>> optimizer = build_component(AdamW(), expected_type=Optimizer, spec_name='optimizer') [](https://microsoft.github.io/agent-lightning/stable/reference/trainer/#__codelineno-0-3) >>> [](https://microsoft.github.io/agent-lightning/stable/reference/trainer/#__codelineno-0-4) >>> # String import path [](https://microsoft.github.io/agent-lightning/stable/reference/trainer/#__codelineno-0-5) >>> optimizer = build_component('torch.optim.AdamW', expected_type=Optimizer, spec_name='optimizer') [](https://microsoft.github.io/agent-lightning/stable/reference/trainer/#__codelineno-0-6) >>> [](https://microsoft.github.io/agent-lightning/stable/reference/trainer/#__codelineno-0-7) >>> # Dict with type and kwargs [](https://microsoft.github.io/agent-lightning/stable/reference/trainer/#__codelineno-0-8) >>> spec = {'type': 'torch.optim.AdamW', 'lr': 0.001} [](https://microsoft.github.io/agent-lightning/stable/reference/trainer/#__codelineno-0-9) >>> optimizer = build_component(spec, expected_type=Optimizer, spec_name='optimizer') [](https://microsoft.github.io/agent-lightning/stable/reference/trainer/#__codelineno-0-10) >>> [](https://microsoft.github.io/agent-lightning/stable/reference/trainer/#__codelineno-0-11) >>> # Class type [](https://microsoft.github.io/agent-lightning/stable/reference/trainer/#__codelineno-0-12) >>> optimizer = build_component(AdamW, expected_type=Optimizer, spec_name='optimizer') [](https://microsoft.github.io/agent-lightning/stable/reference/trainer/#__codelineno-0-13) >>> [](https://microsoft.github.io/agent-lightning/stable/reference/trainer/#__codelineno-0-14) >>> # Factory function [](https://microsoft.github.io/agent-lightning/stable/reference/trainer/#__codelineno-0-15) >>> optimizer = build_component(lambda: AdamW(lr=0.001), expected_type=Optimizer, [](https://microsoft.github.io/agent-lightning/stable/reference/trainer/#__codelineno-0-16) ... spec_name='optimizer')` Execution Strategy[¶](https://microsoft.github.io/agent-lightning/stable/reference/trainer/#execution-strategy "Permanent link") --------------------------------------------------------------------------------------------------------------------------------- ### `agentlightning.ExecutionStrategy` [¶](https://microsoft.github.io/agent-lightning/stable/reference/trainer/#agentlightning.ExecutionStrategy "Permanent link") Coordinate algorithm and runner bundles within a single process abstraction. Strategies decide how many worker bundles to launch, whether to communicate through shared memory or an HTTP boundary, and how to react to shutdown signals. They intentionally avoid inspecting the bundle internals; instead, each bundle remains responsible for its own scheduling semantics. Note Implementations must honor the [execute()](https://microsoft.github.io/agent-lightning/stable/reference/trainer/#agentlightning.ExecutionStrategy.execute " execute(algorithm, runner, store)") contract by propagating `KeyboardInterrupt` and ensuring resources are released when an error occurs on either side of the algorithm/runner pair. #### `execute(algorithm, runner, store)` [¶](https://microsoft.github.io/agent-lightning/stable/reference/trainer/#agentlightning.ExecutionStrategy.execute "Permanent link") Run the provided bundles using the configured orchestration model. Parameters: * **`algorithm`** (`AlgorithmBundle`) – Callable bundle responsible for algorithm execution. * **`runner`** (`RunnerBundle`) – Callable bundle for runner workers. * **`store`** (`[LightningStore](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.LightningStore " agentlightning.LightningStore (agentlightning.store.base.LightningStore)") `) – Concrete [`LightningStore`](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.LightningStore " agentlightning.LightningStore") shared across bundles. Raises: * `NotImplementedError` – Subclasses must provide the orchestration implementation. ### `agentlightning.ClientServerExecutionStrategy` [¶](https://microsoft.github.io/agent-lightning/stable/reference/trainer/#agentlightning.ClientServerExecutionStrategy "Permanent link") Bases: `[ExecutionStrategy](https://microsoft.github.io/agent-lightning/stable/reference/trainer/#agentlightning.ExecutionStrategy " agentlightning.ExecutionStrategy (agentlightning.execution.base.ExecutionStrategy)") ` Run algorithm and runner bundles as separate processes over HTTP. Execution Roles: * `"algorithm"`: Start [`LightningStoreServer`](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.LightningStoreServer " agentlightning.LightningStoreServer") in-process and execute the algorithm bundle against it. * `"runner"`: Connect to an existing server with [`LightningStoreClient`](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.LightningStoreClient " agentlightning.LightningStoreClient") and run the runner bundle locally (spawning multiple processes when requested). * `"both"`: Spawn runner processes first, then execute the algorithm and server on the same machine. This mode orchestrates the full loop locally. When `role == "both"` you may choose which side runs on the main process via `main_process`. The runner-on-main option is limited to `n_runners == 1` because each additional runner requires its own event loop and process. Warning When `main_process == "runner"` the algorithm and HTTP server execute in a child process. Store mutations remain isolated inside that process, so the original store instance passed to [execute()](https://microsoft.github.io/agent-lightning/stable/reference/trainer/#agentlightning.ExecutionStrategy.execute " execute(algorithm, runner, store)") is not updated. Abort Model (four-step escalation): 1. Cooperative stop. Every bundle receives a shared [`MultiprocessingEvent`](https://microsoft.github.io/agent-lightning/stable/reference/trainer/#agentlightning.MultiprocessingEvent " agentlightning.MultiprocessingEvent") (`stop_evt`). Any failure flips the event so peers can exit cleanly. Ctrl+C on the main process also sets the flag. 2. KeyboardInterrupt synthesis. Remaining subprocesses receive `SIGINT` to trigger `KeyboardInterrupt` handlers. 3. Termination. Stubborn processes are asked to `terminate()` (`SIGTERM` on POSIX). 4. Kill. As a last resort `kill()` is invoked (`SIGKILL` on POSIX). This mirrors the semantics implemented in [`SharedMemoryExecutionStrategy`](https://microsoft.github.io/agent-lightning/stable/reference/trainer/#agentlightning.SharedMemoryExecutionStrategy " agentlightning.SharedMemoryExecutionStrategy") but adapts them to multiple processes and the HTTP client/server boundary. #### `__init__(role=None, server_host=None, server_port=None, n_runners=1, graceful_timeout=10.0, terminate_timeout=10.0, main_process='algorithm', managed_store=None, allowed_exit_codes=(0, -15))` [¶](https://microsoft.github.io/agent-lightning/stable/reference/trainer/#agentlightning.ClientServerExecutionStrategy.__init__ "Permanent link") Configure the strategy. Parameters: * **`role`** (`Literal['algorithm', 'runner', 'both'] | None`, default: `None` ) – Which side(s) to run in this process. When omitted, the `AGL_CURRENT_ROLE` environment variable is used. * **`server_host`** (`str | None`, default: `None` ) – Interface the HTTP server binds to when running the algorithm bundle locally. Defaults to `AGL_SERVER_HOST` or `"localhost"` if unset. * **`server_port`** (`int | None`, default: `None` ) – Port for the HTTP server in "algorithm"/"both" modes. Defaults to `AGL_SERVER_PORT` or `4747` if unset. * **`n_runners`** (`int`, default: `1` ) – Number of runner processes to spawn in "runner"/"both". * **`graceful_timeout`** (`float`, default: `10.0` ) – How long to wait (seconds) after setting the stop event before escalating to signals. * **`terminate_timeout`** (`float`, default: `10.0` ) – How long to wait between escalation steps beyond the cooperative phase (re-used for SIGINT, terminate, and kill). * **`main_process`** (`Literal['algorithm', 'runner']`, default: `'algorithm'` ) – Which bundle runs on the main process when `role == "both"`. `"runner"` requires `n_runners == 1` and is primarily intended for debugging. * **`managed_store`** (`bool | None`, default: `None` ) – When `True` (default) the strategy constructs LightningStore client/server wrappers automatically. When `False` the provided `store` is passed directly to the bundles, allowing callers to manage store wrappers manually. * **`allowed_exit_codes`** (`Iterable[int]`, default: `(0, -15)` ) – Allowed exit codes for subprocesses. By default, runner can exit gracefully with code 0 or terminated by SIGTERM (-15). ### `agentlightning.SharedMemoryExecutionStrategy` [¶](https://microsoft.github.io/agent-lightning/stable/reference/trainer/#agentlightning.SharedMemoryExecutionStrategy "Permanent link") Bases: `[ExecutionStrategy](https://microsoft.github.io/agent-lightning/stable/reference/trainer/#agentlightning.ExecutionStrategy " agentlightning.ExecutionStrategy (agentlightning.execution.base.ExecutionStrategy)") ` Execute bundles in a single process with cooperative worker threads. Stop Model: * All bundles share one [`ThreadingEvent`](https://microsoft.github.io/agent-lightning/stable/reference/trainer/#agentlightning.ThreadingEvent " agentlightning.ThreadingEvent") named `stop_evt`. * Only the main thread receives `KeyboardInterrupt`. When Ctrl+C occurs we set `stop_evt`. * Any exception raised inside a bundle sets `stop_evt` so other threads can unwind cooperatively. * Once the bundle running on the main thread exits successfully the treatment depends on `main_thread`: * `"algorithm"`: the runners are asked to stop by setting `stop_evt`. * `"runner"`: the algorithm keeps running until it exits naturally. * Background threads are marked as daemons. We join them briefly and log any stragglers before shutting down. Note Signals other than `SIGINT` (such as `SIGTERM`) are not intercepted; Python's default behavior for those signals is preserved. Events[¶](https://microsoft.github.io/agent-lightning/stable/reference/trainer/#events "Permanent link") --------------------------------------------------------------------------------------------------------- ### `agentlightning.ExecutionEvent` [¶](https://microsoft.github.io/agent-lightning/stable/reference/trainer/#agentlightning.ExecutionEvent "Permanent link") Bases: `Protocol` Protocol capturing the cooperative stop contract shared by strategies. Implementations mirror the API of `threading.Event` and `multiprocessing.Event` so the rest of the execution layer can remain agnostic to the underlying concurrency primitive. Methods: `set: Signal cancellation. The call must be idempotent. clear: Reset the event to the unsignaled state. is_set: Return ``True`` when cancellation has been requested. wait: Block until the event is signaled or an optional timeout elapses.` ### `agentlightning.ThreadingEvent` [¶](https://microsoft.github.io/agent-lightning/stable/reference/trainer/#agentlightning.ThreadingEvent "Permanent link") Thread-safe implementation of [`ExecutionEvent`](https://microsoft.github.io/agent-lightning/stable/reference/trainer/#agentlightning.ExecutionEvent " agentlightning.ExecutionEvent") . ### `agentlightning.MultiprocessingEvent` [¶](https://microsoft.github.io/agent-lightning/stable/reference/trainer/#agentlightning.MultiprocessingEvent "Permanent link") Process-safe implementation of [`ExecutionEvent`](https://microsoft.github.io/agent-lightning/stable/reference/trainer/#agentlightning.ExecutionEvent " agentlightning.ExecutionEvent") . CLI Builder[¶](https://microsoft.github.io/agent-lightning/stable/reference/trainer/#cli-builder "Permanent link") ------------------------------------------------------------------------------------------------------------------- ### `agentlightning.lightning_cli(*classes)` [¶](https://microsoft.github.io/agent-lightning/stable/reference/trainer/#agentlightning.lightning_cli "Permanent link") `[](https://microsoft.github.io/agent-lightning/stable/reference/trainer/#__codelineno-0-1) lightning_cli(cls1: Type[_C1]) -> _C1` `[](https://microsoft.github.io/agent-lightning/stable/reference/trainer/#__codelineno-0-1) lightning_cli( [](https://microsoft.github.io/agent-lightning/stable/reference/trainer/#__codelineno-0-2) cls1: Type[_C1], cls2: Type[_C2] [](https://microsoft.github.io/agent-lightning/stable/reference/trainer/#__codelineno-0-3) ) -> Tuple[_C1, _C2]` `[](https://microsoft.github.io/agent-lightning/stable/reference/trainer/#__codelineno-0-1) lightning_cli( [](https://microsoft.github.io/agent-lightning/stable/reference/trainer/#__codelineno-0-2) cls1: Type[_C1], cls2: Type[_C2], cls3: Type[_C3] [](https://microsoft.github.io/agent-lightning/stable/reference/trainer/#__codelineno-0-3) ) -> Tuple[_C1, _C2, _C3]` `[](https://microsoft.github.io/agent-lightning/stable/reference/trainer/#__codelineno-0-1) lightning_cli( [](https://microsoft.github.io/agent-lightning/stable/reference/trainer/#__codelineno-0-2) cls1: Type[_C1], [](https://microsoft.github.io/agent-lightning/stable/reference/trainer/#__codelineno-0-3) cls2: Type[_C2], [](https://microsoft.github.io/agent-lightning/stable/reference/trainer/#__codelineno-0-4) cls3: Type[_C3], [](https://microsoft.github.io/agent-lightning/stable/reference/trainer/#__codelineno-0-5) cls4: Type[_C4], [](https://microsoft.github.io/agent-lightning/stable/reference/trainer/#__codelineno-0-6) ) -> Tuple[_C1, _C2, _C3, _C4]` `[](https://microsoft.github.io/agent-lightning/stable/reference/trainer/#__codelineno-0-1) lightning_cli( [](https://microsoft.github.io/agent-lightning/stable/reference/trainer/#__codelineno-0-2) *classes: Type[CliConfigurable], [](https://microsoft.github.io/agent-lightning/stable/reference/trainer/#__codelineno-0-3) ) -> Tuple[CliConfigurable, ...]` Parses command-line arguments to configure and instantiate provided CliConfigurable classes. Parameters: * **`*classes`** (`Type[CliConfigurable]`, default: `()` ) – One or more classes that inherit from CliConfigurable. Each class's **init** parameters will be exposed as command-line arguments. Returns: * `CliConfigurable | Tuple[CliConfigurable, ...]` – A tuple of instantiated objects, corresponding to the input classes in order. Logging[¶](https://microsoft.github.io/agent-lightning/stable/reference/trainer/#logging "Permanent link") ----------------------------------------------------------------------------------------------------------- ### `agentlightning.configure_logger(level=logging.INFO, name='agentlightning')` [¶](https://microsoft.github.io/agent-lightning/stable/reference/trainer/#agentlightning.configure_logger "Permanent link") Create or reset a namespaced logger with a consistent console format. This helper clears any previously attached handlers before binding a single `StreamHandler` that writes to standard output. The resulting logger does not propagate to the root logger, preventing duplicate log emission when applications compose multiple logging configurations. Danger This function is deprecated in favor of [`setup_logging`](https://microsoft.github.io/agent-lightning/stable/reference/trainer/#agentlightning.setup_logging " agentlightning.setup_logging(level='INFO', *, console=True, color=True, propagate=False, disable_existing_loggers=False, capture_warnings=False, submodule_levels=None, extra_handlers=None, formatter=None, apply_to=None, files=None)") . Parameters: * **`level`** (`int`, default: `INFO` ) – Logging level applied both to the logger and the installed handler. Defaults to `logging.INFO`. * **`name`** (`str`, default: `'agentlightning'` ) – Dotted path for the logger instance. Defaults to `"agentlightning"`. Returns: * `Logger` – Configured logger instance ready for immediate use. Examples: `[](https://microsoft.github.io/agent-lightning/stable/reference/trainer/#__codelineno-0-1) from agentlightning import configure_logger [](https://microsoft.github.io/agent-lightning/stable/reference/trainer/#__codelineno-0-2) [](https://microsoft.github.io/agent-lightning/stable/reference/trainer/#__codelineno-0-3) logger = configure_logger(level=logging.INFO) [](https://microsoft.github.io/agent-lightning/stable/reference/trainer/#__codelineno-0-4) logger.info("agent-lightning is ready!")` ### `agentlightning.setup_module_logging(level='INFO', *, name='agentlightning', console=True, color=True, propagate=False, disable_existing_loggers=False)` [¶](https://microsoft.github.io/agent-lightning/stable/reference/trainer/#agentlightning.setup_module_logging "Permanent link") Initializes and returns the base logger for `agentlightning`. This function constructs and applies a `dictConfig` configuration for the logger hierarchy rooted at `name`. It supports either rich console formatting (via `RichHandler`) or plain text formatting, based on the `color` argument. Unlike [`setup_logging`](https://microsoft.github.io/agent-lightning/stable/reference/trainer/#agentlightning.setup_logging " agentlightning.setup_logging(level='INFO', *, console=True, color=True, propagate=False, disable_existing_loggers=False, capture_warnings=False, submodule_levels=None, extra_handlers=None, formatter=None, apply_to=None, files=None)") , this function configures only a single logger namespace and does not attach extra handlers or submodule levels. It is primarily used internally by [`setup_logging`](https://microsoft.github.io/agent-lightning/stable/reference/trainer/#agentlightning.setup_logging " agentlightning.setup_logging(level='INFO', *, console=True, color=True, propagate=False, disable_existing_loggers=False, capture_warnings=False, submodule_levels=None, extra_handlers=None, formatter=None, apply_to=None, files=None)") but is also suitable for direct integration in custom logging workflows. ### `agentlightning.setup_logging(level='INFO', *, console=True, color=True, propagate=False, disable_existing_loggers=False, capture_warnings=False, submodule_levels=None, extra_handlers=None, formatter=None, apply_to=None, files=None)` [¶](https://microsoft.github.io/agent-lightning/stable/reference/trainer/#agentlightning.setup_logging "Permanent link") Configures logging for the `agentlightning` logger hierarchy. This function provides a one-stop setup utility for configuring the `agentlightning` root logger and optionally its submodules or external loggers. It supports console logging, colored rich output, per-submodule log levels, and optional handler/formatter injection. The setup is intentionally isolated: it does not modify the global root logger or loggers belonging to other libraries unless explicitly directed via `apply_to`. Parameters: * **`level`** (`int | str`, default: `'INFO'` ) – Logging level for the base `agentlightning` logger. Accepts either an integer (e.g., `logging.DEBUG`) or a string level name (e.g., `"INFO"`). Defaults to `"INFO"`. * **`console`** (`bool`, default: `True` ) – Whether to attach a console handler to the logger. Defaults to `True`. * **`color`** (`bool | Dict[str, Any]`, default: `True` ) – Enables rich-formatted output using `RichHandler` when `True` or a configuration dict. If `False`, a plain text formatter is used instead. Defaults to `True`. * **`propagate`** (`bool`, default: `False` ) – Whether `agentlightning` logs should propagate to ancestor loggers. Defaults to `False`. * **`disable_existing_loggers`** (`bool`, default: `False` ) – Passed to `logging.config.dictConfig`. If `True`, disables all existing configured loggers before applying this configuration. Defaults to `False`. * **`capture_warnings`** (`bool`, default: `False` ) – If `True`, redirects Python `warnings` emitted via the `warnings` module into the logging system. Defaults to `False`. * **`submodule_levels`** (`Optional[dict[str, int | str]]`, default: `None` ) – Mapping of submodule logger names to logging levels. If a specified submodule level is more verbose than the base level, a warning is emitted. * **`extra_handlers`** (`Optional[list[Handler]]`, default: `None` ) – A list of user-provided handlers to attach to the `agentlightning` logger. Handlers are added idempotently; duplicates are not reattached. * **`formatter`** (`Optional[Formatter]`, default: `None` ) – A formatter to apply to any handler under `agentlightning` that does not already have one assigned. Useful for customizing output without overwriting formatters on custom handlers. * **`apply_to`** (`Optional[list[str]]`, default: `None` ) – A list of additional logger names to configure identically to `agentlightning` base logger. Their handlers are replaced with copies of the base handlers, and propagation is disabled to avoid duplicate log emission. * **`files`** (`Optional[str | dict[str, str]]`, default: `None` ) – If a string, attach a FileHandler to the base `agentlightning` logger. If a dict, for each `(logger_name, filename)` pair, attach a FileHandler directly to that logger. Each file handler should use the logger's effective level at creation. Notes * On Windows, this function forces UTF-8 mode in the console to prevent issues with rich output or special characters. * Submodule loggers can generate records below the handler's emission threshold. Whether such records appear depends on both the logger's level and the handler's level. * `apply_to` loggers inherit the same handlers but do not propagate upward, yielding isolated, consistent behavior. Examples: Basic setup: `[](https://microsoft.github.io/agent-lightning/stable/reference/trainer/#__codelineno-0-1) >>> setup()` Enabling debug mode with no color: `[](https://microsoft.github.io/agent-lightning/stable/reference/trainer/#__codelineno-0-1) >>> setup(level="DEBUG", color=False)` Overriding specific submodule levels: `[](https://microsoft.github.io/agent-lightning/stable/reference/trainer/#__codelineno-0-1) >>> setup(submodule_levels={"agentlightning.io": "DEBUG"})` Attaching an additional file handler: `[](https://microsoft.github.io/agent-lightning/stable/reference/trainer/#__codelineno-0-1) >>> fh = logging.FileHandler("app.log") [](https://microsoft.github.io/agent-lightning/stable/reference/trainer/#__codelineno-0-2) >>> setup(extra_handlers=[fh])` Back to top --- # Utilities - Agent-lightning [Skip to content](https://microsoft.github.io/agent-lightning/stable/reference/utilities/#utility-references) ⚠️ You are viewing development documentation. For stable documentation, please visit [https://microsoft.github.io/agent-lightning/stable/](https://microsoft.github.io/agent-lightning/stable/) Utility References[¶](https://microsoft.github.io/agent-lightning/stable/reference/utilities/#utility-references "Permanent link") =================================================================================================================================== ID[¶](https://microsoft.github.io/agent-lightning/stable/reference/utilities/#id "Permanent link") --------------------------------------------------------------------------------------------------- ### `agentlightning.utils.id.generate_id(length)` [¶](https://microsoft.github.io/agent-lightning/stable/reference/utilities/#agentlightning.utils.id.generate_id "Permanent link") Generate a random ID of the given length. Parameters: * **`length`** (`int`) – The length of the ID to generate. Returns: * `str` – A random ID of the given length. Metrics[¶](https://microsoft.github.io/agent-lightning/stable/reference/utilities/#metrics "Permanent link") ------------------------------------------------------------------------------------------------------------- ### `agentlightning.utils.metrics.MetricsBackend` [¶](https://microsoft.github.io/agent-lightning/stable/reference/utilities/#agentlightning.utils.metrics.MetricsBackend "Permanent link") Abstract base class for metrics backends. #### `has_prometheus()` [¶](https://microsoft.github.io/agent-lightning/stable/reference/utilities/#agentlightning.utils.metrics.MetricsBackend.has_prometheus "Permanent link") Check if the backend has prometheus support. #### `inc_counter(name, amount=1.0, labels=None)` `async` [¶](https://microsoft.github.io/agent-lightning/stable/reference/utilities/#agentlightning.utils.metrics.MetricsBackend.inc_counter "Permanent link") Increments a registered counter. Parameters: * **`name`** (`str`) – Metric name (must be registered as a counter). * **`amount`** (`float`, default: `1.0` ) – Increment amount. * **`labels`** (`Optional[LabelDict]`, default: `None` ) – Label values. Raises: * `ValueError` – If the metric is not registered, has the wrong type, or label keys do not match the registered label names. #### `observe_histogram(name, value, labels=None)` `async` [¶](https://microsoft.github.io/agent-lightning/stable/reference/utilities/#agentlightning.utils.metrics.MetricsBackend.observe_histogram "Permanent link") Records an observation for a registered histogram. Parameters: * **`name`** (`str`) – Metric name (must be registered as a histogram). * **`value`** (`float`) – Observed value. * **`labels`** (`Optional[LabelDict]`, default: `None` ) – Label values. Raises: * `ValueError` – If the metric is not registered, has the wrong type, or label keys do not match the registered label names. #### `register_counter(name, label_names=None, group_level=None)` [¶](https://microsoft.github.io/agent-lightning/stable/reference/utilities/#agentlightning.utils.metrics.MetricsBackend.register_counter "Permanent link") Registers a counter metric. Parameters: * **`name`** (`str`) – Metric name. * **`label_names`** (`Optional[Sequence[str]]`, default: `None` ) – List of label names. Order determines the truncation priority for group-level logging. * **`group_level`** (`Optional[int]`, default: `None` ) – Optional per-metric grouping depth for backends that support label grouping (Console). Global backend settings take precedence when provided. Raises: * `ValueError` – If the metric is already registered with a different type or label set. #### `register_histogram(name, label_names=None, buckets=None, group_level=None)` [¶](https://microsoft.github.io/agent-lightning/stable/reference/utilities/#agentlightning.utils.metrics.MetricsBackend.register_histogram "Permanent link") Registers a histogram metric. Parameters: * **`name`** (`str`) – Metric name. * **`label_names`** (`Optional[Sequence[str]]`, default: `None` ) – List of label names. Order determines the truncation priority for group-level logging. * **`buckets`** (`Optional[Sequence[float]]`, default: `None` ) – Bucket boundaries (exclusive upper bounds). If None, the backend may choose defaults. * **`group_level`** (`Optional[int]`, default: `None` ) – Optional per-metric grouping depth for backends that support label grouping (Console). Global backend settings take precedence when provided. Raises: * `ValueError` – If the metric is already registered with a different type or label set. ### `agentlightning.utils.metrics.ConsoleMetricsBackend` [¶](https://microsoft.github.io/agent-lightning/stable/reference/utilities/#agentlightning.utils.metrics.ConsoleMetricsBackend "Permanent link") Bases: `[MetricsBackend](https://microsoft.github.io/agent-lightning/stable/reference/utilities/#agentlightning.utils.metrics.MetricsBackend " agentlightning.utils.metrics.MetricsBackend") ` Console backend with sliding-window aggregations and label grouping. This backend: * Requires explicit metric registration. * Stores timestamped events per (metric\_name, labels) key. * Computes rate and percentiles (P50, P95, P99) over a sliding time window. * Uses a single global logging decision: when logging is triggered, it logs all metric groups, not just the one being updated. Rate is always per second. Label grouping: When logging, label dictionaries are truncated to the first `group_level` label pairs (following the registered label order) and metrics with identical truncated labels are aggregated together. For example: `[](https://microsoft.github.io/agent-lightning/stable/reference/utilities/#__codelineno-0-1) labels = {"method": "GET", "path": "/", "status": "200"} [](https://microsoft.github.io/agent-lightning/stable/reference/utilities/#__codelineno-0-2) group_level = 2 # aggregated labels {"method": "GET", "path": "/"}` If `group_level` is None or < 1, all label combinations for a metric are merged into a single log entry (equivalent to grouping by zero labels). Individual counters or histograms can set their own `group_level` during registration; those values apply only when the backend-level `group_level` is unset, allowing selective overrides. Thread-safety: Runtime updates and snapshotting use two aiologic locks: one for mutating shared state and another that serializes the global logging decision/snapshot capture so other tasks can continue writing. Metric registration happens during initialization, so it is intentionally left lock-free; this assumption is documented here to avoid blocking writes unnecessarily. #### `__init__(window_seconds=60.0, log_interval_seconds=10.0, group_level=None)` [¶](https://microsoft.github.io/agent-lightning/stable/reference/utilities/#agentlightning.utils.metrics.ConsoleMetricsBackend.__init__ "Permanent link") Initializes ConsoleMetricsBackend. Parameters: * **`window_seconds`** (`Optional[float]`, default: `60.0` ) – Sliding window size (in seconds) used when computing rate and percentiles. If None, all in-memory events are used. * **`log_interval_seconds`** (`float`, default: `10.0` ) – Minimum time (in seconds) between log bursts. When the interval elapses, the next metric event triggers a snapshot and logging of all metrics. * **`group_level`** (`Optional[int]`, default: `None` ) – Label grouping depth. When logging, only the first `group_level` labels (following registered order) are retained and metric events sharing those labels are aggregated. If None or < 1, all label combinations collapse into a single group per metric. #### `inc_counter(name, amount=1.0, labels=None)` `async` [¶](https://microsoft.github.io/agent-lightning/stable/reference/utilities/#agentlightning.utils.metrics.ConsoleMetricsBackend.inc_counter "Permanent link") Increments a registered counter metric. See base class for behavior and error conditions. #### `observe_histogram(name, value, labels=None)` `async` [¶](https://microsoft.github.io/agent-lightning/stable/reference/utilities/#agentlightning.utils.metrics.ConsoleMetricsBackend.observe_histogram "Permanent link") Records an observation for a registered histogram metric. See base class for behavior and error conditions. #### `register_counter(name, label_names=None, group_level=None)` [¶](https://microsoft.github.io/agent-lightning/stable/reference/utilities/#agentlightning.utils.metrics.ConsoleMetricsBackend.register_counter "Permanent link") Registers a counter metric. See base class for argument documentation. #### `register_histogram(name, label_names=None, buckets=None, group_level=None)` [¶](https://microsoft.github.io/agent-lightning/stable/reference/utilities/#agentlightning.utils.metrics.ConsoleMetricsBackend.register_histogram "Permanent link") Registers a histogram metric. See base class for argument documentation. ### `agentlightning.utils.metrics.PrometheusMetricsBackend` [¶](https://microsoft.github.io/agent-lightning/stable/reference/utilities/#agentlightning.utils.metrics.PrometheusMetricsBackend "Permanent link") Bases: `[MetricsBackend](https://microsoft.github.io/agent-lightning/stable/reference/utilities/#agentlightning.utils.metrics.MetricsBackend " agentlightning.utils.metrics.MetricsBackend") ` Metrics backend that forwards events to prometheus\_client. All metrics must be registered before use. This backend does not compute any aggregations; it only updates Prometheus metrics. Thread-safety: Registration is protected by a lock. Metric updates assume metrics are registered during initialization and then remain stable. Due to the nature of Prometheus, this backend is only suitable for recording high-volume metrics. Low-volume metrics might be lost if the event has only appeared once. #### `__init__()` [¶](https://microsoft.github.io/agent-lightning/stable/reference/utilities/#agentlightning.utils.metrics.PrometheusMetricsBackend.__init__ "Permanent link") Initializes PrometheusMetricsBackend. Raises: * `ImportError` – If prometheus\_client is not installed. #### `has_prometheus()` [¶](https://microsoft.github.io/agent-lightning/stable/reference/utilities/#agentlightning.utils.metrics.PrometheusMetricsBackend.has_prometheus "Permanent link") Check if the backend has prometheus support. #### `inc_counter(name, amount=1.0, labels=None)` `async` [¶](https://microsoft.github.io/agent-lightning/stable/reference/utilities/#agentlightning.utils.metrics.PrometheusMetricsBackend.inc_counter "Permanent link") Increments a registered Prometheus counter. #### `observe_histogram(name, value, labels=None)` `async` [¶](https://microsoft.github.io/agent-lightning/stable/reference/utilities/#agentlightning.utils.metrics.PrometheusMetricsBackend.observe_histogram "Permanent link") Records an observation for a registered Prometheus histogram. #### `register_counter(name, label_names=None, group_level=None)` [¶](https://microsoft.github.io/agent-lightning/stable/reference/utilities/#agentlightning.utils.metrics.PrometheusMetricsBackend.register_counter "Permanent link") Registers a Prometheus counter metric. #### `register_histogram(name, label_names=None, buckets=None, group_level=None)` [¶](https://microsoft.github.io/agent-lightning/stable/reference/utilities/#agentlightning.utils.metrics.PrometheusMetricsBackend.register_histogram "Permanent link") Registers a Prometheus histogram metric. ### `agentlightning.utils.metrics.MultiMetricsBackend` [¶](https://microsoft.github.io/agent-lightning/stable/reference/utilities/#agentlightning.utils.metrics.MultiMetricsBackend "Permanent link") Bases: `[MetricsBackend](https://microsoft.github.io/agent-lightning/stable/reference/utilities/#agentlightning.utils.metrics.MetricsBackend " agentlightning.utils.metrics.MetricsBackend") ` Metrics backend that forwards calls to multiple underlying backends. #### `__init__(backends)` [¶](https://microsoft.github.io/agent-lightning/stable/reference/utilities/#agentlightning.utils.metrics.MultiMetricsBackend.__init__ "Permanent link") Initializes MultiMetricsBackend. Parameters: * **`backends`** (`Sequence[[MetricsBackend](https://microsoft.github.io/agent-lightning/stable/reference/utilities/#agentlightning.utils.metrics.MetricsBackend " agentlightning.utils.metrics.MetricsBackend") ]`) – Sequence of underlying backends. Raises: * `ValueError` – If no backends are provided. #### `has_prometheus()` [¶](https://microsoft.github.io/agent-lightning/stable/reference/utilities/#agentlightning.utils.metrics.MultiMetricsBackend.has_prometheus "Permanent link") Check if the backend has prometheus support. #### `inc_counter(name, amount=1.0, labels=None)` `async` [¶](https://microsoft.github.io/agent-lightning/stable/reference/utilities/#agentlightning.utils.metrics.MultiMetricsBackend.inc_counter "Permanent link") Increments a counter metric in all underlying backends. #### `observe_histogram(name, value, labels=None)` `async` [¶](https://microsoft.github.io/agent-lightning/stable/reference/utilities/#agentlightning.utils.metrics.MultiMetricsBackend.observe_histogram "Permanent link") Records a histogram observation in all underlying backends. #### `register_counter(name, label_names=None, group_level=None)` [¶](https://microsoft.github.io/agent-lightning/stable/reference/utilities/#agentlightning.utils.metrics.MultiMetricsBackend.register_counter "Permanent link") Registers a counter metric in all underlying backends. #### `register_histogram(name, label_names=None, buckets=None, group_level=None)` [¶](https://microsoft.github.io/agent-lightning/stable/reference/utilities/#agentlightning.utils.metrics.MultiMetricsBackend.register_histogram "Permanent link") Registers a histogram metric in all underlying backends. ### `agentlightning.utils.metrics.setup_multiprocess_prometheus()` [¶](https://microsoft.github.io/agent-lightning/stable/reference/utilities/#agentlightning.utils.metrics.setup_multiprocess_prometheus "Permanent link") Set up prometheus multiprocessing directory if not already configured. ### `agentlightning.utils.metrics.get_prometheus_registry()` [¶](https://microsoft.github.io/agent-lightning/stable/reference/utilities/#agentlightning.utils.metrics.get_prometheus_registry "Permanent link") Get the appropriate prometheus registry based on multiprocessing configuration. ### `agentlightning.utils.metrics.shutdown_metrics(server=None, worker=None, *args, **kwargs)` [¶](https://microsoft.github.io/agent-lightning/stable/reference/utilities/#agentlightning.utils.metrics.shutdown_metrics "Permanent link") Shutdown prometheus metrics. Server Launcher[¶](https://microsoft.github.io/agent-lightning/stable/reference/utilities/#server-launcher "Permanent link") ----------------------------------------------------------------------------------------------------------------------------- ### `agentlightning.utils.server_launcher.PythonServerLauncher` [¶](https://microsoft.github.io/agent-lightning/stable/reference/utilities/#agentlightning.utils.server_launcher.PythonServerLauncher "Permanent link") Unified launcher for FastAPI, using uvicorn or gunicorn per mode/worker count. See [`PythonServerLauncherArgs`](https://microsoft.github.io/agent-lightning/stable/reference/utilities/#agentlightning.utils.server_launcher.PythonServerLauncherArgs " agentlightning.utils.server_launcher.PythonServerLauncherArgs dataclass ") for configuration options. Parameters: * **`app`** (`FastAPI`) – The FastAPI app to launch. * **`args`** (`[PythonServerLauncherArgs](https://microsoft.github.io/agent-lightning/stable/reference/utilities/#agentlightning.utils.server_launcher.PythonServerLauncherArgs " agentlightning.utils.server_launcher.PythonServerLauncherArgs dataclass ") `) – The configuration for the server. * **`serve_context`** (`Optional[AsyncContextManager[Any]]`, default: `None` ) – An optional context manager to apply around the server startup. #### `access_endpoint` `property` [¶](https://microsoft.github.io/agent-lightning/stable/reference/utilities/#agentlightning.utils.server_launcher.PythonServerLauncher.access_endpoint "Permanent link") Return a loopback-friendly URL so health checks succeed even when binding to 0.0.0.0. #### `endpoint` `property` [¶](https://microsoft.github.io/agent-lightning/stable/reference/utilities/#agentlightning.utils.server_launcher.PythonServerLauncher.endpoint "Permanent link") Return the externally advertised host:port pair regardless of accessibility. #### `health_url` `property` [¶](https://microsoft.github.io/agent-lightning/stable/reference/utilities/#agentlightning.utils.server_launcher.PythonServerLauncher.health_url "Permanent link") Build the absolute health-check endpoint from args, if one is configured. #### `__getstate__()` [¶](https://microsoft.github.io/agent-lightning/stable/reference/utilities/#agentlightning.utils.server_launcher.PythonServerLauncher.__getstate__ "Permanent link") Control pickling to prevent server state from being sent to subprocesses. #### `__init__(app, args, serve_context=None)` [¶](https://microsoft.github.io/agent-lightning/stable/reference/utilities/#agentlightning.utils.server_launcher.PythonServerLauncher.__init__ "Permanent link") Initialize the launcher with the FastAPI app, configuration, and optional serve context. #### `is_running()` [¶](https://microsoft.github.io/agent-lightning/stable/reference/utilities/#agentlightning.utils.server_launcher.PythonServerLauncher.is_running "Permanent link") Return True if the server has been started and not yet stopped. #### `reload()` `async` [¶](https://microsoft.github.io/agent-lightning/stable/reference/utilities/#agentlightning.utils.server_launcher.PythonServerLauncher.reload "Permanent link") Restart the server by stopping it if necessary and invoking start again. #### `run_forever()` `async` [¶](https://microsoft.github.io/agent-lightning/stable/reference/utilities/#agentlightning.utils.server_launcher.PythonServerLauncher.run_forever "Permanent link") Start the server and block the caller until it exits, respecting the configured mode. #### `start()` `async` [¶](https://microsoft.github.io/agent-lightning/stable/reference/utilities/#agentlightning.utils.server_launcher.PythonServerLauncher.start "Permanent link") Starts the server according to launch\_mode and n\_workers. #### `stop()` `async` [¶](https://microsoft.github.io/agent-lightning/stable/reference/utilities/#agentlightning.utils.server_launcher.PythonServerLauncher.stop "Permanent link") Stop the server using the inverse of whatever launch mode was used to start it. ### `agentlightning.utils.server_launcher.PythonServerLauncherArgs` `dataclass` [¶](https://microsoft.github.io/agent-lightning/stable/reference/utilities/#agentlightning.utils.server_launcher.PythonServerLauncherArgs "Permanent link") #### `access_host = None` `class-attribute` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/stable/reference/utilities/#agentlightning.utils.server_launcher.PythonServerLauncherArgs.access_host "Permanent link") The hostname or IP address to advertise to the client. If not provided, the server will use the default outbound IPv4 address for this machine. #### `access_log = False` `class-attribute` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/stable/reference/utilities/#agentlightning.utils.server_launcher.PythonServerLauncherArgs.access_log "Permanent link") Whether to turn on access logs. #### `healthcheck_url = None` `class-attribute` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/stable/reference/utilities/#agentlightning.utils.server_launcher.PythonServerLauncherArgs.healthcheck_url "Permanent link") The health check URL to use. If not provided, the server will not be checked for healthiness after starting. #### `host = None` `class-attribute` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/stable/reference/utilities/#agentlightning.utils.server_launcher.PythonServerLauncherArgs.host "Permanent link") The hostname or IP address to bind the server to. #### `kill_unhealthy_server = True` `class-attribute` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/stable/reference/utilities/#agentlightning.utils.server_launcher.PythonServerLauncherArgs.kill_unhealthy_server "Permanent link") Whether to kill the server if it is not healthy after startup. This setting is ignored when `launch_mode` is not `asyncio`. #### `launch_mode = 'asyncio'` `class-attribute` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/stable/reference/utilities/#agentlightning.utils.server_launcher.PythonServerLauncherArgs.launch_mode "Permanent link") The launch mode. `asyncio` is the default mode to runs the server in the current thread. `thread` runs the server in a separate thread. `mp` runs the server in a separate process. #### `log_level = logging.INFO` `class-attribute` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/stable/reference/utilities/#agentlightning.utils.server_launcher.PythonServerLauncherArgs.log_level "Permanent link") The log level to use. #### `n_workers = 1` `class-attribute` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/stable/reference/utilities/#agentlightning.utils.server_launcher.PythonServerLauncherArgs.n_workers "Permanent link") The number of workers to run in the server. Only applicable for `mp` mode. When `n_workers > 1`, the server will be run using Gunicorn. #### `port = None` `class-attribute` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/stable/reference/utilities/#agentlightning.utils.server_launcher.PythonServerLauncherArgs.port "Permanent link") The TCP port to listen on. If not provided, the server will use a random available port. #### `process_join_timeout = 10.0` `class-attribute` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/stable/reference/utilities/#agentlightning.utils.server_launcher.PythonServerLauncherArgs.process_join_timeout "Permanent link") The timeout to wait for the process to join. #### `startup_timeout = 60.0` `class-attribute` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/stable/reference/utilities/#agentlightning.utils.server_launcher.PythonServerLauncherArgs.startup_timeout "Permanent link") The timeout to wait for the server to start up. #### `thread_join_timeout = 10.0` `class-attribute` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/stable/reference/utilities/#agentlightning.utils.server_launcher.PythonServerLauncherArgs.thread_join_timeout "Permanent link") The timeout to wait for the thread to join. #### `timeout_keep_alive = 30` `class-attribute` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/stable/reference/utilities/#agentlightning.utils.server_launcher.PythonServerLauncherArgs.timeout_keep_alive "Permanent link") The timeout to keep the connection alive. ### `agentlightning.utils.server_launcher.LaunchMode = Literal['asyncio', 'thread', 'mp']` `module-attribute` [¶](https://microsoft.github.io/agent-lightning/stable/reference/utilities/#agentlightning.utils.server_launcher.LaunchMode "Permanent link") The launch mode for the server. OpenTelemetry[¶](https://microsoft.github.io/agent-lightning/stable/reference/utilities/#opentelemetry "Permanent link") ------------------------------------------------------------------------------------------------------------------------- ### `agentlightning.utils.otel.full_qualified_name(obj)` [¶](https://microsoft.github.io/agent-lightning/stable/reference/utilities/#agentlightning.utils.otel.full_qualified_name "Permanent link") ### `agentlightning.utils.otel.get_tracer_provider(inspect=True)` [¶](https://microsoft.github.io/agent-lightning/stable/reference/utilities/#agentlightning.utils.otel.get_tracer_provider "Permanent link") Get the OpenTelemetry tracer provider configured for Agent Lightning. Parameters: * **`inspect`** (`bool`, default: `True` ) – Whether to inspect the tracer provider and log its configuration. When it's on, make sure you also set the logger level to DEBUG to see the logs. ### `agentlightning.utils.otel.get_tracer(use_active_span_processor=True)` [¶](https://microsoft.github.io/agent-lightning/stable/reference/utilities/#agentlightning.utils.otel.get_tracer "Permanent link") Resolve the OpenTelemetry tracer configured for Agent Lightning. Parameters: * **`use_active_span_processor`** (`bool`, default: `True` ) – Whether to use the active span processor. Returns: * `Tracer` – OpenTelemetry tracer tagged with the `agentlightning` instrumentation name. Raises: * `RuntimeError` – If OpenTelemetry was not initialized before calling this helper. ### `agentlightning.utils.otel.make_tag_attributes(tags)` [¶](https://microsoft.github.io/agent-lightning/stable/reference/utilities/#agentlightning.utils.otel.make_tag_attributes "Permanent link") Convert a list of tags into flattened attributes for span tagging. There is no syntax enforced for tags, they are just strings. For example: `[](https://microsoft.github.io/agent-lightning/stable/reference/utilities/#__codelineno-0-1) ["gen_ai.model:gpt-4", "reward.extrinsic"]` ### `agentlightning.utils.otel.extract_tags_from_attributes(attributes)` [¶](https://microsoft.github.io/agent-lightning/stable/reference/utilities/#agentlightning.utils.otel.extract_tags_from_attributes "Permanent link") Extract tag attributes from flattened span attributes. Parameters: * **`attributes`** (`Dict[str, Any]`) – A dictionary of flattened span attributes. ### `agentlightning.utils.otel.make_link_attributes(links)` [¶](https://microsoft.github.io/agent-lightning/stable/reference/utilities/#agentlightning.utils.otel.make_link_attributes "Permanent link") Convert a dictionary of links into flattened attributes for span linking. Links example: `[](https://microsoft.github.io/agent-lightning/stable/reference/utilities/#__codelineno-0-1) { [](https://microsoft.github.io/agent-lightning/stable/reference/utilities/#__codelineno-0-2) "gen_ai.response.id": "response-123", [](https://microsoft.github.io/agent-lightning/stable/reference/utilities/#__codelineno-0-3) "span_id": "abcd-efgh-ijkl", [](https://microsoft.github.io/agent-lightning/stable/reference/utilities/#__codelineno-0-4) }` ### `agentlightning.utils.otel.query_linked_spans(spans, links)` [¶](https://microsoft.github.io/agent-lightning/stable/reference/utilities/#agentlightning.utils.otel.query_linked_spans "Permanent link") Query spans that are linked by the given link attributes. Parameters: * **`spans`** (`Sequence[T_SpanLike]`) – A sequence of spans to search. * **`links`** (`List[[LinkPydanticModel](https://microsoft.github.io/agent-lightning/stable/reference/semconv/#agentlightning.semconv.LinkPydanticModel " LinkPydanticModel (agentlightning.semconv.LinkPydanticModel)") ]`) – A list of link attributes to match. Returns: * `List[T_SpanLike]` – A list of spans that match the given link attributes. ### `agentlightning.utils.otel.extract_links_from_attributes(attributes)` [¶](https://microsoft.github.io/agent-lightning/stable/reference/utilities/#agentlightning.utils.otel.extract_links_from_attributes "Permanent link") Extract link attributes from flattened span attributes. Parameters: * **`attributes`** (`Dict[str, Any]`) – A dictionary of flattened span attributes. ### `agentlightning.utils.otel.filter_attributes(attributes, prefix)` [¶](https://microsoft.github.io/agent-lightning/stable/reference/utilities/#agentlightning.utils.otel.filter_attributes "Permanent link") Filter attributes that start with the given prefix. The attribute must start with `prefix.` or be exactly `prefix` to be included. Parameters: * **`attributes`** (`Dict[str, Any]`) – A dictionary of span attributes. * **`prefix`** (`str`) – The prefix to filter by. Returns: * `Dict[str, Any]` – A dictionary of attributes that start with the given prefix. ### `agentlightning.utils.otel.filter_and_unflatten_attributes(attributes, prefix)` [¶](https://microsoft.github.io/agent-lightning/stable/reference/utilities/#agentlightning.utils.otel.filter_and_unflatten_attributes "Permanent link") Filter attributes that start with the given prefix and unflatten them. The prefix will be removed during unflattening. Parameters: * **`attributes`** (`Dict[str, Any]`) – A dictionary of span attributes. * **`prefix`** (`str`) – The prefix to filter by. Returns: * `Union[Dict[str, Any], List[Any]]` – A nested dictionary or list of attributes that start with the given prefix. ### `agentlightning.utils.otel.flatten_attributes(nested_data, *, expand_leaf_lists=False)` [¶](https://microsoft.github.io/agent-lightning/stable/reference/utilities/#agentlightning.utils.otel.flatten_attributes "Permanent link") Flatten a nested dictionary or list into a flat dictionary with dotted keys. This function recursively traverses dictionaries and lists, producing a flat key-value mapping where nested paths are represented via dot-separated keys. Lists are indexed numerically. Example: `>>> flatten_attributes({"a": {"b": 1, "c": [2, 3]}}, expand_leaf_lists=True) {"a.b": 1, "a.c.0": 2, "a.c.1": 3}` Parameters: * **`nested_data`** (`Union[Dict[str, Any], List[Any]]`) – A nested structure composed of dictionaries, lists, or primitive values. * **`expand_leaf_lists`** (`bool`, default: `False` ) – Whether to expand lists composed only of primitive values. When `False` (the default), lists of str/int/float/bool are treated as leaf values and stored without enumerating their indices. Returns: * `Dict[str, Any]` – A flat dictionary mapping dotted-string paths to primitive values. ### `agentlightning.utils.otel.unflatten_attributes(flat_data)` [¶](https://microsoft.github.io/agent-lightning/stable/reference/utilities/#agentlightning.utils.otel.unflatten_attributes "Permanent link") Reconstruct a nested dictionary/list structure from a flat dictionary. Keys are dot-separated paths. Segments that are digit strings will only become list indices if _all_ keys in that dict form a consecutive 0..n-1 range. Otherwise they remain dict keys. Example: `>>> unflatten_attributes({"a.b": 1, "a.c.0": 2, "a.c.1": 3}) {"a": {"b": 1, "c": [2, 3]}}` Parameters: * **`flat_data`** (`Dict[str, Any]`) – A dictionary whose keys are dot-separated paths and whose values are primitive data elements. Returns: * `Union[Dict[str, Any], List[Any]]` – A nested dictionary (and lists where appropriate) corresponding to * `Union[Dict[str, Any], List[Any]]` – the flattened structure. ### `agentlightning.utils.otel.sanitize_attribute_value(object, force=True)` [¶](https://microsoft.github.io/agent-lightning/stable/reference/utilities/#agentlightning.utils.otel.sanitize_attribute_value "Permanent link") Sanitize an attribute value to be a valid OpenTelemetry attribute value. ### `agentlightning.utils.otel.sanitize_attributes(attributes, force=True)` [¶](https://microsoft.github.io/agent-lightning/stable/reference/utilities/#agentlightning.utils.otel.sanitize_attributes "Permanent link") Sanitize a dictionary of attributes to be a valid OpenTelemetry attributes. Parameters: * **`attributes`** (`Dict[str, Any]`) – A dictionary of attributes to sanitize. * **`force`** (`bool`, default: `True` ) – Whether to force sanitization even when the value is not JSON serializable. ### `agentlightning.utils.otel.sanitize_list_attribute_sanity(maybe_list)` [¶](https://microsoft.github.io/agent-lightning/stable/reference/utilities/#agentlightning.utils.otel.sanitize_list_attribute_sanity "Permanent link") Try to sanitize a list of attributes to be a valid OpenTelemetry attribute value. Raise error if the list contains multiple types of primitive values. ### `agentlightning.utils.otel.check_attributes_sanity(attributes)` [¶](https://microsoft.github.io/agent-lightning/stable/reference/utilities/#agentlightning.utils.otel.check_attributes_sanity "Permanent link") Check if a dictionary of attributes is a valid OpenTelemetry attributes. ### `agentlightning.utils.otel.format_exception_attributes(exception)` [¶](https://microsoft.github.io/agent-lightning/stable/reference/utilities/#agentlightning.utils.otel.format_exception_attributes "Permanent link") Format an exception into a dictionary of attributes. OTLP[¶](https://microsoft.github.io/agent-lightning/stable/reference/utilities/#otlp "Permanent link") ------------------------------------------------------------------------------------------------------- ### `agentlightning.utils.otlp.handle_otlp_export(request, request_message_cls, response_message_cls, message_callback, signal_name)` `async` [¶](https://microsoft.github.io/agent-lightning/stable/reference/utilities/#agentlightning.utils.otlp.handle_otlp_export "Permanent link") Generic handler for /v1/traces, /v1/metrics, /v1/logs. Convert the OTLP Protobuf request to a JSON-like object. ### `agentlightning.utils.otlp.spans_from_proto(request, sequence_id_bulk_issuer)` `async` [¶](https://microsoft.github.io/agent-lightning/stable/reference/utilities/#agentlightning.utils.otlp.spans_from_proto "Permanent link") Parse an OTLP proto payload into List\[Span\]. A store is needed here for generating a sequence ID for each span. System Snapshot[¶](https://microsoft.github.io/agent-lightning/stable/reference/utilities/#system-snapshot "Permanent link") ----------------------------------------------------------------------------------------------------------------------------- ### `agentlightning.utils.system_snapshot.system_snapshot(include_gpu=False)` [¶](https://microsoft.github.io/agent-lightning/stable/reference/utilities/#agentlightning.utils.system_snapshot.system_snapshot "Permanent link") Capture a snapshot of the system's hardware and software information. Parameters: * **`include_gpu`** (`bool`, default: `False` ) – Whether to include GPU information. Returns: * `Dict[str, Any]` – A dictionary containing the system's hardware and software information. Back to top --- # Types - Agent-lightning [Skip to content](https://microsoft.github.io/agent-lightning/stable/reference/types/#type-references) ⚠️ You are viewing development documentation. For stable documentation, please visit [https://microsoft.github.io/agent-lightning/stable/](https://microsoft.github.io/agent-lightning/stable/) Type References[¶](https://microsoft.github.io/agent-lightning/stable/reference/types/#type-references "Permanent link") ========================================================================================================================= Core Types[¶](https://microsoft.github.io/agent-lightning/stable/reference/types/#core-types "Permanent link") --------------------------------------------------------------------------------------------------------------- ### `agentlightning.Triplet` [¶](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.Triplet "Permanent link") Bases: `BaseModel` Single interaction turn captured during reinforcement learning. ### `agentlightning.RolloutRawResult = Union[None, float, List[ReadableSpan], List[Span], List[SpanCoreFields]]` `module-attribute` [¶](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.RolloutRawResult "Permanent link") Rollout result type. Possible return values of [`rollout`](https://microsoft.github.io/agent-lightning/stable/reference/agent/#agentlightning.LitAgent.rollout " rollout(task, resources, rollout)") . ### `agentlightning.RolloutMode = Literal['train', 'val', 'test']` `module-attribute` [¶](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.RolloutMode "Permanent link") Possible rollout modes. ### `agentlightning.GenericResponse` [¶](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.GenericResponse "Permanent link") Bases: `BaseModel` Generic server response used by compatibility endpoints. Deprecated This response is no longer used by the new [`LightningStore`](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.LightningStore " agentlightning.LightningStore") APIs. Attributes: * **`status`** (`str`) – Status string describing the result of the request. * **`message`** (`Optional[str]`) – Optional human readable explanation. * **`data`** (`Optional[Dict[str, Any]]`) – Arbitrary payload serialized as JSON. ### `agentlightning.ParallelWorkerBase` [¶](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.ParallelWorkerBase "Permanent link") Base class for workloads executed across multiple worker processes. The lifecycle is orchestrated by the main process: * [`init()`](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.ParallelWorkerBase.init " init(*args, **kwargs)") prepares shared state. * Each worker calls [`init_worker()`](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.ParallelWorkerBase.init_worker " init_worker(worker_id, *args, **kwargs)") during start-up. * [`run()`](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.ParallelWorkerBase.run " run(*args, **kwargs)") performs the parallel workload. * Workers call [`teardown_worker()`](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.ParallelWorkerBase.teardown_worker " teardown_worker(worker_id, *args, **kwargs)") before exiting. * The main process finalizes through [`teardown()`](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.ParallelWorkerBase.teardown " teardown(*args, **kwargs)") . Subclasses must implement [`run()`](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.ParallelWorkerBase.run " run(*args, **kwargs)") and can override other lifecycle hooks. #### `__init__()` [¶](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.ParallelWorkerBase.__init__ "Permanent link") Initialize the base class. This method can be overridden by subclasses. #### `init(*args, **kwargs)` [¶](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.ParallelWorkerBase.init "Permanent link") Initialize before spawning the workers. This method can be overridden by subclasses. #### `init_worker(worker_id, *args, **kwargs)` [¶](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.ParallelWorkerBase.init_worker "Permanent link") Initialize the worker. This method can be overridden by subclasses. #### `run(*args, **kwargs)` [¶](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.ParallelWorkerBase.run "Permanent link") Run the workload. This method can be overridden by subclasses. #### `teardown(*args, **kwargs)` [¶](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.ParallelWorkerBase.teardown "Permanent link") Teardown after the workers have exited. This method can be overridden by subclasses. #### `teardown_worker(worker_id, *args, **kwargs)` [¶](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.ParallelWorkerBase.teardown_worker "Permanent link") Teardown the worker. This method can be overridden by subclasses. ### `agentlightning.Dataset` [¶](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.Dataset "Permanent link") Bases: `Protocol`, `Generic[T_co]` The general interface for a dataset. It's currently implemented as a protocol, having a similar interface to `torch.utils.data.Dataset`. You don't have to inherit from this class; you can use a simple list if you want to. ### `agentlightning.AttemptStatus = Literal['preparing', 'running', 'failed', 'succeeded', 'unresponsive', 'timeout']` `module-attribute` [¶](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.AttemptStatus "Permanent link") The status of an attempt. ### `agentlightning.RolloutStatus = Literal['queuing', 'preparing', 'running', 'failed', 'succeeded', 'cancelled', 'requeuing']` `module-attribute` [¶](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.RolloutStatus "Permanent link") The status of a rollout. ### `agentlightning.RolloutConfig` [¶](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.RolloutConfig "Permanent link") Bases: `BaseModel` Configuration controlling rollout retries and timeouts. #### `max_attempts = Field(default=1, ge=1)` `class-attribute` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.RolloutConfig.max_attempts "Permanent link") The maximum number of attempts for the rollout, including the first attempt. #### `retry_condition = Field(default_factory=(cast(Callable[[], List[AttemptStatus]], list)))` `class-attribute` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.RolloutConfig.retry_condition "Permanent link") The list of statuses that should trigger a retry. #### `timeout_seconds = None` `class-attribute` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.RolloutConfig.timeout_seconds "Permanent link") The timeout for the rollout, in seconds. None indicates no timeout. #### `unresponsive_seconds = None` `class-attribute` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.RolloutConfig.unresponsive_seconds "Permanent link") The unresponsive timeout for the rollout, in seconds. None indicates no unresponsive timeout. ### `agentlightning.Rollout` [¶](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.Rollout "Permanent link") Bases: `BaseModel` #### `config = Field(default_factory=RolloutConfig)` `class-attribute` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.Rollout.config "Permanent link") Retry and timeout configuration associated with the rollout. #### `end_time = None` `class-attribute` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.Rollout.end_time "Permanent link") Timestamp when the rollout ended. #### `input` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.Rollout.input "Permanent link") Task input used to generate the rollout. #### `metadata = None` `class-attribute` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.Rollout.metadata "Permanent link") Additional metadata attached to the rollout. #### `mode = None` `class-attribute` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.Rollout.mode "Permanent link") Execution mode such as `"train"`, `"val"` or `"test"`. See [`RolloutMode`](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.RolloutMode " agentlightning.RolloutMode = Literal['train', 'val', 'test'] module-attribute ") . #### `resources_id = None` `class-attribute` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.Rollout.resources_id "Permanent link") Identifier of the resources required to execute the rollout. #### `rollout_id` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.Rollout.rollout_id "Permanent link") Unique identifier for the rollout. #### `start_time` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.Rollout.start_time "Permanent link") Timestamp when the rollout started. #### `status = 'queuing'` `class-attribute` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.Rollout.status "Permanent link") Latest status emitted by the controller. ### `agentlightning.EnqueueRolloutRequest` [¶](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.EnqueueRolloutRequest "Permanent link") Bases: `BaseModel` Payload describing a rollout to be queued via [`enqueue_rollout`](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.LightningStore.enqueue_rollout " enqueue_rollout(input, mode=None, resources_id=None, config=None, metadata=None) async ") . A subset of fields from [`Rollout`](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.Rollout " agentlightning.Rollout") used for queuing new rollouts. #### `config = None` `class-attribute` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.EnqueueRolloutRequest.config "Permanent link") Retry and timeout configuration associated with the rollout. #### `input` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.EnqueueRolloutRequest.input "Permanent link") Task input used to generate the rollout. #### `metadata = None` `class-attribute` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.EnqueueRolloutRequest.metadata "Permanent link") Additional metadata attached to the rollout. #### `mode = None` `class-attribute` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.EnqueueRolloutRequest.mode "Permanent link") Execution mode such as `"train"`, `"val"` or `"test"`. See [`RolloutMode`](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.RolloutMode " agentlightning.RolloutMode = Literal['train', 'val', 'test'] module-attribute ") . #### `resources_id = None` `class-attribute` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.EnqueueRolloutRequest.resources_id "Permanent link") Identifier of the resources required to execute the rollout. ### `agentlightning.Attempt` [¶](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.Attempt "Permanent link") Bases: `BaseModel` Execution attempt for a rollout, including metadata for retries. #### `attempt_id` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.Attempt.attempt_id "Permanent link") The universal id for current attempt. #### `end_time = None` `class-attribute` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.Attempt.end_time "Permanent link") The time when the attempt has ended. #### `last_heartbeat_time = None` `class-attribute` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.Attempt.last_heartbeat_time "Permanent link") The last time when the worker has reported progress (i.e., a span). #### `metadata = None` `class-attribute` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.Attempt.metadata "Permanent link") A bucket for any other relevant information. #### `rollout_id` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.Attempt.rollout_id "Permanent link") The rollout which this attempt belongs to. #### `sequence_id` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.Attempt.sequence_id "Permanent link") The sequence number of the attempt, starting from 1. #### `start_time` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.Attempt.start_time "Permanent link") The time when the attempt has started. #### `status = 'preparing'` `class-attribute` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.Attempt.status "Permanent link") The status of the attempt. #### `worker_id = None` `class-attribute` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.Attempt.worker_id "Permanent link") The rollout worker which is executing this attempt. ### `agentlightning.AttemptedRollout` [¶](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.AttemptedRollout "Permanent link") Bases: `[Rollout](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.Rollout " agentlightning.Rollout (agentlightning.types.core.Rollout)") ` Rollout paired with the currently active attempt. #### `attempt` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.AttemptedRollout.attempt "Permanent link") The attempt that is currently processing the rollout. ### `agentlightning.Worker` [¶](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.Worker "Permanent link") Bases: `BaseModel` Worker information. This is actually the same as Runner info. #### `current_attempt_id = None` `class-attribute` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.Worker.current_attempt_id "Permanent link") The ID of the current attempt that the worker is processing. #### `current_rollout_id = None` `class-attribute` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.Worker.current_rollout_id "Permanent link") The ID of the current rollout that the worker is processing. #### `heartbeat_stats = None` `class-attribute` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.Worker.heartbeat_stats "Permanent link") Statistics about the worker's heartbeat. #### `last_busy_time = None` `class-attribute` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.Worker.last_busy_time "Permanent link") The last time when the worker has started an attempt and became busy. #### `last_dequeue_time = None` `class-attribute` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.Worker.last_dequeue_time "Permanent link") The last time when the worker has tried to dequeue a rollout. #### `last_heartbeat_time = None` `class-attribute` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.Worker.last_heartbeat_time "Permanent link") The last time when the worker has reported the stats. #### `last_idle_time = None` `class-attribute` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.Worker.last_idle_time "Permanent link") The last time when the worker has triggered the end of an attempt and became idle. #### `status = 'unknown'` `class-attribute` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.Worker.status "Permanent link") The status of the worker. #### `worker_id` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.Worker.worker_id "Permanent link") The ID of the worker. ### `agentlightning.WorkerStatus = Literal['idle', 'busy', 'unknown']` `module-attribute` [¶](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.WorkerStatus "Permanent link") ### `agentlightning.Hook` [¶](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.Hook "Permanent link") Bases: `[ParallelWorkerBase](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.ParallelWorkerBase " agentlightning.ParallelWorkerBase (agentlightning.types.core.ParallelWorkerBase)") ` Base class for defining hooks in the agent runner's lifecycle. #### `on_rollout_end(*, agent, runner, rollout, spans)` `async` [¶](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.Hook.on_rollout_end "Permanent link") Hook called after a rollout _attempt_ completes. Parameters: * **`agent`** (`[LitAgent](https://microsoft.github.io/agent-lightning/stable/reference/agent/#agentlightning.LitAgent " agentlightning.LitAgent (agentlightning.litagent.LitAgent)") [Any]`) – The [`LitAgent`](https://microsoft.github.io/agent-lightning/stable/reference/agent/#agentlightning.LitAgent " agentlightning.LitAgent") instance associated with the runner. * **`runner`** (`[Runner](https://microsoft.github.io/agent-lightning/stable/reference/runner/#agentlightning.Runner " agentlightning.Runner (agentlightning.runner.base.Runner)") [Any]`) – The [`Runner`](https://microsoft.github.io/agent-lightning/stable/reference/runner/#agentlightning.Runner " agentlightning.Runner") managing the rollout. * **`rollout`** (`[Rollout](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.Rollout " agentlightning.Rollout (agentlightning.types.core.Rollout)") `) – The [`Rollout`](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.Rollout " agentlightning.Rollout") object that has been processed. * **`spans`** (`Union[List[ReadableSpan], List[[Span](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.Span " agentlightning.Span (agentlightning.types.tracer.Span)") ]]`) – The spans that have been added to the store. Subclasses can override this method for cleanup or additional logging. By default, this is a no-op. #### `on_rollout_start(*, agent, runner, rollout)` `async` [¶](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.Hook.on_rollout_start "Permanent link") Hook called immediately before a rollout _attempt_ begins. Parameters: * **`agent`** (`[LitAgent](https://microsoft.github.io/agent-lightning/stable/reference/agent/#agentlightning.LitAgent " agentlightning.LitAgent (agentlightning.litagent.LitAgent)") [Any]`) – The [`LitAgent`](https://microsoft.github.io/agent-lightning/stable/reference/agent/#agentlightning.LitAgent " agentlightning.LitAgent") instance associated with the runner. * **`runner`** (`[Runner](https://microsoft.github.io/agent-lightning/stable/reference/runner/#agentlightning.Runner " agentlightning.Runner (agentlightning.runner.base.Runner)") [Any]`) – The [`Runner`](https://microsoft.github.io/agent-lightning/stable/reference/runner/#agentlightning.Runner " agentlightning.Runner") managing the rollout. * **`rollout`** (`[Rollout](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.Rollout " agentlightning.Rollout (agentlightning.types.core.Rollout)") `) – The [`Rollout`](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.Rollout " agentlightning.Rollout") object that will be processed. Subclasses can override this method to implement custom logic such as logging, metric collection, or resource setup. By default, this is a no-op. #### `on_trace_end(*, agent, runner, tracer, rollout)` `async` [¶](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.Hook.on_trace_end "Permanent link") Hook called immediately after the rollout completes but before the tracer exits the trace context. Parameters: * **`agent`** (`[LitAgent](https://microsoft.github.io/agent-lightning/stable/reference/agent/#agentlightning.LitAgent " agentlightning.LitAgent (agentlightning.litagent.LitAgent)") [Any]`) – The [`LitAgent`](https://microsoft.github.io/agent-lightning/stable/reference/agent/#agentlightning.LitAgent " agentlightning.LitAgent") instance associated with the runner. * **`runner`** (`[Runner](https://microsoft.github.io/agent-lightning/stable/reference/runner/#agentlightning.Runner " agentlightning.Runner (agentlightning.runner.base.Runner)") [Any]`) – The [`Runner`](https://microsoft.github.io/agent-lightning/stable/reference/runner/#agentlightning.Runner " agentlightning.Runner") managing the rollout. * **`tracer`** (`[Tracer](https://microsoft.github.io/agent-lightning/stable/reference/runner/#agentlightning.Tracer " agentlightning.Tracer (agentlightning.tracer.base.Tracer)") `) – The [`Tracer`](https://microsoft.github.io/agent-lightning/stable/reference/runner/#agentlightning.Tracer " agentlightning.Tracer") instance associated with the runner. * **`rollout`** (`[Rollout](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.Rollout " agentlightning.Rollout (agentlightning.types.core.Rollout)") `) – The [`Rollout`](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.Rollout " agentlightning.Rollout") object that has been processed. Subclasses can override this method to implement custom logic such as logging, metric collection, or resource cleanup. By default, this is a no-op. #### `on_trace_start(*, agent, runner, tracer, rollout)` `async` [¶](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.Hook.on_trace_start "Permanent link") Hook called immediately after the tracer enters the trace context but before the rollout begins. Parameters: * **`agent`** (`[LitAgent](https://microsoft.github.io/agent-lightning/stable/reference/agent/#agentlightning.LitAgent " agentlightning.LitAgent (agentlightning.litagent.LitAgent)") [Any]`) – The [`LitAgent`](https://microsoft.github.io/agent-lightning/stable/reference/agent/#agentlightning.LitAgent " agentlightning.LitAgent") instance associated with the runner. * **`runner`** (`[Runner](https://microsoft.github.io/agent-lightning/stable/reference/runner/#agentlightning.Runner " agentlightning.Runner (agentlightning.runner.base.Runner)") [Any]`) – The [`Runner`](https://microsoft.github.io/agent-lightning/stable/reference/runner/#agentlightning.Runner " agentlightning.Runner") managing the rollout. * **`tracer`** (`[Tracer](https://microsoft.github.io/agent-lightning/stable/reference/runner/#agentlightning.Tracer " agentlightning.Tracer (agentlightning.tracer.base.Tracer)") `) – The [`Tracer`](https://microsoft.github.io/agent-lightning/stable/reference/runner/#agentlightning.Tracer " agentlightning.Tracer") instance associated with the runner. * **`rollout`** (`[Rollout](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.Rollout " agentlightning.Rollout (agentlightning.types.core.Rollout)") `) – The [`Rollout`](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.Rollout " agentlightning.Rollout") object that will be processed. Subclasses can override this method to implement custom logic such as logging, metric collection, or resource setup. By default, this is a no-op. ### `agentlightning.PaginatedResult` [¶](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.PaginatedResult "Permanent link") Bases: `BaseModel`, `Sequence[T_item]` Result of a paginated query. Behaves like a sequence, but also carries pagination metadata (limit, offset, total). #### `items` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.PaginatedResult.items "Permanent link") Items in the result. #### `limit` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.PaginatedResult.limit "Permanent link") Limit of the result. #### `offset` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.PaginatedResult.offset "Permanent link") Offset of the result. #### `total` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.PaginatedResult.total "Permanent link") Total number of items in the collection. ### `agentlightning.FilterOptions = Mapping[Union[str, Literal['_aggregate', '_must']], Union[FilterField, Literal['and', 'or'], Mapping[str, FilterField]]]` `module-attribute` [¶](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.FilterOptions "Permanent link") A mapping of field name -> operator dict. Each operator dict can contain: * "exact": value for exact equality. * "within": iterable of allowed values. * "contains": substring to search for in string fields. The filter can also have a special field called "\_aggregate" that can be used to specify the logic to combine the results of the filters: * "and": all conditions must match. This is the default value if not specified. * "or": at least one condition must match. All conditions within a field and between different fields are stored in a unified pool and combined using `_aggregate`. The filter can also have a special group called "\_must", which is a mapping of filters that must all match, no matter whether the aggregate logic is "and" or "or". Example: `[](https://microsoft.github.io/agent-lightning/stable/reference/types/#__codelineno-0-1) { [](https://microsoft.github.io/agent-lightning/stable/reference/types/#__codelineno-0-2) "_aggregate": "or", [](https://microsoft.github.io/agent-lightning/stable/reference/types/#__codelineno-0-3) "_must": { [](https://microsoft.github.io/agent-lightning/stable/reference/types/#__codelineno-0-4) "city": {"exact": "New York"}, [](https://microsoft.github.io/agent-lightning/stable/reference/types/#__codelineno-0-5) "timezone": {"within": ["America/New_York", "America/Los_Angeles"]}, [](https://microsoft.github.io/agent-lightning/stable/reference/types/#__codelineno-0-6) }, [](https://microsoft.github.io/agent-lightning/stable/reference/types/#__codelineno-0-7) "status": {"exact": "active"}, [](https://microsoft.github.io/agent-lightning/stable/reference/types/#__codelineno-0-8) "id": {"within": [1, 2, 3]}, [](https://microsoft.github.io/agent-lightning/stable/reference/types/#__codelineno-0-9) "name": {"contains": "foo"}, [](https://microsoft.github.io/agent-lightning/stable/reference/types/#__codelineno-0-10) }` ### `agentlightning.SortOptions` [¶](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.SortOptions "Permanent link") Bases: `TypedDict` Options for sorting the collection. #### `name` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.SortOptions.name "Permanent link") The name of the field to sort by. #### `order` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.SortOptions.order "Permanent link") The order to sort by. ### `agentlightning.FilterField` [¶](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.FilterField "Permanent link") Bases: `TypedDict` An operator dict for a single field. Resources[¶](https://microsoft.github.io/agent-lightning/stable/reference/types/#resources "Permanent link") ------------------------------------------------------------------------------------------------------------- ### `agentlightning.Resource` [¶](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.Resource "Permanent link") Bases: `BaseModel` Base class for tunable resources distributed to executors. #### `resource_type` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.Resource.resource_type "Permanent link") Alias of the resource type. ### `agentlightning.LLM` [¶](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.LLM "Permanent link") Bases: `[Resource](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.Resource " agentlightning.Resource (agentlightning.types.resources.Resource)") ` Resource that identifies an LLM endpoint and its configuration. #### `api_key = None` `class-attribute` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.LLM.api_key "Permanent link") Optional secret used to authenticate requests. #### `endpoint` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.LLM.endpoint "Permanent link") The URL of the LLM API endpoint. #### `model` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.LLM.model "Permanent link") The identifier for the model to be used (e.g., 'gpt-4o'). #### `sampling_parameters = Field(default_factory=dict)` `class-attribute` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.LLM.sampling_parameters "Permanent link") A dictionary of hyperparameters for model inference, such as temperature, top\_p, etc. #### `get_base_url(*args, **kwargs)` [¶](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.LLM.get_base_url "Permanent link") Return the base URL consumed by OpenAI-compatible clients. Users are encouraged to use `get_base_url(rollout_id, attempt_id)` to get the LLM endpoint instead of accessing `.endpoint` directly. ### `agentlightning.ProxyLLM` [¶](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.ProxyLLM "Permanent link") Bases: `[LLM](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.LLM " agentlightning.LLM (agentlightning.types.resources.LLM)") ` LLM resource that rewrites endpoints through [`LLMProxy`](https://microsoft.github.io/agent-lightning/stable/reference/algorithm/#agentlightning.LLMProxy " agentlightning.LLMProxy") . The proxy injects rollout- and attempt-specific routing information into the endpoint so that downstream services can attribute requests correctly. #### `__getattribute__(name)` [¶](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.ProxyLLM.__getattribute__ "Permanent link") Emit a warning when `endpoint` is accessed directly after initialization. #### `get_base_url(rollout_id, attempt_id)` [¶](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.ProxyLLM.get_base_url "Permanent link") Return the routed endpoint for a specific rollout/attempt pair. Parameters: * **`rollout_id`** (`Optional[str]`) – Identifier of the rollout making the request. * **`attempt_id`** (`Optional[str]`) – Identifier of the attempt within that rollout. Returns: * `str` – Fully qualified endpoint including rollout metadata. Raises: * `ValueError` – If exactly one of `rollout_id` or `attempt_id` is provided. #### `model_post_init(__context)` [¶](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.ProxyLLM.model_post_init "Permanent link") Mark initialization as complete after Pydantic finishes setup. #### `with_attempted_rollout(rollout)` [¶](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.ProxyLLM.with_attempted_rollout "Permanent link") Bake rollout metadata into a concrete [`LLM`](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.LLM " agentlightning.LLM") instance. ### `agentlightning.PromptTemplate` [¶](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.PromptTemplate "Permanent link") Bases: `[Resource](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.Resource " agentlightning.Resource (agentlightning.types.resources.Resource)") ` Resource describing a reusable prompt template. #### `engine` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.PromptTemplate.engine "Permanent link") The templating engine to use for rendering the prompt. #### `template` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.PromptTemplate.template "Permanent link") The template string. The format depends on the engine. #### `format(**kwargs)` [¶](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.PromptTemplate.format "Permanent link") Format the prompt using keyword arguments. Warning Only the `f-string` engine is supported for now. ### `agentlightning.ResourceUnion = Annotated[Union[LLM, ProxyLLM, PromptTemplate], Field(discriminator='resource_type')]` `module-attribute` [¶](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.ResourceUnion "Permanent link") ### `agentlightning.NamedResources = Dict[str, ResourceUnion]` `module-attribute` [¶](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.NamedResources "Permanent link") Mapping from resource names to their configured instances. Examples: `[](https://microsoft.github.io/agent-lightning/stable/reference/types/#__codelineno-0-1) resources: NamedResources = { [](https://microsoft.github.io/agent-lightning/stable/reference/types/#__codelineno-0-2) "main_llm": LLM( [](https://microsoft.github.io/agent-lightning/stable/reference/types/#__codelineno-0-3) endpoint="http://localhost:8080", [](https://microsoft.github.io/agent-lightning/stable/reference/types/#__codelineno-0-4) model="llama3", [](https://microsoft.github.io/agent-lightning/stable/reference/types/#__codelineno-0-5) sampling_parameters={"temperature": 0.7, "max_tokens": 100}, [](https://microsoft.github.io/agent-lightning/stable/reference/types/#__codelineno-0-6) ), [](https://microsoft.github.io/agent-lightning/stable/reference/types/#__codelineno-0-7) "system_prompt": PromptTemplate( [](https://microsoft.github.io/agent-lightning/stable/reference/types/#__codelineno-0-8) template="You are a helpful assistant.", [](https://microsoft.github.io/agent-lightning/stable/reference/types/#__codelineno-0-9) engine="f-string", [](https://microsoft.github.io/agent-lightning/stable/reference/types/#__codelineno-0-10) ), [](https://microsoft.github.io/agent-lightning/stable/reference/types/#__codelineno-0-11) }` ### `agentlightning.ResourcesUpdate` [¶](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.ResourcesUpdate "Permanent link") Bases: `BaseModel` Update payload broadcast to clients when resources change. #### `create_time` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.ResourcesUpdate.create_time "Permanent link") Timestamp of the creation time of the resources. #### `resources` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.ResourcesUpdate.resources "Permanent link") Mapping of resource names to their definitions. #### `resources_id` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.ResourcesUpdate.resources_id "Permanent link") Identifier used to version the resources. #### `update_time` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.ResourcesUpdate.update_time "Permanent link") Timestamp of the last update time of the resources. #### `version` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.ResourcesUpdate.version "Permanent link") Version of the resources. Traces[¶](https://microsoft.github.io/agent-lightning/stable/reference/types/#traces "Permanent link") ------------------------------------------------------------------------------------------------------- ### `agentlightning.AttributeValue = Union[str, bool, int, float, Sequence[str], Sequence[bool], Sequence[int], Sequence[float]]` `module-attribute` [¶](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.AttributeValue "Permanent link") Possible values for OpenTelemetry attributes. ### `agentlightning.Attributes = Dict[str, AttributeValue]` `module-attribute` [¶](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.Attributes "Permanent link") Mapping from attribute names to their values. Same as OpenTelemetry `Attributes` type. ### `agentlightning.TraceState = Dict[str, str]` `module-attribute` [¶](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.TraceState "Permanent link") Mapping from trace state key to its value. Same as OpenTelemetry `TraceState` type. ### `agentlightning.SpanContext` [¶](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.SpanContext "Permanent link") Bases: `BaseModel` Pydantic representation of `opentelemetry.trace.SpanContext` values. #### `is_remote` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.SpanContext.is_remote "Permanent link") Whether the span is remote. #### `span_id` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.SpanContext.span_id "Permanent link") The span ID of the span. #### `trace_id` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.SpanContext.trace_id "Permanent link") The trace ID of the span. #### `trace_state` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.SpanContext.trace_state "Permanent link") Mapping from trace state key to its value. #### `from_opentelemetry(src)` `classmethod` [¶](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.SpanContext.from_opentelemetry "Permanent link") Construct a [`SpanContext`](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.SpanContext " agentlightning.SpanContext") from OpenTelemetry data. ### `agentlightning.TraceStatus` [¶](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.TraceStatus "Permanent link") Bases: `BaseModel` Serializable variant of `opentelemetry.trace.Status`. #### `description = None` `class-attribute` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.TraceStatus.description "Permanent link") The description of the span. Same as OpenTelemetry `Status.description` type. #### `status_code` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.TraceStatus.status_code "Permanent link") The status code of the span. Same as OpenTelemetry `Status.status_code` type. #### `from_opentelemetry(src)` `classmethod` [¶](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.TraceStatus.from_opentelemetry "Permanent link") Create a [`TraceStatus`](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.TraceStatus " agentlightning.TraceStatus") from OpenTelemetry metadata. ### `agentlightning.Event` [¶](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.Event "Permanent link") Bases: `BaseModel` Serializable representation of OpenTelemetry `Event` values. #### `attributes` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.Event.attributes "Permanent link") Mapping from attribute names to their values. Same as OpenTelemetry `Attributes` type. #### `name` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.Event.name "Permanent link") The name of the event. #### `timestamp = None` `class-attribute` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.Event.timestamp "Permanent link") The timestamp of the event. Same as OpenTelemetry `Event.timestamp` type. #### `from_opentelemetry(src)` `classmethod` [¶](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.Event.from_opentelemetry "Permanent link") Create an [`Event`](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.Event " agentlightning.Event") from an OpenTelemetry event. ### `agentlightning.Link` [¶](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.Link "Permanent link") Bases: `BaseModel` Serializable representation of OpenTelemetry `Link` values. #### `attributes = None` `class-attribute` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.Link.attributes "Permanent link") Optional attributes. #### `context` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.Link.context "Permanent link") The context of the link. #### `from_opentelemetry(src)` `classmethod` [¶](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.Link.from_opentelemetry "Permanent link") Create a [`Link`](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.Link " agentlightning.Link") from an OpenTelemetry link. ### `agentlightning.Resource` [¶](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.Resource "Permanent link") Bases: `BaseModel` Base class for tunable resources distributed to executors. #### `resource_type` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.Resource.resource_type "Permanent link") Alias of the resource type. ### `agentlightning.Span` [¶](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.Span "Permanent link") Bases: `BaseModel` Agent Lightning's canonical span model used for persistence and analytics. The model captures the most relevant fields from `opentelemetry.sdk.trace.ReadableSpan` instances while preserving unmodeled attributes in Pydantic `BaseModel`'s extra storage. This keeps the serialized format stable even as upstream OpenTelemetry types evolve. #### `attempt_id` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.Span.attempt_id "Permanent link") The attempt which this span belongs to. #### `attributes` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.Span.attributes "Permanent link") The attributes of the span. See [OpenTelemetry docs](https://opentelemetry.io/docs/concepts/signals/traces/) . #### `context` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.Span.context "Permanent link") The context of the span. See [OpenTelemetry docs](https://opentelemetry.io/docs/concepts/signals/traces/) . #### `end_time` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.Span.end_time "Permanent link") The end time of the span. See [OpenTelemetry docs](https://opentelemetry.io/docs/concepts/signals/traces/) . #### `events` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.Span.events "Permanent link") The events of the span. See [OpenTelemetry docs](https://opentelemetry.io/docs/concepts/signals/traces/) . #### `links` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.Span.links "Permanent link") The links of the span. See [OpenTelemetry docs](https://opentelemetry.io/docs/concepts/signals/traces/) . #### `name` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.Span.name "Permanent link") The name of the span. See [OpenTelemetry docs](https://opentelemetry.io/docs/concepts/signals/traces/) . #### `parent` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.Span.parent "Permanent link") The parent context of the span. See [OpenTelemetry docs](https://opentelemetry.io/docs/concepts/signals/traces/) . #### `parent_id` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.Span.parent_id "Permanent link") The parent span ID of the span. #### `resource` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.Span.resource "Permanent link") The resource of the span. See [OpenTelemetry docs](https://opentelemetry.io/docs/concepts/signals/traces/) . #### `rollout_id` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.Span.rollout_id "Permanent link") The rollout which this span belongs to. #### `sequence_id` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.Span.sequence_id "Permanent link") The ID to make spans ordered within a single attempt. #### `span_id` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.Span.span_id "Permanent link") The span ID of the span. This ID comes from the OpenTelemetry span ID generator. #### `start_time` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.Span.start_time "Permanent link") The start time of the span. See [OpenTelemetry docs](https://opentelemetry.io/docs/concepts/signals/traces/) . #### `status` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.Span.status "Permanent link") The status of the span. See [OpenTelemetry docs](https://opentelemetry.io/docs/concepts/signals/traces/) . #### `trace_id` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.Span.trace_id "Permanent link") The trace ID of the span. One rollout/attempt can have multiple traces. This ID comes from the OpenTelemetry trace ID generator. #### `from_attributes(*, attributes, rollout_id=None, attempt_id=None, sequence_id=None, name=None, trace_id=None, span_id=None, parent_id=None, start_time=None, end_time=None, resource=None, status=None)` `classmethod` [¶](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.Span.from_attributes "Permanent link") Build a synthetic span from raw attributes. Different from the [`from_opentelemetry`](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.Span.from_opentelemetry " from_opentelemetry(src, rollout_id, attempt_id, sequence_id) classmethod ") method, all parameters other than `attributes` are optional and will be generated if not provided. Parameters: * **`attributes`** (`[Attributes](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.Attributes " agentlightning.Attributes = Dict[str, AttributeValue] module-attribute (agentlightning.types.tracer.Attributes)") `) – Span attributes to persist. * **`rollout_id`** (`Optional[str]`, default: `None` ) – Optional rollout identifier associated with the span. * **`attempt_id`** (`Optional[str]`, default: `None` ) – Optional attempt identifier associated with the span. * **`sequence_id`** (`Optional[int]`, default: `None` ) – Optional sequence number to preserve ordering. * **`name`** (`Optional[str]`, default: `None` ) – Optional human-readable span name. * **`trace_id`** (`Optional[str]`, default: `None` ) – Custom trace identifier. When omitted, a random identifier is generated. * **`span_id`** (`Optional[str]`, default: `None` ) – Custom span identifier. When omitted, a random identifier is generated. * **`parent_id`** (`Optional[str]`, default: `None` ) – Optional parent span identifier. * **`start_time`** (`Optional[float]`, default: `None` ) – Span start timestamp in seconds. * **`end_time`** (`Optional[float]`, default: `None` ) – Span end timestamp in seconds. * **`resource`** (`Optional[OtelResource]`, default: `None` ) – Explicit resource information to attach to the span. * **`status`** (`Optional[[TraceStatus](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.TraceStatus " agentlightning.TraceStatus (agentlightning.types.tracer.TraceStatus)") ]`, default: `None` ) – Optional status of the span. Returns: * `'Span'` – [`Span`](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.Span " agentlightning.Span") populated with the provided attributes. #### `from_core_fields(core, *, rollout_id=None, attempt_id=None, sequence_id=None)` `classmethod` [¶](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.Span.from_core_fields "Permanent link") Build a span from a core span. Parameters: * **`core`** (`[SpanCoreFields](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.SpanCoreFields " agentlightning.SpanCoreFields (agentlightning.types.tracer.SpanCoreFields)") `) – Core span to build from. * **`rollout_id`** (`Optional[str]`, default: `None` ) – Optional rollout identifier associated with the span. * **`attempt_id`** (`Optional[str]`, default: `None` ) – Optional attempt identifier associated with the span. * **`sequence_id`** (`Optional[int]`, default: `None` ) – Optional sequence number to preserve ordering. Returns: * `[Span](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.Span " agentlightning.Span (agentlightning.types.tracer.Span)") ` – [`Span`](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.Span " agentlightning.Span") populated with the provided attributes. #### `from_opentelemetry(src, rollout_id, attempt_id, sequence_id)` `classmethod` [¶](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.Span.from_opentelemetry "Permanent link") Convert an OpenTelemetry span into the Agent Lightning data model. Parameters: * **`src`** (`ReadableSpan`) – Span captured by OpenTelemetry. * **`rollout_id`** (`str`) – Identifier for the rollout that produced the span. * **`attempt_id`** (`str`) – Identifier of the attempt within the rollout. * **`sequence_id`** (`int`) – Monotonically increasing identifier assigned to the span. Returns: * `'Span'` – Parsed [`Span`](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.Span " agentlightning.Span") instance suitable for persistence. ### `agentlightning.SpanAttributeNames` [¶](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.SpanAttributeNames "Permanent link") Bases: `str`, `Enum` Canonical attribute names written by Agent Lightning emitters. Deprecated in favor of [semconv](https://microsoft.github.io/agent-lightning/stable/reference/semconv/#agentlightning.semconv " agentlightning.semconv") . #### `MESSAGE = 'message'` `class-attribute` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.SpanAttributeNames.MESSAGE "Permanent link") The name of the message attribute. #### `OBJECT = 'object'` `class-attribute` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.SpanAttributeNames.OBJECT "Permanent link") The name of the object attribute. ### `agentlightning.SpanLike = Union[ReadableSpan, Span]` `module-attribute` [¶](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.SpanLike "Permanent link") Union type of OpenTelemetry `ReadableSpan` and Agent-lightning [`Span`](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.Span " agentlightning.Span") . ### `agentlightning.SpanCoreFields` [¶](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.SpanCoreFields "Permanent link") Bases: `BaseModel` Core fields of a span. Used by span creators who don't care about the full span model. If the spans are managed by some OTel tracer provider, it's not advised to create spans via this path. #### `attributes` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.SpanCoreFields.attributes "Permanent link") The attributes of the span. #### `end_time` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.SpanCoreFields.end_time "Permanent link") The end time of the span. #### `name` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.SpanCoreFields.name "Permanent link") The name of the span. #### `start_time` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.SpanCoreFields.start_time "Permanent link") The start time of the span. #### `status` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.SpanCoreFields.status "Permanent link") The status of the span. ### `agentlightning.SpanRecordingContext` [¶](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.SpanRecordingContext "Permanent link") Bases: `Protocol` Context for recording operations on a span. It doesn't have to finalize the span; the caller will do it. #### `get_recorded_span()` [¶](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.SpanRecordingContext.get_recorded_span "Permanent link") Get the recording of the span. #### `record_attributes(attributes)` [¶](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.SpanRecordingContext.record_attributes "Permanent link") Record attributes on the span. #### `record_exception(exception)` [¶](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.SpanRecordingContext.record_exception "Permanent link") Record an exception on the span. #### `record_status(status_code, description=None)` [¶](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.SpanRecordingContext.record_status "Permanent link") Record the status of the span. Environment Variables[¶](https://microsoft.github.io/agent-lightning/stable/reference/types/#environment-variables "Permanent link") ------------------------------------------------------------------------------------------------------------------------------------- ### `agentlightning.LightningEnvVar` [¶](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.LightningEnvVar "Permanent link") Bases: `Enum` Environment variables for Agent Lightning. #### `AGL_CURRENT_ROLE = 'AGL_CURRENT_ROLE'` `class-attribute` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.LightningEnvVar.AGL_CURRENT_ROLE "Permanent link") Which side(s) to run in this process. Used in [`ClientServerExecutionStrategy`](https://microsoft.github.io/agent-lightning/stable/reference/trainer/#agentlightning.ClientServerExecutionStrategy " agentlightning.ClientServerExecutionStrategy") . #### `AGL_EMITTER_DEBUG = 'AGL_EMITTER_DEBUG'` `class-attribute` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.LightningEnvVar.AGL_EMITTER_DEBUG "Permanent link") Enable debug logging for the emitter. #### `AGL_MANAGED_STORE = 'AGL_MANAGED_STORE'` `class-attribute` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.LightningEnvVar.AGL_MANAGED_STORE "Permanent link") If yes, the [`ExecutionStrategy`](https://microsoft.github.io/agent-lightning/stable/reference/trainer/#agentlightning.ExecutionStrategy " agentlightning.ExecutionStrategy") constructs LightningStore wrappers automatically. When `False` the provided `store` is passed directly to the bundles, allowing callers to manage store wrappers manually. #### `AGL_SERVER_HOST = 'AGL_SERVER_HOST'` `class-attribute` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.LightningEnvVar.AGL_SERVER_HOST "Permanent link") Interface the [`LightningStoreServer`](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.LightningStoreServer " agentlightning.LightningStoreServer") binds to when running the algorithm bundle locally. #### `AGL_SERVER_PORT = 'AGL_SERVER_PORT'` `class-attribute` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.LightningEnvVar.AGL_SERVER_PORT "Permanent link") Port the [`LightningStoreServer`](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.LightningStoreServer " agentlightning.LightningStoreServer") listens to. ### `agentlightning.resolve_bool_env_var(env_var, override=None, fallback=None)` [¶](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.resolve_bool_env_var "Permanent link") `[](https://microsoft.github.io/agent-lightning/stable/reference/types/#__codelineno-0-1) resolve_bool_env_var( [](https://microsoft.github.io/agent-lightning/stable/reference/types/#__codelineno-0-2) env_var: LightningEnvVar, override: bool, fallback: bool [](https://microsoft.github.io/agent-lightning/stable/reference/types/#__codelineno-0-3) ) -> bool` `[](https://microsoft.github.io/agent-lightning/stable/reference/types/#__codelineno-0-1) resolve_bool_env_var( [](https://microsoft.github.io/agent-lightning/stable/reference/types/#__codelineno-0-2) env_var: LightningEnvVar, *, fallback: bool [](https://microsoft.github.io/agent-lightning/stable/reference/types/#__codelineno-0-3) ) -> bool` `[](https://microsoft.github.io/agent-lightning/stable/reference/types/#__codelineno-0-1) resolve_bool_env_var( [](https://microsoft.github.io/agent-lightning/stable/reference/types/#__codelineno-0-2) env_var: LightningEnvVar, [](https://microsoft.github.io/agent-lightning/stable/reference/types/#__codelineno-0-3) override: bool | None = None, [](https://microsoft.github.io/agent-lightning/stable/reference/types/#__codelineno-0-4) fallback: bool | None = None, [](https://microsoft.github.io/agent-lightning/stable/reference/types/#__codelineno-0-5) ) -> bool | None` Resolve a boolean environment variable. Parameters: * **`env_var`** (`[LightningEnvVar](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.LightningEnvVar " agentlightning.LightningEnvVar (agentlightning.env_var.LightningEnvVar)") `) – The environment variable to resolve. * **`override`** (`bool | None`, default: `None` ) – Optional override supplied by the caller. * **`fallback`** (`bool | None`, default: `None` ) – Default value if the environment variable is not set. ### `agentlightning.resolve_int_env_var(env_var, override=None, fallback=None)` [¶](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.resolve_int_env_var "Permanent link") `[](https://microsoft.github.io/agent-lightning/stable/reference/types/#__codelineno-0-1) resolve_int_env_var( [](https://microsoft.github.io/agent-lightning/stable/reference/types/#__codelineno-0-2) env_var: LightningEnvVar, override: int, fallback: int [](https://microsoft.github.io/agent-lightning/stable/reference/types/#__codelineno-0-3) ) -> int` `[](https://microsoft.github.io/agent-lightning/stable/reference/types/#__codelineno-0-1) resolve_int_env_var( [](https://microsoft.github.io/agent-lightning/stable/reference/types/#__codelineno-0-2) env_var: LightningEnvVar, *, fallback: int [](https://microsoft.github.io/agent-lightning/stable/reference/types/#__codelineno-0-3) ) -> int` `[](https://microsoft.github.io/agent-lightning/stable/reference/types/#__codelineno-0-1) resolve_int_env_var( [](https://microsoft.github.io/agent-lightning/stable/reference/types/#__codelineno-0-2) env_var: LightningEnvVar, [](https://microsoft.github.io/agent-lightning/stable/reference/types/#__codelineno-0-3) override: int | None = None, [](https://microsoft.github.io/agent-lightning/stable/reference/types/#__codelineno-0-4) fallback: int | None = None, [](https://microsoft.github.io/agent-lightning/stable/reference/types/#__codelineno-0-5) ) -> int | None` Resolve an integer environment variable. Parameters: * **`env_var`** (`[LightningEnvVar](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.LightningEnvVar " agentlightning.LightningEnvVar (agentlightning.env_var.LightningEnvVar)") `) – The environment variable to resolve. * **`override`** (`int | None`, default: `None` ) – Optional override supplied by the caller. * **`fallback`** (`int | None`, default: `None` ) – Default value if the environment variable is not set. ### `agentlightning.resolve_str_env_var(env_var, override=None, fallback=None)` [¶](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.resolve_str_env_var "Permanent link") `[](https://microsoft.github.io/agent-lightning/stable/reference/types/#__codelineno-0-1) resolve_str_env_var( [](https://microsoft.github.io/agent-lightning/stable/reference/types/#__codelineno-0-2) env_var: LightningEnvVar, override: str, fallback: str [](https://microsoft.github.io/agent-lightning/stable/reference/types/#__codelineno-0-3) ) -> str` `[](https://microsoft.github.io/agent-lightning/stable/reference/types/#__codelineno-0-1) resolve_str_env_var( [](https://microsoft.github.io/agent-lightning/stable/reference/types/#__codelineno-0-2) env_var: LightningEnvVar, *, fallback: str [](https://microsoft.github.io/agent-lightning/stable/reference/types/#__codelineno-0-3) ) -> str` `[](https://microsoft.github.io/agent-lightning/stable/reference/types/#__codelineno-0-1) resolve_str_env_var( [](https://microsoft.github.io/agent-lightning/stable/reference/types/#__codelineno-0-2) env_var: LightningEnvVar, [](https://microsoft.github.io/agent-lightning/stable/reference/types/#__codelineno-0-3) override: str | None = None, [](https://microsoft.github.io/agent-lightning/stable/reference/types/#__codelineno-0-4) fallback: str | None = None, [](https://microsoft.github.io/agent-lightning/stable/reference/types/#__codelineno-0-5) ) -> str | None` Resolve a string environment variable. Parameters: * **`env_var`** (`[LightningEnvVar](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.LightningEnvVar " agentlightning.LightningEnvVar (agentlightning.env_var.LightningEnvVar)") `) – The environment variable to resolve. * **`override`** (`str | None`, default: `None` ) – Optional override supplied by the caller. * **`fallback`** (`str | None`, default: `None` ) – Default value if the environment variable is not set. Back to top --- # Internal - Agent-lightning [Skip to content](https://microsoft.github.io/agent-lightning/stable/reference/internal/#internal-api-references) ⚠️ You are viewing development documentation. For stable documentation, please visit [https://microsoft.github.io/agent-lightning/stable/](https://microsoft.github.io/agent-lightning/stable/) Internal API References[¶](https://microsoft.github.io/agent-lightning/stable/reference/internal/#internal-api-references "Permanent link") ============================================================================================================================================ Danger The following APIs should be used with extra caution because they are very likely to change in the future. Algorithms and Adapters[¶](https://microsoft.github.io/agent-lightning/stable/reference/internal/#algorithms-and-adapters "Permanent link") -------------------------------------------------------------------------------------------------------------------------------------------- ### `agentlightning.adapter.messages.OpenAIMessages` [¶](https://microsoft.github.io/agent-lightning/stable/reference/internal/#agentlightning.adapter.messages.OpenAIMessages "Permanent link") Bases: `TypedDict` OpenAI-style chat messages with optional tool definitions. Attributes: * **`messages`** (`List[ChatCompletionMessageParam]`) – Ordered chat messages that describe the conversation. * **`tools`** (`Optional[List[ChatCompletionFunctionToolParam]]`) – Tool specifications available to the assistant, if any. ### `agentlightning.adapter.triplet.TraceTree` [¶](https://microsoft.github.io/agent-lightning/stable/reference/internal/#agentlightning.adapter.triplet.TraceTree "Permanent link") Tree representation of a trace span and its descendants. Attributes: * **`id`** – Unique identifier for the span node. * **`span`** – [`Span`](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.Span " agentlightning.Span") backing this node. * **`children`** – Child nodes connected to the current span. #### `agent_name()` [¶](https://microsoft.github.io/agent-lightning/stable/reference/internal/#agentlightning.adapter.triplet.TraceTree.agent_name "Permanent link") Return the agent name associated with the span, if any. Returns: * `Optional[str]` – Agent name extracted from known attributes, otherwise `None`. #### `extract_prompt_image_urls(prompt_raw_content)` [¶](https://microsoft.github.io/agent-lightning/stable/reference/internal/#agentlightning.adapter.triplet.TraceTree.extract_prompt_image_urls "Permanent link") Extract image URLs from the span attributes, in order of appearance. Parameters: * **`prompt_raw_content`** (`Any`) – The raw content of the prompt, which can be in one of several formats: * List\[dict\]: A list of message entries, each being a dict with at least a "content" key. * Dict\[str, Any\]: A dictionary, often with numeric string keys (e.g., `{"0": {...}, "1": {...}}`), where each value is a message entry. If the dict does not have numeric keys, it is treated as a single message entry. #### `find_llm_calls(*, llm_call_match, agent_match, within_matching_subtree=None, within_reward=None, within_llm_call=None, existing_llm_call_response_ids=None)` [¶](https://microsoft.github.io/agent-lightning/stable/reference/internal/#agentlightning.adapter.triplet.TraceTree.find_llm_calls "Permanent link") Find LLM call spans matching the supplied filters. Parameters: * **`llm_call_match`** (`str`) – Regular expression used to match span names that qualify as LLM calls. * **`agent_match`** (`Optional[str]`) – Optional regular expression that must match the enclosing agent span name. * **`within_matching_subtree`** (`str | None`, default: `None` ) – Marker propagated through recursive calls to record matching agents. * **`within_reward`** (`Optional[bool]`, default: `None` ) – When `True`, suppresses LLM matches under reward spans. * **`within_llm_call`** (`Optional[bool]`, default: `None` ) – When `True`, prevents duplicate matches for nested LLM calls. * **`existing_llm_call_response_ids`** (`Optional[set[str]]`, default: `None` ) – Known response identifiers used to deduplicate spans. Returns: * `List[Tuple['TraceTree', str]]` – A list of tuples pairing the matching node with the agent subtree label that triggered the * `List[Tuple['TraceTree', str]]` – match. #### `from_spans(spans)` `classmethod` [¶](https://microsoft.github.io/agent-lightning/stable/reference/internal/#agentlightning.adapter.triplet.TraceTree.from_spans "Permanent link") Construct a tree from a flat list of spans. Parameters: * **`spans`** (`List[[Span](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.Span " agentlightning.Span (agentlightning.types.Span)") ]`) – Spans that collectively form a single trace segment. Returns: * `'TraceTree'` – A [`TraceTree`](https://microsoft.github.io/agent-lightning/stable/reference/internal/#agentlightning.adapter.triplet.TraceTree " agentlightning.adapter.triplet.TraceTree") rooted at either the * `'TraceTree'` – discovered root span or a synthetic root when multiple roots are present. Raises: * `ValueError` – If the span list is empty or no root span can be inferred. #### `is_reward_span()` [¶](https://microsoft.github.io/agent-lightning/stable/reference/internal/#agentlightning.adapter.triplet.TraceTree.is_reward_span "Permanent link") Return whether the span explicitly encodes a reward. Returns: * `bool` – `True` when the span payload describes a reward, otherwise `False`. #### `match_rewards(reward_match, llm_calls)` [¶](https://microsoft.github.io/agent-lightning/stable/reference/internal/#agentlightning.adapter.triplet.TraceTree.match_rewards "Permanent link") Assign rewards to previously matched LLM calls. Parameters: * **`reward_match`** (`str`) – Strategy identifier from [`RewardMatchPolicy`](https://microsoft.github.io/agent-lightning/stable/reference/internal/#agentlightning.adapter.triplet.RewardMatchPolicy " agentlightning.adapter.triplet.RewardMatchPolicy") . * **`llm_calls`** (`List['TraceTree']`) – Trace nodes representing LLM call spans. Returns: * `dict[str, Optional[float]]` – Mapping from span identifier to reward value or `None` when no reward is available. #### `maybe_reward_dict()` [¶](https://microsoft.github.io/agent-lightning/stable/reference/internal/#agentlightning.adapter.triplet.TraceTree.maybe_reward_dict "Permanent link") Return a reward payload if the span encodes one. Returns: * `dict[str, Any]` – Dictionary containing reward metadata, or an empty dictionary when no reward is found. #### `names_tuple()` [¶](https://microsoft.github.io/agent-lightning/stable/reference/internal/#agentlightning.adapter.triplet.TraceTree.names_tuple "Permanent link") Return the span name alongside nested child names. Returns: * `str` – A tuple of the current span name and a list of tuples for each child containing the * `List[Any]` – child name and its descendants. #### `repair_hierarchy()` [¶](https://microsoft.github.io/agent-lightning/stable/reference/internal/#agentlightning.adapter.triplet.TraceTree.repair_hierarchy "Permanent link") Repair missing parent-child relationships introduced by mixed tracing systems. Some agent frameworks emit spans via multiple subsystems, which can cause LLM completion spans to float directly under the root span instead of being nested under the correct agent. The method re-parents those spans to the closest ancestor that fully envelopes the child in time. If we don't, when we want to select the LLM completion span with agent as filter. We will never get the correct span underneath. #### `span_to_triplet(span, agent_name)` [¶](https://microsoft.github.io/agent-lightning/stable/reference/internal/#agentlightning.adapter.triplet.TraceTree.span_to_triplet "Permanent link") Convert a span to a triplet. Subclass can override this method to add more fields to the triplet, such as chat messages and tool calls. #### `to_json()` [¶](https://microsoft.github.io/agent-lightning/stable/reference/internal/#agentlightning.adapter.triplet.TraceTree.to_json "Permanent link") Convert the tree node into a JSON-serialisable structure. #### `to_trajectory(llm_call_match='openai\\.chat\\.completion', agent_match=None, exclude_llm_call_in_reward=True, dedup_llm_call=True, reward_match=RewardMatchPolicy.FIRST_OCCURRENCE, final_reward=None, _skip_empty_token_spans=False)` [¶](https://microsoft.github.io/agent-lightning/stable/reference/internal/#agentlightning.adapter.triplet.TraceTree.to_trajectory "Permanent link") Convert the trace tree into a trajectory of [`Triplet`](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.Triplet " agentlightning.Triplet") items. Parameters: * **`llm_call_match`** (`str`, default: `'openai\\.chat\\.completion'` ) – Regular expression for LLM call span names. * **`agent_match`** (`Optional[str]`, default: `None` ) – Optional regular expression for agent span names. * **`exclude_llm_call_in_reward`** (`bool`, default: `True` ) – When `True`, prevents searching for rewards under the LLM call subtree. * **`dedup_llm_call`** (`bool`, default: `True` ) – When `True`, deduplicates spans using the LLM response identifier. * **`reward_match`** (`[RewardMatchPolicy](https://microsoft.github.io/agent-lightning/stable/reference/internal/#agentlightning.adapter.triplet.RewardMatchPolicy " agentlightning.adapter.triplet.RewardMatchPolicy") `, default: `[FIRST_OCCURRENCE](https://microsoft.github.io/agent-lightning/stable/reference/internal/#agentlightning.adapter.triplet.RewardMatchPolicy.FIRST_OCCURRENCE " FIRST_OCCURRENCE = 'first_occurrence' class-attribute instance-attribute (agentlightning.adapter.triplet.RewardMatchPolicy.FIRST_OCCURRENCE)") ` ) – Reward matching policy used to associate reward spans with LLM calls. * **`final_reward`** (`Optional[float]`, default: `None` ) – Optional reward appended to the final transition when provided. Returns: * `List[[Triplet](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.Triplet " agentlightning.Triplet (agentlightning.types.Triplet)") ]` – A list of [`Triplet`](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.Triplet " agentlightning.Triplet") objects ordered by call sequence. #### `traverse()` [¶](https://microsoft.github.io/agent-lightning/stable/reference/internal/#agentlightning.adapter.triplet.TraceTree.traverse "Permanent link") Traverse the tree depth first and return every node. #### `visualize(filename, interested_span_match=None)` [¶](https://microsoft.github.io/agent-lightning/stable/reference/internal/#agentlightning.adapter.triplet.TraceTree.visualize "Permanent link") Render the trace tree with Graphviz for debugging purposes. Parameters: * **`filename`** (`str`) – Base filename for the generated `.png` diagram. * **`interested_span_match`** (`str | None`, default: `None` ) – Optional regular expression used to keep only matching spans (and their ancestors) in the output. Note The method requires the optional `graphviz` dependency to be available in the runtime environment. ### `agentlightning.adapter.triplet.Transition` [¶](https://microsoft.github.io/agent-lightning/stable/reference/internal/#agentlightning.adapter.triplet.Transition "Permanent link") Bases: `BaseModel` A single transition within a reinforcement learning trajectory. Attributes: * **`state`** (`List[int]`) – Token identifiers describing the model input state. * **`action`** (`List[int]`) – Token identifiers representing the model output. * **`response_id`** (`Optional[str]`) – Identifier of the LLM response used to deduplicate spans. * **`agent_name`** (`str`) – Human-readable agent name captured from the trace. * **`reward`** (`Optional[float]`) – Scalar reward associated with the transition, if available. ### `agentlightning.adapter.triplet.RewardMatchPolicy` [¶](https://microsoft.github.io/agent-lightning/stable/reference/internal/#agentlightning.adapter.triplet.RewardMatchPolicy "Permanent link") Bases: `str`, `Enum` Strategies for matching rewards to LLM call spans. Note Each reward span must expose a payload shaped like `{"type": "reward", "value": |None}` as described in `reward.py`. #### `FIRST_OCCURRENCE = 'first_occurrence'` `class-attribute` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/stable/reference/internal/#agentlightning.adapter.triplet.RewardMatchPolicy.FIRST_OCCURRENCE "Permanent link") Use the first reward encountered in chronological order after the current LLM call match. #### `FIRST_SIBLING = 'first_sibling'` `class-attribute` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/stable/reference/internal/#agentlightning.adapter.triplet.RewardMatchPolicy.FIRST_SIBLING "Permanent link") Use the first sibling in the current trace subtree as the reward unless another LLM call match is found. ### `agentlightning.algorithm.decorator.FunctionalAlgorithm` [¶](https://microsoft.github.io/agent-lightning/stable/reference/internal/#agentlightning.algorithm.decorator.FunctionalAlgorithm "Permanent link") Bases: `[Algorithm](https://microsoft.github.io/agent-lightning/stable/reference/algorithm/#agentlightning.Algorithm " agentlightning.Algorithm (agentlightning.algorithm.base.Algorithm)") `, `Generic[AF]` An algorithm wrapper built from a callable implementation. Functional algorithms let you provide an ordinary function instead of subclassing [`Algorithm`](https://microsoft.github.io/agent-lightning/stable/reference/algorithm/#agentlightning.Algorithm " agentlightning.Algorithm") . The wrapper inspects the callable signature to supply optional dependencies such as the store, adapter, and LLM proxy. #### `__init__(algorithm_func)` [¶](https://microsoft.github.io/agent-lightning/stable/reference/internal/#agentlightning.algorithm.decorator.FunctionalAlgorithm.__init__ "Permanent link") `[](https://microsoft.github.io/agent-lightning/stable/reference/internal/#__codelineno-0-1) __init__(algorithm_func: AlgorithmFuncSyncLike) -> None` `[](https://microsoft.github.io/agent-lightning/stable/reference/internal/#__codelineno-0-1) __init__(algorithm_func: AlgorithmFuncAsyncLike) -> None` Wrap a function that implements algorithm behaviour. Parameters: * **`algorithm_func`** (`Union[AlgorithmFuncSyncLike, AlgorithmFuncAsyncLike]`) – Sync or async callable implementing the algorithm contract. Arguments are detected automatically based on the function signature. #### `run(train_dataset=None, val_dataset=None)` [¶](https://microsoft.github.io/agent-lightning/stable/reference/internal/#agentlightning.algorithm.decorator.FunctionalAlgorithm.run "Permanent link") `[](https://microsoft.github.io/agent-lightning/stable/reference/internal/#__codelineno-0-1) run( [](https://microsoft.github.io/agent-lightning/stable/reference/internal/#__codelineno-0-2) train_dataset: Optional[Dataset[Any]] = None, [](https://microsoft.github.io/agent-lightning/stable/reference/internal/#__codelineno-0-3) val_dataset: Optional[Dataset[Any]] = None, [](https://microsoft.github.io/agent-lightning/stable/reference/internal/#__codelineno-0-4) ) -> None` `[](https://microsoft.github.io/agent-lightning/stable/reference/internal/#__codelineno-0-1) run( [](https://microsoft.github.io/agent-lightning/stable/reference/internal/#__codelineno-0-2) train_dataset: Optional[Dataset[Any]] = None, [](https://microsoft.github.io/agent-lightning/stable/reference/internal/#__codelineno-0-3) val_dataset: Optional[Dataset[Any]] = None, [](https://microsoft.github.io/agent-lightning/stable/reference/internal/#__codelineno-0-4) ) -> Awaitable[None]` Execute the wrapped function with injected dependencies. Parameters: * **`train_dataset`** (`Optional[[Dataset](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.Dataset " agentlightning.Dataset (agentlightning.types.Dataset)") [Any]]`, default: `None` ) – Optional training dataset passed through when the callable declares a `train_dataset` parameter. * **`val_dataset`** (`Optional[[Dataset](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.Dataset " agentlightning.Dataset (agentlightning.types.Dataset)") [Any]]`, default: `None` ) – Optional validation dataset passed through when the callable declares a `val_dataset` parameter. Returns: * `Union[None, Awaitable[None]]` – None for sync callables or an awaitable when the callable is async. Raises: * `TypeError` – If a dataset is provided but the function signature does not accept the corresponding argument. LitAgent[¶](https://microsoft.github.io/agent-lightning/stable/reference/internal/#litagent "Permanent link") -------------------------------------------------------------------------------------------------------------- ### `agentlightning.litagent.decorator.FunctionalLitAgent` [¶](https://microsoft.github.io/agent-lightning/stable/reference/internal/#agentlightning.litagent.decorator.FunctionalLitAgent "Permanent link") Bases: `[LitAgent](https://microsoft.github.io/agent-lightning/stable/reference/agent/#agentlightning.LitAgent " agentlightning.LitAgent (agentlightning.litagent.litagent.LitAgent)") [T]` Adapter that turns plain rollout functions into [`LitAgent`](https://microsoft.github.io/agent-lightning/stable/reference/agent/#agentlightning.LitAgent " agentlightning.LitAgent") instances. The helper inspects the wrapped function to determine which resources to inject, allowing both synchronous and asynchronous callables to participate in the training loop without writing a dedicated subclass. #### `__call__(*args, **kwargs)` [¶](https://microsoft.github.io/agent-lightning/stable/reference/internal/#agentlightning.litagent.decorator.FunctionalLitAgent.__call__ "Permanent link") Make the agent instance callable, preserving the original function behavior. #### `__init__(rollout_func, *, strip_proxy=True)` [¶](https://microsoft.github.io/agent-lightning/stable/reference/internal/#agentlightning.litagent.decorator.FunctionalLitAgent.__init__ "Permanent link") Initialize the wrapper around a rollout function. Parameters: * **`rollout_func`** (`FunctionalLitAgentFunc[T]`) – Callable that implements the rollout. It may be synchronous or asynchronous and can optionally receive a [`Rollout`](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.Rollout " agentlightning.Rollout") alongside resources such as `llm` or `prompt_template`. * **`strip_proxy`** (`bool`, default: `True` ) – When `True`, convert [`ProxyLLM`](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.ProxyLLM " agentlightning.ProxyLLM") inputs into [`LLM`](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.LLM " agentlightning.LLM") instances before calling the rollout function. Defaults to `True`. #### `rollout(task, resources, rollout)` [¶](https://microsoft.github.io/agent-lightning/stable/reference/internal/#agentlightning.litagent.decorator.FunctionalLitAgent.rollout "Permanent link") Execute a synchronous rollout using the wrapped function. Parameters: * **`task`** (`T`) – Task input data. * **`resources`** (`[NamedResources](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.NamedResources " agentlightning.NamedResources = Dict[str, ResourceUnion] module-attribute (agentlightning.types.NamedResources)") `) – Mapping of named resources available to the agent. * **`rollout`** (`[Rollout](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.Rollout " agentlightning.Rollout (agentlightning.types.Rollout)") `) – Rollout metadata provided by the runtime. Returns: * `[RolloutRawResult](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.RolloutRawResult " agentlightning.RolloutRawResult = Union[None, float, List[ReadableSpan], List[Span], List[SpanCoreFields]] module-attribute (agentlightning.types.RolloutRawResult)") ` – Result produced by the wrapped rollout function. Raises: * `RuntimeError` – If the wrapped function is asynchronous. #### `rollout_async(task, resources, rollout)` `async` [¶](https://microsoft.github.io/agent-lightning/stable/reference/internal/#agentlightning.litagent.decorator.FunctionalLitAgent.rollout_async "Permanent link") Execute an asynchronous rollout using the wrapped function. Parameters: * **`task`** (`T`) – Task input data. * **`resources`** (`[NamedResources](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.NamedResources " agentlightning.NamedResources = Dict[str, ResourceUnion] module-attribute (agentlightning.types.NamedResources)") `) – Mapping of named resources available to the agent. * **`rollout`** (`[Rollout](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.Rollout " agentlightning.Rollout (agentlightning.types.Rollout)") `) – Rollout metadata provided by the runtime. Returns: * `[RolloutRawResult](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.RolloutRawResult " agentlightning.RolloutRawResult = Union[None, float, List[ReadableSpan], List[Span], List[SpanCoreFields]] module-attribute (agentlightning.types.RolloutRawResult)") ` – Result produced by the wrapped rollout coroutine. Raises: * `RuntimeError` – If the wrapped function is synchronous. ### `agentlightning.litagent.decorator.llm_rollout(func=None, *, strip_proxy=True)` [¶](https://microsoft.github.io/agent-lightning/stable/reference/internal/#agentlightning.litagent.decorator.llm_rollout "Permanent link") `[](https://microsoft.github.io/agent-lightning/stable/reference/internal/#__codelineno-0-1) llm_rollout( [](https://microsoft.github.io/agent-lightning/stable/reference/internal/#__codelineno-0-2) func: LlmRolloutFunc[T], [](https://microsoft.github.io/agent-lightning/stable/reference/internal/#__codelineno-0-3) ) -> FunctionalLitAgent[T]` `[](https://microsoft.github.io/agent-lightning/stable/reference/internal/#__codelineno-0-1) llm_rollout( [](https://microsoft.github.io/agent-lightning/stable/reference/internal/#__codelineno-0-2) *, strip_proxy: bool = True [](https://microsoft.github.io/agent-lightning/stable/reference/internal/#__codelineno-0-3) ) -> Callable[[LlmRolloutFunc[T]], FunctionalLitAgent[T]]` Create a [`FunctionalLitAgent`](https://microsoft.github.io/agent-lightning/stable/reference/internal/#agentlightning.litagent.decorator.FunctionalLitAgent " agentlightning.litagent.decorator.FunctionalLitAgent") for LLM-based rollouts. Parameters: * **`func`** (`LlmRolloutFunc[T] | None`, default: `None` ) – Callable defining the agent's behaviour. Supported signatures include: * `(task, llm) -> result` * `(task, llm, rollout) -> result` * `async (task, llm) -> result` * `async (task, llm, rollout) -> result` * **`strip_proxy`** (`bool`, default: `True` ) – When `True`, convert proxy resources into concrete [`LLM`](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.LLM " agentlightning.LLM") instances before calling the function. Defaults to `True`. Returns: * `[FunctionalLitAgent](https://microsoft.github.io/agent-lightning/stable/reference/internal/#agentlightning.litagent.decorator.FunctionalLitAgent " agentlightning.litagent.decorator.FunctionalLitAgent") [T] | Callable[[LlmRolloutFunc[T]], [FunctionalLitAgent](https://microsoft.github.io/agent-lightning/stable/reference/internal/#agentlightning.litagent.decorator.FunctionalLitAgent " agentlightning.litagent.decorator.FunctionalLitAgent") [T]]` – [`FunctionalLitAgent`](https://microsoft.github.io/agent-lightning/stable/reference/internal/#agentlightning.litagent.decorator.FunctionalLitAgent " agentlightning.litagent.decorator.FunctionalLitAgent") that * `[FunctionalLitAgent](https://microsoft.github.io/agent-lightning/stable/reference/internal/#agentlightning.litagent.decorator.FunctionalLitAgent " agentlightning.litagent.decorator.FunctionalLitAgent") [T] | Callable[[LlmRolloutFunc[T]], [FunctionalLitAgent](https://microsoft.github.io/agent-lightning/stable/reference/internal/#agentlightning.litagent.decorator.FunctionalLitAgent " agentlightning.litagent.decorator.FunctionalLitAgent") [T]]` – wraps the supplied function. Examples: `[](https://microsoft.github.io/agent-lightning/stable/reference/internal/#__codelineno-0-1) @llm_rollout [](https://microsoft.github.io/agent-lightning/stable/reference/internal/#__codelineno-0-2) def my_agent(task, llm): [](https://microsoft.github.io/agent-lightning/stable/reference/internal/#__codelineno-0-3) return llm.endpoint [](https://microsoft.github.io/agent-lightning/stable/reference/internal/#__codelineno-0-4) [](https://microsoft.github.io/agent-lightning/stable/reference/internal/#__codelineno-0-5) @llm_rollout(strip_proxy=False) [](https://microsoft.github.io/agent-lightning/stable/reference/internal/#__codelineno-0-6) def my_agent_no_strip(task, llm): [](https://microsoft.github.io/agent-lightning/stable/reference/internal/#__codelineno-0-7) return llm.model [](https://microsoft.github.io/agent-lightning/stable/reference/internal/#__codelineno-0-8) [](https://microsoft.github.io/agent-lightning/stable/reference/internal/#__codelineno-0-9) result = my_agent(task, llm) [](https://microsoft.github.io/agent-lightning/stable/reference/internal/#__codelineno-0-10) result = my_agent.rollout(task, resources, rollout)` ### `agentlightning.litagent.decorator.prompt_rollout(func=None)` [¶](https://microsoft.github.io/agent-lightning/stable/reference/internal/#agentlightning.litagent.decorator.prompt_rollout "Permanent link") `[](https://microsoft.github.io/agent-lightning/stable/reference/internal/#__codelineno-0-1) prompt_rollout( [](https://microsoft.github.io/agent-lightning/stable/reference/internal/#__codelineno-0-2) func: PromptRolloutFunc[T], [](https://microsoft.github.io/agent-lightning/stable/reference/internal/#__codelineno-0-3) ) -> FunctionalLitAgent[T]` `[](https://microsoft.github.io/agent-lightning/stable/reference/internal/#__codelineno-0-1) prompt_rollout() -> ( [](https://microsoft.github.io/agent-lightning/stable/reference/internal/#__codelineno-0-2) Callable[[PromptRolloutFunc[T]], FunctionalLitAgent[T]] [](https://microsoft.github.io/agent-lightning/stable/reference/internal/#__codelineno-0-3) )` Create a [`FunctionalLitAgent`](https://microsoft.github.io/agent-lightning/stable/reference/internal/#agentlightning.litagent.decorator.FunctionalLitAgent " agentlightning.litagent.decorator.FunctionalLitAgent") for prompt-based rollouts. This decorator is designed for agents that work with tunable prompt templates. It enables a workflow where algorithms manage and optimize the prompt template, while agents consume the template to perform rollouts. This is particularly useful for prompt optimization scenarios. Parameters: * **`func`** (`PromptRolloutFunc[T] | None`, default: `None` ) – Callable defining the agent's behavior. Supported signatures include: * `(task, prompt_template) -> result` * `(task, prompt_template, rollout) -> result` * `async (task, prompt_template) -> result` * `async (task, prompt_template, rollout) -> result` Returns: * `[FunctionalLitAgent](https://microsoft.github.io/agent-lightning/stable/reference/internal/#agentlightning.litagent.decorator.FunctionalLitAgent " agentlightning.litagent.decorator.FunctionalLitAgent") [T] | Callable[[PromptRolloutFunc[T]], [FunctionalLitAgent](https://microsoft.github.io/agent-lightning/stable/reference/internal/#agentlightning.litagent.decorator.FunctionalLitAgent " agentlightning.litagent.decorator.FunctionalLitAgent") [T]]` – [`FunctionalLitAgent`](https://microsoft.github.io/agent-lightning/stable/reference/internal/#agentlightning.litagent.decorator.FunctionalLitAgent " agentlightning.litagent.decorator.FunctionalLitAgent") that * `[FunctionalLitAgent](https://microsoft.github.io/agent-lightning/stable/reference/internal/#agentlightning.litagent.decorator.FunctionalLitAgent " agentlightning.litagent.decorator.FunctionalLitAgent") [T] | Callable[[PromptRolloutFunc[T]], [FunctionalLitAgent](https://microsoft.github.io/agent-lightning/stable/reference/internal/#agentlightning.litagent.decorator.FunctionalLitAgent " agentlightning.litagent.decorator.FunctionalLitAgent") [T]]` – wraps the supplied function. Examples: `[](https://microsoft.github.io/agent-lightning/stable/reference/internal/#__codelineno-0-1) @prompt_rollout [](https://microsoft.github.io/agent-lightning/stable/reference/internal/#__codelineno-0-2) def my_agent(task, prompt_template): [](https://microsoft.github.io/agent-lightning/stable/reference/internal/#__codelineno-0-3) messages = prompt_template.format(task=task.input) [](https://microsoft.github.io/agent-lightning/stable/reference/internal/#__codelineno-0-4) return messages [](https://microsoft.github.io/agent-lightning/stable/reference/internal/#__codelineno-0-5) [](https://microsoft.github.io/agent-lightning/stable/reference/internal/#__codelineno-0-6) result = my_agent(task, prompt_template) [](https://microsoft.github.io/agent-lightning/stable/reference/internal/#__codelineno-0-7) result = my_agent.rollout(task, resources, rollout)` ### `agentlightning.emitter.annotation.OperationContext` [¶](https://microsoft.github.io/agent-lightning/stable/reference/internal/#agentlightning.emitter.annotation.OperationContext "Permanent link") Context manager and decorator for tracing operations. This class manages a tracer-backed span for a logical unit of work. It can be used either: * As a decorator, in which case inputs and outputs are inferred automatically from the wrapped function's signature. * As a context manager, in which case inputs and outputs can be recorded explicitly via [`set_input`](https://microsoft.github.io/agent-lightning/stable/reference/internal/#agentlightning.emitter.annotation.OperationContext.set_input " set_input(*args, **kwargs)") and [`set_output`](https://microsoft.github.io/agent-lightning/stable/reference/internal/#agentlightning.emitter.annotation.OperationContext.set_output " set_output(output)") . Attributes: * **`name`** – Human-readable span name. * **`initial_attributes`** – Attributes applied when the span is created. * **`tracer`** – Tracer implementation used to create spans. #### `__call__(fn)` [¶](https://microsoft.github.io/agent-lightning/stable/reference/internal/#agentlightning.emitter.annotation.OperationContext.__call__ "Permanent link") Wrap a callable so its execution is traced in a span. When used as a decorator, a new span is created for each call to the wrapped function. The bound arguments are recorded as input attributes, the return value is recorded as an output attribute, and any exception is recorded and marks the span as an error. Parameters: * **`fn`** (`_FnType`) – The function or coroutine function to wrap. Returns: * `_FnType` – The wrapped callable. #### `__enter__()` [¶](https://microsoft.github.io/agent-lightning/stable/reference/internal/#agentlightning.emitter.annotation.OperationContext.__enter__ "Permanent link") Enter the context manager and start a new span. Returns: * `[OperationContext](https://microsoft.github.io/agent-lightning/stable/reference/internal/#agentlightning.emitter.annotation.OperationContext " agentlightning.emitter.annotation.OperationContext") ` – The current :class:`OperationContext` instance with an active span. #### `__exit__(exc_type, exc_val, exc_tb)` [¶](https://microsoft.github.io/agent-lightning/stable/reference/internal/#agentlightning.emitter.annotation.OperationContext.__exit__ "Permanent link") Exit the context manager and finish the span. #### `__init__(name, attributes, propagate=True)` [¶](https://microsoft.github.io/agent-lightning/stable/reference/internal/#agentlightning.emitter.annotation.OperationContext.__init__ "Permanent link") Initialize a new operation context. Parameters: * **`name`** (`str`) – Human-readable name of the span. * **`attributes`** (`Dict[str, Any]`) – Initial attributes attached to the span. Values are JSON-serialized where necessary. * **`propagate`** (`bool`, default: `True` ) – Whether the span should be sent to active exporters. #### `set_input(*args, **kwargs)` [¶](https://microsoft.github.io/agent-lightning/stable/reference/internal/#agentlightning.emitter.annotation.OperationContext.set_input "Permanent link") Record input arguments on the current span. Positional arguments are stored under the `input.args.` attributes, and keyword arguments are stored under `input.` attributes. This is intended for use inside a `with operation(...) as op` block. Parameters: * **`*args`** (`Any`, default: `()` ) – Positional arguments to record. * **`**kwargs`** (`Any`, default: `{}` ) – Keyword arguments to record. #### `set_output(output)` [¶](https://microsoft.github.io/agent-lightning/stable/reference/internal/#agentlightning.emitter.annotation.OperationContext.set_output "Permanent link") Record the output value on the current span. This is intended for use inside a `with operation(...) as op` block. Parameters: * **`output`** (`Any`) – The output value to record. #### `span()` [¶](https://microsoft.github.io/agent-lightning/stable/reference/internal/#agentlightning.emitter.annotation.OperationContext.span "Permanent link") Get the span that was created by this context manager. LLM Proxy[¶](https://microsoft.github.io/agent-lightning/stable/reference/internal/#llm-proxy "Permanent link") ---------------------------------------------------------------------------------------------------------------- ### `agentlightning.llm_proxy.ModelConfig` [¶](https://microsoft.github.io/agent-lightning/stable/reference/internal/#agentlightning.llm_proxy.ModelConfig "Permanent link") Bases: `TypedDict` LiteLLM model registration entry. This mirrors the items in LiteLLM's `model_list` section. Attributes: * **`model_name`** (`str`) – Logical model name exposed by the proxy. * **`litellm_params`** (`Dict[str, Any]`) – Parameters passed to LiteLLM for this model (e.g., backend model id, api\_base, additional options). ### `agentlightning.llm_proxy.LightningSpanExporter` [¶](https://microsoft.github.io/agent-lightning/stable/reference/internal/#agentlightning.llm_proxy.LightningSpanExporter "Permanent link") Bases: `SpanExporter` Buffered OTEL span exporter with subtree flushing and training-store sink. Design: * Spans are buffered until a root span's entire subtree is available. * A private event loop on a daemon thread runs async flush logic. * Rollout/attempt/sequence metadata is reconstructed by merging headers from any span within a subtree. Thread-safety: * Buffer access is protected by a re-entrant lock. * Export is synchronous to the caller yet schedules an async flush on the internal loop, then waits for completion. #### `export(spans)` [¶](https://microsoft.github.io/agent-lightning/stable/reference/internal/#agentlightning.llm_proxy.LightningSpanExporter.export "Permanent link") Export spans via buffered subtree flush. Appends spans to the internal buffer, then triggers an async flush on the private event loop. Blocks until that flush completes. Parameters: * **`spans`** (`Sequence[ReadableSpan]`) – Sequence of spans to export. Returns: * **`SpanExportResult`** ( `SpanExportResult` ) – SUCCESS on flush success, else FAILURE. #### `shutdown()` [¶](https://microsoft.github.io/agent-lightning/stable/reference/internal/#agentlightning.llm_proxy.LightningSpanExporter.shutdown "Permanent link") Shut down the exporter event loop. Safe to call at process exit. ### `agentlightning.llm_proxy.LightningOpenTelemetry` [¶](https://microsoft.github.io/agent-lightning/stable/reference/internal/#agentlightning.llm_proxy.LightningOpenTelemetry "Permanent link") Bases: `OpenTelemetry` OpenTelemetry integration that exports spans to the Lightning store. Responsibilities: * Ensures each request is annotated with a per-attempt sequence id so spans are ordered deterministically even with clock skew across nodes. * Uses [`LightningSpanExporter`](https://microsoft.github.io/agent-lightning/stable/reference/internal/#agentlightning.llm_proxy.LightningSpanExporter " agentlightning.llm_proxy.LightningSpanExporter") to persist spans for analytics and training. #### `async_pre_call_deployment_hook(kwargs, call_type=None)` `async` [¶](https://microsoft.github.io/agent-lightning/stable/reference/internal/#agentlightning.llm_proxy.LightningOpenTelemetry.async_pre_call_deployment_hook "Permanent link") The root span is sometimes missing (e.g., when Anthropic endpoint is used). It is created in an auth module in LiteLLM. If it's missing, we create it here. ### `agentlightning.llm_proxy.AddReturnTokenIds` [¶](https://microsoft.github.io/agent-lightning/stable/reference/internal/#agentlightning.llm_proxy.AddReturnTokenIds "Permanent link") Bases: `CustomLogger` LiteLLM logger hook to request token ids from vLLM. This mutates the outgoing request payload to include `return_token_ids=True` for backends that support token id return (e.g., vLLM). See also [vLLM PR #22587](https://github.com/vllm-project/vllm/pull/22587) #### `async_pre_call_hook(*args, **kwargs)` `async` [¶](https://microsoft.github.io/agent-lightning/stable/reference/internal/#agentlightning.llm_proxy.AddReturnTokenIds.async_pre_call_hook "Permanent link") Async pre-call hook to adjust request payload. Parameters: * **`args`** (`Any`, default: `()` ) – Positional args from LiteLLM. * **`kwargs`** (`Any`, default: `{}` ) – Keyword args from LiteLLM. Returns: * `Optional[Union[Exception, str, Dict[str, Any]]]` – Either an updated payload dict or an Exception to short-circuit. ### `agentlightning.llm_proxy.StreamConversionMiddleware` [¶](https://microsoft.github.io/agent-lightning/stable/reference/internal/#agentlightning.llm_proxy.StreamConversionMiddleware "Permanent link") Bases: `BaseHTTPMiddleware` Middleware to convert streaming responses to non-streaming responses. Useful for backend that only supports non-streaming responses. LiteLLM's OpenTelemetry is also buggy with streaming responses. The conversion will hopefully bypass the bug. #### `anthropic_stream_generator(original_response)` `async` [¶](https://microsoft.github.io/agent-lightning/stable/reference/internal/#agentlightning.llm_proxy.StreamConversionMiddleware.anthropic_stream_generator "Permanent link") Generate Anthropic SSE-formatted chunks from complete content blocks This is a dirty hack for Anthropic-style streaming from non-streaming response. The sse format is subject to change based on Anthropic's implementation. If so, try to use `MessageInspectionMiddleware` to inspect the update and fix accordingly. #### `openai_stream_generator(response_json)` `async` [¶](https://microsoft.github.io/agent-lightning/stable/reference/internal/#agentlightning.llm_proxy.StreamConversionMiddleware.openai_stream_generator "Permanent link") Convert a _complete_ OpenAI chat.completions choice into a stream of OpenAI-compatible SSE chunks. This emits: * an initial delta with the role ("assistant"), * a sequence of deltas for message.content (split into small chunks), * deltas for any tool\_calls (including id/name and chunked arguments), * a terminal chunk with finish\_reason, * and finally the literal '\[DONE\]'. Notes: * We only handle a _single_ choice (index 0 typically). * We purposefully don't attempt to stream logprobs. * Chunking strategy is simple and conservative to avoid splitting multi-byte characters: we slice on spaces where possible, then fall back to fixed-size substrings. ### `agentlightning.llm_proxy.MessageInspectionMiddleware` [¶](https://microsoft.github.io/agent-lightning/stable/reference/internal/#agentlightning.llm_proxy.MessageInspectionMiddleware "Permanent link") Bases: `BaseHTTPMiddleware` Middleware to inspect the request and response bodies. It's for debugging purposes. Add it via "message\_inspection" middleware alias. ### `agentlightning.llm_proxy.RolloutAttemptMiddleware` [¶](https://microsoft.github.io/agent-lightning/stable/reference/internal/#agentlightning.llm_proxy.RolloutAttemptMiddleware "Permanent link") Bases: `BaseHTTPMiddleware` Rewrites /rollout/{rid}/attempt/{aid}/... -> /... and injects x-rollout-id, x-attempt-id, x-sequence-id headers. LLMProxy can update store later without rebuilding middleware. Store[¶](https://microsoft.github.io/agent-lightning/stable/reference/internal/#store "Permanent link") -------------------------------------------------------------------------------------------------------- ### `agentlightning.store.base.UNSET = _UnsetType()` `module-attribute` [¶](https://microsoft.github.io/agent-lightning/stable/reference/internal/#agentlightning.store.base.UNSET "Permanent link") ### `agentlightning.store.utils.rollout_status_from_attempt(attempt, config)` `async` [¶](https://microsoft.github.io/agent-lightning/stable/reference/internal/#agentlightning.store.utils.rollout_status_from_attempt "Permanent link") Propagate the status of an attempt to the rollout. Returns: * `[RolloutStatus](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.RolloutStatus " agentlightning.RolloutStatus = Literal['queuing', 'preparing', 'running', 'failed', 'succeeded', 'cancelled', 'requeuing'] module-attribute (agentlightning.types.RolloutStatus)") ` – The status of the rollout from the perspective of the attempt. ### `agentlightning.store.utils.scan_unhealthy_rollouts(rollouts)` `async` [¶](https://microsoft.github.io/agent-lightning/stable/reference/internal/#agentlightning.store.utils.scan_unhealthy_rollouts "Permanent link") Perform health check on all running rollouts in the store. This method should be called periodically to: 1. Check for unresponsive attempts (no heartbeat or spans for a while) 2. Check for timed-out rollouts (running too long since start\_time) This operation is completely unlocked. The caller is responsible for locking the store. Parameters: * **`rollouts`** (`List[[AttemptedRollout](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.AttemptedRollout " agentlightning.AttemptedRollout (agentlightning.types.AttemptedRollout)") ]`) – The list of running rollouts to check. Returns: * `Dict[Tuple[str, str], [AttemptStatus](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.AttemptStatus " agentlightning.AttemptStatus = Literal['preparing', 'running', 'failed', 'succeeded', 'unresponsive', 'timeout'] module-attribute (agentlightning.types.AttemptStatus)") ]` – A dictionary of updates to the rollouts. Tracing and OpenTelemetry[¶](https://microsoft.github.io/agent-lightning/stable/reference/internal/#tracing-and-opentelemetry "Permanent link") ------------------------------------------------------------------------------------------------------------------------------------------------ ### `agentlightning.tracer.otel.LightningSpanProcessor` [¶](https://microsoft.github.io/agent-lightning/stable/reference/internal/#agentlightning.tracer.otel.LightningSpanProcessor "Permanent link") Bases: `SpanProcessor` Span processor that subclasses OpenTelemetry's `SpanProcessor` and adds support to dump traces to a [`LightningStore`](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.LightningStore " agentlightning.LightningStore") . It serves two purposes: 1. Records all the spans in a local buffer. 2. Submits the spans to the event loop to be added to the store. #### `attempt_id` `property` [¶](https://microsoft.github.io/agent-lightning/stable/reference/internal/#agentlightning.tracer.otel.LightningSpanProcessor.attempt_id "Permanent link") The attempt ID to submit the spans to. #### `disable_store_submission` `property` `writable` [¶](https://microsoft.github.io/agent-lightning/stable/reference/internal/#agentlightning.tracer.otel.LightningSpanProcessor.disable_store_submission "Permanent link") Whether to disable submitting spans to the store. #### `rollout_id` `property` [¶](https://microsoft.github.io/agent-lightning/stable/reference/internal/#agentlightning.tracer.otel.LightningSpanProcessor.rollout_id "Permanent link") The rollout ID to submit the spans to. #### `store` `property` [¶](https://microsoft.github.io/agent-lightning/stable/reference/internal/#agentlightning.tracer.otel.LightningSpanProcessor.store "Permanent link") The store to submit the spans to. #### `on_end(span)` [¶](https://microsoft.github.io/agent-lightning/stable/reference/internal/#agentlightning.tracer.otel.LightningSpanProcessor.on_end "Permanent link") Process a span when it ends. Parameters: * **`span`** (`ReadableSpan`) – The span that has ended. #### `spans()` [¶](https://microsoft.github.io/agent-lightning/stable/reference/internal/#agentlightning.tracer.otel.LightningSpanProcessor.spans "Permanent link") Get the list of spans collected by this processor. This is useful for debugging and testing purposes. Returns: * `List[[Span](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.Span " agentlightning.Span (agentlightning.types.Span)") ]` – List of [`Span`](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.Span " agentlightning.Span") objects collected during tracing. Deprecated APIs[¶](https://microsoft.github.io/agent-lightning/stable/reference/internal/#deprecated-apis "Permanent link") ---------------------------------------------------------------------------------------------------------------------------- ### `agentlightning.emitter.reward.reward(fn)` [¶](https://microsoft.github.io/agent-lightning/stable/reference/internal/#agentlightning.emitter.reward.reward "Permanent link") Decorate a reward function so its outputs are tracked as spans. The decorator integrates with AgentOps when it is available and falls back to the built-in telemetry otherwise. Both synchronous and asynchronous functions are supported transparently. Deprecated This decorator is deprecated. Use [`emit_reward`](https://microsoft.github.io/agent-lightning/stable/reference/agent/#agentlightning.emit_reward " agentlightning.emit_reward(reward, *, primary_key=None, attributes=None, propagate=True)") instead. Parameters: * **`fn`** (`_FnType`) – Callable that produces a numeric reward. Returns: * `_FnType` – Wrapped callable that preserves the original signature. ### `agentlightning.server.AgentLightningServer` [¶](https://microsoft.github.io/agent-lightning/stable/reference/internal/#agentlightning.server.AgentLightningServer "Permanent link") High-level controller for the legacy Agent Lightning FastAPI server. The controller orchestrates server start-up, task queueing, resource updates, and retrieval of client rollouts. It is primarily used by existing systems that still rely on the HTTP-based workflow. Deprecated [`AgentLightningServer`](https://microsoft.github.io/agent-lightning/stable/reference/internal/#agentlightning.server.AgentLightningServer " agentlightning.server.AgentLightningServer") is part of the legacy client/server stack. Prefer the store-based runtime for new integrations. #### `__init__(host='127.0.0.1', port=8000, task_timeout_seconds=300.0)` [¶](https://microsoft.github.io/agent-lightning/stable/reference/internal/#agentlightning.server.AgentLightningServer.__init__ "Permanent link") Initialize the controller. Parameters: * **`host`** (`str`, default: `'127.0.0.1'` ) – Hostname or IP address to bind the HTTP server to. * **`port`** (`int`, default: `8000` ) – TCP port exposed by the server. * **`task_timeout_seconds`** (`float`, default: `300.0` ) – Seconds before a claimed task is considered stale and re-queued. #### `get_completed_rollout(rollout_id)` `async` [¶](https://microsoft.github.io/agent-lightning/stable/reference/internal/#agentlightning.server.AgentLightningServer.get_completed_rollout "Permanent link") Retrieve a specific completed rollout by identifier. #### `poll_completed_rollout(rollout_id, timeout=None)` `async` [¶](https://microsoft.github.io/agent-lightning/stable/reference/internal/#agentlightning.server.AgentLightningServer.poll_completed_rollout "Permanent link") Poll for a completed rollout until it becomes available or a timeout expires. Parameters: * **`rollout_id`** (`str`) – Identifier of the rollout to wait for. * **`timeout`** (`Optional[float]`, default: `None` ) – Maximum number of seconds to wait. `None` waits indefinitely. Returns: * `Optional[[RolloutLegacy](https://microsoft.github.io/agent-lightning/stable/reference/internal/#agentlightning.RolloutLegacy " agentlightning.RolloutLegacy (agentlightning.types.RolloutLegacy)") ]` – Retrieved rollout, or `None` when the timeout is reached without success. #### `queue_task(sample, mode=None, resources_id=None, metadata=None)` `async` [¶](https://microsoft.github.io/agent-lightning/stable/reference/internal/#agentlightning.server.AgentLightningServer.queue_task "Permanent link") Add a task to the queue for a client to process. #### `retrieve_completed_rollouts()` `async` [¶](https://microsoft.github.io/agent-lightning/stable/reference/internal/#agentlightning.server.AgentLightningServer.retrieve_completed_rollouts "Permanent link") Return every completed rollout and clear the internal buffer. #### `run_forever()` `async` [¶](https://microsoft.github.io/agent-lightning/stable/reference/internal/#agentlightning.server.AgentLightningServer.run_forever "Permanent link") Run the server indefinitely until `stop()` is invoked. #### `start()` `async` [¶](https://microsoft.github.io/agent-lightning/stable/reference/internal/#agentlightning.server.AgentLightningServer.start "Permanent link") Start the FastAPI server in the background. #### `stop()` `async` [¶](https://microsoft.github.io/agent-lightning/stable/reference/internal/#agentlightning.server.AgentLightningServer.stop "Permanent link") Stop the FastAPI server and wait for a graceful shutdown. #### `update_resources(resources)` `async` [¶](https://microsoft.github.io/agent-lightning/stable/reference/internal/#agentlightning.server.AgentLightningServer.update_resources "Permanent link") Publish a new resource bundle and return its generated identifier. ### `agentlightning.server.ServerDataStore` [¶](https://microsoft.github.io/agent-lightning/stable/reference/internal/#agentlightning.server.ServerDataStore "Permanent link") Async-safe container for in-memory server state. The store tracks queued tasks, claimed tasks, uploaded rollouts, and the currently published resources. All interactions are guarded by asyncio locks so that the FastAPI handlers can safely run in parallel. Deprecated [`ServerDataStore`](https://microsoft.github.io/agent-lightning/stable/reference/internal/#agentlightning.server.ServerDataStore " agentlightning.server.ServerDataStore") is part of the legacy client/server stack. Use [`LightningStore`](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.LightningStore " agentlightning.LightningStore") instead. #### `add_task(sample, mode=None, resources_id=None, metadata=None)` `async` [¶](https://microsoft.github.io/agent-lightning/stable/reference/internal/#agentlightning.server.ServerDataStore.add_task "Permanent link") Enqueue a new task and return the generated rollout identifier. Parameters: * **`sample`** (`Any`) – Payload that describes the task input. * **`mode`** (`Literal['train', 'val', 'test'] | None`, default: `None` ) – Phase in which the sample should be executed (`"train"`, `"val"`, or `"test"`). * **`resources_id`** (`str | None`, default: `None` ) – Identifier of a resource bundle that the executor should load before running the task. * **`metadata`** (`Dict[str, Any] | None`, default: `None` ) – Optional metadata forwarded to the executor. Returns: * `str` – Unique rollout identifier assigned to the task. #### `get_latest_resources()` `async` [¶](https://microsoft.github.io/agent-lightning/stable/reference/internal/#agentlightning.server.ServerDataStore.get_latest_resources "Permanent link") Return the most recent resource bundle, if one exists. #### `get_next_task()` `async` [¶](https://microsoft.github.io/agent-lightning/stable/reference/internal/#agentlightning.server.ServerDataStore.get_next_task "Permanent link") Retrieve the next task from the queue without blocking. Returns: * `Optional[[Task](https://microsoft.github.io/agent-lightning/stable/reference/internal/#agentlightning.Task " agentlightning.Task (agentlightning.types.Task)") ]` – Next [`Task`](https://microsoft.github.io/agent-lightning/stable/reference/internal/#agentlightning.Task " agentlightning.Task") ready to execute, or `None` * `Optional[[Task](https://microsoft.github.io/agent-lightning/stable/reference/internal/#agentlightning.Task " agentlightning.Task (agentlightning.types.Task)") ]` – when the queue is empty. #### `get_processing_tasks()` [¶](https://microsoft.github.io/agent-lightning/stable/reference/internal/#agentlightning.server.ServerDataStore.get_processing_tasks "Permanent link") Return a copy of currently processing tasks for timeout checking. #### `get_resources_by_id(resources_id)` `async` [¶](https://microsoft.github.io/agent-lightning/stable/reference/internal/#agentlightning.server.ServerDataStore.get_resources_by_id "Permanent link") Retrieve a specific resource bundle by identifier. Parameters: * **`resources_id`** (`str`) – Identifier that was previously published to the store. Returns: * `Optional[[ResourcesUpdate](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.ResourcesUpdate " agentlightning.ResourcesUpdate (agentlightning.types.ResourcesUpdate)") ]` – Matching [`ResourcesUpdate`](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.ResourcesUpdate " agentlightning.ResourcesUpdate") * `Optional[[ResourcesUpdate](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.ResourcesUpdate " agentlightning.ResourcesUpdate (agentlightning.types.ResourcesUpdate)") ]` – instance, or `None` when the identifier is unknown. #### `requeue_task(task)` `async` [¶](https://microsoft.github.io/agent-lightning/stable/reference/internal/#agentlightning.server.ServerDataStore.requeue_task "Permanent link") Requeue a task that timed out while being processed. #### `retrieve_completed_rollouts()` `async` [¶](https://microsoft.github.io/agent-lightning/stable/reference/internal/#agentlightning.server.ServerDataStore.retrieve_completed_rollouts "Permanent link") Return all completed rollouts and clear the internal buffer. #### `retrieve_rollout(rollout_id)` `async` [¶](https://microsoft.github.io/agent-lightning/stable/reference/internal/#agentlightning.server.ServerDataStore.retrieve_rollout "Permanent link") Retrieve and remove a stored rollout by identifier. Parameters: * **`rollout_id`** (`str`) – Identifier of the rollout to fetch. Returns: * `Optional[[RolloutLegacy](https://microsoft.github.io/agent-lightning/stable/reference/internal/#agentlightning.RolloutLegacy " agentlightning.RolloutLegacy (agentlightning.types.RolloutLegacy)") ]` – Stored [`RolloutLegacy`](https://microsoft.github.io/agent-lightning/stable/reference/internal/#agentlightning.RolloutLegacy " agentlightning.RolloutLegacy") , or `None` * `Optional[[RolloutLegacy](https://microsoft.github.io/agent-lightning/stable/reference/internal/#agentlightning.RolloutLegacy " agentlightning.RolloutLegacy (agentlightning.types.RolloutLegacy)") ]` – when the identifier is unknown. #### `store_rollout(rollout)` `async` [¶](https://microsoft.github.io/agent-lightning/stable/reference/internal/#agentlightning.server.ServerDataStore.store_rollout "Permanent link") Persist a completed rollout for later inspection. Parameters: * **`rollout`** (`[RolloutLegacy](https://microsoft.github.io/agent-lightning/stable/reference/internal/#agentlightning.RolloutLegacy " agentlightning.RolloutLegacy (agentlightning.types.RolloutLegacy)") `) – Rollout returned by a client. #### `update_resources(update)` `async` [¶](https://microsoft.github.io/agent-lightning/stable/reference/internal/#agentlightning.server.ServerDataStore.update_resources "Permanent link") Persist a new resource bundle and mark it as the latest version. Parameters: * **`update`** (`[ResourcesUpdate](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.ResourcesUpdate " agentlightning.ResourcesUpdate (agentlightning.types.ResourcesUpdate)") `) – Resource payload received from a client. ### `agentlightning.client.AgentLightningClient` [¶](https://microsoft.github.io/agent-lightning/stable/reference/internal/#agentlightning.client.AgentLightningClient "Permanent link") Client wrapper for the legacy version-aware Agent Lightning server. The client exposes synchronous and asynchronous helpers for polling tasks, retrieving resource bundles, and submitting rollouts. It also maintains a simple in-memory cache keyed by the server-provided resource identifier to avoid redundant network requests. Deprecated [`AgentLightningClient`](https://microsoft.github.io/agent-lightning/stable/reference/internal/#agentlightning.client.AgentLightningClient " agentlightning.client.AgentLightningClient") is part of the legacy client/server stack. New code should rely on the store-based APIs implemented in `agentlightning.store`. Attributes: * **`endpoint`** – Base URL of the Agent Lightning server. * **`poll_interval`** – Delay in seconds between polling attempts when no task is available. * **`timeout`** – Timeout in seconds applied to HTTP requests. * **`task_count`** – Number of tasks claimed during the lifetime of this client. #### `__init__(endpoint, poll_interval=5.0, timeout=10.0)` [¶](https://microsoft.github.io/agent-lightning/stable/reference/internal/#agentlightning.client.AgentLightningClient.__init__ "Permanent link") Initialize the client. Parameters: * **`endpoint`** (`str`) – Root URL of the Agent Lightning server. * **`poll_interval`** (`float`, default: `5.0` ) – Seconds to wait between polling attempts. * **`timeout`** (`float`, default: `10.0` ) – Seconds before a request to the server is considered timed out. #### `get_latest_resources()` [¶](https://microsoft.github.io/agent-lightning/stable/reference/internal/#agentlightning.client.AgentLightningClient.get_latest_resources "Permanent link") Fetch the most recent resource bundle advertised by the server. Returns: * `Optional[[ResourcesUpdate](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.ResourcesUpdate " agentlightning.ResourcesUpdate (agentlightning.types.ResourcesUpdate)") ]` – [`ResourcesUpdate`](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.ResourcesUpdate " agentlightning.ResourcesUpdate") for the * `Optional[[ResourcesUpdate](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.ResourcesUpdate " agentlightning.ResourcesUpdate (agentlightning.types.ResourcesUpdate)") ]` – newest version, or `None` when unavailable. #### `get_latest_resources_async()` `async` [¶](https://microsoft.github.io/agent-lightning/stable/reference/internal/#agentlightning.client.AgentLightningClient.get_latest_resources_async "Permanent link") Fetch the most recent resource bundle advertised by the server. Returns: * `Optional[[ResourcesUpdate](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.ResourcesUpdate " agentlightning.ResourcesUpdate (agentlightning.types.ResourcesUpdate)") ]` – [`ResourcesUpdate`](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.ResourcesUpdate " agentlightning.ResourcesUpdate") for the * `Optional[[ResourcesUpdate](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.ResourcesUpdate " agentlightning.ResourcesUpdate (agentlightning.types.ResourcesUpdate)") ]` – newest version, or `None` when unavailable. #### `get_resources_by_id(resource_id)` [¶](https://microsoft.github.io/agent-lightning/stable/reference/internal/#agentlightning.client.AgentLightningClient.get_resources_by_id "Permanent link") Fetch a specific resource bundle by identifier. Parameters: * **`resource_id`** (`str`) – Identifier sourced from the task metadata. Returns: * `Optional[[ResourcesUpdate](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.ResourcesUpdate " agentlightning.ResourcesUpdate (agentlightning.types.ResourcesUpdate)") ]` – Cached or freshly downloaded * `Optional[[ResourcesUpdate](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.ResourcesUpdate " agentlightning.ResourcesUpdate (agentlightning.types.ResourcesUpdate)") ]` – [`ResourcesUpdate`](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.ResourcesUpdate " agentlightning.ResourcesUpdate") , or * `Optional[[ResourcesUpdate](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.ResourcesUpdate " agentlightning.ResourcesUpdate (agentlightning.types.ResourcesUpdate)") ]` – `None` when the server returns an error. #### `get_resources_by_id_async(resource_id)` `async` [¶](https://microsoft.github.io/agent-lightning/stable/reference/internal/#agentlightning.client.AgentLightningClient.get_resources_by_id_async "Permanent link") Fetch a specific resource bundle by identifier. Parameters: * **`resource_id`** (`str`) – Identifier sourced from the task metadata. Returns: * `Optional[[ResourcesUpdate](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.ResourcesUpdate " agentlightning.ResourcesUpdate (agentlightning.types.ResourcesUpdate)") ]` – Cached or freshly downloaded * `Optional[[ResourcesUpdate](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.ResourcesUpdate " agentlightning.ResourcesUpdate (agentlightning.types.ResourcesUpdate)") ]` – [`ResourcesUpdate`](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.ResourcesUpdate " agentlightning.ResourcesUpdate") , or * `Optional[[ResourcesUpdate](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.ResourcesUpdate " agentlightning.ResourcesUpdate (agentlightning.types.ResourcesUpdate)") ]` – `None` when the server returns an error. #### `poll_next_task()` [¶](https://microsoft.github.io/agent-lightning/stable/reference/internal/#agentlightning.client.AgentLightningClient.poll_next_task "Permanent link") Poll the server synchronously until a task becomes available. Returns: * `Optional[[Task](https://microsoft.github.io/agent-lightning/stable/reference/internal/#agentlightning.Task " agentlightning.Task (agentlightning.types.Task)") ]` – The next [`Task`](https://microsoft.github.io/agent-lightning/stable/reference/internal/#agentlightning.Task " agentlightning.Task") available for execution, or * `Optional[[Task](https://microsoft.github.io/agent-lightning/stable/reference/internal/#agentlightning.Task " agentlightning.Task (agentlightning.types.Task)") ]` – `None` if polling fails. #### `poll_next_task_async()` `async` [¶](https://microsoft.github.io/agent-lightning/stable/reference/internal/#agentlightning.client.AgentLightningClient.poll_next_task_async "Permanent link") Poll the server asynchronously until a task becomes available. Returns: * `Optional[[Task](https://microsoft.github.io/agent-lightning/stable/reference/internal/#agentlightning.Task " agentlightning.Task (agentlightning.types.Task)") ]` – The next [`Task`](https://microsoft.github.io/agent-lightning/stable/reference/internal/#agentlightning.Task " agentlightning.Task") exposed by the server, * `Optional[[Task](https://microsoft.github.io/agent-lightning/stable/reference/internal/#agentlightning.Task " agentlightning.Task (agentlightning.types.Task)") ]` – or `None` if polling fails. #### `post_rollout(rollout)` [¶](https://microsoft.github.io/agent-lightning/stable/reference/internal/#agentlightning.client.AgentLightningClient.post_rollout "Permanent link") Submit a completed rollout back to the server. Parameters: * **`rollout`** (`[RolloutLegacy](https://microsoft.github.io/agent-lightning/stable/reference/internal/#agentlightning.RolloutLegacy " agentlightning.RolloutLegacy (agentlightning.types.RolloutLegacy)") `) – Legacy rollout payload produced by the executor. Returns: * `Optional[Dict[str, Any]]` – Parsed JSON response returned by the server, or `None` when the request fails. #### `post_rollout_async(rollout)` `async` [¶](https://microsoft.github.io/agent-lightning/stable/reference/internal/#agentlightning.client.AgentLightningClient.post_rollout_async "Permanent link") Submit a completed rollout back to the server. Parameters: * **`rollout`** (`[RolloutLegacy](https://microsoft.github.io/agent-lightning/stable/reference/internal/#agentlightning.RolloutLegacy " agentlightning.RolloutLegacy (agentlightning.types.RolloutLegacy)") `) – Legacy rollout payload produced by the executor. Returns: * `Optional[Dict[str, Any]]` – Parsed JSON response returned by the server, or `None` when the request fails. ### `agentlightning.client.DevTaskLoader` [¶](https://microsoft.github.io/agent-lightning/stable/reference/internal/#agentlightning.client.DevTaskLoader "Permanent link") Bases: `[AgentLightningClient](https://microsoft.github.io/agent-lightning/stable/reference/internal/#agentlightning.client.AgentLightningClient " agentlightning.client.AgentLightningClient") ` In-memory task loader used for development and integration tests. The loader mimics the behavior of the legacy HTTP server by storing tasks and resources locally. Polling methods simply iterate over the provided collection, allowing rapid iteration without provisioning any external infrastructure. Deprecated [`DevTaskLoader`](https://microsoft.github.io/agent-lightning/stable/reference/internal/#agentlightning.client.DevTaskLoader " agentlightning.client.DevTaskLoader") is a compatibility shim. Prefer [`Trainer.dev`](https://microsoft.github.io/agent-lightning/stable/reference/trainer/#agentlightning.Trainer.dev " dev(agent, train_dataset=None, *, val_dataset=None)") for new code. #### `rollouts` `property` [¶](https://microsoft.github.io/agent-lightning/stable/reference/internal/#agentlightning.client.DevTaskLoader.rollouts "Permanent link") Return the rollouts posted back to the loader during development runs. #### `__init__(tasks, resources, **kwargs)` [¶](https://microsoft.github.io/agent-lightning/stable/reference/internal/#agentlightning.client.DevTaskLoader.__init__ "Permanent link") Initialize the loader with predefined tasks and resources. Parameters: * **`tasks`** (`Union[List[[TaskInput](https://microsoft.github.io/agent-lightning/stable/reference/internal/#agentlightning.TaskInput " agentlightning.TaskInput = Any module-attribute (agentlightning.types.TaskInput)") ], List[[Task](https://microsoft.github.io/agent-lightning/stable/reference/internal/#agentlightning.Task " agentlightning.Task (agentlightning.types.Task)") ]]`) – Sequence of task inputs or preconstructed tasks that will be served in order. * **`resources`** (`Union[[NamedResources](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.NamedResources " agentlightning.NamedResources = Dict[str, ResourceUnion] module-attribute (agentlightning.types.NamedResources)") , [ResourcesUpdate](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.ResourcesUpdate " agentlightning.ResourcesUpdate (agentlightning.types.ResourcesUpdate)") ]`) – Static resources returned for any `resources_id` query. * **`**kwargs`** (`Any`, default: `{}` ) – Additional keyword arguments forwarded to the parent client. Raises: * `ValueError` – If no tasks are provided or both [`Task`](https://microsoft.github.io/agent-lightning/stable/reference/internal/#agentlightning.Task " agentlightning.Task") and [`TaskInput`](https://microsoft.github.io/agent-lightning/stable/reference/internal/#agentlightning.TaskInput " agentlightning.TaskInput = Any module-attribute ") instances are mixed. #### `poll_next_task()` [¶](https://microsoft.github.io/agent-lightning/stable/reference/internal/#agentlightning.client.DevTaskLoader.poll_next_task "Permanent link") Return the next task from the local queue. If [`TaskInput`](https://microsoft.github.io/agent-lightning/stable/reference/internal/#agentlightning.TaskInput " agentlightning.TaskInput = Any module-attribute ") instances were provided, they are converted into [`Task`](https://microsoft.github.io/agent-lightning/stable/reference/internal/#agentlightning.Task " agentlightning.Task") objects on the fly. Otherwise, the preconstructed tasks are returned in sequence. Returns: * `Optional[[Task](https://microsoft.github.io/agent-lightning/stable/reference/internal/#agentlightning.Task " agentlightning.Task (agentlightning.types.Task)") ]` – Next task to execute. ### `agentlightning.Task` [¶](https://microsoft.github.io/agent-lightning/stable/reference/internal/#agentlightning.Task "Permanent link") Bases: `BaseModel` Rollout request served to client agents. Deprecated The legacy HTTP client/server stack still uses this model. Prefer [`LightningStore`](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.LightningStore " agentlightning.LightningStore") APIs for new workflows. ### `agentlightning.TaskInput = Any` `module-attribute` [¶](https://microsoft.github.io/agent-lightning/stable/reference/internal/#agentlightning.TaskInput "Permanent link") Task input type. Accepts arbitrary payloads. ### `agentlightning.TaskIfAny` [¶](https://microsoft.github.io/agent-lightning/stable/reference/internal/#agentlightning.TaskIfAny "Permanent link") Bases: `BaseModel` A task or indication that no task is available. Deprecated Use [`LightningStore`](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.LightningStore " agentlightning.LightningStore") APIs for new workflows. #### `is_available` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/stable/reference/internal/#agentlightning.TaskIfAny.is_available "Permanent link") Indication that a task is available. ### `agentlightning.RolloutRawResultLegacy = Union[None, float, List[Triplet], List[Dict[str, Any]], List[ReadableSpan], RolloutLegacy]` `module-attribute` [¶](https://microsoft.github.io/agent-lightning/stable/reference/internal/#agentlightning.RolloutRawResultLegacy "Permanent link") Legacy rollout result type. Deprecated Use [`RolloutRawResult`](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.RolloutRawResult " agentlightning.RolloutRawResult = Union[None, float, List[ReadableSpan], List[Span], List[SpanCoreFields]] module-attribute ") instead. ### `agentlightning.RolloutLegacy` [¶](https://microsoft.github.io/agent-lightning/stable/reference/internal/#agentlightning.RolloutLegacy "Permanent link") Bases: `BaseModel` Legacy reporting payload exchanged with the deprecated HTTP server. Deprecated Use [`Rollout`](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.Rollout " agentlightning.Rollout") instead. Back to top --- # Changelog - Agent-lightning [Skip to content](https://microsoft.github.io/agent-lightning/stable/changelog/#changelog) ⚠️ You are viewing development documentation. For stable documentation, please visit [https://microsoft.github.io/agent-lightning/stable/](https://microsoft.github.io/agent-lightning/stable/) Changelog[¶](https://microsoft.github.io/agent-lightning/stable/changelog/#changelog "Permanent link") ======================================================================================================= Agent-lightning v0.3.0 (12/24/2025)[¶](https://microsoft.github.io/agent-lightning/stable/changelog/#agent-lightning-v030-12242025 "Permanent link") ----------------------------------------------------------------------------------------------------------------------------------------------------- Agent-lightning v0.3.0 is a major release that introduces several new features and bug fixes. The release is a collaborative effort between Agent-lightning core teams and the community. Thanks to all the contributors who made this release possible. ### Highlights[¶](https://microsoft.github.io/agent-lightning/stable/changelog/#highlights "Permanent link") * **Tinker integration**: Support Tinker as an alternative backend for Reinforcement Learning (#226 #245 #264 #269 #327). See [example code](https://github.com/microsoft/agent-lightning/tree/v0.3.0/examples/tinker) , [blog 1](https://medium.com/@yugez/tuning-any-ai-agent-with-tinker-agent-lightning-part-1-1d8c9a397f0e) and [blog 2](https://medium.com/@yugez/tuning-any-ai-agent-with-tinker-agent-lightning-part-2-332c5437f0dc) . * **Azure OpenAI integration**: Support Azure OpenAI as a backend for LLM inference and supervised fine-tuning (#256 #327). [Example code](https://github.com/microsoft/agent-lightning/tree/v0.3.0/examples/azure) . * **MongoDB-based Lightning Store** is added as an alternative backend for Lightning Store (#323). [Documentation](https://microsoft.github.io/agent-lightning/v0.3.0/tutorials/parallelize/#parallelizing-lightningstore) . * **Contrib package**: Add contrib package for community projects. Search-R1 is integrated as a contrib recipe. More coming. (#239 #396 #410 #412 #417). * **RESTful API**: Stabilize and document RESTful API for Lightning Store (#241 #275). [Documentation](https://microsoft.github.io/agent-lightning/v0.3.0/reference/restful/) . * **OTel Semantic Conventions** that are specifically designed for Agent-optimization areas (#340). [Documentation](http://microsoft.github.io/agent-lightning/v0.3.0/reference/semconv/) . * _\[Preview\]_ **Agent-lightning Dashboard** is now available (#288 #289 #291 #296 #371 #375). It's the official web application for inspecting and debugging Agent-lightning experiments. See details [here](https://microsoft.github.io/agent-lightning/v0.3.0/tutorials/debug/) . * _\[Preview\]_ **Multi-modality example** featuring VERL and a LangGraph agent on ChartQA dataset (#379). [Example code](https://github.com/microsoft/agent-lightning/tree/v0.3.0/examples/chartqa) . * _\[Preview\]_ Integrate **Claude Code** as a LitAgent and support training on SWE-Bench (#332 #346 #348). [Example code](https://github.com/microsoft/agent-lightning/tree/v0.3.0/examples/claude_code) . * _\[Preview\]_ **Weave tracer** as a substitute for AgentOps tracer (#277 #411 #420 #423). [Documentation](https://microsoft.github.io/agent-lightning/v0.3.0/tutorials/traces/#weave-tracer-experimental) . * _\[Preview\]_ **Trajectory Level Aggregation** for more efficient training with VERL. See [blog](https://agent-lightning.github.io/posts/trajectory_level_aggregation/) and [documentation](https://microsoft.github.io/agent-lightning/v0.3.0/algorithm-zoo/verl/) . ### Store Benchmark[¶](https://microsoft.github.io/agent-lightning/stable/changelog/#store-benchmark "Permanent link") In this release, the Lightning Store core was redesigned for significantly greater efficiency and scalability (#315 #318 #328 #342 #344 #356 #380 #388 #418 #421). The benchmark results below demonstrate the impact: with large numbers of concurrent runners, v0.3.0 delivers up to a 15x increase in throughput compared to v0.2.2. | Throughput (#rollout/sec) | v0.2.2 | v0.3.0 (in-memory) | v0.3.0 (Mongo) | | --- | --- | --- | --- | | Minimal (batch, #runner=32, #turns=6) | 8.73 | 9.06 | 8.71 | | Medium (batch, #runners=100, #turns=10) | 12.03 | 23.26 | 32.79 | | Mid-high (batch, #runners=300, #turns=6) | 10.61 | 24.42 | 40.24 | | Large (batch, #runners=1000, #turns=3) | 3.36 | 14.60 | 50.05 | | Long queue (queue, #runners=256, #turns=4) | 7.42 | 30.86 | 57.01 | | Heavy trace (queue, #runners=512, #turns=20) | 5.93 | 13.28 | 29.41 | _Notes:_ 1. Benchmarks were run on a single Standard\_D32as\_v4 Azure VM (Large and heavy trace tests used Standard\_D64ads\_v5), executed via GitHub Actions. 2. Two algorithm patterns are evaluated: the batch pattern submits a group of rollouts and waits for all to finish before starting the next group, while the queue pattern maintains a set number of in-flight rollouts, submitting new ones as soon as capacity frees up. Configuration details are available [here](https://github.com/microsoft/agent-lightning/blob/v0.3.0/.github/workflows/benchmark.yml) . 3. The number of turns is directly proportional to the number of spans each rollout generates. ### Maintenance and Bug fixes[¶](https://microsoft.github.io/agent-lightning/stable/changelog/#maintenance-and-bug-fixes "Permanent link") #### Core (Store, Interfaces, etc.)[¶](https://microsoft.github.io/agent-lightning/stable/changelog/#core-store-interfaces-etc "Permanent link") * Add Trainer port option for client-server strategies (#198) * Fix store port conflict handling (#227) * Unified PythonServerLauncher (#286 #292 #303) * Make health timeout configurable (#305) * Refactor logging (#306) * Support OTLP in LightningStore (#313) * Centralized metrics helper (#368) * Fix redundant cancel tracebacks on Ctrl+C (#370) #### Proxy, Adapters and Algorithms[¶](https://microsoft.github.io/agent-lightning/stable/changelog/#proxy-adapters-and-algorithms "Permanent link") * Fix training metrics before and after processing in VERL (#145) * Forward streaming requests for Anthropic and OpenAI APIs (as non-streaming requests) (#299) * Check traces with reward for VERL (#317) * Patch LiteLLM root span (#341) * Handle ref\_in\_actor flag for LoRA compatibility (#386) * Support `with_llm_proxy` and `with_store` in algorithms (#398) * Support image URL export in TracerTraceToTriplets (#400) * Fix match\_rewards assign\_to elements in TraceTree (#403) * Support customizing trainer and daemon in VERL (#407) #### Runners, Tracers and Agents[¶](https://microsoft.github.io/agent-lightning/stable/changelog/#runners-tracers-and-agents "Permanent link") * Refactor tracer initialization (#321) * Fix OpenAI Agents 0.6 compatibility (#322) * `emit_operation`, `emit_annotation`, tags and links (#359) * Sunset HTTP tracer (#402) #### Examples[¶](https://microsoft.github.io/agent-lightning/stable/changelog/#examples "Permanent link") * Fix typos in train-first-agent.md (#263) * Fix room\_selector example which always runs the first task (#270) * Fix typo in SQL agent example (#285) * Add the README and script files for training SQL agent on NPU (#272) * Examples Catalog and Refine Contribution Guide (#331) * Upgrade LangChain to 1.x (#364) * Update RAG example to Agent-lightning v0.2.x (#349) #### Miscellaneous[¶](https://microsoft.github.io/agent-lightning/stable/changelog/#miscellaneous "Permanent link") * DeepWiki Badge (#263) * Add AGENTS.md (#374) ### New Contributors[¶](https://microsoft.github.io/agent-lightning/stable/changelog/#new-contributors "Permanent link") Warm welcome to our first-time contributors: @cptnm3, @TerryChan, @genji970, @zxgx, @xiaochulaoban, @lspinheiro, @Kwanghoon-Choi, @Vasuk12, @totoluo, @jinghuan-Chen 🎉 **Full Changelog**: https://github.com/microsoft/agent-lightning/compare/v0.2.0...v0.3.0 * * * Agent-lightning v0.2.2 (11/12/2025)[¶](https://microsoft.github.io/agent-lightning/stable/changelog/#agent-lightning-v022-11122025 "Permanent link") ----------------------------------------------------------------------------------------------------------------------------------------------------- Agent-lightning v0.2.2 is a stabilization release for v0.2.1. It introduces several bug fixes. * Fix compatibility issues with VERL 0.6.0. * Fix model name for pre-downloaded models in VERL. * Fix preparing status transition on rollout when creating attempts. * Fix OpenAI Agents SDK compatibility issues. **Full Changelog**: https://github.com/microsoft/agent-lightning/compare/v0.2.1...v0.2.2 * * * Agent-lightning v0.2.1 (10/30/2025)[¶](https://microsoft.github.io/agent-lightning/stable/changelog/#agent-lightning-v021-10302025 "Permanent link") ----------------------------------------------------------------------------------------------------------------------------------------------------- Agent-lightning v0.2.1 is a stabilization release for v0.2.0. It introduces several bug fixes and new features, plus a number of unlisted CI improvements. ### Bug fixes[¶](https://microsoft.github.io/agent-lightning/stable/changelog/#bug-fixes "Permanent link") * Fix LiteLLM issues when restarting the proxy multiple times in the same process (#174 #206) * Fix LiteLLM model name selection when multiple servers use the same model (#197) * Fix store port conflict handling (#227) ### New Features[¶](https://microsoft.github.io/agent-lightning/stable/changelog/#new-features "Permanent link") * Add trainer port option for client-server strategies (#198) ### Documentation[¶](https://microsoft.github.io/agent-lightning/stable/changelog/#documentation "Permanent link") * Add tutorial for launching workers on separate machines (#213) * Add link to VERL framework (#210) * Add link to vLLM blog (#215) * Fix a couple of typos and avoid emacs backup files (#237) ### New Contributors[¶](https://microsoft.github.io/agent-lightning/stable/changelog/#new-contributors_1 "Permanent link") A warm welcome to our first-time contributors: @scott-vsi, @ddsfda99, @jeis4wpi 🎉 **Full Changelog**: https://github.com/microsoft/agent-lightning/compare/v0.2.0...v0.2.1 * * * Agent-lightning v0.2.0 (10/22/2025)[¶](https://microsoft.github.io/agent-lightning/stable/changelog/#agent-lightning-v020-10222025 "Permanent link") ----------------------------------------------------------------------------------------------------------------------------------------------------- Agent-Lightning v0.2.0 introduces major framework improvements, new execution strategies, expanded documentation, and enhanced reliability across the agent training and deployment workflow. This release includes **78 pull requests** since v0.1.2. ### Core Enhancements[¶](https://microsoft.github.io/agent-lightning/stable/changelog/#core-enhancements "Permanent link") * **Lightning Store**: Added unified interface and implementation for Agent-lightning's core storage. * **Emitter**: Emitting any objects as spans to the store. * **Adapter** and **Tracer**: Adapting to OpenAI-like messages, and OpenTelemetry dummy tracer. * **LLM Proxy**: Added LLM Proxy as the first-class citizen in Agent-lightning. * **Agent Runner**: New version providing a more modular and robust runner design. * **Embedded Algorithms**: Algorithms are now embedded directly into trainers for simplicity. * **New Execution Strategies**: Introduced _Client-Server_ and _Shared Memory_ execution models. * **Trainer Updates**: Integrated v0.2 interfaces and FastAlgorithm validation. ### Documentation & Examples[¶](https://microsoft.github.io/agent-lightning/stable/changelog/#documentation-examples "Permanent link") * Revamped documentation with new guides for **agent creation**, **training**, **debugging**, and **store concepts**. * Improved quickstart tutorials, clarified installation and new deep-dive articles. * Added and updated examples: _SQL Agent_, _Calc-X_, _Local SFT_, _Search-R1_, and _APO algorithm_. ### Developer Experience[¶](https://microsoft.github.io/agent-lightning/stable/changelog/#developer-experience "Permanent link") * Migrated build and CI pipelines to **1ES**, split workflows and aggregate badges for clarity. * Adopted **uv** as the dependency manager. * Added GPU-based pytest workflows for full test coverage. * Enhanced debugging UX, pre-commit configs, and linting (Pyright fixes, import sorting). ### Ecosystem & Integrations[¶](https://microsoft.github.io/agent-lightning/stable/changelog/#ecosystem-integrations "Permanent link") * Added support for agents built with [**Agent-framework**](https://github.com/microsoft/agent-framework) . * Added new community listings: [_DeepWerewolf_](https://github.com/af-74413592/DeepWerewolf) and [_AgentFlow_](https://agentflow.stanford.edu/) . ### New Contributors[¶](https://microsoft.github.io/agent-lightning/stable/changelog/#new-contributors_2 "Permanent link") A warm welcome to our first-time contributors: @hzy46, @lunaqiu, @syeehyn, @linhx1999, @SiyunZhao, and @acured 🎉 **Full changelog:** [v0.1.2 → v0.2.0](https://github.com/microsoft/agent-lightning/compare/v0.1.2...v0.2.0) * * * Agent-lightning v0.1.2 (08/12/2025)[¶](https://microsoft.github.io/agent-lightning/stable/changelog/#agent-lightning-v012-08122025 "Permanent link") ----------------------------------------------------------------------------------------------------------------------------------------------------- ### What's Changed[¶](https://microsoft.github.io/agent-lightning/stable/changelog/#whats-changed "Permanent link") * Add basic documentation in https://github.com/microsoft/agent-lightning/pull/33 * RAG example by @wizardlancet in https://github.com/microsoft/agent-lightning/pull/21 ### New Contributors[¶](https://microsoft.github.io/agent-lightning/stable/changelog/#new-contributors_3 "Permanent link") * @wizardlancet made their first contribution in https://github.com/microsoft/agent-lightning/pull/21 **Full Changelog**: https://github.com/microsoft/agent-lightning/compare/v0.1.1...v0.1.2 * * * Agent-lightning v0.1.1 (08/06/2025)[¶](https://microsoft.github.io/agent-lightning/stable/changelog/#agent-lightning-v011-08062025 "Permanent link") ----------------------------------------------------------------------------------------------------------------------------------------------------- ### What's Changed[¶](https://microsoft.github.io/agent-lightning/stable/changelog/#whats-changed_1 "Permanent link") * Disable HTTP tracer tests and bump to 0.1.1 in https://github.com/microsoft/agent-lightning/pull/26 * Fix trainer bugs in v0.1 in https://github.com/microsoft/agent-lightning/pull/24 **Full Changelog**: https://github.com/microsoft/agent-lightning/compare/v0.1...v0.1.1 * * * Agent-lightning v0.1.0 (08/04/2025)[¶](https://microsoft.github.io/agent-lightning/stable/changelog/#agent-lightning-v010-08042025 "Permanent link") ----------------------------------------------------------------------------------------------------------------------------------------------------- The first release of Agent-lightning! * Turn your agent into an optimizable beast with **ZERO CODE CHANGE** (almost)! 💤 * Build with **ANY** agent framework (LangChain, OpenAI Agent SDK, AutoGen, CrewAI, ...); or even WITHOUT agent framework (Python OpenAI). You name it! 🤖 * **Selectively** optimize one or more agents in a multi-agent system. 🎯 * Embraces Reinforcement Learning, Automatic Prompt Optimization and more **algorithms**. 🤗 Install via `pip install agentlightning`. Back to top --- # Store - Agent-lightning [Skip to content](https://microsoft.github.io/agent-lightning/stable/reference/store/#store-references) ⚠️ You are viewing development documentation. For stable documentation, please visit [https://microsoft.github.io/agent-lightning/stable/](https://microsoft.github.io/agent-lightning/stable/) Store References[¶](https://microsoft.github.io/agent-lightning/stable/reference/store/#store-references "Permanent link") =========================================================================================================================== ### `agentlightning.LightningStore` [¶](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.LightningStore "Permanent link") Contract for the persistent control-plane that coordinates training rollouts. A `LightningStore` mediates every interaction between algorithms and runners: * **Rollout lifecycle:** accept new rollouts, queue them for execution, create attempts, and drive the rollout status machine (`"queuing"` → `"preparing"` → `"running"` → `{"succeeded","failed","cancelled"}` or `"requeuing"` when a retry is justified). * **Attempt tracking:** record each execution attempt, including progress heartbeats, retry sequencing, and terminal states such as `"timeout"` or `"unresponsive"`. * **Span ingest:** capture structured telemetry emitted by runners (either as native [`Span`](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.Span " agentlightning.Span") objects or as `opentelemetry.sdk.trace.ReadableSpan` instances) so that algorithms can reconstruct trajectories and rewards. * **Resource versioning:** manage immutable snapshots of named resources (prompt templates, model checkpoints, proxy endpoints, …) and expose a single "latest" snapshot that runners can fetch just after claiming work. Implementations must provide thread-safe/async-safe semantics: each coroutine should appear atomic to callers even when multiple algorithms or runners call the API concurrently. Unless stated otherwise, missing identifiers should result in a `ValueError`. #### `capabilities` `property` [¶](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.LightningStore.capabilities "Permanent link") Return the capabilities of the store. #### `add_many_spans(spans)` `async` [¶](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.LightningStore.add_many_spans "Permanent link") Persist a sequence of pre-constructed spans emitted during rollout execution. Implementations can simply delegate to [`add_span()`](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.LightningStore.add_span " add_span(span) async ") for each span. However, if the store supports bulk insertion, it can implement this method to improve performance. #### `add_otel_span(rollout_id, attempt_id, readable_span, sequence_id=None)` `async` [¶](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.LightningStore.add_otel_span "Permanent link") Convert and persist an OpenTelemetry span for a particular attempt. Implementations must transform the `readable_span` into a [`Span`](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.Span " agentlightning.Span") (typically via [`Span.from_opentelemetry()`](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.Span.from_opentelemetry " from_opentelemetry(src, rollout_id, attempt_id, sequence_id) classmethod ") ), assign a strictly increasing `sequence_id` when one is not provided, and persist it using the same semantics as [`add_span()`](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.LightningStore.add_span " add_span(span) async ") . Parameters: * **`rollout_id`** (`str`) – Identifier of the rollout that produced the span. * **`attempt_id`** (`str`) – Attempt identifier the span belongs to. * **`readable_span`** (`ReadableSpan`) – OpenTelemetry span in SDK form. * **`sequence_id`** (`int | None`, default: `None` ) – Optional explicit ordering hint. When omitted, call [`get_next_span_sequence_id()`](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.LightningStore.get_next_span_sequence_id " get_next_span_sequence_id(rollout_id, attempt_id) async ") automatically. Returns: * `Optional[[Span](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.Span " agentlightning.Span (agentlightning.types.Span)") ]` – The stored span record. Return `None` if the span was not added due to a duplicate. Raises: * `NotImplementedError` – Subclasses must implement span persistence. * `ValueError` – Implementations must raise when the rollout or attempt is unknown. #### `add_resources(resources)` `async` [¶](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.LightningStore.add_resources "Permanent link") Persist a new immutable snapshot of named resources and mark it as latest. Implementations must assign a fresh `resources_id` and ensure subsequent calls to [`get_latest_resources()`](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.LightningStore.get_latest_resources " get_latest_resources() async ") return the snapshot produced here. Parameters: * **`resources`** (`[NamedResources](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.NamedResources " agentlightning.NamedResources = Dict[str, ResourceUnion] module-attribute (agentlightning.types.NamedResources)") `) – Mapping of resource names to their serialized payloads. Returns: * `[ResourcesUpdate](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.ResourcesUpdate " agentlightning.ResourcesUpdate (agentlightning.types.ResourcesUpdate)") ` – The stored [`ResourcesUpdate`](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.ResourcesUpdate " agentlightning.ResourcesUpdate") including its generated id. Raises: * `NotImplementedError` – Subclasses must implement resource persistence. #### `add_span(span)` `async` [¶](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.LightningStore.add_span "Permanent link") Persist a pre-constructed span emitted during rollout execution. The provided [`Span`](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.Span " agentlightning.Span") must already contain the `rollout_id`, `attempt_id`, and `sequence_id`. Implementations must: * Verify that both rollout and attempt exist. * Ensure span ordering remains strictly increasing per attempt (rejecting or keeping duplicates). * Treat the span arrival as a heartbeat: update the attempt's `last_heartbeat_time` and transition both attempt and rollout to `"running"` if they were still `"preparing"` or `"requeuing"`. Parameters: * **`span`** (`[Span](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.Span " agentlightning.Span (agentlightning.types.Span)") `) – Fully populated span to persist. Returns: * `Optional[[Span](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.Span " agentlightning.Span (agentlightning.types.Span)") ]` – The stored span record (implementations may return a copy). * `Optional[[Span](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.Span " agentlightning.Span (agentlightning.types.Span)") ]` – Return `None` if the span was not added due to a duplicate. Raises: * `NotImplementedError` – Subclasses must implement span persistence. * `ValueError` – Implementations must raise when the referenced rollout or attempt is missing. #### `dequeue_many_rollouts(*, limit=1, worker_id=None)` `async` [¶](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.LightningStore.dequeue_many_rollouts "Permanent link") Claim up to `limit` queued rollouts without blocking. The implementation can repeatedly invokes [`dequeue_rollout()`](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.LightningStore.dequeue_rollout " dequeue_rollout(worker_id=None) async ") until reaching the requested limit or the queue is empty. Subclasses can override it to fetch multiple rollouts atomically. Parameters: * **`limit`** (`int`, default: `1` ) – Maximum number of rollouts to claim. Non-positive values return an empty list. * **`worker_id`** (`Optional[str]`, default: `None` ) – Optional worker identifier passed through to each dequeue call. Returns: * `Sequence[[AttemptedRollout](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.AttemptedRollout " agentlightning.AttemptedRollout (agentlightning.types.AttemptedRollout)") ]` – Attempted rollouts claimed in FIFO order. May contain fewer than `limit` entries * `Sequence[[AttemptedRollout](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.AttemptedRollout " agentlightning.AttemptedRollout (agentlightning.types.AttemptedRollout)") ]` – when the queue is exhausted. #### `dequeue_rollout(worker_id=None)` `async` [¶](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.LightningStore.dequeue_rollout "Permanent link") Claim the oldest queued rollout and transition it to `preparing`. This function do not block. Retrieval must be FIFO across rollouts that remain in `queuing` or `requeuing` state. When a rollout is claimed, implementations must: * Transition its status to `"preparing"`. * Create a new attempt with `status="preparing"` and `sequence_id` equal to the number of attempts already registered for the rollout plus one. * Return an [`AttemptedRollout`](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.AttemptedRollout " agentlightning.AttemptedRollout") snapshot so the runner knows both rollout metadata and the attempt identifier. * Optionally refresh the caller's [`Worker`](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.Worker " agentlightning.Worker") telemetry (e.g., `last_dequeue_time`) when `worker_id` is provided. Parameters: * **`worker_id`** (`Optional[str]`, default: `None` ) – Optional worker identifier to associate the claimed attempt with. Returns: * `Optional[[AttemptedRollout](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.AttemptedRollout " agentlightning.AttemptedRollout (agentlightning.types.AttemptedRollout)") ]` – The next attempt to execute, or `None` when no eligible rollouts are queued. Raises: * `NotImplementedError` – Subclasses must implement queue retrieval. #### `enqueue_many_rollouts(rollouts)` `async` [¶](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.LightningStore.enqueue_many_rollouts "Permanent link") Persist multiple rollouts in `queuing` state. The implementation can delegate to [`enqueue_rollout()`](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.LightningStore.enqueue_rollout " enqueue_rollout(input, mode=None, resources_id=None, config=None, metadata=None) async ") per request and preserves the input ordering. Subclasses can override to provide more efficient bulk enqueue semantics. Parameters: * **`rollouts`** (`Sequence[[EnqueueRolloutRequest](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.EnqueueRolloutRequest " agentlightning.EnqueueRolloutRequest (agentlightning.types.EnqueueRolloutRequest)") ]`) – Rollout submission payloads mirroring [`enqueue_rollout()`](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.LightningStore.enqueue_rollout " enqueue_rollout(input, mode=None, resources_id=None, config=None, metadata=None) async ") 's parameters. Each entry requires `input` and can optionally include other fields. Returns: * `Sequence[[Rollout](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.Rollout " agentlightning.Rollout (agentlightning.types.Rollout)") ]` – Rollouts enqueued in the same order as `rollouts`. #### `enqueue_rollout(input, mode=None, resources_id=None, config=None, metadata=None)` `async` [¶](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.LightningStore.enqueue_rollout "Permanent link") Persist a rollout in `queuing` state so runners can claim it later. Note Different from [`start_rollout()`](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.LightningStore.start_rollout " start_rollout(input, mode=None, resources_id=None, config=None, metadata=None, worker_id=None) async ") , this method is called when the caller only wants to submit work for later scheduling. Implementations must generate a unique `rollout_id`, stamp `start_time` with the current time, default `config` to a fresh [`RolloutConfig`](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.RolloutConfig " agentlightning.RolloutConfig") , and insert the rollout at the tail of the scheduling queue. No attempt is created yet. Parameters: * **`input`** (`[TaskInput](https://microsoft.github.io/agent-lightning/stable/reference/internal/#agentlightning.TaskInput " agentlightning.TaskInput = Any module-attribute (agentlightning.types.TaskInput)") `) – Arbitrary task payload supplied by an algorithm. * **`mode`** (`Literal['train', 'val', 'test'] | None`, default: `None` ) – Optional semantic mode indicator (`"train"`, `"val"`, `"test"`). * **`resources_id`** (`str | None`, default: `None` ) – Resource snapshot used when a runner eventually executes the rollout. * **`config`** (`[RolloutConfig](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.RolloutConfig " agentlightning.RolloutConfig (agentlightning.types.RolloutConfig)") | None`, default: `None` ) – Fine-grained retry/timeout parameters to persist with the rollout. * **`metadata`** (`Dict[str, Any] | None`, default: `None` ) – Free-form metadata stored verbatim with the rollout record. Returns: * `[Rollout](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.Rollout " agentlightning.Rollout (agentlightning.types.Rollout)") ` – The stored [`Rollout`](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.Rollout " agentlightning.Rollout") in `queuing` status. Raises: * `NotImplementedError` – Subclasses must persist the rollout. * `ValueError` – Implementations should raise when `resources_id` does not exist. #### `get_latest_attempt(rollout_id)` `async` [¶](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.LightningStore.get_latest_attempt "Permanent link") Fetch the attempt with the highest `sequence_id` for `rollout_id`. Parameters: * **`rollout_id`** (`str`) – Identifier to inspect. Returns: * `Optional[[Attempt](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.Attempt " agentlightning.Attempt (agentlightning.types.Attempt)") ]` – The most recent attempt or `None` when no attempts exist yet. Raises: * `NotImplementedError` – Subclasses must implement retrieval. * `ValueError` – Implementations must raise when the rollout does not exist. #### `get_latest_resources()` `async` [¶](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.LightningStore.get_latest_resources "Permanent link") Fetch the latest resource snapshot marked as the global default. Returns: * `Optional[[ResourcesUpdate](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.ResourcesUpdate " agentlightning.ResourcesUpdate (agentlightning.types.ResourcesUpdate)") ]` – The current latest [`ResourcesUpdate`](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.ResourcesUpdate " agentlightning.ResourcesUpdate") , or `None` when * `Optional[[ResourcesUpdate](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.ResourcesUpdate " agentlightning.ResourcesUpdate (agentlightning.types.ResourcesUpdate)") ]` – no resources have been registered yet. Raises: * `NotImplementedError` – Subclasses must implement retrieval. #### `get_many_span_sequence_ids(rollout_attempt_ids)` `async` [¶](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.LightningStore.get_many_span_sequence_ids "Permanent link") Bulk allocate the next strictly increasing sequence number used to order spans. Implementations may delegate to [`get_next_span_sequence_id()`](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.LightningStore.get_next_span_sequence_id " get_next_span_sequence_id(rollout_id, attempt_id) async ") for each rollout and attempt. Parameters: * **`rollout_attempt_ids`** (`Sequence[Tuple[str, str]]`) – List of tuples of rollout and attempt identifiers. Returns: * `Sequence[int]` – List of sequence numbers. #### `get_next_span_sequence_id(rollout_id, attempt_id)` `async` [¶](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.LightningStore.get_next_span_sequence_id "Permanent link") Allocate the next strictly increasing sequence number used to order spans. Implementations must retain counters so repeated calls return `1, 2, ...` without gaps unless spans were explicitly inserted with a custom `sequence_id`. The counter may be scoped per rollout or per attempt, but the sequence must be strictly increasing for spans emitted by the specified attempt so traces remain totally ordered. See [Distributed Tracing](https://microsoft.github.io/agent-lightning/stable/tutorials/traces/#distributed-tracing "LLM Proxy") for detailed motivations. Parameters: * **`rollout_id`** (`str`) – Identifier of the rollout emitting spans. * **`attempt_id`** (`str`) – Attempt identifier for the upcoming span. Returns: * `int` – The next integer sequence identifier, unique within the attempt. Raises: * `NotImplementedError` – Subclasses must provide the allocator. * `ValueError` – Implementations must raise when the rollout or attempt does not exist. #### `get_resources_by_id(resources_id)` `async` [¶](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.LightningStore.get_resources_by_id "Permanent link") Return a specific named resource snapshot by identifier. Parameters: * **`resources_id`** (`str`) – Identifier of the snapshot. Returns: * `Optional[[ResourcesUpdate](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.ResourcesUpdate " agentlightning.ResourcesUpdate (agentlightning.types.ResourcesUpdate)") ]` – The stored [`ResourcesUpdate`](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.ResourcesUpdate " agentlightning.ResourcesUpdate") , or `None` when missing. Raises: * `NotImplementedError` – Subclasses must implement retrieval. #### `get_rollout_by_id(rollout_id)` `async` [¶](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.LightningStore.get_rollout_by_id "Permanent link") Fetch a rollout by identifier without mutating its state. Parameters: * **`rollout_id`** (`str`) – Identifier to retrieve. Returns: * `Optional[[Rollout](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.Rollout " agentlightning.Rollout (agentlightning.types.Rollout)") ]` – The rollout when found, otherwise `None`. Raises: * `NotImplementedError` – Subclasses must implement retrieval. #### `get_worker_by_id(worker_id)` `async` [¶](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.LightningStore.get_worker_by_id "Permanent link") Retrieve a single worker by identifier. Parameters: * **`worker_id`** (`str`) – Identifier of the worker. Returns: * `Optional[[Worker](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.Worker " agentlightning.Worker (agentlightning.types.Worker)") ]` – The worker record if it exists, otherwise `None`. Raises: * `NotImplementedError` – Subclasses must implement lookup semantics. #### `otlp_traces_endpoint()` [¶](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.LightningStore.otlp_traces_endpoint "Permanent link") Return the OTLP/HTTP traces endpoint of the store. The traces can have rollout ID and attempt ID (and optionally sequence ID) saved in the "resource" of the spans. The store, if it supports OTLP, should be able to receive the traces and save them via [`add_span`](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.LightningStore.add_span " add_span(span) async ") or [`add_otel_span`](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.LightningStore.add_otel_span " add_otel_span(rollout_id, attempt_id, readable_span, sequence_id=None) async ") . The endpoint should be compatible with [OTLP HTTP protocol](https://opentelemetry.io/docs/specs/otlp/) . It's not necessarily compatible with OTLP gRPC protocol. The returned endpoint will usually ends with `/v1/traces`. #### `query_attempts(rollout_id, *, sort_by='sequence_id', sort_order='asc', limit=-1, offset=0)` `async` [¶](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.LightningStore.query_attempts "Permanent link") Return every attempt ever created for `rollout_id` in ascending sequence order. The parameters allow callers to re-order or paginate the attempts so that large retry histories can be streamed lazily. Parameters: * **`rollout_id`** (`str`) – Identifier of the rollout being inspected. * **`sort_by`** (`Optional[str]`, default: `'sequence_id'` ) – Field to sort by. Must be a numeric or string field of [`Attempt`](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.Attempt " agentlightning.Attempt") . Defaults to `sequence_id` (oldest first). * **`sort_order`** (`Literal['asc', 'desc']`, default: `'asc'` ) – Order to sort by. * **`limit`** (`int`, default: `-1` ) – Limit on the number of results. `-1` for unlimited. * **`offset`** (`int`, default: `0` ) – Offset into the results. Returns: * `Sequence[[Attempt](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.Attempt " agentlightning.Attempt (agentlightning.types.Attempt)") ]` – Sequence of Attempts. Returns an empty sequence when none exist. * `Sequence[[Attempt](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.Attempt " agentlightning.Attempt (agentlightning.types.Attempt)") ]` – The return value is not guaranteed to be a list. Raises: * `NotImplementedError` – Subclasses must implement the query. * `ValueError` – Implementations must raise when the rollout does not exist. #### `query_resources(*, resources_id=None, resources_id_contains=None, sort_by=None, sort_order='asc', limit=-1, offset=0)` `async` [¶](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.LightningStore.query_resources "Permanent link") List every stored resource snapshot in insertion order. Supports lightweight filtering, sorting, and pagination for embedding in dashboards. Parameters: * **`resources_id`** (`Optional[str]`, default: `None` ) – Optional identifier of the resources to include. * **`resources_id_contains`** (`Optional[str]`, default: `None` ) – Optional substring match for resources identifiers. * **`sort_by`** (`Optional[str]`, default: `None` ) – Optional field to sort by (must be numeric or string on [`ResourcesUpdate`](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.ResourcesUpdate " agentlightning.ResourcesUpdate") ). * **`sort_order`** (`Literal['asc', 'desc']`, default: `'asc'` ) – Order to sort by. * **`limit`** (`int`, default: `-1` ) – Limit on the number of results. `-1` for unlimited. * **`offset`** (`int`, default: `0` ) – Offset into the results. Returns: * `Sequence[[ResourcesUpdate](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.ResourcesUpdate " agentlightning.ResourcesUpdate (agentlightning.types.ResourcesUpdate)") ]` – [`ResourcesUpdate`](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.ResourcesUpdate " agentlightning.ResourcesUpdate") objects. * `Sequence[[ResourcesUpdate](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.ResourcesUpdate " agentlightning.ResourcesUpdate (agentlightning.types.ResourcesUpdate)") ]` – By default, resources are sorted in a deterministic but undefined order. * `Sequence[[ResourcesUpdate](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.ResourcesUpdate " agentlightning.ResourcesUpdate (agentlightning.types.ResourcesUpdate)") ]` – The return value is not guaranteed to be a list. Raises: * `NotImplementedError` – Subclasses must implement retrieval. #### `query_rollouts(*, status_in=None, rollout_id_in=None, rollout_id_contains=None, filter_logic='and', sort_by=None, sort_order='asc', limit=-1, offset=0, status=None, rollout_ids=None)` `async` [¶](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.LightningStore.query_rollouts "Permanent link") Retrieve rollouts filtered by status and/or explicit identifiers. This interface supports structured filtering, sorting, and pagination so callers can build simple dashboards without copying data out of the store. The legacy parameters `status` and `rollout_ids` remain valid and are treated as aliases for `status_in` and `rollout_id_in` respectively—when both the new and deprecated parameters are supplied the new parameters take precedence. Parameters: * **`status_in`** (`Optional[Sequence[[RolloutStatus](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.RolloutStatus " agentlightning.RolloutStatus = Literal['queuing', 'preparing', 'running', 'failed', 'succeeded', 'cancelled', 'requeuing'] module-attribute (agentlightning.types.RolloutStatus)") ]]`, default: `None` ) – Optional whitelist of [`RolloutStatus`](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.RolloutStatus " agentlightning.RolloutStatus = Literal['queuing', 'preparing', 'running', 'failed', 'succeeded', 'cancelled', 'requeuing'] module-attribute ") values. * **`rollout_id_in`** (`Optional[Sequence[str]]`, default: `None` ) – Optional whitelist of rollout identifiers to include. * **`rollout_id_contains`** (`Optional[str]`, default: `None` ) – Optional substring match for rollout identifiers. * **`filter_logic`** (`Literal['and', 'or']`, default: `'and'` ) – Logical operator to combine filters. * **`sort_by`** (`Optional[str]`, default: `None` ) – Optional field to sort by. Must reference a numeric or string field on [`Rollout`](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.Rollout " agentlightning.Rollout") . * **`sort_order`** (`Literal['asc', 'desc']`, default: `'asc'` ) – Direction to sort when `sort_by` is provided. * **`limit`** (`int`, default: `-1` ) – Maximum number of rows to return. Use `-1` for "no limit". * **`offset`** (`int`, default: `0` ) – Number of rows to skip before returning results. * **`status`** (`Optional[Sequence[[RolloutStatus](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.RolloutStatus " agentlightning.RolloutStatus = Literal['queuing', 'preparing', 'running', 'failed', 'succeeded', 'cancelled', 'requeuing'] module-attribute (agentlightning.types.RolloutStatus)") ]]`, default: `None` ) – Deprecated field. Use `status_in` instead. * **`rollout_ids`** (`Optional[Sequence[str]]`, default: `None` ) – Deprecated field. Use `rollout_id_in` instead. Returns: * `Sequence[[Rollout](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.Rollout " agentlightning.Rollout (agentlightning.types.Rollout)") ]` – A sequence of matching rollouts (or [`AttemptedRollout`](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.AttemptedRollout " agentlightning.AttemptedRollout") * `Sequence[[Rollout](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.Rollout " agentlightning.Rollout (agentlightning.types.Rollout)") ]` – when attempts exist). Ordering is deterministic when `sort_by` is set. * `Sequence[[Rollout](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.Rollout " agentlightning.Rollout (agentlightning.types.Rollout)") ]` – The return value is not guaranteed to be a list. Raises: * `NotImplementedError` – Subclasses must implement the query. #### `query_spans(rollout_id, attempt_id=None, *, trace_id=None, trace_id_contains=None, span_id=None, span_id_contains=None, parent_id=None, parent_id_contains=None, name=None, name_contains=None, filter_logic='and', limit=-1, offset=0, sort_by='sequence_id', sort_order='asc')` `async` [¶](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.LightningStore.query_spans "Permanent link") Return the stored spans for a rollout, optionally scoped to one attempt. Supports a handful of filters that cover the most common debugging scenarios (matching `trace_id`/`span_id`/`parent_id` or substring matches on the span name). `attempt_id="latest"` acts as a convenience that resolves the most recent attempt before evaluating filters. When `attempt_id=None`, spans across every attempt are eligible. By default results are sorted by `sequence_id` (oldest first). Implementations may raise a `RuntimeError` when spans were evicted or expired. Parameters: * **`rollout_id`** (`str`) – Identifier of the rollout being inspected. * **`attempt_id`** (`str | Literal['latest'] | None`, default: `None` ) – Attempt identifier to filter by. Pass `"latest"` to retrieve only the most recent attempt, or `None` to return all spans across attempts. * **`trace_id`** (`Optional[str]`, default: `None` ) – Optional trace ID to filter by. * **`trace_id_contains`** (`Optional[str]`, default: `None` ) – Optional substring match for trace IDs. * **`span_id`** (`Optional[str]`, default: `None` ) – Optional span ID to filter by. * **`span_id_contains`** (`Optional[str]`, default: `None` ) – Optional substring match for span IDs. * **`parent_id`** (`Optional[str]`, default: `None` ) – Optional parent span ID to filter by. * **`parent_id_contains`** (`Optional[str]`, default: `None` ) – Optional substring match for parent span IDs. * **`name`** (`Optional[str]`, default: `None` ) – Optional span name to filter by. * **`name_contains`** (`Optional[str]`, default: `None` ) – Optional substring match for span names. * **`filter_logic`** (`Literal['and', 'or']`, default: `'and'` ) – Logical operator to combine the optional filters above. The `rollout_id` argument is always applied with AND semantics. * **`limit`** (`int`, default: `-1` ) – Limit on the number of results. `-1` for unlimited. * **`offset`** (`int`, default: `0` ) – Offset into the results. * **`sort_by`** (`Optional[str]`, default: `'sequence_id'` ) – Field to sort by. Must be a numeric or string field of [`Span`](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.Span " agentlightning.Span") . * **`sort_order`** (`Literal['asc', 'desc']`, default: `'asc'` ) – Order to sort by. Returns: * `Sequence[[Span](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.Span " agentlightning.Span (agentlightning.types.Span)") ]` – An ordered list of spans (possibly empty). * `Sequence[[Span](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.Span " agentlightning.Span (agentlightning.types.Span)") ]` – The return value is not guaranteed to be a list. Raises: * `NotImplementedError` – Subclasses must implement the query. * `ValueError` – Implementations must raise when the rollout or attempt is unknown. #### `query_workers(*, status_in=None, worker_id_contains=None, filter_logic='and', sort_by=None, sort_order='asc', limit=-1, offset=0)` `async` [¶](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.LightningStore.query_workers "Permanent link") Query all workers in the system. Parameters: * **`status_in`** (`Optional[Sequence[[WorkerStatus](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.WorkerStatus " agentlightning.WorkerStatus = Literal['idle', 'busy', 'unknown'] module-attribute (agentlightning.types.WorkerStatus)") ]]`, default: `None` ) – Optional whitelist of [`WorkerStatus`](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.WorkerStatus " agentlightning.WorkerStatus = Literal['idle', 'busy', 'unknown'] module-attribute ") values. * **`worker_id_contains`** (`Optional[str]`, default: `None` ) – Optional substring match for worker identifiers. * **`filter_logic`** (`Literal['and', 'or']`, default: `'and'` ) – Logical operator to combine the optional filters above. * **`sort_by`** (`Optional[str]`, default: `None` ) – Field to sort by. Must be a numeric or string field of [`Worker`](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.Worker " agentlightning.Worker") . * **`sort_order`** (`Literal['asc', 'desc']`, default: `'asc'` ) – Order to sort by. * **`limit`** (`int`, default: `-1` ) – Limit on the number of results. `-1` for unlimited. * **`offset`** (`int`, default: `0` ) – Offset into the results. Returns: * `Sequence[[Worker](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.Worker " agentlightning.Worker (agentlightning.types.Worker)") ]` – Sequence of Workers. Returns an empty sequence when none exist. * `Sequence[[Worker](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.Worker " agentlightning.Worker (agentlightning.types.Worker)") ]` – The return value is not guaranteed to be a list. #### `start_attempt(rollout_id, worker_id=None)` `async` [¶](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.LightningStore.start_attempt "Permanent link") Create a manual retry attempt for an existing rollout. This is typically invoked by runners that wish to retry outside of the normal queue flow (for example in an online RL setup). Implementations must validate that the rollout exists, allocate a fresh `attempt_id`, increment the `sequence_id` monotonically, stamp the new attempt with `status="preparing"`, and return an up-to-date [`AttemptedRollout`](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.AttemptedRollout " agentlightning.AttemptedRollout") . Parameters: * **`rollout_id`** (`str`) – Unique identifier of the rollout receiving a new attempt. * **`worker_id`** (`Optional[str]`, default: `None` ) – Optional worker identifier to associate the new attempt with. Returns: * `[AttemptedRollout](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.AttemptedRollout " agentlightning.AttemptedRollout (agentlightning.types.AttemptedRollout)") ` – The rollout paired with its newly-created attempt. Raises: * `NotImplementedError` – Subclasses must implement attempt creation. * `ValueError` – Implementations must raise when `rollout_id` is unknown. #### `start_rollout(input, mode=None, resources_id=None, config=None, metadata=None, worker_id=None)` `async` [¶](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.LightningStore.start_rollout "Permanent link") Register a rollout and immediately create its first attempt. Note Use [`enqueue_rollout()`](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.LightningStore.enqueue_rollout " enqueue_rollout(input, mode=None, resources_id=None, config=None, metadata=None) async ") when the caller only wants to submit work for later scheduling. The rollout must be persisted with `status="preparing"` and an initial attempt with `sequence_id == 1` so the caller can begin execution without visiting the public queue. Implementations are expected to: 1. Generate a unique `rollout_id` and `attempt_id`. 2. Record `start_time` for both rollout and attempt based on the current clock. 3. Copy `config` and `metadata` so later mutations do not leak shared references. 4. Resolve `resources_id` to the latest resource snapshot when `None` is supplied. Parameters: * **`input`** (`[TaskInput](https://microsoft.github.io/agent-lightning/stable/reference/internal/#agentlightning.TaskInput " agentlightning.TaskInput = Any module-attribute (agentlightning.types.TaskInput)") `) – Arbitrary task payload supplied by an algorithm. * **`mode`** (`[RolloutMode](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.RolloutMode " agentlightning.RolloutMode = Literal['train', 'val', 'test'] module-attribute (agentlightning.types.RolloutMode)") | None`, default: `None` ) – Optional semantic mode for downstream analytics (`"train"`, `"val"`, `"test"`). * **`resources_id`** (`str | None`, default: `None` ) – Concrete resource snapshot to execute against; defaults to the latest stored snapshot. * **`config`** (`[RolloutConfig](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.RolloutConfig " agentlightning.RolloutConfig (agentlightning.types.RolloutConfig)") | None`, default: `None` ) – Rollout retry/timeout policy. Should default to a fresh [`RolloutConfig`](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.RolloutConfig " agentlightning.RolloutConfig") . * **`metadata`** (`Dict[str, Any] | None`, default: `None` ) – Free-form metadata persisted verbatim with the rollout. * **`worker_id`** (`str | None`, default: `None` ) – Optional worker identifier to associate the new attempt with. Returns: * `[AttemptedRollout](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.AttemptedRollout " agentlightning.AttemptedRollout (agentlightning.types.AttemptedRollout)") ` – The fully-populated [`AttemptedRollout`](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.AttemptedRollout " agentlightning.AttemptedRollout") including * `[AttemptedRollout](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.AttemptedRollout " agentlightning.AttemptedRollout (agentlightning.types.AttemptedRollout)") ` – the just-created attempt. Raises: * `NotImplementedError` – Subclasses must provide durable storage for the rollout. * `ValueError` – Implementations should raise when `resources_id` does not exist. #### `statistics()` `async` [¶](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.LightningStore.statistics "Permanent link") Return the statistics of the store. #### `update_attempt(rollout_id, attempt_id, status=UNSET, worker_id=UNSET, last_heartbeat_time=UNSET, metadata=UNSET)` `async` [¶](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.LightningStore.update_attempt "Permanent link") Update attempt bookkeeping such as status, worker ownership, and heartbeats. When `attempt_id` is `"latest"` the update must target the attempt with the highest `sequence_id`; otherwise it must target the specific attempt. Implementations should propagate status changes to the rollout (for example via [`rollout_status_from_attempt()`](https://microsoft.github.io/agent-lightning/stable/reference/internal/#agentlightning.store.utils.rollout_status_from_attempt " agentlightning.store.utils.rollout_status_from_attempt(attempt, config) async ") ) once the latest attempt transitions to a terminal state. Similar to [`update_rollout()`](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.LightningStore.update_rollout " update_rollout(rollout_id, input=UNSET, mode=UNSET, resources_id=UNSET, status=UNSET, config=UNSET, metadata=UNSET) async ") , parameters also default to the sentinel [`UNSET`](https://microsoft.github.io/agent-lightning/stable/reference/internal/#agentlightning.store.base.UNSET " agentlightning.store.base.UNSET = _UnsetType() module-attribute ") . If `worker_id` is present, the worker status will be updated following the rules: 1. If attempt status is "succeeded" or "failed", the corresponding worker status will be set to "idle". 2. If attempt status is "unresponsive" or "timeout", the corresponding worker status will be set to "unknown". 3. Otherwise, the worker status will be set to "busy". Parameters: * **`rollout_id`** (`str`) – Identifier of the rollout whose attempt will be updated. * **`attempt_id`** (`str | Literal['latest']`) – Attempt identifier or `"latest"` as a convenience. * **`status`** (`[AttemptStatus](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.AttemptStatus " agentlightning.AttemptStatus = Literal['preparing', 'running', 'failed', 'succeeded', 'unresponsive', 'timeout'] module-attribute (agentlightning.types.AttemptStatus)") | Unset`, default: `[UNSET](https://microsoft.github.io/agent-lightning/stable/reference/internal/#agentlightning.store.base.UNSET " agentlightning.store.base.UNSET = _UnsetType() module-attribute (agentlightning.store.base.UNSET)") ` ) – Replacement attempt status. Terminal statuses must set `end_time`. * **`worker_id`** (`str | Unset`, default: `[UNSET](https://microsoft.github.io/agent-lightning/stable/reference/internal/#agentlightning.store.base.UNSET " agentlightning.store.base.UNSET = _UnsetType() module-attribute (agentlightning.store.base.UNSET)") ` ) – Identifier for the worker currently processing the attempt. * **`last_heartbeat_time`** (`float | Unset`, default: `[UNSET](https://microsoft.github.io/agent-lightning/stable/reference/internal/#agentlightning.store.base.UNSET " agentlightning.store.base.UNSET = _UnsetType() module-attribute (agentlightning.store.base.UNSET)") ` ) – Wall-clock timestamp (seconds) of the latest heartbeat/span. * **`metadata`** (`Optional[Dict[str, Any]] | Unset`, default: `[UNSET](https://microsoft.github.io/agent-lightning/stable/reference/internal/#agentlightning.store.base.UNSET " agentlightning.store.base.UNSET = _UnsetType() module-attribute (agentlightning.store.base.UNSET)") ` ) – Replacement metadata dictionary. Returns: * `[Attempt](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.Attempt " agentlightning.Attempt (agentlightning.types.Attempt)") ` – The updated attempt record. Raises: * `NotImplementedError` – Subclasses must implement mutation logic. * `ValueError` – Implementations must raise when the rollout or attempt is unknown. #### `update_resources(resources_id, resources)` `async` [¶](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.LightningStore.update_resources "Permanent link") Overwrite or extend an existing resource snapshot and mark it as latest. This API is typically used by algorithms that maintain mutable resources (e.g., model checkpoints) under a stable identifier. Parameters: * **`resources_id`** (`str`) – Identifier of the snapshot to replace. * **`resources`** (`[NamedResources](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.NamedResources " agentlightning.NamedResources = Dict[str, ResourceUnion] module-attribute (agentlightning.types.NamedResources)") `) – Updated mapping of resource names to payloads. Returns: * `[ResourcesUpdate](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.ResourcesUpdate " agentlightning.ResourcesUpdate (agentlightning.types.ResourcesUpdate)") ` – The persisted [`ResourcesUpdate`](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.ResourcesUpdate " agentlightning.ResourcesUpdate") . Raises: * `NotImplementedError` – Subclasses must implement resource persistence. * `ValueError` – Implementations must raise when `resources_id` does not exist. #### `update_rollout(rollout_id, input=UNSET, mode=UNSET, resources_id=UNSET, status=UNSET, config=UNSET, metadata=UNSET)` `async` [¶](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.LightningStore.update_rollout "Permanent link") Update rollout metadata and, when provided, drive status transitions. Parameters default to the sentinel [`UNSET`](https://microsoft.github.io/agent-lightning/stable/reference/internal/#agentlightning.store.base.UNSET " agentlightning.store.base.UNSET = _UnsetType() module-attribute ") to distinguish omitted fields from explicit `None` assignments. Implementations must: * Validate the rollout exists before mutating it. * Replace each property when a concrete value (including `None`) is supplied. * When the status switches into a terminal state, set `end_time` and signal any waiters. * When the status re-enters a queueing state, ensure the rollout is enqueued exactly once. Parameters: * **`rollout_id`** (`str`) – Identifier of the rollout to update. * **`input`** (`[TaskInput](https://microsoft.github.io/agent-lightning/stable/reference/internal/#agentlightning.TaskInput " agentlightning.TaskInput = Any module-attribute (agentlightning.types.TaskInput)") | Unset`, default: `[UNSET](https://microsoft.github.io/agent-lightning/stable/reference/internal/#agentlightning.store.base.UNSET " agentlightning.store.base.UNSET = _UnsetType() module-attribute (agentlightning.store.base.UNSET)") ` ) – Replacement task payload; pass `None` to explicitly clear the input. * **`mode`** (`Optional[Literal['train', 'val', 'test']] | Unset`, default: `[UNSET](https://microsoft.github.io/agent-lightning/stable/reference/internal/#agentlightning.store.base.UNSET " agentlightning.store.base.UNSET = _UnsetType() module-attribute (agentlightning.store.base.UNSET)") ` ) – Replacement rollout mode. * **`resources_id`** (`Optional[str] | Unset`, default: `[UNSET](https://microsoft.github.io/agent-lightning/stable/reference/internal/#agentlightning.store.base.UNSET " agentlightning.store.base.UNSET = _UnsetType() module-attribute (agentlightning.store.base.UNSET)") ` ) – Replacement resources snapshot reference. * **`status`** (`[RolloutStatus](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.RolloutStatus " agentlightning.RolloutStatus = Literal['queuing', 'preparing', 'running', 'failed', 'succeeded', 'cancelled', 'requeuing'] module-attribute (agentlightning.types.RolloutStatus)") | Unset`, default: `[UNSET](https://microsoft.github.io/agent-lightning/stable/reference/internal/#agentlightning.store.base.UNSET " agentlightning.store.base.UNSET = _UnsetType() module-attribute (agentlightning.store.base.UNSET)") ` ) – Target rollout status. * **`config`** (`[RolloutConfig](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.RolloutConfig " agentlightning.RolloutConfig (agentlightning.types.RolloutConfig)") | Unset`, default: `[UNSET](https://microsoft.github.io/agent-lightning/stable/reference/internal/#agentlightning.store.base.UNSET " agentlightning.store.base.UNSET = _UnsetType() module-attribute (agentlightning.store.base.UNSET)") ` ) – Replacement retry/timeout configuration. * **`metadata`** (`Optional[Dict[str, Any]] | Unset`, default: `[UNSET](https://microsoft.github.io/agent-lightning/stable/reference/internal/#agentlightning.store.base.UNSET " agentlightning.store.base.UNSET = _UnsetType() module-attribute (agentlightning.store.base.UNSET)") ` ) – Replacement metadata dictionary. Returns: * `[Rollout](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.Rollout " agentlightning.Rollout (agentlightning.types.Rollout)") ` – The updated rollout record. Raises: * `NotImplementedError` – Subclasses must implement mutation logic. * `ValueError` – Implementations must raise when the rollout is unknown or the update is invalid. #### `update_worker(worker_id, heartbeat_stats=UNSET)` `async` [¶](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.LightningStore.update_worker "Permanent link") Record a heartbeat for `worker_id` and refresh telemetry. Implementations must treat this API as heartbeat-only: it should snapshot the latest stats when provided, stamp `last_heartbeat_time` with the current wall clock, and rely on other store mutations (`dequeue_rollout`, `update_attempt`, etc.) to drive the worker's busy/idle status, assignment, and activity timestamps. Parameters: * **`worker_id`** (`str`) – Identifier of the worker to update. * **`heartbeat_stats`** (`Dict[str, Any] | Unset`, default: `[UNSET](https://microsoft.github.io/agent-lightning/stable/reference/internal/#agentlightning.store.base.UNSET " agentlightning.store.base.UNSET = _UnsetType() module-attribute (agentlightning.store.base.UNSET)") ` ) – Replacement worker heartbeat statistics (non-null when provided). #### `wait_for_rollouts(*, rollout_ids, timeout=None)` `async` [¶](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.LightningStore.wait_for_rollouts "Permanent link") Block until the targeted rollouts reach a terminal status or the timeout expires. Terminal statuses are `"succeeded"`, `"failed"`, and `"cancelled"`. When the timeout elapses, implementations should return the subset of rollouts that are already terminal and omit the rest. Warning It's dangerous and might be event-loop blocking to call this function with a long timeout. It's a good idea to poll for the method to check if new completed rollouts can coming. Be careful in implementing the sleep logic to avoid busy-waiting. Parameters: * **`rollout_ids`** (`List[str]`) – Identifiers of rollouts to watch. * **`timeout`** (`Optional[float]`, default: `None` ) – Maximum time in seconds to wait. `None` waits indefinitely. Returns: * `List[[Rollout](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.Rollout " agentlightning.Rollout (agentlightning.types.Rollout)") ]` – Rollouts that finished before the deadline, in arbitrary order. Raises: * `NotImplementedError` – Subclasses must implement waiting semantics. * `ValueError` – Implementations must raise when a rollout identifier is unknown. ### `agentlightning.LightningStoreCapabilities` [¶](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.LightningStoreCapabilities "Permanent link") Bases: `TypedDict` Capability of a LightningStore implementation. All keys are optional and false by default. #### `async_safe` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.LightningStoreCapabilities.async_safe "Permanent link") Whether the store is async-safe. #### `otlp_traces` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.LightningStoreCapabilities.otlp_traces "Permanent link") Whether the store supports OTLP/HTTP traces. #### `thread_safe` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.LightningStoreCapabilities.thread_safe "Permanent link") Whether the store is thread-safe. #### `zero_copy` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.LightningStoreCapabilities.zero_copy "Permanent link") Whether the store has only one copy across all threads/processes. Store Implementations[¶](https://microsoft.github.io/agent-lightning/stable/reference/store/#store-implementations "Permanent link") ------------------------------------------------------------------------------------------------------------------------------------- ### `agentlightning.InMemoryLightningStore` [¶](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.InMemoryLightningStore "Permanent link") Bases: `[CollectionBasedLightningStore](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.CollectionBasedLightningStore " agentlightning.CollectionBasedLightningStore (agentlightning.store.collection_based.CollectionBasedLightningStore)") [[InMemoryLightningCollections](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.store.collection.InMemoryLightningCollections " agentlightning.store.collection.InMemoryLightningCollections") ]` In-memory implementation of LightningStore using Python data structures. Thread-safe and async-compatible but data is not persistent. Parameters: * **`thread_safe`** (`bool`, default: `False` ) – Whether the store is thread-safe. * **`eviction_memory_threshold`** (`float | int | None`, default: `None` ) – The threshold for evicting spans in bytes. By default, it's 70% of the total VRAM available. * **`safe_memory_threshold`** (`float | int | None`, default: `None` ) – The threshold for safe memory usage in bytes. By default, it's 80% of the eviction threshold. * **`span_size_estimator`** (`Callable[[[Span](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.Span " agentlightning.Span (agentlightning.types.Span)") ], int] | None`, default: `None` ) – A function to estimate the size of a span in bytes. By default, it's a simple size estimator that uses sys.getsizeof. * **`tracker`** (`[MetricsBackend](https://microsoft.github.io/agent-lightning/stable/reference/utilities/#agentlightning.utils.metrics.MetricsBackend " agentlightning.utils.metrics.MetricsBackend") | None`, default: `None` ) – The metrics tracker to use. * **`scan_debounce_seconds`** (`float`, default: `10.0` ) – The debounce time for the scan for unhealthy rollouts. Set to 0 to disable debouncing. #### `capabilities` `property` [¶](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.InMemoryLightningStore.capabilities "Permanent link") Return the capabilities of the store. #### `statistics()` `async` [¶](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.InMemoryLightningStore.statistics "Permanent link") Return the statistics of the store. #### `wait_for_rollout(rollout_id, timeout=None)` `async` [¶](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.InMemoryLightningStore.wait_for_rollout "Permanent link") Wait for a specific rollout to complete with a timeout. ### `agentlightning.store.mongo.MongoLightningStore` [¶](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.store.mongo.MongoLightningStore "Permanent link") Bases: `[CollectionBasedLightningStore](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.CollectionBasedLightningStore " agentlightning.CollectionBasedLightningStore (agentlightning.store.collection_based.CollectionBasedLightningStore)") [[MongoLightningCollections](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.store.collection.mongo.MongoLightningCollections " agentlightning.store.collection.mongo.MongoLightningCollections") ]` MongoDB implementation of LightningStore using MongoDB collections. Data is persistent and can be shared between multiple processes. Parameters: * **`mongo_uri`** (`str`, default: `'mongodb://localhost:27017/?replicaSet=rs0'` ) – MongoDB connection string (defaults to local replica set). * **`mongo_client_kwargs`** (`Mapping[str, Any] | None`, default: `None` ) – Extra keyword arguments forwarded to `AsyncMongoClient`. * **`database_name`** (`str | None`, default: `None` ) – The MongoDB database name. Defaults to `agentlightning`. * **`partition_id`** (`str | None`, default: `None` ) – The partition id. Useful when sharing the database among multiple Agent-lightning trainers. * **`tracker`** (`[MetricsBackend](https://microsoft.github.io/agent-lightning/stable/reference/utilities/#agentlightning.utils.metrics.MetricsBackend " agentlightning.utils.metrics.MetricsBackend") | None`, default: `None` ) – The metrics tracker to use. * **`scan_debounce_seconds`** (`float`, default: `10.0` ) – The debounce time for the scan for unhealthy rollouts. Set to 0 to disable debouncing. #### `capabilities` `property` [¶](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.store.mongo.MongoLightningStore.capabilities "Permanent link") Return the capabilities of the store. #### `close()` `async` [¶](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.store.mongo.MongoLightningStore.close "Permanent link") Close the store by closing the client pool. #### `wait_for_rollouts(*, rollout_ids, timeout=None)` `async` [¶](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.store.mongo.MongoLightningStore.wait_for_rollouts "Permanent link") Wait for specified rollouts to complete with a timeout. Concurrently wait for all rollouts to complete with a timeout. ### `agentlightning.CollectionBasedLightningStore` [¶](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.CollectionBasedLightningStore "Permanent link") Bases: `[LightningStore](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.LightningStore " agentlightning.LightningStore (agentlightning.store.base.LightningStore)") `, `Generic[T_collections]` It's the standard implementation of LightningStore that uses collections to store data. If the store implementation is to use the store's default behavior, it's recommended to inherit from this class and override the methods if needed. Bring your own collection implementation by using a different `collections` argument. The methods in this class should generally not call each other, especially those that are locked. Parameters: * **`collections`** (`T_collections`) – The collections to use for storage. * **`read_snapshot`** (`bool`, default: `False` ) – Make sure read operations are atomic. If set to true, all read operations like `query_rollouts` will have better consistency. It may use an isolated snapshot that supports repeatable reads. * **`tracker`** (`[MetricsBackend](https://microsoft.github.io/agent-lightning/stable/reference/utilities/#agentlightning.utils.metrics.MetricsBackend " agentlightning.utils.metrics.MetricsBackend") | None`, default: `None` ) – Enable metrics tracking. * **`scan_debounce_seconds`** (`float`, default: `10.0` ) – The debounce time for the scan for unhealthy rollouts. Set to 0 to disable debouncing. The debounce is a non-perfect traffic control. It's isolated for each store instance if there are multiple worker replicas. #### `capabilities` `property` [¶](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.CollectionBasedLightningStore.capabilities "Permanent link") Return the capabilities of the store. This store supports no capability. The capability depends on the underlying collections. #### `add_many_spans(spans)` `async` [¶](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.CollectionBasedLightningStore.add_many_spans "Permanent link") Persist a sequence of pre-converted spans. See [`LightningStore.add_many_spans()`](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.LightningStore.add_many_spans " add_many_spans(spans) async ") for semantics. #### `add_otel_span(rollout_id, attempt_id, readable_span, sequence_id=None)` `async` [¶](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.CollectionBasedLightningStore.add_otel_span "Permanent link") Add an opentelemetry span to the store. See [`LightningStore.add_otel_span()`](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.LightningStore.add_otel_span " add_otel_span(rollout_id, attempt_id, readable_span, sequence_id=None) async ") for semantics. #### `add_resources(resources)` `async` [¶](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.CollectionBasedLightningStore.add_resources "Permanent link") Stores a new version of named resources and sets it as the latest. See [`LightningStore.add_resources()`](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.LightningStore.add_resources " add_resources(resources) async ") for semantics. #### `add_span(span)` `async` [¶](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.CollectionBasedLightningStore.add_span "Permanent link") Persist a pre-converted span. See [`LightningStore.add_span()`](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.LightningStore.add_span " add_span(span) async ") for semantics. #### `dequeue_many_rollouts(*, limit=1, worker_id=None)` `async` [¶](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.CollectionBasedLightningStore.dequeue_many_rollouts "Permanent link") Retrieves up to `limit` tasks from the queue without blocking. #### `dequeue_rollout(worker_id=None)` `async` [¶](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.CollectionBasedLightningStore.dequeue_rollout "Permanent link") Retrieves the next task from the queue without blocking. Returns `None` if the queue is empty. Will set the rollout status to preparing and create a new attempt. See [`LightningStore.dequeue_rollout()`](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.LightningStore.dequeue_rollout " dequeue_rollout(worker_id=None) async ") for semantics. #### `enqueue_many_rollouts(rollouts)` `async` [¶](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.CollectionBasedLightningStore.enqueue_many_rollouts "Permanent link") Adds many rollouts in a batch. #### `enqueue_rollout(input, mode=None, resources_id=None, config=None, metadata=None)` `async` [¶](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.CollectionBasedLightningStore.enqueue_rollout "Permanent link") Adds a new task to the queue with specific metadata and returns the rollout. See [`LightningStore.enqueue_rollout()`](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.LightningStore.enqueue_rollout " enqueue_rollout(input, mode=None, resources_id=None, config=None, metadata=None) async ") for semantics. #### `get_latest_attempt(rollout_id)` `async` [¶](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.CollectionBasedLightningStore.get_latest_attempt "Permanent link") Retrieves the latest attempt for a given rollout ID. See [`LightningStore.get_latest_attempt()`](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.LightningStore.get_latest_attempt " get_latest_attempt(rollout_id) async ") for semantics. #### `get_latest_resources()` `async` [¶](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.CollectionBasedLightningStore.get_latest_resources "Permanent link") Retrieves the latest version of named resources. See [`LightningStore.get_latest_resources()`](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.LightningStore.get_latest_resources " get_latest_resources() async ") for semantics. #### `get_many_span_sequence_ids(rollout_attempt_ids)` `async` [¶](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.CollectionBasedLightningStore.get_many_span_sequence_ids "Permanent link") Get the next span sequence IDs for a given list of rollout and attempt identifiers. #### `get_next_span_sequence_id(rollout_id, attempt_id)` `async` [¶](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.CollectionBasedLightningStore.get_next_span_sequence_id "Permanent link") Get the next span sequence ID for a given rollout and attempt. The number is strictly increasing for each rollout. The store will not issue the same sequence ID twice. See [`LightningStore.get_next_span_sequence_id()`](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.LightningStore.get_next_span_sequence_id " get_next_span_sequence_id(rollout_id, attempt_id) async ") for semantics. #### `get_resources_by_id(resources_id)` `async` [¶](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.CollectionBasedLightningStore.get_resources_by_id "Permanent link") Retrieves a specific version of named resources by its ID. See [`LightningStore.get_resources_by_id()`](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.LightningStore.get_resources_by_id " get_resources_by_id(resources_id) async ") for semantics. #### `get_rollout_by_id(rollout_id)` `async` [¶](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.CollectionBasedLightningStore.get_rollout_by_id "Permanent link") Retrieves a specific rollout by its ID. See [`LightningStore.get_rollout_by_id()`](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.LightningStore.get_rollout_by_id " get_rollout_by_id(rollout_id) async ") for semantics. If the rollout has been attempted, the latest attempt will also be returned. #### `query_attempts(rollout_id, *, sort_by='sequence_id', sort_order='asc', limit=-1, offset=0)` `async` [¶](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.CollectionBasedLightningStore.query_attempts "Permanent link") Retrieve attempts for a rollout with optional ordering/pagination. #### `query_resources(*, resources_id=None, resources_id_contains=None, sort_by=None, sort_order='asc', limit=-1, offset=0)` `async` [¶](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.CollectionBasedLightningStore.query_resources "Permanent link") Return every stored resource snapshot in insertion order. #### `query_rollouts(*, status_in=None, rollout_id_in=None, rollout_id_contains=None, filter_logic='and', sort_by=None, sort_order='asc', limit=-1, offset=0, status=None, rollout_ids=None)` `async` [¶](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.CollectionBasedLightningStore.query_rollouts "Permanent link") Retrieve rollouts with filtering and pagination. See [`LightningStore.query_rollouts()`](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.LightningStore.query_rollouts " query_rollouts(*, status_in=None, rollout_id_in=None, rollout_id_contains=None, filter_logic='and', sort_by=None, sort_order='asc', limit=-1, offset=0, status=None, rollout_ids=None) async ") for semantics. #### `query_spans(rollout_id, attempt_id=None, *, trace_id=None, trace_id_contains=None, span_id=None, span_id_contains=None, parent_id=None, parent_id_contains=None, name=None, name_contains=None, filter_logic='and', limit=-1, offset=0, sort_by='sequence_id', sort_order='asc')` `async` [¶](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.CollectionBasedLightningStore.query_spans "Permanent link") Query and retrieve spans associated with a specific rollout ID. Returns an empty list if no spans are found. See [`LightningStore.query_spans()`](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.LightningStore.query_spans " query_spans(rollout_id, attempt_id=None, *, trace_id=None, trace_id_contains=None, span_id=None, span_id_contains=None, parent_id=None, parent_id_contains=None, name=None, name_contains=None, filter_logic='and', limit=-1, offset=0, sort_by='sequence_id', sort_order='asc') async ") for semantics. #### `query_workers(*, status_in=None, worker_id_contains=None, filter_logic='and', sort_by=None, sort_order='asc', limit=-1, offset=0)` `async` [¶](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.CollectionBasedLightningStore.query_workers "Permanent link") Return the current snapshot of all workers. #### `start_attempt(rollout_id, worker_id=None)` `async` [¶](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.CollectionBasedLightningStore.start_attempt "Permanent link") Creates a new attempt for a given rollout ID and return the attempt details. See [`LightningStore.start_attempt()`](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.LightningStore.start_attempt " start_attempt(rollout_id, worker_id=None) async ") for semantics. #### `start_rollout(input, mode=None, resources_id=None, config=None, metadata=None, worker_id=None)` `async` [¶](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.CollectionBasedLightningStore.start_rollout "Permanent link") Notify the store that I'm about to run a rollout. See [`LightningStore.start_rollout()`](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.LightningStore.start_rollout " start_rollout(input, mode=None, resources_id=None, config=None, metadata=None, worker_id=None) async ") for semantics. #### `statistics()` `async` [¶](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.CollectionBasedLightningStore.statistics "Permanent link") Return the statistics of the store. #### `update_attempt(rollout_id, attempt_id, status=UNSET, worker_id=UNSET, last_heartbeat_time=UNSET, metadata=UNSET)` `async` [¶](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.CollectionBasedLightningStore.update_attempt "Permanent link") Update a specific or latest attempt for a given rollout. See [`LightningStore.update_attempt()`](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.LightningStore.update_attempt " update_attempt(rollout_id, attempt_id, status=UNSET, worker_id=UNSET, last_heartbeat_time=UNSET, metadata=UNSET) async ") for semantics. #### `update_resources(collections, resources_id, resources)` `async` [¶](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.CollectionBasedLightningStore.update_resources "Permanent link") Safely stores a new version of named resources and sets it as the latest. See [`LightningStore.update_resources()`](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.LightningStore.update_resources " update_resources(resources_id, resources) async ") for semantics. #### `update_rollout(rollout_id, input=UNSET, mode=UNSET, resources_id=UNSET, status=UNSET, config=UNSET, metadata=UNSET)` `async` [¶](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.CollectionBasedLightningStore.update_rollout "Permanent link") Update the rollout status and related metadata. See [`LightningStore.update_rollout()`](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.LightningStore.update_rollout " update_rollout(rollout_id, input=UNSET, mode=UNSET, resources_id=UNSET, status=UNSET, config=UNSET, metadata=UNSET) async ") for semantics. #### `update_worker(worker_id, heartbeat_stats=UNSET)` `async` [¶](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.CollectionBasedLightningStore.update_worker "Permanent link") Create or update a worker entry. #### `wait_for_rollout(rollout_id, timeout=None)` `async` [¶](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.CollectionBasedLightningStore.wait_for_rollout "Permanent link") Wait for a specific rollout to complete with a timeout. Subclass may use advanced mechanisms like events to accelerate this. Returns the completed rollout, or None if timeout is reached. #### `wait_for_rollouts(*, rollout_ids, timeout=None)` `async` [¶](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.CollectionBasedLightningStore.wait_for_rollouts "Permanent link") Wait for specified rollouts to complete with a timeout. Returns the completed rollouts, potentially incomplete if timeout is reached. This method does not change the state of the store. See [`LightningStore.wait_for_rollouts()`](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.LightningStore.wait_for_rollouts " wait_for_rollouts(*, rollout_ids, timeout=None) async ") for semantics. Client-Server and Thread-safe Wrappers[¶](https://microsoft.github.io/agent-lightning/stable/reference/store/#client-server-and-thread-safe-wrappers "Permanent link") ----------------------------------------------------------------------------------------------------------------------------------------------------------------------- ### `agentlightning.LightningStoreServer` [¶](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.LightningStoreServer "Permanent link") Bases: `[LightningStore](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.LightningStore " agentlightning.LightningStore (agentlightning.store.base.LightningStore)") ` Server wrapper that exposes a LightningStore via HTTP API. Delegates all operations to an underlying store implementation. Healthcheck and watchdog relies on the underlying store. `agl store` is a convenient CLI to start a store server. When the server is executed in a subprocess, the store will discover itself having a different PID and automatically delegate to an HTTP client instead of using the local store. This ensures one single copy of the store will be shared across all processes. This server exporting OTLP-compatible traces via the `/v1/traces` endpoint. Parameters: * **`store`** (`[LightningStore](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.LightningStore " agentlightning.LightningStore (agentlightning.store.base.LightningStore)") `) – The underlying store to delegate operations to. * **`host`** (`str | None`, default: `None` ) – The hostname or IP address to bind the server to. * **`port`** (`int | None`, default: `None` ) – The TCP port to listen on. * **`cors_allow_origins`** (`Sequence[str] | str | None`, default: `None` ) – A list of CORS origins to allow. Use '\*' to allow all origins. * **`launch_mode`** (`[LaunchMode](https://microsoft.github.io/agent-lightning/stable/reference/utilities/#agentlightning.utils.server_launcher.LaunchMode " agentlightning.utils.server_launcher.LaunchMode = Literal['asyncio', 'thread', 'mp'] module-attribute (agentlightning.utils.server_launcher.LaunchMode)") `, default: `'thread'` ) – The launch mode to use for the server. Defaults to "thread", which runs the server in a separate thread. * **`launcher_args`** (`[PythonServerLauncherArgs](https://microsoft.github.io/agent-lightning/stable/reference/utilities/#agentlightning.utils.server_launcher.PythonServerLauncherArgs " agentlightning.utils.server_launcher.PythonServerLauncherArgs dataclass ") | None`, default: `None` ) – The arguments to use for the server launcher. It's not allowed to set `host`, `port`, `launch_mode` together with `launcher_args`. * **`n_workers`** (`int`, default: `1` ) – The number of workers to run in the server. Only applicable for `mp` launch mode. * **`tracker`** (`[MetricsBackend](https://microsoft.github.io/agent-lightning/stable/reference/utilities/#agentlightning.utils.metrics.MetricsBackend " agentlightning.utils.metrics.MetricsBackend") | None`, default: `None` ) – The metrics tracker to use for the server. #### `capabilities` `property` [¶](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.LightningStoreServer.capabilities "Permanent link") Return the capabilities of the store. #### `endpoint` `property` [¶](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.LightningStoreServer.endpoint "Permanent link") Endpoint is the address that the client will use to connect to the server. #### `__getstate__()` [¶](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.LightningStoreServer.__getstate__ "Permanent link") Control pickling to prevent server state from being sent to subprocesses. When LightningStoreServer is pickled (e.g., passed to a subprocess), we only serialize the underlying store and connection details. The client instance and process-awareness state are excluded as they should not be transferred between processes. The subprocess should create its own server instance if needed. #### `__setstate__(state)` [¶](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.LightningStoreServer.__setstate__ "Permanent link") Restore from pickle by reconstructing only the essential attributes. Note: This creates a new server instance without FastAPI/uvicorn initialized. Call **init**() pattern or create a new LightningStoreServer if you need a fully functional server in the subprocess. The unpickled server will also have no app and store attributes, this is to make sure there is only one copy of the server in the whole system. #### `otlp_traces_endpoint()` [¶](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.LightningStoreServer.otlp_traces_endpoint "Permanent link") Return the OTLP/HTTP traces endpoint of the store. #### `run_forever()` `async` [¶](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.LightningStoreServer.run_forever "Permanent link") Runs the FastAPI server indefinitely. #### `start()` `async` [¶](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.LightningStoreServer.start "Permanent link") Starts the FastAPI server in the background. You need to call this method in the same process as the server was created in. #### `stop()` `async` [¶](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.LightningStoreServer.stop "Permanent link") Gracefully stops the running FastAPI server. You need to call this method in the same process as the server was created in. ### `agentlightning.LightningStoreClient` [¶](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.LightningStoreClient "Permanent link") Bases: `[LightningStore](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.LightningStore " agentlightning.LightningStore (agentlightning.store.base.LightningStore)") ` HTTP client that talks to a remote LightningStoreServer. Parameters: * **`server_address`** (`str`) – The address of the LightningStoreServer to connect to. * **`retry_delays`** (`Sequence[float]`, default: `(1.0, 2.0, 5.0)` ) – Backoff schedule (seconds) used when the initial request fails for a non-application reason. Each entry is a retry attempt. Setting to an empty sequence to disable retries. * **`health_retry_delays`** (`Sequence[float]`, default: `(0.1, 0.2, 0.5)` ) – Delays between /health probes while waiting for the server to come back. Setting to an empty sequence to disable health checks. * **`request_timeout`** (`float`, default: `30.0` ) – Timeout (seconds) for each request. * **`connection_timeout`** (`float`, default: `5.0` ) – Timeout (seconds) for establishing connection. #### `capabilities` `property` [¶](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.LightningStoreClient.capabilities "Permanent link") Return the capabilities of the store. #### `__getstate__()` [¶](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.LightningStoreClient.__getstate__ "Permanent link") When LightningStoreClient is pickled (e.g., passed to a subprocess), we only serialize the server address and retry configurations. The ClientSessions are excluded as they should not be transferred between processes. #### `__setstate__(state)` [¶](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.LightningStoreClient.__setstate__ "Permanent link") Restore from pickle by reconstructing only the essential attributes. Replicating `__init__` logic to create another client instance in the subprocess. #### `close()` `async` [¶](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.LightningStoreClient.close "Permanent link") Close the HTTP session. #### `dequeue_rollout(worker_id=None)` `async` [¶](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.LightningStoreClient.dequeue_rollout "Permanent link") Dequeue a rollout from the server queue. Returns: * `Optional[[AttemptedRollout](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.AttemptedRollout " agentlightning.AttemptedRollout (agentlightning.types.AttemptedRollout)") ]` – AttemptedRollout if a rollout is available, None if queue is empty. Note This method does NOT retry on failures. If any exception occurs (network error, server error, etc.), it logs the error and returns None immediately. #### `get_latest_attempt(rollout_id)` `async` [¶](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.LightningStoreClient.get_latest_attempt "Permanent link") Get the latest attempt for a rollout. Parameters: * **`rollout_id`** (`str`) – ID of the rollout to query. Returns: * `Optional[[Attempt](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.Attempt " agentlightning.Attempt (agentlightning.types.Attempt)") ]` – Attempt if found, None if not found or if all retries are exhausted. Note This method retries on transient failures (network errors, 5xx status codes). If all retries fail, it logs the error and returns None instead of raising an exception. #### `get_latest_resources()` `async` [¶](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.LightningStoreClient.get_latest_resources "Permanent link") Get the latest resources. Returns: * `Optional[[ResourcesUpdate](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.ResourcesUpdate " agentlightning.ResourcesUpdate (agentlightning.types.ResourcesUpdate)") ]` – ResourcesUpdate if found, None if not found or if all retries are exhausted. Note This method retries on transient failures (network errors, 5xx status codes). If all retries fail, it logs the error and returns None instead of raising an exception. #### `get_resources_by_id(resources_id)` `async` [¶](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.LightningStoreClient.get_resources_by_id "Permanent link") Get resources by their ID. Parameters: * **`resources_id`** (`str`) – ID of the resources to retrieve. Returns: * `Optional[[ResourcesUpdate](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.ResourcesUpdate " agentlightning.ResourcesUpdate (agentlightning.types.ResourcesUpdate)") ]` – ResourcesUpdate if found, None if not found or if all retries are exhausted. Note This method retries on transient failures (network errors, 5xx status codes). If all retries fail, it logs the error and returns None instead of raising an exception. #### `get_rollout_by_id(rollout_id)` `async` [¶](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.LightningStoreClient.get_rollout_by_id "Permanent link") Get a rollout by its ID. Parameters: * **`rollout_id`** (`str`) – ID of the rollout to retrieve. Returns: * `Optional[[Rollout](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.Rollout " agentlightning.Rollout (agentlightning.types.Rollout)") ]` – Rollout if found, None if not found or if all retries are exhausted. Note This method retries on transient failures (network errors, 5xx status codes). If all retries fail, it logs the error and returns None instead of raising an exception. #### `otlp_traces_endpoint()` [¶](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.LightningStoreClient.otlp_traces_endpoint "Permanent link") Return the OTLP/HTTP traces endpoint of the store. #### `query_resources(*, resources_id=None, resources_id_contains=None, sort_by=None, sort_order='asc', limit=-1, offset=0)` `async` [¶](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.LightningStoreClient.query_resources "Permanent link") List all resource snapshots stored on the server. #### `wait_for_rollouts(*, rollout_ids, timeout=None)` `async` [¶](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.LightningStoreClient.wait_for_rollouts "Permanent link") Wait for rollouts to complete. Parameters: * **`rollout_ids`** (`List[str]`) – List of rollout IDs to wait for. * **`timeout`** (`Optional[float]`, default: `None` ) – Timeout in seconds. If not None, the method will raise a ValueError if the timeout is greater than 0.1 seconds. Returns: * `List[[Rollout](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.Rollout " agentlightning.Rollout (agentlightning.types.Rollout)") ]` – List of rollouts that are completed. ### `agentlightning.LightningStoreThreaded` [¶](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.LightningStoreThreaded "Permanent link") Bases: `[LightningStore](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.LightningStore " agentlightning.LightningStore (agentlightning.store.base.LightningStore)") ` Facade that delegates all store operations to a underlying store instance. The operations are guaranteed to be thread-safe. Make sure the threaded stores are instantiated before initializing the threads. #### `capabilities` `property` [¶](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.LightningStoreThreaded.capabilities "Permanent link") Return the capabilities of the store. #### `statistics()` `async` [¶](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.LightningStoreThreaded.statistics "Permanent link") Return the statistics of the store. Collections and Collection Implementations[¶](https://microsoft.github.io/agent-lightning/stable/reference/store/#collections-and-collection-implementations "Permanent link") ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- ### `agentlightning.store.collection.AtomicMode = Literal['r', 'w', 'rw']` `module-attribute` [¶](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.store.collection.AtomicMode "Permanent link") What is expected within the atomic context. Can be "read", "write", or "read-write". ### `agentlightning.store.collection.AtomicLabels = Literal['rollouts', 'attempts', 'spans', 'resources', 'workers', 'rollout_queue', 'span_sequence_ids', 'generic']` `module-attribute` [¶](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.store.collection.AtomicLabels "Permanent link") Labels for atomic operations. These labels are used to identify the collections that are affected by the atomic operation. The `generic` label is used to identify atomic operations that are not associated with any specific collection. ### `agentlightning.store.collection.Collection` [¶](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.store.collection.Collection "Permanent link") Bases: `TrackedCollection`, `Generic[T]` Standard collection interface. Behaves like a list of items. Supporting addition, updating, and deletion of items. #### `delete(items)` `async` [¶](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.store.collection.Collection.delete "Permanent link") Delete the given items from the collection. Parameters: * **`items`** (`Sequence[T]`) – The items to delete from the collection. Raises: * `ValueError` – If the items with the primary keys to be deleted do not exist. #### `get(filter=None, sort=None)` `async` [¶](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.store.collection.Collection.get "Permanent link") Get the first item that matches the given filters. Parameters: * **`filter`** (`Optional[[FilterOptions](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.FilterOptions " agentlightning.FilterOptions = Mapping[Union[str, Literal['_aggregate', '_must']], Union[FilterField, Literal['and', 'or'], Mapping[str, FilterField]]] module-attribute (agentlightning.types.FilterOptions)") ]`, default: `None` ) – The filters to apply to the collection. See [`FilterOptions`](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.FilterOptions " agentlightning.FilterOptions = Mapping[Union[str, Literal['_aggregate', '_must']], Union[FilterField, Literal['and', 'or'], Mapping[str, FilterField]]] module-attribute ") . * **`sort`** (`Optional[[SortOptions](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.SortOptions " agentlightning.SortOptions (agentlightning.types.SortOptions)") ]`, default: `None` ) – Sort options. See [`SortOptions`](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.SortOptions " agentlightning.SortOptions") . Returns: * `Optional[T]` – The first item that matches the given filters, or None if no item matches. #### `insert(items)` `async` [¶](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.store.collection.Collection.insert "Permanent link") Add the given items to the collection. Raises: * `ValueError` – If an item with the same primary key already exists. #### `item_type()` [¶](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.store.collection.Collection.item_type "Permanent link") Get the type of the items in the collection. #### `primary_keys()` [¶](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.store.collection.Collection.primary_keys "Permanent link") Get the primary keys of the collection. #### `query(filter=None, sort=None, limit=-1, offset=0)` `async` [¶](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.store.collection.Collection.query "Permanent link") Query the collection with the given filters, sort order, and pagination. Parameters: * **`filter`** (`Optional[[FilterOptions](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.FilterOptions " agentlightning.FilterOptions = Mapping[Union[str, Literal['_aggregate', '_must']], Union[FilterField, Literal['and', 'or'], Mapping[str, FilterField]]] module-attribute (agentlightning.types.FilterOptions)") ]`, default: `None` ) – The filters to apply to the collection. See [`FilterOptions`](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.FilterOptions " agentlightning.FilterOptions = Mapping[Union[str, Literal['_aggregate', '_must']], Union[FilterField, Literal['and', 'or'], Mapping[str, FilterField]]] module-attribute ") . * **`sort`** (`Optional[[SortOptions](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.SortOptions " agentlightning.SortOptions (agentlightning.types.SortOptions)") ]`, default: `None` ) – The options for sorting the collection. See [`SortOptions`](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.SortOptions " agentlightning.SortOptions") . The field must exist in the model. If field might contain null values, in which case the behavior is undefined (i.e., depending on the implementation). * **`limit`** (`int`, default: `-1` ) – Max number of items to return. Use -1 for "no limit". * **`offset`** (`int`, default: `0` ) – Number of items to skip from the start of the _matching_ items. Returns: * `[PaginatedResult](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.PaginatedResult " agentlightning.PaginatedResult (agentlightning.types.PaginatedResult)") [T]` – PaginatedResult with items, limit, offset, and total matched items. #### `size()` `async` [¶](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.store.collection.Collection.size "Permanent link") Get the number of items in the collection. #### `update(items, update_fields=None)` `async` [¶](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.store.collection.Collection.update "Permanent link") Update the given items in the collection. Parameters: * **`items`** (`Sequence[T]`) – The items to update in the collection. * **`update_fields`** (`Sequence[str] | None`, default: `None` ) – The fields to update. If not provided, all fields in the type will be updated. Only applicable if the item type is a Pydantic BaseModel. Raises: * `ValueError` – If an item with the primary keys does not exist. Returns: * `Sequence[T]` – The items that were updated. #### `upsert(items, update_fields=None)` `async` [¶](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.store.collection.Collection.upsert "Permanent link") Upsert the given items into the collection. If the items with the same primary keys already exist, they will be updated. Otherwise, they will be inserted. The operation has three semantics configurable via `update_fields`: * `update_or_insert` via `collection.upsert(items, update_fields=["status", "updated_at"])`. If the item with the same primary keys already exists, only the specified fields will be updated. Otherwise, the item will be inserted. * `get_or_insert` via `collection.upsert(items, update_fields=[])`. If the item with the same primary keys already exists, the item will be left unchanged. Otherwise, the item will be inserted. * `replace_ish` via `collection.upsert(items)`. If the item with the same primary keys already exists, all fields from the item will be set. Otherwise, the item will be inserted. Returns: * `Sequence[T]` – The items that were upserted. ### `agentlightning.store.collection.Queue` [¶](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.store.collection.Queue "Permanent link") Bases: `TrackedCollection`, `Generic[T]` Behaves like a deque. Supporting appending items to the end and popping items from the front. #### `dequeue(limit=1)` `async` [¶](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.store.collection.Queue.dequeue "Permanent link") Pop the given number of items from the front of the queue. Parameters: * **`limit`** (`int`, default: `1` ) – The number of items to pop from the front of the queue. Returns: * `Sequence[T]` – The items that were popped from the front of the queue. * `Sequence[T]` – If there are less than `limit` items in the queue, the remaining items will be returned. #### `enqueue(items)` `async` [¶](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.store.collection.Queue.enqueue "Permanent link") Append the given items to the end of the queue. Parameters: * **`items`** (`Sequence[T]`) – The items to append to the end of the queue. Returns: * `Sequence[T]` – The items that were appended to the end of the queue. #### `has(item)` `async` [¶](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.store.collection.Queue.has "Permanent link") Check if the given item is in the queue. #### `item_type()` [¶](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.store.collection.Queue.item_type "Permanent link") Get the type of the items in the queue. #### `peek(limit=1)` `async` [¶](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.store.collection.Queue.peek "Permanent link") Peek the given number of items from the front of the queue. Parameters: * **`limit`** (`int`, default: `1` ) – The number of items to peek from the front of the queue. Returns: * `Sequence[T]` – The items that were peeked from the front of the queue. * `Sequence[T]` – If there are less than `limit` items in the queue, the remaining items will be returned. #### `size()` `async` [¶](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.store.collection.Queue.size "Permanent link") Get the number of items in the queue. ### `agentlightning.store.collection.KeyValue` [¶](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.store.collection.KeyValue "Permanent link") Bases: `TrackedCollection`, `Generic[K, V]` Behaves like a dictionary. Supporting addition, updating, and deletion of items. #### `chmax(key, value)` `async` [¶](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.store.collection.KeyValue.chmax "Permanent link") Set the value for the given key to the maximum of the current and new value. Raises: * `TypeError` – If the existing value or `value` is not numeric. #### `get(key, default=None)` `async` [¶](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.store.collection.KeyValue.get "Permanent link") Get the value for the given key, or the default value if the key is not found. #### `has(key)` `async` [¶](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.store.collection.KeyValue.has "Permanent link") Check if the given key is in the dictionary. #### `inc(key, amount)` `async` [¶](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.store.collection.KeyValue.inc "Permanent link") Increase the numeric value for the given key by `amount` and return the new value. Raises: * `TypeError` – If the existing value or `amount` is not numeric. #### `pop(key, default=None)` `async` [¶](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.store.collection.KeyValue.pop "Permanent link") Pop the value for the given key, or the default value if the key is not found. #### `set(key, value)` `async` [¶](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.store.collection.KeyValue.set "Permanent link") Set the value for the given key. #### `size()` `async` [¶](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.store.collection.KeyValue.size "Permanent link") Get the number of items in the dictionary. ### `agentlightning.store.collection.LightningCollections` [¶](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.store.collection.LightningCollections "Permanent link") Bases: `TrackedCollection` Collections of rollouts, attempts, spans, resources, and workers. [LightningStore](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.LightningStore " agentlightning.LightningStore") implementations can use this as a storage base to implement the store API. #### `attempts` `property` [¶](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.store.collection.LightningCollections.attempts "Permanent link") Collections of attempts. #### `resources` `property` [¶](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.store.collection.LightningCollections.resources "Permanent link") Collections of resources. #### `rollout_queue` `property` [¶](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.store.collection.LightningCollections.rollout_queue "Permanent link") Queue of rollouts (tasks). #### `rollouts` `property` [¶](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.store.collection.LightningCollections.rollouts "Permanent link") Collections of rollouts. #### `span_sequence_ids` `property` [¶](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.store.collection.LightningCollections.span_sequence_ids "Permanent link") Dictionary (counter) of span sequence IDs. #### `spans` `property` [¶](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.store.collection.LightningCollections.spans "Permanent link") Collections of spans. #### `workers` `property` [¶](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.store.collection.LightningCollections.workers "Permanent link") Collections of workers. #### `atomic(*, mode='rw', snapshot=False, commit=False, labels=None, **kwargs)` [¶](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.store.collection.LightningCollections.atomic "Permanent link") Perform a atomic operation on the collections. Subclass may use args and kwargs to support multiple levels of atomicity. The arguments can be seen as tags. They only imply the behavior of the operation, not the implementation. Parameters: * **`mode`** (`[AtomicMode](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.store.collection.AtomicMode " agentlightning.store.collection.AtomicMode = Literal['r', 'w', 'rw'] module-attribute (agentlightning.store.collection.base.AtomicMode)") `, default: `'rw'` ) – The mode of atomicity. See [`AtomicMode`](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.store.collection.AtomicMode " agentlightning.store.collection.AtomicMode = Literal['r', 'w', 'rw'] module-attribute ") . * **`snapshot`** (`bool`, default: `False` ) – Enable read snapshot for repeatable reads. Data consistency is guaranteed. The real behavior is implementation-dependent. * **`commit`** (`bool`, default: `False` ) – Enable commitment for write operations. Unsuccessful operations will be rolled back depending on the implementation. Recommend to use [`execute()`](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.store.collection.LightningCollections.execute " execute(callback, *, mode='rw', snapshot=False, commit=False, labels=None, **kwargs) async ") for this level to enable automatic retries. Remember that the real behavior is implementation-dependent. * **`labels`** (`Optional[Sequence[[AtomicLabels](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.store.collection.AtomicLabels " agentlightning.store.collection.AtomicLabels = Literal['rollouts', 'attempts', 'spans', 'resources', 'workers', 'rollout_queue', 'span_sequence_ids', 'generic'] module-attribute (agentlightning.store.collection.base.AtomicLabels)") ]]`, default: `None` ) – Labels to add to the atomic operation (commonly used as lock names or collection names). * **`**kwargs`** (`Any`, default: `{}` ) – Keyword arguments to pass to the operation. #### `execute(callback, *, mode='rw', snapshot=False, commit=False, labels=None, **kwargs)` `async` [¶](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.store.collection.LightningCollections.execute "Permanent link") Execute the given callback within an atomic operation. Retry on transient errors is implied. See [`atomic()`](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.store.collection.LightningCollections.atomic " atomic(*, mode='rw', snapshot=False, commit=False, labels=None, **kwargs)") for more details. ### `agentlightning.store.collection.ListBasedCollection` [¶](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.store.collection.ListBasedCollection "Permanent link") Bases: `[Collection](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.store.collection.Collection " agentlightning.store.collection.Collection (agentlightning.store.collection.base.Collection)") [T]` In-memory implementation of Collection using a nested dict for O(1) primary-key lookup. The internal structure is: `{ pk1_value: { pk2_value: { ... pkN_value: item } } }` where the nesting depth equals the number of primary keys. Sorting behavior: 1. If no sort\_by is provided, the items are returned in the order of insertion. 2. If sort\_by is provided, the items are sorted by the value of the sort\_by field. 3. If the sort\_by field is a timestamp, the null values are treated as infinity. 4. If the sort\_by field is not a timestamp, the null values are treated as empty string if the field is str-like, 0 if the field is int-like, 0.0 if the field is float-like. #### `delete(items)` `async` [¶](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.store.collection.ListBasedCollection.delete "Permanent link") Delete the given items. Raises: * `ValueError` – If any item with the given primary keys does not exist. #### `get(filter=None, sort=None)` `async` [¶](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.store.collection.ListBasedCollection.get "Permanent link") Return the first (or best-sorted) item that matches the given filters, or None. #### `insert(items)` `async` [¶](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.store.collection.ListBasedCollection.insert "Permanent link") Insert the given items. Raises: * `DuplicatedPrimaryKeyError` – If any item with the same primary keys already exists. #### `item_type()` [¶](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.store.collection.ListBasedCollection.item_type "Permanent link") Return the Pydantic model type of items stored in this collection. #### `primary_keys()` [¶](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.store.collection.ListBasedCollection.primary_keys "Permanent link") Return the primary key field names for this collection. #### `query(filter=None, sort=None, limit=-1, offset=0)` `async` [¶](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.store.collection.ListBasedCollection.query "Permanent link") Query the collection with filters, sort order, and pagination. Parameters: * **`filter`** (`Optional[[FilterOptions](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.FilterOptions " agentlightning.FilterOptions = Mapping[Union[str, Literal['_aggregate', '_must']], Union[FilterField, Literal['and', 'or'], Mapping[str, FilterField]]] module-attribute (agentlightning.types.FilterOptions)") ]`, default: `None` ) – Mapping of field name to operator dict along with the optional `_aggregate` logic. * **`sort`** (`Optional[[SortOptions](https://microsoft.github.io/agent-lightning/stable/reference/types/#agentlightning.SortOptions " agentlightning.SortOptions (agentlightning.types.SortOptions)") ]`, default: `None` ) – Options describing which field to sort by and in which order. * **`limit`** (`int`, default: `-1` ) – Max number of items to return. Use -1 for "no limit". * **`offset`** (`int`, default: `0` ) – Number of items to skip from the start of the _matching_ items. #### `size()` `async` [¶](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.store.collection.ListBasedCollection.size "Permanent link") Return the number of items stored in the collection. #### `update(items, update_fields=None)` `async` [¶](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.store.collection.ListBasedCollection.update "Permanent link") Update the given items. Raises: * `ValueError` – If any item with the given primary keys does not exist. #### `upsert(items, update_fields=None)` `async` [¶](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.store.collection.ListBasedCollection.upsert "Permanent link") Upsert the given items (insert if missing, otherwise update). ### `agentlightning.store.collection.DequeBasedQueue` [¶](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.store.collection.DequeBasedQueue "Permanent link") Bases: `[Queue](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.store.collection.Queue " agentlightning.store.collection.Queue (agentlightning.store.collection.base.Queue)") [T]` Queue implementation backed by collections.deque. Provides O(1) amortized enqueue (append) and dequeue (popleft). ### `agentlightning.store.collection.DictBasedKeyValue` [¶](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.store.collection.DictBasedKeyValue "Permanent link") Bases: `[KeyValue](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.store.collection.KeyValue " agentlightning.store.collection.KeyValue (agentlightning.store.collection.base.KeyValue)") [K, V]` KeyValue implementation backed by a plain dictionary. ### `agentlightning.store.collection.InMemoryLightningCollections` [¶](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.store.collection.InMemoryLightningCollections "Permanent link") Bases: `[LightningCollections](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.store.collection.LightningCollections " agentlightning.store.collection.LightningCollections (agentlightning.store.collection.base.LightningCollections)") ` In-memory implementation of LightningCollections using Python data structures. Serves as the storage base for [`InMemoryLightningStore`](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.InMemoryLightningStore " agentlightning.InMemoryLightningStore") . #### `atomic(*, mode='rw', snapshot=False, labels=None, **kwargs)` `async` [¶](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.store.collection.InMemoryLightningCollections.atomic "Permanent link") In-memory collections apply a lock outside. It doesn't need to manipulate the collections inside. Skip the locking if mode is "r" and snapshot is False. This collection implementation does NOT support rollback / commit. #### `evict_spans_for_rollout(rollout_id)` `async` [¶](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.store.collection.InMemoryLightningCollections.evict_spans_for_rollout "Permanent link") Evict all spans for a given rollout ID. Uses private API for efficiency. ### `agentlightning.store.collection.mongo.MongoBasedCollection` [¶](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.store.collection.mongo.MongoBasedCollection "Permanent link") Bases: `[Collection](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.store.collection.Collection " agentlightning.store.collection.Collection (agentlightning.store.collection.base.Collection)") [T_model]` Mongo-based implementation of Collection. Parameters: * **`client_pool`** (`[MongoClientPool](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.store.collection.mongo.MongoClientPool " agentlightning.store.collection.mongo.MongoClientPool") [Mapping[str, Any]]`) – The pool of MongoDB clients. * **`database_name`** (`str`) – The name of the database. * **`collection_name`** (`str`) – The name of the collection. * **`partition_id`** (`str`) – The partition ID. Used to partition the collection into multiple collections. * **`primary_keys`** (`Sequence[str]`) – The primary keys of the collection. * **`item_type`** (`Type[T_model]`) – The type of the items in the collection. * **`extra_indexes`** (`Sequence[Sequence[str]]`, default: `[]` ) – The extra indexes to create on the collection. * **`tracker`** (`[MetricsBackend](https://microsoft.github.io/agent-lightning/stable/reference/utilities/#agentlightning.utils.metrics.MetricsBackend " agentlightning.utils.metrics.MetricsBackend") | None`, default: `None` ) – The metrics tracker to use. #### `ensure_collection()` `async` [¶](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.store.collection.mongo.MongoBasedCollection.ensure_collection "Permanent link") Ensure the backing MongoDB collection exists (and optionally its indexes). This method is idempotent and safe to call multiple times. It will also create a unique index across the configured primary key fields. #### `insert(items)` `async` [¶](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.store.collection.mongo.MongoBasedCollection.insert "Permanent link") Insert items into the collection. The implementation does NOT do checks for duplicate primary keys, neither within the same insert call nor across different insert calls. It relies on the database to enforce uniqueness via indexes. #### `primary_keys()` [¶](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.store.collection.mongo.MongoBasedCollection.primary_keys "Permanent link") Return the primary key field names for this collection. #### `query(filter=None, sort=None, limit=-1, offset=0)` `async` [¶](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.store.collection.mongo.MongoBasedCollection.query "Permanent link") Mongo-based implementation of Collection.query. The handling of null-values in sorting is different from memory-based implementation. In MongoDB, null values are treated as less than non-null values. #### `with_session(session)` [¶](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.store.collection.mongo.MongoBasedCollection.with_session "Permanent link") Create a new collection with the same configuration but a new session. ### `agentlightning.store.collection.mongo.MongoBasedQueue` [¶](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.store.collection.mongo.MongoBasedQueue "Permanent link") Bases: `[Queue](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.store.collection.Queue " agentlightning.store.collection.Queue (agentlightning.store.collection.base.Queue)") [T_generic]`, `Generic[T_generic]` Mongo-based implementation of Queue backed by a MongoDB collection. Items are stored append-only; dequeue marks items as consumed instead of deleting them. #### `__init__(client_pool, database_name, collection_name, partition_id, item_type, tracker=None)` [¶](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.store.collection.mongo.MongoBasedQueue.__init__ "Permanent link") Parameters: * **`client_pool`** (`[MongoClientPool](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.store.collection.mongo.MongoClientPool " agentlightning.store.collection.mongo.MongoClientPool") [Mapping[str, Any]]`) – The pool of MongoDB clients. * **`database_name`** (`str`) – The name of the database. * **`collection_name`** (`str`) – The name of the collection backing the queue. * **`partition_id`** (`str`) – Partition identifier; allows multiple logical queues in one collection. * **`item_type`** (`Type[T_generic]`) – The Python type of queue items (primitive or BaseModel subclass). #### `ensure_collection()` `async` [¶](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.store.collection.mongo.MongoBasedQueue.ensure_collection "Permanent link") Ensure the backing collection exists. If it already exists, it returns the existing collection. ### `agentlightning.store.collection.mongo.MongoBasedKeyValue` [¶](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.store.collection.mongo.MongoBasedKeyValue "Permanent link") Bases: `[KeyValue](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.store.collection.KeyValue " agentlightning.store.collection.KeyValue (agentlightning.store.collection.base.KeyValue)") [K, V]`, `Generic[K, V]` Mongo-based implementation of KeyValue. #### `__init__(client_pool, database_name, collection_name, partition_id, key_type, value_type, tracker=None)` [¶](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.store.collection.mongo.MongoBasedKeyValue.__init__ "Permanent link") Parameters: * **`client_pool`** (`[MongoClientPool](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.store.collection.mongo.MongoClientPool " agentlightning.store.collection.mongo.MongoClientPool") [Mapping[str, Any]]`) – The pool of MongoDB clients. * **`database_name`** (`str`) – The name of the database. * **`collection_name`** (`str`) – The name of the collection backing the key-value store. * **`partition_id`** (`str`) – Partition identifier; allows multiple logical maps in one collection. * **`key_type`** (`Type[K]`) – The Python type of keys (primitive or BaseModel). * **`value_type`** (`Type[V]`) – The Python type of values (primitive or BaseModel). * **`tracker`** (`[MetricsBackend](https://microsoft.github.io/agent-lightning/stable/reference/utilities/#agentlightning.utils.metrics.MetricsBackend " agentlightning.utils.metrics.MetricsBackend") | None`, default: `None` ) – The metrics tracker to use. #### `ensure_collection(*, create_indexes=True)` `async` [¶](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.store.collection.mongo.MongoBasedKeyValue.ensure_collection "Permanent link") Ensure the backing collection exists (and optionally its indexes). ### `agentlightning.store.collection.mongo.MongoClientPool` [¶](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.store.collection.mongo.MongoClientPool "Permanent link") Bases: `Generic[T_mapping]` A pool of MongoDB clients, each bound to a specific event loop. The pool lazily creates `AsyncMongoClient` instances per event loop using the provided connection parameters, ensuring we never try to reuse a client across loops. #### `close()` `async` [¶](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.store.collection.mongo.MongoClientPool.close "Permanent link") Close all clients currently tracked by the pool. ### `agentlightning.store.collection.mongo.MongoLightningCollections` [¶](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.store.collection.mongo.MongoLightningCollections "Permanent link") Bases: `[LightningCollections](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.store.collection.LightningCollections " agentlightning.store.collection.LightningCollections (agentlightning.store.collection.base.LightningCollections)") ` Mongo implementation of LightningCollections using MongoDB collections. Serves as the storage base for [`MongoLightningStore`](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.store.mongo.MongoLightningStore " agentlightning.store.mongo.MongoLightningStore") . #### `atomic(mode='rw', snapshot=False, commit=False, labels=None, *args, **kwargs)` `async` [¶](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.store.collection.mongo.MongoLightningCollections.atomic "Permanent link") Perform a atomic operation on the collections. #### `execute(callback, *, mode='rw', snapshot=False, commit=False, labels=None, **kwargs)` `async` [¶](https://microsoft.github.io/agent-lightning/stable/reference/store/#agentlightning.store.collection.mongo.MongoLightningCollections.execute "Permanent link") Execute the given callback within an atomic operation, and with retries on transient errors. Back to top --- # Agent-lightning [Skip to content](https://microsoft.github.io/agent-lightning/0.2.0/#agent-lightning) ⚠️ You are viewing development documentation. For stable documentation, please visit [https://microsoft.github.io/agent-lightning/stable/](https://microsoft.github.io/agent-lightning/stable/) Agent Lightning[¶](https://microsoft.github.io/agent-lightning/0.2.0/#agent-lightning "Permanent link") ======================================================================================================== Agent Lightning is the absolute trainer to light up AI agents. [Join our Discord community](https://discord.gg/RYk7CdvDR7) to connect with other users and contributors. Features[¶](https://microsoft.github.io/agent-lightning/0.2.0/#features "Permanent link") ------------------------------------------------------------------------------------------ * Turn your agent into an optimizable beast with **ZERO CODE CHANGE** (almost)! 💤 * Build with **ANY** agent framework (LangChain, OpenAI Agent SDK, AutoGen, CrewAI, Microsoft Agent Framework...); or even WITHOUT agent framework (Python OpenAI). You name it! 🤖 * **Selectively** optimize one or more agents in a multi-agent system. 🎯 * Embraces **Algorithms** like Reinforcement Learning, Automatic Prompt Optimization, Supervised Fine-tuning and more. 🤗 How to Read this Documentation[¶](https://microsoft.github.io/agent-lightning/0.2.0/#how-to-read-this-documentation "Permanent link") -------------------------------------------------------------------------------------------------------------------------------------- This documentation is organized into the following parts: * [Installation](https://microsoft.github.io/agent-lightning/0.2.0/tutorials/installation/) - Get started with Agent Lightning * How-to Recipes (e.g., [Train SQL Agent with RL](https://microsoft.github.io/agent-lightning/0.2.0/how-to/train-sql-agent/) ) - Practical examples of training agents and customizing algorithms. * Learning More (e.g., [Debugging](https://microsoft.github.io/agent-lightning/0.2.0/tutorials/debug/) ) - Guides on specific topics like debugging or parallelization. * Algorithm Zoo (e.g., [APO](https://microsoft.github.io/agent-lightning/0.2.0/algorithm-zoo/apo/) ) - References for built-in algorithms. * Deep Dive (e.g., [Bird's Eye View](https://microsoft.github.io/agent-lightning/0.2.0/deep-dive/birds-eye-view/) ) - For a deeper understanding of what Agent-lightning is doing under the hood. * API References (e.g., [Agent](https://microsoft.github.io/agent-lightning/0.2.0/reference/agent/) ) - References for the Agent-lightning Python API. Resources[¶](https://microsoft.github.io/agent-lightning/0.2.0/#resources "Permanent link") -------------------------------------------------------------------------------------------- * 8/11/2025 [Training AI Agents to Write and Self-correct SQL with Reinforcement Learning](https://medium.com/@yugez/training-ai-agents-to-write-and-self-correct-sql-with-reinforcement-learning-571ed31281ad) Medium. * 8/5/2025 [Agent Lightning: Train ANY AI Agents with Reinforcement Learning](https://arxiv.org/abs/2508.03680) arXiv paper. * 7/26/2025 [We discovered an approach to train any AI agent with RL, with (almost) zero code changes.](https://www.reddit.com/r/LocalLLaMA/comments/1m9m670/we_discovered_an_approach_to_train_any_ai_agent/) Reddit. * 6/6/2025 [Agent Lightning - Microsoft Research](https://www.microsoft.com/en-us/research/project/agent-lightning/) Project page. Community Projects[¶](https://microsoft.github.io/agent-lightning/0.2.0/#community-projects "Permanent link") -------------------------------------------------------------------------------------------------------------- * [DeepWerewolf](https://github.com/af-74413592/DeepWerewolf) — A case study of agent RL training for the Chinese Werewolf game built with AgentScope and Agent Lightning. * [AgentFlow](https://agentflow.stanford.edu/) — A modular multi-agent framework that combines planner, executor, verifier, and generator agents with the Flow-GRPO algorithm to tackle long-horizon, sparse-reward tasks. Citation[¶](https://microsoft.github.io/agent-lightning/0.2.0/#citation "Permanent link") ------------------------------------------------------------------------------------------ If you find Agent Lightning useful in your research or projects, please cite our paper: `[](https://microsoft.github.io/agent-lightning/0.2.0/#__codelineno-0-1) @misc{luo2025agentlightningtrainai, [](https://microsoft.github.io/agent-lightning/0.2.0/#__codelineno-0-2) title={Agent Lightning: Train ANY AI Agents with Reinforcement Learning}, [](https://microsoft.github.io/agent-lightning/0.2.0/#__codelineno-0-3) author={Xufang Luo and Yuge Zhang and Zhiyuan He and Zilong Wang and Siyun Zhao and Dongsheng Li and Luna K. Qiu and Yuqing Yang}, [](https://microsoft.github.io/agent-lightning/0.2.0/#__codelineno-0-4) year={2025}, [](https://microsoft.github.io/agent-lightning/0.2.0/#__codelineno-0-5) eprint={2508.03680}, [](https://microsoft.github.io/agent-lightning/0.2.0/#__codelineno-0-6) archivePrefix={arXiv}, [](https://microsoft.github.io/agent-lightning/0.2.0/#__codelineno-0-7) primaryClass={cs.AI}, [](https://microsoft.github.io/agent-lightning/0.2.0/#__codelineno-0-8) url={https://arxiv.org/abs/2508.03680}, [](https://microsoft.github.io/agent-lightning/0.2.0/#__codelineno-0-9) }` License[¶](https://microsoft.github.io/agent-lightning/0.2.0/#license "Permanent link") ---------------------------------------------------------------------------------------- See the [LICENSE](https://github.com/microsoft/agent-lightning/blob/main/LICENSE) file for details. Back to top --- # Agent-lightning [Skip to content](https://microsoft.github.io/agent-lightning/0.2.2/#agent-lightning) ⚠️ You are viewing development documentation. For stable documentation, please visit [https://microsoft.github.io/agent-lightning/stable/](https://microsoft.github.io/agent-lightning/stable/) Agent Lightning[¶](https://microsoft.github.io/agent-lightning/0.2.2/#agent-lightning "Permanent link") ======================================================================================================== Agent Lightning is the absolute trainer to light up AI agents. [Join our Discord community](https://discord.gg/RYk7CdvDR7) to connect with other users and contributors. Features[¶](https://microsoft.github.io/agent-lightning/0.2.2/#features "Permanent link") ------------------------------------------------------------------------------------------ * Turn your agent into an optimizable beast with **ZERO CODE CHANGE** (almost)! 💤 * Build with **ANY** agent framework (LangChain, OpenAI Agent SDK, AutoGen, CrewAI, Microsoft Agent Framework...); or even WITHOUT agent framework (Python OpenAI). You name it! 🤖 * **Selectively** optimize one or more agents in a multi-agent system. 🎯 * Embraces **Algorithms** like Reinforcement Learning, Automatic Prompt Optimization, Supervised Fine-tuning and more. 🤗 How to Read this Documentation[¶](https://microsoft.github.io/agent-lightning/0.2.2/#how-to-read-this-documentation "Permanent link") -------------------------------------------------------------------------------------------------------------------------------------- This documentation is organized into the following parts: * [Installation](https://microsoft.github.io/agent-lightning/0.2.2/tutorials/installation/) - Get started with Agent Lightning * How-to Recipes (e.g., [Train SQL Agent with RL](https://microsoft.github.io/agent-lightning/0.2.2/how-to/train-sql-agent/) ) - Practical examples of training agents and customizing algorithms. * Learning More (e.g., [Debugging](https://microsoft.github.io/agent-lightning/0.2.2/tutorials/debug/) ) - Guides on specific topics like debugging or parallelization. * Algorithm Zoo (e.g., [APO](https://microsoft.github.io/agent-lightning/0.2.2/algorithm-zoo/apo/) ) - References for built-in algorithms. * Deep Dive (e.g., [Bird's Eye View](https://microsoft.github.io/agent-lightning/0.2.2/deep-dive/birds-eye-view/) ) - For a deeper understanding of what Agent-lightning is doing under the hood. * API References (e.g., [Agent](https://microsoft.github.io/agent-lightning/0.2.2/reference/agent/) ) - References for the Agent-lightning Python API. Resources[¶](https://microsoft.github.io/agent-lightning/0.2.2/#resources "Permanent link") -------------------------------------------------------------------------------------------- * 10/22/2025 [No More Retokenization Drift: Returning Token IDs via the OpenAI Compatible API Matters in Agent RL](https://blog.vllm.ai/2025/10/22/agent-lightning.html) vLLM blog. See also [Zhihu writeup](https://zhuanlan.zhihu.com/p/1965067274642785725) . * 8/11/2025 [Training AI Agents to Write and Self-correct SQL with Reinforcement Learning](https://medium.com/@yugez/training-ai-agents-to-write-and-self-correct-sql-with-reinforcement-learning-571ed31281ad) Medium. * 8/5/2025 [Agent Lightning: Train ANY AI Agents with Reinforcement Learning](https://arxiv.org/abs/2508.03680) arXiv paper. * 7/26/2025 [We discovered an approach to train any AI agent with RL, with (almost) zero code changes.](https://www.reddit.com/r/LocalLLaMA/comments/1m9m670/we_discovered_an_approach_to_train_any_ai_agent/) Reddit. * 6/6/2025 [Agent Lightning - Microsoft Research](https://www.microsoft.com/en-us/research/project/agent-lightning/) Project page. Community Projects[¶](https://microsoft.github.io/agent-lightning/0.2.2/#community-projects "Permanent link") -------------------------------------------------------------------------------------------------------------- * [DeepWerewolf](https://github.com/af-74413592/DeepWerewolf) — A case study of agent RL training for the Chinese Werewolf game built with AgentScope and Agent Lightning. * [AgentFlow](https://agentflow.stanford.edu/) — A modular multi-agent framework that combines planner, executor, verifier, and generator agents with the Flow-GRPO algorithm to tackle long-horizon, sparse-reward tasks. Citation[¶](https://microsoft.github.io/agent-lightning/0.2.2/#citation "Permanent link") ------------------------------------------------------------------------------------------ If you find Agent Lightning useful in your research or projects, please cite our paper: `[](https://microsoft.github.io/agent-lightning/0.2.2/#__codelineno-0-1) @misc{luo2025agentlightningtrainai, [](https://microsoft.github.io/agent-lightning/0.2.2/#__codelineno-0-2) title={Agent Lightning: Train ANY AI Agents with Reinforcement Learning}, [](https://microsoft.github.io/agent-lightning/0.2.2/#__codelineno-0-3) author={Xufang Luo and Yuge Zhang and Zhiyuan He and Zilong Wang and Siyun Zhao and Dongsheng Li and Luna K. Qiu and Yuqing Yang}, [](https://microsoft.github.io/agent-lightning/0.2.2/#__codelineno-0-4) year={2025}, [](https://microsoft.github.io/agent-lightning/0.2.2/#__codelineno-0-5) eprint={2508.03680}, [](https://microsoft.github.io/agent-lightning/0.2.2/#__codelineno-0-6) archivePrefix={arXiv}, [](https://microsoft.github.io/agent-lightning/0.2.2/#__codelineno-0-7) primaryClass={cs.AI}, [](https://microsoft.github.io/agent-lightning/0.2.2/#__codelineno-0-8) url={https://arxiv.org/abs/2508.03680}, [](https://microsoft.github.io/agent-lightning/0.2.2/#__codelineno-0-9) }` License[¶](https://microsoft.github.io/agent-lightning/0.2.2/#license "Permanent link") ---------------------------------------------------------------------------------------- See the [LICENSE](https://github.com/microsoft/agent-lightning/blob/main/LICENSE) file for details. Back to top --- # Agent-lightning [Skip to content](https://microsoft.github.io/agent-lightning/0.2.1/#agent-lightning) ⚠️ You are viewing development documentation. For stable documentation, please visit [https://microsoft.github.io/agent-lightning/stable/](https://microsoft.github.io/agent-lightning/stable/) Agent Lightning[¶](https://microsoft.github.io/agent-lightning/0.2.1/#agent-lightning "Permanent link") ======================================================================================================== Agent Lightning is the absolute trainer to light up AI agents. [Join our Discord community](https://discord.gg/RYk7CdvDR7) to connect with other users and contributors. Features[¶](https://microsoft.github.io/agent-lightning/0.2.1/#features "Permanent link") ------------------------------------------------------------------------------------------ * Turn your agent into an optimizable beast with **ZERO CODE CHANGE** (almost)! 💤 * Build with **ANY** agent framework (LangChain, OpenAI Agent SDK, AutoGen, CrewAI, Microsoft Agent Framework...); or even WITHOUT agent framework (Python OpenAI). You name it! 🤖 * **Selectively** optimize one or more agents in a multi-agent system. 🎯 * Embraces **Algorithms** like Reinforcement Learning, Automatic Prompt Optimization, Supervised Fine-tuning and more. 🤗 How to Read this Documentation[¶](https://microsoft.github.io/agent-lightning/0.2.1/#how-to-read-this-documentation "Permanent link") -------------------------------------------------------------------------------------------------------------------------------------- This documentation is organized into the following parts: * [Installation](https://microsoft.github.io/agent-lightning/0.2.1/tutorials/installation/) - Get started with Agent Lightning * How-to Recipes (e.g., [Train SQL Agent with RL](https://microsoft.github.io/agent-lightning/0.2.1/how-to/train-sql-agent/) ) - Practical examples of training agents and customizing algorithms. * Learning More (e.g., [Debugging](https://microsoft.github.io/agent-lightning/0.2.1/tutorials/debug/) ) - Guides on specific topics like debugging or parallelization. * Algorithm Zoo (e.g., [APO](https://microsoft.github.io/agent-lightning/0.2.1/algorithm-zoo/apo/) ) - References for built-in algorithms. * Deep Dive (e.g., [Bird's Eye View](https://microsoft.github.io/agent-lightning/0.2.1/deep-dive/birds-eye-view/) ) - For a deeper understanding of what Agent-lightning is doing under the hood. * API References (e.g., [Agent](https://microsoft.github.io/agent-lightning/0.2.1/reference/agent/) ) - References for the Agent-lightning Python API. Resources[¶](https://microsoft.github.io/agent-lightning/0.2.1/#resources "Permanent link") -------------------------------------------------------------------------------------------- * 10/22/2025 [No More Retokenization Drift: Returning Token IDs via the OpenAI Compatible API Matters in Agent RL](https://blog.vllm.ai/2025/10/22/agent-lightning.html) vLLM blog. See also [Zhihu writeup](https://zhuanlan.zhihu.com/p/1965067274642785725) . * 8/11/2025 [Training AI Agents to Write and Self-correct SQL with Reinforcement Learning](https://medium.com/@yugez/training-ai-agents-to-write-and-self-correct-sql-with-reinforcement-learning-571ed31281ad) Medium. * 8/5/2025 [Agent Lightning: Train ANY AI Agents with Reinforcement Learning](https://arxiv.org/abs/2508.03680) arXiv paper. * 7/26/2025 [We discovered an approach to train any AI agent with RL, with (almost) zero code changes.](https://www.reddit.com/r/LocalLLaMA/comments/1m9m670/we_discovered_an_approach_to_train_any_ai_agent/) Reddit. * 6/6/2025 [Agent Lightning - Microsoft Research](https://www.microsoft.com/en-us/research/project/agent-lightning/) Project page. Community Projects[¶](https://microsoft.github.io/agent-lightning/0.2.1/#community-projects "Permanent link") -------------------------------------------------------------------------------------------------------------- * [DeepWerewolf](https://github.com/af-74413592/DeepWerewolf) — A case study of agent RL training for the Chinese Werewolf game built with AgentScope and Agent Lightning. * [AgentFlow](https://agentflow.stanford.edu/) — A modular multi-agent framework that combines planner, executor, verifier, and generator agents with the Flow-GRPO algorithm to tackle long-horizon, sparse-reward tasks. Citation[¶](https://microsoft.github.io/agent-lightning/0.2.1/#citation "Permanent link") ------------------------------------------------------------------------------------------ If you find Agent Lightning useful in your research or projects, please cite our paper: `[](https://microsoft.github.io/agent-lightning/0.2.1/#__codelineno-0-1) @misc{luo2025agentlightningtrainai, [](https://microsoft.github.io/agent-lightning/0.2.1/#__codelineno-0-2) title={Agent Lightning: Train ANY AI Agents with Reinforcement Learning}, [](https://microsoft.github.io/agent-lightning/0.2.1/#__codelineno-0-3) author={Xufang Luo and Yuge Zhang and Zhiyuan He and Zilong Wang and Siyun Zhao and Dongsheng Li and Luna K. Qiu and Yuqing Yang}, [](https://microsoft.github.io/agent-lightning/0.2.1/#__codelineno-0-4) year={2025}, [](https://microsoft.github.io/agent-lightning/0.2.1/#__codelineno-0-5) eprint={2508.03680}, [](https://microsoft.github.io/agent-lightning/0.2.1/#__codelineno-0-6) archivePrefix={arXiv}, [](https://microsoft.github.io/agent-lightning/0.2.1/#__codelineno-0-7) primaryClass={cs.AI}, [](https://microsoft.github.io/agent-lightning/0.2.1/#__codelineno-0-8) url={https://arxiv.org/abs/2508.03680}, [](https://microsoft.github.io/agent-lightning/0.2.1/#__codelineno-0-9) }` License[¶](https://microsoft.github.io/agent-lightning/0.2.1/#license "Permanent link") ---------------------------------------------------------------------------------------- See the [LICENSE](https://github.com/microsoft/agent-lightning/blob/main/LICENSE) file for details. Back to top --- # Agent-lightning [Skip to content](https://microsoft.github.io/agent-lightning/0.3.0/#agent-lightning) ⚠️ You are viewing development documentation. For stable documentation, please visit [https://microsoft.github.io/agent-lightning/stable/](https://microsoft.github.io/agent-lightning/stable/) Agent Lightning[¶](https://microsoft.github.io/agent-lightning/0.3.0/#agent-lightning "Permanent link") ======================================================================================================== Agent Lightning is the absolute trainer to light up AI agents. [Join our Discord community](https://discord.gg/RYk7CdvDR7) to connect with other users and contributors. Features[¶](https://microsoft.github.io/agent-lightning/0.3.0/#features "Permanent link") ------------------------------------------------------------------------------------------ * Turn your agent into an optimizable beast with **ZERO CODE CHANGE** (almost)! 💤 * Build with **ANY** agent framework (LangChain, OpenAI Agent SDK, AutoGen, CrewAI, Microsoft Agent Framework...); or even WITHOUT agent framework (Python OpenAI). You name it! 🤖 * **Selectively** optimize one or more agents in a multi-agent system. 🎯 * Embraces **Algorithms** like Reinforcement Learning, Automatic Prompt Optimization, Supervised Fine-tuning and more. 🤗 How to Read this Documentation[¶](https://microsoft.github.io/agent-lightning/0.3.0/#how-to-read-this-documentation "Permanent link") -------------------------------------------------------------------------------------------------------------------------------------- This documentation is organized into the following parts: * [Installation](https://microsoft.github.io/agent-lightning/0.3.0/tutorials/installation/) - Get started with Agent Lightning * How-to Recipes (e.g., [Train SQL Agent with RL](https://microsoft.github.io/agent-lightning/0.3.0/how-to/train-sql-agent/) ) - Practical examples of training agents and customizing algorithms. * Learning More (e.g., [Debugging](https://microsoft.github.io/agent-lightning/0.3.0/tutorials/debug/) ) - Guides on specific topics like debugging or parallelization. * Algorithm Zoo (e.g., [APO](https://microsoft.github.io/agent-lightning/0.3.0/algorithm-zoo/apo/) ) - References for built-in algorithms. * Deep Dive (e.g., [Bird's Eye View](https://microsoft.github.io/agent-lightning/0.3.0/deep-dive/birds-eye-view/) ) - For a deeper understanding of what Agent-lightning is doing under the hood. * API References (e.g., [Agent](https://microsoft.github.io/agent-lightning/0.3.0/reference/agent/) ) - References for the Agent-lightning Python API. Resources[¶](https://microsoft.github.io/agent-lightning/0.3.0/#resources "Permanent link") -------------------------------------------------------------------------------------------- * 11/4/2025 [Tuning ANY AI agent with Tinker ✕ Agent-lightning](https://medium.com/@yugez/tuning-any-ai-agent-with-tinker-agent-lightning-part-1-1d8c9a397f0e) Medium. See also [Part 2](https://medium.com/@yugez/tuning-any-ai-agent-with-tinker-agent-lightning-part-2-332c5437f0dc) . * 10/22/2025 [No More Retokenization Drift: Returning Token IDs via the OpenAI Compatible API Matters in Agent RL](https://blog.vllm.ai/2025/10/22/agent-lightning.html) vLLM blog. See also [Zhihu writeup](https://zhuanlan.zhihu.com/p/1965067274642785725) . * 8/11/2025 [Training AI Agents to Write and Self-correct SQL with Reinforcement Learning](https://medium.com/@yugez/training-ai-agents-to-write-and-self-correct-sql-with-reinforcement-learning-571ed31281ad) Medium. * 8/5/2025 [Agent Lightning: Train ANY AI Agents with Reinforcement Learning](https://arxiv.org/abs/2508.03680) arXiv paper. * 7/26/2025 [We discovered an approach to train any AI agent with RL, with (almost) zero code changes.](https://www.reddit.com/r/LocalLLaMA/comments/1m9m670/we_discovered_an_approach_to_train_any_ai_agent/) Reddit. * 6/6/2025 [Agent Lightning - Microsoft Research](https://www.microsoft.com/en-us/research/project/agent-lightning/) Project page. Community Projects[¶](https://microsoft.github.io/agent-lightning/0.3.0/#community-projects "Permanent link") -------------------------------------------------------------------------------------------------------------- * [DeepWerewolf](https://github.com/af-74413592/DeepWerewolf) — A case study of agent RL training for the Chinese Werewolf game built with AgentScope and Agent Lightning. * [AgentFlow](https://agentflow.stanford.edu/) — A modular multi-agent framework that combines planner, executor, verifier, and generator agents with the Flow-GRPO algorithm to tackle long-horizon, sparse-reward tasks. * [Youtu-Agent](https://github.com/TencentCloudADP/Youtu-agent) — Youtu-Agent lets you build and train your agent with ease. Built with [a modified branch](https://github.com/microsoft/agent-lightning/tree/contrib/youtu-agent-lightning) of Agent Lightning, Youtu-Agent has verified up to 128 GPUs RL training on maths/code and search capabilities with steady convergence. Also check [the recipe](https://github.com/TencentCloudADP/youtu-agent/tree/rl/agl) and their blog [_Stop Wrestling with Your Agent RL: How Youtu-Agent Achieved Stable, 128-GPU Scaling Without Breaking a Sweat_](https://spotted-coconut-df8.notion.site/Stop-Wrestling-with-Your-Agent-RL-How-Youtu-Agent-Achieved-Stable-128-GPU-Scaling-Without-Breaking-2ca5e8f089ba80539a98c582b65e0233) . Citation[¶](https://microsoft.github.io/agent-lightning/0.3.0/#citation "Permanent link") ------------------------------------------------------------------------------------------ If you find Agent Lightning useful in your research or projects, please cite our paper: `[](https://microsoft.github.io/agent-lightning/0.3.0/#__codelineno-0-1) @misc{luo2025agentlightningtrainai, [](https://microsoft.github.io/agent-lightning/0.3.0/#__codelineno-0-2) title={Agent Lightning: Train ANY AI Agents with Reinforcement Learning}, [](https://microsoft.github.io/agent-lightning/0.3.0/#__codelineno-0-3) author={Xufang Luo and Yuge Zhang and Zhiyuan He and Zilong Wang and Siyun Zhao and Dongsheng Li and Luna K. Qiu and Yuqing Yang}, [](https://microsoft.github.io/agent-lightning/0.3.0/#__codelineno-0-4) year={2025}, [](https://microsoft.github.io/agent-lightning/0.3.0/#__codelineno-0-5) eprint={2508.03680}, [](https://microsoft.github.io/agent-lightning/0.3.0/#__codelineno-0-6) archivePrefix={arXiv}, [](https://microsoft.github.io/agent-lightning/0.3.0/#__codelineno-0-7) primaryClass={cs.AI}, [](https://microsoft.github.io/agent-lightning/0.3.0/#__codelineno-0-8) url={https://arxiv.org/abs/2508.03680}, [](https://microsoft.github.io/agent-lightning/0.3.0/#__codelineno-0-9) }` License[¶](https://microsoft.github.io/agent-lightning/0.3.0/#license "Permanent link") ---------------------------------------------------------------------------------------- See the [LICENSE](https://github.com/microsoft/agent-lightning/blob/main/LICENSE) file for details. Back to top --- # RESTful - Agent-lightning [Skip to content](https://microsoft.github.io/agent-lightning/0.3.0/reference/restful/#restful-api-references) ⚠️ You are viewing development documentation. For stable documentation, please visit [https://microsoft.github.io/agent-lightning/stable/](https://microsoft.github.io/agent-lightning/stable/) RESTful API References[¶](https://microsoft.github.io/agent-lightning/0.3.0/reference/restful/#restful-api-references "Permanent link") ======================================================================================================================================== Note Shown in the following is the RESTful API for Lightning Store. Back to top --- # Semantic Conventions - Agent-lightning [Skip to content](https://microsoft.github.io/agent-lightning/0.3.0/reference/semconv/#semantic-conventions) ⚠️ You are viewing development documentation. For stable documentation, please visit [https://microsoft.github.io/agent-lightning/stable/](https://microsoft.github.io/agent-lightning/stable/) Semantic Conventions[¶](https://microsoft.github.io/agent-lightning/0.3.0/reference/semconv/#semantic-conventions "Permanent link") ==================================================================================================================================== ### `agentlightning.semconv` [¶](https://microsoft.github.io/agent-lightning/0.3.0/reference/semconv/#agentlightning.semconv "Permanent link") Semantic conventions for Agent-lightning spans. Conventions in this file are added on demand. We generally DO NOT add new semantic conventions unless it's absolutely needed for certain algorithms or scenarios. #### `AGL_ANNOTATION = 'agentlightning.annotation'` `module-attribute` [¶](https://microsoft.github.io/agent-lightning/0.3.0/reference/semconv/#agentlightning.semconv.AGL_ANNOTATION "Permanent link") Agent-lightning's standard span name for annotations. Annotations are minimal span units for rewards, tags, and metadatas. They are used to "annotate" a specific event or a part of rollout. #### `AGL_EXCEPTION = 'agentlightning.exception'` `module-attribute` [¶](https://microsoft.github.io/agent-lightning/0.3.0/reference/semconv/#agentlightning.semconv.AGL_EXCEPTION "Permanent link") Agent-lightning's standard span name for exceptions. Used by the exception emitter to record exception details. #### `AGL_MESSAGE = 'agentlightning.message'` `module-attribute` [¶](https://microsoft.github.io/agent-lightning/0.3.0/reference/semconv/#agentlightning.semconv.AGL_MESSAGE "Permanent link") Agent-lightning's standard span name for messages and logs. #### `AGL_OBJECT = 'agentlightning.object'` `module-attribute` [¶](https://microsoft.github.io/agent-lightning/0.3.0/reference/semconv/#agentlightning.semconv.AGL_OBJECT "Permanent link") Agent-lightning's standard span name for customized objects. #### `AGL_OPERATION = 'agentlightning.operation'` `module-attribute` [¶](https://microsoft.github.io/agent-lightning/0.3.0/reference/semconv/#agentlightning.semconv.AGL_OPERATION "Permanent link") Agent-lightning's standard span name for functions. Wrap function or code-blocks as operations. #### `AGL_REWARD = 'agentlightning.reward'` `module-attribute` [¶](https://microsoft.github.io/agent-lightning/0.3.0/reference/semconv/#agentlightning.semconv.AGL_REWARD "Permanent link") Agent-lightning's standard span name for reward operations. #### `AGL_VIRTUAL = 'agentlightning.virtual'` `module-attribute` [¶](https://microsoft.github.io/agent-lightning/0.3.0/reference/semconv/#agentlightning.semconv.AGL_VIRTUAL "Permanent link") Agent-lightning's standard span name for virtual operations. Mostly used in adapter when needing to represent the root or intermediate operations. #### `LightningResourceAttributes` [¶](https://microsoft.github.io/agent-lightning/0.3.0/reference/semconv/#agentlightning.semconv.LightningResourceAttributes "Permanent link") Bases: `Enum` Resource attribute names used in Agent-lightning spans. ##### `ATTEMPT_ID = 'agentlightning.attempt_id'` `class-attribute` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/0.3.0/reference/semconv/#agentlightning.semconv.LightningResourceAttributes.ATTEMPT_ID "Permanent link") Resource name for attempt ID in Agent-lightning spans. ##### `ROLLOUT_ID = 'agentlightning.rollout_id'` `class-attribute` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/0.3.0/reference/semconv/#agentlightning.semconv.LightningResourceAttributes.ROLLOUT_ID "Permanent link") Resource name for rollout ID in Agent-lightning spans. ##### `SPAN_SEQUENCE_ID = 'agentlightning.span_sequence_id'` `class-attribute` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/0.3.0/reference/semconv/#agentlightning.semconv.LightningResourceAttributes.SPAN_SEQUENCE_ID "Permanent link") Resource name for span sequence ID in Agent-lightning spans. ##### `TRACER_NAME = 'agentlightning.tracer.name'` `class-attribute` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/0.3.0/reference/semconv/#agentlightning.semconv.LightningResourceAttributes.TRACER_NAME "Permanent link") Which tracer is used to create this span. #### `LightningSpanAttributes` [¶](https://microsoft.github.io/agent-lightning/0.3.0/reference/semconv/#agentlightning.semconv.LightningSpanAttributes "Permanent link") Bases: `Enum` Attribute names that commonly appear in Agent-lightning spans. Exception types can't be found here because they are defined in OpenTelemetry's official semantic conventions. ##### `LINK = 'agentlightning.link'` `class-attribute` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/0.3.0/reference/semconv/#agentlightning.semconv.LightningSpanAttributes.LINK "Permanent link") Attribute name for linking the current span to another span or other objects like requests/responses. ##### `MESSAGE_BODY = 'agentlightning.message.body'` `class-attribute` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/0.3.0/reference/semconv/#agentlightning.semconv.LightningSpanAttributes.MESSAGE_BODY "Permanent link") Attribute name for message text in message spans. ##### `OBJECT_JSON = 'agentlightning.object.json'` `class-attribute` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/0.3.0/reference/semconv/#agentlightning.semconv.LightningSpanAttributes.OBJECT_JSON "Permanent link") Attribute name for object serialized value (JSON) in object spans. ##### `OBJECT_LITERAL = 'agentlightning.object.literal'` `class-attribute` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/0.3.0/reference/semconv/#agentlightning.semconv.LightningSpanAttributes.OBJECT_LITERAL "Permanent link") Attribute name for object literal value in object spans (for str, int, bool, ...). ##### `OBJECT_TYPE = 'agentlightning.object.type'` `class-attribute` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/0.3.0/reference/semconv/#agentlightning.semconv.LightningSpanAttributes.OBJECT_TYPE "Permanent link") Attribute name for object type (full qualified name) in object spans. I think builtin types like str, int, bool, list, dict are self-explanatory and should also be qualified to use here. ##### `OPERATION_INPUT = 'agentlightning.operation.input'` `class-attribute` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/0.3.0/reference/semconv/#agentlightning.semconv.LightningSpanAttributes.OPERATION_INPUT "Permanent link") Attribute name for operation input in operation spans. ##### `OPERATION_NAME = 'agentlightning.operation.name'` `class-attribute` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/0.3.0/reference/semconv/#agentlightning.semconv.LightningSpanAttributes.OPERATION_NAME "Permanent link") Attribute name for operation name in operation spans, normally the function name. ##### `OPERATION_OUTPUT = 'agentlightning.operation.output'` `class-attribute` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/0.3.0/reference/semconv/#agentlightning.semconv.LightningSpanAttributes.OPERATION_OUTPUT "Permanent link") Attribute name for operation output in operation spans. ##### `REWARD = 'agentlightning.reward'` `class-attribute` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/0.3.0/reference/semconv/#agentlightning.semconv.LightningSpanAttributes.REWARD "Permanent link") Attribute prefix for rewards-related data in reward spans. It should be used as a prefix. For example, "agentlightning.reward.0.value" can be used to track a specific metric. See [RewardAttributes](https://microsoft.github.io/agent-lightning/0.3.0/reference/semconv/#agentlightning.semconv.RewardAttributes " RewardAttributes") . ##### `TAG = 'agentlightning.tag'` `class-attribute` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/0.3.0/reference/semconv/#agentlightning.semconv.LightningSpanAttributes.TAG "Permanent link") Attribute name for tagging spans with customized strings. #### `LinkAttributes` [¶](https://microsoft.github.io/agent-lightning/0.3.0/reference/semconv/#agentlightning.semconv.LinkAttributes "Permanent link") Bases: `Enum` Standard link types used in Agent-lightning spans. The link is more powerful than [OpenTelemetry link](https://opentelemetry.io/docs/specs/otel/trace/api/#link) in that it supports linking to a queryset of spans. It can even link to span object that hasn't been emitted yet. ##### `KEY_MATCH = 'key_match'` `class-attribute` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/0.3.0/reference/semconv/#agentlightning.semconv.LinkAttributes.KEY_MATCH "Permanent link") Linking to spans with matching attribute keys. `trace_id` and `span_id` are reserved and will be used to link to specific spans directly. For example, it can be `gen_ai.response.id` if intended to be link to a chat completion response span. Or it can be `span_id` to link to a specific span by its ID. ##### `VALUE_MATCH = 'value_match'` `class-attribute` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/0.3.0/reference/semconv/#agentlightning.semconv.LinkAttributes.VALUE_MATCH "Permanent link") Linking to spans with corresponding attribute values on those keys. #### `LinkPydanticModel` [¶](https://microsoft.github.io/agent-lightning/0.3.0/reference/semconv/#agentlightning.semconv.LinkPydanticModel "Permanent link") Bases: `BaseModel` A stricter implementation of LinkAttributes used in otel helpers. ##### `key_match` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/0.3.0/reference/semconv/#agentlightning.semconv.LinkPydanticModel.key_match "Permanent link") The attribute key to match on the target spans. ##### `value_match` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/0.3.0/reference/semconv/#agentlightning.semconv.LinkPydanticModel.value_match "Permanent link") The attribute value to match on the target spans. #### `RewardAttributes` [¶](https://microsoft.github.io/agent-lightning/0.3.0/reference/semconv/#agentlightning.semconv.RewardAttributes "Permanent link") Bases: `Enum` Multi-dimensional reward attributes will look like: `[](https://microsoft.github.io/agent-lightning/0.3.0/reference/semconv/#__codelineno-0-1) {"agentlightning.reward.0.name": "efficiency", "agentlightning.reward.0.value": 0.75}` The first reward in the reward list will automatically be the primary reward. If the reward list has greater than 1, it shall be a multi-dimensional case. ##### `REWARD_NAME = 'name'` `class-attribute` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/0.3.0/reference/semconv/#agentlightning.semconv.RewardAttributes.REWARD_NAME "Permanent link") Key for each dimension in multi-dimensional reward spans. ##### `REWARD_VALUE = 'value'` `class-attribute` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/0.3.0/reference/semconv/#agentlightning.semconv.RewardAttributes.REWARD_VALUE "Permanent link") Value for each dimension in multi-dimensional reward spans. #### `RewardPydanticModel` [¶](https://microsoft.github.io/agent-lightning/0.3.0/reference/semconv/#agentlightning.semconv.RewardPydanticModel "Permanent link") Bases: `BaseModel` A stricter implementation of RewardAttributes used in otel helpers. ##### `name` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/0.3.0/reference/semconv/#agentlightning.semconv.RewardPydanticModel.name "Permanent link") Name of the reward dimension. ##### `value` `instance-attribute` [¶](https://microsoft.github.io/agent-lightning/0.3.0/reference/semconv/#agentlightning.semconv.RewardPydanticModel.value "Permanent link") Value of the reward dimension. Back to top --- # VERL - Agent-lightning [Skip to content](https://microsoft.github.io/agent-lightning/0.3.0/algorithm-zoo/verl/#verl) ⚠️ You are viewing development documentation. For stable documentation, please visit [https://microsoft.github.io/agent-lightning/stable/](https://microsoft.github.io/agent-lightning/stable/) VERL[¶](https://microsoft.github.io/agent-lightning/0.3.0/algorithm-zoo/verl/#verl "Permanent link") ===================================================================================================== Shortcut You can use the shortcut `agl.VERL(...)` to create a VERL instance. `[](https://microsoft.github.io/agent-lightning/0.3.0/algorithm-zoo/verl/#__codelineno-0-1) import agentlightning as agl [](https://microsoft.github.io/agent-lightning/0.3.0/algorithm-zoo/verl/#__codelineno-0-2) [](https://microsoft.github.io/agent-lightning/0.3.0/algorithm-zoo/verl/#__codelineno-0-3) agl.VERL(...)` Installation[¶](https://microsoft.github.io/agent-lightning/0.3.0/algorithm-zoo/verl/#installation "Permanent link") --------------------------------------------------------------------------------------------------------------------- `[](https://microsoft.github.io/agent-lightning/0.3.0/algorithm-zoo/verl/#__codelineno-1-1) pip install agentlightning[verl]` Warning To avoid various compatibility issues, follow the steps in the [installation guide](https://microsoft.github.io/agent-lightning/0.3.0/tutorials/installation/) to set up VERL and its dependencies. Installing VERL directly with `pip install agentlightning[verl]` can cause issues unless you already have a compatible version of PyTorch installed. Notes for Readers [VERL](https://microsoft.github.io/agent-lightning/0.3.0/algorithm-zoo/verl/#agentlightning.algorithm.verl.VERL " VERL") in this article refers to a wrapper, provided by Agent-lightning, of the [VERL framework](https://github.com/volcengine/verl) . It's a subclass of [agentlightning.Algorithm](https://microsoft.github.io/agent-lightning/0.3.0/reference/algorithm/#agentlightning.Algorithm " agentlightning.Algorithm") . To differentiate it from the VERL framework, all references to the VERL framework shall use the term "VERL framework", and all references to the Agent-lightning wrapper shall be highlighted with a link. Resources[¶](https://microsoft.github.io/agent-lightning/0.3.0/algorithm-zoo/verl/#resources "Permanent link") --------------------------------------------------------------------------------------------------------------- [VERL](https://microsoft.github.io/agent-lightning/0.3.0/algorithm-zoo/verl/#agentlightning.algorithm.verl.VERL " VERL") expects no initial resources. The first LLM endpoint is directly deployed from the VERL configuration (`.actor_rollout_ref.model.path`). The resource key is always `main_llm`. [VERL](https://microsoft.github.io/agent-lightning/0.3.0/algorithm-zoo/verl/#agentlightning.algorithm.verl.VERL " VERL") currently does not support optimizing multiple [LLM](https://microsoft.github.io/agent-lightning/0.3.0/reference/types/#agentlightning.LLM " agentlightning.LLM") s together. Note The resource type created by [VERL](https://microsoft.github.io/agent-lightning/0.3.0/algorithm-zoo/verl/#agentlightning.algorithm.verl.VERL " VERL") is actually a [ProxyLLM](https://microsoft.github.io/agent-lightning/0.3.0/reference/types/#agentlightning.ProxyLLM " agentlightning.ProxyLLM") , a subclass of the [LLM](https://microsoft.github.io/agent-lightning/0.3.0/reference/types/#agentlightning.LLM " agentlightning.LLM") type. This object contains a **URL template** provided by [VERL](https://microsoft.github.io/agent-lightning/0.3.0/algorithm-zoo/verl/#agentlightning.algorithm.verl.VERL " VERL") , with placeholders for rollout and attempt IDs. When a rollout begins on the agent side, the framework uses the current `rollout_id` and `attempt_id` to format this template, generating a final, unique endpoint URL. This URL points to [VERL](https://microsoft.github.io/agent-lightning/0.3.0/algorithm-zoo/verl/#agentlightning.algorithm.verl.VERL " VERL") 's internal proxy, allowing it to intercept and log all traffic for that specific attempt, for tracing and load balancing purposes. For agents created with the `@rollout` decorator, this resolution of the template is handled automatically ("auto-stripped"). Class-based agents will need to manually resolve the [`ProxyLLM`](https://microsoft.github.io/agent-lightning/0.3.0/reference/types/#agentlightning.ProxyLLM " agentlightning.ProxyLLM") using the rollout context. `[](https://microsoft.github.io/agent-lightning/0.3.0/algorithm-zoo/verl/#__codelineno-2-1) proxy_llm = resources["main_llm"] [](https://microsoft.github.io/agent-lightning/0.3.0/algorithm-zoo/verl/#__codelineno-2-2) proxy_llm.get_base_url(rollout.rollout_id, rollout.attempt.attempt_id)` Customization[¶](https://microsoft.github.io/agent-lightning/0.3.0/algorithm-zoo/verl/#customization "Permanent link") ----------------------------------------------------------------------------------------------------------------------- Internally, [VERL](https://microsoft.github.io/agent-lightning/0.3.0/algorithm-zoo/verl/#agentlightning.algorithm.verl.VERL " VERL") decomposes each agent execution into prompt–response pairs via the [Adapter](https://microsoft.github.io/agent-lightning/0.3.0/reference/algorithm/#agentlightning.Adapter " agentlightning.Adapter") and associates them with their corresponding reward signals as [Triplet](https://microsoft.github.io/agent-lightning/0.3.0/reference/types/#agentlightning.Triplet " agentlightning.Triplet") objects. The final scalar reward, derived from the last triplet in the trajectory, is propagated to all preceding triplets following the [identical assignment strategy](https://arxiv.org/abs/2508.03680) . This ensures that each triplet receives an identical reward signal and can be independently optimized as a valid RLHF trajectory within the VERL framework. At present, [VERL](https://microsoft.github.io/agent-lightning/0.3.0/algorithm-zoo/verl/#agentlightning.algorithm.verl.VERL " VERL") does not expose fine-grained control over its reward propagation or credit assignment mechanisms. Users requiring customized reward shaping or trajectory decomposition are advised to clone and modify the [VERL](https://microsoft.github.io/agent-lightning/0.3.0/algorithm-zoo/verl/#agentlightning.algorithm.verl.VERL " VERL") source implementation directly. Tutorials Using VERL[¶](https://microsoft.github.io/agent-lightning/0.3.0/algorithm-zoo/verl/#tutorials-using-verl "Permanent link") ------------------------------------------------------------------------------------------------------------------------------------- * [Train SQL Agent with RL](https://microsoft.github.io/agent-lightning/0.3.0/how-to/train-sql-agent/) - A practical example of training a SQL agent using VERL. References - Entrypoint[¶](https://microsoft.github.io/agent-lightning/0.3.0/algorithm-zoo/verl/#references-entrypoint "Permanent link") ----------------------------------------------------------------------------------------------------------------------------------------- ### `agentlightning.algorithm.verl` [¶](https://microsoft.github.io/agent-lightning/0.3.0/algorithm-zoo/verl/#agentlightning.algorithm.verl "Permanent link") #### `VERL` [¶](https://microsoft.github.io/agent-lightning/0.3.0/algorithm-zoo/verl/#agentlightning.algorithm.verl.VERL "Permanent link") Bases: `[Algorithm](https://microsoft.github.io/agent-lightning/0.3.0/reference/algorithm/#agentlightning.Algorithm " agentlightning.Algorithm (agentlightning.algorithm.base.Algorithm)") ` VERL-powered algorithm that delegates training to the VERL PPO runner. Warning Advanced customisation currently requires copying the VERL source and modifying it directly. Native hooks for overriding training behaviour will land in a future release. Parameters: * **`config`** (`dict[str, Any]`) – Dictionary mirroring the overrides passed to the VERL CLI. The overrides are merged with VERL's packaged defaults via Hydra before launching training. * **`trainer_cls`** (`Optional[Type[[AgentLightningTrainer](https://microsoft.github.io/agent-lightning/0.3.0/algorithm-zoo/verl/#agentlightning.verl.AgentLightningTrainer " AgentLightningTrainer (agentlightning.verl.trainer.AgentLightningTrainer)") ]]`, default: `None` ) – Optional override for the trainer class. Experimental. * **`daemon_cls`** (`Optional[Type[[AgentModeDaemon](https://microsoft.github.io/agent-lightning/0.3.0/algorithm-zoo/verl/#agentlightning.verl.AgentModeDaemon " AgentModeDaemon (agentlightning.verl.daemon.AgentModeDaemon)") ]]`, default: `None` ) – Optional override for the daemon class. Experimental. Trajectory aggregation (experimental) Trajectory-level aggregation merges an entire multi-turn rollout into a single, masked training sample so GPU time is spent once per trajectory rather than N times per turn. Enable it via: `[](https://microsoft.github.io/agent-lightning/0.3.0/algorithm-zoo/verl/#__codelineno-0-1) config["agentlightning"]["trace_aggregator"] = { [](https://microsoft.github.io/agent-lightning/0.3.0/algorithm-zoo/verl/#__codelineno-0-2) "level": "trajectory", [](https://microsoft.github.io/agent-lightning/0.3.0/algorithm-zoo/verl/#__codelineno-0-3) "trajectory_max_prompt_length": 4096, [](https://microsoft.github.io/agent-lightning/0.3.0/algorithm-zoo/verl/#__codelineno-0-4) "trajectory_max_response_length": 34384, [](https://microsoft.github.io/agent-lightning/0.3.0/algorithm-zoo/verl/#__codelineno-0-5) }` Keep conversations structured (message lists rather than manual string concatenation) so prefix matching can stitch traces. `trajectory_max_prompt_length` should be set to the maximum length of the prompt for the first turn, and `trajectory_max_response_length` should be set to the maximum cumulative length of agent responses in the full trajectory. Toggle `debug=True` plus `mismatch_log_dir` when you need to inspect retokenization or chat-template mismatches. See [this blog post](https://agent-lightning.github.io/posts/trajectory_level_aggregation/) for more details. Examples: `[](https://microsoft.github.io/agent-lightning/0.3.0/algorithm-zoo/verl/#__codelineno-0-1) from agentlightning.algorithm.verl import VERL [](https://microsoft.github.io/agent-lightning/0.3.0/algorithm-zoo/verl/#__codelineno-0-2) [](https://microsoft.github.io/agent-lightning/0.3.0/algorithm-zoo/verl/#__codelineno-0-3) algorithm = VERL( [](https://microsoft.github.io/agent-lightning/0.3.0/algorithm-zoo/verl/#__codelineno-0-4) config={ [](https://microsoft.github.io/agent-lightning/0.3.0/algorithm-zoo/verl/#__codelineno-0-5) "algorithm": { [](https://microsoft.github.io/agent-lightning/0.3.0/algorithm-zoo/verl/#__codelineno-0-6) "adv_estimator": "grpo", [](https://microsoft.github.io/agent-lightning/0.3.0/algorithm-zoo/verl/#__codelineno-0-7) "use_kl_in_reward": False, [](https://microsoft.github.io/agent-lightning/0.3.0/algorithm-zoo/verl/#__codelineno-0-8) }, [](https://microsoft.github.io/agent-lightning/0.3.0/algorithm-zoo/verl/#__codelineno-0-9) "data": { [](https://microsoft.github.io/agent-lightning/0.3.0/algorithm-zoo/verl/#__codelineno-0-10) "train_batch_size": 32, [](https://microsoft.github.io/agent-lightning/0.3.0/algorithm-zoo/verl/#__codelineno-0-11) "max_prompt_length": 4096, [](https://microsoft.github.io/agent-lightning/0.3.0/algorithm-zoo/verl/#__codelineno-0-12) "max_response_length": 2048, [](https://microsoft.github.io/agent-lightning/0.3.0/algorithm-zoo/verl/#__codelineno-0-13) }, [](https://microsoft.github.io/agent-lightning/0.3.0/algorithm-zoo/verl/#__codelineno-0-14) "actor_rollout_ref": { [](https://microsoft.github.io/agent-lightning/0.3.0/algorithm-zoo/verl/#__codelineno-0-15) "rollout": { [](https://microsoft.github.io/agent-lightning/0.3.0/algorithm-zoo/verl/#__codelineno-0-16) "tensor_model_parallel_size": 1, [](https://microsoft.github.io/agent-lightning/0.3.0/algorithm-zoo/verl/#__codelineno-0-17) "n": 4, [](https://microsoft.github.io/agent-lightning/0.3.0/algorithm-zoo/verl/#__codelineno-0-18) "log_prob_micro_batch_size_per_gpu": 4, [](https://microsoft.github.io/agent-lightning/0.3.0/algorithm-zoo/verl/#__codelineno-0-19) "multi_turn": {"format": "hermes"}, [](https://microsoft.github.io/agent-lightning/0.3.0/algorithm-zoo/verl/#__codelineno-0-20) "name": "vllm", [](https://microsoft.github.io/agent-lightning/0.3.0/algorithm-zoo/verl/#__codelineno-0-21) "gpu_memory_utilization": 0.6, [](https://microsoft.github.io/agent-lightning/0.3.0/algorithm-zoo/verl/#__codelineno-0-22) }, [](https://microsoft.github.io/agent-lightning/0.3.0/algorithm-zoo/verl/#__codelineno-0-23) "actor": { [](https://microsoft.github.io/agent-lightning/0.3.0/algorithm-zoo/verl/#__codelineno-0-24) "ppo_mini_batch_size": 32, [](https://microsoft.github.io/agent-lightning/0.3.0/algorithm-zoo/verl/#__codelineno-0-25) "ppo_micro_batch_size_per_gpu": 4, [](https://microsoft.github.io/agent-lightning/0.3.0/algorithm-zoo/verl/#__codelineno-0-26) "optim": {"lr": 1e-6}, [](https://microsoft.github.io/agent-lightning/0.3.0/algorithm-zoo/verl/#__codelineno-0-27) "use_kl_loss": False, [](https://microsoft.github.io/agent-lightning/0.3.0/algorithm-zoo/verl/#__codelineno-0-28) "kl_loss_coef": 0.0, [](https://microsoft.github.io/agent-lightning/0.3.0/algorithm-zoo/verl/#__codelineno-0-29) "entropy_coeff": 0, [](https://microsoft.github.io/agent-lightning/0.3.0/algorithm-zoo/verl/#__codelineno-0-30) "clip_ratio_low": 0.2, [](https://microsoft.github.io/agent-lightning/0.3.0/algorithm-zoo/verl/#__codelineno-0-31) "clip_ratio_high": 0.3, [](https://microsoft.github.io/agent-lightning/0.3.0/algorithm-zoo/verl/#__codelineno-0-32) "fsdp_config": { [](https://microsoft.github.io/agent-lightning/0.3.0/algorithm-zoo/verl/#__codelineno-0-33) "param_offload": True, [](https://microsoft.github.io/agent-lightning/0.3.0/algorithm-zoo/verl/#__codelineno-0-34) "optimizer_offload": True, [](https://microsoft.github.io/agent-lightning/0.3.0/algorithm-zoo/verl/#__codelineno-0-35) }, [](https://microsoft.github.io/agent-lightning/0.3.0/algorithm-zoo/verl/#__codelineno-0-36) }, [](https://microsoft.github.io/agent-lightning/0.3.0/algorithm-zoo/verl/#__codelineno-0-37) "ref": { [](https://microsoft.github.io/agent-lightning/0.3.0/algorithm-zoo/verl/#__codelineno-0-38) "log_prob_micro_batch_size_per_gpu": 8, [](https://microsoft.github.io/agent-lightning/0.3.0/algorithm-zoo/verl/#__codelineno-0-39) "fsdp_config": {"param_offload": True}, [](https://microsoft.github.io/agent-lightning/0.3.0/algorithm-zoo/verl/#__codelineno-0-40) }, [](https://microsoft.github.io/agent-lightning/0.3.0/algorithm-zoo/verl/#__codelineno-0-41) "model": { [](https://microsoft.github.io/agent-lightning/0.3.0/algorithm-zoo/verl/#__codelineno-0-42) "path": "Qwen/Qwen2.5-1.5B-Instruct", [](https://microsoft.github.io/agent-lightning/0.3.0/algorithm-zoo/verl/#__codelineno-0-43) "use_remove_padding": True, [](https://microsoft.github.io/agent-lightning/0.3.0/algorithm-zoo/verl/#__codelineno-0-44) "enable_gradient_checkpointing": True, [](https://microsoft.github.io/agent-lightning/0.3.0/algorithm-zoo/verl/#__codelineno-0-45) }, [](https://microsoft.github.io/agent-lightning/0.3.0/algorithm-zoo/verl/#__codelineno-0-46) }, [](https://microsoft.github.io/agent-lightning/0.3.0/algorithm-zoo/verl/#__codelineno-0-47) "trainer": { [](https://microsoft.github.io/agent-lightning/0.3.0/algorithm-zoo/verl/#__codelineno-0-48) "n_gpus_per_node": 1, [](https://microsoft.github.io/agent-lightning/0.3.0/algorithm-zoo/verl/#__codelineno-0-49) "val_before_train": True, [](https://microsoft.github.io/agent-lightning/0.3.0/algorithm-zoo/verl/#__codelineno-0-50) "critic_warmup": 0, [](https://microsoft.github.io/agent-lightning/0.3.0/algorithm-zoo/verl/#__codelineno-0-51) "logger": ["console", "wandb"], [](https://microsoft.github.io/agent-lightning/0.3.0/algorithm-zoo/verl/#__codelineno-0-52) "project_name": "AgentLightning", [](https://microsoft.github.io/agent-lightning/0.3.0/algorithm-zoo/verl/#__codelineno-0-53) "experiment_name": "calc_x", [](https://microsoft.github.io/agent-lightning/0.3.0/algorithm-zoo/verl/#__codelineno-0-54) "nnodes": 1, [](https://microsoft.github.io/agent-lightning/0.3.0/algorithm-zoo/verl/#__codelineno-0-55) "save_freq": 64, [](https://microsoft.github.io/agent-lightning/0.3.0/algorithm-zoo/verl/#__codelineno-0-56) "test_freq": 32, [](https://microsoft.github.io/agent-lightning/0.3.0/algorithm-zoo/verl/#__codelineno-0-57) "total_epochs": 2, [](https://microsoft.github.io/agent-lightning/0.3.0/algorithm-zoo/verl/#__codelineno-0-58) }, [](https://microsoft.github.io/agent-lightning/0.3.0/algorithm-zoo/verl/#__codelineno-0-59) } [](https://microsoft.github.io/agent-lightning/0.3.0/algorithm-zoo/verl/#__codelineno-0-60) ) [](https://microsoft.github.io/agent-lightning/0.3.0/algorithm-zoo/verl/#__codelineno-0-61) trainer.fit(algorithm, train_dataset=my_train_dataset)` ##### `get_client()` [¶](https://microsoft.github.io/agent-lightning/0.3.0/algorithm-zoo/verl/#agentlightning.algorithm.verl.VERL.get_client "Permanent link") Create a client bound to the VERL-managed Agent Lightning server. Deprecated Since v0.2. ##### `run(train_dataset=None, val_dataset=None)` [¶](https://microsoft.github.io/agent-lightning/0.3.0/algorithm-zoo/verl/#agentlightning.algorithm.verl.VERL.run "Permanent link") Launch the VERL PPO entrypoint with the configured runtime context. Parameters: * **`train_dataset`** (`Optional[[Dataset](https://microsoft.github.io/agent-lightning/0.3.0/reference/types/#agentlightning.Dataset " agentlightning.Dataset (agentlightning.types.Dataset)") [Any]]`, default: `None` ) – Optional dataset forwarded to VERL for training. * **`val_dataset`** (`Optional[[Dataset](https://microsoft.github.io/agent-lightning/0.3.0/reference/types/#agentlightning.Dataset " agentlightning.Dataset (agentlightning.types.Dataset)") [Any]]`, default: `None` ) – Optional dataset forwarded to VERL for evaluation. Raises: * `ValueError` – If required dependencies such as the store, LLM proxy, or adapter have been garbage-collected when using the V1 execution mode. References - Implementation[¶](https://microsoft.github.io/agent-lightning/0.3.0/algorithm-zoo/verl/#references-implementation "Permanent link") ------------------------------------------------------------------------------------------------------------------------------------------------- ### `agentlightning.verl` [¶](https://microsoft.github.io/agent-lightning/0.3.0/algorithm-zoo/verl/#agentlightning.verl "Permanent link") This package contains a _hacky_ integration of VERL with Agent Lightning. #### `AgentLightningTrainer` [¶](https://microsoft.github.io/agent-lightning/0.3.0/algorithm-zoo/verl/#agentlightning.verl.AgentLightningTrainer "Permanent link") Bases: `RayPPOTrainer` Specialized PPO trainer for agent-based reinforcement learning. This trainer is designed specifically for scenarios where the model interacts with external environments, tools, or APIs through an AgentLightningServer. It simplifies the training loop by removing the complex conditional logic present in the original RayPPOTrainer and focusing on the agent mode workflow. Key differences from RayPPOTrainer: 1. Uses AgentModeDaemon for server communication 2. Simplified data flow without pop/union operations 3. Direct batch processing through agent daemon 4. Streamlined validation using agent\_mode validation #### `AgentModeDaemon` [¶](https://microsoft.github.io/agent-lightning/0.3.0/algorithm-zoo/verl/#agentlightning.verl.AgentModeDaemon "Permanent link") AgentModeDaemon using the AgentLightningServer SDK. This class manages the server lifecycle, task queueing, and results retrieval, while also running a proxy server for LLM requests. It maintains the original interface for compatibility with the RayPPOTrainer. ##### `clear_data_and_server()` [¶](https://microsoft.github.io/agent-lightning/0.3.0/algorithm-zoo/verl/#agentlightning.verl.AgentModeDaemon.clear_data_and_server "Permanent link") Resets the internal state of the daemon for the next run. ##### `get_test_metrics()` [¶](https://microsoft.github.io/agent-lightning/0.3.0/algorithm-zoo/verl/#agentlightning.verl.AgentModeDaemon.get_test_metrics "Permanent link") Calculates and returns metrics for a validation run. ##### `get_train_data_batch(max_prompt_length, max_response_length, device, global_steps)` [¶](https://microsoft.github.io/agent-lightning/0.3.0/algorithm-zoo/verl/#agentlightning.verl.AgentModeDaemon.get_train_data_batch "Permanent link") Processes completed rollouts to generate a training data batch. This function reconstructs the logic from the original AgentModeDaemon, using data retrieved from the new server architecture. It handles padding, truncation, and tensor creation for the PPO training loop. ##### `run_until_all_finished(verbose=True)` [¶](https://microsoft.github.io/agent-lightning/0.3.0/algorithm-zoo/verl/#agentlightning.verl.AgentModeDaemon.run_until_all_finished "Permanent link") Synchronously waits for all queued tasks to be completed and reported. ##### `set_up_data_and_server(data, server_addresses, is_train=True)` [¶](https://microsoft.github.io/agent-lightning/0.3.0/algorithm-zoo/verl/#agentlightning.verl.AgentModeDaemon.set_up_data_and_server "Permanent link") Synchronous wrapper for setting up data and server resources. ##### `start()` [¶](https://microsoft.github.io/agent-lightning/0.3.0/algorithm-zoo/verl/#agentlightning.verl.AgentModeDaemon.start "Permanent link") Starts the main AgentLightningServer and the proxy server. #### `get_left_padded_ids_and_attention_mask(ids, max_length, pad_token_id)` [¶](https://microsoft.github.io/agent-lightning/0.3.0/algorithm-zoo/verl/#agentlightning.verl.get_left_padded_ids_and_attention_mask "Permanent link") Left-pad (or truncate) a sequence of token IDs to a fixed length, and build the corresponding attention mask. Parameters: * **`ids`** (`List[int]`) – the original list of token IDs. * **`max_length`** (`int`) – desired total length after padding/truncation. * **`pad_token_id`** (`int`) – ID to use for padding. Returns: * **`padded_ids`** ( `any` ) – list of length == max\_length. * **`attention_mask`** ( `any` ) – list of same length: 1 for non-pad tokens, 0 for pads. #### `get_right_padded_ids_and_attention_mask(ids, max_length, pad_token_id)` [¶](https://microsoft.github.io/agent-lightning/0.3.0/algorithm-zoo/verl/#agentlightning.verl.get_right_padded_ids_and_attention_mask "Permanent link") Right-pad (or truncate) a sequence of token IDs to a fixed length, and build the corresponding attention mask. Parameters: * **`ids`** (`List[int]`) – the original list of token IDs. * **`max_length`** (`int`) – desired total length after padding/truncation. * **`pad_token_id`** (`int`) – ID to use for padding. Returns: * **`padded_ids`** ( `any` ) – list of length == max\_length. * **`attention_mask`** ( `any` ) – list of same length: 1 for non-pad tokens, 0 for pads. Back to top --- # Debugging - Agent-lightning [Skip to content](https://microsoft.github.io/agent-lightning/0.3.0/tutorials/debug/#debugging-and-troubleshooting) ⚠️ You are viewing development documentation. For stable documentation, please visit [https://microsoft.github.io/agent-lightning/stable/](https://microsoft.github.io/agent-lightning/stable/) Debugging and Troubleshooting[¶](https://microsoft.github.io/agent-lightning/0.3.0/tutorials/debug/#debugging-and-troubleshooting "Permanent link") ==================================================================================================================================================== When you train your own agent with Agent-lightning, most failures surface because the agent logic is brittle or simply incorrect. Debugging becomes easier when you peel back the stack: start by driving the rollout logic on its own, dry-run the trainer loop, and only then bring the full algorithm and runner topology online. The [`examples/apo/apo_debug.py`](https://github.com/microsoft/agent-lightning/tree/3b5d733861cf313fc09821a23240bbdf3cb2ee5b/examples/apo/apo_debug.py) script demonstrates these techniques; this guide expands on each approach and helps you decide when to reach for them. Debugging with Dashboard[¶](https://microsoft.github.io/agent-lightning/0.3.0/tutorials/debug/#debugging-with-dashboard "Permanent link") ------------------------------------------------------------------------------------------------------------------------------------------ When you launch an experiment with [`Trainer.fit`](https://microsoft.github.io/agent-lightning/0.3.0/reference/trainer/#agentlightning.Trainer.fit " fit(agent, train_dataset=None, *, val_dataset=None)") or start an isolated store via [`agl store`](https://microsoft.github.io/agent-lightning/0.3.0/reference/cli/) , the terminal prints a message similar to: `[](https://microsoft.github.io/agent-lightning/0.3.0/tutorials/debug/#__codelineno-0-1) INFO Agent-lightning dashboard will be available at http://192.168.0.107:4747` Visit that URL, and you will see the Agent-lightning dashboard: ![Dashboard](https://microsoft.github.io/agent-lightning/0.3.0/assets/dashboard-page-rollouts.png) The dashboard surfaces everything stored inside [the store](https://microsoft.github.io/agent-lightning/0.3.0/deep-dive/store/) . Because the store mediates interactions between algorithms and runners, inspecting it often reveals which side is causing issues such as stale rollouts, unresponsive workers, or empty traces. For example, the VERL algorithm may receive no token IDs and emit `cannot reshape tensor of 0 elements into shape [1, 0, -1, 128] because the unspecified dimension size -1 can be any value and is ambiguous` ([Issue #50](https://github.com/microsoft/agent-lightning/issues/50) , [Issue #76](https://github.com/microsoft/agent-lightning/issues/76) ). Several scenarios can produce that error: the runner might not produce trace spans at all, it might produce spans without token IDs, or the IDs may be present but formatted incorrectly. Inspecting the dashboard traces helps you pinpoint which condition applies. ![Dashboard Traces Page](https://microsoft.github.io/agent-lightning/0.3.0/assets/dashboard-page-traces.png) By checking whether the trace span is empty and whether token IDs appear in the span attributes, you can narrow the issue to either the runner (agent) side or the algorithm side. Then apply the techniques below to debug the faulty component. Debug-level Logging[¶](https://microsoft.github.io/agent-lightning/0.3.0/tutorials/debug/#debug-level-logging "Permanent link") -------------------------------------------------------------------------------------------------------------------------------- Starting from v0.3, detailed signals such as store server access logs, runner lifecycle logs, and span payloads only appear when the log level is `DEBUG` so the default output stays readable. Enable debug-level logging by adding the following snippet near the top of your script: `[](https://microsoft.github.io/agent-lightning/0.3.0/tutorials/debug/#__codelineno-1-1) import agentlightning as agl [](https://microsoft.github.io/agent-lightning/0.3.0/tutorials/debug/#__codelineno-1-2) [](https://microsoft.github.io/agent-lightning/0.3.0/tutorials/debug/#__codelineno-1-3) agl.setup_logging("DEBUG")` Set the log level on every process if your setup involves multiple workers. For example, when [running stores in isolation](https://microsoft.github.io/agent-lightning/0.3.0/tutorials/debug/#debug-with-external-store "Debug the Algorithm-Runner Boundary") , configure the store process explicitly: `[](https://microsoft.github.io/agent-lightning/0.3.0/tutorials/debug/#__codelineno-2-1) agl store --port 4747 --log-level DEBUG` Using [`Runner`](https://microsoft.github.io/agent-lightning/0.3.0/reference/runner/#agentlightning.Runner " agentlightning.Runner") in Isolation[¶](https://microsoft.github.io/agent-lightning/0.3.0/tutorials/debug/#using-runner-in-isolation "Permanent link") -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- [`Runner`](https://microsoft.github.io/agent-lightning/0.3.0/reference/runner/#agentlightning.Runner " agentlightning.Runner") is a long-lived worker that wraps your [`LitAgent`](https://microsoft.github.io/agent-lightning/0.3.0/reference/agent/#agentlightning.LitAgent " agentlightning.LitAgent") , coordinates tracing, and talks to the [`LightningStore`](https://microsoft.github.io/agent-lightning/0.3.0/reference/store/#agentlightning.LightningStore " agentlightning.LightningStore") . In typical training flows the trainer manages runners for you, but being able to spin one up manually is invaluable while debugging. If you define rollout logic with [`@rollout`](https://microsoft.github.io/agent-lightning/0.3.0/reference/agent/#agentlightning.rollout " agentlightning.rollout(func)") or implement a [`LitAgent`](https://microsoft.github.io/agent-lightning/0.3.0/reference/agent/#agentlightning.LitAgent " agentlightning.LitAgent") directly, you will get a [`LitAgent`](https://microsoft.github.io/agent-lightning/0.3.0/reference/agent/#agentlightning.LitAgent " agentlightning.LitAgent") instance and you should be able to execute it with [`LitAgentRunner`](https://microsoft.github.io/agent-lightning/0.3.0/reference/runner/#agentlightning.LitAgentRunner " agentlightning.LitAgentRunner") , which is a subclass of [`Runner`](https://microsoft.github.io/agent-lightning/0.3.0/reference/runner/#agentlightning.Runner " agentlightning.Runner") . The runner needs but does not instantiate a [`Tracer`](https://microsoft.github.io/agent-lightning/0.3.0/reference/runner/#agentlightning.Tracer " agentlightning.Tracer") , so supply one yourself. See [Working with Traces](https://microsoft.github.io/agent-lightning/0.3.0/tutorials/traces/) for a walkthrough of tracer options. [`Runner.run_context`](https://microsoft.github.io/agent-lightning/0.3.0/reference/runner/#agentlightning.Runner.run_context " run_context(*, agent, store, hooks=None, worker_id=None)") prepares the runner to execute a particular agent. Besides the agent and tracer you must provide a store that will collect spans and rollouts. [`InMemoryLightningStore`](https://microsoft.github.io/agent-lightning/0.3.0/reference/store/#agentlightning.InMemoryLightningStore " agentlightning.InMemoryLightningStore") keeps everything in-process, which is perfect for debugging sessions. `[](https://microsoft.github.io/agent-lightning/0.3.0/tutorials/debug/#__codelineno-3-1) import agentlightning as agl [](https://microsoft.github.io/agent-lightning/0.3.0/tutorials/debug/#__codelineno-3-2) [](https://microsoft.github.io/agent-lightning/0.3.0/tutorials/debug/#__codelineno-3-3) tracer = agl.OtelTracer() [](https://microsoft.github.io/agent-lightning/0.3.0/tutorials/debug/#__codelineno-3-4) runner = agl.LitAgentRunner(tracer) [](https://microsoft.github.io/agent-lightning/0.3.0/tutorials/debug/#__codelineno-3-5) store = agl.InMemoryLightningStore() [](https://microsoft.github.io/agent-lightning/0.3.0/tutorials/debug/#__codelineno-3-6) [](https://microsoft.github.io/agent-lightning/0.3.0/tutorials/debug/#__codelineno-3-7) with runner.run_context(agent=apo_rollout, store=store): [](https://microsoft.github.io/agent-lightning/0.3.0/tutorials/debug/#__codelineno-3-8) ...` Inside the [`run_context`](https://microsoft.github.io/agent-lightning/0.3.0/reference/runner/#agentlightning.Runner.run_context " run_context(*, agent, store, hooks=None, worker_id=None)") block you can call [`runner.step(...)`](https://microsoft.github.io/agent-lightning/0.3.0/reference/runner/#agentlightning.Runner.step " step(input, *, resources=None, mode=None, event=None) async ") to execute a single rollout. The payload includes the task input and any [`NamedResources`](https://microsoft.github.io/agent-lightning/0.3.0/reference/types/#agentlightning.NamedResources " agentlightning.NamedResources = Dict[str, ResourceUnion] module-attribute ") the agent expects. Read [introduction to Resources](https://microsoft.github.io/agent-lightning/0.3.0/how-to/write-first-algorithm/#introduction-to-resources "Resources: The Tunable Assets") and [NamedResources](https://microsoft.github.io/agent-lightning/0.3.0/tutorials/write-agents/#introduction-to-named-resources "Class-based Agents") for more details. For example, if your agent references a [`PromptTemplate`](https://microsoft.github.io/agent-lightning/0.3.0/reference/types/#agentlightning.PromptTemplate " agentlightning.PromptTemplate") , pass it through the `resources` argument: `[](https://microsoft.github.io/agent-lightning/0.3.0/tutorials/debug/#__codelineno-4-1) with runner.run_context(agent=apo_rollout, store=store): [](https://microsoft.github.io/agent-lightning/0.3.0/tutorials/debug/#__codelineno-4-2) resource = agl.PromptTemplate(template="You are a helpful assistant. {any_question}", engine="f-string") [](https://microsoft.github.io/agent-lightning/0.3.0/tutorials/debug/#__codelineno-4-3) rollout = await runner.step( [](https://microsoft.github.io/agent-lightning/0.3.0/tutorials/debug/#__codelineno-4-4) "Explain why the sky appears blue using principles of light scattering in 100 words.", [](https://microsoft.github.io/agent-lightning/0.3.0/tutorials/debug/#__codelineno-4-5) resources={"main_prompt": resource}, [](https://microsoft.github.io/agent-lightning/0.3.0/tutorials/debug/#__codelineno-4-6) )` You can do as many things as you want within the [`Runner.run_context`](https://microsoft.github.io/agent-lightning/0.3.0/reference/runner/#agentlightning.Runner.run_context " run_context(*, agent, store, hooks=None, worker_id=None)") block. After the rollout finishes you can query the store to inspect what happened: `[](https://microsoft.github.io/agent-lightning/0.3.0/tutorials/debug/#__codelineno-5-1) print(await store.query_rollouts()) [](https://microsoft.github.io/agent-lightning/0.3.0/tutorials/debug/#__codelineno-5-2) print(await store.query_spans(rollout.rollout_id))` Example output (with a reward span captured): `[](https://microsoft.github.io/agent-lightning/0.3.0/tutorials/debug/#__codelineno-6-1) [Rollout(rollout_id='ro-519769241af8', input='Explain why the sky appears blue using principles of light scattering in 100 words.', start_time=1760706315.6996238, ..., status='succeeded')] [](https://microsoft.github.io/agent-lightning/0.3.0/tutorials/debug/#__codelineno-6-2) [Span(rollout_id='ro-519769241af8', attempt_id='at-a6b62caf', sequence_id=1, ..., name='agentlightning.annotation', attributes={'agentlightning.reward.0.value': 0.95}, ...)]` Swap in an [`AgentOpsTracer`](https://microsoft.github.io/agent-lightning/0.3.0/reference/runner/#agentlightning.AgentOpsTracer " agentlightning.AgentOpsTracer") instead of [`OtelTracer`](https://microsoft.github.io/agent-lightning/0.3.0/reference/runner/#agentlightning.OtelTracer " agentlightning.OtelTracer") to see the underlying LLM spans alongside reward information: `[](https://microsoft.github.io/agent-lightning/0.3.0/tutorials/debug/#__codelineno-7-1) [ [](https://microsoft.github.io/agent-lightning/0.3.0/tutorials/debug/#__codelineno-7-2) Span(rollout_id='ro-519769241af8', attempt_id='at-a6b62caf', sequence_id=1, ..., name='openai.chat.completion', attributes={..., 'gen_ai.prompt.0.role': 'user', 'gen_ai.prompt.0.content': 'You are a helpful assistant. Explain why the sky appears blue using principles of light scattering in 100 words.', ...}), [](https://microsoft.github.io/agent-lightning/0.3.0/tutorials/debug/#__codelineno-7-3) Span(rollout_id='ro-519769241af8', attempt_id='at-a6b62caf', sequence_id=2, ..., name='openai.chat.completion', attributes={..., 'gen_ai.prompt.0.role': 'user', 'gen_ai.prompt.0.content': 'Evaluate how well the output fulfills the task...', ...}), [](https://microsoft.github.io/agent-lightning/0.3.0/tutorials/debug/#__codelineno-7-4) Span(rollout_id='ro-519769241af8', attempt_id='at-a6b62caf', sequence_id=3, ..., name='agentlightning.annotation', attributes={'agentlightning.reward.0.value': 0.95}, ...) [](https://microsoft.github.io/agent-lightning/0.3.0/tutorials/debug/#__codelineno-7-5) ]` Tip Spans too difficult to read? Try using [`Adapter`](https://microsoft.github.io/agent-lightning/0.3.0/reference/algorithm/#agentlightning.Adapter " agentlightning.Adapter") to convert them into a [more readable format](https://microsoft.github.io/agent-lightning/0.3.0/tutorials/traces/) . [`Runner.step`](https://microsoft.github.io/agent-lightning/0.3.0/reference/runner/#agentlightning.Runner.step " step(input, *, resources=None, mode=None, event=None) async ") executes a full rollout even though it is named "step". The companion method [`Runner.iter`](https://microsoft.github.io/agent-lightning/0.3.0/reference/runner/#agentlightning.Runner.iter " iter(*, event=None) async ") executes multiple "steps" by continuously pulling new rollout inputs from the store until a stop event is set. Use `iter` once you are confident the single-step path works and you have another worker [`enqueue_rollout`](https://microsoft.github.io/agent-lightning/0.3.0/reference/store/#agentlightning.LightningStore.enqueue_rollout " enqueue_rollout(input, mode=None, resources_id=None, config=None, metadata=None) async ") to the store. Tip You can also call [`Runner.step`](https://microsoft.github.io/agent-lightning/0.3.0/reference/runner/#agentlightning.Runner.step " step(input, *, resources=None, mode=None, event=None) async ") to inject ad-hoc rollouts into a running store being used by another algorithm, so that the rollouts can be consumed by the algorithms. This is very recently known as the paradigm of ["online RL"](https://cursor.com/blog/tab-rl) . At the moment, no algorithm in the [algorithm zoo](https://microsoft.github.io/agent-lightning/0.3.0/algorithm-zoo/) consumes externally generated rollouts, but the data flow is available there if you need it. Debug with LLM Proxy[¶](https://microsoft.github.io/agent-lightning/0.3.0/tutorials/debug/#debug-with-llm-proxy "Permanent link") ---------------------------------------------------------------------------------------------------------------------------------- If you are dealing with LLM optimization like Reinforcement Learning, we generally recommend using an online stable LLM service for your debugging purposes, like `openai/gpt-4.1-nano`. After the debugging is done, you can switch to a local training endpoint. However, if you want to use a local LLM features like [getting the token IDs](https://microsoft.github.io/agent-lightning/0.3.0/deep-dive/serving-llm/) , you can also manually start a local vLLM server by: `[](https://microsoft.github.io/agent-lightning/0.3.0/tutorials/debug/#__codelineno-8-1) vllm serve Qwen/Qwen2.5-0.5B-Instruct --port 8080` Then start the LLM proxy via the following script: `[](https://microsoft.github.io/agent-lightning/0.3.0/tutorials/debug/#__codelineno-9-1) import asyncio [](https://microsoft.github.io/agent-lightning/0.3.0/tutorials/debug/#__codelineno-9-2) import aiohttp [](https://microsoft.github.io/agent-lightning/0.3.0/tutorials/debug/#__codelineno-9-3) import agentlightning as agl [](https://microsoft.github.io/agent-lightning/0.3.0/tutorials/debug/#__codelineno-9-4) [](https://microsoft.github.io/agent-lightning/0.3.0/tutorials/debug/#__codelineno-9-5) async def serve_llm_proxy(): [](https://microsoft.github.io/agent-lightning/0.3.0/tutorials/debug/#__codelineno-9-6) store = agl.InMemoryLightningStore() [](https://microsoft.github.io/agent-lightning/0.3.0/tutorials/debug/#__codelineno-9-7) store_server = agl.LightningStoreServer(store, "127.0.0.1", 8081) [](https://microsoft.github.io/agent-lightning/0.3.0/tutorials/debug/#__codelineno-9-8) await store_server.start() [](https://microsoft.github.io/agent-lightning/0.3.0/tutorials/debug/#__codelineno-9-9) [](https://microsoft.github.io/agent-lightning/0.3.0/tutorials/debug/#__codelineno-9-10) llm_proxy = agl.LLMProxy( [](https://microsoft.github.io/agent-lightning/0.3.0/tutorials/debug/#__codelineno-9-11) port=8082, [](https://microsoft.github.io/agent-lightning/0.3.0/tutorials/debug/#__codelineno-9-12) model_list=[ [](https://microsoft.github.io/agent-lightning/0.3.0/tutorials/debug/#__codelineno-9-13) { [](https://microsoft.github.io/agent-lightning/0.3.0/tutorials/debug/#__codelineno-9-14) "model_name": "Qwen/Qwen2.5-0.5B-Instruct", [](https://microsoft.github.io/agent-lightning/0.3.0/tutorials/debug/#__codelineno-9-15) "litellm_params": { [](https://microsoft.github.io/agent-lightning/0.3.0/tutorials/debug/#__codelineno-9-16) "model": "hosted_vllm/Qwen/Qwen2.5-0.5B-Instruct", [](https://microsoft.github.io/agent-lightning/0.3.0/tutorials/debug/#__codelineno-9-17) "api_base": "http://localhost:8080/v1", [](https://microsoft.github.io/agent-lightning/0.3.0/tutorials/debug/#__codelineno-9-18) }, [](https://microsoft.github.io/agent-lightning/0.3.0/tutorials/debug/#__codelineno-9-19) } [](https://microsoft.github.io/agent-lightning/0.3.0/tutorials/debug/#__codelineno-9-20) ], [](https://microsoft.github.io/agent-lightning/0.3.0/tutorials/debug/#__codelineno-9-21) store=store_server, [](https://microsoft.github.io/agent-lightning/0.3.0/tutorials/debug/#__codelineno-9-22) ) [](https://microsoft.github.io/agent-lightning/0.3.0/tutorials/debug/#__codelineno-9-23) [](https://microsoft.github.io/agent-lightning/0.3.0/tutorials/debug/#__codelineno-9-24) await llm_proxy.start() [](https://microsoft.github.io/agent-lightning/0.3.0/tutorials/debug/#__codelineno-9-25) await asyncio.sleep(1000000)` Test the served LLM proxy with a client like: `[](https://microsoft.github.io/agent-lightning/0.3.0/tutorials/debug/#__codelineno-10-1) async def test_llm_proxy(): [](https://microsoft.github.io/agent-lightning/0.3.0/tutorials/debug/#__codelineno-10-2) async with aiohttp.ClientSession() as session: [](https://microsoft.github.io/agent-lightning/0.3.0/tutorials/debug/#__codelineno-10-3) async with session.post("http://localhost:8082/v1/chat/completions", json={ [](https://microsoft.github.io/agent-lightning/0.3.0/tutorials/debug/#__codelineno-10-4) "model": "Qwen/Qwen2.5-0.5B-Instruct", [](https://microsoft.github.io/agent-lightning/0.3.0/tutorials/debug/#__codelineno-10-5) "messages": [{"role": "user", "content": "Hello, world!"}], [](https://microsoft.github.io/agent-lightning/0.3.0/tutorials/debug/#__codelineno-10-6) }) as response: [](https://microsoft.github.io/agent-lightning/0.3.0/tutorials/debug/#__codelineno-10-7) print(await response.json())` You can now use the LLM proxy by specifying environment variables: `[](https://microsoft.github.io/agent-lightning/0.3.0/tutorials/debug/#__codelineno-11-1) export OPENAI_API_BASE=http://localhost:8081/v1 [](https://microsoft.github.io/agent-lightning/0.3.0/tutorials/debug/#__codelineno-11-2) export OPENAI_API_KEY=dummy` You might see warnings about `Missing or invalid rollout_id, attempt_id, or sequence_id` in the LLM proxy logs. This is fine because you don't have a rollout and attempt yet when you are debugging. When you started the training, the algorithm will create the rollouts for you and the warnings will go away. Hook into Runner's Lifecycle[¶](https://microsoft.github.io/agent-lightning/0.3.0/tutorials/debug/#hook-into-runners-lifecycle "Permanent link") ------------------------------------------------------------------------------------------------------------------------------------------------- [`Runner.run_context`](https://microsoft.github.io/agent-lightning/0.3.0/reference/runner/#agentlightning.Runner.run_context " run_context(*, agent, store, hooks=None, worker_id=None)") accepts a `hooks` argument so you can observe or augment lifecycle events without editing your agent. Hooks subclass [`Hook`](https://microsoft.github.io/agent-lightning/0.3.0/reference/types/#agentlightning.Hook " agentlightning.Hook") and can respond to four asynchronous callbacks: [`on_trace_start`](https://microsoft.github.io/agent-lightning/0.3.0/reference/types/#agentlightning.Hook.on_trace_start " on_trace_start(*, agent, runner, tracer, rollout) async ") , [`on_rollout_start`](https://microsoft.github.io/agent-lightning/0.3.0/reference/types/#agentlightning.Hook.on_rollout_start " on_rollout_start(*, agent, runner, rollout) async ") , [`on_rollout_end`](https://microsoft.github.io/agent-lightning/0.3.0/reference/types/#agentlightning.Hook.on_rollout_end " on_rollout_end(*, agent, runner, rollout, spans) async ") , and [`on_trace_end`](https://microsoft.github.io/agent-lightning/0.3.0/reference/types/#agentlightning.Hook.on_trace_end " on_trace_end(*, agent, runner, tracer, rollout) async ") . This is useful for: * Capturing raw OpenTelemetry spans before they hit the store and before the [`LitAgentRunner`](https://microsoft.github.io/agent-lightning/0.3.0/reference/runner/#agentlightning.LitAgentRunner " agentlightning.LitAgentRunner") do postprocessing on the rollout * Inspecting the tracer instance after they are activated * Logging rollout inputs before they are processed by the agent The `hook` mode in [`examples/apo/apo_debug.py`](https://github.com/microsoft/agent-lightning/tree/3b5d733861cf313fc09821a23240bbdf3cb2ee5b/examples/apo/apo_debug.py) prints every span collected during a rollout: `[](https://microsoft.github.io/agent-lightning/0.3.0/tutorials/debug/#__codelineno-12-1) import agentlightning as agl [](https://microsoft.github.io/agent-lightning/0.3.0/tutorials/debug/#__codelineno-12-2) [](https://microsoft.github.io/agent-lightning/0.3.0/tutorials/debug/#__codelineno-12-3) # ... Same as previous example [](https://microsoft.github.io/agent-lightning/0.3.0/tutorials/debug/#__codelineno-12-4) [](https://microsoft.github.io/agent-lightning/0.3.0/tutorials/debug/#__codelineno-12-5) class DebugHook(agl.Hook): [](https://microsoft.github.io/agent-lightning/0.3.0/tutorials/debug/#__codelineno-12-6) async def on_trace_end(self, *, agent, runner, tracer, rollout): [](https://microsoft.github.io/agent-lightning/0.3.0/tutorials/debug/#__codelineno-12-7) trace = tracer.get_last_trace() [](https://microsoft.github.io/agent-lightning/0.3.0/tutorials/debug/#__codelineno-12-8) print("Trace spans collected during the rollout:") [](https://microsoft.github.io/agent-lightning/0.3.0/tutorials/debug/#__codelineno-12-9) for span in trace: [](https://microsoft.github.io/agent-lightning/0.3.0/tutorials/debug/#__codelineno-12-10) print(f"- {span.name} (status: {span.status}):\n {span.attributes}") [](https://microsoft.github.io/agent-lightning/0.3.0/tutorials/debug/#__codelineno-12-11) [](https://microsoft.github.io/agent-lightning/0.3.0/tutorials/debug/#__codelineno-12-12) with runner.run_context( [](https://microsoft.github.io/agent-lightning/0.3.0/tutorials/debug/#__codelineno-12-13) agent=apo_rollout, [](https://microsoft.github.io/agent-lightning/0.3.0/tutorials/debug/#__codelineno-12-14) store=store, [](https://microsoft.github.io/agent-lightning/0.3.0/tutorials/debug/#__codelineno-12-15) hooks=[DebugHook()], [](https://microsoft.github.io/agent-lightning/0.3.0/tutorials/debug/#__codelineno-12-16) ): [](https://microsoft.github.io/agent-lightning/0.3.0/tutorials/debug/#__codelineno-12-17) await runner.step( [](https://microsoft.github.io/agent-lightning/0.3.0/tutorials/debug/#__codelineno-12-18) "Explain why the sky appears blue using principles of light scattering in 100 words.", [](https://microsoft.github.io/agent-lightning/0.3.0/tutorials/debug/#__codelineno-12-19) resources={"main_prompt": resource}, [](https://microsoft.github.io/agent-lightning/0.3.0/tutorials/debug/#__codelineno-12-20) )` Because hooks run inside the runner process you can also attach debuggers or breakpoints directly in the callback implementations. Note For a better understanding of where hooks are called, we show a pseudo code of Runner's working flow below: `[](https://microsoft.github.io/agent-lightning/0.3.0/tutorials/debug/#__codelineno-13-1) resources = await store.get_latest_resources() [](https://microsoft.github.io/agent-lightning/0.3.0/tutorials/debug/#__codelineno-13-2) rollout = ... [](https://microsoft.github.io/agent-lightning/0.3.0/tutorials/debug/#__codelineno-13-3) try: [](https://microsoft.github.io/agent-lightning/0.3.0/tutorials/debug/#__codelineno-13-4) # <-- on_rollout_start [](https://microsoft.github.io/agent-lightning/0.3.0/tutorials/debug/#__codelineno-13-5) with tracer.trace_context(...): [](https://microsoft.github.io/agent-lightning/0.3.0/tutorials/debug/#__codelineno-13-6) # <--- on_trace_start [](https://microsoft.github.io/agent-lightning/0.3.0/tutorials/debug/#__codelineno-13-7) result = await agent.rollout(...) [](https://microsoft.github.io/agent-lightning/0.3.0/tutorials/debug/#__codelineno-13-8) # <--- on_trace_end [](https://microsoft.github.io/agent-lightning/0.3.0/tutorials/debug/#__codelineno-13-9) post_process_result(result) [](https://microsoft.github.io/agent-lightning/0.3.0/tutorials/debug/#__codelineno-13-10) except Exception: [](https://microsoft.github.io/agent-lightning/0.3.0/tutorials/debug/#__codelineno-13-11) # <-- on_rollout_end [](https://microsoft.github.io/agent-lightning/0.3.0/tutorials/debug/#__codelineno-13-12) await store.update_attempt(status=...)` Dry-Run the Trainer Loop[¶](https://microsoft.github.io/agent-lightning/0.3.0/tutorials/debug/#dry-run-the-trainer-loop "Permanent link") ------------------------------------------------------------------------------------------------------------------------------------------ Once single rollouts behave, switch to the trainer’s dry-run mode. [`Trainer.dev`](https://microsoft.github.io/agent-lightning/0.3.0/reference/trainer/#agentlightning.Trainer.dev " dev(agent, train_dataset=None, *, val_dataset=None)") spins up a lightweight fast algorithm — [`agentlightning.Baseline`](https://microsoft.github.io/agent-lightning/0.3.0/reference/algorithm/#agentlightning.Baseline " agentlightning.Baseline") by default — so you can exercise the same infrastructure as [`Trainer.fit`](https://microsoft.github.io/agent-lightning/0.3.0/reference/trainer/#agentlightning.Trainer.fit " fit(agent, train_dataset=None, *, val_dataset=None)") without standing up complex stacks like RL or SFT. Warning When you enable multiple runners via `n_runners`, the trainer may execute them in separate worker processes. Attaching a debugger such as `pdb` is only practical when `n_runners=1`, and even then the runner might not live in the main process. `[](https://microsoft.github.io/agent-lightning/0.3.0/tutorials/debug/#__codelineno-14-1) import agentlightning as agl [](https://microsoft.github.io/agent-lightning/0.3.0/tutorials/debug/#__codelineno-14-2) [](https://microsoft.github.io/agent-lightning/0.3.0/tutorials/debug/#__codelineno-14-3) dataset: agl.Dataset[str] = [ [](https://microsoft.github.io/agent-lightning/0.3.0/tutorials/debug/#__codelineno-14-4) "Explain why the sky appears blue using principles of light scattering in 100 words.", [](https://microsoft.github.io/agent-lightning/0.3.0/tutorials/debug/#__codelineno-14-5) "What's the capital of France?", [](https://microsoft.github.io/agent-lightning/0.3.0/tutorials/debug/#__codelineno-14-6) ] [](https://microsoft.github.io/agent-lightning/0.3.0/tutorials/debug/#__codelineno-14-7) resource = agl.PromptTemplate(template="You are a helpful assistant. {any_question}", engine="f-string") [](https://microsoft.github.io/agent-lightning/0.3.0/tutorials/debug/#__codelineno-14-8) [](https://microsoft.github.io/agent-lightning/0.3.0/tutorials/debug/#__codelineno-14-9) trainer = agl.Trainer( [](https://microsoft.github.io/agent-lightning/0.3.0/tutorials/debug/#__codelineno-14-10) n_runners=1, [](https://microsoft.github.io/agent-lightning/0.3.0/tutorials/debug/#__codelineno-14-11) initial_resources={"main_prompt": resource}, [](https://microsoft.github.io/agent-lightning/0.3.0/tutorials/debug/#__codelineno-14-12) ) [](https://microsoft.github.io/agent-lightning/0.3.0/tutorials/debug/#__codelineno-14-13) trainer.dev(apo_rollout, dataset)` Just like [`Runner.run_context`](https://microsoft.github.io/agent-lightning/0.3.0/reference/runner/#agentlightning.Runner.run_context " run_context(*, agent, store, hooks=None, worker_id=None)") , [`Trainer.dev`](https://microsoft.github.io/agent-lightning/0.3.0/reference/trainer/#agentlightning.Trainer.dev " dev(agent, train_dataset=None, *, val_dataset=None)") requires the [`NamedResources`](https://microsoft.github.io/agent-lightning/0.3.0/reference/types/#agentlightning.NamedResources " agentlightning.NamedResources = Dict[str, ResourceUnion] module-attribute ") your agent expects. The key difference is that resources are attached to the trainer rather than the runner. [`Trainer.dev`](https://microsoft.github.io/agent-lightning/0.3.0/reference/trainer/#agentlightning.Trainer.dev " dev(agent, train_dataset=None, *, val_dataset=None)") uses an almost switchable interface from [`Trainer.fit`](https://microsoft.github.io/agent-lightning/0.3.0/reference/trainer/#agentlightning.Trainer.fit " fit(agent, train_dataset=None, *, val_dataset=None)") . It also needs a dataset to iterate over, similar to [`fit`](https://microsoft.github.io/agent-lightning/0.3.0/reference/trainer/#agentlightning.Trainer.fit " fit(agent, train_dataset=None, *, val_dataset=None)") . Under the hood [`dev`](https://microsoft.github.io/agent-lightning/0.3.0/reference/trainer/#agentlightning.Trainer.dev " dev(agent, train_dataset=None, *, val_dataset=None)") uses the same implementation as [`fit`](https://microsoft.github.io/agent-lightning/0.3.0/reference/trainer/#agentlightning.Trainer.fit " fit(agent, train_dataset=None, *, val_dataset=None)") , which means you can spin up multiple runners, observe scheduler behavior, and validate how algorithms adapt rollouts. The default [`Baseline`](https://microsoft.github.io/agent-lightning/0.3.0/reference/algorithm/#agentlightning.Baseline " agentlightning.Baseline") logs detailed traces so you can see each rollout as the algorithm perceives it: `[](https://microsoft.github.io/agent-lightning/0.3.0/tutorials/debug/#__codelineno-15-1) 21:20:30 Initial resources set: {'main_prompt': PromptTemplate(resource_type='prompt_template', template='You are a helpful assistant. {any_question}', engine='f-string')} [](https://microsoft.github.io/agent-lightning/0.3.0/tutorials/debug/#__codelineno-15-2) 21:20:30 Proceeding epoch 1/1. [](https://microsoft.github.io/agent-lightning/0.3.0/tutorials/debug/#__codelineno-15-3) 21:20:30 Enqueued rollout ro-302fb202bd85 in train mode with sample: Explain why the sky appears blue using principles of light scattering in 100 words. [](https://microsoft.github.io/agent-lightning/0.3.0/tutorials/debug/#__codelineno-15-4) 21:20:30 Enqueued rollout ro-e65a3ffaa540 in train mode with sample: What's the capital of France? [](https://microsoft.github.io/agent-lightning/0.3.0/tutorials/debug/#__codelineno-15-5) 21:20:30 Waiting for 2 harvest tasks to complete... [](https://microsoft.github.io/agent-lightning/0.3.0/tutorials/debug/#__codelineno-15-6) 21:20:30 [Rollout ro-302fb202bd85] Status is initialized to queuing. [](https://microsoft.github.io/agent-lightning/0.3.0/tutorials/debug/#__codelineno-15-7) 21:20:30 [Rollout ro-e65a3ffaa540] Status is initialized to queuing. [](https://microsoft.github.io/agent-lightning/0.3.0/tutorials/debug/#__codelineno-15-8) 21:20:35 [Rollout ro-302fb202bd85] Finished with status succeeded in 3.80 seconds. [](https://microsoft.github.io/agent-lightning/0.3.0/tutorials/debug/#__codelineno-15-9) 21:20:35 [Rollout ro-302fb202bd85 | Attempt 1] ID: at-f84ad21c. Status: succeeded. Worker: Worker-0 [](https://microsoft.github.io/agent-lightning/0.3.0/tutorials/debug/#__codelineno-15-10) 21:20:35 [Rollout ro-302fb202bd85 | Attempt at-f84ad21c | Span 3a286a856af6bea8] #1 (openai.chat.completion) ... 1.95 seconds. Attribute keys: ['gen_ai.request.type', 'gen_ai.system', ...] [](https://microsoft.github.io/agent-lightning/0.3.0/tutorials/debug/#__codelineno-15-11) 21:20:35 [Rollout ro-302fb202bd85 | Attempt at-f84ad21c | Span e2f44b775e058dd6] #2 (openai.chat.completion) ... 1.24 seconds. Attribute keys: ['gen_ai.request.type', 'gen_ai.system', ...] [](https://microsoft.github.io/agent-lightning/0.3.0/tutorials/debug/#__codelineno-15-12) 21:20:35 [Rollout ro-302fb202bd85 | Attempt at-f84ad21c | Span 45ee3c94fa1070ec] #3 (agentlightning.annotation) ... 0.00 seconds. Attribute keys: ['agentlightning.reward.0.value'] [](https://microsoft.github.io/agent-lightning/0.3.0/tutorials/debug/#__codelineno-15-13) 21:20:35 [Rollout ro-302fb202bd85] Adapted data: [Triplet(prompt={'token_ids': []}, response={'token_ids': []}, reward=None, metadata={'response_id': '...', 'agent_name': ''}), Triplet(prompt={'token_ids': []}, response={'token_ids': []}, reward=0.95, metadata={'response_id': '...', 'agent_name': ''})] [](https://microsoft.github.io/agent-lightning/0.3.0/tutorials/debug/#__codelineno-15-14) 21:20:35 Finished 1 rollouts. [](https://microsoft.github.io/agent-lightning/0.3.0/tutorials/debug/#__codelineno-15-15) 21:20:35 [Rollout ro-e65a3ffaa540] Status changed to preparing. [](https://microsoft.github.io/agent-lightning/0.3.0/tutorials/debug/#__codelineno-15-16) 21:20:40 [Rollout ro-e65a3ffaa540] Finished with status succeeded in 6.39 seconds. [](https://microsoft.github.io/agent-lightning/0.3.0/tutorials/debug/#__codelineno-15-17) 21:20:40 [Rollout ro-e65a3ffaa540 | Attempt 1] ID: at-eaefa5d4. Status: succeeded. Worker: Worker-0 [](https://microsoft.github.io/agent-lightning/0.3.0/tutorials/debug/#__codelineno-15-18) 21:20:40 [Rollout ro-e65a3ffaa540 | Attempt at-eaefa5d4 | Span 901dd6acc0f50147] #1 (openai.chat.completion) ... 1.30 seconds. Attribute keys: ['gen_ai.request.type', 'gen_ai.system', ...] [](https://microsoft.github.io/agent-lightning/0.3.0/tutorials/debug/#__codelineno-15-19) 21:20:40 [Rollout ro-e65a3ffaa540 | Attempt at-eaefa5d4 | Span 52e0aa63e02be611] #2 (openai.chat.completion) ... 1.26 seconds. Attribute keys: ['gen_ai.request.type', 'gen_ai.system', ...] [](https://microsoft.github.io/agent-lightning/0.3.0/tutorials/debug/#__codelineno-15-20) 21:20:40 [Rollout ro-e65a3ffaa540 | Attempt at-eaefa5d4 | Span 6c452de193fbffd3] #3 (agentlightning.annotation) ... 0.00 seconds. Attribute keys: ['agentlightning.reward.0.value'] [](https://microsoft.github.io/agent-lightning/0.3.0/tutorials/debug/#__codelineno-15-21) 21:20:40 [Rollout ro-e65a3ffaa540] Adapted data: [Triplet(prompt={'token_ids': []}, response={'token_ids': []}, reward=None, metadata={'response_id': '...', 'agent_name': ''}), Triplet(prompt={'token_ids': []}, response={'token_ids': []}, reward=1.0, metadata={'response_id': '...', 'agent_name': ''})] [](https://microsoft.github.io/agent-lightning/0.3.0/tutorials/debug/#__codelineno-15-22) 21:20:40 Finished 2 rollouts.` The only limitation is that resources remain static and components like [`LLMProxy`](https://microsoft.github.io/agent-lightning/0.3.0/reference/algorithm/#agentlightning.LLMProxy " agentlightning.LLMProxy") are not wired in. For richer dry runs you can subclass [`FastAlgorithm`](https://microsoft.github.io/agent-lightning/0.3.0/reference/algorithm/#agentlightning.FastAlgorithm " agentlightning.FastAlgorithm") and override the pieces you care about. Debug the Algorithm-Runner Boundary[¶](https://microsoft.github.io/agent-lightning/0.3.0/tutorials/debug/#debug-the-algorithm-runner-boundary "Permanent link") ---------------------------------------------------------------------------------------------------------------------------------------------------------------- [](https://microsoft.github.io/agent-lightning/0.3.0/tutorials/debug/) Debugging algorithms in Agent-Lightning is often more challenging than debugging agents. Algorithms are typically **stateful** and depend on several moving parts — runners, stores, and trainers — which makes it difficult to isolate and inspect their behavior. Even mocking an agent to cooperate with an algorithm can be costly and error-prone. To simplify this, Agent-Lightning provides a way to run algorithms in isolation so you can attach a debugger and inspect internal state without interference from other components. By default, [`Trainer.fit`](https://microsoft.github.io/agent-lightning/0.3.0/reference/trainer/#agentlightning.Trainer.fit " fit(agent, train_dataset=None, *, val_dataset=None)") runs the algorithm in the main process and thread, but its logs are interleaved with those from the store and runners, making it hard to follow what’s happening inside the algorithm itself. In [_Write Your First Algorithm_](https://microsoft.github.io/agent-lightning/0.3.0/how-to/write-first-algorithm/) , we covered how to stand up a store, algorithm, and runner in isolation for your own implementations. This section extends that approach to cover two common questions: 1. How can I run built-in or class-based algorithms (inheriting from [`Algorithm`](https://microsoft.github.io/agent-lightning/0.3.0/reference/algorithm/#agentlightning.Algorithm " agentlightning.Algorithm") ) in isolation? 2. How can I still use [`Trainer`](https://microsoft.github.io/agent-lightning/0.3.0/reference/trainer/#agentlightning.Trainer " agentlightning.Trainer") features like `n_runners`, `adapter`, or `llm_proxy` while debugging? The solution is to keep using a [`Trainer`](https://microsoft.github.io/agent-lightning/0.3.0/reference/trainer/#agentlightning.Trainer " agentlightning.Trainer") instance but **manage the store yourself**, running the algorithm and runner roles separately. This approach mirrors the internal process orchestration of [`Trainer.fit`](https://microsoft.github.io/agent-lightning/0.3.0/reference/trainer/#agentlightning.Trainer.fit " fit(agent, train_dataset=None, *, val_dataset=None)") , but with more visibility and control. Below, we show a step-by-step guide to achieve this with the [`calc_agent` example](https://github.com/microsoft/agent-lightning/tree/3b5d733861cf313fc09821a23240bbdf3cb2ee5b/examples/calc_x/train_calc_agent.py) . **1\. Launch the store manually.** In a separate terminal, start the store: `[](https://microsoft.github.io/agent-lightning/0.3.0/tutorials/debug/#__codelineno-16-1) agl store --port 4747` Add `--log-level DEBUG` to the command to see the detailed logs. Then, in your training script, create a [`LightningStoreClient`](https://microsoft.github.io/agent-lightning/0.3.0/reference/store/#agentlightning.LightningStoreClient " agentlightning.LightningStoreClient") and pass it to the trainer: `[](https://microsoft.github.io/agent-lightning/0.3.0/tutorials/debug/#__codelineno-17-1) client = agl.LightningStoreClient("http://localhost:4747") [](https://microsoft.github.io/agent-lightning/0.3.0/tutorials/debug/#__codelineno-17-2) trainer = agl.Trainer(store=client, ...)` Set the environment variable `AGL_MANAGED_STORE=0` so the trainer doesn't attempt to manage the store automatically. **2\. Start the runner and algorithm processes separately.** Each process should run the same training script, but with different environment variables specifying the current role. This setup faithfully mirrors how [`Trainer.fit`](https://microsoft.github.io/agent-lightning/0.3.0/reference/trainer/#agentlightning.Trainer.fit " fit(agent, train_dataset=None, *, val_dataset=None)") orchestrates these components behind the scenes. `[](https://microsoft.github.io/agent-lightning/0.3.0/tutorials/debug/#__codelineno-18-1) # Terminal 2 – Runner process [](https://microsoft.github.io/agent-lightning/0.3.0/tutorials/debug/#__codelineno-18-2) AGL_MANAGED_STORE=0 AGL_CURRENT_ROLE=runner \ [](https://microsoft.github.io/agent-lightning/0.3.0/tutorials/debug/#__codelineno-18-3) python train_calc_agent.py --external-store-address http://localhost:4747 --val-file data/test_mini.parquet [](https://microsoft.github.io/agent-lightning/0.3.0/tutorials/debug/#__codelineno-18-4) [](https://microsoft.github.io/agent-lightning/0.3.0/tutorials/debug/#__codelineno-18-5) # Terminal 3 – Algorithm process [](https://microsoft.github.io/agent-lightning/0.3.0/tutorials/debug/#__codelineno-18-6) AGL_MANAGED_STORE=0 AGL_CURRENT_ROLE=algorithm \ [](https://microsoft.github.io/agent-lightning/0.3.0/tutorials/debug/#__codelineno-18-7) python train_calc_agent.py --external-store-address http://localhost:4747 --val-file data/test_mini.parquet` **3\. Reuse your existing trainer configuration.** You can continue using the same datasets, adapters, and proxies as usual. Because the store is now external, you can: * Attach debuggers to either the algorithm or runner process * Add fine-grained logging or tracing * Simulate partial failures or latency in individual components This setup provides a faithful reproduction of the algorithm–runner interaction while keeping the store visible for inspection. Once you’ve resolved the issue, simply set `AGL_MANAGED_STORE=1` (or omit it) to return to the standard managed training workflow. Back to top --- # Installation - Agent-lightning [Skip to content](https://microsoft.github.io/agent-lightning/0.3.0/tutorials/installation/#installation-guide) ⚠️ You are viewing development documentation. For stable documentation, please visit [https://microsoft.github.io/agent-lightning/stable/](https://microsoft.github.io/agent-lightning/stable/) Installation Guide[¶](https://microsoft.github.io/agent-lightning/0.3.0/tutorials/installation/#installation-guide "Permanent link") ===================================================================================================================================== This guide explains how to install **Agent-Lightning**. You can install it from **PyPI** (the Python Package Index) for general use or directly from the **source code** if you plan to contribute or need fine-grained control over dependencies. Platform and Hardware Requirements Agent-Lightning is officially supported on **Linux distributions** (Ubuntu 22.04 or later is recommended). At the moment **macOS and Windows** (outside of WSL2) are not supported. The Python runtime must be **Python 3.10 or newer**. We recommend using the latest patch release of Python 3.10, 3.11, or 3.12 to pick up performance and security updates. A **GPU is optional**—you only need CUDA-capable hardware if you plan to fine-tune model weights or run GPU-accelerated workloads. CPU-only environments are fully supported for evaluation and inference. Installing from PyPI[¶](https://microsoft.github.io/agent-lightning/0.3.0/tutorials/installation/#installing-from-pypi "Permanent link") ----------------------------------------------------------------------------------------------------------------------------------------- The easiest way to get started is by installing Agent-Lightning directly from PyPI. This ensures you get the latest **stable release** of the package, tested for compatibility and reliability. ### Install the Stable Release[¶](https://microsoft.github.io/agent-lightning/0.3.0/tutorials/installation/#install-the-stable-release "Permanent link") Run the following command in your terminal: `[](https://microsoft.github.io/agent-lightning/0.3.0/tutorials/installation/#__codelineno-0-1) pip install --upgrade agentlightning` This installs or upgrades Agent-Lightning to the newest stable version. Tip If you intend to use **Agent-Lightning** with [**VERL**](https://microsoft.github.io/agent-lightning/0.3.0/algorithm-zoo/verl/) or run any of its **example scripts**, you’ll need to install some additional dependencies. See the sections on [Algorithm-specific installation](https://microsoft.github.io/agent-lightning/0.3.0/tutorials/installation/#algorithm-specific-installation) and [Example-specific installation](https://microsoft.github.io/agent-lightning/0.3.0/tutorials/installation/#example-specific-installation) for details. ### Install the Nightly Build (Latest Features)[¶](https://microsoft.github.io/agent-lightning/0.3.0/tutorials/installation/#install-the-nightly-build-latest-features "Permanent link") Agent-Lightning also publishes **nightly builds**, which contain the latest experimental features and improvements from the main branch. These are available via **Test PyPI**. `[](https://microsoft.github.io/agent-lightning/0.3.0/tutorials/installation/#__codelineno-1-1) pip install --upgrade --index-url https://test.pypi.org/simple/ --extra-index-url https://pypi.org/simple/ --pre agentlightning` Warning The nightly builds are cutting-edge but may include unstable or untested changes. Use them **at your own risk**, especially in production environments. Algorithm-specific Installation[¶](https://microsoft.github.io/agent-lightning/0.3.0/tutorials/installation/#algorithm-specific-installation "Permanent link") --------------------------------------------------------------------------------------------------------------------------------------------------------------- Agent-Lightning supports multiple learning algorithms. Some of them like [APO](https://microsoft.github.io/agent-lightning/0.3.0/algorithm-zoo/apo/) or [VERL](https://microsoft.github.io/agent-lightning/0.3.0/algorithm-zoo/verl/) require extra dependencies. You can install them automatically using **optional extras** or manually if you prefer finer control. ### Installing APO[¶](https://microsoft.github.io/agent-lightning/0.3.0/tutorials/installation/#installing-apo "Permanent link") [APO](https://microsoft.github.io/agent-lightning/0.3.0/algorithm-zoo/apo/) is an algorithm module that depends on libraries such as [POML](https://github.com/microsoft/POML) . You can install Agent-Lightning with APO support by running: `[](https://microsoft.github.io/agent-lightning/0.3.0/tutorials/installation/#__codelineno-2-1) pip install agentlightning[apo]` Warning APO also depends on the [OpenAI Python SDK](https://github.com/openai/openai-python) , version **2.0 or newer**. Ensure your SDK version is up to date to avoid compatibility issues. ### Installing VERL[¶](https://microsoft.github.io/agent-lightning/0.3.0/tutorials/installation/#installing-verl "Permanent link") [VERL](https://microsoft.github.io/agent-lightning/0.3.0/algorithm-zoo/verl/) integrates with libraries like **PyTorch**, **vLLM**, and **VERL framework**. Although you _can_ install all dependencies automatically, we recommend doing it manually to avoid version conflicts. `[](https://microsoft.github.io/agent-lightning/0.3.0/tutorials/installation/#__codelineno-3-1) pip install agentlightning[verl]` Recommended Manual Setup (More Stable) Automated installation may cause issues if you don’t have a compatible **PyTorch** or **CUDA** version preinstalled. For a more stable setup, install dependencies step-by-step: `[](https://microsoft.github.io/agent-lightning/0.3.0/tutorials/installation/#__codelineno-4-1) pip install torch==2.8.0 torchvision==0.23.0 --index-url https://download.pytorch.org/whl/cu128 [](https://microsoft.github.io/agent-lightning/0.3.0/tutorials/installation/#__codelineno-4-2) pip install flash-attn --no-build-isolation [](https://microsoft.github.io/agent-lightning/0.3.0/tutorials/installation/#__codelineno-4-3) pip install vllm==0.10.2 [](https://microsoft.github.io/agent-lightning/0.3.0/tutorials/installation/#__codelineno-4-4) pip install verl==0.5.0` This approach ensures compatibility with CUDA 12.8 and minimizes dependency conflicts. Example-specific Installation[¶](https://microsoft.github.io/agent-lightning/0.3.0/tutorials/installation/#example-specific-installation "Permanent link") ----------------------------------------------------------------------------------------------------------------------------------------------------------- Each example in the `examples/` directory may have its own additional dependencies. Please refer to the **README** file of each example for detailed setup instructions: [See Example READMEs](https://github.com/microsoft/agent-lightning/tree/3b5d733861cf313fc09821a23240bbdf3cb2ee5b/examples) . Installing from Source (for Developers and Contributors)[¶](https://microsoft.github.io/agent-lightning/0.3.0/tutorials/installation/#installing-from-source-for-developers-and-contributors "Permanent link") --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- If you plan to contribute to Agent-Lightning or prefer to work with the latest development code, install it directly from the **source repository**. ### Why Install from Source?[¶](https://microsoft.github.io/agent-lightning/0.3.0/tutorials/installation/#why-install-from-source "Permanent link") * You want to **modify or contribute** to the project. * You prefer an **isolated development environment**. * You want to test unreleased features or fix bugs locally. ### Using `uv` for Dependency Management[¶](https://microsoft.github.io/agent-lightning/0.3.0/tutorials/installation/#using-uv-for-dependency-management "Permanent link") Starting with version **0.2**, Agent-Lightning uses [`uv`](https://docs.astral.sh/uv/) as its **default dependency manager**. `uv` is a fast and safe alternative to `pip` that: * Installs packages **in seconds** (instead of minutes), * Prevents **dependency conflicts**, * Supports **grouped dependencies** for optional features. Before proceeding, make sure `uv` is installed. ### Minimal Developer Installation[¶](https://microsoft.github.io/agent-lightning/0.3.0/tutorials/installation/#minimal-developer-installation "Permanent link") `[](https://microsoft.github.io/agent-lightning/0.3.0/tutorials/installation/#__codelineno-5-1) git clone https://github.com/microsoft/agent-lightning [](https://microsoft.github.io/agent-lightning/0.3.0/tutorials/installation/#__codelineno-5-2) cd agent-lightning [](https://microsoft.github.io/agent-lightning/0.3.0/tutorials/installation/#__codelineno-5-3) uv sync --group dev` This command sets up a clean development environment with only the essential dependencies. ### Installing All Extras (CPU or GPU)[¶](https://microsoft.github.io/agent-lightning/0.3.0/tutorials/installation/#installing-all-extras-cpu-or-gpu "Permanent link") `uv sync` can also handle algorithm-specific and example-specific dependencies in one step. For a CPU-only machine: `[](https://microsoft.github.io/agent-lightning/0.3.0/tutorials/installation/#__codelineno-6-1) uv sync --frozen \ [](https://microsoft.github.io/agent-lightning/0.3.0/tutorials/installation/#__codelineno-6-2) --extra apo \ [](https://microsoft.github.io/agent-lightning/0.3.0/tutorials/installation/#__codelineno-6-3) --extra verl \ [](https://microsoft.github.io/agent-lightning/0.3.0/tutorials/installation/#__codelineno-6-4) --group dev \ [](https://microsoft.github.io/agent-lightning/0.3.0/tutorials/installation/#__codelineno-6-5) --group torch-cpu \ [](https://microsoft.github.io/agent-lightning/0.3.0/tutorials/installation/#__codelineno-6-6) --group torch-stable \ [](https://microsoft.github.io/agent-lightning/0.3.0/tutorials/installation/#__codelineno-6-7) --group trl \ [](https://microsoft.github.io/agent-lightning/0.3.0/tutorials/installation/#__codelineno-6-8) --group agents \ [](https://microsoft.github.io/agent-lightning/0.3.0/tutorials/installation/#__codelineno-6-9) --no-default-groups` For a GPU-equipped machine that is CUDA 12.8 compatible: `[](https://microsoft.github.io/agent-lightning/0.3.0/tutorials/installation/#__codelineno-7-1) uv sync --frozen \ [](https://microsoft.github.io/agent-lightning/0.3.0/tutorials/installation/#__codelineno-7-2) --extra apo \ [](https://microsoft.github.io/agent-lightning/0.3.0/tutorials/installation/#__codelineno-7-3) --extra verl \ [](https://microsoft.github.io/agent-lightning/0.3.0/tutorials/installation/#__codelineno-7-4) --group dev \ [](https://microsoft.github.io/agent-lightning/0.3.0/tutorials/installation/#__codelineno-7-5) --group torch-gpu-stable \ [](https://microsoft.github.io/agent-lightning/0.3.0/tutorials/installation/#__codelineno-7-6) --group trl \ [](https://microsoft.github.io/agent-lightning/0.3.0/tutorials/installation/#__codelineno-7-7) --group agents \ [](https://microsoft.github.io/agent-lightning/0.3.0/tutorials/installation/#__codelineno-7-8) --no-default-groups` Read more about Agent-lightning managed dependency groups [here](https://github.com/microsoft/agent-lightning/tree/3b5d733861cf313fc09821a23240bbdf3cb2ee5b/pyproject.toml) . ### Building the Dashboard[¶](https://microsoft.github.io/agent-lightning/0.3.0/tutorials/installation/#building-the-dashboard "Permanent link") The Agent-Lightning dashboard is built using [Vite](https://vite.dev/) . To build the dashboard, run the following command: `[](https://microsoft.github.io/agent-lightning/0.3.0/tutorials/installation/#__codelineno-8-1) cd dashboard [](https://microsoft.github.io/agent-lightning/0.3.0/tutorials/installation/#__codelineno-8-2) npm ci [](https://microsoft.github.io/agent-lightning/0.3.0/tutorials/installation/#__codelineno-8-3) npm run build` Some HTML and JavaScript assets will be generated in the `agentlightning/dashboard` directory. ### Activating Your Environment[¶](https://microsoft.github.io/agent-lightning/0.3.0/tutorials/installation/#activating-your-environment "Permanent link") After syncing dependencies, `uv` automatically creates a virtual environment inside the `.venv/` directory. You can use it in two ways: `[](https://microsoft.github.io/agent-lightning/0.3.0/tutorials/installation/#__codelineno-9-1) # Option 1: Prefix commands with uv run [](https://microsoft.github.io/agent-lightning/0.3.0/tutorials/installation/#__codelineno-9-2) uv run python your_script.py [](https://microsoft.github.io/agent-lightning/0.3.0/tutorials/installation/#__codelineno-9-3) [](https://microsoft.github.io/agent-lightning/0.3.0/tutorials/installation/#__codelineno-9-4) # Option 2: Activate the virtual environment [](https://microsoft.github.io/agent-lightning/0.3.0/tutorials/installation/#__codelineno-9-5) source .venv/bin/activate [](https://microsoft.github.io/agent-lightning/0.3.0/tutorials/installation/#__codelineno-9-6) python your_script.py` Before Contributing Agent-Lightning enforces code style and linting rules via **pre-commit hooks**. Installing them early prevents many avoidable formatting issues. `[](https://microsoft.github.io/agent-lightning/0.3.0/tutorials/installation/#__codelineno-10-1) uv run pre-commit install [](https://microsoft.github.io/agent-lightning/0.3.0/tutorials/installation/#__codelineno-10-2) uv run pre-commit run --all-files --show-diff-on-failure --color=always` Back to top --- # Train the First Agent - Agent-lightning [Skip to content](https://microsoft.github.io/agent-lightning/0.3.0/how-to/train-first-agent/#train-the-first-agent-with-agent-lightning) ⚠️ You are viewing development documentation. For stable documentation, please visit [https://microsoft.github.io/agent-lightning/stable/](https://microsoft.github.io/agent-lightning/stable/) Train the First Agent with Agent-lightning[¶](https://microsoft.github.io/agent-lightning/0.3.0/how-to/train-first-agent/#train-the-first-agent-with-agent-lightning "Permanent link") ======================================================================================================================================================================================= Welcome! This tutorial is your first step into making AI agents smarter using the **Agent-lightning** framework. We'll show you how to take a simple agent and automatically improve its performance through a process called [**Automatic Prompt Optimization (APO)**](https://microsoft.github.io/agent-lightning/0.3.0/algorithm-zoo/apo/) . The main goal of Agent-lightning is to provide a structured way to **train your agents**. Just like you train a machine learning model on data, you can train an agent on a task dataset. This could involve using Reinforcement Learning (RL) to teach it new behaviors or, as we'll do today, optimizing its prompts to make it more accurate and reliable. Tip You can open the sample code [room\_selector\_apo.py](https://github.com/microsoft/agent-lightning/tree/3b5d733861cf313fc09821a23240bbdf3cb2ee5b/examples/apo/room_selector_apo.py) and [room\_selector.py](https://github.com/microsoft/agent-lightning/tree/3b5d733861cf313fc09821a23240bbdf3cb2ee5b/examples/apo/room_selector.py) as you go through this tutorial. Our Example: The Room Selector Agent[¶](https://microsoft.github.io/agent-lightning/0.3.0/how-to/train-first-agent/#our-example-the-room-selector-agent "Permanent link") -------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Today, we'll work with an agent whose job is to book a meeting room. It's a common but tricky task with multiple constraints. Here's how the agent works: * **Input:** It receives a task with specific requirements, like "`Find a room for 4 people at 10:00 AM with a whiteboard.`" * **Action:** The agent uses a Large Language Model (LLM) to understand the request. It can also use tools, which are pre-defined functions it can call, to get more information, such as checking room availability in an external database. * **Output:** Its final decision is the ID of the best room it found, like "`A103`". * **Reward:** After the agent makes its choice, a separate "grader" function scores its performance on a scale of 0 to 1. This score is called its **reward**. A perfect choice gets a 1.0, while a wrong one gets a 0.0. The agent's logic is sound, but its performance heavily depends on its initial prompt. A poorly worded prompt will confuse the LLM, leading to bad decisions. Our goal is to use Agent-lightning to find the best possible prompt automatically. A Closer Look at the Agent's Logic Modern LLMs can do more than just generate text; they can decide to call functions you provide. This is often called tool use or function calling. Our agent uses this capability to make informed decisions. If you're new to this concept, you can read more about it in [OpenAI's documentation](https://platform.openai.com/docs/guides/function-calling) . Here is a sketch of the agent's logic, adhering closely to the OpenAI API: `[](https://microsoft.github.io/agent-lightning/0.3.0/how-to/train-first-agent/#__codelineno-0-1) # Pseudo-code for the Room Selector agent [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/train-first-agent/#__codelineno-0-2) [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/train-first-agent/#__codelineno-0-3) import openai [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/train-first-agent/#__codelineno-0-4) import json [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/train-first-agent/#__codelineno-0-5) [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/train-first-agent/#__codelineno-0-6) def room_selector_agent(task, prompt): [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/train-first-agent/#__codelineno-0-7) client = openai.OpenAI() [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/train-first-agent/#__codelineno-0-8) messages = [{"role": "user", "content": prompt.format(**task)}] [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/train-first-agent/#__codelineno-0-9) tools = [ ... ] # Tool definition for the LLM [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/train-first-agent/#__codelineno-0-10) [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/train-first-agent/#__codelineno-0-11) # 1. First LLM call to decide if a tool is needed. [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/train-first-agent/#__codelineno-0-12) response = client.chat.completions.create( [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/train-first-agent/#__codelineno-0-13) model="gpt-5-mini", [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/train-first-agent/#__codelineno-0-14) messages=messages, [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/train-first-agent/#__codelineno-0-15) tools=tools, [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/train-first-agent/#__codelineno-0-16) tool_choice="auto", [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/train-first-agent/#__codelineno-0-17) ) [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/train-first-agent/#__codelineno-0-18) response_message = response.choices[0].message [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/train-first-agent/#__codelineno-0-19) tool_calls = response_message.tool_calls [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/train-first-agent/#__codelineno-0-20) [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/train-first-agent/#__codelineno-0-21) # 2. Check if the LLM wants to use a tool. [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/train-first-agent/#__codelineno-0-22) if tool_calls: [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/train-first-agent/#__codelineno-0-23) messages.append(response_message) # Append assistant's reply [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/train-first-agent/#__codelineno-0-24) [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/train-first-agent/#__codelineno-0-25) # 3. Execute the tool and get the real-world data. [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/train-first-agent/#__codelineno-0-26) for tool_call in tool_calls: [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/train-first-agent/#__codelineno-0-27) function_name = tool_call.function.name [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/train-first-agent/#__codelineno-0-28) if function_name == "get_rooms_and_availability": [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/train-first-agent/#__codelineno-0-29) function_args = json.loads(tool_call.function.arguments) [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/train-first-agent/#__codelineno-0-30) # Query the local room database [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/train-first-agent/#__codelineno-0-31) function_response = get_rooms_and_availability( [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/train-first-agent/#__codelineno-0-32) date=function_args.get("date"), [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/train-first-agent/#__codelineno-0-33) time_str=function_args.get("time"), [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/train-first-agent/#__codelineno-0-34) duration_min=function_args.get("duration_min"), [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/train-first-agent/#__codelineno-0-35) ) [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/train-first-agent/#__codelineno-0-36) messages.append({ [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/train-first-agent/#__codelineno-0-37) "tool_call_id": tool_call.id, [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/train-first-agent/#__codelineno-0-38) "role": "tool", [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/train-first-agent/#__codelineno-0-39) "name": function_name, [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/train-first-agent/#__codelineno-0-40) "content": json.dumps(function_response), [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/train-first-agent/#__codelineno-0-41) }) [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/train-first-agent/#__codelineno-0-42) [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/train-first-agent/#__codelineno-0-43) # 4. Second LLM call with the tool's output to get a final choice. [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/train-first-agent/#__codelineno-0-44) second_response = client.chat.completions.create( [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/train-first-agent/#__codelineno-0-45) model="gpt-5-mini", [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/train-first-agent/#__codelineno-0-46) messages=messages, [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/train-first-agent/#__codelineno-0-47) ) [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/train-first-agent/#__codelineno-0-48) final_choice = second_response.choices[0].message.content [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/train-first-agent/#__codelineno-0-49) else: [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/train-first-agent/#__codelineno-0-50) final_choice = response_message.content [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/train-first-agent/#__codelineno-0-51) [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/train-first-agent/#__codelineno-0-52) # 5. Grade the final choice to get a reward. [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/train-first-agent/#__codelineno-0-53) reward = grade_the_choice(final_choice, task["expected_choice"]) [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/train-first-agent/#__codelineno-0-54) return reward` In Agent-lightning, you wrap this logic in a Python function marked with the [`@rollout`](https://microsoft.github.io/agent-lightning/0.3.0/reference/agent/#agentlightning.rollout " agentlightning.rollout(func)") decorator, so that the agent can be managed and tuned by Agent-lightning's runner and trainer. The `prompt_template` that the APO algorithm tunes is passed in as an argument: `[](https://microsoft.github.io/agent-lightning/0.3.0/how-to/train-first-agent/#__codelineno-1-1) import agentlightning as agl [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/train-first-agent/#__codelineno-1-2) [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/train-first-agent/#__codelineno-1-3) @agl.rollout [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/train-first-agent/#__codelineno-1-4) def room_selector(task: RoomSelectionTask, prompt_template: agl.PromptTemplate) -> float: [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/train-first-agent/#__codelineno-1-5) # ... agent logic using the prompt_template ... [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/train-first-agent/#__codelineno-1-6) [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/train-first-agent/#__codelineno-1-7) # The final reward is determined by a grader function [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/train-first-agent/#__codelineno-1-8) reward = room_selection_grader(client, final_message, task["expected_choice"]) [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/train-first-agent/#__codelineno-1-9) return reward` Core Concepts: Tasks, Rollouts, Spans, and Prompt Templates[¶](https://microsoft.github.io/agent-lightning/0.3.0/how-to/train-first-agent/#core-concepts-tasks-rollouts-spans-and-prompt-templates "Permanent link") --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- To understand how Agent-lightning works, you need to know these key terms. ### Task[¶](https://microsoft.github.io/agent-lightning/0.3.0/how-to/train-first-agent/#task "Permanent link") A task is a specific input or problem statement given to the agent. It defines what the agent needs to accomplish. Analogy: Task If the agent is a chef, a task is the recipe request: "Bake a chocolate cake." ### Rollout[¶](https://microsoft.github.io/agent-lightning/0.3.0/how-to/train-first-agent/#rollout "Permanent link") A rollout is a single, complete execution of an agent attempting to solve a given **task**. It's the entire story from receiving the task to producing a final result and receiving a reward. A rollout captures a full trace of the agent's execution. Analogy: Rollout A rollout is one full attempt by the chef to bake the chocolate cake, from gathering ingredients to the final taste test. ### Span[¶](https://microsoft.github.io/agent-lightning/0.3.0/how-to/train-first-agent/#span "Permanent link") A span represents a single unit of work or an operation within a **rollout**. Spans are the building blocks of a trace. They have a start and end time and contain details about the specific operation, like an LLM call, a tool execution, or a reward calculation. For a more precise definition, see the [OpenTelemetry documentation](https://opentelemetry.io/docs/concepts/signals/traces/) . Analogy: Span If the rollout is "baking a cake," a span could be "preheating the oven," "mixing flour and sugar," or "adding frosting." Each is a distinct step or unit of work. The picture below from [ADK](https://google.github.io/adk-docs/observability/cloud-trace/) shows a typical rollout, where each rectangle in the waterfall visualizes a span. As can be seen in the visualization, spans can be sequential, parallel or nested among each other. In other frameworks, the terminology might be slightly different. Agent-lightning follows the terminologies used by OpenTelemetry to avoid confusion. ![AgentOps Waterfall Visualization](https://microsoft.github.io/agent-lightning/0.3.0/assets/agentops-waterfall-visualization.jpg) ### Prompt Template[¶](https://microsoft.github.io/agent-lightning/0.3.0/how-to/train-first-agent/#prompt-template "Permanent link") A prompt template is a reusable instruction for the agent, often containing placeholders that can be filled in with specific details from a task. It is a key **"resource"** that the algorithm learns and improves over time. Analogy: Resource (Prompt Template) If the task is the recipe request, the prompt template is the master recipe card that the chef follows. The algorithm's job is to edit this recipe card to make the instructions clearer and the final dish better. The Training Loop: How the Magic Happens[¶](https://microsoft.github.io/agent-lightning/0.3.0/how-to/train-first-agent/#the-training-loop-how-the-magic-happens "Permanent link") ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Training in Agent-lightning revolves around a clear, managed loop, orchestrated by the **Trainer**. The diagram below illustrates this core interaction: ![Loop of Tasks and Spans](https://microsoft.github.io/agent-lightning/0.3.0/assets/tasks-spans-loop.svg) **The Loop Explained:** * **Algorithm to Agent (via Trainer):** The **Algorithm** (the "brain") creates an improved **Prompt Template** and selects **Tasks**. The Trainer then sends both to the Agent. * **Agent to Algorithm (via Trainer):** For each task it receives, the Agent uses the provided prompt template to perform a Rollout, executing its logic and potentially using tools. During this rollout, the runner that runs the agent captures Spans that detail every step. The agent also calculates a Reward for its performance on the task. These spans and rewards are then sent back to the Algorithm via the Trainer. * **Algorithm Learning:** The Algorithm then analyzes these spans and rewards to learn how to improve the agent's behavior, for example, by generating a better prompt. This improved prompt is then used in the next iteration of tasks. This cycle continues, allowing the agent to continuously learn and get better at solving tasks. Note In the next tutorial, we will see that the "via Trainer" here is not accurate. It's actually via the runner and store. ### The Algorithm[¶](https://microsoft.github.io/agent-lightning/0.3.0/how-to/train-first-agent/#the-algorithm "Permanent link") The algorithm is the smart part of the system that drives the improvement. In this tutorial, we use [**APO**](https://microsoft.github.io/agent-lightning/0.3.0/algorithm-zoo/apo/#agentlightning.algorithm.apo.APO " APO") (Automatic Prompt Optimization). It works in a few steps: 1. **Evaluate:** The algorithm first asks for rollouts to be run using the current prompt template to see how well it performs. 2. **Critique:** It then looks at the detailed spans from those rollouts. Using a powerful LLM (`gpt-5-mini`), it generates a "textual gradient", which is a natural language critique of the prompt. For example: "The prompt is ambiguous about how to handle tie-breakers for equally good rooms." 3. **Rewrite:** Finally, it gives the critique and the original prompt to another LLM (`gpt-4.1-mini`) and asks it to apply the edits, generating a new, improved prompt template. This cycle repeats, with each round producing a slightly better prompt. To use it, you simply initialize the APO class with your desired hyperparameters. `[](https://microsoft.github.io/agent-lightning/0.3.0/how-to/train-first-agent/#__codelineno-2-1) # In the main training script: run_apo.py [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/train-first-agent/#__codelineno-2-2) from openai import AsyncOpenAI [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/train-first-agent/#__codelineno-2-3) [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/train-first-agent/#__codelineno-2-4) openai = AsyncOpenAI() [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/train-first-agent/#__codelineno-2-5) algo = agl.APO(openai)` Tip Make sure you have `OPENAI_API_KEY` set in your environment variables. ### The Trainer[¶](https://microsoft.github.io/agent-lightning/0.3.0/how-to/train-first-agent/#the-trainer "Permanent link") The Trainer is the central component you'll interact with. It connects everything and manages the entire workflow by running the loop described above. You configure the Trainer, providing the algorithm, the number of parallel runners, and the initial prompt. A single call to [`trainer.fit()`](https://microsoft.github.io/agent-lightning/0.3.0/reference/trainer/#agentlightning.Trainer.fit " fit(agent, train_dataset=None, *, val_dataset=None)") kicks off the entire process! ``[](https://microsoft.github.io/agent-lightning/0.3.0/how-to/train-first-agent/#__codelineno-3-1) # 1. Configure the Trainer with the algorithm and initial prompt [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/train-first-agent/#__codelineno-3-2) trainer = agl.Trainer( [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/train-first-agent/#__codelineno-3-3) algorithm=algo, [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/train-first-agent/#__codelineno-3-4) n_runners=8, # Run 8 agents in parallel to try out the prompts [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/train-first-agent/#__codelineno-3-5) initial_resources={ [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/train-first-agent/#__codelineno-3-6) # The initial prompt template to be tuned [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/train-first-agent/#__codelineno-3-7) "prompt_template": prompt_template_baseline() [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/train-first-agent/#__codelineno-3-8) }, [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/train-first-agent/#__codelineno-3-9) # This is used to convert the span data into a message format consumable by APO algorithm [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/train-first-agent/#__codelineno-3-10) adapter=agl.TraceToMessages(), [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/train-first-agent/#__codelineno-3-11) ) [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/train-first-agent/#__codelineno-3-12) [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/train-first-agent/#__codelineno-3-13) # 2. Load datasets: They can be list of task objects consumable by `room_selector`. [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/train-first-agent/#__codelineno-3-14) dataset_train, dataset_val = ... [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/train-first-agent/#__codelineno-3-15) [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/train-first-agent/#__codelineno-3-16) # 3. Start the training process! [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/train-first-agent/#__codelineno-3-17) trainer.fit( [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/train-first-agent/#__codelineno-3-18) agent=room_selector, [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/train-first-agent/#__codelineno-3-19) train_dataset=dataset_train, [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/train-first-agent/#__codelineno-3-20) val_dataset=dataset_val [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/train-first-agent/#__codelineno-3-21) )`` Tip [`TraceToMessages`](https://microsoft.github.io/agent-lightning/0.3.0/reference/algorithm/#agentlightning.TraceToMessages " agentlightning.TraceToMessages") is a convenience adapter that converts spans into OpenAI chat messages. It requires `openai >= 1.100.0` to be installed. Training Results[¶](https://microsoft.github.io/agent-lightning/0.3.0/how-to/train-first-agent/#training-results "Permanent link") ----------------------------------------------------------------------------------------------------------------------------------- The APO algorithm successfully improved the agent's performance. We ran the example with the following hyper-parameters: * `val_batch_size` = 10 * `gradient_batch_size` = 4 * `beam_width` = 2 * `branch_factor` = 2 * `beam_rounds` = 2 The validation accuracy on the 29 samples of datasets steadily increase from 0.569 (baseline) to **0.721** (after round 2). The tuning takes around 10 minutes with 8 runners. We ran twice, and the results are shown in the chart below. This demonstrates how Agent-lightning can efficiently and automatically enhance your agent's capabilities with just a few lines of code. Back to top --- # Write the First Algorithm - Agent-lightning [Skip to content](https://microsoft.github.io/agent-lightning/0.3.0/how-to/write-first-algorithm/#write-the-first-algorithm-with-agent-lightning) ⚠️ You are viewing development documentation. For stable documentation, please visit [https://microsoft.github.io/agent-lightning/stable/](https://microsoft.github.io/agent-lightning/stable/) Write the First Algorithm with Agent-lightning[¶](https://microsoft.github.io/agent-lightning/0.3.0/how-to/write-first-algorithm/#write-the-first-algorithm-with-agent-lightning "Permanent link") =================================================================================================================================================================================================== In the [first tutorial](https://microsoft.github.io/agent-lightning/0.3.0/how-to/train-first-agent/) , "Train the First Agent," we introduced the [Trainer](https://microsoft.github.io/agent-lightning/0.3.0/reference/trainer/#agentlightning.Trainer " agentlightning.Trainer") and showed how to use a pre-built algorithm like **Automatic Prompt Optimization (APO)** to improve an agent's performance. The [Trainer](https://microsoft.github.io/agent-lightning/0.3.0/reference/trainer/#agentlightning.Trainer " agentlightning.Trainer") handled all the complex interactions, letting us focus on the agent's logic. Now, we'll go a step deeper. What if you have a unique training idea that doesn't fit a standard algorithm? This tutorial will show you how to write your own custom algorithm from scratch. We'll build a simple algorithm that systematically tests a list of prompt templates and identifies the one with the highest reward. By the end, you'll understand the core mechanics of how the **Algorithm**, **Runner**, and a new component, the **Store**, work together to create the powerful training loop at the heart of Agent-lightning. Tip This tutorial helps you build a basic understanding of how to interact with Agent-lightning's core components. It's recommended that all users customizing algorithms should read this tutorial, even for those who are not planning to do prompt optimization. Core Concepts for Training[¶](https://microsoft.github.io/agent-lightning/0.3.0/how-to/write-first-algorithm/#core-concepts-for-training "Permanent link") ----------------------------------------------------------------------------------------------------------------------------------------------------------- Before diving into the [LightningStore](https://microsoft.github.io/agent-lightning/0.3.0/reference/store/#agentlightning.LightningStore " agentlightning.LightningStore") , let's define two key concepts that are central to any training process in Agent-lightning: **Resources** and the **Tracer**. ### Resources: The Tunable Assets[¶](https://microsoft.github.io/agent-lightning/0.3.0/how-to/write-first-algorithm/#resources-the-tunable-assets "Permanent link") [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/write-first-algorithm/) **Resources** are the assets your algorithm is trying to improve. Think of them as the "recipe" an agent uses to perform its task. This recipe can be: * A **prompt template** that guides an LLM. * The **weights** of a machine learning model. * Any other configuration or data your agent needs. The algorithm's job is to run experiments and iteratively update these resources to find the best-performing version. ### Tracer: The Data Collector[¶](https://microsoft.github.io/agent-lightning/0.3.0/how-to/write-first-algorithm/#tracer-the-data-collector "Permanent link") How does the algorithm know if a change was an improvement? It needs data. This is where the **Tracer** comes in. The Tracer automatically **instruments** (aka modifies / patches) the agent's code. This means it watches for important events, like an LLM call, a tool being used, or **reward signals**, and records a detailed log of what happened. Each of these logs is called a **Span** (which has already been introduced in the [last tutorial](https://microsoft.github.io/agent-lightning/0.3.0/how-to/train-first-agent/) ). A collection of spans from a single task execution gives the algorithm a complete, step-by-step trace of the agent's behavior, which is essential for learning and making improvements. Our default tracer is built on the AgentOps SDK to support instrumenting code written in various Agent/non-agent frameworks. The Central Hub: The LightningStore[¶](https://microsoft.github.io/agent-lightning/0.3.0/how-to/write-first-algorithm/#the-central-hub-the-lightningstore "Permanent link") ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Now, where do all these resources, tasks, and spans live? They are all managed by the **LightningStore**. The LightningStore acts as the central database and message queue for the entire system. It's the single source of truth that decouples the Algorithm from the Runners. Note In the [last tutorial](https://microsoft.github.io/agent-lightning/0.3.0/how-to/train-first-agent/) we simplified the training loop, saying the Algorithm and Agent communicate "via the Trainer." That's true at a high level, but the component that makes it all possible is actually the **LightningStore**. * The **Algorithm** connects to the Store to `enqueue_rollout` (tasks) and `update_resources` (like new prompt templates). It also queries the Store to retrieve the resulting spans and rewards from completed rollouts. * The **Runners** connect to the Store to `dequeue_rollout` (polling for available tasks). After executing a task, they use the `Tracer` to write the resulting spans and status updates back to the Store. This architecture is key to Agent-lightning's scalability. Since the Algorithm and Runners only talk to the Store, they can run in different processes or even on different machines. ![Store Architecture](https://microsoft.github.io/agent-lightning/0.3.0/assets/store-api-visualized.svg) A Mental Model of What the Store Contains The [LightningStore](https://microsoft.github.io/agent-lightning/0.3.0/reference/store/#agentlightning.LightningStore " agentlightning.LightningStore") isn't just a simple database; it's an organized system for managing the entire training lifecycle. Here's what it keeps track of: * **Task Queue**: A queue of pending **Rollouts** waiting for a Runner to pick them up, interactable via `enqueue_rollout` and `dequeue_rollout`. * **Rollouts**: The record of a single task. A rollout contains metadata about the task and tracks all **Attempts** to complete it, interactable via `query_rollouts` and `wait_for_rollouts`. * **Attempts**: A single execution of a rollout. If an attempt fails (e.g., due to a network error), the Store can automatically schedules a retry if it's configured. Each attempt is linked to its parent rollout and contains the status and timing information. The rollout status is [synced](https://microsoft.github.io/agent-lightning/0.3.0/deep-dive/store/) with its children's status. **For beginners, you can assume each rollout has only one attempt unless you have explicitly configure the retry.** * **Spans**: The detailed, structured logs generated by the `Tracer` during an attempt. Each span is linked to its parent attempt and rollout. * **Resources**: A versioned collection of the assets (like prompt templates) that the algorithm creates. Each rollout is linked to the specific version of the resources it should use. Building a Custom Algorithm[¶](https://microsoft.github.io/agent-lightning/0.3.0/how-to/write-first-algorithm/#building-a-custom-algorithm "Permanent link") ------------------------------------------------------------------------------------------------------------------------------------------------------------- Let's build an algorithm that finds the best system prompt from a predefined list. The logic is straightforward: 1. Start with a list of candidate prompt templates. 2. For each template, create a "resource" bundle in the Store. 3. Enqueue a rollout (a task), telling the Runner to use this specific resource. 4. Wait for a Runner to pick up the task and complete it. 5. Query the Store to get the final reward from the rollout's spans. 6. After testing all templates, compare the rewards and declare the best one. We can implement this as a simple Python function that interacts directly with the [LightningStore](https://microsoft.github.io/agent-lightning/0.3.0/reference/store/#agentlightning.LightningStore " agentlightning.LightningStore") . `[](https://microsoft.github.io/agent-lightning/0.3.0/how-to/write-first-algorithm/#__codelineno-0-1) async def find_best_prompt(store, prompts_to_test, task_input): [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/write-first-algorithm/#__codelineno-0-2) """A simple algorithm to find the best prompt from a list.""" [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/write-first-algorithm/#__codelineno-0-3) results = [] [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/write-first-algorithm/#__codelineno-0-4) [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/write-first-algorithm/#__codelineno-0-5) # Iterate through each prompt to test it [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/write-first-algorithm/#__codelineno-0-6) for prompt in prompts_to_test: [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/write-first-algorithm/#__codelineno-0-7) print(f"[Algo] Updating prompt template to: '{prompt}'") [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/write-first-algorithm/#__codelineno-0-8) [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/write-first-algorithm/#__codelineno-0-9) # 1. Update the resources in the store with the new prompt [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/write-first-algorithm/#__codelineno-0-10) resources_update = await store.add_resources( [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/write-first-algorithm/#__codelineno-0-11) resources={"prompt_template": prompt} [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/write-first-algorithm/#__codelineno-0-12) ) [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/write-first-algorithm/#__codelineno-0-13) [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/write-first-algorithm/#__codelineno-0-14) # 2. Enqueue a rollout task for a runner to execute [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/write-first-algorithm/#__codelineno-0-15) print("[Algo] Queuing task for clients...") [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/write-first-algorithm/#__codelineno-0-16) rollout = await store.enqueue_rollout( [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/write-first-algorithm/#__codelineno-0-17) input=task_input, [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/write-first-algorithm/#__codelineno-0-18) resources_id=resources_update.resources_id, [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/write-first-algorithm/#__codelineno-0-19) ) [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/write-first-algorithm/#__codelineno-0-20) print(f"[Algo] Task '{rollout.rollout_id}' is now available for clients.") [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/write-first-algorithm/#__codelineno-0-21) [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/write-first-algorithm/#__codelineno-0-22) # 3. Wait for the rollout to be completed by a runner [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/write-first-algorithm/#__codelineno-0-23) await store.wait_for_rollouts([rollout.rollout_id]) [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/write-first-algorithm/#__codelineno-0-24) [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/write-first-algorithm/#__codelineno-0-25) # 4. Query the completed rollout and its spans [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/write-first-algorithm/#__codelineno-0-26) completed_rollout = await store.get_rollout_by_id(rollout.rollout_id) [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/write-first-algorithm/#__codelineno-0-27) print(f"[Algo] Received Result: {completed_rollout.model_dump_json(indent=None)}") [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/write-first-algorithm/#__codelineno-0-28) [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/write-first-algorithm/#__codelineno-0-29) spans = await store.query_spans(rollout.rollout_id) [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/write-first-algorithm/#__codelineno-0-30) # We expect at least two spans: one for the LLM call and one for the final reward [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/write-first-algorithm/#__codelineno-0-31) print(f"[Algo] Queried Spans:\n - " + "\n - ".join(str(span) for span in spans)) [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/write-first-algorithm/#__codelineno-0-32) # find_final_reward is a helper function to extract the reward span [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/write-first-algorithm/#__codelineno-0-33) final_reward = find_final_reward(spans) [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/write-first-algorithm/#__codelineno-0-34) print(f"[Algo] Final reward: {final_reward}\n") [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/write-first-algorithm/#__codelineno-0-35) [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/write-first-algorithm/#__codelineno-0-36) results.append((prompt, final_reward)) [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/write-first-algorithm/#__codelineno-0-37) [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/write-first-algorithm/#__codelineno-0-38) # 5. Find and print the best prompt based on the collected rewards [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/write-first-algorithm/#__codelineno-0-39) print(f"[Algo] All prompts and their rewards: {results}") [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/write-first-algorithm/#__codelineno-0-40) best_prompt, best_reward = max(results, key=lambda item: item[1]) [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/write-first-algorithm/#__codelineno-0-41) print(f"[Algo] Best prompt found: '{best_prompt}' with reward {best_reward}")` Asynchronous Operations You'll notice the `async` and `await` keywords. Agent-lightning is built on asyncio to handle concurrent operations efficiently. All interactions with the store are asynchronous network calls, so they must be awaited. The Agent and Runner[¶](https://microsoft.github.io/agent-lightning/0.3.0/how-to/write-first-algorithm/#the-agent-and-runner "Permanent link") ----------------------------------------------------------------------------------------------------------------------------------------------- Our algorithm needs an **agent** to execute the tasks and a **runner** to manage the process. The runner is a long-lived worker process. Its job is simple: 1. Connect to the [LightningStore](https://microsoft.github.io/agent-lightning/0.3.0/reference/store/#agentlightning.LightningStore " agentlightning.LightningStore") via a [LightningStoreClient](https://microsoft.github.io/agent-lightning/0.3.0/reference/store/#agentlightning.LightningStoreClient " agentlightning.LightningStoreClient") . 2. Enter a loop, constantly asking the [LightningStore](https://microsoft.github.io/agent-lightning/0.3.0/reference/store/#agentlightning.LightningStore " agentlightning.LightningStore") for new tasks (`dequeue_rollout`). 3. When it gets a task, it runs the `simple_agent` function. 4. Crucially, the runner wraps the agent execution with a **Tracer**. The tracer automatically captures all the important events (like the LLM call and the final reward) as spans and sends them back to the [LightningStore](https://microsoft.github.io/agent-lightning/0.3.0/reference/store/#agentlightning.LightningStore " agentlightning.LightningStore") . `[](https://microsoft.github.io/agent-lightning/0.3.0/how-to/write-first-algorithm/#__codelineno-1-1) # Connecting to Store [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/write-first-algorithm/#__codelineno-1-2) store = agl.LightningStoreClient("http://localhost:4747") # or some other address [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/write-first-algorithm/#__codelineno-1-3) runner = LitAgentRunner[str](tracer=AgentOpsTracer()) [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/write-first-algorithm/#__codelineno-1-4) with runner.run_context(agent=simple_agent, store=store): # <-- where the wrapping and instrumentation happens [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/write-first-algorithm/#__codelineno-1-5) await runner.iter() # polling for new tasks forever` For this example, the agent's job is to take the prompt from the resources, use it to ask an LLM a question, and return a score. `[](https://microsoft.github.io/agent-lightning/0.3.0/how-to/write-first-algorithm/#__codelineno-2-1) def simple_agent(task: str, prompt_template: PromptTemplate) -> float: [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/write-first-algorithm/#__codelineno-2-2) """An agent that answers a question and gets judged by an LLM.""" [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/write-first-algorithm/#__codelineno-2-3) client = OpenAI() [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/write-first-algorithm/#__codelineno-2-4) [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/write-first-algorithm/#__codelineno-2-5) # Generate a response using the provided prompt template [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/write-first-algorithm/#__codelineno-2-6) prompt = prompt_template.format(any_question=task) [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/write-first-algorithm/#__codelineno-2-7) response = client.chat.completions.create( [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/write-first-algorithm/#__codelineno-2-8) model="gpt-4.1-nano", messages=[{"role": "user", "content": prompt}] [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/write-first-algorithm/#__codelineno-2-9) ) [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/write-first-algorithm/#__codelineno-2-10) llm_output = response.choices[0].message.content [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/write-first-algorithm/#__codelineno-2-11) print(f"[Rollout] LLM returned: {llm_output}") [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/write-first-algorithm/#__codelineno-2-12) [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/write-first-algorithm/#__codelineno-2-13) # This llm_output and the final score are automatically logged as spans by the Tracer [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/write-first-algorithm/#__codelineno-2-14) score = random.uniform(0, 1) # Replace with actual scoring logic if needed [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/write-first-algorithm/#__codelineno-2-15) return score` Running the Example[¶](https://microsoft.github.io/agent-lightning/0.3.0/how-to/write-first-algorithm/#running-the-example "Permanent link") --------------------------------------------------------------------------------------------------------------------------------------------- To see everything in action, you'll need three separate terminal windows. Tip If you want to follow along, you can find the complete code for this example in the [apo\_custom\_algorithm.py](https://github.com/microsoft/agent-lightning/tree/3b5d733861cf313fc09821a23240bbdf3cb2ee5b/examples/apo/apo_custom_algorithm.py) file. **1\. Start the Store:** In the first terminal, start the LightningStore server. This component will wait for connections from the algorithm and the runner. The store will be listening on port `4747` ⚡ by default. `[](https://microsoft.github.io/agent-lightning/0.3.0/how-to/write-first-algorithm/#__codelineno-3-1) agl store` **2\. Start the Runner:** In the second terminal, start the runner process. It will connect to the store and wait for tasks. The code to start the runner looks like the following: `[](https://microsoft.github.io/agent-lightning/0.3.0/how-to/write-first-algorithm/#__codelineno-4-1) export OPENAI_API_KEY=sk-... # Your OpenAI API key [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/write-first-algorithm/#__codelineno-4-2) python apo_custom_algorithm.py runner` You will see output indicating the runner has started and is waiting for rollouts. `[](https://microsoft.github.io/agent-lightning/0.3.0/how-to/write-first-algorithm/#__codelineno-5-1) 2025-10-14 22:23:41,339 [INFO] ... [Worker 0] Setting up tracer... [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/write-first-algorithm/#__codelineno-5-2) 2025-10-14 22:23:41,343 [INFO] ... [Worker 0] Instrumentation applied. [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/write-first-algorithm/#__codelineno-5-3) 2025-10-14 22:23:41,494 [INFO] ... [Worker 0] AgentOps client initialized. [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/write-first-algorithm/#__codelineno-5-4) 2025-10-14 22:23:41,494 [INFO] ... [Worker 0] Started async rollouts (max: unlimited).` **3\. Start the Algorithm:** In the third terminal, run the algorithm. This will kick off the entire process. For example, we run the algorithm code shown above with the following parameters: `[](https://microsoft.github.io/agent-lightning/0.3.0/how-to/write-first-algorithm/#__codelineno-6-1) prompts_to_test = [ [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/write-first-algorithm/#__codelineno-6-2) "You are a helpful assistant. {any_question}", [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/write-first-algorithm/#__codelineno-6-3) "You are a knowledgeable AI. {any_question}", [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/write-first-algorithm/#__codelineno-6-4) "You are a friendly chatbot. {any_question}", [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/write-first-algorithm/#__codelineno-6-5) ] [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/write-first-algorithm/#__codelineno-6-6) task_input = "Why is the sky blue?" [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/write-first-algorithm/#__codelineno-6-7) store = agl.LightningStoreClient("http://localhost:4747") [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/write-first-algorithm/#__codelineno-6-8) find_best_prompt(store, prompts_to_test, task_input)` Or you can simply use our pre-written script to try out: `[](https://microsoft.github.io/agent-lightning/0.3.0/how-to/write-first-algorithm/#__codelineno-7-1) python apo_custom_algorithm.py algo` ### Understanding the Output[¶](https://microsoft.github.io/agent-lightning/0.3.0/how-to/write-first-algorithm/#understanding-the-output "Permanent link") As the algorithm runs, you'll see logs appear across all three terminals, showing the components interacting in real-time. **Algorithm Output:** The algorithm terminal shows the main control flow: updating prompts, queuing tasks, and receiving the final results. You can also see the raw span data it retrieves from the store. `[](https://microsoft.github.io/agent-lightning/0.3.0/how-to/write-first-algorithm/#__codelineno-8-1) [Algo] Updating prompt template to: 'You are a helpful assistant. {any_question}' [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/write-first-algorithm/#__codelineno-8-2) [Algo] Queuing task for clients... [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/write-first-algorithm/#__codelineno-8-3) [Algo] Task 'ro-1d18988581cd' is now available for clients. [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/write-first-algorithm/#__codelineno-8-4) [Algo] Received Result: rollout_id='ro-1d18988581cd' ... status='succeeded' ... [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/write-first-algorithm/#__codelineno-8-5) [Algo] Queried Spans: [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/write-first-algorithm/#__codelineno-8-6) - Span(name='openai.chat.completion', attributes={'gen_ai.prompt.0.content': 'You are a helpful assistant...', 'gen_ai.completion.0.content': 'The sky appears blue...'}) [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/write-first-algorithm/#__codelineno-8-7) - Span(name='reward', attributes={'value': 0.95}) [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/write-first-algorithm/#__codelineno-8-8) [Algo] Final reward: 0.95 [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/write-first-algorithm/#__codelineno-8-9) [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/write-first-algorithm/#__codelineno-8-10) [Algo] Updating prompt template to: 'You are a knowledgeable AI. {any_question}' [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/write-first-algorithm/#__codelineno-8-11) ... [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/write-first-algorithm/#__codelineno-8-12) [Algo] Final reward: 0.95 [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/write-first-algorithm/#__codelineno-8-13) [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/write-first-algorithm/#__codelineno-8-14) [Algo] Updating prompt template to: 'You are a friendly chatbot. {any_question}' [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/write-first-algorithm/#__codelineno-8-15) ... [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/write-first-algorithm/#__codelineno-8-16) [Algo] Final reward: 1.0 [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/write-first-algorithm/#__codelineno-8-17) [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/write-first-algorithm/#__codelineno-8-18) [Algo] All prompts and their rewards: [('You are a helpful assistant. {any_question}', 0.95), ('You are a knowledgeable AI. {any_question}', 0.95), ('You are a friendly chatbot. {any_question}', 1.0)] [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/write-first-algorithm/#__codelineno-8-19) [Algo] Best prompt found: 'You are a friendly chatbot. {any_question}' with reward 1.0` **Runner Output:** The runner terminal shows it picking up each task, executing the agent logic, and reporting the completion. `[](https://microsoft.github.io/agent-lightning/0.3.0/how-to/write-first-algorithm/#__codelineno-9-1) [Rollout] LLM returned: The sky appears blue due to Rayleigh scattering... [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/write-first-algorithm/#__codelineno-9-2) 2025-10-14 22:25:50,803 [INFO] ... [Worker 0 | Rollout ro-a9f54ac19af5] Completed in 4.24s. ... [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/write-first-algorithm/#__codelineno-9-3) [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/write-first-algorithm/#__codelineno-9-4) [Rollout] LLM returned: The sky looks blue because of a process called Rayleigh scattering... [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/write-first-algorithm/#__codelineno-9-5) 2025-10-14 22:25:59,863 [INFO] ... [Worker 0 | Rollout ro-c67eaa9016b6] Completed in 4.06s. ...` **Store Server Output:** The store terminal shows a detailed log of every interaction, confirming its role as the central hub. You can see requests to enqueue and dequeue rollouts, add spans, and update statuses. `[](https://microsoft.github.io/agent-lightning/0.3.0/how-to/write-first-algorithm/#__codelineno-10-1) ... "POST /enqueue_rollout HTTP/1.1" 200 ... [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/write-first-algorithm/#__codelineno-10-2) ... "GET /dequeue_rollout HTTP/1.1" 200 ... [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/write-first-algorithm/#__codelineno-10-3) ... "POST /add_span HTTP/1.1" 200 ... [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/write-first-algorithm/#__codelineno-10-4) ... "POST /update_attempt HTTP/1.1" 200 ... [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/write-first-algorithm/#__codelineno-10-5) ... "POST /wait_for_rollouts HTTP/1.1" 200 ... [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/write-first-algorithm/#__codelineno-10-6) ... "GET /query_spans/ro-c67eaa9016b6 HTTP/1.1" 200 ...` So Where is Trainer? You might be wondering why the [last tutorial](https://microsoft.github.io/agent-lightning/0.3.0/how-to/train-first-agent/) focused on the [Trainer](https://microsoft.github.io/agent-lightning/0.3.0/reference/trainer/#agentlightning.Trainer " agentlightning.Trainer") class, but we haven't used it here. Think of the [Trainer](https://microsoft.github.io/agent-lightning/0.3.0/reference/trainer/#agentlightning.Trainer " agentlightning.Trainer") as a convenient wrapper that manages the entire training process for you. It's perfect when you want to apply a pre-built algorithm to your agent without worrying about the underlying mechanics. The [Trainer](https://microsoft.github.io/agent-lightning/0.3.0/reference/trainer/#agentlightning.Trainer " agentlightning.Trainer") handles starting the [LightningStore](https://microsoft.github.io/agent-lightning/0.3.0/reference/store/#agentlightning.LightningStore " agentlightning.LightningStore") , coordinating the [Runners](https://microsoft.github.io/agent-lightning/0.3.0/reference/runner/#agentlightning.Runner " agentlightning.Runner") , managing their lifecycles, and handling errors. In this tutorial, however, our goal is to _build a new algorithm_. To do that, we need to interact directly with the core components: the [Store](https://microsoft.github.io/agent-lightning/0.3.0/reference/store/#agentlightning.LightningStore " agentlightning.LightningStore") , the [Runner](https://microsoft.github.io/agent-lightning/0.3.0/reference/runner/#agentlightning.Runner " agentlightning.Runner") , and the algorithm logic itself. Running them separately gives you more control and clearer, isolated logs, which is ideal for development and debugging. Once your custom algorithm is mature, you can package it to comply with our standard interface ([@algo](https://microsoft.github.io/agent-lightning/0.3.0/reference/algorithm/#agentlightning.algo " agentlightning.algo(func)") or [Algorithm](https://microsoft.github.io/agent-lightning/0.3.0/reference/algorithm/#agentlightning.Algorithm " agentlightning.Algorithm") ). This allows you to use it with the [Trainer](https://microsoft.github.io/agent-lightning/0.3.0/reference/trainer/#agentlightning.Trainer " agentlightning.Trainer") again, getting all the benefits of automated lifecycle management while using your own custom logic. A sample code doing this is available in [apo\_custom\_algorithm\_trainer.py](https://github.com/microsoft/agent-lightning/tree/3b5d733861cf313fc09821a23240bbdf3cb2ee5b/examples/apo/apo_custom_algorithm_trainer.py) . Back to top --- # Installation - Agent-lightning [Skip to content](https://microsoft.github.io/agent-lightning/0.2.2/tutorials/installation/#installation-guide) ⚠️ You are viewing development documentation. For stable documentation, please visit [https://microsoft.github.io/agent-lightning/stable/](https://microsoft.github.io/agent-lightning/stable/) Installation Guide[¶](https://microsoft.github.io/agent-lightning/0.2.2/tutorials/installation/#installation-guide "Permanent link") ===================================================================================================================================== This guide explains how to install **Agent-Lightning**. You can install it from **PyPI** (the Python Package Index) for general use or directly from the **source code** if you plan to contribute or need fine-grained control over dependencies. Platform and Hardware Requirements Agent-Lightning is officially supported on **Linux distributions** (Ubuntu 22.04 or later is recommended). At the moment **macOS and Windows** (outside of WSL2) are not supported. The Python runtime must be **Python 3.10 or newer**. We recommend using the latest patch release of Python 3.10, 3.11, or 3.12 to pick up performance and security updates. A **GPU is optional**—you only need CUDA-capable hardware if you plan to fine-tune model weights or run GPU-accelerated workloads. CPU-only environments are fully supported for evaluation and inference. Installing from PyPI[¶](https://microsoft.github.io/agent-lightning/0.2.2/tutorials/installation/#installing-from-pypi "Permanent link") ----------------------------------------------------------------------------------------------------------------------------------------- The easiest way to get started is by installing Agent-Lightning directly from PyPI. This ensures you get the latest **stable release** of the package, tested for compatibility and reliability. ### Install the Stable Release[¶](https://microsoft.github.io/agent-lightning/0.2.2/tutorials/installation/#install-the-stable-release "Permanent link") Run the following command in your terminal: `[](https://microsoft.github.io/agent-lightning/0.2.2/tutorials/installation/#__codelineno-0-1) pip install --upgrade agentlightning` This installs or upgrades Agent-Lightning to the newest stable version. Tip If you intend to use **Agent-Lightning** with [**VERL**](https://microsoft.github.io/agent-lightning/0.2.2/algorithm-zoo/verl/) or run any of its **example scripts**, you’ll need to install some additional dependencies. See the sections on [Algorithm-specific installation](https://microsoft.github.io/agent-lightning/0.2.2/tutorials/installation/#algorithm-specific-installation) and [Example-specific installation](https://microsoft.github.io/agent-lightning/0.2.2/tutorials/installation/#example-specific-installation) for details. ### Install the Nightly Build (Latest Features)[¶](https://microsoft.github.io/agent-lightning/0.2.2/tutorials/installation/#install-the-nightly-build-latest-features "Permanent link") Agent-Lightning also publishes **nightly builds**, which contain the latest experimental features and improvements from the main branch. These are available via **Test PyPI**. `[](https://microsoft.github.io/agent-lightning/0.2.2/tutorials/installation/#__codelineno-1-1) pip install --upgrade --index-url https://test.pypi.org/simple/ --extra-index-url https://pypi.org/simple/ agentlightning` Warning The nightly builds are cutting-edge but may include unstable or untested changes. Use them **at your own risk**, especially in production environments. Algorithm-specific Installation[¶](https://microsoft.github.io/agent-lightning/0.2.2/tutorials/installation/#algorithm-specific-installation "Permanent link") --------------------------------------------------------------------------------------------------------------------------------------------------------------- Agent-Lightning supports multiple learning algorithms. Some of them like [APO](https://microsoft.github.io/agent-lightning/0.2.2/algorithm-zoo/apo/) or [VERL](https://microsoft.github.io/agent-lightning/0.2.2/algorithm-zoo/verl/) require extra dependencies. You can install them automatically using **optional extras** or manually if you prefer finer control. ### Installing APO[¶](https://microsoft.github.io/agent-lightning/0.2.2/tutorials/installation/#installing-apo "Permanent link") [APO](https://microsoft.github.io/agent-lightning/0.2.2/algorithm-zoo/apo/) is an algorithm module that depends on libraries such as [POML](https://github.com/microsoft/POML) . You can install Agent-Lightning with APO support by running: `[](https://microsoft.github.io/agent-lightning/0.2.2/tutorials/installation/#__codelineno-2-1) pip install agentlightning[apo]` Warning APO also depends on the [OpenAI Python SDK](https://github.com/openai/openai-python) , version **2.0 or newer**. Ensure your SDK version is up to date to avoid compatibility issues. ### Installing VERL[¶](https://microsoft.github.io/agent-lightning/0.2.2/tutorials/installation/#installing-verl "Permanent link") [VERL](https://microsoft.github.io/agent-lightning/0.2.2/algorithm-zoo/verl/) integrates with libraries like **PyTorch**, **vLLM**, and **VERL framework**. Although you _can_ install all dependencies automatically, we recommend doing it manually to avoid version conflicts. `[](https://microsoft.github.io/agent-lightning/0.2.2/tutorials/installation/#__codelineno-3-1) pip install agentlightning[verl]` Recommended Manual Setup (More Stable) Automated installation may cause issues if you don’t have a compatible **PyTorch** or **CUDA** version preinstalled. For a more stable setup, install dependencies step-by-step: `[](https://microsoft.github.io/agent-lightning/0.2.2/tutorials/installation/#__codelineno-4-1) pip install torch==2.8.0 torchvision==0.23.0 --index-url https://download.pytorch.org/whl/cu128 [](https://microsoft.github.io/agent-lightning/0.2.2/tutorials/installation/#__codelineno-4-2) pip install flash-attn --no-build-isolation [](https://microsoft.github.io/agent-lightning/0.2.2/tutorials/installation/#__codelineno-4-3) pip install vllm==0.10.2 [](https://microsoft.github.io/agent-lightning/0.2.2/tutorials/installation/#__codelineno-4-4) pip install verl==0.5.0` This approach ensures compatibility with CUDA 12.8 and minimizes dependency conflicts. Example-specific Installation[¶](https://microsoft.github.io/agent-lightning/0.2.2/tutorials/installation/#example-specific-installation "Permanent link") ----------------------------------------------------------------------------------------------------------------------------------------------------------- Each example in the `examples/` directory may have its own additional dependencies. Please refer to the **README** file of each example for detailed setup instructions: [See Example READMEs](https://github.com/microsoft/agent-lightning/tree/289c9ca25b9cc8de23aba597e4e6a02a3c69d3a1/examples) . Installing from Source (for Developers and Contributors)[¶](https://microsoft.github.io/agent-lightning/0.2.2/tutorials/installation/#installing-from-source-for-developers-and-contributors "Permanent link") --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- If you plan to contribute to Agent-Lightning or prefer to work with the latest development code, install it directly from the **source repository**. ### Why Install from Source?[¶](https://microsoft.github.io/agent-lightning/0.2.2/tutorials/installation/#why-install-from-source "Permanent link") * You want to **modify or contribute** to the project. * You prefer an **isolated development environment**. * You want to test unreleased features or fix bugs locally. ### Using `uv` for Dependency Management[¶](https://microsoft.github.io/agent-lightning/0.2.2/tutorials/installation/#using-uv-for-dependency-management "Permanent link") Starting with version **0.2**, Agent-Lightning uses [`uv`](https://docs.astral.sh/uv/) as its **default dependency manager**. `uv` is a fast and safe alternative to `pip` that: * Installs packages **in seconds** (instead of minutes), * Prevents **dependency conflicts**, * Supports **grouped dependencies** for optional features. Before proceeding, make sure `uv` is installed. ### Minimal Developer Installation[¶](https://microsoft.github.io/agent-lightning/0.2.2/tutorials/installation/#minimal-developer-installation "Permanent link") `[](https://microsoft.github.io/agent-lightning/0.2.2/tutorials/installation/#__codelineno-5-1) git clone https://github.com/microsoft/agent-lightning [](https://microsoft.github.io/agent-lightning/0.2.2/tutorials/installation/#__codelineno-5-2) cd agent-lightning [](https://microsoft.github.io/agent-lightning/0.2.2/tutorials/installation/#__codelineno-5-3) uv sync --group dev` This command sets up a clean development environment with only the essential dependencies. ### Installing All Extras (CPU or GPU)[¶](https://microsoft.github.io/agent-lightning/0.2.2/tutorials/installation/#installing-all-extras-cpu-or-gpu "Permanent link") `uv sync` can also handle algorithm-specific and example-specific dependencies in one step. For a CPU-only machine: `[](https://microsoft.github.io/agent-lightning/0.2.2/tutorials/installation/#__codelineno-6-1) uv sync --frozen \ [](https://microsoft.github.io/agent-lightning/0.2.2/tutorials/installation/#__codelineno-6-2) --extra apo \ [](https://microsoft.github.io/agent-lightning/0.2.2/tutorials/installation/#__codelineno-6-3) --extra verl \ [](https://microsoft.github.io/agent-lightning/0.2.2/tutorials/installation/#__codelineno-6-4) --group dev \ [](https://microsoft.github.io/agent-lightning/0.2.2/tutorials/installation/#__codelineno-6-5) --group torch-cpu \ [](https://microsoft.github.io/agent-lightning/0.2.2/tutorials/installation/#__codelineno-6-6) --group torch-stable \ [](https://microsoft.github.io/agent-lightning/0.2.2/tutorials/installation/#__codelineno-6-7) --group trl \ [](https://microsoft.github.io/agent-lightning/0.2.2/tutorials/installation/#__codelineno-6-8) --group agents \ [](https://microsoft.github.io/agent-lightning/0.2.2/tutorials/installation/#__codelineno-6-9) --no-default-groups` For a GPU-equipped machine that is CUDA 12.8 compatible: `[](https://microsoft.github.io/agent-lightning/0.2.2/tutorials/installation/#__codelineno-7-1) uv sync --frozen \ [](https://microsoft.github.io/agent-lightning/0.2.2/tutorials/installation/#__codelineno-7-2) --extra apo \ [](https://microsoft.github.io/agent-lightning/0.2.2/tutorials/installation/#__codelineno-7-3) --extra verl \ [](https://microsoft.github.io/agent-lightning/0.2.2/tutorials/installation/#__codelineno-7-4) --group dev \ [](https://microsoft.github.io/agent-lightning/0.2.2/tutorials/installation/#__codelineno-7-5) --group torch-gpu-stable \ [](https://microsoft.github.io/agent-lightning/0.2.2/tutorials/installation/#__codelineno-7-6) --group trl \ [](https://microsoft.github.io/agent-lightning/0.2.2/tutorials/installation/#__codelineno-7-7) --group agents \ [](https://microsoft.github.io/agent-lightning/0.2.2/tutorials/installation/#__codelineno-7-8) --no-default-groups` Read more about Agent-lightning managed dependency groups [here](https://github.com/microsoft/agent-lightning/tree/289c9ca25b9cc8de23aba597e4e6a02a3c69d3a1/pyproject.toml) . ### Activating Your Environment[¶](https://microsoft.github.io/agent-lightning/0.2.2/tutorials/installation/#activating-your-environment "Permanent link") After syncing dependencies, `uv` automatically creates a virtual environment inside the `.venv/` directory. You can use it in two ways: `[](https://microsoft.github.io/agent-lightning/0.2.2/tutorials/installation/#__codelineno-8-1) # Option 1: Prefix commands with uv run [](https://microsoft.github.io/agent-lightning/0.2.2/tutorials/installation/#__codelineno-8-2) uv run python your_script.py [](https://microsoft.github.io/agent-lightning/0.2.2/tutorials/installation/#__codelineno-8-3) [](https://microsoft.github.io/agent-lightning/0.2.2/tutorials/installation/#__codelineno-8-4) # Option 2: Activate the virtual environment [](https://microsoft.github.io/agent-lightning/0.2.2/tutorials/installation/#__codelineno-8-5) source .venv/bin/activate [](https://microsoft.github.io/agent-lightning/0.2.2/tutorials/installation/#__codelineno-8-6) python your_script.py` Before Contributing Agent-Lightning enforces code style and linting rules via **pre-commit hooks**. Installing them early prevents many avoidable formatting issues. `[](https://microsoft.github.io/agent-lightning/0.2.2/tutorials/installation/#__codelineno-9-1) uv run pre-commit install [](https://microsoft.github.io/agent-lightning/0.2.2/tutorials/installation/#__codelineno-9-2) uv run pre-commit run --all-files --show-diff-on-failure --color=always` Back to top --- # Installation - Agent-lightning [Skip to content](https://microsoft.github.io/agent-lightning/0.2.1/tutorials/installation/#installation-guide) ⚠️ You are viewing development documentation. For stable documentation, please visit [https://microsoft.github.io/agent-lightning/stable/](https://microsoft.github.io/agent-lightning/stable/) Installation Guide[¶](https://microsoft.github.io/agent-lightning/0.2.1/tutorials/installation/#installation-guide "Permanent link") ===================================================================================================================================== This guide explains how to install **Agent-Lightning**. You can install it from **PyPI** (the Python Package Index) for general use or directly from the **source code** if you plan to contribute or need fine-grained control over dependencies. Platform and Hardware Requirements Agent-Lightning is officially supported on **Linux distributions** (Ubuntu 22.04 or later is recommended). At the moment **macOS and Windows** (outside of WSL2) are not supported. The Python runtime must be **Python 3.10 or newer**. We recommend using the latest patch release of Python 3.10, 3.11, or 3.12 to pick up performance and security updates. A **GPU is optional**—you only need CUDA-capable hardware if you plan to fine-tune model weights or run GPU-accelerated workloads. CPU-only environments are fully supported for evaluation and inference. Installing from PyPI[¶](https://microsoft.github.io/agent-lightning/0.2.1/tutorials/installation/#installing-from-pypi "Permanent link") ----------------------------------------------------------------------------------------------------------------------------------------- The easiest way to get started is by installing Agent-Lightning directly from PyPI. This ensures you get the latest **stable release** of the package, tested for compatibility and reliability. ### Install the Stable Release[¶](https://microsoft.github.io/agent-lightning/0.2.1/tutorials/installation/#install-the-stable-release "Permanent link") Run the following command in your terminal: `[](https://microsoft.github.io/agent-lightning/0.2.1/tutorials/installation/#__codelineno-0-1) pip install --upgrade agentlightning` This installs or upgrades Agent-Lightning to the newest stable version. Tip If you intend to use **Agent-Lightning** with [**VERL**](https://microsoft.github.io/agent-lightning/0.2.1/algorithm-zoo/verl/) or run any of its **example scripts**, you’ll need to install some additional dependencies. See the sections on [Algorithm-specific installation](https://microsoft.github.io/agent-lightning/0.2.1/tutorials/installation/#algorithm-specific-installation) and [Example-specific installation](https://microsoft.github.io/agent-lightning/0.2.1/tutorials/installation/#example-specific-installation) for details. ### Install the Nightly Build (Latest Features)[¶](https://microsoft.github.io/agent-lightning/0.2.1/tutorials/installation/#install-the-nightly-build-latest-features "Permanent link") Agent-Lightning also publishes **nightly builds**, which contain the latest experimental features and improvements from the main branch. These are available via **Test PyPI**. `[](https://microsoft.github.io/agent-lightning/0.2.1/tutorials/installation/#__codelineno-1-1) pip install --upgrade --index-url https://test.pypi.org/simple/ --extra-index-url https://pypi.org/simple/ agentlightning` Warning The nightly builds are cutting-edge but may include unstable or untested changes. Use them **at your own risk**, especially in production environments. Algorithm-specific Installation[¶](https://microsoft.github.io/agent-lightning/0.2.1/tutorials/installation/#algorithm-specific-installation "Permanent link") --------------------------------------------------------------------------------------------------------------------------------------------------------------- Agent-Lightning supports multiple learning algorithms. Some of them like [APO](https://microsoft.github.io/agent-lightning/0.2.1/algorithm-zoo/apo/) or [VERL](https://microsoft.github.io/agent-lightning/0.2.1/algorithm-zoo/verl/) require extra dependencies. You can install them automatically using **optional extras** or manually if you prefer finer control. ### Installing APO[¶](https://microsoft.github.io/agent-lightning/0.2.1/tutorials/installation/#installing-apo "Permanent link") [APO](https://microsoft.github.io/agent-lightning/0.2.1/algorithm-zoo/apo/) is an algorithm module that depends on libraries such as [POML](https://github.com/microsoft/POML) . You can install Agent-Lightning with APO support by running: `[](https://microsoft.github.io/agent-lightning/0.2.1/tutorials/installation/#__codelineno-2-1) pip install agentlightning[apo]` Warning APO also depends on the [OpenAI Python SDK](https://github.com/openai/openai-python) , version **2.0 or newer**. Ensure your SDK version is up to date to avoid compatibility issues. ### Installing VERL[¶](https://microsoft.github.io/agent-lightning/0.2.1/tutorials/installation/#installing-verl "Permanent link") [VERL](https://microsoft.github.io/agent-lightning/0.2.1/algorithm-zoo/verl/) integrates with libraries like **PyTorch**, **vLLM**, and **VERL framework**. Although you _can_ install all dependencies automatically, we recommend doing it manually to avoid version conflicts. `[](https://microsoft.github.io/agent-lightning/0.2.1/tutorials/installation/#__codelineno-3-1) pip install agentlightning[verl]` Recommended Manual Setup (More Stable) Automated installation may cause issues if you don’t have a compatible **PyTorch** or **CUDA** version preinstalled. For a more stable setup, install dependencies step-by-step: `[](https://microsoft.github.io/agent-lightning/0.2.1/tutorials/installation/#__codelineno-4-1) pip install torch==2.8.0 torchvision==0.23.0 --index-url https://download.pytorch.org/whl/cu128 [](https://microsoft.github.io/agent-lightning/0.2.1/tutorials/installation/#__codelineno-4-2) pip install flash-attn --no-build-isolation [](https://microsoft.github.io/agent-lightning/0.2.1/tutorials/installation/#__codelineno-4-3) pip install vllm==0.10.2 [](https://microsoft.github.io/agent-lightning/0.2.1/tutorials/installation/#__codelineno-4-4) pip install verl==0.5.0` This approach ensures compatibility with CUDA 12.8 and minimizes dependency conflicts. Example-specific Installation[¶](https://microsoft.github.io/agent-lightning/0.2.1/tutorials/installation/#example-specific-installation "Permanent link") ----------------------------------------------------------------------------------------------------------------------------------------------------------- Each example in the `examples/` directory may have its own additional dependencies. Please refer to the **README** file of each example for detailed setup instructions: [See Example READMEs](https://github.com/microsoft/agent-lightning/tree/01101a64ba5375b9ef3afb824b2e0af81af4ddce/examples) . Installing from Source (for Developers and Contributors)[¶](https://microsoft.github.io/agent-lightning/0.2.1/tutorials/installation/#installing-from-source-for-developers-and-contributors "Permanent link") --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- If you plan to contribute to Agent-Lightning or prefer to work with the latest development code, install it directly from the **source repository**. ### Why Install from Source?[¶](https://microsoft.github.io/agent-lightning/0.2.1/tutorials/installation/#why-install-from-source "Permanent link") * You want to **modify or contribute** to the project. * You prefer an **isolated development environment**. * You want to test unreleased features or fix bugs locally. ### Using `uv` for Dependency Management[¶](https://microsoft.github.io/agent-lightning/0.2.1/tutorials/installation/#using-uv-for-dependency-management "Permanent link") Starting with version **0.2**, Agent-Lightning uses [`uv`](https://docs.astral.sh/uv/) as its **default dependency manager**. `uv` is a fast and safe alternative to `pip` that: * Installs packages **in seconds** (instead of minutes), * Prevents **dependency conflicts**, * Supports **grouped dependencies** for optional features. Before proceeding, make sure `uv` is installed. ### Minimal Developer Installation[¶](https://microsoft.github.io/agent-lightning/0.2.1/tutorials/installation/#minimal-developer-installation "Permanent link") `[](https://microsoft.github.io/agent-lightning/0.2.1/tutorials/installation/#__codelineno-5-1) git clone https://github.com/microsoft/agent-lightning [](https://microsoft.github.io/agent-lightning/0.2.1/tutorials/installation/#__codelineno-5-2) cd agent-lightning [](https://microsoft.github.io/agent-lightning/0.2.1/tutorials/installation/#__codelineno-5-3) uv sync --group dev` This command sets up a clean development environment with only the essential dependencies. ### Installing All Extras (CPU or GPU)[¶](https://microsoft.github.io/agent-lightning/0.2.1/tutorials/installation/#installing-all-extras-cpu-or-gpu "Permanent link") `uv sync` can also handle algorithm-specific and example-specific dependencies in one step. For a CPU-only machine: `[](https://microsoft.github.io/agent-lightning/0.2.1/tutorials/installation/#__codelineno-6-1) uv sync --frozen \ [](https://microsoft.github.io/agent-lightning/0.2.1/tutorials/installation/#__codelineno-6-2) --extra apo \ [](https://microsoft.github.io/agent-lightning/0.2.1/tutorials/installation/#__codelineno-6-3) --extra verl \ [](https://microsoft.github.io/agent-lightning/0.2.1/tutorials/installation/#__codelineno-6-4) --group dev \ [](https://microsoft.github.io/agent-lightning/0.2.1/tutorials/installation/#__codelineno-6-5) --group torch-cpu \ [](https://microsoft.github.io/agent-lightning/0.2.1/tutorials/installation/#__codelineno-6-6) --group torch-stable \ [](https://microsoft.github.io/agent-lightning/0.2.1/tutorials/installation/#__codelineno-6-7) --group trl \ [](https://microsoft.github.io/agent-lightning/0.2.1/tutorials/installation/#__codelineno-6-8) --group agents \ [](https://microsoft.github.io/agent-lightning/0.2.1/tutorials/installation/#__codelineno-6-9) --no-default-groups` For a GPU-equipped machine that is CUDA 12.8 compatible: `[](https://microsoft.github.io/agent-lightning/0.2.1/tutorials/installation/#__codelineno-7-1) uv sync --frozen \ [](https://microsoft.github.io/agent-lightning/0.2.1/tutorials/installation/#__codelineno-7-2) --extra apo \ [](https://microsoft.github.io/agent-lightning/0.2.1/tutorials/installation/#__codelineno-7-3) --extra verl \ [](https://microsoft.github.io/agent-lightning/0.2.1/tutorials/installation/#__codelineno-7-4) --group dev \ [](https://microsoft.github.io/agent-lightning/0.2.1/tutorials/installation/#__codelineno-7-5) --group torch-gpu-stable \ [](https://microsoft.github.io/agent-lightning/0.2.1/tutorials/installation/#__codelineno-7-6) --group trl \ [](https://microsoft.github.io/agent-lightning/0.2.1/tutorials/installation/#__codelineno-7-7) --group agents \ [](https://microsoft.github.io/agent-lightning/0.2.1/tutorials/installation/#__codelineno-7-8) --no-default-groups` Read more about Agent-lightning managed dependency groups [here](https://github.com/microsoft/agent-lightning/tree/01101a64ba5375b9ef3afb824b2e0af81af4ddce/pyproject.toml) . ### Activating Your Environment[¶](https://microsoft.github.io/agent-lightning/0.2.1/tutorials/installation/#activating-your-environment "Permanent link") After syncing dependencies, `uv` automatically creates a virtual environment inside the `.venv/` directory. You can use it in two ways: `[](https://microsoft.github.io/agent-lightning/0.2.1/tutorials/installation/#__codelineno-8-1) # Option 1: Prefix commands with uv run [](https://microsoft.github.io/agent-lightning/0.2.1/tutorials/installation/#__codelineno-8-2) uv run python your_script.py [](https://microsoft.github.io/agent-lightning/0.2.1/tutorials/installation/#__codelineno-8-3) [](https://microsoft.github.io/agent-lightning/0.2.1/tutorials/installation/#__codelineno-8-4) # Option 2: Activate the virtual environment [](https://microsoft.github.io/agent-lightning/0.2.1/tutorials/installation/#__codelineno-8-5) source .venv/bin/activate [](https://microsoft.github.io/agent-lightning/0.2.1/tutorials/installation/#__codelineno-8-6) python your_script.py` Before Contributing Agent-Lightning enforces code style and linting rules via **pre-commit hooks**. Installing them early prevents many avoidable formatting issues. `[](https://microsoft.github.io/agent-lightning/0.2.1/tutorials/installation/#__codelineno-9-1) uv run pre-commit install [](https://microsoft.github.io/agent-lightning/0.2.1/tutorials/installation/#__codelineno-9-2) uv run pre-commit run --all-files --show-diff-on-failure --color=always` Back to top --- # Installation - Agent Lightning [Skip to content](https://microsoft.github.io/agent-lightning/0.1.2/quickstart/installation/#installation) Installation[¶](https://microsoft.github.io/agent-lightning/0.1.2/quickstart/installation/#installation "Permanent link") ========================================================================================================================== Install from PyPI[¶](https://microsoft.github.io/agent-lightning/0.1.2/quickstart/installation/#install-from-pypi "Permanent link") ------------------------------------------------------------------------------------------------------------------------------------ ### Set Up Your Environment[¶](https://microsoft.github.io/agent-lightning/0.1.2/quickstart/installation/#set-up-your-environment "Permanent link") We strongly recommend creating a new virtual environment to avoid conflicts with other packages. You can use either `conda` or `venv`. **Python 3.10 or later** is recommended. ### Install Core Training Dependencies (Optional)[¶](https://microsoft.github.io/agent-lightning/0.1.2/quickstart/installation/#install-core-training-dependencies-optional "Permanent link") If you are running RL with Agent-Lightning, the next step is to install the essential packages: `PyTorch`, `FlashAttention`, `vLLM` and `VERL`. The following versions and installation order have been tested and are confirmed to work. `[](https://microsoft.github.io/agent-lightning/0.1.2/quickstart/installation/#__codelineno-0-1) pip install torch==2.7.0 torchvision==0.22.0 torchaudio==2.7.0 --index-url https://download.pytorch.org/whl/cu128 [](https://microsoft.github.io/agent-lightning/0.1.2/quickstart/installation/#__codelineno-0-2) pip install flash-attn --no-build-isolation [](https://microsoft.github.io/agent-lightning/0.1.2/quickstart/installation/#__codelineno-0-3) pip install vllm==0.9.2 [](https://microsoft.github.io/agent-lightning/0.1.2/quickstart/installation/#__codelineno-0-4) pip install verl==0.5.0` See [this script](https://github.com/microsoft/agent-lightning/tree/3ec5ebdcc79524f78902ce98178c35830cfe1811/scripts/setup_stable_gpu.sh) for a full installation script. ### Install Agent Lightning[¶](https://microsoft.github.io/agent-lightning/0.1.2/quickstart/installation/#install-agent-lightning "Permanent link") Now, you're ready to install Agent Lightning itself. `[](https://microsoft.github.io/agent-lightning/0.1.2/quickstart/installation/#__codelineno-1-1) pip install agentlightning` ### Install Agent Frameworks (Optional)[¶](https://microsoft.github.io/agent-lightning/0.1.2/quickstart/installation/#install-agent-frameworks-optional "Permanent link") If you plan to use other agent frameworks, you can install them with the following commands. If you don't need these, feel free to skip this step. We recommend doing this as the final step to avoid dependency versions being overwritten by mistake. `[](https://microsoft.github.io/agent-lightning/0.1.2/quickstart/installation/#__codelineno-2-1) # AutoGen (Recommended to install first) [](https://microsoft.github.io/agent-lightning/0.1.2/quickstart/installation/#__codelineno-2-2) pip install "autogen-agentchat" "autogen-ext[openai]" [](https://microsoft.github.io/agent-lightning/0.1.2/quickstart/installation/#__codelineno-2-3) [](https://microsoft.github.io/agent-lightning/0.1.2/quickstart/installation/#__codelineno-2-4) # LiteLLM [](https://microsoft.github.io/agent-lightning/0.1.2/quickstart/installation/#__codelineno-2-5) pip install "litellm[proxy]" [](https://microsoft.github.io/agent-lightning/0.1.2/quickstart/installation/#__codelineno-2-6) [](https://microsoft.github.io/agent-lightning/0.1.2/quickstart/installation/#__codelineno-2-7) # MCP [](https://microsoft.github.io/agent-lightning/0.1.2/quickstart/installation/#__codelineno-2-8) pip install mcp [](https://microsoft.github.io/agent-lightning/0.1.2/quickstart/installation/#__codelineno-2-9) [](https://microsoft.github.io/agent-lightning/0.1.2/quickstart/installation/#__codelineno-2-10) # UV [](https://microsoft.github.io/agent-lightning/0.1.2/quickstart/installation/#__codelineno-2-11) pip install uv [](https://microsoft.github.io/agent-lightning/0.1.2/quickstart/installation/#__codelineno-2-12) [](https://microsoft.github.io/agent-lightning/0.1.2/quickstart/installation/#__codelineno-2-13) # OpenAI Agents [](https://microsoft.github.io/agent-lightning/0.1.2/quickstart/installation/#__codelineno-2-14) pip install openai-agents [](https://microsoft.github.io/agent-lightning/0.1.2/quickstart/installation/#__codelineno-2-15) [](https://microsoft.github.io/agent-lightning/0.1.2/quickstart/installation/#__codelineno-2-16) # LangChain [](https://microsoft.github.io/agent-lightning/0.1.2/quickstart/installation/#__codelineno-2-17) pip install langgraph "langchain[openai]" langchain-community langchain-text-splitters [](https://microsoft.github.io/agent-lightning/0.1.2/quickstart/installation/#__codelineno-2-18) [](https://microsoft.github.io/agent-lightning/0.1.2/quickstart/installation/#__codelineno-2-19) # SQL-related dependencies [](https://microsoft.github.io/agent-lightning/0.1.2/quickstart/installation/#__codelineno-2-20) pip install sqlparse nltk` ### Shortcuts for installing Extra Dependencies[¶](https://microsoft.github.io/agent-lightning/0.1.2/quickstart/installation/#shortcuts-for-installing-extra-dependencies "Permanent link") For development: `[](https://microsoft.github.io/agent-lightning/0.1.2/quickstart/installation/#__codelineno-3-1) pip install agentlightning[dev]` For agent support: `[](https://microsoft.github.io/agent-lightning/0.1.2/quickstart/installation/#__codelineno-4-1) pip install agentlightning[agent]` Install from Source[¶](https://microsoft.github.io/agent-lightning/0.1.2/quickstart/installation/#install-from-source "Permanent link") ---------------------------------------------------------------------------------------------------------------------------------------- `[](https://microsoft.github.io/agent-lightning/0.1.2/quickstart/installation/#__codelineno-5-1) git clone https://github.com/microsoft/agent-lightning [](https://microsoft.github.io/agent-lightning/0.1.2/quickstart/installation/#__codelineno-5-2) cd agent-lightning [](https://microsoft.github.io/agent-lightning/0.1.2/quickstart/installation/#__codelineno-5-3) pip install -e .[dev]` Please run pre-commit hooks before checking in code: `[](https://microsoft.github.io/agent-lightning/0.1.2/quickstart/installation/#__codelineno-6-1) pre-commit install [](https://microsoft.github.io/agent-lightning/0.1.2/quickstart/installation/#__codelineno-6-2) pre-commit run --all-files --show-diff-on-failure --color=always` Back to top --- # Installation - Agent-lightning [Skip to content](https://microsoft.github.io/agent-lightning/0.2.0/tutorials/installation/#installation-guide) ⚠️ You are viewing development documentation. For stable documentation, please visit [https://microsoft.github.io/agent-lightning/stable/](https://microsoft.github.io/agent-lightning/stable/) Installation Guide[¶](https://microsoft.github.io/agent-lightning/0.2.0/tutorials/installation/#installation-guide "Permanent link") ===================================================================================================================================== This guide explains how to install **Agent-Lightning**. You can install it from **PyPI** (the Python Package Index) for general use or directly from the **source code** if you plan to contribute or need fine-grained control over dependencies. Platform and Hardware Requirements Agent-Lightning is officially supported on **Linux distributions** (Ubuntu 22.04 or later is recommended). At the moment **macOS and Windows** (outside of WSL2) are not supported. The Python runtime must be **Python 3.10 or newer**. We recommend using the latest patch release of Python 3.10, 3.11, or 3.12 to pick up performance and security updates. A **GPU is optional**—you only need CUDA-capable hardware if you plan to fine-tune model weights or run GPU-accelerated workloads. CPU-only environments are fully supported for evaluation and inference. Installing from PyPI[¶](https://microsoft.github.io/agent-lightning/0.2.0/tutorials/installation/#installing-from-pypi "Permanent link") ----------------------------------------------------------------------------------------------------------------------------------------- The easiest way to get started is by installing Agent-Lightning directly from PyPI. This ensures you get the latest **stable release** of the package, tested for compatibility and reliability. ### Install the Stable Release[¶](https://microsoft.github.io/agent-lightning/0.2.0/tutorials/installation/#install-the-stable-release "Permanent link") Run the following command in your terminal: `[](https://microsoft.github.io/agent-lightning/0.2.0/tutorials/installation/#__codelineno-0-1) pip install --upgrade agentlightning` This installs or upgrades Agent-Lightning to the newest stable version. Tip If you intend to use **Agent-Lightning** with [**VERL**](https://microsoft.github.io/agent-lightning/0.2.0/algorithm-zoo/verl/) or run any of its **example scripts**, you’ll need to install some additional dependencies. See the sections on [Algorithm-specific installation](https://microsoft.github.io/agent-lightning/0.2.0/tutorials/installation/#algorithm-specific-installation) and [Example-specific installation](https://microsoft.github.io/agent-lightning/0.2.0/tutorials/installation/#example-specific-installation) for details. ### Install the Nightly Build (Latest Features)[¶](https://microsoft.github.io/agent-lightning/0.2.0/tutorials/installation/#install-the-nightly-build-latest-features "Permanent link") Agent-Lightning also publishes **nightly builds**, which contain the latest experimental features and improvements from the main branch. These are available via **Test PyPI**. `[](https://microsoft.github.io/agent-lightning/0.2.0/tutorials/installation/#__codelineno-1-1) pip install --upgrade --index-url https://test.pypi.org/simple/ --extra-index-url https://pypi.org/simple/ agentlightning` Warning The nightly builds are cutting-edge but may include unstable or untested changes. Use them **at your own risk**, especially in production environments. Algorithm-specific Installation[¶](https://microsoft.github.io/agent-lightning/0.2.0/tutorials/installation/#algorithm-specific-installation "Permanent link") --------------------------------------------------------------------------------------------------------------------------------------------------------------- Agent-Lightning supports multiple learning algorithms. Some of them like [APO](https://microsoft.github.io/agent-lightning/0.2.0/algorithm-zoo/apo/) or [VERL](https://microsoft.github.io/agent-lightning/0.2.0/algorithm-zoo/verl/) require extra dependencies. You can install them automatically using **optional extras** or manually if you prefer finer control. ### Installing APO[¶](https://microsoft.github.io/agent-lightning/0.2.0/tutorials/installation/#installing-apo "Permanent link") [APO](https://microsoft.github.io/agent-lightning/0.2.0/algorithm-zoo/apo/) is an algorithm module that depends on libraries such as [POML](https://github.com/microsoft/POML) . You can install Agent-Lightning with APO support by running: `[](https://microsoft.github.io/agent-lightning/0.2.0/tutorials/installation/#__codelineno-2-1) pip install agentlightning[apo]` Warning APO also depends on the [OpenAI Python SDK](https://github.com/openai/openai-python) , version **2.0 or newer**. Ensure your SDK version is up to date to avoid compatibility issues. ### Installing VERL[¶](https://microsoft.github.io/agent-lightning/0.2.0/tutorials/installation/#installing-verl "Permanent link") [VERL](https://microsoft.github.io/agent-lightning/0.2.0/algorithm-zoo/verl/) integrates with libraries like **PyTorch**, **vLLM**, and **VERL framework**. Although you _can_ install all dependencies automatically, we recommend doing it manually to avoid version conflicts. `[](https://microsoft.github.io/agent-lightning/0.2.0/tutorials/installation/#__codelineno-3-1) pip install agentlightning[verl]` Recommended Manual Setup (More Stable) Automated installation may cause issues if you don’t have a compatible **PyTorch** or **CUDA** version preinstalled. For a more stable setup, install dependencies step-by-step: `[](https://microsoft.github.io/agent-lightning/0.2.0/tutorials/installation/#__codelineno-4-1) pip install torch==2.8.0 torchvision==0.23.0 --index-url https://download.pytorch.org/whl/cu128 [](https://microsoft.github.io/agent-lightning/0.2.0/tutorials/installation/#__codelineno-4-2) pip install flash-attn --no-build-isolation [](https://microsoft.github.io/agent-lightning/0.2.0/tutorials/installation/#__codelineno-4-3) pip install vllm==0.10.2 [](https://microsoft.github.io/agent-lightning/0.2.0/tutorials/installation/#__codelineno-4-4) pip install verl==0.5.0` This approach ensures compatibility with CUDA 12.8 and minimizes dependency conflicts. Example-specific Installation[¶](https://microsoft.github.io/agent-lightning/0.2.0/tutorials/installation/#example-specific-installation "Permanent link") ----------------------------------------------------------------------------------------------------------------------------------------------------------- Each example in the `examples/` directory may have its own additional dependencies. Please refer to the **README** file of each example for detailed setup instructions: [See Example READMEs](https://github.com/microsoft/agent-lightning/tree/22454adedbfe9d14d59996356e1e493128f8ee9d/examples) . Installing from Source (for Developers and Contributors)[¶](https://microsoft.github.io/agent-lightning/0.2.0/tutorials/installation/#installing-from-source-for-developers-and-contributors "Permanent link") --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- If you plan to contribute to Agent-Lightning or prefer to work with the latest development code, install it directly from the **source repository**. ### Why Install from Source?[¶](https://microsoft.github.io/agent-lightning/0.2.0/tutorials/installation/#why-install-from-source "Permanent link") * You want to **modify or contribute** to the project. * You prefer an **isolated development environment**. * You want to test unreleased features or fix bugs locally. ### Using `uv` for Dependency Management[¶](https://microsoft.github.io/agent-lightning/0.2.0/tutorials/installation/#using-uv-for-dependency-management "Permanent link") Starting with version **0.2**, Agent-Lightning uses [`uv`](https://docs.astral.sh/uv/) as its **default dependency manager**. `uv` is a fast and safe alternative to `pip` that: * Installs packages **in seconds** (instead of minutes), * Prevents **dependency conflicts**, * Supports **grouped dependencies** for optional features. Before proceeding, make sure `uv` is installed. ### Minimal Developer Installation[¶](https://microsoft.github.io/agent-lightning/0.2.0/tutorials/installation/#minimal-developer-installation "Permanent link") `[](https://microsoft.github.io/agent-lightning/0.2.0/tutorials/installation/#__codelineno-5-1) git clone https://github.com/microsoft/agent-lightning [](https://microsoft.github.io/agent-lightning/0.2.0/tutorials/installation/#__codelineno-5-2) cd agent-lightning [](https://microsoft.github.io/agent-lightning/0.2.0/tutorials/installation/#__codelineno-5-3) uv sync --group dev` This command sets up a clean development environment with only the essential dependencies. ### Installing All Extras (CPU or GPU)[¶](https://microsoft.github.io/agent-lightning/0.2.0/tutorials/installation/#installing-all-extras-cpu-or-gpu "Permanent link") `uv sync` can also handle algorithm-specific and example-specific dependencies in one step. For a CPU-only machine: `[](https://microsoft.github.io/agent-lightning/0.2.0/tutorials/installation/#__codelineno-6-1) uv sync --frozen \ [](https://microsoft.github.io/agent-lightning/0.2.0/tutorials/installation/#__codelineno-6-2) --extra apo \ [](https://microsoft.github.io/agent-lightning/0.2.0/tutorials/installation/#__codelineno-6-3) --extra verl \ [](https://microsoft.github.io/agent-lightning/0.2.0/tutorials/installation/#__codelineno-6-4) --group dev \ [](https://microsoft.github.io/agent-lightning/0.2.0/tutorials/installation/#__codelineno-6-5) --group torch-cpu \ [](https://microsoft.github.io/agent-lightning/0.2.0/tutorials/installation/#__codelineno-6-6) --group torch-stable \ [](https://microsoft.github.io/agent-lightning/0.2.0/tutorials/installation/#__codelineno-6-7) --group trl \ [](https://microsoft.github.io/agent-lightning/0.2.0/tutorials/installation/#__codelineno-6-8) --group agents \ [](https://microsoft.github.io/agent-lightning/0.2.0/tutorials/installation/#__codelineno-6-9) --no-default-groups` For a GPU-equipped machine that is CUDA 12.8 compatible: `[](https://microsoft.github.io/agent-lightning/0.2.0/tutorials/installation/#__codelineno-7-1) uv sync --frozen \ [](https://microsoft.github.io/agent-lightning/0.2.0/tutorials/installation/#__codelineno-7-2) --extra apo \ [](https://microsoft.github.io/agent-lightning/0.2.0/tutorials/installation/#__codelineno-7-3) --extra verl \ [](https://microsoft.github.io/agent-lightning/0.2.0/tutorials/installation/#__codelineno-7-4) --group dev \ [](https://microsoft.github.io/agent-lightning/0.2.0/tutorials/installation/#__codelineno-7-5) --group torch-gpu-stable \ [](https://microsoft.github.io/agent-lightning/0.2.0/tutorials/installation/#__codelineno-7-6) --group trl \ [](https://microsoft.github.io/agent-lightning/0.2.0/tutorials/installation/#__codelineno-7-7) --group agents \ [](https://microsoft.github.io/agent-lightning/0.2.0/tutorials/installation/#__codelineno-7-8) --no-default-groups` Read more about Agent-lightning managed dependency groups [here](https://github.com/microsoft/agent-lightning/tree/22454adedbfe9d14d59996356e1e493128f8ee9d/pyproject.toml) . ### Activating Your Environment[¶](https://microsoft.github.io/agent-lightning/0.2.0/tutorials/installation/#activating-your-environment "Permanent link") After syncing dependencies, `uv` automatically creates a virtual environment inside the `.venv/` directory. You can use it in two ways: `[](https://microsoft.github.io/agent-lightning/0.2.0/tutorials/installation/#__codelineno-8-1) # Option 1: Prefix commands with uv run [](https://microsoft.github.io/agent-lightning/0.2.0/tutorials/installation/#__codelineno-8-2) uv run python your_script.py [](https://microsoft.github.io/agent-lightning/0.2.0/tutorials/installation/#__codelineno-8-3) [](https://microsoft.github.io/agent-lightning/0.2.0/tutorials/installation/#__codelineno-8-4) # Option 2: Activate the virtual environment [](https://microsoft.github.io/agent-lightning/0.2.0/tutorials/installation/#__codelineno-8-5) source .venv/bin/activate [](https://microsoft.github.io/agent-lightning/0.2.0/tutorials/installation/#__codelineno-8-6) python your_script.py` Before Contributing Agent-Lightning enforces code style and linting rules via **pre-commit hooks**. Installing them early prevents many avoidable formatting issues. `[](https://microsoft.github.io/agent-lightning/0.2.0/tutorials/installation/#__codelineno-9-1) uv run pre-commit install [](https://microsoft.github.io/agent-lightning/0.2.0/tutorials/installation/#__codelineno-9-2) uv run pre-commit run --all-files --show-diff-on-failure --color=always` Back to top --- # Examples Catalog - Agent-lightning [Skip to content](https://microsoft.github.io/agent-lightning/0.3.0/how-to/examples-catalog/#examples-catalog) ⚠️ You are viewing development documentation. For stable documentation, please visit [https://microsoft.github.io/agent-lightning/stable/](https://microsoft.github.io/agent-lightning/stable/) Examples Catalog[¶](https://microsoft.github.io/agent-lightning/0.3.0/how-to/examples-catalog/#examples-catalog "Permanent link") ================================================================================================================================== Want to Contribute? We welcome contributions to the examples catalog! Please refer to the [Contributing](https://microsoft.github.io/agent-lightning/0.3.0/community/contributing/) guide for more details. * **APO room selector** * * * Prompt-optimize a room-booking agent with the built-in APO algorithm, then contrast it with the write-your-own algorithm and debugging workflows in the tutorials. Pairs well with the [Train the First Agent how-to](https://github.com/microsoft/agent-lightning/tree/3b5d733861cf313fc09821a23240bbdf3cb2ee5b/docs/how-to/train-first-agent.md) and the [Write the First Algorithm guide](https://github.com/microsoft/agent-lightning/tree/3b5d733861cf313fc09821a23240bbdf3cb2ee5b/docs/how-to/write-first-algorithm.md) . [Browse source](https://github.com/microsoft/agent-lightning/tree/3b5d733861cf313fc09821a23240bbdf3cb2ee5b/examples/apo) * **Azure OpenAI SFT** * * * Run a supervised fine-tuning loop against Azure OpenAI: roll out the capital-lookup agent, turn traces into JSONL, launch fine-tunes, and redeploy the resulting checkpoints through Azure CLI. [Browse source](https://github.com/microsoft/agent-lightning/tree/3b5d733861cf313fc09821a23240bbdf3cb2ee5b/examples/azure) * **Calc-X VERL math** * * * VERL-based reinforcement learning setup for a math-reasoning agent that uses AutoGen plus an MCP calculator tool to solve Calc-X problems end to end. [Browse source](https://github.com/microsoft/agent-lightning/tree/3b5d733861cf313fc09821a23240bbdf3cb2ee5b/examples/calc_x) * **ChartQA vision-language RL** * * * LangGraph-powered workflow for answering chart questions end to end: rollout the multi-modality agent with GPT or vLLM, and train with VERL/GRPO plus self-refinement loops. [Browse source](https://github.com/microsoft/agent-lightning/tree/3b5d733861cf313fc09821a23240bbdf3cb2ee5b/examples/chartqa) * **Claude Code SWE-bench** * * * Instrumented driver that runs Anthropic's Claude Code workflow on SWE-bench instances while streaming traces through Agent-lightning—supports hosted vLLM, official Anthropic, or any OpenAI-compatible backend and emits datasets for downstream tuning. [Browse source](https://github.com/microsoft/agent-lightning/tree/3b5d733861cf313fc09821a23240bbdf3cb2ee5b/examples/claude_code) * **Minimal building blocks** * * * Bite-sized scripts that isolate Agent-lightning primitives (e.g., LightningStore usage, LLM proxying, minimal vLLM host) so you can study each part before composing larger workflows. [Browse source](https://github.com/microsoft/agent-lightning/tree/3b5d733861cf313fc09821a23240bbdf3cb2ee5b/examples/minimal) * **RAG (MuSiQue)** * * * Retrieval-Augmented Generation pipeline that preps a Wikipedia retriever via MCP and trains a MuSiQue QA agent with GRPO. Documented for historical reference (verified on Agent-lightning v0.1.x). [Browse source](https://github.com/microsoft/agent-lightning/tree/3b5d733861cf313fc09821a23240bbdf3cb2ee5b/examples/rag) * **Spider SQL agent** * * * LangGraph-powered text-to-SQL workflow for the Spider benchmark, combining LangChain tooling with Agent-lightning rollouts; follow along with the [how-to for training SQL agents](https://github.com/microsoft/agent-lightning/tree/3b5d733861cf313fc09821a23240bbdf3cb2ee5b/docs/how-to/train-sql-agent.md) . [Browse source](https://github.com/microsoft/agent-lightning/tree/3b5d733861cf313fc09821a23240bbdf3cb2ee5b/examples/spider) * **Tinker integration** * * * Adapter package ([`agl_tinker`](https://github.com/microsoft/agent-lightning/tree/3b5d733861cf313fc09821a23240bbdf3cb2ee5b/examples/tinker/agl_tinker) ) with Tinker plus sample CrewAI/OpenAI agents that feed Agent-lightning traces into Tinker’s reinforcement-learning backend for both toy and 20-Questions-style workflows. [Browse source](https://github.com/microsoft/agent-lightning/tree/3b5d733861cf313fc09821a23240bbdf3cb2ee5b/examples/tinker) * **Unsloth SFT** * * * Supervised fine-tuning loop that ranks math-agent rollouts, fine-tunes with Unsloth’s 4-bit LoRA stack, and mirrors the [Fine-tune with Unsloth recipe](https://github.com/microsoft/agent-lightning/tree/3b5d733861cf313fc09821a23240bbdf3cb2ee5b/docs/how-to/unsloth-sft.md) . [Browse source](https://github.com/microsoft/agent-lightning/tree/3b5d733861cf313fc09821a23240bbdf3cb2ee5b/examples/unsloth) Back to top --- # Train the First Agent - Agent-lightning [Skip to content](https://microsoft.github.io/agent-lightning/0.2.2/how-to/train-first-agent/#train-the-first-agent-with-agent-lightning) ⚠️ You are viewing development documentation. For stable documentation, please visit [https://microsoft.github.io/agent-lightning/stable/](https://microsoft.github.io/agent-lightning/stable/) Train the First Agent with Agent-lightning[¶](https://microsoft.github.io/agent-lightning/0.2.2/how-to/train-first-agent/#train-the-first-agent-with-agent-lightning "Permanent link") ======================================================================================================================================================================================= Welcome! This tutorial is your first step into making AI agents smarter using the **Agent-lightning** framework. We'll show you how to take a simple agent and automatically improve its performance through a process called [**Automatic Prompt Optimization (APO)**](https://microsoft.github.io/agent-lightning/0.2.2/algorithm-zoo/apo/) . The main goal of Agent-lightning is to provide a structured way to **train your agents**. Just like you train a machine learning model on data, you can train an agent on a task dataset. This could involve using Reinforcement Learning (RL) to teach it new behaviors or, as we'll do today, optimizing its prompts to make it more accurate and reliable. Tip You can open the sample code [room\_selector\_apo.py](https://github.com/microsoft/agent-lightning/tree/289c9ca25b9cc8de23aba597e4e6a02a3c69d3a1/examples/apo/room_selector_apo.py) and [room\_selector.py](https://github.com/microsoft/agent-lightning/tree/289c9ca25b9cc8de23aba597e4e6a02a3c69d3a1/examples/apo/room_selector.py) as you go through this tutorial. Our Example: The Room Selector Agent[¶](https://microsoft.github.io/agent-lightning/0.2.2/how-to/train-first-agent/#our-example-the-room-selector-agent "Permanent link") -------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Today, we'll work with an agent whose job is to book a meeting room. It's a common but tricky task with multiple constraints. Here's how the agent works: * **Input:** It receives a task with specific requirements, like "`Find a room for 4 people at 10:00 AM with a whiteboard.`" * **Action:** The agent uses a Large Language Model (LLM) to understand the request. It can also use tools, which are pre-defined functions it can call, to get more information, such as checking room availability in an external database. * **Output:** Its final decision is the ID of the best room it found, like "`A103`". * **Reward:** After the agent makes its choice, a separate "grader" function scores its performance on a scale of 0 to 1. This score is called its **reward**. A perfect choice gets a 1.0, while a wrong one gets a 0.0. The agent's logic is sound, but its performance heavily depends on its initial prompt. A poorly worded prompt will confuse the LLM, leading to bad decisions. Our goal is to use Agent-lightning to find the best possible prompt automatically. A Closer Look at the Agent's Logic Modern LLMs can do more than just generate text; they can decide to call functions you provide. This is often called tool use or function calling. Our agent uses this capability to make informed decisions. If you're new to this concept, you can read more about it in [OpenAI's documentation](https://platform.openai.com/docs/guides/function-calling) . Here is a sketch of the agent's logic, adhering closely to the OpenAI API: `[](https://microsoft.github.io/agent-lightning/0.2.2/how-to/train-first-agent/#__codelineno-0-1) # Pseudo-code for the Room Selector agent [](https://microsoft.github.io/agent-lightning/0.2.2/how-to/train-first-agent/#__codelineno-0-2) [](https://microsoft.github.io/agent-lightning/0.2.2/how-to/train-first-agent/#__codelineno-0-3) import openai [](https://microsoft.github.io/agent-lightning/0.2.2/how-to/train-first-agent/#__codelineno-0-4) import json [](https://microsoft.github.io/agent-lightning/0.2.2/how-to/train-first-agent/#__codelineno-0-5) [](https://microsoft.github.io/agent-lightning/0.2.2/how-to/train-first-agent/#__codelineno-0-6) def room_selector_agent(task, prompt): [](https://microsoft.github.io/agent-lightning/0.2.2/how-to/train-first-agent/#__codelineno-0-7) client = openai.OpenAI() [](https://microsoft.github.io/agent-lightning/0.2.2/how-to/train-first-agent/#__codelineno-0-8) messages = [{"role": "user", "content": prompt.format(**task)}] [](https://microsoft.github.io/agent-lightning/0.2.2/how-to/train-first-agent/#__codelineno-0-9) tools = [ ... ] # Tool definition for the LLM [](https://microsoft.github.io/agent-lightning/0.2.2/how-to/train-first-agent/#__codelineno-0-10) [](https://microsoft.github.io/agent-lightning/0.2.2/how-to/train-first-agent/#__codelineno-0-11) # 1. First LLM call to decide if a tool is needed. [](https://microsoft.github.io/agent-lightning/0.2.2/how-to/train-first-agent/#__codelineno-0-12) response = client.chat.completions.create( [](https://microsoft.github.io/agent-lightning/0.2.2/how-to/train-first-agent/#__codelineno-0-13) model="gpt-5-mini", [](https://microsoft.github.io/agent-lightning/0.2.2/how-to/train-first-agent/#__codelineno-0-14) messages=messages, [](https://microsoft.github.io/agent-lightning/0.2.2/how-to/train-first-agent/#__codelineno-0-15) tools=tools, [](https://microsoft.github.io/agent-lightning/0.2.2/how-to/train-first-agent/#__codelineno-0-16) tool_choice="auto", [](https://microsoft.github.io/agent-lightning/0.2.2/how-to/train-first-agent/#__codelineno-0-17) ) [](https://microsoft.github.io/agent-lightning/0.2.2/how-to/train-first-agent/#__codelineno-0-18) response_message = response.choices[0].message [](https://microsoft.github.io/agent-lightning/0.2.2/how-to/train-first-agent/#__codelineno-0-19) tool_calls = response_message.tool_calls [](https://microsoft.github.io/agent-lightning/0.2.2/how-to/train-first-agent/#__codelineno-0-20) [](https://microsoft.github.io/agent-lightning/0.2.2/how-to/train-first-agent/#__codelineno-0-21) # 2. Check if the LLM wants to use a tool. [](https://microsoft.github.io/agent-lightning/0.2.2/how-to/train-first-agent/#__codelineno-0-22) if tool_calls: [](https://microsoft.github.io/agent-lightning/0.2.2/how-to/train-first-agent/#__codelineno-0-23) messages.append(response_message) # Append assistant's reply [](https://microsoft.github.io/agent-lightning/0.2.2/how-to/train-first-agent/#__codelineno-0-24) [](https://microsoft.github.io/agent-lightning/0.2.2/how-to/train-first-agent/#__codelineno-0-25) # 3. Execute the tool and get the real-world data. [](https://microsoft.github.io/agent-lightning/0.2.2/how-to/train-first-agent/#__codelineno-0-26) for tool_call in tool_calls: [](https://microsoft.github.io/agent-lightning/0.2.2/how-to/train-first-agent/#__codelineno-0-27) function_name = tool_call.function.name [](https://microsoft.github.io/agent-lightning/0.2.2/how-to/train-first-agent/#__codelineno-0-28) if function_name == "get_rooms_and_availability": [](https://microsoft.github.io/agent-lightning/0.2.2/how-to/train-first-agent/#__codelineno-0-29) function_args = json.loads(tool_call.function.arguments) [](https://microsoft.github.io/agent-lightning/0.2.2/how-to/train-first-agent/#__codelineno-0-30) # Query the local room database [](https://microsoft.github.io/agent-lightning/0.2.2/how-to/train-first-agent/#__codelineno-0-31) function_response = get_rooms_and_availability( [](https://microsoft.github.io/agent-lightning/0.2.2/how-to/train-first-agent/#__codelineno-0-32) date=function_args.get("date"), [](https://microsoft.github.io/agent-lightning/0.2.2/how-to/train-first-agent/#__codelineno-0-33) time_str=function_args.get("time"), [](https://microsoft.github.io/agent-lightning/0.2.2/how-to/train-first-agent/#__codelineno-0-34) duration_min=function_args.get("duration_min"), [](https://microsoft.github.io/agent-lightning/0.2.2/how-to/train-first-agent/#__codelineno-0-35) ) [](https://microsoft.github.io/agent-lightning/0.2.2/how-to/train-first-agent/#__codelineno-0-36) messages.append({ [](https://microsoft.github.io/agent-lightning/0.2.2/how-to/train-first-agent/#__codelineno-0-37) "tool_call_id": tool_call.id, [](https://microsoft.github.io/agent-lightning/0.2.2/how-to/train-first-agent/#__codelineno-0-38) "role": "tool", [](https://microsoft.github.io/agent-lightning/0.2.2/how-to/train-first-agent/#__codelineno-0-39) "name": function_name, [](https://microsoft.github.io/agent-lightning/0.2.2/how-to/train-first-agent/#__codelineno-0-40) "content": json.dumps(function_response), [](https://microsoft.github.io/agent-lightning/0.2.2/how-to/train-first-agent/#__codelineno-0-41) }) [](https://microsoft.github.io/agent-lightning/0.2.2/how-to/train-first-agent/#__codelineno-0-42) [](https://microsoft.github.io/agent-lightning/0.2.2/how-to/train-first-agent/#__codelineno-0-43) # 4. Second LLM call with the tool's output to get a final choice. [](https://microsoft.github.io/agent-lightning/0.2.2/how-to/train-first-agent/#__codelineno-0-44) second_response = client.chat.completions.create( [](https://microsoft.github.io/agent-lightning/0.2.2/how-to/train-first-agent/#__codelineno-0-45) model="gpt-5-mini", [](https://microsoft.github.io/agent-lightning/0.2.2/how-to/train-first-agent/#__codelineno-0-46) messages=messages, [](https://microsoft.github.io/agent-lightning/0.2.2/how-to/train-first-agent/#__codelineno-0-47) ) [](https://microsoft.github.io/agent-lightning/0.2.2/how-to/train-first-agent/#__codelineno-0-48) final_choice = second_response.choices[0].message.content [](https://microsoft.github.io/agent-lightning/0.2.2/how-to/train-first-agent/#__codelineno-0-49) else: [](https://microsoft.github.io/agent-lightning/0.2.2/how-to/train-first-agent/#__codelineno-0-50) final_choice = response_message.content [](https://microsoft.github.io/agent-lightning/0.2.2/how-to/train-first-agent/#__codelineno-0-51) [](https://microsoft.github.io/agent-lightning/0.2.2/how-to/train-first-agent/#__codelineno-0-52) # 5. Grade the final choice to get a reward. [](https://microsoft.github.io/agent-lightning/0.2.2/how-to/train-first-agent/#__codelineno-0-53) reward = grade_the_choice(final_choice, task["expected_choice"]) [](https://microsoft.github.io/agent-lightning/0.2.2/how-to/train-first-agent/#__codelineno-0-54) return reward` In Agent-lightning, you wrap this logic in a Python function marked with the [`@rollout`](https://microsoft.github.io/agent-lightning/0.2.2/reference/agent/#agentlightning.rollout " agentlightning.rollout(func)") decorator, so that the agent can be managed and tuned by Agent-lightning's runner and trainer. The `prompt_template` that the APO algorithm tunes is passed in as an argument: `[](https://microsoft.github.io/agent-lightning/0.2.2/how-to/train-first-agent/#__codelineno-1-1) import agentlightning as agl [](https://microsoft.github.io/agent-lightning/0.2.2/how-to/train-first-agent/#__codelineno-1-2) [](https://microsoft.github.io/agent-lightning/0.2.2/how-to/train-first-agent/#__codelineno-1-3) @agl.rollout [](https://microsoft.github.io/agent-lightning/0.2.2/how-to/train-first-agent/#__codelineno-1-4) def room_selector(task: RoomSelectionTask, prompt_template: agl.PromptTemplate) -> float: [](https://microsoft.github.io/agent-lightning/0.2.2/how-to/train-first-agent/#__codelineno-1-5) # ... agent logic using the prompt_template ... [](https://microsoft.github.io/agent-lightning/0.2.2/how-to/train-first-agent/#__codelineno-1-6) [](https://microsoft.github.io/agent-lightning/0.2.2/how-to/train-first-agent/#__codelineno-1-7) # The final reward is determined by a grader function [](https://microsoft.github.io/agent-lightning/0.2.2/how-to/train-first-agent/#__codelineno-1-8) reward = room_selection_grader(client, final_message, task["expected_choice"]) [](https://microsoft.github.io/agent-lightning/0.2.2/how-to/train-first-agent/#__codelineno-1-9) return reward` Core Concepts: Tasks, Rollouts, Spans, and Prompt Templates[¶](https://microsoft.github.io/agent-lightning/0.2.2/how-to/train-first-agent/#core-concepts-tasks-rollouts-spans-and-prompt-templates "Permanent link") --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- To understand how Agent-lightning works, you need to know these key terms. ### Task[¶](https://microsoft.github.io/agent-lightning/0.2.2/how-to/train-first-agent/#task "Permanent link") A task is a specific input or problem statement given to the agent. It defines what the agent needs to accomplish. Analogy: Task If the agent is a chef, a task is the recipe request: "Bake a chocolate cake." ### Rollout[¶](https://microsoft.github.io/agent-lightning/0.2.2/how-to/train-first-agent/#rollout "Permanent link") A rollout is a single, complete execution of an agent attempting to solve a given **task**. It's the entire story from receiving the task to producing a final result and receiving a reward. A rollout captures a full trace of the agent's execution. Analogy: Rollout A rollout is one full attempt by the chef to bake the chocolate cake, from gathering ingredients to the final taste test. ### Span[¶](https://microsoft.github.io/agent-lightning/0.2.2/how-to/train-first-agent/#span "Permanent link") A span represents a single unit of work or an operation within a **rollout**. Spans are the building blocks of a trace. They have a start and end time and contain details about the specific operation, like an LLM call, a tool execution, or a reward calculation. For a more precise definition, see the [OpenTelemetry documentation](https://opentelemetry.io/docs/concepts/signals/traces/) . Analogy: Span If the rollout is "baking a cake," a span could be "preheating the oven," "mixing flour and sugar," or "adding frosting." Each is a distinct step or unit of work. The picture below from [ADK](https://google.github.io/adk-docs/observability/cloud-trace/) shows a typical rollout, where each rectangle in the waterfall visualizes a span. As can be seen in the visualization, spans can be sequential, parallel or nested among each other. In other frameworks, the terminilogy might be slightly different. Agent-lightning follows the terminalogies used by OpenTelemetry to avoid confusion. ![AgentOps Waterfall Visualization](https://microsoft.github.io/agent-lightning/0.2.2/assets/agentops-waterfall-visualization.jpg) ### Prompt Template[¶](https://microsoft.github.io/agent-lightning/0.2.2/how-to/train-first-agent/#prompt-template "Permanent link") A prompt template is a reusable instruction for the agent, often containing placeholders that can be filled in with specific details from a task. It is a key **"resource"** that the algorithm learns and improves over time. Analogy: Resource (Prompt Template) If the task is the recipe request, the prompt template is the master recipe card that the chef follows. The algorithm's job is to edit this recipe card to make the instructions clearer and the final dish better. The Training Loop: How the Magic Happens[¶](https://microsoft.github.io/agent-lightning/0.2.2/how-to/train-first-agent/#the-training-loop-how-the-magic-happens "Permanent link") ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Training in Agent-lightning revolves around a clear, managed loop, orchestrated by the **Trainer**. The diagram below illustrates this core interaction: ![Loop of Tasks and Spans](https://microsoft.github.io/agent-lightning/0.2.2/assets/tasks-spans-loop.svg) **The Loop Explained:** * **Algorithm to Agent (via Trainer):** The **Algorithm** (the "brain") creates an improved **Prompt Template** and selects **Tasks**. The Trainer then sends both to the Trainer. * **Agent to Algorithm (via Trainer):** For each task it receives, the Agent uses the provided prompt template to perform a Rollout, executing its logic and potentially using tools. During this rollout, the runner that runs the agent captures Spans that detail every step. The agent also calculates a Reward for its performance on the task. These spans and rewards are then sent back to the Algorithm via the Trainer. * **Algorithm Learning:** The Algorithm then analyzes these spans and rewards to learn how to improve the agent's behavior, for example, by generating a better prompt. This improved prompt is then used in the next iteration of tasks. This cycle continues, allowing the agent to continuously learn and get better at solving tasks. Note In the next tutorial, we will see that the "via Trainer" here is not accurate. It's actually via the runner and store. ### The Algorithm[¶](https://microsoft.github.io/agent-lightning/0.2.2/how-to/train-first-agent/#the-algorithm "Permanent link") The algorithm is the smart part of the system that drives the improvement. In this tutorial, we use [**APO**](https://microsoft.github.io/agent-lightning/0.2.2/algorithm-zoo/apo/#agentlightning.algorithm.apo.APO " APO") (Automatic Prompt Optimization). It works in a few steps: 1. **Evaluate:** The algorithm first asks for rollouts to be run using the current prompt template to see how well it performs. 2. **Critique:** It then looks at the detailed spans from those rollouts. Using a powerful LLM (`gpt-5-mini`), it generates a "textual gradient", which is a natural language critique of the prompt. For example: "The prompt is ambiguous about how to handle tie-breakers for equally good rooms." 3. **Rewrite:** Finally, it gives the critique and the original prompt to another LLM (`gpt-4.1-mini`) and asks it to apply the edits, generating a new, improved prompt template. This cycle repeats, with each round producing a slightly better prompt. To use it, you simply initialize the APO class with your desired hyperparameters. `[](https://microsoft.github.io/agent-lightning/0.2.2/how-to/train-first-agent/#__codelineno-2-1) # In the main training script: run_apo.py [](https://microsoft.github.io/agent-lightning/0.2.2/how-to/train-first-agent/#__codelineno-2-2) from openai import AsyncOpenAI [](https://microsoft.github.io/agent-lightning/0.2.2/how-to/train-first-agent/#__codelineno-2-3) [](https://microsoft.github.io/agent-lightning/0.2.2/how-to/train-first-agent/#__codelineno-2-4) openai = AsyncOpenAI() [](https://microsoft.github.io/agent-lightning/0.2.2/how-to/train-first-agent/#__codelineno-2-5) algo = agl.APO(openai)` Tip Make sure you have `OPENAI_API_KEY` set in your environment variables. ### The Trainer[¶](https://microsoft.github.io/agent-lightning/0.2.2/how-to/train-first-agent/#the-trainer "Permanent link") The Trainer is the central component you'll interact with. It connects everything and manages the entire workflow by running the loop described above. You configure the Trainer, providing the algorithm, the number of parallel runners, and the initial prompt. A single call to [`trainer.fit()`](https://microsoft.github.io/agent-lightning/0.2.2/reference/trainer/#agentlightning.Trainer.fit " fit(agent, train_dataset=None, *, val_dataset=None)") kicks off the entire process! ``[](https://microsoft.github.io/agent-lightning/0.2.2/how-to/train-first-agent/#__codelineno-3-1) # 1. Configure the Trainer with the algorithm and initial prompt [](https://microsoft.github.io/agent-lightning/0.2.2/how-to/train-first-agent/#__codelineno-3-2) trainer = agl.Trainer( [](https://microsoft.github.io/agent-lightning/0.2.2/how-to/train-first-agent/#__codelineno-3-3) algorithm=algo, [](https://microsoft.github.io/agent-lightning/0.2.2/how-to/train-first-agent/#__codelineno-3-4) n_runners=8, # Run 8 agents in parallel to try out the prompts [](https://microsoft.github.io/agent-lightning/0.2.2/how-to/train-first-agent/#__codelineno-3-5) initial_resources={ [](https://microsoft.github.io/agent-lightning/0.2.2/how-to/train-first-agent/#__codelineno-3-6) # The initial prompt template to be tuned [](https://microsoft.github.io/agent-lightning/0.2.2/how-to/train-first-agent/#__codelineno-3-7) "prompt_template": prompt_template_baseline() [](https://microsoft.github.io/agent-lightning/0.2.2/how-to/train-first-agent/#__codelineno-3-8) }, [](https://microsoft.github.io/agent-lightning/0.2.2/how-to/train-first-agent/#__codelineno-3-9) # This is used to convert the span data into a message format consumable by APO algorithm [](https://microsoft.github.io/agent-lightning/0.2.2/how-to/train-first-agent/#__codelineno-3-10) adapter=agl.TraceToMessages(), [](https://microsoft.github.io/agent-lightning/0.2.2/how-to/train-first-agent/#__codelineno-3-11) ) [](https://microsoft.github.io/agent-lightning/0.2.2/how-to/train-first-agent/#__codelineno-3-12) [](https://microsoft.github.io/agent-lightning/0.2.2/how-to/train-first-agent/#__codelineno-3-13) # 2. Load datasets: They can be list of task objects consumable by `room_selector`. [](https://microsoft.github.io/agent-lightning/0.2.2/how-to/train-first-agent/#__codelineno-3-14) dataset_train, dataset_val = ... [](https://microsoft.github.io/agent-lightning/0.2.2/how-to/train-first-agent/#__codelineno-3-15) [](https://microsoft.github.io/agent-lightning/0.2.2/how-to/train-first-agent/#__codelineno-3-16) # 3. Start the training process! [](https://microsoft.github.io/agent-lightning/0.2.2/how-to/train-first-agent/#__codelineno-3-17) trainer.fit( [](https://microsoft.github.io/agent-lightning/0.2.2/how-to/train-first-agent/#__codelineno-3-18) agent=room_selector, [](https://microsoft.github.io/agent-lightning/0.2.2/how-to/train-first-agent/#__codelineno-3-19) train_dataset=dataset_train, [](https://microsoft.github.io/agent-lightning/0.2.2/how-to/train-first-agent/#__codelineno-3-20) val_dataset=dataset_val [](https://microsoft.github.io/agent-lightning/0.2.2/how-to/train-first-agent/#__codelineno-3-21) )`` Tip [`TraceToMessages`](https://microsoft.github.io/agent-lightning/0.2.2/reference/algorithm/#agentlightning.TraceToMessages " agentlightning.TraceToMessages") is a convenience adapter that converts spans into OpenAI chat messages. It requires `openai >= 1.100.0` to be installed. Training Results[¶](https://microsoft.github.io/agent-lightning/0.2.2/how-to/train-first-agent/#training-results "Permanent link") ----------------------------------------------------------------------------------------------------------------------------------- The APO algorithm successfully improved the agent's performance. We ran the example with the following hyper-parameters: * `val_batch_size` = 10 * `gradient_batch_size` = 4 * `beam_width` = 2 * `branch_factor` = 2 * `beam_rounds` = 2 The validation accuracy on the 29 samples of datasets steadily increase from 0.569 (baseline) to **0.721** (after round 2). The tuning takes around 10 minutes with 8 runners. We ran twice, and the results are shown in the chart below. This demonstrates how Agent-lightning can efficiently and automatically enhance your agent's capabilities with just a few lines of code. Back to top --- # Train the First Agent - Agent-lightning [Skip to content](https://microsoft.github.io/agent-lightning/0.2.1/how-to/train-first-agent/#train-the-first-agent-with-agent-lightning) ⚠️ You are viewing development documentation. For stable documentation, please visit [https://microsoft.github.io/agent-lightning/stable/](https://microsoft.github.io/agent-lightning/stable/) Train the First Agent with Agent-lightning[¶](https://microsoft.github.io/agent-lightning/0.2.1/how-to/train-first-agent/#train-the-first-agent-with-agent-lightning "Permanent link") ======================================================================================================================================================================================= Welcome! This tutorial is your first step into making AI agents smarter using the **Agent-lightning** framework. We'll show you how to take a simple agent and automatically improve its performance through a process called [**Automatic Prompt Optimization (APO)**](https://microsoft.github.io/agent-lightning/0.2.1/algorithm-zoo/apo/) . The main goal of Agent-lightning is to provide a structured way to **train your agents**. Just like you train a machine learning model on data, you can train an agent on a task dataset. This could involve using Reinforcement Learning (RL) to teach it new behaviors or, as we'll do today, optimizing its prompts to make it more accurate and reliable. Tip You can open the sample code [room\_selector\_apo.py](https://github.com/microsoft/agent-lightning/tree/01101a64ba5375b9ef3afb824b2e0af81af4ddce/examples/apo/room_selector_apo.py) and [room\_selector.py](https://github.com/microsoft/agent-lightning/tree/01101a64ba5375b9ef3afb824b2e0af81af4ddce/examples/apo/room_selector.py) as you go through this tutorial. Our Example: The Room Selector Agent[¶](https://microsoft.github.io/agent-lightning/0.2.1/how-to/train-first-agent/#our-example-the-room-selector-agent "Permanent link") -------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Today, we'll work with an agent whose job is to book a meeting room. It's a common but tricky task with multiple constraints. Here's how the agent works: * **Input:** It receives a task with specific requirements, like "`Find a room for 4 people at 10:00 AM with a whiteboard.`" * **Action:** The agent uses a Large Language Model (LLM) to understand the request. It can also use tools, which are pre-defined functions it can call, to get more information, such as checking room availability in an external database. * **Output:** Its final decision is the ID of the best room it found, like "`A103`". * **Reward:** After the agent makes its choice, a separate "grader" function scores its performance on a scale of 0 to 1. This score is called its **reward**. A perfect choice gets a 1.0, while a wrong one gets a 0.0. The agent's logic is sound, but its performance heavily depends on its initial prompt. A poorly worded prompt will confuse the LLM, leading to bad decisions. Our goal is to use Agent-lightning to find the best possible prompt automatically. A Closer Look at the Agent's Logic Modern LLMs can do more than just generate text; they can decide to call functions you provide. This is often called tool use or function calling. Our agent uses this capability to make informed decisions. If you're new to this concept, you can read more about it in [OpenAI's documentation](https://platform.openai.com/docs/guides/function-calling) . Here is a sketch of the agent's logic, adhering closely to the OpenAI API: `[](https://microsoft.github.io/agent-lightning/0.2.1/how-to/train-first-agent/#__codelineno-0-1) # Pseudo-code for the Room Selector agent [](https://microsoft.github.io/agent-lightning/0.2.1/how-to/train-first-agent/#__codelineno-0-2) [](https://microsoft.github.io/agent-lightning/0.2.1/how-to/train-first-agent/#__codelineno-0-3) import openai [](https://microsoft.github.io/agent-lightning/0.2.1/how-to/train-first-agent/#__codelineno-0-4) import json [](https://microsoft.github.io/agent-lightning/0.2.1/how-to/train-first-agent/#__codelineno-0-5) [](https://microsoft.github.io/agent-lightning/0.2.1/how-to/train-first-agent/#__codelineno-0-6) def room_selector_agent(task, prompt): [](https://microsoft.github.io/agent-lightning/0.2.1/how-to/train-first-agent/#__codelineno-0-7) client = openai.OpenAI() [](https://microsoft.github.io/agent-lightning/0.2.1/how-to/train-first-agent/#__codelineno-0-8) messages = [{"role": "user", "content": prompt.format(**task)}] [](https://microsoft.github.io/agent-lightning/0.2.1/how-to/train-first-agent/#__codelineno-0-9) tools = [ ... ] # Tool definition for the LLM [](https://microsoft.github.io/agent-lightning/0.2.1/how-to/train-first-agent/#__codelineno-0-10) [](https://microsoft.github.io/agent-lightning/0.2.1/how-to/train-first-agent/#__codelineno-0-11) # 1. First LLM call to decide if a tool is needed. [](https://microsoft.github.io/agent-lightning/0.2.1/how-to/train-first-agent/#__codelineno-0-12) response = client.chat.completions.create( [](https://microsoft.github.io/agent-lightning/0.2.1/how-to/train-first-agent/#__codelineno-0-13) model="gpt-5-mini", [](https://microsoft.github.io/agent-lightning/0.2.1/how-to/train-first-agent/#__codelineno-0-14) messages=messages, [](https://microsoft.github.io/agent-lightning/0.2.1/how-to/train-first-agent/#__codelineno-0-15) tools=tools, [](https://microsoft.github.io/agent-lightning/0.2.1/how-to/train-first-agent/#__codelineno-0-16) tool_choice="auto", [](https://microsoft.github.io/agent-lightning/0.2.1/how-to/train-first-agent/#__codelineno-0-17) ) [](https://microsoft.github.io/agent-lightning/0.2.1/how-to/train-first-agent/#__codelineno-0-18) response_message = response.choices[0].message [](https://microsoft.github.io/agent-lightning/0.2.1/how-to/train-first-agent/#__codelineno-0-19) tool_calls = response_message.tool_calls [](https://microsoft.github.io/agent-lightning/0.2.1/how-to/train-first-agent/#__codelineno-0-20) [](https://microsoft.github.io/agent-lightning/0.2.1/how-to/train-first-agent/#__codelineno-0-21) # 2. Check if the LLM wants to use a tool. [](https://microsoft.github.io/agent-lightning/0.2.1/how-to/train-first-agent/#__codelineno-0-22) if tool_calls: [](https://microsoft.github.io/agent-lightning/0.2.1/how-to/train-first-agent/#__codelineno-0-23) messages.append(response_message) # Append assistant's reply [](https://microsoft.github.io/agent-lightning/0.2.1/how-to/train-first-agent/#__codelineno-0-24) [](https://microsoft.github.io/agent-lightning/0.2.1/how-to/train-first-agent/#__codelineno-0-25) # 3. Execute the tool and get the real-world data. [](https://microsoft.github.io/agent-lightning/0.2.1/how-to/train-first-agent/#__codelineno-0-26) for tool_call in tool_calls: [](https://microsoft.github.io/agent-lightning/0.2.1/how-to/train-first-agent/#__codelineno-0-27) function_name = tool_call.function.name [](https://microsoft.github.io/agent-lightning/0.2.1/how-to/train-first-agent/#__codelineno-0-28) if function_name == "get_rooms_and_availability": [](https://microsoft.github.io/agent-lightning/0.2.1/how-to/train-first-agent/#__codelineno-0-29) function_args = json.loads(tool_call.function.arguments) [](https://microsoft.github.io/agent-lightning/0.2.1/how-to/train-first-agent/#__codelineno-0-30) # Query the local room database [](https://microsoft.github.io/agent-lightning/0.2.1/how-to/train-first-agent/#__codelineno-0-31) function_response = get_rooms_and_availability( [](https://microsoft.github.io/agent-lightning/0.2.1/how-to/train-first-agent/#__codelineno-0-32) date=function_args.get("date"), [](https://microsoft.github.io/agent-lightning/0.2.1/how-to/train-first-agent/#__codelineno-0-33) time_str=function_args.get("time"), [](https://microsoft.github.io/agent-lightning/0.2.1/how-to/train-first-agent/#__codelineno-0-34) duration_min=function_args.get("duration_min"), [](https://microsoft.github.io/agent-lightning/0.2.1/how-to/train-first-agent/#__codelineno-0-35) ) [](https://microsoft.github.io/agent-lightning/0.2.1/how-to/train-first-agent/#__codelineno-0-36) messages.append({ [](https://microsoft.github.io/agent-lightning/0.2.1/how-to/train-first-agent/#__codelineno-0-37) "tool_call_id": tool_call.id, [](https://microsoft.github.io/agent-lightning/0.2.1/how-to/train-first-agent/#__codelineno-0-38) "role": "tool", [](https://microsoft.github.io/agent-lightning/0.2.1/how-to/train-first-agent/#__codelineno-0-39) "name": function_name, [](https://microsoft.github.io/agent-lightning/0.2.1/how-to/train-first-agent/#__codelineno-0-40) "content": json.dumps(function_response), [](https://microsoft.github.io/agent-lightning/0.2.1/how-to/train-first-agent/#__codelineno-0-41) }) [](https://microsoft.github.io/agent-lightning/0.2.1/how-to/train-first-agent/#__codelineno-0-42) [](https://microsoft.github.io/agent-lightning/0.2.1/how-to/train-first-agent/#__codelineno-0-43) # 4. Second LLM call with the tool's output to get a final choice. [](https://microsoft.github.io/agent-lightning/0.2.1/how-to/train-first-agent/#__codelineno-0-44) second_response = client.chat.completions.create( [](https://microsoft.github.io/agent-lightning/0.2.1/how-to/train-first-agent/#__codelineno-0-45) model="gpt-5-mini", [](https://microsoft.github.io/agent-lightning/0.2.1/how-to/train-first-agent/#__codelineno-0-46) messages=messages, [](https://microsoft.github.io/agent-lightning/0.2.1/how-to/train-first-agent/#__codelineno-0-47) ) [](https://microsoft.github.io/agent-lightning/0.2.1/how-to/train-first-agent/#__codelineno-0-48) final_choice = second_response.choices[0].message.content [](https://microsoft.github.io/agent-lightning/0.2.1/how-to/train-first-agent/#__codelineno-0-49) else: [](https://microsoft.github.io/agent-lightning/0.2.1/how-to/train-first-agent/#__codelineno-0-50) final_choice = response_message.content [](https://microsoft.github.io/agent-lightning/0.2.1/how-to/train-first-agent/#__codelineno-0-51) [](https://microsoft.github.io/agent-lightning/0.2.1/how-to/train-first-agent/#__codelineno-0-52) # 5. Grade the final choice to get a reward. [](https://microsoft.github.io/agent-lightning/0.2.1/how-to/train-first-agent/#__codelineno-0-53) reward = grade_the_choice(final_choice, task["expected_choice"]) [](https://microsoft.github.io/agent-lightning/0.2.1/how-to/train-first-agent/#__codelineno-0-54) return reward` In Agent-lightning, you wrap this logic in a Python function marked with the [`@rollout`](https://microsoft.github.io/agent-lightning/0.2.1/reference/agent/#agentlightning.rollout " agentlightning.rollout(func)") decorator, so that the agent can be managed and tuned by Agent-lightning's runner and trainer. The `prompt_template` that the APO algorithm tunes is passed in as an argument: `[](https://microsoft.github.io/agent-lightning/0.2.1/how-to/train-first-agent/#__codelineno-1-1) import agentlightning as agl [](https://microsoft.github.io/agent-lightning/0.2.1/how-to/train-first-agent/#__codelineno-1-2) [](https://microsoft.github.io/agent-lightning/0.2.1/how-to/train-first-agent/#__codelineno-1-3) @agl.rollout [](https://microsoft.github.io/agent-lightning/0.2.1/how-to/train-first-agent/#__codelineno-1-4) def room_selector(task: RoomSelectionTask, prompt_template: agl.PromptTemplate) -> float: [](https://microsoft.github.io/agent-lightning/0.2.1/how-to/train-first-agent/#__codelineno-1-5) # ... agent logic using the prompt_template ... [](https://microsoft.github.io/agent-lightning/0.2.1/how-to/train-first-agent/#__codelineno-1-6) [](https://microsoft.github.io/agent-lightning/0.2.1/how-to/train-first-agent/#__codelineno-1-7) # The final reward is determined by a grader function [](https://microsoft.github.io/agent-lightning/0.2.1/how-to/train-first-agent/#__codelineno-1-8) reward = room_selection_grader(client, final_message, task["expected_choice"]) [](https://microsoft.github.io/agent-lightning/0.2.1/how-to/train-first-agent/#__codelineno-1-9) return reward` Core Concepts: Tasks, Rollouts, Spans, and Prompt Templates[¶](https://microsoft.github.io/agent-lightning/0.2.1/how-to/train-first-agent/#core-concepts-tasks-rollouts-spans-and-prompt-templates "Permanent link") --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- To understand how Agent-lightning works, you need to know these key terms. ### Task[¶](https://microsoft.github.io/agent-lightning/0.2.1/how-to/train-first-agent/#task "Permanent link") A task is a specific input or problem statement given to the agent. It defines what the agent needs to accomplish. Analogy: Task If the agent is a chef, a task is the recipe request: "Bake a chocolate cake." ### Rollout[¶](https://microsoft.github.io/agent-lightning/0.2.1/how-to/train-first-agent/#rollout "Permanent link") A rollout is a single, complete execution of an agent attempting to solve a given **task**. It's the entire story from receiving the task to producing a final result and receiving a reward. A rollout captures a full trace of the agent's execution. Analogy: Rollout A rollout is one full attempt by the chef to bake the chocolate cake, from gathering ingredients to the final taste test. ### Span[¶](https://microsoft.github.io/agent-lightning/0.2.1/how-to/train-first-agent/#span "Permanent link") A span represents a single unit of work or an operation within a **rollout**. Spans are the building blocks of a trace. They have a start and end time and contain details about the specific operation, like an LLM call, a tool execution, or a reward calculation. For a more precise definition, see the [OpenTelemetry documentation](https://opentelemetry.io/docs/concepts/signals/traces/) . Analogy: Span If the rollout is "baking a cake," a span could be "preheating the oven," "mixing flour and sugar," or "adding frosting." Each is a distinct step or unit of work. The picture below from [ADK](https://google.github.io/adk-docs/observability/cloud-trace/) shows a typical rollout, where each rectangle in the waterfall visualizes a span. As can be seen in the visualization, spans can be sequential, parallel or nested among each other. In other frameworks, the terminilogy might be slightly different. Agent-lightning follows the terminalogies used by OpenTelemetry to avoid confusion. ![AgentOps Waterfall Visualization](https://microsoft.github.io/agent-lightning/0.2.1/assets/agentops-waterfall-visualization.jpg) ### Prompt Template[¶](https://microsoft.github.io/agent-lightning/0.2.1/how-to/train-first-agent/#prompt-template "Permanent link") A prompt template is a reusable instruction for the agent, often containing placeholders that can be filled in with specific details from a task. It is a key **"resource"** that the algorithm learns and improves over time. Analogy: Resource (Prompt Template) If the task is the recipe request, the prompt template is the master recipe card that the chef follows. The algorithm's job is to edit this recipe card to make the instructions clearer and the final dish better. The Training Loop: How the Magic Happens[¶](https://microsoft.github.io/agent-lightning/0.2.1/how-to/train-first-agent/#the-training-loop-how-the-magic-happens "Permanent link") ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Training in Agent-lightning revolves around a clear, managed loop, orchestrated by the **Trainer**. The diagram below illustrates this core interaction: ![Loop of Tasks and Spans](https://microsoft.github.io/agent-lightning/0.2.1/assets/tasks-spans-loop.svg) **The Loop Explained:** * **Algorithm to Agent (via Trainer):** The **Algorithm** (the "brain") creates an improved **Prompt Template** and selects **Tasks**. The Trainer then sends both to the Trainer. * **Agent to Algorithm (via Trainer):** For each task it receives, the Agent uses the provided prompt template to perform a Rollout, executing its logic and potentially using tools. During this rollout, the runner that runs the agent captures Spans that detail every step. The agent also calculates a Reward for its performance on the task. These spans and rewards are then sent back to the Algorithm via the Trainer. * **Algorithm Learning:** The Algorithm then analyzes these spans and rewards to learn how to improve the agent's behavior, for example, by generating a better prompt. This improved prompt is then used in the next iteration of tasks. This cycle continues, allowing the agent to continuously learn and get better at solving tasks. Note In the next tutorial, we will see that the "via Trainer" here is not accurate. It's actually via the runner and store. ### The Algorithm[¶](https://microsoft.github.io/agent-lightning/0.2.1/how-to/train-first-agent/#the-algorithm "Permanent link") The algorithm is the smart part of the system that drives the improvement. In this tutorial, we use [**APO**](https://microsoft.github.io/agent-lightning/0.2.1/algorithm-zoo/apo/#agentlightning.algorithm.apo.APO " APO") (Automatic Prompt Optimization). It works in a few steps: 1. **Evaluate:** The algorithm first asks for rollouts to be run using the current prompt template to see how well it performs. 2. **Critique:** It then looks at the detailed spans from those rollouts. Using a powerful LLM (`gpt-5-mini`), it generates a "textual gradient", which is a natural language critique of the prompt. For example: "The prompt is ambiguous about how to handle tie-breakers for equally good rooms." 3. **Rewrite:** Finally, it gives the critique and the original prompt to another LLM (`gpt-4.1-mini`) and asks it to apply the edits, generating a new, improved prompt template. This cycle repeats, with each round producing a slightly better prompt. To use it, you simply initialize the APO class with your desired hyperparameters. `[](https://microsoft.github.io/agent-lightning/0.2.1/how-to/train-first-agent/#__codelineno-2-1) # In the main training script: run_apo.py [](https://microsoft.github.io/agent-lightning/0.2.1/how-to/train-first-agent/#__codelineno-2-2) from openai import AsyncOpenAI [](https://microsoft.github.io/agent-lightning/0.2.1/how-to/train-first-agent/#__codelineno-2-3) [](https://microsoft.github.io/agent-lightning/0.2.1/how-to/train-first-agent/#__codelineno-2-4) openai = AsyncOpenAI() [](https://microsoft.github.io/agent-lightning/0.2.1/how-to/train-first-agent/#__codelineno-2-5) algo = agl.APO(openai)` Tip Make sure you have `OPENAI_API_KEY` set in your environment variables. ### The Trainer[¶](https://microsoft.github.io/agent-lightning/0.2.1/how-to/train-first-agent/#the-trainer "Permanent link") The Trainer is the central component you'll interact with. It connects everything and manages the entire workflow by running the loop described above. You configure the Trainer, providing the algorithm, the number of parallel runners, and the initial prompt. A single call to [`trainer.fit()`](https://microsoft.github.io/agent-lightning/0.2.1/reference/trainer/#agentlightning.Trainer.fit " fit(agent, train_dataset=None, *, val_dataset=None)") kicks off the entire process! ``[](https://microsoft.github.io/agent-lightning/0.2.1/how-to/train-first-agent/#__codelineno-3-1) # 1. Configure the Trainer with the algorithm and initial prompt [](https://microsoft.github.io/agent-lightning/0.2.1/how-to/train-first-agent/#__codelineno-3-2) trainer = agl.Trainer( [](https://microsoft.github.io/agent-lightning/0.2.1/how-to/train-first-agent/#__codelineno-3-3) algorithm=algo, [](https://microsoft.github.io/agent-lightning/0.2.1/how-to/train-first-agent/#__codelineno-3-4) n_runners=8, # Run 8 agents in parallel to try out the prompts [](https://microsoft.github.io/agent-lightning/0.2.1/how-to/train-first-agent/#__codelineno-3-5) initial_resources={ [](https://microsoft.github.io/agent-lightning/0.2.1/how-to/train-first-agent/#__codelineno-3-6) # The initial prompt template to be tuned [](https://microsoft.github.io/agent-lightning/0.2.1/how-to/train-first-agent/#__codelineno-3-7) "prompt_template": prompt_template_baseline() [](https://microsoft.github.io/agent-lightning/0.2.1/how-to/train-first-agent/#__codelineno-3-8) }, [](https://microsoft.github.io/agent-lightning/0.2.1/how-to/train-first-agent/#__codelineno-3-9) # This is used to convert the span data into a message format consumable by APO algorithm [](https://microsoft.github.io/agent-lightning/0.2.1/how-to/train-first-agent/#__codelineno-3-10) adapter=agl.TraceToMessages(), [](https://microsoft.github.io/agent-lightning/0.2.1/how-to/train-first-agent/#__codelineno-3-11) ) [](https://microsoft.github.io/agent-lightning/0.2.1/how-to/train-first-agent/#__codelineno-3-12) [](https://microsoft.github.io/agent-lightning/0.2.1/how-to/train-first-agent/#__codelineno-3-13) # 2. Load datasets: They can be list of task objects consumable by `room_selector`. [](https://microsoft.github.io/agent-lightning/0.2.1/how-to/train-first-agent/#__codelineno-3-14) dataset_train, dataset_val = ... [](https://microsoft.github.io/agent-lightning/0.2.1/how-to/train-first-agent/#__codelineno-3-15) [](https://microsoft.github.io/agent-lightning/0.2.1/how-to/train-first-agent/#__codelineno-3-16) # 3. Start the training process! [](https://microsoft.github.io/agent-lightning/0.2.1/how-to/train-first-agent/#__codelineno-3-17) trainer.fit( [](https://microsoft.github.io/agent-lightning/0.2.1/how-to/train-first-agent/#__codelineno-3-18) agent=room_selector, [](https://microsoft.github.io/agent-lightning/0.2.1/how-to/train-first-agent/#__codelineno-3-19) train_dataset=dataset_train, [](https://microsoft.github.io/agent-lightning/0.2.1/how-to/train-first-agent/#__codelineno-3-20) val_dataset=dataset_val [](https://microsoft.github.io/agent-lightning/0.2.1/how-to/train-first-agent/#__codelineno-3-21) )`` Tip [`TraceToMessages`](https://microsoft.github.io/agent-lightning/0.2.1/reference/algorithm/#agentlightning.TraceToMessages " agentlightning.TraceToMessages") is a convenience adapter that converts spans into OpenAI chat messages. It requires `openai >= 1.100.0` to be installed. Training Results[¶](https://microsoft.github.io/agent-lightning/0.2.1/how-to/train-first-agent/#training-results "Permanent link") ----------------------------------------------------------------------------------------------------------------------------------- The APO algorithm successfully improved the agent's performance. We ran the example with the following hyper-parameters: * `val_batch_size` = 10 * `gradient_batch_size` = 4 * `beam_width` = 2 * `branch_factor` = 2 * `beam_rounds` = 2 The validation accuracy on the 29 samples of datasets steadily increase from 0.569 (baseline) to **0.721** (after round 2). The tuning takes around 10 minutes with 8 runners. We ran twice, and the results are shown in the chart below. This demonstrates how Agent-lightning can efficiently and automatically enhance your agent's capabilities with just a few lines of code. Back to top --- # Getting Started - Agent Lightning [Skip to content](https://microsoft.github.io/agent-lightning/0.1.2/quickstart/getting-started/#getting-started) Getting Started[¶](https://microsoft.github.io/agent-lightning/0.1.2/quickstart/getting-started/#getting-started "Permanent link") =================================================================================================================================== This guide walks you through building your first Agent Lightning application - a simple prompt optimization system that finds the best system prompt for an AI agent. What You'll Build[¶](https://microsoft.github.io/agent-lightning/0.1.2/quickstart/getting-started/#what-youll-build "Permanent link") -------------------------------------------------------------------------------------------------------------------------------------- You'll create a distributed training system with a server that manages optimization algorithms and tasks, a client with multiple workers that execute tasks in parallel, and built-in telemetry for monitoring and debugging. Before starting, ensure you have Python 3.10 or later, Agent Lightning installed (`pip install agentlightning`), and an OpenAI API key. The complete code is available in the [examples/apo](https://github.com/microsoft/agent-lightning/tree/3ec5ebdcc79524f78902ce98178c35830cfe1811/examples/apo) directory. Part 1: Building Your Agent[¶](https://microsoft.github.io/agent-lightning/0.1.2/quickstart/getting-started/#part-1-building-your-agent "Permanent link") ---------------------------------------------------------------------------------------------------------------------------------------------------------- Let's start by creating a simple agent that can answer questions using OpenAI's API. Your agent needs to inherit from `LitAgent` and implement a `training_rollout` method. ### Step 1: Create Your Agent Class[¶](https://microsoft.github.io/agent-lightning/0.1.2/quickstart/getting-started/#step-1-create-your-agent-class "Permanent link") First, import the necessary dependencies and create your agent class: `[](https://microsoft.github.io/agent-lightning/0.1.2/quickstart/getting-started/#__codelineno-0-1) from agentlightning.litagent import LitAgent [](https://microsoft.github.io/agent-lightning/0.1.2/quickstart/getting-started/#__codelineno-0-2) [](https://microsoft.github.io/agent-lightning/0.1.2/quickstart/getting-started/#__codelineno-0-3) class SimpleAgent(LitAgent): [](https://microsoft.github.io/agent-lightning/0.1.2/quickstart/getting-started/#__codelineno-0-4) def training_rollout(self, task, rollout_id, resources): [](https://microsoft.github.io/agent-lightning/0.1.2/quickstart/getting-started/#__codelineno-0-5) """Execute a single training rollout."""` The `training_rollout` method is the heart of your agent. It receives three parameters: a `task` dictionary containing the work to do (like "What is the capital of France?"), a unique `rollout_id` for tracking this execution, and `resources` from the server - in our case, the system prompt we're testing. ### Step 2: Execute the Task[¶](https://microsoft.github.io/agent-lightning/0.1.2/quickstart/getting-started/#step-2-execute-the-task "Permanent link") Inside the training\_rollout method, extract the system prompt from resources and use it to complete the task: `[](https://microsoft.github.io/agent-lightning/0.1.2/quickstart/getting-started/#__codelineno-1-1) # Extract the system prompt being tested [](https://microsoft.github.io/agent-lightning/0.1.2/quickstart/getting-started/#__codelineno-1-2) system_prompt = resources["system_prompt"].template [](https://microsoft.github.io/agent-lightning/0.1.2/quickstart/getting-started/#__codelineno-1-3) [](https://microsoft.github.io/agent-lightning/0.1.2/quickstart/getting-started/#__codelineno-1-4) # Call OpenAI with this prompt [](https://microsoft.github.io/agent-lightning/0.1.2/quickstart/getting-started/#__codelineno-1-5) result = openai.chat.completions.create( [](https://microsoft.github.io/agent-lightning/0.1.2/quickstart/getting-started/#__codelineno-1-6) model="gpt-4o-mini", [](https://microsoft.github.io/agent-lightning/0.1.2/quickstart/getting-started/#__codelineno-1-7) messages=[ [](https://microsoft.github.io/agent-lightning/0.1.2/quickstart/getting-started/#__codelineno-1-8) {"role": "system", "content": system_prompt}, [](https://microsoft.github.io/agent-lightning/0.1.2/quickstart/getting-started/#__codelineno-1-9) {"role": "user", "content": task["prompt"]}, [](https://microsoft.github.io/agent-lightning/0.1.2/quickstart/getting-started/#__codelineno-1-10) ], [](https://microsoft.github.io/agent-lightning/0.1.2/quickstart/getting-started/#__codelineno-1-11) )` The server sends different prompts to test, and your agent uses each one to answer the same question. This lets us compare which prompt works best. ### Step 3: Return a Reward Score[¶](https://microsoft.github.io/agent-lightning/0.1.2/quickstart/getting-started/#step-3-return-a-reward-score "Permanent link") After executing the task, return a reward score between 0 and 1: `[](https://microsoft.github.io/agent-lightning/0.1.2/quickstart/getting-started/#__codelineno-2-1) # In real scenarios, calculate based on response quality [](https://microsoft.github.io/agent-lightning/0.1.2/quickstart/getting-started/#__codelineno-2-2) return random.uniform(0, 1)` Higher rewards mean better performance. In a real system, you'd calculate this with rules, or even an LLM as a judge. For now, we're using random values to demonstrate the flow. ### Step 4: Set Up the Trainer[¶](https://microsoft.github.io/agent-lightning/0.1.2/quickstart/getting-started/#step-4-set-up-the-trainer "Permanent link") To run your agent with multiple workers in parallel: `[](https://microsoft.github.io/agent-lightning/0.1.2/quickstart/getting-started/#__codelineno-3-1) from agentlightning.trainer import Trainer [](https://microsoft.github.io/agent-lightning/0.1.2/quickstart/getting-started/#__codelineno-3-2) [](https://microsoft.github.io/agent-lightning/0.1.2/quickstart/getting-started/#__codelineno-3-3) agent = SimpleAgent() [](https://microsoft.github.io/agent-lightning/0.1.2/quickstart/getting-started/#__codelineno-3-4) trainer = Trainer(n_workers=2) # Create 2 parallel workers [](https://microsoft.github.io/agent-lightning/0.1.2/quickstart/getting-started/#__codelineno-3-5) trainer.fit(agent, backend="http://127.0.0.1:9997")` The trainer creates separate processes for each worker, allowing them to execute tasks independently. This parallelization significantly speeds up the optimization process - with 2 workers, you can test prompts twice as fast. Part 2: Building the Optimization Server[¶](https://microsoft.github.io/agent-lightning/0.1.2/quickstart/getting-started/#part-2-building-the-optimization-server "Permanent link") ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ The server coordinates the training process and implements your optimization algorithm. It manages resources, distributes tasks, and collects results. ### Step 1: Initialize the Server[¶](https://microsoft.github.io/agent-lightning/0.1.2/quickstart/getting-started/#step-1-initialize-the-server "Permanent link") Create an async function to run your optimization: `[](https://microsoft.github.io/agent-lightning/0.1.2/quickstart/getting-started/#__codelineno-4-1) import asyncio [](https://microsoft.github.io/agent-lightning/0.1.2/quickstart/getting-started/#__codelineno-4-2) from agentlightning.server import AgentLightningServer [](https://microsoft.github.io/agent-lightning/0.1.2/quickstart/getting-started/#__codelineno-4-3) from agentlightning.types import PromptTemplate [](https://microsoft.github.io/agent-lightning/0.1.2/quickstart/getting-started/#__codelineno-4-4) [](https://microsoft.github.io/agent-lightning/0.1.2/quickstart/getting-started/#__codelineno-4-5) async def prompt_optimization(): [](https://microsoft.github.io/agent-lightning/0.1.2/quickstart/getting-started/#__codelineno-4-6) server = AgentLightningServer(host="127.0.0.1", port=9997) [](https://microsoft.github.io/agent-lightning/0.1.2/quickstart/getting-started/#__codelineno-4-7) await server.start()` We use async/await because the server handles multiple clients simultaneously. This allows it to queue tasks without blocking and process results as they arrive from different workers. ### Step 2: Test Different Prompts[¶](https://microsoft.github.io/agent-lightning/0.1.2/quickstart/getting-started/#step-2-test-different-prompts "Permanent link") Define the prompts you want to test and iterate through them: `[](https://microsoft.github.io/agent-lightning/0.1.2/quickstart/getting-started/#__codelineno-5-1) prompt_candidates = [ [](https://microsoft.github.io/agent-lightning/0.1.2/quickstart/getting-started/#__codelineno-5-2) "You are a helpful assistant.", [](https://microsoft.github.io/agent-lightning/0.1.2/quickstart/getting-started/#__codelineno-5-3) "You are a knowledgeable AI.", [](https://microsoft.github.io/agent-lightning/0.1.2/quickstart/getting-started/#__codelineno-5-4) "You are a friendly chatbot.", [](https://microsoft.github.io/agent-lightning/0.1.2/quickstart/getting-started/#__codelineno-5-5) ] [](https://microsoft.github.io/agent-lightning/0.1.2/quickstart/getting-started/#__codelineno-5-6) [](https://microsoft.github.io/agent-lightning/0.1.2/quickstart/getting-started/#__codelineno-5-7) for prompt in prompt_candidates: [](https://microsoft.github.io/agent-lightning/0.1.2/quickstart/getting-started/#__codelineno-5-8) # Send this prompt to all connected clients [](https://microsoft.github.io/agent-lightning/0.1.2/quickstart/getting-started/#__codelineno-5-9) resources = { [](https://microsoft.github.io/agent-lightning/0.1.2/quickstart/getting-started/#__codelineno-5-10) "system_prompt": PromptTemplate(template=prompt, engine="f-string") [](https://microsoft.github.io/agent-lightning/0.1.2/quickstart/getting-started/#__codelineno-5-11) } [](https://microsoft.github.io/agent-lightning/0.1.2/quickstart/getting-started/#__codelineno-5-12) await server.update_resources(resources)` When you update resources, all connected clients immediately receive the new system prompt. The format of resources can be arbitrary. We use the key `"system_prompt"` here as an example. The resources here are exactly you would expect at the client side, who will use this prompt for the next task they process. ### Step 3: Queue Tasks and Collect Results[¶](https://microsoft.github.io/agent-lightning/0.1.2/quickstart/getting-started/#step-3-queue-tasks-and-collect-results "Permanent link") For each prompt, queue a task and wait for results. The `{"prompt": ...}` format here is exact what you would expect from the client side code. `[](https://microsoft.github.io/agent-lightning/0.1.2/quickstart/getting-started/#__codelineno-6-1) # Queue a task for clients to process [](https://microsoft.github.io/agent-lightning/0.1.2/quickstart/getting-started/#__codelineno-6-2) task_id = await server.queue_task( [](https://microsoft.github.io/agent-lightning/0.1.2/quickstart/getting-started/#__codelineno-6-3) sample={"prompt": "What is the capital of France?"}, [](https://microsoft.github.io/agent-lightning/0.1.2/quickstart/getting-started/#__codelineno-6-4) mode="train" [](https://microsoft.github.io/agent-lightning/0.1.2/quickstart/getting-started/#__codelineno-6-5) ) [](https://microsoft.github.io/agent-lightning/0.1.2/quickstart/getting-started/#__codelineno-6-6) [](https://microsoft.github.io/agent-lightning/0.1.2/quickstart/getting-started/#__codelineno-6-7) # Wait for a client to complete it (30 second timeout) [](https://microsoft.github.io/agent-lightning/0.1.2/quickstart/getting-started/#__codelineno-6-8) rollout = await server.poll_completed_rollout(task_id, timeout=30) [](https://microsoft.github.io/agent-lightning/0.1.2/quickstart/getting-started/#__codelineno-6-9) [](https://microsoft.github.io/agent-lightning/0.1.2/quickstart/getting-started/#__codelineno-6-10) # Extract and store the reward (this comes from the return value of the client side) [](https://microsoft.github.io/agent-lightning/0.1.2/quickstart/getting-started/#__codelineno-6-11) reward = rollout.final_reward` The server queues the same question for each prompt. The rollout object contains not just the reward, but also detailed telemetry and trace information for debugging and optimization. ### Step 4: Find the Best Prompt[¶](https://microsoft.github.io/agent-lightning/0.1.2/quickstart/getting-started/#step-4-find-the-best-prompt "Permanent link") After testing all candidates, identify the winner: `[](https://microsoft.github.io/agent-lightning/0.1.2/quickstart/getting-started/#__codelineno-7-1) best_prompt = max(prompt_and_rewards, key=lambda x: x[1]) [](https://microsoft.github.io/agent-lightning/0.1.2/quickstart/getting-started/#__codelineno-7-2) print(f"Best prompt: '{best_prompt[0]}' with reward {best_prompt[1]:.3f}")` Running Your System[¶](https://microsoft.github.io/agent-lightning/0.1.2/quickstart/getting-started/#running-your-system "Permanent link") ------------------------------------------------------------------------------------------------------------------------------------------- The [Complete example code](https://github.com/microsoft/agent-lightning/tree/3ec5ebdcc79524f78902ce98178c35830cfe1811/examples/apo) can be found on the GitHub repository. To run it: Create a `.env` file with your API credentials: `[](https://microsoft.github.io/agent-lightning/0.1.2/quickstart/getting-started/#__codelineno-8-1) OPENAI_API_KEY=your-api-key-here [](https://microsoft.github.io/agent-lightning/0.1.2/quickstart/getting-started/#__codelineno-8-2) OPENAI_API_BASE=https://api.openai.com/v1 # Optional` Start the server first in one terminal: `[](https://microsoft.github.io/agent-lightning/0.1.2/quickstart/getting-started/#__codelineno-9-1) python server.py` Then start the client in another terminal: `[](https://microsoft.github.io/agent-lightning/0.1.2/quickstart/getting-started/#__codelineno-10-1) python client.py` Understanding the Output[¶](https://microsoft.github.io/agent-lightning/0.1.2/quickstart/getting-started/#understanding-the-output "Permanent link") ----------------------------------------------------------------------------------------------------------------------------------------------------- When you run the system, you'll see detailed logs from both the client and server. Understanding these logs helps you debug issues and optimize performance. ### Client Output Explained[¶](https://microsoft.github.io/agent-lightning/0.1.2/quickstart/getting-started/#client-output-explained "Permanent link") `[](https://microsoft.github.io/agent-lightning/0.1.2/quickstart/getting-started/#__codelineno-11-1) 2025-08-10 12:59:38,224 [INFO] Initializing Trainer...` The trainer is starting up and preparing to create workers. `[](https://microsoft.github.io/agent-lightning/0.1.2/quickstart/getting-started/#__codelineno-12-1) [INFO] Starting AgentOps local server on port 52081...` A local telemetry server starts to collect metrics and traces. You can access this at `http://localhost:52081` to see detailed execution traces. `[](https://microsoft.github.io/agent-lightning/0.1.2/quickstart/getting-started/#__codelineno-13-1) [INFO] Starting worker process 0... [](https://microsoft.github.io/agent-lightning/0.1.2/quickstart/getting-started/#__codelineno-13-2) [INFO] Starting worker process 1...` Two separate processes are created. Each can execute tasks independently, doubling your throughput. `[](https://microsoft.github.io/agent-lightning/0.1.2/quickstart/getting-started/#__codelineno-14-1) [INFO] [Task 1 Received] ID: rollout-c1eb987b... [](https://microsoft.github.io/agent-lightning/0.1.2/quickstart/getting-started/#__codelineno-14-2) Resources: {'system_prompt': PromptTemplate(...)}` A worker receives a task from the server along with the current prompt to test. `[](https://microsoft.github.io/agent-lightning/0.1.2/quickstart/getting-started/#__codelineno-15-1) [INFO] [Worker 0 | Rollout] Completed in 1.09s. Reward: 0.631` Worker 0 finished executing the task in 1.09 seconds and calculated a reward of 0.631. This tells you both performance (execution time) and quality (reward score). ### Server Output Explained[¶](https://microsoft.github.io/agent-lightning/0.1.2/quickstart/getting-started/#server-output-explained "Permanent link") `[](https://microsoft.github.io/agent-lightning/0.1.2/quickstart/getting-started/#__codelineno-16-1) [Algo] Testing prompt: 'You are a helpful assistant.'` The optimization algorithm selects the next prompt to test. `[](https://microsoft.github.io/agent-lightning/0.1.2/quickstart/getting-started/#__codelineno-17-1) [Algo] Task 'rollout-c1eb987b...' is now available for clients.` The task is queued and waiting for an available worker to pick it up. `[](https://microsoft.github.io/agent-lightning/0.1.2/quickstart/getting-started/#__codelineno-18-1) [Algo] Received reward: 0.631` A client completed the task and returned a performance score. The server uses this to compare prompts. `[](https://microsoft.github.io/agent-lightning/0.1.2/quickstart/getting-started/#__codelineno-19-1) [Algo] Best prompt: 'You are a knowledgeable AI.' (reward: 0.925)` After testing all prompts, the server identifies which one performed best. What's Happening Behind the Scenes[¶](https://microsoft.github.io/agent-lightning/0.1.2/quickstart/getting-started/#whats-happening-behind-the-scenes "Permanent link") ------------------------------------------------------------------------------------------------------------------------------------------------------------------------ Agent Lightning handles several complex operations automatically. Multiple workers process tasks simultaneously. Every API call and execution is tracked through the telemetry tracing system, providing detailed traces for debugging and optimization. If a worker fails, others continue processing, ensuring your optimization doesn't stop. The AgentOps tracer, enabled by default, collects comprehensive data about each execution, including API calls, timing information, token usage, and error traces. The data is sent to the server and can be accessed via `rollout.triplets` and `rollout.traces` at the server side to build more advanced automatic optimization algorithms. Next Steps[¶](https://microsoft.github.io/agent-lightning/0.1.2/quickstart/getting-started/#next-steps "Permanent link") ------------------------------------------------------------------------------------------------------------------------- Now that you have a working system, how about: * Replacing the random reward with actual quality metrics based on real response accuracy? * Testing the system prompt on a batch of different questions to see how it performs across various tasks? * Making the algorithm automatically improve the system prompt based on the best-performing ones? * Setting up a real agent system that consists of multiple prompts, and optimizing them together? Back to top --- # Train the First Agent - Agent-lightning [Skip to content](https://microsoft.github.io/agent-lightning/0.2.0/how-to/train-first-agent/#train-the-first-agent-with-agent-lightning) ⚠️ You are viewing development documentation. For stable documentation, please visit [https://microsoft.github.io/agent-lightning/stable/](https://microsoft.github.io/agent-lightning/stable/) Train the First Agent with Agent-lightning[¶](https://microsoft.github.io/agent-lightning/0.2.0/how-to/train-first-agent/#train-the-first-agent-with-agent-lightning "Permanent link") ======================================================================================================================================================================================= Welcome! This tutorial is your first step into making AI agents smarter using the **Agent-lightning** framework. We'll show you how to take a simple agent and automatically improve its performance through a process called [**Automatic Prompt Optimization (APO)**](https://microsoft.github.io/agent-lightning/0.2.0/algorithm-zoo/apo/) . The main goal of Agent-lightning is to provide a structured way to **train your agents**. Just like you train a machine learning model on data, you can train an agent on a task dataset. This could involve using Reinforcement Learning (RL) to teach it new behaviors or, as we'll do today, optimizing its prompts to make it more accurate and reliable. Tip You can open the sample code [room\_selector\_apo.py](https://github.com/microsoft/agent-lightning/tree/22454adedbfe9d14d59996356e1e493128f8ee9d/examples/apo/room_selector_apo.py) and [room\_selector.py](https://github.com/microsoft/agent-lightning/tree/22454adedbfe9d14d59996356e1e493128f8ee9d/examples/apo/room_selector.py) as you go through this tutorial. Our Example: The Room Selector Agent[¶](https://microsoft.github.io/agent-lightning/0.2.0/how-to/train-first-agent/#our-example-the-room-selector-agent "Permanent link") -------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Today, we'll work with an agent whose job is to book a meeting room. It's a common but tricky task with multiple constraints. Here's how the agent works: * **Input:** It receives a task with specific requirements, like "`Find a room for 4 people at 10:00 AM with a whiteboard.`" * **Action:** The agent uses a Large Language Model (LLM) to understand the request. It can also use tools, which are pre-defined functions it can call, to get more information, such as checking room availability in an external database. * **Output:** Its final decision is the ID of the best room it found, like "`A103`". * **Reward:** After the agent makes its choice, a separate "grader" function scores its performance on a scale of 0 to 1. This score is called its **reward**. A perfect choice gets a 1.0, while a wrong one gets a 0.0. The agent's logic is sound, but its performance heavily depends on its initial prompt. A poorly worded prompt will confuse the LLM, leading to bad decisions. Our goal is to use Agent-lightning to find the best possible prompt automatically. A Closer Look at the Agent's Logic Modern LLMs can do more than just generate text; they can decide to call functions you provide. This is often called tool use or function calling. Our agent uses this capability to make informed decisions. If you're new to this concept, you can read more about it in [OpenAI's documentation](https://platform.openai.com/docs/guides/function-calling) . Here is a sketch of the agent's logic, adhering closely to the OpenAI API: `[](https://microsoft.github.io/agent-lightning/0.2.0/how-to/train-first-agent/#__codelineno-0-1) # Pseudo-code for the Room Selector agent [](https://microsoft.github.io/agent-lightning/0.2.0/how-to/train-first-agent/#__codelineno-0-2) [](https://microsoft.github.io/agent-lightning/0.2.0/how-to/train-first-agent/#__codelineno-0-3) import openai [](https://microsoft.github.io/agent-lightning/0.2.0/how-to/train-first-agent/#__codelineno-0-4) import json [](https://microsoft.github.io/agent-lightning/0.2.0/how-to/train-first-agent/#__codelineno-0-5) [](https://microsoft.github.io/agent-lightning/0.2.0/how-to/train-first-agent/#__codelineno-0-6) def room_selector_agent(task, prompt): [](https://microsoft.github.io/agent-lightning/0.2.0/how-to/train-first-agent/#__codelineno-0-7) client = openai.OpenAI() [](https://microsoft.github.io/agent-lightning/0.2.0/how-to/train-first-agent/#__codelineno-0-8) messages = [{"role": "user", "content": prompt.format(**task)}] [](https://microsoft.github.io/agent-lightning/0.2.0/how-to/train-first-agent/#__codelineno-0-9) tools = [ ... ] # Tool definition for the LLM [](https://microsoft.github.io/agent-lightning/0.2.0/how-to/train-first-agent/#__codelineno-0-10) [](https://microsoft.github.io/agent-lightning/0.2.0/how-to/train-first-agent/#__codelineno-0-11) # 1. First LLM call to decide if a tool is needed. [](https://microsoft.github.io/agent-lightning/0.2.0/how-to/train-first-agent/#__codelineno-0-12) response = client.chat.completions.create( [](https://microsoft.github.io/agent-lightning/0.2.0/how-to/train-first-agent/#__codelineno-0-13) model="gpt-5-mini", [](https://microsoft.github.io/agent-lightning/0.2.0/how-to/train-first-agent/#__codelineno-0-14) messages=messages, [](https://microsoft.github.io/agent-lightning/0.2.0/how-to/train-first-agent/#__codelineno-0-15) tools=tools, [](https://microsoft.github.io/agent-lightning/0.2.0/how-to/train-first-agent/#__codelineno-0-16) tool_choice="auto", [](https://microsoft.github.io/agent-lightning/0.2.0/how-to/train-first-agent/#__codelineno-0-17) ) [](https://microsoft.github.io/agent-lightning/0.2.0/how-to/train-first-agent/#__codelineno-0-18) response_message = response.choices[0].message [](https://microsoft.github.io/agent-lightning/0.2.0/how-to/train-first-agent/#__codelineno-0-19) tool_calls = response_message.tool_calls [](https://microsoft.github.io/agent-lightning/0.2.0/how-to/train-first-agent/#__codelineno-0-20) [](https://microsoft.github.io/agent-lightning/0.2.0/how-to/train-first-agent/#__codelineno-0-21) # 2. Check if the LLM wants to use a tool. [](https://microsoft.github.io/agent-lightning/0.2.0/how-to/train-first-agent/#__codelineno-0-22) if tool_calls: [](https://microsoft.github.io/agent-lightning/0.2.0/how-to/train-first-agent/#__codelineno-0-23) messages.append(response_message) # Append assistant's reply [](https://microsoft.github.io/agent-lightning/0.2.0/how-to/train-first-agent/#__codelineno-0-24) [](https://microsoft.github.io/agent-lightning/0.2.0/how-to/train-first-agent/#__codelineno-0-25) # 3. Execute the tool and get the real-world data. [](https://microsoft.github.io/agent-lightning/0.2.0/how-to/train-first-agent/#__codelineno-0-26) for tool_call in tool_calls: [](https://microsoft.github.io/agent-lightning/0.2.0/how-to/train-first-agent/#__codelineno-0-27) function_name = tool_call.function.name [](https://microsoft.github.io/agent-lightning/0.2.0/how-to/train-first-agent/#__codelineno-0-28) if function_name == "get_rooms_and_availability": [](https://microsoft.github.io/agent-lightning/0.2.0/how-to/train-first-agent/#__codelineno-0-29) function_args = json.loads(tool_call.function.arguments) [](https://microsoft.github.io/agent-lightning/0.2.0/how-to/train-first-agent/#__codelineno-0-30) # Query the local room database [](https://microsoft.github.io/agent-lightning/0.2.0/how-to/train-first-agent/#__codelineno-0-31) function_response = get_rooms_and_availability( [](https://microsoft.github.io/agent-lightning/0.2.0/how-to/train-first-agent/#__codelineno-0-32) date=function_args.get("date"), [](https://microsoft.github.io/agent-lightning/0.2.0/how-to/train-first-agent/#__codelineno-0-33) time_str=function_args.get("time"), [](https://microsoft.github.io/agent-lightning/0.2.0/how-to/train-first-agent/#__codelineno-0-34) duration_min=function_args.get("duration_min"), [](https://microsoft.github.io/agent-lightning/0.2.0/how-to/train-first-agent/#__codelineno-0-35) ) [](https://microsoft.github.io/agent-lightning/0.2.0/how-to/train-first-agent/#__codelineno-0-36) messages.append({ [](https://microsoft.github.io/agent-lightning/0.2.0/how-to/train-first-agent/#__codelineno-0-37) "tool_call_id": tool_call.id, [](https://microsoft.github.io/agent-lightning/0.2.0/how-to/train-first-agent/#__codelineno-0-38) "role": "tool", [](https://microsoft.github.io/agent-lightning/0.2.0/how-to/train-first-agent/#__codelineno-0-39) "name": function_name, [](https://microsoft.github.io/agent-lightning/0.2.0/how-to/train-first-agent/#__codelineno-0-40) "content": json.dumps(function_response), [](https://microsoft.github.io/agent-lightning/0.2.0/how-to/train-first-agent/#__codelineno-0-41) }) [](https://microsoft.github.io/agent-lightning/0.2.0/how-to/train-first-agent/#__codelineno-0-42) [](https://microsoft.github.io/agent-lightning/0.2.0/how-to/train-first-agent/#__codelineno-0-43) # 4. Second LLM call with the tool's output to get a final choice. [](https://microsoft.github.io/agent-lightning/0.2.0/how-to/train-first-agent/#__codelineno-0-44) second_response = client.chat.completions.create( [](https://microsoft.github.io/agent-lightning/0.2.0/how-to/train-first-agent/#__codelineno-0-45) model="gpt-5-mini", [](https://microsoft.github.io/agent-lightning/0.2.0/how-to/train-first-agent/#__codelineno-0-46) messages=messages, [](https://microsoft.github.io/agent-lightning/0.2.0/how-to/train-first-agent/#__codelineno-0-47) ) [](https://microsoft.github.io/agent-lightning/0.2.0/how-to/train-first-agent/#__codelineno-0-48) final_choice = second_response.choices[0].message.content [](https://microsoft.github.io/agent-lightning/0.2.0/how-to/train-first-agent/#__codelineno-0-49) else: [](https://microsoft.github.io/agent-lightning/0.2.0/how-to/train-first-agent/#__codelineno-0-50) final_choice = response_message.content [](https://microsoft.github.io/agent-lightning/0.2.0/how-to/train-first-agent/#__codelineno-0-51) [](https://microsoft.github.io/agent-lightning/0.2.0/how-to/train-first-agent/#__codelineno-0-52) # 5. Grade the final choice to get a reward. [](https://microsoft.github.io/agent-lightning/0.2.0/how-to/train-first-agent/#__codelineno-0-53) reward = grade_the_choice(final_choice, task["expected_choice"]) [](https://microsoft.github.io/agent-lightning/0.2.0/how-to/train-first-agent/#__codelineno-0-54) return reward` In Agent-lightning, you wrap this logic in a Python function marked with the [`@rollout`](https://microsoft.github.io/agent-lightning/0.2.0/reference/agent/#agentlightning.rollout " agentlightning.rollout(func)") decorator, so that the agent can be managed and tuned by Agent-lightning's runner and trainer. The `prompt_template` that the APO algorithm tunes is passed in as an argument: `[](https://microsoft.github.io/agent-lightning/0.2.0/how-to/train-first-agent/#__codelineno-1-1) import agentlightning as agl [](https://microsoft.github.io/agent-lightning/0.2.0/how-to/train-first-agent/#__codelineno-1-2) [](https://microsoft.github.io/agent-lightning/0.2.0/how-to/train-first-agent/#__codelineno-1-3) @agl.rollout [](https://microsoft.github.io/agent-lightning/0.2.0/how-to/train-first-agent/#__codelineno-1-4) def room_selector(task: RoomSelectionTask, prompt_template: agl.PromptTemplate) -> float: [](https://microsoft.github.io/agent-lightning/0.2.0/how-to/train-first-agent/#__codelineno-1-5) # ... agent logic using the prompt_template ... [](https://microsoft.github.io/agent-lightning/0.2.0/how-to/train-first-agent/#__codelineno-1-6) [](https://microsoft.github.io/agent-lightning/0.2.0/how-to/train-first-agent/#__codelineno-1-7) # The final reward is determined by a grader function [](https://microsoft.github.io/agent-lightning/0.2.0/how-to/train-first-agent/#__codelineno-1-8) reward = room_selection_grader(client, final_message, task["expected_choice"]) [](https://microsoft.github.io/agent-lightning/0.2.0/how-to/train-first-agent/#__codelineno-1-9) return reward` Core Concepts: Tasks, Rollouts, Spans, and Prompt Templates[¶](https://microsoft.github.io/agent-lightning/0.2.0/how-to/train-first-agent/#core-concepts-tasks-rollouts-spans-and-prompt-templates "Permanent link") --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- To understand how Agent-lightning works, you need to know these key terms. ### Task[¶](https://microsoft.github.io/agent-lightning/0.2.0/how-to/train-first-agent/#task "Permanent link") A task is a specific input or problem statement given to the agent. It defines what the agent needs to accomplish. Analogy: Task If the agent is a chef, a task is the recipe request: "Bake a chocolate cake." ### Rollout[¶](https://microsoft.github.io/agent-lightning/0.2.0/how-to/train-first-agent/#rollout "Permanent link") A rollout is a single, complete execution of an agent attempting to solve a given **task**. It's the entire story from receiving the task to producing a final result and receiving a reward. A rollout captures a full trace of the agent's execution. Analogy: Rollout A rollout is one full attempt by the chef to bake the chocolate cake, from gathering ingredients to the final taste test. ### Span[¶](https://microsoft.github.io/agent-lightning/0.2.0/how-to/train-first-agent/#span "Permanent link") A span represents a single unit of work or an operation within a **rollout**. Spans are the building blocks of a trace. They have a start and end time and contain details about the specific operation, like an LLM call, a tool execution, or a reward calculation. For a more precise definition, see the [OpenTelemetry documentation](https://opentelemetry.io/docs/concepts/signals/traces/) . Analogy: Span If the rollout is "baking a cake," a span could be "preheating the oven," "mixing flour and sugar," or "adding frosting." Each is a distinct step or unit of work. The picture below from [ADK](https://google.github.io/adk-docs/observability/cloud-trace/) shows a typical rollout, where each rectangle in the waterfall visualizes a span. As can be seen in the visualization, spans can be sequential, parallel or nested among each other. In other frameworks, the terminilogy might be slightly different. Agent-lightning follows the terminalogies used by OpenTelemetry to avoid confusion. ![AgentOps Waterfall Visualization](https://microsoft.github.io/agent-lightning/0.2.0/assets/agentops-waterfall-visualization.jpg) ### Prompt Template[¶](https://microsoft.github.io/agent-lightning/0.2.0/how-to/train-first-agent/#prompt-template "Permanent link") A prompt template is a reusable instruction for the agent, often containing placeholders that can be filled in with specific details from a task. It is a key **"resource"** that the algorithm learns and improves over time. Analogy: Resource (Prompt Template) If the task is the recipe request, the prompt template is the master recipe card that the chef follows. The algorithm's job is to edit this recipe card to make the instructions clearer and the final dish better. The Training Loop: How the Magic Happens[¶](https://microsoft.github.io/agent-lightning/0.2.0/how-to/train-first-agent/#the-training-loop-how-the-magic-happens "Permanent link") ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Training in Agent-lightning revolves around a clear, managed loop, orchestrated by the **Trainer**. The diagram below illustrates this core interaction: ![Loop of Tasks and Spans](https://microsoft.github.io/agent-lightning/0.2.0/assets/tasks-spans-loop.svg) **The Loop Explained:** * **Algorithm to Agent (via Trainer):** The **Algorithm** (the "brain") creates an improved **Prompt Template** and selects **Tasks**. The Trainer then sends both to the Trainer. * **Agent to Algorithm (via Trainer):** For each task it receives, the Agent uses the provided prompt template to perform a Rollout, executing its logic and potentially using tools. During this rollout, the runner that runs the agent captures Spans that detail every step. The agent also calculates a Reward for its performance on the task. These spans and rewards are then sent back to the Algorithm via the Trainer. * **Algorithm Learning:** The Algorithm then analyzes these spans and rewards to learn how to improve the agent's behavior, for example, by generating a better prompt. This improved prompt is then used in the next iteration of tasks. This cycle continues, allowing the agent to continuously learn and get better at solving tasks. Note In the next tutorial, we will see that the "via Trainer" here is not accurate. It's actually via the runner and store. ### The Algorithm[¶](https://microsoft.github.io/agent-lightning/0.2.0/how-to/train-first-agent/#the-algorithm "Permanent link") The algorithm is the smart part of the system that drives the improvement. In this tutorial, we use [**APO**](https://microsoft.github.io/agent-lightning/0.2.0/algorithm-zoo/apo/#agentlightning.algorithm.apo.APO " APO") (Automatic Prompt Optimization). It works in a few steps: 1. **Evaluate:** The algorithm first asks for rollouts to be run using the current prompt template to see how well it performs. 2. **Critique:** It then looks at the detailed spans from those rollouts. Using a powerful LLM (`gpt-5-mini`), it generates a "textual gradient", which is a natural language critique of the prompt. For example: "The prompt is ambiguous about how to handle tie-breakers for equally good rooms." 3. **Rewrite:** Finally, it gives the critique and the original prompt to another LLM (`gpt-4.1-mini`) and asks it to apply the edits, generating a new, improved prompt template. This cycle repeats, with each round producing a slightly better prompt. To use it, you simply initialize the APO class with your desired hyperparameters. `[](https://microsoft.github.io/agent-lightning/0.2.0/how-to/train-first-agent/#__codelineno-2-1) # In the main training script: run_apo.py [](https://microsoft.github.io/agent-lightning/0.2.0/how-to/train-first-agent/#__codelineno-2-2) from openai import AsyncOpenAI [](https://microsoft.github.io/agent-lightning/0.2.0/how-to/train-first-agent/#__codelineno-2-3) [](https://microsoft.github.io/agent-lightning/0.2.0/how-to/train-first-agent/#__codelineno-2-4) openai = AsyncOpenAI() [](https://microsoft.github.io/agent-lightning/0.2.0/how-to/train-first-agent/#__codelineno-2-5) algo = agl.APO(openai)` Tip Make sure you have `OPENAI_API_KEY` set in your environment variables. ### The Trainer[¶](https://microsoft.github.io/agent-lightning/0.2.0/how-to/train-first-agent/#the-trainer "Permanent link") The Trainer is the central component you'll interact with. It connects everything and manages the entire workflow by running the loop described above. You configure the Trainer, providing the algorithm, the number of parallel runners, and the initial prompt. A single call to [`trainer.fit()`](https://microsoft.github.io/agent-lightning/0.2.0/reference/trainer/#agentlightning.Trainer.fit " fit(agent, train_dataset=None, *, val_dataset=None)") kicks off the entire process! ``[](https://microsoft.github.io/agent-lightning/0.2.0/how-to/train-first-agent/#__codelineno-3-1) # 1. Configure the Trainer with the algorithm and initial prompt [](https://microsoft.github.io/agent-lightning/0.2.0/how-to/train-first-agent/#__codelineno-3-2) trainer = agl.Trainer( [](https://microsoft.github.io/agent-lightning/0.2.0/how-to/train-first-agent/#__codelineno-3-3) algorithm=algo, [](https://microsoft.github.io/agent-lightning/0.2.0/how-to/train-first-agent/#__codelineno-3-4) n_runners=8, # Run 8 agents in parallel to try out the prompts [](https://microsoft.github.io/agent-lightning/0.2.0/how-to/train-first-agent/#__codelineno-3-5) initial_resources={ [](https://microsoft.github.io/agent-lightning/0.2.0/how-to/train-first-agent/#__codelineno-3-6) # The initial prompt template to be tuned [](https://microsoft.github.io/agent-lightning/0.2.0/how-to/train-first-agent/#__codelineno-3-7) "prompt_template": prompt_template_baseline() [](https://microsoft.github.io/agent-lightning/0.2.0/how-to/train-first-agent/#__codelineno-3-8) }, [](https://microsoft.github.io/agent-lightning/0.2.0/how-to/train-first-agent/#__codelineno-3-9) # This is used to convert the span data into a message format consumable by APO algorithm [](https://microsoft.github.io/agent-lightning/0.2.0/how-to/train-first-agent/#__codelineno-3-10) adapter=agl.TraceToMessages(), [](https://microsoft.github.io/agent-lightning/0.2.0/how-to/train-first-agent/#__codelineno-3-11) ) [](https://microsoft.github.io/agent-lightning/0.2.0/how-to/train-first-agent/#__codelineno-3-12) [](https://microsoft.github.io/agent-lightning/0.2.0/how-to/train-first-agent/#__codelineno-3-13) # 2. Load datasets: They can be list of task objects consumable by `room_selector`. [](https://microsoft.github.io/agent-lightning/0.2.0/how-to/train-first-agent/#__codelineno-3-14) dataset_train, dataset_val = ... [](https://microsoft.github.io/agent-lightning/0.2.0/how-to/train-first-agent/#__codelineno-3-15) [](https://microsoft.github.io/agent-lightning/0.2.0/how-to/train-first-agent/#__codelineno-3-16) # 3. Start the training process! [](https://microsoft.github.io/agent-lightning/0.2.0/how-to/train-first-agent/#__codelineno-3-17) trainer.fit( [](https://microsoft.github.io/agent-lightning/0.2.0/how-to/train-first-agent/#__codelineno-3-18) agent=room_selector, [](https://microsoft.github.io/agent-lightning/0.2.0/how-to/train-first-agent/#__codelineno-3-19) train_dataset=dataset_train, [](https://microsoft.github.io/agent-lightning/0.2.0/how-to/train-first-agent/#__codelineno-3-20) val_dataset=dataset_val [](https://microsoft.github.io/agent-lightning/0.2.0/how-to/train-first-agent/#__codelineno-3-21) )`` Tip [`TraceToMessages`](https://microsoft.github.io/agent-lightning/0.2.0/reference/algorithm/#agentlightning.TraceToMessages " agentlightning.TraceToMessages") is a convenience adapter that converts spans into OpenAI chat messages. It requires `openai >= 1.100.0` to be installed. Training Results[¶](https://microsoft.github.io/agent-lightning/0.2.0/how-to/train-first-agent/#training-results "Permanent link") ----------------------------------------------------------------------------------------------------------------------------------- The APO algorithm successfully improved the agent's performance. We ran the example with the following hyper-parameters: * `val_batch_size` = 10 * `gradient_batch_size` = 4 * `beam_width` = 2 * `branch_factor` = 2 * `beam_rounds` = 2 The validation accuracy on the 29 samples of datasets steadily increase from 0.569 (baseline) to **0.721** (after round 2). The tuning takes around 10 minutes with 8 runners. We ran twice, and the results are shown in the chart below. This demonstrates how Agent-lightning can efficiently and automatically enhance your agent's capabilities with just a few lines of code. Back to top --- # Train SQL Agent - Agent Lightning [Skip to content](https://microsoft.github.io/agent-lightning/0.1.2/how-to/train-sql-agent/#sql-agent-with-agent-lightning) SQL Agent with Agent Lightning[¶](https://microsoft.github.io/agent-lightning/0.1.2/how-to/train-sql-agent/#sql-agent-with-agent-lightning "Permanent link") ============================================================================================================================================================= > This tutorial is tested with `verl==0.5.0` and `vllm==0.10.0`. This example demonstrates how to build and train a self-correcting SQL agent. It leverages [Agent Lightning](https://github.com/microsoft/agent-lightning) and the `verl` framework for Reinforcement Learning (RL) based training, and LangGraph to define the agent's complex, cyclical reasoning workflow. The goal is to fine-tune a Large Language Model (LLM) to accurately convert natural language questions into executable SQL queries. SQL Agent Implementation[¶](https://microsoft.github.io/agent-lightning/0.1.2/how-to/train-sql-agent/#sql-agent-implementation "Permanent link") ------------------------------------------------------------------------------------------------------------------------------------------------- The design of Agent-lightning **allows flexible integration with various agent frameworks**, including AutoGen, CrewAI, OpenAI Agent SDK, LangGraph, and more. It can also work without agent frameworks, allowing you to train an agent built from scratch with Python code. See [our example gallery](https://github.com/microsoft/agent-lightning/tree/3ec5ebdcc79524f78902ce98178c35830cfe1811/examples) for more details. The core of the agent is a state machine built with LangGraph, which allows for a robust and transparent workflow. The agent's logic, as visualized below, starts by writing a query, executes it, and then enters a refinement loop where it checks and rewrites the query until it is deemed correct or a turn limit is reached. --- config: flowchart: curve: linear --- graph LR; __start__([

__start__

]):::first write_query(write_query) execute_query(execute_query) check_query(check_query) rewrite_query(rewrite_query) __end__([

__end__

]):::last __start__ --> write_query; check_query -.-> __end__; check_query -.-> rewrite_query; execute_query --> check_query; rewrite_query --> execute_query; write_query --> execute_query; classDef default fill:#f2f2f2,line-height:1.2 classDef first fill-opacity:0 classDef last fill:#cccccc This workflow is implemented in the `SQLAgent` class within `sql_agent.py`. It consists of the following key steps: 1. **write\_query**: Given a user's question and database schema, the agent makes an initial attempt to write a SQL query. 2. **execute\_query**: The generated query is run against the target database. 3. **check\_query**: The agent analyzes the original query and its execution result (or error) to check for mistakes. It uses a specific prompt (`CHECK_QUERY_PROMPT`) to determine if the query is correct. 4. **rewrite\_query**: If the `check_query` step finds errors, the agent enters this step. It uses the feedback from the previous step to generate a corrected SQL query. The process then loops back to `check_query` for re-evaluation. 5. **END**: The loop terminates when `check_query` confirms the query is correct or the maximum number of turns (`max_turns`) is exceeded. One turn corresponds to a complete cycle of `write_query` (if first round), `execute_query`, `check_query`, and potentially `rewrite_query`. We aim to train **write\_query** and **rewrite\_query** step in the setup of this example. The **check\_query** step is not trained but will share the same LLM weights as the other steps. Client-Server Training with Agent Lightning[¶](https://microsoft.github.io/agent-lightning/0.1.2/how-to/train-sql-agent/#client-server-training-with-agent-lightning "Permanent link") --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- The training process uses a distributed client-server architecture designed by Agent Lightning to efficiently fine-tune the underlying LLM. This separation allows for scalable data generation across multiple clients while centralizing the computationally intensive model training on a dedicated server with GPUs, and also provides opportunities for customizing algorithms and training strategies (like [prompt optimization](https://github.com/microsoft/agent-lightning/tree/3ec5ebdcc79524f78902ce98178c35830cfe1811/examples/apo) ) with minimal code changes. * **Training Server (`agentlightning.verl`)**: The server, launched with the first command below, manages the core training loop. It runs an RL algorithm (with `verl` of course) and hosts an OpenAI-compatible LLM endpoint (with `verl`'s async server). The server's sole purpose is to receive interaction data from clients and update the LLM's weights to improve its performance. [This link](https://github.com/microsoft/agent-lightning/tree/3ec5ebdcc79524f78902ce98178c35830cfe1811/agentlightning/verl) points to the implementation of the server, which is built upon `verl`. * **Agent Clients (`sql_agent.py`)**: The clients run the LangGraph agent logic described above. They connect to the server to fetch tasks (natural language questions) and use the server's **OpenAI-compatible endpoint** for all generation steps (`write_query`, `check_query`, `rewrite_query`). After completing a task, the client exports its interaction traces (traced by [AgentOps](https://www.agentops.ai/) and filtered by trace hierarchy), evaluates its correctness to calculate a reward, and sends the entire interaction history (the "trajectory") back to the server for training. To adapt any agent to an "agent client", you do not need to change the agent logic, but only need to invoke the client's `run` method with `agentlightning.trainer`. ![Difference between the original agent and modified agent client](https://microsoft.github.io/agent-lightning/0.1.2/assets/sql-agent-diff.png) Running the Example[¶](https://microsoft.github.io/agent-lightning/0.1.2/how-to/train-sql-agent/#running-the-example "Permanent link") --------------------------------------------------------------------------------------------------------------------------------------- 1. Prepare the dataset: download from [here](https://drive.google.com/file/d/1oi9J1jZP9TyM35L85CL3qeGWl2jqlnL6/view) and unzip it to the `data` folder. It's basically a [Spider V1](https://yale-lily.github.io/spider) dataset converted to Parquet format. The dataset contains about 8000 training samples and about 2000 test samples, from which we sampled 500 samples for evaluation. `[](https://microsoft.github.io/agent-lightning/0.1.2/how-to/train-sql-agent/#__codelineno-0-1) pip install gdown [](https://microsoft.github.io/agent-lightning/0.1.2/how-to/train-sql-agent/#__codelineno-0-2) gdown --fuzzy https://drive.google.com/file/d/1oi9J1jZP9TyM35L85CL3qeGWl2jqlnL6/view [](https://microsoft.github.io/agent-lightning/0.1.2/how-to/train-sql-agent/#__codelineno-0-3) unzip -q spider-data.zip -d data [](https://microsoft.github.io/agent-lightning/0.1.2/how-to/train-sql-agent/#__codelineno-0-4) rm spider-data.zip` 2. Install the required dependencies: `[](https://microsoft.github.io/agent-lightning/0.1.2/how-to/train-sql-agent/#__codelineno-1-1) pip install -r requirements.txt` 3. Launch the training server: `bash python -m agentlightning.verl \ agentlightning.port=9997 \ algorithm.adv_estimator=grpo \ data.train_files=data/train_spider.parquet \ data.val_files=data/test_dev_500.parquet \ actor_rollout_ref.rollout.tensor_model_parallel_size=1 \ trainer.n_gpus_per_node=1 \ data.train_batch_size=32 \ actor_rollout_ref.rollout.n=4 \ actor_rollout_ref.actor.ppo_mini_batch_size=32 \ actor_rollout_ref.actor.ppo_micro_batch_size_per_gpu=4 \ actor_rollout_ref.rollout.log_prob_micro_batch_size_per_gpu=4 \ actor_rollout_ref.rollout.multi_turn.format=hermes \ actor_rollout_ref.model.path=meta-llama/Llama-3.2-3B-Instruct \ data.max_prompt_length=4096 \ data.max_response_length=2048 \ data.truncation='error' \ trainer.val_before_train=True \ actor_rollout_ref.actor.optim.lr=1e-6 \ actor_rollout_ref.model.use_remove_padding=True \ actor_rollout_ref.actor.use_kl_loss=False \ actor_rollout_ref.actor.kl_loss_coef=0.000 \ actor_rollout_ref.actor.entropy_coeff=0 \ actor_rollout_ref.actor.clip_ratio_low=0.2 \ actor_rollout_ref.actor.clip_ratio_high=0.3 \ actor_rollout_ref.model.enable_gradient_checkpointing=True \ actor_rollout_ref.actor.fsdp_config.param_offload=True \ actor_rollout_ref.actor.fsdp_config.optimizer_offload=True \ actor_rollout_ref.rollout.name=vllm \ actor_rollout_ref.rollout.gpu_memory_utilization=0.8 \ actor_rollout_ref.ref.log_prob_micro_batch_size_per_gpu=8 \ actor_rollout_ref.ref.fsdp_config.param_offload=True \ algorithm.use_kl_in_reward=False \ trainer.critic_warmup=0 \ trainer.logger=['console','wandb'] \ trainer.project_name=AgentLightning \ trainer.experiment_name=train_sql_agent \ trainer.nnodes=1 \ trainer.save_freq=256 \ trainer.test_freq=32 \ trainer.total_epochs=2` 4. Launch agent clients that connect with the server: `[](https://microsoft.github.io/agent-lightning/0.1.2/how-to/train-sql-agent/#__codelineno-2-1) export VERL_API_BASE=http://localhost:9997/ # Same as the server port. This is used for receiving tasks and sending results. [](https://microsoft.github.io/agent-lightning/0.1.2/how-to/train-sql-agent/#__codelineno-2-2) python sql_agent.py \ [](https://microsoft.github.io/agent-lightning/0.1.2/how-to/train-sql-agent/#__codelineno-2-3) --litsqlagent.trained-agents write \ # Will only train the write and rewrite agent. [](https://microsoft.github.io/agent-lightning/0.1.2/how-to/train-sql-agent/#__codelineno-2-4) --trainer.n-workers 16 \ [](https://microsoft.github.io/agent-lightning/0.1.2/how-to/train-sql-agent/#__codelineno-2-5) --litsqlagent.val-temperature 0` There is no hard requirement in the launching order of the server and clients. But remember to kill the long-running agent clients after the training is done. Debug the Agent without verl[¶](https://microsoft.github.io/agent-lightning/0.1.2/how-to/train-sql-agent/#debug-the-agent-without-verl "Permanent link") --------------------------------------------------------------------------------------------------------------------------------------------------------- You can run the agent client alone without the `verl` server. This is useful for debugging the agent logic and SQL execution. 1. Copy `.env.example` to `.env` and fill in your OpenAI API key. `VERL_API_BASE` does not really matter here because you are not connecting to the server end. 2. Run the agent client: `[](https://microsoft.github.io/agent-lightning/0.1.2/how-to/train-sql-agent/#__codelineno-3-1) dotenv run python sql_agent.py \ [](https://microsoft.github.io/agent-lightning/0.1.2/how-to/train-sql-agent/#__codelineno-3-2) --litsqlagent.trained-agents write \ # Will only select the trajectories related to write and rewrite. [](https://microsoft.github.io/agent-lightning/0.1.2/how-to/train-sql-agent/#__codelineno-3-3) --trainer.n-workers 1 \ # For debug, use single process. [](https://microsoft.github.io/agent-lightning/0.1.2/how-to/train-sql-agent/#__codelineno-3-4) --trainer.dev true # Enable the dev debug mode.` Evaluation[¶](https://microsoft.github.io/agent-lightning/0.1.2/how-to/train-sql-agent/#evaluation "Permanent link") --------------------------------------------------------------------------------------------------------------------- The example is evaluated using Llama-3.2-Instruct models. The models are trained on the Spider dataset for 2 epochs, with evaluation performed on a randomly selected subset of 500 test samples to compute held-out accuracy. The default setup for running agent clients during evaluation is as follows: `[](https://microsoft.github.io/agent-lightning/0.1.2/how-to/train-sql-agent/#__codelineno-4-1) python sql_agent.py \ [](https://microsoft.github.io/agent-lightning/0.1.2/how-to/train-sql-agent/#__codelineno-4-2) --litsqlagent.trained-agents write \ [](https://microsoft.github.io/agent-lightning/0.1.2/how-to/train-sql-agent/#__codelineno-4-3) --trainer.n-workers 16 \ [](https://microsoft.github.io/agent-lightning/0.1.2/how-to/train-sql-agent/#__codelineno-4-4) --trainer.daemon true \ [](https://microsoft.github.io/agent-lightning/0.1.2/how-to/train-sql-agent/#__codelineno-4-5) --litsqlagent.val-temperature 0 \ [](https://microsoft.github.io/agent-lightning/0.1.2/how-to/train-sql-agent/#__codelineno-4-6) --litsqlagent.max-turns 3 \ [](https://microsoft.github.io/agent-lightning/0.1.2/how-to/train-sql-agent/#__codelineno-4-7) --litsqlagent.table-info-truncate 2048 \ [](https://microsoft.github.io/agent-lightning/0.1.2/how-to/train-sql-agent/#__codelineno-4-8) --litsqlagent.execution-truncate 2048` The setup of training server is the same as the command above. ### W&B Report[¶](https://microsoft.github.io/agent-lightning/0.1.2/how-to/train-sql-agent/#wb-report "Permanent link") [link](https://api.wandb.ai/links/ultmaster/4cid500g) ### Performance Metrics[¶](https://microsoft.github.io/agent-lightning/0.1.2/how-to/train-sql-agent/#performance-metrics "Permanent link") ![](https://microsoft.github.io/agent-lightning/0.1.2/assets/sql-agent-val-reward-curve.png) | Model | Size | Context | Max Turns | Agents | Acc (Initial) | Acc (Final) | Transitions | Prompt Length | Response Length | | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | | Llama3.2 | 1B | 2048 | 3 | write\|rewrite | 21 | 49.6 | 2.87 → 3.08 | 821.2 | 319.2 → 249.4 | | Llama3.2 | 3B | 2048 | 3 | write\|rewrite | 51.8 | 66.4 | 2.20 → 2.72 | 865.6 | 116.2 → 314.3 | **Notes:** 1. **Context Length**: Controlled via `--litsqlagent.table-info-truncate ` and `--litsqlagent.execution-truncate ` 2. **Max Turns**: Set using `--litsqlagent.max-turns ` 3. **Agents**: Specified with `--litsqlagent.agents ` (defaults to `write`, which matches both write and rewrite agents) 4. **Transitions**: Represents the number of prompt-response pairs traced (collected) during each rollout. Note that this differs from the turn count in the SQL agent workflow, where one turn may encompass 2-3 transitions in the check-rewrite cycle. The number of transitions is also related to which _agents_ get involved in the training. 5. **Prompt/Response Length**: Average token count per **traced** prompt/transition response. ### Efficiency Metrics[¶](https://microsoft.github.io/agent-lightning/0.1.2/how-to/train-sql-agent/#efficiency-metrics "Permanent link") | Model | Size | Context | Max Turns | Agents | \# GPUs | \# Steps | Time (h) | Time/Step (s) | Rollout Time (%) | Update Actor Time (%) | | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | | Llama3.2 | 1B | 2048 | 3 | write\|rewrite | 1 | 436 | 13.06 | 98.9 | 66.7 | 25.2 | | Llama3.2 | 3B | 2048 | 3 | write\|rewrite | 2 | 436 | 10.3 | 181.3 | 63.9 | 27.9 | Back to top --- # SFT with Unsloth - Agent-lightning [Skip to content](https://microsoft.github.io/agent-lightning/0.3.0/how-to/unsloth-sft/#fine-tune-with-unsloth-sft) ⚠️ You are viewing development documentation. For stable documentation, please visit [https://microsoft.github.io/agent-lightning/stable/](https://microsoft.github.io/agent-lightning/stable/) Fine-tune with Unsloth SFT[¶](https://microsoft.github.io/agent-lightning/0.3.0/how-to/unsloth-sft/#fine-tune-with-unsloth-sft "Permanent link") ================================================================================================================================================= Prerequisites Please make sure you have read [Write the First Algorithm](https://microsoft.github.io/agent-lightning/0.3.0/how-to/write-first-algorithm/) . Although that recipe is based on a simple prompt tuning algorithm, it introduces the core concepts of Agent-lightning and you should be familiar with them before proceeding. This recipe builds on [Write the First Algorithm](https://microsoft.github.io/agent-lightning/0.3.0/how-to/write-first-algorithm/) . Instead of iterating on a prompt, we will fine-tune a large language model with [Unsloth](https://docs.unsloth.ai/) 's SFT Trainer and keep the whole loop inside Agent-lightning. The new pieces you will meet are the **LLM proxy**, the **trace-to-triplet adapter**, a [vLLM](https://github.com/vllm-project/vllm) inference endpoint, and an agent implemented with the [OpenAI Agents SDK](https://openai.github.io/openai-agents-python/) . The full sample code is available in the [`examples/unsloth`](https://github.com/microsoft/agent-lightning/tree/3b5d733861cf313fc09821a23240bbdf3cb2ee5b/examples/unsloth) folder. Warning You need a GPU that can host the Unsloth base model and run vLLM. The sample defaults to `unsloth/Qwen3-4B-Instruct-2507`, which requires at least 16GB of GPU memory under 4-bit quantization. The Data and Serving Loop[¶](https://microsoft.github.io/agent-lightning/0.3.0/how-to/unsloth-sft/#the-data-and-serving-loop "Permanent link") ----------------------------------------------------------------------------------------------------------------------------------------------- To tune a large language model in Supervised Fine-Tuning (SFT), we commonly need a dataset with input/output samples. For example, the [TRL SFT Trainer](https://huggingface.co/docs/trl/sft_trainer) expects a dataset with samples like the following: `[](https://microsoft.github.io/agent-lightning/0.3.0/how-to/unsloth-sft/#__codelineno-0-1) {"messages": [{"role": "user", "content": "What color is the sky?"}, [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/unsloth-sft/#__codelineno-0-2) {"role": "assistant", "content": "It is blue."}]}` With supervised fine-tuning, the LLM learns to generate the "assistant" response as close as possible to the completion in the dataset. Typically, the dataset used in SFT should be a curated set of samples. The samples can be either hand-written by humans, or generated by a more powerful model, which is known as [data distillation](https://docs.nvidia.com/nemo-framework/user-guide/24.12/modelalignment/knowledge-distillation.html) . However, in this recipe, we use a different setup that relies on samples generated by the model itself. We use the reward emitted by the agent to select the top-performing samples. Overall, the flow of the algorithm is an iteration of the following steps: 1. Serve the current checkpoint (with vLLM). 2. Publish the vLLM endpoint through the LLM proxy and let runners roll out some tasks with the current model. 3. Collect the traces from the rollouts and transform the highest-rewarded ones into a dataset that is acceptable for Unsloth SFT Trainer. 4. Launch Unsloth to fine-tune on the dataset and save a new checkpoint. You will find the full source code of this iteration in `sft_one_iter` in [sft\_algorithm.py](https://github.com/microsoft/agent-lightning/tree/3b5d733861cf313fc09821a23240bbdf3cb2ee5b/examples/unsloth/sft_algorithm.py) . We will elaborate on each part below. ### Serving the Model with vLLM and Proxy[¶](https://microsoft.github.io/agent-lightning/0.3.0/how-to/unsloth-sft/#serving-the-model-with-vllm-and-proxy "Permanent link") Most modern agents do not use the model directly; instead, they use an API like the OpenAI chat completions API to interact with the model. Therefore, we need a vLLM-based inference server launched before rollouts. The serving code looks like the following. See the `vllm_server` function in [sft\_algorithm.py](https://github.com/microsoft/agent-lightning/tree/3b5d733861cf313fc09821a23240bbdf3cb2ee5b/examples/unsloth/sft_algorithm.py) if you want to see a more robust version. `[](https://microsoft.github.io/agent-lightning/0.3.0/how-to/unsloth-sft/#__codelineno-1-1) from openai import OpenAI [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/unsloth-sft/#__codelineno-1-2) [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/unsloth-sft/#__codelineno-1-3) vllm_process = subprocess.Popen([ [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/unsloth-sft/#__codelineno-1-4) "vllm", "serve", model_path, "--port", str(port), [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/unsloth-sft/#__codelineno-1-5) "--enable-auto-tool-choice", "--tool-call-parser", "hermes" [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/unsloth-sft/#__codelineno-1-6) ]) [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/unsloth-sft/#__codelineno-1-7) [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/unsloth-sft/#__codelineno-1-8) # Wait for the server to be ready [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/unsloth-sft/#__codelineno-1-9) url = f"http://localhost:{port}/health" [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/unsloth-sft/#__codelineno-1-10) start = time.time() [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/unsloth-sft/#__codelineno-1-11) client = httpx.Client() [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/unsloth-sft/#__codelineno-1-12) [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/unsloth-sft/#__codelineno-1-13) while True: [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/unsloth-sft/#__codelineno-1-14) if client.get(url).status_code == 200: [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/unsloth-sft/#__codelineno-1-15) break [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/unsloth-sft/#__codelineno-1-16) [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/unsloth-sft/#__codelineno-1-17) server_address = f"http://localhost:{port}/v1" [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/unsloth-sft/#__codelineno-1-18) [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/unsloth-sft/#__codelineno-1-19) # Try using the vLLM server [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/unsloth-sft/#__codelineno-1-20) openai = OpenAI(base_url=server_address) [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/unsloth-sft/#__codelineno-1-21) ...` In this recipe, we do not expose the server address directly to the agent runners, because we want to install a "middleware" to collect the prompts and responses of all the requests. In general, it's up to you to decide whether to hide the vLLM server behind a proxy or not. The "middleware" here is [`LLMProxy`](https://microsoft.github.io/agent-lightning/0.3.0/reference/algorithm/#agentlightning.LLMProxy " agentlightning.LLMProxy") , which is an independent [LiteLLM](https://docs.litellm.ai/) server that forwards the requests to the vLLM server. It also exposes an OpenAI-compatible API that the runners can target without caring about where the model lives. The benefits of using the proxy are: 1. **Traces:** The proxy automatically logs the prompts and responses of all the requests into the store. 2. **Token IDs:** The proxy augments the requests so that the vLLM server can return the prompt and response token IDs (see more details in [Serving LLM](https://microsoft.github.io/agent-lightning/0.3.0/deep-dive/serving-llm/) ). The [`LLMProxy`](https://microsoft.github.io/agent-lightning/0.3.0/reference/algorithm/#agentlightning.LLMProxy " agentlightning.LLMProxy") accepts a list of model configurations, in the same syntax as LiteLLM's [`model_list`](https://docs.litellm.ai/docs/proxy/configs) . Include a `hosted_vllm/` prefix to the models to activate LiteLLM's [vLLM integration](https://docs.litellm.ai/docs/providers/vllm) . `[](https://microsoft.github.io/agent-lightning/0.3.0/how-to/unsloth-sft/#__codelineno-2-1) import agentlightning as agl [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/unsloth-sft/#__codelineno-2-2) [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/unsloth-sft/#__codelineno-2-3) llm_proxy = agl.LLMProxy(port=port, store=store) [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/unsloth-sft/#__codelineno-2-4) model_list = [ [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/unsloth-sft/#__codelineno-2-5) { [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/unsloth-sft/#__codelineno-2-6) "model_name": "Qwen3-4B-Instruct", [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/unsloth-sft/#__codelineno-2-7) "litellm_params": {"model": f"hosted_vllm/{model_path}", "api_base": server_address}, [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/unsloth-sft/#__codelineno-2-8) } [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/unsloth-sft/#__codelineno-2-9) ] [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/unsloth-sft/#__codelineno-2-10) llm_proxy.update_model_list(model_list) [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/unsloth-sft/#__codelineno-2-11) # If the proxy is not running, it will start automatically. [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/unsloth-sft/#__codelineno-2-12) await llm_proxy.restart() [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/unsloth-sft/#__codelineno-2-13) # Add the proxy as a resource to the store so that the runners can access it via URL. [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/unsloth-sft/#__codelineno-2-14) resource_update = await store.add_resources({"main_llm": llm_proxy.as_resource()})` ### Spawn Rollout and Collect Spans[¶](https://microsoft.github.io/agent-lightning/0.3.0/how-to/unsloth-sft/#spawn-rollout-and-collect-spans "Permanent link") Once the proxy is registered as a resource, the algorithm schedules work for the rollout runners. Each problem from a training dataset becomes a rollout with the proxy baked into its resources: `[](https://microsoft.github.io/agent-lightning/0.3.0/how-to/unsloth-sft/#__codelineno-3-1) rollouts: list[Rollout] = [] [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/unsloth-sft/#__codelineno-3-2) for sample in train_dataset: [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/unsloth-sft/#__codelineno-3-3) rollouts.append( [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/unsloth-sft/#__codelineno-3-4) await store.enqueue_rollout( [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/unsloth-sft/#__codelineno-3-5) input=sample, [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/unsloth-sft/#__codelineno-3-6) mode="train", [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/unsloth-sft/#__codelineno-3-7) resources_id=resources_update.resources_id, [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/unsloth-sft/#__codelineno-3-8) ) [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/unsloth-sft/#__codelineno-3-9) )` `resources_id` ties every rollout to the `main_llm` proxy resource we just uploaded. The runners on the other side poll the store ([`LitAgentRunner.iter()`](https://microsoft.github.io/agent-lightning/0.3.0/reference/runner/#agentlightning.LitAgentRunner.iter " iter(*, event=None) async ") ) and execute the agent for each rollout. On the algorithm side we wait for completions with a non-blocking polling loop: `[](https://microsoft.github.io/agent-lightning/0.3.0/how-to/unsloth-sft/#__codelineno-4-1) completed_rollouts: list[Rollout] = [] [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/unsloth-sft/#__codelineno-4-2) while True: [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/unsloth-sft/#__codelineno-4-3) completed_rollouts = await store.wait_for_rollouts( [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/unsloth-sft/#__codelineno-4-4) rollout_ids=[r.rollout_id for r in rollouts], [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/unsloth-sft/#__codelineno-4-5) timeout=0.0, [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/unsloth-sft/#__codelineno-4-6) ) [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/unsloth-sft/#__codelineno-4-7) if len(completed_rollouts) == len(rollouts): [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/unsloth-sft/#__codelineno-4-8) break [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/unsloth-sft/#__codelineno-4-9) await asyncio.sleep(5.0)` Note The `timeout=0.0` is needed here because this example uses a [`LightningStoreClient`](https://microsoft.github.io/agent-lightning/0.3.0/reference/store/#agentlightning.LightningStoreClient " agentlightning.LightningStoreClient") , and `wait_for_rollouts` establishes an HTTP connection to that store. Currently, only non-blocking wait requests are supported, which avoids holding the store connection open. Once the rollouts complete, we terminate the vLLM server to free up GPU memory. `[](https://microsoft.github.io/agent-lightning/0.3.0/how-to/unsloth-sft/#__codelineno-5-1) vllm_process.terminate() [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/unsloth-sft/#__codelineno-5-2) vllm_process.join(timeout=10.0)` ### Adapt the Spans to HuggingFace Dataset[¶](https://microsoft.github.io/agent-lightning/0.3.0/how-to/unsloth-sft/#adapt-the-spans-to-huggingface-dataset "Permanent link") [`LlmProxyTraceToTriplet`](https://microsoft.github.io/agent-lightning/0.3.0/reference/algorithm/#agentlightning.LlmProxyTraceToTriplet " agentlightning.LlmProxyTraceToTriplet") converts the proxy’s spans (which might be dozens to hundreds per rollout) into [`Triplet`](https://microsoft.github.io/agent-lightning/0.3.0/reference/types/#agentlightning.Triplet " agentlightning.Triplet") objects that contain prompt/response token IDs plus an optional reward. The adapter may return multiple triplets per rollout (one per chat-completion call). To bias training toward successful reasoning chains the algorithm walks the triplets in reverse order, keeps the most recent reward, and turns each prompt/response pair into Hugging Face dataset rows: `[](https://microsoft.github.io/agent-lightning/0.3.0/how-to/unsloth-sft/#__codelineno-6-1) all_triplets = [] [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/unsloth-sft/#__codelineno-6-2) data_adapter = agl.LlmProxyTraceToTriplet() [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/unsloth-sft/#__codelineno-6-3) [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/unsloth-sft/#__codelineno-6-4) for rollout in completed_rollouts: [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/unsloth-sft/#__codelineno-6-5) spans = await store.query_spans(rollout.rollout_id, "latest") [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/unsloth-sft/#__codelineno-6-6) triplets = data_adapter.adapt(spans) [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/unsloth-sft/#__codelineno-6-7) [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/unsloth-sft/#__codelineno-6-8) recent_reward = None [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/unsloth-sft/#__codelineno-6-9) for triplet in reversed(triplets): [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/unsloth-sft/#__codelineno-6-10) if triplet.reward is not None: [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/unsloth-sft/#__codelineno-6-11) recent_reward = triplet.reward [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/unsloth-sft/#__codelineno-6-12) if recent_reward is None: [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/unsloth-sft/#__codelineno-6-13) continue [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/unsloth-sft/#__codelineno-6-14) [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/unsloth-sft/#__codelineno-6-15) input_ids = triplet.prompt["token_ids"] + triplet.response["token_ids"] [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/unsloth-sft/#__codelineno-6-16) # We don't train on prompt tokens, so they are masked out by setting to -100. [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/unsloth-sft/#__codelineno-6-17) labels = [-100] * len(triplet.prompt["token_ids"]) + triplet.response["token_ids"] [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/unsloth-sft/#__codelineno-6-18) # This matches the dataset format required by the Unsloth SFT trainer. [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/unsloth-sft/#__codelineno-6-19) all_triplets.append( [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/unsloth-sft/#__codelineno-6-20) { [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/unsloth-sft/#__codelineno-6-21) "input_ids": input_ids, [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/unsloth-sft/#__codelineno-6-22) "attention_mask": [1] * len(input_ids), [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/unsloth-sft/#__codelineno-6-23) "labels": labels, [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/unsloth-sft/#__codelineno-6-24) "reward": recent_reward, [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/unsloth-sft/#__codelineno-6-25) } [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/unsloth-sft/#__codelineno-6-26) )` Note You might notice that the dataset format used here differs from the format described in the **SFT Trainer** documentation. According to the documentation, dataset samples should be provided as plain text strings or message objects. As a matter of fact, this example leverages some [undocumented behavior](https://github.com/huggingface/trl/blob/e0eec055b412c48ad754149c475a87a8fca34fb4/trl/trainer/sft_trainer.py#L887) in the SFT Trainer implementation. When the dataset already includes a `"input_ids"` column, the Trainer automatically marks it as `is_processed` and skips the internal tokenization step. Since we already have spans with token IDs generated by the [`LLMProxy`](https://microsoft.github.io/agent-lightning/0.3.0/reference/algorithm/#agentlightning.LLMProxy " agentlightning.LLMProxy") , providing them directly avoids unnecessary [**re-tokenization** and related complications](https://microsoft.github.io/agent-lightning/0.3.0/deep-dive/serving-llm/) . This approach will both save processing time and increase consistency between training and inference. After aggregating every rollout we shuffle, sort by reward, and keep the top fraction (e.g., 50%) before shuffling again. The resulting list feeds directly into `datasets.Dataset.from_list`, which is the format Unsloth’s SFT trainer expects. `[](https://microsoft.github.io/agent-lightning/0.3.0/how-to/unsloth-sft/#__codelineno-7-1) from datasets import Dataset as HuggingFaceDataset [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/unsloth-sft/#__codelineno-7-2) [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/unsloth-sft/#__codelineno-7-3) random.shuffle(all_triplets) [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/unsloth-sft/#__codelineno-7-4) all_triplets.sort(key=lambda x: x["reward"], reverse=True) [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/unsloth-sft/#__codelineno-7-5) sliced_triplets = all_triplets[: max(1, int(len(all_triplets) * triplet_fraction))] [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/unsloth-sft/#__codelineno-7-6) # Shuffle the sliced triplets again [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/unsloth-sft/#__codelineno-7-7) random.shuffle(sliced_triplets) [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/unsloth-sft/#__codelineno-7-8) [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/unsloth-sft/#__codelineno-7-9) sft_dataset = HuggingFaceDataset.from_list(sliced_triplets)` ### Launch Unsloth Training[¶](https://microsoft.github.io/agent-lightning/0.3.0/how-to/unsloth-sft/#launch-unsloth-training "Permanent link") The heavy lifting happens in [`trl.SFTTrainer`](https://huggingface.co/docs/trl/sft_trainer) (see [unsloth\_helper.py](https://github.com/microsoft/agent-lightning/tree/3b5d733861cf313fc09821a23240bbdf3cb2ee5b/examples/unsloth/unsloth_helper.py) on how it's used). We launch it in a fresh process created with `multiprocessing.get_context("spawn")` so CUDA memory is reliably reclaimed when training ends. Launching it in the same process will also work for the first iteration, but we found that the memory won't be freed properly for subsequent vLLM serving. `[](https://microsoft.github.io/agent-lightning/0.3.0/how-to/unsloth-sft/#__codelineno-8-1) context = multiprocessing.get_context("spawn") [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/unsloth-sft/#__codelineno-8-2) unsloth_process = context.Process( [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/unsloth-sft/#__codelineno-8-3) target=unsloth_training, [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/unsloth-sft/#__codelineno-8-4) args=(model_path, sft_dataset, next_model_path), [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/unsloth-sft/#__codelineno-8-5) daemon=True, [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/unsloth-sft/#__codelineno-8-6) ) [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/unsloth-sft/#__codelineno-8-7) unsloth_process.start() [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/unsloth-sft/#__codelineno-8-8) unsloth_process.join(timeout=600.0)` Inside the `unsloth_training` subprocess, Unsloth loads the previous checkpoint in 4-bit, applies LoRA adapters, and forwards the Hugging Face dataset to [`trl.SFTTrainer`](https://huggingface.co/docs/trl/sft_trainer) with the configuration defined in [`SFTConfig`](https://huggingface.co/docs/trl/sft_trainer#trl.SFTConfig) (batch size, accumulation steps, learning rate, etc.). The merged 16-bit weights are saved under `models/version_` so the next iteration can immediately serve them with vLLM. `[](https://microsoft.github.io/agent-lightning/0.3.0/how-to/unsloth-sft/#__codelineno-9-1) from unsloth import FastLanguageModel [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/unsloth-sft/#__codelineno-9-2) # TRL is patched by unsloth. [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/unsloth-sft/#__codelineno-9-3) from trl import SFTConfig, SFTTrainer [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/unsloth-sft/#__codelineno-9-4) [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/unsloth-sft/#__codelineno-9-5) model, tokenizer = FastLanguageModel.from_pretrained( [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/unsloth-sft/#__codelineno-9-6) model_name=model_path, [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/unsloth-sft/#__codelineno-9-7) load_in_4bit=True, # 4 bit quantization to reduce memory [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/unsloth-sft/#__codelineno-9-8) ) [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/unsloth-sft/#__codelineno-9-9) [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/unsloth-sft/#__codelineno-9-10) # Config the model to use LoRA [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/unsloth-sft/#__codelineno-9-11) model = FastLanguageModel.get_peft_model( [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/unsloth-sft/#__codelineno-9-12) model, [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/unsloth-sft/#__codelineno-9-13) r=32, [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/unsloth-sft/#__codelineno-9-14) ... [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/unsloth-sft/#__codelineno-9-15) ) [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/unsloth-sft/#__codelineno-9-16) [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/unsloth-sft/#__codelineno-9-17) trainer = SFTTrainer( [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/unsloth-sft/#__codelineno-9-18) model=model, [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/unsloth-sft/#__codelineno-9-19) tokenizer=tokenizer, [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/unsloth-sft/#__codelineno-9-20) train_dataset=sft_dataset, [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/unsloth-sft/#__codelineno-9-21) ... [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/unsloth-sft/#__codelineno-9-22) ) [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/unsloth-sft/#__codelineno-9-23) [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/unsloth-sft/#__codelineno-9-24) # This is the heaviest step. [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/unsloth-sft/#__codelineno-9-25) trainer_stats = trainer.train() [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/unsloth-sft/#__codelineno-9-26) [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/unsloth-sft/#__codelineno-9-27) # Save in 16-bit for vLLM inference later [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/unsloth-sft/#__codelineno-9-28) model.save_pretrained_merged(next_model_path, tokenizer, save_method="merged_16bit")` Math Agent: OpenAI Agents SDK with MCP[¶](https://microsoft.github.io/agent-lightning/0.3.0/how-to/unsloth-sft/#math-agent-openai-agents-sdk-with-mcp "Permanent link") ------------------------------------------------------------------------------------------------------------------------------------------------------------------------ We build an agent with the [OpenAI Agents SDK](https://openai.github.io/openai-agents-python/) to wire a calculator MCP tool and an OpenAI-compatible chat completion model together. The agent aims to solve a math problem and returns a reward indicating whether the answer is correct or not. The runner injects the `LLM` resource supplied by the algorithm side: `[](https://microsoft.github.io/agent-lightning/0.3.0/how-to/unsloth-sft/#__codelineno-10-1) import os [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/unsloth-sft/#__codelineno-10-2) from typing import TypedDict [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/unsloth-sft/#__codelineno-10-3) [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/unsloth-sft/#__codelineno-10-4) import agentlightning as agl [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/unsloth-sft/#__codelineno-10-5) from agents import Agent, ModelSettings, OpenAIChatCompletionsModel, Runner as OpenAIRunner [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/unsloth-sft/#__codelineno-10-6) from agents.mcp import MCPServerStdio [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/unsloth-sft/#__codelineno-10-7) from openai import AsyncOpenAI [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/unsloth-sft/#__codelineno-10-8) [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/unsloth-sft/#__codelineno-10-9) class GsmProblem(TypedDict): [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/unsloth-sft/#__codelineno-10-10) input: str [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/unsloth-sft/#__codelineno-10-11) target: float [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/unsloth-sft/#__codelineno-10-12) [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/unsloth-sft/#__codelineno-10-13) def compute_reward(result: str, target: float) -> float: [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/unsloth-sft/#__codelineno-10-14) ... [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/unsloth-sft/#__codelineno-10-15) [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/unsloth-sft/#__codelineno-10-16) @agl.rollout [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/unsloth-sft/#__codelineno-10-17) async def math_agent(task: GsmProblem, llm: agl.LLM) -> float: [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/unsloth-sft/#__codelineno-10-18) async with MCPServerStdio( [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/unsloth-sft/#__codelineno-10-19) name="Calculator via uvx", [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/unsloth-sft/#__codelineno-10-20) params={"command": "uvx", "args": ["mcp-server-calculator"]}, [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/unsloth-sft/#__codelineno-10-21) ) as server: [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/unsloth-sft/#__codelineno-10-22) agent = Agent( [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/unsloth-sft/#__codelineno-10-23) name="Assistant", [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/unsloth-sft/#__codelineno-10-24) instructions=( [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/unsloth-sft/#__codelineno-10-25) "Use the calculator tool for every question. " [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/unsloth-sft/#__codelineno-10-26) "Return only the numeric answer wrapped like ### ###." [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/unsloth-sft/#__codelineno-10-27) ), [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/unsloth-sft/#__codelineno-10-28) mcp_servers=[server], [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/unsloth-sft/#__codelineno-10-29) model=OpenAIChatCompletionsModel( [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/unsloth-sft/#__codelineno-10-30) model=llm.model, [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/unsloth-sft/#__codelineno-10-31) openai_client=AsyncOpenAI( [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/unsloth-sft/#__codelineno-10-32) base_url=llm.endpoint, [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/unsloth-sft/#__codelineno-10-33) api_key=llm.api_key or "dummy", [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/unsloth-sft/#__codelineno-10-34) ), [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/unsloth-sft/#__codelineno-10-35) ), [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/unsloth-sft/#__codelineno-10-36) model_settings=ModelSettings( [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/unsloth-sft/#__codelineno-10-37) temperature=llm.sampling_parameters.get("temperature", 0.0), [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/unsloth-sft/#__codelineno-10-38) ), [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/unsloth-sft/#__codelineno-10-39) ) [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/unsloth-sft/#__codelineno-10-40) result = await OpenAIRunner.run(agent, task["input"]) [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/unsloth-sft/#__codelineno-10-41) return compute_reward(result.final_output, task["target"])` Tip You can test the agent with a dry run: `[](https://microsoft.github.io/agent-lightning/0.3.0/how-to/unsloth-sft/#__codelineno-11-1) import asyncio [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/unsloth-sft/#__codelineno-11-2) [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/unsloth-sft/#__codelineno-11-3) llm = agl.LLM( [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/unsloth-sft/#__codelineno-11-4) endpoint=os.environ["OPENAI_BASE_URL"], [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/unsloth-sft/#__codelineno-11-5) api_key=os.environ["OPENAI_API_KEY"], [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/unsloth-sft/#__codelineno-11-6) model="gpt-4.1-mini", [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/unsloth-sft/#__codelineno-11-7) ) [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/unsloth-sft/#__codelineno-11-8) asyncio.run(math_agent({"input": "What is 1 + 1?", "target": 2.0}, llm))` Run this Recipe[¶](https://microsoft.github.io/agent-lightning/0.3.0/how-to/unsloth-sft/#run-this-recipe "Permanent link") --------------------------------------------------------------------------------------------------------------------------- The full runnable script for this recipe resides in [`examples/unsloth`](https://github.com/microsoft/agent-lightning/tree/3b5d733861cf313fc09821a23240bbdf3cb2ee5b/examples/unsloth) folder. Before running this example, install `unsloth`, `vllm`, and the other libraries used in the examples (the project uses CUDA tooling, TRL, rich, datasets, etc.). We tested with `unsloth==2025.10.1`. `unsloth==2025.10.2` and `2025.10.3` are not working because of an [issue](https://github.com/unslothai/unsloth/issues/3451) we have been investigating with the unsloth team. It's recommended to download the base model before running the example, such that the first iteration and subsequent iterations can both load from local checkpoints. `[](https://microsoft.github.io/agent-lightning/0.3.0/how-to/unsloth-sft/#__codelineno-12-1) hf download unsloth/Qwen3-4B-Instruct-2507 --local-dir models/version_0` The repository already contains `examples/unsloth/data_gsmhard.jsonl` (which is a very small subset of the [GSM-hard math dataset](https://huggingface.co/datasets/reasoning-machines/gsm-hard) for demonstration purposes). ### Run Manually[¶](https://microsoft.github.io/agent-lightning/0.3.0/how-to/unsloth-sft/#run-manually "Permanent link") Similar to the [Write the First Algorithm](https://microsoft.github.io/agent-lightning/0.3.0/how-to/write-first-algorithm/) recipe, you can open three terminals and start each component in parallel. `[](https://microsoft.github.io/agent-lightning/0.3.0/how-to/unsloth-sft/#__codelineno-13-1) agl store --port 4747 [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/unsloth-sft/#__codelineno-13-2) python examples/unsloth/sft_rollout_runners.py [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/unsloth-sft/#__codelineno-13-3) python examples/unsloth/sft_algorithm.py` In this case, [`sft_rollout_runners.py`](https://github.com/microsoft/agent-lightning/tree/3b5d733861cf313fc09821a23240bbdf3cb2ee5b/examples/unsloth/sft_rollout_runners.py) is a simple spawner implemented in Python that spawns 4 runners in parallel. The runners all connect to the same store server executing in another terminal. `[](https://microsoft.github.io/agent-lightning/0.3.0/how-to/unsloth-sft/#__codelineno-14-1) import agentlightning as agl [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/unsloth-sft/#__codelineno-14-2) [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/unsloth-sft/#__codelineno-14-3) def run_rollout(store: agl.LightningStore, worker_id: int) -> None: [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/unsloth-sft/#__codelineno-14-4) # Since the server side has already used LiteLLM proxy to collect traces, [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/unsloth-sft/#__codelineno-14-5) # a simple OtelTracer to collect the rewards is enough. [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/unsloth-sft/#__codelineno-14-6) tracer = agl.OtelTracer() [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/unsloth-sft/#__codelineno-14-7) [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/unsloth-sft/#__codelineno-14-8) runner = agl.LitAgentRunner(tracer=tracer) [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/unsloth-sft/#__codelineno-14-9) [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/unsloth-sft/#__codelineno-14-10) with runner.run_context(agent=math_agent, store=store, worker_id=worker_id): [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/unsloth-sft/#__codelineno-14-11) asyncio.run(runner.iter()) [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/unsloth-sft/#__codelineno-14-12) [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/unsloth-sft/#__codelineno-14-13) [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/unsloth-sft/#__codelineno-14-14) def spawn_runners(store: agl.LightningStore, n_runners: int) -> None: [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/unsloth-sft/#__codelineno-14-15) runners = [ [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/unsloth-sft/#__codelineno-14-16) multiprocessing.Process(target=run_rollout, args=(store, worker_id)) [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/unsloth-sft/#__codelineno-14-17) for worker_id in range(n_runners) [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/unsloth-sft/#__codelineno-14-18) ] [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/unsloth-sft/#__codelineno-14-19) for runner in runners: [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/unsloth-sft/#__codelineno-14-20) runner.start() [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/unsloth-sft/#__codelineno-14-21) [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/unsloth-sft/#__codelineno-14-22) for runner in runners: [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/unsloth-sft/#__codelineno-14-23) runner.join() [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/unsloth-sft/#__codelineno-14-24) [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/unsloth-sft/#__codelineno-14-25) [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/unsloth-sft/#__codelineno-14-26) store = agl.LightningStoreClient("http://localhost:4747") [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/unsloth-sft/#__codelineno-14-27) spawn_runners(store=store, n_runners=4)` Tip Try to swap [`OtelTracer`](https://microsoft.github.io/agent-lightning/0.3.0/reference/runner/#agentlightning.OtelTracer " agentlightning.OtelTracer") in the runners with other tracers like [`AgentOpsTracer`](https://microsoft.github.io/agent-lightning/0.3.0/reference/runner/#agentlightning.AgentOpsTracer " agentlightning.AgentOpsTracer") . Try to use a different adapter at the algorithm side such as [`TracerTraceToTriplet`](https://microsoft.github.io/agent-lightning/0.3.0/reference/algorithm/#agentlightning.TracerTraceToTriplet " agentlightning.TracerTraceToTriplet") to see what happens. ### Run Everything with Trainer[¶](https://microsoft.github.io/agent-lightning/0.3.0/how-to/unsloth-sft/#run-everything-with-trainer "Permanent link") We also show how to wrap everything into a single script using [`Trainer`](https://microsoft.github.io/agent-lightning/0.3.0/reference/trainer/#agentlightning.Trainer " agentlightning.Trainer") . [`sft_allinone.py`](https://github.com/microsoft/agent-lightning/tree/3b5d733861cf313fc09821a23240bbdf3cb2ee5b/examples/unsloth/sft_allinone.py) wires the same components together, replacing the manual management of runners above. `[](https://microsoft.github.io/agent-lightning/0.3.0/how-to/unsloth-sft/#__codelineno-15-1) class UnslothSupervisedFinetuning(agl.Algorithm): [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/unsloth-sft/#__codelineno-15-2) [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/unsloth-sft/#__codelineno-15-3) async def run( [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/unsloth-sft/#__codelineno-15-4) self, [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/unsloth-sft/#__codelineno-15-5) train_dataset: Optional[Dataset[GsmProblem]] = None, [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/unsloth-sft/#__codelineno-15-6) val_dataset: Optional[Dataset[GsmProblem]] = None, [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/unsloth-sft/#__codelineno-15-7) ): [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/unsloth-sft/#__codelineno-15-8) # Use the store, llm_proxy, and adapter from the trainer [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/unsloth-sft/#__codelineno-15-9) store = self.get_store() [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/unsloth-sft/#__codelineno-15-10) llm_proxy = self.get_llm_proxy() [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/unsloth-sft/#__codelineno-15-11) data_adapter = self.get_adapter() [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/unsloth-sft/#__codelineno-15-12) [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/unsloth-sft/#__codelineno-15-13) for iteration in range(self.max_iterations): [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/unsloth-sft/#__codelineno-15-14) ... # Same logic as sft_algorithm.py [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/unsloth-sft/#__codelineno-15-15) [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/unsloth-sft/#__codelineno-15-16) algo = UnslothSupervisedFinetuning( [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/unsloth-sft/#__codelineno-15-17) max_iterations=2, [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/unsloth-sft/#__codelineno-15-18) vllm_port=12316, [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/unsloth-sft/#__codelineno-15-19) train_triplet_fraction=0.5, [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/unsloth-sft/#__codelineno-15-20) initial_model_path="models/version_0", [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/unsloth-sft/#__codelineno-15-21) ) [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/unsloth-sft/#__codelineno-15-22) [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/unsloth-sft/#__codelineno-15-23) # The LLM proxy can be created before Trainer [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/unsloth-sft/#__codelineno-15-24) trainer = Trainer( [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/unsloth-sft/#__codelineno-15-25) n_runners=4, [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/unsloth-sft/#__codelineno-15-26) algorithm=algo, [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/unsloth-sft/#__codelineno-15-27) llm_proxy=LLMProxy(port=12358), [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/unsloth-sft/#__codelineno-15-28) ) [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/unsloth-sft/#__codelineno-15-29) [](https://microsoft.github.io/agent-lightning/0.3.0/how-to/unsloth-sft/#__codelineno-15-30) trainer.fit(math_agent, load_math_dataset())` You might wonder where the initialization of [`Adapter`](https://microsoft.github.io/agent-lightning/0.3.0/reference/algorithm/#agentlightning.Adapter " agentlightning.Adapter") happens in this code. It turns out that [`TracerTraceToTriplet`](https://microsoft.github.io/agent-lightning/0.3.0/reference/algorithm/#agentlightning.TracerTraceToTriplet " agentlightning.TracerTraceToTriplet") is the default adapter in [`Trainer`](https://microsoft.github.io/agent-lightning/0.3.0/reference/trainer/#agentlightning.Trainer " agentlightning.Trainer") , so we don't need to create one manually. Now you can run the example with: `[](https://microsoft.github.io/agent-lightning/0.3.0/how-to/unsloth-sft/#__codelineno-16-1) python examples/unsloth/sft_allinone.py` It starts an [`InMemoryLighningStore`](https://microsoft.github.io/agent-lightning/0.3.0/reference/store/#agentlightning.InMemoryLightningStore " agentlightning.InMemoryLightningStore") for you, launches four worker processes, iterates the SFT loop, and prints the final checkpoint path when done. Adjust `max_iterations`, `train_triplet_fraction`, `n_runners`, or the proxy port to match your hardware or training goals. If you already run an external store or proxy you can also pass those objects into [`Trainer`](https://microsoft.github.io/agent-lightning/0.3.0/reference/trainer/#agentlightning.Trainer " agentlightning.Trainer") instead of relying on the [Trainer-managed defaults](https://microsoft.github.io/agent-lightning/0.3.0/tutorials/debug/#debug-with-external-store "Debug the Algorithm-Runner Boundary") . Info As a future plan, we might graduate this example into a more powerful SFT algorithm bundled into [Algorithm Zoo](https://microsoft.github.io/agent-lightning/0.3.0/algorithm-zoo/) . Currently, this `UnslothSupervisedFinetuning` is still for demo purposes. Back to top --- # Write the First Algorithm - Agent-lightning [Skip to content](https://microsoft.github.io/agent-lightning/0.2.2/how-to/write-first-algorithm/#write-the-first-algorithm-with-agent-lightning) ⚠️ You are viewing development documentation. For stable documentation, please visit [https://microsoft.github.io/agent-lightning/stable/](https://microsoft.github.io/agent-lightning/stable/) Write the First Algorithm with Agent-lightning[¶](https://microsoft.github.io/agent-lightning/0.2.2/how-to/write-first-algorithm/#write-the-first-algorithm-with-agent-lightning "Permanent link") =================================================================================================================================================================================================== In the [first tutorial](https://microsoft.github.io/agent-lightning/0.2.2/how-to/train-first-agent/) , "Train the First Agent," we introduced the [Trainer](https://microsoft.github.io/agent-lightning/0.2.2/reference/trainer/#agentlightning.Trainer " agentlightning.Trainer") and showed how to use a pre-built algorithm like **Automatic Prompt Optimization (APO)** to improve an agent's performance. The [Trainer](https://microsoft.github.io/agent-lightning/0.2.2/reference/trainer/#agentlightning.Trainer " agentlightning.Trainer") handled all the complex interactions, letting us focus on the agent's logic. Now, we'll go a step deeper. What if you have a unique training idea that doesn't fit a standard algorithm? This tutorial will show you how to write your own custom algorithm from scratch. We'll build a simple algorithm that systematically tests a list of prompt templates and identifies the one with the highest reward. By the end, you'll understand the core mechanics of how the **Algorithm**, **Runner**, and a new component, the **Store**, work together to create the powerful training loop at the heart of Agent-lightning. Tip This tutorial helps you build a basic understanding of how to interact with Agent-lightning's core components. It's recommended that all users customizing algorithms should read this tutorial, even for those who are not planning to do prompt optimization. Core Concepts for Training[¶](https://microsoft.github.io/agent-lightning/0.2.2/how-to/write-first-algorithm/#core-concepts-for-training "Permanent link") ----------------------------------------------------------------------------------------------------------------------------------------------------------- Before diving into the [LightningStore](https://microsoft.github.io/agent-lightning/0.2.2/reference/store/#agentlightning.LightningStore " agentlightning.LightningStore") , let's define two key concepts that are central to any training process in Agent-lightning: **Resources** and the **Tracer**. ### Resources: The Tunable Assets[¶](https://microsoft.github.io/agent-lightning/0.2.2/how-to/write-first-algorithm/#resources-the-tunable-assets "Permanent link") [](https://microsoft.github.io/agent-lightning/0.2.2/how-to/write-first-algorithm/) **Resources** are the assets your algorithm is trying to improve. Think of them as the "recipe" an agent uses to perform its task. This recipe can be: * A **prompt template** that guides an LLM. * The **weights** of a machine learning model. * Any other configuration or data your agent needs. The algorithm's job is to run experiments and iteratively update these resources to find the best-performing version. ### Tracer: The Data Collector[¶](https://microsoft.github.io/agent-lightning/0.2.2/how-to/write-first-algorithm/#tracer-the-data-collector "Permanent link") How does the algorithm know if a change was an improvement? It needs data. This is where the **Tracer** comes in. The Tracer automatically **instruments** (aka modifies / patches) the agent's code. This means it watches for important events, like an LLM call, a tool being used, or **reward signals**, and records a detailed log of what happened. Each of these logs is called a **Span** (which has already been introduced in the [last tutorial](https://microsoft.github.io/agent-lightning/0.2.2/how-to/train-first-agent/) ). A collection of spans from a single task execution gives the algorithm a complete, step-by-step trace of the agent's behavior, which is essential for learning and making improvements. Our default tracer is built on the AgentOps SDK to support instrumenting code written in various Agent/non-agent frameworks. The Central Hub: The LightningStore[¶](https://microsoft.github.io/agent-lightning/0.2.2/how-to/write-first-algorithm/#the-central-hub-the-lightningstore "Permanent link") ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Now, where do all these resources, tasks, and spans live? They are all managed by the **LightningStore**. The LightningStore acts as the central database and message queue for the entire system. It's the single source of truth that decouples the Algorithm from the Runners. Note In the [last tutorial](https://microsoft.github.io/agent-lightning/0.2.2/how-to/train-first-agent/) we simplified the training loop, saying the Algorithm and Agent communicate "via the Trainer." That's true at a high level, but the component that makes it all possible is actually the **LightningStore**. * The **Algorithm** connects to the Store to `enqueue_rollout` (tasks) and `update_resources` (like new prompt templates). It also queries the Store to retrieve the resulting spans and rewards from completed rollouts. * The **Runners** connect to the Store to `dequeue_rollout` (polling for available tasks). After executing a task, they use the `Tracer` to write the resulting spans and status updates back to the Store. This architecture is key to Agent-lightning's scalability. Since the Algorithm and Runners only talk to the Store, they can run in different processes or even on different machines. ![Store Architecture](https://microsoft.github.io/agent-lightning/0.2.2/assets/store-api-visualized.svg) A Mental Model of What the Store Contains The [LightningStore](https://microsoft.github.io/agent-lightning/0.2.2/reference/store/#agentlightning.LightningStore " agentlightning.LightningStore") isn't just a simple database; it's an organized system for managing the entire training lifecycle. Here's what it keeps track of: * **Task Queue**: A queue of pending **Rollouts** waiting for a Runner to pick them up, interactable via `enqueue_rollout` and `dequeue_rollout`. * **Rollouts**: The record of a single task. A rollout contains metadata about the task and tracks all **Attempts** to complete it, interactable via `query_rollouts` and `wait_for_rollouts`. * **Attempts**: A single execution of a rollout. If an attempt fails (e.g., due to a network error), the Store can automatically schedules a retry if it's configured. Each attempt is linked to its parent rollout and contains the status and timing information. The rollout status is [synced](https://microsoft.github.io/agent-lightning/0.2.2/deep-dive/store/) with its children's status. **For beginners, you can assume each rollout has only one attempt unless you have explicitly configure the retry.** * **Spans**: The detailed, structured logs generated by the `Tracer` during an attempt. Each span is linked to its parent attempt and rollout. * **Resources**: A versioned collection of the assets (like prompt templates) that the algorithm creates. Each rollout is linked to the specific version of the resources it should use. Building a Custom Algorithm[¶](https://microsoft.github.io/agent-lightning/0.2.2/how-to/write-first-algorithm/#building-a-custom-algorithm "Permanent link") ------------------------------------------------------------------------------------------------------------------------------------------------------------- Let's build an algorithm that finds the best system prompt from a predefined list. The logic is straightforward: 1. Start with a list of candidate prompt templates. 2. For each template, create a "resource" bundle in the Store. 3. Enqueue a rollout (a task), telling the Runner to use this specific resource. 4. Wait for a Runner to pick up the task and complete it. 5. Query the Store to get the final reward from the rollout's spans. 6. After testing all templates, compare the rewards and declare the best one. We can implement this as a simple Python function that interacts directly with the [LightningStore](https://microsoft.github.io/agent-lightning/0.2.2/reference/store/#agentlightning.LightningStore " agentlightning.LightningStore") . `[](https://microsoft.github.io/agent-lightning/0.2.2/how-to/write-first-algorithm/#__codelineno-0-1) async def find_best_prompt(store, prompts_to_test, task_input): [](https://microsoft.github.io/agent-lightning/0.2.2/how-to/write-first-algorithm/#__codelineno-0-2) """A simple algorithm to find the best prompt from a list.""" [](https://microsoft.github.io/agent-lightning/0.2.2/how-to/write-first-algorithm/#__codelineno-0-3) results = [] [](https://microsoft.github.io/agent-lightning/0.2.2/how-to/write-first-algorithm/#__codelineno-0-4) [](https://microsoft.github.io/agent-lightning/0.2.2/how-to/write-first-algorithm/#__codelineno-0-5) # Iterate through each prompt to test it [](https://microsoft.github.io/agent-lightning/0.2.2/how-to/write-first-algorithm/#__codelineno-0-6) for prompt in prompts_to_test: [](https://microsoft.github.io/agent-lightning/0.2.2/how-to/write-first-algorithm/#__codelineno-0-7) print(f"[Algo] Updating prompt template to: '{prompt}'") [](https://microsoft.github.io/agent-lightning/0.2.2/how-to/write-first-algorithm/#__codelineno-0-8) [](https://microsoft.github.io/agent-lightning/0.2.2/how-to/write-first-algorithm/#__codelineno-0-9) # 1. Update the resources in the store with the new prompt [](https://microsoft.github.io/agent-lightning/0.2.2/how-to/write-first-algorithm/#__codelineno-0-10) resources_update = await store.add_resources( [](https://microsoft.github.io/agent-lightning/0.2.2/how-to/write-first-algorithm/#__codelineno-0-11) resources={"prompt_template": prompt} [](https://microsoft.github.io/agent-lightning/0.2.2/how-to/write-first-algorithm/#__codelineno-0-12) ) [](https://microsoft.github.io/agent-lightning/0.2.2/how-to/write-first-algorithm/#__codelineno-0-13) [](https://microsoft.github.io/agent-lightning/0.2.2/how-to/write-first-algorithm/#__codelineno-0-14) # 2. Enqueue a rollout task for a runner to execute [](https://microsoft.github.io/agent-lightning/0.2.2/how-to/write-first-algorithm/#__codelineno-0-15) print("[Algo] Queuing task for clients...") [](https://microsoft.github.io/agent-lightning/0.2.2/how-to/write-first-algorithm/#__codelineno-0-16) rollout = await store.enqueue_rollout( [](https://microsoft.github.io/agent-lightning/0.2.2/how-to/write-first-algorithm/#__codelineno-0-17) input=task_input, [](https://microsoft.github.io/agent-lightning/0.2.2/how-to/write-first-algorithm/#__codelineno-0-18) resources_id=resources_update.resources_id, [](https://microsoft.github.io/agent-lightning/0.2.2/how-to/write-first-algorithm/#__codelineno-0-19) ) [](https://microsoft.github.io/agent-lightning/0.2.2/how-to/write-first-algorithm/#__codelineno-0-20) print(f"[Algo] Task '{rollout.rollout_id}' is now available for clients.") [](https://microsoft.github.io/agent-lightning/0.2.2/how-to/write-first-algorithm/#__codelineno-0-21) [](https://microsoft.github.io/agent-lightning/0.2.2/how-to/write-first-algorithm/#__codelineno-0-22) # 3. Wait for the rollout to be completed by a runner [](https://microsoft.github.io/agent-lightning/0.2.2/how-to/write-first-algorithm/#__codelineno-0-23) await store.wait_for_rollouts([rollout.rollout_id]) [](https://microsoft.github.io/agent-lightning/0.2.2/how-to/write-first-algorithm/#__codelineno-0-24) [](https://microsoft.github.io/agent-lightning/0.2.2/how-to/write-first-algorithm/#__codelineno-0-25) # 4. Query the completed rollout and its spans [](https://microsoft.github.io/agent-lightning/0.2.2/how-to/write-first-algorithm/#__codelineno-0-26) completed_rollout = (await store.query_rollouts([rollout.rollout_id]))[0] [](https://microsoft.github.io/agent-lightning/0.2.2/how-to/write-first-algorithm/#__codelineno-0-27) print(f"[Algo] Received Result: {completed_rollout.model_dump_json(indent=None)}") [](https://microsoft.github.io/agent-lightning/0.2.2/how-to/write-first-algorithm/#__codelineno-0-28) [](https://microsoft.github.io/agent-lightning/0.2.2/how-to/write-first-algorithm/#__codelineno-0-29) spans = await store.query_spans(rollout.rollout_id) [](https://microsoft.github.io/agent-lightning/0.2.2/how-to/write-first-algorithm/#__codelineno-0-30) # We expect at least two spans: one for the LLM call and one for the final reward [](https://microsoft.github.io/agent-lightning/0.2.2/how-to/write-first-algorithm/#__codelineno-0-31) print(f"[Algo] Queried Spans:\n - " + "\n - ".join(str(span) for span in spans)) [](https://microsoft.github.io/agent-lightning/0.2.2/how-to/write-first-algorithm/#__codelineno-0-32) # find_final_reward is a helper function to extract the reward span [](https://microsoft.github.io/agent-lightning/0.2.2/how-to/write-first-algorithm/#__codelineno-0-33) final_reward = find_final_reward(spans) [](https://microsoft.github.io/agent-lightning/0.2.2/how-to/write-first-algorithm/#__codelineno-0-34) print(f"[Algo] Final reward: {final_reward}\n") [](https://microsoft.github.io/agent-lightning/0.2.2/how-to/write-first-algorithm/#__codelineno-0-35) [](https://microsoft.github.io/agent-lightning/0.2.2/how-to/write-first-algorithm/#__codelineno-0-36) results.append((prompt, final_reward)) [](https://microsoft.github.io/agent-lightning/0.2.2/how-to/write-first-algorithm/#__codelineno-0-37) [](https://microsoft.github.io/agent-lightning/0.2.2/how-to/write-first-algorithm/#__codelineno-0-38) # 5. Find and print the best prompt based on the collected rewards [](https://microsoft.github.io/agent-lightning/0.2.2/how-to/write-first-algorithm/#__codelineno-0-39) print(f"[Algo] All prompts and their rewards: {results}") [](https://microsoft.github.io/agent-lightning/0.2.2/how-to/write-first-algorithm/#__codelineno-0-40) best_prompt, best_reward = max(results, key=lambda item: item[1]) [](https://microsoft.github.io/agent-lightning/0.2.2/how-to/write-first-algorithm/#__codelineno-0-41) print(f"[Algo] Best prompt found: '{best_prompt}' with reward {best_reward}")` Asynchronous Operations You'll notice the `async` and `await` keywords. Agent-lightning is built on asyncio to handle concurrent operations efficiently. All interactions with the store are asynchronous network calls, so they must be awaited. The Agent and Runner[¶](https://microsoft.github.io/agent-lightning/0.2.2/how-to/write-first-algorithm/#the-agent-and-runner "Permanent link") ----------------------------------------------------------------------------------------------------------------------------------------------- Our algorithm needs an **agent** to execute the tasks and a **runner** to manage the process. The runner is a long-lived worker process. Its job is simple: 1. Connect to the [LightningStore](https://microsoft.github.io/agent-lightning/0.2.2/reference/store/#agentlightning.LightningStore " agentlightning.LightningStore") via a [LightningStoreClient](https://microsoft.github.io/agent-lightning/0.2.2/reference/store/#agentlightning.LightningStoreClient " agentlightning.LightningStoreClient") . 2. Enter a loop, constantly asking the [LightningStore](https://microsoft.github.io/agent-lightning/0.2.2/reference/store/#agentlightning.LightningStore " agentlightning.LightningStore") for new tasks (`dequeue_rollout`). 3. When it gets a task, it runs the `simple_agent` function. 4. Crucially, the runner wraps the agent execution with a **Tracer**. The tracer automatically captures all the important events (like the LLM call and the final reward) as spans and sends them back to the [LightningStore](https://microsoft.github.io/agent-lightning/0.2.2/reference/store/#agentlightning.LightningStore " agentlightning.LightningStore") . `[](https://microsoft.github.io/agent-lightning/0.2.2/how-to/write-first-algorithm/#__codelineno-1-1) # Connecting to Store [](https://microsoft.github.io/agent-lightning/0.2.2/how-to/write-first-algorithm/#__codelineno-1-2) store = agl.LightningStoreClient("http://localhost:4747") # or some other address [](https://microsoft.github.io/agent-lightning/0.2.2/how-to/write-first-algorithm/#__codelineno-1-3) runner = LitAgentRunner[str](tracer=AgentOpsTracer()) [](https://microsoft.github.io/agent-lightning/0.2.2/how-to/write-first-algorithm/#__codelineno-1-4) with runner.run_context(agent=simple_agent, store=store): # <-- where the wrapping and instrumentation happens [](https://microsoft.github.io/agent-lightning/0.2.2/how-to/write-first-algorithm/#__codelineno-1-5) await runner.iter() # polling for new tasks forever` For this example, the agent's job is to take the prompt from the resources, use it to ask an LLM a question, and return a score. `[](https://microsoft.github.io/agent-lightning/0.2.2/how-to/write-first-algorithm/#__codelineno-2-1) def simple_agent(task: str, prompt_template: PromptTemplate) -> float: [](https://microsoft.github.io/agent-lightning/0.2.2/how-to/write-first-algorithm/#__codelineno-2-2) """An agent that answers a question and gets judged by an LLM.""" [](https://microsoft.github.io/agent-lightning/0.2.2/how-to/write-first-algorithm/#__codelineno-2-3) client = OpenAI() [](https://microsoft.github.io/agent-lightning/0.2.2/how-to/write-first-algorithm/#__codelineno-2-4) [](https://microsoft.github.io/agent-lightning/0.2.2/how-to/write-first-algorithm/#__codelineno-2-5) # Generate a response using the provided prompt template [](https://microsoft.github.io/agent-lightning/0.2.2/how-to/write-first-algorithm/#__codelineno-2-6) prompt = prompt_template.format(any_question=task) [](https://microsoft.github.io/agent-lightning/0.2.2/how-to/write-first-algorithm/#__codelineno-2-7) response = client.chat.completions.create( [](https://microsoft.github.io/agent-lightning/0.2.2/how-to/write-first-algorithm/#__codelineno-2-8) model="gpt-4.1-nano", messages=[{"role": "user", "content": prompt}] [](https://microsoft.github.io/agent-lightning/0.2.2/how-to/write-first-algorithm/#__codelineno-2-9) ) [](https://microsoft.github.io/agent-lightning/0.2.2/how-to/write-first-algorithm/#__codelineno-2-10) llm_output = response.choices[0].message.content [](https://microsoft.github.io/agent-lightning/0.2.2/how-to/write-first-algorithm/#__codelineno-2-11) print(f"[Rollout] LLM returned: {llm_output}") [](https://microsoft.github.io/agent-lightning/0.2.2/how-to/write-first-algorithm/#__codelineno-2-12) [](https://microsoft.github.io/agent-lightning/0.2.2/how-to/write-first-algorithm/#__codelineno-2-13) # This llm_output and the final score are automatically logged as spans by the Tracer [](https://microsoft.github.io/agent-lightning/0.2.2/how-to/write-first-algorithm/#__codelineno-2-14) score = random.uniform(0, 1) # Replace with actual scoring logic if needed [](https://microsoft.github.io/agent-lightning/0.2.2/how-to/write-first-algorithm/#__codelineno-2-15) return score` Running the Example[¶](https://microsoft.github.io/agent-lightning/0.2.2/how-to/write-first-algorithm/#running-the-example "Permanent link") --------------------------------------------------------------------------------------------------------------------------------------------- To see everything in action, you'll need three separate terminal windows. Tip If you want to follow along, you can find the complete code for this example in the [apo\_custom\_algorithm.py](https://github.com/microsoft/agent-lightning/tree/289c9ca25b9cc8de23aba597e4e6a02a3c69d3a1/examples/apo/apo_custom_algorithm.py) file. **1\. Start the Store:** In the first terminal, start the LightningStore server. This component will wait for connections from the algorithm and the runner. The store will be listening on port `4747` ⚡ by default. `[](https://microsoft.github.io/agent-lightning/0.2.2/how-to/write-first-algorithm/#__codelineno-3-1) agl store` **2\. Start the Runner:** In the second terminal, start the runner process. It will connect to the store and wait for tasks. The code to start the runner looks like the following: `[](https://microsoft.github.io/agent-lightning/0.2.2/how-to/write-first-algorithm/#__codelineno-4-1) export OPENAI_API_KEY=sk-... # Your OpenAI API key [](https://microsoft.github.io/agent-lightning/0.2.2/how-to/write-first-algorithm/#__codelineno-4-2) python apo_custom_algorithm.py runner` You will see output indicating the runner has started and is waiting for rollouts. `[](https://microsoft.github.io/agent-lightning/0.2.2/how-to/write-first-algorithm/#__codelineno-5-1) 2025-10-14 22:23:41,339 [INFO] ... [Worker 0] Setting up tracer... [](https://microsoft.github.io/agent-lightning/0.2.2/how-to/write-first-algorithm/#__codelineno-5-2) 2025-10-14 22:23:41,343 [INFO] ... [Worker 0] Instrumentation applied. [](https://microsoft.github.io/agent-lightning/0.2.2/how-to/write-first-algorithm/#__codelineno-5-3) 2025-10-14 22:23:41,494 [INFO] ... [Worker 0] AgentOps client initialized. [](https://microsoft.github.io/agent-lightning/0.2.2/how-to/write-first-algorithm/#__codelineno-5-4) 2025-10-14 22:23:41,494 [INFO] ... [Worker 0] Started async rollouts (max: unlimited).` **3\. Start the Algorithm:** In the third terminal, run the algorithm. This will kick off the entire process. For example, we run the algorithm code shown above with the following parameters: `[](https://microsoft.github.io/agent-lightning/0.2.2/how-to/write-first-algorithm/#__codelineno-6-1) prompts_to_test = [ [](https://microsoft.github.io/agent-lightning/0.2.2/how-to/write-first-algorithm/#__codelineno-6-2) "You are a helpful assistant. {any_question}", [](https://microsoft.github.io/agent-lightning/0.2.2/how-to/write-first-algorithm/#__codelineno-6-3) "You are a knowledgeable AI. {any_question}", [](https://microsoft.github.io/agent-lightning/0.2.2/how-to/write-first-algorithm/#__codelineno-6-4) "You are a friendly chatbot. {any_question}", [](https://microsoft.github.io/agent-lightning/0.2.2/how-to/write-first-algorithm/#__codelineno-6-5) ] [](https://microsoft.github.io/agent-lightning/0.2.2/how-to/write-first-algorithm/#__codelineno-6-6) task_input = "Why is the sky blue?" [](https://microsoft.github.io/agent-lightning/0.2.2/how-to/write-first-algorithm/#__codelineno-6-7) store = agl.LightningStoreClient("http://localhost:4747") [](https://microsoft.github.io/agent-lightning/0.2.2/how-to/write-first-algorithm/#__codelineno-6-8) find_best_prompt(store, prompts_to_test, task_input)` Or you can simply use our pre-written script to try out: `[](https://microsoft.github.io/agent-lightning/0.2.2/how-to/write-first-algorithm/#__codelineno-7-1) python apo_custom_algorithm.py algo` ### Understanding the Output[¶](https://microsoft.github.io/agent-lightning/0.2.2/how-to/write-first-algorithm/#understanding-the-output "Permanent link") As the algorithm runs, you'll see logs appear across all three terminals, showing the components interacting in real-time. **Algorithm Output:** The algorithm terminal shows the main control flow: updating prompts, queuing tasks, and receiving the final results. You can also see the raw span data it retrieves from the store. `[](https://microsoft.github.io/agent-lightning/0.2.2/how-to/write-first-algorithm/#__codelineno-8-1) [Algo] Updating prompt template to: 'You are a helpful assistant. {any_question}' [](https://microsoft.github.io/agent-lightning/0.2.2/how-to/write-first-algorithm/#__codelineno-8-2) [Algo] Queuing task for clients... [](https://microsoft.github.io/agent-lightning/0.2.2/how-to/write-first-algorithm/#__codelineno-8-3) [Algo] Task 'ro-1d18988581cd' is now available for clients. [](https://microsoft.github.io/agent-lightning/0.2.2/how-to/write-first-algorithm/#__codelineno-8-4) [Algo] Received Result: rollout_id='ro-1d18988581cd' ... status='succeeded' ... [](https://microsoft.github.io/agent-lightning/0.2.2/how-to/write-first-algorithm/#__codelineno-8-5) [Algo] Queried Spans: [](https://microsoft.github.io/agent-lightning/0.2.2/how-to/write-first-algorithm/#__codelineno-8-6) - Span(name='openai.chat.completion', attributes={'gen_ai.prompt.0.content': 'You are a helpful assistant...', 'gen_ai.completion.0.content': 'The sky appears blue...'}) [](https://microsoft.github.io/agent-lightning/0.2.2/how-to/write-first-algorithm/#__codelineno-8-7) - Span(name='reward', attributes={'value': 0.95}) [](https://microsoft.github.io/agent-lightning/0.2.2/how-to/write-first-algorithm/#__codelineno-8-8) [Algo] Final reward: 0.95 [](https://microsoft.github.io/agent-lightning/0.2.2/how-to/write-first-algorithm/#__codelineno-8-9) [](https://microsoft.github.io/agent-lightning/0.2.2/how-to/write-first-algorithm/#__codelineno-8-10) [Algo] Updating prompt template to: 'You are a knowledgeable AI. {any_question}' [](https://microsoft.github.io/agent-lightning/0.2.2/how-to/write-first-algorithm/#__codelineno-8-11) ... [](https://microsoft.github.io/agent-lightning/0.2.2/how-to/write-first-algorithm/#__codelineno-8-12) [Algo] Final reward: 0.95 [](https://microsoft.github.io/agent-lightning/0.2.2/how-to/write-first-algorithm/#__codelineno-8-13) [](https://microsoft.github.io/agent-lightning/0.2.2/how-to/write-first-algorithm/#__codelineno-8-14) [Algo] Updating prompt template to: 'You are a friendly chatbot. {any_question}' [](https://microsoft.github.io/agent-lightning/0.2.2/how-to/write-first-algorithm/#__codelineno-8-15) ... [](https://microsoft.github.io/agent-lightning/0.2.2/how-to/write-first-algorithm/#__codelineno-8-16) [Algo] Final reward: 1.0 [](https://microsoft.github.io/agent-lightning/0.2.2/how-to/write-first-algorithm/#__codelineno-8-17) [](https://microsoft.github.io/agent-lightning/0.2.2/how-to/write-first-algorithm/#__codelineno-8-18) [Algo] All prompts and their rewards: [('You are a helpful assistant. {any_question}', 0.95), ('You are a knowledgeable AI. {any_question}', 0.95), ('You are a friendly chatbot. {any_question}', 1.0)] [](https://microsoft.github.io/agent-lightning/0.2.2/how-to/write-first-algorithm/#__codelineno-8-19) [Algo] Best prompt found: 'You are a friendly chatbot. {any_question}' with reward 1.0` **Runner Output:** The runner terminal shows it picking up each task, executing the agent logic, and reporting the completion. `[](https://microsoft.github.io/agent-lightning/0.2.2/how-to/write-first-algorithm/#__codelineno-9-1) [Rollout] LLM returned: The sky appears blue due to Rayleigh scattering... [](https://microsoft.github.io/agent-lightning/0.2.2/how-to/write-first-algorithm/#__codelineno-9-2) 2025-10-14 22:25:50,803 [INFO] ... [Worker 0 | Rollout ro-a9f54ac19af5] Completed in 4.24s. ... [](https://microsoft.github.io/agent-lightning/0.2.2/how-to/write-first-algorithm/#__codelineno-9-3) [](https://microsoft.github.io/agent-lightning/0.2.2/how-to/write-first-algorithm/#__codelineno-9-4) [Rollout] LLM returned: The sky looks blue because of a process called Rayleigh scattering... [](https://microsoft.github.io/agent-lightning/0.2.2/how-to/write-first-algorithm/#__codelineno-9-5) 2025-10-14 22:25:59,863 [INFO] ... [Worker 0 | Rollout ro-c67eaa9016b6] Completed in 4.06s. ...` **Store Server Output:** The store terminal shows a detailed log of every interaction, confirming its role as the central hub. You can see requests to enqueue and dequeue rollouts, add spans, and update statuses. `[](https://microsoft.github.io/agent-lightning/0.2.2/how-to/write-first-algorithm/#__codelineno-10-1) ... "POST /enqueue_rollout HTTP/1.1" 200 ... [](https://microsoft.github.io/agent-lightning/0.2.2/how-to/write-first-algorithm/#__codelineno-10-2) ... "GET /dequeue_rollout HTTP/1.1" 200 ... [](https://microsoft.github.io/agent-lightning/0.2.2/how-to/write-first-algorithm/#__codelineno-10-3) ... "POST /add_span HTTP/1.1" 200 ... [](https://microsoft.github.io/agent-lightning/0.2.2/how-to/write-first-algorithm/#__codelineno-10-4) ... "POST /update_attempt HTTP/1.1" 200 ... [](https://microsoft.github.io/agent-lightning/0.2.2/how-to/write-first-algorithm/#__codelineno-10-5) ... "POST /wait_for_rollouts HTTP/1.1" 200 ... [](https://microsoft.github.io/agent-lightning/0.2.2/how-to/write-first-algorithm/#__codelineno-10-6) ... "GET /query_spans/ro-c67eaa9016b6 HTTP/1.1" 200 ...` So Where is Trainer? You might be wondering why the [last tutorial](https://microsoft.github.io/agent-lightning/0.2.2/how-to/train-first-agent/) focused on the [Trainer](https://microsoft.github.io/agent-lightning/0.2.2/reference/trainer/#agentlightning.Trainer " agentlightning.Trainer") class, but we haven't used it here. Think of the [Trainer](https://microsoft.github.io/agent-lightning/0.2.2/reference/trainer/#agentlightning.Trainer " agentlightning.Trainer") as a convenient wrapper that manages the entire training process for you. It's perfect when you want to apply a pre-built algorithm to your agent without worrying about the underlying mechanics. The [Trainer](https://microsoft.github.io/agent-lightning/0.2.2/reference/trainer/#agentlightning.Trainer " agentlightning.Trainer") handles starting the [LightningStore](https://microsoft.github.io/agent-lightning/0.2.2/reference/store/#agentlightning.LightningStore " agentlightning.LightningStore") , coordinating the [Runners](https://microsoft.github.io/agent-lightning/0.2.2/reference/runner/#agentlightning.Runner " agentlightning.Runner") , managing their lifecycles, and handling errors. In this tutorial, however, our goal is to _build a new algorithm_. To do that, we need to interact directly with the core components: the [Store](https://microsoft.github.io/agent-lightning/0.2.2/reference/store/#agentlightning.LightningStore " agentlightning.LightningStore") , the [Runner](https://microsoft.github.io/agent-lightning/0.2.2/reference/runner/#agentlightning.Runner " agentlightning.Runner") , and the algorithm logic itself. Running them separately gives you more control and clearer, isolated logs, which is ideal for development and debugging. Once your custom algorithm is mature, you can package it to comply with our standard interface ([@algo](https://microsoft.github.io/agent-lightning/0.2.2/reference/algorithm/#agentlightning.algo " agentlightning.algo(func)") or [Algorithm](https://microsoft.github.io/agent-lightning/0.2.2/reference/algorithm/#agentlightning.Algorithm " agentlightning.Algorithm") ). This allows you to use it with the [Trainer](https://microsoft.github.io/agent-lightning/0.2.2/reference/trainer/#agentlightning.Trainer " agentlightning.Trainer") again, getting all the benefits of automated lifecycle management while using your own custom logic. A sample code doing this is available in [apo\_custom\_algorithm\_trainer.py](https://github.com/microsoft/agent-lightning/tree/289c9ca25b9cc8de23aba597e4e6a02a3c69d3a1/examples/apo/apo_custom_algorithm_trainer.py) . Back to top --- # Write the First Algorithm - Agent-lightning [Skip to content](https://microsoft.github.io/agent-lightning/0.2.1/how-to/write-first-algorithm/#write-the-first-algorithm-with-agent-lightning) ⚠️ You are viewing development documentation. For stable documentation, please visit [https://microsoft.github.io/agent-lightning/stable/](https://microsoft.github.io/agent-lightning/stable/) Write the First Algorithm with Agent-lightning[¶](https://microsoft.github.io/agent-lightning/0.2.1/how-to/write-first-algorithm/#write-the-first-algorithm-with-agent-lightning "Permanent link") =================================================================================================================================================================================================== In the [first tutorial](https://microsoft.github.io/agent-lightning/0.2.1/how-to/train-first-agent/) , "Train the First Agent," we introduced the [Trainer](https://microsoft.github.io/agent-lightning/0.2.1/reference/trainer/#agentlightning.Trainer " agentlightning.Trainer") and showed how to use a pre-built algorithm like **Automatic Prompt Optimization (APO)** to improve an agent's performance. The [Trainer](https://microsoft.github.io/agent-lightning/0.2.1/reference/trainer/#agentlightning.Trainer " agentlightning.Trainer") handled all the complex interactions, letting us focus on the agent's logic. Now, we'll go a step deeper. What if you have a unique training idea that doesn't fit a standard algorithm? This tutorial will show you how to write your own custom algorithm from scratch. We'll build a simple algorithm that systematically tests a list of prompt templates and identifies the one with the highest reward. By the end, you'll understand the core mechanics of how the **Algorithm**, **Runner**, and a new component, the **Store**, work together to create the powerful training loop at the heart of Agent-lightning. Tip This tutorial helps you build a basic understanding of how to interact with Agent-lightning's core components. It's recommended that all users customizing algorithms should read this tutorial, even for those who are not planning to do prompt optimization. Core Concepts for Training[¶](https://microsoft.github.io/agent-lightning/0.2.1/how-to/write-first-algorithm/#core-concepts-for-training "Permanent link") ----------------------------------------------------------------------------------------------------------------------------------------------------------- Before diving into the [LightningStore](https://microsoft.github.io/agent-lightning/0.2.1/reference/store/#agentlightning.LightningStore " agentlightning.LightningStore") , let's define two key concepts that are central to any training process in Agent-lightning: **Resources** and the **Tracer**. ### Resources: The Tunable Assets[¶](https://microsoft.github.io/agent-lightning/0.2.1/how-to/write-first-algorithm/#resources-the-tunable-assets "Permanent link") [](https://microsoft.github.io/agent-lightning/0.2.1/how-to/write-first-algorithm/) **Resources** are the assets your algorithm is trying to improve. Think of them as the "recipe" an agent uses to perform its task. This recipe can be: * A **prompt template** that guides an LLM. * The **weights** of a machine learning model. * Any other configuration or data your agent needs. The algorithm's job is to run experiments and iteratively update these resources to find the best-performing version. ### Tracer: The Data Collector[¶](https://microsoft.github.io/agent-lightning/0.2.1/how-to/write-first-algorithm/#tracer-the-data-collector "Permanent link") How does the algorithm know if a change was an improvement? It needs data. This is where the **Tracer** comes in. The Tracer automatically **instruments** (aka modifies / patches) the agent's code. This means it watches for important events, like an LLM call, a tool being used, or **reward signals**, and records a detailed log of what happened. Each of these logs is called a **Span** (which has already been introduced in the [last tutorial](https://microsoft.github.io/agent-lightning/0.2.1/how-to/train-first-agent/) ). A collection of spans from a single task execution gives the algorithm a complete, step-by-step trace of the agent's behavior, which is essential for learning and making improvements. Our default tracer is built on the AgentOps SDK to support instrumenting code written in various Agent/non-agent frameworks. The Central Hub: The LightningStore[¶](https://microsoft.github.io/agent-lightning/0.2.1/how-to/write-first-algorithm/#the-central-hub-the-lightningstore "Permanent link") ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Now, where do all these resources, tasks, and spans live? They are all managed by the **LightningStore**. The LightningStore acts as the central database and message queue for the entire system. It's the single source of truth that decouples the Algorithm from the Runners. Note In the [last tutorial](https://microsoft.github.io/agent-lightning/0.2.1/how-to/train-first-agent/) we simplified the training loop, saying the Algorithm and Agent communicate "via the Trainer." That's true at a high level, but the component that makes it all possible is actually the **LightningStore**. * The **Algorithm** connects to the Store to `enqueue_rollout` (tasks) and `update_resources` (like new prompt templates). It also queries the Store to retrieve the resulting spans and rewards from completed rollouts. * The **Runners** connect to the Store to `dequeue_rollout` (polling for available tasks). After executing a task, they use the `Tracer` to write the resulting spans and status updates back to the Store. This architecture is key to Agent-lightning's scalability. Since the Algorithm and Runners only talk to the Store, they can run in different processes or even on different machines. ![Store Architecture](https://microsoft.github.io/agent-lightning/0.2.1/assets/store-api-visualized.svg) A Mental Model of What the Store Contains The [LightningStore](https://microsoft.github.io/agent-lightning/0.2.1/reference/store/#agentlightning.LightningStore " agentlightning.LightningStore") isn't just a simple database; it's an organized system for managing the entire training lifecycle. Here's what it keeps track of: * **Task Queue**: A queue of pending **Rollouts** waiting for a Runner to pick them up, interactable via `enqueue_rollout` and `dequeue_rollout`. * **Rollouts**: The record of a single task. A rollout contains metadata about the task and tracks all **Attempts** to complete it, interactable via `query_rollouts` and `wait_for_rollouts`. * **Attempts**: A single execution of a rollout. If an attempt fails (e.g., due to a network error), the Store can automatically schedules a retry if it's configured. Each attempt is linked to its parent rollout and contains the status and timing information. The rollout status is [synced](https://microsoft.github.io/agent-lightning/0.2.1/deep-dive/store/) with its children's status. **For beginners, you can assume each rollout has only one attempt unless you have explicitly configure the retry.** * **Spans**: The detailed, structured logs generated by the `Tracer` during an attempt. Each span is linked to its parent attempt and rollout. * **Resources**: A versioned collection of the assets (like prompt templates) that the algorithm creates. Each rollout is linked to the specific version of the resources it should use. Building a Custom Algorithm[¶](https://microsoft.github.io/agent-lightning/0.2.1/how-to/write-first-algorithm/#building-a-custom-algorithm "Permanent link") ------------------------------------------------------------------------------------------------------------------------------------------------------------- Let's build an algorithm that finds the best system prompt from a predefined list. The logic is straightforward: 1. Start with a list of candidate prompt templates. 2. For each template, create a "resource" bundle in the Store. 3. Enqueue a rollout (a task), telling the Runner to use this specific resource. 4. Wait for a Runner to pick up the task and complete it. 5. Query the Store to get the final reward from the rollout's spans. 6. After testing all templates, compare the rewards and declare the best one. We can implement this as a simple Python function that interacts directly with the [LightningStore](https://microsoft.github.io/agent-lightning/0.2.1/reference/store/#agentlightning.LightningStore " agentlightning.LightningStore") . `[](https://microsoft.github.io/agent-lightning/0.2.1/how-to/write-first-algorithm/#__codelineno-0-1) async def find_best_prompt(store, prompts_to_test, task_input): [](https://microsoft.github.io/agent-lightning/0.2.1/how-to/write-first-algorithm/#__codelineno-0-2) """A simple algorithm to find the best prompt from a list.""" [](https://microsoft.github.io/agent-lightning/0.2.1/how-to/write-first-algorithm/#__codelineno-0-3) results = [] [](https://microsoft.github.io/agent-lightning/0.2.1/how-to/write-first-algorithm/#__codelineno-0-4) [](https://microsoft.github.io/agent-lightning/0.2.1/how-to/write-first-algorithm/#__codelineno-0-5) # Iterate through each prompt to test it [](https://microsoft.github.io/agent-lightning/0.2.1/how-to/write-first-algorithm/#__codelineno-0-6) for prompt in prompts_to_test: [](https://microsoft.github.io/agent-lightning/0.2.1/how-to/write-first-algorithm/#__codelineno-0-7) print(f"[Algo] Updating prompt template to: '{prompt}'") [](https://microsoft.github.io/agent-lightning/0.2.1/how-to/write-first-algorithm/#__codelineno-0-8) [](https://microsoft.github.io/agent-lightning/0.2.1/how-to/write-first-algorithm/#__codelineno-0-9) # 1. Update the resources in the store with the new prompt [](https://microsoft.github.io/agent-lightning/0.2.1/how-to/write-first-algorithm/#__codelineno-0-10) resources_update = await store.add_resources( [](https://microsoft.github.io/agent-lightning/0.2.1/how-to/write-first-algorithm/#__codelineno-0-11) resources={"prompt_template": prompt} [](https://microsoft.github.io/agent-lightning/0.2.1/how-to/write-first-algorithm/#__codelineno-0-12) ) [](https://microsoft.github.io/agent-lightning/0.2.1/how-to/write-first-algorithm/#__codelineno-0-13) [](https://microsoft.github.io/agent-lightning/0.2.1/how-to/write-first-algorithm/#__codelineno-0-14) # 2. Enqueue a rollout task for a runner to execute [](https://microsoft.github.io/agent-lightning/0.2.1/how-to/write-first-algorithm/#__codelineno-0-15) print("[Algo] Queuing task for clients...") [](https://microsoft.github.io/agent-lightning/0.2.1/how-to/write-first-algorithm/#__codelineno-0-16) rollout = await store.enqueue_rollout( [](https://microsoft.github.io/agent-lightning/0.2.1/how-to/write-first-algorithm/#__codelineno-0-17) input=task_input, [](https://microsoft.github.io/agent-lightning/0.2.1/how-to/write-first-algorithm/#__codelineno-0-18) resources_id=resources_update.resources_id, [](https://microsoft.github.io/agent-lightning/0.2.1/how-to/write-first-algorithm/#__codelineno-0-19) ) [](https://microsoft.github.io/agent-lightning/0.2.1/how-to/write-first-algorithm/#__codelineno-0-20) print(f"[Algo] Task '{rollout.rollout_id}' is now available for clients.") [](https://microsoft.github.io/agent-lightning/0.2.1/how-to/write-first-algorithm/#__codelineno-0-21) [](https://microsoft.github.io/agent-lightning/0.2.1/how-to/write-first-algorithm/#__codelineno-0-22) # 3. Wait for the rollout to be completed by a runner [](https://microsoft.github.io/agent-lightning/0.2.1/how-to/write-first-algorithm/#__codelineno-0-23) await store.wait_for_rollouts([rollout.rollout_id]) [](https://microsoft.github.io/agent-lightning/0.2.1/how-to/write-first-algorithm/#__codelineno-0-24) [](https://microsoft.github.io/agent-lightning/0.2.1/how-to/write-first-algorithm/#__codelineno-0-25) # 4. Query the completed rollout and its spans [](https://microsoft.github.io/agent-lightning/0.2.1/how-to/write-first-algorithm/#__codelineno-0-26) completed_rollout = (await store.query_rollouts([rollout.rollout_id]))[0] [](https://microsoft.github.io/agent-lightning/0.2.1/how-to/write-first-algorithm/#__codelineno-0-27) print(f"[Algo] Received Result: {completed_rollout.model_dump_json(indent=None)}") [](https://microsoft.github.io/agent-lightning/0.2.1/how-to/write-first-algorithm/#__codelineno-0-28) [](https://microsoft.github.io/agent-lightning/0.2.1/how-to/write-first-algorithm/#__codelineno-0-29) spans = await store.query_spans(rollout.rollout_id) [](https://microsoft.github.io/agent-lightning/0.2.1/how-to/write-first-algorithm/#__codelineno-0-30) # We expect at least two spans: one for the LLM call and one for the final reward [](https://microsoft.github.io/agent-lightning/0.2.1/how-to/write-first-algorithm/#__codelineno-0-31) print(f"[Algo] Queried Spans:\n - " + "\n - ".join(str(span) for span in spans)) [](https://microsoft.github.io/agent-lightning/0.2.1/how-to/write-first-algorithm/#__codelineno-0-32) # find_final_reward is a helper function to extract the reward span [](https://microsoft.github.io/agent-lightning/0.2.1/how-to/write-first-algorithm/#__codelineno-0-33) final_reward = find_final_reward(spans) [](https://microsoft.github.io/agent-lightning/0.2.1/how-to/write-first-algorithm/#__codelineno-0-34) print(f"[Algo] Final reward: {final_reward}\n") [](https://microsoft.github.io/agent-lightning/0.2.1/how-to/write-first-algorithm/#__codelineno-0-35) [](https://microsoft.github.io/agent-lightning/0.2.1/how-to/write-first-algorithm/#__codelineno-0-36) results.append((prompt, final_reward)) [](https://microsoft.github.io/agent-lightning/0.2.1/how-to/write-first-algorithm/#__codelineno-0-37) [](https://microsoft.github.io/agent-lightning/0.2.1/how-to/write-first-algorithm/#__codelineno-0-38) # 5. Find and print the best prompt based on the collected rewards [](https://microsoft.github.io/agent-lightning/0.2.1/how-to/write-first-algorithm/#__codelineno-0-39) print(f"[Algo] All prompts and their rewards: {results}") [](https://microsoft.github.io/agent-lightning/0.2.1/how-to/write-first-algorithm/#__codelineno-0-40) best_prompt, best_reward = max(results, key=lambda item: item[1]) [](https://microsoft.github.io/agent-lightning/0.2.1/how-to/write-first-algorithm/#__codelineno-0-41) print(f"[Algo] Best prompt found: '{best_prompt}' with reward {best_reward}")` Asynchronous Operations You'll notice the `async` and `await` keywords. Agent-lightning is built on asyncio to handle concurrent operations efficiently. All interactions with the store are asynchronous network calls, so they must be awaited. The Agent and Runner[¶](https://microsoft.github.io/agent-lightning/0.2.1/how-to/write-first-algorithm/#the-agent-and-runner "Permanent link") ----------------------------------------------------------------------------------------------------------------------------------------------- Our algorithm needs an **agent** to execute the tasks and a **runner** to manage the process. The runner is a long-lived worker process. Its job is simple: 1. Connect to the [LightningStore](https://microsoft.github.io/agent-lightning/0.2.1/reference/store/#agentlightning.LightningStore " agentlightning.LightningStore") via a [LightningStoreClient](https://microsoft.github.io/agent-lightning/0.2.1/reference/store/#agentlightning.LightningStoreClient " agentlightning.LightningStoreClient") . 2. Enter a loop, constantly asking the [LightningStore](https://microsoft.github.io/agent-lightning/0.2.1/reference/store/#agentlightning.LightningStore " agentlightning.LightningStore") for new tasks (`dequeue_rollout`). 3. When it gets a task, it runs the `simple_agent` function. 4. Crucially, the runner wraps the agent execution with a **Tracer**. The tracer automatically captures all the important events (like the LLM call and the final reward) as spans and sends them back to the [LightningStore](https://microsoft.github.io/agent-lightning/0.2.1/reference/store/#agentlightning.LightningStore " agentlightning.LightningStore") . `[](https://microsoft.github.io/agent-lightning/0.2.1/how-to/write-first-algorithm/#__codelineno-1-1) # Connecting to Store [](https://microsoft.github.io/agent-lightning/0.2.1/how-to/write-first-algorithm/#__codelineno-1-2) store = agl.LightningStoreClient("http://localhost:4747") # or some other address [](https://microsoft.github.io/agent-lightning/0.2.1/how-to/write-first-algorithm/#__codelineno-1-3) runner = LitAgentRunner[str](tracer=AgentOpsTracer()) [](https://microsoft.github.io/agent-lightning/0.2.1/how-to/write-first-algorithm/#__codelineno-1-4) with runner.run_context(agent=simple_agent, store=store): # <-- where the wrapping and instrumentation happens [](https://microsoft.github.io/agent-lightning/0.2.1/how-to/write-first-algorithm/#__codelineno-1-5) await runner.iter() # polling for new tasks forever` For this example, the agent's job is to take the prompt from the resources, use it to ask an LLM a question, and return a score. `[](https://microsoft.github.io/agent-lightning/0.2.1/how-to/write-first-algorithm/#__codelineno-2-1) def simple_agent(task: str, prompt_template: PromptTemplate) -> float: [](https://microsoft.github.io/agent-lightning/0.2.1/how-to/write-first-algorithm/#__codelineno-2-2) """An agent that answers a question and gets judged by an LLM.""" [](https://microsoft.github.io/agent-lightning/0.2.1/how-to/write-first-algorithm/#__codelineno-2-3) client = OpenAI() [](https://microsoft.github.io/agent-lightning/0.2.1/how-to/write-first-algorithm/#__codelineno-2-4) [](https://microsoft.github.io/agent-lightning/0.2.1/how-to/write-first-algorithm/#__codelineno-2-5) # Generate a response using the provided prompt template [](https://microsoft.github.io/agent-lightning/0.2.1/how-to/write-first-algorithm/#__codelineno-2-6) prompt = prompt_template.format(any_question=task) [](https://microsoft.github.io/agent-lightning/0.2.1/how-to/write-first-algorithm/#__codelineno-2-7) response = client.chat.completions.create( [](https://microsoft.github.io/agent-lightning/0.2.1/how-to/write-first-algorithm/#__codelineno-2-8) model="gpt-4.1-nano", messages=[{"role": "user", "content": prompt}] [](https://microsoft.github.io/agent-lightning/0.2.1/how-to/write-first-algorithm/#__codelineno-2-9) ) [](https://microsoft.github.io/agent-lightning/0.2.1/how-to/write-first-algorithm/#__codelineno-2-10) llm_output = response.choices[0].message.content [](https://microsoft.github.io/agent-lightning/0.2.1/how-to/write-first-algorithm/#__codelineno-2-11) print(f"[Rollout] LLM returned: {llm_output}") [](https://microsoft.github.io/agent-lightning/0.2.1/how-to/write-first-algorithm/#__codelineno-2-12) [](https://microsoft.github.io/agent-lightning/0.2.1/how-to/write-first-algorithm/#__codelineno-2-13) # This llm_output and the final score are automatically logged as spans by the Tracer [](https://microsoft.github.io/agent-lightning/0.2.1/how-to/write-first-algorithm/#__codelineno-2-14) score = random.uniform(0, 1) # Replace with actual scoring logic if needed [](https://microsoft.github.io/agent-lightning/0.2.1/how-to/write-first-algorithm/#__codelineno-2-15) return score` Running the Example[¶](https://microsoft.github.io/agent-lightning/0.2.1/how-to/write-first-algorithm/#running-the-example "Permanent link") --------------------------------------------------------------------------------------------------------------------------------------------- To see everything in action, you'll need three separate terminal windows. Tip If you want to follow along, you can find the complete code for this example in the [apo\_custom\_algorithm.py](https://github.com/microsoft/agent-lightning/tree/01101a64ba5375b9ef3afb824b2e0af81af4ddce/examples/apo/apo_custom_algorithm.py) file. **1\. Start the Store:** In the first terminal, start the LightningStore server. This component will wait for connections from the algorithm and the runner. The store will be listening on port `4747` ⚡ by default. `[](https://microsoft.github.io/agent-lightning/0.2.1/how-to/write-first-algorithm/#__codelineno-3-1) agl store` **2\. Start the Runner:** In the second terminal, start the runner process. It will connect to the store and wait for tasks. The code to start the runner looks like the following: `[](https://microsoft.github.io/agent-lightning/0.2.1/how-to/write-first-algorithm/#__codelineno-4-1) export OPENAI_API_KEY=sk-... # Your OpenAI API key [](https://microsoft.github.io/agent-lightning/0.2.1/how-to/write-first-algorithm/#__codelineno-4-2) python apo_custom_algorithm.py runner` You will see output indicating the runner has started and is waiting for rollouts. `[](https://microsoft.github.io/agent-lightning/0.2.1/how-to/write-first-algorithm/#__codelineno-5-1) 2025-10-14 22:23:41,339 [INFO] ... [Worker 0] Setting up tracer... [](https://microsoft.github.io/agent-lightning/0.2.1/how-to/write-first-algorithm/#__codelineno-5-2) 2025-10-14 22:23:41,343 [INFO] ... [Worker 0] Instrumentation applied. [](https://microsoft.github.io/agent-lightning/0.2.1/how-to/write-first-algorithm/#__codelineno-5-3) 2025-10-14 22:23:41,494 [INFO] ... [Worker 0] AgentOps client initialized. [](https://microsoft.github.io/agent-lightning/0.2.1/how-to/write-first-algorithm/#__codelineno-5-4) 2025-10-14 22:23:41,494 [INFO] ... [Worker 0] Started async rollouts (max: unlimited).` **3\. Start the Algorithm:** In the third terminal, run the algorithm. This will kick off the entire process. For example, we run the algorithm code shown above with the following parameters: `[](https://microsoft.github.io/agent-lightning/0.2.1/how-to/write-first-algorithm/#__codelineno-6-1) prompts_to_test = [ [](https://microsoft.github.io/agent-lightning/0.2.1/how-to/write-first-algorithm/#__codelineno-6-2) "You are a helpful assistant. {any_question}", [](https://microsoft.github.io/agent-lightning/0.2.1/how-to/write-first-algorithm/#__codelineno-6-3) "You are a knowledgeable AI. {any_question}", [](https://microsoft.github.io/agent-lightning/0.2.1/how-to/write-first-algorithm/#__codelineno-6-4) "You are a friendly chatbot. {any_question}", [](https://microsoft.github.io/agent-lightning/0.2.1/how-to/write-first-algorithm/#__codelineno-6-5) ] [](https://microsoft.github.io/agent-lightning/0.2.1/how-to/write-first-algorithm/#__codelineno-6-6) task_input = "Why is the sky blue?" [](https://microsoft.github.io/agent-lightning/0.2.1/how-to/write-first-algorithm/#__codelineno-6-7) store = agl.LightningStoreClient("http://localhost:4747") [](https://microsoft.github.io/agent-lightning/0.2.1/how-to/write-first-algorithm/#__codelineno-6-8) find_best_prompt(store, prompts_to_test, task_input)` Or you can simply use our pre-written script to try out: `[](https://microsoft.github.io/agent-lightning/0.2.1/how-to/write-first-algorithm/#__codelineno-7-1) python apo_custom_algorithm.py algo` ### Understanding the Output[¶](https://microsoft.github.io/agent-lightning/0.2.1/how-to/write-first-algorithm/#understanding-the-output "Permanent link") As the algorithm runs, you'll see logs appear across all three terminals, showing the components interacting in real-time. **Algorithm Output:** The algorithm terminal shows the main control flow: updating prompts, queuing tasks, and receiving the final results. You can also see the raw span data it retrieves from the store. `[](https://microsoft.github.io/agent-lightning/0.2.1/how-to/write-first-algorithm/#__codelineno-8-1) [Algo] Updating prompt template to: 'You are a helpful assistant. {any_question}' [](https://microsoft.github.io/agent-lightning/0.2.1/how-to/write-first-algorithm/#__codelineno-8-2) [Algo] Queuing task for clients... [](https://microsoft.github.io/agent-lightning/0.2.1/how-to/write-first-algorithm/#__codelineno-8-3) [Algo] Task 'ro-1d18988581cd' is now available for clients. [](https://microsoft.github.io/agent-lightning/0.2.1/how-to/write-first-algorithm/#__codelineno-8-4) [Algo] Received Result: rollout_id='ro-1d18988581cd' ... status='succeeded' ... [](https://microsoft.github.io/agent-lightning/0.2.1/how-to/write-first-algorithm/#__codelineno-8-5) [Algo] Queried Spans: [](https://microsoft.github.io/agent-lightning/0.2.1/how-to/write-first-algorithm/#__codelineno-8-6) - Span(name='openai.chat.completion', attributes={'gen_ai.prompt.0.content': 'You are a helpful assistant...', 'gen_ai.completion.0.content': 'The sky appears blue...'}) [](https://microsoft.github.io/agent-lightning/0.2.1/how-to/write-first-algorithm/#__codelineno-8-7) - Span(name='reward', attributes={'value': 0.95}) [](https://microsoft.github.io/agent-lightning/0.2.1/how-to/write-first-algorithm/#__codelineno-8-8) [Algo] Final reward: 0.95 [](https://microsoft.github.io/agent-lightning/0.2.1/how-to/write-first-algorithm/#__codelineno-8-9) [](https://microsoft.github.io/agent-lightning/0.2.1/how-to/write-first-algorithm/#__codelineno-8-10) [Algo] Updating prompt template to: 'You are a knowledgeable AI. {any_question}' [](https://microsoft.github.io/agent-lightning/0.2.1/how-to/write-first-algorithm/#__codelineno-8-11) ... [](https://microsoft.github.io/agent-lightning/0.2.1/how-to/write-first-algorithm/#__codelineno-8-12) [Algo] Final reward: 0.95 [](https://microsoft.github.io/agent-lightning/0.2.1/how-to/write-first-algorithm/#__codelineno-8-13) [](https://microsoft.github.io/agent-lightning/0.2.1/how-to/write-first-algorithm/#__codelineno-8-14) [Algo] Updating prompt template to: 'You are a friendly chatbot. {any_question}' [](https://microsoft.github.io/agent-lightning/0.2.1/how-to/write-first-algorithm/#__codelineno-8-15) ... [](https://microsoft.github.io/agent-lightning/0.2.1/how-to/write-first-algorithm/#__codelineno-8-16) [Algo] Final reward: 1.0 [](https://microsoft.github.io/agent-lightning/0.2.1/how-to/write-first-algorithm/#__codelineno-8-17) [](https://microsoft.github.io/agent-lightning/0.2.1/how-to/write-first-algorithm/#__codelineno-8-18) [Algo] All prompts and their rewards: [('You are a helpful assistant. {any_question}', 0.95), ('You are a knowledgeable AI. {any_question}', 0.95), ('You are a friendly chatbot. {any_question}', 1.0)] [](https://microsoft.github.io/agent-lightning/0.2.1/how-to/write-first-algorithm/#__codelineno-8-19) [Algo] Best prompt found: 'You are a friendly chatbot. {any_question}' with reward 1.0` **Runner Output:** The runner terminal shows it picking up each task, executing the agent logic, and reporting the completion. `[](https://microsoft.github.io/agent-lightning/0.2.1/how-to/write-first-algorithm/#__codelineno-9-1) [Rollout] LLM returned: The sky appears blue due to Rayleigh scattering... [](https://microsoft.github.io/agent-lightning/0.2.1/how-to/write-first-algorithm/#__codelineno-9-2) 2025-10-14 22:25:50,803 [INFO] ... [Worker 0 | Rollout ro-a9f54ac19af5] Completed in 4.24s. ... [](https://microsoft.github.io/agent-lightning/0.2.1/how-to/write-first-algorithm/#__codelineno-9-3) [](https://microsoft.github.io/agent-lightning/0.2.1/how-to/write-first-algorithm/#__codelineno-9-4) [Rollout] LLM returned: The sky looks blue because of a process called Rayleigh scattering... [](https://microsoft.github.io/agent-lightning/0.2.1/how-to/write-first-algorithm/#__codelineno-9-5) 2025-10-14 22:25:59,863 [INFO] ... [Worker 0 | Rollout ro-c67eaa9016b6] Completed in 4.06s. ...` **Store Server Output:** The store terminal shows a detailed log of every interaction, confirming its role as the central hub. You can see requests to enqueue and dequeue rollouts, add spans, and update statuses. `[](https://microsoft.github.io/agent-lightning/0.2.1/how-to/write-first-algorithm/#__codelineno-10-1) ... "POST /enqueue_rollout HTTP/1.1" 200 ... [](https://microsoft.github.io/agent-lightning/0.2.1/how-to/write-first-algorithm/#__codelineno-10-2) ... "GET /dequeue_rollout HTTP/1.1" 200 ... [](https://microsoft.github.io/agent-lightning/0.2.1/how-to/write-first-algorithm/#__codelineno-10-3) ... "POST /add_span HTTP/1.1" 200 ... [](https://microsoft.github.io/agent-lightning/0.2.1/how-to/write-first-algorithm/#__codelineno-10-4) ... "POST /update_attempt HTTP/1.1" 200 ... [](https://microsoft.github.io/agent-lightning/0.2.1/how-to/write-first-algorithm/#__codelineno-10-5) ... "POST /wait_for_rollouts HTTP/1.1" 200 ... [](https://microsoft.github.io/agent-lightning/0.2.1/how-to/write-first-algorithm/#__codelineno-10-6) ... "GET /query_spans/ro-c67eaa9016b6 HTTP/1.1" 200 ...` So Where is Trainer? You might be wondering why the [last tutorial](https://microsoft.github.io/agent-lightning/0.2.1/how-to/train-first-agent/) focused on the [Trainer](https://microsoft.github.io/agent-lightning/0.2.1/reference/trainer/#agentlightning.Trainer " agentlightning.Trainer") class, but we haven't used it here. Think of the [Trainer](https://microsoft.github.io/agent-lightning/0.2.1/reference/trainer/#agentlightning.Trainer " agentlightning.Trainer") as a convenient wrapper that manages the entire training process for you. It's perfect when you want to apply a pre-built algorithm to your agent without worrying about the underlying mechanics. The [Trainer](https://microsoft.github.io/agent-lightning/0.2.1/reference/trainer/#agentlightning.Trainer " agentlightning.Trainer") handles starting the [LightningStore](https://microsoft.github.io/agent-lightning/0.2.1/reference/store/#agentlightning.LightningStore " agentlightning.LightningStore") , coordinating the [Runners](https://microsoft.github.io/agent-lightning/0.2.1/reference/runner/#agentlightning.Runner " agentlightning.Runner") , managing their lifecycles, and handling errors. In this tutorial, however, our goal is to _build a new algorithm_. To do that, we need to interact directly with the core components: the [Store](https://microsoft.github.io/agent-lightning/0.2.1/reference/store/#agentlightning.LightningStore " agentlightning.LightningStore") , the [Runner](https://microsoft.github.io/agent-lightning/0.2.1/reference/runner/#agentlightning.Runner " agentlightning.Runner") , and the algorithm logic itself. Running them separately gives you more control and clearer, isolated logs, which is ideal for development and debugging. Once your custom algorithm is mature, you can package it to comply with our standard interface ([@algo](https://microsoft.github.io/agent-lightning/0.2.1/reference/algorithm/#agentlightning.algo " agentlightning.algo(func)") or [Algorithm](https://microsoft.github.io/agent-lightning/0.2.1/reference/algorithm/#agentlightning.Algorithm " agentlightning.Algorithm") ). This allows you to use it with the [Trainer](https://microsoft.github.io/agent-lightning/0.2.1/reference/trainer/#agentlightning.Trainer " agentlightning.Trainer") again, getting all the benefits of automated lifecycle management while using your own custom logic. A sample code doing this is available in [apo\_custom\_algorithm\_trainer.py](https://github.com/microsoft/agent-lightning/tree/01101a64ba5375b9ef3afb824b2e0af81af4ddce/examples/apo/apo_custom_algorithm_trainer.py) . Back to top --- # Overview - Agent-lightning [Skip to content](https://microsoft.github.io/agent-lightning/0.2.1/algorithm-zoo/#algorithm-zoo) ⚠️ You are viewing development documentation. For stable documentation, please visit [https://microsoft.github.io/agent-lightning/stable/](https://microsoft.github.io/agent-lightning/stable/) Algorithm Zoo[¶](https://microsoft.github.io/agent-lightning/0.2.1/algorithm-zoo/#algorithm-zoo "Permanent link") ================================================================================================================== AgentLightning includes several popular and frequently requested algorithms in its built-in library, allowing agent developers to use them directly. These algorithms are designed to be compatible with most agent scenarios. For customizing algorithms, see [Algorithm-side References](https://microsoft.github.io/agent-lightning/0.2.1/reference/algorithm/) . | Algorithm | Optimizing Resources | Description | | --- | --- | --- | | [APO](https://microsoft.github.io/agent-lightning/0.2.1/algorithm-zoo/apo/) | `{: [PromptTemplate][agentlightning.PromptTemplate]}` | Automatic Prompt Optimization (APO) algorithm using textual gradients and beam search. | | [VERL](https://microsoft.github.io/agent-lightning/0.2.1/algorithm-zoo/verl/) | `{"main_llm": [LLM][agentlightning.LLM]}` | Reinforcement Learning with [VERL framework](https://github.com/volcengine/verl)
. | Back to top --- # Write the First Algorithm - Agent-lightning [Skip to content](https://microsoft.github.io/agent-lightning/0.2.0/how-to/write-first-algorithm/#write-the-first-algorithm-with-agent-lightning) ⚠️ You are viewing development documentation. For stable documentation, please visit [https://microsoft.github.io/agent-lightning/stable/](https://microsoft.github.io/agent-lightning/stable/) Write the First Algorithm with Agent-lightning[¶](https://microsoft.github.io/agent-lightning/0.2.0/how-to/write-first-algorithm/#write-the-first-algorithm-with-agent-lightning "Permanent link") =================================================================================================================================================================================================== In the [first tutorial](https://microsoft.github.io/agent-lightning/0.2.0/how-to/train-first-agent/) , "Train the First Agent," we introduced the [Trainer](https://microsoft.github.io/agent-lightning/0.2.0/reference/trainer/#agentlightning.Trainer " agentlightning.Trainer") and showed how to use a pre-built algorithm like **Automatic Prompt Optimization (APO)** to improve an agent's performance. The [Trainer](https://microsoft.github.io/agent-lightning/0.2.0/reference/trainer/#agentlightning.Trainer " agentlightning.Trainer") handled all the complex interactions, letting us focus on the agent's logic. Now, we'll go a step deeper. What if you have a unique training idea that doesn't fit a standard algorithm? This tutorial will show you how to write your own custom algorithm from scratch. We'll build a simple algorithm that systematically tests a list of prompt templates and identifies the one with the highest reward. By the end, you'll understand the core mechanics of how the **Algorithm**, **Runner**, and a new component, the **Store**, work together to create the powerful training loop at the heart of Agent-lightning. Tip This tutorial helps you build a basic understanding of how to interact with Agent-lightning's core components. It's recommended that all users customizing algorithms should read this tutorial, even for those who are not planning to do prompt optimization. Core Concepts for Training[¶](https://microsoft.github.io/agent-lightning/0.2.0/how-to/write-first-algorithm/#core-concepts-for-training "Permanent link") ----------------------------------------------------------------------------------------------------------------------------------------------------------- Before diving into the [LightningStore](https://microsoft.github.io/agent-lightning/0.2.0/reference/store/#agentlightning.LightningStore " agentlightning.LightningStore") , let's define two key concepts that are central to any training process in Agent-lightning: **Resources** and the **Tracer**. ### Resources: The Tunable Assets[¶](https://microsoft.github.io/agent-lightning/0.2.0/how-to/write-first-algorithm/#resources-the-tunable-assets "Permanent link") [](https://microsoft.github.io/agent-lightning/0.2.0/how-to/write-first-algorithm/) **Resources** are the assets your algorithm is trying to improve. Think of them as the "recipe" an agent uses to perform its task. This recipe can be: * A **prompt template** that guides an LLM. * The **weights** of a machine learning model. * Any other configuration or data your agent needs. The algorithm's job is to run experiments and iteratively update these resources to find the best-performing version. ### Tracer: The Data Collector[¶](https://microsoft.github.io/agent-lightning/0.2.0/how-to/write-first-algorithm/#tracer-the-data-collector "Permanent link") How does the algorithm know if a change was an improvement? It needs data. This is where the **Tracer** comes in. The Tracer automatically **instruments** (aka modifies / patches) the agent's code. This means it watches for important events, like an LLM call, a tool being used, or **reward signals**, and records a detailed log of what happened. Each of these logs is called a **Span** (which has already been introduced in the [last tutorial](https://microsoft.github.io/agent-lightning/0.2.0/how-to/train-first-agent/) ). A collection of spans from a single task execution gives the algorithm a complete, step-by-step trace of the agent's behavior, which is essential for learning and making improvements. Our default tracer is built on the AgentOps SDK to support instrumenting code written in various Agent/non-agent frameworks. The Central Hub: The LightningStore[¶](https://microsoft.github.io/agent-lightning/0.2.0/how-to/write-first-algorithm/#the-central-hub-the-lightningstore "Permanent link") ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Now, where do all these resources, tasks, and spans live? They are all managed by the **LightningStore**. The LightningStore acts as the central database and message queue for the entire system. It's the single source of truth that decouples the Algorithm from the Runners. Note In the [last tutorial](https://microsoft.github.io/agent-lightning/0.2.0/how-to/train-first-agent/) we simplified the training loop, saying the Algorithm and Agent communicate "via the Trainer." That's true at a high level, but the component that makes it all possible is actually the **LightningStore**. * The **Algorithm** connects to the Store to `enqueue_rollout` (tasks) and `update_resources` (like new prompt templates). It also queries the Store to retrieve the resulting spans and rewards from completed rollouts. * The **Runners** connect to the Store to `dequeue_rollout` (polling for available tasks). After executing a task, they use the `Tracer` to write the resulting spans and status updates back to the Store. This architecture is key to Agent-lightning's scalability. Since the Algorithm and Runners only talk to the Store, they can run in different processes or even on different machines. ![Store Architecture](https://microsoft.github.io/agent-lightning/0.2.0/assets/store-api-visualized.svg) A Mental Model of What the Store Contains The [LightningStore](https://microsoft.github.io/agent-lightning/0.2.0/reference/store/#agentlightning.LightningStore " agentlightning.LightningStore") isn't just a simple database; it's an organized system for managing the entire training lifecycle. Here's what it keeps track of: * **Task Queue**: A queue of pending **Rollouts** waiting for a Runner to pick them up, interactable via `enqueue_rollout` and `dequeue_rollout`. * **Rollouts**: The record of a single task. A rollout contains metadata about the task and tracks all **Attempts** to complete it, interactable via `query_rollouts` and `wait_for_rollouts`. * **Attempts**: A single execution of a rollout. If an attempt fails (e.g., due to a network error), the Store can automatically schedules a retry if it's configured. Each attempt is linked to its parent rollout and contains the status and timing information. The rollout status is [synced](https://microsoft.github.io/agent-lightning/0.2.0/deep-dive/store/) with its children's status. **For beginners, you can assume each rollout has only one attempt unless you have explicitly configure the retry.** * **Spans**: The detailed, structured logs generated by the `Tracer` during an attempt. Each span is linked to its parent attempt and rollout. * **Resources**: A versioned collection of the assets (like prompt templates) that the algorithm creates. Each rollout is linked to the specific version of the resources it should use. Building a Custom Algorithm[¶](https://microsoft.github.io/agent-lightning/0.2.0/how-to/write-first-algorithm/#building-a-custom-algorithm "Permanent link") ------------------------------------------------------------------------------------------------------------------------------------------------------------- Let's build an algorithm that finds the best system prompt from a predefined list. The logic is straightforward: 1. Start with a list of candidate prompt templates. 2. For each template, create a "resource" bundle in the Store. 3. Enqueue a rollout (a task), telling the Runner to use this specific resource. 4. Wait for a Runner to pick up the task and complete it. 5. Query the Store to get the final reward from the rollout's spans. 6. After testing all templates, compare the rewards and declare the best one. We can implement this as a simple Python function that interacts directly with the [LightningStore](https://microsoft.github.io/agent-lightning/0.2.0/reference/store/#agentlightning.LightningStore " agentlightning.LightningStore") . `[](https://microsoft.github.io/agent-lightning/0.2.0/how-to/write-first-algorithm/#__codelineno-0-1) async def find_best_prompt(store, prompts_to_test, task_input): [](https://microsoft.github.io/agent-lightning/0.2.0/how-to/write-first-algorithm/#__codelineno-0-2) """A simple algorithm to find the best prompt from a list.""" [](https://microsoft.github.io/agent-lightning/0.2.0/how-to/write-first-algorithm/#__codelineno-0-3) results = [] [](https://microsoft.github.io/agent-lightning/0.2.0/how-to/write-first-algorithm/#__codelineno-0-4) [](https://microsoft.github.io/agent-lightning/0.2.0/how-to/write-first-algorithm/#__codelineno-0-5) # Iterate through each prompt to test it [](https://microsoft.github.io/agent-lightning/0.2.0/how-to/write-first-algorithm/#__codelineno-0-6) for prompt in prompts_to_test: [](https://microsoft.github.io/agent-lightning/0.2.0/how-to/write-first-algorithm/#__codelineno-0-7) print(f"[Algo] Updating prompt template to: '{prompt}'") [](https://microsoft.github.io/agent-lightning/0.2.0/how-to/write-first-algorithm/#__codelineno-0-8) [](https://microsoft.github.io/agent-lightning/0.2.0/how-to/write-first-algorithm/#__codelineno-0-9) # 1. Update the resources in the store with the new prompt [](https://microsoft.github.io/agent-lightning/0.2.0/how-to/write-first-algorithm/#__codelineno-0-10) resources_update = await store.add_resources( [](https://microsoft.github.io/agent-lightning/0.2.0/how-to/write-first-algorithm/#__codelineno-0-11) resources={"prompt_template": prompt} [](https://microsoft.github.io/agent-lightning/0.2.0/how-to/write-first-algorithm/#__codelineno-0-12) ) [](https://microsoft.github.io/agent-lightning/0.2.0/how-to/write-first-algorithm/#__codelineno-0-13) [](https://microsoft.github.io/agent-lightning/0.2.0/how-to/write-first-algorithm/#__codelineno-0-14) # 2. Enqueue a rollout task for a runner to execute [](https://microsoft.github.io/agent-lightning/0.2.0/how-to/write-first-algorithm/#__codelineno-0-15) print("[Algo] Queuing task for clients...") [](https://microsoft.github.io/agent-lightning/0.2.0/how-to/write-first-algorithm/#__codelineno-0-16) rollout = await store.enqueue_rollout( [](https://microsoft.github.io/agent-lightning/0.2.0/how-to/write-first-algorithm/#__codelineno-0-17) input=task_input, [](https://microsoft.github.io/agent-lightning/0.2.0/how-to/write-first-algorithm/#__codelineno-0-18) resources_id=resources_update.resources_id, [](https://microsoft.github.io/agent-lightning/0.2.0/how-to/write-first-algorithm/#__codelineno-0-19) ) [](https://microsoft.github.io/agent-lightning/0.2.0/how-to/write-first-algorithm/#__codelineno-0-20) print(f"[Algo] Task '{rollout.rollout_id}' is now available for clients.") [](https://microsoft.github.io/agent-lightning/0.2.0/how-to/write-first-algorithm/#__codelineno-0-21) [](https://microsoft.github.io/agent-lightning/0.2.0/how-to/write-first-algorithm/#__codelineno-0-22) # 3. Wait for the rollout to be completed by a runner [](https://microsoft.github.io/agent-lightning/0.2.0/how-to/write-first-algorithm/#__codelineno-0-23) await store.wait_for_rollouts([rollout.rollout_id]) [](https://microsoft.github.io/agent-lightning/0.2.0/how-to/write-first-algorithm/#__codelineno-0-24) [](https://microsoft.github.io/agent-lightning/0.2.0/how-to/write-first-algorithm/#__codelineno-0-25) # 4. Query the completed rollout and its spans [](https://microsoft.github.io/agent-lightning/0.2.0/how-to/write-first-algorithm/#__codelineno-0-26) completed_rollout = (await store.query_rollouts([rollout.rollout_id]))[0] [](https://microsoft.github.io/agent-lightning/0.2.0/how-to/write-first-algorithm/#__codelineno-0-27) print(f"[Algo] Received Result: {completed_rollout.model_dump_json(indent=None)}") [](https://microsoft.github.io/agent-lightning/0.2.0/how-to/write-first-algorithm/#__codelineno-0-28) [](https://microsoft.github.io/agent-lightning/0.2.0/how-to/write-first-algorithm/#__codelineno-0-29) spans = await store.query_spans(rollout.rollout_id) [](https://microsoft.github.io/agent-lightning/0.2.0/how-to/write-first-algorithm/#__codelineno-0-30) # We expect at least two spans: one for the LLM call and one for the final reward [](https://microsoft.github.io/agent-lightning/0.2.0/how-to/write-first-algorithm/#__codelineno-0-31) print(f"[Algo] Queried Spans:\n - " + "\n - ".join(str(span) for span in spans)) [](https://microsoft.github.io/agent-lightning/0.2.0/how-to/write-first-algorithm/#__codelineno-0-32) # find_final_reward is a helper function to extract the reward span [](https://microsoft.github.io/agent-lightning/0.2.0/how-to/write-first-algorithm/#__codelineno-0-33) final_reward = find_final_reward(spans) [](https://microsoft.github.io/agent-lightning/0.2.0/how-to/write-first-algorithm/#__codelineno-0-34) print(f"[Algo] Final reward: {final_reward}\n") [](https://microsoft.github.io/agent-lightning/0.2.0/how-to/write-first-algorithm/#__codelineno-0-35) [](https://microsoft.github.io/agent-lightning/0.2.0/how-to/write-first-algorithm/#__codelineno-0-36) results.append((prompt, final_reward)) [](https://microsoft.github.io/agent-lightning/0.2.0/how-to/write-first-algorithm/#__codelineno-0-37) [](https://microsoft.github.io/agent-lightning/0.2.0/how-to/write-first-algorithm/#__codelineno-0-38) # 5. Find and print the best prompt based on the collected rewards [](https://microsoft.github.io/agent-lightning/0.2.0/how-to/write-first-algorithm/#__codelineno-0-39) print(f"[Algo] All prompts and their rewards: {results}") [](https://microsoft.github.io/agent-lightning/0.2.0/how-to/write-first-algorithm/#__codelineno-0-40) best_prompt, best_reward = max(results, key=lambda item: item[1]) [](https://microsoft.github.io/agent-lightning/0.2.0/how-to/write-first-algorithm/#__codelineno-0-41) print(f"[Algo] Best prompt found: '{best_prompt}' with reward {best_reward}")` Asynchronous Operations You'll notice the `async` and `await` keywords. Agent-lightning is built on asyncio to handle concurrent operations efficiently. All interactions with the store are asynchronous network calls, so they must be awaited. The Agent and Runner[¶](https://microsoft.github.io/agent-lightning/0.2.0/how-to/write-first-algorithm/#the-agent-and-runner "Permanent link") ----------------------------------------------------------------------------------------------------------------------------------------------- Our algorithm needs an **agent** to execute the tasks and a **runner** to manage the process. The runner is a long-lived worker process. Its job is simple: 1. Connect to the [LightningStore](https://microsoft.github.io/agent-lightning/0.2.0/reference/store/#agentlightning.LightningStore " agentlightning.LightningStore") via a [LightningStoreClient](https://microsoft.github.io/agent-lightning/0.2.0/reference/store/#agentlightning.LightningStoreClient " agentlightning.LightningStoreClient") . 2. Enter a loop, constantly asking the [LightningStore](https://microsoft.github.io/agent-lightning/0.2.0/reference/store/#agentlightning.LightningStore " agentlightning.LightningStore") for new tasks (`dequeue_rollout`). 3. When it gets a task, it runs the `simple_agent` function. 4. Crucially, the runner wraps the agent execution with a **Tracer**. The tracer automatically captures all the important events (like the LLM call and the final reward) as spans and sends them back to the [LightningStore](https://microsoft.github.io/agent-lightning/0.2.0/reference/store/#agentlightning.LightningStore " agentlightning.LightningStore") . `[](https://microsoft.github.io/agent-lightning/0.2.0/how-to/write-first-algorithm/#__codelineno-1-1) # Connecting to Store [](https://microsoft.github.io/agent-lightning/0.2.0/how-to/write-first-algorithm/#__codelineno-1-2) store = agl.LightningStoreClient("http://localhost:4747") # or some other address [](https://microsoft.github.io/agent-lightning/0.2.0/how-to/write-first-algorithm/#__codelineno-1-3) runner = LitAgentRunner[str](tracer=AgentOpsTracer()) [](https://microsoft.github.io/agent-lightning/0.2.0/how-to/write-first-algorithm/#__codelineno-1-4) with runner.run_context(agent=simple_agent, store=store): # <-- where the wrapping and instrumentation happens [](https://microsoft.github.io/agent-lightning/0.2.0/how-to/write-first-algorithm/#__codelineno-1-5) await runner.iter() # polling for new tasks forever` For this example, the agent's job is to take the prompt from the resources, use it to ask an LLM a question, and return a score. `[](https://microsoft.github.io/agent-lightning/0.2.0/how-to/write-first-algorithm/#__codelineno-2-1) def simple_agent(task: str, prompt_template: PromptTemplate) -> float: [](https://microsoft.github.io/agent-lightning/0.2.0/how-to/write-first-algorithm/#__codelineno-2-2) """An agent that answers a question and gets judged by an LLM.""" [](https://microsoft.github.io/agent-lightning/0.2.0/how-to/write-first-algorithm/#__codelineno-2-3) client = OpenAI() [](https://microsoft.github.io/agent-lightning/0.2.0/how-to/write-first-algorithm/#__codelineno-2-4) [](https://microsoft.github.io/agent-lightning/0.2.0/how-to/write-first-algorithm/#__codelineno-2-5) # Generate a response using the provided prompt template [](https://microsoft.github.io/agent-lightning/0.2.0/how-to/write-first-algorithm/#__codelineno-2-6) prompt = prompt_template.format(any_question=task) [](https://microsoft.github.io/agent-lightning/0.2.0/how-to/write-first-algorithm/#__codelineno-2-7) response = client.chat.completions.create( [](https://microsoft.github.io/agent-lightning/0.2.0/how-to/write-first-algorithm/#__codelineno-2-8) model="gpt-4.1-nano", messages=[{"role": "user", "content": prompt}] [](https://microsoft.github.io/agent-lightning/0.2.0/how-to/write-first-algorithm/#__codelineno-2-9) ) [](https://microsoft.github.io/agent-lightning/0.2.0/how-to/write-first-algorithm/#__codelineno-2-10) llm_output = response.choices[0].message.content [](https://microsoft.github.io/agent-lightning/0.2.0/how-to/write-first-algorithm/#__codelineno-2-11) print(f"[Rollout] LLM returned: {llm_output}") [](https://microsoft.github.io/agent-lightning/0.2.0/how-to/write-first-algorithm/#__codelineno-2-12) [](https://microsoft.github.io/agent-lightning/0.2.0/how-to/write-first-algorithm/#__codelineno-2-13) # This llm_output and the final score are automatically logged as spans by the Tracer [](https://microsoft.github.io/agent-lightning/0.2.0/how-to/write-first-algorithm/#__codelineno-2-14) score = random.uniform(0, 1) # Replace with actual scoring logic if needed [](https://microsoft.github.io/agent-lightning/0.2.0/how-to/write-first-algorithm/#__codelineno-2-15) return score` Running the Example[¶](https://microsoft.github.io/agent-lightning/0.2.0/how-to/write-first-algorithm/#running-the-example "Permanent link") --------------------------------------------------------------------------------------------------------------------------------------------- To see everything in action, you'll need three separate terminal windows. Tip If you want to follow along, you can find the complete code for this example in the [apo\_custom\_algorithm.py](https://github.com/microsoft/agent-lightning/tree/22454adedbfe9d14d59996356e1e493128f8ee9d/examples/apo/apo_custom_algorithm.py) file. **1\. Start the Store:** In the first terminal, start the LightningStore server. This component will wait for connections from the algorithm and the runner. The store will be listening on port `4747` ⚡ by default. `[](https://microsoft.github.io/agent-lightning/0.2.0/how-to/write-first-algorithm/#__codelineno-3-1) agl store` **2\. Start the Runner:** In the second terminal, start the runner process. It will connect to the store and wait for tasks. The code to start the runner looks like the following: `[](https://microsoft.github.io/agent-lightning/0.2.0/how-to/write-first-algorithm/#__codelineno-4-1) export OPENAI_API_KEY=sk-... # Your OpenAI API key [](https://microsoft.github.io/agent-lightning/0.2.0/how-to/write-first-algorithm/#__codelineno-4-2) python apo_custom_algorithm.py runner` You will see output indicating the runner has started and is waiting for rollouts. `[](https://microsoft.github.io/agent-lightning/0.2.0/how-to/write-first-algorithm/#__codelineno-5-1) 2025-10-14 22:23:41,339 [INFO] ... [Worker 0] Setting up tracer... [](https://microsoft.github.io/agent-lightning/0.2.0/how-to/write-first-algorithm/#__codelineno-5-2) 2025-10-14 22:23:41,343 [INFO] ... [Worker 0] Instrumentation applied. [](https://microsoft.github.io/agent-lightning/0.2.0/how-to/write-first-algorithm/#__codelineno-5-3) 2025-10-14 22:23:41,494 [INFO] ... [Worker 0] AgentOps client initialized. [](https://microsoft.github.io/agent-lightning/0.2.0/how-to/write-first-algorithm/#__codelineno-5-4) 2025-10-14 22:23:41,494 [INFO] ... [Worker 0] Started async rollouts (max: unlimited).` **3\. Start the Algorithm:** In the third terminal, run the algorithm. This will kick off the entire process. For example, we run the algorithm code shown above with the following parameters: `[](https://microsoft.github.io/agent-lightning/0.2.0/how-to/write-first-algorithm/#__codelineno-6-1) prompts_to_test = [ [](https://microsoft.github.io/agent-lightning/0.2.0/how-to/write-first-algorithm/#__codelineno-6-2) "You are a helpful assistant. {any_question}", [](https://microsoft.github.io/agent-lightning/0.2.0/how-to/write-first-algorithm/#__codelineno-6-3) "You are a knowledgeable AI. {any_question}", [](https://microsoft.github.io/agent-lightning/0.2.0/how-to/write-first-algorithm/#__codelineno-6-4) "You are a friendly chatbot. {any_question}", [](https://microsoft.github.io/agent-lightning/0.2.0/how-to/write-first-algorithm/#__codelineno-6-5) ] [](https://microsoft.github.io/agent-lightning/0.2.0/how-to/write-first-algorithm/#__codelineno-6-6) task_input = "Why is the sky blue?" [](https://microsoft.github.io/agent-lightning/0.2.0/how-to/write-first-algorithm/#__codelineno-6-7) store = agl.LightningStoreClient("http://localhost:4747") [](https://microsoft.github.io/agent-lightning/0.2.0/how-to/write-first-algorithm/#__codelineno-6-8) find_best_prompt(store, prompts_to_test, task_input)` Or you can simply use our pre-written script to try out: `[](https://microsoft.github.io/agent-lightning/0.2.0/how-to/write-first-algorithm/#__codelineno-7-1) python apo_custom_algorithm.py algo` ### Understanding the Output[¶](https://microsoft.github.io/agent-lightning/0.2.0/how-to/write-first-algorithm/#understanding-the-output "Permanent link") As the algorithm runs, you'll see logs appear across all three terminals, showing the components interacting in real-time. **Algorithm Output:** The algorithm terminal shows the main control flow: updating prompts, queuing tasks, and receiving the final results. You can also see the raw span data it retrieves from the store. `[](https://microsoft.github.io/agent-lightning/0.2.0/how-to/write-first-algorithm/#__codelineno-8-1) [Algo] Updating prompt template to: 'You are a helpful assistant. {any_question}' [](https://microsoft.github.io/agent-lightning/0.2.0/how-to/write-first-algorithm/#__codelineno-8-2) [Algo] Queuing task for clients... [](https://microsoft.github.io/agent-lightning/0.2.0/how-to/write-first-algorithm/#__codelineno-8-3) [Algo] Task 'ro-1d18988581cd' is now available for clients. [](https://microsoft.github.io/agent-lightning/0.2.0/how-to/write-first-algorithm/#__codelineno-8-4) [Algo] Received Result: rollout_id='ro-1d18988581cd' ... status='succeeded' ... [](https://microsoft.github.io/agent-lightning/0.2.0/how-to/write-first-algorithm/#__codelineno-8-5) [Algo] Queried Spans: [](https://microsoft.github.io/agent-lightning/0.2.0/how-to/write-first-algorithm/#__codelineno-8-6) - Span(name='openai.chat.completion', attributes={'gen_ai.prompt.0.content': 'You are a helpful assistant...', 'gen_ai.completion.0.content': 'The sky appears blue...'}) [](https://microsoft.github.io/agent-lightning/0.2.0/how-to/write-first-algorithm/#__codelineno-8-7) - Span(name='reward', attributes={'value': 0.95}) [](https://microsoft.github.io/agent-lightning/0.2.0/how-to/write-first-algorithm/#__codelineno-8-8) [Algo] Final reward: 0.95 [](https://microsoft.github.io/agent-lightning/0.2.0/how-to/write-first-algorithm/#__codelineno-8-9) [](https://microsoft.github.io/agent-lightning/0.2.0/how-to/write-first-algorithm/#__codelineno-8-10) [Algo] Updating prompt template to: 'You are a knowledgeable AI. {any_question}' [](https://microsoft.github.io/agent-lightning/0.2.0/how-to/write-first-algorithm/#__codelineno-8-11) ... [](https://microsoft.github.io/agent-lightning/0.2.0/how-to/write-first-algorithm/#__codelineno-8-12) [Algo] Final reward: 0.95 [](https://microsoft.github.io/agent-lightning/0.2.0/how-to/write-first-algorithm/#__codelineno-8-13) [](https://microsoft.github.io/agent-lightning/0.2.0/how-to/write-first-algorithm/#__codelineno-8-14) [Algo] Updating prompt template to: 'You are a friendly chatbot. {any_question}' [](https://microsoft.github.io/agent-lightning/0.2.0/how-to/write-first-algorithm/#__codelineno-8-15) ... [](https://microsoft.github.io/agent-lightning/0.2.0/how-to/write-first-algorithm/#__codelineno-8-16) [Algo] Final reward: 1.0 [](https://microsoft.github.io/agent-lightning/0.2.0/how-to/write-first-algorithm/#__codelineno-8-17) [](https://microsoft.github.io/agent-lightning/0.2.0/how-to/write-first-algorithm/#__codelineno-8-18) [Algo] All prompts and their rewards: [('You are a helpful assistant. {any_question}', 0.95), ('You are a knowledgeable AI. {any_question}', 0.95), ('You are a friendly chatbot. {any_question}', 1.0)] [](https://microsoft.github.io/agent-lightning/0.2.0/how-to/write-first-algorithm/#__codelineno-8-19) [Algo] Best prompt found: 'You are a friendly chatbot. {any_question}' with reward 1.0` **Runner Output:** The runner terminal shows it picking up each task, executing the agent logic, and reporting the completion. `[](https://microsoft.github.io/agent-lightning/0.2.0/how-to/write-first-algorithm/#__codelineno-9-1) [Rollout] LLM returned: The sky appears blue due to Rayleigh scattering... [](https://microsoft.github.io/agent-lightning/0.2.0/how-to/write-first-algorithm/#__codelineno-9-2) 2025-10-14 22:25:50,803 [INFO] ... [Worker 0 | Rollout ro-a9f54ac19af5] Completed in 4.24s. ... [](https://microsoft.github.io/agent-lightning/0.2.0/how-to/write-first-algorithm/#__codelineno-9-3) [](https://microsoft.github.io/agent-lightning/0.2.0/how-to/write-first-algorithm/#__codelineno-9-4) [Rollout] LLM returned: The sky looks blue because of a process called Rayleigh scattering... [](https://microsoft.github.io/agent-lightning/0.2.0/how-to/write-first-algorithm/#__codelineno-9-5) 2025-10-14 22:25:59,863 [INFO] ... [Worker 0 | Rollout ro-c67eaa9016b6] Completed in 4.06s. ...` **Store Server Output:** The store terminal shows a detailed log of every interaction, confirming its role as the central hub. You can see requests to enqueue and dequeue rollouts, add spans, and update statuses. `[](https://microsoft.github.io/agent-lightning/0.2.0/how-to/write-first-algorithm/#__codelineno-10-1) ... "POST /enqueue_rollout HTTP/1.1" 200 ... [](https://microsoft.github.io/agent-lightning/0.2.0/how-to/write-first-algorithm/#__codelineno-10-2) ... "GET /dequeue_rollout HTTP/1.1" 200 ... [](https://microsoft.github.io/agent-lightning/0.2.0/how-to/write-first-algorithm/#__codelineno-10-3) ... "POST /add_span HTTP/1.1" 200 ... [](https://microsoft.github.io/agent-lightning/0.2.0/how-to/write-first-algorithm/#__codelineno-10-4) ... "POST /update_attempt HTTP/1.1" 200 ... [](https://microsoft.github.io/agent-lightning/0.2.0/how-to/write-first-algorithm/#__codelineno-10-5) ... "POST /wait_for_rollouts HTTP/1.1" 200 ... [](https://microsoft.github.io/agent-lightning/0.2.0/how-to/write-first-algorithm/#__codelineno-10-6) ... "GET /query_spans/ro-c67eaa9016b6 HTTP/1.1" 200 ...` So Where is Trainer? You might be wondering why the [last tutorial](https://microsoft.github.io/agent-lightning/0.2.0/how-to/train-first-agent/) focused on the [Trainer](https://microsoft.github.io/agent-lightning/0.2.0/reference/trainer/#agentlightning.Trainer " agentlightning.Trainer") class, but we haven't used it here. Think of the [Trainer](https://microsoft.github.io/agent-lightning/0.2.0/reference/trainer/#agentlightning.Trainer " agentlightning.Trainer") as a convenient wrapper that manages the entire training process for you. It's perfect when you want to apply a pre-built algorithm to your agent without worrying about the underlying mechanics. The [Trainer](https://microsoft.github.io/agent-lightning/0.2.0/reference/trainer/#agentlightning.Trainer " agentlightning.Trainer") handles starting the [LightningStore](https://microsoft.github.io/agent-lightning/0.2.0/reference/store/#agentlightning.LightningStore " agentlightning.LightningStore") , coordinating the [Runners](https://microsoft.github.io/agent-lightning/0.2.0/reference/runner/#agentlightning.Runner " agentlightning.Runner") , managing their lifecycles, and handling errors. In this tutorial, however, our goal is to _build a new algorithm_. To do that, we need to interact directly with the core components: the [Store](https://microsoft.github.io/agent-lightning/0.2.0/reference/store/#agentlightning.LightningStore " agentlightning.LightningStore") , the [Runner](https://microsoft.github.io/agent-lightning/0.2.0/reference/runner/#agentlightning.Runner " agentlightning.Runner") , and the algorithm logic itself. Running them separately gives you more control and clearer, isolated logs, which is ideal for development and debugging. Once your custom algorithm is mature, you can package it to comply with our standard interface ([@algo](https://microsoft.github.io/agent-lightning/0.2.0/reference/algorithm/#agentlightning.algo " agentlightning.algo(func)") or [Algorithm](https://microsoft.github.io/agent-lightning/0.2.0/reference/algorithm/#agentlightning.Algorithm " agentlightning.Algorithm") ). This allows you to use it with the [Trainer](https://microsoft.github.io/agent-lightning/0.2.0/reference/trainer/#agentlightning.Trainer " agentlightning.Trainer") again, getting all the benefits of automated lifecycle management while using your own custom logic. A sample code doing this is available in [apo\_custom\_algorithm\_trainer.py](https://github.com/microsoft/agent-lightning/tree/22454adedbfe9d14d59996356e1e493128f8ee9d/examples/apo/apo_custom_algorithm_trainer.py) . Back to top --- # Server-Client Architecture - Agent Lightning [Skip to content](https://microsoft.github.io/agent-lightning/0.1.2/deep-dive/server-client-architecture/#server-client-architecture) Server-client Architecture[¶](https://microsoft.github.io/agent-lightning/0.1.2/deep-dive/server-client-architecture/#server-client-architecture "Permanent link") =================================================================================================================================================================== Article to be written. sequenceDiagram participant RL as RL Framework participant TS as Training Server participant AC as Agent Client participant AG as Agent AC->>TS: Upload Dataset (1) RL->>TS: Start RL Server (2) TS->>RL: Latest Model (3) loop for each batch of tasks loop for each task in the batch AC->>TS: Request Task (4) TS->>AC: Send Task & Model API (5) AC->>AG: Run Agent with Task & Model API (6) loop for each LLM call AC->>AG: Prompt (7) AG->>AC: Response (8) end AG->>AC: Rewarded Trace (9) AC->>TS: Send Rewarded Trace (10) end TS->>RL: Send Batch of Traces (11) RL->>TS: Return Updated Model (12) end Back to top --- # Parallelize - Agent-lightning [Skip to content](https://microsoft.github.io/agent-lightning/0.2.0/tutorials/parallelize/#scaling-out-algorithms-and-rollouts) ⚠️ You are viewing development documentation. For stable documentation, please visit [https://microsoft.github.io/agent-lightning/stable/](https://microsoft.github.io/agent-lightning/stable/) Scaling out Algorithms and Rollouts[¶](https://microsoft.github.io/agent-lightning/0.2.0/tutorials/parallelize/#scaling-out-algorithms-and-rollouts "Permanent link") ====================================================================================================================================================================== Agent-lightning splits training into an **algorithm bundle** and a **runner bundle** that exchange work through the [`LightningStore`](https://microsoft.github.io/agent-lightning/0.2.0/reference/store/#agentlightning.LightningStore " agentlightning.LightningStore") . This tutorial shows how to increase rollout throughput, place bundles across processes or machines, and keep the algorithm side scalable with external frameworks. Parallelizing Rollouts with [`Trainer`](https://microsoft.github.io/agent-lightning/0.2.0/reference/trainer/#agentlightning.Trainer " agentlightning.Trainer") [¶](https://microsoft.github.io/agent-lightning/0.2.0/tutorials/parallelize/#parallelizing-rollouts-with-trainer "Permanent link") ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Before we dive into the details of the bundles and execution strategies, let's first revisit how to parallelize rollouts with [`Trainer`](https://microsoft.github.io/agent-lightning/0.2.0/reference/trainer/#agentlightning.Trainer " agentlightning.Trainer") . [`Trainer`](https://microsoft.github.io/agent-lightning/0.2.0/reference/trainer/#agentlightning.Trainer " agentlightning.Trainer") is the quickest way to dial up parallelism. Even when `n_runners = 1`, calling [`Trainer.fit`](https://microsoft.github.io/agent-lightning/0.2.0/reference/trainer/#agentlightning.Trainer.fit " fit(agent, train_dataset=None, *, val_dataset=None)") runs the algorithm and runners in parallel. The algorithm enqueues rollouts; runners dequeue them and execute your [`LitAgent`](https://microsoft.github.io/agent-lightning/0.2.0/reference/agent/#agentlightning.LitAgent " agentlightning.LitAgent") , and the algorithm collects spans via its [`Adapter`](https://microsoft.github.io/agent-lightning/0.2.0/reference/algorithm/#agentlightning.Adapter " agentlightning.Adapter") before scheduling the next batch. Note One of the most important features of [`Trainer`](https://microsoft.github.io/agent-lightning/0.2.0/reference/trainer/#agentlightning.Trainer " agentlightning.Trainer") is the ability to abort things gracefully. For example, if you press `Ctrl+C` in the terminal, the algorithm will abort and the runners will stop executing. If the algorithm crashes, the runners will also stop executing. Increase throughput by setting `n_runners` when constructing the trainer. The following example comes from [train\_calc\_agent.py](https://github.com/microsoft/agent-lightning/tree/22454adedbfe9d14d59996356e1e493128f8ee9d/examples/calc_x/train_calc_agent.py) . Since backend LLMs usually use techniques like [continuous batching](https://docs.vllm.ai/en/latest/) to increase throughput, you do not have to worry about overwhelming the backend with too many requests. `[](https://microsoft.github.io/agent-lightning/0.2.0/tutorials/parallelize/#__codelineno-0-1) import agentlightning as agl [](https://microsoft.github.io/agent-lightning/0.2.0/tutorials/parallelize/#__codelineno-0-2) from datasets import Dataset as HFDataset [](https://microsoft.github.io/agent-lightning/0.2.0/tutorials/parallelize/#__codelineno-0-3) from calc_agent import calc_agent [](https://microsoft.github.io/agent-lightning/0.2.0/tutorials/parallelize/#__codelineno-0-4) [](https://microsoft.github.io/agent-lightning/0.2.0/tutorials/parallelize/#__codelineno-0-5) train_dataset = HFDataset.from_parquet("data/train.parquet").to_list() [](https://microsoft.github.io/agent-lightning/0.2.0/tutorials/parallelize/#__codelineno-0-6) val_dataset = HFDataset.from_parquet("data/test.parquet").to_list() [](https://microsoft.github.io/agent-lightning/0.2.0/tutorials/parallelize/#__codelineno-0-7) [](https://microsoft.github.io/agent-lightning/0.2.0/tutorials/parallelize/#__codelineno-0-8) algorithm = agl.VERL(verl_config) [](https://microsoft.github.io/agent-lightning/0.2.0/tutorials/parallelize/#__codelineno-0-9) [](https://microsoft.github.io/agent-lightning/0.2.0/tutorials/parallelize/#__codelineno-0-10) trainer = agl.Trainer( [](https://microsoft.github.io/agent-lightning/0.2.0/tutorials/parallelize/#__codelineno-0-11) algorithm=algorithm, [](https://microsoft.github.io/agent-lightning/0.2.0/tutorials/parallelize/#__codelineno-0-12) n_runners=8, # launch eight rollout workers [](https://microsoft.github.io/agent-lightning/0.2.0/tutorials/parallelize/#__codelineno-0-13) tracer=agl.OtelTracer(), [](https://microsoft.github.io/agent-lightning/0.2.0/tutorials/parallelize/#__codelineno-0-14) adapter=agl.LlmProxyTraceToTriplet(), [](https://microsoft.github.io/agent-lightning/0.2.0/tutorials/parallelize/#__codelineno-0-15) ) [](https://microsoft.github.io/agent-lightning/0.2.0/tutorials/parallelize/#__codelineno-0-16) [](https://microsoft.github.io/agent-lightning/0.2.0/tutorials/parallelize/#__codelineno-0-17) trainer.fit(calc_agent, train_dataset=train_dataset, val_dataset=val_dataset)` In [`Trainer`](https://microsoft.github.io/agent-lightning/0.2.0/reference/trainer/#agentlightning.Trainer " agentlightning.Trainer") , there are multiple other initialization parameters that you can use to customize the training process. For example, you can use `max_rollouts` to keep smoke tests short. Pass a concrete [`LightningStore`](https://microsoft.github.io/agent-lightning/0.2.0/reference/store/#agentlightning.LightningStore " agentlightning.LightningStore") instance when you need persistence or want to share the queue across multiple scripts. Tip Before scaling out, run [`Trainer.dev()`](https://microsoft.github.io/agent-lightning/0.2.0/reference/trainer/#agentlightning.Trainer.dev " dev(agent, train_dataset=None, *, val_dataset=None)") with `n_runners=1` to verify the rollout logic and spans without burning GPU hours. Bundles and Execution Strategies[¶](https://microsoft.github.io/agent-lightning/0.2.0/tutorials/parallelize/#bundles-and-execution-strategies "Permanent link") ---------------------------------------------------------------------------------------------------------------------------------------------------------------- When [`Trainer`](https://microsoft.github.io/agent-lightning/0.2.0/reference/trainer/#agentlightning.Trainer " agentlightning.Trainer") starts, it packages its configuration into two callable **bundles**: ![Illustration of bundles and execution strategies](https://microsoft.github.io/agent-lightning/0.2.0/assets/execution-bundles.svg) The **algorithm bundle** wraps your [`Algorithm`](https://microsoft.github.io/agent-lightning/0.2.0/reference/algorithm/#agentlightning.Algorithm " agentlightning.Algorithm") , adapter, and any LLM proxy into a single callable that can be aborted via a signal event. `[](https://microsoft.github.io/agent-lightning/0.2.0/tutorials/parallelize/#__codelineno-1-1) async def algorithm_bundle(store: LightningStore, event: ExecutionEvent) -> None: [](https://microsoft.github.io/agent-lightning/0.2.0/tutorials/parallelize/#__codelineno-1-2) ...` The **runner bundle** wraps the [`Runner`](https://microsoft.github.io/agent-lightning/0.2.0/reference/runner/#agentlightning.Runner " agentlightning.Runner") , tracer, hooks, and agent into a single callable that can be aborted via a signal event. Unlike the algorithm bundle, the runner bundle is expected to be replicated. `[](https://microsoft.github.io/agent-lightning/0.2.0/tutorials/parallelize/#__codelineno-2-1) async def runner_bundle(store: LightningStore, worker_id: int, event: ExecutionEvent) -> None: [](https://microsoft.github.io/agent-lightning/0.2.0/tutorials/parallelize/#__codelineno-2-2) ...` An **execution strategy** then decides where those bundles are placed (threads vs processes vs multiple machines), how many runner replicas to launch, and how lifecycle events such as shutdown are coordinated. By default, the trainer builds an [`InMemoryLightningStore`](https://microsoft.github.io/agent-lightning/0.2.0/reference/store/#agentlightning.InMemoryLightningStore " agentlightning.InMemoryLightningStore") if you do not provide one. Because that store has no locking or cross-process transport, the execution strategy is the component that wraps it in thread-safe or HTTP-safe facades ([`LightningStoreThreaded`](https://microsoft.github.io/agent-lightning/0.2.0/reference/store/#agentlightning.LightningStoreThreaded " agentlightning.LightningStoreThreaded") , [`LightningStoreServer`](https://microsoft.github.io/agent-lightning/0.2.0/reference/store/#agentlightning.LightningStoreServer " agentlightning.LightningStoreServer") ) before handing it to bundles. For a deeper look at these facades, see [Understanding the Store](https://microsoft.github.io/agent-lightning/0.2.0/deep-dive/store/) and [Birds' Eye View](https://microsoft.github.io/agent-lightning/0.2.0/deep-dive/birds-eye-view/) . Agent-lightning provides two built-in execution strategies: [`SharedMemoryExecutionStrategy`](https://microsoft.github.io/agent-lightning/0.2.0/reference/trainer/#agentlightning.SharedMemoryExecutionStrategy " agentlightning.SharedMemoryExecutionStrategy") and [`ClientServerExecutionStrategy`](https://microsoft.github.io/agent-lightning/0.2.0/reference/trainer/#agentlightning.ClientServerExecutionStrategy " agentlightning.ClientServerExecutionStrategy") . You can pass a string alias, a configuration dictionary, or a pre-built strategy instance: ``[](https://microsoft.github.io/agent-lightning/0.2.0/tutorials/parallelize/#__codelineno-3-1) import agentlightning as agl [](https://microsoft.github.io/agent-lightning/0.2.0/tutorials/parallelize/#__codelineno-3-2) [](https://microsoft.github.io/agent-lightning/0.2.0/tutorials/parallelize/#__codelineno-3-3) algorithm = agl.Baseline() [](https://microsoft.github.io/agent-lightning/0.2.0/tutorials/parallelize/#__codelineno-3-4) [](https://microsoft.github.io/agent-lightning/0.2.0/tutorials/parallelize/#__codelineno-3-5) # Short alias for the shared-memory strategy. [](https://microsoft.github.io/agent-lightning/0.2.0/tutorials/parallelize/#__codelineno-3-6) # Because the runner lives on the main thread in this mode, [](https://microsoft.github.io/agent-lightning/0.2.0/tutorials/parallelize/#__codelineno-3-7) # n_runners must be 1 unless you move the algorithm to the main thread. [](https://microsoft.github.io/agent-lightning/0.2.0/tutorials/parallelize/#__codelineno-3-8) trainer = agl.Trainer(algorithm=algorithm, n_runners=1, strategy="shm") [](https://microsoft.github.io/agent-lightning/0.2.0/tutorials/parallelize/#__codelineno-3-9) [](https://microsoft.github.io/agent-lightning/0.2.0/tutorials/parallelize/#__codelineno-3-10) # Dict with overrides; keep the algorithm on the main thread so multiple runner threads can spawn. [](https://microsoft.github.io/agent-lightning/0.2.0/tutorials/parallelize/#__codelineno-3-11) # Specifying `n_runners` inside strategy is equivalent to passing `n_runners` to the trainer. [](https://microsoft.github.io/agent-lightning/0.2.0/tutorials/parallelize/#__codelineno-3-12) trainer = agl.Trainer( [](https://microsoft.github.io/agent-lightning/0.2.0/tutorials/parallelize/#__codelineno-3-13) algorithm=algorithm, [](https://microsoft.github.io/agent-lightning/0.2.0/tutorials/parallelize/#__codelineno-3-14) strategy={ [](https://microsoft.github.io/agent-lightning/0.2.0/tutorials/parallelize/#__codelineno-3-15) "type": "shm", [](https://microsoft.github.io/agent-lightning/0.2.0/tutorials/parallelize/#__codelineno-3-16) "n_runners": 8, [](https://microsoft.github.io/agent-lightning/0.2.0/tutorials/parallelize/#__codelineno-3-17) "main_thread": "algorithm", [](https://microsoft.github.io/agent-lightning/0.2.0/tutorials/parallelize/#__codelineno-3-18) }, [](https://microsoft.github.io/agent-lightning/0.2.0/tutorials/parallelize/#__codelineno-3-19) ) [](https://microsoft.github.io/agent-lightning/0.2.0/tutorials/parallelize/#__codelineno-3-20) [](https://microsoft.github.io/agent-lightning/0.2.0/tutorials/parallelize/#__codelineno-3-21) # Pass an existing strategy instance – Trainer respects the strategy's own `n_runners`. [](https://microsoft.github.io/agent-lightning/0.2.0/tutorials/parallelize/#__codelineno-3-22) strategy = agl.SharedMemoryExecutionStrategy(main_thread="algorithm", n_runners=4) [](https://microsoft.github.io/agent-lightning/0.2.0/tutorials/parallelize/#__codelineno-3-23) trainer = agl.Trainer(algorithm=algorithm, strategy=strategy)`` If you omit the strategy, the trainer defaults to `ClientServerExecutionStrategy(n_runners=trainer.n_runners)`. You can still re-specify the client-server strategy through aliases or configuration to tweak ports and other settings: `[](https://microsoft.github.io/agent-lightning/0.2.0/tutorials/parallelize/#__codelineno-4-1) trainer = agl.Trainer( [](https://microsoft.github.io/agent-lightning/0.2.0/tutorials/parallelize/#__codelineno-4-2) algorithm=algorithm, [](https://microsoft.github.io/agent-lightning/0.2.0/tutorials/parallelize/#__codelineno-4-3) n_runners=8, [](https://microsoft.github.io/agent-lightning/0.2.0/tutorials/parallelize/#__codelineno-4-4) strategy={"type": "cs", "server_port": 9999}, [](https://microsoft.github.io/agent-lightning/0.2.0/tutorials/parallelize/#__codelineno-4-5) )` Environment variables give you another layer of control. For example: `[](https://microsoft.github.io/agent-lightning/0.2.0/tutorials/parallelize/#__codelineno-5-1) import os [](https://microsoft.github.io/agent-lightning/0.2.0/tutorials/parallelize/#__codelineno-5-2) [](https://microsoft.github.io/agent-lightning/0.2.0/tutorials/parallelize/#__codelineno-5-3) os.environ["AGL_SERVER_PORT"] = "10000" [](https://microsoft.github.io/agent-lightning/0.2.0/tutorials/parallelize/#__codelineno-5-4) os.environ["AGL_CURRENT_ROLE"] = "algorithm" [](https://microsoft.github.io/agent-lightning/0.2.0/tutorials/parallelize/#__codelineno-5-5) os.environ["AGL_MANAGED_STORE"] = "0" [](https://microsoft.github.io/agent-lightning/0.2.0/tutorials/parallelize/#__codelineno-5-6) [](https://microsoft.github.io/agent-lightning/0.2.0/tutorials/parallelize/#__codelineno-5-7) trainer = agl.Trainer(algorithm=algorithm, n_runners=8, strategy="cs")` The resulting [`ClientServerExecutionStrategy`](https://microsoft.github.io/agent-lightning/0.2.0/reference/trainer/#agentlightning.ClientServerExecutionStrategy " agentlightning.ClientServerExecutionStrategy") picks up the port, role, and managed-store flag from the environment. Tip The same configuration patterns apply to other trainer components. For example, `[](https://microsoft.github.io/agent-lightning/0.2.0/tutorials/parallelize/#__codelineno-6-1) trainer = agl.Trainer(algorithm=algorithm, tracer=agl.OtelTracer())` wires in a custom tracer, while `[](https://microsoft.github.io/agent-lightning/0.2.0/tutorials/parallelize/#__codelineno-7-1) trainer = agl.Trainer(algorithm=algorithm, adapter="agentlightning.adapter.TraceToMessages")` swaps in a different adapter. Passing a dict lets you tweak the init parameters of defaults without naming the class explicitly: `[](https://microsoft.github.io/agent-lightning/0.2.0/tutorials/parallelize/#__codelineno-8-1) trainer = agl.Trainer( [](https://microsoft.github.io/agent-lightning/0.2.0/tutorials/parallelize/#__codelineno-8-2) algorithm=algorithm, [](https://microsoft.github.io/agent-lightning/0.2.0/tutorials/parallelize/#__codelineno-8-3) adapter={"agent_match": "plan_agent", "repair_hierarchy": False}, [](https://microsoft.github.io/agent-lightning/0.2.0/tutorials/parallelize/#__codelineno-8-4) )` The next sections walk through the two built-in strategies and how they affect placement and store access. Client-server Architecture[¶](https://microsoft.github.io/agent-lightning/0.2.0/tutorials/parallelize/#client-server-architecture "Permanent link") ---------------------------------------------------------------------------------------------------------------------------------------------------- The default [`ClientServerExecutionStrategy`](https://microsoft.github.io/agent-lightning/0.2.0/reference/trainer/#agentlightning.ClientServerExecutionStrategy " agentlightning.ClientServerExecutionStrategy") starts a [`LightningStoreServer`](https://microsoft.github.io/agent-lightning/0.2.0/reference/store/#agentlightning.LightningStoreServer " agentlightning.LightningStoreServer") alongside the algorithm and spawns runner processes that talk to it through [`LightningStoreClient`](https://microsoft.github.io/agent-lightning/0.2.0/reference/store/#agentlightning.LightningStoreClient " agentlightning.LightningStoreClient") . All runners share the HTTP endpoint, so the queue and spans stay consistent across processes or machines. If you simply instantiate [`Trainer`](https://microsoft.github.io/agent-lightning/0.2.0/reference/trainer/#agentlightning.Trainer " agentlightning.Trainer") (as above), it will send the algorithm bundle and runner bundle to [`ClientServerExecutionStrategy`](https://microsoft.github.io/agent-lightning/0.2.0/reference/trainer/#agentlightning.ClientServerExecutionStrategy " agentlightning.ClientServerExecutionStrategy") , which will then: 1. Launch \\(N+1\\) processes: \\(N\\) runner processes and 1 algorithm process (one of them could live in the main process). 2. The algorithm process will take the store received from [`Trainer`](https://microsoft.github.io/agent-lightning/0.2.0/reference/trainer/#agentlightning.Trainer " agentlightning.Trainer") , wrap it in a [`LightningStoreServer`](https://microsoft.github.io/agent-lightning/0.2.0/reference/store/#agentlightning.LightningStoreServer " agentlightning.LightningStoreServer") , and start serving it over HTTP. 3. The runner processes discard the store and create a new store, which is a client that connects to the algorithm process through [`LightningStoreClient`](https://microsoft.github.io/agent-lightning/0.2.0/reference/store/#agentlightning.LightningStoreClient " agentlightning.LightningStoreClient") , and start executing the runner bundle. 4. The strategy automatically escalates shutdown (cooperative stop → `SIGINT` → `terminate()` → `kill()`) so long-running runners do not linger. You can override server placement or ports, and whether to automatically wrap the store, through constructor arguments or environment variables: `[](https://microsoft.github.io/agent-lightning/0.2.0/tutorials/parallelize/#__codelineno-9-1) trainer = agl.Trainer( [](https://microsoft.github.io/agent-lightning/0.2.0/tutorials/parallelize/#__codelineno-9-2) algorithm=algorithm, [](https://microsoft.github.io/agent-lightning/0.2.0/tutorials/parallelize/#__codelineno-9-3) n_runners=1, [](https://microsoft.github.io/agent-lightning/0.2.0/tutorials/parallelize/#__codelineno-9-4) strategy={ [](https://microsoft.github.io/agent-lightning/0.2.0/tutorials/parallelize/#__codelineno-9-5) "type": "cs", [](https://microsoft.github.io/agent-lightning/0.2.0/tutorials/parallelize/#__codelineno-9-6) "server_host": "0.0.0.0", [](https://microsoft.github.io/agent-lightning/0.2.0/tutorials/parallelize/#__codelineno-9-7) "server_port": 9999, [](https://microsoft.github.io/agent-lightning/0.2.0/tutorials/parallelize/#__codelineno-9-8) "main_process": "runner", [](https://microsoft.github.io/agent-lightning/0.2.0/tutorials/parallelize/#__codelineno-9-9) }, [](https://microsoft.github.io/agent-lightning/0.2.0/tutorials/parallelize/#__codelineno-9-10) )` Set `AGL_SERVER_HOST` and `AGL_SERVER_PORT` if you prefer environment-based configuration. You can also use `AGL_MANAGED_STORE` if you do not want the execution strategy to wrap the store for you. An example is shown in [Debugging with External Store](https://microsoft.github.io/agent-lightning/0.2.0/tutorials/debug/#debug-with-external-store "Debug the Algorithm-Runner Boundary") . Algorithms sometimes require heterogeneous computation resources, such as GPU accelerators, while runners sometimes require a specific environment to run because many agent frameworks are fragile in their dependencies. A role-based launch pattern helps you place the algorithm on a dedicated machine with more GPU memory, while runners can live on another machine with more flexible dependencies. This is possible via `AGL_CURRENT_ROLE="algorithm"` or `AGL_CURRENT_ROLE="runner"` environment variables. When running on different machines, you also need to set `AGL_SERVER_HOST` and `AGL_SERVER_PORT` to the IP address and port of the algorithm machine. You might recognize that this convention is very similar to `MASTER_ADDR` and `MASTER_PORT` in [PyTorch distributed training](https://docs.pytorch.org/docs/stable/notes/ddp.html) . ### Shared-memory Strategy[¶](https://microsoft.github.io/agent-lightning/0.2.0/tutorials/parallelize/#shared-memory-strategy "Permanent link") [`SharedMemoryExecutionStrategy`](https://microsoft.github.io/agent-lightning/0.2.0/reference/trainer/#agentlightning.SharedMemoryExecutionStrategy " agentlightning.SharedMemoryExecutionStrategy") keeps everything inside one process. The runner runs on the main thread (by default) while the algorithm lives on a Python thread guarded by [`LightningStoreThreaded`](https://microsoft.github.io/agent-lightning/0.2.0/reference/store/#agentlightning.LightningStoreThreaded " agentlightning.LightningStoreThreaded") . Use it when you want easier debugging with shared breakpoints and no serialization overhead, or minimal startup time for unit tests. It's not a good choice for many algorithms that require heavy model training because [`LightningStoreThreaded`](https://microsoft.github.io/agent-lightning/0.2.0/reference/store/#agentlightning.LightningStoreThreaded " agentlightning.LightningStoreThreaded") does not work for multiprocessing. Using it with multiprocessing algorithms will lead to undefined behavior. Sample configuration: `[](https://microsoft.github.io/agent-lightning/0.2.0/tutorials/parallelize/#__codelineno-10-1) trainer = agl.Trainer( [](https://microsoft.github.io/agent-lightning/0.2.0/tutorials/parallelize/#__codelineno-10-2) algorithm=algorithm, [](https://microsoft.github.io/agent-lightning/0.2.0/tutorials/parallelize/#__codelineno-10-3) strategy="shm", [](https://microsoft.github.io/agent-lightning/0.2.0/tutorials/parallelize/#__codelineno-10-4) )` You can further customize the init parameters of [`SharedMemoryExecutionStrategy`](https://microsoft.github.io/agent-lightning/0.2.0/reference/trainer/#agentlightning.SharedMemoryExecutionStrategy " agentlightning.SharedMemoryExecutionStrategy") . With `main_thread="runner"`, the runner occupies the main thread and `n_runners` must be `1`. The strategy respects `AGL_MANAGED_STORE`; set it to `0` to opt out of the `LightningStoreThreaded` wrapper. Parallelizing Algorithms[¶](https://microsoft.github.io/agent-lightning/0.2.0/tutorials/parallelize/#parallelizing-algorithms "Permanent link") ------------------------------------------------------------------------------------------------------------------------------------------------ Runner parallelism scales rollout throughput, but the algorithm loop remains a single-process loop inside the execution strategy. We understand that many algorithms have parallelization built in, but that's outside the parallelization scope of Agent-lightning. Agent-lightning strives to make algorithms’ own parallelization work well under our execution strategies. The biggest challenge turns out to come from the store. For example, [`VERL`](https://microsoft.github.io/agent-lightning/0.2.0/algorithm-zoo/verl/#agentlightning.algorithm.verl.VERL " VERL") uses [Ray](https://www.ray.io/) and launches [FSDP](https://docs.pytorch.org/tutorials/intermediate/FSDP_tutorial.html) and [vLLM](https://vllm.ai/) components internally. [`ClientServerExecutionStrategy`](https://microsoft.github.io/agent-lightning/0.2.0/reference/trainer/#agentlightning.ClientServerExecutionStrategy " agentlightning.ClientServerExecutionStrategy") has to make sure that the server is not simultaneously serving in multiple processes or Ray workers, and that there is only one single authoritative source of truth for all subprocesses to connect to. Subprocesses connect to the store via a small [`LightningStoreClient`](https://microsoft.github.io/agent-lightning/0.2.0/reference/store/#agentlightning.LightningStoreClient " agentlightning.LightningStoreClient") bundled within [`LightningStoreServer`](https://microsoft.github.io/agent-lightning/0.2.0/reference/store/#agentlightning.LightningStoreServer " agentlightning.LightningStoreServer") . Note The [birds' eye view](https://microsoft.github.io/agent-lightning/0.2.0/deep-dive/birds-eye-view/#birds-eye-view-client-server-strategy "Client-server Strategy") illustrates how adapters, proxies, and stores interact when the algorithm spawns additional workers. Use that diagram as a checklist when introducing new distributed components. Back to top --- # Overview - Agent-lightning [Skip to content](https://microsoft.github.io/agent-lightning/0.2.0/algorithm-zoo/#algorithm-zoo) ⚠️ You are viewing development documentation. For stable documentation, please visit [https://microsoft.github.io/agent-lightning/stable/](https://microsoft.github.io/agent-lightning/stable/) Algorithm Zoo[¶](https://microsoft.github.io/agent-lightning/0.2.0/algorithm-zoo/#algorithm-zoo "Permanent link") ================================================================================================================== AgentLightning includes several popular and frequently requested algorithms in its built-in library, allowing agent developers to use them directly. These algorithms are designed to be compatible with most agent scenarios. For customizing algorithms, see [Algorithm-side References](https://microsoft.github.io/agent-lightning/0.2.0/reference/algorithm/) . | Algorithm | Optimizing Resources | Description | | --- | --- | --- | | [APO](https://microsoft.github.io/agent-lightning/0.2.0/algorithm-zoo/apo/) | `{: [PromptTemplate][agentlightning.PromptTemplate]}` | Automatic Prompt Optimization (APO) algorithm using textual gradients and beam search. | | [VERL](https://microsoft.github.io/agent-lightning/0.2.0/algorithm-zoo/verl/) | `{"main_llm": [LLM][agentlightning.LLM]}` | Reinforcement Learning with [VERL framework](https://github.com/volcengine/verl)
. | Back to top --- # APO - Agent-lightning [Skip to content](https://microsoft.github.io/agent-lightning/0.2.2/algorithm-zoo/apo/#apo) ⚠️ You are viewing development documentation. For stable documentation, please visit [https://microsoft.github.io/agent-lightning/stable/](https://microsoft.github.io/agent-lightning/stable/) APO[¶](https://microsoft.github.io/agent-lightning/0.2.2/algorithm-zoo/apo/#apo "Permanent link") ================================================================================================== Shortcut You can use the shortcut `agl.APO(...)` to create an APO instance. `[](https://microsoft.github.io/agent-lightning/0.2.2/algorithm-zoo/apo/#__codelineno-0-1) import agentlightning as agl [](https://microsoft.github.io/agent-lightning/0.2.2/algorithm-zoo/apo/#__codelineno-0-2) [](https://microsoft.github.io/agent-lightning/0.2.2/algorithm-zoo/apo/#__codelineno-0-3) agl.APO(...)` Installation[¶](https://microsoft.github.io/agent-lightning/0.2.2/algorithm-zoo/apo/#installation "Permanent link") -------------------------------------------------------------------------------------------------------------------- `[](https://microsoft.github.io/agent-lightning/0.2.2/algorithm-zoo/apo/#__codelineno-1-1) pip install agentlightning[apo]` Scope of Current Implementation[¶](https://microsoft.github.io/agent-lightning/0.2.2/algorithm-zoo/apo/#scope-of-current-implementation "Permanent link") ---------------------------------------------------------------------------------------------------------------------------------------------------------- APO is currently scoped to optimize a single prompt template. Optimizing multiple prompt templates is not supported yet. There is however no restriction on the number of variable placeholders in the prompt template (can range from zero to many). It's possible that invalid prompts are created during the optimization process. It is up to the agent developer to ensure that the prompt template is valid for the agent's task. Initial Prompt[¶](https://microsoft.github.io/agent-lightning/0.2.2/algorithm-zoo/apo/#initial-prompt "Permanent link") ------------------------------------------------------------------------------------------------------------------------ APO expects the initial prompt to be provided in the `initial_resources` dictionary. This can be done in two approaches: 1. Pass to the [Trainer](https://microsoft.github.io/agent-lightning/0.2.2/reference/trainer/#agentlightning.Trainer " agentlightning.Trainer") constructor: `[](https://microsoft.github.io/agent-lightning/0.2.2/algorithm-zoo/apo/#__codelineno-2-1) trainer = agl.Trainer( [](https://microsoft.github.io/agent-lightning/0.2.2/algorithm-zoo/apo/#__codelineno-2-2) algorithm=agl.APO(...), [](https://microsoft.github.io/agent-lightning/0.2.2/algorithm-zoo/apo/#__codelineno-2-3) initial_resources={"main_prompt": agl.PromptTemplate(template="You are a helpful assistant.", engine="f-string")}, [](https://microsoft.github.io/agent-lightning/0.2.2/algorithm-zoo/apo/#__codelineno-2-4) )` 1. Pass to the `[APO][agentlightning.algorithm.apo.APO].set_initial_resources()` method: `[](https://microsoft.github.io/agent-lightning/0.2.2/algorithm-zoo/apo/#__codelineno-3-1) algo = agl.APO(...) [](https://microsoft.github.io/agent-lightning/0.2.2/algorithm-zoo/apo/#__codelineno-3-2) algo.set_initial_resources( [](https://microsoft.github.io/agent-lightning/0.2.2/algorithm-zoo/apo/#__codelineno-3-3) {"this_is_also_valid_key": agl.PromptTemplate(template="You are a helpful assistant.", engine="f-string")} [](https://microsoft.github.io/agent-lightning/0.2.2/algorithm-zoo/apo/#__codelineno-3-4) )` The resource key can be arbitrary, which is used to identify the prompt template in [class-based implementations](https://microsoft.github.io/agent-lightning/0.2.2/tutorials/write-agents/) when you have multiple resources. When the key changes, the agent developer needs to update the key in the `rollout` method. Tutorials Using APO[¶](https://microsoft.github.io/agent-lightning/0.2.2/algorithm-zoo/apo/#tutorials-using-apo "Permanent link") ---------------------------------------------------------------------------------------------------------------------------------- * [Train the First Agent with APO](https://microsoft.github.io/agent-lightning/0.2.2/how-to/train-first-agent/) - A step-by-step guide to training your first agent using APO. References[¶](https://microsoft.github.io/agent-lightning/0.2.2/algorithm-zoo/apo/#references "Permanent link") ---------------------------------------------------------------------------------------------------------------- ### `agentlightning.algorithm.apo` [¶](https://microsoft.github.io/agent-lightning/0.2.2/algorithm-zoo/apo/#agentlightning.algorithm.apo "Permanent link") #### `APO` [¶](https://microsoft.github.io/agent-lightning/0.2.2/algorithm-zoo/apo/#agentlightning.algorithm.apo.APO "Permanent link") Bases: `[Algorithm](https://microsoft.github.io/agent-lightning/0.2.2/reference/algorithm/#agentlightning.Algorithm " agentlightning.Algorithm (agentlightning.algorithm.base.Algorithm)") `, `Generic[T_task]` Automatic Prompt Optimization (APO) algorithm using textual gradients and beam search. APO is an iterative prompt optimization algorithm that uses LLM-generated textual gradients to improve prompts through a beam search process. It evaluates prompts on rollouts, computes critiques based on the results, and applies edits to generate improved prompts. The algorithm operates in rounds, where each round: 1. Samples parent prompts from the current beam 2. Generates new prompts by computing textual gradients and applying edits 3. Evaluates all candidates on a validation set 4. Selects the top-k prompts for the next round Based on the ideas from: * [ProTeGi](https://aclanthology.org/2023.emnlp-main.494.pdf) * [TextGrad](https://github.com/zou-group/textgrad) ##### `__init__(async_openai_client, *, gradient_model='gpt-5-mini', apply_edit_model='gpt-4.1-mini', diversity_temperature=1.0, gradient_batch_size=4, val_batch_size=16, beam_width=4, branch_factor=4, beam_rounds=3, rollout_batch_timeout=3600.0, run_initial_validation=True, _poml_trace=False)` [¶](https://microsoft.github.io/agent-lightning/0.2.2/algorithm-zoo/apo/#agentlightning.algorithm.apo.APO.__init__ "Permanent link") Initialize the APO algorithm with configuration parameters. Parameters: * **`async_openai_client`** (`AsyncOpenAI`) – AsyncOpenAI client for making LLM API calls. * **`gradient_model`** (`str`, default: `'gpt-5-mini'` ) – Model name for computing textual gradients (critiques). * **`apply_edit_model`** (`str`, default: `'gpt-4.1-mini'` ) – Model name for applying edits based on critiques. * **`diversity_temperature`** (`float`, default: `1.0` ) – Temperature parameter for LLM calls to control diversity. * **`gradient_batch_size`** (`int`, default: `4` ) – Number of rollout results to sample for gradient computation. * **`val_batch_size`** (`int`, default: `16` ) – Number of validation examples to use for evaluation. * **`beam_width`** (`int`, default: `4` ) – Number of top-scoring prompts to keep in the beam at each round. * **`branch_factor`** (`int`, default: `4` ) – Number of new prompt candidates to generate from each parent prompt by applying textual gradient edits. This controls the expansion of the search tree. * **`beam_rounds`** (`int`, default: `3` ) – Number of beam search rounds to perform. * **`rollout_batch_timeout`** (`float`, default: `3600.0` ) – Maximum time in seconds to wait for rollout batch completion. * **`run_initial_validation`** (`bool`, default: `True` ) – If True, runs validation on the seed prompt before starting optimization to establish a baseline score. Defaults to True. ##### `compute_textual_gradient(current_prompt, rollout_results, *, prefix=None)` `async` [¶](https://microsoft.github.io/agent-lightning/0.2.2/algorithm-zoo/apo/#agentlightning.algorithm.apo.APO.compute_textual_gradient "Permanent link") Compute a textual gradient (critique) for the current prompt based on rollout results. This method samples rollout results, sends them to an LLM along with the current prompt, and generates a critique describing how the prompt could be improved. Parameters: * **`current_prompt`** (`VersionedPromptTemplate`) – The prompt template to critique. * **`rollout_results`** (`List[RolloutResultForAPO]`) – List of rollout results containing spans, messages, and rewards. Returns: * `Optional[str]` – A textual critique generated by the LLM, or None if generation fails. ##### `evaluate_prompt_on_batch(prompt, resource_name, dataset, mode, *, prefix=None)` `async` [¶](https://microsoft.github.io/agent-lightning/0.2.2/algorithm-zoo/apo/#agentlightning.algorithm.apo.APO.evaluate_prompt_on_batch "Permanent link") Evaluate a prompt on a batch of tasks by running rollouts and computing average reward. This method: 1. Adds the prompt as a named resource to the store 2. Enqueues rollouts for each task in the dataset 3. Waits for rollouts to complete (with timeout) 4. Computes and returns the average reward Parameters: * **`prompt`** (`VersionedPromptTemplate`) – The prompt template string to evaluate. * **`resource_name`** (`str`) – The name to register the prompt under in the store. * **`dataset`** (`Sequence[T_task]`) – Sequence of tasks to evaluate the prompt on. * **`mode`** (`[RolloutMode](https://microsoft.github.io/agent-lightning/0.2.2/reference/types/#agentlightning.RolloutMode " agentlightning.RolloutMode = Literal['train', 'val', 'test'] module-attribute (agentlightning.types.RolloutMode)") `) – Rollout mode ("train" or "val") for logging/tracking. Returns: * `List[RolloutResultForAPO]` – A tuple of (rollout\_results, average\_reward) where rollout\_results contains * `float` – detailed information for each rollout and average\_reward is the mean final reward. ##### `get_adapter()` [¶](https://microsoft.github.io/agent-lightning/0.2.2/algorithm-zoo/apo/#agentlightning.algorithm.apo.APO.get_adapter "Permanent link") Get the adapter for converting spans to messages. Returns: * `[TraceToMessages](https://microsoft.github.io/agent-lightning/0.2.2/reference/algorithm/#agentlightning.TraceToMessages " agentlightning.TraceToMessages (agentlightning.adapter.messages.TraceToMessages)") ` – The TraceToMessages instance for this algorithm. Raises: * `ValueError` – If the adapter is not a TraceToMessages. ##### `get_best_prompt()` [¶](https://microsoft.github.io/agent-lightning/0.2.2/algorithm-zoo/apo/#agentlightning.algorithm.apo.APO.get_best_prompt "Permanent link") Retrieve the best prompt discovered during optimization. Returns: * `[PromptTemplate](https://microsoft.github.io/agent-lightning/0.2.2/reference/types/#agentlightning.PromptTemplate " agentlightning.PromptTemplate (agentlightning.types.PromptTemplate)") ` – The prompt template with the highest validation score found so far. Raises: * `ValueError` – If no best prompt has been found yet (run() not called). ##### `get_rollout_results(rollout, *, prefix=None)` `async` [¶](https://microsoft.github.io/agent-lightning/0.2.2/algorithm-zoo/apo/#agentlightning.algorithm.apo.APO.get_rollout_results "Permanent link") Convert completed rollouts to APO-compatible result format. Fetches spans for each rollout, adapts them to messages, and packages them with rewards and status information for gradient computation. Parameters: * **`rollout`** (`List[[Rollout](https://microsoft.github.io/agent-lightning/0.2.2/reference/types/#agentlightning.Rollout " agentlightning.Rollout (agentlightning.types.Rollout)") ]`) – List of completed rollout metadata. Returns: * `List[RolloutResultForAPO]` – List of rollout results formatted for APO processing. ##### `get_seed_prompt_template()` [¶](https://microsoft.github.io/agent-lightning/0.2.2/algorithm-zoo/apo/#agentlightning.algorithm.apo.APO.get_seed_prompt_template "Permanent link") Extract the initial prompt template from the algorithm's resources. Returns: * `Tuple[str, [PromptTemplate](https://microsoft.github.io/agent-lightning/0.2.2/reference/types/#agentlightning.PromptTemplate " agentlightning.PromptTemplate (agentlightning.types.PromptTemplate)") ]` – A tuple of (resource\_name, prompt\_template) representing the seed prompt. Raises: * `ValueError` – If initial\_resources is not set or no PromptTemplate is found. ##### `run(train_dataset=None, val_dataset=None)` `async` [¶](https://microsoft.github.io/agent-lightning/0.2.2/algorithm-zoo/apo/#agentlightning.algorithm.apo.APO.run "Permanent link") Execute the APO algorithm to optimize prompts through beam search with textual gradients. The algorithm performs iterative prompt optimization over multiple rounds: * Each round: samples parent prompts, generates new candidates via textual gradients, evaluates all candidates on validation data, and keeps the top performers * Tracks the historically best prompt across all rounds * Uses different training data samples for each gradient computation to ensure diversity Parameters: * **`train_dataset`** (`Optional[[Dataset](https://microsoft.github.io/agent-lightning/0.2.2/reference/types/#agentlightning.Dataset " agentlightning.Dataset (agentlightning.types.Dataset)") [T_task]]`, default: `None` ) – Dataset of tasks for computing textual gradients. Required. * **`val_dataset`** (`Optional[[Dataset](https://microsoft.github.io/agent-lightning/0.2.2/reference/types/#agentlightning.Dataset " agentlightning.Dataset (agentlightning.types.Dataset)") [T_task]]`, default: `None` ) – Dataset of tasks for evaluating and selecting prompts. Required. Raises: * `ValueError` – If train\_dataset or val\_dataset is None, or if resources are not set. ##### `textual_gradient_and_apply_edit(current_prompt, rollout, *, prefix=None)` `async` [¶](https://microsoft.github.io/agent-lightning/0.2.2/algorithm-zoo/apo/#agentlightning.algorithm.apo.APO.textual_gradient_and_apply_edit "Permanent link") Generate an improved prompt by computing a textual gradient and applying an edit. This is the main optimization step that: 1. Computes a critique (textual gradient) based on rollout performance 2. Uses another LLM to apply the critique and generate an improved prompt Parameters: * **`current_prompt`** (`VersionedPromptTemplate`) – The current prompt template to improve. * **`rollout`** (`List[RolloutResultForAPO]`) – List of rollout results to base the critique on. Returns: * `Optional[str]` – The improved prompt text, or the original prompt if gradient computation fails. Back to top --- # APO - Agent-lightning [Skip to content](https://microsoft.github.io/agent-lightning/0.2.0/algorithm-zoo/apo/#apo) ⚠️ You are viewing development documentation. For stable documentation, please visit [https://microsoft.github.io/agent-lightning/stable/](https://microsoft.github.io/agent-lightning/stable/) APO[¶](https://microsoft.github.io/agent-lightning/0.2.0/algorithm-zoo/apo/#apo "Permanent link") ================================================================================================== Shortcut You can use the shortcut `agl.APO(...)` to create an APO instance. `[](https://microsoft.github.io/agent-lightning/0.2.0/algorithm-zoo/apo/#__codelineno-0-1) import agentlightning as agl [](https://microsoft.github.io/agent-lightning/0.2.0/algorithm-zoo/apo/#__codelineno-0-2) [](https://microsoft.github.io/agent-lightning/0.2.0/algorithm-zoo/apo/#__codelineno-0-3) agl.APO(...)` Installation[¶](https://microsoft.github.io/agent-lightning/0.2.0/algorithm-zoo/apo/#installation "Permanent link") -------------------------------------------------------------------------------------------------------------------- `[](https://microsoft.github.io/agent-lightning/0.2.0/algorithm-zoo/apo/#__codelineno-1-1) pip install agentlightning[apo]` Scope of Current Implementation[¶](https://microsoft.github.io/agent-lightning/0.2.0/algorithm-zoo/apo/#scope-of-current-implementation "Permanent link") ---------------------------------------------------------------------------------------------------------------------------------------------------------- APO is currently scoped to optimize a single prompt template. Optimizing multiple prompt templates is not supported yet. There is however no restriction on the number of variable placeholders in the prompt template (can range from zero to many). It's possible that invalid prompts are created during the optimization process. It is up to the agent developer to ensure that the prompt template is valid for the agent's task. Initial Prompt[¶](https://microsoft.github.io/agent-lightning/0.2.0/algorithm-zoo/apo/#initial-prompt "Permanent link") ------------------------------------------------------------------------------------------------------------------------ APO expects the initial prompt to be provided in the `initial_resources` dictionary. This can be done in two approaches: 1. Pass to the [Trainer](https://microsoft.github.io/agent-lightning/0.2.0/reference/trainer/#agentlightning.Trainer " agentlightning.Trainer") constructor: `[](https://microsoft.github.io/agent-lightning/0.2.0/algorithm-zoo/apo/#__codelineno-2-1) trainer = agl.Trainer( [](https://microsoft.github.io/agent-lightning/0.2.0/algorithm-zoo/apo/#__codelineno-2-2) algorithm=agl.APO(...), [](https://microsoft.github.io/agent-lightning/0.2.0/algorithm-zoo/apo/#__codelineno-2-3) initial_resources={"main_prompt": agl.PromptTemplate(template="You are a helpful assistant.", engine="f-string")}, [](https://microsoft.github.io/agent-lightning/0.2.0/algorithm-zoo/apo/#__codelineno-2-4) )` 1. Pass to the `[APO][agentlightning.algorithm.apo.APO].set_initial_resources()` method: `[](https://microsoft.github.io/agent-lightning/0.2.0/algorithm-zoo/apo/#__codelineno-3-1) algo = agl.APO(...) [](https://microsoft.github.io/agent-lightning/0.2.0/algorithm-zoo/apo/#__codelineno-3-2) algo.set_initial_resources( [](https://microsoft.github.io/agent-lightning/0.2.0/algorithm-zoo/apo/#__codelineno-3-3) {"this_is_also_valid_key": agl.PromptTemplate(template="You are a helpful assistant.", engine="f-string")} [](https://microsoft.github.io/agent-lightning/0.2.0/algorithm-zoo/apo/#__codelineno-3-4) )` The resource key can be arbitrary, which is used to identify the prompt template in [class-based implementations](https://microsoft.github.io/agent-lightning/0.2.0/tutorials/write-agents/) when you have multiple resources. When the key changes, the agent developer needs to update the key in the `rollout` method. Tutorials Using APO[¶](https://microsoft.github.io/agent-lightning/0.2.0/algorithm-zoo/apo/#tutorials-using-apo "Permanent link") ---------------------------------------------------------------------------------------------------------------------------------- * [Train the First Agent with APO](https://microsoft.github.io/agent-lightning/0.2.0/how-to/train-first-agent/) - A step-by-step guide to training your first agent using APO. References[¶](https://microsoft.github.io/agent-lightning/0.2.0/algorithm-zoo/apo/#references "Permanent link") ---------------------------------------------------------------------------------------------------------------- ### `agentlightning.algorithm.apo` [¶](https://microsoft.github.io/agent-lightning/0.2.0/algorithm-zoo/apo/#agentlightning.algorithm.apo "Permanent link") #### `APO` [¶](https://microsoft.github.io/agent-lightning/0.2.0/algorithm-zoo/apo/#agentlightning.algorithm.apo.APO "Permanent link") Bases: `[Algorithm](https://microsoft.github.io/agent-lightning/0.2.0/reference/algorithm/#agentlightning.Algorithm " agentlightning.Algorithm (agentlightning.algorithm.base.Algorithm)") `, `Generic[T_task]` Automatic Prompt Optimization (APO) algorithm using textual gradients and beam search. APO is an iterative prompt optimization algorithm that uses LLM-generated textual gradients to improve prompts through a beam search process. It evaluates prompts on rollouts, computes critiques based on the results, and applies edits to generate improved prompts. The algorithm operates in rounds, where each round: 1. Samples parent prompts from the current beam 2. Generates new prompts by computing textual gradients and applying edits 3. Evaluates all candidates on a validation set 4. Selects the top-k prompts for the next round Based on the ideas from: - ProTeGi: https://aclanthology.org/2023.emnlp-main.494.pdf - TextGrad: https://github.com/zou-group/textgrad ##### `__init__(async_openai_client, *, gradient_model='gpt-5-mini', apply_edit_model='gpt-4.1-mini', diversity_temperature=1.0, gradient_batch_size=4, val_batch_size=16, beam_width=4, branch_factor=4, beam_rounds=3, rollout_batch_timeout=3600.0, run_initial_validation=True, _poml_trace=False)` [¶](https://microsoft.github.io/agent-lightning/0.2.0/algorithm-zoo/apo/#agentlightning.algorithm.apo.APO.__init__ "Permanent link") Initialize the APO algorithm with configuration parameters. Parameters: * **`async_openai_client`** (`AsyncOpenAI`) – AsyncOpenAI client for making LLM API calls. * **`gradient_model`** (`str`, default: `'gpt-5-mini'` ) – Model name for computing textual gradients (critiques). * **`apply_edit_model`** (`str`, default: `'gpt-4.1-mini'` ) – Model name for applying edits based on critiques. * **`diversity_temperature`** (`float`, default: `1.0` ) – Temperature parameter for LLM calls to control diversity. * **`gradient_batch_size`** (`int`, default: `4` ) – Number of rollout results to sample for gradient computation. * **`val_batch_size`** (`int`, default: `16` ) – Number of validation examples to use for evaluation. * **`beam_width`** (`int`, default: `4` ) – Number of top-scoring prompts to keep in the beam at each round. * **`branch_factor`** (`int`, default: `4` ) – Number of new prompt candidates to generate from each parent prompt by applying textual gradient edits. This controls the expansion of the search tree. * **`beam_rounds`** (`int`, default: `3` ) – Number of beam search rounds to perform. * **`rollout_batch_timeout`** (`float`, default: `3600.0` ) – Maximum time in seconds to wait for rollout batch completion. * **`run_initial_validation`** (`bool`, default: `True` ) – If True, runs validation on the seed prompt before starting optimization to establish a baseline score. Defaults to True. ##### `compute_textual_gradient(current_prompt, rollout_results, *, prefix=None)` `async` [¶](https://microsoft.github.io/agent-lightning/0.2.0/algorithm-zoo/apo/#agentlightning.algorithm.apo.APO.compute_textual_gradient "Permanent link") Compute a textual gradient (critique) for the current prompt based on rollout results. This method samples rollout results, sends them to an LLM along with the current prompt, and generates a critique describing how the prompt could be improved. Parameters: * **`current_prompt`** (`VersionedPromptTemplate`) – The prompt template to critique. * **`rollout_results`** (`List[RolloutResultForAPO]`) – List of rollout results containing spans, messages, and rewards. Returns: * `Optional[str]` – A textual critique generated by the LLM, or None if generation fails. ##### `evaluate_prompt_on_batch(prompt, resource_name, dataset, mode, *, prefix=None)` `async` [¶](https://microsoft.github.io/agent-lightning/0.2.0/algorithm-zoo/apo/#agentlightning.algorithm.apo.APO.evaluate_prompt_on_batch "Permanent link") Evaluate a prompt on a batch of tasks by running rollouts and computing average reward. This method: 1. Adds the prompt as a named resource to the store 2. Enqueues rollouts for each task in the dataset 3. Waits for rollouts to complete (with timeout) 4. Computes and returns the average reward Parameters: * **`prompt`** (`VersionedPromptTemplate`) – The prompt template string to evaluate. * **`resource_name`** (`str`) – The name to register the prompt under in the store. * **`dataset`** (`Sequence[T_task]`) – Sequence of tasks to evaluate the prompt on. * **`mode`** (`[RolloutMode](https://microsoft.github.io/agent-lightning/0.2.0/reference/types/#agentlightning.RolloutMode " agentlightning.RolloutMode = Literal['train', 'val', 'test'] module-attribute (agentlightning.types.RolloutMode)") `) – Rollout mode ("train" or "val") for logging/tracking. Returns: * `List[RolloutResultForAPO]` – A tuple of (rollout\_results, average\_reward) where rollout\_results contains * `float` – detailed information for each rollout and average\_reward is the mean final reward. ##### `get_adapter()` [¶](https://microsoft.github.io/agent-lightning/0.2.0/algorithm-zoo/apo/#agentlightning.algorithm.apo.APO.get_adapter "Permanent link") Get the adapter for converting spans to messages. Returns: * `[TraceToMessages](https://microsoft.github.io/agent-lightning/0.2.0/reference/algorithm/#agentlightning.TraceToMessages " agentlightning.TraceToMessages (agentlightning.adapter.messages.TraceToMessages)") ` – The TraceToMessages instance for this algorithm. Raises: * `ValueError` – If the adapter is not a TraceToMessages. ##### `get_best_prompt()` [¶](https://microsoft.github.io/agent-lightning/0.2.0/algorithm-zoo/apo/#agentlightning.algorithm.apo.APO.get_best_prompt "Permanent link") Retrieve the best prompt discovered during optimization. Returns: * `[PromptTemplate](https://microsoft.github.io/agent-lightning/0.2.0/reference/types/#agentlightning.PromptTemplate " agentlightning.PromptTemplate (agentlightning.types.PromptTemplate)") ` – The prompt template with the highest validation score found so far. Raises: * `ValueError` – If no best prompt has been found yet (run() not called). ##### `get_rollout_results(rollout, *, prefix=None)` `async` [¶](https://microsoft.github.io/agent-lightning/0.2.0/algorithm-zoo/apo/#agentlightning.algorithm.apo.APO.get_rollout_results "Permanent link") Convert completed rollouts to APO-compatible result format. Fetches spans for each rollout, adapts them to messages, and packages them with rewards and status information for gradient computation. Parameters: * **`rollout`** (`List[[Rollout](https://microsoft.github.io/agent-lightning/0.2.0/reference/types/#agentlightning.Rollout " agentlightning.Rollout (agentlightning.types.Rollout)") ]`) – List of completed rollout metadata. Returns: * `List[RolloutResultForAPO]` – List of rollout results formatted for APO processing. ##### `get_seed_prompt_template()` [¶](https://microsoft.github.io/agent-lightning/0.2.0/algorithm-zoo/apo/#agentlightning.algorithm.apo.APO.get_seed_prompt_template "Permanent link") Extract the initial prompt template from the algorithm's resources. Returns: * `Tuple[str, [PromptTemplate](https://microsoft.github.io/agent-lightning/0.2.0/reference/types/#agentlightning.PromptTemplate " agentlightning.PromptTemplate (agentlightning.types.PromptTemplate)") ]` – A tuple of (resource\_name, prompt\_template) representing the seed prompt. Raises: * `ValueError` – If initial\_resources is not set or no PromptTemplate is found. ##### `run(train_dataset=None, val_dataset=None)` `async` [¶](https://microsoft.github.io/agent-lightning/0.2.0/algorithm-zoo/apo/#agentlightning.algorithm.apo.APO.run "Permanent link") Execute the APO algorithm to optimize prompts through beam search with textual gradients. The algorithm performs iterative prompt optimization over multiple rounds: - Each round: samples parent prompts, generates new candidates via textual gradients, evaluates all candidates on validation data, and keeps the top performers - Tracks the historically best prompt across all rounds - Uses different training data samples for each gradient computation to ensure diversity Parameters: * **`train_dataset`** (`Optional[[Dataset](https://microsoft.github.io/agent-lightning/0.2.0/reference/types/#agentlightning.Dataset " agentlightning.Dataset (agentlightning.types.Dataset)") [T_task]]`, default: `None` ) – Dataset of tasks for computing textual gradients. Required. * **`val_dataset`** (`Optional[[Dataset](https://microsoft.github.io/agent-lightning/0.2.0/reference/types/#agentlightning.Dataset " agentlightning.Dataset (agentlightning.types.Dataset)") [T_task]]`, default: `None` ) – Dataset of tasks for evaluating and selecting prompts. Required. Raises: * `ValueError` – If train\_dataset or val\_dataset is None, or if resources are not set. ##### `textual_gradient_and_apply_edit(current_prompt, rollout, *, prefix=None)` `async` [¶](https://microsoft.github.io/agent-lightning/0.2.0/algorithm-zoo/apo/#agentlightning.algorithm.apo.APO.textual_gradient_and_apply_edit "Permanent link") Generate an improved prompt by computing a textual gradient and applying an edit. This is the main optimization step that: 1. Computes a critique (textual gradient) based on rollout performance 2. Uses another LLM to apply the critique and generate an improved prompt Parameters: * **`current_prompt`** (`VersionedPromptTemplate`) – The current prompt template to improve. * **`rollout`** (`List[RolloutResultForAPO]`) – List of rollout results to base the critique on. Returns: * `Optional[str]` – The improved prompt text, or the original prompt if gradient computation fails. Back to top --- # APO - Agent-lightning [Skip to content](https://microsoft.github.io/agent-lightning/0.2.1/algorithm-zoo/apo/#apo) ⚠️ You are viewing development documentation. For stable documentation, please visit [https://microsoft.github.io/agent-lightning/stable/](https://microsoft.github.io/agent-lightning/stable/) APO[¶](https://microsoft.github.io/agent-lightning/0.2.1/algorithm-zoo/apo/#apo "Permanent link") ================================================================================================== Shortcut You can use the shortcut `agl.APO(...)` to create an APO instance. `[](https://microsoft.github.io/agent-lightning/0.2.1/algorithm-zoo/apo/#__codelineno-0-1) import agentlightning as agl [](https://microsoft.github.io/agent-lightning/0.2.1/algorithm-zoo/apo/#__codelineno-0-2) [](https://microsoft.github.io/agent-lightning/0.2.1/algorithm-zoo/apo/#__codelineno-0-3) agl.APO(...)` Installation[¶](https://microsoft.github.io/agent-lightning/0.2.1/algorithm-zoo/apo/#installation "Permanent link") -------------------------------------------------------------------------------------------------------------------- `[](https://microsoft.github.io/agent-lightning/0.2.1/algorithm-zoo/apo/#__codelineno-1-1) pip install agentlightning[apo]` Scope of Current Implementation[¶](https://microsoft.github.io/agent-lightning/0.2.1/algorithm-zoo/apo/#scope-of-current-implementation "Permanent link") ---------------------------------------------------------------------------------------------------------------------------------------------------------- APO is currently scoped to optimize a single prompt template. Optimizing multiple prompt templates is not supported yet. There is however no restriction on the number of variable placeholders in the prompt template (can range from zero to many). It's possible that invalid prompts are created during the optimization process. It is up to the agent developer to ensure that the prompt template is valid for the agent's task. Initial Prompt[¶](https://microsoft.github.io/agent-lightning/0.2.1/algorithm-zoo/apo/#initial-prompt "Permanent link") ------------------------------------------------------------------------------------------------------------------------ APO expects the initial prompt to be provided in the `initial_resources` dictionary. This can be done in two approaches: 1. Pass to the [Trainer](https://microsoft.github.io/agent-lightning/0.2.1/reference/trainer/#agentlightning.Trainer " agentlightning.Trainer") constructor: `[](https://microsoft.github.io/agent-lightning/0.2.1/algorithm-zoo/apo/#__codelineno-2-1) trainer = agl.Trainer( [](https://microsoft.github.io/agent-lightning/0.2.1/algorithm-zoo/apo/#__codelineno-2-2) algorithm=agl.APO(...), [](https://microsoft.github.io/agent-lightning/0.2.1/algorithm-zoo/apo/#__codelineno-2-3) initial_resources={"main_prompt": agl.PromptTemplate(template="You are a helpful assistant.", engine="f-string")}, [](https://microsoft.github.io/agent-lightning/0.2.1/algorithm-zoo/apo/#__codelineno-2-4) )` 1. Pass to the `[APO][agentlightning.algorithm.apo.APO].set_initial_resources()` method: `[](https://microsoft.github.io/agent-lightning/0.2.1/algorithm-zoo/apo/#__codelineno-3-1) algo = agl.APO(...) [](https://microsoft.github.io/agent-lightning/0.2.1/algorithm-zoo/apo/#__codelineno-3-2) algo.set_initial_resources( [](https://microsoft.github.io/agent-lightning/0.2.1/algorithm-zoo/apo/#__codelineno-3-3) {"this_is_also_valid_key": agl.PromptTemplate(template="You are a helpful assistant.", engine="f-string")} [](https://microsoft.github.io/agent-lightning/0.2.1/algorithm-zoo/apo/#__codelineno-3-4) )` The resource key can be arbitrary, which is used to identify the prompt template in [class-based implementations](https://microsoft.github.io/agent-lightning/0.2.1/tutorials/write-agents/) when you have multiple resources. When the key changes, the agent developer needs to update the key in the `rollout` method. Tutorials Using APO[¶](https://microsoft.github.io/agent-lightning/0.2.1/algorithm-zoo/apo/#tutorials-using-apo "Permanent link") ---------------------------------------------------------------------------------------------------------------------------------- * [Train the First Agent with APO](https://microsoft.github.io/agent-lightning/0.2.1/how-to/train-first-agent/) - A step-by-step guide to training your first agent using APO. References[¶](https://microsoft.github.io/agent-lightning/0.2.1/algorithm-zoo/apo/#references "Permanent link") ---------------------------------------------------------------------------------------------------------------- ### `agentlightning.algorithm.apo` [¶](https://microsoft.github.io/agent-lightning/0.2.1/algorithm-zoo/apo/#agentlightning.algorithm.apo "Permanent link") #### `APO` [¶](https://microsoft.github.io/agent-lightning/0.2.1/algorithm-zoo/apo/#agentlightning.algorithm.apo.APO "Permanent link") Bases: `[Algorithm](https://microsoft.github.io/agent-lightning/0.2.1/reference/algorithm/#agentlightning.Algorithm " agentlightning.Algorithm (agentlightning.algorithm.base.Algorithm)") `, `Generic[T_task]` Automatic Prompt Optimization (APO) algorithm using textual gradients and beam search. APO is an iterative prompt optimization algorithm that uses LLM-generated textual gradients to improve prompts through a beam search process. It evaluates prompts on rollouts, computes critiques based on the results, and applies edits to generate improved prompts. The algorithm operates in rounds, where each round: 1. Samples parent prompts from the current beam 2. Generates new prompts by computing textual gradients and applying edits 3. Evaluates all candidates on a validation set 4. Selects the top-k prompts for the next round Based on the ideas from: * [ProTeGi](https://aclanthology.org/2023.emnlp-main.494.pdf) * [TextGrad](https://github.com/zou-group/textgrad) ##### `__init__(async_openai_client, *, gradient_model='gpt-5-mini', apply_edit_model='gpt-4.1-mini', diversity_temperature=1.0, gradient_batch_size=4, val_batch_size=16, beam_width=4, branch_factor=4, beam_rounds=3, rollout_batch_timeout=3600.0, run_initial_validation=True, _poml_trace=False)` [¶](https://microsoft.github.io/agent-lightning/0.2.1/algorithm-zoo/apo/#agentlightning.algorithm.apo.APO.__init__ "Permanent link") Initialize the APO algorithm with configuration parameters. Parameters: * **`async_openai_client`** (`AsyncOpenAI`) – AsyncOpenAI client for making LLM API calls. * **`gradient_model`** (`str`, default: `'gpt-5-mini'` ) – Model name for computing textual gradients (critiques). * **`apply_edit_model`** (`str`, default: `'gpt-4.1-mini'` ) – Model name for applying edits based on critiques. * **`diversity_temperature`** (`float`, default: `1.0` ) – Temperature parameter for LLM calls to control diversity. * **`gradient_batch_size`** (`int`, default: `4` ) – Number of rollout results to sample for gradient computation. * **`val_batch_size`** (`int`, default: `16` ) – Number of validation examples to use for evaluation. * **`beam_width`** (`int`, default: `4` ) – Number of top-scoring prompts to keep in the beam at each round. * **`branch_factor`** (`int`, default: `4` ) – Number of new prompt candidates to generate from each parent prompt by applying textual gradient edits. This controls the expansion of the search tree. * **`beam_rounds`** (`int`, default: `3` ) – Number of beam search rounds to perform. * **`rollout_batch_timeout`** (`float`, default: `3600.0` ) – Maximum time in seconds to wait for rollout batch completion. * **`run_initial_validation`** (`bool`, default: `True` ) – If True, runs validation on the seed prompt before starting optimization to establish a baseline score. Defaults to True. ##### `compute_textual_gradient(current_prompt, rollout_results, *, prefix=None)` `async` [¶](https://microsoft.github.io/agent-lightning/0.2.1/algorithm-zoo/apo/#agentlightning.algorithm.apo.APO.compute_textual_gradient "Permanent link") Compute a textual gradient (critique) for the current prompt based on rollout results. This method samples rollout results, sends them to an LLM along with the current prompt, and generates a critique describing how the prompt could be improved. Parameters: * **`current_prompt`** (`VersionedPromptTemplate`) – The prompt template to critique. * **`rollout_results`** (`List[RolloutResultForAPO]`) – List of rollout results containing spans, messages, and rewards. Returns: * `Optional[str]` – A textual critique generated by the LLM, or None if generation fails. ##### `evaluate_prompt_on_batch(prompt, resource_name, dataset, mode, *, prefix=None)` `async` [¶](https://microsoft.github.io/agent-lightning/0.2.1/algorithm-zoo/apo/#agentlightning.algorithm.apo.APO.evaluate_prompt_on_batch "Permanent link") Evaluate a prompt on a batch of tasks by running rollouts and computing average reward. This method: 1. Adds the prompt as a named resource to the store 2. Enqueues rollouts for each task in the dataset 3. Waits for rollouts to complete (with timeout) 4. Computes and returns the average reward Parameters: * **`prompt`** (`VersionedPromptTemplate`) – The prompt template string to evaluate. * **`resource_name`** (`str`) – The name to register the prompt under in the store. * **`dataset`** (`Sequence[T_task]`) – Sequence of tasks to evaluate the prompt on. * **`mode`** (`[RolloutMode](https://microsoft.github.io/agent-lightning/0.2.1/reference/types/#agentlightning.RolloutMode " agentlightning.RolloutMode = Literal['train', 'val', 'test'] module-attribute (agentlightning.types.RolloutMode)") `) – Rollout mode ("train" or "val") for logging/tracking. Returns: * `List[RolloutResultForAPO]` – A tuple of (rollout\_results, average\_reward) where rollout\_results contains * `float` – detailed information for each rollout and average\_reward is the mean final reward. ##### `get_adapter()` [¶](https://microsoft.github.io/agent-lightning/0.2.1/algorithm-zoo/apo/#agentlightning.algorithm.apo.APO.get_adapter "Permanent link") Get the adapter for converting spans to messages. Returns: * `[TraceToMessages](https://microsoft.github.io/agent-lightning/0.2.1/reference/algorithm/#agentlightning.TraceToMessages " agentlightning.TraceToMessages (agentlightning.adapter.messages.TraceToMessages)") ` – The TraceToMessages instance for this algorithm. Raises: * `ValueError` – If the adapter is not a TraceToMessages. ##### `get_best_prompt()` [¶](https://microsoft.github.io/agent-lightning/0.2.1/algorithm-zoo/apo/#agentlightning.algorithm.apo.APO.get_best_prompt "Permanent link") Retrieve the best prompt discovered during optimization. Returns: * `[PromptTemplate](https://microsoft.github.io/agent-lightning/0.2.1/reference/types/#agentlightning.PromptTemplate " agentlightning.PromptTemplate (agentlightning.types.PromptTemplate)") ` – The prompt template with the highest validation score found so far. Raises: * `ValueError` – If no best prompt has been found yet (run() not called). ##### `get_rollout_results(rollout, *, prefix=None)` `async` [¶](https://microsoft.github.io/agent-lightning/0.2.1/algorithm-zoo/apo/#agentlightning.algorithm.apo.APO.get_rollout_results "Permanent link") Convert completed rollouts to APO-compatible result format. Fetches spans for each rollout, adapts them to messages, and packages them with rewards and status information for gradient computation. Parameters: * **`rollout`** (`List[[Rollout](https://microsoft.github.io/agent-lightning/0.2.1/reference/types/#agentlightning.Rollout " agentlightning.Rollout (agentlightning.types.Rollout)") ]`) – List of completed rollout metadata. Returns: * `List[RolloutResultForAPO]` – List of rollout results formatted for APO processing. ##### `get_seed_prompt_template()` [¶](https://microsoft.github.io/agent-lightning/0.2.1/algorithm-zoo/apo/#agentlightning.algorithm.apo.APO.get_seed_prompt_template "Permanent link") Extract the initial prompt template from the algorithm's resources. Returns: * `Tuple[str, [PromptTemplate](https://microsoft.github.io/agent-lightning/0.2.1/reference/types/#agentlightning.PromptTemplate " agentlightning.PromptTemplate (agentlightning.types.PromptTemplate)") ]` – A tuple of (resource\_name, prompt\_template) representing the seed prompt. Raises: * `ValueError` – If initial\_resources is not set or no PromptTemplate is found. ##### `run(train_dataset=None, val_dataset=None)` `async` [¶](https://microsoft.github.io/agent-lightning/0.2.1/algorithm-zoo/apo/#agentlightning.algorithm.apo.APO.run "Permanent link") Execute the APO algorithm to optimize prompts through beam search with textual gradients. The algorithm performs iterative prompt optimization over multiple rounds: * Each round: samples parent prompts, generates new candidates via textual gradients, evaluates all candidates on validation data, and keeps the top performers * Tracks the historically best prompt across all rounds * Uses different training data samples for each gradient computation to ensure diversity Parameters: * **`train_dataset`** (`Optional[[Dataset](https://microsoft.github.io/agent-lightning/0.2.1/reference/types/#agentlightning.Dataset " agentlightning.Dataset (agentlightning.types.Dataset)") [T_task]]`, default: `None` ) – Dataset of tasks for computing textual gradients. Required. * **`val_dataset`** (`Optional[[Dataset](https://microsoft.github.io/agent-lightning/0.2.1/reference/types/#agentlightning.Dataset " agentlightning.Dataset (agentlightning.types.Dataset)") [T_task]]`, default: `None` ) – Dataset of tasks for evaluating and selecting prompts. Required. Raises: * `ValueError` – If train\_dataset or val\_dataset is None, or if resources are not set. ##### `textual_gradient_and_apply_edit(current_prompt, rollout, *, prefix=None)` `async` [¶](https://microsoft.github.io/agent-lightning/0.2.1/algorithm-zoo/apo/#agentlightning.algorithm.apo.APO.textual_gradient_and_apply_edit "Permanent link") Generate an improved prompt by computing a textual gradient and applying an edit. This is the main optimization step that: 1. Computes a critique (textual gradient) based on rollout performance 2. Uses another LLM to apply the critique and generate an improved prompt Parameters: * **`current_prompt`** (`VersionedPromptTemplate`) – The current prompt template to improve. * **`rollout`** (`List[RolloutResultForAPO]`) – List of rollout results to base the critique on. Returns: * `Optional[str]` – The improved prompt text, or the original prompt if gradient computation fails. Back to top --- # VERL - Agent-lightning [Skip to content](https://microsoft.github.io/agent-lightning/0.2.2/algorithm-zoo/verl/#verl) ⚠️ You are viewing development documentation. For stable documentation, please visit [https://microsoft.github.io/agent-lightning/stable/](https://microsoft.github.io/agent-lightning/stable/) VERL[¶](https://microsoft.github.io/agent-lightning/0.2.2/algorithm-zoo/verl/#verl "Permanent link") ===================================================================================================== Shortcut You can use the shortcut `agl.VERL(...)` to create a VERL instance. `[](https://microsoft.github.io/agent-lightning/0.2.2/algorithm-zoo/verl/#__codelineno-0-1) import agentlightning as agl [](https://microsoft.github.io/agent-lightning/0.2.2/algorithm-zoo/verl/#__codelineno-0-2) [](https://microsoft.github.io/agent-lightning/0.2.2/algorithm-zoo/verl/#__codelineno-0-3) agl.VERL(...)` Installation[¶](https://microsoft.github.io/agent-lightning/0.2.2/algorithm-zoo/verl/#installation "Permanent link") --------------------------------------------------------------------------------------------------------------------- `[](https://microsoft.github.io/agent-lightning/0.2.2/algorithm-zoo/verl/#__codelineno-1-1) pip install agentlightning[verl]` Warning To avoid various compatibility issues, follow the steps in the [installation guide](https://microsoft.github.io/agent-lightning/0.2.2/tutorials/installation/) to set up VERL and its dependencies. Installing VERL directly with `pip install agentlightning[verl]` can cause issues unless you already have a compatible version of PyTorch installed. Notes for Readers [VERL](https://microsoft.github.io/agent-lightning/0.2.2/algorithm-zoo/verl/#agentlightning.algorithm.verl.VERL " VERL") in this article refers to a wrapper, provided by Agent-lightning, of the [VERL framework](https://github.com/volcengine/verl) . It's a subclass of [agentlightning.Algorithm](https://microsoft.github.io/agent-lightning/0.2.2/reference/algorithm/#agentlightning.Algorithm " agentlightning.Algorithm") . To differentiate it from the VERL framework, all references to the VERL framework shall use the term "VERL framework", and all references to the Agent-lightning wrapper shall be highlighted with a link. Resources[¶](https://microsoft.github.io/agent-lightning/0.2.2/algorithm-zoo/verl/#resources "Permanent link") --------------------------------------------------------------------------------------------------------------- [VERL](https://microsoft.github.io/agent-lightning/0.2.2/algorithm-zoo/verl/#agentlightning.algorithm.verl.VERL " VERL") expects no initial resources. The first LLM endpoint is directly deployed from the VERL configuration (`.actor_rollout_ref.model.path`). The resource key is always `main_llm`. [VERL](https://microsoft.github.io/agent-lightning/0.2.2/algorithm-zoo/verl/#agentlightning.algorithm.verl.VERL " VERL") currently does not support optimizing multiple [LLM](https://microsoft.github.io/agent-lightning/0.2.2/reference/types/#agentlightning.LLM " agentlightning.LLM") s together. Note The resource type created by [VERL](https://microsoft.github.io/agent-lightning/0.2.2/algorithm-zoo/verl/#agentlightning.algorithm.verl.VERL " VERL") is actually a [ProxyLLM](https://microsoft.github.io/agent-lightning/0.2.2/reference/types/#agentlightning.ProxyLLM " agentlightning.ProxyLLM") , a subclass of the [LLM](https://microsoft.github.io/agent-lightning/0.2.2/reference/types/#agentlightning.LLM " agentlightning.LLM") type. This object contains a **URL template** provided by [VERL](https://microsoft.github.io/agent-lightning/0.2.2/algorithm-zoo/verl/#agentlightning.algorithm.verl.VERL " VERL") , with placeholders for rollout and attempt IDs. When a rollout begins on the agent side, the framework uses the current `rollout_id` and `attempt_id` to format this template, generating a final, unique endpoint URL. This URL points to [VERL](https://microsoft.github.io/agent-lightning/0.2.2/algorithm-zoo/verl/#agentlightning.algorithm.verl.VERL " VERL") 's internal proxy, allowing it to intercept and log all traffic for that specific attempt, for tracing and load balancing purposes. For agents created with the `@rollout` decorator, this resolution of the template is handled automatically ("auto-stripped"). Class-based agents will need to manually resolve the [`ProxyLLM`](https://microsoft.github.io/agent-lightning/0.2.2/reference/types/#agentlightning.ProxyLLM " agentlightning.ProxyLLM") using the rollout context. `[](https://microsoft.github.io/agent-lightning/0.2.2/algorithm-zoo/verl/#__codelineno-2-1) proxy_llm = resources["main_llm"] [](https://microsoft.github.io/agent-lightning/0.2.2/algorithm-zoo/verl/#__codelineno-2-2) proxy_llm.get_base_url(rollout.rollout_id, rollout.attempt.attempt_id)` Customization[¶](https://microsoft.github.io/agent-lightning/0.2.2/algorithm-zoo/verl/#customization "Permanent link") ----------------------------------------------------------------------------------------------------------------------- Internally, [VERL](https://microsoft.github.io/agent-lightning/0.2.2/algorithm-zoo/verl/#agentlightning.algorithm.verl.VERL " VERL") decomposes each agent execution into prompt–response pairs via the [Adapter](https://microsoft.github.io/agent-lightning/0.2.2/reference/algorithm/#agentlightning.Adapter " agentlightning.Adapter") and associates them with their corresponding reward signals as [Triplet](https://microsoft.github.io/agent-lightning/0.2.2/reference/types/#agentlightning.Triplet " agentlightning.Triplet") objects. The final scalar reward, derived from the last triplet in the trajectory, is propagated to all preceding triplets following the [identical assignment strategy](https://arxiv.org/abs/2508.03680) . This ensures that each triplet receives an identical reward signal and can be independently optimized as a valid RLHF trajectory within the VERL framework. At present, [VERL](https://microsoft.github.io/agent-lightning/0.2.2/algorithm-zoo/verl/#agentlightning.algorithm.verl.VERL " VERL") does not expose fine-grained control over its reward propagation or credit assignment mechanisms. Users requiring customized reward shaping or trajectory decomposition are advised to clone and modify the [VERL](https://microsoft.github.io/agent-lightning/0.2.2/algorithm-zoo/verl/#agentlightning.algorithm.verl.VERL " VERL") source implementation directly. Tutorials Using VERL[¶](https://microsoft.github.io/agent-lightning/0.2.2/algorithm-zoo/verl/#tutorials-using-verl "Permanent link") ------------------------------------------------------------------------------------------------------------------------------------- * [Train SQL Agent with RL](https://microsoft.github.io/agent-lightning/0.2.2/how-to/train-sql-agent/) - A practical example of training a SQL agent using VERL. References - Entrypoint[¶](https://microsoft.github.io/agent-lightning/0.2.2/algorithm-zoo/verl/#references-entrypoint "Permanent link") ----------------------------------------------------------------------------------------------------------------------------------------- ### `agentlightning.algorithm.verl` [¶](https://microsoft.github.io/agent-lightning/0.2.2/algorithm-zoo/verl/#agentlightning.algorithm.verl "Permanent link") #### `VERL` [¶](https://microsoft.github.io/agent-lightning/0.2.2/algorithm-zoo/verl/#agentlightning.algorithm.verl.VERL "Permanent link") Bases: `[Algorithm](https://microsoft.github.io/agent-lightning/0.2.2/reference/algorithm/#agentlightning.Algorithm " agentlightning.Algorithm (agentlightning.algorithm.base.Algorithm)") ` VERL-powered algorithm that delegates training to the VERL PPO runner. Warning Advanced customisation currently requires copying the VERL source and modifying it directly. Native hooks for overriding training behaviour will land in a future release. Parameters: * **`config`** (`dict[str, Any]`) – Dictionary mirroring the overrides passed to the VERL CLI. The overrides are merged with VERL's packaged defaults via Hydra before launching training. Examples: `[](https://microsoft.github.io/agent-lightning/0.2.2/algorithm-zoo/verl/#__codelineno-0-1) from agentlightning.algorithm.verl import VERL [](https://microsoft.github.io/agent-lightning/0.2.2/algorithm-zoo/verl/#__codelineno-0-2) [](https://microsoft.github.io/agent-lightning/0.2.2/algorithm-zoo/verl/#__codelineno-0-3) algorithm = VERL( [](https://microsoft.github.io/agent-lightning/0.2.2/algorithm-zoo/verl/#__codelineno-0-4) config={ [](https://microsoft.github.io/agent-lightning/0.2.2/algorithm-zoo/verl/#__codelineno-0-5) "algorithm": { [](https://microsoft.github.io/agent-lightning/0.2.2/algorithm-zoo/verl/#__codelineno-0-6) "adv_estimator": "grpo", [](https://microsoft.github.io/agent-lightning/0.2.2/algorithm-zoo/verl/#__codelineno-0-7) "use_kl_in_reward": False, [](https://microsoft.github.io/agent-lightning/0.2.2/algorithm-zoo/verl/#__codelineno-0-8) }, [](https://microsoft.github.io/agent-lightning/0.2.2/algorithm-zoo/verl/#__codelineno-0-9) "data": { [](https://microsoft.github.io/agent-lightning/0.2.2/algorithm-zoo/verl/#__codelineno-0-10) "train_batch_size": 32, [](https://microsoft.github.io/agent-lightning/0.2.2/algorithm-zoo/verl/#__codelineno-0-11) "max_prompt_length": 4096, [](https://microsoft.github.io/agent-lightning/0.2.2/algorithm-zoo/verl/#__codelineno-0-12) "max_response_length": 2048, [](https://microsoft.github.io/agent-lightning/0.2.2/algorithm-zoo/verl/#__codelineno-0-13) }, [](https://microsoft.github.io/agent-lightning/0.2.2/algorithm-zoo/verl/#__codelineno-0-14) "actor_rollout_ref": { [](https://microsoft.github.io/agent-lightning/0.2.2/algorithm-zoo/verl/#__codelineno-0-15) "rollout": { [](https://microsoft.github.io/agent-lightning/0.2.2/algorithm-zoo/verl/#__codelineno-0-16) "tensor_model_parallel_size": 1, [](https://microsoft.github.io/agent-lightning/0.2.2/algorithm-zoo/verl/#__codelineno-0-17) "n": 4, [](https://microsoft.github.io/agent-lightning/0.2.2/algorithm-zoo/verl/#__codelineno-0-18) "log_prob_micro_batch_size_per_gpu": 4, [](https://microsoft.github.io/agent-lightning/0.2.2/algorithm-zoo/verl/#__codelineno-0-19) "multi_turn": {"format": "hermes"}, [](https://microsoft.github.io/agent-lightning/0.2.2/algorithm-zoo/verl/#__codelineno-0-20) "name": "vllm", [](https://microsoft.github.io/agent-lightning/0.2.2/algorithm-zoo/verl/#__codelineno-0-21) "gpu_memory_utilization": 0.6, [](https://microsoft.github.io/agent-lightning/0.2.2/algorithm-zoo/verl/#__codelineno-0-22) }, [](https://microsoft.github.io/agent-lightning/0.2.2/algorithm-zoo/verl/#__codelineno-0-23) "actor": { [](https://microsoft.github.io/agent-lightning/0.2.2/algorithm-zoo/verl/#__codelineno-0-24) "ppo_mini_batch_size": 32, [](https://microsoft.github.io/agent-lightning/0.2.2/algorithm-zoo/verl/#__codelineno-0-25) "ppo_micro_batch_size_per_gpu": 4, [](https://microsoft.github.io/agent-lightning/0.2.2/algorithm-zoo/verl/#__codelineno-0-26) "optim": {"lr": 1e-6}, [](https://microsoft.github.io/agent-lightning/0.2.2/algorithm-zoo/verl/#__codelineno-0-27) "use_kl_loss": False, [](https://microsoft.github.io/agent-lightning/0.2.2/algorithm-zoo/verl/#__codelineno-0-28) "kl_loss_coef": 0.0, [](https://microsoft.github.io/agent-lightning/0.2.2/algorithm-zoo/verl/#__codelineno-0-29) "entropy_coeff": 0, [](https://microsoft.github.io/agent-lightning/0.2.2/algorithm-zoo/verl/#__codelineno-0-30) "clip_ratio_low": 0.2, [](https://microsoft.github.io/agent-lightning/0.2.2/algorithm-zoo/verl/#__codelineno-0-31) "clip_ratio_high": 0.3, [](https://microsoft.github.io/agent-lightning/0.2.2/algorithm-zoo/verl/#__codelineno-0-32) "fsdp_config": { [](https://microsoft.github.io/agent-lightning/0.2.2/algorithm-zoo/verl/#__codelineno-0-33) "param_offload": True, [](https://microsoft.github.io/agent-lightning/0.2.2/algorithm-zoo/verl/#__codelineno-0-34) "optimizer_offload": True, [](https://microsoft.github.io/agent-lightning/0.2.2/algorithm-zoo/verl/#__codelineno-0-35) }, [](https://microsoft.github.io/agent-lightning/0.2.2/algorithm-zoo/verl/#__codelineno-0-36) }, [](https://microsoft.github.io/agent-lightning/0.2.2/algorithm-zoo/verl/#__codelineno-0-37) "ref": { [](https://microsoft.github.io/agent-lightning/0.2.2/algorithm-zoo/verl/#__codelineno-0-38) "log_prob_micro_batch_size_per_gpu": 8, [](https://microsoft.github.io/agent-lightning/0.2.2/algorithm-zoo/verl/#__codelineno-0-39) "fsdp_config": {"param_offload": True}, [](https://microsoft.github.io/agent-lightning/0.2.2/algorithm-zoo/verl/#__codelineno-0-40) }, [](https://microsoft.github.io/agent-lightning/0.2.2/algorithm-zoo/verl/#__codelineno-0-41) "model": { [](https://microsoft.github.io/agent-lightning/0.2.2/algorithm-zoo/verl/#__codelineno-0-42) "path": "Qwen/Qwen2.5-1.5B-Instruct", [](https://microsoft.github.io/agent-lightning/0.2.2/algorithm-zoo/verl/#__codelineno-0-43) "use_remove_padding": True, [](https://microsoft.github.io/agent-lightning/0.2.2/algorithm-zoo/verl/#__codelineno-0-44) "enable_gradient_checkpointing": True, [](https://microsoft.github.io/agent-lightning/0.2.2/algorithm-zoo/verl/#__codelineno-0-45) }, [](https://microsoft.github.io/agent-lightning/0.2.2/algorithm-zoo/verl/#__codelineno-0-46) }, [](https://microsoft.github.io/agent-lightning/0.2.2/algorithm-zoo/verl/#__codelineno-0-47) "trainer": { [](https://microsoft.github.io/agent-lightning/0.2.2/algorithm-zoo/verl/#__codelineno-0-48) "n_gpus_per_node": 1, [](https://microsoft.github.io/agent-lightning/0.2.2/algorithm-zoo/verl/#__codelineno-0-49) "val_before_train": True, [](https://microsoft.github.io/agent-lightning/0.2.2/algorithm-zoo/verl/#__codelineno-0-50) "critic_warmup": 0, [](https://microsoft.github.io/agent-lightning/0.2.2/algorithm-zoo/verl/#__codelineno-0-51) "logger": ["console", "wandb"], [](https://microsoft.github.io/agent-lightning/0.2.2/algorithm-zoo/verl/#__codelineno-0-52) "project_name": "AgentLightning", [](https://microsoft.github.io/agent-lightning/0.2.2/algorithm-zoo/verl/#__codelineno-0-53) "experiment_name": "calc_x", [](https://microsoft.github.io/agent-lightning/0.2.2/algorithm-zoo/verl/#__codelineno-0-54) "nnodes": 1, [](https://microsoft.github.io/agent-lightning/0.2.2/algorithm-zoo/verl/#__codelineno-0-55) "save_freq": 64, [](https://microsoft.github.io/agent-lightning/0.2.2/algorithm-zoo/verl/#__codelineno-0-56) "test_freq": 32, [](https://microsoft.github.io/agent-lightning/0.2.2/algorithm-zoo/verl/#__codelineno-0-57) "total_epochs": 2, [](https://microsoft.github.io/agent-lightning/0.2.2/algorithm-zoo/verl/#__codelineno-0-58) }, [](https://microsoft.github.io/agent-lightning/0.2.2/algorithm-zoo/verl/#__codelineno-0-59) } [](https://microsoft.github.io/agent-lightning/0.2.2/algorithm-zoo/verl/#__codelineno-0-60) ) [](https://microsoft.github.io/agent-lightning/0.2.2/algorithm-zoo/verl/#__codelineno-0-61) trainer.fit(algorithm, train_dataset=my_train_dataset)` ##### `get_client()` [¶](https://microsoft.github.io/agent-lightning/0.2.2/algorithm-zoo/verl/#agentlightning.algorithm.verl.VERL.get_client "Permanent link") Create a client bound to the VERL-managed Agent Lightning server. Deprecated Since v0.2. ##### `run(train_dataset=None, val_dataset=None)` [¶](https://microsoft.github.io/agent-lightning/0.2.2/algorithm-zoo/verl/#agentlightning.algorithm.verl.VERL.run "Permanent link") Launch the VERL PPO entrypoint with the configured runtime context. Parameters: * **`train_dataset`** (`Optional[[Dataset](https://microsoft.github.io/agent-lightning/0.2.2/reference/types/#agentlightning.Dataset " agentlightning.Dataset (agentlightning.types.Dataset)") [Any]]`, default: `None` ) – Optional dataset forwarded to VERL for training. * **`val_dataset`** (`Optional[[Dataset](https://microsoft.github.io/agent-lightning/0.2.2/reference/types/#agentlightning.Dataset " agentlightning.Dataset (agentlightning.types.Dataset)") [Any]]`, default: `None` ) – Optional dataset forwarded to VERL for evaluation. Raises: * `ValueError` – If required dependencies such as the store, LLM proxy, or adapter have been garbage-collected when using the V1 execution mode. References - Implementation[¶](https://microsoft.github.io/agent-lightning/0.2.2/algorithm-zoo/verl/#references-implementation "Permanent link") ------------------------------------------------------------------------------------------------------------------------------------------------- ### `agentlightning.verl` [¶](https://microsoft.github.io/agent-lightning/0.2.2/algorithm-zoo/verl/#agentlightning.verl "Permanent link") This package contains a _hacky_ integration of VERL with Agent Lightning. #### `AgentLightningTrainer` [¶](https://microsoft.github.io/agent-lightning/0.2.2/algorithm-zoo/verl/#agentlightning.verl.AgentLightningTrainer "Permanent link") Bases: `RayPPOTrainer` Specialized PPO trainer for agent-based reinforcement learning. This trainer is designed specifically for scenarios where the model interacts with external environments, tools, or APIs through an AgentLightningServer. It simplifies the training loop by removing the complex conditional logic present in the original RayPPOTrainer and focusing on the agent mode workflow. Key differences from RayPPOTrainer: 1. Uses AgentModeDaemon for server communication 2. Simplified data flow without pop/union operations 3. Direct batch processing through agent daemon 4. Streamlined validation using agent\_mode validation #### `AgentModeDaemon` [¶](https://microsoft.github.io/agent-lightning/0.2.2/algorithm-zoo/verl/#agentlightning.verl.AgentModeDaemon "Permanent link") AgentModeDaemon using the AgentLightningServer SDK. This class manages the server lifecycle, task queueing, and results retrieval, while also running a proxy server for LLM requests. It maintains the original interface for compatibility with the RayPPOTrainer. ##### `clear_data_and_server()` [¶](https://microsoft.github.io/agent-lightning/0.2.2/algorithm-zoo/verl/#agentlightning.verl.AgentModeDaemon.clear_data_and_server "Permanent link") Resets the internal state of the daemon for the next run. ##### `get_test_metrics()` [¶](https://microsoft.github.io/agent-lightning/0.2.2/algorithm-zoo/verl/#agentlightning.verl.AgentModeDaemon.get_test_metrics "Permanent link") Calculates and returns metrics for a validation run. ##### `get_train_data_batch(max_prompt_length, max_response_length, device)` [¶](https://microsoft.github.io/agent-lightning/0.2.2/algorithm-zoo/verl/#agentlightning.verl.AgentModeDaemon.get_train_data_batch "Permanent link") Processes completed rollouts to generate a training data batch. This function reconstructs the logic from the original AgentModeDaemon, using data retrieved from the new server architecture. It handles padding, truncation, and tensor creation for the PPO training loop. ##### `run_until_all_finished(verbose=True)` [¶](https://microsoft.github.io/agent-lightning/0.2.2/algorithm-zoo/verl/#agentlightning.verl.AgentModeDaemon.run_until_all_finished "Permanent link") Synchronously waits for all queued tasks to be completed and reported. ##### `set_up_data_and_server(data, server_addresses, is_train=True)` [¶](https://microsoft.github.io/agent-lightning/0.2.2/algorithm-zoo/verl/#agentlightning.verl.AgentModeDaemon.set_up_data_and_server "Permanent link") Synchronous wrapper for setting up data and server resources. ##### `start()` [¶](https://microsoft.github.io/agent-lightning/0.2.2/algorithm-zoo/verl/#agentlightning.verl.AgentModeDaemon.start "Permanent link") Starts the main AgentLightningServer and the proxy server. #### `get_left_padded_ids_and_attention_mask(ids, max_length, pad_token_id)` [¶](https://microsoft.github.io/agent-lightning/0.2.2/algorithm-zoo/verl/#agentlightning.verl.get_left_padded_ids_and_attention_mask "Permanent link") Left-pad (or truncate) a sequence of token IDs to a fixed length, and build the corresponding attention mask. Parameters: * **`ids`** (`List[int]`) – the original list of token IDs. * **`max_length`** (`int`) – desired total length after padding/truncation. * **`pad_token_id`** (`int`) – ID to use for padding. Returns: * **`padded_ids`** ( `any` ) – list of length == max\_length. * **`attention_mask`** ( `any` ) – list of same length: 1 for non-pad tokens, 0 for pads. #### `get_right_padded_ids_and_attention_mask(ids, max_length, pad_token_id)` [¶](https://microsoft.github.io/agent-lightning/0.2.2/algorithm-zoo/verl/#agentlightning.verl.get_right_padded_ids_and_attention_mask "Permanent link") Right-pad (or truncate) a sequence of token IDs to a fixed length, and build the corresponding attention mask. Parameters: * **`ids`** (`List[int]`) – the original list of token IDs. * **`max_length`** (`int`) – desired total length after padding/truncation. * **`pad_token_id`** (`int`) – ID to use for padding. Returns: * **`padded_ids`** ( `any` ) – list of length == max\_length. * **`attention_mask`** ( `any` ) – list of same length: 1 for non-pad tokens, 0 for pads. Back to top --- # VERL - Agent-lightning [Skip to content](https://microsoft.github.io/agent-lightning/0.2.0/algorithm-zoo/verl/#verl) ⚠️ You are viewing development documentation. For stable documentation, please visit [https://microsoft.github.io/agent-lightning/stable/](https://microsoft.github.io/agent-lightning/stable/) VERL[¶](https://microsoft.github.io/agent-lightning/0.2.0/algorithm-zoo/verl/#verl "Permanent link") ===================================================================================================== Shortcut You can use the shortcut `agl.VERL(...)` to create a VERL instance. `[](https://microsoft.github.io/agent-lightning/0.2.0/algorithm-zoo/verl/#__codelineno-0-1) import agentlightning as agl [](https://microsoft.github.io/agent-lightning/0.2.0/algorithm-zoo/verl/#__codelineno-0-2) [](https://microsoft.github.io/agent-lightning/0.2.0/algorithm-zoo/verl/#__codelineno-0-3) agl.VERL(...)` Installation[¶](https://microsoft.github.io/agent-lightning/0.2.0/algorithm-zoo/verl/#installation "Permanent link") --------------------------------------------------------------------------------------------------------------------- `[](https://microsoft.github.io/agent-lightning/0.2.0/algorithm-zoo/verl/#__codelineno-1-1) pip install agentlightning[verl]` Warning To avoid various compatibility issues, follow the steps in the [installation guide](https://microsoft.github.io/agent-lightning/0.2.0/tutorials/installation/) to set up VERL and its dependencies. Installing VERL directly with `pip install agentlightning[verl]` can cause issues unless you already have a compatible version of PyTorch installed. Notes for Readers [VERL](https://microsoft.github.io/agent-lightning/0.2.0/algorithm-zoo/verl/#agentlightning.algorithm.verl.VERL " VERL") in this article refers to a wrapper, provided by Agent-lightning, of the VERL framework. It's a subclass of [agentlightning.Algorithm](https://microsoft.github.io/agent-lightning/0.2.0/reference/algorithm/#agentlightning.Algorithm " agentlightning.Algorithm") . To differentiate it from the VERL framework, all references to the VERL framework shall use the term "VERL framework", and all references to the Agent-lightning wrapper shall be highlighted with a link. Resources[¶](https://microsoft.github.io/agent-lightning/0.2.0/algorithm-zoo/verl/#resources "Permanent link") --------------------------------------------------------------------------------------------------------------- [VERL](https://microsoft.github.io/agent-lightning/0.2.0/algorithm-zoo/verl/#agentlightning.algorithm.verl.VERL " VERL") expects no initial resources. The first LLM endpoint is directly deployed from the VERL configuration (`.actor_rollout_ref.model.path`). The resource key is always `main_llm`. [VERL](https://microsoft.github.io/agent-lightning/0.2.0/algorithm-zoo/verl/#agentlightning.algorithm.verl.VERL " VERL") currently does not support optimizing multiple [LLM](https://microsoft.github.io/agent-lightning/0.2.0/reference/types/#agentlightning.LLM " agentlightning.LLM") s together. Note The resource type created by [VERL](https://microsoft.github.io/agent-lightning/0.2.0/algorithm-zoo/verl/#agentlightning.algorithm.verl.VERL " VERL") is actually a [ProxyLLM](https://microsoft.github.io/agent-lightning/0.2.0/reference/types/#agentlightning.ProxyLLM " agentlightning.ProxyLLM") , a subclass of the [LLM](https://microsoft.github.io/agent-lightning/0.2.0/reference/types/#agentlightning.LLM " agentlightning.LLM") type. This object contains a **URL template** provided by [VERL](https://microsoft.github.io/agent-lightning/0.2.0/algorithm-zoo/verl/#agentlightning.algorithm.verl.VERL " VERL") , with placeholders for rollout and attempt IDs. When a rollout begins on the agent side, the framework uses the current `rollout_id` and `attempt_id` to format this template, generating a final, unique endpoint URL. This URL points to [VERL](https://microsoft.github.io/agent-lightning/0.2.0/algorithm-zoo/verl/#agentlightning.algorithm.verl.VERL " VERL") 's internal proxy, allowing it to intercept and log all traffic for that specific attempt, for tracing and load balancing purposes. For agents created with the `@rollout` decorator, this resolution of the template is handled automatically ("auto-stripped"). Class-based agents will need to manually resolve the `ProxyLLM` using the rollout context. `[](https://microsoft.github.io/agent-lightning/0.2.0/algorithm-zoo/verl/#__codelineno-2-1) proxy_llm = resources["main_llm"] [](https://microsoft.github.io/agent-lightning/0.2.0/algorithm-zoo/verl/#__codelineno-2-2) proxy_llm.get_base_url(rollout.rollout_id, rollout.attempt.attempt_id)` Customization[¶](https://microsoft.github.io/agent-lightning/0.2.0/algorithm-zoo/verl/#customization "Permanent link") ----------------------------------------------------------------------------------------------------------------------- Internally, [VERL](https://microsoft.github.io/agent-lightning/0.2.0/algorithm-zoo/verl/#agentlightning.algorithm.verl.VERL " VERL") decomposes each agent execution into prompt–response pairs via the [Adapter](https://microsoft.github.io/agent-lightning/0.2.0/reference/algorithm/#agentlightning.Adapter " agentlightning.Adapter") and associates them with their corresponding reward signals as [Triplet](https://microsoft.github.io/agent-lightning/0.2.0/reference/types/#agentlightning.Triplet " agentlightning.Triplet") objects. The final scalar reward, derived from the last triplet in the trajectory, is propagated to all preceding triplets following the [identical assignment strategy](https://arxiv.org/abs/2508.03680) . This ensures that each triplet receives an identical reward signal and can be independently optimized as a valid RLHF trajectory within the VERL framework. At present, [VERL](https://microsoft.github.io/agent-lightning/0.2.0/algorithm-zoo/verl/#agentlightning.algorithm.verl.VERL " VERL") does not expose fine-grained control over its reward propagation or credit assignment mechanisms. Users requiring customized reward shaping or trajectory decomposition are advised to clone and modify the [VERL](https://microsoft.github.io/agent-lightning/0.2.0/algorithm-zoo/verl/#agentlightning.algorithm.verl.VERL " VERL") source implementation directly. Tutorials Using VERL[¶](https://microsoft.github.io/agent-lightning/0.2.0/algorithm-zoo/verl/#tutorials-using-verl "Permanent link") ------------------------------------------------------------------------------------------------------------------------------------- * [Train SQL Agent with RL](https://microsoft.github.io/agent-lightning/0.2.0/how-to/train-sql-agent/) - A practical example of training a SQL agent using VERL. References - Entrypoint[¶](https://microsoft.github.io/agent-lightning/0.2.0/algorithm-zoo/verl/#references-entrypoint "Permanent link") ----------------------------------------------------------------------------------------------------------------------------------------- ### `agentlightning.algorithm.verl` [¶](https://microsoft.github.io/agent-lightning/0.2.0/algorithm-zoo/verl/#agentlightning.algorithm.verl "Permanent link") #### `VERL` [¶](https://microsoft.github.io/agent-lightning/0.2.0/algorithm-zoo/verl/#agentlightning.algorithm.verl.VERL "Permanent link") Bases: `[Algorithm](https://microsoft.github.io/agent-lightning/0.2.0/reference/algorithm/#agentlightning.Algorithm " agentlightning.Algorithm (agentlightning.algorithm.base.Algorithm)") ` VERL-powered algorithm that delegates training to the VERL PPO runner. Warning Advanced customisation currently requires copying the VERL source and modifying it directly. Native hooks for overriding training behaviour will land in a future release. Parameters: * **`config`** (`dict[str, Any]`) – Dictionary mirroring the overrides passed to the VERL CLI. The overrides are merged with VERL's packaged defaults via Hydra before launching training. Examples: `[](https://microsoft.github.io/agent-lightning/0.2.0/algorithm-zoo/verl/#__codelineno-0-1) from agentlightning.algorithm.verl import VERL [](https://microsoft.github.io/agent-lightning/0.2.0/algorithm-zoo/verl/#__codelineno-0-2) [](https://microsoft.github.io/agent-lightning/0.2.0/algorithm-zoo/verl/#__codelineno-0-3) algorithm = VERL( [](https://microsoft.github.io/agent-lightning/0.2.0/algorithm-zoo/verl/#__codelineno-0-4) config={ [](https://microsoft.github.io/agent-lightning/0.2.0/algorithm-zoo/verl/#__codelineno-0-5) "algorithm": { [](https://microsoft.github.io/agent-lightning/0.2.0/algorithm-zoo/verl/#__codelineno-0-6) "adv_estimator": "grpo", [](https://microsoft.github.io/agent-lightning/0.2.0/algorithm-zoo/verl/#__codelineno-0-7) "use_kl_in_reward": False, [](https://microsoft.github.io/agent-lightning/0.2.0/algorithm-zoo/verl/#__codelineno-0-8) }, [](https://microsoft.github.io/agent-lightning/0.2.0/algorithm-zoo/verl/#__codelineno-0-9) "data": { [](https://microsoft.github.io/agent-lightning/0.2.0/algorithm-zoo/verl/#__codelineno-0-10) "train_batch_size": 32, [](https://microsoft.github.io/agent-lightning/0.2.0/algorithm-zoo/verl/#__codelineno-0-11) "max_prompt_length": 4096, [](https://microsoft.github.io/agent-lightning/0.2.0/algorithm-zoo/verl/#__codelineno-0-12) "max_response_length": 2048, [](https://microsoft.github.io/agent-lightning/0.2.0/algorithm-zoo/verl/#__codelineno-0-13) }, [](https://microsoft.github.io/agent-lightning/0.2.0/algorithm-zoo/verl/#__codelineno-0-14) "actor_rollout_ref": { [](https://microsoft.github.io/agent-lightning/0.2.0/algorithm-zoo/verl/#__codelineno-0-15) "rollout": { [](https://microsoft.github.io/agent-lightning/0.2.0/algorithm-zoo/verl/#__codelineno-0-16) "tensor_model_parallel_size": 1, [](https://microsoft.github.io/agent-lightning/0.2.0/algorithm-zoo/verl/#__codelineno-0-17) "n": 4, [](https://microsoft.github.io/agent-lightning/0.2.0/algorithm-zoo/verl/#__codelineno-0-18) "log_prob_micro_batch_size_per_gpu": 4, [](https://microsoft.github.io/agent-lightning/0.2.0/algorithm-zoo/verl/#__codelineno-0-19) "multi_turn": {"format": "hermes"}, [](https://microsoft.github.io/agent-lightning/0.2.0/algorithm-zoo/verl/#__codelineno-0-20) "name": "vllm", [](https://microsoft.github.io/agent-lightning/0.2.0/algorithm-zoo/verl/#__codelineno-0-21) "gpu_memory_utilization": 0.6, [](https://microsoft.github.io/agent-lightning/0.2.0/algorithm-zoo/verl/#__codelineno-0-22) }, [](https://microsoft.github.io/agent-lightning/0.2.0/algorithm-zoo/verl/#__codelineno-0-23) "actor": { [](https://microsoft.github.io/agent-lightning/0.2.0/algorithm-zoo/verl/#__codelineno-0-24) "ppo_mini_batch_size": 32, [](https://microsoft.github.io/agent-lightning/0.2.0/algorithm-zoo/verl/#__codelineno-0-25) "ppo_micro_batch_size_per_gpu": 4, [](https://microsoft.github.io/agent-lightning/0.2.0/algorithm-zoo/verl/#__codelineno-0-26) "optim": {"lr": 1e-6}, [](https://microsoft.github.io/agent-lightning/0.2.0/algorithm-zoo/verl/#__codelineno-0-27) "use_kl_loss": False, [](https://microsoft.github.io/agent-lightning/0.2.0/algorithm-zoo/verl/#__codelineno-0-28) "kl_loss_coef": 0.0, [](https://microsoft.github.io/agent-lightning/0.2.0/algorithm-zoo/verl/#__codelineno-0-29) "entropy_coeff": 0, [](https://microsoft.github.io/agent-lightning/0.2.0/algorithm-zoo/verl/#__codelineno-0-30) "clip_ratio_low": 0.2, [](https://microsoft.github.io/agent-lightning/0.2.0/algorithm-zoo/verl/#__codelineno-0-31) "clip_ratio_high": 0.3, [](https://microsoft.github.io/agent-lightning/0.2.0/algorithm-zoo/verl/#__codelineno-0-32) "fsdp_config": { [](https://microsoft.github.io/agent-lightning/0.2.0/algorithm-zoo/verl/#__codelineno-0-33) "param_offload": True, [](https://microsoft.github.io/agent-lightning/0.2.0/algorithm-zoo/verl/#__codelineno-0-34) "optimizer_offload": True, [](https://microsoft.github.io/agent-lightning/0.2.0/algorithm-zoo/verl/#__codelineno-0-35) }, [](https://microsoft.github.io/agent-lightning/0.2.0/algorithm-zoo/verl/#__codelineno-0-36) }, [](https://microsoft.github.io/agent-lightning/0.2.0/algorithm-zoo/verl/#__codelineno-0-37) "ref": { [](https://microsoft.github.io/agent-lightning/0.2.0/algorithm-zoo/verl/#__codelineno-0-38) "log_prob_micro_batch_size_per_gpu": 8, [](https://microsoft.github.io/agent-lightning/0.2.0/algorithm-zoo/verl/#__codelineno-0-39) "fsdp_config": {"param_offload": True}, [](https://microsoft.github.io/agent-lightning/0.2.0/algorithm-zoo/verl/#__codelineno-0-40) }, [](https://microsoft.github.io/agent-lightning/0.2.0/algorithm-zoo/verl/#__codelineno-0-41) "model": { [](https://microsoft.github.io/agent-lightning/0.2.0/algorithm-zoo/verl/#__codelineno-0-42) "path": "Qwen/Qwen2.5-1.5B-Instruct", [](https://microsoft.github.io/agent-lightning/0.2.0/algorithm-zoo/verl/#__codelineno-0-43) "use_remove_padding": True, [](https://microsoft.github.io/agent-lightning/0.2.0/algorithm-zoo/verl/#__codelineno-0-44) "enable_gradient_checkpointing": True, [](https://microsoft.github.io/agent-lightning/0.2.0/algorithm-zoo/verl/#__codelineno-0-45) }, [](https://microsoft.github.io/agent-lightning/0.2.0/algorithm-zoo/verl/#__codelineno-0-46) }, [](https://microsoft.github.io/agent-lightning/0.2.0/algorithm-zoo/verl/#__codelineno-0-47) "trainer": { [](https://microsoft.github.io/agent-lightning/0.2.0/algorithm-zoo/verl/#__codelineno-0-48) "n_gpus_per_node": 1, [](https://microsoft.github.io/agent-lightning/0.2.0/algorithm-zoo/verl/#__codelineno-0-49) "val_before_train": True, [](https://microsoft.github.io/agent-lightning/0.2.0/algorithm-zoo/verl/#__codelineno-0-50) "critic_warmup": 0, [](https://microsoft.github.io/agent-lightning/0.2.0/algorithm-zoo/verl/#__codelineno-0-51) "logger": ["console", "wandb"], [](https://microsoft.github.io/agent-lightning/0.2.0/algorithm-zoo/verl/#__codelineno-0-52) "project_name": "AgentLightning", [](https://microsoft.github.io/agent-lightning/0.2.0/algorithm-zoo/verl/#__codelineno-0-53) "experiment_name": "calc_x", [](https://microsoft.github.io/agent-lightning/0.2.0/algorithm-zoo/verl/#__codelineno-0-54) "nnodes": 1, [](https://microsoft.github.io/agent-lightning/0.2.0/algorithm-zoo/verl/#__codelineno-0-55) "save_freq": 64, [](https://microsoft.github.io/agent-lightning/0.2.0/algorithm-zoo/verl/#__codelineno-0-56) "test_freq": 32, [](https://microsoft.github.io/agent-lightning/0.2.0/algorithm-zoo/verl/#__codelineno-0-57) "total_epochs": 2, [](https://microsoft.github.io/agent-lightning/0.2.0/algorithm-zoo/verl/#__codelineno-0-58) }, [](https://microsoft.github.io/agent-lightning/0.2.0/algorithm-zoo/verl/#__codelineno-0-59) } [](https://microsoft.github.io/agent-lightning/0.2.0/algorithm-zoo/verl/#__codelineno-0-60) ) [](https://microsoft.github.io/agent-lightning/0.2.0/algorithm-zoo/verl/#__codelineno-0-61) trainer.fit(algorithm, train_dataset=my_train_dataset)` ##### `get_client()` [¶](https://microsoft.github.io/agent-lightning/0.2.0/algorithm-zoo/verl/#agentlightning.algorithm.verl.VERL.get_client "Permanent link") Create a client bound to the VERL-managed Agent Lightning server. Deprecated Since v0.2. ##### `run(train_dataset=None, val_dataset=None)` [¶](https://microsoft.github.io/agent-lightning/0.2.0/algorithm-zoo/verl/#agentlightning.algorithm.verl.VERL.run "Permanent link") Launch the VERL PPO entrypoint with the configured runtime context. Parameters: * **`train_dataset`** (`Optional[[Dataset](https://microsoft.github.io/agent-lightning/0.2.0/reference/types/#agentlightning.Dataset " agentlightning.Dataset (agentlightning.types.Dataset)") [Any]]`, default: `None` ) – Optional dataset forwarded to VERL for training. * **`val_dataset`** (`Optional[[Dataset](https://microsoft.github.io/agent-lightning/0.2.0/reference/types/#agentlightning.Dataset " agentlightning.Dataset (agentlightning.types.Dataset)") [Any]]`, default: `None` ) – Optional dataset forwarded to VERL for evaluation. Raises: * `ValueError` – If required dependencies such as the store, LLM proxy, or adapter have been garbage-collected when using the V1 execution mode. References - Implementation[¶](https://microsoft.github.io/agent-lightning/0.2.0/algorithm-zoo/verl/#references-implementation "Permanent link") ------------------------------------------------------------------------------------------------------------------------------------------------- ### `agentlightning.verl` [¶](https://microsoft.github.io/agent-lightning/0.2.0/algorithm-zoo/verl/#agentlightning.verl "Permanent link") This package contains a _hacky_ integration of VERL with Agent Lightning. #### `AgentLightningTrainer` [¶](https://microsoft.github.io/agent-lightning/0.2.0/algorithm-zoo/verl/#agentlightning.verl.AgentLightningTrainer "Permanent link") Bases: `RayPPOTrainer` Specialized PPO trainer for agent-based reinforcement learning. This trainer is designed specifically for scenarios where the model interacts with external environments, tools, or APIs through an AgentLightningServer. It simplifies the training loop by removing the complex conditional logic present in the original RayPPOTrainer and focusing on the agent mode workflow. Key differences from RayPPOTrainer: 1. Uses AgentModeDaemon for server communication 2. Simplified data flow without pop/union operations 3. Direct batch processing through agent daemon 4. Streamlined validation using agent\_mode validation #### `AgentModeDaemon` [¶](https://microsoft.github.io/agent-lightning/0.2.0/algorithm-zoo/verl/#agentlightning.verl.AgentModeDaemon "Permanent link") AgentModeDaemon using the AgentLightningServer SDK. This class manages the server lifecycle, task queueing, and results retrieval, while also running a proxy server for LLM requests. It maintains the original interface for compatibility with the RayPPOTrainer. ##### `clear_data_and_server()` [¶](https://microsoft.github.io/agent-lightning/0.2.0/algorithm-zoo/verl/#agentlightning.verl.AgentModeDaemon.clear_data_and_server "Permanent link") Resets the internal state of the daemon for the next run. ##### `get_test_metrics()` [¶](https://microsoft.github.io/agent-lightning/0.2.0/algorithm-zoo/verl/#agentlightning.verl.AgentModeDaemon.get_test_metrics "Permanent link") Calculates and returns metrics for a validation run. ##### `get_train_data_batch(max_prompt_length, max_response_length, device)` [¶](https://microsoft.github.io/agent-lightning/0.2.0/algorithm-zoo/verl/#agentlightning.verl.AgentModeDaemon.get_train_data_batch "Permanent link") Processes completed rollouts to generate a training data batch. This function reconstructs the logic from the original AgentModeDaemon, using data retrieved from the new server architecture. It handles padding, truncation, and tensor creation for the PPO training loop. ##### `run_until_all_finished(verbose=True)` [¶](https://microsoft.github.io/agent-lightning/0.2.0/algorithm-zoo/verl/#agentlightning.verl.AgentModeDaemon.run_until_all_finished "Permanent link") Synchronously waits for all queued tasks to be completed and reported. ##### `set_up_data_and_server(data, server_addresses, is_train=True)` [¶](https://microsoft.github.io/agent-lightning/0.2.0/algorithm-zoo/verl/#agentlightning.verl.AgentModeDaemon.set_up_data_and_server "Permanent link") Synchronous wrapper for setting up data and server resources. ##### `start()` [¶](https://microsoft.github.io/agent-lightning/0.2.0/algorithm-zoo/verl/#agentlightning.verl.AgentModeDaemon.start "Permanent link") Starts the main AgentLightningServer and the proxy server. #### `get_left_padded_ids_and_attention_mask(ids, max_length, pad_token_id)` [¶](https://microsoft.github.io/agent-lightning/0.2.0/algorithm-zoo/verl/#agentlightning.verl.get_left_padded_ids_and_attention_mask "Permanent link") Left-pad (or truncate) a sequence of token IDs to a fixed length, and build the corresponding attention mask. Parameters: * **`ids`** (`List[int]`) – the original list of token IDs. * **`max_length`** (`int`) – desired total length after padding/truncation. * **`pad_token_id`** (`int`) – ID to use for padding. Returns: * **`padded_ids`** ( `any` ) – list of length == max\_length. * **`attention_mask`** ( `any` ) – list of same length: 1 for non-pad tokens, 0 for pads. #### `get_right_padded_ids_and_attention_mask(ids, max_length, pad_token_id)` [¶](https://microsoft.github.io/agent-lightning/0.2.0/algorithm-zoo/verl/#agentlightning.verl.get_right_padded_ids_and_attention_mask "Permanent link") Right-pad (or truncate) a sequence of token IDs to a fixed length, and build the corresponding attention mask. Parameters: * **`ids`** (`List[int]`) – the original list of token IDs. * **`max_length`** (`int`) – desired total length after padding/truncation. * **`pad_token_id`** (`int`) – ID to use for padding. Returns: * **`padded_ids`** ( `any` ) – list of length == max\_length. * **`attention_mask`** ( `any` ) – list of same length: 1 for non-pad tokens, 0 for pads. Back to top --- # VERL - Agent-lightning [Skip to content](https://microsoft.github.io/agent-lightning/0.2.1/algorithm-zoo/verl/#verl) ⚠️ You are viewing development documentation. For stable documentation, please visit [https://microsoft.github.io/agent-lightning/stable/](https://microsoft.github.io/agent-lightning/stable/) VERL[¶](https://microsoft.github.io/agent-lightning/0.2.1/algorithm-zoo/verl/#verl "Permanent link") ===================================================================================================== Shortcut You can use the shortcut `agl.VERL(...)` to create a VERL instance. `[](https://microsoft.github.io/agent-lightning/0.2.1/algorithm-zoo/verl/#__codelineno-0-1) import agentlightning as agl [](https://microsoft.github.io/agent-lightning/0.2.1/algorithm-zoo/verl/#__codelineno-0-2) [](https://microsoft.github.io/agent-lightning/0.2.1/algorithm-zoo/verl/#__codelineno-0-3) agl.VERL(...)` Installation[¶](https://microsoft.github.io/agent-lightning/0.2.1/algorithm-zoo/verl/#installation "Permanent link") --------------------------------------------------------------------------------------------------------------------- `[](https://microsoft.github.io/agent-lightning/0.2.1/algorithm-zoo/verl/#__codelineno-1-1) pip install agentlightning[verl]` Warning To avoid various compatibility issues, follow the steps in the [installation guide](https://microsoft.github.io/agent-lightning/0.2.1/tutorials/installation/) to set up VERL and its dependencies. Installing VERL directly with `pip install agentlightning[verl]` can cause issues unless you already have a compatible version of PyTorch installed. Notes for Readers [VERL](https://microsoft.github.io/agent-lightning/0.2.1/algorithm-zoo/verl/#agentlightning.algorithm.verl.VERL " VERL") in this article refers to a wrapper, provided by Agent-lightning, of the [VERL framework](https://github.com/volcengine/verl) . It's a subclass of [agentlightning.Algorithm](https://microsoft.github.io/agent-lightning/0.2.1/reference/algorithm/#agentlightning.Algorithm " agentlightning.Algorithm") . To differentiate it from the VERL framework, all references to the VERL framework shall use the term "VERL framework", and all references to the Agent-lightning wrapper shall be highlighted with a link. Resources[¶](https://microsoft.github.io/agent-lightning/0.2.1/algorithm-zoo/verl/#resources "Permanent link") --------------------------------------------------------------------------------------------------------------- [VERL](https://microsoft.github.io/agent-lightning/0.2.1/algorithm-zoo/verl/#agentlightning.algorithm.verl.VERL " VERL") expects no initial resources. The first LLM endpoint is directly deployed from the VERL configuration (`.actor_rollout_ref.model.path`). The resource key is always `main_llm`. [VERL](https://microsoft.github.io/agent-lightning/0.2.1/algorithm-zoo/verl/#agentlightning.algorithm.verl.VERL " VERL") currently does not support optimizing multiple [LLM](https://microsoft.github.io/agent-lightning/0.2.1/reference/types/#agentlightning.LLM " agentlightning.LLM") s together. Note The resource type created by [VERL](https://microsoft.github.io/agent-lightning/0.2.1/algorithm-zoo/verl/#agentlightning.algorithm.verl.VERL " VERL") is actually a [ProxyLLM](https://microsoft.github.io/agent-lightning/0.2.1/reference/types/#agentlightning.ProxyLLM " agentlightning.ProxyLLM") , a subclass of the [LLM](https://microsoft.github.io/agent-lightning/0.2.1/reference/types/#agentlightning.LLM " agentlightning.LLM") type. This object contains a **URL template** provided by [VERL](https://microsoft.github.io/agent-lightning/0.2.1/algorithm-zoo/verl/#agentlightning.algorithm.verl.VERL " VERL") , with placeholders for rollout and attempt IDs. When a rollout begins on the agent side, the framework uses the current `rollout_id` and `attempt_id` to format this template, generating a final, unique endpoint URL. This URL points to [VERL](https://microsoft.github.io/agent-lightning/0.2.1/algorithm-zoo/verl/#agentlightning.algorithm.verl.VERL " VERL") 's internal proxy, allowing it to intercept and log all traffic for that specific attempt, for tracing and load balancing purposes. For agents created with the `@rollout` decorator, this resolution of the template is handled automatically ("auto-stripped"). Class-based agents will need to manually resolve the `ProxyLLM` using the rollout context. `[](https://microsoft.github.io/agent-lightning/0.2.1/algorithm-zoo/verl/#__codelineno-2-1) proxy_llm = resources["main_llm"] [](https://microsoft.github.io/agent-lightning/0.2.1/algorithm-zoo/verl/#__codelineno-2-2) proxy_llm.get_base_url(rollout.rollout_id, rollout.attempt.attempt_id)` Customization[¶](https://microsoft.github.io/agent-lightning/0.2.1/algorithm-zoo/verl/#customization "Permanent link") ----------------------------------------------------------------------------------------------------------------------- Internally, [VERL](https://microsoft.github.io/agent-lightning/0.2.1/algorithm-zoo/verl/#agentlightning.algorithm.verl.VERL " VERL") decomposes each agent execution into prompt–response pairs via the [Adapter](https://microsoft.github.io/agent-lightning/0.2.1/reference/algorithm/#agentlightning.Adapter " agentlightning.Adapter") and associates them with their corresponding reward signals as [Triplet](https://microsoft.github.io/agent-lightning/0.2.1/reference/types/#agentlightning.Triplet " agentlightning.Triplet") objects. The final scalar reward, derived from the last triplet in the trajectory, is propagated to all preceding triplets following the [identical assignment strategy](https://arxiv.org/abs/2508.03680) . This ensures that each triplet receives an identical reward signal and can be independently optimized as a valid RLHF trajectory within the VERL framework. At present, [VERL](https://microsoft.github.io/agent-lightning/0.2.1/algorithm-zoo/verl/#agentlightning.algorithm.verl.VERL " VERL") does not expose fine-grained control over its reward propagation or credit assignment mechanisms. Users requiring customized reward shaping or trajectory decomposition are advised to clone and modify the [VERL](https://microsoft.github.io/agent-lightning/0.2.1/algorithm-zoo/verl/#agentlightning.algorithm.verl.VERL " VERL") source implementation directly. Tutorials Using VERL[¶](https://microsoft.github.io/agent-lightning/0.2.1/algorithm-zoo/verl/#tutorials-using-verl "Permanent link") ------------------------------------------------------------------------------------------------------------------------------------- * [Train SQL Agent with RL](https://microsoft.github.io/agent-lightning/0.2.1/how-to/train-sql-agent/) - A practical example of training a SQL agent using VERL. References - Entrypoint[¶](https://microsoft.github.io/agent-lightning/0.2.1/algorithm-zoo/verl/#references-entrypoint "Permanent link") ----------------------------------------------------------------------------------------------------------------------------------------- ### `agentlightning.algorithm.verl` [¶](https://microsoft.github.io/agent-lightning/0.2.1/algorithm-zoo/verl/#agentlightning.algorithm.verl "Permanent link") #### `VERL` [¶](https://microsoft.github.io/agent-lightning/0.2.1/algorithm-zoo/verl/#agentlightning.algorithm.verl.VERL "Permanent link") Bases: `[Algorithm](https://microsoft.github.io/agent-lightning/0.2.1/reference/algorithm/#agentlightning.Algorithm " agentlightning.Algorithm (agentlightning.algorithm.base.Algorithm)") ` VERL-powered algorithm that delegates training to the VERL PPO runner. Warning Advanced customisation currently requires copying the VERL source and modifying it directly. Native hooks for overriding training behaviour will land in a future release. Parameters: * **`config`** (`dict[str, Any]`) – Dictionary mirroring the overrides passed to the VERL CLI. The overrides are merged with VERL's packaged defaults via Hydra before launching training. Examples: `[](https://microsoft.github.io/agent-lightning/0.2.1/algorithm-zoo/verl/#__codelineno-0-1) from agentlightning.algorithm.verl import VERL [](https://microsoft.github.io/agent-lightning/0.2.1/algorithm-zoo/verl/#__codelineno-0-2) [](https://microsoft.github.io/agent-lightning/0.2.1/algorithm-zoo/verl/#__codelineno-0-3) algorithm = VERL( [](https://microsoft.github.io/agent-lightning/0.2.1/algorithm-zoo/verl/#__codelineno-0-4) config={ [](https://microsoft.github.io/agent-lightning/0.2.1/algorithm-zoo/verl/#__codelineno-0-5) "algorithm": { [](https://microsoft.github.io/agent-lightning/0.2.1/algorithm-zoo/verl/#__codelineno-0-6) "adv_estimator": "grpo", [](https://microsoft.github.io/agent-lightning/0.2.1/algorithm-zoo/verl/#__codelineno-0-7) "use_kl_in_reward": False, [](https://microsoft.github.io/agent-lightning/0.2.1/algorithm-zoo/verl/#__codelineno-0-8) }, [](https://microsoft.github.io/agent-lightning/0.2.1/algorithm-zoo/verl/#__codelineno-0-9) "data": { [](https://microsoft.github.io/agent-lightning/0.2.1/algorithm-zoo/verl/#__codelineno-0-10) "train_batch_size": 32, [](https://microsoft.github.io/agent-lightning/0.2.1/algorithm-zoo/verl/#__codelineno-0-11) "max_prompt_length": 4096, [](https://microsoft.github.io/agent-lightning/0.2.1/algorithm-zoo/verl/#__codelineno-0-12) "max_response_length": 2048, [](https://microsoft.github.io/agent-lightning/0.2.1/algorithm-zoo/verl/#__codelineno-0-13) }, [](https://microsoft.github.io/agent-lightning/0.2.1/algorithm-zoo/verl/#__codelineno-0-14) "actor_rollout_ref": { [](https://microsoft.github.io/agent-lightning/0.2.1/algorithm-zoo/verl/#__codelineno-0-15) "rollout": { [](https://microsoft.github.io/agent-lightning/0.2.1/algorithm-zoo/verl/#__codelineno-0-16) "tensor_model_parallel_size": 1, [](https://microsoft.github.io/agent-lightning/0.2.1/algorithm-zoo/verl/#__codelineno-0-17) "n": 4, [](https://microsoft.github.io/agent-lightning/0.2.1/algorithm-zoo/verl/#__codelineno-0-18) "log_prob_micro_batch_size_per_gpu": 4, [](https://microsoft.github.io/agent-lightning/0.2.1/algorithm-zoo/verl/#__codelineno-0-19) "multi_turn": {"format": "hermes"}, [](https://microsoft.github.io/agent-lightning/0.2.1/algorithm-zoo/verl/#__codelineno-0-20) "name": "vllm", [](https://microsoft.github.io/agent-lightning/0.2.1/algorithm-zoo/verl/#__codelineno-0-21) "gpu_memory_utilization": 0.6, [](https://microsoft.github.io/agent-lightning/0.2.1/algorithm-zoo/verl/#__codelineno-0-22) }, [](https://microsoft.github.io/agent-lightning/0.2.1/algorithm-zoo/verl/#__codelineno-0-23) "actor": { [](https://microsoft.github.io/agent-lightning/0.2.1/algorithm-zoo/verl/#__codelineno-0-24) "ppo_mini_batch_size": 32, [](https://microsoft.github.io/agent-lightning/0.2.1/algorithm-zoo/verl/#__codelineno-0-25) "ppo_micro_batch_size_per_gpu": 4, [](https://microsoft.github.io/agent-lightning/0.2.1/algorithm-zoo/verl/#__codelineno-0-26) "optim": {"lr": 1e-6}, [](https://microsoft.github.io/agent-lightning/0.2.1/algorithm-zoo/verl/#__codelineno-0-27) "use_kl_loss": False, [](https://microsoft.github.io/agent-lightning/0.2.1/algorithm-zoo/verl/#__codelineno-0-28) "kl_loss_coef": 0.0, [](https://microsoft.github.io/agent-lightning/0.2.1/algorithm-zoo/verl/#__codelineno-0-29) "entropy_coeff": 0, [](https://microsoft.github.io/agent-lightning/0.2.1/algorithm-zoo/verl/#__codelineno-0-30) "clip_ratio_low": 0.2, [](https://microsoft.github.io/agent-lightning/0.2.1/algorithm-zoo/verl/#__codelineno-0-31) "clip_ratio_high": 0.3, [](https://microsoft.github.io/agent-lightning/0.2.1/algorithm-zoo/verl/#__codelineno-0-32) "fsdp_config": { [](https://microsoft.github.io/agent-lightning/0.2.1/algorithm-zoo/verl/#__codelineno-0-33) "param_offload": True, [](https://microsoft.github.io/agent-lightning/0.2.1/algorithm-zoo/verl/#__codelineno-0-34) "optimizer_offload": True, [](https://microsoft.github.io/agent-lightning/0.2.1/algorithm-zoo/verl/#__codelineno-0-35) }, [](https://microsoft.github.io/agent-lightning/0.2.1/algorithm-zoo/verl/#__codelineno-0-36) }, [](https://microsoft.github.io/agent-lightning/0.2.1/algorithm-zoo/verl/#__codelineno-0-37) "ref": { [](https://microsoft.github.io/agent-lightning/0.2.1/algorithm-zoo/verl/#__codelineno-0-38) "log_prob_micro_batch_size_per_gpu": 8, [](https://microsoft.github.io/agent-lightning/0.2.1/algorithm-zoo/verl/#__codelineno-0-39) "fsdp_config": {"param_offload": True}, [](https://microsoft.github.io/agent-lightning/0.2.1/algorithm-zoo/verl/#__codelineno-0-40) }, [](https://microsoft.github.io/agent-lightning/0.2.1/algorithm-zoo/verl/#__codelineno-0-41) "model": { [](https://microsoft.github.io/agent-lightning/0.2.1/algorithm-zoo/verl/#__codelineno-0-42) "path": "Qwen/Qwen2.5-1.5B-Instruct", [](https://microsoft.github.io/agent-lightning/0.2.1/algorithm-zoo/verl/#__codelineno-0-43) "use_remove_padding": True, [](https://microsoft.github.io/agent-lightning/0.2.1/algorithm-zoo/verl/#__codelineno-0-44) "enable_gradient_checkpointing": True, [](https://microsoft.github.io/agent-lightning/0.2.1/algorithm-zoo/verl/#__codelineno-0-45) }, [](https://microsoft.github.io/agent-lightning/0.2.1/algorithm-zoo/verl/#__codelineno-0-46) }, [](https://microsoft.github.io/agent-lightning/0.2.1/algorithm-zoo/verl/#__codelineno-0-47) "trainer": { [](https://microsoft.github.io/agent-lightning/0.2.1/algorithm-zoo/verl/#__codelineno-0-48) "n_gpus_per_node": 1, [](https://microsoft.github.io/agent-lightning/0.2.1/algorithm-zoo/verl/#__codelineno-0-49) "val_before_train": True, [](https://microsoft.github.io/agent-lightning/0.2.1/algorithm-zoo/verl/#__codelineno-0-50) "critic_warmup": 0, [](https://microsoft.github.io/agent-lightning/0.2.1/algorithm-zoo/verl/#__codelineno-0-51) "logger": ["console", "wandb"], [](https://microsoft.github.io/agent-lightning/0.2.1/algorithm-zoo/verl/#__codelineno-0-52) "project_name": "AgentLightning", [](https://microsoft.github.io/agent-lightning/0.2.1/algorithm-zoo/verl/#__codelineno-0-53) "experiment_name": "calc_x", [](https://microsoft.github.io/agent-lightning/0.2.1/algorithm-zoo/verl/#__codelineno-0-54) "nnodes": 1, [](https://microsoft.github.io/agent-lightning/0.2.1/algorithm-zoo/verl/#__codelineno-0-55) "save_freq": 64, [](https://microsoft.github.io/agent-lightning/0.2.1/algorithm-zoo/verl/#__codelineno-0-56) "test_freq": 32, [](https://microsoft.github.io/agent-lightning/0.2.1/algorithm-zoo/verl/#__codelineno-0-57) "total_epochs": 2, [](https://microsoft.github.io/agent-lightning/0.2.1/algorithm-zoo/verl/#__codelineno-0-58) }, [](https://microsoft.github.io/agent-lightning/0.2.1/algorithm-zoo/verl/#__codelineno-0-59) } [](https://microsoft.github.io/agent-lightning/0.2.1/algorithm-zoo/verl/#__codelineno-0-60) ) [](https://microsoft.github.io/agent-lightning/0.2.1/algorithm-zoo/verl/#__codelineno-0-61) trainer.fit(algorithm, train_dataset=my_train_dataset)` ##### `get_client()` [¶](https://microsoft.github.io/agent-lightning/0.2.1/algorithm-zoo/verl/#agentlightning.algorithm.verl.VERL.get_client "Permanent link") Create a client bound to the VERL-managed Agent Lightning server. Deprecated Since v0.2. ##### `run(train_dataset=None, val_dataset=None)` [¶](https://microsoft.github.io/agent-lightning/0.2.1/algorithm-zoo/verl/#agentlightning.algorithm.verl.VERL.run "Permanent link") Launch the VERL PPO entrypoint with the configured runtime context. Parameters: * **`train_dataset`** (`Optional[[Dataset](https://microsoft.github.io/agent-lightning/0.2.1/reference/types/#agentlightning.Dataset " agentlightning.Dataset (agentlightning.types.Dataset)") [Any]]`, default: `None` ) – Optional dataset forwarded to VERL for training. * **`val_dataset`** (`Optional[[Dataset](https://microsoft.github.io/agent-lightning/0.2.1/reference/types/#agentlightning.Dataset " agentlightning.Dataset (agentlightning.types.Dataset)") [Any]]`, default: `None` ) – Optional dataset forwarded to VERL for evaluation. Raises: * `ValueError` – If required dependencies such as the store, LLM proxy, or adapter have been garbage-collected when using the V1 execution mode. References - Implementation[¶](https://microsoft.github.io/agent-lightning/0.2.1/algorithm-zoo/verl/#references-implementation "Permanent link") ------------------------------------------------------------------------------------------------------------------------------------------------- ### `agentlightning.verl` [¶](https://microsoft.github.io/agent-lightning/0.2.1/algorithm-zoo/verl/#agentlightning.verl "Permanent link") This package contains a _hacky_ integration of VERL with Agent Lightning. #### `AgentLightningTrainer` [¶](https://microsoft.github.io/agent-lightning/0.2.1/algorithm-zoo/verl/#agentlightning.verl.AgentLightningTrainer "Permanent link") Bases: `RayPPOTrainer` Specialized PPO trainer for agent-based reinforcement learning. This trainer is designed specifically for scenarios where the model interacts with external environments, tools, or APIs through an AgentLightningServer. It simplifies the training loop by removing the complex conditional logic present in the original RayPPOTrainer and focusing on the agent mode workflow. Key differences from RayPPOTrainer: 1. Uses AgentModeDaemon for server communication 2. Simplified data flow without pop/union operations 3. Direct batch processing through agent daemon 4. Streamlined validation using agent\_mode validation #### `AgentModeDaemon` [¶](https://microsoft.github.io/agent-lightning/0.2.1/algorithm-zoo/verl/#agentlightning.verl.AgentModeDaemon "Permanent link") AgentModeDaemon using the AgentLightningServer SDK. This class manages the server lifecycle, task queueing, and results retrieval, while also running a proxy server for LLM requests. It maintains the original interface for compatibility with the RayPPOTrainer. ##### `clear_data_and_server()` [¶](https://microsoft.github.io/agent-lightning/0.2.1/algorithm-zoo/verl/#agentlightning.verl.AgentModeDaemon.clear_data_and_server "Permanent link") Resets the internal state of the daemon for the next run. ##### `get_test_metrics()` [¶](https://microsoft.github.io/agent-lightning/0.2.1/algorithm-zoo/verl/#agentlightning.verl.AgentModeDaemon.get_test_metrics "Permanent link") Calculates and returns metrics for a validation run. ##### `get_train_data_batch(max_prompt_length, max_response_length, device)` [¶](https://microsoft.github.io/agent-lightning/0.2.1/algorithm-zoo/verl/#agentlightning.verl.AgentModeDaemon.get_train_data_batch "Permanent link") Processes completed rollouts to generate a training data batch. This function reconstructs the logic from the original AgentModeDaemon, using data retrieved from the new server architecture. It handles padding, truncation, and tensor creation for the PPO training loop. ##### `run_until_all_finished(verbose=True)` [¶](https://microsoft.github.io/agent-lightning/0.2.1/algorithm-zoo/verl/#agentlightning.verl.AgentModeDaemon.run_until_all_finished "Permanent link") Synchronously waits for all queued tasks to be completed and reported. ##### `set_up_data_and_server(data, server_addresses, is_train=True)` [¶](https://microsoft.github.io/agent-lightning/0.2.1/algorithm-zoo/verl/#agentlightning.verl.AgentModeDaemon.set_up_data_and_server "Permanent link") Synchronous wrapper for setting up data and server resources. ##### `start()` [¶](https://microsoft.github.io/agent-lightning/0.2.1/algorithm-zoo/verl/#agentlightning.verl.AgentModeDaemon.start "Permanent link") Starts the main AgentLightningServer and the proxy server. #### `get_left_padded_ids_and_attention_mask(ids, max_length, pad_token_id)` [¶](https://microsoft.github.io/agent-lightning/0.2.1/algorithm-zoo/verl/#agentlightning.verl.get_left_padded_ids_and_attention_mask "Permanent link") Left-pad (or truncate) a sequence of token IDs to a fixed length, and build the corresponding attention mask. Parameters: * **`ids`** (`List[int]`) – the original list of token IDs. * **`max_length`** (`int`) – desired total length after padding/truncation. * **`pad_token_id`** (`int`) – ID to use for padding. Returns: * **`padded_ids`** ( `any` ) – list of length == max\_length. * **`attention_mask`** ( `any` ) – list of same length: 1 for non-pad tokens, 0 for pads. #### `get_right_padded_ids_and_attention_mask(ids, max_length, pad_token_id)` [¶](https://microsoft.github.io/agent-lightning/0.2.1/algorithm-zoo/verl/#agentlightning.verl.get_right_padded_ids_and_attention_mask "Permanent link") Right-pad (or truncate) a sequence of token IDs to a fixed length, and build the corresponding attention mask. Parameters: * **`ids`** (`List[int]`) – the original list of token IDs. * **`max_length`** (`int`) – desired total length after padding/truncation. * **`pad_token_id`** (`int`) – ID to use for padding. Returns: * **`padded_ids`** ( `any` ) – list of length == max\_length. * **`attention_mask`** ( `any` ) – list of same length: 1 for non-pad tokens, 0 for pads. Back to top --- # Understanding Store - Agent-lightning [Skip to content](https://microsoft.github.io/agent-lightning/0.2.0/deep-dive/store/#understanding-store) ⚠️ You are viewing development documentation. For stable documentation, please visit [https://microsoft.github.io/agent-lightning/stable/](https://microsoft.github.io/agent-lightning/stable/) Understanding Store[¶](https://microsoft.github.io/agent-lightning/0.2.0/deep-dive/store/#understanding-store "Permanent link") ================================================================================================================================ The **[`LightningStore`](https://microsoft.github.io/agent-lightning/0.2.0/reference/store/#agentlightning.LightningStore " agentlightning.LightningStore") ** is the central coordination point for Agent-lightning. It holds the task queue, rollouts, attempts, spans, and versioned resources, and exposes a small API both Runners and Algorithms use to communicate. This document explains what’s in the store, how statuses transition, how spans are recorded, and the concurrency model (threads & processes). What’s in the Store?[¶](https://microsoft.github.io/agent-lightning/0.2.0/deep-dive/store/#whats-in-the-store "Permanent link") -------------------------------------------------------------------------------------------------------------------------------- ![Store Architecture](https://microsoft.github.io/agent-lightning/0.2.0/assets/store-api-visualized.svg) At a high level: * **Task Queue** – [`enqueue_rollout`](https://microsoft.github.io/agent-lightning/0.2.0/reference/store/#agentlightning.LightningStore.enqueue_rollout " enqueue_rollout(input, mode=None, resources_id=None, config=None, metadata=None) async ") adds work; workers poll with [`dequeue_rollout`](https://microsoft.github.io/agent-lightning/0.2.0/reference/store/#agentlightning.LightningStore.dequeue_rollout " dequeue_rollout() async ") . When a rollout is dequeued, it automatically creates a new attempt associated with itself. * **Rollouts** – A rollout is one unit of work. It has input, metadata, links to resources, and a lifecycle (`queuing → preparing → running → ...`). Valid [RolloutStatus](https://microsoft.github.io/agent-lightning/0.2.0/reference/types/#agentlightning.RolloutStatus " agentlightning.RolloutStatus = Literal['queuing', 'preparing', 'running', 'failed', 'succeeded', 'cancelled', 'requeuing'] module-attribute ") are **`queuing`**, `preparing`, `running`, `succeeded`, `failed`, **`requeuing`**, **`cancelled`**. For algorithms and runners, the rollout can be seen as a whole, without worrying about the internal attempts. * **Attempts** – Each rollout can have multiple executions (retries). Attempts track [`status`](https://microsoft.github.io/agent-lightning/0.2.0/reference/types/#agentlightning.Attempt.status " status = 'preparing' class-attribute instance-attribute ") , [`start_time`](https://microsoft.github.io/agent-lightning/0.2.0/reference/types/#agentlightning.Attempt.start_time " start_time instance-attribute ") , [`end_time`](https://microsoft.github.io/agent-lightning/0.2.0/reference/types/#agentlightning.Attempt.end_time " end_time = None class-attribute instance-attribute ") , [`last_heartbeat_time`](https://microsoft.github.io/agent-lightning/0.2.0/reference/types/#agentlightning.Attempt.last_heartbeat_time " last_heartbeat_time = None class-attribute instance-attribute ") and link to spans. Valid [AttemptStatus](https://microsoft.github.io/agent-lightning/0.2.0/reference/types/#agentlightning.AttemptStatus " agentlightning.AttemptStatus = Literal['preparing', 'running', 'failed', 'succeeded', 'unresponsive', 'timeout'] module-attribute ") are `preparing`, `running`, `succeeded`, `failed`, `requeuing`, `cancelled`. * **Spans** – Structured trace events produced by the Tracer during an attempt. Spans are ordered by a **monotonic sequence id** per `(rollout_id, attempt_id)`. * **Resources** – Versioned, named bundles (e.g., prompt templates) referenced by rollouts. Rollout and Task share the same surface in practice: [`Rollout.input`](https://microsoft.github.io/agent-lightning/0.2.0/reference/types/#agentlightning.Rollout " agentlightning.Rollout") is the task input. The queue stores rollouts that are not yet running; [Runners](https://microsoft.github.io/agent-lightning/0.2.0/reference/runner/#agentlightning.Runner " agentlightning.Runner") dequeue them and update the same rollout’s status as work progresses. All [`LightningStore`](https://microsoft.github.io/agent-lightning/0.2.0/reference/store/#agentlightning.LightningStore " agentlightning.LightningStore") implementations must inherit from [`LightningStore`](https://microsoft.github.io/agent-lightning/0.2.0/reference/store/#agentlightning.LightningStore " agentlightning.LightningStore") and override the methods to implement the storage logic. Before we look at status transitions, it helps to keep in mind that rollouts are the “outside view,” while attempts are the “inside view.” Attempts are what actually run; rollouts summarize the latest attempt plus a small set of control actions like queueing and cancellation. Attempt Status Transitions[¶](https://microsoft.github.io/agent-lightning/0.2.0/deep-dive/store/#attempt-status-transitions "Permanent link") ---------------------------------------------------------------------------------------------------------------------------------------------- The status model is intentionally small and operationally clear. stateDiagram-v2 direction LR [*] --> preparing: Runner calls dequeue_rollout()
or start_rollout()
or start_attempt() preparing --> running: Runner calls
add_[otel_]span()
for the first time state c_runner <> state c_watch <> preparing --> c_runner: Runner calls
update_attempt(...) running --> c_runner: Runner calls
update_attempt(...) running --> c_watch: Watchdog checks preparing --> c_watch: Watchdog checks state "Client-set outcome" as Client { direction TB succeeded failed } state "Watchdog / policy" as Watch { direction TB timeout unresponsive } c_runner --> succeeded: update_attempt(status=succeeded) c_runner --> failed: update_attempt(status=failed) c_watch --> timeout: now - start_time > timeout_seconds c_watch --> unresponsive: now - last_heartbeat > unresponsive_seconds unresponsive --> running: Runner calls
add_[otel_]span() Each attempt begins in **preparing**, created either when a rollout is dequeued or explicitly started. It transitions to **running** the first time a span is recorded. From there, a few clear rules govern how it can change: * When the runner explicitly marks completion, the attempt becomes **succeeded** or **failed** (when the runner catches exception thrown out by the agent). * When the watchdog detects that the total elapsed time since start exceeds the configured limit, it marks the attempt as **timeout**. * If heartbeats stop arriving for too long, the watchdog marks it **unresponsive**. * A new span from the runner can immediately revive an **unresponsive** attempt back to **running**. What's a Watchdog? The watchdog enforces timing and liveness rules defined by each rollout’s [`RolloutConfig`](https://microsoft.github.io/agent-lightning/0.2.0/reference/types/#agentlightning.RolloutConfig " agentlightning.RolloutConfig") . It’s not a separate thread or service, but a function periodically invoked (e.g., before store mutations) to keep attempts healthy and consistent. This simple model allows the system to distinguish between normal termination, abnormal stalling, and recoverable interruption without additional state flags. Rollout Transition Map[¶](https://microsoft.github.io/agent-lightning/0.2.0/deep-dive/store/#rollout-transition-map "Permanent link") -------------------------------------------------------------------------------------------------------------------------------------- Rollout status is an **aggregated view** of its latest attempt’s status, with additional transitions for queueing and explicit cancellation. A rollout’s retry behavior is controlled by [`Rollout.config`](https://microsoft.github.io/agent-lightning/0.2.0/reference/types/#agentlightning.Rollout " agentlightning.Rollout") (a [`RolloutConfig`](https://microsoft.github.io/agent-lightning/0.2.0/reference/types/#agentlightning.RolloutConfig " agentlightning.RolloutConfig") ). The key fields are: * [`timeout_seconds`](https://microsoft.github.io/agent-lightning/0.2.0/reference/types/#agentlightning.RolloutConfig.timeout_seconds " timeout_seconds = None class-attribute instance-attribute ") – maximum wall-clock time for an attempt before it is marked `timeout`. * [`unresponsive_seconds`](https://microsoft.github.io/agent-lightning/0.2.0/reference/types/#agentlightning.RolloutConfig.unresponsive_seconds " unresponsive_seconds = None class-attribute instance-attribute ") – maximum silence between heartbeats before an attempt is marked `unresponsive`. * [`max_attempts`](https://microsoft.github.io/agent-lightning/0.2.0/reference/types/#agentlightning.RolloutConfig.max_attempts " max_attempts = Field(default=1, ge=1) class-attribute instance-attribute ") – total number of attempts allowed for the rollout (including the first). * [`retry_condition`](https://microsoft.github.io/agent-lightning/0.2.0/reference/types/#agentlightning.RolloutConfig.retry_condition " retry_condition = Field(default_factory=(cast(Callable[[], List[AttemptStatus]], list))) class-attribute instance-attribute ") – which attempt terminal statuses should trigger a retry (e.g., `["failed", "timeout", "unresponsive"]`). **How it plays out:** The runner works on attempt `k`. If the attempt ends in a status that is listed in `retry_condition`, and `k < max_attempts`, the rollout moves to **requeuing** and the store creates attempt `k+1`. Otherwise, the rollout becomes **failed** (or **succeeded** if the runner marked it so). `timeout_seconds` and `unresponsive_seconds` are enforced by the watchdog and feed into the same decision flow. A minimal example of how to use `RolloutConfig`: `[](https://microsoft.github.io/agent-lightning/0.2.0/deep-dive/store/#__codelineno-0-1) from agentlightning import RolloutConfig [](https://microsoft.github.io/agent-lightning/0.2.0/deep-dive/store/#__codelineno-0-2) [](https://microsoft.github.io/agent-lightning/0.2.0/deep-dive/store/#__codelineno-0-3) # Retry on explicit failures or timeouts, up to 3 attempts in total. [](https://microsoft.github.io/agent-lightning/0.2.0/deep-dive/store/#__codelineno-0-4) cfg = RolloutConfig( [](https://microsoft.github.io/agent-lightning/0.2.0/deep-dive/store/#__codelineno-0-5) timeout_seconds=600, [](https://microsoft.github.io/agent-lightning/0.2.0/deep-dive/store/#__codelineno-0-6) unresponsive_seconds=120, [](https://microsoft.github.io/agent-lightning/0.2.0/deep-dive/store/#__codelineno-0-7) max_attempts=3, [](https://microsoft.github.io/agent-lightning/0.2.0/deep-dive/store/#__codelineno-0-8) retry_condition=["failed", "timeout"] [](https://microsoft.github.io/agent-lightning/0.2.0/deep-dive/store/#__codelineno-0-9) ) [](https://microsoft.github.io/agent-lightning/0.2.0/deep-dive/store/#__codelineno-0-10) [](https://microsoft.github.io/agent-lightning/0.2.0/deep-dive/store/#__codelineno-0-11) # When creating/enqueuing a rollout, attach this config. [](https://microsoft.github.io/agent-lightning/0.2.0/deep-dive/store/#__codelineno-0-12) # The store will propagate attempt outcomes according to cfg. [](https://microsoft.github.io/agent-lightning/0.2.0/deep-dive/store/#__codelineno-0-13) rollout = await store.enqueue_rollout(input, config=cfg)` | Latest attempt status | Rollout transition | Notes / guards | | --- | --- | --- | | N/A | `queuing` | Created by `enqueue_rollout()`. | | `preparing` | `queuing/requeuing` → `preparing` | Typically `dequeue_rollout()` or `start_rollout()`/`start_attempt()` creates a new attempt. | | `running` | `preparing/queuing/requeuing` → `running` | First `add_[otel_]span()` flips the attempt to `running`; rollout follows via `propagate_status`. | | `succeeded` | `*` → `succeeded` | Terminal. Rollout `end_time` set. | | `failed` / `timeout` / `unresponsive` | `*` → `requeuing` | **Only if** `status ∈ retry_condition ∧ sequence_id < max_attempts`. | | `failed` / `timeout` / `unresponsive` | `*` → `failed` | Otherwise (no retries left or retries disabled). | | `*` | `*` → `cancelled` | Explicitly set by `update_rollout(status=cancelled)`. | Why aggregation? In code, we use `propagate_status()` which actively updates the rollout based on the latest attempt. Reading the table above is usually easier than reverse-engineering the propagation logic in the code: think of the rollout’s transitions as _callbacks_ on attempt state changes, plus queue/cancel paths. Spans[¶](https://microsoft.github.io/agent-lightning/0.2.0/deep-dive/store/#spans "Permanent link") ---------------------------------------------------------------------------------------------------- Every traceable operation in a rollout is stored as a [Span](https://microsoft.github.io/agent-lightning/0.2.0/reference/types/#agentlightning.Span " agentlightning.Span") . Spans not only capture fine-grained instrumentation but also act as periodic heartbeats that demonstrate liveness. The first span marks activation; each subsequent one both extends the trace and refreshes the attempt’s [`last_heartbeat_time`](https://microsoft.github.io/agent-lightning/0.2.0/reference/types/#agentlightning.Attempt.last_heartbeat_time " last_heartbeat_time = None class-attribute instance-attribute ") . If no span arrives within the configured [`unresponsive_seconds`](https://microsoft.github.io/agent-lightning/0.2.0/reference/types/#agentlightning.RolloutConfig.unresponsive_seconds " unresponsive_seconds = None class-attribute instance-attribute ") , the watchdog downgrades the attempt to **unresponsive** until activity resumes. Spans are indexed by `(rollout_id, attempt_id, sequence_id)` where the sequence is monotonic. Tracing analysis tools like [Adapter](https://microsoft.github.io/agent-lightning/0.2.0/reference/algorithm/#agentlightning.Adapter " agentlightning.Adapter") usually rely on "time order" to reconstruct the trace. However, in a distributed system, the recorded start time and end time of a span are not necessarily in the right order when they aggregated into a central store. Therefore, we enforce every span creation to retrieve a monotonically increasing [`sequence_id`](https://microsoft.github.io/agent-lightning/0.2.0/reference/types/#agentlightning.Span.sequence_id " sequence_id instance-attribute ") from the store before adding the span. Note In practice, one `sequence_id` can be used to create multiple spans. In that case, the orders between the multiple spans are determined by the order of `start_time` and `end_time` of the spans. ### OpenTelemetry conversion[¶](https://microsoft.github.io/agent-lightning/0.2.0/deep-dive/store/#opentelemetry-conversion "Permanent link") Runners often produce [OpenTelemetry `ReadableSpan`](https://opentelemetry.io/docs/concepts/signals/traces/#attributes) objects directly. The store normalizes them into [`Span`](https://microsoft.github.io/agent-lightning/0.2.0/reference/types/#agentlightning.Span " agentlightning.Span") as follows: 1. The runner first requests [`get_next_span_sequence_id`](https://microsoft.github.io/agent-lightning/0.2.0/reference/store/#agentlightning.LightningStore.get_next_span_sequence_id " get_next_span_sequence_id(rollout_id, attempt_id) async ") via `sequence_id = await store.get_next_span_sequence_id(rollout_id, attempt_id)`. This guarantees ordering within the attempt regardless of clock skew. 2. `trace_id`, `span_id`, `parent_id`, `name`, `status`, timestamps, attributes, events, links, and resource are copied from the OTEL span. Timestamps are auto-normalized to seconds (nanoseconds are converted). 3. OTEL `SpanContext` and parent context are preserved so downstream tools can correlate traces across systems. 4. Any additional serializable fields present on the `ReadableSpan` are retained in the stored span (after safe JSON serialization), which keeps the representation forward-compatible. Programmatically this is encapsulated by [`Span.from_opentelemetry(readable_span, rollout_id, attempt_id, sequence_id)`](https://microsoft.github.io/agent-lightning/0.2.0/reference/types/#agentlightning.Span.from_opentelemetry " from_opentelemetry(src, rollout_id, attempt_id, sequence_id) classmethod ") ; [`store.add_otel_span(...)`](https://microsoft.github.io/agent-lightning/0.2.0/reference/store/#agentlightning.LightningStore.add_otel_span " add_otel_span(rollout_id, attempt_id, readable_span, sequence_id=None) async ") simply wraps the fetch-then-add flow. The end result is a store span that is stable to sort, merge, and query, while still preserving the richness of the original OTEL payload. Tip [`add_span`](https://microsoft.github.io/agent-lightning/0.2.0/reference/store/#agentlightning.LightningStore.add_span " add_span(span) async ") or [`add_otel_span`](https://microsoft.github.io/agent-lightning/0.2.0/reference/store/#agentlightning.LightningStore.add_otel_span " add_otel_span(rollout_id, attempt_id, readable_span, sequence_id=None) async ") both appends a span _and_ acts as a heartbeat that can revive `unresponsive` → `running`. Store Implementations[¶](https://microsoft.github.io/agent-lightning/0.2.0/deep-dive/store/#store-implementations "Permanent link") ------------------------------------------------------------------------------------------------------------------------------------ Currently, the only out-of-the-box implementation is [`InMemoryLightningStore`](https://microsoft.github.io/agent-lightning/0.2.0/reference/store/#agentlightning.InMemoryLightningStore " agentlightning.InMemoryLightningStore") : * Fast startup, zero external dependencies, and ideal for local development, CI, and unit tests. * Fully asyncio-safe for writes; most reader operations can iterate without locks, except those that need to perform multiple queries. * Includes a best-effort span eviction policy once memory crosses a configured watermark; querying evicted spans raises a clear error so callers can fall back. For production you will likely want persistence. We’re actively building a SQLite-backed store that keeps the same API surface while adding durability, crash recovery, and better historical span queries. If you need something sooner, implement your own store by subclassing [`LightningStore`](https://microsoft.github.io/agent-lightning/0.2.0/reference/store/#agentlightning.LightningStore " agentlightning.LightningStore") and providing concrete storage for the small set of abstract methods (`enqueue_rollout`, `dequeue_rollout`, `update_attempt`, `add_span`, etc.). This document plus the tests in `tests/store/` illustrate the expected behavior. Thread Safety[¶](https://microsoft.github.io/agent-lightning/0.2.0/deep-dive/store/#thread-safety "Permanent link") -------------------------------------------------------------------------------------------------------------------- **[`LightningStoreThreaded`](https://microsoft.github.io/agent-lightning/0.2.0/reference/store/#agentlightning.LightningStoreThreaded " agentlightning.LightningStoreThreaded") ** is a subclass of [`LightningStore`](https://microsoft.github.io/agent-lightning/0.2.0/reference/store/#agentlightning.LightningStore " agentlightning.LightningStore") that wraps another underlying store to make a store instance safe for multi-threaded callers. It wraps every state-mutating call in a mutex. Specifically: * Methods like [`start_rollout`](https://microsoft.github.io/agent-lightning/0.2.0/reference/store/#agentlightning.LightningStore.start_rollout " start_rollout(input, mode=None, resources_id=None, config=None, metadata=None) async ") , [`enqueue_rollout`](https://microsoft.github.io/agent-lightning/0.2.0/reference/store/#agentlightning.LightningStore.enqueue_rollout " enqueue_rollout(input, mode=None, resources_id=None, config=None, metadata=None) async ") , [`update_attempt`](https://microsoft.github.io/agent-lightning/0.2.0/reference/store/#agentlightning.LightningStore.update_attempt " update_attempt(rollout_id, attempt_id, status=UNSET, worker_id=UNSET, last_heartbeat_time=UNSET, metadata=UNSET) async ") , [`add_span`](https://microsoft.github.io/agent-lightning/0.2.0/reference/store/#agentlightning.LightningStore.add_span " add_span(span) async ") , etc. are guarded by a lock. * Non-mutating, potentially blocking calls remain pass-through by design (e.g., [`wait_for_rollouts`](https://microsoft.github.io/agent-lightning/0.2.0/reference/store/#agentlightning.LightningStore.wait_for_rollouts " wait_for_rollouts(*, rollout_ids, timeout=None) async ") ), as they don’t modify shared state and should not hold the lock for long periods. Process Safety and Client-server Store[¶](https://microsoft.github.io/agent-lightning/0.2.0/deep-dive/store/#process-safety-and-client-server-store "Permanent link") ---------------------------------------------------------------------------------------------------------------------------------------------------------------------- **[`LightningStoreServer`](https://microsoft.github.io/agent-lightning/0.2.0/reference/store/#agentlightning.LightningStoreServer " agentlightning.LightningStoreServer") ** wraps another underlying store and runs a FastAPI app to expose the store API over HTTP. [`LightningStoreClient`](https://microsoft.github.io/agent-lightning/0.2.0/reference/store/#agentlightning.LightningStoreClient " agentlightning.LightningStoreClient") is a small [`LightningStore`](https://microsoft.github.io/agent-lightning/0.2.0/reference/store/#agentlightning.LightningStore " agentlightning.LightningStore") implementation that talks to the HTTP API. Warning The server HTTP API is not considered a stable API at this moment. Users are encouraged to use the [`LightningStoreClient`](https://microsoft.github.io/agent-lightning/0.2.0/reference/store/#agentlightning.LightningStoreClient " agentlightning.LightningStoreClient") to communicate with the server as a stable interface. The server tracks the creator PID. In the owner process it delegates directly to the in-memory store; in other processes it lazily constructs a [`LightningStoreClient`](https://microsoft.github.io/agent-lightning/0.2.0/reference/store/#agentlightning.LightningStoreClient " agentlightning.LightningStoreClient") to talk to the HTTP API. This prevents accidental cross-process mutation of the wrong memory image. When the server is pickled (e.g., via `multiprocessing`), only the minimal fields are serialized, but **NOT** the FastAPI/uvicorn objects. Subprocesses won’t accidentally carry live server state. Forked subprocess should also use [`LightningStoreClient`](https://microsoft.github.io/agent-lightning/0.2.0/reference/store/#agentlightning.LightningStoreClient " agentlightning.LightningStoreClient") to communicate with the server in the main process. On the client side, the client retries network/5xx failures using a small backoff, and probes `/health` between attempts. Application exceptions inside the server are wrapped as HTTP 400 with a traceback—these are **not retried**. The client also maintains a **per-event-loop** `aiohttp.ClientSession` map so that tracer callbacks (often on separate loops/threads) don’t hang by reusing a session from another loop. Minimal lifecycle: `[](https://microsoft.github.io/agent-lightning/0.2.0/deep-dive/store/#__codelineno-1-1) import agentlightning as agl [](https://microsoft.github.io/agent-lightning/0.2.0/deep-dive/store/#__codelineno-1-2) [](https://microsoft.github.io/agent-lightning/0.2.0/deep-dive/store/#__codelineno-1-3) # Server (owner process) [](https://microsoft.github.io/agent-lightning/0.2.0/deep-dive/store/#__codelineno-1-4) in_memory_store = agl.InMemoryLightningStore() [](https://microsoft.github.io/agent-lightning/0.2.0/deep-dive/store/#__codelineno-1-5) server = agl.LightningStoreServer(store=in_memory_store, host="0.0.0.0", port=4747) [](https://microsoft.github.io/agent-lightning/0.2.0/deep-dive/store/#__codelineno-1-6) await server.start() # starts uvicorn in a daemon thread and waits for /health [](https://microsoft.github.io/agent-lightning/0.2.0/deep-dive/store/#__codelineno-1-7) # or keep your own event loop and stop via await server.stop() [](https://microsoft.github.io/agent-lightning/0.2.0/deep-dive/store/#__codelineno-1-8) # await server.run_forever() [](https://microsoft.github.io/agent-lightning/0.2.0/deep-dive/store/#__codelineno-1-9) [](https://microsoft.github.io/agent-lightning/0.2.0/deep-dive/store/#__codelineno-1-10) # Client (same or different process) [](https://microsoft.github.io/agent-lightning/0.2.0/deep-dive/store/#__codelineno-1-11) client = agl.LightningStoreClient("http://localhost:4747") [](https://microsoft.github.io/agent-lightning/0.2.0/deep-dive/store/#__codelineno-1-12) [](https://microsoft.github.io/agent-lightning/0.2.0/deep-dive/store/#__codelineno-1-13) print(await client.query_rollouts(status=["queuing"])) [](https://microsoft.github.io/agent-lightning/0.2.0/deep-dive/store/#__codelineno-1-14) [](https://microsoft.github.io/agent-lightning/0.2.0/deep-dive/store/#__codelineno-1-15) await client.close() [](https://microsoft.github.io/agent-lightning/0.2.0/deep-dive/store/#__codelineno-1-16) await server.stop()` Another approach is to use a dedicated command line to start a long running server process, possibly sharable across multiple processes. In the main process, you can always use [`LightningStoreClient`](https://microsoft.github.io/agent-lightning/0.2.0/reference/store/#agentlightning.LightningStoreClient " agentlightning.LightningStoreClient") to communicate with the server. `[](https://microsoft.github.io/agent-lightning/0.2.0/deep-dive/store/#__codelineno-2-1) agl store --port 4747` Note [`LightningStoreClient.wait_for_rollouts`](https://microsoft.github.io/agent-lightning/0.2.0/reference/store/#agentlightning.LightningStoreClient.wait_for_rollouts " wait_for_rollouts(*, rollout_ids, timeout=None) async ") intentionally enforces a tiny timeout (≤ 0.1s) to avoid blocking event loops. Poll with short timeouts or compose with `asyncio.wait_for` at a higher layer. Back to top --- # Serving LLM - Agent-lightning [Skip to content](https://microsoft.github.io/agent-lightning/0.2.2/deep-dive/serving-llm/#serving-llms-under-agent-lightning) ⚠️ You are viewing development documentation. For stable documentation, please visit [https://microsoft.github.io/agent-lightning/stable/](https://microsoft.github.io/agent-lightning/stable/) Serving LLMs under Agent-lightning[¶](https://microsoft.github.io/agent-lightning/0.2.2/deep-dive/serving-llm/#serving-llms-under-agent-lightning "Permanent link") ==================================================================================================================================================================== Agent-lightning focuses on data, learning signals, and control flow — **not** on running model inference. This deep dive explains how to **serve** a model alongside Agent-lightning so runners can call it reliably, how the **LLM Proxy** fits into the loop, and why **token IDs** matter if you care about correctness in training and evaluation. General background on LLM serving[¶](https://microsoft.github.io/agent-lightning/0.2.2/deep-dive/serving-llm/#general-background-on-llm-serving "Permanent link") ------------------------------------------------------------------------------------------------------------------------------------------------------------------ [](https://microsoft.github.io/agent-lightning/0.2.2/deep-dive/serving-llm/) Serving a model is essential if you want to train it, especially when you use the model’s own generations as training data. We’ll briefly review the general background to ensure all readers are aligned. Modern LLM servers solve a difficult scheduling problem: keeping GPUs fully utilized while handling prompts of different lengths, streaming tokens as they arrive, and fitting large KV caches into limited memory. Techniques like [**continuous batching**](https://www.anyscale.com/blog/continuous-batching-llm-inference) and [**paged attention**](https://arxiv.org/abs/2309.06180) address these challenges. Continuous batching interleaves decoding across requests to reuse weights efficiently; with careful memory planning, it achieves major throughput gains without increasing latency. PagedAttention reduces KV-cache fragmentation so batching remains effective as sequences grow. See [vLLM’s PagedAttention paper](https://arxiv.org/abs/2309.06180) and [industry analyses](https://www.baseten.co/blog/continuous-vs-dynamic-batching-for-ai-inference/) for details. Balancing inference correctness and efficiency is difficult — a [recent blog](https://thinkingmachines.ai/blog/defeating-nondeterminism-in-llm-inference/) from Thinking Machines Labs highlights how inference nondeterminism ultimately affects training. Beyond scheduling, servers expose an HTTP API, often **OpenAI-compatible** (`/v1/chat/completions` and `/v1/responses`), which is itself a complex stack. In addition to text prompts and chat messages, the API defines many parameters and response fields such as [tool calls](https://platform.openai.com/docs/guides/function-calling) , [structured output](https://platform.openai.com/docs/guides/structured-outputs) , and [multimodal support](https://platform.openai.com/docs/guides/images-vision) . Much effort has been put into implementing all these parameters for many frameworks. Popular engines like **vLLM** and [**SGLang**](https://github.com/sgl-project/sglang) ship with OpenAI-compatible frontends so you can reuse existing client code. [Ollama](https://ollama.com/blog/openai-compatibility) and [llama.cpp](https://llama-cpp-python.readthedocs.io/en/latest/server/) provide similar capabilities. However, because models differ internally, each framework interprets and implements the API slightly differently. Even with identical requests, the tokens passed to the model can vary substantially across frameworks. What Agent-lightning expects from a served LLM[¶](https://microsoft.github.io/agent-lightning/0.2.2/deep-dive/serving-llm/#what-agent-lightning-expects-from-a-served-llm "Permanent link") -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Most of the issues above either have workarounds or remain open research problems. Keep them in mind, but the key question is: what does Agent-lightning expect from a served LLM? The answer includes at least two things: * An OpenAI-compatible **Chat Completions** or **Responses** endpoint the agent can call during rollouts. * Optional training and debugging signals: **logprobs**, **usage**, and ideally **token IDs**. (OpenAI’s public API exposes usage and logprobs, but **not** token IDs — more on [why IDs matter](https://microsoft.github.io/agent-lightning/0.2.2/deep-dive/serving-llm/#token-ids-matter "Token IDs and why they matter") later.) Launching a serving framework[¶](https://microsoft.github.io/agent-lightning/0.2.2/deep-dive/serving-llm/#launching-a-serving-framework "Permanent link") ---------------------------------------------------------------------------------------------------------------------------------------------------------- For many algorithms, you’ll start an engine (e.g., **vLLM** or **SGLang**) before rollouts, then shut it down afterward to free GPU memory. Most frameworks provide a one-line “serve” command to launch the OpenAI-compatible server. You can use those to bring up `/v1/chat/completions` with your checkpoint, ensuring streaming and any required tool-calling features are enabled. A working example is shown in [Unsloth SFT](https://microsoft.github.io/agent-lightning/0.2.2/how-to/unsloth-sft/) . Weight updates — which occur after each training step — are trickier. Some frameworks like [vLLM](https://vllm.ai/) support hot-updating model weights, but it’s usually simpler and more reliable to restart the engine to load new weights. For medium-sized tasks (hundreds of rollouts taking 10+ minutes), the restart overhead (under 30 seconds) is typically negligible. If you’re using Agent-lightning’s [**VERL**](https://microsoft.github.io/agent-lightning/0.2.2/algorithm-zoo/verl/#agentlightning.algorithm.verl.VERL " VERL") integration, the algorithm can **manage the server automatically**. The [VERL framework](https://github.com/volcengine/verl) intelligently allocates compute resources and wraps vLLM/SGLang behind an `AsyncLLMServer` abstraction. You can directly use this as the LLM endpoint for agents. Since VERL can spawn multiple vLLM replicas, using [`LLMProxy`](https://microsoft.github.io/agent-lightning/0.2.2/reference/algorithm/#agentlightning.LLMProxy " agentlightning.LLMProxy") to manage them adds an additional safety layer. A full sequence diagram of how [VERL](https://microsoft.github.io/agent-lightning/0.2.2/algorithm-zoo/verl/#agentlightning.algorithm.verl.VERL " VERL") interacts with the LLM server and proxy is available [here](https://microsoft.github.io/agent-lightning/0.2.2/deep-dive/birds-eye-view/#birds-eye-view-verl-example "Putting It All Together: A Reinforcement Learning Example (VERL)") . LLM Proxy[¶](https://microsoft.github.io/agent-lightning/0.2.2/deep-dive/serving-llm/#llm-proxy "Permanent link") ------------------------------------------------------------------------------------------------------------------ The **LLM Proxy** is a utility class in Agent-lightning, built on [LiteLLM](https://docs.litellm.ai/) , that sits between runners and your backend engine(s) or server(s). In Agent-lightning it acts as a single URL registered as a [`Resource`](https://microsoft.github.io/agent-lightning/0.2.2/reference/types/#agentlightning.Resource " agentlightning.Resource") in the store, offering three key benefits: 1. **Unified endpoint & hot-swaps.** You can redirect traffic between OpenAI, Anthropic, local vLLM/SGLang, or canary checkpoints without modifying agent code — simply repoint the proxy. 2. **First-class tracing.** The proxy emits **OpenTelemetry** spans for every call and sends them to the [`LightningStore`](https://microsoft.github.io/agent-lightning/0.2.2/reference/store/#agentlightning.LightningStore " agentlightning.LightningStore") . It includes rollout and attempt identifiers in request headers so spans are correctly attributed. Sequence numbers are allocated monotonically via the store to [prevent clock-skew issues](https://microsoft.github.io/agent-lightning/0.2.2/tutorials/traces/#distributed-tracing "LLM Proxy") and allow reliable reconstruction of execution trees. 3. **Token IDs.** The proxy can return prompt and response token IDs along with the model output. More details are available in the [next section](https://microsoft.github.io/agent-lightning/0.2.2/deep-dive/serving-llm/#token-ids-matter "Token IDs and why they matter") . Operationally, running the proxy alongside the algorithm works best: the algorithm registers the backend (e.g., the vLLM URL) via [`LLMProxy.update_model_list`](https://microsoft.github.io/agent-lightning/0.2.2/reference/algorithm/#agentlightning.LLMProxy.update_model_list " update_model_list(model_list)") , publishes the proxy URL as a resource via [`LightningStore.add_resources`](https://microsoft.github.io/agent-lightning/0.2.2/reference/store/#agentlightning.LightningStore.add_resources " add_resources(resources) async ") , and runners simply use that URL during rollouts. This mirrors many production client–server setups. Token IDs and why they matter[¶](https://microsoft.github.io/agent-lightning/0.2.2/deep-dive/serving-llm/#token-ids-and-why-they-matter "Permanent link") ---------------------------------------------------------------------------------------------------------------------------------------------------------- [](https://microsoft.github.io/agent-lightning/0.2.2/deep-dive/serving-llm/) This section explains how Agent-lightning handles and uses token IDs — a subtle but important detail for training stability and accuracy. Most agents interact with LLMs via **Chat Completion APIs**, exchanging chat messages. There are two main approaches to collecting training data from such agents. Note Tokenization here refers to the process of converting **Chat Messages** into **Token IDs**. Detokenization is the reverse process of converting **Token IDs** back to **Chat Messages**. Normally, the tokenizer is published along with the pretrained model, which includes a vocabulary, special tokens, and a chat template to dealing with chat messages. **1\. Retokenizing chat messages.** In this approach, you store chat messages as text and let training algorithms **retokenize** them later, as done in many SFT workflows (e.g., [HuggingFace SFT](https://huggingface.co/docs/trl/sft_trainer) ). In practice, we’ve found this method unstable and less accurate. The chart below compares training results. The retokenization approach is run twice. All settings are the same except for the retokenization approach. This instability has three causes. Firstly, chat template used in different frameworks could be slightly different. For example, one single LLaMA model can work with multiple chat templates (multiple in [vLLM](https://github.com/vllm-project/vllm/tree/1d165d6d859d3c50720f0c07209db2363c4fd33b/examples) and one in [HuggingFace](https://huggingface.co/meta-llama) ). It's possible that the chat template used in detokenization is different from the one used in tokenization (this is actually an implementation bug). Secondly, a word might be generated as two tokens (e.g., `H + AVING`) but later retokenized as `HAV + ING`. The text looks identical, but the token IDs differ from what the model originally produced. Thirdly, a generated tool call text like `{ "name": ... }` is parsed by tool call parser into an object that is required by chat completion API. Later, the object is rendered back to `{ "name": ... }` and retokenized again, tool call parsing and re-rendering might cause changes in whitespace and formatting. In some situations, JSON errors may even be auto-corrected by the tool call parser — masking the model’s true generation errors and preventing them from being trained away. * * * **2\. Saving token IDs directly.** The alternative is to save the token IDs generated by the model, as done in RL setups like [Tinker](https://thinkingmachines.ai/tinker/) . This requires a training pipeline that treats tokens as first-class entities, meaning agents must communicate with the inference engine at the token level. However, most agents — especially those built with frameworks like LangChain — rely on OpenAI-compatible APIs and can’t tokenize or detokenize themselves. As mentioned [earlier](https://microsoft.github.io/agent-lightning/0.2.2/deep-dive/serving-llm/#general-llm-serving-background "General background on LLM serving") , implementing this layer manually is complex and error-prone. Some frameworks implement custom solutions (e.g., [VERL Agent Loop](https://github.com/volcengine/verl/blob/4da0d3d3188072772cb2ec817b3d6cf4a463821f/recipe/langgraph_agent/chat_model.py) , [Tinker Renderer](https://github.com/thinking-machines-lab/tinker-cookbook/blob/34a6588d7055040c259985d98e71c0140b389ba7/tinker_cookbook/renderers.py) ), while others leave it to users (e.g., [SkyRL Search-R1](https://novasky-ai.notion.site/skyrl-searchr1) ). * * * A better solution is to use an **OpenAI-compatible API that returns token IDs directly.** This lets agents continue using familiar APIs while capturing token IDs via [tracing](https://microsoft.github.io/agent-lightning/0.2.2/tutorials/traces/) for training. The limitation, of course, is that the serving framework must actually support this capability. When Agent-lightning was first released, we implemented an [instrumented vLLM server](https://github.com/microsoft/agent-lightning/blob/v0.1/agentlightning/instrumentation/vllm.py) that monkey-patched vLLM’s OpenAI server to return token IDs. Since then, the Agent-lightning and vLLM teams have collaborated to add this feature directly to [vLLM core](https://github.com/vllm-project/vllm/pull/22587) . Starting with **vLLM v0.10.2**, the OpenAI-compatible API includes a [`return_token_ids` parameter](https://docs.vllm.ai/en/v0.10.2/serving/openai_compatible_server.html#api-reference) , allowing token IDs to be requested alongside chat messages. SGLang has tracked [similar feature requests](https://github.com/sgl-project/sglang/issues/2634) , though its OpenAI-compatible layer doesn’t yet support them. In short, when using vLLM v0.10.2 or newer, [`LLMProxy`](https://microsoft.github.io/agent-lightning/0.2.2/reference/algorithm/#agentlightning.LLMProxy " agentlightning.LLMProxy") automatically adds `return_token_ids` to each request so the engine includes token IDs in its response. For older vLLM versions, you still need the instrumented version (via `agl vllm` CLI command). Finally, if you only save token IDs in spans, it will have its own limitations — if you train one model using spans from another model with a different tokenizer, incompatibilities can arise. In practice, though, spans in Agent-lightning always store both chat messages and token IDs (actually the full request and response objects), allowing you to fall back to retokenization when necessary. Back to top --- # Understanding Store - Agent-lightning [Skip to content](https://microsoft.github.io/agent-lightning/0.2.2/deep-dive/store/#understanding-store) ⚠️ You are viewing development documentation. For stable documentation, please visit [https://microsoft.github.io/agent-lightning/stable/](https://microsoft.github.io/agent-lightning/stable/) Understanding Store[¶](https://microsoft.github.io/agent-lightning/0.2.2/deep-dive/store/#understanding-store "Permanent link") ================================================================================================================================ The **[`LightningStore`](https://microsoft.github.io/agent-lightning/0.2.2/reference/store/#agentlightning.LightningStore " agentlightning.LightningStore") ** is the central coordination point for Agent-lightning. It holds the task queue, rollouts, attempts, spans, and versioned resources, and exposes a small API both Runners and Algorithms use to communicate. This document explains what’s in the store, how statuses transition, how spans are recorded, and the concurrency model (threads & processes). What’s in the Store?[¶](https://microsoft.github.io/agent-lightning/0.2.2/deep-dive/store/#whats-in-the-store "Permanent link") -------------------------------------------------------------------------------------------------------------------------------- ![Store Architecture](https://microsoft.github.io/agent-lightning/0.2.2/assets/store-api-visualized.svg) At a high level: * **Task Queue** – [`enqueue_rollout`](https://microsoft.github.io/agent-lightning/0.2.2/reference/store/#agentlightning.LightningStore.enqueue_rollout " enqueue_rollout(input, mode=None, resources_id=None, config=None, metadata=None) async ") adds work; workers poll with [`dequeue_rollout`](https://microsoft.github.io/agent-lightning/0.2.2/reference/store/#agentlightning.LightningStore.dequeue_rollout " dequeue_rollout() async ") . When a rollout is dequeued, it automatically creates a new attempt associated with itself. * **Rollouts** – A rollout is one unit of work. It has input, metadata, links to resources, and a lifecycle (`queuing → preparing → running → ...`). Valid [RolloutStatus](https://microsoft.github.io/agent-lightning/0.2.2/reference/types/#agentlightning.RolloutStatus " agentlightning.RolloutStatus = Literal['queuing', 'preparing', 'running', 'failed', 'succeeded', 'cancelled', 'requeuing'] module-attribute ") are **`queuing`**, `preparing`, `running`, `succeeded`, `failed`, **`requeuing`**, **`cancelled`**. For algorithms and runners, the rollout can be seen as a whole, without worrying about the internal attempts. * **Attempts** – Each rollout can have multiple executions (retries). Attempts track [`status`](https://microsoft.github.io/agent-lightning/0.2.2/reference/types/#agentlightning.Attempt.status " status = 'preparing' class-attribute instance-attribute ") , [`start_time`](https://microsoft.github.io/agent-lightning/0.2.2/reference/types/#agentlightning.Attempt.start_time " start_time instance-attribute ") , [`end_time`](https://microsoft.github.io/agent-lightning/0.2.2/reference/types/#agentlightning.Attempt.end_time " end_time = None class-attribute instance-attribute ") , [`last_heartbeat_time`](https://microsoft.github.io/agent-lightning/0.2.2/reference/types/#agentlightning.Attempt.last_heartbeat_time " last_heartbeat_time = None class-attribute instance-attribute ") and link to spans. Valid [AttemptStatus](https://microsoft.github.io/agent-lightning/0.2.2/reference/types/#agentlightning.AttemptStatus " agentlightning.AttemptStatus = Literal['preparing', 'running', 'failed', 'succeeded', 'unresponsive', 'timeout'] module-attribute ") are `preparing`, `running`, `succeeded`, `failed`, `requeuing`, `cancelled`. * **Spans** – Structured trace events produced by the Tracer during an attempt. Spans are ordered by a **monotonic sequence id** per `(rollout_id, attempt_id)`. * **Resources** – Versioned, named bundles (e.g., prompt templates) referenced by rollouts. Rollout and Task share the same surface in practice: [`Rollout.input`](https://microsoft.github.io/agent-lightning/0.2.2/reference/types/#agentlightning.Rollout " agentlightning.Rollout") is the task input. The queue stores rollouts that are not yet running; [Runners](https://microsoft.github.io/agent-lightning/0.2.2/reference/runner/#agentlightning.Runner " agentlightning.Runner") dequeue them and update the same rollout’s status as work progresses. All [`LightningStore`](https://microsoft.github.io/agent-lightning/0.2.2/reference/store/#agentlightning.LightningStore " agentlightning.LightningStore") implementations must inherit from [`LightningStore`](https://microsoft.github.io/agent-lightning/0.2.2/reference/store/#agentlightning.LightningStore " agentlightning.LightningStore") and override the methods to implement the storage logic. Before we look at status transitions, it helps to keep in mind that rollouts are the “outside view,” while attempts are the “inside view.” Attempts are what actually run; rollouts summarize the latest attempt plus a small set of control actions like queueing and cancellation. Attempt Status Transitions[¶](https://microsoft.github.io/agent-lightning/0.2.2/deep-dive/store/#attempt-status-transitions "Permanent link") ---------------------------------------------------------------------------------------------------------------------------------------------- The status model is intentionally small and operationally clear. stateDiagram-v2 direction LR [*] --> preparing: Runner calls dequeue_rollout()
or start_rollout()
or start_attempt() preparing --> running: Runner calls
add_[otel_]span()
for the first time state c_runner <> state c_watch <> preparing --> c_runner: Runner calls
update_attempt(...) running --> c_runner: Runner calls
update_attempt(...) running --> c_watch: Watchdog checks preparing --> c_watch: Watchdog checks state "Client-set outcome" as Client { direction TB succeeded failed } state "Watchdog / policy" as Watch { direction TB timeout unresponsive } c_runner --> succeeded: update_attempt(status=succeeded) c_runner --> failed: update_attempt(status=failed) c_watch --> timeout: now - start_time > timeout_seconds c_watch --> unresponsive: now - last_heartbeat > unresponsive_seconds unresponsive --> running: Runner calls
add_[otel_]span() Each attempt begins in **preparing**, created either when a rollout is dequeued or explicitly started. It transitions to **running** the first time a span is recorded. From there, a few clear rules govern how it can change: * When the runner explicitly marks completion, the attempt becomes **succeeded** or **failed** (when the runner catches exception thrown out by the agent). * When the watchdog detects that the total elapsed time since start exceeds the configured limit, it marks the attempt as **timeout**. * If heartbeats stop arriving for too long, the watchdog marks it **unresponsive**. * A new span from the runner can immediately revive an **unresponsive** attempt back to **running**. What's a Watchdog? The watchdog enforces timing and liveness rules defined by each rollout’s [`RolloutConfig`](https://microsoft.github.io/agent-lightning/0.2.2/reference/types/#agentlightning.RolloutConfig " agentlightning.RolloutConfig") . It’s not a separate thread or service, but a function periodically invoked (e.g., before store mutations) to keep attempts healthy and consistent. This simple model allows the system to distinguish between normal termination, abnormal stalling, and recoverable interruption without additional state flags. Rollout Transition Map[¶](https://microsoft.github.io/agent-lightning/0.2.2/deep-dive/store/#rollout-transition-map "Permanent link") -------------------------------------------------------------------------------------------------------------------------------------- Rollout status is an **aggregated view** of its latest attempt’s status, with additional transitions for queueing and explicit cancellation. A rollout’s retry behavior is controlled by [`Rollout.config`](https://microsoft.github.io/agent-lightning/0.2.2/reference/types/#agentlightning.Rollout " agentlightning.Rollout") (a [`RolloutConfig`](https://microsoft.github.io/agent-lightning/0.2.2/reference/types/#agentlightning.RolloutConfig " agentlightning.RolloutConfig") ). The key fields are: * [`timeout_seconds`](https://microsoft.github.io/agent-lightning/0.2.2/reference/types/#agentlightning.RolloutConfig.timeout_seconds " timeout_seconds = None class-attribute instance-attribute ") – maximum wall-clock time for an attempt before it is marked `timeout`. * [`unresponsive_seconds`](https://microsoft.github.io/agent-lightning/0.2.2/reference/types/#agentlightning.RolloutConfig.unresponsive_seconds " unresponsive_seconds = None class-attribute instance-attribute ") – maximum silence between heartbeats before an attempt is marked `unresponsive`. * [`max_attempts`](https://microsoft.github.io/agent-lightning/0.2.2/reference/types/#agentlightning.RolloutConfig.max_attempts " max_attempts = Field(default=1, ge=1) class-attribute instance-attribute ") – total number of attempts allowed for the rollout (including the first). * [`retry_condition`](https://microsoft.github.io/agent-lightning/0.2.2/reference/types/#agentlightning.RolloutConfig.retry_condition " retry_condition = Field(default_factory=(cast(Callable[[], List[AttemptStatus]], list))) class-attribute instance-attribute ") – which attempt terminal statuses should trigger a retry (e.g., `["failed", "timeout", "unresponsive"]`). **How it plays out:** The runner works on attempt `k`. If the attempt ends in a status that is listed in `retry_condition`, and `k < max_attempts`, the rollout moves to **requeuing** and the store creates attempt `k+1`. Otherwise, the rollout becomes **failed** (or **succeeded** if the runner marked it so). `timeout_seconds` and `unresponsive_seconds` are enforced by the watchdog and feed into the same decision flow. A minimal example of how to use `RolloutConfig`: `[](https://microsoft.github.io/agent-lightning/0.2.2/deep-dive/store/#__codelineno-0-1) from agentlightning import RolloutConfig [](https://microsoft.github.io/agent-lightning/0.2.2/deep-dive/store/#__codelineno-0-2) [](https://microsoft.github.io/agent-lightning/0.2.2/deep-dive/store/#__codelineno-0-3) # Retry on explicit failures or timeouts, up to 3 attempts in total. [](https://microsoft.github.io/agent-lightning/0.2.2/deep-dive/store/#__codelineno-0-4) cfg = RolloutConfig( [](https://microsoft.github.io/agent-lightning/0.2.2/deep-dive/store/#__codelineno-0-5) timeout_seconds=600, [](https://microsoft.github.io/agent-lightning/0.2.2/deep-dive/store/#__codelineno-0-6) unresponsive_seconds=120, [](https://microsoft.github.io/agent-lightning/0.2.2/deep-dive/store/#__codelineno-0-7) max_attempts=3, [](https://microsoft.github.io/agent-lightning/0.2.2/deep-dive/store/#__codelineno-0-8) retry_condition=["failed", "timeout"] [](https://microsoft.github.io/agent-lightning/0.2.2/deep-dive/store/#__codelineno-0-9) ) [](https://microsoft.github.io/agent-lightning/0.2.2/deep-dive/store/#__codelineno-0-10) [](https://microsoft.github.io/agent-lightning/0.2.2/deep-dive/store/#__codelineno-0-11) # When creating/enqueuing a rollout, attach this config. [](https://microsoft.github.io/agent-lightning/0.2.2/deep-dive/store/#__codelineno-0-12) # The store will propagate attempt outcomes according to cfg. [](https://microsoft.github.io/agent-lightning/0.2.2/deep-dive/store/#__codelineno-0-13) rollout = await store.enqueue_rollout(input, config=cfg)` | Latest attempt status | Rollout transition | Notes / guards | | --- | --- | --- | | N/A | `queuing` | Created by `enqueue_rollout()`. | | `preparing` | `queuing/requeuing` → `preparing` | Typically `dequeue_rollout()` or `start_rollout()`/`start_attempt()` creates a new attempt. | | `running` | `preparing/queuing/requeuing` → `running` | First `add_[otel_]span()` flips the attempt to `running`; rollout follows via `propagate_status`. | | `succeeded` | `*` → `succeeded` | Terminal. Rollout `end_time` set. | | `failed` / `timeout` / `unresponsive` | `*` → `requeuing` | **Only if** `status ∈ retry_condition ∧ sequence_id < max_attempts`. | | `failed` / `timeout` / `unresponsive` | `*` → `failed` | Otherwise (no retries left or retries disabled). | | `*` | `*` → `cancelled` | Explicitly set by `update_rollout(status=cancelled)`. | Why aggregation? In code, we use `propagate_status()` which actively updates the rollout based on the latest attempt. Reading the table above is usually easier than reverse-engineering the propagation logic in the code: think of the rollout’s transitions as _callbacks_ on attempt state changes, plus queue/cancel paths. Spans[¶](https://microsoft.github.io/agent-lightning/0.2.2/deep-dive/store/#spans "Permanent link") ---------------------------------------------------------------------------------------------------- Every traceable operation in a rollout is stored as a [Span](https://microsoft.github.io/agent-lightning/0.2.2/reference/types/#agentlightning.Span " agentlightning.Span") . Spans not only capture fine-grained instrumentation but also act as periodic heartbeats that demonstrate liveness. The first span marks activation; each subsequent one both extends the trace and refreshes the attempt’s [`last_heartbeat_time`](https://microsoft.github.io/agent-lightning/0.2.2/reference/types/#agentlightning.Attempt.last_heartbeat_time " last_heartbeat_time = None class-attribute instance-attribute ") . If no span arrives within the configured [`unresponsive_seconds`](https://microsoft.github.io/agent-lightning/0.2.2/reference/types/#agentlightning.RolloutConfig.unresponsive_seconds " unresponsive_seconds = None class-attribute instance-attribute ") , the watchdog downgrades the attempt to **unresponsive** until activity resumes. Spans are indexed by `(rollout_id, attempt_id, sequence_id)` where the sequence is monotonic. Tracing analysis tools like [Adapter](https://microsoft.github.io/agent-lightning/0.2.2/reference/algorithm/#agentlightning.Adapter " agentlightning.Adapter") usually rely on "time order" to reconstruct the trace. However, in a distributed system, the recorded start time and end time of a span are not necessarily in the right order when they aggregated into a central store. Therefore, we enforce every span creation to retrieve a monotonically increasing [`sequence_id`](https://microsoft.github.io/agent-lightning/0.2.2/reference/types/#agentlightning.Span.sequence_id " sequence_id instance-attribute ") from the store before adding the span. Note In practice, one `sequence_id` can be used to create multiple spans. In that case, the orders between the multiple spans are determined by the order of `start_time` and `end_time` of the spans. ### OpenTelemetry conversion[¶](https://microsoft.github.io/agent-lightning/0.2.2/deep-dive/store/#opentelemetry-conversion "Permanent link") Runners often produce [OpenTelemetry `ReadableSpan`](https://opentelemetry.io/docs/concepts/signals/traces/#attributes) objects directly. The store normalizes them into [`Span`](https://microsoft.github.io/agent-lightning/0.2.2/reference/types/#agentlightning.Span " agentlightning.Span") as follows: 1. The runner first requests [`get_next_span_sequence_id`](https://microsoft.github.io/agent-lightning/0.2.2/reference/store/#agentlightning.LightningStore.get_next_span_sequence_id " get_next_span_sequence_id(rollout_id, attempt_id) async ") via `sequence_id = await store.get_next_span_sequence_id(rollout_id, attempt_id)`. This guarantees ordering within the attempt regardless of clock skew. 2. `trace_id`, `span_id`, `parent_id`, `name`, `status`, timestamps, attributes, events, links, and resource are copied from the OTEL span. Timestamps are auto-normalized to seconds (nanoseconds are converted). 3. OTEL `SpanContext` and parent context are preserved so downstream tools can correlate traces across systems. 4. Any additional serializable fields present on the `ReadableSpan` are retained in the stored span (after safe JSON serialization), which keeps the representation forward-compatible. Programmatically this is encapsulated by [`Span.from_opentelemetry(readable_span, rollout_id, attempt_id, sequence_id)`](https://microsoft.github.io/agent-lightning/0.2.2/reference/types/#agentlightning.Span.from_opentelemetry " from_opentelemetry(src, rollout_id, attempt_id, sequence_id) classmethod ") ; [`store.add_otel_span(...)`](https://microsoft.github.io/agent-lightning/0.2.2/reference/store/#agentlightning.LightningStore.add_otel_span " add_otel_span(rollout_id, attempt_id, readable_span, sequence_id=None) async ") simply wraps the fetch-then-add flow. The end result is a store span that is stable to sort, merge, and query, while still preserving the richness of the original OTEL payload. Tip [`add_span`](https://microsoft.github.io/agent-lightning/0.2.2/reference/store/#agentlightning.LightningStore.add_span " add_span(span) async ") or [`add_otel_span`](https://microsoft.github.io/agent-lightning/0.2.2/reference/store/#agentlightning.LightningStore.add_otel_span " add_otel_span(rollout_id, attempt_id, readable_span, sequence_id=None) async ") both appends a span _and_ acts as a heartbeat that can revive `unresponsive` → `running`. Store Implementations[¶](https://microsoft.github.io/agent-lightning/0.2.2/deep-dive/store/#store-implementations "Permanent link") ------------------------------------------------------------------------------------------------------------------------------------ Currently, the only out-of-the-box implementation is [`InMemoryLightningStore`](https://microsoft.github.io/agent-lightning/0.2.2/reference/store/#agentlightning.InMemoryLightningStore " agentlightning.InMemoryLightningStore") : * Fast startup, zero external dependencies, and ideal for local development, CI, and unit tests. * Fully asyncio-safe for writes; most reader operations can iterate without locks, except those that need to perform multiple queries. * Includes a best-effort span eviction policy once memory crosses a configured watermark; querying evicted spans raises a clear error so callers can fall back. For production you will likely want persistence. We’re actively building a SQLite-backed store that keeps the same API surface while adding durability, crash recovery, and better historical span queries. If you need something sooner, implement your own store by subclassing [`LightningStore`](https://microsoft.github.io/agent-lightning/0.2.2/reference/store/#agentlightning.LightningStore " agentlightning.LightningStore") and providing concrete storage for the small set of abstract methods (`enqueue_rollout`, `dequeue_rollout`, `update_attempt`, `add_span`, etc.). This document plus the tests in `tests/store/` illustrate the expected behavior. Thread Safety[¶](https://microsoft.github.io/agent-lightning/0.2.2/deep-dive/store/#thread-safety "Permanent link") -------------------------------------------------------------------------------------------------------------------- **[`LightningStoreThreaded`](https://microsoft.github.io/agent-lightning/0.2.2/reference/store/#agentlightning.LightningStoreThreaded " agentlightning.LightningStoreThreaded") ** is a subclass of [`LightningStore`](https://microsoft.github.io/agent-lightning/0.2.2/reference/store/#agentlightning.LightningStore " agentlightning.LightningStore") that wraps another underlying store to make a store instance safe for multi-threaded callers. It wraps every state-mutating call in a mutex. Specifically: * Methods like [`start_rollout`](https://microsoft.github.io/agent-lightning/0.2.2/reference/store/#agentlightning.LightningStore.start_rollout " start_rollout(input, mode=None, resources_id=None, config=None, metadata=None) async ") , [`enqueue_rollout`](https://microsoft.github.io/agent-lightning/0.2.2/reference/store/#agentlightning.LightningStore.enqueue_rollout " enqueue_rollout(input, mode=None, resources_id=None, config=None, metadata=None) async ") , [`update_attempt`](https://microsoft.github.io/agent-lightning/0.2.2/reference/store/#agentlightning.LightningStore.update_attempt " update_attempt(rollout_id, attempt_id, status=UNSET, worker_id=UNSET, last_heartbeat_time=UNSET, metadata=UNSET) async ") , [`add_span`](https://microsoft.github.io/agent-lightning/0.2.2/reference/store/#agentlightning.LightningStore.add_span " add_span(span) async ") , etc. are guarded by a lock. * Non-mutating, potentially blocking calls remain pass-through by design (e.g., [`wait_for_rollouts`](https://microsoft.github.io/agent-lightning/0.2.2/reference/store/#agentlightning.LightningStore.wait_for_rollouts " wait_for_rollouts(*, rollout_ids, timeout=None) async ") ), as they don’t modify shared state and should not hold the lock for long periods. Process Safety and Client-server Store[¶](https://microsoft.github.io/agent-lightning/0.2.2/deep-dive/store/#process-safety-and-client-server-store "Permanent link") ---------------------------------------------------------------------------------------------------------------------------------------------------------------------- **[`LightningStoreServer`](https://microsoft.github.io/agent-lightning/0.2.2/reference/store/#agentlightning.LightningStoreServer " agentlightning.LightningStoreServer") ** wraps another underlying store and runs a FastAPI app to expose the store API over HTTP. [`LightningStoreClient`](https://microsoft.github.io/agent-lightning/0.2.2/reference/store/#agentlightning.LightningStoreClient " agentlightning.LightningStoreClient") is a small [`LightningStore`](https://microsoft.github.io/agent-lightning/0.2.2/reference/store/#agentlightning.LightningStore " agentlightning.LightningStore") implementation that talks to the HTTP API. Warning The server HTTP API is not considered a stable API at this moment. Users are encouraged to use the [`LightningStoreClient`](https://microsoft.github.io/agent-lightning/0.2.2/reference/store/#agentlightning.LightningStoreClient " agentlightning.LightningStoreClient") to communicate with the server as a stable interface. The server tracks the creator PID. In the owner process it delegates directly to the in-memory store; in other processes it lazily constructs a [`LightningStoreClient`](https://microsoft.github.io/agent-lightning/0.2.2/reference/store/#agentlightning.LightningStoreClient " agentlightning.LightningStoreClient") to talk to the HTTP API. This prevents accidental cross-process mutation of the wrong memory image. When the server is pickled (e.g., via `multiprocessing`), only the minimal fields are serialized, but **NOT** the FastAPI/uvicorn objects. Subprocesses won’t accidentally carry live server state. Forked subprocess should also use [`LightningStoreClient`](https://microsoft.github.io/agent-lightning/0.2.2/reference/store/#agentlightning.LightningStoreClient " agentlightning.LightningStoreClient") to communicate with the server in the main process. On the client side, the client retries network/5xx failures using a small backoff, and probes `/health` between attempts. Application exceptions inside the server are wrapped as HTTP 400 with a traceback—these are **not retried**. The client also maintains a **per-event-loop** `aiohttp.ClientSession` map so that tracer callbacks (often on separate loops/threads) don’t hang by reusing a session from another loop. Minimal lifecycle: `[](https://microsoft.github.io/agent-lightning/0.2.2/deep-dive/store/#__codelineno-1-1) import agentlightning as agl [](https://microsoft.github.io/agent-lightning/0.2.2/deep-dive/store/#__codelineno-1-2) [](https://microsoft.github.io/agent-lightning/0.2.2/deep-dive/store/#__codelineno-1-3) # Server (owner process) [](https://microsoft.github.io/agent-lightning/0.2.2/deep-dive/store/#__codelineno-1-4) in_memory_store = agl.InMemoryLightningStore() [](https://microsoft.github.io/agent-lightning/0.2.2/deep-dive/store/#__codelineno-1-5) server = agl.LightningStoreServer(store=in_memory_store, host="0.0.0.0", port=4747) [](https://microsoft.github.io/agent-lightning/0.2.2/deep-dive/store/#__codelineno-1-6) await server.start() # starts uvicorn in a daemon thread and waits for /health [](https://microsoft.github.io/agent-lightning/0.2.2/deep-dive/store/#__codelineno-1-7) # or keep your own event loop and stop via await server.stop() [](https://microsoft.github.io/agent-lightning/0.2.2/deep-dive/store/#__codelineno-1-8) # await server.run_forever() [](https://microsoft.github.io/agent-lightning/0.2.2/deep-dive/store/#__codelineno-1-9) [](https://microsoft.github.io/agent-lightning/0.2.2/deep-dive/store/#__codelineno-1-10) # Client (same or different process) [](https://microsoft.github.io/agent-lightning/0.2.2/deep-dive/store/#__codelineno-1-11) client = agl.LightningStoreClient("http://localhost:4747") [](https://microsoft.github.io/agent-lightning/0.2.2/deep-dive/store/#__codelineno-1-12) [](https://microsoft.github.io/agent-lightning/0.2.2/deep-dive/store/#__codelineno-1-13) print(await client.query_rollouts(status=["queuing"])) [](https://microsoft.github.io/agent-lightning/0.2.2/deep-dive/store/#__codelineno-1-14) [](https://microsoft.github.io/agent-lightning/0.2.2/deep-dive/store/#__codelineno-1-15) await client.close() [](https://microsoft.github.io/agent-lightning/0.2.2/deep-dive/store/#__codelineno-1-16) await server.stop()` Another approach is to use a dedicated command line to start a long running server process, possibly sharable across multiple processes. In the main process, you can always use [`LightningStoreClient`](https://microsoft.github.io/agent-lightning/0.2.2/reference/store/#agentlightning.LightningStoreClient " agentlightning.LightningStoreClient") to communicate with the server. `[](https://microsoft.github.io/agent-lightning/0.2.2/deep-dive/store/#__codelineno-2-1) agl store --port 4747` Note [`LightningStoreClient.wait_for_rollouts`](https://microsoft.github.io/agent-lightning/0.2.2/reference/store/#agentlightning.LightningStoreClient.wait_for_rollouts " wait_for_rollouts(*, rollout_ids, timeout=None) async ") intentionally enforces a tiny timeout (≤ 0.1s) to avoid blocking event loops. Poll with short timeouts or compose with `asyncio.wait_for` at a higher layer. Back to top --- # Serving LLM - Agent-lightning [Skip to content](https://microsoft.github.io/agent-lightning/0.2.0/deep-dive/serving-llm/#serving-llms-under-agent-lightning) ⚠️ You are viewing development documentation. For stable documentation, please visit [https://microsoft.github.io/agent-lightning/stable/](https://microsoft.github.io/agent-lightning/stable/) Serving LLMs under Agent-lightning[¶](https://microsoft.github.io/agent-lightning/0.2.0/deep-dive/serving-llm/#serving-llms-under-agent-lightning "Permanent link") ==================================================================================================================================================================== Agent-lightning focuses on data, learning signals, and control flow — **not** on running model inference. This deep dive explains how to **serve** a model alongside Agent-lightning so runners can call it reliably, how the **LLM Proxy** fits into the loop, and why **token IDs** matter if you care about correctness in training and evaluation. General background on LLM serving[¶](https://microsoft.github.io/agent-lightning/0.2.0/deep-dive/serving-llm/#general-background-on-llm-serving "Permanent link") ------------------------------------------------------------------------------------------------------------------------------------------------------------------ [](https://microsoft.github.io/agent-lightning/0.2.0/deep-dive/serving-llm/) Serving a model is essential if you want to train it, especially when you use the model’s own generations as training data. We’ll briefly review the general background to ensure all readers are aligned. Modern LLM servers solve a difficult scheduling problem: keeping GPUs fully utilized while handling prompts of different lengths, streaming tokens as they arrive, and fitting large KV caches into limited memory. Techniques like [**continuous batching**](https://www.anyscale.com/blog/continuous-batching-llm-inference) and [**paged attention**](https://arxiv.org/abs/2309.06180) address these challenges. Continuous batching interleaves decoding across requests to reuse weights efficiently; with careful memory planning, it achieves major throughput gains without increasing latency. PagedAttention reduces KV-cache fragmentation so batching remains effective as sequences grow. See [vLLM’s PagedAttention paper](https://arxiv.org/abs/2309.06180) and [industry analyses](https://www.baseten.co/blog/continuous-vs-dynamic-batching-for-ai-inference/) for details. Balancing inference correctness and efficiency is difficult — a [recent blog](https://thinkingmachines.ai/blog/defeating-nondeterminism-in-llm-inference/) from Thinking Machines Labs highlights how inference nondeterminism ultimately affects training. Beyond scheduling, servers expose an HTTP API, often **OpenAI-compatible** (`/v1/chat/completions` and `/v1/responses`), which is itself a complex stack. In addition to text prompts and chat messages, the API defines many parameters and response fields such as [tool calls](https://platform.openai.com/docs/guides/function-calling) , [structured output](https://platform.openai.com/docs/guides/structured-outputs) , and [multimodal support](https://platform.openai.com/docs/guides/images-vision) . Much effort has been put into implementing all these parameters for many frameworks. Popular engines like **vLLM** and [**SGLang**](https://github.com/sgl-project/sglang) ship with OpenAI-compatible frontends so you can reuse existing client code. [Ollama](https://ollama.com/blog/openai-compatibility) and [llama.cpp](https://llama-cpp-python.readthedocs.io/en/latest/server/) provide similar capabilities. However, because models differ internally, each framework interprets and implements the API slightly differently. Even with identical requests, the tokens passed to the model can vary substantially across frameworks. What Agent-lightning expects from a served LLM[¶](https://microsoft.github.io/agent-lightning/0.2.0/deep-dive/serving-llm/#what-agent-lightning-expects-from-a-served-llm "Permanent link") -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Most of the issues above either have workarounds or remain open research problems. Keep them in mind, but the key question is: what does Agent-lightning expect from a served LLM? The answer includes at least two things: * An OpenAI-compatible **Chat Completions** or **Responses** endpoint the agent can call during rollouts. * Optional training and debugging signals: **logprobs**, **usage**, and ideally **token IDs**. (OpenAI’s public API exposes usage and logprobs, but **not** token IDs — more on [why IDs matter](https://microsoft.github.io/agent-lightning/0.2.0/deep-dive/serving-llm/#token-ids-matter "Token IDs and why they matter") later.) Launching a serving framework[¶](https://microsoft.github.io/agent-lightning/0.2.0/deep-dive/serving-llm/#launching-a-serving-framework "Permanent link") ---------------------------------------------------------------------------------------------------------------------------------------------------------- For many algorithms, you’ll start an engine (e.g., **vLLM** or **SGLang**) before rollouts, then shut it down afterward to free GPU memory. Most frameworks provide a one-line “serve” command to launch the OpenAI-compatible server. You can use those to bring up `/v1/chat/completions` with your checkpoint, ensuring streaming and any required tool-calling features are enabled. A working example is shown in [Unsloth SFT](https://microsoft.github.io/agent-lightning/0.2.0/how-to/unsloth-sft/) . Weight updates — which occur after each training step — are trickier. Some frameworks like [vLLM](https://vllm.ai/) support hot-updating model weights, but it’s usually simpler and more reliable to restart the engine to load new weights. For medium-sized tasks (hundreds of rollouts taking 10+ minutes), the restart overhead (under 30 seconds) is typically negligible. If you’re using Agent-lightning’s [**VERL**](https://microsoft.github.io/agent-lightning/0.2.0/algorithm-zoo/verl/#agentlightning.algorithm.verl.VERL " VERL") integration, the algorithm can **manage the server automatically**. The [VERL framework](https://github.com/volcengine/verl) intelligently allocates compute resources and wraps vLLM/SGLang behind an `AsyncLLMServer` abstraction. You can directly use this as the LLM endpoint for agents. Since VERL can spawn multiple vLLM replicas, using [`LLMProxy`](https://microsoft.github.io/agent-lightning/0.2.0/reference/algorithm/#agentlightning.LLMProxy " agentlightning.LLMProxy") to manage them adds an additional safety layer. A full sequence diagram of how [VERL](https://microsoft.github.io/agent-lightning/0.2.0/algorithm-zoo/verl/#agentlightning.algorithm.verl.VERL " VERL") interacts with the LLM server and proxy is available [here](https://microsoft.github.io/agent-lightning/0.2.0/deep-dive/birds-eye-view/#birds-eye-view-verl-example "Putting It All Together: A Reinforcement Learning Example (VERL)") . LLM Proxy[¶](https://microsoft.github.io/agent-lightning/0.2.0/deep-dive/serving-llm/#llm-proxy "Permanent link") ------------------------------------------------------------------------------------------------------------------ The **LLM Proxy** is a utility class in Agent-lightning, built on [LiteLLM](https://docs.litellm.ai/) , that sits between runners and your backend engine(s) or server(s). In Agent-lightning it acts as a single URL registered as a [`Resource`](https://microsoft.github.io/agent-lightning/0.2.0/reference/types/#agentlightning.Resource " agentlightning.Resource") in the store, offering three key benefits: 1. **Unified endpoint & hot-swaps.** You can redirect traffic between OpenAI, Anthropic, local vLLM/SGLang, or canary checkpoints without modifying agent code — simply repoint the proxy. 2. **First-class tracing.** The proxy emits **OpenTelemetry** spans for every call and sends them to the [`LightningStore`](https://microsoft.github.io/agent-lightning/0.2.0/reference/store/#agentlightning.LightningStore " agentlightning.LightningStore") . It includes rollout and attempt identifiers in request headers so spans are correctly attributed. Sequence numbers are allocated monotonically via the store to [prevent clock-skew issues](https://microsoft.github.io/agent-lightning/0.2.0/tutorials/traces/#distributed-tracing "LLM Proxy") and allow reliable reconstruction of execution trees. 3. **Token IDs.** The proxy can return prompt and response token IDs along with the model output. More details are available in the [next section](https://microsoft.github.io/agent-lightning/0.2.0/deep-dive/serving-llm/#token-ids-matter "Token IDs and why they matter") . Operationally, running the proxy alongside the algorithm works best: the algorithm registers the backend (e.g., the vLLM URL) via [`LLMProxy.update_model_list`](https://microsoft.github.io/agent-lightning/0.2.0/reference/algorithm/#agentlightning.LLMProxy.update_model_list " update_model_list(model_list)") , publishes the proxy URL as a resource via [`LightningStore.add_resources`](https://microsoft.github.io/agent-lightning/0.2.0/reference/store/#agentlightning.LightningStore.add_resources " add_resources(resources) async ") , and runners simply use that URL during rollouts. This mirrors many production client–server setups. Token IDs and why they matter[¶](https://microsoft.github.io/agent-lightning/0.2.0/deep-dive/serving-llm/#token-ids-and-why-they-matter "Permanent link") ---------------------------------------------------------------------------------------------------------------------------------------------------------- [](https://microsoft.github.io/agent-lightning/0.2.0/deep-dive/serving-llm/) This section explains how Agent-lightning handles and uses token IDs — a subtle but important detail for training stability and accuracy. Most agents interact with LLMs via **Chat Completion APIs**, exchanging chat messages. There are two main approaches to collecting training data from such agents. Note Tokenization here refers to the process of converting **Chat Messages** into **Token IDs**. Detokenization is the reverse process of converting **Token IDs** back to **Chat Messages**. Normally, the tokenizer is published along with the pretrained model, which includes a vocabulary, special tokens, and a chat template to dealing with chat messages. **1\. Retokenizing chat messages.** In this approach, you store chat messages as text and let training algorithms **retokenize** them later, as done in many SFT workflows (e.g., [HuggingFace SFT](https://huggingface.co/docs/trl/sft_trainer) ). In practice, we’ve found this method unstable and less accurate. The chart below compares training results. The retokenization approach is run twice. All settings are the same except for the retokenization approach. This instability has three causes. Firstly, chat template used in different frameworks could be slightly different. For example, one single LLaMA model can work with multiple chat templates (multiple in [vLLM](https://github.com/vllm-project/vllm/tree/1d165d6d859d3c50720f0c07209db2363c4fd33b/examples) and one in [HuggingFace](https://huggingface.co/meta-llama) ). It's possible that the chat template used in detokenization is different from the one used in tokenization (this is actually an implementation bug). Secondly, a word might be generated as two tokens (e.g., `H + AVING`) but later retokenized as `HAV + ING`. The text looks identical, but the token IDs differ from what the model originally produced. Thirdly, a generated tool call text like `{ "name": ... }` is parsed by tool call parser into an object that is required by chat completion API. Later, the object is rendered back to `{ "name": ... }` and retokenized again, tool call parsing and re-rendering might cause changes in whitespace and formatting. In some situations, JSON errors may even be auto-corrected by the tool call parser — masking the model’s true generation errors and preventing them from being trained away. * * * **2\. Saving token IDs directly.** The alternative is to save the token IDs generated by the model, as done in RL setups like [Tinker](https://thinkingmachines.ai/tinker/) . This requires a training pipeline that treats tokens as first-class entities, meaning agents must communicate with the inference engine at the token level. However, most agents — especially those built with frameworks like LangChain — rely on OpenAI-compatible APIs and can’t tokenize or detokenize themselves. As mentioned [earlier](https://microsoft.github.io/agent-lightning/0.2.0/deep-dive/serving-llm/#general-llm-serving-background "General background on LLM serving") , implementing this layer manually is complex and error-prone. Some frameworks implement custom solutions (e.g., [VERL Agent Loop](https://github.com/volcengine/verl/blob/4da0d3d3188072772cb2ec817b3d6cf4a463821f/recipe/langgraph_agent/chat_model.py) , [Tinker Renderer](https://github.com/thinking-machines-lab/tinker-cookbook/blob/34a6588d7055040c259985d98e71c0140b389ba7/tinker_cookbook/renderers.py) ), while others leave it to users (e.g., [SkyRL Search-R1](https://novasky-ai.notion.site/skyrl-searchr1) ). * * * A better solution is to use an **OpenAI-compatible API that returns token IDs directly.** This lets agents continue using familiar APIs while capturing token IDs via [tracing](https://microsoft.github.io/agent-lightning/0.2.0/tutorials/traces/) for training. The limitation, of course, is that the serving framework must actually support this capability. When Agent-lightning was first released, we implemented an [instrumented vLLM server](https://github.com/microsoft/agent-lightning/blob/v0.1/agentlightning/instrumentation/vllm.py) that monkey-patched vLLM’s OpenAI server to return token IDs. Since then, the Agent-lightning and vLLM teams have collaborated to add this feature directly to [vLLM core](https://github.com/vllm-project/vllm/pull/22587) . Starting with **vLLM v0.10.2**, the OpenAI-compatible API includes a [`return_token_ids` parameter](https://docs.vllm.ai/en/v0.10.2/serving/openai_compatible_server.html#api-reference) , allowing token IDs to be requested alongside chat messages. SGLang has tracked [similar feature requests](https://github.com/sgl-project/sglang/issues/2634) , though its OpenAI-compatible layer doesn’t yet support them. In short, when using vLLM v0.10.2 or newer, [`LLMProxy`](https://microsoft.github.io/agent-lightning/0.2.0/reference/algorithm/#agentlightning.LLMProxy " agentlightning.LLMProxy") automatically adds `return_token_ids` to each request so the engine includes token IDs in its response. For older vLLM versions, you still need the instrumented version (via `agl vllm` CLI command). Finally, if you only save token IDs in spans, it will have its own limitations — if you train one model using spans from another model with a different tokenizer, incompatibilities can arise. In practice, though, spans in Agent-lightning always store both chat messages and token IDs (actually the full request and response objects), allowing you to fall back to retokenization when necessary. Back to top --- # Bird's Eye View - Agent-lightning [Skip to content](https://microsoft.github.io/agent-lightning/0.2.2/deep-dive/birds-eye-view/#the-birds-eye-view-of-agent-lightning) ⚠️ You are viewing development documentation. For stable documentation, please visit [https://microsoft.github.io/agent-lightning/stable/](https://microsoft.github.io/agent-lightning/stable/) The Bird's Eye View of Agent-lightning[¶](https://microsoft.github.io/agent-lightning/0.2.2/deep-dive/birds-eye-view/#the-birds-eye-view-of-agent-lightning "Permanent link") ============================================================================================================================================================================== High Volume of Information Ahead This article provides an in-depth exploration of the Agent-lightning architecture. It is not intended as a beginner’s guide or usage tutorial. This article summarizes how Agent-lightning (as of v0.2) wires the [Algorithm](https://microsoft.github.io/agent-lightning/0.2.2/reference/algorithm/#agentlightning.Algorithm " agentlightning.Algorithm") , [Runner](https://microsoft.github.io/agent-lightning/0.2.2/reference/runner/#agentlightning.Runner " agentlightning.Runner") , and [LightningStore](https://microsoft.github.io/agent-lightning/0.2.2/reference/store/#agentlightning.LightningStore " agentlightning.LightningStore") loop together and shows where auxiliary components (the [Tracer](https://microsoft.github.io/agent-lightning/0.2.2/reference/runner/#agentlightning.Tracer " agentlightning.Tracer") , [Adapter](https://microsoft.github.io/agent-lightning/0.2.2/reference/algorithm/#agentlightning.Adapter " agentlightning.Adapter") , and [LLM Proxy](https://microsoft.github.io/agent-lightning/0.2.2/reference/algorithm/#agentlightning.LLMProxy " agentlightning.LLMProxy") ) plug into the loop. Each section provides a diagram for a different perspective of the system. Algorithm ↔ Runner ↔ Store data flow[¶](https://microsoft.github.io/agent-lightning/0.2.2/deep-dive/birds-eye-view/#algorithm-runner-store-data-flow "Permanent link") ----------------------------------------------------------------------------------------------------------------------------------------------------------------------- At its heart, Agent-lightning is built on three main components that work in a coordinated loop: * **[Algorithm](https://microsoft.github.io/agent-lightning/0.2.2/reference/algorithm/#agentlightning.Algorithm " agentlightning.Algorithm") :** The "brain" of the system. It decides what tasks to run, learns from the results, and updates resources (like AI models or prompts). * **[Runner](https://microsoft.github.io/agent-lightning/0.2.2/reference/runner/#agentlightning.Runner " agentlightning.Runner") :** The "worker" of the system. It executes tasks assigned by the algorithm, runs the agent, and records the results. * **[LightningStore](https://microsoft.github.io/agent-lightning/0.2.2/reference/store/#agentlightning.LightningStore " agentlightning.LightningStore") :** The central "database" and message queue. It acts as the single source of truth, storing tasks, results, and resources, and enabling communication between the Algorithm and Runner. The typical data flow in a training loop is as follows: The **[Algorithm](https://microsoft.github.io/agent-lightning/0.2.2/reference/algorithm/#agentlightning.Algorithm " agentlightning.Algorithm") ** enqueues tasks (called **[Rollouts](https://microsoft.github.io/agent-lightning/0.2.2/reference/types/#agentlightning.Rollout " agentlightning.Rollout") **) into the **[LightningStore](https://microsoft.github.io/agent-lightning/0.2.2/reference/store/#agentlightning.LightningStore " agentlightning.LightningStore") **. A **[Runner](https://microsoft.github.io/agent-lightning/0.2.2/reference/runner/#agentlightning.Runner " agentlightning.Runner") ** then dequeues a task, executes it, and streams the results (called **[Spans](https://microsoft.github.io/agent-lightning/0.2.2/reference/types/#agentlightning.Span " agentlightning.Span") **) back to the store. Once the task is complete, the algorithm can query the new data from the store to learn and update its resources. The diagram below shows this fundamental interaction in a simple, non-parallel setup. sequenceDiagram autonumber participant Algo as Algorithm participant Store as LightningStore participant Runner participant Agent loop Over the dataset Algo-->>Store: add_resources + enqueue_rollout Store-->>Runner: dequeue_rollout → AttemptedRollout Store-->>Runner: get_latest_resources Runner-->>Store: update_attempt("running", worker_id) Runner->>Agent: rollout + resources Agent->>Runner: reward / spans Runner-->>Store: add_span or add_otel_span Runner-->>Store: update_attempt("finished", status) Store-->>Algo: query_rollouts + spans Algo-->>Algo: Update resources (optional) end Solid lines represent direct calls, while dashed lines are asynchronous or long-running operations. ### Key Terminology[¶](https://microsoft.github.io/agent-lightning/0.2.2/deep-dive/birds-eye-view/#key-terminology "Permanent link") We define the following terms, which may be helpful for understanding the diagram above. * **[Resources](https://microsoft.github.io/agent-lightning/0.2.2/reference/types/#agentlightning.Resource " agentlightning.Resource") :** A collection of assets to be tuned or trained. Agents perform rollouts against resources and collect span data. Algorithms use those data to update the resources. In RL training, the resources are a tunable model. In prompt tuning, the resources are prompt templates. * **[Rollout](https://microsoft.github.io/agent-lightning/0.2.2/reference/types/#agentlightning.Rollout " agentlightning.Rollout") :** A unit of work that an agent performs against a resource. A rollout (noun) can be incomplete, in which case it is also known as a **task**, **sample**, or **job** (these terms are used interchangeably). The agent executes its own defined workflow against the rollout — the process is also called "to rollout" (verb). After execution, the rollout (noun) is considered _complete_. * **[Attempt](https://microsoft.github.io/agent-lightning/0.2.2/reference/types/#agentlightning.Attempt " agentlightning.Attempt") :** A single execution of a rollout. One rollout can have multiple attempts in case of failures or timeouts. * **[Span](https://microsoft.github.io/agent-lightning/0.2.2/reference/types/#agentlightning.Span " agentlightning.Span") :** During the rollout, the agent can generate multiple spans (also known as "traces" or "events"). The recorded spans are collected in the store, which is crucial for understanding agent behavior and optimizing agents. * **[Reward](https://microsoft.github.io/agent-lightning/0.2.2/reference/agent/#agentlightning.emit_reward " agentlightning.emit_reward(reward)") :** A special span that is defined as a number judging the quality of the rollout during some period of the rollout. * **[Dataset](https://microsoft.github.io/agent-lightning/0.2.2/reference/types/#agentlightning.Dataset " agentlightning.Dataset") :** A collection of incomplete rollouts (i.e., tasks) for the agent to process. The dual datasets (train, val) serve as the initial input for the algorithm to enqueue the first batch of rollouts. ### Store[¶](https://microsoft.github.io/agent-lightning/0.2.2/deep-dive/birds-eye-view/#store "Permanent link") As discussed previously, the [LightningStore](https://microsoft.github.io/agent-lightning/0.2.2/reference/store/#agentlightning.LightningStore " agentlightning.LightningStore") is the central hub for all data in Agent-lightning. The store exposes a set of APIs for algorithms and runners to interact with the data; the most important ones are: `[](https://microsoft.github.io/agent-lightning/0.2.2/deep-dive/birds-eye-view/#__codelineno-0-1) from agentlightning.types import AttemptedRollout, ResourcesUpdate, Span, TaskInput [](https://microsoft.github.io/agent-lightning/0.2.2/deep-dive/birds-eye-view/#__codelineno-0-2) [](https://microsoft.github.io/agent-lightning/0.2.2/deep-dive/birds-eye-view/#__codelineno-0-3) class LightningStore: [](https://microsoft.github.io/agent-lightning/0.2.2/deep-dive/birds-eye-view/#__codelineno-0-4) [](https://microsoft.github.io/agent-lightning/0.2.2/deep-dive/birds-eye-view/#__codelineno-0-5) async def enqueue_rollout(self, input: TaskInput, ...) -> Rollout: ... [](https://microsoft.github.io/agent-lightning/0.2.2/deep-dive/birds-eye-view/#__codelineno-0-6) [](https://microsoft.github.io/agent-lightning/0.2.2/deep-dive/birds-eye-view/#__codelineno-0-7) async def dequeue_rollout(self) -> AttemptedRollout | None: ... [](https://microsoft.github.io/agent-lightning/0.2.2/deep-dive/birds-eye-view/#__codelineno-0-8) [](https://microsoft.github.io/agent-lightning/0.2.2/deep-dive/birds-eye-view/#__codelineno-0-9) async def add_span(self, span: Span) -> Span: ... [](https://microsoft.github.io/agent-lightning/0.2.2/deep-dive/birds-eye-view/#__codelineno-0-10) [](https://microsoft.github.io/agent-lightning/0.2.2/deep-dive/birds-eye-view/#__codelineno-0-11) async def get_latest_resources(self) -> Optional[ResourcesUpdate]: ... [](https://microsoft.github.io/agent-lightning/0.2.2/deep-dive/birds-eye-view/#__codelineno-0-12) [](https://microsoft.github.io/agent-lightning/0.2.2/deep-dive/birds-eye-view/#__codelineno-0-13) async def wait_for_rollouts(self, rollout_ids: List[str], ...): ... [](https://microsoft.github.io/agent-lightning/0.2.2/deep-dive/birds-eye-view/#__codelineno-0-14) [](https://microsoft.github.io/agent-lightning/0.2.2/deep-dive/birds-eye-view/#__codelineno-0-15) async def query_spans(self, rollout_id: str, ...): ... [](https://microsoft.github.io/agent-lightning/0.2.2/deep-dive/birds-eye-view/#__codelineno-0-16) [](https://microsoft.github.io/agent-lightning/0.2.2/deep-dive/birds-eye-view/#__codelineno-0-17) async def update_attempt(self, rollout_id: str, attempt_id: str, status: str, ...): ... [](https://microsoft.github.io/agent-lightning/0.2.2/deep-dive/birds-eye-view/#__codelineno-0-18) [](https://microsoft.github.io/agent-lightning/0.2.2/deep-dive/birds-eye-view/#__codelineno-0-19) ...` These interfaces operate on [`AttemptedRollout`](https://microsoft.github.io/agent-lightning/0.2.2/reference/types/#agentlightning.AttemptedRollout " agentlightning.AttemptedRollout") , [`ResourcesUpdate`](https://microsoft.github.io/agent-lightning/0.2.2/reference/types/#agentlightning.ResourcesUpdate " agentlightning.ResourcesUpdate") , [`Span`](https://microsoft.github.io/agent-lightning/0.2.2/reference/types/#agentlightning.Span " agentlightning.Span") , and [`TaskInput`](https://microsoft.github.io/agent-lightning/0.2.2/reference/internal/#agentlightning.TaskInput " agentlightning.TaskInput = Any module-attribute ") instances from `agentlightning.types`. As the APIs show, the store essentially provides a queue for rollouts and storage for resources, spans, and attempts. Developers should implement the store carefully to ensure data integrity and consistency, especially when multiple runners work in parallel across multiple attempts. The store is designed to be extensible. Users can implement their own store by inheriting from [`LightningStore`](https://microsoft.github.io/agent-lightning/0.2.2/reference/store/#agentlightning.LightningStore " agentlightning.LightningStore") and overriding methods. Agent-lightning provides a few reference implementations, such as [`InMemoryLightningStore`](https://microsoft.github.io/agent-lightning/0.2.2/reference/store/#agentlightning.InMemoryLightningStore " agentlightning.InMemoryLightningStore") (default) and `SqliteLightningStore` (under construction). When parallelized, the store may need special wrappers to ensure thread/process safety or delegate computation to a store in another process or machine. Supporting Components in the Loop[¶](https://microsoft.github.io/agent-lightning/0.2.2/deep-dive/birds-eye-view/#supporting-components-in-the-loop "Permanent link") --------------------------------------------------------------------------------------------------------------------------------------------------------------------- While the core loop is simple, Agent-lightning provides several components to make development easier and more powerful. ### Tracer[¶](https://microsoft.github.io/agent-lightning/0.2.2/deep-dive/birds-eye-view/#tracer "Permanent link") The [`Tracer`](https://microsoft.github.io/agent-lightning/0.2.2/reference/runner/#agentlightning.Tracer " agentlightning.Tracer") is a component within the [`Runner`](https://microsoft.github.io/agent-lightning/0.2.2/reference/runner/#agentlightning.Runner " agentlightning.Runner") that records detailed spans (events) during an agent's execution and sends them to the [`LightningStore`](https://microsoft.github.io/agent-lightning/0.2.2/reference/store/#agentlightning.LightningStore " agentlightning.LightningStore") . Instead of requiring the agent to manually log every span, the tracer automatically instruments key methods (e.g., LLM calls) and captures their inputs, outputs, and metadata. This provides a detailed log of the agent's behavior with minimal effort. sequenceDiagram autonumber participant Store participant Runner participant Tracer participant Agent Note over Runner,Tracer: Runner manages tracer as member Tracer->>Agent: Apply instrumentation loop Until no more rollouts Store-->>Runner: dequeue_rollout → AttemptedRollout Store-->>Runner: get_latest_resources Runner->>Agent: training_rollout / validation_rollout loop For each finished span Agent-->>Tracer: openai.chat.completion invoked
agent.execute invoked
... Agent->>Tracer: emit intermediate reward Tracer-->>Store: add_otel_span(rollout_id, attempt_id, span) end Agent->>Runner: final reward + extra spans (if any) Runner-->>Store: add_span(rollout_id, attempt_id, span) Runner-->>Store: update_attempt(status) end Tracer->>Agent: Unapply instrumentation The above diagram shows the overall data flow between store, tracer and agent. In realistic, it's a bit more complicated than that. Spans are not emitted actively by the agent; they are intercepted by the tracer by hooking and instrumenting key methods used in the agents. The tracer uses a callback (called exporter) to monitor events and log to the store. Before a rollout starts, the runner enters a [`trace_context`](https://microsoft.github.io/agent-lightning/0.2.2/reference/runner/#agentlightning.Tracer.trace_context " trace_context(name=None, *, store=None, rollout_id=None, attempt_id=None)") before invoking the agent, wiring store identifiers into the tracer. Each span completion streams back to the store through `LightningSpanProcessor`, so the agent’s instrumentation lands in [`add_otel_span`](https://microsoft.github.io/agent-lightning/0.2.2/reference/store/#agentlightning.LightningStore.add_otel_span " add_otel_span(rollout_id, attempt_id, readable_span, sequence_id=None) async ") . If the agent’s rollout method returns a numeric reward, the runner emits one more OpenTelemetry span before finalizing the attempt. ### Hooks[¶](https://microsoft.github.io/agent-lightning/0.2.2/deep-dive/birds-eye-view/#hooks "Permanent link") [`Hook`](https://microsoft.github.io/agent-lightning/0.2.2/reference/types/#agentlightning.Hook " agentlightning.Hook") implementations are user-defined callback functions that allow you to augment a [`Runner`](https://microsoft.github.io/agent-lightning/0.2.2/reference/runner/#agentlightning.Runner " agentlightning.Runner") 's behavior at specific points in its lifecycle. You can use hooks to add custom logging, set up resources before a rollout begins, or tear them down after it ends. Hooks can be triggered at four key moments: `on_rollout_start`, `on_trace_start`, `on_trace_end`, and `on_rollout_end`. Users should pay special attention to the difference between `on_trace_end` and `on_rollout_end`. The former is called right before the tracer exits the trace context, while the latter is called after the runner processes the final leftover rewards and spans, and finalizes the attempt in the store. sequenceDiagram autonumber participant Store participant Hooks participant Runner participant Tracer participant Agent Note over Runner,Hooks: Runner manages hooks as member loop Until no more rollouts Store-->>Runner: dequeue_rollout → AttemptedRollout Store-->>Runner: get_latest_resources Runner->>Hooks: on_rollout_start(agent, runner, rollout) Runner->>Agent: training_rollout / validation_rollout Tracer->>Agent: enter_trace_context activate Tracer Runner->>Hooks: on_trace_start(agent, runner, tracer, rollout) Note over Runner,Agent: Agent rollout omitted Runner->>Hooks: on_trace_end(agent, runner, tracer, rollout) Tracer->>Agent: exit_trace_context deactivate Tracer Agent->>Runner: final reward + extra spans (if any) Runner-->>Store: add_span(rollout_id, attempt_id, span) Runner->>Hooks: on_rollout_end(agent, runner, rollout, status) end ### Adapter[¶](https://microsoft.github.io/agent-lightning/0.2.2/deep-dive/birds-eye-view/#adapter "Permanent link") The [`Adapter`](https://microsoft.github.io/agent-lightning/0.2.2/reference/algorithm/#agentlightning.Adapter " agentlightning.Adapter") is a component used by the [`Algorithm`](https://microsoft.github.io/agent-lightning/0.2.2/reference/algorithm/#agentlightning.Algorithm " agentlightning.Algorithm") to transform raw data from the [`LightningStore`](https://microsoft.github.io/agent-lightning/0.2.2/reference/store/#agentlightning.LightningStore " agentlightning.LightningStore") into a format suitable for learning. Runners stream raw spans into the store during execution. Later, the algorithm queries these spans and uses an adapter to convert them into structured data, like training examples for a reinforcement learning model. For instance, the [`TracerTraceToTriplet`](https://microsoft.github.io/agent-lightning/0.2.2/reference/algorithm/#agentlightning.TracerTraceToTriplet " agentlightning.TracerTraceToTriplet") processes OpenTelemetry spans to create `(prompt, response, reward)` triplets, which are the fundamental data structure for many RL fine-tuning algorithms. flowchart LR Runner -- (1) add_otel_span --> Store Store -- (2) query_spans --> Algorithm Algorithm -- (3) spans --> Adapter Adapter -- (4) transformed data --> Algorithm ### LLM Proxy[¶](https://microsoft.github.io/agent-lightning/0.2.2/deep-dive/birds-eye-view/#llm-proxy "Permanent link") The [`LLMProxy`](https://microsoft.github.io/agent-lightning/0.2.2/reference/algorithm/#agentlightning.LLMProxy " agentlightning.LLMProxy") is an optional bridge component that sits between an agent and the algorithms' resources. It acts as a centralized endpoint for all LLM calls. Usually the proxy URL is added to the store as a special resource, so that the [`Runner`](https://microsoft.github.io/agent-lightning/0.2.2/reference/runner/#agentlightning.Runner " agentlightning.Runner") can fetch it along with other resources when dequeuing a rollout. During rollouts, the runner invokes the proxy's HTTP endpoint instead of calling a model backend directly. This design offers several benefits: 1. **Instrumentation:** It automatically captures detailed traces of LLM interactions (prompts, responses, metadata) and sends them to the store, complementing the tracer, especially when the agent's code is hard to instrument directly. 2. **Backend Abstraction:** It provides a unified interface for various LLM backends (OpenAI, Anthropic, local models) and can add features like retry logic, rate limiting, and caching. 3. **Resource Management:** The algorithm can dynamically update which LLM the agent uses (e.g., swapping to a newly fine-tuned model) by simply swapping the backend model the proxy is using, without interrupting the agent's code. The benefits above seem to be all discussed within the context of model fine-tuning. As a matter of fact, the proxy can be useful for prompt tuning as well. The algorithm can register one of the following two types of endpoints into the proxy: 1. **Endpoint served by the algorithm:** If the algorithm is internally updating the LLM weights (e.g., RL), it can launch an LLM inference engine (i.e., a model server) and register the endpoint URL with the proxy. The proxy then forwards all LLM calls to that endpoint. 2. **Third-party LLM endpoint:** If the algorithm is not updating the LLM weights (e.g., prompt tuning), it can register a third-party LLM endpoint into the proxy. We show a diagram below that illustrates how the proxy fits into the overall data flow. sequenceDiagram autonumber participant Algo as Algorithm participant LLMProxy as LLM Proxy participant Store participant Runner participant Agent Note over Algo,LLMProxy: Algorithm manages LLMProxy as member loop Over the Dataset Algo->>Algo: Launch LLM Inference Engine
(optional) Algo->>LLMProxy: Register Inference Engine
(optional) Algo-->>Store: enqueue_rollout LLMProxy->>Store: Proxy URL added as Resource Store-->>Runner: dequeue_rollout → AttemptedRollout Store-->>Runner: get_latest_resources Runner->>Agent: rollout + resources
(LLM Proxy URL as resource) loop Defined by Agent Agent-->>LLMProxy: LLM calls activate LLMProxy LLMProxy-->>Store: add_span or add_otel_span LLMProxy-->>Agent: LLM responses deactivate LLMProxy Agent-->>Runner: rewards Runner-->>Store: add_span or add_otel_span end Runner-->>Store: update_attempt("finished", status) Store-->>Algo: query_rollouts + spans Algo-->>Algo: Update LLM Weights
(optional) end In this diagram, the store receives spans from both the proxy and the runner. We will see a problem later with parallelism where the proxy and runner are in different machines, and spans need to obtain a special counter from the store to ensure the ordering of spans. ### Trainer[¶](https://microsoft.github.io/agent-lightning/0.2.2/deep-dive/birds-eye-view/#trainer "Permanent link") The [Trainer](https://microsoft.github.io/agent-lightning/0.2.2/reference/trainer/#agentlightning.Trainer " agentlightning.Trainer") is the high-level orchestrator that initializes and connects all major components -- [Algorithm](https://microsoft.github.io/agent-lightning/0.2.2/reference/algorithm/#agentlightning.Algorithm " agentlightning.Algorithm") , [Runner](https://microsoft.github.io/agent-lightning/0.2.2/reference/runner/#agentlightning.Runner " agentlightning.Runner") , [LightningStore](https://microsoft.github.io/agent-lightning/0.2.2/reference/store/#agentlightning.LightningStore " agentlightning.LightningStore") , [Tracer](https://microsoft.github.io/agent-lightning/0.2.2/reference/runner/#agentlightning.Tracer " agentlightning.Tracer") , [Adapter](https://microsoft.github.io/agent-lightning/0.2.2/reference/algorithm/#agentlightning.Adapter " agentlightning.Adapter") , [LLM Proxy](https://microsoft.github.io/agent-lightning/0.2.2/reference/algorithm/#agentlightning.LLMProxy " agentlightning.LLMProxy") , and [Hook](https://microsoft.github.io/agent-lightning/0.2.2/reference/types/#agentlightning.Hook " agentlightning.Hook") . The components can have a lifecycle as long as the trainer. The trainer manages their lifecycles and handles dependency injection, ensuring that every part of the system operates within a consistent and shared environment. Below, we demonstrate how the components relate to each other and their roles. We first clarify the roles and relationships shown in the diagram: 1. **Owns:** components that the trainer constructs and manages directly (e.g., runner, tracer). 2. **Injects:** components passed into others as dependencies. 3. **References:** weak links for coordination without ownership. 4. **Uses:** components that are temporarily interacted with. For example, the [LightningStore](https://microsoft.github.io/agent-lightning/0.2.2/reference/store/#agentlightning.LightningStore " agentlightning.LightningStore") is injected into the [Algorithm](https://microsoft.github.io/agent-lightning/0.2.2/reference/algorithm/#agentlightning.Algorithm " agentlightning.Algorithm") and [Runner](https://microsoft.github.io/agent-lightning/0.2.2/reference/runner/#agentlightning.Runner " agentlightning.Runner") . The [Tracer](https://microsoft.github.io/agent-lightning/0.2.2/reference/runner/#agentlightning.Tracer " agentlightning.Tracer") and [LitAgent](https://microsoft.github.io/agent-lightning/0.2.2/reference/agent/#agentlightning.LitAgent " agentlightning.LitAgent") are injected into the runner. The [Adapter](https://microsoft.github.io/agent-lightning/0.2.2/reference/algorithm/#agentlightning.Adapter " agentlightning.Adapter") and [LLM Proxy](https://microsoft.github.io/agent-lightning/0.2.2/reference/algorithm/#agentlightning.LLMProxy " agentlightning.LLMProxy") are injected into the algorithm. The store is further injected into the tracer, adapter and LLM proxy by the runner and algorithm respectively. flowchart TD %% === Left side: Algorithm domain === subgraph L["Algorithm Side"] Algorithm["Algorithm
(no default)"] Adapter["Adapter
(TracerTraceToTriplet*)"] LLMProxy["LLM Proxy
(no default)"] Algorithm -.injects.-> Adapter Algorithm -.injects.-> LLMProxy end linkStyle 0,1 stroke:#896978,stroke-width:2px; %% === Middle: Core trainer and store === subgraph M["Core"] Trainer["Trainer"] Store["LightningStore
(InMemory* default)"] Trainer --has--> Algorithm Trainer --has--> Store Trainer --has--> Adapter Trainer --has--> LLMProxy end linkStyle 2,3,4,5 stroke:#839791,stroke-width:2px; %% === Right side: Runner side === subgraph R["Runner Side"] Runner["Runner
(LitAgentRunner* default)"] Tracer["Tracer
(AgentOpsTracer*)"] Hooks["Hooks (empty default)"] Agent["Agent
(LitAgent*)"] Runner -.injects.-> Tracer Runner -.injects.-> Store Runner -.injects.-> Agent Runner -.injects.-> Hooks Tracer -.injects.-> Store Hooks -.uses.-> Runner Hooks -.uses.-> Agent Hooks -.uses.-> Tracer end linkStyle 6,7,8,9,10 stroke:#896978,stroke-width:2px; linkStyle 11,12,13 stroke:#7a89c2,stroke-width:2px; %% === Cross-section connections === Trainer --has--> Runner Trainer --has--> Tracer Trainer --has--> Hooks Trainer --uses--> Agent Algorithm -.injects.-> Store LLMProxy -.injects.-> Store Agent -.references.-> Trainer Runner -.references.-> Trainer Algorithm -.references.-> Trainer linkStyle 14,15,16 stroke:#839791,stroke-width:2px; linkStyle 17,20,21,22 stroke:#7a89c2,stroke-width:2px; linkStyle 18,19 stroke:#896978,stroke-width:2px; style L fill:none; style M fill:none; style R fill:none; Putting It All Together: A Reinforcement Learning Example (VERL)[¶](https://microsoft.github.io/agent-lightning/0.2.2/deep-dive/birds-eye-view/#putting-it-all-together-a-reinforcement-learning-example-verl "Permanent link") -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- [](https://microsoft.github.io/agent-lightning/0.2.2/deep-dive/birds-eye-view/) VERL shows how an algorithm consumes the shared infrastructure. For historical reasons, code lives in `agentlightning.algorithm.verl` and `agentlightning.verl`. The latter is legacy and reuses terms like `Trainer` in confusing ways. The former is a thin wrapper that conforms to the new algorithm interface. Future versions will merge the two. Reinforcement learning aims to learn a policy that takes actions in states to maximize expected reward. For agents, the policy is usually a language model. Inputs are prompts (state). Outputs are generated text (action). A numeric score judges quality (reward). The `(state, action, reward)` **triplet** is the basic learning unit. In Agent-lightning, the environment is implicit in the agent’s workflow, which orchestrates one or more LLM calls and often self-judges using rules or additional model calls. During a rollout, the agent emits spans that contain everything needed for RL training, including LLM call traces and numeric judge/reward signals. The "algorithm", on the other hand, have more responsibilities. 1. Providing a language model deployment that is currently learning and improving for the agent to interact with; 2. Preparing the tasks that the agents will perform; 3. Querying the spans generated, extracting triplets, and converting them into a format that the underlying RL library can consume; 4. Updating the language model based on the learning signals. In the VERL integration, the algorithm launches a chat completion endpoint using `vLLM` and wraps training with `FSDP` for distributed optimization. It enqueues tasks from the dataset. After rollouts finish, it queries spans and converts them to triplets with `TracerTraceToTriplet`. VERL’s native training loop then consumes these triplets to update model weights. The workflow can be summarized in the following diagram. sequenceDiagram autonumber participant vLLM as vLLM Chat
Completion Endpoint participant FSDP as FSDP / Megatron
Weights Optimizer participant Algo as Algorithm
Main Controller
(Main Process) participant Adapter as TracerTraceToTriplet participant LLMProxy as LLM Proxy participant Store as LightningStore participant Runner as Runner + Agent Note over Algo,LLMProxy: LLMProxy and Adapter are injected by Trainer as member Note over vLLM,Algo: Algorithm creates and owns vLLM and FSDP loop Over the Dataset in Batches Algo->>vLLM: Create Chat Completion Endpoint activate vLLM vLLM->>LLMProxy: Registered as Backend Endpoint LLMProxy->>Store: Proxy URL added as Resource par Over data samples in the batch Algo-->>Store: enqueue_rollout Store-->>Runner: Dequeue Rollout +
Resources (i.e., URL) loop One Rollout Attempt Runner-->>LLMProxy: LLM calls LLMProxy-->>vLLM: Forwarded LLM calls vLLM-->>LLMProxy: LLM responses LLMProxy-->>Store: add_span / add_otel_span LLMProxy-->>Runner: Forwarded LLM responses Runner-->>Store: add_span / add_otel_span
(by tracer, including rewards) end Runner-->>Store: update_attempt("finished", status) end Algo-->>Store: Poll for completed rollouts + spans Algo->>vLLM: Chat Completion Endpoint Sleeps deactivate vLLM Algo->>Adapter: adapt(spans) Adapter->>FSDP: Triplets (state, action, reward) activate FSDP FSDP-->>Algo: Updated LLM weights deactivate FSDP end **Notes:** 1. There are interactions between different components injected into or owned by algorithms in the diagram, such as the output of the adapter feeding into the FSDP optimizer. This is for simplicity of illustration and slightly different from the actual implementation, where it's the algorithm main controller that orchestrates the data flow between components. 2. **On mapping to VERL.** VERL uses a classic RLHF setup where each action is a single token, the state is the full conversation history up to that token, and reward is given at the end. This is very different from our setup where each action is actually a chunk of text, although they are both called RL! Therefore, after the adapter produces triplets, the algorithm converts each `(state, action, reward)` into a VERL trajectory (`DataProto`) with keys like `input_ids`, `position_ids`, `attention_mask`, and `token_level_scores`. That conversion happens after triplet generation and is not shown in the diagram. Execution Strategies and Parallelism[¶](https://microsoft.github.io/agent-lightning/0.2.2/deep-dive/birds-eye-view/#execution-strategies-and-parallelism "Permanent link") --------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Readers might have observed from the diagram above that there is absolutely no communication between (1) runner and agents and (2) algorithm. The only overlap of them is the [Trainer](https://microsoft.github.io/agent-lightning/0.2.2/reference/trainer/#agentlightning.Trainer " agentlightning.Trainer") and [LightningStore](https://microsoft.github.io/agent-lightning/0.2.2/reference/store/#agentlightning.LightningStore " agentlightning.LightningStore") . This observation is very clear with the diagram within the trainer section. This design allows us to flexibly scale the runner and algorithm independently, which is crucial for large-scale training. Agent-lightning packages two executable bundles: a runner bundle ([Runner](https://microsoft.github.io/agent-lightning/0.2.2/reference/runner/#agentlightning.Runner " agentlightning.Runner") , [Tracer](https://microsoft.github.io/agent-lightning/0.2.2/reference/runner/#agentlightning.Tracer " agentlightning.Tracer") , [Hook](https://microsoft.github.io/agent-lightning/0.2.2/reference/types/#agentlightning.Hook " agentlightning.Hook") , [LitAgent](https://microsoft.github.io/agent-lightning/0.2.2/reference/agent/#agentlightning.LitAgent " agentlightning.LitAgent") ) and an algorithm bundle ([Algorithm](https://microsoft.github.io/agent-lightning/0.2.2/reference/algorithm/#agentlightning.Algorithm " agentlightning.Algorithm") , [Adapter](https://microsoft.github.io/agent-lightning/0.2.2/reference/algorithm/#agentlightning.Adapter " agentlightning.Adapter") , [LLM Proxy](https://microsoft.github.io/agent-lightning/0.2.2/reference/algorithm/#agentlightning.LLMProxy " agentlightning.LLMProxy") ). Both share the [LightningStore](https://microsoft.github.io/agent-lightning/0.2.2/reference/store/#agentlightning.LightningStore " agentlightning.LightningStore") . The trainer initializes and connects the bundles. graph TD subgraph Runner_Side["Runner Bundle"] direction LR R[Runner] --- T[Tracer] --- H[Hooks] --- A1[Agent] end subgraph Algorithm_Side["Algorithm Bundle"] direction LR ALG[Algorithm] --- AD[Adapter] --- LLM[LLM Proxy] end S[(Store)] TR[Trainer] Runner_Side <--> S Algorithm_Side <--> S TR --> Runner_Side TR --> Algorithm_Side linkStyle 0,1,2,3,4 opacity:0; An [execution strategy](https://microsoft.github.io/agent-lightning/0.2.2/reference/trainer/#agentlightning.ExecutionStrategy " agentlightning.ExecutionStrategy") , defined and owned by the trainer, governs how algorithm and runner bundles are placed, connected, scaled, and aborted. It serves four primary purposes. Execution strategies first determine **bundle placement** — whether the two bundles run in the same thread, process, machine, or across separate machines. They also define **store management**, wrapping the store and specifying how data is shared between bundles. In terms of **scalability**, the strategy can replicate the runner bundle across multiple threads, processes, or machines to expand throughput on the runner side. The algorithm side remains single-process due to the complexity of parallelization. Mature frameworks such as _DeepSpeed_ and _Megatron_ already support distributed model training, so scaling of the algorithm bundle is delegated to those implementations. **Abort handling** is another core responsibility. Aborts may be triggered by normal exits, failures in either bundle, or user interrupts. The trainer must include cancellation interfaces for the bundles so that bundles can be cleanly aborted. When the algorithm bundle exits normally, the strategy signals the runner bundle to terminate. If the runner exits first, no signal is sent to the algorithm, as it may still be processing completed rollouts. In cases of failure or user interruption, the strategy signals both bundles to abort; if a bundle fails to respond, the strategy should attempt a forceful termination. Agent-lightning currently provides two execution strategies: **shared-memory** and **client-server**, described in the following sections. ### Shared-memory Strategy[¶](https://microsoft.github.io/agent-lightning/0.2.2/deep-dive/birds-eye-view/#shared-memory-strategy "Permanent link") [`SharedMemoryExecutionStrategy`](https://microsoft.github.io/agent-lightning/0.2.2/reference/trainer/#agentlightning.SharedMemoryExecutionStrategy " agentlightning.SharedMemoryExecutionStrategy") runs algorithm and runner bundles as threads in one process. The strategy wraps the store with [`LightningStoreThreaded`](https://microsoft.github.io/agent-lightning/0.2.2/reference/store/#agentlightning.LightningStoreThreaded " agentlightning.LightningStoreThreaded") , which guards calls with a lock for safe concurrency. This is good for lightweight debugging because components share one Python heap and avoid serialization. It is not suitable for heavy RL training or compute-intensive agents. flowchart TB subgraph MainProcess direction TB subgraph AlgorithmThread [Thread 0] Algorithm[Algorithm bundle] end subgraph RunnerThread1 [Thread 1] Runner1[Runner bundle #1] end subgraph RunnerThread2 [Thread 2] Runner2[Runner bundle #2] end subgraph RunnerThread3 [Thread 3] RunnerN[Runner bundle #N] end LightningStoreFacade[LightningStoreThreaded] BaseStore[Underlying LightningStore] end Algorithm -- async calls --> LightningStoreFacade Runner1 -- async calls --> LightningStoreFacade Runner2 -- async calls --> LightningStoreFacade RunnerN -- async calls --> LightningStoreFacade LightningStoreFacade -->|thread-safe delegates| BaseStore You can configure which role runs on the main thread. If the main thread runs the algorithm, it is able to spawn multiple runner threads. If it runs a runner, `n_runners` must be 1 and the runner lives on the main thread. ### Client-server Strategy[¶](https://microsoft.github.io/agent-lightning/0.2.2/deep-dive/birds-eye-view/#client-server-strategy "Permanent link") [](https://microsoft.github.io/agent-lightning/0.2.2/deep-dive/birds-eye-view/) [`ClientServerExecutionStrategy`](https://microsoft.github.io/agent-lightning/0.2.2/reference/trainer/#agentlightning.ClientServerExecutionStrategy " agentlightning.ClientServerExecutionStrategy") splits concerns across processes. The algorithm bundle starts a [`LightningStoreServer`](https://microsoft.github.io/agent-lightning/0.2.2/reference/store/#agentlightning.LightningStoreServer " agentlightning.LightningStoreServer") (HTTP API) that wraps the underlying store. Runners connect via [`LightningStoreClient`](https://microsoft.github.io/agent-lightning/0.2.2/reference/store/#agentlightning.LightningStoreClient " agentlightning.LightningStoreClient") to call the same interface over REST. The server embeds a client to support algorithm-launched subprocesses (e.g., an LLM proxy worker) that need to talk back to the algorithm’s process through the same API. Currently this design introduces an extra wrapper in the Server side (as shown in the diagram), which helps debugging and improves fault tolerance. We might revisit this design in the future and enforce the client to be the only way to communicate with the store. flowchart TD subgraph Algorithm Process Group subgraph StoreServer[LightningStoreServer] StoreHttpClient[HTTP Client] StoreHttpServer[HTTP Server] StoreWrapper[LightningStore Wrapper] StoreHttpClient -- HTTP --> StoreHttpServer end subgraph Algorithm Bundle Algorithm[Algorithm Main Process] subgraph Another subprocess LLMProxy[LLM Proxy] end end LLMProxy -- async calls --> StoreHttpClient Algorithm -- async calls --> StoreWrapper end subgraph RunnerSide ["Runner Side"] subgraph Runner Process 1 Runner1[Runner bundle #1] Runner1 -- async calls --> LightningStoreClient1 LightningStoreClient1[LightningStoreClient] end subgraph Runner Process 2 Runner2[Runner bundle #2] Runner2 -- async calls --> LightningStoreClient2 LightningStoreClient2[LightningStoreClient] end subgraph Runner Process N RunnerN[Runner bundle #N] RunnerN -- async calls --> LightningStoreClientN LightningStoreClientN[LightningStoreClient] end end LocalStore[Underlying LightningStore] StoreHttpServer -->|delegates| StoreWrapper StoreWrapper -->|delegates| LocalStore LightningStoreClient1 -- HTTP --> StoreHttpServer LightningStoreClient2 -- HTTP --> StoreHttpServer LightningStoreClientN -- HTTP --> StoreHttpServer style RunnerSide fill:none; Online/Continuous Learning[¶](https://microsoft.github.io/agent-lightning/0.2.2/deep-dive/birds-eye-view/#onlinecontinuous-learning "Permanent link") ------------------------------------------------------------------------------------------------------------------------------------------------------ Continuous learning keeps the algorithm loop running while runners report tasks and spans opportunistically. Key differences from batch mode: 1. The algorithm does not enqueue rollouts from a fixed dataset. Runners report tasks/rollouts and spans spontaneously. 2. The algorithm can wait for rollouts with a expected set of rollout IDs, but more often polls for new rollouts and spans or waits for a count to arrive. 3. The [`Runner`](https://microsoft.github.io/agent-lightning/0.2.2/reference/runner/#agentlightning.Runner " agentlightning.Runner") processes one rollout at a time via [`step(task)`](https://microsoft.github.io/agent-lightning/0.2.2/reference/runner/#agentlightning.Runner.step " step(input, *, resources=None, mode=None, event=None) async ") instead of exhausting a task queue. It notifies the store when starting a rollout so the store records it. 4. A user or higher-level loop controls which resources the next step uses and when to retry. [Spans](https://microsoft.github.io/agent-lightning/0.2.2/reference/types/#agentlightning.Span " agentlightning.Span") , [Adapter](https://microsoft.github.io/agent-lightning/0.2.2/reference/algorithm/#agentlightning.Adapter " agentlightning.Adapter") implementations, and the [LLM Proxy](https://microsoft.github.io/agent-lightning/0.2.2/reference/algorithm/#agentlightning.LLMProxy " agentlightning.LLMProxy") work the same way. sequenceDiagram autonumber actor User participant Runner participant Agent participant Store as LightningStore participant Algorithm Note over Algorithm: Algorithm is long-running and loops continuously loop Continuous Learning Loop activate User opt Decide what to do next User-->>Store: get_resources_by_id Store-->>User: Resources User-->>User: Prepare input for next step end User->>Runner: step(input, resources) activate Runner Runner-->>Store: Notify: start_rollout(input) Runner->>Agent: rollout(input, resources) Agent-->>Runner: add_span / reward spans Runner-->>Store: add_span or add_otel_span Runner-->>Store: update_attempt(status="finished") deactivate Runner deactivate User Algorithm->>Store: poll for new rollouts and spans opt If there is enough new data Store-->>Algorithm: new spans Algorithm->>Algorithm: adapt spans → learning signal Algorithm->>Store: update_resources end end Back to top --- # Serving LLM - Agent-lightning [Skip to content](https://microsoft.github.io/agent-lightning/0.2.1/deep-dive/serving-llm/#serving-llms-under-agent-lightning) ⚠️ You are viewing development documentation. For stable documentation, please visit [https://microsoft.github.io/agent-lightning/stable/](https://microsoft.github.io/agent-lightning/stable/) Serving LLMs under Agent-lightning[¶](https://microsoft.github.io/agent-lightning/0.2.1/deep-dive/serving-llm/#serving-llms-under-agent-lightning "Permanent link") ==================================================================================================================================================================== Agent-lightning focuses on data, learning signals, and control flow — **not** on running model inference. This deep dive explains how to **serve** a model alongside Agent-lightning so runners can call it reliably, how the **LLM Proxy** fits into the loop, and why **token IDs** matter if you care about correctness in training and evaluation. General background on LLM serving[¶](https://microsoft.github.io/agent-lightning/0.2.1/deep-dive/serving-llm/#general-background-on-llm-serving "Permanent link") ------------------------------------------------------------------------------------------------------------------------------------------------------------------ [](https://microsoft.github.io/agent-lightning/0.2.1/deep-dive/serving-llm/) Serving a model is essential if you want to train it, especially when you use the model’s own generations as training data. We’ll briefly review the general background to ensure all readers are aligned. Modern LLM servers solve a difficult scheduling problem: keeping GPUs fully utilized while handling prompts of different lengths, streaming tokens as they arrive, and fitting large KV caches into limited memory. Techniques like [**continuous batching**](https://www.anyscale.com/blog/continuous-batching-llm-inference) and [**paged attention**](https://arxiv.org/abs/2309.06180) address these challenges. Continuous batching interleaves decoding across requests to reuse weights efficiently; with careful memory planning, it achieves major throughput gains without increasing latency. PagedAttention reduces KV-cache fragmentation so batching remains effective as sequences grow. See [vLLM’s PagedAttention paper](https://arxiv.org/abs/2309.06180) and [industry analyses](https://www.baseten.co/blog/continuous-vs-dynamic-batching-for-ai-inference/) for details. Balancing inference correctness and efficiency is difficult — a [recent blog](https://thinkingmachines.ai/blog/defeating-nondeterminism-in-llm-inference/) from Thinking Machines Labs highlights how inference nondeterminism ultimately affects training. Beyond scheduling, servers expose an HTTP API, often **OpenAI-compatible** (`/v1/chat/completions` and `/v1/responses`), which is itself a complex stack. In addition to text prompts and chat messages, the API defines many parameters and response fields such as [tool calls](https://platform.openai.com/docs/guides/function-calling) , [structured output](https://platform.openai.com/docs/guides/structured-outputs) , and [multimodal support](https://platform.openai.com/docs/guides/images-vision) . Much effort has been put into implementing all these parameters for many frameworks. Popular engines like **vLLM** and [**SGLang**](https://github.com/sgl-project/sglang) ship with OpenAI-compatible frontends so you can reuse existing client code. [Ollama](https://ollama.com/blog/openai-compatibility) and [llama.cpp](https://llama-cpp-python.readthedocs.io/en/latest/server/) provide similar capabilities. However, because models differ internally, each framework interprets and implements the API slightly differently. Even with identical requests, the tokens passed to the model can vary substantially across frameworks. What Agent-lightning expects from a served LLM[¶](https://microsoft.github.io/agent-lightning/0.2.1/deep-dive/serving-llm/#what-agent-lightning-expects-from-a-served-llm "Permanent link") -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Most of the issues above either have workarounds or remain open research problems. Keep them in mind, but the key question is: what does Agent-lightning expect from a served LLM? The answer includes at least two things: * An OpenAI-compatible **Chat Completions** or **Responses** endpoint the agent can call during rollouts. * Optional training and debugging signals: **logprobs**, **usage**, and ideally **token IDs**. (OpenAI’s public API exposes usage and logprobs, but **not** token IDs — more on [why IDs matter](https://microsoft.github.io/agent-lightning/0.2.1/deep-dive/serving-llm/#token-ids-matter "Token IDs and why they matter") later.) Launching a serving framework[¶](https://microsoft.github.io/agent-lightning/0.2.1/deep-dive/serving-llm/#launching-a-serving-framework "Permanent link") ---------------------------------------------------------------------------------------------------------------------------------------------------------- For many algorithms, you’ll start an engine (e.g., **vLLM** or **SGLang**) before rollouts, then shut it down afterward to free GPU memory. Most frameworks provide a one-line “serve” command to launch the OpenAI-compatible server. You can use those to bring up `/v1/chat/completions` with your checkpoint, ensuring streaming and any required tool-calling features are enabled. A working example is shown in [Unsloth SFT](https://microsoft.github.io/agent-lightning/0.2.1/how-to/unsloth-sft/) . Weight updates — which occur after each training step — are trickier. Some frameworks like [vLLM](https://vllm.ai/) support hot-updating model weights, but it’s usually simpler and more reliable to restart the engine to load new weights. For medium-sized tasks (hundreds of rollouts taking 10+ minutes), the restart overhead (under 30 seconds) is typically negligible. If you’re using Agent-lightning’s [**VERL**](https://microsoft.github.io/agent-lightning/0.2.1/algorithm-zoo/verl/#agentlightning.algorithm.verl.VERL " VERL") integration, the algorithm can **manage the server automatically**. The [VERL framework](https://github.com/volcengine/verl) intelligently allocates compute resources and wraps vLLM/SGLang behind an `AsyncLLMServer` abstraction. You can directly use this as the LLM endpoint for agents. Since VERL can spawn multiple vLLM replicas, using [`LLMProxy`](https://microsoft.github.io/agent-lightning/0.2.1/reference/algorithm/#agentlightning.LLMProxy " agentlightning.LLMProxy") to manage them adds an additional safety layer. A full sequence diagram of how [VERL](https://microsoft.github.io/agent-lightning/0.2.1/algorithm-zoo/verl/#agentlightning.algorithm.verl.VERL " VERL") interacts with the LLM server and proxy is available [here](https://microsoft.github.io/agent-lightning/0.2.1/deep-dive/birds-eye-view/#birds-eye-view-verl-example "Putting It All Together: A Reinforcement Learning Example (VERL)") . LLM Proxy[¶](https://microsoft.github.io/agent-lightning/0.2.1/deep-dive/serving-llm/#llm-proxy "Permanent link") ------------------------------------------------------------------------------------------------------------------ The **LLM Proxy** is a utility class in Agent-lightning, built on [LiteLLM](https://docs.litellm.ai/) , that sits between runners and your backend engine(s) or server(s). In Agent-lightning it acts as a single URL registered as a [`Resource`](https://microsoft.github.io/agent-lightning/0.2.1/reference/types/#agentlightning.Resource " agentlightning.Resource") in the store, offering three key benefits: 1. **Unified endpoint & hot-swaps.** You can redirect traffic between OpenAI, Anthropic, local vLLM/SGLang, or canary checkpoints without modifying agent code — simply repoint the proxy. 2. **First-class tracing.** The proxy emits **OpenTelemetry** spans for every call and sends them to the [`LightningStore`](https://microsoft.github.io/agent-lightning/0.2.1/reference/store/#agentlightning.LightningStore " agentlightning.LightningStore") . It includes rollout and attempt identifiers in request headers so spans are correctly attributed. Sequence numbers are allocated monotonically via the store to [prevent clock-skew issues](https://microsoft.github.io/agent-lightning/0.2.1/tutorials/traces/#distributed-tracing "LLM Proxy") and allow reliable reconstruction of execution trees. 3. **Token IDs.** The proxy can return prompt and response token IDs along with the model output. More details are available in the [next section](https://microsoft.github.io/agent-lightning/0.2.1/deep-dive/serving-llm/#token-ids-matter "Token IDs and why they matter") . Operationally, running the proxy alongside the algorithm works best: the algorithm registers the backend (e.g., the vLLM URL) via [`LLMProxy.update_model_list`](https://microsoft.github.io/agent-lightning/0.2.1/reference/algorithm/#agentlightning.LLMProxy.update_model_list " update_model_list(model_list)") , publishes the proxy URL as a resource via [`LightningStore.add_resources`](https://microsoft.github.io/agent-lightning/0.2.1/reference/store/#agentlightning.LightningStore.add_resources " add_resources(resources) async ") , and runners simply use that URL during rollouts. This mirrors many production client–server setups. Token IDs and why they matter[¶](https://microsoft.github.io/agent-lightning/0.2.1/deep-dive/serving-llm/#token-ids-and-why-they-matter "Permanent link") ---------------------------------------------------------------------------------------------------------------------------------------------------------- [](https://microsoft.github.io/agent-lightning/0.2.1/deep-dive/serving-llm/) This section explains how Agent-lightning handles and uses token IDs — a subtle but important detail for training stability and accuracy. Most agents interact with LLMs via **Chat Completion APIs**, exchanging chat messages. There are two main approaches to collecting training data from such agents. Note Tokenization here refers to the process of converting **Chat Messages** into **Token IDs**. Detokenization is the reverse process of converting **Token IDs** back to **Chat Messages**. Normally, the tokenizer is published along with the pretrained model, which includes a vocabulary, special tokens, and a chat template to dealing with chat messages. **1\. Retokenizing chat messages.** In this approach, you store chat messages as text and let training algorithms **retokenize** them later, as done in many SFT workflows (e.g., [HuggingFace SFT](https://huggingface.co/docs/trl/sft_trainer) ). In practice, we’ve found this method unstable and less accurate. The chart below compares training results. The retokenization approach is run twice. All settings are the same except for the retokenization approach. This instability has three causes. Firstly, chat template used in different frameworks could be slightly different. For example, one single LLaMA model can work with multiple chat templates (multiple in [vLLM](https://github.com/vllm-project/vllm/tree/1d165d6d859d3c50720f0c07209db2363c4fd33b/examples) and one in [HuggingFace](https://huggingface.co/meta-llama) ). It's possible that the chat template used in detokenization is different from the one used in tokenization (this is actually an implementation bug). Secondly, a word might be generated as two tokens (e.g., `H + AVING`) but later retokenized as `HAV + ING`. The text looks identical, but the token IDs differ from what the model originally produced. Thirdly, a generated tool call text like `{ "name": ... }` is parsed by tool call parser into an object that is required by chat completion API. Later, the object is rendered back to `{ "name": ... }` and retokenized again, tool call parsing and re-rendering might cause changes in whitespace and formatting. In some situations, JSON errors may even be auto-corrected by the tool call parser — masking the model’s true generation errors and preventing them from being trained away. * * * **2\. Saving token IDs directly.** The alternative is to save the token IDs generated by the model, as done in RL setups like [Tinker](https://thinkingmachines.ai/tinker/) . This requires a training pipeline that treats tokens as first-class entities, meaning agents must communicate with the inference engine at the token level. However, most agents — especially those built with frameworks like LangChain — rely on OpenAI-compatible APIs and can’t tokenize or detokenize themselves. As mentioned [earlier](https://microsoft.github.io/agent-lightning/0.2.1/deep-dive/serving-llm/#general-llm-serving-background "General background on LLM serving") , implementing this layer manually is complex and error-prone. Some frameworks implement custom solutions (e.g., [VERL Agent Loop](https://github.com/volcengine/verl/blob/4da0d3d3188072772cb2ec817b3d6cf4a463821f/recipe/langgraph_agent/chat_model.py) , [Tinker Renderer](https://github.com/thinking-machines-lab/tinker-cookbook/blob/34a6588d7055040c259985d98e71c0140b389ba7/tinker_cookbook/renderers.py) ), while others leave it to users (e.g., [SkyRL Search-R1](https://novasky-ai.notion.site/skyrl-searchr1) ). * * * A better solution is to use an **OpenAI-compatible API that returns token IDs directly.** This lets agents continue using familiar APIs while capturing token IDs via [tracing](https://microsoft.github.io/agent-lightning/0.2.1/tutorials/traces/) for training. The limitation, of course, is that the serving framework must actually support this capability. When Agent-lightning was first released, we implemented an [instrumented vLLM server](https://github.com/microsoft/agent-lightning/blob/v0.1/agentlightning/instrumentation/vllm.py) that monkey-patched vLLM’s OpenAI server to return token IDs. Since then, the Agent-lightning and vLLM teams have collaborated to add this feature directly to [vLLM core](https://github.com/vllm-project/vllm/pull/22587) . Starting with **vLLM v0.10.2**, the OpenAI-compatible API includes a [`return_token_ids` parameter](https://docs.vllm.ai/en/v0.10.2/serving/openai_compatible_server.html#api-reference) , allowing token IDs to be requested alongside chat messages. SGLang has tracked [similar feature requests](https://github.com/sgl-project/sglang/issues/2634) , though its OpenAI-compatible layer doesn’t yet support them. In short, when using vLLM v0.10.2 or newer, [`LLMProxy`](https://microsoft.github.io/agent-lightning/0.2.1/reference/algorithm/#agentlightning.LLMProxy " agentlightning.LLMProxy") automatically adds `return_token_ids` to each request so the engine includes token IDs in its response. For older vLLM versions, you still need the instrumented version (via `agl vllm` CLI command). Finally, if you only save token IDs in spans, it will have its own limitations — if you train one model using spans from another model with a different tokenizer, incompatibilities can arise. In practice, though, spans in Agent-lightning always store both chat messages and token IDs (actually the full request and response objects), allowing you to fall back to retokenization when necessary. Back to top --- # Bird's Eye View - Agent-lightning [Skip to content](https://microsoft.github.io/agent-lightning/0.2.0/deep-dive/birds-eye-view/#the-birds-eye-view-of-agent-lightning) ⚠️ You are viewing development documentation. For stable documentation, please visit [https://microsoft.github.io/agent-lightning/stable/](https://microsoft.github.io/agent-lightning/stable/) The Bird's Eye View of Agent-lightning[¶](https://microsoft.github.io/agent-lightning/0.2.0/deep-dive/birds-eye-view/#the-birds-eye-view-of-agent-lightning "Permanent link") ============================================================================================================================================================================== High Volume of Information Ahead This article provides an in-depth exploration of the Agent-lightning architecture. It is not intended as a beginner’s guide or usage tutorial. This article summarizes how Agent-lightning (as of v0.2) wires the [Algorithm](https://microsoft.github.io/agent-lightning/0.2.0/reference/algorithm/#agentlightning.Algorithm " agentlightning.Algorithm") , [Runner](https://microsoft.github.io/agent-lightning/0.2.0/reference/runner/#agentlightning.Runner " agentlightning.Runner") , and [LightningStore](https://microsoft.github.io/agent-lightning/0.2.0/reference/store/#agentlightning.LightningStore " agentlightning.LightningStore") loop together and shows where auxiliary components (the [Tracer](https://microsoft.github.io/agent-lightning/0.2.0/reference/runner/#agentlightning.Tracer " agentlightning.Tracer") , [Adapter](https://microsoft.github.io/agent-lightning/0.2.0/reference/algorithm/#agentlightning.Adapter " agentlightning.Adapter") , and [LLM Proxy](https://microsoft.github.io/agent-lightning/0.2.0/reference/algorithm/#agentlightning.LLMProxy " agentlightning.LLMProxy") ) plug into the loop. Each section provides a diagram for a different perspective of the system. Algorithm ↔ Runner ↔ Store data flow[¶](https://microsoft.github.io/agent-lightning/0.2.0/deep-dive/birds-eye-view/#algorithm-runner-store-data-flow "Permanent link") ----------------------------------------------------------------------------------------------------------------------------------------------------------------------- At its heart, Agent-lightning is built on three main components that work in a coordinated loop: * **[Algorithm](https://microsoft.github.io/agent-lightning/0.2.0/reference/algorithm/#agentlightning.Algorithm " agentlightning.Algorithm") :** The "brain" of the system. It decides what tasks to run, learns from the results, and updates resources (like AI models or prompts). * **[Runner](https://microsoft.github.io/agent-lightning/0.2.0/reference/runner/#agentlightning.Runner " agentlightning.Runner") :** The "worker" of the system. It executes tasks assigned by the algorithm, runs the agent, and records the results. * **[LightningStore](https://microsoft.github.io/agent-lightning/0.2.0/reference/store/#agentlightning.LightningStore " agentlightning.LightningStore") :** The central "database" and message queue. It acts as the single source of truth, storing tasks, results, and resources, and enabling communication between the Algorithm and Runner. The typical data flow in a training loop is as follows: The **[Algorithm](https://microsoft.github.io/agent-lightning/0.2.0/reference/algorithm/#agentlightning.Algorithm " agentlightning.Algorithm") ** enqueues tasks (called **[Rollouts](https://microsoft.github.io/agent-lightning/0.2.0/reference/types/#agentlightning.Rollout " agentlightning.Rollout") **) into the **[LightningStore](https://microsoft.github.io/agent-lightning/0.2.0/reference/store/#agentlightning.LightningStore " agentlightning.LightningStore") **. A **[Runner](https://microsoft.github.io/agent-lightning/0.2.0/reference/runner/#agentlightning.Runner " agentlightning.Runner") ** then dequeues a task, executes it, and streams the results (called **[Spans](https://microsoft.github.io/agent-lightning/0.2.0/reference/types/#agentlightning.Span " agentlightning.Span") **) back to the store. Once the task is complete, the algorithm can query the new data from the store to learn and update its resources. The diagram below shows this fundamental interaction in a simple, non-parallel setup. sequenceDiagram autonumber participant Algo as Algorithm participant Store as LightningStore participant Runner participant Agent loop Over the dataset Algo-->>Store: add_resources + enqueue_rollout Store-->>Runner: dequeue_rollout → AttemptedRollout Store-->>Runner: get_latest_resources Runner-->>Store: update_attempt("running", worker_id) Runner->>Agent: rollout + resources Agent->>Runner: reward / spans Runner-->>Store: add_span or add_otel_span Runner-->>Store: update_attempt("finished", status) Store-->>Algo: query_rollouts + spans Algo-->>Algo: Update resources (optional) end Solid lines represent direct calls, while dashed lines are asynchronous or long-running operations. ### Key Terminology[¶](https://microsoft.github.io/agent-lightning/0.2.0/deep-dive/birds-eye-view/#key-terminology "Permanent link") We define the following terms, which may be helpful for understanding the diagram above. * **[Resources](https://microsoft.github.io/agent-lightning/0.2.0/reference/types/#agentlightning.Resource " agentlightning.Resource") :** A collection of assets to be tuned or trained. Agents perform rollouts against resources and collect span data. Algorithms use those data to update the resources. In RL training, the resources are a tunable model. In prompt tuning, the resources are prompt templates. * **[Rollout](https://microsoft.github.io/agent-lightning/0.2.0/reference/types/#agentlightning.Rollout " agentlightning.Rollout") :** A unit of work that an agent performs against a resource. A rollout (noun) can be incomplete, in which case it is also known as a **task**, **sample**, or **job** (these terms are used interchangeably). The agent executes its own defined workflow against the rollout — the process is also called "to rollout" (verb). After execution, the rollout (noun) is considered _complete_. * **[Attempt](https://microsoft.github.io/agent-lightning/0.2.0/reference/types/#agentlightning.Attempt " agentlightning.Attempt") :** A single execution of a rollout. One rollout can have multiple attempts in case of failures or timeouts. * **[Span](https://microsoft.github.io/agent-lightning/0.2.0/reference/types/#agentlightning.Span " agentlightning.Span") :** During the rollout, the agent can generate multiple spans (also known as "traces" or "events"). The recorded spans are collected in the store, which is crucial for understanding agent behavior and optimizing agents. * **[Reward](https://microsoft.github.io/agent-lightning/0.2.0/reference/agent/#agentlightning.emit_reward " agentlightning.emit_reward(reward)") :** A special span that is defined as a number judging the quality of the rollout during some period of the rollout. * **[Dataset](https://microsoft.github.io/agent-lightning/0.2.0/reference/types/#agentlightning.Dataset " agentlightning.Dataset") :** A collection of incomplete rollouts (i.e., tasks) for the agent to process. The dual datasets (train, val) serve as the initial input for the algorithm to enqueue the first batch of rollouts. ### Store[¶](https://microsoft.github.io/agent-lightning/0.2.0/deep-dive/birds-eye-view/#store "Permanent link") As discussed previously, the [LightningStore](https://microsoft.github.io/agent-lightning/0.2.0/reference/store/#agentlightning.LightningStore " agentlightning.LightningStore") is the central hub for all data in Agent-lightning. The store exposes a set of APIs for algorithms and runners to interact with the data; the most important ones are: `[](https://microsoft.github.io/agent-lightning/0.2.0/deep-dive/birds-eye-view/#__codelineno-0-1) from agentlightning.types import AttemptedRollout, ResourcesUpdate, Span, TaskInput [](https://microsoft.github.io/agent-lightning/0.2.0/deep-dive/birds-eye-view/#__codelineno-0-2) [](https://microsoft.github.io/agent-lightning/0.2.0/deep-dive/birds-eye-view/#__codelineno-0-3) class LightningStore: [](https://microsoft.github.io/agent-lightning/0.2.0/deep-dive/birds-eye-view/#__codelineno-0-4) [](https://microsoft.github.io/agent-lightning/0.2.0/deep-dive/birds-eye-view/#__codelineno-0-5) async def enqueue_rollout(self, input: TaskInput, ...) -> Rollout: ... [](https://microsoft.github.io/agent-lightning/0.2.0/deep-dive/birds-eye-view/#__codelineno-0-6) [](https://microsoft.github.io/agent-lightning/0.2.0/deep-dive/birds-eye-view/#__codelineno-0-7) async def dequeue_rollout(self) -> AttemptedRollout | None: ... [](https://microsoft.github.io/agent-lightning/0.2.0/deep-dive/birds-eye-view/#__codelineno-0-8) [](https://microsoft.github.io/agent-lightning/0.2.0/deep-dive/birds-eye-view/#__codelineno-0-9) async def add_span(self, span: Span) -> Span: ... [](https://microsoft.github.io/agent-lightning/0.2.0/deep-dive/birds-eye-view/#__codelineno-0-10) [](https://microsoft.github.io/agent-lightning/0.2.0/deep-dive/birds-eye-view/#__codelineno-0-11) async def get_latest_resources(self) -> Optional[ResourcesUpdate]: ... [](https://microsoft.github.io/agent-lightning/0.2.0/deep-dive/birds-eye-view/#__codelineno-0-12) [](https://microsoft.github.io/agent-lightning/0.2.0/deep-dive/birds-eye-view/#__codelineno-0-13) async def wait_for_rollouts(self, rollout_ids: List[str], ...): ... [](https://microsoft.github.io/agent-lightning/0.2.0/deep-dive/birds-eye-view/#__codelineno-0-14) [](https://microsoft.github.io/agent-lightning/0.2.0/deep-dive/birds-eye-view/#__codelineno-0-15) async def query_spans(self, rollout_id: str, ...): ... [](https://microsoft.github.io/agent-lightning/0.2.0/deep-dive/birds-eye-view/#__codelineno-0-16) [](https://microsoft.github.io/agent-lightning/0.2.0/deep-dive/birds-eye-view/#__codelineno-0-17) async def update_attempt(self, rollout_id: str, attempt_id: str, status: str, ...): ... [](https://microsoft.github.io/agent-lightning/0.2.0/deep-dive/birds-eye-view/#__codelineno-0-18) [](https://microsoft.github.io/agent-lightning/0.2.0/deep-dive/birds-eye-view/#__codelineno-0-19) ...` These interfaces operate on [`AttemptedRollout`](https://microsoft.github.io/agent-lightning/0.2.0/reference/types/#agentlightning.AttemptedRollout " agentlightning.AttemptedRollout") , [`ResourcesUpdate`](https://microsoft.github.io/agent-lightning/0.2.0/reference/types/#agentlightning.ResourcesUpdate " agentlightning.ResourcesUpdate") , [`Span`](https://microsoft.github.io/agent-lightning/0.2.0/reference/types/#agentlightning.Span " agentlightning.Span") , and [`TaskInput`](https://microsoft.github.io/agent-lightning/0.2.0/reference/internal/#agentlightning.TaskInput " agentlightning.TaskInput = Any module-attribute ") instances from `agentlightning.types`. As the APIs show, the store essentially provides a queue for rollouts and storage for resources, spans, and attempts. Developers should implement the store carefully to ensure data integrity and consistency, especially when multiple runners work in parallel across multiple attempts. The store is designed to be extensible. Users can implement their own store by inheriting from [`LightningStore`](https://microsoft.github.io/agent-lightning/0.2.0/reference/store/#agentlightning.LightningStore " agentlightning.LightningStore") and overriding methods. Agent-lightning provides a few reference implementations, such as [`InMemoryLightningStore`](https://microsoft.github.io/agent-lightning/0.2.0/reference/store/#agentlightning.InMemoryLightningStore " agentlightning.InMemoryLightningStore") (default) and `SqliteLightningStore` (under construction). When parallelized, the store may need special wrappers to ensure thread/process safety or delegate computation to a store in another process or machine. Supporting Components in the Loop[¶](https://microsoft.github.io/agent-lightning/0.2.0/deep-dive/birds-eye-view/#supporting-components-in-the-loop "Permanent link") --------------------------------------------------------------------------------------------------------------------------------------------------------------------- While the core loop is simple, Agent-lightning provides several components to make development easier and more powerful. ### Tracer[¶](https://microsoft.github.io/agent-lightning/0.2.0/deep-dive/birds-eye-view/#tracer "Permanent link") The [`Tracer`](https://microsoft.github.io/agent-lightning/0.2.0/reference/runner/#agentlightning.Tracer " agentlightning.Tracer") is a component within the [`Runner`](https://microsoft.github.io/agent-lightning/0.2.0/reference/runner/#agentlightning.Runner " agentlightning.Runner") that records detailed spans (events) during an agent's execution and sends them to the [`LightningStore`](https://microsoft.github.io/agent-lightning/0.2.0/reference/store/#agentlightning.LightningStore " agentlightning.LightningStore") . Instead of requiring the agent to manually log every span, the tracer automatically instruments key methods (e.g., LLM calls) and captures their inputs, outputs, and metadata. This provides a detailed log of the agent's behavior with minimal effort. sequenceDiagram autonumber participant Store participant Runner participant Tracer participant Agent Note over Runner,Tracer: Runner manages tracer as member Tracer->>Agent: Apply instrumentation loop Until no more rollouts Store-->>Runner: dequeue_rollout → AttemptedRollout Store-->>Runner: get_latest_resources Runner->>Agent: training_rollout / validation_rollout loop For each finished span Agent-->>Tracer: openai.chat.completion invoked
agent.execute invoked
... Agent->>Tracer: emit intermediate reward Tracer-->>Store: add_otel_span(rollout_id, attempt_id, span) end Agent->>Runner: final reward + extra spans (if any) Runner-->>Store: add_span(rollout_id, attempt_id, span) Runner-->>Store: update_attempt(status) end Tracer->>Agent: Unapply instrumentation The above diagram shows the overall data flow between store, tracer and agent. In realistic, it's a bit more complicated than that. Spans are not emitted actively by the agent; they are intercepted by the tracer by hooking and instrumenting key methods used in the agents. The tracer uses a callback (called exporter) to monitor events and log to the store. Before a rollout starts, the runner enters a [`trace_context`](https://microsoft.github.io/agent-lightning/0.2.0/reference/runner/#agentlightning.Tracer.trace_context " trace_context(name=None, *, store=None, rollout_id=None, attempt_id=None)") before invoking the agent, wiring store identifiers into the tracer. Each span completion streams back to the store through `LightningSpanProcessor`, so the agent’s instrumentation lands in [`add_otel_span`](https://microsoft.github.io/agent-lightning/0.2.0/reference/store/#agentlightning.LightningStore.add_otel_span " add_otel_span(rollout_id, attempt_id, readable_span, sequence_id=None) async ") . If the agent’s rollout method returns a numeric reward, the runner emits one more OpenTelemetry span before finalizing the attempt. ### Hooks[¶](https://microsoft.github.io/agent-lightning/0.2.0/deep-dive/birds-eye-view/#hooks "Permanent link") [`Hook`](https://microsoft.github.io/agent-lightning/0.2.0/reference/types/#agentlightning.Hook " agentlightning.Hook") implementations are user-defined callback functions that allow you to augment a [`Runner`](https://microsoft.github.io/agent-lightning/0.2.0/reference/runner/#agentlightning.Runner " agentlightning.Runner") 's behavior at specific points in its lifecycle. You can use hooks to add custom logging, set up resources before a rollout begins, or tear them down after it ends. Hooks can be triggered at four key moments: `on_rollout_start`, `on_trace_start`, `on_trace_end`, and `on_rollout_end`. Users should pay special attention to the difference between `on_trace_end` and `on_rollout_end`. The former is called right before the tracer exits the trace context, while the latter is called after the runner processes the final leftover rewards and spans, and finalizes the attempt in the store. sequenceDiagram autonumber participant Store participant Hooks participant Runner participant Tracer participant Agent Note over Runner,Hooks: Runner manages hooks as member loop Until no more rollouts Store-->>Runner: dequeue_rollout → AttemptedRollout Store-->>Runner: get_latest_resources Runner->>Hooks: on_rollout_start(agent, runner, rollout) Runner->>Agent: training_rollout / validation_rollout Tracer->>Agent: enter_trace_context activate Tracer Runner->>Hooks: on_trace_start(agent, runner, tracer, rollout) Note over Runner,Agent: Agent rollout omitted Runner->>Hooks: on_trace_end(agent, runner, tracer, rollout) Tracer->>Agent: exit_trace_context deactivate Tracer Agent->>Runner: final reward + extra spans (if any) Runner-->>Store: add_span(rollout_id, attempt_id, span) Runner->>Hooks: on_rollout_end(agent, runner, rollout, status) end ### Adapter[¶](https://microsoft.github.io/agent-lightning/0.2.0/deep-dive/birds-eye-view/#adapter "Permanent link") The [`Adapter`](https://microsoft.github.io/agent-lightning/0.2.0/reference/algorithm/#agentlightning.Adapter " agentlightning.Adapter") is a component used by the [`Algorithm`](https://microsoft.github.io/agent-lightning/0.2.0/reference/algorithm/#agentlightning.Algorithm " agentlightning.Algorithm") to transform raw data from the [`LightningStore`](https://microsoft.github.io/agent-lightning/0.2.0/reference/store/#agentlightning.LightningStore " agentlightning.LightningStore") into a format suitable for learning. Runners stream raw spans into the store during execution. Later, the algorithm queries these spans and uses an adapter to convert them into structured data, like training examples for a reinforcement learning model. For instance, the [`TracerTraceToTriplet`](https://microsoft.github.io/agent-lightning/0.2.0/reference/algorithm/#agentlightning.TracerTraceToTriplet " agentlightning.TracerTraceToTriplet") processes OpenTelemetry spans to create `(prompt, response, reward)` triplets, which are the fundamental data structure for many RL fine-tuning algorithms. flowchart LR Runner -- (1) add_otel_span --> Store Store -- (2) query_spans --> Algorithm Algorithm -- (3) spans --> Adapter Adapter -- (4) transformed data --> Algorithm ### LLM Proxy[¶](https://microsoft.github.io/agent-lightning/0.2.0/deep-dive/birds-eye-view/#llm-proxy "Permanent link") The [`LLMProxy`](https://microsoft.github.io/agent-lightning/0.2.0/reference/algorithm/#agentlightning.LLMProxy " agentlightning.LLMProxy") is an optional bridge component that sits between an agent and the algorithms' resources. It acts as a centralized endpoint for all LLM calls. Usually the proxy URL is added to the store as a special resource, so that the [`Runner`](https://microsoft.github.io/agent-lightning/0.2.0/reference/runner/#agentlightning.Runner " agentlightning.Runner") can fetch it along with other resources when dequeuing a rollout. During rollouts, the runner invokes the proxy's HTTP endpoint instead of calling a model backend directly. This design offers several benefits: 1. **Instrumentation:** It automatically captures detailed traces of LLM interactions (prompts, responses, metadata) and sends them to the store, complementing the tracer, especially when the agent's code is hard to instrument directly. 2. **Backend Abstraction:** It provides a unified interface for various LLM backends (OpenAI, Anthropic, local models) and can add features like retry logic, rate limiting, and caching. 3. **Resource Management:** The algorithm can dynamically update which LLM the agent uses (e.g., swapping to a newly fine-tuned model) by simply swapping the backend model the proxy is using, without interrupting the agent's code. The benefits above seem to be all discussed within the context of model fine-tuning. As a matter of fact, the proxy can be useful for prompt tuning as well. The algorithm can register one of the following two types of endpoints into the proxy: 1. **Endpoint served by the algorithm:** If the algorithm is internally updating the LLM weights (e.g., RL), it can launch an LLM inference engine (i.e., a model server) and register the endpoint URL with the proxy. The proxy then forwards all LLM calls to that endpoint. 2. **Third-party LLM endpoint:** If the algorithm is not updating the LLM weights (e.g., prompt tuning), it can register a third-party LLM endpoint into the proxy. We show a diagram below that illustrates how the proxy fits into the overall data flow. sequenceDiagram autonumber participant Algo as Algorithm participant LLMProxy as LLM Proxy participant Store participant Runner participant Agent Note over Algo,LLMProxy: Algorithm manages LLMProxy as member loop Over the Dataset Algo->>Algo: Launch LLM Inference Engine
(optional) Algo->>LLMProxy: Register Inference Engine
(optional) Algo-->>Store: enqueue_rollout LLMProxy->>Store: Proxy URL added as Resource Store-->>Runner: dequeue_rollout → AttemptedRollout Store-->>Runner: get_latest_resources Runner->>Agent: rollout + resources
(LLM Proxy URL as resource) loop Defined by Agent Agent-->>LLMProxy: LLM calls activate LLMProxy LLMProxy-->>Store: add_span or add_otel_span LLMProxy-->>Agent: LLM responses deactivate LLMProxy Agent-->>Runner: rewards Runner-->>Store: add_span or add_otel_span end Runner-->>Store: update_attempt("finished", status) Store-->>Algo: query_rollouts + spans Algo-->>Algo: Update LLM Weights
(optional) end In this diagram, the store receives spans from both the proxy and the runner. We will see a problem later with parallelism where the proxy and runner are in different machines, and spans need to obtain a special counter from the store to ensure the ordering of spans. ### Trainer[¶](https://microsoft.github.io/agent-lightning/0.2.0/deep-dive/birds-eye-view/#trainer "Permanent link") The [Trainer](https://microsoft.github.io/agent-lightning/0.2.0/reference/trainer/#agentlightning.Trainer " agentlightning.Trainer") is the high-level orchestrator that initializes and connects all major components -- [Algorithm](https://microsoft.github.io/agent-lightning/0.2.0/reference/algorithm/#agentlightning.Algorithm " agentlightning.Algorithm") , [Runner](https://microsoft.github.io/agent-lightning/0.2.0/reference/runner/#agentlightning.Runner " agentlightning.Runner") , [LightningStore](https://microsoft.github.io/agent-lightning/0.2.0/reference/store/#agentlightning.LightningStore " agentlightning.LightningStore") , [Tracer](https://microsoft.github.io/agent-lightning/0.2.0/reference/runner/#agentlightning.Tracer " agentlightning.Tracer") , [Adapter](https://microsoft.github.io/agent-lightning/0.2.0/reference/algorithm/#agentlightning.Adapter " agentlightning.Adapter") , [LLM Proxy](https://microsoft.github.io/agent-lightning/0.2.0/reference/algorithm/#agentlightning.LLMProxy " agentlightning.LLMProxy") , and [Hook](https://microsoft.github.io/agent-lightning/0.2.0/reference/types/#agentlightning.Hook " agentlightning.Hook") . The components can have a lifecycle as long as the trainer. The trainer manages their lifecycles and handles dependency injection, ensuring that every part of the system operates within a consistent and shared environment. Below, we demonstrate how the components relate to each other and their roles. We first clarify the roles and relationships shown in the diagram: 1. **Owns:** components that the trainer constructs and manages directly (e.g., runner, tracer). 2. **Injects:** components passed into others as dependencies. 3. **References:** weak links for coordination without ownership. 4. **Uses:** components that are temporarily interacted with. For example, the [LightningStore](https://microsoft.github.io/agent-lightning/0.2.0/reference/store/#agentlightning.LightningStore " agentlightning.LightningStore") is injected into the [Algorithm](https://microsoft.github.io/agent-lightning/0.2.0/reference/algorithm/#agentlightning.Algorithm " agentlightning.Algorithm") and [Runner](https://microsoft.github.io/agent-lightning/0.2.0/reference/runner/#agentlightning.Runner " agentlightning.Runner") . The [Tracer](https://microsoft.github.io/agent-lightning/0.2.0/reference/runner/#agentlightning.Tracer " agentlightning.Tracer") and [LitAgent](https://microsoft.github.io/agent-lightning/0.2.0/reference/agent/#agentlightning.LitAgent " agentlightning.LitAgent") are injected into the runner. The [Adapter](https://microsoft.github.io/agent-lightning/0.2.0/reference/algorithm/#agentlightning.Adapter " agentlightning.Adapter") and [LLM Proxy](https://microsoft.github.io/agent-lightning/0.2.0/reference/algorithm/#agentlightning.LLMProxy " agentlightning.LLMProxy") are injected into the algorithm. The store is further injected into the tracer, adapter and LLM proxy by the runner and algorithm respectively. flowchart TD %% === Left side: Algorithm domain === subgraph L["Algorithm Side"] Algorithm["Algorithm
(no default)"] Adapter["Adapter
(TracerTraceToTriplet*)"] LLMProxy["LLM Proxy
(no default)"] Algorithm -.injects.-> Adapter Algorithm -.injects.-> LLMProxy end linkStyle 0,1 stroke:#896978,stroke-width:2px; %% === Middle: Core trainer and store === subgraph M["Core"] Trainer["Trainer"] Store["LightningStore
(InMemory* default)"] Trainer --has--> Algorithm Trainer --has--> Store Trainer --has--> Adapter Trainer --has--> LLMProxy end linkStyle 2,3,4,5 stroke:#839791,stroke-width:2px; %% === Right side: Runner side === subgraph R["Runner Side"] Runner["Runner
(LitAgentRunner* default)"] Tracer["Tracer
(AgentOpsTracer*)"] Hooks["Hooks (empty default)"] Agent["Agent
(LitAgent*)"] Runner -.injects.-> Tracer Runner -.injects.-> Store Runner -.injects.-> Agent Runner -.injects.-> Hooks Tracer -.injects.-> Store Hooks -.uses.-> Runner Hooks -.uses.-> Agent Hooks -.uses.-> Tracer end linkStyle 6,7,8,9,10 stroke:#896978,stroke-width:2px; linkStyle 11,12,13 stroke:#7a89c2,stroke-width:2px; %% === Cross-section connections === Trainer --has--> Runner Trainer --has--> Tracer Trainer --has--> Hooks Trainer --uses--> Agent Algorithm -.injects.-> Store LLMProxy -.injects.-> Store Agent -.references.-> Trainer Runner -.references.-> Trainer Algorithm -.references.-> Trainer linkStyle 14,15,16 stroke:#839791,stroke-width:2px; linkStyle 17,20,21,22 stroke:#7a89c2,stroke-width:2px; linkStyle 18,19 stroke:#896978,stroke-width:2px; style L fill:none; style M fill:none; style R fill:none; Putting It All Together: A Reinforcement Learning Example (VERL)[¶](https://microsoft.github.io/agent-lightning/0.2.0/deep-dive/birds-eye-view/#putting-it-all-together-a-reinforcement-learning-example-verl "Permanent link") -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- [](https://microsoft.github.io/agent-lightning/0.2.0/deep-dive/birds-eye-view/) VERL shows how an algorithm consumes the shared infrastructure. For historical reasons, code lives in `agentlightning.algorithm.verl` and `agentlightning.verl`. The latter is legacy and reuses terms like `Trainer` in confusing ways. The former is a thin wrapper that conforms to the new algorithm interface. Future versions will merge the two. Reinforcement learning aims to learn a policy that takes actions in states to maximize expected reward. For agents, the policy is usually a language model. Inputs are prompts (state). Outputs are generated text (action). A numeric score judges quality (reward). The `(state, action, reward)` **triplet** is the basic learning unit. In Agent-lightning, the environment is implicit in the agent’s workflow, which orchestrates one or more LLM calls and often self-judges using rules or additional model calls. During a rollout, the agent emits spans that contain everything needed for RL training, including LLM call traces and numeric judge/reward signals. The "algorithm", on the other hand, have more responsibilities. 1. Providing a language model deployment that is currently learning and improving for the agent to interact with; 2. Preparing the tasks that the agents will perform; 3. Querying the spans generated, extracting triplets, and converting them into a format that the underlying RL library can consume; 4. Updating the language model based on the learning signals. In the VERL integration, the algorithm launches a chat completion endpoint using `vLLM` and wraps training with `FSDP` for distributed optimization. It enqueues tasks from the dataset. After rollouts finish, it queries spans and converts them to triplets with `TracerTraceToTriplet`. VERL’s native training loop then consumes these triplets to update model weights. The workflow can be summarized in the following diagram. sequenceDiagram autonumber participant vLLM as vLLM Chat
Completion Endpoint participant FSDP as FSDP / Megatron
Weights Optimizer participant Algo as Algorithm
Main Controller
(Main Process) participant Adapter as TracerTraceToTriplet participant LLMProxy as LLM Proxy participant Store as LightningStore participant Runner as Runner + Agent Note over Algo,LLMProxy: LLMProxy and Adapter are injected by Trainer as member Note over vLLM,Algo: Algorithm creates and owns vLLM and FSDP loop Over the Dataset in Batches Algo->>vLLM: Create Chat Completion Endpoint activate vLLM vLLM->>LLMProxy: Registered as Backend Endpoint LLMProxy->>Store: Proxy URL added as Resource par Over data samples in the batch Algo-->>Store: enqueue_rollout Store-->>Runner: Dequeue Rollout +
Resources (i.e., URL) loop One Rollout Attempt Runner-->>LLMProxy: LLM calls LLMProxy-->>vLLM: Forwarded LLM calls vLLM-->>LLMProxy: LLM responses LLMProxy-->>Store: add_span / add_otel_span LLMProxy-->>Runner: Forwarded LLM responses Runner-->>Store: add_span / add_otel_span
(by tracer, including rewards) end Runner-->>Store: update_attempt("finished", status) end Algo-->>Store: Poll for completed rollouts + spans Algo->>vLLM: Chat Completion Endpoint Sleeps deactivate vLLM Algo->>Adapter: adapt(spans) Adapter->>FSDP: Triplets (state, action, reward) activate FSDP FSDP-->>Algo: Updated LLM weights deactivate FSDP end **Notes:** 1. There are interactions between different components injected into or owned by algorithms in the diagram, such as the output of the adapter feeding into the FSDP optimizer. This is for simplicity of illustration and slightly different from the actual implementation, where it's the algorithm main controller that orchestrates the data flow between components. 2. **On mapping to VERL.** VERL uses a classic RLHF setup where each action is a single token, the state is the full conversation history up to that token, and reward is given at the end. This is very different from our setup where each action is actually a chunk of text, although they are both called RL! Therefore, after the adapter produces triplets, the algorithm converts each `(state, action, reward)` into a VERL trajectory (`DataProto`) with keys like `input_ids`, `position_ids`, `attention_mask`, and `token_level_scores`. That conversion happens after triplet generation and is not shown in the diagram. Execution Strategies and Parallelism[¶](https://microsoft.github.io/agent-lightning/0.2.0/deep-dive/birds-eye-view/#execution-strategies-and-parallelism "Permanent link") --------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Readers might have observed from the diagram above that there is absolutely no communication between (1) runner and agents and (2) algorithm. The only overlap of them is the [Trainer](https://microsoft.github.io/agent-lightning/0.2.0/reference/trainer/#agentlightning.Trainer " agentlightning.Trainer") and [LightningStore](https://microsoft.github.io/agent-lightning/0.2.0/reference/store/#agentlightning.LightningStore " agentlightning.LightningStore") . This observation is very clear with the diagram within the trainer section. This design allows us to flexibly scale the runner and algorithm independently, which is crucial for large-scale training. Agent-lightning packages two executable bundles: a runner bundle ([Runner](https://microsoft.github.io/agent-lightning/0.2.0/reference/runner/#agentlightning.Runner " agentlightning.Runner") , [Tracer](https://microsoft.github.io/agent-lightning/0.2.0/reference/runner/#agentlightning.Tracer " agentlightning.Tracer") , [Hook](https://microsoft.github.io/agent-lightning/0.2.0/reference/types/#agentlightning.Hook " agentlightning.Hook") , [LitAgent](https://microsoft.github.io/agent-lightning/0.2.0/reference/agent/#agentlightning.LitAgent " agentlightning.LitAgent") ) and an algorithm bundle ([Algorithm](https://microsoft.github.io/agent-lightning/0.2.0/reference/algorithm/#agentlightning.Algorithm " agentlightning.Algorithm") , [Adapter](https://microsoft.github.io/agent-lightning/0.2.0/reference/algorithm/#agentlightning.Adapter " agentlightning.Adapter") , [LLM Proxy](https://microsoft.github.io/agent-lightning/0.2.0/reference/algorithm/#agentlightning.LLMProxy " agentlightning.LLMProxy") ). Both share the [LightningStore](https://microsoft.github.io/agent-lightning/0.2.0/reference/store/#agentlightning.LightningStore " agentlightning.LightningStore") . The trainer initializes and connects the bundles. graph TD subgraph Runner_Side["Runner Bundle"] direction LR R[Runner] --- T[Tracer] --- H[Hooks] --- A1[Agent] end subgraph Algorithm_Side["Algorithm Bundle"] direction LR ALG[Algorithm] --- AD[Adapter] --- LLM[LLM Proxy] end S[(Store)] TR[Trainer] Runner_Side <--> S Algorithm_Side <--> S TR --> Runner_Side TR --> Algorithm_Side linkStyle 0,1,2,3,4 opacity:0; An [execution strategy](https://microsoft.github.io/agent-lightning/0.2.0/reference/trainer/#agentlightning.ExecutionStrategy " agentlightning.ExecutionStrategy") , defined and owned by the trainer, governs how algorithm and runner bundles are placed, connected, scaled, and aborted. It serves four primary purposes. Execution strategies first determine **bundle placement** — whether the two bundles run in the same thread, process, machine, or across separate machines. They also define **store management**, wrapping the store and specifying how data is shared between bundles. In terms of **scalability**, the strategy can replicate the runner bundle across multiple threads, processes, or machines to expand throughput on the runner side. The algorithm side remains single-process due to the complexity of parallelization. Mature frameworks such as _DeepSpeed_ and _Megatron_ already support distributed model training, so scaling of the algorithm bundle is delegated to those implementations. **Abort handling** is another core responsibility. Aborts may be triggered by normal exits, failures in either bundle, or user interrupts. The trainer must include cancellation interfaces for the bundles so that bundles can be cleanly aborted. When the algorithm bundle exits normally, the strategy signals the runner bundle to terminate. If the runner exits first, no signal is sent to the algorithm, as it may still be processing completed rollouts. In cases of failure or user interruption, the strategy signals both bundles to abort; if a bundle fails to respond, the strategy should attempt a forceful termination. Agent-lightning currently provides two execution strategies: **shared-memory** and **client-server**, described in the following sections. ### Shared-memory Strategy[¶](https://microsoft.github.io/agent-lightning/0.2.0/deep-dive/birds-eye-view/#shared-memory-strategy "Permanent link") [`SharedMemoryExecutionStrategy`](https://microsoft.github.io/agent-lightning/0.2.0/reference/trainer/#agentlightning.SharedMemoryExecutionStrategy " agentlightning.SharedMemoryExecutionStrategy") runs algorithm and runner bundles as threads in one process. The strategy wraps the store with [`LightningStoreThreaded`](https://microsoft.github.io/agent-lightning/0.2.0/reference/store/#agentlightning.LightningStoreThreaded " agentlightning.LightningStoreThreaded") , which guards calls with a lock for safe concurrency. This is good for lightweight debugging because components share one Python heap and avoid serialization. It is not suitable for heavy RL training or compute-intensive agents. flowchart TB subgraph MainProcess direction TB subgraph AlgorithmThread [Thread 0] Algorithm[Algorithm bundle] end subgraph RunnerThread1 [Thread 1] Runner1[Runner bundle #1] end subgraph RunnerThread2 [Thread 2] Runner2[Runner bundle #2] end subgraph RunnerThread3 [Thread 3] RunnerN[Runner bundle #N] end LightningStoreFacade[LightningStoreThreaded] BaseStore[Underlying LightningStore] end Algorithm -- async calls --> LightningStoreFacade Runner1 -- async calls --> LightningStoreFacade Runner2 -- async calls --> LightningStoreFacade RunnerN -- async calls --> LightningStoreFacade LightningStoreFacade -->|thread-safe delegates| BaseStore You can configure which role runs on the main thread. If the main thread runs the algorithm, it is able to spawn multiple runner threads. If it runs a runner, `n_runners` must be 1 and the runner lives on the main thread. ### Client-server Strategy[¶](https://microsoft.github.io/agent-lightning/0.2.0/deep-dive/birds-eye-view/#client-server-strategy "Permanent link") [](https://microsoft.github.io/agent-lightning/0.2.0/deep-dive/birds-eye-view/) [`ClientServerExecutionStrategy`](https://microsoft.github.io/agent-lightning/0.2.0/reference/trainer/#agentlightning.ClientServerExecutionStrategy " agentlightning.ClientServerExecutionStrategy") splits concerns across processes. The algorithm bundle starts a [`LightningStoreServer`](https://microsoft.github.io/agent-lightning/0.2.0/reference/store/#agentlightning.LightningStoreServer " agentlightning.LightningStoreServer") (HTTP API) that wraps the underlying store. Runners connect via [`LightningStoreClient`](https://microsoft.github.io/agent-lightning/0.2.0/reference/store/#agentlightning.LightningStoreClient " agentlightning.LightningStoreClient") to call the same interface over REST. The server embeds a client to support algorithm-launched subprocesses (e.g., an LLM proxy worker) that need to talk back to the algorithm’s process through the same API. Currently this design introduces an extra wrapper in the Server side (as shown in the diagram), which helps debugging and improves fault tolerance. We might revisit this design in the future and enforce the client to be the only way to communicate with the store. flowchart TD subgraph Algorithm Process Group subgraph StoreServer[LightningStoreServer] StoreHttpClient[HTTP Client] StoreHttpServer[HTTP Server] StoreWrapper[LightningStore Wrapper] StoreHttpClient -- HTTP --> StoreHttpServer end subgraph Algorithm Bundle Algorithm[Algorithm Main Process] subgraph Another subprocess LLMProxy[LLM Proxy] end end LLMProxy -- async calls --> StoreHttpClient Algorithm -- async calls --> StoreWrapper end subgraph RunnerSide ["Runner Side"] subgraph Runner Process 1 Runner1[Runner bundle #1] Runner1 -- async calls --> LightningStoreClient1 LightningStoreClient1[LightningStoreClient] end subgraph Runner Process 2 Runner2[Runner bundle #2] Runner2 -- async calls --> LightningStoreClient2 LightningStoreClient2[LightningStoreClient] end subgraph Runner Process N RunnerN[Runner bundle #N] RunnerN -- async calls --> LightningStoreClientN LightningStoreClientN[LightningStoreClient] end end LocalStore[Underlying LightningStore] StoreHttpServer -->|delegates| StoreWrapper StoreWrapper -->|delegates| LocalStore LightningStoreClient1 -- HTTP --> StoreHttpServer LightningStoreClient2 -- HTTP --> StoreHttpServer LightningStoreClientN -- HTTP --> StoreHttpServer style RunnerSide fill:none; Online/Continuous Learning[¶](https://microsoft.github.io/agent-lightning/0.2.0/deep-dive/birds-eye-view/#onlinecontinuous-learning "Permanent link") ------------------------------------------------------------------------------------------------------------------------------------------------------ Continuous learning keeps the algorithm loop running while runners report tasks and spans opportunistically. Key differences from batch mode: 1. The algorithm does not enqueue rollouts from a fixed dataset. Runners report tasks/rollouts and spans spontaneously. 2. The algorithm can wait for rollouts with a expected set of rollout IDs, but more oftenly polls for new rollouts and spans or waits for a count to arrive. 3. The [`Runner`](https://microsoft.github.io/agent-lightning/0.2.0/reference/runner/#agentlightning.Runner " agentlightning.Runner") processes one rollout at a time via [`step(task)`](https://microsoft.github.io/agent-lightning/0.2.0/reference/runner/#agentlightning.Runner.step " step(input, *, resources=None, mode=None, event=None) async ") instead of exhausting a task queue. It notifies the store when starting a rollout so the store records it. 4. A user or higher-level loop controls which resources the next step uses and when to retry. [Spans](https://microsoft.github.io/agent-lightning/0.2.0/reference/types/#agentlightning.Span " agentlightning.Span") , [Adapter](https://microsoft.github.io/agent-lightning/0.2.0/reference/algorithm/#agentlightning.Adapter " agentlightning.Adapter") implementations, and the [LLM Proxy](https://microsoft.github.io/agent-lightning/0.2.0/reference/algorithm/#agentlightning.LLMProxy " agentlightning.LLMProxy") work the same way. sequenceDiagram autonumber actor User participant Runner participant Agent participant Store as LightningStore participant Algorithm Note over Algorithm: Algorithm is long-running and loops continuously loop Continuous Learning Loop activate User opt Decide what to do next User-->>Store: get_resources_by_id Store-->>User: Resources User-->>User: Prepare input for next step end User->>Runner: step(input, resources) activate Runner Runner-->>Store: Notify: start_rollout(input) Runner->>Agent: rollout(input, resources) Agent-->>Runner: add_span / reward spans Runner-->>Store: add_span or add_otel_span Runner-->>Store: update_attempt(status="finished") deactivate Runner deactivate User Algorithm->>Store: poll for new rollouts and spans opt If there is enough new data Store-->>Algorithm: new spans Algorithm->>Algorithm: adapt spans → learning signal Algorithm->>Store: update_resources end end Back to top --- # Understanding Store - Agent-lightning [Skip to content](https://microsoft.github.io/agent-lightning/0.2.1/deep-dive/store/#understanding-store) ⚠️ You are viewing development documentation. For stable documentation, please visit [https://microsoft.github.io/agent-lightning/stable/](https://microsoft.github.io/agent-lightning/stable/) Understanding Store[¶](https://microsoft.github.io/agent-lightning/0.2.1/deep-dive/store/#understanding-store "Permanent link") ================================================================================================================================ The **[`LightningStore`](https://microsoft.github.io/agent-lightning/0.2.1/reference/store/#agentlightning.LightningStore " agentlightning.LightningStore") ** is the central coordination point for Agent-lightning. It holds the task queue, rollouts, attempts, spans, and versioned resources, and exposes a small API both Runners and Algorithms use to communicate. This document explains what’s in the store, how statuses transition, how spans are recorded, and the concurrency model (threads & processes). What’s in the Store?[¶](https://microsoft.github.io/agent-lightning/0.2.1/deep-dive/store/#whats-in-the-store "Permanent link") -------------------------------------------------------------------------------------------------------------------------------- ![Store Architecture](https://microsoft.github.io/agent-lightning/0.2.1/assets/store-api-visualized.svg) At a high level: * **Task Queue** – [`enqueue_rollout`](https://microsoft.github.io/agent-lightning/0.2.1/reference/store/#agentlightning.LightningStore.enqueue_rollout " enqueue_rollout(input, mode=None, resources_id=None, config=None, metadata=None) async ") adds work; workers poll with [`dequeue_rollout`](https://microsoft.github.io/agent-lightning/0.2.1/reference/store/#agentlightning.LightningStore.dequeue_rollout " dequeue_rollout() async ") . When a rollout is dequeued, it automatically creates a new attempt associated with itself. * **Rollouts** – A rollout is one unit of work. It has input, metadata, links to resources, and a lifecycle (`queuing → preparing → running → ...`). Valid [RolloutStatus](https://microsoft.github.io/agent-lightning/0.2.1/reference/types/#agentlightning.RolloutStatus " agentlightning.RolloutStatus = Literal['queuing', 'preparing', 'running', 'failed', 'succeeded', 'cancelled', 'requeuing'] module-attribute ") are **`queuing`**, `preparing`, `running`, `succeeded`, `failed`, **`requeuing`**, **`cancelled`**. For algorithms and runners, the rollout can be seen as a whole, without worrying about the internal attempts. * **Attempts** – Each rollout can have multiple executions (retries). Attempts track [`status`](https://microsoft.github.io/agent-lightning/0.2.1/reference/types/#agentlightning.Attempt.status " status = 'preparing' class-attribute instance-attribute ") , [`start_time`](https://microsoft.github.io/agent-lightning/0.2.1/reference/types/#agentlightning.Attempt.start_time " start_time instance-attribute ") , [`end_time`](https://microsoft.github.io/agent-lightning/0.2.1/reference/types/#agentlightning.Attempt.end_time " end_time = None class-attribute instance-attribute ") , [`last_heartbeat_time`](https://microsoft.github.io/agent-lightning/0.2.1/reference/types/#agentlightning.Attempt.last_heartbeat_time " last_heartbeat_time = None class-attribute instance-attribute ") and link to spans. Valid [AttemptStatus](https://microsoft.github.io/agent-lightning/0.2.1/reference/types/#agentlightning.AttemptStatus " agentlightning.AttemptStatus = Literal['preparing', 'running', 'failed', 'succeeded', 'unresponsive', 'timeout'] module-attribute ") are `preparing`, `running`, `succeeded`, `failed`, `requeuing`, `cancelled`. * **Spans** – Structured trace events produced by the Tracer during an attempt. Spans are ordered by a **monotonic sequence id** per `(rollout_id, attempt_id)`. * **Resources** – Versioned, named bundles (e.g., prompt templates) referenced by rollouts. Rollout and Task share the same surface in practice: [`Rollout.input`](https://microsoft.github.io/agent-lightning/0.2.1/reference/types/#agentlightning.Rollout " agentlightning.Rollout") is the task input. The queue stores rollouts that are not yet running; [Runners](https://microsoft.github.io/agent-lightning/0.2.1/reference/runner/#agentlightning.Runner " agentlightning.Runner") dequeue them and update the same rollout’s status as work progresses. All [`LightningStore`](https://microsoft.github.io/agent-lightning/0.2.1/reference/store/#agentlightning.LightningStore " agentlightning.LightningStore") implementations must inherit from [`LightningStore`](https://microsoft.github.io/agent-lightning/0.2.1/reference/store/#agentlightning.LightningStore " agentlightning.LightningStore") and override the methods to implement the storage logic. Before we look at status transitions, it helps to keep in mind that rollouts are the “outside view,” while attempts are the “inside view.” Attempts are what actually run; rollouts summarize the latest attempt plus a small set of control actions like queueing and cancellation. Attempt Status Transitions[¶](https://microsoft.github.io/agent-lightning/0.2.1/deep-dive/store/#attempt-status-transitions "Permanent link") ---------------------------------------------------------------------------------------------------------------------------------------------- The status model is intentionally small and operationally clear. stateDiagram-v2 direction LR [*] --> preparing: Runner calls dequeue_rollout()
or start_rollout()
or start_attempt() preparing --> running: Runner calls
add_[otel_]span()
for the first time state c_runner <> state c_watch <> preparing --> c_runner: Runner calls
update_attempt(...) running --> c_runner: Runner calls
update_attempt(...) running --> c_watch: Watchdog checks preparing --> c_watch: Watchdog checks state "Client-set outcome" as Client { direction TB succeeded failed } state "Watchdog / policy" as Watch { direction TB timeout unresponsive } c_runner --> succeeded: update_attempt(status=succeeded) c_runner --> failed: update_attempt(status=failed) c_watch --> timeout: now - start_time > timeout_seconds c_watch --> unresponsive: now - last_heartbeat > unresponsive_seconds unresponsive --> running: Runner calls
add_[otel_]span() Each attempt begins in **preparing**, created either when a rollout is dequeued or explicitly started. It transitions to **running** the first time a span is recorded. From there, a few clear rules govern how it can change: * When the runner explicitly marks completion, the attempt becomes **succeeded** or **failed** (when the runner catches exception thrown out by the agent). * When the watchdog detects that the total elapsed time since start exceeds the configured limit, it marks the attempt as **timeout**. * If heartbeats stop arriving for too long, the watchdog marks it **unresponsive**. * A new span from the runner can immediately revive an **unresponsive** attempt back to **running**. What's a Watchdog? The watchdog enforces timing and liveness rules defined by each rollout’s [`RolloutConfig`](https://microsoft.github.io/agent-lightning/0.2.1/reference/types/#agentlightning.RolloutConfig " agentlightning.RolloutConfig") . It’s not a separate thread or service, but a function periodically invoked (e.g., before store mutations) to keep attempts healthy and consistent. This simple model allows the system to distinguish between normal termination, abnormal stalling, and recoverable interruption without additional state flags. Rollout Transition Map[¶](https://microsoft.github.io/agent-lightning/0.2.1/deep-dive/store/#rollout-transition-map "Permanent link") -------------------------------------------------------------------------------------------------------------------------------------- Rollout status is an **aggregated view** of its latest attempt’s status, with additional transitions for queueing and explicit cancellation. A rollout’s retry behavior is controlled by [`Rollout.config`](https://microsoft.github.io/agent-lightning/0.2.1/reference/types/#agentlightning.Rollout " agentlightning.Rollout") (a [`RolloutConfig`](https://microsoft.github.io/agent-lightning/0.2.1/reference/types/#agentlightning.RolloutConfig " agentlightning.RolloutConfig") ). The key fields are: * [`timeout_seconds`](https://microsoft.github.io/agent-lightning/0.2.1/reference/types/#agentlightning.RolloutConfig.timeout_seconds " timeout_seconds = None class-attribute instance-attribute ") – maximum wall-clock time for an attempt before it is marked `timeout`. * [`unresponsive_seconds`](https://microsoft.github.io/agent-lightning/0.2.1/reference/types/#agentlightning.RolloutConfig.unresponsive_seconds " unresponsive_seconds = None class-attribute instance-attribute ") – maximum silence between heartbeats before an attempt is marked `unresponsive`. * [`max_attempts`](https://microsoft.github.io/agent-lightning/0.2.1/reference/types/#agentlightning.RolloutConfig.max_attempts " max_attempts = Field(default=1, ge=1) class-attribute instance-attribute ") – total number of attempts allowed for the rollout (including the first). * [`retry_condition`](https://microsoft.github.io/agent-lightning/0.2.1/reference/types/#agentlightning.RolloutConfig.retry_condition " retry_condition = Field(default_factory=(cast(Callable[[], List[AttemptStatus]], list))) class-attribute instance-attribute ") – which attempt terminal statuses should trigger a retry (e.g., `["failed", "timeout", "unresponsive"]`). **How it plays out:** The runner works on attempt `k`. If the attempt ends in a status that is listed in `retry_condition`, and `k < max_attempts`, the rollout moves to **requeuing** and the store creates attempt `k+1`. Otherwise, the rollout becomes **failed** (or **succeeded** if the runner marked it so). `timeout_seconds` and `unresponsive_seconds` are enforced by the watchdog and feed into the same decision flow. A minimal example of how to use `RolloutConfig`: `[](https://microsoft.github.io/agent-lightning/0.2.1/deep-dive/store/#__codelineno-0-1) from agentlightning import RolloutConfig [](https://microsoft.github.io/agent-lightning/0.2.1/deep-dive/store/#__codelineno-0-2) [](https://microsoft.github.io/agent-lightning/0.2.1/deep-dive/store/#__codelineno-0-3) # Retry on explicit failures or timeouts, up to 3 attempts in total. [](https://microsoft.github.io/agent-lightning/0.2.1/deep-dive/store/#__codelineno-0-4) cfg = RolloutConfig( [](https://microsoft.github.io/agent-lightning/0.2.1/deep-dive/store/#__codelineno-0-5) timeout_seconds=600, [](https://microsoft.github.io/agent-lightning/0.2.1/deep-dive/store/#__codelineno-0-6) unresponsive_seconds=120, [](https://microsoft.github.io/agent-lightning/0.2.1/deep-dive/store/#__codelineno-0-7) max_attempts=3, [](https://microsoft.github.io/agent-lightning/0.2.1/deep-dive/store/#__codelineno-0-8) retry_condition=["failed", "timeout"] [](https://microsoft.github.io/agent-lightning/0.2.1/deep-dive/store/#__codelineno-0-9) ) [](https://microsoft.github.io/agent-lightning/0.2.1/deep-dive/store/#__codelineno-0-10) [](https://microsoft.github.io/agent-lightning/0.2.1/deep-dive/store/#__codelineno-0-11) # When creating/enqueuing a rollout, attach this config. [](https://microsoft.github.io/agent-lightning/0.2.1/deep-dive/store/#__codelineno-0-12) # The store will propagate attempt outcomes according to cfg. [](https://microsoft.github.io/agent-lightning/0.2.1/deep-dive/store/#__codelineno-0-13) rollout = await store.enqueue_rollout(input, config=cfg)` | Latest attempt status | Rollout transition | Notes / guards | | --- | --- | --- | | N/A | `queuing` | Created by `enqueue_rollout()`. | | `preparing` | `queuing/requeuing` → `preparing` | Typically `dequeue_rollout()` or `start_rollout()`/`start_attempt()` creates a new attempt. | | `running` | `preparing/queuing/requeuing` → `running` | First `add_[otel_]span()` flips the attempt to `running`; rollout follows via `propagate_status`. | | `succeeded` | `*` → `succeeded` | Terminal. Rollout `end_time` set. | | `failed` / `timeout` / `unresponsive` | `*` → `requeuing` | **Only if** `status ∈ retry_condition ∧ sequence_id < max_attempts`. | | `failed` / `timeout` / `unresponsive` | `*` → `failed` | Otherwise (no retries left or retries disabled). | | `*` | `*` → `cancelled` | Explicitly set by `update_rollout(status=cancelled)`. | Why aggregation? In code, we use `propagate_status()` which actively updates the rollout based on the latest attempt. Reading the table above is usually easier than reverse-engineering the propagation logic in the code: think of the rollout’s transitions as _callbacks_ on attempt state changes, plus queue/cancel paths. Spans[¶](https://microsoft.github.io/agent-lightning/0.2.1/deep-dive/store/#spans "Permanent link") ---------------------------------------------------------------------------------------------------- Every traceable operation in a rollout is stored as a [Span](https://microsoft.github.io/agent-lightning/0.2.1/reference/types/#agentlightning.Span " agentlightning.Span") . Spans not only capture fine-grained instrumentation but also act as periodic heartbeats that demonstrate liveness. The first span marks activation; each subsequent one both extends the trace and refreshes the attempt’s [`last_heartbeat_time`](https://microsoft.github.io/agent-lightning/0.2.1/reference/types/#agentlightning.Attempt.last_heartbeat_time " last_heartbeat_time = None class-attribute instance-attribute ") . If no span arrives within the configured [`unresponsive_seconds`](https://microsoft.github.io/agent-lightning/0.2.1/reference/types/#agentlightning.RolloutConfig.unresponsive_seconds " unresponsive_seconds = None class-attribute instance-attribute ") , the watchdog downgrades the attempt to **unresponsive** until activity resumes. Spans are indexed by `(rollout_id, attempt_id, sequence_id)` where the sequence is monotonic. Tracing analysis tools like [Adapter](https://microsoft.github.io/agent-lightning/0.2.1/reference/algorithm/#agentlightning.Adapter " agentlightning.Adapter") usually rely on "time order" to reconstruct the trace. However, in a distributed system, the recorded start time and end time of a span are not necessarily in the right order when they aggregated into a central store. Therefore, we enforce every span creation to retrieve a monotonically increasing [`sequence_id`](https://microsoft.github.io/agent-lightning/0.2.1/reference/types/#agentlightning.Span.sequence_id " sequence_id instance-attribute ") from the store before adding the span. Note In practice, one `sequence_id` can be used to create multiple spans. In that case, the orders between the multiple spans are determined by the order of `start_time` and `end_time` of the spans. ### OpenTelemetry conversion[¶](https://microsoft.github.io/agent-lightning/0.2.1/deep-dive/store/#opentelemetry-conversion "Permanent link") Runners often produce [OpenTelemetry `ReadableSpan`](https://opentelemetry.io/docs/concepts/signals/traces/#attributes) objects directly. The store normalizes them into [`Span`](https://microsoft.github.io/agent-lightning/0.2.1/reference/types/#agentlightning.Span " agentlightning.Span") as follows: 1. The runner first requests [`get_next_span_sequence_id`](https://microsoft.github.io/agent-lightning/0.2.1/reference/store/#agentlightning.LightningStore.get_next_span_sequence_id " get_next_span_sequence_id(rollout_id, attempt_id) async ") via `sequence_id = await store.get_next_span_sequence_id(rollout_id, attempt_id)`. This guarantees ordering within the attempt regardless of clock skew. 2. `trace_id`, `span_id`, `parent_id`, `name`, `status`, timestamps, attributes, events, links, and resource are copied from the OTEL span. Timestamps are auto-normalized to seconds (nanoseconds are converted). 3. OTEL `SpanContext` and parent context are preserved so downstream tools can correlate traces across systems. 4. Any additional serializable fields present on the `ReadableSpan` are retained in the stored span (after safe JSON serialization), which keeps the representation forward-compatible. Programmatically this is encapsulated by [`Span.from_opentelemetry(readable_span, rollout_id, attempt_id, sequence_id)`](https://microsoft.github.io/agent-lightning/0.2.1/reference/types/#agentlightning.Span.from_opentelemetry " from_opentelemetry(src, rollout_id, attempt_id, sequence_id) classmethod ") ; [`store.add_otel_span(...)`](https://microsoft.github.io/agent-lightning/0.2.1/reference/store/#agentlightning.LightningStore.add_otel_span " add_otel_span(rollout_id, attempt_id, readable_span, sequence_id=None) async ") simply wraps the fetch-then-add flow. The end result is a store span that is stable to sort, merge, and query, while still preserving the richness of the original OTEL payload. Tip [`add_span`](https://microsoft.github.io/agent-lightning/0.2.1/reference/store/#agentlightning.LightningStore.add_span " add_span(span) async ") or [`add_otel_span`](https://microsoft.github.io/agent-lightning/0.2.1/reference/store/#agentlightning.LightningStore.add_otel_span " add_otel_span(rollout_id, attempt_id, readable_span, sequence_id=None) async ") both appends a span _and_ acts as a heartbeat that can revive `unresponsive` → `running`. Store Implementations[¶](https://microsoft.github.io/agent-lightning/0.2.1/deep-dive/store/#store-implementations "Permanent link") ------------------------------------------------------------------------------------------------------------------------------------ Currently, the only out-of-the-box implementation is [`InMemoryLightningStore`](https://microsoft.github.io/agent-lightning/0.2.1/reference/store/#agentlightning.InMemoryLightningStore " agentlightning.InMemoryLightningStore") : * Fast startup, zero external dependencies, and ideal for local development, CI, and unit tests. * Fully asyncio-safe for writes; most reader operations can iterate without locks, except those that need to perform multiple queries. * Includes a best-effort span eviction policy once memory crosses a configured watermark; querying evicted spans raises a clear error so callers can fall back. For production you will likely want persistence. We’re actively building a SQLite-backed store that keeps the same API surface while adding durability, crash recovery, and better historical span queries. If you need something sooner, implement your own store by subclassing [`LightningStore`](https://microsoft.github.io/agent-lightning/0.2.1/reference/store/#agentlightning.LightningStore " agentlightning.LightningStore") and providing concrete storage for the small set of abstract methods (`enqueue_rollout`, `dequeue_rollout`, `update_attempt`, `add_span`, etc.). This document plus the tests in `tests/store/` illustrate the expected behavior. Thread Safety[¶](https://microsoft.github.io/agent-lightning/0.2.1/deep-dive/store/#thread-safety "Permanent link") -------------------------------------------------------------------------------------------------------------------- **[`LightningStoreThreaded`](https://microsoft.github.io/agent-lightning/0.2.1/reference/store/#agentlightning.LightningStoreThreaded " agentlightning.LightningStoreThreaded") ** is a subclass of [`LightningStore`](https://microsoft.github.io/agent-lightning/0.2.1/reference/store/#agentlightning.LightningStore " agentlightning.LightningStore") that wraps another underlying store to make a store instance safe for multi-threaded callers. It wraps every state-mutating call in a mutex. Specifically: * Methods like [`start_rollout`](https://microsoft.github.io/agent-lightning/0.2.1/reference/store/#agentlightning.LightningStore.start_rollout " start_rollout(input, mode=None, resources_id=None, config=None, metadata=None) async ") , [`enqueue_rollout`](https://microsoft.github.io/agent-lightning/0.2.1/reference/store/#agentlightning.LightningStore.enqueue_rollout " enqueue_rollout(input, mode=None, resources_id=None, config=None, metadata=None) async ") , [`update_attempt`](https://microsoft.github.io/agent-lightning/0.2.1/reference/store/#agentlightning.LightningStore.update_attempt " update_attempt(rollout_id, attempt_id, status=UNSET, worker_id=UNSET, last_heartbeat_time=UNSET, metadata=UNSET) async ") , [`add_span`](https://microsoft.github.io/agent-lightning/0.2.1/reference/store/#agentlightning.LightningStore.add_span " add_span(span) async ") , etc. are guarded by a lock. * Non-mutating, potentially blocking calls remain pass-through by design (e.g., [`wait_for_rollouts`](https://microsoft.github.io/agent-lightning/0.2.1/reference/store/#agentlightning.LightningStore.wait_for_rollouts " wait_for_rollouts(*, rollout_ids, timeout=None) async ") ), as they don’t modify shared state and should not hold the lock for long periods. Process Safety and Client-server Store[¶](https://microsoft.github.io/agent-lightning/0.2.1/deep-dive/store/#process-safety-and-client-server-store "Permanent link") ---------------------------------------------------------------------------------------------------------------------------------------------------------------------- **[`LightningStoreServer`](https://microsoft.github.io/agent-lightning/0.2.1/reference/store/#agentlightning.LightningStoreServer " agentlightning.LightningStoreServer") ** wraps another underlying store and runs a FastAPI app to expose the store API over HTTP. [`LightningStoreClient`](https://microsoft.github.io/agent-lightning/0.2.1/reference/store/#agentlightning.LightningStoreClient " agentlightning.LightningStoreClient") is a small [`LightningStore`](https://microsoft.github.io/agent-lightning/0.2.1/reference/store/#agentlightning.LightningStore " agentlightning.LightningStore") implementation that talks to the HTTP API. Warning The server HTTP API is not considered a stable API at this moment. Users are encouraged to use the [`LightningStoreClient`](https://microsoft.github.io/agent-lightning/0.2.1/reference/store/#agentlightning.LightningStoreClient " agentlightning.LightningStoreClient") to communicate with the server as a stable interface. The server tracks the creator PID. In the owner process it delegates directly to the in-memory store; in other processes it lazily constructs a [`LightningStoreClient`](https://microsoft.github.io/agent-lightning/0.2.1/reference/store/#agentlightning.LightningStoreClient " agentlightning.LightningStoreClient") to talk to the HTTP API. This prevents accidental cross-process mutation of the wrong memory image. When the server is pickled (e.g., via `multiprocessing`), only the minimal fields are serialized, but **NOT** the FastAPI/uvicorn objects. Subprocesses won’t accidentally carry live server state. Forked subprocess should also use [`LightningStoreClient`](https://microsoft.github.io/agent-lightning/0.2.1/reference/store/#agentlightning.LightningStoreClient " agentlightning.LightningStoreClient") to communicate with the server in the main process. On the client side, the client retries network/5xx failures using a small backoff, and probes `/health` between attempts. Application exceptions inside the server are wrapped as HTTP 400 with a traceback—these are **not retried**. The client also maintains a **per-event-loop** `aiohttp.ClientSession` map so that tracer callbacks (often on separate loops/threads) don’t hang by reusing a session from another loop. Minimal lifecycle: `[](https://microsoft.github.io/agent-lightning/0.2.1/deep-dive/store/#__codelineno-1-1) import agentlightning as agl [](https://microsoft.github.io/agent-lightning/0.2.1/deep-dive/store/#__codelineno-1-2) [](https://microsoft.github.io/agent-lightning/0.2.1/deep-dive/store/#__codelineno-1-3) # Server (owner process) [](https://microsoft.github.io/agent-lightning/0.2.1/deep-dive/store/#__codelineno-1-4) in_memory_store = agl.InMemoryLightningStore() [](https://microsoft.github.io/agent-lightning/0.2.1/deep-dive/store/#__codelineno-1-5) server = agl.LightningStoreServer(store=in_memory_store, host="0.0.0.0", port=4747) [](https://microsoft.github.io/agent-lightning/0.2.1/deep-dive/store/#__codelineno-1-6) await server.start() # starts uvicorn in a daemon thread and waits for /health [](https://microsoft.github.io/agent-lightning/0.2.1/deep-dive/store/#__codelineno-1-7) # or keep your own event loop and stop via await server.stop() [](https://microsoft.github.io/agent-lightning/0.2.1/deep-dive/store/#__codelineno-1-8) # await server.run_forever() [](https://microsoft.github.io/agent-lightning/0.2.1/deep-dive/store/#__codelineno-1-9) [](https://microsoft.github.io/agent-lightning/0.2.1/deep-dive/store/#__codelineno-1-10) # Client (same or different process) [](https://microsoft.github.io/agent-lightning/0.2.1/deep-dive/store/#__codelineno-1-11) client = agl.LightningStoreClient("http://localhost:4747") [](https://microsoft.github.io/agent-lightning/0.2.1/deep-dive/store/#__codelineno-1-12) [](https://microsoft.github.io/agent-lightning/0.2.1/deep-dive/store/#__codelineno-1-13) print(await client.query_rollouts(status=["queuing"])) [](https://microsoft.github.io/agent-lightning/0.2.1/deep-dive/store/#__codelineno-1-14) [](https://microsoft.github.io/agent-lightning/0.2.1/deep-dive/store/#__codelineno-1-15) await client.close() [](https://microsoft.github.io/agent-lightning/0.2.1/deep-dive/store/#__codelineno-1-16) await server.stop()` Another approach is to use a dedicated command line to start a long running server process, possibly sharable across multiple processes. In the main process, you can always use [`LightningStoreClient`](https://microsoft.github.io/agent-lightning/0.2.1/reference/store/#agentlightning.LightningStoreClient " agentlightning.LightningStoreClient") to communicate with the server. `[](https://microsoft.github.io/agent-lightning/0.2.1/deep-dive/store/#__codelineno-2-1) agl store --port 4747` Note [`LightningStoreClient.wait_for_rollouts`](https://microsoft.github.io/agent-lightning/0.2.1/reference/store/#agentlightning.LightningStoreClient.wait_for_rollouts " wait_for_rollouts(*, rollout_ids, timeout=None) async ") intentionally enforces a tiny timeout (≤ 0.1s) to avoid blocking event loops. Poll with short timeouts or compose with `asyncio.wait_for` at a higher layer. Back to top --- # Command Line - Agent-lightning [Skip to content](https://microsoft.github.io/agent-lightning/0.2.0/reference/cli/#command-line-interface) ⚠️ You are viewing development documentation. For stable documentation, please visit [https://microsoft.github.io/agent-lightning/stable/](https://microsoft.github.io/agent-lightning/stable/) Command Line Interface[¶](https://microsoft.github.io/agent-lightning/0.2.0/reference/cli/#command-line-interface "Permanent link") ==================================================================================================================================== Warning This document is a work in progress and might not be updated with the latest changes. Try to use `agl -h` to get the latest help message. Tip Agent-lightning also provides utilities to help you build your own CLI for [LitAgent](https://microsoft.github.io/agent-lightning/0.2.0/reference/agent/#agentlightning.LitAgent " agentlightning.LitAgent") and [Trainer](https://microsoft.github.io/agent-lightning/0.2.0/reference/trainer/#agentlightning.Trainer " agentlightning.Trainer") . See [Trainer](https://microsoft.github.io/agent-lightning/0.2.0/reference/trainer/) for references. agl[¶](https://microsoft.github.io/agent-lightning/0.2.0/reference/cli/#agl "Permanent link") ---------------------------------------------------------------------------------------------- `[](https://microsoft.github.io/agent-lightning/0.2.0/reference/cli/#__codelineno-0-1) usage: agl [-h] {vllm,store,agentops} [](https://microsoft.github.io/agent-lightning/0.2.0/reference/cli/#__codelineno-0-2) [](https://microsoft.github.io/agent-lightning/0.2.0/reference/cli/#__codelineno-0-3) Agent Lightning CLI entry point. [](https://microsoft.github.io/agent-lightning/0.2.0/reference/cli/#__codelineno-0-4) [](https://microsoft.github.io/agent-lightning/0.2.0/reference/cli/#__codelineno-0-5) Available subcommands: [](https://microsoft.github.io/agent-lightning/0.2.0/reference/cli/#__codelineno-0-6) vllm Run the vLLM CLI with Agent Lightning instrumentation. [](https://microsoft.github.io/agent-lightning/0.2.0/reference/cli/#__codelineno-0-7) store Run a LightningStore server. [](https://microsoft.github.io/agent-lightning/0.2.0/reference/cli/#__codelineno-0-8) agentops Start the AgentOps server manager. [](https://microsoft.github.io/agent-lightning/0.2.0/reference/cli/#__codelineno-0-9) [](https://microsoft.github.io/agent-lightning/0.2.0/reference/cli/#__codelineno-0-10) positional arguments: [](https://microsoft.github.io/agent-lightning/0.2.0/reference/cli/#__codelineno-0-11) {vllm,store,agentops} [](https://microsoft.github.io/agent-lightning/0.2.0/reference/cli/#__codelineno-0-12) Subcommand to run. [](https://microsoft.github.io/agent-lightning/0.2.0/reference/cli/#__codelineno-0-13) [](https://microsoft.github.io/agent-lightning/0.2.0/reference/cli/#__codelineno-0-14) options: [](https://microsoft.github.io/agent-lightning/0.2.0/reference/cli/#__codelineno-0-15) -h, --help show this help message and exit` agl vllm[¶](https://microsoft.github.io/agent-lightning/0.2.0/reference/cli/#agl-vllm "Permanent link") -------------------------------------------------------------------------------------------------------- Agent-lightning's instrumented vLLM CLI. `[](https://microsoft.github.io/agent-lightning/0.2.0/reference/cli/#__codelineno-1-1) usage: agl vllm [-h] [-v] {chat,complete,serve,bench,collect-env,run-batch} ... [](https://microsoft.github.io/agent-lightning/0.2.0/reference/cli/#__codelineno-1-2) [](https://microsoft.github.io/agent-lightning/0.2.0/reference/cli/#__codelineno-1-3) vLLM CLI [](https://microsoft.github.io/agent-lightning/0.2.0/reference/cli/#__codelineno-1-4) [](https://microsoft.github.io/agent-lightning/0.2.0/reference/cli/#__codelineno-1-5) positional arguments: [](https://microsoft.github.io/agent-lightning/0.2.0/reference/cli/#__codelineno-1-6) {chat,complete,serve,bench,collect-env,run-batch} [](https://microsoft.github.io/agent-lightning/0.2.0/reference/cli/#__codelineno-1-7) chat Generate chat completions via the running API server. [](https://microsoft.github.io/agent-lightning/0.2.0/reference/cli/#__codelineno-1-8) complete Generate text completions based on the given prompt via the running API server. [](https://microsoft.github.io/agent-lightning/0.2.0/reference/cli/#__codelineno-1-9) collect-env Start collecting environment information. [](https://microsoft.github.io/agent-lightning/0.2.0/reference/cli/#__codelineno-1-10) run-batch Run batch prompts and write results to file. [](https://microsoft.github.io/agent-lightning/0.2.0/reference/cli/#__codelineno-1-11) [](https://microsoft.github.io/agent-lightning/0.2.0/reference/cli/#__codelineno-1-12) options: [](https://microsoft.github.io/agent-lightning/0.2.0/reference/cli/#__codelineno-1-13) -h, --help show this help message and exit [](https://microsoft.github.io/agent-lightning/0.2.0/reference/cli/#__codelineno-1-14) -v, --version show program's version number and exit [](https://microsoft.github.io/agent-lightning/0.2.0/reference/cli/#__codelineno-1-15) [](https://microsoft.github.io/agent-lightning/0.2.0/reference/cli/#__codelineno-1-16) For full list: vllm [subcommand] --help=all [](https://microsoft.github.io/agent-lightning/0.2.0/reference/cli/#__codelineno-1-17) For a section: vllm [subcommand] --help=ModelConfig (case-insensitive) [](https://microsoft.github.io/agent-lightning/0.2.0/reference/cli/#__codelineno-1-18) For a flag: vllm [subcommand] --help=max-model-len (_ or - accepted) [](https://microsoft.github.io/agent-lightning/0.2.0/reference/cli/#__codelineno-1-19) Documentation: https://docs.vllm.ai` agl store[¶](https://microsoft.github.io/agent-lightning/0.2.0/reference/cli/#agl-store "Permanent link") ---------------------------------------------------------------------------------------------------------- Agent-lightning's LightningStore CLI. Use it to start an independent LightningStore server. Currently the store data are stored in memory and will be lost when the server is stopped. `[](https://microsoft.github.io/agent-lightning/0.2.0/reference/cli/#__codelineno-2-1) usage: agl store [-h] [--port PORT] [](https://microsoft.github.io/agent-lightning/0.2.0/reference/cli/#__codelineno-2-2) [](https://microsoft.github.io/agent-lightning/0.2.0/reference/cli/#__codelineno-2-3) Run a LightningStore server [](https://microsoft.github.io/agent-lightning/0.2.0/reference/cli/#__codelineno-2-4) [](https://microsoft.github.io/agent-lightning/0.2.0/reference/cli/#__codelineno-2-5) options: [](https://microsoft.github.io/agent-lightning/0.2.0/reference/cli/#__codelineno-2-6) -h, --help show this help message and exit [](https://microsoft.github.io/agent-lightning/0.2.0/reference/cli/#__codelineno-2-7) --port PORT Port to run the server on` agl agentops[¶](https://microsoft.github.io/agent-lightning/0.2.0/reference/cli/#agl-agentops "Permanent link") ---------------------------------------------------------------------------------------------------------------- Start a mock AgentOps server to bypass the online service of AgentOps. `[](https://microsoft.github.io/agent-lightning/0.2.0/reference/cli/#__codelineno-3-1) usage: agl agentops [-h] [--daemon] [--port PORT] [](https://microsoft.github.io/agent-lightning/0.2.0/reference/cli/#__codelineno-3-2) [](https://microsoft.github.io/agent-lightning/0.2.0/reference/cli/#__codelineno-3-3) Start AgentOps server [](https://microsoft.github.io/agent-lightning/0.2.0/reference/cli/#__codelineno-3-4) [](https://microsoft.github.io/agent-lightning/0.2.0/reference/cli/#__codelineno-3-5) options: [](https://microsoft.github.io/agent-lightning/0.2.0/reference/cli/#__codelineno-3-6) -h, --help show this help message and exit [](https://microsoft.github.io/agent-lightning/0.2.0/reference/cli/#__codelineno-3-7) --daemon Run server as a daemon [](https://microsoft.github.io/agent-lightning/0.2.0/reference/cli/#__codelineno-3-8) --port PORT Port to run the server on` Back to top --- # Command Line - Agent-lightning [Skip to content](https://microsoft.github.io/agent-lightning/0.2.2/reference/cli/#command-line-interface) ⚠️ You are viewing development documentation. For stable documentation, please visit [https://microsoft.github.io/agent-lightning/stable/](https://microsoft.github.io/agent-lightning/stable/) Command Line Interface[¶](https://microsoft.github.io/agent-lightning/0.2.2/reference/cli/#command-line-interface "Permanent link") ==================================================================================================================================== Warning This document is a work in progress and might not be updated with the latest changes. Try to use `agl -h` to get the latest help message. Tip Agent-lightning also provides utilities to help you build your own CLI for [LitAgent](https://microsoft.github.io/agent-lightning/0.2.2/reference/agent/#agentlightning.LitAgent " agentlightning.LitAgent") and [Trainer](https://microsoft.github.io/agent-lightning/0.2.2/reference/trainer/#agentlightning.Trainer " agentlightning.Trainer") . See [Trainer](https://microsoft.github.io/agent-lightning/0.2.2/reference/trainer/) for references. agl[¶](https://microsoft.github.io/agent-lightning/0.2.2/reference/cli/#agl "Permanent link") ---------------------------------------------------------------------------------------------- `[](https://microsoft.github.io/agent-lightning/0.2.2/reference/cli/#__codelineno-0-1) usage: agl [-h] {vllm,store,agentops} [](https://microsoft.github.io/agent-lightning/0.2.2/reference/cli/#__codelineno-0-2) [](https://microsoft.github.io/agent-lightning/0.2.2/reference/cli/#__codelineno-0-3) Agent Lightning CLI entry point. [](https://microsoft.github.io/agent-lightning/0.2.2/reference/cli/#__codelineno-0-4) [](https://microsoft.github.io/agent-lightning/0.2.2/reference/cli/#__codelineno-0-5) Available subcommands: [](https://microsoft.github.io/agent-lightning/0.2.2/reference/cli/#__codelineno-0-6) vllm Run the vLLM CLI with Agent Lightning instrumentation. [](https://microsoft.github.io/agent-lightning/0.2.2/reference/cli/#__codelineno-0-7) store Run a LightningStore server. [](https://microsoft.github.io/agent-lightning/0.2.2/reference/cli/#__codelineno-0-8) agentops Start the AgentOps server manager. [](https://microsoft.github.io/agent-lightning/0.2.2/reference/cli/#__codelineno-0-9) [](https://microsoft.github.io/agent-lightning/0.2.2/reference/cli/#__codelineno-0-10) positional arguments: [](https://microsoft.github.io/agent-lightning/0.2.2/reference/cli/#__codelineno-0-11) {vllm,store,agentops} [](https://microsoft.github.io/agent-lightning/0.2.2/reference/cli/#__codelineno-0-12) Subcommand to run. [](https://microsoft.github.io/agent-lightning/0.2.2/reference/cli/#__codelineno-0-13) [](https://microsoft.github.io/agent-lightning/0.2.2/reference/cli/#__codelineno-0-14) options: [](https://microsoft.github.io/agent-lightning/0.2.2/reference/cli/#__codelineno-0-15) -h, --help show this help message and exit` agl vllm[¶](https://microsoft.github.io/agent-lightning/0.2.2/reference/cli/#agl-vllm "Permanent link") -------------------------------------------------------------------------------------------------------- Agent-lightning's instrumented vLLM CLI. `[](https://microsoft.github.io/agent-lightning/0.2.2/reference/cli/#__codelineno-1-1) usage: agl vllm [-h] [-v] {chat,complete,serve,bench,collect-env,run-batch} ... [](https://microsoft.github.io/agent-lightning/0.2.2/reference/cli/#__codelineno-1-2) [](https://microsoft.github.io/agent-lightning/0.2.2/reference/cli/#__codelineno-1-3) vLLM CLI [](https://microsoft.github.io/agent-lightning/0.2.2/reference/cli/#__codelineno-1-4) [](https://microsoft.github.io/agent-lightning/0.2.2/reference/cli/#__codelineno-1-5) positional arguments: [](https://microsoft.github.io/agent-lightning/0.2.2/reference/cli/#__codelineno-1-6) {chat,complete,serve,bench,collect-env,run-batch} [](https://microsoft.github.io/agent-lightning/0.2.2/reference/cli/#__codelineno-1-7) chat Generate chat completions via the running API server. [](https://microsoft.github.io/agent-lightning/0.2.2/reference/cli/#__codelineno-1-8) complete Generate text completions based on the given prompt via the running API server. [](https://microsoft.github.io/agent-lightning/0.2.2/reference/cli/#__codelineno-1-9) collect-env Start collecting environment information. [](https://microsoft.github.io/agent-lightning/0.2.2/reference/cli/#__codelineno-1-10) run-batch Run batch prompts and write results to file. [](https://microsoft.github.io/agent-lightning/0.2.2/reference/cli/#__codelineno-1-11) [](https://microsoft.github.io/agent-lightning/0.2.2/reference/cli/#__codelineno-1-12) options: [](https://microsoft.github.io/agent-lightning/0.2.2/reference/cli/#__codelineno-1-13) -h, --help show this help message and exit [](https://microsoft.github.io/agent-lightning/0.2.2/reference/cli/#__codelineno-1-14) -v, --version show program's version number and exit [](https://microsoft.github.io/agent-lightning/0.2.2/reference/cli/#__codelineno-1-15) [](https://microsoft.github.io/agent-lightning/0.2.2/reference/cli/#__codelineno-1-16) For full list: vllm [subcommand] --help=all [](https://microsoft.github.io/agent-lightning/0.2.2/reference/cli/#__codelineno-1-17) For a section: vllm [subcommand] --help=ModelConfig (case-insensitive) [](https://microsoft.github.io/agent-lightning/0.2.2/reference/cli/#__codelineno-1-18) For a flag: vllm [subcommand] --help=max-model-len (_ or - accepted) [](https://microsoft.github.io/agent-lightning/0.2.2/reference/cli/#__codelineno-1-19) Documentation: https://docs.vllm.ai` agl store[¶](https://microsoft.github.io/agent-lightning/0.2.2/reference/cli/#agl-store "Permanent link") ---------------------------------------------------------------------------------------------------------- Agent-lightning's LightningStore CLI. Use it to start an independent LightningStore server. Currently the store data are stored in memory and will be lost when the server is stopped. `[](https://microsoft.github.io/agent-lightning/0.2.2/reference/cli/#__codelineno-2-1) usage: agl store [-h] [--port PORT] [](https://microsoft.github.io/agent-lightning/0.2.2/reference/cli/#__codelineno-2-2) [](https://microsoft.github.io/agent-lightning/0.2.2/reference/cli/#__codelineno-2-3) Run a LightningStore server [](https://microsoft.github.io/agent-lightning/0.2.2/reference/cli/#__codelineno-2-4) [](https://microsoft.github.io/agent-lightning/0.2.2/reference/cli/#__codelineno-2-5) options: [](https://microsoft.github.io/agent-lightning/0.2.2/reference/cli/#__codelineno-2-6) -h, --help show this help message and exit [](https://microsoft.github.io/agent-lightning/0.2.2/reference/cli/#__codelineno-2-7) --port PORT Port to run the server on` agl agentops[¶](https://microsoft.github.io/agent-lightning/0.2.2/reference/cli/#agl-agentops "Permanent link") ---------------------------------------------------------------------------------------------------------------- Start a mock AgentOps server to bypass the online service of AgentOps. `[](https://microsoft.github.io/agent-lightning/0.2.2/reference/cli/#__codelineno-3-1) usage: agl agentops [-h] [--daemon] [--port PORT] [](https://microsoft.github.io/agent-lightning/0.2.2/reference/cli/#__codelineno-3-2) [](https://microsoft.github.io/agent-lightning/0.2.2/reference/cli/#__codelineno-3-3) Start AgentOps server [](https://microsoft.github.io/agent-lightning/0.2.2/reference/cli/#__codelineno-3-4) [](https://microsoft.github.io/agent-lightning/0.2.2/reference/cli/#__codelineno-3-5) options: [](https://microsoft.github.io/agent-lightning/0.2.2/reference/cli/#__codelineno-3-6) -h, --help show this help message and exit [](https://microsoft.github.io/agent-lightning/0.2.2/reference/cli/#__codelineno-3-7) --daemon Run server as a daemon [](https://microsoft.github.io/agent-lightning/0.2.2/reference/cli/#__codelineno-3-8) --port PORT Port to run the server on` Back to top --- # Bird's Eye View - Agent-lightning [Skip to content](https://microsoft.github.io/agent-lightning/0.2.1/deep-dive/birds-eye-view/#the-birds-eye-view-of-agent-lightning) ⚠️ You are viewing development documentation. For stable documentation, please visit [https://microsoft.github.io/agent-lightning/stable/](https://microsoft.github.io/agent-lightning/stable/) The Bird's Eye View of Agent-lightning[¶](https://microsoft.github.io/agent-lightning/0.2.1/deep-dive/birds-eye-view/#the-birds-eye-view-of-agent-lightning "Permanent link") ============================================================================================================================================================================== High Volume of Information Ahead This article provides an in-depth exploration of the Agent-lightning architecture. It is not intended as a beginner’s guide or usage tutorial. This article summarizes how Agent-lightning (as of v0.2) wires the [Algorithm](https://microsoft.github.io/agent-lightning/0.2.1/reference/algorithm/#agentlightning.Algorithm " agentlightning.Algorithm") , [Runner](https://microsoft.github.io/agent-lightning/0.2.1/reference/runner/#agentlightning.Runner " agentlightning.Runner") , and [LightningStore](https://microsoft.github.io/agent-lightning/0.2.1/reference/store/#agentlightning.LightningStore " agentlightning.LightningStore") loop together and shows where auxiliary components (the [Tracer](https://microsoft.github.io/agent-lightning/0.2.1/reference/runner/#agentlightning.Tracer " agentlightning.Tracer") , [Adapter](https://microsoft.github.io/agent-lightning/0.2.1/reference/algorithm/#agentlightning.Adapter " agentlightning.Adapter") , and [LLM Proxy](https://microsoft.github.io/agent-lightning/0.2.1/reference/algorithm/#agentlightning.LLMProxy " agentlightning.LLMProxy") ) plug into the loop. Each section provides a diagram for a different perspective of the system. Algorithm ↔ Runner ↔ Store data flow[¶](https://microsoft.github.io/agent-lightning/0.2.1/deep-dive/birds-eye-view/#algorithm-runner-store-data-flow "Permanent link") ----------------------------------------------------------------------------------------------------------------------------------------------------------------------- At its heart, Agent-lightning is built on three main components that work in a coordinated loop: * **[Algorithm](https://microsoft.github.io/agent-lightning/0.2.1/reference/algorithm/#agentlightning.Algorithm " agentlightning.Algorithm") :** The "brain" of the system. It decides what tasks to run, learns from the results, and updates resources (like AI models or prompts). * **[Runner](https://microsoft.github.io/agent-lightning/0.2.1/reference/runner/#agentlightning.Runner " agentlightning.Runner") :** The "worker" of the system. It executes tasks assigned by the algorithm, runs the agent, and records the results. * **[LightningStore](https://microsoft.github.io/agent-lightning/0.2.1/reference/store/#agentlightning.LightningStore " agentlightning.LightningStore") :** The central "database" and message queue. It acts as the single source of truth, storing tasks, results, and resources, and enabling communication between the Algorithm and Runner. The typical data flow in a training loop is as follows: The **[Algorithm](https://microsoft.github.io/agent-lightning/0.2.1/reference/algorithm/#agentlightning.Algorithm " agentlightning.Algorithm") ** enqueues tasks (called **[Rollouts](https://microsoft.github.io/agent-lightning/0.2.1/reference/types/#agentlightning.Rollout " agentlightning.Rollout") **) into the **[LightningStore](https://microsoft.github.io/agent-lightning/0.2.1/reference/store/#agentlightning.LightningStore " agentlightning.LightningStore") **. A **[Runner](https://microsoft.github.io/agent-lightning/0.2.1/reference/runner/#agentlightning.Runner " agentlightning.Runner") ** then dequeues a task, executes it, and streams the results (called **[Spans](https://microsoft.github.io/agent-lightning/0.2.1/reference/types/#agentlightning.Span " agentlightning.Span") **) back to the store. Once the task is complete, the algorithm can query the new data from the store to learn and update its resources. The diagram below shows this fundamental interaction in a simple, non-parallel setup. sequenceDiagram autonumber participant Algo as Algorithm participant Store as LightningStore participant Runner participant Agent loop Over the dataset Algo-->>Store: add_resources + enqueue_rollout Store-->>Runner: dequeue_rollout → AttemptedRollout Store-->>Runner: get_latest_resources Runner-->>Store: update_attempt("running", worker_id) Runner->>Agent: rollout + resources Agent->>Runner: reward / spans Runner-->>Store: add_span or add_otel_span Runner-->>Store: update_attempt("finished", status) Store-->>Algo: query_rollouts + spans Algo-->>Algo: Update resources (optional) end Solid lines represent direct calls, while dashed lines are asynchronous or long-running operations. ### Key Terminology[¶](https://microsoft.github.io/agent-lightning/0.2.1/deep-dive/birds-eye-view/#key-terminology "Permanent link") We define the following terms, which may be helpful for understanding the diagram above. * **[Resources](https://microsoft.github.io/agent-lightning/0.2.1/reference/types/#agentlightning.Resource " agentlightning.Resource") :** A collection of assets to be tuned or trained. Agents perform rollouts against resources and collect span data. Algorithms use those data to update the resources. In RL training, the resources are a tunable model. In prompt tuning, the resources are prompt templates. * **[Rollout](https://microsoft.github.io/agent-lightning/0.2.1/reference/types/#agentlightning.Rollout " agentlightning.Rollout") :** A unit of work that an agent performs against a resource. A rollout (noun) can be incomplete, in which case it is also known as a **task**, **sample**, or **job** (these terms are used interchangeably). The agent executes its own defined workflow against the rollout — the process is also called "to rollout" (verb). After execution, the rollout (noun) is considered _complete_. * **[Attempt](https://microsoft.github.io/agent-lightning/0.2.1/reference/types/#agentlightning.Attempt " agentlightning.Attempt") :** A single execution of a rollout. One rollout can have multiple attempts in case of failures or timeouts. * **[Span](https://microsoft.github.io/agent-lightning/0.2.1/reference/types/#agentlightning.Span " agentlightning.Span") :** During the rollout, the agent can generate multiple spans (also known as "traces" or "events"). The recorded spans are collected in the store, which is crucial for understanding agent behavior and optimizing agents. * **[Reward](https://microsoft.github.io/agent-lightning/0.2.1/reference/agent/#agentlightning.emit_reward " agentlightning.emit_reward(reward)") :** A special span that is defined as a number judging the quality of the rollout during some period of the rollout. * **[Dataset](https://microsoft.github.io/agent-lightning/0.2.1/reference/types/#agentlightning.Dataset " agentlightning.Dataset") :** A collection of incomplete rollouts (i.e., tasks) for the agent to process. The dual datasets (train, val) serve as the initial input for the algorithm to enqueue the first batch of rollouts. ### Store[¶](https://microsoft.github.io/agent-lightning/0.2.1/deep-dive/birds-eye-view/#store "Permanent link") As discussed previously, the [LightningStore](https://microsoft.github.io/agent-lightning/0.2.1/reference/store/#agentlightning.LightningStore " agentlightning.LightningStore") is the central hub for all data in Agent-lightning. The store exposes a set of APIs for algorithms and runners to interact with the data; the most important ones are: `[](https://microsoft.github.io/agent-lightning/0.2.1/deep-dive/birds-eye-view/#__codelineno-0-1) from agentlightning.types import AttemptedRollout, ResourcesUpdate, Span, TaskInput [](https://microsoft.github.io/agent-lightning/0.2.1/deep-dive/birds-eye-view/#__codelineno-0-2) [](https://microsoft.github.io/agent-lightning/0.2.1/deep-dive/birds-eye-view/#__codelineno-0-3) class LightningStore: [](https://microsoft.github.io/agent-lightning/0.2.1/deep-dive/birds-eye-view/#__codelineno-0-4) [](https://microsoft.github.io/agent-lightning/0.2.1/deep-dive/birds-eye-view/#__codelineno-0-5) async def enqueue_rollout(self, input: TaskInput, ...) -> Rollout: ... [](https://microsoft.github.io/agent-lightning/0.2.1/deep-dive/birds-eye-view/#__codelineno-0-6) [](https://microsoft.github.io/agent-lightning/0.2.1/deep-dive/birds-eye-view/#__codelineno-0-7) async def dequeue_rollout(self) -> AttemptedRollout | None: ... [](https://microsoft.github.io/agent-lightning/0.2.1/deep-dive/birds-eye-view/#__codelineno-0-8) [](https://microsoft.github.io/agent-lightning/0.2.1/deep-dive/birds-eye-view/#__codelineno-0-9) async def add_span(self, span: Span) -> Span: ... [](https://microsoft.github.io/agent-lightning/0.2.1/deep-dive/birds-eye-view/#__codelineno-0-10) [](https://microsoft.github.io/agent-lightning/0.2.1/deep-dive/birds-eye-view/#__codelineno-0-11) async def get_latest_resources(self) -> Optional[ResourcesUpdate]: ... [](https://microsoft.github.io/agent-lightning/0.2.1/deep-dive/birds-eye-view/#__codelineno-0-12) [](https://microsoft.github.io/agent-lightning/0.2.1/deep-dive/birds-eye-view/#__codelineno-0-13) async def wait_for_rollouts(self, rollout_ids: List[str], ...): ... [](https://microsoft.github.io/agent-lightning/0.2.1/deep-dive/birds-eye-view/#__codelineno-0-14) [](https://microsoft.github.io/agent-lightning/0.2.1/deep-dive/birds-eye-view/#__codelineno-0-15) async def query_spans(self, rollout_id: str, ...): ... [](https://microsoft.github.io/agent-lightning/0.2.1/deep-dive/birds-eye-view/#__codelineno-0-16) [](https://microsoft.github.io/agent-lightning/0.2.1/deep-dive/birds-eye-view/#__codelineno-0-17) async def update_attempt(self, rollout_id: str, attempt_id: str, status: str, ...): ... [](https://microsoft.github.io/agent-lightning/0.2.1/deep-dive/birds-eye-view/#__codelineno-0-18) [](https://microsoft.github.io/agent-lightning/0.2.1/deep-dive/birds-eye-view/#__codelineno-0-19) ...` These interfaces operate on [`AttemptedRollout`](https://microsoft.github.io/agent-lightning/0.2.1/reference/types/#agentlightning.AttemptedRollout " agentlightning.AttemptedRollout") , [`ResourcesUpdate`](https://microsoft.github.io/agent-lightning/0.2.1/reference/types/#agentlightning.ResourcesUpdate " agentlightning.ResourcesUpdate") , [`Span`](https://microsoft.github.io/agent-lightning/0.2.1/reference/types/#agentlightning.Span " agentlightning.Span") , and [`TaskInput`](https://microsoft.github.io/agent-lightning/0.2.1/reference/internal/#agentlightning.TaskInput " agentlightning.TaskInput = Any module-attribute ") instances from `agentlightning.types`. As the APIs show, the store essentially provides a queue for rollouts and storage for resources, spans, and attempts. Developers should implement the store carefully to ensure data integrity and consistency, especially when multiple runners work in parallel across multiple attempts. The store is designed to be extensible. Users can implement their own store by inheriting from [`LightningStore`](https://microsoft.github.io/agent-lightning/0.2.1/reference/store/#agentlightning.LightningStore " agentlightning.LightningStore") and overriding methods. Agent-lightning provides a few reference implementations, such as [`InMemoryLightningStore`](https://microsoft.github.io/agent-lightning/0.2.1/reference/store/#agentlightning.InMemoryLightningStore " agentlightning.InMemoryLightningStore") (default) and `SqliteLightningStore` (under construction). When parallelized, the store may need special wrappers to ensure thread/process safety or delegate computation to a store in another process or machine. Supporting Components in the Loop[¶](https://microsoft.github.io/agent-lightning/0.2.1/deep-dive/birds-eye-view/#supporting-components-in-the-loop "Permanent link") --------------------------------------------------------------------------------------------------------------------------------------------------------------------- While the core loop is simple, Agent-lightning provides several components to make development easier and more powerful. ### Tracer[¶](https://microsoft.github.io/agent-lightning/0.2.1/deep-dive/birds-eye-view/#tracer "Permanent link") The [`Tracer`](https://microsoft.github.io/agent-lightning/0.2.1/reference/runner/#agentlightning.Tracer " agentlightning.Tracer") is a component within the [`Runner`](https://microsoft.github.io/agent-lightning/0.2.1/reference/runner/#agentlightning.Runner " agentlightning.Runner") that records detailed spans (events) during an agent's execution and sends them to the [`LightningStore`](https://microsoft.github.io/agent-lightning/0.2.1/reference/store/#agentlightning.LightningStore " agentlightning.LightningStore") . Instead of requiring the agent to manually log every span, the tracer automatically instruments key methods (e.g., LLM calls) and captures their inputs, outputs, and metadata. This provides a detailed log of the agent's behavior with minimal effort. sequenceDiagram autonumber participant Store participant Runner participant Tracer participant Agent Note over Runner,Tracer: Runner manages tracer as member Tracer->>Agent: Apply instrumentation loop Until no more rollouts Store-->>Runner: dequeue_rollout → AttemptedRollout Store-->>Runner: get_latest_resources Runner->>Agent: training_rollout / validation_rollout loop For each finished span Agent-->>Tracer: openai.chat.completion invoked
agent.execute invoked
... Agent->>Tracer: emit intermediate reward Tracer-->>Store: add_otel_span(rollout_id, attempt_id, span) end Agent->>Runner: final reward + extra spans (if any) Runner-->>Store: add_span(rollout_id, attempt_id, span) Runner-->>Store: update_attempt(status) end Tracer->>Agent: Unapply instrumentation The above diagram shows the overall data flow between store, tracer and agent. In realistic, it's a bit more complicated than that. Spans are not emitted actively by the agent; they are intercepted by the tracer by hooking and instrumenting key methods used in the agents. The tracer uses a callback (called exporter) to monitor events and log to the store. Before a rollout starts, the runner enters a [`trace_context`](https://microsoft.github.io/agent-lightning/0.2.1/reference/runner/#agentlightning.Tracer.trace_context " trace_context(name=None, *, store=None, rollout_id=None, attempt_id=None)") before invoking the agent, wiring store identifiers into the tracer. Each span completion streams back to the store through `LightningSpanProcessor`, so the agent’s instrumentation lands in [`add_otel_span`](https://microsoft.github.io/agent-lightning/0.2.1/reference/store/#agentlightning.LightningStore.add_otel_span " add_otel_span(rollout_id, attempt_id, readable_span, sequence_id=None) async ") . If the agent’s rollout method returns a numeric reward, the runner emits one more OpenTelemetry span before finalizing the attempt. ### Hooks[¶](https://microsoft.github.io/agent-lightning/0.2.1/deep-dive/birds-eye-view/#hooks "Permanent link") [`Hook`](https://microsoft.github.io/agent-lightning/0.2.1/reference/types/#agentlightning.Hook " agentlightning.Hook") implementations are user-defined callback functions that allow you to augment a [`Runner`](https://microsoft.github.io/agent-lightning/0.2.1/reference/runner/#agentlightning.Runner " agentlightning.Runner") 's behavior at specific points in its lifecycle. You can use hooks to add custom logging, set up resources before a rollout begins, or tear them down after it ends. Hooks can be triggered at four key moments: `on_rollout_start`, `on_trace_start`, `on_trace_end`, and `on_rollout_end`. Users should pay special attention to the difference between `on_trace_end` and `on_rollout_end`. The former is called right before the tracer exits the trace context, while the latter is called after the runner processes the final leftover rewards and spans, and finalizes the attempt in the store. sequenceDiagram autonumber participant Store participant Hooks participant Runner participant Tracer participant Agent Note over Runner,Hooks: Runner manages hooks as member loop Until no more rollouts Store-->>Runner: dequeue_rollout → AttemptedRollout Store-->>Runner: get_latest_resources Runner->>Hooks: on_rollout_start(agent, runner, rollout) Runner->>Agent: training_rollout / validation_rollout Tracer->>Agent: enter_trace_context activate Tracer Runner->>Hooks: on_trace_start(agent, runner, tracer, rollout) Note over Runner,Agent: Agent rollout omitted Runner->>Hooks: on_trace_end(agent, runner, tracer, rollout) Tracer->>Agent: exit_trace_context deactivate Tracer Agent->>Runner: final reward + extra spans (if any) Runner-->>Store: add_span(rollout_id, attempt_id, span) Runner->>Hooks: on_rollout_end(agent, runner, rollout, status) end ### Adapter[¶](https://microsoft.github.io/agent-lightning/0.2.1/deep-dive/birds-eye-view/#adapter "Permanent link") The [`Adapter`](https://microsoft.github.io/agent-lightning/0.2.1/reference/algorithm/#agentlightning.Adapter " agentlightning.Adapter") is a component used by the [`Algorithm`](https://microsoft.github.io/agent-lightning/0.2.1/reference/algorithm/#agentlightning.Algorithm " agentlightning.Algorithm") to transform raw data from the [`LightningStore`](https://microsoft.github.io/agent-lightning/0.2.1/reference/store/#agentlightning.LightningStore " agentlightning.LightningStore") into a format suitable for learning. Runners stream raw spans into the store during execution. Later, the algorithm queries these spans and uses an adapter to convert them into structured data, like training examples for a reinforcement learning model. For instance, the [`TracerTraceToTriplet`](https://microsoft.github.io/agent-lightning/0.2.1/reference/algorithm/#agentlightning.TracerTraceToTriplet " agentlightning.TracerTraceToTriplet") processes OpenTelemetry spans to create `(prompt, response, reward)` triplets, which are the fundamental data structure for many RL fine-tuning algorithms. flowchart LR Runner -- (1) add_otel_span --> Store Store -- (2) query_spans --> Algorithm Algorithm -- (3) spans --> Adapter Adapter -- (4) transformed data --> Algorithm ### LLM Proxy[¶](https://microsoft.github.io/agent-lightning/0.2.1/deep-dive/birds-eye-view/#llm-proxy "Permanent link") The [`LLMProxy`](https://microsoft.github.io/agent-lightning/0.2.1/reference/algorithm/#agentlightning.LLMProxy " agentlightning.LLMProxy") is an optional bridge component that sits between an agent and the algorithms' resources. It acts as a centralized endpoint for all LLM calls. Usually the proxy URL is added to the store as a special resource, so that the [`Runner`](https://microsoft.github.io/agent-lightning/0.2.1/reference/runner/#agentlightning.Runner " agentlightning.Runner") can fetch it along with other resources when dequeuing a rollout. During rollouts, the runner invokes the proxy's HTTP endpoint instead of calling a model backend directly. This design offers several benefits: 1. **Instrumentation:** It automatically captures detailed traces of LLM interactions (prompts, responses, metadata) and sends them to the store, complementing the tracer, especially when the agent's code is hard to instrument directly. 2. **Backend Abstraction:** It provides a unified interface for various LLM backends (OpenAI, Anthropic, local models) and can add features like retry logic, rate limiting, and caching. 3. **Resource Management:** The algorithm can dynamically update which LLM the agent uses (e.g., swapping to a newly fine-tuned model) by simply swapping the backend model the proxy is using, without interrupting the agent's code. The benefits above seem to be all discussed within the context of model fine-tuning. As a matter of fact, the proxy can be useful for prompt tuning as well. The algorithm can register one of the following two types of endpoints into the proxy: 1. **Endpoint served by the algorithm:** If the algorithm is internally updating the LLM weights (e.g., RL), it can launch an LLM inference engine (i.e., a model server) and register the endpoint URL with the proxy. The proxy then forwards all LLM calls to that endpoint. 2. **Third-party LLM endpoint:** If the algorithm is not updating the LLM weights (e.g., prompt tuning), it can register a third-party LLM endpoint into the proxy. We show a diagram below that illustrates how the proxy fits into the overall data flow. sequenceDiagram autonumber participant Algo as Algorithm participant LLMProxy as LLM Proxy participant Store participant Runner participant Agent Note over Algo,LLMProxy: Algorithm manages LLMProxy as member loop Over the Dataset Algo->>Algo: Launch LLM Inference Engine
(optional) Algo->>LLMProxy: Register Inference Engine
(optional) Algo-->>Store: enqueue_rollout LLMProxy->>Store: Proxy URL added as Resource Store-->>Runner: dequeue_rollout → AttemptedRollout Store-->>Runner: get_latest_resources Runner->>Agent: rollout + resources
(LLM Proxy URL as resource) loop Defined by Agent Agent-->>LLMProxy: LLM calls activate LLMProxy LLMProxy-->>Store: add_span or add_otel_span LLMProxy-->>Agent: LLM responses deactivate LLMProxy Agent-->>Runner: rewards Runner-->>Store: add_span or add_otel_span end Runner-->>Store: update_attempt("finished", status) Store-->>Algo: query_rollouts + spans Algo-->>Algo: Update LLM Weights
(optional) end In this diagram, the store receives spans from both the proxy and the runner. We will see a problem later with parallelism where the proxy and runner are in different machines, and spans need to obtain a special counter from the store to ensure the ordering of spans. ### Trainer[¶](https://microsoft.github.io/agent-lightning/0.2.1/deep-dive/birds-eye-view/#trainer "Permanent link") The [Trainer](https://microsoft.github.io/agent-lightning/0.2.1/reference/trainer/#agentlightning.Trainer " agentlightning.Trainer") is the high-level orchestrator that initializes and connects all major components -- [Algorithm](https://microsoft.github.io/agent-lightning/0.2.1/reference/algorithm/#agentlightning.Algorithm " agentlightning.Algorithm") , [Runner](https://microsoft.github.io/agent-lightning/0.2.1/reference/runner/#agentlightning.Runner " agentlightning.Runner") , [LightningStore](https://microsoft.github.io/agent-lightning/0.2.1/reference/store/#agentlightning.LightningStore " agentlightning.LightningStore") , [Tracer](https://microsoft.github.io/agent-lightning/0.2.1/reference/runner/#agentlightning.Tracer " agentlightning.Tracer") , [Adapter](https://microsoft.github.io/agent-lightning/0.2.1/reference/algorithm/#agentlightning.Adapter " agentlightning.Adapter") , [LLM Proxy](https://microsoft.github.io/agent-lightning/0.2.1/reference/algorithm/#agentlightning.LLMProxy " agentlightning.LLMProxy") , and [Hook](https://microsoft.github.io/agent-lightning/0.2.1/reference/types/#agentlightning.Hook " agentlightning.Hook") . The components can have a lifecycle as long as the trainer. The trainer manages their lifecycles and handles dependency injection, ensuring that every part of the system operates within a consistent and shared environment. Below, we demonstrate how the components relate to each other and their roles. We first clarify the roles and relationships shown in the diagram: 1. **Owns:** components that the trainer constructs and manages directly (e.g., runner, tracer). 2. **Injects:** components passed into others as dependencies. 3. **References:** weak links for coordination without ownership. 4. **Uses:** components that are temporarily interacted with. For example, the [LightningStore](https://microsoft.github.io/agent-lightning/0.2.1/reference/store/#agentlightning.LightningStore " agentlightning.LightningStore") is injected into the [Algorithm](https://microsoft.github.io/agent-lightning/0.2.1/reference/algorithm/#agentlightning.Algorithm " agentlightning.Algorithm") and [Runner](https://microsoft.github.io/agent-lightning/0.2.1/reference/runner/#agentlightning.Runner " agentlightning.Runner") . The [Tracer](https://microsoft.github.io/agent-lightning/0.2.1/reference/runner/#agentlightning.Tracer " agentlightning.Tracer") and [LitAgent](https://microsoft.github.io/agent-lightning/0.2.1/reference/agent/#agentlightning.LitAgent " agentlightning.LitAgent") are injected into the runner. The [Adapter](https://microsoft.github.io/agent-lightning/0.2.1/reference/algorithm/#agentlightning.Adapter " agentlightning.Adapter") and [LLM Proxy](https://microsoft.github.io/agent-lightning/0.2.1/reference/algorithm/#agentlightning.LLMProxy " agentlightning.LLMProxy") are injected into the algorithm. The store is further injected into the tracer, adapter and LLM proxy by the runner and algorithm respectively. flowchart TD %% === Left side: Algorithm domain === subgraph L["Algorithm Side"] Algorithm["Algorithm
(no default)"] Adapter["Adapter
(TracerTraceToTriplet*)"] LLMProxy["LLM Proxy
(no default)"] Algorithm -.injects.-> Adapter Algorithm -.injects.-> LLMProxy end linkStyle 0,1 stroke:#896978,stroke-width:2px; %% === Middle: Core trainer and store === subgraph M["Core"] Trainer["Trainer"] Store["LightningStore
(InMemory* default)"] Trainer --has--> Algorithm Trainer --has--> Store Trainer --has--> Adapter Trainer --has--> LLMProxy end linkStyle 2,3,4,5 stroke:#839791,stroke-width:2px; %% === Right side: Runner side === subgraph R["Runner Side"] Runner["Runner
(LitAgentRunner* default)"] Tracer["Tracer
(AgentOpsTracer*)"] Hooks["Hooks (empty default)"] Agent["Agent
(LitAgent*)"] Runner -.injects.-> Tracer Runner -.injects.-> Store Runner -.injects.-> Agent Runner -.injects.-> Hooks Tracer -.injects.-> Store Hooks -.uses.-> Runner Hooks -.uses.-> Agent Hooks -.uses.-> Tracer end linkStyle 6,7,8,9,10 stroke:#896978,stroke-width:2px; linkStyle 11,12,13 stroke:#7a89c2,stroke-width:2px; %% === Cross-section connections === Trainer --has--> Runner Trainer --has--> Tracer Trainer --has--> Hooks Trainer --uses--> Agent Algorithm -.injects.-> Store LLMProxy -.injects.-> Store Agent -.references.-> Trainer Runner -.references.-> Trainer Algorithm -.references.-> Trainer linkStyle 14,15,16 stroke:#839791,stroke-width:2px; linkStyle 17,20,21,22 stroke:#7a89c2,stroke-width:2px; linkStyle 18,19 stroke:#896978,stroke-width:2px; style L fill:none; style M fill:none; style R fill:none; Putting It All Together: A Reinforcement Learning Example (VERL)[¶](https://microsoft.github.io/agent-lightning/0.2.1/deep-dive/birds-eye-view/#putting-it-all-together-a-reinforcement-learning-example-verl "Permanent link") -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- [](https://microsoft.github.io/agent-lightning/0.2.1/deep-dive/birds-eye-view/) VERL shows how an algorithm consumes the shared infrastructure. For historical reasons, code lives in `agentlightning.algorithm.verl` and `agentlightning.verl`. The latter is legacy and reuses terms like `Trainer` in confusing ways. The former is a thin wrapper that conforms to the new algorithm interface. Future versions will merge the two. Reinforcement learning aims to learn a policy that takes actions in states to maximize expected reward. For agents, the policy is usually a language model. Inputs are prompts (state). Outputs are generated text (action). A numeric score judges quality (reward). The `(state, action, reward)` **triplet** is the basic learning unit. In Agent-lightning, the environment is implicit in the agent’s workflow, which orchestrates one or more LLM calls and often self-judges using rules or additional model calls. During a rollout, the agent emits spans that contain everything needed for RL training, including LLM call traces and numeric judge/reward signals. The "algorithm", on the other hand, have more responsibilities. 1. Providing a language model deployment that is currently learning and improving for the agent to interact with; 2. Preparing the tasks that the agents will perform; 3. Querying the spans generated, extracting triplets, and converting them into a format that the underlying RL library can consume; 4. Updating the language model based on the learning signals. In the VERL integration, the algorithm launches a chat completion endpoint using `vLLM` and wraps training with `FSDP` for distributed optimization. It enqueues tasks from the dataset. After rollouts finish, it queries spans and converts them to triplets with `TracerTraceToTriplet`. VERL’s native training loop then consumes these triplets to update model weights. The workflow can be summarized in the following diagram. sequenceDiagram autonumber participant vLLM as vLLM Chat
Completion Endpoint participant FSDP as FSDP / Megatron
Weights Optimizer participant Algo as Algorithm
Main Controller
(Main Process) participant Adapter as TracerTraceToTriplet participant LLMProxy as LLM Proxy participant Store as LightningStore participant Runner as Runner + Agent Note over Algo,LLMProxy: LLMProxy and Adapter are injected by Trainer as member Note over vLLM,Algo: Algorithm creates and owns vLLM and FSDP loop Over the Dataset in Batches Algo->>vLLM: Create Chat Completion Endpoint activate vLLM vLLM->>LLMProxy: Registered as Backend Endpoint LLMProxy->>Store: Proxy URL added as Resource par Over data samples in the batch Algo-->>Store: enqueue_rollout Store-->>Runner: Dequeue Rollout +
Resources (i.e., URL) loop One Rollout Attempt Runner-->>LLMProxy: LLM calls LLMProxy-->>vLLM: Forwarded LLM calls vLLM-->>LLMProxy: LLM responses LLMProxy-->>Store: add_span / add_otel_span LLMProxy-->>Runner: Forwarded LLM responses Runner-->>Store: add_span / add_otel_span
(by tracer, including rewards) end Runner-->>Store: update_attempt("finished", status) end Algo-->>Store: Poll for completed rollouts + spans Algo->>vLLM: Chat Completion Endpoint Sleeps deactivate vLLM Algo->>Adapter: adapt(spans) Adapter->>FSDP: Triplets (state, action, reward) activate FSDP FSDP-->>Algo: Updated LLM weights deactivate FSDP end **Notes:** 1. There are interactions between different components injected into or owned by algorithms in the diagram, such as the output of the adapter feeding into the FSDP optimizer. This is for simplicity of illustration and slightly different from the actual implementation, where it's the algorithm main controller that orchestrates the data flow between components. 2. **On mapping to VERL.** VERL uses a classic RLHF setup where each action is a single token, the state is the full conversation history up to that token, and reward is given at the end. This is very different from our setup where each action is actually a chunk of text, although they are both called RL! Therefore, after the adapter produces triplets, the algorithm converts each `(state, action, reward)` into a VERL trajectory (`DataProto`) with keys like `input_ids`, `position_ids`, `attention_mask`, and `token_level_scores`. That conversion happens after triplet generation and is not shown in the diagram. Execution Strategies and Parallelism[¶](https://microsoft.github.io/agent-lightning/0.2.1/deep-dive/birds-eye-view/#execution-strategies-and-parallelism "Permanent link") --------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Readers might have observed from the diagram above that there is absolutely no communication between (1) runner and agents and (2) algorithm. The only overlap of them is the [Trainer](https://microsoft.github.io/agent-lightning/0.2.1/reference/trainer/#agentlightning.Trainer " agentlightning.Trainer") and [LightningStore](https://microsoft.github.io/agent-lightning/0.2.1/reference/store/#agentlightning.LightningStore " agentlightning.LightningStore") . This observation is very clear with the diagram within the trainer section. This design allows us to flexibly scale the runner and algorithm independently, which is crucial for large-scale training. Agent-lightning packages two executable bundles: a runner bundle ([Runner](https://microsoft.github.io/agent-lightning/0.2.1/reference/runner/#agentlightning.Runner " agentlightning.Runner") , [Tracer](https://microsoft.github.io/agent-lightning/0.2.1/reference/runner/#agentlightning.Tracer " agentlightning.Tracer") , [Hook](https://microsoft.github.io/agent-lightning/0.2.1/reference/types/#agentlightning.Hook " agentlightning.Hook") , [LitAgent](https://microsoft.github.io/agent-lightning/0.2.1/reference/agent/#agentlightning.LitAgent " agentlightning.LitAgent") ) and an algorithm bundle ([Algorithm](https://microsoft.github.io/agent-lightning/0.2.1/reference/algorithm/#agentlightning.Algorithm " agentlightning.Algorithm") , [Adapter](https://microsoft.github.io/agent-lightning/0.2.1/reference/algorithm/#agentlightning.Adapter " agentlightning.Adapter") , [LLM Proxy](https://microsoft.github.io/agent-lightning/0.2.1/reference/algorithm/#agentlightning.LLMProxy " agentlightning.LLMProxy") ). Both share the [LightningStore](https://microsoft.github.io/agent-lightning/0.2.1/reference/store/#agentlightning.LightningStore " agentlightning.LightningStore") . The trainer initializes and connects the bundles. graph TD subgraph Runner_Side["Runner Bundle"] direction LR R[Runner] --- T[Tracer] --- H[Hooks] --- A1[Agent] end subgraph Algorithm_Side["Algorithm Bundle"] direction LR ALG[Algorithm] --- AD[Adapter] --- LLM[LLM Proxy] end S[(Store)] TR[Trainer] Runner_Side <--> S Algorithm_Side <--> S TR --> Runner_Side TR --> Algorithm_Side linkStyle 0,1,2,3,4 opacity:0; An [execution strategy](https://microsoft.github.io/agent-lightning/0.2.1/reference/trainer/#agentlightning.ExecutionStrategy " agentlightning.ExecutionStrategy") , defined and owned by the trainer, governs how algorithm and runner bundles are placed, connected, scaled, and aborted. It serves four primary purposes. Execution strategies first determine **bundle placement** — whether the two bundles run in the same thread, process, machine, or across separate machines. They also define **store management**, wrapping the store and specifying how data is shared between bundles. In terms of **scalability**, the strategy can replicate the runner bundle across multiple threads, processes, or machines to expand throughput on the runner side. The algorithm side remains single-process due to the complexity of parallelization. Mature frameworks such as _DeepSpeed_ and _Megatron_ already support distributed model training, so scaling of the algorithm bundle is delegated to those implementations. **Abort handling** is another core responsibility. Aborts may be triggered by normal exits, failures in either bundle, or user interrupts. The trainer must include cancellation interfaces for the bundles so that bundles can be cleanly aborted. When the algorithm bundle exits normally, the strategy signals the runner bundle to terminate. If the runner exits first, no signal is sent to the algorithm, as it may still be processing completed rollouts. In cases of failure or user interruption, the strategy signals both bundles to abort; if a bundle fails to respond, the strategy should attempt a forceful termination. Agent-lightning currently provides two execution strategies: **shared-memory** and **client-server**, described in the following sections. ### Shared-memory Strategy[¶](https://microsoft.github.io/agent-lightning/0.2.1/deep-dive/birds-eye-view/#shared-memory-strategy "Permanent link") [`SharedMemoryExecutionStrategy`](https://microsoft.github.io/agent-lightning/0.2.1/reference/trainer/#agentlightning.SharedMemoryExecutionStrategy " agentlightning.SharedMemoryExecutionStrategy") runs algorithm and runner bundles as threads in one process. The strategy wraps the store with [`LightningStoreThreaded`](https://microsoft.github.io/agent-lightning/0.2.1/reference/store/#agentlightning.LightningStoreThreaded " agentlightning.LightningStoreThreaded") , which guards calls with a lock for safe concurrency. This is good for lightweight debugging because components share one Python heap and avoid serialization. It is not suitable for heavy RL training or compute-intensive agents. flowchart TB subgraph MainProcess direction TB subgraph AlgorithmThread [Thread 0] Algorithm[Algorithm bundle] end subgraph RunnerThread1 [Thread 1] Runner1[Runner bundle #1] end subgraph RunnerThread2 [Thread 2] Runner2[Runner bundle #2] end subgraph RunnerThread3 [Thread 3] RunnerN[Runner bundle #N] end LightningStoreFacade[LightningStoreThreaded] BaseStore[Underlying LightningStore] end Algorithm -- async calls --> LightningStoreFacade Runner1 -- async calls --> LightningStoreFacade Runner2 -- async calls --> LightningStoreFacade RunnerN -- async calls --> LightningStoreFacade LightningStoreFacade -->|thread-safe delegates| BaseStore You can configure which role runs on the main thread. If the main thread runs the algorithm, it is able to spawn multiple runner threads. If it runs a runner, `n_runners` must be 1 and the runner lives on the main thread. ### Client-server Strategy[¶](https://microsoft.github.io/agent-lightning/0.2.1/deep-dive/birds-eye-view/#client-server-strategy "Permanent link") [](https://microsoft.github.io/agent-lightning/0.2.1/deep-dive/birds-eye-view/) [`ClientServerExecutionStrategy`](https://microsoft.github.io/agent-lightning/0.2.1/reference/trainer/#agentlightning.ClientServerExecutionStrategy " agentlightning.ClientServerExecutionStrategy") splits concerns across processes. The algorithm bundle starts a [`LightningStoreServer`](https://microsoft.github.io/agent-lightning/0.2.1/reference/store/#agentlightning.LightningStoreServer " agentlightning.LightningStoreServer") (HTTP API) that wraps the underlying store. Runners connect via [`LightningStoreClient`](https://microsoft.github.io/agent-lightning/0.2.1/reference/store/#agentlightning.LightningStoreClient " agentlightning.LightningStoreClient") to call the same interface over REST. The server embeds a client to support algorithm-launched subprocesses (e.g., an LLM proxy worker) that need to talk back to the algorithm’s process through the same API. Currently this design introduces an extra wrapper in the Server side (as shown in the diagram), which helps debugging and improves fault tolerance. We might revisit this design in the future and enforce the client to be the only way to communicate with the store. flowchart TD subgraph Algorithm Process Group subgraph StoreServer[LightningStoreServer] StoreHttpClient[HTTP Client] StoreHttpServer[HTTP Server] StoreWrapper[LightningStore Wrapper] StoreHttpClient -- HTTP --> StoreHttpServer end subgraph Algorithm Bundle Algorithm[Algorithm Main Process] subgraph Another subprocess LLMProxy[LLM Proxy] end end LLMProxy -- async calls --> StoreHttpClient Algorithm -- async calls --> StoreWrapper end subgraph RunnerSide ["Runner Side"] subgraph Runner Process 1 Runner1[Runner bundle #1] Runner1 -- async calls --> LightningStoreClient1 LightningStoreClient1[LightningStoreClient] end subgraph Runner Process 2 Runner2[Runner bundle #2] Runner2 -- async calls --> LightningStoreClient2 LightningStoreClient2[LightningStoreClient] end subgraph Runner Process N RunnerN[Runner bundle #N] RunnerN -- async calls --> LightningStoreClientN LightningStoreClientN[LightningStoreClient] end end LocalStore[Underlying LightningStore] StoreHttpServer -->|delegates| StoreWrapper StoreWrapper -->|delegates| LocalStore LightningStoreClient1 -- HTTP --> StoreHttpServer LightningStoreClient2 -- HTTP --> StoreHttpServer LightningStoreClientN -- HTTP --> StoreHttpServer style RunnerSide fill:none; Online/Continuous Learning[¶](https://microsoft.github.io/agent-lightning/0.2.1/deep-dive/birds-eye-view/#onlinecontinuous-learning "Permanent link") ------------------------------------------------------------------------------------------------------------------------------------------------------ Continuous learning keeps the algorithm loop running while runners report tasks and spans opportunistically. Key differences from batch mode: 1. The algorithm does not enqueue rollouts from a fixed dataset. Runners report tasks/rollouts and spans spontaneously. 2. The algorithm can wait for rollouts with a expected set of rollout IDs, but more often polls for new rollouts and spans or waits for a count to arrive. 3. The [`Runner`](https://microsoft.github.io/agent-lightning/0.2.1/reference/runner/#agentlightning.Runner " agentlightning.Runner") processes one rollout at a time via [`step(task)`](https://microsoft.github.io/agent-lightning/0.2.1/reference/runner/#agentlightning.Runner.step " step(input, *, resources=None, mode=None, event=None) async ") instead of exhausting a task queue. It notifies the store when starting a rollout so the store records it. 4. A user or higher-level loop controls which resources the next step uses and when to retry. [Spans](https://microsoft.github.io/agent-lightning/0.2.1/reference/types/#agentlightning.Span " agentlightning.Span") , [Adapter](https://microsoft.github.io/agent-lightning/0.2.1/reference/algorithm/#agentlightning.Adapter " agentlightning.Adapter") implementations, and the [LLM Proxy](https://microsoft.github.io/agent-lightning/0.2.1/reference/algorithm/#agentlightning.LLMProxy " agentlightning.LLMProxy") work the same way. sequenceDiagram autonumber actor User participant Runner participant Agent participant Store as LightningStore participant Algorithm Note over Algorithm: Algorithm is long-running and loops continuously loop Continuous Learning Loop activate User opt Decide what to do next User-->>Store: get_resources_by_id Store-->>User: Resources User-->>User: Prepare input for next step end User->>Runner: step(input, resources) activate Runner Runner-->>Store: Notify: start_rollout(input) Runner->>Agent: rollout(input, resources) Agent-->>Runner: add_span / reward spans Runner-->>Store: add_span or add_otel_span Runner-->>Store: update_attempt(status="finished") deactivate Runner deactivate User Algorithm->>Store: poll for new rollouts and spans opt If there is enough new data Store-->>Algorithm: new spans Algorithm->>Algorithm: adapt spans → learning signal Algorithm->>Store: update_resources end end Back to top --- # Bird's Eye View - Agent-lightning [Skip to content](https://microsoft.github.io/agent-lightning/0.3.0/deep-dive/birds-eye-view/#the-birds-eye-view-of-agent-lightning) ⚠️ You are viewing development documentation. For stable documentation, please visit [https://microsoft.github.io/agent-lightning/stable/](https://microsoft.github.io/agent-lightning/stable/) The Bird's Eye View of Agent-lightning[¶](https://microsoft.github.io/agent-lightning/0.3.0/deep-dive/birds-eye-view/#the-birds-eye-view-of-agent-lightning "Permanent link") ============================================================================================================================================================================== High Volume of Information Ahead This article provides an in-depth exploration of the Agent-lightning architecture. It is not intended as a beginner’s guide or usage tutorial. This article summarizes how Agent-lightning (as of v0.2) wires the [Algorithm](https://microsoft.github.io/agent-lightning/0.3.0/reference/algorithm/#agentlightning.Algorithm " agentlightning.Algorithm") , [Runner](https://microsoft.github.io/agent-lightning/0.3.0/reference/runner/#agentlightning.Runner " agentlightning.Runner") , and [LightningStore](https://microsoft.github.io/agent-lightning/0.3.0/reference/store/#agentlightning.LightningStore " agentlightning.LightningStore") loop together and shows where auxiliary components (the [Tracer](https://microsoft.github.io/agent-lightning/0.3.0/reference/runner/#agentlightning.Tracer " agentlightning.Tracer") , [Adapter](https://microsoft.github.io/agent-lightning/0.3.0/reference/algorithm/#agentlightning.Adapter " agentlightning.Adapter") , and [LLM Proxy](https://microsoft.github.io/agent-lightning/0.3.0/reference/algorithm/#agentlightning.LLMProxy " agentlightning.LLMProxy") ) plug into the loop. Each section provides a diagram for a different perspective of the system. Algorithm ↔ Runner ↔ Store data flow[¶](https://microsoft.github.io/agent-lightning/0.3.0/deep-dive/birds-eye-view/#algorithm-runner-store-data-flow "Permanent link") ----------------------------------------------------------------------------------------------------------------------------------------------------------------------- At its heart, Agent-lightning is built on three main components that work in a coordinated loop: * **[Algorithm](https://microsoft.github.io/agent-lightning/0.3.0/reference/algorithm/#agentlightning.Algorithm " agentlightning.Algorithm") :** The "brain" of the system. It decides what tasks to run, learns from the results, and updates resources (like AI models or prompts). * **[Runner](https://microsoft.github.io/agent-lightning/0.3.0/reference/runner/#agentlightning.Runner " agentlightning.Runner") :** The "worker" of the system. It executes tasks assigned by the algorithm, runs the agent, and records the results. * **[LightningStore](https://microsoft.github.io/agent-lightning/0.3.0/reference/store/#agentlightning.LightningStore " agentlightning.LightningStore") :** The central "database" and message queue. It acts as the single source of truth, storing tasks, results, and resources, and enabling communication between the Algorithm and Runner. The typical data flow in a training loop is as follows: The **[Algorithm](https://microsoft.github.io/agent-lightning/0.3.0/reference/algorithm/#agentlightning.Algorithm " agentlightning.Algorithm") ** enqueues tasks (called **[Rollouts](https://microsoft.github.io/agent-lightning/0.3.0/reference/types/#agentlightning.Rollout " agentlightning.Rollout") **) into the **[LightningStore](https://microsoft.github.io/agent-lightning/0.3.0/reference/store/#agentlightning.LightningStore " agentlightning.LightningStore") **. A **[Runner](https://microsoft.github.io/agent-lightning/0.3.0/reference/runner/#agentlightning.Runner " agentlightning.Runner") ** then dequeues a task, executes it, and streams the results (called **[Spans](https://microsoft.github.io/agent-lightning/0.3.0/reference/types/#agentlightning.Span " agentlightning.Span") **) back to the store. Once the task is complete, the algorithm can query the new data from the store to learn and update its resources. The diagram below shows this fundamental interaction in a simple, non-parallel setup. sequenceDiagram autonumber participant Algo as Algorithm participant Store as LightningStore participant Runner participant Agent loop Over the dataset Algo-->>Store: add_resources + enqueue_rollout Store-->>Runner: dequeue_rollout → AttemptedRollout Store-->>Runner: get_latest_resources Runner-->>Store: update_attempt("running", worker_id) Runner->>Agent: rollout + resources Agent->>Runner: reward / spans Runner-->>Store: add_span or add_otel_span Runner-->>Store: update_attempt("finished", status) Store-->>Algo: query_rollouts + spans Algo-->>Algo: Update resources (optional) end Solid lines represent direct calls, while dashed lines are asynchronous or long-running operations. ### Key Terminology[¶](https://microsoft.github.io/agent-lightning/0.3.0/deep-dive/birds-eye-view/#key-terminology "Permanent link") We define the following terms, which may be helpful for understanding the diagram above. * **[Resources](https://microsoft.github.io/agent-lightning/0.3.0/reference/types/#agentlightning.Resource " agentlightning.Resource") :** A collection of assets to be tuned or trained. Agents perform rollouts against resources and collect span data. Algorithms use those data to update the resources. In RL training, the resources are a tunable model. In prompt tuning, the resources are prompt templates. * **[Rollout](https://microsoft.github.io/agent-lightning/0.3.0/reference/types/#agentlightning.Rollout " agentlightning.Rollout") :** A unit of work that an agent performs against a resource. A rollout (noun) can be incomplete, in which case it is also known as a **task**, **sample**, or **job** (these terms are used interchangeably). The agent executes its own defined workflow against the rollout — the process is also called "to rollout" (verb). After execution, the rollout (noun) is considered _complete_. * **[Attempt](https://microsoft.github.io/agent-lightning/0.3.0/reference/types/#agentlightning.Attempt " agentlightning.Attempt") :** A single execution of a rollout. One rollout can have multiple attempts in case of failures or timeouts. * **[Span](https://microsoft.github.io/agent-lightning/0.3.0/reference/types/#agentlightning.Span " agentlightning.Span") :** During the rollout, the agent can generate multiple spans (also known as "traces" or "events"). The recorded spans are collected in the store, which is crucial for understanding agent behavior and optimizing agents. * **[Reward](https://microsoft.github.io/agent-lightning/0.3.0/reference/agent/#agentlightning.emit_reward " agentlightning.emit_reward(reward, *, primary_key=None, attributes=None, propagate=True)") :** A special span that is defined as a number judging the quality of the rollout during some period of the rollout. * **[Dataset](https://microsoft.github.io/agent-lightning/0.3.0/reference/types/#agentlightning.Dataset " agentlightning.Dataset") :** A collection of incomplete rollouts (i.e., tasks) for the agent to process. The dual datasets (train, val) serve as the initial input for the algorithm to enqueue the first batch of rollouts. ### Store[¶](https://microsoft.github.io/agent-lightning/0.3.0/deep-dive/birds-eye-view/#store "Permanent link") As discussed previously, the [LightningStore](https://microsoft.github.io/agent-lightning/0.3.0/reference/store/#agentlightning.LightningStore " agentlightning.LightningStore") is the central hub for all data in Agent-lightning. The store exposes a set of APIs for algorithms and runners to interact with the data; the most important ones are: `[](https://microsoft.github.io/agent-lightning/0.3.0/deep-dive/birds-eye-view/#__codelineno-0-1) from agentlightning.types import AttemptedRollout, ResourcesUpdate, Span, TaskInput [](https://microsoft.github.io/agent-lightning/0.3.0/deep-dive/birds-eye-view/#__codelineno-0-2) [](https://microsoft.github.io/agent-lightning/0.3.0/deep-dive/birds-eye-view/#__codelineno-0-3) class LightningStore: [](https://microsoft.github.io/agent-lightning/0.3.0/deep-dive/birds-eye-view/#__codelineno-0-4) [](https://microsoft.github.io/agent-lightning/0.3.0/deep-dive/birds-eye-view/#__codelineno-0-5) async def enqueue_rollout(self, input: TaskInput, ...) -> Rollout: ... [](https://microsoft.github.io/agent-lightning/0.3.0/deep-dive/birds-eye-view/#__codelineno-0-6) [](https://microsoft.github.io/agent-lightning/0.3.0/deep-dive/birds-eye-view/#__codelineno-0-7) async def dequeue_rollout(self) -> AttemptedRollout | None: ... [](https://microsoft.github.io/agent-lightning/0.3.0/deep-dive/birds-eye-view/#__codelineno-0-8) [](https://microsoft.github.io/agent-lightning/0.3.0/deep-dive/birds-eye-view/#__codelineno-0-9) async def add_span(self, span: Span) -> Span: ... [](https://microsoft.github.io/agent-lightning/0.3.0/deep-dive/birds-eye-view/#__codelineno-0-10) [](https://microsoft.github.io/agent-lightning/0.3.0/deep-dive/birds-eye-view/#__codelineno-0-11) async def get_latest_resources(self) -> Optional[ResourcesUpdate]: ... [](https://microsoft.github.io/agent-lightning/0.3.0/deep-dive/birds-eye-view/#__codelineno-0-12) [](https://microsoft.github.io/agent-lightning/0.3.0/deep-dive/birds-eye-view/#__codelineno-0-13) async def wait_for_rollouts(self, rollout_ids: List[str], ...): ... [](https://microsoft.github.io/agent-lightning/0.3.0/deep-dive/birds-eye-view/#__codelineno-0-14) [](https://microsoft.github.io/agent-lightning/0.3.0/deep-dive/birds-eye-view/#__codelineno-0-15) async def query_spans(self, rollout_id: str, ...): ... [](https://microsoft.github.io/agent-lightning/0.3.0/deep-dive/birds-eye-view/#__codelineno-0-16) [](https://microsoft.github.io/agent-lightning/0.3.0/deep-dive/birds-eye-view/#__codelineno-0-17) async def update_attempt(self, rollout_id: str, attempt_id: str, status: str, ...): ... [](https://microsoft.github.io/agent-lightning/0.3.0/deep-dive/birds-eye-view/#__codelineno-0-18) [](https://microsoft.github.io/agent-lightning/0.3.0/deep-dive/birds-eye-view/#__codelineno-0-19) ...` These interfaces operate on [`AttemptedRollout`](https://microsoft.github.io/agent-lightning/0.3.0/reference/types/#agentlightning.AttemptedRollout " agentlightning.AttemptedRollout") , [`ResourcesUpdate`](https://microsoft.github.io/agent-lightning/0.3.0/reference/types/#agentlightning.ResourcesUpdate " agentlightning.ResourcesUpdate") , [`Span`](https://microsoft.github.io/agent-lightning/0.3.0/reference/types/#agentlightning.Span " agentlightning.Span") , and [`TaskInput`](https://microsoft.github.io/agent-lightning/0.3.0/reference/internal/#agentlightning.TaskInput " agentlightning.TaskInput = Any module-attribute ") instances from `agentlightning.types`. As the APIs show, the store essentially provides a queue for rollouts and storage for resources, spans, and attempts. Developers should implement the store carefully to ensure data integrity and consistency, especially when multiple runners work in parallel across multiple attempts. The store is designed to be extensible. Users can implement their own store by inheriting from [`LightningStore`](https://microsoft.github.io/agent-lightning/0.3.0/reference/store/#agentlightning.LightningStore " agentlightning.LightningStore") and overriding methods. Agent-lightning provides a few reference implementations, such as [`InMemoryLightningStore`](https://microsoft.github.io/agent-lightning/0.3.0/reference/store/#agentlightning.InMemoryLightningStore " agentlightning.InMemoryLightningStore") (default) and `SqliteLightningStore` (under construction). When parallelized, the store may need special wrappers to ensure thread/process safety or delegate computation to a store in another process or machine. Supporting Components in the Loop[¶](https://microsoft.github.io/agent-lightning/0.3.0/deep-dive/birds-eye-view/#supporting-components-in-the-loop "Permanent link") --------------------------------------------------------------------------------------------------------------------------------------------------------------------- While the core loop is simple, Agent-lightning provides several components to make development easier and more powerful. ### Tracer[¶](https://microsoft.github.io/agent-lightning/0.3.0/deep-dive/birds-eye-view/#tracer "Permanent link") The [`Tracer`](https://microsoft.github.io/agent-lightning/0.3.0/reference/runner/#agentlightning.Tracer " agentlightning.Tracer") is a component within the [`Runner`](https://microsoft.github.io/agent-lightning/0.3.0/reference/runner/#agentlightning.Runner " agentlightning.Runner") that records detailed spans (events) during an agent's execution and sends them to the [`LightningStore`](https://microsoft.github.io/agent-lightning/0.3.0/reference/store/#agentlightning.LightningStore " agentlightning.LightningStore") . Instead of requiring the agent to manually log every span, the tracer automatically instruments key methods (e.g., LLM calls) and captures their inputs, outputs, and metadata. This provides a detailed log of the agent's behavior with minimal effort. sequenceDiagram autonumber participant Store participant Runner participant Tracer participant Agent Note over Runner,Tracer: Runner manages tracer as member Tracer->>Agent: Apply instrumentation loop Until no more rollouts Store-->>Runner: dequeue_rollout → AttemptedRollout Store-->>Runner: get_latest_resources Runner->>Agent: training_rollout / validation_rollout loop For each finished span Agent-->>Tracer: openai.chat.completion invoked
agent.execute invoked
... Agent->>Tracer: emit intermediate reward Tracer-->>Store: add_otel_span(rollout_id, attempt_id, span) end Agent->>Runner: final reward + extra spans (if any) Runner-->>Store: add_span(rollout_id, attempt_id, span) Runner-->>Store: update_attempt(status) end Tracer->>Agent: Unapply instrumentation The above diagram shows the overall data flow between store, tracer and agent. In realistic, it's a bit more complicated than that. Spans are not emitted actively by the agent; they are intercepted by the tracer by hooking and instrumenting key methods used in the agents. The tracer uses a callback (called exporter) to monitor events and log to the store. Before a rollout starts, the runner enters a [`trace_context`](https://microsoft.github.io/agent-lightning/0.3.0/reference/runner/#agentlightning.Tracer.trace_context " trace_context(name=None, *, store=None, rollout_id=None, attempt_id=None)") before invoking the agent, wiring store identifiers into the tracer. Each span completion streams back to the store through `LightningSpanProcessor`, so the agent’s instrumentation lands in [`add_otel_span`](https://microsoft.github.io/agent-lightning/0.3.0/reference/store/#agentlightning.LightningStore.add_otel_span " add_otel_span(rollout_id, attempt_id, readable_span, sequence_id=None) async ") . If the agent’s rollout method returns a numeric reward, the runner emits one more OpenTelemetry span before finalizing the attempt. ### Hooks[¶](https://microsoft.github.io/agent-lightning/0.3.0/deep-dive/birds-eye-view/#hooks "Permanent link") [`Hook`](https://microsoft.github.io/agent-lightning/0.3.0/reference/types/#agentlightning.Hook " agentlightning.Hook") implementations are user-defined callback functions that allow you to augment a [`Runner`](https://microsoft.github.io/agent-lightning/0.3.0/reference/runner/#agentlightning.Runner " agentlightning.Runner") 's behavior at specific points in its lifecycle. You can use hooks to add custom logging, set up resources before a rollout begins, or tear them down after it ends. Hooks can be triggered at four key moments: `on_rollout_start`, `on_trace_start`, `on_trace_end`, and `on_rollout_end`. Users should pay special attention to the difference between `on_trace_end` and `on_rollout_end`. The former is called right before the tracer exits the trace context, while the latter is called after the runner processes the final leftover rewards and spans, and finalizes the attempt in the store. sequenceDiagram autonumber participant Store participant Hooks participant Runner participant Tracer participant Agent Note over Runner,Hooks: Runner manages hooks as member loop Until no more rollouts Store-->>Runner: dequeue_rollout → AttemptedRollout Store-->>Runner: get_latest_resources Runner->>Hooks: on_rollout_start(agent, runner, rollout) Runner->>Agent: training_rollout / validation_rollout Tracer->>Agent: enter_trace_context activate Tracer Runner->>Hooks: on_trace_start(agent, runner, tracer, rollout) Note over Runner,Agent: Agent rollout omitted Runner->>Hooks: on_trace_end(agent, runner, tracer, rollout) Tracer->>Agent: exit_trace_context deactivate Tracer Agent->>Runner: final reward + extra spans (if any) Runner-->>Store: add_span(rollout_id, attempt_id, span) Runner->>Hooks: on_rollout_end(agent, runner, rollout, status) end ### Adapter[¶](https://microsoft.github.io/agent-lightning/0.3.0/deep-dive/birds-eye-view/#adapter "Permanent link") The [`Adapter`](https://microsoft.github.io/agent-lightning/0.3.0/reference/algorithm/#agentlightning.Adapter " agentlightning.Adapter") is a component used by the [`Algorithm`](https://microsoft.github.io/agent-lightning/0.3.0/reference/algorithm/#agentlightning.Algorithm " agentlightning.Algorithm") to transform raw data from the [`LightningStore`](https://microsoft.github.io/agent-lightning/0.3.0/reference/store/#agentlightning.LightningStore " agentlightning.LightningStore") into a format suitable for learning. Runners stream raw spans into the store during execution. Later, the algorithm queries these spans and uses an adapter to convert them into structured data, like training examples for a reinforcement learning model. For instance, the [`TracerTraceToTriplet`](https://microsoft.github.io/agent-lightning/0.3.0/reference/algorithm/#agentlightning.TracerTraceToTriplet " agentlightning.TracerTraceToTriplet") processes OpenTelemetry spans to create `(prompt, response, reward)` triplets, which are the fundamental data structure for many RL fine-tuning algorithms. flowchart LR Runner -- (1) add_otel_span --> Store Store -- (2) query_spans --> Algorithm Algorithm -- (3) spans --> Adapter Adapter -- (4) transformed data --> Algorithm ### LLM Proxy[¶](https://microsoft.github.io/agent-lightning/0.3.0/deep-dive/birds-eye-view/#llm-proxy "Permanent link") The [`LLMProxy`](https://microsoft.github.io/agent-lightning/0.3.0/reference/algorithm/#agentlightning.LLMProxy " agentlightning.LLMProxy") is an optional bridge component that sits between an agent and the algorithms' resources. It acts as a centralized endpoint for all LLM calls. Usually the proxy URL is added to the store as a special resource, so that the [`Runner`](https://microsoft.github.io/agent-lightning/0.3.0/reference/runner/#agentlightning.Runner " agentlightning.Runner") can fetch it along with other resources when dequeuing a rollout. During rollouts, the runner invokes the proxy's HTTP endpoint instead of calling a model backend directly. This design offers several benefits: 1. **Instrumentation:** It automatically captures detailed traces of LLM interactions (prompts, responses, metadata) and sends them to the store, complementing the tracer, especially when the agent's code is hard to instrument directly. 2. **Backend Abstraction:** It provides a unified interface for various LLM backends (OpenAI, Anthropic, local models) and can add features like retry logic, rate limiting, and caching. 3. **Resource Management:** The algorithm can dynamically update which LLM the agent uses (e.g., swapping to a newly fine-tuned model) by simply swapping the backend model the proxy is using, without interrupting the agent's code. The benefits above seem to be all discussed within the context of model fine-tuning. As a matter of fact, the proxy can be useful for prompt tuning as well. The algorithm can register one of the following two types of endpoints into the proxy: 1. **Endpoint served by the algorithm:** If the algorithm is internally updating the LLM weights (e.g., RL), it can launch an LLM inference engine (i.e., a model server) and register the endpoint URL with the proxy. The proxy then forwards all LLM calls to that endpoint. 2. **Third-party LLM endpoint:** If the algorithm is not updating the LLM weights (e.g., prompt tuning), it can register a third-party LLM endpoint into the proxy. We show a diagram below that illustrates how the proxy fits into the overall data flow. sequenceDiagram autonumber participant Algo as Algorithm participant LLMProxy as LLM Proxy participant Store participant Runner participant Agent Note over Algo,LLMProxy: Algorithm manages LLMProxy as member loop Over the Dataset Algo->>Algo: Launch LLM Inference Engine
(optional) Algo->>LLMProxy: Register Inference Engine
(optional) Algo-->>Store: enqueue_rollout LLMProxy->>Store: Proxy URL added as Resource Store-->>Runner: dequeue_rollout → AttemptedRollout Store-->>Runner: get_latest_resources Runner->>Agent: rollout + resources
(LLM Proxy URL as resource) loop Defined by Agent Agent-->>LLMProxy: LLM calls activate LLMProxy LLMProxy-->>Store: add_span or add_otel_span LLMProxy-->>Agent: LLM responses deactivate LLMProxy Agent-->>Runner: rewards Runner-->>Store: add_span or add_otel_span end Runner-->>Store: update_attempt("finished", status) Store-->>Algo: query_rollouts + spans Algo-->>Algo: Update LLM Weights
(optional) end In this diagram, the store receives spans from both the proxy and the runner. We will see a problem later with parallelism where the proxy and runner are in different machines, and spans need to obtain a special counter from the store to ensure the ordering of spans. ### Trainer[¶](https://microsoft.github.io/agent-lightning/0.3.0/deep-dive/birds-eye-view/#trainer "Permanent link") The [Trainer](https://microsoft.github.io/agent-lightning/0.3.0/reference/trainer/#agentlightning.Trainer " agentlightning.Trainer") is the high-level orchestrator that initializes and connects all major components -- [Algorithm](https://microsoft.github.io/agent-lightning/0.3.0/reference/algorithm/#agentlightning.Algorithm " agentlightning.Algorithm") , [Runner](https://microsoft.github.io/agent-lightning/0.3.0/reference/runner/#agentlightning.Runner " agentlightning.Runner") , [LightningStore](https://microsoft.github.io/agent-lightning/0.3.0/reference/store/#agentlightning.LightningStore " agentlightning.LightningStore") , [Tracer](https://microsoft.github.io/agent-lightning/0.3.0/reference/runner/#agentlightning.Tracer " agentlightning.Tracer") , [Adapter](https://microsoft.github.io/agent-lightning/0.3.0/reference/algorithm/#agentlightning.Adapter " agentlightning.Adapter") , [LLM Proxy](https://microsoft.github.io/agent-lightning/0.3.0/reference/algorithm/#agentlightning.LLMProxy " agentlightning.LLMProxy") , and [Hook](https://microsoft.github.io/agent-lightning/0.3.0/reference/types/#agentlightning.Hook " agentlightning.Hook") . The components can have a lifecycle as long as the trainer. The trainer manages their lifecycles and handles dependency injection, ensuring that every part of the system operates within a consistent and shared environment. Below, we demonstrate how the components relate to each other and their roles. We first clarify the roles and relationships shown in the diagram: 1. **Owns:** components that the trainer constructs and manages directly (e.g., runner, tracer). 2. **Injects:** components passed into others as dependencies. 3. **References:** weak links for coordination without ownership. 4. **Uses:** components that are temporarily interacted with. For example, the [LightningStore](https://microsoft.github.io/agent-lightning/0.3.0/reference/store/#agentlightning.LightningStore " agentlightning.LightningStore") is injected into the [Algorithm](https://microsoft.github.io/agent-lightning/0.3.0/reference/algorithm/#agentlightning.Algorithm " agentlightning.Algorithm") and [Runner](https://microsoft.github.io/agent-lightning/0.3.0/reference/runner/#agentlightning.Runner " agentlightning.Runner") . The [Tracer](https://microsoft.github.io/agent-lightning/0.3.0/reference/runner/#agentlightning.Tracer " agentlightning.Tracer") and [LitAgent](https://microsoft.github.io/agent-lightning/0.3.0/reference/agent/#agentlightning.LitAgent " agentlightning.LitAgent") are injected into the runner. The [Adapter](https://microsoft.github.io/agent-lightning/0.3.0/reference/algorithm/#agentlightning.Adapter " agentlightning.Adapter") and [LLM Proxy](https://microsoft.github.io/agent-lightning/0.3.0/reference/algorithm/#agentlightning.LLMProxy " agentlightning.LLMProxy") are injected into the algorithm. The store is further injected into the tracer, adapter and LLM proxy by the runner and algorithm respectively. flowchart TD %% === Left side: Algorithm domain === subgraph L["Algorithm Side"] Algorithm["Algorithm
(no default)"] Adapter["Adapter
(TracerTraceToTriplet*)"] LLMProxy["LLM Proxy
(no default)"] Algorithm -.injects.-> Adapter Algorithm -.injects.-> LLMProxy end linkStyle 0,1 stroke:#896978,stroke-width:2px; %% === Middle: Core trainer and store === subgraph M["Core"] Trainer["Trainer"] Store["LightningStore
(InMemory* default)"] Trainer --has--> Algorithm Trainer --has--> Store Trainer --has--> Adapter Trainer --has--> LLMProxy end linkStyle 2,3,4,5 stroke:#839791,stroke-width:2px; %% === Right side: Runner side === subgraph R["Runner Side"] Runner["Runner
(LitAgentRunner* default)"] Tracer["Tracer
(AgentOpsTracer*)"] Hooks["Hooks (empty default)"] Agent["Agent
(LitAgent*)"] Runner -.injects.-> Tracer Runner -.injects.-> Store Runner -.injects.-> Agent Runner -.injects.-> Hooks Tracer -.injects.-> Store Hooks -.uses.-> Runner Hooks -.uses.-> Agent Hooks -.uses.-> Tracer end linkStyle 6,7,8,9,10 stroke:#896978,stroke-width:2px; linkStyle 11,12,13 stroke:#7a89c2,stroke-width:2px; %% === Cross-section connections === Trainer --has--> Runner Trainer --has--> Tracer Trainer --has--> Hooks Trainer --uses--> Agent Algorithm -.injects.-> Store LLMProxy -.injects.-> Store Agent -.references.-> Trainer Runner -.references.-> Trainer Algorithm -.references.-> Trainer linkStyle 14,15,16 stroke:#839791,stroke-width:2px; linkStyle 17,20,21,22 stroke:#7a89c2,stroke-width:2px; linkStyle 18,19 stroke:#896978,stroke-width:2px; style L fill:none; style M fill:none; style R fill:none; Putting It All Together: A Reinforcement Learning Example (VERL)[¶](https://microsoft.github.io/agent-lightning/0.3.0/deep-dive/birds-eye-view/#putting-it-all-together-a-reinforcement-learning-example-verl "Permanent link") -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- [](https://microsoft.github.io/agent-lightning/0.3.0/deep-dive/birds-eye-view/) VERL shows how an algorithm consumes the shared infrastructure. For historical reasons, code lives in `agentlightning.algorithm.verl` and `agentlightning.verl`. The latter is legacy and reuses terms like `Trainer` in confusing ways. The former is a thin wrapper that conforms to the new algorithm interface. Future versions will merge the two. Reinforcement learning aims to learn a policy that takes actions in states to maximize expected reward. For agents, the policy is usually a language model. Inputs are prompts (state). Outputs are generated text (action). A numeric score judges quality (reward). The `(state, action, reward)` **triplet** is the basic learning unit. In Agent-lightning, the environment is implicit in the agent’s workflow, which orchestrates one or more LLM calls and often self-judges using rules or additional model calls. During a rollout, the agent emits spans that contain everything needed for RL training, including LLM call traces and numeric judge/reward signals. The "algorithm", on the other hand, have more responsibilities. 1. Providing a language model deployment that is currently learning and improving for the agent to interact with; 2. Preparing the tasks that the agents will perform; 3. Querying the spans generated, extracting triplets, and converting them into a format that the underlying RL library can consume; 4. Updating the language model based on the learning signals. In the VERL integration, the algorithm launches a chat completion endpoint using `vLLM` and wraps training with `FSDP` for distributed optimization. It enqueues tasks from the dataset. After rollouts finish, it queries spans and converts them to triplets with `TracerTraceToTriplet`. VERL’s native training loop then consumes these triplets to update model weights. The workflow can be summarized in the following diagram. sequenceDiagram autonumber participant vLLM as vLLM Chat
Completion Endpoint participant FSDP as FSDP / Megatron
Weights Optimizer participant Algo as Algorithm
Main Controller
(Main Process) participant Adapter as TracerTraceToTriplet participant LLMProxy as LLM Proxy participant Store as LightningStore participant Runner as Runner + Agent Note over Algo,LLMProxy: LLMProxy and Adapter are injected by Trainer as member Note over vLLM,Algo: Algorithm creates and owns vLLM and FSDP loop Over the Dataset in Batches Algo->>vLLM: Create Chat Completion Endpoint activate vLLM vLLM->>LLMProxy: Registered as Backend Endpoint LLMProxy->>Store: Proxy URL added as Resource par Over data samples in the batch Algo-->>Store: enqueue_rollout Store-->>Runner: Dequeue Rollout +
Resources (i.e., URL) loop One Rollout Attempt Runner-->>LLMProxy: LLM calls LLMProxy-->>vLLM: Forwarded LLM calls vLLM-->>LLMProxy: LLM responses LLMProxy-->>Store: add_span / add_otel_span LLMProxy-->>Runner: Forwarded LLM responses Runner-->>Store: add_span / add_otel_span
(by tracer, including rewards) end Runner-->>Store: update_attempt("finished", status) end Algo-->>Store: Poll for completed rollouts + spans Algo->>vLLM: Chat Completion Endpoint Sleeps deactivate vLLM Algo->>Adapter: adapt(spans) Adapter->>FSDP: Triplets (state, action, reward) activate FSDP FSDP-->>Algo: Updated LLM weights deactivate FSDP end **Notes:** 1. There are interactions between different components injected into or owned by algorithms in the diagram, such as the output of the adapter feeding into the FSDP optimizer. This is for simplicity of illustration and slightly different from the actual implementation, where it's the algorithm main controller that orchestrates the data flow between components. 2. **On mapping to VERL.** VERL uses a classic RLHF setup where each action is a single token, the state is the full conversation history up to that token, and reward is given at the end. This is very different from our setup where each action is actually a chunk of text, although they are both called RL! Therefore, after the adapter produces triplets, the algorithm converts each `(state, action, reward)` into a VERL trajectory (`DataProto`) with keys like `input_ids`, `position_ids`, `attention_mask`, and `token_level_scores`. That conversion happens after triplet generation and is not shown in the diagram. Execution Strategies and Parallelism[¶](https://microsoft.github.io/agent-lightning/0.3.0/deep-dive/birds-eye-view/#execution-strategies-and-parallelism "Permanent link") --------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Readers might have observed from the diagram above that there is absolutely no communication between (1) runner and agents and (2) algorithm. The only overlap of them is the [Trainer](https://microsoft.github.io/agent-lightning/0.3.0/reference/trainer/#agentlightning.Trainer " agentlightning.Trainer") and [LightningStore](https://microsoft.github.io/agent-lightning/0.3.0/reference/store/#agentlightning.LightningStore " agentlightning.LightningStore") . This observation is very clear with the diagram within the trainer section. This design allows us to flexibly scale the runner and algorithm independently, which is crucial for large-scale training. Agent-lightning packages two executable bundles: a runner bundle ([Runner](https://microsoft.github.io/agent-lightning/0.3.0/reference/runner/#agentlightning.Runner " agentlightning.Runner") , [Tracer](https://microsoft.github.io/agent-lightning/0.3.0/reference/runner/#agentlightning.Tracer " agentlightning.Tracer") , [Hook](https://microsoft.github.io/agent-lightning/0.3.0/reference/types/#agentlightning.Hook " agentlightning.Hook") , [LitAgent](https://microsoft.github.io/agent-lightning/0.3.0/reference/agent/#agentlightning.LitAgent " agentlightning.LitAgent") ) and an algorithm bundle ([Algorithm](https://microsoft.github.io/agent-lightning/0.3.0/reference/algorithm/#agentlightning.Algorithm " agentlightning.Algorithm") , [Adapter](https://microsoft.github.io/agent-lightning/0.3.0/reference/algorithm/#agentlightning.Adapter " agentlightning.Adapter") , [LLM Proxy](https://microsoft.github.io/agent-lightning/0.3.0/reference/algorithm/#agentlightning.LLMProxy " agentlightning.LLMProxy") ). Both share the [LightningStore](https://microsoft.github.io/agent-lightning/0.3.0/reference/store/#agentlightning.LightningStore " agentlightning.LightningStore") . The trainer initializes and connects the bundles. graph TD subgraph Runner_Side["Runner Bundle"] direction LR R[Runner] --- T[Tracer] --- H[Hooks] --- A1[Agent] end subgraph Algorithm_Side["Algorithm Bundle"] direction LR ALG[Algorithm] --- AD[Adapter] --- LLM[LLM Proxy] end S[(Store)] TR[Trainer] Runner_Side <--> S Algorithm_Side <--> S TR --> Runner_Side TR --> Algorithm_Side linkStyle 0,1,2,3,4 opacity:0; An [execution strategy](https://microsoft.github.io/agent-lightning/0.3.0/reference/trainer/#agentlightning.ExecutionStrategy " agentlightning.ExecutionStrategy") , defined and owned by the trainer, governs how algorithm and runner bundles are placed, connected, scaled, and aborted. It serves four primary purposes. Execution strategies first determine **bundle placement** — whether the two bundles run in the same thread, process, machine, or across separate machines. They also define **store management**, wrapping the store and specifying how data is shared between bundles. In terms of **scalability**, the strategy can replicate the runner bundle across multiple threads, processes, or machines to expand throughput on the runner side. The algorithm side remains single-process due to the complexity of parallelization. Mature frameworks such as _DeepSpeed_ and _Megatron_ already support distributed model training, so scaling of the algorithm bundle is delegated to those implementations. **Abort handling** is another core responsibility. Aborts may be triggered by normal exits, failures in either bundle, or user interrupts. The trainer must include cancellation interfaces for the bundles so that bundles can be cleanly aborted. When the algorithm bundle exits normally, the strategy signals the runner bundle to terminate. If the runner exits first, no signal is sent to the algorithm, as it may still be processing completed rollouts. In cases of failure or user interruption, the strategy signals both bundles to abort; if a bundle fails to respond, the strategy should attempt a forceful termination. Agent-lightning currently provides two execution strategies: **shared-memory** and **client-server**, described in the following sections. ### Shared-memory Strategy[¶](https://microsoft.github.io/agent-lightning/0.3.0/deep-dive/birds-eye-view/#shared-memory-strategy "Permanent link") [`SharedMemoryExecutionStrategy`](https://microsoft.github.io/agent-lightning/0.3.0/reference/trainer/#agentlightning.SharedMemoryExecutionStrategy " agentlightning.SharedMemoryExecutionStrategy") runs algorithm and runner bundles as threads in one process. The strategy wraps the store with [`LightningStoreThreaded`](https://microsoft.github.io/agent-lightning/0.3.0/reference/store/#agentlightning.LightningStoreThreaded " agentlightning.LightningStoreThreaded") , which guards calls with a lock for safe concurrency. This is good for lightweight debugging because components share one Python heap and avoid serialization. It is not suitable for heavy RL training or compute-intensive agents. flowchart TB subgraph MainProcess direction TB subgraph AlgorithmThread [Thread 0] Algorithm[Algorithm bundle] end subgraph RunnerThread1 [Thread 1] Runner1[Runner bundle #1] end subgraph RunnerThread2 [Thread 2] Runner2[Runner bundle #2] end subgraph RunnerThread3 [Thread 3] RunnerN[Runner bundle #N] end LightningStoreFacade[LightningStoreThreaded] BaseStore[Underlying LightningStore] end Algorithm -- async calls --> LightningStoreFacade Runner1 -- async calls --> LightningStoreFacade Runner2 -- async calls --> LightningStoreFacade RunnerN -- async calls --> LightningStoreFacade LightningStoreFacade -->|thread-safe delegates| BaseStore You can configure which role runs on the main thread. If the main thread runs the algorithm, it is able to spawn multiple runner threads. If it runs a runner, `n_runners` must be 1 and the runner lives on the main thread. ### Client-server Strategy[¶](https://microsoft.github.io/agent-lightning/0.3.0/deep-dive/birds-eye-view/#client-server-strategy "Permanent link") [](https://microsoft.github.io/agent-lightning/0.3.0/deep-dive/birds-eye-view/) [`ClientServerExecutionStrategy`](https://microsoft.github.io/agent-lightning/0.3.0/reference/trainer/#agentlightning.ClientServerExecutionStrategy " agentlightning.ClientServerExecutionStrategy") splits concerns across processes. The algorithm bundle starts a [`LightningStoreServer`](https://microsoft.github.io/agent-lightning/0.3.0/reference/store/#agentlightning.LightningStoreServer " agentlightning.LightningStoreServer") (HTTP API) that wraps the underlying store. Runners connect via [`LightningStoreClient`](https://microsoft.github.io/agent-lightning/0.3.0/reference/store/#agentlightning.LightningStoreClient " agentlightning.LightningStoreClient") to call the same interface over REST. The server embeds a client to support algorithm-launched subprocesses (e.g., an LLM proxy worker) that need to talk back to the algorithm’s process through the same API. Currently this design introduces an extra wrapper in the Server side (as shown in the diagram), which helps debugging and improves fault tolerance. We might revisit this design in the future and enforce the client to be the only way to communicate with the store. flowchart TD subgraph Algorithm Process Group subgraph StoreServer[LightningStoreServer] StoreHttpClient[HTTP Client] StoreHttpServer[HTTP Server] StoreWrapper[LightningStore Wrapper] StoreHttpClient -- HTTP --> StoreHttpServer end subgraph Algorithm Bundle Algorithm[Algorithm Main Process] subgraph Another subprocess LLMProxy[LLM Proxy] end end LLMProxy -- async calls --> StoreHttpClient Algorithm -- async calls --> StoreWrapper end subgraph RunnerSide ["Runner Side"] subgraph Runner Process 1 Runner1[Runner bundle #1] Runner1 -- async calls --> LightningStoreClient1 LightningStoreClient1[LightningStoreClient] end subgraph Runner Process 2 Runner2[Runner bundle #2] Runner2 -- async calls --> LightningStoreClient2 LightningStoreClient2[LightningStoreClient] end subgraph Runner Process N RunnerN[Runner bundle #N] RunnerN -- async calls --> LightningStoreClientN LightningStoreClientN[LightningStoreClient] end end LocalStore[Underlying LightningStore] StoreHttpServer -->|delegates| StoreWrapper StoreWrapper -->|delegates| LocalStore LightningStoreClient1 -- HTTP --> StoreHttpServer LightningStoreClient2 -- HTTP --> StoreHttpServer LightningStoreClientN -- HTTP --> StoreHttpServer style RunnerSide fill:none; Online/Continuous Learning[¶](https://microsoft.github.io/agent-lightning/0.3.0/deep-dive/birds-eye-view/#onlinecontinuous-learning "Permanent link") ------------------------------------------------------------------------------------------------------------------------------------------------------ Continuous learning keeps the algorithm loop running while runners report tasks and spans opportunistically. Key differences from batch mode: 1. The algorithm does not enqueue rollouts from a fixed dataset. Runners report tasks/rollouts and spans spontaneously. 2. The algorithm can wait for rollouts with a expected set of rollout IDs, but more often polls for new rollouts and spans or waits for a count to arrive. 3. The [`Runner`](https://microsoft.github.io/agent-lightning/0.3.0/reference/runner/#agentlightning.Runner " agentlightning.Runner") processes one rollout at a time via [`step(task)`](https://microsoft.github.io/agent-lightning/0.3.0/reference/runner/#agentlightning.Runner.step " step(input, *, resources=None, mode=None, event=None) async ") instead of exhausting a task queue. It notifies the store when starting a rollout so the store records it. 4. A user or higher-level loop controls which resources the next step uses and when to retry. [Spans](https://microsoft.github.io/agent-lightning/0.3.0/reference/types/#agentlightning.Span " agentlightning.Span") , [Adapter](https://microsoft.github.io/agent-lightning/0.3.0/reference/algorithm/#agentlightning.Adapter " agentlightning.Adapter") implementations, and the [LLM Proxy](https://microsoft.github.io/agent-lightning/0.3.0/reference/algorithm/#agentlightning.LLMProxy " agentlightning.LLMProxy") work the same way. sequenceDiagram autonumber actor User participant Runner participant Agent participant Store as LightningStore participant Algorithm Note over Algorithm: Algorithm is long-running and loops continuously loop Continuous Learning Loop activate User opt Decide what to do next User-->>Store: get_resources_by_id Store-->>User: Resources User-->>User: Prepare input for next step end User->>Runner: step(input, resources) activate Runner Runner-->>Store: Notify: start_rollout(input) Runner->>Agent: rollout(input, resources) Agent-->>Runner: add_span / reward spans Runner-->>Store: add_span or add_otel_span Runner-->>Store: update_attempt(status="finished") deactivate Runner deactivate User Algorithm->>Store: poll for new rollouts and spans opt If there is enough new data Store-->>Algorithm: new spans Algorithm->>Algorithm: adapt spans → learning signal Algorithm->>Store: update_resources end end Back to top --- # Command Line - Agent-lightning [Skip to content](https://microsoft.github.io/agent-lightning/0.2.1/reference/cli/#command-line-interface) ⚠️ You are viewing development documentation. For stable documentation, please visit [https://microsoft.github.io/agent-lightning/stable/](https://microsoft.github.io/agent-lightning/stable/) Command Line Interface[¶](https://microsoft.github.io/agent-lightning/0.2.1/reference/cli/#command-line-interface "Permanent link") ==================================================================================================================================== Warning This document is a work in progress and might not be updated with the latest changes. Try to use `agl -h` to get the latest help message. Tip Agent-lightning also provides utilities to help you build your own CLI for [LitAgent](https://microsoft.github.io/agent-lightning/0.2.1/reference/agent/#agentlightning.LitAgent " agentlightning.LitAgent") and [Trainer](https://microsoft.github.io/agent-lightning/0.2.1/reference/trainer/#agentlightning.Trainer " agentlightning.Trainer") . See [Trainer](https://microsoft.github.io/agent-lightning/0.2.1/reference/trainer/) for references. agl[¶](https://microsoft.github.io/agent-lightning/0.2.1/reference/cli/#agl "Permanent link") ---------------------------------------------------------------------------------------------- `[](https://microsoft.github.io/agent-lightning/0.2.1/reference/cli/#__codelineno-0-1) usage: agl [-h] {vllm,store,agentops} [](https://microsoft.github.io/agent-lightning/0.2.1/reference/cli/#__codelineno-0-2) [](https://microsoft.github.io/agent-lightning/0.2.1/reference/cli/#__codelineno-0-3) Agent Lightning CLI entry point. [](https://microsoft.github.io/agent-lightning/0.2.1/reference/cli/#__codelineno-0-4) [](https://microsoft.github.io/agent-lightning/0.2.1/reference/cli/#__codelineno-0-5) Available subcommands: [](https://microsoft.github.io/agent-lightning/0.2.1/reference/cli/#__codelineno-0-6) vllm Run the vLLM CLI with Agent Lightning instrumentation. [](https://microsoft.github.io/agent-lightning/0.2.1/reference/cli/#__codelineno-0-7) store Run a LightningStore server. [](https://microsoft.github.io/agent-lightning/0.2.1/reference/cli/#__codelineno-0-8) agentops Start the AgentOps server manager. [](https://microsoft.github.io/agent-lightning/0.2.1/reference/cli/#__codelineno-0-9) [](https://microsoft.github.io/agent-lightning/0.2.1/reference/cli/#__codelineno-0-10) positional arguments: [](https://microsoft.github.io/agent-lightning/0.2.1/reference/cli/#__codelineno-0-11) {vllm,store,agentops} [](https://microsoft.github.io/agent-lightning/0.2.1/reference/cli/#__codelineno-0-12) Subcommand to run. [](https://microsoft.github.io/agent-lightning/0.2.1/reference/cli/#__codelineno-0-13) [](https://microsoft.github.io/agent-lightning/0.2.1/reference/cli/#__codelineno-0-14) options: [](https://microsoft.github.io/agent-lightning/0.2.1/reference/cli/#__codelineno-0-15) -h, --help show this help message and exit` agl vllm[¶](https://microsoft.github.io/agent-lightning/0.2.1/reference/cli/#agl-vllm "Permanent link") -------------------------------------------------------------------------------------------------------- Agent-lightning's instrumented vLLM CLI. `[](https://microsoft.github.io/agent-lightning/0.2.1/reference/cli/#__codelineno-1-1) usage: agl vllm [-h] [-v] {chat,complete,serve,bench,collect-env,run-batch} ... [](https://microsoft.github.io/agent-lightning/0.2.1/reference/cli/#__codelineno-1-2) [](https://microsoft.github.io/agent-lightning/0.2.1/reference/cli/#__codelineno-1-3) vLLM CLI [](https://microsoft.github.io/agent-lightning/0.2.1/reference/cli/#__codelineno-1-4) [](https://microsoft.github.io/agent-lightning/0.2.1/reference/cli/#__codelineno-1-5) positional arguments: [](https://microsoft.github.io/agent-lightning/0.2.1/reference/cli/#__codelineno-1-6) {chat,complete,serve,bench,collect-env,run-batch} [](https://microsoft.github.io/agent-lightning/0.2.1/reference/cli/#__codelineno-1-7) chat Generate chat completions via the running API server. [](https://microsoft.github.io/agent-lightning/0.2.1/reference/cli/#__codelineno-1-8) complete Generate text completions based on the given prompt via the running API server. [](https://microsoft.github.io/agent-lightning/0.2.1/reference/cli/#__codelineno-1-9) collect-env Start collecting environment information. [](https://microsoft.github.io/agent-lightning/0.2.1/reference/cli/#__codelineno-1-10) run-batch Run batch prompts and write results to file. [](https://microsoft.github.io/agent-lightning/0.2.1/reference/cli/#__codelineno-1-11) [](https://microsoft.github.io/agent-lightning/0.2.1/reference/cli/#__codelineno-1-12) options: [](https://microsoft.github.io/agent-lightning/0.2.1/reference/cli/#__codelineno-1-13) -h, --help show this help message and exit [](https://microsoft.github.io/agent-lightning/0.2.1/reference/cli/#__codelineno-1-14) -v, --version show program's version number and exit [](https://microsoft.github.io/agent-lightning/0.2.1/reference/cli/#__codelineno-1-15) [](https://microsoft.github.io/agent-lightning/0.2.1/reference/cli/#__codelineno-1-16) For full list: vllm [subcommand] --help=all [](https://microsoft.github.io/agent-lightning/0.2.1/reference/cli/#__codelineno-1-17) For a section: vllm [subcommand] --help=ModelConfig (case-insensitive) [](https://microsoft.github.io/agent-lightning/0.2.1/reference/cli/#__codelineno-1-18) For a flag: vllm [subcommand] --help=max-model-len (_ or - accepted) [](https://microsoft.github.io/agent-lightning/0.2.1/reference/cli/#__codelineno-1-19) Documentation: https://docs.vllm.ai` agl store[¶](https://microsoft.github.io/agent-lightning/0.2.1/reference/cli/#agl-store "Permanent link") ---------------------------------------------------------------------------------------------------------- Agent-lightning's LightningStore CLI. Use it to start an independent LightningStore server. Currently the store data are stored in memory and will be lost when the server is stopped. `[](https://microsoft.github.io/agent-lightning/0.2.1/reference/cli/#__codelineno-2-1) usage: agl store [-h] [--port PORT] [](https://microsoft.github.io/agent-lightning/0.2.1/reference/cli/#__codelineno-2-2) [](https://microsoft.github.io/agent-lightning/0.2.1/reference/cli/#__codelineno-2-3) Run a LightningStore server [](https://microsoft.github.io/agent-lightning/0.2.1/reference/cli/#__codelineno-2-4) [](https://microsoft.github.io/agent-lightning/0.2.1/reference/cli/#__codelineno-2-5) options: [](https://microsoft.github.io/agent-lightning/0.2.1/reference/cli/#__codelineno-2-6) -h, --help show this help message and exit [](https://microsoft.github.io/agent-lightning/0.2.1/reference/cli/#__codelineno-2-7) --port PORT Port to run the server on` agl agentops[¶](https://microsoft.github.io/agent-lightning/0.2.1/reference/cli/#agl-agentops "Permanent link") ---------------------------------------------------------------------------------------------------------------- Start a mock AgentOps server to bypass the online service of AgentOps. `[](https://microsoft.github.io/agent-lightning/0.2.1/reference/cli/#__codelineno-3-1) usage: agl agentops [-h] [--daemon] [--port PORT] [](https://microsoft.github.io/agent-lightning/0.2.1/reference/cli/#__codelineno-3-2) [](https://microsoft.github.io/agent-lightning/0.2.1/reference/cli/#__codelineno-3-3) Start AgentOps server [](https://microsoft.github.io/agent-lightning/0.2.1/reference/cli/#__codelineno-3-4) [](https://microsoft.github.io/agent-lightning/0.2.1/reference/cli/#__codelineno-3-5) options: [](https://microsoft.github.io/agent-lightning/0.2.1/reference/cli/#__codelineno-3-6) -h, --help show this help message and exit [](https://microsoft.github.io/agent-lightning/0.2.1/reference/cli/#__codelineno-3-7) --daemon Run server as a daemon [](https://microsoft.github.io/agent-lightning/0.2.1/reference/cli/#__codelineno-3-8) --port PORT Port to run the server on` Back to top --- # Instrumentation - Agent-lightning [Skip to content](https://microsoft.github.io/agent-lightning/0.2.0/reference/instrumentation/#instrumentation-api) ⚠️ You are viewing development documentation. For stable documentation, please visit [https://microsoft.github.io/agent-lightning/stable/](https://microsoft.github.io/agent-lightning/stable/) Instrumentation API[¶](https://microsoft.github.io/agent-lightning/0.2.0/reference/instrumentation/#instrumentation-api "Permanent link") ========================================================================================================================================== ### `agentlightning.instrumentation.instrument_all()` [¶](https://microsoft.github.io/agent-lightning/0.2.0/reference/instrumentation/#agentlightning.instrumentation.instrument_all "Permanent link") Instrument all the instrumentation libraries. ### `agentlightning.instrumentation.uninstrument_all()` [¶](https://microsoft.github.io/agent-lightning/0.2.0/reference/instrumentation/#agentlightning.instrumentation.uninstrument_all "Permanent link") Uninstrument all the instrumentation libraries. AgentOps LangChain[¶](https://microsoft.github.io/agent-lightning/0.2.0/reference/instrumentation/#agentops-langchain "Permanent link") ---------------------------------------------------------------------------------------------------------------------------------------- ### `agentlightning.instrumentation.agentops_langchain` [¶](https://microsoft.github.io/agent-lightning/0.2.0/reference/instrumentation/#agentlightning.instrumentation.agentops_langchain "Permanent link") #### `instrument_agentops_langchain()` [¶](https://microsoft.github.io/agent-lightning/0.2.0/reference/instrumentation/#agentlightning.instrumentation.agentops_langchain.instrument_agentops_langchain "Permanent link") Bypass AgentOp's native support for Langchain. #### `uninstrument_agentops_langchain()` [¶](https://microsoft.github.io/agent-lightning/0.2.0/reference/instrumentation/#agentlightning.instrumentation.agentops_langchain.uninstrument_agentops_langchain "Permanent link") Restore AgentOp's native support for Langchain. AgentOps[¶](https://microsoft.github.io/agent-lightning/0.2.0/reference/instrumentation/#agentops "Permanent link") -------------------------------------------------------------------------------------------------------------------- ### `agentlightning.instrumentation.agentops` [¶](https://microsoft.github.io/agent-lightning/0.2.0/reference/instrumentation/#agentlightning.instrumentation.agentops "Permanent link") #### `AgentOpsServerManager` [¶](https://microsoft.github.io/agent-lightning/0.2.0/reference/instrumentation/#agentlightning.instrumentation.agentops.AgentOpsServerManager "Permanent link") Manages a AgentOps local server to bypass the online service of AgentOps. #### `agentops_local_server()` [¶](https://microsoft.github.io/agent-lightning/0.2.0/reference/instrumentation/#agentlightning.instrumentation.agentops.agentops_local_server "Permanent link") Returns a Flask app that can be used to test agentops integration. This server provides endpoints for token fetching and a catch-all endpoint. #### `instrument_agentops()` [¶](https://microsoft.github.io/agent-lightning/0.2.0/reference/instrumentation/#agentlightning.instrumentation.agentops.instrument_agentops "Permanent link") Instrument agentops to capture token IDs. Automatically detects and uses the appropriate patching method based on the installed agentops version. #### `uninstrument_agentops()` [¶](https://microsoft.github.io/agent-lightning/0.2.0/reference/instrumentation/#agentlightning.instrumentation.agentops.uninstrument_agentops "Permanent link") Uninstrument agentops to stop capturing token IDs. LiteLLM[¶](https://microsoft.github.io/agent-lightning/0.2.0/reference/instrumentation/#litellm "Permanent link") ------------------------------------------------------------------------------------------------------------------ ### `agentlightning.instrumentation.litellm` [¶](https://microsoft.github.io/agent-lightning/0.2.0/reference/instrumentation/#agentlightning.instrumentation.litellm "Permanent link") LiteLLM instrumentations. It's unclear whether or not this file is useful. It seems that LiteLLM owns its own telemetry from their own entrance [Related documentation](https://docs.litellm.ai/docs/observability/agentops_integration) . #### `instrument_litellm()` [¶](https://microsoft.github.io/agent-lightning/0.2.0/reference/instrumentation/#agentlightning.instrumentation.litellm.instrument_litellm "Permanent link") Instrument litellm to capture token IDs. #### `uninstrument_litellm()` [¶](https://microsoft.github.io/agent-lightning/0.2.0/reference/instrumentation/#agentlightning.instrumentation.litellm.uninstrument_litellm "Permanent link") Uninstrument litellm to stop capturing token IDs. vLLM[¶](https://microsoft.github.io/agent-lightning/0.2.0/reference/instrumentation/#vllm "Permanent link") ------------------------------------------------------------------------------------------------------------ ### `agentlightning.instrumentation.vllm` [¶](https://microsoft.github.io/agent-lightning/0.2.0/reference/instrumentation/#agentlightning.instrumentation.vllm "Permanent link") #### `instrument_vllm()` [¶](https://microsoft.github.io/agent-lightning/0.2.0/reference/instrumentation/#agentlightning.instrumentation.vllm.instrument_vllm "Permanent link") Instrument vLLM to capture token IDs generated by engine. This instrumentation has been merged to upstream vLLM since v0.10.2. #### `uninstrument_vllm()` [¶](https://microsoft.github.io/agent-lightning/0.2.0/reference/instrumentation/#agentlightning.instrumentation.vllm.uninstrument_vllm "Permanent link") Uninstrument vLLM to stop capturing token IDs generated by engine. Back to top --- # Agent - Agent-lightning [Skip to content](https://microsoft.github.io/agent-lightning/0.2.2/reference/agent/#agent-developer-apis) ⚠️ You are viewing development documentation. For stable documentation, please visit [https://microsoft.github.io/agent-lightning/stable/](https://microsoft.github.io/agent-lightning/stable/) Agent Developer APIs[¶](https://microsoft.github.io/agent-lightning/0.2.2/reference/agent/#agent-developer-apis "Permanent link") ================================================================================================================================== Agent Decorators[¶](https://microsoft.github.io/agent-lightning/0.2.2/reference/agent/#agent-decorators "Permanent link") -------------------------------------------------------------------------------------------------------------------------- Tip These are convenient helpers for creating agents from functions. First-time users are recommended to use these decorators to create agents. ### `agentlightning.rollout(func)` [¶](https://microsoft.github.io/agent-lightning/0.2.2/reference/agent/#agentlightning.rollout "Permanent link") Create a [`FunctionalLitAgent`](https://microsoft.github.io/agent-lightning/0.2.2/reference/internal/#agentlightning.litagent.decorator.FunctionalLitAgent " agentlightning.litagent.decorator.FunctionalLitAgent") from an arbitrary rollout function. This function inspects the provided callable and creates the appropriate agent type based on its signature. It supports both LLM-based and prompt-template-based agents. The returned agent instance is callable, preserving the original function's behavior and type hints. See [`llm_rollout`](https://microsoft.github.io/agent-lightning/0.2.2/reference/internal/#agentlightning.litagent.decorator.llm_rollout " agentlightning.litagent.decorator.llm_rollout(func=None, *, strip_proxy=True)") and [`prompt_rollout`](https://microsoft.github.io/agent-lightning/0.2.2/reference/internal/#agentlightning.litagent.decorator.prompt_rollout " agentlightning.litagent.decorator.prompt_rollout(func=None)") for more details. Parameters: * **`func`** (`Union[LlmRolloutFunc[T], PromptRolloutFunc[T], Callable[..., Any]]`) – Callable that implements the rollout. Supported signatures: * `[async ](task, llm[, rollout])` for LLM-based agents * `[async ](task, prompt_template[, rollout])` for prompt-template-based agents The supported output types of `func` is same as the return type of [`rollout`](https://microsoft.github.io/agent-lightning/0.2.2/reference/agent/#agentlightning.LitAgent.rollout " rollout(task, resources, rollout)") . Returns: * `[FunctionalLitAgent](https://microsoft.github.io/agent-lightning/0.2.2/reference/internal/#agentlightning.litagent.decorator.FunctionalLitAgent " agentlightning.litagent.decorator.FunctionalLitAgent") [T]` – [`FunctionalLitAgent`](https://microsoft.github.io/agent-lightning/0.2.2/reference/internal/#agentlightning.litagent.decorator.FunctionalLitAgent " agentlightning.litagent.decorator.FunctionalLitAgent") that * `[FunctionalLitAgent](https://microsoft.github.io/agent-lightning/0.2.2/reference/internal/#agentlightning.litagent.decorator.FunctionalLitAgent " agentlightning.litagent.decorator.FunctionalLitAgent") [T]` – wraps the supplied function. Examples: `[](https://microsoft.github.io/agent-lightning/0.2.2/reference/agent/#__codelineno-0-1) # LLM-based agent [](https://microsoft.github.io/agent-lightning/0.2.2/reference/agent/#__codelineno-0-2) @rollout [](https://microsoft.github.io/agent-lightning/0.2.2/reference/agent/#__codelineno-0-3) def my_llm_agent(task, llm): [](https://microsoft.github.io/agent-lightning/0.2.2/reference/agent/#__codelineno-0-4) client = OpenAI(base_url=llm.endpoint) [](https://microsoft.github.io/agent-lightning/0.2.2/reference/agent/#__codelineno-0-5) response = client.chat.completions.create( [](https://microsoft.github.io/agent-lightning/0.2.2/reference/agent/#__codelineno-0-6) model=llm.model, [](https://microsoft.github.io/agent-lightning/0.2.2/reference/agent/#__codelineno-0-7) messages=[{"role": "user", "content": task.input}], [](https://microsoft.github.io/agent-lightning/0.2.2/reference/agent/#__codelineno-0-8) ) [](https://microsoft.github.io/agent-lightning/0.2.2/reference/agent/#__codelineno-0-9) return response [](https://microsoft.github.io/agent-lightning/0.2.2/reference/agent/#__codelineno-0-10) [](https://microsoft.github.io/agent-lightning/0.2.2/reference/agent/#__codelineno-0-11) # Prompt-template-based agent [](https://microsoft.github.io/agent-lightning/0.2.2/reference/agent/#__codelineno-0-12) @rollout [](https://microsoft.github.io/agent-lightning/0.2.2/reference/agent/#__codelineno-0-13) def my_prompt_agent(task, prompt_template): [](https://microsoft.github.io/agent-lightning/0.2.2/reference/agent/#__codelineno-0-14) messages = prompt_template.format(task=task.input) [](https://microsoft.github.io/agent-lightning/0.2.2/reference/agent/#__codelineno-0-15) # ... perform rollout with the formatted prompt [](https://microsoft.github.io/agent-lightning/0.2.2/reference/agent/#__codelineno-0-16) return response [](https://microsoft.github.io/agent-lightning/0.2.2/reference/agent/#__codelineno-0-17) [](https://microsoft.github.io/agent-lightning/0.2.2/reference/agent/#__codelineno-0-18) # Function is still callable with original behavior [](https://microsoft.github.io/agent-lightning/0.2.2/reference/agent/#__codelineno-0-19) result = my_llm_agent(task, llm) [](https://microsoft.github.io/agent-lightning/0.2.2/reference/agent/#__codelineno-0-20) [](https://microsoft.github.io/agent-lightning/0.2.2/reference/agent/#__codelineno-0-21) # Agent methods are also available [](https://microsoft.github.io/agent-lightning/0.2.2/reference/agent/#__codelineno-0-22) result = my_llm_agent.rollout(task, resources, rollout)` Raises: * `NotImplementedError` – If the function signature doesn't match any known patterns. Warning The following two decorators are implementations of [`agentlightning.rollout`](https://microsoft.github.io/agent-lightning/0.2.2/reference/agent/#agentlightning.rollout " agentlightning.rollout(func)") . They are not recommended for new users. ### `agentlightning.llm_rollout(func=None, *, strip_proxy=True)` [¶](https://microsoft.github.io/agent-lightning/0.2.2/reference/agent/#agentlightning.llm_rollout "Permanent link") `[](https://microsoft.github.io/agent-lightning/0.2.2/reference/agent/#__codelineno-0-1) llm_rollout( [](https://microsoft.github.io/agent-lightning/0.2.2/reference/agent/#__codelineno-0-2) func: LlmRolloutFunc[T], [](https://microsoft.github.io/agent-lightning/0.2.2/reference/agent/#__codelineno-0-3) ) -> FunctionalLitAgent[T]` `[](https://microsoft.github.io/agent-lightning/0.2.2/reference/agent/#__codelineno-0-1) llm_rollout( [](https://microsoft.github.io/agent-lightning/0.2.2/reference/agent/#__codelineno-0-2) *, strip_proxy: bool = True [](https://microsoft.github.io/agent-lightning/0.2.2/reference/agent/#__codelineno-0-3) ) -> Callable[[LlmRolloutFunc[T]], FunctionalLitAgent[T]]` Create a [`FunctionalLitAgent`](https://microsoft.github.io/agent-lightning/0.2.2/reference/internal/#agentlightning.litagent.decorator.FunctionalLitAgent " agentlightning.litagent.decorator.FunctionalLitAgent") for LLM-based rollouts. Parameters: * **`func`** (`LlmRolloutFunc[T] | None`, default: `None` ) – Callable defining the agent's behaviour. Supported signatures include: * `(task, llm) -> result` * `(task, llm, rollout) -> result` * `async (task, llm) -> result` * `async (task, llm, rollout) -> result` * **`strip_proxy`** (`bool`, default: `True` ) – When `True`, convert proxy resources into concrete [`LLM`](https://microsoft.github.io/agent-lightning/0.2.2/reference/types/#agentlightning.LLM " agentlightning.LLM") instances before calling the function. Defaults to `True`. Returns: * `[FunctionalLitAgent](https://microsoft.github.io/agent-lightning/0.2.2/reference/internal/#agentlightning.litagent.decorator.FunctionalLitAgent " agentlightning.litagent.decorator.FunctionalLitAgent") [T] | Callable[[LlmRolloutFunc[T]], [FunctionalLitAgent](https://microsoft.github.io/agent-lightning/0.2.2/reference/internal/#agentlightning.litagent.decorator.FunctionalLitAgent " agentlightning.litagent.decorator.FunctionalLitAgent") [T]]` – [`FunctionalLitAgent`](https://microsoft.github.io/agent-lightning/0.2.2/reference/internal/#agentlightning.litagent.decorator.FunctionalLitAgent " agentlightning.litagent.decorator.FunctionalLitAgent") that * `[FunctionalLitAgent](https://microsoft.github.io/agent-lightning/0.2.2/reference/internal/#agentlightning.litagent.decorator.FunctionalLitAgent " agentlightning.litagent.decorator.FunctionalLitAgent") [T] | Callable[[LlmRolloutFunc[T]], [FunctionalLitAgent](https://microsoft.github.io/agent-lightning/0.2.2/reference/internal/#agentlightning.litagent.decorator.FunctionalLitAgent " agentlightning.litagent.decorator.FunctionalLitAgent") [T]]` – wraps the supplied function. Examples: `[](https://microsoft.github.io/agent-lightning/0.2.2/reference/agent/#__codelineno-0-1) @llm_rollout [](https://microsoft.github.io/agent-lightning/0.2.2/reference/agent/#__codelineno-0-2) def my_agent(task, llm): [](https://microsoft.github.io/agent-lightning/0.2.2/reference/agent/#__codelineno-0-3) return llm.endpoint [](https://microsoft.github.io/agent-lightning/0.2.2/reference/agent/#__codelineno-0-4) [](https://microsoft.github.io/agent-lightning/0.2.2/reference/agent/#__codelineno-0-5) @llm_rollout(strip_proxy=False) [](https://microsoft.github.io/agent-lightning/0.2.2/reference/agent/#__codelineno-0-6) def my_agent_no_strip(task, llm): [](https://microsoft.github.io/agent-lightning/0.2.2/reference/agent/#__codelineno-0-7) return llm.model [](https://microsoft.github.io/agent-lightning/0.2.2/reference/agent/#__codelineno-0-8) [](https://microsoft.github.io/agent-lightning/0.2.2/reference/agent/#__codelineno-0-9) result = my_agent(task, llm) [](https://microsoft.github.io/agent-lightning/0.2.2/reference/agent/#__codelineno-0-10) result = my_agent.rollout(task, resources, rollout)` ### `agentlightning.prompt_rollout(func=None)` [¶](https://microsoft.github.io/agent-lightning/0.2.2/reference/agent/#agentlightning.prompt_rollout "Permanent link") `[](https://microsoft.github.io/agent-lightning/0.2.2/reference/agent/#__codelineno-0-1) prompt_rollout( [](https://microsoft.github.io/agent-lightning/0.2.2/reference/agent/#__codelineno-0-2) func: PromptRolloutFunc[T], [](https://microsoft.github.io/agent-lightning/0.2.2/reference/agent/#__codelineno-0-3) ) -> FunctionalLitAgent[T]` `[](https://microsoft.github.io/agent-lightning/0.2.2/reference/agent/#__codelineno-0-1) prompt_rollout() -> ( [](https://microsoft.github.io/agent-lightning/0.2.2/reference/agent/#__codelineno-0-2) Callable[[PromptRolloutFunc[T]], FunctionalLitAgent[T]] [](https://microsoft.github.io/agent-lightning/0.2.2/reference/agent/#__codelineno-0-3) )` Create a [`FunctionalLitAgent`](https://microsoft.github.io/agent-lightning/0.2.2/reference/internal/#agentlightning.litagent.decorator.FunctionalLitAgent " agentlightning.litagent.decorator.FunctionalLitAgent") for prompt-based rollouts. This decorator is designed for agents that work with tunable prompt templates. It enables a workflow where algorithms manage and optimize the prompt template, while agents consume the template to perform rollouts. This is particularly useful for prompt optimization scenarios. Parameters: * **`func`** (`PromptRolloutFunc[T] | None`, default: `None` ) – Callable defining the agent's behavior. Supported signatures include: * `(task, prompt_template) -> result` * `(task, prompt_template, rollout) -> result` * `async (task, prompt_template) -> result` * `async (task, prompt_template, rollout) -> result` Returns: * `[FunctionalLitAgent](https://microsoft.github.io/agent-lightning/0.2.2/reference/internal/#agentlightning.litagent.decorator.FunctionalLitAgent " agentlightning.litagent.decorator.FunctionalLitAgent") [T] | Callable[[PromptRolloutFunc[T]], [FunctionalLitAgent](https://microsoft.github.io/agent-lightning/0.2.2/reference/internal/#agentlightning.litagent.decorator.FunctionalLitAgent " agentlightning.litagent.decorator.FunctionalLitAgent") [T]]` – [`FunctionalLitAgent`](https://microsoft.github.io/agent-lightning/0.2.2/reference/internal/#agentlightning.litagent.decorator.FunctionalLitAgent " agentlightning.litagent.decorator.FunctionalLitAgent") that * `[FunctionalLitAgent](https://microsoft.github.io/agent-lightning/0.2.2/reference/internal/#agentlightning.litagent.decorator.FunctionalLitAgent " agentlightning.litagent.decorator.FunctionalLitAgent") [T] | Callable[[PromptRolloutFunc[T]], [FunctionalLitAgent](https://microsoft.github.io/agent-lightning/0.2.2/reference/internal/#agentlightning.litagent.decorator.FunctionalLitAgent " agentlightning.litagent.decorator.FunctionalLitAgent") [T]]` – wraps the supplied function. Examples: `[](https://microsoft.github.io/agent-lightning/0.2.2/reference/agent/#__codelineno-0-1) @prompt_rollout [](https://microsoft.github.io/agent-lightning/0.2.2/reference/agent/#__codelineno-0-2) def my_agent(task, prompt_template): [](https://microsoft.github.io/agent-lightning/0.2.2/reference/agent/#__codelineno-0-3) messages = prompt_template.format(task=task.input) [](https://microsoft.github.io/agent-lightning/0.2.2/reference/agent/#__codelineno-0-4) return messages [](https://microsoft.github.io/agent-lightning/0.2.2/reference/agent/#__codelineno-0-5) [](https://microsoft.github.io/agent-lightning/0.2.2/reference/agent/#__codelineno-0-6) result = my_agent(task, prompt_template) [](https://microsoft.github.io/agent-lightning/0.2.2/reference/agent/#__codelineno-0-7) result = my_agent.rollout(task, resources, rollout)` Class-based Agents[¶](https://microsoft.github.io/agent-lightning/0.2.2/reference/agent/#class-based-agents "Permanent link") ------------------------------------------------------------------------------------------------------------------------------ ### `agentlightning.LitAgent` [¶](https://microsoft.github.io/agent-lightning/0.2.2/reference/agent/#agentlightning.LitAgent "Permanent link") Bases: `Generic[T]` Base class for implementing agent rollouts. Subclasses override the rollout methods to process tasks while the trainer and runner infrastructure manages orchestration, tracing, and persistence. #### `runner` `property` [¶](https://microsoft.github.io/agent-lightning/0.2.2/reference/agent/#agentlightning.LitAgent.runner "Permanent link") Return the runner responsible for executing rollouts. #### `tracer` `property` [¶](https://microsoft.github.io/agent-lightning/0.2.2/reference/agent/#agentlightning.LitAgent.tracer "Permanent link") Return the tracer configured for this agent. #### `trainer` `property` [¶](https://microsoft.github.io/agent-lightning/0.2.2/reference/agent/#agentlightning.LitAgent.trainer "Permanent link") Return the trainer associated with this agent. #### `__init__(*, trained_agents=None)` [¶](https://microsoft.github.io/agent-lightning/0.2.2/reference/agent/#agentlightning.LitAgent.__init__ "Permanent link") Initialize the agent instance. Parameters: * **`trained_agents`** (`Optional[str]`, default: `None` ) – Optional identifier used by legacy tooling to mark trained agents. Deprecated The `trained_agents` flag is deprecated. Configure `agent_match` in the adapter layer instead. See [`TracerTraceToTriplet`](https://microsoft.github.io/agent-lightning/0.2.2/reference/algorithm/#agentlightning.TracerTraceToTriplet " agentlightning.TracerTraceToTriplet") for more details. #### `get_runner()` [¶](https://microsoft.github.io/agent-lightning/0.2.2/reference/agent/#agentlightning.LitAgent.get_runner "Permanent link") Return the runner responsible for executing rollouts. #### `get_tracer()` [¶](https://microsoft.github.io/agent-lightning/0.2.2/reference/agent/#agentlightning.LitAgent.get_tracer "Permanent link") Return the tracer configured for this agent. #### `get_trainer()` [¶](https://microsoft.github.io/agent-lightning/0.2.2/reference/agent/#agentlightning.LitAgent.get_trainer "Permanent link") Return the trainer associated with this agent. #### `is_async()` [¶](https://microsoft.github.io/agent-lightning/0.2.2/reference/agent/#agentlightning.LitAgent.is_async "Permanent link") Return `True` when the agent overrides any asynchronous rollout methods. Override this method for customized async detection logic. #### `on_rollout_end(task, rollout, runner, tracer)` [¶](https://microsoft.github.io/agent-lightning/0.2.2/reference/agent/#agentlightning.LitAgent.on_rollout_end "Permanent link") Hook invoked after a rollout completes. Subclasses can override this method for cleanup or additional logging. The default implementation is a no-op. Parameters: * **`task`** (`[Task](https://microsoft.github.io/agent-lightning/0.2.2/reference/internal/#agentlightning.Task " agentlightning.Task (agentlightning.types.Task)") `) – [`Task`](https://microsoft.github.io/agent-lightning/0.2.2/reference/internal/#agentlightning.Task " agentlightning.Task") that was processed. * **`rollout`** (`[Rollout](https://microsoft.github.io/agent-lightning/0.2.2/reference/types/#agentlightning.Rollout " agentlightning.Rollout (agentlightning.types.Rollout)") `) – Resulting [`Rollout`](https://microsoft.github.io/agent-lightning/0.2.2/reference/types/#agentlightning.Rollout " agentlightning.Rollout") . * **`runner`** (`[Runner](https://microsoft.github.io/agent-lightning/0.2.2/reference/runner/#agentlightning.Runner " agentlightning.Runner (agentlightning.runner.Runner)") [T]`) – [`Runner`](https://microsoft.github.io/agent-lightning/0.2.2/reference/runner/#agentlightning.Runner " agentlightning.Runner") managing the rollout. * **`tracer`** (`[Tracer](https://microsoft.github.io/agent-lightning/0.2.2/reference/runner/#agentlightning.Tracer " agentlightning.Tracer (agentlightning.tracer.Tracer)") `) – [`Tracer`](https://microsoft.github.io/agent-lightning/0.2.2/reference/runner/#agentlightning.Tracer " agentlightning.Tracer") associated with the runner. Deprecated Override [`Hook.on_rollout_end`](https://microsoft.github.io/agent-lightning/0.2.2/reference/types/#agentlightning.Hook.on_rollout_end " on_rollout_end(*, agent, runner, rollout, spans) async ") instead of this method when extending agents. #### `on_rollout_start(task, runner, tracer)` [¶](https://microsoft.github.io/agent-lightning/0.2.2/reference/agent/#agentlightning.LitAgent.on_rollout_start "Permanent link") Hook invoked immediately before a rollout begins. Subclasses can override this method to implement custom logic such as logging, metric collection, or resource setup. The default implementation is a no-op. Parameters: * **`task`** (`[Task](https://microsoft.github.io/agent-lightning/0.2.2/reference/internal/#agentlightning.Task " agentlightning.Task (agentlightning.types.Task)") `) – [`Task`](https://microsoft.github.io/agent-lightning/0.2.2/reference/internal/#agentlightning.Task " agentlightning.Task") that will be processed. * **`runner`** (`[Runner](https://microsoft.github.io/agent-lightning/0.2.2/reference/runner/#agentlightning.Runner " agentlightning.Runner (agentlightning.runner.Runner)") [T]`) – [`Runner`](https://microsoft.github.io/agent-lightning/0.2.2/reference/runner/#agentlightning.Runner " agentlightning.Runner") managing the rollout. * **`tracer`** (`[Tracer](https://microsoft.github.io/agent-lightning/0.2.2/reference/runner/#agentlightning.Tracer " agentlightning.Tracer (agentlightning.tracer.Tracer)") `) – [`Tracer`](https://microsoft.github.io/agent-lightning/0.2.2/reference/runner/#agentlightning.Tracer " agentlightning.Tracer") associated with the runner. Deprecated Override [`Hook.on_rollout_start`](https://microsoft.github.io/agent-lightning/0.2.2/reference/types/#agentlightning.Hook.on_rollout_start " on_rollout_start(*, agent, runner, rollout) async ") instead of this method when extending agents. #### `rollout(task, resources, rollout)` [¶](https://microsoft.github.io/agent-lightning/0.2.2/reference/agent/#agentlightning.LitAgent.rollout "Permanent link") Execute a rollout synchronously. If you don't wish to implement both training rollout and validation rollout separately, you can just implement `rollout` which will work for both. Parameters: * **`task`** (`T`) – Task payload provided by the scheduler. * **`resources`** (`[NamedResources](https://microsoft.github.io/agent-lightning/0.2.2/reference/types/#agentlightning.NamedResources " agentlightning.NamedResources = Dict[str, ResourceUnion] module-attribute (agentlightning.types.NamedResources)") `) – Mapping of named resources (for example LLMs or prompt templates). * **`rollout`** (`[Rollout](https://microsoft.github.io/agent-lightning/0.2.2/reference/types/#agentlightning.Rollout " agentlightning.Rollout (agentlightning.types.Rollout)") `) – Rollout metadata. Avoid mutating this object directly unless a subclass needs to override defaults. Returns: * `[RolloutRawResult](https://microsoft.github.io/agent-lightning/0.2.2/reference/types/#agentlightning.RolloutRawResult " agentlightning.RolloutRawResult = Union[None, float, List[ReadableSpan], List[Span]] module-attribute (agentlightning.types.RolloutRawResult)") ` – One of the following values: * `[RolloutRawResult](https://microsoft.github.io/agent-lightning/0.2.2/reference/types/#agentlightning.RolloutRawResult " agentlightning.RolloutRawResult = Union[None, float, List[ReadableSpan], List[Span]] module-attribute (agentlightning.types.RolloutRawResult)") ` – * `None` when tracing is handled by the runner. * `[RolloutRawResult](https://microsoft.github.io/agent-lightning/0.2.2/reference/types/#agentlightning.RolloutRawResult " agentlightning.RolloutRawResult = Union[None, float, List[ReadableSpan], List[Span]] module-attribute (agentlightning.types.RolloutRawResult)") ` – * `float` representing the final reward. * `[RolloutRawResult](https://microsoft.github.io/agent-lightning/0.2.2/reference/types/#agentlightning.RolloutRawResult " agentlightning.RolloutRawResult = Union[None, float, List[ReadableSpan], List[Span]] module-attribute (agentlightning.types.RolloutRawResult)") ` – * `List[ReadableSpan]` with OpenTelemetry spans. * `[RolloutRawResult](https://microsoft.github.io/agent-lightning/0.2.2/reference/types/#agentlightning.RolloutRawResult " agentlightning.RolloutRawResult = Union[None, float, List[ReadableSpan], List[Span]] module-attribute (agentlightning.types.RolloutRawResult)") ` – * `List[Span]` with Agent Lightning spans. #### `rollout_async(task, resources, rollout)` `async` [¶](https://microsoft.github.io/agent-lightning/0.2.2/reference/agent/#agentlightning.LitAgent.rollout_async "Permanent link") Execute a rollout asynchronously. Parameters: * **`task`** (`T`) – Task payload provided by the scheduler. * **`resources`** (`[NamedResources](https://microsoft.github.io/agent-lightning/0.2.2/reference/types/#agentlightning.NamedResources " agentlightning.NamedResources = Dict[str, ResourceUnion] module-attribute (agentlightning.types.NamedResources)") `) – Mapping of named resources (for example LLMs or prompt templates). * **`rollout`** (`[Rollout](https://microsoft.github.io/agent-lightning/0.2.2/reference/types/#agentlightning.Rollout " agentlightning.Rollout (agentlightning.types.Rollout)") `) – Rollout metadata. Avoid mutating this object directly unless a subclass needs to override defaults. Returns: * `[RolloutRawResult](https://microsoft.github.io/agent-lightning/0.2.2/reference/types/#agentlightning.RolloutRawResult " agentlightning.RolloutRawResult = Union[None, float, List[ReadableSpan], List[Span]] module-attribute (agentlightning.types.RolloutRawResult)") ` – Same possible return values as * `[RolloutRawResult](https://microsoft.github.io/agent-lightning/0.2.2/reference/types/#agentlightning.RolloutRawResult " agentlightning.RolloutRawResult = Union[None, float, List[ReadableSpan], List[Span]] module-attribute (agentlightning.types.RolloutRawResult)") ` – [`rollout`](https://microsoft.github.io/agent-lightning/0.2.2/reference/agent/#agentlightning.LitAgent.rollout " rollout(task, resources, rollout)") . #### `set_runner(runner)` [¶](https://microsoft.github.io/agent-lightning/0.2.2/reference/agent/#agentlightning.LitAgent.set_runner "Permanent link") Attach the runner responsible for executing rollouts. Parameters: * **`runner`** (`[Runner](https://microsoft.github.io/agent-lightning/0.2.2/reference/runner/#agentlightning.Runner " agentlightning.Runner (agentlightning.runner.Runner)") [T]`) – [`Runner`](https://microsoft.github.io/agent-lightning/0.2.2/reference/runner/#agentlightning.Runner " agentlightning.Runner") coordinating execution. #### `set_trainer(trainer)` [¶](https://microsoft.github.io/agent-lightning/0.2.2/reference/agent/#agentlightning.LitAgent.set_trainer "Permanent link") Attach the trainer responsible for orchestration. Parameters: * **`trainer`** (`[Trainer](https://microsoft.github.io/agent-lightning/0.2.2/reference/trainer/#agentlightning.Trainer " agentlightning.Trainer (agentlightning.trainer.Trainer)") `) – [`Trainer`](https://microsoft.github.io/agent-lightning/0.2.2/reference/trainer/#agentlightning.Trainer " agentlightning.Trainer") that manages the agent. #### `training_rollout(task, resources, rollout)` [¶](https://microsoft.github.io/agent-lightning/0.2.2/reference/agent/#agentlightning.LitAgent.training_rollout "Permanent link") Process a single training task synchronously. By default, this method delegates to [`rollout`](https://microsoft.github.io/agent-lightning/0.2.2/reference/agent/#agentlightning.LitAgent.rollout " rollout(task, resources, rollout)") . #### `training_rollout_async(task, resources, rollout)` `async` [¶](https://microsoft.github.io/agent-lightning/0.2.2/reference/agent/#agentlightning.LitAgent.training_rollout_async "Permanent link") Process a single training task asynchronously. By default, this method delegates to [`rollout_async`](https://microsoft.github.io/agent-lightning/0.2.2/reference/agent/#agentlightning.LitAgent.rollout_async " rollout_async(task, resources, rollout) async ") . #### `validation_rollout(task, resources, rollout)` [¶](https://microsoft.github.io/agent-lightning/0.2.2/reference/agent/#agentlightning.LitAgent.validation_rollout "Permanent link") Process a single validation task synchronously. Override this method when validation should differ from training. The default implementation delegates to [`training_rollout`](https://microsoft.github.io/agent-lightning/0.2.2/reference/agent/#agentlightning.LitAgent.training_rollout " training_rollout(task, resources, rollout)") . #### `validation_rollout_async(task, resources, rollout)` `async` [¶](https://microsoft.github.io/agent-lightning/0.2.2/reference/agent/#agentlightning.LitAgent.validation_rollout_async "Permanent link") Process a single validation task asynchronously. Override this method when validation should differ from training. The default implementation delegates to [`training_rollout_async`](https://microsoft.github.io/agent-lightning/0.2.2/reference/agent/#agentlightning.LitAgent.training_rollout_async " training_rollout_async(task, resources, rollout) async ") . Emitter[¶](https://microsoft.github.io/agent-lightning/0.2.2/reference/agent/#emitter "Permanent link") -------------------------------------------------------------------------------------------------------- ### `agentlightning.emit_reward(reward)` [¶](https://microsoft.github.io/agent-lightning/0.2.2/reference/agent/#agentlightning.emit_reward "Permanent link") Emit a reward value as an OpenTelemetry span. Parameters: * **`reward`** (`float`) – Numeric reward to record. Integers and booleans are converted to floating point numbers for consistency. Returns: * `ReadableSpan` – Readable span capturing the recorded reward. Raises: * `ValueError` – If the provided reward cannot be interpreted as a float or the resulting span is not a [`ReadableSpan`](https://opentelemetry.io/docs/concepts/signals/traces/) instance. ### `agentlightning.emit_message(message)` [¶](https://microsoft.github.io/agent-lightning/0.2.2/reference/agent/#agentlightning.emit_message "Permanent link") Emit a textual message as an OpenTelemetry span. Parameters: * **`message`** (`str`) – Human readable message to attach as a span attribute. Note OpenTelemetry distinguishes between logs and spans. Emitting the message as a span keeps all Agent Lightning telemetry in a single data store for analysis. ### `agentlightning.emit_object(object)` [¶](https://microsoft.github.io/agent-lightning/0.2.2/reference/agent/#agentlightning.emit_object "Permanent link") Emit an object's serialized representation as an OpenTelemetry span. Parameters: * **`object`** (`Any`) – Data structure to encode as JSON and attach to the span payload. Note The payload must be JSON serializable. Non-serializable objects are ignored and an error is logged to aid debugging. ### `agentlightning.emit_exception(exception)` [¶](https://microsoft.github.io/agent-lightning/0.2.2/reference/agent/#agentlightning.emit_exception "Permanent link") Record an exception with OpenTelemetry metadata. Parameters: * **`exception`** (`BaseException`) – Raised exception instance to serialize into telemetry attributes. Note The helper validates its input. Non-exception values are ignored to prevent noisy telemetry and indicate programming mistakes via the logger. Reward Helpers[¶](https://microsoft.github.io/agent-lightning/0.2.2/reference/agent/#reward-helpers "Permanent link") ---------------------------------------------------------------------------------------------------------------------- ### `agentlightning.find_final_reward(spans)` [¶](https://microsoft.github.io/agent-lightning/0.2.2/reference/agent/#agentlightning.find_final_reward "Permanent link") Return the last reward value present in the provided spans. Parameters: * **`spans`** (`Sequence[[SpanLike](https://microsoft.github.io/agent-lightning/0.2.2/reference/types/#agentlightning.SpanLike " agentlightning.SpanLike = Union[ReadableSpan, Span] module-attribute (agentlightning.types.SpanLike)") ]`) – Sequence containing [`ReadableSpan`](https://opentelemetry.io/docs/concepts/signals/traces/) objects or mocked span-like values. Returns: * `Optional[float]` – Reward value from the latest reward span, or `None` when none are found. ### `agentlightning.find_reward_spans(spans)` [¶](https://microsoft.github.io/agent-lightning/0.2.2/reference/agent/#agentlightning.find_reward_spans "Permanent link") Return all reward spans in the provided sequence. Parameters: * **`spans`** (`Sequence[[SpanLike](https://microsoft.github.io/agent-lightning/0.2.2/reference/types/#agentlightning.SpanLike " agentlightning.SpanLike = Union[ReadableSpan, Span] module-attribute (agentlightning.types.SpanLike)") ]`) – Sequence containing [`ReadableSpan`](https://opentelemetry.io/docs/concepts/signals/traces/) objects or mocked span-like values. Returns: * `List[[SpanLike](https://microsoft.github.io/agent-lightning/0.2.2/reference/types/#agentlightning.SpanLike " agentlightning.SpanLike = Union[ReadableSpan, Span] module-attribute (agentlightning.types.SpanLike)") ]` – List of spans that could be parsed as rewards. ### `agentlightning.get_reward_value(span)` [¶](https://microsoft.github.io/agent-lightning/0.2.2/reference/agent/#agentlightning.get_reward_value "Permanent link") Extract the reward value from a span, if available. Parameters: * **`span`** (`[SpanLike](https://microsoft.github.io/agent-lightning/0.2.2/reference/types/#agentlightning.SpanLike " agentlightning.SpanLike = Union[ReadableSpan, Span] module-attribute (agentlightning.types.SpanLike)") `) – Span object produced by AgentOps or Agent Lightning emitters. Returns: * `Optional[float]` – The reward encoded in the span or `None` when the span does not represent a reward. ### `agentlightning.is_reward_span(span)` [¶](https://microsoft.github.io/agent-lightning/0.2.2/reference/agent/#agentlightning.is_reward_span "Permanent link") Return `True` when the provided span encodes a reward value. Legacy Emitter Decorators[¶](https://microsoft.github.io/agent-lightning/0.2.2/reference/agent/#legacy-emitter-decorators "Permanent link") -------------------------------------------------------------------------------------------------------------------------------------------- ### `agentlightning.reward.reward(fn)` [¶](https://microsoft.github.io/agent-lightning/0.2.2/reference/agent/#agentlightning.reward.reward "Permanent link") Decorate a reward function so its outputs are tracked as spans. The decorator integrates with AgentOps when it is available and falls back to the built-in telemetry otherwise. Both synchronous and asynchronous functions are supported transparently. Deprecated This decorator is deprecated. Use [`emit_reward`](https://microsoft.github.io/agent-lightning/0.2.2/reference/agent/#agentlightning.emit_reward " agentlightning.emit_reward(reward)") instead. Parameters: * **`fn`** (`FnType`) – Callable that produces a numeric reward. Returns: * `FnType` – Wrapped callable that preserves the original signature. Back to top --- # Instrumentation - Agent-lightning [Skip to content](https://microsoft.github.io/agent-lightning/0.2.2/reference/instrumentation/#instrumentation-api) ⚠️ You are viewing development documentation. For stable documentation, please visit [https://microsoft.github.io/agent-lightning/stable/](https://microsoft.github.io/agent-lightning/stable/) Instrumentation API[¶](https://microsoft.github.io/agent-lightning/0.2.2/reference/instrumentation/#instrumentation-api "Permanent link") ========================================================================================================================================== ### `agentlightning.instrumentation.instrument_all()` [¶](https://microsoft.github.io/agent-lightning/0.2.2/reference/instrumentation/#agentlightning.instrumentation.instrument_all "Permanent link") Instrument all the instrumentation libraries. ### `agentlightning.instrumentation.uninstrument_all()` [¶](https://microsoft.github.io/agent-lightning/0.2.2/reference/instrumentation/#agentlightning.instrumentation.uninstrument_all "Permanent link") Uninstrument all the instrumentation libraries. AgentOps LangChain[¶](https://microsoft.github.io/agent-lightning/0.2.2/reference/instrumentation/#agentops-langchain "Permanent link") ---------------------------------------------------------------------------------------------------------------------------------------- ### `agentlightning.instrumentation.agentops_langchain` [¶](https://microsoft.github.io/agent-lightning/0.2.2/reference/instrumentation/#agentlightning.instrumentation.agentops_langchain "Permanent link") #### `instrument_agentops_langchain()` [¶](https://microsoft.github.io/agent-lightning/0.2.2/reference/instrumentation/#agentlightning.instrumentation.agentops_langchain.instrument_agentops_langchain "Permanent link") Bypass AgentOp's native support for Langchain. #### `uninstrument_agentops_langchain()` [¶](https://microsoft.github.io/agent-lightning/0.2.2/reference/instrumentation/#agentlightning.instrumentation.agentops_langchain.uninstrument_agentops_langchain "Permanent link") Restore AgentOp's native support for Langchain. AgentOps[¶](https://microsoft.github.io/agent-lightning/0.2.2/reference/instrumentation/#agentops "Permanent link") -------------------------------------------------------------------------------------------------------------------- ### `agentlightning.instrumentation.agentops` [¶](https://microsoft.github.io/agent-lightning/0.2.2/reference/instrumentation/#agentlightning.instrumentation.agentops "Permanent link") #### `AgentOpsServerManager` [¶](https://microsoft.github.io/agent-lightning/0.2.2/reference/instrumentation/#agentlightning.instrumentation.agentops.AgentOpsServerManager "Permanent link") Manages a AgentOps local server to bypass the online service of AgentOps. #### `agentops_local_server()` [¶](https://microsoft.github.io/agent-lightning/0.2.2/reference/instrumentation/#agentlightning.instrumentation.agentops.agentops_local_server "Permanent link") Returns a Flask app that can be used to test agentops integration. This server provides endpoints for token fetching and a catch-all endpoint. #### `instrument_agentops()` [¶](https://microsoft.github.io/agent-lightning/0.2.2/reference/instrumentation/#agentlightning.instrumentation.agentops.instrument_agentops "Permanent link") Instrument agentops to capture token IDs. Automatically detects and uses the appropriate patching method based on the installed agentops version. #### `uninstrument_agentops()` [¶](https://microsoft.github.io/agent-lightning/0.2.2/reference/instrumentation/#agentlightning.instrumentation.agentops.uninstrument_agentops "Permanent link") Uninstrument agentops to stop capturing token IDs. LiteLLM[¶](https://microsoft.github.io/agent-lightning/0.2.2/reference/instrumentation/#litellm "Permanent link") ------------------------------------------------------------------------------------------------------------------ ### `agentlightning.instrumentation.litellm` [¶](https://microsoft.github.io/agent-lightning/0.2.2/reference/instrumentation/#agentlightning.instrumentation.litellm "Permanent link") LiteLLM instrumentations. It's unclear whether or not this file is useful. It seems that LiteLLM owns its own telemetry from their own entrance [Related documentation](https://docs.litellm.ai/docs/observability/agentops_integration) . #### `instrument_litellm()` [¶](https://microsoft.github.io/agent-lightning/0.2.2/reference/instrumentation/#agentlightning.instrumentation.litellm.instrument_litellm "Permanent link") Instrument litellm to capture token IDs. #### `uninstrument_litellm()` [¶](https://microsoft.github.io/agent-lightning/0.2.2/reference/instrumentation/#agentlightning.instrumentation.litellm.uninstrument_litellm "Permanent link") Uninstrument litellm to stop capturing token IDs. vLLM[¶](https://microsoft.github.io/agent-lightning/0.2.2/reference/instrumentation/#vllm "Permanent link") ------------------------------------------------------------------------------------------------------------ ### `agentlightning.instrumentation.vllm` [¶](https://microsoft.github.io/agent-lightning/0.2.2/reference/instrumentation/#agentlightning.instrumentation.vllm "Permanent link") #### `instrument_vllm()` [¶](https://microsoft.github.io/agent-lightning/0.2.2/reference/instrumentation/#agentlightning.instrumentation.vllm.instrument_vllm "Permanent link") Instrument vLLM to capture token IDs generated by engine. This instrumentation has been merged to upstream vLLM since v0.10.2. #### `uninstrument_vllm()` [¶](https://microsoft.github.io/agent-lightning/0.2.2/reference/instrumentation/#agentlightning.instrumentation.vllm.uninstrument_vllm "Permanent link") Uninstrument vLLM to stop capturing token IDs generated by engine. Back to top --- # Instrumentation - Agent-lightning [Skip to content](https://microsoft.github.io/agent-lightning/0.2.1/reference/instrumentation/#instrumentation-api) ⚠️ You are viewing development documentation. For stable documentation, please visit [https://microsoft.github.io/agent-lightning/stable/](https://microsoft.github.io/agent-lightning/stable/) Instrumentation API[¶](https://microsoft.github.io/agent-lightning/0.2.1/reference/instrumentation/#instrumentation-api "Permanent link") ========================================================================================================================================== ### `agentlightning.instrumentation.instrument_all()` [¶](https://microsoft.github.io/agent-lightning/0.2.1/reference/instrumentation/#agentlightning.instrumentation.instrument_all "Permanent link") Instrument all the instrumentation libraries. ### `agentlightning.instrumentation.uninstrument_all()` [¶](https://microsoft.github.io/agent-lightning/0.2.1/reference/instrumentation/#agentlightning.instrumentation.uninstrument_all "Permanent link") Uninstrument all the instrumentation libraries. AgentOps LangChain[¶](https://microsoft.github.io/agent-lightning/0.2.1/reference/instrumentation/#agentops-langchain "Permanent link") ---------------------------------------------------------------------------------------------------------------------------------------- ### `agentlightning.instrumentation.agentops_langchain` [¶](https://microsoft.github.io/agent-lightning/0.2.1/reference/instrumentation/#agentlightning.instrumentation.agentops_langchain "Permanent link") #### `instrument_agentops_langchain()` [¶](https://microsoft.github.io/agent-lightning/0.2.1/reference/instrumentation/#agentlightning.instrumentation.agentops_langchain.instrument_agentops_langchain "Permanent link") Bypass AgentOp's native support for Langchain. #### `uninstrument_agentops_langchain()` [¶](https://microsoft.github.io/agent-lightning/0.2.1/reference/instrumentation/#agentlightning.instrumentation.agentops_langchain.uninstrument_agentops_langchain "Permanent link") Restore AgentOp's native support for Langchain. AgentOps[¶](https://microsoft.github.io/agent-lightning/0.2.1/reference/instrumentation/#agentops "Permanent link") -------------------------------------------------------------------------------------------------------------------- ### `agentlightning.instrumentation.agentops` [¶](https://microsoft.github.io/agent-lightning/0.2.1/reference/instrumentation/#agentlightning.instrumentation.agentops "Permanent link") #### `AgentOpsServerManager` [¶](https://microsoft.github.io/agent-lightning/0.2.1/reference/instrumentation/#agentlightning.instrumentation.agentops.AgentOpsServerManager "Permanent link") Manages a AgentOps local server to bypass the online service of AgentOps. #### `agentops_local_server()` [¶](https://microsoft.github.io/agent-lightning/0.2.1/reference/instrumentation/#agentlightning.instrumentation.agentops.agentops_local_server "Permanent link") Returns a Flask app that can be used to test agentops integration. This server provides endpoints for token fetching and a catch-all endpoint. #### `instrument_agentops()` [¶](https://microsoft.github.io/agent-lightning/0.2.1/reference/instrumentation/#agentlightning.instrumentation.agentops.instrument_agentops "Permanent link") Instrument agentops to capture token IDs. Automatically detects and uses the appropriate patching method based on the installed agentops version. #### `uninstrument_agentops()` [¶](https://microsoft.github.io/agent-lightning/0.2.1/reference/instrumentation/#agentlightning.instrumentation.agentops.uninstrument_agentops "Permanent link") Uninstrument agentops to stop capturing token IDs. LiteLLM[¶](https://microsoft.github.io/agent-lightning/0.2.1/reference/instrumentation/#litellm "Permanent link") ------------------------------------------------------------------------------------------------------------------ ### `agentlightning.instrumentation.litellm` [¶](https://microsoft.github.io/agent-lightning/0.2.1/reference/instrumentation/#agentlightning.instrumentation.litellm "Permanent link") LiteLLM instrumentations. It's unclear whether or not this file is useful. It seems that LiteLLM owns its own telemetry from their own entrance [Related documentation](https://docs.litellm.ai/docs/observability/agentops_integration) . #### `instrument_litellm()` [¶](https://microsoft.github.io/agent-lightning/0.2.1/reference/instrumentation/#agentlightning.instrumentation.litellm.instrument_litellm "Permanent link") Instrument litellm to capture token IDs. #### `uninstrument_litellm()` [¶](https://microsoft.github.io/agent-lightning/0.2.1/reference/instrumentation/#agentlightning.instrumentation.litellm.uninstrument_litellm "Permanent link") Uninstrument litellm to stop capturing token IDs. vLLM[¶](https://microsoft.github.io/agent-lightning/0.2.1/reference/instrumentation/#vllm "Permanent link") ------------------------------------------------------------------------------------------------------------ ### `agentlightning.instrumentation.vllm` [¶](https://microsoft.github.io/agent-lightning/0.2.1/reference/instrumentation/#agentlightning.instrumentation.vllm "Permanent link") #### `instrument_vllm()` [¶](https://microsoft.github.io/agent-lightning/0.2.1/reference/instrumentation/#agentlightning.instrumentation.vllm.instrument_vllm "Permanent link") Instrument vLLM to capture token IDs generated by engine. This instrumentation has been merged to upstream vLLM since v0.10.2. #### `uninstrument_vllm()` [¶](https://microsoft.github.io/agent-lightning/0.2.1/reference/instrumentation/#agentlightning.instrumentation.vllm.uninstrument_vllm "Permanent link") Uninstrument vLLM to stop capturing token IDs generated by engine. Back to top --- # Agent - Agent-lightning [Skip to content](https://microsoft.github.io/agent-lightning/0.2.1/reference/agent/#agent-developer-apis) ⚠️ You are viewing development documentation. For stable documentation, please visit [https://microsoft.github.io/agent-lightning/stable/](https://microsoft.github.io/agent-lightning/stable/) Agent Developer APIs[¶](https://microsoft.github.io/agent-lightning/0.2.1/reference/agent/#agent-developer-apis "Permanent link") ================================================================================================================================== Agent Decorators[¶](https://microsoft.github.io/agent-lightning/0.2.1/reference/agent/#agent-decorators "Permanent link") -------------------------------------------------------------------------------------------------------------------------- Tip These are convenient helpers for creating agents from functions. First-time users are recommended to use these decorators to create agents. ### `agentlightning.rollout(func)` [¶](https://microsoft.github.io/agent-lightning/0.2.1/reference/agent/#agentlightning.rollout "Permanent link") Create a [`FunctionalLitAgent`](https://microsoft.github.io/agent-lightning/0.2.1/reference/internal/#agentlightning.litagent.decorator.FunctionalLitAgent " agentlightning.litagent.decorator.FunctionalLitAgent") from an arbitrary rollout function. This function inspects the provided callable and creates the appropriate agent type based on its signature. It supports both LLM-based and prompt-template-based agents. The returned agent instance is callable, preserving the original function's behavior and type hints. See [`llm_rollout`](https://microsoft.github.io/agent-lightning/0.2.1/reference/internal/#agentlightning.litagent.decorator.llm_rollout " agentlightning.litagent.decorator.llm_rollout(func=None, *, strip_proxy=True)") and [`prompt_rollout`](https://microsoft.github.io/agent-lightning/0.2.1/reference/internal/#agentlightning.litagent.decorator.prompt_rollout " agentlightning.litagent.decorator.prompt_rollout(func=None)") for more details. Parameters: * **`func`** (`Union[LlmRolloutFunc[T], PromptRolloutFunc[T], Callable[..., Any]]`) – Callable that implements the rollout. Supported signatures: * `[async ](task, llm[, rollout])` for LLM-based agents * `[async ](task, prompt_template[, rollout])` for prompt-template-based agents The supported output types of `func` is same as the return type of [`rollout`](https://microsoft.github.io/agent-lightning/0.2.1/reference/agent/#agentlightning.LitAgent.rollout " rollout(task, resources, rollout)") . Returns: * `[FunctionalLitAgent](https://microsoft.github.io/agent-lightning/0.2.1/reference/internal/#agentlightning.litagent.decorator.FunctionalLitAgent " agentlightning.litagent.decorator.FunctionalLitAgent") [T]` – [`FunctionalLitAgent`](https://microsoft.github.io/agent-lightning/0.2.1/reference/internal/#agentlightning.litagent.decorator.FunctionalLitAgent " agentlightning.litagent.decorator.FunctionalLitAgent") that * `[FunctionalLitAgent](https://microsoft.github.io/agent-lightning/0.2.1/reference/internal/#agentlightning.litagent.decorator.FunctionalLitAgent " agentlightning.litagent.decorator.FunctionalLitAgent") [T]` – wraps the supplied function. Examples: `[](https://microsoft.github.io/agent-lightning/0.2.1/reference/agent/#__codelineno-0-1) # LLM-based agent [](https://microsoft.github.io/agent-lightning/0.2.1/reference/agent/#__codelineno-0-2) @rollout [](https://microsoft.github.io/agent-lightning/0.2.1/reference/agent/#__codelineno-0-3) def my_llm_agent(task, llm): [](https://microsoft.github.io/agent-lightning/0.2.1/reference/agent/#__codelineno-0-4) client = OpenAI(base_url=llm.endpoint) [](https://microsoft.github.io/agent-lightning/0.2.1/reference/agent/#__codelineno-0-5) response = client.chat.completions.create( [](https://microsoft.github.io/agent-lightning/0.2.1/reference/agent/#__codelineno-0-6) model=llm.model, [](https://microsoft.github.io/agent-lightning/0.2.1/reference/agent/#__codelineno-0-7) messages=[{"role": "user", "content": task.input}], [](https://microsoft.github.io/agent-lightning/0.2.1/reference/agent/#__codelineno-0-8) ) [](https://microsoft.github.io/agent-lightning/0.2.1/reference/agent/#__codelineno-0-9) return response [](https://microsoft.github.io/agent-lightning/0.2.1/reference/agent/#__codelineno-0-10) [](https://microsoft.github.io/agent-lightning/0.2.1/reference/agent/#__codelineno-0-11) # Prompt-template-based agent [](https://microsoft.github.io/agent-lightning/0.2.1/reference/agent/#__codelineno-0-12) @rollout [](https://microsoft.github.io/agent-lightning/0.2.1/reference/agent/#__codelineno-0-13) def my_prompt_agent(task, prompt_template): [](https://microsoft.github.io/agent-lightning/0.2.1/reference/agent/#__codelineno-0-14) messages = prompt_template.format(task=task.input) [](https://microsoft.github.io/agent-lightning/0.2.1/reference/agent/#__codelineno-0-15) # ... perform rollout with the formatted prompt [](https://microsoft.github.io/agent-lightning/0.2.1/reference/agent/#__codelineno-0-16) return response [](https://microsoft.github.io/agent-lightning/0.2.1/reference/agent/#__codelineno-0-17) [](https://microsoft.github.io/agent-lightning/0.2.1/reference/agent/#__codelineno-0-18) # Function is still callable with original behavior [](https://microsoft.github.io/agent-lightning/0.2.1/reference/agent/#__codelineno-0-19) result = my_llm_agent(task, llm) [](https://microsoft.github.io/agent-lightning/0.2.1/reference/agent/#__codelineno-0-20) [](https://microsoft.github.io/agent-lightning/0.2.1/reference/agent/#__codelineno-0-21) # Agent methods are also available [](https://microsoft.github.io/agent-lightning/0.2.1/reference/agent/#__codelineno-0-22) result = my_llm_agent.rollout(task, resources, rollout)` Raises: * `NotImplementedError` – If the function signature doesn't match any known patterns. Warning The following two decorators are implementations of [`agentlightning.rollout`](https://microsoft.github.io/agent-lightning/0.2.1/reference/agent/#agentlightning.rollout " agentlightning.rollout(func)") . They are not recommended for new users. ### `agentlightning.llm_rollout(func=None, *, strip_proxy=True)` [¶](https://microsoft.github.io/agent-lightning/0.2.1/reference/agent/#agentlightning.llm_rollout "Permanent link") `[](https://microsoft.github.io/agent-lightning/0.2.1/reference/agent/#__codelineno-0-1) llm_rollout( [](https://microsoft.github.io/agent-lightning/0.2.1/reference/agent/#__codelineno-0-2) func: LlmRolloutFunc[T], [](https://microsoft.github.io/agent-lightning/0.2.1/reference/agent/#__codelineno-0-3) ) -> FunctionalLitAgent[T]` `[](https://microsoft.github.io/agent-lightning/0.2.1/reference/agent/#__codelineno-0-1) llm_rollout( [](https://microsoft.github.io/agent-lightning/0.2.1/reference/agent/#__codelineno-0-2) *, strip_proxy: bool = True [](https://microsoft.github.io/agent-lightning/0.2.1/reference/agent/#__codelineno-0-3) ) -> Callable[[LlmRolloutFunc[T]], FunctionalLitAgent[T]]` Create a [`FunctionalLitAgent`](https://microsoft.github.io/agent-lightning/0.2.1/reference/internal/#agentlightning.litagent.decorator.FunctionalLitAgent " agentlightning.litagent.decorator.FunctionalLitAgent") for LLM-based rollouts. Parameters: * **`func`** (`LlmRolloutFunc[T] | None`, default: `None` ) – Callable defining the agent's behaviour. Supported signatures include: * `(task, llm) -> result` * `(task, llm, rollout) -> result` * `async (task, llm) -> result` * `async (task, llm, rollout) -> result` * **`strip_proxy`** (`bool`, default: `True` ) – When `True`, convert proxy resources into concrete [`LLM`](https://microsoft.github.io/agent-lightning/0.2.1/reference/types/#agentlightning.LLM " agentlightning.LLM") instances before calling the function. Defaults to `True`. Returns: * `[FunctionalLitAgent](https://microsoft.github.io/agent-lightning/0.2.1/reference/internal/#agentlightning.litagent.decorator.FunctionalLitAgent " agentlightning.litagent.decorator.FunctionalLitAgent") [T] | Callable[[LlmRolloutFunc[T]], [FunctionalLitAgent](https://microsoft.github.io/agent-lightning/0.2.1/reference/internal/#agentlightning.litagent.decorator.FunctionalLitAgent " agentlightning.litagent.decorator.FunctionalLitAgent") [T]]` – [`FunctionalLitAgent`](https://microsoft.github.io/agent-lightning/0.2.1/reference/internal/#agentlightning.litagent.decorator.FunctionalLitAgent " agentlightning.litagent.decorator.FunctionalLitAgent") that * `[FunctionalLitAgent](https://microsoft.github.io/agent-lightning/0.2.1/reference/internal/#agentlightning.litagent.decorator.FunctionalLitAgent " agentlightning.litagent.decorator.FunctionalLitAgent") [T] | Callable[[LlmRolloutFunc[T]], [FunctionalLitAgent](https://microsoft.github.io/agent-lightning/0.2.1/reference/internal/#agentlightning.litagent.decorator.FunctionalLitAgent " agentlightning.litagent.decorator.FunctionalLitAgent") [T]]` – wraps the supplied function. Examples: `[](https://microsoft.github.io/agent-lightning/0.2.1/reference/agent/#__codelineno-0-1) @llm_rollout [](https://microsoft.github.io/agent-lightning/0.2.1/reference/agent/#__codelineno-0-2) def my_agent(task, llm): [](https://microsoft.github.io/agent-lightning/0.2.1/reference/agent/#__codelineno-0-3) return llm.endpoint [](https://microsoft.github.io/agent-lightning/0.2.1/reference/agent/#__codelineno-0-4) [](https://microsoft.github.io/agent-lightning/0.2.1/reference/agent/#__codelineno-0-5) @llm_rollout(strip_proxy=False) [](https://microsoft.github.io/agent-lightning/0.2.1/reference/agent/#__codelineno-0-6) def my_agent_no_strip(task, llm): [](https://microsoft.github.io/agent-lightning/0.2.1/reference/agent/#__codelineno-0-7) return llm.model [](https://microsoft.github.io/agent-lightning/0.2.1/reference/agent/#__codelineno-0-8) [](https://microsoft.github.io/agent-lightning/0.2.1/reference/agent/#__codelineno-0-9) result = my_agent(task, llm) [](https://microsoft.github.io/agent-lightning/0.2.1/reference/agent/#__codelineno-0-10) result = my_agent.rollout(task, resources, rollout)` ### `agentlightning.prompt_rollout(func=None)` [¶](https://microsoft.github.io/agent-lightning/0.2.1/reference/agent/#agentlightning.prompt_rollout "Permanent link") `[](https://microsoft.github.io/agent-lightning/0.2.1/reference/agent/#__codelineno-0-1) prompt_rollout( [](https://microsoft.github.io/agent-lightning/0.2.1/reference/agent/#__codelineno-0-2) func: PromptRolloutFunc[T], [](https://microsoft.github.io/agent-lightning/0.2.1/reference/agent/#__codelineno-0-3) ) -> FunctionalLitAgent[T]` `[](https://microsoft.github.io/agent-lightning/0.2.1/reference/agent/#__codelineno-0-1) prompt_rollout() -> ( [](https://microsoft.github.io/agent-lightning/0.2.1/reference/agent/#__codelineno-0-2) Callable[[PromptRolloutFunc[T]], FunctionalLitAgent[T]] [](https://microsoft.github.io/agent-lightning/0.2.1/reference/agent/#__codelineno-0-3) )` Create a [`FunctionalLitAgent`](https://microsoft.github.io/agent-lightning/0.2.1/reference/internal/#agentlightning.litagent.decorator.FunctionalLitAgent " agentlightning.litagent.decorator.FunctionalLitAgent") for prompt-based rollouts. This decorator is designed for agents that work with tunable prompt templates. It enables a workflow where algorithms manage and optimize the prompt template, while agents consume the template to perform rollouts. This is particularly useful for prompt optimization scenarios. Parameters: * **`func`** (`PromptRolloutFunc[T] | None`, default: `None` ) – Callable defining the agent's behavior. Supported signatures include: * `(task, prompt_template) -> result` * `(task, prompt_template, rollout) -> result` * `async (task, prompt_template) -> result` * `async (task, prompt_template, rollout) -> result` Returns: * `[FunctionalLitAgent](https://microsoft.github.io/agent-lightning/0.2.1/reference/internal/#agentlightning.litagent.decorator.FunctionalLitAgent " agentlightning.litagent.decorator.FunctionalLitAgent") [T] | Callable[[PromptRolloutFunc[T]], [FunctionalLitAgent](https://microsoft.github.io/agent-lightning/0.2.1/reference/internal/#agentlightning.litagent.decorator.FunctionalLitAgent " agentlightning.litagent.decorator.FunctionalLitAgent") [T]]` – [`FunctionalLitAgent`](https://microsoft.github.io/agent-lightning/0.2.1/reference/internal/#agentlightning.litagent.decorator.FunctionalLitAgent " agentlightning.litagent.decorator.FunctionalLitAgent") that * `[FunctionalLitAgent](https://microsoft.github.io/agent-lightning/0.2.1/reference/internal/#agentlightning.litagent.decorator.FunctionalLitAgent " agentlightning.litagent.decorator.FunctionalLitAgent") [T] | Callable[[PromptRolloutFunc[T]], [FunctionalLitAgent](https://microsoft.github.io/agent-lightning/0.2.1/reference/internal/#agentlightning.litagent.decorator.FunctionalLitAgent " agentlightning.litagent.decorator.FunctionalLitAgent") [T]]` – wraps the supplied function. Examples: `[](https://microsoft.github.io/agent-lightning/0.2.1/reference/agent/#__codelineno-0-1) @prompt_rollout [](https://microsoft.github.io/agent-lightning/0.2.1/reference/agent/#__codelineno-0-2) def my_agent(task, prompt_template): [](https://microsoft.github.io/agent-lightning/0.2.1/reference/agent/#__codelineno-0-3) messages = prompt_template.format(task=task.input) [](https://microsoft.github.io/agent-lightning/0.2.1/reference/agent/#__codelineno-0-4) return messages [](https://microsoft.github.io/agent-lightning/0.2.1/reference/agent/#__codelineno-0-5) [](https://microsoft.github.io/agent-lightning/0.2.1/reference/agent/#__codelineno-0-6) result = my_agent(task, prompt_template) [](https://microsoft.github.io/agent-lightning/0.2.1/reference/agent/#__codelineno-0-7) result = my_agent.rollout(task, resources, rollout)` Class-based Agents[¶](https://microsoft.github.io/agent-lightning/0.2.1/reference/agent/#class-based-agents "Permanent link") ------------------------------------------------------------------------------------------------------------------------------ ### `agentlightning.LitAgent` [¶](https://microsoft.github.io/agent-lightning/0.2.1/reference/agent/#agentlightning.LitAgent "Permanent link") Bases: `Generic[T]` Base class for implementing agent rollouts. Subclasses override the rollout methods to process tasks while the trainer and runner infrastructure manages orchestration, tracing, and persistence. #### `runner` `property` [¶](https://microsoft.github.io/agent-lightning/0.2.1/reference/agent/#agentlightning.LitAgent.runner "Permanent link") Return the runner responsible for executing rollouts. #### `tracer` `property` [¶](https://microsoft.github.io/agent-lightning/0.2.1/reference/agent/#agentlightning.LitAgent.tracer "Permanent link") Return the tracer configured for this agent. #### `trainer` `property` [¶](https://microsoft.github.io/agent-lightning/0.2.1/reference/agent/#agentlightning.LitAgent.trainer "Permanent link") Return the trainer associated with this agent. #### `__init__(*, trained_agents=None)` [¶](https://microsoft.github.io/agent-lightning/0.2.1/reference/agent/#agentlightning.LitAgent.__init__ "Permanent link") Initialize the agent instance. Parameters: * **`trained_agents`** (`Optional[str]`, default: `None` ) – Optional identifier used by legacy tooling to mark trained agents. Deprecated The `trained_agents` flag is deprecated. Configure `agent_match` in the adapter layer instead. See [`TracerTraceToTriplet`](https://microsoft.github.io/agent-lightning/0.2.1/reference/algorithm/#agentlightning.TracerTraceToTriplet " agentlightning.TracerTraceToTriplet") for more details. #### `get_runner()` [¶](https://microsoft.github.io/agent-lightning/0.2.1/reference/agent/#agentlightning.LitAgent.get_runner "Permanent link") Return the runner responsible for executing rollouts. #### `get_tracer()` [¶](https://microsoft.github.io/agent-lightning/0.2.1/reference/agent/#agentlightning.LitAgent.get_tracer "Permanent link") Return the tracer configured for this agent. #### `get_trainer()` [¶](https://microsoft.github.io/agent-lightning/0.2.1/reference/agent/#agentlightning.LitAgent.get_trainer "Permanent link") Return the trainer associated with this agent. #### `is_async()` [¶](https://microsoft.github.io/agent-lightning/0.2.1/reference/agent/#agentlightning.LitAgent.is_async "Permanent link") Return `True` when the agent overrides any asynchronous rollout methods. Override this method for customized async detection logic. #### `on_rollout_end(task, rollout, runner, tracer)` [¶](https://microsoft.github.io/agent-lightning/0.2.1/reference/agent/#agentlightning.LitAgent.on_rollout_end "Permanent link") Hook invoked after a rollout completes. Subclasses can override this method for cleanup or additional logging. The default implementation is a no-op. Parameters: * **`task`** (`[Task](https://microsoft.github.io/agent-lightning/0.2.1/reference/internal/#agentlightning.Task " agentlightning.Task (agentlightning.types.Task)") `) – [`Task`](https://microsoft.github.io/agent-lightning/0.2.1/reference/internal/#agentlightning.Task " agentlightning.Task") that was processed. * **`rollout`** (`[Rollout](https://microsoft.github.io/agent-lightning/0.2.1/reference/types/#agentlightning.Rollout " agentlightning.Rollout (agentlightning.types.Rollout)") `) – Resulting [`Rollout`](https://microsoft.github.io/agent-lightning/0.2.1/reference/types/#agentlightning.Rollout " agentlightning.Rollout") . * **`runner`** (`[Runner](https://microsoft.github.io/agent-lightning/0.2.1/reference/runner/#agentlightning.Runner " agentlightning.Runner (agentlightning.runner.Runner)") [T]`) – [`Runner`](https://microsoft.github.io/agent-lightning/0.2.1/reference/runner/#agentlightning.Runner " agentlightning.Runner") managing the rollout. * **`tracer`** (`[Tracer](https://microsoft.github.io/agent-lightning/0.2.1/reference/runner/#agentlightning.Tracer " agentlightning.Tracer (agentlightning.tracer.Tracer)") `) – [`Tracer`](https://microsoft.github.io/agent-lightning/0.2.1/reference/runner/#agentlightning.Tracer " agentlightning.Tracer") associated with the runner. Deprecated Override [`Hook.on_rollout_end`](https://microsoft.github.io/agent-lightning/0.2.1/reference/types/#agentlightning.Hook.on_rollout_end " on_rollout_end(*, agent, runner, rollout, spans) async ") instead of this method when extending agents. #### `on_rollout_start(task, runner, tracer)` [¶](https://microsoft.github.io/agent-lightning/0.2.1/reference/agent/#agentlightning.LitAgent.on_rollout_start "Permanent link") Hook invoked immediately before a rollout begins. Subclasses can override this method to implement custom logic such as logging, metric collection, or resource setup. The default implementation is a no-op. Parameters: * **`task`** (`[Task](https://microsoft.github.io/agent-lightning/0.2.1/reference/internal/#agentlightning.Task " agentlightning.Task (agentlightning.types.Task)") `) – [`Task`](https://microsoft.github.io/agent-lightning/0.2.1/reference/internal/#agentlightning.Task " agentlightning.Task") that will be processed. * **`runner`** (`[Runner](https://microsoft.github.io/agent-lightning/0.2.1/reference/runner/#agentlightning.Runner " agentlightning.Runner (agentlightning.runner.Runner)") [T]`) – [`Runner`](https://microsoft.github.io/agent-lightning/0.2.1/reference/runner/#agentlightning.Runner " agentlightning.Runner") managing the rollout. * **`tracer`** (`[Tracer](https://microsoft.github.io/agent-lightning/0.2.1/reference/runner/#agentlightning.Tracer " agentlightning.Tracer (agentlightning.tracer.Tracer)") `) – [`Tracer`](https://microsoft.github.io/agent-lightning/0.2.1/reference/runner/#agentlightning.Tracer " agentlightning.Tracer") associated with the runner. Deprecated Override [`Hook.on_rollout_start`](https://microsoft.github.io/agent-lightning/0.2.1/reference/types/#agentlightning.Hook.on_rollout_start " on_rollout_start(*, agent, runner, rollout) async ") instead of this method when extending agents. #### `rollout(task, resources, rollout)` [¶](https://microsoft.github.io/agent-lightning/0.2.1/reference/agent/#agentlightning.LitAgent.rollout "Permanent link") Execute a rollout synchronously. If you don't wish to implement both training rollout and validation rollout separately, you can just implement `rollout` which will work for both. Parameters: * **`task`** (`T`) – Task payload provided by the scheduler. * **`resources`** (`[NamedResources](https://microsoft.github.io/agent-lightning/0.2.1/reference/types/#agentlightning.NamedResources " agentlightning.NamedResources = Dict[str, ResourceUnion] module-attribute (agentlightning.types.NamedResources)") `) – Mapping of named resources (for example LLMs or prompt templates). * **`rollout`** (`[Rollout](https://microsoft.github.io/agent-lightning/0.2.1/reference/types/#agentlightning.Rollout " agentlightning.Rollout (agentlightning.types.Rollout)") `) – Rollout metadata. Avoid mutating this object directly unless a subclass needs to override defaults. Returns: * `[RolloutRawResult](https://microsoft.github.io/agent-lightning/0.2.1/reference/types/#agentlightning.RolloutRawResult " agentlightning.RolloutRawResult = Union[None, float, List[ReadableSpan], List[Span]] module-attribute (agentlightning.types.RolloutRawResult)") ` – One of the following values: * `[RolloutRawResult](https://microsoft.github.io/agent-lightning/0.2.1/reference/types/#agentlightning.RolloutRawResult " agentlightning.RolloutRawResult = Union[None, float, List[ReadableSpan], List[Span]] module-attribute (agentlightning.types.RolloutRawResult)") ` – * `None` when tracing is handled by the runner. * `[RolloutRawResult](https://microsoft.github.io/agent-lightning/0.2.1/reference/types/#agentlightning.RolloutRawResult " agentlightning.RolloutRawResult = Union[None, float, List[ReadableSpan], List[Span]] module-attribute (agentlightning.types.RolloutRawResult)") ` – * `float` representing the final reward. * `[RolloutRawResult](https://microsoft.github.io/agent-lightning/0.2.1/reference/types/#agentlightning.RolloutRawResult " agentlightning.RolloutRawResult = Union[None, float, List[ReadableSpan], List[Span]] module-attribute (agentlightning.types.RolloutRawResult)") ` – * `List[ReadableSpan]` with OpenTelemetry spans. * `[RolloutRawResult](https://microsoft.github.io/agent-lightning/0.2.1/reference/types/#agentlightning.RolloutRawResult " agentlightning.RolloutRawResult = Union[None, float, List[ReadableSpan], List[Span]] module-attribute (agentlightning.types.RolloutRawResult)") ` – * `List[Span]` with Agent Lightning spans. #### `rollout_async(task, resources, rollout)` `async` [¶](https://microsoft.github.io/agent-lightning/0.2.1/reference/agent/#agentlightning.LitAgent.rollout_async "Permanent link") Execute a rollout asynchronously. Parameters: * **`task`** (`T`) – Task payload provided by the scheduler. * **`resources`** (`[NamedResources](https://microsoft.github.io/agent-lightning/0.2.1/reference/types/#agentlightning.NamedResources " agentlightning.NamedResources = Dict[str, ResourceUnion] module-attribute (agentlightning.types.NamedResources)") `) – Mapping of named resources (for example LLMs or prompt templates). * **`rollout`** (`[Rollout](https://microsoft.github.io/agent-lightning/0.2.1/reference/types/#agentlightning.Rollout " agentlightning.Rollout (agentlightning.types.Rollout)") `) – Rollout metadata. Avoid mutating this object directly unless a subclass needs to override defaults. Returns: * `[RolloutRawResult](https://microsoft.github.io/agent-lightning/0.2.1/reference/types/#agentlightning.RolloutRawResult " agentlightning.RolloutRawResult = Union[None, float, List[ReadableSpan], List[Span]] module-attribute (agentlightning.types.RolloutRawResult)") ` – Same possible return values as * `[RolloutRawResult](https://microsoft.github.io/agent-lightning/0.2.1/reference/types/#agentlightning.RolloutRawResult " agentlightning.RolloutRawResult = Union[None, float, List[ReadableSpan], List[Span]] module-attribute (agentlightning.types.RolloutRawResult)") ` – [`rollout`](https://microsoft.github.io/agent-lightning/0.2.1/reference/agent/#agentlightning.LitAgent.rollout " rollout(task, resources, rollout)") . #### `set_runner(runner)` [¶](https://microsoft.github.io/agent-lightning/0.2.1/reference/agent/#agentlightning.LitAgent.set_runner "Permanent link") Attach the runner responsible for executing rollouts. Parameters: * **`runner`** (`[Runner](https://microsoft.github.io/agent-lightning/0.2.1/reference/runner/#agentlightning.Runner " agentlightning.Runner (agentlightning.runner.Runner)") [T]`) – [`Runner`](https://microsoft.github.io/agent-lightning/0.2.1/reference/runner/#agentlightning.Runner " agentlightning.Runner") coordinating execution. #### `set_trainer(trainer)` [¶](https://microsoft.github.io/agent-lightning/0.2.1/reference/agent/#agentlightning.LitAgent.set_trainer "Permanent link") Attach the trainer responsible for orchestration. Parameters: * **`trainer`** (`[Trainer](https://microsoft.github.io/agent-lightning/0.2.1/reference/trainer/#agentlightning.Trainer " agentlightning.Trainer (agentlightning.trainer.Trainer)") `) – [`Trainer`](https://microsoft.github.io/agent-lightning/0.2.1/reference/trainer/#agentlightning.Trainer " agentlightning.Trainer") that manages the agent. #### `training_rollout(task, resources, rollout)` [¶](https://microsoft.github.io/agent-lightning/0.2.1/reference/agent/#agentlightning.LitAgent.training_rollout "Permanent link") Process a single training task synchronously. By default, this method delegates to [`rollout`](https://microsoft.github.io/agent-lightning/0.2.1/reference/agent/#agentlightning.LitAgent.rollout " rollout(task, resources, rollout)") . #### `training_rollout_async(task, resources, rollout)` `async` [¶](https://microsoft.github.io/agent-lightning/0.2.1/reference/agent/#agentlightning.LitAgent.training_rollout_async "Permanent link") Process a single training task asynchronously. By default, this method delegates to [`rollout_async`](https://microsoft.github.io/agent-lightning/0.2.1/reference/agent/#agentlightning.LitAgent.rollout_async " rollout_async(task, resources, rollout) async ") . #### `validation_rollout(task, resources, rollout)` [¶](https://microsoft.github.io/agent-lightning/0.2.1/reference/agent/#agentlightning.LitAgent.validation_rollout "Permanent link") Process a single validation task synchronously. Override this method when validation should differ from training. The default implementation delegates to [`training_rollout`](https://microsoft.github.io/agent-lightning/0.2.1/reference/agent/#agentlightning.LitAgent.training_rollout " training_rollout(task, resources, rollout)") . #### `validation_rollout_async(task, resources, rollout)` `async` [¶](https://microsoft.github.io/agent-lightning/0.2.1/reference/agent/#agentlightning.LitAgent.validation_rollout_async "Permanent link") Process a single validation task asynchronously. Override this method when validation should differ from training. The default implementation delegates to [`training_rollout_async`](https://microsoft.github.io/agent-lightning/0.2.1/reference/agent/#agentlightning.LitAgent.training_rollout_async " training_rollout_async(task, resources, rollout) async ") . Emitter[¶](https://microsoft.github.io/agent-lightning/0.2.1/reference/agent/#emitter "Permanent link") -------------------------------------------------------------------------------------------------------- ### `agentlightning.emit_reward(reward)` [¶](https://microsoft.github.io/agent-lightning/0.2.1/reference/agent/#agentlightning.emit_reward "Permanent link") Emit a reward value as an OpenTelemetry span. Parameters: * **`reward`** (`float`) – Numeric reward to record. Integers and booleans are converted to floating point numbers for consistency. Returns: * `ReadableSpan` – Readable span capturing the recorded reward. Raises: * `ValueError` – If the provided reward cannot be interpreted as a float or the resulting span is not a [`ReadableSpan`](https://opentelemetry.io/docs/concepts/signals/traces/) instance. ### `agentlightning.emit_message(message)` [¶](https://microsoft.github.io/agent-lightning/0.2.1/reference/agent/#agentlightning.emit_message "Permanent link") Emit a textual message as an OpenTelemetry span. Parameters: * **`message`** (`str`) – Human readable message to attach as a span attribute. Note OpenTelemetry distinguishes between logs and spans. Emitting the message as a span keeps all Agent Lightning telemetry in a single data store for analysis. ### `agentlightning.emit_object(object)` [¶](https://microsoft.github.io/agent-lightning/0.2.1/reference/agent/#agentlightning.emit_object "Permanent link") Emit an object's serialized representation as an OpenTelemetry span. Parameters: * **`object`** (`Any`) – Data structure to encode as JSON and attach to the span payload. Note The payload must be JSON serializable. Non-serializable objects are ignored and an error is logged to aid debugging. ### `agentlightning.emit_exception(exception)` [¶](https://microsoft.github.io/agent-lightning/0.2.1/reference/agent/#agentlightning.emit_exception "Permanent link") Record an exception with OpenTelemetry metadata. Parameters: * **`exception`** (`BaseException`) – Raised exception instance to serialize into telemetry attributes. Note The helper validates its input. Non-exception values are ignored to prevent noisy telemetry and indicate programming mistakes via the logger. Reward Helpers[¶](https://microsoft.github.io/agent-lightning/0.2.1/reference/agent/#reward-helpers "Permanent link") ---------------------------------------------------------------------------------------------------------------------- ### `agentlightning.find_final_reward(spans)` [¶](https://microsoft.github.io/agent-lightning/0.2.1/reference/agent/#agentlightning.find_final_reward "Permanent link") Return the last reward value present in the provided spans. Parameters: * **`spans`** (`Sequence[[SpanLike](https://microsoft.github.io/agent-lightning/0.2.1/reference/types/#agentlightning.SpanLike " agentlightning.SpanLike = Union[ReadableSpan, Span] module-attribute (agentlightning.types.SpanLike)") ]`) – Sequence containing [`ReadableSpan`](https://opentelemetry.io/docs/concepts/signals/traces/) objects or mocked span-like values. Returns: * `Optional[float]` – Reward value from the latest reward span, or `None` when none are found. ### `agentlightning.find_reward_spans(spans)` [¶](https://microsoft.github.io/agent-lightning/0.2.1/reference/agent/#agentlightning.find_reward_spans "Permanent link") Return all reward spans in the provided sequence. Parameters: * **`spans`** (`Sequence[[SpanLike](https://microsoft.github.io/agent-lightning/0.2.1/reference/types/#agentlightning.SpanLike " agentlightning.SpanLike = Union[ReadableSpan, Span] module-attribute (agentlightning.types.SpanLike)") ]`) – Sequence containing [`ReadableSpan`](https://opentelemetry.io/docs/concepts/signals/traces/) objects or mocked span-like values. Returns: * `List[[SpanLike](https://microsoft.github.io/agent-lightning/0.2.1/reference/types/#agentlightning.SpanLike " agentlightning.SpanLike = Union[ReadableSpan, Span] module-attribute (agentlightning.types.SpanLike)") ]` – List of spans that could be parsed as rewards. ### `agentlightning.get_reward_value(span)` [¶](https://microsoft.github.io/agent-lightning/0.2.1/reference/agent/#agentlightning.get_reward_value "Permanent link") Extract the reward value from a span, if available. Parameters: * **`span`** (`[SpanLike](https://microsoft.github.io/agent-lightning/0.2.1/reference/types/#agentlightning.SpanLike " agentlightning.SpanLike = Union[ReadableSpan, Span] module-attribute (agentlightning.types.SpanLike)") `) – Span object produced by AgentOps or Agent Lightning emitters. Returns: * `Optional[float]` – The reward encoded in the span or `None` when the span does not represent a reward. ### `agentlightning.is_reward_span(span)` [¶](https://microsoft.github.io/agent-lightning/0.2.1/reference/agent/#agentlightning.is_reward_span "Permanent link") Return `True` when the provided span encodes a reward value. Legacy Emitter Decorators[¶](https://microsoft.github.io/agent-lightning/0.2.1/reference/agent/#legacy-emitter-decorators "Permanent link") -------------------------------------------------------------------------------------------------------------------------------------------- ### `agentlightning.reward.reward(fn)` [¶](https://microsoft.github.io/agent-lightning/0.2.1/reference/agent/#agentlightning.reward.reward "Permanent link") Decorate a reward function so its outputs are tracked as spans. The decorator integrates with AgentOps when it is available and falls back to the built-in telemetry otherwise. Both synchronous and asynchronous functions are supported transparently. Deprecated This decorator is deprecated. Use [`emit_reward`](https://microsoft.github.io/agent-lightning/0.2.1/reference/agent/#agentlightning.emit_reward " agentlightning.emit_reward(reward)") instead. Parameters: * **`fn`** (`FnType`) – Callable that produces a numeric reward. Returns: * `FnType` – Wrapped callable that preserves the original signature. Back to top --- # Agent - Agent-lightning [Skip to content](https://microsoft.github.io/agent-lightning/0.2.0/reference/agent/#agent-developer-apis) ⚠️ You are viewing development documentation. For stable documentation, please visit [https://microsoft.github.io/agent-lightning/stable/](https://microsoft.github.io/agent-lightning/stable/) Agent Developer APIs[¶](https://microsoft.github.io/agent-lightning/0.2.0/reference/agent/#agent-developer-apis "Permanent link") ================================================================================================================================== Agent Decorators[¶](https://microsoft.github.io/agent-lightning/0.2.0/reference/agent/#agent-decorators "Permanent link") -------------------------------------------------------------------------------------------------------------------------- Tip These are convenient helpers for creating agents from functions. First-time users are recommended to use these decorators to create agents. ### `agentlightning.rollout(func)` [¶](https://microsoft.github.io/agent-lightning/0.2.0/reference/agent/#agentlightning.rollout "Permanent link") Create a [`FunctionalLitAgent`](https://microsoft.github.io/agent-lightning/0.2.0/reference/internal/#agentlightning.litagent.decorator.FunctionalLitAgent " agentlightning.litagent.decorator.FunctionalLitAgent") from an arbitrary rollout function. This function inspects the provided callable and creates the appropriate agent type based on its signature. It supports both LLM-based and prompt-template-based agents. The returned agent instance is callable, preserving the original function's behavior and type hints. See [`llm_rollout`](https://microsoft.github.io/agent-lightning/0.2.0/reference/internal/#agentlightning.litagent.decorator.llm_rollout " agentlightning.litagent.decorator.llm_rollout(func=None, *, strip_proxy=True)") and [`prompt_rollout`](https://microsoft.github.io/agent-lightning/0.2.0/reference/internal/#agentlightning.litagent.decorator.prompt_rollout " agentlightning.litagent.decorator.prompt_rollout(func=None)") for more details. Parameters: * **`func`** (`Union[LlmRolloutFunc[T], PromptRolloutFunc[T], Callable[..., Any]]`) – Callable that implements the rollout. Supported signatures: * `[async ](task, llm[, rollout])` for LLM-based agents * `[async ](task, prompt_template[, rollout])` for prompt-template-based agents The supported output types of `func` is same as the return type of [`rollout`](https://microsoft.github.io/agent-lightning/0.2.0/reference/agent/#agentlightning.LitAgent.rollout " rollout(task, resources, rollout)") . Returns: * `[FunctionalLitAgent](https://microsoft.github.io/agent-lightning/0.2.0/reference/internal/#agentlightning.litagent.decorator.FunctionalLitAgent " agentlightning.litagent.decorator.FunctionalLitAgent") [T]` – [`FunctionalLitAgent`](https://microsoft.github.io/agent-lightning/0.2.0/reference/internal/#agentlightning.litagent.decorator.FunctionalLitAgent " agentlightning.litagent.decorator.FunctionalLitAgent") that * `[FunctionalLitAgent](https://microsoft.github.io/agent-lightning/0.2.0/reference/internal/#agentlightning.litagent.decorator.FunctionalLitAgent " agentlightning.litagent.decorator.FunctionalLitAgent") [T]` – wraps the supplied function. Examples: `[](https://microsoft.github.io/agent-lightning/0.2.0/reference/agent/#__codelineno-0-1) # LLM-based agent [](https://microsoft.github.io/agent-lightning/0.2.0/reference/agent/#__codelineno-0-2) @rollout [](https://microsoft.github.io/agent-lightning/0.2.0/reference/agent/#__codelineno-0-3) def my_llm_agent(task, llm): [](https://microsoft.github.io/agent-lightning/0.2.0/reference/agent/#__codelineno-0-4) client = OpenAI(base_url=llm.endpoint) [](https://microsoft.github.io/agent-lightning/0.2.0/reference/agent/#__codelineno-0-5) response = client.chat.completions.create( [](https://microsoft.github.io/agent-lightning/0.2.0/reference/agent/#__codelineno-0-6) model=llm.model, [](https://microsoft.github.io/agent-lightning/0.2.0/reference/agent/#__codelineno-0-7) messages=[{"role": "user", "content": task.input}], [](https://microsoft.github.io/agent-lightning/0.2.0/reference/agent/#__codelineno-0-8) ) [](https://microsoft.github.io/agent-lightning/0.2.0/reference/agent/#__codelineno-0-9) return response [](https://microsoft.github.io/agent-lightning/0.2.0/reference/agent/#__codelineno-0-10) [](https://microsoft.github.io/agent-lightning/0.2.0/reference/agent/#__codelineno-0-11) # Prompt-template-based agent [](https://microsoft.github.io/agent-lightning/0.2.0/reference/agent/#__codelineno-0-12) @rollout [](https://microsoft.github.io/agent-lightning/0.2.0/reference/agent/#__codelineno-0-13) def my_prompt_agent(task, prompt_template): [](https://microsoft.github.io/agent-lightning/0.2.0/reference/agent/#__codelineno-0-14) messages = prompt_template.format(task=task.input) [](https://microsoft.github.io/agent-lightning/0.2.0/reference/agent/#__codelineno-0-15) # ... perform rollout with the formatted prompt [](https://microsoft.github.io/agent-lightning/0.2.0/reference/agent/#__codelineno-0-16) return response [](https://microsoft.github.io/agent-lightning/0.2.0/reference/agent/#__codelineno-0-17) [](https://microsoft.github.io/agent-lightning/0.2.0/reference/agent/#__codelineno-0-18) # Function is still callable with original behavior [](https://microsoft.github.io/agent-lightning/0.2.0/reference/agent/#__codelineno-0-19) result = my_llm_agent(task, llm) [](https://microsoft.github.io/agent-lightning/0.2.0/reference/agent/#__codelineno-0-20) [](https://microsoft.github.io/agent-lightning/0.2.0/reference/agent/#__codelineno-0-21) # Agent methods are also available [](https://microsoft.github.io/agent-lightning/0.2.0/reference/agent/#__codelineno-0-22) result = my_llm_agent.rollout(task, resources, rollout)` Raises: * `NotImplementedError` – If the function signature doesn't match any known patterns. Warning The following two decorators are implementations of [`agentlightning.rollout`](https://microsoft.github.io/agent-lightning/0.2.0/reference/agent/#agentlightning.rollout " agentlightning.rollout(func)") . They are not recommended for new users. ### `agentlightning.llm_rollout(func=None, *, strip_proxy=True)` [¶](https://microsoft.github.io/agent-lightning/0.2.0/reference/agent/#agentlightning.llm_rollout "Permanent link") `[](https://microsoft.github.io/agent-lightning/0.2.0/reference/agent/#__codelineno-0-1) llm_rollout( [](https://microsoft.github.io/agent-lightning/0.2.0/reference/agent/#__codelineno-0-2) func: LlmRolloutFunc[T], [](https://microsoft.github.io/agent-lightning/0.2.0/reference/agent/#__codelineno-0-3) ) -> FunctionalLitAgent[T]` `[](https://microsoft.github.io/agent-lightning/0.2.0/reference/agent/#__codelineno-0-1) llm_rollout( [](https://microsoft.github.io/agent-lightning/0.2.0/reference/agent/#__codelineno-0-2) *, strip_proxy: bool = True [](https://microsoft.github.io/agent-lightning/0.2.0/reference/agent/#__codelineno-0-3) ) -> Callable[[LlmRolloutFunc[T]], FunctionalLitAgent[T]]` Create a [`FunctionalLitAgent`](https://microsoft.github.io/agent-lightning/0.2.0/reference/internal/#agentlightning.litagent.decorator.FunctionalLitAgent " agentlightning.litagent.decorator.FunctionalLitAgent") for LLM-based rollouts. Parameters: * **`func`** (`LlmRolloutFunc[T] | None`, default: `None` ) – Callable defining the agent's behaviour. Supported signatures include: * `(task, llm) -> result` * `(task, llm, rollout) -> result` * `async (task, llm) -> result` * `async (task, llm, rollout) -> result` * **`strip_proxy`** (`bool`, default: `True` ) – When `True`, convert proxy resources into concrete [`LLM`](https://microsoft.github.io/agent-lightning/0.2.0/reference/types/#agentlightning.LLM " agentlightning.LLM") instances before calling the function. Defaults to `True`. Returns: * `[FunctionalLitAgent](https://microsoft.github.io/agent-lightning/0.2.0/reference/internal/#agentlightning.litagent.decorator.FunctionalLitAgent " agentlightning.litagent.decorator.FunctionalLitAgent") [T] | Callable[[LlmRolloutFunc[T]], [FunctionalLitAgent](https://microsoft.github.io/agent-lightning/0.2.0/reference/internal/#agentlightning.litagent.decorator.FunctionalLitAgent " agentlightning.litagent.decorator.FunctionalLitAgent") [T]]` – [`FunctionalLitAgent`](https://microsoft.github.io/agent-lightning/0.2.0/reference/internal/#agentlightning.litagent.decorator.FunctionalLitAgent " agentlightning.litagent.decorator.FunctionalLitAgent") that * `[FunctionalLitAgent](https://microsoft.github.io/agent-lightning/0.2.0/reference/internal/#agentlightning.litagent.decorator.FunctionalLitAgent " agentlightning.litagent.decorator.FunctionalLitAgent") [T] | Callable[[LlmRolloutFunc[T]], [FunctionalLitAgent](https://microsoft.github.io/agent-lightning/0.2.0/reference/internal/#agentlightning.litagent.decorator.FunctionalLitAgent " agentlightning.litagent.decorator.FunctionalLitAgent") [T]]` – wraps the supplied function. Examples: `[](https://microsoft.github.io/agent-lightning/0.2.0/reference/agent/#__codelineno-0-1) @llm_rollout [](https://microsoft.github.io/agent-lightning/0.2.0/reference/agent/#__codelineno-0-2) def my_agent(task, llm): [](https://microsoft.github.io/agent-lightning/0.2.0/reference/agent/#__codelineno-0-3) return llm.endpoint [](https://microsoft.github.io/agent-lightning/0.2.0/reference/agent/#__codelineno-0-4) [](https://microsoft.github.io/agent-lightning/0.2.0/reference/agent/#__codelineno-0-5) @llm_rollout(strip_proxy=False) [](https://microsoft.github.io/agent-lightning/0.2.0/reference/agent/#__codelineno-0-6) def my_agent_no_strip(task, llm): [](https://microsoft.github.io/agent-lightning/0.2.0/reference/agent/#__codelineno-0-7) return llm.model [](https://microsoft.github.io/agent-lightning/0.2.0/reference/agent/#__codelineno-0-8) [](https://microsoft.github.io/agent-lightning/0.2.0/reference/agent/#__codelineno-0-9) result = my_agent(task, llm) [](https://microsoft.github.io/agent-lightning/0.2.0/reference/agent/#__codelineno-0-10) result = my_agent.rollout(task, resources, rollout)` ### `agentlightning.prompt_rollout(func=None)` [¶](https://microsoft.github.io/agent-lightning/0.2.0/reference/agent/#agentlightning.prompt_rollout "Permanent link") `[](https://microsoft.github.io/agent-lightning/0.2.0/reference/agent/#__codelineno-0-1) prompt_rollout( [](https://microsoft.github.io/agent-lightning/0.2.0/reference/agent/#__codelineno-0-2) func: PromptRolloutFunc[T], [](https://microsoft.github.io/agent-lightning/0.2.0/reference/agent/#__codelineno-0-3) ) -> FunctionalLitAgent[T]` `[](https://microsoft.github.io/agent-lightning/0.2.0/reference/agent/#__codelineno-0-1) prompt_rollout() -> ( [](https://microsoft.github.io/agent-lightning/0.2.0/reference/agent/#__codelineno-0-2) Callable[[PromptRolloutFunc[T]], FunctionalLitAgent[T]] [](https://microsoft.github.io/agent-lightning/0.2.0/reference/agent/#__codelineno-0-3) )` Create a [`FunctionalLitAgent`](https://microsoft.github.io/agent-lightning/0.2.0/reference/internal/#agentlightning.litagent.decorator.FunctionalLitAgent " agentlightning.litagent.decorator.FunctionalLitAgent") for prompt-based rollouts. This decorator is designed for agents that work with tunable prompt templates. It enables a workflow where algorithms manage and optimize the prompt template, while agents consume the template to perform rollouts. This is particularly useful for prompt optimization scenarios. Parameters: * **`func`** (`PromptRolloutFunc[T] | None`, default: `None` ) – Callable defining the agent's behavior. Supported signatures include: * `(task, prompt_template) -> result` * `(task, prompt_template, rollout) -> result` * `async (task, prompt_template) -> result` * `async (task, prompt_template, rollout) -> result` Returns: * `[FunctionalLitAgent](https://microsoft.github.io/agent-lightning/0.2.0/reference/internal/#agentlightning.litagent.decorator.FunctionalLitAgent " agentlightning.litagent.decorator.FunctionalLitAgent") [T] | Callable[[PromptRolloutFunc[T]], [FunctionalLitAgent](https://microsoft.github.io/agent-lightning/0.2.0/reference/internal/#agentlightning.litagent.decorator.FunctionalLitAgent " agentlightning.litagent.decorator.FunctionalLitAgent") [T]]` – [`FunctionalLitAgent`](https://microsoft.github.io/agent-lightning/0.2.0/reference/internal/#agentlightning.litagent.decorator.FunctionalLitAgent " agentlightning.litagent.decorator.FunctionalLitAgent") that * `[FunctionalLitAgent](https://microsoft.github.io/agent-lightning/0.2.0/reference/internal/#agentlightning.litagent.decorator.FunctionalLitAgent " agentlightning.litagent.decorator.FunctionalLitAgent") [T] | Callable[[PromptRolloutFunc[T]], [FunctionalLitAgent](https://microsoft.github.io/agent-lightning/0.2.0/reference/internal/#agentlightning.litagent.decorator.FunctionalLitAgent " agentlightning.litagent.decorator.FunctionalLitAgent") [T]]` – wraps the supplied function. Examples: `[](https://microsoft.github.io/agent-lightning/0.2.0/reference/agent/#__codelineno-0-1) @prompt_rollout [](https://microsoft.github.io/agent-lightning/0.2.0/reference/agent/#__codelineno-0-2) def my_agent(task, prompt_template): [](https://microsoft.github.io/agent-lightning/0.2.0/reference/agent/#__codelineno-0-3) messages = prompt_template.format(task=task.input) [](https://microsoft.github.io/agent-lightning/0.2.0/reference/agent/#__codelineno-0-4) return messages [](https://microsoft.github.io/agent-lightning/0.2.0/reference/agent/#__codelineno-0-5) [](https://microsoft.github.io/agent-lightning/0.2.0/reference/agent/#__codelineno-0-6) result = my_agent(task, prompt_template) [](https://microsoft.github.io/agent-lightning/0.2.0/reference/agent/#__codelineno-0-7) result = my_agent.rollout(task, resources, rollout)` Class-based Agents[¶](https://microsoft.github.io/agent-lightning/0.2.0/reference/agent/#class-based-agents "Permanent link") ------------------------------------------------------------------------------------------------------------------------------ ### `agentlightning.LitAgent` [¶](https://microsoft.github.io/agent-lightning/0.2.0/reference/agent/#agentlightning.LitAgent "Permanent link") Bases: `Generic[T]` Base class for implementing agent rollouts. Subclasses override the rollout methods to process tasks while the trainer and runner infrastructure manages orchestration, tracing, and persistence. #### `runner` `property` [¶](https://microsoft.github.io/agent-lightning/0.2.0/reference/agent/#agentlightning.LitAgent.runner "Permanent link") Return the runner responsible for executing rollouts. #### `tracer` `property` [¶](https://microsoft.github.io/agent-lightning/0.2.0/reference/agent/#agentlightning.LitAgent.tracer "Permanent link") Return the tracer configured for this agent. #### `trainer` `property` [¶](https://microsoft.github.io/agent-lightning/0.2.0/reference/agent/#agentlightning.LitAgent.trainer "Permanent link") Return the trainer associated with this agent. #### `__init__(*, trained_agents=None)` [¶](https://microsoft.github.io/agent-lightning/0.2.0/reference/agent/#agentlightning.LitAgent.__init__ "Permanent link") Initialize the agent instance. Parameters: * **`trained_agents`** (`Optional[str]`, default: `None` ) – Optional identifier used by legacy tooling to mark trained agents. Deprecated The `trained_agents` flag is deprecated. Configure `agent_match` in the adapter layer instead. See [`TracerTraceToTriplet`](https://microsoft.github.io/agent-lightning/0.2.0/reference/algorithm/#agentlightning.TracerTraceToTriplet " agentlightning.TracerTraceToTriplet") for more details. #### `get_runner()` [¶](https://microsoft.github.io/agent-lightning/0.2.0/reference/agent/#agentlightning.LitAgent.get_runner "Permanent link") Return the runner responsible for executing rollouts. #### `get_tracer()` [¶](https://microsoft.github.io/agent-lightning/0.2.0/reference/agent/#agentlightning.LitAgent.get_tracer "Permanent link") Return the tracer configured for this agent. #### `get_trainer()` [¶](https://microsoft.github.io/agent-lightning/0.2.0/reference/agent/#agentlightning.LitAgent.get_trainer "Permanent link") Return the trainer associated with this agent. #### `is_async()` [¶](https://microsoft.github.io/agent-lightning/0.2.0/reference/agent/#agentlightning.LitAgent.is_async "Permanent link") Return `True` when the agent overrides any asynchronous rollout methods. Override this method for customized async detection logic. #### `on_rollout_end(task, rollout, runner, tracer)` [¶](https://microsoft.github.io/agent-lightning/0.2.0/reference/agent/#agentlightning.LitAgent.on_rollout_end "Permanent link") Hook invoked after a rollout completes. Subclasses can override this method for cleanup or additional logging. The default implementation is a no-op. Parameters: * **`task`** (`[Task](https://microsoft.github.io/agent-lightning/0.2.0/reference/internal/#agentlightning.Task " agentlightning.Task (agentlightning.types.Task)") `) – [`Task`](https://microsoft.github.io/agent-lightning/0.2.0/reference/internal/#agentlightning.Task " agentlightning.Task") that was processed. * **`rollout`** (`[Rollout](https://microsoft.github.io/agent-lightning/0.2.0/reference/types/#agentlightning.Rollout " agentlightning.Rollout (agentlightning.types.Rollout)") `) – Resulting [`Rollout`](https://microsoft.github.io/agent-lightning/0.2.0/reference/types/#agentlightning.Rollout " agentlightning.Rollout") . * **`runner`** (`[Runner](https://microsoft.github.io/agent-lightning/0.2.0/reference/runner/#agentlightning.Runner " agentlightning.Runner (agentlightning.runner.Runner)") [T]`) – [`Runner`](https://microsoft.github.io/agent-lightning/0.2.0/reference/runner/#agentlightning.Runner " agentlightning.Runner") managing the rollout. * **`tracer`** (`[Tracer](https://microsoft.github.io/agent-lightning/0.2.0/reference/runner/#agentlightning.Tracer " agentlightning.Tracer (agentlightning.tracer.Tracer)") `) – [`Tracer`](https://microsoft.github.io/agent-lightning/0.2.0/reference/runner/#agentlightning.Tracer " agentlightning.Tracer") associated with the runner. Deprecated Override [`Hook.on_rollout_end`](https://microsoft.github.io/agent-lightning/0.2.0/reference/types/#agentlightning.Hook.on_rollout_end " on_rollout_end(*, agent, runner, rollout, spans) async ") instead of this method when extending agents. #### `on_rollout_start(task, runner, tracer)` [¶](https://microsoft.github.io/agent-lightning/0.2.0/reference/agent/#agentlightning.LitAgent.on_rollout_start "Permanent link") Hook invoked immediately before a rollout begins. Subclasses can override this method to implement custom logic such as logging, metric collection, or resource setup. The default implementation is a no-op. Parameters: * **`task`** (`[Task](https://microsoft.github.io/agent-lightning/0.2.0/reference/internal/#agentlightning.Task " agentlightning.Task (agentlightning.types.Task)") `) – [`Task`](https://microsoft.github.io/agent-lightning/0.2.0/reference/internal/#agentlightning.Task " agentlightning.Task") that will be processed. * **`runner`** (`[Runner](https://microsoft.github.io/agent-lightning/0.2.0/reference/runner/#agentlightning.Runner " agentlightning.Runner (agentlightning.runner.Runner)") [T]`) – [`Runner`](https://microsoft.github.io/agent-lightning/0.2.0/reference/runner/#agentlightning.Runner " agentlightning.Runner") managing the rollout. * **`tracer`** (`[Tracer](https://microsoft.github.io/agent-lightning/0.2.0/reference/runner/#agentlightning.Tracer " agentlightning.Tracer (agentlightning.tracer.Tracer)") `) – [`Tracer`](https://microsoft.github.io/agent-lightning/0.2.0/reference/runner/#agentlightning.Tracer " agentlightning.Tracer") associated with the runner. Deprecated Override [`Hook.on_rollout_start`](https://microsoft.github.io/agent-lightning/0.2.0/reference/types/#agentlightning.Hook.on_rollout_start " on_rollout_start(*, agent, runner, rollout) async ") instead of this method when extending agents. #### `rollout(task, resources, rollout)` [¶](https://microsoft.github.io/agent-lightning/0.2.0/reference/agent/#agentlightning.LitAgent.rollout "Permanent link") Execute a rollout synchronously. If you don't wish to implement both training rollout and validation rollout separately, you can just implement `rollout` which will work for both. Parameters: * **`task`** (`T`) – Task payload provided by the scheduler. * **`resources`** (`[NamedResources](https://microsoft.github.io/agent-lightning/0.2.0/reference/types/#agentlightning.NamedResources " agentlightning.NamedResources = Dict[str, ResourceUnion] module-attribute (agentlightning.types.NamedResources)") `) – Mapping of named resources (for example LLMs or prompt templates). * **`rollout`** (`[Rollout](https://microsoft.github.io/agent-lightning/0.2.0/reference/types/#agentlightning.Rollout " agentlightning.Rollout (agentlightning.types.Rollout)") `) – Rollout metadata. Avoid mutating this object directly unless a subclass needs to override defaults. Returns: * `[RolloutRawResult](https://microsoft.github.io/agent-lightning/0.2.0/reference/types/#agentlightning.RolloutRawResult " agentlightning.RolloutRawResult = Union[None, float, List[ReadableSpan], List[Span]] module-attribute (agentlightning.types.RolloutRawResult)") ` – One of the following values: * `[RolloutRawResult](https://microsoft.github.io/agent-lightning/0.2.0/reference/types/#agentlightning.RolloutRawResult " agentlightning.RolloutRawResult = Union[None, float, List[ReadableSpan], List[Span]] module-attribute (agentlightning.types.RolloutRawResult)") ` – * `None` when tracing is handled by the runner. * `[RolloutRawResult](https://microsoft.github.io/agent-lightning/0.2.0/reference/types/#agentlightning.RolloutRawResult " agentlightning.RolloutRawResult = Union[None, float, List[ReadableSpan], List[Span]] module-attribute (agentlightning.types.RolloutRawResult)") ` – * `float` representing the final reward. * `[RolloutRawResult](https://microsoft.github.io/agent-lightning/0.2.0/reference/types/#agentlightning.RolloutRawResult " agentlightning.RolloutRawResult = Union[None, float, List[ReadableSpan], List[Span]] module-attribute (agentlightning.types.RolloutRawResult)") ` – * `List[ReadableSpan]` with OpenTelemetry spans. * `[RolloutRawResult](https://microsoft.github.io/agent-lightning/0.2.0/reference/types/#agentlightning.RolloutRawResult " agentlightning.RolloutRawResult = Union[None, float, List[ReadableSpan], List[Span]] module-attribute (agentlightning.types.RolloutRawResult)") ` – * `List[Span]` with Agent Lightning spans. #### `rollout_async(task, resources, rollout)` `async` [¶](https://microsoft.github.io/agent-lightning/0.2.0/reference/agent/#agentlightning.LitAgent.rollout_async "Permanent link") Execute a rollout asynchronously. Parameters: * **`task`** (`T`) – Task payload provided by the scheduler. * **`resources`** (`[NamedResources](https://microsoft.github.io/agent-lightning/0.2.0/reference/types/#agentlightning.NamedResources " agentlightning.NamedResources = Dict[str, ResourceUnion] module-attribute (agentlightning.types.NamedResources)") `) – Mapping of named resources (for example LLMs or prompt templates). * **`rollout`** (`[Rollout](https://microsoft.github.io/agent-lightning/0.2.0/reference/types/#agentlightning.Rollout " agentlightning.Rollout (agentlightning.types.Rollout)") `) – Rollout metadata. Avoid mutating this object directly unless a subclass needs to override defaults. Returns: * `[RolloutRawResult](https://microsoft.github.io/agent-lightning/0.2.0/reference/types/#agentlightning.RolloutRawResult " agentlightning.RolloutRawResult = Union[None, float, List[ReadableSpan], List[Span]] module-attribute (agentlightning.types.RolloutRawResult)") ` – Same possible return values as * `[RolloutRawResult](https://microsoft.github.io/agent-lightning/0.2.0/reference/types/#agentlightning.RolloutRawResult " agentlightning.RolloutRawResult = Union[None, float, List[ReadableSpan], List[Span]] module-attribute (agentlightning.types.RolloutRawResult)") ` – [`rollout`](https://microsoft.github.io/agent-lightning/0.2.0/reference/agent/#agentlightning.LitAgent.rollout " rollout(task, resources, rollout)") . #### `set_runner(runner)` [¶](https://microsoft.github.io/agent-lightning/0.2.0/reference/agent/#agentlightning.LitAgent.set_runner "Permanent link") Attach the runner responsible for executing rollouts. Parameters: * **`runner`** (`[Runner](https://microsoft.github.io/agent-lightning/0.2.0/reference/runner/#agentlightning.Runner " agentlightning.Runner (agentlightning.runner.Runner)") [T]`) – [`Runner`](https://microsoft.github.io/agent-lightning/0.2.0/reference/runner/#agentlightning.Runner " agentlightning.Runner") coordinating execution. #### `set_trainer(trainer)` [¶](https://microsoft.github.io/agent-lightning/0.2.0/reference/agent/#agentlightning.LitAgent.set_trainer "Permanent link") Attach the trainer responsible for orchestration. Parameters: * **`trainer`** (`[Trainer](https://microsoft.github.io/agent-lightning/0.2.0/reference/trainer/#agentlightning.Trainer " agentlightning.Trainer (agentlightning.trainer.Trainer)") `) – [`Trainer`](https://microsoft.github.io/agent-lightning/0.2.0/reference/trainer/#agentlightning.Trainer " agentlightning.Trainer") that manages the agent. #### `training_rollout(task, resources, rollout)` [¶](https://microsoft.github.io/agent-lightning/0.2.0/reference/agent/#agentlightning.LitAgent.training_rollout "Permanent link") Process a single training task synchronously. By default, this method delegates to [`rollout`](https://microsoft.github.io/agent-lightning/0.2.0/reference/agent/#agentlightning.LitAgent.rollout " rollout(task, resources, rollout)") . #### `training_rollout_async(task, resources, rollout)` `async` [¶](https://microsoft.github.io/agent-lightning/0.2.0/reference/agent/#agentlightning.LitAgent.training_rollout_async "Permanent link") Process a single training task asynchronously. By default, this method delegates to [`rollout_async`](https://microsoft.github.io/agent-lightning/0.2.0/reference/agent/#agentlightning.LitAgent.rollout_async " rollout_async(task, resources, rollout) async ") . #### `validation_rollout(task, resources, rollout)` [¶](https://microsoft.github.io/agent-lightning/0.2.0/reference/agent/#agentlightning.LitAgent.validation_rollout "Permanent link") Process a single validation task synchronously. Override this method when validation should differ from training. The default implementation delegates to [`training_rollout`](https://microsoft.github.io/agent-lightning/0.2.0/reference/agent/#agentlightning.LitAgent.training_rollout " training_rollout(task, resources, rollout)") . #### `validation_rollout_async(task, resources, rollout)` `async` [¶](https://microsoft.github.io/agent-lightning/0.2.0/reference/agent/#agentlightning.LitAgent.validation_rollout_async "Permanent link") Process a single validation task asynchronously. Override this method when validation should differ from training. The default implementation delegates to [`training_rollout_async`](https://microsoft.github.io/agent-lightning/0.2.0/reference/agent/#agentlightning.LitAgent.training_rollout_async " training_rollout_async(task, resources, rollout) async ") . Emitter[¶](https://microsoft.github.io/agent-lightning/0.2.0/reference/agent/#emitter "Permanent link") -------------------------------------------------------------------------------------------------------- ### `agentlightning.emit_reward(reward)` [¶](https://microsoft.github.io/agent-lightning/0.2.0/reference/agent/#agentlightning.emit_reward "Permanent link") Emit a reward value as an OpenTelemetry span. Parameters: * **`reward`** (`float`) – Numeric reward to record. Integers and booleans are converted to floating point numbers for consistency. Returns: * `ReadableSpan` – Readable span capturing the recorded reward. Raises: * `ValueError` – If the provided reward cannot be interpreted as a float or the resulting span is not a [`ReadableSpan`](https://opentelemetry.io/docs/concepts/signals/traces/) instance. ### `agentlightning.emit_message(message)` [¶](https://microsoft.github.io/agent-lightning/0.2.0/reference/agent/#agentlightning.emit_message "Permanent link") Emit a textual message as an OpenTelemetry span. Parameters: * **`message`** (`str`) – Human readable message to attach as a span attribute. Note OpenTelemetry distinguishes between logs and spans. Emitting the message as a span keeps all Agent Lightning telemetry in a single data store for analysis. ### `agentlightning.emit_object(object)` [¶](https://microsoft.github.io/agent-lightning/0.2.0/reference/agent/#agentlightning.emit_object "Permanent link") Emit an object's serialized representation as an OpenTelemetry span. Parameters: * **`object`** (`Any`) – Data structure to encode as JSON and attach to the span payload. Note The payload must be JSON serializable. Non-serializable objects are ignored and an error is logged to aid debugging. ### `agentlightning.emit_exception(exception)` [¶](https://microsoft.github.io/agent-lightning/0.2.0/reference/agent/#agentlightning.emit_exception "Permanent link") Record an exception with OpenTelemetry metadata. Parameters: * **`exception`** (`BaseException`) – Raised exception instance to serialize into telemetry attributes. Note The helper validates its input. Non-exception values are ignored to prevent noisy telemetry and indicate programming mistakes via the logger. Reward Helpers[¶](https://microsoft.github.io/agent-lightning/0.2.0/reference/agent/#reward-helpers "Permanent link") ---------------------------------------------------------------------------------------------------------------------- ### `agentlightning.find_final_reward(spans)` [¶](https://microsoft.github.io/agent-lightning/0.2.0/reference/agent/#agentlightning.find_final_reward "Permanent link") Return the last reward value present in the provided spans. Parameters: * **`spans`** (`Sequence[[SpanLike](https://microsoft.github.io/agent-lightning/0.2.0/reference/types/#agentlightning.SpanLike " agentlightning.SpanLike = Union[ReadableSpan, Span] module-attribute (agentlightning.types.SpanLike)") ]`) – Sequence containing [`ReadableSpan`](https://opentelemetry.io/docs/concepts/signals/traces/) objects or mocked span-like values. Returns: * `Optional[float]` – Reward value from the latest reward span, or `None` when none are found. ### `agentlightning.find_reward_spans(spans)` [¶](https://microsoft.github.io/agent-lightning/0.2.0/reference/agent/#agentlightning.find_reward_spans "Permanent link") Return all reward spans in the provided sequence. Parameters: * **`spans`** (`Sequence[[SpanLike](https://microsoft.github.io/agent-lightning/0.2.0/reference/types/#agentlightning.SpanLike " agentlightning.SpanLike = Union[ReadableSpan, Span] module-attribute (agentlightning.types.SpanLike)") ]`) – Sequence containing [`ReadableSpan`](https://opentelemetry.io/docs/concepts/signals/traces/) objects or mocked span-like values. Returns: * `List[[SpanLike](https://microsoft.github.io/agent-lightning/0.2.0/reference/types/#agentlightning.SpanLike " agentlightning.SpanLike = Union[ReadableSpan, Span] module-attribute (agentlightning.types.SpanLike)") ]` – List of spans that could be parsed as rewards. ### `agentlightning.get_reward_value(span)` [¶](https://microsoft.github.io/agent-lightning/0.2.0/reference/agent/#agentlightning.get_reward_value "Permanent link") Extract the reward value from a span, if available. Parameters: * **`span`** (`[SpanLike](https://microsoft.github.io/agent-lightning/0.2.0/reference/types/#agentlightning.SpanLike " agentlightning.SpanLike = Union[ReadableSpan, Span] module-attribute (agentlightning.types.SpanLike)") `) – Span object produced by AgentOps or Agent Lightning emitters. Returns: * `Optional[float]` – The reward encoded in the span or `None` when the span does not represent a reward. ### `agentlightning.is_reward_span(span)` [¶](https://microsoft.github.io/agent-lightning/0.2.0/reference/agent/#agentlightning.is_reward_span "Permanent link") Return `True` when the provided span encodes a reward value. Legacy Emitter Decorators[¶](https://microsoft.github.io/agent-lightning/0.2.0/reference/agent/#legacy-emitter-decorators "Permanent link") -------------------------------------------------------------------------------------------------------------------------------------------- ### `agentlightning.reward.reward(fn)` [¶](https://microsoft.github.io/agent-lightning/0.2.0/reference/agent/#agentlightning.reward.reward "Permanent link") Decorate a reward function so its outputs are tracked as spans. The decorator integrates with AgentOps when it is available and falls back to the built-in telemetry otherwise. Both synchronous and asynchronous functions are supported transparently. Deprecated This decorator is deprecated. Use [`emit_reward`](https://microsoft.github.io/agent-lightning/0.2.0/reference/agent/#agentlightning.emit_reward " agentlightning.emit_reward(reward)") instead. Parameters: * **`fn`** (`FnType`) – Callable that produces a numeric reward. Returns: * `FnType` – Wrapped callable that preserves the original signature. Back to top --- # Understanding Store - Agent-lightning [Skip to content](https://microsoft.github.io/agent-lightning/0.3.0/deep-dive/store/#understanding-store) ⚠️ You are viewing development documentation. For stable documentation, please visit [https://microsoft.github.io/agent-lightning/stable/](https://microsoft.github.io/agent-lightning/stable/) Understanding Store[¶](https://microsoft.github.io/agent-lightning/0.3.0/deep-dive/store/#understanding-store "Permanent link") ================================================================================================================================ The **[`LightningStore`](https://microsoft.github.io/agent-lightning/0.3.0/reference/store/#agentlightning.LightningStore " agentlightning.LightningStore") ** is the central coordination point for Agent-lightning. It holds the task queue, rollouts, attempts, spans, and versioned resources, and exposes a small API both Runners and Algorithms use to communicate. This document explains what's in the store, how statuses transition, how spans are recorded, and the concurrency model (threads & processes). What's in the Store?[¶](https://microsoft.github.io/agent-lightning/0.3.0/deep-dive/store/#whats-in-the-store "Permanent link") -------------------------------------------------------------------------------------------------------------------------------- ![Store Architecture](https://microsoft.github.io/agent-lightning/0.3.0/assets/store-api-visualized.svg) At a high level: * **Task Queue** – [`enqueue_rollout`](https://microsoft.github.io/agent-lightning/0.3.0/reference/store/#agentlightning.LightningStore.enqueue_rollout " enqueue_rollout(input, mode=None, resources_id=None, config=None, metadata=None) async ") adds work; workers poll with [`dequeue_rollout`](https://microsoft.github.io/agent-lightning/0.3.0/reference/store/#agentlightning.LightningStore.dequeue_rollout " dequeue_rollout(worker_id=None) async ") . When a rollout is dequeued, it automatically creates a new attempt associated with itself. * **Rollouts** – A rollout is one unit of work. It has input, metadata, links to resources, and a lifecycle (`queuing → preparing → running → ...`). Valid [RolloutStatus](https://microsoft.github.io/agent-lightning/0.3.0/reference/types/#agentlightning.RolloutStatus " agentlightning.RolloutStatus = Literal['queuing', 'preparing', 'running', 'failed', 'succeeded', 'cancelled', 'requeuing'] module-attribute ") are **`queuing`**, `preparing`, `running`, `succeeded`, `failed`, **`requeuing`**, **`cancelled`**. For algorithms and runners, the rollout can be seen as a whole, without worrying about the internal attempts. * **Attempts** – Each rollout can have multiple executions (retries). Attempts track [`status`](https://microsoft.github.io/agent-lightning/0.3.0/reference/types/#agentlightning.Attempt.status " status = 'preparing' class-attribute instance-attribute ") , [`start_time`](https://microsoft.github.io/agent-lightning/0.3.0/reference/types/#agentlightning.Attempt.start_time " start_time instance-attribute ") , [`end_time`](https://microsoft.github.io/agent-lightning/0.3.0/reference/types/#agentlightning.Attempt.end_time " end_time = None class-attribute instance-attribute ") , [`last_heartbeat_time`](https://microsoft.github.io/agent-lightning/0.3.0/reference/types/#agentlightning.Attempt.last_heartbeat_time " last_heartbeat_time = None class-attribute instance-attribute ") and link to spans. Valid [AttemptStatus](https://microsoft.github.io/agent-lightning/0.3.0/reference/types/#agentlightning.AttemptStatus " agentlightning.AttemptStatus = Literal['preparing', 'running', 'failed', 'succeeded', 'unresponsive', 'timeout'] module-attribute ") are `preparing`, `running`, `succeeded`, `failed`, `requeuing`, `cancelled`. * **Spans** – Structured trace events produced by the Tracer during an attempt. Spans are ordered by a **monotonic sequence id** per `(rollout_id, attempt_id)`. * **Resources** – Versioned, named bundles (e.g., prompt templates) referenced by rollouts. * **Workers** – Metadata about runner instances: heartbeat timestamps, current assignment, and status. Rollout and Task share the same surface in practice: [`Rollout.input`](https://microsoft.github.io/agent-lightning/0.3.0/reference/types/#agentlightning.Rollout " agentlightning.Rollout") is the task input. The queue stores rollouts that are not yet running; [Runners](https://microsoft.github.io/agent-lightning/0.3.0/reference/runner/#agentlightning.Runner " agentlightning.Runner") dequeue them and update the same rollout's status as work progresses. Before we look at status transitions, it helps to keep in mind that rollouts are the "outside view," while attempts are the "inside view." Attempts are what actually run; rollouts summarize the latest attempt plus a small set of control actions like queueing and cancellation. Attempt Status Transitions[¶](https://microsoft.github.io/agent-lightning/0.3.0/deep-dive/store/#attempt-status-transitions "Permanent link") ---------------------------------------------------------------------------------------------------------------------------------------------- The status model is intentionally small and operationally clear. stateDiagram-v2 direction LR [*] --> preparing: Runner calls dequeue_rollout()
or start_rollout()
or start_attempt() preparing --> running: Runner calls
add_[otel_]span()
for the first time state c_runner <> state c_watch <> preparing --> c_runner: Runner calls
update_attempt(...) running --> c_runner: Runner calls
update_attempt(...) running --> c_watch: Watchdog checks preparing --> c_watch: Watchdog checks state "Client-set outcome" as Client { direction TB succeeded failed } state "Watchdog / policy" as Watch { direction TB timeout unresponsive } c_runner --> succeeded: update_attempt(status=succeeded) c_runner --> failed: update_attempt(status=failed) c_watch --> timeout: now - start_time > timeout_seconds c_watch --> unresponsive: now - last_heartbeat > unresponsive_seconds unresponsive --> running: Runner calls
add_[otel_]span() Each attempt begins in **preparing**, created either when a rollout is dequeued or explicitly started. It transitions to **running** the first time a span is recorded. From there, a few clear rules govern how it can change: * When the runner explicitly marks completion, the attempt becomes **succeeded** or **failed** (when the runner catches exception thrown out by the agent). * When the watchdog detects that the total elapsed time since start exceeds the configured limit, it marks the attempt as **timeout**. * If heartbeats stop arriving for too long, the watchdog marks it **unresponsive**. * A new span from the runner can immediately revive an **unresponsive** attempt back to **running**. What's a Watchdog? The watchdog enforces timing and liveness rules defined by each rollout’s [`RolloutConfig`](https://microsoft.github.io/agent-lightning/0.3.0/reference/types/#agentlightning.RolloutConfig " agentlightning.RolloutConfig") . It’s not a separate thread or service, but a function periodically invoked (e.g., before store mutations) to keep attempts healthy and consistent. This simple model allows the system to distinguish between normal termination, abnormal stalling, and recoverable interruption without additional state flags. Worker Telemetry[¶](https://microsoft.github.io/agent-lightning/0.3.0/deep-dive/store/#worker-telemetry "Permanent link") -------------------------------------------------------------------------------------------------------------------------- Workers track runner-level activity timestamps (`last_heartbeat_time`, `last_dequeue_time`, `last_busy_time`, `last_idle_time`) plus their current rollout assignment. Those fields are now derived automatically: * [`dequeue_rollout(worker_id=...)`](https://microsoft.github.io/agent-lightning/0.3.0/reference/store/#agentlightning.LightningStore.dequeue_rollout " dequeue_rollout(worker_id=None) async ") records which worker polled the queue and refreshes `last_dequeue_time`. * [`update_attempt(..., worker_id=...)`](https://microsoft.github.io/agent-lightning/0.3.0/reference/store/#agentlightning.LightningStore.update_attempt " update_attempt(rollout_id, attempt_id, status=UNSET, worker_id=UNSET, last_heartbeat_time=UNSET, metadata=UNSET) async ") drives the worker status machine. Assigning an attempt marks the worker **busy** and stamps `last_busy_time`; finishing with `status in {"succeeded","failed"}` switches to **idle**, while watchdog transitions such as `timeout`/`unresponsive` make the worker **unknown** and clear `current_rollout_id` / `current_attempt_id`. * [`update_worker(...)`](https://microsoft.github.io/agent-lightning/0.3.0/reference/store/#agentlightning.LightningStore.update_worker " update_worker(worker_id, heartbeat_stats=UNSET) async ") is reserved for heartbeats. It snapshots optional `heartbeat_stats` and always updates `last_heartbeat_time`. Because every transition flows through these APIs, worker status is derived automatically from rollout execution and heartbeats. Note, however, that calling `update_worker` with a new `worker_id` will create a new worker record with status "unknown" if one does not exist. Thus, while manual status changes are not allowed, new worker records can be created externally via heartbeats. Rollout Transition Map[¶](https://microsoft.github.io/agent-lightning/0.3.0/deep-dive/store/#rollout-transition-map "Permanent link") -------------------------------------------------------------------------------------------------------------------------------------- Rollout status is an **aggregated view** of its latest attempt’s status, with additional transitions for queueing and explicit cancellation. A rollout’s retry behavior is controlled by [`Rollout.config`](https://microsoft.github.io/agent-lightning/0.3.0/reference/types/#agentlightning.Rollout " agentlightning.Rollout") (a [`RolloutConfig`](https://microsoft.github.io/agent-lightning/0.3.0/reference/types/#agentlightning.RolloutConfig " agentlightning.RolloutConfig") ). The key fields are: * [`timeout_seconds`](https://microsoft.github.io/agent-lightning/0.3.0/reference/types/#agentlightning.RolloutConfig.timeout_seconds " timeout_seconds = None class-attribute instance-attribute ") – maximum wall-clock time for an attempt before it is marked `timeout`. * [`unresponsive_seconds`](https://microsoft.github.io/agent-lightning/0.3.0/reference/types/#agentlightning.RolloutConfig.unresponsive_seconds " unresponsive_seconds = None class-attribute instance-attribute ") – maximum silence between heartbeats before an attempt is marked `unresponsive`. * [`max_attempts`](https://microsoft.github.io/agent-lightning/0.3.0/reference/types/#agentlightning.RolloutConfig.max_attempts " max_attempts = Field(default=1, ge=1) class-attribute instance-attribute ") – total number of attempts allowed for the rollout (including the first). * [`retry_condition`](https://microsoft.github.io/agent-lightning/0.3.0/reference/types/#agentlightning.RolloutConfig.retry_condition " retry_condition = Field(default_factory=(cast(Callable[[], List[AttemptStatus]], list))) class-attribute instance-attribute ") – which attempt terminal statuses should trigger a retry (e.g., `["failed", "timeout", "unresponsive"]`). **How it plays out:** The runner works on attempt `k`. If the attempt ends in a status that is listed in `retry_condition`, and `k < max_attempts`, the rollout moves to **requeuing** and the store creates attempt `k+1`. Otherwise, the rollout becomes **failed** (or **succeeded** if the runner marked it so). `timeout_seconds` and `unresponsive_seconds` are enforced by the watchdog and feed into the same decision flow. A minimal example of how to use `RolloutConfig`: `[](https://microsoft.github.io/agent-lightning/0.3.0/deep-dive/store/#__codelineno-0-1) from agentlightning import RolloutConfig [](https://microsoft.github.io/agent-lightning/0.3.0/deep-dive/store/#__codelineno-0-2) [](https://microsoft.github.io/agent-lightning/0.3.0/deep-dive/store/#__codelineno-0-3) # Retry on explicit failures or timeouts, up to 3 attempts in total. [](https://microsoft.github.io/agent-lightning/0.3.0/deep-dive/store/#__codelineno-0-4) cfg = RolloutConfig( [](https://microsoft.github.io/agent-lightning/0.3.0/deep-dive/store/#__codelineno-0-5) timeout_seconds=600, [](https://microsoft.github.io/agent-lightning/0.3.0/deep-dive/store/#__codelineno-0-6) unresponsive_seconds=120, [](https://microsoft.github.io/agent-lightning/0.3.0/deep-dive/store/#__codelineno-0-7) max_attempts=3, [](https://microsoft.github.io/agent-lightning/0.3.0/deep-dive/store/#__codelineno-0-8) retry_condition=["failed", "timeout"] [](https://microsoft.github.io/agent-lightning/0.3.0/deep-dive/store/#__codelineno-0-9) ) [](https://microsoft.github.io/agent-lightning/0.3.0/deep-dive/store/#__codelineno-0-10) [](https://microsoft.github.io/agent-lightning/0.3.0/deep-dive/store/#__codelineno-0-11) # When creating/enqueuing a rollout, attach this config. [](https://microsoft.github.io/agent-lightning/0.3.0/deep-dive/store/#__codelineno-0-12) # The store will propagate attempt outcomes according to cfg. [](https://microsoft.github.io/agent-lightning/0.3.0/deep-dive/store/#__codelineno-0-13) rollout = await store.enqueue_rollout(input, config=cfg)` | Latest attempt status | Rollout transition | Notes / guards | | --- | --- | --- | | N/A | `queuing` | Created by `enqueue_rollout()`. | | `preparing` | `queuing/requeuing` → `preparing` | Typically `dequeue_rollout()` or `start_rollout()`/`start_attempt()` creates a new attempt. | | `running` | `preparing/queuing/requeuing` → `running` | First `add_[otel_]span()` flips the attempt to `running`; rollout follows via `rollout_status_from_attempt`. | | `succeeded` | `*` → `succeeded` | Terminal. Rollout `end_time` set. | | `failed` / `timeout` / `unresponsive` | `*` → `requeuing` | **Only if** `status ∈ retry_condition ∧ sequence_id < max_attempts`. | | `failed` / `timeout` / `unresponsive` | `*` → `failed` | Otherwise (no retries left or retries disabled). | | `*` | `*` → `cancelled` | Explicitly set by `update_rollout(status=cancelled)`. | Why aggregation? In code, we use `rollout_status_from_attempt()` which actively updates the rollout based on the latest attempt. Reading the table above is usually easier than reverse-engineering the propagation logic in the code: think of the rollout’s transitions as _callbacks_ on attempt state changes, plus queue/cancel paths. Spans[¶](https://microsoft.github.io/agent-lightning/0.3.0/deep-dive/store/#spans "Permanent link") ---------------------------------------------------------------------------------------------------- Every traceable operation in a rollout is stored as a [Span](https://microsoft.github.io/agent-lightning/0.3.0/reference/types/#agentlightning.Span " agentlightning.Span") . Spans not only capture fine-grained instrumentation but also act as periodic heartbeats that demonstrate liveness. The first span marks activation; each subsequent one both extends the trace and refreshes the attempt’s [`last_heartbeat_time`](https://microsoft.github.io/agent-lightning/0.3.0/reference/types/#agentlightning.Attempt.last_heartbeat_time " last_heartbeat_time = None class-attribute instance-attribute ") . If no span arrives within the configured [`unresponsive_seconds`](https://microsoft.github.io/agent-lightning/0.3.0/reference/types/#agentlightning.RolloutConfig.unresponsive_seconds " unresponsive_seconds = None class-attribute instance-attribute ") , the watchdog downgrades the attempt to **unresponsive** until activity resumes. Spans are indexed by `(rollout_id, attempt_id, sequence_id)` where the sequence is monotonic. Tracing analysis tools like [Adapter](https://microsoft.github.io/agent-lightning/0.3.0/reference/algorithm/#agentlightning.Adapter " agentlightning.Adapter") usually rely on "time order" to reconstruct the trace. However, in a distributed system, the recorded start time and end time of a span are not necessarily in the right order when they aggregated into a central store. Therefore, we enforce every span creation to retrieve a monotonically increasing [`sequence_id`](https://microsoft.github.io/agent-lightning/0.3.0/reference/types/#agentlightning.Span.sequence_id " sequence_id instance-attribute ") from the store before adding the span. Note In practice, one `sequence_id` can be used to create multiple spans. In that case, the orders between the multiple spans are determined by the order of `start_time` and `end_time` of the spans. ### OpenTelemetry conversion[¶](https://microsoft.github.io/agent-lightning/0.3.0/deep-dive/store/#opentelemetry-conversion "Permanent link") Runners often produce [OpenTelemetry `ReadableSpan`](https://opentelemetry.io/docs/concepts/signals/traces/#attributes) objects directly. The store normalizes them into [`Span`](https://microsoft.github.io/agent-lightning/0.3.0/reference/types/#agentlightning.Span " agentlightning.Span") as follows: 1. The runner first requests [`get_next_span_sequence_id`](https://microsoft.github.io/agent-lightning/0.3.0/reference/store/#agentlightning.LightningStore.get_next_span_sequence_id " get_next_span_sequence_id(rollout_id, attempt_id) async ") via `sequence_id = await store.get_next_span_sequence_id(rollout_id, attempt_id)`. This guarantees ordering within the attempt regardless of clock skew. 2. `trace_id`, `span_id`, `parent_id`, `name`, `status`, timestamps, attributes, events, links, and resource are copied from the OTEL span. Timestamps are auto-normalized to seconds (nanoseconds are converted). 3. OTEL `SpanContext` and parent context are preserved so downstream tools can correlate traces across systems. 4. Any additional serializable fields present on the `ReadableSpan` are retained in the stored span (after safe JSON serialization), which keeps the representation forward-compatible. Programmatically this is encapsulated by [`Span.from_opentelemetry(readable_span, rollout_id, attempt_id, sequence_id)`](https://microsoft.github.io/agent-lightning/0.3.0/reference/types/#agentlightning.Span.from_opentelemetry " from_opentelemetry(src, rollout_id, attempt_id, sequence_id) classmethod ") ; [`store.add_otel_span(...)`](https://microsoft.github.io/agent-lightning/0.3.0/reference/store/#agentlightning.LightningStore.add_otel_span " add_otel_span(rollout_id, attempt_id, readable_span, sequence_id=None) async ") simply wraps the fetch-then-add flow. The end result is a store span that is stable to sort, merge, and query, while still preserving the richness of the original OTEL payload. Tip [`add_span`](https://microsoft.github.io/agent-lightning/0.3.0/reference/store/#agentlightning.LightningStore.add_span " add_span(span) async ") or [`add_otel_span`](https://microsoft.github.io/agent-lightning/0.3.0/reference/store/#agentlightning.LightningStore.add_otel_span " add_otel_span(rollout_id, attempt_id, readable_span, sequence_id=None) async ") both appends a span _and_ acts as a heartbeat that can revive `unresponsive` → `running`. ### OTLP Compatibility[¶](https://microsoft.github.io/agent-lightning/0.3.0/deep-dive/store/#otlp-compatibility "Permanent link") Some of the LightningStore implementations support exporting traces via the [OTLP/HTTP specification](https://opentelemetry.io/docs/specs/otlp/) . For example, [`LightningStoreServer`](https://microsoft.github.io/agent-lightning/0.3.0/reference/store/#agentlightning.LightningStoreServer " agentlightning.LightningStoreServer") exposes `/v1/traces` endpoint, it implements the binary Protobuf variant defined by the spec, including the required `Content-Type: application/x-protobuf`, optional `Content-Encoding: gzip`, and status responses encoded as `google.rpc.Status`. Agent-lightning helps parsing `ExportTraceServiceRequest` messages, validate identifiers, normalize resource metadata, and allocate sequence numbers so store implementations only need to persist [`Span`](https://microsoft.github.io/agent-lightning/0.3.0/reference/types/#agentlightning.Span " agentlightning.Span") objects in order. Because the interface speaks standard OTLP, any OpenTelemetry-compatible SDK or collector can emit spans directly to a LightningStore OTLP endpoint without custom shims. The server responds according to the OTLP contract (status code, encoding, and error payloads), which keeps Agent-lightning interoperable with existing observability tooling. This compatibility serves as a strong complement to the OpenTelemetry conversion discussed above. Check whether the store supports OTLP traces via the [`capabilities["otlp_traces"]`](https://microsoft.github.io/agent-lightning/0.3.0/reference/store/#agentlightning.LightningStore.capabilities " capabilities property ") property. Implementation Overview[¶](https://microsoft.github.io/agent-lightning/0.3.0/deep-dive/store/#implementation-overview "Permanent link") ---------------------------------------------------------------------------------------------------------------------------------------- The `agentlightning.store` module is organized into two distinct layers plus optional wrappers: classDiagram direction TB class LightningStore { <> +enqueue_rollout() +dequeue_rollout() +update_attempt() +add_span() +query_rollouts() ... } class LightningCollections { <> +rollouts: Collection +attempts: Collection +spans: Collection +resources: Collection +workers: Collection +rollout_queue: Queue +span_sequence_ids: KeyValue +atomic() } class CollectionBasedLightningStore~T~ { +collections: T -healthcheck_before() -tracked() } class InMemoryLightningStore class MongoLightningStore class InMemoryLightningCollections class MongoLightningCollections class LightningStoreServer { +store: LightningStore +start() +stop() } class LightningStoreClient { +server_address: str } class LightningStoreThreaded { +store: LightningStore } LightningStore <|-- CollectionBasedLightningStore LightningStore <|-- LightningStoreServer LightningStore <|-- LightningStoreClient LightningStore <|-- LightningStoreThreaded CollectionBasedLightningStore <|-- InMemoryLightningStore CollectionBasedLightningStore <|-- MongoLightningStore LightningCollections <|-- InMemoryLightningCollections LightningCollections <|-- MongoLightningCollections InMemoryLightningStore ..> InMemoryLightningCollections : uses MongoLightningStore ..> MongoLightningCollections : uses LightningStoreServer o-- LightningStore : wraps LightningStoreThreaded o-- LightningStore : wraps 1. **Collections Layer** – Low-level storage primitives ([`LightningCollections`](https://microsoft.github.io/agent-lightning/0.3.0/reference/store/#agentlightning.store.collection.LightningCollections " agentlightning.store.collection.LightningCollections") ) providing CRUD operations via [`Collection`](https://microsoft.github.io/agent-lightning/0.3.0/reference/store/#agentlightning.store.collection.Collection " agentlightning.store.collection.Collection") , [`Queue`](https://microsoft.github.io/agent-lightning/0.3.0/reference/store/#agentlightning.store.collection.Queue " agentlightning.store.collection.Queue") , and [`KeyValue`](https://microsoft.github.io/agent-lightning/0.3.0/reference/store/#agentlightning.store.collection.KeyValue " agentlightning.store.collection.KeyValue") interfaces. Each backend (in-memory, MongoDB) implements these primitives. 2. **Store Layer** – All [`LightningStore`](https://microsoft.github.io/agent-lightning/0.3.0/reference/store/#agentlightning.LightningStore " agentlightning.LightningStore") implementations must inherit from [`LightningStore`](https://microsoft.github.io/agent-lightning/0.3.0/reference/store/#agentlightning.LightningStore " agentlightning.LightningStore") and override the methods to implement the storage logic. [`CollectionBasedLightningStore`](https://microsoft.github.io/agent-lightning/0.3.0/reference/store/#agentlightning.CollectionBasedLightningStore " agentlightning.CollectionBasedLightningStore") builds on collections to implement the full [`LightningStore`](https://microsoft.github.io/agent-lightning/0.3.0/reference/store/#agentlightning.LightningStore " agentlightning.LightningStore") API, including business logic like status transitions, watchdog health checks, and retry policies. 3. **Wrappers** – Cross-cutting concerns live in thin wrappers: * [`LightningStoreThreaded`](https://microsoft.github.io/agent-lightning/0.3.0/reference/store/#agentlightning.LightningStoreThreaded " agentlightning.LightningStoreThreaded") adds mutex-based thread safety. * [`LightningStoreServer`](https://microsoft.github.io/agent-lightning/0.3.0/reference/store/#agentlightning.LightningStoreServer " agentlightning.LightningStoreServer") / [`LightningStoreClient`](https://microsoft.github.io/agent-lightning/0.3.0/reference/store/#agentlightning.LightningStoreClient " agentlightning.LightningStoreClient") enable multi-process access over HTTP. Collections[¶](https://microsoft.github.io/agent-lightning/0.3.0/deep-dive/store/#collections "Permanent link") ---------------------------------------------------------------------------------------------------------------- The collections layer provides storage primitives that [`CollectionBasedLightningStore`](https://microsoft.github.io/agent-lightning/0.3.0/reference/store/#agentlightning.CollectionBasedLightningStore " agentlightning.CollectionBasedLightningStore") builds upon. This separation keeps business logic (status transitions, watchdog, retries) in the store layer while allowing different backends to focus purely on persistence. The off-the-shelf implementations are [`InMemoryLightningCollections`](https://microsoft.github.io/agent-lightning/0.3.0/reference/store/#agentlightning.store.collection.InMemoryLightningCollections " agentlightning.store.collection.InMemoryLightningCollections") and [`MongoLightningCollections`](https://microsoft.github.io/agent-lightning/0.3.0/reference/store/#agentlightning.store.collection.mongo.MongoLightningCollections " agentlightning.store.collection.mongo.MongoLightningCollections") , which are the underlying collections for [`InMemoryLightningStore`](https://microsoft.github.io/agent-lightning/0.3.0/reference/store/#agentlightning.InMemoryLightningStore " agentlightning.InMemoryLightningStore") and [`MongoLightningStore`](https://microsoft.github.io/agent-lightning/0.3.0/reference/store/#agentlightning.store.mongo.MongoLightningStore " agentlightning.store.mongo.MongoLightningStore") , respectively. ### Collection Primitives[¶](https://microsoft.github.io/agent-lightning/0.3.0/deep-dive/store/#collection-primitives "Permanent link") [`LightningCollections`](https://microsoft.github.io/agent-lightning/0.3.0/reference/store/#agentlightning.store.collection.LightningCollections " agentlightning.store.collection.LightningCollections") bundles three primitive types: | Primitive | Purpose | Methods | | --- | --- | --- | | [`Collection[T]`](https://microsoft.github.io/agent-lightning/0.3.0/reference/store/#agentlightning.store.collection.Collection " agentlightning.store.collection.Collection") | Indexed storage with primary keys | [`query()`](https://microsoft.github.io/agent-lightning/0.3.0/reference/store/#agentlightning.store.collection.Collection.query " query(filter=None, sort=None, limit=-1, offset=0)


async
")
, [`get()`](https://microsoft.github.io/agent-lightning/0.3.0/reference/store/#agentlightning.store.collection.Collection.get " get(filter=None, sort=None)


async
")
, [`insert()`](https://microsoft.github.io/agent-lightning/0.3.0/reference/store/#agentlightning.store.collection.Collection.insert " insert(items)


async
")
, [`update()`](https://microsoft.github.io/agent-lightning/0.3.0/reference/store/#agentlightning.store.collection.Collection.update " update(items, update_fields=None)


async
")
, [`upsert()`](https://microsoft.github.io/agent-lightning/0.3.0/reference/store/#agentlightning.store.collection.Collection.upsert " upsert(items, update_fields=None)


async
")
, [`delete()`](https://microsoft.github.io/agent-lightning/0.3.0/reference/store/#agentlightning.store.collection.Collection.delete " delete(items)


async
") | | [`Queue[T]`](https://microsoft.github.io/agent-lightning/0.3.0/reference/store/#agentlightning.store.collection.Queue " agentlightning.store.collection.Queue") | FIFO queue for task scheduling | [`enqueue()`](https://microsoft.github.io/agent-lightning/0.3.0/reference/store/#agentlightning.store.collection.Queue.enqueue " enqueue(items)


async
")
, [`dequeue()`](https://microsoft.github.io/agent-lightning/0.3.0/reference/store/#agentlightning.store.collection.Queue.dequeue " dequeue(limit=1)


async
")
, [`peek()`](https://microsoft.github.io/agent-lightning/0.3.0/reference/store/#agentlightning.store.collection.Queue.peek " peek(limit=1)


async
")
, [`size()`](https://microsoft.github.io/agent-lightning/0.3.0/reference/store/#agentlightning.store.collection.Queue.size " size()


async
") | | [`KeyValue[K, V]`](https://microsoft.github.io/agent-lightning/0.3.0/reference/store/#agentlightning.store.collection.KeyValue " agentlightning.store.collection.KeyValue") | Simple key-value store | [`get()`](https://microsoft.github.io/agent-lightning/0.3.0/reference/store/#agentlightning.store.collection.KeyValue.get " get(key, default=None)


async
")
, [`set()`](https://microsoft.github.io/agent-lightning/0.3.0/reference/store/#agentlightning.store.collection.KeyValue.set " set(key, value)


async
")
, [`inc()`](https://microsoft.github.io/agent-lightning/0.3.0/reference/store/#agentlightning.store.collection.KeyValue.inc " inc(key, amount)


async
")
, [`chmax()`](https://microsoft.github.io/agent-lightning/0.3.0/reference/store/#agentlightning.store.collection.KeyValue.chmax " chmax(key, value)


async
")
, [`pop()`](https://microsoft.github.io/agent-lightning/0.3.0/reference/store/#agentlightning.store.collection.KeyValue.pop " pop(key, default=None)


async
") | Every [`LightningCollections`](https://microsoft.github.io/agent-lightning/0.3.0/reference/store/#agentlightning.store.collection.LightningCollections " agentlightning.store.collection.LightningCollections") instance exposes these named collections: * `rollouts` – [`Collection[Rollout]`](https://microsoft.github.io/agent-lightning/0.3.0/reference/store/#agentlightning.store.collection.Collection " agentlightning.store.collection.Collection") keyed by `rollout_id` * `attempts` – [`Collection[Attempt]`](https://microsoft.github.io/agent-lightning/0.3.0/reference/store/#agentlightning.store.collection.Collection " agentlightning.store.collection.Collection") keyed by `(rollout_id, attempt_id)` * `spans` – [`Collection[Span]`](https://microsoft.github.io/agent-lightning/0.3.0/reference/store/#agentlightning.store.collection.Collection " agentlightning.store.collection.Collection") keyed by `(rollout_id, attempt_id, span_id)` * `resources` – [`Collection[ResourcesUpdate]`](https://microsoft.github.io/agent-lightning/0.3.0/reference/store/#agentlightning.store.collection.Collection " agentlightning.store.collection.Collection") keyed by `resources_id` * `workers` – [`Collection[Worker]`](https://microsoft.github.io/agent-lightning/0.3.0/reference/store/#agentlightning.store.collection.Collection " agentlightning.store.collection.Collection") keyed by `worker_id` * `rollout_queue` – [`Queue[str]`](https://microsoft.github.io/agent-lightning/0.3.0/reference/store/#agentlightning.store.collection.Queue " agentlightning.store.collection.Queue") holding rollout IDs awaiting execution * `span_sequence_ids` – [`KeyValue[str, int]`](https://microsoft.github.io/agent-lightning/0.3.0/reference/store/#agentlightning.store.collection.KeyValue " agentlightning.store.collection.KeyValue") tracking monotonic sequence counters ### Atomic Operations[¶](https://microsoft.github.io/agent-lightning/0.3.0/deep-dive/store/#atomic-operations "Permanent link") Collections support atomic operations through the [`atomic()`](https://microsoft.github.io/agent-lightning/0.3.0/reference/store/#agentlightning.store.collection.LightningCollections.atomic " atomic(*, mode='rw', snapshot=False, commit=False, labels=None, **kwargs)") context manager: `[](https://microsoft.github.io/agent-lightning/0.3.0/deep-dive/store/#__codelineno-1-1) async with collections.atomic(mode="rw", labels=["rollouts", "attempts"]) as ctx: [](https://microsoft.github.io/agent-lightning/0.3.0/deep-dive/store/#__codelineno-1-2) rollout = await ctx.rollouts.get(filter={"rollout_id": {"exact": rollout_id}}) [](https://microsoft.github.io/agent-lightning/0.3.0/deep-dive/store/#__codelineno-1-3) # modify and update within the same transaction [](https://microsoft.github.io/agent-lightning/0.3.0/deep-dive/store/#__codelineno-1-4) await ctx.rollouts.update([updated_rollout])` The arguments passed to [`atomic()`](https://microsoft.github.io/agent-lightning/0.3.0/reference/store/#agentlightning.store.collection.LightningCollections.atomic " atomic(*, mode='rw', snapshot=False, commit=False, labels=None, **kwargs)") are quite arbitrary and flexible. Different implementations may have different interpretations of the arguments. For example, to [`InMemoryLightningCollections`](https://microsoft.github.io/agent-lightning/0.3.0/reference/store/#agentlightning.store.collection.InMemoryLightningCollections " agentlightning.store.collection.InMemoryLightningCollections") , the `mode` parameter controls locking behavior (`"r"` for read-only, `"rw"` for read-write), while `labels` specifies which collections to lock. Acquiring locks in sorted order prevents deadlocks when multiple operations run concurrently. ### Implementing a Custom Backend[¶](https://microsoft.github.io/agent-lightning/0.3.0/deep-dive/store/#implementing-a-custom-backend "Permanent link") To add a new storage backend, implement [`LightningCollections`](https://microsoft.github.io/agent-lightning/0.3.0/reference/store/#agentlightning.store.collection.LightningCollections " agentlightning.store.collection.LightningCollections") : `[](https://microsoft.github.io/agent-lightning/0.3.0/deep-dive/store/#__codelineno-2-1) from agentlightning.store.collection import LightningCollections, Collection, Queue, KeyValue [](https://microsoft.github.io/agent-lightning/0.3.0/deep-dive/store/#__codelineno-2-2) [](https://microsoft.github.io/agent-lightning/0.3.0/deep-dive/store/#__codelineno-2-3) class MyLightningCollections(LightningCollections): [](https://microsoft.github.io/agent-lightning/0.3.0/deep-dive/store/#__codelineno-2-4) @property [](https://microsoft.github.io/agent-lightning/0.3.0/deep-dive/store/#__codelineno-2-5) def rollouts(self) -> Collection[Rollout]: [](https://microsoft.github.io/agent-lightning/0.3.0/deep-dive/store/#__codelineno-2-6) return self._rollouts # your implementation [](https://microsoft.github.io/agent-lightning/0.3.0/deep-dive/store/#__codelineno-2-7) [](https://microsoft.github.io/agent-lightning/0.3.0/deep-dive/store/#__codelineno-2-8) @property [](https://microsoft.github.io/agent-lightning/0.3.0/deep-dive/store/#__codelineno-2-9) def rollout_queue(self) -> Queue[str]: [](https://microsoft.github.io/agent-lightning/0.3.0/deep-dive/store/#__codelineno-2-10) return self._queue # your implementation [](https://microsoft.github.io/agent-lightning/0.3.0/deep-dive/store/#__codelineno-2-11) [](https://microsoft.github.io/agent-lightning/0.3.0/deep-dive/store/#__codelineno-2-12) # ... implement remaining properties [](https://microsoft.github.io/agent-lightning/0.3.0/deep-dive/store/#__codelineno-2-13) [](https://microsoft.github.io/agent-lightning/0.3.0/deep-dive/store/#__codelineno-2-14) async def atomic(self, *, mode, snapshot=False, labels=None, **kwargs): [](https://microsoft.github.io/agent-lightning/0.3.0/deep-dive/store/#__codelineno-2-15) # provide transaction / locking semantics [](https://microsoft.github.io/agent-lightning/0.3.0/deep-dive/store/#__codelineno-2-16) ...` Then instantiate your store: `[](https://microsoft.github.io/agent-lightning/0.3.0/deep-dive/store/#__codelineno-3-1) from agentlightning.store.collection_based import CollectionBasedLightningStore [](https://microsoft.github.io/agent-lightning/0.3.0/deep-dive/store/#__codelineno-3-2) [](https://microsoft.github.io/agent-lightning/0.3.0/deep-dive/store/#__codelineno-3-3) store = CollectionBasedLightningStore(collections=MyLightningCollections())` The store layer handles all business logic; your collections just need to provide correct CRUD semantics. Collection-based Store Implementations[¶](https://microsoft.github.io/agent-lightning/0.3.0/deep-dive/store/#collection-based-store-implementations "Permanent link") ---------------------------------------------------------------------------------------------------------------------------------------------------------------------- Agent-lightning ships with two collection-based store implementations: ### InMemoryLightningStore[¶](https://microsoft.github.io/agent-lightning/0.3.0/deep-dive/store/#inmemorylightningstore "Permanent link") [`InMemoryLightningStore`](https://microsoft.github.io/agent-lightning/0.3.0/reference/store/#agentlightning.InMemoryLightningStore " agentlightning.InMemoryLightningStore") uses [`InMemoryLightningCollections`](https://microsoft.github.io/agent-lightning/0.3.0/reference/store/#agentlightning.store.collection.InMemoryLightningCollections " agentlightning.store.collection.InMemoryLightningCollections") backed by Python data structures. It supports **fast startup** with zero external dependencies—ideal for local development, CI, and unit tests. It also provides two lock modes, configurable between `"asyncio"` (single-thread, multiple coroutines) and `"thread"` (multi-threaded via [aiologic](https://github.com/x42005e1f/aiologic) ). [`InMemoryLightningCollections`](https://microsoft.github.io/agent-lightning/0.3.0/reference/store/#agentlightning.store.collection.InMemoryLightningCollections " agentlightning.store.collection.InMemoryLightningCollections") use nested dictionaries for O(1) primary-key lookup and `deque` for the task queue. ### MongoLightningStore[¶](https://microsoft.github.io/agent-lightning/0.3.0/deep-dive/store/#mongolightningstore "Permanent link") [`MongoLightningStore`](https://microsoft.github.io/agent-lightning/0.3.0/reference/store/#agentlightning.store.mongo.MongoLightningStore " agentlightning.store.mongo.MongoLightningStore") uses [`MongoLightningCollections`](https://microsoft.github.io/agent-lightning/0.3.0/reference/store/#agentlightning.store.collection.mongo.MongoLightningCollections " agentlightning.store.collection.mongo.MongoLightningCollections") backed by MongoDB. It supports **persistent storage** suitable for production deployments and **multi-process safe** via database-level atomicity. It also supports **partition support** via `partition_id` for running multiple trainers against the same database. `[](https://microsoft.github.io/agent-lightning/0.3.0/deep-dive/store/#__codelineno-4-1) from agentlightning.store.mongo import MongoLightningStore [](https://microsoft.github.io/agent-lightning/0.3.0/deep-dive/store/#__codelineno-4-2) [](https://microsoft.github.io/agent-lightning/0.3.0/deep-dive/store/#__codelineno-4-3) store = MongoLightningStore( [](https://microsoft.github.io/agent-lightning/0.3.0/deep-dive/store/#__codelineno-4-4) mongo_uri="mongodb://localhost:27017/?replicaSet=rs0", [](https://microsoft.github.io/agent-lightning/0.3.0/deep-dive/store/#__codelineno-4-5) database_name="agentlightning", [](https://microsoft.github.io/agent-lightning/0.3.0/deep-dive/store/#__codelineno-4-6) partition_id="trainer-1", # optional: isolate data per trainer [](https://microsoft.github.io/agent-lightning/0.3.0/deep-dive/store/#__codelineno-4-7) )` Note [`MongoLightningStore`](https://microsoft.github.io/agent-lightning/0.3.0/reference/store/#agentlightning.store.mongo.MongoLightningStore " agentlightning.store.mongo.MongoLightningStore") requires the `mongo` optional dependency. Install with `pip install agentlightning[mongo]`. ### Capabilities[¶](https://microsoft.github.io/agent-lightning/0.3.0/deep-dive/store/#capabilities "Permanent link") [](https://microsoft.github.io/agent-lightning/0.3.0/deep-dive/store/) Different stores have different capabilities. Check the [`capabilities`](https://microsoft.github.io/agent-lightning/0.3.0/reference/store/#agentlightning.LightningStore.capabilities " capabilities property ") property to understand what a store supports: | Capability | Description | InMemory | Mongo | Server | Client | | --- | --- | --- | --- | --- | --- | | `thread_safe` | Safe for concurrent access from multiple threads | configurable | ✓ | ✓ | ✓ | | `async_safe` | Safe for concurrent access from multiple coroutines | ✓ | ✓ | ✓ | ✓ | | `zero_copy` | Can be shared across processes without serialization | ✗ | ✓ | ✓ | ✓ | | `otlp_traces` | Exposes an OTLP-compatible `/v1/traces` endpoint | ✗ | ✗ | ✓ | ✓ | Thread Safety[¶](https://microsoft.github.io/agent-lightning/0.3.0/deep-dive/store/#thread-safety "Permanent link") -------------------------------------------------------------------------------------------------------------------- Thread safety can be achieved at different layers: **At the collections layer**: [`InMemoryLightningCollections`](https://microsoft.github.io/agent-lightning/0.3.0/reference/store/#agentlightning.store.collection.InMemoryLightningCollections " agentlightning.store.collection.InMemoryLightningCollections") accepts a `lock_type` parameter: * `"asyncio"` – Uses per-event-loop `asyncio.Lock` for single-threaded, multi-coroutine scenarios. * `"thread"` – Uses `aiologic.Lock` for true multi-threaded access. **At the store layer**: [`LightningStoreThreaded`](https://microsoft.github.io/agent-lightning/0.3.0/reference/store/#agentlightning.LightningStoreThreaded " agentlightning.LightningStoreThreaded") wraps any [`LightningStore`](https://microsoft.github.io/agent-lightning/0.3.0/reference/store/#agentlightning.LightningStore " agentlightning.LightningStore") to add mutex-based thread safety: * Methods like [`start_rollout`](https://microsoft.github.io/agent-lightning/0.3.0/reference/store/#agentlightning.LightningStore.start_rollout " start_rollout(input, mode=None, resources_id=None, config=None, metadata=None, worker_id=None) async ") , [`enqueue_rollout`](https://microsoft.github.io/agent-lightning/0.3.0/reference/store/#agentlightning.LightningStore.enqueue_rollout " enqueue_rollout(input, mode=None, resources_id=None, config=None, metadata=None) async ") , [`update_attempt`](https://microsoft.github.io/agent-lightning/0.3.0/reference/store/#agentlightning.LightningStore.update_attempt " update_attempt(rollout_id, attempt_id, status=UNSET, worker_id=UNSET, last_heartbeat_time=UNSET, metadata=UNSET) async ") , [`add_span`](https://microsoft.github.io/agent-lightning/0.3.0/reference/store/#agentlightning.LightningStore.add_span " add_span(span) async ") , etc. are guarded by a lock. * Non-mutating, potentially blocking calls remain pass-through by design (e.g., [`wait_for_rollouts`](https://microsoft.github.io/agent-lightning/0.3.0/reference/store/#agentlightning.LightningStore.wait_for_rollouts " wait_for_rollouts(*, rollout_ids, timeout=None) async ") ), as they don't modify shared state and should not hold the lock for long periods. Database-based stores like [`MongoLightningStore`](https://microsoft.github.io/agent-lightning/0.3.0/reference/store/#agentlightning.store.mongo.MongoLightningStore " agentlightning.store.mongo.MongoLightningStore") are inherently thread-safe through database atomicity guarantees. Process Safety and Client-server Store[¶](https://microsoft.github.io/agent-lightning/0.3.0/deep-dive/store/#process-safety-and-client-server-store "Permanent link") ---------------------------------------------------------------------------------------------------------------------------------------------------------------------- **[`LightningStoreServer`](https://microsoft.github.io/agent-lightning/0.3.0/reference/store/#agentlightning.LightningStoreServer " agentlightning.LightningStoreServer") ** wraps another underlying store and runs a FastAPI app to expose the store API over HTTP. [`LightningStoreClient`](https://microsoft.github.io/agent-lightning/0.3.0/reference/store/#agentlightning.LightningStoreClient " agentlightning.LightningStoreClient") is a small [`LightningStore`](https://microsoft.github.io/agent-lightning/0.3.0/reference/store/#agentlightning.LightningStore " agentlightning.LightningStore") implementation that talks to the HTTP API. Warning The server HTTP API is not considered a stable API at this moment. Users are encouraged to use the [`LightningStoreClient`](https://microsoft.github.io/agent-lightning/0.3.0/reference/store/#agentlightning.LightningStoreClient " agentlightning.LightningStoreClient") to communicate with the server as a stable interface. The server tracks the creator PID. In the owner process it delegates directly to the in-memory store; in other processes it lazily constructs a [`LightningStoreClient`](https://microsoft.github.io/agent-lightning/0.3.0/reference/store/#agentlightning.LightningStoreClient " agentlightning.LightningStoreClient") to talk to the HTTP API. This prevents accidental cross-process mutation of the wrong memory image. When the server is pickled (e.g., via `multiprocessing`), only the minimal fields are serialized, but **NOT** the FastAPI/uvicorn objects. Subprocesses won’t accidentally carry live server state. Forked subprocess should also use [`LightningStoreClient`](https://microsoft.github.io/agent-lightning/0.3.0/reference/store/#agentlightning.LightningStoreClient " agentlightning.LightningStoreClient") to communicate with the server in the main process. On the client side, the client retries network/5xx failures using a small backoff, and probes `/v1/agl/health` between attempts. Application exceptions inside the server are wrapped as HTTP 400 with a traceback—these are **not retried**. The client also maintains a **per-event-loop** `aiohttp.ClientSession` map so that tracer callbacks (often on separate loops/threads) don’t hang by reusing a session from another loop. Minimal lifecycle: `[](https://microsoft.github.io/agent-lightning/0.3.0/deep-dive/store/#__codelineno-5-1) import agentlightning as agl [](https://microsoft.github.io/agent-lightning/0.3.0/deep-dive/store/#__codelineno-5-2) [](https://microsoft.github.io/agent-lightning/0.3.0/deep-dive/store/#__codelineno-5-3) # Server (owner process) [](https://microsoft.github.io/agent-lightning/0.3.0/deep-dive/store/#__codelineno-5-4) in_memory_store = agl.InMemoryLightningStore() [](https://microsoft.github.io/agent-lightning/0.3.0/deep-dive/store/#__codelineno-5-5) server = agl.LightningStoreServer(store=in_memory_store, host="0.0.0.0", port=4747) [](https://microsoft.github.io/agent-lightning/0.3.0/deep-dive/store/#__codelineno-5-6) await server.start() # starts uvicorn in a daemon thread and waits for /health [](https://microsoft.github.io/agent-lightning/0.3.0/deep-dive/store/#__codelineno-5-7) # or keep your own event loop and stop via await server.stop() [](https://microsoft.github.io/agent-lightning/0.3.0/deep-dive/store/#__codelineno-5-8) # await server.run_forever() [](https://microsoft.github.io/agent-lightning/0.3.0/deep-dive/store/#__codelineno-5-9) [](https://microsoft.github.io/agent-lightning/0.3.0/deep-dive/store/#__codelineno-5-10) # Client (same or different process) [](https://microsoft.github.io/agent-lightning/0.3.0/deep-dive/store/#__codelineno-5-11) client = agl.LightningStoreClient("http://localhost:4747") [](https://microsoft.github.io/agent-lightning/0.3.0/deep-dive/store/#__codelineno-5-12) [](https://microsoft.github.io/agent-lightning/0.3.0/deep-dive/store/#__codelineno-5-13) print(await client.query_rollouts(status_in=["queuing"])) [](https://microsoft.github.io/agent-lightning/0.3.0/deep-dive/store/#__codelineno-5-14) [](https://microsoft.github.io/agent-lightning/0.3.0/deep-dive/store/#__codelineno-5-15) await client.close() [](https://microsoft.github.io/agent-lightning/0.3.0/deep-dive/store/#__codelineno-5-16) await server.stop()` Another approach is to use a dedicated command line to start a long running server process, possibly sharable across multiple processes. In the main process, you can always use [`LightningStoreClient`](https://microsoft.github.io/agent-lightning/0.3.0/reference/store/#agentlightning.LightningStoreClient " agentlightning.LightningStoreClient") to communicate with the server. `[](https://microsoft.github.io/agent-lightning/0.3.0/deep-dive/store/#__codelineno-6-1) agl store --port 4747` Note [`LightningStoreClient.wait_for_rollouts`](https://microsoft.github.io/agent-lightning/0.3.0/reference/store/#agentlightning.LightningStoreClient.wait_for_rollouts " wait_for_rollouts(*, rollout_ids, timeout=None) async ") intentionally enforces a tiny timeout (≤ 0.1s) to avoid blocking event loops. Poll with short timeouts or compose with `asyncio.wait_for` at a higher layer. Back to top --- # Serving LLM - Agent-lightning [Skip to content](https://microsoft.github.io/agent-lightning/0.3.0/deep-dive/serving-llm/#serving-llms-under-agent-lightning) ⚠️ You are viewing development documentation. For stable documentation, please visit [https://microsoft.github.io/agent-lightning/stable/](https://microsoft.github.io/agent-lightning/stable/) Serving LLMs under Agent-lightning[¶](https://microsoft.github.io/agent-lightning/0.3.0/deep-dive/serving-llm/#serving-llms-under-agent-lightning "Permanent link") ==================================================================================================================================================================== Agent-lightning focuses on data, learning signals, and control flow — **not** on running model inference. This deep dive explains how to **serve** a model alongside Agent-lightning so runners can call it reliably, how the **LLM Proxy** fits into the loop, and why **token IDs** matter if you care about correctness in training and evaluation. General background on LLM serving[¶](https://microsoft.github.io/agent-lightning/0.3.0/deep-dive/serving-llm/#general-background-on-llm-serving "Permanent link") ------------------------------------------------------------------------------------------------------------------------------------------------------------------ [](https://microsoft.github.io/agent-lightning/0.3.0/deep-dive/serving-llm/) Serving a model is essential if you want to train it, especially when you use the model’s own generations as training data. We’ll briefly review the general background to ensure all readers are aligned. Modern LLM servers solve a difficult scheduling problem: keeping GPUs fully utilized while handling prompts of different lengths, streaming tokens as they arrive, and fitting large KV caches into limited memory. Techniques like [**continuous batching**](https://www.anyscale.com/blog/continuous-batching-llm-inference) and [**paged attention**](https://arxiv.org/abs/2309.06180) address these challenges. Continuous batching interleaves decoding across requests to reuse weights efficiently; with careful memory planning, it achieves major throughput gains without increasing latency. PagedAttention reduces KV-cache fragmentation so batching remains effective as sequences grow. See [vLLM’s PagedAttention paper](https://arxiv.org/abs/2309.06180) and [industry analyses](https://www.baseten.co/blog/continuous-vs-dynamic-batching-for-ai-inference/) for details. Balancing inference correctness and efficiency is difficult — a [recent blog](https://thinkingmachines.ai/blog/defeating-nondeterminism-in-llm-inference/) from Thinking Machines Labs highlights how inference nondeterminism ultimately affects training. Beyond scheduling, servers expose an HTTP API, often **OpenAI-compatible** (`/v1/chat/completions` and `/v1/responses`), which is itself a complex stack. In addition to text prompts and chat messages, the API defines many parameters and response fields such as [tool calls](https://platform.openai.com/docs/guides/function-calling) , [structured output](https://platform.openai.com/docs/guides/structured-outputs) , and [multimodal support](https://platform.openai.com/docs/guides/images-vision) . Much effort has been put into implementing all these parameters for many frameworks. Popular engines like **vLLM** and [**SGLang**](https://github.com/sgl-project/sglang) ship with OpenAI-compatible frontends so you can reuse existing client code. [Ollama](https://ollama.com/blog/openai-compatibility) and [llama.cpp](https://llama-cpp-python.readthedocs.io/en/latest/server/) provide similar capabilities. However, because models differ internally, each framework interprets and implements the API slightly differently. Even with identical requests, the tokens passed to the model can vary substantially across frameworks. What Agent-lightning expects from a served LLM[¶](https://microsoft.github.io/agent-lightning/0.3.0/deep-dive/serving-llm/#what-agent-lightning-expects-from-a-served-llm "Permanent link") -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Most of the issues above either have workarounds or remain open research problems. Keep them in mind, but the key question is: what does Agent-lightning expect from a served LLM? The answer includes at least two things: * An OpenAI-compatible **Chat Completions** or **Responses** endpoint the agent can call during rollouts. * Optional training and debugging signals: **logprobs**, **usage**, and ideally **token IDs**. (OpenAI’s public API exposes usage and logprobs, but **not** token IDs — more on [why IDs matter](https://microsoft.github.io/agent-lightning/0.3.0/deep-dive/serving-llm/#token-ids-matter "Token IDs and why they matter") later.) Launching a serving framework[¶](https://microsoft.github.io/agent-lightning/0.3.0/deep-dive/serving-llm/#launching-a-serving-framework "Permanent link") ---------------------------------------------------------------------------------------------------------------------------------------------------------- For many algorithms, you’ll start an engine (e.g., **vLLM** or **SGLang**) before rollouts, then shut it down afterward to free GPU memory. Most frameworks provide a one-line “serve” command to launch the OpenAI-compatible server. You can use those to bring up `/v1/chat/completions` with your checkpoint, ensuring streaming and any required tool-calling features are enabled. A working example is shown in [Unsloth SFT](https://microsoft.github.io/agent-lightning/0.3.0/how-to/unsloth-sft/) . Weight updates — which occur after each training step — are trickier. Some frameworks like [vLLM](https://vllm.ai/) support hot-updating model weights, but it’s usually simpler and more reliable to restart the engine to load new weights. For medium-sized tasks (hundreds of rollouts taking 10+ minutes), the restart overhead (under 30 seconds) is typically negligible. If you’re using Agent-lightning’s [**VERL**](https://microsoft.github.io/agent-lightning/0.3.0/algorithm-zoo/verl/#agentlightning.algorithm.verl.VERL " VERL") integration, the algorithm can **manage the server automatically**. The [VERL framework](https://github.com/volcengine/verl) intelligently allocates compute resources and wraps vLLM/SGLang behind an `AsyncLLMServer` abstraction. You can directly use this as the LLM endpoint for agents. Since VERL can spawn multiple vLLM replicas, using [`LLMProxy`](https://microsoft.github.io/agent-lightning/0.3.0/reference/algorithm/#agentlightning.LLMProxy " agentlightning.LLMProxy") to manage them adds an additional safety layer. A full sequence diagram of how [VERL](https://microsoft.github.io/agent-lightning/0.3.0/algorithm-zoo/verl/#agentlightning.algorithm.verl.VERL " VERL") interacts with the LLM server and proxy is available [here](https://microsoft.github.io/agent-lightning/0.3.0/deep-dive/birds-eye-view/#birds-eye-view-verl-example "Putting It All Together: A Reinforcement Learning Example (VERL)") . LLM Proxy[¶](https://microsoft.github.io/agent-lightning/0.3.0/deep-dive/serving-llm/#llm-proxy "Permanent link") ------------------------------------------------------------------------------------------------------------------ The **LLM Proxy** is a utility class in Agent-lightning, built on [LiteLLM](https://docs.litellm.ai/) , that sits between runners and your backend engine(s) or server(s). In Agent-lightning it acts as a single URL registered as a [`Resource`](https://microsoft.github.io/agent-lightning/0.3.0/reference/types/#agentlightning.Resource " agentlightning.Resource") in the store, offering three key benefits: 1. **Unified endpoint & hot-swaps.** You can redirect traffic between OpenAI, Anthropic, local vLLM/SGLang, or canary checkpoints without modifying agent code — simply repoint the proxy. 2. **First-class tracing.** The proxy emits **OpenTelemetry** spans for every call and sends them to the [`LightningStore`](https://microsoft.github.io/agent-lightning/0.3.0/reference/store/#agentlightning.LightningStore " agentlightning.LightningStore") . It includes rollout and attempt identifiers in request headers so spans are correctly attributed. Sequence numbers are allocated monotonically via the store to [prevent clock-skew issues](https://microsoft.github.io/agent-lightning/0.3.0/tutorials/traces/#distributed-tracing "LLM Proxy") and allow reliable reconstruction of execution trees. 3. **Token IDs.** The proxy can return prompt and response token IDs along with the model output. More details are available in the [next section](https://microsoft.github.io/agent-lightning/0.3.0/deep-dive/serving-llm/#token-ids-matter "Token IDs and why they matter") . Operationally, running the proxy alongside the algorithm works best: the algorithm registers the backend (e.g., the vLLM URL) via [`LLMProxy.update_model_list`](https://microsoft.github.io/agent-lightning/0.3.0/reference/algorithm/#agentlightning.LLMProxy.update_model_list " update_model_list(model_list)") , publishes the proxy URL as a resource via [`LightningStore.add_resources`](https://microsoft.github.io/agent-lightning/0.3.0/reference/store/#agentlightning.LightningStore.add_resources " add_resources(resources) async ") , and runners simply use that URL during rollouts. This mirrors many production client–server setups. Token IDs and why they matter[¶](https://microsoft.github.io/agent-lightning/0.3.0/deep-dive/serving-llm/#token-ids-and-why-they-matter "Permanent link") ---------------------------------------------------------------------------------------------------------------------------------------------------------- [](https://microsoft.github.io/agent-lightning/0.3.0/deep-dive/serving-llm/) This section explains how Agent-lightning handles and uses token IDs — a subtle but important detail for training stability and accuracy. Most agents interact with LLMs via **Chat Completion APIs**, exchanging chat messages. There are two main approaches to collecting training data from such agents. Note Tokenization here refers to the process of converting **Chat Messages** into **Token IDs**. Detokenization is the reverse process of converting **Token IDs** back to **Chat Messages**. Normally, the tokenizer is published along with the pretrained model, which includes a vocabulary, special tokens, and a chat template to dealing with chat messages. **1\. Retokenizing chat messages.** In this approach, you store chat messages as text and let training algorithms **retokenize** them later, as done in many SFT workflows (e.g., [HuggingFace SFT](https://huggingface.co/docs/trl/sft_trainer) ). In practice, we’ve found this method unstable and less accurate. The chart below compares training results. The retokenization approach is run twice. All settings are the same except for the retokenization approach. This instability has three causes. Firstly, chat template used in different frameworks could be slightly different. For example, one single LLaMA model can work with multiple chat templates (multiple in [vLLM](https://github.com/vllm-project/vllm/tree/1d165d6d859d3c50720f0c07209db2363c4fd33b/examples) and one in [HuggingFace](https://huggingface.co/meta-llama) ). It's possible that the chat template used in detokenization is different from the one used in tokenization (this is actually an implementation bug). Secondly, a word might be generated as two tokens (e.g., `H + AVING`) but later retokenized as `HAV + ING`. The text looks identical, but the token IDs differ from what the model originally produced. Thirdly, a generated tool call text like `{ "name": ... }` is parsed by tool call parser into an object that is required by chat completion API. Later, the object is rendered back to `{ "name": ... }` and retokenized again, tool call parsing and re-rendering might cause changes in whitespace and formatting. In some situations, JSON errors may even be auto-corrected by the tool call parser — masking the model’s true generation errors and preventing them from being trained away. * * * **2\. Saving token IDs directly.** The alternative is to save the token IDs generated by the model, as done in RL setups like [Tinker](https://thinkingmachines.ai/tinker/) . This requires a training pipeline that treats tokens as first-class entities, meaning agents must communicate with the inference engine at the token level. However, most agents — especially those built with frameworks like LangChain — rely on OpenAI-compatible APIs and can’t tokenize or detokenize themselves. As mentioned [earlier](https://microsoft.github.io/agent-lightning/0.3.0/deep-dive/serving-llm/#general-llm-serving-background "General background on LLM serving") , implementing this layer manually is complex and error-prone. Some frameworks implement custom solutions (e.g., [VERL Agent Loop](https://github.com/volcengine/verl/blob/4da0d3d3188072772cb2ec817b3d6cf4a463821f/recipe/langgraph_agent/chat_model.py) , [Tinker Renderer](https://github.com/thinking-machines-lab/tinker-cookbook/blob/34a6588d7055040c259985d98e71c0140b389ba7/tinker_cookbook/renderers.py) ), while others leave it to users (e.g., [SkyRL Search-R1](https://novasky-ai.notion.site/skyrl-searchr1) ). * * * A better solution is to use an **OpenAI-compatible API that returns token IDs directly.** This lets agents continue using familiar APIs while capturing token IDs via [tracing](https://microsoft.github.io/agent-lightning/0.3.0/tutorials/traces/) for training. The limitation, of course, is that the serving framework must actually support this capability. When Agent-lightning was first released, we implemented an [instrumented vLLM server](https://github.com/microsoft/agent-lightning/blob/v0.1/agentlightning/instrumentation/vllm.py) that monkey-patched vLLM’s OpenAI server to return token IDs. Since then, the Agent-lightning and vLLM teams have collaborated to add this feature directly to [vLLM core](https://github.com/vllm-project/vllm/pull/22587) . Starting with **vLLM v0.10.2**, the OpenAI-compatible API includes a [`return_token_ids` parameter](https://docs.vllm.ai/en/v0.10.2/serving/openai_compatible_server.html#api-reference) , allowing token IDs to be requested alongside chat messages. SGLang has tracked [similar feature requests](https://github.com/sgl-project/sglang/issues/2634) , though its OpenAI-compatible layer doesn’t yet support them. In short, when using vLLM v0.10.2 or newer, [`LLMProxy`](https://microsoft.github.io/agent-lightning/0.3.0/reference/algorithm/#agentlightning.LLMProxy " agentlightning.LLMProxy") automatically adds `return_token_ids` to each request so the engine includes token IDs in its response. For older vLLM versions, you still need the instrumented version (via `agl vllm` CLI command). Finally, if you only save token IDs in spans, it will have its own limitations — if you train one model using spans from another model with a different tokenizer, incompatibilities can arise. In practice, though, spans in Agent-lightning always store both chat messages and token IDs (actually the full request and response objects), allowing you to fall back to retokenization when necessary. Back to top --- # Overview - Agent-lightning [Skip to content](https://microsoft.github.io/agent-lightning/0.2.2/algorithm-zoo/#algorithm-zoo) ⚠️ You are viewing development documentation. For stable documentation, please visit [https://microsoft.github.io/agent-lightning/stable/](https://microsoft.github.io/agent-lightning/stable/) Algorithm Zoo[¶](https://microsoft.github.io/agent-lightning/0.2.2/algorithm-zoo/#algorithm-zoo "Permanent link") ================================================================================================================== AgentLightning includes several popular and frequently requested algorithms in its built-in library, allowing agent developers to use them directly. These algorithms are designed to be compatible with most agent scenarios. For customizing algorithms, see [Algorithm-side References](https://microsoft.github.io/agent-lightning/0.2.2/reference/algorithm/) . | Algorithm | Optimizing Resources | Description | | --- | --- | --- | | [APO](https://microsoft.github.io/agent-lightning/0.2.2/algorithm-zoo/apo/) | `{: [PromptTemplate][agentlightning.PromptTemplate]}` | Automatic Prompt Optimization (APO) algorithm using textual gradients and beam search. | | [VERL](https://microsoft.github.io/agent-lightning/0.2.2/algorithm-zoo/verl/) | `{"main_llm": [LLM][agentlightning.LLM]}` | Reinforcement Learning with [VERL framework](https://github.com/volcengine/verl)
. | Back to top ---