# Table of Contents
- [Ultravox Realtime - Ultradox](#ultravox-realtime-ultradox)
- [Ultravox REST API Overview - Ultradox](#ultravox-rest-api-overview-ultradox)
- [News & Updates - Ultradox](#news-updates-ultradox)
- [Web Quickstart - Ultradox](#web-quickstart-ultradox)
- [Prompting Guide - Ultradox](#prompting-guide-ultradox)
- [Using Tools - Ultradox](#using-tools-ultradox)
- [Ultravox Client SDK - Ultradox](#ultravox-client-sdk-ultradox)
- [Creating an API Key - Ultradox](#creating-an-api-key-ultradox)
- [SDKs - Ultradox](#sdks-ultradox)
- [Call Stages - Ultradox](#call-stages-ultradox)
- [Voice Cloning - Ultradox](#voice-cloning-ultradox)
- [Adding Knowledge (RAG) - Ultradox](#adding-knowledge-rag-ultradox)
- [Multilingual Agents - Ultradox](#multilingual-agents-ultradox)
- [Debugging & Troubleshooting - Ultradox](#debugging-troubleshooting-ultradox)
- [Data Messages - Ultradox](#data-messages-ultradox)
- [WebSocket Integration - Ultradox](#websocket-integration-ultradox)
- [Telephony - Ultradox](#telephony-ultradox)
- [Webhooks - Ultradox](#webhooks-ultradox)
- [Get Account - Ultradox](#get-account-ultradox)
- [Outgoing Phone Calls - Ultradox](#outgoing-phone-calls-ultradox)
- [List Calls - Ultradox](#list-calls-ultradox)
- [Calls Overview - Ultradox](#calls-overview-ultradox)
- [Deprecation Guide - Ultradox](#deprecation-guide-ultradox)
- [Available Models - Ultradox](#available-models-ultradox)
- [List Call Messages - Ultradox](#list-call-messages-ultradox)
- [Get Call - Ultradox](#get-call-ultradox)
- [Get Call Recording - Ultradox](#get-call-recording-ultradox)
- [It's All Prompting - Ultradox](#it-s-all-prompting-ultradox)
- [List Call Tools - Ultradox](#list-call-tools-ultradox)
- [Base Tool Definition - Ultradox](#base-tool-definition-ultradox)
- [Customer Success & Support - Ultradox](#customer-success-support-ultradox)
- [Customer Acquisition - Ultradox](#customer-acquisition-ultradox)
- [Operations - Ultradox](#operations-ultradox)
- [List Call Stage Messages - Ultradox](#list-call-stage-messages-ultradox)
---
# Ultravox Realtime - Ultradox
[Ultradox home page](/)
Search or ask...
Search...
Navigation
Getting Started
Ultravox Realtime
[Documentation](/introduction)
[API Reference](/api-reference/introduction)
[SDK Reference](/sdk-reference/introduction)
[Changelog](/changelog/news)
[Sign-up\
-------](https://app.ultravox.ai?utm_source=docs)
[Quickstart\
----------](gettingstarted/quickstart-phone-outgoing)
[API Reference + Playground\
--------------------------](api-reference)
[](#key-features)
Key Features
----------------------------------
Built on our best-in-class, [open-weight model](https://github.com/fixie-ai/ultravox)
, Ultravox Realtime provides a fully hosted platform for creating real-time, low-latency voice AI applications.
We provide everything you need:
[**Multi-lingual**](/essentials/multilingual-agents)
→ Support for 26 spoken languages (Arabic, Bulgarian, Chinese, Czech, Danish, Dutch, English, Finnish, French, German, Greek, Hindi, Hungarian, Italian, Japanese, Polish, Portuguese, Romanian, Russian, Slovak, Spanish, Swedish, Tamil, Turkish, Ukrainian, Vietnamese)
[**Tools**](/essentials/tools)
→ Give your agent the ability to connect to the world (AKA function calling)
[**Knowledge (RAG)**](/essentials/rag)
→ Amp up your agent’s knowledge with custom, contextual knowledge
[**Call Stages**](/guides/callstages)
→ Advanced agent branching and flows
[**Conversation History**](/api-reference/calls/calls-messages-list)
→ Audio recordings and full text transcripts
[**Voice Cloning**](/essentials/voices)
→ Create the perfect voice for your agent
[**SDKs**](/essentials/sdk)
→ Build in your favorite language
[**Telephony Integrations**](/essentials/telephony)
→ Quickly use Ultravox with telephony partners
[**Webhooks**](/guides/webhooks)
→ Get realtime notifications for key events
[](#build-powerful-ai-agents)
Build Powerful AI Agents
----------------------------------------------------------
Build any type of voice AI agent. We have guides to help you quickly get started.
[Customer Success & Support\
--------------------------\
\
Create customer support agents who have access to your full product knowledge base. Onboard new accounts.](/usecases/customersuccess)
[Customer Acquisition\
--------------------\
\
Qualify leads. Do outbound sales calls and then handoff to humans.](/usecases/customeracquisition)
[Operations\
----------\
\
Create an AI receptionist to route calls. Perform customer surveys. Make reservations or book meetings.](/usecases/operations)
[It's All Prompting](/gettingstarted/rule1)
---
# Ultravox REST API Overview - Ultradox
[Ultradox home page](/)
Search or ask...
Search...
Navigation
Ultravox REST API Overview
[Documentation](/introduction)
[API Reference](/api-reference/introduction)
[SDK Reference](/sdk-reference/introduction)
[Changelog](/changelog/news)
**Get an API Key**
Using the Ultravox API requires an API key.
You can [sign-up](https://app.ultravox.ai)
for a free account that comes with 30 free minutes for creating calls.
[](#base-url)
Base URL
--------------------------
The Ultravox API is available at `https://api.ultravox.ai/api/`.
[](#api-keys)
API Keys
--------------------------
Ultravox API keys are 41 characters long and are made up of two alphanumeric parts separated by a period. The first part is 8 characters long and the second is 32 characters.
For example: `Zk9Ht7Lm.wX7pN9fM3kLj6tRq2bGhA8yE5cZvD4sT`
Throughout the docs we use `aBCDef.123456` for brevity.
[](#x-api-key-header)
X-API-Key Header
------------------------------------------
When making API calls, pass your key in using the `X-API-Key` header.
**You should never expose your API key to client code**
If you _really_ want to ignore this advice for a local demo, use the X-Unsafe-API-Key header instead at your own risk. It works the same way except that our server will allow it in CORS preflight requests.
Here’s an example showing how to use the fictional API key `aBCDef.123456` to get a list of calls:
curl
JavaScript
curl --request GET \
--url https://api.ultravox.ai/api/calls \
--header 'X-API-Key: aBCDef.123456'
[](#playground)
Playground
------------------------------
If you want to quickly experiment with prompts and voices, the fastest way to do that is in the [Ultravox Dashboard](https://app.ultravox.ai/playground)
.
You can also paste in an Ultravox API key throughout the API reference (look for “Authorization” and paste your key where it asks for `X-API-Key`) and test the REST API endpoints.
[Get Account](/api-reference/accounts/accounts-me-get)
On this page
* [Base URL](#base-url)
* [API Keys](#api-keys)
* [X-API-Key Header](#x-api-key-header)
* [Playground](#playground)
---
# News & Updates - Ultradox
[Ultradox home page](/)
Search or ask...
Search...
Navigation
Changelog
News & Updates
[Documentation](/introduction)
[API Reference](/api-reference/introduction)
[SDK Reference](/sdk-reference/introduction)
[Changelog](/changelog/news)
Be in the Know
--------------
All Ultravox customers automatically receive email updates. Create a [free account](https://app.ultravox.ai)
to start building with the the best voice AI and to stay in the loop.
[](#latest-update)
Latest Update
------------------------------------
###
[](#2025-01-03-upgrades-to-model%2C-tools-performance%2C-and-docs)
2025-01-03 - Upgrades to Model, Tools Performance, and Docs
We’re kicking off 2025 with exciting improvements and community-driven developments build on our leading platform for voice AI (5 cents per minute, highest quality voices, low latency, and SDKs for all major languages).
####
[](#what%E2%80%99s-hot)
What’s Hot
1. Model Upgrades and Performance Improvements
2. New Dashboard Features and Documentation
3. Community Hack Day Results
##### Model Upgrades and Performance Improvements
* **Model Upgrade** → Upgraded Ultravox 0.4.1 to run on the newest Llama 3.3 model, delivering significantly improved instruction following and tool usage capabilities. This new model is now the [default in the Ultravox Realtime](/availablemodels)
service and the [HF model](https://huggingface.co/fixie-ai)
and model card are available.
* **Improved Tool Performance** → Updated our vLLM tool parser with more lenient processing to better handle Llama’s tool calling patterns, resulting in more consistent performance. We fixed an issue with large tool calls being unstable.
* **More Improvements** → Added [`vadSettings`](/api-reference/calls/calls-post)
parameters for more control, new public [`hang-up`](/essentials/tools#changing-call-state)
tool, and added [`timeout`](/api-reference/schema/base-tool-definition)
for tools.
##### New Dashboard Features and Documentation
* **Call History** → Introducing the [Call History](https://app.ultravox.ai/calls)
page at app.ultravox.ai - dive deep into detailed call analytics, including tool error tracking. This is just the first of many dashboard improvements coming your way.
* **New Docs** → Major documentation updates include enhanced [context](/gettingstarted/rule1)
around Ultravox Realtime, new phone integration [quickstarts](/gettingstarted/quickstart-phone-outgoing)
, interactive [API playground](/api-reference)
(test the REST API directly in the docs with your API key), and expanded content covering [tools](/essentials/tools)
, [RAG](/essentials/rag)
, and [voice cloning](/essentials/voices)
.
##### Community Hack Day Results
Last week’s impromptu hack day brought our community together to share their vision for future voice agents.
Members were particularly interested in implementing features like knowledge lookup/RAG, human agent handoff, end-of-call transcript retrieval, Make.com integration, and Cal.com API calendar availability checking.
In response, we’ve released a new sample application that implements all these features as tools for the voice AI agent to call when needed (with end-of-call transcripts handled via webhooks). You can find the complete implementation on our [GitHub repo](https://github.com/fixie-ai/ultradox/tree/main/examples/twilio-incoming-advanced-js)
, along with a detailed [video walkthrough](https://youtu.be/sa9uF5Rr9Os)
.
####
[](#what%E2%80%99s-next)
What’s Next
**Dashboard Evolution** → The new [Call History](https://app.ultravox.ai/calls)
page is just the beginning. We’re developing a no-code builder that will allow you to create and customize voice AI agents without writing any code. This will make it easier than ever to implement complex features like those showcased in our latest sample application, while still maintaining the flexibility for developers who prefer to code their own solutions.
**Other Improvements** → Building on our recent Llama 3.3 integration, we’re focusing on further enhancing tool usage reliability and model performance. We are also working on providing automatic conversation summaries on call end.
[](#prior-updates)
Prior Updates
------------------------------------
###
[](#2024-12-10-websockets-and-more)
2024-12-10 - WebSockets and More
We’re excited to announce several new features and improvements to the Ultravox platform, including new integration options, model support, and infrastructure updates.
####
[](#what%E2%80%99s-hot-2)
What’s Hot
1. New Features: WebSockets, Telnyx, and Plivo
2. SDK and Other Improvements
3. Docs Updates (Including “News” and “Deprecation” pages)
##### New Features: WebSockets, Telnyx, and Plivo
* **WebSockets:** You can now integrate on the server side via [WebSockets](/guides/connectionoptions#websocket-integration)
.
* **Telnyx & Plivo:** New telephony integrations for Telnyx and Plivo are now available in addition to our existing support for Twilio. Check the [docs](/guides/connectionoptions#phone-integration)
.
##### SDK and Other Improvements
* **SDK Updates:** New [client version](/sdk#joincall)
tracking allows you to set an arbitrary value that is tied to calls (retrieve with GET on /calls endpoint).
* **Enhanced Call Transcripts:** For more accurate transcripts, you can now pass in the `languageHint` at call creation time to help guide the model.
* **Bug Fix:** Fixed an issue where errant connections could affect proper call termination.
##### Docs Updates
* Introduced new pages for [News](/updates/news)
and [Deprecations](/updates/deprecation)
to help you stay informed.
* Added comprehensive documentation for [initialMessages](/api/calls#initialmessages)
and [inactivityMessages](/api/calls#inactivitymessages)
.
####
[](#what%E2%80%99s-not)
What’s Not
We have one active deprecation: `initiator` will be deleted at the end of the month. This has been replaced with `firstSpeaker`. Not using `initiator`? You can ignore this. Otherwise, check out the [migration guide](/migration/firstspeaker)
.
####
[](#what%E2%80%99s-next-2)
What’s Next
We’re actively working on several exciting features and improvements:
**Language and Voice Expansion:** Finnish language support is up next. We welcome your input on additional language requirements. Pop into [#feature-requests](https://discord.com/channels/1240071833798184990/1315065334058713198)
to let us know which voices you’d like to see added to our roadmap!
**Infrastructure and Compliance:** EU datacenter planning in progress and GDPR compliance implementation is underway. If these initiatives are important to your operations, please schedule a meeting using my calendar link to discuss your specific requirements.
**Platform Enhancements:** Enhanced call visibility in the dashboard to help you more easily monitor usage and debug issues. We are working on adding a low latency RAG service and are continuing to work on additional optimizations for transcripts and function calling.
**Holiday Schedule Update:** Our team will be operating on a reduced schedule between Christmas and New Year’s. While we’ll maintain system health and provide emergency support, response times on Discord and email may be longer than usual during this period. Rest assured that all critical systems will remain fully monitored and supported by our on-call team.
###
[](#2024-11-14-ultravox-v0-4-1-release)
2024-11-14 - Ultravox v0.4.1 Release
We’re excited to announce the release of Ultravox v0.4.1, which brings significant improvements to the model you’re already using. We’ve also added a new web console and have enabled your agents to start conversations via text.
####
[](#what%E2%80%99s-hot-3)
What’s Hot
1. Ultravox v0.4.1: Six new languages, higher quality, new variants.
2. Ultravox Console: Your web playground and place to manage your account.
3. initialOutputMedium: Agents can now start conversations via text.
##### Ultravox v0.4.1
**Expanded Language Coverage**
* Added 6 new languages (Chinese, Dutch, Hindi, Swedish, Turkish, and Ukrainian).
* Total of 15 languages are now supported by the model.
**Enhanced Performance**
* Improved BLEU scores across all languages.
* Now achieving average BLEU score of 38.97 (vs. GPT-4’s 40.35).
**New Model Variants**
* Added Mistral NeMo variant.
* Updated Llama variants (8B model and 70B model) trained on 8xH100s.
The 0.4.1 updates are now live as the default on our managed Ultravox Realtime APIs. Pricing starts at just 5 cents per minute (⅓ the cost of GPT-4o).
The model weights are available on [Hugging Face](https://huggingface.co/fixie-ai)
, and you can find detailed release notes on our [GitHub repository](https://github.com/fixie-ai/ultravox)
. If you need on-premises support for end-to-end data sovereignty, please reach out via email or set-up a call to discuss.
For insights into our roadmap and strategy and to see a live demonstration of the new model in action, check out our latest [blog post](https://www.ultravox.ai/blog/ultravox-an-open-weight-alternative-to-gpt-4o-realtime)
.
##### Ultravox Console
There’s now a web-based console application at [https://app.ultravox.ai](https://app.ultravox.ai)
that you can use for keeping track of usage, generating API keys, managing your subscription, and playing around with different voices and system prompts. The console is a work-in-progress so don’t hesitate to reach out with requests for new features!
##### initialOutputMedium
This new property can be set at call creation to have the agent’s initial output be text (voice remains the default). This enables text-based scenarios and can be used with the SDK’s [`setOutputMedium()`](../sdk/#setoutputmedium)
to toggle between text and voice. Check out the Create Call docs for more info.
####
[](#what%E2%80%99s-next-3)
What’s Next
We’re already working on the next major release of Ultravox with even more exciting features. Your feedback has been invaluable in shaping our development, and we’d love to hear your thoughts on these latest improvements.
###
[](#2024-10-18-call-stages-and-client-implemented-tools)
2024-10-18 - Call Stages and Client-Implemented Tools
We’re thrilled to share the latest updates we’ve made to the Ultravox APIs. All of these enhancements have been made due to feedback from our community. Please keep the feedback coming! If there’s anything we can do to make things work better for you, don’t hesitate to get in touch!
####
[](#what%E2%80%99s-hot-4)
What’s Hot
1. Call Stages: Dynamic, Multi-Stage Conversations
2. Client-Implemented Tools: Implement Tools in Your App
3. More Improvements: setOutputMedium + Webhooks
##### Call Stages: Dynamic, Multi-Stage Conversations
* **What’s new:** Stages enable more complex and nuanced agent interactions, giving you fine-grained control over the conversation flow.
* **Why it matters:** Each stage can have a new system prompt, a different set of tools, a new voice, an updated conversation history, and more.
* **Where to use:** Stages are designed for complex conversational flows like data gathering (job applications, medical intake forms, applying for a mortgage) or context switching (customer support escalation, triaging IT issues).
* **Where to start:** Check our [docs](../guides/stages/)
for the details on how to get started.
##### Client-Implemented Tools: Implement Tools in Your App
* **What’s new:** In our previous update we added support for tools. Those were “server” tools and required you to implement the logic on a server and expose things via a URL. Client-implemented tools enable putting all the logic in your client application and are still called by your agent.
* **Why it matters:** Enable dynamic UI or other interactivity in your app without having to rely on putting all the logic on a server.
* **Learn more:** Visit our [SDK page](../sdk/#client-tools)
for more info.
##### More Improvements: setOutputMedium + Webhooks
* **setOutputMedium():** Added to our SDKs to give you more control over how your agents respond. Allows toggling the agent’s output between text and voice. See the [docs](../sdk/#setoutputmedium)
.
* **Webhooks:** Ultravox now has [webhooks](../api/webhooks/)
for two key events: `call.started` and `call.ended`. This opens up new opportunities for triggering external processes when calls start/end, logging call data in real-time to your own systems, or integrating Ultravox more deeply with other workflows.
####
[](#what%E2%80%99s-not-2)
What’s Not
1. Breaking Change: SDK SessionState
2. Deprecation Notice: initiator on new call creation
We recognize that breaking changes and deprecation notices are not fun and we try to avoid them when possible. However, we are committed to having our APIs and SDKs work better and be as clear as possible. That means we will inevitably need to revisit some choices early on.
##### Breaking Change: SDK SessionState
In the latest versions of our client SDKs, the UltravoxSession joinCall() method no longer returns an object. UltravoxSession now exposes properties for `status` and `transcripts`.
##### Deprecation: `initiator` is now `firstSpeaker`
This change is being made because `firstSpeaker` is more descriptive of what is happening when the call starts. For example, if you are making an outgoing call, you expect the user to answer the call and be the first to speak. When creating a new call, you should start using `firstSpeaker` and choose either “FIRST\_SPEAKER\_AGENT” (the default) or “FIRST\_SPEAKER\_USER” (for outgoing calls) as the value.
_`initiator` will be removed at the end of November, 2024._
####
[](#what%E2%80%99s-next-4)
What’s Next
We are working on a new version of the Ultravox model that will add new language support for Chinese, Dutch, Hindi, Swedish, Turkish, and Ukrainian. We are also creating a web-based application for the Ultravox service (sign-up, API key management, usage tracking) and are adding a Swift client SDK for iOS developers.
If you have any suggestions for new features or improvements, please don’t hesitate to reach out.
###
[](#2024-09-30-70b-and-tools)
2024-09-30 - 70B and Tools
We are continuing to get great feedback (thank you!) and have been working to add more capabilities.
####
[](#what%E2%80%99s-hot-5)
What’s Hot
1. Ultravox 70B: Our Smartest Model Yet
2. Tools Support: Give Your Agents New Abilities
3. Expanded SDK Coverage: Flutter, Kotlin, and Python
##### 1\. Ultravox 70B: Brains Meet Brawn
* **Why it matters:** More complex reasoning, better understanding
* **How to use:** It’s now the default! Just use ‘fixie-ai/ultravox’ in your API calls
* **Pro tip:** Need the 8B version? Use ‘fixie-ai/ultravox-8B’ (Note: Tools not supported)
* **Model weights:** Available on [HuggingFace](https://huggingface.co/fixie-ai/ultravox-v0_4-llama-3_1-70b)
##### 2\. Tools: Your AI’s New Superpowers
* **What’s new:** Durable tools (create once, use often) and Temporary tools (perfect for iterating)
* **Where to start:** Check our [docs](../tools/)
for the how-to
* **See it in action:** Try our [tools demo](https://demo.ultravox.ai/)
on our website
##### 3\. New SDKs: Code Your Way
* **New additions:** Flutter, Kotlin, and Python join our JavaScript SDK
* **Cool features:** [Debug Messages](../sdk/#debug-messages)
, mic/speaker controls
* **Learn more:** Visit our [SDK page](../sdk/)
for details
####
[](#in-case-you-missed-it)
In Case You Missed It
* **Price drop:** Now just $0.05/minute (cheaper than coffee, and way more talkative!)
* **Voice cloning:** Create [customized voices](../api/voices/#create-clone-voice)
for your agents
* **Conversation continuity:** Because [why start over](../api/calls/#create-call)
?
####
[](#what%E2%80%99s-next%3F)
What’s Next?
You tell us! We’re all ears for your suggestions to make Ultravox even better for you.
###
[](#2024-09-04-price-reduction%2C-resume-calls%2C-voice-cloning)
2024-09-04 - Price Reduction, Resume Calls, Voice Cloning
* Our managed Ultravox APIs are getting _much_ cheaper. We’re decreasing our price to **$0.05/min**. That’s full-on, real-time, speech-to-speech voice chat. We think this is the highest quality, lowest cost system out there.
* We continue to offer 30 minutes of free usage to try it out for yourself. If you’d like to continue using our managed APIs after that, you’ll need to set up a Stripe subscription. You can now do that by accessing the billingUrl from the new [/accounts API](../api/accounts/)
.
* We’ve added the ability to seamlessly continue a prior conversation. This is as simple as passing in a priorCallId parameter when [starting a call](../api/calls/)
.
* We’ve added support for [Voice Cloning](../api/voices/#create-clone-voice)
.
* We released a new version of the Ultravox Model, [v0.4](https://github.com/fixie-ai/ultravox/releases/tag/v0.4)
.
* Tool support is coming very soon!
[Deprecation Guide](/changelog/deprecation)
On this page
* [Latest Update](#latest-update)
* [2025-01-03 - Upgrades to Model, Tools Performance, and Docs](#2025-01-03-upgrades-to-model%2C-tools-performance%2C-and-docs)
* [What’s Hot](#what%E2%80%99s-hot)
* [What’s Next](#what%E2%80%99s-next)
* [Prior Updates](#prior-updates)
* [2024-12-10 - WebSockets and More](#2024-12-10-websockets-and-more)
* [What’s Hot](#what%E2%80%99s-hot-2)
* [What’s Not](#what%E2%80%99s-not)
* [What’s Next](#what%E2%80%99s-next-2)
* [2024-11-14 - Ultravox v0.4.1 Release](#2024-11-14-ultravox-v0-4-1-release)
* [What’s Hot](#what%E2%80%99s-hot-3)
* [What’s Next](#what%E2%80%99s-next-3)
* [2024-10-18 - Call Stages and Client-Implemented Tools](#2024-10-18-call-stages-and-client-implemented-tools)
* [What’s Hot](#what%E2%80%99s-hot-4)
* [What’s Not](#what%E2%80%99s-not-2)
* [What’s Next](#what%E2%80%99s-next-4)
* [2024-09-30 - 70B and Tools](#2024-09-30-70b-and-tools)
* [What’s Hot](#what%E2%80%99s-hot-5)
* [In Case You Missed It](#in-case-you-missed-it)
* [What’s Next?](#what%E2%80%99s-next%3F)
* [2024-09-04 - Price Reduction, Resume Calls, Voice Cloning](#2024-09-04-price-reduction%2C-resume-calls%2C-voice-cloning)
---
# Web Quickstart - Ultradox
[Ultradox home page](/)
Search or ask...
Search...
Navigation
Getting Started
Web Quickstart
[Documentation](/introduction)
[API Reference](/api-reference/introduction)
[SDK Reference](/sdk-reference/introduction)
[Changelog](/changelog/news)
[](#%5Bunder-construction%5D)
\[Under Construction\]
--------------------------------------------------------
[Incoming Phone Calls](/gettingstarted/quickstart-phone-incoming)
[Creating an API Key](/essentials/apikeys)
On this page
* [\[Under Construction\]](#%5Bunder-construction%5D)
---
# Prompting Guide - Ultradox
[Ultradox home page](/)
Search or ask...
Search...
Navigation
Getting Started
Prompting Guide
[Documentation](/introduction)
[API Reference](/api-reference/introduction)
[SDK Reference](/sdk-reference/introduction)
[Changelog](/changelog/news)
[Ultravox Realtime is Fast and Cheap](/gettingstarted/rule6)
[Outgoing Phone Calls](/gettingstarted/quickstart-phone-outgoing)
---
# Using Tools - Ultradox
[Ultradox home page](/)
Search or ask...
Search...
Navigation
Essentials
Using Tools
[Documentation](/introduction)
[API Reference](/api-reference/introduction)
[SDK Reference](/sdk-reference/introduction)
[Changelog](/changelog/news)
Tools in Ultravox (also known as function calling) are a powerful way to extend your agents' capabilities by connecting them to external services and systems. At their core, tools are simply functions that agents can invoke to perform specific actions or retrieve information.
Ultravox includes [built-in tools](/essentials/tools#built-in-tools)
and you can [create custom tools](/essentials/tools#creating-your-first-custom-tool)
.
Here are some of the things you can do with tools:
Communicate with the Outside world
----------------------------------
Lookup the weather, get movie times, create calendar events, or send emails.
Order Lookup
------------
Lookup orders, backordered items, or provide shipment updates.
Knowledge Base
--------------
Consult product and support documentation for contextual support.
Create Support Case
-------------------
Open tailored support cases for human follow-up.
Transfer Call
-------------
Hand-off or escalate calls to human support agents.
End Call
--------
End calls due to user inactivity or after successful resolution.
Any functionality you can encapsulate in a function can be exposed to your agents as a tool. Addtionally, Ultravox automatically calls the underlying function so you don't have to sweat gluing things together.
**Tools in Ultravox are Different**
Unlike using tools with single-generation LLM APIs, Ultravox actually calls your tool. This means you need to do a bit more work upfront in defining tools with the proper authentication and parameters.
**Use ultravox-70B for Tools**
While tools are supported across multiple variants of the Ultravox model, using tools with smaller models (i.e. 8B) typically don’t work well. YMMV. See [available models](/availablemodels)
for more info.
[](#built-in-tools)
Built-in Tools
--------------------------------------
Ultravox currently includes the following built-in tools.
| Tool Name | Tool Action |
| --- | --- |
| queryCorpus | Retrieves relevant information from an existing corpus (AKA knowledge base). Also see [Query Corpus](/api-reference/corpora/corpus-query)
. |
| hangUp | Terminates the call. You can also have your own [custom tools end the call](/_sites/docs.ultravox.ai/essentials/tools#changing-call-state)
. |
| playDtmfSounds | Plays dual-tone multi-frequency (a.k.a dialpad) tones. See [DTMF](/essentials/telephony#dtmf)
for details on sending and receiving tones in your voice applications. |
###
[](#using-a-built-in-tool)
Using a Built-in Tool
You give your AI agent access tools when you [create the call](/api-reference/calls/calls-post)
or create a new [call stage](/guides/callstages#creating-and-managing-stages)
.
// Example request body for creating a call with a built-in tool
{
"systemPrompt":"You are a helpful assistant. When the call naturally wraps up, use the 'hangUp' tool to end the call.",
"selectedTools":[\
{ "toolName": "hangUp" }\
]
};
// The toolId can also be used
{
"systemPrompt":"You are a helpful assistant. When the call naturally wraps up, use the 'hangUp' tool to end the call.",
"selectedTools":[\
{ "toolId": "56294126-5a7d-4948-b67d-3b7e13d55ea7" }\
]
};
**Note:** Using built-in tools is the same as using any other [custom durable tool](/_sites/docs.ultravox.ai/essentials/tools#durable-tools)
that you have created except for one difference: you can override built-in tools by using the same name.
For example, if you created a durable tool named “hangUp” and then provide that tool by name (i.e. not by the toolId), then your tool would be used instead of the built-in hangUp tool.
###
[](#viewing-tools)
Viewing Tools
To view all tools available to you, including built-in tools, use the [List Tools](/api-reference/tools/tools-list)
API.
The [List Tools](/api-reference/tools/tools-list)
API returns all available built-in tools along with any custom tools that you have created.
[](#custom-tools)
Custom Tools
----------------------------------
Custom tools enable you to communicate with the outside world. Anything that you can do in a function can now be done by your agent via a custom tool.
###
[](#creating-your-first-custom-tool)
Creating Your First Custom Tool
Let’s look at creating a tool that sends an email with a summary of the conversation.
There are three steps:
1
Define the Tool
We need to define the tool and provide it to our agent. Everything we provide here will be seen by the agent so we need to be thoughtful of names, descriptions, parameters, etc.
// Creating a tool called 'sendConversationSummary'
//
// A 'string' parameter named 'conversationSummary'
// is passed in the body of a POST request to https://foo.bar/sendSummary
{
"systemPrompt": "You are a helpful assistant...",
"model": "fixie-ai/ultravox",
"selectedTools": [\
{\
"temporaryTool": {\
"modelToolName": "sendConversationSummary",\
"description": "Use this tool at the end of a conversation to send the caller a summary of the conversation.",\
"dynamicParameters": [\
{\
"name": "conversationSummary",\
"location": "PARAMETER_LOCATION_BODY",\
"schema": {\
"description": "A 2-3 sentence summary of the conversation.",\
"type": "string"\
},\
"required": true\
}\
],\
"http": {\
"baseUrlPattern": "https://foo.bar/sendSummary",\
"httpMethod": "POST"\
}\
}\
}\
]
}
What’s happening here:
* We are adding `selectedTools` to the request body of the [`Create Call`](/api-reference/calls/calls-post)
.
* There’s a single tool named `sendConversationSummary`.
* This tool requires a single [dynamic parameter](/_sites/docs.ultravox.ai/essentials/tools#dynamic-parameters)
called `conversationSummary` that is passed in the request body.
* The tool’s functionality is available via POST at the url `https://foo.bar/sendSummary`.
2
Implement the Function
Now that we’ve defined the tool, let’s implement the functionality. This is a simplified example using Express.js and imagines a generic email API provider.
const express = require('express');
const router = express.Router();
router.post('/sendSummary', async (req, res) => {
try {
const { conversationSummary } = req.body;
// Send the email using our email provider
sendEmail(conversationSummary);
return res.status(200).json({
message: 'Conversation summary sent successfully. Continue the conversation with the user.'
});
} catch (error) {
return res.status(500).json({
message: 'Internal server error',
error: error.message
});
}
});
module.exports = router;
This function does the following:
* Accepts the `conversationSummary` via a POST.
* Passes the data along to another function (`sendEmail`) that will send it via email.
3
Instruct the Agent on Tool Use
The last thing we need to do is provide additional instructions to the agent on how to use the tool. This is done by adding to the `systemPrompt`. Let’s update what we used in the first step.
const updatedPrompt = `
You're a friendly and fun guy. You like to chat casually while learning more about the person you're chatting with (name, hobbies, likes/dislikes).
Be casual. Be fun to chat with. Don't talk too much. Keep your sentences pretty short and fun. Let the user guide the conversation.
As you chat, try and learn more about the person you are talking to such as their name, hobbies, and their likes/dislikes.
Once you have all the information, call the 'sendSummary' tool to send a summary of the conversation.
`;
{
"systemPrompt": updatedPrompt,
"model": "fixie-ai/ultravox",
"selectedTools": [\
// Same as before\
]
}
We’ve updated the system prompt that is used when the Ultravox call is created to instruct the agent when and how to use the tool.
###
[](#server-vs-client-tools)
Server vs. Client Tools
You can implement your tools as either server (the tool’s functionality is exposed via a URL) or client (the tool’s functionality is implemented in your client application) tools. From the perspective of Ultravox Realtime, there is no difference between server or client tools. When the model chooses to call a tool, Ultravox will simply call the tool however it’s defined.
There are a couple things to note:
Server tools are defined using \`http\`. Client tools are defined using \`client\`
{
"systemPrompt": "You are a helpful assistant...",
"selectedTools": [\
{\
"temporaryTool": {\
"modelToolName": "sendEmail",\
...\
"http": {\
"baseUrlPattern": "https://foo.bar/sendEmail",\
"httpMethod": "POST"\
}\
}\
},\
{\
"temporaryTool": {\
"modelToolName": "updateScreen",\
...\
"client": {}\
}\
}\
]
}
Client tools are only available if you are using our client SDK.
See [Client Tools](/sdk-reference/introduction#client-tools)
to learn how those are registered and used with the [Ultravox Client SDK](/sdk-reference)
.
###
[](#tool-authentication)
Tool Authentication
Ultravox has rich support for tools auth. When creating a tool, you must specify what is required for successful auth to the backend service.
Three methods for passing API keys are supported and are used when creating the tool:
1
Query Parameter
The API key will be passed via the query string. The name of the parameter must be provided when the tool is created.
2
Header
The API key will be passed via a custom header. The name of the header must be provided when the tool is created.
3
HTTP Authentication
The API key will be passed via the HTTP Authentication header. The name of the scheme (e.g. `Bearer`) must be provided when the tool is created.
You then pass in the key(s) in the `authTokens` property of `selectedTools` when creating a call.
* Query Parameter
* Header
* HTTP Authentication
// Create a tool that uses a query parameter called 'apiKey'
{
"name": "stock_price"
"definition": {
"description": "Get the current stock price for a given symbol",
"requirements": {
"httpSecurityOptions": {
"options": [\
"requirements": {\
"mySeviceApiKey": {\
"queryApiKey": {\
"name": "apiKey"\
}\
}\
}\
]
}
}
}
}
// Pass the API key during call creation
{
"model": "fixie-ai/ultravox-70B"
"systemPrompt": ...
"selectedTools": [\
{\
"toolName": "stock_price"\
"authTokens": {\
"myServiceApiKey": "your_token_here"\
}\
}\
]
}
###
[](#tool-parameters)
Tool Parameters
Tool parameters define what gets passed in to your backend service when the tool is called. When creating a tool, parameters are defined as one of three types:
1
Dynamic
The model will choose which values to pass. These are the parameters you’d use for a single-generation LLM API. Can be overridden using Parameter Overrides (see [below](/_sites/docs.ultravox.ai/essentials/tools#dynamic-parameters)
).
2
Static
Value is known when the tool is defined and is unconditionally set on invocations. Not exposed to or set by the model.
3
Automatic
Like “Static”, except that the value may not be known when the tool is defined but will instead be populated by the system when the tool is invoked.
####
[](#dynamic-parameters)
Dynamic Parameters
Dynamic parameters will have their values set by the model. Creating a dynamic parameter on a tool looks like this:
// Adding a dynamic parameter to a stock price tool
// The parameter will be named 'symbol' and will be passed as a query parameter
{
"name": "stock_price",
"description": "Get the current stock price for a given symbol",
"dynamicParameters": [\
{\
"name": "symbol",\
"location": "PARAMETER_LOCATION_QUERY",\
"schema": {\
"type": "string",\
"description": "Stock symbol (e.g., AAPL for Apple Inc.)"\
},\
"required": true\
}\
]
}
##### Parameter Overrides
You can choose to set static values for dynamic parameters when you start a call. The model won’t see any parameters that you override. When creating a call simply pass in the overrides with each tool:
// Overriding dynamic parameter when starting a new call
// Always set the stock symbol to 'NVDA'
{
"model": "fixie-ai/ultravox-70B",
"systemPrompt": ...
"selectedTools": [\
"toolName": "stock_price",\
"parameterOverrides": {\
"symbol": "NVDA"\
}\
]
}
Parameter overrides don’t make sense for temporary tools. Instead of overriding a dynamic parameter, use a static parameter instead.
####
[](#static-parameters)
Static Parameters
If you have parameters that are known at the time you create the tool, static parameters can be used.
// Adding a static parameter that always sends utm=ultravox
{
"name": "stock_price",
"description": "Get the current stock price for a given symbol",
"staticParameters": [\
{\
"name": "utm",\
"location": "PARAMETER_LOCATION_QUERY",\
"value": "ultravox"\
}\
]
}
####
[](#automatic-parameters)
Automatic Parameters
Automatic parameters are used when you want a consistent, predictable value (not generated by the model) but you don’t know the value when the tool is created.
There are currently two use cases:
* **Call ID** → Used for sending the current Ultravox call ID to the tool. Can be used to provide a unique identifier or for logging purposes. Example: you want to store the call ID in your CRM as part of the current caller’s record.
Use `"knownValue": "KNOWN_PARAM_CALL_ID"`.
* **Conversation History** → Sends the entire conversation history up to that point to the tool. Primarily used for tools that create a new call stage or when you want a tool to do something with the messages. Example: Pre-process conversation history messages when changing call stage and dynamically updating the system prompt for the new stage.
Use `"knownValue": "KNOWN_PARAM_CONVERSATION_HISTORY"`.
// Adding automatic parameters to a profile creation tool
// There are two parameters added:
// 'call_id' which is sent as a query param
// 'conversation_history' which is sent in the request body
{
"name": "create_profile",
"description": "Creates a profile for the current caller",
"automaticParameters": [\
{\
"name": "call_id",\
"location": "PARAMETER_LOCATION_QUERY",\
"knownValue": "KNOWN_PARAM_CALL_ID"\
},\
{\
"name": "conversation_history",\
"location": "PARAMETER_LOCATION_BODY",\
"knownValue": "KNOWN_PARAM_CONVERSATION_HISTORY"\
}\
]
}
You can find more on Automatic Parameters in the [Base Tool Definition](/api-reference/schema/base-tool-definition)
.
###
[](#temporary-vs-durable-tools)
Temporary vs. Durable Tools
Custom tools come in two varieties: temporary and durable. There is much more information below but there are a few things to consider right upfront:
1
Creation
Temporary tools are created in the request body when a new call is created. Durable tools are created using the [Ultravox REST API](/api-reference/tools/tools-post)
.
2
No Functional Difference
There is no functional difference within the context of an Ultravox call between the two tool types.
3
Iteration Speed
Temporary tools are great when you are building out a new application and need to iterate.
4
Reuse & Collaboration
Durable tools are best when you have things dialed in and want to reuse tools across applications and/or work with a team and want to divide ownership of tools from the rest of your app.
The distinction between temporary and durable only applies to custom tools. [Built-in](/_sites/docs.ultravox.ai/essentials/tools#built-in-tools)
tools are always durable.
####
[](#temporary-tools)
Temporary Tools
Temporary tools are created each time you create a new Call and exist exclusively within the context of that call. (Temporary tools aren’t visible in the [List Tools](/api-reference/tools/tools-list)
response for example.)
Iteration is faster when using temporary tools because you don’t have to create/update/delete tools as you build out your application. You can simply adjust the JSON in the request body and start a new call.
####
[](#creating-%26-using-temporary-tools)
Creating & Using Temporary Tools
Temporary tools are defined and passed in the request body of the [`Create Call`](/api-reference/calls/calls-post)
endpoint. They are available during the current call.
{
"model": "fixie-ai/ultravox-70B",
"systemPrompt": ...
"selectedTools": [\
"temporaryTool": {\
"modelToolName": "stock_price",\
"description": "Get the current stock price for a given symbol",\
"dynamicParameters": [\
{\
"name": "symbol",\
"location": "PARAMETER_LOCATION_QUERY",\
"schema": {\
"type": "string",\
"description": "Stock symbol (e.g., AAPL for Apple Inc.)"\
},\
"required": true\
}\
],\
"http": {\
"baseUrlPattern": "https://api.stockmarket.com/v1/price",\
"httpMethod": "GET"\
}\
}\
]
}
####
[](#durable-tools)
Durable Tools
In addition to temporary tools, Ultravox supports the creation of durable tools. There is no functional difference between durable and temporary tools within the context of a call.
Durable tools are persisted and can be reused across calls or applications. They shine once you have things dialed in, when you want to share tools across multiple applications, or if you have split responsibilities on the team.
####
[](#creating-durable-tools)
Creating Durable Tools
The `/tools` endpoint in the Ultravox API is for working with durable tools. You create durable tools either by uploading an OpenAPI spec or via the request body in the [`Create Tool`](/api-reference/tools/tools-post)
endpoint. Your OpenAPI spec must be either `json` or `yaml` format.
####
[](#using-durable-tools)
Using Durable Tools
To use a durable tool in a call, set the `toolName` or `toolId` field instead of using a `temporaryTool`.
For example:
// Request body for creating an Ultravox call with a durable tool
{
"model": "fixie-ai/ultravox-70B",
"systemPrompt": ...
"selectedTools": [\
"toolName": "stock_price",\
]
}
[](#additional-information)
Additional Information
------------------------------------------------------
###
[](#changing-call-state)
Changing Call State
For most tools, the response will include data you want the model to use (e.g. the results of a lookup). However, Ultravox has support for special tool actions that can end the call or change the call stage.
These tool actions require setting a special response type.
| Response Type | Tool Action |
| --- | --- |
| hang-up | Terminates the call. In addition to having Ultravox end the call after [periods of user inactivity](/api-reference/calls/overview#inactivitymessages)
, your custom tool can end the call. |
| new-stage | Creates a new call stage. See [here](/guides/callstages)
for more. |
How you set the response type depends on if you are using a server/HTTP or a client tool:
**Server/HTTP Tools**
Server tools must respond with the `X-Ultravox-Response-Type` header set to either `hang-up` or `new-stage`
**Client Tools**
For client tools, set `responseType="hang-up"` or `responseType="new-stage"` on your `ClientToolResult` object.
###
[](#debugging)
Debugging
The Ultravox SDK enables viewing [debug messages](/sdk-reference/introduction#debug-messages)
for any active call. These messages include tool calls.
###
[](#tool-definition-schema)
Tool Definition Schema
The `definition` object in the tool creation and update requests follows the [BaseToolDefinition schema](/api-reference/schema/base-tool-definition)
.
Here’s a breakdown of its main components:
* `description` (string): A clear, concise description of what the tool does.
* `dynamicParameters` (array, optional): List of parameters that can be set by the AI model when using the tool. Each parameter is an object containining:
* `name` (string): The name of the parameter.
* `location` (string): Where the parameter is used (“PARAMETER\_LOCATION\_QUERY”, “PARAMETER\_LOCATION\_PATH”, “PARAMETER\_LOCATION\_HEADER”, “PARAMETER\_LOCATION\_BODY”).
* `schema` (object): JSON Schema definition of the parameter. This typically includes things like type, description, enum values, format, other restrictions, etc.
* `required` (boolean): Whether the parameter is required.
* `staticParameters` (array, optional): List of parameters that are always set to a known, fixed value when the tool is used. These are unconditionally added when the tool is invoked. These parameters are not exposed to or set by the model. Example: you use an API for various things but want to track which requests come from your Ultravox app so you always append `utm=ultravox` to the query parameters.
* `automaticParameters` (array, optional): Additional parameters automatically set by the system. Used when the value is not known when the tool is created but that will be known when the tool is called. Example: you want to use the unique `call_id` from ultravox as a key in your backend and you have the tool include `call_id` in the request body when your tool’s API is called.
* `requirements` (object, optional): Any specific requirements for using the tool. Currently this is used for security (e.g. API keys or HTTP Auth).
* `http` (object): Details for invoking the tool via HTTP. For server tools.
* `baseUrlPattern` (string): The base URL pattern for the tool, possibly with placeholders for path parameters.
* `httpMethod` (string): The HTTP method for the tool (e.g., “GET”, “POST”).
* `client` (object): Declares the tool as a client tool. Exactly one of `http` or `client` must be set for a tool.
###
[](#best-practices-for-creating-tools)
Best Practices for Creating Tools
1. **Clear Naming**: Choose a descriptive and unique name for your tool that clearly indicates its function.
2. **Detailed Description**: Provide a comprehensive description of what the tool does, including any important details about its usage or limitations. This and the name will help the model decide when and how to use your tool.
3. **Precise Parameters**: Define your dynamic parameters carefully, ensuring that the AI model has all the information it needs to use the tool effectively.
4. **Error Handling**: Consider how your tool will handle errors or unexpected inputs, and document this behavior in the tool’s description.
5. **Iterate Faster**: Use temporary tools when you are building your application. Persist durable tools in the system when things have stabilized.
6. **Version Control**: When updating tools, consider creating a new version (e.g., “stock\_price\_v2”) rather than modifying the existing tool. This allows testing of the new tool before impacting new calls made with the prior version of the tool.
7. **Security**: Be mindful of security when creating tools, especially when they interact with external APIs. Use appropriate authentication methods and avoid exposing sensitive information.
8. **Testing**: Thoroughly test your tools before deploying them in production conversations to ensure they function as expected.
[Creating an API Key](/essentials/apikeys)
[Adding Knowledge (RAG)](/essentials/rag)
On this page
* [Built-in Tools](#built-in-tools)
* [Using a Built-in Tool](#using-a-built-in-tool)
* [Viewing Tools](#viewing-tools)
* [Custom Tools](#custom-tools)
* [Creating Your First Custom Tool](#creating-your-first-custom-tool)
* [Server vs. Client Tools](#server-vs-client-tools)
* [Tool Authentication](#tool-authentication)
* [Tool Parameters](#tool-parameters)
* [Dynamic Parameters](#dynamic-parameters)
* [Static Parameters](#static-parameters)
* [Automatic Parameters](#automatic-parameters)
* [Temporary vs. Durable Tools](#temporary-vs-durable-tools)
* [Temporary Tools](#temporary-tools)
* [Creating & Using Temporary Tools](#creating-%26-using-temporary-tools)
* [Durable Tools](#durable-tools)
* [Creating Durable Tools](#creating-durable-tools)
* [Using Durable Tools](#using-durable-tools)
* [Additional Information](#additional-information)
* [Changing Call State](#changing-call-state)
* [Debugging](#debugging)
* [Tool Definition Schema](#tool-definition-schema)
* [Best Practices for Creating Tools](#best-practices-for-creating-tools)
---
# Ultravox Client SDK - Ultradox
[Ultradox home page](/)
Search or ask...
Search...
Navigation
SDK
Ultravox Client SDK
[Documentation](/introduction)
[API Reference](/api-reference/introduction)
[SDK Reference](/sdk-reference/introduction)
[Changelog](/changelog/news)
The Ultravox [REST API](/api-reference)
is used to create calls but you must use one of the Ultravox client SDKs to join and end calls. This page primarily uses examples in JavaScript. The concepts are the same across all the [different SDK implementations](/_sites/docs.ultravox.ai/sdk-reference/introduction#sdk-implementations)
.
[](#methods)
Methods
------------------------
The core of the SDK is the `UltravoxSession`. The session is used to join and leave calls. The `UltravoxSession` contains methods for:
* Joining/leaving a call
* Sending text messages to the agent
* Changing the output medium for how the agent replies
* Registering client tools
* Muting the microphone/speaker
###
[](#joincall)
joinCall()
Joins a call.
joinCall(joinUrl: string, clientVersion?: string): void
[](#param-join-url)
joinUrl
string
required
The joinUrl that was returned from the [`Create Call`](/api-reference/calls/calls-post)
request.
[](#param-client-version)
clientVersion
string
Optional string that can be used for application version tracking. Will be appended to the call and be available in the `clientVersion` field in the [`Get Call`](/api-reference/calls/calls-get)
response.
###
[](#leavecall)
leaveCall()
Leaves the current call. Returns a promise (with no return value) that resolves when the call has successfully been left.
async leaveCall(): Promise
###
[](#sendtext)
sendText()
Sends a text message to the agent.
sendText(text: string): void
[](#param-text)
text
string
required
The message to send to the agent.
###
[](#setoutputmedium)
setOutputMedium()
Sets the agent’s output medium for future utterances. If the agent is currently speaking, this will take effect at the end of the agent’s utterance.
setOutputMedium(medium: Medium): void
[](#param-medium)
medium
Medium
required
How replies are communicated. Must be either `'text'` or `'voice'`.
Also see [muteSpeaker](/_sites/docs.ultravox.ai/sdk-reference/introduction#mutespeaker)
and [unmuteSpeaker](/_sites/docs.ultravox.ai/sdk-reference/introduction#unmutespeaker)
below.
###
[](#registertoolimplementation)
registerToolImplementation()
Registers a client tool implementation with the given name. If the call is started with a client-implemented tool, this implementation will be invoked when the model calls the tool.
registerToolImplementation(name: string, implementation: ClientToolImplementation): void
[](#param-name)
name
string
required
The name of the tool. Must match what is defined in `selectedTools` during [`Create Call`](/api-reference/calls/calls-post)
.
If `nameOverride` is set then must match that name. Otherwise must match `modelToolName`.
[](#param-implementation)
implementation
ClientToolImplementation
required
The function that implements the tool’s logic. This is a function that:
**Accepts `parameters`** → An object containing key-value pairs for the tool’s parameters. The keys will be strings.
**Returns** → Either a string result, or an object with a result string and a responseType, or a Promise that resolves to one of these.
For example:
const stock_price = (parameters) => {
... // to be implemented
return `Stock price is ${value}`;
};
###
[](#registertoolimplementations)
registerToolImplementations()
Convenience batch wrapper for [`registerToolImplementation`](/_sites/docs.ultravox.ai/sdk-reference/introduction#registertoolimplementation)
.
registerToolImplementations(implementationMap: { [name: string]: ClientToolImplementation }): void
[](#param-implementation-map)
implementationMap
Object
required
An object where each key (a string) represents the name of the tool and each value is a `ClientToolImplementation` function.
###
[](#ismicmuted)
isMicMuted()
Returns a boolen indicating if the end user’s microphone is muted. This is scoped to the Ultravox SDK and does not detect muting done by the user outside of your application.
isMicMuted(): boolean
###
[](#isspeakermuted)
isSpeakerMuted()
Returns a boolen indicating if the speaker (the agent’s voice output) is muted. This is scoped to the Ultravox SDK and does not detect muting done by the user outside of your application.
isSpeakerMuted(): boolean
###
[](#mutemic)
muteMic()
Mutes the end user’s microphone. This is scoped to the Ultravox SDK.
muteMic(): void
###
[](#unmutemic)
unmuteMic()
Unmutes the end user’s microphone. This is scoped to the Ultravox SDK.
unmuteMic(): void
###
[](#mutespeaker)
muteSpeaker()
Mutes the end user’s speaker (the agent’s voice output). This is scoped to the Ultravox SDK.
muteSpeaker(): void
###
[](#unmutespeaker)
unmuteSpeaker()
Unmutes the end user’s speaker (the agent’s voice output). This is scoped to the Ultravox SDK.
unmuteSpeaker(): void
[](#client-tools)
Client Tools
----------------------------------
Ultravox has robust support for [tools](/essentials/tools)
. The SDK has support for client tools. Client tools will be invoked in your client code and enable you to add interactivity in your app that is driven by user interactions with your agent. For example, your agent could choose to invoke a tool that would trigger some UI change.
**Message Size Limit**
Messages larger than 15-16KB may cause timeouts. Keep payload sizes within this limit.
**Creating Client Tools**
Client tools are defined just like “server” tools with three exceptions:
1
'client' not 'http'
You don’t add the URL and HTTP method for client tools.
Instead, you add `"client": {}` to the tool definition.
Using a Client Tool
Using a Server Tool
{
"model": "fixie-ai/ultravox-70B",
"systemPrompt": ...
"selectedTools": [\
"temporaryTool": {\
"modelToolName": "stock_price",\
"description": "Get the current stock price for a given symbol",\
"dynamicParameters": [\
{\
"name": "symbol",\
"location": "PARAMETER_LOCATION_BODY",\
"schema": {\
"type": "string",\
"description": "Stock symbol (e.g., AAPL for Apple Inc.)"\
},\
"required": true\
}\
],\
"client": {}\
}\
]
}
2
Register Tool with Client
Your client tool must be registered in your client code. Here’s a snippet that might be found in client code to register the client tool and implement the logic for the tool.
See the SDK method [`registerToolImplementation()`](/_sites/docs.ultravox.ai/sdk-reference/introduction#registertoolimplementation)
for more information.
Registering a Client Tool
// Start up our Ultravox Session
uvSession = new UltravoxSession();
// Register our client-side tool
uvSession.registerToolImplementation(
"stock_price",
stock_price
);
uvSession.joinCall(joinUrl);
// Function that implements tool logic
const stock_price = (parameters) => {
... // to be implemented
return `Stock price is ${value}`;
};
3
Only Body Parameters Allowed
Unlike server tools (which accept parameters passed by path, header, body, etc.), client tools only allow parameters to be passed in the body of the request. That means client tools will always have parameter location set like this:
"location": "PARAMETER_LOCATION_BODY"
[](#session-status)
Session Status
--------------------------------------
The `UltravoxSession` exposes status. Based on the `UltravoxSessionStatus` enum, status can be one of the following:
| status | description |
| --- | --- |
| disconnected | Session is not connected. This is the initial state prior to joinCall. |
| disconnecting | Session is in the process of disconnecting. |
| connecting | Session is establishing the connection. |
| idle | Session is connected but not yet active. |
| listening | Listening to the end user. |
| thinking | The model is processing/thinking. |
| speaking | The model is speaking. |
**Status Events**
The status can be retrieved by adding an event listener to the session status.
Get Session Status Events
// Listen for status changing events
session.addEventListener('status', (event) => {
console.log('Session status changed: ', session.status);
});
[](#transcripts)
Transcripts
--------------------------------
Sometimes you may want to augment the audio with text transcripts (e.g. if you want to show the end user the model’s output in real-time). Transcripts can be retrieved by adding an event listener:
Get Transcripts
// Listen for transcripts changing events
session.addEventListener('transcripts', (event) => {
console.log('Transcripts updated: ', session.transcripts);
});
Transcripts are returned as an array of transcript objects.
[](#param-transcript)
transcript
Transcript Object
Hide properties
[](#param-text-1)
text
string
Text transcript of the speech from the end user or the agent.
[](#param-is-final)
isFinal
boolean
True if the transcript represents a complete utterance. False if it is a fragment of an utterance that is still underway.
[](#param-speaker)
speaker
Role
Either “user” or “agent”. Denotes who was speaking.
[](#param-medium-1)
medium
Medium
Either “voice” or “text”. Denotes how the message was sent.
[](#debug-messages)
Debug Messages
--------------------------------------
**No Guarantee**
Debug messages from Ultravox should be treated as debug logs. They can change regularly and don’t have a contract. Relying on the specific structure or content should be avoided.
The `UltravoxSession` object also provides debug messages. Debug messages must be enabled when creating a new session and then are available via an event listener similar to status and transcripts:
Get Debug Messages
// Listen for debug messages
session.addEventListener('experimental_message', (msg) => {
console.log('Got a debug message: ', JSON.stringify(msg));
});
**Debug Message: Tool Call**
When the agent invokes a tool, the message contains the function, all arguments, and an invocation ID:
LLM response: Tool calls: [FunctionCall(name='createProfile', args='{"firstName":"Ron","lastName":"Burgandy","organization":"Fixie.ai","useCase":"creating a talking AI news reporter"}', invocation_id='call_D2qQVS8OQc998aMEw5PRa9cF')]
**Debug Message: Tool Call Result**
When the tool call completes, the message contains an array of messages. Multiple tools can be invoked by the model. This message array will conatain all the calls followed by all the results. These messages are also available via [List Call Messages](/api-reference/calls/calls-messages-list)
.
Here’s an example of what we might see from a single tool invocation:
Tool call complete.
Result: [\
role: MESSAGE_ROLE_TOOL_CALL ordinal: 6 text: "{\"firstName\":\"Ron\",\"lastName\":\"Burgandy\",\"organization\":\"Fixie.ai\",\"useCase\":\"creating a talking AI news reporter\"}" tool_name: "createProfile" invocation_id: "call_D2qQVS8OQc998aMEw5PRa9cF" tool_id: "aa737e12-0989-4adb-9895-f387f40557d8" ,\
role: MESSAGE_ROLE_TOOL_RESULT ordinal: 7 text: "{\"firstName\":\"Ron\",\"lastName\":\"Burgandy\",\"emailAddress\":null,\"organization\":\"Fixie\",\"useCase\":\"creating a talking AI news reporter\"}" tool_name: "createProfile" invocation_id: "call_D2qQVS8OQc998aMEw5PRa9cF" tool_id: "aa737e12-0989-4adb-9895-f387f40557d8"\
]
[](#sdk-implementations)
SDK Implementations
------------------------------------------------
There are currently five implementations of the SDK available:
JavaScript
----------
`npm install ultravox-client`
[](https://github.com/fixie-ai/ultravox-client-sdk-js)
Flutter
-------
`flutter add ultravox_client`
[](https://github.com/fixie-ai/ultravox-client-sdk-flutter)
Python
------
`pip install ultravox-client`
[](https://github.com/fixie-ai/ultravox-client-sdk-python)
Kotlin (Android)
----------------
`Find it on Maven Central`
[](https://github.com/fixie-ai/ultravox-client-sdk-android)
Swift (iOS)
-----------
`Find it on Swift Package Index`
[](https://github.com/fixie-ai/ultravox-client-sdk-ios)
On this page
* [Methods](#methods)
* [joinCall()](#joincall)
* [leaveCall()](#leavecall)
* [sendText()](#sendtext)
* [setOutputMedium()](#setoutputmedium)
* [registerToolImplementation()](#registertoolimplementation)
* [registerToolImplementations()](#registertoolimplementations)
* [isMicMuted()](#ismicmuted)
* [isSpeakerMuted()](#isspeakermuted)
* [muteMic()](#mutemic)
* [unmuteMic()](#unmutemic)
* [muteSpeaker()](#mutespeaker)
* [unmuteSpeaker()](#unmutespeaker)
* [Client Tools](#client-tools)
* [Session Status](#session-status)
* [Transcripts](#transcripts)
* [Debug Messages](#debug-messages)
* [SDK Implementations](#sdk-implementations)
---
# Creating an API Key - Ultradox
[Ultradox home page](/)
Search or ask...
Search...
Navigation
Essentials
Creating an API Key
[Documentation](/introduction)
[API Reference](/api-reference/introduction)
[SDK Reference](/sdk-reference/introduction)
[Changelog](/changelog/news)
If you don’t have an Ultravox account, you can sign up at [https://app.ultravox.ai](https://app.ultravox.ai)
. All new accounts get 30 free minutes of call time.
API keys must be managed using the Ultravox console. Once you have created an API key, you can use the REST API.
1
Sign-In
Make sure you are signed into you account at [https://app.ultravox.ai](https://app.ultravox.ai)
2
Go to Settings
In the left nav, click on Settings. Or navigate to [https://app.ultravox.ai/settings/](https://app.ultravox.ai/settings/)
.
3
Click 'Generate New Key'
In the API Keys sections, click on “Generate New Key”.
4
Name the Key and Save
Create a name for your new API key and create it. Make sure you save the key in a secure secrets manager.
[](#)
----------
[Web Quickstart](/gettingstarted/quickstart-web)
[Using Tools](/essentials/tools)
---
# SDKs - Ultradox
[Ultradox home page](/)
Search or ask...
Search...
Navigation
Essentials
SDKs
[Documentation](/introduction)
[API Reference](/api-reference/introduction)
[SDK Reference](/sdk-reference/introduction)
[Changelog](/changelog/news)
If you are building a voice AI application that has a front end (e.g. web, mobile, desktop), then you should use our client SDK which is designed to deliver high-quality audio at low-latency. The Ultravox Client SDK uses WebRTC. There is also support for [Telephony](/essentials/telephony)
and [WebSockets](/guides/websockets)
.
[](#sdk-features)
SDK Features
----------------------------------
All the features of the SDK are documented in the [SDK Reference](/sdk-reference/introduction)
.
[](#sdk-implementations)
SDK Implementations
------------------------------------------------
There are currently five implementations of the SDK available:
JavaScript
----------
`npm install ultravox-client`
[](https://github.com/fixie-ai/ultravox-client-sdk-js)
Flutter
-------
`flutter add ultravox_client`
[](https://github.com/fixie-ai/ultravox-client-sdk-flutter)
Python
------
`pip install ultravox-client`
[](https://github.com/fixie-ai/ultravox-client-sdk-python)
Kotlin (Android)
----------------
`Find it on Maven Central`
[](https://github.com/fixie-ai/ultravox-client-sdk-android)
Swift (iOS)
-----------
`Find it on Swift Package Index`
[](https://github.com/fixie-ai/ultravox-client-sdk-ios)
[Adding Knowledge (RAG)](/essentials/rag)
[Telephony](/essentials/telephony)
On this page
* [SDK Features](#sdk-features)
* [SDK Implementations](#sdk-implementations)
---
# Call Stages - Ultradox
[Ultradox home page](/)
Search or ask...
Search...
Navigation
Advanced Guides + Tutorials
Call Stages
[Documentation](/introduction)
[API Reference](/api-reference/introduction)
[SDK Reference](/sdk-reference/introduction)
[Changelog](/changelog/news)
The Ultravox API’s Call Stages functionality allows you to create dynamic, multi-stage conversations. Stages enable more complex and nuanced agent interactions, giving you fine-grained control over the conversation flow.
Each stage can have a new system prompt, a different set of tools, a new voice, an updated conversation history, and more.
**Advanced Feature**
Call stages require planning and careful implementation and are likely not required for simple use cases.
[](#understanding-call-stages)
Understanding Call Stages
------------------------------------------------------------
Call Stages (“Stages”) provide a way to segment a conversation into distinct phases, each with its own system prompt and potentially different parameters. This enables interactions that can adapt and change focus as the conversation progresses.
Key points to understand about Stages:
**Dynamic System Prompts** → Stages allow you to give granular system prompts to the model as the conversation progresses.
**Flexibility** → You have full control to determine when and how you want the conversation to progress to the next stage.
**Thoughtful Design** → Implementing stages requires careful planning and consideration of the conversation structure. Consider how to handle stage transitions and test thoroughly to ensure a natural flow to the conversation.
**Maintain Context** → Think about how the agent will maintain context about the user between stages if you need to ensure a coherent conversation.
[](#creating-and-managing-stages)
Creating and Managing Stages
------------------------------------------------------------------
To implement Call Stages in your Ultravox application, follow these steps:
1
Plan Your Stages
Determine the different phases of your conversation and what prompts or parameters should change at each stage.
2
Implement a Stage Change Tool
Create a custom tool that will trigger stage changes when called. This tool should:
* Respond with a `new-stage` response type. This creates the new stage. How you send the response depends on the tool type:
* For server/HTTP tools, set the `X-Ultravox-Response-Type` header to `new-stage`.
* For [client tools](/sdk-reference/introduction#client-tools)
, set `responseType="new-stage"` on your `ClientToolResult` object.
* Provide the updated parameters (e.g., system prompt, tools, voice) for the new stage in the response body.
Unless overridden, stages inherit all properties of the existing call. See [Stages Call Properties](/_sites/docs.ultravox.ai/guides/callstages#stages-call-properties)
for the list of call properties that can be changed.
3
Configure Stage Transitions
* Prompt the agent to use the stage change tool at appropriate points in the conversation.
* Ensure the stage change tool is part of `selectedTools` when creating the call as well as during new stages (if needed).
* Update your system prompt as needed to instruct the agent on when/how to use the stage change tool.
**Things to Remember**
* New stages inherit all properties from the previous stage unless explicitly overridden.
* Refer to [Stages Call Properties](/_sites/docs.ultravox.ai/guides/callstages#stages-call-properties)
to understand which call properties can be changed as part of a new stage.
* Test your stage transitions thoroughly to ensure the conversation flows naturally.
###
[](#example-stage-change-implementation)
Example Stage Change Implementation
Here’s a basic example of how to implement a new call stage.
First, we create a tool that is responsible for changing stages:
function changeStage(requestBody) {
const responseBody = {
systemPrompt: "...", // new prompt
..., // other properties to change, like the voice
// You may optionally also set toolResultText, which will be the content
// of the tool result message in conversation history. The tool result
// will be the most recent message the model sees during its next generation
// unless you set initialMessages. Defaults to "OK".
toolResultText: "(New Stage) Next, focus on..."
};
return {
body: responseBody,
headers: {
'X-Ultravox-Response-Type': 'new-stage'
}
};
}
We also need to ensure that we have instructed our agent to use the tool and that we add the tool to our `selectedTools` during the creation of the call.
// Instruct the agent on how to use the stage management tool
// Add the tool to selectedTools
{
systemPrompt: "You are a helpful assistant...you have access to a tool called changeStage...",
...
selectedTools: [\
{\
"temporaryTool": {\
"modelToolName": "changeStage",\
"description": ...,\
"dynamicParameters": [...],\
}\
}\
]
}
**Inheritance**
New stages inherit all properties from the previous stage. You can selectively overwrite properties as needed when defining a new stage.
See [Stages Call Properties](/_sites/docs.ultravox.ai/guides/callstages#stages-call-properties)
for more.
[](#ultravox-api-implications)
Ultravox API Implications
------------------------------------------------------------
If you are not using stages for a call, retrieving calls or call messages via the API (e.g. [`List Calls`](/api-reference/calls/calls-list)
) works as expected.
However, if you are using call stages then you most likely want to use the stage-centric API endpoints to get stage-specific settings, messages, etc.
Use [`List Call Stages`](/api-reference/calls/calls-stages-list)
to get all the stages for a given call.
| Ultravox API | Stage-Centric Equivalent |
| --- | --- |
| [`Get Call`](/api-reference/calls/calls-get) | [`Get Call Stage`](/api-reference/calls/calls-stages-get) |
| [`List Call Messages`](/api-reference/calls/calls-messages-list) | [`List Call Stage Messages`](/api-reference/calls/calls-stages-messages-list) |
| [`List Call Tools`](/api-reference/calls/calls-tools-list) | [`List Call Stage Tools`](/api-reference/calls/calls-stages-tools-list) |
[](#stages-call-properties)
Stages Call Properties
------------------------------------------------------
The schema used for a Stages response body is a subset of the request body schema used when creating a new call. The response body must contain the new values for any properties you want to change in the new stage.
Unless overridden, stages inherit all properties of the existing call.
Here is the list of all call properties that can and cannot be changed during a new stage:
| property | change with new stage? |
| --- | --- |
| systemPrompt | Yes |
| temperature | Yes |
| voice | Yes |
| languageHint | Yes |
| initialMessages | Yes |
| selectedTools | Yes |
| firstSpeaker | No |
| model | No |
| joinTimeout | No |
| maxDuration | No |
| timeExceededMessage | No |
| inactivityMessages | No |
| medium | No |
| recordingEnabled | No |
[](#use-cases-for-call-stages)
Use Cases for Call Stages
------------------------------------------------------------
Call Stages are particularly useful for complex conversational flows. Here are some example scenarios:
**Data Gathering** → Scenarios where the agent needs to collect a lot of data. Examples: job applications, medical intake forms, applying for a mortgage.
Here are potential stages for a **Mortgage Application**:
* Stage 1: Greeting and basic information gathering
* Stage 2: Financial assessment
* Stage 3: Property evaluation
* Stage 4: Presentation of loan options
* Stage 5: Hand-off to loan officer
**Switching Contexts** → Scenarios where the agent needs to navigate different contexts. Examples: customer support escalation, triaging IT issues.
Let’s consider what the potential stages might be for **Customer Support**:
* Stage 1: Initial greeting and problem identification
* Stage 2: Troubleshooting
* Stage 3: Resolution or escalation (to another stage or to a human support agent)
[](#conclusion)
Conclusion
------------------------------
Call Stages in the Ultravox API give you the ability to create adaptive conversations for more complex scenarios like data gathering or switching contexts. By allowing granular control over system prompts and conversation parameters at different stages, you can create more dynamic and context-aware interactions.
[Multilingual Agents](/essentials/multilingual-agents)
[Debugging & Troubleshooting](/guides/debugging)
On this page
* [Understanding Call Stages](#understanding-call-stages)
* [Creating and Managing Stages](#creating-and-managing-stages)
* [Example Stage Change Implementation](#example-stage-change-implementation)
* [Ultravox API Implications](#ultravox-api-implications)
* [Stages Call Properties](#stages-call-properties)
* [Use Cases for Call Stages](#use-cases-for-call-stages)
* [Conclusion](#conclusion)
---
# Voice Cloning - Ultradox
[Ultradox home page](/)
Search or ask...
Search...
Navigation
Essentials
Voice Cloning
[Documentation](/introduction)
[API Reference](/api-reference/introduction)
[SDK Reference](/sdk-reference/introduction)
[Changelog](/changelog/news)
Ultravox Realtime includes multiple, high-quality voices for all supported languages. The fastest way to experience the included voices is in the [Playground](https://app.ultravox.ai/playground)
. You can also use the [List Voices](/api-reference/voices/voices-list)
endpoint to see all voices and their details.
[](#creating-a-custom-voice)
Creating a Custom Voice
--------------------------------------------------------
Currently, we support one cloned voice per account. If you need more cloned voices, please [reach out](mailto:hello@fixie.ai?subject=Additional%20Voices)
.
You can create a custom voice by uploading an audio sample using the [Create (Clone) Voice](/api-reference/voices/voices-post)
endpoint. This process allows you to generate a unique voice that matches the characteristics of your audio sample.
###
[](#prerequisites)
Prerequisites
* An Ultravox Realtime API key
* A single audio file containing a clear voice sample (30 seconds recommended)
* The audio file must be in .mp3 or .wav format
###
[](#using-the-api)
Using the API
To create a custom voice, send a POST request to the `/api/voices` endpoint with your audio file. Note: multiple files are not supported.
Here’s how to do it:
curl --request POST https://api.ultravox.ai/api/voices \
--header 'Content-Type: multipart/form-data' \
--header 'X-API-Key: YOUR_API_KEY' \
--form 'file=@"/path/to/your/audio-sample.wav"'
--form 'name=My Custom Voice' \
--form 'description=Voice recorded on Jan 1, 2024'
###
[](#requirements-for-audio-samples)
Requirements for Audio Samples
For optimal results, ensure your audio sample meets these criteria:
* Clear, high-quality audio without background noise or echo
* Single speaker throughout the recording
* Natural speaking pace and tone
* No music or other voices in the background
* 30-60 seconds in length (longer samples do not typically lead to better clones)
###
[](#limitations)
Limitations
* Maximum of one audio file per voice
* 10MB file size maximum
[Telephony](/essentials/telephony)
[Multilingual Agents](/essentials/multilingual-agents)
On this page
* [Creating a Custom Voice](#creating-a-custom-voice)
* [Prerequisites](#prerequisites)
* [Using the API](#using-the-api)
* [Requirements for Audio Samples](#requirements-for-audio-samples)
* [Limitations](#limitations)
---
# Adding Knowledge (RAG) - Ultradox
[Ultradox home page](/)
Search or ask...
Search...
Navigation
Essentials
Adding Knowledge (RAG)
[Documentation](/introduction)
[API Reference](/api-reference/introduction)
[SDK Reference](/sdk-reference/introduction)
[Changelog](/changelog/news)
LLMs have tremendous knowledge about the world, but they don’t have all the up-to-date specifics about your organization, your products, or other relevant information you might want to provide to your AI voice agents.
RAG is a common technique for grounding agents in the relevant information for your use case.
[](#examples-of-knowledge-sources)
Examples of Knowledge Sources
--------------------------------------------------------------------
Let’s consider some of the content that might be useful to serve some popular use cases:
Customer Success & Support
* Product Documentation → user guides, FAQs, troubleshooting steps.
* Onboarding Materials → getting started guides, best practices, transcripts from training videos.
Customer Acquisition
* Product Information → features, pricing tiers, competitive comparisons.
* Sales Scripts → qualification questions, objection handling, industry-specific use cases.
Operations
* Internal Processes → call routing rules, department directories, business hours & holiday schedules.
* Survey Materials → question banks, follow-up questions, rating scales.
[](#adding-rag-to-ultravox)
Adding RAG to Ultravox
------------------------------------------------------
As we saw in the [Using Tools](/essentials/tools)
guide, tools provide power-ups for your agents. To use RAG with an Ultravox agent, it’s as simple as creating a tool and instructing the agent on how to use the tool.
###
[](#the-easy-way)
The Easy Way
Ultravox provides the [corpus service](/api-reference/corpora)
for RAG.
1
Create a Corpus
Use the [Create Corpus](/api-reference/corpora/corpora-post)
endpoint. Give your new corpus a name and (optional) description. This returns a `corpusId`.
2
Create a Source
Add a website to crawl using [Create Corpus Source](/api-reference/corpora/corpora-sources-post)
. Each source is given a unique `sourceId`. We will crawl the URL(s) and ingest all the content.
3
Query the Corpus
After everything is loaded, try some queries using the [Query Corpus](/api-reference/corpora/corpus-query)
endpoint.
4
Use the queryCorpus Tool
Give the built-in [queryCorpus](/essentials/tools#built-in-tools)
tool. to your agents and provide the `corpusId`. For example, if we wanted to create a voice agent to answer questions about Seattle, we could provide the tool like this:
{
"systemPrompt": "Use the queryCorpus tool to answer questions about Seattle.",
"selectedTools": [\
{\
"toolName": "queryCorpus", \
"parameterOverrides": {\
"corpus_id": "",\
"max_results": 5\
}\
}\
]
}
###
[](#the-other-way)
The Other Way
Let’s assume we have already stored our product documentation in a vector database and can search that content at `https://foo.bar/lookupProductInfo`.
Here’s how we might create a tool for our Ultravox agent to use:
Example: Adding a RAG tool
{
"systemPrompt": "You are a helpful assistant. You have a tool called 'lookupProductInfo' that you must use to find answers.",
"model": "fixie-ai/ultravox",
"selectedTools": [\
{\
"temporaryTool": {\
"modelToolName": "lookupProductInfo",\
"description": "Searches official product documentation using semantic similarity to find relevant information. Use this tool to look up specific product features, specifications, limitations, pricing, or support information. The tool returns the most relevant text chunks from the documentation.",\
"dynamicParameters": [\
{\
"name": "query",\
"location": "PARAMETER_LOCATION_BODY",\
"schema": {\
"description": "A specific, focused search query to find relevant product information",\
"type": "string"\
},\
"required": true\
}\
],\
"http": {\
"baseUrlPattern": "https://foo.bar/lookupProductInfo",\
"httpMethod": "POST"\
}\
}\
}\
]
}
[Using Tools](/essentials/tools)
[SDKs](/essentials/sdk)
On this page
* [Examples of Knowledge Sources](#examples-of-knowledge-sources)
* [Adding RAG to Ultravox](#adding-rag-to-ultravox)
* [The Easy Way](#the-easy-way)
* [The Other Way](#the-other-way)
---
# Multilingual Agents - Ultradox
[Ultradox home page](/)
Search or ask...
Search...
Navigation
Essentials
Multilingual Agents
[Documentation](/introduction)
[API Reference](/api-reference/introduction)
[SDK Reference](/sdk-reference/introduction)
[Changelog](/changelog/news)
[Voice Cloning](/essentials/voices)
[Call Stages](/guides/callstages)
---
# Debugging & Troubleshooting - Ultradox
[Ultradox home page](/)
Search or ask...
Search...
Navigation
Advanced Guides + Tutorials
Debugging & Troubleshooting
[Documentation](/introduction)
[API Reference](/api-reference/introduction)
[SDK Reference](/sdk-reference/introduction)
[Changelog](/changelog/news)
[Call Stages](/guides/callstages)
[WebSocket Integration](/guides/websockets)
---
# Data Messages - Ultradox
[Ultradox home page](/)
Search or ask...
Search...
Navigation
Advanced Guides + Tutorials
Data Messages
[Documentation](/introduction)
[API Reference](/api-reference/introduction)
[SDK Reference](/sdk-reference/introduction)
[Changelog](/changelog/news)
Data messages are used to communicate non-audio information between your client and an Ultravox server during calls. These messages work across [WebRTC](/essentials/sdk)
data channels and [WebSocket](/guides/websockets)
connections.
All messages are JSON objects with camelCase keys containing:
* A required `type` field identifying the message type
* Additional fields specific to each message type
[](#messages-at-a-glance)
Messages at a Glance
--------------------------------------------------
This table provides all messages at a glance. Details on each message type appears below. Sender indicates client or server message. Client messages are sent from the client to the server. Server messages are sent from the server to the client.
| Message | Sender | Description |
| --- | --- | --- |
| [Ping](/_sites/docs.ultravox.ai/datamessages#ping) | Client | Measures round-trip data latency. |
| [Pong](/_sites/docs.ultravox.ai/datamessages#pong) | Server | Server reply to a ping message. |
| [State](/_sites/docs.ultravox.ai/datamessages#state) | Server | Indicates the server’s current state. |
| [Transcript](/_sites/docs.ultravox.ai/datamessages#transcript) | Server | Contains text for an utterance made during the call. |
| [InputTextMessage](/_sites/docs.ultravox.ai/datamessages#inputtextmessage) | Client | Used to send a user message to the agent via text. |
| [SetOutputMedium](/_sites/docs.ultravox.ai/datamessages#setoutputmedium) | Client | Sets server’s output medium to text or voice. |
| [ClientToolInvocation](/_sites/docs.ultravox.ai/datamessages#clienttoolinvocation) | Server | Asks the client to invoke a client tool. |
| [ClientToolResult](/_sites/docs.ultravox.ai/datamessages#clienttoolresult) | Client | Contains the result of a client tool invocation. |
| [Debug](/_sites/docs.ultravox.ai/datamessages#debug) | Server | Useful for application debugging. |
| [PlaybackClearBuffer](/_sites/docs.ultravox.ai/datamessages#playbackclearbuffer) | Server | Used to clear buffered output audio. WebSocket only. |
[](#ping)
Ping
------------------
A message sent by the client to measure round-trip data message latency.
* `type: "ping"`
* `timestamp`: Float. Client timestamp for latency measurement.
[](#pong)
Pong
------------------
A message sent by the server in response to a PingMessage. The timestamp is copied from the PingMessage.
* `type: "pong"`
* `timestamp`: Float. Matching ping timestamp.
[](#state)
State
--------------------
A message sent by the server to indicate its current state.
* `type: "state"`
* `state`: Current session state
[](#transcript)
Transcript
------------------------------
A message containing text transcripts of user and agent utterances.
* `type: "transcript"`
* `role`: “user” or “agent”. Who emitted the utterance.
* `medium`: “text” or “voice”. The medium through which the utterance was emitted.
* `text`: String. Full transcript text (exclusive with delta). The full text of the transcript so far. Either this or delta will be set.
* `delta`: String. Incremental transcript update (exclusive with text). The additional transcript text added since the last agent transcript message.
* `final`: Boolean. Whether more updates are expected for this utterance.
* `ordinal`: int. Used for ordering transcripts within a call.
[](#inputtextmessage)
InputTextMessage
------------------------------------------
A user message sent via text.
* `type: "input_text_message"`
* `text`: String. The content of the user message.
[](#setoutputmedium)
SetOutputMedium
----------------------------------------
Message sent by the client to set the server’s output medium.
* `type: "set_output_medium"`
* `medium`: Either “voice” or “text”.
[](#clienttoolinvocation)
ClientToolInvocation
--------------------------------------------------
Sent by the server to ask the client to invoke a client-implemented tool with the given parameters. The client is expected to send back a ClientToolResultMessage with a matching invocation\_id.
* `type: "client_tool_invocation"`
* `tool_name`: String. Tool to invoke
* `invocation_id`: String. Unique invocation ID
* `parameters`: Dict\[String, Any\]. Tool parameters
[](#clienttoolresult)
ClientToolResult
------------------------------------------
Contains the result of a client-implemented tool invocation.
* `type: "client_tool_result"`
* `invocation_id`: String. Matches corresponding invocation.
* `result`: String. Tool execution result. Often a JSON string. May be omitted for errors.
* `response_type`: String. Defaults to “tool-response”.
* `error_type`: Optional string. Should be omitted when result is set. Otherwise, should be “undefined” if the a tool with the given name does not exist or “implementation-error” otherwise.
* `error_message`: String. Error details if failed (optional).
[](#debug)
Debug
--------------------
A message sent by the server to communicate debug information.
* `type: "debug"`
* `message`: String. Debug information
* Disabled by default
[](#playbackclearbuffer)
PlaybackClearBuffer
------------------------------------------------
Message sent by our server to clear buffered output audio. Integrators should drop as much unplayed output audio as possible in order for interruptions to function properly.
* `type: "playback_clear_buffer"`
* WebSocket connections only
[WebSocket Integration](/guides/websockets)
[Webhooks](/guides/webhooks)
On this page
* [Messages at a Glance](#messages-at-a-glance)
* [Ping](#ping)
* [Pong](#pong)
* [State](#state)
* [Transcript](#transcript)
* [InputTextMessage](#inputtextmessage)
* [SetOutputMedium](#setoutputmedium)
* [ClientToolInvocation](#clienttoolinvocation)
* [ClientToolResult](#clienttoolresult)
* [Debug](#debug)
* [PlaybackClearBuffer](#playbackclearbuffer)
---
# WebSocket Integration - Ultradox
[Ultradox home page](/)
Search or ask...
Search...
Navigation
Advanced Guides + Tutorials
WebSocket Integration
[Documentation](/introduction)
[API Reference](/api-reference/introduction)
[SDK Reference](/sdk-reference/introduction)
[Changelog](/changelog/news)
The Ultravox API allows you to create AI-powered voice applications that can interact through various protocols:
* **WebRTC** → Default protocol for browser and mobile applications.
* **Regular Phone Numbers** → Connect Ultravox to phone calls you make or receive (via Telnxy, Twilio, or Plivo).
* **WebSockets** → Direct server-to-server integration.
For more information about WebRTC and Phone, consult the [Telephony](/essentials/telephony)
guide.
**Server-to-Server Only**
WebSocket connections are designed for server-to-server communication. For browser or mobile applications, use our client SDKs with WebRTC for optimal performance. WebSocket connections over TCP can experience audio blocking and ordering constraints that make them unsuitable for direct client use.
###
[](#creating-a-websocket-call)
Creating a WebSocket Call
Creating a WebSocket-based call with Ultravox requires setting `medium` to `serverWebSocket` and passing in parameters for sample rates and buffer size.
* **inputSampleRate** (required): Sample rate for input (user) audio (e.g., 48000).
* **outputSampleRate** (optional): Sample rate for output (agent) audio (defaults to inputSampleRate).
* **clientBufferSizeMs** (optional): Size of the client-side audio buffer in milliseconds. Smaller buffers allow for faster interruptions but may cause audio underflow if network latency fluctuates too greatly. For the best of both worlds, set this to some large value (e.g. 30000) and implement support for [PlaybackClearBuffer](/datamessages#playbackclearbuffer)
messages. (Defaults to 60).
Example: Creating an Ultravox Call for WebSockets
const response = await fetch('https://api.ultravox.ai/api/calls', {
method: 'POST',
headers: {
'X-API-Key': 'your_api_key',
'Content-Type': 'application/json'
},
body: JSON.stringify({
systemPrompt: "You are a helpful assistant...",
model: "fixie-ai/ultravox",
voice: "Mark",
medium: {
serverWebSocket: {
inputSampleRate: 48000,
outputSampleRate: 48000,
}
}
})
});
const { joinUrl } = await response.json();
Example: Joining an Ultravox Call via Websockets
import websockets
socket = await websockets.connect(join_url)
audio_send_task = asyncio.create_task(_send_audio(socket))
async for message in socket:
if isinstance(message, bytes):
# Handle agent audio data
else:
# Handle data message. See "Data Messages"
...
async def _send_audio(socket: websockets.WebSocketClientProtocol):
async for chunk in some_audio_source:
# chunk should be a bytes object containing s16le PCM audio from the user
self._socket.send(chunk)
See [Data Messages](/datamessages)
for more information on all available messages.
**Data Messages**
WebSocket connections use the same message format as WebRTC data channels. See our [Data Messages](/datamessages)
documentation for detailed message specifications.
[Debugging & Troubleshooting](/guides/debugging)
[Data Messages](/datamessages)
On this page
* [Creating a WebSocket Call](#creating-a-websocket-call)
---
# Telephony - Ultradox
[Ultradox home page](/)
Search or ask...
Search...
Navigation
Essentials
Telephony
[Documentation](/introduction)
[API Reference](/api-reference/introduction)
[SDK Reference](/sdk-reference/introduction)
[Changelog](/changelog/news)
The Ultravox API allows you to create AI-powered voice applications that can interact through various protocols:
* **WebRTC** → Default protocol for browser and mobile applications.
* **Regular Phone Numbers** → Connect Ultravox to phone calls you make or receive (via Telnxy, Twilio, or Plivo).
* **WebSockets** → Direct server-to-server integration.
[](#choosing-a-protocol)
Choosing a Protocol
------------------------------------------------
Choose your integration method based on your needs:
* **WebRTC**: Best for most integrations, especially for any client deployment (for example, browsers or mobile clients). This is the default. Get started with the Ultravox client [SDK](/essentials/sdk)
.
* **Phone**: For traditional phone network integration. Ultravox integrates directly with Telnyx, Twilio, and Plivo.
* **WebSocket**: For server-to-server integration, especially when you already have high-bandwidth connections between your server and clients. Check out the [WebSocket Integration](/guides/websockets)
guide for more information.
[](#phone-integration)
Phone Integration
--------------------------------------------
Ultravox integrates with multiple telephony providers, enabling you to create AI-powered voice applications that can interact through regular phone networks. You can build AI agents that can make outgoing calls and answer incoming calls, opening up possibilities for customer service, automated outreach, and other voice-based AI applications.
###
[](#supported-providers)
Supported Providers
* **Twilio** → Uses Twilio [Media Streams](https://www.twilio.com/docs/voice/media-streams)
.
* **Telnyx** → Uses Telnyx [Media Streaming](https://developers.telnyx.com/docs/voice/programmable-voice/media-streaming)
.
* **Plivo** → Uses Plivo [AudioStream](https://www.plivo.com/docs/voice/xml/the-stream-element/)
.
###
[](#prerequisites)
Prerequisites
**Prerequisites**
Make sure you have:
1. An active account with your chosen provider (Telnyx, Twilio, or Plivo)
2. A phone number purchased from your provider
###
[](#connecting-ultravox-to-a-phone-call)
Connecting Ultravox to a Phone Call
**”Calls” is Overloaded**
It can be a bit confusing because Ultravox Realtime uses the concept of a “call” to mean a voice session between an AI agent and another party. For phone calls, you will accept incoming or make outgoing calls via your chosen telephony provider and then connect those calls to an Ultravox call.
Creating an Ultravox call that you can connect with a phone call is similar to creating a WebRTC call, but requires specific parameters in the [Create Call](/api-reference/calls/calls-post)
command:
[](#param-medium)
medium
object
default:
"{'webRtc': {}}"
Tells Ultravox which protocol to use. For phone calls, must be set to one of:
`{"telnyx": {}}`, `{"twilio": {}}`, or `{"plivo": {}}`.
Defaults to `{"webRtc": {}}`.
[](#param-first-speaker)
firstSpeaker
string
default:
"FIRST\_SPEAKER\_AGENT"
Tells Ultravox who should speak first.
For outgoing calls, typically set to `"FIRST_SPEAKER_USER"`.
The default is `"FIRST_SPEAKER_AGENT"`.
Example: Outgoing Call via Telnyx
{
"systemPrompt": "You are a helpful assistant...",
...
"medium": {
"telnyx": {} // or "twilio": {} or "plivo": {}
},
"firstSpeaker": "FIRST_SPEAKER_USER"
}
###
[](#provider-specific-integration)
Provider-Specific Integration
* Telnyx
* Twilio
* Plivo
#### Outgoing Calls with Telnyx
1
Create an Ultravox Call
Create a new call as shown above with `medium: { "telnyx": {} }`, `firstSpeaker: "FIRST_SPEAKER_USER"`, and get a `joinUrl`.
2
Connect Ultravox to the Telnyx Phone Call
Use the `joinUrl` with a TeXML ``:
// Example using the telnyx node library
const call = await telnyx.calls.create({
connection_id: "uuid",
to: phoneNumber,
from: telnyxPhoneNumber,
stream_url: joinUrl,
stream_track: "both_tracks",
stream_bidirectional_mode: "rtp"
});
Or using TeXML:
#### Incoming Calls with Telnyx
1
Create an Ultravox Call
Create a new call with `medium: { "telnyx": {} }` and `firstSpeaker` set to `"FIRST_SPEAKER_AGENT"`.
2
Connect the Inbound Telnyx Call to Ultravox
Use the `joinUrl` with a TeXML ``:
**Telnyx `codec`**
Telnyx allows setting both `codec` and `bidirectionalCodec`. The former controls user audio while the latter controls agent audio. When using with Ultravox, **these must have the same value** because Telnyx only tells us about one of them! If your users are mostly in Europe, you’ll likely want to set both to PCMA. Otherwise setting both to PCMU is preferred but leaving them both unset is fine to get started.
For more details, see the [Telnyx documentation](https://developers.telnyx.com/)
.
**Provider Support**
We currently integrate with Telnyx, Twilio, and Plivo. Please [let us know](https://discord.com/channels/1240071833798184990/1315065334058713198)
if there’s another integration you’d like to see.
###
[](#dtmf)
DTMF
Ultravox provides comprehensive support for DTMF (Dual-Tone Multi-Frequency) tones, enabling both [sending](/_sites/docs.ultravox.ai/essentials/telephony#sending-dtmf-tones)
and [receiving](/_sites/docs.ultravox.ai/essentials/telephony#receiving-dtmf-tones)
tones during phone calls. This enables AI agents to interact with traditional phone systems and allows you to build voice applications that can respond to keypad inputs.
**DTMF and WebRTC**
Due to the audio codec used in WebRTC connections, DTMF tones are inaudible when using WebRTC. The `playDtmfSounds` tool is intended for use with [telephony integrations](/essentials/telephony)
.
####
[](#receiving-dtmf-tones)
Receiving DTMF Tones
Ultravox automatically converts incoming DTMF tones to text, making it easy to build interactive voice applications that respond to keypad input. When a caller presses keys on their phone keypad, the tones are converted to text that your AI agent can understand and respond to.
For example, if a caller presses “5” on their keypad, your agent will receive this as text and can respond accordingly:
// Example system prompt for an agent that handles DTMF input
{
"systemPrompt": `You are an automated phone system.
When a caller joins, say: "Welcome! Press 1 for sales, 2 for support, or 3 for billing."
If they press 1, transfer them to sales using the transfer tool.
If they press 2, transfer them to support.
If they press 3, transfer them to billing.
If they press any other key, ask them to try again with a valid option."`
}
####
[](#sending-dtmf-tones)
Sending DTMF Tones
The [built-in](/essentials/tools#built-in-tools)
`playDtmfSounds` tool allows your AI agent to send DTMF tones, which is useful for navigating Interactive Voice Response (IVR) systems or other phone trees. To enable the tool, add it to the `selectedTools` array when creating a call or call stage:
// Example request body for creating a call with DTMF capability
{
"systemPrompt": "You are a helpful assistant. When prompted to dial an extension, use the 'playDtmfSounds' tool to send the appropriate tones.",
"selectedTools": [\
{ "toolName": "playDtmfSounds" }\
]
}
The `playDtmfSounds` tool accepts a string parameter named `digits` and works with the following tones: 0-9, \*, #, A-D.
For example:
// Example of using the playDtmfSounds tool to dial an extension
{
"digits": "123#" // Will play tones for 1, 2, 3, and # in sequence
}
Note: the `playDtmfSounds` tool uses an [automatic parameter](/essentials/tools#automatic-parameters)
that sends the proper sample rate of the source audio and should be treated as an implementation detail.
####
[](#common-use-cases)
Common Use Cases
* Building interactive phone trees or IVR systems
* Creating agents that can navigate existing phone systems
* Enabling quick responses through keypad input
* Collecting numeric input (e.g., account numbers, PIN codes)
* Building hybrid voice/keypad interfaces
[SDKs](/essentials/sdk)
[Voice Cloning](/essentials/voices)
On this page
* [Choosing a Protocol](#choosing-a-protocol)
* [Phone Integration](#phone-integration)
* [Supported Providers](#supported-providers)
* [Prerequisites](#prerequisites)
* [Connecting Ultravox to a Phone Call](#connecting-ultravox-to-a-phone-call)
* [Provider-Specific Integration](#provider-specific-integration)
* [DTMF](#dtmf)
* [Receiving DTMF Tones](#receiving-dtmf-tones)
* [Sending DTMF Tones](#sending-dtmf-tones)
* [Common Use Cases](#common-use-cases)
---
# Webhooks - Ultradox
[Ultradox home page](/)
Search or ask...
Search...
Navigation
Advanced Guides + Tutorials
Webhooks
[Documentation](/introduction)
[API Reference](/api-reference/introduction)
[SDK Reference](/sdk-reference/introduction)
[Changelog](/changelog/news)
Webhooks allow you to receive real-time notifications about events in your Ultravox account. You can configure webhooks to send HTTP POST requests to a specified URL when certain events occur.
[](#available-events)
Available Events
------------------------------------------
The following events are available and can be specified when creating or updating a webhoook.
| event | description |
| --- | --- |
| call.started | Fired when a call starts. |
| call.ended | Fired when a call ends. |
| call.joined | Fired when a call is joined. |
###
[](#event-payloads)
Event Payloads
The payloads that are sent with each webhook are passed in the request body as follows:
{
"event": {event_name}
"call": {call_object}
}
The `{event_name}` is the same as the event you subscribed to when creating your webhook (e.g. `call.started`). The `{call_object}` matches what is returned from the [Get Call](/api-reference/calls/calls-get)
endpoint.
[](#webhooks-in-ultravox)
Webhooks in Ultravox
--------------------------------------------------
When creating a webhook, you must provide a URL that will receive the webhook notification. Your service must return an acceptable HTTP status code.
###
[](#http-status-codes)
HTTP Status Codes
Your service should return a 2xx status code (we recommend 204) to confirm receipt of all webhooks. Any 4xx or 5xx response will result in a [retry](/_sites/docs.ultravox.ai/guides/webhooks#retries)
.
###
[](#securing-webhooks)
Securing Webhooks
You can optionally choose to secure your webhooks with a key. When creating a webhook, a secret key is automatically generated for you or you can choose to provide your own secret. You can update or patch your webhooks to change secrets in the event of a leak or as part of regular key rotation.
Each time your server receives an incoming webhook from Ultravox here’s how you can ensure the webhook was sent by Ultravox and hasn’t been tampered with:
1
Timestamp Verification
* Each incoming webhook request includes a `X-Ultravox-Webhook-Timestamp` header with the time the webhook was sent.
* Verify that this timestamp is recent (e.g. within the last minute) to prevent replay attacks.
2
Signature Verification
* Ultravox signs each webhook using HMAC-SHA256.
* The signature is included in the `X-Ultravox-Webhook-Signature` header.
* To verify the signature:
* Concatenate the raw request body with the timestamp.
* Create an HMAC-SHA256 hash of this concatenated string using your webhook secret as the key.
* Compare this hash with the provided signature.
Verifying Webhook Signature
import datetime
import hmac
request_timestamp = request.headers["X-Ultravox-Webhook-Timestamp"]
if datetime.datetime.now() - datetime.datetime.fromisoformat(request_timestamp) > datetime.timedelta(minutes=1):
raise RuntimeError("Expired message")
expected_signature = hmac.new(WEBHOOK_SECRET.encode(), request.content + request_timestamp.encode(), "sha256").hexdigest()
for signature in request.headers["X-Ultravox-Webhook-Signature"].split(","):
if hmac.compare_digest(signature, expected_signature):
break # Valid signature
else:
raise RuntimeError("Message or timestamp was tampered with")
3
Multiple Signatures
* `The X-Ultravox-Webhook-Signature` header may contain multiple signatures separated by commas.
* This allows for key rotation without downtime.
* Your code should check if any of the provided signatures match your computed signature.
By implementing these checks, you ensure that only authentic, recent, and unmodified webhooks from Ultravox are processed by your system. Remember to store your webhook secret securely and never expose it in client-side code or public repositories.
###
[](#retries)
Retries
When a destination URL fails to respond to an incoming webhook with a 2xx, we will retry using a random exponential backoff strategy as follows:
* First retry will occur approximately 30 seconds later.
* Subsequent retries will double the retry interval. (e.g. second retry again after 1m, third retry after 2m, etc.)
* Total of 10 retries.
If your receiving service is still down, you can use the API to retrieve information about any calls that occurred.
[Data Messages](/datamessages)
[Tutorial: Customer Escalation with Call Stages](/tutorials/callstages)
On this page
* [Available Events](#available-events)
* [Event Payloads](#event-payloads)
* [Webhooks in Ultravox](#webhooks-in-ultravox)
* [HTTP Status Codes](#http-status-codes)
* [Securing Webhooks](#securing-webhooks)
* [Retries](#retries)
---
# Get Account - Ultradox
[Ultradox home page](/)
Search or ask...
Search...
Navigation
Accounts
Get Account
[Documentation](/introduction)
[API Reference](/api-reference/introduction)
[SDK Reference](/sdk-reference/introduction)
[Changelog](/changelog/news)
GET
/
api
/
accounts
/
me
Try it
cURL
Python
JavaScript
PHP
Go
Java
curl --request GET \
--url https://api.ultravox.ai/api/accounts/me \
--header 'X-API-Key: '
200
{
"name": "",
"billingUrl": "",
"freeTimeUsed": "",
"freeTimeRemaining": "",
"hasActiveSubscription": true,
"activeCalls": 123,
"allowedConcurrentCalls": 123,
"allowedVoices": 123
}
#### Authorizations
[](#authorization-x-api-key)
X-API-Key
string
header
required
API key
#### Response
200 - application/json
[](#response-name)
name
string
required
[](#response-billing-url)
billingUrl
string
required
[](#response-free-time-used)
freeTimeUsed
string
required
How much free time has been used by previous (or ongoing) calls.
[](#response-free-time-remaining)
freeTimeRemaining
string
required
How much free call time this account has remaining. (This could increase if an existing call ends without using its maximum duration or an unjoined call times out.)
[](#response-has-active-subscription)
hasActiveSubscription
boolean
required
Whether the account has an active subscription.
[](#response-active-calls)
activeCalls
integer
required
The number of active calls for this account.
[](#response-allowed-concurrent-calls)
allowedConcurrentCalls
integer
required
The maximum number of concurrent calls allowed for this account.
[](#response-allowed-voices)
allowedVoices
integer
required
The maximum number of custom voices allowed for this account.
[Ultravox REST API Overview](/api-reference/introduction)
[Calls Overview](/api-reference/calls/overview)
cURL
Python
JavaScript
PHP
Go
Java
curl --request GET \
--url https://api.ultravox.ai/api/accounts/me \
--header 'X-API-Key: '
200
{
"name": "",
"billingUrl": "",
"freeTimeUsed": "",
"freeTimeRemaining": "",
"hasActiveSubscription": true,
"activeCalls": 123,
"allowedConcurrentCalls": 123,
"allowedVoices": 123
}
---
# Outgoing Phone Calls - Ultradox
[Ultradox home page](/)
Search or ask...
Search...
Navigation
Phone Quickstart
Outgoing Phone Calls
[Documentation](/introduction)
[API Reference](/api-reference/introduction)
[SDK Reference](/sdk-reference/introduction)
[Changelog](/changelog/news)
This guide will help you set up and make your first automated call using Ultravox and Twilio.
[](#prerequisites)
Prerequisites
------------------------------------
* Node.js 20 or higher
* A Twilio account with:
* Account SID
* Auth Token
* Phone Number
* An Ultravox API key
[](#set-up-and-installation)
Set-up and Installation
--------------------------------------------------------
1
Get the Source Code
Copy all the code locally from the [`twilio-outgoing-quickstart-js`](https://github.com/fixie-ai/ultradox/tree/main/examples/twilio-outgoing-quickstart-js)
example.
2
Install the Required Dependencies
pnpm install
or
npm install
[](#making-your-first-call)
Making Your First Call
------------------------------------------------------
The AI assistant will introduce itself as Steve and have a conversation with the recipient. To make a call you need to update the variables for keys and phone numbers. You may also update the system prompt.
###
[](#update-configuration)
Update Configuration
* `TWILIO_ACCOUNT_SID`: Your Twilio Account SID from the Twilio Console
* `TWILIO_AUTH_TOKEN`: Your Twilio Auth Token
* `TWILIO_PHONE_NUMBER`: The Twilio phone number to make calls from
* `DESTINATION_PHONE_NUMBER`: The recipient’s phone number
* `ULTRAVOX_API_KEY`: Your Ultravox API key
* `SYSTEM_PROMPT`: Instructions for the AI agent’s behavior
import twilio from 'twilio';
import https from 'https';
// Twilio configuration
const TWILIO_ACCOUNT_SID = 'your_twilio_account_sid_here';
const TWILIO_AUTH_TOKEN = 'your_twilio_auth_token_here';
const TWILIO_PHONE_NUMBER = 'your_twilio_phone_number_here';
const DESTINATION_PHONE_NUMBER = 'the_destination_phone_number_here';
// Ultravox configuration
const ULTRAVOX_API_KEY = 'your_ultravox_api_key_here';
const SYSTEM_PROMPT = 'Your name is Steve and you are calling a person on the phone. Ask them their name and see how they are doing.';
###
[](#start-the-call)
Start the Call
Once you’ve configured and saved everything, start the call:
pnpm start
[](#next-steps)
Next Steps
------------------------------
Ultravox Realtime provides telephony integrations for Telnyx, Twilio, and Plivo. Learn more [here](/essentials/telephony)
.
[Prompting Guide](/gettingstarted/prompting)
[Incoming Phone Calls](/gettingstarted/quickstart-phone-incoming)
On this page
* [Prerequisites](#prerequisites)
* [Set-up and Installation](#set-up-and-installation)
* [Making Your First Call](#making-your-first-call)
* [Update Configuration](#update-configuration)
* [Start the Call](#start-the-call)
* [Next Steps](#next-steps)
---
# List Calls - Ultradox
[Ultradox home page](/)
Search or ask...
Search...
Navigation
Calls, Messages, Stages
List Calls
[Documentation](/introduction)
[API Reference](/api-reference/introduction)
[SDK Reference](/sdk-reference/introduction)
[Changelog](/changelog/news)
GET
/
api
/
calls
Try it
cURL
Python
JavaScript
PHP
Go
Java
curl --request GET \
--url https://api.ultravox.ai/api/calls \
--header 'X-API-Key: '
200
{
"next": "http://api.example.org/accounts/?cursor=cD00ODY%3D\"",
"previous": "http://api.example.org/accounts/?cursor=cj0xJnA9NDg3",
"results": [\
{\
"callId": "3c90c3cc-0d44-4b50-8888-8dd25736052a",\
"clientVersion": "",\
"created": "2023-11-07T05:31:56Z",\
"joined": "2023-11-07T05:31:56Z",\
"ended": "2023-11-07T05:31:56Z",\
"endReason": "unjoined",\
"firstSpeaker": "FIRST_SPEAKER_AGENT",\
"firstSpeakerSettings": {\
"user": {},\
"agent": {\
"uninterruptible": true,\
"text": ""\
}\
},\
"inactivityMessages": [\
{\
"duration": "",\
"message": "",\
"endBehavior": "END_BEHAVIOR_UNSPECIFIED"\
}\
],\
"initialOutputMedium": "MESSAGE_MEDIUM_VOICE",\
"joinTimeout": "30s",\
"joinUrl": "",\
"languageHint": "",\
"maxDuration": "3600s",\
"medium": {\
"webRtc": {},\
"twilio": {},\
"serverWebSocket": {\
"inputSampleRate": 123,\
"outputSampleRate": 123,\
"clientBufferSizeMs": 123\
},\
"telnyx": {},\
"plivo": {},\
"exotel": {}\
},\
"model": "fixie-ai/ultravox",\
"recordingEnabled": false,\
"systemPrompt": "",\
"temperature": 0,\
"timeExceededMessage": "",\
"voice": "",\
"transcriptOptional": true,\
"errorCount": 0,\
"vadSettings": {\
"turnEndpointDelay": "",\
"minimumTurnDuration": "",\
"minimumInterruptionDuration": "",\
"frameActivationThreshold": 123\
},\
"shortSummary": "",\
"summary": "",\
"experimentalSettings": ""\
}\
],
"total": 123
}
#### Authorizations
[](#authorization-x-api-key)
X-API-Key
string
header
required
API key
#### Query Parameters
[](#parameter-cursor)
cursor
string
The pagination cursor value.
[](#parameter-page-size)
pageSize
integer
Number of results to return per page.
#### Response
200 - application/json
[](#response-results)
results
object\[\]
required
Show child attributes
[](#response-results-call-id)
results.callId
string
required
[](#response-results-client-version)
results.clientVersion
string | null
required
The version of the client that joined this call.
[](#response-results-created)
results.created
string
required
[](#response-results-joined)
results.joined
string | null
required
[](#response-results-ended)
results.ended
string | null
required
[](#response-results-end-reason)
results.endReason
enum | nullany | null
required
The reason the call ended.
* `unjoined` - Client never joined
* `hangup` - Client hung up
* `agent_hangup` - Agent hung up
* `timeout` - Call timed out
* `connection_error` - Connection error
Available options:
`unjoined`,
`hangup`,
`agent_hangup`,
`timeout`,
`connection_error`
[](#response-results-first-speaker)
results.firstSpeaker
enum
requireddeprecated
Who was supposed to talk first when the call started. Typically set to FIRST\_SPEAKER\_USER for outgoing calls and left as the default (FIRST\_SPEAKER\_AGENT) otherwise.
Available options:
`FIRST_SPEAKER_AGENT`,
`FIRST_SPEAKER_USER`
[](#response-results-first-speaker-settings)
results.firstSpeakerSettings
object
required
Settings for the initial message to get the call started.
Show child attributes
[](#response-results-first-speaker-settings-user)
results.firstSpeakerSettings.user
object
If set, the user should speak first.
[](#response-results-first-speaker-settings-agent)
results.firstSpeakerSettings.agent
object
If set, the agent should speak first.
Show child attributes
[](#response-results-first-speaker-settings-agent-uninterruptible)
results.firstSpeakerSettings.agent.uninterruptible
boolean
Whether the user should be prevented from interrupting the agent's first message. Defaults to false (meaning the agent is interruptible as usual).
[](#response-results-first-speaker-settings-agent-text)
results.firstSpeakerSettings.agent.text
string
What the agent should say. If unset, the model will generate a greeting.
[](#response-results-initial-output-medium)
results.initialOutputMedium
enum
required
The medium used initially by the agent. May later be changed by the client.
Available options:
`MESSAGE_MEDIUM_VOICE`,
`MESSAGE_MEDIUM_TEXT`
[](#response-results-join-url)
results.joinUrl
string | null
required
[](#response-results-error-count)
results.errorCount
integer
default:
0
required
The number of errors in this call.
[](#response-results-short-summary)
results.shortSummary
string | null
required
A short summary of the call.
[](#response-results-summary)
results.summary
string | null
required
A summary of the call.
[](#response-results-experimental-settings)
results.experimentalSettings
any
required
Experimental settings for the call.
[](#response-results-inactivity-messages)
results.inactivityMessages
object\[\]
Messages spoken by the agent when the user is inactive for the specified duration. Durations are cumulative, so a message m > 1 with duration 30s will be spoken 30 seconds after message m-1.
A message the agent should say after some duration. The duration's meaning varies depending on the context.
Show child attributes
[](#response-results-inactivity-messages-duration)
results.inactivityMessages.duration
string
The duration after which the message should be spoken.
[](#response-results-inactivity-messages-message)
results.inactivityMessages.message
string
The message to speak.
[](#response-results-inactivity-messages-end-behavior)
results.inactivityMessages.endBehavior
enum
The behavior to exhibit when the message is finished being spoken.
Available options:
`END_BEHAVIOR_UNSPECIFIED`,
`END_BEHAVIOR_HANG_UP_SOFT`,
`END_BEHAVIOR_HANG_UP_STRICT`
[](#response-results-join-timeout)
results.joinTimeout
string
default:
30s
[](#response-results-language-hint)
results.languageHint
string | null
BCP47 language code that may be used to guide speech recognition.
Maximum length: `16`
[](#response-results-max-duration)
results.maxDuration
string
default:
3600s
[](#response-results-medium)
results.medium
object
Details about a call's protocol. By default, calls occur over WebRTC using the Ultravox client SDK. Setting a different call medium will prepare the server for a call using a different protocol. At most one call medium may be set.
Show child attributes
[](#response-results-medium-web-rtc)
results.medium.webRtc
object
The call will use WebRTC with the Ultravox client SDK. This is the default.
[](#response-results-medium-twilio)
results.medium.twilio
object
The call will use Twilio's "Media Streams" protocol. Once you have a join URL from starting a call, include it in your TwiML like so: This works for both inbound and outbound calls.
[](#response-results-medium-server-web-socket)
results.medium.serverWebSocket
object
The call will use a plain websocket connection. This is unlikely to yield an acceptable user experience if used from a browser or mobile client, but may be suitable for a server-to-server connection. This option provides a simple way to connect your own server to an Ultravox inference instance.
Show child attributes
[](#response-results-medium-server-web-socket-input-sample-rate)
results.medium.serverWebSocket.inputSampleRate
integer
The sample rate for input (user) audio. Required.
[](#response-results-medium-server-web-socket-output-sample-rate)
results.medium.serverWebSocket.outputSampleRate
integer
The desired sample rate for output (agent) audio. If unset, defaults to the input\_sample\_rate.
[](#response-results-medium-server-web-socket-client-buffer-size-ms)
results.medium.serverWebSocket.clientBufferSizeMs
integer
The size of the client-side audio buffer in milliseconds. Smaller buffers allow for faster interruptions but may cause audio underflow if network latency fluctuates too greatly. For the best of both worlds, set this to some large value (e.g. 30000) and implement support for playback\_clear\_buffer messages. Defaults to 60.
[](#response-results-medium-telnyx)
results.medium.telnyx
object
The call will use Telnyx's media streaming protocol. Once you have a join URL from starting a call, include it in your TexML like so: This works for both inbound and outbound calls.
[](#response-results-medium-plivo)
results.medium.plivo
object
The call will use Plivo's AudioStreams protocol. Once you have a join URL from starting a call, include it in your Plivo XML like so: ${your-join-url} This works for both inbound and outbound calls.
[](#response-results-medium-exotel)
results.medium.exotel
object
The call will use Exotel's "Voicebot" protocol. Once you have a join URL from starting a call, provide it to Exotel as the wss target URL for your Voicebot (either directly or more likely dynamically from your own server).
[](#response-results-model)
results.model
string
default:
fixie-ai/ultravox
[](#response-results-recording-enabled)
results.recordingEnabled
boolean
default:
false
[](#response-results-system-prompt)
results.systemPrompt
string | null
[](#response-results-temperature)
results.temperature
number
default:
0
Required range: `0 <= x <= 1`
[](#response-results-time-exceeded-message)
results.timeExceededMessage
string | null
[](#response-results-voice)
results.voice
string | null
[](#response-results-transcript-optional)
results.transcriptOptional
boolean
default:
true
deprecated
Indicates whether a transcript is optional for the call.
[](#response-results-vad-settings)
results.vadSettings
object
VAD settings for the call.
Show child attributes
[](#response-results-vad-settings-turn-endpoint-delay)
results.vadSettings.turnEndpointDelay
string
The minimum amount of time the agent will wait to respond after the user seems to be done speaking. Increasing this value will make the agent less eager to respond, which may increase perceived response latency but will also make the agent less likely to jump in before the user is really done speaking.
Built-in VAD currently operates on 32ms frames, so only multiples of 32ms are meaningful. (Anything from 1ms to 31ms will produce the same result.)
Defaults to "0.384s" (384ms) as a starting point, but there's nothing special about this value aside from it corresponding to 12 VAD frames.
[](#response-results-vad-settings-minimum-turn-duration)
results.vadSettings.minimumTurnDuration
string
The minimum duration of user speech required to be considered a user turn. Increasing this value will cause the agent to ignore short user audio. This may be useful in particularly noisy environments, but it comes at the cost of possibly ignoring very short user responses such as "yes" or "no".
Defaults to "0s" meaning the agent considers all user audio inputs (that make it through built-in noise cancellation).
[](#response-results-vad-settings-minimum-interruption-duration)
results.vadSettings.minimumInterruptionDuration
string
The minimum duration of user speech required to interrupt the agent. This works the same way as minimumTurnDuration, but allows for a higher threshold for interrupting the agent. (This value will be ignored if it is less than minimumTurnDuration.)
Defaults to "0.09s" (90ms) as a starting point, but there's nothing special about this value.
[](#response-results-vad-settings-frame-activation-threshold)
results.vadSettings.frameActivationThreshold
number
The threshold for the VAD to consider a frame as speech. This is a value between 0.1 and 1.
Miniumum value is 0.1, which is the default value.
[](#response-next)
next
string | null
[](#response-previous)
previous
string | null
[](#response-total)
total
integer
[Calls Overview](/api-reference/calls/overview)
[Get Call](/api-reference/calls/calls-get)
cURL
Python
JavaScript
PHP
Go
Java
curl --request GET \
--url https://api.ultravox.ai/api/calls \
--header 'X-API-Key: '
200
{
"next": "http://api.example.org/accounts/?cursor=cD00ODY%3D\"",
"previous": "http://api.example.org/accounts/?cursor=cj0xJnA9NDg3",
"results": [\
{\
"callId": "3c90c3cc-0d44-4b50-8888-8dd25736052a",\
"clientVersion": "",\
"created": "2023-11-07T05:31:56Z",\
"joined": "2023-11-07T05:31:56Z",\
"ended": "2023-11-07T05:31:56Z",\
"endReason": "unjoined",\
"firstSpeaker": "FIRST_SPEAKER_AGENT",\
"firstSpeakerSettings": {\
"user": {},\
"agent": {\
"uninterruptible": true,\
"text": ""\
}\
},\
"inactivityMessages": [\
{\
"duration": "",\
"message": "",\
"endBehavior": "END_BEHAVIOR_UNSPECIFIED"\
}\
],\
"initialOutputMedium": "MESSAGE_MEDIUM_VOICE",\
"joinTimeout": "30s",\
"joinUrl": "",\
"languageHint": "",\
"maxDuration": "3600s",\
"medium": {\
"webRtc": {},\
"twilio": {},\
"serverWebSocket": {\
"inputSampleRate": 123,\
"outputSampleRate": 123,\
"clientBufferSizeMs": 123\
},\
"telnyx": {},\
"plivo": {},\
"exotel": {}\
},\
"model": "fixie-ai/ultravox",\
"recordingEnabled": false,\
"systemPrompt": "",\
"temperature": 0,\
"timeExceededMessage": "",\
"voice": "",\
"transcriptOptional": true,\
"errorCount": 0,\
"vadSettings": {\
"turnEndpointDelay": "",\
"minimumTurnDuration": "",\
"minimumInterruptionDuration": "",\
"frameActivationThreshold": 123\
},\
"shortSummary": "",\
"summary": "",\
"experimentalSettings": ""\
}\
],
"total": 123
}
---
# Calls Overview - Ultradox
[Ultradox home page](/)
Search or ask...
Search...
Navigation
Calls, Messages, Stages
Calls Overview
[Documentation](/introduction)
[API Reference](/api-reference/introduction)
[SDK Reference](/sdk-reference/introduction)
[Changelog](/changelog/news)
This section of the API reference covers the APIs for managing voice calls. The APIs provide methods to create and manage calls, retrieve recordings, handle user inactivity, and access conversation history through stages and messages. The system supports features like call recording, language hints, and maximum duration limits to help manage the conversation flow.
**Calls** → At the core are Calls, which represent complete conversations between users and AI agents, configured with parameters like system prompts, voice settings, and language preferences.
**Stages** → Calls can be broken down into Stages, which represent distinct segments of conversation where different parameters (like prompts or tools) may be used.
**Messages** → Within each stage, Messages capture the back-and-forth dialogue between users and agents.
[](#more-info)
More Info
----------------------------
This section contains additional details for some properties.
###
[](#inactivitymessages)
inactivityMessages
Inactivity messages allow your agent to gracefully handle periods of user silence and end the call after a period of user inactivity. This feature helps maintain engagement and ensures calls don’t remain open indefinitely when users have disconnected or finished their interaction.
* **Messages are Ordered** → Messages are delivered by the agent in the order provided.
* **Message Durations are Cumulative** → The first message is delivered when the user has been inactive for its duration. Each subsequent message m is delivered its duration after message m-1 (provided the user remains inactive).
* **User Interactions Reset** → Any activity from the user will reset the message sequence.
* **Different Behaviors** → Messages can have different end behaviors and can terminate the call.
**Best Practices**
* Keep messages concise and natural-sounding.
* Start with friendly check-in messages before moving to call termination.
* Provide clear context in messages if the call will be terminated.
**Message Format**
When creating a new call, `inactivityMessages` are an array of message objects.
[](#param-message)
message
Message Object
Hide properties
[](#param-duration)
duration
string
required
The duration (in seconds) after which the message should be spoken.
Pattern: `^-?(?:0|[1-9][0-9]{0,11})(?:\.[0-9]{1,9})?s$`
Examples: ”60s”, “5.5s”
[](#param-message-1)
message
string
required
The message to speak.
Examples: “Are you still there?”, “If there’s nothing else, I will end the call now.”
[](#param-end-behavior)
endBehavior
string
The behavior to exhibit when the message is finished being spoken. Must be one of the enumerated values.
Possible values:
`END_BEHAVIOR_UNSPECIFIED` - Default. The system will continue to wait for user input.
`END_BEHAVIOR_HANG_UP_SOFT` - Will end the call unless the user interacts while the agent is delivering the message.
`END_BEHAVIOR_HANG_UP_STRICT` - Will end the call after speaking the message, regardless of whether the user interrupts.
####
[](#example%3A-adding-inactivity-messages)
Example: Adding Inactivity Messages
Let’s look at how we could add inactivity messages to a call.
Example: Adding Inactivity Messages
{
"systemPrompt": "You are a helpful assistant.",
"inactivityMessages": [\
{\
"duration": "30s",\
"message": "Are you still there?"\
},\
{\
"duration": "15s",\
"message": "If there's nothing else, may I end the call?"\
},\
{\
"duration": "10s",\
"message": "Thank you for calling. Have a great day. Goodbye.",\
"endBehavior": "END_BEHAVIOR_HANG_UP_SOFT"\
}\
]
}
1
Call Starts
Using the `inactivityMessages` above, the call is created and joined.
2
User Stops Interacting - First Message
After 30 seconds of no user interaction, agent says “Are you still there?”.
If user interacts, call continues.
3
Inactivity Continues - Second Message
If no user interaction occurs for another 15 seconds, agent says “If there’s nothing else, may I end the call?”.
4
Inactivity Continues - Call Ends
If no user interaction occurs for another 10 seconds, agent says the provided message and the call ends unless the agent is interrupted during this final message.
###
[](#initialmessages)
initialMessages
When creating a new call or a new call stage, you can provide messages to the agent via `initialMessages`. By default, new calls don’t have initial messages and call stages inherit the prior stage’s messages. New calls will inherit messages if `priorCallId` is set.
These messages can serve the purpose of giving the agent call history or to give examples for few-shotting (e.g. if you want the agent to learn how to respond in a specific way to user input).
####
[](#message-format)
Message Format
`initialMessages` must be an array of message objects where each message contains a `role` and `text`.
See “Response” under [List Call Messages](/api-reference/calls/calls-messages-list)
for more details.
Example: Providing Initial Messages
{
"systemPrompt": "You are a helpful assistant.",
"initialMessages": [\
{\
"role": "MESSAGE_ROLE_USER",\
"text": "My name is Steve"\
},\
{\
"role": "MESSAGE_ROLE_AGENT",\
"text": "Great to meet you, Steve! How can I help?"\
},\
]
}
[Get Account](/api-reference/accounts/accounts-me-get)
[List Calls](/api-reference/calls/calls-list)
On this page
* [More Info](#more-info)
* [inactivityMessages](#inactivitymessages)
* [Example: Adding Inactivity Messages](#example%3A-adding-inactivity-messages)
* [initialMessages](#initialmessages)
* [Message Format](#message-format)
---
# Deprecation Guide - Ultradox
[Ultradox home page](/)
Search or ask...
Search...
Navigation
Changelog
Deprecation Guide
[Documentation](/introduction)
[API Reference](/api-reference/introduction)
[SDK Reference](/sdk-reference/introduction)
[Changelog](/changelog/news)
[](#active-deprecations)
Active Deprecations
------------------------------------------------
###
[](#currently-scheduled-changes)
Currently Scheduled Changes
| Feature/API | Status | Deprecation Date | Removal Date | Migration Guidance |
| --- | --- | --- | --- | --- |
| `initiator` | ⚠️ < 30 days | 2024-10-01 | 2024-12-31 | [Guide](/changelog/migration/firstspeaker) |
[](#deprecation-process)
Deprecation Process
------------------------------------------------
We recognize that breaking changes and deprecation notices are not fun and we try to avoid them when possible.
However, the Ultravox APIs have not yet reached v1 and we are committed to having our APIs and SDKs work better and be as clear as possible. This means we will inevitably need to revisit some choices early on.
This process will evolve as the APIs mature. Please share your feedback with us if you’d like to see any changes to the process or policy.
###
[](#lifecycle-stages)
Lifecycle Stages
* 1\. Announcement
* 2\. Deprecation Period
* 3\. Removal
* Feature marked as deprecated in documentation
* Migration guidance published
[](#deprecation-policy)
Deprecation Policy
----------------------------------------------
###
[](#standard-timeline)
Standard Timeline
* Pre-release features: 30-day minimum deprecation period
* Breaking changes require publication of migration guidance
###
[](#security-exceptions)
Security Exceptions
Critical security updates may bypass the standard deprecation timeline. These will be:
* Clearly marked and documented
* Communicated directly to affected users
* Accompanied by immediate mitigation steps
[](#migration-guides)
Migration Guides
------------------------------------------
###
[](#current-migrations)
Current Migrations
* [Migrating from Call `initiator`](/changelog/migration/firstspeaker)
**Need Help?**
If you need assistance with a migration, please visit our [Discord community](https://discord.gg/Qw6KHxv8YB)
.
[News & Updates](/changelog/news)
On this page
* [Active Deprecations](#active-deprecations)
* [Currently Scheduled Changes](#currently-scheduled-changes)
* [Deprecation Process](#deprecation-process)
* [Lifecycle Stages](#lifecycle-stages)
* [Deprecation Policy](#deprecation-policy)
* [Standard Timeline](#standard-timeline)
* [Security Exceptions](#security-exceptions)
* [Migration Guides](#migration-guides)
* [Current Migrations](#current-migrations)
---
# Available Models - Ultradox
[Ultradox home page](/)
Search or ask...
Search...
Navigation
Available Models
[Documentation](/introduction)
[API Reference](/api-reference/introduction)
[SDK Reference](/sdk-reference/introduction)
[Changelog](/changelog/news)
The Ultravox API currently provides the following models.
| Model | Description |
| --- | --- |
| fixie-ai/ultravox | This is an alias that points to `ultravox-70B`. The default if no model is specified when creating a call with the API. |
| fixie-ai/ultravox-70B | Based on Llama 3.3 70B. Supports tools. |
| fixie-ai/ultravox-8B | Based on Llama 3.3 8B. Not recommended for most use cases. Tools are unlikely to work. |
| Model | Description |
| --- | --- |
| fixie-ai/ultravox | This is an alias that points to `ultravox-70B`. The default if no model is specified when creating a call with the API. |
| fixie-ai/ultravox-70B | Based on Llama 3.3 70B. Supports tools. |
| fixie-ai/ultravox-8B | Based on Llama 3.3 8B. Not recommended for most use cases. Tools are unlikely to work. |
[](#using-models)
Using Models
----------------------------------
Each model can be used when [creating a call](./api/calls/#create-call)
via the API. For example:
// Example request body to create an Ultravox call
{
"systemPrompt": "You are a helpful assistant...",
"model": "fixie-ai/ultravox-70B"
}
On this page
* [Using Models](#using-models)
---
# List Call Messages - Ultradox
[Ultradox home page](/)
Search or ask...
Search...
Navigation
Calls, Messages, Stages
List Call Messages
[Documentation](/introduction)
[API Reference](/api-reference/introduction)
[SDK Reference](/sdk-reference/introduction)
[Changelog](/changelog/news)
GET
/
api
/
calls
/
{call\_id}
/
messages
Try it
cURL
Python
JavaScript
PHP
Go
Java
curl --request GET \
--url https://api.ultravox.ai/api/calls/{call_id}/messages \
--header 'X-API-Key: '
200
{
"next": "http://api.example.org/accounts/?cursor=cD00ODY%3D\"",
"previous": "http://api.example.org/accounts/?cursor=cj0xJnA9NDg3",
"results": [\
{\
"role": "MESSAGE_ROLE_UNSPECIFIED",\
"text": "",\
"invocationId": "",\
"toolName": "",\
"errorDetails": "",\
"medium": "MESSAGE_MEDIUM_UNSPECIFIED",\
"callStageMessageIndex": 123,\
"callStageId": ""\
}\
],
"total": 123
}
#### Authorizations
[](#authorization-x-api-key)
X-API-Key
string
header
required
API key
#### Path Parameters
[](#parameter-call-id)
call\_id
string
required
#### Query Parameters
[](#parameter-cursor)
cursor
string
The pagination cursor value.
[](#parameter-page-size)
pageSize
integer
Number of results to return per page.
#### Response
200 - application/json
[](#response-results)
results
object\[\]
required
A message exchanged during a call.
Show child attributes
[](#response-results-role)
results.role
enum
The message's role.
Available options:
`MESSAGE_ROLE_UNSPECIFIED`,
`MESSAGE_ROLE_USER`,
`MESSAGE_ROLE_AGENT`,
`MESSAGE_ROLE_TOOL_CALL`,
`MESSAGE_ROLE_TOOL_RESULT`
[](#response-results-text)
results.text
string
The message text for user and agent messages, tool arguments for tool\_call messages, tool results for tool\_result messages.
[](#response-results-invocation-id)
results.invocationId
string
The invocation ID for tool messages. Used to pair tool calls with their results.
[](#response-results-tool-name)
results.toolName
string
The tool name for tool messages.
[](#response-results-error-details)
results.errorDetails
string
For failed tool calls, additional debugging information. While the text field is presented to the model so it can respond to failures gracefully, the full details are only exposed via the Ultravox REST API.
[](#response-results-medium)
results.medium
enum
The medium of the message.
Available options:
`MESSAGE_MEDIUM_UNSPECIFIED`,
`MESSAGE_MEDIUM_VOICE`,
`MESSAGE_MEDIUM_TEXT`
[](#response-results-call-stage-message-index)
results.callStageMessageIndex
integer
The index of the message within the call stage.
[](#response-results-call-stage-id)
results.callStageId
string
The call stage this message appeared in.
[](#response-next)
next
string | null
[](#response-previous)
previous
string | null
[](#response-total)
total
integer
[Create Call](/api-reference/calls/calls-post)
[List Call Tools](/api-reference/calls/calls-tools-list)
cURL
Python
JavaScript
PHP
Go
Java
curl --request GET \
--url https://api.ultravox.ai/api/calls/{call_id}/messages \
--header 'X-API-Key: '
200
{
"next": "http://api.example.org/accounts/?cursor=cD00ODY%3D\"",
"previous": "http://api.example.org/accounts/?cursor=cj0xJnA9NDg3",
"results": [\
{\
"role": "MESSAGE_ROLE_UNSPECIFIED",\
"text": "",\
"invocationId": "",\
"toolName": "",\
"errorDetails": "",\
"medium": "MESSAGE_MEDIUM_UNSPECIFIED",\
"callStageMessageIndex": 123,\
"callStageId": ""\
}\
],
"total": 123
}
---
# Get Call - Ultradox
[Ultradox home page](/)
Search or ask...
Search...
Navigation
Calls, Messages, Stages
Get Call
[Documentation](/introduction)
[API Reference](/api-reference/introduction)
[SDK Reference](/sdk-reference/introduction)
[Changelog](/changelog/news)
GET
/
api
/
calls
/
{call\_id}
Try it
cURL
Python
JavaScript
PHP
Go
Java
curl --request GET \
--url https://api.ultravox.ai/api/calls/{call_id} \
--header 'X-API-Key: '
200
{
"callId": "3c90c3cc-0d44-4b50-8888-8dd25736052a",
"clientVersion": "",
"created": "2023-11-07T05:31:56Z",
"joined": "2023-11-07T05:31:56Z",
"ended": "2023-11-07T05:31:56Z",
"endReason": "unjoined",
"firstSpeaker": "FIRST_SPEAKER_AGENT",
"firstSpeakerSettings": {
"user": {},
"agent": {
"uninterruptible": true,
"text": ""
}
},
"inactivityMessages": [\
{\
"duration": "",\
"message": "",\
"endBehavior": "END_BEHAVIOR_UNSPECIFIED"\
}\
],
"initialOutputMedium": "MESSAGE_MEDIUM_VOICE",
"joinTimeout": "30s",
"joinUrl": "",
"languageHint": "",
"maxDuration": "3600s",
"medium": {
"webRtc": {},
"twilio": {},
"serverWebSocket": {
"inputSampleRate": 123,
"outputSampleRate": 123,
"clientBufferSizeMs": 123
},
"telnyx": {},
"plivo": {},
"exotel": {}
},
"model": "fixie-ai/ultravox",
"recordingEnabled": false,
"systemPrompt": "",
"temperature": 0,
"timeExceededMessage": "",
"voice": "",
"transcriptOptional": true,
"errorCount": 0,
"vadSettings": {
"turnEndpointDelay": "",
"minimumTurnDuration": "",
"minimumInterruptionDuration": "",
"frameActivationThreshold": 123
},
"shortSummary": "",
"summary": "",
"experimentalSettings": ""
}
#### Authorizations
[](#authorization-x-api-key)
X-API-Key
string
header
required
API key
#### Path Parameters
[](#parameter-call-id)
call\_id
string
required
#### Response
200 - application/json
[](#response-call-id)
callId
string
required
[](#response-client-version)
clientVersion
string | null
required
The version of the client that joined this call.
[](#response-created)
created
string
required
[](#response-joined)
joined
string | null
required
[](#response-ended)
ended
string | null
required
[](#response-end-reason)
endReason
enum | nullany | null
required
The reason the call ended.
* `unjoined` - Client never joined
* `hangup` - Client hung up
* `agent_hangup` - Agent hung up
* `timeout` - Call timed out
* `connection_error` - Connection error
Available options:
`unjoined`,
`hangup`,
`agent_hangup`,
`timeout`,
`connection_error`
[](#response-first-speaker)
firstSpeaker
enum
requireddeprecated
Who was supposed to talk first when the call started. Typically set to FIRST\_SPEAKER\_USER for outgoing calls and left as the default (FIRST\_SPEAKER\_AGENT) otherwise.
Available options:
`FIRST_SPEAKER_AGENT`,
`FIRST_SPEAKER_USER`
[](#response-first-speaker-settings)
firstSpeakerSettings
object
required
Settings for the initial message to get the call started.
Show child attributes
[](#response-first-speaker-settings-user)
firstSpeakerSettings.user
object
If set, the user should speak first.
[](#response-first-speaker-settings-agent)
firstSpeakerSettings.agent
object
If set, the agent should speak first.
Show child attributes
[](#response-first-speaker-settings-agent-uninterruptible)
firstSpeakerSettings.agent.uninterruptible
boolean
Whether the user should be prevented from interrupting the agent's first message. Defaults to false (meaning the agent is interruptible as usual).
[](#response-first-speaker-settings-agent-text)
firstSpeakerSettings.agent.text
string
What the agent should say. If unset, the model will generate a greeting.
[](#response-initial-output-medium)
initialOutputMedium
enum
required
The medium used initially by the agent. May later be changed by the client.
Available options:
`MESSAGE_MEDIUM_VOICE`,
`MESSAGE_MEDIUM_TEXT`
[](#response-join-url)
joinUrl
string | null
required
[](#response-error-count)
errorCount
integer
default:
0
required
The number of errors in this call.
[](#response-short-summary)
shortSummary
string | null
required
A short summary of the call.
[](#response-summary)
summary
string | null
required
A summary of the call.
[](#response-experimental-settings)
experimentalSettings
any
required
Experimental settings for the call.
[](#response-inactivity-messages)
inactivityMessages
object\[\]
Messages spoken by the agent when the user is inactive for the specified duration. Durations are cumulative, so a message m > 1 with duration 30s will be spoken 30 seconds after message m-1.
A message the agent should say after some duration. The duration's meaning varies depending on the context.
Show child attributes
[](#response-inactivity-messages-duration)
inactivityMessages.duration
string
The duration after which the message should be spoken.
[](#response-inactivity-messages-message)
inactivityMessages.message
string
The message to speak.
[](#response-inactivity-messages-end-behavior)
inactivityMessages.endBehavior
enum
The behavior to exhibit when the message is finished being spoken.
Available options:
`END_BEHAVIOR_UNSPECIFIED`,
`END_BEHAVIOR_HANG_UP_SOFT`,
`END_BEHAVIOR_HANG_UP_STRICT`
[](#response-join-timeout)
joinTimeout
string
default:
30s
[](#response-language-hint)
languageHint
string | null
BCP47 language code that may be used to guide speech recognition.
Maximum length: `16`
[](#response-max-duration)
maxDuration
string
default:
3600s
[](#response-medium)
medium
object
Details about a call's protocol. By default, calls occur over WebRTC using the Ultravox client SDK. Setting a different call medium will prepare the server for a call using a different protocol. At most one call medium may be set.
Show child attributes
[](#response-medium-web-rtc)
medium.webRtc
object
The call will use WebRTC with the Ultravox client SDK. This is the default.
[](#response-medium-twilio)
medium.twilio
object
The call will use Twilio's "Media Streams" protocol. Once you have a join URL from starting a call, include it in your TwiML like so: This works for both inbound and outbound calls.
[](#response-medium-server-web-socket)
medium.serverWebSocket
object
The call will use a plain websocket connection. This is unlikely to yield an acceptable user experience if used from a browser or mobile client, but may be suitable for a server-to-server connection. This option provides a simple way to connect your own server to an Ultravox inference instance.
Show child attributes
[](#response-medium-server-web-socket-input-sample-rate)
medium.serverWebSocket.inputSampleRate
integer
The sample rate for input (user) audio. Required.
[](#response-medium-server-web-socket-output-sample-rate)
medium.serverWebSocket.outputSampleRate
integer
The desired sample rate for output (agent) audio. If unset, defaults to the input\_sample\_rate.
[](#response-medium-server-web-socket-client-buffer-size-ms)
medium.serverWebSocket.clientBufferSizeMs
integer
The size of the client-side audio buffer in milliseconds. Smaller buffers allow for faster interruptions but may cause audio underflow if network latency fluctuates too greatly. For the best of both worlds, set this to some large value (e.g. 30000) and implement support for playback\_clear\_buffer messages. Defaults to 60.
[](#response-medium-telnyx)
medium.telnyx
object
The call will use Telnyx's media streaming protocol. Once you have a join URL from starting a call, include it in your TexML like so: This works for both inbound and outbound calls.
[](#response-medium-plivo)
medium.plivo
object
The call will use Plivo's AudioStreams protocol. Once you have a join URL from starting a call, include it in your Plivo XML like so: ${your-join-url} This works for both inbound and outbound calls.
[](#response-medium-exotel)
medium.exotel
object
The call will use Exotel's "Voicebot" protocol. Once you have a join URL from starting a call, provide it to Exotel as the wss target URL for your Voicebot (either directly or more likely dynamically from your own server).
[](#response-model)
model
string
default:
fixie-ai/ultravox
[](#response-recording-enabled)
recordingEnabled
boolean
default:
false
[](#response-system-prompt)
systemPrompt
string | null
[](#response-temperature)
temperature
number
default:
0
Required range: `0 <= x <= 1`
[](#response-time-exceeded-message)
timeExceededMessage
string | null
[](#response-voice)
voice
string | null
[](#response-transcript-optional)
transcriptOptional
boolean
default:
true
deprecated
Indicates whether a transcript is optional for the call.
[](#response-vad-settings)
vadSettings
object
VAD settings for the call.
Show child attributes
[](#response-vad-settings-turn-endpoint-delay)
vadSettings.turnEndpointDelay
string
The minimum amount of time the agent will wait to respond after the user seems to be done speaking. Increasing this value will make the agent less eager to respond, which may increase perceived response latency but will also make the agent less likely to jump in before the user is really done speaking.
Built-in VAD currently operates on 32ms frames, so only multiples of 32ms are meaningful. (Anything from 1ms to 31ms will produce the same result.)
Defaults to "0.384s" (384ms) as a starting point, but there's nothing special about this value aside from it corresponding to 12 VAD frames.
[](#response-vad-settings-minimum-turn-duration)
vadSettings.minimumTurnDuration
string
The minimum duration of user speech required to be considered a user turn. Increasing this value will cause the agent to ignore short user audio. This may be useful in particularly noisy environments, but it comes at the cost of possibly ignoring very short user responses such as "yes" or "no".
Defaults to "0s" meaning the agent considers all user audio inputs (that make it through built-in noise cancellation).
[](#response-vad-settings-minimum-interruption-duration)
vadSettings.minimumInterruptionDuration
string
The minimum duration of user speech required to interrupt the agent. This works the same way as minimumTurnDuration, but allows for a higher threshold for interrupting the agent. (This value will be ignored if it is less than minimumTurnDuration.)
Defaults to "0.09s" (90ms) as a starting point, but there's nothing special about this value.
[](#response-vad-settings-frame-activation-threshold)
vadSettings.frameActivationThreshold
number
The threshold for the VAD to consider a frame as speech. This is a value between 0.1 and 1.
Miniumum value is 0.1, which is the default value.
[List Calls](/api-reference/calls/calls-list)
[Create Call](/api-reference/calls/calls-post)
cURL
Python
JavaScript
PHP
Go
Java
curl --request GET \
--url https://api.ultravox.ai/api/calls/{call_id} \
--header 'X-API-Key: '
200
{
"callId": "3c90c3cc-0d44-4b50-8888-8dd25736052a",
"clientVersion": "",
"created": "2023-11-07T05:31:56Z",
"joined": "2023-11-07T05:31:56Z",
"ended": "2023-11-07T05:31:56Z",
"endReason": "unjoined",
"firstSpeaker": "FIRST_SPEAKER_AGENT",
"firstSpeakerSettings": {
"user": {},
"agent": {
"uninterruptible": true,
"text": ""
}
},
"inactivityMessages": [\
{\
"duration": "",\
"message": "",\
"endBehavior": "END_BEHAVIOR_UNSPECIFIED"\
}\
],
"initialOutputMedium": "MESSAGE_MEDIUM_VOICE",
"joinTimeout": "30s",
"joinUrl": "",
"languageHint": "",
"maxDuration": "3600s",
"medium": {
"webRtc": {},
"twilio": {},
"serverWebSocket": {
"inputSampleRate": 123,
"outputSampleRate": 123,
"clientBufferSizeMs": 123
},
"telnyx": {},
"plivo": {},
"exotel": {}
},
"model": "fixie-ai/ultravox",
"recordingEnabled": false,
"systemPrompt": "",
"temperature": 0,
"timeExceededMessage": "",
"voice": "",
"transcriptOptional": true,
"errorCount": 0,
"vadSettings": {
"turnEndpointDelay": "",
"minimumTurnDuration": "",
"minimumInterruptionDuration": "",
"frameActivationThreshold": 123
},
"shortSummary": "",
"summary": "",
"experimentalSettings": ""
}
---
# Get Call Recording - Ultradox
[Ultradox home page](/)
Search or ask...
Search...
Navigation
Calls, Messages, Stages
Get Call Recording
[Documentation](/introduction)
[API Reference](/api-reference/introduction)
[SDK Reference](/sdk-reference/introduction)
[Changelog](/changelog/news)
GET
/
api
/
calls
/
{call\_id}
/
recording
Try it
cURL
Python
JavaScript
PHP
Go
Java
curl --request GET \
--url https://api.ultravox.ai/api/calls/{call_id}/recording \
--header 'X-API-Key: '
#### Authorizations
[](#authorization-x-api-key)
X-API-Key
string
header
required
API key
#### Path Parameters
[](#parameter-call-id)
call\_id
string
required
#### Response
200
No response body
[List Call Tools](/api-reference/calls/calls-tools-list)
[List Call Stages](/api-reference/calls/calls-stages-list)
cURL
Python
JavaScript
PHP
Go
Java
curl --request GET \
--url https://api.ultravox.ai/api/calls/{call_id}/recording \
--header 'X-API-Key: '
---
# It's All Prompting - Ultradox
[Ultradox home page](/)
Search or ask...
Search...
Navigation
The Five Rules of Ultravox
It's All Prompting
[Documentation](/introduction)
[API Reference](/api-reference/introduction)
[SDK Reference](/sdk-reference/introduction)
[Changelog](/changelog/news)
[](#the-first-rule-of-building-with-ultravox%3A-%E2%80%9Cit%E2%80%99s-all-prompting%E2%80%9D)
The first rule of building with Ultravox: “It’s all prompting”.
-----------------------------------------------------------------------------------------------------------------------------------------------------------------
Everything that your agents do is based on the prompt instructions you give them. It’s tempting then to write very complicated, verbose prompts. The problem with that approach is that the more instructions we provide in the prompt, the less well the model does at attending to the various instructions.
[Ultravox Realtime](/introduction)
[It's All Prompting](/gettingstarted/rule2)
---
# List Call Tools - Ultradox
[Ultradox home page](/)
Search or ask...
Search...
Navigation
Calls, Messages, Stages
List Call Tools
[Documentation](/introduction)
[API Reference](/api-reference/introduction)
[SDK Reference](/sdk-reference/introduction)
[Changelog](/changelog/news)
GET
/
api
/
calls
/
{call\_id}
/
tools
Try it
cURL
Python
JavaScript
PHP
Go
Java
curl --request GET \
--url https://api.ultravox.ai/api/calls/{call_id}/tools \
--header 'X-API-Key: '
200
[\
{\
"callToolId": "3c90c3cc-0d44-4b50-8888-8dd25736052a",\
"toolId": "3c90c3cc-0d44-4b50-8888-8dd25736052a",\
"name": "",\
"definition": {\
"description": "",\
"dynamicParameters": [\
{\
"name": "",\
"location": "PARAMETER_LOCATION_UNSPECIFIED",\
"schema": {},\
"required": true\
}\
],\
"staticParameters": [\
{\
"name": "",\
"location": "PARAMETER_LOCATION_UNSPECIFIED",\
"value": ""\
}\
],\
"automaticParameters": [\
{\
"name": "",\
"location": "PARAMETER_LOCATION_UNSPECIFIED",\
"knownValue": "KNOWN_PARAM_UNSPECIFIED"\
}\
],\
"timeout": "",\
"precomputable": true,\
"http": {\
"baseUrlPattern": "",\
"httpMethod": "",\
"authHeaders": [\
""\
],\
"authQueryParams": [\
""\
],\
"callTokenScopes": [\
""\
]\
},\
"client": {}\
}\
}\
]
#### Authorizations
[](#authorization-x-api-key)
X-API-Key
string
header
required
API key
#### Path Parameters
[](#parameter-call-id)
call\_id
string
required
#### Response
200 - application/json
[](#response-call-tool-id)
callToolId
string
required
[](#response-tool-id)
toolId
string | null
required
[](#response-name)
name
string
required
The possibly overridden name of the tool.
[](#response-definition)
definition
object
required
A tool as used for a particular call (omitting auth details).
Show child attributes
[](#response-definition-description)
definition.description
string
The description of the tool.
[](#response-definition-dynamic-parameters)
definition.dynamicParameters
object\[\]
The parameters presented to the model.
A dynamic parameter the tool accepts that may be set by the model.
Show child attributes
[](#response-definition-dynamic-parameters-name)
definition.dynamicParameters.name
string
The name of the parameter.
[](#response-definition-dynamic-parameters-location)
definition.dynamicParameters.location
enum
Where the parameter is used.
Available options:
`PARAMETER_LOCATION_UNSPECIFIED`,
`PARAMETER_LOCATION_QUERY`,
`PARAMETER_LOCATION_PATH`,
`PARAMETER_LOCATION_HEADER`,
`PARAMETER_LOCATION_BODY`
[](#response-definition-dynamic-parameters-schema)
definition.dynamicParameters.schema
object
The JsonSchema definition of the parameter. This typically includes things like type, description, enum values, format, other restrictions, etc.
[](#response-definition-dynamic-parameters-required)
definition.dynamicParameters.required
boolean
Whether the parameter is required.
[](#response-definition-static-parameters)
definition.staticParameters
object\[\]
Parameters added unconditionally when the tool is invoked.
A static parameter that is unconditionally added when the tool is invoked. This parameter is not exposed to or set by the model.
Show child attributes
[](#response-definition-static-parameters-name)
definition.staticParameters.name
string
The name of the parameter.
[](#response-definition-static-parameters-location)
definition.staticParameters.location
enum
Where the parameter is used.
Available options:
`PARAMETER_LOCATION_UNSPECIFIED`,
`PARAMETER_LOCATION_QUERY`,
`PARAMETER_LOCATION_PATH`,
`PARAMETER_LOCATION_HEADER`,
`PARAMETER_LOCATION_BODY`
[](#response-definition-static-parameters-value)
definition.staticParameters.value
any
The value of the parameter.
[](#response-definition-automatic-parameters)
definition.automaticParameters
object\[\]
Parameters automatically set by the system.
A parameter that is automatically set by the system.
Show child attributes
[](#response-definition-automatic-parameters-name)
definition.automaticParameters.name
string
The name of the parameter.
[](#response-definition-automatic-parameters-location)
definition.automaticParameters.location
enum
Where the parameter is used.
Available options:
`PARAMETER_LOCATION_UNSPECIFIED`,
`PARAMETER_LOCATION_QUERY`,
`PARAMETER_LOCATION_PATH`,
`PARAMETER_LOCATION_HEADER`,
`PARAMETER_LOCATION_BODY`
[](#response-definition-automatic-parameters-known-value)
definition.automaticParameters.knownValue
enum
The value to set for the parameter.
Available options:
`KNOWN_PARAM_UNSPECIFIED`,
`KNOWN_PARAM_CALL_ID`,
`KNOWN_PARAM_CONVERSATION_HISTORY`,
`KNOWN_PARAM_OUTPUT_SAMPLE_RATE`
[](#response-definition-timeout)
definition.timeout
string
The maximum amount of time the tool is allowed for execution. The conversation is frozen while tools run, so prefer sticking to the default unless you're comfortable with that consequence. If your tool is too slow for the default and can't be made faster, still try to keep this timeout as low as possible.
[](#response-definition-precomputable)
definition.precomputable
boolean
The tool is guaranteed to be non-mutating, repeatable, and free of side-effects. Such tools can safely be executed speculatively, reducing their effective latency. However, the fact they were called may not be reflected in the call history if their result ends up unused.
[](#response-definition-http)
definition.http
object
Details for an HTTP tool.
Show child attributes
[](#response-definition-http-base-url-pattern)
definition.http.baseUrlPattern
string
The base URL pattern for the tool, possibly with placeholders for path parameters.
[](#response-definition-http-http-method)
definition.http.httpMethod
string
The HTTP method for the tool.
[](#response-definition-http-auth-headers)
definition.http.authHeaders
string\[\]
Auth headers added when the tool is invoked.
[](#response-definition-http-auth-query-params)
definition.http.authQueryParams
string\[\]
Auth query parameters added when the tool is invoked.
[](#response-definition-http-call-token-scopes)
definition.http.callTokenScopes
string\[\]
If the tool requires a call token, the scopes that must be present in the token. If this is empty, no call token will be created.
[](#response-definition-client)
definition.client
object
Details for a client-implemented tool. Only body parameters are allowed for client tools.
[List Call Messages](/api-reference/calls/calls-messages-list)
[Get Call Recording](/api-reference/calls/calls-recording-get)
cURL
Python
JavaScript
PHP
Go
Java
curl --request GET \
--url https://api.ultravox.ai/api/calls/{call_id}/tools \
--header 'X-API-Key: '
200
[\
{\
"callToolId": "3c90c3cc-0d44-4b50-8888-8dd25736052a",\
"toolId": "3c90c3cc-0d44-4b50-8888-8dd25736052a",\
"name": "",\
"definition": {\
"description": "",\
"dynamicParameters": [\
{\
"name": "",\
"location": "PARAMETER_LOCATION_UNSPECIFIED",\
"schema": {},\
"required": true\
}\
],\
"staticParameters": [\
{\
"name": "",\
"location": "PARAMETER_LOCATION_UNSPECIFIED",\
"value": ""\
}\
],\
"automaticParameters": [\
{\
"name": "",\
"location": "PARAMETER_LOCATION_UNSPECIFIED",\
"knownValue": "KNOWN_PARAM_UNSPECIFIED"\
}\
],\
"timeout": "",\
"precomputable": true,\
"http": {\
"baseUrlPattern": "",\
"httpMethod": "",\
"authHeaders": [\
""\
],\
"authQueryParams": [\
""\
],\
"callTokenScopes": [\
""\
]\
},\
"client": {}\
}\
}\
]
---
# Base Tool Definition - Ultradox
[Ultradox home page](/)
Search or ask...
Search...
Navigation
Schema
Base Tool Definition
[Documentation](/introduction)
[API Reference](/api-reference/introduction)
[SDK Reference](/sdk-reference/introduction)
[Changelog](/changelog/news)
The base definition of a tool that can be used during a call. Exactly one implementation (http or client) should be set.
[](#schema-model-tool-name)
modelToolName
string
The name of the tool, as presented to the model. Must match ^\[a-zA-Z0-9\_-\]{1,64}$.
[](#schema-description)
description
string
The description of the tool.
[](#schema-dynamic-parameters)
dynamicParameters
object\[\]
The parameters that the tool accepts.
A dynamic parameter the tool accepts that may be set by the model.
Show child attributes
[](#schema-dynamic-parameters-name)
dynamicParameters.name
string
The name of the parameter.
[](#schema-dynamic-parameters-location)
dynamicParameters.location
enum
Where the parameter is used.
Available options:
`PARAMETER_LOCATION_UNSPECIFIED`,
`PARAMETER_LOCATION_QUERY`,
`PARAMETER_LOCATION_PATH`,
`PARAMETER_LOCATION_HEADER`,
`PARAMETER_LOCATION_BODY`
[](#schema-dynamic-parameters-schema)
dynamicParameters.schema
object
The JsonSchema definition of the parameter. This typically includes things like type, description, enum values, format, other restrictions, etc.
[](#schema-dynamic-parameters-required)
dynamicParameters.required
boolean
Whether the parameter is required.
[](#schema-static-parameters)
staticParameters
object\[\]
The static parameters added when the tool is invoked.
A static parameter that is unconditionally added when the tool is invoked. This parameter is not exposed to or set by the model.
Show child attributes
[](#schema-static-parameters-name)
staticParameters.name
string
The name of the parameter.
[](#schema-static-parameters-location)
staticParameters.location
enum
Where the parameter is used.
Available options:
`PARAMETER_LOCATION_UNSPECIFIED`,
`PARAMETER_LOCATION_QUERY`,
`PARAMETER_LOCATION_PATH`,
`PARAMETER_LOCATION_HEADER`,
`PARAMETER_LOCATION_BODY`
[](#schema-static-parameters-value)
staticParameters.value
any
The value of the parameter.
[](#schema-automatic-parameters)
automaticParameters
object\[\]
Additional parameters that are automatically set by the system when the tool is invoked.
A parameter that is automatically set by the system.
Show child attributes
[](#schema-automatic-parameters-name)
automaticParameters.name
string
The name of the parameter.
[](#schema-automatic-parameters-location)
automaticParameters.location
enum
Where the parameter is used.
Available options:
`PARAMETER_LOCATION_UNSPECIFIED`,
`PARAMETER_LOCATION_QUERY`,
`PARAMETER_LOCATION_PATH`,
`PARAMETER_LOCATION_HEADER`,
`PARAMETER_LOCATION_BODY`
[](#schema-automatic-parameters-known-value)
automaticParameters.knownValue
enum
The value to set for the parameter.
Available options:
`KNOWN_PARAM_UNSPECIFIED`,
`KNOWN_PARAM_CALL_ID`,
`KNOWN_PARAM_CONVERSATION_HISTORY`,
`KNOWN_PARAM_OUTPUT_SAMPLE_RATE`
[](#schema-requirements)
requirements
object
Requirements that must be fulfilled when creating a call for the tool to be used.
Show child attributes
[](#schema-requirements-http-security-options)
requirements.httpSecurityOptions
object
Security requirements for an HTTP tool.
Show child attributes
[](#schema-requirements-http-security-options-options)
requirements.httpSecurityOptions.options
object\[\]
The options for security. Only one must be met. The first one that can be satisfied will be used in general. The single exception to this rule is that we always prefer a non-empty set of requirements over an empty set unless no non-empty set can be satisfied.
The security requirements for a request. All requirements must be met.
Show child attributes
[](#schema-requirements-http-security-options-options-requirements)
requirements.httpSecurityOptions.options.requirements
object
Requirements keyed by name.
Show child attributes
[](#schema-requirements-http-security-options-options-requirements-key)
requirements.httpSecurityOptions.options.requirements.{key}
object
A single security requirement that must be met for a tool to be available. Exactly one of query\_api\_key, header\_api\_key, or http\_auth should be set.
Show child attributes
[](#schema-requirements-http-security-options-options-ultravox-call-token-requirement)
requirements.httpSecurityOptions.options.ultravoxCallTokenRequirement
object
An additional special security requirement that can be automatically fulfilled during call creation. If a tool has this requirement set, a token identifying the call and relevant scopes will be created during call creation and set as an X-Ultravox-Call-Token header when the tool is invoked. Such tokens are only verifiable by the Ultravox service and primarily exist for built-in tools (though it's possible for third-party tools that wrap a built-in tool to make use of them as well).
Show child attributes
[](#schema-requirements-http-security-options-options-ultravox-call-token-requirement-scopes)
requirements.httpSecurityOptions.options.ultravoxCallTokenRequirement.scopes
string\[\]
The scopes that must be present in the token.
[](#schema-requirements-required-parameter-overrides)
requirements.requiredParameterOverrides
string\[\]
Dynamic parameters that must be overridden with an explicit (static) value.
[](#schema-timeout)
timeout
string
The maximum amount of time the tool is allowed for execution. The conversation is frozen while tools run, so prefer sticking to the default unless you're comfortable with that consequence. If your tool is too slow for the default and can't be made faster, still try to keep this timeout as low as possible.
[](#schema-precomputable)
precomputable
boolean
The tool is guaranteed to be non-mutating, repeatable, and free of side-effects. Such tools can safely be executed speculatively, reducing their effective latency. However, the fact they were called may not be reflected in the call history if their result ends up unused.
[](#schema-http)
http
object
Details for an HTTP tool.
Show child attributes
[](#schema-http-base-url-pattern)
http.baseUrlPattern
string
The base URL pattern for the tool, possibly with placeholders for path parameters.
[](#schema-http-http-method)
http.httpMethod
string
The HTTP method for the tool.
[](#schema-client)
client
object
Details for a client-implemented tool. Only body parameters are allowed for client tools.
[Delete Webhook](/api-reference/webhooks/webhooks-delete)
[Ultravox Data Message Protocol](/api-reference/schema/datamessages)
Example
{
"modelToolName": "",
"description": "",
"dynamicParameters": [\
{\
"name": "",\
"location": "PARAMETER_LOCATION_UNSPECIFIED",\
"schema": {},\
"required": true\
}\
],
"staticParameters": [\
{\
"name": "",\
"location": "PARAMETER_LOCATION_UNSPECIFIED",\
"value": ""\
}\
],
"automaticParameters": [\
{\
"name": "",\
"location": "PARAMETER_LOCATION_UNSPECIFIED",\
"knownValue": "KNOWN_PARAM_UNSPECIFIED"\
}\
],
"requirements": {
"httpSecurityOptions": {
"options": [\
{\
"requirements": {},\
"ultravoxCallTokenRequirement": {\
"scopes": [\
""\
]\
}\
}\
]
},
"requiredParameterOverrides": [\
""\
]
},
"timeout": "",
"precomputable": true,
"http": {
"baseUrlPattern": "",
"httpMethod": ""
},
"client": {}
}
---
# Customer Success & Support - Ultradox
[Ultradox home page](/)
Search or ask...
Search...
Navigation
Customer Success & Support
[Documentation](/introduction)
[API Reference](/api-reference/introduction)
[SDK Reference](/sdk-reference/introduction)
[Changelog](/changelog/news)
---
# Customer Acquisition - Ultradox
[Ultradox home page](/)
Search or ask...
Search...
Navigation
Customer Acquisition
[Documentation](/introduction)
[API Reference](/api-reference/introduction)
[SDK Reference](/sdk-reference/introduction)
[Changelog](/changelog/news)
---
# Operations - Ultradox
[Ultradox home page](/)
Search or ask...
Search...
Navigation
Operations
[Documentation](/introduction)
[API Reference](/api-reference/introduction)
[SDK Reference](/sdk-reference/introduction)
[Changelog](/changelog/news)
---
# List Call Stage Messages - Ultradox
[Ultradox home page](/)
Search or ask...
Search...
Navigation
Calls, Messages, Stages
List Call Stage Messages
[Documentation](/introduction)
[API Reference](/api-reference/introduction)
[SDK Reference](/sdk-reference/introduction)
[Changelog](/changelog/news)
GET
/
api
/
calls
/
{call\_id}
/
stages
/
{call\_stage\_id}
/
messages
Try it
cURL
Python
JavaScript
PHP
Go
Java
curl --request GET \
--url https://api.ultravox.ai/api/calls/{call_id}/stages/{call_stage_id}/messages \
--header 'X-API-Key: '
200
{
"next": "http://api.example.org/accounts/?cursor=cD00ODY%3D\"",
"previous": "http://api.example.org/accounts/?cursor=cj0xJnA9NDg3",
"results": [\
{\
"role": "MESSAGE_ROLE_UNSPECIFIED",\
"text": "",\
"invocationId": "",\
"toolName": "",\
"errorDetails": "",\
"medium": "MESSAGE_MEDIUM_UNSPECIFIED",\
"callStageMessageIndex": 123,\
"callStageId": ""\
}\
],
"total": 123
}
#### Authorizations
[](#authorization-x-api-key)
X-API-Key
string
header
required
API key
#### Path Parameters
[](#parameter-call-id)
call\_id
string
required
[](#parameter-call-stage-id)
call\_stage\_id
string
required
#### Query Parameters
[](#parameter-cursor)
cursor
string
The pagination cursor value.
[](#parameter-page-size)
pageSize
integer
Number of results to return per page.
#### Response
200 - application/json
[](#response-results)
results
object\[\]
required
A message exchanged during a call.
Show child attributes
[](#response-results-role)
results.role
enum
The message's role.
Available options:
`MESSAGE_ROLE_UNSPECIFIED`,
`MESSAGE_ROLE_USER`,
`MESSAGE_ROLE_AGENT`,
`MESSAGE_ROLE_TOOL_CALL`,
`MESSAGE_ROLE_TOOL_RESULT`
[](#response-results-text)
results.text
string
The message text for user and agent messages, tool arguments for tool\_call messages, tool results for tool\_result messages.
[](#response-results-invocation-id)
results.invocationId
string
The invocation ID for tool messages. Used to pair tool calls with their results.
[](#response-results-tool-name)
results.toolName
string
The tool name for tool messages.
[](#response-results-error-details)
results.errorDetails
string
For failed tool calls, additional debugging information. While the text field is presented to the model so it can respond to failures gracefully, the full details are only exposed via the Ultravox REST API.
[](#response-results-medium)
results.medium
enum
The medium of the message.
Available options:
`MESSAGE_MEDIUM_UNSPECIFIED`,
`MESSAGE_MEDIUM_VOICE`,
`MESSAGE_MEDIUM_TEXT`
[](#response-results-call-stage-message-index)
results.callStageMessageIndex
integer
The index of the message within the call stage.
[](#response-results-call-stage-id)
results.callStageId
string
The call stage this message appeared in.
[](#response-next)
next
string | null
[](#response-previous)
previous
string | null
[](#response-total)
total
integer
[Get Call Stage](/api-reference/calls/calls-stages-get)
[List Call Stage Tools](/api-reference/calls/calls-stages-tools-list)
cURL
Python
JavaScript
PHP
Go
Java
curl --request GET \
--url https://api.ultravox.ai/api/calls/{call_id}/stages/{call_stage_id}/messages \
--header 'X-API-Key: '
200
{
"next": "http://api.example.org/accounts/?cursor=cD00ODY%3D\"",
"previous": "http://api.example.org/accounts/?cursor=cj0xJnA9NDg3",
"results": [\
{\
"role": "MESSAGE_ROLE_UNSPECIFIED",\
"text": "",\
"invocationId": "",\
"toolName": "",\
"errorDetails": "",\
"medium": "MESSAGE_MEDIUM_UNSPECIFIED",\
"callStageMessageIndex": 123,\
"callStageId": ""\
}\
],
"total": 123
}
---