# Table of Contents - [Welcome to Vapi's Developer Documentation — Vapi](#welcome-to-vapi-s-developer-documentation-vapi) - [Orchestration Models — Vapi](#orchestration-models-vapi) - [Core Models — Vapi](#core-models-vapi) - [Creating Custom Knowledge Bases for Your Voice AI Assistants — Vapi](#creating-custom-knowledge-bases-for-your-voice-ai-assistants-vapi) - [Billing Limits — Vapi](#billing-limits-vapi) - [Cost Routing — Vapi](#cost-routing-vapi) - [Estimating Costs — Vapi](#estimating-costs-vapi) - [Pricing Overview — Vapi](#pricing-overview-vapi) - [Billing Examples — Vapi](#billing-examples-vapi) - [Introduction — Vapi](#introduction-vapi) - [Vapi Enterprise — Vapi](#vapi-enterprise-vapi) - [HIPAA Compliance — Vapi](#hipaa-compliance-vapi) - [Support — Vapi](#support-vapi) - [Client SDKs — Vapi](#client-sdks-vapi) - [On-Prem Deployments — Vapi](#on-prem-deployments-vapi) - [Server SDKs — Vapi](#server-sdks-vapi) - [Web Snippet — Vapi](#web-snippet-vapi) - [Code Resources — Vapi](#code-resources-vapi) - [Inbound Calling — Vapi](#inbound-calling-vapi) - [Outbound Calls from Python 📞 — Vapi](#outbound-calls-from-python-vapi) - [Dashboard — Vapi](#dashboard-vapi) - [Provider Keys — Vapi](#provider-keys-vapi) - [Billing FAQ — Vapi](#billing-faq-vapi) - [Pizza Website Example 🍕 — Vapi](#pizza-website-example-vapi) - [Outbound Sales Example 📞 — Vapi](#outbound-sales-example-vapi) - [Inbound Support Example ⚙️ — Vapi](#inbound-support-example-vapi) - [Web Calling — Vapi](#web-calling-vapi) - [Outbound Calling — Vapi](#outbound-calling-vapi) - [Web SDK — Vapi](#web-sdk-vapi) - [Fine-tuned OpenAI models — Vapi](#fine-tuned-openai-models-vapi) - [Multilingual — Vapi](#multilingual-vapi) - [Introduction — Vapi](#introduction-vapi) - [Elevenlabs — Vapi](#elevenlabs-vapi) - [Connecting Your Custom LLM to Vapi: A Comprehensive Guide — Vapi](#connecting-your-custom-llm-to-vapi-a-comprehensive-guide-vapi) - [JWT Authentication — Vapi](#jwt-authentication-vapi) - [PlayHT — Vapi](#playht-vapi) - [Introduction — Vapi](#introduction-vapi) - [Background Messaging — Vapi](#background-messaging-vapi) - [Tavus — Vapi](#tavus-vapi) - [Voice Formatting Plan — Vapi](#voice-formatting-plan-vapi) - [Custom Keywords — Vapi](#custom-keywords-vapi) - [Persistent Assistants — Vapi](#persistent-assistants-vapi) - [Function Calling — Vapi](#function-calling-vapi) - [Dynamic Variables — Vapi](#dynamic-variables-vapi) - [Speech Configuration — Vapi](#speech-configuration-vapi) - [Bring your own chunks/vectors from Trieve — Vapi](#bring-your-own-chunks-vectors-from-trieve-vapi) - [Call Analysis — Vapi](#call-analysis-vapi) - [Introduction — Vapi](#introduction-vapi) - [Block Types — Vapi](#block-types-vapi) - [Phone Calling — Vapi](#phone-calling-vapi) - [Squads — Vapi](#squads-vapi) - [Configuring Inbound and Outbound Calls for Squads — Vapi](#configuring-inbound-and-outbound-calls-for-squads-vapi) - [Server URLs — Vapi](#server-urls-vapi) - [Steps — Vapi](#steps-vapi) - [Setting Server URLs — Vapi](#setting-server-urls-vapi) - [Server Events — Vapi](#server-events-vapi) --- # Welcome to Vapi's Developer Documentation — Vapi Everything you need to build, test, & deploy voice AI agents in minutes rather than months [Quickstart\ \ Get started now with the Vapi web dashboard.](/quickstart/dashboard) [Documentation\ \ Reference our documentation for how to use Vapi’s Voice AI platform.](/introduction) [Community\ \ Connect with our team & other developers using Vapi.](https://discord.gg/pUFNcf2WmH) [GitHub\ \ Check out our GitHub to see what the Vapi team has been up to.](https://github.com/VapiAI) [Built with](https://buildwithfern.com/?utm_campaign=buildWith&utm_medium=docs&utm_source=docs.vapi.ai) --- # Orchestration Models — Vapi Vapi also runs a suite of audio and text models that make it’s latency-optimized Speech-to-Text (STT), Large Language Model (LLM), & Text-to-Speech (TTS) pipeline feel human. Here’s a high-level overview of the Vapi architecture: ![](https://files.buildwithfern.com/https://vapi.docs.buildwithfern.com/2025-01-31T05:46:58.861Z/static/images/learn/platform/vapi-orchestration.png) These are some of the models that are part of the Orchestration suite. We currently have lots of other models in the pipeline that will be added to the orchestration suite soon. The ultimate goal is to achieve human performance. ### Endpointing Endpointing is a fancy word for knowing when the user is done speaking. Traditional methods use silence detection with a timeout. Unfortunately, if we want sub-second response-times, that’s not going to work. Vapi’s uses a custom fusion audio-text model to know when a user has completed their turn. Based on both the user’s tone and what they’re saying, it decides how long to pause before hitting the LLM. This is critical to make sure the user isn’t interrupted mid-thought while still providing sub-second response times when they’re done speaking. ### Interruptions (Barge-in) Interruptions (aka. barge-in in research circles) is the ability to detect when the user would like to interject and stop the assistant’s speech. Vapi uses a custom model to distinguish when there is a true interruption, like “stop”, “hold up”, “that’s not what I mean, and when there isn’t, like “yeah”, “oh gotcha”, “okay.” It also keeps track of where the assistant was cut off, so the LLM knows what it wasn’t able to say. ### Background Noise Filtering Many of our models, including the transcriber, are audio-based. In the real world, things like music and car horns can interfere with model performance. We use a proprietary real-time noise filtering model to ensure the audio is cleaned without sacrificing latency, before it reaches the inner models of the pipeline. ### Background Voice Filtering We rely quite heavily on the transcription model to know what’s going on, for interruptions, endpointing, backchanneling, and for the user’s statement passed to the LLM. Transcription models are built to pick up everything that sounds like speech, so this can be a problem. As you can imagine, having a TV on in the background or echo coming back into the mic can severely impact the conversation ability of a system like Vapi. Background noise cancellation is a well-researched problem. Background voice cancellation is not. To solve this, we built proprietary audio filtering model that’s able to **focus in** on the primary speaker and block everything else out. ### Backchanneling Humans like to affirm each other while they speak with statements like “yeah”, “uh-huh”, “got it”, “oh no!” They’re not considered interruptions, they’re just used to let the speaker know that their statement has been understood, and encourage the user to continue their statement. A backchannel cue used at the wrong moment can derail a user’s statement. Vapi uses a proprietary fusion audio text model to determine the best moment to backchannel and to decide which backchannel cue is most appropriate to use. ### Emotion Detection How a person says something is just as important as what they’re saying. So we’ve trained a real-time audio model to extract the emotional inflection of the user’s statement. This emotional information is then fed into the LLM, so knows to behave differently if the user is angry, annoyed, or confused. ### Filler Injection The output of LLMs tends to be formal, and not conversational. People speak with phrases like “umm”, “ahh”, “i mean”, “like”, “so”, etc. You can prompt the model to output like this, but we treat our user’s prompts as **sacred**. Making a change like this to a prompt can change the behavior in unintended ways. To ensure we don’t add additional latency transforming the output, we’ve built a custom model that’s able to convert streaming input and make it sound conversational in real-time. [Built with](https://buildwithfern.com/?utm_campaign=buildWith&utm_medium=docs&utm_source=docs.vapi.ai) --- # Core Models — Vapi At it’s core, Vapi is an orchestration layer over three modules: the **transcriber**, the **model**, and the **voice**. ![](https://files.buildwithfern.com/https://vapi.docs.buildwithfern.com/2025-01-31T05:46:58.861Z/static/images/quickstart/quickstart-banner.png) These three modules can be swapped out with **any provider** of your choosing; OpenAI, Groq, Deepgram, ElevenLabs, PlayHT, etc. You can even plug in your server to act as the LLM. Vapi takes these three modules, optimizes the latency, manages the scaling & streaming, and orchestrates the conversation flow to make it sound human. [1](/quickstart#listen-intake-raw-audio) ### Listen (intake raw audio) When a person speaks, the client device (whether it is a laptop, phone, etc) will record raw audio (1’s & 0’s at the core of it). This raw audio will have to either be transcribed on the client device itself, or get shipped off to a server somewhere to turn into transcription text. [2](/quickstart#run-an-llm) ### Run an LLM That transcript text will then get fed into a prompt & run through an LLM ([LLM inference](/glossary#inference) ). The LLM is the core intelligence that simulates a person behind-the-scenes. [3](/quickstart#speak-text--raw-audio) ### Speak (text → raw audio) The LLM outputs text that now must be spoken. That text is turned back into raw audio (again, 1’s & 0’s), that is playable back at the user’s device. This process can also either happen on the user’s device itself, or on a server somewhere (then the raw speech audio be shipped back to the user). The idea is to perform each phase in realtime (sensitive down to 50-100ms level), streaming between every layer. Ideally the whole flow [voice-to-voice](/glossary#voice-to-voice) clocks in at <500-700ms. Vapi pulls all these pieces together, ensuring a smooth & responsive conversation (in addition to providing you with a simple set of tools to manage these inner-workings). [Built with](https://buildwithfern.com/?utm_campaign=buildWith&utm_medium=docs&utm_source=docs.vapi.ai) --- # Creating Custom Knowledge Bases for Your Voice AI Assistants — Vapi **What is Vapi’s Knowledge Base?** ---------------------------------- A Knowledge Base is a collection of custom files that contain information on specific topics or domains. By integrating a Knowledge Base into your voice AI assistant, you can enable it to provide more accurate and informative responses to user queries. This is currently available in Vapi via the API, and will be on the dashboard soon. ### **Why Use a Knowledge Base?** Using a Knowledge Base with your voice AI assistant offers several benefits: * **Improved accuracy**: By integrating custom files into your assistant, you can ensure that it provides accurate and up-to-date information to users. * **Enhanced capabilities**: A Knowledge Base enables your assistant to answer complex queries and provide detailed responses to user inquiries. * **Customization**: With a Knowledge Base, you can tailor your assistant’s responses to specific domains or topics, making it more effective and informative. **How to Create a Knowledge Base** ---------------------------------- To create a Knowledge Base, follow these steps: ### **Step 1: Upload Your Files** Navigate to Platform > Files and upload your custom files in Markdown, PDF, plain text, or Microsoft Word (.doc and .docx) format to Vapi’s Knowledge Base. ![Adding files to your Knowledge Base](https://files.buildwithfern.com/https://vapi.docs.buildwithfern.com/2025-01-31T05:46:58.861Z/static/images/knowledge-base/files.png) Adding files to your Knowledge Base Alternatively you can upload your files via the API. ` | | | | --- | --- | | $ | curl --location 'https://api.vapi.ai/file' \ | | > | --header 'Authorization: Bearer ' \ | | > | --form 'file=@""' | ` ### **Step 2: Create a Knowledge Base** Use the ID of the uploaded file to create a Knowledge Base. Currently we support [trieve](https://trieve.ai/) as a provider. ` | | | | --- | --- | | $ | curl --location 'http://localhost:3001/knowledge-base' \ | | > | --header 'Content-Type: text/plain' \ | | > | --header 'Authorization: Bearer 4beb7e10-f4be-4588-be65-712235f07f68' \ | | > | --data '{ | | > | "name": "v2", | | > | "provider": "trieve", | | > | "searchPlan": { | | > | "scoreThreshold": 0.2, | | > | "searchType": "semantic" | | > | }, | | > | "createPlan": { | | > | "type": "create", | | > | "chunkPlans": [ | | > | { | | > | "fileIds": ["", ""], | | > | "websites": ["", ""], | | > | "targetSplitsPerChunk": 50, | | > | "rebalanceChunks": true | | > | } | | > | ] | | > | } | | > | }'' | ` ### **Step 3: Create an Assistant** Create a new assistant in Vapi and, on the right sidebar menu. Add the Knowledge Base to your assistant via the PATCH endpoint. Also make sure you customize your assistant’s system prompt to utilize the Knowledge Base for responding to user queries. ` | | | | --- | --- | | $ | curl --location --request PATCH 'https://api.vapi.ai/assistant/' \ | | > | --header 'Content-Type: text/plain' \ | | > | --header 'Authorization: Bearer ' \ | | > | --data '{ | | > | "model": { | | > | "knowledgeBaseId": "", | | > | "temperature": 0.2, | | > | "provider": "openai", | | > | "model": "gpt-4o", | | > | "messages": [ | | > | { | | > | "content": "You are a smart assistant who responds to user queries using the information you know, or information supplied by outside context.", | | > | "role": "system" | | > | } | | > | ] | | > | } | | > | }' | ` **Best Practices for Creating Effective Knowledge Bases** --------------------------------------------------------- * **Organize Your files**: Organize your files by topic or category to ensure that your assistant can quickly retrieve relevant information. * **Use Clear and concise language**: Use clear and concise language in your files to ensure that your assistant can accurately understand and respond to user queries. * **Keep your files up-to-date**: Regularly update your files to ensure that your assistant provides the most accurate and up-to-date information. For more information on creating effective Knowledge Bases, check out our tutorial on [Best Practices for Knowledge Base Creation](https://youtu.be/i5mvqC5sZxU) . By following these guidelines, you can create a comprehensive Knowledge Base that enhances the capabilities of your voice AI assistant and provides valuable information to users. [Built with](https://buildwithfern.com/?utm_campaign=buildWith&utm_medium=docs&utm_source=docs.vapi.ai) --- # Billing Limits — Vapi You can set billing limits in the billing section of your dashboard. You can access your billing settings at [dashboard.vapi.ai/org/billing](https://dashboard.vapi.ai/org/billing) ### Concurrency Limits Vapi has concurrency limits on both inbound and outbound calls. These limits define the maximum number of simultaneous calls your account can handle. Exceeding your concurrency limit causes new requests to queue or be rejected until existing calls finish. * The default concurrency limit is 10 simultaneous calls(inbound and outbound calls combined). This limit applies to your entire account and is not dependent on the number of users or organizations associated with it. * To increase your concurrency limit beyond the default of 10, you must purchase additional concurrent lines through the dashboard section. ### Setting a Monthly Billing Limit In your billing settings you can set a monthly billing limit: ![](https://files.buildwithfern.com/https://vapi.docs.buildwithfern.com/2025-01-31T05:46:58.861Z/static/images/learn/billing/billing-limits.png) ### Exceeding Billing Limits Once you have used all of your starter credits, or exceeded your set monthly usage limit, you will start seeing errors in your dashboard & via the API mentioning `Billing Limits Exceeded`. ![](https://files.buildwithfern.com/https://vapi.docs.buildwithfern.com/2025-01-31T05:46:58.861Z/static/images/learn/billing/billing-limits-exceeded.png) Once you have gone over on your monthly billing limits, the API & dashboard will throw billing-related errors. [Built with](https://buildwithfern.com/?utm_campaign=buildWith&utm_medium=docs&utm_source=docs.vapi.ai) --- # Cost Routing — Vapi ![](https://files.buildwithfern.com/https://vapi.docs.buildwithfern.com/2025-01-31T05:46:58.861Z/static/images/learn/billing/cost-routing.png) Provider expenses can either end up provider-side, or at-cost in your Vapi account. During calls, requests will be made to different providers in the voice pipeline: * **transcription providers:** providers conducting speech-to-text * **model providers:** LLM providers * **voice providers:** providers conducting text-to-speech * **telephony providers:** providers like [Twilio](https://www.twilio.com/) /[Vonage](https://www.vonage.com/) that facilitate phone calls Per-minute telephony costs only occur during inbound/outbound phone calling. Web calls do not incur this cost. Where Provider Costs End-up --------------------------- There are 2 places these charges can end up: 1. **Provider-side:** in the account you have with the provider. 2. **With Vapi:** in your Vapi account. ###### Billed Provider-side If we have [provider keys](/billing/customization/provider-keys) on file for a provider, the cost will be seen directly in your account with the provider. Vapi will have made the request on your behalf with your provider key. No charge will be made to your Vapi account. Charges for inbound/outbound phone calling (telephony) will always end up where the phone number was provisioned. If you import a phone number from Twilio or Vonage, per-minute charges for calling those numbers will appear with them. ###### Billed with Vapi If no key is found on-file for the provider, Vapi will make the API request itself (with Vapi’s own keys, at Vapi’s expense). This expense is then passed on [**at-cost**](/glossary#at-cost) to be billed directly to your Vapi account. No charge will show up provider-side. Billing That “Just Works” ------------------------- The central idea is that everything is designed to “just work”. Whether you are billed provider-side, or on Vapi’s side, you will never be charged with any margin for provider fees incurred during calls. [Built with](https://buildwithfern.com/?utm_campaign=buildWith&utm_medium=docs&utm_source=docs.vapi.ai) --- # Estimating Costs — Vapi Since there are so many moving parts to the voice pipeline that can incur cost, it would be ideal for us to get a good estimate of our final projected per-minute cost for calls. ### Dashboard Cost Estimates The Vapi dashboard provides static cost projections on a per-assistant basis, so you can get a rough idea of the costs your assistant will incur during live execution. You can view your dashboard at [dashboard.vapi.ai](https://dashboard.vapi.ai/) & get started with our [dashboard quickstart](/quickstart/dashboard) . ![](https://files.buildwithfern.com/https://vapi.docs.buildwithfern.com/2025-01-31T05:46:58.861Z/static/images/learn/billing/cost-estimate.gif) Vapi-computed per-minute cost projection, as well as latency projection. ### General Provider Estimates The provider costs listed below are subject to change as we get more data, but they will always reflect our best estimate of the provider costs per minute: ###### Transcription Provider Estimates | Provider | $/min (≈) | $/hour | | --- | --- | --- | | Deepgram | **$0.01/min** | $0.60/hr | ###### Model Provider Estimates | Provider | $/min (≈) | $/hour | | --- | --- | --- | | OpenAI (gpt-4-turbo) | **$0.20/min** | $12.00/hr | | OpenAI (gpt-3.5-turbo) | **$0.02/min** | $1.20/hr | ###### Voice Provider Estimates | Provider | $/min (≈) | $/hour | | --- | --- | --- | | ElevenLabs | **$0.04/min** | $2.40/hr | | PlayHT | **$0.07/min** | $4.20/hr | | Deepgram | **$0.02/min** | $1.20/hr | | OpenAI | **$0.02/min** | $1.20/hr | | RimeAI | **$0.03/min** | $1.80/hr | | Azure | **$0.02/min** | $1.20/hr | | Neets | **$0.005/min** | $0.30/hr | | LMNT | **$0.03/min** | $1.80/hr | ###### Telephony Provider Estimates | Provider | $/min (≈) | $/hour | | --- | --- | --- | | Twilio | **$0.01/min** | $0.60/hr | | Vonage | **$0.01/min** | $0.60/hr | ### Provider Pricings Here are direct links to different provider’s pricing pages to assist in estimating cost: ###### Transcription Providers [Deepgram\ \ Deepgram transcription pricing.](https://deepgram.com/pricing) ###### Model Providers [OpenAI\ \ OpenAI model pricing.](https://openai.com/pricing) ###### Voice Providers [ElevenLabs\ \ ElevenLabs voice pricing.](https://elevenlabs.io/pricing) [PlayHT\ \ PlayHT voice pricing.](https://play.ht/pricing) [Deepgram\ \ Deepgram voice pricing.](https://deepgram.com/pricing) [OpenAI\ \ OpenAI voice pricing.](https://openai.com/pricing) [RimeAI\ \ RimeAI voice pricing.](https://rime.ai/pricing) [Azure\ \ Azure voice pricing.](https://azure.microsoft.com/en-us/pricing/details/cognitive-services/speech-services) [Neets\ \ Neets voice pricing.](https://neets.ai/#pricing) [LMNT\ \ LMNT voice pricing.](https://app.lmnt.com/pricing) ###### Telephony Providers [Twilio\ \ Twilio phone call pricing.](https://www.twilio.com/en-us/voice/pricing) [Vonage\ \ Vonage phone call pricing.](https://www.vonage.com/communications-apis/voice/pricing) ### Calling Your Assistant One good way to get an empirical per-minute cost on your whole voice pipeline is to actually call in, use it for a few minutes, & observe the average cost/minute at the call level. You can view a breakdown of your cost per call in your dashboard at [dashboard.vapi.ai/calls](https://dashboard.vapi.ai/calls) Your call cost breakdowns will look something like this: ![](https://files.buildwithfern.com/https://vapi.docs.buildwithfern.com/2025-01-31T05:46:58.861Z/static/images/learn/billing/call-pricing-breakdown.png) Cost breakdown for a call viewed in the Vapi web dashboard. Here is what each line item corresponds to: * `STT`: Speech-to-text (providers often bill per-minute, prorated) * `LLM`: LLM inference (providers often bill per-million or per-thousand tokens) * `TTS`: Text-to-speech (providers often bill per-character) * `Vapi`: the Vapi platform fee of 5¢/minute (prorated per-second) * `Transport`: telephony costs (incurred for inbound/outbound phone calls to/from a phone number) (providers often bill per-minute) This method can be effective because **per-minute costs will not scale** with the amount of call minutes you consume. The cost for the 1st minute will be the same as the 10,000th minute. Volume pricing is available on enterprise plans. Check out [enterprise](/enterprise) to learn more. [Built with](https://buildwithfern.com/?utm_campaign=buildWith&utm_medium=docs&utm_source=docs.vapi.ai) --- # Pricing Overview — Vapi ![](https://files.buildwithfern.com/https://vapi.docs.buildwithfern.com/2025-01-31T05:46:58.861Z/static/images/pricing/voice-pipeline-cost-breakdown.png) Vapi charges a flat 5¢/min for calls. 5¢/min for Calls Vapi itself charges $0.05 per minute for calls. Prorated to the second. At-Cost for Providers Transcriber, model, voice, & telephony costs charged at-cost. Bring Your Own Keys Bring your own API keys for providers, Vapi makes requests on your behalf. $2/mo for Phone Numbers Phone numbers purchased through Vapi bill at $2/mo. ### Starter Credits Every new account is granted **$10 in free credits** to begin testing voice workflows. You can [begin using Vapi](/quickstart/dashboard) without a credit card. * * * Enterprise ---------- Handling a large volume of calls? You can find more information on our Enterprise plans [here](/enterprise) . * Higher concurrency and rate limits * Hands-on 24/7 support * Shared Slack channel with our team * Included minutes with volume pricing * Calls with our engineering team 2-3 times per week Further Reading --------------- [Routing Provider Cost\ \ Learn more about how Vapi routes provider costs.](/billing/cost-routing) [Estimating Costs\ \ Learn more about estimating costs for your voice pipeline.](/billing/estimating-costs) [Billing Limits\ \ Learn how to set billing limits for your account.](/billing/billing-limits) [Billing Examples\ \ Read full end-to-end billing breakdowns to better understand how Vapi bills.](/billing/examples) [Built with](https://buildwithfern.com/?utm_campaign=buildWith&utm_medium=docs&utm_source=docs.vapi.ai) --- # Billing Examples — Vapi ![](https://files.buildwithfern.com/https://vapi.docs.buildwithfern.com/2025-01-31T05:46:58.861Z/static/images/learn/billing/billing-example-template.png) Case Examples ------------- Here are a few case-examples of what billing would look like on Vapi for different voice pipeline configurations. ###### Call Center (Inbound Phone) A customer is looking to use Vapi to assist their call center staff taking phone calls inbound: [1](/billing/examples#scenario) ### Scenario “I want to use Vapi voice assistants to support my human customer service reps in a call center. However, I have a custom LLM I would prefer to use instead of the ones offered through the platform. Expected monthly usage will be 10,000 calls, with an average of 2 minutes per call. For Voice, PlayHT will suit our needs. What is my pricing breakdown?” [2](/billing/examples#providers) ### Providers The providers used will determine per-minute cost. The following providers will be involved: **Transcriber:** Deepgram **Model:** custom model **Voice:** PlayHT **Telephony:** Twilio (receiving inbound phone calls) ![](https://files.buildwithfern.com/https://vapi.docs.buildwithfern.com/2025-01-31T05:46:58.861Z/static/images/learn/billing/custom-model-inbound-phone-example.png) The customer will be using a custom model & taking inbound phone calls. [3](/billing/examples#cost-breakdown) ### Cost Breakdown We will break down the costs of each piece of the voice pipeline, then later multiply by call volume: **Deepgram:** ≈ $0.01/min **Custom Model:** ≈ $0.04/min (vague assumption, can vary widely) **PlayHT:** ≈ $0.07/min **Twilio:** ≈ $0.02/min (inbound, toll-free) (see Twilio [phone call pricing](https://www.twilio.com/en-us/voice/pricing) ) **Vapi:** $0.05/min Our [estimating costs](/billing/estimating-costs) guide can help you determine these values. [4](/billing/examples#final-estimate) ### Final Estimate Call Minutes / Month: 10,000 calls x 2 min/call = **20,000 call minutes** **Transcription:** $0.01/min x 20,000 = **$200** **Custom Model:** $0.04/min x 20,000 = **$800** **Voice:** ≈ $0.07/min x 20,000 = **$1,400** **Telephony:** ≈ $0.02/min x 20,000 = **$400** **Vapi:** $0.05/min x 20,000 = **$1,000** **Total**: **$3,800**/mo ###### Real Estate Lead Generation (Outbound Phone) A customer doing real estate lead generation is looking to use Vapi to automate parts of their sales calling operation: [1](/billing/examples#scenario-1) ### Scenario “I have a company that does real estate lead generation, and would like to use Vapi voice assistants to automate parts of my sales process. Calls would average ~4 minutes, for Model I want to use GPT-3.5-turbo through your platform, and for Voice I will be using a ElevenLabs. I’d like a breakdown based on sending 1,000 outbound calls in one month.” [2](/billing/examples#providers-1) ### Providers **Transcriber:** Deepgram **Model:** OpenAI (GPT-3.5 Turbo) **Voice:** ElevenLabs **Telephony:** Vonage (sending outbound phone calls) ![](https://files.buildwithfern.com/https://vapi.docs.buildwithfern.com/2025-01-31T05:46:58.861Z/static/images/learn/billing/outbound-phone-example.png) The customer will be using GPT-4 & making outbound phone calls. [3](/billing/examples#cost-breakdown-1) ### Cost Breakdown **Deepgram:** ≈ $0.01/min **OpenAI (GPT-3.5 Turbo):** ≈ $0.02/min **ElevenLabs:** ≈ $0.04/min **Vonage:** ≈ $0.01/min (outbound call) (see Vonage’s [phone call pricing](https://www.vonage.com/communications-apis/voice/pricing) ) **Vapi:** $0.05/min Our [estimating costs](/billing/estimating-costs) guide can help you determine these values. [4](/billing/examples#final-estimate-1) ### Final Estimate Call Minutes / Month: 1,000 calls x 4 min/call = **4,000 call minutes** **Transcription:** $0.01/min x 4,000 = **$40** **Model:** $0.02/min x 4,000 = **$80** **Voice:** ≈ $0.04/min x 4,000 = **$160** **Telephony:** ≈ $0.01/min x 4,000 = **$40** **Vapi:** $0.05/min x 4,000 = **$200** **Total**: **$520**/mo ###### Mock Intervieweing Application (Web) A web engineer is looking to develop a website that helps job candidates practice for job interviews. They are looking to use Vapi for their virtual interviewers: [1](/billing/examples#scenario-2) ### Scenario “Hi, I’m looking to develop a web application for mock interviews. Users will be able to practice for a variety of job interviews with AI interviewers. Interviews will be 30-minutes each (at max), for model I’ll be using a custom open-source model hosted with Baseten & for voice I’ll be using PlayHT. How much would this cost me each month if I service 1,000 interviews per month?” [2](/billing/examples#providers-2) ### Providers **Transcriber:** Deepgram **Model:** custom model **Voice:** PlayHT ![](https://files.buildwithfern.com/https://vapi.docs.buildwithfern.com/2025-01-31T05:46:58.861Z/static/images/learn/billing/web-interviews-example.png) The customer will be using a custom model & making web calls (no telephony will be involved). [3](/billing/examples#cost-breakdown-2) ### Cost Breakdown **Deepgram:** ≈ $0.01/min **Custom Model:** ≈ $0.02/min (vague assumption, can vary widely) **PlayHT:** ≈ $0.07/min **Vapi:** $0.05/min Our [estimating costs](/billing/estimating-costs) guide can help you determine these values. [4](/billing/examples#final-estimate-2) ### Final Estimate Call Minutes / Month: 1,000 calls x 30 min/call = **30,000 call minutes** **Transcription:** $0.01/min x 30,000 = **$300** **Model:** $0.02/min x 30,000 = **$600** **Voice:** ≈ $0.07/min x 30,000 = **$2,100** **Vapi:** $0.05/min x 30,000 = **$1,500** **Total**: **$4,500**/mo ### Further Reading [Provider Costs\ \ Learn more about where provider costs end up getting billed.](/billing/cost-routing) [Estimating Costs\ \ Learn more about determining per-minute costs for providers.](/billing/estimating-costs) [Built with](https://buildwithfern.com/?utm_campaign=buildWith&utm_medium=docs&utm_source=docs.vapi.ai) --- # Introduction — Vapi ![](https://files.buildwithfern.com/https://vapi.docs.buildwithfern.com/2025-01-31T05:46:58.861Z/static/images/intro/custom-vs-vapi.png) Vapi lets developers build, test, & deploy voice AI agents in minutes rather than months — solving for the foundational challenges voice AI applications face: Simulating the Flow of Natural Human Conversation Turn-taking, interruption handling, backchanneling, and more. Realtime/Low Latency Demands Responsive conversation demands low latency. Internationally. (<500-800ms voice-to-voice). Taking Actions (Function Calling) Taking actions during conversation, getting data to your services for custom actions. Extracting Conversation Data Review conversation audio, transcripts, & metadata. Implemented from scratch, this functionality can take months to build, and large, continuous, resources to maintain & improve. Vapi abstracts away these complexities, allowing developers to focus on the core of their voice AI application’s business logic. **Shipping in days, not months.** Quickstart Guides ----------------- Get up & running in minutes with one of our [quickstart](/quickstart) guides: #### No Code [Dashboard Quickstart\ \ The easiest way to start with Vapi. Run a voice agent in minutes.](/quickstart/dashboard) [Inbound Calling\ \ Quickly get started handling inbound phone calls.](/quickstart/phone/inbound) [Outbound Calling\ \ Quickly get started sending outbound phone calls.](/quickstart/phone/outbound) #### Platform-Specific [Web Quickstart\ \ Quickly get started making web calls. Web developers, this is for you.](/quickstart/web) Examples -------- Explore end-to-end examples for some common voice workflows: [Outbound Sales\ \ We’ll build an outbound sales agent that can schedule appointments.](/examples/outbound-sales) [Inbound Support\ \ We’ll build an technical support assistant that remembers where we left off.](/examples/inbound-support) [Pizza Website\ \ We’ll build an order taking agent for our pizza website.](/examples/pizza-website) Key Concepts ------------ Gain a deep understanding of key concepts in Vapi, as well as how Vapi works: #### Core Concepts [Assistants\ \ Assistants set the foundation for applications built on Vapi.](/assistants) [Server URLs\ \ Server URLs allow Vapi to deliver your application data in realtime.](/server-url) [Phone Calling\ \ Learn the ins-and-outs of telephony & conducting phone calls on Vapi.](/phone-calling) [Privacy\ \ Learn about privacy concepts like HIPAA & data privacy on Vapi.](/security-and-privacy/hipaa) #### Platform [How Vapi Works\ \ Learn what goes on behind-the-scenes to make Vapi work.](/quickstart) Explore Our SDKs ---------------- Our SDKs are open source, and available on [our GitHub](https://github.com/VapiAI) : [Vapi Web\ \ Add a Vapi assistant to your web application.](/sdk/web) [Vapi iOS\ \ Add a Vapi assistant to your iOS app.](https://github.com/VapiAI/ios) [Vapi Flutter\ \ Add a Vapi assistant to your Flutter app.](https://github.com/VapiAI/flutter) [Vapi React Native\ \ Add a Vapi assistant to your React Native app.](https://github.com/VapiAI/react-native-sdk) [Vapi Python\ \ Multi-platform. Mac, Windows, and Linux.](https://github.com/VapiAI/python) FAQ --- Common questions asked by other users: ###### Is Vapi right for my usecase? If you are **a developer building a voice AI application simulating human conversation** (w/ LLMs — to whatever degree of application complexity) — Vapi is built for you. Whether you are building for a completely “turn-based” use case (like appointment setting), all the way to robust agentic voice applications (like virtual assistants), Vapi is tooled to solve for your voice AI workflow. Vapi runs on any platform: the web, mobile, or even embedded systems (given network access). ###### Sounds good, but I’m building a custom X for Y... Not a problem, we can likely already support it. Vapi is designed to be modular at every level of the voice pipeline: Text-to-speech, LLM, Speech-to-text. You can bring your own custom models for any part of the pipeline. * **If they’re hosted with one of our providers:** you just need to add your [provider keys](/customization/provider-keys) , then specify the custom model in your API requests. * **If they are hosted elsewhere:** you can use the `Custom LLM` provider and specify the [URL to your model](/customization/custom-llm/fine-tuned-openai-models) in your API request. Everything is interchangeable, mix & match to suit your usecase. ###### Couldn’t I build this myself and save money? You could (and the person writing this right now did, from scratch) — but there are good reasons for not doing so. Writing a great realtime voice AI application from scratch is a fairly challenging task (more on those challenges [here](/challenges-of-realtime-conversation) ). Most of these challenges are not apparent until you face them, then you realize you are 3 weeks into a rabbit hole that may take months to properly solve out of. Think of Vapi as hiring a software engineering team for this hard problem, while you focus on what uniquely generates value for your voice AI application. * * * But to address cost, the vast majority of cost in running your application will come from provider cost (Speect-to-text, LLM, Text-to-speech) direct with vendors (Deepgram, OpenAI, ElevenLabs, etc) — where we add no fee (vendor cost passes-through). These would have to be incurred anyway. Vapi only charges its small fee on top of these for the continuous maintenance & improvement of these hardest components of your system (which would have costed you time to write/maintain). No matter what, some cost is inescapable (in money, time, etc) to solve this challenging technical problem. Our focus is solely on foundational Voice AI orchestration, & it’s what we put our full time and resources into. To learn more about Vapi’s pricing, you can visit our [pricing page](/pricing) . ###### Is it going to be hard to set up? No — in fact, the setup could not be easier: * **Web Dashboard:** It can take minutes to get up & running with our [dashboard](https://dashboard.vapi.ai/) . * **Client SDKs:** You can start calls with 1 line of code with any of our [client SDKs](/sdks) . For more advanced features like function calling, you will have to set up a [Server URL](/server-url) to receive and respond to messages. ###### How is Vapi different from other Voice AI services? Vapi focuses on developers. Giving developers modular, simple, & robust tooling to build any voice AI application imaginable. Vapi also has some of the lowest latency & (equally important) highest reliability amongst any other voice AI platform built for developers. Get Support ----------- Join our Discord to connect with other developers & connect with our team: [Join Our Discord\ \ Connect with our team & other developers using Vapi.](https://discord.gg/pUFNcf2WmH) [Email Support\ \ Send our support team an email.](https://tally.so/r/3yD9Wx) [Built with](https://buildwithfern.com/?utm_campaign=buildWith&utm_medium=docs&utm_source=docs.vapi.ai) --- # Vapi Enterprise — Vapi If you’re building a production application on Vapi, we can help you every step of the way from idea to full-scale deployment. #### Enterprise Plans include: * Unlimited concurrency and higher rate limits * Reserved capacity on our dedicated Enterprise Cluster * Hands-on 24/7 support with dedicated solutions engineer * Shared Slack channel with our team * Regular check-in calls with our team Contact us ---------- To get started on Vapi Enterprise, [fill out this form](https://book.vapi.ai/) . [Built with](https://buildwithfern.com/?utm_campaign=buildWith&utm_medium=docs&utm_source=docs.vapi.ai) --- # HIPAA Compliance — Vapi Introduction to Privacy at Vapi ------------------------------- At Vapi, we are committed to delivering exceptional voice assistant services while upholding the highest standards of privacy and data protection for our users. We understand the importance of balancing service quality with the need to respect and protect personal and sensitive information. Our privacy policies and practices are designed to give you control over your data while benefiting from the full capabilities of our platform. Understanding HIPAA Compliance Basics ------------------------------------- The Health Insurance Portability and Accountability Act (HIPAA) is a United States legislation that provides data privacy and security provisions for safeguarding medical information. HIPAA compliance is crucial for any entity that deals with protected health information (PHI), ensuring that sensitive patient data is handled, stored, and transmitted with the highest standards of security and confidentiality. The key concepts of HIPAA compliance include the Privacy Rule, which protects the privacy of individually identifiable health information; the Security Rule, which sets standards for the security of electronic protected health information (e-PHI); and the Breach Notification Rule, which requires covered entities to notify individuals, HHS, and in some cases, the media of a breach of unsecured PHI. Compliance with these rules is not just about adhering to legal requirements but also about building trust with your customers by demonstrating your commitment to protecting their sensitive data. By enabling the `hipaaEnabled` configuration in Vapi’s voice assistant platform, you are taking a significant step towards aligning your operations with these HIPAA principles, ensuring that your use of technology adheres to these critical privacy and security standards. Understanding Default Settings ------------------------------ By default, Vapi records your calls and stores logs and transcriptions. This practice is aimed at continuously improving the quality of our service, ensuring that you receive the best possible experience. However, we recognize the importance of privacy and provide options for users who prefer more control over their data. Opting for Privacy: The HIPAA Compliance Option ----------------------------------------------- For users prioritizing privacy, particularly in compliance with the Health Insurance Portability and Accountability Act (HIPAA), Vapi offers the flexibility to opt out of our default data recording settings. Choosing HIPAA compliance through our platform ensures that you can still use our voice assistant services without compromising on privacy requirements. Enabling HIPAA Compliance ------------------------- HIPAA compliance can be ensured by enabling the `hipaaEnabled` configuration in your assistant settings. This simple yet effective setting guarantees that no call logs, recordings, or transcriptions are stored during or after your calls. An end-of-call report message will be generated and stored on your server for record-keeping, ensuring compliance without storing sensitive data on Vapi’s systems. To enable HIPAA compliance, set hipaaEnabled to true within your assistant’s configuration: ` | | | | --- | --- | | 1 | { | | 2 | "hipaaEnabled": true | | 3 | } | ` Note: The default value for hipaaEnabled is false. Activating this setting is a proactive measure to align with HIPAA standards, requiring manual configuration adjustment. FAQs ---- **Q: Will enabling HIPAA compliance affect the quality of Vapi’s service?** A: Enabling HIPAA compliance does not degrade the quality of the voice assistant services. However, it limits access to certain features, such as reviewing call logs or transcriptions, that some users may find valuable for quality improvement purposes. **Q: Who should use the HIPAA compliance feature?** A: This feature is particularly useful for businesses and organizations in the healthcare sector or any entity that handles sensitive health information and must comply with HIPAA regulations. **Q: Can I switch between default and HIPAA-compliant settings?** A: Yes, users can toggle the hipaaEnabled setting as needed. However, we recommend carefully considering the implications of each option on your data privacy and compliance requirements. Need Further Assistance? ------------------------ If you have more questions about privacy, HIPAA compliance, or how to configure your Vapi assistant, our support team is here to help. Contact us at [\[email protected\]](/cdn-cgi/l/email-protection#5b283e382e29322f221b2d3a2b32753a32) for personalized assistance and more information on how to make the most of Vapi’s voice assistant platform while ensuring your data remains protected. [Built with](https://buildwithfern.com/?utm_campaign=buildWith&utm_medium=docs&utm_source=docs.vapi.ai) --- # Support — Vapi Join Vapi community ------------------- * To take part in community discussion join our [Discord](https://discord.com/invite/pUFNcf2WmH) server to collaborate with other developers. * For quick support: Visit #support channel to submit support requests. [Built with](https://buildwithfern.com/?utm_campaign=buildWith&utm_medium=docs&utm_source=docs.vapi.ai) --- # Client SDKs — Vapi The Vapi Client SDKs automatically configure audio streaming to and from the client, and provide a simple interface for starting calls. The interface is equivalent across all the SDKs. The SDKs are open source, and available on GitHub: [Vapi Web\ \ Add a Vapi assistant to your web application.](/sdk/web) [Vapi iOS\ \ Add a Vapi assistant to your iOS app.](https://github.com/VapiAI/ios) [Vapi Flutter\ \ Add a Vapi assistant to your Flutter app.](https://github.com/VapiAI/flutter) [Vapi React Native\ \ Add a Vapi assistant to your React Native app.](https://github.com/VapiAI/react-native-sdk) [Vapi Python\ \ Multi-platform. Mac, Windows, and Linux.](https://github.com/VapiAI/python) * * * ###### Events * `speech-start`, `speech-end`, and `volume-level` for creating animations. - `message` for receiving messages sent to the [Server URL](/server-url) locally on the client, so you can show live transcriptions and use function calls to perform actions on the client. [Built with](https://buildwithfern.com/?utm_campaign=buildWith&utm_medium=docs&utm_source=docs.vapi.ai) --- # On-Prem Deployments — Vapi Vapi On-Prem allows you to deploy Vapi’s best in class enterprise voice infrastructure AI directly in your own cloud. It can be deployed in a dockerized format on any cloud provider, in any geographic location, running on your GPUs. With On-Prem, your audio and text data stays in your cloud. Data never passes through Vapi’s servers. If you’re are handling sensitive data (e.g. health, financial, legal) and are under strict data requirements, you should consider deploying on-prem. Your device regularly sends performance and usage information to Vapi’s cloud. This data helps adjust your device’s GPU resources and is also used for billing. All network traffic from your device is tracked in an audit log, letting your engineering or security team see what the device is doing at all times. Frequently Asked Questions -------------------------- #### Can the appliance adjust to my needs? Yes, the Vapi On-Prem appliance automatically adjusts its GPU resources to handle your workload as required by our service agreement. It can take a few minutes to adjust to changes in your workload. If you need quicker adjustments, you might want to ask for more GPUs by contacting [\[email protected\]](/cdn-cgi/l/email-protection#50232520203f222410263120397e3139) . #### What if I can’t get enough GPUs from my cloud provider? If you’re struggling to get more GPUs from your provider, contact [\[email protected\]](/cdn-cgi/l/email-protection#2a595f5a5a45585e6a5c4b5a43044b43) for help. #### Can I access Vapi’s AI models? No, our AI models are on secure machines in your Isolated VPC and you can’t log into these machines or check their files. #### How can I make sure my data stays within my cloud? Your device operates in VPCs that you control. You can check the network settings and firewall rules, and look at traffic logs to make sure everything is as it should be. The Control VPC uses open source components, allowing you to make sure the policies are being followed. Performance data and model updates are sent to Vapi, but all other traffic leaving your device is logged, except for the data sent back to your API clients. Contact us ---------- For more information about Vapi On-Prem, please contact us at [\[email protected\]](/cdn-cgi/l/email-protection#9deee8ededf2efe9ddebfcedf4b3fcf4) [Built with](https://buildwithfern.com/?utm_campaign=buildWith&utm_medium=docs&utm_source=docs.vapi.ai) --- # Server SDKs — Vapi Vapi provides server-side SDKs to help developers quickly integrate and manage voice AI capabilities into their applications. Our SDKs allow seamless interaction with Vapi’s API across a wide range of programming languages, ensuring you can choose the stack that best suits your needs. The SDKs are open source, and available on GitHub: [Vapi Python\ \ Add a Vapi assistant to your Python application.](https://github.com/VapiAI/server-sdk-python) [Vapi TypeScript\ \ Add a Vapi assistant to your TypeScript application.](https://github.com/VapiAI/server-sdk-typescript) [Vapi Java\ \ Add a Vapi assistant to your Java application.](https://github.com/VapiAI/server-sdk-java) [Vapi Ruby\ \ Add a Vapi assistant to your Ruby application.](https://github.com/VapiAI/server-sdk-ruby) [Vapi C#\ \ Add a Vapi assistant to your C#/.NET application.](https://github.com/VapiAI/server-sdk-csharp) [Vapi Go\ \ Add a Vapi assistant to your Go application.](https://github.com/VapiAI/server-sdk-go) [Built with](https://buildwithfern.com/?utm_campaign=buildWith&utm_medium=docs&utm_source=docs.vapi.ai) --- # Web Snippet — Vapi Improve your website’s user interaction with the Vapi Voice Widget. This robust tool enables your visitors to engage with a voice assistant for support and interaction, offering a smooth and contemporary way to connect with your services. Steps for Installation ---------------------- [1](/examples/voice-widget#insert-the-widget-snippet) ### Insert the Widget Snippet Copy the snippet below and insert it into your website’s HTML, ideally before the closing `` tag. ` | | | | --- | --- | | 1 | | ` [2](/examples/voice-widget#generate-your-assistant) ### Generate Your Assistant From your Vapi dashboard, create an assistant to get the assistant ID. Alternatively, define an assistant configuration directly in your website’s code as demonstrated in the example below. ` | | | | --- | --- | | 1 | const assistant = { | | 2 | model: { | | 3 | provider: "openai", | | 4 | model: "gpt-3.5-turbo", | | 5 | systemPrompt: | | 6 | "You're a versatile AI assistant named Vapi who is fun to talk with.", | | 7 | }, | | 8 | voice: { | | 9 | provider: "11labs", | | 10 | voiceId: "paula", | | 11 | }, | | 12 | firstMessage: "Hi, I am Vapi how can I assist you today?", | | 13 | }; | ` [3](/examples/voice-widget#modify-the-button) ### Modify the Button Modify the `buttonConfig` object to align with your website’s style and branding. Choose between a pill or round button and set colors, positions, and icons. `` | | | | --- | --- | | 1 | const buttonConfig = { | | 2 | position: "bottom-right", // "bottom" \| "top" \| "left" \| "right" \| "top-right" \| "top-left" \| "bottom-left" \| "bottom-right" | | 3 | offset: "40px", // decide how far the button should be from the edge | | 4 | width: "50px", // min-width of the button | | 5 | height: "50px", // height of the button | | 6 | idle: { // button state when the call is not active. | | 7 | color: `rgb(93, 254, 202)`, | | 8 | type: "pill", // or "round" | | 9 | title: "Have a quick question?", // only required in case of Pill | | 10 | subtitle: "Talk with our AI assistant", // only required in case of pill | | 11 | icon: `https://unpkg.com/[[email protected]](/cdn-cgi/l/email-protection)
/icons/phone.svg`, | | 12 | }, | | 13 | loading: { // button state when the call is connecting | | 14 | color: `rgb(93, 124, 202)`, | | 15 | type: "pill", // or "round" | | 16 | title: "Connecting...", // only required in case of Pill | | 17 | subtitle: "Please wait", // only required in case of pill | | 18 | icon: `https://unpkg.com/[[email protected]](/cdn-cgi/l/email-protection)
/icons/loader-2.svg`, | | 19 | }, | | 20 | active: { // button state when the call is in progress or active. | | 21 | color: `rgb(255, 0, 0)`, | | 22 | type: "pill", // or "round" | | 23 | title: "Call is in progress...", // only required in case of Pill | | 24 | subtitle: "End the call.", // only required in case of pill | | 25 | icon: `https://unpkg.com/[[email protected]](/cdn-cgi/l/email-protection)
/icons/phone-off.svg`, | | 26 | }, | | 27 | }; | `` [4](/examples/voice-widget#add-functionality-to-vapi-instance) ### Add Functionality to Vapi Instance You can use the `vapiInstance` returned from the run function in the snippet to further customize the behaviour. For instance, you might want to listen to various EventSource, or even send some messages to the bot programmatically. `` | | | | --- | --- | | 1 | vapiInstance.on('speech-start', () => { | | 2 | console.log('Speech has started'); | | 3 | }); | | 4 | | | 5 | vapiInstance.on('speech-end', () => { | | 6 | console.log('Speech has ended'); | | 7 | }); | | 8 | | | 9 | vapiInstance.on('call-start', () => { | | 10 | console.log('Call has started'); | | 11 | }); | | 12 | | | 13 | vapiInstance.on('call-end', () => { | | 14 | console.log('Call has stopped'); | | 15 | }); | | 16 | | | 17 | vapiInstance.on('volume-level', (volume) => { | | 18 | console.log(`Assistant volume level: ${volume}`); | | 19 | }); | | 20 | | | 21 | // Function calls and transcripts will be sent via messages | | 22 | vapiInstance.on('message', (message) => { | | 23 | console.log(message); | | 24 | }); | | 25 | | | 26 | vapiInstance.on('error', (e) => { | | 27 | console.error(e) | | 28 | }); | `` Customization ------------- Modify your assistant’s behavior and the initial message users will see. Refer to the provided examples to customize the assistant’s model, voice, and initial greeting. UI Customization ---------------- For advanced styling, target the exposed CSS and other classes to ensure the widget’s appearance aligns with your website’s design. Here is a list of the classes you can customize: * `.vapi-btn`: The primary class for the Vapi button. * `.vapi-btn-is-idle`: The class for the Vapi button when the call is disconnected. * `.vapi-btn-is-active`: The class for the Vapi button when the call is active. * `.vapi-btn-is-loading`: The class for the Vapi button when the call is connecting. * `.vapi-btn-is-speaking`: The class for the Vapi button when the bot is speaking. * `.vapi-btn-pill`: The class for Vapi button to set pill variant. * `.vapi-btn-round`: The class for Vapi button to set round variant. [Built with](https://buildwithfern.com/?utm_campaign=buildWith&utm_medium=docs&utm_source=docs.vapi.ai) --- # Code Resources — Vapi | Vapi AI Ecosystem | | | --- | --- | | Real-time SDKs | [Web](https://github.com/VapiAI/web)
· [Flutter](https://github.com/VapiAI/flutter)
· [React Native](https://github.com/VapiAI/react-native-sdk)
· [iOS](https://github.com/VapiAI/ios)
· [Python](https://github.com/VapiAI/python)
· [Vanilla](https://github.com/VapiAI/html-script-tag) | | Client Examples | [Next.js](https://github.com/VapiAI/client-side-example-javascript-next)
· [React](https://github.com/VapiAI/client-side-example-javascript-react)
· [Flutter](https://github.com/VapiAI/flutter/tree/main/example)
· [React Native](https://github.com/VapiAI/client-side-example-react-native) | | Server Examples | [Vercel](https://github.com/VapiAI/server-side-example-serverless-vercel)
· [Cloudflare](https://github.com/VapiAI/server-side-example-serverless-cloudflare)
· [Supabase](https://github.com/VapiAI/server-side-example-serverless-supabase)
· [Node](https://github.com/VapiAI/server-side-example-javascript-node)
· [Bun](https://github.com/VapiAI/server-side-example-javascript-bun)
· [Deno](https://github.com/VapiAI/server-side-example-javascript-deno)
· [Flask](https://github.com/VapiAI/server-side-example-python-flask)
· [Laravel](https://github.com/VapiAI/server-side-example-php-laravel)
· [Go](https://github.com/VapiAI/server-side-example-go-gin)
· [Rust](https://github.com/VapiAI/server-side-example-rust-actix) | | Resources | [Official Docs](https://docs.vapi.ai/)
· [API Reference](https://api.vapi.ai/api) | | Community | [Videos](/community/videos)
. [UI Library](https://www.vapiblocks.com/) | [Built with](https://buildwithfern.com/?utm_campaign=buildWith&utm_medium=docs&utm_source=docs.vapi.ai) --- # Inbound Calling — Vapi An inbound call is a phone call that comes **“in”** towards a phone number, & in our case, our AI assistant will be there to pick up the phone call. There are **4 steps** we will cover to handle our first inbound phone call: 1. **Create an Assistant:** we will create an [assistant](/assistants) & instruct it on how to conduct the call 2. **Get a Phone Number:** we can either import existing numbers we own, or purchase one through Vapi 3. **Attach Our Assistant:** we will put our assistant behind the phone number to pick up calls 4. **Call the Number:** we can then call the number & talk to our assistant Vapi’s Pizzeria --------------- We will be implementing a simple order-taking assistant that receives customer calls at a pizza shop called “Vapi’s Pizzeria”. Vapi’s has 3 types of menu items: `pizza`, `side`s, & `drink`s. Customers will be ordering 1 of each. ![](https://files.buildwithfern.com/https://vapi.docs.buildwithfern.com/2025-01-31T05:46:58.861Z/static/images/quickstart/vapis-pizzeria.png) Customers will order 3 items: 1 pizza, 1 side, & 1 drink. The assistant will pick up the phone & take the customer's order. Assistant Setup --------------- First we’re going to set up our assistant in the dashboard. Once our assistant’s **transcriber**, **model**, & **voice** are set up, we can call it to place our order. You can visit your dashboard by going to [dashboard.vapi.ai](https://dashboard.vapi.ai/) ###### Sign-up or Log-in to Vapi If you haven’t already signed-up, you’re going to need an account before you can use the web dashboard. When you visit [dashboard.vapi.ai](https://dashboard.vapi.ai/) you may see something like this: ![](https://files.buildwithfern.com/https://vapi.docs.buildwithfern.com/2025-01-31T05:46:58.861Z/static/images/quickstart/dashboard/auth-ui.png) Sign-up for an account (or log-in to your existing account) — you will then find yourself inside the web dashboard. It will look something like this: ![](https://files.buildwithfern.com/https://vapi.docs.buildwithfern.com/2025-01-31T05:46:58.861Z/static/images/quickstart/dashboard/vapi-dashboard-post-signup.png) Your dashboard may look a bit different if you already have an account with assistants in it. The main idea is that we’re in the dashboard now. ###### Create an Assistant Now that you’re in your dashboard, we’re going to create an [assistant](/assistants) . Assistants are at the heart of how Vapi models AI voice agents — we will be setting certain properties on a new assistant to model an order-taking agent. Once in the “Assistants” dashboard tab (you should be in it by-default after log-in), you will see a button to create a new assistant. ![](https://files.buildwithfern.com/https://vapi.docs.buildwithfern.com/2025-01-31T05:46:58.861Z/static/images/quickstart/dashboard/create-new-assistant-button.png) Ensure you are in the 'Assistants' dashboard tab, then this button will allow you to begin the assistant creation flow. After clicking the create new assistant button, you will see a pop-up modal that asks you to pick a starter template. For our example we will start from a blank slate so choose the `Blank Template` option. ![](https://files.buildwithfern.com/https://vapi.docs.buildwithfern.com/2025-01-31T05:46:58.861Z/static/images/quickstart/dashboard/choose-blank-template.png) Ensure you are in the 'Assistants' dashboard tab, then this button will allow you to begin the assistant creation flow. You will then be able to name your assistant — you can name it whatever you’d like (`Vapi’s Pizza Front Desk`, for example): This name is only for internal labeling use. It is not an identifier, nor will the assistant be aware of this name. ![](https://files.buildwithfern.com/https://vapi.docs.buildwithfern.com/2025-01-31T05:46:58.861Z/static/images/quickstart/dashboard/name-your-assistant.png) Name your assistant. Once you have named your assistant, you can hit “Create” to create it. You will then see something like this: ![](https://files.buildwithfern.com/https://vapi.docs.buildwithfern.com/2025-01-31T05:46:58.861Z/static/images/quickstart/dashboard/assistant-created.png) The assistant overview. You can edit your assistant’s transcriber, model, & voice — and edit other advanced configuration. This is the assistant overview view — it gives you the ability to edit different attributes about your assistant, as well as see **cost** & **latency** projection information for each portion of it’s voice pipeline (this is very important data to have handy when building out your assistants). ###### Model Setup Now we’re going to set the “brains” of the assistant, the large language model. We’re going to be using `GPT-4` (from [OpenAI](https://openai.com/) ) for this demo (though you’re free to use `GPT-3.5`, or any one of your favorite LLMs). ###### Set Your OpenAI Provider Key (optional) Before we proceed, we can set our [provider key](https://docs.vapi.ai/customization/provider-keys) for OpenAI (this is just your OpenAI secret key). You can see all of your provider keys in the “Provider Keys” dashboard tab. You can also go directly to [dashboard.vapi.ai/keys](https://dashboard.vapi.ai/keys) . Vapi uses [provider keys](https://docs.vapi.ai/customization/provider-keys) you provide to communicate with LLM, TTS, & STT vendors on your behalf. It is most ideal that we set keys for the vendors we intend to use ahead of time. ![](https://files.buildwithfern.com/https://vapi.docs.buildwithfern.com/2025-01-31T05:46:58.861Z/static/images/quickstart/dashboard/model-provider-keys.png) We set our provider key for OpenAI so Vapi can make requests to their API. While we’re here it’d be ideal for you to go & set up provider keys for other providers you’re familiar with & intend to use later. ###### Set a First Message Assistants can **optionally** be configured with a `First Message`. This first message will be spoken by your assistant when either: * **A Web Call Connects:** when a web call is started with your assistant * **An Inbound Call is Picked-up:** an [inbound call](/glossary#inbound-call) is picked-up & answered by your assistant * **An Outbound Call is Dialed & Picked-up:** an [outbound call](/glossary#outbound-call) is dialed by your assistant & a person picks up Note that this first message cannot be interrupted & is guaranteed to be spoken. Certain use cases need a first message, while others do not. For our use case, we will want a first message. It would be ideal for us to have a first message like this: ` Vappy’s Pizzeria speaking, how can I help you? ` Some text-to-speech voices may struggle to pronounce ‘Vapi’ correctly, compartmentalizing it to be spoken letter by letter “V. A. P. I.” Some aspects of configuring your voice pipeline will require tweaks like this to get the target behaviour you want. This will be spoken by the assistant when a web or inbound phone call is received. ###### Set the System Prompt We will now set the `System Prompt` for our assistant. If you’re familiar with OpenAI’s API, this is the first prompt in the message list that we feed our LLM (learn more about prompt engineering on the [OpenAI docs](https://platform.openai.com/docs/guides/prompt-engineering) ). The system prompt can be used to configure the context, role, personality, instructions and so on for the assistant. In our case, a system prompt like this will give us the behaviour we want: ` | | | --- | | You are a voice assistant for Vappy’s Pizzeria, | | a pizza shop located on the Internet. | | | | Your job is to take the order of customers calling in. The menu has only 3 types | | of items: pizza, sides, and drinks. There are no other types of items on the menu. | | | | 1) There are 3 kinds of pizza: cheese pizza, pepperoni pizza, and vegetarian pizza | | (often called "veggie" pizza). | | 2) There are 3 kinds of sides: french fries, garlic bread, and chicken wings. | | 3) There are 2 kinds of drinks: soda, and water. (if a customer asks for a | | brand name like "coca cola", just let them know that we only offer "soda") | | | | Customers can only order 1 of each item. If a customer tries to order more | | than 1 item within each category, politely inform them that only 1 item per | | category may be ordered. | | | | Customers must order 1 item from at least 1 category to have a complete order. | | They can order just a pizza, or just a side, or just a drink. | | | | Be sure to introduce the menu items, don't assume that the caller knows what | | is on the menu (most appropriate at the start of the conversation). | | | | If the customer goes off-topic or off-track and talks about anything but the | | process of ordering, politely steer the conversation back to collecting their order. | | | | Once you have all the information you need pertaining to their order, you can | | end the conversation. You can say something like "Awesome, we'll have that ready | | for you in 10-20 minutes." to naturally let the customer know the order has been | | fully communicated. | | | | It is important that you collect the order in an efficient manner (succinct replies | | & direct questions). You only have 1 task here, and it is to collect the customers | | order, then end the conversation. | | | | - Be sure to be kind of funny and witty! | | - Keep all your responses short and simple. Use casual language, phrases like "Umm...", "Well...", and "I mean" are preferred. | | - This is a voice conversation, so keep your responses short, like in a real conversation. Don't ramble for too long. | ` You can copy & paste the above prompt into the `System Prompt` field. Now the model configuration for your assistant should look something like this: ![](https://files.buildwithfern.com/https://vapi.docs.buildwithfern.com/2025-01-31T05:46:58.861Z/static/images/quickstart/dashboard/assistant-model-set-up.png) Note how our model provider is set to OpenAI & the model is set to GPT-4. ###### Transcriber Setup The transcriber is what turns user speech into processable text for our LLM. This is the first step in the end-to-end voice pipeline. ###### Set Your Deepgram Provider Key (optional) We will be using [Deepgram](https://deepgram.com/) (which provides blazing-fast & accurate Speech-to-Text) as our STT provider. We will set our provider key for them in “Provider Keys”: ![](https://files.buildwithfern.com/https://vapi.docs.buildwithfern.com/2025-01-31T05:46:58.861Z/static/images/quickstart/dashboard/transcriber-providers-keys.png) ###### Set Transcriber We will set the model to `Nova 2` & the language to `en` for English. Now your assistant’s transcriber configuration should look something like this: ![](https://files.buildwithfern.com/https://vapi.docs.buildwithfern.com/2025-01-31T05:46:58.861Z/static/images/quickstart/dashboard/assistant-transcriber-config.png) Note how our transcriber is set to 'deepgram', the model is set to 'Nova 2', & the language is set to English. ###### Voice Setup The final portion of the voice pipeline is turning LLM output-text into speech. This process is called “Text-to-speech” (or TTS for short). We will be using a voice provider called [PlayHT](https://play.ht/) (they have very conversational voices), & a voice provided by them labeled `Jennifer` (`female`, `en-US`). You are free to use your favorite TTS voice platform here. [ElevenLabs](https://elevenlabs.io/) is another alternative — by now you should get the flow of plugging in vendors into Vapi (add provider key + pick provider in assistant config). You can skip the next step(s) if you don’t intend to use PlayHT. ###### Set Your PlayHT Provider Key (optional) If you haven’t already, sign up for an account with PlayHT at [play.ht](https://play.ht/) . Since their flows are liable to change — you can just grab your `API Key` & `User ID` from them. ![](https://files.buildwithfern.com/https://vapi.docs.buildwithfern.com/2025-01-31T05:46:58.861Z/static/images/quickstart/dashboard/voice-provider-keys.png) ###### Set Voice You will want to select `playht` in the “provider” field, & `Jennifer` in the “voice” field. We will leave all of the other settings untouched. ![](https://files.buildwithfern.com/https://vapi.docs.buildwithfern.com/2025-01-31T05:46:58.861Z/static/images/quickstart/dashboard/assistant-voice-config.png) Each voice provider offers a host of settings you can modulate to customize voices. Here we will leave all the defaults alone. Get a Phone Number ------------------ Now that we’ve configured how our assistant will behave, we want to figure out how to call it. We will need a phone number that we can make phone calls to. There are **2 ways** we can get a phone number into our Vapi account: 1. **Purchase a Number Through Vapi:** we can directly purchase phone numbers through Vapi. * Vapi will provision the phone number for us via Twilio * This can be done in the dashboard, or via the API (we will use the dashboard) 2. **Import from Twilio or Vonage:** if we already have a phone number with an external telephony provider (like Twilio or Vonage), we can import them into our Vapi account. ###### Provision via Vapi (faster) The quickest way to secure a phone number for your assistant is to purchase a phone number directly through Vapi. Ensure you have a card on file that Vapi can bill before proceeding, you can add your billing information in your dashboard at [dashboard.vapi.ai/billing](https://dashboard.vapi.ai/billing) Navigate to the “Phone Numbers” section & click the “Buy number” button: ![](https://files.buildwithfern.com/https://vapi.docs.buildwithfern.com/2025-01-31T05:46:58.861Z/static/images/quickstart/dashboard/buy-a-phone-number.png) Make sure you are in the 'Phone Numbers' dashboard tab. We will use the area code `415` for our phone number (these are area codes domestic to the US & Canada). ![](https://files.buildwithfern.com/https://vapi.docs.buildwithfern.com/2025-01-31T05:46:58.861Z/static/images/quickstart/dashboard/buy-phone-number-modal.png) Choose an area code for your phone number. Currently, only US & Canada phone numbers can be directly purchased through Vapi. Phone numbers in other regions must be imported, see our [phone calling](/phone-calling) guide. Click “Buy”, after purchasing a phone number you should see something like this: ![](https://files.buildwithfern.com/https://vapi.docs.buildwithfern.com/2025-01-31T05:46:58.861Z/static/images/quickstart/dashboard/phone-number-config.png) Here we can attach an assistant to the number for inbound calls (or perform an outbound call, with a select assistant). The phone number is now ready to be used (either for inbound or outbound calling). ###### Import from Twilio or Vonage We can also import an existing phone number we already own with either Twilio or Vonage. For example’s sake, we will proceed with [**Twilio**](https://twilio.com/) (though the steps are the same for Vonage as well). ###### Buy a Phone Number via Twilio (optional) If you don’t already have a number in Twilio, you can purchase one by going to your Twilio console’s “Buy a number” section: ![](https://files.buildwithfern.com/https://vapi.docs.buildwithfern.com/2025-01-31T05:46:58.861Z/static/images/quickstart/phone/buy-phone-number-twilio.png) The Twilio 'Buy a Number' page in the Twilio console. Once you’ve purchased a number, it will immediately be ready for import into Vapi. ###### Locate Twilio Account SID & Auth Token To complete the import on Vapi’s side, we will need to grab our Twilio **“Account SID”** & **“Auth Token”**. You should see a section for “API keys & tokens”, the credentials we will need for the import will live here. ![](https://files.buildwithfern.com/https://vapi.docs.buildwithfern.com/2025-01-31T05:46:58.861Z/static/images/quickstart/phone/twilio-api-key-nav.png) We will want to navigate to the credentials section of our account. Once we are in our “API keys & tokens” section, we will grab the Account SID & Auth Token: ![](https://files.buildwithfern.com/https://vapi.docs.buildwithfern.com/2025-01-31T05:46:58.861Z/static/images/quickstart/phone/twilio-credentials.png) We will use both of these credentials in the next step of importing via the Dashboard. ###### Import via Dashboard Navigate to the “Phone Numbers” section & click the “Import” button: ![](https://files.buildwithfern.com/https://vapi.docs.buildwithfern.com/2025-01-31T05:46:58.861Z/static/images/quickstart/phone/dashboard-import-phone-number.png) Click 'Import' in the 'Phone Numbers' tab of your dashboard. There you will input your phone number, as well as the credentials you retrieved in the previous step: ![](https://files.buildwithfern.com/https://vapi.docs.buildwithfern.com/2025-01-31T05:46:58.861Z/static/images/quickstart/phone/import-twilio-number-dashboard.png) Hit “Import” & you will come to the phone number detail page: ![](https://files.buildwithfern.com/https://vapi.docs.buildwithfern.com/2025-01-31T05:46:58.861Z/static/images/quickstart/phone/phone-number-import-complete.png) The phone number detail page, we can configure our phone number here. Your number is now ready to be attached to an assistant for inbound or outbound phone calling. Attach Your Assistant --------------------- Now that we have a configured assistant & a phone number, we will put our assistant behind the phone number to pick up incoming phone calls. In the `Inbound` area of the phone number detail view, select your assistant in the dropdown under `Assistant`. ![](https://files.buildwithfern.com/https://vapi.docs.buildwithfern.com/2025-01-31T05:46:58.861Z/static/images/quickstart/dashboard/inbound-assistant-set.png) Your assistant will now pick-up calls made to this phone number. Call the Number --------------- You can now make a phone call to the number. Your assistant will pick up the phone & manage the order-taking conversation. Happy ordering! Your assistant won’t yet be able to hang-up the phone at the end of the call. We will learn more about configuring call end behaviour in later guides. [Built with](https://buildwithfern.com/?utm_campaign=buildWith&utm_medium=docs&utm_source=docs.vapi.ai) --- # Outbound Calls from Python 📞 — Vapi ` | | | | --- | --- | | 1 | import requests | | 2 | | | 3 | # Your Vapi API Authorization token | | 4 | auth_token = '' | | 5 | # The Phone Number ID, and the Customer details for the call | | 6 | phone_number_id = '' | | 7 | customer_number = "+14151231234" | | 8 | | | 9 | # Create the header with Authorization token | | 10 | headers = { | | 11 | 'Authorization': f'Bearer {auth_token}', | | 12 | 'Content-Type': 'application/json', | | 13 | } | | 14 | | | 15 | # Create the data payload for the API request | | 16 | data = { | | 17 | 'assistant': { | | 18 | "firstMessage": "Hey, what's up?", | | 19 | "model": { | | 20 | "provider": "openai", | | 21 | "model": "gpt-3.5-turbo", | | 22 | "messages": [ | | 23 | { | | 24 | "role": "system", | | 25 | "content": "You are an assistant." | | 26 | } | | 27 | ] | | 28 | }, | | 29 | "voice": "jennifer-playht" | | 30 | }, | | 31 | 'phoneNumberId': phone_number_id, | | 32 | 'customer': { | | 33 | 'number': customer_number, | | 34 | }, | | 35 | } | | 36 | | | 37 | # Make the POST request to Vapi to create the phone call | | 38 | response = requests.post( | | 39 | 'https://api.vapi.ai/call/phone', headers=headers, json=data) | | 40 | | | 41 | # Check if the request was successful and print the response | | 42 | if response.status_code == 201: | | 43 | print('Call created successfully') | | 44 | print(response.json()) | | 45 | else: | | 46 | print('Failed to create call') | | 47 | print(response.text) | ` [Built with](https://buildwithfern.com/?utm_campaign=buildWith&utm_medium=docs&utm_source=docs.vapi.ai) --- # Dashboard — Vapi ### The Web Dashboard One of the easiest ways to get started with Vapi is by using the web dashboard. You can visit your dashboard by going to [dashboard.vapi.ai](https://dashboard.vapi.ai/) The web dashboard gives you the ability to: * **view, create, & modify [assistants](/assistants) ** associated with your account * **provision & manage phone numbers** assistants can dial outbound from or receive inbound calls to * **review conversation data** (such as audio recordings, call metadata, etc) * **manage your [provider keys](/customization/provider-keys) ** (used in communication with external [TTS](/glossary#tts) , LLM, & [STT](/glossary#stt) vendors) We will be walking through the core necessities you need to get up and running in this guide. The web dashboard wraps over much of the realtime call functionality of Vapi. The dashboard actually uses the [web SDK](/sdk/web) beneath-the-hood to make web calls. Vapi’s Pizzeria --------------- In this guide we will be implementing a simple order-taking assistant at a pizza shop called “Vapi’s Pizzeria”. Vapi’s has 3 types of menu items: `pizza`, `side`s, & `drink`s. Customers will be ordering 1 of each. ![](https://files.buildwithfern.com/https://vapi.docs.buildwithfern.com/2025-01-31T05:46:58.861Z/static/images/quickstart/vapis-pizzeria.png) Customers will order 3 items: 1 pizza, 1 side, & 1 drink. The assistant will handle the full order taking conversation. Assistant Setup --------------- First we’re going to set up our assistant in the dashboard. Once our assistant’s **transcriber**, **model**, & **voice** are set up, we can call it to place our order. You can visit your dashboard at [dashboard.vapi.ai](https://dashboard.vapi.ai/) ###### Sign-up or Log-in to Vapi If you haven’t already signed-up, you’re going to need an account before you can use the web dashboard. When you visit [dashboard.vapi.ai](https://dashboard.vapi.ai/) you may see something like this: ![](https://files.buildwithfern.com/https://vapi.docs.buildwithfern.com/2025-01-31T05:46:58.861Z/static/images/quickstart/dashboard/auth-ui.png) Sign-up for an account (or log-in to your existing account) — you will then find yourself inside the web dashboard. It will look something like this: ![](https://files.buildwithfern.com/https://vapi.docs.buildwithfern.com/2025-01-31T05:46:58.861Z/static/images/quickstart/dashboard/vapi-dashboard-post-signup.png) Your dashboard may look a bit different if you already have an account with assistants in it. The main idea is that we’re in the dashboard now. ###### Create an Assistant Now that you’re in your dashboard, we’re going to create an [assistant](/assistants) . Assistants are at the heart of how Vapi models AI voice agents — we will be setting certain properties on a new assistant to model an order-taking agent. Once in the “Assistants” dashboard tab (you should be in it by-default after log-in), you will see a button to create a new assistant. ![](https://files.buildwithfern.com/https://vapi.docs.buildwithfern.com/2025-01-31T05:46:58.861Z/static/images/quickstart/dashboard/create-new-assistant-button.png) Ensure you are in the 'Assistants' dashboard tab, then this button will allow you to begin the assistant creation flow. After clicking the create new assistant button, you will see a pop-up modal that asks you to pick a starter template. For our example we will start from a blank slate so choose the `Blank Template` option. ![](https://files.buildwithfern.com/https://vapi.docs.buildwithfern.com/2025-01-31T05:46:58.861Z/static/images/quickstart/dashboard/choose-blank-template.png) Ensure you are in the 'Assistants' dashboard tab, then this button will allow you to begin the assistant creation flow. You will then be able to name your assistant — you can name it whatever you’d like (`Vapi’s Pizza Front Desk`, for example): This name is only for internal labeling use. It is not an identifier, nor will the assistant be aware of this name. ![](https://files.buildwithfern.com/https://vapi.docs.buildwithfern.com/2025-01-31T05:46:58.861Z/static/images/quickstart/dashboard/name-your-assistant.png) Name your assistant. Once you have named your assistant, you can hit “Create” to create it. You will then see something like this: ![](https://files.buildwithfern.com/https://vapi.docs.buildwithfern.com/2025-01-31T05:46:58.861Z/static/images/quickstart/dashboard/assistant-created.png) The assistant overview. You can edit your assistant’s transcriber, model, & voice — and edit other advanced configuration. This is the assistant overview view — it gives you the ability to edit different attributes about your assistant, as well as see **cost** & **latency** projection information for each portion of it’s voice pipeline (this is very important data to have handy when building out your assistants). ###### Model Setup Now we’re going to set the “brains” of the assistant, the large language model. We’re going to be using `GPT-4` (from [OpenAI](https://openai.com/) ) for this demo (though you’re free to use `GPT-3.5`, or any one of your favorite LLMs). ###### Set Your OpenAI Provider Key (optional) Before we proceed, we can set our [provider key](https://docs.vapi.ai/customization/provider-keys) for OpenAI (this is just your OpenAI secret key). You can see all of your provider keys in the “Provider Keys” dashboard tab. You can also go directly to [dashboard.vapi.ai/keys](https://dashboard.vapi.ai/keys) . Vapi uses [provider keys](https://docs.vapi.ai/customization/provider-keys) you provide to communicate with LLM, TTS, & STT vendors on your behalf. It is most ideal that we set keys for the vendors we intend to use ahead of time. ![](https://files.buildwithfern.com/https://vapi.docs.buildwithfern.com/2025-01-31T05:46:58.861Z/static/images/quickstart/dashboard/model-provider-keys.png) We set our provider key for OpenAI so Vapi can make requests to their API. While we’re here it’d be ideal for you to go & set up provider keys for other providers you’re familiar with & intend to use later. ###### Set a First Message Assistants can **optionally** be configured with a `First Message`. This first message will be spoken by your assistant when either: * **A Web Call Connects:** when a web call is started with your assistant * **An Inbound Call is Picked-up:** an [inbound call](/glossary#inbound-call) is picked-up & answered by your assistant * **An Outbound Call is Dialed & Picked-up:** an [outbound call](/glossary#outbound-call) is dialed by your assistant & a person picks up Note that this first message cannot be interrupted & is guaranteed to be spoken. Certain use cases need a first message, while others do not. For our use case, we will want a first message. It would be ideal for us to have a first message like this: ` Vappy’s Pizzeria speaking, how can I help you? ` Some text-to-speech voices may struggle to pronounce ‘Vapi’ correctly, compartmentalizing it to be spoken letter by letter “V. A. P. I.” Some aspects of configuring your voice pipeline will require tweaks like this to get the target behaviour you want. This will be spoken by the assistant when a web or inbound phone call is received. ###### Set the System Prompt We will now set the `System Prompt` for our assistant. If you’re familiar with OpenAI’s API, this is the first prompt in the message list that we feed our LLM (learn more about prompt engineering on the [OpenAI docs](https://platform.openai.com/docs/guides/prompt-engineering) ). The system prompt can be used to configure the context, role, personality, instructions and so on for the assistant. In our case, a system prompt like this will give us the behaviour we want: ` | | | --- | | You are a voice assistant for Vappy’s Pizzeria, | | a pizza shop located on the Internet. | | | | Your job is to take the order of customers calling in. The menu has only 3 types | | of items: pizza, sides, and drinks. There are no other types of items on the menu. | | | | 1) There are 3 kinds of pizza: cheese pizza, pepperoni pizza, and vegetarian pizza | | (often called "veggie" pizza). | | 2) There are 3 kinds of sides: french fries, garlic bread, and chicken wings. | | 3) There are 2 kinds of drinks: soda, and water. (if a customer asks for a | | brand name like "coca cola", just let them know that we only offer "soda") | | | | Customers can only order 1 of each item. If a customer tries to order more | | than 1 item within each category, politely inform them that only 1 item per | | category may be ordered. | | | | Customers must order 1 item from at least 1 category to have a complete order. | | They can order just a pizza, or just a side, or just a drink. | | | | Be sure to introduce the menu items, don't assume that the caller knows what | | is on the menu (most appropriate at the start of the conversation). | | | | If the customer goes off-topic or off-track and talks about anything but the | | process of ordering, politely steer the conversation back to collecting their order. | | | | Once you have all the information you need pertaining to their order, you can | | end the conversation. You can say something like "Awesome, we'll have that ready | | for you in 10-20 minutes." to naturally let the customer know the order has been | | fully communicated. | | | | It is important that you collect the order in an efficient manner (succinct replies | | & direct questions). You only have 1 task here, and it is to collect the customers | | order, then end the conversation. | | | | - Be sure to be kind of funny and witty! | | - Keep all your responses short and simple. Use casual language, phrases like "Umm...", "Well...", and "I mean" are preferred. | | - This is a voice conversation, so keep your responses short, like in a real conversation. Don't ramble for too long. | ` You can copy & paste the above prompt into the `System Prompt` field. Now the model configuration for your assistant should look something like this: ![](https://files.buildwithfern.com/https://vapi.docs.buildwithfern.com/2025-01-31T05:46:58.861Z/static/images/quickstart/dashboard/assistant-model-set-up.png) Note how our model provider is set to OpenAI & the model is set to GPT-4. ###### Transcriber Setup The transcriber is what turns user speech into processable text for our LLM. This is the first step in the end-to-end voice pipeline. ###### Set Your Deepgram Provider Key (optional) We will be using [Deepgram](https://deepgram.com/) (which provides blazing-fast & accurate Speech-to-Text) as our STT provider. We will set our provider key for them in “Provider Keys”: ![](https://files.buildwithfern.com/https://vapi.docs.buildwithfern.com/2025-01-31T05:46:58.861Z/static/images/quickstart/dashboard/transcriber-providers-keys.png) ###### Set Transcriber We will set the model to `Nova 2` & the language to `en` for English. Now your assistant’s transcriber configuration should look something like this: ![](https://files.buildwithfern.com/https://vapi.docs.buildwithfern.com/2025-01-31T05:46:58.861Z/static/images/quickstart/dashboard/assistant-transcriber-config.png) Note how our transcriber is set to 'deepgram', the model is set to 'Nova 2', & the language is set to English. ###### Voice Setup The final portion of the voice pipeline is turning LLM output-text into speech. This process is called “Text-to-speech” (or TTS for short). We will be using a voice provider called [PlayHT](https://play.ht/) (they have very conversational voices), & a voice provided by them labeled `Jennifer` (`female`, `en-US`). You are free to use your favorite TTS voice platform here. [ElevenLabs](https://elevenlabs.io/) is another alternative — by now you should get the flow of plugging in vendors into Vapi (add provider key + pick provider in assistant config). You can skip the next step(s) if you don’t intend to use PlayHT. ###### Set Your PlayHT Provider Key (optional) If you haven’t already, sign up for an account with PlayHT at [play.ht](https://play.ht/) . Since their flows are liable to change — you can just grab your `API Key` & `User ID` from them. ![](https://files.buildwithfern.com/https://vapi.docs.buildwithfern.com/2025-01-31T05:46:58.861Z/static/images/quickstart/dashboard/voice-provider-keys.png) ###### Set Voice You will want to select `playht` in the “provider” field, & `Jennifer` in the “voice” field. We will leave all of the other settings untouched. ![](https://files.buildwithfern.com/https://vapi.docs.buildwithfern.com/2025-01-31T05:46:58.861Z/static/images/quickstart/dashboard/assistant-voice-config.png) Each voice provider offers a host of settings you can modulate to customize voices. Here we will leave all the defaults alone. Calling Your Assistant ---------------------- Now that your assistant is fully setup & configured, we will want to contact it. There are 2 ways to “call in” to an assistant: * **Over the Internet:** Network-enabled devices can contact Vapi via the Internet (i.e. web applications, mobile applications). No phone number is involved. * **Via Telephony:** Phones can communicate to Vapi over a cellular network (i.e. phone call). One phone number dials to another phone number. For our use case, it is most appropriate that customers will contact our assistant via an inbound phone call. Though, we will look at both ways of calling in. ###### Call in the Dashboard The quickest way to contact your new assistant is by simply using the call button on the assistant detail page: ![](https://files.buildwithfern.com/https://vapi.docs.buildwithfern.com/2025-01-31T05:46:58.861Z/static/images/quickstart/dashboard/call-assistant-web-dashboard.png) Call into your assistant via the dashboard. The dashboard uses the [web SDK](/sdk/web) underneath to make web calls. This will start a web call with your assistant, you can now speak to it to order your pizza & sides! ###### Call via Phone Since our assistant is meant to take orders over the phone, we will want to set up [inbound calling](/phone-calling) to our assistant. We will need to do 2 things: 1. **provision a new phone number** to sit our agent behind (it will pick-up calls that come in — hence “inbound calling”) 2. **place our agent behind that phone number** If you already have your own phone numbers (purchased via Twilio or Vonage, etc), you can import them into Vapi for use. Learn more about [telephony](/phone-calling) on Vapi. ###### Provision a Phone Number The quickest way to secure a phone number for your assistant is to purchase a phone number directly through Vapi. Ensure you have a card on file that Vapi can bill before proceeding, you can add your billing information in your dashboard at [dashboard.vapi.ai/billing](https://dashboard.vapi.ai/billing) Navigate to the “Phone Numbers” section & click the “Buy number” button: ![](https://files.buildwithfern.com/https://vapi.docs.buildwithfern.com/2025-01-31T05:46:58.861Z/static/images/quickstart/dashboard/buy-a-phone-number.png) Make sure you are in the 'Phone Numbers' dashboard tab. We will use the area code `415` for our phone number (these are area codes domestic to the US & Canada). ![](https://files.buildwithfern.com/https://vapi.docs.buildwithfern.com/2025-01-31T05:46:58.861Z/static/images/quickstart/dashboard/buy-phone-number-modal.png) Choose an area code for your phone number. Currently, only US & Canada phone numbers can be directly purchased through Vapi. Phone numbers in other regions must be imported, see our [phone calling](/phone-calling) guide. Click “Buy”, after purchasing a phone number you should see something like this: ![](https://files.buildwithfern.com/https://vapi.docs.buildwithfern.com/2025-01-31T05:46:58.861Z/static/images/quickstart/dashboard/phone-number-config.png) Here we can attach an assistant to the number for inbound calls (or perform an outbound call, with a select assistant). The phone number is now ready to be used (either for inbound or outbound calling). ###### Attach Your Assistant In the `Inbound` area of the phone number detail view, select your assistant in the dropdown under `Assistant`. ![](https://files.buildwithfern.com/https://vapi.docs.buildwithfern.com/2025-01-31T05:46:58.861Z/static/images/quickstart/dashboard/inbound-assistant-set.png) Your assistant will now pick-up calls made to this phone number. This will put your assistant behind the phone number for inbound calls. Your assistant is now ready to take calls. Your assistant should be able to accept calls & maintain a basic conversation. Happy ordering! Your assistant won’t yet be able to hang-up the phone at the end of the call. We will learn more about configuring call end behaviour in later guides. [Built with](https://buildwithfern.com/?utm_campaign=buildWith&utm_medium=docs&utm_source=docs.vapi.ai) --- # Provider Keys — Vapi Have a custom model or voice with one of the providers? Or an enterprise account with volume pricing? No problem! You can bring your own API keys to Vapi. You can add them in the [Dashboard](https://dashboard.vapi.ai/) under the **Provider Keys** tab. Once your API key is validated, you won’t be charged when using that provider through Vapi. Instead, you’ll be charged directly by the provider. Transcription Providers ----------------------- Currently, the only available transcription provider is `deepgram`. To use a custom model, you can specify the deepgram model ID in the `transcriber.model` parameter of the [Assistant](/api-reference/assistants/create-assistant) . Model Providers --------------- We are currently have support for any OpenAI-compatible endpoint. This includes services like [OpenRouter](https://openrouter.ai/) , [AnyScale](https://www.anyscale.com/) , [Together AI](https://www.together.ai/) , or your own server. To use one of these providers, you can specify the `provider` and `model` in the `model` parameter of the [Assistant](/api-reference/assistants/create-assistant) . You can find more details in the [Custom LLMs](/customization/custom-llm/fine-tuned-openai-models) section of the documentation. Voice Providers --------------- All voice providers are supported. Once you’ve validated your API through the [Dashboard](https://dashboard.vapi.ai/) , any voice ID from your provider can be used in the `voice.voiceId` field of the [Assistant](/api-reference/assistants/create-assistant) . Cloud Providers --------------- Vapi stores recordings of conversations with assistants in the cloud. By default, Vapi stores these recordings in its own bucket in Cloudflare R2. You can configure Vapi to store recordings in your own bucket in AWS S3, GCP, or Cloudflare R2. You can find more details on how to configure your Cloud Provider keys here: * [AWS S3](/providers/cloud/s3) * [GCP Cloud Storage](/providers/cloud/gcp) * [Cloudflare R2](/providers/cloud/cloudflare) [Built with](https://buildwithfern.com/?utm_campaign=buildWith&utm_medium=docs&utm_source=docs.vapi.ai) --- # Billing FAQ — Vapi ### Overview Vapi uses a credit-based billing system where credits serve as the platform’s currency. Every billable action, including making calls and purchasing add-ons, requires credits. One credit equals one US dollar ($1 = 1 credit). Each new trial account receives 10 complimentary credits in their wallet to help you get started. These credits can be used for making calls, but not purchasing add-ons. To continue using Vapi after your trial credits are depleted, you’ll need to: 1. Add your payment details 2. Purchase additional credits 3. Your wallet will then be upgraded to pay-as-you-go status. Your wallet will be marked as frozen while your credit balance is negative. Frozen wallets are unable to make calls or purchase add-ons. ### What is auto-reload? Auto-reload is a billing mechanism in which Vapi will automatically top up your credits when it hits a certain threshold. It can be enabled through the [billing page](https://dashboard.vapi.ai/org/billing) and we highly recommend enabling it to prevent billing-related operational issues. Auto-reload amounts must be at least $10. We recommend setting a threshold above $0. ![](https://files.buildwithfern.com/https://vapi.docs.buildwithfern.com/2025-01-31T05:46:58.861Z/static/images/quickstart/billing/auto-reload.png) Enabling auto-reload through the Vapi dashboard. ### What are add-ons? Add-ons are platform enhancements that you may purchase. They are purchased through the [billing add-ons page](https://dashboard.vapi.ai/org/billing/add-ons) . The following add-ons are currently available: | Add-on | Price (credits / month) | Description | | --- | --- | --- | | Phone Numbers | 2 | Twilio phone numbers for your calls | | Reserved Concurrency Lines | 10 | Guaranteed capacity for concurrent calls | | HIPAA Compliance | 1,000 | HIPAA Compliance and a BAA agreement | | Slack Support | 5,000 | Priority support via dedicated Slack channel | Here’s how the billing works: * You’ll be charged a prorated amount for the remainder of the current billing cycle when you first purchase an add-on * Subsequently, you’ll be billed the full amount at the start of each billing cycle (the 1st of each month) * If you cancel an add-on, you’ll receive a prorated refund for the unused portion of the billing cycle. * Add-ons can be cancelled at any time through the billing page If your wallet is frozen at the time of billing, all add-ons will be automatically cancelled. You’ll need to repurchase them once your wallet is reactivated. ### How do I download invoices for my credit purchases? You may download invoices within the [billing page](https://dashboard.vapi.ai/billing) . Find the relevant credit purchase within the payment history table, and click on **Download Invoice**. You may then optionally choose to fill in extra details which will be reflected within the Invoice. ![](https://files.buildwithfern.com/https://vapi.docs.buildwithfern.com/2025-01-31T05:46:58.861Z/static/images/quickstart/billing/download-invoice.png) Downloading an invoice. ![](https://files.buildwithfern.com/https://vapi.docs.buildwithfern.com/2025-01-31T05:46:58.861Z/static/images/quickstart/billing/invoice-detail-form.png) Optional invoice details. ![](https://files.buildwithfern.com/https://vapi.docs.buildwithfern.com/2025-01-31T05:46:58.861Z/static/images/quickstart/billing/sample-invoice.png) Sample invoice. [Built with](https://buildwithfern.com/?utm_campaign=buildWith&utm_medium=docs&utm_source=docs.vapi.ai) --- # Pizza Website Example 🍕 — Vapi In this example, we’ll be using the [Web SDK](https://github.com/VapiAI/web) to create an assistant that can take a pizza order. Since all the [Client SDKs](/sdks) have equivalent functionality, you can use this example as a guide for any Vapi client. We want to add a button to the page to start a call, update our UI with the call status, and display what the user’s saying while they say it. When the user mentions a topping, we should add it to the pizza. When they’re done, we should redirect them to checkout. [1](/examples/pizza-website#create-an-assistant) ### Create an assistant We’ll start by taking a look at the [Assistant API reference](/api-reference/assistants/create-assistant) and define our assistant: ` | | | | --- | --- | | 1 | { | | 2 | "model": { | | 3 | "provider": "openai", | | 4 | "model": "gpt-4", | | 5 | "messages": [ | | 6 | { | | 7 | "role": "system", | | 8 | "content": "You're a pizza ordering assistant. The user will ask for toppings, you'll add them. When they're done, you'll redirect them to checkout." | | 9 | } | | 10 | ], | | 11 | "functions": [ | | 12 | { | | 13 | "name": "addTopping", | | 14 | "description": "Used to add a topping to the pizza.", | | 15 | "parameters": { | | 16 | "type": "object", | | 17 | "properties": { | | 18 | "topping": { | | 19 | "type": "string", | | 20 | "description": "The name of the topping. For example, 'pepperoni'." | | 21 | } | | 22 | } | | 23 | } | | 24 | }, | | 25 | { | | 26 | "name": "goToCheckout", | | 27 | "description": "Redirects the user to checkout and order their pizza.", | | 28 | "parameters": {"type": "object", "properties": {}} | | 29 | } | | 30 | ] | | 31 | }, | | 32 | "firstMessage": "Hi, I'm the pizza ordering assistant. What toppings would you like?", | | 33 | } | ` Let’s break this down: * `model` - We’re using the OpenAI GPT-4 model, which is better at function calling. * `messages` - We’re defining the assistant’s instructions for how to run the call. * `functions` - We’re providing a addTopping function with a topping parameter. The assistant can call this during the conversation to add a topping. We’re also adding goToCheckout, with an empty parameters object. The assistant can call this to redirect the user to checkout. * `firstMessage` - This is the first message the assistant will say when the user starts the call. We’ll then make a POST request to the [Create Assistant](/api-reference/assistants/create-assistant) endpoint to create the assistant. [2](/examples/pizza-website#set-up-the-web-sdk) ### Set up the Web SDK We’ll follow the `README` for the [Web SDK](https://github.com/VapiAI/web) to get it installed. We’ll then get our **Public Key** from the [Vapi Dashboard](https://dashboard.vapi.ai/) and initialize the SDK: ` | | | | --- | --- | | 1 | import Vapi from '@vapi-ai/web'; | | 2 | | | 3 | const vapi = new Vapi('your-web-token'); | ` [3](/examples/pizza-website#add-the-call-buttons) ### Add the call buttons We’ll add a button to the page that starts the call when clicked: ` | | | | --- | --- | | 1 | | | 2 | | ` ` | | | | --- | --- | | 1 | const startCallButton = document.getElementById('start-call'); | | 2 | | | 3 | startCallButton.addEventListener('click', async () => { | | 4 | await vapi.start('your-assistant-id'); | | 5 | }); | | 6 | | | 7 | const stopCallButton = document.getElementById('stop-call'); | | 8 | | | 9 | stopCallButton.addEventListener('click', async () => { | | 10 | await vapi.stop(); | | 11 | }); | ` [4](/examples/pizza-website#handle-call-status-events) ### Handle call status events ` | | | | --- | --- | | 1 | vapi.on('call-start', () => { | | 2 | // Update UI to show that the call has started | | 3 | }); | | 4 | | | 5 | vapi.on('call-end', () => { | | 6 | // Update UI to show that the call has ended | | 7 | }); | ` [5](/examples/pizza-website#handle-speaking-events) ### Handle speaking events ` | | | | --- | --- | | 1 | vapi.on('speech-start', () => { | | 2 | // Update UI to show that the assistant is speaking | | 3 | }); | | 4 | | | 5 | vapi.on('speech-end', () => { | | 6 | // Update UI to show that the assistant is done speaking | | 7 | }); | ` [6](/examples/pizza-website#handle-transcription-events) ### Handle transcription events All messages send to the [Server URL](/server-url) , including `transcript` and `function-call` messages, are also sent to the client as `message` events. We’ll need to check the `type` of the message to see what type it is. ` | | | | --- | --- | | 1 | vapi.on("message", (msg) => { | | 2 | if (msg.type !== "transcript") return; | | 3 | | | 4 | if (msg.transcriptType === "partial") { | | 5 | // Update UI to show the live partial transcript | | 6 | } | | 7 | | | 8 | if (msg.transcriptType === "final") { | | 9 | // Update UI to show the final transcript | | 10 | } | | 11 | }); | ` [7](/examples/pizza-website#handle-function-call-events) ### Handle function call events ` | | | | --- | --- | | 1 | vapi.on('message', (msg) => { | | 2 | if (msg.type !== "function-call") return; | | 3 | | | 4 | if (msg.functionCall.name === "addTopping") { | | 5 | const topping = msg.functionCall.parameters.topping; | | 6 | // Add the topping to the pizza | | 7 | } | | 8 | | | 9 | if (msg.functionCall.name === "goToCheckout") { | | 10 | // Redirect the user to checkout | | 11 | } | | 12 | }); | ` [8](/examples/pizza-website#order-your-pizza) ### Order your pizza! You should now have a working pizza ordering assistant! 🍕 [Built with](https://buildwithfern.com/?utm_campaign=buildWith&utm_medium=docs&utm_source=docs.vapi.ai) --- # Outbound Sales Example 📞 — Vapi We want this agent to be able to call a list of leads and schedule appointments. We’ll create our assistant, create a phone number for it, then we’ll configure our server for function calling to book the appointments. [1](/examples/outbound-sales#create-an-assistant) ### Create an assistant We’ll start by taking a look at the [Assistant API reference](/api-reference/assistants/create-assistant) and define our assistant: ` | | | | --- | --- | | 1 | { | | 2 | "transcriber":{ | | 3 | "provider": "deepgram", | | 4 | "keywords": ["Bicky:1"] | | 5 | }, | | 6 | "model": { | | 7 | "provider": "openai", | | 8 | "model": "gpt-4", | | 9 | "messages": [ | | 10 | { | | 11 | "role": "system", | | 12 | "content": "You're a sales agent for a Bicky Realty. You're calling a list of leads to schedule appointments to show them houses..." | | 13 | } | | 14 | ], | | 15 | "functions": [ | | 16 | { | | 17 | "name": "bookAppointment", | | 18 | "description": "Used to book the appointment.", | | 19 | "parameters": { | | 20 | "type": "object", | | 21 | "properties": { | | 22 | "datetime": { | | 23 | "type": "string", | | 24 | "description": "The date and time of the appointment in ISO format." | | 25 | } | | 26 | } | | 27 | } | | 28 | } | | 29 | ] | | 30 | }, | | 31 | "voice": { | | 32 | "provider": "openai", | | 33 | "voiceId": "onyx" | | 34 | }, | | 35 | "forwardingPhoneNumber": "+16054440129", | | 36 | "voicemailMessage": "Hi, this is Jennifer from Bicky Realty. We were just calling to let you know...", | | 37 | "firstMessage": "Hi, this Jennifer from Bicky Realty. We're calling to schedule an appointment to show you a house. When would be a good time for you?", | | 38 | "endCallMessage": "Thanks for your time.", | | 39 | "endCallFunctionEnabled": true, | | 40 | "recordingEnabled": false, | | 41 | } | ` Let’s break this down: * `transcriber` - We’re defining this to make sure the transcriber picks up the custom word “Bicky” * `model` - We’re using the OpenAI GPT-4 model, which is better at function calling. * `messages` - We’re defining the assistant’s instructions for how to run the call. * `functions` - We’re providing a bookAppointment function with a datetime parameter. The assistant can call this during the conversation to book the appointment. * `voice` - We’re using the Onyx voice from OpenAI. * `forwardingPhoneNumber` - Since we’ve added this, the assistant will be provided the [transferCall](/assistants#transfer-call) function to use. * `voicemailMessage` - If the call goes to voicemail, this message will be played. * `firstMessage` - This is the first message the assistant will say when the user picks up. * `endCallMessage` - This is the message the assistant will deciding to hang up. * `endCallFunctionEnabled` - This will give the assistant the [endCall](/assistants#end-call) function. * `recordingEnabled` - We’ve disabled recording, since we don’t have the user’s consent to record the call. We’ll then make a POST request to the [Create Assistant](/api-reference/assistants/create-assistant) endpoint to create the assistant. [2](/examples/outbound-sales#buy-a-phone-number) ### Buy a phone number We’ll buy a phone number for outbound calls using the [Phone Numbers API](/phone-calling#set-up-a-phone-number) . ` | | | | --- | --- | | 1 | { | | 2 | "id": "c86b5177-5cd8-447f-9013-99e307a8a7bb", | | 3 | "orgId": "aa4c36ba-db21-4ce0-9c6e-99e307a8a7bb", | | 4 | "number": "+11234567890", | | 5 | "createdAt": "2023-09-29T21:44:37.946Z", | | 6 | "updatedAt": "2023-12-08T00:57:24.706Z", | | 7 | } | ` Great, let’s take note of that `id` field- we’ll need it later. [3](/examples/outbound-sales#configure-your-server-url) ### Configure your Server URL When the assistant calls that `bookAppointment` function, we’ll want to handle that function call and actually book the appointment. We also want to let the user know if booking the appointment was unsuccessful. First, we’ll create an endpoint on our server for Vapi to hit. It’ll receive messages as shown in the [Function Calling](/server-url#function-calling) docs. Once created, we’ll add that endpoint URL to the **Server URL** field in the Account page on the [Vapi Dashboard](https://dashboard.vapi.ai/) . [4](/examples/outbound-sales#handle-function-calls) ### Handle function calls So now, when the assistant decides to call `bookAppointment`, our server will get something like this: ` | | | | --- | --- | | 1 | { | | 2 | "message": { | | 3 | "type": "function-call", | | 4 | "call": { Call Object }, | | 5 | "functionCall": { | | 6 | "name": "bookAppointment", | | 7 | "parameters": "{ \"datetime\": \"2023-09-29T21:44:37.946Z\"}" | | 8 | } | | 9 | } | | 10 | } | ` We’ll do our own logic to book the appointment, then we’ll respond to the request with the result to let the assistant know it was booked: ` | | | | --- | --- | | 1 | { "result": "The appointment was booked successfully." } | ` or, if it failed: ` | | | | --- | --- | | 1 | { "result": "The appointment time is unavailable, please try another time." } | ` So, when the assistant calls this function, these results will be appended to the conversation, and the assistant will respond to the user knowing the result. Great, now we’re ready to start calling leads! [5](/examples/outbound-sales#place-a-call) ### Place a call We’ll use the [Create Phone Call](/api-reference/calls/create-phone-call) endpoint to place a call to a lead: ` | | | | --- | --- | | 1 | { | | 2 | "phoneNumberId": "c86b5177-5cd8-447f-9013-99e307a8a7bb", | | 3 | "assistantId": "d87b5177-5cd8-447f-9013-99e307a8a7bb", | | 4 | "customer": { | | 5 | "number": "+11234567890" | | 6 | } | | 7 | } | ` Since we also defined a `forwardingPhoneNumber`, when the user asks to speak to a human, the assistant will transfer the call to that number automatically. We can then check the [Dashboard](https://dashboard.vapi.ai/) to see the call logs and read the transcripts. [Built with](https://buildwithfern.com/?utm_campaign=buildWith&utm_medium=docs&utm_source=docs.vapi.ai) --- # Inbound Support Example ⚙️ — Vapi We want a phone number we can call to get technical support. We want the assistant to use a provided set of troubleshooting guides to help walk the caller through solving their issue. As a bonus, we also want the assistant to remember by the phone number of the caller where we left off if we get disconnected. [1](/examples/inbound-support#create-an-assistant) ### Create an assistant We’ll start by taking a look at the [Assistant API reference](/api-reference/assistants/create-assistant) and define our assistant: ` | | | | --- | --- | | 1 | { | | 2 | "transcriber":{ | | 3 | "provider": "deepgram", | | 4 | "keywords": ["iPhone:1", "MacBook:1.5", "iPad:1", "iMac:0.8", "Watch:1", "TV:1", "Apple:2"], | | 5 | }, | | 6 | "model": { | | 7 | "provider": "openai", | | 8 | "model": "gpt-4", | | 9 | "messages": [ | | 10 | { | | 11 | "role": "system", | | 12 | "content": "You're a technical support assistant. You're helping a customer troubleshoot their Apple device. You can ask the customer questions, and you can use the following troubleshooting guides to help the customer solve their issue: ..." | | 13 | } | | 14 | ] | | 15 | }, | | 16 | "forwardingPhoneNumber": "+16054440129", | | 17 | "firstMessage": "Hey, I'm an A.I. assistant for Apple. I can help you troubleshoot your Apple device. What's the issue?", | | 18 | "recordingEnabled": true, | | 19 | } | ` Let's break this down * `transcriber` - We’re defining this to make sure the transcriber picks up the custom words related to our devices. * `model` - We’re using the OpenAI GPT-3.5-turbo model. It’s much faster and preferred if we don’t need GPT-4. * `messages` - We’re defining the assistant’s instructions for how to run the call. * `forwardingPhoneNumber` - Since we’ve added this, the assistant will be provided the [transferCall](/assistants#transfer-call) function to use if the caller asks to be transferred to a person. * `firstMessage` - This is the first message the assistant will say when the user picks up. * `recordingEnabled` - We’re recording the call so we can hear the conversation later. Since we want the assistant to remember where we left off, its configuration is going to change based on the caller. So, we’re not going to use [temporary assistants](/assistants/persistent-assistants) . For this example, we’re going to store the conversation on our server between calls and use the [Server URL’s `assistant-request`](/server-url#retrieving-assistants) to fetch a new configuration based on the caller every time someone calls. [2](/examples/inbound-support#buy-a-phone-number) ### Buy a phone number We’ll buy a phone number for inbound calls using the [Phone Numbers API](/api-reference/phone-numbers/buy-phone-number) . ` | | | | --- | --- | | 1 | { | | 2 | "id": "c86b5177-5cd8-447f-9013-99e307a8a7bb", | | 3 | "orgId": "aa4c36ba-db21-4ce0-9c6e-99e307a8a7bb", | | 4 | "number": "+11234567890", | | 5 | "createdAt": "2023-09-29T21:44:37.946Z", | | 6 | "updatedAt": "2023-12-08T00:57:24.706Z", | | 7 | } | ` [3](/examples/inbound-support#configure-your-server-url) ### Configure your Server URL When someone calls our number, we want to fetch the assistant configuration from our server. We’ll use the [Server URL’s `assistant-request`](/server-url#retrieving-assistants) to do this. First, we’ll create an endpoint on our server for Vapi to hit. It’ll receive messages as shown in the [Assistant Request](/server-url#retrieving-assistants-calling) docs. Once created, we’ll add that endpoint URL to the **Server URL** field in the Account page on the [Vapi Dashboard](https://dashboard.vapi.ai/) . [4](/examples/inbound-support#save-the-conversation-at-the-end-of-the-call) ### Save the conversation at the end of the call We’ll want to save the conversation at the end of the call for the next time they call. We’ll use the [Server URL’s `end-of-call-report`](/server-url#end-of-call-report) message to do this. At the end of each call, we’ll get a message like this: ` | | | | --- | --- | | 1 | { | | 2 | "message": { | | 3 | "type": "end-of-call-report", | | 4 | "endedReason": "hangup", | | 5 | "call": { Call Object }, | | 6 | "recordingUrl": "https://vapi-public.s3.amazonaws.com/recordings/1234.wav", | | 7 | "summary": "The user mentioned they were having an issue with their iPhone restarting randomly. They restarted their phone, but the issue persisted. They mentioned they were using an iPhone 12 Pro Max. They mentioned they were using iOS 15.", | | 8 | "transcript": "Hey, I'm an A.I. assistant for Apple...", | | 9 | "messages":[ | | 10 | { | | 11 | "role": "assistant", | | 12 | "message": "Hey, I'm an A.I. assistant for Apple. I can help you troubleshoot your Apple device. What's the issue?", | | 13 | }, | | 14 | { | | 15 | "role": "user", | | 16 | "message": "Yeah I'm having an issue with my iPhone restarting randomly.", | | 17 | }, | | 18 | ... | | 19 | ] | | 20 | } | | 21 | } | ` We’ll save the `call.customer.number` and `summary` fields to our database for the next time they call. [5](/examples/inbound-support#handle-assistant-requests) ### Handle assistant requests. When our number receives a call, Vapi will also hit our server’s endpoint with a message like this: ` | | | | --- | --- | | 1 | { | | 2 | "message": { | | 3 | "type": "assistant-request", | | 4 | "call": { Call Object }, | | 5 | } | | 6 | } | ` We’ll check our database to see if we have a conversation for this caller. If we do, we’ll create an assistant configuration like in Step 1 and respond with it: ` | | | | --- | --- | | 1 | { | | 2 | "assistant": { | | 3 | ... | | 4 | "model": { | | 5 | "provider": "openai", | | 6 | "model": "gpt-4", | | 7 | "messages": [ | | 8 | { | | 9 | "role": "system", | | 10 | "content": "You're a technical support assistant. Here's where we left off: ..." | | 11 | } | | 12 | ] | | 13 | }, | | 14 | ... | | 15 | } | | 16 | } | ` If we don’t, we’ll just respond with the assistant configuration from Step 1. [6](/examples/inbound-support#try-calling-it) ### Try calling it! We’ll call our number and see if it works. Give it a call, and tell it you’re having an issue with your iPhone restarting randomly. Hang up, and call back. Then ask what the issue was. The assistant should remember where we left off. [Built with](https://buildwithfern.com/?utm_campaign=buildWith&utm_medium=docs&utm_source=docs.vapi.ai) --- # Web Calling — Vapi Anywhere you can run client-side JavaScript, you can run Vapi. All the way from vanilla to complex component-based applications with React and Next.js. [The Web SDK\ \ Explore the full Vapi Web SDK.](/sdk/web) [Live React Demo\ \ Follow along as you read.](https://stackblitz.com/~/github.com/VapiAI/quickstart-react) Installation ------------ Install the package: ` | | | | --- | --- | | $ | yarn add @vapi-ai/web | ` or w/ npm: ` | | | | --- | --- | | $ | npm install @vapi-ai/web | ` Import the package: ` | | | | --- | --- | | 1 | import Vapi from "@vapi-ai/web"; | ` Then, create a new instance of the Vapi class, passing your **Public Key** as a parameter to the constructor: ` | | | | --- | --- | | 1 | const vapi = new Vapi("your-public-key"); | ` You can find your public key in the [Vapi Dashboard](https://dashboard.vapi.ai/account) . Starting a Call --------------- Assistants can either be created on the fly (temporary) or created & persisted to your account (persistent). ### Option 1: Temporary Assistant If you want to customize properties from the frontend on the fly, you can create an assistant configuration object and pass it to the `.start()` method. ###### Assistant Configuration Here are the options we will pass to `.start()`: `` | | | | --- | --- | | 1 | const assistantOptions = { | | 2 | name: "Vapi’s Pizza Front Desk", | | 3 | firstMessage: "Vappy’s Pizzeria speaking, how can I help you?", | | 4 | transcriber: { | | 5 | provider: "deepgram", | | 6 | model: "nova-2", | | 7 | language: "en-US", | | 8 | }, | | 9 | voice: { | | 10 | provider: "playht", | | 11 | voiceId: "jennifer", | | 12 | }, | | 13 | model: { | | 14 | provider: "openai", | | 15 | model: "gpt-4", | | 16 | messages: [ | | 17 | { | | 18 | role: "system", | | 19 | content: `You are a voice assistant for Vappy’s Pizzeria, a pizza shop located on the Internet. | | 20 | | | 21 | Your job is to take the order of customers calling in. The menu has only 3 types | | 22 | of items: pizza, sides, and drinks. There are no other types of items on the menu. | | 23 | | | 24 | 1) There are 3 kinds of pizza: cheese pizza, pepperoni pizza, and vegetarian pizza | | 25 | (often called "veggie" pizza). | | 26 | 2) There are 3 kinds of sides: french fries, garlic bread, and chicken wings. | | 27 | 3) There are 2 kinds of drinks: soda, and water. (if a customer asks for a | | 28 | brand name like "coca cola", just let them know that we only offer "soda") | | 29 | | | 30 | Customers can only order 1 of each item. If a customer tries to order more | | 31 | than 1 item within each category, politely inform them that only 1 item per | | 32 | category may be ordered. | | 33 | | | 34 | Customers must order 1 item from at least 1 category to have a complete order. | | 35 | They can order just a pizza, or just a side, or just a drink. | | 36 | | | 37 | Be sure to introduce the menu items, don't assume that the caller knows what | | 38 | is on the menu (most appropriate at the start of the conversation). | | 39 | | | 40 | If the customer goes off-topic or off-track and talks about anything but the | | 41 | process of ordering, politely steer the conversation back to collecting their order. | | 42 | | | 43 | Once you have all the information you need pertaining to their order, you can | | 44 | end the conversation. You can say something like "Awesome, we'll have that ready | | 45 | for you in 10-20 minutes." to naturally let the customer know the order has been | | 46 | fully communicated. | | 47 | | | 48 | It is important that you collect the order in an efficient manner (succinct replies | | 49 | & direct questions). You only have 1 task here, and it is to collect the customers | | 50 | order, then end the conversation. | | 51 | | | 52 | - Be sure to be kind of funny and witty! | | 53 | - Keep all your responses short and simple. Use casual language, phrases like "Umm...", "Well...", and "I mean" are preferred. | | 54 | - This is a voice conversation, so keep your responses short, like in a real conversation. Don't ramble for too long.`, | | 55 | }, | | 56 | ], | | 57 | }, | | 58 | }; | `` Let’s break down the configuration options we passed: * **name:** the display name for the assistant in our dashboard (for internal purposes only) * **firstMessage:** the first message that our assistant will say when it picks up the web call * **transcriber:** the transcriber is what turns user speech into processable text for our LLM. This is the first step in the end-to-end voice pipeline. We are using Deepgram for transcription, specifically, their `Nova 2` model. We also set the language to be transcribed as English. * **voice:** the final portion of the voice pipeline is turning LLM output-text into speech. This process is called “Text-to-speech” (or TTS for short). We use a voice provider called PlayHT, & a voice provided by them called `jennifer`. * **model:** for our LLM, we use `gpt-4` (from OpenAI) & set our system prompt for the assistant. The system prompt configures the context, role, personality, instructions and so on for the assistant. In our case, the system prompt above will give us the behaviour we want. Now we can call `.start()`, passing the temporary assistant configuration: ` | | | | --- | --- | | 1 | vapi.start(assistantOptions); | ` More configuration options can be found in the [Assistant](/api-reference/assistants/create-assistant) API reference. ### Option 2: Persistent Assistant If you want to create an assistant that you can reuse across multiple calls, you can create a persistent assistant in the [Vapi Dashboard](https://dashboard.vapi.ai/) . Here’s how you can do that: ###### Sign-up or Log-in to Vapi If you haven’t already signed-up, you’re going to need an account before you can use the web dashboard. When you visit [dashboard.vapi.ai](https://dashboard.vapi.ai/) you may see something like this: ![](https://files.buildwithfern.com/https://vapi.docs.buildwithfern.com/2025-01-31T05:46:58.861Z/static/images/quickstart/dashboard/auth-ui.png) Sign-up for an account (or log-in to your existing account) — you will then find yourself inside the web dashboard. It will look something like this: ![](https://files.buildwithfern.com/https://vapi.docs.buildwithfern.com/2025-01-31T05:46:58.861Z/static/images/quickstart/dashboard/vapi-dashboard-post-signup.png) Your dashboard may look a bit different if you already have an account with assistants in it. The main idea is that we’re in the dashboard now. ###### Create an Assistant Now that you’re in your dashboard, we’re going to create an [assistant](/assistants) . Assistants are at the heart of how Vapi models AI voice agents — we will be setting certain properties on a new assistant to model an order-taking agent. Once in the “Assistants” dashboard tab (you should be in it by-default after log-in), you will see a button to create a new assistant. ![](https://files.buildwithfern.com/https://vapi.docs.buildwithfern.com/2025-01-31T05:46:58.861Z/static/images/quickstart/dashboard/create-new-assistant-button.png) Ensure you are in the 'Assistants' dashboard tab, then this button will allow you to begin the assistant creation flow. After clicking the create new assistant button, you will see a pop-up modal that asks you to pick a starter template. For our example we will start from a blank slate so choose the `Blank Template` option. ![](https://files.buildwithfern.com/https://vapi.docs.buildwithfern.com/2025-01-31T05:46:58.861Z/static/images/quickstart/dashboard/choose-blank-template.png) Ensure you are in the 'Assistants' dashboard tab, then this button will allow you to begin the assistant creation flow. You will then be able to name your assistant — you can name it whatever you’d like (`Vapi’s Pizza Front Desk`, for example): This name is only for internal labeling use. It is not an identifier, nor will the assistant be aware of this name. ![](https://files.buildwithfern.com/https://vapi.docs.buildwithfern.com/2025-01-31T05:46:58.861Z/static/images/quickstart/dashboard/name-your-assistant.png) Name your assistant. Once you have named your assistant, you can hit “Create” to create it. You will then see something like this: ![](https://files.buildwithfern.com/https://vapi.docs.buildwithfern.com/2025-01-31T05:46:58.861Z/static/images/quickstart/dashboard/assistant-created.png) The assistant overview. You can edit your assistant’s transcriber, model, & voice — and edit other advanced configuration. This is the assistant overview view — it gives you the ability to edit different attributes about your assistant, as well as see **cost** & **latency** projection information for each portion of it’s voice pipeline (this is very important data to have handy when building out your assistants). ###### Model Setup Now we’re going to set the “brains” of the assistant, the large language model. We’re going to be using `GPT-4` (from [OpenAI](https://openai.com/) ) for this demo (though you’re free to use `GPT-3.5`, or any one of your favorite LLMs). ###### Set Your OpenAI Provider Key (optional) Before we proceed, we can set our [provider key](https://docs.vapi.ai/customization/provider-keys) for OpenAI (this is just your OpenAI secret key). You can see all of your provider keys in the “Provider Keys” dashboard tab. You can also go directly to [dashboard.vapi.ai/keys](https://dashboard.vapi.ai/keys) . Vapi uses [provider keys](https://docs.vapi.ai/customization/provider-keys) you provide to communicate with LLM, TTS, & STT vendors on your behalf. It is most ideal that we set keys for the vendors we intend to use ahead of time. ![](https://files.buildwithfern.com/https://vapi.docs.buildwithfern.com/2025-01-31T05:46:58.861Z/static/images/quickstart/dashboard/model-provider-keys.png) We set our provider key for OpenAI so Vapi can make requests to their API. While we’re here it’d be ideal for you to go & set up provider keys for other providers you’re familiar with & intend to use later. ###### Set a First Message Assistants can **optionally** be configured with a `First Message`. This first message will be spoken by your assistant when either: * **A Web Call Connects:** when a web call is started with your assistant * **An Inbound Call is Picked-up:** an [inbound call](/glossary#inbound-call) is picked-up & answered by your assistant * **An Outbound Call is Dialed & Picked-up:** an [outbound call](/glossary#outbound-call) is dialed by your assistant & a person picks up Note that this first message cannot be interrupted & is guaranteed to be spoken. Certain use cases need a first message, while others do not. For our use case, we will want a first message. It would be ideal for us to have a first message like this: ` Vappy’s Pizzeria speaking, how can I help you? ` Some text-to-speech voices may struggle to pronounce ‘Vapi’ correctly, compartmentalizing it to be spoken letter by letter “V. A. P. I.” Some aspects of configuring your voice pipeline will require tweaks like this to get the target behaviour you want. This will be spoken by the assistant when a web or inbound phone call is received. ###### Set the System Prompt We will now set the `System Prompt` for our assistant. If you’re familiar with OpenAI’s API, this is the first prompt in the message list that we feed our LLM (learn more about prompt engineering on the [OpenAI docs](https://platform.openai.com/docs/guides/prompt-engineering) ). The system prompt can be used to configure the context, role, personality, instructions and so on for the assistant. In our case, a system prompt like this will give us the behaviour we want: ` | | | --- | | You are a voice assistant for Vappy’s Pizzeria, | | a pizza shop located on the Internet. | | | | Your job is to take the order of customers calling in. The menu has only 3 types | | of items: pizza, sides, and drinks. There are no other types of items on the menu. | | | | 1) There are 3 kinds of pizza: cheese pizza, pepperoni pizza, and vegetarian pizza | | (often called "veggie" pizza). | | 2) There are 3 kinds of sides: french fries, garlic bread, and chicken wings. | | 3) There are 2 kinds of drinks: soda, and water. (if a customer asks for a | | brand name like "coca cola", just let them know that we only offer "soda") | | | | Customers can only order 1 of each item. If a customer tries to order more | | than 1 item within each category, politely inform them that only 1 item per | | category may be ordered. | | | | Customers must order 1 item from at least 1 category to have a complete order. | | They can order just a pizza, or just a side, or just a drink. | | | | Be sure to introduce the menu items, don't assume that the caller knows what | | is on the menu (most appropriate at the start of the conversation). | | | | If the customer goes off-topic or off-track and talks about anything but the | | process of ordering, politely steer the conversation back to collecting their order. | | | | Once you have all the information you need pertaining to their order, you can | | end the conversation. You can say something like "Awesome, we'll have that ready | | for you in 10-20 minutes." to naturally let the customer know the order has been | | fully communicated. | | | | It is important that you collect the order in an efficient manner (succinct replies | | & direct questions). You only have 1 task here, and it is to collect the customers | | order, then end the conversation. | | | | - Be sure to be kind of funny and witty! | | - Keep all your responses short and simple. Use casual language, phrases like "Umm...", "Well...", and "I mean" are preferred. | | - This is a voice conversation, so keep your responses short, like in a real conversation. Don't ramble for too long. | ` You can copy & paste the above prompt into the `System Prompt` field. Now the model configuration for your assistant should look something like this: ![](https://files.buildwithfern.com/https://vapi.docs.buildwithfern.com/2025-01-31T05:46:58.861Z/static/images/quickstart/dashboard/assistant-model-set-up.png) Note how our model provider is set to OpenAI & the model is set to GPT-4. ###### Transcriber Setup The transcriber is what turns user speech into processable text for our LLM. This is the first step in the end-to-end voice pipeline. ###### Set Your Deepgram Provider Key (optional) We will be using [Deepgram](https://deepgram.com/) (which provides blazing-fast & accurate Speech-to-Text) as our STT provider. We will set our provider key for them in “Provider Keys”: ![](https://files.buildwithfern.com/https://vapi.docs.buildwithfern.com/2025-01-31T05:46:58.861Z/static/images/quickstart/dashboard/transcriber-providers-keys.png) ###### Set Transcriber We will set the model to `Nova 2` & the language to `en` for English. Now your assistant’s transcriber configuration should look something like this: ![](https://files.buildwithfern.com/https://vapi.docs.buildwithfern.com/2025-01-31T05:46:58.861Z/static/images/quickstart/dashboard/assistant-transcriber-config.png) Note how our transcriber is set to 'deepgram', the model is set to 'Nova 2', & the language is set to English. ###### Voice Setup The final portion of the voice pipeline is turning LLM output-text into speech. This process is called “Text-to-speech” (or TTS for short). We will be using a voice provider called [PlayHT](https://play.ht/) (they have very conversational voices), & a voice provided by them labeled `Jennifer` (`female`, `en-US`). You are free to use your favorite TTS voice platform here. [ElevenLabs](https://elevenlabs.io/) is another alternative — by now you should get the flow of plugging in vendors into Vapi (add provider key + pick provider in assistant config). You can skip the next step(s) if you don’t intend to use PlayHT. ###### Set Your PlayHT Provider Key (optional) If you haven’t already, sign up for an account with PlayHT at [play.ht](https://play.ht/) . Since their flows are liable to change — you can just grab your `API Key` & `User ID` from them. ![](https://files.buildwithfern.com/https://vapi.docs.buildwithfern.com/2025-01-31T05:46:58.861Z/static/images/quickstart/dashboard/voice-provider-keys.png) ###### Set Voice You will want to select `playht` in the “provider” field, & `Jennifer` in the “voice” field. We will leave all of the other settings untouched. ![](https://files.buildwithfern.com/https://vapi.docs.buildwithfern.com/2025-01-31T05:46:58.861Z/static/images/quickstart/dashboard/assistant-voice-config.png) Each voice provider offers a host of settings you can modulate to customize voices. Here we will leave all the defaults alone. To customize additional fields, this can be done via the [Assistant](/api-reference/assistants/create-assistant) API instead. Then, you can copy the assistant’s ID at the top of the assistant detail page: ![](https://files.buildwithfern.com/https://vapi.docs.buildwithfern.com/2025-01-31T05:46:58.861Z/static/images/quickstart/assistant-id-dashboard.png) Now we can call `.start()`, passing the persistent assistant’s ID: ` | | | | --- | --- | | 1 | vapi.start("79f3XXXX-XXXX-XXXX-XXXX-XXXXXXXXce48"); | ` If you need to override any assistant settings or set template variables, you can pass `assistantOverrides` as the second argument. For example, if the first message is “Hello `{{name}}`”, you can set `assistantOverrides` to replace `{{name}}` with `John`: ` | | | | --- | --- | | 1 | const assistantOverrides = { | | 2 | transcriber: { | | 3 | provider: "deepgram", | | 4 | model: "nova-2", | | 5 | language: "en-US", | | 6 | }, | | 7 | recordingEnabled: false, | | 8 | variableValues: { | | 9 | name: "John", | | 10 | }, | | 11 | }; | | 12 | | | 13 | vapi.start("79f3XXXX-XXXX-XXXX-XXXX-XXXXXXXXce48", assistantOverrides); | ` [Built with](https://buildwithfern.com/?utm_campaign=buildWith&utm_medium=docs&utm_source=docs.vapi.ai) --- # Outbound Calling — Vapi An outbound call is a phone call that is dialed and goes **“out”** from a phone number, & in our case, our AI assistant will be doing the dialing. There are **3 steps** we will cover to send our first outbound phone call: 1. **Create an Assistant:** we will create an [assistant](/assistants) & instruct it on how to conduct itself during the call 2. **Get a Phone Number:** we can either import existing numbers we own, or purchase one through Vapi 3. **Call Your Number:** we will set our assistant as the dialer, set the destination phone number, then make the call We can then send the outbound call, hopefully someone picks up! It is a violation of FCC law to dial phone numbers without consent in an automated manner. See [Telemarketing Sales Rule](/glossary#telemarketing-sales-rule) to learn more. Vapi’s Pizzeria --------------- We will be implementing a simple order-taking assistant for a pizza shop called “Vapi’s Pizzeria”. Vapi’s has 3 types of menu items: `pizza`, `side`s, & `drink`s. Customers will be ordering 1 of each. **Outbound Scenario:** We will imagine we are calling back a customer who originally called in to place an order. Our assistant is calling back to complete the ordering process with the customer. ![](https://files.buildwithfern.com/https://vapi.docs.buildwithfern.com/2025-01-31T05:46:58.861Z/static/images/quickstart/vapis-pizzeria.png) Customers can order 3 items: 1 pizza, 1 side, & 1 drink. The assistant will call the customer (who got disconnected) & finish the ordering process. Assistant Setup --------------- First we’re going to set up our assistant in the dashboard. Once our assistant’s **transcriber**, **model**, & **voice** are set up, we can have it call the customer to finish the order. You can visit your dashboard by going to [dashboard.vapi.ai](https://dashboard.vapi.ai/) ###### Sign-up or Log-in to Vapi If you haven’t already signed-up, you’re going to need an account before you can use the web dashboard. When you visit [dashboard.vapi.ai](https://dashboard.vapi.ai/) you may see something like this: ![](https://files.buildwithfern.com/https://vapi.docs.buildwithfern.com/2025-01-31T05:46:58.861Z/static/images/quickstart/dashboard/auth-ui.png) Sign-up for an account (or log-in to your existing account) — you will then find yourself inside the web dashboard. It will look something like this: ![](https://files.buildwithfern.com/https://vapi.docs.buildwithfern.com/2025-01-31T05:46:58.861Z/static/images/quickstart/dashboard/vapi-dashboard-post-signup.png) Your dashboard may look a bit different if you already have an account with assistants in it. The main idea is that we’re in the dashboard now. ###### Create an Assistant Now that you’re in your dashboard, we’re going to create an [assistant](/assistants) . Assistants are at the heart of how Vapi models AI voice agents — we will be setting certain properties on a new assistant to model an order-taking agent. Once in the “Assistants” dashboard tab (you should be in it by-default after log-in), you will see a button to create a new assistant. ![](https://files.buildwithfern.com/https://vapi.docs.buildwithfern.com/2025-01-31T05:46:58.861Z/static/images/quickstart/dashboard/create-new-assistant-button.png) Ensure you are in the 'Assistants' dashboard tab, then this button will allow you to begin the assistant creation flow. After clicking the create new assistant button, you will see a pop-up modal that asks you to pick a starter template. For our example we will start from a blank slate so choose the `Blank Template` option. ![](https://files.buildwithfern.com/https://vapi.docs.buildwithfern.com/2025-01-31T05:46:58.861Z/static/images/quickstart/dashboard/choose-blank-template.png) Ensure you are in the 'Assistants' dashboard tab, then this button will allow you to begin the assistant creation flow. You will then be able to name your assistant — you can name it whatever you’d like (`Vapi’s Pizza Front Desk`, for example): This name is only for internal labeling use. It is not an identifier, nor will the assistant be aware of this name. ![](https://files.buildwithfern.com/https://vapi.docs.buildwithfern.com/2025-01-31T05:46:58.861Z/static/images/quickstart/dashboard/name-your-assistant.png) Name your assistant. Once you have named your assistant, you can hit “Create” to create it. You will then see something like this: ![](https://files.buildwithfern.com/https://vapi.docs.buildwithfern.com/2025-01-31T05:46:58.861Z/static/images/quickstart/dashboard/assistant-created.png) The assistant overview. You can edit your assistant’s transcriber, model, & voice — and edit other advanced configuration. This is the assistant overview view — it gives you the ability to edit different attributes about your assistant, as well as see **cost** & **latency** projection information for each portion of it’s voice pipeline (this is very important data to have handy when building out your assistants). ###### Model Setup Now we’re going to set the “brains” of the assistant, the large language model. We’re going to be using `GPT-4` (from [OpenAI](https://openai.com/) ) for this demo (though you’re free to use `GPT-3.5`, or any one of your favorite LLMs). ###### Set Your OpenAI Provider Key (optional) Before we proceed, we can set our [provider key](/quickstart/phone/customization/provider-keys) for OpenAI (this is just your OpenAI secret key). You can see all of your provider keys in the “Provider Keys” dashboard tab. You can also go directly to [dashboard.vapi.ai/keys](https://dashboard.vapi.ai/keys) . Vapi uses [provider keys](/quickstart/phone/customization/provider-keys) you provide to communicate with LLM, TTS, & STT vendors on your behalf. It is most ideal that we set keys for the vendors we intend to use ahead of time. ![](https://files.buildwithfern.com/https://vapi.docs.buildwithfern.com/2025-01-31T05:46:58.861Z/static/images/quickstart/dashboard/model-provider-keys.png) We set our provider key for OpenAI so Vapi can make requests to their API. While we’re here it’d be ideal for you to go & set up provider keys for other providers you’re familiar with & intend to use later. ###### Set a First Message Assistants can **optionally** be configured with a `First Message`. This first message will be spoken by your assistant when either: * **A Web Call Connects:** when a web call is started with your assistant * **An Inbound Call is Picked-up:** an [inbound call](/glossary#inbound-call) is picked-up & answered by your assistant * **An Outbound Call is Dialed & Picked-up:** an [outbound call](/glossary#outbound-call) is dialed by your assistant & a person picks up Note that this first message cannot be interrupted & is guaranteed to be spoken. Certain use cases need a first message, while others do not. For our use case, we will want a first message. Since we are calling the customer back it would be ideal for us to have a first message like this: ` Hi this is Jennifer from Vappy’s Pizzeria giving you a call back since we got disconnected. Would you like to finish your order with us? ` Some text-to-speech voices may struggle to pronounce ‘Vapi’ correctly, compartmentalizing it to be spoken letter by letter “V. A. P. I.” Some aspects of configuring your voice pipeline will require tweaks like this to get the target behaviour you want. This will be spoken by the assistant when a web or inbound phone call is received. ###### Set the System Prompt We will now set the `System Prompt` for our assistant. If you’re familiar with OpenAI’s API, this is the first prompt in the message list that we feed our LLM (learn more about prompt engineering on the [OpenAI docs](https://platform.openai.com/docs/guides/prompt-engineering) ). The system prompt can be used to configure the context, role, personality, instructions and so on for the assistant. Since we are calling the customer back, we will tweak the base prompt a bit so the model understands the situation & new goal (recovering the order). A system prompt like this will give us the behaviour we want: ` | | | --- | | You are a voice assistant for Vappy’s Pizzeria, | | a pizza shop located on the Internet. | | | | Your job is to take the order of customers calling in. The menu has only 3 types | | of items: pizza, sides, and drinks. There are no other types of items on the menu. | | | | 1) There are 3 kinds of pizza: cheese pizza, pepperoni pizza, and vegetarian pizza | | (often called "veggie" pizza). | | 2) There are 3 kinds of sides: french fries, garlic bread, and chicken wings. | | 3) There are 2 kinds of drinks: soda, and water. (if a customer asks for a | | brand name like "coca cola", just let them know that we only offer "soda") | | | | Customers can only order 1 of each item. If a customer tries to order more | | than 1 item within each category, politely inform them that only 1 item per | | category may be ordered. | | | | Customers must order 1 item from at least 1 category to have a complete order. | | They can order just a pizza, or just a side, or just a drink. | | | | Be sure to introduce the menu items, don't assume that the caller knows what | | is on the menu (most appropriate at the start of the conversation). | | | | If the customer goes off-topic or off-track and talks about anything but the | | process of ordering, politely steer the conversation back to collecting their order. | | | | Once you have all the information you need pertaining to their order, you can | | end the conversation. You can say something like "Awesome, we'll have that ready | | for you in 10-20 minutes." to naturally let the customer know the order has been | | fully communicated. | | | | It is important that you collect the order in an efficient manner (succinct replies | | & direct questions). You only have 1 task here, and it is to collect the customers | | order, then end the conversation. | | | | - Be sure to be kind of funny and witty! | | - Keep all your responses short and simple. Use casual language, phrases like "Umm...", "Well...", and "I mean" are preferred. | | - This is a voice conversation, so keep your responses short, like in a real conversation. Don't ramble for too long. | | | | You are calling back a customer after the call got disconnected while they were | | ordering. Your job is to help them complete their order. | ` You can copy & paste the above prompt into the `System Prompt` field. Now the model configuration for your assistant should look something like this: ![](https://files.buildwithfern.com/https://vapi.docs.buildwithfern.com/2025-01-31T05:46:58.861Z/static/images/quickstart/phone/outbound/assistant-model-setup.png) Note how our model provider is set to OpenAI & the model is set to GPT-4. ###### Transcriber Setup The transcriber is what turns user speech into processable text for our LLM. This is the first step in the end-to-end voice pipeline. ###### Set Your Deepgram Provider Key (optional) We will be using [Deepgram](https://deepgram.com/) (which provides blazing-fast & accurate Speech-to-Text) as our STT provider. We will set our provider key for them in “Provider Keys”: ![](https://files.buildwithfern.com/https://vapi.docs.buildwithfern.com/2025-01-31T05:46:58.861Z/static/images/quickstart/dashboard/transcriber-providers-keys.png) ###### Set Transcriber We will set the model to `Nova 2` & the language to `en` for English. Now your assistant’s transcriber configuration should look something like this: ![](https://files.buildwithfern.com/https://vapi.docs.buildwithfern.com/2025-01-31T05:46:58.861Z/static/images/quickstart/dashboard/assistant-transcriber-config.png) Note how our transcriber is set to 'deepgram', the model is set to 'Nova 2', & the language is set to English. ###### Voice Setup The final portion of the voice pipeline is turning LLM output-text into speech. This process is called “Text-to-speech” (or TTS for short). We will be using a voice provider called [PlayHT](https://play.ht/) (they have very conversational voices), & a voice provided by them labeled `Jennifer` (`female`, `en-US`). You are free to use your favorite TTS voice platform here. [ElevenLabs](https://elevenlabs.io/) is another alternative — by now you should get the flow of plugging in vendors into Vapi (add provider key + pick provider in assistant config). You can skip the next step(s) if you don’t intend to use PlayHT. ###### Set Your PlayHT Provider Key (optional) If you haven’t already, sign up for an account with PlayHT at [play.ht](https://play.ht/) . Since their flows are liable to change — you can just grab your `API Key` & `User ID` from them. ![](https://files.buildwithfern.com/https://vapi.docs.buildwithfern.com/2025-01-31T05:46:58.861Z/static/images/quickstart/dashboard/voice-provider-keys.png) ###### Set Voice You will want to select `playht` in the “provider” field, & `Jennifer` in the “voice” field. We will leave all of the other settings untouched. ![](https://files.buildwithfern.com/https://vapi.docs.buildwithfern.com/2025-01-31T05:46:58.861Z/static/images/quickstart/dashboard/assistant-voice-config.png) Each voice provider offers a host of settings you can modulate to customize voices. Here we will leave all the defaults alone. Get a Phone Number ------------------ Now that we’ve configured how our assistant will behave, we want to figure out how to dial calls with it. We will need a phone number that we can call from. There are **2 ways** we can get a phone number into our Vapi account: 1. **Purchase a Number Through Vapi:** we can directly purchase phone numbers through Vapi. * Vapi will provision the phone number for us via Twilio * This can be done in the dashboard, or via the API (we will use the dashboard) 2. **Import from Twilio or Vonage:** if we already have a phone number with an external telephony provider (like Twilio or Vonage), we can import them into our Vapi account. ###### Provision via Vapi (faster) The quickest way to secure a phone number for your assistant is to purchase a phone number directly through Vapi. Ensure you have a card on file that Vapi can bill before proceeding, you can add your billing information in your dashboard at [dashboard.vapi.ai/billing](https://dashboard.vapi.ai/billing) Navigate to the “Phone Numbers” section & click the “Buy number” button: ![](https://files.buildwithfern.com/https://vapi.docs.buildwithfern.com/2025-01-31T05:46:58.861Z/static/images/quickstart/dashboard/buy-a-phone-number.png) Make sure you are in the 'Phone Numbers' dashboard tab. We will use the area code `415` for our phone number (these are area codes domestic to the US & Canada). ![](https://files.buildwithfern.com/https://vapi.docs.buildwithfern.com/2025-01-31T05:46:58.861Z/static/images/quickstart/dashboard/buy-phone-number-modal.png) Choose an area code for your phone number. Currently, only US & Canada phone numbers can be directly purchased through Vapi. Phone numbers in other regions must be imported, see our [phone calling](/phone-calling) guide. Click “Buy”, after purchasing a phone number you should see something like this: ![](https://files.buildwithfern.com/https://vapi.docs.buildwithfern.com/2025-01-31T05:46:58.861Z/static/images/quickstart/dashboard/phone-number-config.png) Here we can attach an assistant to the number for inbound calls (or perform an outbound call, with a select assistant). The phone number is now ready to be used (either for inbound or outbound calling). ###### Import from Twilio or Vonage We can also import an existing phone number we already own with either Twilio or Vonage. For example’s sake, we will proceed with [**Twilio**](https://twilio.com/) (though the steps are the same for Vonage as well). ###### Buy a Phone Number via Twilio (optional) If you don’t already have a number in Twilio, you can purchase one by going to your Twilio console’s “Buy a number” section: ![](https://files.buildwithfern.com/https://vapi.docs.buildwithfern.com/2025-01-31T05:46:58.861Z/static/images/quickstart/phone/buy-phone-number-twilio.png) The Twilio 'Buy a Number' page in the Twilio console. Once you’ve purchased a number, it will immediately be ready for import into Vapi. ###### Locate Twilio Account SID & Auth Token To complete the import on Vapi’s side, we will need to grab our Twilio **“Account SID”** & **“Auth Token”**. You should see a section for “API keys & tokens”, the credentials we will need for the import will live here. ![](https://files.buildwithfern.com/https://vapi.docs.buildwithfern.com/2025-01-31T05:46:58.861Z/static/images/quickstart/phone/twilio-api-key-nav.png) We will want to navigate to the credentials section of our account. Once we are in our “API keys & tokens” section, we will grab the Account SID & Auth Token: ![](https://files.buildwithfern.com/https://vapi.docs.buildwithfern.com/2025-01-31T05:46:58.861Z/static/images/quickstart/phone/twilio-credentials.png) We will use both of these credentials in the next step of importing via the Dashboard. ###### Import via Dashboard Navigate to the “Phone Numbers” section & click the “Import” button: ![](https://files.buildwithfern.com/https://vapi.docs.buildwithfern.com/2025-01-31T05:46:58.861Z/static/images/quickstart/phone/dashboard-import-phone-number.png) Click 'Import' in the 'Phone Numbers' tab of your dashboard. There you will input your phone number, as well as the credentials you retrieved in the previous step: ![](https://files.buildwithfern.com/https://vapi.docs.buildwithfern.com/2025-01-31T05:46:58.861Z/static/images/quickstart/phone/import-twilio-number-dashboard.png) Hit “Import” & you will come to the phone number detail page: ![](https://files.buildwithfern.com/https://vapi.docs.buildwithfern.com/2025-01-31T05:46:58.861Z/static/images/quickstart/phone/phone-number-import-complete.png) The phone number detail page, we can configure our phone number here. Your number is now ready to be attached to an assistant for inbound or outbound phone calling. Call Your Number ---------------- We can now make outbound calls to phone numbers, setting our assistant as the one doing the dialing. In the phone numbers section of the dashboard, go to your phone number detail page. We will: 1. fill out **our own phone number** as the number to dial 2. set our assistant as the one doing the calling ![](https://files.buildwithfern.com/https://vapi.docs.buildwithfern.com/2025-01-31T05:46:58.861Z/static/images/quickstart/phone/outbound/dial-outbound-call-dashboard.png) When we hit the call button, our assistant will make the outbound call to the phone number. You can now hit the call button to make the outbound call. Your assistant will dial the phone number & manage the order recovery process. Your assistant won’t yet be able to hang-up the phone at the end of the call. We will learn more about configuring call end behaviour in later guides. [Built with](https://buildwithfern.com/?utm_campaign=buildWith&utm_medium=docs&utm_source=docs.vapi.ai) --- # Web SDK — Vapi The Vapi Web SDK provides web developers a simple API for interacting with the realtime call functionality of Vapi. ### Installation Install the package: ` | | | | --- | --- | | $ | yarn add @vapi-ai/web | ` or w/ npm: ` | | | | --- | --- | | $ | npm install @vapi-ai/web | ` ### Importing Import the package: ` | | | | --- | --- | | 1 | import Vapi from "@vapi-ai/web"; | ` Then, create a new instance of the Vapi class, passing your **Public Key** as a parameter to the constructor: ` | | | | --- | --- | | 1 | const vapi = new Vapi("your-public-key"); | ` You can find your public key in the [Vapi Dashboard](https://dashboard.vapi.ai/account) . * * * Usage ----- ### `.start()` You can start a web call by calling the `.start()` function. The `start` function can either accept: 1. **a string**, representing an assistant ID 2. **an object**, representing a set of assistant configs (see [Create Assistant](/api-reference/assistants/create-assistant) ) The `start` function returns a promise that resolves to a call object. For example: ` | | | | --- | --- | | 1 | const call = await vapi.start(assistantId); | | 2 | // { "id": "bd2184a1-bdea-4d4f-9503-b09ca8b185e6", "orgId": "6da6841c-0fca-4604-8941-3d5d65f43a17", "createdAt": "2024-11-13T19:20:24.606Z", "updatedAt": "2024-11-13T19:20:24.606Z", "type": "webCall", ... } | ` #### Passing an Assistant ID If you already have an assistant that you created (either via [the Dashboard](/quickstart/dashboard) or [the API](/api-reference/assistants/create-assistant) ), you can start the call with the assistant’s ID: ` | | | | --- | --- | | 1 | vapi.start("79f3XXXX-XXXX-XXXX-XXXX-XXXXXXXXce48"); | ` #### Passing Assistant Configuration Inline You can also specify configuration for your assistant inline. This will not create a [persistent assistant](/assistants/persistent-assistants) that is saved to your account, rather it will create an ephemeral assistant only used for this call specifically. You can pass the assistant’s configuration in an object (see [Create Assistant](/api-reference/assistants/create-assistant) for a list of acceptable fields): ` | | | | --- | --- | | 1 | vapi.start({ | | 2 | transcriber: { | | 3 | provider: "deepgram", | | 4 | model: "nova-2", | | 5 | language: "en-US", | | 6 | }, | | 7 | model: { | | 8 | provider: "openai", | | 9 | model: "gpt-3.5-turbo", | | 10 | messages: [ | | 11 | { | | 12 | role: "system", | | 13 | content: "You are a helpful assistant.", | | 14 | }, | | 15 | ], | | 16 | }, | | 17 | voice: { | | 18 | provider: "playht", | | 19 | voiceId: "jennifer", | | 20 | }, | | 21 | name: "My Inline Assistant", | | 22 | ... | | 23 | }); | ` #### Overriding Assistant Configurations To override assistant settings or set template variables, you can pass `assistantOverrides` as the second argument. For example, if the first message is “Hello `{{name}}`”, set `assistantOverrides` to the following to replace `{{name}}` with `John`: ` | | | | --- | --- | | 1 | const assistantOverrides = { | | 2 | transcriber: { | | 3 | provider: "deepgram", | | 4 | model: "nova-2", | | 5 | language: "en-US", | | 6 | }, | | 7 | recordingEnabled: false, | | 8 | variableValues: { | | 9 | name: "Alice", | | 10 | }, | | 11 | }; | | 12 | | | 13 | vapi.start("79f3XXXX-XXXX-XXXX-XXXX-XXXXXXXXce48", assistantOverrides); | ` ### `.send()` During the call, you can send intermediate messages to the assistant (like [background messages](/assistants/background-messages) ). * `type` will always be `"add-message"` * the `message` field will have 2 items, `role` and `content`. ` | | | | --- | --- | | 1 | vapi.send({ | | 2 | type: "add-message", | | 3 | message: { | | 4 | role: "system", | | 5 | content: "The user has pressed the button, say peanuts", | | 6 | }, | | 7 | }); | ` Possible values for role are `system`, `user`, `assistant`, `tool` or `function`. ### `.stop()` You can stop the call session by calling the `stop` method: ` | | | | --- | --- | | 1 | vapi.stop(); | ` This will stop the recording and close the connection. ### `.isMuted()` Check if the user’s microphone is muted: ` | | | | --- | --- | | 1 | vapi.isMuted(); | ` ### `.setMuted(muted: boolean)` You can mute & unmute the user’s microphone with `setMuted`: ` | | | | --- | --- | | 1 | vapi.isMuted(); // false | | 2 | vapi.setMuted(true); | | 3 | vapi.isMuted(); // true | ` ### `say(message: string, endCallAfterSpoken?: boolean)` The `say` method can be used to invoke speech and gracefully terminate the call if needed ` | | | | --- | --- | | 1 | vapi.say("Our time's up, goodbye!", true) | ` Events ------ You can listen on the `vapi` instance for events. These events allow you to react to changes in the state of the call or user speech. #### `speech-start` Occurs when your AI assistant has started speaking. ` | | | | --- | --- | | 1 | vapi.on("speech-start", () => { | | 2 | console.log("Assistant speech has started."); | | 3 | }); | ` #### `speech-end` Occurs when your AI assistant has finished speaking. ` | | | | --- | --- | | 1 | vapi.on("speech-end", () => { | | 2 | console.log("Assistant speech has ended."); | | 3 | }); | ` #### `call-start` Occurs when the call has connected & begins. ` | | | | --- | --- | | 1 | vapi.on("call-start", () => { | | 2 | console.log("Call has started."); | | 3 | }); | ` #### `call-end` Occurs when the call has disconnected & ended. ` | | | | --- | --- | | 1 | vapi.on("call-end", () => { | | 2 | console.log("Call has ended."); | | 3 | }); | ` #### `volume-level` Realtime volume level updates for the assistant. A floating-point number between `0` & `1`. `` | | | | --- | --- | | 1 | vapi.on("volume-level", (volume) => { | | 2 | console.log(`Assistant volume level: ${volume}`); | | 3 | }); | `` #### `message` Various assistant messages can be sent back to the client during the call. These are the same messages that your [server](/server-url) would receive. At [assistant creation time](/api-reference/assistants/create-assistant) , you can specify on the `clientMessages` field the set of messages you’d like the assistant to send back to the client. Those messages will come back via the `message` event: ` | | | | --- | --- | | 1 | // Various assistant messages can come back (like function calls, transcripts, etc) | | 2 | vapi.on("message", (message) => { | | 3 | console.log(message); | | 4 | }); | ` #### `error` Handle errors that occur during the call. ` | | | | --- | --- | | 1 | vapi.on("error", (e) => { | | 2 | console.error(e); | | 3 | }); | ` * * * Resources --------- [NPM\ \ View the package on NPM.](https://www.npmjs.com/package/@vapi-ai/web) [GitHub\ \ View the package on GitHub.](https://github.com/VapiAI/web) [Try Our Quickstart\ \ Get up and running quickly with the Web SDK.](/quickstart/web) [Built with](https://buildwithfern.com/?utm_campaign=buildWith&utm_medium=docs&utm_source=docs.vapi.ai) --- # Fine-tuned OpenAI models — Vapi Vapi supports using any OpenAI-compatible endpoint as the LLM. This includes services like [OpenRouter](https://openrouter.ai/) , [AnyScale](https://www.anyscale.com/) , [Together AI](https://www.together.ai/) , or your own server. ###### When to Use Custom LLMs * For an open-source LLM, like Mixtral * To update the context during the conversation * To customize the messages before they’re sent to an LLM Using an LLM provider --------------------- You’ll first want to POST your API key via the `/credential` endpoint: ` | | | | --- | --- | | 1 | { | | 2 | "provider": "openrouter", | | 3 | "apiKey": "" | | 4 | } | ` Then, you can create an assistant with the model provider: ` | | | | --- | --- | | 1 | { | | 2 | "name": "My Assistant", | | 3 | "model": { | | 4 | "provider": "openrouter", | | 5 | "model": "cognitivecomputations/dolphin-mixtral-8x7b", | | 6 | "messages": [ | | 7 | { | | 8 | "role": "system", | | 9 | "content": "You are an assistant." | | 10 | } | | 11 | ], | | 12 | "temperature": 0.7 | | 13 | } | | 14 | } | ` Using Fine-Tuned OpenAI Models ------------------------------ To set up your OpenAI Fine-Tuned model, you need to follow these steps: 1. Set the custom llm URL to `https://api.openai.com/v1`. 2. Assign the custom llm key to the OpenAI key. 3. Update the model to their model. 4. Execute a PATCH request to the `/assistant` endpoint and ensure that `model.metadataSendMode` is set to off. Using your server ----------------- To set up your server to act as the LLM, you’ll need to create an endpoint that is compatible with the [OpenAI Client](https://platform.openai.com/docs/api-reference/making-requests) . For best results, your endpoint should also support streaming completions. If your server is making calls to an OpenAI compatble API, you can pipe the requests directly back in your response to Vapi. If you’d like your OpenAI-compatible endpoint to be authenticated, you can POST your server’s API key and URL via the `/credential` endpoint: ` | | | | --- | --- | | 1 | { | | 2 | "provider": "custom-llm", | | 3 | "apiKey": "" | | 4 | } | ` If your server isn’t authenticated, you can skip this step. Then, you can create an assistant with the `custom-llm` model provider: ` | | | | --- | --- | | 1 | { | | 2 | "name": "My Assistant", | | 3 | "model": { | | 4 | "provider": "custom-llm", | | 5 | "url": "", | | 6 | "model": "my-cool-model", | | 7 | "messages": [ | | 8 | { | | 9 | "role": "system", | | 10 | "content": "You are an assistant." | | 11 | } | | 12 | ], | | 13 | "temperature": 0.7 | | 14 | } | | 15 | } | ` [Built with](https://buildwithfern.com/?utm_campaign=buildWith&utm_medium=docs&utm_source=docs.vapi.ai) --- # Multilingual — Vapi Vapi’s multilingual support is primarily facilitated through transcribers, which are part of the speech-to-text process. The pipeline consists of three key elements: text-to-speech, speech-to-text, and the llm model, which acts as the brain of the operation. Each of these elements can be customized using different providers. Transcribers (Speech-to-Text) ----------------------------- Currently, Vapi supports two providers for speech-to-text transcriptions: * `Deepgram` (nova - family models) * `Talkscriber` (whisper model) Each provider supports different languages. For more detailed information, you can visit your dashboard and navigate to the transcribers tab on the assistant page. Here, you can see the languages supported by each provider and the available models. **Note that not all models support all languages**. For specific details, you can refer to the documentation for the corresponding providers. Voice (Text-to-Speech) ---------------------- Once you have set your transcriber and corresponding language, you can choose a voice for text-to-speech in that language. For example, you can choose a voice with a Spanish accent if needed. Vapi currently supports the following providers for text-to-speech: * `PlayHT` * `11labs` * `Rime-ai` * `Deepgram` * `OpenAI` * `Azure` * `Lmnt` * `Neets` Each provider offers varying degrees of language support. Azure, for instance, supports the most languages, with approximately 400 prebuilt voices across 140 languages and variants. You can also create your own custom languages with other providers. Multilingual Support -------------------- For multilingual support, you can choose providers like Eleven Labs or Azure, which have models and voices designed for this purpose. This allows your voice assistant to understand and respond in multiple languages, enhancing the user experience for non-English speakers. To set up multilingual support, you no longer need to specify the desired language when configuring the voice assistant. This configuration in the voice section is deprecated. Instead, you directly choose a voice that supports the desired language from your voice provider. This can be done when you are setting up or modifying your voice assistant. Here is an example of how to set up a voice assistant that speaks Spanish: ` | | | | --- | --- | | 1 | { | | 2 | "voice": { | | 3 | "provider": "azure", | | 4 | "voiceId": "es-ES-ElviraNeural" | | 5 | } | | 6 | } | ` In this example, the voice `es-ES-ElviraNeural` from the provider `azure` supports Spanish. You can replace `es-ES-ElviraNeural` with the ID of any other voice that supports your desired language. By leveraging Vapi’s multilingual support, you can make your voice assistant more accessible and user-friendly, reaching a wider audience and providing a better user experience. [Built with](https://buildwithfern.com/?utm_campaign=buildWith&utm_medium=docs&utm_source=docs.vapi.ai) --- # Introduction — Vapi Vapi lets you use various providers with some preset voice. At the same time you can also create your own custom voices in the supported providers and use them with Vapi. You can update the `voice` property in the assistant configuration when you are creating the assistant to use your custom voice. ` | | | | --- | --- | | 1 | { | | 2 | "voice": { | | 3 | "provider": "deepgram", | | 4 | "voiceId": "your-voice-id" | | 5 | } | | 6 | } | ` [Built with](https://buildwithfern.com/?utm_campaign=buildWith&utm_medium=docs&utm_source=docs.vapi.ai) --- # Elevenlabs — Vapi This guide outlines the procedure for integrating your cloned voice with 11labs through the VAPI platform. An subscription is required for this process to work. To integrate your cloned voice with 11labs using the VAPI platform, follow these steps. 1. **Obtain an 11labs API Subscription:** Visit the [11labs pricing page](https://elevenlabs.io/pricing) and subscribe to an API plan that suits your needs. 2. **Retrieve Your API Key:** Go to the ‘Profile + Keys’ section on the 11labs website to get your API key. 3. **Enter Your API Key in VAPI:** Navigate to the [VAPI Provider Key section](https://dashboard.vapi.ai/keys) and input your 11labs API key under the 11labs section. 4. **Sync Your Cloned Voice:** From the [Voice Library](https://dashboard.vapi.ai/voice-library) in VAPI, select 11labs as your voice provider and click on “Sync with 11labs.” 5. **Search and Use Your Cloned Voice:** After syncing, you can search for your cloned voice within the voice library and directly use it with your assistant. By following these steps, you will successfully integrate your cloned voice from 11labs with VAPI. **Video Tutorial:** [Built with](https://buildwithfern.com/?utm_campaign=buildWith&utm_medium=docs&utm_source=docs.vapi.ai) --- # Connecting Your Custom LLM to Vapi: A Comprehensive Guide — Vapi This guide provides a comprehensive walkthrough on integrating Vapi with OpenAI’s gpt-3.5-turbo-instruct model using a custom LLM configuration. We’ll leverage Ngrok to expose a local development environment for testing and demonstrate the communication flow between Vapi and your LLM. Prerequisites ------------- * **Vapi Account**: Access to the Vapi Dashboard for configuration. * **OpenAI API Key**: With access to the gpt-3.5-turbo-instruct model. * **Python Environment**: Set up with the OpenAI library (`pip install openai`). * **Ngrok**: For exposing your local server to the internet. * **Code Reference**: Familiarize yourself with the `/openai-sse/chat/completions` endpoint function in the provided Github repository: [Server-Side Example Python Flask](https://github.com/VapiAI/server-side-example-python-flask/blob/main/app/api/custom_llm.py) . Step 1: Setting Up Your Local Development Environment ----------------------------------------------------- **1\. Create a Python Script (app.py):** ` | | | | --- | --- | | 1 | from flask import Flask, request, jsonify | | 2 | import openai | | 3 | | | 4 | app = Flask(__name__) | | 5 | openai.api_key = "YOUR_OPENAI_API_KEY" # Replace with your actual API key | | 6 | | | 7 | @app.route("/chat/completions", methods=["POST"]) | | 8 | def chat_completions(): | | 9 | data = request.get_json() | | 10 | # Extract relevant information from data (e.g., prompt, conversation history) | | 11 | # ... | | 12 | | | 13 | response = openai.ChatCompletion.create( | | 14 | model="gpt-3.5-turbo-instruct", | | 15 | messages=[ | | 16 | {"role": "system", "content": "You are a helpful assistant."}, | | 17 | # ... (Add messages from conversation history and current prompt) | | 18 | ] | | 19 | ) | | 20 | # Format response according to Vapi's structure | | 21 | # ... | | 22 | return jsonify(formatted_response) | | 23 | | | 24 | if __name__ == "__main__": | | 25 | app.run(debug=True, port=5000) # You can adjust the port if needed | ` **2\. Run the Script:** Execute the Python script using python app.py in your terminal. This will start the Flask server on the specified port (5000 in this example). **3\. Expose with Ngrok:** Open a new terminal window and run ngrok http 5000 (replace 5000 with your chosen port) to create a public URL that tunnels to your local server. Step 2: Configuring Vapi with Custom LLM ---------------------------------------- **1\. Access Vapi Dashboard:** Log in to your Vapi account and navigate to the “Model” section. **2\. Select Custom LLM:** Choose the “Custom LLM” option to set up the integration. **3\. Enter Ngrok URL:** Paste the public URL generated by ngrok (e.g., [https://your-unique-id.ngrok.io](https://your-unique-id.ngrok.io/) ) into the endpoint field. This will be the URL Vapi uses to communicate with your local server. **4\. Test the Connection:** Send a test message through the Vapi interface to ensure it reaches your local server and receives a response from the OpenAI API. Verify that the response is displayed correctly in Vapi. Step 3: Understanding the Communication Flow -------------------------------------------- **1\. Vapi Sends POST Request:** When a user interacts with your Vapi application, Vapi sends a POST request containing conversation context and metadata to the configured endpoint (your ngrok URL). **2\. Local Server Processes Request:** Your Python script receives the POST request and the chat\_completions function is invoked. **3\. Extract and Prepare Data:** The script parses the JSON data, extracts relevant information (prompt, conversation history), and builds the prompt for the OpenAI API call. **4\. Call to OpenAI API:** The constructed prompt is sent to the gpt-3.5-turbo-instruct model using the openai.ChatCompletion.create method. **5\. Receive and Format Response:** The response from OpenAI, containing the generated text, is received and formatted according to Vapi’s expected structure. **6\. Send Response to Vapi:** The formatted response is sent back to Vapi as a JSON object. **7\. Vapi Displays Response:** Vapi receives the response and displays the generated text within the conversation interface to the user. By following these detailed steps and understanding the communication flow, you can successfully connect Vapi to OpenAI’s gpt-3.5-turbo-instruct model and create powerful conversational experiences within your Vapi applications. The provided code example and reference serve as a starting point for you to build and customize your integration based on your specific needs. **Video Tutorial:** [Built with](https://buildwithfern.com/?utm_campaign=buildWith&utm_medium=docs&utm_source=docs.vapi.ai) --- # JWT Authentication — Vapi This documentation provides an overview of JWT (JSON Web Token) Authentication and demonstrates how to generate a JWT token and use it to authenticate API requests securely. Prerequisites ------------- Before you proceed, ensure you have the following: * An environment that supports JWT generation and API calls (e.g., a programming language or framework) * An account with a service that requires JWT authentication * Environment variables set up for the necessary credentials (e.g., organization ID and private key, both can be found in your Vapi portal) Generating a JWT Token ---------------------- The following steps outline how to generate a JWT token: 1. **Define the Payload**: The payload contains the data you want to include in the token. In this case, it includes an `orgId`. 2. **Get the Private Key**: The private key (provided by Vapi) is used to sign the token. Ensure it is securely stored, often in environment variables. 3. **Set Token Options**: Define options for the token, such as the expiration time (`expiresIn`). 4. **Generate the Token**: Use a JWT library or built-in functionality to generate the token with the payload, key, and options. ### Example ` | | | | --- | --- | | 1 | // Define the payload | | 2 | const payload = { | | 3 | orgId: process.env.ORG_ID, | | 4 | }; | | 5 | | | 6 | // Get the private key from environment variables | | 7 | const key = process.env.PRIVATE_KEY; | | 8 | | | 9 | // Define token options | | 10 | const options = { | | 11 | expiresIn: '1h', | | 12 | }; | | 13 | | | 14 | // Generate the token using a JWT library or built-in functionality | | 15 | const token = generateJWT(payload, key, options); | ` ### Explanation * **Payload**: The payload includes the `orgId`, representing the organization ID. * **Key**: The private key is used to sign the token, ensuring its authenticity. * **Options**: The `expiresIn` option specifies that the token will expire in 1 hour. * **Token Generation**: The `generateJWT` function (a placeholder for the actual JWT generation method) creates the token using the provided payload, key, and options. Making an Authenticated API Request ----------------------------------- Once the token is generated, you can use it to make authenticated API requests. The following steps outline how to make an authenticated request: 1. **Define the API Endpoint**: Specify the URL of the API you want to call. 2. **Set the Headers**: Include the `Content-Type` and `Authorization` headers in your request. The `Authorization` header should include the generated JWT token prefixed with `Bearer`. 3. **Make the API Call**: Use an appropriate method to send the request and handle the response. ### Example `` | | | | --- | --- | | 1 | async function getAssistants() { | | 2 | const response = await fetch('https://api.vapi.ai/assistant', { | | 3 | method: 'GET', | | 4 | headers: { | | 5 | 'Content-Type': 'application/json', | | 6 | Authorization: `Bearer ${token}`, | | 7 | }, | | 8 | }); | | 9 | | | 10 | const data = await response.json(); | | 11 | console.log(data); | | 12 | } | | 13 | | | 14 | fetchData().catch(console.error); | `` ### Explanation * **API Endpoint**: The URL of the API you want to call. * **Headers**: The `Content-Type` is set to `application/json`, and the `Authorization` header includes the generated JWT token. * **API Call**: The `fetchData` function makes an asynchronous GET request to the specified API endpoint and logs the response. ### Usage With the generated token, you can authenticate API requests to any endpoint requiring authentication. The token will be valid for the duration specified in the options (1 hour in this case). Conclusion ---------- This documentation covered the basics of generating a JWT token and demonstrated how to use the token to make authenticated API requests. Ensure that your environment variables (e.g., `ORG_ID` and `PRIVATE_KEY`) are correctly set up before running the code. [Built with](https://buildwithfern.com/?utm_campaign=buildWith&utm_medium=docs&utm_source=docs.vapi.ai) --- # PlayHT — Vapi This guide outlines the procedure for integrating your cloned voice with Play.ht through the VAPI platform. An API subscription is required for this process to work. To integrate your cloned voice with [Play.ht](http://play.ht/) using the VAPI platform, follow these steps. 1. **Obtain a Play.ht API Subscription:** Visit the [Play.ht pricing page](https://play.ht/studio/pricing) and subscribe to an API plan. 2. **Retrieve Your User ID and Secret Key:** Go to the [API Access section](https://play.ht/studio/api-access) on Play.ht to get your User ID and Secret Key. 3. **Enter Your API Keys in VAPI:** Navigate to the [VAPI Provider Key section](https://dashboard.vapi.ai/keys) and input your Play.ht API keys under the Play.ht section. 4. **Sync Your Cloned Voice:** From the [Voice Library](https://dashboard.vapi.ai/voice-library) in VAPI, select Play.ht as your voice provider and click on “Sync with Play.ht.” 5. **Search and Use Your Cloned Voice:** After syncing, you can search for your cloned voice within the voice library and directly use it with your assistant. **Video Tutorial:** [Built with](https://buildwithfern.com/?utm_campaign=buildWith&utm_medium=docs&utm_source=docs.vapi.ai) --- # Introduction — Vapi **Assistant** is a fancy word for an AI configuration that can be used across phone calls and Vapi clients. Your voice assistant can augment your customer support and experience for call centers, business websites, mobile apps, and much more. There are three core components: **Transcriber**, **Model**, and **Voice**. These can be configured, mixed, and matched for your use case. There are also various other configurable properties you can find [here](/api-reference/assistants/create-assistant) Below, check out some ways you can layer in powerful customizations and features to meet any use case. Advanced Concepts ----------------- [Provider Keys\ \ Add your API keys for other providers](/customization/provider-keys) [Custom LLM URL\ \ Plug in your own LLM](/customization/custom-llm/fine-tuned-openai-models) [Call Functions\ \ Forward and hang up with function calls](/assistants/function-calling) [Persistent / Temporary Assistants\ \ Which setup is best for you?](/assistants/persistent-assistants) [Built with](https://buildwithfern.com/?utm_campaign=buildWith&utm_medium=docs&utm_source=docs.vapi.ai) --- # Background Messaging — Vapi Scenario Overview ----------------- As a developer you may run into scenarios where a user action, such as pressing a button, needs to be logged in the chat history without overt user involvement. This could be crucial for maintaining conversation context or system logging purposes. [1](/assistants/background-messages#add-a-button-to-trigger-the-message) ### Add a Button to Trigger the Message Add a button to your interface with an `onClick` event handler that will call a function to send the system message: ` | | | | --- | --- | | 1 | | ` [2](/assistants/background-messages#log-the-action-as-a-system-message) ### Log the Action as a System Message When the button is clicked, the `logUserAction` function will silently insert a system message into the chat history: ` | | | | --- | --- | | 1 | function logUserAction() { | | 2 | // Function to log the user action | | 3 | vapi.send({ | | 4 | type: "add-message", | | 5 | message: { | | 6 | role: "system", | | 7 | content: "The user has pressed the button, say peanuts", | | 8 | }, | | 9 | }); | | 10 | } | ` * `vapi.send`: The primary function to interact with your assistant, handling various requests or commands. * `type: "add-message"`: Specifies the command to add a new message. * `message`: This is the actual message that you want to add to the message history. * `role`: “system” Designates the message origin as ‘system’, ensuring the addition is unobtrusive. Other possible values of role are ‘user’ | ‘assistant’ | ‘tool’ | ‘function’ * `content`: The actual message text to be added. Practical Use Cases * Silent logging of user activities. * Contextual updates in conversations triggered by background processes. * Non-intrusive user experience enhancements through additional information provision. [Built with](https://buildwithfern.com/?utm_campaign=buildWith&utm_medium=docs&utm_source=docs.vapi.ai) --- # Tavus — Vapi This guide outlines the procedure for integrating your custom replica with Tavus through the VAPI platform. An API subscription is required for this process to work. This process is only required if you would like to use your **custom Tavus replicas**. This process is not required to use stock replicas on the Vapi platform. To integrate your custom replica with [Tavus](https://platform.tavus.io/) using the VAPI platform, follow these steps. 1. **Obtain a Tavus API Subscription:** Visit the [Tavus pricing page](https://platform.tavus.io/billing) and subscribe to an API plan. 2. **Retrieve Your API Key:** Go to the [API Keys section](https://platform.tavus.io/api-keys) on Tavus to get your API key. 3. **Enter Your API Key in VAPI:** Navigate to the [VAPI Provider Key section](https://dashboard.vapi.ai/keys) and input your Tavus API key under the Tavus section. 4. **Enter Your Custom Replica ID:** After adding your API key, you can select Tavus as your assistant’s voice provider and add your Custom Replica ID manually through the dashboard. Alternatively, you may use the API and specify the replica ID within the `voiceId` field. **Video Tutorial:** [Built with](https://buildwithfern.com/?utm_campaign=buildWith&utm_medium=docs&utm_source=docs.vapi.ai) --- # Voice Formatting Plan — Vapi What is Voice Input Formatted? ------------------------------ When interacting with voice assistants, you might notice terms like `Voice Input Formatted` in call logs or system outputs. This article explains what this means, how it works, and why it’s important for delivering clear and natural voice interactions. Voice Input Formatted is a function that takes raw text from a language model (LLM) and cleans it up so text-to-speech (TTS) provider can read it more naturally. It’s **on by default** in your assistant’s voice provider settings, because it helps turn things like: * `$42.50` → `forty two dollars and fifty cents` * `ST` → `STREET`, * or phone numbers → spaced digits (“1 2 3 4 5 6 7 8 9 0”). If you prefer the raw, unchanged text, you can **turn off** these transformations, which we’ll show you later. ### Log Example ![Screenshot 2025-01-21 at 10.23.19.png](https://img.notionusercontent.com/s3/prod-files-secure%2Ffdafdda2-774c-49e6-8896-a352ff4d44f3%2Ff603f2bd-36cf-4085-a3bc-f76c89a1ef75%2FScreenshot_2025-01-21_at_10.23.19.png/size/w=2000?exp=1737581744&sig=yoEEQF-BcTTgEVBNdcZh9MWHye2moRsbUcxGPjATNX8) 1\. Step-by-Step Transformations -------------------------------- When `Voice Input Formatted` runs, it calls a bunch of helper functions in a row. Each one focuses on a different kind of text pattern. The entire process happens in this order: 1. **removeAngleBracketContent** 2. **removeMarkdownSymbols** 3. **removePhrasesInAsterisks** 4. **replaceNewLinesWithPeriods** 5. **replaceColonsWithPeriods** 6. **formatAcronyms** 7. **formatDollarAmounts** 8. **formatEmails** 9. **formatDates** 10. **formatTimes** 11. **formatDistances, formatUnits, formatPercentages, formatPhoneNumbers** 12. **formatNumbers** 13. **Applying Replacements** We’ll walk you through them using a **shorter example** than before. ### 1.1 Our Simpler Example Input ` | | | --- | | Hello world | | **Wanted** to say *hi* | | We have NASA and .NET here, | | call me at 123-456-7890, | | price: $42.50 | | and the date is 2023 05 10 | | and time is 14:00 | | Distance is 5km | | We might see 9999 | | the address is 320 ST 21 RD | | my email is [[email protected]](/cdn-cgi/l/email-protection) | ` ### 1.2 removeAngleBracketContent * **What it does**: Removes `` unless it’s ``, ``, or double angle brackets `<< >>`. * **Example effect**: `` gets removed. **Result so far**: ` | | | --- | | Hello world | | **Wanted** to say *hi* | | We have NASA and .NET here, | | call me at 123-456-7890, | | price: $42.50 | | and the date is 2023 05 10 | | and time is 14:00 | | Distance is 5km | | We might see 9999 | | the address is 320 ST 21 RD | | my email is [[email protected]](/cdn-cgi/l/email-protection) | ` ### 1.3 removeMarkdownSymbols * **What it does**: Removes `_`, \`\`\`, or `~`. Some versions also remove double asterisks, but that might happen in a later step (next function). In this example, there’s `**Wanted**`, which _might_ remain if we strictly only remove `_`, backticks, and tildes. If the code does remove `**` as well, it’ll vanish here or in the next step. Let’s assume it doesn’t remove them in this step. **Result**: \_No real change if the code only targets `_` , \`\`\`, and `~`.\_ ` | | | --- | | Hello world | | **Wanted** to say *hi* | | ... | ` ### 1.4 removePhrasesInAsterisks * **What it does**: Looks for `some text*` or `*some text**` and cuts it out. In our text, we have `**Wanted**` and `*hi*`. Both get removed if the function is broad enough to remove single and double-asterisk blocks. **Result**: ` | | | --- | | Hello world | | to say | | We have NASA and .NET here, | | call me at 123-456-7890, | | price: $42.50 | | and the date is 2023 05 10 | | and time is 14:00 | | Distance is 5km | | We might see 9999 | | the address is 320 ST 21 RD | | my email is [[email protected]](/cdn-cgi/l/email-protection) | ` ### 1.5 replaceNewLinesWithPeriods * **What it does**: Turns line breaks into `.` or `.` and merges repeated periods. Let’s say the above text has line breaks. After this step, it’s more of a single line (or fewer lines), each newline replaced by a period. **Result** (roughly): ` Hello world . to say . We have NASA and .NET here, call me at 123-456-7890, price: $42.50 and the date is 2023 05 10 and time is 14:00 Distance is 5km We might see 9999 the address is 320 ST 21 RD my email is [[email protected]](/cdn-cgi/l/email-protection) ` ### 1.6 replaceColonsWithPeriods * **What it does**: `:` → `.` Our text has `price: $42.50`. That becomes `price. $42.50`. **Result**: ` Hello world . to say . We have NASA and .NET here, call me at 123-456-7890, price. $42.50 ... ` ### 1.7 formatAcronyms * **What it does**: * If something is in a known “to-lower” list (like `NASA`, `.NET`), it becomes lowercase (`nasa`, `.net`). * If it’s all-caps but not recognized, it might get spaced letters. If it has vowels, it’s left alone. In the example: * `NASA` → `nasa` * `.NET` → `.net` ### 1.8 formatDollarAmounts * **What it does**: `$42.50` → “forty two dollars and fifty cents.” ### 1.9 formatEmails * **What it does**: Replaces `@` with “ at ” and `.` with “ dot ” in emails. * `[[email protected]](/cdn-cgi/l/email-protection) ` → `JOHN dot DOE at example dot COM` ### 1.10 formatDates * **What it does**: `YYYY MM DD` → e.g. “Wednesday, May 10, 2023” (if valid). * `2023 05 10` become “Wednesday, May 10, 2023” (day name depends on how the code calculates it). ### 1.11 formatTimes * **What it does**: `14:00` → `14` (since minutes are “00,” it remove them). * If it was `14:30`, it might become `14 30`. ### 1.12 formatDistances, formatUnits, formatPercentages, formatPhoneNumbers * **Distances**: `5km` → “5 kilometers.” * **Units**: e.g. `43 lb` → “forty three pounds.” * **Percentages**: `50%` → “50 percent.” * **PhoneNumbers**: `123-456-7890` → `1 2 3 4 5 6 7 8 9 0`. ### 1.13 formatNumbers * **What it does**: * Skips year-like numbers if they’re below current year(2025). * For large numbers above a cutoff (e.g. 1000 or 5000), it reads as digits. * Negative numbers: `9` → “minus nine.” * Decimals: `2.5` → “two point five.” In our case, `9999` might be big enough to become spelled out (nine thousand nine hundred ninety nine) or digits spaced out, depending on the cutoff. `2023` used with `05 10` might get turned into a date, so it’s handled by the date logic, not the plain number logic. ### 1.14 Applying Replacements (street-suffix expansions) * **Runs last**. If you have user-defined replacements like `\bST\b` → `STREET`, `\bRD\b` → `ROAD`, it changes them after all the other steps. * So `320 ST 21 RD` → `320 STREET 21 ROAD`. **End Result**: A single line of text with all the helpful expansions and transformations done. 2\. Formatting Plan: Customization Options ------------------------------------------ The **Formatting Plan** governs how Voice Input Formatted works. Here are the main settings you can customize: ### 2.1 Enabled Determines whether the formatting is applied. * **Default**: `true` * To disable: Set `voice.chunkPlan.formatPlan.enabled = false`. ### 2.2 Number-to-Digits Cutoff This decides when numbers are read as digits instead of words. * **Default**: `2025` (current year). * The code generally **doesn’t** convert numbers below the current year (like `2025`) into spelled-out words, so it stays as digits if it’s obviously a year. * If a number is bigger than the cutoff (`numberToDigitsCutoff`), it reads digits out loud. * Negative numbers become “minus,” decimals get “point,” etc. * Example: With a cutoff of `2025`, numbers like `12345` will remain digits. * To ensure larger numbers are spelled out, set the cutoff higher, like `300000`. For example: * `30003` → “thirty thousand and three” (with a cutoff of `300000`). ### 2.3 Replacements Allows exact or regex-based substitutions in text. * **Example 1**: Replace `hello` with `hi`:`{ type: 'exact', key: 'hello', value: 'hi' }`. * **Example 2**: Replace words matching a pattern:`{ type: 'regex', regex: '\\\\b[a-zA-Z]{5}\\\\b', value: 'hi' }`. ### Note Currently, only **replacements** and **number-to-digits cutoff** are exposed for customization. Other options, such as toggling acronym replacement, are not exposed to be toggled. 3\. How to Turn It Off ---------------------- By default, the entire pipeline is **on** because it helps TTS read better. To **turn it off**, set: ` | | | --- | | voice.chunkPlan.enabled = false; | | // or | | voice.chunkPlan.formatPlan.enabled = false; | ` Any of those flags being `false` means we **skip** calling `Voice Input Formatted`. 4\. Conclusion -------------- * `Voice Input Formatted` orchestrates a chain of mini-functions that together fix punctuation, expand abbreviations, and make text more readable out loud. * You can keep it **on** for better TTS results or **off** if you need the raw LLM output. * The final transformations, especially the user-supplied replacements (like street expansions), happen **last**, so keep that in mind it rely on other expansions earlier. [Built with](https://buildwithfern.com/?utm_campaign=buildWith&utm_medium=docs&utm_source=docs.vapi.ai) --- # Custom Keywords — Vapi VAPI allows you to improve the accuracy of your transcriptions by leveraging Deepgram’s keyword boosting feature. This is particularly useful when dealing with specialized terminology or uncommon proper nouns. By providing specific keywords to the Deepgram model, you can enhance transcription quality directly through VAPI. ### Why Use Keyword Boosting? Keyword boosting is beneficial for: * Enhancing the recognition of specialized terms and proper nouns. * Improving transcription accuracy without the need for a custom-trained model. * Quickly updating the model’s vocabulary with new or uncommon words. ### Important Notes * Keywords should be uncommon words or proper nouns not frequently recognized by the model. * Custom model training is the most effective way to ensure accurate keyword recognition. * For more than 50 keywords, consider custom model training by contacting Deepgram. Enabling Keyword Boosting in VAPI --------------------------------- ### API Call Integration To enable keyword boosting, you need to add a `keywords` parameter to your VAPI assistant’s transcriber section. This parameter should include the keywords and their respective intensifiers. ### Example of POST Request To create an assistant with keyword boosting enabled, you can make the following POST request to VAPI: ` | | | | --- | --- | | $ | bashCopy code | | > | curl \ | | > | --request POST \ | | > | --header 'Authorization: Bearer ' \ | | > | --header 'Content-Type: application/json' \ | | > | --data '{ | | > | "name": "Emma", | | > | "model": { | | > | "model": "gpt-4o", | | > | "provider": "openai" | | > | }, | | > | "voice": { | | > | "voiceId": "emma", | | > | "provider": "azure" | | > | }, | | > | "transcriber": { | | > | "provider": "deepgram", | | > | "model": "nova-2", | | > | "language": "bg", | | > | "smartFormat": true, | | > | "keywords": [ | | > | "snuffleupagus:1" | | > | ] | | > | }, | | > | "firstMessage": "Hi, I am Emma, what is your name?", | | > | "firstMessageMode": "assistant-speaks-first" | | > | }' \ | | > | https://api.vapi.ai/assistant | ` In this configuration: * **name**: The name of the assistant. * **model**: Specifies the model and provider for the assistant’s conversational capabilities. * **voice**: Specifies the voice and provider for the assistant’s speech. * **transcriber**: Specifies Deepgram as the transcription provider, along with the model, language, smart formatting, and keywords for boosting. * **firstMessage**: The initial message the assistant will speak. * **firstMessageMode**: Specifies that the assistant speaks first. ### Intensifiers Intensifiers are exponential factors that boost or suppress the likelihood of the specified keyword being recognized. The default intensifier is `1`. Higher values increase the likelihood, while `0` is equivalent to not specifying a keyword. * **Boosting Example:** `keywords=snuffleupagus:5` * **Suppressing Example:** `keywords=kansas:-10` ### Best Practices for Keyword Boosting 1. **Send Uncommon Keywords:** Focus on keywords not successfully transcribed by the model. 2. **Send Keywords Once:** Avoid repeating keywords. 3. **Use Individual Keywords:** Prefer individual terms over phrases. 4. **Use Proper Spelling:** Spell proper nouns as you want them to appear in transcripts. 5. **Moderate Intensifiers:** Start with small increments to avoid false positives. 6. **Custom Model Training:** For extensive vocabulary needs, consider custom model training. ### Additional Resources For more detailed information on Deepgram’s keyword boosting feature, refer to the Deepgram Keyword Boosting Documentation. By following these guidelines, you can effectively utilize Deepgram’s keyword boosting feature within your VAPI assistant, ensuring enhanced transcription accuracy for specialized terminology and uncommon proper nouns. [Built with](https://buildwithfern.com/?utm_campaign=buildWith&utm_medium=docs&utm_source=docs.vapi.ai) --- # Persistent Assistants — Vapi You might be wondering whether or not you should create an assistant using the `/assistant` endpoint with its `assistantId`. Or, can you just specify the assistant configuration when starting a call? The `/assistant` endpoint is there for convenience to save you creating your own assistants table. ###### Use cases * You won’t be adding more assistant properties on top of ours. * You want to use the same assistant across multiple calls. Otherwise, you can just specify the assistant configuration when starting a call. [Built with](https://buildwithfern.com/?utm_campaign=buildWith&utm_medium=docs&utm_source=docs.vapi.ai) --- # Function Calling — Vapi Vapi voice assistants are given three additional functions: `transferCall`,`endCall`, and `dialKeypad`. These functions can be used to transfer calls, hang up calls, and enter digits on the keypad. You **do not** need to add these functions to your model’s `functions` array. #### Transfer Call When a `forwardingPhoneNumber` is present on an assistant, the assistant will be given a `transferCall` function. This function can be used to transfer the call to the `forwardingPhoneNumber`. ` | | | | --- | --- | | 1 | { | | 2 | "model": { | | 3 | "provider": "openai", | | 4 | "model": "gpt-3.5-turbo", | | 5 | "messages": [ | | 6 | { | | 7 | "role": "system", | | 8 | "content": "You are an assistant at a law firm. When the user asks to be transferred, use the transferCall function." | | 9 | } | | 10 | ] | | 11 | }, | | 12 | "forwardingPhoneNumber": "+16054440129" | | 13 | } | ` #### End Call This function is provided when `endCallFunctionEnabled` is enabled on the assistant. The assistant can use this function to end the call. ` | | | | --- | --- | | 1 | { | | 2 | "model": { | | 3 | "provider": "openai", | | 4 | "model": "gpt-3.5-turbo", | | 5 | "messages": [ | | 6 | { | | 7 | "role": "system", | | 8 | "content": "You are an assistant at a law firm. If the user is being mean, use the endCall function." | | 9 | } | | 10 | ] | | 11 | }, | | 12 | "endCallFunctionEnabled": true | | 13 | } | ` #### Dial Keypad This function is provided when `dialKeypadFunctionEnabled` is enabled on the assistant. The assistant will be able to enter digits on the keypad. ` | | | | --- | --- | | 1 | { | | 2 | "model": { | | 3 | "provider": "openai", | | 4 | "model": "gpt-3.5-turbo", | | 5 | "messages": [ | | 6 | { | | 7 | "role": "system", | | 8 | "content": "You are an assistant at a law firm. When you hit a menu, use the dialKeypad function to enter the digits." | | 9 | } | | 10 | ] | | 11 | }, | | 12 | "dialKeypadFunctionEnabled": true | | 13 | } | ` ### Custom Functions The **Custom Functions** feature is being deprecated in favor of [Tools](/tools-calling) . Please refer to the **Tools** section instead. We’re working on a solution to migrate your existing functions over to make this a seamless transtion. In addition to the predefined functions, you can also define custom functions. These functions are similar to OpenAI functions and your chosen LLM will trigger them as needed based on your instructions. The functions array in the assistant definition allows you to define custom functions that the assistant can call during a conversation. Each function is an object with the following properties: * `name`: The name of the function. It must be a string containing a-z, A-Z, 0-9, underscores, or dashes, with a maximum length of 64. * `description`: A brief description of what the function does. This is used by the AI to decide when and how to call the function. * `parameters`: An object that describes the parameters the function accepts. The type property should be “object”, and the properties property should be an object where each key is a parameter name and each value is an object describing the type and purpose of the parameter. Here’s an example of a function definition: ` | | | | --- | --- | | 1 | { | | 2 | "functions": [ | | 3 | { | | 4 | "name": "bookAppointment", | | 5 | "description": "Used to book the appointment.", | | 6 | "parameters": { | | 7 | "type": "object", | | 8 | "properties": { | | 9 | "datetime": { | | 10 | "type": "string", | | 11 | "description": "The date and time of the appointment in ISO format." | | 12 | } | | 13 | } | | 14 | } | | 15 | } | | 16 | ] | | 17 | } | ` In this example, the bookAppointment function accepts one parameter, `datetime`, which is a string representing the date and time of the appointment in ISO format. In addition to defining custom functions, you can specify a `serverUrl` where Vapi will send the function call information. This URL can be configured at the account level or at the assistant level. At the account level, the `serverUrl` is set in the Vapi Dashboard. All assistants under the account will use this URL by default for function calls. At the assistant level, the `serverUrl` can be specified in the assistant configuration when creating or updating an assistant. This allows different assistants to use different URLs for function calls. If a `serverUrl` is specified at the assistant level, it will override the account-level Server URL. If the `serverUrl` is not defined either at the account level or the assistant level, the function call will simply be added to the chat history. This can be particularly useful when you want a function call to trigger an action on the frontend. For instance, the frontend can listen for specific function calls in the chat history and respond by updating the user interface or performing other actions. This allows for a dynamic and interactive user experience, where the frontend can react to changes in the conversation in real time. [Built with](https://buildwithfern.com/?utm_campaign=buildWith&utm_medium=docs&utm_source=docs.vapi.ai) --- # Dynamic Variables — Vapi Prompts, messages, and other assistant properties can be dynamically set when starting a call based on templates. These templates are defined using double curly braces `{{variableName}}`. This is useful when you want to customize the assistant for a specific call. For example, you could set the assistant’s first message to “Hello, `{{name}}`!” and then set `name` to `John` when starting the call by passing `assistantOverrides` with `variableValues` to the API or SDK: ` | | | | --- | --- | | 1 | { | | 2 | "variableValues": { | | 3 | "name": "John" | | 4 | } | | 5 | } | ` Utilizing Dynamic Variables in Phone Calls ------------------------------------------ To leverage dynamic variables during phone calls, follow these steps: 1. **Prepare Your Request:** Construct a JSON payload containing the following key-value pairs: * `assistantId`: Replace `"your-assistant-id"` with the actual ID of your assistant. * `assistantOverride`: This object is used to customize your assistant’s behavior. * `variableValues`: An object containing the dynamic variables you want to use, in the format `{ "variableName": "variableValue" }`. For example, `{ "name": "John" }`. * `customer`: An object representing the call recipient. * `number`: Replace `"+1xxxxxxxxxx"` with the phone number you wish to call (in E.164 format). * `phoneNumberId`: Replace `"your-phone-id"` with the ID of your registered phone number. You can get it from the [Phone number](https://dashboard.vapi.ai/phone-numbers) in the dashboard. 2. **Send the Request:** Dispatch the JSON payload to the `/call/phone` endpoint using your preferred method (e.g., HTTP POST request). ` | | | | --- | --- | | 1 | { | | 2 | "assistantId": "your-assistant-id", | | 3 | "assistantOverrides": { | | 4 | "variableValues": { | | 5 | "name": "John" | | 6 | } | | 7 | }, | | 8 | "customer": { | | 9 | "number": "+1xxxxxxxxxx" | | 10 | }, | | 11 | "phoneNumberId": "your-phone-id" | | 12 | } | ` **Note:** You will need to add the `{{variableName}}` in this format in all your prompts, whether it is the first message or anywhere else you want to use it. Default Variables ----------------- By default, the following variables are automatically filled based on the current (UTC) time, meaning that you don’t need to set them manually in `variableValues`: | Variable | Description | Example | | --- | --- | --- | | `{{now}}` | Current date and time (UTC) | Jan 1, 2024 12:00 PM | | `{{date}}` | Current date (UTC) | Jan 1, 2024 | | `{{time}}` | Current time (UTC) | 12:00 PM | | `{{month}}` | Current month (UTC) | January | | `{{day}}` | Current day of month (UTC) | 1 | | `{{year}}` | Current year (UTC) | 2024 | | `{{customer.number}}` | Customer’s phone number | +1xxxxxxxxxx | | `{{customer.X}}` | Any other customer property | | Advanced Date and Time Usage ---------------------------- We use [LiquidJS](https://liquidjs.com/) for dynamic variables. You can use the `date` filter to format the date and time in the timezone you want. ` | | | | --- | --- | | 1 | {{"now" \| date: "%b %d, %Y, %I:%M %p", "America/New_York"}} | ` This should return the current date and time in New York. [Built with](https://buildwithfern.com/?utm_campaign=buildWith&utm_medium=docs&utm_source=docs.vapi.ai) --- # Speech Configuration — Vapi The Speaking Plan and Stop Speaking Plan are essential configurations designed to optimize the timing of when the assistant begins and stops speaking during interactions with a customer. These plans ensure that the assistant does not interrupt the customer and also prevents awkward pauses that can occur if the assistant starts speaking too late. Adjusting these parameters helps tailor the assistant’s responsiveness to different conversational dynamics. **Note**: At the moment these configurations can currently only be made via API. Start Speaking Plan ------------------- This plan defines the parameters for when the assistant begins speaking after the customer pauses or finishes. * **Wait Time Before Speaking**: You can set how long the assistant waits before speaking after the customer finishes. The default is 0.4 seconds, but you can increase it if the assistant is speaking too soon, or decrease it if there’s too much delay. **Example:** For tech support calls, set `waitSeconds` for the assistant to more than 1.0 seconds to give customers time to complete their thoughts, even if they have some pauses in between. * **Smart Endpointing**: This feature uses advanced processing to detect when the customer has truly finished speaking, especially if they pause mid-thought. It’s off by default but can be turned on if needed. **Example:** In insurance claims, `smartendpointingEnabled` helps avoid interruptions while customers think through responses and as they formulate responses. Example: The assistant mentions “do you want a loan,” triggering the system to check the customer’s response. If the customer responds with “yes” (matching the CustomerRegex for “yes|no”), the system waits for 1.1 seconds before proceeding, allowing time for further clarification. For responses requiring number sequences like “What’s your account number?”, set longer timeouts like 5 seconds or more to accommodate pauses between digits. * **Transcription-Based Detection**: Customize how the assistant determines that the customer has stopped speaking based on what they’re saying. This offers more control over the timing. **Example:** When a customer says, “My account number is 123456789, I want to transfer $500.” * The system detects the number “123456789” and waits for 0.5 seconds (`WaitSeconds`) to ensure the customer isn’t still speaking. * If the customer were to finish with an additional line, “I want to transfer $500.”, the system uses `onPunctuationSeconds` to confirm the end of the speech and then proceed with the request processing. * In a scenario where the customer has been silent for a long and has already finished speaking but the transcriber is not confident to punctuate the transcription, `onNoPunctuationSeconds` is used for 1.5 seconds. Here’s a code snippet for Start Speaking Plan - `| | | | --- | --- | | 1 | "startSpeakingPlan": { | | 2 | "waitSeconds": 0.4, | | 3 | "smartEndpointingEnabled": false, | | 4 | "customEndpointingRules": [ | | 5 | { | | 6 | "type": "both", | | 7 | "assistantRegex": "customEndpointingRules", | | 8 | "customerRegex": "customEndpointingRules", | | 9 | "timeoutSeconds": 1.1 | | 10 | } | | 11 | ], | | 12 | "transcriptionEndpointingPlan": { | | 13 | "onPunctuationSeconds": 0.1, | | 14 | "onNoPunctuationSeconds": 1.5, | | 15 | "onNumberSeconds": 0.5 | | 16 | } | | 17 | } |` Stop Speaking Plan ------------------ The Stop Speaking Plan defines when the assistant stops talking after detecting customer speech. * **Words to Stop Speaking**: Define how many words the customer needs to say before the assistant stops talking. If you want immediate reaction, set this to 0. Increase it to avoid interruptions by brief acknowledgments like “okay” or “right”. **Example:** While setting an appointment with a clinic, set `numWords` to 2-3 seconds to allow customers to finish brief clarifications without triggering interruptions. * **Voice Activity Detection**: Adjust how long the customer needs to be speaking before the assistant stops. The default is 0.2 seconds, but you can tweak this to balance responsiveness and avoid false triggers. **Example:** For a banking call center, setting a higher `voiceSeconds` value ensures accuracy by reducing false positives. This avoids interruptions caused by background sounds, even if it slightly delays the detection of speech onset. This tradeoff is essential to ensure the assistant processes only correct and intended information. * **Pause Before Resuming**: Control how long the assistant waits before starting to talk again after being interrupted. The default is 1 second, but you can adjust it depending on how quickly the assistant should resume. **Example:** For quick queries (e.g., “What’s the total order value in my cart?”), set `backoffSeconds` to 1 second. Here’s a code snippet for Start Speaking Plan - ` | | | | --- | --- | | 1 | "stopSpeakingPlan": { | | 2 | "numWords": 0, | | 3 | "voiceSeconds": 0.2, | | 4 | "backoffSeconds": 1 | | 5 | } | ` Considerations for Configuration -------------------------------- * **Customer Style**: Think about whether the customer pauses mid-thought or provides continuous speech. Adjust wait times and enable smart endpointing as needed. * **Background Noise**: If there’s a lot of background noise, you may need to tweak the settings to avoid false triggers. Default for phone calls is ‘office’ and default for web calls is ‘off’. ` | | | | --- | --- | | 1 | "backgroundSound": "off", | ` * **Conversation Flow**: Aim for a balance where the assistant is responsive but not intrusive. Test different settings to find the best fit for your needs. [Built with](https://buildwithfern.com/?utm_campaign=buildWith&utm_medium=docs&utm_source=docs.vapi.ai) --- # Bring your own chunks/vectors from Trieve — Vapi Using Trieve with Vapi ====================== Vapi offers two ways to integrate with [Trieve](https://trieve.ai/) : 1. **Direct Integration**: Create and manage Trieve datasets directly through Vapi 2. **BYOK (Bring Your Own Key)**: Use your Trieve API key to import your existing Trieve datasets into Vapi Direct Integration with Trieve ------------------------------ When using Trieve directly through Vapi, you can create and manage datasets, but they’ll be tied to Vapi’s account. This approach offers: * Quick setup with minimal configuration * Basic dataset management through Vapi’s API * Limited customization options ### Setting up Direct Integration 1. Navigate to the [Vapi dashboard credentials page](https://dashboard.vapi.ai/keys) 2. Add your Trieve API key from [Trieve’s dashboard](https://dashboard.trieve.ai/org/keys) 3. Create a new knowledge base with Trieve as the provider: ` | | | | --- | --- | | 1 | { | | 2 | "name": "my-trieve-kb", | | 3 | "provider": "trieve", | | 4 | "searchPlan": { | | 5 | "scoreThreshold": 0.2, | | 6 | "searchType": "semantic" | | 7 | }, | | 8 | "createPlan": { | | 9 | "type": "create", | | 10 | "chunkPlans": [ | | 11 | { | | 12 | "fileIds": ["file-123", "file-456"], | | 13 | "websites": ["https://example.com"], | | 14 | "targetSplitsPerChunk": 50, | | 15 | "rebalanceChunks": true | | 16 | } | | 17 | ] | | 18 | } | | 19 | } | ` BYOK with Trieve (Recommended) ------------------------------ The BYOK approach offers more flexibility and control over your datasets. You can: * Fully manage your datasets in Trieve’s native interface * Use Trieve’s advanced features like: * Custom chunking rules * Search playground testing * Manual chunk editing * Website crawling * Dataset visualization ### Step 1: Set Up Trieve Dataset 1. Create an account at [Trieve](https://trieve.ai/) 2. Create a new dataset using Trieve’s dashboard ![Create dataset in Trieve](https://files.buildwithfern.com/https://vapi.docs.buildwithfern.com/2025-01-31T05:46:58.861Z/static/images/knowledge-base/create-dataset.png) When creating your dataset in Trieve, selecting the right embedding model is crucial for optimizing performance and accuracy. Here are some of the available options: ### jina-base-en * **Provider**: Jina AI (Hosted by Trieve) * **Performance**: Fast * **Description**: This model is designed for speed and efficiency, making it suitable for applications where quick response times are critical. It provides a good balance of performance and accuracy for general use cases. ### text-embedding-3-small * **Provider**: OpenAI * **Performance**: Moderate * **Description**: A smaller model from OpenAI that offers a compromise between speed and accuracy. It is suitable for applications that require a balance between computational efficiency and the quality of embeddings. ### text-embedding-3-large * **Provider**: OpenAI * **Performance**: Slow * **Description**: This larger model provides the highest accuracy among the options but at the cost of slower processing times. It is ideal for applications where the quality of embeddings is prioritized over speed. 3. Add content through various methods: #### Upload Documents Upload documents directly through Trieve’s interface: ![Upload files in Trieve](https://files.buildwithfern.com/https://vapi.docs.buildwithfern.com/2025-01-31T05:46:58.861Z/static/images/knowledge-base/upload-files.png) When uploading files, you can configure advanced chunking options: ![Upload files advanced options in Trieve](https://files.buildwithfern.com/https://vapi.docs.buildwithfern.com/2025-01-31T05:46:58.861Z/static/images/knowledge-base/upload-files-advanced.png) #### Edit Individual Chunks After uploading documents, you can edit individual chunks to refine their content: ![Edit chunk interface in Trieve](https://files.buildwithfern.com/https://vapi.docs.buildwithfern.com/2025-01-31T05:46:58.861Z/static/images/knowledge-base/edit-chunk.png) ##### Editing Options * **Chunk Content**: Modify the text directly in the rich text editor * Fix formatting issues * Correct errors or typos * Split or combine chunks manually * Add or remove content * **Metadata Fields**: * Date: Update document timestamps * Number Value: Adjust numeric metadata for filtering * Location: Set or modify geographical coordinates * Weight: Fine-tune search relevance with custom weights * Fulltext Boost: Add terms to enhance search visibility * Semantic Boost: Adjust vector embedding influence ##### Best Practices for Chunk Editing 1. **Content Length** * Keep chunks between 200-1000 tokens * Maintain logical content boundaries * Ensure complete thoughts within each chunk 2. **Metadata Optimization** * Use consistent date formats * Add relevant numeric values for filtering * Apply weights strategically for important content 3. **Search Enhancement** * Use boost terms for critical keywords * Balance semantic and fulltext boosts * Test search results after significant edits ### Advanced Chunking Options #### Metadata * Add custom metadata as JSON to associate with your chunks * Useful for filtering and organizing content (e.g., `{"author": "John Doe", "category": "technical"}`) * Keep metadata concise and relevant to avoid storage overhead * Use consistent keys across related documents for better searchability #### Date * Specify the creation or relevant date for the document * Important for version control and content freshness * Helps with filtering outdated information * Use actual document creation dates when possible #### Split Delimiters * Define custom delimiters (e.g., ”.,?\\n”) to control where chunks are split * Recommended defaults: ”.,?\\n” for general content * Add semicolons (;) for technical documentation * Use “\\n\\n” for markdown or structured content * Avoid over-aggressive splitting that might break context #### Target Splits Per Chunk * Set the desired number of splits per chunk * Default: 20 splits * Recommended ranges: * 15-25 for general content * 10-15 for technical documentation * 25-30 for narrative content * Lower values create more granular chunks, better for precise retrieval * Higher values maintain more context but may retrieve irrelevant information #### Rebalance Chunks * Enable to redistribute content evenly across chunks * Recommended for documents with varying section lengths * Helps maintain consistent chunk sizes * May slightly impact natural content boundaries * Best used with technical documentation or structured content #### Use gpt4o chunking * Enable GPT-4 optimized chunking for improved semantic coherence * Recommended for: * Complex technical documentation * Content with intricate relationships * Documents where context preservation is crucial * Note: Increases processing time and cost * Best for high-value content where accuracy is paramount #### Heading Based Chunking * Split content based on document headings * Ideal for well-structured documents (e.g., documentation, reports) * Works best with consistent heading hierarchy * Consider enabling for: * Technical documentation * User manuals * Research papers * May create uneven chunk sizes based on section lengths #### System Prompt * Provide custom instructions for the chunking process * Optional but powerful for specific use cases * Example prompts: * “Preserve code blocks as single chunks" * "Keep API endpoint descriptions together" * "Maintain question-answer pairs in the same chunk” * Keep prompts clear and specific * Test different prompts with sample content to optimize results #### Website Crawling Trieve offers powerful website crawling capabilities with extensive configuration options: ![Website crawling in Trieve](https://files.buildwithfern.com/https://vapi.docs.buildwithfern.com/2025-01-31T05:46:58.861Z/static/images/knowledge-base/crawl.png) ##### Crawl Configuration Options * **Crawl Interval**: Set how often to refresh content * Options: Daily, Weekly, Monthly * Recommended: Daily for frequently updated content * **Page Limit**: Control the maximum number of pages to crawl * Default: 1000 pages * Adjust based on your site size and content relevance * **URL Patterns** * Include/Exclude specific URL patterns using regex * Example includes: `https://docs.example.com/*` * Example excludes: `https://example.com/internal/*` * **Query Selectors** * Include specific HTML elements for targeted content extraction * Exclude navigation, footers, and other non-content elements * Common excludes: `navbar`, `footer`, `aside`, `nav`, `form` * **Special Content Types** * OpenAPI Spec: Toggle for API documentation crawling * Shopify: Enable for e-commerce content * YouTube Channel: Include video transcripts and descriptions * **Advanced Options** * Boost Titles: Increase weight of page titles in search results * Allow External Links: Include content from linked domains * Ignore Sitemap: Skip sitemap-based crawling * Remove Strings: Clean up headers and body content ##### Best Practices for Crawling 1. **Start Small** * Begin with a low page limit * Test with specific sections of your site * Gradually expand coverage 2. **Optimize Selectors** * Remove navigation and UI elements * Focus on main content areas * Use browser inspector to identify key selectors 3. **Monitor Performance** * Check crawl logs regularly * Adjust patterns based on results * Balance frequency with server load ### Step 2: Test and Refine Use Trieve’s search playground to: * Test semantic search queries * Adjust chunk sizes * Edit chunks manually * Visualize vector embeddings * Fine-tune relevance scores ![Search playground in Trieve](https://files.buildwithfern.com/https://vapi.docs.buildwithfern.com/2025-01-31T05:46:58.861Z/static/images/knowledge-base/search-playground.png) ### Step 3: Import to Vapi 1. Create your Trieve API key from [Trieve’s dashboard](https://dashboard.trieve.ai/org/keys) 2. Add your Trieve API key to Vapi [Provider Credentials](https://dashboard.vapi.ai/keys) ![Add Trieve API key in Vapi](https://files.buildwithfern.com/https://vapi.docs.buildwithfern.com/2025-01-31T05:46:58.861Z/static/images/knowledge-base/trieve-credential.png) 3. Once your dataset is optimized in Trieve, import it to Vapi: ` | | | | --- | --- | | 1 | { | | 2 | "name": "trieve-byok", | | 3 | "provider": "trieve", | | 4 | "searchPlan": { | | 5 | "scoreThreshold": 0.2, | | 6 | "searchType": "semantic" | | 7 | }, | | 8 | "createPlan": { | | 9 | "type": "import", | | 10 | "providerId": "" | | 11 | } | | 12 | } | ` Best Practices -------------- 1. **Dataset Organization** * Segment datasets by domain knowledge boundaries * Use semantic-based dataset naming (e.g., “api-docs-v2”, “user-guides-2024”) * Version control chunking configurations in your codebase 2. **Content Quality** * Implement text normalization (Unicode normalization, whitespace standardization) * Use regex patterns to clean formatting artifacts * Validate chunk semantic coherence through embedding similarity scores 3. **Performance Optimization** * Target chunk sizes: 200-1000 tokens (optimal for current embedding models) * Configure hybrid search with BM25 boost = 0.3 for technical content * Set score thresholds dynamically based on embedding model (0.2 for text-embedding-3-small, 0.25 for text-embedding-3-large) 4. **Maintenance** * Implement automated content refresh cycles via Trieve’s API * Track search result relevance metrics (MRR, NDCG) * Rotate API keys on 90-day cycles Troubleshooting --------------- Common issues and solutions: 1. **Search Relevance Issues** * Implement cross-encoder reranking for critical queries * Fine-tune BM25 vs semantic weights (recommended ratio: 0.3:0.7) * Analyze chunk boundary overlap percentage (aim for 15-20%) 2. **Integration Errors** * Validate dataset permissions (READ\_DATASET scope required) * Check for dataset ID format compliance (UUID v4) * Monitor rate limits (default: 100 requests/min) 3. **Performance Optimization** * Implement chunk size normalization (max variance: 20%) * Enable query caching for frequent searches * Use batch operations for bulk updates (max 100 chunks/request) Need help? Contact [\[email protected\]](/cdn-cgi/l/email-protection#6c1f191c1c031e182c1a0d1c05420d05) for assistance. [Built with](https://buildwithfern.com/?utm_campaign=buildWith&utm_medium=docs&utm_source=docs.vapi.ai) --- # Call Analysis — Vapi The Call Analysis feature allows you to summarize and evaluate calls, providing valuable insights into their effectiveness. This feature uses a combination of prompts and schemas to generate structured data and success evaluations based on the call’s content. The underlying models driving our call analysis pipeline are the latest version of Anthropic’s Claude Sonnet and in case of failure OpenAI’s GPT 4o. You can customize the below in the assistant’s `assistant.analysisPlan`. Summary Prompt -------------- The summary prompt is used to create a concise summary of the call. This summary is stored in `call.analysis.summary`. ### Default Summary Prompt The default summary prompt is: ` You are an expert note-taker. You will be given a transcript of a call. Summarize the call in 2-3 sentences, if applicable. ` ### Customizing the Summary Prompt You can customize the summary prompt by setting the `summaryPrompt` property in the API or SDK: ` | | | | --- | --- | | 1 | { | | 2 | "summaryPrompt": "Custom summary prompt text" | | 3 | } | ` To disable the summary prompt, set it to an empty string `""` or `"off"`: ` | | | | --- | --- | | 1 | { | | 2 | "summaryPrompt": "" | | 3 | } | ` Structured Data Prompt ---------------------- The structured data prompt extracts specific pieces of data from the call. This data is stored in `call.analysis.structuredData`. ### Default Structured Data Prompt The default structured data prompt is: ` You are an expert data extractor. You will be given a transcript of a call. Extract structured data per the JSON Schema. ` ### Customizing the Structured Data Prompt You can set a custom structured data prompt using the `structuredDataPrompt` property: ` | | | | --- | --- | | 1 | { | | 2 | "structuredDataPrompt": "Custom structured data prompt text" | | 3 | } | ` Structured Data Schema ---------------------- The structured data schema enforces the format of the extracted data. It is defined using JSON Schema standards. ### Customizing the Structured Data Schema You can set a custom structured data schema using the `structuredDataSchema` property: ` | | | | --- | --- | | 1 | { | | 2 | "structuredDataSchema": { | | 3 | "type": "object", | | 4 | "properties": { | | 5 | "field1": { "type": "string" }, | | 6 | "field2": { "type": "number" } | | 7 | }, | | 8 | "required": ["field1", "field2"] | | 9 | } | | 10 | } | ` Success Evaluation Prompt ------------------------- The success evaluation prompt is used to determine if the call was successful. This evaluation is stored in `call.analysis.successEvaluation`. ### Default Success Evaluation Prompt The default success evaluation prompt is: ` You are an expert call evaluator. You will be given a transcript of a call and the system prompt of the AI participant. Determine if the call was successful based on the objectives inferred from the system prompt. ` ### Customizing the Success Evaluation Prompt You can set a custom success evaluation prompt using the `successEvaluationPrompt` property: ` | | | | --- | --- | | 1 | { | | 2 | "successEvaluationPrompt": "Custom success evaluation prompt text" | | 3 | } | ` To disable the success evaluation prompt, set it to an empty string `""` or `"off"`: ` | | | | --- | --- | | 1 | { | | 2 | "successEvaluationPrompt": "" | | 3 | } | ` Success Evaluation Rubric ------------------------- The success evaluation rubric defines the criteria used to evaluate the call’s success. The available rubrics are: * `NumericScale`: A scale of 1 to 10. * `DescriptiveScale`: A scale of Excellent, Good, Fair, Poor. * `Checklist`: A checklist of criteria and their status. * `Matrix`: A grid that evaluates multiple criteria across different performance levels. * `PercentageScale`: A scale of 0% to 100%. * `LikertScale`: A scale of Strongly Agree, Agree, Neutral, Disagree, Strongly Disagree. * `AutomaticRubric`: Automatically break down evaluation into several criteria, each with its own score. * `PassFail`: A simple ‘true’ if the call passed, ‘false’ if not. ### Customizing the Success Evaluation Rubric You can set a custom success evaluation rubric using the `successEvaluationRubric` property: ` | | | | --- | --- | | 1 | { | | 2 | "successEvaluationRubric": "NumericScale" | | 3 | } | ` Combining Prompts and Rubrics ----------------------------- You can use prompts and rubrics in combination to create detailed instructions for the call analysis: ` | | | | --- | --- | | 1 | { | | 2 | "successEvaluationPrompt": "Evaluate the call based on these criteria:...", | | 3 | "successEvaluationRubric": "Checklist" | | 4 | } | ` By customizing these properties, you can tailor the call analysis to meet your specific needs and gain valuable insights from your calls. [Built with](https://buildwithfern.com/?utm_campaign=buildWith&utm_medium=docs&utm_source=docs.vapi.ai) --- # Introduction — Vapi We’re currently running a beta for **Blocks**, an upcoming feature from [Vapi.ai](http://vapi.ai/) aimed at improving bot conversations. The problem we’ve noticed is that single LLM prompts are prone to hallucinations, unreliable tool calls, and can’t handle many-step complex instructions. **By breaking the conversation into smaller, more manageable prompts**, we can guarantee the bot will do this, then that, or if this happens, then that happens. It’s like having a checklist for conversations — less room for error, more room for getting things right. Here’s an example: For food ordering, this is what a prompt would look like. ###### Without Blocks Example Prompt ` | | | | --- | --- | | 1 | [Identity] | | 2 | You are a friendly and efficient assistant for a food truck that serves burgers, fries, and drinks. | | 3 | | | 4 | [Task] | | 5 | 1. Greet the customer warmly and inquire about their main order. | | 6 | 2. Offer suggestions for the main order if needed. | | 7 | 3. If they choose a burger, suggest upgrading to a combo with fries and a drink, offering clear options (e.g., regular or special fries, different drink choices). | | 8 | 4. Confirm the entire order to ensure accuracy. | | 9 | 5. Suggest any additional items like desserts or sauces. | | 10 | 6. Thank the customer and let them know when their order will be ready. | ` ###### With Blocks ![](https://files.buildwithfern.com/https://vapi.docs.buildwithfern.com/2025-01-31T05:46:58.861Z/static/images/blocks/food-order-steps.png) There are three core types of Blocks: [Conversation](https://api.vapi.ai/api#:~:text=ConversationBlock) , [Tool-call](https://api.vapi.ai/api#:~:text=ToolCallBlock) , and [Workflow](https://api.vapi.ai/api#:~:text=WorkflowBlock) . Each type serves a different role in shaping how your assistant engages with users. Blocks is currently in beta. We’re excited to have you try this new feature and welcome your [feedback](https://discord.com/invite/pUFNcf2WmH) as we continue to refine and improve the experience. Advanced Concepts ----------------- [Steps\ \ Learn how to structure the flow of your conversation](/blocks/steps) [Block Types\ \ Explore the different block types and how to use them](/blocks/block-types) [Built with](https://buildwithfern.com/?utm_campaign=buildWith&utm_medium=docs&utm_source=docs.vapi.ai) --- # Block Types — Vapi [**Blocks**](https://api.vapi.ai/api#/Blocks/BlockController_create) are the functional units within a Step, defining what action happens at each stage of a conversation. Each Step can contain only one Block, and there are three main types of Blocks, each designed to handle different aspects of conversation flow. Blocks is currently in beta. We’re excited to have you try this new feature and welcome your [feedback](https://discord.com/invite/pUFNcf2WmH) as we continue to refine and improve the experience. #### Types * [**Conversation:**](https://api.vapi.ai/api#:~:text=ConversationBlock) This block type manages interactions between the assistant and the user. A conversation block is used when the assistant needs to ask the user for specific information, such as contact details or preferences. * [**Tool-call:**](https://api.vapi.ai/api#:~:text=ToolCallBlock) This block allows the assistant to make external tool calls. * [**Workflow:**](https://api.vapi.ai/api#:~:text=WorkflowBlock) This block type enables the creation of subflows, which are smaller sets of steps executed within a Block. It can contain an array of steps (`steps[]`) and uses an `inputSchema` to define the data needed to initiate the workflow, along with an `outputSchema` to handle the data returned after completing the subflow. Workflow blocks are ideal for organizing complex processes or reusing workflows across different parts of the conversation. [Built with](https://buildwithfern.com/?utm_campaign=buildWith&utm_medium=docs&utm_source=docs.vapi.ai) --- # Phone Calling — Vapi ###### Set up a Phone Number You can set up a phone number to place and receive phone calls. Phone numbers can be bought directly through Vapi, or you can use your own from Twilio. You can buy a phone number through the dashboard or use the [`/phone-numbers/buy`](/api-reference/phone-numbers/buy-phone-number) \` endpoint. If you want to use your own phone number, you can also use the dashboard or the [`/phone-numbers/import`](/api-reference/phone-numbers/import-twilio-number) endpoint. This will use your Twilio credentials to verify the number and configure it with Vapi services. ###### Outbound Calls You can place an outbound call from one of your phone numbers using the [`/call/phone`](/api-reference/calls/create-phone-call) endpoint. If the system message will be different with every call, you can specify a temporary assistant in the `assistant` field. If you want to reuse an assistant, you can specify its ID in the `assistantId` field. ###### Inbound Calls You can provide an `assistantId` to a phone number and it will use that assistant when receiving inbound calls. You may want to specify the assistant based on the caller’s phone number. If a phone number doesn’t have an `assistantId`, Vapi will attempt to retrieve the assistant from your server using your [Server URL](/server-url#retrieving-assistants) . Video Tutorial on How to Import Numbers from Twilio for International Calls: [Built with](https://buildwithfern.com/?utm_campaign=buildWith&utm_medium=docs&utm_source=docs.vapi.ai) --- # Squads — Vapi Sometimes, complex workflows are easier to manage with multiple assistants. You can think of each assistant in a Squad as a leg of a conversation tree. For example, you might have one assistant for lead qualification, which transfers to another for booking an appointment if they’re qualified. Prior to Squads you would put all functionality in one assistant, but Squads were added to break up the complexity of larger prompts into smaller specialized assistants with specific tools and fewer goals. Squads enable calls to transfer assistants mid-conversation, while maintaining full conversation context. Usage ----- To use Squads, you can create a `squad` when starting a call and specify `members` as a list of assistants and destinations. The first member is the assistant that will start the call, and assistants can be either persistent or transient. Each assistant should be assigned the relevant assistant transfer destinations. Transfers are specified by assistant name and are used when the model recognizes a specific trigger. ` | | | | --- | --- | | 1 | { | | 2 | "squad": { | | 3 | "members": [ | | 4 | { | | 5 | "assistantId": "information-gathering-assistant-id", | | 6 | "assistantDestinations": [{ | | 7 | "type": "assistant", | | 8 | "assistantName": "Appointment Booking", | | 9 | "message": "Please hold on while I transfer you to our appointment booking assistant.", | | 10 | "description": "Transfer the user to the appointment booking assistant after they say their name." | | 11 | }], | | 12 | }, | | 13 | { | | 14 | "assistant": { | | 15 | "name": "Appointment Booking", | | 16 | ... | | 17 | }, | | 18 | } | | 19 | ] | | 20 | } | | 21 | } | ` Best practices -------------- The following are some best practices for using Squads to reduce errors: * Group assistants by closely related tasks * Create as few assistants as possible to reduce complexity * Make sure descriptions for transfers are clear and concise [Built with](https://buildwithfern.com/?utm_campaign=buildWith&utm_medium=docs&utm_source=docs.vapi.ai) --- # Configuring Inbound and Outbound Calls for Squads — Vapi This guide details how to set up and manage inbound and outbound call functionality within Squads, leveraging AI assistants. ### Key Concepts * **Transient Assistant:** A temporary assistant configuration passed directly in the request payload. * **Assistant ID:** A unique identifier referring to a pre-existing assistant configuration. When using Assistant IDs, ensure the `name` property in the payload matches the associated assistant’s name accurately. ### Inbound Call Configuration When your server receives a request of type `assistant-request`, respond with a JSON payload structured as follows: ` | | | | --- | --- | | 1 | { | | 2 | "squad": { | | 3 | "members": [ | | 4 | { | | 5 | "assistant": { | | 6 | "name": "Emma", | | 7 | "model": { "model": "gpt-4o", "provider": "openai" }, | | 8 | "voice": { "voiceId": "emma", "provider": "azure" }, | | 9 | "transcriber": { "provider": "deepgram" }, | | 10 | "firstMessage": "Hi, I am Emma, what is your name?", | | 11 | "firstMessageMode": "assistant-speaks-first" | | 12 | }, | | 13 | "assistantDestinations": [ | | 14 | { | | 15 | "type": "assistant", | | 16 | "assistantName": "Mary", | | 17 | "message": "Please hold on while I transfer you to our appointment booking assistant Mary.", | | 18 | "description": "Transfer the user to the appointment booking assistant." | | 19 | } | | 20 | ] | | 21 | }, | | 22 | { | | 23 | "assistantId": "your-assistant-id" | | 24 | } | | 25 | ] | | 26 | } | | 27 | } | ` **In this example:** * The first `members` entry is a **transient assistant** (full configuration provided). * The second `members` entry uses an **Assistant ID**. * `assistantDestinations` defines how to **transfer the call** to another assistant. ### Outbound Call Configuration To initiate an outbound call, send a POST request to the API endpoint /call/phone with a JSON payload structured as follows: ` | | | | --- | --- | | 1 | { | | 2 | "squad": { | | 3 | "members": [ | | 4 | { | | 5 | "assistant": { | | 6 | "name": "Emma", | | 7 | "model": { "model": "gpt-4o", "provider": "openai" }, | | 8 | "voice": { "voiceId": "emma", "provider": "azure" }, | | 9 | "transcriber": { "provider": "deepgram" }, | | 10 | "firstMessage": "Hi, I am Emma, what is your name?", | | 11 | "firstMessageMode": "assistant-speaks-first" | | 12 | }, | | 13 | "assistantDestinations": [ | | 14 | { | | 15 | "type": "assistant", | | 16 | "assistantName": "Mary", | | 17 | "message": "Please hold on while I transfer you to our appointment booking assistant Mary.", | | 18 | "description": "Transfer the user to the appointment booking assistant." | | 19 | } | | 20 | ] | | 21 | }, | | 22 | { | | 23 | "assistantId": "your-assistant-id" | | 24 | } | | 25 | ] | | 26 | }, | | 27 | "customer": { | | 28 | "number": "your-phone-number" | | 29 | }, | | 30 | "phoneNumberId": "your-phone-number-id" | | 31 | } | ` **Key points:** * `customer.number` is the phone number to call. * `phoneNumberId` is a unique identifier for the phone number (obtain this from your provider). [Built with](https://buildwithfern.com/?utm_campaign=buildWith&utm_medium=docs&utm_source=docs.vapi.ai) --- # Server URLs — Vapi ![](https://files.buildwithfern.com/https://vapi.docs.buildwithfern.com/2025-01-31T05:46:58.861Z/static/images/server-url/overview-graphic.png) Server URLs give Vapi a location to send real-time conversation data (as well as query for data Vapi needs). Server URLs allow your application to **receive data** & **communicate with Vapi** during conversations. Conversation events can include: * **Status Updates:** updates on the status of a call * **Transcript Updates**: call transcripts * **Function Calls:** payloads delivered when your assistant wants certain actions executed * **Assistant Requests:** in certain circumstances, Vapi may ping your server to get dynamic configuration for an assistant handling a specific call * **End of Call Report:** call summary data at the end of a call * **Hang Notifications:** get notified when your assistant fails to reply for a certain amount of time In our [quickstart guides](/quickstart) we learned how to setup a basic back-and-forth conversation with a Vapi assistant. To build more complex & custom applications, we’re going to need to get real-time conversation data to our backend. **This is where server URLs come in.** If you’re familiar with functional programming, Server URLs are like callback functions. But instead of specifying a function to get data back on, we specify a URL to a server (to POST data back to). Get Started ----------- To get started using server URLs, read our guides: [Setting Server URLs\ \ Server URLs can be set in multiple places. Learn where here.](/server-url/setting-server-urls) [Events\ \ Read about the different types of events Vapi can send to your server.](/server-url/events) [Developing Locally\ \ Learn about receiving server events in your local development environment.](/server-url/developing-locally) FAQ --- ###### Where can the server be located? The server URL can be any publicly accessible URL pointing to an HTTP endpoint. This can be a: * **Cloud Server:** your application might be deployed on a cloud platform like [Railway](https://railway.app/) , [AWS](https://aws.com/) , [GCP](https://cloud.google.com/gcp) , etc — as a persistent web server. * **Serverless Function:** services like [Vercel](https://vercel.com/docs/functions) , [AWS Lambda](https://aws.amazon.com/lambda/) , [Google Cloud Functions](https://cloud.google.com/functions) , [Cloudflare](https://developers.cloudflare.com/workers/) , etc — allow you to host on-demand cloud functions. * **Workflow Orchestrator:** platforms like [Pipedream](https://pipedream.com/) & [Make](https://www.make.com/) allow you to program workflows (often without code) that can receive events via HTTP triggers. The main idea is that Vapi needs a location on the Internet that it can drop data to & converse with your application. ###### Why not just call them webhooks? [Webhooks](/glossary#webhook) are traditionally unidirectional & stateless, with the target endpoint usually only replying with a status code to acknowledge message reception. Certain server URL events (like assistant requests) may require a meaningful reply from your server. ”Server URL” is a more general term that encompasses both webhooks & bidirectional communication. [Built with](https://buildwithfern.com/?utm_campaign=buildWith&utm_medium=docs&utm_source=docs.vapi.ai) --- # Steps — Vapi [**Steps**](https://api.vapi.ai/api#:~:text=HandoffStep) are the core building blocks that dictate how conversations progress in a bot interaction. Each Step represents a distinct point in the conversation where the bot performs an action, gathers information, or decides where to go next. Think of Steps as checkpoints in a conversation that guide the flow, manage user inputs, and determine outcomes. Blocks is currently in beta. We’re excited to have you try this new feature and welcome your [feedback](https://discord.com/invite/pUFNcf2WmH) as we continue to refine and improve the experience. #### Features * **Output:** The data or response expected from the step, as outlined in the block’s `outputSchema`. * **Input:** The data necessary for the step to execute, defined in the block’s `inputSchema`. * [**Destinations:**](https://api.vapi.ai/api#:~:text=StepDestination) This can be determined by a simple linear progression or based on specific criteria, like conditions or rules set within the Step. This enables dynamic decision-making, allowing the assistant to choose the next Step depending on what happens during the conversation (e.g., user input, a specific value, or a condition being met). #### Example ` | | | | --- | --- | | 1 | { | | 2 | "type": "handoff", | | 3 | "name": "get_user_order", | | 4 | "input": { | | 5 | "name": "John Doe", | | 6 | "email": "[[email protected]](/cdn-cgi/l/email-protection)
" | | 7 | }, | | 8 | "destinations": [ | | 9 | { | | 10 | "type": "step", | | 11 | "stepName": "confirm_order", | | 12 | "conditions": [ | | 13 | { | | 14 | "type": "model-based", | | 15 | "instruction": "If the user has provided an order" | | 16 | } | | 17 | ] | | 18 | } | | 19 | ], | | 20 | "block": { | | 21 | "name": "ask_for_order", | | 22 | "type": "conversation", | | 23 | "inputSchema": { | | 24 | "type": "object", | | 25 | "required": ["name", "email"], | | 26 | "properties": { | | 27 | "name": { "type": "string", "description": "The customer's name" }, | | 28 | "email": { "type": "string", "description": "The customer's email" } | | 29 | } | | 30 | }, | | 31 | "instruction": "Greet the customer and ask for their name and email. Then ask them what they'd like to order.", | | 32 | "outputSchema": { | | 33 | "type": "object", | | 34 | "required": ["orders", "name"], | | 35 | "properties": { | | 36 | "orders": { | | 37 | "type": "string", | | 38 | "description": "The customer's order, e.g., 'burger with fries'" | | 39 | }, | | 40 | "name": { | | 41 | "type": "string", | | 42 | "description": "The customer's name" | | 43 | } | | 44 | } | | 45 | } | | 46 | } | | 47 | } | ` [Built with](https://buildwithfern.com/?utm_campaign=buildWith&utm_medium=docs&utm_source=docs.vapi.ai) --- # Setting Server URLs — Vapi ![](https://files.buildwithfern.com/https://vapi.docs.buildwithfern.com/2025-01-31T05:46:58.861Z/static/images/server-url/settings-server-urls/server-url-priority.png) Server URLs can be set at multiple levels in Vapi. Server URLs can be set in multiple places in Vapi. Each level has a different priority. The server URL with the highest priority for a relevant event will be the one that Vapi uses to send the event to. Server URLs can be set at **4 levels** in Vapi: * **Account-wide:** you can set a server URL for your broader account * **Phone Number:** server URLs can be attached to phone numbers themselves * **Assistant:** assistants can be configured with a server URL * **Function:** function calls themselves (under an assistant) can have a corresponding server URL Setting Server URLs ------------------- Here’s a breakdown of where you can set server URLs in Vapi: ###### Organization You can set an organization-wide server URL in the [organization section](https://dashboard.vapi.ai/vapi-api) of your dashboard. ![](https://files.buildwithfern.com/https://vapi.docs.buildwithfern.com/2025-01-31T05:46:58.861Z/static/images/server-url/settings-server-urls/org-settings-server-urls.png) Setting your organization-wide server URL. If no other server URL is set, Vapi will use this one. ###### Phone Number Phone numbers can have a server URL attached to them via the [phone number API](/api-reference/phone-numbers) . The server URL for phone numbers can be set **3 ways**: * **At Time of Purchase:** when you [buy a number](/api-reference/phone-numbers/buy-phone-number) through Vapi * **At Import:** when you [import from Twilio](/api-reference/phone-numbers/import-twilio-number) or [Vonage](/api-reference/phone-numbers/import-vonage-number) * **Via Update:** you can [update a number](/api-reference/phone-numbers/update-phone-number) already in your account The field `phoneNumber.serverUrl` will contain the server URL for the phone number. ###### Assistant Assistants themselves can have a server URL attached to them. There are **2 ways** this can be done: ###### In the Dashboard If you go to the [assistant section](https://dashboard.vapi.ai/assistants) of your dashboard, in the **“Advanced”** tab you will see a setting to set the assistant’s server URL: ![](https://files.buildwithfern.com/https://vapi.docs.buildwithfern.com/2025-01-31T05:46:58.861Z/static/images/server-url/settings-server-urls/assistant-server-url-dashboard.png) Setting server URL at the assistant level. ###### Via the API At [assistant creation](/api-reference/assistants/create-assistant) (or via an [update](/api-reference/assistants/update-assistant) ) you can set the assistant’s server URL. The server URL for an assistant is stored in the `assistant.serverUrl` field. ###### Function Call The most granular level server URLs can be set is at the function call level. This can also be done either in the dashboard, or via code. ###### In the Dashboard In the [assistant section](https://dashboard.vapi.ai/assistants) of your dashboard, in the **“Functions”** tab you can add function calls & optionally give each a specific server URL: ![](https://files.buildwithfern.com/https://vapi.docs.buildwithfern.com/2025-01-31T05:46:58.861Z/static/images/server-url/settings-server-urls/function-call-server-url-dashboard.png) Setting server URL at the function call level. ###### Via the API The server URL for a function call can be found on an assistant at `assistant.model.functions[].serverUrl`. You can either set the URL for a function call at [assistant creation](/api-reference/assistants/create-assistant) , or in an [assistant update](/api-reference/assistants/update-assistant) . URL Priority ------------ Events are only sent/assigned to 1 server URL in the priority stack. Here’s the order of priority: 1. **Function:** if a function call has a server URL, the function call event will be sent to that URL 2. **Assistant:** assistant server URLs are the next highest priority 3. **Phone Number:** if a phone number has a server URL, it will be used over the account-wide URL 4. **Account-wide:** Default / “lowest” importance. It will be used if no other server URL is set. You will most commonly set a server URL on your account, and/or on specific assistants. [Built with](https://buildwithfern.com/?utm_campaign=buildWith&utm_medium=docs&utm_source=docs.vapi.ai) --- # Server Events — Vapi All messages sent to your Server URL will be `POST` requests with the following body: ` | | | | --- | --- | | 1 | { | | 2 | "message": { | | 3 | "type": "function-call", | | 4 | "call": { Call Object }, | | 5 | ...other message properties | | 6 | } | | 7 | } | ` They include the type of message, the call object, and any other properties that are relevant to the message type. Below are the different types of messages that can be sent to your Server URL. ### Function Calling Vapi fully supports [OpenAI’s function calling API](https://platform.openai.com/docs/guides/gpt/function-calling) , so you can have assistants ping your server to perform actions like sending emails, retrieve information, and more. With each response, the assistant will automatically determine what functions to call based on the directions provided in the system message in `messages`. Here’s an example of what the assistant might look like: ` | | | | --- | --- | | 1 | { | | 2 | "name": "Ryan's Assistant", | | 3 | "model": { | | 4 | "provider": "openai", | | 5 | "model": "gpt-3.5-turbo", | | 6 | "functions": [ | | 7 | { | | 8 | "name": "sendEmail", | | 9 | "description": "Used to send an email to a client.", | | 10 | "parameters": { | | 11 | "type": "object", | | 12 | "properties": { | | 13 | "color": { "type": "string" } | | 14 | } | | 15 | } | | 16 | } | | 17 | ] | | 18 | } | | 19 | } | ` Once a function is triggered, the assistant will send a message to your Server URL with the following body: ` | | | | --- | --- | | 1 | { | | 2 | "message": { | | 3 | "type": "function-call", | | 4 | "call": { Call Object }, | | 5 | "functionCall": { | | 6 | "name": "sendEmail", | | 7 | "parameters": "{ \"emailAddress\": \"[[email protected]](/cdn-cgi/l/email-protection)
\"}" | | 8 | } | | 9 | } | | 10 | } | ` Your server should respond with a JSON object containing the function’s response, like so: ` | | | | --- | --- | | 1 | { "result": "Your email has been sent." } | ` Or if it’s an object: ` | | | | --- | --- | | 1 | { | | 2 | "result": "{ \"message\": \"Your email has been sent.\", \"email\": \"[[email protected]](/cdn-cgi/l/email-protection)
\" }" | | 3 | } | ` The result will be appended to the conversation, and the assistant will decide what to do with the response based on its system prompt. If you don’t need to return a response, you can use the `async: true` parameter in your assitant’s function configuration. This will prevent the assistant from waiting for a response from your server. ### Retrieving Assistants For inbound phone calls, you may want to specify the assistant based on the caller’s phone number. If a PhoneNumber doesn’t have an `assistantId`, Vapi will attempt to retrieve the assistant from your server. ` | | | | --- | --- | | 1 | { | | 2 | "message": { | | 3 | "type": "assistant-request", | | 4 | "call": { Call Object }, | | 5 | } | | 6 | } | ` Your server should respond with a JSON object containing the assistant, like so: ` | | | | --- | --- | | 1 | { | | 2 | "assistant": { | | 3 | "firstMessage": "Hey Ryan, how are you?", | | 4 | "model": { | | 5 | "provider": "openai", | | 6 | "model": "gpt-3.5-turbo", | | 7 | "messages": [ | | 8 | { | | 9 | "role": "system", | | 10 | "content": "You're Ryan's assistant..." | | 11 | } | | 12 | ] | | 13 | } | | 14 | } | | 15 | } | ` If you’d like to play an error message instead, you can respond with: ` | | | | --- | --- | | 1 | { "error": "Sorry, not enough credits on your account, please refill." } | ` ### Call Status Updates During the call, the assistant will make multiple `POST` requests to the Server URL with the following body: ` | | | | --- | --- | | 1 | { | | 2 | "message": { | | 3 | "type": "status-update", | | 4 | "call": { Call Object }, | | 5 | "status": "ended", | | 6 | } | | 7 | } | ` Status Events * `in-progress`: The call has started. - `forwarding`: The call is about to be forwarded to `forwardingPhoneNumber`. - `ended`: The call has ended. ### End of Call Report When a call ends, the assistant will make a `POST` request to the Server URL with the following body: ` | | | | --- | --- | | 1 | { | | 2 | "message": { | | 3 | "type": "end-of-call-report", | | 4 | "endedReason": "hangup", | | 5 | "call": { Call Object }, | | 6 | "recordingUrl": "https://vapi-public.s3.amazonaws.com/recordings/1234.wav", | | 7 | "summary": "The user picked up the phone then asked about the weather...", | | 8 | "transcript": "AI: How can I help? User: What's the weather? ...", | | 9 | "messages":[ | | 10 | { | | 11 | "role": "assistant", | | 12 | "message": "How can I help?", | | 13 | }, | | 14 | { | | 15 | "role": "user", | | 16 | "message": "What's the weather?" | | 17 | }, | | 18 | ... | | 19 | ] | | 20 | } | | 21 | } | ` `endedReason` can be any of the options defined on the [Call Object](/api-reference/calls/get-call) . ### Hang Notifications Whenever the assistant fails to respond for 5+ seconds, the assistant will make a `POST` requests to the Server URL with the following body: ` | | | | --- | --- | | 1 | { | | 2 | "message": { | | 3 | "type": "hang", | | 4 | "call": { Call Object }, | | 5 | } | | 6 | } | ` You can use this to display an error message to the user, or to send a notification to your team. [Built with](https://buildwithfern.com/?utm_campaign=buildWith&utm_medium=docs&utm_source=docs.vapi.ai) ---