# Table of Contents - [Introduction](#introduction) - [API Key and Python Client](#api-key-and-python-client) - [Pricing](#pricing) - [Rate Limits](#rate-limits) - [Snowflake](#snowflake) - [Flexible Dimensions and Quantization](#flexible-dimensions-and-quantization) - [Organizations and Projects](#organizations-and-projects) - [Error Codes](#error-codes) - [Service Level Objectives](#service-level-objectives) - [Text Embeddings](#text-embeddings) - [Multimodal Embeddings](#multimodal-embeddings) - [Tokenization](#tokenization) - [Rerankers](#rerankers) - [AWS Marketplace Model Package](#aws-marketplace-model-package) - [Quickstart Tutorial](#quickstart-tutorial) - [Azure Marketplace Managed Application](#azure-marketplace-managed-application) - [Integrations](#integrations) - [Community SDKs](#community-sdks) - [Discord](#discord) - [FAQ](#faq) --- # Introduction Voyage AI provides cutting-edge embedding and rerankers. Embedding models are neural net models (e.g., transformers) that convert unstructured and complex data, such as documents, images, audios, videos, or tabular data, into dense numerical vectors (i.e. embeddings) that capture their semantic meanings. These vectors serve as representations/indices for datapoints and are essential building blocks for semantic search and retrieval-augmented generation (RAG), which is the predominant approach for domain-specific or company-specific chatbots and other AI applications. Rerankers are neural nets that output relevance scores between a query and multiple documents. It is common practice to use the relevance scores to rerank the documents initially retrieved with embedding-based methods (or with lexical search algorithms such as [BM25](https://en.wikipedia.org/wiki/Okapi_BM25) and [TF-IDF](https://en.wikipedia.org/wiki/Tf–idf) . Selecting the highest-scored documents refines the retrieval results into a more relevant subset. Voyage AI provides API endpoints for embedding and reranking models that take in your data (e.g., documents, queries, or query-document pairs) and return their embeddings or relevance scores. Embedding models and rerankers, as modular components, seamlessly integrate with other parts of a RAG stack, including vector stores and generative Large Language Models (LLMs). Voyage AI’s embedding models and rerankers are **state-of-the-art** in retrieval accuracy. Please read our announcing [blog post](https://blog.voyageai.com/2023/10/29/voyage-embeddings/) for details. Please also check out a high-level [introduction](https://www.pinecone.io/learn/retrieval-augmented-generation/) of embedding models, semantic search, and RAG, and our step-by-step [quickstart tutorial](/docs/quickstart-tutorial) on implementing a minimalist RAG chatbot using Voyage model endpoints. Updated 3 months ago * * * --- # API Key and Python Client Authentication with API Keys [](#authentication-with-api-keys) ------------------------------------------------------------------ Voyage AI utilizes API keys to monitor usage and manage permissions. To obtain your key, please sign in with your Voyage AI account and click the "Create new secret key" button in the [**API keys**](https://dashboard.voyageai.com/organization/api-keys) section of the Voyage [dashboard](https://dashboard.voyageai.com/organization/api-keys) . We recommend setting the API key as an environment variable. For example, in MacOS or Linux, type the following command in the terminal, replacing `` with your actual API key: Shell `export VOYAGE_API_KEY=""` You can verify the setup by typing `echo $VOYAGE_API_KEY` in the terminal. It should display your API key. Your API key is supposed to be secret -- please avoid sharing it or exposing it in browsers or apps. Please store your API key securely for future use. Install Voyage Python Package [](#install-voyage-python-package) -------------------------------------------------------------------- You can interact with the API through HTTP requests from any language. For Python users, we offer an official package which can be installed via `pip` : Shell `pip install -U voyageai` We recommend using the `-U` or `--upgrade` option to ensure you are installing the latest version of the package. This helps you access the most recent features and bug fixes. After installation, you can test it by running: Shell `python -c "import voyageai"` The installation is successful if this command runs without any errors. `voyageai.Client` [](#voyageaiclient) ----------------------------------------- The Python package offers the `voyageai.Client` class as the interface to invoke Voyage's API. You can create a client object and use it to access the predictions by our models. [`class voyageai.Client`](https://github.com/voyage-ai/voyageai-python/blob/main/voyageai/client.py) **Parameters** * **api\_key** (str, optional, defaults to `None`) - Voyage API key. If `None`, the client will search for the API key in the following order: * `voyageai.api_key_path`, path to the file containing the key; * environment variable `VOYAGE_API_KEY_PATH`, which can be set to the path to the file containing the key; * `voyageai.api_key`, an attribute of the `voyageai` module, which can be used to store the key; * environment variable `VOYAGE_API_KEY`. * **max\_retries** (int, defaults to 0) - Maximum number of retries for each API request in case of rate limit errors or temporary server unavailability. The client employs a wait-and-retry strategy to handle such errors, and will raise an exception upon reaching the maximum retry limit. By default, the client does not retry. * **timeout** (int, optional, defaults to `None`) - Maximum time in seconds to wait for a response from the API before aborting the request. If the specified timeout is exceeded, the request is terminated and a timeout exception is raised. By default, no timeout constraint is enforced. **Example** Python `import voyageai vo = voyageai.Client() # This will automatically use the environment variable VOYAGE_API_KEY. # Alternatively, you can use vo = voyageai.Client(api_key="") result = vo.embed(["hello world"], model="voyage-3")` `voyageai.AsyncClient` [](#voyageaiasyncclient) --------------------------------------------------- The Python package provides a `voyageai.AsyncClient` class designed for asynchronous API calls. This `AsyncClient` class mirrors the `Client` class in terms of method offerings and input/output specifications but is tailored for asynchronous operations, enabling non-blocking API requests. **Example** Python `import voyageai vo = voyageai.AsyncClient() # This will automatically use the environment variable VOYAGE_API_KEY. # Alternatively, you can use vo = voyageai.AsyncClient(api_key="") result = await vo.embed(["hello world"], model="voyage-3")` Updated about 2 months ago * * * --- # Pricing Voyage model pricing is usage-based, with charges billed to the account linked to the API key used for access. Text Embeddings [](#text-embeddings) ======================================== We charge for requests to the Voyage text embedding endpoint based on the number of tokens in the documents/queries. The first 200 million tokens for `voyage-3-large`, `voyage-3`, `voyage-3-lite`, and `voyage-code-3`, or the first 50 million tokens for `voyage-multilingual-2`, `voyage-finance-2`, `voyage-law-2`, and `voyage-code-2`, are free for every account. Subsequent usage is priced per token, as shown in the following table. | Model | Price per thousand tokens | Price per million tokens | Number of free tokens | | --- | --- | --- | --- | | `voyage-3-large` | $0.00018 | $0.18 | 200 million | | `voyage-3` | $0.00006 | $0.06 | 200 million | | `voyage-3-lite` | $0.00002 | $0.02 | 200 million | | `voyage-code-3` | $0.00018 | $0.18 | 200 million | | `voyage-finance-2`
`voyage-law-2`
`voyage-code-2` | $0.00012 | $0.12 | 50 million | Multimodal Embeddings [](#multimodal-embeddings) ==================================================== We charge for requests to the Voyage multimodal endpoint based on the number of tokens in the text and the number of pixels in images. The first 200M text tokens and 150B pixels for `voyage-multimodal-3` are free for every account. Subsequent usage is priced per token and per pixel, as shown in the following table. | Model | Price per million tokens | Price per billion pixels | Number of free tokens and pixels | | --- | --- | --- | --- | | `voyage-multimodal-3` | $0.12 | $0.60 | 200M text tokens and 150B pixels | An image with fewer than 50,000 pixels will be upscaled, processed, and charged as a 50,000-pixel image. Images containing over 2 million pixels will be downsampled and charged as 2 million-pixel images. Therefore, the minimum and maximum costs per image are $0.00003 and $0.0012, respectively. The following table provides examples of image pricing. | Image resolution | Number of pixels | Price per image | Price per thousand images | | --- | --- | --- | --- | | 200px x 200px | 40,000 | $0.00003 | $0.03 | | 1000px x 1000px | 1 million | $0.0006 | $0.60 | | 2000px x 2000px | 4 million | $0.0012 | $1.20 | | 4000px x 4000px | 16 million | $0.0012 | $1.20 | For example, the cost to vectorize a single input with 1000 text tokens ($0.00012) and two 4 million-pixel images (2 x $0.0012) would be $0.00252. Rerankers [](#rerankers) ============================ Pricing for the Voyage reranker endpoint is based on the total number of processed tokens, calculated as “(the number of query tokens × the number of documents) + sum of the number of tokens in all documents.” The first 200 million tokens for `rerank-2` and `rerank-2-lite` are free for each account. Subsequent usage is priced per token, as shown in the following table. | Model | Price per thousand tokens | Price per million tokens | Estimated price per request \* | Number of free tokens | | --- | --- | --- | --- | --- | | `rerank-2` | $0.00005 | $0.05 | $0.0025 | 200 million | | `rerank-2-lite` | $0.00002 | $0.02 | $0.001 | 200 million | \* This price estimate assumes each request includes 100 documents and that the sum of the number of tokens in the query and the number of tokens in each document is 500. * * * _Older models_ The following table shows the pricing for older models. Please note that we do not offer free tokens for them. | Model | Price per thousand tokens | Price per million tokens | Number of free tokens | | --- | --- | --- | --- | | `voyage-multilingual-2`
`voyage-large-2-instruct`
`voyage-large-2` | $0.00012 | $0.12 | 0 | | `voyage-01`
`voyage-lite-01`
`voyage-lite-01-instruct`
`voyage-02`
`voyage-lite-02-instruct`
`voyage-2` | $0.0001 | $0.1 | 0 | | `rerank-1` | $0.00005 | $0.05 | 0 | | `rerank-lite-1` | $0.00002 | $0.02 | 0 | * * * Fine-tuned models [](#fine-tuned-models) ============================================ For fine-tuned models with dedicated instances and support channels, please get in touch with our [sales team](/cdn-cgi/l/email-protection#1b7874756f7a786f5b6d74627a7c7e7a7235787476) . Updated 3 months ago * * * --- # Rate Limits Rate limits are restrictions that we impose on the number of times and tokens a user can access our API services within a specified period of time. | Models | Basic TPM | Basic RPM | | --- | --- | --- | | `voyage-3-large`
`voyage-code-3`
`voyage` 1&2 Series embedding models | 3M | 2000 | | `voyage-3` | 8M | 2000 | | `voyage-3-lite` | 16M | 2000 | | `voyage-multimodal-3` | 2M | 2000 | | `rerank-lite-1`, `rerank-2-lite` | 4M | 2000 | | `rerank-1`, `rerank-2` | 2M | 2000 | For example, the `voyage-3` API rate limits are set at **2000 RPM (requests per minute)** and **8M TPM (tokens per minute)**, which means that a user or client is allowed to make up to 2000 API requests and process at most 8M tokens within one minute. Please refer to [tokenization](/docs/tokenization) for the calculation of the number of tokens. Adhering to this limit ensures a balanced and efficient utilization of the API's resources, preventing excessive traffic that could impact the overall performance and accessibility of the service. ### Usage tiers [](#usage-tiers) You can view your organization’s rate limits in the organization [**Rate Limits**](https://dashboard.voyageai.com/organization/rate-limits) section of the Voyage dashboard. As your usage and spending of the Voyage API increase, we automatically graduate you to the next usage tier, raising rate limits across all models. The table below summarizes the qualification criteria for each usage tier. | Usage Tier | Qualification | Rate Limits | | --- | --- | --- | | 1 | Payment method added. | See table above. | | 2 | ≥ $100 paid | 2x Tier 1 | | 3 | ≥ $1000 paid | 3x Tier 1 | For example, at tier 2 and 3, the API rate limits for `voyage-3` would increase to 16M TPM / 4000 RPM and 24M TPM / 6000 RPM, respectively. If you are at Tier 3, and need a higher rate limit, please [request a rate limit increase](http://www.voyageai.com/request-rate-limit-increase) . **Note**: Usage tier qualification is based on **billed usage**, excluding free tokens for current models. Billing occurs monthly, and you can view your billing history in the Voyage [dashboard](https://dashboard.voyageai.com/organization/billing/history) . Purchasing usage credits count towards billed usage–see our [FAQ](/docs/faq#how-can-i-set-up-prepaid-billing) for details. Once you qualify for a tier, you will never be downgraded to a lower tier. ### Project rate limits [](#project-rate-limits) The rate limits above apply to your entire organization. Rate limits can also be set at the project level by the organization Admin. Project-level rate limits for each model can be any value less than or equal to the organization’s corresponding rate limit. For example, at usage tier 1, the `voyage-3` API rate limit for a project can be set to 2000 RPM (requests per minute) and 8M TPM (tokens per minute), or lower. _**Manage project rate limits**_ To manage project rate limits, select the desired project and navigate to the project [**Rate Limits**](https://dashboard.voyageai.com/rate-limits) section. Click the **Select Models** button in the upper-right corner. ![](https://files.readme.io/cb83e24e0b5c4df58ca303572ae8931d73cafe217221c6501ddc8cb31d66f913-project-rate-limits-1.png) To edit a model’s rate limit, click the pencil (Edit) icon in the model’s row. For example, in the screenshot below, we’ll be editing the `voyage-3` model. ![](https://files.readme.io/cc0da19c1a5b3f7e04ee6d7e34f0bb0c938c293d19ead086d1ca7b7d632b06e5-project-rate-limits-2.png) Enter the desired rate limits in the corresponding fields and click **Save** in the upper-right corner to apply your changes. ![](https://files.readme.io/1af34665c46f6ed118daca804acfff75db1dc05ab763a0f7e9275f8125c83bdc-project-rate-limits-3.png) Any project rate limits not directly inherited from the organization are summarized on the project’s [**Rate Limits**](https://dashboard.voyageai.com/rate-limits) section. ![](https://files.readme.io/20725a109dce5a6afbdc540ae9a566317648f38a46128a1afcc3bb9000278a9b-project-rate-limits-4.png) Why do we have rate limits? [](#why-do-we-have-rate-limits) =============================================================== Rate limits are commonly adopted in APIs, serving several vital purposes: 1. **Rate limits promote equitable access to the API for all users.** If one individual or organization generates an excessive volume of requests, it could potentially impede the API's performance for others. Through rate limiting, Voyage ensures that a larger number of users can utilize the API without encountering performance issues. 2. **Rate limits enable Voyage AI to effectively manage the workload on its infrastructure.** Sudden and substantial spikes in API requests could strain server resources and lead to performance degradation. By establishing rate limits, Voyage can effectively maintain a consistent and reliable experience for all users. 3. **They act as a safeguard against potential abuse or misuse of the API.** For instance, malicious actors might attempt to inundate the API with excessive requests to overload it or disrupt its services. By instituting rate limits, Voyage can thwart such nefarious activities. What happens if I exceed the rate limit? [](#what-happens-if-i-exceed-the-rate-limit) ========================================================================================= If you exceed the rate limit, you will receive an error message with the code [429](/docs/error-codes) . For ways to avoid hitting rate limits, see the section [below](/docs/rate-limits#how-can-i-avoid-hitting-rate-limits) . Your rate limits automatically increase as your Voyage API usage and spending grow, progressing you into higher [usage tiers](/docs/rate-limits#usage-tiers) . Tier 1 requires you add a payment method in the billing page for the appropriate organization in Voyage [dashboard](https://dashboard.voyageai.com/organization/billing/payment-methods) . Even with a payment method entered, the free tokens (e.g., 200M tokens for Voyage series 3) will still apply. See our [pricing](/docs/pricing) for the free tokens for your model. You can also purchase usage credits to accelerate tier progression and increase your rate limits faster. For example, purchasing **$100** of usage credits automatically upgrades you to **Tier 2**, doubling the rate limits of usage Tier 1. See the [FAQ](/docs/faq#how-can-i-set-up-prepaid-billing) for details and instructions. How can I avoid hitting rate limits? [](#how-can-i-avoid-hitting-rate-limits) ================================================================================= In this section, we’ll offer some tips to avoid and manage rate limit errors. We will provide code snippets to demonstrate these tips and as a starting point for you to use. The code snippets assume you've properly [installed the Voyage AI python package and have a configured API key](/docs/api-key-and-installation) . You can use the following boilerplate code with all of the provided code snippets. Python `import voyageai vo = voyageai.Client() documents = [ "The Mediterranean diet emphasizes fish, olive oil, and vegetables, believed to reduce chronic diseases.", "Photosynthesis in plants converts light energy into glucose and produces essential oxygen.", "20th-century innovations, from radios to smartphones, centered on electronic advancements.", "Rivers provide water, irrigation, and habitat for aquatic species, vital for ecosystems.", "Apple’s conference call to discuss fourth fiscal quarter results and business updates is scheduled for Thursday, November 2, 2023 at 2:00 p.m. PT / 5:00 p.m. ET.", "Shakespeare's works, like 'Hamlet' and 'A Midsummer Night's Dream,' endure in literature." ]` ### Use larger batches [](#use-larger-batches) If you have many documents to embed, you can increase the number of documents you embed per request and increase your overall throughput by sending larger batches. A "batch" is the collection of documents you are embedding in one request, and the "batch size" is the number of documents in the batch, meaning the length of the list of documents. For example, assume you want to vectorize 512 documents. If you used a batch size of 1, then this would require 512 requests and you could hit your RPM limit. However, if you used a batch size of 128, then this would require only 4 requests and you would not hit your RPM limit. You can control the batch size by changing the number of documents you provide in the request, and using larger batch sizes will reduce your overall RPM for a given number of documents. The following are a compact code to vectorize documents in batches of size 128, and a more verbose code printing out useful runtime logs. Python (compact)Python (verbose) `# Embed more than 128 documents in a for loop. batch_size = 128 embeddings = [] for i in range(0, len(documents), batch_size): embeddings += vo.embed( documents[i:i + batch_size], model="voyage-3", input_type="document" ).embeddings` ``# Embed more than 128 documents in a for loop. # Note: it requires the tokenizers package, which can be installed with `pip install tokenizers` batch_size = 128 print("Total documents:", len(documents)) embeddings = [] for i in range(0, len(documents), batch_size): batch = documents[i:i + batch_size] print(f"Embedding documents {i} to {i + len(batch) - 1}") print("Total tokens:", vo.count_tokens(batch)) batch_embeddings = vo.embed( batch, model="voyage-3", input_type="document" ).embeddings embeddings += batch_embeddings print("Embeddings preview:", embeddings[0][:5])`` Finally, you will need to consider the [API maximum batch size and tokens](/reference/embeddings-api) when selecting your batch size. You cannot exceed the API max batch size, and if you have longer documents, the token limit per request may constrain you to a smaller batch size. ### Set a wait period [](#set-a-wait-period) Another way to manage rate limit errors is to simply make requests less frequently. This applies to both RPM and TPM. You can do this by pacing your requests, and the most straightforward approach is inserting a wait period between each request. Python `import time batch_size = 128 embeddings = [] for i in range(0, len(documents), batch_size): batch = documents[i:i + batch_size] embeddings += vo.embed( batch, model="voyage-3", input_type="document" ).embeddings # Add a small amount of sleep in between each iteration time.sleep(0.1)` ### Exponential Backoff [](#exponential-backoff) One strategy is to backoff once you’ve hit your rate limit (i.e., receive a 429 error). With this strategy, you wait for an exponentially increased time after receiving a rate limit error before trying again. This continues until the request is successful or until a maximum number of retries is reached. For example, if your initial wait time was 1 second and you got three consecutive rate limit errors before success, you would wait 1, 2, and 4 seconds after each rate limit error, respectively, before re-sending the request. The following code snippet demonstrates how to implement exponential backoff (with a little bit of jitter) in Python using the [tenacity package](https://tenacity.readthedocs.io/en/latest/) . Python ``# Note: it requires the tenacity package, which can be installed with `pip install tenacity` from tenacity import ( retry, stop_after_attempt, wait_random_exponential, ) @retry(wait=wait_random_exponential(multiplier=1, max=60), stop=stop_after_attempt(6)) def embed_with_backoff(**kwargs): return vo.embed(**kwargs) embed_with_backoff(texts=documents, model="voyage-3", input_type="document")`` ### Request rate limit increase [](#request-rate-limit-increase) If the above methods are insufficient for your use case, please [let us know and request a rate limit increase](http://www.voyageai.com/request-rate-limit-increase) . How do organization limits affect my project limits? [](#how-do-organization-limits-affect-my-project-limits) ================================================================================================================= Rate limits can be set at the project level by the organization Admin. Project-level rate limits must not exceed the organization’s corresponding rate limit. However, if the organization rate limit is reached first, projects may be rate-limited to a lower rate. This can occur when the sum of all project rate limits exceeds the organization limit. Consider an organization rate limit O with three projects with rate limits P1, P2, and P3. The table below illustrates three scenarios where the sum of the project rate limits is less than, equal to, or greater than the organization rate limit. For each scenario, the table indicates whether the organization limit can be reached and whether one project’s usage can impact another. | | Scenario 1
P1 + P2 + P3 < O | Scenario 2
P1 + P2 + P3 = O | Scenario 3
P1 + P2 + P3 > O | | --- | --- | --- | --- | | **Scenario Description** | Sum of all project rate limits **_less than_** organization limit | Sum of all project rate limits **_equal to_** organization limit | Sum of all project rate limits _**greater than**_ organization limit | | **Can the organization limit be reached?** | **No**, even if all projects reach their rate limits, the organization rate limit will not be exceeded. | **Yes**, if all projects reach their rate limits, the organization limit will also be reached. | **Yes**, since the sum of all project rate limits exceeds the organization limit, the organization limit can be reached before individual projects hit their own limits | | **Can one project’s usage impact another?** | **No**. | **No**. | **Yes**. If projects collectively consume enough usage to reach the organization limit before any or all projects reach their individual limits, projects can be rate-limited to a lower rate than their individual limits. | Updated 19 days ago * * * --- # Snowflake We’ve partnered with Snowflake to [integrate](https://venturebeat.com/ai/why-snowflake-is-backing-embedding-startup-voyage-ai-to-improve-enterprise-rag/) our embedding models as part of the [Cortex AI](https://docs.snowflake.com/en/release-notes/2024/other/2024-09-12-voyage-embed-model) service within the [Snowflake AI Data Cloud](https://www.snowflake.com/en/data-cloud/platform/) . The product integration is part of the broader Snowflake [investment](https://www.snowflake.com/en/blog/snowflake-invests-in-voyage-ai-optimizing-multilingual-rag/) in Voyage AI. `voyage-multilingual-2` is available in the text embedding function ([EMBED\_TEXT\_1024](https://docs.snowflake.com/en/sql-reference/functions/embed_text_1024-snowflake-cortex) ) of [Snowflake Cortex AI](https://docs.snowflake.com/en/release-notes/2024/other/2024-09-12-voyage-embed-model) . Updated 6 months ago * * * --- # Flexible Dimensions and Quantization Storage and search costs in vector-based search can become significant for large corpora, such as in code retrieval with massive repositories. The costs scale linearly in the embedding dimensionality and precision (i.e., the number of bits used to encode each number). Lower dimensional embeddings and quantization (e.g., binary or int8 representations) are used to dramatically lower costs without losing much retrieval quality. These are enabled by [Matryoshka learning](https://arxiv.org/abs/2205.13147)  and [quantization-aware training](https://arxiv.org/abs/1702.00758) . Matryoshka Embeddings [](#matryoshka-embeddings) ---------------------------------------------------- Matryoshka learning creates embeddings with a nested family of embeddings with various lengths within a single vector. Concretely, for each _k_ in {256, 512, and 1024}, the first _k_ entries of a 2048-dimensional embedding also form a valid _k_\-dimensional embedding that is shorter with a slight loss of retrieval quality. This allows users to vectorize documents into a long 2048-dimensional vector in advance and have the flexibility to use shorter versions of the embedding (by taking the first _k_ entries) without re-invoking the embedding model. Newer Voyage models, such as `voyage-3-large`, generate Matryoshka embeddings and support multiple output dimensions directly through the `output_dimension` parameter (see supported model embedding dimensions [here](/docs/embeddings) ). ### How can I truncate Matryoshka embeddings? [](#how-can-i-truncate-matryoshka-embeddings) You can truncate Matryoshka embeddings by keeping the leading subset of dimensions. For example, the following Python code demonstrates how to truncate 1024-dimensional vectors to 256 dimensions: Python `import voyageai import numpy as np def embd_normalize(v: np.ndarray) -> np.ndarray: """ Normalize the rows of a 2D NumPy array to unit vectors by dividing each row by its Euclidean norm. Raises a ValueError if any row has a norm of zero to prevent division by zero. """ row_norms = np.linalg.norm(v, axis=1, keepdims=True) if np.any(row_norms == 0): raise ValueError("Cannot normalize rows with a norm of zero.") return v / row_norms vo = voyageai.Client() # Generate voyage-3-large vectors, which by default are 1024-dimensional floating-point numbers embd = vo.embed(['Sample text 1', 'Sample text 2'], model='voyage-3-large').embeddings # Set shorter dimension short_dim = 256 # Resize and normalize vectors to shorter dimension resized_embd = embd_normalize(np.array(embd)[:, :short_dim]).tolist()` Quantization [](#quantization) ---------------------------------- Quantized embeddings have lower precision, with 8 bits or 1 bit per dimension, reducing storage costs by 4x or 32x compared to 32-bit floats. Newer Voyage embedding models, such as `voyage-3-large`, support lower-precision embeddings in various data types: `int8` (8-bit signed integer), `uint8` (8-bit unsigned integer), `binary` (bit-packed `int8`), and `ubinary` (bit-packed `uint8`). Most vector databases directly support storing and searching with quantized embeddings, including Milvus, Qdrant, Weaviate, Elasticsearch, and Vespa AI. Supported Voyage models enable quantization by specifying the output data type with the `output_dtype` parameter: * `float`: Each returned embedding is a list of 32-bit (4-byte) [single-precision floating-point](https://en.wikipedia.org/wiki/Single-precision_floating-point_format)  numbers. This is the default and provides the highest precision / retrieval accuracy. * `int8` and `uint8`: Each returned embedding is a list of 8-bit (1-byte) integers ranging from -128 to 127 and 0 to 255, respectively. * `binary` and `ubinary`: Each returned embedding is a list of 8-bit integers that represent bit-packed, quantized single-bit embedding values: `int8` for `binary` and `uint8` for `ubinary`. The length of the returned list of integers is **1/8** of the actual dimension of the embedding. The `binary` type uses the offset binary method, explained below. _Binary quantization example_Consider the following eight embedding values: -0.03955078, 0.006214142, -0.07446289, -0.039001465, 0.0046463013, 0.00030612946, -0.08496094, and 0.03994751. With binary quantization, values less than or equal to zero will be quantized to a binary zero, and positive values to a binary one, resulting in the following binary sequence: 0, 1, 0, 0, 1, 1, 0, 1. These eight bits are then packed into a single 8-bit integer: 01001101 (with the leftmost bit as the most significant bit). * `ubinary`: The binary sequence is directly converted and represented as the unsigned integer (`uint8`) 77. * `binary`: The binary sequence is represented as the signed integer (`int8`) -51, calculated using the [offset binary](#offset-binary) method (77 - 128 = -51). ### Offset binary [](#offset-binary) [Offset binary](https://en.wikipedia.org/wiki/Offset_binary) is a method for representing negative numbers in binary form (i.e., [signed number representations](https://en.wikipedia.org/wiki/Signed_number_representations) ). This approach is used when representing quantized binary embedding values, specifically when the `output_dtype` parameter is set to `binary`. The binary values are bit-packed, with each 8-bit sequence represented as an integer calculated using the offset binary method. In this method, an offset is _added_ to an integer before converting to binary and _subtracted_ when converting from binary to a signed integer. For signed 8-bit integers, which have a range of -128 to 127, the offset is typically 128. _Signed integer to binary example_ To represent -32 as an 8-bit binary number: 1. Add the offset (128) to -32, resulting in 96. 2. Convert 96 to binary: 01100000. _Binary to signed integer example_ To determine the signed integer from the 8-bit binary number 01010101: 1. Convert it directly to an integer: 85. 2. _Subtract_ the offset (128) from 85, resulting in -43. Examples [](#examples) -------------------------- Below are Python code examples for converting and working with binary embeddings. Convert float to binary & ubinaryUnpack binary & ubinary embeddings `import numpy as np import voyageai vo = voyageai.Client() # Generate float embeddings embd_float = vo.embed('Sample text 1', model='voyage-3-large', output_dimension=2048).embeddings[0] # Compute 512-dimensional bit-packed binary and ubinary embeddings from 2048-dimensional float embeddings embd_binary_calc = (np.packbits(np.array(embd_float) > 0, axis=0) - 128).astype(np.int8).tolist() # Quantize, binary offset embd_binary_512_calc = embd_binary_calc[0:64] # Truncate. Binary is 1/8 length of embedding dimension. embd_ubinary_calc = (np.packbits(np.array(embd_float) > 0, axis=0)).astype(np.uint8).tolist() # Quantize, binary offset embd_ubinary_512_calc = embd_ubinary_calc[0:64] # Truncate. Binary is 1/8 length of embedding dimension.` `import numpy as np import voyageai vo = voyageai.Client() # Generate binary embeddings embd_binary = vo.embed('Sample text 1', model='voyage-3-large', output_dtype='binary', output_dimension=2048).embeddings[0] embd_ubinary = vo.embed('Sample text 1', model='voyage-3-large', output_dtype='ubinary', output_dimension=2048).embeddings[0] # Unpack bits embd_binary_bits = [format(x, f'08b') for x in np.array(embd_binary) + 128] # List of (bits) strings embd_binary_unpacked = [bit == '1' for bit in ''.join(embd_binary_bits)] # List of booleans embd_ubinary_bits = [format(x, f'08b') for x in np.array(embd_ubinary)] # List of (bits) strings embd_ubinary_unpacked = [bit == '1' for bit in ''.join(embd_ubinary_bits)] # List of booleans` We also have a getting started tutorial in Google Colab [here](https://colab.research.google.com/drive/1JcIZ3dHLjsuxwgXsGTL79VwawMl9XXYt) . Updated 2 months ago * * * --- # Organizations and Projects An **organization** is the top-level administrative entity in a Voyage account, with each account starting with a default organization. Billing, budget limits, and data controls are managed at the organization level. Rate limits and API keys are also managed at this level but can be further constrained within **projects**, which are sub-groupings of resources and settings often tied to a specific team or use case. An organization can contain multiple projects. This guide will show you how to manage your organization and projects. All walkthroughs in this guide are done in the Voyage [dashboard](https://dash.voyageai.com/) . What are the different user roles within organizations and projects, and what permissions do they entail? [](#what-are-the-different-user-roles-within-organizations-and-projects-and-what-permissions-do-they-entail) ========================================================================================================================================================================================================================== Organizations have two roles: **Admin** and **Member**. Admins have full control of and access to their organizations, while Members have more limited permissions. Admins can create and manage projects, invite members, and assign roles. They are also automatically included in all projects within their organization. Within projects, Admins can designate members as project **Owners**, who have full control and access within their projects. A summary of these roles is provided in the table below. | Role | Scope | Description | | --- | --- | --- | | Admin | Organization | Full control of and access to the organization and can:

\- Manage all billing information and controls- Invite, remove, and manage all members
\- Create, view, and archive all projects
\- Create, revoke, and manage all API keys
\- Manage all rate limits
\- Manage all budget limits
\- Manage all permissions to view usage information for others in the organization | | Owner | Project | Full control of and access to the project and can:

\- Invite, remove, and manage all project members
\- Create, revoke, and manage all project API keys
\- Manage project rate limits
\- Manage project budget limits | | Member | Organization and Project | Can create keys for projects they are members of. Can view created API keys and keys within projects they are Owners of. Has view-only access to all other entities. Can leave organization or project. | _Granular role permissions_ The following table details granular permissions for organization and project roles. | | Org Admin | Project Owner | Member | | --- | --- | --- | --- | | Org Usage / Costs | Full access | View only or no access\* | View only or no access\* | | Org API Keys | Full access | Access to created keys and keys within owned projects. | Access to created keys | | Org Members | Full access | View only admins, owners, and members within the same project. Can leave org. | View only admins, owners, and members within the same project. Can leave org. | | Org Manage Projects | Full access | View only projects user is a member of. | View only projects user is a member of. | | Org General | Full access | View only | View only | | Org Billing | Full access | View only | View only | | Org Rate Limits | Full access | View only | View only | | Org Budget Limits | Full access | View only | View only | | Org Data Control | Full access | No access | No access | | Org Terms of Service | Full access | View only | View only | | Project General | Full access | View only | View only | | Project Members | Full access | Full access | View only. Can leave project. | | Project Usage | Full access | Full access | View only | | Project Budget Limits | Full access | Full access | View only | | Project API Keys | Full access | Full access | Access to created keys | | Project Rate Limits | Full access | Full access | View only | \* No access if organization Admin has set usage dashboard visibility to organization owners only. Can I belong to multiple organizations? [](#can-i-belong-to-multiple-organizations) ======================================================================================= Yes, you can belong to multiple organizations. An admin from an organization must invite you. To switch between organizations, hover over your organization’s name in the top-left of the dashboard and select the organization from the list. ![](https://files.readme.io/302a75c78eaeafab5e86eecdc7ae1ef5c95ca5f2039c49d011eff43aacdf5094-switch-organization.png) How do I create a project? [](#how-do-i-create-a-project) ============================================================= You can create projects in two ways. 1. **Project selector.** Hover on the project name in the upper left-corner of the page and select **Add project**. ![](https://files.readme.io/b6126080ef7f9434f324751b3410b994994a6ba7a41f5bab419544e39845b4eb-project-selector.png) 2. **Manage projects**. Navigate to the [**Manage Projects**](https://dashboard.voyageai.com/organization/projects) section under **Organization**. Click on the **Create** button in the upper right-corner of the page. ![](https://files.readme.io/a32c03276e15bf07b31ce7894c41420448ed7c2eee1a672bb62eca87cf26faf7-manage-projects.png) Both options will launch a **Create a new project** modal. Provide a name and click the **Create** button. ![](https://files.readme.io/d5b230ffefac0d450fa0c5190ac221bb6c2021c701585bcd6a7f9a06b281aa8f-image.png) How can I add users to an organization and assign them to a project? [](#how-can-i-add-users-to-an-organization-and-assign-them-to-a-project) ================================================================================================================================================= To add users to an organization, you must be an Admin. Navigate to the [**Members**](https://dashboard.voyageai.com/organization/members) section under **Organization** in the navigation sidebar. Click the **Invite** button in the upper-right corner to open the **Invite Team Members** modal. ![](https://files.readme.io/82c6fa304a890605bf3a1cdccf38e42fd1c47ec3e6287a389111b5f45fe6e805-image.png) Enter the users' email addresses in the **Emails** field, assign their organization role using the **Role** dropdown, and select a project for them using the **Project** dropdown. Finally, click **Invite**. Invited users will receive an email to accept the invitation. ![](https://files.readme.io/518a111fc90d76526ded6228c9a16392506db973d2e7c65fadfa7143e74fcd82-image.png) [](#section-) ================ Updated 2 months ago * * * --- # Error Codes The following table shows the possible error codes that might be returned from the API. | Error Code | Error Type | Description | | --- | --- | --- | | 🔴 400 | `Invalid Request` | **Cause:** The request is invalid, which might be due to one of the following reasons:

* The request body is not a valid [JSON](https://en.wikipedia.org/wiki/JSON)
;
* A parameter is invalid or has the wrong type.
* Batch size is too large;
* Total number of tokens in the batch exceeds the limit;
* Number of tokens in an example exceeds the context length;

**Solution:** The specific error will be indicated in the error message. Double check the request body and correct the errors. | | 🔴 401 | `Unauthorized` | **Cause:** Invalid authentication.
**Solution:** Ensure the API key is correctly specified in the HTTP request header. Please create or view your API key in our [dashboard](https://dash.voyageai.com)
. | | 🔴 403 | `Forbidden` | **Cause:** The IP address you are sending the request from might be forbidden.
**Solution:** Try to use a different IP address to call the API. | | 🔴 429 | `Rate Limit Exceeded` | **Cause:** The frequency of your requests is too high.
**Solution:** Please pace your requests. Read the [rate limit guide](/docs/rate-limits)
for more information. | | 🔴 500 | `Server Error` | **Cause:** Unexpected issue on our servers.
**Solution:** Retry your request after a brief wait. If the error persists, please contact us at [\[email protected\]](/cdn-cgi/l/email-protection#0b6864657f6a687f4b7d64726a6c6e6a6225686466)
. | | 🔴 502
🔴 503
🔴 504 | `Service Unavailable` | **Cause:** Our servers are currently experiencing high traffic.
**Solution:** Retry your requests after a brief wait. | Updated 7 months ago * * * --- # Service Level Objectives Last Updated: September 18, 2024 Voyage AI will use commercially reasonable efforts to make the Voyage AI API SaaS Services available on a 24 x 7 x 365 basis with an objective of achieving the following "Monthly Uptime Percentages" for the corresponding “Covered Service” as set out in the following table (the "**Service Level Objective**" or "**SLO**"): | Covered Services | Monthly Uptime Percentage | | --- | --- | | `api.voyageai.com` | ≥ 99.5% | | `https://dash.voyageai.com/` | ≥ 99.5% | **Status Page**: [https://voyageai-status.statuspage.io](https://voyageai-status.statuspage.io) Please email us at [\[email protected\]](/cdn-cgi/l/email-protection#9efdf1f0eafffdeadee8f1e7fff9fbfff7b0fdf1f3) for inquiries. Definitions [](#definitions) ================================ Capitalized terms used but not defined herein, will have the meanings set out in the Voyage AI Agreement or [Terms of Use](https://www.voyageai.com/tos) (collectively, the "**Agreement**"). "**Covered Service**" means services that comprise the Voyage AI API SaaS Services available at the following Websites: (i) [`api.voyageai.com`](http://api.voyageai.com) and (ii) [`https://dash.voyageai.com/`](https://dash.voyageai.com/) "**Monthly Uptime Percentage**" means the total percentage of availability of a Covered Service within any given month, excluding unavailability of such Covered Service due to one or more of the following: (i) services marked as Alpha or Beta, (ii) acts or omissions of Customer or its Permitted Users, including any modifications made to the Covered Service or any breach of the terms of the Agreement; (iii) Customer's failure to adhere to Voyage’s recommendations, including hardware or software configuration necessary to meet minimum system requirements for the Covered Service; (iv) acts or omissions of Voyage AI when complying with the request or acting under the direction of Customer; (v) spikes in demand for system resources driven by Customer for which Customer and Voyage AI did not previously agree in writing; (vi) scheduled or emergency maintenance of the Covered Service; (vii) downtime of third party service providers; (viii) Force Majeure; or (ix) suspension permitted under the Agreement. Updated about 2 months ago * * * --- # Text Embeddings Model Choices [](#model-choices) ==================================== Voyage currently provides the following text embedding models: | Model | Context Length (tokens) | Embedding Dimension | Description | | --- | --- | --- | --- | | `voyage-3-large` | 32,000 | 1024 (default), 256, 512, 2048 | The best general-purpose and multilingual retrieval quality. See [blog post](https://blog.voyageai.com/2025/01/07/voyage-3-large/)
for details. | | `voyage-3` | 32,000 | 1024 | Optimized for general-purpose and multilingual retrieval quality. See [blog post](https://blog.voyageai.com/2024/09/18/voyage-3/)
for details. | | `voyage-3-lite` | 32,000 | 512 | Optimized for latency and cost. See [blog post](https://blog.voyageai.com/2024/09/18/voyage-3/)
for details. | | `voyage-code-3` | 32,000 | 1024 (default), 256, 512, 2048 | Optimized for **code** retrieval. See [blog post](https://blog.voyageai.com/2024/12/04/voyage-code-3/)
for details. | | `voyage-finance-2` | 32,000 | 1024 | Optimized for **finance** retrieval and RAG. See [blog post](https://blog.voyageai.com/2024/06/03/domain-specific-embeddings-finance-edition-voyage-finance-2/)
for details. | | `voyage-law-2` | 16,000 | 1024 | Optimized for **legal** retrieval and RAG. Also improved performance across all domains. See [blog post](https://blog.voyageai.com/2024/04/15/domain-specific-embeddings-and-retrieval-legal-edition-voyage-law-2/)
for details. | | `voyage-code-2` | 16,000 | 1536 | Optimized for code retrieval (17% better than alternatives) / Previous generation of code embeddings. See [blog post](https://blog.voyageai.com/2024/01/23/voyage-code-2-elevate-your-code-retrieval/)
for details. | Need help deciding which text embedding model to use? Check out our [FAQ](/docs/faq#what-embedding-models-are-available-and-which-one-should-i-use) . _Older models_ The following are our earlier models, which are still accessible from our API. We recommend using the new models above for better quality and efficiency. Our latest models listed in the above table will be strictly better than the legacy models in all aspects, such as quality, context length, latency, and throughput. | Model | Context Length (tokens) | Embedding Dimension | Description | | --- | --- | --- | --- | | `voyage-multilingual-2` | 32,000 | 1024 | Optimized for **multilingual** retrieval and RAG. See [blog post](https://blog.voyageai.com/2024/06/10/voyage-multilingual-2-multilingual-embedding-model/)
for details. | | `voyage-large-2-instruct` | 16,000 | 1024 | Top of [MTEB leaderboard](https://huggingface.co/spaces/mteb/leaderboard)
. Instruction-tuned general-purpose embedding model optimized for clustering, classification, and retrieval. For retrieval, please use `input_type` parameter to specify whether the text is a query or document. For classification and clustering, please use the instructions [here](https://github.com/voyage-ai/voyage-large-2-instruct)
. See [blog post](https://blog.voyageai.com/2024/05/05/voyage-large-2-instruct-instruction-tuned-and-rank-1-on-mteb/)
for details. We recommend existing `voyage-large-2-instruct` users to transition to `voyage-3`. | | `voyage-large-2` | 16,000 | 1536 | General-purpose embedding model that is optimized for retrieval quality (e.g., better than OpenAI V3 Large). Please transition to `voyage-3`. | | `voyage-2` | 4000 | 1024 | General-purpose embedding model optimized for a balance between cost, latency, and retrieval quality. Please transition to `voyage-3-lite`. | | `voyage-lite-02-instruct` | 4000 | 1024 | \[_Deprecated_\] [Instruction-tuned](https://github.com/voyage-ai/voyage-lite-02-instruct/blob/main/instruct.json)
for classification, clustering, and sentence textual similarity tasks, which are the only recommended use cases. Please transition to `voyage-3`. | | `voyage-02` | 4000 | 1024 | \[_Deprecated_\] This is our pilot-version v2 embedding model. We kindly ask you to transition to `voyage-3` as detailed above. | | `voyage-01` | 4000 | 1024 | \[_Deprecated_\] This is our v1 embedding model. Please transition to `voyage-3`. | | `voyage-lite-01` | 4000 | 1024 | \[_Deprecated_\] This is our v1 embedding model. Please transition to `voyage-3`. | | `voyage-lite-01-instruct` | 4000 | 1024 | \[_Deprecated_\] Tweaked on top of `voyage-lite-01` for classification and clustering tasks. Please transition to `voyage-3`. | * * * Python API [](#python-api) ============================== Voyage text embeddings are accessible in Python through the `voyageai` [package](/docs/api-key-and-installation#install-voyage-python-package) . Please install the `voyageai` package, [set up](/docs/api-key-and-installation) the API key, and use the `voyageai.Client.embed()` function to vectorize your inputs. > [`voyageai.Client.embed`](https://github.com/voyage-ai/voyageai-python/blob/870e95486d4f373a9dab9061f5e2397ff230db06/voyageai/client.py#L31) > `(texts : List[str], model : str, input_type : Optional[str] = None, truncation : Optional[bool] = None, output_dimension: Optional[int] = None, output_dtype: Optional[str] = "float")` **Parameters** * **texts** (List\[str\]) - A list of texts as a list of strings, such as `["I like cats", "I also like dogs"]`. Currently, we have two constraints on the list: * The maximum length of the list is 128. * The total number of tokens in the list is at most 1M for `voyage-3-lite`; 320K for `voyage-3` and `voyage-2`; and 120K for `voyage-3-large`, `voyage-code-3`, `voyage-large-2-instruct`, `voyage-finance-2`, `voyage-multilingual-2`, `voyage-law-2`, and `voyage-large-2`. * **model** (str) - Name of the model. Recommended options: `voyage-3-large`, `voyage-3`, `voyage-3-lite`, `voyage-code-3`, `voyage-finance-2`, `voyage-law-2`. * **input\_type** (str, optional, defaults to `None`) - Type of the input text. Options: `None`, `query`, `document`. * When `input_type` is `None` , the embedding model directly converts the inputs (`texts`) into numerical vectors. For retrieval/search purposes, where a "query" is used to search for relevant information among a collection of data, referred to as "documents", we recommend specifying whether your inputs (`texts`) are intended as queries or documents by setting `input_type` to `query` or `document` , respectively. In these cases, Voyage automatically prepends a prompt to your inputs (`texts`) before vectorizing them, creating vectors more tailored for retrieval/search tasks. Embeddings generated with and without the `input_type` argument are compatible. * For transparency, the following prompts are prepended to your input. * For query, the prompt is "_Represent the query for retrieving supporting documents:_ ". * For document, the prompt is "_Represent the document for retrieval:_ ". * **truncation** (bool, optional, defaults to `True`) - Whether to truncate the input texts to fit within the context length. * If `True`, an over-length input texts will be truncated to fit within the context length, before vectorized by the embedding model. * If `False`, an error will be raised if any given text exceeds the context length. * **output\_dimension** (int, optional, defaults to `None`) - The number of dimensions for resulting output embeddings. * Most models only support a single default dimension, used when `output_dimension` is set to `None` (see model embedding dimensions [above](/docs/embeddings#model-choices) ). * `voyage-3-large` and `voyage-code-3` support the following `output_dimension` values: 2048, 1024 (default), 512, and 256. * **output\_dtype** (str, optional, defaults to `float`) - The data type for the embeddings to be returned. Options: `float`, `int8`, `uint8`, `binary`, `ubinary`. `float` is supported for all models. `int8`, `uint8`, `binary`, and `ubinary` are supported by `voyage-3-large` and `voyage-code-3`. Please see our [guide](/docs/flexible-dimensions-and-quantization#quantization) for more details about output data types. * `float`: Each returned embedding is a list of 32-bit (4-byte) [single-precision floating-point](https://en.wikipedia.org/wiki/Single-precision_floating-point_format) numbers. This is the default and provides the highest precision / retrieval accuracy. * `int8` and `uint8`: Each returned embedding is a list of 8-bit (1-byte) integers ranging from -128 to 127 and 0 to 255, respectively. * `binary` and `ubinary`: Each returned embedding is a list of 8-bit integers that represent bit-packed, quantized single-bit embedding values: `int8` for `binary` and `uint8` for `ubinary`. The length of the returned list of integers is 1/8 of `output_dimension` (which is the actual dimension of the embedding). The `binary` type uses the offset binary method. Please refer to our guide for details on [offset binary](/docs/flexible-dimensions-and-quantization#offset-binary) and [binary embeddings](/docs/flexible-dimensions-and-quantization#quantization) . **Returns** * A `EmbeddingsObject`, containing the following attributes: * **embeddings** (List\[List\[float\]\] or List\[List\[int\]\]) - A list of embeddings for the corresponding list of input texts. Each embedding is a vector represented as a list of [floats](https://docs.python.org/3/library/functions.html#float) when `output_dtype` is set to `float` and as a list of [integers](https://docs.python.org/3/library/functions.html#int) for all other values of `output_dtype` (`int8`, `uint8`, `binary`, `ubinary`). * **total\_tokens** (int) - The total number of tokens in the input texts. **Example** PythonOutput `import voyageai vo = voyageai.Client() # This will automatically use the environment variable VOYAGE_API_KEY. # Alternatively, you can use vo = voyageai.Client(api_key="") texts = [ "The Mediterranean diet emphasizes fish, olive oil, and vegetables, believed to reduce chronic diseases.", "Photosynthesis in plants converts light energy into glucose and produces essential oxygen.", "20th-century innovations, from radios to smartphones, centered on electronic advancements.", "Rivers provide water, irrigation, and habitat for aquatic species, vital for ecosystems.", "Apple’s conference call to discuss fourth fiscal quarter results and business updates is scheduled for Thursday, November 2, 2023 at 2:00 p.m. PT / 5:00 p.m. ET.", "Shakespeare's works, like 'Hamlet' and 'A Midsummer Night's Dream,' endure in literature." ] # Embed the documents result = vo.embed(texts, model="voyage-3", input_type="document") print(result.embeddings)` `[ [0.005100732669234276, -0.04479769989848137, 0.023507222533226013, ...], [0.014439012855291367, -0.05365084111690521, -0.019395021721720695, ...], [0.048720553517341614, -0.05633316561579704, -0.00036325387191027403, ...], [0.011985979974269867, -0.10603463649749756, -0.021402806043624878, ...], [0-0.004056323319673538, -0.029253670945763588, 0.0067815352231264114, ...], [0.00726663414388895, -0.0137932812795043, -0.015689561143517494, ...] ]` * * * _Deprecated Functions_ The following functions are deprecated and will be removed in the future. > `get_embedding(text, model="voyage-01", input_type=None)`[](https://github.com/voyage-ai/voyageai-python/blob/870e95486d4f373a9dab9061f5e2397ff230db06/voyageai/embeddings_utils.py#L72) **Parameters** * **text** - A single document/query as a string, such as `"I like cats"` . * **model** - Name of the model. Options: `voyage-01` (default), `voyage-lite-01`. * **input\_type** - Type of the input text. Defalut to `None`, meaning the type is unspecified. Other options: `query`, `document`. **Returns** * An embedding vector (a list of floating-point numbers) for the document. > `get_embeddings(list_of_text, model="voyage-01", input_type=None)`[](https://github.com/voyage-ai/voyageai-python/blob/870e95486d4f373a9dab9061f5e2397ff230db06/voyageai/embeddings_utils.py#L89) **Parameters** * **list\_of\_text** - A list of documents as a list of strings, such as `["I like cats", "I also like dogs"]`. The maximum length of the list is 64. * **model** - Name of the model. Options: `voyage-01` (default), `voyage-lite-01`. * **input\_type** - Type of the input text. Defalut to `None`, meaning the type is unspecified. Other options: `query`, `document`. **Returns** * A list of embedding vectors. * * * REST API [](#rest-api) ========================== Voyage text embeddings can be accessed by calling the endpoint `POST https://api.voyageai.com/v1/embeddings`. Please refer to the [Text Embeddings API Reference](/reference/embeddings-api) for the specification. **Example** Embed a single stringEmbed a list of strings `curl https://api.voyageai.com/v1/embeddings \ -H "Content-Type: application/json" \ -H "Authorization: Bearer $VOYAGE_API_KEY" \ -d '{ "input": "Sample text", "model": "voyage-3", "input_type": "document" }'` `curl https://api.voyageai.com/v1/embeddings \ -H "Content-Type: application/json" \ -H "Authorization: Bearer $VOYAGE_API_KEY" \ -d '{ "input": ["Sample text 1", "Sample text 2"], "model": "voyage-3", "input_type": "document" }'` * * * TypeScript Library [](#typescript-library) ============================================== Voyage text embeddings are accessible in TypeScript through the [Voyage TypeScript Library](https://www.npmjs.com/package/voyageai) , which exposes all the functionality of our text embeddings endpoint (see [Text Embeddings API Reference](/reference/embeddings-api) ). Updated about 2 months ago * * * --- # Multimodal Embeddings Multimodal embedding models transform unstructured data from multiple modalities into a shared vector space. Voyage multimodal embedding models support text and content-rich images — such as figures, photos, slide decks, and document screenshots — eliminating the need for complex text extraction or ETL pipelines. Unlike traditional multimodal models like CLIP, which process text and images separately, Voyage multimodal embedding models can directly vectorize inputs containing interleaved text + images. The architecture of CLIP also prevents it from being usable in mixed-modality searches, as text and image vectors often align with irrelevant items of the same modality. Voyage multimodal embedding models eliminate this bias by processing all inputs through a single backbone. Model Choices [](#model-choices) ==================================== Voyage currently provides the following multimodal embedding models: | Model | Context Length (tokens) | Embedding Dimension | Description | | --- | --- | --- | --- | | `voyage-multimodal-3` | 32,000 | 1024 | Rich multimodal embedding model that can vectorize interleaved text and content-rich images, such as screenshots of PDFs, slides, tables, figures, and more. See [blog post](https://blog.voyageai.com/2024/11/12/voyage-multimodal-3/)
for details. | * * * Python API [](#python-api) ============================== Voyage multimodal embeddings are accessible in Python through the `voyageai` [package](/docs/api-key-and-installation#install-voyage-python-package) . Please install the `voyageai` package, [set up](/docs/api-key-and-installation) the API key, and use the `voyageai.Client.multimodal_embed()` function to vectorize your inputs. > [`voyageai.Client.multimodal_embed`](https://github.com/voyage-ai/voyageai-python/blob/main/voyageai/client.py#L99) > `(inputs : List[dict] or List[List[Union[str, PIL.Image.Image]]], model : str, input_type : Optional[str] = None, truncation : Optional[bool] = True)` **Parameters** * **inputs** (List\[dict\] or List\[List\[Union\[str, PIL.Image.Image\]\]\]) - A list of multimodal inputs to be vectorized. * Each input is a sequence of text and images, which can be represented in either of the following two ways: **(1)** A list containing text strings and/or PIL image objects (List\[Union\[str, PIL.Image.Image\]\]), where each image is an instance of the [Pillow Image class](https://pillow.readthedocs.io/en/stable/reference/Image.html#the-image-class) . For example: `["This is a banana.", PIL.Image.open("banana.jpg")]`. _PIL Image Object_ [Pillow](https://python-pillow.org/) is a widely used Python library for image processing. In the above example, [`PIL.Image.open()`](https://pillow.readthedocs.io/en/stable/reference/Image.html#PIL.Image.open) opens an image file and returns a PIL Image. Please see our [FAQ](/docs/faq#what-is-the-python-pillow-library) for more details about Pillow. **(2)** A dictionary that contains a single key `"content"`, whose value represents a sequence of text and images. The dictionary schema is identical to that of an input in the `inputs` parameter of the [REST API](/reference/multimodal-embeddings-api) . * The following constraints apply to the `inputs` list: * The list must not contain more than 1000 inputs. * Each image must not contain more than 16 million pixels or be larger than 20 MB in size. * With every 560 pixels of an image being counted as a token, each input in the list must not exceed 32,000 tokens, and the total number of tokens across all inputs must not exceed 320,000. * **model** (str) - Name of the model. Currently, the only supported model is `voyage-multimodal-3`. * **input\_type** (str, optional, defaults to `None`) - Type of the input. Options: `None`, `query`, `document`. * When `input_type` is `None`, the embedding model directly converts the `inputs` into numerical vectors. For retrieval/search purposes, where a "query", which can be text or image in this case, is used to search for relevant information among a collection of data referred to as "documents," we recommend specifying whether your `inputs` are intended as queries or documents by setting `input_type` to `query` or `document`, respectively. In these cases, Voyage automatically prepends a prompt to your `inputs` before vectorizing them, creating vectors more tailored for retrieval/search tasks. Since `inputs` can be multimodal, "queries" and "documents" can be text, images, or an interleaving of both modalities. Embeddings generated with and without the `input_type` argument are compatible. * For transparency, the following prompts are prepended to your input. * For `query`, the prompt is "_Represent the query for retrieving supporting documents:_ ". * For `document`, the prompt is "_Represent the document for retrieval:_ ". * **truncation** (bool, optional, defaults to `True`) - Whether to truncate the inputs to fit within the context length. * If `True`, an over-length input will be truncated to fit within the context length before being vectorized by the embedding model. If the truncation happens in the middle of an image, the entire image will be discarded. * If `False`, an error will be raised if any input exceeds the context length. **Returns** * A `MultimodalEmbeddingsObject`, containing the following attributes: * **embeddings** (List\[List\[float\]\]) - A list of embeddings for the corresponding list of inputs, where each embedding is a vector represented as a list of floats. * **text\_tokens** (int) - The total number of text tokens in the list of inputs. * **image\_pixels** (int) - The total number of image pixels in the list of inputs. * **total\_tokens** (int) - The combined total of text and image tokens. Every 560 pixels counts as a token. **Example** PythonOutput `import voyageai import PIL vo = voyageai.Client() # This will automatically use the environment variable VOYAGE_API_KEY. # Alternatively, you can use vo = voyageai.Client(api_key="") # Example input containing a text string and PIL image object inputs = [ ["This is a banana.", PIL.Image.open('banana.jpg')] ] # Vectorize inputs result = vo.multimodal_embed(inputs, model="voyage-multimodal-3") print(result.embeddings)` `[ [0.027587890625, -0.021240234375, 0.018310546875,...] ]` * * * REST API [](#rest-api) ========================== Voyage multimodal embeddings can be accessed by calling the endpoint `POST https://api.voyageai.com/v1/multimodalembeddings`. Please refer to the [Multimodal Embeddings API Reference](/reference/multimodal-embeddings-api) for the specification and an example. * * * TypeScript Library [](#typescript-library) ============================================== Voyage multimodal embeddings are accessible in TypeScript through the [Voyage TypeScript Library](https://www.npmjs.com/package/voyageai) , which exposes all the functionality of our multimodal embeddings endpoint (see [Multimodal Embeddings API Reference](/reference/multimodal-embeddings-api) ). Updated 3 months ago * * * --- # Tokenization Given an input, the first step of the embedding/reranking process is to dissect it into a list of tokens. This tokenization step is automatically performed on our servers when you call the API. We open-source the tokenizers so that you can preview the tokenized results and verify the number of tokens the API uses. > 📘 > > Voyage's Tokenizers on Hugging Face 🤗 > > > -------------------------------------------- > > Voyage's tokenizers are available on [Hugging Face 🤗](https://huggingface.co/voyageai) > You can access the tokenizer associated with a particular model using the following code: > > Python > > `from transformers import AutoTokenizer tokenizer = AutoTokenizer.from_pretrained('voyageai/voyage-3')` > > _Update on Voyage tokenizer_ > > Our earlier models, including embedding models `voyage-01`, `voyage-lite-01`, `voyage-lite-01-instruct`, `voyage-lite-02-instruct`, `voyage-2`, `voyage-large-2`, `voyage-code-2`, `voyage-law-2`, `voyage-large-2-instruct`, and reranking model `rerank-lite-1`, use the same tokenizer as [Llama 2](https://ai.meta.com/llama/) > . However, our new models have adopted different tokenizers for optimized performance. Therefore, in the future, please specify the model you use when calling the tokenizer. In our [Python package](/docs/api-key-and-installation#install-voyage-python-package) , we provide functions in `voyageai.Client` which allows you to try the tokenizer before calling the API: > [`voyageai.Client.tokenize`](https://github.com/voyage-ai/voyageai-python/blob/v0.3.1/voyageai/_base.py#L112) > `(texts : List[str], model: str)` **Parameters** * **texts** (List\[str\]) - A list of texts to be tokenized. * **model** (str) - Name of the model to be tokenized for. For example, `voyage-3-large`, `voyage-3`, `voyage-3-lite`, `rerank-2`, `rerank-2-lite`, `voyage-multimodal-3`. _Note_ The "model" parameter was added in June 2024. Our earlier models, including embedding models `voyage-01`, `voyage-lite-01`, `voyage-lite-01-instruct`, `voyage-lite-02-instruct`, `voyage-2`, `voyage-large-2`, `voyage-code-2`, `voyage-law-2`, `voyage-large-2-instruct`, and reranking model `rerank-lite-1`, used the same tokenizer. However, our new models have adopted different tokenizers. **Please specify the "model" when using this function.** If "model" is unspecified, the old tokenizer will be loaded, which may produce mismatched results if you are using our latest models. **Returns** * A list of [`tokenizers.Encoding`](https://huggingface.co/docs/tokenizers/main/en/api/encoding#encoding) , each of which represents the tokenized results of an input text string. * * * **Example - Return Tokens** PythonOutput `import voyageai vo = voyageai.Client() # This will automatically use the environment variable VOYAGE_API_KEY. # Alternatively, you can use vo = voyageai.Client(api_key="") texts = [ "The Mediterranean diet emphasizes fish, olive oil, and vegetables, believed to reduce chronic diseases.", "Photosynthesis in plants converts light energy into glucose and produces essential oxygen." ] tokenized = vo.tokenize(texts, model="voyage-3") for i in range(len(texts)): print(tokenized[i].tokens)` `['The', 'ĠMediterranean', 'Ġdiet', 'Ġemphasizes', 'Ġfish', ',', 'Ġolive', 'Ġoil', ',', 'Ġand', 'Ġvegetables', ',', 'Ġbelieved', 'Ġto', 'Ġreduce', 'Ġchronic', 'Ġdiseases', '.'] ['Photos', 'ynthesis', 'Ġin', 'Ġplants', 'Ġconverts', 'Ġlight', 'Ġenergy', 'Ġinto', 'Ġglucose', 'Ġand', 'Ġproduces', 'Ġessential', 'Ġoxygen', '.']` > [`voyageai.Client.count_tokens`](https://github.com/voyage-ai/voyageai-python/blob/v0.3.1/voyageai/_base.py#L129) > `(texts : List[str], model: str)` **Parameters** * **texts** (List\[str\]) - A list of texts to count the tokens for. * **model** (str) - Name of the model to be counted for. For example, `voyage-3-large`, `voyage-3`, `voyage-3-lite`, `rerank-2`, `rerank-2-lite`, `voyage-multimodal-3`. _Note_ The "model" parameter was added in June 2024. Our earlier models, including embedding models `voyage-01`, `voyage-lite-01`, `voyage-lite-01-instruct`, `voyage-lite-02-instruct`, `voyage-2`, `voyage-large-2`, `voyage-code-2`, `voyage-law-2`, `voyage-large-2-instruct`, and reranking model `rerank-lite-1`, used the same tokenizer. However, our new models have adopted different tokenizers. **Please specify the "model" when using this function.** If "model" is unspecified, the old tokenizer will be loaded, which may produce mismatched results if you are using our latest models. **Returns** * The total number of tokens in the input texts, as an integer. * * * **Example - Return Count of Tokens** PythonOutput `import voyageai vo = voyageai.Client() # This will automatically use the environment variable VOYAGE_API_KEY. # Alternatively, you can use vo = voyageai.Client(api_key="") texts = [ "The Mediterranean diet emphasizes fish, olive oil, and vegetables, believed to reduce chronic diseases.", "Photosynthesis in plants converts light energy into glucose and produces essential oxygen." ] total_tokens = vo.count_tokens(texts, model="voyage-3") print(total_tokens)` `32` > [`voyageai.Client.count_usage`](https://github.com/voyage-ai/voyageai-python/blob/v0.3.1/voyageai/_base.py#L137) > `(inputs : List[dict] or List[List[Union[str, PIL.Image.Image]]] , model: str)` **Parameters** * **inputs** (List\[dict\] or List\[List\[Union\[str, PIL.Image.Image\]\]\]) - A list of text and image sequences for which to count text tokens, image pixels, and total tokens. The list elements follow the same format as the `inputs` parameter of `voyageai.Client.multimodal_embed()`, except that image URLs are not supported. For additional details, refer to the [Python API for Multimodal Embeddings](/docs/multimodal-embeddings#python-api) . * **model** (str) - Name of the model (which affects how inputs are counted). Currently, the only supported model is `voyage-multimodal-3`. For other models that support only text, use the `voyageai.Client.count_tokens()` function to calculate token counts. **Returns** * A dictionary containing the following attributes: * **text\_tokens** (int) - The total number of text tokens in the list of inputs. * **image\_pixels** (int) - The total number of image pixels in the list of inputs. * **total\_tokens** (int) - The combined total of text and image tokens. Every 560 pixels counts as a token. * * * **Example** PythonOutput `import voyageai import PIL vo = voyageai.Client() # This will automatically use the environment variable VOYAGE_API_KEY. # Alternatively, you can use vo = voyageai.Client(api_key="") inputs = [ ["This is a banana.", PIL.Image.open('banana.jpg')] ] usage = vo.count_usage(inputs, model="voyage-multimodal-3") print(usage)` `{'text_tokens': 5, 'image_pixels': 2000000, 'total_tokens': 3576}` Our embedding models have [context length limits](/docs/embeddings#model-choices) . If your text exceeds the limit, you would need to truncate the text before calling the API, or specify the `truncation` argument so that we can do it for you. > 📘 > > Tokens, words, and characters > > > ----------------------------------- > > Modern NLP models typically convert a text string into a list of tokens. Frequent words, such as "you" and "apple," will be tokens by themselves. In contrast, rare or long words will be broken into multiple tokens, e.g., "uncharacteristically" is dissected into four tokens, "▁un", "character", "ist", and "ically". One word roughly corresponds to 1.2 - 1.5 tokens on average, depending on the complexity of the domain. The tokens produced by our tokenizer have an average of 5 characters, suggesting that you could roughly estimate the number of tokens by dividing the number of characters in the text string by 5. To determine the exact number of tokens, please use the `count_tokens()` function. > 📘 > > tiktoken > > > -------------- > > `tiktoken` is the open-source version of OpenAI's tokenizer. Voyage models use different tokenizers, which can be accessed from [Hugging Face 🤗](https://huggingface.co/voyageai) > . Therefore, our tokenizer may generate a different list of tokens for a given text compared to `tiktoken`. Statistically, the number of tokens produced by our tokenizer is on average 1.1 - 1.2 times that of `tiktoken`. To determine the exact number of tokens, please use the `count_tokens()` function. Updated about 2 months ago * * * --- # Rerankers A reranker, given a query and many documents, returns the (ranks of) relevancy between the query and documents. The documents oftentimes are the preliminary results from an embedding-based retrieval system, and the reranker refines the ranks of these candidate documents and provides more accurate relevancy scores. Unlike [embedding](/docs/embeddings) models that encode queries and documents separately, rerankers are [cross-encoders](https://www.sbert.net/examples/applications/cross-encoder/README.html) that jointly process a pair of query and document, enabling more accurate relevancy prediction. Thus, it is a common practice to apply a reranker on the top candidates retrieved with embedding-based search (or with lexical search algorithms such as [BM25](https://en.wikipedia.org/wiki/Okapi_BM25) and [TF-IDF](https://en.wikipedia.org/wiki/Tf%E2%80%93idf) ). Model Choice [](#model-choice) ================================== Voyage currently provides the following reranker models: | Model | Context Length (tokens) | Description | | --- | --- | --- | | `rerank-2` | 16,000 | Our generalist reranker optimized for quality with multilingual support. See [blog post](https://blog.voyageai.com/2024/09/30/rerank-2/)
for details. | | `rerank-2-lite` | 8000 | Our generalist reranker optimized for both latency and quality with multilingual support. See [blog post](https://blog.voyageai.com/2024/09/30/rerank-2/)
for details. | * * * _Older models_ The following are our earlier models, which are still accessible from our API. We recommend using the new models above for better quality and efficiency. Our latest models listed in the above table will be strictly better than the legacy models in all aspects, such as quality, context length, latency, and throughput. | Model | Context Length (tokens) | Description | | --- | --- | --- | | `rerank-1` | 8000 | Our generalist reranker optimized for quality. Multilingual support. See [blog post](https://blog.voyageai.com/2024/05/29/voyage-rerank-1-cutting-edge-general-purpose-and-multilingual-reranker/)
for details. | | `rerank-lite-1` | 4000 | Our generalist reranker optimized for both latency and quality. See [blog post](https://blog.voyageai.com/2024/03/15/boosting-your-search-and-rag-with-voyages-rerankers/)
for details. | * * * [](#section-) ================ Python API [](#python-api) ============================== Voyage reranker is accessible in Python through the `voyageai` package. Please first [install the `voyageai` package and setup the API key](/docs/api-key-and-installation) . Voyage reranker receives as input a query and a list of candidate documents, e.g., the documents retrieved by a nearest neighbor search with embeddings. It reranks the candidate documents according to their semantic relevances to the search query, and returns the list of relevance scores. To access the reranker, create a [`voyageai.Client`](/docs/api-key-and-installation#voyageaiclient) object and use its `rerank()` method. > [`voyageai.Client.rerank`](https://github.com/voyage-ai/voyageai-python/blob/602ba757eb2e072c95e765f9b83992affbb4ea41/voyageai/client.py#L56) > `(query: str, documents: List[str], model: str, top_k: Optional[int] = None, truncation: bool = True)` **Parameters** * **query** (str) - The query as a string. The query can contain a maximum of 4000 tokens for `rerank-2`, 2000 tokens for `rerank-2-lite` and `rerank-1`, and 1000 tokens for `rerank-lite-1`. * **documents** (List\[str\]) - The documents to be reranked as a list of strings. * The number of documents cannot exceed 1000. * The sum of the number of tokens in the query and the number of tokens in any single document cannot exceed 16000 for `rerank-2`; 8000 for `rerank-2-lite` and `rerank-1`; and 4000 for `rerank-lite-1`. * The total number of tokens, defined as "the number of query tokens × the number of documents + sum of the number of tokens in all documents", cannot exceed 600K for `rerank-2` and `rerank-2-lite`, and 300K for `rerank-1` and `rerank-lite-1`. Please see our [FAQ](/docs/faq#what-is-the-total-number-of-tokens-for-the-rerankers) . * **model** (str) - Name of the model. Recommended options: `rerank-2`, `rerank-2-lite`. * **top\_k** (int, optional, defaults to `None`) - The number of most relevant documents to return. If not specified, the reranking results of all documents will be returned. * **truncation** (bool, optional, defaults to `True`) - Whether to truncate the input to satisfy the "context length limit" on the query and the documents. * If `True`, the query and documents will be truncated to fit within the context length limit, before processed by the reranker model. * If `False`, an error will be raised when the query exceeds 4000 tokens for `rerank-2`; 2000 tokens `rerank-2-lite` and `rerank-1`; and 1000 tokens for `rerank-lite-1`, or the sum of the number of tokens in the query and the number of tokens in any single document exceeds 16000 for `rerank-2`; 8000 for `rerank-2-lite` and `rerank-1`; and 4000 for `rerank-lite-1`. **Returns** * A `RerankingObject`, containing the following attributes: * **results** (List\[`RerankingResult`\]) - A list of `RerankingResult`, with format specified below, sorted by the descending order of relevance scores. The length of the list equals to top\_k if this argument is specified, otherwise the number of the input documents. Each element in the list is a `RerankingResult` object, which contains attributes: * **index** (int) - The index of the document in the input list. * **document** (str) - The document as a string. * **relevance\_score** (float) - The relevance score of the document with respect to the query. * **total\_tokens** (int) - The total number of tokens in the input, which is defined as "the number of query tokens × the number of documents + sum of the number of tokens in all documents". **Example** PythonOutput `import voyageai vo = voyageai.Client() # This will automatically use the environment variable VOYAGE_API_KEY. # Alternatively, you can use vo = voyageai.Client(api_key="") query = "When is Apple's conference call scheduled?" documents = [ "The Mediterranean diet emphasizes fish, olive oil, and vegetables, believed to reduce chronic diseases.", "Photosynthesis in plants converts light energy into glucose and produces essential oxygen.", "20th-century innovations, from radios to smartphones, centered on electronic advancements.", "Rivers provide water, irrigation, and habitat for aquatic species, vital for ecosystems.", "Apple’s conference call to discuss fourth fiscal quarter results and business updates is scheduled for Thursday, November 2, 2023 at 2:00 p.m. PT / 5:00 p.m. ET.", "Shakespeare's works, like 'Hamlet' and 'A Midsummer Night's Dream,' endure in literature." ] reranking = vo.rerank(query, documents, model="rerank-2", top_k=3) for r in reranking.results: print(f"Document: {r.document}") print(f"Relevance Score: {r.relevance_score}") print()` `Document: Apple’s conference call to discuss fourth fiscal quarter results and business updates is scheduled for Thursday, November 2, 2023 at 2:00 p.m. PT / 5:00 p.m. ET. Relevance Score: 0.9296875 Document: The Mediterranean diet emphasizes fish, olive oil, and vegetables, believed to reduce chronic diseases. Relevance Score: 0.40625 Document: Photosynthesis in plants converts light energy into glucose and produces essential oxygen. Relevance Score: 0.39453125` * * * REST API [](#rest-api) ========================== Voyage reranker can be accessed by calling the endpoint `POST https://api.voyageai.com/v1/rerank`. Please refer to the [Reranker API Reference](/reference/reranker-api) for the specification. **Example** Shell `curl https://api.voyageai.com/v1/rerank \ -H "Content-Type: application/json" \ -H "Authorization: Bearer $VOYAGE_API_KEY" \ -d '{ "query": "Sample query", "documents": [ "Sample document 1", "Sample document 2" ], "model": "rerank-2" }'` * * * TypeScript Library [](#typescript-library) ============================================== Voyage rerankers are accessible in TypeScript through the [Voyage TypeScript Library](https://www.npmjs.com/package/voyageai) , which exposes all the functionality of our reranker endpoint (see [Reranker API Reference](/reference/reranker-api) ). Updated about 2 months ago * * * --- # AWS Marketplace Model Package AWS Marketplace [model packages](https://aws.amazon.com/marketplace/seller-profile?id=seller-snt4gb6fd7ljg) are containerized solutions that include the model and inference code, designed to be deployed in a customer account and virtual private cloud (VPC). Model packages offer several benefits to customers: 1. **Data flow and access control**. Because models are deployed into their account and VPC, customers maintain full control over data flow and API access, effectively addressing data privacy risks associated with third-party or multi-tenant serving. 2. **Reliability and compliance backed by AWS**. AWS will be the customer sub-processor, and customers inherit all the reliability and compliance guarantees of AWS. 3. **Billing and payment through AWS**. By transacting through a marketplace listing, customers can utilize their existing AWS billing information and credits to procure Voyage models. This streamlined process eliminates the need to manage a separate third-party payment and billing system. A model package can be deployed in two ways — as a [real-time inference API endpoint](https://docs.aws.amazon.com/sagemaker/latest/dg/realtime-endpoints.html) or a [batch transform job](https://docs.aws.amazon.com/sagemaker/latest/dg/batch-transform.html) . A real-time inference API endpoint is a persistent, fully managed API endpoint designed for request-by-request inference. In contrast, a batch transform job is a finite execution process intended for bulk inference on a dataset, with predictions written to a file. In both cases, the model packages are running on some underlying hardware, such as GPUs, which are called AWS instances. **Pricing.** Pricing for the use of a Voyage model package consists of software pricing and infrastructure pricing, both at an hourly rate. Software pricing covers the cost of model usage, while your total hourly cost is the sum of the software pricing (e.g., $5.71 per hour for [`voyage-multilingual-2`](https://aws.amazon.com/marketplace/pp/prodview-ebotgiywtalec?sr=0-4&ref_=beagle&applicationId=AWSMPContessa) ) and the infrastructure pricing (e.g., $1.408 per hour for a single ml.g5.xlarge). Pricing rates vary based on deployment type (i.e., real-time inference API endpoint versus batch transform job), instance type, and region. All Voyage AI models come with a free trial. Below, we present a short tutorial on subscribing to and deploying the models. Then, we will discuss advanced deployment options and provide information on latency and throughput. Model Package Subscription [](#model-package-subscription) ============================================================== You will need the following AWS identity access management (IAM) permission to subscribe to an AWS Marketplace listing. To add them, [sign into your AWS account console](https://console.aws.amazon.com/iam/) and see this [page](https://docs.aws.amazon.com/IAM/latest/UserGuide/access_policies_manage-attach-detach.html#add-policies-console) for instructions. * [AmazonSageMakerFullAccess](https://docs.aws.amazon.com/aws-managed-policy/latest/reference/AmazonSageMakerFullAccess.html) (AWS Managed Policy) * aws-marketplace:ViewSubscriptions * aws-marketplace:Subscribe * aws-marketplace:Unsubscribe To subscribe via the AWS Marketplace: 1. Select the Voyage model package you would like to subscribe to in the [AWS Marketplace](https://aws.amazon.com/marketplace/seller-profile?id=seller-snt4gb6fd7ljg) . 2. Click **Continue to Subscribe**. ![Voyage AI model package AWS marketplace product listing](https://files.readme.io/a961efdfd4d005d154197aed321fc1c52a1df6bc8e324e5d25702222fcbe7d4f-image.png) 3. Click **Accept Offer**. ![Voyage AI model package AWS marketplace subscribe](https://files.readme.io/36ea2d0d6696e815311c34119a2a30e98054d2b45bd74223664b38311fa18a0b-image.png) 4. Confirm you have successfully subscribed (see confirmation toast in the figure below). You can now safely close the window. ![Voyage AI model package AWS marketplace subscribe confirmation](https://files.readme.io/207453ae5fc0c0e2ae5415b7f527cb43d061353384e648de73380b83ae6b5e85-image.png) You can also confirm and manage your AWS Marketplace subscriptions through the console’s [manage subscription page](https://aws.amazon.com/marketplace/library/ref=gtw_navgno_library) . You can [cancel your subscription](https://docs.aws.amazon.com/marketplace/latest/buyerguide/cancel-machine-learning-subscription.html) at any time, but note that canceling your subscription [**_does not_**](https://docs.aws.amazon.com/marketplace/latest/buyerguide/cancel-machine-learning-subscription.html) terminate your existing real-time inference endpoints or batch transform jobs (see [Delete Real-Time Inference Endpoints](#delete-real-time-inference-endpoints) ). Model package deployment requires specific SageMaker instances (e.g., ml.g5.xlarge). The exact quota names for these instances end with “endpoint usage” and “transform job usage” (e.g., “ml.g5.2xlarge for endpoint usage”, and “ml.g5.2xlarge for transform job usage”. These [quotas](https://docs.aws.amazon.com/general/latest/gr/sagemaker.html#limits_sagemaker) are often set to zero by default. Please go to the [SageMaker Service Quotas console](https://console.aws.amazon.com/servicequotas/home/services/sagemaker/quotas) to [request quota increases](https://docs.aws.amazon.com/servicequotas/latest/userguide/request-quota-increase.html) if needed. Model Package Deployment [](#model-package-deployment) ========================================================== This section covers the mechanics of how to deploy a model package using Amazon SageMaker Studio and our example Jupyter Notebooks. Amazon SageMaker Studio [](#amazon-sagemaker-studio) -------------------------------------------------------- [Amazon SageMaker Studio](https://aws.amazon.com/sagemaker/studio/) is a web-based interface for ML and AI development that includes a hosted notebook environment already authenticated to your AWS account. You can skip this section if you have another preferred Jupyter notebook execution environment, such as your local machine, and you know how to properly authenticate to your AWS account from that environment. Follow the SageMaker documentation to first [launch SageMaker Studio](https://docs.aws.amazon.com/sagemaker/latest/dg/studio-updated-launch.html) , and then [launch a JupyterLab environment](https://docs.aws.amazon.com/sagemaker/latest/dg/studio-updated-jl-user-guide.html) . ![Amazon SageMaker Studio](https://files.readme.io/97ce1bf372af3f435910e8c98c38405d61ddf7fffa37c98eee82fd522c581fd0-image.png) Jupyter Notebook [](#jupyter-notebook) ------------------------------------------ We provide an [example Jupyter notebook](https://github.com/voyage-ai/voyageai-aws/blob/main/deploy_voyage_model_package_sagemaker.ipynb) to get started with Python using the AWS SDK ([Boto3](https://boto3.amazonaws.com/v1/documentation/api/latest/index.html) ) and the [Amazon SageMaker Python SDK](https://sagemaker.readthedocs.io/en/stable/) . You can clone the notebook into SageMaker Studio, or your preferred Jupyter notebook execution environment, by cloning the [Voyage AI AWS repo](https://github.com/voyage-ai/voyageai-aws/tree/main) (i.e., `git clone https://github.com/voyage-ai/voyageai-aws.git`). ![JupyterLab git clone](https://files.readme.io/f7669bb78111a58f7dca9326090a48332eb2a33676b1521e00ed043447515d36-image.png) Alternatively, you can directly download the notebook from GitHub and move it to your notebook execution environment (e.g., upload it to SageMaker Studio). Once accessible to SageMaker Studio or your preferred execution environment, you can open the notebook and follow the steps in it to deploy the models. Delete Real-Time Inference Endpoints [](#delete-real-time-inference-endpoints) ---------------------------------------------------------------------------------- Be careful to not have real-time inference endpoints running unnecessarily. They will incur wasteful costs, potentially leading to unexpected charges. If you are using a provided example Jupyter notebook, be sure to run the “Clean-up” section, which deletes the endpoint and associated endpoint configuration. You can manage and delete endpoints through SageMaker Studio or the SageMaker console (see [here](https://docs.aws.amazon.com/sagemaker/latest/dg/realtime-endpoints-manage.html) for instructions). Advanced Deployment [](#advanced-deployment) ================================================ The Jupyter notebook described in the previous section is meant to get you started and build your understanding of deploying model packages. However, we’d like to make you aware of several other ways to deploy model packages, such as CloudFormation, the SageMaker Console, and the AWS CLI, though the details are beyond the scope of this documentation. These alternative methods may better suit the existing production workflow you have: CloudFormation for declarative infrastructure specification, SageMaker Console for interactive UI-based deployment, and AWS CLI for programmatic shell orchestration. A subscribed listing **Configure and launch** page provides instructions and resources for the aforementioned deployment methods, which you can return to by: 1. Returning to the product listing page for your subscribed model of interest from the [AWS Marketplace](https://aws.amazon.com/marketplace/seller-profile?id=seller-snt4gb6fd7ljg) . 2. Clicking on **Continue to Subscribe** (upper right), which will take you to the **Subscribe to this software** page. ![Voyage AI model package AWS marketplace continue to subscribe](https://files.readme.io/381d847f8dee6e615388cac16325459edef2de3f540de0ed1f0122710dd54b22-image.png) 3. In the **Subscribe to this software** page, you should notice an “Already Subscribed” indicator. Click on the **Continue to configuration** button (upper right). ![Voyage AI model package AWS marketplace already subscribed](https://files.readme.io/15d7c516c24758984e31704e30f43e3cadb3e2fa82e17346bb4a53826ead82e1-image.png) 4. You are now back at the **Configure and launch** page. You can select your desired launch method, and the user interface will change with the appropriate instructions and resources for that method. ![Voyage AI model package AWS marketplace configure and launch](https://files.readme.io/4a33160c71a2daae8ee71a10887bc3fdeb976fcdf9c51801b3edbb9be928a78c-image.png) Latency and Throughput [](#latency-and-throughput) ====================================================== The following table provides representative latency and throughput numbers for a deployed real-time inference endpoint running on an ml.g6.xlarge. | Model | Latency | Throughput | | --- | --- | --- | | `voyage-3` | 75 ms for a single query with at most 200 tokens. | 57M tokens per hour. | | `voyage-3-lite` | 20 ms for a single query with at most 200 tokens. | 182M tokens per hour. | | `voyage-large-2`
`voyage-large-2-instruct`
`voyage-code-2`
`voyage-law-2`
`voyage-multilingual-2`
`voyage-finance-2` | 90 ms for single query with at most 100 tokens.
185 ms for 500 tokens. | 12.6M tokens per hour. | | `voyage-2` | 75 ms for single query with at most 200 tokens
90 ms for 500 tokens. | 36M tokens per hour. | | `rerank-2` | For 25K tokens: 1.5 s (1 GPU), 415 ms (4 GPUs), and 245 ms (8 GPUs). | 60M tokens per hour. | | `rerank-2-lite` | For 25K tokens: 565 ms (1 GPU), 170 ms (4 GPUs), and 120 ms (8 GPUs). | 160M tokens per hour. | | `rerank-lite-1` | For 25K tokens: 445 ms (1 GPU), 135 ms (4 GPUs), and 90 ms (8 GPUs). | 202M tokens per hour. | If you need assistance subscribing and deploying a Voyage model package from the AWS Marketplace, please send an email to [\[email protected\]](/cdn-cgi/l/email-protection#83e0ecedf7e2e0f7c3f5ecfae2e4e6e2eaade0ecee) or join our [Discord](https://discord.gg/zAU7GQEmvT) . Updated 4 months ago * * * --- # Quickstart Tutorial This tutorial is a step-by-step guidance on implementing a specialized chatbot with RAG stack using embedding models (e.g., Voyage embeddings) and large language models (LLMs). We start with a brief overview of the retrieval augmented generation (RAG) stack. Then, we’ll briefly go through the preparation and vectorization of data (i.e. embeddings). We’ll show how to do retrieval with embeddings as well as some additional refinements with rerankers. Finally, we’ll put this all together to create a simple RAG chatbot. Brief overview of the RAG stack [](#brief-overview-of-the-rag-stack) ------------------------------------------------------------------------ A typical RAG stack is illustrated in Figure 1. When presented with a search query, our initial step involves employing the embedding model, such as Voyage embeddings, to derive the vector representation of the query. Subsequently, we conduct a document search, identifying the most relevant documents from a [vector store](https://www.pinecone.io/learn/vector-database/) The most relevant document is then selected and combined with the original query. This composite input is then submitted to a generative model to generate a comprehensive response to the query. This RAG stack can be further refined with reranking, which we’ll discuss in a later section. ![Figure 1: Typical RAG stack](https://files.readme.io/754fddc90f8c41cfe4a9846d5ed2e72cb39110ae3311a041df582600de568850-rag_with_embedding.webp) Figure 1: Typical RAG stack Prepare data [](#prepare-data) ---------------------------------- You will need a corpus of documents that your chatbot will specialize in. You can choose to save your documents as demonstrated below or use the following set of documents as a starting point. Python `documents = [ "The Mediterranean diet emphasizes fish, olive oil, and vegetables, believed to reduce chronic diseases.", "Photosynthesis in plants converts light energy into glucose and produces essential oxygen.", "20th-century innovations, from radios to smartphones, centered on electronic advancements.", "Rivers provide water, irrigation, and habitat for aquatic species, vital for ecosystems.", "Apple’s conference call to discuss fourth fiscal quarter results and business updates is scheduled for Thursday, November 2, 2023 at 2:00 p.m. PT / 5:00 p.m. ET.", "Shakespeare's works, like 'Hamlet' and 'A Midsummer Night's Dream,' endure in literature." ]` We have additional examples available in this [link](https://drive.google.com/file/d/1Is0J1chn27bR3NvcwC-zqOWa4cdPDmzJ/view?usp=sharing) for you to download and test. Vectorize/embed the documents [](#vectorizeembed-the-documents) ------------------------------------------------------------------- First, follow the [installation](/docs/api-key-and-installation) guide to install the Voyage Python package and get your API key. Then, we can use the Python client to create embeddings. Embed a small number of documentsEmbed a large number of documents `import voyageai vo = voyageai.Client() # This will automatically use the environment variable VOYAGE_API_KEY. # Alternatively, you can use vo = voyageai.Client(api_key="") # Embed the documents documents_embeddings = vo.embed( documents, model="voyage-3", input_type="document" ).embeddings` `import voyageai vo = voyageai.Client() # This will automatically use the environment variable VOYAGE_API_KEY. # Alternatively, you can use vo = voyageai.Client(api_key="") # Embed more than 128 documents with a for loop batch_size = 128 documents_embeddings = [ vo.embed( documents[i : i + batch_size], model="voyage-3", input_type="document", ).embeddings for i in range(0, len(documents), batch_size) ]` * * * _Notes on **Tokenization**_ > 📘 > > Tokenization > > > ------------------ > > Voyage has a limit on the context length and the number of total tokens in each request (see [here](/docs/embeddings#python-api) > for details). To comply with this limit, you can preview the tokenized results by our [tokenizer](/docs/tokenization) > and count the total tokens in the input. > > Python > > `# Print the tokenized results tokenized = vo.tokenize(documents) for i in range(len(documents)): print(tokenized[i].tokens) # Count the total tokens print(vo.count_tokens(documents))` * * * A minimalist retrieval system [](#a-minimalist-retrieval-system) -------------------------------------------------------------------- The main feature of the embeddings is that the cosine similarity between two embeddings captures the semantic relatedness of the corresponding original passages. This allows us to use the embeddings to do semantic retrieval / search. Suppose the user sends a "query" (e.g., a question or a comment) to the chatbot: Python `query = "When is Apple's conference call scheduled?"` To find out the document that is most similar to the query among the existing data, we can first embed/vectorize the query: Python `# Get the embedding of the query query_embedding = vo.embed([query], model="voyage-3", input_type="query").embeddings[0]` **Nearest neighbor Search:** We can find the closest embedding among the documents based on the cosine similarity, and retrieve the corresponding document. Python `# Compute the similarity # Voyage embeddings are normalized to length 1, therefore dot-product and cosine # similarity are the same. similarities = np.dot(doc_embds, query_embd) retrieved_id = np.argmax(similarities) print(documents[retrieved_id])` _k_**\-nearest neighbors Search (**_k_**\-NN):** It is often useful to retrieve not only the closest document but also the _k_ most closest documents. We can use any `k_nearest_neighbors` search algorithm to achieve this goal. Python `# Use the k-nearest neighbor algorithm to identify the top-k documents with the highest similarity retrieved_embds, retrieved_embd_indices = k_nearest_neighbors( query_embedding, documents_embeddings, k=3 ) retrieved_docs = [documents[index] for index in retrieved_embd_indices]` * * * _Notes on **Cosine Similarity**, **Nearest Neighbor Search**, and **Vector Database**_ > 📘 > > Cosine Similarity > > > ----------------------- > > **Cosine Similarity:** Given vectors _q_ and _d_, the cosine similarity is defined as > > ![](https://files.readme.io/9e643e2-image.png) > > where _q⋅d_ is the dot product of the vectors _q_ and _d_, and ||_q_||, ||_d_|| are the magnitude (or length) of vectors _q_ and _d_, respectively. The term "cosine similarity" derives its name from the fact that it measures the cosine of the angle between two vectors. A bigger cosine similarity means the two vectors are closer. > 📘 > > Nearest Neighbor Search > > > ----------------------------- > > **Nearest neighbor Search:** Given a vector _q_ (the embedding for the query) and a sequence of embeddings _d\_1, ..., d\_n_ for the _n_ documents in the pool, we can define the nearest neighbor of vector _q_ in the pool through the computation of cosine similarity > > ![](https://files.readme.io/a9eadef-image.png) > > _k_**\-nearest neighbors Search (**_k_**\-NN):** Sort the documents based on their cosine similarities to the query in descending order (higher similarity comes first), obtaining an ordered list of documents. Select the _k_ documents with the highest cosine similarities, which represent the _k_ nearest neighbors of the query. > > The `k_nearest_neighbors` function is implemented below. > > Python > > `from sklearn.metrics.pairwise import cosine_similarity import numpy as np def k_nearest_neighbors(query_embedding, documents_embeddings, k=5): # Convert to numpy array query_embedding = np.array(query_embedding) documents_embeddings = np.array(documents_embeddings) # Reshape the query vector embedding to a matrix of shape (1, n) to make it # compatible with cosine_similarity query_embedding = query_embedding.reshape(1, -1) # Calculate the similarity for each item in data cosine_sim = cosine_similarity(query_embedding, documents_embeddings) # Sort the data by similarity in descending order and take the top k items sorted_indices = np.argsort(cosine_sim[0])[::-1] # Take the top k related embeddings top_k_related_indices = sorted_indices[:k] top_k_related_embeddings = documents_embeddings[sorted_indices[:k]] top_k_related_embeddings = [ list(row[:]) for row in top_k_related_embeddings ] # convert to list return top_k_related_embeddings, top_k_related_indices` > > When the number of document is huge, one will need to use a vector database (e.g. [Pinecone](https://www.pinecone.io/) > ) to compute (approximate) $k$-nearest neighbor efficiently, which is beyond the scope of this tutorial. > 📘 > > Vector Database > > > --------------------- > > **Vector Database** is a specialized database or data platform designed to cater to the unique needs of applications and industries that rely heavily on vector-based data. This database is engineered to efficiently store, manage, and retrieve vector data, which can include a wide range of information such as spatial data, molecular sequences, time-series data, and more. VectorDB offers a robust set of tools and features tailored to support the indexing, querying, and analysis of vector data, making it a valuable resource for researchers, data scientists, and businesses working with complex datasets that exhibit vector-like characteristics. * * * Refinement with rerankers [](#refinement-with-rerankers) ------------------------------------------------------------ We can further refine our embedding-based retrieval with rerankers. The refined RAG stack with a reranker is illustrated in Figure 2. Here, the retrieved documents from the vector store are subsequently passed to a reranker, which then reranks the documents for semantic relevance against the query and produces a more relevant and smaller set of documents for inputting to the generative model. ![Figure 2: RAG stack with reranker ](https://files.readme.io/15e2e50dc20a85d99fa3e43a52bcf336a99b612347686ca272e10a52eaafae35-rag_with_embedding_reranker.webp) Figure 2: RAG stack with reranker Below, we send initially retrieved documents to the reranker to obtain the top-3 most relevant documents. Python `# Reranking documents_reranked = vo.rerank(query, documents, model="rerank-2", top_k=3)` We see that the reranker properly ranks the Apple conference call document as the most relevant to the query. Python `for r in documents_reranked.results: print(f"Document: {r.document}") print(f"Relevance Score: {r.relevance_score}") print(f"Index: {r.index}") print()` Output: Text `Document: Apple’s conference call to discuss fourth fiscal quarter results and business updates is scheduled for Thursday, November 2, 2023 at 2:00 p.m. PT / 5:00 p.m. ET. Relevance Score: 0.9296875 Index: 4 Document: The Mediterranean diet emphasizes fish, olive oil, and vegetables, believed to reduce chronic diseases. Relevance Score: 0.40625 Index: 0 Document: Photosynthesis in plants converts light energy into glucose and produces essential oxygen. Relevance Score: 0.39453125 Index: 1` A minimalist RAG chatbot [](#a-minimalist-rag-chatbot) ---------------------------------------------------------- The [Retrieval-Augmented Generation](https://www.pinecone.io/learn/retrieval-augmented-generation/) (RAG) chatbot represents a cutting-edge approach in conversational artificial intelligence. RAG combines the powers of retrieval-based and generative methods to produce more accurate and contextually relevant responses. RAG can leverage a large corpora of text to retrieve relevant documents and then send those documents to language models, such as Claude or GPT, to generate replies. This methodology ensures that the chatbot's answers are both informed by vast amounts of information and tailored to the specifics of the user's query. Suppose you have implemented a semantic search system as described in the previous section---either with or without a reranker. As a result of the search process, you have retrieved the most relevant document, referred to as `retrieved_doc`. We can craft a prompt with this context which we can use as input to the language model. Python `# Take the retrieved document and use it as a prompt for the text generation model prompt = f"Based on the information: '{retrieved_doc}', generate a response of {query}"` Now you can utilize a text generation model like Claude 3.5 Sonnet to craft a response based on the provided query and the retrieved document. Install the `anthropic` package first: Shell `pip install anthropic` Then run the following code: Python `import anthropic # Initialize Anthropic API client = anthropic.Anthropic(api_key="YOUR ANTHROPIC API KEY") message = client.messages.create( model="claude-3-5-sonnet-20240620", max_tokens=1024, messages=[ {"role": "user", "content": prompt} ] ) print(message.content[0].text)` Output: `Apple's conference call is scheduled for Thursday, November 2, 2023 at 2:00 p.m. PT / 5:00 p.m. ET.` Output without using Voyage retrieved documents `I don't have information about a specific upcoming Apple conference call. Apple typically holds quarterly earnings conference calls, but without a more precise timeframe or context, I can't provide the exact date of their next scheduled call. For the most up-to-date information on Apple's upcoming conference calls or earnings releases, you should check Apple's investor relations website or contact their investor relations department directly.` You can do the same with GPT-4o as well. Install the `openai` package first: Shell `pip install openai` Then run the following code: Python `from openai import OpenAI # Initialize OpenAI client client = OpenAI(api_key="YOUR OPENAI API KEY") response = client.chat.completions.create( model="gpt-4o", messages=[ {"role": "system", "content": "You are a helpful assistant."}, {"role": "user", "content": prompt}, ], ) print(response.choices[0].message.content)` Output: Text `Apple's conference call is scheduled for Thursday, November 2, 2023 at 2:00 p.m. PT / 5:00 p.m. ET.` Output without using Voyage retrieved documents: Text `Apple's conference calls are typically scheduled to discuss quarterly earnings. They usually announce these dates a few weeks in advance. For the specific date and time of the next Apple conference call, I recommend checking Apple's Investor Relations website or recent press releases, as they will have the most accurate and up-to-date information. If you're looking for the scheduled call for a specific quarter, these events usually occur a few weeks after the end of a fiscal quarter, with Apple's fiscal year ending on the last Saturday of September.` Colab examples [](#colab-examples) -------------------------------------- To execute the code examples provided above in Google Colab, please review and run the code snippets in [Google Colaboratory](https://colab.research.google.com/drive/14Us9q_zsMbOsT1mKEypu0RnFYGj3MyP3?usp=sharing) . Updated 3 months ago * * * --- # Azure Marketplace Managed Application Voyage [Azure managed applications](https://azuremarketplace.microsoft.com/en-us/marketplace/apps?search=voyageaiinnovationsinc1718340344903) are Voyage models deployed as real-time inference API endpoints in a customer Azure account and virtual network (VNet). Azure managed applications offer several benefits to customers: 1. **Data flow and access control**. Data never leave the customers' virtual network, comprehensively addressing data privacy risks associated with third-party or multi-tenant serving. 2. **Reliability and compliance backed by Azure**. Only Azure will be the customer's sub-processor, and customers will inherit all Azure's reliability and compliance guarantees. 3. **Billing and payment through Azure**. By transacting through a marketplace listing, customers can utilize their existing Azure billing information and credits to procure Voyage models. This streamlined process eliminates the need to manage a separate third-party payment and billing system. **Pricing.** Voyage Azure managed applications are billed hourly. The total hourly cost will be the sum of the virtual machine (VM) price ($5 per hour) and the price of the underlying instance in your region. There is no management cost for this managed Azure application; this is why the listed Azure managed application monthly price is $0. Prerequisites [](#prerequisites) ==================================== 1. **Azure Subscription**. Ensure you have an existing [Azure subscription](https://learn.microsoft.com/en-us/microsoft-365/enterprise/subscriptions-licenses-accounts-and-tenants-for-microsoft-cloud-offerings?view=o365-worldwide#subscriptions) . 2. **Virtual Network.** You must have an existing virtual network into which you will deploy a Voyage Azure managed application. If do not have an existing one and need help creating one, see the **[Create a Virtual Network](#create-a-virtual-network) ** section below. 3. **Quota in Subscription and Region**. You must have 24 Standard NCADS\_A100\_v4 Family vCPUs quota for each Voyage Azure managed application you want to deploy. The quota is specific to an Azure subscription and region. If you need help checking your quota or requesting a quota increase, see the **[Quota](#quota)** section below. If you already meet all the prerequisites, you can jump to the **[Create a Voyage API Endpoint in Virtual Network](#create-a-voyage-api-endpoint-in-virtual-network)** section below. Quota [](#quota) ==================== In this section, we walk through how to check your quota and request an increase if necessary. You can skip this section if you know you have sufficient quota. Check Quota [](#check-quota) -------------------------------- 1. Go to **Subscriptions** in your [Azure portal](https://portal.azure.com/) . ![check-quota-subscriptions](https://files.readme.io/526af4ac1a7d0826f5cd09bc7048bfbc61f4eb620a38d401335443d7572a2145-01-ba0d965becdd40c4a2fbadce985619ccf474a167b02bc437030a418b81a1c875-image.png) 2. Select the subscription you want to use for the Azure managed application. In the example below, we selected our "Pay-As-You-Go" subscription. Within your subscription page, select **Usage + quotas** under **Settings**. ![](https://files.readme.io/9204ea20c76ff1eb90d841a1fe24f8809e1c501ed5f276ae25b712da9a5402f8-02-c8828e1c9de19b41032c0241af2c3bebd4647b3d9a50f50bd74012d4ff8e7898-image.png) 3. Enter "Standard NCADS\_A100\_v4 Family vCPUs" in the search bar. Filter the **Region** for your desired region; in the example below, we filtered the region for "West US 2." ![](https://files.readme.io/968d1040ec7ccf9e183a1a14c7bc74ae17b633639d6badcac3f7029d3968692e-03-b8bb479d5a5f79db160259792e642afd7cb1be0870bb849900fee5bfcd1e1a64-image.png) 4. Ensure you have sufficient "Standard NCADSA100\_v4 Family vCPUs" quota for the number of Voyage Azure managed applications you want to deploy. Each Voyage Azure managed application requires **_24 Standard NCADS\_A100\_v4 Family vCPUs_**. If you do not have sufficient quota, continue to the next section (**[Request Quota Increase](#request-quota-increase) **) to request more quota. Request Quota Increase [](#request-quota-increase) ------------------------------------------------------ From the **Usage + quotas** page of your desired subscription, you can request to increase your quota: 1. Check the checkbox for the “Standard NCADS\_A100\_v4 Family vCPUs” quota. Select the **New Quota Request** drop down, and select **Enter a new limit**. A **New Quota Request** panel will appear. ![](https://files.readme.io/b371fea6e263b5aa4cc81941af4f185d3499890f727b6934d22a10bb0a59949e-04-97b0f232574324adf08c63e0df5ab41164207dcff3ee5f93951eb1abc438cb8f-image.png) 2. In the **New Quota Request** panel, enter your desired quota in the **New limit** field. Click **Submit**. ![](https://files.readme.io/c4de97511cd0881a1ab658b4dd9fa38733e13f918dc1a0feab993881fa1f3c93-05-94f263575a45b4865cff4e413e8ae5af631d8a67f8cb2f8e31869aacfa0d906d-image.png) Create a Virtual Network [](#create-a-virtual-network) ========================================================== You can skip this step if you already have an existing virtual network into which you want to deploy your Voyage Azure managed application. 1. Click **Create a resource**. ![](https://files.readme.io/fa55b425236e87ce84368bcf2c53142f045c7a820f82b22b22f7c0601762b770-06-3208d8add8859d1db608a7ee53e9e49d5f8f37ebaf5529483b72fd23b4f72e43-image.png) 2. Search for “virtual network”. Select **Virtual network** from the results. ![](https://files.readme.io/c41769f19d1d48aed5123c30ac1ba553375d0afffcafd7f555aa78df12401464-07-65a1a4137855fb754c1fb715b9572a01a890af895fe845b1cf280c7dc6fa43f4-image.png) 3. Click **Create**. ![](https://files.readme.io/268da57e6560c032fb19edde4d2c4956fcc51b6fc62b7ad906ef6f14650ea59a-08-ce17fb57d92bb75fd1daefeb755ab7a2ed85ba0665ad2e890483c629812a4522-image.png) 4. Fill out the fields in the **Basics** tab. ![](https://files.readme.io/2cc558bf4f65305e591b551ceccec74de3f722e19ee2e346620f6f25976a5d32-09-827176ae570255127d420b5c2ab577c9f73c71279a9c40dcaedbb8fd6f53a74e-image.png) | **Field** | **Value** | | --- | --- | | **Subscription** | Select your subscription. | | **Resource group** | Select an existing resource group or create a new resource group. | | **Virtual network name** | Provide a name for your virtual network. | | **Region** | Select your desired region. We assume your desired region has the required resource quota (see [Quota in Subscription and Region](#prerequisites)
prerequisite). | 5. There are several advanced configurations like security, IP addresses, and tags. If you have advanced configuration requirements, feel free to enter them in the appropriate tabs. Voyage does not require any advanced configuration to deploy our application. If you do not need any advanced configuration or do not know, click **Review + create** to directly review and create your application. 6. In the **Review + create** tab, review your configuration and if it looks good, click **Create**. Otherwise, go to previous tabs to make changes; you can click on the **Previous** button at the bottom or directly click on the tab of interest. ![](https://files.readme.io/b79d79ecb5516542e46fb9291ad18e7b9eb575ea36e34e7800c40df8b8d61cef-10-e9656a6d141c20fcb548f2e8c2470321b21250f39490c71ae16702726a9cee0d-image.png) 7. Verify your virtual network creation. As it is being created, you will see the status as "Deployment is in progress." ![](https://files.readme.io/e9d5f4ab4143803372f0bf94c8dd7ae1f3b8beb549bbdd0b0562e725507e3501-11-1cacbda6382614b186eddac8ee524cae208257edf1cf342d420582a7f0b9bf44-image.png) You will see the status change to "Your deployment is complete" when your virtual network has been successfully created. ![](https://files.readme.io/9954c1014212da3db1cd47a4484ab2fbaeb50cecc660cff2fe0405557388d348-12-8fbbe50dc6ef0651e23c68fe780c408ff015bd4f8724183e48092716aaa233bc-image.png) Create a Voyage API Endpoint in Virtual Network [](#create-a-voyage-api-endpoint-in-virtual-network) ======================================================================================================== We will deploy a Voyage Azure managed application to create a Voyage API endpoint in your virtual network. 1. Select the Voyage Azure managed application you would like to subscribe to in the [Azure Marketplace](https://azuremarketplace.microsoft.com/en-us/marketplace/apps?search=voyageaiinnovationsinc1718340344903) . ![](https://files.readme.io/97d9063eaba59b64a99909cc6d1e0188900af38e517755f12924335c38bbc1f5-13-193c48353f338928fddb8dbbf5f691ac7fff90590e1e7022adf00d154dd3aeeb-image.png) 2. Click **Get it Now**. ![](https://files.readme.io/f9a65d3312566ab470625d4d9d98103828ea8a4f6e499dcdd303646c6e1c2642-14-dc4eb276003c8d899a83e0b495aef7f31d1120971567b247e36f884c11a52925-image.png) 3. Click **Continue** in the **Create this app in Azure** modal. ![](https://files.readme.io/17040734609878099df55e28fd9c509c112c86bd20b2b2362cc6877d502c8398-15-687ebd870be081be72036e335e929a949d186fa6b45ccae4159890eee1aa19ed-image.png) 4. Create a Standard Plan Azure managed application by selecting "Standard Plan" and clicking **Create**. ![](https://files.readme.io/5095a0db7b89f93a482331a61a2f92f33588c47a06b99291e23f81d1c5877452-16-57c69f8431d9b5b7ae57b83f7c3ba794b87a67f8fd7e25cf653ba95d41c511b1-image.png) 5. Fill out the fields in the **Basics** tab. Click **Next** to move to **Network Settings**. ![](https://files.readme.io/84ed094d47fe2b6b58f89a07d91439b0003a9758e082e6be508d6e6f5fd7a2ec-17-f19b6b1dd0f5858d89f5c0ed7ca32362e1b39bfd33844e23fa266c183301c978-image.png) | **Field** | **Value** | | --- | --- | | **Subscription** | Select your subscription. | | **Resource group** | Select the resource group that contains your desired virtual network. | | **Region** | Select your desired region. We assume your desired region has the required resource quota (see [Quota in Subscription and Region](#prerequisites)
prerequisite). | | **Application Name** | Provide a name for your application. | 6. Fill out the fields in the **Network Settings** tab. Once complete, click **Next** to move to **Review + create**. ![](https://files.readme.io/979a4d22572dd95fb3316e1366033c3fd97154b965bbfed5b8289b9e55b8c6d6-18-9eaf9112dbc1fe6ae7a3a183f98855aa592b34c4026748a2cdb6549a178dcfc8-image.png) | **Field** | **Value** | | --- | --- | | **Select an existing virtual network (do not create a new vnet)** | Select your desired existing virtual network. The Voyage API endpoint will be deployed into this virtual network. | | **Select or create a subnet that allows HTTP traffic** | Select your desired subnet. Select "default" if you do not have custom or advanced subnets within your virtual network. The deployed Voyage API endpoint will have a private IP address within this subnet. | 7. In the **Review + create** tab, review your configuration and if it looks good, click **Create**. Otherwise, go to previous tabs to make changes; you can click on the **Previous** button at the bottom or directly click on the tab of interest. ![](https://files.readme.io/57c81cb2d59dcd55a78d4f13a4eea2dbb4e4408219cb846d3a85768df98ae036-19-33d2c5ac310a829fc88a5049399bb072d5a8d59c2440a64dd0f36993523c3dd0-image.png) 8. Verify your Voyage API endpoint creation. As it is being created, you will see the status as "Deployment is in progress." ![](https://files.readme.io/e2495183969559ddfa0be5be4205fce43d2a8cdaf1e144739b60535242e9a79d-20-4f907d50eb7f06a3b443255ecc6746455cec3973c0f37ec8b0ac99e99758a3b0-image.png) You will see the status change to "Your deployment is complete" when your Voyage API endpoint has been successfully created. **IMPORTANT**: Click **Go to resource** to view the application properties. ![](https://files.readme.io/a7707576f7154006aec5e1f10a17eeb2d33e2ffc5606f617ab978d902448d554-21-e77db7b63c3778a39b81368025361bd75a9c375c95a24a9b519402a99220e7e5-image.png) 9. **IMPORTANT**: From the application resource page, get the private IP of the Voyage endpoint by clicking on **Parameters and Outputs** under the **Settings** section. Take note of the private IP (”privateIPAddress” output); you will need this IP address to invoke the application endpoints. ![](https://files.readme.io/ea76b2fad26c5c513b220e5ffc1f8bb373ca94122cd2c30ed19b1905c33dbda0-22-c94089609500c6126b3f83d1fda9e7f75744995aa9fa13cbf43f33f78eb8d99a-image.png) ![](https://files.readme.io/35dd0543c3473fd0cdc3193723644791aeaf69134d79176e4b57e7b226d6abfa-23-e2546c18acc5212d72950134e775ec7a76b3011c3ad3e1d099b0ef45e9e76cbd-image.png) Accessing Voyage API Endpoints [](#accessing-voyage-api-endpoints) ====================================================================== Once deployed, the Voyage API endpoints can be accessed via HTTP requests by any service within the same virtual network. Create Bastion Virtual Machine [](#create-bastion-virtual-machine) ---------------------------------------------------------------------- To demonstrate how to access the Voyage API endpoints, we will create a bastion virtual machine within the same virtual network. If you’d like to use or have other existing machines/servers in the network to query the model endpoints, you can skip this section and jump to the [Invoke Voyage API Endpoint](#invoke-voyage-api-endpoint) section. 1. From the Azure portal home, click **Create a resource**. ![](https://files.readme.io/f8d0fa1e9287a016143308dc7ee48e45c0b8d43e0d9598572066a288e9be955b-24-19f56d7ce7a991a93ad62a0b4dc6a80fecbf796985fbd573ded38c0b83aae103-image.png) 2. Search for “virtual machine”. Select **Virtual machine** from the results. ![](https://files.readme.io/bf693f97df9e00ccda37c94af24be295e6f27cbe6aa261ea0d3c15d1cb083c6a-25-37b32d246ddc359d64575d026a03c40eb406ea918d203928c69ddb24963023c9-image.png) 3. Click **Create**. ![](https://files.readme.io/02974d104f31157473991d20aad181d62693322fd86309198e977c251773d272-26-e69f3d70639926612f9a66a5ab633eadb43da27eb66bf293aae41379ae70e72f-image.png) We will walk through a low-cost virtual machine configuration for demo purposes only. Advanced users are welcome to make customized configurations. ### Basics [](#basics) Fill out the fields in the **Basics** tab. Once complete, click **Next** to move to **Disks**. ![](https://files.readme.io/17f67cf56fccadd607b97b498107d6c1b29bba2b6ffa03d116cdcb4924fcb693-27-4828fa782ceefde617eec94197b4f8c721e3261ad88ddea12e5a80e555eb7c29-image.png) | **Field** | **Value** | | --- | --- | | **Subscription** | Select your subscription. | | **Resource group** | Select the resource group that contains your Voyage API endpoint. | | **Virtual machine name** | Provide a name for your virtual machine. | | **Region** | Select the region of your Voyage API endpoint. | | **Availability options** | Select “No infrastructure redundancy required” | | **Security type** | Select “Standard”. | | **Image** | Select “Ubuntu Server 24.04 LTS - x64 Gen2”. | | **VM architecture** | Select “x64”. | | **Run with Azure Spot discount** | Leave box unchecked. | | **Size** | Select “Standard\_B1s - 1 vcpu, 1GiB memory”. | | **Enable Hibernation** | Leave box unchecked. | | **Authentication type** | Select “SSH public key”. | | **Username** | Enter “voyage”. | | **SSH public key source** | Select “Generate new key pair”. | | **SSH Key Type** | Select “RSA SSH Format”. | | **Key pair name** | Enter “vm-voyage\_key”. | | **Public inbound ports** | Select “Allow selected ports”. | | **Select inbound ports** | Select “SSH (22)”. | ### Disks [](#disks) Fill out the fields in the **Disks** tab. Once complete, click **Next** to move to **Networking**. ![](https://files.readme.io/c5099e85c5e9ee96e6e736a6f2144e88220c94c32118aab56836d9bbc62773bd-28-3589c1ac0df326385f2b0c4caeb76963f297d00d43723333985a2e5dc1fc1c6d-image.png) | **Field** | **Value** | | --- | --- | | **Encryption at host** | Leave box unchecked. | | **OS disk size** | Select “Image default (30 GiB)”. | | **OS disk type** | Select “Standard HDD (locally-redundant storage)”. | | **Delete with VM** | Check the box. | | **Key management** | Select “Platform-managed key”. | | **Enable Ultra Disk compatibility** | Leave box unchecked. | ### Networking [](#networking) Fill out the fields in the **Networking** tab. Once complete, click **Review + create**. ![](https://files.readme.io/146842d7a029274fd29e983a11a027d02c895d097c7dc089ae3e687eae0aec33-29-ac559c4050f1a8404ead35276f76670e480de5c8f08c22810a652d42a5264b6e-image.png) | **Field** | **Value** | | --- | --- | | Virtual network | **IMPORTANT**. Select the virtual network in which your Voyage API endpoint is deployed. | | Subnet | **IMPORTANT**. Select the subnet within the virtual network in which your Voyage API endpoint is deployed. | | Public IP | **IMPORTANT**. Create a new public IP. If you do not create this, you will not be able to access your virtual machine from outside the virtual network. | | NIC network security group | Select “Basic”. | | Public inbound ports | Select “Allow selected port”. | | Select inbound ports | Select “SSH (22)”. | | Delete public IP and NIC when VM is deleted | Leave box unchecked. | | Enable accelerated networking | Leave box unchecked. | | Load balancing options | Select “None”. | ### Review and Create [](#review-and-create) 1. In the **Review + create** tab, review your configuration and if it looks good, click **Create**. Otherwise, go to previous tabs to make changes; you can click on the **Previous** button at the bottom or directly click on the tab of interest. ![](https://files.readme.io/0b90c0c44ff0271102db31c4e37e824030c24306f0182da3c015dc17cf10a297-30-3221f77a902ad4c823b73ff06c1be83ac9661ded41fcb9ae03edd8caaf1f2504-image.png) 2. A new **Generate new key pair** modal will appear. Click **Download private key and create resource** to save the key to your local machine. Note where you saved the key. ![](https://files.readme.io/22aae4743ba74276d69baea03c950dc15669e773a9db3ea715ea7cf552cde02e-31-f1b7922951d2b3b1d206536596ba2e5f2c5cade23d51770f97dabfd576f03535-image.png) 3. Verify your bastion virtual machine creation. As it is being created, you will see the status as "Deployment is in progress." You will see the status change to "Your deployment is complete" when your bastion virtual machine has been successfully created. Click **Go to resource** to get your bastion virtual machine public IP address. ![](https://files.readme.io/4c5c6e2e636b9c615612db61d220bda3413d08baed9f9e966b911cb55cba6c4a-32-c0d0b5f3efc8abe1793eb87eae0117e6cd25489ba4f8761adf912b97834fa951-image.png) 4. In your bastion virtual machine page, you can find the public IP address in the **Overview** section, under **Essentials**. ![](https://files.readme.io/1afdff8a906e187859aa9c1475b99897aaa232afb99f80a46a3fab773674d1ff-33-b29341b4397dac410b0f387f7788258b881f05513bfc377f89cc4c977bbcbf2e-image.png) 5. Update the permissions of the SSH private key you downloaded to read-only: `chmod 400 vm-voyage_key.pem` 6. Securely login into the bastion virtual machine shell (SSH) using your private key. The format is: `ssh -i {private-ssh-key} voyage@{bastion-vm-public-ip-address}`. For example: `ssh -i vm-voyage_key.pem [[email protected]](/cdn-cgi/l/email-protection) `. Invoke Voyage API Endpoint [](#invoke-voyage-api-endpoint) -------------------------------------------------------------- 1. Verify Voyage API is reachable with the following endpoint: `http://{voyage-app-vnet-private-ip}/info`. For example, using cURL: `curl -X POST http://10.0.0.4/info`. An example successful response: "Voyage AI model \[voyage-multilingual-2\] up and running!" 2. You can make a request to the Voyage API with the following endpoint: `http://{voyage-app-vnet-private-ip}/embeddings`. For example, using cURL: cURL `curl -X POST "http://10.0.0.4/embeddings" -H "Content-Type: application/json" -d '{ "input": ["Best-in-class embedding models and rerankers"], "input_type": "query" }'` An example response: cURL `{"data":[{"object":"embedding","embedding":["-0.031104945",...,"-0.037366055"],"index":0}],"object":"list","model":"voyage-multilingual-2","usage":{"total_tokens":13}}` For all the endpoint parameters, please see our [API reference](/reference/) . Delete a Voyage API Application [](#delete-a-voyage-api-application) ======================================================================== To delete your Voyage API endpoint, go to your application and click the **Delete** button in the **Overview** section. A confirmation modal will appear; click **Yes** in the modal to confirm the deletion. Note, you do not need to worry about deleting the associated managed resource group; it will be deleted as part of the application deletion. ![](https://files.readme.io/e47bb5b8e9088f590d8b1b460163b6b360c0a03baf04cebc9f88bc9331af36ba-34-619f7f35bd6e9a1789061c793d19690c6d9eaf3e940a0767a1f774bb079763f7-image.png) Delete Bastion Virtual Machine [](#delete-bastion-virtual-machine) ---------------------------------------------------------------------- If you provisioned a bastion virtual machine, be sure to delete the virtual machine if you no longer need it. It will incur wasteful costs, potentially leading to unexpected charges. 1. Go to your bastion virtual machine and click the **Delete** button in the **Overview** section. A delete panel will appear. ![](https://files.readme.io/4b6ff53f2c36a1251569f2148bca43121d452f6d89ebb1d23f51aebd9e7c0a6e-35-42681d59da66b617a69d19b255d70a8e922e2a907000142a58c4c12dc4f95529-image.png) 2. In the delete panel, ensure all the virtual machine resources are deleted by checking all the checkboxes shown in the screenshot below: **Apply force delete**, **OS disk**, **Network interfaces**, **Public IP addresses**. Check the acknowledgment checkbox: "I have read and understand that this virtual machine as well as any selected associated resources listed above will be deleted." Finally, click **Delete**. ![](https://files.readme.io/04ef6f6c62e538788990c6bd383007e8a2a6846fad249f9e043c8728899295e6-36-4b73f5a126dbffdf5315f55e4508101bd8723fac6921a3b54e6c437b1d2fc783-image.png) If you need assistance subscribing and deploying a Voyage Azure managed application from the Azure Marketplace, please send an email to [\[email protected\]](/cdn-cgi/l/email-protection#2e4d41405a4f4d5a6e5841574f494b4f47004d4143)  or join our [Discord](https://discord.gg/zAU7GQEmvT) . Updated 3 months ago * * * --- # Integrations Voyage embeddings have been integrated into a number of popular community libraries and vector databases, including: * DataStax: [Astra DB Serverless Vectorize Embedding Provider](https://docs.datastax.com/en/astra-db-serverless/integrations/embedding-providers/voyage-ai.html) * Epsila: [Platform Integration](https://epsilla-inc.gitbook.io/epsilladb/platform/integrations/voyage-ai) * Haystack: [Embedder and Reranker](https://haystack.deepset.ai/integrations/voyage) * LanceDB: [Embedding Function](https://lancedb.github.io/lancedb/embeddings/available_embedding_models/text_embedding_functions/voyageai_embedding/) , [Reranker](https://lancedb.github.io/lancedb/reranking/voyageai/) * LangChain: [Embeddings](https://python.langchain.com/docs/integrations/text_embedding/voyageai/) , [Rerank Document Compressor](https://python.langchain.com/v0.1/docs/integrations/document_transformers/voyageai-reranker/) * LlamaIndex: [Embeddings](https://docs.llamaindex.ai/en/stable/examples/embeddings/voyageai/#voyage-embeddings) , [Rerank Node Postprocessor](https://docs.llamaindex.ai/en/stable/examples/node_postprocessor/VoyageAIRerank/) * MyScale: [EmbedText Function](https://myscale.com/docs/en/functions/ai-functions/embedding-functions/#voyage-ai-embedding) * Pgai: [Vectorizer](https://github.com/timescale/pgai/blob/main/docs/vectorizer-quick-start-voyage.md) * Redis: [Vectorizer](https://docs.redisvl.com/en/latest/user_guide/vectorizers_04.html#voyageai) * Unstructured: [Embedding Encoder](https://docs.unstructured.io/open-source/core-functionality/embedding#voyageaiembeddingencoder) * Vectorize: [Platform Integration](https://docs.vectorize.io/integrations/ai-platforms) * Weaviate: [Vectorizer Module](https://weaviate.io/developers/weaviate/modules/retriever-vectorizer-modules/text2vec-voyageai) , [Ranker Module](https://weaviate.io/developers/weaviate/modules/retriever-vectorizer-modules/reranker-voyageai) * Zilliz: [Cloud Pipeline Support](https://zilliz.com/product/integrations/voyage-ai) Voyage’s Python client is compatible with virtually all Python libraries, including, but not limited to, [IBM](https://www.ibm.com/watsonx/partners#Embeddings) , [Milvus](https://milvus.io/docs/embed-with-voyage.md) , [Pinecone](https://docs.pinecone.io/integrations/voyage) , and [Qdrant](https://qdrant.tech/documentation/embeddings/voyage/) . If you would like to add a new library here or need assistance with the integration, please contact the [Voyage team](/cdn-cgi/l/email-protection#a6c5c9c8d2c7c5d2e6d0c9dfc7c1c3c7cf88c5c9cb) ! Updated about 2 months ago * * * --- # Community SDKs Here are various SDKs contributed by community members: * [Ruby](https://voyageai.ksylvest.com/) by [Kevin Sylvestre](https://ksylvest.com/) * [Vercel](https://www.npmjs.com/package/voyage-ai-provider) by [Vivek Patel](https://patelvivek.dev) * [Go](https://pkg.go.dev/github.com/austinfhunter/voyageai) by [Austin Hunter](https://austinhunter.dev) * [Typescript](https://github.com/lunary-ai/abso) by [Lunary](https://lunary.ai/) A big thank you to all of the authors above for their contributions. If you would like to get your SDK listed here, please reach out to [\[email protected\]](/cdn-cgi/l/email-protection#8ae9e5e4feebe9fecafce5f3ebedefebe3a4e9e5e7) . Updated 5 days ago * * * --- # Discord * **Already familiar with Discord?** Join our Discord [here](https://discord.gg/zAU7GQEmvT) . * **New to Discord?** * Download and install Discord: [https://discord.com/download](https://discord.com/download) * Register a Discord account. * Join our Discord [here](https://discord.gg/zAU7GQEmvT) . Updated 7 months ago * * * --- # FAQ General [](#general) ======================== ### Why do Voyage embeddings have superior quality? [](#why-do-voyage-embeddings-have-superior-quality) Embedding models, much like generative models, rely on powerful neural networks (often transformer-based) to capture and compress semantic context. And, much like generative models, they’re incredibly hard to train. Voyage is a team of leading AI researchers who each have experience in training embedding models for 5+ years. We refine every component, from model architecture and data collection to selecting suitable loss functions and optimizers. Please see our [blog](https://blog.voyageai.com/) for more details. * * * Model [](#model) ==================== ### What text embedding models are available, and which one should I use? [](#what-text-embedding-models-are-available-and-which-one-should-i-use) For general-purpose embedding, our recommendation is `voyage-3-large` for the best quality, `voyage-3-lite` for the lowest latency and cost, and `voyage-3` for a balance between the two. Notably, `voyage-3` outperforms all major competitors in retrieval quality while being offered at a lower price point. For retrieval, please use the `input_type` parameter to specify whether the text is a query or document, which adds instructions on the backend. If your application is in a domain addressed by one of our domain-specific embedding models, we recommend using that model. Specifically: * `voyage-law-2` is recommended for retrieval tasks in the legal domain. * `voyage-code-3` is recommended for code-related tasks and programming documentation. * `voyage-finance-2` is recommended for finance-related tasks. ### Which similarity function should I use? [](#which-similarity-function-should-i-use) You can use Voyage embeddings with either dot-product similarity, cosine similarity, or Euclidean distance. An explanation about embedding similarity can be found [here](https://www.pinecone.io/learn/vector-similarity/) . Voyage AI embeddings are normalized to length 1, which means that: * Cosine similarity is equivalent to dot-product similarity, while the latter can be computed more quickly. * Cosine similarity and Euclidean distance will result in the identical rankings. ### What is the relationship between characters, words, and tokens? [](#what-is-the-relationship-between-characters-words-and-tokens) Please see this [section](/docs/tokenization#tokens-words-characters) . ### When and how should I use the `input_type` parameter? [](#when-and-how-should-i-use-the-input_type-parameter) For retrieval/search tasks and use cases (e.g., RAG), where a "query" is used to search for relevant information among a collection of data referred to as "documents," we recommend specifying whether your inputs are intended as queries or documents by setting `input_type` to `query` or `document`, respectively. Do not omit `input_type` or set `input_type=None`. Specifying whether inputs are queries or documents helps the model create more effective vectors tailored for retrieval/search tasks. When using the `input_type` parameter, special prompts are prepended to the input text prior to embedding. Specifically: > 📘 > > Prompts associated with `input_type` > > > ------------------------------------------ > > * For a query, the prompt is “Represent the query for retrieving supporting documents: “. > * For a document, the prompt is “Represent the document for retrieval: “. > > Example > > * When `input_type="query"`, a query like "When is Apple's conference call scheduled?" will become "**Represent the query for retrieving supporting documents:** When is Apple's conference call scheduled?" > * When `input_type="document"`, a query like "Apple’s conference call to discuss fourth fiscal quarter results and business updates is scheduled for Thursday, November 2, 2023 at 2:00 p.m. PT / 5:00 p.m. ET." will become "**Represent the document for retrieval:** Apple’s conference call to discuss fourth fiscal quarter results and business updates is scheduled for Thursday, November 2, 2023 at 2:00 p.m. PT / 5:00 p.m. ET." `voyage-large-2-instruct`, as the name suggests, is trained to be responsive to additional instructions that are prepended to the input text. For classification, clustering, or other [MTEB](https://huggingface.co/mteb) subtasks, please use the instructions [here](https://github.com/voyage-ai/voyage-large-2-instruct) . ### What is the total number of tokens for the rerankers? [](#what-is-the-total-number-of-tokens-for-the-rerankers) We define the total number of tokens as the “(number of query tokens × the number of documents) + sum of the number of tokens in all documents". This cannot exceed 600K for `rerank-2` and `rerank-2-lite`, and 300K for `rerank-1` and `rerank-lite-1`. However, if you are latency-sensitive, we recommend you to use `rerank-2-lite` and use no more than 200K total tokens per request. ### What is quantization and output data types? [](#what-is-quantization-and-output-data-types) Quantization in embeddings converts high-precision values, like 32-bit single-precision floating-point numbers, to lower-precision formats such as 8-bit integers or 1-bit binary values, reducing storage, memory, and costs by 4x and 32x, respectively. Supported Voyage models enable quantization by specifying the output data type with the `output_dtype` parameter: * `float`: Each returned embedding is a list of 32-bit (4-byte) [single-precision floating-point](https://en.wikipedia.org/wiki/Single-precision_floating-point_format) numbers. This is the default and provides the highest precision / retrieval accuracy. * `int8` and `uint8`: Each returned embedding is a list of 8-bit (1-byte) integers ranging from -128 to 127 and 0 to 255, respectively. * `binary` and `ubinary`: Each returned embedding is a list of 8-bit integers that represent bit-packed, quantized single-bit embedding values: `int8` for `binary` and `uint8` for `ubinary`. The length of the returned list of integers is 1/8 of the actual dimension of the embedding. The `binary` type uses the offset binary method, which you can learn more about in the FAQ [below](/docs/faq#what-is-offset-binary) . _Binary quantization example_Consider the following eight embedding values: -0.03955078, 0.006214142, -0.07446289, -0.039001465, 0.0046463013, 0.00030612946, -0.08496094, and 0.03994751. With binary quantization, values less than or equal to zero will be quantized to a binary zero, and positive values to a binary one, resulting in the following binary sequence: 0, 1, 0, 0, 1, 1, 0, 1. These eight bits are then packed into a single 8-bit integer, 01001101 (with the leftmost bit as the most significant bit). * `ubinary`: The binary sequence is directly converted and represented as the unsigned integer (`uint8`) 77. * `binary`: The binary sequence is represented as the signed integer (`int8`) -51, calculated using the [offset binary](#what-is-offset-binary) method (77 - 128 = -51). ### What is offset binary? [](#what-is-offset-binary) [Offset binary](https://en.wikipedia.org/wiki/Offset_binary) is a method for representing negative numbers in binary form (i.e., [signed number representations](https://en.wikipedia.org/wiki/Signed_number_representations) ). This approach is used when representing quantized binary embedding values, specifically when the `output_dtype` parameter is set to `binary`. The binary values are bit-packed, with each 8-bit sequence represented as an integer calculated using the offset binary method. In this method, an offset is _added_ to an integer before converting to binary and _subtracted_ when converting from binary to a signed integer. For signed 8-bit integers, which have a range of -128 to 127, the offset is typically 128. _Signed integer to binary example_ To represent -32 as an 8-bit binary number: 1. Add the offset (128) to -32, resulting in 96. 2. Convert 96 to binary: 01100000. _Binary to signed integer example_ To determine the signed integer from the 8-bit binary number 01010101: 1. Convert it directly to an integer: 85. 2. _Subtract_ the offset (128) from 85, resulting in -43. ### How can I truncate Matryoshka embeddings? [](#how-can-i-truncate-matryoshka-embeddings) [Matryoshka learning](https://arxiv.org/abs/2205.13147) creates embeddings with coarse-to-fine representations within a single vector. Voyage models, such as `voyage-3-large`, that support multiple output dimensions generate such Matryoshka embeddings. You can truncate these vectors by keeping the leading subset of dimensions. For example, the following Python code demonstrates how to truncate 1024-dimensional vectors to 256 dimensions: Python `import voyageai import numpy as np def embd_normalize(v: np.ndarray) -> np.ndarray: """ Normalize the rows of a 2D numpy array to unit vectors by dividing each row by its Euclidean norm. Raises a ValueError if any row has a norm of zero to prevent division by zero. """ row_norms = np.linalg.norm(v, axis=1, keepdims=True) if np.any(row_norms == 0): raise ValueError("Cannot normalize rows with a norm of zero.") return v / row_norms vo = voyageai.Client() # Generate voyage-code-3 vectors, which by default are 1024-dimensional floating-point numbers embd = vo.embed(['Sample text 1', 'Sample text 2'], model='voyage-3-large').embeddings # Set shorter dimension short_dim = 256 # Resize and normalize vectors to shorter dimension resized_embd = embd_normalize(np.array(embd)[:, :short_dim]).tolist()` * * * Usage [](#usage) ==================== ### How do I get the Voyage API key? [](#how-do-i-get-the-voyage-api-key) Upon creating an account, we instantly generate an API key for you. Once signed in, access your API key by clicking the "Create new API key" button in the dashboard. ### What are the rate limits for the Voyage API? [](#what-are-the-rate-limits-for-the-voyage-api) Please see the [rate limit guide](/docs/rate-limits) . ### How can I retrieve nearest text quickly if I have a large corpus? [](#how-can-i-retrieve-nearest-text-quickly-if-i-have-a-large-corpus) To efficiently retrieve the nearest texts from a sizable corpus, you can use a vector database. We recommend using: * [MongoDB](https://www.mongodb.com/products/platform/atlas-vector-search) , a versatile database with best-in-class vector search capabilities MongoDB and the document data model are the best for representing all the variety of data in the world—whether it's structured, semi-structured, or unstructured (vector) data. Here are some other choices: * [Pinecone](https://www.pinecone.io/) , a fully managed vector database * [Zilliz](https://zilliz.com) , a vector database for enterprise * [Chroma](https://www.chroma.com/) , an open-source embeddings store * [Elasticsearch](https://www.elastic.co/?utm_campaign=B-Stack-Trials-AMER-US-W&utm_content=Stack-Core-EXT&utm_source=google&utm_medium=cpc&device=c&utm_term=elasticsearch&gad_source=1&gclid=CjwKCAjw1t2pBhAFEiwA_-A-NNbaQ9G9Wtujlv6oM9PI_fT5IJtHYLFrKw-XtkDmfXI9SjLLdVl5PhoCr7QQAvD_BwE) , a popular search/analytics engine and vector database * [Milvus](https://zilliz.com/what-is-milvus?utm_source=google&utm_medium=cpc&utm_campaign=WP_Milvus_Search&utm_content=WP_Milvus_TKAG&utm_term=milvus&utm_campaign=WP_Milvus_Search&utm_source=adwords&utm_medium=ppc&hsa_acc=3636806625&hsa_cam=20033301076&hsa_grp=151285141111&hsa_ad=656191017493&hsa_src=g&hsa_tgt=kwd-2942447257&hsa_kw=milvus&hsa_mt=e&hsa_net=adwords&hsa_ver=3&gad_source=1&gclid=CjwKCAjw1t2pBhAFEiwA_-A-NMWAZ5gCgJa33WVewl4QS2MUURaM7puFTbRtxOSh8J3MzsglrfeowhoC6h0QAvD_BwE) , a vector database built for scalable similarity search * [Qdrant](https://qdrant.tech/) , a vector search engine * [Weaviate](https://weaviate.io) , an open source, AI-native vector database ### What is Base64 encoding? [](#what-is-base64-encoding) Base64 encoding converts binary data, such as images, into a sequence of printable characters. It is commonly used to encode data that needs to be stored or transferred over media designed to handle textual data, such as HTTPS requests. The encoding ensures that the data remains intact without modification during transmission. Python comes with a native [`base64` library](https://docs.python.org/3/library/base64.html) . Here’s an example using the Python `base64` library to create a Base64 image: Python `import base64 # Replace 'banana.jpg' with the path to your image file image_path = 'banana.jpg' # Open the image file in binary mode with open(image_path, "rb") as image_file: # Read the binary data image_data = image_file.read() # Encode the binary data to Base64 base64_encoded_data = base64.b64encode(image_data) # Convert the base64 bytes to a string base64_string = base64_encoded_data.decode("utf-8")` Here’s how you can create a Base64 image with a function and vectorize it with `voyage-multimodal-3`: PythonOutput `import base64 import mimetypes import voyageai vo = voyageai.Client() # This will automatically use the environment variable VOYAGE_API_KEY. # Alternatively, you can use vo = voyageai.Client(api_key="") # Convert image to Base64 image def b64encode_image(image_path): with open(image_path, "rb") as image_file: return base64.b64encode(image_file.read()).decode('utf-8') # Image path image_path = 'banana.jpg' # Get media type mediatype, _ = mimetypes.guess_type(image_path) # Input with text and a Base64 encoded image inputs = [ { "content": [ { "type": "text", "text": "This is a banana." }, { "type": "image_base64", "image_base64": f"data:{mediatype};base64,{b64encode_image(image_path)}" } ] } ] # Vectorize inputs result = vo.multimodal_embed(inputs, model="voyage-multimodal-3", input_type="document") print(result.embeddings)` `[ [0.0181884765625, -0.01019287109375, 0.021240234375,...] ]` ### What is Pillow? [](#what-is-pillow) [Pillow](https://python-pillow.org/) is a widely used Python library for image processing, providing powerful tools to open, manipulate, and save various image file formats. It is popular because it is simple to use, well-documented, and covers many use cases. The following example demonstrates some common tasks with Pillow, such as opening, showing, resizing, and saving images. It also demonstrates retrieving metadata like width, height, and pixel count. Check out the [Pillow docs](https://pillow.readthedocs.io/en/stable/) for more details. Python `from PIL import Image # Open an image img = Image.open('banana.jpg') # Show image img.show() # Get image width, height, and pixels width, height = img.size # Get image width and height num_pixels = width * height # Calculate number of pixels print(f"Width: {width} pixels") print(f"Height: {height} pixels") print(f"Total Pixels: {num_pixels}") # Resize image to 200 pixels by 150 pixels new_size = (200, 150) # Example size: 200x150 pixels # Resize the image resized_img = img.resize(new_size) # Show resized image resized_img.show() # Save resized image resized_img.save('banana-200px-by-150px.jpg')` * * * Billing and Budgets [](#billing-and-budgets) ================================================ ### When will I receive the bill? [](#when-will-i-receive-the-bill) The first 200 million tokens are free for every account, and subsequent usage is priced on a per-token basis. See our [Pricing page](/docs/pricing) for details You can add payment methods to your account in the [dashboard](https://dash.voyageai.com) . We will bill monthly. You can expect a credit card charge around the 2nd of each month for the usage of the past month. ### How can I set up prepaid billing? [](#how-can-i-set-up-prepaid-billing) Prepaid billing allows API users to pre-purchase usage credits, which are applied to their monthly invoice. API usage is first deducted from prepaid credits, and any usage exceeding the purchased credits is billed separately. Prepaid billing helps developers plan upfront, providing greater predictability for budgeting and spend management. **Setting up prepaid billing**. To prepay and add a credit balance, you must have a payment method on file. Navigate to the [**Billing**](https://dashboard.voyageai.com/organization/billing) section under Organization in the navigation sidebar. In the **Overview** tab, click **Add to credit balance** to open the **Configure Payment** modal. ![](https://files.readme.io/f40d41f609ce039ab91c7f6ef7ca98d69a0db985207a4a2fb822469cf28a23ba-add-to-credit-balance.png) Enter the amount you want to prepay in the **Initial credit purchase** field. Optionally, you can enable automatic recharge and specify an amount to restore your credit balance when it falls below a set threshold. ![](https://files.readme.io/1f9f6cba6e10a71fd7ead1b127166e9fe0a0076c37da7dc45d538c22e52a70f9-configure-payment.png) **Service Credit Terms and Expiration**. Please note that any purchased credits will expire after 1 year, and they are non-refundable. * * * Others [](#others) ====================== ### What are your policies in regard to protecting customer data? [](#what-are-your-policies-in-regard-to-protecting-customer-data) For Voyage-hosted model API endpoints, customers can opt-out from Voyage storing and using their data for future model training so that there is a zero-day retention of the data. To opt out, you must have a payment method on file and be an organization Admin. Log into your account dashboard and navigate to the [**Terms of Service**](https://dashboard.voyageai.com/organization/tos) section under **Organization** in the navigation sidebar. Scroll to the bottom and toggle the **Opted In** slider to **Opted Out**. ![](https://files.readme.io/b4921e853b7b71dd58277dad680af87d83fa73f39e80464e7639bc0e19c8620c-opt-out.png) > 🚧 > -- > > You won’t be able to opt-in again in the dashboard after you opt out. If you are interested in opting-in again after opting-out, please contact [\[email protected\]](/cdn-cgi/l/email-protection#325e5755535e72445d4b535557535b1c515d5f) > . ### Is fine-tuning available? [](#is-fine-tuning-available) Currently we offer fine-tuned embeddings through subscription. Please email Tengyu Ma (CEO) at [\[email protected\]](/cdn-cgi/l/email-protection#61150c0021170e1800060400084f020e0c) if you are interested. ### How to contact us? [](#how-to-contact-us) Please email us at [\[email protected\]](/cdn-cgi/l/email-protection#54373b3a2035372014223b2d353331353d7a373b39) for inquiries and customer support. ### How to get updates from Voyage? [](#how-to-get-updates-from-voyage) Follow us on [twitter](https://x.com/VoyageAI) and/or [linkedin](https://www.linkedin.com/company/voyageai) for more updates! To subscribe to our newsletter, feel free to send us an email at [\[email protected\]](/cdn-cgi/l/email-protection#6e0d01001a0f0d1a2e1801170f090b0f07400d0103) . Updated 28 days ago * * * ---