# Table of Contents - [Online Evals & Monitoring for Agents in Production (Mosaic AI) | Cookbooks | Arize Docs](#online-evals-monitoring-for-agents-in-production-mosaic-ai-cookbooks-arize-docs) - [Arize Private Connect | Arize Docs](#arize-private-connect-arize-docs) - [AI Powered Trace Analysis | Arize Docs](#ai-powered-trace-analysis-arize-docs) - [AI Powered Search & Filter | Arize Docs](#ai-powered-search-filter-arize-docs) - [Arize AI for Agents | Arize Docs](#arize-ai-for-agents-arize-docs) - [AI Span Analysis & Evaluation | Arize Docs](#ai-span-analysis-evaluation-arize-docs) - [Alyx - AI Engineering Agent | Arize Docs](#alyx-ai-engineering-agent-arize-docs) - [Alyx: Eval & RAG Analysis | Arize Docs](#alyx-eval-rag-analysis-arize-docs) - [Arize AI Named to Forbes AI 50 List of Most Promising Artificial Intelligence Companies of 2021 - Arize AI](#arize-ai-named-to-forbes-ai-50-list-of-most-promising-artificial-intelligence-companies-of-2021-arize-ai) - [Alyx: ArizeQL Generator | Arize Docs](#alyx-arizeql-generator-arize-docs) - [Arize AI Wins 2020 AI TechAward for Enterprise AI - Arize AI](#arize-ai-wins-2020-ai-techaward-for-enterprise-ai-arize-ai) - [Pandas Batch Logging | Arize Docs](#pandas-batch-logging-arize-docs) - [Single Record Logging | Arize Docs](#single-record-logging-arize-docs) - [Alyx: ArizeQL Generator | Arize Docs](#alyx-arizeql-generator-arize-docs) - [Arize Receives Certifications Validating Health Information Security for HIPAA Compliance - Arize AI](#arize-receives-certifications-validating-health-information-security-for-hipaa-compliance-arize-ai) - [Arize AI Raises $19 Million Series A As Organizations Move To Address ML Observability, the Missing Foundational Piece of ML infrastructure - Arize AI](#arize-ai-raises-19-million-series-a-as-organizations-move-to-address-ml-observability-the-missing-foundational-piece-of-ml-infrastructure-arize-ai) - [Arize Audit Log | Arize Docs](#arize-audit-log-arize-docs) - [Arize Release Notes: Aug 8, 2024 - Arize AI](#arize-release-notes-aug-8-2024-arize-ai) - [Arize AI Brings LLM Evaluation, Observability To Microsoft Azure AI Model Catalog - Arize AI](#arize-ai-brings-llm-evaluation-observability-to-microsoft-azure-ai-model-catalog-arize-ai) - [Arize AI’s Next Era of Growth - Arize AI](#arize-ai-s-next-era-of-growth-arize-ai) - [Arize, Vertex AI API: Evaluation Workflows to Accelerate Generative App Development and AI ROI - Arize AI](#arize-vertex-ai-api-evaluation-workflows-to-accelerate-generative-app-development-and-ai-roi-arize-ai) - [Arize Release Notes: Test Tasks, Filter Experiments, and More - Arize AI](#arize-release-notes-test-tasks-filter-experiments-and-more-arize-ai) - [Arize AI Accelerates Enterprise AI Adoption On-Premises With NVIDIA - Arize AI](#arize-ai-accelerates-enterprise-ai-adoption-on-premises-with-nvidia-arize-ai) - [Arize AI Raises $70M Series C to Build the Gold Standard for AI Evaluation & Observability - Arize AI](#arize-ai-raises-70m-series-c-to-build-the-gold-standard-for-ai-evaluation-observability-arize-ai) - [Arize AI Named TiE50 Award Winner at TiEcon - Arize AI](#arize-ai-named-tie50-award-winner-at-tiecon-arize-ai) - [Arize AI Partners with Algorithmia to Enable Better MLOps and Observability for Enterprises - Arize AI](#arize-ai-partners-with-algorithmia-to-enable-better-mlops-and-observability-for-enterprises-arize-ai) - [Arize AI Unveils Prompt Engineering and Retrieval Tracing Workflows For LLM Troubleshooting - Arize AI](#arize-ai-unveils-prompt-engineering-and-retrieval-tracing-workflows-for-llm-troubleshooting-arize-ai) - [Arize Platform demo - Arize AI](#arize-platform-demo-arize-ai) - [Arize AI and Infogain Partner to Accelerate Enterprise AI Outcomes With Ignis - Arize AI](#arize-ai-and-infogain-partner-to-accelerate-enterprise-ai-outcomes-with-ignis-arize-ai) - [Arize Partners with UbiOps to Accelerate Model Building & Deployment - Arize AI](#arize-partners-with-ubiops-to-accelerate-model-building-deployment-arize-ai) - [Arize Phoenix: Datasets - Arize AI](#arize-phoenix-datasets-arize-ai) - [Arize AI Is Growing! - Arize AI](#arize-ai-is-growing-arize-ai) - [Arize AI Partners with Spell to Bring ML Observability to the Spell Platform - Arize AI](#arize-ai-partners-with-spell-to-bring-ml-observability-to-the-spell-platform-arize-ai) - [Arize - Quickstart Guide - Arize AI](#arize-quickstart-guide-arize-ai) - [Arize AI Selected For insideBIGDATA's Impact 50 List - Arize AI](#arize-ai-selected-for-insidebigdata-s-impact-50-list-arize-ai) - [Hugging Face + Arize: Partnership and Code Example - Arize AI](#hugging-face-arize-partnership-and-code-example-arize-ai) - [Arize AI Listed In 2021 Gartner Market Guide for AI Trust, Risk and Security Management (AI TRiSM) - Arize AI](#arize-ai-listed-in-2021-gartner-market-guide-for-ai-trust-risk-and-security-management-ai-trism-arize-ai) - [Arize AI and Paperspace Announce a Partnership to Bring Deep ML Observability Solutions to Data Science Teams - Arize AI](#arize-ai-and-paperspace-announce-a-partnership-to-bring-deep-ml-observability-solutions-to-data-science-teams-arize-ai) - [Arize AI: Support for EU Data Residency - Arize AI](#arize-ai-support-for-eu-data-residency-arize-ai) - [Arize AI + OpenAI - Arize AI](#arize-ai-openai-arize-ai) - [Arize Release Notes: Sep 5, 2024 - Arize AI](#arize-release-notes-sep-5-2024-arize-ai) - [Arize AI + MongoDB: Leveraging Agent Evaluation and Memory to Build Robust Agentic Systems - Arize AI](#arize-ai-mongodb-leveraging-agent-evaluation-and-memory-to-build-robust-agentic-systems-arize-ai) - [Arize Release Notes: AI Search V2, Copilot Updates, and More - Arize AI](#arize-release-notes-ai-search-v2-copilot-updates-and-more-arize-ai) - [Arize Release Notes: Aug 23, 2024 - Arize AI](#arize-release-notes-aug-23-2024-arize-ai) - [Arize Release Notes: New Copilot Skills, Local Explainability, and More. - Arize AI](#arize-release-notes-new-copilot-skills-local-explainability-and-more-arize-ai) - [Arize Release Notes: Embeddings Tracing, Experiments Details, and More. - Arize AI](#arize-release-notes-embeddings-tracing-experiments-details-and-more-arize-ai) - [Arize Release Notes: Copilot Enhancements, Experiment Projects, and More - Arize AI](#arize-release-notes-copilot-enhancements-experiment-projects-and-more-arize-ai) - [Arize Release Notes: Monitor Runtime, Create a Dataset from CSV, and More - Arize AI](#arize-release-notes-monitor-runtime-create-a-dataset-from-csv-and-more-arize-ai) - [Arize Phoenix: 2024 in Review - Arize AI](#arize-phoenix-2024-in-review-arize-ai) - [Embracing Google's Agent-To-Agent (A2A) Protocol - Arize AI](#embracing-google-s-agent-to-agent-a2a-protocol-arize-ai) - [Arize AI Achieves ISO/IEC 27001 Certification - Arize AI](#arize-ai-achieves-iso-iec-27001-certification-arize-ai) - [Arize Release Notes: Prompt Hub, Managed Code Evaluators and More - Arize AI](#arize-release-notes-prompt-hub-managed-code-evaluators-and-more-arize-ai) - [Arize Release Notes: Labeling Queues, Expand/Collapse Rows in Trace Table - Arize AI](#arize-release-notes-labeling-queues-expand-collapse-rows-in-trace-table-arize-ai) - [Arize AI Now Generally Available As Part of Azure Native Integrations - Arize AI](#arize-ai-now-generally-available-as-part-of-azure-native-integrations-arize-ai) - [Self-Improving Agents: Automating LLM Performance Optimization using Arize and NVIDIA NeMo - Arize AI](#self-improving-agents-automating-llm-performance-optimization-using-arize-and-nvidia-nemo-arize-ai) - [Arize AI Introduces Next Generation of Its Machine Learning Observability Platform, Goes Self-Serve For Any Organization Seeking Optimize AI Investments - Arize AI](#arize-ai-introduces-next-generation-of-its-machine-learning-observability-platform-goes-self-serve-for-any-organization-seeking-optimize-ai-investments-arize-ai) - [Arize AI Announces SOC 2 Type II Certification - Arize AI](#arize-ai-announces-soc-2-type-ii-certification-arize-ai) - [Arize AI Recognized For MLOps Innovation in 2022 Artificial Intelligence Breakthrough Awards Program - Arize AI](#arize-ai-recognized-for-mlops-innovation-in-2022-artificial-intelligence-breakthrough-awards-program-arize-ai) - [Arize AI Named To Fast Company’s List of the Best Workplaces for Innovators - Arize AI](#arize-ai-named-to-fast-company-s-list-of-the-best-workplaces-for-innovators-arize-ai) - [Arize AI Expands Partnership with Google Cloud To Accelerate Machine Learning Observability - Arize AI](#arize-ai-expands-partnership-with-google-cloud-to-accelerate-machine-learning-observability-arize-ai) - [Arize AI Launches Industry-First LLM Observability Tool - Arize AI](#arize-ai-launches-industry-first-llm-observability-tool-arize-ai) - [Arize AI Recognized As 2023 “Best MLOps Company” in Sixth Annual AI Breakthrough Awards - Arize AI](#arize-ai-recognized-as-2023-best-mlops-company-in-sixth-annual-ai-breakthrough-awards-arize-ai) - [Arize AI Raises $38 Million Series B To Scale Machine Learning Observability Platform - Arize AI](#arize-ai-raises-38-million-series-b-to-scale-machine-learning-observability-platform-arize-ai) - [Arize AI Honored In “On The Rise” Category of Fast Company’s 2023 World Changing Ideas Awards - Arize AI](#arize-ai-honored-in-on-the-rise-category-of-fast-company-s-2023-world-changing-ideas-awards-arize-ai) - [Arize Debuts Phoenix, the First Open Source Library for Evaluating Large Language Models - Arize AI](#arize-debuts-phoenix-the-first-open-source-library-for-evaluating-large-language-models-arize-ai) - [Arize AI Debuts General Availability of Embedding Drift Measurement - Arize AI](#arize-ai-debuts-general-availability-of-embedding-drift-measurement-arize-ai) - [Arize AI Debuts Monitoring for Unstructured Data - Arize AI](#arize-ai-debuts-monitoring-for-unstructured-data-arize-ai) - [Arize Debuts Data Lake Connectors - Arize AI](#arize-debuts-data-lake-connectors-arize-ai) - [Arize AI Debuts Observe Copilot, Winning “Coolest Technology” at VB Transform’s Innovation Showcase - Arize AI](#arize-ai-debuts-observe-copilot-winning-coolest-technology-at-vb-transform-s-innovation-showcase-arize-ai) - [Arize AI, LlamaIndex Roll Out Joint Platform for Evaluating LLM Applications - Arize AI](#arize-ai-llamaindex-roll-out-joint-platform-for-evaluating-llm-applications-arize-ai) - [Arize:Observe To Gather Top Minds In Generative AI for Day of Learning - Arize AI](#arize-observe-to-gather-top-minds-in-generative-ai-for-day-of-learning-arize-ai) - [Arize AI Debuts Prompt Variable Monitoring - Arize AI](#arize-ai-debuts-prompt-variable-monitoring-arize-ai) - [Arize AI Introduces AI Copilot - Arize AI](#arize-ai-introduces-ai-copilot-arize-ai) - [Arize AI Is Named An Emerging Leader In the Generative AI Engineering Gartner® Emerging Market Quadrant - Arize AI](#arize-ai-is-named-an-emerging-leader-in-the-generative-ai-engineering-gartner-emerging-market-quadrant-arize-ai) - [Arize Premieres Open Source LLM Evals Library and Support for Traces and Spans - Arize AI](#arize-premieres-open-source-llm-evals-library-and-support-for-traces-and-spans-arize-ai) - [Arize AI Acquires Velvet - Arize AI](#arize-ai-acquires-velvet-arize-ai) - [Arize AI Secures $70M Series C to Fix AI’s Biggest Problem: Making LLMs and AI Agents Work in the Real World - Arize AI](#arize-ai-secures-70m-series-c-to-fix-ai-s-biggest-problem-making-llms-and-ai-agents-work-in-the-real-world-arize-ai) - [Arize AI’s AI Engineering Platform for R&D Selected by AFWERX - Arize AI](#arize-ai-s-ai-engineering-platform-for-r-d-selected-by-afwerx-arize-ai) - [Arize AI and Tokyo Electron Device Announce Partnership - Arize AI](#arize-ai-and-tokyo-electron-device-announce-partnership-arize-ai) - [Arize AI Makes Fast Company’s Fifth Annual List of the Best Workplaces for Innovators In the Enterprise Category - Arize AI](#arize-ai-makes-fast-company-s-fifth-annual-list-of-the-best-workplaces-for-innovators-in-the-enterprise-category-arize-ai) - [Arize Platform Demo 2024 - Arize AI](#arize-platform-demo-2024-arize-ai) - [Arize Holiday Special - Arize AI](#arize-holiday-special-arize-ai) - [Arize:Observe - Tecton Workshop - Building Production Ready batch, Streaming, and... in Jupyter Notebook - Arize AI](#arize-observe-tecton-workshop-building-production-ready-batch-streaming-and-in-jupyter-notebook-arize-ai) - [Arize:Observe - Introducing Phoenix - ML Observability in Your Notebook - Arize AI](#arize-observe-introducing-phoenix-ml-observability-in-your-notebook-arize-ai) - [Arize Phoenix OSS - ML Observability in a Notebook - Arize AI](#arize-phoenix-oss-ml-observability-in-a-notebook-arize-ai) - [Arize:Observe - Are You Flying Blind With Your Chatbots - Arize AI](#arize-observe-are-you-flying-blind-with-your-chatbots-arize-ai) - [Prost to improving Model Performance - Arize AI](#prost-to-improving-model-performance-arize-ai) - [Productionizing Machine Learning with Observability, Quality and Flexibility at Scale - Arize AI](#productionizing-machine-learning-with-observability-quality-and-flexibility-at-scale-arize-ai) - [Arize: Observe Unstructured 2022 - Arize AI](#arize-observe-unstructured-2022-arize-ai) - [Arize:Observe Unstructured - How to improve performance of unstructured models with less data - Arize AI](#arize-observe-unstructured-how-to-improve-performance-of-unstructured-models-with-less-data-arize-ai) - [Productionizing Machine Learning with Observability, Quality and Flexibility at Scale - Arize AI](#productionizing-machine-learning-with-observability-quality-and-flexibility-at-scale-arize-ai) - [Arize:Observe Unstructured - A Theory Primer for UMAP: Uniform Manifold Approximation and Projection - Arize AI](#arize-observe-unstructured-a-theory-primer-for-umap-uniform-manifold-approximation-and-projection-arize-ai) - [Arize:Observe Unstructured - Accelerating Machine Learning from Research to Production with Hugging Face - Arize AI](#arize-observe-unstructured-accelerating-machine-learning-from-research-to-production-with-hugging-face-arize-ai) - [Arize:Observe Unstructured - Keynote presentation - Arize AI](#arize-observe-unstructured-keynote-presentation-arize-ai) - [Arize:Observe Unstructured - Powering the Next Generation of Products with AI - Arize AI](#arize-observe-unstructured-powering-the-next-generation-of-products-with-ai-arize-ai) - [See It In Action: Arize Platform Demo, Live Q&A Recording - Arize AI](#see-it-in-action-arize-platform-demo-live-q-a-recording-arize-ai) - [Arize:Observe Unstructured - Handling the Challenges of Unstructured Data, The Unsung Hero of Machine Learning - Arize AI](#arize-observe-unstructured-handling-the-challenges-of-unstructured-data-the-unsung-hero-of-machine-learning-arize-ai) - [Arize:Observe - Keynote - Arize AI](#arize-observe-keynote-arize-ai) - [See It In Action: Arize Platform Demo, Live Q&A - Arize AI](#see-it-in-action-arize-platform-demo-live-q-a-arize-ai) - [Arize Platform Demo - Arize AI](#arize-platform-demo-arize-ai) - [Arize AI Un/Summit - Epic Fail: How Models Fail - Arize AI](#arize-ai-un-summit-epic-fail-how-models-fail-arize-ai) - [Arize AI Un/Summit - ML Observability in Finance - Arize AI](#arize-ai-un-summit-ml-observability-in-finance-arize-ai) - [Arize AI Un/Summit - Model Improvement - Arize AI](#arize-ai-un-summit-model-improvement-arize-ai) - [Arize AI Un/Summit - Eye on the Prize: AI Ethics - Arize AI](#arize-ai-un-summit-eye-on-the-prize-ai-ethics-arize-ai) --- # Online Evals & Monitoring for Agents in Production (Mosaic AI) | Cookbooks | Arize Docs This notebook will only run in a Databricks workspace environment. This notebook is adapted from Databricks's "[Mosaic AI Agent Framework: Author and deploy a tool-calling LangGraph agent](https://docs.databricks.com/aws/en/notebooks/source/generative-ai/langgraph-tool-calling-agent.html) " [![Logo](https://arize.com/docs/ax/~gitbook/image?url=https%3A%2F%2Fssl.gstatic.com%2Fcolaboratory-static%2Fcommon%2F20a9eda6ca436612e5341069a97a79fb%2Fimg%2Ffavicon.ico&width=20&dpr=4&quality=100&sign=bd67196d&sv=2)Google Colabcolab.research.google.com](https://colab.research.google.com/github/Arize-ai/tutorials/blob/main/python/llm/agents/databricks-mosaicai-tracing-evals.ipynb) In this notebook you learn to: * Author a tool-calling LangGraph agent wrapped with `ChatAgent` and Arize auto-instrumentation for tracing * This agent has the capability to generate and execute python code in a stateless sandboxed environment * Log and deploy the agent * Evaluate the agent's python code using Arize AX LLM as a Judge evaluation * Invoke the agent and view traces and evaluation results in Arize AX * Set up evaluation custom metrics and view them in monitors and dashboards in Arize AX To learn more about authoring an agent using Mosaic AI Agent Framework, see Databricks documentation ([AWS](https://docs.databricks.com/aws/generative-ai/agent-framework/author-agent) | [Azure](https://learn.microsoft.com/azure/databricks/generative-ai/agent-framework/create-chat-model) ). [](https://arize.com/docs/ax/cookbooks/agents/arize-+-mosaic-ai-agent-framework#prerequisites) Prerequisites ----------------------------------------------------------------------------------------------------------------- * Databricks account and workspace ([Sign up for free](https://docs.databricks.com/aws/en/getting-started/free-trial) ) * Arize AX account ([Sign up for free](https://app.arize.com/auth/join) ) * Address all `TODO`s in this notebook. ### [](https://arize.com/docs/ax/cookbooks/agents/arize-+-mosaic-ai-agent-framework#install-dependencies) Install Dependencies Copy %pip install -U -qqqq mlflow databricks-langchain databricks-agents uv langgraph==0.3.4 arize-otel openinference-instrumentation-langchain dbutils.library.restartPython() ### [](https://arize.com/docs/ax/cookbooks/agents/arize-+-mosaic-ai-agent-framework#access-arize-space-and-api-keys-from-databricks-secrets-and-set-them-as-environment-variables) Access Arize Space and API Keys from Databricks Secrets and set them as Environment Variables Create a [Arize AX API key and Space ID](https://docs.arize.com/arize/reference/authentication-and-security/api-keys) for the items below. Set up Arize credentials using [Databricks Secrets](https://docs.databricks.com/aws/en/security/secrets/) for secure access of keys. Copy # Reading the secure keys from secrets ARIZE_API_KEY = dbutils.secrets.get(scope="ryoung", key="ARIZE_API_KEY") ARIZE_SPACE_ID = dbutils.secrets.get(scope="ryoung", key="ARIZE_SPACE_ID") # setting as environment variables to be used by the chain import os os.environ["ARIZE_API_KEY"] = ARIZE_API_KEY os.environ["ARIZE_SPACE_ID"] = ARIZE_SPACE_ID ### [](https://arize.com/docs/ax/cookbooks/agents/arize-+-mosaic-ai-agent-framework#create-a-local-configuration-file-to-store-project-settings) Create a local configuration file to store project settings: Create a file named "`chain_config.yaml`" with variables below. It should reside in the same folder as the notebook. These variables will be accessed from the agent code. Replace the example values with your own values: `ARIZE_PROJECT_NAME="databricks-langgraph-tool-calling-agent"` `LLM_ENDPOINT_NAME="databricks-claude-3-7-sonnet"` [](https://arize.com/docs/ax/cookbooks/agents/arize-+-mosaic-ai-agent-framework#define-the-agent-in-code) Define the agent in code --------------------------------------------------------------------------------------------------------------------------------------- Define the agent code in a single cell below. This lets you easily write the agent code to a local Python file, using the `%%writefile` magic command, for subsequent logging and deployment. **Tracing auto-instrumentation** Opentelemetry based auto-instrumentation for Langgraph exports traces to Arize AX. **Agent tools** This agent code adds the built-in Unity Catalog function `system.ai.python_exec` to the agent. The agent code also includes commented-out sample code for adding a vector search index to perform unstructured data retrieval. `system.ai.python_exec` - Executes Python code in a stateless sandboxed environment and returns its stdout. The runtime cannot access files or read previous executions' output. All operations must be self-contained, using only standard Python libraries. Calls to other tools are prohibited. For more examples of tools to add to your agent, see Databricks documentation ([AWS](https://docs.databricks.com/aws/generative-ai/agent-framework/agent-tool) | [Azure](https://learn.microsoft.com/en-us/azure/databricks/generative-ai/agent-framework/agent-tool) ) **Wrap the LangGraph agent using the** `**ChatAgent**` **interface** For compatibility with Databricks AI features, the `LangGraphChatAgent` class implements the `ChatAgent` interface to wrap the LangGraph agent. This example uses the provided convenience APIs [`ChatAgentState`](https://mlflow.org/docs/latest/python_api/mlflow.langchain.html#mlflow.langchain.chat_agent_langgraph.ChatAgentState) and [`ChatAgentToolNode`](https://mlflow.org/docs/latest/python_api/mlflow.langchain.html#mlflow.langchain.chat_agent_langgraph.ChatAgentToolNode) for ease of use. Databricks recommends using `ChatAgent` as it simplifies authoring multi-turn conversational agents using an open source standard. See MLflow's [ChatAgent documentation](https://mlflow.org/docs/latest/python_api/mlflow.pyfunc.html#mlflow.pyfunc.ChatAgent) . Copy %%writefile agent.py from typing import Any, Generator, Optional, Sequence, Union import mlflow from databricks_langchain import ( ChatDatabricks, UCFunctionToolkit, VectorSearchRetrieverTool, ) from langchain_core.language_models import LanguageModelLike from langchain_core.runnables import RunnableConfig, RunnableLambda from langchain_core.tools import BaseTool from langgraph.graph import END, StateGraph from langgraph.graph.graph import CompiledGraph from langgraph.graph.state import CompiledStateGraph from langgraph.prebuilt.tool_node import ToolNode from mlflow.langchain.chat_agent_langgraph import ChatAgentState, ChatAgentToolNode from mlflow.pyfunc import ChatAgent from mlflow.types.agent import ( ChatAgentChunk, ChatAgentMessage, ChatAgentResponse, ChatContext, ) import os import logging logging.getLogger("openinference.instrumentation.langchain._tracer").setLevel(logging.CRITICAL) ############################################ # Arize Tracing Setup ############################################ #register tracer provider to send traces to Arize from arize.otel import register model_config = mlflow.models.ModelConfig(development_config="chain_config.yaml") tracer_provider = register( space_id = os.getenv("ARIZE_SPACE_ID"), api_key = os.getenv("ARIZE_API_KEY"), project_name = model_config.get("ARIZE_PROJECT_NAME"), #log_to_console=True ) # 1 line auto instrumentation from openinference.instrumentation.langchain import LangChainInstrumentor LangChainInstrumentor().instrument(tracer_provider=tracer_provider) ############################################ # Define your LLM endpoint and system prompt ############################################ # TODO: Replace with your model serving endpoint LLM_ENDPOINT_NAME = model_config.get("LLM_ENDPOINT_NAME") llm = ChatDatabricks(endpoint=LLM_ENDPOINT_NAME) # TODO: Update with your system prompt system_prompt = "You are a helpful assistant. Take the user's request and where applicable, use the appropriate tool if necessary to accomplish the task. If tools are not necessary, response directly to the user's request." ############################################################################### ## Define tools for your agent, enabling it to retrieve data or take actions ## beyond text generation ## To create and see usage examples of more tools, see ## https://docs.databricks.com/en/generative-ai/agent-framework/agent-tool.html ############################################################################### tools = [] # You can use UDFs in Unity Catalog as agent tools # Below, we add the `system.ai.python_exec` UDF, which provides # a python code interpreter tool to our agent # You can also add local LangChain python tools. See https://python.langchain.com/docs/concepts/tools # TODO: Add additional tools uc_tool_names = ["system.ai.python_exec"] uc_toolkit = UCFunctionToolkit(function_names=uc_tool_names) tools.extend(uc_toolkit.tools) # Use Databricks vector search indexes as tools # See https://docs.databricks.com/en/generative-ai/agent-framework/unstructured-retrieval-tools.html # for details # TODO: Add vector search indexes # vector_search_tools = [\ # VectorSearchRetrieverTool(\ # index_name="",\ # # filters="..."\ # )\ # ] # tools.extend(vector_search_tools) ##################### ## Define agent logic ##################### def create_tool_calling_agent( model: LanguageModelLike, tools: Union[ToolNode, Sequence[BaseTool]], system_prompt: Optional[str] = None, ) -> CompiledGraph: model = model.bind_tools(tools) # Define the function that determines which node to go to def should_continue(state: ChatAgentState): messages = state["messages"] last_message = messages[-1] # If there are function calls, continue. else, end if last_message.get("tool_calls"): return "continue" else: return "end" if system_prompt: preprocessor = RunnableLambda( lambda state: [{"role": "system", "content": system_prompt}] + state["messages"] ) else: preprocessor = RunnableLambda(lambda state: state["messages"]) model_runnable = preprocessor | model def call_model( state: ChatAgentState, config: RunnableConfig, ): response = model_runnable.invoke(state, config) return {"messages": [response]} workflow = StateGraph(ChatAgentState) workflow.add_node("agent", RunnableLambda(call_model)) workflow.add_node("tools", ChatAgentToolNode(tools)) workflow.set_entry_point("agent") workflow.add_conditional_edges( "agent", should_continue, { "continue": "tools", "end": END, }, ) workflow.add_edge("tools", "agent") return workflow.compile() class LangGraphChatAgent(ChatAgent): def __init__(self, agent: CompiledStateGraph): self.agent = agent def predict( self, messages: list[ChatAgentMessage], context: Optional[ChatContext] = None, custom_inputs: Optional[dict[str, Any]] = None, ) -> ChatAgentResponse: request = {"messages": self._convert_messages_to_dict(messages)} messages = [] for event in self.agent.stream(request, stream_mode="updates"): for node_data in event.values(): messages.extend( ChatAgentMessage(**msg) for msg in node_data.get("messages", []) ) return ChatAgentResponse(messages=messages) def predict_stream( self, messages: list[ChatAgentMessage], context: Optional[ChatContext] = None, custom_inputs: Optional[dict[str, Any]] = None, ) -> Generator[ChatAgentChunk, None, None]: request = {"messages": self._convert_messages_to_dict(messages)} for event in self.agent.stream(request, stream_mode="updates"): for node_data in event.values(): yield from ( ChatAgentChunk(**{"delta": msg}) for msg in node_data["messages"] ) # Create the agent object, and specify it as the agent object to use when # loading the agent back for inference via mlflow.models.set_model() agent = create_tool_calling_agent(llm, tools, system_prompt) AGENT = LangGraphChatAgent(agent) mlflow.models.set_model(AGENT) [](https://arize.com/docs/ax/cookbooks/agents/arize-+-mosaic-ai-agent-framework#restart-python-and-reset-environment-variables) Restart Python and reset environment variables ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Copy dbutils.library.restartPython() Copy # Reading the secure keys from secrets ARIZE_API_KEY = dbutils.secrets.get(scope="ryoung", key="ARIZE_API_KEY") ARIZE_SPACE_ID = dbutils.secrets.get(scope="ryoung", key="ARIZE_SPACE_ID") # setting as environment variables to be used by the chain import os os.environ["ARIZE_API_KEY"] = ARIZE_API_KEY os.environ["ARIZE_SPACE_ID"] = ARIZE_SPACE_ID [](https://arize.com/docs/ax/cookbooks/agents/arize-+-mosaic-ai-agent-framework#log-the-agent-as-an-mlflow-model) Log the agent as an MLflow model ------------------------------------------------------------------------------------------------------------------------------------------------------- Log the agent as code from the `agent.py` file. See [MLflow - Models from Code](https://mlflow.org/docs/latest/models.html#models-from-code) . #### [](https://arize.com/docs/ax/cookbooks/agents/arize-+-mosaic-ai-agent-framework#enable-automatic-authentication-for-databricks-resources) Enable automatic authentication for Databricks resources For the most common Databricks resource types, Databricks supports and recommends declaring resource dependencies for the agent upfront during logging. This enables automatic authentication passthrough when you deploy the agent. With automatic authentication passthrough, Databricks automatically provisions, rotates, and manages short-lived credentials to securely access these resource dependencies from within the agent endpoint. To enable automatic authentication, specify the dependent Databricks resources when calling `mlflow.pyfunc.log_model().` * **TODO**: If your Unity Catalog tool queries a \[vector search index\](docs link) or leverages \[external functions\](docs link), you need to include the dependent vector search index and UC connection objects, respectively, as resources. See docs ([AWS](https://docs.databricks.com/generative-ai/agent-framework/log-agent.html#specify-resources-for-automatic-authentication-passthrough) | [Azure](https://learn.microsoft.com/azure/databricks/generative-ai/agent-framework/log-agent#resources) ). Copy import mlflow from agent import tools, LLM_ENDPOINT_NAME from databricks_langchain import VectorSearchRetrieverTool from mlflow.models.resources import DatabricksFunction, DatabricksServingEndpoint from unitycatalog.ai.langchain.toolkit import UnityCatalogTool from pkg_resources import get_distribution model_config = mlflow.models.ModelConfig(development_config="chain_config.yaml") resources = [DatabricksServingEndpoint(endpoint_name=model_config.get("LLM_ENDPOINT_NAME"))] for tool in tools: if isinstance(tool, VectorSearchRetrieverTool): resources.extend(tool.resources) elif isinstance(tool, UnityCatalogTool): resources.append(DatabricksFunction(function_name=tool.uc_function_name)) with mlflow.start_run(): logged_agent_info = mlflow.pyfunc.log_model( artifact_path="agent", python_model="agent.py", model_config="chain_config.yaml", extra_pip_requirements= [\ f"databricks-connect=={get_distribution('databricks-connect').version}",\ "arize-otel", "openinference.instrumentation.langchain"\ ], resources=resources, ) ### [](https://arize.com/docs/ax/cookbooks/agents/arize-+-mosaic-ai-agent-framework#pre-deployment-agent-validation) Pre-deployment agent validation Before registering and deploying the agent, perform pre-deployment checks using the [mlflow.models.predict()](https://mlflow.org/docs/latest/python_api/mlflow.models.html#mlflow.models.predict) API. See Databricks documentation ([AWS](https://docs.databricks.com/en/machine-learning/model-serving/model-serving-debug.html#validate-inputs) | [Azure](https://learn.microsoft.com/en-us/azure/databricks/machine-learning/model-serving/model-serving-debug#before-model-deployment-validation-checks) ). Copy mlflow.models.predict( model_uri=f"runs:/{logged_agent_info.run_id}/agent", input_data={"messages": [{"role": "user", "content": "Hello!"}]}, env_manager="uv", ) ### [](https://arize.com/docs/ax/cookbooks/agents/arize-+-mosaic-ai-agent-framework#register-the-model-to-unity-catalog) Register the model to Unity Catalog Before you deploy the agent, you must register the agent to Unity Catalog. * **TODO** Update the `catalog`, `schema`, and `model_name` below to register the MLflow model to Unity Catalog. Copy mlflow.set_registry_uri("databricks-uc") # TODO: define the catalog, schema, and model name for your UC model catalog = "prasad_kona_isv" schema = "demo" model_name = "langgraph-tool-calling-agent" UC_MODEL_NAME = f"{catalog}.{schema}.{model_name}" # register the model to UC uc_registered_model_info = mlflow.register_model( model_uri=logged_agent_info.model_uri, name=UC_MODEL_NAME ) [](https://arize.com/docs/ax/cookbooks/agents/arize-+-mosaic-ai-agent-framework#deploy-the-agent) Deploy the agent ----------------------------------------------------------------------------------------------------------------------- Copy from databricks import agents agents.deploy( UC_MODEL_NAME, uc_registered_model_info.version, tags = {"endpointSource": "docs"}, scale_to_zero_enabled=True, environment_vars={ "ARIZE_API_KEY": "{{secrets//ARIZE_API_KEY}}", "ARIZE_SPACE_ID": "{{secrets//ARIZE_SPACE_ID}}", } ) [](https://arize.com/docs/ax/cookbooks/agents/arize-+-mosaic-ai-agent-framework#configure-online-evaluations-in-arize-ax) Configure Online Evaluations in Arize AX ----------------------------------------------------------------------------------------------------------------------------------------------------------------------- Follow instructions [here](https://docs.arize.com/arize/evaluate/online-evals/run-evaluations-in-the-ui) to setup up online evaluations in Arize AX. Arize's Online Evaluations automatically run LLM-as-a-Judge based evaluations directly on the traces collected in Arize AX from our Agent runs. This provides continuous quality monitoring without manual intervention. This approach scales to thousands of interactions, enabling data-driven improvements to your agent's performance. These evaluations are for assessing code generation quality that the agent produces, specifically: * Code Correctness: Does the generated code solve the user's problem accurately? * Code Readability: Is the code clean, well-structured, and maintainable? References: * LLM-as-a-Judge evaluation best practices: (Arize docs * Agent evaluation best practices: (Arize Docs * Automate running evaluations on your Traces and Spans: (Docs ![](https://arize.com/docs/ax/~gitbook/image?url=https%3A%2F%2Fstorage.googleapis.com%2Farize-assets%2Ftutorials%2Fimages%2Fdatabricks-eval.gif&width=768&dpr=4&quality=100&sign=8778ced&sv=2) [](https://arize.com/docs/ax/cookbooks/agents/arize-+-mosaic-ai-agent-framework#call-the-agent) Call the Agent ------------------------------------------------------------------------------------------------------------------- There are several methods we can use to call our newly deployed agent in Databricks. * REST API Calls: You can invoke your deployed agent through HTTP POST requests to the model serving endpoint. This method provides programmatic access, allowing you to integrate the agent into applications or automated workflows by sending JSON payloads with your input data and receiving structured responses. * Model Serving UI: Databricks provides a built-in web interface where you can directly test your deployed agent. Simply navigate to the serving endpoint in the Databricks workspace, use the "Test" tab to input sample data, and see real-time responses without writing any code. * Databricks AI Playground: This interactive environment lets you experiment with your agent in a conversational interface. You can test different prompts, observe the agent's behavior, and refine your interactions before implementing them in production scenarios. Copy # Example REST API Call via Curl # #1 - Basic question (no code generation) curl \ -u token:$DATABRICKS_TOKEN \ -X POST \ -H "Content-Type: application/json" \ -d '{"prompt": "What is a lakehouse?", "max_tokens": 64}' \ https://.databricks.com/serving-endpoints//invocations # #2 - Math question (code generation) curl \ -u token:$DATABRICKS_TOKEN \ -X POST \ -H "Content-Type: application/json" \ -d '{"prompt": "What is 5*5 in python?", "max_tokens": 64}' \ https://.databricks.com/serving-endpoints//invocations Copy # Example calling the agent using openai sdk from openai import OpenAI import os # In a Databricks notebook you can use this: DATABRICKS_HOSTNAME = dbutils.notebook.entry_point.getDbutils().notebook().getContext().browserHostName().get() DATABRICKS_TOKEN = dbutils.notebook.entry_point.getDbutils().notebook().getContext().apiToken().get() serving_endpoint_name = "" client = OpenAI( api_key=DATABRICKS_TOKEN, base_url=f"https://{DATABRICKS_HOSTNAME}/serving-endpoints" ) chat_completion = client.chat.completions.create( messages=[\ {\ "role": "system",\ "content": "You are an AI assistant"\ },\ {\ "role": "user",\ "content": "Tell me about Large Language Models in one sentence"\ }\ ], model=serving_endpoint_name, max_tokens=256 ) print(chat_completion.choices[0].message.content) if chat_completion and chat_completion.choices else print(chat_completion) ### [](https://arize.com/docs/ax/cookbooks/agents/arize-+-mosaic-ai-agent-framework#view-traces-and-evaluation-results-in-arize-ax) View traces and evaluation results in Arize AX As you run your agent, traces are automatically sent to Arize AX. In Arize AX, you can see agent execution details, tool invocations, latency breakdown by component, token usage and costs, errors and metadata captured for each span and function call. Additionally, evaluation labels are captured for every trace based on the code correctness and code readability evals we setup earlier. ![](https://arize.com/docs/ax/~gitbook/image?url=https%3A%2F%2Fstorage.googleapis.com%2Farize-assets%2Ftutorials%2Fimages%2Fdatabricks-trace.gif&width=768&dpr=4&quality=100&sign=1f43eecc&sv=2) ![](https://arize.com/docs/ax/~gitbook/image?url=https%3A%2F%2Fstorage.googleapis.com%2Farize-assets%2Ftutorials%2Fimages%2Fdatabricks-trace-screenshot.jpg&width=768&dpr=4&quality=100&sign=76e8951a&sv=2) [](https://arize.com/docs/ax/cookbooks/agents/arize-+-mosaic-ai-agent-framework#monitoring-alerting-and-kpi-dashboards-in-arize-ax) Monitoring, alerting and KPI dashboards in Arize AX -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Turn any trace attribute and evaluation label into [custom metrics](https://docs.arize.com/arize/machine-learning/machine-learning/how-to-ml/custom-metrics-api/12.-custom-metrics) . Build KPI driven [dashboards](https://docs.arize.com/arize/observe/dashboards) and [monitors](https://docs.arize.com/arize/observe/production-monitoring) that proactively alert you when any degradation in performance or quality of your agent occurs. ![](https://arize.com/docs/ax/~gitbook/image?url=https%3A%2F%2Fstorage.googleapis.com%2Farize-assets%2Ftutorials%2Fimages%2Fdatabricks-kpi-dashboard.jpg&width=768&dpr=4&quality=100&sign=180bda53&sv=2) ![](https://arize.com/docs/ax/~gitbook/image?url=https%3A%2F%2Fstorage.googleapis.com%2Farize-assets%2Ftutorials%2Fimages%2Fdatabricks-monitor.jpg&width=768&dpr=4&quality=100&sign=fb20ba6&sv=2) ### [](https://arize.com/docs/ax/cookbooks/agents/arize-+-mosaic-ai-agent-framework#next-steps) Next steps After your agent is deployed, you can chat with it in AI playground to perform additional checks, share it with SMEs in your organization for feedback, or embed it in a production application. See Databricks documentation ([AWS](https://docs.databricks.com/en/generative-ai/deploy-agent.html) | [Azure](https://learn.microsoft.com/en-us/azure/databricks/generative-ai/deploy-agent) ). ### [](https://arize.com/docs/ax/cookbooks/agents/arize-+-mosaic-ai-agent-framework#resources) Resources Databricks Resources * [Mosaic AI Agent Framework Documentation](https://docs.databricks.com/aws/en/data-governance/unity-catalog/) * [Unity Catalog Tools Guide](https://docs.databricks.com/generative-ai/agent-framework/agent-tool) Arize Resources * [Agent Evaluation Best Practices](https://arize.com/docs/ax/learn/evaluation-concepts/agent-evaluation) Last updated 14 days ago Was this helpful? --- # Arize Private Connect | Arize Docs Arize AX provides multiple deployment options including SaaS, Arize PrivateConnect (supporting AWS PrivateLink, Azure Private Link, and Google Cloud's Private Service Connect), and VPC deployments to ensure secure cloud connectivity. This document outlines why Arize PrivateConnect is the recommended solution for most customers, offering secure and seamless integration with existing cloud infrastructure while maintaining data privacy and operational agility. Talk to our customer solutions team for more detailed setup instructions by reaching out to _support@arize.com_ [](https://arize.com/docs/ax/security-and-settings/arize-private-connect#architecture-overview) Architecture Overview -------------------------------------------------------------------------------------------------------------------------- When you set up Arize with PrivateConnect, your cloud storage account creates a VPC endpoint that connects directly to Arize's cloud environment. When your application sends AI/ML telemetry data to Arize, it flows through this private endpoint inside your cloud provider's network - never touching the public internet. The data first goes to your secure virtual private cloud environment, then through the private service endpoint, and finally reaches Arize's compute resources. Everything stays within the cloud provider's private network backbone, ensuring secure and private data transmission between your infrastructure and Arize's platform. By having PrivateConnect, any API or UI related operation will also be driven by this PrivateConnect endpoint and user operations will not touch the public internet at all. Here is a sample diagram for Arize PrivateConnect using AWS PrivateLink: ![](https://arize.com/docs/ax/~gitbook/image?url=https%3A%2F%2Flh7-rt.googleusercontent.com%2Fdocsz%2FAD_4nXfLWi_GrNYzZDY3R_HY_K5w57O71_cAZuwMSv32qm9GIYJisIz6ds7wyPQqkO6mgCD6UxFtvjWafUkAqj97CF-FK8phg_wdDGnn1FGSKLIP23i6SyxCLgBPMNDf727Uuw9ilG4tqQ%3Fkey%3DGi0bSyIS28ZjzAT30a2DLZ-_&width=768&dpr=4&quality=100&sign=f869f933&sv=2) Example Architecture Setup for AWS [](https://arize.com/docs/ax/security-and-settings/arize-private-connect#benefits-of-deploying-arize-private-connect) Benefits of Deploying Arize Private Connect ---------------------------------------------------------------------------------------------------------------------------------------------------------------------- * **Ease of Setup and Faster Time-to-Value**: Setting up Arize PrivateConnect is typically faster than a full VPC installation across all cloud providers. This quicker setup reduces time-to-value and lets customers start using the platform immediately. * **Reduced Operational Complexity**: With Arize PrivateConnect, customers don't have to manage complex networking or infrastructure updates within their virtual network. Instead, they can connect directly to Arize AX without the need for extensive configurations, upgrades, or ongoing maintenance. * **Data Security & Isolation**: Arize PrivateConnect provides secure, private connections without exposing traffic to the internet, maintaining a high level of security similar to VPC deployments. The service is accessible from a customer's virtual network directly, keeping traffic isolated and secure within each cloud provider's network backbone. * **Scalability & Flexibility**: Arize PrivateConnect allows customers to easily connect to Arize AX from any of their virtual networks within a single region or even across regions (with appropriate network peering). This flexibility can be especially beneficial as usage grows or if multi-region access is needed without reconfiguring or expanding a full VPC deployment. * **Ongoing Compliance and Updates**: By connecting through Arize PrivateConnect, customers can leverage the latest security updates, patches, and feature improvements of Arize AX without requiring downtime or maintenance within their own virtual network due to Arize's CI/CD deployment practices. * **Cost Efficiency**: Arize PrivateConnect can significantly reduce costs compared to a full virtual network deployment. A VPC deployment requires additional infrastructure and ongoing maintenance, while Arize PrivateConnect uses managed network connections, cutting down infrastructure and operational expenses. * **Simplified Access Control**: Arize PrivateConnect integrates easily with each cloud provider's identity and access management services (AWS IAM, Azure AD, Google Cloud IAM), providing granular control over access to the platform without requiring complex configuration in addition to SSO. Using Arize PrivateConnect provides a private, secure, and high-performance connection with all the advantages of using a managed SaaS platform without the added overhead of deploying and managing the entire software stack in their environment. This solution works seamlessly whether you're using AWS PrivateLink, Azure Private Link, or Google Cloud's Private Service Connect. Last updated 14 days ago Was this helpful? --- # AI Powered Trace Analysis | Arize Docs **LLM Analysis Lite** * **Description:** Lightweight analysis skill to find patterns and provide filter suggestions. * **Suggested Prompt:** "What are the top 5 types of questions asked?" * **Use When:** You need a lightweight analysis of your data. ![](https://arize.com/docs/ax/~gitbook/image?url=https%3A%2F%2F2088270005-files.gitbook.io%2F%7E%2Ffiles%2Fv0%2Fb%2Fgitbook-x-prod.appspot.com%2Fo%2Fspaces%252F-MAlgpMyBRcl2qFZRQ67%252Fuploads%252Fgit-blob-018992e06322958ec1063d398e59c697ba6c16e0%252Ftrace_analysis.gif%3Falt%3Dmedia&width=768&dpr=4&quality=100&sign=95ae752e&sv=2) Last updated 1 year ago Was this helpful? --- # AI Powered Search & Filter | Arize Docs Arize has 3 ways users can filter and search across their traces. 1. [Use AI Search with natural language](https://arize.com/docs/ax/alyx/arize-copilot/filter-traces-1#use-ai-search-with-natural-language) 2. [Use AI Search to construct the filter syntax](https://arize.com/docs/ax/alyx/arize-copilot/filter-traces-1#use-ai-search-to-construct-the-filter-syntax) 3. Directly use Filter Syntax [](https://arize.com/docs/ax/alyx/arize-copilot/filter-traces-1#use-ai-search-with-natural-language) Use AI Search with natural language --------------------------------------------------------------------------------------------------------------------------------------------- 1. **Users can search across a Column** * **Description:** Refined semantic search within a single column based on user criteria. * **Suggested Prompt:** “Find me confused inputs” * **Use When:** Alyx will automatically select this skill when you need to search within a specific column. 2. **Users can also search across Multiple or ALL Columns in Table Search** * **Description:** Search across multiple or all columns within a table to find patterns and outliers. * **Suggested Prompt:** “Find inputs that reference pricing that are hallucinated” * **Use When:** Alyx will automatically select this skill when you want to search across mutliple columns or entire table. ![](https://arize.com/docs/ax/~gitbook/image?url=https%3A%2F%2F2088270005-files.gitbook.io%2F%7E%2Ffiles%2Fv0%2Fb%2Fgitbook-x-prod.appspot.com%2Fo%2Fspaces%252F-MAlgpMyBRcl2qFZRQ67%252Fuploads%252Fgit-blob-5f3d1e4ad0696327cda308a49914806ab2628d18%252Fai_search.gif%3Falt%3Dmedia&width=768&dpr=4&quality=100&sign=e424006f&sv=2) [](https://arize.com/docs/ax/alyx/arize-copilot/filter-traces-1#use-ai-search-to-construct-the-filter-syntax) Use AI Search to Construct the Filter Syntax --------------------------------------------------------------------------------------------------------------------------------------------------------------- You can also use AI Search to automatically generate query filters. Simply use the keyword "filter" followed by your desired criteria. * **Description:** Generates query filters from natural language commands to refine your data search. Use keyword `Filter` to trigger this skill. * **Suggested Prompt:** "Filter by input contains SDK" * **Use When:** You want Alyx to construct a query filter for you. ![](https://arize.com/docs/ax/~gitbook/image?url=https%3A%2F%2F2088270005-files.gitbook.io%2F%7E%2Ffiles%2Fv0%2Fb%2Fgitbook-x-prod.appspot.com%2Fo%2Fspaces%252F-MAlgpMyBRcl2qFZRQ67%252Fuploads%252Fgit-blob-5bb6f8f0baecd7e219f74ac4114bfd5596cbff73%252FExport-1729825775546.gif%3Falt%3Dmedia&width=768&dpr=4&quality=100&sign=1cc1b3dd&sv=2) [](https://arize.com/docs/ax/alyx/arize-copilot/filter-traces-1#use-filter-syntax) Use Filter Syntax --------------------------------------------------------------------------------------------------------- Users can also directly use the filter syntax to query their traces. Last updated 4 months ago Was this helpful? --- # Arize AI for Agents | Arize Docs Arize AI is a powerful AI engineering platform designed to support the development, evaluation, and observability of AI agents. Arize helps developers create robust, high performing agents. It has first-class support for agent frameworks such as Autogen, OpenAI-agents, LangGraph, and smolagents. [](https://arize.com/docs/ax/arize-ai-for-agents#why-arize-ai-for-agents) Why Arize AI for Agents? ------------------------------------------------------------------------------------------------------- ### [](https://arize.com/docs/ax/arize-ai-for-agents#id-1.-agent-observability-with-auto-instrumentation) 1\. Agent Observability with Auto Instrumentation Observability is critical for understanding how agents behave in real-world scenarios. Arize AI provides robust tracing capabilities through our open source [OpenInference](https://github.com/Arize-ai/openinference) library, automatically instrumenting your agent applications to capture traces and spans. This includes LLM calls, tool invocations, and data retrieval steps, giving you a detailed view of your agent's workflow. With just a few lines of code, you can set up tracing for popular frameworks like OpenAI Agents, LangGraph, and Autogen. Learn more about Tracing. **Code Example: Auto Instrumentation for OpenAI Agents** Copy from arize.otel import register tracer_provider = register( space_id = "your-space-id", api_key = "your-api-key", project_name="agents" ) from openinference.instrumentation.openai_agents import OpenAIAgentsInstrumentor OpenAIAgentsInstrumentor().instrument(tracer_provider=tracer_provider) ### [](https://arize.com/docs/ax/arize-ai-for-agents#id-2.-agent-evaluations-with-online-evals) 2\. Agent Evaluations with Online Evals Evaluating agent performance is essential to ensure reliability and accuracy. Arize AI's online evaluations automatically tag spans with performance labels, helping you identify problematic interactions and measure key metrics. * **Comprehensive Evaluation Templates**: Arize AX provides templates for evaluating various agent components, such as Tool Calling, Path Convergence, and Planning. * **Online Evals**: With Online Evals, you can run continuous evaluations on production data to monitor correctness, hallucination, relevance, and latency. This ensures your agents perform consistently across diverse scenarios. * **Custom Metrics and Alerts**: Track key metrics on custom dashboards and receive alerts when performance deviates from the norm, allowing proactive optimization of agent behavior. **Code Example: Logging Evaluations to Arize** Copy # Example of logging an evaluation for an agent's response from arize.api import Client from arize.utils.types import Environments, ModelTypes arize_client = Client(space_key="YOUR_SPACE_KEY", api_key="YOUR_API_KEY") response = arize_client.log_evaluation( model_id="agent-model-v1", environment=Environments.PRODUCTION, model_type=ModelTypes.GENERATIVE_LLM, prompt="Plan a trip to Paris.", response="Here is a 5-day itinerary for Paris...", evaluation_name="Correctness", evaluation_score=0.9 ) ### [](https://arize.com/docs/ax/arize-ai-for-agents#id-3.-testing-agents-in-prompt-playground-with-tool-calling-support) 3\. Testing Agents in Prompt Playground with Tool Calling Support Arize's Prompt Playground is a no-code environment for iterating on prompts and testing agent behaviors, including support for tool calling—a critical feature for agents that interact with external APIs or functions. * **Iterate on Prompts**: Test different prompt templates, models, and parameters side by side to refine how your agent responds to user inputs. * **Tool Calling Support**: Debug tool calling directly in the Playground to ensure your agent selects the right tools and parameters. Learn more about Using Tools in Playground. * **Save as Experiment**: Run systematic A/B tests on datasets to validate agent performance and share results with your team via experiments. ### [](https://arize.com/docs/ax/arize-ai-for-agents#id-4.-sessions-for-agent-interaction-tracking) 4\. Sessions for Agent Interaction Tracking ![](https://arize.com/docs/ax/~gitbook/image?url=https%3A%2F%2F2088270005-files.gitbook.io%2F%7E%2Ffiles%2Fv0%2Fb%2Fgitbook-x-prod.appspot.com%2Fo%2Fspaces%252F-MAlgpMyBRcl2qFZRQ67%252Fuploads%252Fgit-blob-a1641b7274d588ad4cd232ffdacaf163ead96c39%252FScreenshot%25202024-05-09%2520at%25205.21.51%25E2%2580%25AFPM.png%3Falt%3Dmedia&width=768&dpr=4&quality=100&sign=f2b8cb16&sv=2) For chatbot or multi-turn agent applications, tracking sessions is invaluable for debugging and performance analysis. Arize AI supports session tracking to group traces based on interactions. * **Session ID and User ID**: Add `session.id` and `user.id` as attributes to spans to group interactions and analyze conversation flows. This helps identify where conversations break or user frustration increases. * **Debugging Sessions**: Use Arize AX to filter sessions and find underperforming groups of traces. Learn more about Sessions and Users. **Code Example: Adding Session ID for Agent Chatbot** Copy from openinference.instrumentation import using_session with using_session(session_id="chat-session-456"): # Agent interaction within a session response = client.chat.completions.create( model="gpt-3.5-turbo", messages=[{"role": "user", "content": "Book a flight to Paris."}], max_tokens=50, ) ### [](https://arize.com/docs/ax/arize-ai-for-agents#id-5.-agent-replay-and-agent-pathing) 5\. Agent Replay and Agent Pathing ![](https://arize.com/docs/ax/~gitbook/image?url=https%3A%2F%2F2088270005-files.gitbook.io%2F%7E%2Ffiles%2Fv0%2Fb%2Fgitbook-x-prod.appspot.com%2Fo%2Fspaces%252F-MAlgpMyBRcl2qFZRQ67%252Fuploads%252Fgit-blob-98c6589abb596d63351cfea79380f7a3abf6f275%252Fimage.png%3Falt%3Dmedia&width=768&dpr=4&quality=100&sign=12972070&sv=2) * **Agent Replay**: Replay agent interactions to debug agent tool calling in a controlled environment. Replay will help you simulate past sessions to test improvements without impacting live users. * **Agent Pathing:** Analyze and optimize the pathways your agents take to complete tasks. Understand whether agents are taking efficient routes or getting stuck in loops, with tools to refine planning and convergence strategies. ### [](https://arize.com/docs/ax/arize-ai-for-agents#additional-resources-for-agent-development) Additional Resources for Agent Development [](https://arize.com/ai-agents/) **Agent Evaluation Guide** Learn how to evaluate every component of your agent. [](https://arize.com/docs/ax/cookbooks/agents/tracing-and-evaluating-agents) **Try our Tutorials** Explore example notebooks for agents, RAG, tracing, and evaluations. [](https://arize.com/ai-research-papers/) **Watch our Paper Readings** Dive into video discussions on the latest AI research, including agent architectures. [](https://arize.com/community/) **Join our Slack Community** Connect with other developers to ask questions, share insights, and provide feedback on agent development with Arize. Last updated 14 days ago Was this helpful? --- # AI Span Analysis & Evaluation | Arize Docs [](https://arize.com/docs/ax/alyx/arize-copilot/ai-span-analysis-and-evaluation#overview) **Overview** ----------------------------------------------------------------------------------------------------------- The **Alyx** **Span Chat** simplifies reviewing spans by enabling intuitive, natural-language interactions. Whether you need to analyze span data, ask questions, or evaluate performance, Alyx makes it fast and easy. ### [](https://arize.com/docs/ax/alyx/arize-copilot/ai-span-analysis-and-evaluation#key-features) **Key Features** * **Analyze Spans**: Extract key insights and actionable information from complex span data. * **Ask Questions**: Use natural language to query specific details or gain clarity on span content. * **Run Evaluations**: Seamlessly execute evaluations for individual spans directly from the chat. [](https://arize.com/docs/ax/alyx/arize-copilot/ai-span-analysis-and-evaluation#how-to-use-span-chat) How to Use Span Chat ------------------------------------------------------------------------------------------------------------------------------- #### [](https://arize.com/docs/ax/alyx/arize-copilot/ai-span-analysis-and-evaluation#navigate-to-a-span) Navigate to a Span Start by navigating to the span you want to analyze and click into the Alyx input ![](https://arize.com/docs/ax/~gitbook/image?url=https%3A%2F%2F2088270005-files.gitbook.io%2F%7E%2Ffiles%2Fv0%2Fb%2Fgitbook-x-prod.appspot.com%2Fo%2Fspaces%252F-MAlgpMyBRcl2qFZRQ67%252Fuploads%252Fgit-blob-45009b9c12d7f5341e32e7e450ea891e1466898f%252Fspanchat.gif%3Falt%3Dmedia&width=768&dpr=4&quality=100&sign=919d9673&sv=2) #### [](https://arize.com/docs/ax/alyx/arize-copilot/ai-span-analysis-and-evaluation#ask-questions-or-analyze-a-span) Ask Questions or Analyze a Span * Type natural-language questions like: * "Is X in the prompt variable schema?" * "Can you summarize the system prompt?" * Alyx will extract key insights and summarize the data for you. ![](https://arize.com/docs/ax/~gitbook/image?url=https%3A%2F%2F2088270005-files.gitbook.io%2F%7E%2Ffiles%2Fv0%2Fb%2Fgitbook-x-prod.appspot.com%2Fo%2Fspaces%252F-MAlgpMyBRcl2qFZRQ67%252Fuploads%252Fgit-blob-92d9581a780a77765b54826df216c69690156cbc%252Fspanchat_analysis.gif%3Falt%3Dmedia&width=768&dpr=4&quality=100&sign=5aba858c&sv=2) #### [](https://arize.com/docs/ax/alyx/arize-copilot/ai-span-analysis-and-evaluation#run-evaluations) Run Evaluations 1. Request Alyx to evaluate a span: * Example: "Evaluate this span based on X criteria." 2. Review the results and iterate as needed. ![](https://arize.com/docs/ax/~gitbook/image?url=https%3A%2F%2F2088270005-files.gitbook.io%2F%7E%2Ffiles%2Fv0%2Fb%2Fgitbook-x-prod.appspot.com%2Fo%2Fspaces%252F-MAlgpMyBRcl2qFZRQ67%252Fuploads%252Fgit-blob-526d835e14a4a32cd4ad48d67596b4544aee6872%252Fspanchat_eval.gif%3Falt%3Dmedia&width=768&dpr=4&quality=100&sign=9a51dd5c&sv=2) Last updated 4 months ago Was this helpful? --- # Alyx - AI Engineering Agent | Arize Docs Alyx is an AI powered assistant designed to help AI engineers build and improve their applications. Alyx is equipped with over 30 purpose built applications, called skills, to assist with tasks such as optimizing your prompt, building an eval, semantic search, and more. Alyx is integrated throughout Arize AX where skills are applicable, and is available through the chat interface. [](https://arize.com/docs/ax/alyx/arize-copilot#alyx-skills) Alyx Skills ----------------------------------------------------------------------------- Alyx is built with skills to assist with tasks throughout AX. Skills are built with industry expertise to accelerate development, embed best-practice workflows, and solve common pain points. There are a variety of skills available through Alyx. Click on each skill below to learn how to use it effectively and integrate it into your workflow. ### [](https://arize.com/docs/ax/alyx/arize-copilot#llm-applications) LLM Applications **Skill** **Description** [**AI Search - Column Search**](https://arize.com/docs/ax/alyx/arize-copilot/filter-traces-1#use-ai-search-with-natural-language) Searches a specific column based on the user’s input to find relevant data. Example: "Find me confused inputs." [**AI Search - Table Search**](https://arize.com/docs/ax/alyx/arize-copilot/filter-traces-1#use-ai-search-with-natural-language) Searches across the entire table to identify patterns, anomalies, or outliers. Example: "Find inputs that reference pricing that are hallucinated." [**AI Search - Text to Filter**](https://arize.com/docs/ax/alyx/arize-copilot/filter-traces-1#use-ai-search-to-construct-the-filter-syntax) Generates query filters based on natural language commands. Example: "Filter by input contains SDK." [**AI Search - LLM Analysis Lite**](https://arize.com/docs/ax/alyx/arize-copilot/ai-powered-trace-analysis) Provides suggestions for search results and finds patterns in the data. Example: "What are the top 5 types of questions asked?" [**Trace Troubleshooting**](https://arize.com/docs/ax/release-notes/alyx/arize-copilot#arize-copilot-3.0-and-trace-troubleshooting) Enables end-to-end trace navigation powered by o3 to pinpoint issues and diagnose with ease. [**Create Custom Evaluations**](https://github.com/Arize-ai/docs/blob/main/arize/alyx/arize-copilot/broken-reference/README.md) Writes a tailored eval for your application based on specified goals or data analysis. [**Diagnose RAG Issues**](https://github.com/Arize-ai/docs/blob/main/arize/alyx/arize-copilot/broken-reference/README.md) Analyzes responses in the retrieval process, ensuring relevance and accuracy, offering improvements. [**Optimize Prompts**](https://arize.com/docs/ax/prompts/prompt-optimization/ai-powered-prompt-builder) Optimizes prompts to enhance response quality or address specific issues. [**Summarize Evaluation Metrics**](https://arize.com/docs/ax/alyx/arize-copilot/ai-powered-eval-analysis) Assesses and summarizes evaluation metrics, providing suggestions for enhancing performance. [**ArizeQL Generator**](https://arize.com/docs/ax/alyx/arize-copilot/arizeql-generator) Generates custom metrics by translating natural language descriptions or existing code (e.g., SQL, Python) into AQL for easy application. [**Dashboard Generator**](https://arize.com/docs/ax/observe/dashboards/dashboard-widget-creation) Generates dashboard widgets by translating natural language descriptions or existing code. [**Span Chat**](https://arize.com/docs/ax/alyx/arize-copilot/ai-span-analysis-and-evaluation) Enables effortless analysis and evaluation of spans through natural-language interactions, providing insights, answering questions, and running evaluations with ease. ### [](https://arize.com/docs/ax/alyx/arize-copilot#ml-and-cv-models) ML & CV Models **Skill** **Description** [**Get Model Insights**](https://arize.com/docs/ax/machine-learning/machine-learning/how-to-ml/performance-tracing/ai-powered-performance-insights) Provides a high-level analysis of model performance, including trends over time, prediction volumes, and drift. Best for general inquiries, not suited for detailed debugging. [**Cohort Performance Analysis**](https://arize.com/docs/ax/machine-learning/machine-learning/how-to-ml/performance-tracing/ai-powered-performance-insights) Analyzes model performance across different cohorts or slices of data, identifying poorly performing segments. Provides insights into behavior over a specific period. [**Detect Data Drift**](https://arize.com/docs/ax/machine-learning/machine-learning/how-to-ml/drift-tracing/ai-powered-drift-insights) Pinpoints sudden input quality issues by examining features and tags for drift. Compares current distributions to a baseline to detect significant shifts. [**Check for Missing Data**](https://arize.com/docs/ax/machine-learning/machine-learning/how-to-ml/data-quality-troubleshooting/ai-powered-data-quality-insights) Analyzes input data to report the percentage of missing data in features and tags, highlighting any sudden spikes or changes that could impact model inputs. [**Assess Feature Data Quality**](https://arize.com/docs/ax/machine-learning/machine-learning/how-to-ml/data-quality-troubleshooting/ai-powered-data-quality-insights) Assists in debugging issues by analyzing dataset metrics and focusing on specific dimensions. Identifies critical changes and provides actionable suggestions. [**Evaluate Distribution Shifts**](https://arize.com/docs/ax/machine-learning/machine-learning/how-to-ml/data-quality-troubleshooting/ai-powered-data-quality-insights) Analyzes a dimension’s distribution to understand shifts in percentage over time. [**Review Cardinality Trends**](https://arize.com/docs/ax/machine-learning/machine-learning/how-to-ml/data-quality-troubleshooting/ai-powered-data-quality-insights) Analyzes changes in the cardinality of features and tags over time, highlighting unusual variations that may indicate data quality issues. [**Embedding Summarization**](https://arize.com/docs/ax/machine-learning/computer-vision/how-to-cv/embedding-summarization) Provides concise summaries of embedding data, helping you quickly understand patterns and insights from your models' embeddings [**ArizeQL Generator**](https://arize.com/docs/ax/machine-learning/machine-learning/how-to-ml/custom-metrics-api/arizeql-generator) Generates custom metrics by translating natural language descriptions or existing code (e.g., SQL, Python) into AQL for easy application. [**Dashboard Generator**](https://arize.com/docs/ax/observe/dashboards/dashboard-widget-creation) Generates dashboard widgets by translating natural language descriptions or existing code. [](https://arize.com/docs/ax/alyx/arize-copilot#data-privacy) Data Privacy ------------------------------------------------------------------------------- Arize's Alyx is built on Azure OpenAI because of its built-in security and compliance features. This ensures that customer data is protected and not directly exposed to third-party providers. Here's how it works: * **Data Processing:** Azure acts as the data processor for the prompts and outputs sent to and generated by Alyx. The models are stateless, meaning no prompts or outputs are stored in the model. * **No Data Sharing or Model Improvement:** * Inputs and outputs of Alyx are **NOT** used to improve OpenAI models. * They are **NOT** used to improve any Microsoft or third-party products or services. * They are **NOT** used for automatically improving Azure OpenAI models. * **Full Control by Microsoft:** The Azure OpenAI Service is fully controlled by Microsoft, and the OpenAI models are hosted in Microsoft’s Azure environment. The service does **NOT** interact with any other OpenAI-operated services, such as ChatGPT or the OpenAI API. * **Security & Compliance:** Azure OpenAI ensures that we meet industry-standard security and compliance measures, protecting your data throughout the process. For a detailed breakdown of the data flow and additional privacy measures, refer to the diagram below: ![](https://arize.com/docs/ax/~gitbook/image?url=https%3A%2F%2F2088270005-files.gitbook.io%2F%7E%2Ffiles%2Fv0%2Fb%2Fgitbook-x-prod.appspot.com%2Fo%2Fspaces%252F-MAlgpMyBRcl2qFZRQ67%252Fuploads%252Fgit-blob-4c188401f175c6e2c507e2361a0c90915eef9342%252FScreenshot%25202024-09-10%2520at%25209.14.49%25E2%2580%25AFPM.png%3Falt%3Dmedia&width=768&dpr=4&quality=100&sign=81e26235&sv=2) To read more about Azure Data privacy, see their documentation [here](https://learn.microsoft.com/en-us/legal/cognitive-services/openai/data-privacy?tabs=azure-portal) . If you have further questions or need more clarification on how your data is managed, feel free to contact our team at support@arize.com. [](https://arize.com/docs/ax/alyx/arize-copilot#alyx-settings) Alyx Settings --------------------------------------------------------------------------------- Alyx ships with a built-in shortcut so you can easily open Alyx or pass in highlighted text as context * **macOS:** ⌘ + L * **Windows / Linux:** Ctrl + L Edit your hotkey via the **Settings** modal, accessible through the menu in the top right corner of the Alyx chat interface. [](https://arize.com/docs/ax/alyx/arize-copilot#third-party-integrations) Third-Party Integrations ------------------------------------------------------------------------------------------------------- Alyx includes a support skill designed to help answer user questions. When you ask a support-related question, the question itself is sent to a third-party service, **RunLLM**, for processing. It's important to note that: * **Limited Data Sharing:** Only the specific question you ask is shared with RunLLM. No additional model information or user data is shared beyond the question itself. * **User Control:** You retain control over your interaction with this skill. If at any time you wish to modify or revoke your consent to share support questions with RunLLM, please contact us at support@arize.com * **Disclaimer Acknowledgement:** Before using the support skill, users must acknowledge a one-time disclaimer outlining the involvement of RunLLM. Last updated 4 months ago Was this helpful? --- # Alyx: Eval & RAG Analysis | Arize Docs [](https://arize.com/docs/ax/alyx/arize-copilot/ai-powered-eval-analysis#use-alyx-to-understand-your-evaluations) Use Alyx to Understand Your Evaluations -------------------------------------------------------------------------------------------------------------------------------------------------------------- Rather than manually combing through your evaluation results, ✨Alyx [assesses and summarizes](https://arize.com/docs/ax/alyx/arize-copilot/ai-powered-eval-analysis) your evaluation metrics, offering targeted suggestions for improving your LLM application. These insights help you identify shortcomings in your application and make the necessary adjustments to enhance performance. ### [](https://arize.com/docs/ax/alyx/arize-copilot/ai-powered-eval-analysis#id-1.-summarize-evaluation-metrics) 1\. Summarize Evaluation Metrics * **Suggested Prompt:** "Summarize my eval" * **Use When:** You want to understand the performance of your LLM application. * **Description:** Assesses and summarizes your evaluation metrics, offering targeted suggestions for enhancing your LLM application. ### [](https://arize.com/docs/ax/alyx/arize-copilot/ai-powered-eval-analysis#id-2.-rag-analysis) 2\. RAG Analysis **Debug retrieval directly from trace details or in the main chat** * **Suggested Prompt:** "Debug retrieval step of RAG app for " * **Use When:** You need to refine your retrieval process. * **Description:** Evaluate and refine your retrieval process. Analyzes responses to ensure relevance and accuracy, and provides improvement strategies. Last updated 1 month ago Was this helpful? --- # Arize AI Named to Forbes AI 50 List of Most Promising Artificial Intelligence Companies of 2021 - Arize AI ![](https://arize.com/wp-content/uploads/2021/06/krystal-headshot-e1624425208666-196x196.jpg "krystal headshot") [Krystal Kirkland](https://arize.com/author/krystal-kirkland/) Software Engineer Share * [![](https://arize.com/wp-content/themes/arize-2022/images/icon-tw-circle.svg)](https://twitter.com/intent/tweet?url=https%3A%2F%2Farize.com%2Fblog%2Farize-ai-named-to-forbes-ai-50-2021%2F&text=Arize%20AI%20Named%20to%20Forbes%20AI%2050%20List%20of%20Most%20Promising%20Artificial%20Intelligence%20Companies%20of%202021) * [![](https://arize.com/wp-content/themes/arize-2022/images/icon-fb-circle.svg)](https://www.facguide.com/sharer/sharer.php?u=https://arize.com/blog/arize-ai-named-to-forbes-ai-50-2021/&picture=https://arize.com) * [![](https://arize.com/wp-content/themes/arize-2022/images/icon-in-circle.svg)](https://www.linkedin.com/shareArticle?mini=true&url=https%3A%2F%2Farize.com%2Fblog%2Farize-ai-named-to-forbes-ai-50-2021%2F&title=Arize%20AI%20Named%20to%20Forbes%20AI%2050%20List%20of%20Most%20Promising%20Artificial%20Intelligence%20Companies%20of%202021) ![](https://arize.com/wp-content/themes/arize-2022/images/icon-newsletter.svg) Subscribe to the Arize blog [Get the latest](https://arize.com/blog/arize-ai-named-to-forbes-ai-50-2021/#blog-subscribe-modal) #### On this page #### Suggested reading ![](https://arize.com/wp-content/uploads/2025/11/microsoft-foundry-arize-ax.png) [Evaluating and Improving AI Agents at Scale with Microsoft Foundry](https://arize.com/blog/evaluating-and-improving-ai-agents-at-scale-with-microsoft-foundry/) ![](https://arize.com/wp-content/uploads/2025/10/llm-tracing-blog-cover.png) [Top LLM Tracing Tools](https://arize.com/blog/top-llm-tracing-tools/) ![](https://arize.com/wp-content/uploads/2021/06/Screen-Shot-2021-06-23-at-4.30.00-PM-2142x1120.png "Screen Shot 2021-06-23 at 4.30.00 PM") Arize AI Named to Forbes AI 50 List of Most Promising Artificial Intelligence Companies of 2021 =============================================================================================== Published Apr 30, 2021 * [Company](https://arize.com/blog/?cat=company) * [Uncategorized](https://arize.com/blog/?cat=uncategorized) ![](https://arize.com/wp-content/uploads/2021/06/krystal-headshot-e1624425208666-196x196.jpg "krystal headshot") #### [Krystal Kirkland](https://arize.com/author/krystal-kirkland/) ##### Software Engineer We’re thrilled to announce that [Arize AI](https://arize.com/) , the leading Machine Learning (ML) Observability company, has been named to the [Forbes AI 50](https://www.forbes.com/sites/alanohnsman/2021/04/26/ai-50-americas-most-promising-artificial-intelligence-companies/?sh=71bb2f177cf1) , a list of the top private companies using artificial intelligence to transform industries! The [Forbes AI 50](https://c212.net/c/link/?t=0&l=en&o=3145224-1&h=2695710924&u=https%3A%2F%2Fwww.forbes.com%2Fsites%2Falanohnsman%2F2021%2F04%2F26%2Fai-50-americas-most-promising-artificial-intelligence-companies%2F%3Fsh%3D2af289ed77cf&a=Forbes+AI+50) list, in its third year, includes a list of private North American companies using artificial intelligence in ways that are fundamental to their operations, such as machine learning, natural language processing, and computer vision. Today, companies spend millions of dollars developing and implementing ML models, only to see a myriad of unexpected performance degradation issues arise. Models that don’t perform after the code is shipped are painful to troubleshoot and negatively impact business operations and results. “Arize AI is squarely focused on the last mile of AI: models that are in production and making decisions that can cost businesses millions of dollars a day,” said Jason Lopatecki, co-founder and CEO of Arize. “We are excited that the AI 50 panel recognizes the importance of software that can monitor, troubleshoot, explain and provide guardrails on AI, as it is deployed into the real world, and views Arize AI as a leader in this category.” In partnership with Sequoia Capital and Meritech Capital, Forbes evaluated hundreds of submissions from the U.S. and Canada. A panel of expert AI judges then reviewed the finalists to hand-pick the 50 most compelling companies. **About Arize AI** Arize AI was founded by leaders in the Machine Learning (ML) Infrastructure and analytics space to bring better visibility and performance management over AI. Arize AI built the first ML Observability platform to help make machine learning models work in production. As models move from research to the real world, we provide a real-time platform to monitor, explain and troubleshoot model/data issues. Share * [![](https://arize.com/wp-content/themes/arize-2022/images/icon-tw-circle.svg)](https://twitter.com/intent/tweet?url=https%3A%2F%2Farize.com%2Fblog%2Farize-ai-named-to-forbes-ai-50-2021%2F&text=Arize%20AI%20Named%20to%20Forbes%20AI%2050%20List%20of%20Most%20Promising%20Artificial%20Intelligence%20Companies%20of%202021) * [![](https://arize.com/wp-content/themes/arize-2022/images/icon-fb-circle.svg)](https://www.facguide.com/sharer/sharer.php?u=https://arize.com/blog/arize-ai-named-to-forbes-ai-50-2021/&picture=https://arize.com) * [![](https://arize.com/wp-content/themes/arize-2022/images/icon-in-circle.svg)](https://www.linkedin.com/shareArticle?mini=true&url=https%3A%2F%2Farize.com%2Fblog%2Farize-ai-named-to-forbes-ai-50-2021%2F&title=Arize%20AI%20Named%20to%20Forbes%20AI%2050%20List%20of%20Most%20Promising%20Artificial%20Intelligence%20Companies%20of%202021) #### Suggested reading ![](https://arize.com/wp-content/uploads/2025/11/microsoft-foundry-arize-ax.png) [Evaluating and Improving AI Agents at Scale with Microsoft Foundry](https://arize.com/blog/evaluating-and-improving-ai-agents-at-scale-with-microsoft-foundry/) ![](https://arize.com/wp-content/uploads/2025/10/llm-tracing-blog-cover.png) [Top LLM Tracing Tools](https://arize.com/blog/top-llm-tracing-tools/) Sign up for our monthly newsletter, The Evaluator. -------------------------------------------------- [Subscribe](https://arize.com/blog/arize-ai-named-to-forbes-ai-50-2021/#blog-subscribe-modal) Subscribe to The Evaluator -------------------------- We’ll send you the latest news, expertise, and product updates from Arize. Your inbox is sacred, so we’ll only curate and send the best stuff. \*We’re committed to your privacy. Arize uses the information you provide to contact you about relevant content, products, and services. You may unsubscribe from these communications at any time. For more information, check out our [privacy policy](https://arize.com/privacy-policy) . --- # Alyx: ArizeQL Generator | Arize Docs Need help creating a custom metric? Alyx can assist! Simply describe your desired metric or provide existing code in any language like SQL or Python, and Copilot will translate it to AQL. Apply the metric, save it, and you're all set. ![](https://arize.com/docs/ax/~gitbook/image?url=https%3A%2F%2F2088270005-files.gitbook.io%2F%7E%2Ffiles%2Fv0%2Fb%2Fgitbook-x-prod.appspot.com%2Fo%2Fspaces%252F-MAlgpMyBRcl2qFZRQ67%252Fuploads%252Fgit-blob-f9e4d4d884640beb82673e7ab898f8f46c66e0f3%252Fcustom_metrics.gif%3Falt%3Dmedia&width=768&dpr=4&quality=100&sign=c7d912b4&sv=2) Last updated 4 months ago Was this helpful? --- # Arize AI Wins 2020 AI TechAward for Enterprise AI - Arize AI ![](https://arize.com/wp-content/uploads/2021/06/krystal-headshot-e1624425208666-196x196.jpg "krystal headshot") [Krystal Kirkland](https://arize.com/author/krystal-kirkland/) Software Engineer Share * [![](https://arize.com/wp-content/themes/arize-2022/images/icon-tw-circle.svg)](https://twitter.com/intent/tweet?url=https%3A%2F%2Farize.com%2Fblog%2Farize-ai-wins-2020-ai-techaward-for-enterprise-ai%2F&text=Arize%20AI%20Wins%202020%20AI%20TechAward%20for%20Enterprise%20AI) * [![](https://arize.com/wp-content/themes/arize-2022/images/icon-fb-circle.svg)](https://www.facguide.com/sharer/sharer.php?u=https://arize.com/blog/arize-ai-wins-2020-ai-techaward-for-enterprise-ai/&picture=https://arize.com) * [![](https://arize.com/wp-content/themes/arize-2022/images/icon-in-circle.svg)](https://www.linkedin.com/shareArticle?mini=true&url=https%3A%2F%2Farize.com%2Fblog%2Farize-ai-wins-2020-ai-techaward-for-enterprise-ai%2F&title=Arize%20AI%20Wins%202020%20AI%20TechAward%20for%20Enterprise%20AI) ![](https://arize.com/wp-content/themes/arize-2022/images/icon-newsletter.svg) Subscribe to the Arize blog [Get the latest](https://arize.com/blog/arize-ai-wins-2020-ai-techaward-for-enterprise-ai/#blog-subscribe-modal) #### On this page #### Suggested reading ![](https://arize.com/wp-content/uploads/2025/11/microsoft-foundry-arize-ax.png) [Evaluating and Improving AI Agents at Scale with Microsoft Foundry](https://arize.com/blog/evaluating-and-improving-ai-agents-at-scale-with-microsoft-foundry/) ![](https://arize.com/wp-content/uploads/2025/10/llm-tracing-blog-cover.png) [Top LLM Tracing Tools](https://arize.com/blog/top-llm-tracing-tools/) ![](https://arize.com/wp-content/uploads/2020/10/AdobeStock_369481241-2142x1120.jpeg "Team of happy employees winning award and celebrating success. Business people enjoying victory, getting gold cup trophy. Vector illustration for reward, prize, champions concepts") Arize AI Wins 2020 AI TechAward for Enterprise AI ================================================= Published Oct 21, 2020 * [Company](https://arize.com/blog/?cat=company) * [Uncategorized](https://arize.com/blog/?cat=uncategorized) ![](https://arize.com/wp-content/uploads/2021/06/krystal-headshot-e1624425208666-196x196.jpg "krystal headshot") #### [Krystal Kirkland](https://arize.com/author/krystal-kirkland/) ##### Software Engineer BERKELEY, Calif., Oct. 21, 2020 /PRNewswire/ — [Arize AI](https://c212.net/c/link/?t=0&l=en&o=2956148-1&h=962430787&u=http%3A%2F%2Farize.com%2F%3Futm_source%3Dprsnews%26utm_medium%3Dtext%26utm_campaign%3DPrsnews1&a=Arize+AI) , We’re excited to announce that Arize AI has won an 2020 AI TechAward in the category: **Enterprise AI.** The [](https://c212.net/c/link/?t=0&l=en&o=2956148-1&h=1438162186&u=https%3A%2F%2Faidevworld.com%2Fawards%2F&a=%C2%A0) [2020 AI Tech Awards](https://c212.net/c/link/?t=0&l=en&o=2956148-1&h=2724143631&u=https%3A%2F%2Faidevworld.com%2Fawards%2F&a=2020+AI+Tech+Awards)  celebrates technical innovation, adoption and reception in the AI, Machine Learning & Data Science industry and by the developer community. The 2020 AI TechAwards will be presented at the 2020 AI TechAwards Ceremony during  [](https://c212.net/c/link/?t=0&l=en&o=2956148-1&h=1030521444&u=https%3A%2F%2Faidevworld.com%2F&a=%C2%A0) [AI DevWorld Virtual](https://c212.net/c/link/?t=0&l=en&o=2956148-1&h=2898935231&u=https%3A%2F%2Faidevworld.com%2F&a=AI+DevWorld+Virtual)  (Oct 27-29, 2020), the largest Artificial Intelligence, Machine Learning & Data Science conference with tracks covering NLP, Open Source AI, AI for the Enterprise, Deep AI, Neural Networks and more. The 2020 AI TechAwards received hundreds of nominations, and the Advisory Board to the AI TechAwards selected our product/technology based on three criteria: (1) attracting notable attention and awareness in the AI, Machine Learning & Data Science industry; (2) general regard and use by the developer & engineering community; and (3) being a leader in its sector for innovation. “Arize AI is a great example of the newest AI & Machine Learning technologies now allowing developers & engineers to build upon the burgeoning AI industry. Today’s cloud-based software and hardware increasingly runs on systems needing increased data and intelligence, and **Arize AI’s** win here at the 2020 AI TechAwards is evidence of their leading role in the growth of the AI ecosystem,” said Jonathan Pasky, Executive Producer & Co-Founder of DevNetwork, producer of AI Dev World & the 2019 AI TechAwards. “We are excited to be recognized by AI TechAwards. Arize AI is the leading ML Observability platform in the market. We help make models successful in production with deep troubleshooting, monitoring, and explainability solutions. As businesses deploy more models deploy more models into production and these models get more complex, model observability is key to making models successful.” — Jason Lopatecki, CEO of Arize AI **About Arize AI** Arize AI was founded by leaders in the Machine Learning (ML) Infrastructure and analytics space to bring better visibility and performance management over AI. Arize AI built the first ML Observability platform to help make machine learning models work in production. We provide a real time platform to monitor, explain and troubleshoot model/data issues, as models move from research to real world. Share * [![](https://arize.com/wp-content/themes/arize-2022/images/icon-tw-circle.svg)](https://twitter.com/intent/tweet?url=https%3A%2F%2Farize.com%2Fblog%2Farize-ai-wins-2020-ai-techaward-for-enterprise-ai%2F&text=Arize%20AI%20Wins%202020%20AI%20TechAward%20for%20Enterprise%20AI) * [![](https://arize.com/wp-content/themes/arize-2022/images/icon-fb-circle.svg)](https://www.facguide.com/sharer/sharer.php?u=https://arize.com/blog/arize-ai-wins-2020-ai-techaward-for-enterprise-ai/&picture=https://arize.com) * [![](https://arize.com/wp-content/themes/arize-2022/images/icon-in-circle.svg)](https://www.linkedin.com/shareArticle?mini=true&url=https%3A%2F%2Farize.com%2Fblog%2Farize-ai-wins-2020-ai-techaward-for-enterprise-ai%2F&title=Arize%20AI%20Wins%202020%20AI%20TechAward%20for%20Enterprise%20AI) #### Suggested reading ![](https://arize.com/wp-content/uploads/2025/11/microsoft-foundry-arize-ax.png) [Evaluating and Improving AI Agents at Scale with Microsoft Foundry](https://arize.com/blog/evaluating-and-improving-ai-agents-at-scale-with-microsoft-foundry/) ![](https://arize.com/wp-content/uploads/2025/10/llm-tracing-blog-cover.png) [Top LLM Tracing Tools](https://arize.com/blog/top-llm-tracing-tools/) Sign up for our monthly newsletter, The Evaluator. -------------------------------------------------- [Subscribe](https://arize.com/blog/arize-ai-wins-2020-ai-techaward-for-enterprise-ai/#blog-subscribe-modal) Subscribe to The Evaluator -------------------------- We’ll send you the latest news, expertise, and product updates from Arize. Your inbox is sacred, so we’ll only curate and send the best stuff. \*We’re committed to your privacy. Arize uses the information you provide to contact you about relevant content, products, and services. You may unsubscribe from these communications at any time. For more information, check out our [privacy policy](https://arize.com/privacy-policy) . --- # Pandas Batch Logging | Arize Docs Use the `arize` Python library to monitor machine learning predictions with a few lines of code in a Jupyter Notebook or a Python server that batch processes backend data The most commonly used functions/objects are: [`Client`](https://arize.com/docs/ax/machine-learning/machine-learning/api-reference-ml/python-sdk/arize.pandas/client) — Initialize to begin logging model data to Arize [`Schema`](https://arize.com/docs/ax/machine-learning/machine-learning/api-reference-ml/python-sdk/arize.pandas/schema) — Organize and map column names containing model data within your Pandas dataframe. [`log`](https://arize.com/docs/ax/machine-learning/machine-learning/api-reference-ml/python-sdk/arize.pandas/log) — Log inferences within a dataframe to Arize via a POST request. ### [](https://arize.com/docs/ax/machine-learning/machine-learning/api-reference-ml/python-sdk/arize.pandas#python-pandas-example) Python Pandas Example For examples and interactive notebooks, see [Cookbooks](https://arize.com/docs/ax/cookbooks) Copy # install and import dependencies !pip install -q arize import datetime from arize.pandas.logger import Client from arize.utils.types import ModelTypes, Environments, Schema, Metrics import numpy as np import pandas as pd # create Arize client SPACE_ID = "SPACE_ID" API_KEY = "API_KEY" arize_client = Client(space_id=SPACE_ID, api_key=API_KEY) # define schema schema = Schema( prediction_id_column_name="prediction_id", timestamp_column_name="prediction_ts", prediction_label_column_name="predicted_label", actual_label_column_name="actual_label", feature_column_names=feature_column_names, tag_column_names=TypedSchema( inferred=["tag1", "tag3"], to_int=["tag2"], ) ) #log data response = arize_client.log( dataframe=df, schema=schema, model_id="binary-classification-metrics-only-batch-ingestion-tutorial", model_version="1.0.0", model_type=ModelTypes.BINARY_CLASSIFICATION, metrics_validation=[Metrics.CLASSIFICATION], validate=True, environment=Environments.PRODUCTION ) Follow this example in Google Colab: [![Logo](https://arize.com/docs/ax/~gitbook/image?url=https%3A%2F%2Fssl.gstatic.com%2Fcolaboratory-static%2Fcommon%2F20a9eda6ca436612e5341069a97a79fb%2Fimg%2Ffavicon.ico&width=20&dpr=4&quality=100&sign=bd67196d&sv=2)Google Colabcolab.research.google.com](https://colab.research.google.com/github/Arize-ai/tutorials_python/blob/main/Arize_Tutorials/Data_Ingestion/Binary_Classification/binary_classification_metrics_only_batch_ingestion_tutorial.ipynb) ### [](https://arize.com/docs/ax/machine-learning/machine-learning/api-reference-ml/python-sdk/arize.pandas#benchmark-tests) Benchmark Tests The ability to ingest data with low latency is important to many customers. Below is a benchmarking colab that demonstrates the efficiency with which Arize uploads data from a Python environment. Sending 10 Million Inferences to Arize in 90 Seconds [Colab Link](https://colab.research.google.com/github/Arize-ai/tutorials_python/blob/main/Benchmarks/arize_3_1_1_test.ipynb) Last updated 4 months ago Was this helpful? --- # Single Record Logging | Arize Docs This API is designed for record-by-record data ingestion. **It is not designed for a large set of data**. If you need a faster experience for batch logging, check out the [arize.pandas](https://arize.com/docs/ax/machine-learning/machine-learning/api-reference-ml/python-sdk/arize.pandas) . The most commonly used functions/objects are: [`Client`](https://arize.com/docs/ax/machine-learning/machine-learning/api-reference-ml/python-sdk/arize-log/client) — Initialize to begin logging model data to Arize [`log`](https://arize.com/docs/ax/machine-learning/machine-learning/api-reference-ml/java-sdk/log) — Log inferences record-by-record ### [](https://arize.com/docs/ax/machine-learning/machine-learning/api-reference-ml/python-sdk/arize-log#install-the-package) Install the Package Copy pip install arize ### [](https://arize.com/docs/ax/machine-learning/machine-learning/api-reference-ml/python-sdk/arize-log#initialize-arize-client) Initialize Arize Client Initialize Arize [`Client`](https://arize.com/docs/ax/machine-learning/machine-learning/api-reference-ml/python-sdk/arize-log/client) to begin logging model inferences. Copy from arize.api import Client # create Arize client SPACE_ID = "YOUR_SPACE_ID" API_KEY = "YOUR_API_KEY" arize_client = Client(space_id=SPACE_ID, api_key=API_KEY) [](https://arize.com/docs/ax/machine-learning/machine-learning/api-reference-ml/python-sdk/arize-log#single-record-examples) Single Record Examples -------------------------------------------------------------------------------------------------------------------------------------------------------- For examples and interactive notebooks, see [Cookbooks](https://arize.com/docs/ax/cookbooks) ### [](https://arize.com/docs/ax/machine-learning/machine-learning/api-reference-ml/python-sdk/arize-log#example-1-logging-features-tags-and-predictions-only) Example 1: Logging Features, Tags, and Predictions Only Copy # Example features; features & tags can be optionally defined with typing features = { 'state': 'ca', 'city': 'berkeley', 'merchant_name': 'Peets Coffee', 'pos_approved': TypedValue(value=False, type=ArizeTypes.INT), 'item_count': 10, 'merchant_type': 'coffee shop', 'charge_amount': TypedValue(value=20.11, type=ArizeTypes.FLOAT), } # example tags tags = { 'age': 30, 'zip_code': '94610', 'device_os': 'iOS', 'server_node_id': 12, } # example embeddings embedding_features = { 'image_embedding': Embedding( vector=np.array([1.0, 2, 3]), link_to_data='https://my-bucket.s3.us-west-2.amazonaws.com/puppy.png', ), 'nlp_embedding_sentence': Embedding( vector=pd.Series([4.0, 5.0, 6.0, 7.0]), data='This is a test sentence', ), 'nlp_embedding_tokens': Embedding( vector=pd.Series([4.0, 5.0, 6.0, 7.0]), data=['This', 'is', 'a', 'sample', 'token', 'array'], ), } # log the prediction response = arize_client.log( prediction_id='plED4eERDCasd9797ca34', model_id='sample-model-1', model_type=ModelTypes.SCORE_CATEGORICAL, environment=Environments.PRODUCTION, model_version='v1', prediction_timestamp=1618590882, prediction_label=('Fraud',.4) features=features, embedding_features=embedding_features tags=tags ) # Listen to response code to ensure successful delivery res = response.result() if res.status_code == 200: print('Success sending Prediction!') else: print(f'Log failed with response code {res.status_code}, {res.text}') ### [](https://arize.com/docs/ax/machine-learning/machine-learning/api-reference-ml/python-sdk/arize-log#example-2-logging-features-and-predictions-first-then-delayed-actuals) Example 2: Logging Features & Predictions First, Then Delayed Actuals Copy # Example features; features & tags can be optionally defined with typing features = { 'state': 'ca', 'city': 'berkeley', 'merchant_name': 'Peets Coffee', 'pos_approved': TypedValue(value=False, type=ArizeTypes.INT), 'item_count': 10, 'merchant_type': 'coffee shop', 'charge_amount': TypedValue(value=20.11, type=ArizeTypes.FLOAT), } # log the features & prediction response = arize_client.log( prediction_id='plED4eERDCasd9797ca34', model_id='sample-model-1', model_type=ModelTypes.SCORE_CATEGORICAL, environment=Environments.PRODUCTION, model_version='v1', prediction_timestamp=1618590882, features=features, prediction_label=('Fraud',.4), tags=tags ) res = response.result() if res.status_code == 200: print('Success sending Prediction!') else: print(f'Log failed with response code {res.status_code}, {res.text}') # log the actual actual_response = arize_client.log( prediction_id='plED4eERDCasd9797ca34', model_id='sample-model-1', model_type=ModelTypes.SCORE_CATEGORICAL, environment=Environments.PRODUCTION, actual_label=('Fraud',1), tags=tags) # Listen to response code to ensure successful delivery res = actual_response.result() if res.status_code == 200: print('Success sending Actual!') else: print(f'Log failed with response code {res.status_code}, {res.text}') ### [](https://arize.com/docs/ax/machine-learning/machine-learning/api-reference-ml/python-sdk/arize-log#example-3-logging-features-predictions-and-actuals-together) Example 3: Logging Features, Predictions and Actuals Together Copy # Example features; features & tags can be optionally defined with typing features = { 'state': 'ca', 'city': 'berkeley', 'merchant_name': 'Peets Coffee', 'pos_approved': TypedValue(value=False, type=ArizeTypes.INT), 'item_count': 10, 'merchant_type': 'coffee shop', 'charge_amount': TypedValue(value=20.11, type=ArizeTypes.FLOAT), } # log the prediction, actual, and features response = arize_client.log( prediction_id='plED4eERDCasd9797ca34', model_id='sample-model-1', model_type=ModelTypes.SCORE_CATEGORICAL, environment=Environments.PRODUCTION, model_version='v1', prediction_timestamp=1618590882, features=features, prediction_label=('False', .4), actual_label=('True', 1), tags=tags ) # Listen to response code to ensure successful delivery res = response.result() if res.status_code == 200: print('Success sending Prediction and Actual!') else: print(f'Log failed with response code {res.status_code}, {res.text}') ### [](https://arize.com/docs/ax/machine-learning/machine-learning/api-reference-ml/python-sdk/arize-log#example-4-logging-predictions-actuals-and-shap-together) Example 4: Logging Predictions, Actuals, and SHAP Together Copy # Example features; features & tags can be optionally defined with typing features = { 'state': 'ca', 'city': 'berkeley', 'merchant_name': 'Peets Coffee', 'pos_approved': TypedValue(value=False, type=ArizeTypes.INT), 'item_count': 10, 'merchant_type': 'coffee shop', 'charge_amount': TypedValue(value=20.11, type=ArizeTypes.FLOAT), } # example SHAP values shaps = { 'state': 0.23, 'city': 0.31, 'merchant_name': 0.10, 'pos_approved': 0.02, 'item_count': 0.06, 'merchant_type': 0.11, 'charge_amount': 0.29, } # log the prediction, actual, features, and shap response = arize_client.log( prediction_id='plED4eERDCasd9797ca34', model_id='sample-model-1', model_type=ModelTypes.SCORE_CATEGORICAL, environment=Environments.PRODUCTION, model_version='v1', prediction_timestamp=1618590882, features=features, prediction_label=('False', .4), actual_label=('True',1), tags=tags, shap_values=shaps) # Listen to response code to ensure successful delivery res = response.result() if res.status_code == 200: print('Success sending Prediction, Actual, and SHAPs!') else: print(f'Log failed with response code {res.status_code}, {res.text}') Questions? Email us at [support@arize.com](mailto::support@arize.com) or [Slack us](https://arize-ai.slack.com/) in the #arize-support channel Last updated 1 month ago Was this helpful? --- # Alyx: ArizeQL Generator | Arize Docs Need help creating a custom metric? Alyx can assist! Simply describe your desired metric or provide existing code in any language like SQL or Python, and Alyx will translate it to AQL. Apply the metric, save it, and you're all set. ![](https://arize.com/docs/ax/~gitbook/image?url=https%3A%2F%2F2088270005-files.gitbook.io%2F%7E%2Ffiles%2Fv0%2Fb%2Fgitbook-x-prod.appspot.com%2Fo%2Fspaces%252F-MAlgpMyBRcl2qFZRQ67%252Fuploads%252Fgit-blob-f9e4d4d884640beb82673e7ab898f8f46c66e0f3%252Fcustom_metrics.gif%3Falt%3Dmedia&width=768&dpr=4&quality=100&sign=c7d912b4&sv=2) Last updated 4 months ago Was this helpful? --- # Arize Receives Certifications Validating Health Information Security for HIPAA Compliance - Arize AI ![jim groff compliance officer arize ai](https://arize.com/wp-content/uploads/2022/03/Jim-Groff-196x196.png "Jim Groff") [Jim Groff](https://arize.com/author/jim-groff/) Compliance Officer Share * [![](https://arize.com/wp-content/themes/arize-2022/images/icon-tw-circle.svg)](https://twitter.com/intent/tweet?url=https%3A%2F%2Farize.com%2Fblog%2Farize-receives-certifications-validating-health-information-security-for-hipaa-compliance%2F&text=Arize%20Receives%20Certifications%20Validating%20Health%20Information%20Security%20for%20HIPAA%20Compliance) * [![](https://arize.com/wp-content/themes/arize-2022/images/icon-fb-circle.svg)](https://www.facguide.com/sharer/sharer.php?u=https://arize.com/blog/arize-receives-certifications-validating-health-information-security-for-hipaa-compliance/&picture=https://arize.com) * [![](https://arize.com/wp-content/themes/arize-2022/images/icon-in-circle.svg)](https://www.linkedin.com/shareArticle?mini=true&url=https%3A%2F%2Farize.com%2Fblog%2Farize-receives-certifications-validating-health-information-security-for-hipaa-compliance%2F&title=Arize%20Receives%20Certifications%20Validating%20Health%20Information%20Security%20for%20HIPAA%20Compliance) ![](https://arize.com/wp-content/themes/arize-2022/images/icon-newsletter.svg) Subscribe to the Arize blog [Get the latest](https://arize.com/blog/arize-receives-certifications-validating-health-information-security-for-hipaa-compliance/#blog-subscribe-modal) #### On this page #### Suggested reading ![](https://arize.com/wp-content/uploads/2025/10/iso-iec-27001-certified-icon.jpg) [Arize AI Achieves ISO/IEC 27001 Certification](https://arize.com/blog/arize-ai-achieves-iso-iec-27001-certification/) ![](https://arize.com/wp-content/uploads/2025/03/NVIDIA-Arize-blog.jpg) [Self-Improving Agents: Automating LLM Performance Optimization using Arize and NVIDIA NeMo](https://arize.com/blog/arize-nvidia-nemo-integration/) ![arize certified for health information security hipaa compliance](https://arize.com/wp-content/uploads/2022/08/arize-hipaa-compliant.jpg "arize-hipaa-compliant") Arize Receives Certifications Validating Health Information Security for HIPAA Compliance ========================================================================================= Published Aug 29, 2022 * [Company](https://arize.com/blog/?cat=company) ![jim groff compliance officer arize ai](https://arize.com/wp-content/uploads/2022/03/Jim-Groff-196x196.png "Jim Groff") #### [Jim Groff](https://arize.com/author/jim-groff/) ##### Compliance Officer Artificial intelligence is transforming modern healthcare. AI-focused healthcare startups [raised](https://www.cbinsights.com/research/healthcare-ai-funding-trends/) over $12 billion last year, delivering everything from life-saving [interventions in cancer care](https://www.wsj.com/articles/fda-authorizes-ai-software-designed-to-help-spot-prostate-cancer-11632780683) to [reductions](https://www.wsj.com/articles/anthem-looks-to-fuel-ai-efforts-with-petabytes-of-synthetic-data-11652781602?mod=pls_whats_news_us_business_f) in claims fraud. Large insurers, providers, and pharmaceutical companies are also [investing](https://www.unitedhealthgroup.com/newsroom/2020/2020-10-27-optum-growing-need-ai-expertise.html) in machine learning to improve health outcomes and the overall patient experience. Altogether, Accenture [predicts](https://www.forbes.com/sites/insights-intelai/2019/02/11/ai-and-healthcare-a-giant-opportunity/) that deployed AI may save the healthcare industry over $150 billion by 2026. Despite this early progress, the industry still faces formidable challenges in embracing AI at scale. Building, deploying, and maintaining models (i.e. NLP models scanning electronic medical records for insights) is uniquely challenging in healthcare. According to a [recent survey](https://arize.com/wp-content/uploads/2022/02/The-Industry-is-Ready-for-Machine-Learning-Observability-at-Scale-Final.pdf) , over one in four (27.3%) machine learning engineers and data scientists in healthcare say it takes their team a week or more to detect and fix an issue with a model in production – more than most other industries surveyed. These delays and blindspots [can be costly](https://www.hsph.harvard.edu/news/hsph-in-the-news/study-widely-used-health-care-algorithm-has-racial-bias/) . Arize’s New Certifications -------------------------- In order to detect and resolve ML model issues faster, an increasing number of healthcare organizations are implementing Arize for [ML observability](https://arize.com/ml-observability/) . To give these companies peace of mind and prove compliance with applicable standards, Arize recently received certifications from an independent auditor validating that the company’s health information security program is fairly represented and includes the essential elements of the U.S. Health Insurance Portability and Accountability Act (HIPAA) Security Rule and the Health Information Technology for Economic and Clinical Health (HITECH) Act. Specifically, the independent auditor verified – via a _Type 1 Attestation: AT-C 105 and AT-C 205_ –  that applicable requirements under both laws can be met if controls at Arize are suitably designed. These healthcare-specific certifications supplement Arize’s broader SOC 2 Type II compliance, which the company [received earlier this year](https://arize.com/press/arize-ai-announces-soc-2-type-ii-certification/) . Background on HIPAA ------------------- [The Health Insurance Portability and Accountability Act of 1996 (HIPAA)](https://www.cdc.gov/phlp/publications/topic/hipaa.html) is a federal law that requires the creation of national standards to protect sensitive patient health information from being disclosed without the patient’s consent or knowledge. The US Department of Health and Human Services (HHS) issued the HIPAA Privacy Rule to implement the requirements of HIPAA. The HIPAA Security Rule protects a subset of information covered by the Privacy Rule. The Privacy Rule standards address the use and disclosure of individuals’ health information (known as “protected health information or PHI”) by entities subject to the Privacy Rule. These individuals and organizations are called “covered entities.” There are four types of covered entities under HIPAA: healthcare providers, health plans, healthcare clearinghouses, and business associates. Arize is considered a business associate, which by definition is a person or organization (other than a member of a covered entity’s workforce) using individually identifiable health information to perform or provide functions, activities, or services for a covered entity. In Arize’s case, these functions and services include things like data analysis and utilization review. Arize’s Approach To Protecting Customer Data -------------------------------------------- At Arize, we believe that safeguarding data is a core function – fundamental to earning and maintaining the trust of users, customers, and partners. That’s especially true in healthcare, which is why Arize is rolling out internal training on the importance of HIPAA. As always, security for protected health information at Arize rests on three pillars: * **Auditability** ensures that Arize always knows what happens on company systems and can fill in the key details to facilitate both internal and third party investigations. * **Prevention** is about observing company systems to consistently identify weak points to add protections and controls to ensure protected health information is secure. * **Preparedness** is also critical in a world where healthcare organizations are often a [target of hackers](https://www.healthcaredive.com/news/phishing-scam-at-presbyterian-exposes-183k-patients-data/561745/) , and HIPAA violations [even more costly](https://www.hhs.gov/hipaa/for-professionals/special-topics/hitech-act-enforcement-interim-final-rule/index.html) in the wake of the HITECH Act. Arize regularly simulates a variety of scenarios, fine-tuning written plans and processes for incident response. To learn more about security at Arize or to obtain a full copy of the report, visit the [Arize Trust Center](https://arize.com/trust-center/) or reach out directly in the [Arize community.](https://arize.com/community/) Share * [![](https://arize.com/wp-content/themes/arize-2022/images/icon-tw-circle.svg)](https://twitter.com/intent/tweet?url=https%3A%2F%2Farize.com%2Fblog%2Farize-receives-certifications-validating-health-information-security-for-hipaa-compliance%2F&text=Arize%20Receives%20Certifications%20Validating%20Health%20Information%20Security%20for%20HIPAA%20Compliance) * [![](https://arize.com/wp-content/themes/arize-2022/images/icon-fb-circle.svg)](https://www.facguide.com/sharer/sharer.php?u=https://arize.com/blog/arize-receives-certifications-validating-health-information-security-for-hipaa-compliance/&picture=https://arize.com) * [![](https://arize.com/wp-content/themes/arize-2022/images/icon-in-circle.svg)](https://www.linkedin.com/shareArticle?mini=true&url=https%3A%2F%2Farize.com%2Fblog%2Farize-receives-certifications-validating-health-information-security-for-hipaa-compliance%2F&title=Arize%20Receives%20Certifications%20Validating%20Health%20Information%20Security%20for%20HIPAA%20Compliance) #### Suggested reading ![](https://arize.com/wp-content/uploads/2025/10/iso-iec-27001-certified-icon.jpg) [Arize AI Achieves ISO/IEC 27001 Certification](https://arize.com/blog/arize-ai-achieves-iso-iec-27001-certification/) ![](https://arize.com/wp-content/uploads/2025/03/NVIDIA-Arize-blog.jpg) [Self-Improving Agents: Automating LLM Performance Optimization using Arize and NVIDIA NeMo](https://arize.com/blog/arize-nvidia-nemo-integration/) Recommended resources --------------------- [![Hugging Face and Arize Partnership and Integration Colab](https://arize.com/wp-content/uploads/2022/08/Arize-Hugging_Face-blog_cover-1.jpg)\ \ post\ \ #### Hugging Face + Arize: Partnership and Code Example\ \ Read more](https://arize.com/blog/arize-hugging-face/) [![](https://arize.com/wp-content/uploads/2022/07/claire-longo-arize-.jpg)\ \ post\ \ #### Introducing Claire Longo, Arize’s New Customer Success Lead\ \ Read more](https://arize.com/blog/introducing-claire-longo-arizes-new-customer-success-lead/) [![](https://arize.com/wp-content/uploads/2020/05/AdobeStock_351369604-2048x1024.jpeg)\ \ post\ \ #### AI in the Time of Corona\ \ Read more](https://arize.com/blog/ai-in-the-time-of-corona/) Sign up for our monthly newsletter, The Evaluator. -------------------------------------------------- [Subscribe](https://arize.com/blog/arize-receives-certifications-validating-health-information-security-for-hipaa-compliance/#blog-subscribe-modal) Subscribe to The Evaluator -------------------------- We’ll send you the latest news, expertise, and product updates from Arize. Your inbox is sacred, so we’ll only curate and send the best stuff. \*We’re committed to your privacy. Arize uses the information you provide to contact you about relevant content, products, and services. You may unsubscribe from these communications at any time. For more information, check out our [privacy policy](https://arize.com/privacy-policy) . --- # Arize AI Raises $19 Million Series A As Organizations Move To Address ML Observability, the Missing Foundational Piece of ML infrastructure - Arize AI ![](https://arize.com/wp-content/uploads/2021/03/jasonlopatecki-196x196.jpeg "jasonlopatecki") [Jason Lopatecki](https://arize.com/author/jason-lopatecki/) Co-founder and CEO Share * [![](https://arize.com/wp-content/themes/arize-2022/images/icon-tw-circle.svg)](https://twitter.com/intent/tweet?url=https%3A%2F%2Farize.com%2Fblog%2Farize-series-a-ml-observability%2F&text=Arize%20AI%20Raises%20$19%20Million%20Series%20A%20As%20Organizations%20Move%20To%20Address%20ML%20Observability,%20the%20Missing%20Foundational%20Piece%20of%20ML%20infrastructure) * [![](https://arize.com/wp-content/themes/arize-2022/images/icon-fb-circle.svg)](https://www.facguide.com/sharer/sharer.php?u=https://arize.com/blog/arize-series-a-ml-observability/&picture=https://arize.com) * [![](https://arize.com/wp-content/themes/arize-2022/images/icon-in-circle.svg)](https://www.linkedin.com/shareArticle?mini=true&url=https%3A%2F%2Farize.com%2Fblog%2Farize-series-a-ml-observability%2F&title=Arize%20AI%20Raises%20$19%20Million%20Series%20A%20As%20Organizations%20Move%20To%20Address%20ML%20Observability,%20the%20Missing%20Foundational%20Piece%20of%20ML%20infrastructure) ![](https://arize.com/wp-content/themes/arize-2022/images/icon-newsletter.svg) Subscribe to the Arize blog [Get the latest](https://arize.com/blog/arize-series-a-ml-observability/#blog-subscribe-modal) #### On this page #### Suggested reading ![](https://arize.com/wp-content/uploads/2025/10/iso-iec-27001-certified-icon.jpg) [Arize AI Achieves ISO/IEC 27001 Certification](https://arize.com/blog/arize-ai-achieves-iso-iec-27001-certification/) ![](https://arize.com/wp-content/uploads/2025/03/NVIDIA-Arize-blog.jpg) [Self-Improving Agents: Automating LLM Performance Optimization using Arize and NVIDIA NeMo](https://arize.com/blog/arize-nvidia-nemo-integration/) ![](https://arize.com/wp-content/uploads/2021/09/arize-collage-large.jpg "arize-collage-large") Arize AI Raises $19 Million Series A As Organizations Move To Address ML Observability, the Missing Foundational Piece of ML infrastructure =========================================================================================================================================== Published Sep 28, 2021 * [Company](https://arize.com/blog/?cat=company) ![](https://arize.com/wp-content/uploads/2021/03/jasonlopatecki-196x196.jpeg "jasonlopatecki") #### [Jason Lopatecki](https://arize.com/author/jason-lopatecki/) ##### Co-founder and CEO Today, almost every business is making massive investments in artificial intelligence to gain a competitive advantage. Yet, if you’re not Google, Facebook or Uber you most likely lack purpose-built, in-house systems to scale your MLOps and tools that can discover issues, diagnose problems, and improve the performance of your ML models. In response to the growing demand for observability solutions amongst organizations that rely on ML models for critical business applications, [Arize has raised $19 million in Series A funding](https://techcrunch.com/2021/09/28/battery-ventures-leads-arize-ais-19m-round-for-ml-observability/) in a round led by Battery Ventures. This partnership allows us to strengthen our commitment to helping ML practitioners obtain a deeper understanding of model performance across all stages of the model development cycle: as they are being built, once they have been deployed, and long into their life in production. ### A Pivotal Moment for AI Ethics The news could not come at a more important time as society rightly questions the ethics and potential peril of AI systems. Many of us know all too well that once a model moves from the research lab into production, modern ML systems often lack the ability to identify, evaluate and mitigate ethical risks that can disproportionately harm underrepresented and disadvantaged communities. At Arize, our vision for the future is one where today’s ethical deficits are overcome by tools that monitor, troubleshoot, explain, and provide guardrails on AI. These tools will provide a way to understand how machine learning models are making their decisions and how we can improve their results in a way that benefits businesses and society. Funding from Battery Ventures is the catalyst we need to make this vision a reality. ### About Arize AI’s Toolset We know that without the right tools to reason about mistakes a model is making in the wild, teams are investing massive amounts of time, resources and money in research environments only to fly blind when the model is up and running. With Arize, teams can observe, manage and improve machine learning models through a single pane of glass, allowing them to quickly get to the root cause of any regressions in performance or anomalous behavior.  The ability to connect points across training, validation and production allows Arize users to understand the source of an issue and build workflows to improve model outcomes over time, a key differentiator between Arize model observability and traditional model monitoring. A key attribute of the Arize platform is the ability to surface problems in a timely manner pre-and post-deployment. This includes helping a model builder detect problems before deployment, exposing specific cohorts of problematic predictions or features, and, ultimately, facilitating a tighter iteration loop between data scientists and ML engineers. In a production scenario, Arize’s platform provides automated monitoring of key model performance metrics out-of-the-box, with fully customizable dashboards and the ability to export data to share with stakeholders. ![arize](https://arize.com/wp-content/uploads/2021/09/arize-train-validate-ml-lifecycle.png) ### Arize and Battery Ventures During our most recent fundraising efforts, it was critical that our new investors were aligned with our mission and values. We found this with Battery Ventures, and are thankful for the valuable insight from Dharmesh Thakker and the rest of the Battery team. Battery has backed high-profile players in the MLOps space such as Databricks and Dataiku as well as ML Infrastructure Observability leaders AppDynamics, Sumo Logic and Datadog. Just like us, they believe that tackling the huge problem of how organizations continuously monitor and enhance ML-model performance will accelerate the use of machine learning at a greater scale across industries. And, just like us, they share deep-seated concerns about the design and future use of AI and the need for solutions that prepare organizations to implement and use AI in an ethical way. Battery invested in Arize because they believe in what we do and how we’re doing it. They know our team is uniquely positioned to make ML observability the standard for troubleshooting issues and improving the ROI of ML investments. “Data, systems, features and code all impact a model’s performance making ML observability critical to MLOps, yet incredibly complex,” notes Battery’s Thakker. “We think there is a large market opportunity for ML observability tooling to provide a deep understanding of model performance as companies continue to invest significant resources into operationalizing AI.” ### Working at Arize AI ​​As a team focused on machine learning fairness, responsible AI, and mitigating bias, we can not turn a blind eye to our team, partners, and community. Diversity and inclusion are at the very core of our DNA and we are committed to relentlessly ensuring that our values are reflected within the workplace. At Arize, we carefully develop our recruitment, [hiring](https://arize.com/careers/) , and retention processes with the goal of ensuring that our team is representative of gender, race, and identity. Today, the Arize team is 32% women, while 62% of the organization are people of color. We have explicit communication practices designed for accomplishing company and product goals. We also have established standards that ensure all team members feel welcome in our organization and that teamwork, collaboration, and [team-building](https://asana.com/resources/icebreaker-questions-team-building) create trust, empathy and a deep connection to the organizational mission. We know firsthand that implementing systems that are free from bias and ethical concerns is essential to the future or ML and cannot be achieved without diverse teams with broad skill sets and viewpoints. We’re committed to doing our part to get there. Share * [![](https://arize.com/wp-content/themes/arize-2022/images/icon-tw-circle.svg)](https://twitter.com/intent/tweet?url=https%3A%2F%2Farize.com%2Fblog%2Farize-series-a-ml-observability%2F&text=Arize%20AI%20Raises%20$19%20Million%20Series%20A%20As%20Organizations%20Move%20To%20Address%20ML%20Observability,%20the%20Missing%20Foundational%20Piece%20of%20ML%20infrastructure) * [![](https://arize.com/wp-content/themes/arize-2022/images/icon-fb-circle.svg)](https://www.facguide.com/sharer/sharer.php?u=https://arize.com/blog/arize-series-a-ml-observability/&picture=https://arize.com) * [![](https://arize.com/wp-content/themes/arize-2022/images/icon-in-circle.svg)](https://www.linkedin.com/shareArticle?mini=true&url=https%3A%2F%2Farize.com%2Fblog%2Farize-series-a-ml-observability%2F&title=Arize%20AI%20Raises%20$19%20Million%20Series%20A%20As%20Organizations%20Move%20To%20Address%20ML%20Observability,%20the%20Missing%20Foundational%20Piece%20of%20ML%20infrastructure) #### Suggested reading ![](https://arize.com/wp-content/uploads/2025/10/iso-iec-27001-certified-icon.jpg) [Arize AI Achieves ISO/IEC 27001 Certification](https://arize.com/blog/arize-ai-achieves-iso-iec-27001-certification/) ![](https://arize.com/wp-content/uploads/2025/03/NVIDIA-Arize-blog.jpg) [Self-Improving Agents: Automating LLM Performance Optimization using Arize and NVIDIA NeMo](https://arize.com/blog/arize-nvidia-nemo-integration/) Sign up for our monthly newsletter, The Evaluator. -------------------------------------------------- [Subscribe](https://arize.com/blog/arize-series-a-ml-observability/#blog-subscribe-modal) Subscribe to The Evaluator -------------------------- We’ll send you the latest news, expertise, and product updates from Arize. Your inbox is sacred, so we’ll only curate and send the best stuff. \*We’re committed to your privacy. Arize uses the information you provide to contact you about relevant content, products, and services. You may unsubscribe from these communications at any time. For more information, check out our [privacy policy](https://arize.com/privacy-policy) . --- # Arize Audit Log | Arize Docs [](https://arize.com/docs/ax/security-and-settings/compliance/arize-audit-log#overview) Overview ----------------------------------------------------------------------------------------------------- Audit Logs provide a comprehensive record of user activities within your Arize account. They are valuable for: * Security monitoring and compliance * Tracking login attempts * Monitoring data access * Ensuring adherence to internal policies * Investigating suspicious activities [](https://arize.com/docs/ax/security-and-settings/compliance/arize-audit-log#types-of-audit-logs) Types of Audit Logs --------------------------------------------------------------------------------------------------------------------------- Arize AX provides three types of audit logs, each capturing different aspects of user interaction with the platform: ### [](https://arize.com/docs/ax/security-and-settings/compliance/arize-audit-log#id-1.-unauthenticated-audit-logs) 1\. Unauthenticated Audit Logs These logs capture user login attempts, whether successful or unsuccessful. They include: * Email address * IP address * Success/failure status * Timestamp of the login attempt **Example Query** Copy query GetUnauthenticatedAuditLogs( $num: Int!, $startTime: DateTime, $endTime: DateTime, $cursor: String ) { account { unauthenticatedAuditLogs( first: $num, startTime: $startTime, endTime: $endTime, after: $cursor ) { pageInfo { hasNextPage endCursor } edges { node { id user { email } ip success mutationName loggedAt } } } } } ### [](https://arize.com/docs/ax/security-and-settings/compliance/arize-audit-log#id-2.-authenticated-audit-logs) 2\. Authenticated Audit Logs These logs record mutations (operations) performed by users after they have successfully logged in. They include: * Email address of the user * Operation name * Operation text (the GraphQL query) * Variables passed to the operation * Timestamp of the operation > **Note:** The `operationName` field is client-supplied. When using the Arize AX UI, these will be consistent, but be aware that this field can be manipulated by clients and should not be solely relied upon for security-critical decisions. **Example Query** Copy query GetAuthenticatedAuditLogs( $num: Int!, $startTime: DateTime, $endTime: DateTime, $cursor: String ) { account { authenticatedAuditLogs( first: $num, startTime: $startTime, endTime: $endTime, after: $cursor ) { pageInfo { hasNextPage endCursor } edges { node { id user { email } operationName operationText variables loggedAt } } } } } ### [](https://arize.com/docs/ax/security-and-settings/compliance/arize-audit-log#id-3.-exporter-audit-logs) 3\. Exporter Audit Logs These logs track when data is exported from your Arize account, helping you monitor who is downloading data and from which models. They include: * Email address of the user * Model name from which data was exported * Timestamp of the export > **Note:** Exporter logs are only created when data is actually returned. Export requests that return zero rows will not be logged. Additionally, export requests for demo models are not logged. **Example Query** Copy query GetExporterAuditLogs( $num: Int!, $startTime: DateTime, $endTime: DateTime, $cursor: String ) { account { exporterAuditLogs( first: $num, after: $cursor, startTime: $startTime, endTime: $endTime ) { pageInfo { hasNextPage endCursor } edges { node { id user { email } model { name } loggedAt } } } } } ### [](https://arize.com/docs/ax/security-and-settings/compliance/arize-audit-log#query-parameters) Query Parameters All audit log queries accept the following parameters: * `num` (required): Number of records to retrieve per page * `startTime` (optional): Start of the time range to query (ISO format). If not provided, defaults to last 30 days. * `endTime` (optional): End of the time range to query (ISO format). If not provided, defaults to now. * `cursor` (optional): Pagination cursor for retrieving additional pages of results ### [](https://arize.com/docs/ax/security-and-settings/compliance/arize-audit-log#pagination) Pagination Pagination is encouraged when working with large volumes of audit logs. Each query response includes a `pageInfo` object with: * `hasNextPage`: Boolean indicating if more records are available * `endCursor`: Cursor to use for fetching the next page of results Here's an example implementation of pagination for retrieving authenticated audit logs: Copy # Initialize variables authenticated_logs = [] params = { "num": 100, "startTime": "2025-01-01T00:00:00Z" } # Loop through all pages while True: paged_response = client.execute(authenticated_audit_logs_query, params) # Append the logs to your list authenticated_logs.extend(paged_response["account"]["authenticatedAuditLogs"]["edges"]) # If there is another page of information, point the cursor to the next page and fetch more end_cursor = paged_response["account"]["authenticatedAuditLogs"]["pageInfo"]["endCursor"] print("pageInfo end_cursor %s" % (end_cursor)) if end_cursor: print("There is another page of logs. Loading more.") params["cursor"] = end_cursor else: # No more logs to pull. The list is complete! break Last updated 14 days ago Was this helpful? --- # Arize Release Notes: Aug 8, 2024 - Arize AI ![](https://arize.com/wp-content/uploads/2022/03/David-Burch-1-196x196.png "David-Burch") [David Burch](https://arize.com/author/david-burch/) Share * [![](https://arize.com/wp-content/themes/arize-2022/images/icon-tw-circle.svg)](https://twitter.com/intent/tweet?url=https%3A%2F%2Farize.com%2Fblog%2Farize-release-notes-aug-8-2024%2F&text=Arize%20Release%20Notes:%20Aug%208,%202024) * [![](https://arize.com/wp-content/themes/arize-2022/images/icon-fb-circle.svg)](https://www.facguide.com/sharer/sharer.php?u=https://arize.com/blog/arize-release-notes-aug-8-2024/&picture=https://arize.com) * [![](https://arize.com/wp-content/themes/arize-2022/images/icon-in-circle.svg)](https://www.linkedin.com/shareArticle?mini=true&url=https%3A%2F%2Farize.com%2Fblog%2Farize-release-notes-aug-8-2024%2F&title=Arize%20Release%20Notes:%20Aug%208,%202024) ![](https://arize.com/wp-content/themes/arize-2022/images/icon-newsletter.svg) Subscribe to the Arize blog [Get the latest](https://arize.com/blog/arize-release-notes-aug-8-2024/#blog-subscribe-modal) #### On this page #### Suggested reading ![](https://arize.com/wp-content/uploads/2025/11/arize-ax-october-updates-2025.jpg) [New In Arize AX: Tags, Data Fabric, Automatic Threshold Ranges for Monitors and More](https://arize.com/blog/new-in-arize-ax-tags-data-fabric-automatic-threshold-ranges-for-monitors-and-more/) ![adb - Arize AI](https://arize.com/wp-content/uploads/2025/09/abd-cover-image-dark-3.jpg) [adb Benchmarks](https://arize.com/blog/adb-benchmarks/) ![](https://arize.com/wp-content/uploads/2024/08/release-notes-aug-2024.png "release-notes-aug-2024") Arize Release Notes: Aug 8, 2024 ================================ Published Aug 8, 2024 * [Product](https://arize.com/blog/?cat=product) * [Product Releases](https://arize.com/blog/?cat=product-releases) * [Release Notes](https://arize.com/blog/?cat=release-notes) ![](https://arize.com/wp-content/uploads/2022/03/David-Burch-1-196x196.png "David-Burch") #### [David Burch](https://arize.com/author/david-burch/) Welcome to our regular update on new releases, enhancements, and changes. What’s New ---------- ### Auto Instrumentation Automatically collect traces from an expanded set of frameworks and libraries. * [Haystack](https://docs.arize.com/arize/large-language-models/tracing/auto-instrumentation/haystack) * [LiteLLM](https://docs.arize.com/arize/large-language-models/tracing/auto-instrumentation/litellm) * [CrewAI](https://docs.arize.com/arize/large-language-models/tracing/auto-instrumentation/crewai) * [PromptFlow](https://docs.arize.com/arize/large-language-models/tracing/auto-instrumentation/prompt-flow) * [Groq](https://docs.arize.com/arize/large-language-models/tracing/auto-instrumentation/groq) ### Examples If helpful or illustrative, example notebooks are available from OpenInference on everything from RAG pipelines to building fallbacks with conditional routing with [Haystack](https://github.com/Arize-ai/openinference/tree/main/python/instrumentation/openinference-instrumentation-haystack) , [Groq](https://github.com/Arize-ai/openinference/tree/main/python/instrumentation/openinference-instrumentation-groq) , and other libraries. ### How To Instrument Your LLM Application Want to learn more about instrumentation? This new new tutorial covers key frameworks for [instrumenting an LLM application](https://arize.com/blog/different-ways-to-instrument-your-llm-application/) — including OpenTelemetry (Otel) and OpenInference — and the tradeoffs between automatic and manual instrumentation. It also shows three ways to set up manual instrumentation: using decorators, the \`with\` clause, and starting spans directly. 📚 New Content -------------- Recapping the latest video tutorials, paper readings, ebooks, self-guided learning modules, and technical posts: * ![:arrow_forward:](https://a.slack-edge.com/production-standard-emoji-assets/14.0/apple-medium/25b6-fe0f@2x.png) [Prompt Playground](https://arize.com/resource/prompt-playground/) * ![:arrow_forward:](https://a.slack-edge.com/production-standard-emoji-assets/14.0/apple-medium/25b6-fe0f@2x.png) [Online LLM Evaluations](https://arize.com/resource/online-llm-evaluations/) * ![:arrow_forward:](https://a.slack-edge.com/production-standard-emoji-assets/14.0/apple-medium/25b6-fe0f@2x.png) [LLM Token Counter](https://arize.com/resource/llm-token-counting/) * ![:construction:](https://a.slack-edge.com/production-standard-emoji-assets/14.0/apple-medium/1f6a7@2x.png) [LLM Guardrails](https://arize.com/blog-course/llm-guardrails-protecting-your-ai-application/) : How To Protect Your LLM Application, Including from Itself * ![:sparkles:](https://a.slack-edge.com/production-standard-emoji-assets/14.0/apple-medium/2728@2x.png) [Text To SQL: Evaluating SQL Gen with LLM as a Judge](https://arize.com/blog/text-to-sql-evaluating-sql-generation-with-llm-as-a-judge/) * ![:robot_face:](https://a.slack-edge.com/production-standard-emoji-assets/14.0/apple-medium/1f916@2x.png) [How To Build an Agent](https://arize.com/blog/developing-copilot-what-ai-engineers-can-learn-from-our-experience-building-an-ai-assistant/) : Lessons Learned with Arize Copilot ### Get In Touch Questions? Feel free to [join the Arize community](https://join.slack.com/t/arize-ai/shared_invite/zt-26zg4u3lw-OjUNoLvKQ2Yv53EfvxW6Kg) . Share * [![](https://arize.com/wp-content/themes/arize-2022/images/icon-tw-circle.svg)](https://twitter.com/intent/tweet?url=https%3A%2F%2Farize.com%2Fblog%2Farize-release-notes-aug-8-2024%2F&text=Arize%20Release%20Notes:%20Aug%208,%202024) * [![](https://arize.com/wp-content/themes/arize-2022/images/icon-fb-circle.svg)](https://www.facguide.com/sharer/sharer.php?u=https://arize.com/blog/arize-release-notes-aug-8-2024/&picture=https://arize.com) * [![](https://arize.com/wp-content/themes/arize-2022/images/icon-in-circle.svg)](https://www.linkedin.com/shareArticle?mini=true&url=https%3A%2F%2Farize.com%2Fblog%2Farize-release-notes-aug-8-2024%2F&title=Arize%20Release%20Notes:%20Aug%208,%202024) #### Suggested reading ![](https://arize.com/wp-content/uploads/2025/11/arize-ax-october-updates-2025.jpg) [New In Arize AX: Tags, Data Fabric, Automatic Threshold Ranges for Monitors and More](https://arize.com/blog/new-in-arize-ax-tags-data-fabric-automatic-threshold-ranges-for-monitors-and-more/) ![adb - Arize AI](https://arize.com/wp-content/uploads/2025/09/abd-cover-image-dark-3.jpg) [adb Benchmarks](https://arize.com/blog/adb-benchmarks/) Sign up for our monthly newsletter, The Evaluator. -------------------------------------------------- [Subscribe](https://arize.com/blog/arize-release-notes-aug-8-2024/#blog-subscribe-modal) Subscribe to The Evaluator -------------------------- We’ll send you the latest news, expertise, and product updates from Arize. Your inbox is sacred, so we’ll only curate and send the best stuff. \*We’re committed to your privacy. Arize uses the information you provide to contact you about relevant content, products, and services. You may unsubscribe from these communications at any time. For more information, check out our [privacy policy](https://arize.com/privacy-policy) . --- # Arize AI Brings LLM Evaluation, Observability To Microsoft Azure AI Model Catalog - Arize AI ![](https://arize.com/wp-content/uploads/2024/05/azure-ai-model-catalog-arize-ai-integration-collaboration.png) Arize AI Brings LLM Evaluation, Observability To Microsoft Azure AI Model Catalog ================================================================================= Published May 21, 2024 ---------------------- ![](https://arize.com/wp-content/uploads/2024/05/azure-ai-model-catalog-arize-ai-integration-collaboration.png) Generative AI is reshaping the modern enterprise. According to a recent survey, over half (61%) of developers say they plan to deploy LLM applications into production in the next 12 months or “as soon as possible.” However, challenges remain in getting a generative application from toy to production – and staying there. This week at Microsoft Build, Arize AI is rolling out a deepened partnership and integration with Microsoft Azure to help AI engineers speed the reliable deployment of LLM applications. Microsoft Azure Model-as-a-Service ---------------------------------- Many Fortune 500 airlines, financial services firms, retailers, technology companies, and others rely on Azure along with Arize for robust ML and LLM observability. For these users, Azure AI model catalog provides a great starting point. Featuring popular open source models curated by Azure AI – including from Azure Open AI Service, Meta, Mistral, Cohere, and others – the catalog leverages a partnership with Hugging Face to offer thousands of OSS models for inference. Critically, Azure AI model catalog offers several pay-as-you-go inference APIs through Models- as-a-Service (MaaS). With Azure managing the infrastructure and GPU and users accessing a curated set of models – including Llama 3, Mistral Large, Cohere Command R/R+ and Embed models – via a pay-as-you-go API with billing based on tokens for LLMs, users can do hosted fine-tuning without provisioning GPUs and integrate seamlessly with LLMOps tools like prompt flow, LlamaIndex, and Arize – ultimately getting to production sooner. With MaaS available on these GenAI development platforms, now developers can continue using their preferred tools to build GenAI apps while leveraging enterprise grade pay-as-you-go APIs. These APIs are subject to [Azure’s data, privacy, and security commitments](https://learn.microsoft.com/en-us/azure/ai-studio/how-to/concept-data-privacy) , ensuring Microsoft does not share your data with third parties without your permission. Your data, including the data generated through your organization’s use of Models as a Service on Azure – such as prompts and responses – are kept private and are not disclosed to third parties. Azure Model As a Service Integration with Arize ----------------------------------------------- Arize has integrated the Azure AI Model as a Service into the Arize LLM evaluation and observability platform. By offering access to a curated set of easy-to-use hosted models like Llama 3 and Mistral Large as a service, Azure is making it faster to build generative AI applications than ever. Through a seamless integration with Arize AI — which offers a leading platform for LLM evaluation, tracing, and observability — AI engineers and developers using [Azure AI Studio](https://ai.azure.com/) can ensure enterprise grade deployments of sophisticated LLM systems. ![azure model as a service in arize observability platform](https://arize.com/wp-content/uploads/2024/05/azure-model-as-a-service-in-arize.png) The Need for Better LLM Evaluation and Observability ---------------------------------------------------- Given the wealth of models available and variation in performance, the need for objective [evaluation for specialized tasks](https://arize.com/blog-course/llm-evaluation-the-definitive-guide/) and use cases as well as observability to detect and troubleshoot issues once an LLM app is in production becomes paramount. This is often easier said than done given how complex it is to select the right model and get applications working with use cases like retrieval augmented generation (RAG), co-pilots, or agents. ![llm app development flow](https://arize.com/wp-content/uploads/2024/05/large-language-model-application-pain-points.png) Pain points in an LLM-based application RAG deployments include a complex set of technology that is required to work together in order to connect private customer data to LLM generative models. The problems that occur as this technology is deployed into the real world can be hard for teams to troubleshoot. These include Incorrect retrieval, hallucinations on private data, incomplete context chunks causing incomplete answers, and questions with no context data to answer. Complex AI systems like these require world-class technology solutions for observability and analysis. Paired with the high visibility of these initiatives in executive suites, there is little room for error or mistakes that snowball into PR issues. In short, the need for LLM observability is an imperative to any team deploying LLMs in their applications or services. ![llm tracing in arize ai](https://arize.com/wp-content/uploads/2024/05/arize-ai-llm-tracing-observability.png) Arize AI observability platform Arize AI Provides LLM Evaluation and Observability Across the Emerging Stack ---------------------------------------------------------------------------- Arize AI is deepening its collaboration with Azure AI Studio to help teams objectively evaluate LLM apps, working backwards from the output to pinpoint where exactly an issue is stemming from across an LLM stack. ![phoenix arize lifecycle ](https://arize.com/wp-content/uploads/2024/05/phoenix-arize-lifecycle.png) For LLM evaluation, Phoenix – [an open source library from Arize](https://docs.arize.com/phoenix) – offers simple, fast, and accurate LLM-based evaluations for a variety of tasks including code generation, context relevance, hallucination, Q&A correctness, summarization, and toxicity. All evaluation templates are tested against golden datasets that are available as part of the LLM eval library’s benchmarked datasets and target precision at 70-90% and F1 at 70-85%. ![](https://arize.com/wp-content/uploads/2024/05/arize-observability.png) AI observability evaluation in Arize AI Arize’s platform also offers tools for collecting evaluation data, troubleshooting search and retrieval, and tracing to see where an LLM app chain fails. Arize AI: LLM Observability for Azure ------------------------------------- Arize AI’s observability platform features native support for Azure customers leveraging Azure AI Studio, offering complete visibility into every layer of an LLM-based software system: the application, the prompt, and the response. Through the lightweight integration, teams can easily enable AI observability for any LLM or ML-powered application built on top of Azure’s stack. Arize AI’s joint offering with Azure covers the [five pillars of LLM observability](https://arize.com/blog-course/large-language-model-monitoring-observability/#five-pillars) : * **LLM Evaluation**: the Phoenix open source framework allows robust, fast LLM-as-a-judge evaluations underpinning the AI technology of Azure. * **Retrieval Augmented Generation (RAG)**: Arize’s troubleshooting solutions combined with Azure AI Studio create a powerhouse for using private data to create more robust LLM applications powered by your own data. * **LLM Traces and Spans**: with agent tracing and spans, teams can understand what calls failed, or where in the span issues occurred. * **Prompt Engineering**: Azure AI model catalog is easily integrated with Arize’s Prompt Playground, enabling teams to improve prompt templates and iterate in real-time, verifying results. **Fine-Tuning**: curate golden datasets using Arize tools, with integrations back to Azure AI Studio for fine tuning. ![](https://arize.com/wp-content/uploads/2024/05/pillars-llm-observability.png) Fundamental pillars of LLM observability LLM Traces and Spans -------------------- In a deployed chatbot application, every conversation thread creates a large set of traces and spans that include calls to LLMs and vector retrieval systems. These are critical to debugging any LLM application. The simplified example below shows calls to retrievers, embeddings, tools, chains, and more. Each span does a specific job, and troubleshooting each span type requires specific types of evaluations. ![](https://arize.com/wp-content/uploads/2024/05/traces-viz.png) Chatbot LLM framework tracing The traces are first collected using instrumentation. The instrumentation options include auto-instrumentation for frameworks like LlamaIndex,and DSPy. In the case of manual instrumentation, OTEL instrumentation is supported using standard interfaces along with PythonSDK dataframe based instrumentation. ![](https://arize.com/wp-content/uploads/2024/05/ways-send-data-arize.png) Arize instrumentation options Each span powering a chatbot interaction can be visualized and analyzed with Phoenix OSS and persisted to Azure when you are in the development phase. As the application approaches production, you can seamlessly graduate to Arize’s SaaS platform (or deploy on VPC) for always-on monitoring and troubleshooting to automatically monitor for problematic spans. ![](https://arize.com/wp-content/uploads/2024/05/poor-retrieval-troubleshooting.png) RAG evaluations with Q&A correctness and hallucination Evaluations: LLM As a Judge --------------------------- The foremost way in which teams evaluate LLMs at scale is by using an approach called LLM as a judge. The Phoenix open-source evaluation library developed by the Arize team is designed to run LLM as a judge at scale, in parallel, across lots of data – providing high quality, pre-tested templates for analysis. Azure customers can set any model from Azure AI model catalog as the evaluation LLM. ![](https://arize.com/wp-content/uploads/2024/05/llm-as-judge-visual-azure.png) How LLM as a judge works The eval library comes enabled with pre-set eval templates for specific tasks such as RAG relevance, Q&A, and hallucination detection: ![](https://arize.com/wp-content/uploads/2024/05/pre-tested-evals.png) Pretested eval performance The Arize LLM library is designed to be fast, using parallel calls with evals run over large datasets utilizing models from Azure AI model catalog or Azure OpenAI Service. The library supports explanations and uses async calls with concurrency. The following features are supported across Azure AI Studio: * Fast, customizable parallelization of LLM calls * Explanations for all eval results * Bring your own eval with custom eval support * Pre-tested evals for toxicity, human vs AI, relevance, Q&A, citation link checks, and retrieval relevance * Context window overflow RAG: Retrieval Troubleshooting ------------------------------ RAG is the key component in connecting companies’ private data to LLMs to create AI applications. Making RAG work consistently and at scale is critical to building LLM-powered applications that are differentiated for your products and services. Arize helps you identify several critical issues can emerge during the retrieval process, namely: * Lack of content similar to the query in question * Lack of relevant content in your vector database * LLM hallucinates in the response ![](https://arize.com/wp-content/uploads/2024/05/search-retrieval-problems.png) Common problems with search and retrieval systems The Arize platform helps detect clusters of problems, determine why those clusters have retrieval problems, and sort those clusters based on evaluation metrics. ![](https://arize.com/wp-content/uploads/2024/05/retrieval-troubleshooting-workflow-umap.png) Retrieval troubleshooting workflows Arize has RAG troubleshooting flows that allow teams to visualize both the embeddings of chunks and the embeddings of queries to your application. These workflows enable teams powerful debugging capabilities to pinpoint problematic groups of queries in RAG systems. The RAG troubleshooting workflows include: * Prompt and chunk embedding analysis * Clusters of query performance sorted by evals * Relevance evals, Q&A evals, citation link correctness evals, and AI vs human evals * Eval monitoring and generation * Explanation on retrieval evals The Azure AI Studio and Arize suite of tools allow teams to go from benchmarking to production deployment analysis of retrieval results, narrowing down retrieval problems in minutes. Navigating a New Era of Generative AI with Azure and Arize AI ------------------------------------------------------------- In addition to the push-button data integration, the Arize platform has multiple integration points with Azure. The combination of foundation models offered as Model as a Service via Azure AI model catalog and LLM troubleshooting with Arize provides customers a powerful suite for evaluating models and debugging production deployments. Subscribe to The Evaluator -------------------------- We’ll send you the latest news, expertise, and product updates from Arize. Your inbox is sacred, so we’ll only curate and send the best stuff. \*We’re committed to your privacy. Arize uses the information you provide to contact you about relevant content, products, and services. You may unsubscribe from these communications at any time. For more information, check out our [privacy policy](https://arize.com/privacy-policy) . --- # Arize AI’s Next Era of Growth - Arize AI ![](https://arize.com/wp-content/uploads/2021/03/jasonlopatecki-196x196.jpeg "jasonlopatecki") [Jason Lopatecki](https://arize.com/author/jason-lopatecki/) Co-founder and CEO Share * [![](https://arize.com/wp-content/themes/arize-2022/images/icon-tw-circle.svg)](https://twitter.com/intent/tweet?url=https%3A%2F%2Farize.com%2Fblog%2Farize-ais-next-era-of-growth%2F&text=Arize%20AI%E2%80%99s%20Next%20Era%20of%20Growth) * [![](https://arize.com/wp-content/themes/arize-2022/images/icon-fb-circle.svg)](https://www.facguide.com/sharer/sharer.php?u=https://arize.com/blog/arize-ais-next-era-of-growth/&picture=https://arize.com) * [![](https://arize.com/wp-content/themes/arize-2022/images/icon-in-circle.svg)](https://www.linkedin.com/shareArticle?mini=true&url=https%3A%2F%2Farize.com%2Fblog%2Farize-ais-next-era-of-growth%2F&title=Arize%20AI%E2%80%99s%20Next%20Era%20of%20Growth) ![](https://arize.com/wp-content/themes/arize-2022/images/icon-newsletter.svg) Subscribe to the Arize blog [Get the latest](https://arize.com/blog/arize-ais-next-era-of-growth/#blog-subscribe-modal) #### On this page #### Suggested reading ![](https://arize.com/wp-content/uploads/2025/10/iso-iec-27001-certified-icon.jpg) [Arize AI Achieves ISO/IEC 27001 Certification](https://arize.com/blog/arize-ai-achieves-iso-iec-27001-certification/) ![](https://arize.com/wp-content/uploads/2025/03/NVIDIA-Arize-blog.jpg) [Self-Improving Agents: Automating LLM Performance Optimization using Arize and NVIDIA NeMo](https://arize.com/blog/arize-nvidia-nemo-integration/) ![Arize AI Careers](https://arize.com/wp-content/uploads/2022/09/arize-ai-collage-2142x1120.jpg "Arize AI Team") Arize AI’s Next Era of Growth ============================= Published Sep 7, 2022 * [Company](https://arize.com/blog/?cat=company) ![](https://arize.com/wp-content/uploads/2021/03/jasonlopatecki-196x196.jpeg "jasonlopatecki") #### [Jason Lopatecki](https://arize.com/author/jason-lopatecki/) ##### Co-founder and CEO Today, we announced a $38 million Series B led by TCV, with participation from existing investors Battery Ventures, Foundation Capital, and Swift Ventures. The investment is the industry’s largest-ever in a machine learning observability platform. Our Role In a Changing World ---------------------------- When Aparna and I started Arize, we were on a mission to make AI work and work for the people. As ML practitioners, we experienced firsthand the heartache of spending months building and training machine learning models, deploying them to production, and having no insight into how the models actually performed. Were the models helping or hurting our products? Or were the predictions unintentionally perpetuating bias or unfair outcomes for certain groups? We set out to answer these and other fundamental questions. While our raison d’etre has not changed since the company’s founding in 2020, the world has become a much different place. AI is now being deployed across industries, informing everything from holiday apparel orders for retailers to potentially life-saving interventions in cancer care. In all, spending on AI is forecast to [eclipse](https://www.wsj.com/articles/retail-set-to-overtake-banking-in-ai-spending-11631007001) $200 billion within three years. The pace of technical progress in areas like deep learning is also [accelerating](https://arize.com/blog/four-takeaways-from-arizeobserve-unstructured/) . As models get more complex and prolific, detecting and troubleshooting problems gets harder. Unfortunately, [most companies](https://arize.com/resource/survey-machine-learning-observability-results/) still lack visibility into how their ML models are performing in production and run the risk of [models impacting earnings](https://arize.com/blog/when-ai-attacks-earnings/) or acting in unfair ways. Nearly one in ten Fortune 500 companies even [disclose AI risk](https://arize.com/resource/rise-of-ai-risk-disclosure/) on their annual financial and ESG reports. A revolution in machine learning observability is needed for AI to flourish and achieve long term sustainability. That’s why Arize pioneered this space two years ago and is now proud to count Uber, Chime, Ebay, New York Life, ShareChat, Spotify, and Stitch Fix as customers and design partners. Arize’s ML observability platform offers ML teams an easy way to streamline performance monitoring, drift detection, data quality checks, and model validation. A slew of recent innovations supplement these core capabilities. Most recently, we debuted [embedding analysis and embedding drift monitoring](https://arize.com/blog/monitor-unstructured-data-with-arize/) – a much-needed breakthrough given computer vision and natural language processing models are notoriously difficult to troubleshoot and data labeling for retraining is costly. We also released [bias tracing](https://arize.com/blog/machine-learning-bias-tracing/) , a tool designed to help monitor and take action on model fairness metrics, earlier this year. Most importantly, we democratized access to these tools, offering a [free Arize plan](https://app.arize.com/auth/join) that makes it easy for any ML engineer anywhere in the world to get up and running in minutes. Welcoming New Partners ---------------------- It is an incredible privilege to lead Arize along with my co-founder Aparna Dhinakaran. We are both incredibly proud and grateful to our early team, customers, partners, and investors. In taking Arize to the next level, we could not ask for a better partner than TCV. The firm’s track record working with growth-stage technology companies – including many household names and B2B giants – is well-known. Partner Morgan Gerlak brings a great strategic lens to our board of directors. We are also excited to expand hiring and welcome new faces to our team. With [roles](https://arize.com/careers/) now open across engineering, product, marketing and sales, we hope you will consider joining Arize. You will find an open and collaborative team that was recently [recognized by _Fast Company_](https://arize.com/press/arize-ai-named-to-fast-companys-list-of-the-best-workplaces-for-innovators/) as among the “Best Workplaces for Innovators” and a diversity category standout. Share * [![](https://arize.com/wp-content/themes/arize-2022/images/icon-tw-circle.svg)](https://twitter.com/intent/tweet?url=https%3A%2F%2Farize.com%2Fblog%2Farize-ais-next-era-of-growth%2F&text=Arize%20AI%E2%80%99s%20Next%20Era%20of%20Growth) * [![](https://arize.com/wp-content/themes/arize-2022/images/icon-fb-circle.svg)](https://www.facguide.com/sharer/sharer.php?u=https://arize.com/blog/arize-ais-next-era-of-growth/&picture=https://arize.com) * [![](https://arize.com/wp-content/themes/arize-2022/images/icon-in-circle.svg)](https://www.linkedin.com/shareArticle?mini=true&url=https%3A%2F%2Farize.com%2Fblog%2Farize-ais-next-era-of-growth%2F&title=Arize%20AI%E2%80%99s%20Next%20Era%20of%20Growth) #### Suggested reading ![](https://arize.com/wp-content/uploads/2025/10/iso-iec-27001-certified-icon.jpg) [Arize AI Achieves ISO/IEC 27001 Certification](https://arize.com/blog/arize-ai-achieves-iso-iec-27001-certification/) ![](https://arize.com/wp-content/uploads/2025/03/NVIDIA-Arize-blog.jpg) [Self-Improving Agents: Automating LLM Performance Optimization using Arize and NVIDIA NeMo](https://arize.com/blog/arize-nvidia-nemo-integration/) Recommended resources --------------------- [![arize certified for health information security hipaa compliance](https://arize.com/wp-content/uploads/2022/08/arize-hipaa-compliant.jpg)\ \ post\ \ #### Arize Receives Certifications Validating Health Information Security for HIPAA Compliance\ \ Read more](https://arize.com/blog/arize-receives-certifications-validating-health-information-security-for-hipaa-compliance/) [![When AI Attacks Earnings](https://arize.com/wp-content/uploads/2022/08/When-AI-Attacks-Earnings.png)\ \ post\ \ #### When AI Attacks Earnings\ \ Read more](https://arize.com/blog/when-ai-attacks-earnings/) [![](https://arize.com/wp-content/uploads/2022/04/cover-rise-of-ai-risk-disclosure.jpg)\ \ resource\ \ #### The Rise of AI Risk Disclosure\ \ Read more](https://arize.com/resource/rise-of-ai-risk-disclosure/) Sign up for our monthly newsletter, The Evaluator. -------------------------------------------------- [Subscribe](https://arize.com/blog/arize-ais-next-era-of-growth/#blog-subscribe-modal) Subscribe to The Evaluator -------------------------- We’ll send you the latest news, expertise, and product updates from Arize. Your inbox is sacred, so we’ll only curate and send the best stuff. \*We’re committed to your privacy. Arize uses the information you provide to contact you about relevant content, products, and services. You may unsubscribe from these communications at any time. For more information, check out our [privacy policy](https://arize.com/privacy-policy) . --- # Arize, Vertex AI API: Evaluation Workflows to Accelerate Generative App Development and AI ROI - Arize AI ![](https://arize.com/wp-content/uploads/2024/11/GCP-Arize-AI-Blog-Post-1.png) Arize, Vertex AI API: Evaluation Workflows to Accelerate Generative App Development and AI ROI ============================================================================================== Published November 1, 2024 -------------------------- ![](https://arize.com/wp-content/uploads/2024/11/GCP-Arize-AI-Blog-Post-1.png) _Written in collaboration with Christian Williams, Principal Architect AI/ML, Google Cloud._  In the rapidly evolving landscape of artificial intelligence, enterprise AI engineering teams must constantly seek cutting-edge solutions to drive innovation, enhance productivity, and maintain a competitive edge. In leveraging an AI observability and evaluation platform like Arize AI with the advanced capabilities of Google’s suite of AI tools, enterprises looking to push the boundaries of what’s possible with their AI applications have a robust, compelling option. As a state-of-the-art large language model (LLM) with multi-modal capabilities, Vertex AI API serving Gemini 1.5 Pro offers enterprise teams a powerful model that can be integrated across a wide range of applications and use cases. From improving customer interactions and automating complex processes to enhancing data analysis and decision-making, the potential to transform business operations is significant. By adopting Vertex AI API for Gemini, enterprise AI teams can: * **Accelerate development:** Leverage advanced natural language processing and generation capabilities to streamline code development, debugging, and documentation processes. * **Enhance customer experiences:** Implement sophisticated chatbots and virtual assistants capable of understanding and responding to customer queries across multiple modalities. * **Boost data analysis:** Utilize the ability to process and interpret various data types, including text, images, and audio, for more comprehensive and insightful data analysis. * **Improve decision-making:** Harness advanced reasoning capabilities to provide data-driven insights and recommendations to support strategic decision-making. Teams using the Vertex AI API further gain from implementing a telemetry system, or AI observability and LLM evaluation, as they’re developing generative applications to validate performance and accelerate the iteration cycle. By adopting Arize AI in tandem with their Google AI tools, AI teams can: * **Help ensure application reliability:** Continuously validate and monitor generative app performance as input data shifts and new use cases arise, to quickly address issues in development and after deployment. * **Speed development cycles:** Rapidly iterate using pre-production app evaluations and workflows to test and compare the results of various prompt iterations. * **Implement guardrails for protection:** Systematically test app responses to a wide range of inputs and edge cases to ensure outputs comply in the boundaries of expectations. * **Make improvements with dynamic data:** Automatically flag low-performing sessions for review and identify challenging or ambiguous examples for further analysis and fine-tuning. * **Consistent evaluation from development to deployment:** Use Arize’s open-source evaluation solution during development, alongside an enterprise-ready platform as applications become ready for production. Solutions to Common Challenges Afflicting AI Engineering Teams -------------------------------------------------------------- In working with hundreds of AI engineering teams building and deploying generative-powered applications, a common set of challenges emerged: * **Small changes can lead to performance regressions** – even minor alterations in prompts or underlying data can result in expected degradations. It’s difficult to anticipate or track down these regressions. * **Hard to discover data for testing and improvement** – identifying edge cases, underrepresented scenarios or high-impact failure modes requires complex data mining techniques to extract meaningful subsets of data. * **Bad LLM responses can lead to outsized business impact** – a single factually incorrect or inappropriate response can result in legal issues, loss of trust, or financial liabilities. Arize’s AI observability and evaluation platform enables engineering teams to tackle these challenges head on, building a foundation during the app development phase to carry through to online production observability. Let’s delve deeper into the specific applications and integration strategies for Arize and Vertex AI, and how an enterprise AI engineering team can build better AI using the two solutions together. ![Pillars of LLM observability](https://arize.com/wp-content/uploads/2024/11/1_Pillars_of_LLM_Observability.max-1000x1000-1.png) Gain Visibility with LLM Tracing in Development ----------------------------------------------- Arize’s LLM tracing capabilities provides visibility into each call in an LLM-powered system to facilitate application development and troubleshooting. This is especially critical for systems that implement an orchestration or agentic framework, as those abstractions can mask an immense number of distributed system calls that are nearly impossible to debug without programmatic tracing. With LLM tracing, teams gain a comprehensive understanding of how the Vertex AI API serving Gemini 1.5 Pro processes input data through each layer of the application: query, retriever, embedding, LLM call, synthesis, etc. Traces are available from the session-level down to specific span — e.g., retrieval of an individual document — which let AI engineers pinpoint the exact source of an issue and how it might propagate through the system’s layers. ![LLM tracing with document retrieval](https://arize.com/wp-content/uploads/2024/11/2_LLM_tracing_with_document_retrieval.max-2000x2000-1.png) LLM tracing with document retrieval. LLM tracing also surfaces fundamental telemetry data such as token usage and latency in system steps and Vertex AI API calls. This allows for identification of bottlenecks and inefficiencies for further application performance optimization. Instrumenting Arize tracing on apps takes just a few lines of code — traces are collected automatically from over a dozen frameworks such as OpenAI, DSPy, LlamaIndex, and LangChain, or manually set up using the OpenTelemetry Trace API. Replay and Fix Issues with Vertex AI in Prompt + Data Playground ---------------------------------------------------------------- Replaying problems and prompt engineering with your application data is an incredibly effective way to improve the outputs of LLM-powered applications. The prompt + data playground in Arize offers an interactive environment for optimizing prompts used with Vertex AI API for Gemini, providing developers real-time feedback into the results using app development data. Use it to import trace data and explore the impact of changes to prompt templates, input variables, and model parameters. Workflows in Arize allow developers to take a prompt from an app trace of interest and replay scenarios directly in the platform. This is a convenient method to rapidly iterate and test different prompt configurations as new use cases are being deployed, or encountered by Vertex AI API serving Gemini 1.5 Pro once apps are live. ![Prompt data playground using Vertex AI](https://arize.com/wp-content/uploads/2024/11/3_Prompt__Data_Playground_Using_Vertex_AI_.max-2000x2000-1.png) Prompt + Data Playground Using Vertex AI API serving Gemini 1.5 Pro Validate Performance with Online LLM Evaluation ----------------------------------------------- Once tracing is implemented, Arize helps developers validate performance with a systematic approach to LLM evaluation. The Arize evaluation library is a set of pre-tested evaluation frameworks to score the quality of LLM outputs on specific tasks such as: hallucination, relevance, Q&A on retrieved data, code generation, user frustration, summarization, among many others. Google customers can use Vertex AI API serving Gemini models to automate and scale evaluation tasks, in a process called Online LLM as a judge. With Online LLM as a judge, developers define the evaluation criteria in a prompt template in Arize and designate Vertex AI API serving Gemini as the evaluator in the platform. As the LLM app runs, the model scores, or evaluates, the outputs generated by the system based on the criteria defined. ![LLM as a Jodge](https://arize.com/wp-content/uploads/2024/11/4_Online_LLM_Evaluation_Method_Using_Verte.max-2000x2000-1.png) Online LLM Evaluation Method Using Vertex AI API serving Gemini 1.5 Pro as Evaluator LLM. Furthermore, Vertex AI API serving Gemini can be used to explain the evaluations generated. In many cases it can be difficult to understand why an LLM responds in a specific way — explanations expose the rationale and can help further improve the accuracy of evaluations downstream. ![](https://arize.com/wp-content/uploads/2024/11/5_LLM_evaluation_with_explanations_generat.max-2000x2000-1.png) LLM evaluation in Arize with explanations generated by Vertex AI API serving Gemini 1.5 Pro. Teams greatly gain from employing evaluations while they are actively developing their AI applications, as this serves as an early benchmark for performance to base subsequent iterations and fine-tuning. Curate Dynamic Datasets for Experimentation ------------------------------------------- The ability to curate dynamic datasets in Arize arms developers with a workflow to capture examples of interest — whether high-quality evaluations or edge cases where the LLM struggles to perform — to run experiments and track improvements to their prompts, LLM, or other parts of their application. Paired with Vertex AI Vector Search, developers can bring together offline and online data streams in a curation process that leverages AI to find similar data points to the ones of interest, curating the examples into a dataset that continues to evolve as the application runs. Developers can automate online tasks in Arize that identify examples of interest as traces are collected to continuously validate performance. Examples can be further augmented manually or with Vertex AI API for Gemini driven labeling and annotation. ![Screenshot Curate dynamic datasets in Arize for experimentation](https://arize.com/wp-content/uploads/2024/11/6_Curate_dynamic_datasets_in_Arize_for_exp.max-2000x2000-1.png) Curate dynamic datasets in Arize for experimentation. Once a dataset is created, it can be leveraged for experimentation, offering developers workflows to conduct A/B testing against prompt template changes, prompt variable changes, or even validate newly tuned versions of Vertex AI API serving Gemini against specific use cases. This systematic experimentation is vital for identifying the optimal configuration to balance model performance and efficiency, particularly in production environments where response times are critical. Safeguard Your Business with Arize and Vertex AI API Serving Gemini ------------------------------------------------------------------- Together, Arize and Google AI can help safeguard your AI from undesirable outcomes for your customers and business. LLM guardrails are essential for real-time safety against malicious attempts like jailbreaks, context management, compliance, and overall user experience. Arize guardrails can be configured using custom datasets and a fine-tuned Vertex AI Gemini model for the following detections: * **Embeddings guards:** Uses your examples of “bad” messages to guard against similar inputs based on analysis of cosine distance between embeddings. The benefit of this approach is the continuous iteration on breaks so the guard gets more advanced over time. * **Few-shot LLM prompt**: With your few-shot examples, the model classifies the input as “pass” or “fail”. This is especially advantageous when you want to define a completely customized guardrail. * **LLM evaluations:** Uses Vertex AI API serving Gemini to evaluate for PII data, user frustration, hallucination, etc. as trigger. This approach leverages scaled LLM evaluations as its foundation. ![](https://arize.com/wp-content/uploads/2024/11/7_Arize_Guardrails_Using_Vertex_AI_API_ser.max-1100x1100-1.png) Arize guardrails using Vertex API serving Gemini 1.5 Pro for detection.If these detections are flagged in Arize, an immediate corrective action will kick in to protect your application from outputting an undesired response. Developers can specify the remediation to block, retry, or default an answer such as “I cannot answer your query”. Your own Arize AI Copilot Powered by Vertex AI API Serving Gemini 1.5 Pro ------------------------------------------------------------------------- To further streamline the AI observability and evaluation process, developers can leverage Arize AI Copilot, powered by Vertex AI API serving Gemini. This in-platform assistant streamlines the workflows for AI teams, automating tasks and analysis to lighten the daily operational effort for team members. With Arize Copilot, engineers can: * **Initiate AI Search with Vertex AI API serving Gemini** – find specific examples such as “angry responses” or “frustrated user inquiries” to add to a dataset. * **Perform quick actions and analysis** – configure monitors for dashboards or ask questions about your models and data. * **Automate a task** – define and build LLM evaluations. * **Prompt engineering** – ask Vertex AI API serving Gemini to generate prompt playground iterations for you. ![](https://arize.com/wp-content/uploads/2024/11/8_Arize_Copilot_utilizing_the_Vertex_AI_AP.max-2000x2000-1.png) Arize Copilot utilizing the Vertex AI API serving Gemini 1.5 Pro. Accelerating AI Innovation with Arize and Vertex AI --------------------------------------------------- As enterprises push the boundaries of AI, the integration of Arize AI with Vertex AI API serving Gemini offers a compelling solution for optimizing and safeguarding generative applications. By leveraging Arize’s observability and evaluation platform and Google’s advanced LLM capabilities, AI teams can streamline development, enhance application performance, and help ensure reliability from development to deployment. Whether it’s through dynamic dataset curation, real-time guardrails, or the automated workflows of Arize AI Copilot, these tools work in harmony to accelerate innovation and drive meaningful business outcomes. As you continue to develop and scale AI applications, Arize and Vertex AI API serving Gemini models provide the critical infrastructure to navigate the complexities of modern AI engineering, so your projects remain efficient, resilient, and impactful. To get started with tracing and evaluation for your Google-powered AI applications, visit [Arize’s product documentation here](https://docs.arize.com/arize/llm-tracing/tracing-integrations-auto/vertex-ai-gemini) . Or [try out this Colab](https://colab.research.google.com/github/Arize-ai/phoenix/blob/main/tutorials/tracing/langchain_vertex_ai_tracing_tutorial.ipynb) to visualize how the tools work together to help accelerate your AI development journey. Want to simplify your AI observability even further? You can find [Arize on Google Cloud Marketplace](https://console.cloud.google.com/marketplace/product/arize/arize-ai?pli=1) ! This integration makes it easier than ever to deploy Arize and monitor the performance of your production models. Visit the Arize listing on Google Cloud Marketplace today to learn more and get started. Subscribe to The Evaluator -------------------------- We’ll send you the latest news, expertise, and product updates from Arize. Your inbox is sacred, so we’ll only curate and send the best stuff. \*We’re committed to your privacy. Arize uses the information you provide to contact you about relevant content, products, and services. You may unsubscribe from these communications at any time. For more information, check out our [privacy policy](https://arize.com/privacy-policy) . --- # Arize Release Notes: Test Tasks, Filter Experiments, and More - Arize AI ![](https://arize.com/wp-content/uploads/2023/01/Sarah_headshot-196x196.jpg "Sarah_headshot") [Sarah Welsh](https://arize.com/author/sarah-welsh/) Contributor Share * [![](https://arize.com/wp-content/themes/arize-2022/images/icon-tw-circle.svg)](https://twitter.com/intent/tweet?url=https%3A%2F%2Farize.com%2Fblog%2Farize-release-notes-test-tasks-filter-experiments-and-more%2F&text=Arize%20Release%20Notes:%20Test%20Tasks,%20Filter%20Experiments,%20and%20More) * [![](https://arize.com/wp-content/themes/arize-2022/images/icon-fb-circle.svg)](https://www.facguide.com/sharer/sharer.php?u=https://arize.com/blog/arize-release-notes-test-tasks-filter-experiments-and-more/&picture=https://arize.com) * [![](https://arize.com/wp-content/themes/arize-2022/images/icon-in-circle.svg)](https://www.linkedin.com/shareArticle?mini=true&url=https%3A%2F%2Farize.com%2Fblog%2Farize-release-notes-test-tasks-filter-experiments-and-more%2F&title=Arize%20Release%20Notes:%20Test%20Tasks,%20Filter%20Experiments,%20and%20More) ![](https://arize.com/wp-content/themes/arize-2022/images/icon-newsletter.svg) Subscribe to the Arize blog [Get the latest](https://arize.com/blog/arize-release-notes-test-tasks-filter-experiments-and-more/#blog-subscribe-modal) #### On this page #### Suggested reading ![](https://arize.com/wp-content/uploads/2025/11/arize-ax-october-updates-2025.jpg) [New In Arize AX: Tags, Data Fabric, Automatic Threshold Ranges for Monitors and More](https://arize.com/blog/new-in-arize-ax-tags-data-fabric-automatic-threshold-ranges-for-monitors-and-more/) ![adb - Arize AI](https://arize.com/wp-content/uploads/2025/09/abd-cover-image-dark-3.jpg) [adb Benchmarks](https://arize.com/blog/adb-benchmarks/) ![](https://arize.com/wp-content/uploads/2024/10/Release-notes-10-24.jpg "Release notes-10-24") Arize Release Notes: Test Tasks, Filter Experiments, and More ============================================================= Published Oct 24, 2024 * [Product Releases](https://arize.com/blog/?cat=product-releases) * [Release Notes](https://arize.com/blog/?cat=release-notes) ![](https://arize.com/wp-content/uploads/2023/01/Sarah_headshot-196x196.jpg "Sarah_headshot") #### [Sarah Welsh](https://arize.com/author/sarah-welsh/) ##### Contributor Welcome to our regular update on new releases, enhancements, and changes. What’s New ---------- ### Run Task Once Users now have the option to to test a task, such as online eval, by running it once on existing data, or apply evaluation labels to older traces. To use the feature, simply select “Run once on historical data” in the new Schedule Run component of Tasks, and choose the time range for the data you wish to run evaluations on. In the Task Details pane, you can view logs and check if a task is set to run continuously or just once. [Learn more](https://docs.arize.com/arize/llm-evaluation-and-annotations/catching-hallucinations/tasks-for-online-evals) ![Arize interface screenshot run task once](https://arize.com/wp-content/uploads/2024/10/Run-Task-Once.png) ### Experiment Filters Users can now filter experiments based on dataset attributes or experiment results, making it easy to identify areas for improvement and track their experiment progress with more precision. [Learn more](https://docs.arize.com/arize/llm-experiments-and-testing/how-to-experiments/filter-experiments) ![Experiment filters](https://arize.com/wp-content/uploads/2024/10/experiment_filters.gif) Filtering experiments by experiment name 📚 New Content -------------- The latest video tutorials, paper readings, ebooks, self-guided learning modules, and technical posts: ✏️ [Tracing LLM Function Calls](https://arize.com/resource/tracing-llm-function-calls-in-arize/) 🤖 [Intro to LangGraph](https://arize.com/blog/langgraph/) 📘 [Exploring Google’s NotebookLM](https://arize.com/blog/exploring-google-notebook-lm/) 📡 [OpenTelemetry and LLM Observability](https://arize.com/blog/the-role-of-opentelemetry-in-llm-observability/) 👓 [Object Detection Modeling](https://arize.com/resource/object-detection-modeling-arize/) 🛠️ [How to Build Better AI](https://www.youtube.com/watch?v=Q9tw9gCPOag) Share * [![](https://arize.com/wp-content/themes/arize-2022/images/icon-tw-circle.svg)](https://twitter.com/intent/tweet?url=https%3A%2F%2Farize.com%2Fblog%2Farize-release-notes-test-tasks-filter-experiments-and-more%2F&text=Arize%20Release%20Notes:%20Test%20Tasks,%20Filter%20Experiments,%20and%20More) * [![](https://arize.com/wp-content/themes/arize-2022/images/icon-fb-circle.svg)](https://www.facguide.com/sharer/sharer.php?u=https://arize.com/blog/arize-release-notes-test-tasks-filter-experiments-and-more/&picture=https://arize.com) * [![](https://arize.com/wp-content/themes/arize-2022/images/icon-in-circle.svg)](https://www.linkedin.com/shareArticle?mini=true&url=https%3A%2F%2Farize.com%2Fblog%2Farize-release-notes-test-tasks-filter-experiments-and-more%2F&title=Arize%20Release%20Notes:%20Test%20Tasks,%20Filter%20Experiments,%20and%20More) #### Suggested reading ![](https://arize.com/wp-content/uploads/2025/11/arize-ax-october-updates-2025.jpg) [New In Arize AX: Tags, Data Fabric, Automatic Threshold Ranges for Monitors and More](https://arize.com/blog/new-in-arize-ax-tags-data-fabric-automatic-threshold-ranges-for-monitors-and-more/) ![adb - Arize AI](https://arize.com/wp-content/uploads/2025/09/abd-cover-image-dark-3.jpg) [adb Benchmarks](https://arize.com/blog/adb-benchmarks/) Sign up for our monthly newsletter, The Evaluator. -------------------------------------------------- [Subscribe](https://arize.com/blog/arize-release-notes-test-tasks-filter-experiments-and-more/#blog-subscribe-modal) Subscribe to The Evaluator -------------------------- We’ll send you the latest news, expertise, and product updates from Arize. Your inbox is sacred, so we’ll only curate and send the best stuff. \*We’re committed to your privacy. Arize uses the information you provide to contact you about relevant content, products, and services. You may unsubscribe from these communications at any time. For more information, check out our [privacy policy](https://arize.com/privacy-policy) . --- # Arize AI Accelerates Enterprise AI Adoption On-Premises With NVIDIA - Arize AI ![](https://arize.com/wp-content/uploads/2025/05/Arize-x-Nvidia.jpg) Arize AI Accelerates Enterprise AI Adoption On-Premises With NVIDIA =================================================================== Published May 18, 2025 ---------------------- ![](https://arize.com/wp-content/uploads/2025/05/Arize-x-Nvidia.jpg) Arize AI, a leader in large language model (LLM) evaluation and AI observability, today announced it is delivering a high-performance, on-premises AI for enterprises seeking to deploy and scale AI applications, including AI agents, in collaboration with NVIDIA. At the heart of this innovation is the Arize AI platform integration with the new NVIDIA Enterprise AI Factory full-stack, validated design to help enterprises build and deploy their own on-premises AI factory. **The Rise of On-Premise AI** As enterprises leverage AI to drive innovation and efficiency, many organizations – particularly those in highly regulated industries – require the control and security of on-premise solutions. This approach offers several advantages, from data security and compliance to flexibility and lower latency from processing data locally for real-time AI applications. This collaboration addresses these critical needs, empowering enterprises from financial institutions to public sector agencies to confidently embrace AI while maintaining control over their data and infrastructure. **Arize AI and NVIDIA Enterprise AI Factory** This collaboration brings together several key components to create a comprehensive solution for enterprise AI: * **Arize AI Platform:** Arize AI provides an AI engineering platform that helps teams test, evaluate and monitor complex AI systems — such as semi-autonomous multi-agent systems, voice assistants, and sophisticated consumer-facing AI applications — to ensure they deliver accurate, reliable results. * **NVIDIA Enterprise AI Factory:** The [NVIDIA Enterprise AI Factory validated design](https://blogs.nvidia.com/blog/enterprise-ai-factory-agents) provides guidance for developing, deploying, and managing agentic AI, physical AI, and HPC workloads on the [NVIDIA Blackwell](https://www.nvidia.com/en-us/data-center/generative-ai-in-practice/?ncid=pa-srch-goog-775790-GenAI-Brand-Broad&_bt=746488506849&_bk=nvidia%20blackwell&_bm=e&_bn=g&_bg=178253341379&gad_source=1&gad_campaignid=22447665005&gbraid=0AAAAAD4XAoFwMpvZkwLpTdyFSxWSUigol&gclid=EAIaIQobChMIq4iq3tqjjQMVzSdECB2mDjEJEAAYASAAEgKfXvD_BwE) platform on-premises, including systems featuring [NVIDIA RTX PRO 6000 Blackwell GPU](https://nvidianews.nvidia.com/news/nvidia-rtx-pro-blackwell-servers-speed-trillion-dollar-enterprise-it-industry-transition-to-ai-factories) s. Designed for enterprise IT, the validated design recommends NVIDIA AI Infrastructure including, networking, storage, and software to help deliver faster time-to-value AI factory deployments while mitigating deployment risks. The integration of Arize AI’s self-hosted deployment option with NVIDIA Blackwell and NVIDIA’s software solutions for Enterprise AI creates a seamless workflow for developing, deploying, and managing AI applications on-premise.  **Integrated Features and Optimized Performance** It empowers enterprises to achieve significant business outcomes faster and bring real-time AI solutions to regulated clients’ data in new ways. “We are incredibly excited to collaborate with NVIDIA to bring this powerful on-premise AI solution to enterprises,” notes Rich Young, Director of Partner Solutions Architecture at Arize. “By combining the Arize AI Platform with NVIDIA Blackwell computing and NVIDIA NeMo microservices, we are empowering organizations to apply cutting-edge generative AI agents and systems in new ways while maintaining strict compliance and control.” Subscribe to The Evaluator -------------------------- We’ll send you the latest news, expertise, and product updates from Arize. Your inbox is sacred, so we’ll only curate and send the best stuff. \*We’re committed to your privacy. Arize uses the information you provide to contact you about relevant content, products, and services. You may unsubscribe from these communications at any time. For more information, check out our [privacy policy](https://arize.com/privacy-policy) . --- # Arize AI Raises $70M Series C to Build the Gold Standard for AI Evaluation & Observability - Arize AI ![](https://arize.com/wp-content/uploads/2021/03/jasonlopatecki-196x196.jpeg "jasonlopatecki") [Jason Lopatecki](https://arize.com/author/jason-lopatecki/) Co-founder and CEO Share * [![](https://arize.com/wp-content/themes/arize-2022/images/icon-tw-circle.svg)](https://twitter.com/intent/tweet?url=https%3A%2F%2Farize.com%2Fblog%2Farize-ai-raises-70m-series-c-to-build-the-gold-standard-for-ai-evaluation-observability%2F&text=Arize%20AI%20Raises%20$70M%20Series%20C%20to%20Build%20the%20Gold%20Standard%20for%20AI%20Evaluation%20&%20Observability) * [![](https://arize.com/wp-content/themes/arize-2022/images/icon-fb-circle.svg)](https://www.facguide.com/sharer/sharer.php?u=https://arize.com/blog/arize-ai-raises-70m-series-c-to-build-the-gold-standard-for-ai-evaluation-observability/&picture=https://arize.com) * [![](https://arize.com/wp-content/themes/arize-2022/images/icon-in-circle.svg)](https://www.linkedin.com/shareArticle?mini=true&url=https%3A%2F%2Farize.com%2Fblog%2Farize-ai-raises-70m-series-c-to-build-the-gold-standard-for-ai-evaluation-observability%2F&title=Arize%20AI%20Raises%20$70M%20Series%20C%20to%20Build%20the%20Gold%20Standard%20for%20AI%20Evaluation%20&%20Observability) ![](https://arize.com/wp-content/uploads/2020/12/bg-speaker-196x196.png "bg-speaker") [Aparna Dhinakaran](https://arize.com/author/aparna-dhinakaran/) Co-founder & Chief Product Officer Share * [![](https://arize.com/wp-content/themes/arize-2022/images/icon-tw-circle.svg)](https://twitter.com/intent/tweet?url=https%3A%2F%2Farize.com%2Fblog%2Farize-ai-raises-70m-series-c-to-build-the-gold-standard-for-ai-evaluation-observability%2F&text=Arize%20AI%20Raises%20$70M%20Series%20C%20to%20Build%20the%20Gold%20Standard%20for%20AI%20Evaluation%20&%20Observability) * [![](https://arize.com/wp-content/themes/arize-2022/images/icon-fb-circle.svg)](https://www.facguide.com/sharer/sharer.php?u=https://arize.com/blog/arize-ai-raises-70m-series-c-to-build-the-gold-standard-for-ai-evaluation-observability/&picture=https://arize.com) * [![](https://arize.com/wp-content/themes/arize-2022/images/icon-in-circle.svg)](https://www.linkedin.com/shareArticle?mini=true&url=https%3A%2F%2Farize.com%2Fblog%2Farize-ai-raises-70m-series-c-to-build-the-gold-standard-for-ai-evaluation-observability%2F&title=Arize%20AI%20Raises%20$70M%20Series%20C%20to%20Build%20the%20Gold%20Standard%20for%20AI%20Evaluation%20&%20Observability) ![](https://arize.com/wp-content/themes/arize-2022/images/icon-newsletter.svg) Subscribe to the Arize blog [Get the latest](https://arize.com/blog/arize-ai-raises-70m-series-c-to-build-the-gold-standard-for-ai-evaluation-observability/#blog-subscribe-modal) #### On this page #### Suggested reading ![](https://arize.com/wp-content/uploads/2025/10/iso-iec-27001-certified-icon.jpg) [Arize AI Achieves ISO/IEC 27001 Certification](https://arize.com/blog/arize-ai-achieves-iso-iec-27001-certification/) ![](https://arize.com/wp-content/uploads/2025/03/NVIDIA-Arize-blog.jpg) [Self-Improving Agents: Automating LLM Performance Optimization using Arize and NVIDIA NeMo](https://arize.com/blog/arize-nvidia-nemo-integration/) ![Text reads: Arize AI raises $70M Series C](https://arize.com/wp-content/uploads/2025/02/Series-C-foudners-blog.jpg "Series C founders blog") Arize AI Raises $70M Series C to Build the Gold Standard for AI Evaluation & Observability ========================================================================================== Published Feb 20, 2025 * [Company](https://arize.com/blog/?cat=company) ![](https://arize.com/wp-content/uploads/2021/03/jasonlopatecki-196x196.jpeg "jasonlopatecki") #### [Jason Lopatecki](https://arize.com/author/jason-lopatecki/) ##### Co-founder and CEO ![](https://arize.com/wp-content/uploads/2020/12/bg-speaker-196x196.png "bg-speaker") #### [Aparna Dhinakaran](https://arize.com/author/aparna-dhinakaran/) ##### Co-founder & Chief Product Officer In 2020, we founded Arize with a clear mission: to give teams the tools they need to understand, troubleshoot, and improve AI performance in the real world. Our initial seed investment deck started with the simple line “We Make AI Work.” Since then, AI has evolved at breakneck speed—expanding beyond traditional machine learning into generative models, multi-agent systems, and autonomous decision-making. But with all this progress comes one of the most momentous challenges that AI builders have faced: How to make Artificial Intelligence really work. That’s why today, we’re thrilled to announce our [$70 million Series C](https://www.prnewswire.com/news-releases/arize-ai-secures-70m-series-c-to-fix-ais-biggest-problem-making-llms-and-ai-agents-work-in-the-real-world-302381601.html?tc=eml_cleartime) to accelerate our mission: ensuring LLMs and AI agents don’t just work—but work reliably at scale in the real world. Powering The Next Generation of AI Agents ----------------------------------------- This round, the largest-ever investment in AI observability, was led by Adams Street Partners, with participation from M12 (Microsoft’s venture fund), Sinewave Ventures, OMERS Ventures, Datadog, PagerDuty, Industry Ventures, and Archerman Capital. We’re also grateful to our existing investors—Foundation Capital, Battery Ventures, TCV, and Swift VC—for doubling down on their commitment to our vision. Why now? AI is no longer confined to research labs or X/Twitter demos— AI agents will be making real-world decisions in trading, logistics, and critical infrastructure, often without direct human oversight. As a result, trust, evaluation, and reliability have never been more important. Arize ensures that AI teams can test, debug, and optimize their systems before failure cascades into production. Unified Platform: Evaluation & Observability -------------------------------------------- > I have Cursor open in one window and Arize open in another – Arize Customer Our vision of building the next generation of intelligent applications is radically different from how we build software today: In software development, you have different systems for development and production. In AI, data is the fuel that drives development and the data derived from production will power development. In software, tracing is an afterthought. In AI, tracing is a first class citizen that is instrumental in your local AI development. In software, testing is fairly simple code. In AI, testing requires AI evaluations and those same AI evaluations are used to perfect your product in production. In software, there is code and a small number of people who can edit that code. In AI, there are prompts & models and anyone that can write English can edit a prompt. In software, you deliver fixed deterministic systems that process data. In AI, we believe that self learning systems powered by production data, directed by AI evaluations, will be optimizing themselves in self learning iterative loops. _Simply put, in Artificial Intelligence, there will be a single Unified Platform across development and production, Evaluation and Observability, unified by data._ Independence Matters -------------------- The burgeoning ecosystem of agent frameworks, gateways and model providers means that independence matters more than ever. In response, we’ve built a best-in-class, framework independent, AI evaluation and observability suite to help AI engineers debug, monitor, and optimize AI systems: * Arize AX for Enterprise – The leading evaluation and observability platform for AI engineers, spanning generative AI, AI agents, machine learning, and computer vision. * Arize Phoenix OSS – The open-source AI observability and performance tracing tool launched in 2023, now with over two million monthly downloads and growing. * Arize AI Copilot – The first AI assistant for AI engineers, launched in 2024, with over 50 built-in skills, from debugging agent traces to writing evals and optimizing prompts. AI teams need better infrastructure for debugging and evaluation—not just for today’s AI applications, but for the future of multi-agent systems, reinforcement learning, and autonomous AI. That’s why we’re also expanding our partnership with Microsoft, bringing deeper integrations with Azure AI Studio, the Azure AI Foundry portal, SDK, and CLI. Additionally, we continue to deepen technical integrations with Google Cloud and NVIDIA’s AI microservices, making it easier for AI engineers to standardize observability across any stack. Shaping the Future of Trustworthy LLMs & AI Agents -------------------------------------------------- At Arize, we believe AI can only reach its full potential if it’s built on a foundation of reliability, transparency, and accountability. As AI takes on high-stakes roles in finance, healthcare, and autonomous systems, ensuring its trustworthiness isn’t just important—it’s mission-critical. From day one, we’ve been committed to building the infrastructure AI engineers need to push the field forward—whether that’s debugging complex models, closing gaps in training data, reducing bias, or optimizing multi-agent systems. Our goal isn’t just to make AI work; it’s to make AI work responsibly, explainably, and in ways that amplify human decision-making. This funding isn’t just about our growth—it’s about investing in the broader AI ecosystem. We’re doubling down on our work with customers, partners, and the open-source community to ensure AI remains a force for progress—rather than an unchecked risk. What’s Next? ------------ With this new round of funding, we’re doubling down on our mission: * Scaling AI evaluation and monitoring for LLMs, AI agents, and multi-agent systems * Expanding Arize Phoenix OSS, now the most widely adopted AI observability library * Advancing research through OpenEvals and AgentEvals initiatives * Hiring world-class engineers to shape the future of AI observability We’re Hiring! Join Us in Shaping the Future of AI Observability --------------------------------------------------------------- Building the future of AI observability isn’t just an exciting technical challenge—it’s a mission-critical problem that will define how AI is built and deployed for years to come. At Arize, we don’t just build tools; we tackle the hardest problems in AI reliability. Our engineers, researchers, and product teams work at the intersection of machine learning, software engineering, and AI infrastructure, developing technology that help companies push the boundaries of what’s possible with LLMs, autonomous agents, and reinforcement learning. We’re looking for curious, driven engineers, researchers, and GTM builders who are passionate about AI’s future and want to ensure it’s built on a solid foundation. If you want to work on the bleeding-edge of AI infrastructure, we’d love to hear from you. Check out our [open roles here](https://arize.com/careers/) . Big things are ahead for 2025, and we’re just getting started. _–Jason & Aparna_ Share * [![](https://arize.com/wp-content/themes/arize-2022/images/icon-tw-circle.svg)](https://twitter.com/intent/tweet?url=https%3A%2F%2Farize.com%2Fblog%2Farize-ai-raises-70m-series-c-to-build-the-gold-standard-for-ai-evaluation-observability%2F&text=Arize%20AI%20Raises%20$70M%20Series%20C%20to%20Build%20the%20Gold%20Standard%20for%20AI%20Evaluation%20&%20Observability) * [![](https://arize.com/wp-content/themes/arize-2022/images/icon-fb-circle.svg)](https://www.facguide.com/sharer/sharer.php?u=https://arize.com/blog/arize-ai-raises-70m-series-c-to-build-the-gold-standard-for-ai-evaluation-observability/&picture=https://arize.com) * [![](https://arize.com/wp-content/themes/arize-2022/images/icon-in-circle.svg)](https://www.linkedin.com/shareArticle?mini=true&url=https%3A%2F%2Farize.com%2Fblog%2Farize-ai-raises-70m-series-c-to-build-the-gold-standard-for-ai-evaluation-observability%2F&title=Arize%20AI%20Raises%20$70M%20Series%20C%20to%20Build%20the%20Gold%20Standard%20for%20AI%20Evaluation%20&%20Observability) #### Suggested reading ![](https://arize.com/wp-content/uploads/2025/10/iso-iec-27001-certified-icon.jpg) [Arize AI Achieves ISO/IEC 27001 Certification](https://arize.com/blog/arize-ai-achieves-iso-iec-27001-certification/) ![](https://arize.com/wp-content/uploads/2025/03/NVIDIA-Arize-blog.jpg) [Self-Improving Agents: Automating LLM Performance Optimization using Arize and NVIDIA NeMo](https://arize.com/blog/arize-nvidia-nemo-integration/) Sign up for our monthly newsletter, The Evaluator. -------------------------------------------------- [Subscribe](https://arize.com/blog/arize-ai-raises-70m-series-c-to-build-the-gold-standard-for-ai-evaluation-observability/#blog-subscribe-modal) Subscribe to The Evaluator -------------------------- We’ll send you the latest news, expertise, and product updates from Arize. Your inbox is sacred, so we’ll only curate and send the best stuff. \*We’re committed to your privacy. Arize uses the information you provide to contact you about relevant content, products, and services. You may unsubscribe from these communications at any time. For more information, check out our [privacy policy](https://arize.com/privacy-policy) . --- # Arize AI Named TiE50 Award Winner at TiEcon - Arize AI ![](https://arize.com/wp-content/uploads/2020/12/bg-speaker-196x196.png "bg-speaker") [Aparna Dhinakaran](https://arize.com/author/aparna-dhinakaran/) Co-founder & Chief Product Officer Share * [![](https://arize.com/wp-content/themes/arize-2022/images/icon-tw-circle.svg)](https://twitter.com/intent/tweet?url=https%3A%2F%2Farize.com%2Fblog%2Farize-ai-named-tie50-award-winner-at-tiecon%2F&text=Arize%20AI%20Named%20TiE50%20Award%20Winner%20at%20TiEcon) * [![](https://arize.com/wp-content/themes/arize-2022/images/icon-fb-circle.svg)](https://www.facguide.com/sharer/sharer.php?u=https://arize.com/blog/arize-ai-named-tie50-award-winner-at-tiecon/&picture=https://arize.com) * [![](https://arize.com/wp-content/themes/arize-2022/images/icon-in-circle.svg)](https://www.linkedin.com/shareArticle?mini=true&url=https%3A%2F%2Farize.com%2Fblog%2Farize-ai-named-tie50-award-winner-at-tiecon%2F&title=Arize%20AI%20Named%20TiE50%20Award%20Winner%20at%20TiEcon) ![](https://arize.com/wp-content/themes/arize-2022/images/icon-newsletter.svg) Subscribe to the Arize blog [Get the latest](https://arize.com/blog/arize-ai-named-tie50-award-winner-at-tiecon/#blog-subscribe-modal) #### On this page #### Suggested reading ![](https://arize.com/wp-content/uploads/2025/11/microsoft-foundry-arize-ax.png) [Evaluating and Improving AI Agents at Scale with Microsoft Foundry](https://arize.com/blog/evaluating-and-improving-ai-agents-at-scale-with-microsoft-foundry/) ![](https://arize.com/wp-content/uploads/2025/10/llm-tracing-blog-cover.png) [Top LLM Tracing Tools](https://arize.com/blog/top-llm-tracing-tools/) ![](https://arize.com/wp-content/uploads/2020/10/AdobeStock_369481241-2142x1120.jpeg "Team of happy employees winning award and celebrating success. Business people enjoying victory, getting gold cup trophy. Vector illustration for reward, prize, champions concepts") Arize AI Named TiE50 Award Winner at TiEcon =========================================== Published Aug 31, 2020 * [Company](https://arize.com/blog/?cat=company) * [ML Observability](https://arize.com/blog/?cat=ml-observability) * [Uncategorized](https://arize.com/blog/?cat=uncategorized) ![](https://arize.com/wp-content/uploads/2020/12/bg-speaker-196x196.png "bg-speaker") #### [Aparna Dhinakaran](https://arize.com/author/aparna-dhinakaran/) ##### Co-founder & Chief Product Officer ![](https://arize.com/wp-content/uploads/2021/03/1_7wH6Y-mcpUTALh0wp0MAJA.png) Arize AI, the first-to-market ML Observability Platform, is excited to announce that it has been selected as a 2020 TiE50 Winner in the prestigious TiE50 Awards Program. This ten year old awards competition is a program of TiEcon, the world’s largest conference for tech entrepreneurs. Arize AI was recognized for its innovative platform to troubleshoot, monitor, and explain AI. “Arize AI is the first go-to-market ML Observability platform. We are the only platform to gather actuals and capture true model performance. As businesses deploy more models into production and these models get more complex, model observability is key to making models successful.” — Jason Lopatecki, CEO of Arize AI “TiE50 again attracted high potential startups bringing innovation from different parts of the world. Besides the recognition associated with the TiE50 award, this year TiE50 also partnered with Meet the Draper’s, a ground-breaking reality show to give an opportunity to some companies to pitch to the show,” said Kamal Anand, TiE50 Program Chair. “For over 28 years as a not-for-profit organization dedicated to fostering entrepreneurship and with a global footprint of half million entrepreneurs, enterprise executives, and investment professionals, at TiE Silicon Valley we take pride in the fact that we have created TiE50, a strong 10-year-old brand for recognizing high potential startups,” said B.J. Arun, President, TiE Silicon Valley. The TiE50 Awards will be presented virtually to the winners during a ceremony on September 3rd. For more information, please go to [https://www.tiecon.org/TiE50Awards2020/](https://www.tiecon.org/TiE50Awards2020/) **About Arize AI** Arize AI is building the first ML Observability platform to help make machine learning models work in production. They provide a real time platform to monitor, explain and troubleshoot model & data issues as models move from research to production. Their team comes top ML and engineering teams including Google, Facebook, Uber, Adobe, etc. **About TiE50** Now celebrating its tenth year, TiE50 Awards provides a one-of-a-kind showcase for the world’s top technology and technology-enabled startups. TiE Silicon Valley’s premier annual awards program is keenly contested by thousands of early- to mid-stage startups of all sizes representing a wide range of verticals. Applications are rigorously reviewed by a panel of judges including venture capitalists, angels, successful entrepreneurs, and corporate executives. Since its inception, 84 percent of TiE50 winners and top startups have been funded at a total of over $1 billion. Many of these companies went on to acquisition or IPO with 29 of the exits at over $100 million. **About TiEcon** TiEcon is the world’s largest conference for entrepreneurs and intrapreneurs with participation from top technology companies, leading venture capital firms, and global service providers. Delegates range from CEOs of top companies to first-time entrepreneurs as well as corporate executives and investment professionals. TiEcon was listed as one of the 10 best conferences for ideas and entrepreneurship by Worth Magazine, along with TED and the World Economic Forum. Previous TiEcon events have attracted 5,000+ attendees from 22 countries. More information: [TiEcon.org](https://www.tiecon.org/about-us/) Share * [![](https://arize.com/wp-content/themes/arize-2022/images/icon-tw-circle.svg)](https://twitter.com/intent/tweet?url=https%3A%2F%2Farize.com%2Fblog%2Farize-ai-named-tie50-award-winner-at-tiecon%2F&text=Arize%20AI%20Named%20TiE50%20Award%20Winner%20at%20TiEcon) * [![](https://arize.com/wp-content/themes/arize-2022/images/icon-fb-circle.svg)](https://www.facguide.com/sharer/sharer.php?u=https://arize.com/blog/arize-ai-named-tie50-award-winner-at-tiecon/&picture=https://arize.com) * [![](https://arize.com/wp-content/themes/arize-2022/images/icon-in-circle.svg)](https://www.linkedin.com/shareArticle?mini=true&url=https%3A%2F%2Farize.com%2Fblog%2Farize-ai-named-tie50-award-winner-at-tiecon%2F&title=Arize%20AI%20Named%20TiE50%20Award%20Winner%20at%20TiEcon) #### Suggested reading ![](https://arize.com/wp-content/uploads/2025/11/microsoft-foundry-arize-ax.png) [Evaluating and Improving AI Agents at Scale with Microsoft Foundry](https://arize.com/blog/evaluating-and-improving-ai-agents-at-scale-with-microsoft-foundry/) ![](https://arize.com/wp-content/uploads/2025/10/llm-tracing-blog-cover.png) [Top LLM Tracing Tools](https://arize.com/blog/top-llm-tracing-tools/) Sign up for our monthly newsletter, The Evaluator. -------------------------------------------------- [Subscribe](https://arize.com/blog/arize-ai-named-tie50-award-winner-at-tiecon/#blog-subscribe-modal) Subscribe to The Evaluator -------------------------- We’ll send you the latest news, expertise, and product updates from Arize. Your inbox is sacred, so we’ll only curate and send the best stuff. \*We’re committed to your privacy. Arize uses the information you provide to contact you about relevant content, products, and services. You may unsubscribe from these communications at any time. For more information, check out our [privacy policy](https://arize.com/privacy-policy) . --- # Arize AI Partners with Algorithmia to Enable Better MLOps and Observability for Enterprises - Arize AI ![](https://arize.com/wp-content/uploads/2020/12/bg-speaker-196x196.png "bg-speaker") [Aparna Dhinakaran](https://arize.com/author/aparna-dhinakaran/) Co-founder & Chief Product Officer Share * [![](https://arize.com/wp-content/themes/arize-2022/images/icon-tw-circle.svg)](https://twitter.com/intent/tweet?url=https%3A%2F%2Farize.com%2Fblog%2Farize-ai-partners-with-algorithmia%2F&text=Arize%20AI%20Partners%20with%20Algorithmia%20to%20Enable%20Better%20MLOps%20and%20Observability%20for%20Enterprises) * [![](https://arize.com/wp-content/themes/arize-2022/images/icon-fb-circle.svg)](https://www.facguide.com/sharer/sharer.php?u=https://arize.com/blog/arize-ai-partners-with-algorithmia/&picture=https://arize.com) * [![](https://arize.com/wp-content/themes/arize-2022/images/icon-in-circle.svg)](https://www.linkedin.com/shareArticle?mini=true&url=https%3A%2F%2Farize.com%2Fblog%2Farize-ai-partners-with-algorithmia%2F&title=Arize%20AI%20Partners%20with%20Algorithmia%20to%20Enable%20Better%20MLOps%20and%20Observability%20for%20Enterprises) ![](https://arize.com/wp-content/themes/arize-2022/images/icon-newsletter.svg) Subscribe to the Arize blog [Get the latest](https://arize.com/blog/arize-ai-partners-with-algorithmia/#blog-subscribe-modal) #### On this page #### Suggested reading ![](https://arize.com/wp-content/uploads/2025/11/microsoft-foundry-arize-ax.png) [Evaluating and Improving AI Agents at Scale with Microsoft Foundry](https://arize.com/blog/evaluating-and-improving-ai-agents-at-scale-with-microsoft-foundry/) ![](https://arize.com/wp-content/uploads/2025/10/llm-tracing-blog-cover.png) [Top LLM Tracing Tools](https://arize.com/blog/top-llm-tracing-tools/) ![Algorithmia - Arize](https://arize.com/wp-content/uploads/2022/08/Algorithmia-Arize-blog-cover.jpg "Algorithmia -Arize blog cover") Arize AI Partners with Algorithmia to Enable Better MLOps and Observability for Enterprises =========================================================================================== Published Apr 19, 2021 * [Company](https://arize.com/blog/?cat=company) * [ML Monitoring](https://arize.com/blog/?cat=ml-monitoring) * [ML Observability](https://arize.com/blog/?cat=ml-observability) * [MLOps](https://arize.com/blog/?cat=mlops) * [Uncategorized](https://arize.com/blog/?cat=uncategorized) ![](https://arize.com/wp-content/uploads/2020/12/bg-speaker-196x196.png "bg-speaker") #### [Aparna Dhinakaran](https://arize.com/author/aparna-dhinakaran/) ##### Co-founder & Chief Product Officer _Written in collaboration with Ezra Citron, Customer Solutions Consultant at Algorithmia._ We’re excited to share that Arize AI and Algorithmia are partnering to help organizations deliver more models to production, maximize their performance, and minimize model risk. From optimizing delivery ETAs to protecting against defaults, modern companies of all sizes and industries understand the value of leveraging machine learning and AI to achieve key business objectives. However, despite the often significant and accelerating R&D investments in ML systems, taking a model from research to production isn’t an easy feat. [Machine learning operations](https://algorithmia.com/mlops?utm_medium=arize&utm_source=guest-blog) (MLOps) and ML observability are some of the most significant challenges faced by teams trying to scale their ML efforts. AI Investment Problem --------------------- In the last decade, digital transformation has been elevated to a mission-critical imperative for many organizations. At its core, this shift has accelerated companies’ use of cloud computing to power how they build and operate their technology stacks—the most advanced teams have leveraged this transition to adopt a data-centric approach to tackle key business objectives and fuel customer engagement. As Clive Humby put simply: Data is the new oil. How teams harness the power of data and applications can make or break a business. The effects of this are especially acute when considering the increasing reliance on automation and machine-learned systems that power AI. Today, a wealth of business intelligence and data processing solutions are available to help organizations make sense of their data and build machine learning models into their businesses. The north star is to leverage unique data and machine learning to give one’s business an edge over the competition. However, the reality is that despite an ever-growing investment in AI, many organizations fail to reach this desired result for multiple reasons. Algorithmia’s [2021 enterprise trends in machine learning report](https://info.algorithmia.com/2021?utm_medium=arize&utm_source=guest-blog&utm_campaign=IC-2012-2021-ML-Trends) revealed that 83% of all organizations had increased their AI/ML budgets in the past year. The average number of data scientists employed has grown by 76%. Despite this level of investment, the time required to deploy a trained model to production has increased, with 64% of all organizations taking a month or longer. Organizations are struggling with the operational components needed to deploy and operate ML models in production after the development stage. Once over the initial hurdle of getting ML models into production, another challenge is monitoring and optimizing model performance. Without proper model observability, the reality is that AI investment can sometimes feel like you’re throwing money into a black box. A model that worked for one customer or scenario doesn’t work for another. What worked last Thursday doesn’t work this Tuesday. A recommendation from a model for credit or purchase angers a customer. Have you added risk and cost to your business that you don’t understand? Data is the new oil, but only if harnessed in the right way. How can organizations that invest heavily in data science reap the rewards of that investment? Simply investing in ML and developing models is not enough. To unlock the value in your ML investment, you need MLOps—and a critical component of your MLOps—ML observability. That’s why we’re excited to share that Arize AI and Algorithmia are partnering to help organizations deliver more models to production, maximize their performance, and minimize model risk. With a simple integration, customers will be able to leverage the combined power of Algorithmia’s enterprise MLOps platform and Arize’s ML observability platform to deploy models and manage performance at scale. Streamline Machine Learning Operations -------------------------------------- Algorithmia’s [enterprise MLOps platform](https://algorithmia.com/mlops?utm_medium=arize&utm_source=guest-blog) manages all stages of the production ML lifecycle from deployment and operations to governance and security—enabling data science and ML teams to deliver more models quicker while protecting the business. With Algorithmia, organizations of all sizes can easily: * Connect, load, catalog, version, and validate models for production in a central platform. * Manage costs, control infrastructure usage, monitor operations, and deliver models and services at high velocity. * Minimize risk with enterprise-grade security and governance across all data, models, and infrastructure. **Bridge the Gap Between Data Science and ML Engineering** As enterprise AI/ML systems grow to operate at scale, deep model observability is critical to make well-informed business investments and build a high-performing MLOps practice. Arize AI provides real-time monitoring and observability to help teams understand how their models perform in the real world and improve their performance. The ability to upload offline (training or validation) baselines into an evaluation store for automated drift, data quality, and performance analysis creates an active feedback mechanism between data science and engineering teams so that they can: * Manage and improve machine learning investment through a single pane of glass. * Map drift changes to actual performance changes. * Make real-time drift assessments, multi-model performance comparisons, fairness/bias evaluations, and performance monitoring assessments with support for delayed ground truth. * Complete root-cause analyses to troubleshoot model failures/performance degradation using explainability and slice analysis. Leveraging the Arize and Algorithmia platforms give struggling teams a streamlined ability to test and compare model performance, discover the root cause of issues in production, manage risks and costs, and uncover retraining opportunities. This solution empowers teams to increase the velocity at which they can develop and iterate on models, increase model quality, and decrease cost, delivering AI into the business as if it were a product. **Get Started with the Algorithmia-Arize Integration** For existing Algorithmia customers, integrating with the Arize AI platform is simple. If you’re not familiar with Algorithmia or Arize, sign up for an [Algorithmia demo](https://algorithmia.com/demo?utm_medium=arize&utm_source=guest-blog) and [early access](https://arize.com/) to Arize. Below, we’ll explain the basic components of the integration; to test it out, you’ll need accounts on both platforms. The general workflow is that when you deploy your model on Algorithmia, you’ll add some code to establish a connection with Arize and to log the features, prediction, and actuals every time the model is called. You can then use these data on the Arize side for model tracking and explainability. In the code samples below, we’re just showing the parts associated with establishing the connection to Arize and logging the data; for the complete code showing the integration end-to-end, visit [Algorithmia’s Developer Center](https://algorithmia.com/developers/integrations/arize) . Before incorporating the logging functionality into an Algorithmia algorithm, you can use a Jupyter notebook or your favorite local IDE to test the part of the code that sends data to Arize, to make sure the connection is configured properly and the library dependencies are in place. We recommend this workflow for development, as debugging is often more efficient in a local environment. Begin by importing the necessary Arize classes from the Arize Python client library, as well as some additional Python modules and libraries: from arize.api import Client from arize.types import ModelTypes import datetime import joblib import pandas as pd Note that the code sample below assumes a trained model serialized as _MODEL\_NAME.joblib_ and some test data in the pandas _DataFrames X\_test_ and _y\_test_. Calling the to\_json() method on X\_test, the first row of data looks like this: { "mean radius":{"204":12.470000267}, "mean texture":{"204":18.6000003815}, ... "worst symmetry":{"204":0.3014000058}, "worst fractal dimension":{"204":0.0874999985} } The _arize.api.Client_ object establishes the connection to your account on Arize, so you’ll need to provide your secret Arize credentials. The _Client’s log\_bulk\_predictions()_ and _log\_bulk\_actuals()_ methods do the heavy lifting, sending the specified data to the Arize platform: import Algorithmia import pandas as pd #Provide Algorithmia secret; we recommend reading from an environment variable. ALGORITHMIA\_API\_KEY = "ALGORITHMIA\_API\_KEY" #Establish a connection with Algorithmia. client = Algorithmia.client(ALGORITHMIA\_API\_KEY) #Identify your new algorithm and instantiate an algorithm object. ALGO\_ENDPOINT = "ALGO\_OWNER/ALGO\_NAME/ALGO\_VERSION" algo = client.algo(ALGO\_ENDPOINT) #Optionally set timeout parameters for testing purposes. algo.set\_options(timeout=60) #Pipe JSON payload into algorithm and convert JSON output back to DataFrame. input = X\_test.to\_json() result\_json = algo.pipe(input).result result\_df = pd.read\_json(res) After you verify that the sample data are being logged to Arize, simply move the logging functionality into an Algorithmia algorithm, which will be exposed as an API endpoint to be called in real-time. Once you’ve uploaded your serialized model and published the algorithm, you can send data straight into it through the pipe() function, as demonstrated in the example code below. Note that since the Algorithmia API requires JSON-formatted data for its apply() function, you’ll need to convert any production and/or test data to JSON before sending them as algorithm input. Algorithmia will return the response directly as another JSON object with the algorithm output stored in the object’s result attribute. You can then convert the JSON output into the desired type—in this example, back into a pandas DataFrame. Below is an example of code that can be used to call your published model on Algorithmia. The Algorithm code itself is shown in the [integration guide on the Developer Center](https://algorithmia.com/developers/integrations/arize) . import Algorithmia import pandas as pd # Provide Algorithmia secret; we recommend reading from an environment variable. ALGORITHMIA\_API\_KEY = "ALGORITHMIA\_API\_KEY" # Establish a connection with Algorithmia. client = Algorithmia.client(ALGORITHMIA\_API\_KEY) # Identify your new algorithm and instantiate an algorithm object. ALGO\_ENDPOINT = "ALGO\_OWNER/ALGO\_NAME/ALGO\_VERSION" algo = client.algo(ALGO\_ENDPOINT) # Optionally set timeout parameters for testing purposes. algo.set\_options(timeout=60) # Pipe JSON payload into algorithm and convert JSON output back to DataFrame. input = X\_test.to\_json() result\_json = algo.pipe(input).result result\_df = pd.read\_json(res) When you call your algorithm, your prediction events are now logged to Arize, and the platform discovers your model and sets up dashboards, monitors, and analytics for your predictions. Default dashboards are set up to highlight critical evaluation and data metrics. In addition to inference metrics, you can also send operational metrics from Algorithmia to Arize. For a detailed, step-by-step walk-through of this integration, visit [Algorithmia’s Developer Center](https://algorithmia.com/developers/integrations/arize) . ![](https://arize.com/wp-content/uploads/2021/04/arize1.png) Example Arize Model Performance Metrics Dashboard ![](https://arize.com/wp-content/uploads/2021/04/arize2.png) _Example PSI Monitor_ ![](https://arize.com/wp-content/uploads/2021/04/arize3.png) Local, Slice, or Global Explainability **About Arize AI** Arize AI is a [Machine Learning Observabililty](https://arize.com/model-monitoring "https://arize.com/model-monitoring") platform that helps ML practitioners successfully take models from research to production, with ease. Arize’s automated [model monitoring](https://arize.com/ml-monitoring/ "https://arize.com/ml-monitoring/") and analytics platform help ML teams quickly detect issues the moment they emerge, troubleshoot why they happened, and improve overall model performance. By connecting offline training and validation datasets to online production data in a central inference store, ML teams are able to streamline [model validation](https://arize.com/ml-model-failure-modes/ "https://arize.com/ml-model-failure-modes/") , [drift detection](https://arize.com/take-my-drift-away/ "https://arize.com/take-my-drift-away/") , [data quality checks](https://arize.com/data-quality-monitoring/ "https://arize.com/data-quality-monitoring/") , and [model performance management](https://arize.com/monitor-your-model-in-production/ "https://arize.com/monitor-your-model-in-production/") . Arize AI acts as the guardrail on deployed AI, providing transparency and introspection into historically black box systems to ensure more effective and [responsible AI](https://www.forbes.com/sites/aparnadhinakaran/?sh=5d7691024958 "https://www.forbes.com/sites/aparnadhinakaran/?sh=5d7691024958") . To learn more about Arize or machine learning observability and monitoring, visit our [blog](https://arize.com/blog/ "https://arize.com/blog/") and [resource hub](https://arize.com/resource-hub/ "https://arize.com/resource-hub/") ! **About Algorithmia** Algorithmia is the enterprise MLOps platform. It manages all stages of the production ML lifecycle within existing operational processes, so you can put models into production quickly, securely, and cost-effectively. Unlike inefficient and expensive do-it-yourself MLOps management solutions that lock users into specific technology stacks, Algorithmia automates ML deployment, optimizes collaboration between operations and development, leverages existing SDLC and CI/CD systems, integrates with best-of-breed tools, and provides advanced security and governance. Over 130,000 engineers and data scientists have used Algorithmia’s platform to date, including the United Nations, government intelligence agencies, and Fortune 500 companies. To learn more, [explore the Algorithmia platform](https://algorithmia.com/product) and [get your demo](https://algorithmia.com/demo?utm_medium=arize&utm_source=guest-blog) today. Share * [![](https://arize.com/wp-content/themes/arize-2022/images/icon-tw-circle.svg)](https://twitter.com/intent/tweet?url=https%3A%2F%2Farize.com%2Fblog%2Farize-ai-partners-with-algorithmia%2F&text=Arize%20AI%20Partners%20with%20Algorithmia%20to%20Enable%20Better%20MLOps%20and%20Observability%20for%20Enterprises) * [![](https://arize.com/wp-content/themes/arize-2022/images/icon-fb-circle.svg)](https://www.facguide.com/sharer/sharer.php?u=https://arize.com/blog/arize-ai-partners-with-algorithmia/&picture=https://arize.com) * [![](https://arize.com/wp-content/themes/arize-2022/images/icon-in-circle.svg)](https://www.linkedin.com/shareArticle?mini=true&url=https%3A%2F%2Farize.com%2Fblog%2Farize-ai-partners-with-algorithmia%2F&title=Arize%20AI%20Partners%20with%20Algorithmia%20to%20Enable%20Better%20MLOps%20and%20Observability%20for%20Enterprises) #### Suggested reading ![](https://arize.com/wp-content/uploads/2025/11/microsoft-foundry-arize-ax.png) [Evaluating and Improving AI Agents at Scale with Microsoft Foundry](https://arize.com/blog/evaluating-and-improving-ai-agents-at-scale-with-microsoft-foundry/) ![](https://arize.com/wp-content/uploads/2025/10/llm-tracing-blog-cover.png) [Top LLM Tracing Tools](https://arize.com/blog/top-llm-tracing-tools/) Sign up for our monthly newsletter, The Evaluator. -------------------------------------------------- [Subscribe](https://arize.com/blog/arize-ai-partners-with-algorithmia/#blog-subscribe-modal) Subscribe to The Evaluator -------------------------- We’ll send you the latest news, expertise, and product updates from Arize. Your inbox is sacred, so we’ll only curate and send the best stuff. \*We’re committed to your privacy. Arize uses the information you provide to contact you about relevant content, products, and services. You may unsubscribe from these communications at any time. For more information, check out our [privacy policy](https://arize.com/privacy-policy) . --- # Arize AI Unveils Prompt Engineering and Retrieval Tracing Workflows For LLM Troubleshooting - Arize AI Arize AI Unveils Prompt Engineering and Retrieval Tracing Workflows For LLM Troubleshooting =========================================================================================== ![](https://arize.com/wp-content/uploads/2023/08/arize-embeddings-prod.jpg) **San Francisco, CA, August 30, 2023** – Arize AI, a market leader in machine learning observability, debuted industry-first capabilities for troubleshooting large language models (LLMs) at Google Cloud Next ‘23 today. Arize’s new prompt engineering workflows, including a new prompt playground, enables teams to find prompt templates that need to be improved, iterate on them in real time, and verify improved LLM outputs. Prompt analysis is an important component in troubleshooting an LLM’s performance. Often, LLM performance can be improved simply by testing different prompt templates, or iterating on one to achieve better responses. With these new workflows, teams can: * Uncover responses with poor user feedback or evaluation scores * Identify the template associated with poor responses * Iterate on the existing prompt template * Compare responses across prompt templates in a Prompt Playground Arize is also launching additional search and retrieval workflows to help teams using [retrieval augmented generation](https://arize.com/blog-course/introduction-to-retrieval-augmented-generation/) (RAG) troubleshoot where and how the retrieval needs to be improved. These new workflows will help teams identify where they may need to add additional context into their knowledge base (or vector database), when the retrieval didn’t retrieve the most relevant information, and ultimately understand why their LLM may have hallucinated or generated sub-optimal responses. “Building LLM-powered systems that responsibly work in the real-world is still too difficult today,” said Aparna Dhinakaran, Co-Founder and Chief Product Officer of Arize. “These industry-first prompt engineering and RAG workflows will help teams get to value and resolve issues faster, ultimately improving outcomes and proving the value of generative AI across industries.” ![retrieval tracing llm workflows](https://arize.com/wp-content/uploads/2023/08/arize-embeddings-prod.jpg) Retrieval tracing ![prompt playground analysis](https://arize.com/wp-content/uploads/2023/08/arize-prompt-playground.jpg) Prompt analysis **About Arize AI**  Arize AI is a machine learning observability platform that helps ML teams deliver and maintain more successful AI in production. Arize’s automated model monitoring and observability platform allows ML teams to quickly detect issues when they emerge, troubleshoot why they happened, and improve overall model performance across both structured data and image and large language models. Arize is a remote-first company with headquarters in Berkeley, CA. Subscribe to The Evaluator -------------------------- We’ll send you the latest news, expertise, and product updates from Arize. Your inbox is sacred, so we’ll only curate and send the best stuff. \*We’re committed to your privacy. Arize uses the information you provide to contact you about relevant content, products, and services. You may unsubscribe from these communications at any time. For more information, check out our [privacy policy](https://arize.com/privacy-policy) . --- # Arize Platform demo - Arize AI Arize Platform demo =================== Subscribe to The Evaluator -------------------------- We’ll send you the latest news, expertise, and product updates from Arize. Your inbox is sacred, so we’ll only curate and send the best stuff. \*We’re committed to your privacy. Arize uses the information you provide to contact you about relevant content, products, and services. You may unsubscribe from these communications at any time. For more information, check out our [privacy policy](https://arize.com/privacy-policy) . --- # Arize AI and Infogain Partner to Accelerate Enterprise AI Outcomes With Ignis - Arize AI Arize AI and Infogain Partner to Accelerate Enterprise AI Outcomes With Ignis ============================================================================= ![](https://arize.com/wp-content/uploads/2025/10/infogain-arize.png) Collaboration combines Infogain’s consulting and delivery expertise with Arize’s vendor-agnostic AI engineering & observability platform to operationalize reliable, scalable AI systems ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- **Berkeley, CA — October 30, 2025** — Arize AI, a leading AI observability and LLM evals company, today announced a strategic partnership with Infogain to help joint customers design, evaluate, and scale AI systems from first prototype to production outcomes.  Infogain’s Ignis — an AI innovation engine that leverages agentic frameworks and a broad partner ecosystem — will integrate Arize’s AI agent engineering and observability capabilities to ensure complex AI applications deliver value and are reliable and measurable in production.  ### **Why It Matters** Enterprises are racing to translate successful GenAI pilots into durable business value. This  collaboration pairs Infogain’s AI-first offerings, delivery accelerators, and _Ignis Agentic Platform_ (an LLM and cloud-agnostic agentic platform) with Arize’s end-to-end evaluation, tracing, and monitoring. The results enable organizations to move faster while meeting governance and ROI requirements. “Our clients want AI that performs in the real world, not just in the lab. By bringing Arize AX into Ignis Agentic Platform, we bring a comprehensive platform with the best of our AI solutions & partnerships to help enterprises during the end-to-end of the agent lifecycle and tie the results to business KPIs,” said Mohit Bhat, Chief Delivery & Innovation Officer at Infogain.. ### **How Arize Complements Ignis** Arize AX helps teams test, evaluate, and monitor complex AI systems — from semi-autonomous multi-agent workflows and copilots to customer-facing applications — with: * **Agent Tracing & Replay:** Visibility across tools, LLM calls, and decision nodes; quickly reproduce and diagnose issues before they impact users. * **Evaluations & Online Evals:** Continuous, KPI-aligned checks to catch regressions and improve quality pre- and post-deployment. * **Prompt & Model Optimization:** Compare prompts and models across datasets; choose the best stack for each use case. * **Real-Time Monitors & Alerts:** Production observability tied to business outcomes and SLAs. Coupled with Ignis’ AI-first approach, partner ecosystem, and _Ignis Agentic_ framework, AI-ready delivery teams can accelerate time-to-value while maintaining rigor on security, scale, and governance. “Infogain’s Ignis meets enterprises where they are and gets them to value fast. Together, we bring the discipline of evaluation and observability to every stage — from discovery to scaled rollout,” said Noah Smolen, Head of Partnerships at Arize AI.  Arize is independent and framework-agnostic, working across clouds, LLMs, and agent frameworks. With native support for OpenInference — the open standard developed at the company — teams can unify traces and evaluations quickly. Under the hood, Arize’s purpose-built database (adb) is engineered to scale real-time ingest, search, and analytics at enterprise scale. About Infogain -------------- Infogain is a leader in digital customer experience engineering based in Silicon Valley. Infogain engineers business outcomes for Fortune 500 companies and digital natives in the technology, healthcare, insurance, travel, telecom, and retail/CPG industries. It accelerates experience-led transformation in the delivery of digital platforms using technologies such as cloud, microservices, automation, IoT, and artificial intelligence. Infogain is a multi-cloud expert across hyperscale cloud providers – Microsoft Azure, Google Cloud Platform and Amazon Web Services. Infogain, an Apax Funds portfolio company, has offices in California, Washington, Texas, the UK, and Singapore, with delivery centers in Seattle, Dallas, Montevideo, Kraków, Noida, Bengaluru, Pune, Gurgaon, and Mumbai. To learn more, visit [www.infogain.com](http://www.infogain.com/) About Arize AI -------------- Arize is a leading AI and agent engineering platform that helps teams design, evaluate, and operate production-grade AI systems with reliability and speed. Built to be independent and interoperable across models, frameworks, and clouds — with open-standard instrumentation at its core — Arize unifies development and production workflows in Arize AX (with Alyx for copilot-powered analysis) and complements them with the open-source Phoenix and OpenInference projects. Arize is headquartered in Berkeley, CA. Subscribe to The Evaluator -------------------------- We’ll send you the latest news, expertise, and product updates from Arize. Your inbox is sacred, so we’ll only curate and send the best stuff. \*We’re committed to your privacy. Arize uses the information you provide to contact you about relevant content, products, and services. You may unsubscribe from these communications at any time. For more information, check out our [privacy policy](https://arize.com/privacy-policy) . --- # Arize Partners with UbiOps to Accelerate Model Building & Deployment - Arize AI ![](https://arize.com/wp-content/uploads/2021/06/krystal-headshot-e1624425208666-196x196.jpg "krystal headshot") [Krystal Kirkland](https://arize.com/author/krystal-kirkland/) Software Engineer Share * [![](https://arize.com/wp-content/themes/arize-2022/images/icon-tw-circle.svg)](https://twitter.com/intent/tweet?url=https%3A%2F%2Farize.com%2Fblog%2Farize-partners-with-ubiops%2F&text=Arize%20Partners%20with%20UbiOps%20to%20Accelerate%20Model%20Building%20&%20Deployment) * [![](https://arize.com/wp-content/themes/arize-2022/images/icon-fb-circle.svg)](https://www.facguide.com/sharer/sharer.php?u=https://arize.com/blog/arize-partners-with-ubiops/&picture=https://arize.com) * [![](https://arize.com/wp-content/themes/arize-2022/images/icon-in-circle.svg)](https://www.linkedin.com/shareArticle?mini=true&url=https%3A%2F%2Farize.com%2Fblog%2Farize-partners-with-ubiops%2F&title=Arize%20Partners%20with%20UbiOps%20to%20Accelerate%20Model%20Building%20&%20Deployment) ![](https://arize.com/wp-content/themes/arize-2022/images/icon-newsletter.svg) Subscribe to the Arize blog [Get the latest](https://arize.com/blog/arize-partners-with-ubiops/#blog-subscribe-modal) #### On this page #### Suggested reading ![](https://arize.com/wp-content/uploads/2025/11/microsoft-foundry-arize-ax.png) [Evaluating and Improving AI Agents at Scale with Microsoft Foundry](https://arize.com/blog/evaluating-and-improving-ai-agents-at-scale-with-microsoft-foundry/) ![](https://arize.com/wp-content/uploads/2025/10/llm-tracing-blog-cover.png) [Top LLM Tracing Tools](https://arize.com/blog/top-llm-tracing-tools/) ![](https://arize.com/wp-content/uploads/2021/06/Group-496-1024x573-1.png "Group-496-1024x573") Arize Partners with UbiOps to Accelerate Model Building & Deployment ==================================================================== Published Jun 7, 2021 * [Company](https://arize.com/blog/?cat=company) * [Data Science](https://arize.com/blog/?cat=data-science) * [ML Observability](https://arize.com/blog/?cat=ml-observability) * [MLOps](https://arize.com/blog/?cat=mlops) * [Uncategorized](https://arize.com/blog/?cat=uncategorized) ![](https://arize.com/wp-content/uploads/2021/06/krystal-headshot-e1624425208666-196x196.jpg "krystal headshot") #### [Krystal Kirkland](https://arize.com/author/krystal-kirkland/) ##### Software Engineer _Written in collaboration with UbiOps._ ### **UbiOps and Arize** UbiOps is the easy-to-use serving and hosting layer for data science code. UbiOps stands out for its ease of use, freedom to write any code you want while eliminating the need for in-depth IT knowledge. It is a serving, hosting and management layer on top of your preferred infrastructure. Accessible via the UI, client library, or CLI, it’s suitable for every type of data scientist.  UbiOps is specifically useful for real-time applications that require both simple processing scripts or complex ML models. Thanks to the scalable infrastructure every piece of code can be scaled up and down according to your specifications.  Arize allows for real time observability of machine learning models. Arize extends beyond traditional monitoring and is uniquely focused on enabling ML engineers with a comprehensive observability platform to more effectively detect and troubleshoot issues, perform analysis, and improve model performance.  The platform is backed by an evaluation store, which allows teams to connect datasets across training, validation and production environments. By storing performance metrics for each model version in an evaluation store, users can leverage any dataset as a baseline reference to monitor and explain model performance in production. The evaluation store can hook into an existing feature store and model store to create a virtuous feedback loop for model improvements. **Why this integration?** The more business-critical a model is, the more important observability is to keep a pulse on its health and to quickly resolve any issues that arise.  While deployment of a production-worthy AI model poses a challenge to many, observability is another, deeper challenge that awaits a model in production. With this integration, data scientists and ML engineers can work together to develop a model, push it to production, and gain full visibility and control of its performance.  Teams using Arize and UbiOps together are able to: * Validate model quality and performance prior to deploying to production. * Accelerate model deployment (time to value) and iterations without high ops overhead. * Automatically diagnose issues that emerge in production, with ability to analyze specific cohorts of problematic predictions. * Gain deeper visibility into how models are performing with features such as performance heatmaps, and find opportunities to deliver improvements / retraining.  ![](https://arize.com/wp-content/uploads/2021/06/ubiops_arize.png) Figure 1: architecture overview of the integration **1\. Integration walkthrough and instructions** To demonstrate how Arize and UbiOps can work together we’ll use a (locally trained) TensorFlow model that predicts the miles per gallon usage of a car based on specific attributes such as the amount of cylinders, horsepower, weight and model year.  We’ll work in a jupyter notebook and make use of the UbiOps client libraries to communicate with the backend to host and serve the code. The full notebook can be found [here](https://docs.arize.com/arize/integrations/integrations/ubiops) . The below code snippets show how UbiOps and Arize integrate. This code block is the deployment.py file that UbiOps uses to deploy models on its platform. When new data is sent in, it goes through the request function in order for the model to make predictions. In this example, we send in both the input feature data and the actual data to this function, making it the perfect place to place our Arize logging code. We simply use Arize’s bulk\_log method, passing in features, predictions, actuals, and optional prediction timestamps, and just like that we have our model logged and ready to explore on the Arize platform. class Deployment:    def \_\_init\_\_(self, base\_directory, context):       model\_file = os.path.join(base\_directory, "tensorflow\_model.h5")        self.model = load\_model(model\_file)        self.arize = Client(organization\_key=os.environ.get('ARIZE\_ORGANIZATION\_KEY'), api\_key=os.environ.get('ARIZE\_API\_KEY'))   def request(self, data):        input\_data = pd.read\_csv(data\['data'\])        actuals = input\_data.pop('MPG')        prediction = self.model.predict(input\_data)        ########### ARIZE CODE HERE ###########        ids = pd.DataFrame(input\_data.index.values).applymap(str)       # OPTIONAL: Simulate predictions evenly distributed over 30 days by manually specifying prediction time       current\_time = datetime.datetime.now().timestamp()       earlier\_time = (datetime.datetime.now() - datetime.timedelta(days=30)).timestamp()       optional\_prediction\_timestamps = np.linspace(earlier\_time, current\_time, num=len(ids))       optional\_prediction\_timestamps = pd.Series(optional\_prediction\_timestamps.astype(int))        responses = self.arize.bulk\_log(           model\_id="arize-ubiops-tutorial",           model\_type=ModelTypes.NUMERIC,           model\_version="v1",           prediction\_ids= ids,           prediction\_labels=pd.DataFrame(prediction),           prediction\_timestamps=optional\_prediction\_timestamps,           actual\_labels=actuals,           features=input\_data)        #######################################        # Writing the prediction to a csv for further use        print('Writing prediction to csv')        pd.DataFrame(prediction).to\_csv('prediction.csv', header = \['MPG'\], index\_label= 'index')        return {            "prediction": 'prediction.csv',       } **2\. Example end result visualised in Arize**  #### Here’s an example of how Arize visualises model performance in production, with data coming in on a daily basis. The platform provides a snapshot of the overall health of a model, surfacing key metrics such as accuracy, false positive rate, recall, amongst others (see fig. 2). Moreover, the current performance distributions can be compared against training, validation or historical performance baselines (see fig. 3). Arize Performance Dashboard ![](https://arize.com/wp-content/uploads/2021/06/arize-dashboard.png) Figure 2: Arize performance dashboard Arize PSI Monitor ![](https://arize.com/wp-content/uploads/2021/06/arize-PSI-monitor.png) Figure 3: Arize PSI monitor example This example shows how one can simply deploy, in a fully scalable (containerised) environment, a TensorFlow model that is directly available for high frequency requests. In this case, anyone that likes to see what the expected MPG is of a car, can receive the results in a matter of seconds. This is ideal for example a webapp providing such a service. What’s more, with Arize’s monitoring functionality you can keep track of the model’s performance, automatically monitor it and conduct pre-launch validations to ensure a successful launch of your project.  Using the provided integration notebook you can deploy and monitor your own model quickly. The full notebook can be found [here](https://docs.arize.com/arize/integrations/integrations/ubiops) . If you have questions, remarks or suggestions, please don’t hesitate to contact Ubiops via their [Slack channel](https://join.slack.com/t/ubiops-community/shared_invite/zt-np02blts-5xyFK0azBOuhJzdRSYwM_w) or get in touch with Arize via their [Community on Slack](https://join.slack.com/t/arize-ai/shared_invite/zt-h3t7afis-5AuUGzDjTRpdDijiWv3jWA) .  Share * [![](https://arize.com/wp-content/themes/arize-2022/images/icon-tw-circle.svg)](https://twitter.com/intent/tweet?url=https%3A%2F%2Farize.com%2Fblog%2Farize-partners-with-ubiops%2F&text=Arize%20Partners%20with%20UbiOps%20to%20Accelerate%20Model%20Building%20&%20Deployment) * [![](https://arize.com/wp-content/themes/arize-2022/images/icon-fb-circle.svg)](https://www.facguide.com/sharer/sharer.php?u=https://arize.com/blog/arize-partners-with-ubiops/&picture=https://arize.com) * [![](https://arize.com/wp-content/themes/arize-2022/images/icon-in-circle.svg)](https://www.linkedin.com/shareArticle?mini=true&url=https%3A%2F%2Farize.com%2Fblog%2Farize-partners-with-ubiops%2F&title=Arize%20Partners%20with%20UbiOps%20to%20Accelerate%20Model%20Building%20&%20Deployment) #### Suggested reading ![](https://arize.com/wp-content/uploads/2025/11/microsoft-foundry-arize-ax.png) [Evaluating and Improving AI Agents at Scale with Microsoft Foundry](https://arize.com/blog/evaluating-and-improving-ai-agents-at-scale-with-microsoft-foundry/) ![](https://arize.com/wp-content/uploads/2025/10/llm-tracing-blog-cover.png) [Top LLM Tracing Tools](https://arize.com/blog/top-llm-tracing-tools/) Sign up for our monthly newsletter, The Evaluator. -------------------------------------------------- [Subscribe](https://arize.com/blog/arize-partners-with-ubiops/#blog-subscribe-modal) Subscribe to The Evaluator -------------------------- We’ll send you the latest news, expertise, and product updates from Arize. Your inbox is sacred, so we’ll only curate and send the best stuff. \*We’re committed to your privacy. Arize uses the information you provide to contact you about relevant content, products, and services. You may unsubscribe from these communications at any time. For more information, check out our [privacy policy](https://arize.com/privacy-policy) . --- # Arize Phoenix: Datasets - Arize AI ### Experiments Arize Phoenix: Datasets ======================= Datasets are a new core feature in Phoenix that live alongside your projects. They can be imported, exported, created, curated, manipulated, and viewed within the platform, and should make a few flows much easier: 👉 Fine-tuning – you can now create a dataset based on conditions in the UI, or by manually choosing examples, then export these into csv or jsonl formats ready-made for fine-tuning APIs. 👉 Experimentation – external datasets can be uploaded into Phoenix to serve as the test cases for experiments run in the platform (more on this on Day 3! For more details on using datasets see our documentation or example notebook! [📓 Docs](https://docs.arize.com/phoenix/datasets-and-experiments/overview-datasets) [👩‍🍳 Cookbook](https://colab.research.google.com/drive/1e4vZR5VPelXXYGtWfvM3CErPhItHAIp2?usp=sharing) [⭐️ Star Phoenix on GitHub](https://github.com/Arize-ai/phoenix) Subscribe to The Evaluator -------------------------- We’ll send you the latest news, expertise, and product updates from Arize. Your inbox is sacred, so we’ll only curate and send the best stuff. \*We’re committed to your privacy. Arize uses the information you provide to contact you about relevant content, products, and services. You may unsubscribe from these communications at any time. For more information, check out our [privacy policy](https://arize.com/privacy-policy) . --- # Arize AI Is Growing! - Arize AI ![](https://arize.com/wp-content/uploads/2021/06/krystal-headshot-e1624425208666-196x196.jpg "krystal headshot") [Krystal Kirkland](https://arize.com/author/krystal-kirkland/) Software Engineer Share * [![](https://arize.com/wp-content/themes/arize-2022/images/icon-tw-circle.svg)](https://twitter.com/intent/tweet?url=https%3A%2F%2Farize.com%2Fblog%2Farize-ai-is-growing%2F&text=Arize%20AI%20Is%20Growing!) * [![](https://arize.com/wp-content/themes/arize-2022/images/icon-fb-circle.svg)](https://www.facguide.com/sharer/sharer.php?u=https://arize.com/blog/arize-ai-is-growing/&picture=https://arize.com) * [![](https://arize.com/wp-content/themes/arize-2022/images/icon-in-circle.svg)](https://www.linkedin.com/shareArticle?mini=true&url=https%3A%2F%2Farize.com%2Fblog%2Farize-ai-is-growing%2F&title=Arize%20AI%20Is%20Growing!) ![](https://arize.com/wp-content/themes/arize-2022/images/icon-newsletter.svg) Subscribe to the Arize blog [Get the latest](https://arize.com/blog/arize-ai-is-growing/#blog-subscribe-modal) #### On this page #### Suggested reading ![](https://arize.com/wp-content/uploads/2025/11/microsoft-foundry-arize-ax.png) [Evaluating and Improving AI Agents at Scale with Microsoft Foundry](https://arize.com/blog/evaluating-and-improving-ai-agents-at-scale-with-microsoft-foundry/) ![](https://arize.com/wp-content/uploads/2025/10/llm-tracing-blog-cover.png) [Top LLM Tracing Tools](https://arize.com/blog/top-llm-tracing-tools/) ![](https://arize.com/wp-content/uploads/2021/06/Team-Series-3.png "Team Series 3") Arize AI Is Growing! ==================== Published Jun 17, 2021 * [Company](https://arize.com/blog/?cat=company) * [Uncategorized](https://arize.com/blog/?cat=uncategorized) ![](https://arize.com/wp-content/uploads/2021/06/krystal-headshot-e1624425208666-196x196.jpg "krystal headshot") #### [Krystal Kirkland](https://arize.com/author/krystal-kirkland/) ##### Software Engineer Arize AI has grown, and we’re thrilled to welcome our new team members to the Arize AI team! ![](https://miro.medium.com/max/1000/1*n-RG2peSKW0VqTww3wpTVg.jpeg) Andy Lu [Andy Lu](https://www.linkedin.com/in/theandylu/) -------------------------------------------------- _Senior Designer_ Andy joins the Design Team as a Senior Designer. He was previously a product designer at Chase Bank. “While the opportunity to help build the Arize platform from the ground up initially drew me to my position at Arize, Arize’s strong stance on diversity, fairness & ethics, and inclusion has fueled my passion for building a brighter future in the AI/ML space.” ![](https://miro.medium.com/max/1000/1*djZqR4Nq3imCWEMI8Cyptw.png) Harrison Chu [Harrison Chu](https://www.linkedin.com/in/hchu1/) --------------------------------------------------- _Senior Software Engineer_ Harrison joins the Engineering Team as a Senior Software Engineer. Previously, Harrison was an Engineering Manager at Lyft. “After years of working with ML-based teams across industries, I saw how often teams were unequipped to tackle some of the most complicated and annoying operational challenges of running models in production. I saw that this was true even for the most well-funded and well-staffed engineering organizations. These problems compelled me to join Arize where I can have a hand in solving this for teams of all sizes. Especially the small ones because I like the idea of arming the rebels.” ![](https://miro.medium.com/max/1000/1*sQeWWreTmVAJcwZvITtFGQ.jpeg) Krystal Kirkland [Krystal Kirkland](https://www.linkedin.com/in/krystal-kirkland/) ------------------------------------------------------------------ _Product Marketing Manager_ Krystal joins the marketing team as a Product Marketing Manager. She holds a bachelor’s degree in Sociology from UC Berkeley. “It was clear from the get-go that Arize has the relationship between technology and society front-of-mind. I’m excited to contribute to such a passionate team aimed at bettering the evermore complex sociotechnical world.” ![](https://miro.medium.com/max/1000/1*4SeG6RY9Fgmd2Yb6BOc98g.jpeg) David Monical [David Monical](https://www.linkedin.com/in/david-monical-8a14401b2/) ---------------------------------------------------------------------- _Application Engineer_ David joins the engineering team as an Application Engineer. “What originally drew me to Arize was scrolling through the website and wondering how on Earth I had got by without ML observability before. Having just graduated from UC Berkeley, I thought if my school project Jupyter Notebooks could benefit immensely from Arize, imagine how much use a full-scale MLOps team running thousands of models could get. Between that thought and the fun, comfortable, and inspiring culture I felt the very first time I met the team, I knew Arize was the place I wanted to be.” ![](https://miro.medium.com/max/1000/1*5sdypdkPXdhoq1RxzjTsNg.png) Eric Senzig [Eric Senzig](https://www.linkedin.com/in/ericsenzig/) ------------------------------------------------------- _Enterprise Sales Lead_ Eric joins the sales team as an Enterprise Sales Lead. He was previously on the sales teams at H20.ai and Qubole. “It is tough to beat the mix of people, technology, and market timing at Arize. I’m grateful to be a part of the team and I’m excited to help develop a path for ML observability.” ![](https://miro.medium.com/max/856/1*vcAdYQinSwsstuDubsIrXQ.jpeg) Francisco Castillo [Francisco Castillo](https://www.linkedin.com/in/f-castillo-carrasco/) ----------------------------------------------------------------------- _Data Scientist_ Francisco (aka Kiko) joins the data science team as a Data Scientist. He is currently a Ph.D. candidate in Applied Mathematics at Arizona State University. “As AI rapidly moves from a research field to the real world, it is paramount that we hold it accountable. I’m thrilled to join Arize AI because of its mission to help companies observe and explain their models, thereby facilitating to make systems more transparent and ethical.” Share * [![](https://arize.com/wp-content/themes/arize-2022/images/icon-tw-circle.svg)](https://twitter.com/intent/tweet?url=https%3A%2F%2Farize.com%2Fblog%2Farize-ai-is-growing%2F&text=Arize%20AI%20Is%20Growing!) * [![](https://arize.com/wp-content/themes/arize-2022/images/icon-fb-circle.svg)](https://www.facguide.com/sharer/sharer.php?u=https://arize.com/blog/arize-ai-is-growing/&picture=https://arize.com) * [![](https://arize.com/wp-content/themes/arize-2022/images/icon-in-circle.svg)](https://www.linkedin.com/shareArticle?mini=true&url=https%3A%2F%2Farize.com%2Fblog%2Farize-ai-is-growing%2F&title=Arize%20AI%20Is%20Growing!) #### Suggested reading ![](https://arize.com/wp-content/uploads/2025/11/microsoft-foundry-arize-ax.png) [Evaluating and Improving AI Agents at Scale with Microsoft Foundry](https://arize.com/blog/evaluating-and-improving-ai-agents-at-scale-with-microsoft-foundry/) ![](https://arize.com/wp-content/uploads/2025/10/llm-tracing-blog-cover.png) [Top LLM Tracing Tools](https://arize.com/blog/top-llm-tracing-tools/) Sign up for our monthly newsletter, The Evaluator. -------------------------------------------------- [Subscribe](https://arize.com/blog/arize-ai-is-growing/#blog-subscribe-modal) Subscribe to The Evaluator -------------------------- We’ll send you the latest news, expertise, and product updates from Arize. Your inbox is sacred, so we’ll only curate and send the best stuff. \*We’re committed to your privacy. Arize uses the information you provide to contact you about relevant content, products, and services. You may unsubscribe from these communications at any time. For more information, check out our [privacy policy](https://arize.com/privacy-policy) . --- # Arize AI Partners with Spell to Bring ML Observability to the Spell Platform - Arize AI ![](https://arize.com/wp-content/uploads/2021/06/krystal-headshot-e1624425208666-196x196.jpg "krystal headshot") [Krystal Kirkland](https://arize.com/author/krystal-kirkland/) Software Engineer Share * [![](https://arize.com/wp-content/themes/arize-2022/images/icon-tw-circle.svg)](https://twitter.com/intent/tweet?url=https%3A%2F%2Farize.com%2Fblog%2Farize-ai-partners-with-spell%2F&text=Arize%20AI%20Partners%20with%20Spell%20to%20Bring%20ML%20Observability%20to%20the%20Spell%20Platform) * [![](https://arize.com/wp-content/themes/arize-2022/images/icon-fb-circle.svg)](https://www.facguide.com/sharer/sharer.php?u=https://arize.com/blog/arize-ai-partners-with-spell/&picture=https://arize.com) * [![](https://arize.com/wp-content/themes/arize-2022/images/icon-in-circle.svg)](https://www.linkedin.com/shareArticle?mini=true&url=https%3A%2F%2Farize.com%2Fblog%2Farize-ai-partners-with-spell%2F&title=Arize%20AI%20Partners%20with%20Spell%20to%20Bring%20ML%20Observability%20to%20the%20Spell%20Platform) ![](https://arize.com/wp-content/themes/arize-2022/images/icon-newsletter.svg) Subscribe to the Arize blog [Get the latest](https://arize.com/blog/arize-ai-partners-with-spell/#blog-subscribe-modal) #### On this page #### Suggested reading ![](https://arize.com/wp-content/uploads/2025/11/microsoft-foundry-arize-ax.png) [Evaluating and Improving AI Agents at Scale with Microsoft Foundry](https://arize.com/blog/evaluating-and-improving-ai-agents-at-scale-with-microsoft-foundry/) ![](https://arize.com/wp-content/uploads/2025/10/llm-tracing-blog-cover.png) [Top LLM Tracing Tools](https://arize.com/blog/top-llm-tracing-tools/) ![](https://arize.com/wp-content/uploads/2021/06/Partnership-Announcement-2-1534x1120.png "Partnership Announcement 2") Arize AI Partners with Spell to Bring ML Observability to the Spell Platform ============================================================================ Published Feb 8, 2021 * [Company](https://arize.com/blog/?cat=company) * [ML Monitoring](https://arize.com/blog/?cat=ml-monitoring) * [ML Observability](https://arize.com/blog/?cat=ml-observability) * [MLOps](https://arize.com/blog/?cat=mlops) * [Uncategorized](https://arize.com/blog/?cat=uncategorized) ![](https://arize.com/wp-content/uploads/2021/06/krystal-headshot-e1624425208666-196x196.jpg "krystal headshot") #### [Krystal Kirkland](https://arize.com/author/krystal-kirkland/) ##### Software Engineer This week we’re announcing our new partnership with [](https://arize.com/) [Spell](https://spell.ml/) ! There is a vast difference between the offline environments where models are trained and production environments where they are served. This training/serving skew often leads to data science teams trying to troubleshoot their models performance once they are deployed. However, most machine learning teams have little to no telemetry about their models once they are deployed. **ML observability** helps teams easily transition from research to production — maintaining the results delivered, and helping teams troubleshoot problems quickly — without eating up Data Science cycles. The ability to explain, understand and get answers quickly builds a necessary trust between research teams and end users. Combining Spell model servers with Arize model observability lets you have the best of both worlds — easy-to-use autoscaling online model APIs, powerful model monitoring, explainability, and troubleshooting. If you already are using Spell, the integration with Arize for Model Observability is easy. In this blog post we will showcase deploying a [lightgbm churn prediction model](https://spell.ml/blog/churn-prediction-with-lightgbm-and-artificial-neural-X0a8XxIAAA_Nfdkk)  on Spell that’s tracked and monitored using Arize. Deploying a model server on Spell ================================= To begin, we’ll need to train and save a model on Spell. We can do so using the following spell run command, which uses a training script from the [spellml/examples GH repo](https://github.com/spellml/examples/tree/master/arize/train.py) : $spell run \\\\ --github-url \\\\ --machine-type cpu \\\\ --mount public/tutorial/churn\_data/:/mnt/churn\_prediction/ \\\\ --pip arize — pip lightgbm \\\\ --python arize/train.py#replace $RUN\_ID with the ID number of the run that just finished$spell model create churn-prediction runs/$RUN\_ID Next, we’ll need a model server script. This file will be used to serve the model and to log it to Arize. Here’s the one’s we’ll use: \` import os import uuid from asyncio import wrap\_futureimport numpy as np import lightgbm as lgbfrom spell.serving import BasePredictor from arize.api import Clientclass PythonPredictor(BasePredictor): def \_\_init\_\_(self): self.model = lgb.Booster( model\_file=”/model/churn\_model/lgb\_classifier.txt”) self.arize\_client = Client(organization\_key=os.environ\['\ ARIZE\_ORG\_KEY’\], api\_key=os.environ\[‘ARIZE\_API\_KEY’\]) self.model\_id = ‘churn-model’ self.model\_version = ‘0.0.1’async def predict(self, request): payload = request\[‘payload’\] #use np.round to squeeze to binary {0,1} results = list(np.round(self.model.predict(payload))) futures = \[\] for result in results: prediction\_id = str(uuid.uuid4()) future = self.arize\_client.log\_prediction( model\_id=self.model\_id, model\_version=self.model\_version, prediction\_id=prediction\_id, prediction\_label=bool(result) # {1,0} => {true,false}) future = wrap\_future(future) # SO#34376938 futures.append(future)for future in futures: await future status\_code = future.result().status\_code if status\_code != 200: raise IOError( f”Could not reach Arize! Got error code {status\_code}.” ) response = {‘result’: results} return response \` The PythonPredictor class inherits from Spell’s BasePredictor class, which expects two functions: an \_\_init\_\_, which runs once at server initialization time, and a predict, which runs at model serving time. In this example \_\_init\_\_ does two things: it loads the model artifact from disk, and it initializes the Arize Client we’ll use for observability logging. As a best practice, this client reads its authentication secrets (organization\_key and api\_key) from environment variables, which we’ll pass through to the server via Spell. The predict method is what actually handles request-response flow. Spell will unpack the payload of the POST request and pass it to the request parameter. We then generate a model prediction (self.model.predict(payload)) and return it to the caller (return response). However, before we return the response, we log the predicted value Arize first (makes a POST request to an Arize web endpoint) before returning to the caller: future = self.arize\_client.log\_prediction( model\_id=self.model\_id, model\_version=self.model\_version, prediction\_id=prediction\_id, prediction\_label=bool(result) # {1,0} => {true,false}) model\_id and model\_version identify this model to Arize (churn-model, version 0.0.1). prediction\_id is a unique identifier _for_ _this specific prediction_; the easiest way to create a [UUID](https://en.wikipedia.org/wiki/Universally_unique_identifier) , which is what we did here (using uuid from Python stdlib). The Arize API is asynchronous for performance, and returns a future as its result. In order to guarantee that the prediction actually gets logged, we need to ensure that this future resolves before the function exits and its contents gets garbage collected. To achieve this, we make the predict function asnyc, await every Arize request, and throw an error (which Spell will intercept and log) if the Spell server can’t reach the Arize server: for future in futures: await future status\_code = future.result().status\_code if status\_code != 200: raise IOError( f”Could not reach Arize! Got error code {status\_code}.”) The [demo repo](https://github.com/spellml/examples/tree/arize/)  also has a higher performance, fully async version of this script. All that’s left now is deploying the server: $ spell server serve \\\\ --node-group default \\\\ --min-pods 1 — max-pods 3 \\\\ --target-requests-per-second 100 \\\\ --pip lightgbm — pip arize \\\\ --env ARIZE\_ORG\_KEY=$ARIZE\_ORG\_KEY \\\\ --env ARIZE\_API\_KEY=$ARIZE\_API\_KEY \\\\churn-prediction:v1 serve\_sync.py # or serve\_async.py At this point this model server should be up and running: ![](https://miro.medium.com/max/1400/1*9kU0qvMIrvkziRbwnaYjIQ.png) You can test everything works by running the following curl command (replacing the variables with the ones appropriate for your Spell cluster instance): $ curl -X POST -d '@test\_payload.txt' \\\\, <.https://$REGION.$CLUSTER.spell.services/$ORGANIZATION/churn- prediction/predict> {"result":\[1.0\]}% ML Observability with Arize =========================== Once you’ve added this code snippet, your prediction events are logged to Arize and the platform discovers your model and sets up dashboards, monitors, and analytics for your predictions. Default dashboards are set up to highlight important evaluation metrics and data metrics. Operational metrics from the Spell platform can also be sent to the Arize platform. The dashboards are customizable for your specific custom model metrics. ![](https://miro.medium.com/max/1400/1*bK8jOj9zbnT7OONCaq18ng.png) The Arize platform can log inferences across the entire ML Workflow — training, validation, and production. The platform sets up default dashboards for quick model analysis, but also has powerful tools for troubleshooting and analyzing prediction slices. The platform surfaces what data caused poor performance so it can be used for testing and retraining. Monitoring your model with Arize ================================ Models on Arize can be set up with drift, performance, and data quality monitors. Here is an example of a drift monitor setup on the predictions of a model. The PSI monitor can indicate when the predictions are drifting and the model needs to be revisited. ![](https://miro.medium.com/max/1400/1*uL_eaMwhYbjl1xhh_rxmjg.png) Getting Access to Spell and Arize ================================= Through this partnership, the Spell and Arize platforms team up to make MLOps even easier. Spell users will have early access to Arize AI’s model observability platform. Arize users can leverage Spell as their powerful, iterative MLOps platform for building and managing machine learning projects. About Arize =========== Arize AI is a [Machine Learning Observabililty](https://arize.com/model-monitoring)  platform that helps ML practitioners successfully take models from research to production, with ease. Arize’s automated [model monitoring](https://arize.com/ml-monitoring/)  and analytics platform help ML teams quickly detect issues the moment they emerge, troubleshoot why they happened, and improve overall model performance. By connecting offline training and validation datasets to online production data in a central inference store, ML teams are able to streamline [model validation](https://arize.com/ml-model-failure-modes/) , [drift detection](https://arize.com/take-my-drift-away/) , [data quality checks](https://arize.com/data-quality-monitoring/) , and [model performance management](https://arize.com/monitor-your-model-in-production/) . Arize AI acts as the guardrail on deployed AI, providing transparency and introspection into historically black box systems to ensure more effective and [responsible AI](https://www.forbes.com/sites/aparnadhinakaran/?sh=5d7691024958) . To learn more about Arize or machine learning observability and monitoring, visit our [blog](https://arize.com/blog/)  and [resource hub](https://arize.com/resource-hub/) ! About Spell =========== Spell is the MLOps platform built to meet the unique challenges of operationalizing deep learning at scale. For engineers, it eliminates drudgery and enhances collaboration. For managers, it provides real-time project visibility and accountability. And for stakeholders, it reduces cost and shortens time to value. Spell DLOps is comprehensive and inclusive, meeting the needs of the engineer, the team, and the enterprise for effective development, deployment, and management of deep learning models. Spell operates on public, private, and hybrid clouds, or on dedicated on-premises compute infrastructure. It easily integrates with existing workflows, frameworks, infrastructure, and datastores. Spell doesn’t force users to learn new deep learning tools and technologies; it makes existing ones easier to use. To try the Spell platform, sign up here: [spell.ml/get-started/](http://spell.ml/get-started/) To request access to Arize, sign up here: [https://arize.com/sign-in/](https://arize.com/sign-in/) Share * [![](https://arize.com/wp-content/themes/arize-2022/images/icon-tw-circle.svg)](https://twitter.com/intent/tweet?url=https%3A%2F%2Farize.com%2Fblog%2Farize-ai-partners-with-spell%2F&text=Arize%20AI%20Partners%20with%20Spell%20to%20Bring%20ML%20Observability%20to%20the%20Spell%20Platform) * [![](https://arize.com/wp-content/themes/arize-2022/images/icon-fb-circle.svg)](https://www.facguide.com/sharer/sharer.php?u=https://arize.com/blog/arize-ai-partners-with-spell/&picture=https://arize.com) * [![](https://arize.com/wp-content/themes/arize-2022/images/icon-in-circle.svg)](https://www.linkedin.com/shareArticle?mini=true&url=https%3A%2F%2Farize.com%2Fblog%2Farize-ai-partners-with-spell%2F&title=Arize%20AI%20Partners%20with%20Spell%20to%20Bring%20ML%20Observability%20to%20the%20Spell%20Platform) #### Suggested reading ![](https://arize.com/wp-content/uploads/2025/11/microsoft-foundry-arize-ax.png) [Evaluating and Improving AI Agents at Scale with Microsoft Foundry](https://arize.com/blog/evaluating-and-improving-ai-agents-at-scale-with-microsoft-foundry/) ![](https://arize.com/wp-content/uploads/2025/10/llm-tracing-blog-cover.png) [Top LLM Tracing Tools](https://arize.com/blog/top-llm-tracing-tools/) Sign up for our monthly newsletter, The Evaluator. -------------------------------------------------- [Subscribe](https://arize.com/blog/arize-ai-partners-with-spell/#blog-subscribe-modal) Subscribe to The Evaluator -------------------------- We’ll send you the latest news, expertise, and product updates from Arize. Your inbox is sacred, so we’ll only curate and send the best stuff. \*We’re committed to your privacy. Arize uses the information you provide to contact you about relevant content, products, and services. You may unsubscribe from these communications at any time. For more information, check out our [privacy policy](https://arize.com/privacy-policy) . --- # Arize - Quickstart Guide - Arize AI ![marquee background](https://arize.com/wp-content/uploads/2022/08/Solutions-marquee-bg@2x-scaled.jpg "marquee background") Quickstart guide ================ Overview -------- If you’re new to Arize, this document will be a great reference for you as you get started. Feel free to skip sections that don’t apply and go straight to the ones that pertain to you. We hope you enjoy observing your models and find cool insights with Arize, if you get stuck or have any questions, feel free to reach out to us on our [community slack channel](http://arize-ai.slack.com/) in #arize-platform-support for assistance. Objective --------- This tutorial will familiarize you with the necessary steps to start using Arize. Upon completing these steps, you will have set up your first model and gained familiarity with Arize’s core capabilities. ##### [**Step 1: Sending data**](https://arize.com/resource/arize-quickstart-guide/#Step-1) * 1.1 Identify Data to Send  * 1.2 Send Your Data  * 1.3 Verify Your Data * 1.4 Verify You Data (cont.)  ##### [**Step 2: Configure your model**](https://arize.com/resource/arize-quickstart-guide/#Step-2) * 2.1 Set Your Model Baseline * 2.2 Set Your Default Performance Metric ##### [**Step 3. Find Insights**](https://arize.com/resource/arize-quickstart-guide/#Step-3) * 3.1 Inspect Your Model Performance Over Time * 3.2 Explore Performance Tracing * 3.3 Explore Drift Troubleshooting ##### [**Step 4: Set Up Monitors**](https://arize.com/resource/arize-quickstart-guide/#Step-4) * 4.1 Bulk Monitor Creation    * 4.2 Custom Monitors ##### [**Step 5: Set Up a Dashboard**](https://arize.com/resource/arize-quickstart-guide/#Step-5) * 5.1 Gain Insights Step 1: Sending Data ==================== ### 1.1 Identify Data to Send It’s important to note that Arize does not ingest your model itself; rather, it ingests the data surrounding your model. Arize can ingest training, validation, and production data, and you can send in some or all of these datasets depending on your use-case. ![Identify Data to Send ](https://arize.com/wp-content/uploads/2022/09/Identify-Data-to-Send-@2x-2048x1066.png) You will need to define the data columns within your dataset so Arize knows how to interpret each column (i.e. arize differentiates between predictions and features). The following model data can be sent to Arize:  _(Please refer to our_ [_model schema documentation_](https://docs.arize.com/arize/data-ingestion/model-schema) _to learn about the data types)_ * Model Name * Model Version * Model Type * Environment * Prediction ID * Timestamp * Features * Feature Importance * Tags * Prediction Score / Prediction Label / Prediction Value This table helps you map your **data columns** to the **data schema** expected by Arize. ![Data columns - data schema](https://arize.com/wp-content/uploads/2022/09/data_columns-data_schema.png)   * * * ### 1.2 Send Your Data To Arize Send your data to Arize using the SDK or File importer.  #### SDKs The first method of getting data in is through our SDKs. You can think of these as wrappers around our REST API, where you are pushing data via a HTTP POST. Whether you want to send data in batch or real time, both architectures are supported. #### File Importer The second method is for those teams that persist inferences in a data lake style architecture. If you store your data in a cloud bucket, you can set up a file import job (which is probably the preferred choice at Spotify) to ingest data to Arize.  ⚡ **Pro Tip:** No matter what method you choose, continuously send your model data into the platform through a recurring data ingestion pipeline. This way, Arize receives your most up to date model data as it becomes available for real-time feedback.   * * * ### 1.3 Verify Your Data in Arize Once you send your data into the platform, verify that your data was correctly received by navigating to the ‘Data Ingestion’ tab. From there, cross check your expected values with what’s showing on the platform. ![Verify Your Data in Arize](https://arize.com/wp-content/uploads/2022/09/Verify-Your-Data-in-Arize-2048x1132.png) **Coffee break!** ☕ Once you confirm Arize received your data, sit back and relax! Your data will take a bit to be processed and indexed by Arizer before it appears everywhere in the platform. You can return after a few minutes to continue working through this tutorial. _During this time, Arize indexes and processes the data before it will show up anywhere else in the platform._   * * * ### 1.4 Verify Your Data (cont.) After about 10 minutes, verify your data within the ‘Datasets’ tab.  ⚡ **Pro tip:** We recommend uploading a small sample of data so you can verify the data looks correct in the platform before uploading larger datasets. #### Keep An Eye Out Here are some things to ask yourself as you verify your data: * **Features and Tags**: _What features and tags have been ingested?_ * **Data Type**: _Do the data types (numeric and categorical) and data ranges look as expected?_  * **Missing Values**: _Are there any missing values that are not expected?_ * **Timestamp**: Does the prediction timestamp look correct? If the data does not look right, it’s likely an issue with the data schema you sent to arize. You can jump into our [model schema](https://docs.arize.com/arize/data-ingestion/model-schema) documentation to easily resolve the issues.  #### Meaningful Insights While looking at the ‘Datasets’ tab, you may have noticed a few things that can help you set up monitors and reduce time to resolution. Some context clues include:  * Unexpected missing values in your data * Keep this in mind when we set up data quality monitors * Specific expectations around missing data * Set up alerts if these expectations are violated. Step 2. Configure Your Model in Arize ===================================== Once you set up your data ingestion pipeline, you’ll need to choose a **baseline** and configure your model’s **performance metric**. Both these things are set up in the ‘Config’ tab within your model.   Arize helps you surface, resolve, and improve your models. This takes the form of setting monitors (performance, drift, and data quality), alerting you when your models trigger, and an easy root cause analysis workflow with performance tracing. The first actionable step in the platform to achieve this is to set a baseline and pick your performance metric.  ### 2.1 Set Your Model Baseline A model baseline is a reference data set of either training, validation, or prior time periods in production. A baseline is a dataset used to compare against your current data. Once you set a baseline, the Arize platform can automatically detect drift, data quality issues, and anomalous performance degradations. Your model is preset with a baseline defined by a moving time range of your model’s production data for 2 weeks delayed by 3 days. However, you can set a custom model baseline by navigating to the “Config” tab. Learn when to choose a different baseline [here](https://docs.arize.com/arize/product-guides/baselines#choosing-a-baseline) . ![Set up a baseline](https://arize.com/wp-content/uploads/2022/09/Set-up-a-baseline.png) ### 2.2 Set Your Default Model Performance Metric Performance metrics compare how well your model should behave with how it’s actually behaving, and helps break down poor performing areas of your model for an in-depth understanding of your model’s behavior.  Under the ‘Model Baseline’ card in the ‘Config’ tab, navigate to the ‘Performance Configs’ card to choose a default performance metric for your model.  Typically, you will want the default metric to be the metric you used when training your model. Arize offers many common model performance metrics to choose from, and your default metric manifests in the performance over time chart and your performance monitor. ![Performance Configs](https://arize.com/wp-content/uploads/2022/09/Performance-Configs.jpg) * * * Step 3. Find Insights In Your Model =================================== Once you’ve sent data into the Arize platform, set up a model baseline, and configured your default performance metric – gain immediate model insights with Performance Tracing and the Drift Tab.  3.1 Inspect Your Model Performance Over Time  --------------------------------------------- The Model Overview page gives you key model health metrics and visualizes your performance over time for at-a-glance observability. This will be the first page you land on whenever you click on a model from the home page, so get familiar with key metrics that matter to you!  Navigate to the ‘Overview’ tab to see your model performance metric plotted overtime overlaid with a timeseries chart of your prediction volume.  **Pro Tip:** Look for trends in your performance metric chart over time. This can help you better understand problematic areas for further analysis. ![Performance Over Time](https://arize.com/wp-content/uploads/2022/09/Performance-Over-Time.png) ### 3.2 Explore Arize Performance Tracing Performance Tracing gives you a wealth of information to resolve and improve your models. From surfacing your worst performing slices to visualizing your performance breakdown by feature and tags, this functionality in the platform helps you easily uncover areas to retrain or rebuild.  From the ‘Overview’ tab, navigate to the ‘Performance Tracing’ tab to uncover insights on your mode’s performance:  * Visualize model performance over time  * Compare model performance on multiple datasets * Break down model performance across different cohorts within feature and tag data **Pro tip:** We use ‘Cohort’ and ‘Slice’ interchangeably. They are just segments within features and tags data.  Performance tracing is a highly flexible feature that can help you drill down into specific cohorts of your model.  Some useful performance workflows include: * Comparing your production performance against your training performance * Comparing training/validation versions against each other * Determining which slices are affecting performance the most _\* You don’t have to worry about all of these workflows now, just note that they’re there once you need them in the future. We’re also happy to help you navigate the best way to use performance tracing for your use case, just reach out to us on slack!_ ![Fraud use case](https://arize.com/wp-content/uploads/2022/09/Fraud-use-case.png) **Pro tip**: If you notice cohorts where your model performance is unusually high or low – this might indicate:  * An area to continually monitor (more on monitoring in the next step)  * A possible area to retrain your model with additional data improvements  ### 3.3 Explore Arize Drift Troubleshooting When ground truths are not present or come at a severe lag time, drift is often a great proxy metric to assess risk to a model. Not only can we look at drift at the model output level, but we can also break down drift at the feature level. In the Arize platform, we also rank up the most drifting features by weighting the feature drift, by feature importance values.  **Pro Tip:** ‘Ground truth’ and ‘actuals’ can be used interchangeably.  Navigate to the ‘Drift’ tab to visualize PSI plotted for your model performance. Use the Arize Drift tab to discover any drift in your data.  **![Prediction drift over time](https://arize.com/wp-content/uploads/2022/09/Prediction-drift-over-time.png)** **Pro Tip:** Our visualizations help you easily identify where drift occurs. Click on a point in the graph in red, and scroll down to surface a distribution comparison for that point. From there, scroll down to the ‘Feature Drift’ card to uncover which features affect your model the most.  **Pro Tip:** If you have actuals in your data, performance tracing will be the most helpful guide. If you do not have actuals, use drift monitoring as a proxy for detecting performance degradation in your data. * * * Step 4. Set up Monitors ======================= Now that you’ve taken a few notes down on some areas within your model you want to keep an eye on, and are familiar with how Arize can help you get to the root cause of your problems, let’s set up the way we catch your problems in the first place!  Even if your model performed as expected last week, yesterday, and today – monitors guarantee your model continues to predict as expected by quickly and automatically alerting you when something goes wrong. This way, you don’t have to second guess that the ball might drop any second.  Arize has three different categories of monitors.   ### **Performance Monitors**  Performance monitors allow you to troubleshoot your models performance with granularity down to the hourly level, enabling a deep understanding of your model’s problematic areas. If you receive ground truth, performance monitoring will be your bread and butter monitor to better understand how to resolve your model issues at a glance.  _Docs_  ### **Drift Monitors** Drift monitors detect changes in data distributions. Drift can occur in both your features and in your predictions, so it’s important to account for both when analyzing how drift impacts your model. As we mentioned before, drift monitors act as a proxy metric for performance when you don’t have ground truth.  **Pro tip:** In most cases, use [PSI](https://docs.arize.com/arize/glossary/model-metric-definitions#population-stability-index-psi) as your metric to measure drift, but you can also use [KL Divergence](https://docs.arize.com/arize/glossary/model-metric-definitions#docs-internal-guid-44a01e33-7fff-6353-c3e2-e79a7789042e) in the case that your distributions have large variance, or [JS Distance](https://docs.arize.com/arize/glossary/model-metric-definitions#docs-internal-guid-44a01e33-7fff-6353-c3e2-e79a7789042e-1) if your distributions have low variance.  __Docs_ _\*As always, feel free reach out to us on our_ [_community slack channel_](http://arize-ai.slack.com/) _in_ _#arize-platform-support_ _if you need help picking a drift metric._  ### **Data Quality Monitors**  Very often, bad data can be the sneaky culprate to your ML model woes. Data quality monitors ensure clean and accurate data is sourced to your model. We do this by monitoring changes in cardinality and other data quality elements to make sure your data conforms with what is expected. **Pro tip:** Remember the ‘Model Overview’ page? Return to that page for at-a-glance data quality metrics.  __Docs_ Step 5. Set up a Dashboard ========================== Now that you have a general understanding of how your model behaves, what to look out for, and Arize’s core functionalities – aggregate your key metrics and visualizations with dashboards.  **Pro Tip:** Use dashboards to answer specific questions you may have about your model. Dashboards are great assets to share across your team since you can have multiple models, metrics, and visualizations all in one place.  Create dashboards from scratch or with one of our templates. To create a new Dashboard, click the ‘Dashboards’ tab and select ‘Create Dashboard’ then click ‘Create blank dashboard’. From there, you can select between three widget types: Time Series, Distribution, and Statistic. ![widget types - Arize](https://arize.com/wp-content/uploads/2022/09/Widget-types-Arize.png) ### Time Series  Use the time series widget to graph key metrics (evaluation and data) over time to help you understand your model’s behavior.  [https://docs.arize.com/arize/product-guides/dashboards/widgets/timeseries-widgets](https://docs.arize.com/arize/product-guides/dashboards/widgets/timeseries-widgets) ### Distribution  Choose the distribution widget to visualize the distribution of any feature, prediction, and actuals, and compare distributions against each other.  [https://docs.arize.com/arize/product-guides/dashboards/widgets/distribution-widgets](https://docs.arize.com/arize/product-guides/dashboards/widgets/distribution-widgets) ### Statistic  Select the statistic widget to represent an aggregate of the metric you picked. The value shown represents what you would get if you summed up all the time series data points. [https://docs.arize.com/arize/product-guides/dashboards/widgets/statistic-widgets](https://docs.arize.com/arize/product-guides/dashboards/widgets/statistic-widgets) ### **5.1 Gain Insights** Once you create the dashboard, you can filter your dashboard to analyze a specific cohort for an even more granular approach. To do this, select the model you want to filter by under your dashboard name, and pick between all your model versions and model dimensions.  Once you’ve identified areas to fix or improve, export your data via email to retrain, rebuild, or start anew. Cick on the ellipses icon in the top right corner of each widget and click on ‘Export data’. From there, an email will be sent to your user email with a link to your data. Step 6: Continue Exploring Arize ================================ Congrats on completing this tutorial! We hope you found some great model insights along the way.  This tutorial is just scratching the surface of our platform’s capabilities, continue to explore the platform by referring to our documentation: [https://docs.arize.com/arize/](https://docs.arize.com/arize/) Please refer to the FAQ sections to search for common questions: * [https://docs.arize.com/arize/data-ingestion/data-ingestion](https://docs.arize.com/arize/data-ingestion/data-ingestion)   * [https://docs.arize.com/arize/product-guides/product](https://docs.arize.com/arize/product-guides/product) **Pro Tip:** We’re always here to help! Work with your Arize representatives directly if you have any questions along the way, or have feedback to share.  We love collaborating with our customers! <3 Subscribe to The Evaluator -------------------------- We’ll send you the latest news, expertise, and product updates from Arize. Your inbox is sacred, so we’ll only curate and send the best stuff. \*We’re committed to your privacy. Arize uses the information you provide to contact you about relevant content, products, and services. You may unsubscribe from these communications at any time. For more information, check out our [privacy policy](https://arize.com/privacy-policy) . --- # Arize AI Selected For insideBIGDATA's Impact 50 List - Arize AI ![](https://arize.com/wp-content/uploads/2021/06/krystal-headshot-e1624425208666-196x196.jpg "krystal headshot") [Krystal Kirkland](https://arize.com/author/krystal-kirkland/) Software Engineer Share * [![](https://arize.com/wp-content/themes/arize-2022/images/icon-tw-circle.svg)](https://twitter.com/intent/tweet?url=https%3A%2F%2Farize.com%2Fblog%2Farize-ai-selected-for-insidebigdatas-impact-50-list%2F&text=Arize%20AI%20Selected%20For%20insideBIGDATA%E2%80%99s%20Impact%2050%20List) * [![](https://arize.com/wp-content/themes/arize-2022/images/icon-fb-circle.svg)](https://www.facguide.com/sharer/sharer.php?u=https://arize.com/blog/arize-ai-selected-for-insidebigdatas-impact-50-list/&picture=https://arize.com) * [![](https://arize.com/wp-content/themes/arize-2022/images/icon-in-circle.svg)](https://www.linkedin.com/shareArticle?mini=true&url=https%3A%2F%2Farize.com%2Fblog%2Farize-ai-selected-for-insidebigdatas-impact-50-list%2F&title=Arize%20AI%20Selected%20For%20insideBIGDATA%E2%80%99s%20Impact%2050%20List) ![](https://arize.com/wp-content/themes/arize-2022/images/icon-newsletter.svg) Subscribe to the Arize blog [Get the latest](https://arize.com/blog/arize-ai-selected-for-insidebigdatas-impact-50-list/#blog-subscribe-modal) #### On this page #### Suggested reading ![](https://arize.com/wp-content/uploads/2025/11/microsoft-foundry-arize-ax.png) [Evaluating and Improving AI Agents at Scale with Microsoft Foundry](https://arize.com/blog/evaluating-and-improving-ai-agents-at-scale-with-microsoft-foundry/) ![](https://arize.com/wp-content/uploads/2025/10/llm-tracing-blog-cover.png) [Top LLM Tracing Tools](https://arize.com/blog/top-llm-tracing-tools/) ![](https://arize.com/wp-content/uploads/2020/10/AdobeStock_96089349-scaled.jpeg "Big idea isometric flat vector concept. Mans hands take a trophy cup that looks like a lightbulb on pedestal.") Arize AI Selected For insideBIGDATA’s Impact 50 List ==================================================== Published Oct 28, 2020 * [Company](https://arize.com/blog/?cat=company) * [Uncategorized](https://arize.com/blog/?cat=uncategorized) ![](https://arize.com/wp-content/uploads/2021/06/krystal-headshot-e1624425208666-196x196.jpg "krystal headshot") #### [Krystal Kirkland](https://arize.com/author/krystal-kirkland/) ##### Software Engineer BERKELEY, Calif., Oct. 28, 2020 /PRNewswire/ — [Arize AI](https://c212.net/c/link/?t=0&l=en&o=2963991-1&h=325218129&u=http%3A%2F%2Farize.com%2F%3Futm_source%3Dprsnews%26utm_medium%3Dtext%26utm_campaign%3DPrsnews1&a=Arize+AI) , We’re excited to announce that Arize AI is selected for [insideBigData’s Impact 50 list in Q4 2020](https://c212.net/c/link/?t=0&l=en&o=2963991-1&h=3891321908&u=https%3A%2F%2Finsidebigdata.com%2F2020%2F10%2F13%2Fthe-insidebigdata-impact-50-list-for-q4-2020%2F&a=insideBigData%27s+Impact+50+list+in+Q4+2020) . Companies on the list exhibit technology leadership, strength of offering, proven innovation, positivity of message, quality perception in the enterprise, intensity and frequency of social media buzz, high profile of members of the C-suite, and so much more! The team at insideBIGDATA is deeply entrenched in following the big data ecosystem of companies from around the globe. We’re in close contact with most of the firms making waves in the technology areas of big data, data science, machine learning, AI and deep learning. Our in-box is filled each day with new announcements, commentaries, and insights about what’s driving the success of our industry so we’re in a unique position to publish our quarterly **IMPACT 50 List** of the most important movers and shakers in our industry. These companies have proven their relevance by the way they’re impacting the enterprise through leading edge products and services. We’re happy to publish this evolving list of the industry’s most impactful companies! “We are thrilled to be recognized in the prestigious insideBIGDATA Impact50 List for Q4 2020. Arize AI is the leading ML Observability platform in the market designed to troubleshoot, monitor and explain AI deployed in the real world. With Arize AI, Data Scientists and Machine Learning engineers are able to deploy their models with confidence, creating a more transparent and trustworthy future with AI”  — Jason Lopatecki, CEO of Arize AI **About Arize AI** Arize AI was founded by leaders in the Machine Learning (ML) Infrastructure and analytics space to bring better visibility and performance management over AI. Arize AI built the first ML Observability platform to help make machine learning models work in production. We provide a real time platform to monitor, explain and troubleshoot model/data issues, as models move from research to real world. Share * [![](https://arize.com/wp-content/themes/arize-2022/images/icon-tw-circle.svg)](https://twitter.com/intent/tweet?url=https%3A%2F%2Farize.com%2Fblog%2Farize-ai-selected-for-insidebigdatas-impact-50-list%2F&text=Arize%20AI%20Selected%20For%20insideBIGDATA%E2%80%99s%20Impact%2050%20List) * [![](https://arize.com/wp-content/themes/arize-2022/images/icon-fb-circle.svg)](https://www.facguide.com/sharer/sharer.php?u=https://arize.com/blog/arize-ai-selected-for-insidebigdatas-impact-50-list/&picture=https://arize.com) * [![](https://arize.com/wp-content/themes/arize-2022/images/icon-in-circle.svg)](https://www.linkedin.com/shareArticle?mini=true&url=https%3A%2F%2Farize.com%2Fblog%2Farize-ai-selected-for-insidebigdatas-impact-50-list%2F&title=Arize%20AI%20Selected%20For%20insideBIGDATA%E2%80%99s%20Impact%2050%20List) #### Suggested reading ![](https://arize.com/wp-content/uploads/2025/11/microsoft-foundry-arize-ax.png) [Evaluating and Improving AI Agents at Scale with Microsoft Foundry](https://arize.com/blog/evaluating-and-improving-ai-agents-at-scale-with-microsoft-foundry/) ![](https://arize.com/wp-content/uploads/2025/10/llm-tracing-blog-cover.png) [Top LLM Tracing Tools](https://arize.com/blog/top-llm-tracing-tools/) Sign up for our monthly newsletter, The Evaluator. -------------------------------------------------- [Subscribe](https://arize.com/blog/arize-ai-selected-for-insidebigdatas-impact-50-list/#blog-subscribe-modal) Subscribe to The Evaluator -------------------------- We’ll send you the latest news, expertise, and product updates from Arize. Your inbox is sacred, so we’ll only curate and send the best stuff. \*We’re committed to your privacy. Arize uses the information you provide to contact you about relevant content, products, and services. You may unsubscribe from these communications at any time. For more information, check out our [privacy policy](https://arize.com/privacy-policy) . --- # Hugging Face + Arize: Partnership and Code Example - Arize AI ![](https://arize.com/wp-content/uploads/2022/06/francisco-castillo-carrasco-196x196.jpg "francisco-castillo-carrasco") [Francisco Castillo](https://arize.com/author/francisco-castillo/) Data Scientist / Software Engineer Share * [![](https://arize.com/wp-content/themes/arize-2022/images/icon-tw-circle.svg)](https://twitter.com/intent/tweet?url=https%3A%2F%2Farize.com%2Fblog%2Farize-hugging-face%2F&text=Hugging%20Face%20+%20Arize:%20Partnership%20and%20Code%20Example) * [![](https://arize.com/wp-content/themes/arize-2022/images/icon-fb-circle.svg)](https://www.facguide.com/sharer/sharer.php?u=https://arize.com/blog/arize-hugging-face/&picture=https://arize.com) * [![](https://arize.com/wp-content/themes/arize-2022/images/icon-in-circle.svg)](https://www.linkedin.com/shareArticle?mini=true&url=https%3A%2F%2Farize.com%2Fblog%2Farize-hugging-face%2F&title=Hugging%20Face%20+%20Arize:%20Partnership%20and%20Code%20Example) ![](https://arize.com/wp-content/themes/arize-2022/images/icon-newsletter.svg) Subscribe to the Arize blog [Get the latest](https://arize.com/blog/arize-hugging-face/#blog-subscribe-modal) #### On this page #### Suggested reading ![azure native llm observability tracing evaluation with arize](https://arize.com/wp-content/uploads/2025/05/Arize-x-Azure-1.jpg) [Arize AI Now Generally Available As Part of Azure Native Integrations](https://arize.com/blog/arize-ai-now-generally-available-as-part-of-azure-native-integrations/) ![](https://arize.com/wp-content/uploads/2025/05/Arize-x-Nvidia.jpg) [Arize AI Accelerates Enterprise AI Adoption On-Premises With NVIDIA](https://arize.com/blog/arize-ai-accelerates-enterprise-ai-adoption-on-premises-with-nvidia/) ![Hugging Face and Arize Partnership and Integration Colab](https://arize.com/wp-content/uploads/2022/08/Arize-Hugging_Face-blog_cover-1.jpg "Arize-Hugging_Face-blog_cover (1)") Hugging Face + Arize: Partnership and Code Example ================================================== Published Dec 22, 2022 * [Integrations](https://arize.com/blog/?cat=integrations) ![](https://arize.com/wp-content/uploads/2022/06/francisco-castillo-carrasco-196x196.jpg "francisco-castillo-carrasco") #### [Francisco Castillo](https://arize.com/author/francisco-castillo/) ##### Data Scientist / Software Engineer _This article was written in collaboration with [Amit Goren](https://www.linkedin.com/in/amit-goren/) , Senior Product Marketing Manager at Arize_ We’re excited to share that Arize AI and Hugging Face are partnering to help organizations train unstructured models and monitor and troubleshoot those models in production, lowering costs and maximizing performance. **_Want to dive right in? Sign up for your_ [_free Arize account_](https://arize.com/join) _and check out our Hugging Face_ [_colabs_](https://docs.arize.com/arize/examples/embedding-examples-nlp) _._**  Tools That Paved the Way ------------------------ The _transformer_ architecture, first introduced in 2017 in the paper “[Attention Is All You Need](https://arxiv.org/pdf/1706.03762.pdf) ,” has taken the natural language processing (NLP) field by storm and supplanted many previous architectures. It is so good at capturing patterns in long sequences of data that it is being used beyond NLP, in both computer vision and reinforcement learning. These transformer models are hungry for huge amounts of data – so much that their use would likely be prohibited from individuals with domestic hardware/machines. Thanks to **transfer learning**, we are able to download a _pre-trained_ model trained on a generic dataset on a generic task. All that is left to do is to _fine-tune_ that model with your specific dataset to perform a specific task. Transfer learning allows almost anyone to obtain SOA results on their specific problem. If only the AI community had an ecosystem that allowed for collaboration… Enter Hugging Face! Transformer architecture and transfer learning have made it possible for the AI community to focus on a consistent set of tools to achieve state of the art results, and Hugging Face has positioned itself as the center of that ecosystem, invaluable for the community. Techniques for visualizing embeddings have also come a long way in the past few decades, with new algorithms made possible by the successful combination of mathematics, computer science and machine learning. The evolution from [SNE to t-SNE and UMAP](https://arize.com/blog/t-sne-vs-umap/) opens up new possibilities for data scientists and machine learning engineers to better understand their data and troubleshoot models. When it comes to understanding the underlying structure of the data your model is dealing with as well as how your model is interpreting and acting on that structure, [neighbor graph algorithms such as UMAP](https://towardsdatascience.com/visualizing-your-embeddings-4c79332581a9) are great tools for the AI community. Arize allows its users to observe this topological structure on the fly using their interactive UMAP implementation with both 2D and 3D views. Teams can quickly visualize their high dimensional data in a low dimensional space to isolate new or emerging patterns, underlying data changes, and data quality issues. What Is Hugging Face? --------------------- Hugging Face is on a mission to democratize state-of-the-art machine learning. The Hugging Face Hub makes the latest innovations coming from the global AI community accessible and easy to use. With a community-driven Hub, Hugging Face provides model implementations through an open-source library and model files, also known as checkpoints. In the Hub, ML teams can easily find the most optimal pre-trained or fine-tuned model to solve their business needs. Similarly, teams can find or contribute datasets based on their use case. The Hugging Face Hub represents the global contribution of thousands of open source contributors who have provided new changes, features, model architectures, and more. In addition to democratizing AI through a community-driven, open sourced hub, Hugging Face is removing the barriers of cost and time when it comes to training deep learning models. Building tools for Transfer Learning, Hugging Face Transformers provides APIs to easily download and fine-tune state-of-the-art pre-trained models, reducing compute costs and time from training a model from scratch. Hugging Face also offers a no-code solution, AutoTrain, to fine tune models on a specific dataset. Users just need to upload a dataset, and they will get state-of-the-art models back that are already fine-tuned, evaluated, and deployed. Lastly, Hugging Face offers an Inference API, helping teams improve and iterate on their models. What is Arize? -------------- Arize is an [ML observability platform](https://arize.com/) that enables teams to log models with both structured and unstructured data to detect, root cause, and resolve model performance issues faster. Tracing a model issue through the data it is built and acts upon is a time-consuming feat. With Arize’s purpose-built workflows for root cause analysis, teams can reduce time-to-resolution for even the most complex models. With tools such as automated monitors for drift, data quality, and performance, bias tracing to root out algorithmic bias, and powerful dashboards, teams can quickly catch model and data issues, diagnose the root cause, and continuously improve performance for their products and business. Arize’s latest release, includes the support of embeddings to monitor and troubleshoot unstructured data models. By monitoring [embeddings](https://arize.com/blog/getting-started-with-embeddings-is-easier-than-you-think/) of their unstructured data, teams can proactively identify when their data is drifting, and troubleshoot using Arize’s interactive UMAP visualization to identify new patterns, detect data quality issues, or export segments for high-value labeling. Challenges with NLP Models -------------------------- ### Challenge 1: The Bias Problem / Our Responsibility / Large Language Models Dangers While public access to **large language models** (LLMs) is at the core of the democratization of AI, these models don’t come without possible dangers. If misused, or used without human supervision, LLMs can operate with harmful bias issues. These problems have been [clearly](https://dl.acm.org/doi/abs/10.1145/3461702.3462624) [documented](https://arxiv.org/abs/2004.09456) in the [literature](https://proceedings.neurips.cc/paper/2020/hash/92650b2e92217715fe312e6fa7b90d82-Abstract.html) over the past several years. Finding solutions to these issues is particularly difficult when dealing with LLMs. Currently, the community does not have a full comprehensive solution to this critical problem, nor help to mitigate the potential harm that could arise. ### Challenge 2: Monitoring / Data Patterns / Data Quality According to [multiple estimates](https://mitsloan.mit.edu/ideas-made-to-matter/tapping-power-unstructured-data) , 80% of data generated is unstructured images, text, or audio. Despite this, ML teams spend the most time and money training deep learning models and lack the tools to monitor and troubleshoot them in production. Unfortunately, ML teams working with unstructured data end up shipping models blind as a result. Arize helps lower ML teams’ cost and time training, monitoring, and troubleshooting unstructured data models. ### Solution: Improving the Unstructured Data Workflow with Hugging Face and Arize Arize and Hugging Face tackle these problems head on and are committed to making sure the whole AI chain is transparent in its design with insight and monitoring of production data, ensuring that AI is never blind. ![great ai responsibility meme](https://arize.com/wp-content/uploads/2022/08/great-responsibility.png) Starting with the dataset, thanks to the Hugging Face Hub everything is public and transparent. In the _Dataset Card_ you can see where the data is sourced from and can check the dataset quality with the respect to metrics that measure bias risk. When it comes to models, it is key that their architecture is open sourced in the Hugging Face Hub. But, once we know their architecture, can we know how they are performing? Can we observe how models understand the inputs? Arize can help teams observe model performance in production. Arize measures drift by comparing euclidean distance of production data to a baseline and alerts ML teams that there may be a new pattern, an underlying data change, or data quality issue. Troubleshooting unstructured models is simple with Arize’s interactive UMAP implementation. The 2D and 3D views enable teams to easily visualize their high dimensional data in a low dimensional space. This embedding visualization helps ML teams understand the topological structure of their data and how their model is understanding that structure to make decisions. It can help you identify human errors on construction of the training data, which once fixed can improve your model without touching its architecture. Code Example: Obtain Embeddings From a Transformer Model -------------------------------------------------------- There is not a one-size-fits-all approach for computing embeddings. Depending on the problem at hand and the architecture of the model, you may choose to compute your embeddings in different ways and compare them to see which version of the embedding is best for you and your problem. In this section, we will go through how to obtain one embedding vector representing a sentence. At the end, we will put it all together so we can obtain embedding vectors for the entire dataset. Embeddings are, in essence, a dense vector representation of the inputs made by our model. Thus, we will need to run inputs through our model and obtain outputs, from which we will extract embedding vectors. For this extraction to be possible, these outputs should contain, in addition to the classification logits, the activation values of the hidden state layers. Specifically, the vector components are obtained from the activation values of the hidden layers of your model. Hence, we will need to run the input text through our model to obtain the outputs. First, let’s [tokenize](https://arize.com/blog-course/tokenization/) the input text: `input = tokenizer(input_text)` Once we have the tokenized input, we can pass it through our model. We use no\_grad() because we are not in the training phase, hence we do not need the gradients for back propagation. `with torch.no_grad(): # Get model outputs from batched inputs output = model(**inputs)` This output should contain, in addition to the classification logits, the activation values of the hidden state layers. We can select the hidden state layers as we would in a dictionary (to be able to obtain hidden states, we need to pass \`output\_hidden\_states=True\` when we instantiate the model using Hugging Face’s 🤗Transformer library): `# Get hidden states from model output hidden_states = out.hidden_states # Shape (num_hidden_states, seq_length, hidden_size)` The shape of the hidden\_state tensor is (num\_hidden\_states, seq\_length, hidden\_size), where: * **Num\_hidden\_states** represents the number of hidden states layers present in the model. * **Seq\_length** is the chosen token sequence length established. If our tokenized text has more(less) tokens, the sequence will be truncated(padded) before passed to the model. * **Hidden\_size** represents the size of each hidden layer. As mentioned above, embedding vectors are arrays with values equal to the activation values on the hidden layers. Hence, since a hidden layer has hidden\_size activation values, this parameter gives you the embedding dimensionality. For instance, the BERT model has a hidden\_size of 768, and a consequent embedding dimension of 768. Next, we choose to select the last hidden state layer to form our embeddings. You could choose other options, such as the average of all hidden layers, the maximum, the minimum, or any other combination of your liking. We kept it simple and chose the last layer. `# Select last hidden state layer last_hidden_state = hidden_states[-1] # (seq_length, hidden_size)` The shape now has been reduced to (seq\_length, hidden\_size), the last hidden state layer contains one embedding vector per token in the tokenized input. Here you have another opportunity to generate your embeddings in different ways. For instance, you could average out all the embedding vectors of all the tokens. Being a sentiment classification problem, and having used a BERT-like tokenizer, we chose to select the embedding vector associated with the \[CLS\] token, also known as the classification token (in sentiment classification problems, the \[CLS\] token embedding is fed to a feed-forward neural network to perform the classification and return the logits). . `# Select CLS token vector, across the batch embeddings = last_hidden_state[:,0,:] # (hidden_size)` This is a method used in the [original BERT paper](https://arxiv.org/pdf/1810.04805.pdf) , and is illustrated in the image below. In sentence classification, you often pass the embedding vector associated with the \[CLS\] token to a feed-forward neural network to obtain the classification predictions. In our case, we also extract that vector as a representation of the input sentence for observability using Arize. ![transformer encoder example ](https://arize.com/wp-content/uploads/2022/08/transformer-encoder.jpg) Now you know how to calculate an embedding vector from input text. However, you will also probably want to calculate embeddings for batches of data at once. The code snippet below gives an example of how you could calculate embedding vectors representing sentences in a sentiment classification problem, for input batches. And there you have it, this is how we chose to calculate embedding vectors, representing input sentences, in a sentiment classification problem. As we discussed above, even within the same use case, there are many opportunities for design decisions. In our [sentiment classification tutorial](https://docs.arize.com/arize/examples/embedding-examples-nlp) , we have a function like the one above with more ways of computing embeddings implemented – check it out!. To learn more about how to generate embeddings in other NLP use cases or in computer vision, check Arize’s example tutorials in the company’s [documentation](https://docs.arize.com/arize/sending-data/examples#embeddings-unstructured-data-tutorials) . How Can You Help Make AI More Transparent? ------------------------------------------ ### Using Arize Many times, knowing the architecture of a model is not enough to know a-priori how it’s going to perform on a specific use-case. Arize allows you to observe your model’s performance, see how it interacts with your data, and visualize your dataset’s topological structure. Arize can help you identify problems that were not possible to notice before, allowing you to report back to the community and make public changes to your dataset and/or model in a public setting such as the Hugging Face Hub. ### Using Hugging Face Hugging Face’s Hub has thousands of models and datasets, open sourced for you to see and use. If you are working on a specific use-case, you may detect problems with the way the data was sourced, problems with the data itself, possible improvements on the model, etc. Making these observations and possible improvements public is a fantastic way to help the community strive for better AI. Start Your Journey ------------------ Want to get started? Check out [Hugging Face’s Hub](https://huggingface.co/) and sign up for your [free Arize account](https://arize.com/join) . Share * [![](https://arize.com/wp-content/themes/arize-2022/images/icon-tw-circle.svg)](https://twitter.com/intent/tweet?url=https%3A%2F%2Farize.com%2Fblog%2Farize-hugging-face%2F&text=Hugging%20Face%20+%20Arize:%20Partnership%20and%20Code%20Example) * [![](https://arize.com/wp-content/themes/arize-2022/images/icon-fb-circle.svg)](https://www.facguide.com/sharer/sharer.php?u=https://arize.com/blog/arize-hugging-face/&picture=https://arize.com) * [![](https://arize.com/wp-content/themes/arize-2022/images/icon-in-circle.svg)](https://www.linkedin.com/shareArticle?mini=true&url=https%3A%2F%2Farize.com%2Fblog%2Farize-hugging-face%2F&title=Hugging%20Face%20+%20Arize:%20Partnership%20and%20Code%20Example) #### Suggested reading ![azure native llm observability tracing evaluation with arize](https://arize.com/wp-content/uploads/2025/05/Arize-x-Azure-1.jpg) [Arize AI Now Generally Available As Part of Azure Native Integrations](https://arize.com/blog/arize-ai-now-generally-available-as-part-of-azure-native-integrations/) ![](https://arize.com/wp-content/uploads/2025/05/Arize-x-Nvidia.jpg) [Arize AI Accelerates Enterprise AI Adoption On-Premises With NVIDIA](https://arize.com/blog/arize-ai-accelerates-enterprise-ai-adoption-on-premises-with-nvidia/) Recommended resources --------------------- [![](https://arize.com/wp-content/uploads/2022/06/arize-embeddings-cover.jpg)\ \ post\ \ #### Monitor Unstructured Data with Arize\ \ Read more](https://arize.com/blog/monitor-unstructured-data-with-arize/) [![Jeff Boudier Hugging Face](https://arize.com/wp-content/uploads/2022/06/Screen-Shot-2022-08-22-at-4.14.33-PM.png)\ \ resource\ \ #### Arize:Observe Unstructured – Accelerating Machine Learning from Research to Production with Hugging Face\ \ Read more](https://arize.com/resource/arizeobserve-unstructured-accelerating-machine-learning-from-research-to-production-with-hugging-face/) [![](https://arize.com/wp-content/uploads/2022/06/blog-embeddings-101-cover.jpg)\ \ post\ \ #### Getting Started With Embeddings Is Easier Than You Think\ \ Read more](https://arize.com/blog/getting-started-with-embeddings-is-easier-than-you-think/) Sign up for our monthly newsletter, The Evaluator. -------------------------------------------------- [Subscribe](https://arize.com/blog/arize-hugging-face/#blog-subscribe-modal) Subscribe to The Evaluator -------------------------- We’ll send you the latest news, expertise, and product updates from Arize. Your inbox is sacred, so we’ll only curate and send the best stuff. \*We’re committed to your privacy. Arize uses the information you provide to contact you about relevant content, products, and services. You may unsubscribe from these communications at any time. For more information, check out our [privacy policy](https://arize.com/privacy-policy) . --- # Arize AI Listed In 2021 Gartner Market Guide for AI Trust, Risk and Security Management (AI TRiSM) - Arize AI ![](https://arize.com/wp-content/uploads/2021/03/tammyle-196x196.jpg "Version 2") [Tammy Le](https://arize.com/author/tammy-le/) Chief Marketing Officer Share * [![](https://arize.com/wp-content/themes/arize-2022/images/icon-tw-circle.svg)](https://twitter.com/intent/tweet?url=https%3A%2F%2Farize.com%2Fblog%2Farize-ai-listed-in-gartner-market-guide%2F&text=Arize%20AI%20Listed%20In%202021%20Gartner%20Market%20Guide%20for%20AI%20Trust,%20Risk%20and%20Security%20Management%20(AI%20TRiSM)) * [![](https://arize.com/wp-content/themes/arize-2022/images/icon-fb-circle.svg)](https://www.facguide.com/sharer/sharer.php?u=https://arize.com/blog/arize-ai-listed-in-gartner-market-guide/&picture=https://arize.com) * [![](https://arize.com/wp-content/themes/arize-2022/images/icon-in-circle.svg)](https://www.linkedin.com/shareArticle?mini=true&url=https%3A%2F%2Farize.com%2Fblog%2Farize-ai-listed-in-gartner-market-guide%2F&title=Arize%20AI%20Listed%20In%202021%20Gartner%20Market%20Guide%20for%20AI%20Trust,%20Risk%20and%20Security%20Management%20(AI%20TRiSM)) ![](https://arize.com/wp-content/themes/arize-2022/images/icon-newsletter.svg) Subscribe to the Arize blog [Get the latest](https://arize.com/blog/arize-ai-listed-in-gartner-market-guide/#blog-subscribe-modal) #### On this page #### Suggested reading ![](https://arize.com/wp-content/uploads/2025/10/iso-iec-27001-certified-icon.jpg) [Arize AI Achieves ISO/IEC 27001 Certification](https://arize.com/blog/arize-ai-achieves-iso-iec-27001-certification/) ![](https://arize.com/wp-content/uploads/2025/03/NVIDIA-Arize-blog.jpg) [Self-Improving Agents: Automating LLM Performance Optimization using Arize and NVIDIA NeMo](https://arize.com/blog/arize-nvidia-nemo-integration/) ![](https://arize.com/wp-content/uploads/2021/09/GartnerAward_Graphic.png "GartnerAward_Graphic") Arize AI Listed In 2021 Gartner Market Guide for AI Trust, Risk and Security Management (AI TRiSM) ================================================================================================== Published Sep 27, 2021 * [Company](https://arize.com/blog/?cat=company) ![](https://arize.com/wp-content/uploads/2021/03/tammyle-196x196.jpg "Version 2") #### [Tammy Le](https://arize.com/author/tammy-le/) ##### Chief Marketing Officer As more and more teams embrace machine learning to streamline their businesses or turn previously impractical technologies into reality, there has been a rising interest in tools that can help bring a model from the research lab into customers’ hands. Google built TFX, Facebook built FBLearner, Uber built Michaelangelo, Airbnb built Bighead, and these systems have allowed teams to scale their MLOps and AI ambitions. Outside of these large tech companies; however, building machine learning proof of concepts in the lab is drastically different from deploying models that work in the real world. Two years ago, based on our founders’ experience managing complex ML systems at large, data-driven enterprises and hundreds of conversations with ML practitioners, Arize set out to develop a [Machine Learning Observability](https://arize.com/model-monitoring) platform to help teams successfully take models from research to production with ease. Today, Arize’s automated [model monitoring](https://arize.com/ml-monitoring/) and analytics platform is counted on by top enterprises to quickly detect issues when they emerge, troubleshoot why they happened, and improve overall model performance. We are excited to announce that Gartner has listed Arize as a Representative Explainability Vendor in the 2021 Market Guide for AI Trust, Risk and Security Management (AI TRiSM). Gartner views AI TriSM as comprising multiple software segments that ensure AI model governance, trustworthiness, fairness, reliability, efficacy, security and data protection. According to Gartner, “AI Trust, Risk and Security Management typically requires organizations to implement a best-of-breed tool portfolio approach, as most AI platforms will not provide all required functionality.” The firm also notes that, “security and risk concerns are almost always an afterthought in any system development and deployment. When it comes to AI, this is an especially poor design choice since there are so many moving parts and so many of them are opaque to most users. There’s no need to rely on a black box running critical functions for your enterprise. There are in fact many solutions that bring transparency and trust, keep the bad guys out, prevent benign mistakes, protect sensitive data, and keep AI models functioning as intended. These solutions just need to be used.” To us, this recognition highlights the growing need for and our leadership in delivering innovative solutions that act as the guardrail on deployed AI, providing transparency and introspection into historically black box systems to ensure more effective and [responsible AI](https://www.forbes.com/sites/aparnadhinakaran/?sh=5d7691024958) . While this market is still emerging, the pain is real and we are working closely with our customers towards a future where every organization can easily deploy, observe and maintain ML models that drive critical business decisions and processes. If you’re a Gartner client, you can access the full report [here](https://www.gartner.com/account/signin?method=initialize&TARGET=http%253A%252F%252Fwww.gartner.com%252Fdocument%252F4005344%253Fref%253DsolrAll%2526refval%253D299261666%2526_ga%253D2.8903798.2128181334.1632499103-1543138590.1632499103) . _Disclaimer: Gartner Market Guide for AI Trust, Risk and Security Management (AI TRiSM), Avivah Litan, Farhan Choudhary, Jeremy D’Hoinne, September 1, 2021. Gartner does not endorse any vendor, product or service depicted in its research publications, and does not advise technology users to select only those vendors with the highest ratings or other designation._ Share * [![](https://arize.com/wp-content/themes/arize-2022/images/icon-tw-circle.svg)](https://twitter.com/intent/tweet?url=https%3A%2F%2Farize.com%2Fblog%2Farize-ai-listed-in-gartner-market-guide%2F&text=Arize%20AI%20Listed%20In%202021%20Gartner%20Market%20Guide%20for%20AI%20Trust,%20Risk%20and%20Security%20Management%20(AI%20TRiSM)) * [![](https://arize.com/wp-content/themes/arize-2022/images/icon-fb-circle.svg)](https://www.facguide.com/sharer/sharer.php?u=https://arize.com/blog/arize-ai-listed-in-gartner-market-guide/&picture=https://arize.com) * [![](https://arize.com/wp-content/themes/arize-2022/images/icon-in-circle.svg)](https://www.linkedin.com/shareArticle?mini=true&url=https%3A%2F%2Farize.com%2Fblog%2Farize-ai-listed-in-gartner-market-guide%2F&title=Arize%20AI%20Listed%20In%202021%20Gartner%20Market%20Guide%20for%20AI%20Trust,%20Risk%20and%20Security%20Management%20(AI%20TRiSM)) #### Suggested reading ![](https://arize.com/wp-content/uploads/2025/10/iso-iec-27001-certified-icon.jpg) [Arize AI Achieves ISO/IEC 27001 Certification](https://arize.com/blog/arize-ai-achieves-iso-iec-27001-certification/) ![](https://arize.com/wp-content/uploads/2025/03/NVIDIA-Arize-blog.jpg) [Self-Improving Agents: Automating LLM Performance Optimization using Arize and NVIDIA NeMo](https://arize.com/blog/arize-nvidia-nemo-integration/) Sign up for our monthly newsletter, The Evaluator. -------------------------------------------------- [Subscribe](https://arize.com/blog/arize-ai-listed-in-gartner-market-guide/#blog-subscribe-modal) Subscribe to The Evaluator -------------------------- We’ll send you the latest news, expertise, and product updates from Arize. Your inbox is sacred, so we’ll only curate and send the best stuff. \*We’re committed to your privacy. Arize uses the information you provide to contact you about relevant content, products, and services. You may unsubscribe from these communications at any time. For more information, check out our [privacy policy](https://arize.com/privacy-policy) . --- # Arize AI and Paperspace Announce a Partnership to Bring Deep ML Observability Solutions to Data Science Teams - Arize AI ![](https://arize.com/wp-content/uploads/2021/06/krystal-headshot-e1624425208666-196x196.jpg "krystal headshot") [Krystal Kirkland](https://arize.com/author/krystal-kirkland/) Software Engineer Share * [![](https://arize.com/wp-content/themes/arize-2022/images/icon-tw-circle.svg)](https://twitter.com/intent/tweet?url=https%3A%2F%2Farize.com%2Fblog%2Farize-ai-and-paperspace-partnership%2F&text=Arize%20AI%20and%20Paperspace%20Announce%20a%20Partnership%20to%20Bring%20Deep%20ML%20Observability%20Solutions%20to%20Data%20Science%20Teams) * [![](https://arize.com/wp-content/themes/arize-2022/images/icon-fb-circle.svg)](https://www.facguide.com/sharer/sharer.php?u=https://arize.com/blog/arize-ai-and-paperspace-partnership/&picture=https://arize.com) * [![](https://arize.com/wp-content/themes/arize-2022/images/icon-in-circle.svg)](https://www.linkedin.com/shareArticle?mini=true&url=https%3A%2F%2Farize.com%2Fblog%2Farize-ai-and-paperspace-partnership%2F&title=Arize%20AI%20and%20Paperspace%20Announce%20a%20Partnership%20to%20Bring%20Deep%20ML%20Observability%20Solutions%20to%20Data%20Science%20Teams) ![](https://arize.com/wp-content/themes/arize-2022/images/icon-newsletter.svg) Subscribe to the Arize blog [Get the latest](https://arize.com/blog/arize-ai-and-paperspace-partnership/#blog-subscribe-modal) #### On this page #### Suggested reading ![](https://arize.com/wp-content/uploads/2025/11/microsoft-foundry-arize-ax.png) [Evaluating and Improving AI Agents at Scale with Microsoft Foundry](https://arize.com/blog/evaluating-and-improving-ai-agents-at-scale-with-microsoft-foundry/) ![](https://arize.com/wp-content/uploads/2025/10/llm-tracing-blog-cover.png) [Top LLM Tracing Tools](https://arize.com/blog/top-llm-tracing-tools/) ![](https://arize.com/wp-content/uploads/2021/03/Partnership-Announcement-1-e1624424337129-1445x1120.png "Partnership Announcement 1") Arize AI and Paperspace Announce a Partnership to Bring Deep ML Observability Solutions to Data Science Teams ============================================================================================================= Published Oct 16, 2020 * [Company](https://arize.com/blog/?cat=company) * [ML Monitoring](https://arize.com/blog/?cat=ml-monitoring) * [ML Observability](https://arize.com/blog/?cat=ml-observability) * [Product](https://arize.com/blog/?cat=product) * [Uncategorized](https://arize.com/blog/?cat=uncategorized) ![](https://arize.com/wp-content/uploads/2021/06/krystal-headshot-e1624425208666-196x196.jpg "krystal headshot") #### [Krystal Kirkland](https://arize.com/author/krystal-kirkland/) ##### Software Engineer Arize and Paperspace are pleased to announce a partnership available to Paperspace platform customers. A simple pre-tested integration that is easy to set up, is now available to Paperspace users. Paperspace customers will have priority access to the Arize platform available for model monitoring, troubleshooting and explainability. The integration allows, with a few lines of code, simple integration, the ability to monitor data drift and model drift, and troubleshoot those problems in a purpose built platform designed for ML Observability. Why Observability ----------------- The difference between research environments and production can cause large issues for models deployed in the real world. The inputs models see, the degradation over time and the performance problems that arise, can be painful to troubleshoot. Observability helps teams go from research to production maintaining the results delivered, and helps teams troubleshoot problems quickly without eating up Data Science cycles. The ability to explain, understand and get quick answers builds trust between research teams and end users. Arize AI Integration Setup -------------------------- ![](https://arize.com/wp-content/uploads/2020/10/0_KdjyA4JAm43rVKDg.png) Arize Integration What Paperspace Users Will Get ------------------------------ **Model Drift Detection** Push button setup for access to performance by any performance statistic. ![](https://arize.com/wp-content/uploads/2020/10/0_bNavZynhO8aqKKsl.png) Easy access to in-depth tools to troubleshoot changes on any model launched on the PaperSpace platform. Production performance can be tracked and all performance metrics can be monitored for any cohort of predictions. **Data Drift Detection** Any feature data or prediction output can be analyzed for data drift and quality statistics. ![](https://arize.com/wp-content/uploads/2020/10/0_q_yvbwC667geyV4y.png) In depth tools for statistical distance checks on input features allowing for quick analysis of complex changes. ![](https://arize.com/wp-content/uploads/2020/10/0_GZ3Hok7ytz38pAwL.png) Highly configurable checks on any data distribution compared to any reference. **Production Troubleshooting** ![](https://arize.com/wp-content/uploads/2020/10/0_RWuXGHMSyLmNXHPO.png) The ability to quickly troubleshoot issues accessing millions of facets of predictions based on any value in seconds. Arize helps connect distribution changes to performance issues instantly. Arize and Paperspace Partnership -------------------------------- Arize AI is excited to enable Paperspace users with ML Observability tools on top of the high-performance cloud computing and deep learning development platform Paperspace provides. Get on the waitlist for the Arize AI model observability integration optimized to run on Gradient and coming soon to the Paperspace console. Get your name down to be at the front of the line! [https://arizeai.typeform.com/to/tGkiUiCN](https://arizeai.typeform.com/to/tGkiUiCN) **About Arize AI**  Arize AI is a [Machine Learning Observabililty](https://arize.com/model-monitoring "https://arize.com/model-monitoring") platform that helps ML practitioners successfully take models from research to production, with ease. Arize’s automated [model monitoring](https://arize.com/ml-monitoring/ "https://arize.com/ml-monitoring/") and analytics platform help ML teams quickly detect issues the moment they emerge, troubleshoot why they happened, and improve overall model performance. By connecting offline training and validation datasets to online production data in a central inference store, ML teams are able to streamline [model validation](https://arize.com/ml-model-failure-modes/ "https://arize.com/ml-model-failure-modes/") , [drift detection](https://arize.com/take-my-drift-away/ "https://arize.com/take-my-drift-away/") , [data quality checks](https://arize.com/data-quality-monitoring/ "https://arize.com/data-quality-monitoring/") , and [model performance management](https://arize.com/monitor-your-model-in-production/ "https://arize.com/monitor-your-model-in-production/") . Arize AI acts as the guardrail on deployed AI, providing transparency and introspection into historically black box systems to ensure more effective and [responsible AI](https://www.forbes.com/sites/aparnadhinakaran/?sh=5d7691024958 "https://www.forbes.com/sites/aparnadhinakaran/?sh=5d7691024958") . To learn more about Arize or machine learning observability and monitoring, visit our [blog](https://arize.com/blog/ "https://arize.com/blog/") and [resource hub](https://arize.com/resource-hub/ "https://arize.com/resource-hub/") ! Share * [![](https://arize.com/wp-content/themes/arize-2022/images/icon-tw-circle.svg)](https://twitter.com/intent/tweet?url=https%3A%2F%2Farize.com%2Fblog%2Farize-ai-and-paperspace-partnership%2F&text=Arize%20AI%20and%20Paperspace%20Announce%20a%20Partnership%20to%20Bring%20Deep%20ML%20Observability%20Solutions%20to%20Data%20Science%20Teams) * [![](https://arize.com/wp-content/themes/arize-2022/images/icon-fb-circle.svg)](https://www.facguide.com/sharer/sharer.php?u=https://arize.com/blog/arize-ai-and-paperspace-partnership/&picture=https://arize.com) * [![](https://arize.com/wp-content/themes/arize-2022/images/icon-in-circle.svg)](https://www.linkedin.com/shareArticle?mini=true&url=https%3A%2F%2Farize.com%2Fblog%2Farize-ai-and-paperspace-partnership%2F&title=Arize%20AI%20and%20Paperspace%20Announce%20a%20Partnership%20to%20Bring%20Deep%20ML%20Observability%20Solutions%20to%20Data%20Science%20Teams) #### Suggested reading ![](https://arize.com/wp-content/uploads/2025/11/microsoft-foundry-arize-ax.png) [Evaluating and Improving AI Agents at Scale with Microsoft Foundry](https://arize.com/blog/evaluating-and-improving-ai-agents-at-scale-with-microsoft-foundry/) ![](https://arize.com/wp-content/uploads/2025/10/llm-tracing-blog-cover.png) [Top LLM Tracing Tools](https://arize.com/blog/top-llm-tracing-tools/) Sign up for our monthly newsletter, The Evaluator. -------------------------------------------------- [Subscribe](https://arize.com/blog/arize-ai-and-paperspace-partnership/#blog-subscribe-modal) Subscribe to The Evaluator -------------------------- We’ll send you the latest news, expertise, and product updates from Arize. Your inbox is sacred, so we’ll only curate and send the best stuff. \*We’re committed to your privacy. Arize uses the information you provide to contact you about relevant content, products, and services. You may unsubscribe from these communications at any time. For more information, check out our [privacy policy](https://arize.com/privacy-policy) . --- # Arize AI: Support for EU Data Residency - Arize AI ![](https://arize.com/wp-content/uploads/2022/03/David-Burch-1-196x196.png "David-Burch") [David Burch](https://arize.com/author/david-burch/) Share * [![](https://arize.com/wp-content/themes/arize-2022/images/icon-tw-circle.svg)](https://twitter.com/intent/tweet?url=https%3A%2F%2Farize.com%2Fblog%2Farize-ai-support-for-eu-data-residency%2F&text=Arize%20AI:%20Support%20for%20EU%20Data%20Residency) * [![](https://arize.com/wp-content/themes/arize-2022/images/icon-fb-circle.svg)](https://www.facguide.com/sharer/sharer.php?u=https://arize.com/blog/arize-ai-support-for-eu-data-residency/&picture=https://arize.com) * [![](https://arize.com/wp-content/themes/arize-2022/images/icon-in-circle.svg)](https://www.linkedin.com/shareArticle?mini=true&url=https%3A%2F%2Farize.com%2Fblog%2Farize-ai-support-for-eu-data-residency%2F&title=Arize%20AI:%20Support%20for%20EU%20Data%20Residency) ![](https://arize.com/wp-content/themes/arize-2022/images/icon-newsletter.svg) Subscribe to the Arize blog [Get the latest](https://arize.com/blog/arize-ai-support-for-eu-data-residency/#blog-subscribe-modal) #### On this page #### Suggested reading ![](https://arize.com/wp-content/uploads/2025/10/iso-iec-27001-certified-icon.jpg) [Arize AI Achieves ISO/IEC 27001 Certification](https://arize.com/blog/arize-ai-achieves-iso-iec-27001-certification/) ![](https://arize.com/wp-content/uploads/2025/03/NVIDIA-Arize-blog.jpg) [Self-Improving Agents: Automating LLM Performance Optimization using Arize and NVIDIA NeMo](https://arize.com/blog/arize-nvidia-nemo-integration/) ![](https://arize.com/wp-content/uploads/2024/08/arize-eu-residency.png "arize-eu-residency") Arize AI: Support for EU Data Residency ======================================= Published Aug 1, 2024 * [Company](https://arize.com/blog/?cat=company) ![](https://arize.com/wp-content/uploads/2022/03/David-Burch-1-196x196.png "David-Burch") #### [David Burch](https://arize.com/author/david-burch/) Arize AI recently rolled out EU data residency for all users, enabling customers to host their data within the European Union. By offering EU data residency, Arize enables organizations to use its AI observability and LLM evaluation tools while adhering to local data protection laws. Companies can comply with the EU’s General Data Protection Regulation (GDPR) by ensuring that personal data remains within EU borders. This development is particularly important for sectors like finance and healthcare, where regulations often require data to be stored locally. Arize is SOC 2 Type II and HIPAA compliant and has achieved PCI DSS 4.0 certification. To learn more about Arize AI’s commitment to supporting compliance and data privacy requirements globally, check out the [Arize Trust Center](https://arize.com/trust-center/) . Share * [![](https://arize.com/wp-content/themes/arize-2022/images/icon-tw-circle.svg)](https://twitter.com/intent/tweet?url=https%3A%2F%2Farize.com%2Fblog%2Farize-ai-support-for-eu-data-residency%2F&text=Arize%20AI:%20Support%20for%20EU%20Data%20Residency) * [![](https://arize.com/wp-content/themes/arize-2022/images/icon-fb-circle.svg)](https://www.facguide.com/sharer/sharer.php?u=https://arize.com/blog/arize-ai-support-for-eu-data-residency/&picture=https://arize.com) * [![](https://arize.com/wp-content/themes/arize-2022/images/icon-in-circle.svg)](https://www.linkedin.com/shareArticle?mini=true&url=https%3A%2F%2Farize.com%2Fblog%2Farize-ai-support-for-eu-data-residency%2F&title=Arize%20AI:%20Support%20for%20EU%20Data%20Residency) #### Suggested reading ![](https://arize.com/wp-content/uploads/2025/10/iso-iec-27001-certified-icon.jpg) [Arize AI Achieves ISO/IEC 27001 Certification](https://arize.com/blog/arize-ai-achieves-iso-iec-27001-certification/) ![](https://arize.com/wp-content/uploads/2025/03/NVIDIA-Arize-blog.jpg) [Self-Improving Agents: Automating LLM Performance Optimization using Arize and NVIDIA NeMo](https://arize.com/blog/arize-nvidia-nemo-integration/) Sign up for our monthly newsletter, The Evaluator. -------------------------------------------------- [Subscribe](https://arize.com/blog/arize-ai-support-for-eu-data-residency/#blog-subscribe-modal) Subscribe to The Evaluator -------------------------- We’ll send you the latest news, expertise, and product updates from Arize. Your inbox is sacred, so we’ll only curate and send the best stuff. \*We’re committed to your privacy. Arize uses the information you provide to contact you about relevant content, products, and services. You may unsubscribe from these communications at any time. For more information, check out our [privacy policy](https://arize.com/privacy-policy) . --- # Arize AI + OpenAI - Arize AI ![](https://arize.com/wp-content/uploads/2022/06/francisco-castillo-carrasco-196x196.jpg "francisco-castillo-carrasco") [Francisco Castillo](https://arize.com/author/francisco-castillo/) Data Scientist / Software Engineer Share * [![](https://arize.com/wp-content/themes/arize-2022/images/icon-tw-circle.svg)](https://twitter.com/intent/tweet?url=https%3A%2F%2Farize.com%2Fblog%2Farize-ai-openai%2F&text=Arize%20AI%20+%20OpenAI) * [![](https://arize.com/wp-content/themes/arize-2022/images/icon-fb-circle.svg)](https://www.facguide.com/sharer/sharer.php?u=https://arize.com/blog/arize-ai-openai/&picture=https://arize.com) * [![](https://arize.com/wp-content/themes/arize-2022/images/icon-in-circle.svg)](https://www.linkedin.com/shareArticle?mini=true&url=https%3A%2F%2Farize.com%2Fblog%2Farize-ai-openai%2F&title=Arize%20AI%20+%20OpenAI) ![](https://arize.com/wp-content/themes/arize-2022/images/icon-newsletter.svg) Subscribe to the Arize blog [Get the latest](https://arize.com/blog/arize-ai-openai/#blog-subscribe-modal) #### On this page #### Suggested reading ![Peter Leimbigler Data Science Team Leader at Klick Consulting](https://arize.com/wp-content/uploads/2024/02/klick-health-interview.jpg) [What Does It Take To Pioneer Successful LLM Applications In Healthcare and the Life Sciences?](https://arize.com/blog/ai-llm-in-healthcare-and-the-life-sciences-klick-health/) ![umap abstract art](https://arize.com/wp-content/uploads/2022/12/umap-abstract-art.png) [Measuring Embedding Drift](https://arize.com/blog/embedding-drift/) ![openai arize](https://arize.com/wp-content/uploads/2022/09/arize-openai.jpg "arize-openai") Arize AI + OpenAI ================= Published Sep 30, 2022 * [Embeddings](https://arize.com/blog/?cat=embeddings) ![](https://arize.com/wp-content/uploads/2022/06/francisco-castillo-carrasco-196x196.jpg "francisco-castillo-carrasco") #### [Francisco Castillo](https://arize.com/author/francisco-castillo/) ##### Data Scientist / Software Engineer **_This blog was written in collaboration with Amit Goren, Group Product Marketing Manager at Arize_** Using Arize and OpenAI together can help organizations better build unstructured models and monitor and troubleshoot those models in production, lowering costs and maximizing performance. NLP Models Are Tough To Get Out Into the Real World and Monitor Once Deployed ----------------------------------------------------------------------------- According to [multiple analyst estimates](https://mitsloan.mit.edu/ideas-made-to-matter/tapping-power-unstructured-data) , over 80% of data is unstructured information like text, images, video, or audio. Leveraging this data for deep learning is time and resource-intensive. Unstructured data such as text requires some human labeling or annotation for teams to be able to group the data and find trends and insights. Given the difficulty in finding similarities, generating embeddings can help lower the dimensions and enable teams to better understand and visualize unstructured data. When an unstructured model such as a natural language processing (NLP) model is ready to be deployed to production, teams frequently lack adequate tools to monitor and troubleshoot issues that may exist or emerge after deployment. OpenAI Helps You Build Unstructured Models ------------------------------------------ OpenAI is an AI research and deployment company. Their mission is to ensure that artificial general intelligence — highly autonomous systems that outperform humans at most economically valuable work — benefits all of humanity. With AI systems like GPT-3, Codex and DALL-E, OpenAI provides the AI building blocks to power the next generation of products. Given a simple text-based instruction in natural language, GPT-3 and Codex returns a text or code completion. Given a text-based prompt, DALL-E renders photorealistic images or art. Together, these generative models open up a new world of use cases and applications. Earlier this year, OpenAI released three families of embedding models for different functionalities: text similarity, text search and code search. * Text Similarity: Text similarity models provide embeddings that capture the semantic similarity of pieces of text. * Text Search: Text search models provide embeddings that enable large-scale search tasks, like finding a relevant document among a collection of documents given a text query. * Code Search: Code search models provide code and text embeddings for code search tasks. Arize Helps You Monitor and Improve Your Unstructured Models ------------------------------------------------------------ Arize is an ML observability platform that enables teams to log both structured and unstructured data to detect, root cause, and resolve model performance issues faster. With Arize’s purpose-built workflows for root cause analysis, teams can reduce time-to-resolution for even the most complex models. Arize’s latest release includes support for embeddings to [monitor and troubleshoot unstructured data models](https://arize.com/blog/monitor-unstructured-data-with-arize/) . By monitoring embeddings of their unstructured data, teams can proactively identify when their data is drifting and troubleshoot using Arize’s interactive [UMAP](https://arize.com/glossary/umap/) visualization to identify new patterns or export segments for high-value labeling. ### Example With Code Here is an example of how to use OpenAI and Arize AI together for an NLP use case. #### OpenAI First, we need a vector representation or _embedding_ of our input text. OpenAI offers a variety of models that can extract said embedding using just a couple lines of code. #### Arize AI Once we have our data, including the embeddings associated with the input text, the first step is to set up the Arize client. We will log the data afterward. Copy the Arize `API_KEY` and `SPACE_KEY` from your Arize Space Settings page to the variables in the cell below. We will also be setting up some metadata to use across all logging. Next, we set up the _Schema._ A Schema instance specifies the column names for corresponding data in the dataframe. While we could define different Schemas for training and production datasets, the dataframes have the same column names, so the Schema will be the same. Arize allows you to ingest not only the embedding vector, but the raw data associated with that embedding, or a URL link to that raw data. Therefore, up to three columns can be associated to the same embedding object\*. To be able to do this, Arize’s SDK provides the `EmbeddingColumnNames` class, used below. \*NOTE: This is how we refer to the 3 possible pieces of information that can be sent as embedding objects: * Embedding `vector` (required) * Embedding `data` (optional): raw text associated with the embedding vector * Embedding `link_to_data` (optional): link to the data file (image, audio, …) associated with the embedding vector. Not represented in this example, learn more [here](https://docs.arize.com/arize/sending-data/model-schema-reference#8.-embedding-features-unstructured) . Finally, we can send our data to Arize by using the `log()`method. Once the data is in Arize and you set a baseline (i.e. training data), you can begin to troubleshoot. Here is an example of a drift tracking plot and a UMAP visualization of the data. In this case, both training and production data are superimposed, but another cluster of production data has appeared. This indicates that the model sees data in production qualitatively different from the data it was trained on, causing performance degradation. ![](https://arize.com/wp-content/uploads/2022/09/umap-new-cluster.png) ### Start Your Journey Check out the full Colab on multi-class sentiment classification using OpenAI [here](https://docs.arize.com/arize/sending-data/examples#embeddings-nlp-examples) and get started by signing up for your [free Arize account](https://app.arize.com/auth/join) . Questions? [Join the Arize community](https://arize.com/community/) to learn from peers and get notified about workshops and events. Share * [![](https://arize.com/wp-content/themes/arize-2022/images/icon-tw-circle.svg)](https://twitter.com/intent/tweet?url=https%3A%2F%2Farize.com%2Fblog%2Farize-ai-openai%2F&text=Arize%20AI%20+%20OpenAI) * [![](https://arize.com/wp-content/themes/arize-2022/images/icon-fb-circle.svg)](https://www.facguide.com/sharer/sharer.php?u=https://arize.com/blog/arize-ai-openai/&picture=https://arize.com) * [![](https://arize.com/wp-content/themes/arize-2022/images/icon-in-circle.svg)](https://www.linkedin.com/shareArticle?mini=true&url=https%3A%2F%2Farize.com%2Fblog%2Farize-ai-openai%2F&title=Arize%20AI%20+%20OpenAI) #### Suggested reading ![Peter Leimbigler Data Science Team Leader at Klick Consulting](https://arize.com/wp-content/uploads/2024/02/klick-health-interview.jpg) [What Does It Take To Pioneer Successful LLM Applications In Healthcare and the Life Sciences?](https://arize.com/blog/ai-llm-in-healthcare-and-the-life-sciences-klick-health/) ![umap abstract art](https://arize.com/wp-content/uploads/2022/12/umap-abstract-art.png) [Measuring Embedding Drift](https://arize.com/blog/embedding-drift/) Sign up for our monthly newsletter, The Evaluator. -------------------------------------------------- [Subscribe](https://arize.com/blog/arize-ai-openai/#blog-subscribe-modal) Subscribe to The Evaluator -------------------------- We’ll send you the latest news, expertise, and product updates from Arize. Your inbox is sacred, so we’ll only curate and send the best stuff. \*We’re committed to your privacy. Arize uses the information you provide to contact you about relevant content, products, and services. You may unsubscribe from these communications at any time. For more information, check out our [privacy policy](https://arize.com/privacy-policy) . --- # Arize Release Notes: Sep 5, 2024 - Arize AI ![](https://arize.com/wp-content/uploads/2023/01/Sarah_headshot-196x196.jpg "Sarah_headshot") [Sarah Welsh](https://arize.com/author/sarah-welsh/) Contributor Share * [![](https://arize.com/wp-content/themes/arize-2022/images/icon-tw-circle.svg)](https://twitter.com/intent/tweet?url=https%3A%2F%2Farize.com%2Fblog%2Farize-release-notes-sep-5-2024&text=Arize%20Release%20Notes:%20Sep%205,%202024) * [![](https://arize.com/wp-content/themes/arize-2022/images/icon-fb-circle.svg)](https://www.facguide.com/sharer/sharer.php?u=https://arize.com/blog/arize-release-notes-sep-5-2024&picture=https://arize.com) * [![](https://arize.com/wp-content/themes/arize-2022/images/icon-in-circle.svg)](https://www.linkedin.com/shareArticle?mini=true&url=https%3A%2F%2Farize.com%2Fblog%2Farize-release-notes-sep-5-2024&title=Arize%20Release%20Notes:%20Sep%205,%202024) ![](https://arize.com/wp-content/themes/arize-2022/images/icon-newsletter.svg) Subscribe to the Arize blog [Get the latest](https://arize.com/blog/arize-release-notes-sep-5-2024#blog-subscribe-modal) #### On this page #### Suggested reading ![](https://arize.com/wp-content/uploads/2025/11/arize-ax-october-updates-2025.jpg) [New In Arize AX: Tags, Data Fabric, Automatic Threshold Ranges for Monitors and More](https://arize.com/blog/new-in-arize-ax-tags-data-fabric-automatic-threshold-ranges-for-monitors-and-more/) ![adb - Arize AI](https://arize.com/wp-content/uploads/2025/09/abd-cover-image-dark-3.jpg) [adb Benchmarks](https://arize.com/blog/adb-benchmarks/) ![Text reads: Release Notes, September 5, 2024](https://arize.com/wp-content/uploads/2024/09/Release-notes-9-5.jpg "Release notes 9-5") Arize Release Notes: Sep 5, 2024 ================================ Published Sep 5, 2024 * [Product Releases](https://arize.com/blog/?cat=product-releases) * [Release Notes](https://arize.com/blog/?cat=release-notes) ![](https://arize.com/wp-content/uploads/2023/01/Sarah_headshot-196x196.jpg "Sarah_headshot") #### [Sarah Welsh](https://arize.com/author/sarah-welsh/) ##### Contributor Welcome to our regular update on new releases, enhancements, and changes. What’s New ---------- ### Annotations (Beta) Annotations are custom labels that can be added to traces. Use annotations to: manually label data, categorize spans or traces, curate a dataset for experimentation, log human feedback. Reach out to your account rep to get access. [Learn more about annotations here.](https://docs.arize.com/arize/llm-evaluation-and-annotations/annotations) ![Arize annotations custom labels](https://arize.com/wp-content/uploads/2024/09/Annotations_Beta.png) Enhancements ------------ ### Models API Update Users can now query Model Versions and set the model baseline using GraphQL. [Learn more about the Models API update here](https://docs.arize.com/arize/api-reference/graphql-api/models-api) . ### Metrics API Update Users can now query for average metrics or metrics over time directly from the model node. [Learn more about the Metrics API update here.](https://docs.arize.com/arize/api-reference/graphql-api/metrics-api)   📚 New Content -------------- 🤖 Intro to [Agent Architectures](https://arize.com/blog-course/llm-agent-how-to-set-up/agent-architecture/) 🖼️ [Evaluating an Image Classifier](https://arize.com/blog/evaluate-image-classifier/) 💂 [Advanced Guardrails](https://arize.com/blog-course/advanced-guardrails-for-llm-applications/) 🛠️ Survey: [The State of AI Engineering](https://arize.com/blog/state-of-ai-engineering-survey/) ✏️ [LLM Tracing Primer](https://arize.com/blog-course/llm-tracing-from-automatically-collecting-traces-to-troubleshooting-your-llm-app/) 🛍️ [Bazaarvoice](https://arize.com/blog/how-bazaarvoice-navigated-the-challenges-of-deploying-an-llm-app/) : Navigating the Challenges of Deploying an LLM App 👨‍💼 Report: [The Rise of Gen AI in SEC Filings](https://arize.com/resource/rise-of-gen-ai-in-sec-filings/) Share * [![](https://arize.com/wp-content/themes/arize-2022/images/icon-tw-circle.svg)](https://twitter.com/intent/tweet?url=https%3A%2F%2Farize.com%2Fblog%2Farize-release-notes-sep-5-2024&text=Arize%20Release%20Notes:%20Sep%205,%202024) * [![](https://arize.com/wp-content/themes/arize-2022/images/icon-fb-circle.svg)](https://www.facguide.com/sharer/sharer.php?u=https://arize.com/blog/arize-release-notes-sep-5-2024&picture=https://arize.com) * [![](https://arize.com/wp-content/themes/arize-2022/images/icon-in-circle.svg)](https://www.linkedin.com/shareArticle?mini=true&url=https%3A%2F%2Farize.com%2Fblog%2Farize-release-notes-sep-5-2024&title=Arize%20Release%20Notes:%20Sep%205,%202024) #### Suggested reading ![](https://arize.com/wp-content/uploads/2025/11/arize-ax-october-updates-2025.jpg) [New In Arize AX: Tags, Data Fabric, Automatic Threshold Ranges for Monitors and More](https://arize.com/blog/new-in-arize-ax-tags-data-fabric-automatic-threshold-ranges-for-monitors-and-more/) ![adb - Arize AI](https://arize.com/wp-content/uploads/2025/09/abd-cover-image-dark-3.jpg) [adb Benchmarks](https://arize.com/blog/adb-benchmarks/) Sign up for our monthly newsletter, The Evaluator. -------------------------------------------------- [Subscribe](https://arize.com/blog/arize-release-notes-sep-5-2024#blog-subscribe-modal) Subscribe to The Evaluator -------------------------- We’ll send you the latest news, expertise, and product updates from Arize. Your inbox is sacred, so we’ll only curate and send the best stuff. \*We’re committed to your privacy. Arize uses the information you provide to contact you about relevant content, products, and services. You may unsubscribe from these communications at any time. For more information, check out our [privacy policy](https://arize.com/privacy-policy) . --- # Arize AI + MongoDB: Leveraging Agent Evaluation and Memory to Build Robust Agentic Systems - Arize AI ![](https://arize.com/wp-content/uploads/2024/09/MongoDB-partnership-blog@2x.jpg) Arize AI + MongoDB: Leveraging Agent Evaluation and Memory to Build Robust Agentic Systems ========================================================================================== Published September 30, 2024 ---------------------------- ![](https://arize.com/wp-content/uploads/2024/09/MongoDB-partnership-blog@2x.jpg) In the evolving landscape of artificial intelligence, agentic systems—autonomous agents capable of making decisions and learning from feedback loops in their environment—are becoming increasingly sophisticated. At the same time, as retrieval augmented generation (RAG) applications become more complex, a critical component of these systems is memory. AI agents depend on memory to perform effectively, adapt to new situations, and make informed decisions. However, a single request to these systems can generate hundreds of calls under the hood, making debugging issues and understanding how they come to their outputs challenging for the AI engineering teams building and maintaining applications.  As more businesses begin adopting and integrating LLM applications with robust agentic systems, it is imperative that teams are able to evaluate, troubleshoot, and improve the performance of their applications. Arize AI and MongoDB have come together to help AI engineers develop and deploy their LLM applications with confidence.  As large language models (LLMs) continue to advance, efficient and scalable memory systems are essential. Vector databases are critical in this context, particularly for managing the memory of AI agents. MongoDB provides a full document data store and a robust query api that supports  integrated full text search and vector search capabilities which lays a powerful foundation for implementing these systems, and when combined with Arize AI’s advanced evaluation and observability capabilities, it becomes possible to build, troubleshoot, and optimize robust agentic systems.  **Fast and Scalable Retrieval** ------------------------------- For AI engineers working with RAG-based systems, the combination of MongoDB and Arize AI offers a powerful toolkit for building and maintaining generative-powered systems. MongoDB’s vector search capabilities ensure rapid, scalable retrieval of relevant vectors that RAG applications rely on. This capability is essential for real-time memory recall, enabling agents to perform effectively even as data volumes grow.  Arize AI’s platform offers comprehensive observability tools that allow engineers to trace the flow of data through the AI system, from input to final output. This tracing capability is especially valuable in complex, multi-layered architectures like RAG, where understanding the impact of each component on the final result is critical for effective debugging and optimization. **Contextual Memory Management and Interactive RAG Strategy** ------------------------------------------------------------- By leveraging its document-based architecture and vector search capabilities, MongoDB’s flexible schema allows agents to manage contextual memory effectively. By storing complex documents that include vectors and related context, MongoDB helps agents maintain a nuanced understanding of interactions, ensuring coherence and context-awareness. MongoDB’s schema flexibility also supports the differentiation between short-term and long-term memory, enabling agents to manage their memory resources efficiently. Arize offers a library of LLM evaluations that are pre-tested on tasks such as code generation, Q&A accuracy, embedding cluster summarization, and more. Leveraging the LLM as a judge approach, an evaluator LLM scores application output based on relevance, toxicity, etc. LLM-generated explanations detail why the output was scored a certain way, providing a scaled mechanism to understand how the LLM application came to its output and potential ways to improve performance of these complex systems. ![Eval chart](https://arize.com/wp-content/uploads/2024/09/image2.png) Employing an interactive RAG approach, allows the knowledge base to access and process real-time information from external sources such as online databases and APIs. This enables it to provide up-to-date and relevant responses, making it suitable for applications requiring access to constantly changing data. Powered by MongoDB Atlas, interactive RAG enables teams to dynamically adjust their RAG strategy in real-time, using the function calling API of large language models—optimizing for a truly interactive and personalized experience.  ![Traces screenshot](https://arize.com/wp-content/uploads/2024/09/image4.png) Leveraging Arize’s retrieval evaluation with explanations, developers can quickly see that the LLM hallucinated, see the exact chunk of the retrieval used in the call, and receive an explanation of why the LLM was incorrect: ![Explanation of why the LLM was incorrect](https://arize.com/wp-content/uploads/2024/09/image1.png) **Visibility into the System with Tracing** ------------------------------------------- Arize’s LLM tracing capabilities provide visibility into each call in an LLM-powered system to facilitate application development and troubleshooting. This is especially critical for systems that implement an orchestration or agentic framework, as those abstractions can mask an immense number of distributed system calls that are nearly impossible to debug without programmatic tracing.  ![Visibility into the system with tracing](https://arize.com/wp-content/uploads/2024/09/image3.png) Arize Tracing provides visibility into entire system. **Evaluate Agent and Retriever Performance** -------------------------------------------- Evaluations help teams understand their LLM application’s performance. Evals can be used to measure an application across several dimensions such as correctness, hallucination, relevance, latency, tool calling and more. This enables teams to evaluate their application’s performance at every step.  Arize has built an evaluation framework with: * **Pre-tested Evaluators Backed by Research:** Arize evaluators are thoroughly tested against the latest capabilities from LLM providers, such as needle in a haystack tests. * **Multi-level Custom Evaluation:** Arize provides several types of evaluations complete with explanations out of the box, enabling users to customize their evaluation using their own criteria and prompt templates. * **Designed for Speed:** Arize evals are designed to handle large volumes of data, with parallel calls, batch processing, and rate limiting. * **Ease of Onboarding:** Arize’s framework integrates seamlessly with popular LLM frameworks like LangChain and LlamaIndex, providing straightforward setup and execution. * **Extensive Compatibility:** Arize’s library is compatible with all common LLMs and offers unparalleled RAG debugging and troubleshooting. Arize also offers teams the option to create and run automated actions on your LLM spans as their application scales—known as Tasks. During development, engineers can automatically run an evaluation on every trace that doesn’t have an evaluation yet. In production, they can sample a set of your traffic to run evaluations for monitoring that run every few minutes.  ![llm evals screenshot](https://arize.com/wp-content/uploads/2024/09/image5-1.png) Arize Tasks – Evaluating Tool Call Performance **Curated Datasets for Experimentation** ---------------------------------------- In AI development, it’s hard to understand how a change will affect performance. This breaks the dev flow, making iteration more guesswork than engineering. In Arize, datasets and experiments help solve this. Developers can select examples of interest in Arize—such as cases where an agent failed to perform—to then run experiments and optimize for performance. Teams can track improvements to their prompts, LLM, or other parts of their application across experiments in order to continuously iterate on and improve their application. This systematic experimentation is vital for identifying the optimal configuration to balance agent performance and efficiency.  Developers can leverage Arize’s prompt + data playground to replay problems within their application, test different prompts across their data, as an effective way to improve the outputs of their applications. The interactive environment provides developers real-time feedback into the results, providing valuable insight during experimentation.  ![prompt playground screenshot](https://arize.com/wp-content/uploads/2024/09/image6-1.png) Experimenting in Arize’s Prompt Playground on a Dataset **Develop and Deploy Robust Agentic Systems with Confidence** Vector databases are essential for managing memory in LLM-based agentic systems, and MongoDB offers a robust solution for storing, retrieving, and managing this data. When combined with Arize AI’s advanced features like Tracing, Datasets, Experiments, and LLM Evaluations, developers have a comprehensive toolkit for building, evaluating, and optimizing their AI agents: * **Data Ingestion and Storage:** MongoDB has a flexible schema allowing it to  ingest and store diverse datasets, including structured data, time series dataset, graph dataset, vector embeddings, and unstructured text. This data forms the knowledge base from which the AI agent draws context and information. * **Unified Query API:** MongoDB is a document data store with full fledged query api to query data with multiple patterns such as vector search, vector search with pre-filtering, vector search with post-filter, full text search, hybrid search. The post-filtering step in the query pipeline also allows the user to couple graph traversal of data along with vector search. When the AI agent receives a query or task, it first retrieves relevant information from MongoDB using one or more techniques mentioned above. The retrieved information is then passed to the LLM for further processing. * **LLM Processing and Generation:** The LLM, powered by models like OpenAI’s latest embeddings, processes the retrieved data to generate a response or decision. This process is iterative, with the agent potentially making several retrievals and adjustments before finalizing its output. * **Agent Evaluation and Fine-Tuning:** As the AI agent completes its task, Arize AI’s evaluation tools kick in, scoring the quality of the output and identifying any areas for improvement. This feedback loop is crucial for refining the agent’s behavior over time, ensuring that it remains effective and reliable as new data and scenarios are encountered. By leveraging MongoDB’s scalability and Arize AI’s powerful evaluation and troubleshooting capabilities, developers can ensure that their agentic systems not only perform well in the short term but also adapt and improve over time. This combination of technologies ensures that AI agents are equipped to handle complex, real-world scenarios with reliability, safety, and efficiency. See how MongoDB and Arize work together **[in this Colab tutorial.](https://colab.research.google.com/github/Arize-ai/phoenix/blob/main/tutorials/integrations/tracing_and_evals_with_mongodb_and_llama_index.ipynb)**  Subscribe to The Evaluator -------------------------- We’ll send you the latest news, expertise, and product updates from Arize. Your inbox is sacred, so we’ll only curate and send the best stuff. \*We’re committed to your privacy. Arize uses the information you provide to contact you about relevant content, products, and services. You may unsubscribe from these communications at any time. For more information, check out our [privacy policy](https://arize.com/privacy-policy) . --- # Arize Release Notes: AI Search V2, Copilot Updates, and More - Arize AI ![](https://arize.com/wp-content/uploads/2023/01/Sarah_headshot-196x196.jpg "Sarah_headshot") [Sarah Welsh](https://arize.com/author/sarah-welsh/) Contributor Share * [![](https://arize.com/wp-content/themes/arize-2022/images/icon-tw-circle.svg)](https://twitter.com/intent/tweet?url=https%3A%2F%2Farize.com%2Fblog%2Farize-release-notes-ai-search-v2-copilot-updates-and-more%2F&text=Arize%20Release%20Notes:%20AI%20Search%20V2,%20Copilot%20Updates,%20and%20More) * [![](https://arize.com/wp-content/themes/arize-2022/images/icon-fb-circle.svg)](https://www.facguide.com/sharer/sharer.php?u=https://arize.com/blog/arize-release-notes-ai-search-v2-copilot-updates-and-more/&picture=https://arize.com) * [![](https://arize.com/wp-content/themes/arize-2022/images/icon-in-circle.svg)](https://www.linkedin.com/shareArticle?mini=true&url=https%3A%2F%2Farize.com%2Fblog%2Farize-release-notes-ai-search-v2-copilot-updates-and-more%2F&title=Arize%20Release%20Notes:%20AI%20Search%20V2,%20Copilot%20Updates,%20and%20More) ![](https://arize.com/wp-content/themes/arize-2022/images/icon-newsletter.svg) Subscribe to the Arize blog [Get the latest](https://arize.com/blog/arize-release-notes-ai-search-v2-copilot-updates-and-more/#blog-subscribe-modal) #### On this page #### Suggested reading ![](https://arize.com/wp-content/uploads/2025/11/arize-ax-october-updates-2025.jpg) [New In Arize AX: Tags, Data Fabric, Automatic Threshold Ranges for Monitors and More](https://arize.com/blog/new-in-arize-ax-tags-data-fabric-automatic-threshold-ranges-for-monitors-and-more/) ![adb - Arize AI](https://arize.com/wp-content/uploads/2025/09/abd-cover-image-dark-3.jpg) [adb Benchmarks](https://arize.com/blog/adb-benchmarks/) ![Release Notes Title and Arize Logo](https://arize.com/wp-content/uploads/2024/09/Release-notes-9-19.jpg "Release notes 9-19") Arize Release Notes: AI Search V2, Copilot Updates, and More ============================================================ Published Sep 19, 2024 * [Product Releases](https://arize.com/blog/?cat=product-releases) * [Release Notes](https://arize.com/blog/?cat=release-notes) ![](https://arize.com/wp-content/uploads/2023/01/Sarah_headshot-196x196.jpg "Sarah_headshot") #### [Sarah Welsh](https://arize.com/author/sarah-welsh/) ##### Contributor Welcome to our regular update on new releases, enhancements, and changes. What’s New ---------- ### AI Search V2 We’re excited to announce the release of AI Search V2, packed with new features and improvements designed to enhance the user experience. Here’s what’s new: * **Column Search** (Improved) The original AI search skill now offers refined semantic search capabilities within a single column based on user criteria. Try it out: _“Find me confused inputs”_ * **Table Search** (New) Our newest skill allows users to search across multiple or all columns within a table, making it easier to catch patterns and outliers. Try it out: _“Find inputs that reference pricing that are hallucinated”_ * **Text to Filter** (New) This skill generates precise query filters based on natural language commands. Simply use “filter by” to trigger it and narrow down data based on your needs. Try it out: _“Filter by input contains SDK”_ * **LLM Analysis Lite** (New) This lightweight analysis skill helps users find patterns in their data and provides filter suggestions to improve their search results. Try it out: _“What are the top 5 types of questions asked?”_ ![Ai Search V2 screenshot](https://arize.com/wp-content/uploads/2024/09/Ai-Search.png) ### Documentation Questions in Copilot Copilot can now answer questions about the Arize product! Try it out: _“How do I send in traces?”_ _Note: Arize has partnered with_ [_Run LLM_](https://runllm.com/) _to provide support via Copilot. Users must first authorize the use of Run LLM. Only the user’s question is sent to Run LLM, no other data will be included._ [Learn more about annotations here.](https://docs.arize.com/arize/llm-evaluation-and-annotations/annotations) Enhancements ------------ ### Experiments Overview Visualization There’s a new way to visualize experiment results on the Experiments Overview page. Users can now see up to the 10 most recent experiments and select which evaluations they’d like to visualize. This is just the beginning of our investment in visualization tools, so stay tuned for more exciting updates! ![Experiments overview ](https://arize.com/wp-content/uploads/2024/09/Experiments-overview.avif) ### Data API Users can now query for drift over time using GraphQL. [Learn more here](https://docs.arize.com/arize/api-reference/graphql-api/metrics-api#drift-over-time) . ### Admin API Users can now query for organization users, update space membership or delete a user from a space. [Learn more here](https://docs.arize.com/arize/api-reference/graphql-api/admin-api#query-for-users-in-an-organization) . New Content ----------- ✏️ [Tracing a Groq Application](https://arize.com/blog/tracing-groq/) 🤖 [Composable Interventions for Language Models](https://arize.com/blog/composable-interventions-for-language-models/) 🧠 [Creating and Validating Synthetic Datasets for LLM Evaluation & Experimentation](https://arize.com/blog/creating-and-validating-synthetic-datasets-for-llm-evaluation-experimentation/) Share * [![](https://arize.com/wp-content/themes/arize-2022/images/icon-tw-circle.svg)](https://twitter.com/intent/tweet?url=https%3A%2F%2Farize.com%2Fblog%2Farize-release-notes-ai-search-v2-copilot-updates-and-more%2F&text=Arize%20Release%20Notes:%20AI%20Search%20V2,%20Copilot%20Updates,%20and%20More) * [![](https://arize.com/wp-content/themes/arize-2022/images/icon-fb-circle.svg)](https://www.facguide.com/sharer/sharer.php?u=https://arize.com/blog/arize-release-notes-ai-search-v2-copilot-updates-and-more/&picture=https://arize.com) * [![](https://arize.com/wp-content/themes/arize-2022/images/icon-in-circle.svg)](https://www.linkedin.com/shareArticle?mini=true&url=https%3A%2F%2Farize.com%2Fblog%2Farize-release-notes-ai-search-v2-copilot-updates-and-more%2F&title=Arize%20Release%20Notes:%20AI%20Search%20V2,%20Copilot%20Updates,%20and%20More) #### Suggested reading ![](https://arize.com/wp-content/uploads/2025/11/arize-ax-october-updates-2025.jpg) [New In Arize AX: Tags, Data Fabric, Automatic Threshold Ranges for Monitors and More](https://arize.com/blog/new-in-arize-ax-tags-data-fabric-automatic-threshold-ranges-for-monitors-and-more/) ![adb - Arize AI](https://arize.com/wp-content/uploads/2025/09/abd-cover-image-dark-3.jpg) [adb Benchmarks](https://arize.com/blog/adb-benchmarks/) Sign up for our monthly newsletter, The Evaluator. -------------------------------------------------- [Subscribe](https://arize.com/blog/arize-release-notes-ai-search-v2-copilot-updates-and-more/#blog-subscribe-modal) Subscribe to The Evaluator -------------------------- We’ll send you the latest news, expertise, and product updates from Arize. Your inbox is sacred, so we’ll only curate and send the best stuff. \*We’re committed to your privacy. Arize uses the information you provide to contact you about relevant content, products, and services. You may unsubscribe from these communications at any time. For more information, check out our [privacy policy](https://arize.com/privacy-policy) . --- # Arize Release Notes: Aug 23, 2024 - Arize AI ![](https://arize.com/wp-content/uploads/2022/03/David-Burch-1-196x196.png "David-Burch") [David Burch](https://arize.com/author/david-burch/) Share * [![](https://arize.com/wp-content/themes/arize-2022/images/icon-tw-circle.svg)](https://twitter.com/intent/tweet?url=https%3A%2F%2Farize.com%2Fblog%2Farize-release-notes-aug-23-2024%2F&text=Arize%20Release%20Notes:%20Aug%2023,%202024) * [![](https://arize.com/wp-content/themes/arize-2022/images/icon-fb-circle.svg)](https://www.facguide.com/sharer/sharer.php?u=https://arize.com/blog/arize-release-notes-aug-23-2024/&picture=https://arize.com) * [![](https://arize.com/wp-content/themes/arize-2022/images/icon-in-circle.svg)](https://www.linkedin.com/shareArticle?mini=true&url=https%3A%2F%2Farize.com%2Fblog%2Farize-release-notes-aug-23-2024%2F&title=Arize%20Release%20Notes:%20Aug%2023,%202024) ![](https://arize.com/wp-content/themes/arize-2022/images/icon-newsletter.svg) Subscribe to the Arize blog [Get the latest](https://arize.com/blog/arize-release-notes-aug-23-2024/#blog-subscribe-modal) #### On this page #### Suggested reading ![](https://arize.com/wp-content/uploads/2025/11/arize-ax-october-updates-2025.jpg) [New In Arize AX: Tags, Data Fabric, Automatic Threshold Ranges for Monitors and More](https://arize.com/blog/new-in-arize-ax-tags-data-fabric-automatic-threshold-ranges-for-monitors-and-more/) ![adb - Arize AI](https://arize.com/wp-content/uploads/2025/09/abd-cover-image-dark-3.jpg) [adb Benchmarks](https://arize.com/blog/adb-benchmarks/) ![](https://arize.com/wp-content/uploads/2024/08/release-notes-20240823.png "release-notes-20240823") Arize Release Notes: Aug 23, 2024 ================================= Published Aug 23, 2024 * [Product Releases](https://arize.com/blog/?cat=product-releases) * [Release Notes](https://arize.com/blog/?cat=release-notes) ![](https://arize.com/wp-content/uploads/2022/03/David-Burch-1-196x196.png "David-Burch") #### [David Burch](https://arize.com/author/david-burch/) Welcome to our regular update on new releases, enhancements, and changes. What’s New ---------- ### Create Spaces Programmatically Users can now [create spaces programmatically with graphQL](https://docs.arize.com/arize/api-reference/graphql-api/admin-api) . ### Online Evals Update We added support for [three new LLM integrations for online tasks](https://docs.arize.com/arize/large-language-models/tasks-for-online-evals) : Azure OpenAI, Bedrock, and Vertex / Gemini. ### Event-Based Snowflake Jobs Users can now [trigger Snowflake queries using graphQL](https://docs.arize.com/arize/api-reference/graphql-api/table-importer-api#event-based-table-import-jobs) . Enhancements ------------ ### Python SDK v7.20.1 * Enable delayed tags for stream logging * Experiment eval metadata * Ingest data to Arize using space\_id Learn about Python SDK fixes and improvements [here](https://pypi.org/project/arize/) . 📚 New Content -------------- * 🪡 [How To Trace Your Haystack LLM App](https://arize.com/blog/trace-your-haystack-application-with-phoenix/) * 🦙 [LlamaIndex Workflows](https://arize.com/blog/llamaindex-workflows-a-new-way-to-build-cyclical-agents/) : Navigating a New Way To Build Cyclical Agents * 🚧 [Types of LLM Guardrails](https://arize.com/blog-course/llm-guardrails-types-of-guards/) * ✋ How To Use [Annotations for Human Feedback](https://arize.com/blog/how-to-use-annotations-to-collect-human-feedback-on-your-llm-application/) * 🧑🏽‍⚖️ [Judging the Judges: Evaluating Alignment and Vulnerabilities in LLMs-as-Judges](https://arize.com/blog/judging-the-judges-llm-as-a-judge/) * 🦙 Breaking Down [Llama 3](https://arize.com/blog/breaking-down-meta-llama-3/) * ✨ [How Flipkart Leverages Generative AI for 600 Million Users](https://arize.com/blog/how-flipkart-leverages-generative-ai-for-600-million-users/) * 🏥 [Atropos Health](https://arize.com/blog/how-atropos-health-accelerates-research-with-llm-observability/) : Leveraging LLM Observability Share * [![](https://arize.com/wp-content/themes/arize-2022/images/icon-tw-circle.svg)](https://twitter.com/intent/tweet?url=https%3A%2F%2Farize.com%2Fblog%2Farize-release-notes-aug-23-2024%2F&text=Arize%20Release%20Notes:%20Aug%2023,%202024) * [![](https://arize.com/wp-content/themes/arize-2022/images/icon-fb-circle.svg)](https://www.facguide.com/sharer/sharer.php?u=https://arize.com/blog/arize-release-notes-aug-23-2024/&picture=https://arize.com) * [![](https://arize.com/wp-content/themes/arize-2022/images/icon-in-circle.svg)](https://www.linkedin.com/shareArticle?mini=true&url=https%3A%2F%2Farize.com%2Fblog%2Farize-release-notes-aug-23-2024%2F&title=Arize%20Release%20Notes:%20Aug%2023,%202024) #### Suggested reading ![](https://arize.com/wp-content/uploads/2025/11/arize-ax-october-updates-2025.jpg) [New In Arize AX: Tags, Data Fabric, Automatic Threshold Ranges for Monitors and More](https://arize.com/blog/new-in-arize-ax-tags-data-fabric-automatic-threshold-ranges-for-monitors-and-more/) ![adb - Arize AI](https://arize.com/wp-content/uploads/2025/09/abd-cover-image-dark-3.jpg) [adb Benchmarks](https://arize.com/blog/adb-benchmarks/) Sign up for our monthly newsletter, The Evaluator. -------------------------------------------------- [Subscribe](https://arize.com/blog/arize-release-notes-aug-23-2024/#blog-subscribe-modal) Subscribe to The Evaluator -------------------------- We’ll send you the latest news, expertise, and product updates from Arize. Your inbox is sacred, so we’ll only curate and send the best stuff. \*We’re committed to your privacy. Arize uses the information you provide to contact you about relevant content, products, and services. You may unsubscribe from these communications at any time. For more information, check out our [privacy policy](https://arize.com/privacy-policy) . --- # Arize Release Notes: New Copilot Skills, Local Explainability, and More. - Arize AI ![](https://arize.com/wp-content/uploads/2023/01/Sarah_headshot-196x196.jpg "Sarah_headshot") [Sarah Welsh](https://arize.com/author/sarah-welsh/) Contributor Share * [![](https://arize.com/wp-content/themes/arize-2022/images/icon-tw-circle.svg)](https://twitter.com/intent/tweet?url=https%3A%2F%2Farize.com%2Fblog%2Farize-release-notes-new-copilot-skills-local-explainability-and-more%2F&text=Arize%20Release%20Notes:%20New%20Copilot%20Skills,%20Local%20Explainability,%20and%20More.) * [![](https://arize.com/wp-content/themes/arize-2022/images/icon-fb-circle.svg)](https://www.facguide.com/sharer/sharer.php?u=https://arize.com/blog/arize-release-notes-new-copilot-skills-local-explainability-and-more/&picture=https://arize.com) * [![](https://arize.com/wp-content/themes/arize-2022/images/icon-in-circle.svg)](https://www.linkedin.com/shareArticle?mini=true&url=https%3A%2F%2Farize.com%2Fblog%2Farize-release-notes-new-copilot-skills-local-explainability-and-more%2F&title=Arize%20Release%20Notes:%20New%20Copilot%20Skills,%20Local%20Explainability,%20and%20More.) ![](https://arize.com/wp-content/themes/arize-2022/images/icon-newsletter.svg) Subscribe to the Arize blog [Get the latest](https://arize.com/blog/arize-release-notes-new-copilot-skills-local-explainability-and-more/#blog-subscribe-modal) #### On this page #### Suggested reading ![](https://arize.com/wp-content/uploads/2025/11/arize-ax-october-updates-2025.jpg) [New In Arize AX: Tags, Data Fabric, Automatic Threshold Ranges for Monitors and More](https://arize.com/blog/new-in-arize-ax-tags-data-fabric-automatic-threshold-ranges-for-monitors-and-more/) ![adb - Arize AI](https://arize.com/wp-content/uploads/2025/09/abd-cover-image-dark-3.jpg) [adb Benchmarks](https://arize.com/blog/adb-benchmarks/) ![Arize logo with text and reads: Release notes, November 7, 2024](https://arize.com/wp-content/uploads/2024/11/Release-notes-11-7.jpg "Release notes 11-7") Arize Release Notes: New Copilot Skills, Local Explainability, and More. ======================================================================== Published Nov 7, 2024 * [Product Releases](https://arize.com/blog/?cat=product-releases) * [Release Notes](https://arize.com/blog/?cat=release-notes) ![](https://arize.com/wp-content/uploads/2023/01/Sarah_headshot-196x196.jpg "Sarah_headshot") #### [Sarah Welsh](https://arize.com/author/sarah-welsh/) ##### Contributor Welcome to our regular update on new releases, enhancements, and changes. What’s New ---------- ### New Copilot Skills **Custom Metric Skill**: Copilot now writes custom metrics! Users can generate their desired metric by having copilot translate natural language descriptions or existing code (e.g., SQL, Python) into AQL. [Learn more](https://docs.arize.com/arize/llm-monitoring-and-guardrails/custom-metrics-api/arizeql-generator) ![Custom metrics](https://arize.com/wp-content/uploads/2024/11/custom_metrics.gif) **Embedding Summarization Skill**: Copilot now works for embeddings! Users can select embedding data point and Copilot will analyze for patterns and insights. [Learn more](https://docs.arize.com/arize/computer-vision-cv/how-to-cv/embedding-summarization) ![cluster summarization](https://arize.com/wp-content/uploads/2024/11/cluster_summarization.gif) Enhancements ------------ ### Local Explainability Report Local Explainability is now live, providing both a table view and waterfall style plot for detailed, per-feature SHAP values on individual predictions. [Learn more](https://docs.arize.com/arize/machine-learning/how-to-ml/explainability/interpreting-and-analyzing-feature-importance-values#local-feature-importance) ![](https://arize.com/wp-content/uploads/2024/11/local_explainability.gif) ### Experiment Over Time Widget This widget allows users to integrate experiment data directly into their dashboards for ongoing visibility and analysis. Users can now: * Select dataset of interest * Choose specific evaluations they want to visualize over time * Complete with direct connectivity to experiment details, making it easy to access the individual experiment results [Learn more](https://docs.arize.com/arize/llm-monitoring-and-guardrails/dashboards/widgets#experiments) ![](https://arize.com/wp-content/uploads/2024/11/experiment_widget.gif) ### Full Function Calling Replay in Prompt Playground Now users can follow the full [function calling tutorial](https://platform.openai.com/docs/guides/function-calling) from OpenAI and iterate on different functions in different messages from within the Prompt Playground. ![full function calling replay](https://arize.com/wp-content/uploads/2024/11/full-function-calling-replay.avif) ### Instrumentation Enhancements * **Context Attribute Propagation**: Arize now has a set of utilities (eg: using\_session) that allow users to set properties on context. All of these properties will be picked up by all of our auto instrumentations and added to spans. [Learn more](https://docs.arize.com/arize/llm-tracing/how-to-tracing-manual/hybrid-instrumentation#add-attributes-to-multiple-spans-at-once)   * **Typescript Trace Configuration**: Typescript auto instrumentations now accept a trace configuration which allows for data masking and configuration of attribute values on spans. [Learn more](https://docs.arize.com/arize/llm-tracing/how-to-tracing-manual/masking-span-attributes)   * **Vercel AI SDK**: Users can now ingest traces created by the Vercel AI SDK into Arize. [Learn more](https://docs.arize.com/arize/llm-tracing/tracing-integrations-auto/vercel-ai-sdk)   * **LangChain Auto Instrumentation**: Arize’s LangChain auto instrumentation now supports langchain.js version 0.3 and is backwards compatible with all previous versions. [Learn more](https://docs.arize.com/arize/llm-tracing/tracing-integrations-auto/langchain)   📚 New Content -------------- The latest video tutorials, paper readings, ebooks, self-guided learning modules, and technical posts: 🧑‍🏫 [Prompt Optimization Course](https://arize.com/course/prompt-optimization/) 📊 [Evaluation Workflows to Accelerate Generative App Development and AI ROI](https://arize.com/blog/arize-vertex-ai-api/) 🐝 [Swarm: OpenAI’s Experimental Approach to Multi-Agent Systems](https://arize.com/blog/swarm-openai-experimental-approach-to-multi-agent-systems/) ✏️ [LLM Evaluation Course](https://arize.com/llm-evaluation/) 🤖[Techniques for Self-Improving LLM Evals](https://arize.com/blog/techniques-for-self-improving-llm-evals/) Share * [![](https://arize.com/wp-content/themes/arize-2022/images/icon-tw-circle.svg)](https://twitter.com/intent/tweet?url=https%3A%2F%2Farize.com%2Fblog%2Farize-release-notes-new-copilot-skills-local-explainability-and-more%2F&text=Arize%20Release%20Notes:%20New%20Copilot%20Skills,%20Local%20Explainability,%20and%20More.) * [![](https://arize.com/wp-content/themes/arize-2022/images/icon-fb-circle.svg)](https://www.facguide.com/sharer/sharer.php?u=https://arize.com/blog/arize-release-notes-new-copilot-skills-local-explainability-and-more/&picture=https://arize.com) * [![](https://arize.com/wp-content/themes/arize-2022/images/icon-in-circle.svg)](https://www.linkedin.com/shareArticle?mini=true&url=https%3A%2F%2Farize.com%2Fblog%2Farize-release-notes-new-copilot-skills-local-explainability-and-more%2F&title=Arize%20Release%20Notes:%20New%20Copilot%20Skills,%20Local%20Explainability,%20and%20More.) #### Suggested reading ![](https://arize.com/wp-content/uploads/2025/11/arize-ax-october-updates-2025.jpg) [New In Arize AX: Tags, Data Fabric, Automatic Threshold Ranges for Monitors and More](https://arize.com/blog/new-in-arize-ax-tags-data-fabric-automatic-threshold-ranges-for-monitors-and-more/) ![adb - Arize AI](https://arize.com/wp-content/uploads/2025/09/abd-cover-image-dark-3.jpg) [adb Benchmarks](https://arize.com/blog/adb-benchmarks/) Sign up for our monthly newsletter, The Evaluator. -------------------------------------------------- [Subscribe](https://arize.com/blog/arize-release-notes-new-copilot-skills-local-explainability-and-more/#blog-subscribe-modal) Subscribe to The Evaluator -------------------------- We’ll send you the latest news, expertise, and product updates from Arize. Your inbox is sacred, so we’ll only curate and send the best stuff. \*We’re committed to your privacy. Arize uses the information you provide to contact you about relevant content, products, and services. You may unsubscribe from these communications at any time. For more information, check out our [privacy policy](https://arize.com/privacy-policy) . --- # Arize Release Notes: Embeddings Tracing, Experiments Details, and More. - Arize AI ![](https://arize.com/wp-content/uploads/2023/01/Sarah_headshot-196x196.jpg "Sarah_headshot") [Sarah Welsh](https://arize.com/author/sarah-welsh/) Contributor Share * [![](https://arize.com/wp-content/themes/arize-2022/images/icon-tw-circle.svg)](https://twitter.com/intent/tweet?url=https%3A%2F%2Farize.com%2Fblog%2Farize-release-notes-embeddings-tracing-experiments-details-and-more%2F&text=Arize%20Release%20Notes:%20Embeddings%20Tracing,%20Experiments%20Details,%20and%20More.) * [![](https://arize.com/wp-content/themes/arize-2022/images/icon-fb-circle.svg)](https://www.facguide.com/sharer/sharer.php?u=https://arize.com/blog/arize-release-notes-embeddings-tracing-experiments-details-and-more/&picture=https://arize.com) * [![](https://arize.com/wp-content/themes/arize-2022/images/icon-in-circle.svg)](https://www.linkedin.com/shareArticle?mini=true&url=https%3A%2F%2Farize.com%2Fblog%2Farize-release-notes-embeddings-tracing-experiments-details-and-more%2F&title=Arize%20Release%20Notes:%20Embeddings%20Tracing,%20Experiments%20Details,%20and%20More.) ![](https://arize.com/wp-content/themes/arize-2022/images/icon-newsletter.svg) Subscribe to the Arize blog [Get the latest](https://arize.com/blog/arize-release-notes-embeddings-tracing-experiments-details-and-more/#blog-subscribe-modal) #### On this page #### Suggested reading ![](https://arize.com/wp-content/uploads/2025/11/arize-ax-october-updates-2025.jpg) [New In Arize AX: Tags, Data Fabric, Automatic Threshold Ranges for Monitors and More](https://arize.com/blog/new-in-arize-ax-tags-data-fabric-automatic-threshold-ranges-for-monitors-and-more/) ![adb - Arize AI](https://arize.com/wp-content/uploads/2025/09/abd-cover-image-dark-3.jpg) [adb Benchmarks](https://arize.com/blog/adb-benchmarks/) ![Release Notes October 3, 2024](https://arize.com/wp-content/uploads/2024/10/Release-notes-10-3.jpg "Release notes 10-3") Arize Release Notes: Embeddings Tracing, Experiments Details, and More. ======================================================================= Published Oct 3, 2024 * [Product Releases](https://arize.com/blog/?cat=product-releases) * [Release Notes](https://arize.com/blog/?cat=release-notes) ![](https://arize.com/wp-content/uploads/2023/01/Sarah_headshot-196x196.jpg "Sarah_headshot") #### [Sarah Welsh](https://arize.com/author/sarah-welsh/) ##### Contributor Welcome to our regular update on new releases, enhancements, and changes. What’s New ---------- Embeddings Tracing ------------------ With Embeddings Tracing, you can effortlessly select embedding spans and dive straight into the UMAP visualizer, simplifying troubleshooting for your genAI applications. How this works: * Users can now select embedding spans and go to the embedding visualizer (also available on project level nav for all generative models) * All embedding spans get pulled over for UMAP + clustering * Users can select an individual point/cluster and the embedding span attributes get pulled over * “Color by” span attributes is also available, and whichever attribute users “color by” can also be viewed as a column in the table _Note: this functionality is only viewable for embedding spans after October 2, 2024._ [Learn more about our embeddings visualizer here](https://docs.arize.com/arize/computer-vision-cv/how-to-cv/embedding-and-cluster-analyzer) . [https://arize.com/wp-content/uploads/2024/10/Embeddings-Tracing.mp4](https://arize.com/wp-content/uploads/2024/10/Embeddings-Tracing.mp4) Experiments Details Visualization --------------------------------- Users can now view a detailed breakdown of labels for their experiments on the Experiments Details page. Note: this feature is currently only available for label-based evaluations. Support for score-exclusive evaluations will be coming in a future update. ![Compare Experiments](https://arize.com/wp-content/uploads/2024/10/Compare-Experiments.avif) Experiments Details Visualization Enhancements ------------ ### Prompt Playground Improvements * **Full OpenAI Models Support:** We’ve added full support for all available OpenAI models in the playground including the o1-mini and o1-preview. * **OpenAI Function/Tool Calls:** With OpenAI’s addition of function calling in their models, we have added support in Arize starting from the trace all the way to the playground. View function/tool calls in your traces, and open the playground to further experiment and test. * **Full-screen Data Mode:** Enter full screen mode to view your data more easily. * **Features For Datasets:** * **Prompt Overriding:** When using a dataset to test out different prompts, users can now replace the prompt template with the dataset. * **Pop Up Windows for Long Outputs & Variables:** Viewing longer outputs is now easier with pop up windows. Along with these features, we’ve added improvements such as better input variable behavior, autocompletion enhancements, support for mustache/f-string input variables, and more. ![prompt playground](https://arize.com/wp-content/uploads/2024/10/Playground.png) Filters Updates --------------- * **Filter History:** We now store the last three filters used by a user! Users can easily access their filter history in the query filters dropdown, making it simpler to reuse filters for future queries. ![filter history ](https://arize.com/wp-content/uploads/2024/10/Filter-histroy.png)Traces Page Updates ------------------------------------------------------------------------------------------------------ * **Parent Spans:** We’ve enhanced the Traces page to give users a better experience when filtering spans. Now, even if the filter doesn’t match the parent span, users will still see the the parent span with the relevant spans nested under it. * **Quick Filters:** We’ve introduced quick filters, allowing users to apply filters directly from the table by hovering over the text to reveal the filter icon. ![Spans](https://arize.com/wp-content/uploads/2024/10/Spans.png) New Arize-Otel Package for LLM App Tracing ------------------------------------------ We heard you, and we made it much simpler to add automatic tracing to your applications! It’s now just a few lines of code to use OpenTelemetry to trace your LLM application. [Check out our new quickstart guide which uses our arize-otel package](https://docs.arize.com/arize/llm-tracing/quickstart-llm) . Workflow Improvements --------------------- * Support for new line creation while reviewing prompts. * Easily add spans to a dataset from the Traces page using the “Add to Dataset” button. * Quickly create an evaluation task within a trace if no evaluations currently exist using the “Setup Task” button. ![trace details](https://arize.com/wp-content/uploads/2024/10/Trace-details.png) 📚 New Content -------------- The latest video tutorials, paper readings, ebooks, self-guided learning modules, and technical posts: 🧑‍⚖️ [Selecting the Right Model for LLM-as-a-Judge Evaluations](https://arize.com/blog/choosing-the-best-llm-evaluation-model/) 🤖 [Leveraging Agent Evaluation and Memory to Build Robust Agentic Systems](https://arize.com/blog/arize-ai-mongodb-agentic-systems/) 🔭 [Exploring OpenAI’s o1 Models](https://arize.com/blog/exploring-openai-o1-preview-and-o1-mini/) 🪞[Breaking Down Reflection Tuning](https://arize.com/blog/breaking-down-reflection-tuning-enhancing-llm-performance-with-self-learning/) Share * [![](https://arize.com/wp-content/themes/arize-2022/images/icon-tw-circle.svg)](https://twitter.com/intent/tweet?url=https%3A%2F%2Farize.com%2Fblog%2Farize-release-notes-embeddings-tracing-experiments-details-and-more%2F&text=Arize%20Release%20Notes:%20Embeddings%20Tracing,%20Experiments%20Details,%20and%20More.) * [![](https://arize.com/wp-content/themes/arize-2022/images/icon-fb-circle.svg)](https://www.facguide.com/sharer/sharer.php?u=https://arize.com/blog/arize-release-notes-embeddings-tracing-experiments-details-and-more/&picture=https://arize.com) * [![](https://arize.com/wp-content/themes/arize-2022/images/icon-in-circle.svg)](https://www.linkedin.com/shareArticle?mini=true&url=https%3A%2F%2Farize.com%2Fblog%2Farize-release-notes-embeddings-tracing-experiments-details-and-more%2F&title=Arize%20Release%20Notes:%20Embeddings%20Tracing,%20Experiments%20Details,%20and%20More.) #### Suggested reading ![](https://arize.com/wp-content/uploads/2025/11/arize-ax-october-updates-2025.jpg) [New In Arize AX: Tags, Data Fabric, Automatic Threshold Ranges for Monitors and More](https://arize.com/blog/new-in-arize-ax-tags-data-fabric-automatic-threshold-ranges-for-monitors-and-more/) ![adb - Arize AI](https://arize.com/wp-content/uploads/2025/09/abd-cover-image-dark-3.jpg) [adb Benchmarks](https://arize.com/blog/adb-benchmarks/) Sign up for our monthly newsletter, The Evaluator. -------------------------------------------------- [Subscribe](https://arize.com/blog/arize-release-notes-embeddings-tracing-experiments-details-and-more/#blog-subscribe-modal) Subscribe to The Evaluator -------------------------- We’ll send you the latest news, expertise, and product updates from Arize. Your inbox is sacred, so we’ll only curate and send the best stuff. \*We’re committed to your privacy. Arize uses the information you provide to contact you about relevant content, products, and services. You may unsubscribe from these communications at any time. For more information, check out our [privacy policy](https://arize.com/privacy-policy) . --- # Arize Release Notes: Copilot Enhancements, Experiment Projects, and More - Arize AI ![](https://arize.com/wp-content/uploads/2023/01/Sarah_headshot-196x196.jpg "Sarah_headshot") [Sarah Welsh](https://arize.com/author/sarah-welsh/) Contributor Share * [![](https://arize.com/wp-content/themes/arize-2022/images/icon-tw-circle.svg)](https://twitter.com/intent/tweet?url=https%3A%2F%2Farize.com%2Fblog%2Farize-release-notes-copilot-enhancements-experiment-projects-and-more%2F&text=Arize%20Release%20Notes:%20Copilot%20Enhancements,%20Experiment%20Projects,%20and%20More) * [![](https://arize.com/wp-content/themes/arize-2022/images/icon-fb-circle.svg)](https://www.facguide.com/sharer/sharer.php?u=https://arize.com/blog/arize-release-notes-copilot-enhancements-experiment-projects-and-more/&picture=https://arize.com) * [![](https://arize.com/wp-content/themes/arize-2022/images/icon-in-circle.svg)](https://www.linkedin.com/shareArticle?mini=true&url=https%3A%2F%2Farize.com%2Fblog%2Farize-release-notes-copilot-enhancements-experiment-projects-and-more%2F&title=Arize%20Release%20Notes:%20Copilot%20Enhancements,%20Experiment%20Projects,%20and%20More) ![](https://arize.com/wp-content/themes/arize-2022/images/icon-newsletter.svg) Subscribe to the Arize blog [Get the latest](https://arize.com/blog/arize-release-notes-copilot-enhancements-experiment-projects-and-more/#blog-subscribe-modal) #### On this page #### Suggested reading ![](https://arize.com/wp-content/uploads/2025/11/arize-ax-october-updates-2025.jpg) [New In Arize AX: Tags, Data Fabric, Automatic Threshold Ranges for Monitors and More](https://arize.com/blog/new-in-arize-ax-tags-data-fabric-automatic-threshold-ranges-for-monitors-and-more/) ![adb - Arize AI](https://arize.com/wp-content/uploads/2025/09/abd-cover-image-dark-3.jpg) [adb Benchmarks](https://arize.com/blog/adb-benchmarks/) ![](https://arize.com/wp-content/uploads/2024/12/Release-notes-12-5.jpg "Release notes 12-5") Arize Release Notes: Copilot Enhancements, Experiment Projects, and More ======================================================================== Published Dec 5, 2024 * [Product Releases](https://arize.com/blog/?cat=product-releases) * [Release Notes](https://arize.com/blog/?cat=release-notes) ![](https://arize.com/wp-content/uploads/2023/01/Sarah_headshot-196x196.jpg "Sarah_headshot") #### [Sarah Welsh](https://arize.com/author/sarah-welsh/) ##### Contributor Welcome to our regular update on new releases, enhancements, and changes. What’s New ---------- ### Copilot Enhancements #### **Span Chat** The Copilot Span Chat skill makes getting value from spans faster and easier. Rather than spending time scrolling through and deciphering span data , teams can now: * Analyze spans to extract key insights * Ask questions to quickly understand span data * Run evaluations on individual spans ![spanchat eval](https://arize.com/wp-content/uploads/2024/12/spanchat_eval.gif) Span Chat Evaluation #### **Dashboard Widget Generator** Building dashboard plots just got way easier. The dashboard skill lets teams… * Create time series plots or distributions from natural language * Translate code (like Plotly) into ready-to-go visualizations * Handle ambiguous filters like “west coast states” and plot multiple widgets at once ![Dashboard generator screenshot](https://arize.com/wp-content/uploads/2024/12/dashboard-generator.png) Dashboard generator #### **Misc. Copilot Updates** * We’ve revamped the main chat experience to be always accessible on the page, with an option to collapse the input bar * The Custom Metric skill now supports a conversational flow, making it easier for users to iterate and refine metrics dynamically Additional Enhancements ----------------------- ### Experiment Projects Experiment traces for a dataset are now consolidated and can be accessed under “Experiment Projects” on the “Projects & Models” page. ![experiment projects screenshot](https://arize.com/wp-content/uploads/2024/12/experiment-projects.png) Experiment projects ### Multi-Class/Label Per-Class Calibration & Chart We’ve just rolled out per-class calibration metrics and calibration chart. Users can see calibration scores for each class separately and view the calibration chart all in one place. * To view per-class calibration simply select calibration from the metric dropdown and choose a class ![Per-class calibration screenshot](https://arize.com/wp-content/uploads/2024/12/calibration.png) Per-class calibration * The calibration chart can be found under the “More Charts” tab ![Calibration chart ](https://arize.com/wp-content/uploads/2024/12/Calibration-chart-2048x690.avif) Calibration chart ### SDK Version 7.29.0 * Log experiments from a previously created dataframe 📚 New Content -------------- The latest video tutorials, paper readings, ebooks, self-guided learning modules, and technical posts: 🧑‍⚖️ [Agent-as-a-Judge: Evaluate Agents with Agents](https://arize.com/blog/agent-as-a-judge-evaluate-agents-with-agents/) 🤖 [LLM-as-a-Judge Evaluation for GenAI Use-Cases](https://arize.com/blog-course/llm-as-a-judge/) 🌎 [Building an AI Agent that Thrives in the Real World](https://arize.com/blog/building-an-ai-agent-that-thrives-in-the-real-world/) 🛠️ [AI Agent Workflows and Architectures Masterclass](https://arize.com/blog/ai-agent-workflows-and-architectures/) 🔬 [Agents in the Wild: Geotab](https://www.youtube.com/watch?v=0WYO7Sx2pfw) Share * [![](https://arize.com/wp-content/themes/arize-2022/images/icon-tw-circle.svg)](https://twitter.com/intent/tweet?url=https%3A%2F%2Farize.com%2Fblog%2Farize-release-notes-copilot-enhancements-experiment-projects-and-more%2F&text=Arize%20Release%20Notes:%20Copilot%20Enhancements,%20Experiment%20Projects,%20and%20More) * [![](https://arize.com/wp-content/themes/arize-2022/images/icon-fb-circle.svg)](https://www.facguide.com/sharer/sharer.php?u=https://arize.com/blog/arize-release-notes-copilot-enhancements-experiment-projects-and-more/&picture=https://arize.com) * [![](https://arize.com/wp-content/themes/arize-2022/images/icon-in-circle.svg)](https://www.linkedin.com/shareArticle?mini=true&url=https%3A%2F%2Farize.com%2Fblog%2Farize-release-notes-copilot-enhancements-experiment-projects-and-more%2F&title=Arize%20Release%20Notes:%20Copilot%20Enhancements,%20Experiment%20Projects,%20and%20More) #### Suggested reading ![](https://arize.com/wp-content/uploads/2025/11/arize-ax-october-updates-2025.jpg) [New In Arize AX: Tags, Data Fabric, Automatic Threshold Ranges for Monitors and More](https://arize.com/blog/new-in-arize-ax-tags-data-fabric-automatic-threshold-ranges-for-monitors-and-more/) ![adb - Arize AI](https://arize.com/wp-content/uploads/2025/09/abd-cover-image-dark-3.jpg) [adb Benchmarks](https://arize.com/blog/adb-benchmarks/) Sign up for our monthly newsletter, The Evaluator. -------------------------------------------------- [Subscribe](https://arize.com/blog/arize-release-notes-copilot-enhancements-experiment-projects-and-more/#blog-subscribe-modal) Subscribe to The Evaluator -------------------------- We’ll send you the latest news, expertise, and product updates from Arize. Your inbox is sacred, so we’ll only curate and send the best stuff. \*We’re committed to your privacy. Arize uses the information you provide to contact you about relevant content, products, and services. You may unsubscribe from these communications at any time. For more information, check out our [privacy policy](https://arize.com/privacy-policy) . --- # Arize Release Notes: Monitor Runtime, Create a Dataset from CSV, and More - Arize AI ![](https://arize.com/wp-content/uploads/2023/01/Sarah_headshot-196x196.jpg "Sarah_headshot") [Sarah Welsh](https://arize.com/author/sarah-welsh/) Contributor Share * [![](https://arize.com/wp-content/themes/arize-2022/images/icon-tw-circle.svg)](https://twitter.com/intent/tweet?url=https%3A%2F%2Farize.com%2Fblog%2Farize-release-notes-feb-14%2F&text=Arize%20Release%20Notes:%20Monitor%20Runtime,%20Create%20a%20Dataset%20from%20CSV,%20and%20More) * [![](https://arize.com/wp-content/themes/arize-2022/images/icon-fb-circle.svg)](https://www.facguide.com/sharer/sharer.php?u=https://arize.com/blog/arize-release-notes-feb-14/&picture=https://arize.com) * [![](https://arize.com/wp-content/themes/arize-2022/images/icon-in-circle.svg)](https://www.linkedin.com/shareArticle?mini=true&url=https%3A%2F%2Farize.com%2Fblog%2Farize-release-notes-feb-14%2F&title=Arize%20Release%20Notes:%20Monitor%20Runtime,%20Create%20a%20Dataset%20from%20CSV,%20and%20More) ![](https://arize.com/wp-content/themes/arize-2022/images/icon-newsletter.svg) Subscribe to the Arize blog [Get the latest](https://arize.com/blog/arize-release-notes-feb-14/#blog-subscribe-modal) #### On this page #### Suggested reading ![](https://arize.com/wp-content/uploads/2025/11/arize-ax-october-updates-2025.jpg) [New In Arize AX: Tags, Data Fabric, Automatic Threshold Ranges for Monitors and More](https://arize.com/blog/new-in-arize-ax-tags-data-fabric-automatic-threshold-ranges-for-monitors-and-more/) ![adb - Arize AI](https://arize.com/wp-content/uploads/2025/09/abd-cover-image-dark-3.jpg) [adb Benchmarks](https://arize.com/blog/adb-benchmarks/) ![Text reads: Release Notes, February 14, 2025](https://arize.com/wp-content/uploads/2025/02/Release-notes-Feb-14.jpg "Release notes Feb 14") Arize Release Notes: Monitor Runtime, Create a Dataset from CSV, and More ========================================================================= Published Feb 14, 2025 * [Product Releases](https://arize.com/blog/?cat=product-releases) * [Release Notes](https://arize.com/blog/?cat=release-notes) ![](https://arize.com/wp-content/uploads/2023/01/Sarah_headshot-196x196.jpg "Sarah_headshot") #### [Sarah Welsh](https://arize.com/author/sarah-welsh/) ##### Contributor Enhancements ------------ ### Monitor Runtime Users can now schedule when monitors run. Users can configure their monitors to run: * Hourly & Daily: Select specific days of the week. * Daily, Weekly & Monthly: Runs at 12 AM UTC after creation. * Default Behavior: Monitors will continue running every 3 hours, 7 days a week unless configured otherwise. ![Screenshot of what it looks like to schedule when monitors run in Arize](https://arize.com/wp-content/uploads/2025/02/Define-the-Alerting.png) Column Specification With Exporting Data ---------------------------------------- Users can now export only the columns they care about for large datasets, reducing SDK export time by up to 95%. * Specify which columns of data you’d like to export when exporting data via the [ArizeExportClient](https://arize-client-python.readthedocs.io/en/latest/llm-api/exporter.html) * When using the `export_model_to_df` function, users can specify the `columns` parameter to only export specific columns. ![Screenshot of Export to notebook](https://arize.com/wp-content/uploads/2025/02/Export-to-notebook.png) Create a Dataset from CSV ------------------------- Users can now upload CSVs as a dataset in Arize. Columns in the file will be attributes that users can access in Experiments or in Prompt Playground. [Learn more here](https://docs.arize.com/arize/llm-datasets-and-experiments/how-to-datasets/create-a-dataset-from-csv) . ![Screenshot of how you can upload a dataset via csv in Arize. ](https://arize.com/wp-content/uploads/2025/02/Create-dataset-from-CSV.png) Monitor Improvements -------------------- We’ve made some updates to make monitors more organized, searchable, and user-friendly. Here’s what’s new: * Cardless Design – A sleek, modern table view for better readability. * Project-Level Monitors – LLM and ML monitors now have separate tabs. * Search & Sort – Find monitors by name or dimension, plus sort by any column. * Summary Stats – See how many monitors triggered in the last 24 hours * New LLM Monitor Types – Clearer categories: * Custom Metric Monitor → Performance Monitor with a custom metric preselected. * Span Property Monitor → Data Quality Monitor for span properties. * Evaluation Monitor → Data Quality Monitor for evaluations. * Quick Monitor for Errors – Easily enable error count monitoring (count, status\_code = ERROR). ![](https://arize.com/wp-content/uploads/2025/02/Screenshot-2025-02-19-at-2.33.54%E2%80%AFPM.png) OTEL Tracing Via HTTP --------------------- We’ve added support for HTTP protocol when sending traces to Arize through an OTEL tracer. To use: Specify `/v1/traces` as the endpoint and `Transport.HTTP` as the transport in our register helper // tracer_provider = register( endpoint="https://otlp.arize.com/v1/traces", # NEW transport=Transport.HTTP, # NEW space_id=SPACE_ID, api_key=API_KEY project_name="test-project-http", ) 📚 New Content -------------- The latest video tutorials, paper readings, ebooks, self-guided learning modules, and technical posts: 💯 [How 100X AI Uses Phoenix to Supercharge AI-Driven Troubleshooting](https://arize.com/blog/how-100x-ai-uses-phoenix-to-supercharge-ai-driven-troubleshooting/) 🤖 [Understanding Agentic RAG](https://arize.com/blog/understanding-agentic-rag/) ⚙️ [Multiagent Finetuning: A Conversation with Researcher Yilun Du](https://arize.com/blog/multiagent-finetuning-a-conversation-with-researcher-yilun-du/) Share * [![](https://arize.com/wp-content/themes/arize-2022/images/icon-tw-circle.svg)](https://twitter.com/intent/tweet?url=https%3A%2F%2Farize.com%2Fblog%2Farize-release-notes-feb-14%2F&text=Arize%20Release%20Notes:%20Monitor%20Runtime,%20Create%20a%20Dataset%20from%20CSV,%20and%20More) * [![](https://arize.com/wp-content/themes/arize-2022/images/icon-fb-circle.svg)](https://www.facguide.com/sharer/sharer.php?u=https://arize.com/blog/arize-release-notes-feb-14/&picture=https://arize.com) * [![](https://arize.com/wp-content/themes/arize-2022/images/icon-in-circle.svg)](https://www.linkedin.com/shareArticle?mini=true&url=https%3A%2F%2Farize.com%2Fblog%2Farize-release-notes-feb-14%2F&title=Arize%20Release%20Notes:%20Monitor%20Runtime,%20Create%20a%20Dataset%20from%20CSV,%20and%20More) #### Suggested reading ![](https://arize.com/wp-content/uploads/2025/11/arize-ax-october-updates-2025.jpg) [New In Arize AX: Tags, Data Fabric, Automatic Threshold Ranges for Monitors and More](https://arize.com/blog/new-in-arize-ax-tags-data-fabric-automatic-threshold-ranges-for-monitors-and-more/) ![adb - Arize AI](https://arize.com/wp-content/uploads/2025/09/abd-cover-image-dark-3.jpg) [adb Benchmarks](https://arize.com/blog/adb-benchmarks/) Sign up for our monthly newsletter, The Evaluator. -------------------------------------------------- [Subscribe](https://arize.com/blog/arize-release-notes-feb-14/#blog-subscribe-modal) Subscribe to The Evaluator -------------------------- We’ll send you the latest news, expertise, and product updates from Arize. Your inbox is sacred, so we’ll only curate and send the best stuff. \*We’re committed to your privacy. Arize uses the information you provide to contact you about relevant content, products, and services. You may unsubscribe from these communications at any time. For more information, check out our [privacy policy](https://arize.com/privacy-policy) . --- # Arize Phoenix: 2024 in Review - Arize AI ![](https://arize.com/wp-content/uploads/2024/12/DALL%C2%B7E-2024-12-30-08.33.20-A-visually-striking-illustration-of-a-phoenix-rising-into-the-sky-symbolizing-growth-and-progress.-The-phoenix-features-intricate-feather-patterns-in.webp) Arize Phoenix: 2024 in Review ============================= Published December 30, 2024 --------------------------- ![](https://arize.com/wp-content/uploads/2024/12/DALL%C2%B7E-2024-12-30-08.33.20-A-visually-striking-illustration-of-a-phoenix-rising-into-the-sky-symbolizing-growth-and-progress.-The-phoenix-features-intricate-feather-patterns-in.webp) 2024 was [Arize Phoenix](http://phoenix.arize.com/) ‘s biggest year ever. Granted, it was also Phoenix’s first full year ever, but given how much we crammed into this year we think it still counts 🤗 Over the past year, Phoenix’s [open-source](https://github.com/Arize-ai/phoenix) LLM evaluation and tracing solution has grown from ~20k monthly downloads to over 2.5 million. Our [community](https://join.slack.com/t/arize-ai/shared_invite/zt-2w57bhem8-hq24MB6u7yE_ZF_ilOYSBw) grew to over 6,000 members. We ran dozens of hackathons, meetups, paper reading sessions, workshops, tech talks, and conferences, and got to spend countless hours connecting with the AI developer community. Throughout all of this, we saw a few notable themes in 2024.  1/ The AI industry moved rapidly and completely toward **agents** in 2024. [New development tools](https://towardsdatascience.com/navigating-the-new-types-of-llm-agents-and-architectures-309382ce9f88) launched seemingly weekly, but despite this rapid progress, key challenges remain unsolved. The question of how to properly [evaluate agents](https://arize.com/blog-course/llm-agent-how-to-set-up/evaluating-ai-agents/) – beyond basic function calling evals and skill tests – stayed an open debate in the field. Expect a lot more from us here in 2025! 2/ **OpenTelemetry** solidifies its position as the preferred standard to build on top of when it comes to LLM observability. By now, the majority of observability tools have shifted or are building support for OTEL.  We luckily made this bet back in Jan 2024, and designed Arize Phoenix to run entirely on OpenTelemetry. This change was a huge unlock for our team, allowing us to iterate more quickly, and take lessons from pre-LLM observability platforms. But this was not an easy shift. We had to figure out everything from dealing with latent data, to properly instrumenting streaming, to handling something as basic as lists. If you’re curious about those, check out our [lessons learned post](https://arize.com/blog/zero-to-a-million-instrumenting-llms-with-otel/) . 3/ The industry’s **adoption of LLM evaluations** matured significantly. “LLM as a Judge” is now a recognizable concept for many AI builders. OpenAI launched their own evals product. And how to structure your evals is now a popular discussion topic on X and BlueSky. We launched a ton of features throughout the year to help AI engineers looking to run their evals. [Datasets & Experiments](https://docs.arize.com/phoenix/datasets-and-experiments/overview-datasets) made Evaluation Driven Development possible, giving users the ability to test iterations of their applications on a consistent set of test cases. [Prompt Playground](https://docs.arize.com/phoenix/prompt-engineering/quickstart-prompts) moved the debugging process into the dashboard, letting devs replay time and test tweaks to their prompts (our team is especially proud of this one, [check it out](https://www.youtube.com/watch?v=wLK5RwHNLUM) if you haven’t!) Looking back over this year has also made us realize just what a special opportunity we have in Arize Phoenix.That gives us, the Phoenix team, the support to build the AI engineering platform that we want as developers, and the ability to put it in as many people’s hands as possible, with nothing held back. We can spend cycles testing the newest experimental framework. We don’t have to gate features behind paywalls (if you’re looking for our SaaS counterpart, check out [Arize](https://arize.com/) !). We have the opportunity to build the best possible AI platform there is, and we can do it with no reservations. But what makes all of the building worth it is seeing the community, aka YOU, using what we’ve built.  Every social mention, every inclusion of Phoenix in a tutorial, every piece of swag seen in the wild, and every star on Github is fuel for the Phoenix team’s fire. We’ve got big plans for 2025. Expect more features, more community events, more experiments, and many, many more releases in the year to come. To the skies, The Phoenix Team (Mikyo, Xander, Dustin, Roger, Parker, Tony, and John) Subscribe to The Evaluator -------------------------- We’ll send you the latest news, expertise, and product updates from Arize. Your inbox is sacred, so we’ll only curate and send the best stuff. \*We’re committed to your privacy. Arize uses the information you provide to contact you about relevant content, products, and services. You may unsubscribe from these communications at any time. For more information, check out our [privacy policy](https://arize.com/privacy-policy) . --- # Embracing Google's Agent-To-Agent (A2A) Protocol - Arize AI ![Rich Young headshot](https://arize.com/wp-content/uploads/2025/04/Rich-Young-196x196.jpeg "Rich Young") [Richard Young](https://arize.com/author/richard-young/) Director, Partner Solutions Architecture Share * [![](https://arize.com/wp-content/themes/arize-2022/images/icon-tw-circle.svg)](https://twitter.com/intent/tweet?url=https%3A%2F%2Farize.com%2Fblog%2Farize-ai-and-future-of-agent-interoperability-embracing-googles-a2a-protocol%2F&text=Embracing%20Google%E2%80%99s%20Agent-To-Agent%20(A2A)%20Protocol) * [![](https://arize.com/wp-content/themes/arize-2022/images/icon-fb-circle.svg)](https://www.facguide.com/sharer/sharer.php?u=https://arize.com/blog/arize-ai-and-future-of-agent-interoperability-embracing-googles-a2a-protocol/&picture=https://arize.com) * [![](https://arize.com/wp-content/themes/arize-2022/images/icon-in-circle.svg)](https://www.linkedin.com/shareArticle?mini=true&url=https%3A%2F%2Farize.com%2Fblog%2Farize-ai-and-future-of-agent-interoperability-embracing-googles-a2a-protocol%2F&title=Embracing%20Google%E2%80%99s%20Agent-To-Agent%20(A2A)%20Protocol) ![](https://arize.com/wp-content/themes/arize-2022/images/icon-newsletter.svg) Subscribe to the Arize blog [Get the latest](https://arize.com/blog/arize-ai-and-future-of-agent-interoperability-embracing-googles-a2a-protocol/#blog-subscribe-modal) #### On this page #### Suggested reading ![](https://arize.com/wp-content/uploads/2025/11/improving-ai-agent-security-cover-image.png) [How To Improve AI Agent Security with Microsoft’s AI Red Teaming Agent in Microsoft Foundry](https://arize.com/blog/how-to-improve-ai-agent-security-with-microsofts-ai-red-teaming-agent-in-microsoft-foundry/) ![](https://arize.com/wp-content/uploads/2025/10/coding-agents-cover.png) [Optimizing Coding Agent Rules (./clinerules) for Improved Accuracy](https://arize.com/blog/optimizing-coding-agent-rules-claude-md-agents-md-clinerules-cursor-rules-for-improved-accuracy/) ![](https://arize.com/wp-content/uploads/2025/04/Google-Arize-blog-image-1-2142x1120.jpg "Google-Arize blog image 1") Embracing Google’s Agent-To-Agent (A2A) Protocol ================================================ Published Apr 9, 2025 * [Agents](https://arize.com/blog/?cat=agents) * [AI In the Enterprise](https://arize.com/blog/?cat=ai-in-the-enterprise) ![Rich Young headshot](https://arize.com/wp-content/uploads/2025/04/Rich-Young-196x196.jpeg "Rich Young") #### [Richard Young](https://arize.com/author/richard-young/) ##### Director, Partner Solutions Architecture We’re excited to announce that Arize AI is partnering with Google as a launch partner for the Agent Interop Protocol (A2A), an open standard enabling seamless communication between AI agents across different platforms and organizational boundaries. What is Agent Interop Protocol (A2A)? ------------------------------------- The Agent Interop Protocol (A2A) addresses a fundamental challenge in today’s AI ecosystem: enabling different AI agents to collaborate effectively without sharing internal resources. As specialized AI agents proliferate across various frameworks, A2A creates a standardized way for “opaque agents” – those operating across business, policy, or competitive boundaries – to work together while respecting organizational boundaries. Another way to think about this: Where Anthropic’s Model Context Protocol (MCP) is a protocol that standardizes connection of LLM-based applications to information, resources, and tools, A2A is a protocol that standardizes collaboration between autonomous agents. This protocol establishes a universal language and interaction framework allowing AI agents to coordinate regardless of who built them or what technology they use – all while maintaining security and privacy controls. Why Agent Interoperability Matters ---------------------------------- * The fragmentation of today’s AI agent ecosystem creates several challenges: * Increasing variability of frameworks to build agents * Difficulty in representing an “agent as a tool” given multimodal, unstructured, dynamic interaction with users. * Agents across (business, policy, competitive) boundaries cannot share tools, memory, plans, thoughts, etc. A2A addresses these issues through four foundational capabilities: * Capability Discovery: Agents can advertise their abilities, enabling clients to utilize them for specific tasks. * User Experience Negotiation: Clients and agents can agree on appropriate communication methods. * Task and State Management: Clear mechanisms for communicating task status and dependencies. * Dynamic Collaboration: Support for agents to request clarifications or additional information as needed. Arize AI’s Role in A2A ---------------------- As agents begin to collaborate across organizational boundaries, the ability to trace interactions, evaluate performance, identify failures and observe behavior at each agent level becomes even more critical. As an AI observability and evaluation platform, Arize’s mission is to make AI systems more transparent, reliable, and trustworthy. Our involvement in the A2A Protocol further extends this mission into the emerging world of autonomous agents. Our expertise in tracing, evaluation, and observability for AI systems—including our OpenTelemetry-compliant architecture—makes Arize uniquely positioned to support and enhance this standard, regardless of the type of agent development framework or approach taken. “Arize AI is proud to partner with Google as a launch partner for the A2A interoperability protocol. A2A represents a meaningful advancement toward seamless, secure interaction across AI agents,” says Jason Lopatecki, Cofounder & CEO of Arize AI. “Arize’s commitment to open-source evaluation and observability frameworks positions us uniquely to help define and enhance this open standard.” The Path Forward ---------------- We invite developers, enterprises, and AI practitioners to explore how the A2A Protocol can transform their agent architectures. By embracing open standards for agent interoperability, we can collectively build a more connected, capable, and trustworthy AI ecosystem. For organizations looking to prepare for this next wave of AI innovation, Arize’s observability platform provides the foundation needed to confidently deploy and optimize agents in both standalone and collaborative contexts. [Read Google’s announcement here](https://developers.googleblog.com/en/a2a-a-new-era-of-agent-interoperability/) . Learn More ---------- Explore how Arize is helping teams trace, evaluate, and improve AI systems at scale, or book a demo. * [Transform Agent Architectures for Enterprise](https://arize.com/generative-ai/) * [Book a demo](https://arize.com/request-a-demo/) Share * [![](https://arize.com/wp-content/themes/arize-2022/images/icon-tw-circle.svg)](https://twitter.com/intent/tweet?url=https%3A%2F%2Farize.com%2Fblog%2Farize-ai-and-future-of-agent-interoperability-embracing-googles-a2a-protocol%2F&text=Embracing%20Google%E2%80%99s%20Agent-To-Agent%20(A2A)%20Protocol) * [![](https://arize.com/wp-content/themes/arize-2022/images/icon-fb-circle.svg)](https://www.facguide.com/sharer/sharer.php?u=https://arize.com/blog/arize-ai-and-future-of-agent-interoperability-embracing-googles-a2a-protocol/&picture=https://arize.com) * [![](https://arize.com/wp-content/themes/arize-2022/images/icon-in-circle.svg)](https://www.linkedin.com/shareArticle?mini=true&url=https%3A%2F%2Farize.com%2Fblog%2Farize-ai-and-future-of-agent-interoperability-embracing-googles-a2a-protocol%2F&title=Embracing%20Google%E2%80%99s%20Agent-To-Agent%20(A2A)%20Protocol) #### Suggested reading ![](https://arize.com/wp-content/uploads/2025/11/improving-ai-agent-security-cover-image.png) [How To Improve AI Agent Security with Microsoft’s AI Red Teaming Agent in Microsoft Foundry](https://arize.com/blog/how-to-improve-ai-agent-security-with-microsofts-ai-red-teaming-agent-in-microsoft-foundry/) ![](https://arize.com/wp-content/uploads/2025/10/coding-agents-cover.png) [Optimizing Coding Agent Rules (./clinerules) for Improved Accuracy](https://arize.com/blog/optimizing-coding-agent-rules-claude-md-agents-md-clinerules-cursor-rules-for-improved-accuracy/) Sign up for our monthly newsletter, The Evaluator. -------------------------------------------------- [Subscribe](https://arize.com/blog/arize-ai-and-future-of-agent-interoperability-embracing-googles-a2a-protocol/#blog-subscribe-modal) Subscribe to The Evaluator -------------------------- We’ll send you the latest news, expertise, and product updates from Arize. Your inbox is sacred, so we’ll only curate and send the best stuff. \*We’re committed to your privacy. Arize uses the information you provide to contact you about relevant content, products, and services. You may unsubscribe from these communications at any time. For more information, check out our [privacy policy](https://arize.com/privacy-policy) . --- # Arize AI Achieves ISO/IEC 27001 Certification - Arize AI ![](https://arize.com/wp-content/uploads/2025/10/iso-iec-27001-certified-icon.jpg) Arize AI Achieves ISO/IEC 27001 Certification ============================================= Published October 20, 2025 -------------------------- ![](https://arize.com/wp-content/uploads/2025/10/iso-iec-27001-certified-icon.jpg) Organizations running AI agents in production depend on Arize to operate securely at scale, logging over 1 trillion inferences and spans and 10 million evaluation runs monthly. Today, we’re proud to share that Arize AI has achieved ISO/IEC 27001 certification, underscoring an already-robust commitment to the highest standards of information security. Why This Matters ---------------- ISO/IEC 27001 is a globally recognized benchmark published by the International Organization for Standardization (ISO) and the International Electrotechnical Commission (IEC) for establishing, implementing, maintaining, and continually improving an Information Security Management System (ISMS). Our certification validates that Arize’s controls, processes, and governance align with best practices, helping customers meet their own security and compliance requirements with confidence. Independently Assessed by A-LIGN -------------------------------- Arize’s audit and certification were conducted by A-LIGN, an ISO/IEC 27001 certification body accredited by ANAB and UKAS and trusted by over 4,000 organizations for security and compliance assessments. > “Congratulations to Arize AI for earning ISO/IEC 27001 certification, a widely recognized signal of trust and security. It’s great to work with organizations like Arize AI, who understand the value of expertise in driving an efficient audit and the importance of a high-quality final report.” — Steve Simmons, COO, A-LIGN Learn More About Security At Arize ---------------------------------- Safeguarding data is a core function at Arize and a key part of how we earn and maintain the trust of users, customers, and partners. Our industry certifications and validations include SOC 2 Type II, PCI DSS 4.0 (where applicable), Clone Guard certification, a Cloud Security Alliance (CSA) STAR Level 1 self-assessment, and independent validations of GDPR compliance and HIPAA health-information security. Arize also supports EU and other regional data-residency requirements. To learn more—or to request a full copy of our ISO/IEC 27001 certificate—visit the [Arize Trust Center](https://arize.com/trust-center/) or reach out directly in the [Arize community](https://arize.com/community/) . Subscribe to The Evaluator -------------------------- We’ll send you the latest news, expertise, and product updates from Arize. Your inbox is sacred, so we’ll only curate and send the best stuff. \*We’re committed to your privacy. Arize uses the information you provide to contact you about relevant content, products, and services. You may unsubscribe from these communications at any time. For more information, check out our [privacy policy](https://arize.com/privacy-policy) . --- # Arize Release Notes: Prompt Hub, Managed Code Evaluators and More - Arize AI ![](https://arize.com/wp-content/uploads/2023/01/Sarah_headshot-196x196.jpg "Sarah_headshot") [Sarah Welsh](https://arize.com/author/sarah-welsh/) Contributor Share * [![](https://arize.com/wp-content/themes/arize-2022/images/icon-tw-circle.svg)](https://twitter.com/intent/tweet?url=https%3A%2F%2Farize.com%2Fblog%2Farize-release-notes-prompt-hub-managed-code-evaluators-and-more%2F&text=Arize%20Release%20Notes:%20Prompt%20Hub,%20Managed%20Code%20Evaluators%20and%20More) * [![](https://arize.com/wp-content/themes/arize-2022/images/icon-fb-circle.svg)](https://www.facguide.com/sharer/sharer.php?u=https://arize.com/blog/arize-release-notes-prompt-hub-managed-code-evaluators-and-more/&picture=https://arize.com) * [![](https://arize.com/wp-content/themes/arize-2022/images/icon-in-circle.svg)](https://www.linkedin.com/shareArticle?mini=true&url=https%3A%2F%2Farize.com%2Fblog%2Farize-release-notes-prompt-hub-managed-code-evaluators-and-more%2F&title=Arize%20Release%20Notes:%20Prompt%20Hub,%20Managed%20Code%20Evaluators%20and%20More) ![](https://arize.com/wp-content/themes/arize-2022/images/icon-newsletter.svg) Subscribe to the Arize blog [Get the latest](https://arize.com/blog/arize-release-notes-prompt-hub-managed-code-evaluators-and-more/#blog-subscribe-modal) #### On this page #### Suggested reading ![](https://arize.com/wp-content/uploads/2025/09/arize-ax-august-2025-updates-cover-art.jpg) [New In Arize AX: Experiment Comparisons, Better Data Visualization, and a Dedicated Agent Graph Tab](https://arize.com/blog/new-in-arize-ax-experiment-comparisons-better-data-visualization-and-a-dedicated-agent-graph-tab/) ![](https://arize.com/wp-content/uploads/2025/05/arize-ai-agents-tracing-graphs-dark.avif) [Arize Observe 2025 – Product Releases](https://arize.com/blog/observe-2025-releases/) ![Text reads: Release Notes, December 19, 2024 with the Arize logo.](https://arize.com/wp-content/uploads/2024/12/Release-notes-12-19.jpg "Release notes 12-19") Arize Release Notes: Prompt Hub, Managed Code Evaluators and More ================================================================= Published Dec 19, 2024 * [Release Notes](https://arize.com/blog/?cat=release-notes) ![](https://arize.com/wp-content/uploads/2023/01/Sarah_headshot-196x196.jpg "Sarah_headshot") #### [Sarah Welsh](https://arize.com/author/sarah-welsh/) ##### Contributor Welcome to our regular update on new releases, enhancements, and changes. What’s New ---------- ### Prompt Hub [The Prompt Hub](https://docs.arize.com/arize/prompt-engineering/prompt-hub) is a centralized repository for managing, iterating, and deploying prompt templates within the Arize platform. It serves as a collaborative workspace for users to refine and store templates for various use cases, including production applications and experimentation. Key features of the Prompt Hub include: * **Template Management**: Users can save templates directly from the Prompt Playground along with associated LLM parameters, function definitions, and metadata required to reproduce specific LLM calls. * **Version Control**: Every saved template supports versioning, enabling users to track updates, experiment with variations, and revert to previous versions if needed. * **Collaboration and Reusability**: Saved templates can be shared across teams, facilitating collaboration and consistency in production workflows. Templates can also be reloaded into the Prompt Playground or accessed via APIs for seamless integration into codebases and online tasks. * **Evaluation and Optimization**: By saving outputs as experiments, users can compare templates, compute evaluation metrics, and analyze performance both quantitatively and qualitatively. ### Managed Code Evaluators We recently launched [a set of pre-built, off-the-shelf evaluators](https://docs.arize.com/arize/llm-evaluation-and-annotations/catching-hallucinations/code-evaluations) to enable users to evaluate their spans without requiring requests to an LLM-as-a-Judge. Evaluators available: * **Matches Regex**: Checks if text matches a specific regular expression pattern. * **JSON Parseable**: Validate JSON output from LLMs. * **Contains Any Keyword**: Check if any keywords appear in the text. * **Contains All Keywords**: Validate that all specified keywords are present. Enhancements ------------ ### Experiment Creation From Playground We just released a [new flow for creating experiments](https://docs.arize.com/arize/prompt-engineering/prompt-playground#save-outputs-as-experiment) from outputs generated with the Prompt Playground. What’s new? **Quickly Experiment**: After running the playground successfully on a dataset, click the “Save as Experiment” button. * **Debug**: In addition to the newly outputted response, we save the LLM invocation parameters & prompt template message structure for greater replay functionality. * **Compare**: Just like our existing experiments, you can compare the playground outputs as well. ### New Monitor Visualization We’ve rolled out the first part of our monitor improvements! Here’s what’s new: * **Alert Status Graph**: Maps directly to the alerts users see, giving them a transparent and seamless way to line up alerts with the real-time metric visualization. * **Cleaner UX**: Updates include removing “last run monitor time,” aligning card titles and Y-axis with metric names, and simplifying by removing granularity. _Note: Alert ticks are limited—users may need to zoom into specific dates to see all alerts._ ![Screenshot of monitor visualization in Arize](https://arize.com/wp-content/uploads/2024/12/AI-Search-errors-2048x1187.png) ### LangChain Instrumentation Support for sessions via LangChain native thread tracking in TypeScript is now available. Easily track multi-turn conversations / threads using LangChain.js. 📚 New Content -------------- The latest video tutorials, paper readings, ebooks, self-guided learning modules, and technical posts: ✈️ [How](https://arize.com/blog/how-booking-com-enhances-travel-planning-with-ai-trip-planner-and-arize-ai/)  [Booking.com](http://booking.com/)  [Personalizes Travel Planning with AI Trip Planner and Arize AI](https://arize.com/blog/how-booking-com-enhances-travel-planning-with-ai-trip-planner-and-arize-ai/) ♾️ [How to Add LLM Evaluations to CI/CD Pipelines](https://arize.com/blog/how-to-add-llm-evaluations-to-ci-cd-pipelines/) 📛 [2025 AI Conferences](https://arize.com/blog/2025-ai-conferences/) 🤝 [Merge, Ensemble, and Cooperate! A Survey on Collaborative LLM Strategies](https://arize.com/blog/merge-ensemble-and-cooperate-a-survey-on-collaborative-llm-strategies/) Share * [![](https://arize.com/wp-content/themes/arize-2022/images/icon-tw-circle.svg)](https://twitter.com/intent/tweet?url=https%3A%2F%2Farize.com%2Fblog%2Farize-release-notes-prompt-hub-managed-code-evaluators-and-more%2F&text=Arize%20Release%20Notes:%20Prompt%20Hub,%20Managed%20Code%20Evaluators%20and%20More) * [![](https://arize.com/wp-content/themes/arize-2022/images/icon-fb-circle.svg)](https://www.facguide.com/sharer/sharer.php?u=https://arize.com/blog/arize-release-notes-prompt-hub-managed-code-evaluators-and-more/&picture=https://arize.com) * [![](https://arize.com/wp-content/themes/arize-2022/images/icon-in-circle.svg)](https://www.linkedin.com/shareArticle?mini=true&url=https%3A%2F%2Farize.com%2Fblog%2Farize-release-notes-prompt-hub-managed-code-evaluators-and-more%2F&title=Arize%20Release%20Notes:%20Prompt%20Hub,%20Managed%20Code%20Evaluators%20and%20More) #### Suggested reading ![](https://arize.com/wp-content/uploads/2025/09/arize-ax-august-2025-updates-cover-art.jpg) [New In Arize AX: Experiment Comparisons, Better Data Visualization, and a Dedicated Agent Graph Tab](https://arize.com/blog/new-in-arize-ax-experiment-comparisons-better-data-visualization-and-a-dedicated-agent-graph-tab/) ![](https://arize.com/wp-content/uploads/2025/05/arize-ai-agents-tracing-graphs-dark.avif) [Arize Observe 2025 – Product Releases](https://arize.com/blog/observe-2025-releases/) Sign up for our monthly newsletter, The Evaluator. -------------------------------------------------- [Subscribe](https://arize.com/blog/arize-release-notes-prompt-hub-managed-code-evaluators-and-more/#blog-subscribe-modal) Subscribe to The Evaluator -------------------------- We’ll send you the latest news, expertise, and product updates from Arize. Your inbox is sacred, so we’ll only curate and send the best stuff. \*We’re committed to your privacy. Arize uses the information you provide to contact you about relevant content, products, and services. You may unsubscribe from these communications at any time. For more information, check out our [privacy policy](https://arize.com/privacy-policy) . --- # Arize Release Notes: Labeling Queues, Expand/Collapse Rows in Trace Table - Arize AI ![](https://arize.com/wp-content/uploads/2023/01/Sarah_headshot-196x196.jpg "Sarah_headshot") [Sarah Welsh](https://arize.com/author/sarah-welsh/) Contributor Share * [![](https://arize.com/wp-content/themes/arize-2022/images/icon-tw-circle.svg)](https://twitter.com/intent/tweet?url=https%3A%2F%2Farize.com%2Fblog%2Farize-releases-labeling-queues-expand-collapse-rows-in-trace-table%2F&text=Arize%20Release%20Notes:%20Labeling%20Queues,%20Expand/Collapse%20Rows%20in%20Trace%20Table) * [![](https://arize.com/wp-content/themes/arize-2022/images/icon-fb-circle.svg)](https://www.facguide.com/sharer/sharer.php?u=https://arize.com/blog/arize-releases-labeling-queues-expand-collapse-rows-in-trace-table/&picture=https://arize.com) * [![](https://arize.com/wp-content/themes/arize-2022/images/icon-in-circle.svg)](https://www.linkedin.com/shareArticle?mini=true&url=https%3A%2F%2Farize.com%2Fblog%2Farize-releases-labeling-queues-expand-collapse-rows-in-trace-table%2F&title=Arize%20Release%20Notes:%20Labeling%20Queues,%20Expand/Collapse%20Rows%20in%20Trace%20Table) ![](https://arize.com/wp-content/themes/arize-2022/images/icon-newsletter.svg) Subscribe to the Arize blog [Get the latest](https://arize.com/blog/arize-releases-labeling-queues-expand-collapse-rows-in-trace-table/#blog-subscribe-modal) #### On this page #### Suggested reading ![](https://arize.com/wp-content/uploads/2025/11/arize-ax-october-updates-2025.jpg) [New In Arize AX: Tags, Data Fabric, Automatic Threshold Ranges for Monitors and More](https://arize.com/blog/new-in-arize-ax-tags-data-fabric-automatic-threshold-ranges-for-monitors-and-more/) ![adb - Arize AI](https://arize.com/wp-content/uploads/2025/09/abd-cover-image-dark-3.jpg) [adb Benchmarks](https://arize.com/blog/adb-benchmarks/) ![](https://arize.com/wp-content/uploads/2025/03/Release-notes-Feb-27.jpg "Release notes Feb 27") Arize Release Notes: Labeling Queues, Expand/Collapse Rows in Trace Table ========================================================================= Published Mar 4, 2025 * [Product Releases](https://arize.com/blog/?cat=product-releases) * [Release Notes](https://arize.com/blog/?cat=release-notes) ![](https://arize.com/wp-content/uploads/2023/01/Sarah_headshot-196x196.jpg "Sarah_headshot") #### [Sarah Welsh](https://arize.com/author/sarah-welsh/) ##### Contributor What’s New ---------- ### Labeling Queues Labeling Queues are now live, making dataset annotation more scalable and efficient with features such as: * **New Annotator Role** – A dedicated RBAC role with focused permissions, ensuring annotators only see assigned records while keeping other data secure. * **Seamless Queue Creation** – Create Labeling Queues directly from dataset records, with annotations automatically written back for easy tracking. * **Annotation Resets** – AI Engineers can reset annotations, allowing re-labeling when needed. * **Flexible Assignment Methods** – Choose between Random or All assignments for annotators in a queue. * **Fast & Streamlined UI** – Optimized for quick labeling workflows with: Hotkey support, background data fetching & pagination, and background submissions. [Learn More](https://docs.arize.com/arize/llm-evaluation-and-annotations/how-to-labeling-queues) ### Enhancements #### Expand/Collapse Rows in the Trace Table You can now collapse rows to see more data at a glance or expand them to view more text. ![Traces collapse gif](https://arize.com/wp-content/uploads/2025/03/traces_collapse.gif) 📚 New Content -------------- The latest video tutorials, paper readings, ebooks, self-guided learning modules, and technical posts: 🐳 [DeepSeek Deep Dive](https://arize.com/blog/how-deepseek-is-pushing-the-boundaries-of-ai-development/) 🤖 [How to Build an AI Agent](https://arize.com/blog/how-to-build-an-ai-agent/) 🎉 [We Raised $70M: A Note from our Founders](https://arize.com/blog/arize-ai-raises-70m-series-c-to-build-the-gold-standard-for-ai-evaluation-observability/) Share * [![](https://arize.com/wp-content/themes/arize-2022/images/icon-tw-circle.svg)](https://twitter.com/intent/tweet?url=https%3A%2F%2Farize.com%2Fblog%2Farize-releases-labeling-queues-expand-collapse-rows-in-trace-table%2F&text=Arize%20Release%20Notes:%20Labeling%20Queues,%20Expand/Collapse%20Rows%20in%20Trace%20Table) * [![](https://arize.com/wp-content/themes/arize-2022/images/icon-fb-circle.svg)](https://www.facguide.com/sharer/sharer.php?u=https://arize.com/blog/arize-releases-labeling-queues-expand-collapse-rows-in-trace-table/&picture=https://arize.com) * [![](https://arize.com/wp-content/themes/arize-2022/images/icon-in-circle.svg)](https://www.linkedin.com/shareArticle?mini=true&url=https%3A%2F%2Farize.com%2Fblog%2Farize-releases-labeling-queues-expand-collapse-rows-in-trace-table%2F&title=Arize%20Release%20Notes:%20Labeling%20Queues,%20Expand/Collapse%20Rows%20in%20Trace%20Table) #### Suggested reading ![](https://arize.com/wp-content/uploads/2025/11/arize-ax-october-updates-2025.jpg) [New In Arize AX: Tags, Data Fabric, Automatic Threshold Ranges for Monitors and More](https://arize.com/blog/new-in-arize-ax-tags-data-fabric-automatic-threshold-ranges-for-monitors-and-more/) ![adb - Arize AI](https://arize.com/wp-content/uploads/2025/09/abd-cover-image-dark-3.jpg) [adb Benchmarks](https://arize.com/blog/adb-benchmarks/) Sign up for our monthly newsletter, The Evaluator. -------------------------------------------------- [Subscribe](https://arize.com/blog/arize-releases-labeling-queues-expand-collapse-rows-in-trace-table/#blog-subscribe-modal) Subscribe to The Evaluator -------------------------- We’ll send you the latest news, expertise, and product updates from Arize. Your inbox is sacred, so we’ll only curate and send the best stuff. \*We’re committed to your privacy. Arize uses the information you provide to contact you about relevant content, products, and services. You may unsubscribe from these communications at any time. For more information, check out our [privacy policy](https://arize.com/privacy-policy) . --- # Arize AI Now Generally Available As Part of Azure Native Integrations - Arize AI ![](https://arize.com/wp-content/uploads/2025/05/Arize-x-Azure-1.jpg) Arize AI Now Generally Available As Part of Azure Native Integrations ===================================================================== Published May 19, 2025 ---------------------- ![](https://arize.com/wp-content/uploads/2025/05/Arize-x-Azure-1.jpg) Arize AI, a leading platform for AI observability and LLM evaluation, today announced the general availability of its platform to developers as part of Azure Native Integrations. The debut follows a successful public preview unveiled at Microsoft Ignite 2024. Developed in collaboration with Microsoft, this fully managed integration allows AI teams to provision and run Arize directly from the Azure portal with native support for Azure SDK, CLI, and billing infrastructure. With single sign-on through Azure Active Directory and a unified billing experience, enterprises can deploy observability and evaluation workflows with minimal setup. Arize’s platform brings essential tooling for AI and agent evaluation, observability, and performance testing to enterprise AI teams deploying LLM and generative applications and agents at scale. With this integration, AI engineers and developers working on AI agents and applications can: * Seamlessly test, evaluate, and automate observability  * Trace prompts, variables, tool calls, and agents to debug faster * Test models and workflows pre-deployment with CI/CD rigor * Improve reliability and iteration speed with real-time feedback loops “Microsoft Azure AI platform is relied on by top enterprises to deploy AI and agents at scale globally across a wide range of use cases, and we’re pleased to offer fully integrated tools to ensure safe and effective production deployments,” says Rich Young, Director of Partner Solutions Architecture at Arize AI. Clients can get started with the Arize AI Cloud Service [here](https://portal.azure.com/#browse/ArizeAi.ObservabilityEval%2Forganizations) . Subscribe to The Evaluator -------------------------- We’ll send you the latest news, expertise, and product updates from Arize. Your inbox is sacred, so we’ll only curate and send the best stuff. \*We’re committed to your privacy. Arize uses the information you provide to contact you about relevant content, products, and services. You may unsubscribe from these communications at any time. For more information, check out our [privacy policy](https://arize.com/privacy-policy) . --- # Self-Improving Agents: Automating LLM Performance Optimization using Arize and NVIDIA NeMo - Arize AI ![](https://arize.com/wp-content/uploads/2025/03/NVIDIA-Arize-blog.jpg) Self-Improving Agents: Automating LLM Performance Optimization using Arize and NVIDIA NeMo ========================================================================================== Published March 18, 2025 ------------------------ ![](https://arize.com/wp-content/uploads/2025/03/NVIDIA-Arize-blog.jpg) Enterprises face a critical challenge in keeping their LLM models accurate and reliable over time. Traditional model improvement approaches are slow, manual, and reactive, making it difficult to scale and adapt to evolving data patterns. The Arize integration of [NVIDIA NeMo](https://www.nvidia.com/en-us/ai-data-science/products/nemo/) empowers AI teams with an automated, self-improving AI data flywheel to enhance LLM performance. The powerful Arize solution automatically identifies production LLM failure modes through online evaluations, routes challenging cases for human annotation, and continuously refines models through targeted fine-tuning and validation against golden datasets—enabling enterprises to maintain optimal LLM performance through a streamlined human-in-the-loop workflow. By leveraging Arize’s AI-driven evaluation tools and datasets, alongside NVIDIA NeMo for model training, evaluation, and guardrailing, organizations can continuously improve and deploy state-of-the-art LLMs at scale. Additionally, the same iterative loop can be applied to improve the accuracy and reliability of LLM-based Judge evaluators. In this workflow, examples with low-confidence evaluations are automatically aggregated and routed to human annotators, who provide correct labels. These annotated examples then drive targeted fine-tuning, ensuring continuous enhancement of Judge evaluator quality and consistency, enabling Judges to continuously improve alongside the production application. Over time, less and less human intervention is needed. ![Self-Improving Agents](https://arize.com/wp-content/uploads/2025/03/image1-3.png) How it Works ------------ 1. **Identify Failure Modes:** Automatically detect and log failure cases using [Arize Online Evaluations](https://docs.arize.com/arize/llm-evaluation-and-annotations/catching-hallucinations/tasks-for-online-evals) , which supports both LLM-as-a-Judge and code evaluators. 2. **Targeted Annotation:** Route critical examples through Arize’s [labeling queue](https://docs.arize.com/arize/llm-evaluation-and-annotations/annotations) for expert human validation, only adding human-in-the-loop workflows when needed. 3. **Train Smarter:** Arize automatically kicks off fine-tuning jobs using [NeMo Customizer](https://developer.nvidia.com/blog/fine-tune-and-align-llms-easily-with-nvidia-nemo-customizer/) as new examples are added to the dataset, where the NVIDIA training configs can be fully configured in the Arize UI. 4. **Benchmark for Excellence:** After fine-tuning, evaluations are automatically run on [Arize golden datasets](https://docs.arize.com/phoenix/datasets-and-experiments/concepts-datasets) built from production data, as well as public benchmarks using [NeMo Evaluator](https://developer.nvidia.com/blog/streamline-evaluation-of-llms-for-accuracy-with-nvidia-nemo-evaluator/) . This provides a final check that the fine-tuned model has improved before it is deployed. 5. **Assess Results:** Evaluation results can be analyzed in further detail on the [Arize Experiments](https://docs.arize.com/arize/llm-datasets-and-experiments/how-to-use-experiments/filter-experiments) page, where the model output, evaluation labels and aggregate metrics are displayed. This includes both custom datasets in Arize and academic benchmarks from NVIDIA’s eval harness. 6. **Enforce in Real Time:** Once the fine-tuned LLM Judge meets performance standards, it is deployed in Arize online evals and NeMo Guardrails for real-time enforcement. Any unsafe or undesired outputs are blocked before reaching users, with every guardrails activation traced and logged in Arize for full observability. Why it Matters -------------- The Arize + NVIDIA NeMo integration eliminates bottlenecks in generative AI development, providing a no-code solution that empowers domain experts—regardless of coding ability—to actively drive model improvement workflows. This continuous, automated loop enables models to progressively enhance their performance without manual dataset curation or training job configuration by ML specialists. Human involvement is streamlined to efficient annotation tasks, significantly reducing the costs typically associated with model development. As a result, organizations can effortlessly scale their AI model improvement processes, consistently delivering more reliable and accurate generative AI applications at reduced operational cost. Learn More ---------- * [Try Arize AX for Enterprise](https://app.arize.com/auth/join) * [Get started with Arize Phoenix OSS](https://phoenix.arize.com/) * [Book a demo](https://arize.com/request-a-demo/) **Attending NVIDIA’s GTC conference?** [Meet the Arize team](https://arize.com/nvidia-gtc-2025) ! Subscribe to The Evaluator -------------------------- We’ll send you the latest news, expertise, and product updates from Arize. Your inbox is sacred, so we’ll only curate and send the best stuff. \*We’re committed to your privacy. Arize uses the information you provide to contact you about relevant content, products, and services. You may unsubscribe from these communications at any time. For more information, check out our [privacy policy](https://arize.com/privacy-policy) . --- # Arize AI Introduces Next Generation of Its Machine Learning Observability Platform, Goes Self-Serve For Any Organization Seeking Optimize AI Investments - Arize AI Arize AI Introduces Next Generation of Its Machine Learning Observability Platform, Goes Self-Serve For Any Organization Seeking Optimize AI Investments ======================================================================================================================================================== ![](https://arize.com/wp-content/uploads/2022/04/g3-logo.png) BERKELEY, Calif., March 29, 2022 — Arize AI, the leader in [machine learning (ML) observability](https://arize.com/ml-observability/) and model performance monitoring, today introduced the next generation of its ML observability platform at its Arize:Observe 2022 summit. Arize is the industry’s first and only full-stack ML observability and model performance monitoring platform that is built specifically to solve troubleshooting bottlenecks and pain points experienced every day by thousands of ML engineers, data scientists and other practitioners responsible for deploying and maintaining ML models. With this release, Arize marks a milestone in its evolution, becoming the first ML observability company to offer a full complement of self-serve signup options for every organization – including a free offering that makes it easy for ML engineers to get up and running in minutes. The next-generation Arize platform is battle-proven, deployed by some of the world’s most respected and advanced ML organizations to help quickly detect issues the moment they emerge, troubleshoot why they happened, and improve overall model performance. In all, Arize processes hundreds of billions of predictions a month. Included in the release are enhancements to platform features used every day by ML engineers tasked with solving some of their organizations’ most important challenges, allowing teams to better: * Monitor and Identify Drift–Pinpoint drift across model dimensions and values. Track for prediction, data, and concept drift across model dimensions and values, and compare across training, validation, and production environments. * Ensure Data Integrity–Guarantee the quality of model data inputs and outputs with automated checks for missing, unexpected, or extreme values. * Improve Model Performance–Use [ML performance tracing](https://arize.com/blog/machine-learning-performance-tracing/) to automatically pinpoint the source of model performance problems and map back to underlying data issues. * Leverage Explainability–See how a model dimension affects prediction distributions, and leverage SHAP to explain feature importance for specific cohorts. Introducing Arize’s New Self-Serve Options ------------------------------------------ An early pioneer and leader in machine learning (ML) observability and monitoring, Arize AI already tracks hundreds of billions of predictions a month on behalf of large enterprises and disruptive startups. Arize’s newly released self-serve options remove barriers to adoption to ensure that every organization can detect, root cause, and resolve model performance issues faster regardless of the number of models deployed in production. _**[New users can sign up here](https://app.arize.com/auth/join) **_. Featuring an easy integration via an SDK or file ingestion from major cloud storage providers, Arize enables ML teams to begin monitoring and troubleshooting model performance in minutes. “The reality today is that most teams are only doing ‘red light; green light’ model monitoring and haven’t yet embraced true ML observability with ML performance tracing to pinpoint the source of model performance problems before they impact customers or the bottom line,” said Arize Co-Founder and Chief Product Officer Aparna Dhinakaran. “We are changing that with a platform that is purpose-built to tackle the toughest ML observability challenges of the world’s most respected organizations. Customers of all sizes are now able to try, buy and deploy our AI [model monitoring](https://arize.com/model-monitoring/) capabilities and expand their model coverage as their needs change.” Free Offering Jumpstarts AI Observability and Model Monitoring -------------------------------------------------------------- In a [recent survey](https://arize.com/resource/survey-machine-learning-observability-results/) of more than 900 data scientists, engineers and executives, Arize found that 84.3% of data scientists and ML engineers say the time it takes to detect and diagnose problems with a model is an issue for their teams at least some of the time. This challenge is most significant when teams are reliant upon solutions that are not optimized to detect, root cause, and quickly resolve model performance issues. New Arize customers can now select from Free, Pro, Business and Enterprise versions that map directly to the number of models, features used and predictions in production. Any organization that deploys any Arize tier can easily add new capacity and advanced capabilities as their needs expand. The free version of Arize delivers access to the full version of the platform for up to two models, 500 features per model and 500K production predictions per month. About Arize AI Arize AI is a Machine Learning Observability platform that helps ML practitioners successfully take models from research to production with ease. Arize’s automated model monitoring and analytics platform helps ML teams quickly detect issues when they emerge, troubleshoot why they happened, and improve overall model performance. By connecting offline training and validation datasets to online production data in a central inference store, ML teams can streamline model validation, drift detection, data quality checks, and model performance management. Arize AI acts as the guardrail on deployed AI, providing transparency and introspection into historically black box systems to ensure more effective and responsible AI. To learn more about Arize or machine learning observability and monitoring, visit our blog and resource hub. Subscribe to The Evaluator -------------------------- We’ll send you the latest news, expertise, and product updates from Arize. Your inbox is sacred, so we’ll only curate and send the best stuff. \*We’re committed to your privacy. Arize uses the information you provide to contact you about relevant content, products, and services. You may unsubscribe from these communications at any time. For more information, check out our [privacy policy](https://arize.com/privacy-policy) . --- # Arize AI Announces SOC 2 Type II Certification - Arize AI Arize AI Announces SOC 2 Type II Certification ============================================== ![](https://arize.com/wp-content/uploads/2022/04/soc2typeII.png) **Berkeley, California, April 21, 2022** – Arize AI, a leading ML observability company, today announced that the company achieved SOC 2 Type II certification under standards set by the American Institute of Certified Public Accountants (AICPA). SOC 2 (System and Organization Controls) requires a third-party audit that analyzes key criteria such as organization and management, communication, risk assessment, select controls, monitoring controls, system operations, and more. **Arize’s Foundation Of Security, Availability, and Privacy**  Arize’s SOC 2 security certification validates that the company has adequate processes and policies to securely handle both customer and organizational data. With a third-party-vetted security program in place, users can confidently use the Arize platform knowing their data is safe and secure. The ability to dependably handle organizational and customer data starts with processes and policies that Arize has implemented to ensure security is both operationalized and always top of mind. Arize’s security strategy pillars include: * **Business Continuity Plan** – Sets safeguards to ensure Arize is prepared to provide its services regardless of circumstance * **Mobile Device Management** – Ensures all Arize devices are controlled and secured * **Secure Development Lifecycle** – Guarantees the highest quality security guidelines to Arize’s development process and minimizes the number of vulnerabilities within Arize’s software * **Encryption Policies** – Secure data at REST and in transit by using the most modern encryption algorithms This certification comes on the heels of the company’s recent debut of its [self-serve](https://arize.com/pricing/) [ML observability platform](https://arize.com/) , which already tracks hundreds of billions of predictions a month on behalf of large enterprises and disruptive startups. “Our SOC2 Certification is a validation of Arize AI’s security strategy, but it’s really just the beginning,” said Remi Cattiau, Chief Information Security Officer at Arize AI. “Realizing Arize’s mission of making AI work and work for the people necessarily starts with putting security and privacy at the heart of everything we do.” To request a copy of the report, please contact us [here](https://arize.com/contact/) . **About Arize AI** Arize AI is a [Machine Learning Observability](https://arize.com/ml-observability/) platform that helps ML practitioners successfully take models from research to production with ease. Arize’s automated [model monitoring](https://arize.com/model-monitoring/) and analytics platform help ML teams quickly detect issues when they emerge, troubleshoot why they happened, and improve overall model performance. By connecting offline training and validation datasets to online production data in a central inference store, ML teams can streamline [model validation](https://arize.com/blog/ml-model-failure-modes/) , [drift detection](https://arize.com/model-drift/) , [data quality checks](https://arize.com/blog/data-quality-monitoring/) , and [model performance management](https://arize.com/blog/machine-learning-performance-tracing/) . Arize AI acts as the guardrail on deployed AI, providing transparency and introspection into historically black box systems to ensure more effective and [responsible AI](https://www.forbes.com/sites/aparnadhinakaran/?sh=44d750b84958) . To learn more about Arize or machine learning observability and monitoring, visit our [blog](https://arize.com/blog/) and [resource hub](https://arize.com/resource-hub/) . Subscribe to The Evaluator -------------------------- We’ll send you the latest news, expertise, and product updates from Arize. Your inbox is sacred, so we’ll only curate and send the best stuff. \*We’re committed to your privacy. Arize uses the information you provide to contact you about relevant content, products, and services. You may unsubscribe from these communications at any time. For more information, check out our [privacy policy](https://arize.com/privacy-policy) . --- # Arize AI Recognized For MLOps Innovation in 2022 Artificial Intelligence Breakthrough Awards Program - Arize AI Arize AI Recognized For MLOps Innovation in 2022 Artificial Intelligence Breakthrough Awards Program ==================================================================================================== ![](https://arize.com/wp-content/uploads/2022/06/aibreakthrough.jpg) **BERKELEY, Calif., June 23, 2022** – Arize AI, the leader in machine learning (ML) observability and model performance monitoring, today announced that its [Bias Tracing](https://arize.com/blog/machine-learning-bias-tracing/) tool has been named “MLOps Innovation of the Year” in the fifth annual [AI Breakthrough Awards](https://aibreakthroughawards.com/) program conducted by AI Breakthrough, a market intelligence organization that recognizes the top companies, technologies and products in the global Artificial Intelligence (AI) market today. The mission of the AI Breakthrough Awards is to honor excellence and recognize the innovation, hard work and success in a range of AI and machine learning related categories. This year’s program attracted more than 2,950 nominations from over 18 different countries throughout the world. “In today’s world, it has become all too common to read about AI acting in discriminatory ways and existing solutions built to monitor fairness metrics for ML models lack actionability,” said James Johnson, managing director, AI Breakthrough. “Arize Bias Tracing represents a breakthrough innovation in addressing these challenges, helping monitor and take action on model fairness metrics and helps enterprises quickly get to the bottom of where and why disparate impacts are happening. We extend our sincere congratulations to Arize AI for taking home a well-deserved 2022 AI Breakthrough Award.” #### About Arize AI Arize AI is a [machine learning observability](https://arize.com/ml-observability/) platform that helps ML practitioners successfully take models from research to production with ease. Arize’s automated [model monitoring](https://arize.com/model-monitoring/) and analytics platform helps ML teams quickly detect issues when they emerge, troubleshoot why they happened, and improve overall model performance. By connecting offline training and validation datasets to online production data in a central inference store, ML teams can streamline model validation, drift detection, data quality checks, and [model performance management](https://arize.com/wp-content/uploads/2022/04/Model-Performance-Management-Paper.pdf) . Arize AI acts as the guardrail on deployed AI, providing transparency and introspection into historically black box systems to ensure more effective and responsible AI. To learn more about Arize or machine learning observability and monitoring, visit our blog and resource hub. About AI Breakthrough Part of Tech Breakthrough, a leading market intelligence and recognition platform for global technology innovation and leadership, the AI Breakthrough Awards program is devoted to honoring excellence in Artificial Intelligence technologies, services, companies and products. The AI Breakthrough Awards provide public recognition for the achievements of AI companies and products in categories including AI Platforms, Robotics, Business Intelligence, AI Hardware, NLP, Vision, Biometrics and more. For more information visit AIBreakthroughAwards.com. Subscribe to The Evaluator -------------------------- We’ll send you the latest news, expertise, and product updates from Arize. Your inbox is sacred, so we’ll only curate and send the best stuff. \*We’re committed to your privacy. Arize uses the information you provide to contact you about relevant content, products, and services. You may unsubscribe from these communications at any time. For more information, check out our [privacy policy](https://arize.com/privacy-policy) . --- # Arize AI Named To Fast Company’s List of the Best Workplaces for Innovators - Arize AI Arize AI Named To Fast Company’s List of the Best Workplaces for Innovators =========================================================================== ![](https://arize.com/wp-content/uploads/2022/03/Arize-logo-pink-black.png) ![](https://arize.com/wp-content/uploads/2022/08/Award-Announcement_Fast-Company.jpg) **Berkeley, Calif., August 2, 2022 –** _Fast Company_ today announced its fourth annual “[Best Workplaces for Innovators](https://www.fastcompany.com/90769079/best-workplaces-for-innovators-2022-4-diversity-category-standouts) ” list, honoring organizations and businesses that demonstrate a steadfast commitment to encouraging innovation at all levels. Arize is listed in the all-new [Diverse Innovators](https://www.fastcompany.com/90769079/best-workplaces-for-innovators-2022-4-diversity-category-standouts) category. Developed in collaboration with Accenture, the 2022 Best Workplaces for Innovators ranks 100 winners from a variety of industries, including computer science, biotech, consumer packaged goods, nonprofit, education, financial services, cybersecurity, engineering, diversity, sustainability, B2B, and consumer products and services. _Fast Company_ editors and Accenture researchers worked together to score nearly 1,500 applications, and a panel of eight eminent judges reviewed and endorsed the top 100 companies. The 2022 awards feature workplaces from around the world. “We are proud to be recognized not only as a top workplace for innovators, but also for helping to create a more representative future for AI,” says Aparna Dhinakaran, Co-Founder and Chief Product Officer at Arize. “Diversity propels innovation because you need different perspectives to truly understand a problem and the potential impacts on society at large.” Today’s recognition comes on the heels of a slew of recent product innovations from Arize. Most recently, the company debuted a groundbreaking product for [monitoring unstructured data](https://arize.com/blog/monitor-unstructured-data-with-arize/) . The company also launched [Bias Tracing](https://arize.com/blog/machine-learning-bias-tracing/) , a tool designed to help monitor and take action on model fairness metrics, earlier this year. “This year’s list of the Best Workplaces for Innovators recognizes organizations that have demonstrated a deep commitment to cultivating creativity across the board,” says Brendan Vaughan, editor-in-chief of _Fast Company_. “In the face of powerful headwinds, these leaders and teams continue to spur innovation.” **About Fast Company** Fast Company is the only media brand fully dedicated to the vital intersection of business, innovation, and design, engaging the most influential leaders, companies, and thinkers on the future of business. Headquartered in New York City, Fast Company is published by Mansueto Ventures LLC, along with our sister publication Inc., and can be found online at [www.fastcompany.com](http://www.fastcompany.com/) . **About Accenture** Accenture is a global professional services company with leading capabilities in digital, cloud and security. Combining unmatched experience and specialized skills across more than 40 industries, we offer Strategy and Consulting, Technology and Operations services and Accenture Song —all powered by the world’s largest network of Advanced Technology and Intelligent Operations centers. Our 710,000 people deliver on the promise of technology and human ingenuity every day, serving clients in more than 120 countries. We embrace the power of change to create value and shared success for our clients, people, shareholders, partners and communities. Visit us at accenture.com. **About Arize AI**  Arize AI is a [machine learning observability](https://arize.com/model-monitoring) platform that helps ML practitioners successfully take models from research to production with ease. Arize’s automated [model monitoring](https://arize.com/model-monitoring/) and analytics platform helps ML teams quickly detect issues when they emerge, troubleshoot why they happened, and improve overall model performance. By connecting offline training and validation datasets to online production data in a central inference store, ML teams can streamline [model validation](https://arize.com/ml-model-failure-modes/) , [drift detection](https://arize.com/take-my-drift-away/) , [data quality checks](https://arize.com/data-quality-monitoring/) , and [model performance management](https://arize.com/resource/modern-model-performance-management/) . Arize AI acts as the guardrail on deployed AI, providing transparency and introspection into historically black box systems to ensure more effective and [responsible AI](https://www.forbes.com/sites/aparnadhinakaran/?sh=5d7691024958) . [Sign up for a free account](https://app.arize.com/auth/join) or [request a demo](https://arize.com/request-a-demo/) for your team at Arize.com. Subscribe to The Evaluator -------------------------- We’ll send you the latest news, expertise, and product updates from Arize. Your inbox is sacred, so we’ll only curate and send the best stuff. \*We’re committed to your privacy. Arize uses the information you provide to contact you about relevant content, products, and services. You may unsubscribe from these communications at any time. For more information, check out our [privacy policy](https://arize.com/privacy-policy) . --- # Arize AI Expands Partnership with Google Cloud To Accelerate Machine Learning Observability - Arize AI Arize AI Expands Partnership with Google Cloud To Accelerate Machine Learning Observability =========================================================================================== ![](https://arize.com/wp-content/uploads/2022/03/Arize-logo-pink-black.png) Berkeley, CA, September 29, 2022 – Arize AI announced today that its machine learning (ML) observability platform is [now available on Google Cloud Marketplace](https://console.cloud.google.com/marketplace/product/arize/arize-ai?pli=1) . This availability marks the expansion of the company’s partnership with Google Cloud, which will help Arize deliver its platform – which tracks billions of model predictions daily – to more customers globally. With Arize available on Google Cloud Marketplace, customers already leveraging Google Cloud can easily deploy Arize to their cloud environment, speeding their time-to-value. In a matter of minutes, teams can start streamlining their ML troubleshooting efforts with Arize through purpose-built workflows and analytics for [model performance management](https://arize.com/resource/modern-model-performance-management/) , drift detection, data quality checks, and model validation. This will also provide greater migration support to existing Arize customers as they move their on-prem Arize instances onto Google Cloud’s global infrastructure. In addition to bringing its platform to Google Cloud Marketplace, Arize is significantly expanding its use of Google Cloud technology and services as a part of this expanded partnership. Arize, which runs its platform on Google Cloud’s secure infrastructure, will increase its use of Google Kubernetes Engine (GKE) to support its hosting production, developer onboarding, and application data management as it scales to meet the growing demand for its solution. “AI continues to change the way organizations automate operations and deliver innovative products and solutions to customers,” said Drew Bradstock, Director, GKE Product Management, Google Cloud. “We’re proud to support the growth of innovators like Arize with the infrastructure, cloud technologies, and go-to-market expertise needed to empower more customers and their machine learning teams with robust ML observability solutions.” “ML teams count on Arize to be able to handle current and future analytics complexity and scale, and we built the platform with that in mind,” said Michael Schiff, Chief Architect and Founding Engineer at Arize. “Having a partner like Google Cloud has been invaluable to Arize’s growth, and we look forward to delivering our solutions to more companies through this expanded partnership.” **About Arize AI**  Arize AI is a machine learning observability platform that helps ML teams deliver and maintain more successful AI in production. Arize’s automated model monitoring and observability platform allows ML teams to quickly detect issues when they emerge, troubleshoot why they happened, and improve overall model performance across both structured and unstructured data. Arize is a remote first company with headquarters in Berkeley, CA. [Sign up for a free account](https://app.arize.com/auth/join) or [book a demo for your team](https://arize.com/request-a-demo/) at Arize.com. Subscribe to The Evaluator -------------------------- We’ll send you the latest news, expertise, and product updates from Arize. Your inbox is sacred, so we’ll only curate and send the best stuff. \*We’re committed to your privacy. Arize uses the information you provide to contact you about relevant content, products, and services. You may unsubscribe from these communications at any time. For more information, check out our [privacy policy](https://arize.com/privacy-policy) . --- # Arize AI Launches Industry-First LLM Observability Tool - Arize AI Arize AI Launches Industry-First LLM Observability Tool ======================================================= ![](https://arize.com/wp-content/uploads/2023/04/logo-arize.svg) **Berkeley, CA, May 18, 2023** – Arize AI, a market leader in machine learning observability, debuted new capabilities for fine tuning and monitoring large language models (LLMs) today. The offering brings greater control and insight to teams looking to build with LLMs. As the industry re-tools and data scientists begin to apply foundational models to new use cases, there is a distinct need for new LLMOps tools to reliably evaluate, monitor, and troubleshoot these models. According to a [recent survey](https://arize.com/blog/survey-massive-retooling-around-large-language-models-underway/) , 43% of machine learning teams cite “accuracy of responses and hallucinations” as among the biggest barriers to production deployment of LLMs. Now available as part of the free product, Arize’s [LLM observability](https://arize.com/llm/) tool is the first to evaluate LLM responses, pinpoint where to improve with prompt engineering, and identify fine-tuning opportunities using vector similarity search. The new offering is built to work in tandem with [Phoenix](https://phoenix.arize.com/) , an open source library for LLM evaluation that launched onstage at Arize:Observe. [![](https://arize.com/wp-content/uploads/2023/05/llm-arize-observability.jpg)](https://arize.com/llm/) Leveraging Arize, teams can: * **Detect Problematic Prompts and Responses**: By monitoring a model’s prompt/response embeddings performance using LLM evaluation scores and cluster analysis, teams can narrow in on areas their LLM needs improvement. * **Analyze Clusters Using LLM Evaluation Metrics and GPT-4**: Automatically generate clusters of semantically similar data points and sort by performance. Arize supports LLM-assisted evaluation metrics, task-specific metrics, along with user feedback. An integration with ChatGPT also enables teams to analyze clusters for deeper insights. * **Improve LLM Responses with Prompt Engineering**: Pinpoint prompt/response clusters with low evaluation scores. Workflows suggest ways to augment prompts to help your LLM models generate better responses and improve acceptance rates. * **Fine-Tune Your LLM Using Vector Similarity Search:** Find problematic clusters, such as inaccurate or unhelpful responses, to fine-tune with better data. Vector-similarity search clues you into other examples of issues emerging, so you can begin data augmentation before they become systemic. * **Leverage Pre-Built Clusters for Prescriptive Analysis:** Use pre-built global clusters identified by Arize algorithms, or define custom clusters of your own to simplify RCA and make prescriptive improvements to your generative models. “Despite the power of these models, the risk of deploying LLMs in high risk environments can be immense,” notes Jason Lopatecki, CEO and Co-Founder of Arize. “As new applications get built, Arize LLM observability is here to provide the right guardrails to innovate with this new technology safely.” **About Arize AI**  Arize AI is a machine learning observability platform that helps ML teams deliver and maintain more successful AI in production. Arize’s automated model monitoring and observability platform allows ML teams to quickly detect issues when they emerge, troubleshoot why they happened, and improve overall model performance across both structured data and image and large language models. Arize is a remote-first company with headquarters in Berkeley, CA. Subscribe to The Evaluator -------------------------- We’ll send you the latest news, expertise, and product updates from Arize. Your inbox is sacred, so we’ll only curate and send the best stuff. \*We’re committed to your privacy. Arize uses the information you provide to contact you about relevant content, products, and services. You may unsubscribe from these communications at any time. For more information, check out our [privacy policy](https://arize.com/privacy-policy) . --- # Arize AI Recognized As 2023 “Best MLOps Company” in Sixth Annual AI Breakthrough Awards - Arize AI Arize AI Recognized As 2023 “Best MLOps Company” in Sixth Annual AI Breakthrough Awards ======================================================================================= ![](https://arize.com/wp-content/uploads/2023/06/AI-Breakthrough-Awards-reources-arize.jpg) **BERKELEY, Calif., June 21, 2023 –** Arize AI, a market leader in machine learning observability, today announced that it has been selected as winner of the “Best MLOps Company” award in the sixth annual AI Breakthrough Awards program conducted by AI Breakthrough, a market intelligence organization that recognizes the top companies, technologies and products in the global artificial intelligence (AI) market. Key factors in Arize winning the category include the company’s [open source contributions](https://phoenix.arize.com/) and pioneering work in [LLMOps](https://arize.com/blog-course/llmops-operationalizing-llms-at-scale/) . ![mlops company of the year 2023](https://arize.com/wp-content/uploads/2023/06/AI-Breakthrough-Awards-MLOps-Arize.jpg) The mission of the AI Breakthrough Awards is to honor excellence and recognize the innovation, hard work and success in a range of AI and machine learning related categories, including generative AI, computer vision, AIOps, deep learning, robotics, natural language processing, industry-specific AI applications and many more. This year’s program attracted more than 3,200 nominations from over 20 different countries throughout the world. “Arize helps teams monitor and improve model performance across billions of daily predictions and troubleshoot incredibly complex systems. We’re thrilled to name them ‘Best MLOps Company’ at such a vital time in AI development. Despite calls to halt this development, the reality is that innovation will continue to accelerate,” said James Johnson, managing director, AI Breakthrough. “What the industry needs is guardrails to ensure that AI isn’t fueling inequity, inadvertently releasing sensitive information, churning out false information, or any other host of problems an LLM might create if not monitored. Arize provides ML teams with tools to understand whether their models are performing as expected and to quickly get to the cause behind any issues before there’s a problem.” **About Arize AI** Arize AI is a machine learning observability platform that helps ML teams deliver and maintain more successful AI in production. Arize’s automated model monitoring and observability platform allows ML teams to quickly detect issues when they emerge, troubleshoot why they happened, and improve overall model performance across both structured data and image and large language models. Arize is a remote-first company with headquarters in Berkeley, CA. **About AI Breakthrough** Part of Tech Breakthrough, a leading market intelligence and recognition platform for global technology innovation and leadership, the AI Breakthrough Awards program is devoted to honoring excellence in Artificial Intelligence technologies, services, companies and products. The AI Breakthrough Awards provide public recognition for the achievements of AI companies and products in categories including AI Platforms, Robotics, Business Intelligence, AI Hardware, NLP, Computer Vision and more. For more information visit AIBreakthroughAwards.com. Subscribe to The Evaluator -------------------------- We’ll send you the latest news, expertise, and product updates from Arize. Your inbox is sacred, so we’ll only curate and send the best stuff. \*We’re committed to your privacy. Arize uses the information you provide to contact you about relevant content, products, and services. You may unsubscribe from these communications at any time. For more information, check out our [privacy policy](https://arize.com/privacy-policy) . --- # Arize AI Raises $38 Million Series B To Scale Machine Learning Observability Platform - Arize AI Arize AI Raises $38 Million Series B To Scale Machine Learning Observability Platform ===================================================================================== ![](https://arize.com/wp-content/uploads/2022/03/Arize-logo-pink-black.png) Berkeley, CA, September 8, 2022 – Arize AI, a leader in machine learning observability, announced today that it has raised $38 million in Series B funding. TCV led the round with participation from existing investors Battery Ventures, Foundation Capital, and Swift Ventures. The investment is the largest-ever in a machine learning observability platform and comes at an important inflection point for the industry. Machine learning models are now being deployed in nearly every sector of the economy, with companies [investing billions](https://www.wsj.com/articles/retail-set-to-overtake-banking-in-ai-spending-11631007001) to turn artificial intelligence (AI) and machine learning (ML) into a competitive advantage. Despite a decade of investment in data infrastructure and the pre-production ML toolchain, most companies still [lack visibility](https://arize.com/resource/survey-machine-learning-observability-results/) into how their ML models are performing in production and run the risk of models [impacting earnings](https://arize.com/blog/when-ai-attacks-earnings/) or [acting in unfair ways](https://arize.com/blog/machine-learning-bias-tracing/) . This is especially true as the industry embraces computer vision and natural language processing models that are notoriously difficult to troubleshoot since they rely on unstructured data and manual labeling by humans. Launched in 2020, Arize’s ML observability platform is already counted on by a growing list of enterprises and disruptive technology companies – including Uber, Spotify, Ebay, Chime, Neustar, Nextdoor, New York Life, Stitch Fix, and more – to track hundreds of billions of predictions per month. Arize is [beloved by ML engineering teams](https://arize.com/customers/) because it enables them to streamline troubleshooting efforts with purpose-built workflows and analytics for model performance management, drift detection, data quality checks, and model validation. Arize also enables users to log models with both structured and [unstructured data](https://arize.com/blog/monitor-unstructured-data-with-arize/) to the platform for monitoring. “Michaelangelo is Uber’s end-to-end ML platform that powers 100% business-critical ML use cases at Uber to deliver a consistent user experience across billions of rides and deliveries,” says Kai Wang, Product Lead, Uber AI Platform. “Given the vital role ML plays in this process, it’s critical to have tools that build on Michalangelo’s core capabilities and help us stay ahead of potential production ML problems. We’re excited to work with Arize AI to enhance platform ML observability capabilities and make it easier to detect and resolve model performance issues.” “Arize’s platform finally makes it easy for ML engineers to scalably detect data and drift issues, troubleshoot what happened, and improve overall model performance” says Morgan Gerlak, Partner at TCV. “Like other areas of observability, the end user really matters — and we were impressed by Arize’s ability to build a practical solution that ML engineers love.” “As the pace of innovation in AI accelerates, it’s more important than ever for organizations to have machine learning observability in place to surface potential problems and improve ML models in production,” says Dharmesh Thakker, a general partner with Battery Ventures. “In the past year, Arize has emerged as one of the highest-profile companies in this space — the platform of choice for many prominent ML teams. We’re proud to expand our investment and partnership with Arize as it pushes into new frontiers.” “Speaking of observability, I’ve been watching Arize grow since its inception,” says Ashu Garg, general partner at Foundation Capital, “and I’m thrilled at how far the company has come. Its product is far ahead of the competition and is being deployed by best-in-class AI enterprises, which all acknowledge the seriousness of the problem. In two short years, Arize has become _the_ breakout company in its category.” “The reality is that if you’re not injecting AI into every major business decision, you are going to be left behind,” notes Brett Wilson, Co-Founder and General Partner at Swift Ventures. “Having machine learning observability in place to monitor models and get ahead of potential problems is table stakes, especially in a challenging economic environment. Arize is the category leader in this space and is pushing it to new frontiers.” **About Arize AI**  Arize AI is a machine learning observability platform that helps ML teams deliver and maintain more successful AI in production. Arize’s automated model monitoring and observability platform allows ML teams to quickly detect issues when they emerge, troubleshoot why they happened, and improve overall model performance across both structured and unstructured data. Arize is a remote first company with headquarters in Berkeley, CA. [Sign up for a free account](https://app.arize.com/auth/join) or [book a demo for your team](https://arize.com/request-a-demo/) at Arize.com. Subscribe to The Evaluator -------------------------- We’ll send you the latest news, expertise, and product updates from Arize. Your inbox is sacred, so we’ll only curate and send the best stuff. \*We’re committed to your privacy. Arize uses the information you provide to contact you about relevant content, products, and services. You may unsubscribe from these communications at any time. For more information, check out our [privacy policy](https://arize.com/privacy-policy) . --- # Arize AI Honored In “On The Rise” Category of Fast Company’s 2023 World Changing Ideas Awards - Arize AI Arize AI Honored In “On The Rise” Category of Fast Company’s 2023 World Changing Ideas Awards ============================================================================================= ![](https://arize.com/wp-content/uploads/2023/04/logo-arize.svg) Berkeley, CA, May 2, 2023 — The honorees of Fast Company’s 2023 World Changing Ideas Awards were announced today, honoring sustainable designs, innovative products, bold social initiatives, and other creative projects that are changing the way we work, live, and interact with the world. Arize AI, a leader in machine learning observability, was honored in the “On the Rise: 0-4 Years In Business” category for delivering tools that empower practitioners to achieve greater fairness in AI. A panel of Fast Company editors and reporters selected winners from a pool of more than 2,200 entries across urban design, education, nature, politics, technology, corporate social responsibility, and more. The 2023 awards feature entries from across the globe, from Italy to Singapore to New Zealand. ![world changing ideas](https://arize.com/wp-content/uploads/2023/05/FastCompany-World-Changing-Ideas-On-the-Rise-2023.jpg) “It’s thrilling to see the creativity and innovation that are so abundant among this year’s honorees,” says Fast Company editor-in-chief Brendan Vaughan. “While it’s easy to feel discouraged by the state of the world, the entrepreneurs, companies, and nonprofits featured in this package show the limitless potential to address society’s most urgent problems. Our journalists have highlighted some of the most exciting and impactful work being done today—from housing to equity to sustainability—and we look forward to seeing not only how these projects evolve but how they inspire others to develop solutions of their own.” **About the World Changing Ideas Awards**  World Changing Ideas is one of Fast Company’s major annual awards programs and is focused on social good, seeking to elevate finished products and brave concepts that make the world better. A panel of judges from across sectors choose winners, finalists, and honorable mentions based on feasibility and the potential for impact. With the goals of awarding ingenuity and fostering innovation, Fast Company draws attention to ideas with great potential and helps them expand their reach to inspire more people to start working on solving the problems that affect us all. **About Arize AI**    Arize AI is a machine learning observability platform that helps ML teams deliver and maintain more successful AI in production. Arize’s automated [model monitoring](https://arize.com/model-monitoring/) and observability platform allows ML teams to quickly detect issues when they emerge, troubleshoot why they happened, and improve overall model performance across both structured and unstructured data. Arize is a remote first company with headquarters in Berkeley, CA. Subscribe to The Evaluator -------------------------- We’ll send you the latest news, expertise, and product updates from Arize. Your inbox is sacred, so we’ll only curate and send the best stuff. \*We’re committed to your privacy. Arize uses the information you provide to contact you about relevant content, products, and services. You may unsubscribe from these communications at any time. For more information, check out our [privacy policy](https://arize.com/privacy-policy) . --- # Arize Debuts Phoenix, the First Open Source Library for Evaluating Large Language Models - Arize AI Arize Debuts Phoenix, the First Open Source Library for Evaluating Large Language Models ======================================================================================== ![](https://arize.com/wp-content/uploads/2023/06/Data-Talks-Club-promo.gif) **Berkeley, CA, April 26, 2023** – Arize AI, a market leader in machine learning observability, debuted deeper support on the Arize platform for generative AI and a first-of-its-kind open source observability library for evaluating large language models (LLMs) at its Arize:Observe 2023 summit today. The launch comes at a critical moment for the future of AI. Generative AI is fueling a technical renaissance, with models like GPT-4 showing [sparks](https://arxiv.org/pdf/2303.12712.pdf) of artificial general intelligence and new breakthroughs and use cases emerging daily. On the other hand, most leading large language models are black boxes that have [known](https://www.nytimes.com/2023/03/15/technology/gpt-4-artificial-intelligence-openai.html) [issues](https://www.nytimes.com/2023/03/29/technology/ai-chatbots-hallucinations.html) around hallucination and problematic biases. Available today, Phoenix is the first [open source observability library](https://phoenix.arize.com/) specifically built to help data scientists evaluate outputs from LLMs like OpenAI’s GPT-4, Google’s Bard, Anthropic’s Claude, and others. Leveraging Phoenix, data scientists can visualize complex LLM decision-making, monitor LLMs when they produce false or misleading results, and narrow in on fixes to improve outcomes. ![arize phoenix in action llm observability](https://arize.com/wp-content/uploads/2023/04/arize-phoenix-viz-image.png) “A huge barrier in getting LLMs and Generative Agents to be deployed into production is because of the lack of observability into these systems,” says Harrison Chase, Co-Founder of LangChain. “With Phoenix, Arize is offering an open source way to visualize complex LLM decision-making.” “Phoenix is a much-appreciated advancement in model observability and production,” says Christopher Brown, CEO and Co-Founder of AI-focused consulting firm Decision Patterns and a former Computer Science lecturer at UC Berkeley. “The integration of observability utilities directly into the development process not only saves time but encourages model development and production teams to actively think about model use and ongoing improvements before releasing to production. This is a big win for management of the model lifecycle.” “Despite calls to halt AI development, the reality is that innovation will continue to accelerate,” said Jason Lopatecki, CEO and Co-Founder of Arize AI. “Phoenix is the first software designed to help data scientists understand how GPT-4 and LLMs think, monitor their responses and fix the inevitable issues as they arise.” Phoenix is instantiated by a simple import call in a Jupyter notebook and is built to interactively run on top of Pandas dataframes. The tool works easily with unstructured text and images, with embeddings and latent structure analysis designed as a core foundation of the toolset. Leveraging Phoenix, data scientists can: * **_Evaluate LLM Tasks_****:** Troubleshoot tasks such as summarization or question/answering to find problem clusters with misleading or false answers. * **_Detect Anomalies_****:** Using LLM embeddings * **_Find Clusters of Issues to Export for Model Improvement_**: Find clusters of problems using performance metrics or drift. Export clusters for fine-tuning workflows. * **_Surface Model Drift and Multivariate Drift_**: Use embedding drift to surface data drift for generative AI, LLMs, computer vision (CV) and tabular models. * **_Easily Compare A/B Datasets_**: Uncover high-impact clusters of data points missing from model training data when comparing training and production datasets. * **_Discover How Embeddings Represent Your Data_**: Map structured features onto embeddings for deeper insights into how embeddings represent your data. * **_Monitoring Analysis to Pinpoint Issues_**: Monitor model performance and track down issues through exploratory data analysis. **About Arize AI**    Arize AI is a machine learning observability platform that helps ML teams deliver and maintain more successful AI in production. Arize’s automated [model monitoring](https://arize.com/model-monitoring/) and observability platform allows ML teams to quickly detect issues when they emerge, troubleshoot why they happened, and improve overall model performance across both structured and unstructured data. Arize is a remote first company with headquarters in Berkeley, CA. Subscribe to The Evaluator -------------------------- We’ll send you the latest news, expertise, and product updates from Arize. Your inbox is sacred, so we’ll only curate and send the best stuff. \*We’re committed to your privacy. Arize uses the information you provide to contact you about relevant content, products, and services. You may unsubscribe from these communications at any time. For more information, check out our [privacy policy](https://arize.com/privacy-policy) . --- # Arize AI Debuts General Availability of Embedding Drift Measurement - Arize AI Arize AI Debuts General Availability of Embedding Drift Measurement =================================================================== ![](https://arize.com/wp-content/uploads/2022/03/Arize-logo-pink-black.png) **Berkeley, CA, January 9, 2022** – Arize AI announced today that it is making embedding drift monitoring available to all customers and free users of the company’s leading machine learning (ML) observability platform. The debut follows a beta in which over 20 enterprises and startups tested Arize’s embedding drift monitoring across billions of model predictions, resulting in over $10 million in savings from improved model performance and reductions in labeling costs. Arize’s full rollout of [embedding drift](https://arize.com/blog/embedding-drift/) monitoring comes at a time of great need in the industry. Despite [investing billions](https://www.wsj.com/articles/tech-giants-pour-billions-into-ai-but-hype-doesnt-always-match-reality-11656508394&sa=D&source=docs&ust=1673033009865698&usg=AOvVaw2kwIZF60EJp_cFDLC6Tdsg) in computer vision and natural language processing models to do everything from detect cancer to improve crop yields, most organizations still [lack visibility](https://arize.com/resource/survey-machine-learning-observability-results/) into what is happening when unstructured models are put into production. Since metrics typically used to measure changes in data distributions (drift) in structured data simply do not extend to unstructured data, ML teams often miss upstream data quality issues and new patterns in the data before they impact model performance and business results. Arize’s tool helps ameliorate this problem by enabling ML teams to easily compare [embeddings](https://arize.com/blog/getting-started-with-embeddings-is-easier-than-you-think/) (vector representations of data) across different periods of time using sensitive and scalable metrics like euclidean distance and cosine distance. Leveraging this unique approach to embedding drift monitoring, ML practitioners can now better identify new patterns in the data, prioritize what to label next, and focus retraining efforts to proactively improve model performance. “Gone are the days of shipping CV and NLP models blind,” notes Jason Lopatecki, CEO and Co-Founder of Arize. “We’re proud to make embedding drift measurement available to all after a year of [extensive research](https://arize.com/blog/embedding-drift/) and development on a large variation of scenarios and data.” ![embedding drift in arize](https://arize.com/wp-content/uploads/2023/01/embedding-drift.png) **About Arize AI**    Arize AI is a machine learning observability platform that helps ML teams deliver and maintain more successful AI in production. Arize’s automated [model monitoring](https://arize.com/model-monitoring/) and observability platform allows ML teams to quickly detect issues when they emerge, troubleshoot why they happened, and improve overall model performance across both structured and unstructured data. Arize is a remote first company with headquarters in Berkeley, CA. [Sign up](https://app.arize.com/auth/join) for a free account or [book a demo](https://arize.com/request-a-demo/) for your team at Arize.com. Subscribe to The Evaluator -------------------------- We’ll send you the latest news, expertise, and product updates from Arize. Your inbox is sacred, so we’ll only curate and send the best stuff. \*We’re committed to your privacy. Arize uses the information you provide to contact you about relevant content, products, and services. You may unsubscribe from these communications at any time. For more information, check out our [privacy policy](https://arize.com/privacy-policy) . --- # Arize AI Debuts Monitoring for Unstructured Data - Arize AI Arize AI Debuts Monitoring for Unstructured Data ================================================ ![](https://arize.com/wp-content/uploads/2022/03/Arize-logo-pink-black.png) **BERKELEY, Calif., June 30, 2022 —** [Arize AI, the leader in](https://arize.com/) machine learning (ML) observability, debuted a groundbreaking product for embedding drift monitoring and embedding analysis today. According to [multiple estimates](https://mitsloan.mit.edu/ideas-made-to-matter/tapping-power-unstructured-data) , 80% of data generated is unstructured audio, images, text, or video (as opposed to structured data like rows of dates, numbers, and addresses). Machine learning teams are putting this data to great use, with computer vision and natural language processing (NLP) models powering everything from self-driving cars to classifying long legal documents. Despite a decade of investment in deep learning, however, there has not been a great way to monitor these models as performance shifts in production – until now. Now available as part of Arize’s free subscription tier, Arize for embedding analysis enables users to log models with both structured and unstructured data to Arize for monitoring. By monitoring [embeddings](https://arize.com/blog/getting-started-with-embeddings-is-easier-than-you-think/) of their unstructured data, teams can proactively identify when their unstructured data is drifting. Troubleshooting is simplified with interactive visualizations to help isolate new or emerging patterns, underlying data changes, and data quality issues. This update is designed to tackle several common pain points of working with deep learning models:  * **ML teams often lack visibility into what’s happening to the data when an unstructured data model is put into production**. With no monitoring for drift or performance, picking up on upstream data quality issues or change in the data is practically impossible. * **Deep learning models are expensive to train**. Since labeling is expensive, ML teams often only label as much as 0.1% of their data. When models are then put into production, it often results in new patterns emerging that the model hadn’t encountered in training. Gone unnoticed, these new patterns lead to performance degradation. Arize’s interactive UMAP implementation with both 2D and 3D views enables teams to quickly visualize their high dimensional data in a low dimensional space. By visualizing drift between embeddings with production data layered on top of training data, teams are able to see groupings of embeddings and easily identify patterns or data that were not present in training. ![](https://1591756861-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-MAlgpMyBRcl2qFZRQ67%2Fuploads%2FmeI86JeNt3VHq8zyamRJ%2FScreen%20Recording%202022-06-08%20at%2011.30.09%20AM.gif?alt=media&token=ffeda305-0423-4cb4-9c36-e72a605d9cee) “It has been an amazing journey over the last year scaling Arize to track hundreds of billions predictions a month, and we have learned a lot. We plowed many of those insights into an architecture for embedding analysis that is the first of its kind,” says Jason Lopatecki, CEO and Co-Founder of Arize. “Most teams are shipping deep learning models blind today, and this product is built to help change that.” Aparna Dhinakaran, Arize’s Co-Founder and Chief Product Officer, concurs: “Cutting edge deep learning still relies on human labeling teams looking at around 1% of the data to help train models and – hopefully – capture what will happen in the real world. Better monitoring is needed to surface issues with models in production. Arize’s new capabilities for monitoring unstructured data promise to change the game for ML teams.” **About Arize AI** Arize AI is a machine learning observability platform that helps ML practitioners successfully take models from research to production with ease. Arize’s automated model monitoring and analytics platform helps ML teams quickly detect issues when they emerge, troubleshoot why they happened, and improve overall model performance. By connecting offline training and validation datasets to online production data in a central inference store, ML teams can streamline model validation, drift detection, data quality checks, and model performance management. Arize AI acts as the guardrail on deployed AI, providing transparency and introspection into historically black box systems to ensure more effective and responsible AI. To learn more about Arize or machine learning observability and monitoring, visit our blog and resource hub. Subscribe to The Evaluator -------------------------- We’ll send you the latest news, expertise, and product updates from Arize. Your inbox is sacred, so we’ll only curate and send the best stuff. \*We’re committed to your privacy. Arize uses the information you provide to contact you about relevant content, products, and services. You may unsubscribe from these communications at any time. For more information, check out our [privacy policy](https://arize.com/privacy-policy) . --- # Arize Debuts Data Lake Connectors - Arize AI Arize Debuts Data Lake Connectors ================================= ![](https://arize.com/wp-content/uploads/2022/03/Arize-logo-pink-black.png) **Berkeley, CA, January 23, 2023** – [Arize AI](https://arize.com/) , a market leader in machine learning observability, today launched a data lake connectivity solution for BigQuery, Delta Lake, Redshift, and Snowflake. Through Arize Data Lake Connectors, Arize clients with centralized inference stores can instantly link up their ML table data to Arize for robust model observability. Arize leads the industry in both volume of models and predictions monitored, topping billions of predictions daily. To date, ML observability platforms have struggled to make their deployments easy while handling billions of predictions and complex monitoring services, such as [embedding drift](https://arize.com/blog/embedding-drift/) . The new Arize release for data connectors further extends integration options for customers to the most-used data lakes. The launch comes as the ML ecosystem begins to converge on a number of MLOps architectures. One modern approach to ML data architecture is designed around storing inference data in a data lake. ML teams are designing these ML data lakes to power feature stores for feature serving and an inference store for analysis. Arize Data Lake Connectors are designed to fit seamlessly into modern data lake architectures. The advantages of connecting directly to the ML data store include: * Teams can run off of a single source of truth * Integration and on-boarding are faster and easier * Financial savings can be significant relative to other approaches to ML monitoring “The growing pool of ML data that is stored and used for ad hoc operational analysis is largely sitting untapped by ML engineering teams,” notes Jason Lopatecki, CEO and co-founder of Arize. “That data, when connected to Arize, empowers iterative workflows around model performance analysis and data improvement – ultimately saving teams time and improving the ROI on AI investments.” Arize already integrates with cloud storage providers (including Amazon Web Services, Google Cloud Platform, and Microsoft Azure), Python pipelines through an SDK, and Kafka Streaming.  With today’s launch, it’s now easier than ever for users of data lakes to access real-time model analytics. Arize offers built-in connectors that are fully managed as part of its cloud and virtual private cloud (VPC) platform, obviating the need for users to build and manage complicated data pipelines or use a separate ETL tool and enabling real-time model performance analysis and monitoring. ML teams interested in learning more can [book a demo](https://arize.com/request-a-demo/) or [sign up](https://app.arize.com/auth/join) for an Arize account. **About Arize AI** Arize AI is a machine learning observability platform that helps ML teams deliver and maintain more successful AI in production. Arize’s automated [model monitoring](https://arize.com/model-monitoring/) and observability platform allows ML teams to quickly detect issues when they emerge, troubleshoot why they happened, and improve overall model performance across both structured and unstructured data. Arize is a remote first company with headquarters in Berkeley, CA. Media Contact: press@arize.com Subscribe to The Evaluator -------------------------- We’ll send you the latest news, expertise, and product updates from Arize. Your inbox is sacred, so we’ll only curate and send the best stuff. \*We’re committed to your privacy. Arize uses the information you provide to contact you about relevant content, products, and services. You may unsubscribe from these communications at any time. For more information, check out our [privacy policy](https://arize.com/privacy-policy) . --- # Arize AI Debuts Observe Copilot, Winning “Coolest Technology” at VB Transform’s Innovation Showcase - Arize AI Arize AI Debuts Observe Copilot, Winning “Coolest Technology” at VB Transform’s Innovation Showcase =================================================================================================== ![](https://arize.com/wp-content/uploads/2021/10/logo2.svg) San Francisco, CA, July 13, 2023 – Arize AI, a market leader in machine learning observability, won “Coolest Technology” at the VB Transform 2023 Innovation Showcase for a newly-launched ChatGPT plugin for LLM observability. The debut comes at a time of great upheaval around generative AI as a majority of machine learning teams plan production deployments of large language models (LLMs) but encounter early challenges like accuracy of responses, poor data retrieval, and hallucinations. Arize Observe Copilot is an [LLM observability](https://arize.com/llm/) add-on designed to assist machine learning and data science practitioners explore and analyze how their LLM models are performing in production. Through an integration with GPT-4, Observe Copilot provides an intuitive, chat-like experience for LLM practitioners to ask questions about their model, perform EDA, and uncover problematic clusters of data points. Ultimately, Observe Copilot enables LLM developers and teams with an easier way to troubleshoot the root cause of ML issues and understand where to focus their improvement and fine-tuning efforts. “As AI gets more complicated in the generative era – with prompts and responses and potentially layers of agents – finding problems in operational data becomes more difficult,” says Jason Lopatecki, CEO and Co-Founder of Arize. “We believe that the future will involve using AI to troubleshoot AI and built Observe Copilot as a first-of-its-kind solution to do just that.” About Arize AI Arize AI is a machine learning observability platform that helps ML teams deliver and maintain more successful AI in production. Arize’s automated model monitoring and observability platform allows ML teams to quickly detect issues when they emerge, troubleshoot why they happened, and improve overall model performance across both structured data and image and large language models. Arize is a remote-first company with headquarters in Berkeley, CA. Subscribe to The Evaluator -------------------------- We’ll send you the latest news, expertise, and product updates from Arize. Your inbox is sacred, so we’ll only curate and send the best stuff. \*We’re committed to your privacy. Arize uses the information you provide to contact you about relevant content, products, and services. You may unsubscribe from these communications at any time. For more information, check out our [privacy policy](https://arize.com/privacy-policy) . --- # Arize AI, LlamaIndex Roll Out Joint Platform for Evaluating LLM Applications - Arize AI Arize AI, LlamaIndex Roll Out Joint Platform for Evaluating LLM Applications ============================================================================ ![](https://arize.com/wp-content/uploads/2022/03/Arize-logo-pink-black.png) Strategic alliance and joint product promises to broaden the adoption of generative AI across industries -------------------------------------------------------------------------------------------------------- **San Francisco, CA, July 11, 2024** – Arize AI, a pioneer and leader in AI observability and LLM evaluation, and LlamaIndex, a leading data framework for LLM applications, debuted a new joint offering today called [LlamaTrace, a hosted version of Arize OSS Phoenix](https://docs.arize.com/phoenix/hosted-phoenix) . According to a soon-to-release survey, 47.7% of AI engineers and developers building generative AI applications are leveraging retrieval today in their LLM Applications. By connecting data to generative AI, orchestration frameworks like LlamaIndex can be game-changers in accelerating generative AI development. However, for many teams and enterprises technical challenges remain in getting modern LLM systems – with layers of abstraction – ready for the real world. To help, Arize and LlamaIndex are debuting an LLM tracing and observability platform that works natively with the LlamaIndex and Arize ecosystem. With a foundation based on [Arize Phoenix OSS](https://phoenix.arize.com/) , the hosted version of Phoenix offers the ability to persist application telemetry data generated during AI development in order to better experiment, iterate, and collaborate in development or production. The solution has a foundation in open source and features a fully hosted, online, persistent deployment option for teams that do not want to self host. AI engineers can instantly log traces, persist datasets, run experiments, run evaluations – and share those insights with colleagues. The new offering is available today, and can be accessed through either a LlamaIndex or Arize account. “We share a vision with LlamaIndex in enabling builders to reduce the time it takes to deploy generative AI into production but in a way that is super battle hardened for business-critical use cases,” said Jason Lopatecki, CEO and Co-Founder of Arize. “As leaders in our respective spaces with a common philosophy in empowering AI engineers and developers, we’re uniquely positioned here to do something that can move modern LLMOps forward and broaden adoption.” “Prototyping a RAG pipeline or agent is easy, but every AI engineer needs the right data processing layer, orchestration framework, and experimentation/monitoring tool in order to take these applications to production. LlamaTrace by Arize offers the richest toolkit we’ve seen in enabling developers to observe, debug, and evaluate every granular step of a very complex LLM workflow, and it nicely complements the production-ready data platform and orchestration framework that LlamaCloud and LlamaIndex offer,” said Jerry Liu, CEO of LlamaIndex. About Arize AI Arize AI is an AI observability and LLM evaluation platform that helps teams deliver and maintain more successful AI in production. Arize’s automated monitoring and observability platform allows teams to quickly detect issues when they emerge, troubleshoot why they happened, and improve overall performance across both traditional ML and generative use cases. Arize is headquartered in Berkeley, CA About LlamaIndex LlamaIndex is a data framework and platform which lets developers easily build LLM applications over their data. LlamaIndex provides an enterprise offering, LlamaCloud, which lets developers efficiently parse, index, and retrieve over a wide range of data sources. Developers can then use the core open-source framework to orchestrate workflows with LLMs to build production-grade applications, ranging from question-answering chatbots to document extraction and summarization to autonomous agents. Subscribe to The Evaluator -------------------------- We’ll send you the latest news, expertise, and product updates from Arize. Your inbox is sacred, so we’ll only curate and send the best stuff. \*We’re committed to your privacy. Arize uses the information you provide to contact you about relevant content, products, and services. You may unsubscribe from these communications at any time. For more information, check out our [privacy policy](https://arize.com/privacy-policy) . --- # Arize:Observe To Gather Top Minds In Generative AI for Day of Learning - Arize AI Arize:Observe To Gather Top Minds In Generative AI for Day of Learning ====================================================================== ![](https://arize.com/wp-content/uploads/2022/03/Arize-logo-pink-black.png) Event will feature research and tech talks from OpenAI, Microsoft, Mistral AI, Meta AI, Fireworks AI, LlamaIndex, Doordash, NATO, and others -------------------------------------------------------------------------------------------------------------------------------------------- San Francisco, CA, June 20, 2024 – Arize AI, a pioneer and market leader in AI observability and large language model (LLM) evaluation, announced an action-packed agenda and speaker roster for the company’s annual conference today. [Now open for registration](https://arize.com/observe-2024/) , Arize:Observe kicks off in-person on July 11th at Shack 15 in San Francisco’s iconic Ferry Building. Event sponsors include Microsoft – which is presenting the Builder’s Track – as well as Battery Ventures, Cerebral Valley, Modelbit, MindsDB, PromptLayer, and Swift Ventures. The event comes at a critical inflection point for the industry. Although enterprises are racing to deploy foundation models, [barriers remain](https://arize.com/blog/llm-survey/) that stand in the way of reliable production deployments. Arize:Observe aims to help AI engineers achieve breakthroughs with generative AI systems in the real world. Tracks include: * **_AI Builders Guild_**:  Hands-on sessions offer pragmatic experience on innovative open-source tools and evaluation methodologies. * **_AI Research Frontiers_**: Cutting-edge research, emerging techniques, and theoretical advancements. * **_AI Innovators_**: Real world use cases, challenges in deploying products, and scaling of AI across organizations; industry-specific considerations to navigate the complexities of enterprise AI deployment. Presenters include prominent foundation model creators, popular open source LLMOps tool developers, and AI researchers. Speakers hail from OpenAI, Anthropic, Google (Gemini team), Meta AI (Llama team), Microsoft (AutoGen and Phi-2 teams), Mistral, LlamaIndex, PromptLayer, TripAdvisor, Lowe’s, NATO, Wayfair, Stanford, UC Berkeley, and others. “We put a lot of thought into building an event that offers both cutting edge learning on research and pragmatic, helpful workshops. We believe Arize:Observe is a must-attend for any AI engineer that wants to move past the tinkering or toy phase,” says Jason Lopatecki, CEO and Co-Founder of Arize AI. **About Arize AI**  Arize AI is an AI observability and [LLM evaluation](https://arize.com/blog-course/llm-evaluation-the-definitive-guide/) platform that helps teams deliver and maintain more successful AI in production. Arize’s automated monitoring and observability platform allows teams to quickly detect issues when they emerge, troubleshoot why they happened, and improve overall performance across both traditional ML and generative use cases. Arize is headquartered in Berkeley, CA. Subscribe to The Evaluator -------------------------- We’ll send you the latest news, expertise, and product updates from Arize. Your inbox is sacred, so we’ll only curate and send the best stuff. \*We’re committed to your privacy. Arize uses the information you provide to contact you about relevant content, products, and services. You may unsubscribe from these communications at any time. For more information, check out our [privacy policy](https://arize.com/privacy-policy) . --- # Arize AI Debuts Prompt Variable Monitoring - Arize AI Arize AI Debuts Prompt Variable Monitoring ========================================== ![](https://arize.com/wp-content/uploads/2022/03/Arize-logo-pink-black.png) Launch at Google Cloud Next ‘24 gives AI engineers unique visibility and control over complex LLM applications -------------------------------------------------------------------------------------------------------------- Las Vegas, NV, April 11, 2023 – Arize AI, a pioneer and market leader in [AI observability](https://arize.com/) and large language model (LLM) evaluation, debuted industry-first capabilities for prompt variable monitoring and analysis onstage at Google Cloud Next ’24 today. The launch comes at a time of critical need. Although enterprises are racing to deploy foundation models to stay competitive in an increasingly AI-driven world, hallucinations and accuracy of responses [remain barriers](https://arize.com/blog/llm-survey/) to production deployments. Arize’s new prompt variable monitoring (create a [free account](https://app.arize.com/auth/join) ) helps AI engineers and ML teams solve that problem by automatically detecting bugs in prompt variables and pinpointing problematic datasets. Through introspection and refinement of the prompts used in LLM-powered applications, teams can ensure that generated outputs align with expectations around metrics such as accuracy, relevance, and correctness. Additional context window management tools also launching today allow for further examination. ![prompt variable monitoring](https://arize.com/wp-content/uploads/2024/04/arize-ai-prompt-variable-analysis.png) “Debugging LLM systems is far too painful today,” says Jason Lopatecki, CEO and Co-Founder of Arize AI. “By analyzing how AI systems respond to a myriad of prompts and offering deeper insights into model behavior, Arize’s new prompt variable analysis tools promise to help AI engineers have more successful outcomes in production — with training and feedback loops to inform ongoing refinement.” **About Arize AI**  Arize AI is an AI observability and LLM evaluation platform that helps teams deliver and maintain more successful AI in production. Arize’s automated monitoring and observability platform allows teams to quickly detect issues when they emerge, troubleshoot why they happened, and improve overall performance across both traditional ML and generative use cases. Arize is headquartered in Berkeley, CA. Subscribe to The Evaluator -------------------------- We’ll send you the latest news, expertise, and product updates from Arize. Your inbox is sacred, so we’ll only curate and send the best stuff. \*We’re committed to your privacy. Arize uses the information you provide to contact you about relevant content, products, and services. You may unsubscribe from these communications at any time. For more information, check out our [privacy policy](https://arize.com/privacy-policy) . --- # Arize AI Introduces AI Copilot - Arize AI Arize AI Introduces AI Copilot ============================== ![](https://arize.com/wp-content/uploads/2022/03/Arize-logo-pink-black.png) Industry-first AI assistant for troubleshooting AI and other new updates promise to speed development for AI engineers ---------------------------------------------------------------------------------------------------------------------- San Francisco, CA, July 11, 2024 – Arize AI, a pioneer and leader in AI observability and LLM evaluation, today debuted new capabilities to help AI developers evaluate and debug LLM systems. The premiere is one among many taking place at the Arize:Observe conference today, where speakers – including OpenAI, Lowe’s, Mistral, Microsoft, NATO, and others – are sharing the latest research, engineering best practices, and open source frameworks. [Arize Copilot](https://docs.arize.com/arize/copilot/how-to-copilot) – the industry’s first AI assistant to troubleshoot AI systems – is a new tool that surfaces relevant information and suggests actions in the Arize platform, automating complex tasks and taking actions to help AI engineers save time and improve app performance. Examples where Copilot can help out of the box include getting model insights, prompt optimization, building a custom evaluation, and AI search. “Using AI to troubleshoot complex AI systems is a logical next step in the evolution of building generative AI applications, and we are proud to offer Arize Copilot to teams that want to improve the development and performance of LLM systems,” said Aparna Dhinakaran, Chief Product Officer and Co-Founder of Arize. Other new workflows debuting today in the Arize platform promise to help engineers find issues with LLM apps once they are deployed. With AI search, for example, teams can select an example span and easily discover all similar issues (i.e. finding all data points where a customer is frustrated). Teams can then save those data points into a curated dataset to apply annotations, run evaluation experiments, or kick off fine-tuning workflows. Altogether, the updates make Arize a powerhouse for experimentation as well as production observability. Leveraging Arize, AI engineers can make adjustments – editing a prompt template, for example, or swapping out the LLM they are using – and then see if performance across a test dataset decreases or there are other impacts (i.e. around latency, retrieval, and hallucinations) before safely deploying a change into production. About Arize AI Arize AI is an AI observability and LLM evaluation platform that helps teams deliver and maintain more successful AI in production. Arize’s automated monitoring and observability platform allows teams to quickly detect issues when they emerge, troubleshoot why they happened, and improve overall performance across both traditional ML and generative use cases. Arize is headquartered in Berkeley, CA Subscribe to The Evaluator -------------------------- We’ll send you the latest news, expertise, and product updates from Arize. Your inbox is sacred, so we’ll only curate and send the best stuff. \*We’re committed to your privacy. Arize uses the information you provide to contact you about relevant content, products, and services. You may unsubscribe from these communications at any time. For more information, check out our [privacy policy](https://arize.com/privacy-policy) . --- # Arize AI Is Named An Emerging Leader In the Generative AI Engineering Gartner® Emerging Market Quadrant - Arize AI Arize AI Is Named An Emerging Leader In the Generative AI Engineering Gartner® Emerging Market Quadrant ======================================================================================================= ![](https://arize.com/wp-content/uploads/2022/03/Arize-logo-pink-black.png) The industry is at a crossroads. Despite the fact that a [majority](https://arize.com/blog/state-of-ai-engineering-survey/) of teams are planning to deploy generative AI,an [estimated](https://www.wsj.com/articles/companies-had-fun-experimenting-with-ai-now-they-have-to-show-the-returns-2a683592) 90% of projects are not out of the prototyping phase. For enterprise leaders who want to overcome these hurdles faster, rapidly assessing the right vendors is critical. Gartner’s recently-released “Innovation Guide for Generative AI Technologies” can help simplify that process, breaking down everything from market dynamics to piloting and evaluating vendors. The report also includes handy Emerging Market Quadrants to “represent the vendors’ capabilities in a dynamic and fast-moving market” across four categories: GenAI specialized cloud infrastructure, GenAI engineering, AI knowledge management apps, and GenAI model providers. Arize is featured as an “Emerging Leader” in the AI engineering submarket, which Gartner defines as including technologies providing “full-model life cycle management, specifically adjusted to and catering for developing, refining and deploying generative models…and other GenAI artifacts in production applications.” As the report notes: “Since generative models respond to prompting or seed conditions in different ways, it is important to monitor the interaction between the prompt and completion, observing how the model interprets the inbound prompt and how it responds to it.” Arize’s platform is relied on every day by enterprises to do that and more, from performing end-to-end tracing to evaluating and troubleshooting agents and AI applications. This latest update follows several other recognitions by Gartner. Previously, Arize was [listed](https://arize.com/blog/gartner-trism/) in the Gartner Market Guide for AI Trust, Risk, and Security Management (AI TRiSM) and was also [named](https://arize.com/blog/gartner-ai-operationalization-arize/) as a Cool Vendor in Gartner’s Enterprise AI Operationalization and Engineering Report. ![Generative AI engineering emerging leaders and challengers](https://arize.com/wp-content/uploads/2024/11/image1-3.png) Gartner clients can [download the full “Innovation Guide for Generative AI Technologies”here](https://www.gartner.com/document-reader/document/4584399?ref=solrAll&refval=432404969) . _Disclaimer: Gartner does not endorse any vendor, product or service depicted in its research publications, and does not advise technology users to select only those vendors with the highest ratings or other designation. Gartner research publications consist of the opinions of Gartner’s research organization and should not be construed as statements of fact. Gartner disclaims all warranties, expressed or implied, with respect to this research, including any warranties of merchantability or fitness for a particular purpose._ Subscribe to The Evaluator -------------------------- We’ll send you the latest news, expertise, and product updates from Arize. Your inbox is sacred, so we’ll only curate and send the best stuff. \*We’re committed to your privacy. Arize uses the information you provide to contact you about relevant content, products, and services. You may unsubscribe from these communications at any time. For more information, check out our [privacy policy](https://arize.com/privacy-policy) . --- # Arize Premieres Open Source LLM Evals Library and Support for Traces and Spans - Arize AI Arize Premieres Open Source LLM Evals Library and Support for Traces and Spans ============================================================================== ![](https://arize.com/wp-content/uploads/2023/04/logo-arize.svg) Popular open source tool Phoenix continues to expand what is possible in LLM evaluation, troubleshooting, and observability --------------------------------------------------------------------------------------------------------------------------- **Berkeley, CA, October 2, 2023** – [Arize Phoenix](https://phoenix.arize.com/) , a popular open-source library for visualizing datasets and troubleshooting large language model (LLM)-powered applications, rolled out several industry-first capabilities in its latest release. The update comes at a crossroads for generative AI, as new LLMOps tools race to keep up with the latest capabilities of foundation models. Over half (53.3%) of machine learning teams are planning production deployments of LLMs in the next year, but many continue to cite issues like hallucinations and responsible deployment as barriers in moving LLM-powered systems into the real world. While the rise of LlamaIndex and LangChain has enabled developers to accelerate the development of applications powered by LLMs, the abstractions created by these frameworks can also make them complicated to debug. Phoenix’s new support for LLM traces and spans means that AI engineers and developers can get visibility at a span-level and see exactly where an app breaks, with tools to analyze each step rather than just the end-result. This capability is particularly useful for early app developers because it doesn’t require them to send data to a SaaS platform to perform LLM evaluation and troubleshooting — instead, the open-source solution provides a mechanism for pre-deployment LLM observability directly from their local machine. Phoenix supports all common spans and has a native integration into LlamaIndex and LangChain. ![rag traces](https://arize.com/wp-content/uploads/2023/10/RAG_trace_details_PHX.png) The new [Phoenix LLM evals library](https://docs.arize.com/phoenix/concepts/llm-evals/) is also designed for fast and accurate LLM-assisted evaluations, ultimately making the use of the evaluation LLM easy to implement. Applying data science rigor to the testing of model and template combinations, Phoenix offers proven LLM evals for common use cases and needs around retrieval (RAG) relevance, reducing hallucinations, question-and-answer on retrieved data, toxicity, code generation, summarization, and classification. The Phoenix LLM evals library is optimized to run evaluations quickly with support for the notebook, Python pipeline, and app frameworks such as LangChain and LlamaIndex. “As LLM-powered applications increase in sophistication and new use cases emerge, deeper capabilities around LLM observability are needed to help debug and troubleshoot. We’re pleased to see this open-source solution from Arize, along with a one-click integration to LlamaIndex, and recommend any AI engineers or developers building with LlamaIndex check it out,” says Jerry Liu, CEO and Co-Founder of LlamaIndex. “Large language models are poised to transform industries and society, but when it comes to robust performance going from toy to production remains a challenge,” said Jason Lopatecki, CEO and Co-Founder of Arize AI. “These industry-first updates from Phoenix promise to provide better LLM evals and deeper troubleshooting to make complex LLM-powered systems ready and reliable in the real world.” **About Arize AI** Arize AI is a machine learning observability platform that helps ML teams deliver and maintain more successful AI in production. Arize’s automated model monitoring and observability platform allows ML teams to quickly detect issues when they emerge, troubleshoot why they happened, and improve overall model performance across both structured data and image and large language models. Arize is a remote-first company with headquarters in Berkeley, CA. Subscribe to The Evaluator -------------------------- We’ll send you the latest news, expertise, and product updates from Arize. Your inbox is sacred, so we’ll only curate and send the best stuff. \*We’re committed to your privacy. Arize uses the information you provide to contact you about relevant content, products, and services. You may unsubscribe from these communications at any time. For more information, check out our [privacy policy](https://arize.com/privacy-policy) . --- # Arize AI Acquires Velvet - Arize AI Arize AI Acquires Velvet ======================== ![](https://arize.com/wp-content/uploads/2023/04/logo-arize.svg) Addition to accelerate developer-first AI infrastructure -------------------------------------------------------- **Berkeley, CA – March 13, 2025** — Arize, a leader in AI observability and LLM evaluation, today announced the acquisition of Velvet, a developer-first AI gateway startup. Arize helps organizations ensure AI systems work reliably in production through advanced testing, model monitoring, evaluation frameworks, and debugging tools. The platform supports AI agents, voice assistants, and other sophisticated gen-AI applications as well as traditional ML. Velvet provides developers with lightweight tooling to analyze, evaluate, and monitor AI features. “We’re thrilled to welcome the Velvet team and work toward a shared vision around AI observability,” said Jason Lopatecki, CEO and Co-Founder of Arize AI. The Velvet team will focus on developer experience initiatives, helping to accelerate adoption of Arize’s unified AI platform, spanning: * Arize AX for Enterprise – AI evaluation and observability platform * Arize Phoenix OSS – Open-source AI observability with 2M+ monthly downloads * Arize AI Copilot – An AI-powered agent that helps engineers debug, optimize, and analyze models faster with 30+ skills. “Joining Arize lets us bring sophisticated AI infrastructure to more full-stack product developers,” said Emma Lawler, former CEO of Velvet. “Our shared vision is to make AI observability accessible while maintaining the enterprise-grade capabilities teams need as they scale.” Chris Hendel, Velvet’s co-founder and CTO, will lead platform engineering initiatives at Arize. Velvet’s existing customers will have the option to transition to Arize’s platform. The terms of the acquisition were not disclosed. **About Arize **Arize AI is a unified AI observability and LLM evaluation platform that helps teams develop and maintain more successful AI. Arize’s automated monitoring and observability platform allows teams to quickly detect issues when they emerge, troubleshoot why they happened, and improve overall performance across both traditional ML and generative use cases. Arize is headquartered in Berkeley, CA. **About Velvet **Velvet provides an AI gateway to help developers analyze, evaluate, and monitor AI features in production. The product enables teams to seamlessly observe LLM requests, analyze performance, and continuously test AI features with minimal setup overhead. Subscribe to The Evaluator -------------------------- We’ll send you the latest news, expertise, and product updates from Arize. Your inbox is sacred, so we’ll only curate and send the best stuff. \*We’re committed to your privacy. Arize uses the information you provide to contact you about relevant content, products, and services. You may unsubscribe from these communications at any time. For more information, check out our [privacy policy](https://arize.com/privacy-policy) . --- # Arize AI Secures $70M Series C to Fix AI’s Biggest Problem: Making LLMs and AI Agents Work in the Real World - Arize AI Arize AI Secures $70M Series C to Fix AI’s Biggest Problem: Making LLMs and AI Agents Work in the Real World ============================================================================================================ ![](https://arize.com/wp-content/uploads/2025/02/General-featured.jpg) Largest-ever investment in AI observability for development and production underscores the critical need for better testing, evaluation, and reliability of AI agents, voice assistants and other gen-AI applications --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- **Berkeley, CA – February 20, 2025** – Arize AI, a leader in AI observability and LLM evaluation, today announced a $70 million Series C to accelerate its mission of making AI work reliably in production. The round—the largest-ever investment in AI observability—was led by Adams Street Partners, with participation from M12 (Microsoft’s venture fund), Sinewave Ventures,OMERS Ventures, Datadog, PagerDuty, Industry Ventures, and Archerman Capital. Existing investors Foundation Capital, Battery Ventures, TCV, and Swift Ventures also reaffirmed their confidence in Arize’s vision. AI adoption is skyrocketing—business spending surpassed $13.8 billion in 2024, with 68% of enterprises planning to invest between $50 million and $250 million in generative AI in 2025. Yet, while AI models are more powerful than ever, most LLMs struggle to perform reliably in real-world applications like voice assistants. A growing number of cutting-edge AI models are trained and optimized using synthetic data—data generated by other AI models rather than real-world sources. But what happens when those models can’t accurately evaluate the results of their own synthetic data? In a research effort called OpenEvals, Arize has demonstrated that LLMs struggle to reliably assess correctness of synthetic datasets compared to non-synthetic data—a major blind spot as enterprises rush to scale generative AI. These findings highlight serious risks in AI model training and self-improvement loops, where unchecked errors in synthetic data can compound over time. For engineering teams, LLMs are still a black box—unpredictable, difficult to troubleshoot, and prone to failures that can derail entire projects. As the industry grapples with these challenges, AI engineers need better tools to ensure their models aren’t building on faulty foundations. With Arize’s AI observability and LLM evaluation platform, teams can test, troubleshoot, and course-correct AI systems before failures escalate into real-world consequences. This is especially important as enterprises race to implement semi-autonomous multi-agent systems, voice assistants, and increasingly sophisticated consumer-facing AI applications. “Building AI is easy. Making it work in the real world is the hard part,” said Jason Lopatecki, CEO and Co-Founder of Arize AI. “Enterprises can’t afford to deploy unreliable AI. Engineering teams need better infrastructure to test, evaluate, and troubleshoot their models before they impact customers. That’s exactly what Arize delivers—whether through our enterprise platform, Arize AX, or our open-source offering, Arize Phoenix.” “As AI research and real-world applications accelerate, Arize will continue to pioneer new tools, like our recent first-to-market launch of audio evaluation for voice assistants, to help engineers working on these systems better evaluation, debug, and improve what they build” added Aparna Dhinakaran, Chief Product Officer and Co-Founder of Arize. Since launching in 2020, Arize has become an AI observability and evaluation backbone for the world’s top enterprises and government agencies—including Booking.com, Condé Nast, Duolingo, Hyatt, PepsiCo, Priceline, TripAdvisor, Uber, and Wayfair, among hundreds more. Arize Phoenix, the company’s open-source offering, has emerged as the most widely adopted AI observability and evaluation library for development, with over two million monthly downloads. Arize’s partnership with Microsoft is also expanding, with M12’s investment reinforcing along-standing collaboration. The company recently launched deeper integrations with Azure AI Studio and the Azure AI Foundry portal, SDK, and CLI, making it easier than ever for AI engineers to integrate observability and evaluation into their workflows. “We believe AI observability is the missing piece in making AI truly enterprise-ready,” said Fred Wang, Partner at Adams Street Partners. “As AI adoption accelerates, companies need robust, cohesive tools to ensure their AI systems are performant, reliable, and aligned with business goals. Through our research and diligence in this market, we believe Arize AI has built the category-defining platform for AI observability and evaluation, trusted by leading enterprises and AI-first organizations. We’re excited to support their vision as they scale to meet the growing demand for production-grade AI.” “Arize AI’s innovative approach to AI observability and LLM evaluation is transforming the way enterprises deploy and manage AI systems. Our investment reflects our confidence in their ability to set new standards in the industry and empower AI engineers and developers to achieve real-world results,” said Todd Graham, Managing Partner at M12. “Tripadvisor’s billion-plus reviews and contributions are becoming even more important in a world of AI search and recommendations where travel experiences are more conversational, personal and even agentic. As we build out new AI products and capabilities, having the right infrastructure in place to evaluate and observe AI is important. Arize has been a valuable partner on that front,” said Rahul Todkar, Head of Data and AI at Tripadvisor. “With GenAI, we’re facilitating more tailored experiences that adapt and respond to travelers’ needs faster than ever before. As we continue to innovate, our technical teams blend an approach of pioneering new tools in-house and using platforms like Arize to help in testing, evaluating and tracing new AI-powered applications and workflows,” said Jeroen Hofman, ML Engineering Manager at Booking. “Arize AI deserves a lot of credit for pioneering AI observability and creating a de facto standard for enterprises that want to achieve real-world results with generative AI,” said Brett Wilson, General Partner at Swift Ventures.“ We’re proud to continue to back the company as it scales.” **About Arize** Arize AI is a unified AI observability and LLM evaluation platform that helps teams develop and maintain more successful AI. Arize’s automated monitoring and observability platform allows teams to quickly detect issues when they emerge, troubleshoot why they happened, and improve overall performance across both traditional ML and generative use cases. Arize is headquartered in Berkeley, CA. Subscribe to The Evaluator -------------------------- We’ll send you the latest news, expertise, and product updates from Arize. Your inbox is sacred, so we’ll only curate and send the best stuff. \*We’re committed to your privacy. Arize uses the information you provide to contact you about relevant content, products, and services. You may unsubscribe from these communications at any time. For more information, check out our [privacy policy](https://arize.com/privacy-policy) . --- # Arize AI’s AI Engineering Platform for R&D Selected by AFWERX - Arize AI Arize AI’s AI Engineering Platform for R&D Selected by AFWERX ============================================================= ![](https://arize.com/wp-content/uploads/2025/08/arize-ai-department-air-force-afwerx-afrl.png) Arize’s AI and agent engineering tools will be adapted to the Department of the Air Force’s secure NIPRNet environment ---------------------------------------------------------------------------------------------------------------------- (Berkeley, Calif.) – Arize AI announces it has been selected by AFWERX for a Direct-to-Phase II Small Business Innovation Research (SBIR) contract in the amount of approximately $1.2 million per year to conduct a 12‑month R&D focused effort on AI engineering capabilities for NIPRGPT / GCP AI enhancement to address the most pressing challenges in the Department of the Air Force (DAF). The Air Force Research Laboratory and AFWERX have partnered to streamline the Small Business Innovation Research (SBIR) and Small Business Technology Transfer (STTR) process by accelerating the small business experience through faster proposal to award timelines, changing the pool of potential applicants by expanding opportunities to small business and eliminating bureaucratic overhead by continually implementing process improvement changes in contract execution. The DAF began offering the Open Topic SBIR/STTR program in 2018 which expanded the range of innovations the DAF funded and now effective July 22, 2025 Arize AI starts its journey to create and provide an R&D AI Engineering Platform that will strengthen the national defense of the United States of America by accelerating the deployment of powerful generative AI applications such as agentic AI and RAG based use cases. > **Arize AI is excited to support DAF in maintaining and accelerating global AI dominance and innovation for the American warfighter.** — _Jason Lopatecki, co‑founder & CEO, Arize AI_ **NIPRGPT** is a prototype large language model (LLM) authorized at Impact Level 5 (IL5), developed by the Air Force Research Laboratory for the Department of the Air Force’s Controlled Unclassified Information (CUI) workloads. It offers a ChatGPT-like experience with the ability to maintain chat history, enabling critical assessments of the feasibility and security of deploying Generative AI on the Non-classified Internet Protocol Router Network (NIPRNet). NIPRGPT provides valuable insights into technical challenges, security implications, and operational benefits, assisting the Department of the Air Force in the secure and ethical adoption of commercial AI technologies. The Arize AX R&D effort promises to help NIPRGPT by automating prompt engineering and online evaluation within CAC/NIPRNet constraints with metrics for quality, safety, efficiency, and compliance so leaders can see what works in practice. Insights from user feedback and key operational telemetry metrics will flow back to inform future policy, acquisition, and investment decisions across the DAF. **The views expressed are those of the author and do not necessarily reflect the official policy or position of the Department of the Air Force, the Department of Defense, or the U.S. government.** ### **About Arize AI** Arize AI is an AI and agent engineering platform. Arize’s tools allow teams to quickly detect issues when they emerge, troubleshoot why they happened, and improve overall performance across both traditional ML and AI agent engineering. Arize is headquartered in Berkeley, CA ### **About AFRL**  
The Air Force Research Laboratory, or AFRL, is the primary scientific research and development center for the Department of the Air Force. AFRL plays an integral role in leading the discovery, development and integration of affordable warfighting technologies for our air, space and cyberspace forces. With a workforce spanning across nine technology areas and 40 other operations around the globe, AFRL provides a diverse portfolio of science and technology ranging from fundamental to advanced research and technology development. For more information, visit [afresearchlab.com](http://afresearchlab.com/) .   ### **About AFWERX** As the innovation arm of the DAF and a directorate within the Air Force Research Laboratory, AFWERX brings cutting-edge American ingenuity from small businesses and start-ups to address the most pressing challenges of the DAF. AFWERX employs approximately 370 military, civilian and contractor personnel at four hubs and sites executing an annual $1.4 billion budget. Since 2019, AFWERX has awarded over 10,400 contracts worth more than $7.24 billion to strengthen the U.S. defense industrial base and drive faster technology transition to operational capability. For more information, visit: [afwerx.com](https://afwerx.com/) .   **Contract No.:** FA864925P0276 **Award / Selection Date:** July 22, 2025 **Period of Performance:** July 22, 2026 **TPOC:** William Jinkins Subscribe to The Evaluator -------------------------- We’ll send you the latest news, expertise, and product updates from Arize. Your inbox is sacred, so we’ll only curate and send the best stuff. \*We’re committed to your privacy. Arize uses the information you provide to contact you about relevant content, products, and services. You may unsubscribe from these communications at any time. For more information, check out our [privacy policy](https://arize.com/privacy-policy) . --- # Arize AI and Tokyo Electron Device Announce Partnership - Arize AI Arize AI and Tokyo Electron Device Announce Partnership ======================================================= ![](https://arize.com/wp-content/uploads/2025/08/arize-tokyo-electron-device.png) Tokyo Electron Device Limited to roll out the Arize AX agent engineering platform in Japan ------------------------------------------------------------------------------------------ _Tokyo, Japan_  – August 27, 2025 —Tokyo Electron Device Limited, (headquartered in Shibuya ward, Tokyo; President & CEO: Atsuyuki Tokushige, hereinafter referred to as TED) has entered into a sales agency agreement with Arize AI, Inc., (headquartered in Berkeley, California, USA; Co-Founder & CEO: Jason Lopatecki, hereinafter referred to as Arize) and will begin selling the AI agent and engineering platform as the first partner in Japan starting on August 26, 2025. “The rise of AI agents is creating new demands on engineering teams to iterate quickly and keep systems reliable at scale. With Tokyo Electron Device as our inaugural partner in Japan, we’re combining their deep enterprise reach with Arize AX to help AI engineers accelerate time-to-value as they build, evaluate, and operate AI systems end-to-end.” — Jason Lopatecki, CEO and Co-Founder of Arize AI.   Recently, the development and utilization of unique services leveraging generative AI have become more active in Japan, resulting in an increase in companies advancing the development of businesses and applications through the operation of LLMs and AI agents. However, achieving reliable performance from AI systems is not an easy task, and addressing challenges such as hallucinations is a significant concern. Realizing improvements requires a tremendous amount of effort and time from engineers. Arize AX helps address these challenges. Arize AX offers a single platform to help enterprises accelerate development of AI apps and agents – then perfect them in production. Featuring an embedded AI copilot with agent mode, AX offers tools across the full lifecycle: * Development tools to build high-quality agents and AI apps * Evaluation that powers reliable, production-ready AI applications and agents * Observability to debug, trace, and improve  TED supports implementation, construction, and verification assistance, as well as providing help desk services. URL: [http://cn.teldevice.co.jp/support/detail/supportservice/ss](http://cn.teldevice.co.jp/support/detail/supportservice/ss) Through the provision of the AI observability platform “Arize AX,” TED strongly supports quality management and operational efficiency of AI applications, creating an environment where AI can be utilized with confidence. ![](https://arize.com/wp-content/uploads/2025/08/arize-lifecycle.png) **About Arize AI, Inc.** Arize AI is an AI and agent engineering platform. Arize’s tools allow teams to quickly detect issues when they emerge, troubleshoot why they happened, and improve overall performance across both traditional ML and AI agent engineering. Arize is headquartered in Berkeley, CA. **About Tokyo Electron Device Limited** Tokyo Electron Device aims to resolve potential social challenges through the power of manufacturers and technology trading companies, promoting the social implementation of cutting-edge technologies centered around semiconductors and IT. Through the identification of advanced products and services cultivated as a technology trading company and the development of innovative solutions by strengthening manufacturing functions, it contributes to the realization of a super-smart society and sustainable development. URL: [http://www.teldevice.co.jp/](http://www.teldevice.co.jp/) **For Press Inquiries Regarding This Matter** Tokyo Electron Device Limited, Marketing Communication Department, Public Relations Group Inquiry form: [https://www.teldevice.co.jp/cgi-bin/form/contact.php](https://www.teldevice.co.jp/cgi-bin/form/contact.php) _Company names and product names mentioned in this news release are registered trademarks or trademarks of their respective companies._ Subscribe to The Evaluator -------------------------- We’ll send you the latest news, expertise, and product updates from Arize. Your inbox is sacred, so we’ll only curate and send the best stuff. \*We’re committed to your privacy. Arize uses the information you provide to contact you about relevant content, products, and services. You may unsubscribe from these communications at any time. For more information, check out our [privacy policy](https://arize.com/privacy-policy) . --- # Arize AI Makes Fast Company’s Fifth Annual List of the Best Workplaces for Innovators In the Enterprise Category - Arize AI Arize AI Makes Fast Company’s Fifth Annual List of the Best Workplaces for Innovators In the Enterprise Category ================================================================================================================ ![](https://arize.com/wp-content/uploads/2022/08/fastco-logo.jpg) Berkeley, CA, July 11, 2023 — Fast Company today announced its fifth annual Best Workplaces for Innovators list, honoring organizations and businesses that demonstrate an inspiring commitment to encourage and develop innovation at all levels. Arize AI is recognized alongside Asana and Merck in the _Enterprise Products and Services_ category for “creating a strong culture of innovation.” ![](https://arize.com/wp-content/uploads/2023/07/fastco-innovator-back-to-back-fastcompany.jpg) Developed in collaboration with Accenture, the 2023 Best Workplaces for Innovators ranks companies from a variety of industries, including entertainment, biotech, consumer packaged goods, marketing, education, healthcare, and many more. Fast Company editors and Accenture researchers collaborated together to score nearly 1,000 submissions, and a panel of eight distinguished judges reviewed and endorsed the top companies. The 2023 awards feature workplaces from around the world. “Innovation is a global priority, and this year’s list has a decidedly international flavor,” says Brendan Vaughan, editor-in-chief of Fast Company. “Five of the top 10 ranked companies, including No. 1, Canva, are not headquartered in the U.S.” **About Fast Company** Fast Company is the only media brand fully dedicated to the vital intersection of business, innovation, and design, engaging the most influential leaders, companies, and thinkers on the future of business. Headquartered in New York City, Fast Company is published by Mansueto Ventures LLC, along with our sister publication Inc., and can be found online at www.fastcompany.com. **About Arize AI** Arize AI is a machine learning observability platform that helps ML teams deliver and maintain more successful AI in production. Arize’s automated model monitoring and observability platform allows ML teams to quickly detect issues when they emerge, troubleshoot why they happened, and improve overall model performance across both structured data and image and large language models. Arize is a remote-first company with headquarters in Berkeley, CA. Subscribe to The Evaluator -------------------------- We’ll send you the latest news, expertise, and product updates from Arize. Your inbox is sacred, so we’ll only curate and send the best stuff. \*We’re committed to your privacy. Arize uses the information you provide to contact you about relevant content, products, and services. You may unsubscribe from these communications at any time. For more information, check out our [privacy policy](https://arize.com/privacy-policy) . --- # Arize Platform Demo 2024 - Arize AI Arize Platform Demo 2024 ======================== Arize AI Platform Demo ---------------------- Subscribe to The Evaluator -------------------------- We’ll send you the latest news, expertise, and product updates from Arize. Your inbox is sacred, so we’ll only curate and send the best stuff. \*We’re committed to your privacy. Arize uses the information you provide to contact you about relevant content, products, and services. You may unsubscribe from these communications at any time. For more information, check out our [privacy policy](https://arize.com/privacy-policy) . --- # Arize Holiday Special - Arize AI ![](https://arize.com/wp-content/uploads/2023/11/holiday-banner-bg.jpg) #### ARIZE HOLIDAY SPECIAL ![](https://arize.com/wp-content/uploads/2023/11/holiday-hero-title.png) ======================================================================== December 15, 2023 Let’s get into the LLM spirit together! Join us for live, virtual sessions focused on prompt engineering, search and retrieval workflows, and LLM system evaluations. Each session is designed to be a hands-on workshop where participants will apply technical skills to construct evaluation approaches for Retrieval-Augmented Generation (RAG) systems and generative AI language models. ![](https://arize.com/wp-content/uploads/2023/12/holiday-logo1.png)   ![](https://arize.com/wp-content/uploads/2023/12/holiday-logo2.png)   ![](https://arize.com/wp-content/uploads/2023/12/holiday-logo3.png) [Save your seat](https://events.zoom.us/ev/AsAS8DWlSExvMrHCMR8-QwCWTfu6-A6ynvhdamDK0X23_TychKib~Ao71Yly3iNhqlyvpPtSGs2n92yLXLyOYjt4sR_04fqju5RRpTrZv-q7WLQ) ![](https://arize.com/wp-content/uploads/2023/11/holiday-banner-right.png) Topics include -------------- From RAGtag to RAGing: Evaluating Search and Retrieval Use-cases with Phoenix Tracing This tutorial will look into building a RAG pipeline and evaluating it with Phoenix Evals. It will include: Understanding Retrieval Augmented Generation (RAG), Building RAG (with the help of a framework such as LlamaIndex), and Evaluating RAG with Phoenix Evals. Constructing an Evaluation Approach for Generative AI Models As Large Language Models (LLMs) revolutionize data science with generative use cases, their real-world application challenges traditional evaluation methods built for discriminative use cases. Building your own RAGs with LangChain A practical guide to constructing a Retrieval-Augmented Generation (RAG) model using the LangChain framework. We’ll cover the essentials of RAG, its integration with LLMs, and the unique advantages it offers in natural language processing. Vibe-Based Prompt Engineering Including insights from PromptLayer’s collaboration with top-tier teams, this talk highlights the need for iteration over “silver-bullet” MLOps-style evaluations. “Vibe-based” evaluation is the scientific process, try a prompt and check the output. Speakers -------- ![](https://arize.com/wp-content/uploads/2023/11/Amber-scanline.jpg) ### Amber Roberts #### ML Growth Lead, Arize AI ![](https://arize.com/wp-content/uploads/2023/11/Madhav-scanline.jpg) ### Madhav Thaker #### Senior Data Scientist, Shopify ![](https://arize.com/wp-content/uploads/2023/11/Jared-scanline.jpg) ### Jared Zoneraich #### Founder, PromptLayer ![](https://arize.com/wp-content/uploads/2023/11/Rajiv-scanline.jpg) ### Rajiv Shah #### Machine Learning Engineer, HuggingFace Registration is open -------------------- [Save your seat](https://events.zoom.us/ev/AsAS8DWlSExvMrHCMR8-QwCWTfu6-A6ynvhdamDK0X23_TychKib~Ao71Yly3iNhqlyvpPtSGs2n92yLXLyOYjt4sR_04fqju5RRpTrZv-q7WLQ) Subscribe to The Evaluator -------------------------- We’ll send you the latest news, expertise, and product updates from Arize. Your inbox is sacred, so we’ll only curate and send the best stuff. \*We’re committed to your privacy. Arize uses the information you provide to contact you about relevant content, products, and services. You may unsubscribe from these communications at any time. For more information, check out our [privacy policy](https://arize.com/privacy-policy) . --- # Arize:Observe - Tecton Workshop - Building Production Ready batch, Streaming, and... in Jupyter Notebook - Arize AI ### Arize:Observe 2023 Arize:Observe – Tecton Workshop – Building Production Ready batch, Streaming, and… in Jupyter Notebook ====================================================================================================== Subscribe to The Evaluator -------------------------- We’ll send you the latest news, expertise, and product updates from Arize. Your inbox is sacred, so we’ll only curate and send the best stuff. \*We’re committed to your privacy. Arize uses the information you provide to contact you about relevant content, products, and services. You may unsubscribe from these communications at any time. For more information, check out our [privacy policy](https://arize.com/privacy-policy) . --- # Arize:Observe - Introducing Phoenix - ML Observability in Your Notebook - Arize AI ### Arize:Observe 2023 Arize:Observe – Introducing Phoenix – ML Observability in Your Notebook ======================================================================= Subscribe to The Evaluator -------------------------- We’ll send you the latest news, expertise, and product updates from Arize. Your inbox is sacred, so we’ll only curate and send the best stuff. \*We’re committed to your privacy. Arize uses the information you provide to contact you about relevant content, products, and services. You may unsubscribe from these communications at any time. For more information, check out our [privacy policy](https://arize.com/privacy-policy) . --- # Arize Phoenix OSS - ML Observability in a Notebook - Arize AI ### Arize:Observe 2023 Arize Phoenix OSS – ML Observability in a Notebook ================================================== Subscribe to The Evaluator -------------------------- We’ll send you the latest news, expertise, and product updates from Arize. Your inbox is sacred, so we’ll only curate and send the best stuff. \*We’re committed to your privacy. Arize uses the information you provide to contact you about relevant content, products, and services. You may unsubscribe from these communications at any time. For more information, check out our [privacy policy](https://arize.com/privacy-policy) . --- # Arize:Observe - Are You Flying Blind With Your Chatbots - Arize AI Arize:Observe – Are You Flying Blind With Your Chatbots ======================================================= Subscribe to The Evaluator -------------------------- We’ll send you the latest news, expertise, and product updates from Arize. Your inbox is sacred, so we’ll only curate and send the best stuff. \*We’re committed to your privacy. Arize uses the information you provide to contact you about relevant content, products, and services. You may unsubscribe from these communications at any time. For more information, check out our [privacy policy](https://arize.com/privacy-policy) . --- # Prost to improving Model Performance - Arize AI ### Webinar Prost ===== ![](https://arize.com/wp-content/uploads/2022/09/H2-Prost_to_improving_model_performance.svg) ### →  October 20th, 3pm PST | Virtual Experience Do not operate heavy machinery or deploy new models into production during this fun Oktoberfest virtual tasting and beer-themed dive into the world of monitoring unstructured data. While drinking beer, we’ll be exploring two common problems and how beer and model monitoring can help solve them: the difficulty in figuring out what to label next for CV and NLP models and the lack of visibility into when models encounter new patterns they did not see in training. You’ll want to stay for the full presentation because at the end there will be a data labeling contest for who can accurately identify the most beers! The tasting box will arrive at your home by October 20th. Once you get it, hold off on cracking the bottles open! (We know this part is tricky!) On **Thursday, October 20th, 2022 at 3:00pm PT | 5:00 CT | 6:00 ET**, get your glasses and your favorite people (age 21+) and join us for a fun, informative get-together. _Since we need 2 weeks to ship the kit out, please register by October 6th and block your calendar_. We will send you an invitation with more details once your spot is confirmed. ### Registration is Now Closed Subscribe to The Evaluator -------------------------- We’ll send you the latest news, expertise, and product updates from Arize. Your inbox is sacred, so we’ll only curate and send the best stuff. \*We’re committed to your privacy. Arize uses the information you provide to contact you about relevant content, products, and services. You may unsubscribe from these communications at any time. For more information, check out our [privacy policy](https://arize.com/privacy-policy) . --- # Productionizing Machine Learning with Observability, Quality and Flexibility at Scale - Arize AI ### Webinar Productionizing Machine Learning with Observability, Quality and Flexibility at Scale ===================================================================================== ### Subscribe to our resources and blogs [Subscribe](https://arize.com/resource/arize-anyscale-webinar/thank-you/#blog-subscribe-modal) Subscribe to The Evaluator -------------------------- We’ll send you the latest news, expertise, and product updates from Arize. Your inbox is sacred, so we’ll only curate and send the best stuff. \*We’re committed to your privacy. Arize uses the information you provide to contact you about relevant content, products, and services. You may unsubscribe from these communications at any time. For more information, check out our [privacy policy](https://arize.com/privacy-policy) . --- # Arize: Observe Unstructured 2022 - Arize AI ![AO Unstructured logo](https://arize.com/wp-content/uploads/2022/06/AO-Unstructured-logo.png "AO Unstructured logo") From images and video to natural language and audio, unstructured data coupled with machine learning can unlock deeper AI potential and ROI for many organizations and use cases. Embeddings are the core of how deep learning models represent structures and are fundamental to how the next generation of ML models work. Join us at Arize:Observe Unstructured to learn about emerging techniques like UMAP to transform unstructured data into embeddings that can be more efficiently processed by ML models, hear from leading-edge ML platform teams implementing new technologies to monitor and improve models in production, and try out these approaches and tools in a hands-on workshop. ![](https://arize.com/wp-content/uploads/2022/02/Group_2315-1.png "Group_2315 (1)") View Sessions On-Demand ----------------------- ![Arize Keynote](https://arize.com/wp-content/uploads/2022/06/Arize-Keynote-thumbnail.png) #### Arize:Observe Unstructured keynote Jason Lopatecki, Founder and CEO, Arize AI Aparna Dhinakaran, Co-Founder and CPO, Arize AI [Play the video →](https://arize.com/resource/arizeobserve-unstructured-keynote-presentation/) ![Hugging Face](https://arize.com/wp-content/uploads/2022/06/Hugging-Face-thumbnail.png) #### Accelerating Machine Learning from Research to Production with Hugging Face Jeff Boudier, Product Director, Hugging Face Francisco Castillo, Software Engineer, Arize AI [Play the video →](https://arize.com/resource/arizeobserve-unstructured-accelerating-machine-learning-from-research-to-production-with-hugging-face/) ![Workshop](https://arize.com/wp-content/uploads/2022/06/Workshop-thumbnail.png) #### Workshop: Monitor & Troubleshoot Embeddings Amber Roberts, Machine Learning Engineer, Arize AI [Video coming soon →](https://arize.com/arize-observe-unstructured-2022/#) ![Pachyderm](https://arize.com/wp-content/uploads/2022/06/Pachyderm-thumbnail.png) #### Handling the Challenges of Unstructured Data, The Unsung Hero of Machine Learning Dan Jeffries, Chief Technical Evangelist, Pachyderm / Managing Director, AIIA [Play the video →](https://arize.com/resource/arizeobserve-unstructured-handling-the-challenges-of-unstructured-data-the-unsung-hero-of-machine-learning/) ![UMAP](https://arize.com/wp-content/uploads/2022/06/UMAP-thumbnail.png) #### A Theory Primer for UMAP: Uniform Manifold Approximation and Projection Leland McInnes, Founder, UMAP [Play the video →](https://arize.com/resource/arizeobserve-unstructured-a-theory-primer-for-umap-uniform-manifold-approximation-and-projection/) ![Labelbox](https://arize.com/wp-content/uploads/2022/06/Labelbox-thumbnail.png) #### How to improve performance of unstructured models with less data Maxime Voisin, Head of Catalog and Models, Labelbox Claire Longo, Customer Success Lead, Arize AI [Play the video →](https://arize.com/resource/arizeobserve-unstructured-how-to-improve-performance-of-unstructured-models-with-less-data/) Featured Speakers ----------------- ![](https://arize.com/wp-content/uploads/2022/06/Peter_Welinder.301.png "Peter_Welinder.301") ###### Peter Welinder VP of Product & Partnerships, OpenAI ![Jeff_Boudier](https://arize.com/wp-content/uploads/2022/06/Jeff_Boudier.301.png "Jeff_Boudier.301") ###### Jeff Boudier Product Director, Hugging Face ![Leiland](https://arize.com/wp-content/uploads/2022/06/Leiland-website.301.png "Leiland-website.301") ###### Leland McInnes Creator of UMAP ![Dan](https://arize.com/wp-content/uploads/2022/06/Dan.301.png "Dan.301") ###### Dan Jeffries Chief Technical Evangelist, Pachyderm / Managing Director, AIIA ![Maxime Voisin](https://arize.com/wp-content/uploads/2022/06/Maxime-Voisin.301.png "Maxime Voisin.301") ###### Maxime Voisin Head of Catalog and Models, Labelbox ![](https://arize.com/wp-content/themes/arize-2022/images/bg-bottom.png) Subscribe to The Evaluator -------------------------- We’ll send you the latest news, expertise, and product updates from Arize. Your inbox is sacred, so we’ll only curate and send the best stuff. \*We’re committed to your privacy. Arize uses the information you provide to contact you about relevant content, products, and services. You may unsubscribe from these communications at any time. For more information, check out our [privacy policy](https://arize.com/privacy-policy) . --- # Arize:Observe Unstructured - How to improve performance of unstructured models with less data - Arize AI ### Videos Arize:Observe Unstructured – How to improve performance of unstructured models with less data ============================================================================================= Arize:Observe Unstructured - How to improve performance of unstructured models with less data --------------------------------------------------------------------------------------------- Join our talk with Maxime Voisin, Product at Labelbox, and Claire Longo, Customer Success at Arize, to hear how ML teams can improve the performance of their unstructured models in training and production with less data. Today, leading ML teams extend their focus to carefully selecting their training data, training the model, examining its performance, and modifying the next training dataset accordingly. Learn about best practices when moving unstructured data through R&D, training, and all the way to production, and how to apply these techniques in your own organization. [![](https://arize.com/wp-content/uploads/2022/08/Arize-Observe-Unstructured.png "Arize-Observe Unstructured")\ \ #### Arize:Observe Unstructured – Keynote presentation\ \ By Joel Bowman |](https://arize.com/resource/arizeobserve-unstructured-keynote-presentation/) [![Dan Jeffries](https://arize.com/wp-content/uploads/2022/08/Dan.png "Dan Jeffries")\ \ #### Arize:Observe Unstructured – Handling the Challenges of Unstructured Data, The Unsung Hero of Machine Learning\ \ By Joel Bowman |](https://arize.com/resource/arizeobserve-unstructured-handling-the-challenges-of-unstructured-data-the-unsung-hero-of-machine-learning/) [![Jeff Boudier Hugging Face](https://arize.com/wp-content/uploads/2022/06/Screen-Shot-2022-08-22-at-4.14.33-PM.png "Jeff Boudier")\ \ #### Arize:Observe Unstructured – Accelerating Machine Learning from Research to Production with Hugging Face\ \ By Joel Bowman |](https://arize.com/resource/arizeobserve-unstructured-accelerating-machine-learning-from-research-to-production-with-hugging-face/) [![Leland](https://arize.com/wp-content/uploads/2022/08/Leland.png "Leland")\ \ #### Arize:Observe Unstructured – A Theory Primer for UMAP: Uniform Manifold Approximation and Projection\ \ By Joel Bowman |](https://arize.com/resource/arizeobserve-unstructured-a-theory-primer-for-umap-uniform-manifold-approximation-and-projection/) Subscribe to The Evaluator -------------------------- We’ll send you the latest news, expertise, and product updates from Arize. Your inbox is sacred, so we’ll only curate and send the best stuff. \*We’re committed to your privacy. Arize uses the information you provide to contact you about relevant content, products, and services. You may unsubscribe from these communications at any time. For more information, check out our [privacy policy](https://arize.com/privacy-policy) . --- # Productionizing Machine Learning with Observability, Quality and Flexibility at Scale - Arize AI ### Webinar Productionizing Machine Learning with Observability, Quality and Flexibility at Scale ===================================================================================== **![](https://arize.com/wp-content/uploads/2022/08/Calendar.svg) On-Demand**   **![](https://arize.com/wp-content/uploads/2022/08/Duration.svg)  30 Minutes** See how **Ray** and **Arize** combine to provide highly scalable and easily managed ML deployments, with automatic issue detection and quick troubleshooting.  Don’t miss this live webinar on Feb, 7, at 10 am PT hosted by **Anyscale,** the company behind **Ray**, **the unified framework for scalable computin****g**, and **Arize****, the leader in machine learning observability**. This interactive webinar discusses how Ray and Arize combine to provide ease of AI/ML development and observability along with the ability to understand performance, data quality and drift issues. Hear how leading AI teams: * **Bridge the gap between development and production** * **Scale across multiple dimensions** * **Increase developer velocity and speed experimentation**  * **Understand model drift** * **Automate monitoring at scale** * **Find and fix problems faster** ![Arize + Anyscale](https://arize.com/wp-content/uploads/2023/01/Resources_1920x1080-1024x576.png) ### Watch Recording Speakers -------- ![](https://arize.com/wp-content/uploads/2022/08/dat-ngo-arize-.jpg) ##### Dat Ngo ###### ML Solutions Architect, Arize AI ![](https://arize.com/wp-content/uploads/2023/01/Phi_Nguyen-1.png) ##### Phi Nguyen ###### GTM Tech Lead, Anyscale Subscribe to The Evaluator -------------------------- We’ll send you the latest news, expertise, and product updates from Arize. Your inbox is sacred, so we’ll only curate and send the best stuff. \*We’re committed to your privacy. Arize uses the information you provide to contact you about relevant content, products, and services. You may unsubscribe from these communications at any time. For more information, check out our [privacy policy](https://arize.com/privacy-policy) . --- # Arize:Observe Unstructured - A Theory Primer for UMAP: Uniform Manifold Approximation and Projection - Arize AI ### Videos Arize:Observe Unstructured – A Theory Primer for UMAP: Uniform Manifold Approximation and Projection ==================================================================================================== Arize:Observe Unstructured - A Theory Primer for UMAP: Uniform Manifold Approximation and Projection ---------------------------------------------------------------------------------------------------- Join our talk with Leland McInnes, the creator of UMAP, as he walks through the theory and mathematics behind UMAP. UMAP visualizations of embeddings can be used in practice to troubleshoot high dimensional data in a low dimensional space. Embeddings are vector (mathematical) representations of data where linear distances capture structure in the original datasets, and are proliferating in modern ML systems. This talk will cover the evolution of UMAP, and how UMAP can be used in practice to troubleshoot high dimensional data. [![](https://arize.com/wp-content/uploads/2022/08/Arize-Observe-Unstructured.png "Arize-Observe Unstructured")\ \ #### Arize:Observe Unstructured – Keynote presentation\ \ By Joel Bowman |](https://arize.com/resource/arizeobserve-unstructured-keynote-presentation/) [![Arize:Observe Unstructured featured image](https://arize.com/wp-content/uploads/2022/06/Featured-image-video-resources.jpg "Arize:Observe Unstructured featured image")\ \ #### Arize:Observe Unstructured – Powering the Next Generation of Products with AI\ \ By Joel Bowman |](https://arize.com/resource/arizeobserve-unstructured-powering-the-next-generation-of-products-with-ai/) [![Jeff Boudier Hugging Face](https://arize.com/wp-content/uploads/2022/06/Screen-Shot-2022-08-22-at-4.14.33-PM.png "Jeff Boudier")\ \ #### Arize:Observe Unstructured – Accelerating Machine Learning from Research to Production with Hugging Face\ \ By Joel Bowman |](https://arize.com/resource/arizeobserve-unstructured-accelerating-machine-learning-from-research-to-production-with-hugging-face/) Subscribe to The Evaluator -------------------------- We’ll send you the latest news, expertise, and product updates from Arize. Your inbox is sacred, so we’ll only curate and send the best stuff. \*We’re committed to your privacy. Arize uses the information you provide to contact you about relevant content, products, and services. You may unsubscribe from these communications at any time. For more information, check out our [privacy policy](https://arize.com/privacy-policy) . --- # Arize:Observe Unstructured - Accelerating Machine Learning from Research to Production with Hugging Face - Arize AI ### Videos Arize:Observe Unstructured – Accelerating Machine Learning from Research to Production with Hugging Face ======================================================================================================== Accelerating Machine Learning from Research to Production with Hugging Face --------------------------------------------------------------------------- Join our talk with Jeff Boudier, Product Director at Hugging Face to learn how Transformers can help ML teams get state of the art models up in production faster than ever. Hear more about the latest trends and use cases in Machine Learning. [![](https://arize.com/wp-content/uploads/2022/08/Arize-Observe-Unstructured.png "Arize-Observe Unstructured")\ \ #### Arize:Observe Unstructured – Keynote presentation\ \ By Joel Bowman |](https://arize.com/resource/arizeobserve-unstructured-keynote-presentation/) [![Arize:Observe Unstructured featured image](https://arize.com/wp-content/uploads/2022/06/Featured-image-video-resources.jpg "Arize:Observe Unstructured featured image")\ \ #### Arize: Observe Unstructured 2022\ \ By Joel Bowman |](https://arize.com/arize-observe-unstructured-2022/) [![Arize:Observe Unstructured featured image](https://arize.com/wp-content/uploads/2022/06/Featured-image-video-resources.jpg "Arize:Observe Unstructured featured image")\ \ #### Arize:Observe Unstructured – Powering the Next Generation of Products with AI\ \ By Joel Bowman |](https://arize.com/resource/arizeobserve-unstructured-powering-the-next-generation-of-products-with-ai/) Subscribe to The Evaluator -------------------------- We’ll send you the latest news, expertise, and product updates from Arize. Your inbox is sacred, so we’ll only curate and send the best stuff. \*We’re committed to your privacy. Arize uses the information you provide to contact you about relevant content, products, and services. You may unsubscribe from these communications at any time. For more information, check out our [privacy policy](https://arize.com/privacy-policy) . --- # Arize:Observe Unstructured - Keynote presentation - Arize AI ### Videos Arize:Observe Unstructured – Keynote presentation ================================================= Join Jason Lopatecki, Co-founder and CEO and Aparna Dhinakaran, Co-founder and CPO of Arize AI as they kick off Arize: Observe Unstructured 2022. [![Arize:Observe Unstructured featured image](https://arize.com/wp-content/uploads/2022/06/Featured-image-video-resources.jpg "Arize:Observe Unstructured featured image")\ \ #### Arize: Observe Unstructured 2022\ \ By Joel Bowman |](https://arize.com/arize-observe-unstructured-2022/) [![Arize:Observe Unstructured featured image](https://arize.com/wp-content/uploads/2022/06/Featured-image-video-resources.jpg "Arize:Observe Unstructured featured image")\ \ #### Arize:Observe Unstructured – Powering the Next Generation of Products with AI\ \ By Joel Bowman |](https://arize.com/resource/arizeobserve-unstructured-powering-the-next-generation-of-products-with-ai/) [![Leland](https://arize.com/wp-content/uploads/2022/08/Leland.png "Leland")\ \ #### Arize:Observe Unstructured – A Theory Primer for UMAP: Uniform Manifold Approximation and Projection\ \ By Joel Bowman |](https://arize.com/resource/arizeobserve-unstructured-a-theory-primer-for-umap-uniform-manifold-approximation-and-projection/) Subscribe to The Evaluator -------------------------- We’ll send you the latest news, expertise, and product updates from Arize. Your inbox is sacred, so we’ll only curate and send the best stuff. \*We’re committed to your privacy. Arize uses the information you provide to contact you about relevant content, products, and services. You may unsubscribe from these communications at any time. For more information, check out our [privacy policy](https://arize.com/privacy-policy) . --- # Arize:Observe Unstructured - Powering the Next Generation of Products with AI - Arize AI Arize:Observe Unstructured – Powering the Next Generation of Products with AI ============================================================================= Powering the Next Generation of Products with AI ------------------------------------------------ With AI systems like GPT-3, Codex and DALL-E, OpenAI provides the AI building blocks to power the next generation of products. Given a simple text-based instruction in natural language, GPT-3 and Codex returns a text- or code completion. Given a text-based prompt, DALL-E draws photorealistic images or art. Together, these generative models open up a new world of use cases and applications. [![Arize:Observe Unstructured featured image](https://arize.com/wp-content/uploads/2022/06/Featured-image-video-resources.jpg "Arize:Observe Unstructured featured image")\ \ #### Arize: Observe Unstructured 2022\ \ By Joel Bowman |](https://arize.com/arize-observe-unstructured-2022/) [![](https://arize.com/wp-content/uploads/2022/08/Arize-Observe-Unstructured.png "Arize-Observe Unstructured")\ \ #### Arize:Observe Unstructured – Keynote presentation\ \ By Joel Bowman |](https://arize.com/resource/arizeobserve-unstructured-keynote-presentation/) [![Arize:Observe Unstructured featured image](https://arize.com/wp-content/uploads/2022/06/Featured-image-video-resources.jpg "Arize:Observe Unstructured featured image")\ \ #### Arize:Observe Unstructured – Powering the Next Generation of Products with AI\ \ By Joel Bowman |](https://arize.com/resource/arizeobserve-unstructured-powering-the-next-generation-of-products-with-ai/) Subscribe to The Evaluator -------------------------- We’ll send you the latest news, expertise, and product updates from Arize. Your inbox is sacred, so we’ll only curate and send the best stuff. \*We’re committed to your privacy. Arize uses the information you provide to contact you about relevant content, products, and services. You may unsubscribe from these communications at any time. For more information, check out our [privacy policy](https://arize.com/privacy-policy) . --- # See It In Action: Arize Platform Demo, Live Q&A Recording - Arize AI ### Events See It In Action: Arize Platform Demo, Live Q&A Recording ========================================================= ### Recommended Resources [![](https://arize.com/wp-content/uploads/2021/12/Data-Quality-Arize-Superb.png "Data-Quality-Arize-Superb")\ \ MLOps\ \ #### Solving Data Quality with ML Observability and Data Operations\ \ By Krystal Kirkland | 8 minutes read](https://arize.com/blog/solving-data-quality-with-ml-observability-and-data-operations/) [![](https://arize.com/wp-content/uploads/2021/12/New-edition-mockup-large@2x.png "New edition mockup - large@2x")\ \ #### The Definitive Machine Learning Observability Checklist\ \ By Aparna Dhinakaran | 38 seconds read](https://arize.com/resource/machine-learning-observability-checklist/) [![ml observability fraud models](https://arize.com/wp-content/uploads/2021/10/arize-fraud-models-whack-a-mole.png "arize-fraud-models-whack-a-mole")\ \ Use-Case\ \ #### Best Practices In ML Observability for Monitoring, Mitigating and Preventing Fraud\ \ By Tammy Le | 9 minutes read](https://arize.com/blog/best-practices-in-ml-observability-for-monitoring-mitigating-and-preventing-fraud/) ### Subscribe to our resources and blogs [Subscribe](https://arize.com/resource/arize-platform-demo-jan12/on-demand/#blog-subscribe-modal) Subscribe to The Evaluator -------------------------- We’ll send you the latest news, expertise, and product updates from Arize. Your inbox is sacred, so we’ll only curate and send the best stuff. \*We’re committed to your privacy. Arize uses the information you provide to contact you about relevant content, products, and services. You may unsubscribe from these communications at any time. For more information, check out our [privacy policy](https://arize.com/privacy-policy) . --- # Arize:Observe Unstructured - Handling the Challenges of Unstructured Data, The Unsung Hero of Machine Learning - Arize AI ### Videos Arize:Observe Unstructured – Handling the Challenges of Unstructured Data, The Unsung Hero of Machine Learning ============================================================================================================== Arize:Observe Unstructured - Handling the Challenges of Unstructured Data, The Unsung Hero of Machine Learning -------------------------------------------------------------------------------------------------------------- 80% of data is unstructured. So why do most AI/ML tools only handle structured data? We’ve known how to deal with structured data for decades, so it’s no surprise that most companies focus on the tried and true database as their backend. But for the tremendous amounts of unstructured data pouring into data centers, everything from high resolution satellite images, to video from film and TV, to music and audio recordings, to genetics data, financial reports, and chat logs databases just don’t work very well. You need a platform that treats unstructured data as a first class citizen to do some of the most cutting edge work in machine learning today. In this webinar attendees will learn: - How to extract value from your team’s unstructured data - The differences in the ML tech stack between handling unstructured vs structured data - What tools your team can take advantage of today [![](https://arize.com/wp-content/uploads/2022/08/Arize-Observe-Unstructured.png "Arize-Observe Unstructured")\ \ #### Arize:Observe Unstructured – Keynote presentation\ \ By Joel Bowman |](https://arize.com/resource/arizeobserve-unstructured-keynote-presentation/) [![Arize:Observe Unstructured featured image](https://arize.com/wp-content/uploads/2022/06/Featured-image-video-resources.jpg "Arize:Observe Unstructured featured image")\ \ #### Arize:Observe Unstructured – Powering the Next Generation of Products with AI\ \ By Joel Bowman |](https://arize.com/resource/arizeobserve-unstructured-powering-the-next-generation-of-products-with-ai/) [![Jeff Boudier Hugging Face](https://arize.com/wp-content/uploads/2022/06/Screen-Shot-2022-08-22-at-4.14.33-PM.png "Jeff Boudier")\ \ #### Arize:Observe Unstructured – Accelerating Machine Learning from Research to Production with Hugging Face\ \ By Joel Bowman |](https://arize.com/resource/arizeobserve-unstructured-accelerating-machine-learning-from-research-to-production-with-hugging-face/) Subscribe to The Evaluator -------------------------- We’ll send you the latest news, expertise, and product updates from Arize. Your inbox is sacred, so we’ll only curate and send the best stuff. \*We’re committed to your privacy. Arize uses the information you provide to contact you about relevant content, products, and services. You may unsubscribe from these communications at any time. For more information, check out our [privacy policy](https://arize.com/privacy-policy) . --- # Arize:Observe - Keynote - Arize AI ### Videos Arize:Observe – Keynote ======================= Observability is arguably the hottest area of machine learning today. The massive investments companies have put toward digital transformation and building data-centric businesses in the last decade are manifesting as machine learning models in production – yet there’s a gap in the infrastructure required to maintain and improve these models once they are deployed into the real world. In this session, we will explore the state of the ML infrastructure ecosystem, key considerations when building an ML observability practice that can deliver tangible ROI across your organization, and see what’s on the horizon of Arize’s product roadmap. Speakers -------- [![](https://arize.com/wp-content/uploads/2022/04/Aparna-purple.png)\ \ #### Aparna Dhinakaran\ \ Co-founder and CPO, Arize AI](https://arize.com/resource/arize-observe-keynote/#speaker-1) [![Jason Lopatecki](https://arize.com/wp-content/uploads/2022/04/Jason-1.png)\ \ #### Jason Lopatecki\ \ Co-founder and CEO, Arize AI](https://arize.com/resource/arize-observe-keynote/#speaker-2) ![](https://arize.com/wp-content/uploads/2022/04/Aparna-purple.png) #### Aparna Dhinakaran Co-founder and CPO, Arize AI Aparna Dhinakaran is the Co-Founder and Chief Product Officer at Arize AI, a pioneer and early leader in machine learning (ML) observability. A frequent speaker at top conferences and thought leader in the space, Dhinakaran was recently named to the Forbes 30 Under 30 in the Enterprise Technology category. Before Arize, Dhinakaran was an ML engineer and leader at Uber, Apple, and TubeMogul (acquired by Adobe). During her time at Uber, she built several core ML infrastructure platforms, including Michealangelo. She has a bachelor’s from Berkeley’s Electrical Engineering and Computer Science program, where she published research with Berkeley’s AI Research group. She is on a leave of absence from the Computer Vision Ph.D. program at Cornell University. ![Jason Lopatecki](https://arize.com/wp-content/uploads/2022/04/Jason-1.png) #### Jason Lopatecki Co-founder and CEO, Arize AI Jason Lopatecki is co-founder and CEO of Arize AI, a machine learning observability company. He is a garage-to-IPO executive with an extensive background in building marketing-leading products and businesses that heavily leverage analytics. Prior to Arize, Jason was co-founder and chief innovation officer at TubeMogul where he scaled the business into a public company and eventual acquisition by Adobe. Jason has hands-on knowledge of big data architectures, programmatic advertising systems, distributed systems, and machine learning and data processing architectures. In his free time, Jason tinkers with personal machine learning projects as a hobby, with a special interest in unsupervised learning and deep neural networks. He holds an electrical engineering and computer science degree from UC Berkeley - Go Bears! ### Recommended Resources [![Arize-Observe-3](https://arize.com/wp-content/uploads/2022/04/Arize-Observe-3.jpg "Arize-Observe-3")\ \ #### Arize:Observe – Scaling Your ML Practice\ \ By Joel Bowman |](https://arize.com/resource/scaling-your-ml-practice/) [![Arize-Observe-2](https://arize.com/wp-content/uploads/2022/04/Arize-Observe-2.jpg "Arize-Observe-2")\ \ #### Arize:Observe – Bracing yourself for a world of data-centric AI\ \ By Joel Bowman |](https://arize.com/resource/bracing-yourself-for-a-world-of-data-centricai/) [![Arize-Observe-alt](https://arize.com/wp-content/uploads/2022/04/Arize-Observe-alt.jpg "Arize-Observe-alt")\ \ #### Arize:Observe – Closing the Gap Between AI Teams and Business Executives\ \ By Joel Bowman |](https://arize.com/resource/closing-the-gap-between-ai-teams-and-business-executives/) Subscribe to The Evaluator -------------------------- We’ll send you the latest news, expertise, and product updates from Arize. Your inbox is sacred, so we’ll only curate and send the best stuff. \*We’re committed to your privacy. Arize uses the information you provide to contact you about relevant content, products, and services. You may unsubscribe from these communications at any time. For more information, check out our [privacy policy](https://arize.com/privacy-policy) . --- # See It In Action: Arize Platform Demo, Live Q&A - Arize AI ### Events See It In Action: Arize Platform Demo, Live Q&A =============================================== **On-Demand** Arize AI, an early pioneer and leader in ML observability, tracks billions of ML predictions on behalf of enterprises and disruptive startups. Curious about how Arize AI might help your team?  These 30-minute open sessions offer: * An interactive demo of the Arize platform, covering a variety of use-cases * Opportunities to casually interject, asking anything throughout the demo or at the end * A casual way to see the platform in action and hear from peers This session will be led by Amber Roberts, a sales engineer at Arize who previously worked as both a data scientist and ML engineer (and an astrophysicist) ![](https://arize.com/wp-content/uploads/2021/12/Blog-Amber-500x343-1.jpg) ### Watch On-Demand Speakers -------- ![](https://arize.com/wp-content/uploads/2021/11/Amber.png) ##### Amber Roberts ###### Machine Learning Engineer Amber Roberts is an astrophysicist and machine learning engineer who was previously the Head of AI at Insight Data Science. Since then she has been at Splunk in their ML Product Org to build out ML feature solutions as a ML Product Manager. She now joins us at Arize as a ML Sales Engineer looking to help teams across industries build ML Observability into their productionalized AI environments. Subscribe to The Evaluator -------------------------- We’ll send you the latest news, expertise, and product updates from Arize. Your inbox is sacred, so we’ll only curate and send the best stuff. \*We’re committed to your privacy. Arize uses the information you provide to contact you about relevant content, products, and services. You may unsubscribe from these communications at any time. For more information, check out our [privacy policy](https://arize.com/privacy-policy) . --- # Arize Platform Demo - Arize AI ### Videos Arize Platform Demo =================== A short demo of the Arize AI observability and LLM evaluation platform. In this video, we’ll cover the basics of monitoring and troubleshooting performance issues, so you can deploy better models to production with confidence. [https://arize.com/wp-content/uploads/2023/02/Arize-Product-Demo-720.mp4](https://arize.com/wp-content/uploads/2023/02/Arize-Product-Demo-720.mp4) Recommended Resources --------------------- [![](https://arize.com/wp-content/uploads/2025/01/Building-an-Agent-Router-video-thumbnail-1030x600.jpg)](https://arize.com/resource/building-an-agent-router-best-practices/) Videos #### [Building an Agent Router: Best Practices](https://arize.com/resource/building-an-agent-router-best-practices/) By Samantha White | [![Text reads: Agentic RAG](https://arize.com/wp-content/uploads/2024/12/Agentic-RAG-video-thumbnail-1030x600.jpg)](https://arize.com/resource/understanding-agentic-rag/) Videos #### [Understanding Agentic RAG](https://arize.com/resource/understanding-agentic-rag/) By Trevor LaViale | [![Eric Xiao and an AI teddy bear, building better AI](https://arize.com/wp-content/uploads/2024/10/Building-Better-AI-1030x600.jpg)](https://arize.com/resource/improving-safety-and-reliability-of-llm-applications/) Videos #### [Improving Safety and Reliability of LLM Applications](https://arize.com/resource/improving-safety-and-reliability-of-llm-applications/) By Eric Xiao | ### Subscribe to our resources and blogs [Subscribe](https://arize.com/resource/arize-platform-demo/#blog-subscribe-modal) Subscribe to The Evaluator -------------------------- We’ll send you the latest news, expertise, and product updates from Arize. Your inbox is sacred, so we’ll only curate and send the best stuff. \*We’re committed to your privacy. Arize uses the information you provide to contact you about relevant content, products, and services. You may unsubscribe from these communications at any time. For more information, check out our [privacy policy](https://arize.com/privacy-policy) . --- # Arize AI Un/Summit - Epic Fail: How Models Fail - Arize AI ### Summit Arize AI Un/Summit – Epic Fail: How Models Fail =============================================== This chat is a part of Arize AI’s ML Observability Un/Summit 2020 Panelists: Anthony Goldbloom, Founder & CEO @ Kaggle Chintan Turakhia, Senior Engineering Manager @ Uber Recommended Resources --------------------- [![](https://arize.com/wp-content/uploads/2021/06/pic-un-summit@2x.jpg)](https://arize.com/resource/arize-ai-un-summit-model-improvement/) Summit #### [Arize AI Un/Summit – Model Improvement](https://arize.com/resource/arize-ai-un-summit-model-improvement/) By Krystal Kirkland | 10 seconds read [![](https://arize.com/wp-content/uploads/2021/06/pic-un-summit@2x.jpg)](https://arize.com/resource/arize-ai-un-summit-eye-on-the-prize-ai-ethics/) Summit #### [Arize AI Un/Summit – Eye on the Prize: AI Ethics](https://arize.com/resource/arize-ai-un-summit-eye-on-the-prize-ai-ethics/) By Krystal Kirkland | 12 seconds read [![](https://arize.com/wp-content/uploads/2021/06/pic-un-summit@2x.jpg)](https://arize.com/resource/ml-monitoring-vs-ml-observability/) Summit #### [Arize AI Un/Summit – ML Observability v. ML Monitoring](https://arize.com/resource/ml-monitoring-vs-ml-observability/) By Jason Lopatecki | 9 seconds read ### Subscribe to our resources and blogs [Subscribe](https://arize.com/resource/arize-ai-un-summit-epic-fail-how-models-fail/#blog-subscribe-modal) Subscribe to The Evaluator -------------------------- We’ll send you the latest news, expertise, and product updates from Arize. Your inbox is sacred, so we’ll only curate and send the best stuff. \*We’re committed to your privacy. Arize uses the information you provide to contact you about relevant content, products, and services. You may unsubscribe from these communications at any time. For more information, check out our [privacy policy](https://arize.com/privacy-policy) . --- # Arize AI Un/Summit - ML Observability in Finance - Arize AI ### Summit Arize AI Un/Summit – ML Observability in Finance ================================================ This chat is a part of Arize AI’s ML Observability Un/Summit 2020 Panelists: Sterling Blood – Data Engineering Manager @ Divvy Zach Ernst – Machine Learning Lead @ Clearcover Brandon Duderstadt – Senior Machine Learning Engineer @ Square Recommended Resources --------------------- [![](https://arize.com/wp-content/uploads/2025/01/Building-an-Agent-Router-video-thumbnail-1030x600.jpg)](https://arize.com/resource/building-an-agent-router-best-practices/) Videos #### [Building an Agent Router: Best Practices](https://arize.com/resource/building-an-agent-router-best-practices/) By Samantha White | [![Text reads: Agentic RAG](https://arize.com/wp-content/uploads/2024/12/Agentic-RAG-video-thumbnail-1030x600.jpg)](https://arize.com/resource/understanding-agentic-rag/) Videos #### [Understanding Agentic RAG](https://arize.com/resource/understanding-agentic-rag/) By Trevor LaViale | [![Eric Xiao and an AI teddy bear, building better AI](https://arize.com/wp-content/uploads/2024/10/Building-Better-AI-1030x600.jpg)](https://arize.com/resource/improving-safety-and-reliability-of-llm-applications/) Videos #### [Improving Safety and Reliability of LLM Applications](https://arize.com/resource/improving-safety-and-reliability-of-llm-applications/) By Eric Xiao | ### Subscribe to our resources and blogs [Subscribe](https://arize.com/resource/arize-ai-un-summit-ml-observability-in-finance/#blog-subscribe-modal) Subscribe to The Evaluator -------------------------- We’ll send you the latest news, expertise, and product updates from Arize. Your inbox is sacred, so we’ll only curate and send the best stuff. \*We’re committed to your privacy. Arize uses the information you provide to contact you about relevant content, products, and services. You may unsubscribe from these communications at any time. For more information, check out our [privacy policy](https://arize.com/privacy-policy) . --- # Arize AI Un/Summit - Model Improvement - Arize AI ### Summit Arize AI Un/Summit – Model Improvement ====================================== This chat is a part of Arize AI’s ML Observability Un/Summit 2020 Panelists: John Trenkle – Principal Machine Learning Engineer @ TubiChristopher Brown- Chief Data Scientist @ ADARA, Inc. Recommended Resources --------------------- [![](https://arize.com/wp-content/uploads/2025/01/Building-an-Agent-Router-video-thumbnail-1030x600.jpg)](https://arize.com/resource/building-an-agent-router-best-practices/) Videos #### [Building an Agent Router: Best Practices](https://arize.com/resource/building-an-agent-router-best-practices/) By Samantha White | [![Text reads: Agentic RAG](https://arize.com/wp-content/uploads/2024/12/Agentic-RAG-video-thumbnail-1030x600.jpg)](https://arize.com/resource/understanding-agentic-rag/) Videos #### [Understanding Agentic RAG](https://arize.com/resource/understanding-agentic-rag/) By Trevor LaViale | [![Eric Xiao and an AI teddy bear, building better AI](https://arize.com/wp-content/uploads/2024/10/Building-Better-AI-1030x600.jpg)](https://arize.com/resource/improving-safety-and-reliability-of-llm-applications/) Videos #### [Improving Safety and Reliability of LLM Applications](https://arize.com/resource/improving-safety-and-reliability-of-llm-applications/) By Eric Xiao | ### Subscribe to our resources and blogs [Subscribe](https://arize.com/resource/arize-ai-un-summit-model-improvement/#blog-subscribe-modal) Subscribe to The Evaluator -------------------------- We’ll send you the latest news, expertise, and product updates from Arize. Your inbox is sacred, so we’ll only curate and send the best stuff. \*We’re committed to your privacy. Arize uses the information you provide to contact you about relevant content, products, and services. You may unsubscribe from these communications at any time. For more information, check out our [privacy policy](https://arize.com/privacy-policy) . --- # Arize AI Un/Summit - Eye on the Prize: AI Ethics - Arize AI ### Summit Arize AI Un/Summit – Eye on the Prize: AI Ethics ================================================ This chat is a part of Arize AI’s ML Observability Un/Summit 2020 Panelist: Margarita Boenig-Liptsin – Instructor and Program Lead, Human Contexts and Ethics, Division of Data Science and Information, UC Berkeley Recommended Resources --------------------- [![](https://arize.com/wp-content/uploads/2025/01/Building-an-Agent-Router-video-thumbnail-1030x600.jpg)](https://arize.com/resource/building-an-agent-router-best-practices/) Videos #### [Building an Agent Router: Best Practices](https://arize.com/resource/building-an-agent-router-best-practices/) By Samantha White | [![Text reads: Agentic RAG](https://arize.com/wp-content/uploads/2024/12/Agentic-RAG-video-thumbnail-1030x600.jpg)](https://arize.com/resource/understanding-agentic-rag/) Videos #### [Understanding Agentic RAG](https://arize.com/resource/understanding-agentic-rag/) By Trevor LaViale | [![Eric Xiao and an AI teddy bear, building better AI](https://arize.com/wp-content/uploads/2024/10/Building-Better-AI-1030x600.jpg)](https://arize.com/resource/improving-safety-and-reliability-of-llm-applications/) Videos #### [Improving Safety and Reliability of LLM Applications](https://arize.com/resource/improving-safety-and-reliability-of-llm-applications/) By Eric Xiao | ### Subscribe to our resources and blogs [Subscribe](https://arize.com/resource/arize-ai-un-summit-eye-on-the-prize-ai-ethics/#blog-subscribe-modal) Subscribe to The Evaluator -------------------------- We’ll send you the latest news, expertise, and product updates from Arize. Your inbox is sacred, so we’ll only curate and send the best stuff. \*We’re committed to your privacy. Arize uses the information you provide to contact you about relevant content, products, and services. You may unsubscribe from these communications at any time. For more information, check out our [privacy policy](https://arize.com/privacy-policy) . ---