# Table of Contents

- [Features - Mage AI](#features-mage-ai)
- [Frequently Asked Questions - Mage AI](#frequently-asked-questions-mage-ai)
- [Roadmap - Mage AI](#roadmap-mage-ai)
- [AI Clients - Mage AI](#ai-clients-mage-ai)
- [Code of Conduct - Mage AI](#code-of-conduct-mage-ai)
- [Releases - Mage AI](#releases-mage-ai)
- [Using AI (artificial intelligence) - Mage AI](#using-ai-artificial-intelligence-mage-ai)
- [Help improve the tool - Mage AI](#help-improve-the-tool-mage-ai)
- [Changelog - Mage AI](#changelog-mage-ai)
- [Retrieval Augmented Generation (RAG) pipeline builder - Mage AI](#retrieval-augmented-generation-rag-pipeline-builder-mage-ai)
- [Customized AI resources for training and inference - Mage AI](#customized-ai-resources-for-training-and-inference-mage-ai)
- [Machine learning pipeline tutorial - Mage AI](#machine-learning-pipeline-tutorial-mage-ai)
- [User defined permissions - Mage AI](#user-defined-permissions-mage-ai)
- [LinkedIn - Mage AI](#linkedin-mage-ai)
- [Twitter - Mage AI](#twitter-mage-ai)
- [Roles - Mage AI](#roles-mage-ai)
- [Your AI data engineer - Mage AI](#your-ai-data-engineer-mage-ai)
- [Adding an IO class - Mage AI](#adding-an-io-class-mage-ai)
- [Slack - Mage AI](#slack-mage-ai)
- [Blog - Mage AI](#blog-mage-ai)

---

# Features - Mage AI

[Mage AI home page![light logo](https://mintlify.s3.us-west-1.amazonaws.com/mage/logo/m-mage-light.svg)![dark logo](https://mintlify.s3.us-west-1.amazonaws.com/mage/logo/m-mage.svg)](/)

Latest

Omniscience... ✨

Search...

Navigation

Features

[Documentation](/introduction/overview)
[Development](/guides/overview)
[Production](/production/ci-cd/overview)
[AI](/ai/sidekick/index)
[Observability](/observability/monitoring)
[Integrations](/integrations/compute/spark-pyspark)
[Extensibility](/extensibility/global-hooks/overview)

[Data pipeline management\
------------------------](/design/data-pipeline-management)
[Notebook for building data pipelines\
------------------------------------](/about/features#1-data-centric-editor)
[Changelog\
---------](https://github.com/mage-ai/mage-ai/releases)
[Roadmap\
-------](https://airtable.com/shrJS0cDOmQywb8vp)

[​](#data-pipeline-management)

Data pipeline management
----------------------------------------------------------

👉 See more [details here](/design/data-pipeline-management)
.

[​](#notebook-for-building-data-pipelines)

Notebook for building data pipelines
----------------------------------------------------------------------------------

### 

[​](#1-data-centric-editor)

1\. Data centric editor

An interactive coding experience designed for preparing data to train ML models.

Visualize the impact of your code every time you load, clean, and transform data.

### 

[​](#2-production-ready-code)

2\. Production ready code

No more writing throw away code or trying to turn notebooks into scripts.

Each block (aka cell) in this editor is a modular file that can be tested, reused, and chained together to create an executable data pipeline locally or in any environment.

Read more about [blocks](/design/blocks)
 and how they work.

Run your data pipeline end-to-end using the command line function: `$ mage run [project] [pipeline]`

You can run your pipeline in production environments with the orchestration tools

*   [Airflow](/guides/integrate-mage-airflow)
    
*   [Prefect](/integrations/prefect)
    

### 

[​](#3-extensible)

3\. Extensible

Easily add new functionality directly in the source code or through plug-ins (coming soon).

Adding new API endpoints ([Tornado](https://www.tornadoweb.org/en/stable/)
), transformations (Python, PySpark, SQL), and charts (using [React](https://reactjs.org/)
) is easy to do (tutorial coming soon).

Was this page helpful?

YesNo

[Suggest edits](https://github.com/mage-ai/mage-ai/edit/master/docs/about/features.mdx)
[Raise issue](https://github.com/mage-ai/mage-ai/issues/new?title=Issue on docs&body=Path: /about/features)

On this page

*   [Data pipeline management](#data-pipeline-management)
    
*   [Notebook for building data pipelines](#notebook-for-building-data-pipelines)
    
*   [1\. Data centric editor](#1-data-centric-editor)
    
*   [2\. Production ready code](#2-production-ready-code)
    
*   [3\. Extensible](#3-extensible)

---

# Frequently Asked Questions - Mage AI

[Mage AI home page![light logo](https://mintlify.s3.us-west-1.amazonaws.com/mage/logo/m-mage-light.svg)![dark logo](https://mintlify.s3.us-west-1.amazonaws.com/mage/logo/m-mage.svg)](/)

Latest

Omniscience... ✨

Search...

Navigation

About

Frequently Asked Questions

[Documentation](/introduction/overview)
[Development](/guides/overview)
[Production](/production/ci-cd/overview)
[AI](/ai/sidekick/index)
[Observability](/observability/monitoring)
[Integrations](/integrations/compute/spark-pyspark)
[Extensibility](/extensibility/global-hooks/overview)

What is Mage?

Mage is an open-source data pipeline tool for transforming and integrating data.

🧙 A mage is someone who uses magic. Advanced technology is indistinguishable from magic.

We’re on a mission to make AI technology more accessible by building data tools for engineers and scientists.

Find out more about our story: [https://www.mage.ai/blog/mage-heros-journey-fantasy-epic-on-how-a-startup-rose-from-the-ashes](https://www.mage.ai/blog/mage-heros-journey-fantasy-epic-on-how-a-startup-rose-from-the-ashes)

Who is the ideal user for this tool?

Our tool was built with data engineers and data scientists in mind, but is not limited to those roles. Other data professionals could find value in the tool.

How difficult is Mage to setup?

You can quickly and easily get started by installing Mage using Docker (recommended), **`pip`**, or **`conda`**. Click [here](https://docs.mage.ai/introduction/overview#quick-start)
 for details.

How much does Mage cost?

Mage is free as long as you are self-hosted (AWS, GCP, Azure, or Digital Ocean).

How is Mage’s data pipeline engine software different from Airflow, Dagster, etc?

Our 4 [core design principles](/design/core-design-principles)
 that differentiate ourselves are:

1.  Easy developer experience
    
2.  Engineering best practices built-in
    
3.  Data is a first-class citizen
    
4.  Scaling is made simple
    

Features that set us apart (some of the others might eventually have these features):

1.  Mix and match SQL and Python in data pipeline tasks.
2.  UI/IDE for building and managing data pipelines.
3.  Data centric: we designed and built a pipeline engine ONLY for moving and transforming data. This makes it possible for us to make datasets a 1st class citizen; enabling native features such as partitioning, versioning, backfilling, data validation, testing, and data quality monitoring.
4.  Extensible: we designed and built the tool with developers in mind, making sure it’s really easy to add new functionality to the source code or through plug-ins.
5.  Scalable: the tool can handle very, very large datasets while transforming the data or charting it.
6.  Production ready: when you build your data pipeline, it runs exactly the same in development as it does in production. Deploying the tool and managing the infrastructure in production is very easy and simple, unlike Airflow.
7.  Modular: every block/cell you write is a standalone file that is interoperable; meaning it can be used in other pipelines or in other code bases.

What’s the difference between Mage and Fivetran?

Check out our blog [Mage vs. Fivetran](https://www.mage.ai/blog/mage-vs-fivetran)
.

What’s the difference between Mage and Airbyte?

Check out our blog [Mage vs Airbyte](https://www.mage.ai/blog/mage-vs-airbyte)
.

What’s the difference between Mage and Prefect?

Mage provides an interactive notebook with built-in engineering best practices for building pipelines, which makes prototyping and building production-ready pipelines much easier.

Mage supports writing pipelines in multiple languages which include Python, SQL, and R.

Mage supports multiple types of pipelines natively such as:

*   Standard batch pipelines
*   Data integration pipelines
*   Streaming pipelines
*   Spark pipelines
*   DBT pipelines

What languages does Mage support?

We currently support SQL, Python, R, and PySpark. Coming soon: Spark SQL.

Does Mage integrate with Spark?

Yes! [Here](https://docs.mage.ai/integrations/spark-pyspark)
 is a step-by-step tutorial to use Mage with Spark on EMR.

What’s the difference between Mage and Sagemaker?

Sagemaker is used to train machine learning models and serve them via api.

Mage is an engine for running data pipelines that can move and transform data. That data can then be stored anywhere (e.g. S3) and used to train models in Sagemaker.

What’s the difference between Mage and Databricks?

Databricks provides infrastructure to run Spark. They also provide notebooks that can run your code in Spark as well.

Mage can execute your code in a Spark cluster, managed by AWS, GCP, or even Databricks.

How do I send pipeline notifications to Slack?

[Here is a doc](/integrations/observability/alerting-slack)
 to help you set up alerting for pipeline status updates in [Slack](https://www.mage.ai/chat)
.

How can I contribute or request features?

We love and welcome community contributions! [Here is a doc](/contributing/overview)
 to get you started.

To request features, add a “Feature request” using the `New issue` button in GitHub from this [link](https://github.com/mage-ai/mage-ai/issues)
, or join our [feature-request](https://www.mage.ai/chat)
 Slack channel.

_Can’t find what you’re looking for? [Ask a question here](https://github.com/mage-ai/mage-ai/discussions/categories/q-a)
 or [join our slack](https://mage.ai/chat)
 for additional support!_

Was this page helpful?

YesNo

[Suggest edits](https://github.com/mage-ai/mage-ai/edit/master/docs/about/frequently-asked-questions.mdx)
[Raise issue](https://github.com/mage-ai/mage-ai/issues/new?title=Issue on docs&body=Path: /about/frequently-asked-questions)

[Usage statistics](/about/statistics)
[Changelog](/about/releases)

---

# Roadmap - Mage AI

[Mage AI home page![light logo](https://mintlify.s3.us-west-1.amazonaws.com/mage/logo/m-mage-light.svg)![dark logo](https://mintlify.s3.us-west-1.amazonaws.com/mage/logo/m-mage.svg)](/)

Latest

Omniscience... ✨

Search...

Navigation

Roadmap

[Documentation](/introduction/overview)
[Development](/guides/overview)
[Production](/production/ci-cd/overview)
[AI](/ai/sidekick/index)
[Observability](/observability/monitoring)
[Integrations](/integrations/compute/spark-pyspark)
[Extensibility](/extensibility/global-hooks/overview)

Roadmap
-------

Mage’s 2024 Roadmap coming soon!

Was this page helpful?

YesNo

[Suggest edits](https://github.com/mage-ai/mage-ai/edit/master/docs/about/roadmap.mdx)
[Raise issue](https://github.com/mage-ai/mage-ai/issues/new?title=Issue on docs&body=Path: /about/roadmap)

---

# AI Clients - Mage AI

[Mage AI home page![light logo](https://mintlify.s3.us-west-1.amazonaws.com/mage/logo/m-mage-light.svg)![dark logo](https://mintlify.s3.us-west-1.amazonaws.com/mage/logo/m-mage.svg)](/)

Latest

Omniscience... ✨

Search...

Navigation

Development

AI Clients

[Documentation](/introduction/overview)
[Development](/guides/overview)
[Production](/production/ci-cd/overview)
[AI](/ai/sidekick/index)
[Observability](/observability/monitoring)
[Integrations](/integrations/compute/spark-pyspark)
[Extensibility](/extensibility/global-hooks/overview)

[​](#ai-client)

AI Client
----------------------------

You now have the option to select from various AI clients to harness [the capabilities of Mage AI](https://docs.mage.ai/guides/ai/overview)
, as detailed in the Mage AI capabilities documentation. Currently, we offer support for OpenAI and Hugging Face, with the promise of additional AI clients being added in the future.

[​](#use-hugging-face-client)

Use Hugging Face Client
--------------------------------------------------------

### 

[​](#setup)

Setup

#### 

[​](#hugging-face-inference-endpoint)

Hugging Face Inference Endpoint

In order to utilize the Hugging Face AI Client, it is necessary to establish a Hugging Face inference endpoint. You can set it up following [this guide](https://huggingface.co/inference-endpoints)
.

This process is quite straightforward. It entails

*   selecting the specific model you wish to use,
*   determining the hosting environment (AWS or Azure),
*   specifying the geographical region,
*   choosing the type of GPU.

For your convenience and based on our testing, we recommend using the “mistralai/Mistral-7B-Instruct-v0.1” model.

Once the Inference endpoint is operational, it will provide you with an API URL and a corresponding token for establishing a secure connection.

#### 

[​](#mage-project-setup)

Mage Project Setup

Within your Mage project’s metadata YAML configuration, please include the subsequent “ai\_config” section:

    ai_config:
      mode: 'hugging_face'
      open_ai_config:
        openai_api_key: key
      hugging_face_config:
        huggingface_api: api_url
        huggingface_inference_api_token: api_token
    

The “mode” parameter determines your selection of the AI client to be employed. It can be specified as either “open\_ai” or “hugging\_face,” with the default value being set to “open\_ai.”

“hugging\_face\_config” as a mandatory configuration if you choose to use the hugging face client. This configuration includes the two essential elements obtained from the Hugging Face inference endpoint, namely, the API and Token.

You are ready to go once the “ai\_config” is setup. At this point, you can fully leverage Mage AI’s capabilities, such as generating blocks with text description, automatically write comments for your functions, etc.

[​](#how-to-add-a-new-ai-client)

How to add a new AI Client
--------------------------------------------------------------

You may find it necessary to employ an AI client other than those offered by OpenAI and Hugging Face. Additionally, you might wish to make direct calls to your Language Model (LLM). This can be accomplished by enabling a new AI client for your specific needs.

This is an [example PR](https://github.com/mage-ai/mage-ai/pull/3850)
.

### 

[​](#create-new-ai-config)

Create new AI config

Create a dedicated configuration to save the params required to connect to LLM in the [config.py](https://github.com/mage-ai/mage-ai/blob/92c372b24e08148863d799d9afcdd44483c11c89/mage_ai/orchestration/ai/config.py#L19)
. For instance, when using the Hugging Face client, the LLM is hosted within the inference endpoint, mandating both the API and Token for invoking the service for inference. In the OpenAI client, the OpenAI key is required to facilitate model inference.

### 

[​](#create-dedicated-ai-client)

Create dedicated AI Client

Inherit the [AIClient interface](https://github.com/mage-ai/mage-ai/blob/master/mage_ai/ai/ai_client.py)
 and implements the two required functions: “inference\_with\_prompt” and “find\_block\_params”.

*   Inference\_with\_prompt function does the LLM model inference. It takes the prompt template, required variables being used in the prompt and return the inference result.
*   Find\_block\_params function does a multi classification based on code description to generate required types including block\_type, pipeline\_type, language, action type and data source.

You can read your configuration in the Setup function and initialize the client to talk to your service.

### 

[​](#enable-in-llm-pipeline-wizard)

Enable in llm\_pipeline\_wizard

The last action to take is modifying the Setup function within “[mage\_ai/ai/llm\_pipeline\_wizard.py](https://github.com/mage-ai/mage-ai/blob/92c372b24e08148863d799d9afcdd44483c11c89/mage_ai/ai/llm_pipeline_wizard.py#L195)
” to introduce a new mode of your client and initialize your AI client.

Was this page helpful?

YesNo

[Suggest edits](https://github.com/mage-ai/mage-ai/edit/master/docs/ai/ai-client.mdx)
[Raise issue](https://github.com/mage-ai/mage-ai/issues/new?title=Issue on docs&body=Path: /ai/ai-client)

[Getting started](/ai/setup)
[Machine learning](/ai/ml/train-model)

On this page

*   [AI Client](#ai-client)
    
*   [Use Hugging Face Client](#use-hugging-face-client)
    
*   [Setup](#setup)
    
*   [Hugging Face Inference Endpoint](#hugging-face-inference-endpoint)
    
*   [Mage Project Setup](#mage-project-setup)
    
*   [How to add a new AI Client](#how-to-add-a-new-ai-client)
    
*   [Create new AI config](#create-new-ai-config)
    
*   [Create dedicated AI Client](#create-dedicated-ai-client)
    
*   [Enable in llm\_pipeline\_wizard](#enable-in-llm-pipeline-wizard)

---

# Code of Conduct - Mage AI

[Mage AI home page![light logo](https://mintlify.s3.us-west-1.amazonaws.com/mage/logo/m-mage-light.svg)![dark logo](https://mintlify.s3.us-west-1.amazonaws.com/mage/logo/m-mage.svg)](/)

Latest

Omniscience... ✨

Search...

Navigation

About

Code of Conduct

[Documentation](/introduction/overview)
[Development](/guides/overview)
[Production](/production/ci-cd/overview)
[AI](/ai/sidekick/index)
[Observability](/observability/monitoring)
[Integrations](/integrations/compute/spark-pyspark)
[Extensibility](/extensibility/global-hooks/overview)

[​](#our-pledge)

Our Pledge
------------------------------

We as members, contributors, and leaders pledge to make participation in our community a _magical_ ✨ experience for everyone.

We pledge to act and interact in ways that contribute to an open, welcoming, diverse, inclusive, and healthy community.

[​](#our-standards)

Our Standards
------------------------------------

Examples of behavior that contributes to a positive environment for our community include:

*   🪄 Deliver magical experiences: demonstrating empathy and kindness toward others. Being respectful of differing opinions, viewpoints, and experiences
*   🔋 Give people power-ups: in data engineering _and_ open-source development, doing your best to help others learn and grow
*   👬 Victorious as a team: working together to build the best product possible, recognizing that teamwork can amplify outcomes beyond that of an individual
*   🙅‍♂️ No Ego: giving and gracefully accepting constructive feedback

Examples of unacceptable behavior include:

*   The use of sexualized language or imagery, and sexual attention or advances of any kind
*   Trolling, insulting or derogatory comments, and personal or political attacks
*   Public or private harassment
*   Publishing others’ private information, such as a physical or email address, without their explicit permission
*   Other conduct which could reasonably be considered inappropriate in a professional setting

[​](#enforcement-responsibilities)

Enforcement Responsibilities
------------------------------------------------------------------

Community leaders are responsible for clarifying and enforcing our standards of acceptable behavior and will take appropriate and fair corrective action in response to any behavior that they deem inappropriate, threatening, offensive, or harmful.

Community leaders have the right and responsibility to remove, edit, or reject comments, commits, code, wiki edits, issues, and other contributions that are not aligned to this Code of Conduct, and will communicate reasons for moderation decisions when appropriate.

[​](#scope)

Scope
--------------------

This Code of Conduct applies within all community spaces, and also applies when an individual is officially representing the community in public spaces. Examples of representing our community include using an official e-mail address, posting via an official social media account, or acting as an appointed representative at an online or offline event.

[​](#enforcement)

Enforcement
--------------------------------

Instances of abusive, harassing, or otherwise unacceptable behavior may be reported to the community leaders responsible for enforcement— please reach out to [the Mage team](mailto:hello@mage.ai)
 with any concerns. All complaints will be reviewed and investigated promptly and fairly.

All community leaders are obligated to respect the privacy and security of the reporter of any incident.

[​](#enforcement-guidelines)

Enforcement Guidelines
------------------------------------------------------

Community leaders will follow these Community Impact Guidelines in determining the consequences for any action they deem in violation of this Code of Conduct:

### 

[​](#1-correction)

1\. Correction

**Community Impact**: Use of inappropriate language or other behavior deemed unprofessional or unwelcome in the community.

**Consequence**: A private, written warning from community leaders, providing clarity around the nature of the violation and an explanation of why the behavior was inappropriate.

### 

[​](#2-warning)

2\. Warning

**Community Impact**: A violation through a single incident or series of actions.

**Consequence**: A warning with consequences for continued behavior. No interaction with the people involved, including unsolicited interaction with those enforcing the Code of Conduct, for a specified period of time. This includes avoiding interactions in community spaces as well as external channels like social media. Violating these terms may lead to a temporary or permanent ban.

### 

[​](#3-temporary-ban)

3\. Temporary Ban

**Community Impact**: A serious violation of community standards, including sustained inappropriate behavior.

**Consequence**: A temporary ban from any sort of interaction or public communication with the community for a specified period of time. No public or private interaction with the people involved, including unsolicited interaction with those enforcing the Code of Conduct, is allowed during this period. Violating these terms may lead to a permanent ban.

### 

[​](#4-permanent-ban)

4\. Permanent Ban

**Community Impact**: Demonstrating a pattern of violation of community standards, including sustained inappropriate behavior, harassment of an individual, or aggression toward or disparagement of classes of individuals.

**Consequence**: A permanent ban from any sort of public interaction within the community.

[​](#attribution)

Attribution
--------------------------------

This Code of Conduct is adapted from the [Contributor Covenant](https://www.contributor-covenant.org)
, version 2.1, available at [https://www.contributor-covenant.org/version/2/1/code\_of\_conduct.html](https://www.contributor-covenant.org/version/2/1/code_of_conduct.html)
.

Community Impact Guidelines were inspired by [Mozilla’s code of conduct enforcement ladder](https://github.com/mozilla/diversity)
.

For answers to common questions about this code of conduct, see the FAQ at [https://www.contributor-covenant.org/faq](https://www.contributor-covenant.org/faq)
. Translations are available at [https://www.contributor-covenant.org/translations](https://www.contributor-covenant.org/translations)
.

Was this page helpful?

YesNo

[Suggest edits](https://github.com/mage-ai/mage-ai/edit/master/docs/about/code-of-conduct.mdx)
[Raise issue](https://github.com/mage-ai/mage-ai/issues/new?title=Issue on docs&body=Path: /about/code-of-conduct)

[Overview](/contributing/documentation/overview)
[Usage statistics](/about/statistics)

On this page

*   [Our Pledge](#our-pledge)
    
*   [Our Standards](#our-standards)
    
*   [Enforcement Responsibilities](#enforcement-responsibilities)
    
*   [Scope](#scope)
    
*   [Enforcement](#enforcement)
    
*   [Enforcement Guidelines](#enforcement-guidelines)
    
*   [1\. Correction](#1-correction)
    
*   [2\. Warning](#2-warning)
    
*   [3\. Temporary Ban](#3-temporary-ban)
    
*   [4\. Permanent Ban](#4-permanent-ban)
    
*   [Attribution](#attribution)

---

# Releases - Mage AI

[Mage AI home page![light logo](https://mintlify.s3.us-west-1.amazonaws.com/mage/logo/m-mage-light.svg)![dark logo](https://mintlify.s3.us-west-1.amazonaws.com/mage/logo/m-mage.svg)](/)

Latest

Omniscience... ✨

Search...

Navigation

About

Releases

[Documentation](/introduction/overview)
[Development](/guides/overview)
[Production](/production/ci-cd/overview)
[AI](/ai/sidekick/index)
[Observability](/observability/monitoring)
[Integrations](/integrations/compute/spark-pyspark)
[Extensibility](/extensibility/global-hooks/overview)

[Update Mage\
-----------\
\
Install the newest version of Mage](/development/updating-mage)

* * *

[All releases\
------------\
\
Install the newest version of Mage](https://github.com/mage-ai/mage-ai/releases)
  

[​](#kaos)

Kaos

0.9.74

[Read full release notes](https://github.com/mage-ai/mage-ai/releases/tag/0.9.74)

### ☁️ Google cloud storage source

You can now effortlessly configure your data integration source blocks to Google Cloud Storage using this snippet. It specifies the destination bucket, sets the file type to Parquet, and provides the path to your service account credentials needed for authentication.

by @TalaatHasanin in [https://github.com/mage-ai/mage-ai/pull/5334](https://github.com/mage-ai/mage-ai/pull/5334)

### 🌬️ 🍽️ Airtable Integration

Effortlessly streamline your data data integration source blocks with Airtable using this tailored script. It establishes a connection to your Airtable base, designates the specific table for integration, and securely integrates your Airtable API key for access.

by @TalaatHasanin in [https://github.com/mage-ai/mage-ai/pull/5404](https://github.com/mage-ai/mage-ai/pull/5404)

### ✂️ Add trim reformat action to transformer block

A new “trim reformat” action has been added to the Python transformer block, allowing for the removal of leading and trailing whitespace from specified text columns. This enhancement ensures cleaner and more consistent data formatting by automatically stripping unnecessary spaces around text entries.

by @cristopheridlc in [https://github.com/mage-ai/mage-ai/pull/5321](https://github.com/mage-ai/mage-ai/pull/5321)

### 🐻‍❄️ Enable polars dataframe in GCS data exporter

This update enhances the export method of the GCS IO module, expanding its functionality to support exporting Polars DataFrames in addition to the previously supported Pandas DataFrames. This new capability allows users to seamlessly work with both data formats, offering greater flexibility in managing and exporting data from their workflows.

by @TalaatHasanin in [https://github.com/mage-ai/mage-ai/pull/5348](https://github.com/mage-ai/mage-ai/pull/5348)

### Add support to scheduler name on k8s executor

This update adds the ability to customize Kubernetes scheduler options when using the Kubernetes Executor. Users can now directly configure scheduler settings, providing increased flexibility and control over how pods are scheduled within Kubernetes environments.

by @messerzen in [https://github.com/mage-ai/mage-ai/pull/5412](https://github.com/mage-ai/mage-ai/pull/5412)

[​](#deadpool-%26-wolverine)

Deadpool & Wolverine

0.9.73

[Read full release notes](https://github.com/mage-ai/mage-ai/releases/tag/0.9.73)

### 🧠 Memory management upgrade

This feature enhances system performance and stability by optimizing allocation, deallocation, and overall utilization of RAM resources, resulting in improved efficiency and reduced memory-related errors.

Check out the [doc](https://www.notion.so/mageai/Memory-management-upgrades-e6c5e6e360ce410094091e00c46e3bb6?pvs=4)
 for details.

  

### 🔄 Dynamic blocks 2.0

Introducing an enhanced Dynamic blocks feature for creating dynamic content blocks that can adapt based on user input or data changes, offering improved flexibility and interactivity for users. This update aims to streamline workflows and enhance the overall user experience by making content more responsive and customizable.

Learn more in this [doc](https://www.notion.so/mageai/Dynamic-Blocks-2-0-d2fc11a4f48148d68b817e9bfb38a732)
.

### 🔐 Azure DB connection via Key Vault

Securely retrieve a database connection URL stored in Azure Key Vault using environment variables for authentication.

    AZURE_KEY_VAULT_URL
    AZURE_CLIENT_ID
    AZURE_CLIENT_SECRET
    AZURE_TENANT_ID
    

by [@wangxiaoyou1993](https://github.com/wangxiaoyou1993)
 in [#5302](https://github.com/mage-ai/mage-ai/pull/5302)

### 👁️ Workspace monitoring

Add an “Overview” page and “Pipeline runs” page to the Workspace Management UI. This provides some overall monitoring for all of the pipeline runs in the different workspaces without having to individually open up each workspace instance.

  

by @johnson-mage in [https://github.com/mage-ai/mage-ai/pull/5311](https://github.com/mage-ai/mage-ai/pull/5311)

[​](#house-targaryen)

House Targaryen

0.9.72

[Read full release notes](https://github.com/mage-ai/mage-ai/releases/tag/0.9.72)

[​](#x-men)

X-Men

0.9.71

[Read full release notes](https://github.com/mage-ai/mage-ai/releases/tag/0.9.71)

[​](#fallout)

Fallout

0.9.70

[Read full release notes](https://github.com/mage-ai/mage-ai/releases/tag/0.9.70)

[​](#invincible)

Invincible

0.9.68

[Read full release notes](https://github.com/mage-ai/mage-ai/releases/tag/0.9.68)

*   [0.9.66 | Shogun Release](https://github.com/mage-ai/mage-ai/releases/tag/0.9.66)
    
*   [0.9.65 | Demon Slayer Release](https://github.com/mage-ai/mage-ai/releases/tag/0.9.65)
    
*   [0.9.64 | Maestro Release](https://github.com/mage-ai/mage-ai/releases/tag/0.9.64)
    
*   [0.9.63 | Halo Release](https://github.com/mage-ai/mage-ai/releases/tag/0.9.63)
    
*   [0.9.62 | The Beekeeper Release](https://github.com/mage-ai/mage-ai/releases/tag/0.9.62)
    
*   [0.9.60 | Yusuke Urameshi Release](https://github.com/mage-ai/mage-ai/releases/tag/0.9.60)
    
*   [0.9.59 | Year of the Dragon Release](https://github.com/mage-ai/mage-ai/releases/tag/0.9.59)
    
*   [0.9.50 | Wonka Release](https://github.com/mage-ai/mage-ai/releases/tag/0.9.50)
    
*   [0.9.48 | The Boy and the Heron Release](https://github.com/mage-ai/mage-ai/releases/tag/0.9.48)
    
*   [0.9.46 | Wish Release](https://github.com/mage-ai/mage-ai/releases/tag/0.9.46)
    
*   [0.9.45 | Yuji Itadori Release](https://github.com/mage-ai/mage-ai/releases/tag/0.9.45)
    
*   [0.9.43 | Attack on Titan Release](https://github.com/mage-ai/mage-ai/releases/tag/0.9.43)
    
*   [0.9.41 | Halloween Release](https://github.com/mage-ai/mage-ai/releases/tag/0.9.41)
    
*   [0.9.38 | Goosebumps Release](https://github.com/mage-ai/mage-ai/releases/tag/0.9.38)
    
*   [0.9.35 | Loki Release](https://github.com/mage-ai/mage-ai/releases/tag/0.9.35)
    
*   [0.9.34 | C-3PO Release](https://github.com/mage-ai/mage-ai/releases/tag/0.9.34)
    
*   [0.9.30 | Cowboy Bebop Release](https://github.com/mage-ai/mage-ai/releases/tag/0.9.30)
    
*   [0.9.28 | The Creator Release](https://github.com/mage-ai/mage-ai/releases/tag/0.9.28)
    
*   [0.9.26 | Expend4bles Release](https://github.com/mage-ai/mage-ai/releases/tag/0.9.26)
    
*   [0.9.23 | One Piece Release](https://github.com/mage-ai/mage-ai/releases/tag/0.9.23)
    
*   [0.9.21 | Ahsoka Release](https://github.com/mage-ai/mage-ai/releases/tag/0.9.21)
    
*   [0.9.19 | The Equalizer Release](https://github.com/mage-ai/mage-ai/releases/tag/0.9.19)
    
*   [0.9.16 | Gran Turismo Release](https://github.com/mage-ai/mage-ai/releases/tag/0.9.16)
    
*   [0.9.14 | Blue Beetle Release](https://github.com/mage-ai/mage-ai/releases/tag/0.9.14)
    
*   [0.9.11 | Mutant Mayhem Release](https://github.com/mage-ai/mage-ai/releases/tag/0.9.11)
    
*   [0.9.10 | Haunted Mansion Release](https://github.com/mage-ai/mage-ai/releases/tag/0.9.10)
    
*   [0.9.80 | Dead Reckoning Release](https://github.com/mage-ai/mage-ai/releases/tag/0.9.8)
    
*   [0.9.40 | Solar Flare Release](https://github.com/mage-ai/mage-ai/releases/tag/0.9.4)
    
*   [0.9.00 | The Dial of Destiny Release](https://github.com/mage-ai/mage-ai/releases/tag/0.9.0)
    
*   [0.8.93 | Rise of the Beasts Release](https://github.com/mage-ai/mage-ai/releases/tag/0.8.93)
    
*   [0.8.86 | Fast Release](https://github.com/mage-ai/mage-ai/releases/tag/0.8.86)
    
*   [0.8.83 | Fury of the Gods Release](https://github.com/mage-ai/mage-ai/releases/tag/0.8.83)
    
*   [0.8.78 | Rise of the Machines Release](https://github.com/mage-ai/mage-ai/releases/tag/0.8.78)
    
*   [0.8.75 | Guardians Release](https://github.com/mage-ai/mage-ai/releases/tag/0.8.75)
    
*   [0.8.69 | Hairy Otter Release](https://github.com/mage-ai/mage-ai/releases/tag/0.8.69)
    
*   [0.8.58 | Once & Always Release](https://github.com/mage-ai/mage-ai/releases/tag/0.8.58)
    
*   [0.8.52 | The Super Mario Bros. Release](https://github.com/mage-ai/mage-ai/releases/tag/0.8.52)
    
*   [0.8.44 | Dungeons and Dragons Release](https://github.com/mage-ai/mage-ai/releases/tag/0.8.44)
    
*   [0.8.37 | Foundation Release](https://github.com/mage-ai/mage-ai/releases/tag/0.8.37)
    
*   [0.8.29 | Wick Release](https://github.com/mage-ai/mage-ai/releases/tag/0.8.29)
    
*   [0.8.27 | Shazam Release](https://github.com/mage-ai/mage-ai/releases/tag/0.8.27)
    
*   [0.8.24 | Merlin Release](https://github.com/mage-ai/mage-ai/releases/tag/0.8.24)
    
*   [0.8.15 | Creed Release](https://github.com/mage-ai/mage-ai/releases/tag/0.8.15)
    
*   [0.8.11 | The Mandalorian Release](https://github.com/mage-ai/mage-ai/releases/tag/0.8.11)
    
*   [0.8.3 | Everything Everywhere All at Once Release](https://github.com/mage-ai/mage-ai/releases/tag/0.8.3)
    
*   [0.7.98 | Quantumania Release](https://github.com/mage-ai/mage-ai/releases/tag/0.7.98)
    
*   [0.7.90 | That ’90s Show Release](https://github.com/mage-ai/mage-ai/releases/tag/0.7.90)
    
*   [0.7.84 | Rabbit Release](https://github.com/mage-ai/mage-ai/releases/tag/0.7.84)
    
*   [0.7.74 | Lunar Release](https://github.com/mage-ai/mage-ai/releases/tag/0.7.74)
    

Was this page helpful?

YesNo

[Suggest edits](https://github.com/mage-ai/mage-ai/edit/master/docs/about/releases.mdx)
[Raise issue](https://github.com/mage-ai/mage-ai/issues/new?title=Issue on docs&body=Path: /about/releases)

[FAQ](/about/frequently-asked-questions)

On this page

*   [Kaos](#kaos)
    
*   [Deadpool & Wolverine](#deadpool-%26-wolverine)
    
*   [House Targaryen](#house-targaryen)
    
*   [X-Men](#x-men)
    
*   [Fallout](#fallout)
    
*   [Invincible](#invincible)

---

# Using AI (artificial intelligence) - Mage AI

[Mage AI home page![light logo](https://mintlify.s3.us-west-1.amazonaws.com/mage/logo/m-mage-light.svg)![dark logo](https://mintlify.s3.us-west-1.amazonaws.com/mage/logo/m-mage.svg)](/)

Latest

Omniscience... ✨

Search...

Navigation

Development

Using AI (artificial intelligence)

[Documentation](/introduction/overview)
[Development](/guides/overview)
[Production](/production/ci-cd/overview)
[AI](/ai/sidekick/index)
[Observability](/observability/monitoring)
[Integrations](/integrations/compute/spark-pyspark)
[Extensibility](/extensibility/global-hooks/overview)

Mage uses AI to help you build data pipelines faster— with our OpenAI integration, you can automate the tedious parts of pipeline development and focus on the fun stuff.

You will need to add an OpenAI API key to your project before you can use AI for various actions.

We currently support the following AI features:

*   Generate pipelines
*   Generate blocks
*   Document code
    *   Block documentation
    *   Block comments
    *   Pipeline documentation
    *   Simultaneously document _all_ blocks in a pipeline

To see examples of how to use AI in Mage, check out our [AI guides](/guides/ai/overview)
.

[​](#setup)

Setup
--------------------

You need to add an OpenAI API key to your project before you can use AI for various actions.

Read [OpenAI’s documentation](https://help.openai.com/en/articles/4936850-where-do-i-find-my-secret-api-key)
 to get your API key.

Once you have your OpenAI API key, go to project settings (click the “wizard” in the top right > Workspace > Preferences) and enter the OpenAI API key under the section labeled **OpenAI**.

[​](#generate-pipeline-using-ai)

Generate pipeline using AI
--------------------------------------------------------------

When creating a new pipeline, select the option labeled **Using AI**.

Then, type the description of what the pipeline should do.

For example:

_Load data from an API, then clean the column names, and finally export the dataframe to PostgreSQL._

[​](#generate-block-using-ai)

Generate block using AI
--------------------------------------------------------

You must turn on the feature named `add_new_block_v2` in your project settings. (click the “wizard” in the top right > Workspace > Preferences)

When adding a new block, type in the description of what you want the block to do. In the autocomplete dropdown list, select the 1st option with a label starting with

**Generate block using AI: …**

[​](#document-code-using-ai)

Document code using AI
------------------------------------------------------

### 

[​](#add-documentation-for-a-block)

Add documentation for a block

On the edit pipeline page in the top right corner of a block, click the AI actions icon. Select the option labeled **Document block**.

### 

[​](#add-documentation-for-a-pipeline-and-all-its-blocks)

Add documentation for a pipeline and all its blocks

On the edit pipeline page in the top right corner of a block, click the AI actions icon. Select the option labeled **Document pipeline and all blocks**.

### 

[​](#add-comments-in-a-block)

Add comments in a block

On the edit pipeline page in the top right corner of a block, click the AI actions icon. Select the option labeled **Add comments in code**.

Was this page helpful?

YesNo

[Suggest edits](https://github.com/mage-ai/mage-ai/edit/master/docs/ai/setup.mdx)
[Raise issue](https://github.com/mage-ai/mage-ai/issues/new?title=Issue on docs&body=Path: /ai/setup)

[Introduction](/ai/sidekick/index)
[Clients](/ai/ai-client)

On this page

*   [Setup](#setup)
    
*   [Generate pipeline using AI](#generate-pipeline-using-ai)
    
*   [Generate block using AI](#generate-block-using-ai)
    
*   [Document code using AI](#document-code-using-ai)
    
*   [Add documentation for a block](#add-documentation-for-a-block)
    
*   [Add documentation for a pipeline and all its blocks](#add-documentation-for-a-pipeline-and-all-its-blocks)
    
*   [Add comments in a block](#add-comments-in-a-block)

---

# Help improve the tool - Mage AI

[Mage AI home page![light logo](https://mintlify.s3.us-west-1.amazonaws.com/mage/logo/m-mage-light.svg)![dark logo](https://mintlify.s3.us-west-1.amazonaws.com/mage/logo/m-mage.svg)](/)

Latest

Omniscience... ✨

Search...

Navigation

About

Help improve the tool

[Documentation](/introduction/overview)
[Development](/guides/overview)
[Production](/production/ci-cd/overview)
[AI](/ai/sidekick/index)
[Observability](/observability/monitoring)
[Integrations](/integrations/compute/spark-pyspark)
[Extensibility](/extensibility/global-hooks/overview)

All usage statistics are completely **anonymous**.

It’s **impossible** for Mage to know which statistics belongs to whom.

[​](#%F0%9F%99%8F-why-is-this-important%3F)

🙏 Why is this important?
------------------------------------------------------------------------

By opting into sending usage statistics to [Mage](https://www.mage.ai/)
, it’ll help the team and community of contributors (Magers) learn what’s going wrong with the tool and what improvements can be made.

In addition to helping reduce potential errors, you’ll help inform which features are useful and which need work.

[​](#%F0%9F%A4%94-what-usage-statistics-am-i-sending%3F)

🤔 What usage statistics am I sending?
--------------------------------------------------------------------------------------------------

### 

[​](#project-uuid)

Project UUID

Each project will have a universally unique identifier. This will help Mage count how many projects have been created.

It’s **impossible** to associate a UUID with a project without knowing the pair together.

Your project UUID is stored in the project’s `metadata.yaml` file, located at the root of your project: `[project_name]/metadata.yaml`.

Here is an example of what it could look like:

    variables_dir: ~/.mage_data
    
    # ...
    
    project_uuid: 4279d28ab1f64644b1f2f4f779be7b7e
    

### 

[​](#number-of-pipelines)

Number of pipelines

Sending usage statistics will include the number of pipelines in a single project. This will help improve the coding experience when building pipelines.

### 

[​](#number-of-pipeline-runs)

Number of pipeline runs

Sending usage statistics will include the number of times any pipeline has ran in a single project. This will help add better pipeline management features.

### 

[​](#number-of-users)

Number of users

Sending usage statistics will include the number of users in a single project. This will help improve the collaboration capabilities of the tool.

This usage statistic is only included if [user authentication](/production/authentication/overview)
 is enabled.

### 

[​](#errors)

Errors

When an application error occurs in Mage, the error type, error message, and offending line of code will be included in the usage statistics. This will help fix bugs and improve the developer experience.

### 

[​](#platform)

Platform

The operating system, release, version, etc of the machine that Mage is running on. This information will help reproduce errors.

[​](#%F0%9F%A4%B7%E2%80%8D%E2%99%80%EF%B8%8F-how-does-this-work%3F)

🤷‍♀️ How does this work?
------------------------------------------------------------------------------------------------

Usage statistics are anonymously sent to Mage’s online server.

Here’s a sample of the JSON payload containing usage statistics that could be sent:

    {
      "usage_statistic": {
        "project_uuid": "4279d28ab1f64644b1f2f4f779be7b7e",
        "pipelines": 40,
        "pipeline_runs": 357,
        "users": 13,
        "platform": "Linux-5.15.49-linuxkit-aarch64-with-glibc2.31",
        "version": "0.8.70",
        "error": {
          "message": "...",
          "traceback": "..."
        }
      }
    }
    

### 

[​](#enable)

Enable

To enable sending usage statistics, add a key in the project’s `metadata.yaml` file called `help_improve_mage` with the value `true`.

Here is an example:

    project_uuid: 4279d28ab1f64644b1f2f4f779be7b7e
    help_improve_mage: true
    

### 

[​](#disable)

Disable

To disable, change the value of `help_improve_mage` to `false`.

Here is an example:

    project_uuid: 4279d28ab1f64644b1f2f4f779be7b7e
    help_improve_mage: false
    

Was this page helpful?

YesNo

[Suggest edits](https://github.com/mage-ai/mage-ai/edit/master/docs/about/statistics.mdx)
[Raise issue](https://github.com/mage-ai/mage-ai/issues/new?title=Issue on docs&body=Path: /about/statistics)

[Code of conduct](/about/code-of-conduct)
[FAQ](/about/frequently-asked-questions)

On this page

*   [🙏 Why is this important?](#%F0%9F%99%8F-why-is-this-important%3F)
    
*   [🤔 What usage statistics am I sending?](#%F0%9F%A4%94-what-usage-statistics-am-i-sending%3F)
    
*   [Project UUID](#project-uuid)
    
*   [Number of pipelines](#number-of-pipelines)
    
*   [Number of pipeline runs](#number-of-pipeline-runs)
    
*   [Number of users](#number-of-users)
    
*   [Errors](#errors)
    
*   [Platform](#platform)
    
*   [🤷‍♀️ How does this work?](#%F0%9F%A4%B7%E2%80%8D%E2%99%80%EF%B8%8F-how-does-this-work%3F)
    
*   [Enable](#enable)
    
*   [Disable](#disable)

---

# Changelog - Mage AI

[Mage AI home page![light logo](https://mintlify.s3.us-west-1.amazonaws.com/mage/logo/m-mage-light.svg)![dark logo](https://mintlify.s3.us-west-1.amazonaws.com/mage/logo/m-mage.svg)](/)

Latest

Omniscience... ✨

Search...

Navigation

Changelog

[Documentation](/introduction/overview)
[Development](/guides/overview)
[Production](/production/ci-cd/overview)
[AI](/ai/sidekick/index)
[Observability](/observability/monitoring)
[Integrations](/integrations/compute/spark-pyspark)
[Extensibility](/extensibility/global-hooks/overview)

[Changelog\
---------\
\
Learn more about the latest features and improvements](https://github.com/mage-ai/mage-ai/releases)

Was this page helpful?

YesNo

[Suggest edits](https://github.com/mage-ai/mage-ai/edit/master/docs/about/changelog.mdx)
[Raise issue](https://github.com/mage-ai/mage-ai/issues/new?title=Issue on docs&body=Path: /about/changelog)

---

# Retrieval Augmented Generation (RAG) pipeline builder - Mage AI

[Mage AI home page![light logo](https://mintlify.s3.us-west-1.amazonaws.com/mage/logo/m-mage-light.svg)![dark logo](https://mintlify.s3.us-west-1.amazonaws.com/mage/logo/m-mage.svg)](/)

Latest

Omniscience... ✨

Search...

Navigation

Pipelines

Retrieval Augmented Generation (RAG) pipeline builder

[Documentation](/introduction/overview)
[Development](/guides/overview)
[Production](/production/ci-cd/overview)
[AI](/ai/sidekick/index)
[Observability](/observability/monitoring)
[Integrations](/integrations/compute/spark-pyspark)
[Extensibility](/extensibility/global-hooks/overview)

[](https://cloud.mage.ai/sign-up?source=rag-pipeline)

[Only in Mage Pro.\
\
Try our fully managed solution to access this advanced feature.](https://cloud.mage.ai/sign-up?source=rag-pipeline)
 

[](https://cloud.mage.ai/sign-up?source=rag-pipeline)

[](https://cloud.mage.ai/sign-up?source=rag-pipeline)
[Get started for free](https://cloud.mage.ai/sign-up?source=rag-pipeline)

Was this page helpful?

YesNo

[Suggest edits](https://github.com/mage-ai/mage-ai/edit/master/docs/ai/rag-pipeline.mdx)
[Raise issue](https://github.com/mage-ai/mage-ai/issues/new?title=Issue on docs&body=Path: /ai/rag-pipeline)

[Machine learning](/ai/ml/train-model)
[Resources](/ai/custom-resources)

---

# Customized AI resources for training and inference - Mage AI

[Mage AI home page![light logo](https://mintlify.s3.us-west-1.amazonaws.com/mage/logo/m-mage-light.svg)![dark logo](https://mintlify.s3.us-west-1.amazonaws.com/mage/logo/m-mage.svg)](/)

Latest

Omniscience... ✨

Search...

Navigation

Pipelines

Customized AI resources for training and inference

[Documentation](/introduction/overview)
[Development](/guides/overview)
[Production](/production/ci-cd/overview)
[AI](/ai/sidekick/index)
[Observability](/observability/monitoring)
[Integrations](/integrations/compute/spark-pyspark)
[Extensibility](/extensibility/global-hooks/overview)

[​](#custom-resources)

Custom resources
------------------------------------------

Customized GPU accelerated resources for running AI/ML/LLM pipelines.

[](https://cloud.mage.ai/sign-up?source=ai-resources)

[Only in Mage Pro.\
\
Try our fully managed solution to access this advanced feature.](https://cloud.mage.ai/sign-up?source=ai-resources)
 

[](https://cloud.mage.ai/sign-up?source=ai-resources)

[](https://cloud.mage.ai/sign-up?source=ai-resources)
[Get started for free](https://cloud.mage.ai/sign-up?source=ai-resources)

[​](#inference-endpoints)

Inference endpoints
------------------------------------------------

Deploy high-performance, low-latency API endpoints for executing blocks and returning output data, such as inference endpoints.

[](https://cloud.mage.ai/sign-up?source=ai-endpoints)

[Only in Mage Pro.\
\
Try our fully managed solution to access this advanced feature.](https://cloud.mage.ai/sign-up?source=ai-endpoints)
 

[](https://cloud.mage.ai/sign-up?source=ai-endpoints)

[](https://cloud.mage.ai/sign-up?source=ai-endpoints)
[Get started for free](https://cloud.mage.ai/sign-up?source=ai-endpoints)

Was this page helpful?

YesNo

[Suggest edits](https://github.com/mage-ai/mage-ai/edit/master/docs/ai/custom-resources.mdx)
[Raise issue](https://github.com/mage-ai/mage-ai/issues/new?title=Issue on docs&body=Path: /ai/custom-resources)

[RAG pipelines](/ai/rag-pipeline)

On this page

*   [Custom resources](#custom-resources)
    
*   [Inference endpoints](#inference-endpoints)

---

# Machine learning pipeline tutorial - Mage AI

[Mage AI home page![light logo](https://mintlify.s3.us-west-1.amazonaws.com/mage/logo/m-mage-light.svg)![dark logo](https://mintlify.s3.us-west-1.amazonaws.com/mage/logo/m-mage.svg)](/)

Latest

Omniscience... ✨

Search...

Navigation

Pipelines

Machine learning pipeline tutorial

[Documentation](/introduction/overview)
[Development](/guides/overview)
[Production](/production/ci-cd/overview)
[AI](/ai/sidekick/index)
[Observability](/observability/monitoring)
[Integrations](/integrations/compute/spark-pyspark)
[Extensibility](/extensibility/global-hooks/overview)

In this tutorial, we’ll create a pipeline that does the following:

1.  Load data from an online endpoint
2.  Select columns and fill in missing values
3.  Train a model to predict which passengers will survive

If you prefer to skip the tutorial and view the finished code, follow [this guide](/guides/train/complete-project)
.

If you haven’t setup a project before, check out the [setup guide](/getting-started/setup)
 before starting.

[​](#1-setup)

1\. Setup
--------------------------

### 

[​](#1a-add-python-packages-to-project)

1a. Add Python packages to project

In the left sidebar (aka file browser), click on the `requirements.txt` file under the `demo_project/` folder.

Then add the following dependencies to that file:

    matplotlib
    requests
    scikit-learn
    

Then, save the file by pressing `⌘ + S`.

### 

[​](#2a-install-dependencies)

2a. Install dependencies

The simplest way is to run pip install from the tool.

Add a scratchpad block by pressing the `+ Scratchpad` button. Then run the following command:

    pip install -r demo_project/requirements.txt
    

Alternatively, here are other ways of installing dependencies (depending on if you are using Docker or not):

#### 

[​](#docker)

Docker

Get the name of the container that is running the tool:

    docker ps
    

Sample output:

Sample output

    CONTAINER ID   IMAGE       COMMAND                  CREATED         STATUS         PORTS     NAMES
    214e1155f5c3   mage/data   "python mage_ai/comm…"   5 seconds ago   Up 2 seconds             mage-ai_server_run_6f8d367ac405
    

The container name in the above sample output is `mage-ai_server_run_6f8d367ac405`.

Then run this command to install Python packages in the `demo_project/requirements.txt` file:

    docker exec [container_name] pip3 install -r demo_project/requirements.txt
    

### 

[​](#pip)

pip

If you aren’t using Docker, just run the following command in your terminal:

    pip3 install -r demo_project/requirements.txt
    

[​](#2-create-new-pipeline)

2\. Create new pipeline
------------------------------------------------------

In the top left corner, click `File > New pipeline`. Then, click the name of the pipeline next to the green dot to rename it to `titanic survivors`.

[​](#3-play-around-with-scratchpad)

3\. Play around with scratchpad
----------------------------------------------------------------------

There are 4 buttons, click on the `+ Scratchpad` button to add a block.

Paste the following sample code in the block:

    import matplotlib.pyplot as plt
    import numpy as np
    
    
    t = np.arange(0.0, 2.0, 0.01)
    s = 1 + np.sin(2*np.pi*t)
    plt.plot(t, s)
    
    plt.xlabel('time (s)')
    plt.ylabel('voltage (mV)')
    plt.title('About as simple as it gets, folks')
    plt.grid(True)
    plt.show()
    

Then click the `Play button` on the right side of the block to run the code. Alternatively, you can use the following keyboard shortcuts to execute code in the block:

*   ⌘ + Enter
*   Control + Enter
*   Shift + Enter (run code and add a new block)

Now that we’re done with the scratchpad, we can leave it there or delete it. To delete a block, click the trash can icon on the right side or use the keyboard shortcut by typing the letter D and then D again.

[​](#4-load-data)

4\. Load data
----------------------------------

1.  Click the `+ Data loader` button, select `Python`, then click the template called `API`.
2.  Rename the block to `load dataset`.
3.  In the function named `load_data_from_api`, set the `url` variable to: `https://raw.githubusercontent.com/datasciencedojo/datasets/master/titanic.csv`.
4.  Run the block by clicking the play icon button or using the keyboard shortcuts `⌘ + Enter`, `Control + Enter`, or `Shift + Enter`.

After you run the block (⌘ + Enter), you can immediately see a sample of the data in the block’s output.

Here is what the code should look like:

    import io
    import pandas as pd
    import requests
    from pandas import DataFrame
    
    if 'data_loader' not in globals():
        from mage_ai.data_preparation.decorators import data_loader
    if 'test' not in globals():
        from mage_ai.data_preparation.decorators import test
    
    
    @data_loader
    def load_data_from_api(**kwargs) -> DataFrame:
        """
        Template for loading data from API
        """
        url = 'https://raw.githubusercontent.com/mage-ai/datasets/master/titanic_survival.csv'
    
        response = requests.get(url)
        return pd.read_csv(io.StringIO(response.text), sep=',')
    
    
    @test
    def test_output(df) -> None:
        """
        Template code for testing the output of the block.
        """
        assert df is not None, 'The output is undefined'
    

[​](#5-transform-data)

5\. Transform data
--------------------------------------------

We’re going to select numerical columns from the original dataset, then fill in missing values for those columns (aka impute).

1.  Click the `+ Transformer` button, select `Python`, then click `Generic (no template)`.
2.  Rename the block to `extract and impute numbers`.
3.  Paste the following code in the block:

    from pandas import DataFrame
    import math
    
    
    if 'transformer' not in globals():
        from mage_ai.data_preparation.decorators import transformer
    
    
    def select_number_columns(df: DataFrame) -> DataFrame:
        return df[['Age', 'Fare', 'Parch', 'Pclass', 'SibSp', 'Survived']]
    
    
    def fill_missing_values_with_median(df: DataFrame) -> DataFrame:
        for col in df.columns:
            values = sorted(df[col].dropna().tolist())
            median_age = values[math.floor(len(values) / 2)]
            df[[col]] = df[[col]].fillna(median_age)
        return df
    
    
    @transformer
    def transform_df(df: DataFrame, *args) -> DataFrame:
        return fill_missing_values_with_median(select_number_columns(df))
    

After you run the block (⌘ + Enter), you can immediately see a sample of the data in the block’s output.

[​](#6-train-model)

6\. Train model
--------------------------------------

In this part, we’re going to accomplish the following:

1.  Split the dataset into a training set and a test set.
2.  Train logistic regression model.
3.  Calculate the model’s accuracy score.
4.  Save the training set, test set, and model artifact to disk.

Here are the steps to take:

1.  Add a new data exporter block by clicking `+ Data exporter` button, select `Python`, then click `Generic (no template)`.
2.  Rename the block to `train model`.
3.  Paste the following code in the block:

    from pandas import DataFrame
    from sklearn.linear_model import LogisticRegression
    from sklearn.metrics import accuracy_score
    from sklearn.model_selection import train_test_split
    import os
    import pickle
    
    if 'data_exporter' not in globals():
        from mage_ai.data_preparation.decorators import data_exporter
    
    
    LABEL_COLUMN = 'Survived'
    
    
    def build_training_and_test_set(df: DataFrame) -> None:
        X = df.drop(columns=[LABEL_COLUMN])
        y = df[LABEL_COLUMN]
    
        return train_test_split(X, y)
    
    
    def train_model(X, y) -> None:
        model = LogisticRegression()
        model.fit(X, y)
    
        return model
    
    
    def score_model(model, X, y) -> None:
        y_pred = model.predict(X)
    
        return accuracy_score(y, y_pred)
    
    
    @data_exporter
    def export_data(df: DataFrame) -> None:
        X_train, X_test, y_train, y_test = build_training_and_test_set(df)
        model = train_model(X_train, y_train)
    
        score = score_model(model, X_test, y_test)
        print(f'Accuracy: {score}')
    
        cwd = os.getcwd()
        filename = f'{cwd}/finalized_model.lib'
        print(f'Saving model to {filename}')
        pickle.dump(model, open(filename, 'wb'))
    
        print(f'Saving training and test set')
        X_train.to_csv(f'{cwd}/X_train')
        X_test.to_csv(f'{cwd}/X_test')
        y_train.to_csv(f'{cwd}/y_train')
        y_test.to_csv(f'{cwd}/y_test')
    

Run the block (⌘ + Enter).

[​](#7-run-pipeline)

7\. Run pipeline
----------------------------------------

We can now run the entire pipeline end-to-end. In your terminal, execute the following command:

Docker

pip

    ./scripts/run.sh demo_project titanic_survivors
    

You can also run the pipeline from the UI. Click on the **Execute pipeline** from right bottom panel.

Your output should look something like this:

    Executing data_loader block: load_dataset...DONE
    Executing transformer block: extract_and_impute_numbers...DONE
    Executing data_exporter block: train_model...Accuracy: 0.757847533632287
    Saving model to /home/src/finalized_model.lib
    Saving training and test set
    DONE
    

* * *

[​](#congratulations)

Congratulations!
-----------------------------------------

You’ve successfully built an ML pipeline that consists of modular code blocks and is reproducible in any environment.

If you have more questions or ideas, please live chat with us in [Slack](https://www.mage.ai/chat)
.

Was this page helpful?

YesNo

[Suggest edits](https://github.com/mage-ai/mage-ai/edit/master/docs/ai/ml/train-model.mdx)
[Raise issue](https://github.com/mage-ai/mage-ai/issues/new?title=Issue on docs&body=Path: /ai/ml/train-model)

[Clients](/ai/ai-client)
[RAG pipelines](/ai/rag-pipeline)

On this page

*   [1\. Setup](#1-setup)
    
*   [1a. Add Python packages to project](#1a-add-python-packages-to-project)
    
*   [2a. Install dependencies](#2a-install-dependencies)
    
*   [Docker](#docker)
    
*   [pip](#pip)
    
*   [2\. Create new pipeline](#2-create-new-pipeline)
    
*   [3\. Play around with scratchpad](#3-play-around-with-scratchpad)
    
*   [4\. Load data](#4-load-data)
    
*   [5\. Transform data](#5-transform-data)
    
*   [6\. Train model](#6-train-model)
    
*   [7\. Run pipeline](#7-run-pipeline)
    
*   [Congratulations!](#congratulations)

---

# User defined permissions - Mage AI

[Mage AI home page![light logo](https://mintlify.s3.us-west-1.amazonaws.com/mage/logo/m-mage-light.svg)![dark logo](https://mintlify.s3.us-west-1.amazonaws.com/mage/logo/m-mage.svg)](/)

Latest

Omniscience... ✨

Search...

Navigation

User defined permissions

[Documentation](/introduction/overview)
[Development](/guides/overview)
[Production](/production/ci-cd/overview)
[AI](/ai/sidekick/index)
[Observability](/observability/monitoring)
[Integrations](/integrations/compute/spark-pyspark)
[Extensibility](/extensibility/global-hooks/overview)

  

Available in version **`0.9.35`** and greater.

[​](#overview)

Overview
--------------------------

Define roles with 1 or more permissions. Each permission can grant or deny read and write operations on specific resources (e.g. API endpoints). One or more roles can be assigned to 1 or multiple users.

### 

[​](#why-is-this-important%3F)

Why is this important?

Control which user can perform specific actions at a granular level.

* * *

[​](#features)

Features
--------------------------

*   Define roles.
*   Define permissions.
*   Add permissions to 1 or more roles.
*   Add roles to 1 or more users.

* * *

[​](#how-to-use)

How to use
------------------------------

### 

[​](#turn-on-and-require-user-permissions)

Turn on and require user permissions

You must have [User authentication](/production/authentication/overview)
 enabled.

Set the environment variable named `REQUIRE_USER_PERMISSIONS` to the value `1`.

You may need to restart your Mage application for the environment variable to be updated and for the application to enable user permissions.

### 

[​](#create-roles)

Create roles

You must be the owner or have a role that grants permissions to read and write on the following entities:

*   `Permission`
*   `Role`
*   `RolePermission`
*   `UserRole`
*   `User`

1.  Go the the Mage project settings and click on the navigation row labeled **Roles**.
    
2.  On the roles list page, there is a button labeled **Create default roles and permissions**. If you click this button, 6 roles and a few hundred permissions will be created.
    
    The roles are:
    
    *   Admin default permissions
    *   Editor default permissions
    *   Editor with notebook edit access
    *   Editor with pipeline edit access
    *   Owner default permissions
    *   Viewer default permissions
    
    Each role will have a set of permissions that match the Mage API policies that are normally used to authenticate operations when the user authentication feature is turned on but the user permissions feature is turned off.
    
3.  Alternatively, click the button **Add new role** to create a single new role.
    
4.  Enter in a unique name for the role.
    
5.  Click the button **Create new role** to save the role.
    
6.  Once you create the role, you’ll be taken to the role detail page.
    
7.  From here, you can add existing permissions to the role.
    
8.  From here, you can add existing users to the role. This will grant permissions or deny operations for user.
    

### 

[​](#create-permissions)

Create permissions

1.  Go the the Mage project settings and click on the navigation row labeled **Permissions**.
    
2.  Click the button **Add new permission** to create a new permission.
    
3.  Select an entity that these permissions will be applied to. An entity refers to a defined API endpoint that currently exists in the Mage application.
    
    For example, the entity `Pipeline` applies to the API endpoints `/pipelines`.
    
4.  Optionally enter an entity UUID that these permissions are applied to. For example, you can grant or deny permissions to a specific pipeline by selecting the entity `Pipeline` and entering the entity `UUID` `example_pipeline`. This permission will only be applied to the pipeline with the UUID `example_pipeline`.
    
5.  Under the section labeled **Access**, toggle 1 or more accesses to grant. For more information on what each access grants or denies, read the [permissions documentation](/authentication/permissions/permissions)
    .
    
6.  Click the button **Create new permission** to save the permission.
    
7.  Once you create the role, you’ll be taken to the role detail page.
    
8.  From here, you can edit the permission.
    
9.  From here, you can add this permission to existing roles.
    

### 

[​](#assign-roles-to-users)

Assign roles to users

1.  Go the the Mage project settings and click on the navigation row labeled **Users**.
2.  Click on a row for an existing user.
3.  From here, you can assign existing roles to the user.
4.  Alternatively, you can add a user to a role by following these steps:
    1.  Go to the [roles list page](http://localhost:6789/settings/workspace/roles)
        .
    2.  Click on a row for an existing role.
    3.  Click the button labeled **+Add user**.
    4.  Select 1 or more users to assign this role to.

Was this page helpful?

YesNo

[Suggest edits](https://github.com/mage-ai/mage-ai/edit/master/docs/authentication/permissions/overview.mdx)
[Raise issue](https://github.com/mage-ai/mage-ai/issues/new?title=Issue on docs&body=Path: /authentication/permissions/overview)

On this page

*   [Overview](#overview)
    
*   [Why is this important?](#why-is-this-important%3F)
    
*   [Features](#features)
    
*   [How to use](#how-to-use)
    
*   [Turn on and require user permissions](#turn-on-and-require-user-permissions)
    
*   [Create roles](#create-roles)
    
*   [Create permissions](#create-permissions)
    
*   [Assign roles to users](#assign-roles-to-users)

---

# LinkedIn - Mage AI

[Mage AI home page![light logo](https://mintlify.s3.us-west-1.amazonaws.com/mage/logo/m-mage-light.svg)![dark logo](https://mintlify.s3.us-west-1.amazonaws.com/mage/logo/m-mage.svg)](/)

Latest

Omniscience... ✨

Search...

Navigation

LinkedIn

[Documentation](/introduction/overview)
[Development](/guides/overview)
[Production](/production/ci-cd/overview)
[AI](/ai/sidekick/index)
[Observability](/observability/monitoring)
[Integrations](/integrations/compute/spark-pyspark)
[Extensibility](/extensibility/global-hooks/overview)

[LinkedIn\
--------\
\
Let’s connect professionally](https://www.linkedin.com/company/magetech)

Was this page helpful?

YesNo

[Suggest edits](https://github.com/mage-ai/mage-ai/edit/master/docs/community/linkedin.mdx)
[Raise issue](https://github.com/mage-ai/mage-ai/issues/new?title=Issue on docs&body=Path: /community/linkedin)

---

# Twitter - Mage AI

[Mage AI home page![light logo](https://mintlify.s3.us-west-1.amazonaws.com/mage/logo/m-mage-light.svg)![dark logo](https://mintlify.s3.us-west-1.amazonaws.com/mage/logo/m-mage.svg)](/)

Latest

Omniscience... ✨

Search...

Navigation

Twitter

[Documentation](/introduction/overview)
[Development](/guides/overview)
[Production](/production/ci-cd/overview)
[AI](/ai/sidekick/index)
[Observability](/observability/monitoring)
[Integrations](/integrations/compute/spark-pyspark)
[Extensibility](/extensibility/global-hooks/overview)

[Twitter\
-------\
\
Ready for funny memes?](https://twitter.com/mage_ai)

Was this page helpful?

YesNo

[Suggest edits](https://github.com/mage-ai/mage-ai/edit/master/docs/community/twitter.mdx)
[Raise issue](https://github.com/mage-ai/mage-ai/issues/new?title=Issue on docs&body=Path: /community/twitter)

---

# Roles - Mage AI

[Mage AI home page![light logo](https://mintlify.s3.us-west-1.amazonaws.com/mage/logo/m-mage-light.svg)![dark logo](https://mintlify.s3.us-west-1.amazonaws.com/mage/logo/m-mage.svg)](/)

Latest

Omniscience... ✨

Search...

Navigation

Roles

[Documentation](/introduction/overview)
[Development](/guides/overview)
[Production](/production/ci-cd/overview)
[AI](/ai/sidekick/index)
[Observability](/observability/monitoring)
[Integrations](/integrations/compute/spark-pyspark)
[Extensibility](/extensibility/global-hooks/overview)

[​](#overview)

Overview
--------------------------

*   A role contains a set of 1 or more permissions.
*   A user can have 1 or more roles.
*   Each role must have a unique name across the entire project.

Was this page helpful?

YesNo

[Suggest edits](https://github.com/mage-ai/mage-ai/edit/master/docs/authentication/permissions/roles.mdx)
[Raise issue](https://github.com/mage-ai/mage-ai/issues/new?title=Issue on docs&body=Path: /authentication/permissions/roles)

On this page

*   [Overview](#overview)

---

# Your AI data engineer - Mage AI

[Mage AI home page![light logo](https://mintlify.s3.us-west-1.amazonaws.com/mage/logo/m-mage-light.svg)![dark logo](https://mintlify.s3.us-west-1.amazonaws.com/mage/logo/m-mage.svg)](/)

Latest

Omniscience... ✨

Search...

Navigation

AI Sidekick

Your AI data engineer

[Documentation](/introduction/overview)
[Development](/guides/overview)
[Production](/production/ci-cd/overview)
[AI](/ai/sidekick/index)
[Observability](/observability/monitoring)
[Integrations](/integrations/compute/spark-pyspark)
[Extensibility](/extensibility/global-hooks/overview)

1.  Create, design, and assemble end-to-end pipelines
2.  Generate code blocks
3.  Fix code bugs, troubleshoot execution errors, and resolve production issues
4.  Run code blocks with synthetic data
5.  Write detailed documentation for your data products

[](https://cloud.mage.ai/sign-up?source=pipeline-dependencies)

[Only in Mage Pro.\
\
Try our fully managed solution to access this advanced feature.](https://cloud.mage.ai/sign-up?source=pipeline-dependencies)
 

[](https://cloud.mage.ai/sign-up?source=pipeline-dependencies)

[](https://cloud.mage.ai/sign-up?source=pipeline-dependencies)
[Get started for free](https://cloud.mage.ai/sign-up?source=pipeline-dependencies)

[​](#your-sidekick)

Your sidekick
------------------------------------

Build pipelines

Write code

Fix errors

* * *

[​](#coming-soon)

Coming soon
--------------------------------

### 

[​](#your-coding-thought-partner)

Your coding thought partner

1.  Optimize your data pipelines for performance
2.  Refactor your code for readability and maintainability
3.  Suggest solutions while you’re writing code

### 

[​](#your-data-oracle)

Your data oracle

1.  Recommend and answer questions related to data engineering and best practices
2.  Provide insights on your past, present, and future data creation and performance trends
3.  Automatically create monitoring dashboards and visualization charts

Was this page helpful?

YesNo

[Suggest edits](https://github.com/mage-ai/mage-ai/edit/master/docs/ai/sidekick/index.mdx)
[Raise issue](https://github.com/mage-ai/mage-ai/issues/new?title=Issue on docs&body=Path: /ai/sidekick/index)

[Getting started](/ai/setup)

On this page

*   [Your sidekick](#your-sidekick)
    
*   [Coming soon](#coming-soon)
    
*   [Your coding thought partner](#your-coding-thought-partner)
    
*   [Your data oracle](#your-data-oracle)

---

# Adding an IO class - Mage AI

[Mage AI home page![light logo](https://mintlify.s3.us-west-1.amazonaws.com/mage/logo/m-mage-light.svg)![dark logo](https://mintlify.s3.us-west-1.amazonaws.com/mage/logo/m-mage.svg)](/)

Latest

Omniscience... ✨

Search...

Navigation

IO classes

Adding an IO class

[Documentation](/introduction/overview)
[Development](/guides/overview)
[Production](/production/ci-cd/overview)
[AI](/ai/sidekick/index)
[Observability](/observability/monitoring)
[Integrations](/integrations/compute/spark-pyspark)
[Extensibility](/extensibility/global-hooks/overview)

[​](#what%E2%80%99s-an-io-class%3F)

What’s an IO class?
----------------------------------------------------------

IO classes are at the heart of Mage— they’re the code that enables Mage to read and write data. When you configure your [`io_config`](https://docs.mage.ai/development/io_config)
 to use a source or destination, you’re telling Mage to use a specific IO class.

IO classes are defined [here](https://github.com/mage-ai/mage-ai/tree/master/mage_ai/io)
 in the Mage repository. You can see that there are a few different types of IO classes— some are for reading data, some are for writing data, and some are for both.

[​](#why-should-i-contribute%3F)

Why should I contribute?
------------------------------------------------------------

If you have a favorite IO class _that’s not currently supported_, you can reach out _or_ contribute it to Mage! Contributing is a great way to build your skills, become a part of the Mage community, and help others.

[​](#how-do-i-contribute%3F)

How do I contribute?
----------------------------------------------------

### 

[​](#configure-your-development-environment)

Configure your development environment

The first step to contributing an IO class is to configure your development environment. You can find instructions for doing so [here](https://docs.mage.ai/contributing/development-environment)
.

### 

[​](#create-a-new-io-class)

Create a new IO class

Once configured, you’ll want to create a new file in the `mage_ai/io` directory. The file should be named after the IO class you’re contributing. For example, if you’re contributing an IO class called `MyIOClass`, you should create a file called `my_io_class.py`.

Some IO classes are relatively straightforward, like [exporting to local files](https://github.com/mage-ai/mage-ai/blob/master/mage_ai/io/file.py)
 or [writing to Google Sheets](https://github.com/mage-ai/mage-ai/blob/master/mage_ai/io/google_sheets.py)
. Others, like full database integrations, can be pretty complex. [Google BigQuery](https://github.com/mage-ai/mage-ai/blob/master/mage_ai/io/bigquery.py)
 is a good example of a complex IO class.

Most classes are platform specific, but for a database, you might need the following methods:

*   `alter_table`: Alter the table schema
*   `load`: Load data from the database
*   `export`: Export data to the database
*   `execute`: Execute a query on the database
    *   `execute_queries`: Execute multiple queries on the database

Additionally, every class will need to have the following methods:

*   `__init__`: Initialize the class
*   `with_config`: Initialize the database client from the configuration loader. You’ll notice that this method is used in _all_ of our templates.

### 

[​](#test-your-io-class)

Test your IO class

In order to test your class, you should perform all of the methods you defined in the class itself. Once you pass our linting checks (outlined in the development environment link above) _and_ your tests pass, you’re ready to submit a pull request! Learn more about pull requests on the contribution.

[​](#create-a-pull-request)

Create a pull request
----------------------------------------------------

1

Fork the Mage GitHub repo

Visit [GitHub](https://github.com/mage-ai/mage-ai)
 and fork the Mage repo, then clone it to your machine.

2

Create a branch

Create a branch in your forked repo and commit your changes.

3

Create a pull request

Once you've [made and tested your changes](/contributing/development-environment)
, create a [pull request](https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/proposing-changes-to-your-work-with-pull-requests/creating-a-pull-request-from-a-fork)
 on the Mage repo.

4

Tag Mage team members for a review

Add members for review in GitHub, please tag one of the following:

*   `@wangxiaoyou1993` (backend/IO)
*   `@johnson-mage` (frontend/docs/website)
*   `@tommydangerous`

Was this page helpful?

YesNo

[Suggest edits](https://github.com/mage-ai/mage-ai/edit/master/docs/contributing/backend/io/adding-a-class.mdx)
[Raise issue](https://github.com/mage-ai/mage-ai/issues/new?title=Issue on docs&body=Path: /contributing/backend/io/adding-a-class)

[Create a new destination](/contributing/data-integrations/add-new-destination)
[Contributing](/contributing/backend/streaming/sources-and-destinations)

On this page

*   [What’s an IO class?](#what%E2%80%99s-an-io-class%3F)
    
*   [Why should I contribute?](#why-should-i-contribute%3F)
    
*   [How do I contribute?](#how-do-i-contribute%3F)
    
*   [Configure your development environment](#configure-your-development-environment)
    
*   [Create a new IO class](#create-a-new-io-class)
    
*   [Test your IO class](#test-your-io-class)
    
*   [Create a pull request](#create-a-pull-request)

---

# Slack - Mage AI

[Mage AI home page![light logo](https://mintlify.s3.us-west-1.amazonaws.com/mage/logo/m-mage-light.svg)![dark logo](https://mintlify.s3.us-west-1.amazonaws.com/mage/logo/m-mage.svg)](/)

Latest

Omniscience... ✨

Search...

Navigation

Slack

[Documentation](/introduction/overview)
[Development](/guides/overview)
[Production](/production/ci-cd/overview)
[AI](/ai/sidekick/index)
[Observability](/observability/monitoring)
[Integrations](/integrations/compute/spark-pyspark)
[Extensibility](/extensibility/global-hooks/overview)

[Slack\
-----\
\
Instantly chat with the team and other community members](https://www.mage.ai/chat)

Was this page helpful?

YesNo

[Suggest edits](https://github.com/mage-ai/mage-ai/edit/master/docs/community/slack.mdx)
[Raise issue](https://github.com/mage-ai/mage-ai/issues/new?title=Issue on docs&body=Path: /community/slack)

---

# Blog - Mage AI

[Mage AI home page![light logo](https://mintlify.s3.us-west-1.amazonaws.com/mage/logo/m-mage-light.svg)![dark logo](https://mintlify.s3.us-west-1.amazonaws.com/mage/logo/m-mage.svg)](/)

Latest

Omniscience... ✨

Search...

Navigation

Blog

[Documentation](/introduction/overview)
[Development](/guides/overview)
[Production](/production/ci-cd/overview)
[AI](/ai/sidekick/index)
[Observability](/observability/monitoring)
[Integrations](/integrations/compute/spark-pyspark)
[Extensibility](/extensibility/global-hooks/overview)

[Blog\
----\
\
Want to be entertained?](https://www.mage.ai/blog)

Was this page helpful?

YesNo

[Suggest edits](https://github.com/mage-ai/mage-ai/edit/master/docs/community/blog.mdx)
[Raise issue](https://github.com/mage-ai/mage-ai/issues/new?title=Issue on docs&body=Path: /community/blog)

---