# Table of Contents
- [Magical Data Engineering Workflows | Mage AI](#magical-data-engineering-workflows-mage-ai)
- [Your AI Data Engineer | Intelligent Data Engineering Platform | Mage AI](#your-ai-data-engineer-intelligent-data-engineering-platform-mage-ai)
- [Get Demo | Customized for Your Needs | Mage AI](#get-demo-customized-for-your-needs-mage-ai)
- [Deploy | Flexible Deployment Options | Mage AI](#deploy-flexible-deployment-options-mage-ai)
- [Data | Intelligent, Scalable Data Pipeline Solutions | Mage AI](#data-intelligent-scalable-data-pipeline-solutions-mage-ai)
- [Build | Create Reusable Data Pipelines in Half the Time | Mage AI](#build-create-reusable-data-pipelines-in-half-the-time-mage-ai)
- [Extend | Unlock Flexible Extensibility | Mage AI](#extend-unlock-flexible-extensibility-mage-ai)
- [Scale | Intelligent Data Pipeline Scaling | Mage AI](#scale-intelligent-data-pipeline-scaling-mage-ai)
- [GitHub - mage-ai/mage-ai: š§ Build, run, and manage data pipelines for integrating and transforming data.](#github-mage-ai-mage-ai-build-run-and-manage-data-pipelines-for-integrating-and-transforming-data-)
- [Terms of Service | AI Data Engineering Platform | Mage AI](#terms-of-service-ai-data-engineering-platform-mage-ai)
- [Privacy Policy | Data Protection and Security | Mage AI](#privacy-policy-data-protection-and-security-mage-ai)
- [Innovations | Unique Differentiators | Mage AI](#innovations-unique-differentiators-mage-ai)
- [Pricing | Find a Mage Pro Plan | Mage AI](#pricing-find-a-mage-pro-plan-mage-ai)
- [Slack](#slack)
- [Data Pipelines & Templates | AI-Powered Solutions | Mage AI](#data-pipelines-templates-ai-powered-solutions-mage-ai)
- [Updates | Changelog and Release Notes | Mage AI](#updates-changelog-and-release-notes-mage-ai)
- [Blog | Data Engineering Hub | Mage AI](#blog-data-engineering-hub-mage-ai)
- [Partners Program | Collaborate and Earn | Mage AI](#partners-program-collaborate-and-earn-mage-ai)
- [Sign in to GitHub Ā· GitHub](#sign-in-to-github-github)
- [GitHub Ā· Build and ship software on a single, collaborative platform Ā· GitHub](#github-build-and-ship-software-on-a-single-collaborative-platform-github)
- [Platform Specs | AI Data Engineering Features | Mage AI](#platform-specs-ai-data-engineering-features-mage-ai)
---
# Magical Data Engineering Workflows | Mage AI
Your AI data engineer
=====================
Your AI data engineer
=====================
Your AI
data engineer
=======================
Build, deploy, and run data pipelines
-------------------------------------
through an intuitive interface in minutes.
------------------------------------------
Run at any scale instantly with Mage Pro.
-----------------------------------------
Build, deploy, and run data pipelines through an intuitive interface in minutes. Run at any scale instantly with Mage Pro.
Build, deploy, and run data pipelines through an intuitive interface in minutes. Run at any scale instantly with Mage Pro.
[Start building](https://cloud.mage.ai/sign-up)
[Get a demo](./getdemo)
Now with Sonnet 3.7 and GPT-4.5 powering your pipelines\*
Now with Sonnet 3.7 and GPT-4.5 powering your pipelines\*
[](./#highlights)
[](./#highlights)
[Move data from Google cloud to our analytics platform.\
\
\
\
Remove any duplicate cells you find.\
\
Done. Hereās your pipeline.\
\
### Intelligent data engineering\
\
### Intelligent data engineering\
\
Write production-ready code faster with AI that understands your data. Get instant debugging, best practice recommendations, and automated data engineering tasks - all within your workflow.\
\
Write production-ready code faster with AI that understands your data. Get instant debugging, best practice recommendations, and automated data engineering tasks - all within your workflow.\
\
Write production-ready code faster with AI that understands your data. Get instant debugging, best practice recommendations, and automated data engineering tasks - all within your workflow.\
\
#### Learn more](./ai)
[Running\
\
In\
\
Out\
\
Retriever.py\
\
1\
\
2\
\
3\
\
4\
\
5\
\
6\
\
7\
\
from dense\_retriever import DenseRetriever\
\
\
\
class CustomRetriever(DenseRetriever):\
\
def retrieve(self, query, documents):\
\
\# Custom retrieval logic\
\
custom\_results = super().retrieve(query, documents)\
\
return custom\_results\
\
### One platform for all data pipelines\
\
### One platform for all data pipelines\
\
Build any pipeline type - batch, streaming, ML, RAG - in our interactive environment. Seamlessly combine Python, SQL, R, and dbt with instant previews at every step.\
\
Build any pipeline type - batch, streaming, ML, RAG - in our interactive environment. Seamlessly combine Python, SQL, R, and dbt with instant previews at every step.\
\
Build any pipeline type - batch, streaming, ML, RAG - in our interactive environment. Seamlessly combine Python, SQL, R, and dbt with instant previews at every step.\
\
#### Learn more](./data)
[\
\
40\
\
Template\
\
\
\
8\
\
Workflow\
\
\
\
100\
\
File\
\
\
\
6\
\
Pipeline\
\
\
\
40\
\
Team\
\
Import\
\
### Build reusable pipelines in half the time\
\
### Build reusable pipelines in half the time\
\
Mage breaks down complex data engineering tasks into smaller, manageable units called blocks. Each block in a pipeline is a standalone file that can be executed independently.\
\
Mage breaks down complex data engineering tasks into smaller, manageable units called blocks. Each block in a pipeline is a standalone file that can be executed independently.\
\
Mage breaks down complex data engineering tasks into smaller, manageable units called blocks. Each block in a pipeline is a standalone file that can be executed independently.\
\
#### Learn more](./build)
[Deploy\
\
Live\
\
main\
\
b83bf2 Optimized pipeline\
\
Deploy\
\
main\
\
4h34xt Added dynamic blocks\
\
### Your DevOps command center\
\
### Your DevOps command center\
\
Launch environments, collaborate in team workspaces, and deploy continuously with zero friction. Keep your pipelines organized and production-ready without the DevOps complexity.\
\
Launch environments, collaborate in team workspaces, and deploy continuously with zero friction. Keep your pipelines organized and production-ready without the DevOps complexity.\
\
Launch environments, collaborate in team workspaces, and deploy continuously with zero friction. Keep your pipelines organized and production-ready without the DevOps complexity.\
\
#### Learn more](./deploy)
[Architecture\
\
\
\
\
\
### Your personal data infrastructure team\
\
### Your personal data infrastructure team\
\
Scale from terabytes to thousands of concurrent tasks automatically. Mageās intelligent architecture handles your most complex workloads without the operational overhead.\
\
Scale from terabytes to thousands of concurrent tasks automatically. Mageās intelligent architecture handles your most complex workloads without the operational overhead.\
\
Scale from terabytes to thousands of concurrent tasks automatically. Mageās intelligent architecture handles your most complex workloads without the operational overhead.\
\
#### Learn more](./scale)
[Engineering\
\
Design\
\
Marketing\
\
Builder\
\
EDITOR\
\
VIEWER\
\
\-\
\
Comments\
\
EDITOR\
\
EDITOR\
\
\-\
\
Triggers\
\
ADMIN\
\
\-\
\
VIEWER\
\
Monitoring\
\
EDITOR\
\
VIEWER\
\
\-\
\
Dashboard\
\
VIEWER\
\
VIEWER\
\
VIEWER\
\
Alerts\
\
EDITOR\
\
\-\
\
\-\
\
### Architectural playground for data teams\
\
### Architectural playground for data teams\
\
Customize Mage AI to your exact operational DNA through surgical-grade extensibility features ā 100+ API endpoints, application behavioral hooks, and granular security permissions.\
\
Customize Mage AI to your exact operational DNA through surgical-grade extensibility features ā 100+ API endpoints, application behavioral hooks, and granular security permissions.\
\
Customize Mage AI to your exact operational DNA through surgical-grade extensibility features ā 100+ API endpoints, application behavioral hooks, and granular security permissions.\
\
#### Learn more](./extend)
Batch
Integration
Streaming
RAG
Spark
ML
Batch
Integration
Streaming
RAG
Spark
ML
Familiar patterns.
Powerful new framework.
Build. Collaborate. Launch. Analyze. Win.
Filter
SQL
Transform along the way using
Py, R, or Sql
Amazon S3
SQL
Combine multiple sources
at different stages
Google Cloud
PY
Source data from anywhere
Aggregate
R
Refine and enhance
modularly and nondestructively
Daily Analytics
PY
Deliver exactly when and
where it is needed
Sensor
PY
Detect and react to data changes
You
Thomas
Sarah
### Available in two spellbinding versions
### Available in two
spellbinding versions
### **For teams.** Fully managed platform for integrating and transforming data.
### **For teams.** Fully managed platform for integrating and transforming data.
### **For teams.** Fully managed platform for integrating and transforming data.
[Try Pro for free](https://cloud.mage.ai/sign-up)
OSS
### **Self hosted.** System to build, run, and manage data pipelines.
### **Self hosted.** System to build, run, and manage data pipelines.
### **Self hosted.** System to build, run, and manage data pipelines.
[Try OSS](https://www.mage.ai/oss)
Ā© 2025 Mage Technologies, Inc.
[Terms of Service](./terms)
[Privacy Policy](./privacy)
Ā© 2025 Mage Technologies, Inc.
[Terms of Service](./terms)
[Privacy Policy](./privacy)
Ā© 2025 Mage Technologies, Inc.
[Terms of Service](./terms)
[Privacy Policy](./privacy)
[](./)
Platform
Explore
Pricing
[Community](https://www.mage.ai/chat)
[Get a demo](./getdemo)
[Start now](https://cloud.mage.ai/sign-up)
[](./)
[](./)
---
# Your AI Data Engineer | Intelligent Data Engineering Platform | Mage AI
[](./)
Platform
Explore
Pricing
[Community](https://www.mage.ai/chat)
[Get a demo](./getdemo)
[Start now](https://cloud.mage.ai/sign-up)
[](./)
AI
AI
Mage AI automates repetitive coding, debugs issues, and stays up 24/7 as your permanent on-call so that you can sleep easy.
Mage AI automates repetitive coding, debugs issues, and stays up 24/7 as your permanent on-call so that you can sleep easy.
[Design](./ai#design)
[Code](./ai#code)
[Debug](./ai#debug)
[Answer](./ai#answer)
Design
Ship pipelines
at the speed of thought
Ship pipelines
at the speed of thought
Transform your ideas into battle-tested pipelines using AI that understands data engineering nuance.
Transform your ideas into battle-tested pipelines using AI that understands data engineering nuance.
From initial concept to production in minutes. Your 30-second prompt just automated 4 hours of work.
From initial concept to production in minutes. Your 30-second prompt just automated 4 hours of work.
[Try it yourself](https://mage.ai/sign-up)
Pick the perfect AI for the job. From writing code to debugging issues, choose from a variety of models.
Pick the perfect AI for the job. From writing code to debugging issues, choose from a variety of models.
[Try it yourself](https://mage.ai/sign-up)
Code
Build like a 10x engineer,
without the burnout
Build like a 10x engineer,
without the burnout
AI that anticipates roadblocks so you stay in flow state.
AI that anticipates roadblocks so you stay in flow state.
Code that teaches itself. Generate code blocks, AI-powered insights on best practices, performance improvements, and potential bottlenecks.
Code that teaches itself. Generate code blocks, AI-powered insights on best practices, performance improvements, and potential bottlenecks.
[Try it yourself](https://mage.ai/sign-up)
Data-aware autocomplete. Our AI has full context of your upstream code blocks and the data your code is expecting as input parameters.
Data-aware autocomplete. Our AI has full context of your upstream code blocks and the data your code is expecting as input parameters.
[Try it yourself](https://mage.ai/sign-up)
Inline code assistance. Never lose focus again by using Mageās AI wherever youāre coding.
Inline code assistance. Never lose focus again by using Mageās AI wherever youāre coding.
[Try it yourself](https://mage.ai/sign-up)
Debug
Resolve production
issues in your sleep
Resolve production
issues in your sleep
AI that squashes bugs, writes tests, and documents code 24/7 so you donāt have to.
AI that squashes bugs, writes tests, and documents code 24/7 so you donāt have to.
Proactive debugging. Fewer production incidents with AI that audits code and data patterns.
Proactive debugging. Fewer production incidents with AI that audits code and data patterns.
[Try it yourself](https://mage.ai/sign-up)
Increase uptime. Mage AI investigates errors, automatically debugs pipelines, and fixes broken code.
Increase uptime. Mage AI investigates errors, automatically debugs pipelines, and fixes broken code.
[Try it yourself](https://mage.ai/sign-up)
Answer
Answer every secret
to data engineering
From idempotent loading, incremental logic, to compliance guardrails, your personal data architect at your fingertips.
Answer
Answer every secret
to data engineering
From idempotent loading, incremental logic, to compliance guardrails, your personal data architect at your fingertips.
Youāre the data engineering lead now. Leverage AI trained on best practices and ready to help.
Youāre the data engineering lead now. Leverage AI trained on best practices and ready to help.
[Try it yourself](https://mage.ai/sign-up)
Youāre the analytics engineering lead now. Automatically write queries and analyze data.
Youāre the analytics engineering lead now. Automatically write queries and analyze data.
[Try it yourself](https://mage.ai/sign-up)
Recover your precious developer time
Now you can focus on the fun, creative, and high-impact data engineering projects and let Mage AI handle the rest.
For engineers
Experience how Mage AI ships data pipelines faster, giving you a better work-life-balance.
[Get started](https://mage.ai/sign-up)
For data teams
See how Mage AI accelerates your team velocity while reducing data and infrastructure costs.
[Enterprise demo](./getdemo)
[Explore the platform](./build)
[\
\
AI\
\
Code faster, smarter](./ai)
[\
\
Data pipelines\
\
Build, run, and monitor](./data)
[\
\
Dev Ex\
\
Made for speed and ease](./build)
[\
\
Deployments\
\
Ship workflows effortlessly](./deploy)
[\
\
Scalabiity\
\
Grow without limits](./scale)
[\
\
Customizations\
\
Total extensibility](./extend)
Recover your precious developer time
Now you can focus on the fun, creative, and high-impact data engineering projects and let Mage AI handle the rest.
For engineers
Experience how Mage AI ships data pipelines faster, giving you a better work-life-balance.
[Get started](https://mage.ai/sign-up)
For data teams
See how Mage AI accelerates your team velocity while reducing data and infrastructure costs.
[Enterprise demo](./getdemo)
[Explore the platform](./build)
[\
\
AI\
\
Code faster, smarter](./ai)
[\
\
Data pipelines\
\
Build, run, and monitor](./data)
[\
\
Dev Ex\
\
Made for speed and ease](./build)
[\
\
Deployments\
\
Ship workflows effortlessly](./deploy)
[\
\
Scalabiity\
\
Grow without limits](./scale)
[\
\
Customizations\
\
Total extensibility](./extend)
How do I activate the Mage AI on my cluster?
How is our AI different from other AI tools?
How is my data used?
Is there a free trial?
How can I opt-out of using the Mage AI or disable specific models/LLMs?
How do I activate the Mage AI on my cluster?
How is our AI different from other AI tools?
How is my data used?
Is there a free trial?
How can I opt-out of using the Mage AI or disable specific models/LLMs?
Your AI data engineer
[Get started](https://mage.ai/sign-up)
[Request a demo](./getdemo)
Product
[AI](./ai)
[Data pipelines](./data)
[Developer experience](./build)
[Deployments](./deploy)
[Scalability](./scale)
[Customizations](./extend)
Resources
[Pricing](./pricing)
[Documentation](https://docs.mage.ai/)
[Blog](./blog)
[Platform features](./specs)
[Unique differentiators](./innovations)
[Mage Pro vs OSS](./oss-vs-pro)
[Changelog](./updates)
Getting started
[Quickstart](https://docs.mage.ai/getting-started/setup)
[Guides](https://docs.mage.ai/guides/overview)
[Templates](./commons)
[Core abstractions](https://docs.mage.ai/design/core-abstractions)
[Dynamic blocks](https://docs.mage.ai/design/blocks/dynamic-blocks)
Tools
[Help & support](https://www.mage.ai/chat)
[Migration support](./support-plan)
[Community](https://www.mage.ai/chat)
[Partner program](./partners)
Company
[Terms](./terms)
[Privacy Policy](./privacy)
Ā© 2025 Mage Technologies, Inc.
Your AI data engineer
[Get started](https://mage.ai/sign-up)
[Request a demo](./getdemo)
Product
[AI](./ai)
[Data pipelines](./data)
[Developer experience](./build)
[Deployments](./deploy)
[Scalability](./scale)
[Customizations](./extend)
Resources
[Pricing](./pricing)
[Documentation](https://docs.mage.ai/)
[Blog](./blog)
[Platform features](./specs)
[Unique differentiators](./innovations)
[Mage Pro vs OSS](./oss-vs-pro)
[Changelog](./updates)
Getting started
[Quickstart](https://docs.mage.ai/getting-started/setup)
[Guides](https://docs.mage.ai/guides/overview)
[Templates](./commons)
[Core abstractions](https://docs.mage.ai/design/core-abstractions)
[Dynamic blocks](https://docs.mage.ai/design/blocks/dynamic-blocks)
Tools
[Help & support](https://www.mage.ai/chat)
[Migration support](./support-plan)
[Community](https://www.mage.ai/chat)
[Partner program](./partners)
Company
[Terms](./terms)
[Privacy Policy](./privacy)
Ā© 2025 Mage Technologies, Inc.
---
# Get Demo | Customized for Your Needs | Mage AI
[](./)
Platform
Explore
Pricing
[Community](https://www.mage.ai/chat)
[Get a demo](./getdemo)
[Start now](https://cloud.mage.ai/sign-up)
[](./)
Get a 15-Min Live Demo
Discover how Mage transforms data workflows ā all while saving you time and money. Chat with our devs, ask anything.
[AI](./ai)
Your AI Data Engineer. Mage AI automates repetitive coding, debugs issues, and stays up 24/7 as your permanent on-call so that you can sleep easy.
Your AI Data Engineer. Mage AI automates repetitive coding, debugs issues, and stays up 24/7 as your permanent on-call so that you can sleep easy.
[Data](./data)
Batch it. Sync it. Stream it. Build resilient, scalable data pipelines for batch processing, data integrations, and real-time streaming.
Batch it. Sync it. Stream it. Build resilient, scalable data pipelines for batch processing, data integrations, and real-time streaming.
[Build](./build)
Workspaces for data engineers. Launch individual coding workspaces for each developer with isolated file system and dedicated resources.
Workspaces for data engineers. Launch individual coding workspaces for each developer with isolated file system and dedicated resources.
[Deploy](./deploy)
Deploy code, deliver results. Ship code snapshots from any branchāno merge conflicts, no deployment drama.
Deploy code, deliver results. Ship code snapshots from any branchāno merge conflicts, no deployment drama.
[Scale](./scale)
Never overpay for unused infr. Mage intelligently scales data pipelines in real-time ā while maintaining peak performance and reducing costs by up to 40%.
Never overpay for unused infr. Mage intelligently scales data pipelines in real-time ā while maintaining peak performance and reducing costs by up to 40%.
Schedule
Email \*
Name \*
Role \*
Selectā¦Developer / EngineerData Scientist / AnalystExecutive / Decision MakerOperations / DevOpsConsultant / FreelancerOther
Company
How can we help?
Select Date & Time
Book directly on our [calendar](https://calendar.google.com/calendar/u/0/appointments/schedules/AcZssZ3_iR1lmvsmrp1pWtUWC98izwbcnUuGpZYI7KDjvpcXkGFOgiUi-hTeXJSc_iPmCWJmmsXCsAKv)
.šļø
Email \*
Name \*
Role \*
Selectā¦Developer / EngineerData Scientist / AnalystExecutive / Decision MakerOperations / DevOpsConsultant / FreelancerOther
Company
How can we help?
Select Date & Time
Book directly on our [calendar](https://calendar.google.com/calendar/u/0/appointments/schedules/AcZssZ3_iR1lmvsmrp1pWtUWC98izwbcnUuGpZYI7KDjvpcXkGFOgiUi-hTeXJSc_iPmCWJmmsXCsAKv)
.šļø
Get a 15-Min Live Demo
Discover how Mage transforms data workflows ā all while saving you time and money. Chat with our devs, ask anything.
Your AI data engineer
[Get started](https://mage.ai/sign-up)
[Request a demo](./getdemo)
Product
[AI](./ai)
[Data pipelines](./data)
[Developer experience](./build)
[Deployments](./deploy)
[Scalability](./scale)
[Customizations](./extend)
Resources
[Pricing](./pricing)
[Documentation](https://docs.mage.ai/)
[Blog](./blog)
[Platform features](./specs)
[Unique differentiators](./innovations)
[Mage Pro vs OSS](./oss-vs-pro)
[Changelog](./updates)
Getting started
[Quickstart](https://docs.mage.ai/getting-started/setup)
[Guides](https://docs.mage.ai/guides/overview)
[Templates](./commons)
[Core abstractions](https://docs.mage.ai/design/core-abstractions)
[Dynamic blocks](https://docs.mage.ai/design/blocks/dynamic-blocks)
Tools
[Help & support](https://www.mage.ai/chat)
[Migration support](./support-plan)
[Community](https://www.mage.ai/chat)
[Partner program](./partners)
Company
[Terms](./terms)
[Privacy Policy](./privacy)
Ā© 2025 Mage Technologies, Inc.
Your AI data engineer
[Get started](https://mage.ai/sign-up)
[Request a demo](./getdemo)
Product
[AI](./ai)
[Data pipelines](./data)
[Developer experience](./build)
[Deployments](./deploy)
[Scalability](./scale)
[Customizations](./extend)
Resources
[Pricing](./pricing)
[Documentation](https://docs.mage.ai/)
[Blog](./blog)
[Platform features](./specs)
[Unique differentiators](./innovations)
[Mage Pro vs OSS](./oss-vs-pro)
[Changelog](./updates)
Getting started
[Quickstart](https://docs.mage.ai/getting-started/setup)
[Guides](https://docs.mage.ai/guides/overview)
[Templates](./commons)
[Core abstractions](https://docs.mage.ai/design/core-abstractions)
[Dynamic blocks](https://docs.mage.ai/design/blocks/dynamic-blocks)
Tools
[Help & support](https://www.mage.ai/chat)
[Migration support](./support-plan)
[Community](https://www.mage.ai/chat)
[Partner program](./partners)
Company
[Terms](./terms)
[Privacy Policy](./privacy)
Ā© 2025 Mage Technologies, Inc.
[Schedule Demo](./getdemo#demo)
---
# Deploy | Flexible Deployment Options | Mage AI
Ready, Set, Ship
Ready, Set, Ship
Deliver to prod in a snap ā no merge conflicts, no deployment drama.
Deliver to prod in a snap ā no merge conflicts, no deployment drama.
[Ship](./deploy#ship)
[Trigger](./deploy#trigger)
[Monitor](./deploy#monitor)
[Reproduce](./deploy#reproduce)
[AI](./deploy#ai)
Ship
Deploy code, deliver results
Deploy code, deliver results
Ship code snapshots from any branchāno merge conflicts, no deployment drama.
Ship code snapshots from any branchāno merge conflicts, no deployment drama.
Stage deployments. Promote changes through dev, staging, and production environments with granular control.
Stage deployments. Promote changes through dev, staging, and production environments with granular control.
Undo disasters in 1 click. Roll back broken deployments instantly while Mage preserves your last stable configuration.
Undo disasters in 1 click. Roll back broken deployments instantly while Mage preserves your last stable configuration.
[Try it yourself](https://mage.ai/sign-up)
Orchestrate
Orchestrate data on your timetable
Orchestrate data on your timetable
Run data processes with surgical precision, trigger from anywhere. AttachĀ multiple triggersĀ to a single pipeline, each with unique schedules and runtime parameter ā code once, run limitless.
Run data processes with surgical precision, trigger from anywhere. AttachĀ multiple triggersĀ to a single pipeline, each with unique schedules and runtime parameter ā code once, run limitless.
API and event based triggers. Execute pipelines through REST endpoints, Webhooks, or streaming events.
API and event based triggers. Execute pipelines through REST endpoints, Webhooks, or streaming events.
Landing times. Define when your pipelineĀ must finishĀ ā Mage automatically back-calculates start times using historical performance data.
Landing times. Define when your pipelineĀ must finishĀ ā Mage automatically back-calculates start times using historical performance data.
[Try it yourself](https://mage.ai/sign-up)
React in
real-time
React in
real-time
Monitor
Observe the past, present, and future
Observe the past, present, and future
Tailored insights from milliseconds to metrics ā see everything, miss nothing.
Tailored insights from milliseconds to metrics ā see everything, miss nothing.
Metrics that matter. Assemble bespoke monitoring boards that blend real-time pipeline metrics, block-level diagnostics, and business KPIs.
Metrics that matter. Assemble bespoke monitoring boards that blend real-time pipeline metrics, block-level diagnostics, and business KPIs.
Detailed structured logging. Every pipeline run generates a forensic-grade audit trail with logs stamped at millisecond resolution, capturing packages, OS versions, infrastructure tags, commits, parameter overrides, and more.
Detailed structured logging. Every pipeline run generates a forensic-grade audit trail with logs stamped at millisecond resolution, capturing packages, OS versions, infrastructure tags, commits, parameter overrides, and more.
Personalized team alerts. Configure surgical alert rules with conditional logic, template messaging, and multi-platform routing. Silence noise with metric-based thresholds while ensuring critical failures scream through email, Slack, Teams, Pager Duty, and more.
Personalized team alerts. Configure surgical alert rules with conditional logic, template messaging, and multi-platform routing. Silence noise with metric-based thresholds while ensuring critical failures scream through email, Slack, Teams, Pager Duty, and more.
[Try it yourself](https://mage.ai/sign-up)
Reproduce
Reproduce history, rewrite futures
Reproduce history, rewrite futures
Re-run pipelines using historical partitions or recreate datasets from any point in time.
Re-run pipelines using historical partitions or recreate datasets from any point in time.
Fix fast, recover faster. Resume workflows from failure points instead of restarting entire processes.
Fix fast, recover faster. Resume workflows from failure points instead of restarting entire processes.
Backfill without bottlenecks. Process years of data in parallel clusters or trickle-feed intervals with custom pacing.
Backfill without bottlenecks. Process years of data in parallel clusters or trickle-feed intervals with custom pacing.
[Try it yourself](https://mage.ai/sign-up)
AI
Let AI handle it
Let AI handle it
Deliver production-ready code with AI that squashes bugs, writes tests, and documents code 24/7 so you donāt have to.
Deliver production-ready code with AI that squashes bugs, writes tests, and documents code 24/7 so you donāt have to.
With proactive debugging jump ahead of production incidents with AI that audits code and data patterns.
[Explore AI features](./ai)
Recover your precious developer time
Now you can focus on the fun, creative, and high-impact data engineering projects and let Mage AI handle the rest.
For engineers
Experience how Mage AI ships data pipelines faster, giving you a better work-life-balance.
[Get started](https://mage.ai/sign-up)
For data teams
See how Mage AI accelerates your team velocity while reducing data and infrastructure costs.
[Enterprise demo](./getdemo)
[Explore the platform](./build)
[\
\
AI\
\
Code faster, smarter](./ai)
[\
\
Data pipelines\
\
Build, run, and monitor](./data)
[\
\
Dev Ex\
\
Made for speed and ease](./build)
[\
\
Deployments\
\
Ship workflows effortlessly](./deploy)
[\
\
Scalabiity\
\
Grow without limits](./scale)
[\
\
Customizations\
\
Total extensibility](./extend)
Recover your precious developer time
Now you can focus on the fun, creative, and high-impact data engineering projects and let Mage AI handle the rest.
For engineers
Experience how Mage AI ships data pipelines faster, giving you a better work-life-balance.
[Get started](https://mage.ai/sign-up)
For data teams
See how Mage AI accelerates your team velocity while reducing data and infrastructure costs.
[Enterprise demo](./getdemo)
[Explore the platform](./build)
[\
\
AI\
\
Code faster, smarter](./ai)
[\
\
Data pipelines\
\
Build, run, and monitor](./data)
[\
\
Dev Ex\
\
Made for speed and ease](./build)
[\
\
Deployments\
\
Ship workflows effortlessly](./deploy)
[\
\
Scalabiity\
\
Grow without limits](./scale)
[\
\
Customizations\
\
Total extensibility](./extend)
How do I activate the Mage AI on my cluster?
How is our AI different from other AI tools?
How is my data used?
Is there a free trial?
How can I opt-out of using the Mage AI or disable specific models/LLMs?
How do I activate the Mage AI on my cluster?
How is our AI different from other AI tools?
How is my data used?
Is there a free trial?
How can I opt-out of using the Mage AI or disable specific models/LLMs?
Your AI data engineer
[Get started](https://mage.ai/sign-up)
[Request a demo](./getdemo)
Product
[AI](./ai)
[Data pipelines](./data)
[Developer experience](./build)
[Deployments](./deploy)
[Scalability](./scale)
[Customizations](./extend)
Resources
[Pricing](./pricing)
[Documentation](https://docs.mage.ai/)
[Blog](./blog)
[Platform features](./specs)
[Unique differentiators](./innovations)
[Mage Pro vs OSS](./oss-vs-pro)
[Changelog](./updates)
Getting started
[Quickstart](https://docs.mage.ai/getting-started/setup)
[Guides](https://docs.mage.ai/guides/overview)
[Templates](./commons)
[Core abstractions](https://docs.mage.ai/design/core-abstractions)
[Dynamic blocks](https://docs.mage.ai/design/blocks/dynamic-blocks)
Tools
[Help & support](https://www.mage.ai/chat)
[Migration support](./support-plan)
[Community](https://www.mage.ai/chat)
[Partner program](./partners)
Company
[Terms](./terms)
[Privacy Policy](./privacy)
Ā© 2025 Mage Technologies, Inc.
Your AI data engineer
[Get started](https://mage.ai/sign-up)
[Request a demo](./getdemo)
Product
[AI](./ai)
[Data pipelines](./data)
[Developer experience](./build)
[Deployments](./deploy)
[Scalability](./scale)
[Customizations](./extend)
Resources
[Pricing](./pricing)
[Documentation](https://docs.mage.ai/)
[Blog](./blog)
[Platform features](./specs)
[Unique differentiators](./innovations)
[Mage Pro vs OSS](./oss-vs-pro)
[Changelog](./updates)
Getting started
[Quickstart](https://docs.mage.ai/getting-started/setup)
[Guides](https://docs.mage.ai/guides/overview)
[Templates](./commons)
[Core abstractions](https://docs.mage.ai/design/core-abstractions)
[Dynamic blocks](https://docs.mage.ai/design/blocks/dynamic-blocks)
Tools
[Help & support](https://www.mage.ai/chat)
[Migration support](./support-plan)
[Community](https://www.mage.ai/chat)
[Partner program](./partners)
Company
[Terms](./terms)
[Privacy Policy](./privacy)
Ā© 2025 Mage Technologies, Inc.
[](./)
Platform
Explore
Pricing
[Community](https://www.mage.ai/chat)
[Get a demo](./getdemo)
[Start now](https://cloud.mage.ai/sign-up)
[](./)
---
# Data | Intelligent, Scalable Data Pipeline Solutions | Mage AI
Moves you
Moves you
Transforms complex data workflows into seamless, automated processes ā so you can focus on outcomes, not infrastructure.
Transforms complex data workflows into seamless, automated processes ā so you can focus on outcomes, not infrastructure.
[Batch](./data#batch)
[Sync](./data#sync)
[Stream](./data#stream)
[Models](./data#models)
Batch
Batch it
Batch it
Transform your ideas into battle-tested pipelines using AI that understands data engineering nuance.
Transform your ideas into battle-tested pipelines using AI that understands data engineering nuance.
Flexible data orchestration. Run pipelines on your terms. Schedule them to execute at specific intervals or trigger them dynamically based on events, webhooks, API requests, or even other pipelines. Adapt to real-time demands and ensure your workflows stay in sync with business needs.
Flexible data orchestration. Run pipelines on your terms. Schedule them to execute at specific intervals or trigger them dynamically based on events, webhooks, API requests, or even other pipelines. Adapt to real-time demands and ensure your workflows stay in sync with business needs.
Dynamic runtime settings. Unlock the power of a single pipeline to handle hundreds of variations. With dynamic runtime settings, you can adjust parameters on the flyāperfect for processing different datasets or configurations without duplicating effort. Scale smarter and streamline operations with unparalleled efficiency.
Dynamic runtime settings. Unlock the power of a single pipeline to handle hundreds of variations. With dynamic runtime settings, you can adjust parameters on the flyāperfect for processing different datasets or configurations without duplicating effort. Scale smarter and streamline operations with unparalleled efficiency.
[Try it yourself](https://mage.ai/sign-up)
Sync
Sync it
Sync it
Effortlessly integrate and synchronize data across hundreds of sources and destinationsāwithout writing a single line of code. With Mageās pre-built integrations and automated workflows, you can build robust data pipelines that are as flexible as they are powerful. Keep your data fresh, accurate, and ready for action.
Effortlessly integrate and synchronize data across hundreds of sources and destinationsāwithout writing a single line of code. With Mageās pre-built integrations and automated workflows, you can build robust data pipelines that are as flexible as they are powerful. Keep your data fresh, accurate, and ready for action.
Composable, low-code data integrations. Connect to virtually any data platform in minutes using Mageās intuitive interfaceāno coding required.
Composable, low-code data integrations. Connect to virtually any data platform in minutes using Mageās intuitive interfaceāno coding required.
Sync and save costs. Save money by avoiding charges based on the number of rows synced or models builtāpay only for the value you create, not busy work.
Sync and save costs. Save money by avoiding charges based on the number of rows synced or models builtāpay only for the value you create, not busy work.
[Try it yourself](https://mage.ai/sign-up)
Stream
Stream it
Stream it
Mageās real-time streaming pipelines make ingesting, transforming, and delivering live events seamless and 95% faster. Designed for performance, reliability, and simplicity, these pipelines empower you to handle high-throughput streams with easeākeeping you ahead in a millisecond-driven world.
Mageās real-time streaming pipelines make ingesting, transforming, and delivering live events seamless and 95% faster. Designed for performance, reliability, and simplicity, these pipelines empower you to handle high-throughput streams with easeākeeping you ahead in a millisecond-driven world.
Zero setup, Zero maintenance. Say goodbye to infrastructure headaches. Mageās fully managed streaming pipelines require no setup or ongoing maintenanceājust configure your sources and sinks, and let Mage handle the rest.
Zero setup, Zero maintenance. Say goodbye to infrastructure headaches. Mageās fully managed streaming pipelines require no setup or ongoing maintenanceājust configure your sources and sinks, and let Mage handle the rest.
Never lose critical data. With built-in replay capabilities, you can reprocess events as neededāwhether for debugging, analytics, or complianceāwithout disrupting your pipeline.
Never lose critical data. With built-in replay capabilities, you can reprocess events as neededāwhether for debugging, analytics, or complianceāwithout disrupting your pipeline.
[Try it yourself](https://mage.ai/sign-up)
Models
Limitless data models without breaking the bank
Limitless data models without breaking the bank
Build, run, and manage data models from a single platform. Forge canonical data products using SQL, Python, or dbt ā then deploy them as reusable blueprints.
Build, run, and manage data models from a single platform. Forge canonical data products using SQL, Python, or dbt ā then deploy them as reusable blueprints.
Dynamic SQL. Inject upstream data, runtime parameters, environment variables, secrets, and macros.
Dynamic SQL. Inject upstream data, runtime parameters, environment variables, secrets, and macros.
Data model catalog. Publish models as versioned artifacts for cross-team reuse.
Data model catalog. Publish models as versioned artifacts for cross-team reuse.
[Try it yourself](https://mage.ai/sign-up)
Recover your precious developer time
Now you can focus on the fun, creative, and high-impact data engineering projects and let Mage AI handle the rest.
For engineers
Experience how Mage AI ships data pipelines faster, giving you a better work-life-balance.
[Get started](https://mage.ai/sign-up)
For data teams
See how Mage AI accelerates your team velocity while reducing data and infrastructure costs.
[Enterprise demo](./getdemo)
[Explore the platform](./build)
[\
\
AI\
\
Code faster, smarter](./ai)
[\
\
Data pipelines\
\
Build, run, and monitor](./data)
[\
\
Dev Ex\
\
Made for speed and ease](./build)
[\
\
Deployments\
\
Ship workflows effortlessly](./deploy)
[\
\
Scalabiity\
\
Grow without limits](./scale)
[\
\
Customizations\
\
Total extensibility](./extend)
Recover your precious developer time
Now you can focus on the fun, creative, and high-impact data engineering projects and let Mage AI handle the rest.
For engineers
Experience how Mage AI ships data pipelines faster, giving you a better work-life-balance.
[Get started](https://mage.ai/sign-up)
For data teams
See how Mage AI accelerates your team velocity while reducing data and infrastructure costs.
[Enterprise demo](./getdemo)
[Explore the platform](./build)
[\
\
AI\
\
Code faster, smarter](./ai)
[\
\
Data pipelines\
\
Build, run, and monitor](./data)
[\
\
Dev Ex\
\
Made for speed and ease](./build)
[\
\
Deployments\
\
Ship workflows effortlessly](./deploy)
[\
\
Scalabiity\
\
Grow without limits](./scale)
[\
\
Customizations\
\
Total extensibility](./extend)
Your AI data engineer
[Get started](https://mage.ai/sign-up)
[Request a demo](./getdemo)
Product
[AI](./ai)
[Data pipelines](./data)
[Developer experience](./build)
[Deployments](./deploy)
[Scalability](./scale)
[Customizations](./extend)
Resources
[Pricing](./pricing)
[Documentation](https://docs.mage.ai/)
[Blog](./blog)
[Platform features](./specs)
[Unique differentiators](./innovations)
[Mage Pro vs OSS](./oss-vs-pro)
[Changelog](./updates)
Getting started
[Quickstart](https://docs.mage.ai/getting-started/setup)
[Guides](https://docs.mage.ai/guides/overview)
[Templates](./commons)
[Core abstractions](https://docs.mage.ai/design/core-abstractions)
[Dynamic blocks](https://docs.mage.ai/design/blocks/dynamic-blocks)
Tools
[Help & support](https://www.mage.ai/chat)
[Migration support](./support-plan)
[Community](https://www.mage.ai/chat)
[Partner program](./partners)
Company
[Terms](./terms)
[Privacy Policy](./privacy)
Ā© 2025 Mage Technologies, Inc.
Your AI data engineer
[Get started](https://mage.ai/sign-up)
[Request a demo](./getdemo)
Product
[AI](./ai)
[Data pipelines](./data)
[Developer experience](./build)
[Deployments](./deploy)
[Scalability](./scale)
[Customizations](./extend)
Resources
[Pricing](./pricing)
[Documentation](https://docs.mage.ai/)
[Blog](./blog)
[Platform features](./specs)
[Unique differentiators](./innovations)
[Mage Pro vs OSS](./oss-vs-pro)
[Changelog](./updates)
Getting started
[Quickstart](https://docs.mage.ai/getting-started/setup)
[Guides](https://docs.mage.ai/guides/overview)
[Templates](./commons)
[Core abstractions](https://docs.mage.ai/design/core-abstractions)
[Dynamic blocks](https://docs.mage.ai/design/blocks/dynamic-blocks)
Tools
[Help & support](https://www.mage.ai/chat)
[Migration support](./support-plan)
[Community](https://www.mage.ai/chat)
[Partner program](./partners)
Company
[Terms](./terms)
[Privacy Policy](./privacy)
Ā© 2025 Mage Technologies, Inc.
[](./)
Platform
Explore
Pricing
[Community](https://www.mage.ai/chat)
[Get a demo](./getdemo)
[Start now](https://cloud.mage.ai/sign-up)
[](./)
---
# Build | Create Reusable Data Pipelines in Half the Time | Mage AI
def happier\_dev():
def happier\_dev():
Thousands of innovations that improve all aspects of the developer experience - speed, quality, joy, and beyond.
Thousands of innovations that improve all aspects of the developer experience - speed, quality, joy, and beyond.
[Compose](./build#compose)
[Preview](./build#preview)
[Test](./build#test)
[Collaborate](./build#collab)
[Enjoy](./build#enjoy)
Compose
Build reusable pipelines in half the time
Build reusable pipelines in half the time
Mage AIās modular blocks act like data engineering Legosāsnap components together to create production-grade workflows.
Mage AIās modular blocks act like data engineering Legosāsnap components together to create production-grade workflows.
Composable architecture. Each block is a standalone file that can be shared and executed independently, accelerating development time by reusing code across different pipelines.
[Try it yourself](https://mage.ai/sign-up)
Preview
Code with live data previews
Code with live data previews
Mageās data pipeline builder shows real-time code execution results ā no more ārun and prayā development.
Mageās data pipeline builder shows real-time code execution results ā no more ārun and prayā development.
Interactive coding experience. This real-time feedback loop accelerates the development process by enabling quick iterations and debugging without the need to switch contexts or navigate through different interfaces.
[Try it yourself](https://mage.ai/sign-up)
Test
Catch bugs
before they crash prod
Catch bugs before they crash prod
Mage bakes reliability and testing into every block with battle-tested workflows.
Mage bakes reliability and testing into every block with battle-tested workflows.
Test-driven development simplified. Write multiple tests directly in blocks to validate data during development and in production.
Test-driven development simplified. Write multiple tests directly in blocks to validate data during development and in production.
Proactive defense system. Failed tests block pipeline execution to prevent broken data flows.
Proactive defense system. Failed tests block pipeline execution to prevent broken data flows.
Reusable validation tests. Write a suite of data quality checks once and apply them to multiple blocks across different pipelines.
Reusable validation tests. Write a suite of data quality checks once and apply them to multiple blocks across different pipelines.
[Try it yourself](https://mage.ai/sign-up)
Collaborate
Collaborate and win together
Collaborate and win together
Data engineering is now more productive, collaborative, and fun.
Data engineering is now more productive, collaborative, and fun.
Version control. Native Git terminal handles commits, rebases, merges and moreĀ without breaking pipeline states.
Version control. Native Git terminal handles commits, rebases, merges and moreĀ without breaking pipeline states.
[Try it yourself](https://mage.ai/sign-up)
Pair debugging. Investigate issues and squash bugs with your teammates.
Pair debugging. Investigate issues and squash bugs with your teammates.
[Try it yourself](https://mage.ai/sign-up)
Workspaces for data engineers. Launch individual coding workspaces for each developer with isolated file system and dedicated resources.
Workspaces for data engineers. Launch individual coding workspaces for each developer with isolated file system and dedicated resources.
[Try it yourself](https://mage.ai/sign-up)
Enjoy
Find your flow state
Find your flow state
Experience development as it should beāfun, smooth, and satisfying. Our tools remove friction, so you can focus on what you love: building great things.
Experience development as it should beāfun, smooth, and satisfying. Our tools remove friction, so you can focus on what you love: building great things.
Integrate your own VSCode without friction. Get instant cloud scale without leaving your local VSCode. Bridge local development and production execution without complex setups.
Integrate your own VSCode without friction. Get instant cloud scale without leaving your local VSCode. Bridge local development and production execution without complex setups.
[Try it yourself](https://mage.ai/sign-up)
Safeguard every code change. Mage automatically tracks every edit, config update, and pipeline change, no Git expertise required.
Safeguard every code change. Mage automatically tracks every edit, config update, and pipeline change, no Git expertise required.
[Try it yourself](https://mage.ai/sign-up)
Code that teaches itself. Generate code blocks, AI-powered insights on best practices, performance improvements, and potential bottlenecks.
Code that teaches itself. Generate code blocks, AI-powered insights on best practices, performance improvements, and potential bottlenecks.
[Try it yourself](https://mage.ai/sign-up)
Data-aware autocomplete. Our AI has full context of your upstream code blocks and the data your code is expecting as input parameters.
Data-aware autocomplete. Our AI has full context of your upstream code blocks and the data your code is expecting as input parameters.
[Try it yourself](https://mage.ai/sign-up)
Inline code assistance. Never lose focus again by using Mageās AI wherever youāre coding.
Inline code assistance. Never lose focus again by using Mageās AI wherever youāre coding.
[Try it yourself](https://mage.ai/sign-up)
Recover your precious developer time
Now you can focus on the fun, creative, and high-impact data engineering projects and let Mage AI handle the rest.
For engineers
Experience how Mage AI ships data pipelines faster, giving you a better work-life-balance.
[Get started](https://mage.ai/sign-up)
For data teams
See how Mage AI accelerates your team velocity while reducing data and infrastructure costs.
[Enterprise demo](./getdemo)
[Explore the platform](./build)
[\
\
AI\
\
Code faster, smarter](./ai)
[\
\
Data pipelines\
\
Build, run, and monitor](./data)
[\
\
Dev Ex\
\
Made for speed and ease](./build)
[\
\
Deployments\
\
Ship workflows effortlessly](./deploy)
[\
\
Scalabiity\
\
Grow without limits](./scale)
[\
\
Customizations\
\
Total extensibility](./extend)
Recover your precious developer time
Now you can focus on the fun, creative, and high-impact data engineering projects and let Mage AI handle the rest.
For engineers
Experience how Mage AI ships data pipelines faster, giving you a better work-life-balance.
[Get started](https://mage.ai/sign-up)
For data teams
See how Mage AI accelerates your team velocity while reducing data and infrastructure costs.
[Enterprise demo](./getdemo)
[Explore the platform](./build)
[\
\
AI\
\
Code faster, smarter](./ai)
[\
\
Data pipelines\
\
Build, run, and monitor](./data)
[\
\
Dev Ex\
\
Made for speed and ease](./build)
[\
\
Deployments\
\
Ship workflows effortlessly](./deploy)
[\
\
Scalabiity\
\
Grow without limits](./scale)
[\
\
Customizations\
\
Total extensibility](./extend)
How do I activate the Mage AI on my cluster?
How is our AI different from other AI tools?
How is my data used?
Is there a free trial?
How can I opt-out of using the Mage AI or disable specific models/LLMs?
How do I activate the Mage AI on my cluster?
How is our AI different from other AI tools?
How is my data used?
Is there a free trial?
How can I opt-out of using the Mage AI or disable specific models/LLMs?
Your AI data engineer
[Get started](https://mage.ai/sign-up)
[Request a demo](./getdemo)
Product
[AI](./ai)
[Data pipelines](./data)
[Developer experience](./build)
[Deployments](./deploy)
[Scalability](./scale)
[Customizations](./extend)
Resources
[Pricing](./pricing)
[Documentation](https://docs.mage.ai/)
[Blog](./blog)
[Platform features](./specs)
[Unique differentiators](./innovations)
[Mage Pro vs OSS](./oss-vs-pro)
[Changelog](./updates)
Getting started
[Quickstart](https://docs.mage.ai/getting-started/setup)
[Guides](https://docs.mage.ai/guides/overview)
[Templates](./commons)
[Core abstractions](https://docs.mage.ai/design/core-abstractions)
[Dynamic blocks](https://docs.mage.ai/design/blocks/dynamic-blocks)
Tools
[Help & support](https://www.mage.ai/chat)
[Migration support](./support-plan)
[Community](https://www.mage.ai/chat)
[Partner program](./partners)
Company
[Terms](./terms)
[Privacy Policy](./privacy)
Ā© 2025 Mage Technologies, Inc.
Your AI data engineer
[Get started](https://mage.ai/sign-up)
[Request a demo](./getdemo)
Product
[AI](./ai)
[Data pipelines](./data)
[Developer experience](./build)
[Deployments](./deploy)
[Scalability](./scale)
[Customizations](./extend)
Resources
[Pricing](./pricing)
[Documentation](https://docs.mage.ai/)
[Blog](./blog)
[Platform features](./specs)
[Unique differentiators](./innovations)
[Mage Pro vs OSS](./oss-vs-pro)
[Changelog](./updates)
Getting started
[Quickstart](https://docs.mage.ai/getting-started/setup)
[Guides](https://docs.mage.ai/guides/overview)
[Templates](./commons)
[Core abstractions](https://docs.mage.ai/design/core-abstractions)
[Dynamic blocks](https://docs.mage.ai/design/blocks/dynamic-blocks)
Tools
[Help & support](https://www.mage.ai/chat)
[Migration support](./support-plan)
[Community](https://www.mage.ai/chat)
[Partner program](./partners)
Company
[Terms](./terms)
[Privacy Policy](./privacy)
Ā© 2025 Mage Technologies, Inc.
[](./)
Platform
Explore
Pricing
[Community](https://www.mage.ai/chat)
[Get a demo](./getdemo)
[Start now](https://cloud.mage.ai/sign-up)
[](./)
---
# Extend | Unlock Flexible Extensibility | Mage AI
The customizable
data platform
The customizable
data platform
Integrate, extend, and evolve.
[API](./extend#api)
[Global Hooks](./extend#hooks)
[Security](./extend#security)
Mage AIās extensibility suite transforms data platforms from static tools intoĀ **living enterprise organisms** ā breathing with custom logic, evolving via API integration, and protected by atomic security controls.
This level of flexibility allows organizations to seamlessly integrate Mage into their existing ecosystems while maintaining stringent security and compliance standards.
Mage AIās extensibility suite transforms data platforms from static tools intoĀ **living enterprise organisms** ā breathing with custom logic, evolving via API integration, and protected by atomic security controls.
This level of flexibility allows organizations to seamlessly integrate Mage into their existing ecosystems while maintaining stringent security and compliance standards.
Drag
API
100+
100+
Production-ready API endpoints
Production-ready API endpoints
The Mage AI platform exposes every platform capability through REST APIs, enabling complete programmatic control over the entire data ecosystem.
The Mage AI platform exposes every platform capability through REST APIs, enabling complete programmatic control over the entire data ecosystem.
Atomic operations. Create / update / delete any Mage resource (pipelines, blocks, schedules, etc).
Metadata management. Query and update data catalogs, lineage, and governance information.
Cluster resource management. Trigger updates, deployments, and more. Monitor infrastructure metrics, system updates, and react to events.
User and permission control. Programmatically manage access and security policies.
[Try it yourself](https://mage.ai/sign-up)
Global Hooks
Runtime mutation engine
Runtime mutation engine
Behavioral DNA editing with Global Hooks, providing a powerful mechanism to inject custom logic at strategic points throughout the Mage ecosystem.
Behavioral DNA editing with Global Hooks, providing a powerful mechanism to inject custom logic at strategic points throughout the Mage ecosystem.
Drag
Universal coverage. Attach hooks to any of the 100+ API endpoints.
Granular targeting. Use predicates for conditional hook execution.
Full context access. Modify request/response payloads, trigger pipelines, interact with external systems.
Asynchronous capability. Long-running hooks wonāt block main request processing.
[Try it yourself](https://mage.ai/sign-up)
Security
Security at the atomic level
Security at the atomic level
Infinitely composable access controls using Mage AIās RBAC system, allowing unprecedented granularity in defining user roles and permissions.
Infinitely composable access controls using Mage AIās RBAC system, allowing unprecedented granularity in defining user roles and permissions.
Drag
Endpoint-level control. Grant or restrict access to specific API operation and resource.
Attribute scoping. Define read/write permissions on individual resource fields.
Query parameter restrictions. Limit allowed values for API query parameters.
Role composition. Build complex permission sets through role aggregation and flexible assignment of roles to users and groups.
[Try it yourself](https://mage.ai/sign-up)
Recover your precious developer time
Now you can focus on the fun, creative, and high-impact data engineering projects and let Mage AI handle the rest.
For engineers
Experience how Mage AI ships data pipelines faster, giving you a better work-life-balance.
[Get started](https://mage.ai/sign-up)
For data teams
See how Mage AI accelerates your team velocity while reducing data and infrastructure costs.
[Enterprise demo](./getdemo)
[Explore the platform](./build)
[\
\
AI\
\
Code faster, smarter](./ai)
[\
\
Data pipelines\
\
Build, run, and monitor](./data)
[\
\
Dev Ex\
\
Made for speed and ease](./build)
[\
\
Deployments\
\
Ship workflows effortlessly](./deploy)
[\
\
Scalabiity\
\
Grow without limits](./scale)
[\
\
Customizations\
\
Total extensibility](./extend)
Recover your precious developer time
Now you can focus on the fun, creative, and high-impact data engineering projects and let Mage AI handle the rest.
For engineers
Experience how Mage AI ships data pipelines faster, giving you a better work-life-balance.
[Get started](https://mage.ai/sign-up)
For data teams
See how Mage AI accelerates your team velocity while reducing data and infrastructure costs.
[Enterprise demo](./getdemo)
[Explore the platform](./build)
[\
\
AI\
\
Code faster, smarter](./ai)
[\
\
Data pipelines\
\
Build, run, and monitor](./data)
[\
\
Dev Ex\
\
Made for speed and ease](./build)
[\
\
Deployments\
\
Ship workflows effortlessly](./deploy)
[\
\
Scalabiity\
\
Grow without limits](./scale)
[\
\
Customizations\
\
Total extensibility](./extend)
Your AI data engineer
[Get started](https://mage.ai/sign-up)
[Request a demo](./getdemo)
Product
[AI](./ai)
[Data pipelines](./data)
[Developer experience](./build)
[Deployments](./deploy)
[Scalability](./scale)
[Customizations](./extend)
Resources
[Pricing](./pricing)
[Documentation](https://docs.mage.ai/)
[Blog](./blog)
[Platform features](./specs)
[Unique differentiators](./innovations)
[Mage Pro vs OSS](./oss-vs-pro)
[Changelog](./updates)
Getting started
[Quickstart](https://docs.mage.ai/getting-started/setup)
[Guides](https://docs.mage.ai/guides/overview)
[Templates](./commons)
[Core abstractions](https://docs.mage.ai/design/core-abstractions)
[Dynamic blocks](https://docs.mage.ai/design/blocks/dynamic-blocks)
Tools
[Help & support](https://www.mage.ai/chat)
[Migration support](./support-plan)
[Community](https://www.mage.ai/chat)
[Partner program](./partners)
Company
[Terms](./terms)
[Privacy Policy](./privacy)
Ā© 2025 Mage Technologies, Inc.
Your AI data engineer
[Get started](https://mage.ai/sign-up)
[Request a demo](./getdemo)
Product
[AI](./ai)
[Data pipelines](./data)
[Developer experience](./build)
[Deployments](./deploy)
[Scalability](./scale)
[Customizations](./extend)
Resources
[Pricing](./pricing)
[Documentation](https://docs.mage.ai/)
[Blog](./blog)
[Platform features](./specs)
[Unique differentiators](./innovations)
[Mage Pro vs OSS](./oss-vs-pro)
[Changelog](./updates)
Getting started
[Quickstart](https://docs.mage.ai/getting-started/setup)
[Guides](https://docs.mage.ai/guides/overview)
[Templates](./commons)
[Core abstractions](https://docs.mage.ai/design/core-abstractions)
[Dynamic blocks](https://docs.mage.ai/design/blocks/dynamic-blocks)
Tools
[Help & support](https://www.mage.ai/chat)
[Migration support](./support-plan)
[Community](https://www.mage.ai/chat)
[Partner program](./partners)
Company
[Terms](./terms)
[Privacy Policy](./privacy)
Ā© 2025 Mage Technologies, Inc.
[](./)
Platform
Explore
Pricing
[Community](https://www.mage.ai/chat)
[Get a demo](./getdemo)
[Start now](https://cloud.mage.ai/sign-up)
[](./)
---
# Scale | Intelligent Data Pipeline Scaling | Mage AI
Scales beyond.
Scales beyond.
Grows with you, from startup to industry icon
Grows with you, from startup to industry icon
[Dynamic blocks](./scale#dynamic)
[Spark](./scale#spark)
[Integrations](./scale#integrations)
Never overpay
for unused infra
Never overpay
for unused infra
Mage doesnāt just process data, it revolutionizes how you think about scalability by intelligently scaling data pipelines, vertically and horizontally, in real-time while maintaining peak performance and reducing costs by up to 40%.
Mage doesnāt just process data, it revolutionizes how you think about scalability by intelligently scaling data pipelines, vertically and horizontally, in real-time while maintaining peak performance and reducing costs by up to 40%.
Dynamic blocks
True dynamos!
True dynamos!
Mageās hyper-concurrency engine splits workloads into independent and self-managing units. These tasks are dynamically generated and distributed across your infrastructure, maximizing speed and processing power across all available resources.
Mageās hyper-concurrency engine splits workloads into independent and self-managing units. These tasks are dynamically generated and distributed across your infrastructure, maximizing speed and processing power across all available resources.
Dynamic blocks adapt their behavior based on input data or runtime conditions, enabling the creation of flexible and complex data pipelines that can easily accommodate varying scalability requirements all without the need to write duplicate code.
**Dynamic blocks** revolutionize pipeline architecture through adaptive parallelism and context-aware execution, transforming static code from rigid sequences into living neural networks of data processing.
**Dynamic blocks** revolutionize pipeline architecture through adaptive parallelism and context-aware execution, transforming static code from rigid sequences into living neural networks of data processing.
[Try it yourself](https://mage.ai/sign-up)
Unlike static DAGs, dynamic blocks enable fractal-like processing trees that auto-scale with data complexity.
This represents a paradigm shift in data pipeline orchestration, enabling intelligent workload distribution and runtime flexibility that sets the platform apart from traditional ETL tools.
Asynchronous Execution Matrix
Asynchronous Execution Matrix
* Sibling blocks execute concurrently without synchronization
* Sibling blocks execute concurrently without synchronization
* Each branch maintains isolated context through UUID-bound metadata
* Each branch maintains isolated context through UUID-bound metadata
* Failure domains constrained to individual data partitions and doesnāt affect the sibling branches
* Failure domains constrained to individual data partitions and doesnāt affect the sibling branches
Stream mode execution
Stream mode execution
* Continuous data hydration enables processing records before the full dataset lands
* Continuous data hydration enables processing records before the full dataset lands
* Achieve 60% faster data delivery SLAs
* Achieve 60% faster data delivery SLAs
* 90% memory reduction vs batch processing
* 90% memory reduction vs batch processing
Adaptive topology support
Adaptive topology support
* Hybrid parentage: Combine static/dynamic upstreams
* Hybrid parentage: Combine static/dynamic upstreams
* Multi-parent orchestration through metadata inheritance
* Multi-parent orchestration through metadata inheritance
* Auto-generated UUIDs prevent namespace collisions
* Auto-generated UUIDs prevent namespace collisions
Recursive reduction engine
Recursive reduction engine
* Fan-in patterns to reduce each blockās data output into a single source of data
* Fan-in patterns to reduce each blockās data output into a single source of data
* Multiple reduction strategies (concat, sum, merge)
* Multiple reduction strategies (concat, sum, merge)
* Preserved data lineage through reduction stages
* Preserved data lineage through reduction stages
Spark
Spark magic in lightning time
Spark magic in lightning time
Run PySpark and SparkSQL alongside vanilla Python ā zero infra tax, maximum data power. Mage AI provides a robust interface for monitoring and debugging your Spark pipelines, offering detailed insights into execution metrics, stages, and SQL operations.
Run PySpark and SparkSQL alongside vanilla Python ā zero infra tax, maximum data power. Mage AI provides a robust interface for monitoring and debugging your Spark pipelines, offering detailed insights into execution metrics, stages, and SQL operations.
Infrastructure autopilot. Mage auto-provisions optimized clusters on-demand per data pipeline needs.
Infrastructure autopilot. Mage auto-provisions optimized clusters on-demand per data pipeline needs.
[Try it yourself](https://mage.ai/sign-up)
Execution metrics overview. Track Spark execution metrics during development and in production.
Execution metrics overview. Track Spark execution metrics during development and in production.
[Try it yourself](https://mage.ai/sign-up)
Code hybridization engine. Seamless context handoff between Spark, Pandas, Polars, PyArrow, and other Python objects.
Code hybridization engine. Seamless context handoff between Spark, Pandas, Polars, PyArrow, and other Python objects.
Stages and tasks analysis. See what is happening along the way.
Stages and tasks analysis. See what is happening along the way.
[Try it yourself](https://mage.ai/sign-up)
Visualize task execution phases (e.g., shuffle read/write, deserialization) to identify bottlenecks.
Visualize task execution phases (e.g., shuffle read/write, deserialization) to identify bottlenecks.
Analyze key metrics such as input records, shuffle bytes, and GC time to optimize performance.
Drill into individual tasks to debug failures or inefficiencies.
SQL execution insights. Gain a deeper understanding of your Spark queries
SQL execution insights. Gain a deeper understanding of your Spark queries
[Try it yourself](https://mage.ai/sign-up)
View the query plan as a graph to understand how Spark processes data (e.g., scans, transformations).
Inspect detailed statistics like scan time, file sizes, and output rows for each stage of the query.
Track SQL statements across multiple jobs with completion status and durations.
Integrations
Build anything,
connect to everything
Build anything,
connect to everything
* 
All your favorites. From zero-copy Polars to petabyte-scale Iceberg ā wield without infra tax.
All your favorites. From zero-copy Polars to petabyte-scale Iceberg ā wield without infra tax.
Avoid vendor lock in. Blend cloud SQL engines with OSS formats.
Avoid vendor lock in. Blend cloud SQL engines with OSS formats.
Cost arbitrage. Process cold data in DuckDB / Polars and hot data in BigQuery / Snowflake.
Cost arbitrage. Process cold data in DuckDB / Polars and hot data in BigQuery / Snowflake.
Big data, small cost
Mage AIās smart resource managementā¦
Automatically matches processing power to workload demands
Automatically matches processing power to workload demands
Eliminates wasted capacity with predictive scaling
Eliminates wasted capacity with predictive scaling
Processes massive datasets without costly hardware upgrades
Processes massive datasets without costly hardware upgrades
Reduces cloud spend while maintaining petabyte-scale throughput
Reduces cloud spend while maintaining petabyte-scale throughput
Recover your precious developer time
Now you can focus on the fun, creative, and high-impact data engineering projects and let Mage AI handle the rest.
For engineers
Experience how Mage AI ships data pipelines faster, giving you a better work-life-balance.
[Get started](https://mage.ai/sign-up)
For data teams
See how Mage AI accelerates your team velocity while reducing data and infrastructure costs.
[Enterprise demo](./getdemo)
[Explore the platform](./build)
[\
\
AI\
\
Code faster, smarter](./ai)
[\
\
Data pipelines\
\
Build, run, and monitor](./data)
[\
\
Dev Ex\
\
Made for speed and ease](./build)
[\
\
Deployments\
\
Ship workflows effortlessly](./deploy)
[\
\
Scalabiity\
\
Grow without limits](./scale)
[\
\
Customizations\
\
Total extensibility](./extend)
Recover your precious developer time
Now you can focus on the fun, creative, and high-impact data engineering projects and let Mage AI handle the rest.
For engineers
Experience how Mage AI ships data pipelines faster, giving you a better work-life-balance.
[Get started](https://mage.ai/sign-up)
For data teams
See how Mage AI accelerates your team velocity while reducing data and infrastructure costs.
[Enterprise demo](./getdemo)
[Explore the platform](./build)
[\
\
AI\
\
Code faster, smarter](./ai)
[\
\
Data pipelines\
\
Build, run, and monitor](./data)
[\
\
Dev Ex\
\
Made for speed and ease](./build)
[\
\
Deployments\
\
Ship workflows effortlessly](./deploy)
[\
\
Scalabiity\
\
Grow without limits](./scale)
[\
\
Customizations\
\
Total extensibility](./extend)
Your AI data engineer
[Get started](https://mage.ai/sign-up)
[Request a demo](./getdemo)
Product
[AI](./ai)
[Data pipelines](./data)
[Developer experience](./build)
[Deployments](./deploy)
[Scalability](./scale)
[Customizations](./extend)
Resources
[Pricing](./pricing)
[Documentation](https://docs.mage.ai/)
[Blog](./blog)
[Platform features](./specs)
[Unique differentiators](./innovations)
[Mage Pro vs OSS](./oss-vs-pro)
[Changelog](./updates)
Getting started
[Quickstart](https://docs.mage.ai/getting-started/setup)
[Guides](https://docs.mage.ai/guides/overview)
[Templates](./commons)
[Core abstractions](https://docs.mage.ai/design/core-abstractions)
[Dynamic blocks](https://docs.mage.ai/design/blocks/dynamic-blocks)
Tools
[Help & support](https://www.mage.ai/chat)
[Migration support](./support-plan)
[Community](https://www.mage.ai/chat)
[Partner program](./partners)
Company
[Terms](./terms)
[Privacy Policy](./privacy)
Ā© 2025 Mage Technologies, Inc.
Your AI data engineer
[Get started](https://mage.ai/sign-up)
[Request a demo](./getdemo)
Product
[AI](./ai)
[Data pipelines](./data)
[Developer experience](./build)
[Deployments](./deploy)
[Scalability](./scale)
[Customizations](./extend)
Resources
[Pricing](./pricing)
[Documentation](https://docs.mage.ai/)
[Blog](./blog)
[Platform features](./specs)
[Unique differentiators](./innovations)
[Mage Pro vs OSS](./oss-vs-pro)
[Changelog](./updates)
Getting started
[Quickstart](https://docs.mage.ai/getting-started/setup)
[Guides](https://docs.mage.ai/guides/overview)
[Templates](./commons)
[Core abstractions](https://docs.mage.ai/design/core-abstractions)
[Dynamic blocks](https://docs.mage.ai/design/blocks/dynamic-blocks)
Tools
[Help & support](https://www.mage.ai/chat)
[Migration support](./support-plan)
[Community](https://www.mage.ai/chat)
[Partner program](./partners)
Company
[Terms](./terms)
[Privacy Policy](./privacy)
Ā© 2025 Mage Technologies, Inc.
[](./)
Platform
Explore
Pricing
[Community](https://www.mage.ai/chat)
[Get a demo](./getdemo)
[Start now](https://cloud.mage.ai/sign-up)
[](./)
---
# GitHub - mage-ai/mage-ai: š§ Build, run, and manage data pipelines for integrating and transforming data.
You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session. You switched accounts on another tab or window. Reload to refresh your session. Dismiss alert
[mage-ai](/mage-ai) / **[mage-ai](/mage-ai/mage-ai)** Public
* [Notifications](/login?return_to=%2Fmage-ai%2Fmage-ai)
You must be signed in to change notification settings
* [Fork 826](/login?return_to=%2Fmage-ai%2Fmage-ai)
* [Star 8.2k](/login?return_to=%2Fmage-ai%2Fmage-ai)
š§ Build, run, and manage data pipelines for integrating and transforming data.
[www.mage.ai](https://www.mage.ai "https://www.mage.ai")
### License
[Apache-2.0 license](/mage-ai/mage-ai/blob/master/LICENSE)
[8.2k stars](/mage-ai/mage-ai/stargazers)
[826 forks](/mage-ai/mage-ai/forks)
[Branches](/mage-ai/mage-ai/branches)
[Tags](/mage-ai/mage-ai/tags)
[Activity](/mage-ai/mage-ai/activity)
[Star](/login?return_to=%2Fmage-ai%2Fmage-ai)
[Notifications](/login?return_to=%2Fmage-ai%2Fmage-ai)
You must be signed in to change notification settings
mage-ai/mage-ai
===============
Ā master
[Branches](/mage-ai/mage-ai/branches)
[Tags](/mage-ai/mage-ai/tags)
[](/mage-ai/mage-ai/branches)
[](/mage-ai/mage-ai/tags)
Go to file
Code
Folders and files
-----------------
| Name | | Name | Last commit message | Last commit date |
| --- | --- | --- | --- |
| Latest commit
-------------
History
-------
[5,557 Commits](/mage-ai/mage-ai/commits/master/)
[](/mage-ai/mage-ai/commits/master/) | | |
| [.git-dev/hooks](/mage-ai/mage-ai/tree/master/.git-dev/hooks "This path skips through empty directories") | | [.git-dev/hooks](/mage-ai/mage-ai/tree/master/.git-dev/hooks "This path skips through empty directories") | | |
| [.github](/mage-ai/mage-ai/tree/master/.github ".github") | | [.github](/mage-ai/mage-ai/tree/master/.github ".github") | | |
| [docs](/mage-ai/mage-ai/tree/master/docs "docs") | | [docs](/mage-ai/mage-ai/tree/master/docs "docs") | | |
| [integrations](/mage-ai/mage-ai/tree/master/integrations "integrations") | | [integrations](/mage-ai/mage-ai/tree/master/integrations "integrations") | | |
| [kube](/mage-ai/mage-ai/tree/master/kube "kube") | | [kube](/mage-ai/mage-ai/tree/master/kube "kube") | | |
| [mage\_ai](/mage-ai/mage-ai/tree/master/mage_ai "mage_ai") | | [mage\_ai](/mage-ai/mage-ai/tree/master/mage_ai "mage_ai") | | |
| [mage\_integrations](/mage-ai/mage-ai/tree/master/mage_integrations "mage_integrations") | | [mage\_integrations](/mage-ai/mage-ai/tree/master/mage_integrations "mage_integrations") | | |
| [scripts](/mage-ai/mage-ai/tree/master/scripts "scripts") | | [scripts](/mage-ai/mage-ai/tree/master/scripts "scripts") | | |
| [templates](/mage-ai/mage-ai/tree/master/templates "templates") | | [templates](/mage-ai/mage-ai/tree/master/templates "templates") | | |
| [.flake8](/mage-ai/mage-ai/blob/master/.flake8 ".flake8") | | [.flake8](/mage-ai/mage-ai/blob/master/.flake8 ".flake8") | | |
| [.gitignore](/mage-ai/mage-ai/blob/master/.gitignore ".gitignore") | | [.gitignore](/mage-ai/mage-ai/blob/master/.gitignore ".gitignore") | | |
| [.pre-commit-config.yaml](/mage-ai/mage-ai/blob/master/.pre-commit-config.yaml ".pre-commit-config.yaml") | | [.pre-commit-config.yaml](/mage-ai/mage-ai/blob/master/.pre-commit-config.yaml ".pre-commit-config.yaml") | | |
| [Dockerfile](/mage-ai/mage-ai/blob/master/Dockerfile "Dockerfile") | | [Dockerfile](/mage-ai/mage-ai/blob/master/Dockerfile "Dockerfile") | | |
| [LICENSE](/mage-ai/mage-ai/blob/master/LICENSE "LICENSE") | | [LICENSE](/mage-ai/mage-ai/blob/master/LICENSE "LICENSE") | | |
| [MANIFEST.in](/mage-ai/mage-ai/blob/master/MANIFEST.in "MANIFEST.in") | | [MANIFEST.in](/mage-ai/mage-ai/blob/master/MANIFEST.in "MANIFEST.in") | | |
| [Makefile](/mage-ai/mage-ai/blob/master/Makefile "Makefile") | | [Makefile](/mage-ai/mage-ai/blob/master/Makefile "Makefile") | | |
| [README.md](/mage-ai/mage-ai/blob/master/README.md "README.md") | | [README.md](/mage-ai/mage-ai/blob/master/README.md "README.md") | | |
| [README\_dev.md](/mage-ai/mage-ai/blob/master/README_dev.md "README_dev.md") | | [README\_dev.md](/mage-ai/mage-ai/blob/master/README_dev.md "README_dev.md") | | |
| [SUMMARY.md](/mage-ai/mage-ai/blob/master/SUMMARY.md "SUMMARY.md") | | [SUMMARY.md](/mage-ai/mage-ai/blob/master/SUMMARY.md "SUMMARY.md") | | |
| [aws](/mage-ai/mage-ai/blob/master/aws "aws") | | [aws](/mage-ai/mage-ai/blob/master/aws "aws") | | |
| [dev.Dockerfile](/mage-ai/mage-ai/blob/master/dev.Dockerfile "dev.Dockerfile") | | [dev.Dockerfile](/mage-ai/mage-ai/blob/master/dev.Dockerfile "dev.Dockerfile") | | |
| [dev.requirements.txt](/mage-ai/mage-ai/blob/master/dev.requirements.txt "dev.requirements.txt") | | [dev.requirements.txt](/mage-ai/mage-ai/blob/master/dev.requirements.txt "dev.requirements.txt") | | |
| [dev.spark.Dockerfile](/mage-ai/mage-ai/blob/master/dev.spark.Dockerfile "dev.spark.Dockerfile") | | [dev.spark.Dockerfile](/mage-ai/mage-ai/blob/master/dev.spark.Dockerfile "dev.spark.Dockerfile") | | |
| [docker-compose.yml](/mage-ai/mage-ai/blob/master/docker-compose.yml "docker-compose.yml") | | [docker-compose.yml](/mage-ai/mage-ai/blob/master/docker-compose.yml "docker-compose.yml") | | |
| [example.ipynb](/mage-ai/mage-ai/blob/master/example.ipynb "example.ipynb") | | [example.ipynb](/mage-ai/mage-ai/blob/master/example.ipynb "example.ipynb") | | |
| [lsp.Dockerfile](/mage-ai/mage-ai/blob/master/lsp.Dockerfile "lsp.Dockerfile") | | [lsp.Dockerfile](/mage-ai/mage-ai/blob/master/lsp.Dockerfile "lsp.Dockerfile") | | |
| [pg-docker-compose.yml](/mage-ai/mage-ai/blob/master/pg-docker-compose.yml "pg-docker-compose.yml") | | [pg-docker-compose.yml](/mage-ai/mage-ai/blob/master/pg-docker-compose.yml "pg-docker-compose.yml") | | |
| [poetry.lock](/mage-ai/mage-ai/blob/master/poetry.lock "poetry.lock") | | [poetry.lock](/mage-ai/mage-ai/blob/master/poetry.lock "poetry.lock") | | |
| [pyproject.toml](/mage-ai/mage-ai/blob/master/pyproject.toml "pyproject.toml") | | [pyproject.toml](/mage-ai/mage-ai/blob/master/pyproject.toml "pyproject.toml") | | |
| [requirements.txt](/mage-ai/mage-ai/blob/master/requirements.txt "requirements.txt") | | [requirements.txt](/mage-ai/mage-ai/blob/master/requirements.txt "requirements.txt") | | |
| [setup.py](/mage-ai/mage-ai/blob/master/setup.py "setup.py") | | [setup.py](/mage-ai/mage-ai/blob/master/setup.py "setup.py") | | |
| [test\_branch.Dockerfile](/mage-ai/mage-ai/blob/master/test_branch.Dockerfile "test_branch.Dockerfile") | | [test\_branch.Dockerfile](/mage-ai/mage-ai/blob/master/test_branch.Dockerfile "test_branch.Dockerfile") | | |
| View all files | | |
Repository files navigation
---------------------------
Mage AI
=======
[](#mage-ai)
Give your data team magical powers.
[](https://mage.ai)
[](https://hub.docker.com/r/mageai/mageai)
[](https://github.com/mage-ai/mage-ai/blob/master/LICENSE)
[](https://www.mage.ai/chat)
[](https://cloud.mage.ai/sign-up)
Mage is a hybrid framework for transforming and integrating data. It combines the best of both worlds: the flexibility of notebooks with the rigor of modular code.
* Extract and synchronize data from 3rd party sources.
* Transform data with real-time and batch pipelines using Python, SQL, and R.
* Load data into your data warehouse or data lake using our pre-built connectors.
* Run, monitor, and orchestrate thousands of pipelines without losing sleep.
Plus hundreds of enterprise-class features, infrastructure innovations, and magical surprises.
#### Available in two spellbinding versions
[](#available-in-two-spellbinding-versions)
| | |
| --- | --- |
| [](https://cloud.mage.ai/sign-up)
**For teams.** Fully managed platform for integrating and transforming data. | [](https://github.com/mage-ai/mage-ai?tab=readme-ov-file#its-magic)
**Self-hosted.** System to build, run, and manage data pipelines. |
[](https://cloud.mage.ai/sign-up)
Itās magic.
===========
[](#its-magic)
For documentation on getting started, how to develop, and how to deploy to production check out the live
[**Developer documentation portal**](https://docs.mage.ai?source=github)
.
šāāļø Install
-------------
[](#ļø-install)
The recommended way to install the latest version of Mage is through Docker with the following command:
```shell
docker pull mageai/mageai:latest
```
You can also install Mage using pip or conda, though this may cause dependency issues without the proper environment.
```shell
pip install mage-ai
```
```shell
conda install -c conda-forge mage-ai
```
Looking for help? The _fastest_ way to get started is by checking out our documentation [here](https://docs.mage.ai/getting-started/setup)
.
Looking for quick examples? Open a [demo](https://demo.mage.ai/)
project right in your browser or check out our [guides](https://docs.mage.ai/guides/overview)
.
š® Demo
-------
[](#-demo)
### Live demo
[](#live-demo)
Build and run a data pipeline with our **[demo app](https://demo.mage.ai/)
**.
> WARNING
>
> The live demo is public to everyone, please donāt save anything sensitive (e.g. passwords, secrets, etc).
### Demo video (5 min)
[](#demo-video-5-min)
[](https://youtu.be/GswOdShLGmg)
_Click the image to play video_
š® [Features](https://docs.mage.ai/about/features)
---------------------------------------------------
[](#-features)
| | | |
| --- | --- | --- |
| š¶ | **[Orchestration](https://docs.mage.ai/design/data-pipeline-management)
** | Schedule and manage data pipelines with observability. |
| š | **[Notebook](https://docs.mage.ai/about/features#notebook-for-building-data-pipelines)
** | Interactive Python, SQL, & R editor for coding data pipelines. |
| šļø | **[Data integrations](https://docs.mage.ai/data-integrations/overview)
** | Synchronize data from 3rd party sources to your internal destinations. |
| š° | **[Streaming pipelines](https://docs.mage.ai/guides/streaming-pipeline)
** | Ingest and transform real-time data. |
| ā | **[dbt](https://docs.mage.ai/dbt/overview)
** | Build, run, and manage your dbt models with Mage. |
**A sample data pipeline defined across 3 files ā**
1. Load data ā
```python
@data_loader
def load_csv_from_file() -> pl.DataFrame:
return pl.read_csv('default_repo/titanic.csv')
```
2. Transform data ā
```python
@transformer
def select_columns_from_df(df: pl.DataFrame, *args) -> pl.DataFrame:
return df[['Age', 'Fare', 'Survived']]
```
3. Export data ā
```python
@data_exporter
def export_titanic_data_to_disk(df: pl.DataFrame) -> None:
df.to_csv('default_repo/titanic_transformed.csv')
```
[](https://www.mage.ai/)
About
-----
š§ Build, run, and manage data pipelines for integrating and transforming data.
[www.mage.ai](https://www.mage.ai "https://www.mage.ai")
### Topics
[python](/topics/python "Topic: python")
[data-science](/topics/data-science "Topic: data-science")
[data](/topics/data "Topic: data")
[machine-learning](/topics/machine-learning "Topic: machine-learning")
[sql](/topics/sql "Topic: sql")
[spark](/topics/spark "Topic: spark")
[pipeline](/topics/pipeline "Topic: pipeline")
[etl](/topics/etl "Topic: etl")
[pipelines](/topics/pipelines "Topic: pipelines")
[orchestration](/topics/orchestration "Topic: orchestration")
[artificial-intelligence](/topics/artificial-intelligence "Topic: artificial-intelligence")
[data-engineering](/topics/data-engineering "Topic: data-engineering")
[data-integration](/topics/data-integration "Topic: data-integration")
[dbt](/topics/dbt "Topic: dbt")
[elt](/topics/elt "Topic: elt")
[transformation](/topics/transformation "Topic: transformation")
[data-pipelines](/topics/data-pipelines "Topic: data-pipelines")
[reverse-etl](/topics/reverse-etl "Topic: reverse-etl")
### Resources
[Readme](#readme-ov-file)
### License
[Apache-2.0 license](#Apache-2.0-1-ov-file)
[Activity](/mage-ai/mage-ai/activity)
[Custom properties](/mage-ai/mage-ai/custom-properties)
### Stars
[**8.2k** stars](/mage-ai/mage-ai/stargazers)
### Watchers
[**62** watching](/mage-ai/mage-ai/watchers)
### Forks
[**826** forks](/mage-ai/mage-ai/forks)
[Report repository](/contact/report-content?content_url=https%3A%2F%2Fgithub.com%2Fmage-ai%2Fmage-ai&report=mage-ai+%28user%29)
[Releases 56](/mage-ai/mage-ai/releases)
-----------------------------------------
[0.9.75 | Wicked š§āāļøš§¹ Latest\
\
Dec 6, 2024](/mage-ai/mage-ai/releases/tag/0.9.75)
[\+ 55 releases](/mage-ai/mage-ai/releases)
[Packages 0](/orgs/mage-ai/packages?repo_name=mage-ai)
-------------------------------------------------------
No packages published
[Used by 199](/mage-ai/mage-ai/network/dependents)
---------------------------------------------------
[* \
* \
* \
* \
* \
* \
* \
* \
\
\+ 191](/mage-ai/mage-ai/network/dependents)
[Contributors 140](/mage-ai/mage-ai/graphs/contributors)
---------------------------------------------------------
* [](https://github.com/tommydangerous)
* [](https://github.com/wangxiaoyou1993)
* [](https://github.com/dy46)
* [](https://github.com/johnson-mage)
* [](https://github.com/ci-mage)
* [](https://github.com/mattppal)
* [](https://github.com/skunichetty)
* [](https://github.com/shrey-mage)
* [](https://github.com/Luishfs)
* [](https://github.com/csharplus)
* [](https://github.com/thomaschung408)
* [](https://github.com/nathaniel-mage)
* [](https://github.com/matrixstone)
* [](https://github.com/anniexcheng)
[\+ 126 contributors](/mage-ai/mage-ai/graphs/contributors)
Languages
---------
* [Python 53.4%](/mage-ai/mage-ai/search?l=python)
* [TypeScript 36.3%](/mage-ai/mage-ai/search?l=typescript)
* [HTML 8.5%](/mage-ai/mage-ai/search?l=html)
* [CSS 0.7%](/mage-ai/mage-ai/search?l=css)
* [SCSS 0.5%](/mage-ai/mage-ai/search?l=scss)
* [JavaScript 0.2%](/mage-ai/mage-ai/search?l=javascript)
* Other 0.4%
You canāt perform that action at this time.
---
# Terms of Service | AI Data Engineering Platform | Mage AI
Terms.
[Services & API](./terms#1)
[Access & use](./terms#2)
[Term & termination](./terms#3)
[Fees & payments](./terms#4)
[Trials & betas](./terms#5)
[Proprietary rights](./terms#6)
[Confidentiality](./terms#7)
[Customer data](./terms#8)
[Indemnification](./terms#9)
[Warranties](./terms#10)
[Disclaimer](./terms#11)
[Liability](./terms#12)
[General provisions](./terms#13)
[Definitions](./terms#14)
Terms of Service
================
Last updated: December 2, 2021
This Master Subscription Agreement (this āMSAā) is entered into by and between Mage Technologies, Inc., a California corporation with offices at 2010 El Camino Real #2073 Santa Clara, CA 95050 (āMageā), and the individual or entity entering into this Agreement (āCustomerā) as of the date Customer clicks to accept this MSA, accesses or uses the Service, or authorizes or permits any Authorized User to access or use the Service (the āEffective Dateā). By clicking to accept this Master Subscription Agreement or otherwise accessing or using the Service, or authorizing or permitting any Authorized User to access or use the Service, Customer agrees to be bound by this Agreement. If the individual entering into this Agreement is doing so on behalf of a company, organization or other legal entity, such individual is agreeing to this Agreement for that legal entity and representing to Mage that such individual has the authority to bind such legal entity to this Agreement, in which case the term āCustomerā refers to such legal entity. If the individual agreeing to this Agreement does not have authority to enter into this Agreement, if Customer does not agree with this Agreement, there will be no agreement entered into between Mage and Customer and Customer must not use or authorize any use of the Service. Certain features of the Services may be subject to additional guidelines, terms, or rules, which will be posted on the Services in connection with such features (āSupplemental Termsā), all of which are incorporated herein by reference.
1. ### Services and API
1.1 Authorization
Subject to Customerās ongoing compliance with the terms of the Agreement, Mage hereby grants to Customer a non-exclusive, non-transferable, non-sublicensable, internal right to access and use, and allow Authorized Users to access and use, the Service and Dashboard solely for Customerās internal business purposes in accordance with any Service Plan and subject to the terms of this Agreement and any limitations set forth in the Order or any Supplemental Terms.
1.2 API License
Subject to Customerās ongoing compliance with the terms of the Agreement (including the terms and conditions of the applicable Order and timely payment of all applicable Fees), Mage hereby grants Customer a non-exclusive, non-transferable, non-sublicensable, internal use only license, during the Subscription Term to: (i) use APIs to submit to and obtain information from the Service in accordance with any associated Documentation solely as necessary in connection with the use of the Service pursuant to this Agreement; and (ii) make only those copies of APIs that are absolutely necessary to exercise Customerās rights under the foregoing (i).
1.3 Documentation
Mage hereby grants Customer a non-exclusive, non-transferable, non-sublicensable, internal use only license, during the Term to use any Documentation in connection with Customerās use of the Service and API and make only those copies of the Documentation reasonably necessary to exercise Customerās rights hereunder.
1.4 Availability
Mage will use commercially reasonable efforts to make the Service available to Customer in accordance with the Service Plan 24 hours a day, 7 days a week, except during scheduled downtime (of which Mage will use commercially reasonable efforts to give advance notice via the Service or email).
1.5 Modifications
Customer acknowledges that Mage may modify the features and functionality of the Service at any time, including during the Subscription Term. Mage shall provide Customer with commercially reasonable advance notice of any deprecation of any material feature or functionality.
2. ### Access and use
2.1 Account and User Logins
In order to access and use the Service, Customer must register an account on the Service (the āCustomer Accountā). Customer agrees to provide all information required in connection with registering the Customer Account and to keep such information, true, accurate and up to date. Customer may be able to provision access under the Customer Account to the specified number of individual Authorized Users permitted under a Service Plan (āUser Loginsā). Customer acknowledges and agrees that each Authorized User must be an individual person and a User Login cannot be shared or used any other person. User Logins may be reassigned to new individuals replacing former individuals who no longer require ongoing access to or use of the Service. Customer and Authorized Users are responsible for maintaining the confidentiality of all access credentials for the Customer Account and User Logins. Should Mage discover that Customerās or any Authorized Usersā use of a Service violates this Agreement or the Service Plan features and limitations, Mage reserves the right to charge Customer, and Customer hereby agrees to pay for said overuse, in addition to other remedies available to Mage.
2.2 Compliance
Customer is responsible for compliance with the provisions of this Agreement by Customerās personnel and agents, including Authorized Users, and for any and all activities that occur under the Customer Account, which Mage may verify from time to time. Without limiting the foregoing, Customer is solely responsible for ensuring that its and its personnelās and agentsā use of the Service is compliant with all applicable laws and regulations as well as any and all privacy policies, agreements or other obligations Customer may maintain or enter into with third parties.
2.3 Restrictions
Customer shall not, directly or indirectly, and shall not permit any Authorized User or third party to: (i) decompile, disassemble, reverse engineer, or otherwise attempt to derive the source code, algorithms, or associated know-how of the Mage Technology (except to the extent expressly made available to Customer by Mage or permitted by applicable law notwithstanding this restriction); (ii) write or develop any program based upon the Mage Technology or any portion of any of the foregoing, or otherwise use the Mage Technology in any manner for the purpose of developing, distributing or making available products or services that are similar to or compete with the Mage Technology; (iii) sell, sublicense, transfer, assign, lease, rent, distribute, or grant a security interest in the Mage Technology or any rights to any of the foregoing; (iv) permit the Mage Technology to be accessed or used by any persons other than Authorized Users accessing or using the Mage Technology in accordance with the Agreement; (v) use the Service in any manner where Customer acts as a service bureau or to provide any outsourced business process services; (vi) alter or remove any trademarks or proprietary notices contained in or on the Mage Technology; (vii) circumvent or otherwise interfere with any authentication or security measures of the Mage Technology or otherwise interfere with or disrupt the integrity or performance of the foregoing; (viii) use the Service to store or transmit any āprotected health informationā as defined by the Health Insurance Portability and Accountability Act of 1996 (āHIPAAā), unless expressly agreed to otherwise in writing by Mage; (ix) use or launch any automated system that accesses a Service (i.e., bot) in a manner that sends more request messages to a Service server in a given period of time than a human can reasonably produce in the same period by using a conventional online web browser; (x) launch or facilitate, whether intentionally or unintentionally, a level of traffic on the Service, or engage in any other conduct that materially and adversely impact the security, availability, or stability of the Service; or (xi) otherwise use the Mage Technology except as expressly permitted hereunder. Customer represents and warrants that it or, as applicable, Authorized Users have all rights, authorizations, and consents to provide Submitted Data to Mage and Customer has all rights, authorizations, and consents to grant Mage the rights and permissions to use and process the Submitted Data as contemplated by this Agreement and the function of the Service. Customer represents and warrants that it and all Authorized Users will, at all times during the Term, comply with all applicable laws in connection with its use of the Mage Technology, Outputs, or Submitted Data.
2.4 Right to Suspend
Customer acknowledges and agrees that Mage may, but is under no obligation to monitor use of the Service under the Customer Account. Without limitation to any other remedy available to Mage, Mage may suspend Customerās, or an Authorized Userās access to the Service for any period during which Customer or an Authorized User is, or Mage has a reasonable basis for alleging Customer or an Authorized User is, in noncompliance with Section 3.3 of this Agreement.
2.5 System Requirements
A high-speed Internet connection is required for proper use of the Service. Customer is responsible for procuring and maintaining the network connections that connect its network to the Service including, but not limited to, browser software that supports protocols used by Mage and to follow procedures for accessing services that support such protocols. Mage assumes no responsibility for the reliability or performance of any connections as described in this Section.
3. ### Term and termination
3.1 Agreement Term
The Agreement will start on the Effective Date and will continue until terminated in accordance with the Agreement (the āTermā).
3.2 Subscription Terms
A Subscription will begin when Mage accepts the applicable Order by providing a confirmation email for such Subscription and continue until completion of the period for which Customer signs up for a Subscription (the āInitial Termā). Upon expiration of the Initial Term, the Subscription shall automatically and continuously renew for successive periods of the same duration as the Initial Term (each such period, a āRenewal Termā and all Renewal Terms together with the Initial Term of that Subscription, the āSubscription Termā) unless either party provides notice of non-renewal at least 30 days prior to the end of the then current Initial Term or Renewal Term.
3.3 Termination
Either party may terminate the Agreement by written notice: (i) if no Subscription is then in effect; (ii) the other party is in material breach of the Agreement, where such material breach is not cured within 30 days after written notice of such breach; or (iii) if: (a) the other party ceases to carry on its business; (b) a receiver or similar officer is appointed for the other partyās business, property, affairs or revenues and such proceedings continue for 45 days; (c) the other party becomes insolvent, admits in writing its inability to pay debts generally as they come due, is adjudicated bankrupt, or enters composition proceedings, makes an assignment for the benefit of its creditors or another arrangement ofĀ similar import; or (d) proceedings under bankruptcy or insolvency laws are commenced by or against the other party and are not dismissed within 45 days. Notwithstanding anything to the contrary herein, if Customer fails to pay any amounts owed to Mage within 15 days after written notice of nonpayment of any amounts owed to Mage, which may be provided any time after any amount becomes past due, Mage may immediately terminate this Agreement. For the avoidance of doubt, Customerās noncompliance with Section 2.4 shall be deemed a material breach of the Agreement.
3.4 Effect of Termination
Upon the effective date of the expiration or termination of the Agreement for any reason: (i) Customerās access to the Service, and the licenses granted to Customer hereunder will automatically terminate; (ii) all outstanding payment obligations of Customer for the remainder of the then current Subscription Term will become due and payable immediately, unless the Agreement is terminated by Customer pursuant to Section 4.3; and (iii) Customer shall immediately return, or at Mageās request destroy and certify the destruction of any tangible embodiments of Mageās Confidential Information, including all copies of APIs. The following provisions will survive the expiration or termination of the Agreement for any reason: SectionsĀ 1, 3.3, 4.4, 5.1, 5.2, 5.5, 6 through 14.
4. ### Fees and payments
4.1 Fees
During the Term, Customer will pay Mage all fees of the type and amount set forth in an Order (āFeesā). All Fees payable for Subscriptions (āSubscription Feesā) are due monthly in advance unless otherwise set forth in the applicable Order. Mage may, at its sole discretion, change the cost or the amount of fees for any features or functionality of the Service at any point on a going forward basis; provided that changes in Subscription Fees for Customerās then current Service Plan will take effect upon the start of the next Renewal Term.
4.2 Payment
Customer is responsible for providing valid and current payment information for payment of any Fees (āPayment Methodā) and Customer agrees to promptly update its Payment Method, with any changes to keep its Payment Method valid and current.Ā Unless otherwise set forth in an Order or any Supplemental Terms all Fees are non-cancellable, non-refundable, and non-recoupable. Interest accrues from the due date at the lesser of 1.5% per month or the highest rate allowed by law. Customer is responsible for all federal, state, local, sales, use, value added, excise, or other taxes, fees, or duties arising out of the Agreement or the transactions contemplated by the Agreement (other than taxes based on Mageās net income).
4.3 Payment Provider
Payments hereunder may be billed and processed by Mageās third party payment service provider, Stripe, Inc. (āStripeā). for payment services (e.g., card acceptance, merchant settlement, and related services). Customer acknowledges that Mage is bound by, and Customer agrees, to the extent applicable, to be bound by Stripeās terms and conditions (available at https://stripe.com/us/terms) along with its privacy policy (available at https://stripe.com/us/privacy). Customer hereby consents and authorizes the Mage and Stripe. Customer hereby authorizes Mage and Stripe to bill Customerās Payment Method in advance on a periodic basis in accordance with the terms of Customerās Subscription, and for other Fees payable hereunder and to share any information and payment instructions Customer provides with one or more third party service provider(s) to the minimum extent required to complete Customerās transactions with Mage. If applicable, Customer hereby authorizes Mage and Stripe to charge Customerās Payment Method to establish prepaid credit.
5. ### Free trials and beta services
5.1 Free Trials
From time to time, Mage may make available to Customer certain features and functionality of the Service on a free or trial basis (āFree Trialsā), including beta versions of non-public or pre-release features and functionality. If Customer registers for a Free Trial, Customer may access and use the applicable features and functionality, as made available, in accordance with this Agreement on a trial basis free of charge until the earlier of (i) the end of the period of the Free Trial for which Customer registered; (ii) the start date of Subscription to such features or functionality is purchased by Customer; and (iii) termination of the Free Trial by Mage in its sole discretion. Notwithstanding anything to the contrary herein, Free Trials are not Service Plans nor are they part of a Service Plan. Free Trials may be subject to, and Customerās use of the applicable features and functionality is conditioned on its compliance with any applicable Supplemental Terms.
5.2 Disclaimer
NOTWITHSTANDING ANYTHING TO THE CONTRARY HEREIN, ALL FREE TRIALS ARE PROVIDED ON AN āAS ISā āAS AVAILABLE BASISā WITHOUT ANY WARRANTIES OF ANY KIND WHETHER EXPRESS OR IMPLIED AND MAGE MAKES NO PROMISES AND WILL HAVE NO OBLIGATION WITH RESPECT TO THE AVAILABILITY OR PERFORMANCE OF THE FEATURES AND FUNCTIONALITY OF ANY FREE TRIAL. FOR THE AVOIDANCE OF DOUBT, MAGE WILL HAVE NO OBLIGATIONS UNDER SECTIONS 2.4, 9.2, 10.1, OR 11.2 OF THIS MSA WITH RESPECT TO ANY FREE TRIALS. ANY SUBMITTED DATA PROVIDED BY CUSTOMER AND ANY OUTPUTS PROVIDED TO CUSTOMER IN CONNECTION WITH A FREE TRIAL, AND ANY CONFIGURATIONS OR CUSTOMIZATIONS MADE TO A SERVICE BY OR FOR CUSTOMER, IN CONNECTION WITH A FREE TRIAL WILL BE PERMANENTLY LOST UNLESS CUSTOMER PURCHASES A SUBSCRIPTION TO THE SAME FEATURES AND FUNCTIONALITY AS COVERED BY THE TRIAL, PURCHASE THE APPLICABLE SERVICE, OR EXPORT SUCH SERVICE DATA, BEFORE THE END OF THE TRIAL PERIOD; PROVIDED THAT MAGE HAS NO OBLIGATION TO MAKE ANY FEATURES OR FUNCTIONALITY AVAILABLE AS A FREE TRIAL AVAILABLE ON A SUBSCRIPTION BASIS.
6. ### Proprietary rights
Customer acknowledges that Mage owns and retains all rights, title, and interest, including all intellectual property rights, in and to the Mage Technology, including all technology, software, algorithms, user interfaces, trade secrets, techniques, designs, inventions, works of authorship, and other tangible and intangible material and information pertaining thereto or included therein, and nothing in the Agreement shall preclude or restrict Mage from using or exploiting any concepts, ideas, techniques or know-how of or related to the Mage Technology or otherwise arising in connection with Mageās performance under the Agreement. Other than as expressly set forth in the Agreement, no licenses or other rights in or to the Mage Technology are granted to Customer and all such rights are hereby expressly reserved.
7. ### Confidentiality
7.1 Definition
āConfidential Informationā means: (i) any information disclosed, directly or indirectly, by or on behalf of one party (āDisclosing Partyā) to the other party (āReceiving Partyā) pursuant to the Agreement that is designated as āconfidential,ā or in some other manner to indicate its confidential nature; and (ii) any information that otherwise should reasonably be expected to be treated in a confidential manner based on the circumstances of its disclosure or the nature of the information itself. Without limiting the foregoing, (a) the Mage Technology, the functionality and performance of the Mage Technology, including any metrics pertaining thereto is the Confidential Information of Mage, and (b) the Submitted Data and Outputs are the Confidential Information of Customer. However, Confidential Information does not include any information that: (I) is or becomes generally known and available to the public through no act of the Receiving Party; (II) was already in the Receiving Partyās possession without a duty of confidentiality owed to the Disclosing Party at the time of disclosure by the Disclosing Party, as shown by the Receiving Partyās contemporaneous records; (III) is lawfully obtained by the Receiving Party from a third party who has the express right to make such disclosure; or (IV) is independently developed by the Receiving Party without breach of an obligation owed to the Disclosing Party.
7.2 Use: Maintenance
Neither party shall use the Confidential Information of the other party for any purpose except to exercise its rights and perform its obligations under the Agreement. Neither party shall disclose, or permit to be disclosed, either directly or indirectly, any Confidential Information of the other party, except: (i) to its advisors, or prospective investors or purchasers, in each case subject to written obligations of confidentiality, or (ii) where the Receiving Party becomes legally compelled to disclose Confidential Information, notwithstanding the Receiving Partyās having given the Disclosing Party prior notice of such legally compelled disclosure and a reasonable opportunity to seek a protective order or other confidential treatment for such Confidential information (if permitted by applicable law). Each party will take reasonable measures and care to protect the secrecy of, and avoid disclosure and unauthorized use of the other partyās Confidential Information, and will take at least those measures taken to protect its own most highly confidential information. Notwithstanding anything to the contrary herein, Customer agrees that Mage may also use and exploit in any manner on a worldwide, irrevocable, perpetual, royalty-free basis, any: (a) aggregated non-personally identifiable information related to any usage of the Service to operate and improve Mageās products and services; and (b) suggestions, requests and feedback provided by or on behalf of Customer regarding the Mage Technology.
8. ### Customer data
8.1 Submitted Data and Outputs
Mage will maintain commercially reasonable administrative, physical, and technical safeguards for protection of the security, confidentiality and integrity of Customer Data. Except as required in Mageās reasonable discretion for purposes of (iv) or (v) of this Section 8.1, Mage will not store any piece of Submitted Data for any longer from the date on which such Submitted Data is submitted than 365 days. Customer agrees that Mage may use Customer Data as necessary to make available the Service, perform its obligations hereunder, and improve the Service, including, without limitation, performing any required, usual, appropriate, or acceptable activities relating to the Service, such as: (i) providing or supporting the use of the Service and carrying out the business of which the Service is a part; (ii) training Mageās machine learning algorithms; (iii) carrying out any benefits, rights, and obligations relating to the Service; (iv) maintaining records relating to the Service; and (v) complying with any legal or self-regulatory obligations relating to the Service. Customer acknowledges and agrees that, notwithstanding anything to the contrary herein, Mage may, in its sole discretion, erase or delete from the Service any Customer Data that it reasonably believes is illegal, harmful, objectionable, lewd, not related to the function of or necessary for the use of the Service, or that Mage determines may, as a result of Mage possessing such data, harm Mageās business or reputation.
8.2 Post-termination Access to Customer Data
For thirty (30) days after the effective date of termination or expiration of a Subscription, upon Customerās request, Mage will make Customer Data in Mageās possession available to Customer for export or download as provided in the Documentation. Thereafter, Mage will have no obligation to maintain or provide any Customer Data, and Mage will, unless prohibited by law or legal order, delete Customer Data in the Service in accordance with its data deletion policies.
9. ### Indemnification
9.1 By Mage
Mage shall: (i) defend, or at its option settle, any claim brought against Customer by a third party to the extent it alleges that the Mage Technology made available to Customer pursuant to a Subscription or Customerās use, as authorized in the Agreement, of the Service pursuant to a Subscription during the Subscription Term constitutes a direct infringement of U.S. copyrights or trade secrets of any third party (an āInfringement Claimā); and (ii) pay any damages awarded in a final judgment (or amounts agreed in a monetary settlement) in any such Infringement Claim defended by Mage; provided that Customer provides Mage (I) prompt written notice of, (II) sole control over the defense and settlement of, and (III) all information and assistance reasonably requested by Mage in connection with the defense or settlement of, any such Infringement Claim. If any Infringement Claim is brought or threatened, Mage may, at its sole option and expense: (w) procure for Customer the right to continue to use the Mage Technology; (x) modify the Mage Technology, as applicable, to make it non-infringing; (y) replace the affected aspect of the Mage Technology with non-infringing technology having substantially similar capabilities; or (z) if none of the foregoing is commercially practicable, terminate the Agreement. Notwithstanding the foregoing, Mage will have no liability to Customer: (1) for any use of the Mage Technology in combination with software, products or services not provided by Mage; to the extent that the Mage Technology would not be infringing but for such combination or modification; (2) for Customerās failure to use the Mage Technology in accordance with the Agreement; or (3) for any claims related to Submitted Data.
9.2 Disclaimer
SECTIONĀ 7.1 STATES THE ENTIRE LIABILITY OF MAGE, AND THE EXCLUSIVE REMEDY OF CUSTOMER, WITH RESPECT TO ANY ACTUAL OR ALLEGED INFRINGEMENT OF ANY INTELLECTUAL PROPERTY RIGHTS BY MAGE, THE MAGE TECHNOLOGY, OR ANY PART THEREOF.
9.3 By Customer
Notwithstanding anything to the contrary in SectionĀ 7.1, Customer shall defend or, at its option, settle, any claim brought against Mage by a third party: (i) alleging that the use by or on behalf of Mage of the Submitted Data in accordance with the Agreement violates a third partyās rights; (ii) arising out of any unauthorized access or use of the Service by Customer or any Authorized Users or any third party utilizing any access credentials of Customer or any Authorized Users; (iii) arising out of the use of the Service in violation or in connection with a violation of applicable law; (iv) arising out of the operation of Customerās business; or (v) arising out of Customerās or Authorized Usersā breach of this Agreement. Customer will indemnify Mage for all costs, liabilities, damages, and expenses incurred by Mage (or the amount of any settlement entered into or approved in writing by Customer) with respect to such a claim. Mage shall provide Customer with: (a) prompt written notice of; (b) sole control over the defense and settlement of; and (c) all information and assistance reasonably requested by Customer in connection with the defense or settlement of, any such claim.
10. ### Warranties
10.1 Mutual Warranties
Each party represents and warrants to the other that (i) this Agreement has been duly executed and delivered and constitutes a valid and binding agreement enforceable against such party in accordance with its terms; (ii) no authorization or approval from any third party is required in connection with such partyās execution, delivery or performance of this Agreement; and (iii) the execution, delivery and performance of the Agreement does not and will not violate the terms or conditions of any other agreement to which it is a party or by which it is otherwise bound.
10.2 Warranties of Mage
Mage warrants that during an applicable Subscription Term (i) this Agreement and the Documentation will accurately describe the applicable administrative, physical, and technical safeguards for protection of the security, confidentiality and integrity of Customer Data; and (ii) the Service Plan will perform materially in accordance with the applicable Documentation. For any breach of a warranty in this section, Customerās exclusive remedies are those described in Section 4.3 to the extent such a breach constitutes a material breach of the Agreement.
11. ### DISCLAIMER
EXCEPT AS EXPRESSLY SET FORTH IN THE AGREEMENT, MAGE HEREBY DISCLAIMS ALL WARRANTIES, WHETHER EXPRESS, IMPLIED OR STATUTORY, INCLUDING ANY AND ALL WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, NON-INFRINGEMENT, LOSS OF DATA, OR ACCURACY OF RESULTS. MAGE DOES NOT WARRANT THAT THE MAGE TECHNOLOGY WILL BE ERROR-FREE, UNINTERRUPTED, OR COMPATIBLE WITH ANY PARTICULAR DEVICE, THAT ANY DATA PROVIDED BY OR THROUGH THE MAGE TECHNOLOGY, INCLUDING PROVIDED DATA, WILL BE ACCURATE OR COMPLETE, OR, EXCEPT AS EXPRESSLY SET FORTH HEREIN, THAT MAGEāS SECURITY MEASURES WILL BE SUFFICIENT TO PREVENT THIRD PARTY ACCESS TO SUBMITTED DATA. CUSTOMER ACKNOWLEDGES AND AGREES THAT CUSTOMER BEARS ALL RESPONSIBILITY, AND MAGE WILL HAVE NO LIABILITY FOR DECISIONS BASED ON ANY OUTPUTS, OR ANY OTHER INFORMATION PROVIDED TO CUSTOMER VIA THE SERVICE OR BY MAGE.
12. ### LIMITATION OF LIABILITY
EXCEPT FOR EITHER PARTYāS INDEMNIFICATION OBLIGATIONS: (i) IN NO EVENT WILL EITHER PARTY BE LIABLE FOR ANY INDIRECT, SPECIAL, INCIDENTAL, EXEMPLARY, PUNITIVE, OR CONSEQUENTIAL DAMAGES ARISING OUT OF OR RELATING TO THE AGREEMENT, HOWEVER CAUSED, AND BASED ON ANY THEORY OF LIABILITY, WHETHER FOR BREACH OF CONTRACT, TORT (INCLUDING NEGLIGENCE), OR OTHERWISE, EVEN IF A PARTY HAS BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGES; AND (ii) NEITHER PARTYāS TOTAL LIABILITY (INCLUDING ATTORNEYSā FEES) ARISING OUT OF OR RELATED TO THE AGREEMENT (EXCEPT FOR CUSTOMERāS PAYMENT OBLIGATIONS) WILL EXCEED THE AMOUNT PAID BY CUSTOMER HEREUNDER DURING THE 12-MONTH PERIOD PRIOR TO THE DATE THE CLAIM AROSE. THESE LIMITATIONS SHALL APPLY NOTWITHSTANDING ANY FAILURE OF THE ESSENTIAL PURPOSE OF ANY LIMITED OR EXCLUSIVE REMEDY. CUSTOMER ACKNOWLEDGES AND AGREES THAT THE ESSENTIAL PURPOSE OF THIS SECTION 15.2 IS TO ALLOCATE THE RISKS UNDER THIS AGREEMENT BETWEEN THE PARTIES AND LIMIT POTENTIAL LIABILITY GIVEN THE SUBSCRIPTION CHARGES AND CONSULTING FEES, WHICH WOULD HAVE BEEN SUBSTANTIALLY HIGHER IF MAGE WERE TO ASSUME ANY FURTHER LIABILITY OTHER THAN AS SET FORTH HEREIN. MAGE HAS RELIED ON THESE LIMITATIONS IN DETERMINING WHETHER TO PROVIDE CUSTOMER WITH THE RIGHTS TO ACCESS AND USE THE SERVICES AND/OR THE CONSULTING SERVICES PROVIDED FOR IN THIS AGREEMENT.
13. ### General provisions
13.1 Assignment
Neither party may assign the Agreement or any of its rights or obligations under the Agreement without the prior written consent of the other party, except that Mage may assign the Agreement without the consent of Customer as part of a corporate reorganization, or upon a change of control, consolidation, merger, sale of all or substantially all of its business or assets related to the Agreement, or a similar transaction or series of transactions. Subject to the foregoing, the Agreement will be binding upon and inure to the benefit of the parties and their respective successors and permitted assigns.
13.2 Force Majeure
Except for the obligation to pay money, neither party will be liable for any failure or delay in its performance under the Agreement due to any cause beyond its reasonable control, including without limitation an act of war, terrorism, act of God, earthquake, flood, pandemic, embargo, riot, sabotage, labor shortage or dispute, governmental act, failure or degradation of the Internet or any third party service provider. The delayed party shall use its commercially reasonable efforts to correct such failure or delay in performance.
13.3 Governing Law
The Agreement shall be governed by and construed under the laws of the State of California without reference to conflict of laws principles. The application of the United Nations Convention on Contracts for the International Sale of Goods is expressly excluded. Subject first to Section 10.6, if a lawsuit or court proceeding is permitted under the Agreement, the parties will be subject to the exclusive jurisdiction of the state and federal courts located in San Francisco County, California, and the parties hereby agree and consent to the exclusive jurisdiction and venue of such courts.
13.4 Publicity
Mage may use Customerās name as a reference for marketing or promotional purposes on Mageās website and in other communication with existing or potential Mage customers; subject to Customerās prior approval of any such use.
13.5 Third-Party Links
The Service may contain links to or offer integrations with third-party websites and services, including supported third party cloud-based applications that may integrate with the Service (āThird-Party Linksā). Such Third-Party Links are not under the control of Mage, and Mage is not responsible for any Third-Party Links. Mage provides access to these Third-Party Links only as a convenience to Customer and does not review, approve, monitor, endorse, warrant, or make any representations with respect to Third-Party Links. Customer uses all Third-Party Links at its own risk, and should apply a suitable level of caution and discretion in doing so. When Customer uses any of the Third-Party Links, the applicable third partyās terms and policies apply, including the third partyās privacy and data gathering practices. Customer should make whatever investigation it feels necessary or appropriate before proceeding with any transaction in connection with such Third-Party Links.
13.6 Government Rights
Mage provides the Mage Technology, including any related software, data, and technology, for ultimate government end use solely in accordance with the following: The Service and API shall constitute ācommercialā computer software. Government technical data and software rights related to the Service and API include only those rights customarily provided to the public as defined in the Agreement. These customary commercial licenses are provided in accordance with FAR 12.211 (Technical Data) and FAR 12.212 (Customer-Side Application) and, for Department of Defense transactions, DFAR 252.227-7015 (Technical Data ā Commercial Items) and DFAR 227.7202-3 (Rights in Commercial Computer Customer-Side Application or Computer Customer-Side Application Documentation). If a government agency has a need for rights not granted under these terms, it must negotiate with Mage to determine if there are acceptable terms for granting those rights, and a mutually acceptable written addendum specifically granting those rights must be included in any applicable agreement.
13.7 Arbitration
The parties agree to resolve all disputes arising under or in connection with the Agreement through binding arbitration. A party who intends to seek arbitration must first send a written notice of the dispute to the other party. The parties will use good faith efforts to resolve the dispute directly, but if the parties do not reach an agreement to do so within 30 days after the notice is received, either party may commence an arbitration proceeding. The arbitration will be conducted in accordance with the applicable rules of the American Arbitration Association (āAAAā). The arbitration will be conducted in English in San Francisco, California, USA. If the parties do not agree on an arbitrator, the arbitrator will be selected in accordance with the applicable rules of the AAA for the appointment of an arbitrator. The selection of an arbitrator under the rules of the AAA will be final and binding on the parties. The arbitrator must be independent of the parties. The arbitratorās decision will be final and binding on both parties, and the arbitrator must issue a reasoned written decision sufficient to explain the essential findings and conclusions on which the decision and award, if any, are based. The costs and expenses of the arbitration will be shared equally by both parties; however, if the arbitrator finds that either the substance of the claim or the relief sought in arbitration is frivolous or brought for an improper purpose (as measured by the standards set forth in Federal Rule of Civil Procedure 11(b)), then the payment of all fees will be governed by the AAA Rules. Notwithstanding the foregoing, this SectionĀ 10.6 will not prohibit either party from: (i) bringing an individual action in small claims court; (ii) seeking injunctive or other equitable relief in a court of competent jurisdiction; (iii) pursuing an enforcement action through the applicable federal, state, or local agency if that action is available; or (iv) filing suit in a court of law to address an intellectual property infringement or misappropriation claim. If this SectionĀ 10.6 is found to be unenforceable, the parties agree that the exclusive jurisdiction and venue described in SectionĀ 10.3 will govern any action arising out of or related to the Agreement.
13.8 Miscellaneous
Mage and Customer are independent contractors. This Agreement does not create a partnership, franchise, joint venture, agency, fiduciary or employment relationship between Mage and Customer. In the event of any inconsistency or conflict between the terms of the Agreement and the terms of any Order, the terms of the Order shall control. The Agreement is the sole agreement of the parties concerning the subject matter hereof, and supersedes all prior agreements and understandings with respect to said subject matter. Customer may not subcontract or delegate any rights or obligations granted to it under the Agreement to any third parties, including its consultants or contractors, without the prior written consent of Mage. No terms of any purchase order, acknowledgement, or other form provided by Customer in connection with an Order will modify the Agreement, regardless of any failure of Mage to object to such terms. Any ambiguity in the Agreement shall be interpreted without regard to which party drafted the Agreement or any part thereof. There are no third party beneficiaries to the Agreement. The Agreement may only be amended by a writing signed by both parties. The Agreement may be executed in counterparts. The headings in the Agreement are inserted for convenience and are not intended to affect the interpretation of the Agreement. All notices provided by Mage to Customer under this Agreement may be delivered in writing by (a) nationally recognized overnight delivery service or U.S. mail to the contact mailing address provided by Customer; or (b) electronic mail to the electronic mail address provided for the Customer Account. Customer must give notice to Mage in writing by nationally recognized overnight delivery service or U.S. mail to the address set forth above. All notices shall be deemed to have been given immediately upon delivery by electronic mail; or, if otherwise delivered upon the earlier of receipt or two (2) business days after being deposited in the mail or with a courier as permitted above. Mage may use subcontractors or otherwise delegate aspects of its performance under the Agreement; provided that Mage shall remain responsible hereunder for any such subcontractorās performance. Waiver of any term of the Agreement or forbearance to enforce any term by either party shall not constitute a waiver as to any subsequent breach or failure of the same term or a waiver of any other term of the Agreement. Any provision found to be unlawful, unenforceable or void shall be severed from the remainder of the Agreement and the remainder of the Agreement will continue in full force and effect without said provision. The parties agree to comply with all applicable export control laws and regulations related to their performance of the Agreement.
14. ### Definitions
āAgreementā
means this MSA together with all Orders, Statements of Work, and Supplemental Terms, all of which are hereby incorporated by reference, and all other terms and conditions have been incorporated herein by reference.
āAPIā
means any application programming interface for sending data to or receiving data from the Service and any corresponding software libraries that are made available to Customer.
āAuthorized Usersā
means individuals who are Customerās employee or contractor personnel authorized by Customer to access and use the Service.
āCustomer Dataā
means Submitted Data and Outputs.
āDashboardā
means the web based user interface for Customer to access portions of the Service.
āDocumentationā
means any user instructions, manuals, on-line help files, or other materials that are provided by Mage in connection with the Service or API.
āMage Technologyā
means, collectively, the Service, API, Documentation, and any other services to be provided pursuant to the Agreement.
āOrderā
means a generated service order form(s) or online ordering document or process completed, executed or approved by Customer with respect to a Subscription to the Service, which may detail, among other things, the number of Authorized Users and the Service Plan applicable to the Subscription.
āOutputsā
means outputs or results based on the processing of Submitted Data via the Service that are returned to Customer.
āServiceā
means Mageās proprietary Software-as-a-Service platform for Machine Learning.
āService Planā
means the features and functionality of the Service as well as any limitations or quantities or usage or Authorized Users applicable to the Subscription purchased by Customer.
āSubmitted Dataā
means any data pertaining to that is submitted by Customer or Authorized Users to the Service to be processed by the Service.
āSubscriptionā
means the right to access and use certain services and features of the Service for a particular period of time.
[Try Mage](https://mage.ai/sign-up)
[Help & support](https://www.mage.ai/chat)
[](./)
Platform
Explore
Pricing
[Community](https://www.mage.ai/chat)
[Get a demo](./getdemo)
[Start now](https://cloud.mage.ai/sign-up)
[](./)
Your AI data engineer
[Get started](https://mage.ai/sign-up)
[Request a demo](./getdemo)
Product
[AI](./ai)
[Data pipelines](./data)
[Developer experience](./build)
[Deployments](./deploy)
[Scalability](./scale)
[Customizations](./extend)
Resources
[Pricing](./pricing)
[Documentation](https://docs.mage.ai/)
[Blog](./blog)
[Platform features](./specs)
[Unique differentiators](./innovations)
[Mage Pro vs OSS](./oss-vs-pro)
[Changelog](./updates)
Getting started
[Quickstart](https://docs.mage.ai/getting-started/setup)
[Guides](https://docs.mage.ai/guides/overview)
[Templates](./commons)
[Core abstractions](https://docs.mage.ai/design/core-abstractions)
[Dynamic blocks](https://docs.mage.ai/design/blocks/dynamic-blocks)
Tools
[Help & support](https://www.mage.ai/chat)
[Migration support](./support-plan)
[Community](https://www.mage.ai/chat)
[Partner program](./partners)
Company
[Terms](./terms)
[Privacy Policy](./privacy)
Ā© 2025 Mage Technologies, Inc.
Your AI data engineer
[Get started](https://mage.ai/sign-up)
[Request a demo](./getdemo)
Product
[AI](./ai)
[Data pipelines](./data)
[Developer experience](./build)
[Deployments](./deploy)
[Scalability](./scale)
[Customizations](./extend)
Resources
[Pricing](./pricing)
[Documentation](https://docs.mage.ai/)
[Blog](./blog)
[Platform features](./specs)
[Unique differentiators](./innovations)
[Mage Pro vs OSS](./oss-vs-pro)
[Changelog](./updates)
Getting started
[Quickstart](https://docs.mage.ai/getting-started/setup)
[Guides](https://docs.mage.ai/guides/overview)
[Templates](./commons)
[Core abstractions](https://docs.mage.ai/design/core-abstractions)
[Dynamic blocks](https://docs.mage.ai/design/blocks/dynamic-blocks)
Tools
[Help & support](https://www.mage.ai/chat)
[Migration support](./support-plan)
[Community](https://www.mage.ai/chat)
[Partner program](./partners)
Company
[Terms](./terms)
[Privacy Policy](./privacy)
Ā© 2025 Mage Technologies, Inc.
---
# Privacy Policy | Data Protection and Security | Mage AI
Privacy.
[Info we collect](./privacy#1)
[How we use](./privacy#2)
[How we share](./privacy#3)
[Sites & services](./privacy#4)
[Security](./privacy#5)
[International data transfer](./privacy#6)
[Children](./privacy#7)
[Notice to end users](./privacy#8)
[Changes to Policy](./privacy#9)
Privacy Policy
==============
Effective as of December 11th, 2020
This Privacy Policy describes the privacy practices of Mage Technologies, Inc. (collectively āMageā, āweā, āusā or āourā) regarding personal information that we collect through our website and online application that link to this privacy policy (collectively, the āServiceā), as well as through social media, our marketing activities, and other activities described in this Privacy Policy.
Mage provides tools for businesses and developers to analyze their own data. This Privacy Policy does not apply to any data that our customers ingest into the Service for analysis which we process as a processor/service provider in accordance with our contracts with those customers.
1. ### Personal information we collect
1.1 Information you provide to us
Personal information you may provide to us through the Service or otherwise, includes:
* Contact data, such as your first and last name, email address, phone number, country, professional title, and organizational affiliation.
* Account data, such as the username, password, API key or other credentials associated with an account on the Service, account preferences, and any other registration data associated with a Service account.
* User-submitted data, which includes any data that individual customers (as opposed to business customers) submit to the Service for processing and analysis.
* Communications that we exchange with you, including when you contact us with questions or feedback, through social media, or otherwise.
* Payment and transactional data, such as your payment card information, billing information and purchase history.
* Marketing data, such as your preferences for receiving our marketing communications and details about your engagement with them.
* Other data not specifically listed here, which we will use as described in this Privacy Policy or as otherwise disclosed at the time of collection.
1.2 Third party sources
We may combine personal information we receive from you with personal information we obtain from other sources, such as:
* Administrators, where your Service account is created and administered for you by an administrator.
* Public sources, such as social media platforms.
1.3 Automatic data collection
We, our service providers, and our business partners may automatically log information about you, your computer or mobile device, and your interaction over time with the Service, our communications and other online services, such as:
* Device data, such as your computerās or mobile deviceās operating system type and version, manufacturer and model, browser type, screen resolution, RAM and disk size, CPU usage, device type (e.g., phone, tablet), IP address, unique identifiers, language settings, mobile device carrier, radio/network information (e.g., WiFi, LTE, 3G), and general location information such as city, state or geographic area.
* Online activity data, such as pages you viewed, how long you spent on a page, the website you visited before browsing to the Service, navigation paths between pages, information about your activity on a page, access times, and duration of access, and whether you have opened our marketing emails or clicked links within them.
2. ### How we use your personal information
We use your personal information for the following purposes or as otherwise described at the time of collection:
2.1 Service delivery
We use your personal information to:
* provide, operate and improve the Service and our business;
* process your payments for the Service;
* communicate with you about the Service, including by sending announcements, updates, security alerts, and support and administrative messages;
* understand your needs and interests, and personalize your experience with the Service and our communications; and
* provide support for the Service, and respond to your requests, questions and feedback.
2.2 Research and development
We may use your personal information for research and development purposes, including to analyze and improve the Service and our business. As part of these activities, we may create aggregated, de-identified or other anonymous data from personal information we collect. We make personal information into anonymous data by removing information that makes the data personally identifiable to you. We may use this anonymous data and share it with third parties for our lawful business purposes, including to analyze and improve the Service and promote our business.
2.3 Marketing
We may send you Mage-related direct marketing communications as permitted by law, including by email. You may opt-out of our marketing communications as described in the Opt-out of marketing section below.
2.4 Compliance and protection
We may use your personal information to:
* comply with applicable laws, lawful requests, and legal process, such as to respond to subpoenas or requests from government authorities;
* protect our, your or othersā rights, privacy, safety or property (including by making and defending legal claims);
* audit our internal processes for compliance with legal and contractual requirements and internal policies;
* enforce the terms and conditions that govern the Service; and
* prevent, identify, investigate and deter fraudulent, harmful, unauthorized, unethical or illegal activity, including cyberattacks and identity theft.
3. ### How we share your personal information
We may share your personal information with the following parties and as otherwise described in this Privacy Policy or at the time of collection:
3.1 Service providers
Companies and individuals that provide services on our behalf or help us operate the Service or our business (such as hosting, information technology, customer support, email delivery, marketing, and analytics).
3.2 Payment processors
Any payment card information you use to make a purchase on the Service is collected and processed directly by our payment processors, such as Stripe, who may use your payment data in accordance with its Privacy Policy, [https://stripe.com/privacy](https://stripe.com/privacy)
.
3.3 Professional advisors
Professional advisors, such as lawyers, auditors, bankers and insurers, where necessary in the course of the professional services that they render to us.
3.4 Authorities and others
Law enforcement, government authorities, and private parties, as we believe in good faith to be necessary or appropriate for the compliance and protection purposes described above.
3.5 Business transferees
Acquirers and other relevant participants in business transactions (or negotiations for such transactions) involving a corporate divestiture, merger, consolidation, acquisition, reorganization, sale or other disposition of all or any portion of the business or assets of, or equity interests in, Mage or our affiliates (including, in connection with a bankruptcy or similar proceedings).
4. ### Your choices
You have the following choices with respect to your personal information. Where the Services are administered for you by an administrator (see Notice to End Users below), you may need to contact your administrator to assist with your requests first.
4.1 Access or update your information
If you have registered for a Service account with us, you may review and update certain account information by logging into the account.
4.2 Opt-out of marketing communications
You may opt-out of marketing-related emails by following the opt-out or unsubscribe instructions at the bottom of the email, or by contacting us. You may continue to receive service-related and other non-marketing emails.
4.3 Do not track
Some Internet browsers may be configured to send āDo Not Trackā signals to the online services that you visit. We currently do not respond to āDo Not Trackā or similar signals. To find out more about āDo Not Track,ā please visit [http://www.allaboutdnt.com](http://www.allaboutdnt.com)
.
5. ### Other sites and services
The Service may contain links to websites, mobile applications, and other online services operated by third parties. In addition, our content may be integrated into web pages or other online services that are not associated with us. These links and integrations are not an endorsement of, or representation that we are affiliated with, any third party. We do not control websites, mobile applications or online services operated by third parties, and we are not responsible for their actions.
6. ### Security
We employ a number of technical, organizational and physical safeguards designed to protect the personal information we collect. However, security risk is inherent in all internet and information technologies and we cannot guarantee the security of your personal information.
7. ### International data transfer
We are headquartered in the United States and may use service providers that operate in other countries. Your personal information may be transferred to the United States or other locations where privacy laws may not be as protective as those in your state, province, or country.
8. ### Children
The Service is not intended for use by children under 16 years of age. If we learn that we have collected personal information through the Service from a child under 16 without the consent of the childās parent or guardian as required by law, we will delete it.
9. ### Notice to end users
Where the Services are made available to you through an organization (e.g. your employer), that organization is the administrator of the Services and is responsible for the accounts over which it has control. If this is the case, please direct your data privacy questions to your administrator, as your use of the Services is subject to that organizationās policies. We are not responsible for the privacy or security practices of an administratorās organization, which may be different than this policy.
Administrators are able to require you to reset your account password; restrict, suspend or terminate your access to the Services; access and retain information in and about your account; change the email address associated with your account; change your information, including profile information; and restrict your ability to edit, restrict, modify or delete information.
10. ### Changes to this Privacy Policy
We reserve the right to modify this Privacy Policy at any time. If we make material changes to this Privacy Policy, we will notify you by updating the date of this Privacy Policy and posting it on the Service. If required by law we will also provide notification of changes in another way that we believe is reasonably likely to reach you, such as via email or another manner through the Service. Any modifications to this Privacy Policy will be effective upon our posting the modified version (or as otherwise indicated at the time of posting). In all cases, your use of the Service after the effective date of any modified Privacy Policy indicates your acceptance of the modified Privacy Policy.
[Try Mage](https://mage.ai/sign-up)
[Help & support](https://www.mage.ai/chat)
[](./)
Platform
Explore
Pricing
[Community](https://www.mage.ai/chat)
[Get a demo](./getdemo)
[Start now](https://cloud.mage.ai/sign-up)
[](./)
Your AI data engineer
[Get started](https://mage.ai/sign-up)
[Request a demo](./getdemo)
Product
[AI](./ai)
[Data pipelines](./data)
[Developer experience](./build)
[Deployments](./deploy)
[Scalability](./scale)
[Customizations](./extend)
Resources
[Pricing](./pricing)
[Documentation](https://docs.mage.ai/)
[Blog](./blog)
[Platform features](./specs)
[Unique differentiators](./innovations)
[Mage Pro vs OSS](./oss-vs-pro)
[Changelog](./updates)
Getting started
[Quickstart](https://docs.mage.ai/getting-started/setup)
[Guides](https://docs.mage.ai/guides/overview)
[Templates](./commons)
[Core abstractions](https://docs.mage.ai/design/core-abstractions)
[Dynamic blocks](https://docs.mage.ai/design/blocks/dynamic-blocks)
Tools
[Help & support](https://www.mage.ai/chat)
[Migration support](./support-plan)
[Community](https://www.mage.ai/chat)
[Partner program](./partners)
Company
[Terms](./terms)
[Privacy Policy](./privacy)
Ā© 2025 Mage Technologies, Inc.
Your AI data engineer
[Get started](https://mage.ai/sign-up)
[Request a demo](./getdemo)
Product
[AI](./ai)
[Data pipelines](./data)
[Developer experience](./build)
[Deployments](./deploy)
[Scalability](./scale)
[Customizations](./extend)
Resources
[Pricing](./pricing)
[Documentation](https://docs.mage.ai/)
[Blog](./blog)
[Platform features](./specs)
[Unique differentiators](./innovations)
[Mage Pro vs OSS](./oss-vs-pro)
[Changelog](./updates)
Getting started
[Quickstart](https://docs.mage.ai/getting-started/setup)
[Guides](https://docs.mage.ai/guides/overview)
[Templates](./commons)
[Core abstractions](https://docs.mage.ai/design/core-abstractions)
[Dynamic blocks](https://docs.mage.ai/design/blocks/dynamic-blocks)
Tools
[Help & support](https://www.mage.ai/chat)
[Migration support](./support-plan)
[Community](https://www.mage.ai/chat)
[Partner program](./partners)
Company
[Terms](./terms)
[Privacy Policy](./privacy)
Ā© 2025 Mage Technologies, Inc.
---
# Innovations | Unique Differentiators | Mage AI
Unique differentiators and key innovations
Better by design
================
Mage AI redefines data pipeline development through a developer-first approach and architectural innovations that streamline end-to-end workflow management.
By combining cutting-edge coding tools with modular design principles, Mage simplifies complex data engineering tasks while maintaining enterprise-grade scalability.
=======================================================================================================================================================================================================================================================================================================================================
Modular block design
====================
Mage AIās modular architecture uses standalone code blocks (individual files) to streamline data engineering workflows. Each pipeline block exists as an independent file that processes and passes data downstream.
This design enables blocks to operate as standalone units with isolated dependencies and execution environments.
==========================================================================================================================================================================================================================================================================================================================================
[Explore Blocks](./build#compose)
[Explore Blocks](./build#compose)
Data engineering-centric workflow
=================================
Great design isnāt just about looksāitās about usability. Mage simplifies complex data operations with a clean, structured experience that enhances productivity and reduces cognitive load.
Developers adopt Mage faster because they donāt have to learn a new syntax specific to Mage, loweringĀ the barrier toĀ entry for newĀ users.
By avoidingĀ domain-specific languagesĀ (DSLs) and focusingĀ on functionalĀ programming, developers canĀ use familiarĀ programming paradigms without the need to read documentation.
This creates a smootherĀ migration forĀ developers fromĀ other platformsĀ and a quickerĀ start to productiveĀ work.
==================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================
[Explore Dev Ex](./build)
[Explore Dev Ex](./build)
Unified pipeline builder and orchestration
==========================================
Build, deploy, orchestrate, and monitor batch processing pipelines, data integration pipelines, and real-time streaming pipelines; all from a single cohesive data platform with a consumer-grade user experience, simplifying the workflow for developers so that they no longer need to buy and learn different tools.
This unified approach reduces costs, simplifies the data pipeline architecture, and streamlines the development process.
======================================================================================================================================================================================================================================================================================================================================================================================================================================================
[Explore Data Pipelines](./data)
[Explore Data Pipelines](./data)
Dynamic scalability
===================
Mage AIās dynamic blocks revolutionize pipeline architecture through adaptive parallelism and context-aware execution; transform static code from rigid sequences into living neural networks of data processing.
Dynamic blocks represent a paradigm shift in data pipeline orchestration, enabling intelligent workload distribution and runtime flexibility that sets the platform apart from traditional ETL tools.
This innovation fundamentally changes how engineers handle variable data loads and complex processing requirements.
=====================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================
[Explore Scalability](./scale)
[Explore Scalability](./scale)
Flexible extensibility
======================
Mage's extensibility suite transforms data platforms from static tools intoĀ living enterprise organismsĀ - breathing with custom logic, evolving via API integration, and protected by atomic security controls.
This level of flexibility allows organizations to seamlessly integrate Mage into their existing ecosystems while maintaining stringent security and compliance standards.
==============================================================================================================================================================================================================================================================================================================================================================================================
[Explore Customizations](./extend)
[Explore Customizations](./extend)
Recover your precious developer time
Now you can focus on the fun, creative, and high-impact data engineering projects and let Mage AI handle the rest.
For engineers
Experience how Mage AI ships data pipelines faster, giving you a better work-life-balance.
[Get started](https://mage.ai/sign-up)
For data teams
See how Mage AI accelerates your team velocity while reducing data and infrastructure costs.
[Enterprise demo](./getdemo)
[Explore the platform](./build)
[\
\
AI\
\
Code faster, smarter](./ai)
[\
\
Data pipelines\
\
Build, run, and monitor](./data)
[\
\
Dev Ex\
\
Made for speed and ease](./build)
[\
\
Deployments\
\
Ship workflows effortlessly](./deploy)
[\
\
Scalabiity\
\
Grow without limits](./scale)
[\
\
Customizations\
\
Total extensibility](./extend)
Recover your precious developer time
Now you can focus on the fun, creative, and high-impact data engineering projects and let Mage AI handle the rest.
For engineers
Experience how Mage AI ships data pipelines faster, giving you a better work-life-balance.
[Get started](https://mage.ai/sign-up)
For data teams
See how Mage AI accelerates your team velocity while reducing data and infrastructure costs.
[Enterprise demo](./getdemo)
[Explore the platform](./build)
[\
\
AI\
\
Code faster, smarter](./ai)
[\
\
Data pipelines\
\
Build, run, and monitor](./data)
[\
\
Dev Ex\
\
Made for speed and ease](./build)
[\
\
Deployments\
\
Ship workflows effortlessly](./deploy)
[\
\
Scalabiity\
\
Grow without limits](./scale)
[\
\
Customizations\
\
Total extensibility](./extend)
Product
[AI](./ai)
[Data pipelines](./data)
[Developer experience](./build)
[Deployments](./deploy)
[Scalability](./scale)
[Customizations](./extend)
Resources
[Pricing](./pricing)
[Documentation](https://docs.mage.ai/)
[Blog](./blog)
[Platform features](./specs)
[Unique differentiators](./innovations)
[Mage Pro vs OSS](./oss-vs-pro)
[Changelog](./updates)
Getting started
[Quickstart](https://docs.mage.ai/getting-started/setup)
[Guides](https://docs.mage.ai/guides/overview)
[Templates](./commons)
[Core abstractions](https://docs.mage.ai/design/core-abstractions)
[Dynamic blocks](https://docs.mage.ai/design/blocks/dynamic-blocks)
Tools
[Help & support](https://www.mage.ai/chat)
[Migration support](./support-plan)
[Community](https://www.mage.ai/chat)
[Partner program](./partners)
Company
[Terms](./terms)
[Privacy Policy](./privacy)
Ā© 2025 Mage Technologies, Inc.
Product
[AI](./ai)
[Data pipelines](./data)
[Developer experience](./build)
[Deployments](./deploy)
[Scalability](./scale)
[Customizations](./extend)
Resources
[Pricing](./pricing)
[Documentation](https://docs.mage.ai/)
[Blog](./blog)
[Platform features](./specs)
[Unique differentiators](./innovations)
[Mage Pro vs OSS](./oss-vs-pro)
[Changelog](./updates)
Getting started
[Quickstart](https://docs.mage.ai/getting-started/setup)
[Guides](https://docs.mage.ai/guides/overview)
[Templates](./commons)
[Core abstractions](https://docs.mage.ai/design/core-abstractions)
[Dynamic blocks](https://docs.mage.ai/design/blocks/dynamic-blocks)
Tools
[Help & support](https://www.mage.ai/chat)
[Migration support](./support-plan)
[Community](https://www.mage.ai/chat)
[Partner program](./partners)
Company
[Terms](./terms)
[Privacy Policy](./privacy)
Ā© 2025 Mage Technologies, Inc.
[](./)
Platform
Explore
Pricing
[Community](https://www.mage.ai/chat)
[Get a demo](./getdemo)
[Start now](https://cloud.mage.ai/sign-up)
[](./)
---
# Pricing | Find a Mage Pro Plan | Mage AI
Prototype
Free
Build & experiment.
Pipeline runtime unavailable.
[Subscribe](https://cloud.mage.ai/sign-up)
[Subscribe](https://cloud.mage.ai/sign-up)
On-demand
$100 / mo
$0.29 /
Billed per 1 CPU-hour or 4 GB RAM-hour of pipeline runtime \*
[Subscribe](https://cloud.mage.ai/sign-up)
[Subscribe](https://cloud.mage.ai/sign-up)
Enterprise
$Custom
Custom /
Billed per 1 CPU-hour or 4 GB RAM-hour of pipeline runtime \*
[Contact Us](mainto:pro@mage.ai)
[Contact Us](mainto:pro@mage.ai)
1 project
Auto-termination after idle
Risk-free trial
2 CPU / 8 GB memory base cluster included
Unlimited autoscaling pipelines
3 day data retention
Customizable environments
AI for coding and debugging pipelines
Built-in version control and CI/CD
Unlimited code storage
Product walkthrough
\*Usage cost is measured by how much CPU or GBs of memory is consumed when a pipeline is run from a trigger, not from the code editor.
Risk-free trial
Custom CPU / GB memory base cluster
Unlimited autoscaling pipelines
3 day data retention
Customizable environments
AI for coding and debugging pipelines
Built-in version control and CI/CD
Unlimited code storage
Multiple workspaces for collaboration
Custom deployment in any region
Single sign-on, VPN, and more
Higher resource limits
Support for dedicated requirements
\*Usage cost is measured by how much CPU or GBs of memory is consumed when a pipeline is run from a trigger, not from the code editor.
Deployment options
Deployment options
Mage AI can be used immediately as a fully managed cloud service or be fully customized and deployed into your private infrastructure.
[Contact us](mailto:pro@mage.ai)
[Learn more](./setup)
[\
\
Managed cloud\
\
Best for teams focused on building pipelines fast.\
\
* Fully hosted and managed by Mage\
\
\
* Zero infrastructure management\
\
\
* Instant setup and automatic updates](./setup#managed)
[\
\
Hybrid cloud\
\
Ideal for companies with strict compliance needs.\
\
* Control plane in Mageās cloud\
\
\
* Data processing in your private cloud\
\
\
* Enhanced data privacy and control](./setup#hybrid-cloud)
[\
\
Private cloud\
\
Perfect for enterprise-grade security requirements.\
\
* Full platform deployed in your cloud\
\
\
* Complete control over all components\
\
\
* End-to-end security and isolation](./setup#private-cloud)
[\
\
On-premises\
\
Suited for strict regulatory environments.\
\
* Deployed in your data center\
\
\
* Maximum data sovereignty\
\
\
* Complete infrastructure control](./setup#on-premises)
Support plans
Support plans
Mage AI offers flexible support options with three tiers of on-demand assistance, providing guaranteed response times, training sessions, feature request handling, migration help, and more
[Contact us](mailto:pro@mage.ai)
[Learn more](./support-plan)
[\
\
Included\
\
Get up and running like magic. (On-demand and enterprise)\
\
* Curated platform walkthrough\
\
\
* Setup management accounts and provision resources\
\
\
* Apply custom configurations](./support-plan)
[\
\
Essential\
\
Core assistance to keep your operations running around the clock.\
\
* Migrate 10 existing pipelines\
\
\
* \>24 hr technical response\
\
\
* \>6 hr critical response](./support-plan#compare-plans)
[\
\
Extended\
\
Expanded support hours for when your workflows need extra coverage.\
\
* Migrate 40 existing pipelines\
\
\
* \>12 hr technical response\
\
\
* \>1 hr critical response](./support-plan#compare-plans)
[\
\
Complete\
\
Broad access to expert help, covering extended and critical times.\
\
* Migrate 100 existing pipelines\
\
\
* \>6 hr technical response\
\
\
* \>15 min critical response](./support-plan#compare-plans)
Features
Features
Mage is the best way to build, deploy, and run data pipelines. Explore the platform through the pages below or start a free trial to experience it firsthand.
[\
\
AI\
\
Code faster, smarter](./ai)
[\
\
Data pipelines\
\
Build, run, and monitor](./data)
[\
\
Dev Ex\
\
Made for speed and ease](./build)
[\
\
Deployments\
\
Ship workflows effortlessly](./deploy)
[\
\
Scalabiity\
\
Grow without limits](./scale)
[\
\
Customizations\
\
Total extensibility](./extend)
[Data Mesh](./data-mesh)
[Specs](./specs)
[Browse features](./explore)
Available upgrades
Available upgrades
Beyond our core features, we also offer additional premium capabilities for enhanced functionality.
[RAG & LLM Pipelines](./ai)
[Pipeline Version Control](./ai)
[Unified Pipeline Triggers](./ai)
[System Metrics & Dashboards](./ai)
[Secure Database Connections](./ai)
[Custom Domains](./ai)
Prototype
Free
Build & experiment.
Pipeline runtime unavailable.
[Subscribe](https://cloud.mage.ai/sign-up)
[Subscribe](https://cloud.mage.ai/sign-up)
On-demand
$100 / mo
$0.29 /
Billed per 1 CPU-hour or 4 GB RAM-hour of pipeline runtime \*
[Subscribe](https://cloud.mage.ai/sign-up)
[Subscribe](https://cloud.mage.ai/sign-up)
Enterprise
$Custom
Custom /
Billed per 1 CPU-hour or 4 GB RAM-hour of pipeline runtime \*
[Contact Us](mainto:pro@mage.ai)
[Contact Us](mainto:pro@mage.ai)
[](./)
Platform
Explore
Pricing
[Community](https://www.mage.ai/chat)
[Get a demo](./getdemo)
[Start now](https://cloud.mage.ai/sign-up)
[](./)
Product
[AI](./ai)
[Data pipelines](./data)
[Developer experience](./build)
[Deployments](./deploy)
[Scalability](./scale)
[Customizations](./extend)
Resources
[Pricing](./pricing)
[Documentation](https://docs.mage.ai/)
[Blog](./blog)
[Platform features](./specs)
[Unique differentiators](./innovations)
[Mage Pro vs OSS](./oss-vs-pro)
[Changelog](./updates)
Getting started
[Quickstart](https://docs.mage.ai/getting-started/setup)
[Guides](https://docs.mage.ai/guides/overview)
[Templates](./commons)
[Core abstractions](https://docs.mage.ai/design/core-abstractions)
[Dynamic blocks](https://docs.mage.ai/design/blocks/dynamic-blocks)
Tools
[Help & support](https://www.mage.ai/chat)
[Migration support](./support-plan)
[Community](https://www.mage.ai/chat)
[Partner program](./partners)
Company
[Terms](./terms)
[Privacy Policy](./privacy)
Ā© 2025 Mage Technologies, Inc.
Product
[AI](./ai)
[Data pipelines](./data)
[Developer experience](./build)
[Deployments](./deploy)
[Scalability](./scale)
[Customizations](./extend)
Resources
[Pricing](./pricing)
[Documentation](https://docs.mage.ai/)
[Blog](./blog)
[Platform features](./specs)
[Unique differentiators](./innovations)
[Mage Pro vs OSS](./oss-vs-pro)
[Changelog](./updates)
Getting started
[Quickstart](https://docs.mage.ai/getting-started/setup)
[Guides](https://docs.mage.ai/guides/overview)
[Templates](./commons)
[Core abstractions](https://docs.mage.ai/design/core-abstractions)
[Dynamic blocks](https://docs.mage.ai/design/blocks/dynamic-blocks)
Tools
[Help & support](https://www.mage.ai/chat)
[Migration support](./support-plan)
[Community](https://www.mage.ai/chat)
[Partner program](./partners)
Company
[Terms](./terms)
[Privacy Policy](./privacy)
Ā© 2025 Mage Technologies, Inc.
---
# Slack
We're very sorry, but your browser is not supported!
====================================================
Please upgrade to a [supported browser](https://slack.com/help/articles/115002037526-Minimum-requirements-for-using-Slack#web-browsers)
, or try one of our apps.
Desktop Apps
------------
[](https://itunes.apple.com/app/slack/id803453959?ls=1&mt=12)
### Mac
[See system requirements](https://slack.com/help/articles/115002037526-Minimum-requirements-for-using-Slack#desktop-apps)
[](https://itunes.apple.com/app/slack/id803453959?ls=1&mt=12 "Download on the Mac App Store")
v4.43.44
[](https://www.microsoft.com/en-us/store/p/slack/9wzdncrdk3wp)
### Windows
[See system requirements](https://slack.com/help/articles/115002037526-Minimum-requirements-for-using-Slack#desktop-apps)
[](https://www.microsoft.com/en-us/store/p/slack/9wzdncrdk3wp "Download on the Microsoft Store")
v4.43.43
[](https://snapcraft.io/slack)
### Linux
[See system requirements](https://slack.com/help/articles/115002037526-Minimum-requirements-for-using-Slack#desktop-apps)
[](https://snapcraft.io/slack "Download from the Snap Store")
v4.41.105
Mobile Apps
-----------
[](https://itunes.apple.com/app/slack-app/id618783545?ls=1&mt=8)
### iOS
[](https://itunes.apple.com/app/slack-app/id618783545?ls=1&mt=8 "Download on the App Store")
[](https://play.google.com/store/apps/details?id=com.Slack)
### Android
[](https://play.google.com/store/apps/details?id=com.Slack "Get it on Google Play")
Don't see the platform you're looking for? [Let us know.](/help/contact)
---
# Data Pipelines & Templates | AI-Powered Solutions | Mage AI
[](./)
Platform
Explore
Pricing
[Community](https://www.mage.ai/chat)
[Get a demo](./getdemo)
[Start now](https://cloud.mage.ai/sign-up)
[](./)
Commons
[Latest](./commons)
[Staff Picks](./commons/topics/staff-picks)
[Batch](./commons/topics/batch)
[dbt](./commons/topics/dbt)
[Dynamic Blocks](./commons/topics/dynamic-blocks)
[Integration](./commons/topics/integration)
[RAG](./commons/topics/rag)
[Streaming](./commons/topics/streaming)
[Staff Picks](./commons/topics/staff-picks)
[Batch](./commons/topics/batch)
[dbt](./commons/topics/dbt)
[Dynamic Blocks](./commons/topics/dynamic-blocks)
[Integration](./commons/topics/integration)
[RAG](./commons/topics/rag)
[Streaming](./commons/topics/streaming)
Commons
Ready-made pipelines and projects crafted to save you time and supercharge your workflows.
Ready-made pipelines and projects crafted to save you time and supercharge your workflows.
[\
\
Titanic Survival Demo](./commons/titanic-survival-demo)
[\
\
Basic DBT pipeline](./commons/demo-dbt-basic)
Your AI data engineer
[Get started](https://mage.ai/sign-up)
[Request a demo](./getdemo)
Product
[AI](./ai)
[Data pipelines](./data)
[Developer experience](./build)
[Deployments](./deploy)
[Scalability](./scale)
[Customizations](./extend)
Resources
[Pricing](./pricing)
[Documentation](https://docs.mage.ai/)
[Blog](./blog)
[Platform features](./specs)
[Unique differentiators](./innovations)
[Mage Pro vs OSS](./oss-vs-pro)
[Changelog](./updates)
Getting started
[Quickstart](https://docs.mage.ai/getting-started/setup)
[Guides](https://docs.mage.ai/guides/overview)
[Templates](./commons)
[Core abstractions](https://docs.mage.ai/design/core-abstractions)
[Dynamic blocks](https://docs.mage.ai/design/blocks/dynamic-blocks)
Tools
[Help & support](https://www.mage.ai/chat)
[Migration support](./support-plan)
[Community](https://www.mage.ai/chat)
[Partner program](./partners)
Company
[Terms](./terms)
[Privacy Policy](./privacy)
Ā© 2025 Mage Technologies, Inc.
Your AI data engineer
[Get started](https://mage.ai/sign-up)
[Request a demo](./getdemo)
Product
[AI](./ai)
[Data pipelines](./data)
[Developer experience](./build)
[Deployments](./deploy)
[Scalability](./scale)
[Customizations](./extend)
Resources
[Pricing](./pricing)
[Documentation](https://docs.mage.ai/)
[Blog](./blog)
[Platform features](./specs)
[Unique differentiators](./innovations)
[Mage Pro vs OSS](./oss-vs-pro)
[Changelog](./updates)
Getting started
[Quickstart](https://docs.mage.ai/getting-started/setup)
[Guides](https://docs.mage.ai/guides/overview)
[Templates](./commons)
[Core abstractions](https://docs.mage.ai/design/core-abstractions)
[Dynamic blocks](https://docs.mage.ai/design/blocks/dynamic-blocks)
Tools
[Help & support](https://www.mage.ai/chat)
[Migration support](./support-plan)
[Community](https://www.mage.ai/chat)
[Partner program](./partners)
Company
[Terms](./terms)
[Privacy Policy](./privacy)
Ā© 2025 Mage Technologies, Inc.
---
# Updates | Changelog and Release Notes | Mage AI
[](./)
Platform
Explore
Pricing
[Community](https://www.mage.ai/chat)
[Get a demo](./getdemo)
[Start now](https://cloud.mage.ai/sign-up)
[](./)
[March 6, 2025](./explore/tags/:m66alHC0y)
1923 Release š¤ š“
=================
1923 Release š¤ š“
=================
Saddle up for 1923 ā our latest Mage Pro release! This update brings the grit and determination of the Dutton Ranch to your data pipelines, forging new paths and expanding your horizons.
**š Enhanced API source for data integrations**
------------------------------------------------
Mage Pro's API Source now delivers unparalleled flexibility and resilience in data integration. Our enhanced connector provides advanced capabilities to handle complex API data retrieval, supporting custom encodings, compressed file formats, dynamic type handling, and seamless ingestion of large-scale data sources. Overcome traditional API integration challenges with a single, powerful solution designed to streamline your data extraction workflows.
Key Enhancements:
* Support custom encoding
* Support syncing zipped csv file
* Support force column types to string type
* Support syncing large files from API
**š Teradata Integration**
---------------------------
š Mage Pro now includes native Teradata connectivity, enabling bidirectional data synchronization for enterprise-scale data warehousing. Leverage our full support for both source and destination configurations with advanced replication capabilities.
**āļø Hybrid Deployment on Azure**
---------------------------------
Experience the seamless integration of Azure cloud scalability and on-premises security with Mage Pro's Hybrid Deployment solution. By combining the robust management capabilities of Microsoft Azure with the control and privacy of your local infrastructure, our deployment offers a flexible and resilient approach to data processing. Unlock the full potential of your data while maintaining stringent security standards and operational efficiency.
**š§ Iceberg Destination**
--------------------------
Mage Pro now supports direct writes to Apache Iceberg tables, empowering data teams to build robust, scalable data lakehouse architectures. Leverage native Iceberg table format capabilities, including schema evolution, time travel, and atomic commits, directly within your data pipelines. Integrate with cloud data platforms and enable advanced data engineering workflows with our Iceberg destination support.
**āØ Airbyte Cloud Integration**
-------------------------------
Expand your data integration capabilities with Mage Pro's native Airbyte Cloud connector. Access an extensive ecosystem of pre-built data sources, enabling rapid connection to hundreds of platforms without complex configuration. Streamline your data ingestion workflow by leveraging Airbyte's robust connector library directly within the Mage Pro platform.
š Check out our documentation for details:Ā [https://docs.mage.ai/integrations/airbyte#airbyte-in-mage](https://docs.mage.ai/integrations/airbyte#airbyte-in-mage)
**š§± Databricks Integration Suite**
-----------------------------------
Mage Pro now offers seamless integration with Databricks SQL, enabling you to run SQL queries directly within your Databricks environment. Weāre taking our integration with Databricks to the next level. Chek out which features we recently added:
* **Databricks SQL Block:**Ā Run SQL queries directly within your Databricks environment using our Databricks SQL block. More info:Ā [https://docs.mage.ai/integrations/databases/Databricks](https://docs.mage.ai/integrations/databases/Databricks)
* **Unity Catalog Integration:**Ā Seamlessly access and manage data in your Unity Catalog.
* **Databricks PySpark Integration:**Ā Harness the power of PySpark for data transformations within Mage. Details:Ā [https://docs.mage.ai/integrations/compute/databricks](https://docs.mage.ai/integrations/compute/databricks)
**ā¾ļø Universal Block Concurrency Limit**
----------------------------------------
Take precise control of your data pipeline's computational resources with Mage Pro's new Universal Block Concurrency Limit. Implement granular execution constraints that prevent system overload, optimize resource allocation, and ensure predictable performance across complex data workflows.
**š¤Ā AI Inline code assistance**
--------------------------------
Revolutionize your coding workflow with Mage Pro's intelligent AI sidekick. Seamlessly generate, debug, and optimize code directly within your development environment. Leverage AI-powered insights to write complex queries, analyze data, and accelerate your development process without ever leaving your coding workspace.
š” Learn more: [https://www.mage.ai/explore/ai-inline-code-assistance](https://www.mage.ai/explore/ai-inline-code-assistance)
š§ AI data-aware autocomplete
-----------------------------
Elevate your coding precision with Mage Pro's intelligent autocomplete that understands your code in all upstream pipeline blocks. Our AI provides context-aware suggestions by analyzing upstream code blocks and anticipating exact input parameters, transforming code completion from a basic hint to a smart, data-intelligent assistant.
š” Learn more: [https://www.mage.ai/explore/ai-data-aware-autocomplete](https://www.mage.ai/explore/ai-data-aware-autocomplete)
This release is all about taming the data frontier and building a robust foundation for your data-driven future. Onward to new horizons!
[Browse Prologs](./updates/all)
Prologs
[Get started](https://mage.ai/sign-up)
[Demo](./getdemo)
[###### 1923 Release š¤ š“\
\
3/6/25](./updates/prolog-1923)
[###### Cobra Kai Release šš„\
\
2/21/25](./updates/prolog-cobra-kai)
[###### Brave New World Release āļø\
\
2/6/25](./updates/prolog-brave-new-world)
[###### Mage Pro Partner Program: Empowering Data Experts and Consultants\
\
2/1/25](./updates/mage-pro-partner-program-empowering-data-experts-and-consultants)
[###### Wolf Man release šŗ\
\
1/17/25](./updates/wolfman-release)
[###### š¤« Psstā¦ No wait, free access to Mage Pro\
\
12/23/24](./updates/psst-no-wait-free-access-to-mage-pro)
[###### Wicked release 0.9.75 (OSS) š«§ š§¹\
\
12/4/24](./updates/oss-wicked-release-0-9-75)
[###### Google cloud storage source\
\
12/3/24](./updates/google-cloud-storage-source)
[Browse all](./updates/all)
Prologs
Your AI data engineer
[Get started](https://mage.ai/sign-up)
[Request a demo](./getdemo)
Product
[AI](./ai)
[Data pipelines](./data)
[Developer experience](./build)
[Deployments](./deploy)
[Scalability](./scale)
[Customizations](./extend)
Resources
[Pricing](./pricing)
[Documentation](https://docs.mage.ai/)
[Blog](./blog)
[Platform features](./specs)
[Unique differentiators](./innovations)
[Mage Pro vs OSS](./oss-vs-pro)
[Changelog](./updates)
Getting started
[Quickstart](https://docs.mage.ai/getting-started/setup)
[Guides](https://docs.mage.ai/guides/overview)
[Templates](./commons)
[Core abstractions](https://docs.mage.ai/design/core-abstractions)
[Dynamic blocks](https://docs.mage.ai/design/blocks/dynamic-blocks)
Tools
[Help & support](https://www.mage.ai/chat)
[Migration support](./support-plan)
[Community](https://www.mage.ai/chat)
[Partner program](./partners)
Company
[Terms](./terms)
[Privacy Policy](./privacy)
Ā© 2025 Mage Technologies, Inc.
Your AI data engineer
[Get started](https://mage.ai/sign-up)
[Request a demo](./getdemo)
Product
[AI](./ai)
[Data pipelines](./data)
[Developer experience](./build)
[Deployments](./deploy)
[Scalability](./scale)
[Customizations](./extend)
Resources
[Pricing](./pricing)
[Documentation](https://docs.mage.ai/)
[Blog](./blog)
[Platform features](./specs)
[Unique differentiators](./innovations)
[Mage Pro vs OSS](./oss-vs-pro)
[Changelog](./updates)
Getting started
[Quickstart](https://docs.mage.ai/getting-started/setup)
[Guides](https://docs.mage.ai/guides/overview)
[Templates](./commons)
[Core abstractions](https://docs.mage.ai/design/core-abstractions)
[Dynamic blocks](https://docs.mage.ai/design/blocks/dynamic-blocks)
Tools
[Help & support](https://www.mage.ai/chat)
[Migration support](./support-plan)
[Community](https://www.mage.ai/chat)
[Partner program](./partners)
Company
[Terms](./terms)
[Privacy Policy](./privacy)
Ā© 2025 Mage Technologies, Inc.
---
# Blog | Data Engineering Hub | Mage AI
From the Grimoire.
From the Grimoire.
[Step-by-step guide to connecting Mage Pro SQL blocks with Databricks\
\
The guide provides a step-by-step tutorial for integrating Mage Pro SQL blocks with Databricks to create automated data pipelines. It walks through configuring Databricks credentials in Mage Pro, creating a pipeline, adding a data loader block to fetch API data (using golf rankings as an example), securely storing API keys, creating SQL blocks to transfer data to Databricks, and finally verifying the data through queries in Databricks. The integration aims to streamline data workflows by eliminating manual processes and creating a seamless connection between data sources and Databricks for analysis.\
\
Read now\
\
\
\
](./blog/step-by-step-guide-to-connecting-mage-pro-sql-blocks-with-databricks)
Stories
Legendary tales and magical insights
Filter
Filter
[Streamline data transfer: The ultimate data integration guide SFTP to BigQuery\
\
\
\
Securely move data from SFTP to Google BigQuery using Mage Pro for automated analytics. This guide walks you through setting up a pipeline, connecting to SFTP, choosing data streams, configuring replication, transforming data, connecting to BigQuery, triggering syncs, and verifying data. Automate your data flow, eliminate manual errors, and get valuable business insights!\
\
Mar 4, 2025](./blog/streamline-data-transfer-the-ultimate-data-integration-guide-sftp-to-bigquery)
[Consolidate your data stack with Mage AI: Cut costs and complexity\
\
\
\
Data teams in companies are drowning in too many different tools. Each tool does one thing well, but using them all together creates a mess that wastes time and money. Mage Pro fixes this by putting everything you need in one place instead of juggling separate programs like Airbyte, dbt Cloud, and others. With one system instead of many, teams spend less money on subscriptions, don't have to connect different tools together, work better as a team, and can adapt more easily when things change.\
\
Feb 28, 2025](./blog/consolidate-your-data-stack-with-mage-ai-cut-costs-and-complexity)
[Guide to accuracy, precision, and recall\
\
\
\
Accuracy tells you how many times the ML model was correct overall. Precision is how good the model is at predicting a specific category. Recall tells you how many times the model was able to detect a specific category.\
\
Jan 29, 2025](./blog/definitive-guide-to-accuracy-precision-recall-for-product-developers)
[Mage AI vs. Airbyte: A comprehensive comparison of modern data integration platforms\
\
\
\
This blog compares Mage AI and Airbyte, two leading data integration platforms. Mage offers a fully managed service with advanced integrations and developer-friendly features, while Airbyte provides an open-source ELT solution with a wide range of connectors. We explore their features, pricing, and key differences to help you choose the best fit for your data integration needs.\
\
Jan 26, 2025](./blog/mage-ai-vs-airbyte-a-comprehensive-comparison-of-modern-data-integration-platforms)
[Exploring the power of Mage: Insights from the data engineering community\
\
\
\
What are people saying about Mage, the modern day open-source data pipeline tool for transforming and integrating your data? Discover the buzz for yourself by checking out these blogs and articles written by those in the data engineering community!\
\
Jan 16, 2025](./blog/exploring-the-power-of-mage-insights-from-the-data-engineering-community)
[Azure Blob Storage file transfer using Mage Proās dynamic blocks\
\
\
\
Companies are always looking for better ways to manage and process their data. In this blog post, weāll explore a proof of concept developed for a leading beauty brand, demonstrating how Mage Pro can turn complicated data tasks into smooth operations. Whether youāre a data engineer, analyst, or simply interested in data management, this article will offer valuable insights into building effective data pipelines using Mage Pro, Azure Blob Storage, and Snowflake.\
\
Oct 31, 2024](./blog/azure-blob-storage-file-transfer-using-mage-pro-s-dynamic-blocks)
[7 essential Mage features for data engineers in 2024\
\
\
\
Explore how Mage can help developers build flexible and scalable data pipelines. This article guides you through features that will help data engineers streamline their workflow design, enhance data processing, and ensure reliable data pipeline execution. By leveraging Mage Proās advanced tools, data engineers can simplify complex processes and achieve greater productivity in their projects.\
\
Oct 10, 2024](./blog/7-essential-mage-features-for-data-engineers-in-2024)
[Boost your data pipeline automation with Mage sensor blocks\
\
\
\
Sensors continuously evaluate specific conditions, ensuring that downstream blocks execute only when prerequisites are met or within a defined timeframe. By integrating sensor blocks, data workflows become more reliable and resource-efficient, minimizing unnecessary computations and maintaining data consistency. The article covers the fundamentals of sensor blocks, their configuration, practical examples across various platforms, and the benefits they bring to data pipeline management.\
\
Sep 26, 2024](./blog/boost-your-data-pipeline-automation-with-mage-sensor-blocks)
Our favorites
Powerful spells for every data pro
[Consolidate your data stack with Mage AI: Cut costs and complexity\
\
](./blog/consolidate-your-data-stack-with-mage-ai-cut-costs-and-complexity)
Wisdom is not a product of schooling but of the lifelong attempt to acquire it.
Wisdom is not a product of schooling but of the lifelong attempt to acquire it.
Albert Einstein
[Guide to accuracy, precision, and recall\
\
](./blog/definitive-guide-to-accuracy-precision-recall-for-product-developers)
[7 essential Mage features for data engineers in 2024\
\
](./blog/7-essential-mage-features-for-data-engineers-in-2024)
[Revolutionizing data pipelines with dynamic blocks in Mage AI\
\
](./blog/revolutionizing-data-pipelines-with-dynamic-blocks-in-mage-ai)
[Make dbt Magic with Mage\
\
](./blog/make-dbt-magic-with-mage)
Explore
Dive into the stories that matter most
[Best Practices](./blog/tags/best-practices)
[Blocks](./blog/tags/blocks)
[Community](./blog/tags/community)
[Concepts](./blog/tags/concepts)
[Data integration](./blog/tags/data-integration)
[dbt](./blog/tags/dbt)
[Guest Post](./blog/tags/guest-post)
[Machine Learning](./blog/tags/machine-learning)
[Solutions](./blog/tags/solutions)
[Spellbook](./blog/tags/spellbook)
[Trends](./blog/tags/trends)
[Tutorials](./blog/tags/tutorial)
Your AI data engineer
[Get started](https://mage.ai/sign-up)
[Request a demo](./getdemo)
Product
[AI](./ai)
[Data pipelines](./data)
[Developer experience](./build)
[Deployments](./deploy)
[Scalability](./scale)
[Customizations](./extend)
Resources
[Pricing](./pricing)
[Documentation](https://docs.mage.ai/)
[Blog](./blog)
[Platform features](./specs)
[Unique differentiators](./innovations)
[Mage Pro vs OSS](./oss-vs-pro)
[Changelog](./updates)
Getting started
[Quickstart](https://docs.mage.ai/getting-started/setup)
[Guides](https://docs.mage.ai/guides/overview)
[Templates](./commons)
[Core abstractions](https://docs.mage.ai/design/core-abstractions)
[Dynamic blocks](https://docs.mage.ai/design/blocks/dynamic-blocks)
Tools
[Help & support](https://www.mage.ai/chat)
[Migration support](./support-plan)
[Community](https://www.mage.ai/chat)
[Partner program](./partners)
Company
[Terms](./terms)
[Privacy Policy](./privacy)
Ā© 2025 Mage Technologies, Inc.
Your AI data engineer
[Get started](https://mage.ai/sign-up)
[Request a demo](./getdemo)
Product
[AI](./ai)
[Data pipelines](./data)
[Developer experience](./build)
[Deployments](./deploy)
[Scalability](./scale)
[Customizations](./extend)
Resources
[Pricing](./pricing)
[Documentation](https://docs.mage.ai/)
[Blog](./blog)
[Platform features](./specs)
[Unique differentiators](./innovations)
[Mage Pro vs OSS](./oss-vs-pro)
[Changelog](./updates)
Getting started
[Quickstart](https://docs.mage.ai/getting-started/setup)
[Guides](https://docs.mage.ai/guides/overview)
[Templates](./commons)
[Core abstractions](https://docs.mage.ai/design/core-abstractions)
[Dynamic blocks](https://docs.mage.ai/design/blocks/dynamic-blocks)
Tools
[Help & support](https://www.mage.ai/chat)
[Migration support](./support-plan)
[Community](https://www.mage.ai/chat)
[Partner program](./partners)
Company
[Terms](./terms)
[Privacy Policy](./privacy)
Ā© 2025 Mage Technologies, Inc.
[](./)
Platform
Explore
Pricing
[Community](https://www.mage.ai/chat)
[Get a demo](./getdemo)
[Start now](https://cloud.mage.ai/sign-up)
[](./)
---
# Partners Program | Collaborate and Earn | Mage AI
Partner with Mage
=================
As a valued partner, youāll earn up to **15% commission** on the annual contract value for every successful customer referral.
[Become a partner](https://airtable.com/appNxTJTLQDiCsk3F/pagWHXOtrxo6T0bDm/form)
[Become a partner](https://airtable.com/appNxTJTLQDiCsk3F/pagWHXOtrxo6T0bDm/form)
[Refer now](https://airtable.com/appNxTJTLQDiCsk3F/pagy3l9uXZqvrXf6q/form)
[Refer now](https://airtable.com/appNxTJTLQDiCsk3F/pagy3l9uXZqvrXf6q/form)
**Generous Rewards**
Earn commissions that grow with your referrals.
**Effortless Onboarding**
Focus on what you do best while we handle the technical work.
**Unmatched Value**
Empower clients with industry-leading tools and support.
Unlock Revenue Potential
* **Earn Big:** Bring new customers to Mage and enjoy substantial commissions on their contracts.
* **Build Recurring Income:** Keep earning year after year with commissions on annual renewals.
Dedicated Partnership Support
* **Weāve Got Your Back:** Our engineering team will handle setup and project migration, ensuring your referrals have a seamless experience.
* **Risk-Free Starter Credits:** Give your prospects $25 in free credits to get started with confidence.
Exclusive Access to Mage Pro
* **Empower Your Clients:** Gain full access to the Mage Pro platform and services to unlock their data potential.
* **See Whatās Possible:** Explore our cutting-edge features: [Mage Product Overview](./product)
.
Grow Together, Succeed Together
* **A Partnership That Works for You:** Our program is designed to align with your business goals, creating opportunities for shared success.
* **Win-Win Philosophy:** Unlock growth for your clients, your business, and Mage - all at the same time.
Who Can Become a Partner?
Our program is perfect for those driving innovation and transformation:
* **System Integrators** - Delivering impactful solutions on a global or regional scale.
* **Consultants & Advisors** ā Helping organizations thrive with expert guidance.
* **Distributors & Resellers** ā Offering cutting-edge tools to meet customer needs.
* **Technology Innovators** ā Adding value with complementary solutions.
Meet our partners
Weāre proud to collaborate with industry leaders and innovators who trust Mage. Here are just a few of the incredible partners in our network:
[](https://www.ada-study.com/)
[](https://www.ada-study.com/)
[](https://www.intodata.nl)
[](https://www.intodata.nl)
[](https://cloudshuttle.com.au)
[](https://cloudshuttle.com.au)
[](https://www.dataplex-consulting.com)
[](https://www.dataplex-consulting.com)
āAn extraordinary team to work alongsideā
āAn extraordinary team to work alongsideā
Take the first step toward unlocking new revenue streams and growth opportunities. Join the Mage Partnership Program
[Become a partner](https://airtable.com/appNxTJTLQDiCsk3F/pagWHXOtrxo6T0bDm/form)
[Become a partner](https://airtable.com/appNxTJTLQDiCsk3F/pagWHXOtrxo6T0bDm/form)
[Refer a team](https://airtable.com/appNxTJTLQDiCsk3F/pagy3l9uXZqvrXf6q/form)
[Refer a team](https://airtable.com/appNxTJTLQDiCsk3F/pagy3l9uXZqvrXf6q/form)
Your AI data engineer
[Get started](https://mage.ai/sign-up)
[Request a demo](./getdemo)
Product
[AI](./ai)
[Data pipelines](./data)
[Developer experience](./build)
[Deployments](./deploy)
[Scalability](./scale)
[Customizations](./extend)
Resources
[Pricing](./pricing)
[Documentation](https://docs.mage.ai/)
[Blog](./blog)
[Platform features](./specs)
[Unique differentiators](./innovations)
[Mage Pro vs OSS](./oss-vs-pro)
[Changelog](./updates)
Getting started
[Quickstart](https://docs.mage.ai/getting-started/setup)
[Guides](https://docs.mage.ai/guides/overview)
[Templates](./commons)
[Core abstractions](https://docs.mage.ai/design/core-abstractions)
[Dynamic blocks](https://docs.mage.ai/design/blocks/dynamic-blocks)
Tools
[Help & support](https://www.mage.ai/chat)
[Migration support](./support-plan)
[Community](https://www.mage.ai/chat)
[Partner program](./partners)
Company
[Terms](./terms)
[Privacy Policy](./privacy)
Ā© 2025 Mage Technologies, Inc.
Your AI data engineer
[Get started](https://mage.ai/sign-up)
[Request a demo](./getdemo)
Product
[AI](./ai)
[Data pipelines](./data)
[Developer experience](./build)
[Deployments](./deploy)
[Scalability](./scale)
[Customizations](./extend)
Resources
[Pricing](./pricing)
[Documentation](https://docs.mage.ai/)
[Blog](./blog)
[Platform features](./specs)
[Unique differentiators](./innovations)
[Mage Pro vs OSS](./oss-vs-pro)
[Changelog](./updates)
Getting started
[Quickstart](https://docs.mage.ai/getting-started/setup)
[Guides](https://docs.mage.ai/guides/overview)
[Templates](./commons)
[Core abstractions](https://docs.mage.ai/design/core-abstractions)
[Dynamic blocks](https://docs.mage.ai/design/blocks/dynamic-blocks)
Tools
[Help & support](https://www.mage.ai/chat)
[Migration support](./support-plan)
[Community](https://www.mage.ai/chat)
[Partner program](./partners)
Company
[Terms](./terms)
[Privacy Policy](./privacy)
Ā© 2025 Mage Technologies, Inc.
[](./)
Platform
Explore
Pricing
[Community](https://www.mage.ai/chat)
[Get a demo](./getdemo)
[Start now](https://cloud.mage.ai/sign-up)
[](./)
---
# Sign in to GitHub Ā· GitHub
You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session. You switched accounts on another tab or window. Reload to refresh your session. Dismiss alert
Sign in to GitHub
=================
Username or email address
Password [Forgot password?](/password_reset)
Password login alternatives
---------------------------
Sign in with a passkey
New to GitHub? [Create an account](/signup?return_to=https%3A%2F%2Fgithub.com%2Fmage-ai%2Fmage-ai&source=login)
You canāt perform that action at this time.
---
# GitHub Ā· Build and ship software on a single, collaborative platform Ā· GitHub
GitHub Copilot is now available for free. [Learn more](https://github.com/features/copilot/?utm_source=github&utm_medium=banner&utm_campaign=copilotfree-bannerheader)
You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session. You switched accounts on another tab or window. Reload to refresh your session. Dismiss alert
Build and ship software on a single, collaborative platform
===========================================================
Join the worldās most widely adopted AI-powered developer platform.
Enter your email
Sign up for GitHub
[Try GitHub Copilot](/features/copilot)
GitHub features
---------------
A demonstration animation of a code editor using GitHub Copilot Chat, where the user requests GitHub Copilot to refactor duplicated logic and extract it into a reusable function for a given code snippet.
CodePlanCollaborateAutomateSecure
Code
Build code quickly and more securely with GitHub Copilot embedded throughout your workflows.
GitHub is used by
-----------------
Pause
A Copilot chat window with extensions enabled. The user inputs the @ symbol to reveal a list of five Copilot Extensions. @Sentry is selected from the list, which shifts the window to a chat directly with that extension. There are three sample prompts at the bottom of the chat window, allowing the user to Get incident information, Edit status on incident, or List the latest issues. The last one is activated to send the prompt: @Sentry List the latest issues. The extension then lists several new issues and their metadata.
### Work 55% faster.[Jump to footnoteĀ 1](#footnote-1)
Increase productivity with AI-powered coding assistance, including code completion, chat, and more.
[Explore GitHub Copilot](/features/copilot)
Duolingo boosts developer speed by 25% with GitHub Copilot
[Read customer story](/customer-stories/duolingo)
2024 GartnerĀ® Magic Quadrantā¢ for AI Code Assistants
[Read report](https://www.gartner.com/doc/reprints?id=1-2IKO4MPE&ct=240819&st=sb)
### Automate any workflow
Optimize your process with simple and secured CI/CD.
A list of workflows displays a heading ā45,167 workflow runsā at the top. Below are five rows of completed workflows accompanied by their completion time and their duration formatted in minutes and seconds.[Discover GitHub Actions](/features/actions)
### Get up and running in seconds
Start building instantly with a comprehensive dev environment in the cloud.
A GitHub Codespaces setup for the landing page of a game called OctoInvaders. On the left is a code editor with some HTML and Javascript files open. On the right is a live render of the page. In front of this split editor window is a screenshot of two active GitHub Codespaces environments with their branch names and a button to āCreate codespace on main.ā[Check out GitHub Codespaces](/features/codespaces)
### Build on the go
Manage projects and chat with GitHub Copilot from anywhere.
Two smartphone screens side by side. The left screen shows a Notification inbox, listing issues and pull requests from different repositories like TensorFlow and GitHubās OctoArcade octoinvaders. The right screen shows a new conversation in GitHub Copilot chat.[Download GitHub Mobile](/mobile)
### Integrate the tools you love
Sync with 17,000+ integrations and a growing library of Copilot Extensions.
A grid of fifty app tiles displays logos for integrations and extensions for companies like Stripe, Slack, and Docker. The tiles extend beyond the bounds of the image to indicate a wide array of apps. [Visit GitHub Marketplace](/marketplace)
### Apply fixes in seconds. Spend less time fixing vulnerabilities and more time building features with Copilot Autofix.
[Explore GitHub Advanced Security](/enterprise/advanced-security)
Solve security debt. Leverage AI-assisted security campaigns to reduce application vulnerabilities and zero-day attacks.
[Discover security campaigns](/enterprise/advanced-security)
Dependencies you can depend on. Update vulnerable dependencies with supported fixes for breaking changes.
[Learn about Dependabot](/features/security/software-supply-chain)
Your secrets, your business: protected. Detect, prevent, and remediate leaked secrets across your organization.
[Read about secret scanning](/features/security/code)
### 7x fastervulnerability fixes with GitHub[Jump to footnoteĀ 2](#footnote-2)
### 90% coverage[of alert types in all supported languages with Copilot Autofix](https://docs.github.com/en/code-security/code-scanning/managing-your-code-scanning-configuration/codeql-query-suites)
### Your workflows, your way. Plan effectively with an adaptable spreadsheet that syncs with your work.
[Jump into GitHub Projects](/features/issues)
ā
> It helps us onboard new software engineers and get them productive right away. We have all our source code, issues, and pull requests in one place... GitHub is a complete platform that frees us from menial tasks and enables us to do our best work.
Fabian FaulhaberApplication manager at Mercedes-Benz
### Keep track of your tasks
Create issues and manage projects with tools that adapt to your code.
Display of task tracking within an issue, showing the status of related sub-issues and their connection to the main issue.[Explore GitHub Issues](/features/issues)
### Share ideas and ask questions
Create space for open-ended conversations alongside your project.
A GitHub Discussions thread where a GitHub user suggests a power-up idea involving Hubot revealing a path and protecting Mona. The post has received 5 upvotes and several reactions. Below, three other users add to the discussion, suggesting Hubot could provide different power-ups depending on levels and appreciating the collaboration idea.[Discover GitHub Discussions](/features/discussions)
### Review code changes together
Create review processes that improve code quality and fit neatly into your workflow.
Two code review approvals by helios-ackmore and amanda-knox, which are followed by three successful checks for āBuild,ā āTest,ā and āPublish.ā[Learn about code review](/features/code-review)
### Fund open source projects
Become an open source partner and support the tools and libraries that power your work.
A GitHub Sponsors popup displays ā$15,000 a monthā with a progress bar showing 87% towards a $15,000 goal.[Dive into GitHub Sponsors](/sponsors)
By industryBy sizeBy use case
By industry
* * *
[\
\
\
\
Technology\
\
Figma streamlines development and strengthens security\
\
Read customer story](https://github.com/customer-stories/figma)
[\
\
\
\
Automotive\
\
Mercedes-Benz standardizes source code and automates onboarding\
\
Read customer story](https://github.com/customer-stories/mercedes-benz)
[\
\
\
\
Financial services\
\
Mercado Libre cuts coding time by 50%\
\
Read customer story](https://github.com/customer-stories/mercado-libre)
[Explore customer stories](/customer-stories)
* * *
[View all solutions](/solutions)
Millions of developers and businesses call GitHub home
------------------------------------------------------
Whether youāre scaling your development process or just learning how to code, GitHub is where you belong. Join the worldās most widely adopted AI-powered developer platform to build the technologies that redefine whatās possible.
Enter your email
Sign up for GitHub
[Try GitHub Copilot](/features/copilot)
Footnotes
---------
1. [Survey: The AI wave continues to grow on software development teams, 2024.](https://github.blog/news-insights/research/survey-ai-wave-grows/)
[](#footnote-ref-1)
2. This 7X times factor is based on data from the industryās longest running analysis of fix rates Veracode State of Software Security 2023, which cites the average time to fix 50% of flaws as 198 days vs. GitHubās fix rates of 72% of flaws with in 28 days which is at a minimum of 7X faster when compared.[](#footnote-ref-2)
[](#hero)
You canāt perform that action at this time.
---
# Platform Specs | AI Data Engineering Features | Mage AI
Specifications
Specifications
From core capabilities to advanced features
From core capabilities to advanced features
Developer experience
Code Editor
Code Editor
Interactive code editor with visual feedback for immediately previewing execution results
=========================================================================================
Execute Python, SQL, and R code from a block within a pipeline
==============================================================
Mix and match dbt models and custom Python, SQL, or R code blocks within the same pipeline
==========================================================================================
Drag-and-drop code blocks in a visual dependency tree graph to customize the order of execution and flow of data between code blocks
====================================================================================================================================
View code side-by-side with the block's execution output while developing and building pipelines
================================================================================================
Configure project, pipeline, or block level settings for limiting the data volume when running code blocks in development
=========================================================================================================================
New code editor with enhanced file browser, code management, and multi-row and multi-column layout
==================================================================================================
Autoscaling code execution framework for running blocks during pipeline development
===================================================================================
Install and run VS Code extensions from the new code editor
===========================================================
New pipeline canvas editor for building complex graphs
======================================================
Retrieval Augmented Generation (RAG) pipeline builder
=====================================================
Al Sidekick for creating pipelines, generating code blocks, and troubleshooting execution errors
================================================================================================
Code Blocks
Code Blocks
Custom
======
Source
======
Dest.
=====
Transform
=========
Sensor
======
Scratchpad
==========
dbt
===
Markdown
========
Automatically retry block runs with customizable number of retries, delay, maximum delay, and exponential backoff
=================================================================================================================
Control the flow of code executions using sensor blocks that pause a branch of code from running until a condition is met
=========================================================================================================================
Search files, using full-text and natural language, across multiple projects and add code blocks to pipelines without duplicating code
======================================================================================================================================
Data Integrations
Data Integrations
Data integrations with sources and destinations from 100+ third-party services
==============================================================================
Data connectors and integrations with data lakes using Apache Iceberg table format
==================================================================================
Data syncs are 12-18x faster using optimized concurrent read/writes with high throughput and capacity
=====================================================================================================
Run no-code data integrations alongside custom code blocks together within a single pipeline
============================================================================================
Change data capture (CDC) with select databases
===============================================
Create custom data integration sources and destinations without changing source code
====================================================================================
Real-time Data Streaming
Real-time Data Streaming
Build streaming pipelines that process real-time data as it arrives using no-code configurations
================================================================================================
Execute custom Python code blocks on incoming real-time data from streaming sources
===================================================================================
SQL Data Models
SQL Data Models
Interpolate runtime variables, environment variables, secrets, and function macros within the SQL command of a SQL code block
=============================================================================================================================
Special SQL block connectors for DuckDB, OracleDB, Teradata, StarRocks, Couchbase, etc.
=======================================================================================
Execute SQL commands on data output from other Python, SQL, or R blocks
=======================================================================
Upgraded developer experience for building, managing, and monitoring thousands of SQL models
============================================================================================
dbt Integrations
dbt Integrations
Develop, build, test, run, document, manage, and monitor dbt models
===================================================================
Upgraded abt developer experience with new and more flexible user interface
===========================================================================
Manage multiple dbt projects, with different remote repositories, across multiple Mage projects from a single application
=========================================================================================================================
Development workflow
Code Templates
Code Templates
Customizable pipeline and block templates on a team-by-team basis, reducing development time and production errors
==================================================================================================================
Use 100+ pre-defined boilerplate templates for loading data, transforming data, and exporting data or re-use existing code blocks
=================================================================================================================================
Data Validation, Quality, and Unit Testing
Data Validation, Quality, and Unit Testing
Create and run unit tests in Python
===================================
Built-in data testing framework for conveniently writing tests within code blocks
=================================================================================
Framework for validating acceptable state of data using data quality test suites
================================================================================
Development Environment
Development Environment
Documentation built-in using native tagging system and markdown blocks to document code blocks within a pipeline
================================================================================================================
Multiple Python virtual environments for development and code block execution
=============================================================================
Automatically format code styling and run linters to fix syntax
===============================================================
Real-time collaboration with commenting, assignable action items, and reminder notes
====================================================================================
Customize workspace themes, personalize accounts, and re-configure Ul component layouts
=======================================================================================
Manage multiple Mage projects and any other files from a single application
===========================================================================
Global search bar, page launcher, browser history navigation, and command shortcuts application
===============================================================================================
Production Environment
Production Environment
Multi-tenant support with granular access controls and isolated workspaces
==========================================================================
Customizable project and pipeline settings per environment
==========================================================
Launch multiple environments and manage shared team workspaces
==============================================================
Integrate your existing CI/CD custom build jobs and deployment steps into Mage Pro
==================================================================================
Automatically deploy new code changes and data pipelines to different environments
==================================================================================
Versioning
Versioning
Terminal for running console commands
=====================================
Built-in version control application
====================================
Git terminal interface with built-in authentication and shortcuts
=================================================================
Remote repository import tool for easily migrating or syncing existing projects
===============================================================================
Local file edit tracking and version history for restoring changes made in the past
===================================================================================
Data version control for data pipeline versioning
=================================================
Data Orchestration
Scheduling
Scheduling
Schedule pipelines to trigger on a regular or custom interval; e.g. hourly, daily, weekly, monthly, CRON expression
===================================================================================================================
Schedule pipelines to start running and complete its execution by a specific date or time of day
================================================================================================
Confgure multiple triggers for a single pipeline with different schedules, frequency, conditions, SLAs, retries, and runtime variables
======================================================================================================================================
Triggering
Triggering
Trigger pipelines from code and optionally wait until completion before proceeding
==================================================================================
Trigger pipelines based on events from external 3rd party services; e.g. object deleted from Azure Blob Storage
===============================================================================================================
Trigger pipelines across different projects and workspaces
==========================================================
Share and reuse a single trigger across different pipelines
===========================================================
Backfills
Backfills
Schedule, run, or re-run backfills multiple times across different windows of time
==================================================================================
Backfill data with dynamically generated configurations and variables at runtime using custom code blocks
=========================================================================================================
Observability & monitoring
Alerting
Alerting
Receive alerts through email, OpsGenie, Slack, Teams, Discord, Telegram, Google Chat, PagerDuty, etc.
=====================================================================================================
Data pipeline execution runtime SLA and alerts
==============================================
Customizable alert notifications and templates, per channel, when pipeline or block run succeeds, is cancelled, or fails
========================================================================================================================
Alerts via email notification fully integrated without the need for external email service providers
====================================================================================================
Custom events, metrics, and alert notification rules
====================================================
Monitoring
Monitoring
Monitoring dashboards for every pipeline and across the entire project
======================================================================
View current and past pipelines runs
====================================
Integrations with major 3rd party and open source tools, including DataDog, Metaplane, New Relic, Sentry, Prometheus, OpenTelemetry
===================================================================================================================================
Manage cross-pipeline dependencies and execution flow across every pipeline within a project
============================================================================================
Customizable monitoring dashboard
=================================
System level metrics and logs with dashboard charts
===================================================
Monitor upcoming pipeline execution runs from across all schedules, workspaces, and projects
============================================================================================
Data catalog, metadata management, and data lineage
===================================================
Logging
Logging
Detailed structured logging for pipeline runs, pipeline triggers, and individual block runs
===========================================================================================
Export events and logs to remote object storage locations and 3rd party services
================================================================================
Find and browse complete log history using full text search
===========================================================
Privacy & security
Data Governance
Data Governance
Securely store and access sensitive data or credentials using 3rd party or Mage's built-in secret manager
=========================================================================================================
Granular pipeline and block run data output retention policies
==============================================================
User audit trail and logging
============================
Single sign on (SSO)
====================
User management and RBAC permission system with fine-grained access controls to create customizable roles and permissions
=========================================================================================================================
Network & Security
Network & Security
Custom security network access rules to allow or deny inbound/outbound traffic from/to a user-defined set of IP addresses
=========================================================================================================================
Virtual Private Network (VPN) for application account sign-in
=============================================================
VPN connection and SSL certificate authentication for select databases
======================================================================
Infrastructure Development
Infrastructure Development
Dedicated static IP address for development and running pipelines
=================================================================
Regional deployment for data processing operations
==================================================
Performance & scaling
Compute Management
Compute Management
Architecture
Build and run data pipelines using Spark, Snowpark, and Databricks
==================================================================
Spark compute management and resource monitoring
================================================
Customized GPU accelerated resources for running AI/ML/LLM pipelines
====================================================================
Dynamic Blocks
Dynamic Blocks
Dynamically create unique instances of a block at runtime and execute its code logic on distinct sets of input data from upstream sources
=========================================================================================================================================
Reduce the output across all dynamically created blocks and produce a single data output for downstream consumers
=================================================================================================================
Combine the data output from multiple dynamically created blocks and standard block types to create unique combinations of data operations
==========================================================================================================================================
Execute 100,000+ dynamically created block runs concurrently
============================================================
Stream block output data to dynamically generated blocks without waiting for the upstream parent block to finish executing
==========================================================================================================================
Scalability & Automation
Scalability & Automation
Application upgrades and new product features are instantly installed in your environment
=========================================================================================
Reusable pipeline data output, accessible across multiple pipelines to reduce duplication, optimize compute, and ensure data consistency
========================================================================================================================================
Granular block settings for controlling read/write data partitions using output size, number of chunks, and item count
======================================================================================================================
Batch generator framework to operate and process 1,000+ gigabytes (GB) of data without running out of memory
============================================================================================================
Autoscaling orchestration scheduler for maximum pipeline trigger frequency
==========================================================================
Automatically and intelligently scale data pipelines, both vertically and horizontally, using predictive analytics and machine learning
=======================================================================================================================================
Customization & extensibility
Flexible Execution
Flexible Execution
Execute data pipelines using fully customized Docker images
===========================================================
Control and execute pipelines from within other pipelines, using webhooks, or via API requests
==============================================================================================
Run pipelines and configure runtime variables using no-code user interface elements, such as dropdown menus and autocomplete inputs
===================================================================================================================================
Deploy high-performance, low-latency API endpoints for executing blocks and returning output data, such as inference endpoints
==============================================================================================================================
Personal Setup
Personal Setup
Install custom Python modules and system libraries on a per-project basis
=========================================================================
Deploy and run third-party or custom services integrated with your application environment's cluster
====================================================================================================
Set up custom domains for different environments and workspaces
===============================================================
Manage, orchestrate, and configure infrastructure settings and system resources via API endpoints
=================================================================================================
Customizable Experience
Customizable Experience
High throughput API endpoints for integrating Mage Pro with any 3rd party or in-house services
==============================================================================================
Fully customize platform application behavior by executing code that transforms any API request payload or response
===================================================================================================================
Recover your precious developer time
Now you can focus on the fun, creative, and high-impact data engineering projects and let Mage AI handle the rest.
For engineers
Experience how Mage AI ships data pipelines faster, giving you a better work-life-balance.
[Get started](https://mage.ai/sign-up)
For data teams
See how Mage AI accelerates your team velocity while reducing data and infrastructure costs.
[Enterprise demo](./getdemo)
[Explore the platform](./build)
[\
\
AI\
\
Code faster, smarter](./ai)
[\
\
Data pipelines\
\
Build, run, and monitor](./data)
[\
\
Dev Ex\
\
Made for speed and ease](./build)
[\
\
Deployments\
\
Ship workflows effortlessly](./deploy)
[\
\
Scalabiity\
\
Grow without limits](./scale)
[\
\
Customizations\
\
Total extensibility](./extend)
Recover your precious developer time
Now you can focus on the fun, creative, and high-impact data engineering projects and let Mage AI handle the rest.
For engineers
Experience how Mage AI ships data pipelines faster, giving you a better work-life-balance.
[Get started](https://mage.ai/sign-up)
For data teams
See how Mage AI accelerates your team velocity while reducing data and infrastructure costs.
[Enterprise demo](./getdemo)
[Explore the platform](./build)
[\
\
AI\
\
Code faster, smarter](./ai)
[\
\
Data pipelines\
\
Build, run, and monitor](./data)
[\
\
Dev Ex\
\
Made for speed and ease](./build)
[\
\
Deployments\
\
Ship workflows effortlessly](./deploy)
[\
\
Scalabiity\
\
Grow without limits](./scale)
[\
\
Customizations\
\
Total extensibility](./extend)
[](./)
Platform
Explore
Pricing
[Community](https://www.mage.ai/chat)
[Get a demo](./getdemo)
[Start now](https://cloud.mage.ai/sign-up)
[](./)
Product
[AI](./ai)
[Data pipelines](./data)
[Developer experience](./build)
[Deployments](./deploy)
[Scalability](./scale)
[Customizations](./extend)
Resources
[Pricing](./pricing)
[Documentation](https://docs.mage.ai/)
[Blog](./blog)
[Platform features](./specs)
[Unique differentiators](./innovations)
[Mage Pro vs OSS](./oss-vs-pro)
[Changelog](./updates)
Getting started
[Quickstart](https://docs.mage.ai/getting-started/setup)
[Guides](https://docs.mage.ai/guides/overview)
[Templates](./commons)
[Core abstractions](https://docs.mage.ai/design/core-abstractions)
[Dynamic blocks](https://docs.mage.ai/design/blocks/dynamic-blocks)
Tools
[Help & support](https://www.mage.ai/chat)
[Migration support](./support-plan)
[Community](https://www.mage.ai/chat)
[Partner program](./partners)
Company
[Terms](./terms)
[Privacy Policy](./privacy)
Ā© 2025 Mage Technologies, Inc.
Product
[AI](./ai)
[Data pipelines](./data)
[Developer experience](./build)
[Deployments](./deploy)
[Scalability](./scale)
[Customizations](./extend)
Resources
[Pricing](./pricing)
[Documentation](https://docs.mage.ai/)
[Blog](./blog)
[Platform features](./specs)
[Unique differentiators](./innovations)
[Mage Pro vs OSS](./oss-vs-pro)
[Changelog](./updates)
Getting started
[Quickstart](https://docs.mage.ai/getting-started/setup)
[Guides](https://docs.mage.ai/guides/overview)
[Templates](./commons)
[Core abstractions](https://docs.mage.ai/design/core-abstractions)
[Dynamic blocks](https://docs.mage.ai/design/blocks/dynamic-blocks)
Tools
[Help & support](https://www.mage.ai/chat)
[Migration support](./support-plan)
[Community](https://www.mage.ai/chat)
[Partner program](./partners)
Company
[Terms](./terms)
[Privacy Policy](./privacy)
Ā© 2025 Mage Technologies, Inc.
---