# Table of Contents

- [Introduction | SlingData.IO](#introduction-slingdata-io)
- [Installation | SlingData.IO](#installation-slingdata-io)
- [Global Variables | SlingData.IO](#global-variables-slingdata-io)
- [VS Code Extension 🔌 | SlingData.IO](#vs-code-extension-slingdata-io)
- [Running Sling | SlingData.IO](#running-sling-slingdata-io)
- [Environment | SlingData.IO](#environment-slingdata-io)
- [Architecture | SlingData.IO](#architecture-slingdata-io)
- [Sling Platform | SlingData.IO](#sling-platform-slingdata-io)
- [Connections | SlingData.IO](#connections-slingdata-io)
- [Editor | SlingData.IO](#editor-slingdata-io)

---

# Introduction | SlingData.IO

Moving data from one platform to another has long been a critical process. Sling is a modern data movement and transformation platform designed to simplify and streamline data operations. It provides both a powerful CLI tool and a comprehensive platform for managing data workflows between various sources and destinations.

[hashtag](https://docs.slingdata.io/#core-features)

Core Features


----------------------------------------------------------------------

*   **Data Movement**: Transfer data between different storage systems and databases efficiently
    
*   **Flexible Connectivity**: Support for numerous databases, data warehouses, and file storage systems
    
*   **Transformation Capabilities**: Built-in data transformation features during transfer
    
*   **Multiple Operation Modes**: Support for various replication modes including full-refresh, incremental, and snapshot
    
*   **Production-Ready**: Deployable with monitoring, scheduling, and error handling
    

[hashtag](https://docs.slingdata.io/#key-components)

Key Components


------------------------------------------------------------------------

### 

[hashtag](https://docs.slingdata.io/#sling-cli)

Sling CLI

Sling CLI Demo

The command-line interface provides direct access to Sling's capabilities, perfect for:

*   Local development and testing
    
*   CI/CD pipeline integration
    
*   Automated data operations
    
*   Quick data transfers and transformations
    

You can by running a command like this:

`cat my_file.csv | sling run --tgt-conn MYDB --tgt-object my_schema.my_table`

### 

[hashtag](https://docs.slingdata.io/#sling-platform)

Sling Platform

Sling Platform UI

The web-based platform offers:

*   Visual interface for creating and managing data workflows
    
*   Agent based architecture for scalable execution
    
*   Team collaboration features
    
*   Monitoring and alerting
    
*   Centralized connection management
    
*   Job scheduling and orchestration
    

#### 

[hashtag](https://docs.slingdata.io/#sling-agents)

Sling Agents

Agents are the workers that execute your data operations:

*   Run in your own infrastructure
    
*   Secure access to your data sources
    
*   Support for both development and production environments
    

[hashtag](https://docs.slingdata.io/#common-use-cases)

Common Use Cases


----------------------------------------------------------------------------

*   Database replication and synchronization
    
*   Data warehouse loading and ETL operations
    
*   File system to database ingestion
    
*   Cross-platform data migration
    
*   Backup and archival operations
    
*   Real-time data copying and transformation
    

[hashtag](https://docs.slingdata.io/#getting-started)

Getting Started


--------------------------------------------------------------------------

To begin using Sling, you can either:

1.  [Install the CLI tool](https://docs.slingdata.io/sling-cli/getting-started)
     for local development and testing
    
2.  [Sign up for the Sling Platformarrow-up-right](https://github.com/slingdata-io/sling-docs/blob/master/sling-platform/getting-started.md)
     for a managed experience
    
3.  Use both in combination for a complete data operations solution
    

Choose the approach that best fits your needs and scale up as your requirements grow.

![](https://docs.slingdata.io/~gitbook/image?url=https%3A%2F%2Fstatic.scarf.sh%2Fa.png%3Fx-pxid%3D29813085-4cb1-4636-ab5d-cce5dbafc8aa&width=768&dpr=3&quality=100&sign=24fc176e&sv=2)

[NextInstallationchevron-right](https://docs.slingdata.io/sling-cli/getting-started)

Last updated 1 year ago

Was this helpful?

---

# Installation | SlingData.IO

Sling CLI Demo

[hashtag](https://docs.slingdata.io/sling-cli/getting-started#getting-started)

Getting Started


---------------------------------------------------------------------------------------------------

`sling` CLI is a free tool that allows data extraction and loading from / into many popular databases / storage platforms. Follow the instructions below to install it on your respective operating system.

### 

[hashtag](https://docs.slingdata.io/sling-cli/getting-started#installation)

Installation

All commands below will install the latest version of sling. If you'd like to determine which version is latest/current, you can check out the [releases pagearrow-up-right](https://github.com/slingdata-io/sling-cli/releases)
 on github.

#### 

[hashtag](https://docs.slingdata.io/sling-cli/getting-started#brew-on-mac)

Brew on Mac

Follow these [directionsarrow-up-right](https://brew.sh/)
 to install HomeBrew for mac if not already installed.

#### 

[hashtag](https://docs.slingdata.io/sling-cli/getting-started#scoop-on-windows)

Scoop on Windows

#### 

[hashtag](https://docs.slingdata.io/sling-cli/getting-started#linux)

Linux

#### 

[hashtag](https://docs.slingdata.io/sling-cli/getting-started#docker)

Docker

#### 

[hashtag](https://docs.slingdata.io/sling-cli/getting-started#other-binary-downloads)

Other Binary Downloads

See [Releasesarrow-up-right](https://github.com/slingdata-io/sling-cli/releases)
 on Github.

### 

[hashtag](https://docs.slingdata.io/sling-cli/getting-started#setting-up-your-connections)

Setting up your Connections

Sling looks for credentials in several places:

*   Environment Variables
    
*   Sling Env File (located at `~/.sling/env.yaml`)
    
*   DBT Profiles Files (located at `~/.dbt/profiles.yml`)
    

Please see [environment](https://docs.slingdata.io/sling-cli/environment)
 for more details.

[hashtag](https://docs.slingdata.io/sling-cli/getting-started#using-sling)

Using Sling


-------------------------------------------------------------------------------------------

`sling` CLI is designed to be easy to use. Say we want to load a CSV file into a PostgreSQL database. We could run the following command:

Linux

Mac

Windows

Docker

### 

[hashtag](https://docs.slingdata.io/sling-cli/getting-started#with-python)

With Python

If you have Python `pip` installed, you can simply run:

You call also check out the [Python wrapperarrow-up-right](https://github.com/slingdata-io/sling-python)
 library on github.

[PreviousIntroductionchevron-left](https://docs.slingdata.io/)
[NextEnvironmentchevron-right](https://docs.slingdata.io/sling-cli/environment)

Last updated 5 months ago

Was this helpful?

*   [Getting Started](https://docs.slingdata.io/sling-cli/getting-started#getting-started)
    
*   [Installation](https://docs.slingdata.io/sling-cli/getting-started#installation)
    
*   [Setting up your Connections](https://docs.slingdata.io/sling-cli/getting-started#setting-up-your-connections)
    
*   [Using Sling](https://docs.slingdata.io/sling-cli/getting-started#using-sling)
    
*   [With Python](https://docs.slingdata.io/sling-cli/getting-started#with-python)
    

Was this helpful?

sun-brightdesktopmoon

Copy

    # Install with brew first
    brew install slingdata-io/sling/sling
    
    # Once, installed, `sling` should be available
    sling -h

Copy

    scoop bucket add sling https://github.com/slingdata-io/scoop-sling.git
    scoop install sling
    
    # You're good to go!
    sling -h

Copy

    # download latest binary
    curl -LO 'https://github.com/slingdata-io/sling-cli/releases/latest/download/sling_linux_amd64.tar.gz' \
      && tar xf sling_linux_amd64.tar.gz \
      && rm -f sling_linux_amd64.tar.gz \
      && chmod +x sling
    
    # You're good to go!
    ./sling -h

Copy

    docker pull slingdata/sling
    
    docker run --rm -i slingdata/sling --help

Copy

    export MY_PG='postgresql://user:[email protected]:5432/db1'
    
    sling run --src-stream file:///path/to/myfile.csv --tgt-conn MY_PG --tgt-object public.my_new_data
    
    # OR pipe it in
    cat /path/to/myfile.csv | sling run --tgt-conn MY_PG --tgt-object public.my_new_data

Copy

    export MY_PG='postgresql://user:[email protected]:5432/db1'
    
    sling run --src-stream file:///path/to/myfile.csv --tgt-conn MY_PG --tgt-object public.my_new_data
    
    # OR pipe it in
    cat /path/to/myfile.csv | sling run --tgt-conn MY_PG --tgt-object public.my_new_data

Copy

    # using windows Powershell
    $env:MY_PG = 'postgresql://user:[email protected]:5432/db1'
    sling run --src-stream file://C:/path/to/myfile.csv --tgt-conn MY_PG --tgt-object public.my_new_data
    
    # OR pipe it in
    cat C:\path\to\myfile.csv | sling run --tgt-conn MY_PG --tgt-object public.my_new_data

Copy

    export MY_PG='postgresql://user:[email protected]:5432/db1'
    
    docker run --rm -i -e MY_PG -v /path/to/myfile.csv slingdata/sling run --src-stream file:///path/to/myfile.csv --tgt-conn MY_PG --tgt-object public.my_new_data

Copy

    pip install sling

Copy

    from sling import Replication, ReplicationStream
    
    replication = Replication(
      source='MY_PG',
      target='MY_AWS_S3',
      steams={
        "my_table": ReplicationStream(
          sql="select * from my_table",
          object='my_folder/new_file.csv',
        ),
      }
    )
    
    replication.run()

sun-brightdesktopmoon

---

# Global Variables | SlingData.IO

Sling utilizes the following global environment variables to further configure the load behavior. You can simply define them in your environment, the `env.yaml` file or the `env` section in a task or replication.

Variable Name

Description

`SLING_HOME_DIR`

The sling home directory, which contains `env.yaml`. Will use [defaultarrow-up-right](https://github.com/slingdata-io/sling-docs/blob/master/environment.md#sling-env-file-env.yaml)
 if not provided.

`SLING_LOADED_AT_COLUMN`

Whether to add an audit timestamp column named `_sling_loaded_at` in target object. Accepts values `true`, `false`, `unix` (for [epocharrow-up-right](https://www.epochconverter.com/)
 integer values) or `timestamp`. `true` defaults to `unix`.

`SLING_SYNCED_AT_COLUMN`

Whether to add sync tracking columns `_sling_synced_at` (timestamp) and `_sling_synced_op` (operation type) in target object. The `_sling_synced_op` column tracks the last operation: `I` = Insert, `U` = Update, `D` = Delete (soft delete). This is useful when streaming from staging to multiple destinations to track when records were last touched. To enable, set to `true`. When enabled, `_sling_synced_at` replaces `_sling_deleted_at` for soft deletes.

`SLING_STREAM_URL_COLUMN`

If source is file, whether to add a column `_sling_stream_url` with the source file path / url in target object. To enable, set to `true`.

`SLING_TIMEOUT`

The maximum number of minutes the sling replication should run. Once reached, it will kill the process. To enable, set a number (`SLING_TIMEOUT=10.5`)

`SLING_RECURSIVE_LIMIT`

The number limit of file names to pull, when listing from cloud file systems such as S3, GCP and Azure Storage.

`SLING_ROW_ID_COLUMN`

Whether to add a column named `_sling_row_id` in the target object, which will have a random UUIDv7 value. This will be unique. To enable, set to `true`.

`SLING_ROW_NUM_COLUMN`

If source is file, whether to add a column named `_sling_row_num` in the target object, which will be the row number of the stream (incremented by record processed). To enable, set to `true`.

`SLING_EXEC_ID_COLUMN`

Whether to add a column named `_sling_exec_id` in the target object, which will have the run / execution string (a random UUIDv7 value). This will be unique per run. To enable, set to `true`.

`SLING_STATE`

The [location](https://docs.slingdata.io/sling-cli/environment#location-string)
 to read/write information such as incremental values. Proper input format is `CONN_NAME/key`. For example: `POSTGRES/sling_state.state_table` , `AWS_S3/my/folder` or `MY_SFTP/my/folder`.

`SLING_ALLOW_EMPTY`

This is useful to create tables / files using the stream columns structure, even if there is no data. To enable, set to `true`.

`SLING_DIRECT_INSERT`

Tells sling to insert directly into the final table (not create a temp table before). To enable, set to `true`.

`SLING_THREADS`

sets the maximum number of concurrent stream runs. Accepts an integer value, default is `1`.

`SLING_RETRIES`

sets the maximum number of retries for a failed stream run. Accepts an integer value, default is `0`.

`SLING_KEEP_TEMP`

Tells sling to keep any temporary files or tables created in the load process. To enable, set to `true`

`SLING_ENV_YAML`

Provide the body of the `env.yaml` file as an environment variable.

`SLING_DISABLE_TELEMETRY`

this disables any anonymous usage reporting. These are used to improve sling. To disable, set this to `true`.

`SLING_SHOW_PROGRESS`

Whether the progress of the stream should be displayed (`true` or `false`).

`SLING_LOGGING`

How sling formats the log lines. Accepts values `JSON`, `NO_COLOR` or `CONSOLE` (default).

`SLING_LOG_DIR`

Directory for automatic date-based debug log files. When set, Sling creates a `sling_debug_YYYY_MM_DD.log` file in the specified directory and automatically cleans up old log files, keeping the latest 15. Supports `~` for home directory (e.g., `~/.sling/logs`).

`SLING_SAMPLE_SIZE`

The number of records to process in order to infer column types (especially for file sources). Default is `900`.

`SLING_DUCKDB_COMPUTE`

Whether to use DuckDB for writing to parquet files and partitioned parquet/CSV files. DuckDB provides optimized performance for these formats. To disable DuckDB compute, set to `false`. Default is `true`.

`SLING_OTEL_ENDPOINT`

The OpenTelemetry HTTP endpoint URL to export logs to (e.g., `http://otel.host.ip:4318/v1/logs`). When set, Sling will send structured logs to the specified OTLP endpoint with execution attributes including `project_id`, `job_id`, and `exec_id` if configured. Requires [CLI Pro](https://docs.slingdata.io/sling-cli/cli-pro)
.

`SLING_PROXY`

Route database connections through a SOCKS5 proxy. Accepts a SOCKS5 URL (e.g., `socks5://user:pass@host:1080`). To proxy a specific connection instead, set `proxy_url` in its `env.yaml` connection entry. Useful for reaching databases behind VPNs or private networks (e.g., Tailscale). Available in v1.5.9+

[PreviousRunning Slingchevron-left](https://docs.slingdata.io/sling-cli/run)
[NextCLI Prochevron-right](https://docs.slingdata.io/sling-cli/cli-pro)

Last updated 1 month ago

Was this helpful?

Was this helpful?

sun-brightdesktopmoon

sun-brightdesktopmoon

---

# VS Code Extension 🔌 | SlingData.IO

The Sling VS Code extension provides schema validation and language server features for SlingData.io configuration files.

[hashtag](https://docs.slingdata.io/sling-cli/vscode#why-use-it)

Why Use It


--------------------------------------------------------------------------------

This extension enhances your productivity when working with Sling configuration files by offering:

*   **Auto-detection** of Sling configuration files (e.g., pipeline.yaml, replication.yaml, spec.yaml).
    
*   **Schema validation** to ensure your configurations adhere to the correct structure.
    
*   **Auto-completion** for properties, values, and expressions.
    
*   **Hover information** providing inline documentation.
    
*   **Diagnostics** for identifying errors, warnings, and suggestions in your configs.
    

These features help you write valid and efficient Sling configurations, reducing errors and speeding up development!

[hashtag](https://docs.slingdata.io/sling-cli/vscode#how-to-get-it)

How to Get It


--------------------------------------------------------------------------------------

You can get the extension here: [https://marketplace.visualstudio.com/items?itemName=sling.sling-vscodearrow-up-right](https://marketplace.visualstudio.com/items?itemName=sling.sling-vscode)

Or simply search `Sling` in the Extensions Market place panel in VS Code.

![](https://docs.slingdata.io/~gitbook/image?url=https%3A%2F%2F3453272330-files.gitbook.io%2F%7E%2Ffiles%2Fv0%2Fb%2Fgitbook-x-prod.appspot.com%2Fo%2Fspaces%252F-M93cpHl7B7NPZlDrubS%252Fuploads%252Fgit-blob-09871c7e220309dd1ae9a8c12da5464734b1ce4b%252Fimage.png%3Falt%3Dmedia&width=768&dpr=3&quality=100&sign=5fe5b8a6&sv=2)

Install via Marketplace

If you're using a forked VS Code editor (such as Cursor or VSCodium), you can install directly from the VSIX package ([download it herearrow-up-right](https://marketplace.visualstudio.com/_apis/public/gallery/publishers/sling/vsextensions/sling-vscode/latest/vspackage)
).

![](https://docs.slingdata.io/~gitbook/image?url=https%3A%2F%2F3453272330-files.gitbook.io%2F%7E%2Ffiles%2Fv0%2Fb%2Fgitbook-x-prod.appspot.com%2Fo%2Fspaces%252F-M93cpHl7B7NPZlDrubS%252Fuploads%252Fgit-blob-3ab86bfba38844ee3bbbace21f5c55577bfbd374%252Fimage-3.png%3Falt%3Dmedia&width=768&dpr=3&quality=100&sign=bb5045f8&sv=2)

Install via VSIX

[hashtag](https://docs.slingdata.io/sling-cli/vscode#working-with-pipelines-and-hooks)

Working with Pipelines & Hooks


--------------------------------------------------------------------------------------------------------------------------

This VSCode extension helps tremendously with developing Sling Hooks & Pipelines. The extension provides:

*   Schema validation for pipeline YAML files
    
*   Auto-completion for step types, properties, and expression (such as functions)
    
*   Hover documentation for pipeline steps and variables
    

This makes authoring and maintaining complex pipelines much easier and error-free. For more on pipelines, see the [Pipelines](https://docs.slingdata.io/concepts/pipeline)
, [Hooks](https://docs.slingdata.io/concepts/hooks)
 and [Functions](https://docs.slingdata.io/concepts/functions)
 documentation documentation.

[hashtag](https://docs.slingdata.io/sling-cli/vscode#screenshots)

Screenshots


----------------------------------------------------------------------------------

![](https://docs.slingdata.io/~gitbook/image?url=https%3A%2F%2F3453272330-files.gitbook.io%2F%7E%2Ffiles%2Fv0%2Fb%2Fgitbook-x-prod.appspot.com%2Fo%2Fspaces%252F-M93cpHl7B7NPZlDrubS%252Fuploads%252Fgit-blob-8301b2d6799c7eb854c1d81ba9a15e31d97b42e7%252Fimage-2.png%3Falt%3Dmedia&width=768&dpr=3&quality=100&sign=2715005b&sv=2)

Input Suggestions

![](https://docs.slingdata.io/~gitbook/image?url=https%3A%2F%2F3453272330-files.gitbook.io%2F%7E%2Ffiles%2Fv0%2Fb%2Fgitbook-x-prod.appspot.com%2Fo%2Fspaces%252F-M93cpHl7B7NPZlDrubS%252Fuploads%252Fgit-blob-1ddeeb85a87822fadcd93837fdb5cdeb25a55b7d%252Fimage-1.png%3Falt%3Dmedia&width=768&dpr=3&quality=100&sign=6b7114c4&sv=2)

Function Suggestions

[PreviousCLI Prochevron-left](https://docs.slingdata.io/sling-cli/cli-pro)
[NextWorking with AI 🤖chevron-right](https://docs.slingdata.io/sling-cli/ai)

Last updated 8 months ago

Was this helpful?

*   [Why Use It](https://docs.slingdata.io/sling-cli/vscode#why-use-it)
    
*   [How to Get It](https://docs.slingdata.io/sling-cli/vscode#how-to-get-it)
    
*   [Working with Pipelines & Hooks](https://docs.slingdata.io/sling-cli/vscode#working-with-pipelines-and-hooks)
    
*   [Screenshots](https://docs.slingdata.io/sling-cli/vscode#screenshots)
    

Was this helpful?

sun-brightdesktopmoon

sun-brightdesktopmoon

---

# Running Sling | SlingData.IO

The `sling run` command is the primary mechanism for executing data movement operations in Sling CLI. It provides a flexible interface for transferring data between various sources and targets, with support for different replication modes and configuration options.

There are 2 primary ways to configure and run sling, using:

*   [**CLI Flags**](https://docs.slingdata.io/sling-cli/run#cli-flags-overview)
    : quick ad-hoc runs from your terminal shell or script.
    
*   [**Replication**](https://docs.slingdata.io/concepts/replication)
    : streams defined in a YAML or JSON file.
    

* * *

Furthermore, you'll find plenty of examples on how to use Sling:

*   [Database to Databasearrow-up-right](https://github.com/slingdata-io/sling-docs/blob/master/sling-cli/run/examples/database-to-database.md)
    
*   [Database to Filearrow-up-right](https://github.com/slingdata-io/sling-docs/blob/master/sling-cli/run/examples/database-to-file.md)
    
*   [File to Databasearrow-up-right](https://github.com/slingdata-io/sling-docs/blob/master/sling-cli/run/examples/file-to-database.md)
    

[hashtag](https://docs.slingdata.io/sling-cli/run#cli-flags-overview)

CLI Flags Overview


---------------------------------------------------------------------------------------------

For quickly running ad-hoc operations from the terminal, using CLI flags is often best. Here are some examples using:

Copy

    # Load all tables in a schema in with 3 threads
    $ export SLING_THREADS=3
    $ sling run \
        --src-conn MY_SOURCE_DB \
        --src-stream 'source_schema.*' \
        --tgt-conn MY_TARGET_DB \
        --tgt-object 'target_schema.{stream_table}'
        --mode full-refresh
        
    # Pipe in your json file and flatten the nested keys into their own columns
    $ cat /tmp/my_file.json | sling run --src-options '{"flatten": "true"}' --tgt-conn MY_TARGET_DB --tgt-object 'target_schema.target_table' --mode full-refresh
    
    # Read folder containing many CSV files
    $ sling run \
        --src-stream 'file:///tmp/my_csv_folder/' \
        --tgt-conn MY_TARGET_DB --tgt-object 'target_schema.target_table' \
        --mode full-refresh
    
    # Load only latest data from one source DB to another.
    $ sling run \
        --src-conn MY_SOURCE_DB \
        --src-stream 'source_schema.source_table' \
        --tgt-conn MY_TARGET_DB \
        --tgt-object 'target_schema.target_table' \
        --mode incremental \
        --primary-key 'id' --update-key 'last_modified_dt' 
    
    # Export / Backup database tables to JSON files
    $ sling run \
        --src-conn MY_SOURCE_DB \
        --src-stream 'source_schema.source_table' \
        --tgt-conn MY_S3_BUCKET \
        --tgt-object 's3://my-bucket/my_json_folder/' \
        --tgt-options '{"file_max_rows": 100000, "format": "jsonlines"}'

[hashtag](https://docs.slingdata.io/sling-cli/run#interface-specifications)

Interface Specifications


---------------------------------------------------------------------------------------------------------

CLI Flag

Description

`--src-conn`

The source database connection (name, conn string or URL).

`--tgt-conn`

The target database connection (name, conn string or URL).

`--src-stream`

The source table (schema.table), local / cloud file path. Can also be the path of sql file or in-line text to use as query. Use `file://` for local paths.

`--tgt-object`

The target table (schema.table) or local / cloud file path. Use `file://` for local paths. See [here](https://docs.slingdata.io/concepts/replication/runtime-variables)
 for details on runtime variables.

`--mode`

The target load [mode](https://docs.slingdata.io/concepts/replication/modes)
 to use: `incremental`, `truncate`, `full-refresh`, `backfill` or `snapshot`. Default is `full-refresh`.

`--primary-key`

The column(s) to use as primary key (for `incremental` mode). If composite key, use a comma-delimited string.

`--update-key`

The column to use as update key (for `incremental` mode).

`--src-options`

In-line options to further configure source (JSON or YAML). See [here](https://docs.slingdata.io/concepts/replication/source-options)
 for details.

`--tgt-options`

In-line options to further configure target (JSON or YAML). See [here](https://docs.slingdata.io/concepts/replication/target-options)
 for details.

`--stdout`

Output the stream to standard output (STDOUT).

`--select`

Select or exclude specific columns from the source stream. (comma separated). Use `-` prefix to exclude.

`--transforms`

An object/map, or array/list of built-in transforms to apply to records (JSON or YAML).

`--columns`

An object/map to specify the type that a column should be cast as (JSON or YAML).

`--streams`

Only run specific streams from a replication (comma separated). See [here](https://docs.slingdata.io/concepts/replication/tags-wildcards)
 for details.

[hashtag](https://docs.slingdata.io/sling-cli/run#features)

Features


-------------------------------------------------------------------------

*   **Flexible Data Sources**: Supports databases, files, cloud storage, and standard input
    
*   **Multiple Load Modes**: Includes full refresh, incremental, snapshot, and truncate modes
    
*   **Data Transformations**: Allows column selection, type casting, and custom transformations
    
*   **Progress Tracking**: Monitors row counts, bytes transferred, and constraint violations
    
*   **Error Handling**: Provides detailed error reporting and validation
    

The `sling run` command is designed to be both powerful and flexible, accommodating various data movement scenarios while maintaining ease of use through consistent parameter patterns and comprehensive documentation.

[PreviousEnvironmentchevron-left](https://docs.slingdata.io/sling-cli/environment)
[NextGlobal Variableschevron-right](https://docs.slingdata.io/sling-cli/variables)

Last updated 1 year ago

Was this helpful?

*   [CLI Flags Overview](https://docs.slingdata.io/sling-cli/run#cli-flags-overview)
    
*   [Interface Specifications](https://docs.slingdata.io/sling-cli/run#interface-specifications)
    
*   [Features](https://docs.slingdata.io/sling-cli/run#features)
    

Was this helpful?

sun-brightdesktopmoon

sun-brightdesktopmoon

---

# Environment | SlingData.IO

Sling looks for connection credentials in several places:

*   [Sling Env File](https://docs.slingdata.io/sling-cli/environment#sling-env-file-env.yaml)
     (located at `~/.sling/env.yaml`)
    
*   [Project Env File](https://docs.slingdata.io/sling-cli/environment#project-env-file-.env.sling)
     (`.env.sling` in the current working directory)
    
*   [DBT Profiles Files](https://docs.slingdata.io/sling-cli/environment#dbt-profiles-dbt-profiles.yml)
     (located at `~/.dbt/profiles.yml`)
    
*   [Environment Variables](https://docs.slingdata.io/sling-cli/environment#environment-variables)
    

One of the easiest ways is to manage your connections is to use the `sling conns` sub-command. Follow in the next section.

[hashtag](https://docs.slingdata.io/sling-cli/environment#managing-connections)

Managing Connections


---------------------------------------------------------------------------------------------------------

Sling makes it easy to **set**, **list** and **test** connections. You can even see the available streams in a connection by using the **discover** sub-command.

Copy

    $ sling conns -h
    conns - Manage and interact with local connections
    
    See more details at https://docs.slingdata.io/sling-cli/
    
      Usage:
        conns [discover|list|set|test]
    
      Subcommands:
        discover   list available streams in connection
        list       list local connections detected
        test       test a local connection
        unset      remove a connection from the sling env file
        set        set a connection in the sling env file
        exec       execute a SQL query on a Database connection
    
      Flags:
           --version   Displays the program version string.
        -h --help      Displays help with available flag, subcommand, and positional value parameters.

### 

[hashtag](https://docs.slingdata.io/sling-cli/environment#set-connections)

Set Connections

Here we can easily set a connection with the `sling conns set` command and later refer to them by their name. This ensures credentials are not visible by other users when using process monitors, for example.

To see what credential keys are necessary/accepted for each type of connector, click below:

*   File/Storage Connections (see [here](https://docs.slingdata.io/connections/file-connections)
    )
    
*   Database Connections (see [here](https://docs.slingdata.io/connections/database-connections)
    )
    

### 

[hashtag](https://docs.slingdata.io/sling-cli/environment#list-connections)

List Connections

Once connections are set, we can run the `sling conns list` command to list our detected connections:

### 

[hashtag](https://docs.slingdata.io/sling-cli/environment#test-connections)

Test Connections

We can also test a connection by running the `sling conns test` command:

### 

[hashtag](https://docs.slingdata.io/sling-cli/environment#discover-connections)

Discover Connections

We can easily discover streams available in a connection with the `sling conns discover` command:

Show column level information:

[hashtag](https://docs.slingdata.io/sling-cli/environment#credentials-location)

Credentials Location


---------------------------------------------------------------------------------------------------------

### 

[hashtag](https://docs.slingdata.io/sling-cli/environment#sling-env-file-env.yaml)

Sling Env File (`env.yaml`)

The Sling Env file is the primary way sling reads connections globally. It needs to be saved in the path `~/.sling/env.yaml` where the `~` denotes the path of the user Home folder, which can have different locations depending on the operating system (see [here for Windowsarrow-up-right](https://stackoverflow.com/a/42966089/2295355)
, [here for Macarrow-up-right](https://apple.stackexchange.com/a/51282)
 and [here for Linuxarrow-up-right](https://www.linuxshelltips.com/find-user-home-directory-linux/)
). Sling automatically creates the `.sling` folder in the user home directory, which is typically as shown below:

*   Linux: `/home/<username>/.sling`, or `/root/.sling` if user is `root`
    
*   Mac: `/Users/<username>/.sling`
    
*   Windows: `C:\Users\<username>\.sling`
    

Once in the user home directory, setting the Sling Env File (named `env.yaml`) is easy, and adheres to the structure below. Running `sling` the first time will auto-create it. You can alternatively provide the environment variable `SLING_HOME_DIR`.

To see what credential keys are necessary/accepted for each type of connector, click below:

*   File/Storage Connections (see [here](https://docs.slingdata.io/connections/file-connections)
    )
    
*   Database Connections (see [here](https://docs.slingdata.io/connections/database-connections)
    )
    

### 

[hashtag](https://docs.slingdata.io/sling-cli/environment#dot-env-file-.env.sling)

Dot-Env File (`.env.sling`)

Sling automatically loads a `.env.sling` file from the **current working directory** when it starts. This lets you define per-project connections and variables without modifying the global `env.yaml`. This applies to version _1.5.10+_.

This is useful when you have multiple projects, each with their own connections or credentials. Simply place a `.env.sling` file in the project directory, and Sling will pick it up automatically when run from that directory.

The file uses a simple `KEY=VALUE` format (one per line). Comments and blank lines are supported. Values can optionally be wrapped in single or double quotes.

circle-info

**Existing environment variables are not overwritten.** If a variable is already set in the shell environment, the value from `.env.sling` will be ignored. This means shell-level overrides always take precedence.

circle-exclamation

Since `.env.sling` may contain sensitive credentials, make sure to add it to your `.gitignore` to avoid accidentally committing secrets to version control.

### 

[hashtag](https://docs.slingdata.io/sling-cli/environment#environment-variables)

Environment Variables

Sling also reads environment variables. Simply export a connection URL (or YAML payload) to the current shell environment to use them.

To see examples of setting environment variables for each type of connector, click below:

*   File/Storage Connections (see [here](https://docs.slingdata.io/connections/file-connections)
    )
    
*   Database Connections (see [here](https://docs.slingdata.io/connections/database-connections)
    )
    

Mac / Linux

Windows

### 

[hashtag](https://docs.slingdata.io/sling-cli/environment#dbt-profiles-dbt-profiles.yml)

DBT Profiles (`~/dbt/profiles.yml`)

Sling also reads dbt profiles connections! If you're already set up with dbt cli locally, you don't need to create additional duplicate connections.

See [herearrow-up-right](https://docs.getdbt.com/dbt-cli/configure-your-profile)
 for more details.

[hashtag](https://docs.slingdata.io/sling-cli/environment#location-string)

Location String


-----------------------------------------------------------------------------------------------

The location string is a way to describe where sling should look for a file object or database object. It is used in a few places, such as the [`SLING_STATE`](https://docs.slingdata.io/sling-cli/cli-pro#file--state-based-incremental-loading)
 env var, as well as [Hooks](https://docs.slingdata.io/concepts/hooks)
 such as [`delete`](https://docs.slingdata.io/concepts/hooks/delete)
, [`copy`](https://docs.slingdata.io/concepts/hooks/copy)
 and [`inspect`](https://docs.slingdata.io/concepts/hooks/inspect)
.

The proper input format is `CONN_NAME/path/to/key` for storage connections, or `CONN_NAME/[database.]schema.table` for database objects.

Local location examples:

*   `local/relative/path`
    
*   `local/../parent/relative/path`
    
*   `local//absolute/linux/path`
    
*   `local/C:/absolute/windows/path`
    

For cloud or remote storage connections (with defined `AWS_S3`, `GCP`, `AZURE`, `SFTP` connections):

*   `aws_s3/path/to/folder`
    
*   `gcp/path/to/folder/file.parquet`
    
*   `azure/path/to/folder/file.log`
    
*   `sftp/relative/path/to/folder/file.log`
    
*   `sftp//absolute/path/to/folder/file.log`
    

Database location examples (for hooks like `inspect`):

*   `postgres/public.users` - PostgreSQL table in public schema
    
*   `mysql_db/analytics.events` - MySQL table in analytics schema
    
*   `snowflake/DATABASE.SCHEMA.TABLE` - Snowflake table with explicit database
    
*   `bigquery/project.dataset.table_name` - BigQuery table
    
*   `oracle_db/HR.EMPLOYEES` - Oracle table in HR schema
    
*   `mssql/dbo.customers` - SQL Server table in dbo schema
    

circle-exclamation

For **file storage connections** (`local`, `ftp` and `sftp`), you can specify a relative or absolute path. For FTP connections, it will be relative to the default folder of the username connecting.

**Relative Path**: You use the typical single slash (`/`) after the connection name:

*   `local/relative/path`
    
*   `sftp/relative/path`
    
*   `ftp/relative/path`
    

**Absolute Path**: You need to add 2 slashes (`//`) after the connection name:

*   `local//absolute/path`
    
*   `local/C:/absolute/path`
    
*   `sftp//absolute/path`
    
*   `ftp//absolute/path`
    

For **database connections**, use the standard database object naming convention: `connection_name/[database.]schema.table_name`

[PreviousInstallationchevron-left](https://docs.slingdata.io/sling-cli/getting-started)
[NextRunning Slingchevron-right](https://docs.slingdata.io/sling-cli/run)

Last updated 1 month ago

Was this helpful?

*   [Managing Connections](https://docs.slingdata.io/sling-cli/environment#managing-connections)
    
*   [Set Connections](https://docs.slingdata.io/sling-cli/environment#set-connections)
    
*   [List Connections](https://docs.slingdata.io/sling-cli/environment#list-connections)
    
*   [Test Connections](https://docs.slingdata.io/sling-cli/environment#test-connections)
    
*   [Discover Connections](https://docs.slingdata.io/sling-cli/environment#discover-connections)
    
*   [Credentials Location](https://docs.slingdata.io/sling-cli/environment#credentials-location)
    
*   [Sling Env File (env.yaml)](https://docs.slingdata.io/sling-cli/environment#sling-env-file-env.yaml)
    
*   [Dot-Env File (.env.sling)](https://docs.slingdata.io/sling-cli/environment#dot-env-file-.env.sling)
    
*   [Environment Variables](https://docs.slingdata.io/sling-cli/environment#environment-variables)
    
*   [DBT Profiles (~/dbt/profiles.yml)](https://docs.slingdata.io/sling-cli/environment#dbt-profiles-dbt-profiles.yml)
    
*   [Location String](https://docs.slingdata.io/sling-cli/environment#location-string)
    

Was this helpful?

sun-brightdesktopmoon

Copy

    # set a connection by providing the key=value pairs
    $ sling conns set AWS_S3 type=s3 bucket=sling-bucket access_key_id=ACCESS_KEY_ID secret_access_key="SECRET_ACCESS_KEY"
    
    # we set a database connection with just the url
    $ sling conns set MY_PG url='postgresql://postgres:myPassword@pghost:5432/postgres'

Copy

    $ sling conns list
    +--------------------------+-----------------+-------------------+
    | CONN NAME                | CONN TYPE       | SOURCE            |
    +--------------------------+-----------------+-------------------+
    | AWS_S3                   | FileSys - S3    | sling env yaml    |
    | FINANCE_BQ               | DB - BigQuery   | sling env yaml    |
    | DO_SPACES                | FileSys - S3    | sling env yaml    |
    | LOCALHOST_DEV            | DB - PostgreSQL | dbt profiles yaml |
    | MSSQL                    | DB - SQLServer  | sling env yaml    |
    | MYSQL                    | DB - MySQL      | sling env yaml    |
    | ORACLE_DB                | DB - Oracle     | env variable      |
    | MY_PG                    | DB - PostgreSQL | sling env yaml    |
    +--------------------------+-----------------+-------------------+

Copy

    $ sling conns test LOCALHOST_DEV
    9:04AM INF success!

Copy

    $ sling conns discover postgres --pattern public.work*
    +---+--------+-------------------+-------+---------+
    | # | SCHEMA | NAME              | TYPE  | COLUMNS |
    +---+--------+-------------------+-------+---------+
    | 1 | public | worker_heartbeats | table |      14 |
    | 2 | public | workers           | table |      20 |
    | 3 | public | workspaces        | table |       9 |
    +---+--------+-------------------+-------+---------+
    
    $ sling conns discover aws_s3
    +---+------------------+-----------+---------+-------------------------------+
    | # | NAME             | TYPE      | SIZE    | LAST UPDATED (UTC)            |
    +---+------------------+-----------+---------+-------------------------------+
    | 1 | logging/         | directory | -       | -                             |
    | 2 | sling_test/      | directory | -       | -                             |
    | 3 | work/            | directory | -       | -                             |
    | 4 | temp/            | directory | -       | -                             |
    | 5 | records.json     | file      | 442 KiB | 2022-12-07 11:05:01 (1y ago)  |
    | 6 | test.sqlite.db   | file      | 4.8 MiB | 2022-12-14 21:00:48 (1y ago)  |
    | 7 | test1.parquet    | file      | 48 KiB  | 2024-03-31 22:54:52 (29d ago) |
    | 8 | test_1000.csv    | file      | 99 KiB  | 2024-02-23 09:53:13 (67d ago) |
    +---+------------------+-----------+---------+-------------------------------+

Copy

    $ sling conns discover postgres -p public.workspaces --columns
    +----------+--------+------------+----+--------------+--------------------------+--------------+
    | DATABASE | SCHEMA | TABLE      | ID | COLUMN       | NATIVE TYPE              | GENERAL TYPE |
    +----------+--------+------------+----+--------------+--------------------------+--------------+
    | postgres | public | workspaces |  1 | id           | bigint                   | bigint       |
    | postgres | public | workspaces |  2 | account_id   | bigint                   | bigint       |
    | postgres | public | workspaces |  3 | name         | text                     | text         |
    | postgres | public | workspaces |  4 | short_name   | varchar                  | string       |
    | postgres | public | workspaces |  5 | token        | text                     | text         |
    | postgres | public | workspaces |  6 | settings     | jsonb                    | json         |
    | postgres | public | workspaces |  7 | created_dt   | timestamp with time zone | timestampz   |
    | postgres | public | workspaces |  8 | updated_dt   | timestamp with time zone | timestampz   |
    | postgres | public | workspaces |  9 | deleted_dt   | timestamp with time zone | timestampz   |
    +----------+--------+------------+----+--------------+--------------------------+--------------+
    
    $ sling conns discover aws_s3 -p test1.parquet --columns
    +---------------------------------+----+------------------+----------------+--------------+
    | FILE                            | ID | COLUMN           | NATIVE TYPE    | GENERAL TYPE |
    +---------------------------------+----+------------------+----------------+--------------+
    | s3://my-bucket/test1.parquet    |  1 | id               | INT_64         | bigint       |
    | s3://my-bucket/test1.parquet    |  2 | first_name       | UTF8           | string       |
    | s3://my-bucket/test1.parquet    |  3 | last_name        | UTF8           | string       |
    | s3://my-bucket/test1.parquet    |  4 | email            | UTF8           | string       |
    | s3://my-bucket/test1.parquet    |  5 | target           | BOOLEAN        | bool         |
    | s3://my-bucket/test1.parquet    |  6 | create_dt        | Timestamp      | datetime     |
    | s3://my-bucket/test1.parquet    |  7 | date             | Timestamp      | datetime     |
    | s3://my-bucket/test1.parquet    |  8 | rating           | DECIMAL        | decimal      |
    | s3://my-bucket/test1.parquet    |  9 | code             | DECIMAL        | decimal      |
    | s3://my-bucket/test1.parquet    | 10 | json_data        | UTF8           | string       |
    | s3://my-bucket/test1.parquet    | 11 | _sling_loaded_at | INT_64         | bigint       |
    +---------------------------------+----+------------------+----------------+--------------+

Copy

    # Holds all connection credentials for Extraction and Loading
    connections:
      marketing_pg:
        url: 'postgres://...' 
        ssh_tunnel: 'ssh://...' # optional
      
      # or dbt profile styled
      marketing_pg:
        type: postgres        
        host: [hostname]      
        user: [username]      
        password: ${PASSWORD} # you can pass in environment variables as well
        port: [port]          
        dbname: [database name]
        schema: [dbt schema]  
        ssh_tunnel: 'ssh://...' 
      
      finance_bq:
        type: bigquery
        method: service-account
        project: [GCP project id]
        dataset: [the name of your dbt dataset]
        keyfile: [/path/to/bigquery/keyfile.json]
    
    # Global variables for specific settings, available to all connections at runtime (Optional)
    variables:
      SLING_CLI_TOKEN: xxxxxxxxxxxxxxxx  # picked up machine wide
      SLING_LOG_DIR: ~/.sling/logs       # write debug logs here
      aws_access_key: '...'
      aws_secret_key: '...'

Copy

    # .env.sling - project-specific connections
    
    # Database connections
    MY_PG='postgresql://user:password@localhost:5432/mydb'
    STAGING_PG='postgresql://user:password@staging-host:5432/mydb'
    
    # File storage
    MY_S3='{type: s3, bucket: my-project-bucket, access_key_id: AKID, secret_access_key: SECRET}'
    
    # API Connection
    SALESFORCE='{ type: api, spec: salesforce, secrets: { client_id: "xxxxxxxx", client_secret: "xxxxxxxx", instance: "mycompany.my.salesforce.com" } }'
    
    # Other variables
    SLING_LOG_DIR=/tmp/logs
    SLING_LOADED_AT_COLUMN=timestamp
    SLING_CLI_TOKEN=xxxxxxxxxxxxxxxx

Copy

    $ export MY_PG='postgresql://user:[email protected]:5432/db1'
    $ export MY_PG='{type: postgres, host: "pg.host", user: user, database: "db1", password: "mypassw", port: 5432}'
    
    $ export MY_SNOWFLAKE='snowflake://user:[email protected]/db1'
    $ export MY_SNOWFLAKE='{type: snowflake, host: "<host>", user: "<user>", database: "<database>", password: "<password>", role: "<role>"}'
    
    $ export ORACLE_DB='oracle://user:[email protected]:1521/db1'
    
    $ export BIGQUERY_DB='{type: bigquery, dataset: public, key_file: /path/to/service.json, project: my-google-project}' # yaml or json form is also accepted
    
    $ sling conns list
    +---------------+------------------+-----------------+
    | CONN NAME     | CONN TYPE        | SOURCE          |
    +---------------+------------------+-----------------+
    | MY_PG         | DB - PostgreSQL  | env variable    |
    | MY_SNOWFLAKE  | DB - Snowflake   | env variable    |
    | ORACLE_DB     | DB - Oracle      | env variable    |
    | BIGQUERY_DB   | DB - Big Query   | env variable    |
    +---------------+------------------+-----------------+

Copy

    $ $env:MY_PG='postgresql://user:[email protected]:5432/db1'
    $ $env:MY_PG='{type: postgres, host: "pg.host", user: user, database: "db1", password: "mypassw", port: 5432}'
    
    $ $env:MY_SNOWFLAKE='snowflake://user:[email protected]/db1'
    $ $env:MY_SNOWFLAKE='{type: snowflake, host: "sf.host", user: user, database: db1, password: "mypassw", role: "<role>"}'
    
    $ $env:ORACLE_DB='oracle://user:[email protected]:1521/db1'
    
    $ $env:BIGQUERY_DB='{type: bigquery, dataset: public, key_file: /path/to/service.json, project: my-google-project}' # yaml or json form is also accepted
    
    $ sling conns list
    +---------------+------------------+-----------------+
    | CONN NAME     | CONN TYPE        | SOURCE          |
    +---------------+------------------+-----------------+
    | MY_PG         | DB - PostgreSQL  | env variable    |
    | MY_SNOWFLAKE  | DB - Snowflake   | env variable    |
    | ORACLE_DB     | DB - Oracle      | env variable    |
    +---------------+------------------+-----------------+

Copy

    $ sling conns list
    +------------------+------------------+-------------------+
    | CONN NAME        | CONN TYPE        | SOURCE            |
    +------------------+------------------+-------------------+
    | SNOWCASTLE_DEV   | DB - Snowflake   | dbt profiles yaml |
    | SNOWCASTLE_PROD  | DB - Snowflake   | dbt profiles yaml |
    +------------------+------------------+-------------------+

sun-brightdesktopmoon

---

# Architecture | SlingData.IO

The Sling Data Platform consists of three main components that work together to provide a scalable and secure data movement solution:

1.  Control Server & Database
    
2.  Agents (Self-hosted)
    
3.  Frontend UI
    

[hashtag](https://docs.slingdata.io/sling-platform/platform/architecture#system-overview)

System Overview


--------------------------------------------------------------------------------------------------------------

![](https://docs.slingdata.io/~gitbook/image?url=https%3A%2F%2F3453272330-files.gitbook.io%2F%7E%2Ffiles%2Fv0%2Fb%2Fgitbook-x-prod.appspot.com%2Fo%2Fspaces%252F-M93cpHl7B7NPZlDrubS%252Fuploads%252Fgit-blob-9e8f30ab5666515edb23e357755204f17a0836bc%252Fsling-platform-architecture.png%3Falt%3Dmedia&width=768&dpr=3&quality=100&sign=17f6e1f2&sv=2)

Sling Platform Architecture

### 

[hashtag](https://docs.slingdata.io/sling-platform/platform/architecture#control-plane)

Control Plane

The Control Plane is the central coordination point of the Sling Platform that:

*   Manages user authentication and authorization
    
*   Stores configuration data (connections, replications, etc.)
    
*   Coordinates job scheduling and execution
    
*   Maintains system state and job history
    
*   Communicates with agents via secure NATS websockets
    
*   Provides REST API endpoints for the frontend UI
    

### 

[hashtag](https://docs.slingdata.io/sling-platform/platform/architecture#agents)

Agents

Agents are distributed workers that:

*   Run the Sling CLI to execute data jobs
    
*   Operate within your infrastructure for secure data access
    
*   Connect to the Control Server via secure NATS websockets
    
*   Can be deployed across multiple environments (dev, prod, etc.)
    
*   Scale horizontally to handle increased workload
    
*   Provide secure access to data sources without exposing credentials
    

### 

[hashtag](https://docs.slingdata.io/sling-platform/platform/architecture#frontend-ui)

Frontend UI

The Frontend UI provides:

*   Web-based interface for managing Sling operations
    
*   Secure communication with Control Server via HTTPS
    
*   Visual workflow creation and management
    
*   Real-time monitoring and alerting
    
*   Team collaboration features
    
*   Connection management interface
    

[hashtag](https://docs.slingdata.io/sling-platform/platform/architecture#communication-flow)

Communication Flow


--------------------------------------------------------------------------------------------------------------------

1.  Users interact with the Frontend UI over HTTPS
    
2.  Frontend UI communicates with Control Server via REST API
    
3.  Control Server coordinates with Agents using NATS secure websockets
    
4.  Agents execute jobs and report status back to Control Server
    
5.  Control Server updates Frontend UI with job status and results
    

[hashtag](https://docs.slingdata.io/sling-platform/platform/architecture#security)

Security


------------------------------------------------------------------------------------------------

*   All communication between components uses encryption (HTTPS, WSS)
    
*   Agents run in your infrastructure, keeping sensitive data and credentials secure. No ports need to be opened.
    
*   Authentication and authorization managed by Control Server
    
*   Credentials stored securely in Control Server database, or on your self-hosted agent.
    

[PreviousSling Platformchevron-left](https://docs.slingdata.io/sling-platform/platform)
[NextAgentschevron-right](https://docs.slingdata.io/sling-platform/platform/agents)

Last updated 1 year ago

Was this helpful?

*   [System Overview](https://docs.slingdata.io/sling-platform/platform/architecture#system-overview)
    
*   [Control Plane](https://docs.slingdata.io/sling-platform/platform/architecture#control-plane)
    
*   [Agents](https://docs.slingdata.io/sling-platform/platform/architecture#agents)
    
*   [Frontend UI](https://docs.slingdata.io/sling-platform/platform/architecture#frontend-ui)
    
*   [Communication Flow](https://docs.slingdata.io/sling-platform/platform/architecture#communication-flow)
    
*   [Security](https://docs.slingdata.io/sling-platform/platform/architecture#security)
    

Was this helpful?

sun-brightdesktopmoon

sun-brightdesktopmoon

---

# Sling Platform | SlingData.IO

Sling Platform UI

[hashtag](https://docs.slingdata.io/sling-platform/platform#why-sling-platform)

Why Sling Platform


-------------------------------------------------------------------------------------------------------

Sling Platform provides a comprehensive web-based interface for managing data operations at scale. Key benefits include:

*   Visual interface for creating and managing data workflows
    
*   Agent-based architecture for secure and scalable execution
    
*   Team collaboration features
    
*   Centralized connection management
    
*   Built-in monitoring and alerting
    
*   Job scheduling and orchestration
    

[hashtag](https://docs.slingdata.io/sling-platform/platform#platform-features)

Platform Features


-----------------------------------------------------------------------------------------------------

The paid platform plan includes exclusive capabilities:

*   ✅ [Smart Editor (IDE)](https://docs.slingdata.io/sling-platform/platform/editor)
     - Live compilation, auto-complete, syntax highlighting, and validation for replications
    
*   ✅ [Alerting](https://docs.slingdata.io/sling-platform/platform/editor#managing-jobs)
     - Email, Slack, and MS Teams notifications for job status (error, warning, success)
    
*   ✅ [Job Scheduling](https://docs.slingdata.io/sling-platform/platform/editor#managing-jobs)
     - Schedule and orchestrate data jobs with cron expressions
    
*   ✅ [Self-Hosted Agents](https://docs.slingdata.io/sling-platform/platform/agents)
     - Deploy agents in your infrastructure for secure data access
    
*   ✅ [REST API](https://docs.slingdata.io/sling-platform/platform/api)
     - Programmatic access to connections, jobs, executions, and git sync
    
*   ✅ [Job Run History](https://docs.slingdata.io/sling-platform/platform/editor#job-run-history)
     - View detailed history and logs for all job executions
    

### 

[hashtag](https://docs.slingdata.io/sling-platform/platform#cli-pro-features)

CLI Pro Features

The paid platform plan also includes all of the [CLI Pro](https://docs.slingdata.io/sling-cli/cli-pro)
 features:

*   ✅ [API Sources](https://docs.slingdata.io/sling-cli/cli-pro#api-sources)
     (extract data from any REST API by using Specs)
    
*   ✅ [Parallel Stream Processing](https://docs.slingdata.io/sling-cli/cli-pro#stream-chunking-and-parallel-processing)
     (run streams in parallel with automatic retries)
    
*   ✅ [Stream Chunking](https://docs.slingdata.io/sling-cli/cli-pro#stream-chunking-and-parallel-processing)
     (split large streams into smaller chunks)
    
*   ✅ [Pipelines & Hooks](https://docs.slingdata.io/sling-cli/cli-pro#pipelines-and-hooks)
     (such as `http`, `query`, `check`, `copy`, `command` and more)
    
*   ✅ [OpenTelemetry Logging](https://docs.slingdata.io/sling-cli/cli-pro#opentelemetry-logging)
     (export structured logs to any OTLP endpoint)
    
*   ✅ [Capture Deletes](https://docs.slingdata.io/sling-cli/cli-pro#capture-deletes-cdc)
     (similar to CDC)
    
*   ✅ [Staged Transforms](https://docs.slingdata.io/sling-cli/cli-pro#staged-transforms)
     (advanced multi-stage transformations with expressions and functions)
    
*   ✅ [State Based Incremental](https://docs.slingdata.io/sling-cli/cli-pro#state-based-incremental-loading)
     (file target incremental mode)
    
*   ✅ [Schema Migration](https://docs.slingdata.io/sling-cli/cli-pro#schema-migration)
     (migrate primary keys, foreign keys, indexes, defaults, and more)
    
*   ✅ [ODBC Connections](https://docs.slingdata.io/sling-cli/cli-pro#odbc-connections)
     (connect to any database via ODBC drivers)
    

### 

[hashtag](https://docs.slingdata.io/sling-platform/platform#advanced-plan-features)

Advanced Plan Features

The following features are available with the [Advanced planarrow-up-right](https://slingdata.io/platform/#pricing)
:

*   ✅ [Platform Self-Hosting](https://docs.slingdata.io/sling-platform/platform/self-hosting)
     - Host the entire platform in your private network for full control and security
    
*   ✅ [Git Integration](https://docs.slingdata.io/sling-platform/platform/api#git-sync)
     - Connect to GitHub, GitLab, or Bitbucket for version control and CI/CD workflows
    
*   ✅ User Roles - Define granular access controls with custom user roles and permissions
    
*   ✅ Audit Logs - Comprehensive logging and tracking of all platform activities for compliance
    

![](https://docs.slingdata.io/~gitbook/image?url=https%3A%2F%2F3453272330-files.gitbook.io%2F%7E%2Ffiles%2Fv0%2Fb%2Fgitbook-x-prod.appspot.com%2Fo%2Fspaces%252F-M93cpHl7B7NPZlDrubS%252Fuploads%252Fgit-blob-22e0411029f6c0376d78bd0f2681dea8f806b073%252Fsling-platform-ui.png%3Falt%3Dmedia&width=768&dpr=3&quality=100&sign=946ebfa2&sv=2)

Sling Platform UI

[hashtag](https://docs.slingdata.io/sling-platform/platform#sign-up)

Sign Up


---------------------------------------------------------------------------------

1.  Visit [platform.slingdata.ioarrow-up-right](https://platform.slingdata.io/)
     to create your account
    
2.  Choose your preferred authentication method
    
3.  Create a Project
    
4.  Get Slinging
    

![](https://docs.slingdata.io/~gitbook/image?url=https%3A%2F%2F3453272330-files.gitbook.io%2F%7E%2Ffiles%2Fv0%2Fb%2Fgitbook-x-prod.appspot.com%2Fo%2Fspaces%252F-M93cpHl7B7NPZlDrubS%252Fuploads%252Fgit-blob-8ba0cee2b2928afb996b911a8078a008c2e158bc%252Fsling-platform-sign-up.png%3Falt%3Dmedia&width=768&dpr=3&quality=100&sign=cbf07fc1&sv=2)

Sling Platform Sign Up

[hashtag](https://docs.slingdata.io/sling-platform/platform#creating-a-project)

Creating a Project


-------------------------------------------------------------------------------------------------------

1.  After logging in, You will need to create a project
    
2.  Enter a project name, and click "Create Project"
    

![](https://docs.slingdata.io/~gitbook/image?url=https%3A%2F%2F3453272330-files.gitbook.io%2F%7E%2Ffiles%2Fv0%2Fb%2Fgitbook-x-prod.appspot.com%2Fo%2Fspaces%252F-M93cpHl7B7NPZlDrubS%252Fuploads%252Fgit-blob-830bf801574c74f0368c2fdcc7802e6702f3fbf4%252Fsling-platform-create-project.png%3Falt%3Dmedia&width=768&dpr=3&quality=100&sign=4d92c921&sv=2)

Sling Platform Create Project

Once your project is created, you can:

*   Set up connections to your data sources and targets
    
*   Deploy agents to securely access your data
    
*   Create and schedule replications
    
*   Monitor your data operations
    
*   Invite team members to collaborate
    

circle-info

Each project is isolated from other projects, meaning that you can have multiple projects, each with their own connections, agents, replications, etc.

For detailed instructions on specific features, please refer to the relevant sections in the documentation.

[PreviousMCP Serverchevron-left](https://docs.slingdata.io/sling-cli/ai/mcp)
[NextArchitecturechevron-right](https://docs.slingdata.io/sling-platform/platform/architecture)

Last updated 1 month ago

Was this helpful?

*   [Why Sling Platform](https://docs.slingdata.io/sling-platform/platform#why-sling-platform)
    
*   [Platform Features](https://docs.slingdata.io/sling-platform/platform#platform-features)
    
*   [CLI Pro Features](https://docs.slingdata.io/sling-platform/platform#cli-pro-features)
    
*   [Advanced Plan Features](https://docs.slingdata.io/sling-platform/platform#advanced-plan-features)
    
*   [Sign Up](https://docs.slingdata.io/sling-platform/platform#sign-up)
    
*   [Creating a Project](https://docs.slingdata.io/sling-platform/platform#creating-a-project)
    

Was this helpful?

sun-brightdesktopmoon

sun-brightdesktopmoon

---

# Connections | SlingData.IO

Connections are used to connect to external systems. For example, they are used to read and write data from databases, files, and other systems when running replications.

![](https://docs.slingdata.io/~gitbook/image?url=https%3A%2F%2F3453272330-files.gitbook.io%2F%7E%2Ffiles%2Fv0%2Fb%2Fgitbook-x-prod.appspot.com%2Fo%2Fspaces%252F-M93cpHl7B7NPZlDrubS%252Fuploads%252Fgit-blob-10858a01ad4753ab62e560b6c686e48e424b560e%252Fsling-platform-connections.png%3Falt%3Dmedia&width=768&dpr=3&quality=100&sign=6b3170a4&sv=2)

Sling Platform Connections

[hashtag](https://docs.slingdata.io/sling-platform/platform/connections#project-env.yaml-vs-agent-env.yaml)

Project env.yaml vs Agent env.yaml


---------------------------------------------------------------------------------------------------------------------------------------------------

Each project has an internal `env.yaml` file, which is used to store the connection credentials. This file is used to authenticate the agent when it needs to access the external system. When using the project env.yaml file, the credentials are securely transmitted over TLS and encrypted at rest.

You can also define a local `env.yaml` file (saved on the agent machine), which can be used to authenticate the agent when it needs to access the external system. When using the local env.yaml file, the credentials are never transmitted outside the agent machine. Only the name and type of the connection are sent to the control server for displaying on the UI.

### 

[hashtag](https://docs.slingdata.io/sling-platform/platform/connections#project-env.yaml)

Project env.yaml

To add a connection to the project env.yaml file, you can click on the `New` button on the connections page and then select the type of connection you want to add.

![](https://docs.slingdata.io/~gitbook/image?url=https%3A%2F%2F3453272330-files.gitbook.io%2F%7E%2Ffiles%2Fv0%2Fb%2Fgitbook-x-prod.appspot.com%2Fo%2Fspaces%252F-M93cpHl7B7NPZlDrubS%252Fuploads%252Fgit-blob-bd298bc45fab3edf1ca872b05e82301a6b1d80de%252Fsling-platform-connections-new.png%3Falt%3Dmedia&width=768&dpr=3&quality=100&sign=a7d75550&sv=2)

Sling Platform Connections New

You can then fill in the connection details and click on the `Save` button to add the connection to the project env.yaml file. You can also click on the `Test` button to test the connection before saving it.

![](https://docs.slingdata.io/~gitbook/image?url=https%3A%2F%2F3453272330-files.gitbook.io%2F%7E%2Ffiles%2Fv0%2Fb%2Fgitbook-x-prod.appspot.com%2Fo%2Fspaces%252F-M93cpHl7B7NPZlDrubS%252Fuploads%252Fgit-blob-4eb94e4d9d348b11b26fbd09d8270295947cda19%252Fsling-platform-connections-save.png%3Falt%3Dmedia&width=768&dpr=3&quality=100&sign=1a4c14cb&sv=2)

Sling Platform Connections Save

[hashtag](https://docs.slingdata.io/sling-platform/platform/connections#explore-connections)

Explore Connections


---------------------------------------------------------------------------------------------------------------------

You can explore the files or tables in a connection by clicking one on the left sidebar. This allows you to see which objects are available, as well as previewing the data inside. This is useful for getting familiar with the data, and for finding the correct object names for your replications.

![](https://docs.slingdata.io/~gitbook/image?url=https%3A%2F%2F3453272330-files.gitbook.io%2F%7E%2Ffiles%2Fv0%2Fb%2Fgitbook-x-prod.appspot.com%2Fo%2Fspaces%252F-M93cpHl7B7NPZlDrubS%252Fuploads%252Fgit-blob-61828ec7b0893808c91216e3fc857bec3cd34329%252Fsling-platform-connections-explore.png%3Falt%3Dmedia&width=768&dpr=3&quality=100&sign=2160263c&sv=2)

Sling Platform Connections Explore

circle-exclamation

Note that objects are accessed in a read-only fashion, therefore no changes can be made to the data.

[PreviousAgentschevron-left](https://docs.slingdata.io/sling-platform/platform/agents)
[NextEditorchevron-right](https://docs.slingdata.io/sling-platform/platform/editor)

Last updated 1 year ago

Was this helpful?

*   [Project env.yaml vs Agent env.yaml](https://docs.slingdata.io/sling-platform/platform/connections#project-env.yaml-vs-agent-env.yaml)
    
*   [Project env.yaml](https://docs.slingdata.io/sling-platform/platform/connections#project-env.yaml)
    
*   [Explore Connections](https://docs.slingdata.io/sling-platform/platform/connections#explore-connections)
    

Was this helpful?

sun-brightdesktopmoon

sun-brightdesktopmoon

---

# Editor | SlingData.IO

The Sling Data Platform Editor is a powerful tool for creating and editing replications. It allows you to visually build replications by incorporating an IDE like experience:

*   Live compilation of replications
    
*   Interactive visual editor for building replications
    
*   Auto-complete for source and target objects
    
*   Syntax highlighting for SQL and Sling
    
*   Validation for your replications
    

![](https://docs.slingdata.io/~gitbook/image?url=https%3A%2F%2F3453272330-files.gitbook.io%2F%7E%2Ffiles%2Fv0%2Fb%2Fgitbook-x-prod.appspot.com%2Fo%2Fspaces%252F-M93cpHl7B7NPZlDrubS%252Fuploads%252Fgit-blob-de09177e8e4eaa3a5bb2b7a339158f93b1b22d1e%252Fsling-platform-editor.png%3Falt%3Dmedia&width=768&dpr=3&quality=100&sign=37c0b8f7&sv=2)

Sling Platform Editor

[hashtag](https://docs.slingdata.io/sling-platform/platform/editor#referring-to-query-files)

Referring to Query Files


--------------------------------------------------------------------------------------------------------------------------

In order to refer to query files (to avoid putting large SQL text in your replication YAML), you can specify query files that exist in your project. Always use the relative path to the file from the root of your project. For example, the file `my_long_query.sql` below is in folder `queries` in the root:

[hashtag](https://docs.slingdata.io/sling-platform/platform/editor#managing-jobs)

Managing Jobs


----------------------------------------------------------------------------------------------------

You can manage your jobs by clicking on the `Jobs` button on the bottom bar. This allows you to see the status of your jobs, as well as creating new jobs.

![](https://docs.slingdata.io/~gitbook/image?url=https%3A%2F%2F3453272330-files.gitbook.io%2F%7E%2Ffiles%2Fv0%2Fb%2Fgitbook-x-prod.appspot.com%2Fo%2Fspaces%252F-M93cpHl7B7NPZlDrubS%252Fuploads%252Fgit-blob-a6fcfc146d71091f9569c70df6e3f1430b31f91e%252Fsling-platform-editor-jobs.png%3Falt%3Dmedia&width=768&dpr=3&quality=100&sign=6d745313&sv=2)

Sling Platform Editor Jobs

### 

[hashtag](https://docs.slingdata.io/sling-platform/platform/editor#using-a-specific-sling-cli-version)

Using a Specific Sling CLI Version

By default, any job will use the latest Sling CLI version to run the jobs. If you'd like to specify a specific version to use, you can do so with:

![](https://docs.slingdata.io/~gitbook/image?url=https%3A%2F%2F3453272330-files.gitbook.io%2F%7E%2Ffiles%2Fv0%2Fb%2Fgitbook-x-prod.appspot.com%2Fo%2Fspaces%252F-M93cpHl7B7NPZlDrubS%252Fuploads%252Fgit-blob-c17358e3498dbb7b4b095ca58ce316d82d23a3bb%252Fversion.png%3Falt%3Dmedia&width=768&dpr=3&quality=100&sign=d9ff66b4&sv=2)

Specific version

### 

[hashtag](https://docs.slingdata.io/sling-platform/platform/editor#using-the-dev-build)

Using the Dev Build

You can set your job to use the latest dev build (which is a preview of the upcoming release).

![](https://docs.slingdata.io/~gitbook/image?url=https%3A%2F%2F3453272330-files.gitbook.io%2F%7E%2Ffiles%2Fv0%2Fb%2Fgitbook-x-prod.appspot.com%2Fo%2Fspaces%252F-M93cpHl7B7NPZlDrubS%252Fuploads%252Fgit-blob-23d8e50a778d61490441c8c388e7d1cf410f555c%252Fdev_build.png%3Falt%3Dmedia&width=768&dpr=3&quality=100&sign=ffdadbb8&sv=2)

Setting the env var

[hashtag](https://docs.slingdata.io/sling-platform/platform/editor#job-run-history)

Job Run History


--------------------------------------------------------------------------------------------------------

You can view the history of your jobs by clicking on the `History` button on the bottom bar. This allows you to see the status of your job runs, as well as details about each run.

![](https://docs.slingdata.io/~gitbook/image?url=https%3A%2F%2F3453272330-files.gitbook.io%2F%7E%2Ffiles%2Fv0%2Fb%2Fgitbook-x-prod.appspot.com%2Fo%2Fspaces%252F-M93cpHl7B7NPZlDrubS%252Fuploads%252Fgit-blob-edf0b68e08c28dd69873a4ae6d345fcd8e8756a4%252Fsling-platform-editor-history.png%3Falt%3Dmedia&width=768&dpr=3&quality=100&sign=fd71b1c7&sv=2)

Sling Platform Job Run Details

[PreviousConnectionschevron-left](https://docs.slingdata.io/sling-platform/platform/connections)
[NextAPIchevron-right](https://docs.slingdata.io/sling-platform/platform/api)

Last updated 7 months ago

Was this helpful?

*   [Referring to Query Files](https://docs.slingdata.io/sling-platform/platform/editor#referring-to-query-files)
    
*   [Managing Jobs](https://docs.slingdata.io/sling-platform/platform/editor#managing-jobs)
    
*   [Using a Specific Sling CLI Version](https://docs.slingdata.io/sling-platform/platform/editor#using-a-specific-sling-cli-version)
    
*   [Using the Dev Build](https://docs.slingdata.io/sling-platform/platform/editor#using-the-dev-build)
    
*   [Job Run History](https://docs.slingdata.io/sling-platform/platform/editor#job-run-history)
    

Was this helpful?

sun-brightdesktopmoon

Copy

    source: MY_SOURCE
    target: MY_TARGET
    
    streams:
      my_custom_stream:
        sql: 'file://queries/my_long_query.sql'

sun-brightdesktopmoon

---