# Table of Contents - [Introduction](#introduction) - [Introduction](#introduction) - [Deploy Sublime](#deploy-sublime) - [AWS GovCloud](#aws-govcloud) - [Add your abuse mailbox](#add-your-abuse-mailbox) - [Add a Microsoft 365 message source](#add-a-microsoft-365-message-source) - [Add a message source](#add-a-message-source) - [Add a Google Workspace message source](#add-a-google-workspace-message-source) - [Sublime Cloud (Fully Managed) - Microsoft 365](#sublime-cloud-fully-managed-microsoft-365) - [Sublime Managed](#sublime-managed) - [Sublime Cloud (Fully Managed) - Google Workspace](#sublime-cloud-fully-managed-google-workspace) - [Azure via ARM](#azure-via-arm) - [Enable user-reported phishing](#enable-user-reported-phishing) - [Self Managed - Microsoft 365](#self-managed-microsoft-365) - [Metrics Collection in Self-Hosted Deployments](#metrics-collection-in-self-hosted-deployments) - [Add an IMAP message source](#add-an-imap-message-source) - [Self Managed - Google Workspace](#self-managed-google-workspace) - [Gmail's "Report phishing" feature](#gmail-s-report-phishing-feature) - [Microsoft 365's Native “Report phishing” feature](#microsoft-365-s-native-report-phishing-feature) - [MQL Detection Rules](#mql-detection-rules) - [AWS CloudFormation](#aws-cloudformation) - [Message types](#message-types) - [Attack Surface Reduction](#attack-surface-reduction) - [Rule Severity](#rule-severity) - [Rule Feeds](#rule-feeds) - [Message groups](#message-groups) - [Docker](#docker) - [Private rule feed authentication](#private-rule-feed-authentication) - [Trash](#trash) - [Warning Banners](#warning-banners) - [Auto-respond to User Reports](#auto-respond-to-user-reports) - [Track Link Clicks (beta)](#track-link-clicks-beta-) - [Rules file format (YAML)](#rules-file-format-yaml-) - [Automations](#automations) - [Auto-review](#auto-review) - [YARA](#yara) - [Email Alert with EML Attached](#email-alert-with-eml-attached) - [Email Alert](#email-alert) - [Exclusions](#exclusions) - [Message Access Controls](#message-access-controls) - [Slack Alert](#slack-alert) - [Configure the org_vips list](#configure-the-org-vips-list) - [Message Query Language (MQL)](#message-query-language-mql-) - [Message Data Model (MDM)](#message-data-model-mdm-) - [ADÉ: Autonomous Detection Engineer](#ad-autonomous-detection-engineer) - [Actions](#actions) - [Move to Spam](#move-to-spam) - [How to set up a custom domain](#how-to-set-up-a-custom-domain) - [Export Message MDMs](#export-message-mdms) - [How to detect text in attachments](#how-to-detect-text-in-attachments) - [How to use message header values in a rule](#how-to-use-message-header-values-in-a-rule) - [ASA: Autonomous Security Analyst](#asa-autonomous-security-analyst) - [Using the MQL Editor](#using-the-mql-editor) - [RegEx functions](#regex-functions) - [How to detect lookalike domains](#how-to-detect-lookalike-domains) - [Missing or null values](#missing-or-null-values) - [How to manage users with SCIM](#how-to-manage-users-with-scim) - [How to detect keywords or phrases in the body content of messages](#how-to-detect-keywords-or-phrases-in-the-body-content-of-messages) - [Lists](#lists) - [How to detect manual outbound forwards](#how-to-detect-manual-outbound-forwards) - [Role-Based Access Control (RBAC)](#role-based-access-control-rbac-) - [Functions](#functions) - [Quarantine](#quarantine) - [How to set up single sign-on (SSO)](#how-to-set-up-single-sign-on-sso-) - [Attack Score](#attack-score) - [How to Integrate Email Threat Intel with a TIP](#how-to-integrate-email-threat-intel-with-a-tip) - [Export to S3](#export-to-s3) - [Export Audit Logs and Message Events](#export-audit-logs-and-message-events) - [Common snippets](#common-snippets) - [Strings functions](#strings-functions) - [Webhook](#webhook) - [Syntax](#syntax) - [Enrichment functions](#enrichment-functions) - [Introduction](#introduction) - [Authentication](#authentication) - [Errors](#errors) - [Upload a binary to be binexploded](#upload-a-binary-to-be-binexploded) - [Get results of a binexplode scan](#get-results-of-a-binexplode-scan) - [Analyze a provided link using ml.link_analysis functionality](#analyze-a-provided-link-using-ml-link-analysis-functionality) - [List events in audit log](#list-events-in-audit-log) - [List event types for audit log](#list-event-types-for-audit-log) - [Introduction to the Message Data Model](#introduction-to-the-message-data-model) - [Writing your first rule](#writing-your-first-rule) - [Analysis API Use cases](#analysis-api-use-cases) --- # Introduction Sublime is a programmable, AI-powered, cloud email security platform for Microsoft 365 and Google Workspace environments. IMAP and APIs for direct ingestion are also supported. Use the Sublime Platform to: * block email attacks such as phishing, BEC, and malware * threat hunt * auto-triage user reports At its core, Sublime is a _rules engine_ that ingests email messages from arbitrary sources, evaluates them using a [powerful query language](https://docs.sublime.security/docs/message-query-language) , and then takes any number of actions like [quarantining a message](https://docs.sublime.security/docs/quarantine) , generating a [webhook notification](https://docs.sublime.security/docs/webhooks) , or inserting a [warning banner](https://docs.sublime.security/docs/warning-banners) . Below is an example of a simple rule: YAML `name: HTML smuggling via attachment severity: high source: | type.inbound and any(attachments, .file_extension in~ ('html', 'htm') and any(file.explode(.), any(.scan.javascript.identifiers, . == "unescape") ) ) tags: - "HTML smuggling"` Updated about 1 year ago * * * * [Installation](https://docs.sublime.security/docs/installation) * [Add a message source](https://docs.sublime.security/docs/add-message-source) --- # Introduction Sublime is a programmable, AI-powered, cloud email security platform for Microsoft 365 and Google Workspace environments. IMAP and APIs for direct ingestion are also supported. Use the Sublime Platform to: * block email attacks such as phishing, BEC, and malware * threat hunt * auto-triage user reports At its core, Sublime is a _rules engine_ that ingests email messages from arbitrary sources, evaluates them using a [powerful query language](https://docs.sublime.security/docs/message-query-language) , and then takes any number of actions like [quarantining a message](https://docs.sublime.security/docs/quarantine) , generating a [webhook notification](https://docs.sublime.security/docs/webhooks) , or inserting a [warning banner](https://docs.sublime.security/docs/warning-banners) . Below is an example of a simple rule: YAML `name: HTML smuggling via attachment severity: high source: | type.inbound and any(attachments, .file_extension in~ ('html', 'htm') and any(file.explode(.), any(.scan.javascript.identifiers, . == "unescape") ) ) tags: - "HTML smuggling"` Updated about 1 year ago * * * * [Installation](https://docs.sublime.security/docs/installation) * [Add a message source](https://docs.sublime.security/docs/add-message-source) --- # Deploy Sublime * [Managed](https://platform.sublime.security/) : For organizations of any size. First 100 mailboxes free. * [Self-Managed: Docker](https://docs.sublime.security/docs/quickstart-docker) : Up to 600 active mailboxes. * [Self-Managed: AWS CloudFormation](https://docs.sublime.security/docs/aws-cloudformation) : Scales to any number of mailboxes. * [Self-Managed: Azure via ARM](https://docs.sublime.security/docs/azure-via-arm) : Scales to any number of mailboxes. * [EML Analyzer](https://analyzer.sublime.security/) : Rapid phishing investigations, no setup required. Updated 5 months ago * * * --- # AWS GovCloud Overview [](https://docs.sublime.security/docs/aws-govcloud#overview) ========================================================================= The Sublime GovCloud stack uses docker images to run AWS Lambda, which powers the message screenshots detections. In CloudFormation stacks in regular AWS regions, we publish these docker images in AWS' Elastic Container Registry in an AWS account we own, and AWS Lambda uses these docker images (the same pattern is used for other Sublime services). In GovCloud, Lambda does not yet support using docker images from another AWS account. We have confirmed this disparity with AWS and are waiting on the expansion of the feature to GovCloud. To work around this limitation, a few manual steps are required _before_ deploying Sublime in GovCloud. > ❗️ > > Deployment failures > > > ------------------------- > > GovCloud CloudFormation deployments are required to complete the steps listed below before moving forward with your AWS deployment. If these steps are not completed, the stack will fail to deploy! Requirements [](https://docs.sublime.security/docs/aws-govcloud#requirements) ================================================================================= 1. AWS credentials for your destination GovCloud AWS account Permissions for ECR login, creating a repository, and pulling and pushing to the repository are necessary. 2. A terminal with the AWS CLI and docker installed Steps to Create the Registry [](https://docs.sublime.security/docs/aws-govcloud#steps-to-create-the-registry) ----------------------------------------------------------------------------------------------------------------- In the steps, we'll assume your credentials for your GovCloud AWS account are in an AWS profile named `gov_cloud`. These steps could also be done in the AWS console, but we'll do this via the CLI since we'll need these credentials later anyway. 1. Confirm credentials `AWS_PROFILE=gov_cloud aws sts get-caller-identity` Confirm that the "Arn" in the response includes "aws-us-gov" and your expected account ID. Take note of the account ID for later steps. 2. Create both required repositories `AWS_PROFILE=gov_cloud aws ecr create-repository --repository-name url-defense --region us-gov-west-1 AWS_PROFILE=gov_cloud aws ecr create-repository --repository-name render-email-html-lambda --region us-gov-west-1` Set Repository Contents [](https://docs.sublime.security/docs/aws-govcloud#set-repository-contents) ------------------------------------------------------------------------------------------------------- 1. Login to ECR repos (`YOUR_GOV_CLOUD_ACCOUNT_ID` must be replaced on the first line) `AWS_PROFILE=gov_cloud aws ecr get-login-password --region us-gov-west-1 | docker login --username AWS --password-stdin YOUR_GOV_CLOUD_ACCOUNT_ID.dkr.ecr.us-gov-west-1.amazonaws.com AWS_PROFILE=gov_cloud aws ecr get-login-password --region us-gov-west-1 | docker login --username AWS --password-stdin 106524781139.dkr.ecr.us-gov-west-1.amazonaws.com` 1. Pull Sublime published docker images from ECR Note `106524781139` is a Sublime managed account that contains the latest versions of the docker images. It should _not_ be replaced. `docker pull 106524781139.dkr.ecr.us-gov-west-1.amazonaws.com/url-defense:latest docker pull 106524781139.dkr.ecr.us-gov-west-1.amazonaws.com/render-email-html-lambda:latest` 2. Retag images for your ECR repo (`YOUR_GOV_CLOUD_ACCOUNT_ID` must be replaced on each line) `docker tag 106524781139.dkr.ecr.us-gov-west-1.amazonaws.com/url-defense:latest YOUR_GOV_CLOUD_ACCOUNT_ID.dkr.ecr.us-gov-west-1.amazonaws.com/url-defense:latest docker tag 106524781139.dkr.ecr.us-gov-west-1.amazonaws.com/render-email-html-lambda:latest YOUR_GOV_CLOUD_ACCOUNT_ID.dkr.ecr.us-gov-west-1.amazonaws.com/render-email-html-lambda:latest` 3. Push docker images to your account `docker push YOUR_GOV_CLOUD_ACCOUNT_ID.dkr.ecr.us-gov-west-1.amazonaws.com/url-defense:latest docker push YOUR_GOV_CLOUD_ACCOUNT_ID.dkr.ecr.us-gov-west-1.amazonaws.com/render-email-html-lambda:latest` _Note: We may ask you to repeat the "Set Repository Contents" step in the future in order to update these services._ Once you’ve completed these steps, you can kick off the AWS CF setup steps described [here](https://docs.sublimesecurity.com/docs/aws-cloudformation#setup) ! Please note that you **must** complete the above steps before moving forward with your AWS CF deployment in order to avoid errors/failures. Updated 9 months ago * * * * [AWS CloudFormation](https://docs.sublime.security/docs/aws-cloudformation) --- # Add your abuse mailbox Introduction [](https://docs.sublime.security/docs/add-your-abuse-mailbox#introduction) =========================================================================================== Configuring your team's abuse mailbox in Sublime allows you to remediate reported phish faster and automate the busywork of abuse mailboxes, including retrieving the original message reported by the user and taking action like trashing messages from real attacks. Sublime automatically groups messages in the same attack, so you can quickly remediate all messages in an attack based on a single user report. > 📘 > -- > > For clarity, throughout this documentation we call the suspected malicious message forwarded by the user the **_reported message_** and the message the abuse mailbox receives when a user sends the reported message the **_reporting message_**, which contains the reported message in the body or as an attachment. Configuration [](https://docs.sublime.security/docs/add-your-abuse-mailbox#configuration) ============================================================================================= Your abuse mailbox can be an actual user mailbox or a mailing list, such as a Microsoft 365 contact group (also known as a "distribution list") or a Google Group. To configure your abuse mailbox, go to **Admin** > **Account** in the Sublime interface, enter your abuse mailbox address in the **Abuse mailbox** section, and click **Save**. (You can configure up to 5 separate abuse mailboxes in Sublime by adding them on their own line) _If you use an actual user mailbox_ as your abuse mailbox, you just need to confirm that the mailbox is listed as Active in your instance. To confirm, head to Admin > Mailboxes in the Sublime interface, search for your abuse mailbox address, and check for a green dot in the Status column. _If you use a mailing list_ as your abuse mailbox, you need to configure a few details: * At least 1 actual mailbox must subscribe to the mailing list. * The subscribed actual mailbox must receive all emails sent to the mailing list. * The subscribed actual mailbox must be listed as Active in your instance. To confirm, head to Admin > Mailboxes in the Sublime interface, search for your abuse mailbox address, and check for a green dot in the Status column. How Sublime processes abuse mailbox messages [](https://docs.sublime.security/docs/add-your-abuse-mailbox#how-sublime-processes-abuse-mailbox-messages) =========================================================================================================================================================== When Sublime processes a reporting message sent to the abuse mailbox, it fetches the reported message by either: 1. Extracting the `In-Reply-To` header from the reporting message and using that identifier to look up the reported message. 2. If the reporting message does not include an `In-Reply-To` header (usually meaning it wasn't sent using an email client's forwarding feature), or if Sublime cannot find a message matching the `In-Reply-To` header, Sublime will look for one or more attachments to the reporting message with an `.eml` extension and use the `Message-ID` header of each such attachment to look up the reported message or messages. 3. If neither `In-Reply-To` or a valid EML attachment is present, the last value from `References` header is used. The identifier is again used to look up the reported message. For security reasons, user reports must be recognized as [internal](https://docs.sublime.security/docs/message-types) messages by default. If you have an SPF, DKIM, or DMARC misconfiguration where messages sent by your users fail sender authentication, you can permit unauthenticated user reports from your users in your Account settings. > 🚧 > > Only messages received by active Sublime mailboxes can be reported > > > ------------------------------------------------------------------------ > > Today, user reported messages must have first been ingested in Sublime in order to be processed as a User Report. This means that only messages from Sublime mailboxes can be reported. > > Support for non-Sublime mailbox user report ingestion is on the roadmap and will be supported in the near future. If you'd like support for this, [drop us a message](https://docs.sublime.security/cdn-cgi/l/email-protection#60131510100f1214201315020c090d054e1305031512091419) > and we'll let you know once it's released. Processing of the reporting message [](https://docs.sublime.security/docs/add-your-abuse-mailbox#processing-of-the-reporting-message) ----------------------------------------------------------------------------------------------------------------------------------------- In most cases, Sublime will store the reporting message, including its [MDM](https://docs.sublime.security/docs/mdm) , so that it can be searched for and inspected, but Sublime **will not** run any rules on the reporting message, both so that you don't end up with additional flagged messages that are redundant with the reported message and so that the reporting message is not modified or removed from either the abuse mailbox or subscribers' mailboxes (typically members of the security team), which can interfere with investigations. > 📘 > -- > > The exception, however, is when an external sender sends a message to the abuse mailbox. Sublime will still run rules on such messages to ensure attackers can't bypass Sublime rules by including the abuse mailbox as a recipient. Viewing user reports [](https://docs.sublime.security/docs/add-your-abuse-mailbox#viewing-user-reports) =========================================================================================================== To view user reports in the Sublime interface, go to **Messages** > **User Reported**. This message list view will show all unreviewed reported message groups, regardless of whether the messages were flagged by any rules. You can modify any message filters from this view. ![](https://files.readme.io/0646b56-Screen_Shot_2021-12-13_at_9.06.18_PM.png "Screen Shot 2021-12-13 at 9.06.18 PM.png") Updated about 2 months ago * * * * [Automations](https://docs.sublime.security/docs/automations) --- # Add a Microsoft 365 message source Overview [](https://docs.sublime.security/docs/add-an-office-365-message-source#overview) ============================================================================================= > 📘 > > Microsoft 365 Global Admin Required > > > ----------------------------------------- > > You must be a Global Admin in Microsoft 365 to authorize access for all users on your domain. You can add a Microsoft 365 message source when setting up your Sublime organization or by going to `Admin` > `Message Sources` > `New message source` in the Sublime dashboard and selecting `Microsoft 365`. Once you select the option to add a Microsoft 365 message source, the next steps will depend on whether you're using: * [Sublime Cloud](https://docs.sublime.security/docs/sublime-cloud-fully-managed-m365) * [A self managed Sublime instance](https://docs.sublime.security/docs/self-managed-m365) Updated 7 months ago * * * --- # Add a message source Connect your Sublime Platform instance to one or more message sources: [Add a Microsoft 365 message source](https://docs.sublime.security/docs/add-an-office-365-message-source) [Add a Google Workspace message source](https://docs.sublime.security/docs/add-a-google-workspace-message-source) [Add an IMAP message source](https://docs.sublime.security/docs/add-an-imap-message-source) Updated almost 3 years ago * * * Add a message source * [Microsoft 365](https://docs.sublime.security/docs/add-an-office-365-message-source) * [IMAP](https://docs.sublime.security/docs/add-an-imap-message-source) * [Google Workspace](https://docs.sublimesecurity.com/docs/add-a-google-workspace-message-source) --- # Add a Google Workspace message source Overview [](https://docs.sublime.security/docs/add-a-google-workspace-message-source#overview) ================================================================================================== > 📘 > > Google Workspace Super Administrator Required > > > --------------------------------------------------- > > To add a Google Workspace message source, you need to be a [super administrator](https://support.google.com/a/answer/2405986#super_admin) > for your Google Workspace organization or get the help of a super administrator. You can add a Google Workspace message source when setting up your Sublime organization or by going to `Admin` > `Message Sources` > `New message source` in the Sublime dashboard and selecting `Google Workspace`. Once you select the option to add a Google Workspace message source, the next steps will depend on whether you're using: * [Sublime Cloud](https://docs.sublime.security/docs/sublime-cloud-fully-managed-gw) * [A self managed Sublime instance](https://docs.sublime.security/docs/self-managed-gw) Updated 7 months ago * * * --- # Sublime Cloud (Fully Managed) - Microsoft 365 > 📘 > > Microsoft 365 Global Admin Required > > > ----------------------------------------- > > You must be a Global Admin in Microsoft 365 to authorize access for all users on your domain. You can add a Microsoft 365 message source when setting up your Sublime organization or by going to `Admin` > `Message Sources` > `New message source` in the Sublime dashboard and selecting `Microsoft 365`. Overview [](https://docs.sublime.security/docs/sublime-cloud-fully-managed-m365#overview) --------------------------------------------------------------------------------------------- If you're using [Sublime Cloud](https://platform.sublime.security/) , simply select "Microsoft 365" and follow the prompts to authorize access on behalf of your organization. ![](https://files.readme.io/0903d69a87b5ebe2cd269fe9ce1f31e086cbc530dc16f0173a03d4a52c47ba8e-253fd63-m365_add_message_source.png) Updated 5 months ago * * * * [Microsoft 365's Native “Report phishing” feature](https://docs.sublime.security/docs/how-to-integrate-microsoft365-report-phishing-with-sublime) --- # Sublime Managed Creating a Sublime Managed deployment only takes a few minutes. Available regions include the US, EU (Dublin, London), Canada (Montréal), and Australia (Sydney). [Create an account here!](https://platform.sublime.security/nux) Looking to check out Sublime without setup? Analyze email messages using [the free unauthenticated Analyzer app](https://analyzer.sublime.security/) or [API](https://docs.sublime.security/reference/analysis-api-introduction) . Updated 6 months ago * * * --- # Sublime Cloud (Fully Managed) - Google Workspace > 📘 > > Google Workspace Super Administrator Required > > > --------------------------------------------------- > > To add a Google Workspace message source, you need to be a [super administrator](https://support.google.com/a/answer/2405986#super_admin) > for your Google Workspace organization or get the help of a super administrator. You can add a Google Workspace message source when setting up your Sublime organization or by going to `Admin` > `Message Sources` > `New message source` in the Sublime dashboard and selecting `Google Workspace`. Overview [](https://docs.sublime.security/docs/sublime-cloud-fully-managed-gw#overview) ------------------------------------------------------------------------------------------- If you're using [Sublime Cloud](https://platform.sublime.security/) , simply select "Google Workspace" and follow the prompts to authorize access on behalf of your organization. The Sublime App for the Google Workspace Marketplace is private so you can't find it by searching the app directory. [Follow this link to the Sublime Cloud Platform Listing](https://workspace.google.com/marketplace/app/sublime_cloud_platform/421484249706) . ![](https://files.readme.io/52d1795-gw_add_message_source.png) Updated 5 months ago * * * * [Add your abuse mailbox](https://docs.sublime.security/docs/add-your-abuse-mailbox) --- # Azure via ARM Overview [](https://docs.sublime.security/docs/azure-via-arm#overview) ========================================================================== Welcome to the Sublime Platform self-managed Azure deployment! This deployment leverages Azure Resource Manager (ARM) to set up the Sublime Platform in your own VPC without sending or storing email data externally. > 🚧 > > Use a dedicated Azure Subscription > > > ---------------------------------------- > > To limit monitoring visibility to just the Sublime deployment, It is recommended to Install this Azure deployment in it's own Azure Subscription > 🚧 > > Azure quota for NVIDIA GPUs is required to deploy and operate Sublime in Azure > > > ------------------------------------------------------------------------------------ > > Sublime recommends the `standard_NC4as_T4_v3` SKU. However availability for the recommended SKU is limited based on Azure Region. If the recommended SKU is not available, customers can also use `standard_NV6ads_A10_v5` Setup [](https://docs.sublime.security/docs/azure-via-arm#setup) ==================================================================== Enabling Monitoring [](https://docs.sublime.security/docs/azure-via-arm#enabling-monitoring) ------------------------------------------------------------------------------------------------ Monitoring allows Sublime to see metrics and other high level telemetry about your deployment and create alerts, monitor via dashboards, and improve the Azure experience. An example of these metrics are DB CPU, pods in the AKS cluster, etc. Logs and more granular data can be collected via the “Send errors and system health information” setting within the platform. Monitoring is an optional step, but it must be enabled to receive support from Sublime. 1. Create a new app registration 1. Select “Microsoft Entra ID” 2. Expand “Manage” and select “App registrations” 3. Click “New registration” 4. Name the application whatever you would like and select “Accounts in this organizational directory only” for “Supported account types”. Leave “Redirect URI” empty. 5. Click “Register” 6. Copy the “Application (client) ID”, you will need it when running the ARM template 2. Create a client secret 1. Navigate to the App registration resource 2. Expand “Manage” and select “Certificates & secrets” 3. Click “Client secrets” 4. Click “New client secret” 5. Give the secret a name and expiration of your choice 6. Click “Add” 7. Copy the “Value” and save it somewhere, you will not be able to retrieve it later and you will need it when running the ARM template 3. Grant permissions to the application 1. Navigate to the subscription you’re using for Sublime 2. Select “Access control (IAM)” 3. Select “Role assignments” 4. Click “Add” and “Add role assignment” 5. Select the “Monitoring Reader” role 6. Click “Members” 7. Select the “User, group, or service principal” radial and click “Select members” 8. Enter the name of the application you created and click “Select” 9. Click “Review + assign” Deploy the template [](https://docs.sublime.security/docs/azure-via-arm#deploy-the-template) ------------------------------------------------------------------------------------------------ 1. Click the following link to open the template and fill out the information. [![](https://aka.ms/deploytoazurebutton)](https://portal.azure.com/#create/Microsoft.Template/uri/https%3A%2F%2Fsublime-security.github.io%2Fazure-platform%2Fsublime-platform.json/createUIDefinitionUri/https%3A%2F%2Fsublime-security.github.io%2Fazure-platform%2Fsublime-platform.ui.json) Verify & complete the installation [](https://docs.sublime.security/docs/azure-via-arm#verify--complete-the-installation) ----------------------------------------------------------------------------------------------------------------------------- 1. To check the installation status go to the Kubernetes resource in the Azure portal, expand “Kubernetes resources”, and click “Run command”. You can run the following command to check installation status (it may take a bit for the release to show up at first): `kubectl get helmrelease -n flux-system`. Once the installation is complete, “Ready” will be “True” and “Status” will be “Release reconciliation succeeded”. 2. Create a DNS record to point to your Sublime dashboard. This should simply be an A record pointing to our Ingress’s IP, which you can find either of two ways: 1. Using the “Run command” page above run `kubectl get ingress -n sublime`. The external IP will be in the “Address” column. 2. Using the Azure portal you can view the “kubernetes” Load Balancer resource, expand “Settings”, select “Frontend IP configuration”, and the configuration with 3 rules will be the external IP. Upload your TLS Certificate [](https://docs.sublime.security/docs/azure-via-arm#upload-your-tls-certificate) ---------------------------------------------------------------------------------------------------------------- 1. Go to the key vault that was created after running the ARM template (it’ll be named `sublime-vault-`) 2. You may have to grant yourself access to the key vault. 1. Click “Access control (IAM)”, “Add”, and then “Add role assignment” 2. Grant your Principal the “Key Vault Certificates Officer” role 3. Click “Review + assign” 3. Expand “Objects” and click “Certificates” 4. Click “Generate/Import” 5. Select “Import” for “Method of Certificate Creation” and then upload your certificate file. 1. If you do not have a preferred Certificate Authority, you can generate one in Azure and upload it here. To do so expand the steps below. * Generating a Certificate in Azure 1. Open “App Service Certificates” in Azure and click “Create”. 2. Set the resource group to the resource group used previously & leave the SKU as is. 3. Enter the DNS name you’ll access Sublime through (e.g. `sublime.my-org.com`), you’ll need the ability to add DNS records for this domain. 4. Enter any name and click “Review and Create” then “Create”. 5. Wait for the “Deployment in Progress” step to complete, then select “Go to resource”. 6. Ignore any errors and select “Configure required Key Vault store”. 7. Click the checkbox next to “Store”, follow “Select from Key Vault”, and select the newly created Sublime key vault, and finish with “Select”. 8. Go back and click the checkbox next to “Verify” and follow Azure’s instructions for validating your certificate. 9. Once your certificate is validated, you can proceed. 10. Now go into the sublime Key Vault and open Objects > Secrets. 11. Select the certificate previously saved into Secrets — it’ll have a Type of “application/x-pkcs12”. 12. Select the ID listed under “Current Version” and then click “Download as Certificate” 13. You may now use this downloaded certificate for the “Import” step (yes this is very convoluted, we’re sorry there’s not a better way in Azure). 6. The certificate name must be `ingress`. 7. Click “Create” Updated 5 months ago * * * * [Add a message source](https://docs.sublime.security/docs/add-message-source) --- # Enable user-reported phishing Sublime can automatically track the messages your users report as phishing through multiple reporting mechanisms. Updated 6 months ago * * * * [Add your abuse mailbox](https://docs.sublime.security/docs/add-your-abuse-mailbox) * [Gmail's "Report phishing" feature](https://docs.sublime.security/docs/gmails-report-phishing-feature) * [Microsoft 365's Native “Report phishing” feature](https://docs.sublime.security/docs/how-to-integrate-microsoft365-report-phishing-with-sublime) --- # Self Managed - Microsoft 365 > 📘 > > Microsoft 365 Global Admin Required > > > ----------------------------------------- > > You must be a Global Admin in Microsoft 365 to authorize access for all users on your domain. You can add a Microsoft 365 message source when setting up your Sublime organization or by going to `Admin` > `Message Sources` > `New message source` in the Sublime dashboard and selecting `Microsoft 365`. Overview [](https://docs.sublime.security/docs/self-managed-m365#overview) ------------------------------------------------------------------------------ In order for your self-hosted instance of Sublime to access mail from your tenant programmatically, you must create a new Azure app and authorize it for access. This is required so that only you have access to your mail, and not anyone else, including the Sublime team. After completing the steps below, you'll copy over the following to add the message source in Sublime: * Application (client) ID * Directory (tenant) ID * Client secret ### Steps [](https://docs.sublime.security/docs/self-managed-m365#steps) 1. Login to [https://portal.azure.com](https://portal.azure.com/) 2. Search for `App registrations` ![1586](https://files.readme.io/079d651-o1.png "o1.png") 3. Click `+ New registration` ![1240](https://files.readme.io/496f7cc-o2.png "o2.png") 4. Enter "Sublime Platform" for the app name. Leave the other defaults (Single-tenant, no Redirect URI). Click "Register". ![1686](https://files.readme.io/1ba2f31-Screen_Shot_2021-03-31_at_12.40.03_PM.png "Screen Shot 2021-03-31 at 12.40.03 PM.png") 5. Copy the application (client) ID and the Directory (tenant) ID over to the Sublime Platform setup ![1348](https://files.readme.io/ec259b1-Screen_Shot_2021-03-31_at_3.51.11_PM.png "Screen Shot 2021-03-31 at 3.51.11 PM.png") 6. Click `View API permissions` and then `+ Add a permission` * Select `Microsoft Graph` * Select `Application permissions` * Choose `Mail.ReadWrite`, `User.Read.All`, `Group.Read.All`, and `Domain.Read.All` * Click `Add permissions` 7. Then, `Grant admin consent` for your domain. Note that if this button is greyed out for you, you may need to have a Domain Admin grant the consent. ![1660](https://files.readme.io/d0cb9b9-Screenshot_2023-09-11_at_1.38.20_PM.png "Screen Shot 2021-05-12 at 6.38.17 PM.png") **You should see green checkmarks like this:** ![2050](https://files.readme.io/7882203-Screenshot_2023-09-11_at_1.38.30_PM.png "Screen Shot 2021-05-12 at 6.37.47 PM.png") 8. Navigate to `Certificates & secrets` in the sidebar ![1530](https://files.readme.io/781219c-Screen_Shot_2021-05-12_at_6.37.31_PM.png "Screen Shot 2021-05-12 at 6.37.31 PM.png") 9. Create a new Client secret. Enter "Sublime Platform" for the description and choose "24 months". ![2532](https://files.readme.io/4aa14dc-Screen_Shot_2021-04-15_at_9.47.49_AM.png "Screen Shot 2021-04-15 at 9.47.49 AM.png") 10. **Copy the client secret "Value" over to the Sublime Platform setup.** 🎉 That's it! Safe Attachments Recommended Policy Settings [](https://docs.sublime.security/docs/self-managed-m365#safe-attachments-recommended-policy-settings) ====================================================================================================================================================== If you have [Safe Attachments](https://learn.microsoft.com/en-us/microsoft-365/security/office-365-security/safe-attachments-about) enabled, the _Dynamic Delivery_ unknown malware response setting will cause issues with scanning in Sublime. Specifically, Sublime currently does not scan attachments that are placed back into a message post-delivery. Any of the other settings will work with Sublime (Block, Monitor, or None). In order to get the best of Microsoft and the best of Sublime together, **our recommendation is Block** (also Microsoft's recommended and default setting). ![](https://files.readme.io/c16d603-image.png) Authorizing a Subset of Mailboxes [](https://docs.sublime.security/docs/self-managed-m365#authorizing-a-subset-of-mailboxes) ================================================================================================================================ If you wish to restrict the access of the Sublime Azure app to a subset of mailboxes while testing out the Platform, you can do so using the [_New-ApplicationAccessPolicy_ cmdlet in PowerShell](https://learn.microsoft.com/en-us/powershell/module/exchange/new-applicationaccesspolicy?view=exchange-ps) . To do so, you'll need to first install the [Exchange Online PowerShell module](https://learn.microsoft.com/en-us/powershell/exchange/exchange-online-powershell-v2?view=exchange-ps#install-and-maintain-the-exchange-online-powershell-module) if you don't already have it. This can be done by copying and pasting the command found on [Microsoft's PowerShell Gallery listing](https://www.powershellgallery.com/packages/ExchangeOnlineManagement/) for the module (the link can also be found on the corresponding docs page). Now, in PowerShell, the module needs to be imported and the tenant authenticated. First, run: PowerShell `Import-Module ExchangeOnlineManagement` Then, replacing _{EMAIL\_OR\_UPN}_ with your applicable email address or UserPrincipalName: PowerShell `Connect-ExchangeOnline -UserPrincipalName {EMAIL_OR_UPN}` This should open a prompt in a web browser to complete authentication. Once successfully authenticated, we can now create a new _ApplicationAccessPolicy_ to restrict access of the Azure app to specific mailboxes. The below command creates an application access policy to restrict access to the user or group email specified. Be sure to replace `{APP_ID}` with the value for `Application (client ID)` for the Azure app that was created as part of the setup above. Replace `{USER_OR_GROUP_EMAIL}`with the user or group you'd like to _allow_ access for. PowerShell `New-ApplicationAccessPolicy -AppId {APP_ID} -PolicyScopeGroupId {USER_OR_GROUP_EMAIL} -AccessRight RestrictAccess -Description "Restrict access to Sublime app"` > 📘 > > Allowing Multiple Mailboxes or Groups > > > ------------------------------------------- > > To authorize multiple different mailboxes or groups, repeat the above command once for each mailbox or group, replacing the value of `{USER_OR_GROUP_EMAIL}` with each email address. All accounts with an ApplicationAccessPolicy created with the above AccessRight will be accessible from the Sublime Platform. Updated about 1 month ago * * * * [Microsoft 365's Native “Report phishing” feature](https://docs.sublime.security/docs/how-to-integrate-microsoft365-report-phishing-with-sublime) --- # Metrics Collection in Self-Hosted Deployments ### Overview [](https://docs.sublime.security/docs/data-collection-in-self-hosted-deployments#overview) Sublime uses [DataDog](https://www.datadoghq.com/) for application performance monitoring, system health monitoring, and operational insights. The DataDog agent is embedded within Sublime's Docker images and can be dynamically enabled or disabled by customers through their Sublime dashboard. The information extracted from your Sublime instance is built with privacy and compliance considerations top of mind. This information is critical to Sublime's ability to provide proactive and reactive support for your deployment. Additionally, we use this diagnostic information to plan for future capacity needs as well as tune your instance to maximize message processing performance. ### Customer Control [](https://docs.sublime.security/docs/data-collection-in-self-hosted-deployments#customer-control) Customers can control DataDog telemetry through their Sublime Dashboard: * **"Admin" > "Account" > "Usage data" > "Send errors and system health information"** ![](https://files.readme.io/2b394efd43eaae13b43c924dec00b995295a15862f3b5d6c1dc69c3d857ed007-image.png) * When disabled, no telemetry data is sent to DataDog * The DataDog agent is only activated when customers explicitly enable monitoring ### Types of Data Collected [](https://docs.sublime.security/docs/data-collection-in-self-hosted-deployments#types-of-data-collected) #### 1\. **Application Logs** [](https://docs.sublime.security/docs/data-collection-in-self-hosted-deployments#1-application-logs) * **Service logs**: Application logs from core services (mantis, bora, nugget, inline\_monitor) * **DataDog agent logs**: Internal agent logging for debugging * **Format**: Structured JSON logs with timestamps and severity levels #### 2\. **Application Performance Monitoring (APM)** [](https://docs.sublime.security/docs/data-collection-in-self-hosted-deployments#2-application-performance-monitoring-apm) * **Traces**: HTTP request traces, task execution spans, database queries * **Service performance**: Response times, error rates, throughput * **Request traces**: API endpoint performance and dependencies * **Background job execution**: Task processing times and success rates * **Database interactions**: Query performance and connection metrics #### 3\. **Custom Business Metrics** [](https://docs.sublime.security/docs/data-collection-in-self-hosted-deployments#3-custom-business-metrics) The application sends various operational metrics including, but not necessarily limited to: **Message Processing Metrics:** * Message processing latency and duration * Email analysis timing (spam detection, content analysis) * Queue processing metrics * SMTP session metrics (connections, data size, success/failure rates) **System Health Metrics:** * Heartbeat metrics for service availability * Cache hit/miss ratios and performance * Active mailbox counts * Processing failure rates * Memory and CPU usage patterns **Security and Detection Metrics:** * Attack score verdicts * Authentication events * Rule execution performance and results #### 4\. **Infrastructure Metrics** [](https://docs.sublime.security/docs/data-collection-in-self-hosted-deployments#4-infrastructure-metrics) * **Container metrics**: Docker container resource usage * **Host metrics**: CPU, memory, disk, network usage * **Process metrics**: Running processes and resource consumption * **System performance**: Load averages, file system usage #### 5\. **Application Profiling Data** [](https://docs.sublime.security/docs/data-collection-in-self-hosted-deployments#5-application-profiling-data) * **CPU profiles**: Application CPU usage patterns * **Memory profiles**: Heap usage and allocation patterns * **Goroutine profiles**: Concurrency and thread usage * **Block profiles**: Lock contention and blocking operations * **Metrics profiles**: Runtime metrics collection #### 6\. **Events and Alerts** [](https://docs.sublime.security/docs/data-collection-in-self-hosted-deployments#6-events-and-alerts) * **Startup/shutdown events**: Service lifecycle events * **Error events**: Application errors and exceptions with stack traces * **Configuration changes**: System configuration updates * **Deployment events**: Version updates and releases ### Data Enrichment and Tagging [](https://docs.sublime.security/docs/data-collection-in-self-hosted-deployments#data-enrichment-and-tagging) All telemetry data is tagged with: * **Environment information**: staging, production, development * **Service identification**: specific Sublime service names * **Version information**: Release version and Git commit SHA * **Infrastructure details**: * Cloud provider (AWS, Azure) * Region and availability zone * Container and host identifiers * **Organization context**: Customer organization IDs (hashed/anonymized) ### Data Transmission and Security [](https://docs.sublime.security/docs/data-collection-in-self-hosted-deployments#data-transmission-and-security) #### Transmission Method [](https://docs.sublime.security/docs/data-collection-in-self-hosted-deployments#transmission-method) * **Protocol**: HTTPS to DataDog intake endpoints * **Compression**: Logs and metrics are compressed before transmission * **Batching**: Data is batched for efficient transmission * **Authentication**: Uses DataDog API keys managed by Sublime #### Security Measures [](https://docs.sublime.security/docs/data-collection-in-self-hosted-deployments#security-measures) * **TLS encryption**: All data transmitted over TLS 1.2+ * **Access controls**: DataDog API keys are securely managed ### Data Processing and Retention [](https://docs.sublime.security/docs/data-collection-in-self-hosted-deployments#data-processing-and-retention) #### DataDog Processing [](https://docs.sublime.security/docs/data-collection-in-self-hosted-deployments#datadog-processing) * **Log processing**: Automatic parsing and indexing of log data * **Metric aggregation**: Statistical aggregation over time windows * **Trace sampling**: Intelligent sampling to manage data volume * **Alert generation**: Automated alerts based on metric thresholds #### Data Retention [](https://docs.sublime.security/docs/data-collection-in-self-hosted-deployments#data-retention) * **Logs**: Logs are retained for a year. * **Metrics**: Long-term retention with data aggregation * **Traces**: Short-term retention (typically 7-15 days) * **Events**: Medium-term retention for audit purposes ### Data Exclusions and Privacy [](https://docs.sublime.security/docs/data-collection-in-self-hosted-deployments#data-exclusions-and-privacy) #### What is NOT Collected [](https://docs.sublime.security/docs/data-collection-in-self-hosted-deployments#what-is-not-collected) * **Email content**: Actual email message bodies and email metadata are not sent to DataDog * **User credentials**: Passwords, API keys, and authentication tokens * **Personal data**: Customer email addresses or personal information * **File contents**: Attachment contents or file data * **Encryption keys**: Security certificates and private keys #### Data Sanitization [](https://docs.sublime.security/docs/data-collection-in-self-hosted-deployments#data-sanitization) * Automatic removal of sensitive patterns from logs * Sanitization of emails and JWTs * IP address masking and anonymization * URL parameter sanitization * Database query parameter scrubbing * We regularly audit code to ensure fields do not contain PII and they are not logged or added to traces or spans ### Azure ARM Deployment Considerations [](https://docs.sublime.security/docs/data-collection-in-self-hosted-deployments#azure-arm-deployment-considerations) For Azure ARM deployments specifically: * **Azure metadata collection**: Subscription ID, resource group, region information * **Identity management**: We only use Azure-managed identities Updated about 1 month ago * * * --- # Add an IMAP message source Gmail [](https://docs.sublime.security/docs/add-an-imap-message-source#gmail) ================================================================================= To add an IMAP message source for your Gmail or Google Workspace account, you need to: 1. Be sure IMAP is enabled for your account. 2. Create an app password. > 📘 > > 2-Step Verification Required > > > ---------------------------------- > > You must have 2-Step Verification enabled on your Google account in order to create an _app password_ for the IMAP integration. > > To enable 2-Step Verification, see [Turn on 2-Step Verification](https://support.google.com/accounts/answer/185839) > . > 📘 > > Advanced Protection > > > ------------------------- > > App passwords are not supported if Google Advanced Protection is enabled, and menu option will be missing to generate. To learn more visit: [https://support.google.com/accounts/answer/185833?hl=en](https://support.google.com/accounts/answer/185833?hl=en) Enabling IMAP for your account [](https://docs.sublime.security/docs/add-an-imap-message-source#enabling-imap-for-your-account) ----------------------------------------------------------------------------------------------------------------------------------- 1. Sign into your Gmail account 2. Click Settings on the top right, then "See all settings" 3. Click on the "Forwarding and POP/IMAP" sub-menu, then "Enable IMAP" 4. Save changes ![Gif showing Gmail's "Forwarding and POP/IMAP" settings where the IMAP setting is switched from "disabled" to "enabled". ](https://files.readme.io/3d17c26-imap_settings.gif) Creating an app password [](https://docs.sublime.security/docs/add-an-imap-message-source#creating-an-app-password) ----------------------------------------------------------------------------------------------------------------------- 1. Head to your Gmail account's [app passwords](https://myaccount.google.com/apppasswords) page. 1. You can alternatively go to the ["Manage your Google Account" section](https://docs.sublime.security/docs/add-an-imap-message-source) and utilize the Search feature to look up "App passwords". 2. To create a new app password, enter "Sublime Platform" in the app name field and click the "Create" button. ![Gif showing the app passwords section of the Gmail settings. "Sublime Platform" app password is created.](https://files.readme.io/904b11e-app_password.gif "Screen Shot 2021-10-02 at 12.11.11 PM.png") 4. Once you've created an app password, enter it into the **IMAP / App Password** field in Sublime when creating the IMAP message source. Microsoft 365 [](https://docs.sublime.security/docs/add-an-imap-message-source#microsoft-365) ================================================================================================= To add an IMAP message source for your Microsoft 365 account, you need to: 1. Be sure IMAP is enabled for your account. 2. Create an app password, if and only if you're using [two-step verification](https://support.microsoft.com/en-us/account-billing/how-to-use-two-step-verification-with-your-microsoft-account-c7910146-672f-01e9-50a0-93b4585e7eb4) with your Microsoft 365 account. Enabling IMAP for your account [](https://docs.sublime.security/docs/add-an-imap-message-source#enabling-imap-for-your-account-1) ------------------------------------------------------------------------------------------------------------------------------------- > 📘 > > Microsoft 365 administrator required > > > ------------------------------------------ > > You'll need to be a Microsoft 365 administrator to follow these steps. If you're not a Microsoft 365 administrator, we recommend skipping this step, attempting to add the message source, and reaching out to your Microsoft 365 administrator for assistance if you receive a signin error when attempting to add the message source. 1. Sign into the Microsoft 365 admin center at [https://admin.microsoft.com](https://admin.microsoft.com/) 2. In the lefthand menu, go to **Users** > **Active Users** 3. Search for your user and click the matching row 4. In the righthand drawer that opens up, click the **Mail** tab 5. Under **Email apps**, click **Manage email apps** ![](https://files.readme.io/d24a8ed-mail_tab_annotated.png "mail_tab_annotated.png") 6. Check the box next to **IMAP** and click the **Save changes** button Creating an app password [](https://docs.sublime.security/docs/add-an-imap-message-source#creating-an-app-password-1) ------------------------------------------------------------------------------------------------------------------------- > 📘 > -- > > You only need to create an app password if you're using two-step verification with your Microsoft 365 account. The pre-requisites are as follows: 1. "Manage email apps" is enabled in M365 admin interface 2. "Authenticated SMTP" is enabled for the user 3. In Entra admin center: "Allow users to create app passwords" is enabled under Per-user multifactor authentication settings Creating the Password [](https://docs.sublime.security/docs/add-an-imap-message-source#creating-the-password) ----------------------------------------------------------------------------------------------------------------- 1. Sign in to your Microsoft 365 account 2. Go to Security Info settings 3. Select "Add sign-in method" 4. Choose "App password" 5. Name your app password (e.g., "Sublime Email Alerts") 6. Copy the generated password and enter it in the Sublime SMTP configuration Custom IMAP servers [](https://docs.sublime.security/docs/add-an-imap-message-source#custom-imap-servers) ============================================================================================================= > 📘 > -- > > If your IMAP server’s capabilities include `SPECIAL-USE` as defined by [`RFC 6154`](https://www.rfc-editor.org/rfc/rfc6154.html#section-2) > , the Sublime Platform will assume that these special use folders are properly configured to adhere to the attributes on the mailbox. For example, when the `SPECIAL-USE` capability is enabled, Sublime will treat folders with the `\All` attribute as a virtual mailbox containing all incoming mail, and will not ingest mail from any other folders in that mailbox. Updated 6 months ago * * * --- # Self Managed - Google Workspace > 📘 > > Google Workspace Super Administrator Required > > > --------------------------------------------------- > > To add a Google Workspace message source, you need to be a [super administrator](https://support.google.com/a/answer/2405986#super_admin) > for your Google Workspace organization or get the help of a super administrator. You can add a Google Workspace message source when setting up your Sublime organization or by going to `Admin` > `Message Sources` > `New message source` in the Sublime dashboard and selecting `Google Workspace`. Overview [](https://docs.sublime.security/docs/self-managed-gw#overview) ---------------------------------------------------------------------------- In order for your self-hosted instance of Sublime to access mail for your Google Workspace organization programmatically, you must upload a Google Cloud Platform service account key to which you've granted [domain-wide delegation of authority](https://support.google.com/a/answer/162106) for your Google Workspace organization. This is required so that only you have access to your mail, and not anyone else, including the Sublime team. > 📘 > > Google Cloud Project Creator Permission Required > > > ------------------------------------------------------ > > To create a Google Workspace message source when self-hosting Sublime, you must have the **Project Creator** permission in Google Cloud, or get the help of a colleague with that permission. > > This requirement for self-hosting is in addition to the Google Workspace super administrator requirement, which applies to both Sublime Cloud and self-hosting. ### Creating a service account key [](https://docs.sublime.security/docs/self-managed-gw#creating-a-service-account-key) There are 3 high-level steps to creating a service account key, which are described in detail below: 1. Set up a Google Cloud Platform project 2. Create a service account for the project 3. Grant domain-wide delegation to the service account #### 1\. Set up a Google Cloud Platform project [](https://docs.sublime.security/docs/self-managed-gw#1-set-up-a-google-cloud-platform-project) 1. Go to [https://console.cloud.google.com](https://console.cloud.google.com/) and sign in with your Google Workspace account 2. If you've never used Google Cloud Platform before, agree to the terms ![](https://files.readme.io/5883324-gcp_terms.png "gcp_terms.png") 3. Click the dropdown in the upper left ![](https://files.readme.io/80a61f5-new_gcp_account__annotated.png "new_gcp_account__annotated.png") 4. Click the **New Project** button 5. Add a project name in the **Project name** field (for example, "Sublime Platform") 6. Click the **Create** button and wait a moment for the new project to be created 7. Click **Select Project** in the notification for your new project in the upper right ![](https://files.readme.io/6e7d1f2-new_project__annotated.png "new_project__annotated.png") 8. Type "admin sdk" into the search bar and select **Admin SDK API** in the search results ![](https://files.readme.io/3c19a8f-admin_api_search_results__annotated.png "admin_api_search_results__annotated.png") 9. Click the **Enable** button and wait for the page to reload when the Admin SDK API is enabled 10. Type "gmail" into the search bar and select **Gmail API** in the search results 11. Click the **Enable** button and wait for the page to reload when the Gmail API is enabled 12. Type "cloud pub/sub" into the search bar and select **Cloud Pub/Sub API** in the search results 13. Click the **Enable** button and wait for the page to reload when the Cloud Pub/Sub API is enabled 14. Type "alert center" into the search bar and select **Google Workspace Alert Center API** in the search results 15. Click the **Enable** button and wait for the page to reload when the Google Workspace Alert Center API is enabled 16. Type "oauth" into the search bar and select **OAuth consent screen** in the search results 17. Under **User Type**, select **Internal** 18. Click the **Create** button 19. Under **App information**, provide an app name (for example, "Sublime Platform") 20. In the **User support email** dropdown, select your email address or any other available email address 21. Scroll to the bottom of the interface and under **Developer contact information**, enter your email address 22. Click the **Save and Continue** button. You can ignore the remaining OAuth options. #### 2\. Create a service account [](https://docs.sublime.security/docs/self-managed-gw#2-create-a-service-account) 1. In the Google Cloud Platform interface, type "service accounts" into the search bar and select **Service Accounts** in the search results 2. Click the **Create Service Account** button 3. Add a service account name (for example, "Sublime Platform") and optionally a service account description 4. Click the **Create and Continue** button 5. Under **Grant this service account access to project**, click the dropdown, search for "Pub/Sub Admin" and select the matching role. 6. Click the **Done** button. You can ignore the remaining service account setup options. 7. In the service accounts list, select the service account you just created 8. Near the bottom of the details view, click **Advanced settings** 9. Copy the **Client ID** that appears under **Domain-wide Delegation** and hold on to it for use later in this guide 10. Click the **Keys** tab in the horizontal tabs list 11. Click the **Add Key** dropdown 12. Select **Create new key** 13. Select the **JSON** key type if it's not selected by default 14. Click the **Create** button and hold on to the file that is saved to your computer. **This is the file you'll need when setting up a Google Workspace message source.** #### 3\. Grant domain-wide delegation [](https://docs.sublime.security/docs/self-managed-gw#3-grant-domain-wide-delegation) 1. Sign into the Google Workspace admin interface at [https://admin.google.com](https://admin.google.com/) 2. In the lefthand sidebar, click **Security** > **Access and data control** > **API controls** 3. Scroll down and click **Manage Domain Wide Delegation** under **Domain wide delegation** 4. Click **Add new** 5. Paste the **Client ID** you copied when configuring the service account 6. Paste this value into the **OAuth scopes** field: `https://mail.google.com/,https://www.googleapis.com/auth/admin.directory.user.readonly,https://www.googleapis.com/auth/admin.directory.customer.readonly,https://www.googleapis.com/auth/admin.directory.domain.readonly,https://www.googleapis.com/auth/admin.directory.group.readonly,https://www.googleapis.com/auth/admin.directory.orgunit.readonly,https://www.googleapis.com/auth/admin.reports.audit.readonly,https://www.googleapis.com/auth/apps.alerts` 7. Click the **Authorize** button #### 4\. Turn on Google's "User-reported phishing" rule [](https://docs.sublime.security/docs/self-managed-gw#4-turn-on-googles-user-reported-phishing-rule) These steps ensure Sublime will receive notifications when users report messages through Gmail's "Report phishing" feature, allowing you to [view these reports in Sublime](https://docs.sublime.security/docs/gmails-report-phishing-feature) . 1. Sign into the Google Workspace admin interface at [https://admin.google.com](https://admin.google.com/) 2. Click "Rules" in the sidebar on the left 3. Locate the system-defined "User-reported phishing" rule and click it 4. Mouse over the **Actions** section and click the edit icon that appears in the upper right 5. If it's not checked, check the box next to "Send to alert center" 6. Click **Next: Review** 7. Click **Update Rule** ### Use your new service account key [](https://docs.sublime.security/docs/self-managed-gw#use-your-new-service-account-key) When setting up a Google Workspace message source in the Sublime Dashboard, paste the contents of the service account key file you downloaded into the **Service Account Key JSON** field. Updated 5 months ago * * * * [Add your abuse mailbox](https://docs.sublime.security/docs/add-your-abuse-mailbox) --- # Gmail's "Report phishing" feature Sublime automatically tracks user phishing reports made through Gmail's built-in "Report phishing" feature: ![638](https://files.readme.io/7a04fad-Screen_Shot_2022-02-07_at_10.05.10_PM.png "Screen Shot 2022-02-07 at 10.05.10 PM.png") Sublime will show messages reported through this feature in **Messages** > **User Reported**. ![](https://files.readme.io/cd5db8b-0646b56-Screen_Shot_2021-12-13_at_9.06.18_PM.png "0646b56-Screen_Shot_2021-12-13_at_9.06.18_PM.png") > 🚧 > -- > > There is typically a delay of 20+ minutes between when a user reports a message as phishing through the "Report phishing" feature and when an alert is created in Google Workspace's alert center. This is a Google limitation, and can't be changed according to Google support. Thus, this user reporting mechanism is far from real-time, in contrast to an [abuse mailbox](https://docs.sublime.security/docs/add-your-abuse-mailbox) > . Setup [](https://docs.sublime.security/docs/gmails-report-phishing-feature#setup) ===================================================================================== Once you've finished [adding a Google Workspace message source](https://docs.sublime.security/docs/add-a-google-workspace-message-source) , Sublime will automatically track messages reported through Gmail's "Report phishing" feature. Note that currently, Sublime must have processed the message that was reported in order for it to show in the User Reported page within Sublime. Updated 5 months ago * * * * [Automations](https://docs.sublime.security/docs/automations) --- # Microsoft 365's Native “Report phishing” feature You can configure Sublime's [Abuse Mailbox](https://docs.sublime.security/docs/add-your-abuse-mailbox) to ingest reports from Microsoft 365's native "Report Phishing" button. 1. Navigate to the [User reported settings](https://security.microsoft.com/securitysettings/userSubmission) in Microsoft Defender 2. Select the "Send reported messages to:" dropdown (pictured below) and choose "Microsoft and my reporting mailbox" or "My reporting mailbox only" ![](https://files.readme.io/f779eac-image.png) 3. Enter the Abuse Mailbox that you have configured in Sublime ![](https://files.readme.io/8c6e000-image.png) 4. Click "Save" > 📘 > > Report Message Options > > > ---------------------------- > > Outlook users may have multiple "Report message" buttons, depending on configuration. These include `Report as Phish`, `Report as Junk`, and `Report as Not Junk`. > > The Abuse Mailbox will ignore any message reports for `Junk` and `Not Junk`. > > Only `Phish` reports generated through the native button will be processed as abuse reports. Updated 5 months ago * * * * [Automations](https://docs.sublime.security/docs/automations) --- # MQL Detection Rules Overview [](https://docs.sublime.security/docs/detection-rules#overview) ============================================================================ Message Query Language (MQL) Detection Rules run on live email flow and are used for identifying phishing attacks, data loss prevention (DLP), and policy enforcement. You can view some of the open-source detection rules available for use today in the [Sublime rules Github repo](https://github.com/sublime-security/sublime-rules) . Here is a non-exhaustive list of some of the categories of phishing attacks and techniques that can be detected today: * Executive impersonation * Brand impersonation * Vendor impersonation * Sextortion * Homoglyph and lookalike domains * Gift card scams * Bitcoin scams * Free file hosting services * Free subdomains * Spoofed messages * URL shorteners * Suspicious Office 365 app authorization requests * COVID-19 scams Get started [](https://docs.sublime.security/docs/detection-rules#get-started) ================================================================================== Create your first detection rule by visiting **Rules > Detection Rules** and clicking "New rule". You can also create and share detection rules in the [MQL Playground](https://mql.new/) . Updated 6 months ago * * * --- # AWS CloudFormation Overview [](https://docs.sublime.security/docs/aws-cloudformation#overview) =============================================================================== Welcome to the Sublime Platform self-managed AWS deployment! This deployment leverages AWS CloudFormation to automatically create resources and set up the Sublime Platform in your own AWS account. > 📘 > > GovCloud deployments > > > -------------------------- > > AWS GovCloud deployments have prerequisite steps that are described [here](https://docs.sublime.security/docs/aws-gov-cloud-cloudformation) > ! You must complete these steps before moving forward with the steps listed below. Requirements [](https://docs.sublime.security/docs/aws-cloudformation#requirements) ======================================================================================= 1. Admin permissions A user with ~Admin permissions is required in order to deploy the stack and create the [resources described below](https://docs.sublime.security/docs/aws-cloudformation#resources-deployed) . Resources used are subject to change in the future, which is why we currently recommend Admin permissions (and an isolated AWS account, mentioned below). 2. Dedicated AWS account **We recommend deploying the Platform in an isolated AWS account.** This reduces the risk of errors, such as hitting AWS service resource limits, and overall makes things simpler to manage. Setup [](https://docs.sublime.security/docs/aws-cloudformation#setup) ========================================================================= To deploy the Stack, press one of the following region-specific deploy buttons, and click through the AWS web GUI console. > 🚧 > > Use a dedicated AWS account > > > --------------------------------- > > **We recommend deploying the Platform in an isolated AWS account.** > > This reduces the risk of errors, such as hitting AWS service resource limits, and overall makes things simpler to manage. Deployment can take 40-60 minutes. CF templates are region specific, do not change your region in the console after following a link. `us-east-2` (Ohio) deployment - [![](https://user-images.githubusercontent.com/9153055/116786049-cbf4b780-aa6a-11eb-9072-41e2123be7ab.png)](https://console.aws.amazon.com/cloudformation/home?region=us-east-2#/stacks/quickcreate?stackName=Sublime-Platform&templateURL=https://sublime-platform-latest-us-east-2.s3.us-east-2.amazonaws.com/Sublime-Platform-us-east-2.template.json) `us-west-2` (Oregon) deployment - [![](https://user-images.githubusercontent.com/9153055/116786049-cbf4b780-aa6a-11eb-9072-41e2123be7ab.png)](https://console.aws.amazon.com/cloudformation/home?region=us-west-2#/stacks/quickcreate?stackName=Sublime-Platform&templateURL=https://sublime-platform-latest-us-west-2.s3.us-west-2.amazonaws.com/Sublime-Platform-us-west-2.template.json) `eu-west-1` (Ireland) deployment - [![](https://user-images.githubusercontent.com/9153055/116786049-cbf4b780-aa6a-11eb-9072-41e2123be7ab.png)](https://console.aws.amazon.com/cloudformation/home?region=eu-west-1#/stacks/quickcreate?stackName=Sublime-Platform&templateURL=https://sublime-platform-latest-eu-west-1.s3.eu-west-1.amazonaws.com/Sublime-Platform-eu-west-1.template.json) `eu-west-2` (London) deployment - [![](https://user-images.githubusercontent.com/9153055/116786049-cbf4b780-aa6a-11eb-9072-41e2123be7ab.png)](https://console.aws.amazon.com/cloudformation/home?region=eu-west-2#/stacks/quickcreate?stackName=Sublime-Platform&templateURL=https://sublime-platform-latest-eu-west-2.s3.eu-west-2.amazonaws.com/Sublime-Platform-eu-west-2.template.json) `eu-central-2` (Zurich) deployment - [![](https://user-images.githubusercontent.com/9153055/116786049-cbf4b780-aa6a-11eb-9072-41e2123be7ab.png)](https://console.aws.amazon.com/cloudformation/home?region=eu-central-2#/stacks/quickcreate?stackName=Sublime-Platform&templateURL=https://sublime-platform-latest-eu-central-2.s3.eu-central-2.amazonaws.com/Sublime-Platform-eu-central-2.template.json) `eu-north-1` (Stockholm) deployment - [![](https://user-images.githubusercontent.com/9153055/116786049-cbf4b780-aa6a-11eb-9072-41e2123be7ab.png)](https://console.aws.amazon.com/cloudformation/home?region=eu-north-1#/stacks/quickcreate?stackName=Sublime-Platform&templateURL=https://sublime-platform-latest-eu-north-1.s3.eu-north-1.amazonaws.com/Sublime-Platform-eu-north-1.template.json) `ap-southeast-2` (Sydney) deployment - [![](https://user-images.githubusercontent.com/9153055/116786049-cbf4b780-aa6a-11eb-9072-41e2123be7ab.png)](https://console.aws.amazon.com/cloudformation/home?region=ap-southeast-2#/stacks/quickcreate?stackName=Sublime-Platform&templateURL=https://sublime-platform-latest-ap-southeast-2.s3.ap-southeast-2.amazonaws.com/Sublime-Platform-ap-southeast-2.template.json) `us-gov-west-1` (US GovCloud West, requires GovCloud account) deployment - [![](https://user-images.githubusercontent.com/9153055/116786049-cbf4b780-aa6a-11eb-9072-41e2123be7ab.png)](https://console.amazonaws-us-gov.com/cloudformation/home?region=us-gov-west-1#/stacks/quickcreate?stackName=Sublime-Platform&templateURL=https://sublime-platform-latest-us-gov-west-1.s3.us-gov-west-1.amazonaws.com/Sublime-Platform-us-gov-west-1.template.json) `us-east-1` (N. Virginia, not recommended, please reach out to Sublime for details) deployment - [![](https://user-images.githubusercontent.com/9153055/116786049-cbf4b780-aa6a-11eb-9072-41e2123be7ab.png)](https://console.aws.amazon.com/cloudformation/home?region=us-east-1#/stacks/quickcreate?stackName=Sublime-Platform&templateURL=https://sublime-platform-latest-us-east-1.s3.us-east-1.amazonaws.com/Sublime-Platform-us-east-1.template.json) `us-west-1` (California, not recommended, please reach out to Sublime for details) deployment - [![](https://user-images.githubusercontent.com/9153055/116786049-cbf4b780-aa6a-11eb-9072-41e2123be7ab.png)](https://console.aws.amazon.com/cloudformation/home?region=us-west-1#/stacks/quickcreate?stackName=Sublime-Platform&templateURL=https://sublime-platform-latest-us-west-1.s3.us-west-1.amazonaws.com/Sublime-Platform-us-west-1.template.json) `ca-central-1` (Canada, not recommended, please reach out to Sublime for details) deployment - [![](https://user-images.githubusercontent.com/9153055/116786049-cbf4b780-aa6a-11eb-9072-41e2123be7ab.png)](https://console.aws.amazon.com/cloudformation/home?region=ca-central-1#/stacks/quickcreate?stackName=Sublime-Platform&templateURL=https://sublime-platform-latest-ca-central-1.s3.ca-central-1.amazonaws.com/Sublime-Platform-ca-central-1.template.json) The default parameters are appropriate for many organizations, but if you have less than 4,000 mailboxes you may change the `dbsizeparam` to `m6gXLarge1TB`. If you have more than 30,000 mailboxes, start with a `dbsizeparam` of `m6g4XLarge4TB` and a `redissizeparam` of `cache.t3.medium`. For organizations over 100,000 mailboxes, start with a `dbsizeparam` of `m6g8XLarge8TB` and a `redissizeparam` of `cache.m5.large`. Leave the two params, "ddapikey" & "ddroleexternalid" empty. These are used for **opt in** enterprise monitoring. Note that even if you discuss monitoring with the Sublime team, you may not be asked to update these fields. > 🚧 > > Acknowledge IAM permissions > > > --------------------------------- > > When you get to the last page in the CloudFormation web app console GUI, right before the orange `Create Stack` button, please ensure you check the following two IAM permissions. The deployment **will fail** if you do not check these two boxes. CloudFormation needs permissions to create these IAM roles for the deployment to succeed. Check **both boxes**. > > ![image](https://files.readme.io/7f06076-docs_aws_cloudformation_permissions.png) View Dashboard [](https://docs.sublime.security/docs/aws-cloudformation#view-dashboard) ------------------------------------------------------------------------------------------- 1. Once the deployment has completed (you should see "CREATE\_COMPLETE" on all items under "Stacks"), click the topmost item in the "Stacks" list. It should contain the word "docker" in the name. ![1126](https://files.readme.io/2aa6855-Screenshot_2023-02-10_at_3.28.35_PM.png "Screenshot 2023-02-10 at 3.28.35 PM.png") 2. Click the "Outputs" tab and copy the Value ending in "amazonaws.com" ![1020](https://files.readme.io/7ef5a3c-Screenshot_2023-02-10_at_3.27.03_PM.png "Screenshot 2023-02-10 at 3.27.03 PM.png") 3. Visit this URL in your web browser. You will see an SSL security warning because the deployment uses a self-signed certificate. Click "Advanced" to bypass this warning and visit your dashboard. 1. You may setup a custom domain and certificate following these [these](https://docs.sublime.security/docs/custom-domains) instructions (this will remove the self signed certificate warning). 4. Bookmark this URL! The Dashboard will guide you through further setup. Resources deployed [](https://docs.sublime.security/docs/aws-cloudformation#resources-deployed) --------------------------------------------------------------------------------------------------- * Virtual Private Cloud (VPC) * Two Public Subnets * Two Private Subnets (with NAT Gateways) * Two Isolated Subnets * RDS Postgres Database * ECS Cluster + AWS Fargate containers * EC2 Instances to support ECS w/ GPUs * EKS (K8s) * API Gateway * SQS * Elasticache (Redis) * S3 Buckets * Elastic File System * EC2 instance used to generate a self-signed certificate * Self-Signed SSL certificate that is imported into AWS Certificate Manager (ACM) and Systems Managers (SSM) * Secrets Manager secrets for Postgres Password and access credentials * Application Load Balancer that leverages the self-signed SSL certificate * Internal Application Load Balancer * Security Groups to restrict access to the infrastructure (only port 443 to the Load Balancer, no access to the RDS databases, etc) Production Readiness [](https://docs.sublime.security/docs/aws-cloudformation#production-readiness) ------------------------------------------------------------------------------------------------------- We advise some additional steps to protect your Sublime Deployment from accidental mistakes. 1. Go to RDS in your deployed region and find the sublime instance -- if you have multiple with random names find the one in the `sublime-platform...` security group. * Alternatively find the nested CloudFormation stack with `rds` in the name and under resources follow the link associated with `AWS::RDS::DBInstance`. * Select `Modify` on the DB Instance, go the the bottom and select the box "Enable deletion protection". Continue and you may opt to apply the change immediately (this change won't cause downtime). * If you ever do need to delete this DB instance you will follow the same steps but unselect the box. 2. Go to CloudFormation and find the "Sublime-Platform" stack. * Select "Stack Actions" and "Edit Termination Protection". Set this to "Enabled" and Save. Usage data [](https://docs.sublime.security/docs/aws-cloudformation#usage-data) =================================================================================== Basic information like version, error logs, and high-level usage metrics are shared with the Sublime Team so we can troubleshoot issues, provide support, and make Sublime better for you and the community. You can change these settings in your Account page. How To Update [](https://docs.sublime.security/docs/aws-cloudformation#how-to-update) ========================================================================================= You only need to update if Sublime reaches out and asks you. These steps allow you to update your cloud infrastructure, and updates to the software itself are made separately (and automatically). This is a straightforward process with lots screens to click through. If you're unsure of anything please reach out! 1. Login into your AWS account and go to "CloudFormation". Find the "Sublime-Platform" stack (there will be many stacks that start with "Sublime-Platform" and include "Nested" -- ignore these). * If you do not see your stack make sure the region in the upper right is the region you deployed to (a US or EU West region). * The name may vary depending on what it was set as originally -- "Sublime-Platform" is just our default. 2. Validate DB configuration. Your DB config may have changed from storage auto scaling, or a version upgrade. 1. Navigate to RDS 2. Find the Sublime database. The name may contain "sublime", otherwise you can differentiate by checking tags -- the sublime DB will have "sublime-version": "platform". 3. Once you've identified the DB, validate that the status is "Available". If the status is anything else (such as "Storage-Optimization"), please hold off updating your stack -- an update will most likely fail. Wait a couple hours and check again, and reach out to Sublime if the status is not resolving. 4. Once you've identified the DB, check the configuration tab and note the "Engine Version", "Instance class", and "Storage" (a value in GiB). 5. Navigate back to AWS CloudFormation. 3. Selected the "Sublime-Platform" stack and then "Update" in the upper right, and "Make a direct change" in the dropdown. ![](https://files.readme.io/df5878c-Screen_Shot_2021-09-15_at_10.04.25_AM.png "Screen Shot 2021-09-15 at 10.04.25 AM.png") 3. Choose "Replace Existing Template" and enter the URL below which corresponds to your deployment region: * us-east-1 * [https://sublime-platform-latest-us-east-1.s3.us-east-1.amazonaws.com/Sublime-Platform-us-east-1.template.json](https://sublime-platform-latest-us-east-1.s3.us-east-1.amazonaws.com/Sublime-Platform-us-east-1.template.json) * us-east-2 * [https://sublime-platform-latest-us-east-2.s3.us-east-2.amazonaws.com/Sublime-Platform-us-east-2.template.json](https://sublime-platform-latest-us-east-2.s3.us-east-2.amazonaws.com/Sublime-Platform-us-east-2.template.json) * us-west-1 * [https://sublime-platform-latest-us-west-1.s3.us-west-1.amazonaws.com/Sublime-Platform-us-west-1.template.json](https://sublime-platform-latest-us-west-1.s3.us-west-1.amazonaws.com/Sublime-Platform-us-west-1.template.json) * us-west-2 * [https://sublime-platform-latest-us-west-2.s3.us-west-2.amazonaws.com/Sublime-Platform-us-west-2.template.json](https://sublime-platform-latest-us-west-2.s3.us-west-2.amazonaws.com/Sublime-Platform-us-west-2.template.json) * eu-west-1 * [https://sublime-platform-latest-eu-west-1.s3.eu-west-1.amazonaws.com/Sublime-Platform-eu-west-1.template.json](https://sublime-platform-latest-eu-west-1.s3.eu-west-1.amazonaws.com/Sublime-Platform-eu-west-1.template.json) * eu-west-2 * [https://sublime-platform-latest-eu-west-2.s3.eu-west-2.amazonaws.com/Sublime-Platform-eu-west-2.template.json](https://sublime-platform-latest-eu-west-2.s3.eu-west-2.amazonaws.com/Sublime-Platform-eu-west-2.template.json) * eu-central-2 * [https://sublime-platform-latest-eu-central-2.s3.eu-central-2.amazonaws.com/Sublime-Platform-eu-central-2.template.json](https://sublime-platform-latest-eu-central-2.s3.eu-central-2.amazonaws.com/Sublime-Platform-eu-central-2.template.json) * eu-north-1 * [https://sublime-platform-latest-eu-north-1.s3.eu-north-1.amazonaws.com/Sublime-Platform-eu-north-1.template.json](https://sublime-platform-latest-eu-north-1.s3.eu-north-1.amazonaws.com/Sublime-Platform-eu-north-1.template.json) * ca-central-1 * [https://sublime-platform-latest-ca-central-1.s3.ca-central-1.amazonaws.com/Sublime-Platform-ca-central-1.template.json](https://sublime-platform-latest-ca-central-1.s3.ca-central-1.amazonaws.com/Sublime-Platform-ca-central-1.template.json) * ap-southeast-2 * [https://sublime-platform-latest-ap-southeast-2.s3.ap-southeast-2.amazonaws.com/Sublime-Platform-ap-southeast-2.template.json](https://sublime-platform-latest-ap-southeast-2.s3.ap-southeast-2.amazonaws.com/Sublime-Platform-ap-southeast-2.template.json) * us-gov-west-1 * [https://sublime-platform-latest-us-gov-west-1.s3.us-gov-west-1.amazonaws.com/Sublime-Platform-us-gov-west-1.template.json](https://sublime-platform-latest-us-gov-west-1.s3.us-gov-west-1.amazonaws.com/Sublime-Platform-us-gov-west-1.template.json) 4. Double check that the URL pasted into shows the same region as the AWS console highlights when you click on the region in the upper right. E.g. if "Virginia" is shown and the drop down highlights `us-east-1` then `us-east-1` should appear the URL (three times). 5. Select Next and edit params as follows 1. `dbsizeparam` this param encapsulates the instance class and configured storage in terrabytes, using a base 10 definition. E.g. `m6g4XLarge6TB` means a `db.m6g.4xlarge` with 6000 GiB of configured storage. Reference your notes from step 2 to select the correct value. 1. The instance class component of the parameter should remain the same, unless you're updating specifically to increase the instance class. 2. The storage value must be the same or greater than your current configured value. If your current storage value is 3000 GiB then ensure your db size param is 3TB. If your current storage was 3300, then you would need to select a 4TB option. 2. `dbversionparam` this should match your currently configured "Engine Version" from step 2. 3. : Leave all other parameters as is, unless asked to do otherwise, and then Next again. * If a parameter is blank or you're otherwise unsure, please reach out! 6. Ensure "Stack failure options" is set to "Roll back all stack resources" and then select Next. * If there are issues with a deploy we may ask you to change this, but if set to preserve CF will refuse to deploy certain types of changes. ![](https://files.readme.io/f9691ec-Screen_Shot_2021-09-15_at_10.07.19_AM.png "Screen Shot 2021-09-15 at 10.07.19 AM.png") 6. Check both of the boxes around permissions at the bottom and select "Update Stack". * You don't need to wait for the change set to load. 7. Occasionally monitor the status of the deploy -- some updates can take hours, others just minutes. * If the stack update does fail please take a screenshot of the failure under "Events" in the "Sublime-Platform" stack. If it says that a nested stack failed to deploy, include a screenshot of the "Events" in the failed nested stack. We apologize for the issue! CloudFormation is a very sensitive and particular system, as well as being subject to transient issues from an array of AWS services! Uninstallation [](https://docs.sublime.security/docs/aws-cloudformation#uninstallation) =========================================================================================== Uninstalling the Sublime Platform and securely wiping all your data is easy: 1. Visit [AWS CloudFormation](https://console.aws.amazon.com/cloudformation) in the region you deployed to. 2. Click on the `sublime-platform` Stack, then click `Delete`. 3. Delete the `/sublime-security/self-signed-acm-certificate-for-sublime-platform` parameter in the [SSM parameter store](https://console.aws.amazon.com/systems-manager/parameters) in your region. 4. Delete the `example.com` certificate in [AWS Certificate Manager](https://console.aws.amazon.com/acm) in your region. You may need to wait for Step 2 to complete in order to delete this cert. 5. CloudFormation sometimes fails to delete resources, including occasionally silent failures. Double check: * EKS for a cluster with "sublime" and/or "strelka" in the name. If present manually delete. * EC2 check for instances or ASG with "hydra" in the name. If present manually terminate/delete. * S3 check for buckets "sublime-screenshots...", "sublime-platform-messages...", "sublime-platform-audit...". If present, empty and delete. Updated about 1 month ago * * * * [Add a message source](https://docs.sublime.security/docs/add-message-source) --- # Message types The Sublime system categorizes the directionality of email messages using the `type` property in the [Message Data Model (MDM)](https://docs.sublime.security/docs/mdm) . The message type lets you construct detection rules to address different use cases in your email environment. For example: * Inbound for phishing * Outbound for DLP * Internal for lateral movement (east-west traffic) Below is how each message type is defined. Message source domains are verified domains associated with your cloud email provider, i.e. Google Workspace or Office 365. Domains are synced daily. | Message Type | Description | | --- | --- | | `type.inbound` | The message was received from a sender outside your message source domains. | | `type.outbound` | Messages sent by your organization to at least 1 recipient outside of your message source domains. | | `type.internal` | Messages sent by your organization where at least 1 recipient is in your message source domains.

Messages **must be authenticated** by either SPF, DKIM, or DMARC and have a `Return-Path` header matching one of your organization’s internal domains to be treated as **internal**.

If a message passes one of SPF, DKIM, or DMARC, _but also fails_ any of the others (including _softfail_), it is treated as an **inbound** message. | A single message could have any of the following combinations of types: 1. Inbound 2. Outbound 3. Internal 4. Outbound _and_ Internal (a message sent by your organization to both an external and internal recipient) Updated almost 2 years ago * * * --- # Attack Surface Reduction Attack Surface Reduction, or ASR, is all about mitigating the ways an attacker can conduct a successful attack. On endpoints, app allowlisting leaves attackers with fewer ways to perform attacks. On networks, Secure Web Gateways can block outbound requests to new domains. Now, you can do the same in Microsoft 365 & Google Workspace using Sublime. If a certain behavior is not normal for your environment, you can block it by default and make exceptions when necessary for legitimate behavior. Here's a [sample ASR Rule](https://sublime.security/feeds/core/detection-rules/new-sender-domain-less10d-from-first-time-sender-d87fa543/) to detect inbound messages from newly registered sender domains that you’ve never spoken to before: MQL `type.inbound and beta.whois(sender․email.domain).days_old < 30 and not profile․by_sender().solicited` When an ASR Rule flags, you’ll see an alert in a view called "Attack Surface Reduction." Similar to Detection and Automation Rules, you can run this in alert-only mode or configure actions to run: Quarantine, Trash, Warning Banner, Webhook, Slack Alert, etc. ![](https://files.readme.io/1b355b0-asr_1.gif) To create your own ASR Rules, simply add the `Attack surface reduction` Tag to your Rule. Check out these Core Feed Rules for more [examples](https://github.com/search?q=repo%3Asublime-security%2Fsublime-rules+%22attack+surface+reduction%22+path%3A%2F%5Edetection-rules%5C%2F%2F&type=code) . Updated 6 months ago * * * --- # Rule Severity Overview [](https://docs.sublime.security/docs/rule-severity#overview) ========================================================================== Rule severities are used to help you prioritize alerts during the triage or investigation process. You can think of severity as **confidence-weighted impact**, where _confidence_ is how likely an alert is a true positive, and _impact_ is the damage the attack the rule is designed to detect could cause. ![604](https://files.readme.io/dc760b4-severity_chart.png "severity chart.png") Severities [](https://docs.sublime.security/docs/rule-severity#severities) ============================================================================== * `critical` is used to identify rules related to CVEs, malware families, and threat actors. `critical` alerts that were not auto-remediated should be reviewed immediately. `critical` can also be used for high-confidence, high-impact alerts that you want prioritized over everything else that is `high`. * `high` alerts that were not auto-remediated should be reviewed quickly. * `medium` alerts that were not auto-remediated should be reviewed frequently. * `low` alerts that were not auto-remediated should be reviewed regularly. A given rule's severity may not be appropriate for everyone because organizations have different compensating controls. For example, a Microsoft 365 credential phishing attack may have a lower severity for organizations enforcing hardware-based MFA. The rules in the Sublime Rules Feed assume minimal compensating controls. Updated over 2 years ago * * * --- # Rule Feeds Overview [](https://docs.sublime.security/docs/rule-feeds#overview) ======================================================================= Git-backed rule feeds let you receive rule updates and new Sublime rules directly in your dashboard from both the Sublime team and the entire Sublime community. To view and manage rule feeds, go to your Sublime Dashboard and click **Feeds** in the navigation bar. ![](https://files.readme.io/3f21bdaffa05e40bdd88c7b0c2b91e40b8be887869acdf1a90543e7a38668d7d-image.png) The Sublime Rules Feed [](https://docs.sublime.security/docs/rule-feeds#the-sublime-rules-feed) =================================================================================================== The **Sublime Rules Feed** is added to your Sublime organization automatically. It's powered by a [public Git repo](https://github.com/sublime-security/sublime-rules/tree/main/detection-rules) managed by the Sublime team with community contributions. Adding feeds [](https://docs.sublime.security/docs/rule-feeds#adding-feeds) =============================================================================== To add a rule feed, such as a Community feed or your own Feed, go to your Sublime Dashboard and click **Feeds** in the navigation bar. Then click **New feed.** Community rule feeds [](https://docs.sublime.security/docs/rule-feeds#community-rule-feeds) ----------------------------------------------------------------------------------------------- Community rule feeds are listed [here](https://github.com/sublime-security/sublime-rules#community-rule-feeds) . Custom rule feeds [](https://docs.sublime.security/docs/rule-feeds#custom-rule-feeds) ----------------------------------------------------------------------------------------- You can create and manage custom rule feeds via both public and private Git repositories. This works with any repo, including those hosted on GitHub, GitLab, or BitBucket, and enables you and other members of the Sublime community to manage email detections the same way you manage other detections: in Git. Private rule feeds [](https://docs.sublime.security/docs/rule-feeds#private-rule-feeds) ------------------------------------------------------------------------------------------- Private rule feeds require authentication. Learn how to set up authentication for private rule feeds [here](https://docs.sublime.security/docs/private-feed-authentication) . Updated 6 months ago * * * --- # Message groups Introduction [](https://docs.sublime.security/docs/message-groups#introduction) ----------------------------------------------------------------------------------- The Sublime Platform automatically groups similar messages to reduce alert fatigue and provide a better user experience when viewing flagged messages in the Dashboard. This means that if 100 users receive the same email, you'll only see one grouping in the Dashboard. And if you have email alerts configured, for example, you'll only receive one alert. ![In this example, 15 similar messages went to 7 different mailboxes. The subject is the same for each message, but the links within the body are different.](https://files.readme.io/bf69776a8976b53c020981093fede6b4cf98c46423fcba6b943504cc0783c21b-select_sample.png) In this example, 15 similar messages went to 7 different mailboxes. The subject is the same for each message, but the links within the body are different. For more information about Sublime message grouping, read [our blog post](https://sublime.security/blog/enhanced-message-groups-improving-efficiency-in-email-incident-response/) . `canonical_id` [](https://docs.sublime.security/docs/message-groups#canonical_id) ------------------------------------------------------------------------------------- When a message is processed, it’s grouped with similar messages that have already been processed. This group ID is called the `canonical_id` and it’s a unique identifier for the group itself. All messages in a group will have similar subjects, body contents, attachments, sender email, headers, etc., but none of these fields are guaranteed to be exactly the same. The `canonical_id` is a unique identifier of a group within an organization, but the same message in a different organization could belong to a group with an entirely different `canonical_id` , even if it’s the only message in the group. It’s not recommended to use the `canonical_id` across different organizations, because the values may not match. > 📘 > > Webhook actions > > > --------------------- > > Since Webhook Actions currently operate on `message.flagged` events (opposed to `message_group.flagged` events), you'll receive a webhook notification for every message that flags in a group. The flagged message's `canonical_id` is included in the [webhook payload](https://docs.sublime.security/docs/webhooks#the-webhook-payload) > . Updated 4 months ago * * * --- # Docker Sublime Platform can be deployed locally on a laptop or remotely to a VPS or VM using Docker. > 📘 > > Docker deployment not intended for production > > > --------------------------------------------------- > > This Docker deployment is intended for small-medium size deployments or for testing purposes ONLY, and should not be used for more than 600 _active_ mailboxes. If you'd like to activate more than 600 mailboxes, use the [AWS Cloud-native deployment](https://docs.sublime.security/docs/aws-cloudformation) > or Sublime Cloud, which can support any number of mailboxes. > > Other limitations apply, see: [Docker-Limitations](https://docs.sublime.security/docs/quickstart-docker#docker-deployment-limitations) Setup [](https://docs.sublime.security/docs/quickstart-docker#setup) ======================================================================== One-line install [](https://docs.sublime.security/docs/quickstart-docker#one-line-install) ---------------------------------------------------------------------------------------------- Copy the command below to the clipboard, then paste it into a bash terminal on macOS or Linux: console `curl -sL https://sublime.security/install.sh | sh` Alternatively, you can follow our [Manual Installation: Docker](https://docs.sublime.security/docs/manual-installation-docker) guide. Requirements [](https://docs.sublime.security/docs/quickstart-docker#requirements) -------------------------------------------------------------------------------------- | Mailboxes | RAM | CPUs | Storage | | --- | --- | --- | --- | | 10 | 8GB | 4 | 50GB | | 100 | 16GB | 8 | 100GB | | 600 | 32GB | 16 | 200GB | * Supported architectures: * amd64 / x86\_64 * arm64 (including Apple Silicon -- M1/M2 Macs) * Supported operating systems: * Linux (Tested on Ubuntu 18-22.10, Amazon Linux 2, CentOS 7, and Fedora 34-37) * Recommended: Ubuntu 22.10 * macOS * The following ports will need to be accessible when deployed remotely: * `3000`: Used by the Sublime Dashboard * `8000`: Used by the Sublime API * `443`: If using SSL * `80`: If using SSL * [Docker](https://docs.docker.com/get-docker/) version: 20.10.0 or greater _Run `docker version` to see your Docker version_ * [Linux installation](https://docs.docker.com/engine/install/ubuntu/#install-using-the-convenience-script) * [macOS installation](https://docs.docker.com/desktop/install/mac-install/) * [Git](https://git-scm.com/downloads) version: 2.7.0 or greater _Run `git version` to see your Git version_ * [docker compose](https://docs.docker.com/compose/install/) version: 2.4.0 or greater _Run `docker compose version` to see your Docker Compose version_ * Linux: comes installed with Docker engine but you can also do the [standalone installation](https://docs.docker.com/compose/install/linux/#install-using-the-repository) * macOS: comes installed with Docker Desktop Docker deployment limitations [](https://docs.sublime.security/docs/quickstart-docker#docker-deployment-limitations) ------------------------------------------------------------------------------------------------------------------------ * This Docker deployment is intended for small-medium size deployments or for testing purposes, and should not be used for more than 600 _active_ mailboxes. If you'd like to activate more than 600 mailboxes, use the [AWS Cloud-native deployment](https://docs.sublime.security/docs/aws-cloudformation) or Sublime Cloud, which can support any number of mailboxes. * The Docker deployment is "polling" based, opposed to callback/subscription based, so there’s naturally at least a 15 second delay between polling intervals when checking for new messages that have arrived. You may see a longer delay with more mailboxes. The cloud-native deployments are callback-based (i.e. push instead of pull), and real-time. * System updates bring the platform fully down, so mailboxes are not protected when standard updates are applied. Downtime for typical updates is usually short (less than a minute), but larger updates could take longer. * Downloading the raw EML for unflagged messages isn't currently supported due to raw message retention limitations. * Audit log export to S3 currently isn't supported. * Message analysis time is slower than cloud-native deployments, and there is no auto-scaling. Inference for some machine learning models are most performant on GPUs, which currently isn't supported via our Docker containers. * Enterprise monitoring is more limited for Docker based deployments. * No built-in backups or instrumentation for backups. * Child orgs are not supported SSL [](https://docs.sublime.security/docs/quickstart-docker#ssl) -------------------------------------------------------------------- To enable SSL for your installation via Nginx and certbot, follow the steps below. **1\. Set up your domain** 1. Register your domain if you don't already have one, or use a subdomain. 2. Create an A record to point to your VPS/host. 3. Verify your A record has propagated. Run this in your terminal: `dig YOUR_DOMAIN A` **2\. Configure Sublime** 1. Navigate to your `sublime-platform` directory. 2. Run `cp certbot.env.example certbot.env` 3. Edit `certbot.env` with your domain and the email associated with it. 4. Create a file called `sublime.env` with the following contents (replace `YOUR_DOMAIN_HERE` with the domain you registered in step 1). `CORS_ALLOW_ORIGINS=https://YOUR_DOMAIN_HERE BASE_URL=https://YOUR_DOMAIN_HERE DASHBOARD_PUBLIC_BASE_URL=https://YOUR_DOMAIN_HERE API_PUBLIC_BASE_URL=https://YOUR_DOMAIN_HERE` **Note**: The URLs for these environment variables should not include a port. For example, if you're hosting Sublime at `sublime.example.com`, the configured URLs should be `https://sublime.example.com`, without any port. 6. Ensure that ports 80 and 443 are open to the web. This is necessary for `certbot` to create your LetsEncrypt certs. 7. Re-run the install script: `clone_platform=false ./install-and-launch.sh` How to update [](https://docs.sublime.security/docs/quickstart-docker#how-to-update) ======================================================================================== > 📘 > > Automatic updates > > > ----------------------- > > If you deployed Sublime with one of our scripts, you may have enabled automatic updates (using Cron), which run nightly. If you want to manually update your Sublime Platform then follow the steps below. 1. Navigate to your `sublime-platform` directory. 2. Copy and paste the appropriate shell commands below depending on how you setup Sublime. This will update your Github repo, then pull the latest images from Docker Hub and restart your instances: Default `docker compose down; git pull; docker compose pull; docker compose up -d;` LetsEncrypt `docker compose --profile letsencrypt down; git pull; docker compose --profile letsencrypt pull --include-deps; docker compose --profile letsencrypt up -d;` Troubleshooting [](https://docs.sublime.security/docs/quickstart-docker#troubleshooting) ============================================================================================ If you run into any issues during setup, or while running Sublime, below are troubleshooting steps that can help you diagnose and fix. Snap is not supported [](https://docs.sublime.security/docs/quickstart-docker#snap-is-not-supported) -------------------------------------------------------------------------------------------------------- Snap installations of Docker are currently not supported because they can cause issues when using Docker Compose in the future. If you installed Docker via Snap, you may run into problems starting the Platform. Even if you have removed the package and reinstalled Docker using the recommended method, you may still experience problems. This is because Snap's Docker uninstall process is incomplete, leaving remnants of the old installation which will corrupt future installations of Docker. Luckily, there is an easy solution: 1. Remove the Snap installation of Docker: Bash `snap remove docker` 2. Install Docker using the [official Docker installation steps](https://docs.docker.com/get-docker/) 3. To finish clean-up of the Snap uninstall process, reboot your machine: Bash `reboot` Docker logs errors [](https://docs.sublime.security/docs/quickstart-docker#docker-logs-errors) -------------------------------------------------------------------------------------------------- Run the following command to look for errors: console `docker compose logs | grep -i 'error\|fatal\|panic'` **If you see the following error: `server error (FATAL: password authentication failed for user \"sublime\")`** This likely means that the `sublime.env` file was overwritten and the credentials are no longer valid, or you have previous installation of Sublime. If this is the case, you can either copy your old `sublime.env` file to your new deployment directory, or follow the steps below to wipe your Postgres volume if you don't have the old sublime.env file, and there is no data that you care to keep in Postgres. **If you see the following error: `exec: "docker-credential-desktop": executable file not found in $PATH`** You'll need to edit `~/.docker/config.json` and delete the JSON key/pair `"credsStore": "desktop",` (see [this issue](https://github.com/docker/docker-credential-helpers/issues/149) for details). You should be able to re-run your script. **If you see the following error: `no space left on device`** See [out of disk space](https://docs.sublime.security/docs/quickstart-docker#out-of-disk-space) > 📘 > > Live tail Docker logs > > > --------------------------- > > You can live tail the logs at any time by running: > > console > > `docker compose logs -f` Docker Compose errors [](https://docs.sublime.security/docs/quickstart-docker#docker-compose-errors) -------------------------------------------------------------------------------------------------------- ### Error: `docker.errors.DockerException: Error while fetching server API version` [](https://docs.sublime.security/docs/quickstart-docker#error-dockererrorsdockerexception-error-while-fetching-server-api-version) If you see this error when running `docker compose`, this likely means there's a permissions issue. You may need to run the command using `sudo docker compose`. ### Error: `no space left on device` [](https://docs.sublime.security/docs/quickstart-docker#error-no-space-left-on-device) See [out of disk space](https://docs.sublime.security/docs/quickstart-docker#out-of-disk-space) Waiting for services to start [](https://docs.sublime.security/docs/quickstart-docker#waiting-for-services-to-start) ------------------------------------------------------------------------------------------------------------------------ It can take several minutes for all backend services to come up and be in a healthy state. If you see this message when accessing the Dashboard after waiting for several minutes, this likely means either the Dashboard cannot access the backend (eg a CORS misconfiguration or firewall setting), or a backend service is not running. Check your **browser's developer tools** to confirm. ### Browser developer tools: Console tab [](https://docs.sublime.security/docs/quickstart-docker#browser-developer-tools-console-tab) If you see a `Cross Origin Resource Sharing` error, such as `Access to fetch at '{...}' from origin '{...}' has been blocked by CORS policy`, this likely confirms there's a CORS misconfiguration. If you deployed Sublime to a remote VPS, but haven't updated the backend's allowed CORS origins, you'll see this error. CORS origins are restricted by default for your security. See step 2 of [Manual Installation: Docker](https://docs.sublime.security/docs/manual-installation-docker) for instructions on how to update the CORS settings for your deployment. If you see `ERR_CONNECTION_REFUSED`, this likely means the backend API is not running. Check the Docker logs (see above) for more information. ### Browser developer tools: Network tab [](https://docs.sublime.security/docs/quickstart-docker#browser-developer-tools-network-tab) See what status codes are being returned from the requests to `/v1/health`. This can be useful for debugging. ### Backend not running [](https://docs.sublime.security/docs/quickstart-docker#backend-not-running) If your backend is not running after installation, then you may want to reset your installation by: * Wiping your Postgres volume (see below) without starting Sublime again * Removing your Sublime Platform directory: `rm -rf ./sublime-platform` * Follow installation steps again ### Clear your local storage [](https://docs.sublime.security/docs/quickstart-docker#clear-your-local-storage) If you're in a bad state (eg if you've wiped your deployment and started fresh, but still running into issues), you can try clearing your local storage from your browser's developer tools: [![](https://user-images.githubusercontent.com/11003450/212777369-380b08f3-c870-425f-a20c-f3350f108c4f.gif)\ \ **View**: https://user-images.githubusercontent.com/11003450/212777369-380b08f3-c870-425f-a20c-f3350f108c4f.gif](https://user-images.githubusercontent.com/11003450/212777369-380b08f3-c870-425f-a20c-f3350f108c4f.gif) SSL errors [](https://docs.sublime.security/docs/quickstart-docker#ssl-errors) ---------------------------------------------------------------------------------- If you are unable to access the dashboard using `https` after configuring SSL, there may have been errors on startup. It can be useful to interrogate the logs from your `sublime_nginx_letsencrypt` container specifically to ensure no errors occurred while creating the certificate. To do so, run: `sudo docker compose logs sublime_nginx_letsencrypt` ### Error: Are you trying to change the key type of the certificate...? [](https://docs.sublime.security/docs/quickstart-docker#error-are-you-trying-to-change-the-key-type-of-the-certificate) If you encounter this error, you most likely have already registered a certificate for your domain using a non-RSA algorithm. The easiest way to fix this is to register a new subdomain, update your configuration, and re-run installation. Since your new subdomain will not yet have been issued a certificate, installation should be able to proceed successfully. > 📘 > > Fixing certificate collisions > > > ----------------------------------- > > 1. Register a new subdomain > 2. Update your `certbot.env` and `sublime.env` files with the new subdomain > 3. Re-run: `clone_platform=false ./install-and-launch.sh` Wipe your data [](https://docs.sublime.security/docs/quickstart-docker#wipe-your-data) ------------------------------------------------------------------------------------------ > ❗️ > > THIS WILL WIPE ALL YOUR SUBLIME DATA STORED IN POSTGRES > > > ------------------------------------------------------------- > > Only follow the steps below if you are certain you don't need access to any data within your Sublime deployment. 1. Navigate to the sublime-platform directory created during setup: Bash `cd sublime-platform` 2. Stop the docker containers: Default `docker compose down` LetsEncrypt `docker compose --profile letsencrypt down` 3. Remove the docker containers: Bash `docker compose rm` 4. Wipe the Sublime docker volumes: Bash `docker volume rm $(docker volume ls -f name=sublime-platform --format "{{.Name}}")` If you wish to remove Sublime completely, you may remove the `sublime-platform/` directory, and the cron job (`crontab -e`) if you configured automatic updates. Out of disk space [](https://docs.sublime.security/docs/quickstart-docker#out-of-disk-space) ------------------------------------------------------------------------------------------------ If you run out of disk space while running Sublime Platform, you have a few options. > 📘 > > After following any of the steps below, **restart your Sublime deployment**: > > > ---------------------------------------------------------------------------------- > > Bash > > `docker compose down && docker compose up` ### 1\. Freeing up Docker disk space [](https://docs.sublime.security/docs/quickstart-docker#1-freeing-up-docker-disk-space) You can prune unused images to free up Docker disk resources. Remove all unused images: Shell `docker image prune` ### 2\. Expand the resources allocated to Docker [](https://docs.sublime.security/docs/quickstart-docker#2-expand-the-resources-allocated-to-docker) For macOS deployments / Docker Desktop, you can solve this by expanding Disk image size in Docker settings: ![](https://files.readme.io/fbde4f5-Docker_settings.png "Docker settings.png") ### 3\. Expand the host file system [](https://docs.sublime.security/docs/quickstart-docker#3-expand-the-host-file-system) If the Docker resources aren't an issue, but your host has run out of disk space, then you'll need to expand your host file system volume. For Ubuntu distributions, [this is a good guide](https://packetpushers.net/ubuntu-extend-your-default-lvm-space) on how to expand your host's file system volume. Using a proxy [](https://docs.sublime.security/docs/quickstart-docker#using-a-proxy) ======================================================================================== If you want to set up a proxy in front of Sublime (e.g., nginx), it's important to note that Sublime's frontend and API are served on different ports when using Docker Compose. The frontend is served on port `3000` while the API is served on port `8000`. This means you should configure your proxy to proxy all API requests (requests with paths starting with `/v0/` or `/v1/`) to port `8000` and all other requests to port `3000`. Allow requests to port `8110` to pass through. Also be certain to update the configuration in your `sublime.env` file to reflect the hostname for your proxy. See step 2 of [Manual Installation: Docker](https://docs.sublime.security/docs/manual-installation-docker) for more information. By default for security reasons, any Sublime features which leverage client IP address (e.g. IP Allow List and Audit Log) will prefer the using the first value from the `X-Forwarded-For` header. * If you're not using a Proxy, you may add `IP_DETECTION=REMOTE_ADDR` to strictly use the remote address and ignore the header. * You may also set `IP_DETECTION=X-FORWARDED-TRUST-LAST` to prefer using last value of the `X-Forwarded-For` header. Deploying to AWS EC2 [](https://docs.sublime.security/docs/quickstart-docker#deploying-to-aws-ec2) ====================================================================================================== If you deploy the Docker deployment to EC2, you must allow ports 3000 and 8000 ingress in the VPC's security group. Updating your Sublime host [](https://docs.sublime.security/docs/quickstart-docker#updating-your-sublime-host) ================================================================================================================== If you installed Sublime on a remote VPS or VM, and you mistakenly used the default `http://localhost` Sublime host, you can follow step 3 of [Manual Installation: Docker](https://docs.sublime.security/docs/manual-installation-docker) to update your `sublime.env` to point to the correct host. This will save you from having to reset your entire Sublime Platform. If you want to setup a proxy such as nginx, see [using a proxy](https://docs.sublime.security/docs/quickstart-docker#using-a-proxy) . Usage data [](https://docs.sublime.security/docs/quickstart-docker#usage-data) ================================================================================== Basic information like version, error logs, and high-level usage metrics are shared with the Sublime Team so we can troubleshoot issues, provide support, and make Sublime better for you and the community. You can change these settings in your Account page. Docker services [](https://docs.sublime.security/docs/quickstart-docker#docker-services) ============================================================================================ These are the Docker containers that are spun up when running Docker Compose: * Backend services: * `mantis`: API service * `bora-lite`: Async task service which is most commonly used for processing emails * `screenshot-service`: Converts email HTML to image * `hydra`: Machine learning (ML) service that powers a number of detection capabilities * Frontend services: * `dashboard`: Serves the dashboard frontend * Infrastructure: `posgres`, `redis`, `sublime3` * `postgres`: DB instance used by backend services * `redis`: Shared cache * `sublime3`: A local version of AWS S3 * [Strelka](https://github.com/target/strelka) : `strelka-frontend`, `strelka-backend`, `strelka-manager`, `strelka-coordinator` * A real-time, container-based file scanning system used for threat hunting, threat detection, and incident response created by Target Support [](https://docs.sublime.security/docs/quickstart-docker#support) ============================================================================ If you run into any issues or have questions, [send us an email](https://docs.sublime.security/cdn-cgi/l/email-protection#a8dbddd8d8c7dadce8dbddcac4c1c5cd86dbcdcbdddac1dcd1) or post in the Slack community. Updated 6 months ago * * * --- # Private rule feed authentication Sublime uses SSH to authenticate to private rule feeds. > 📘 > > SSH URL Required > > > ---------------------- > > When adding a rule feed using SSH authentication to Sublime, you must use the SSH URL for the repository. > > ✅ `[[email protected]](https://docs.sublime.security/cdn-cgi/l/email-protection) :sublime-security/sublime-rules.git` > > ❌ `https://github.com/sublime-security/sublime-rules.git` Adding an SSH key [](https://docs.sublime.security/docs/private-feed-authentication#adding-an-ssh-key) ========================================================================================================== When creating an SSH key, make sure not to put a password on the key, since there is no way for Sublime to enter in the password. Don't worry though, the key is still encrypted in transit and at rest! GitHub [](https://docs.sublime.security/docs/private-feed-authentication#github) ------------------------------------------------------------------------------------ 1. [Generate a new SSH key](https://docs.github.com/en/authentication/connecting-to-github-with-ssh/generating-a-new-ssh-key-and-adding-it-to-the-ssh-agent) 2. Add the new SSH key to either your GitHub account or as a deploy key for the private repository containing your feed * [Add an SSH key as a deploy key for the repository](https://docs.github.com/en/developers/overview/managing-deploy-keys#deploy-keys) (recommended) * [Add an SSH key to your GitHub account](https://docs.github.com/en/authentication/connecting-to-github-with-ssh/adding-a-new-ssh-key-to-your-github-account) GitLab [](https://docs.sublime.security/docs/private-feed-authentication#gitlab) ------------------------------------------------------------------------------------ 1. [Generate a new SSH key](https://docs.gitlab.com/ee/user/ssh.html#generate-an-ssh-key-pair) 2. [Add the SSH key to your GitLab account](https://docs.gitlab.com/ee/user/ssh.html#add-an-ssh-key-to-your-gitlab-account) Known host public key [](https://docs.sublime.security/docs/private-feed-authentication#known-host-public-key) ================================================================================================================== When adding a private feed to Sublime, you can optionally provide a known host public key as extra protection against configuration errors and man-in-the-middle attacks. An SSH connection will fail if the public keys don't match. Use `ssh-keyscan` to look up the public key for your Git server. For example, `ssh-keyscan github.com` will provide GitHub's current public key of: `ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEmKSENjQEezOmxkZMy7opKgwFB9nkt5YRrYMjNuG5N87uRgg6CLrbo5wAdT/y6v0mKV0U2w0WZ2YB/++Tpockg=` Updated over 2 years ago * * * --- # Trash Overview [](https://docs.sublime.security/docs/trash#overview) ------------------------------------------------------------------ This Action sends flagged messages to the trash folder of a mailbox. It can be used automatically via Rules, manually in the dashboard, or via API. In Google Workspace, trashed messages are moved to the `Trash` folder. In Microsoft 365, trashed messages are moved to the [`Recoverable Items` folder](https://learn.microsoft.com/en-us/exchange/security-and-compliance/recoverable-items-folder/recoverable-items-folder) . How to add Auto-trash to Rules [](https://docs.sublime.security/docs/trash#how-to-add-auto-trash-to-rules) -------------------------------------------------------------------------------------------------------------- 1. Click "Detection Rules" or “Automations” in the left nav of your Dashboard 2. Click on the Rule you wish to add the action to, opening the detail view for this Rule 3. Click "Edit" in the top right of this detail view 4. Click the "Actions" dropdown and select your Auto-trash Action 5. Click "Save Rule" Updated 9 months ago * * * --- # Warning Banners Overview [](https://docs.sublime.security/docs/warning-banners#overview) ---------------------------------------------------------------------------- The Warning Banners Action is an Enterprise feature that allows you to surface a warning to mailbox users. Warning Banners are designed to fill the gap between high-fidelity Rules you set to auto-remediate and low-fidelity Rules that only generate alerts. This new Action applies a banner that alerts users to suspicious email messages. You can also customize the Warning Banner to fit your organization's tone or workflows. How to add Warning Banners to Rules [](https://docs.sublime.security/docs/warning-banners#how-to-add-warning-banners-to-rules) ---------------------------------------------------------------------------------------------------------------------------------- 1. Click "Detection Rules" or “Automations” in the left nav of your Dashboard 2. Click on the Rule you wish to add the action to, opening the detail view for this Rule 3. Click "Edit" or "Edit Metadata" in the top right of this detail view 4. Click the "Actions" dropdown and select your Warning Banner Action 5. Click "Save Rule" Sublime's default banner looks like this when applied: ![](https://files.readme.io/5967fc1deb8d0ddb83a04d0e47845f90961e14b5559dd0f3fde75717aaa49ec7-sample_banner.png) How to create a custom Warning Banner [](https://docs.sublime.security/docs/warning-banners#how-to-create-a-custom-warning-banner) -------------------------------------------------------------------------------------------------------------------------------------- 1. Click "Actions" in the left nav of your Dashboard 2. Click on the "Warning Banner" Action 3. Click "Edit" in the top right of this detail view 4. Customize the Header and Description to your liking! Updated 5 months ago * * * --- # Auto-respond to User Reports Overview [](https://docs.sublime.security/docs/auto-respond-to-user-reports#overview) ----------------------------------------------------------------------------------------- With Sublime's automatic responses to user reports, Microsoft365 and Google Workspace users can automatically interact with and give feedback to reporting end-users: 1. Acknowledge user report * After Sublime receives the user report, we'll insert a templated message into the reporting end-user’s mailbox. * In order to insert the message, the reporting mailbox must be `active`. 2. Close the loop with reporter * After the message is classified (`malicious`, `spam`, `graymail`, `benign`, or `simulation`), we'll insert another templated message into the reporting end-user’s mailbox. * Each classification has a corresponding template that's customizable to fit your organization's needs. * In order to insert the message, the reporting mailbox must be `active` in Sublime. If you accept unauthenticated user reports, which are reports coming from a mailbox outside of your organization, automatic responses will only be sent to the authenticated user reports that you receive from `active` mailboxes in your organization. Automatic responses are not available for IMAP users at this time. Enable automatic responses [](https://docs.sublime.security/docs/auto-respond-to-user-reports#enable-automatic-responses) ----------------------------------------------------------------------------------------------------------------------------- Setting up automatic responses is easy: 1. Firstly, ensure you have an [Abuse Mailbox](https://docs.sublime.security/docs/add-your-abuse-mailbox) configured. 2. Activate the two Core Automations: "Acknowledge user report" and "Close the loop with reporter". 3. Optional: customize the 6 available templates to your organization's specifications. 4. Optional: add a 1-minute delay to all close the loop messages to prevent both messages from landing in rapid succession. Once you're done with setup, the templates will be inserted into reporting mailboxes without any additional steps from your analysts. ![](https://files.readme.io/54de5da2fa28b1ea97460581df5649e27aa34bc6583c05c377ab4374a8f756b7-activating_automated_responses.gif) The Automations will show up on your user reported messages when applicable so you can easily identify which messages have already received the acknowledgement and/or the results of the review. Example use cases [](https://docs.sublime.security/docs/auto-respond-to-user-reports#example-use-cases) ----------------------------------------------------------------------------------------------------------- ### Reduce end-user outreach about reported messages [](https://docs.sublime.security/docs/auto-respond-to-user-reports#reduce-end-user-outreach-about-reported-messages) Turn on automatic responses to reduce outreach inquiring about the status of a user report from end-users. Reduce time that analysts spend answering questions like "Did you get my report?", "I didn't see that message in my mailbox. Was it phishing after all?", and more. ### Provide feedback on the quality of user reports [](https://docs.sublime.security/docs/auto-respond-to-user-reports#provide-feedback-on-the-quality-of-user-reports) Constructive feedback is always helpful, and user reports are no different! Give your reporters information on the outcome of their user report so they can better spot similar messages in the future. Reduce the number of benign, spam, and graymail user reports by letting users know when a given report is not malicious. ### Alert reporters after successfully identifying phishing simulations [](https://docs.sublime.security/docs/auto-respond-to-user-reports#alert-reporters-after-successfully-identifying-phishing-simulations) Use Sublime in coordination with your phishing simulation tool to quickly alert users when they successfully report a phishing simulation. Don't wait days or weeks for your phishing simulation to be considered "done" -- let them know immediately with an automatic response. Updated 4 months ago * * * --- # Track Link Clicks (beta) Overview [](https://docs.sublime.security/docs/track-link-clicks#overview) ------------------------------------------------------------------------------ Track Link Clicks is a **beta** Enterprise Action that rewrites URLs within a flagged message’s body and tracks the clicks on that rewritten URL. Information available in the message details page includes: * Count of clicks * Mailboxes that clicked * Time of each click * Original URL that was visited (even if the link was rewritten by another product) **Please note:** this feature is not currently available for Google Workspace. Reach out to the Sublime team at [\[email protected\]](https://docs.sublime.security/cdn-cgi/l/email-protection#790a0c0909160b0d390a0c1b1510141c570a1c1a0c0b100d00) to learn more about participating in the beta! Updated about 1 month ago * * * --- # Rules file format (YAML) Format [](https://docs.sublime.security/docs/yaml-files#format) =================================================================== Rules are stored in [YAML files](https://en.wikipedia.org/wiki/YAML) with a `.yml` extension and have the following format: _(\* denotes required fields)_ YAML `*name: string *description: string references: - string authors: - twitter: string - name: string - github: string - email: string type: string severity: string *source: string tags: - string attack_types: - string tactics_and_techniques: - string detection_methods: - string` | Field | Required | Description | Acceptable values | | --- | --- | --- | --- | | name | **true** | Name for the rule or query | Any string | | source | **true** | Raw MQL | Any string | | description | **true** | Digestible description of the rule and its purpose | Any string | | type | **true** | Entity type | `rule` | | severity | false | Criticality of a triggered rule | `low`
`medium`
`high`
`critical` | | references | false | References to where the rule was derived from. Blogs, tweets, papers, etc. | A list of strings (typically URLs) | | tags | false | A way to categorize the rule or query | A list of any strings | | false\_positives | false | Descriptions of known false positives that can occur | A list of any strings | | authors | false | Original author(s) | A list of Twitter profiles or names | | attack\_types | false | Profiles of cyber threats that pinpoint the attackers' primary intent, whether it's to steal credentials, distribute malware, or commit fraud. | `BEC/Fraud`
`Callback Phishing`
`Credential Phishing`
`Extortion`
`Malware/Ransomware`
`Spam` | | tactics\_and\_techniques | false | Methods and strategies employed by threat actors to execute their attack, focusing on their actions, behaviors, and artifacts. | `Evasion`
`Encryption`
`Exploit`
`Free email provider`
`Free file host`
`Free subdomain host`
`HTML smuggling`
`Image as content`
`Impersonation: Brand`
`Impersonation: Employee`
`Impersonation: VIP`
`IPFS`
`ISO`
`LNK`
`Lookalike domain`
`Macros`
`OneNote`
`Open redirect`
`Out of band pivot`
`PDF`
`Punycode`
`QR code`
`Scripting`
`Social engineering`
`Spoofing` | | detection\_methods | false | Highlights the technical methodologies and Sublime specialized techniques that recognized and flagged the threat, offering insights. | `Archive analysis`
`Computer Vision`
`Content analysis`
`Exif analysis`
`File analysis`
`Header analysis`
`HTML analysis`
`Javascript analysis`
`Macro analysis`
`Natural Language Understanding`
`OLE analysis`
`Optical Character Recognition`
`QR code analysis`
`Sender analysis`
`Threat intelligence`
`URL analysis`
`URL screenshot`
`Whois`
`XML analysis`
`YARA` | Multi-line strings in YAML are denoted using a `|` like so: YAML `name: "Mismatched link" source: | type.inbound and any(body.links, .mismatched) type: rule` Updated 5 months ago * * * --- # Automations Overview [](https://docs.sublime.security/docs/automations#overview) ======================================================================== Automations are used for auto-triaging flagged and user reported messages. Automations take high confidence remediative Actions to improve analyst efficiency. ![The Automations Card View](https://files.readme.io/d8164f83d7f75cb0e7045690c5efd2dcac9a395eafe79bf0af83a6c1a75ed680-automations.png) Here are a few ways organizations are using Automations today: * Auto-quarantine flagged messages with a Malicious Attack Score verdict * Auto-quarantine campaigns when multiple Detection Rules flag * Auto-quarantine entire campaigns based on a number of user reports * Trigger an email alert whenever a VIP user reports a message * Apply a warning banner to any flagged message when the recipient is a VIP What are Automations? [](https://docs.sublime.security/docs/automations#what-are-automations) ================================================================================================= Automations are MQL logic compared against messages, just like a Detection Rule. However, Automations use `triggers` to designate which type of message is eligible for analysis and then apply Actions to matching messages. The `trigger` for an Automation is when a user reports a message and/or when any Detection Rule flags a message. Automations also have a special active status, `passive mode`, that allows you to run an Automation over incoming messages and receive alert-based Actions like webhooks, but without applying remediative Actions like Auto-quarantine, Auto-trash, or Auto-banner. ![](https://files.readme.io/7ff69923fdf67aa2e381bfdebac537f9b8bae3bca0ebefd420c992f8d3ea6048-automations_flagged_message-optimized.gif) Sublime shares recommended Automations in the Core Feed, just like Detection Rules. Core Automations are inactive by default and come with default Actions, so you can easily kick off your automated response to malicious attack patterns. You can set Core Automations to `active` to get coverage against malicious messages or to `passive mode` to observe the Automations in your environment without the default Action! When writing Automations, you may reference special functions -- `beta.attack_score` and `triage.` -- that aren't available in Detection Rules: * `beta` - Beta feature * `.attack_score` - Attack Score information about a given message * `.verdict` -`*string*` - A description — `malicious`, `suspicious`, or `unknown` — to communicate the likelihood of a given message being an attack * `.score` - `*float*` - A number from 0-100 calculated by the ML algorithm, with higher scores indicating a higher correlation with messages involved in attacks * `triage` - Message (group) information available during the evaluation of Automations: * `.user_reports` - Aggregate information about user reports for the message group * `.count` - `*number*` - A count of the number of times this message group has been reported by distinct users * `.flagged_rules` - List of all Detection Rules that matched any message in the group * `.attack_types` - `*array of strings*` - Attack types associated with the Rule * `.detection_methods` - `*array of strings*` - Detection methods associated with the Rule * `.name` - `*string*` - Name of the Rule * `.severity` - `*string*` - Severity level of the Rule * `.tactics_and_techniques` - `*array of strings*` - Tactics and techniques associated with the Rule * `.tags` - `*array of strings*` - Tags associated with the Rule, including user provided tags * `.feed` - Information about the feed containing this rule, or `null` if the rule didn't come from a feed * `.name` - `*string*` - Name of the feed * `.is_core` - `*boolean*` - Whether the feed is the **Sublime Rules Feed** Example Automations [](https://docs.sublime.security/docs/automations#example-automations) ============================================================================================== Flagged or user reported messages with a `malicious` Attack Score verdict * _Trigger: flagged message, user report_ * _Recommended Action: auto-quarantine, auto-review_ MQL `type.inbound and ml.attack_score() .verdict == "malicious"` Flagged or user reported messages with a `suspicious` Attack Score verdict * _Trigger: flagged message, user report_ * _Recommended Action: warning banner_ MQL `type.inbound and ml.attack_score() .verdict == "suspicious"` User reported message with more than 1 report * _Trigger: user report_ * _Recommended Action: warning banner_ MQL `type.inbound and triage.user_reports.count > 1` Flagged messages with more than 3 matching Detection Rules * _Trigger: flagged message_ * _Recommended Action: auto-quarantine_ MQL `type.inbound and length(triage.flagged_rules) > 3` Flagged messages that landed in spam * _Trigger: flagged message_ * _Recommended Action: auto-quarantine_ MQL `type.inbound and external.spam` Flagged or user reported messages with Detection Rules that have specific tags * _Trigger: flagged message, user report_ * _Recommended Action: auto-quarantine_ MQL `type.inbound and any(triage.flagged_rules, any(.tags, . == "tag name here"))` Get started [](https://docs.sublime.security/docs/automations#get-started) ============================================================================== To use the _user report_ trigger, read our docs on [User-reported phishing](https://docs.sublime.security/docs/user-reported-phishing) . To create an Automation, visit **Rules > Automation** and click **New Automation**. When you finish writing your MQL, click **Create Automation**. In the metadata modal, be sure to set the **Triggers** to “user report” and/or “flagged message”. Updated 4 months ago * * * --- # Auto-review Overview [](https://docs.sublime.security/docs/auto-review#overview) ------------------------------------------------------------------------ Auto-review is an Action for highly effective Rules to automate the message review process. Auto-review marks flagged message groups as “auto-reviewed”, adds a classification (Malicious, Benign, Graymail, Spam, or Simulation), and prevents the flagged message from appearing in your default triage view. You can view all auto-reviewed messages via Flagged > Auto-Reviewed in the nav. ![](https://files.readme.io/bacc6c5-auto_activate.gif) How to add auto-review to a Rule [](https://docs.sublime.security/docs/auto-review#how-to-add-auto-review-to-a-rule) ------------------------------------------------------------------------------------------------------------------------ Auto-review can't be added to Rules in-bulk. Instead, you need to add auto-review to each Rule so you can see a classification. 1. Select a Rule and open its details page. 2. On the details page, click "Edit" or "Edit metadata" in the top right. 3. Add the Action "Auto-review". 4. After you select "Auto-review", a new field called "Classification for Auto-Review" will appear. Select a classification (Malicious, Unwanted) that best aligns with the classification you usually select during manual reviews of its matching messages. _For example, if a Rule is very effective at locating spam messages and I want to assign auto-review to it, I will choose the classification "unwanted"._ 5. Click "Save Rule" ![](https://files.readme.io/13b3a23-auto_review_application.gif) ### Auto-review hierarchy [](https://docs.sublime.security/docs/auto-review#auto-review-hierarchy) Auto-review uses a hierarchy to determine which classification to apply in cases where 2 or more classifications are being applied: _Simulation > Benign > Malicious > Spam > Graymail_ This means that if Automation #1 with `auto-review as simulation` and Automation #2 with `auto-review as malicious` both match a given message, auto-review will choose to mark the message as `simulation` instead of `malicious`. Updated 6 months ago * * * --- # YARA Overview [](https://docs.sublime.security/docs/yara#overview) ================================================================= [YARA](http://virustotal.github.io/yara/) is an open-source tool designed to help malware researchers identify and classify malware samples. It makes it possible to create signatures for malware families based on textual and/or binary patterns. YARA support in Sublime enables you to better block known malware using open-source YARA signatures, your own custom YARA signatures, and YARA signatures shared by your peers. With YARA + MQL, you can combine your YARA signatures with powerful email context like first-time sender or Azure AD Group recipients to block malicious files before they're detonated. You can also use YARA signatures to Hunt over historically received files via email. ![](https://files.readme.io/d0f4d4fb42de6cef019c62b39aec8f7856a423e19c1cb291a208aa0507d30f58-image.png) Getting started [](https://docs.sublime.security/docs/yara#getting-started) =============================================================================== In order to use YARA signatures in your Sublime rules, you must first add a Feed that points to a private or public Git repository containing the signatures you wish you use. After adding the Feed, the YARA signatures will be ingested automatically, and will run each time you invoke MQL's [`file.explode()`](https://docs.sublime.security/docs/enrichment-functions#fileexplode) on both the original file passed to the function as well as every file that is recursively extracted. Creating a Feed [](https://docs.sublime.security/docs/yara#creating-a-feed) ------------------------------------------------------------------------------- 1. First, identify or create a Git repo containing the YARA signatures you want to use. If you don't have one yet, you can either: a. [follow our testing instructions](https://docs.sublime.security/docs/yara#testing-yara-in-sublime) to see YARA working end to end b. or use one of the [public YARA repos](https://github.com/YARAHQ/yara-forge/blob/master/yara-forge-config.yml) 2. Within the `Feeds` section of your Dashboard, click `New feed` in the top right-side corner. 3. Name the Feed, for example: `My YARA signatures` 4. Enter the URL for any private or public Git repository consisting of one or more YARA signatures. 5. Enter the respective Git branch. 6. Enter the glob pattern for where the YARA signatures are within the repo. For example, if they’re in the root level yara/ directory, enter `yara/*.yar`. 7. Click `Save`. ![1370](https://files.readme.io/e25a240-yara_new_feed_example.png "yara_new_feed_example.png") YARA signatures will be compiled and validated on Feed creation. The Feed will surface the number of YARA signatures ingested into the platform from the repository. If any files fail validation, they will be excluded from the Feed. Errors will appear in the Admin section under System Errors. > ⚠️ > -- > > To look at each YARA signature, you’ll need to view them in the source repository. Viewing YARA signature source within the Sublime Dashboard is currently not supported. Creating an MQL Rule [](https://docs.sublime.security/docs/yara#creating-an-mql-rule) ----------------------------------------------------------------------------------------- Once YARA signatures are ingested via your Feed, they will run by default any time `file.explode()` is invoked. Any matches will be available in the `.scan.yara.matches` array of the FileExplode output. Since FileExplode runs over any file, this means you can run YARA over any attachment or even files auto-downloaded from links via LinkAnalysis. Using FileExplode's `.scan.yara.matches`, you can look for: 1. _Any YARA signature match with an archive attachment_ using the length of the scan matches: MQL `any(attachments, .file_extension in $file_extensions_common_archives and any(file.explode(.), length(.scan.yara.matches) > 0) )` 2. _Specific YARA signature matches_ using `.name`: MQL `any(attachments, .file_extension in $file_extensions_common_archives and any(file.explode(.), any(.scan.yara.matches, .name == "YARA_SIGNATURE_NAME_HERE") ) )` 3. _Arbitrary [YARA metadata](https://yara.readthedocs.io/en/stable/writingrules.html#metadata) matches_ using `.meta`: _Any match from a specific author:_ MQL `any(attachments, .file_extension in $file_extensions_common_archives and any(file.explode(.), any(.scan.yara.matches, .meta['author'] == "YARA_META_AUTHOR_HERE") ) )` _Any match with a specific severity:_ MQL `any(attachments, .file_extension in $file_extensions_common_archives and any(file.explode(.), any(.scan.yara.matches, .meta['severity'] == "YARA_META_SEVERITY_HERE") ) )` When MQL rules flag, the specific YARA signatures that match will be visible in the Investigation page as a new Insight. In the future, we’ll make these visible in the Triage page as well. Testing YARA in Sublime [](https://docs.sublime.security/docs/yara#testing-yara-in-sublime) =============================================================================================== If you would like to test out YARA in your Sublime deployment, we’ve added a YARA test rule to our public Git repo. Enter the following details: 1. Create a new Feed using the following fields: 1. Feed name: `YARA Test Feed` 2. Git url: `https://github.com/sublime-security/sublime-rules` 3. Git branch: `main` 4. Leave MQL file filter blank 5. YARA file filter: `yara/*.yar` 2. Next, navigate to `Detection` in your left nav and select `Create` 1. Replace the stub rule with this MQL: MQL `type.inbound and any(attachments, .file_extension in $file_extensions_common_archives and any(file.explode(.), any(.scan.yara.matches, .name == "SublimeStandardTestString") ) )` 2. Select `Create Rule` in the top right 1. Activate immediately: `On` 2. Name: `YARA: Test Rule` 3. Type: `Detection` 4. Other fields: optional 3. Download our [benign test attachment](https://github.com/sublime-security/sublime-rules/files/10678536/sublime_yara_test.zip) _(A zip file containing a `test.txt` file with the string `Sublime-Standard-Test-String`)_ 4. Send the zipped content **from an unmonitored mailbox** (e.g. your personal mailbox) to a monitored mailbox (e.g. your work mailbox). Note that if you send the message from a monitored Sublime mailbox, the [message type](https://docs.sublime.security/docs/message-types) will be `type.internal`, not `type.inbound`, and the rule will not flag. 5. You should see the message flagged in your Sublime Dashboard! 🎉 ![](https://files.readme.io/7d5f391-yara_example_message_triage_page.png "yara_example_message_triage_page.png") Updated 3 months ago * * * --- # Email Alert with EML Attached Overview [](https://docs.sublime.security/docs/email-alert-with-eml-attached#overview) ------------------------------------------------------------------------------------------ This action extends the [Email Alert](https://docs.sublime.security/docs/email-alerts) action by additionally attaching the matched message to the email alert as a `.eml` file. It is especially useful for sending [user reports](https://docs.sublime.security/docs/user-reported-phishing) from Sublime to another system for further investigation (for example, your SOAR). See the [Email Alert page](https://docs.sublime.security/docs/email-alerts) for more information on the behavior and configuration of these alerts. Updated 12 months ago * * * --- # Email Alert Overview [](https://docs.sublime.security/docs/email-alerts#overview) ------------------------------------------------------------------------- This action sends an email alert when a new message group is flagged, using SMTP credentials provided during setup. Email alerts are useful as a cross-platform push notification of potential new threats in the email environment. Alert contents [](https://docs.sublime.security/docs/email-alerts#alert-contents) ------------------------------------------------------------------------------------- Email alerts are plaintext emails with the following format: `Subject: Mailbox: Recipients: Rules flagging: Actions triggered:` Any links present are defanged. How to create an email alert [](https://docs.sublime.security/docs/email-alerts#how-to-create-an-email-alert) ----------------------------------------------------------------------------------------------------------------- 1. Click "Actions" in the left nav of your Dashboard 2. Click "New action" in the top right 3. Select "Email alert" 4. Provide a name, recipients, sender email address, and related sender credentials for the alert 5. Click "Save" ### SMTP server configuration settings [](https://docs.sublime.security/docs/email-alerts#smtp-server-configuration-settings) _Any SMTP server/app password should work, but we've included server settings for common services below._ | Service | SMTP server | Port | | --- | --- | --- | | [Google Workspace / Gmail](https://support.google.com/mail/answer/7126229?hl=en#zippy=%2Cstep-check-that-imap-is-turned-on%2Cstep-change-smtp-other-settings-in-your-email-client) | smtp.gmail.com | 587 | | [Microsoft 365](https://support.microsoft.com/en-us/office/pop-imap-and-smtp-settings-8361e398-8af4-4e97-b147-6c6c4ac95353) | smtp.office365.com | 587 | ### SMTP user name [](https://docs.sublime.security/docs/email-alerts#smtp-user-name) For SMTP servers such as [Amazon Simple Email Service](https://aws.amazon.com/ses) that support authentication using a user name other than the sender email address, provide that user name in the optional **SMTP user name (optional)** field when creating your email alert action in Sublime. Updated 5 months ago * * * --- # Exclusions An Exclusion is a set of MQL logic used to avoid alerting on phishing simulations and other benign messages. Exclusions are evaluated **before** Detection Rules or Automations are run on a message. There's three kinds of Exclusions: 1. **Global Exclusion:** messages matching a Global Exclusion are not analyzed by **any** Detection Rules or Automations 2. **Detection Rule Exclusion:** messages matching a Detection Rule Exclusion are not analyzed by any Detection Rules (Automations will still process) 3. **Rule Exclusion:** looks for the unique combination of a specified Detection Rule or Automation and a sender, sender domain, or recipient. If the Rule Exclusion matches the message, it prevents the Rule from matching, thereby reducing false positives. ![](https://files.readme.io/a583540cdab0f993b89e35b7678fb8822d31fb75a1f6f32e5a5951a8168beaac-exclusions.gif) There are three out-of-the-box Global Exclusions for Cofense, KnowBe4, and Hoxhunt that are inactive by default. Exclusions are visible on impacted messages on the message list table and details page. You can view the Exclusion MQL on the message details page or head to the Exclusion details page. Updated 6 months ago * * * --- # Message Access Controls Every time a user views a message in Sublime, that view event is recorded in Sublime's immutable audit log. In addition, Sublime allows administrators to enforce certain requirements before a user can view a message's contents (plaintext body, HTML body, or attachments). These options allow you to enrich the paper trail for message access as you see fit. Changing the setting [](https://docs.sublime.security/docs/view-message-contents-requirements#changing-the-setting) ----------------------------------------------------------------------------------------------------------------------- To change this setting, go to **Admin** > **Account** > **View Message Contents Requirements**. Available settings [](https://docs.sublime.security/docs/view-message-contents-requirements#available-settings) ------------------------------------------------------------------------------------------------------------------- ### Show contents immediately [](https://docs.sublime.security/docs/view-message-contents-requirements#show-contents-immediately) This is the default setting. With this setting, message contents will load as soon as a user views a message's details. ### Require opt-in [](https://docs.sublime.security/docs/view-message-contents-requirements#require-opt-in) With this setting, users will need to click a button to reveal message contents when viewing a message's details. ### Require justification [](https://docs.sublime.security/docs/view-message-contents-requirements#require-justification) With this setting, users will need to provide a justification to reveal message contents when viewing a message's details. ![1630](https://files.readme.io/301d6be-justification.png "justification.png") Updated over 2 years ago * * * --- # Slack Alert Overview [](https://docs.sublime.security/docs/slack-alert#overview) ------------------------------------------------------------------------ This action sends a Slack alert when a new message group is flagged, using a Slack incoming webhook. Alert contents [](https://docs.sublime.security/docs/slack-alert#alert-contents) ------------------------------------------------------------------------------------ Slack alerts contain the following information: `Subject: Sender: Rules flagging: Actions triggered:` Any links present are defanged. How to create a Slack alert [](https://docs.sublime.security/docs/slack-alert#how-to-create-a-slack-alert) -------------------------------------------------------------------------------------------------------------- 1. Click "Actions" in the left nav of your Dashboard 2. Click "New action" in the top right 3. Select "Slack Alert" 4. Follow Slack's guide to [create an incoming webhook](https://api.slack.com/messaging/webhooks) or follow the steps below 5. Provide a name and the webhook URL created in the prior step 6. Click "Save" ### How to create a webhook URL using a new Slack app [](https://docs.sublime.security/docs/slack-alert#how-to-create-a-webhook-url-using-a-new-slack-app) 1. Visit Slack's guide to [create an incoming webhook](https://api.slack.com/messaging/webhooks) and click `Create your Slack app` ![2202](https://files.readme.io/69e03e4-Screen_Shot_2022-07-28_at_10.47.02_PM.png "Screen Shot 2022-07-28 at 10.47.02 PM.png") 2. Click "From scratch" ![1716](https://files.readme.io/500aa80-Screen_Shot_2022-07-28_at_10.47.17_PM.png "Screen Shot 2022-07-28 at 10.47.17 PM.png") 3. Add an app name and choose your Workspace ![1654](https://files.readme.io/793d4cc-Screen_Shot_2022-07-28_at_10.47.36_PM.png "Screen Shot 2022-07-28 at 10.47.36 PM.png") 4. Select "Incoming Webhooks" ![1498](https://files.readme.io/9e694f9-Screen_Shot_2022-07-28_at_10.47.51_PM.png "Screen Shot 2022-07-28 at 10.47.51 PM.png") 5. Toggle "Activate Incoming Webhooks" to **On**, scroll down, and click "Add New Webhook to Workspace" ![1490](https://files.readme.io/6176e60-Screen_Shot_2022-07-28_at_10.48.18_PM.png "Screen Shot 2022-07-28 at 10.48.18 PM.png") 6. Copy the generated Webhook URL, paste it into Sublime, and click "Save" ![2308](https://files.readme.io/a4929f4-Screen_Shot_2022-07-29_at_12.02.44_AM.png "Screen Shot 2022-07-29 at 12.02.44 AM.png") Updated about 3 years ago * * * --- # Configure the org_vips list Overview [](https://docs.sublime.security/docs/configure-org_vips-list#overview) ------------------------------------------------------------------------------------ The Sublime Platform provides a standard [list](https://docs.sublime.security/docs/lists) named `org_vips` that is useful for creating rules to detect VIP/Executive impersonations. [See all the Sublime Core Feed Rules that uses this list](https://sublime.security/labels/impersonation-vip/) . The `org_vips` list is backed by a Google Workspace group or a Microsoft 365 distribution list you specify, which should be a relatively small group or distribution list including VIPs in your organization who are especially likely to be impersonated by attackers. Members of this group are synced to Sublime daily, so that the list stays fresh. > 📘 > > Initial Sync > > > ------------------ > > The initial sync of Google Workspace groups or Microsoft 365 distribution lists can take up to a few hours. If you don't see any user groups available when following the setup instructions below, check back later. Setup [](https://docs.sublime.security/docs/configure-org_vips-list#setup) ------------------------------------------------------------------------------ To configure the `org_vips` list: 1. In the Sublime dashboard, click **Lists** in the navigation bar 2. Click the row for the `org_vips` list 3. Click the **Edit** button in the modal that appears 4. In the User Group section, search for the group or distribution list for your VIPs/executives 5. Select the matching group or list 6. Click **Save** 7. Click the row for the `org_vips` list again to view the list members synced to Sublime from Google Workspace or Microsoft 365 You're now able to use the `org_vips` list in your Sublime rules! MQL Example [](https://docs.sublime.security/docs/configure-org_vips-list#mql-example) ------------------------------------------------------------------------------------------ Here is an MQL snippet that checks if a message's sender matches the display name of any member of the `org_vips` list: Python `any($org_vips, .display_name == sender.display_name)` And here's a [Sublime Feed Rule that uses this list](https://github.com/sublime-security/sublime-rules/blob/main/detection-rules/vip_impersonation.yml) . Updated 3 months ago * * * --- # Message Query Language (MQL) Message Query Language (MQL) makes it easy for anyone to analyze email messages for threats such as phishing, BEC, and malware. MQL can also be used for threat hunting over historical mail. MQL, and the Sublime Platform as a whole, is designed to be email provider agnostic and thus completely shareable and interoperable. To see examples of MQL in action, check out the [Sublime Rules Feed](https://github.com/sublime-security/sublime-rules) or How-to MQL guides (in the left nav). > 👾 > > Hands-on Labs > > > ------------------- > > New to MQL? Apply the concepts in our documentation in the [Email Detection Engineering and Threat Hunting Labs](http://labs.sublime.security/) > . You will gain an understanding of how rule creation works and hunt for common email attacks. Updated 11 months ago * * * * [Syntax](https://docs.sublime.security/docs/syntax) * [Functions](https://docs.sublime.security/docs/functions) * [Enrichment functions](https://docs.sublime.security/docs/enrichment-functions) --- # Message Data Model (MDM) null Updated about 1 year ago * * * --- # ADÉ: Autonomous Detection Engineer ### **Overview** [](https://docs.sublime.security/docs/ade-autonomous-detection-engineer#overview) The Autonomous Detection Engineer (ADÉ) is an AI-powered detection generator that analyzes new and evolving attacks within your environment and automatically generates Detection Rules. ADÉ operates like a well-trained detection engineer, completing detailed analysis of the missed attack, identifying attack patterns and techniques, and reviewing your existing Detection Rules in order to create new detection coverage. ![](https://files.readme.io/5c7e0aca21dc11c7baa4b9c9f94c06bf57d22026b42f6accb40d83b84dfcfb90-ADE_example.png) No tickets, no vendor bottlenecks, and no rule writing. ### Getting access to ADÉ [](https://docs.sublime.security/docs/ade-autonomous-detection-engineer#getting-access-to-ad%C3%A9) ADÉ is currently in private beta for Enterprise users. Interested in getting access for your organization? [Join the waitlist!](https://www.notion.so/24204655fc9d80f1a946d68c857f45ce?pvs=21) Updated 12 days ago * * * --- # Actions Introduction [](https://docs.sublime.security/docs/actions#introduction) ============================================================================ A core part of the Sublime Platform is Actions. You can configure Actions to do something useful when a rule flags a message, or you can manually apply an Action to one or more messages. These actions are supported today: * [Webhooks](https://docs.sublime.security/docs/webhooks) * [Email alerts](https://docs.sublime.security/docs/email-alerts) * [Slack alerts](https://docs.sublime.security/docs/slack-alert) * [Trash](https://docs.sublime.security/docs/auto-trash) * [Quarantine](https://docs.sublime.security/docs/quarantine) (Enterprise) * [Warning Banners](https://docs.sublime.security/docs/warning-banners) (Enterprise) * [Move to Spam](https://docs.sublime.security/docs/move-to-spam) * [Auto-review](https://docs.sublime.security/docs/auto-review) These Actions are coming soon: * Create a Microsoft Teams alert * Send an email to message recipients * Auto-restore * Track link clicks Updated 12 months ago * * * * [Webhooks](https://docs.sublime.security/docs/webhooks) * [Email alerts](https://docs.sublime.security/docs/email-alerts) --- # Move to Spam Overview [](https://docs.sublime.security/docs/move-to-spam#overview) ========================================================================= Move to Spam is an Action that moves messages from the inbox to the spam or junk folder, depending on your email provider. Move to Spam is available for both Core and Enterprise users! Enterprise users often pair Move to Spam with a Warning Banner Action on Rules to automatically handle spam/graymail, while Core users often use Move to Spam to place potentially suspicious messages in an accessible "holding spot". Add `Move to Spam` to Rules for automated blocking [](https://docs.sublime.security/docs/move-to-spam#add-move-to-spam-to-rules-for-automated-blocking) ----------------------------------------------------------------------------------------------------------------------------------------------------------- 1. Click "Detection Rules" or “Automations” in the left nav of your Dashboard 2. Click on the Rule you wish to add the action to, opening the detail view for this Rule 3. Click "Edit" or "Edit Metadata" in the top right of this detail view 4. Click the "Actions" dropdown and select your Move to Spam Action 5. Click "Save Rule" Apply `Move to Spam` via API [](https://docs.sublime.security/docs/move-to-spam#apply-move-to-spam-via-api) --------------------------------------------------------------------------------------------------------------- 1. Leverage our [reviewMessageGroups API](https://docs.sublime.security/reference/reviewmessagegroups) 2. Pass in `move_to_spam` in the `actions` parameter 3. Fill out the remaining required fields 4. Make your API call 💻 Updated 6 months ago * * * --- # How to set up a custom domain If you've deployed the Sublime Platform via our AWS CloudFormation deployment, it's simple to set up a custom domain for your instance, such as sublime.example.com: 1. In the AWS console, navigate to the same region in which you deployed the Sublime Platform 2. [Request a public certificate](https://docs.aws.amazon.com/acm/latest/userguide/gs-acm-request-public.html) for the custom domain you want to use through AWS Certificate Manager 3. In the EC2 interface, locate the `Sublime-Platform-ALB` load balancer CloudFormation created for you and [add the certificate](https://docs.aws.amazon.com/elasticloadbalancing/latest/application/listener-update-certificates.html#add-certificates) you created to the load balancer's HTTPS listener 4. Create a CNAME DNS record pointing your custom domain to the load balancer's DNS name (which should start with `Sublime-Platform-ALB-`) 5. Copy the ARN for the certificate you set in step (4). Navigate to AWS Systems Manager and then Parameter Store. Select the `/sublime-security/self-signed-acm-certificate-for-sublime-platform` parameter and click Edit. Replace the current value with the ARN of your new certificate and Save Changes. That's it! Updated over 1 year ago * * * --- # Export Message MDMs Background [](https://docs.sublime.security/docs/export-message-mdms#background) ==================================================================================== Sublime Enterprise users on Cloud and AWS CloudFormation deployments can configure all Message Data Models (MDMs) to export to an S3 bucket that you own every 10 minutes. _Note: Enterprise users on Docker deployments can request assistance with exports to S3 in your Sublime support Slack channel._ 1\. Setup [](https://docs.sublime.security/docs/export-message-mdms#1-setup) ================================================================================ Before starting, you'll need to: 1. Determine the ID of the AWS account uploads will be coming from (used for `AWS_ACCOUNT_ID` placeholder below): * If you've deployed into a self-managed AWS environment, then the Account ID will be that of the AWS account. * If you are a Sublime Cloud user located in the EU, then the value you'll use is `128225141972`. * For all other Sublime Cloud users, then the value you'll use is `415365657389`. 2. Create an S3 bucket in the AWS account where you want the events exported to. We recommend creating a new S3 bucket specifically for these exports instead of re-using an existing one. 2\. Configure bucket permissions [](https://docs.sublime.security/docs/export-message-mdms#2-configure-bucket-permissions) ============================================================================================================================== These instructions assume you have a bucket with default settings and bucket policies. 1. Navigate to your desired bucket in the AWS S3 dashboard, and go to the Permissions tab. 2. Click Edit on the `Block public access (bucket settings)` section, which should bring you to the following page: ![](https://files.readme.io/bed4ae2-Screenshot_2024-04-19_at_1.54.53_PM.png) 3. Make sure the final checkbox is _deselected_. The other three should remain checked, unless you have a particular reason not to. Save these changes. 4. Now, scroll down to the `Bucket policy` section (still on the Permissions tab). This is where we'll set up the policy to allow cross-account uploads from the account Sublime is running in. Click Edit on the `Bucket Policy` tab. 5. Use the following template as the Policy. Update the placeholders `AWS_ACCOUNT_ID` (use the ID from Setup step 1 -- the AWS account where exports are coming from) and `BUCKET_NAME_HERE` with the corresponding values. Save changes once complete. . JSON `{ "Version": "2012-10-17", "Statement": [ { "Sid": "AllowSublimeWrites", "Effect": "Allow", "Principal": { "AWS": "arn:aws:iam::AWS_ACCOUNT_ID:root" }, "Action": [ "s3:PutObject*" ], "Resource": [ "arn:aws:s3:::BUCKET_NAME_HERE/*" ] } ] }` 6. The bucket is now configured for cross-account uploads from the Sublime platform. * * * 3\. Configure settings in Sublime [](https://docs.sublime.security/docs/export-message-mdms#3-configure-settings-in-sublime) ================================================================================================================================ To complete configuration, head over to `Account > Admin` in your Sublime dashboard and scroll down to the **Message Export** setting. Configure the following fields: * **S3 bucket name**: Bucket names must be globally unique. They can be named anything but here’s a sample format: `sublime-message-events-export-` * **S3 prefix** (optional): Configuring a prefix will store your exports in a folder with the prefix name within the bucket. A prefix is not necessary if the bucket is not used for any other use cases (recommended). Example prefix: `exports` * **S3 region**: the region you created the S3 bucket in ![](https://files.readme.io/d79cba3-export_all_messages.png) After hitting **Apply**, it may take up to 20 minutes before you start seeing events in S3. After the first export, exports take place every 10 minutes. S3 Export Details [](https://docs.sublime.security/docs/export-message-mdms#s3-export-details) ================================================================================================== File Names [](https://docs.sublime.security/docs/export-message-mdms#file-names) ------------------------------------------------------------------------------------ The filename (with prefixes) is output as follows: `ConfiguredKeyPrefix/YYYY/MM/DD/HH/mm/UnixTime_MessageGroupID_MessageID.json` Where each is as follows: `ConfiguredKeyPrefix`: The key prefix that was configured in your Sublime Platform settings. `YYYY`: Year `MM`: Month, zero-padded `DD`: Day, zero-padded `HH`: Hour, zero-padded `mm`: Minute, zero-padded `UnixTime`: Time message was received in Unix Time (Number of seconds elapsed since 1/1/1970 UTC) `MessageGroupID`: The ID of the message group that the message is in `MessageID`: The ID of the specific message stored in the file Content [](https://docs.sublime.security/docs/export-message-mdms#content) ------------------------------------------------------------------------------ The file is a JSON file that contains a top-level, single JSON object representing the MDM that was processed. The full MDM schema can be found [here](https://docs.sublime.security/docs/mdm) . Updated 12 months ago * * * --- # How to detect text in attachments Sublime uses recursive binary explosion and OCR to extract text from files. For more information on how this works, see the [`file.explode` documentation](https://docs.sublime.security/docs/enrichment-functions#fileexplode) . To simply find any string that was extracted out of any file, such as PDFs, images, and Office documents, use string searching functions like [strings.icontains](https://docs.sublime.security/docs/strings-module#contains--icontains) or [regex.icontains](https://docs.sublime.security/docs/regex-module#contains--icontains) on the output of the following property: `file.explode[].scan.strings.strings`. The example below detects the word "norton" in any PDF file: MQL `type.inbound and any(attachments, .file_extension == "pdf" and any(file.explode(.), any(.scan.strings.strings, strings.icontains(., "norton")) ) )` To find text that was extracted from a specific type of file or scanner, use the same string searching functions on the desired scanner output. Examples include `.docx`, `.html`, `.javascript`, `.ocr`, `.vba`, and `.xml`: MQL `type.inbound and any(attachments, .file_extension in~ ("doc", "docm", "docx", "dot", "dotm", "pptm", "ppsm", "xlm", "xls", "xlsb", "xlsm", "xlt", "xltm", "zip") and any(file.explode(.), strings.icontains(.scan.ocr.raw, "enable macros") ) )` > 🚧 > > A note on performance > > > --------------------------- > > The `file.explode` function is a relatively expensive operation, so you typically don't want to run it on _every_ file attachment. Instead, include a pre-filter to limit it to specific file types, such as archives, PDFs, HTML files, etc. > > One way to do this, as shown in the examples above, is by checking the `.file_extension` prior to calling `file.explode`. Testing your rule against a sample EML file [](https://docs.sublime.security/docs/how-to-detect-text-in-attachments#testing-your-rule-against-a-sample-eml-file) -------------------------------------------------------------------------------------------------------------------------------------------------------------------- The MQL rule editor can be used to display the output of the `file.explode` function so you can easily build detection rules for the values extracted. First write your MQL snippet, then click on the `file.explode` function, and then the `Evaluate` play button: ![](https://files.readme.io/2d8b77e-2022-03-27_13.37.13.gif "2022-03-27 13.37.13.gif") Updated 6 months ago * * * --- # How to use message header values in a rule Background [](https://docs.sublime.security/docs/how-to-use-message-header-values-in-a-rule#background) =========================================================================================================== Email headers are metadata within a raw message that contain information like the path the message took from source to destination, authentication results, originating mail client, and more. Often we can use information from within the headers to signature an attacker or attack type. To see how headers are parsed and normalized on the MDM, see the [Message Data Model reference](https://docs.sublime.security/docs/mdm) . Each mail server a message traverses is called a "mail hop." In the MDM, each hop is stored in the `headers.hops` list. Within each hop, Sublime further normalizes common header values into structured objects like `authentication_results`, `received_spf`, and more. ### Use Authentication-Results in a rule [](https://docs.sublime.security/docs/how-to-use-message-header-values-in-a-rule#use-authentication-results-in-a-rule) Header values like ‘Authentication-Results’ and ‘ARC-Authentication-Results’ are all normalized into the `authentication_results` MDM object within the `headers.hops` list. This MQL snippet will return `true` whenever there's a DMARC "fail": MQL `type.inbound and any(headers.hops, .authentication_results.dmarc =~ "fail")` ### Use raw header values in a rule [](https://docs.sublime.security/docs/how-to-use-message-header-values-in-a-rule#use-raw-header-values-in-a-rule) Raw header values are stored within each `hop` object on the MDM, with their order preserved. This MQL snippet will return true whenever there's a header field by the name (case insensitive) "Received-SPF" and the value containing "spf=temperror". MQL `type.inbound and any(headers.hops, any(.fields, .name =~ "Received-SPF") and strings.ilike(.value, "*spf=temperror*")))` Updated 8 months ago * * * --- # ASA: Autonomous Security Analyst Overview [](https://docs.sublime.security/docs/asa-autonomous-security-analyst#overview) -------------------------------------------------------------------------------------------- The Autonomous Security Analyst (ASA) is an AI-powered automation solution that analyzes and triages user-reported messages. ASA operates like a virtual security analyst, investigating reports to the abuse mailbox and providing detailed analysis to help security teams overcome any volume of user-reported messages. ASA addresses key pain points for security teams: * Security analysts overwhelmed by high volumes of user-reported messages can reduce the number of manual reviews required with ASA’s automated analysis. * Security managers struggling with allocating resources to noise and slow response times can automate the analysis, triage, remediation, and response using ASA’s auto-remediation and auto-review along with user report auto-replies. When enabled, ASA is invoked automatically after an end-user reports a message as an Automation with the trigger “user report received”. After being invoked, ASA completes analysis and comes up with a verdict for the message along with a full report. ASA uses the same Sublime tools as an analyst, such as * Analyzing detonated files and determining malicious content * Perform visual analysis on logos * Accessing sender profiles to determine prevalence and previous labels * Identifying potential misclassification of messages and why Getting started with ASA [](https://docs.sublime.security/docs/asa-autonomous-security-analyst#getting-started-with-asa) ---------------------------------------------------------------------------------------------------------------------------- To enable ASA in your environment, you just have to activate the “Send user reports to ASA” Automation. ![](https://files.readme.io/f88b9ebd1310f7b2efef7194022a8f13b3ec40f98ec88d0949a692e499699ca3-asa_automation_activation.gif) To leverage user report Automations, you must be an Enterprise customer with an abuse mailbox set. * To get a demo of Sublime Enterprise, reach out [here](https://sublime.security/demo/) ! * If you’re an Enterprise user but haven’t set up your abuse mailbox yet, check out [our documentation on enabling user-reported phishing](https://docs.sublime.security/docs/user-reported-phishing) on your account. * [Test out ASA in the free, public EML Analyzer by opening any sample message](https://analyzer.sublime.security/) ### Availability [](https://docs.sublime.security/docs/asa-autonomous-security-analyst#availability) ASA operates in our cloud environment (SaaS deployment) or your AWS cloud (self-hosted) at the moment. Data remains within the Sublime environment and is not shared with third parties including model providers. The following AWS regions are currently supported: * USA - Virginia * USA - Oregon * USA - Ohio * EU - Dublin * UK - London * AU - Australia To get on the waitlist for additional region support, reach out to [\[email protected\]](https://docs.sublime.security/cdn-cgi/l/email-protection#adc5c8c1c1c2edded8cfc1c4c0c883dec8ced8dfc4d9d4) ! ### Configurations [](https://docs.sublime.security/docs/asa-autonomous-security-analyst#configurations) ASA defaults to `inactive` mode and can then be set to `active` or `passive`: * `Active`: **“Autonomous Mode”** * In its `active` state, the ASA Automation will analyze, triage, and remediate user reported messages. * You can adjust the auto-remediations, auto-review status, and custom alerts for individual ASA verdicts to give ASA as much or as little autonomy as you’d like! * `Passive`: **“Analyst Mode”** * In its `passive` state, the ASA Automation will simply analyze and alert on user reported messages. * You can set auto-remediations and auto-review status, but the Automation cannot actually apply a remediation or auto-review. * Any custom alert Actions that are set are still applied so that you can receive alerts upon completion of the analysis. ### Verdict-based auto-remediation, auto-review, and custom alert Actions [](https://docs.sublime.security/docs/asa-autonomous-security-analyst#verdict-based-auto-remediation-auto-review-and-custom-alert-actions) ASA classifies messages into the following verdict categories: * `Malicious` * `Spam` * `Graymail` * `Benign` * `Unknown` Outside of `unknown`, all of ASA’s verdicts align 1:1 with a Sublime message classification option for easy message triage and review. `Unknown` verdicts are rare and indicate that a message needs human review. You’re able to select an auto-remediation approach for each of the verdicts, with an exception for `benign` verdicts. Options for auto-remediation include: * Quarantine * Trash * Move to spam * Add a warning banner Auto-review is available for the `malicious`, `spam`, `graymail`, and `benign` verdicts because there’s a corresponding review classification. Auto-review is unavailable for `unknown` verdicts that require human review. Custom alert Actions - webhooks, Slack alerts, and email alerts — can be applied on all ASA verdicts for easy integration with SIEM and SOAR workflows. ASA Report [](https://docs.sublime.security/docs/asa-autonomous-security-analyst#asa-report) ------------------------------------------------------------------------------------------------ ASA provides comprehensive analysis reports for each user-reported message, presenting information in a clear, accessible format. ### One-liner [](https://docs.sublime.security/docs/asa-autonomous-security-analyst#one-liner) A concise summary of the message’s key attributes that’s available on the message details. ![](https://files.readme.io/7d2c1f858075cd4f1d263f60ccdfd3b50b08178794ee3f5f8aa7ba41d51049fc-asa_one_liner_example.png) On the message details page, the one-liner also includes some message statistics: * The ASA verdict (unreviewed messages) * The message classification (reviewed messages) * Actions that have been applied * Messages count * User report count * First user report timing ### Executive Summary [](https://docs.sublime.security/docs/asa-autonomous-security-analyst#executive-summary) A short, detailed summary of message intent, content, links, and attachments with critical information to help analysts quickly make decisions on appropriate response to a given user report. ![](https://files.readme.io/d3cc348b79eea261f3bf6599f3041593ae8cac67ae26ae95f6912bf5123ab8d7-asa_executive_summary_example.png) ### Full report and reasoning [](https://docs.sublime.security/docs/asa-autonomous-security-analyst#full-report-and-reasoning) The full ASA analysis report and agent reasoning is available for in-depth investigations. The report is a full, detailed breakdown including: * Full attack chain analysis (for `malicious` verdicts) * Sender and content analysis * Details from detonated files and links * Contextual findings from sender profiles * Explanation of signals and Detection Rules that fired (or didn't fire) * An overall verdict explanation Use Cases [](https://docs.sublime.security/docs/asa-autonomous-security-analyst#use-cases) ---------------------------------------------------------------------------------------------- ### Expedited analyst decisions with ASA analysis [](https://docs.sublime.security/docs/asa-autonomous-security-analyst#expedited-analyst-decisions-with-asa-analysis) In `passive` ”Analyst Mode”, ASA provides enhanced context to security analysts without taking automatic remediation actions. ![](https://files.readme.io/4fc9a963c81171c3e20ec287e9bb6881117f3cef437f9583306bb04202ef0608-asa_full_report.gif) * ASA kicks off analysis when a message is reported * ASA’s analysis report is available when any analyst opens a user report (and coming soon, via API too!) * Analysts review ASA's one-liner and executive summary to make fast, informed decisions * Analysts check out ASA’s full report as-needed for in-depth investigations * User report mean time to respond (MTTR) is accelerated with ASA’s contextual analysis * Security teams maintain control over final remediation decisions and message classifications while reaping the benefits of ASA’s analysis. ### Autonomous reviews with an analyst escalation path [](https://docs.sublime.security/docs/asa-autonomous-security-analyst#autonomous-reviews-with-an-analyst-escalation-path) In `active` ”Autonomous Mode”, ASA functions as a fully autonomous security agent with the ability to automatically remediate and review user reports. ![](https://files.readme.io/8f213ef14e00bb0b3f07d409972383c13578f46cc7f2b8f0cda4901ae2b0f12a-asa_autotriaged_example.png) * ASA kicks off analysis when a message is reported and automatically remediates and classified messages based on verdict-level settings: * Dismisses benign and graymail false positive user reports * Quarantines malicious messages * Trashes/junks spam and graymail messages * Automatically classifies messages using the ASA verdict * Escalates unknown classifications to human analysts * Security teams only intervene when ASA determines human expertise is needed. * The Automation works 24/7 to process the queue of user reports, even when analysts are offline. This mode significantly reduces the workload on security teams while maintaining effective threat protection. Updated 7 days ago * * * --- # Using the MQL Editor Learning how to write new detection rules with MQL can be easy, with the smarts that are baked into the editor. It has builtin syntax highlighting, autocompletion, live error checking, and debugging so that you can write rules effectively and spot mistakes quickly. Once you start writing a rule or crafting a hunt, you may recognize a familiar look and feel if you've used the [Visual Studio Code](https://code.visualstudio.com/) editor before. Sublime uses the same core framework, tailored to improve the MQL experience. You don't have to have the Sublime editor to try out MQL! Navigate to [mql.new](https://mql.new/) to open the Sublime Sandbox. You can also upload an EML to test, and they are only stored if you generate a sharing link that includes the EML. Live completion and error checking [](https://docs.sublime.security/docs/using-the-mql-editor#live-completion-and-error-checking) ------------------------------------------------------------------------------------------------------------------------------------- As you type, you'll notice that fields from the [Message Data Model (MDM)](https://docs.sublime.security/docs/mdm) are automatically suggested and you can see the corresponding types. As function names, like `any` are typed, you can see the names and types of arguments it accepts, and what type of value it returns. When writing an expression inside an [array function](https://docs.sublime.security/docs/functions#array-functions) , the attributes for the current value are also autocompleted as soon as you type dot`.`. ![600](https://files.readme.io/ea80e7a-mql-autocompletion.gif "mql-autocompletion.gif") If you need a refresher on parts of the [MDM](https://docs.sublime.security/docs/mdm) , hover over a field to reveal its type and description. If it is limited in output, you'll see an additional line with `possible values: ...`, enumerating each possible value. ![600](https://files.readme.io/0fcd435-mdm-description.gif "mdm-description.gif") Fields that have a finite set of values are automatically completed, even within a string. Note that MQL doesn't enforce this restriction when a rule is created, but the editor does try to nudge you in the right direction. ![600](https://files.readme.io/03822b4-enum-example.gif "enum-example.gif") Testing and debugging rules [](https://docs.sublime.security/docs/using-the-mql-editor#testing-and-debugging-rules) ----------------------------------------------------------------------------------------------------------------------- When writing rules, it's valuable to get feedback ASAP. Instead of saving the rule and hoping it works after activating it, upload an EML file to test it against. When an EML is attached to the editor, additional functionality is available to test and debug a rule. When **Test Rule** is pressed, the matching parts of the rule (evaluate to `true`) are immediately highlighted and underlined. If the entire rule matches, a satisfying ✅ is displayed. If not then an ❗ will show. If you get an ❗ and aren't sure why, try running `Evaluate` to debug the intermediate output. Click on any function name, to see the 💡▶️ button to see the intermediate output in a JSON viewer. This is especially useful for functions with a lot of output, such as [enrichment functions](https://docs.sublime.security/docs/enrichment-functions) . ![600](https://files.readme.io/6f7e034-mql-test-rule.gif "mql-test-rule.gif") If you eval on a function that's inside another array function, such as `any` or `all`, the debug output will be an array, as if the function was evaluated with `map(source_array, function(.))`. This can help debug a partial match. Note in the screenshot, that `iregex_match` was highlighted because it matched _at least once_. However, the resulting `any` wasn't flagged. Evaluating `iregex_match` can identify which items matched. ![600](https://files.readme.io/7751950-inner-array-function.gif "inner-array-function.gif") Detecting logical bugs [](https://docs.sublime.security/docs/using-the-mql-editor#detecting-logical-bugs) ------------------------------------------------------------------------------------------------------------- Writing correct syntax and using functions and types correctly is half the battle to writing a strong detection. Sometimes, even the most clean looking rules can have the most dangerous of bugs: logical errors. The MQL editor tries to detect logical errors like using an `or` when you mean an `and`, or if you forgot parentheses. Sometimes, these bugs are easy to spot, but MQL isn't psychic and doesn't know what you _really_ meant. If one of these cases is detected, a warning is presented that there may be a logical error. These go away as soon as you add parentheses, adjust the indentation or switch to the expected operator. Warnings don't prevent MQL from compiling, but just show up when something looks like a mistake. More on this particular bug in [Common logical errors](https://docs.sublime.security/docs/syntax#common-logic-errors) . ![600](https://files.readme.io/4b95455-mql-logical-bug.gif "mql-logical-bug.gif") Stuck? [](https://docs.sublime.security/docs/using-the-mql-editor#stuck) ---------------------------------------------------------------------------- Unsure what functions and fields are at your disposal? At any time, you can view autocomplete suggestions with the shortcut Ctrl+Space. Since the MQL editor is built on top of the same core as VS Code, it has many of the same [default keyboard shortcuts](https://code.visualstudio.com/docs/getstarted/keybindings#_default-keyboard-shortcuts) . ![800](https://files.readme.io/3940f1b-mql-manual-suggestions.gif "mql-manual-suggestions.gif") Updated almost 3 years ago * * * --- # RegEx functions All regular expression functions below currently use `RE2` **(Golang flavor of RE2)** for the regular expression syntax. RE2 shares many features with PCRE, but is typically more efficient and does not include computationally expensive features of PCRE. For more details, see the [official documentation](https://github.com/google/re2/wiki/Syntax) for RE2. [RE2 regular expression tester](https://regex101.com/) > 📘 > > _Single quote `'` vs double quote `"`_ > > > -------------------------------------------- > > Use single quotes for regular expressions so that escape sequences such as `\d` don't have to be escaped twice. > > For instance, to find three consecutive digits anywhere in a string, use `regex_search(field, '\d\d\d')` to avoid double escapes of the `\\`. This is equivalent to `regex_search(field, "\\d\\d\\d")` `match` / `imatch` [](https://docs.sublime.security/docs/regex-module#match--imatch) ======================================================================================== `regex.match(input: string, pattern: regexp, ...) -> bool` `regex.imatch(input: string, pattern: regexp, ...) -> bool` **`match`** is used to match an \_entire string \_against a list of regular expressions. If at least one of the regular expressions matches the entire string, then `match` will return `true`. `match` performs a case-sensitive regular expression match, but `imatch` is case-insensitive. Both functions match against the entire string, so add leading and trailing wildcards (`.*` or `.*?`) to search for a substring within the entire `input` string. If `input` is `null`, then `match` and `imatch` will return null. ### Examples [](https://docs.sublime.security/docs/regex-module#examples) MQL `regex.match("[[email protected]](https://docs.sublime.security/cdn-cgi/l/email-protection) ", ".*[[email protected]](https://docs.sublime.security/cdn-cgi/l/email-protection) ") -> true # use imatch for case-insensitive matches regex.match("[[email protected]](https://docs.sublime.security/cdn-cgi/l/email-protection) ", "@sublimesecurity.com") -> false regex.imatch("[[email protected]](https://docs.sublime.security/cdn-cgi/l/email-protection) ", "@sublimesecurity.com") -> true # if multiple regular expressions are provided, only one needs to match regex.match("[[email protected]](https://docs.sublime.security/cdn-cgi/l/email-protection) ", "@.*.org$", "@.*.com$", "@.*.gov$") -> true` `contains` / `icontains` [](https://docs.sublime.security/docs/regex-module#contains--icontains) ==================================================================================================== `regex.contains(input: string, pattern: regexp, ...) -> bool` `regex.icontains(input: string, pattern: regexp, ...) -> bool` **`regex.contains`** is used to check if a string _contains_ a substring that has matches at least one of a list of regular expressions. Unlike `regex.match`, the full string does not need to match. `regex.contains(field, '\bfoo\b')` has the same behavior as `regex.match(field, '.*\bfoo\b.*')`. For case-insensitive regular expression matching, use `regex.icontains`. ### Examples [](https://docs.sublime.security/docs/regex-module#examples-1) MQL `regex.contains("[[email protected]](https://docs.sublime.security/cdn-cgi/l/email-protection) ", "@(google|sublimesecurity)") -> true # use icontains for case-insensitive substring regex.contains("[[email protected]](https://docs.sublime.security/cdn-cgi/l/email-protection) ", "@(google|sublimesecurity)") -> true # if multiple regular expressions are provided, only one needs to match regex.contains("[[email protected]](https://docs.sublime.security/cdn-cgi/l/email-protection) ", "@google", "@sublimesecurity") -> true` `count` / `icount` [](https://docs.sublime.security/docs/regex-module#count--icount) ======================================================================================== `regex.count(input: string, pattern: regexp) -> bool` `regex.icount(input: string, pattern: regexp) -> bool` **`regex.count`** is used to count the number of times `pattern` matches `input`. Matches are greedy. For example, `regex.count('hello', '.*')` will be 1. For case-insensitive regular expression matching, use `regex.icount`. ### Examples [](https://docs.sublime.security/docs/regex-module#examples-2) MQL `# Find large numbers of unusual characters regex.count(body.current_thread.text, '[^\x00-\x7F]') > 20 # Find multiple uses of excessive punctuation regex.count(body.current_thread.text, '[!?.]{2,}') > 3` `extract` / `iextract` [](https://docs.sublime.security/docs/regex-module#extract--iextract) ================================================================================================ `regex.extract(input: string, pattern: regexp) -> [RegexMatch]` `regex.iextract(input: string, pattern: regexp) -> [Regexmatch]` iframe **`regex.extract`** is used to return all regular expression matches within a string, including submatches for capture groups. This is similar to `regex.contains`, but instead of returning a boolean `true`/`false` for whether a match exists, it returns the complete match and the individual submatches. The returned fields for one of the matches: * `full_match`: Matches the complete regular expression, including all capture gorups * `groups`: A list of all the strings matched by capture groups. This will always be the same length as the number of capture groups, and individual captures are never `null` but `""`. * `named_groups`: A mapping of `string` -> `string` for capture groups with names. In RE2 syntax, this is done via `(?P.*)` like syntax, where `"my_capture"` will be one of the keys to the group. The resulting string, `.named_groups["my_capture"]` is the value matching that group. For case-insensitive regular expression extraction, use `regex.iextract`. ### Examples [](https://docs.sublime.security/docs/regex-module#examples-3) MQL `// With positional capture groups any(regex.iextract(sender.display_name, '\A(.*)\((?:via )?Google'), any(.groups, . in~ $org_display_names) ) // With named capture groups any(regex.iextract(sender.display_name, '\A(?P.*)\((?:via )?Google' ), .named_groups["sender_display_name"] in~ $org_display_names )` Updated 4 months ago * * * --- # How to detect lookalike domains Sublime Core Rules [](https://docs.sublime.security/docs/how-to-detect-lookalike-domains#sublime-core-rules) ================================================================================================================ [Sublime's Core Feed includes over 45 Rules that target lookalike domains](https://sublime.security/feeds/core/?page=1&tactic_or_technique=Lookalike%20domain&view=list) . These Rules are available on every account and can be located by heading to Feeds > Sublime Core Feed > Card View > Lookalike domain: ![](https://files.readme.io/e35ebe6ab4907064c827c3245430a74789d5fa4a10be149171414a0666396e17-lookalike_domains.gif) Background [](https://docs.sublime.security/docs/how-to-detect-lookalike-domains#background) ================================================================================================ Some phishing attacks involve a domain that is a very close 'look alike' to your own domain. For example, if your company's domain is acme.com, an adversary may send an email to one of your organization's executives from a domain such as 'acne.com', hoping that the executive does not notice the minor character substitution that represents a completely different DNS domain. See [here](https://en.wikipedia.org/wiki/IDN_homograph_attack) for more information about homoglyph attacks. Replace `example.com` with your domain, or any domain you want to protect. Consider increasing or decreasing the [levenshtein](https://docs.sublime.security/docs/strings-module#levenshtein--ilevenshtein) value if your domain is longer or shorter. ### Detect inbound messages where the sender domain is a typosquat of your domain [](https://docs.sublime.security/docs/how-to-detect-lookalike-domains#detect-inbound-messages-where-the-sender-domain-is-a-typosquat-of-your-domain) Will return true if the sender's root domain is a typosquat of the protected domain. So exanple.com or ecample.com would be flagged by this rule, but example.com or example12345.com would not be flagged. MQL `type.inbound and strings.levenshtein(sender.email.domain.root_domain, 'example.com') <= 2 and sender.email.domain.root_domain != 'example.com'` ### Detect inbound messages where the sender domain is a typosquat of one of several domains you are protecting [](https://docs.sublime.security/docs/how-to-detect-lookalike-domains#detect-inbound-messages-where-the-sender-domain-is-a-typosquat-of-one-of-several-domains-you-are-protecting) Some organizations may have multiple domains. You can write a rule to detect typosquats and protect as many different domains as you like. For example, let's say you have 3 domains: `example.com`, `example-europe.com`, and `example-finance.com`. The following rule would identify inbound emails that try to typosquat on any of those 3 domains. Will return true if the sender's root domain is a typosquat of the protected domain. MQL `type.inbound and strings.levenshtein(sender.email.domain.root_domain, 'example.com') <= 2 and strings.levenshtein(sender.email.domain.root_domain, 'example-europe.com') <= 2 and strings.levenshtein(sender.email.domain.root_domain, 'example-finance.com') <= 2 and sender.email.domain.root_domain not in ('example.com', 'example-europe.com', 'example-finance.com')` ### Detect inbound messages where a link in the body is a typosquat of a protected domain [](https://docs.sublime.security/docs/how-to-detect-lookalike-domains#detect-inbound-messages-where-a-link-in-the-body-is-a-typosquat-of-a-protected-domain) Will return true if any body link is a typosquat of the protected domain. MQL `type.inbound and any(body.links, strings.levenshtein(.href_url.domain.root_domain, 'example.com') <= 2) and sender.email.domain.root_domain != 'example.com'` Updated 9 months ago * * * --- # Missing or null values Within the [Message Data Model (MDM)](https://docs.sublime.security/docs/mdm) there are many optional fields. For example, `body.html` might be missing if an email message wasn't encoded with HTML. Within Message Query Language (MQL), `null` values are handled carefully and automatically to maintain consistent behavior. > 📘 > > Rule of thumb for how MQL handles missing/null values > > > ----------------------------------------------------------- > > If an expression can't be evaluated because one of the _required_ arguments is missing (`null`), then the result is also `null`. This also means that with `not` or `!=`, you want to take extra caution for the case where a field is `null` > > To see how the boolean operations `and`, `or`, and `not` are performed in the presence of `null`, see [boolean-logic-with-null](https://docs.sublime.security/docs/null-handling#boolean-logic-with-null) For example, a rule might consist of this logic `like(body.html.raw, "*cloudtracking.gif*)`. What happens if `body.html` is missing? What if `body.html.raw` is missing? This is easy to test by substituting `null` for the value. If a field is missing, like `body.html`, then any subfields will always be missing, like `body.html.raw`. To test the behavior ahead of time, substitute `null` in its place. results `like(null, "*cloudtracking.gif*") -> null # if body.html is missing: like(body.html.raw, "*cloudtracking.gif*") -> null` Comparisons to null values [](https://docs.sublime.security/docs/null-handling#comparisons-to-null-values) -------------------------------------------------------------------------------------------------------------- If either side of a comparison (e.g. ,`<`, `<=`, `==`, ...) is `null`, the full comparison will evaluate as `null`, even if both sides are null. results `# if body.html is missing: length(body.html.raw) > 4096 -> null null == null -> null null != null -> null` Test if a value is null/missing [](https://docs.sublime.security/docs/null-handling#test-if-a-value-is-nullmissing) ----------------------------------------------------------------------------------------------------------------------- To explicitly test if a value is missing, use `is null` or `is not null` syntax, which will always return `true` or `false` results `# if body.html is missing body.html is null -> true body.html is not null -> false # if body.html is present body.html is null -> false body.html is not null -> true` Boolean logic with null [](https://docs.sublime.security/docs/null-handling#boolean-logic-with-null) -------------------------------------------------------------------------------------------------------- A `null` boolean adds another level of logic to the boolean operators `and`, `or` and `not`. There are certain logical rules that must be maintained for consistency. For example, `not (x == y)` is the same as `x != y`. Similarly, `not (x < y)` is always equivalent to the expression `x >= y`. Because this is always true when the values are not `null`, it must remain true if a `null` is present. `null` is never implicitly converted to `false` when performing boolean logic. Let's see what happens if we explicitly check for `null` using `is not null` for the below expression. Note how it changes the result so that it no longer returns `null`. MQL `not (length(body.html.raw) > 4096)` body.html.raw is a long stringbody.html.raw is a short stringbody.html.raw is missing `not (length(body.html.raw) > 4096) => not (7000 > 4096) => not (true) => false not (body.html is not null and (length(body.html.raw) > 4096)) => not (true and (7000 > 4096)) => not (true and true) => not (true) => false` `not (length(body.html.raw) > 4096) => not (600 > 4096) => not (false) => true not (body.html is not null and (length(body.html.raw) > 4096)) => not (true and (600 > 4096)) => not (true and false) => not (false) => true` `not (length(body.html.raw) > 4096) => not (null > 4096) => not (null) => null not (body.html is not null and (length(body.html.raw) > 4096)) => not (false and null) => not (false and null) => not (false) => true` The truth tables for boolean operations are below. Both `and` and `or` are associative, meaning that `x and y` is equivalent to `y and x`. **x and y** | x | y | result | | --- | --- | --- | | true | false | false | | true | null | null | | null | null | null | | false | null | false | | false | false | false | **x or y** | x | y | result | | --- | --- | --- | | true | false | true | | true | null | true | | null | null | null | | false | null | null | | false | false | false | **not (x)** | x | result | | | --- | --- | --- | | true | false | | | null | null | | | false | true | | **x of (... N clauses ...)** The table from `of` is derived from the behavior of `and` and `or` and is fully compatible. That is, `2 of (a, b, c)` is always logically equivalent to `(a and b) or (a and c) or (b and c)`. | condition | result | | --- | --- | | `x` or more are `true` | true | | more than `n-x` are false | false | | otherwise | null | Updated over 2 years ago * * * --- # How to manage users with SCIM Introduction [](https://docs.sublime.security/docs/scim#introduction) ========================================================================= Sublime's [SCIM](https://en.wikipedia.org/wiki/System_for_Cross-domain_Identity_Management) API allows you to provision, update, and deprovision Sublime users from your identity provider (e.g., Okta), including managing each user's [Sublime role](https://docs.sublime.security/docs/role-based-access-control-rbac) . > 📘 > > Sublime and identity provider admin required > > > -------------------------------------------------- > > To integrate with SCIM, you'll need to be an administrator for both Sublime and your identity provider, or get the help of an admin for each. Below are the steps for integrating Okta with SCIM. If you need help integrating another identity provider with SCIM, [drop us a message](https://docs.sublime.security/cdn-cgi/l/email-protection#c5b6b0b5b5aab7b185b6b0a7a9aca8a0ebb6a0a6b0b7acb1bc) . Okta Setup [](https://docs.sublime.security/docs/scim#okta-setup) ===================================================================== Gather Sublime info [](https://docs.sublime.security/docs/scim#gather-sublime-info) --------------------------------------------------------------------------------------- To integrate Okta with SCIM, you'll need the following from the Sublime dashboard: 1. The SCIM API base URL for your Sublime instance: 1. In Sublime, navigate to the **API** page 2. Copy the **Base URL**. For example, Sublime Cloud's US region has the API base URL `https://platform.sublime.security` 3. Append `/v0/scim` to the base URL to construct a SCIM API base URL like `https://platform.sublime.security/v0/scim` 4. Hold onto this URL 2. An API key for a Sublime admin: 1. In Sublime, navigate to the **API** page 2. Click **New Key** 3. Enter a key name, such as "SCIM Integration" 4. Click **Save** 5. Copy and hold onto the new key Create a custom SAML app [](https://docs.sublime.security/docs/scim#create-a-custom-saml-app) ------------------------------------------------------------------------------------------------- Follow the instructions [here](https://docs.sublime.security/docs/single-sign-on#okta) to set up a [custom SAML app](https://help.okta.com/en-us/content/topics/apps/apps_app_integration_wizard_saml.htm) in Okta. You'll need an Okta SAML app to integrate Sublime's SCIM API with Okta. Add SCIM provisioning to the Okta app [](https://docs.sublime.security/docs/scim#add-scim-provisioning-to-the-okta-app) --------------------------------------------------------------------------------------------------------------------------- 1. Go to the overview page for the Okta application you created 2. Click the **General** tab 3. In the **App Settings** section, click **Edit** 4. Check **Enable SCIM provisioning** 5. Click **Save** 6. Click the **Provisioning** tab that appears 7. Click **Edit** 8. For **SCIM connector base URL**, enter `https://platform.sublime.security/v0/scim` 9. For **Unique identifier field for users**, enter `userName` 10. For **Supported provisioning actions**, check: 1. `Import New Users and Profile Updates` 2. `Push New Users` 3. `Push Profile Updates` 11. For **Authentication Mode**, select **HTTP Header** 12. For **Authorization**, enter the Sublime API key you created 13. Click **Test Connector Configuration** and confirm you get a **Connector configured successfully** message 14. Close the **Test Connector Configuration** modal 15. Click **Save** 16. Click the **To App** subtab 17. In the **Provisioning to App** section, click **Edit** 18. Check **Enable** for: 1. `Create Users` 2. `Update User Attributes` 3. `Deactivate Users` 19. Click **Save** 20. Under **Sublime Platform Attribute Mappings**, update the mappings as follows. You should only need to remove some of the default mappings. ![](https://files.readme.io/b284129-Screenshot_2024-06-19_at_12.16.30_AM.png) 21. Leave all other provisioning settings unchanged Map Okta groups to Sublime roles [](https://docs.sublime.security/docs/scim#map-okta-groups-to-sublime-roles) ----------------------------------------------------------------------------------------------------------------- 1. In Okta, go to **Directory** > **Profile Editor** 2. Under the **Users** tab, click your Sublime Okta app 3. Click **Add Attribute** 4. Add a new attribute with these properties: 1. **Data type**: `string` 2. **Display name**: `Sublime Role` 3. **Variable name**: `sublimeRole` 4. **External name**: `sublimeRole` 5. **External namespace**: `urn:ietf:params:scim:schemas:core:2.0:User` 6. **Enum**: checked, with these options: 1. `Admin`: `admin` 2. `Engineer`: `engineer` 3. `Analyst`: `analyst` 7. **Attribute length**: between 0 and 50 8. **Attribute required**: unchecked 9. **Attribute type**: `Group` ![](https://files.readme.io/7986ad8-Screenshot_2024-06-19_at_12.49.57_AM.png) 5. Click **Save** When assigning a group to the Okta app, you can now attach a Sublime role: ![](https://files.readme.io/dc180a0-Screenshot_2024-06-19_at_1.01.27_AM.png) Updated 12 months ago * * * --- # How to detect keywords or phrases in the body content of messages Background [](https://docs.sublime.security/docs/how-to-detect-keywords-or-phrases-in-the-body#background) ============================================================================================================== Email messages can contain two sections with text data that get rendered in an email client when a message is viewed by a user: * `text/html` (or just `html`) section * `text/plain` (or just `plain`) section As the section names imply, the `html` section can contain HTML mark-up such as hyperlinks, embedded images, text formatting, and more. The `plain` section does not render HTML, and is displayed raw. Email clients are typically configured to display the HTML section by default to the end-user, if it's present, and only display the `plain` section as a fall-back. Email messages are not required to use both of these sections. ### HTML content in MQL [](https://docs.sublime.security/docs/how-to-detect-keywords-or-phrases-in-the-body#html-content-in-mql) For HTML bodies, content is stored in two fields: * `body.html.raw` * `body.html.inner_text` The original HTML body is preserved in the `raw` field, and the internal decoded is stored in `inner_text`. Unless you need to match specific HTML elements, **it's best to prefer `body.html.inner_text`**. Example HTML body HTML ` This & that are in a hidden span.

Some paragraph content, before a table

Row 1, Column 1. Span contents inside R1C1 Row 1, Column 2
Row 2, Column 1 Row 2, Column 2
Copyright © 2022
` The inner text from the parsed HTML is much more compact. Note that newlines are automatically inserted between tags, regardless of whether they display on the same line visually. `This & that are in a hidden span. Some paragraph content, before a table Row 1, Column 1 Span contents inside R1C1 Row 1, Column 2 Row 2, Column 1 Row 2, Column 2 Copyright © 2022` > 🚧 > > When searching inside HTML contents > > > ----------------------------------------- > > Due to the size of HTML content, searching inside `body.html.raw` can be very time intensive. For better performance, consider writing rules that use `body.html.inner_text` instead, which contains the unescaped text inside the HTML, with different tags over different lines. The parsed HTML field, `body.html.inner_text`, is much smaller and is significantly faster to search. ### Plain content in MQL [](https://docs.sublime.security/docs/how-to-detect-keywords-or-phrases-in-the-body#plain-content-in-mql) For plain bodies, content is stored in `body.plain.raw`. Detect specific keywords [](https://docs.sublime.security/docs/how-to-detect-keywords-or-phrases-in-the-body#detect-specific-keywords) ========================================================================================================================================== We can search both text sections easily to detect specific keywords or phrases: MQL `type.inbound and any([body.plain.raw, body.html.inner_text], strings.ilike(., "*voicemail*", "*password reset*"))` Detect complex phrases [](https://docs.sublime.security/docs/how-to-detect-keywords-or-phrases-in-the-body#detect-complex-phrases) ====================================================================================================================================== We can use regular expressions if we're looking for something more complex, like a Social Security Number: MQL `type.inbound and any([body.plain.raw, body.html.inner_text], regex.contains(., '\b(\d\d\d)-(\d\d)-(\d\d\d\d)\b'))` Example detection rules [](https://docs.sublime.security/docs/how-to-detect-keywords-or-phrases-in-the-body#example-detection-rules) ======================================================================================================================================== View example rules for [body.html](https://github.com/sublime-security/sublime-rules/search?q=body.html) and [body.plain](https://github.com/sublime-security/sublime-rules/search?q=body.plain) . Updated 8 months ago * * * --- # Lists There are a variety of named lists that can be accessed via MQL. List names are always prepended with `$` when referenced in MQL. The lists below are included in all Sublime deployments. You can also create your own arbitrary lists. > 📘 > > How to use lists in rules > > > ------------------------------- > > See the [lists syntax](https://docs.sublime.security/docs/syntax#checking-a-named-list-with-in) > for usage instructions. Email tenant Lists [](https://docs.sublime.security/docs/lists#email-tenant-lists) ====================================================================================== Email tenant lists contain information from your upstream email provider(s) (Google Workspace or Microsoft 365). These lists are synced daily. ### Org domains [](https://docs.sublime.security/docs/lists#org-domains) `$org_domains` All domains associated with your organization’s message sources. Example usage: JavaScript `sender.email.domain.domain in $org_domains` ### Org display names [](https://docs.sublime.security/docs/lists#org-display-names) `$org_display_names` All display names of users in your organization with a valid mail license. Example usage: JavaScript `sender.display_name in $org_display_names` Google Group / Microsoft 365 Group Lists [](https://docs.sublime.security/docs/lists#google-group--microsoft-365-group-lists) --------------------------------------------------------------------------------------------------------------------------------- Groups are synced daily from your upstream tenant. You can create Lists that are backed by these groups, and kept up-to-date automatically, from within your Sublime dashboard. Historical context Lists [](https://docs.sublime.security/docs/lists#historical-context-lists) ================================================================================================== Historical context lists are created and managed by your Sublime instance and include historical communication information specific to your environment. These lists change over time as new messages are processed by the platform. When performing a backtest or a hunt with a list that’s powered by historical context, list entries will always be relative to the time of the message. Using these lists in a hunt or backtest such as `sender.email.email in $sender_emails`, preserves what was true when the message was first processed. ### Recipient domains [](https://docs.sublime.security/docs/lists#recipient-domains) `$recipient_domains` All domains anyone in your organization has sent an email to. This includes any domain that has appeared in the "to", "cc", and "bcc" fields of any outbound message. Example usage: JavaScript `sender.email.domain.domain not in $recipient_domains` ### Recipient emails [](https://docs.sublime.security/docs/lists#recipient-emails) `$recipient_emails` All email addresses anyone in your organization has sent an email to. This includes all email addresses that have appeared in the "to", "cc", and "bcc" fields of any outbound message. Example usage: JavaScript `sender.email.email not in $recipient_emails` ### Sender domains [](https://docs.sublime.security/docs/lists#sender-domains) `$sender_domains` All domains anyone in your organization has received an email from. Example usage: JavaScript `sender.email.domain.domain not in $sender_domains` ### Sender emails [](https://docs.sublime.security/docs/lists#sender-emails) `$sender_emails` All email addresses anyone in your organization has received an email from. Example usage: JavaScript `sender.email.email not in $sender_emails` Intelligence Lists [](https://docs.sublime.security/docs/lists#intelligence-lists) ====================================================================================== Intelligence lists are curated and maintained by the Sublime team. ### Abuse.ch URLHaus URLs: [](https://docs.sublime.security/docs/lists#abusech-urlhaus-urls) `$abuse_ch_urlhaus_urls_trusted_reporters` URLs consumed from URLHaus reports from trusted/top reporters JavaScript `any(body.links, .href_url.url in $abuse_ch_urlhaus_urls_trusted_reporters)` ### Abuse.ch URLHaus Domains: [](https://docs.sublime.security/docs/lists#abusech-urlhaus-domains) `$abuse_ch_urlhaus_domains_trusted_reporters` Domains consumed from URLHaus reports from trusted/top reporters JavaScript `any(body.links, .href_url.domain.domain in $abuse_ch_urlhaus_domains_trusted_reporters)` ### Abuse.ch MalwareBazaar SHA255 Hashes: [](https://docs.sublime.security/docs/lists#abusech-malwarebazaar-sha255-hashes) `$abuse_ch_malwarebazaar_sha256_trusted_reporters` SHA256 Hashes consumed from MalwareBazaar from trusted/top reporters. Javascript `any(attachments, .sha256 in $abuse_ch_malwarebazaar_sha256_trusted_reporters)` Note: Entries are limited to trusted reporters, but are otherwise unfiltered. We recommend using filtering logic in MQL to reduce potential false positives. Sublime Lists [](https://docs.sublime.security/docs/lists#sublime-lists) ============================================================================ Sublime lists are maintained by the Sublime team and are [open-source on GitHub](https://github.com/sublime-security/static-files) . ### Alexa Top 1 Million Domains [](https://docs.sublime.security/docs/lists#alexa-top-1-million-domains) `$alexa_1m` The top 1 million domains, as ranked by [Alexa](https://www.alexa.com/topsites) . Example usage: JavaScript `any(body.links, .href_url.domain.domain not in $alexa_1m)` ### Disposable Email Providers [](https://docs.sublime.security/docs/lists#disposable-email-providers) `$disposable_email_providers` Domains of disposable (or temporary) email providers that generate short-lived email addresses not owned by or attributable to any single user. Example usage: JavaScript `any(recipients.to, .email.domain.domain in $disposable_email_providers)` ### File extensions: common archives [](https://docs.sublime.security/docs/lists#file-extensions-common-archives) `$file_extensions_common_archives` Common archive file extensions, which can be used to deliver malicious files. Uncommon archive extensions are not included in this list because they can typically be blocked holistically. Example usage: JavaScript `any(attachments, .file_extension in~ $file_extensions_common_archives)` ### File extensions: macro-supported documents [](https://docs.sublime.security/docs/lists#file-extensions-macro-supported-documents) `$file_extensions_macros` File extensions of macro-supported documents, such as Microsoft Office files. Also includes file types that can load macros indirectly, eg via remote template injection. Example usage: JavaScript `any(attachments, .file_extension in~ $file_extensions_macros)` ### Free Email Providers [](https://docs.sublime.security/docs/lists#free-email-providers) `$free_email_providers` Domains of free email ("freemail") providers that allow anyone to create an email address. This is important to ensure the email address does not inherit the reputation of the freemail provider's domain. This list also includes the contents of `$disposable_email_providers`. Example usage: JavaScript `sender.email.domain.domain in $free_email_providers` ### Free File Hosts [](https://docs.sublime.security/docs/lists#free-file-hosts) `$free_file_hosts` Domains of free file hosting sites that allow anyone to upload and serve arbitrary files. Example usage: JavaScript `any(body.links, .href_url.domain.domain in $free_file_hosts)` ### Free Subdomain Hosts [](https://docs.sublime.security/docs/lists#free-subdomain-hosts) `$free_subdomain_hosts` Free subdomain sites that allow anyone to create their own subdomain and host arbitrary content. This is important to ensure the subdomains do not inherit the reputation of the root domain. Example usage: JavaScript `any(body.links, .href_url.domain.root_domain in $free_subdomain_hosts)` ### Majestic Million Domains [](https://docs.sublime.security/docs/lists#majestic-million-domains) `$majestic_million` The [Majestic Million](https://majestic.com/reports/majestic-million) is a collection of 1 million domains that have the most referring subnets. Example usage: JavaScript `any(body.links, .href_url.domain.domain not in $majestic_million)` ### Suspicious TLDs [](https://docs.sublime.security/docs/lists#suspicious-tlds) `$suspicious_tlds` Top-level domains that are either frequently abused, free to register, or otherwise not generally used in the normal course of business or email communication. Example usage: JavaScript `any(body.links, .href_url.domain.tld in $suspicious_tlds)` ### Tranco Top 1 Million Domains [](https://docs.sublime.security/docs/lists#tranco-top-1-million-domains) `$tranco_1m` The top 1 million domains found by [Tranco ranking](https://tranco-list.eu/) , a research-oriented top sites ranking hardened against manipulation. Example usage: JavaScript `any(body.links, .href_url.domain.domain not in $tranco_1m)` ### Cisco Umbrella Top 1 Million Domains [](https://docs.sublime.security/docs/lists#cisco-umbrella-top-1-million-domains) `$umbrella_1m` Top 1 million most popular domains as ranked by [Cisco Umbrella](https://umbrella.cisco.com/blog/cisco-umbrella-1-million) (based on passive DNS data). Example usage: JavaScript `any(body.links, .href_url.domain.domain not in $umbrella_1m)` ### Umbrella Top TLDs [](https://docs.sublime.security/docs/lists#umbrella-top-tlds) `$umbrella_1m_tld` The TLDs found in `$umbrella_1m`. Example usage: JavaScript `any(body.links, .href_url.domain.tld not in $umbrella_1m_tld)` ### URL Shorteners [](https://docs.sublime.security/docs/lists#url-shorteners) `$url_shorteners` Known URL shorteners that allow anyone to host arbitrary content. Example usage: JavaScript `any(body.links, .href_url.domain.root_domain in $url_shorteners)` API Reference [](https://docs.sublime.security/docs/lists#api-reference) ============================================================================ For automating the creation and updating of your own custom lists, see the [Lists API Reference](https://docs.sublime.security/reference/getlists-1) . Updated 4 months ago * * * --- # How to detect manual outbound forwards Overview [](https://docs.sublime.security/docs/how-to-detect-manual-outbound-forwards#overview) =================================================================================================== Sublime can dynamically detect end-user forwarding to personal email addresses in a variety of ways. Outbound forward to personal email address [](https://docs.sublime.security/docs/how-to-detect-manual-outbound-forwards#outbound-forward-to-personal-email-address) ======================================================================================================================================================================= Inspects outbound forwarded messages to free or disposable email providers with either: * An exact display name match * A similar display name match * An exact local part match * A similar local part match MQL `type.outbound and regex.imatch(subject.subject, "fw(d):.*") and any(recipients.to, ( // if the sender is [[email protected]](https://docs.sublime.security/cdn-cgi/l/email-protection) , this will match variations like: // - [[email protected]](https://docs.sublime.security/cdn-cgi/l/email-protection) // - [[email protected]](https://docs.sublime.security/cdn-cgi/l/email-protection) strings.levenshtein(.email.local_part, sender.email.local_part) <= 2 or strings.ilevenshtein(.display_name, sender.display_name) <= 2 ) and ( .email.domain.root_domain in $free_email_providers or .email.domain.root_domain in $disposable_email_providers ) )` Any outbound to personal email address [](https://docs.sublime.security/docs/how-to-detect-manual-outbound-forwards#any-outbound-to-personal-email-address) =============================================================================================================================================================== Inspects outbound messages to free or disposable email providers with either: * An exact display name match * A similar display name match * An exact local part match * A similar local part match MQL `type.outbound and any(recipients.to, ( // if the sender is [[email protected]](https://docs.sublime.security/cdn-cgi/l/email-protection) , this will match variations like: // - [[email protected]](https://docs.sublime.security/cdn-cgi/l/email-protection) // - [[email protected]](https://docs.sublime.security/cdn-cgi/l/email-protection) strings.levenshtein(.email.local_part, sender.email.local_part) <= 2 or strings.ilevenshtein(.display_name, sender.display_name) <= 2 ) and ( .email.domain.root_domain in $free_email_providers or .email.domain.root_domain in $disposable_email_providers ) )` Outbound forward to personal email address with \[sensitive data|attachment\] [](https://docs.sublime.security/docs/how-to-detect-manual-outbound-forwards#outbound-forward-to-personal-email-address-with-sensitive-dataattachment) ======================================================================================================================================================================================================================================== > 📘 > > Customize this rule > > > ------------------------- > > You may customize the criteria you deem sensitive within the rule logic below. Inspects outbound messages to free or disposable email providers with the same criteria as above or any of: * Any attachments in [$file\_extensions\_common\_archives](https://docs.sublimesecurity.com/docs/lists#file-extensions-common-archives) * Strings that may convey sensitive content (eg. "_password_", "_sensitive_", "_protected_", "_confidential_") * Natural Language understanding tags in the body, or a PDF attachment: | Name | Description | | --- | --- | | `invoice` | These emails contain language about viewing invoices via links or attachments. | | `payment` | These emails contain language about ACH, EFT, or Wire payments. | | `purchase_order` | These emails contain language about Purchase Orders, Requests for Quotation. | MQL `type.outbound and regex.imatch(subject.subject, "fw(d):.*") and any(recipients.to, ( // if the sender is [[email protected]](https://docs.sublime.security/cdn-cgi/l/email-protection) , this will match variations like: // - [[email protected]](https://docs.sublime.security/cdn-cgi/l/email-protection) // - [[email protected]](https://docs.sublime.security/cdn-cgi/l/email-protection) strings.levenshtein(.email.local_part, sender.email.local_part) <= 2 or strings.ilevenshtein(.display_name, sender.display_name) <= 2 ) and ( .email.domain.root_domain in $free_email_providers or .email.domain.root_domain in $disposable_email_providers ) ) // add any sensitive data indicators below and ( any(ml.nlu_classifier(body.current_thread.text).tags, .name in ("invoice", "payment", "purchase_order") ) or any(attachments, .file_extension in $file_extensions_common_archives or strings.ilike(body.current_thread.text, "*password*", "*sensitive*", "*protected*", "*confidential*" ) or ( .file_type == "pdf" and any(file.explode(.), any(ml.nlu_classifier(.scan.ocr.raw).tags, .name in ("invoice", "payment", "purchase_order") ) ) ) ) )` Updated 5 months ago * * * --- # Role-Based Access Control (RBAC) Sublime's built-in and custom roles make it easy to give your team the perfect amount of access. Managing Roles [](https://docs.sublime.security/docs/role-based-access-control-rbac#managing-roles) ------------------------------------------------------------------------------------------------------- When you create a new user from the **Admin** > **Account** page, you'll have the option to select a role. To change an existing user's role, select the user in the Users table, select **Actions** > **Edit**, and update the user's role. ![](https://files.readme.io/547b8e2b0cfb8edaf5100f4481b7a8d93df1c828b710d874b74402b8bc0cc56a-manage_roles.gif) Roles [](https://docs.sublime.security/docs/role-based-access-control-rbac#roles) ------------------------------------------------------------------------------------- | Role | Description | | --- | --- | | `Admin` | All Sublime features, including the ability to create custom roles. | | `Engineer` | Build detection rules, Backtest, Hunt, investigate and remediate flagged and user reported messages, view system error notifications, and more. | | `Analyst` | Investigate and remediate flagged and user reported messages, view system error notifications. Cannot create or modify rules, Search, Backtest, or Hunt. | | `Custom` | Anyone with the Admin role can create a custom role with granular permissions. | The below table enumerates the Sublime Platform permissions and denotes which role contains it. A green check ✅ denotes that the permission is included in the role. A green check with an asterisk ✅\* denotes that the role has the listed permission, but can only operate on resources that the current user has created themselves (e.g. an `Engineer` can only read API keys that were created by themselves, whereas an `Admin` can read API keys regardless of which user they were created by). | Category | Permission | Admin | Engineer | Analyst | | --- | --- | --- | --- | --- | | Audit Log | `manage_audit_log` | ✅ | | | | Audit Log | `read_audit_log` | ✅ | | | | IP Allowlist | `manage_ip_allowlist` | ✅ | | | | IP Allowlist | `read_ip_allowlist` | ✅ | ✅ | ✅ | | MDM Retention | `update_full_message_retention` | ✅ | | | | Mailbox Auto-Activation | `read_auto_activate` | ✅ | ✅ | ✅ | | Mailbox Auto-Activation | `update_auto_activate` | ✅ | | | | Abuse Mailbox Settings | `manage_abuse_mailbox` | ✅ | ✅ | | | Abuse Mailbox Settings | `read_abuse_mailbox` | ✅ | ✅ | ✅ | | API Keys | `create_api_keys` | ✅ | ✅\* | ✅\* | | API Keys | `read_api_keys` | ✅ | ✅\* | ✅\* | | API Keys | `delete_api_keys` | ✅ | ✅\* | ✅\* | | API Keys | `set_external_api_keys` | ✅ | ✅\* | | | API Keys | `read_external_api_keys` | ✅ | ✅\* | | | Users | `create_users` | ✅ | | | | Users | `read_users` | ✅ | ✅ | ✅ | | Users | `update_users` | ✅ | | | | Users | `delete_users` | ✅ | | | | Users | `invite_users` | ✅ | | | | Message Sources | `create_message_sources` | ✅ | | | | Message Sources | `read_message_sources` | ✅ | ✅ | ✅ | | Message Sources | `read_message_sources_inline_message_meta` | ✅ | | | | Message Sources | `update_message_sources` | ✅ | | | | Message Sources | `delete_message_sources` | ✅ | | | | Mailboxes | `read_mailboxes` | ✅ | ✅ | ✅ | | Mailboxes | `activate_mailbox` | ✅ | | | | Mailboxes | `deactivate_mailbox` | ✅ | | | | Rules | `create_rules` | ✅ | ✅ | | | Rules | `read_rules` | ✅ | ✅ | ✅ | | Rules | `update_rules` | ✅ | ✅ | | | Rules | `delete_rules` | ✅ | ✅ | | | Lists | `create_lists` | ✅ | ✅ | | | Lists | `read_lists` | ✅ | ✅ | ✅ | | Lists | `update_lists` | ✅ | ✅ | ✅ | | Lists | `delete_lists` | ✅ | ✅ | | | Actions | `create_actions` | ✅ | ✅ | | | Actions | `read_actions` | ✅ | ✅ | ✅ | | Actions | `update_actions` | ✅ | ✅ | | | Actions | `delete_actions` | ✅ | ✅ | | | Actions | `associate_rules_to_actions` | ✅ | ✅ | | | Feeds | `create_feeds` | ✅ | ✅ | | | Feeds | `read_feeds` | ✅ | ✅ | ✅ | | Feeds | `update_feeds` | ✅ | ✅ | | | Feeds | `delete_feeds` | ✅ | ✅ | | | Backtest | `backtest` | ✅ | ✅ | | | Hunt | `hunt` | ✅ | ✅ | | | Hunt | `private_hunt` | ✅ | | | | Search | `search` | ✅ | ✅ | | | Investigation | `access_all_message_contents` | ✅ | ✅ | ✅ | | Investigation | `access_flagged_message_contents` | ✅ | ✅ | ✅ | | Investigation | `access_user_reported_message_contents` | ✅ | ✅ | ✅ | | Remediation | `perform_actions` | ✅ | ✅ | ✅ | | Error Log | `access_error_logs` | ✅ | ✅ | ✅ | | URL Defense | `read_url_defense_settings` | ✅ | | | | URL Defense | `manage_url_defense_settings` | ✅ | | | | Content viewing setting | `manage_view_contents_requirements` | ✅ | | | | OIDC setting | `manage_oidc` | ✅ | | | | OIDC setting | `read_oidc` | ✅ | ✅ | ✅ | | SCIM | `read_scim_resources` | ✅ | ✅ | ✅ | | Provider group | `read_provider_groups` | ✅ | ✅ | ✅ | | Provider group | `update_provider_groups` | ✅ | ✅ | ✅ | | Exclusions | `create_exclusions` | ✅ | | | | Exclusions | `read_exclusions` | ✅ | ✅ | ✅ | | Exclusions | `update_exclusions` | ✅ | | | | Exclusions | `delete_exclusions` | ✅ | | | | Historical ingestion | `start_historical_ingestion_jobs` | ✅ | | | | Historical ingestion | `abandon_historical_ingestion_jobs` | ✅ | | | | Historical ingestion | `read_historical_ingestion_jobs` | ✅ | ✅ | ✅ | | Roles | `read_roles` | ✅ | ✅ | ✅ | | Telemetry | `manage_telemetry` | ✅ | | | | Deletion | `delete_org` | ✅ | | | | Polling setting | `manage_message_polling` | ✅ | | | | Overview Report | `read_org_stats` | ✅ | ✅ | ✅ | | Overview Report | `read_remediation_stats` | ✅ | ✅ | ✅ | | Link clicks | `read_link_clicks` | ✅ | | | | S3 export | `manage_message_export` | ✅ | | | | Setup guide | `manage_account_setup_guide` | ✅ | | | | Message creation | `create_new_message` | ✅ | ✅ | ✅ | ### Custom roles [](https://docs.sublime.security/docs/role-based-access-control-rbac#custom-roles) You can create an unlimited number of custom roles within your organization by heading to **Admin** > **Account** and clicking **Create Role**. ![](https://files.readme.io/bf6a13b896743b5cc95c67de8cc0d18662a5a5d0a61c4a12b2dd163f6d47e32c-custom_rbac.gif) #### Multi-tenant custom roles [](https://docs.sublime.security/docs/role-based-access-control-rbac#multi-tenant-custom-roles) For multi-tenant instances, custom roles can only be created or managed by the parent org. While Admins can assign custom roles to team members on any child org, they can't create new roles or edit the permissions on existing roles via the child org. Team members who have access to the parent org will inherit their existing role when they log into the child org for the first time. Admins can change their role afterwards if they want that team member to have a different role on a specific child org. All custom roles created by the parent org are available to all of the child orgs, so we recommend that custom role names **don't** include child org information, such as business names. You can alternatively limit anyone with a custom role from seeing available roles by withholding the `read_roles` permission. Updated 2 months ago * * * --- # Functions MQL has several built-in functions for capabilities that don't neatly fit into concise syntax. Functions can operate on a variable amount of parameters and always evaluate to a value. The basic syntax for functions is `(, , ... )`. > 📘 > > Request a function! > > > ------------------------- > > Don't see a function you want? Let us know via [email](https://docs.sublime.security/cdn-cgi/l/email-protection#f29a979e9e9db28187909e9b9f97dc81979187809b868b) > or Slack! Top level functions [](https://docs.sublime.security/docs/functions#top-level-functions) ============================================================================================ `coalesce` [](https://docs.sublime.security/docs/functions#coalesce) ------------------------------------------------------------------------ `coalesce(values: boolean, ...) -> boolean` `coalesce(values: number, ...) -> number` `coalesce(values: string, ...) -> string` Use **`coalesce`** to handle `null` values, and return the first non-null value. If all arguments are `null` then the first non-`null` value is returned. The `coalesce` function can be used with any type of scalar values, such as a `boolean`, `number`, or a `string`. MQL `# prefer body.html.display_text if it's not null, but fallback to body.plain.raw coalesce(body.html.display_text, body.plain.raw)` `length` (string) [](https://docs.sublime.security/docs/functions#length-string) ------------------------------------------------------------------------------------ `length(input: string) -> integer` Use **`length`** to retrieve the number of Unicode code points used to encode a string in UTF-8. Note that this is not equivalent to the number of bytes or glyphs. MQL `length("sublimesecurity.com") -> 19 length("📧") -> 1 length("émaîl") -> 5` `length` (array) [](https://docs.sublime.security/docs/functions#length-array) ---------------------------------------------------------------------------------- `length(input: [T]) -> integer` Use **`length`** to retrieve the length of an array. If the input array is `null`, the function returns `null`. If the input array is non-null but empty, the function returns `0`. MQL `length(attachments) -> 3 length(filter(attachments, .file_extension == "xls")) -> 2` > 📘 > > Unicode and strings > > > ------------------------- > > Characters in strings are a complicated matter. Within MQL, all strings are always valid unicode and internally encoded as UTF-8. > > Functions within MQL typically operate on unicode strings, and a "character" means a single UTF-8 code point, which can span more than one byte and may be an incomplete glyph. Array looping functions [](https://docs.sublime.security/docs/functions#array-looping-functions) ==================================================================================================== Arrays could come from various places: on the MDM, such as `recipients.to` or `attachments`; [inline](https://docs.sublime.security/docs/syntax#creating-arrays) with `[`...`]` syntax, such as `[body.html.raw, body.plain.raw]`; or returned from functions, such as [`file.explode`](https://docs.sublime.security/docs/enrichment-functions#fileexplode) . To process an array and loop over elements, MQL uses a few builtin functions. Array functions loop over an input array with an expression and return another array or some other value. Array functions typically take two arguments: the source array, and an expression to evaluate over the elements. Within the looping expression, use the prefix `.` to access the loop item. To access an attribute of the looping item, use `.`. To access fields on the root level of the MDM, drop the prefix. You can access a loop item from a parent scope with an additional `.`, such as `..`. For example, to check if an array of strings contains the string `secret`: MQL `any(["super", "secret", "text"], . == "secret")` To check the `body.links` array that the attribute `.mismatched` is _always_ `false`: MQL `all(body.links, .mismatched == false)` To check if either `body.plain.raw` or `body.html.raw` to contain any of the strings `"urgent"`, `"promptly"`, or `"asap"`, use an array to avoid repeating logic: MQL `any([body.plain.raw, body.html.raw], ilike(., "*urgent*", "*promptly*", "*asap*"))` Example MDM for vector functions `{ "attachments": [ { "file_extension": "xls", ... }, { "file_extension": "docx", ... }, { "file_extension": "xls", ... } ], ... }` `all` [](https://docs.sublime.security/docs/functions#all) -------------------------------------------------------------- `all(input: [T], expression: bool) -> bool` **`all`** returns `true` if all of the values in the original array match an expression, which is provided by the second argument. If the array is empty, then `all` is vacuously `true`. ### Example usage [](https://docs.sublime.security/docs/functions#example-usage) results `all(attachments, .file_extension == "xls") -> false all(attachments, .file_extension in ("xls", "docx")) -> true` **Tip**: Use `all` if you want to perform an `and` across the array, and use `any` to perform an `or`. `any` [](https://docs.sublime.security/docs/functions#any) -------------------------------------------------------------- `any(input: [T], expression: bool) -> bool` **`any`** is the counterpart of `all`. Instead of checking for the expression to evaluate `true` for _all_ items, `any` will check if at least one value returns `true`. results `any(attachments, .file_extension == "xls") -> true any(attachments, .file_extension in ("xls", "docx")) -> true` **Tip**: Use `all` if you want to perform an `and` across the array, and use `any` to perform an `or`. `distinct` [](https://docs.sublime.security/docs/functions#distinct) ------------------------------------------------------------------------ `distinct(array: [T]) -> [T]` `distinct(array: [T], key) -> [T]` **`distinct`** filters an array, only returning one match per key. MQL `distinct(attachments, .file_extension) -> [{"file_extension": "xls"}, {"file_extension": "docx"}]` One common use case is to combine it with `length(array)`, to get a distinct count an array. MQL `# how many unique file extensions are there? length(distinct(attachments, .file_extension)) -> 2` `distinct` can also operate without a key if it's evaluated over an array of scalar values `filter` [](https://docs.sublime.security/docs/functions#filter) -------------------------------------------------------------------- `filter(array: [T], expression: bool) -> [T]` **`filter`** is used to generate a new filtered array, only returning items that cause the loop expression to evaluate `true`. results `filter(attachments, .file_extension != "xls") -> [{"file_extension": "docx"}]` `map` [](https://docs.sublime.security/docs/functions#map) -------------------------------------------------------------- `map(array: [T], expression: U) -> [U]` **`map`** is used to generate a new array, by evaluating an expression for each item in the array. results `map(attachments, length(.file_extension)) -> [3, 4, 3]` **`map`** is particularly useful when querying an MDM: results `map(attachments, .file_name) -> ["unpaid invoice.xlsx", "unpaid invoice.rar"]` `sum` [](https://docs.sublime.security/docs/functions#sum) -------------------------------------------------------------- `sum(array: [number]) -> number` `sum(array: [T], expression: number) -> number` **`sum`** returns an aggregate count over all items in an array. results `sum(attachments, .file_size)` **`sum`** doesn't need an expression if the array already consists of numbers. results `sum([1, 2, 3, 4]) -> 10` `ratio` [](https://docs.sublime.security/docs/functions#ratio) ------------------------------------------------------------------ `ratio(array: [T], expression: bool) -> number` **`ratio`** calculates the ratio between items of the array that cause the loop expression to evaluate to `true` and the total number of items in the array. If the array is empty, `ratio` will return `null`. MQL `ratio(attachments, .file_extension in ("xls", "docx")) -> 0.5 ratio([], . == "xls") -> null` `null` values are allowed in the input array. MQL `ratio([1, 2, null, null], . > 0) -> 0.5` Updated 11 months ago * * * --- # Quarantine Overview [](https://docs.sublime.security/docs/quarantine#overview) ----------------------------------------------------------------------- Quarantine is an Enterprise Action that makes messages inaccessible to mailbox end users, thus preventing interaction with malicious message bodies, links, or attachments. In Microsoft 365 environments, quarantined messages are pulled from the inbox and placed in the "Recoverable Items Purges" directory. Mailbox users are not able to access the message in this directory. > 📘 > > Note > > > ---------- > > By default, admins are able to restore Microsoft 365 messages from Quarantine for 14 days after the message was received. This retention period is configurable in Exchange Admin. See the [official Microsoft documentation](https://learn.microsoft.com/en-us/exchange/security-and-compliance/recoverable-items-folder/recoverable-items-folder#deleted-item-retention) > for more details. In Google Workspace environments, quarantined messages are deleted and are completely inaccessible to the mailbox owner, but can still be restored by Sublime admins. Add Quarantine to Rules for automated blocking [](https://docs.sublime.security/docs/quarantine#add-quarantine-to-rules-for-automated-blocking) --------------------------------------------------------------------------------------------------------------------------------------------------- 1. Click "Detection Rules" or “Automations” in the left nav of your Dashboard 2. Click on the Rule you wish to add the action to, opening the detail view for this Rule 3. Click "Edit" or "Edit Metadata" in the top right of this detail view 4. Click the "Actions" dropdown and select your Quarantine Action 5. Click "Save Rule" Manually quarantine a message [](https://docs.sublime.security/docs/quarantine#manually-quarantine-a-message) ----------------------------------------------------------------------------------------------------------------- 1. While viewing the message on the Dashboard, select the "Quarantine" option in the review flow. 2. Optionally add a custom Action, review label, comment, and the Share with Sublime check. 3. Submit your review! ![](https://files.readme.io/ffd5a84-quarantine_workflow.png) _Note: You can quarantine a message using the Quarantine API instead! Check out how [here](https://docs.sublime.security/reference/quarantinemessagecanonicalgroup-1) ._ Restore a quarantined message [](https://docs.sublime.security/docs/quarantine#restore-a-quarantined-message) ----------------------------------------------------------------------------------------------------------------- 1. While viewing the message on the Dashboard, select the "Restore from Quarantine" option in the review flow. 2. Optionally add a custom Action, review label, comment, and the Share with Sublime check. 3. Submit your review! _Note: You can restore a quarantined a message using the Restore API instead! Check out how [here](https://docs.sublime.security/reference/restoremessagecanonicalgroup-1) ._ Updated 9 months ago * * * --- # How to set up single sign-on (SSO) Introduction [](https://docs.sublime.security/docs/single-sign-on#introduction) =================================================================================== The Sublime Platform supports single sign-on SSO through both SAML (Security Assertion Markup Language) and OIDC (OpenID Connect), allowing you to authenticate with Sublime via Okta, OneLogin, Azure Active Directory, or any other SAML or OIDC provider. > 📘 > > Managing users with SCIM > > > ------------------------------ > > Sublime also supports the SCIM (System for Cross-domain Identity Management) standard, which makes it easy to provision, update, and de-provision users from your identity provider. Learn more [here](https://docs.sublime.security/docs/scim) > . Integrating with SAML [](https://docs.sublime.security/docs/single-sign-on#integrating-with-saml) ===================================================================================================== Below are the steps for setting up SAML with Okta. If you need help setting up SAML for another identity provider, [drop us a message](https://docs.sublime.security/cdn-cgi/l/email-protection#8cfff9fcfce3fef8ccfff9eee0e5e1e9a2ffe9eff9fee5f8f5) . Okta [](https://docs.sublime.security/docs/single-sign-on#okta) ------------------------------------------------------------------- ### Get your Sublime SAML signon URL [](https://docs.sublime.security/docs/single-sign-on#get-your-sublime-saml-signon-url) 1. Sign into Sublime 2. Go to **Admin** > **Account** 3. Under **Authentication**, click the button next to **SAML** 4. Copy and hold onto the URL under **Single sign-on URL** ### Create a custom SAML app [](https://docs.sublime.security/docs/single-sign-on#create-a-custom-saml-app) 1. Sign into your Okta admin console 2. Go to **Applications** > **Applications** 3. Click **Create App Integration** 4. In the modal that opens, select **Sign-in method** of **SAML 2.0** 5. Click **Next** 6. Provide an **App name**, such as "Sublime Platform" 7. Optionally add a logo. You can download the Sublime logo [here](https://user-images.githubusercontent.com/11003450/115128085-5805da00-9fa9-11eb-8c7a-dc8b708053ee.png) . 8. Click **Next** 9. For **Single sign-on URL**, paste the SAML signon URL you copied from the Sublime dashboard 10. Be sure **Use this for Recipient URL and Destination URL** is checked 11. For **Audience URI (SP Entity ID)**, also paste the SAML signon URL you copied from the Sublime dashboard 12. For **Name ID format**, select **Email Address** 13. For **Update application username on**, select **Create and update** 14. Leave all other settings as their defaults 15. Click **Next** 16. Click **Finish** 17. Click the **Sign On** tab of the new Okta app 18. Copy the **Metadata URL** Update your SAML settings in Sublime: 1. In Sublime, go to **Admin** > **Account** 2. Under **Authentication**, click the button next to **SAML** 3. Under **Metadata URL**, paste the metadata URL you copied from Okta 4. Click **Save** ### Test SAML [](https://docs.sublime.security/docs/single-sign-on#test-saml) > 📘 > > Matching user required > > > ---------------------------- > > For a user to successfully sign into Sublime with your SAML identity provider, there must already be a matching user with the same email address in Sublime. We recommend provisioning users through [SCIM](https://docs.sublime.security/docs/scim) > . 1. In Sublime, go to **Admin** > **Account** 2. Under **Authentication**, verify **Allowed methods** is set to either **Any method** or **SAML only**, or update accordingly 3. While signed into Okta as the provisioned user, navigate to the SAML signon URL you copied from the Sublime dashboard You should be signed into the Sublime dashboard. You're in business! Integrating with OIDC [](https://docs.sublime.security/docs/single-sign-on#integrating-with-oidc) ===================================================================================================== There are two high-level steps to integrating the Sublime Platform with OpenID Connect: 1. Obtain a set of OIDC settings, consisting of an issuer URL, a client ID, and a client secret 2. Add these settings to Sublime 1\. Obtain OIDC settings [](https://docs.sublime.security/docs/single-sign-on#1-obtain-oidc-settings) --------------------------------------------------------------------------------------------------------- The process for obtaining OIDC settings varies between OIDC identity providers, but typically involves creating an application that includes an OAuth 2.0 client ID and client secret, and configuring the appropriate redirect URI. ### Okta [](https://docs.sublime.security/docs/single-sign-on#okta-1) Below are the steps for obtaining OIDC settings via Okta. Open the OIDC settings modal in Sublime: 1. Log into the Sublime Platform 2. Go to **Admin** > **Account** 3. Under **Authentication**, click the button next to **Open ID Connect** 4. Keep this page open, as you'll use the **Initiate login URL** and **Redirect URI** to set up an application in Okta Create the application in Okta: 1. Sign into your Okta admin console 2. Go to **Applications** > **Applications** 3. Click **Create App Integration** 4. In the modal that opens, select **Sign-in method** of **OIDC - OpenID Connect** 5. Then select **Application type** of **Web Application** 6. Click the **Next** button 7. Provide an **App integration name**, such as "Sublime Platform" 8. Optionally add a logo. You can download the Sublime logo [here](https://user-images.githubusercontent.com/11003450/115128085-5805da00-9fa9-11eb-8c7a-dc8b708053ee.png) . 9. In the **Grant type** section, check **Implicit (Hybrid)** 10. In **Sign-in redirect URIs**, paste the **Redirect URI** from Sublime 11. Remove the default entry in **Sign-out redirect URIs** 12. In the **Controlled access** section, select your preferred option 13. Click the **Save** button 14. Click the **Edit** button in the **General Settings** section 15. In **Login initiated by**, select **Either Okta or App** 16. In **Initiate login URI**, paste the **Initiate login URL** from Sublime 17. Next to **Application visibility**, check **Display application icon to users** and optionally check **Display application icon in the Okta Mobile app** 18. Click the **Save** button 19. Note the **Client ID** and **Client secret** from the current page 20. Click the **Sign On** tab 21. Note the **Issuer** URL under **OpenID Connect ID Token** You'll use the client ID, client secret, and issuer URL you noted in the next section ### Azure Active Directory [](https://docs.sublime.security/docs/single-sign-on#azure-active-directory) To use Azure Active Directory as your SSO provider for Sublime, you'll need to create an Azure application. Note that you have the option of using the same application you created when [setting up a Microsoft 365 message source](https://docs.sublime.security/docs/add-an-office-365-message-source) . > 🚧 > > You must use an Azure AD tenant you control > > > ------------------------------------------------- > > You must create your Azure application for Sublime SSO in a tenant you wholly control to ensure no third party can impersonate your users. First, get the **Redirect URI** for your Sublime organization: 1. Log into the Sublime Platform 2. Go to **Admin** > **Account** 3. Under **Authentication**, click the button next to **Open ID Connect** 4. Note the **Redirect URI**, as you'll use it to set up an application in Azure To create the application in Azure, follow the steps below. If you're using an existing Azure app for SSO, jump to step 7. 1. Sign into [portal.azure.com](https://portal.azure.com/) 2. Click **App Registrations** 3. Click **New Registration** 4. Give your application a name, such as "Sublime Platform" 5. Under **Supported account types** select **Accounts in this organizational directory only** if it's not already selected by default 6. Click **Register** (skip the **Redirect URI** section) 7. In the **Overview** section of the application's settings, note the **Application (client) ID** and the **Directory (tenant) ID**. You'll use these IDs later. 8. Go to the **Authentication** settings page 9. Click **Add a platform** 10. In the panel that opens, click **Web** 11. Under **Redirect URIs**, paste the **Redirect URI** from Sublime 12. Under **Implicit grant and hybrid flows**, check **ID tokens** 13. Click **Configure** 14. Go to the **Certificates & secrets** settings page 15. Click **New client secret** 16. Give the client secret a name like "Sublime SSO" and select an expiration of "24 months" 17. Click **Add** 18. Note the value of the new client secret Now that you've configure your Azure application, you'll use the following values in the next section of this guide: * Your **issuer URL** is `https://login.microsoftonline.com/TENANT_ID/v2.0`, with `TENANT_ID` being the **Directory (tenant) ID** you noted earlier * Your **client ID** is the **Application (client) ID** you noted earlier * Your **client secret** is the client secret you just created ### Google [](https://docs.sublime.security/docs/single-sign-on#google) In order to use Google's OpenID Connect feature, you'll need to create a Google Cloud Platform project with an OAuth app restricted to your organization, then create a client ID/secret pair to use to set up OIDC in Sublime. Here's a video recording showing the steps to configure the app and create the client ID/secret pair. * Note that the GCP settings not configured in the recording are optional. * Add the Redirect URI provided in the "Configure OpenID Connect" modal in Sublime to the "authorized redirect URIs" in GCP. * Add the client ID and client secret from Google to your OIDC settings in Sublime, using `https://accounts.google.com` as the "Issuer URL". 2\. Add OIDC settings to Sublime [](https://docs.sublime.security/docs/single-sign-on#2-add-oidc-settings-to-sublime) ------------------------------------------------------------------------------------------------------------------------- 1. Log into the Sublime Platform 2. Go to **Admin** > **Account** 3. Under **Authentication**, click the button next to **Open ID Connect** 4. Enter your OIDC **issuer URL**, **client ID**, and **client secret** 5. Click the **Save** button Test the integration [](https://docs.sublime.security/docs/single-sign-on#test-the-integration) --------------------------------------------------------------------------------------------------- You should now be able to sign into Sublime with your OIDC identity provider. You can verify the integration is working by either selecting the Sublime Platform application in your provider (for example, Okta), or by loading the **Initiate login URL** from your OIDC settings in Sublime. > 📘 > > Matching user required > > > ---------------------------- > > For a user to successfully sign into Sublime with your OIDC identity provider, there must already be a matching user with the same email address in Sublime. Managing allowed authentication methods [](https://docs.sublime.security/docs/single-sign-on#managing-allowed-authentication-methods) ========================================================================================================================================= Once you set up SAML or OIDC, you can optionally restrict user signin to only SAML or only OIDC. If you'd like to use this option: 1. Sign out of Sublime and verify signin via OpenID Connect is working 2. Go to **Admin** > **Account** 3. Under **Authentication**, click the button next to **Allowed methods** 4. Select **Users must log in with OpenID Connect SSO** or **Users must log in with SAML SSO** 5. Click **Save** Updated 12 months ago * * * --- # Attack Score Overview [](https://docs.sublime.security/docs/attack-score#overview) ========================================================================= Sublime’s Attack Score is a machine learning-based feature that helps security teams understand and prioritize potential threats based on observed attack indicators in an email message. Attack Score is surfaced through a verdict — `malicious`, `spam`, `graymail`, `suspicious`, `unknown`, or `likely_benign` — along with corresponding signals to provide easily digestible context during the alert investigation process about the message and the underlying machine-learning model logic. ![attack_score_example_expanded.png](https://files.readme.io/247f5aba0e96309434d76087a2827e70109d246beffe69fe9cbcc64812c9ca37-Screenshot_2025-02-26_at_4.35.18_PM.png) Attack Score utilizes hundreds of signals written in MQL extracted from various parts of a message, organizational context, past behavior, and more: * Headers * Attachment metadata * Link analysis * Sender behavior * Content understanding * Authentication checks These signals are analyzed to identify common attack patterns and unusual activity. They are derived from attachments, headers, authentication, message body, links, and sender while leveraging enrichment capabilities such as: * Sender behavior profiles * Natural Language Understanding * Computer Vision * WHOIS * Domain reputation * Attachment analysis Every Attack Score has 3 components: * **Verdict:** This is a description — `malicious`, `spam`, `suspicious`, `unknown`, or `likely_benign` — utilized to communicate the likelihood of a given message being an attack. * `Malicious` verdicts signal a threat with strong confidence. * `Spam` verdicts indicate unsolicited and unwanted emails sent to recipients without consent, which are fully undesirable for the mailbox. * `Suspicious` verdicts contain patterns indicative of potential threats but are inconclusive. * `Unknown` verdicts are similar to Suspicious as they contain patterns indicative of potential threats but may also contain common benign behaviors. * `Likely Benign` verdicts contain patterns commonly associated with benign email. * **Top Signals:** These are the top 2-5 characteristics that most contribute to a given raw score and verdict, providing context and transparency through explainable ML. * **Raw Score:** This is a number from 0-100 that is calculated from the ML algorithm. The raw score of a message determines which verdict is selected. Raw scores are not available in the platform at this time, but are available via the [API](https://docs.sublime.security/reference/attackscoreformessage) . Common workflows [](https://docs.sublime.security/docs/attack-score#common-workflows) ========================================================================================= Attack Score can be used in a variety of ways, including: ### Auto-remediate high-confidence threats [](https://docs.sublime.security/docs/attack-score#auto-remediate-high-confidence-threats) Sublime's default [Automations](https://docs.sublime.security/docs/automations) look for flagged or user reported messages with the `malicious` verdict. Activate these Automations in your instance to automatically remediate high-confidence attacks! ### In-platform message reviews [](https://docs.sublime.security/docs/attack-score#in-platform-message-reviews) You can leverage the Attack Score on any investigation by simply opening the message details within the platform. You do not need to view the message contents in order to view a message’s Attack Score verdict and top signals. `Malicious` verdicts are assigned with precision, making them high-confidence indicators of phish and spam in user mailboxes. In general, analysts should spend limited time reviewing these messages, opting instead to classify the message as malicious and move it to quarantine (automatically with [Automations](https://docs.sublime.security/docs/automations) or manually with their review). `Suspicious` and `unknown` verdicts are very similar because they both contain patterns indicative of potential threats, but `suspicious` verdicts indicate a higher likelihood of a malicious message. ### Submitting feedback for future ML models [](https://docs.sublime.security/docs/attack-score#submitting-feedback-for-future-ml-models) All users are encouraged to share feedback with Sublime about the performance of Attack Score in their environment. If you would like to provide us with feedback on any of the verdicts or signals provided, please share the EML with our team using the “Share with Sublime” checkbox during the message review or the “Send EML to Sublime” button on the lower left side of the message investigation page. Comments highlighting the details of your Attack Score feedback are welcome! Enterprise customers can also share specific callouts with the Sublime team through the usual support channels. ### “Auto-reviewing” in SOARs [](https://docs.sublime.security/docs/attack-score#auto-reviewing-in-soars) If you utilize a SOAR, you can call our [Attack Score API endpoint](https://docs.sublime.security/reference/attackscoreformessage) , `/v0/messages/:id/attack_score`, in order to enrich your automation. When the `verdict` is set to `malicious`, you can complete your review without manual analyst review by calling the [`v0/message-groups/review` API endpoint](https://docs.sublime.security/reference/reviewmessagegroups) to eliminate time spent investigating high-confidence threats. Example MQL [](https://docs.sublime.security/docs/attack-score#example-mql) =============================================================================== #### Attack Score is `malicious` [](https://docs.sublime.security/docs/attack-score#attack-score-is-malicious) MQL `type.inbound and ml.attack_score().verdict == "malicious"` #### Attack Score is `spam` [](https://docs.sublime.security/docs/attack-score#attack-score-is-spam) MQL `type.inbound and ml.attack_score().verdict == "spam"` * * * Support for additional workflows, such as MQL Rules and message triage views, is coming soon! Updated 27 days ago * * * --- # How to Integrate Email Threat Intel with a TIP Introduction [](https://docs.sublime.security/docs/how-to-integrate-email-threat-intel-with-a-tip#introduction) =================================================================================================================== This guide outlines how a Sublime user can send Indicators of Compromise (IOC) to commonly used Threat Intelligence Platforms (TIPs) like MISP or RecordedFuture. Sublime provides a significant amount of flexibility for handling these use cases and this article is intended to outline an event-driven, webhook-based approach. Setup [](https://docs.sublime.security/docs/how-to-integrate-email-threat-intel-with-a-tip#setup) ===================================================================================================== Create a Webhook Action [](https://docs.sublime.security/docs/how-to-integrate-email-threat-intel-with-a-tip#create-a-webhook-action) ----------------------------------------------------------------------------------------------------------------------------------------- 1. From the Sublime Dashboard, navigate to the `/actions` page found under Manage on the left navigation menu. Create a new Action > Webhook. 2. Name your Webhook and configure the appropriate endpoint and custom headers for the outbound request. Ensure that the "Finish message processing before sending webhooks?" is checked. > 💡 > -- > > For more information on Sublime webhooks, please see the [documentation here](https://docs.sublime.security/docs/webhooks) > . Create an Automation [](https://docs.sublime.security/docs/how-to-integrate-email-threat-intel-with-a-tip#create-an-automation) ----------------------------------------------------------------------------------------------------------------------------------- 1. From the Sublime Dashboard, navigate to the `/automations` page found under `Automate` on the left navigation menu. Create a new Automation. 2. Set the automation triggers to be for both Flagged Messages as well as User Reports. 3. The following MQL example will send a webhook whenever the message is sent from an Unknown sender or repeat offender and the message includes at least one hyperlink or attachment. MQL `type.inbound and ( ( profile.by_sender().prevalence in ("new", "outlier") and not profile.by_sender().solicited ) or ( profile.by_sender().any_messages_malicious_or_spam and not profile.by_sender().any_false_positives ) ) and (length(body.links) > 0 or length(attachments) > 0)` Feel free to adjust the above MQL logic to fit your use case. See our [common snippets library](https://docs.sublime.security/docs/common-snippets) for inspiration on other ways to trigger the automation. Handle the Webhook Request [](https://docs.sublime.security/docs/how-to-integrate-email-threat-intel-with-a-tip#handle-the-webhook-request) ----------------------------------------------------------------------------------------------------------------------------------------------- By design, Sublime's webhook payload excludes properties that include PII data. Upon receiving the request you will want to retrieve the [message's MDM properties](https://docs.sublime.security/reference/getmessagedatamodel-1) via Sublime's API to pass into your TIP. This API call will require the Sublime Message ID found in the webhook payload. Common IOC strings found in the MDM include: * Attachment hashes * Sender domain * Sender email * Links from the body of the email * Originating IP Addresses That's it! Once the Automation is activated Sublime will pass all flagged messages and user reports that match the automation logic to your TIP. If you have any questions about this guide please reach out to the Sublime Support team. Updated 4 months ago * * * --- # Export to S3 You can export all messages' MDMs or Sublime audit logs and message events to an S3 bucket. Sublime customers often use the S3 bucket in coordination with their SIEM or data lake to enhance their email data management and enrichment workflows. Commonly used SIEMs and data lakes: * [Chronicle](https://cloud.google.com/chronicle/docs/ingestion/ingest-aws-logs-into-chronicle) * [Elastic](https://www.elastic.co/docs/current/integrations/sublime_security) * [Panther](https://panther.com/integrations/logs/sublime-security/) * [Snowflake](https://docs.snowflake.com/en/user-guide/data-load-s3) * [Splunk](https://docs.splunk.com/Documentation/AddOns/released/AWS/S3) * [Sumo Logic](https://help.sumologic.com/docs/send-data/hosted-collectors/amazon-aws/aws-s3-source/) For help with your exports to S3, reach out in our [Slack community](https://join.slack.com/t/sublimecommunity/shared_invite/zt-1hhwosroy-LvflKNVE3HEtgIcbHdB1sw) ! Updated 10 months ago * * * * [Export Message MDMs](https://docs.sublime.security/docs/setting-up-mdm-export-to-s3-bucket) * [Audit Logs and Message Events](https://docs.sublime.security/docs/setting-up-audit-log-and-message-events-export-to-s3-bucket) --- # Export Audit Logs and Message Events Background [](https://docs.sublime.security/docs/export-audit-logs-and-message-events#background) ===================================================================================================== Sublime Enterprise users on Cloud and AWS CloudFormation deployments can configure all audit logs and message events to export to an S3 bucket that you own every 10 minutes. _Note: Enterprise users on Docker deployments can request assistance with exports to S3 in your Sublime support Slack channel._ 1\. Setup [](https://docs.sublime.security/docs/export-audit-logs-and-message-events#1-setup) ================================================================================================= Before starting, you'll need to: 1. Determine the ID of the AWS account uploads will be coming from (used for `AWS_ACCOUNT_ID` placeholder below): * If you've deployed into a self-managed AWS environment, then the Account ID will be that of the AWS account. * If you are a Sublime Cloud user located in the EU, then the value you'll use is `128225141972`. * For all other Sublime Cloud users, then the value you'll use is `415365657389`. 2. Create an S3 bucket in the AWS account where you want the events exported to. We recommend creating a new S3 bucket specifically for these exports instead of re-using an existing one. 2\. Configure bucket permissions [](https://docs.sublime.security/docs/export-audit-logs-and-message-events#2-configure-bucket-permissions) =============================================================================================================================================== These instructions assume you have a bucket with default settings and bucket policies. 1. Navigate to your desired bucket in the AWS S3 dashboard, and go to the Permissions tab. 2. Click Edit on the `Block public access (bucket settings)` section, which should bring you to the following page: ![](https://files.readme.io/d7a499e-image.png) 3. Make sure the final checkbox is _deselected_. The other three should remain checked, unless you have a particular reason not to. Save these changes. 4. Now, scroll down to the `Bucket policy` section (still on the Permissions tab). This is where we'll set up the policy to allow cross-account uploads from the account Sublime is running in. Click Edit on the `Bucket Policy` tab. 5. Use the following template as the Policy. Update the placeholders `AWS_ACCOUNT_ID` (use the ID from Setup step 1 -- the AWS account where exports are coming from) and `BUCKET_NAME_HERE` with the corresponding values. Save changes once complete. JSON `{ "Version": "2012-10-17", "Statement": [ { "Sid": "AllowSublimeWrites", "Effect": "Allow", "Principal": { "AWS": "arn:aws:iam::AWS_ACCOUNT_ID:root" }, "Action": [ "s3:PutObject*" ], "Resource": [ "arn:aws:s3:::BUCKET_NAME_HERE/*" ] } ] }` 6. The bucket is now configured for cross-account uploads from the Sublime platform. * * * 3\. Configure settings in Sublime [](https://docs.sublime.security/docs/export-audit-logs-and-message-events#3-configure-settings-in-sublime) ================================================================================================================================================= To complete configuration, head over to `Account > Admin` in your Sublime dashboard and scroll down to the **Audit Log and Message Events Export** setting. Configure the following fields: * **S3 bucket name**: Bucket names must be globally unique. They can be named anything but here’s a sample format: `sublime-audit-log-and-message-events-export-` * **S3 prefix** (optional): Configuring a prefix will store your exports in a folder with the prefix name within the bucket. A prefix is not necessary if the bucket is not used for any other use cases (recommended). Example prefix: `exports` * **S3 region**: the region you created the S3 bucket in * **JSON Line formatting** (optional): Check this box to configure your exports in JSON Line formatting. ![](https://files.readme.io/c6b7807a0f55044a5ba961c567284fb126415f05e7519b5149545e57502f57e2-s3_export-audit_logs.png) > 📘 > > Panther Integration > > > ------------------------- > > Note that the Panther integration does not support JSON Lines text formatting. That can be disabled when using them as a SIEM provider. After hitting **Apply**, it may take up to 20 minutes before you start seeing events in S3. After the first export, exports take place every 10 minutes. S3 Export Details [](https://docs.sublime.security/docs/export-audit-logs-and-message-events#s3-export-details) =================================================================================================================== File Names [](https://docs.sublime.security/docs/export-audit-logs-and-message-events#file-names) ----------------------------------------------------------------------------------------------------- The filename (with prefixes) is output as follows: `ConfiguredKeyPrefix/AuditType/YYYY/MM/DD/HHMMSSZ-ID.json` Where each is as follows: `ConfiguredKeyPrefix`: The key prefix that was configured in your Sublime Platform settings. If you did not set a prefix, this field will be omitted. `AuditType`: The type of audit content, either `sublime_platform_audit_log` or `sublime_platform_message_events` `YYYY`: Year `MM`: Month, zero-padded `DD`: Day, zero-padded `HHMMSSZ`: The hour, minute, seconds, and timezone, zero-padded `ID`: Six random characters Example Audit Logs [](https://docs.sublime.security/docs/export-audit-logs-and-message-events#example-audit-logs) --------------------------------------------------------------------------------------------------------------------- This includes all audit log events, stored under the `sublime_platform_audit_log` type. Example 1 (empty): json `{ "events": null, "count": 0, "start": "2023-05-03T23:55:01.06552Z", "end": "2023-05-04T00:05:00.309749667Z", "key": "sublime_platform_audit_log/2023/05/04/000500Z-LPPJKV.json" }` Example 2: JSON `{ "events": [ { "id": "beace168-2c8f-43ca-9e4b-6d7f76af1ddc", "type": "message.view_contents", "created_at": "2023-05-04T00:59:29.58391Z", "data": { "request": { "id": "c61dafcb-dfdd-41bb-acdc-e66fee4148c4", "path": "/v1/messages/groups/0c8e2ef2a2eb02fbbdafb74c8dc5e27ab1096527cc1d07483cf4519a5478dd5f", "method": "GET", "query": {}, "body": "", "authentication_method": "user_session", "ip": "71.81.206.41/32", "user_agent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/112.0.0.0 Safari/537.36" } }, "created_by": { "id": "8ccab895-d27f-4250-9d51-ae50793d3a68", "first_name": "Sublime", "last_name": "Test", "email_address": "[[email protected]](https://docs.sublime.security/cdn-cgi/l/email-protection) ", "phone_number": null, "created_at": "2021-06-21T18:09:27.66874Z", "updated_at": "2022-06-14T09:11:38.629119Z", "role": "admin", "is_enrolled": true, "google_oauth_user_id": "", "microsoft_oauth_user_id": "" } }, { "id": "a7e29e07-6ce6-429a-88a7-b29b6fb55d38", "type": "message_group.trash", "created_at": "2023-05-04T00:59:25.252995Z", "data": { "message_group": { "id": "0c8e2ef2a2eb02fbbdafb74c8dc5e27ab1096527cc1d07483cf4519a5478dd5f" }, "request": { "id": "316a16b1-2658-4b9b-bc74-3302c2662665", "path": "/v1/messages/groups/0c8e2ef2a2eb02fbbdafb74c8dc5e27ab1096527cc1d07483cf4519a5478dd5f/delete", "method": "POST", "query": {}, "body": "{\"review_comment\":\"\",\"share_with_sublime\":false}", "authentication_method": "user_session", "ip": "71.81.206.41/32", "user_agent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/112.0.0.0 Safari/537.36" } }, "created_by": { "id": "8ccab895-d27f-4250-9d51-ae50793d3a68", "first_name": "Sublime", "last_name": "Test", "email_address": "[[email protected]](https://docs.sublime.security/cdn-cgi/l/email-protection) ", "phone_number": null, "created_at": "2021-06-21T18:09:27.66874Z", "updated_at": "2022-06-14T09:11:38.629119Z", "role": "admin", "is_enrolled": true, "google_oauth_user_id": "", "microsoft_oauth_user_id": "" } }, { "id": "6930efb2-bc40-4e90-a206-7fb2211cf3aa", "type": "message.view_contents", "created_at": "2023-05-04T00:58:29.179602Z", "data": { "request": { "id": "9ed8b3ee-bab2-493c-a675-44344df32a91", "path": "/v1/messages/groups/0c8e2ef2a2eb02fbbdafb74c8dc5e27ab1096527cc1d07483cf4519a5478dd5f", "method": "GET", "query": {}, "body": "", "authentication_method": "user_session", "ip": "71.81.206.41/32", "user_agent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/112.0.0.0 Safari/537.36" } }, "created_by": { "id": "8ccab895-d27f-4250-9d51-ae50793d3a68", "first_name": "Sublime", "last_name": "Test", "email_address": "[[email protected]](https://docs.sublime.security/cdn-cgi/l/email-protection) ", "phone_number": null, "created_at": "2021-06-21T18:09:27.66874Z", "updated_at": "2022-06-14T09:11:38.629119Z", "role": "admin", "is_enrolled": true, "google_oauth_user_id": "", "microsoft_oauth_user_id": "" } }, { "id": "5545d2da-04d2-482d-a32e-a9e494b782d9", "type": "message.view_contents", "created_at": "2023-05-04T00:58:29.159373Z", "data": { "request": { "id": "16967ed4-7b70-44f1-9a5b-0be4c46f0f02", "path": "/v0/messages/6e079728-0cec-4388-87c2-d35595fed536/image_link", "method": "GET", "query": {}, "body": "", "authentication_method": "user_session", "ip": "71.81.206.41/32", "user_agent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/112.0.0.0 Safari/537.36" } }, "created_by": { "id": "8ccab895-d27f-4250-9d51-ae50793d3a68", "first_name": "Sublime", "last_name": "Test", "email_address": "[[email protected]](https://docs.sublime.security/cdn-cgi/l/email-protection) ", "phone_number": null, "created_at": "2021-06-21T18:09:27.66874Z", "updated_at": "2022-06-14T09:11:38.629119Z", "role": "admin", "is_enrolled": true, "google_oauth_user_id": "", "microsoft_oauth_user_id": "" } }, { "id": "06e1fe91-6963-4bc6-96a6-1f141dedd974", "type": "message.access_justification", "created_at": "2023-05-04T00:58:28.911435Z", "data": { "message": { "id": "6e079728-0cec-4388-87c2-d35595fed536" }, "request": { "id": "934a3a0c-1794-4204-bc4a-e37c9e405f9d", "path": "/v1/messages/6e079728-0cec-4388-87c2-d35595fed536/justification", "method": "POST", "query": {}, "body": "{}", "authentication_method": "user_session", "ip": "71.81.206.41/32", "user_agent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/112.0.0.0 Safari/537.36" } }, "created_by": { "id": "8ccab895-d27f-4250-9d51-ae50793d3a68", "first_name": "Sublime", "last_name": "Test", "email_address": "[[email protected]](https://docs.sublime.security/cdn-cgi/l/email-protection) ", "phone_number": null, "created_at": "2021-06-21T18:09:27.66874Z", "updated_at": "2022-06-14T09:11:38.629119Z", "role": "admin", "is_enrolled": true, "google_oauth_user_id": "", "microsoft_oauth_user_id": "" } }, { "id": "dd529556-4578-4e54-833a-23d69053a6c5", "type": "message.view_contents", "created_at": "2023-05-04T00:58:20.681083Z", "data": { "request": { "id": "d1643ae7-94e6-4f3d-82ca-645456dc96f1", "path": "/v1/messages/groups/0c8e2ef2a2eb02fbbdafb74c8dc5e27ab1096527cc1d07483cf4519a5478dd5f", "method": "GET", "query": {}, "body": "", "authentication_method": "user_session", "ip": "71.81.206.41/32", "user_agent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/112.0.0.0 Safari/537.36" } }, "created_by": { "id": "8ccab895-d27f-4250-9d51-ae50793d3a68", "first_name": "Sublime", "last_name": "Test", "email_address": "[[email protected]](https://docs.sublime.security/cdn-cgi/l/email-protection) ", "phone_number": null, "created_at": "2021-06-21T18:09:27.66874Z", "updated_at": "2022-06-14T09:11:38.629119Z", "role": "admin", "is_enrolled": true, "google_oauth_user_id": "", "microsoft_oauth_user_id": "" } }, { "id": "0e7017b5-60fe-49b4-ac8a-3f45765e7cf7", "type": "message_group.quarantine", "created_at": "2023-05-04T00:58:08.206579Z", "data": { "request": { "id": "059b0f11-0caa-4444-8f2c-6256a8bcd5a1", "path": "/v1/messages/groups/0c8e2ef2a2eb02fbbdafb74c8dc5e27ab1096527cc1d07483cf4519a5478dd5f/quarantine", "method": "POST", "query": {}, "body": "{\"review_comment\":\"\",\"share_with_sublime\":false}", "authentication_method": "user_session", "ip": "71.81.206.41/32", "user_agent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/112.0.0.0 Safari/537.36" } }, "created_by": { "id": "8ccab895-d27f-4250-9d51-ae50793d3a68", "first_name": "Sublime", "last_name": "Test", "email_address": "[[email protected]](https://docs.sublime.security/cdn-cgi/l/email-protection) ", "phone_number": null, "created_at": "2021-06-21T18:09:27.66874Z", "updated_at": "2022-06-14T09:11:38.629119Z", "role": "admin", "is_enrolled": true, "google_oauth_user_id": "", "microsoft_oauth_user_id": "" } }, { "id": "377369a8-e07f-4052-b61b-6ed88c921f15", "type": "message.view_contents", "created_at": "2023-05-04T00:58:04.017724Z", "data": { "request": { "id": "c5494c7e-fcdc-4513-90dc-d0990b63eb7c", "path": "/v0/messages/6e079728-0cec-4388-87c2-d35595fed536/image_link", "method": "GET", "query": {}, "body": "", "authentication_method": "user_session", "ip": "71.81.206.41/32", "user_agent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/112.0.0.0 Safari/537.36" } }, "created_by": { "id": "8ccab895-d27f-4250-9d51-ae50793d3a68", "first_name": "Sublime", "last_name": "Test", "email_address": "[[email protected]](https://docs.sublime.security/cdn-cgi/l/email-protection) ", "phone_number": null, "created_at": "2021-06-21T18:09:27.66874Z", "updated_at": "2022-06-14T09:11:38.629119Z", "role": "admin", "is_enrolled": true, "google_oauth_user_id": "", "microsoft_oauth_user_id": "" } }, { "id": "5d60ce00-57a1-4add-8a2a-a1c84070c320", "type": "message.view_contents", "created_at": "2023-05-04T00:58:03.719453Z", "data": { "request": { "id": "82f4e05b-8649-4ddc-96b5-89c3fe9133bc", "path": "/v1/messages/groups/0c8e2ef2a2eb02fbbdafb74c8dc5e27ab1096527cc1d07483cf4519a5478dd5f", "method": "GET", "query": {}, "body": "", "authentication_method": "user_session", "ip": "71.81.206.41/32", "user_agent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/112.0.0.0 Safari/537.36" } }, "created_by": { "id": "8ccab895-d27f-4250-9d51-ae50793d3a68", "first_name": "Sublime", "last_name": "Test", "email_address": "[[email protected]](https://docs.sublime.security/cdn-cgi/l/email-protection) ", "phone_number": null, "created_at": "2021-06-21T18:09:27.66874Z", "updated_at": "2022-06-14T09:11:38.629119Z", "role": "admin", "is_enrolled": true, "google_oauth_user_id": "", "microsoft_oauth_user_id": "" } } ], "count": 9, "start": "2023-05-04T00:55:00.91963Z", "end": "2023-05-04T01:05:00.246998523Z", "key": "sublime_platform_audit_log/2023/05/04/010500Z-MQABBD.json" }` Example Message Events [](https://docs.sublime.security/docs/export-audit-logs-and-message-events#example-message-events) ----------------------------------------------------------------------------------------------------------------------------- This includes all flagged message data, under the `sublime_platform_message_events` type. Example 1 (empty): JSON `{ "events": null, "count": 0, "start": "2023-05-03T23:55:01.06552Z", "end": "2023-05-04T00:05:00.309749667Z", "key": "sublime_platform_message_events/2023/05/04/000500Z-ONJOFG.json" }` Example2: JSON `{ "events": [ { "created_at": "2023-05-04T01:06:50.73801Z", "data": { "flagged_rules": [ { "id": "958ad820-767f-4c83-ac44-94032151574f", "name": "Test rule1", "severity": null, "tags": [ "Testing" ] }, { "id": "2483da42-6905-4eca-bc8e-a5475ab906a4", "name": "Test rule2", "severity": null, "tags": [ "Testing" ] }, { "id": "9f95f7bb-b1f1-465a-a7a4-23266792c000", "name": "Test rule3", "severity": null, "tags": [ "Testing" ] }, { "id": "862b4f5b-c0f7-4e36-89b2-3091f9ac492b", "name": "Test rule4", "severity": null, "tags": [ "Testing" ] } ], "message": { "canonical_id": "8fdb3acd8441b18478b7e80927c732f44e9086dcf7cfa6c1186ed21637427ef8", "external_id": "187e44c2288f2e8d", "id": "089b2e97-2107-4d1e-83e1-1920314dbb80", "mailbox": { "external_id": "100712120249161713725", "id": "3142d2f8-ac0d-4e37-bb1f-d5e4ff5f7f96" }, "message_source_id": "d76ce576-4687-4182-b8e3-a220d6878b21" }, "triggered_actions": null }, "type": "message.flagged" } ], "count": 1, "start": "2023-05-04T01:05:00.246998Z", "end": "2023-05-04T01:15:00.164976076Z", "key": "sublime_platform_message_events/2023/05/04/011500Z-NBFUQY.json" }` Updated about 1 month ago * * * --- # Common snippets ### Unsolicited [](https://docs.sublime.security/docs/common-snippets#unsolicited) The organization has never sent an email to this sender. MQL `not profile.by_sender().solicited` ### First-time sender [](https://docs.sublime.security/docs/common-snippets#first-time-sender) The organization has never received an email from this sender. MQL `profile.by_sender().prevalence in ("new")` ### Unknown sender or repeat offender [](https://docs.sublime.security/docs/common-snippets#unknown-sender-or-repeat-offender) MQL `and ( ( profile.by_sender().prevalence in ("new", "outlier") and not profile.by_sender().solicited ) or ( profile.by_sender().any_messages_malicious_or_spam and not profile.by_sender().any_false_positives ) )` ### Free subdomain link [](https://docs.sublime.security/docs/common-snippets#free-subdomain-link) MQL `any(body.links, .href_url.domain.subdomain is not null and .href_url.domain.subdomain != "www" and .href_url.domain.root_domain in $free_subdomain_hosts )` ### Free file / content hosting link [](https://docs.sublime.security/docs/common-snippets#free-file--content-hosting-link) MQL `any(body.links, .href_url.domain.domain in $free_file_hosts)` ### Negate high trust senders [](https://docs.sublime.security/docs/common-snippets#negate-high-trust-senders) MQL `// negate highly trusted sender domains unless they fail DMARC authentication and ( ( sender.email.domain.root_domain in $high_trust_sender_root_domains and ( any(distinct(headers.hops, .authentication_results.dmarc is not null), strings.ilike(.authentication_results.dmarc, "*fail") ) ) ) or sender.email.domain.root_domain not in $high_trust_sender_root_domains )` ### Detect Spoofing of Internal Org Domains [](https://docs.sublime.security/docs/common-snippets#detect-spoofing-of-internal-org-domains) MQL `type.inbound and sender.email.domain.domain in $org_domains and (not headers.auth_summary.spf.pass or not headers.auth_summary.dmarc.pass or 'fail' in~ distinct(map(headers.hops, .authentication_results.dkim)) )` ### URL shortener [](https://docs.sublime.security/docs/common-snippets#url-shortener) MQL `any(body.links, .href_url.domain.root_domain in $url_shorteners)` ### Low reputation link [](https://docs.sublime.security/docs/common-snippets#low-reputation-link) Not in tranco 1m, or a free subdomain/file host. MQL `any(body.links, .href_url.domain.root_domain not in $tranco_1m or ( // free subdomain URL .href_url.domain.subdomain is not null and .href_url.domain.subdomain != "www" and .href_url.domain.root_domain in $free_subdomain_hosts ) )` ### Specific domain in the body of a message [](https://docs.sublime.security/docs/common-snippets#specific-domain-in-the-body-of-a-message) MQL `any(body.links, .href_url.domain.domain in ( "foo.com", "foobar.com" ) )` ### Exclude email addresses or aliases in a custom list (To, CC, or BCC'd) [](https://docs.sublime.security/docs/common-snippets#exclude-email-addresses-or-aliases-in-a-custom-list-to-cc-or-bccd) MQL `// exclude emails to support aliases not any([recipients.to, recipients.cc, recipients.bcc], any(., .email.email in $support_email_aliases) )` ### Exclude specific mailboxes [](https://docs.sublime.security/docs/common-snippets#exclude-specific-mailboxes) MQL `not mailbox.email.email in ("[[email protected]](https://docs.sublime.security/cdn-cgi/l/email-protection) ")` ### Only run on specific mailboxes in a custom list [](https://docs.sublime.security/docs/common-snippets#only-run-on-specific-mailboxes-in-a-custom-list) MQL `mailbox.email.email in $highly_targeted_user_email_addresses` ### Recipient's email address is in the URL path of a link in the body [](https://docs.sublime.security/docs/common-snippets#recipients-email-address-is-in-the-url-path-of-a-link-in-the-body) MQL `any(body.links, // is the recipient's email address in the URL? // this method accounts for any encoding we might encounter // in the query_params any(recipients.to, strings.icontains(..href_url.url, .email.local_part) and strings.icontains(..href_url.url, .email.domain.domain) ) )` ### Emoji in the body or subject [](https://docs.sublime.security/docs/common-snippets#emoji-in-the-body-or-subject) MQL `// has an emoji in the subject or body and ( regex.contains(body.plain.raw, '[\x{1F300}-\x{1F5FF}\x{1F600}-\x{1F64F}\x{1F680}-\x{1F6FF}\x{1F700}-\x{1F77F}\x{1F780}-\x{1F7FF}\x{1F900}-\x{1F9FF}\x{2600}-\x{26FF}\x{2700}-\x{27BF}\x{2300}-\x{23FF}]') or regex.contains(subject.subject, '[\x{1F300}-\x{1F5FF}\x{1F600}-\x{1F64F}\x{1F680}-\x{1F6FF}\x{1F700}-\x{1F77F}\x{1F780}-\x{1F7FF}\x{1F900}-\x{1F9FF}\x{2600}-\x{26FF}\x{2700}-\x{27BF}\x{2300}-\x{23FF}]') )` ### Recipient is in a user group list [](https://docs.sublime.security/docs/common-snippets#recipient-is-in-a-user-group-list) MQL `any(recipients.to, any($finance_users, .email == ..email.email))` ### Sender email address [](https://docs.sublime.security/docs/common-snippets#sender-email-address) MQL `sender.email.email == "[[email protected]](https://docs.sublime.security/cdn-cgi/l/email-protection) "` ### Attachment file name [](https://docs.sublime.security/docs/common-snippets#attachment-file-name) MQL `any(attachments, .file_name == "foo.txt")` ### Attachment file name contains a specific string [](https://docs.sublime.security/docs/common-snippets#attachment-file-name-contains-a-specific-string) MQL `any(attachments, .file_type == "pdf" and strings.icontains(.file_name, "foobar") )` ### Outbound message to a specific email address [](https://docs.sublime.security/docs/common-snippets#outbound-message-to-a-specific-email-address) MQL `type.outbound and any(recipients.to, .email.email == "[[email protected]](https://docs.sublime.security/cdn-cgi/l/email-protection) ")` ### Specific mailer [](https://docs.sublime.security/docs/common-snippets#specific-mailer) MQL `strings.istarts_with(headers.mailer, "Foo")` ### Specific string in the current thread of the body [](https://docs.sublime.security/docs/common-snippets#specific-string-in-the-current-thread-of-the-body) MQL `strings.contains(body.current_thread.text, "foo bar")` ### Case insensitive string search in the current thread of the body [](https://docs.sublime.security/docs/common-snippets#case-insensitive-string-search-in-the-current-thread-of-the-body) MQL `strings.icontains(body.current_thread.text, "foo bar")` ### Link in the body using a specific TLD [](https://docs.sublime.security/docs/common-snippets#link-in-the-body-using-a-specific-tld) MQL `any(body.links, .href_url.domain.tld == "ru")` ### Sender / Return-path mismatch [](https://docs.sublime.security/docs/common-snippets#sender--return-path-mismatch) MQL `sender.email.email != headers.return_path.email` ### Match the subject using a case-insensitive regular expression [](https://docs.sublime.security/docs/common-snippets#match-the-subject-using-a-case-insensitive-regular-expression) MQL `regex.icontains(subject.subject, "foo.*bar")` ### Inbound message to a specific recipient email address [](https://docs.sublime.security/docs/common-snippets#inbound-message-to-a-specific-recipient-email-address) MQL `any(recipients.to, .email.email == "[[email protected]](https://docs.sublime.security/cdn-cgi/l/email-protection) ")` ### Any DMARC authentication failures [](https://docs.sublime.security/docs/common-snippets#any-dmarc-authentication-failures) MQL `any(headers.hops, .authentication_results.dmarc == "fail")` ### Failing SPF [](https://docs.sublime.security/docs/common-snippets#failing-spf) MQL `any(headers.hops, .authentication_results.spf == "fail")` ### Lookalike display name (hard-coded string) [](https://docs.sublime.security/docs/common-snippets#lookalike-display-name-hard-coded-string) MQL `// display name is within 3 character substitutions of "Jane Doe" strings.ilevenshtein(sender.display_name, "Jane Doe") < 3` ### Lookalike sender domain (hard-coded string) [](https://docs.sublime.security/docs/common-snippets#lookalike-sender-domain-hard-coded-string) MQL `// don't match exact org domain matches sender.email.domain.domain not in $org_domains // sender's display name is within 3 character substitutions of a specific hard-coded domain and strings.levenshtein(sender.email.domain.domain, "foo.com") < 3` ### Lookalike sender domain (dynamic $org\_domains) [](https://docs.sublime.security/docs/common-snippets#lookalike-sender-domain-dynamic-org_domains) MQL `// don't match exact org domain matches sender.email.domain.domain not in $org_domains // sender's display name is within 3 character substitutions of a known org domain and any($org_domains, strings.levenshtein(sender.email.domain.domain, .) < 3)` ### Email address is a To, CC, or BCC [](https://docs.sublime.security/docs/common-snippets#email-address-is-a-to-cc-or-bcc) MQL `// [[email protected]](https://docs.sublime.security/cdn-cgi/l/email-protection) is on the to, cc, or bcc line any([recipients.to, recipients.cc, recipients.bcc], any(., .email.email == "[[email protected]](https://docs.sublime.security/cdn-cgi/l/email-protection) ") )` ### Email has a link in the body or an attachment [](https://docs.sublime.security/docs/common-snippets#email-has-a-link-in-the-body-or-an-attachment) MQL `// set for a minimum number and length(body.links) > 0) or (length(attachments) > 0` ### Email has between X and Y links [](https://docs.sublime.security/docs/common-snippets#email-has-between-x-and-y-links) MQL `// set for an upper and lower bound and (0 < length(body.links) < 40)` Updated 2 months ago * * * --- # Strings functions `concat` [](https://docs.sublime.security/docs/strings-module#concat) ========================================================================= `strings.concat(sub: string, ...) -> string` **`concat`** combines multiple strings together in order. This could be be used later by other string comparisons or list lookups. ### Examples [](https://docs.sublime.security/docs/strings-module#examples) MQL `strings.concat("no-reply", "@sublimesecurity.com") -> "[[email protected]](https://docs.sublime.security/cdn-cgi/l/email-protection) " strings.concat(null, "@sublimesecurity.com") -> null strings.concat(sender.email.local_part, "@different_domain.com") -> "[[email protected]](https://docs.sublime.security/cdn-cgi/l/email-protection) " // Escaped email in a URL (@ escapes to %40) strings.concat(sender.email.local_part, "%40", sender.email.domain)` `contains` / `icontains` [](https://docs.sublime.security/docs/strings-module#contains--icontains) ====================================================================================================== `strings.contains(source: string, substring: string) -> bool` `strings.icontains(source: string, substring: string) -> bool` **`contains`** is used to check if one string contains a specific substring. `contains` performs a case-sensitive substring search, but `icontains` is case-insensitive. These strings can be literal strings, fields, or values from functions. If either string is `null`, then `contains` and `icontains` will return null. ### Examples [](https://docs.sublime.security/docs/strings-module#examples-1) MQL `strings.contains("[[email protected]](https://docs.sublime.security/cdn-cgi/l/email-protection) ", "@sublimesecurity.com") -> true strings.contains("[[email protected]](https://docs.sublime.security/cdn-cgi/l/email-protection) ", "@sublime") -> true strings.contains("[[email protected]](https://docs.sublime.security/cdn-cgi/l/email-protection) ", "@SuBLiMe") -> false strings.icontains("[[email protected]](https://docs.sublime.security/cdn-cgi/l/email-protection) ", "@SuBLiMe") -> true strings.icontains("[[email protected]](https://docs.sublime.security/cdn-cgi/l/email-protection) ", null) -> null` `count` / `icount` [](https://docs.sublime.security/docs/strings-module#count--icount) ========================================================================================== `strings.count(source: string, substring: string) -> bool` `strings.icount(source: string, substring: string) -> bool` **`count`** is used to count the number of occurrences of a substring within another string. `count` only counts `case-senstive` occurrences but `icount` counts all case-insensitive occurences. These strings can be literal strings, fields, or values from functions. If either string is `null`, then `count` and `icount` will return null. ### Examples [](https://docs.sublime.security/docs/strings-module#examples-2) MQL `strings.count("An example string to search", " ") -> 4 strings.count("Banana nana nana", "na") -> 6 strings.icount("BaNaNa NaNa NaNa", "NA") -> 6` `ends_with` / `iends_with` [](https://docs.sublime.security/docs/strings-module#ends_with--iends_with) ========================================================================================================== `strings.ends_with(source: string, suffix: string) -> bool` `strings.iends_with(source: string, suffix: string) -> bool` **`ends_with`** is used to check if one string has a specific suffix. `ends_with` performs a case-sensitive suffix check, but `iends_with` is case-insensitive. These strings can be literal strings, fields, or values from functions. If either string is `null`, then `contains` and `icontains` will return null. ### Examples [](https://docs.sublime.security/docs/strings-module#examples-3) MQL `strings.ends_with("[[email protected]](https://docs.sublime.security/cdn-cgi/l/email-protection) ", "@sublimesecurity.com") -> true strings.ends_with("[[email protected]](https://docs.sublime.security/cdn-cgi/l/email-protection) ", "@sublime") -> false strings.ends_with("[[email protected]](https://docs.sublime.security/cdn-cgi/l/email-protection) ", "@SublimeSecurity.com") -> false strings.iends_with("[[email protected]](https://docs.sublime.security/cdn-cgi/l/email-protection) ", "@SublimeSecurity.com") -> true strings.iends_with("[[email protected]](https://docs.sublime.security/cdn-cgi/l/email-protection) ", null) -> null` `like` / `ilike` [](https://docs.sublime.security/docs/strings-module#like--ilike) ====================================================================================== `strings.like(input: string, pattern: string, ...) -> bool` `strings.ilike(input: string, pattern: string, ...) -> bool` **`like`** is used to match a string against a list of predefined wildcard patterns. `like` performs a case-sensitive match against the entire string, but `ilike` is case-insensitive. Use wildcard characters to represent unknown substrings: * `*` is a placeholder for a string of any length, including empty strings * `?` is a placeholder for a single character within the string Remember that `like` and `ilike` are evaluated against the entire string. To look for a substring, wrap it in `*`, such as `strings.ilike(body.plain.raw, "*password*")`, or use `strings.icontains(body.plain.raw, "password")`. If `input` is `null`, then `like` and `ilike` will always return null. ### Examples [](https://docs.sublime.security/docs/strings-module#examples-4) MQL `strings.like("[[email protected]](https://docs.sublime.security/cdn-cgi/l/email-protection) ", "*@sublimesecurity.com") -> true # use ilike for case-insensitive matches strings.like("[[email protected]](https://docs.sublime.security/cdn-cgi/l/email-protection) ", "*@sublimesecurity.com") -> false strings.ilike("[[email protected]](https://docs.sublime.security/cdn-cgi/l/email-protection) ", "*@sublimesecurity.com") -> true # if multiple wildcard patterns are specified, only one needs to match strings.like("[[email protected]](https://docs.sublime.security/cdn-cgi/l/email-protection) ", "*@*.org", "*@*.com", "*@*.gov") -> true # use ? to match exactly 1 unknown character strings.like("[[email protected]](https://docs.sublime.security/cdn-cgi/l/email-protection) ", "[[email protected]](https://docs.sublime.security/cdn-cgi/l/email-protection) ") -> true strings.like("[[email protected]](https://docs.sublime.security/cdn-cgi/l/email-protection) ", "[[email protected]](https://docs.sublime.security/cdn-cgi/l/email-protection) ") -> false` `levenshtein` / `ilevenshtein` [](https://docs.sublime.security/docs/strings-module#levenshtein--ilevenshtein) ================================================================================================================== `strings.levenshtein(a: string, b: string) -> integer` `strings.ilevenshtein(a: string, b: string) -> integer` **`levenshtein`** is used to calculate the [Levenshtein edit distance](https://en.wikipedia.org/wiki/Levenshtein_distance) between two strings. The returned value is equal to the minimum number of character operations (deletion, insertion, substitution) required to transform one string into the other. `levenshtein` finds the case-sensitive distance between the two strings, but `ilevenshtein` is case-insensitive. MQL `strings.levenshtein('Bat', 'bot') -> 2 strings.ilevenshtein('Bat', 'bot') -> 1` `parse_domain` [](https://docs.sublime.security/docs/strings-module#parse_domain) ===================================================================================== `strings.parse_domain(text: string) -> Domain` **`strings.parse_domain`** is used to parse a string into an MDM Domain object, like the one at `sender.email.domain.root_domain`. * `text` (required): The input text of the domain If a domain cannot be parsed, the resulting object will be `null`. MQLResult `strings.parse_domain("www.sublimesecurity.com")` `{ "domain": { "domain": "www.sublimesecurity.com", "root_domain": "sublimesecurity.com", "sld": "sublimesecurity", "tld": "com", "valid": true } }` `parse_email` [](https://docs.sublime.security/docs/strings-module#parse_email) =================================================================================== `strings.parse_email(text: string) -> EmailAddress` **`strings.parse_email`** is used to parse a string into an MDM EmailAddress object, like the one at `sender.email`. * `text` (required): The input text of the email address. If an email address cannot be parsed, the resulting object will be `null`. MQLResult `strings.parse_email("[[email protected]](https://docs.sublime.security/cdn-cgi/l/email-protection) ")` `{ "email": "[[email protected]](https://docs.sublime.security/cdn-cgi/l/email-protection) ", "local_part": "foo", "domain": { "domain": "sublimesecurity.com", "root_domain": "sublimesecurity.com", "sld": "sublimesecurity", "tld": "com", "valid": true } }` `parse_json` [](https://docs.sublime.security/docs/strings-module#parse_json) ================================================================================= `strings.parse_json(j: string) -> object` **`strings.parse_json`** is used to parse a raw JSON string into an MQL JSON object. If the JSON cannot be parsed, the resulting object will be `null` You can traverse the resulting JSON object using the syntax as described on the [syntax page](https://docs.sublime.security/docs/syntax#json-type) `parse_url` [](https://docs.sublime.security/docs/strings-module#parse_url) =============================================================================== `strings.parse_url(text: string, strict=true) -> URL` **`strings.parse_url`** parses a string into an MDM URL object, such as the one at `body.links[].href_url`. * `text` (required): The input text of a URL. * `strict` (optional): If true, the URL must have an explicit schema, such as `https://`. To parse a URL without a scheme, manually set `strict = false` MQLResult `strings.parse_url("https://www.sublimesecurity.com")` `{ "url": "https://www.sublimesecurity.com", "domain": { "domain": "www.sublimesecurity.com", "root_domain": "sublimesecurity.com", "sld": "sublimesecurity", "subdomain": "www", "tld": "com", "valid": true }, "scheme": "https" }` MQLResult `strings.parse_url("www.sublimesecurity.com", strict=false)` `{ "url": "https://www.sublimesecurity.com", "domain": { "domain": "www.sublimesecurity.com", "root_domain": "sublimesecurity.com", "sld": "sublimesecurity", "subdomain": "www", "tld": "com", "valid": true }, "scheme": "https" }` `starts_with` / `istarts_with` [](https://docs.sublime.security/docs/strings-module#starts_with--istarts_with) ================================================================================================================== `strings.starts_with(source: string, prefix: string) -> bool` `strings.istarts_with(source: string, prefix: string) -> bool` **`starts_with`** is used to check if one string has a specific prefix. `istarts_with` performs a case-sensitive prefix check, but `iends_with` is case-insensitive. These strings can be literal strings, fields, or values from functions. If either string is `null`, then `contains` and `icontains` will return null. ### Examples [](https://docs.sublime.security/docs/strings-module#examples-5) MQL `strings.starts_with("[[email protected]](https://docs.sublime.security/cdn-cgi/l/email-protection) ", "no-reply") -> true strings.starts_with("[[email protected]](https://docs.sublime.security/cdn-cgi/l/email-protection) ", "@sublime") -> false strings.starts_with("[[email protected]](https://docs.sublime.security/cdn-cgi/l/email-protection) ", "No-Reply") -> false strings.istarts_with("[[email protected]](https://docs.sublime.security/cdn-cgi/l/email-protection) ", "No-Reply") -> true strings.istarts_with("[[email protected]](https://docs.sublime.security/cdn-cgi/l/email-protection) ", null) -> null` Updated 2 months ago * * * --- # Webhook Overview [](https://docs.sublime.security/docs/webhooks#overview) --------------------------------------------------------------------- A webhook Action sends a notification to a URL of your choosing when one or more Rules or Automations with the Action flag an email message. The HTTP notification includes information about the message, the mailbox, the flagged Rules and Automations, and any triggered Actions, so you can set up an unlimited variety of integrations in response to flagged message events. Here is some of what you can do with webhooks: * Send Flagged Message events to your SIEM for analytics or to correlate with events from other tools, such as your EDR * Send Flagged Message events to your SOAR, to trigger sophisticated DFIR workflows that are today only used on reported phish * Trigger an AWS Lambda function How to create a Webhook Action [](https://docs.sublime.security/docs/webhooks#how-to-create-a-webhook-action) ----------------------------------------------------------------------------------------------------------------- 1. Set up the endpoint URL in the external system that you want to send Flagged Message events to (e.g. your SIEM, SOAR, or an intermediary like AWS Lambda or Zapier). Some systems call an integration that provides a URL for receiving webhook notifications a "webhook integration". 2. Go to your Sublime Dashboard and click "Actions" 3. Click "New Action" and select "Webhook" 4. Give your webhook a name, provide the endpoint URL you generated externally in step 1, and optionally specify custom headers that will be included in the HTTP request for webhook notifications 5. Customize the payload to include Attack Score information and MDM properties such as headers or attachments. 6. Click "Save" ![](https://files.readme.io/ab2506cb1cdb3c18499759897c617cf047a4c60806073c8223d9837d439d6292-create_webhook.gif) How to add your new Webhook Action to Rules [](https://docs.sublime.security/docs/webhooks#how-to-add-your-new-webhook-action-to-rules) ------------------------------------------------------------------------------------------------------------------------------------------- 1. Click "Detection Rules" or "Automations" in the left nav of your Dashboard 2. Click on the Rule or Automation you wish to add the Action to, opening the detail view 3. Click "Edit" in the top right of this detail view 4. Click the "Actions" dropdown and select your new webhook Action 5. Click "Save Rule" Custom headers [](https://docs.sublime.security/docs/webhooks#custom-headers) --------------------------------------------------------------------------------- Any custom headers you configure for your webhook Actions will be be included in the HTTP requests for webhook notifications. You can include as many custom headers as you need. The webhook payload [](https://docs.sublime.security/docs/webhooks#the-webhook-payload) ------------------------------------------------------------------------------------------- Once you activate a webhook Action and connect it to a Rule, whenever that Rule flags a message, the endpoint for the webhook will receive an HTTP `POST` request with a body like the following: JSON `{ "id": "5e02026c-55c1-4cbb-8a18-76eb2f3e06d3", "api_version": "0", "created_at": "2021-06-30T16:17:22.937642Z", "type": "message.flagged", "data": { "message": { "id": "dff3be1d-b348-4dab-bb60-582842909a88", "canonical_id": "767f6519458092af9994a79ae73ca055e1365618", "external_id": "17a5db5ea7fa760d", "message_source_id": "a5843e7d-4ba5-40df-954e-73cb2f1e3e7e", "mailbox": { "id": "f0654b68-2686-491e-8fcb-e4ad61c61ad8", "external_id": "109412126579921477721" } }, "flagged_rules": [ { "id": "756b841c-4560-49b3-a8c0-dbacb2900aa5", "name": "Brand impersonation: Internal Revenue Service", "severity": "high", "tags": [ "brand-impersonation", "suspicious-sender" ] }, { "id": "67cfc6d8-3cbb-4338-ae8e-fffb0adccd12", "name": "Firebase storage link", "severity": "medium", "tags": [ "suspicious-content" ] } ], "triggered_actions": [ { "id": "6f45c807-a9c7-47a8-8c3e-5391e4f477ca", "name": "Auto-trash", "type": "trash_message" }, { "id": "8edd8b97-7b85-40d2-9045-a619ae9b4e52", "name": "Notify SIEM", "type": "webhook" } ] } }` ### Payload properties [](https://docs.sublime.security/docs/webhooks#payload-properties) The `data.message` property contains information about the message that triggered the webhook notification, a subset of the data available through the [retrieve message endpoint](https://docs.sublime.security/reference/getmessage-1) , including: * the Sublime-provided `id` of the message, which can be used to fetch additional information through the [retrieve message endpoint](https://docs.sublime.security/reference/getmessage-1) , including if and when the user read, forwarded, or replied to a message, as well as with other message endpoints like the endpoint to [retrieve an image of a message](https://docs.sublime.security/reference/getmessageimage-1) * the `canonical_id` for the [message group](https://docs.sublime.security/docs/message-groups) the message belongs to, which can be used with message group API endpoints like [trash message group](https://docs.sublime.security/reference/trashmessagecanonicalgroup-1) The `data.triggered_actions` array contains a list of actions triggered for that specific message. The `type` property can include any of the following ENUM values: `"trash_message" "auto_review" "dismiss_message" "quarantine_message" "warning_banner" "restore_message" "track_link_clicks" "move_to_spam" "report_message" "webhook"` ### Source IPs [](https://docs.sublime.security/docs/webhooks#source-ips) If the domain receiving your webhook supports IPv6, you'll see frequently changing IPv6 addresses making the calls to your endpoint. If only IPv4 is used, you'll see stable IPs making the calls to your endpoint. If you're using one of Sublime's Managed deployments you can check the table below. If you're using AWS CloudFormation, we recommending sending several test requests to establish to two IP addresses used by your deployment. And if you're using docker-compose, the IP address may vary or be stable based on the networking of the instance docker compose is running on. | Managed Instance | IPs | | --- | --- | | platform.sublime.security | `44.194.118.42`, `34.197.67.5` | | eu.platform.sublime.security | `54.247.166.54`, `52.214.238.48` | | au.platform.sublime.security | `13.238.69.11`, `54.252.178.93` | | ca.platform.sublime.security | `15.157.18.163`, `3.97.76.222` | You may choose to create an IP allowlist based on these IPs, although we recommend checking the webhook signature instead. Responding to webhook notifications [](https://docs.sublime.security/docs/webhooks#responding-to-webhook-notifications) --------------------------------------------------------------------------------------------------------------------------- The endpoint URL you configure for a webhook should respond to the webhook's HTTP requests with an HTTP response code starting with `2` (for example, `200`) within 10 seconds. If your integration needs to do anything in response to webhook notifications that may take more than 10 seconds, it should send an HTTP response before doing that work. Check the webhook signature [](https://docs.sublime.security/docs/webhooks#check-the-webhook-signature) ----------------------------------------------------------------------------------------------------------- Webhook notifications include a signature in the `X-Sublime-Signature` HTTP header. This allows you to verify that the requests were sent by Sublime, not some other party. Before you can verify signatures, you need to retrieve your webhook Action's signing secret. Select a webhook Action from the "Actions" page, then click "Click to reveal", or click the "Copy" button to copy the secret directly to your clipboard. The `X-Sublime-Signature` header included in each request contains a timestamp and one or more signatures. The timestamp is prefixed by `t=`, and each signature is prefixed by a scheme. Schemes start with `v`, followed by an integer. Currently, the only valid signature scheme is `v0`. Example X-Sublime-Signature `t=1626139320,v0=3d35d777510a7ccf73ded723f141b0ada2986aebe63c4e45a9c7e4bb1abaca06` Sublime generates signatures using a hash-based message authentication code ([HMAC](https://en.wikipedia.org/wiki/HMAC) ) with [SHA-256](https://en.wikipedia.org/wiki/SHA-2) . To prevent downgrade attacks, you should ignore all schemes that are not `v0`. ### Preventing replay attacks [](https://docs.sublime.security/docs/webhooks#preventing-replay-attacks) A [replay attack](https://en.wikipedia.org/wiki/Replay_attack) is when an attacker intercepts a valid payload and its signature, then re-transmits them. To mitigate such attacks, Sublime includes a timestamp in the `X-Sublime-Signature` header. Because this timestamp is part of the signed payload, it is also verified by the signature, so an attacker cannot change the timestamp without invalidating the signature. If the signature is valid but the timestamp is too old, you can have your application reject the payload. Sublime generates the timestamp and signature each time a webhook notification is sent to your endpoint. If Sublime retries a notification (for example, your endpoint previously replied with a non-2xx status code), then a new signature and timestamp is generated for the new delivery attempt. ### Verifying signatures [](https://docs.sublime.security/docs/webhooks#verifying-signatures) Follow these steps to verify a request's signature: #### Step 1: Extract the timestamp and signatures from the header [](https://docs.sublime.security/docs/webhooks#step-1-extract-the-timestamp-and-signatures-from-the-header) Split the header, using the `,` character as the separator, to get a list of elements. Then split each element, using the `=` character as the separator, to get a prefix and value pair. The value for the prefix `t` corresponds to the timestamp, and `v0` corresponds to the signature (or signatures). You can discard all other elements. #### Step 2: Prepare the `signed_payload` string [](https://docs.sublime.security/docs/webhooks#step-2-prepare-the-signed_payload-string) The `signed_payload` string is created by concatenating: 1. The timestamp (as a string) 2. The character `.` 3. The actual JSON payload (that is, the request body) #### Step 3: Determine the expected signature [](https://docs.sublime.security/docs/webhooks#step-3-determine-the-expected-signature) Compute an HMAC with the SHA256 hash function. Use the webhook's signing secret as the key, and use the `signed_payload` string as the message. #### Step 4: Compare the signatures [](https://docs.sublime.security/docs/webhooks#step-4-compare-the-signatures) Compare the signature (or signatures) in the header to the expected signature. For an equality match, compute the difference between the current timestamp and the received timestamp, then decide if the difference is within your tolerance. To protect against timing attacks, use a constant-time string comparison to compare the expected signature to each of the received signatures. Updated about 1 month ago * * * --- # Syntax Extract fields from the MDM [](https://docs.sublime.security/docs/syntax#extract-fields-from-the-mdm) --------------------------------------------------------------------------------------------------------- In Message Query Language (MQL), fields can be extracted from the MDM by specifying the field name. To get a subfield, combine two fields together with a `.` in between. For example, to retrieve the sender's display name: MQL `sender.display_name` Literal values [](https://docs.sublime.security/docs/syntax#literal-values) ------------------------------------------------------------------------------- **Strings** can be specified in two forms: _escaped_ and _unescaped_. _Escaped strings_ are surrounded by double quotes `"` and special characters can be escaped with `\`. If an unrecognized character is specified after `\`, that is treated as an invalid escape sequence and causes a syntax error. Strings `"hello world" "line 1\nline2\nline3" "unicode characters like ✉️ are supported"` _Unescaped_ or _raw_ strings are surrounded by single quotes `'`. No escape sequences are supported within a raw string. However, two single quotes `''` can be used to insert a single quote character. Strings `'hello world' 'escaping apostrophes isn''t that difficult' 'this back\slash is interpreted literally'` ### Escape sequences [](https://docs.sublime.security/docs/syntax#escape-sequences) * `\r` carriage return CR (ASCII 0x0d) * `\n` new line LF (ASCII 0x0a) * `\t` tab (ASCII 0x09) * `\'` single quotes `'` (ASCII 0x27) * `\"` double quotes `"` (ASCII 0x22) * `\\` backslash `\` (ASCII 0x5c) * `\u{xxxxxxxx}` Unicode code point between 0x01 and 0x10ffff. Unicode escape sequences can include any valid unicode code point, including ASCII characters, non-printable characters or other unicode characters. Between 2 and 8 hex digits are recognized within `{` and `}`. Example unicode escapes: * `\u{0a}` identical to `\n` for a newline * `\u{0398}` greek capital theta: Θ * `\u{200f}` unicode right-to-left encoding character * `\u{1f4ec}` open mailbox emoji: 📬 * `\u{0001f4ec}` open mailbox emoji, with optional leading zeros: 📬 Comparing values [](https://docs.sublime.security/docs/syntax#comparing-values) ----------------------------------------------------------------------------------- MQL provides eight built-in operators to compare two values. The syntax is ` `: MQL `sender.email.domain.root_domain == "sublimesecurity.com"` **Numbers** can be compared with the below operators. MQL can represent unsigned, signed, or floating-point values. * `<`: less than * `<=`: less than or equal to * `==`: equal to * `!=`: not equal to * `>=`: greater than or equal to * `>`: greater than _Note_: When two values are compared of different numeric types, they are automatically converted to have compatible types. For example, in the comparison `1 < 1.5`, the integer `1` is compared to a floating point `1.5`. First, the `1` is converted to a floating point `1.0`, and then the comparison is performed. This means that an expression like `3 == 3.14` will always evaluate false because it's converted to `3.0 == 3.14`. **Strings** support two additional operators to support case-insensitive equality. Unless explicitly specified, assume that strings are interpreted with case-sensitivity (meaning that `"a"` and `"A"` are distinct). Range operators, such as `<` use lexicographical ordering and are always case-sensitive. _case-sensitive comparisons_ * `<`: less than * `<=`: less than or equal to * `==`: equal to (`"Abc" == "abc"` evaluates as `false`) * `!=`: not equal to * `>=`: greater than or equal to * `>`: greater than _case-insensitive comparisons_ * `=~`: case-insensitive equality. (`"Abc" =~ "abc"` evaluates as `true`) * `!~`: case-insensitive inequality. (`"Abc" !~ "abc"` evaluates as `false`) **Booleans** can only be compared with `==` or `!=`. Booleans also have dedicated operators for traditional boolean logic, such as `and`. (see the next section). ### Range checking [](https://docs.sublime.security/docs/syntax#range-checking) A common pattern when comparing values is to check if a value is within a given range. MQL provides syntax sugar for this kind of comparison with the syntax ` x `. Some examples: MQL `strings.levenshtein(sender.email.email, "[[email protected]](https://docs.sublime.security/cdn-cgi/l/email-protection) ") > 4 and strings.levenshtein(sender.email.email, "[[email protected]](https://docs.sublime.security/cdn-cgi/l/email-protection) ") <= 7 # more compact form 4 < strings.levenshtein(sender.email.email, "[[email protected]](https://docs.sublime.security/cdn-cgi/l/email-protection) ") <= 7` MQL `'abc' <= subject.subject < 'xyz'` Like other [comparisons](https://docs.sublime.security/docs/syntax#comparing-values) , both **strings** and **numbers** are supported; however, range checking only supports the following operators: * `<`: less than * `<=`: less than or equal to Boolean logic [](https://docs.sublime.security/docs/syntax#boolean-logic) ----------------------------------------------------------------------------- Multiple boolean expressions can be combined with traditional boolean operators. Use the `and`, `or`, or `not` keywords for boolean operations: MQL `sender.email.email == "[[email protected]](https://docs.sublime.security/cdn-cgi/l/email-protection) " and subject.subject == "Password reset request" sender.email.email == "[[email protected]](https://docs.sublime.security/cdn-cgi/l/email-protection) " or subject.subject == "Password reset request" not (sender.email.email == "[[email protected]](https://docs.sublime.security/cdn-cgi/l/email-protection) ")` Check against multiple values with `in` [](https://docs.sublime.security/docs/syntax#check-against-multiple-values-with-in) ------------------------------------------------------------------------------------------------------------------------------- One common pattern that often combines [comparisons](https://docs.sublime.security/docs/syntax#comparing-values) with [boolean logic](https://docs.sublime.security/docs/syntax#boolean-logic) is to check the same field against multiple literal values. Instead of combining similar equality checks `==` with a logical `or`, use the `in` keyword to check a value against a set of values: For example, to detect subjects commonly used in BEC attacks: MQL ``subject.subject == "Urgent" or subject.subject == "Can you help" or subject.subject == "Quick errand" # more compact form with `in` subject.subject in ("urgent", "can you help", "quick errand")`` To check the inverse and ensure that a value is _not_ in a list of values, use `not in`: MQL `subject.subject not in ("Urgent", "Can you help", "Quick errand")` `in` can also be used in a case-insensitive way, using `in~`: MQL `subject.subject in~ ("Urgent", "Can you help", "Quick errand")` Checking array fields with `in` [](https://docs.sublime.security/docs/syntax#checking-array-fields-with-in) --------------------------------------------------------------------------------------------------------------- The syntax for `in` and `in~` can also be used to lookup a single value against an array field or function that returns an array. The syntax `x in array_field` is simply shorthand for `any(some_array, . == x)`. Can be used with [`file.explode`](https://docs.sublime.security/docs/enrichment-functions#fileexplode) : MQL `any(attachments, .file_extension in~ ('html', 'htm') and any(file.explode(.), any(.scan.javascript.identifiers, . == "unescape")) )` With `in` to check `.scan.javascript.identifiers`: MQL `any(attachments, .file_extension in~ ('html', 'htm') and any(file.explode(.), "unescape" in .scan.javascript.identifiers) )` Matching several boolean expressions with `of` [](https://docs.sublime.security/docs/syntax#matching-several-boolean-expressions-with-of) --------------------------------------------------------------------------------------------------------------------------------------------- Occasionally when writing rules, a minimum amount of boolean clauses must match. With `and`, all the clauses must be true and with `or` at least one clause has to be true. Use `of` for a hybrid operator when checking for a minimum amount of matches. `X of (...)` evaluates true if at least X terms evaluate `true`. The basic structure for `of` follows: MQL `X of (clause1, clause2, ..., clauseN)` Using `1 of (...)` is identical to an `or` over all of the terms. Similarly, if `X` is the same as the total number of terms, then it's equivalent to `and`. The minimum number of clauses to check with `of` must be between 1 and the total number of clauses between `(` and `)`. For example, MQL `# returns true 3 of (true, false, true, true) # returns false 3 of (true, false, false, true)` Checking a named list with `in` [](https://docs.sublime.security/docs/syntax#checking-a-named-list-with-in) --------------------------------------------------------------------------------------------------------------- A value can be checked against a list by using `in $list` or `not in $list` syntax. See the [full reference of currently available lists](https://docs.sublime.security/docs/lists) . MQL `sender.email.domain.domain in $alexa_1m sender.email.domain.domain not in $alexa_1m` Creating arrays [](https://docs.sublime.security/docs/syntax#creating-arrays) --------------------------------------------------------------------------------- Create arrays on-the-fly with a list of literal or dynamic values. Arrays can be combined with array functions such as `any` and `all` to consolidate logic. To create an array, encapsulate a list of values in `[` `]` MQL `["foo", "bar", "baz"] [body.plain.text, body.html.text]` For example, to check if either `body.plain.text` or `body.html.text` contain a Social Security Number: MQL `any([body.plain.text, body.html.text], regex.contains(., '\b(\d\d\d)-(\d\d)-(\d\d\d\d)\b') )` Using `any` with an custom array is equivalent to writing an `or` with two `regex.contains` calls: MQL `regex.contains(body.plain.text, '\b(\d\d\d)-(\d\d)-(\d\d\d\d)\b') or regex.contains(body.html.text, '\b(\d\d\d)-(\d\d)-(\d\d\d\d)\b')` To check if any of the recipients is a disposable email provider: MQL `any([recipients.to, recipients.cc, recipients.bcc], any(., .email.domain.domain in $disposable_email_providers) )` See [array functions](https://docs.sublime.security/docs/functions#array-functions) for the full list of array functions. Arithmetic [](https://docs.sublime.security/docs/syntax#arithmetic) ----------------------------------------------------------------------- All numbers support standard arithmetic operations. When two numbers of different types are used in arithmetic they are first converted to a matching type. That means `1 * 2.0` is automatically converted to `1.0 * 2.0`. The supported arithmetic operations are: * `+` Add two numbers * `-` Subtract two numbers * `*` Multiply two numbers * `/` Divide two numbers. If both values are integers, this indicates _integer_ division, meaning that `5 / 2` is `2`. To get `2.5`, add a decimal point to either side: `5 / 2.0`, `5.0 / 2` and `5.0 / 2.0` are all equivalent. * `%` Modulo of two numbers JSON type [](https://docs.sublime.security/docs/syntax#json-type) --------------------------------------------------------------------- To parse a JSON string to the JSON type, see `strings.parse_json` on the [strings functions page](https://docs.sublime.security/docs/strings-module#parse_json) In addition to recognizing the specific schema for the Message Data Model and functions, MQL can traverse an arbitrary, schemaless JSON document. Because the schema isn't defined, the syntax to retrieve keys is different than it is from fields on the MDM. JSON objects build a hierarchy. Traversing a JSON type in MQL results in intermediate JSON typed values which can be used by any other part of MQL. The JSON hierachy is: * `{string: JSON}` * `boolean` * `string` * `float` * `[JSON]` For example, if there was a JSON object in a field `.json` that looks like: JSON `{ "company": "Sublime Security", "$url": "https://sublime.security", "meta": { "name": "Sublime Security · The new standard for email security", "description": "Sublime is an adaptive email security platform that combines best-in-class effectiveness with unprecedented visibility and control.", "image": "https://cdn.prod.website-files.com/6734d4696c8f76142b33121b/6759b520893a25f4c27f5024_OG_Image.png" }, "is_responsive": true, "nav": [ "Solutions", "Platform", "About Us", "Blog" ] }` JSON can be traversed in MQL with `[]` syntax to access a key: MQL `.json["company"] # "Sublime Security"` To compare JSON, use it freely with any normal operators: MQL `.json["company"] == "Sublime Security"` Or pass JSON data into functions, where the types are already matched MQL `strings.ilike(.json["$url"], "https://*.security")` Iterating arrays is also possible with [array functions](https://docs.sublime.security/docs/functions#array-looping-functions) : MQL `any(.json["nav"], . == "Solutions")` However, interpreting a JSON as a `boolean` needs to be explicit. This is to make it more clear and readable that in MQL, boolean logic is exact and doesn't operate on ["truthy"](https://en.wikipedia.org/wiki/Truth_value#Computing) values (e.g. in Python, `[]` can be interpreted as `false` in boolean logic). MQL `# Not allowed in MQL not .json["is_responsive"] # Explicit boolean logic is required for JSON types .json["is_responsive"] == true .json["is_responsive"] == false not .json["is_responsive"] .json["company"] == "Sublime Security"` Because JSON has a dynamic type, mismatching types will result in a `null` result: MQL ``# Mismatched type, results in `null` .json["company"] == 100 .json["company"] == .json["nav"]`` Order of operations [](https://docs.sublime.security/docs/syntax#order-of-operations) ----------------------------------------------------------------------------------------- The precedence of operations, ordered from highest to lowest: * `(` ... `)`: parentheses * `*`, `/` and `%`: multiplication, division, and modulus * `+` and `-`: addition and subtraction * `<`, `<=`, `==`, `=~`, `!=`, `!~`, `>=`, `>`, `in`: comparisons * `of`: matching multiple boolean terms * `not`: boolean NOT * `and`: boolean AND * `or`: boolean OR ### Common logic errors [](https://docs.sublime.security/docs/syntax#common-logic-errors) **Missing parentheses** The following rule will match any message with "microsoft team" in the body, regardless of whether `type.inbound` is true. MQL `type.inbound and strings.ilike(subject.subject, "*microsoft team*") or strings.ilike(body.html.inner_text, "*microsoft team*") ^^-- oops!` That's the same as typing: MQL `( type.inbound and strings.ilike(subject.subject, "*microsoft team*") ) or strings.ilike(body.html.inner_text, "*microsoft team*") ^^-- oops!` If you intend for the `or` to cover the two `ilike` checks, you need parentheses to make the order of operations explicit. That way, `type.inbound` must always evaluate as `true`: MQL `type.inbound and ( strings.ilike(subject.subject, "*microsoft team*") or strings.ilike(body.html.inner_text, "*microsoft team*") )` Comments [](https://docs.sublime.security/docs/syntax#comments) ------------------------------------------------------------------- MQL supports single-line comments beginning with `//`. MQL `type.inbound // check if the sender's domain uses the ru TLD and sender.email.domain.tld != 'ru'` Updated 4 months ago * * * --- # Enrichment functions MQL is highly extensible and can integrate with virtually any tool or service to build better detection rules. > 📘 > > Request a function! > > > ------------------------- > > Don't see a function you want? Let us know via [email](https://docs.sublime.security/cdn-cgi/l/email-protection#afc7cac3c3c0efdcdacdc3c6c2ca81dccaccdaddc6dbd6) > or Slack! `beta.message_screenshot` [](https://docs.sublime.security/docs/enrichment-functions#betamessage_screenshot) ---------------------------------------------------------------------------------------------------------------- `beta.message_screenshot() → File` The `beta.message_screenshot` function takes a screenshot of the message using the message body's HTML section. This screenshot is the same as the one that shows in the Message Preview pane when viewing a message, and is a representation of what the end-user would see. The resulting file can be passed into other File analysis functions, such as `file.explode` or `ml.logo_detect`: MQL `// Check for an embedded Microsoft logo any(ml.logo_detect(beta.message_screenshot()).brands, .name == "Microsoft" and .confidence in ("medium", "high") ) // Run OCR on a screenshot of the message any(file.explode(beta.message_screenshot()), strings.ilike(.scan.ocr.raw, "*free cooler*") )` [View detection rules that use this function](https://github.com/search?q=beta.message_screenshot+repo%3Asublime-security%2Fsublime-rules+path%3Adetection-rules%2F&type=Code&ref=advsearch&l=&l=) beta.scan\_base64 [](https://docs.sublime.security/docs/enrichment-functions#betascan_base64) ------------------------------------------------------------------------------------------------- ### Overview [](https://docs.sublime.security/docs/enrichment-functions#overview) The `beta.scan_base64` function identifies base64 encoded strings within text content and decodes them into readable strings. This is particularly useful for detecting phishing attacks where threat actors encode sensitive information like recipient email addresses within URLs or QR codes. #### Key Use Cases [](https://docs.sublime.security/docs/enrichment-functions#key-use-cases) * Detecting recipient information encoded in phishing URLs * Identifying encoded domains or usernames in message content * Analyzing HTML attachments for encoded data ### Technical Specification [](https://docs.sublime.security/docs/enrichment-functions#technical-specification) #### Syntax [](https://docs.sublime.security/docs/enrichment-functions#syntax) MQL `beta.scan_base64( text: string, encodings: [“ascii”, “utf8”, “utf16” “latin1”, “windows1252”], ignore_padding: bool ) -> []string` #### Parameters [](https://docs.sublime.security/docs/enrichment-functions#parameters) * `text` (required): The input text to scan for base64 encoded content * `encodings` (optional): Array of string encodings to attempt when decoding base64 content * Supported values: “ascii”, “utf8”, “utf16” “latin1”, “windows1252” * Default: \[“ascii”, “utf8”, “latin1”, “windows1252”\] * `ignore_padding` (optional): Boolean flag to control base64 padding handling * Default: false * When true: Attempts to decode even with missing padding * When false: Strictly enforces proper base64 padding #### Return Value [](https://docs.sublime.security/docs/enrichment-functions#return-value) Returns an array of successfully decoded strings that were found in the input text. If no base64 string is found returns an empty array of type string. ### Usage Guide [](https://docs.sublime.security/docs/enrichment-functions#usage-guide) #### HTML attachments with recipient email [](https://docs.sublime.security/docs/enrichment-functions#html-attachments-with-recipient-email) MQL `type.inbound // other rule criteria and any(attachments, .file_type == "html" and any(beta.scan_base64(file.parse_html(.).raw), any(recipients.to, strings.icontains(.., .email.email)) ) )` #### PDF attachment(s) with URLs containing the recipient domain [](https://docs.sublime.security/docs/enrichment-functions#pdf-attachments-with-urls-containing-the-recipient-domain) MQL `type.inbound // other rule criteria and any(attachments, .file_type == "pdf" and any(file.explode(.), any(.scan.url.urls, any(beta.scan_base64(.url), any(recipients.to, strings.icontains(.., .email.domain.root_domain) ) ) ) ) )` #### URL redirect history or fragment containing encoded recipient email [](https://docs.sublime.security/docs/enrichment-functions#url-redirect-history-or-fragment-containing-encoded-recipient-email) MQL `type.inbound // other rule criteria and any(body.links, any(ml.link_analysis(.).redirect_history, any(beta.scan_base64(.query_params), any(recipients.to, strings.icontains(.., .email.email)) ) or any(beta.scan_base64(.fragment), any(recipients.to, strings.icontains(.., .email.email)) ) ) )` #### Attachment(s) with a QR code with URLs containing the recipient email [](https://docs.sublime.security/docs/enrichment-functions#attachments-with-a-qr-code-with-urls-containing-the-recipient-email) MQL `type.inbound // other rule criteria and any(attachments, (.file_type in $file_types_images or .file_extension in $file_extensions_macros or .file_type == "pdf") and any(file.explode(.), any(recipients.to, any(beta.scan_base64(..scan.qr.url.url, ignore_padding=true), strings.icontains(., ..email.email) ) ) ) )` File Analysis [](https://docs.sublime.security/docs/enrichment-functions#file-analysis) =========================================================================================== Files can be delivered via email in a variety of ways, including directly as an attachment or auto-downloaded via links. `file.explode` [](https://docs.sublime.security/docs/enrichment-functions#fileexplode) ------------------------------------------------------------------------------------------ `file.explode(input: File | HTML) -> [FileExplodeOutput]` FileExplode uses [Strelka](https://github.com/sublime-security/strelka/) , a file extraction and metadata collection system developed by Target. Strelka uses a variety of scanners to parse files of a specific flavor and performs data collection and/or file extraction on them. Strelka can recursively extract nested files (like a Word doc within a Zip file), identify malicious scripts, suspicious executables and text, run analysis like OCR and Macro detection, and more. For more information on how Strelka works, see the official [Strelka documentation](https://target.github.io/strelka/) . For a list of all available scanners, see the [Github repo](https://github.com/sublime-security/strelka/tree/main/src/python/strelka/scanners) or the official [Strelka docs](https://target.github.io/strelka/#/?id=scanners) . [View detection rules that use this function](https://github.com/search?q=file.explode+repo%3Asublime-security%2Fsublime-rules+path%3Adetection-rules%2F&type=Code&ref=advsearch&l=&l=) MQL `// detect HTML smuggling techniques any(attachments, .file_extension in~ ('html', 'htm') and any(file.explode(.), "unescape" in .scan.javascript.identifiers) ) // detect encrypted zip files any(attachments, any(file.explode(.), 'encrypted_zip' in .flavors.yara ) ) // detect attachments soliciting the user to enable macros using OCR any(attachments, any(file.explode(.), strings.icontains(.scan.ocr.raw, "enable macros") ) ) // detect macros with auto-open any(attachments, any(file.explode(.), any(.scan.vba.auto_exec, . == "AutoOpen") ) ) // detect macros calling an exe any(attachments, any(file.explode(.), any(.scan.vba.hex, strings.ilike(., "*exe*")) ) )` iframe `file.html_screenshot` [](https://docs.sublime.security/docs/enrichment-functions#filehtml_screenshot) ---------------------------------------------------------------------------------------------------------- `file.html_screenshot(input: File) -> File` The `file.html_screenshot` function takes a screenshot of HTML files so that you can query the image. This allows you to run logo detect on HTML attachments — `ml.logo_detect(file.html_screenshot(.))` — or send the result to `file.explode`, empowering you to run OCR and QR analysis. MQL `any(attachments, ( .file_extension in~ ("html", "htm", "shtml", "dhtml") or .file_type == "html" or .content_type == "text/html" ) and any(ml.logo_detect(file.html_screenshot(.)).brands, .name != null and .confidence in ("medium", "high") ) )` `file.oletools` [](https://docs.sublime.security/docs/enrichment-functions#fileoletools) -------------------------------------------------------------------------------------------- `file.oletools(input: File) -> OleToolsOutput` [Oletools](https://github.com/decalage2/oletools/) , developed by [Philippe Lagadec](http://www.decalage.info/python/oletools) , analyzes Microsoft OLE2 files such as Microsoft Office documents for malware and other suspicious indicators. Use **`file.oletools`** to analyze attachments for malware or suspicious indicators like VBA macros, remote OLE objects, encryption, and more. [View detection rules that use this function](https://github.com/sublime-security/sublime-rules/search?q=file.oletools) MQL `// detect suspicious macros any(attachments, file.oletools(.).indicators.vba_macros.exists) any(attachments, file.oletools(.).indicators.vba_macros.risk == "high") // detect potential attempts to exploit CVE-2021-40444 (https://msrc.microsoft.com/update-guide/vulnerability/CVE-2021-40444) any(attachments, any(file.oletools(.).relationships, strings.ilike(.target, "*html:http*"))) // detect external OLE object relationships any(attachments, file.oletools(.).indicators.external_relationships.count > 0) // detect encrypted Office documents any(attachments, file.oletools(.).indicators.encryption.exists) // detect macros that attempt to auto-execute when the document is opened any(attachments, any(file.oletools(.).macros.keywords, .type == "autoexec")) // detect suspicious macro source code any(attachments, strings.ilike(file.oletools(.).macros.vba_code_all_modules, "*kernel32*", "*GetProcessId*"))` iframe `file.parse_eml` [](https://docs.sublime.security/docs/enrichment-functions#fileparse_eml) ---------------------------------------------------------------------------------------------- `file.parse_eml(input: Attachment) -> MessageDataModel` The `file.parse_eml` function takes in an EML attachment (file extension `.eml` or content type `message/rfc822`) and parses it into an MDM. MQL `any(attachments, (.file_extension == "eml" or .content_type == "message/rfc822") and strings.icontains(file.parse_eml(.).subject.subject, "invoice") )` `file.parse_html` [](https://docs.sublime.security/docs/enrichment-functions#fileparse_html) ------------------------------------------------------------------------------------------------ `file.parse_html(input: File) -> HTML` The `file.parse_html` function parses an HTML file from an attachment, returning the full `raw` along with `display_text`, `inner_text`. This empowers detections such as running NLU on the `display_text` and completing a regex on the HTML without custom scanners or YARA signatures. MQL `any(attachments, ( .file_extension in~ ("html", "htm", "shtml", "dhtml") or .file_type == "html" ) and regex.icontains(file.parse_html(.).raw, "fromCharCode", "charCodeAt", "charAt", "parseInt" ) )` `file.parse_text` [](https://docs.sublime.security/docs/enrichment-functions#fileparse_text) ------------------------------------------------------------------------------------------------ `file.parse_text(input: File) -> ParseTextOutput` The `file.parse_text` function parses a file from an attachment, returning the decoded `text`. MQL `any(attachments, ( .file_extension in~ ("html", "htm", "shtml", "dhtml") or .file_type == "html" ) and strings.icontains(file.parse_text(.).text, "invoice") )` iframe Machine Learning functions [](https://docs.sublime.security/docs/enrichment-functions#machine-learning-functions) ===================================================================================================================== `ml.link_analysis` [](https://docs.sublime.security/docs/enrichment-functions#mllink_analysis) -------------------------------------------------------------------------------------------------- `ml.link_analysis(input: Link | URL, mode="default") → LinkAnalysisOutput` LinkAnalysis analyzes a link and classifies them as benign or suspicious. The service sends suspicious URLs to a headless browser which resolves the effective URL and collects a screenshot. The screenshot is sent to an object detection model to detect brand logos, buttons, and input forms. We chose [Phishpedia](https://github.com/lindsey98/Phishpedia) , an Open Source object detection project as our baseline model architecture. If any logos are detected, those logos are cropped from the original screenshot and compared to a set of protected brand logos commonly used in credential phishing attacks. Discovered brands are available to MQL, along with summary information about login input boxes or captchas in the screenshot. `mode` is an optional argument that alters LinkAnalysis's analysis criteria (see note below). By changing `mode` from its default of `"default"` to `"aggressive"`, LinkAnalysis performs extra processing on a link when determining whether to fully analyze the link. For example, LinkAnalysis with `mode="aggressive"` will fetch the destination link of known common click trackers via HEAD and apply normal analysis criteria to that destination link. [View rules that use this function](https://github.com/sublime-security/sublime-rules/search?q=ml.link_analysis) MQL `// detect links to credential phishing pages any(body.links, all([ml.link_analysis(.)], .credphish.disposition == "phishing" and .credphish.brand.confidence in ("medium", "high") ) ) // detect any links to credential phishing pages any(body.links, any([ml.link_analysis(., mode="aggressive")], .credphish.disposition == "phishing" and .credphish.brand.confidence in ("medium", "high") ) ) // detect free subdomain links with a login or captcha any(body.links, all([ml.link_analysis(.)], ( .credphish.contains_login or .credphish.contains_captcha ) and ( .effective_url.domain.root_domain in $free_subdomain_hosts or .original_url.domain.root_domain in $free_subdomain_hosts )) ) // analyze the final DOM of a link within the body any(body.links, strings.icontains(ml.link_analysis(.).final_dom.display_text, "Redirect Notice") and strings.contains(ml.link_analysis(.).final_dom.display_text, ".zip") )` > 📘 > > Analysis criteria > > > ----------------------- > > In order to prevent LinkAnalysis from "clicking" on every link, such as Unsubscribes and one-time password resets, LinkAnalysis uses a URL classification model to determine which links to actually send to the service for analysis. > > You can check whether LinkAnalysis `submitted`, `retrieved`, or `analyzed` the target page by [inspecting the response in the MQL editor](https://docs.sublime.security/docs/using-the-mql-editor#testing-and-debugging-rules) > . > > If you observe LinkAnalysis analyzing links it shouldn't or not analyzing links it should, please [send us an email](https://docs.sublime.security/cdn-cgi/l/email-protection#10636560607f6264506365727c797d753e6375736562796469) > or post in the Slack Community. iframe `ml.logo_detect` [](https://docs.sublime.security/docs/enrichment-functions#mllogo_detect) ---------------------------------------------------------------------------------------------- `ml.logo_detect(input: File) -> [LogoDetectOutput]` LogoDetect uses computer vision to detect common brand logos used in attachment-based credential phishing attacks, such as impersonations of PayPal, Adobe, Microsoft, Outlook, Office365, DocuSign, and more. This includes embedded images in the body of messages as CIDs. Our object detection model identifies logos, which are then cropped into separate images. These images are passed through a Siamese Neural Network to generate a feature vector. We compare this vector to a database of known logos using a similarity calculation. If the score exceeds a predetermined threshold, we confirm it as the respective brand logo. For text-based logos, we utilize OCR, a computer vision technique for extracting text from images. Combined with Siamese Networks, this approach ensures comprehensive logo detection. [View Rules that use this function](https://github.com/search?q=repo%3Asublime-security%2Fsublime-rules%20logo_detect&type=code) MQL `// detect SharePoint logos in attached images any(attachments, .file_type in ('png', 'jpeg', 'jpg', 'bmp') and any(ml.logo_detect(.).brands, .name == "Microsoft SharePoint") ) // detect DocuSign logos in attached images any(attachments, .file_type in ('png', 'jpeg', 'jpg', 'bmp') and any(ml.logo_detect(.).brands, .name == "DocuSign") ) // detect Norton logos in attached PDFs any(attachments, .file_type == "pdf" and any(ml.logo_detect(.).brands, .name == "Norton") )` ### List of Supported Brands [](https://docs.sublime.security/docs/enrichment-functions#list-of-supported-brands) Don't see the brand you're looking for? Want to be able to detect your own company's logo? Contact your account team or send an email to [\[email protected\]](https://docs.sublime.security/cdn-cgi/l/email-protection#02717772726d7076427177606e6b6f672c71676177706b767b) with a few examples of the logo and we'll get back to you. Brands `ABN Amro ADP AT&T Adobe Amazon American Express Apple BB&T Corporation Bank of America Barclays Belastingdienst Benteler Bol.com Box BT Capital One Bank Captcha Chase ChicagoTitle Coinbase DigiD DHL Discover DocuSign Dropbox DPD Ebay Facebook FidelityTitle FirstAm GeekSquad Generic Webmail GLS Google GoogleDrive Gusto Heroku HSBC HSBC Bank Hulu ING IRS Instagram Key Bank KPN LawyersTitle Ledger LinkedIn Lloyds M & T Bank MadisonTitle Mastercard Meta Microsoft Microsoft Office365 Microsoft OneDrive Microsoft Outlook Microsoft SharePoint Microsoft Teams NatWest Navy Federal Credit Union Netflix NHS Norton Okta OldRepublicTitle OVO PayPal Post NL Quickbooks Rabobank Rakuten RBS Royal Mail SBB Silicon Valley Bank Slack Spotify Square StewartTitle SunTrust Bank Swiss Post Swisscom TD Bank TicorTitle U.S. Bank UPS Venmo Visa Vodafone WeTransfer Wells Fargo WhatsApp Ziggo Zoom` iframe `ml.macro_classifier` [](https://docs.sublime.security/docs/enrichment-functions#mlmacro_classifier) -------------------------------------------------------------------------------------------------------- `ml.macro_classifier(input: File) → MLMacrosOutput` The Sublime Macro Classifier introduces machine learning in MQL to detect malicious VBA macro attachments. Combining ML and MQL allows users to combine the model output with custom detection logic to surface what matters most while reducing the noise commonly associated with black-box ML approaches. The classifier uses [XGBoost](https://xgboost.readthedocs.io/en/stable/) to analyze VBA keywords, file metadata, and [Oletools](https://github.com/decalage2/oletools/)  output to predict whether an attachment is likely to cause harm. Use `ml.macro_classifier` to detect suspicious VBA macro attachments. [View rules that use this function](https://github.com/sublime-security/sublime-rules/search?q=ml.macro_classifier) MQL `// detect malicious VBA macros in Office documents, high confidence any(attachments, .file_extension in~ ("doc", "docm", "docx", "dot", "dotm", "pptm", "ppsm", "xlm", "xls", "xlsb", "xlsm", "xlt", "xltm", "zip") and ml.macro_classifier(.).malicious and ml.macro_classifier(.).confidence in ("high") ) // detect malicious VBA macros in Office documents, low or medium confidence any(attachments, .file_extension in~ ("doc", "docm", "docx", "dot", "dotm", "pptm", "ppsm", "xlm", "xls", "xlsb", "xlsm", "xlt", "xltm", "zip") and ml.macro_classifier(.).malicious and ml.macro_classifier(.).confidence in ("low", "medium") )` iframe `ml.nlu_classifier` [](https://docs.sublime.security/docs/enrichment-functions#mlnlu_classifier) ---------------------------------------------------------------------------------------------------- `ml.nlu_classifier(input: str) -> NluResult` Natural Language Understanding, or NLU, provides users with a machine learning service to analyze text-based content. The service has three primary capabilities: * Email Classification * Named Entity Recognition * Topic Recognition ### Email **Classification** [](https://docs.sublime.security/docs/enrichment-functions#email-classification) The Email Classification component takes a body of text as input and provides Intents and/or Tags. **Intent** Intents are top-level categories describing common language attackers use to carry out phishing attacks. | Name | Description | | --- | --- | | `bec` | Emails containing urgent language about quick tasks from C-suite, HR, and Accounting Depts. | | `callback_scam` | Emails containing language about renewing/purchasing services such as tech support, antivirus, or cryptocurrency. | | `cred_theft` | Emails contain language urging users to visit a link leading to a realistic-looking portal that requires their credentials to log in. | | `extortion` | Emails meant to intimidate victims with threats of blackmail. | | `steal_pii` | Emails requesting updates to billing information, personal identification, and tax returns. | | `job_scam` | Deceptive emails disguised as employment offers to dupe students into divulging sensitive data or becoming unwitting accomplices in criminal or fraudulent schemes. | **Tags** Tags are subcategories that provide additional context for financial-themed phishing attacks. The service returns the following values: | Name | Description | | --- | --- | | `invoice` | These emails contain language about viewing invoices via links or attachments. | | `payment` | These emails contain language about ACH, EFT, or Wire payments. | | `purchase_order` | These emails contain language about Purchase Orders, Requests for Quotation. | ### Example Usage [](https://docs.sublime.security/docs/enrichment-functions#example-usage) MQL `type.inbound and any([body.plain.raw, body.html.inner_text], any(ml.nlu_classifier(.).intents, .name == "bec" and .confidence == "high" ) ) // first-time sender and ( ( sender.email.domain.root_domain in $free_email_providers and sender.email.email not in $sender_emails ) or ( sender.email.domain.root_domain not in $free_email_providers and sender.email.domain.domain not in $sender_domains ) )` ### Entity Recognition [](https://docs.sublime.security/docs/enrichment-functions#entity-recognition) Named Entity Recognition (NER) identifies, tags, and extracts important keywords within a body of text. Users can leverage this output to determine if an email contains language commonly associated with _urgency_, _requests_, or _financial_ matters. The available entities are listed below: | Name | Description | Examples | | --- | --- | --- | | `greeting` | Token(s) that aid in the identification of the recipient | hello, dear | | `financial` | Token(s) containing financial details such as payments, bank accounts, or real estate transactions | wire, bank details, ACH payment | | `org` | Token(s) containing an organization name | Google, Microsoft | | `recipient` | Token(s) representing the recipient of the email. Either a name or a generic designator. | Jane Doe, all | | `request` | Token(s) asking the recipient to act on behalf of the sender | "I need you to", "please open" | | `salutation` | Token(s) signifying the end of the correspondence, aids in the identification of the sender | thanks, regards | | `sender` | Token(s) representing the sender of an email. Either a name or a generic designator. | Ms. Tyrell, IT Department | | `urgency` | Token(s) containing language meant to urge recipient to act immediately | ASAP, immediately | ### Example Usage [](https://docs.sublime.security/docs/enrichment-functions#example-usage-1) MQL `type.inbound and sender.display_name in~ $org_display_names and any(ml.nlu_classifier(body.current_thread.text).entities, .name == "urgency") and any(ml.nlu_classifier(body.current_thread.text).entities, .name == "request")` [](https://docs.sublime.security/docs/enrichment-functions) ### Topic Recognition [](https://docs.sublime.security/docs/enrichment-functions#topic-recognition) The Topic Classification component takes a body of text as input and provides topic classification for email content analysis. It analyzes message content to identify the primary topics and themes present in the email, helping to categorize and understand message intent. MQ `ml.nlu_classifier().topics(string, display_name=sender.display_name, subject=subject.subject) → TopicResponse // First parameter is required text to analyze // Optional named parameters provide additional context` #### Parameters [](https://docs.sublime.security/docs/enrichment-functions#parameters-1) * Required text input (string) - The text content to analyze * `display_name` (optional) - Sender display name. Defaults to sender.display\_name if not provided * `subject` (optional) - Subject text. Defaults to subject.subject if not provided When analyzing message body content, use `body.current_thread.text` to ensure valid input. The function can also analyze OCR text or other content sources. ### Supported Topics [](https://docs.sublime.security/docs/enrichment-functions#supported-topics) The function can identify the following topics: * * * #### Business & Professional [](https://docs.sublime.security/docs/enrichment-functions#business--professional) * `"Financial Communications"` - Banking, investments, bills, invoices, financial services * `"Legal and Compliance"` - Legal matters, terms of service, privacy policies, compliance * `"Customer Service and Support"` - Support tickets, inquiries, feedback requests * `"Professional and Career Development"` - Job opportunities, training, industry insights * `"E-Signature"` - Electronic document signing requests and updates #### Technology & Security [](https://docs.sublime.security/docs/enrichment-functions#technology--security) * `"Security and Authentication"` - Account security, password resets, 2FA, login alerts * `"Software and App Updates"` - Software changes, new features, bug fixes * `"File Sharing and Cloud Services"` - Shared files, storage notifications, collaboration * `"Secure Message"` - Encrypted messaging and confidential communications #### Communications & Notifications [](https://docs.sublime.security/docs/enrichment-functions#communications--notifications) * `"Newsletters and Digests"` - Regular content compilations and updates * `"Reminders and Notifications"` - Event/task reminders, calendar notifications * `"Out of Office and Automatic Replies"` - Absence notifications, auto-responses * `"Bounce Back and Delivery Failure Notifications"` - Failed email delivery notices * `"Voicemail Call and Missed Call Notifications"` - Alerts for voicemails, calls, and missed call notifications #### Marketing & Promotions [](https://docs.sublime.security/docs/enrichment-functions#marketing--promotions) * `"Advertising and Promotions"` - Marketing emails, sales, product launches * `"Events and Webinars"` - Event invitations, RSVPs, online/offline gatherings * `"Travel and Transportation"` - Trip planning, bookings, travel updates #### Public & Community [](https://docs.sublime.security/docs/enrichment-functions#public--community) * `"Government Services"` - Official government communications * `"Emergency Alerts"` - Urgent notifications, weather, public safety * `"News and Current Events"` - News updates and current affairs * `"Political Mail"` - Campaign messages, political updates * `"Charity and Non-Profit"` - Fundraising, volunteer opportunities * `"Environmental and Sustainability"` - Updates on environmental initiatives and sustainability efforts #### Health & Education [](https://docs.sublime.security/docs/enrichment-functions#health--education) * `"Health and Wellness"` - Medical appointments, health insurance, wellness * `"Educational and Research"` - Learning materials, academic announcements #### Entertainment & Social [](https://docs.sublime.security/docs/enrichment-functions#entertainment--social) * `"Entertainment and Sports"` - Movies, music, games, sports updates * `"Social Media and Networking"` - Social network notifications, connections ### Common Use Cases [](https://docs.sublime.security/docs/enrichment-functions#common-use-cases) #### Basic Usage [](https://docs.sublime.security/docs/enrichment-functions#basic-usage) MQL `type.inbound and any(ml.nlu_classifier(body.current_thread.text, display_name=sender.display_name, subject=subject.subject).topics, .name == "Voicemail Call and Missed Call Notifications" and .confidence == "high" )` #### Analyze Attachments with OCR [](https://docs.sublime.security/docs/enrichment-functions#analyze-attachments-with-ocr) Combine with OCR functionality to analyze text in attachments: Go `type.inbound and any(attachments, .file_type in ('pdf', 'png', 'jpeg', 'jpg') and any(file.explode(.), any(ml.nlu_classifier(.scan.ocr.raw).topics, .name == "Financial Communications" and .confidence == "high" ) ) )` #### Negate Topics [](https://docs.sublime.security/docs/enrichment-functions#negate-topics) Identify out of office auto replies, and email bound back notification message types: Go `type.inbound // your detection logic that you would like to exclude OOO replies from and not any(ml.nlu_classifier(body.current_thread.text).topics, .name in ( "Out of Office and Automatic Replies", "Bounce Back and Delivery Failure Notifications" ) and .confidence == "high" )` #### Multi-Topic Analysis [](https://docs.sublime.security/docs/enrichment-functions#multi-topic-analysis) Check for multiple topics to further scope a query: Go `type.inbound and any(ml.nlu_classifier(body.current_thread.text).topics, .name in ("Political Mail") and .confidence == "high" ) and not any(ml.nlu_classifier(body.current_thread.text).topics, .name in ( "Charity and Non-Profit", "News and Current Events", "Government Services" ) and (.confidence == "high" or .confidence == "medium") )` ### Best Practices [](https://docs.sublime.security/docs/enrichment-functions#best-practices) 1. **Confidence Levels** * Prefer `confidence == "high"` when topics are critical to a rule or hunt * Consider medium confidence for supplementary signals * Avoid using low confidence results in isolation 2. **Topic Combinations** * Consider both presence and absence of topics * Use with other detection methods for scoping 3. **Performance** * Avoid unnecessary topic analysis on filtered messages * Consider using other methods for simple text matching 4. **Input Selection** * Provide specific content for targeted analysis * Consider context when analyzing extracted text ### Considerations [](https://docs.sublime.security/docs/enrichment-functions#considerations) It is important to remember that the NLU engine only looks at text. Because of this, it needs additional context to be an adequate detector. For example, attackers may craft an email that looks the same as a password reset for your favorite social network. The NLU engine would classify the text as `cred_theft`, but it would also do the same for a legitimate password reset email. But pairing it with a First-Time/Unsolicited Sender or LinkAnalysis provides the necessary context to make an effective detector. iframe Network Analysis [](https://docs.sublime.security/docs/enrichment-functions#network-analysis) ================================================================================================= `network.whois` [](https://docs.sublime.security/docs/enrichment-functions#networkwhois) -------------------------------------------------------------------------------------------- `network.whois(domain: Domain) -> WhoisOutput` `network.whois` performs a WHOIS lookup for domain registration on the `.root_domain` field of a Domain. It returns the domain age, registrar information, and timing information about the age of the registration record and when it was retrieved. This function can be used to identify newly registered domains, by searching for domain age or if a domain is not found. Lookups are performed against Sublime's WHOIS service, which may be delayed by ~24 hours. Since new domains have a slight delay, searching for `.found == false` will identify both unregistered and newly registered domains. For some detections, the `.found == false` could be high enough signal. [View rules that use this function](https://github.com/sublime-security/sublime-rules/search?q=network.whois) emails from a domain registered in the past 7 days `network.whois(sender.email.domain).found == false or network.whois(sender.email.domain).days_old <= 7` links to domains registered in the past 14 days `any(body.links, network.whois(.href_url.domain).days_old <= 14)` iframe HTML Parsing [](https://docs.sublime.security/docs/enrichment-functions#html-parsing) ========================================================================================= `html.xpath` [](https://docs.sublime.security/docs/enrichment-functions#htmlxpath) -------------------------------------------------------------------------------------- `html.xpath -> HTMLXPathResult` `html.xpath` allows you to query the contents of an HTML document (including the body of an email) using [XPath syntax](https://en.wikipedia.org/wiki/XPath) links contained in email that do not match the expected root domain `any(html.xpath(body.html, '//a/@href').nodes, strings.parse_url(.raw).domain.root_domain != 'example.com' )` look for h1 headers in the document that seem to be stating that they are from DocuSign `any(html.xpath(body.html, '//h1').nodes, regex.icontains(.display_text, 'Docu.?Sign') )` check if any suspicious TLDs are included in links on pages found via ml.link\_analysis `any( html.xpath(ml.link_analysis(.).final_dom, '//a/@href' ).nodes, strings.parse_url(.raw).domain.tld in $suspicious_tlds )` Profiling with historical context [](https://docs.sublime.security/docs/enrichment-functions#profiling-with-historical-context) =================================================================================================================================== > 📘 > > Behavior of historical functions > > > -------------------------------------- > > The result of historical functions is always relative to the time of the message that is being evaluated. During live processing, this means the latest possible information is available. However, during a backtest, these functions only take into account messages that are seen _prior to that point in time_. If there's not enough data, some fields like `.prevalence` may be `"unknown"`. This behavior ensures that during a backtest there's never access to "future" data, which would lead to incorrect results and a false sense of confidence in the efficacy of a rule. > > Results are typically and deliberately delayed by several hours, so that the prevalence of a sender can remain as`"new"`for approximately 8-12 hours. `profile.by_sender` [](https://docs.sublime.security/docs/enrichment-functions#profileby_sender) ---------------------------------------------------------------------------------------------------- `profile.by_sender() -> SenderProfile` `profile.by_sender` uses previously ingested inbound messages to build a profile for messages received from a matching Sender. This profile captures information like the `.prevalence` of the sender domain within your environment to assess how common or uncommon it is across messages. It also captures information about flagged messages, such as false positives or true positives. For the `profile.by_sender` function, the list `$free_email_providers` is used to determine whether a sender means a matching email or domain. If the value of `sender.email.domain.domain` is in `$free_email_providers`, then `sender.email.email` is used to determine a matching Sender. Otherwise, all messages with a matching `sender.email.domain.domain` are considered to be from the same Sender. This ensures that for `profile.by_sender`, a matching Sender covers messages from an organization, instead of an individual. Using `profile.by_sender()` to find a first-time sender: MQL `type.inbound and profile.by_sender().prevalence == "new"` Using lists do find a first-time sender is the same but more verbose: MQL `type.inbound and ( ( sender.email.domain.root_domain in $free_email_providers and sender.email.email not in $sender_emails ) or ( sender.email.domain.root_domain not in $free_email_providers and sender.email.domain.domain not in $sender_domains ) )` To check against the historical reputation for a sender, check whether a sender has sent at least 1 message flagged as malicious or spam but no confirmed false positives. MQL `type.inbound and profile.by_sender().any_messages_malicious_or_spam and not profile.by_sender().any_messages_benign // Additional logic on the suspicious sender. and ...` iframe Two more sender profile functions: [`profile.by_sender_domain`](https://docs.sublime.security/docs/enrichment-functions#profileby_sender_domain) and [`profile.by_sender_email`](https://docs.sublime.security/docs/enrichment-functions#profileby_sender_email) exist if the automatic switching between email and domain is not preferred. `profile.by_sender_domain` [](https://docs.sublime.security/docs/enrichment-functions#profileby_sender_domain) ------------------------------------------------------------------------------------------------------------------ `profile.by_sender_domain() -> SenderProfile` `profile.by_sender_domain` uses previously ingested inbound messages to build a profile for messages received from a matching `sender.email.domain.domain`. MQL `type.inbound // filter by first-seen domains or anomalous domains in your environment and profile.by_sender_domain().prevalence in ("outlier", "new") // scrutinize PDF attachments, for example and any(attachments, .file_extension == "pdf" and ...)` iframe `profile.by_sender_email` [](https://docs.sublime.security/docs/enrichment-functions#profileby_sender_email) ---------------------------------------------------------------------------------------------------------------- `profile.by_sender_email() -> SenderProfile` `profile.by_sender_domain` uses previously ingested inbound messages to build a profile for messages received from a matching `sender.email.email`. MQL `type.inbound // filter by first-seen or anomalous email addresses in your environment and profile.by_sender_email().prevalence in ("outlier", "new")` Together, `profile.by_sender_domain` and `profile.by_sender_email` can be used to tell when a domain is common but the sending email address is new: MQL `type.inbound // not a free email provider and sender.email.domain.domain not in $free_email_providers // domain is common in your environment and profile.by_sender_domain().prevalence == "common" // but this is the first time you've received messages from this sender and profile.by_sender_email().prevalence == "new"` iframe Updated 6 days ago * * * --- # Introduction The Sublime Platform API allows you to interact with your Sublime deployment programmatically. Here are some ways you can use the API: * Trash malicious messages from your SOAR * Update a List containing malicious attachment hashes using threat intel * Enrich alerts (e.g. a JIRA or ServiceNow ticket) with message metadata or screenshots The API is organized around [REST](http://en.wikipedia.org/wiki/Representational_State_Transfer) and has predictable resource-oriented URLs, accepts JSON request bodies, returns JSON-encoded responses, and uses standard HTTP response codes, authentication, and verbs. Your Base URL [](https://docs.sublime.security/reference/introduction#your-base-url) ---------------------------------------------------------------------------------------- Your Base URL is the location you send API requests. In this example, the Base URL is `https://platform.sublime.security/`, which is the URL for some Sublime Cloud customers. Base URLs depend on deployment type and region, so head to **Automate > API** on your Dashboard to see your Base URL. ![](https://files.readme.io/1f47be70fcd01ffe29d3279b0ee3c45259dca222ee81d213b6bad15af57c17ef-base_url_location.png) Other Base URL's include: * NA-East: `https://platform.sublime.security` * NA-West: `https://na-west.platform.sublime.security` * Canada: `https://ca.platform.sublime.security` * UK (London): `https://uk.platform.sublime.security` * Europe (Dublin): `https://eu.platform.sublime.security` * Australia: `https://au.platform.sublime.security` Using with webhooks [](https://docs.sublime.security/reference/introduction#using-with-webhooks) ---------------------------------------------------------------------------------------------------- The Sublime API and Sublime's [webhooks](https://docs.sublimesecurity.com/docs/webhooks) go together like warm chocolate chip cookies and cold, cold milk. Use webhooks to receive notifications when a message is flagged, and use the API to take actions like adding message details to a Jira ticket and providing the option to trash a message directly from that ticket. Request IDs [](https://docs.sublime.security/reference/introduction#request-ids) ------------------------------------------------------------------------------------ Every Sublime API response includes a header called `X-Request-ID`. Clients communicating with the Sublime API should log this request ID, so that it can be included in any support requests. --- # Authentication Create an API key [](https://docs.sublime.security/reference/authentication#create-an-api-key) -------------------------------------------------------------------------------------------------- The Sublime API uses API keys for authentication. To generate an API key: 1. Load your Sublime dashboard 2. Click **API** in the left-hand navigation under **Automate** 3. Click the **New Key** button 4. Give your API key a name 5. Click **Save** The API uses the [Bearer authentication scheme](https://swagger.io/docs/specification/authentication/bearer-authentication/) to authenticate requests. Once you've created your API key, include this header in your requests, replacing `YOUR_API_KEY` with your API key: `Authorization: Bearer YOUR_API_KEY` For example, here is a complete curl request for fetching the metadata of a message with ID `3391c0e4-2d7e-4eaa-841c-2d5f7146759e`: `curl \ -H "Authorization: Bearer st4o9hnkltjxpz1dcjo1guga8304hjq8ingb6sx3ldyj96isg8k0xg5749jlusiy" \ https://example.sublime.security/v0/messages/3391c0e4-2d7e-4eaa-841c-2d5f7146759e` --- # Errors Sublime uses standard HTTP response codes to indicate the success or failure of an API request. In general: Codes in the `2xx` range indicate success. Codes in the `4xx` range indicate an error that failed given the information provided (e.g., a required parameter was omitted, etc.). Codes in the `5xx` range indicate an error with Sublime's servers (we try to make these as rare as possible). ### [](https://docs.sublime.security/reference/errors#section-) | HTTP Status Code | Summary | | --- | --- | | 200 - OK | Everything worked as expected. | | 400 - Bad Request | The request was unacceptable, often due to missing a required parameter. | | 401 - Unauthorized | Invalid API key provided. | | 403 - Forbidden | The API key doesn't have permissions to perform the request. | | 404 - Not Found | The requested resource doesn't exist. | | 429 - Too Many Requests | Either too many requests hit the API too quickly, or exceeded allotted rate limit. | | 500, 502, 503, 504 - Server Errors | Something went wrong on Sublime's end. (We try to make these rare.) | --- # Upload a binary to be binexploded ShellNodeRubyPHPPython Base URL https://platform.sublime.security/v0/binexplode/scan --- # Get results of a binexplode scan ShellNodeRubyPHPPython Base URL https://platform.sublime.security/v0/binexplode/scan/{id} --- # Analyze a provided link using ml.link_analysis functionality ShellNodeRubyPHPPython Base URL https://platform.sublime.security/v0/enrichment/link\_analysis/evaluate --- # List events in audit log ShellNodeRubyPHPPython Base URL https://platform.sublime.security/v0/audit-log/events --- # List event types for audit log ShellNodeRubyPHPPython Base URL https://platform.sublime.security/v0/audit-log/events/types --- # Introduction to the Message Data Model Overview [](https://docs.sublime.security/docs/introduction-to-the-message-data-model#overview) =================================================================================================== The [Message Data Model (MDM)](https://docs.sublime.security/docs/mdm) , or MDM, is one of the core building blocks of the Sublime system. It is a structured data model representation of an email message designed to make rule writing predictable, easy, and intuitive. Let's take a look at how a raw email message gets transformed into an MDM. First, download the message [here](https://github.com/sublime-security/sublime-rules/blob/master/tutorial-files/1-intro/impersonation.eml) . ![1726](https://files.readme.io/511b9d3-bec_screenshot.png "bec_screenshot.png") View the contents of the raw message: console `head impersonation.eml` Output: console `Delivered-To: [[email protected]](https://docs.sublime.security/cdn-cgi/l/email-protection) Received: by 2002:a2e:5419:0:0:0:0:0 with SMTP id i25csp4721902ljb; Mon, 21 Oct 2019 11:23:39 -0700 (PDT) X-Received: by 2002:a05:6e02:a:: with SMTP id h10mr3524960ilr.254.1571682219618; Mon, 21 Oct 2019 11:23:39 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1571682219; cv=none; d=google.com; s=arc-20160816; b=UjFMxEI17M6u7hd/V3tM+q/qAYJeKUX6+wZaFWZrXAi/H8RWsiUcBcnPzc8mx1c8d4 q6YIqczF3TEs6wfbbuAgHbel8oAYegOchVgiv0NTgmQsOQ2rzxC2vzyc2ynBdusQUGsv M1TPSJcRHOeimJr5vJA+LDrkKoBhq+Hd8CAqfLycaqnVTK9gWsdP0l2B3phWMGw7a91X` View the sender of the email: console `grep 'From:' impersonation.eml` Output: console `From: Joshua Kamdjou <[[email protected]](https://docs.sublime.security/cdn-cgi/l/email-protection) >` Create a Message Data Model [](https://docs.sublime.security/docs/introduction-to-the-message-data-model#create-a-message-data-model) ========================================================================================================================================= Now, let's create an MDM from our raw EML. Install the Sublime CLI: console `pip3 install sublime-cli` Now, let's create the MDM: console `sublime create -i impersonation.eml` Output: `Output saved to impersonation.mdm` The MDM is stored as a JSON object, and organizes the message into distinct hierarchical objects that make it easy to find what you're looking for in a message. For example, to view the sender of the message, just output the top-level `sender` object using an MQL query (more on that later): console `sublime analyze -i impersonation.mdm -q sender` Output: console `╔═══════════════════════════╗ ║ Results ║ ╚═══════════════════════════╝ File Name: impersonation.mdm Total Rules: 0 Total Queries: 1 QUERIES - Query 1 Result: { "display_name": "Joshua Kamdjou", "email": { "email": "[[email protected]](https://docs.sublime.security/cdn-cgi/l/email-protection) ", "local_part": "joshkamdjou90", "domain": { "domain": "gmail.com", "root_domain": "gmail.com", "sld": "gmail", "tld": "com", "valid": true } } }` Instead of an unformatted string, the `sender` object is tokenized and designed to make rule writing easy! You can view the other objects of the MDM anytime using the [Message Data Model (MDM)](https://docs.sublime.security/docs/mdm) reference. In the next tutorial, we'll learn how to use this information to write a rule for this impersonation attack. Updated over 4 years ago * * * * [Writing your first rule](https://docs.sublime.security/docs/write-your-first-rule) --- # Writing your first rule Now that we understand what the Message Data Model is, it's time to learn about the next core building block of Sublime, [Message Query Language (MQL)](https://docs.sublime.security/docs/message-query-language) , and write our first rule. MQL is the query language that powers Sublime rules. It's very versatile and lets us define any pattern of behavior in our email environment. Let's take a look at the phishing attack from our previous tutorial more closely: ![1726](https://files.readme.io/9bb17ea-bec_screenshot_dropdown.png "bec_screenshot_dropdown.png") The sender's display name is `Joshua Kamdjou`, but this isn't his real email address. Let's write a simple rule to detect this attack. Step 1: Determine what attributes of the message you'll use for your rule [](https://docs.sublime.security/docs/write-your-first-rule#step-1-determine-what-attributes-of-the-message-youll-use-for-your-rule) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ Let's make a list of everything that's suspicious about this message: * the sender's display name impersonates a known contact, "Joshua Kamdjou", but is not from his work email ([\[email protected\]](https://docs.sublime.security/cdn-cgi/l/email-protection#066c69756e467573646a6f6b6375636573746f727f2865696b) ) * suspicious phrases like "Urgent" and "W2" in the subject * suspicious phrases like "need your help" and "send me the w2s" in the body * email signature with Josh's name and title, but not from his known email address For the purposes of this tutorial we'll keep things simple and start with the impersonation of Josh's display name. Step 2: Write a query to build the syntax of your rule [](https://docs.sublime.security/docs/write-your-first-rule#step-2-write-a-query-to-build-the-syntax-of-your-rule) ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Sublime queries are used to extract or summarize information from a message. We start with queries because they'll return more verbose output, and then we'll turn it into a rule when we're done. By looking at the [Message Data Model (MDM)](https://docs.sublime.security/docs/mdm) reference, we can see the display name is located in the `sender` object. Let's confirm this by running a simple query. > 📘 > > CLI hint > > > -------------- > > The Sublime CLI's `analyze` command can run queries and rules on any of the following types of files: `.eml`, `.msg`, `.mdm`, and `.mbox`. Below we're using the MDM we generated from the first tutorial, but you could also use the EML. Both can be downloaded directly [here](https://github.com/sublime-security/sublime-rules/tree/master/tutorial-files/2-write) > . Python `sublime analyze -i impersonation.mdm -q 'sender.display_name'` Output: console `╔═══════════════════════════╗ ║ Results ║ ╚═══════════════════════════╝ File Name: impersonation.mdm Total Rules: 0 Total Queries: 1 QUERIES - Query 1 Result: Joshua Kamdjou` We've successfully extracted the sender's display name using a Sublime query. Now, let's modify our query to look for Josh's full name. We'll start with MQL's [case-insensitive comparison operator](https://docs.sublime.security/docs/syntax#comparing-values) : `=~`: Python `sublime analyze -i impersonation.mdm -q 'sender.display_name =~ "Joshua Kamdjou"'` Output: console `╔═══════════════════════════╗ ║ Results ║ ╚═══════════════════════════╝ File Name: impersonation.mdm Total Rules: 0 Total Queries: 1 QUERIES - Query 1 Result: True` Great! The query above will look for an exact match, so we'll make this a little more robust by using MQL's [ilike function](https://docs.sublime.security/docs/functions#like--ilike) for case-insensitive string search: Python `sublime analyze -i impersonation.mdm -q 'ilike(sender.display_name, "*Josh* Kamdjou*")'` Now, this query will return true for _all_ emails that we receive from Josh, so we'll want to make sure we exclude those: Python `sublime analyze -i impersonation.mdm -q 'ilike(sender.display_name, "*Josh* Kamdjou*") and sender.email.email != "[[email protected]](https://docs.sublime.security/cdn-cgi/l/email-protection) "'` Output: console `╔═══════════════════════════╗ ║ Results ║ ╚═══════════════════════════╝ File Name: impersonation.mdm Total Rules: 0 Total Queries: 1 QUERIES - Query 1 Result: True` Awesome! This is looking pretty good. Now we're ready to turn this into a rule and save it for future use. Step 3: Save and run your rule [](https://docs.sublime.security/docs/write-your-first-rule#step-3-save-and-run-your-rule) ----------------------------------------------------------------------------------------------------------------------------- Sublime rules and queries are stored in [YAML files](https://docs.sublime.security/docs/sublime-rules#format) . Open up a new file called `ceo_impersonation.yml` with your text editor of choice and paste in the following: YAML `name: "CEO impersonation" source: | ilike(sender.display_name, "*Josh* Kamdjou*") and sender.email.email != "[[email protected]](https://docs.sublime.security/cdn-cgi/l/email-protection) " type: "rule"` Let's run our new rule to make sure it works as expected: Python `sublime analyze -i impersonation.mdm -r ceo_impersonation.yml` ![1704](https://files.readme.io/01f6e99-Screen_Shot_2021-02-28_at_9.22.31_PM.png "Screen Shot 2021-02-28 at 9.22.31 PM.png") 🎉 You've done it! Congratulations on writing your first detection rule. Josh is now protected from display name impersonations. Updated over 4 years ago * * * * [Use cases](https://docs.sublime.security/docs/use-cases) * [Detect lookalike domains](https://docs.sublime.security/docs/how-to-detect-lookalike-domains) --- # Analysis API Use cases Analysis API [](https://docs.sublime.security/docs/use-cases#analysis-api) ------------------------------------------------------------------------------ The alpha Analysis API lets you run custom rules on email messages for the following use cases: | Use case | Description | | --- | --- | | Triage reported phish | Detect contact impersonation, homoglyph attacks, and more | | Analyze custom email feeds | Detect phishing attacks and draw insights on your own custom email feeds (see "mail source integrations" below for more information) | Sublime platform [](https://docs.sublime.security/docs/use-cases#sublime-platform) -------------------------------------------------------------------------------------- > 📘 > > Coming soon > > > ----------------- > > You can [request early access](https://sublimesecurity.com/platform.html) > to the Sublime Platform. | Use case | Description | | --- | --- | | Business Email Compromise | Detect executive and employee impersonation, vendor account compromise, invoice fraud, and other targeted phishing attacks | | Credential phishing | Detect credential phishing attacks | | Malware | Detect attachment malware | ### Mail source integrations [](https://docs.sublime.security/docs/use-cases#mail-source-integrations) The alpha Analysis API currently _does not_ support easy integration with live external mail sources like Gmail, G Suite, or Office 365 for on-going analysis. That means you'll need a way to "bring your own emails", whether it's user-reported phish, phishing tools like Cofense, Postfix, honeypots, or by building your own integration with your email provider. Soon we'll be releasing the [Sublime Platform](https://sublimesecurity.com/platform.html) , which enables many other use cases and includes a dashboard, advanced capabilities like historical graph properties, hunt, and easy integrations for IMAP, G Suite, and Office 365 environments. You can [request early access](https://sublimesecurity.com/platform.html) to the Sublime Platform. Updated about 3 years ago * * * ---