# Table of Contents - [Core Concepts in ast-grep's Pattern | ast-grep](#core-concepts-in-ast-grep-s-pattern-ast-grep) - [Custom Language Support | ast-grep](#custom-language-support-ast-grep) - [Frequently Asked Questions | ast-grep](#frequently-asked-questions-ast-grep) - [Find & Patch: A Novel Functional Programming like Code Rewrite Scheme | ast-grep](#find-patch-a-novel-functional-programming-like-code-rewrite-scheme-ast-grep) - [How ast-grep Works: A bird's-eye view | ast-grep](#how-ast-grep-works-a-bird-s-eye-view-ast-grep) - [Deep Dive into ast-grep's Match Algorithm | ast-grep](#deep-dive-into-ast-grep-s-match-algorithm-ast-grep) - [Search Multi-language Documents in ast-grep | ast-grep](#search-multi-language-documents-in-ast-grep-ast-grep) - [Deep Dive into ast-grep's Pattern Syntax | ast-grep](#deep-dive-into-ast-grep-s-pattern-syntax-ast-grep) - [Using ast-grep with AI Tools | ast-grep](#using-ast-grep-with-ai-tools-ast-grep) - [Comparison With Other Frameworks | ast-grep](#comparison-with-other-frameworks-ast-grep) - [ast-grep Blog | ast-grep](#ast-grep-blog-ast-grep) - [Design Space for Code Search Query | ast-grep](#design-space-for-code-search-query-ast-grep) - [An Example of Rust's Fearless Concurrency | ast-grep](#an-example-of-rust-s-fearless-concurrency-ast-grep) - [ast-grep's Journey to AI Generated Rules | ast-grep](#ast-grep-s-journey-to-ai-generated-rules-ast-grep) - [Interactive Code Fixes with Multiple Options! | ast-grep](#interactive-code-fixes-with-multiple-options-ast-grep) - [How to Debug ast-grep Rule Effectively | ast-grep](#how-to-debug-ast-grep-rule-effectively-ast-grep) - [Announcing the Book: Mastering ast-grep | ast-grep](#announcing-the-book-mastering-ast-grep-ast-grep) - [Migrating Bevy can be easier with (semi-)automation | ast-grep](#migrating-bevy-can-be-easier-with-semi-automation-ast-grep) - [ast-grep Gets More LLM Support! | ast-grep](#ast-grep-gets-more-llm-support-ast-grep) - [ast-grep 0.38 is Here | ast-grep](#ast-grep-0-38-is-here-ast-grep) - [ast-grep 0.42: The Answer to Code Searching | ast-grep](#ast-grep-0-42-the-answer-to-code-searching-ast-grep) - [ast-grep 0.39 is Here | ast-grep](#ast-grep-0-39-is-here-ast-grep) - [ast-grep got 6000 stars! | ast-grep](#ast-grep-got-6000-stars-ast-grep) - [Optimize ast-grep to get 10X faster | ast-grep](#optimize-ast-grep-to-get-10x-faster-ast-grep) - [ast-grep Rockets to 8000 Stars! | ast-grep](#ast-grep-rockets-to-8000-stars-ast-grep) - [ast-grep: 5000 stars and beyond! | ast-grep](#ast-grep-5000-stars-and-beyond-ast-grep) - [ast-grep got 3000 stars! | ast-grep](#ast-grep-got-3000-stars-ast-grep) - [YAML vs DSL: comparison is subjective | ast-grep](#yaml-vs-dsl-comparison-is-subjective-ast-grep) - [ast-grep's Journey to Type Safety in Node API | ast-grep](#ast-grep-s-journey-to-type-safety-in-node-api-ast-grep) - [Playground | ast-grep](#playground-ast-grep) - [ast-grep | structural search/rewrite tool for many languages](#ast-grep-structural-search-rewrite-tool-for-many-languages) - [ast-grep](#ast-grep) - [C | ast-grep](#c-ast-grep) - [ast-grep](#ast-grep) - [ast-grep](#ast-grep) - [ast-grep](#ast-grep) - [ast-grep](#ast-grep) - [Cpp | ast-grep](#cpp-ast-grep) - [ast-grep](#ast-grep) - [ast-grep](#ast-grep) - [ast-grep](#ast-grep) - [ast-grep](#ast-grep) - [ast-grep](#ast-grep) - [ast-grep](#ast-grep) - [HTML | ast-grep](#html-ast-grep) - [ast-grep](#ast-grep) - [ast-grep](#ast-grep) - [ast-grep](#ast-grep) - [Go | ast-grep](#go-ast-grep) - [Kotlin | ast-grep](#kotlin-ast-grep) - [Java | ast-grep](#java-ast-grep) - [ast-grep](#ast-grep) - [ast-grep](#ast-grep) - [ast-grep](#ast-grep) - [ast-grep](#ast-grep) - [ast-grep](#ast-grep) - [ast-grep](#ast-grep) - [ast-grep](#ast-grep) - [ast-grep](#ast-grep) - [ast-grep](#ast-grep) - [ast-grep](#ast-grep) - [ast-grep](#ast-grep) - [ast-grep](#ast-grep) - [ast-grep](#ast-grep) - [ast-grep](#ast-grep) - [ast-grep](#ast-grep) - [ast-grep](#ast-grep) - [ast-grep](#ast-grep) - [ast-grep](#ast-grep) - [ast-grep](#ast-grep) - [ast-grep](#ast-grep) - [Ruby | ast-grep](#ruby-ast-grep) - [ast-grep](#ast-grep) - [Python | ast-grep](#python-ast-grep) - [ast-grep](#ast-grep) - [ast-grep](#ast-grep) - [Rust | ast-grep](#rust-ast-grep) - [ast-grep](#ast-grep) - [ast-grep](#ast-grep) - [ast-grep](#ast-grep) - [ast-grep](#ast-grep) - [ast-grep](#ast-grep) - [ast-grep](#ast-grep) - [ast-grep](#ast-grep) - [ast-grep](#ast-grep) - [ast-grep](#ast-grep) - [YAML | ast-grep](#yaml-ast-grep) - [ast-grep](#ast-grep) - [ast-grep](#ast-grep) - [ast-grep](#ast-grep) - [API Usage | ast-grep](#api-usage-ast-grep) - [TSX | ast-grep](#tsx-ast-grep) - [Config Cheat Sheet | ast-grep](#config-cheat-sheet-ast-grep) - [Development Guide | ast-grep](#development-guide-ast-grep) - [Rule Cheat Sheet | ast-grep](#rule-cheat-sheet-ast-grep) - [Add New Language to ast-grep | ast-grep](#add-new-language-to-ast-grep-ast-grep) - [Project Configuration | ast-grep](#project-configuration-ast-grep) - [Performance Tip for napi usage | ast-grep](#performance-tip-for-napi-usage-ast-grep) - [Pattern Syntax | ast-grep](#pattern-syntax-ast-grep) - [Contributing | ast-grep](#contributing-ast-grep) - [Quick Start | ast-grep](#quick-start-ast-grep) - [ast-grep](#ast-grep) - [What is ast-grep? | ast-grep](#what-is-ast-grep-ast-grep) - [TODO: | ast-grep](#todo-ast-grep) - [ast-grep new | ast-grep](#ast-grep-new-ast-grep) - [Handle Error Reports | ast-grep](#handle-error-reports-ast-grep) - [JavaScript API | ast-grep](#javascript-api-ast-grep) - [Relational Rules | ast-grep](#relational-rules-ast-grep) - [Reusing Rule as Utility | ast-grep](#reusing-rule-as-utility-ast-grep) - [Python API | ast-grep](#python-api-ast-grep) - [ast-grep test | ast-grep](#ast-grep-test-ast-grep) - [Scan Your Project! | ast-grep](#scan-your-project-ast-grep) - [transform Code in Rewrite | ast-grep](#transform-code-in-rewrite-ast-grep) - [List of Languages with Built-in Support | ast-grep](#list-of-languages-with-built-in-support-ast-grep) - [Rewriter in Fix | ast-grep](#rewriter-in-fix-ast-grep) - [Editor Integration | ast-grep](#editor-integration-ast-grep) - [Rule Essentials | ast-grep](#rule-essentials-ast-grep) - [Composite Rule | ast-grep](#composite-rule-ast-grep) - [Atomic Rule | ast-grep](#atomic-rule-ast-grep) - [Test Your Rule | ast-grep](#test-your-rule-ast-grep) - [JSON Mode | ast-grep](#json-mode-ast-grep) - [ast-grep run | ast-grep](#ast-grep-run-ast-grep) - [Lint Rule | ast-grep](#lint-rule-ast-grep) - [Rewrite Code | ast-grep](#rewrite-code-ast-grep) - [ast-grep scan | ast-grep](#ast-grep-scan-ast-grep) - [Command Line Tooling Overview | ast-grep](#command-line-tooling-overview-ast-grep) - [Fix | ast-grep](#fix-ast-grep) - [ast-grep Playground Manual | ast-grep](#ast-grep-playground-manual-ast-grep) - [sgconfig.yml Reference | ast-grep](#sgconfig-yml-reference-ast-grep) - [Command Line Reference | ast-grep](#command-line-reference-ast-grep) - [Rule Object Reference | ast-grep](#rule-object-reference-ast-grep) - [Rewriter | ast-grep](#rewriter-ast-grep) - [Configuration Reference | ast-grep](#configuration-reference-ast-grep) - [Transformation Object | ast-grep](#transformation-object-ast-grep) - [API Reference | ast-grep](#api-reference-ast-grep) - [TypeScript | ast-grep](#typescript-ast-grep) - [Rule Catalog | ast-grep](#rule-catalog-ast-grep) - [Unknown](#unknown) - [Playground | ast-grep](#playground-ast-grep) - [ast-grep Playground Manual | ast-grep](#ast-grep-playground-manual-ast-grep) - [404 | ast-grep](#404-ast-grep) - [404 | ast-grep](#404-ast-grep) --- # Core Concepts in ast-grep's Pattern | ast-grep [Skip to content](https://ast-grep.github.io/advanced/core-concepts.html#VPContent) On this page Are you an LLM? You can read better optimized documentation at /advanced/core-concepts.md for this page in Markdown format Core Concepts in ast-grep's Pattern [​](https://ast-grep.github.io/advanced/core-concepts.html#core-concepts-in-ast-grep-s-pattern) ==================================================================================================================================== One key highlight of ast-grep is its pattern. _Pattern is a convenient way to write and read expressions that describe syntax trees_. It resembles code, but with some special syntax and structure that allow you to match parts of a syntax tree based on their structure, type or content. While ast-grep's pattern is **easy to learn**, it is **hard to master**. It requires you to know the Tree-sitter grammar and meaning of the target language, as well as the rules and conventions of ast-grep. In this guide, we will help you grasp the core concepts of ast-grep's pattern that are common to all languages. We will also show you how to leverage the full power of ast-grep pattern for your own usage. What is Tree-sitter? [​](https://ast-grep.github.io/advanced/core-concepts.html#what-is-tree-sitter) ----------------------------------------------------------------------------------------------------- ast-grep is using [Tree-sitter](https://tree-sitter.github.io/) as its underlying parsing framework due to its **popularity**, **performance** and **robustness**. Tree-sitter is a tool that generates parsers and provides an incremental parsing library. A [parser](https://www.wikiwand.com/en/Parser_(programming_language)) is a program that takes a source code file as input and produces a tree structure that describes the organization of the code. (Contrary to ast-grep's name, the tree structure is not abstract syntax tree, as we will see later). Writing good parsers for various programming languages is a laborious task, if even possible, for one single project like ast-grep. Fortunately, Tree-sitter is a venerable and popular tool that has a wide community support. Many mainstream languages such as C, Java, JavaScript, Python, Rust, and more are supported by Tree-sitter. Using Tree-sitter as ast-grep's underlying parsing library allows it to _work with any language that has a well-maintained grammar available_. Another perk of Tree-sitter is its incremental nature. An incremental parser is a parser that can update the syntax tree efficiently when the source code file is edited, without having to re-parse the entire file. _It can run very fast on every code changes in ast-greps' [interactive editing](https://ast-grep.github.io/guide/tooling-overview.html#interactive-mode) ._ Finally, Tree-sitter also handles syntax errors gracefully, and it can parse multiple languages within the same file. _This makes pattern code more robust to parse and easier to write._ In future we can also support multi-language source code like Vue. Textual vs Structural [​](https://ast-grep.github.io/advanced/core-concepts.html#textual-vs-structural) -------------------------------------------------------------------------------------------------------- When you use ast-grep to search for patterns in source code, you need to understand the difference between textual and structural matching. Source code input is text, a sequence of characters that follows certain syntax rules. You can use common search tools like [silver-searcher](https://github.com/ggreer/the_silver_searcher) or [ripgrep](https://github.com/BurntSushi/ripgrep) to search for text patterns in source code. However, ast-grep does not match patterns against the text directly. Instead, it parses the text into a tree structure that represents the syntax of the code. This allows ast-grep to match patterns based on the structure of the code, not just its surface appearance. This is known as [structural](https://docs.sourcegraph.com/code_search/reference/structural) [search](https://docs.sourcegraph.com/code_search/reference/structural) , which searches for code with a specific structure, not just a specific text. _Therefore, the patterns you write must also be of valid syntax that can be compared with the code tree._ Textual Search in ast-grep Though `pattern` structurally matches code, you can use [the atomic rule `regex`](https://ast-grep.github.io/guide/rule-config/atomic-rule.html#regex) to matches the text of a node by specifying a regular expression. This way, it is possible to combine textual and structural matching in ast-grep. AST vs CST [​](https://ast-grep.github.io/advanced/core-concepts.html#ast-vs-cst) ---------------------------------------------------------------------------------- To represent the syntax and structure of code, we have two types of tree structures: [AST](https://www.wikiwand.com/en/Abstract_syntax_tree) and [CST](https://eli.thegreenplace.net/2009/02/16/abstract-vs-concrete-syntax-trees/) . AST stands for Abstract Syntax Tree, which is a **simplified** representation of the code that _omits some details_ like punctuation and whitespaces. CST stands for Concrete Syntax Tree, which is a more **faithful** representation of the code that _includes all the details_. Tree-sitter is a library that can parse code into CSTs for many programming languages. Thusly, _ast-grep, contrary to its name, searches and rewrites code based on CST patterns, instead of AST_. Let's walk through an example to see why CST makes more sense. Consider the JavaScript snippet `1 + 1`. Its AST representation [looks like this](https://ast-grep.github.io/playground.html#eyJtb2RlIjoiUGF0Y2giLCJsYW5nIjoiamF2YXNjcmlwdCIsInF1ZXJ5IjoiY29uc29sZS5sb2coJE1BVENIKSIsImNvbmZpZyI6IiMgQ29uZmlndXJlIFJ1bGUgaW4gWUFNTFxucnVsZTpcbiAgYW55OlxuICAgIC0gcGF0dGVybjogaWYgKGZhbHNlKSB7ICQkJCB9XG4gICAgLSBwYXR0ZXJuOiBpZiAodHJ1ZSkgeyAkJCQgfVxuY29uc3RyYWludHM6XG4gICMgTUVUQV9WQVI6IHBhdHRlcm4iLCJzb3VyY2UiOiIxICsgMSJ9) : binary_expression number number An astute reader should notice the important operator `+` is not encoded in AST. Meanwhile, its CST faithfully represents all critical information. binary_expression number + # note this + operator! number You might wonder if using CST will make trivial whitespaces affect your search results. Fortunately, ast-grep uses a [smart matching algorithm](https://ast-grep.github.io/advanced/match-algorithm.html) that can skip trivial nodes in CST when appropriate, which saves you a lot of trouble. Named vs Unnamed [​](https://ast-grep.github.io/advanced/core-concepts.html#named-vs-unnamed) ---------------------------------------------------------------------------------------------- It is possible to convert CST to AST if we don't care about punctuation and whitespaces. Tree-sitter has two types of nodes: named nodes and unnamed nodes(anonymous nodes). The more important _named nodes_ are defined with a regular name in the grammar rules, such as `binary_expression` or `identifier`. The less important _unnamed nodes_ are defined with literal strings such as `","` or `"+"`. Named nodes are more important for understanding the code's structure and meaning, while unnamed nodes are less important and can be sometimes skipped by ast-grep's matching algorithms. The following example, adapted from [Tree-sitter's official guide](https://tree-sitter.github.io/tree-sitter/creating-parsers#the-first-few-rules) , shows the difference in grammar definition. javascript rules: { // named nodes are defined with the format `kind: parseRule` identifier: $ => /[a-z]+/, // binary_expression is also a named node, // the `+` operator is defined with a string literal, so it is an unnamed node binary_expression: $ => seq($.identifier, '+', $.identifier), // ↑ unnamed node } Practically, named nodes have a property called `kind` that indicates their names. You can use ast-grep's [atomic rule `kind`](https://ast-grep.github.io/guide/rule-config/atomic-rule.html#kind) to find the specific AST node. [Playground link](https://ast-grep.github.io/playground.html#eyJtb2RlIjoiQ29uZmlnIiwibGFuZyI6ImphdmFzY3JpcHQiLCJxdWVyeSI6ImNvbnNvbGUubG9nKCRNQVRDSCkiLCJjb25maWciOiJydWxlOiBcbiAga2luZDogYmluYXJ5X2V4cHJlc3Npb24iLCJzb3VyY2UiOiIxICsgMSAifQ==) for the example below. yaml rule: kind: binary_expression # matches `1 + 1` Further more, ast-grep's meta variable matches only named nodes by default. `return $A` matches only the first statement below. [Playground link](https://ast-grep.github.io/playground.html#eyJtb2RlIjoiUGF0Y2giLCJsYW5nIjoiamF2YXNjcmlwdCIsInF1ZXJ5IjoicmV0dXJuICRBIiwicmV3cml0ZSI6IiIsInN0cmljdG5lc3MiOiJzbWFydCIsInNlbGVjdG9yIjoiIiwiY29uZmlnIjoiIiwic291cmNlIjoicmV0dXJuIDEyM1xucmV0dXJuOyJ9) . js return 123 // `123` is named `number` and matched. return; // `;` is unnamed and not matched. We can use double dollar `$$VAR` to _include unnamed nodes_ in the pattern result. `return $$A` will match both statement above. [Playground link](https://ast-grep.github.io/playground.html#eyJtb2RlIjoiUGF0Y2giLCJsYW5nIjoiamF2YXNjcmlwdCIsInF1ZXJ5IjoicmV0dXJuICQkQSIsInJld3JpdGUiOiIiLCJzdHJpY3RuZXNzIjoic21hcnQiLCJzZWxlY3RvciI6IiIsImNvbmZpZyI6IiIsInNvdXJjZSI6InJldHVybiAxMjNcbnJldHVybjsifQ==) . Kind vs Field [​](https://ast-grep.github.io/advanced/core-concepts.html#kind-vs-field) ---------------------------------------------------------------------------------------- Sometimes, using kind alone is not enough to find the nodes we want. A node may have several children with the same kind, but different roles in the code. For [example](https://ast-grep.github.io/playground.html#eyJtb2RlIjoiQ29uZmlnIiwibGFuZyI6ImphdmFzY3JpcHQiLCJxdWVyeSI6ImNvbnNvbGUubG9nKCRNQVRDSCkiLCJjb25maWciOiJydWxlOlxuICBraW5kOiBzdHJpbmciLCJzb3VyY2UiOiJ2YXIgYSA9IHtcbiAgJ2tleSc6ICd2YWx1ZSdcbn0ifQ==) , in JavaScript, an object may have multiple keys and values, all with the string kind. To distinguish them, we can use `field` to specify the relation between a node and its parent. In ast-grep, `field` can be specified in two [relational rules](https://ast-grep.github.io/guide/rule-config/relational-rule.html#relational-rule-mnemonics) : `has` and `inside`. `has` and `inside` accept a special configuration item called `field`. The value of `field` is the _field name_ of the parent-child relation. For example, the key-value `pair` in JavaScript object has two children: one with field `key` and the other with field `value`. We can use [this rule](https://ast-grep.github.io/playground.html#eyJtb2RlIjoiQ29uZmlnIiwibGFuZyI6ImphdmFzY3JpcHQiLCJxdWVyeSI6ImNvbnNvbGUubG9nKCRNQVRDSCkiLCJjb25maWciOiJydWxlOlxuICBraW5kOiBzdHJpbmdcbiAgaW5zaWRlOlxuICAgIGZpZWxkOiBrZXlcbiAgICBraW5kOiBwYWlyIiwic291cmNlIjoidmFyIGEgPSB7XG4gICdrZXknOiAndmFsdWUnXG59In0=) to match the `key` node of kind `string`. yaml rule: kind: string inside: field: key kind: pair `field` can help us to narrow down the search scope and make the pattern more precise. We can also use `has` to rewrite the rule above, searching the key-value `pair` with `string` key. [Playground link](https://ast-grep.github.io/playground.html#eyJtb2RlIjoiQ29uZmlnIiwibGFuZyI6ImphdmFzY3JpcHQiLCJxdWVyeSI6ImNvbnNvbGUubG9nKCRNQVRDSCkiLCJjb25maWciOiJydWxlOlxuICBraW5kOiBwYWlyXG4gIGhhczpcbiAgICBmaWVsZDoga2V5XG4gICAga2luZDogc3RyaW5nIiwic291cmNlIjoidmFyIG1hdGNoID0geyAna2V5JzogJ3ZhbHVlJyB9XG52YXIgbm9NYXRjaCA9IHsga2V5OiB2YWx1ZX0ifQ==) . yaml rule: kind: pair has: field: key kind: string Key Difference between `kind` and `field` * `kind` is the property of the node itself. Only named nodes have `kind`s. * `field` is the property of the relation between parent and child. Unnamed nodes can also have `field`s. It might be confusing to new users that a node has both `kind` and `field`. `kind` belongs to the node itself, represented by blue text in ast-grep's playground. Child node has a `field` only relative to its parent, and vice-versa. `field` is represented by dark yellow text in the playground. Since field is a property of a node relation, unnamed nodes can also have `field`. For example, the `+` in the binary expression `1 + 1` has the field `operator`. Significant vs Trivial [​](https://ast-grep.github.io/advanced/core-concepts.html#significant-vs-trivial) ---------------------------------------------------------------------------------------------------------- ast-grep goes further beyond Tree-sitter. It has a concept about the "significance" of a node. * If a node is a named node or has a field relative to its parent, it is a **significant** node. * Otherwise, the node is a **trivial** node. Even significance is not enough Most Tree-sitter languages do not encode all critical structures in AST, the tree with named nodes only. Even significant nodes are not sufficient to represent the meaning of code. We have to preserve some trivial nodes for precise matching. Tree-sitter parsers do not encode all semantics with named nodes. For example, `class A { get method() {} }` and `class A { method() {} }` are equivalent in Tree-sitter's AST. The critical token `get` is not named nor has a field name. It is a trivial node! If you do not care about if the method is a getter method, a static method or an instance method, you can use `class $A { method() {} }` to [match all the three methods at once](https://ast-grep.github.io/playground.html#eyJtb2RlIjoiUGF0Y2giLCJsYW5nIjoiamF2YXNjcmlwdCIsInF1ZXJ5IjoiY2xhc3MgJEEgeyBtZXRob2QoKSB7fSB9IiwiY29uZmlnIjoicnVsZTpcbiAga2luZDogcGFpclxuICBoYXM6XG4gICAgZmllbGQ6IGtleVxuICAgIGtpbmQ6IHN0cmluZyIsInNvdXJjZSI6ImNsYXNzIEEgeyBtZXRob2QoKSB7fX1cbmNsYXNzIEIgeyBnZXQgbWV0aG9kKCkge319XG5jbGFzcyBDIHsgc3RhdGljIG1ldGhvZCgpIHt9fSJ9) . Alternatively, you can [fully spell out the method modifier](https://ast-grep.github.io/playground.html#eyJtb2RlIjoiUGF0Y2giLCJsYW5nIjoiamF2YXNjcmlwdCIsInF1ZXJ5IjoiY2xhc3MgJEEgeyBnZXQgbWV0aG9kKCkge30gfSIsImNvbmZpZyI6InJ1bGU6XG4gIGtpbmQ6IHBhaXJcbiAgaGFzOlxuICAgIGZpZWxkOiBrZXlcbiAgICBraW5kOiBzdHJpbmciLCJzb3VyY2UiOiJjbGFzcyBBIHsgbWV0aG9kKCkge319XG5jbGFzcyBCIHsgZ2V0IG1ldGhvZCgpIHt9fVxuY2xhc3MgQyB7IHN0YXRpYyBtZXRob2QoKSB7fX0ifQ==) if you need to tell getter method from normal method. Summary [​](https://ast-grep.github.io/advanced/core-concepts.html#summary) ---------------------------------------------------------------------------- Thank you for reading until here! There are many concepts in this article. Let's summarize them in one paragraph. ast-grep uses Tree-sitter to parse _textual_ source code into a detailed tree _structure_ called **CST**. We can get **AST** from CST by only keeping **named nodes**, which have kinds. To search nodes in a syntax tree, you can use both node **kind** and node **field**, which is a special role of a child node relative to its parent node. A node with either a kind or a field is a **significant** node. --- # Custom Language Support | ast-grep [Skip to content](https://ast-grep.github.io/advanced/custom-language.html#VPContent) On this page Are you an LLM? You can read better optimized documentation at /advanced/custom-language.md for this page in Markdown format Custom Language Support [​](https://ast-grep.github.io/advanced/custom-language.html#custom-language-support) ============================================================================================================== In this guide, we will show you how to use a custom language that is not built into ast-grep. We will use [Mojo 🔥](https://www.modular.com/mojo) as an example! * * * [Tree-sitter](https://tree-sitter.github.io/tree-sitter/) is a popular parser generator library that ast-grep uses to support many languages. However, not all Tree-sitter compatible languages are shipped with ast-grep command line tool. If you want to use a custom language that is not built into ast-grep, you can compile it as a dynamic library first and load it via custom language registration. There will be three steps to achieve this: 1. Install tree-sitter CLI and prepare the grammar file. 2. Compile the custom language as a dynamic library. 3. Register the custom language in ast-grep project config. Pro Tip You can also reuse the dynamic library compiled by neovim. See [this link](https://github.com/nvim-treesitter/nvim-treesitter/#changing-the-parser-install-directory) to find where the parsers are. Prepare Tree-sitter Tool and Parser [​](https://ast-grep.github.io/advanced/custom-language.html#prepare-tree-sitter-tool-and-parser) -------------------------------------------------------------------------------------------------------------------------------------- Before you can compile a custom language as a dynamic library, you need to install the Tree-sitter CLI tool and get the Tree-sitter grammar for your custom language. The recommended way to install the Tree-sitter CLI tool is via [npm](https://www.npmjs.com/package/tree-sitter-cli) : bash npm install -g tree-sitter-cli Alternative installation methods are also available in the [official doc](https://tree-sitter.github.io/tree-sitter/creating-parsers#installation) . For the Tree-sitter grammar, you can either [write your own](https://tree-sitter.github.io/tree-sitter/creating-parsers#writing-the-grammar) or find one from the Tree-sitter grammars [repository](https://github.com/tree-sitter) . Since **Mojo** is a new language, we cannot find an existing repo for it. But I have created a mock [grammar for Mojo](https://github.com/HerringtonDarkholme/tree-sitter-mojo) . You can clone it for the tutorial sake. It is forked from Python and barely contains Mojo syntax(just `struct`/`fn` keywords). bash git clone https://github.com/HerringtonDarkholme/tree-sitter-mojo.git Compile the Parser as Dynamic Library [​](https://ast-grep.github.io/advanced/custom-language.html#compile-the-parser-as-dynamic-library) ------------------------------------------------------------------------------------------------------------------------------------------ Once we have prepared the tool and the grammar, we can compile the parser as dynamic library. _`tree-sitter-cli` is the preferred way to compile dynamic library._ The [official way](https://tree-sitter.github.io/tree-sitter/cli/build.html) to compile a parser as a dynamic library is to use the `tree-sitter build` command. sh tree-sitter build --output mojo.so The build command compiles your parser into a dynamically-loadable library as a shared object (.so, .dylib, or .dll). Another way is to use the following [commands](https://github.com/tree-sitter/tree-sitter/blob/a62bac5370dc5c76c75935834ef083457a6dd0e1/cli/loader/src/lib.rs#L380-L410) to compile the parser manually: shell gcc -shared -fPIC -fno-exceptions -g -I {header_path} -o {lib_path} -O2 {scanner_path} -xc {parser_path} {other_flags} where `{header_path}` is the path to the folder of header file of your custom language parser (usually `src`) and `{lib_path}` is the path where you want to store the dynamic library (in this case `mojo.so`). `{scanner_path}` and `{parser_path}` are the `c` or `cc` files of your parser. You also need to include other gcc flags if needed. For example, in mojo's case, the full command will be: shell gcc -shared -fPIC -fno-exceptions -g -I 'src' -o mojo.so -O2 src/scanner.cc -xc src/parser.c -lstdc++ Old tree-sitter does not have build command [Previously](https://github.com/tree-sitter/tree-sitter/pull/3174) there are no official instructions on how to do this on the internet, but we can get some hints from Tree-sitter's [source code](https://github.com/tree-sitter/tree-sitter/blob/a62bac5370dc5c76c75935834ef083457a6dd0e1/cli/loader/src/lib.rs#L111) . One way is to set an environment variable called `TREE_SITTER_LIBDIR` to the path where you want to store the dynamic library, and then run `tree-sitter test` in the directory of your custom language parser. This will generate a dynamic library at the `TREE_SITTER_LIBDIR` path. For example: sh cd path/to/mojo/parser export TREE_SITTER_LIBDIR=path/to/your/dir tree-sitter test Register Language in `sgconfig.yml` [​](https://ast-grep.github.io/advanced/custom-language.html#register-language-in-sgconfig-yml) ------------------------------------------------------------------------------------------------------------------------------------ Once you have compiled the dynamic library for your custom language, you need to register it in the `sgconfig.yml` file. You can use the command [`ast-grep new`](https://ast-grep.github.io/guide/scan-project.html#create-scaffolding) to create a project and find the configuration file in the project root. You need to add a new entry under the `customLanguages` key with the name of your custom language and some properties: yaml # sgconfig.yml ruleDirs: ["./rules"] customLanguages: mojo: libraryPath: mojo.so # path to dynamic library extensions: [mojo, 🔥] # file extensions for this language expandoChar: _ # optional char to replace $ in your pattern The `libraryPath` property specifies the path to the dynamic library relative to the `sgconfig.yml` file or an absolute path. The `extensions` property specifies a list of file extensions for this language. The `expandoChar` property is optional and specifies a character that can be used instead of `$` for meta-variables in your pattern. What's expandoChar? ast-grep requires pattern to be a valid syntactical construct, but `$VAR` might not be a valid expression in some language. `expandoChar` will replace `$` in the pattern so it can be parsed successfully by Tree-sitter. For example, `$VAR` is not valid in ~[Python](https://github.com/ast-grep/ast-grep/blob/1b999b249110c157ae5026e546a3112cd64344f7/crates/language/src/python.rs#L15) ~ Mojo. So we need to replace it with `_VAR`. You can check the `expandoChar` of ast-grep's built-in languages [here](https://github.com/ast-grep/ast-grep/tree/main/crates/language/src) . Use It! [​](https://ast-grep.github.io/advanced/custom-language.html#use-it) ----------------------------------------------------------------------------- Now you are ready to use your custom language with ast-grep! You can use it as any other supported language with the `-l` flag or the `language` property in your rule. For example, to search for all occurrences of `print` in mojo files, you can run: bash ast-grep -p "print" -l mojo Or you can write a rule in yaml like this: yaml id: my-first-mojo-rule language: mojo # the name we register before! severity: hint rule: pattern: print And that's it! You have successfully used a custom language with ast-grep! Inspect Parser Output [​](https://ast-grep.github.io/advanced/custom-language.html#inspect-parser-output) ---------------------------------------------------------------------------------------------------------- Due to limited bandwidth, ast-grep does not support pretty print Concrete Syntax Trees. However, you can use [tree-sitter-cli](https://github.com/tree-sitter/tree-sitter/tree/master/cli#commands) to dump the AST tree for your file. bash tree-sitter parse [file_path] Quiz Time Can you support parse `main.ʕ◔ϖ◔ʔ` as [Golang](https://github.com/golang/go/issues/59968) ? [Answer](https://twitter.com/hd_nvim/status/1655085184855969797) . --- # Frequently Asked Questions | ast-grep [Skip to content](https://ast-grep.github.io/advanced/faq.html#VPContent) On this page Are you an LLM? You can read better optimized documentation at /advanced/faq.md for this page in Markdown format Frequently Asked Questions [​](https://ast-grep.github.io/advanced/faq.html#frequently-asked-questions) ======================================================================================================== My pattern does not work, why? [​](https://ast-grep.github.io/advanced/faq.html#my-pattern-does-not-work-why) -------------------------------------------------------------------------------------------------------------- 1. **Use the Playground**: Test your pattern in the [ast-grep playground](https://ast-grep.github.io/playground.html) . 2. **Check for Valid Code**: Make sure your pattern is valid code that tree-sitter can parse. 3. **Ensure Correctness**: Use a [pattern object](https://ast-grep.github.io/guide/rule-config/atomic-rule.html#pattern) to ensure your code is correct and unambiguous. 4. **Explore Examples**: See ast-grep's [catalog](https://ast-grep.github.io/catalog/) for more examples. The most common scenario is that you only want to match a sub-expression or one specific AST node in a whole syntax tree. However, the code fragment corresponding to the sub-expression may not be valid code. To make the code can be parsed by tree-sitter, you probably need more context instead of providing just code fragment. For example, if you want to match key-value pair in JSON, writing `"key": "$VAL"` will not work because it is not a legal JSON. Instead, you can provide context via the pattern object. See [playground code](https://ast-grep.github.io/playground.html#eyJtb2RlIjoiQ29uZmlnIiwibGFuZyI6Impzb24iLCJxdWVyeSI6ImZvbygkJCRBLCBiLCAkJCRDKSIsInJld3JpdGUiOiIiLCJjb25maWciOiJydWxlOlxuICBwYXR0ZXJuOiBcbiAgICBjb250ZXh0OiAne1widmVyc2lvblwiOiBcIiRWRVJcIiB9J1xuICAgIHNlbGVjdG9yOiBwYWlyIiwic291cmNlIjoie1xuICAgIFwidmVyc2lvblwiOiBcInZlclwiXG59In0=) . YAML rule: pattern: context: '{"key": "$VAL"}' selector: pair The idea is that you can write full an valid code in the `context` field and use `selector` to select the sub-AST node. This trick can be used in other languages as well, like [C](https://ast-grep.github.io/catalog/c/#match-function-call) and [Go](https://ast-grep.github.io/catalog/go/#match-function-call-in-golang) . That said, pattern is not always the best choice for code search. [Rule](https://ast-grep.github.io/guide/rule-config.html) can be more expressive and powerful. My Rule does not work, why? [​](https://ast-grep.github.io/advanced/faq.html#my-rule-does-not-work-why) -------------------------------------------------------------------------------------------------------- Here are some tips to debug your rule: * Use the [ast-grep playground](https://ast-grep.github.io/playground.html) to test your rule. * Simplify your rule to the minimal possible code that reproduces the issue. * Confirm pattern's matched AST nodes are expected. e.g. statement and expression are [different matches](https://ast-grep.github.io/advanced/pattern-parse.html#extract-effective-ast-for-pattern) . This usually happens when you use `follows` or `precedes` in the rule. * Check the [rule order](https://ast-grep.github.io/advanced/faq.html#why-is-rule-matching-order-sensitive) . The order of rules matters in ast-grep especially when using meta variables with relational rules. CLI and Playground produce different results, why? [​](https://ast-grep.github.io/advanced/faq.html#cli-and-playground-produce-different-results-why) ------------------------------------------------------------------------------------------------------------------------------------------------------ There are two main reasons why the results may differ: * **Parser Version**: The CLI may use a different version of the tree-sitter parser than the Playground. Playground parsers are updated less frequently than the CLI, so there may be differences in the results. * **Text Encoding**: The CLI and Playground use different text encodings. CLI uses utf-8, while the Playground uses utf-16. The encoding difference may cause different fallback parsing during [error recovery](https://github.com/tree-sitter/tree-sitter/issues/224) . To debug the issue, you can use the [`--debug-query`](https://ast-grep.github.io/reference/cli/run.html#debug-query-format) in the CLI to see the parsed AST nodes and meta variables. sh ast-grep run -p --debug-query ast The debug output will show the parsed AST nodes and you can compare them with the [Playground](https://ast-grep.github.io/playground.html) . You can also use different debug formats like `cst` or `pattern`. Different results are usually caused by incomplete or wrong code snippet in the pattern. A common fix is to provide a complete context code via the [pattern object](https://ast-grep.github.io/reference/rule.html#atomic-rules) . yaml rule: pattern: context: 'int main() { return 0; }' selector: function See [Pattern Deep Dive](https://ast-grep.github.io/advanced/pattern-parse.html) for more context. Alternatively, you can try [rule](https://ast-grep.github.io/guide/rule-config.html) instead. Note `--debug-query` is not only for pattern, you can pass source code as `pattern` to see the parsed AST. Text encoding impacts tree-sitter error recovery. Tree-sitter is a robust parser that can recover from syntax errors and continue parsing the rest of the code. The exact strategy for error recovery is implementation-defined and uses a heuristic to determine the best recovery strategy. See [tree-sitter issue](https://github.com/tree-sitter/tree-sitter/issues/224) for more details. Text-encoding will affect the error recovery because it changed the cost of different recovery strategies. If you find the inconsistency between CLI and Playground, try confirming the playground version by hovering over the language label in playground, and the CLI version by [this file](https://github.com/ast-grep/ast-grep/blob/main/crates/language/Cargo.toml) . ![Playground Version](https://ast-grep.github.io/image/playground-parser-version.png) Found inconsistency? You can also [open an issue in the Playground repository](https://github.com/ast-grep/ast-grep.github.io/issues) if you find outdated parsers. Contribution to update the Playground parser is warmly welcome! MetaVariable does not work, why? [​](https://ast-grep.github.io/advanced/faq.html#metavariable-does-not-work-why) ------------------------------------------------------------------------------------------------------------------ 1. **Correct Naming**: Start meta variables with the `$` sign, followed by uppercase letters (A-Z), underscores (`_`), or digits (1-9). 2. **Single AST Node**: A meta variable should be a single AST node. Avoid mixing meta variables with other text in one AST node. For example, `mix$OTHER_VAR` or `use$HOOK` will not work. 3. **Named AST Nodes**: By default, a meta variable matches only named AST nodes. Use double dollar signs like `$$UNNAMED` to match unnamed nodes. Multiple MetaVariable does not work [​](https://ast-grep.github.io/advanced/faq.html#multiple-metavariable-does-not-work) -------------------------------------------------------------------------------------------------------------------------- Multiple meta variables in ast-grep, such as `$$$MULTI`, are lazy. They stop matching nodes if the first node after them can match. For example, `foo($$$A, b, $$$C)` matches `foo(a, c, b, b, c)`. `$$$A` stops before the first `b` and only matches `a, c`. This design follows TypeScript's template literal types (`${infer VAR}Literal`) to ensure multiple meta variables always produce a match or non-match in linear time. Pattern cannot match my use case, how? [​](https://ast-grep.github.io/advanced/faq.html#pattern-cannot-match-my-use-case-how) ------------------------------------------------------------------------------------------------------------------------------ Patterns are a quick and easy way to match code in ast-grep, but they might not handle complex code. YAML rules are much more expressive and make it easier to specify complex code. I want to pattern match function call starts with some prefix string, how can I do that? [​](https://ast-grep.github.io/advanced/faq.html#i-want-to-pattern-match-function-call-starts-with-some-prefix-string-how-can-i-do-that) ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- It is common to find function name or variable name following some naming convention like a function must starts with specific prefix. For example, [React Hook](https://react.dev/learn/reusing-logic-with-custom-hooks#hook-names-always-start-with-use) in JavaScript requires function names start with `use`. Another example will be using `io_uring` in [Linux asynchronous programming](https://unixism.net/loti/genindex.html) . You may start with pattern like `use$HOOK` or `io_uring_$FUNC`. However, they are not valid meta variable names since the AST node text does not start with the dollar sign. The workaround is using [`constraints`](https://ast-grep.github.io/guide/project/lint-rule.html#constraints) in [YAML rule](https://ast-grep.github.io/guide/project/lint-rule.html) and [`regex`](https://ast-grep.github.io/guide/rule-config/atomic-rule.html#regex) rule. yaml rule: pattern: $HOOK($$$ARGS) constraints: HOOK: { regex: '^use' } [Example usage](https://ast-grep.github.io/playground.html#eyJtb2RlIjoiQ29uZmlnIiwibGFuZyI6ImphdmFzY3JpcHQiLCJxdWVyeSI6ImZvbygkJCRBLCBiLCAkJCRDKSIsInJld3JpdGUiOiIiLCJjb25maWciOiJydWxlOlxuICBwYXR0ZXJuOiAkSE9PSygkJCRBUkdTKVxuY29uc3RyYWludHM6XG4gIEhPT0s6IHsgcmVnZXg6IF51c2UgfSIsInNvdXJjZSI6ImZ1bmN0aW9uIFJlYWN0Q29tcG9uZW50KCkge1xuICAgIGNvbnN0IGRhdGEgPSBub3RIb28oKVxuICAgIGNvbnN0IFtmb28sIHNldEZvb10gPSB1c2VTdGF0ZSgnJylcbn0ifQ==) . MetaVariable must be one single AST node Meta variables cannot be mixed with prefix/suffix string . `use$HOOK` and `io_uring_$FUNC` are not valid meta variables. They are parsed as one AST node as whole, and ast-grep will not treat them as valid meta variable name. How to reuse rule for similar languages like TS/JS or C/C++? [​](https://ast-grep.github.io/advanced/faq.html#how-to-reuse-rule-for-similar-languages-like-ts-js-or-c-c) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------- ast-grep does not support multiple languages in one rule because: 1. **Different ASTs**: Similar languages still have different ASTs. For instance, [JS](https://ast-grep.github.io/playground.html#eyJtb2RlIjoiUGF0Y2giLCJsYW5nIjoiamF2YXNjcmlwdCIsInF1ZXJ5IjoiIiwicmV3cml0ZSI6IiIsInN0cmljdG5lc3MiOiJyZWxheGVkIiwic2VsZWN0b3IiOiIiLCJjb25maWciOiIiLCJzb3VyY2UiOiJmdW5jdGlvbiB0ZXN0KGEpIHt9In0=) and [TS](https://ast-grep.github.io/advanced/faq.html#eyJtb2RlIjoiUGF0Y2giLCJsYW5nIjoidHlwZXNjcmlwdCIsInF1ZXJ5IjoiIiwicmV3cml0ZSI6IiIsInN0cmljdG5lc3MiOiJyZWxheGVkIiwic2VsZWN0b3IiOiIiLCJjb25maWciOiIiLCJzb3VyY2UiOiJmdW5jdGlvbiB0ZXN0KGEpIHt9In0=) have different parsing trees for the same function declaration code. 2. **Different Kinds**: Similar languages may have different AST node kinds. Since ast-grep reports non-existing kinds as errors, there is no straightforward way to report error for kind only existing in one language. 3. **Debugging Experience**: Mixing languages in one rule requires users to test the rule in both languages. This can be confusing and error-prone, especially when unexpected results occur. Supporting multi-lang rule is a challenging task for both tool developers and users. Instead, we recommend two approaches: * **Always use the superset language**: Rule reusing usually happens when one language is a superset of another, e.g., TS and JS. In this case, you can use [`languageGlobs`](https://ast-grep.github.io/reference/sgconfig.html#languageglobs) to parse files in the superset language. This is more suitable if you don't need to distinguish between the two languages. * **Write Separate Rules**: Generate separate rules for each language. This approach is suitable when you do need to handle the differences between the languages. If you have a better, clearer and easier proposal to support multi-lang rule, please leave a comment under [this issue](https://github.com/ast-grep/ast-grep/issues/525) . Why is rule matching order sensitive? [​](https://ast-grep.github.io/advanced/faq.html#why-is-rule-matching-order-sensitive) ----------------------------------------------------------------------------------------------------------------------------- ast-grep's rule matching is a step-by-step process. It matches one atomic rule at a time, stores the matched meta-variable, and proceeds to the next rule until all rules are matched. **Rule matching is ordered** because previous rules' matched meta-variables can affect later rules. Only the first rule can specify what a `$META_VAR` matches, and later rules can only match the content captured by the first rule without modifying it. Let's see an example. Suppose we want to find a recursive function in JavaScript. [This rule](https://ast-grep.github.io/playground.html#eyJtb2RlIjoiQ29uZmlnIiwibGFuZyI6ImphdmFzY3JpcHQiLCJxdWVyeSI6ImZvbygkJCRBLCBiLCAkJCRDKSIsInJld3JpdGUiOiIiLCJjb25maWciOiJpZDogcmVjdXJzaXZlLWNhbGxcbmxhbmd1YWdlOiBKYXZhU2NyaXB0XG5ydWxlOlxuICBhbGw6XG4gIC0gcGF0dGVybjogZnVuY3Rpb24gJEYoKSB7ICQkJCB9XG4gIC0gaGFzOlxuICAgICAgcGF0dGVybjogJEYoKVxuICAgICAgc3RvcEJ5OiBlbmRcbiIsInNvdXJjZSI6ImZ1bmN0aW9uIHJlY3Vyc2UoKSB7XG4gICAgZm9vKClcbiAgICByZWN1cnNlKClcbn0ifQ==) can do the trick. rule.ymlmatch.js yml id: recursive-call language: JavaScript rule: all: - pattern: function $F() { $$$ } - has: pattern: $F() stopBy: end js function recurse() { foo() recurse() } The rule works because the pattern `function $F() { $$$ }` matches first, capturing `$F` as `recurse`. The later `has` rule then looks for a `recurse()` call based on the matched `$F`. If we [swap the order of rules](https://ast-grep.github.io/playground.html#eyJtb2RlIjoiQ29uZmlnIiwibGFuZyI6ImphdmFzY3JpcHQiLCJxdWVyeSI6ImZvbygkJCRBLCBiLCAkJCRDKSIsInJld3JpdGUiOiIiLCJjb25maWciOiJpZDogcmVjdXJzaXZlLWNhbGxcbmxhbmd1YWdlOiBKYXZhU2NyaXB0XG5ydWxlOlxuICBhbGw6XG4gIC0gaGFzOlxuICAgICAgcGF0dGVybjogJEYoKVxuICAgICAgc3RvcEJ5OiBlbmRcbiAgLSBwYXR0ZXJuOiBmdW5jdGlvbiAkRigpIHsgJCQkIH1cbiIsInNvdXJjZSI6ImZ1bmN0aW9uIHJlY3Vyc2UoKSB7XG4gICAgZm9vKClcbiAgICByZWN1cnNlKClcbn0ifQ==) , it will produce no match. yml id: recursive-call language: JavaScript rule: all: - has: # N.B. has is the first rule pattern: $F() stopBy: end - pattern: function $F() { $$$ } In this case, the `has` rule matches first and captures `$F` as `foo` since `foo()` is the first function call matching the pattern `$F()`. The later rule `function $F() { $$$ }` will only find the `foo` declaration instead of `recurse`. TIP Using `all` to specify the order of rule matching can be helpful when debugging YAML rules. What does unordered rule object imply? [​](https://ast-grep.github.io/advanced/faq.html#what-does-unordered-rule-object-imply) ------------------------------------------------------------------------------------------------------------------------------- A rule object in ast-grep is an unordered dictionary. The order of rule application is implementation-defined. Currently, ast-grep applies atomic rules first, then composite rules, and finally relational rules. If your rule depends on using meta variables in later rules, the best way is to use the `all` rule to specify the order of rules. `kind` and `pattern` rules are not working together, why? [​](https://ast-grep.github.io/advanced/faq.html#kind-and-pattern-rules-are-not-working-together-why) ---------------------------------------------------------------------------------------------------------------------------------------------------------------- The most common scenario is that your pattern is parsed as a different AST node than you expected. And you may use `kind` rule to filter out the AST node you want to match. This does not work in ast-grep for two reasons: 1. tree-sitter, the underlying parser library, does not offer a way to parse a string of a specific kind. So `kind` rule cannot be used to change the parsing outcome of a `pattern`. 2. ast-grep rules are mostly independent of each other, except sharing meta-variables during a match. `pattern` will behave the same regardless of another `kind` rule. To specify the `kind` of a `pattern`, you need to use [pattern](https://ast-grep.github.io/guide/rule-config/atomic-rule.html#pattern-object) [object](https://ast-grep.github.io/advanced/pattern-parse.html#incomplete-pattern-code) . For example, to match class field in JavaScript, a kind + pattern rule [will not work](https://ast-grep.github.io/playground.html#eyJtb2RlIjoiQ29uZmlnIiwibGFuZyI6ImphdmFzY3JpcHQiLCJxdWVyeSI6IiIsInJld3JpdGUiOiIiLCJzdHJpY3RuZXNzIjoic21hcnQiLCJzZWxlY3RvciI6IiIsImNvbmZpZyI6InJ1bGU6XG4gIHBhdHRlcm46IGEgPSAxMjNcbiAga2luZDogZmllbGRfZGVmaW5pdGlvbiIsInNvdXJjZSI6ImNsYXNzIEEge1xuICAgIGEgPSAxMjNcbn0ifQ==) : yaml # these are two separate rules pattern: a = 123 # rule 1 kind: field_definition # rule 2 This is because pattern `a = 123` is parsed as [`assignment_expression`](https://ast-grep.github.io/playground.html#eyJtb2RlIjoiUGF0Y2giLCJsYW5nIjoiamF2YXNjcmlwdCIsInF1ZXJ5IjoiYSA9IDEyMyIsInJld3JpdGUiOiIiLCJzdHJpY3RuZXNzIjoic21hcnQiLCJzZWxlY3RvciI6IiIsImNvbmZpZyI6IiIsInNvdXJjZSI6IiJ9) . Pattern and kind are two separate rules. And using them together will match nothing because no AST will have both `assignment_expression` and `field_definition` kind at once. Instead, you need to use pattern object to provide enough context code for the parser to parse the code snippet as `field_definition`: yaml # this is one single pattern rule! pattern: context: 'class A { a = 123 }' # provide full context code selector: field_definition # select the effective pattern Note the rule above is one single pattern rule, instead of two. The `context` field provides the full unambiguous code snippet of `class`. So the `a = 123` will be parsed as `field_definition`. The `selector` field then selects the `field_definition` node as the [effective pattern](https://ast-grep.github.io/advanced/pattern-parse.html#steps-to-create-a-pattern) matcher. Does ast-grep support some advanced static analysis? [​](https://ast-grep.github.io/advanced/faq.html#does-ast-grep-support-some-advanced-static-analysis) ----------------------------------------------------------------------------------------------------------------------------------------------------------- Short answer: **NO**. Long answer: ast-grep at the moment does not support the following information: * [scope analysis](https://eslint.org/docs/latest/extend/scope-manager-interface) * [type information](https://semgrep.dev/docs/writing-rules/pattern-syntax#typed-metavariables) * [control flow analysis](https://en.wikipedia.org/wiki/Control-flow_analysis) * [data flow analysis](https://en.wikipedia.org/wiki/Data-flow_analysis) * [taint analysis](https://semgrep.dev/docs/writing-rules/data-flow/taint-mode) * [constant propagation](https://semgrep.dev/docs/writing-rules/data-flow/constant-propagation) More concretely, it is not easy, or even possible, to achieve the following tasks in ast-grep: * Find variables that are not defined/used in the current scope. * Find variables of a specific type. * Find code that is unreachable. * Find code that is always executed. * Identify the flow of user input. Also see [tool comparison](https://ast-grep.github.io/advanced/tool-comparison.html) for more information. I don't want to read the docs / I don't understand the docs / The docs are too long / I have an urgent request [​](https://ast-grep.github.io/advanced/faq.html#i-don-t-want-to-read-the-docs-i-don-t-understand-the-docs-the-docs-are-too-long-i-have-an-urgent-request) -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- [Open Source Software](https://antfu.me/posts/why-reproductions-are-required) is served "as-is" by volunteers. We appreciate your interest in ast-grep, but we also have limited time and resources to address every request. We appreciate constructive feedback and are always looking for ways to improve the documentation and the tool itself. There are several ways you can help us or yourself: * Ask [Copilot](https://copilot.microsoft.com/) or other AI assistants to help you understand the docs. * Provide feedbacks or pull requests on the [documentation](https://github.com/ast-grep/ast-grep.github.io) . * Browse [Discord](https://discord.com/invite/4YZjf6htSQ) , [StackOverflow](https://stackoverflow.com/questions/tagged/ast-grep) or [Reddit](https://www.reddit.com/r/astgrep/) . ~If you just want an answer without effort, let the author [write a rule for you](https://github.com/sponsors/HerringtonDarkholme) .~ --- # Find & Patch: A Novel Functional Programming like Code Rewrite Scheme | ast-grep [Skip to content](https://ast-grep.github.io/advanced/find-n-patch.html#VPContent) On this page Are you an LLM? You can read better optimized documentation at /advanced/find-n-patch.md for this page in Markdown format Find & Patch: A Novel Functional Programming like Code Rewrite Scheme [​](https://ast-grep.github.io/advanced/find-n-patch.html#find-patch-a-novel-functional-programming-like-code-rewrite-scheme) ==================================================================================================================================================================================================== Introduction [​](https://ast-grep.github.io/advanced/find-n-patch.html#introduction) ------------------------------------------------------------------------------------- Code transformation is a powerful technique that allows you to modify your code programmatically. There are many tools that can help you with code transformation, such as [Babel](https://babeljs.io/) /[biome](https://github.com/biomejs/biome/discussions/1762) for JavaScript/TypeScript, [libcst](https://libcst.readthedocs.io/en/latest/) for Python, or [Rector](https://getrector.com/) for PHP. Most of these tools use imperative APIs to manipulate the [abstract syntax tree](https://www.wikiwand.com/en/Abstract_syntax_tree) (AST) of your code. In this post, we will introduce a different approach to code transformation called **Find & Patch**. This scheme lets you rewrite complex code using a fully declarative [Domain-Specific Language](https://www.wikiwand.com/en/Domain-specific_language) (DSL). While the scheme is powerful, the underlying concept is simple: find certain nodes, rewrite them, and recursively repeat the rewriting. The idea of Find & Patch comes from developing [ast-grep](https://ast-grep.github.io/) , a tool using AST to find and replace code patterns. We realized that this approach can be generalized and extended to support more complex and diverse code transformations! At the end of this article, we will compare Find & Patch to functional programming on the tree of syntax nodes. You can apply filter nodes using `rule`, map them via `transform`, and compose them with `rewriters`. This gives you a lot of flexibility and expressiveness to manipulate your code! What is ast-grep? [​](https://ast-grep.github.io/advanced/find-n-patch.html#what-is-ast-grep) ---------------------------------------------------------------------------------------------- [ast-grep](https://github.com/ast-grep/ast-grep) is a tool to search and rewrite code based on ASTs. It is like `grep` for code, but with the power of ASTs. More concretely, ast-grep can find code patterns using its [rule system](https://ast-grep.github.io/guide/rule-config/atomic-rule.html) . It can also rewrite the matched code using [meta-variables](https://ast-grep.github.io/guide/pattern-syntax.html#meta-variable) based on the rule. ast-grep's rewriting can be seen as two steps: finding target nodes and patching them with new text. Find and Patch: How ast-grep Rewrites Code [​](https://ast-grep.github.io/advanced/find-n-patch.html#find-and-patch-how-ast-grep-rewrites-code) ------------------------------------------------------------------------------------------------------------------------------------------------ The basic rewriting workflow of ast-grep is like below: 1. _Find_: search the nodes in the AST that match the rewriter rules (hence the name ast-grep). 2. _Rewrite_: generate a new string based on the matched meta-variables. 3. _Patch_: replace the node text with the generated fix. Let's see a simple example: replace `console.log` with `logger.log`. The following rule will do the trick. yaml rule: pattern: console.log($MSG) fix: logger.log($MSG) The rule above is quite straightforward. It matches the `console.log` call, using the pattern, and replaces it with the `logger.log` call. The meta-variable `$MSG` captures the argument of `console.log` and is used in the `fix` field. ast-grep also has several other fields to fine-tune the process. The core fields in ast-grep's rule map naturally to the idea of **Find & Patch**. * **Find** * Find a target node based on the [`rule`](https://ast-grep.github.io/reference/rule.html) * Filter the matched nodes based on [`constraints`](https://ast-grep.github.io/reference/yaml.html#constraints) * **Patch** * Rewrite the matched meta-variable based on [`transform`](https://ast-grep.github.io/reference/yaml/transformation.html) * Replace the matched node with [`fix`](https://ast-grep.github.io/reference/yaml/fix.html) , which can use the transformed meta-variables. Limitation of the Current Workflow [​](https://ast-grep.github.io/advanced/find-n-patch.html#limitation-of-the-current-workflow) --------------------------------------------------------------------------------------------------------------------------------- However, this workflow has a limitation: it can only replace one node at a time, which means that we cannot handle complex transformations that involve multiple nodes or lists of nodes. For example, suppose we want to rewrite barrel imports to single imports. A [barrel import](https://adrianfaciu.dev/posts/barrel-files/) is a way to consolidate the exports of multiple modules into a single convenient module that can be imported using a single import statement. For instance: js import {a, b, c} from './barrel'; This imports three modules (`a`, `b`, and `c`) from a single barrel file (`barrel.js`) that re-exports them. Rewriting this to single imports has [some](https://vercel.com/blog/how-we-optimized-package-imports-in-next-js) [benefits](https://marvinh.dev/blog/speeding-up-javascript-ecosystem-part-7/) , such as reducing [bundle size](https://dev.to/tassiofront/barrel-files-and-why-you-should-stop-using-them-now-bc4) or avoiding [conflicting names](https://flaming.codes/posts/barrel-files-in-javascript/) . js import a from './barrel/a'; import b from './barrel/b'; import c from './barrel/c'; This imports each module directly from its own file, without going through the barrel file. With the simple "Find and Patch" workflow, we cannot achieve this transformation easily. We either have to rewrite the whole import statement or rewrite each identifier one by one. We cannot replace the whole import statement because we cannot process the multiple identifiers, which requires processing a list of nodes at one time. Can we rewrite the identifiers one by one? This also fails because we cannot replace the whole import statement, so there will be unwanted import statement text surrounding the identifiers. javascript // we cannot rewrite the whole import statements // because we don't know how to rewrite a, b, c as a list import ??? from './barrel'; // we cannot rewrite each identifier // because the replaced text is inside the import statement import { ??, ??, ?? } from './barrel'; We need a better way to rewrite code that involves multiple nodes or lists of nodes. And here comes **Find & Patch**. Extend the Concept of `Find` and `Patch` [​](https://ast-grep.github.io/advanced/find-n-patch.html#extend-the-concept-of-find-and-patch) ----------------------------------------------------------------------------------------------------------------------------------------- Let's reflect: what limits us from rewriting the code above? Our old workflow does not allow us to apply a rule to multiple sub-nodes of a node. (This is like not being able to write for loops.) Nor does it allow us to generate different text for different sub-nodes in a rule. (This is like not being able to write if/switch statements.) I initially thought of adding [list comprehension](https://github.com/ast-grep/ast-grep/issues/723#issuecomment-1890362116) to transform to overcome these limitations. However, list comprehension will introduce more concepts like loops, filters and probably nested loops. I prefer having [Occam's razor](https://www.wikiwand.com/en/Occam%27s_razor) to shave off unnecessary constructs. Luckily, [Mosenkis](https://github.com/emosenkis) proposed the [refreshing idea](https://github.com/ast-grep/ast-grep/issues/723#issuecomment-1883526774) that we can apply sub-rules, called `rewriters`, to specific nodes during matching. It can elegantly solve the issue of processing multiple nodes with multiple different rules! The idea is simple: we will add three new, but similar, steps in the rewriting step. 1. _Find_ a list of different sub-nodes under a meta-variable that match different rewriters. 2. _Generate_ a different fix for each sub-node based on the matched rewriter sub-rule. 3. _Join_ the fixes together and store the string in a new metavariable for later use. The new steps are similar to the existing **"Find and Patch"** workflow. It is like recursively applying the old workflow to matched nodes! We can, taking the previous barrel import as an example, first match the import statement and then apply the rewriter sub-rule to each identifier. Intriguing Example [​](https://ast-grep.github.io/advanced/find-n-patch.html#intriguing-example) ------------------------------------------------------------------------------------------------- The idea above is implemented by a new [`rewriters`](https://ast-grep.github.io/reference/yaml/rewriter.html) field and a new [`rewrite`](https://ast-grep.github.io/reference/yaml/transformation.html#rewrite) transformation. **Our first step is to write a rule to capture the import statement.** yaml rule: pattern: import {$$$IDENTS} from './barrel' This will capture the imported identifiers `a, b, c` in `$$$IDENTS`. **Next, we need to transform `$$$IDENTS` to individual imports.** The idea is that we can find the identifier nodes in the `$$$IDENT` and rewrite them to individual imports. To do this, we register a rewriter that acts as a separate rewriter rule for each identifier. yaml rewriters: - id: rewrite-identifer rule: pattern: $IDENT kind: identifier fix: import $IDENT from './barrel/$IDENT' The `rewrite-identifier` above will: 1. First, find each `identifier` AST node and capture it as `$IDENT`. 2. Rewrite the identifier to a new import statement. For example, the rewriter will change identifier `a` to `import a from './barrel/a'`. **We can now apply the rewriter to the matched variable `$$$IDENTS`.** The counterpart of `rewriter` is the `rewrite` transformation, which applies the rewriter to a matched variable and generates a new string. The yaml fragment below uses `rewrite` to find identifiers in `$$$IDENTS`, as specified in `rewrite-identifier`'s rule, and rewrites it to single import statement. yaml transform: IMPORTS: rewrite: rewriters: [rewrite-identifer] source: $$$IDENTS joinBy: "\n" Note the `joinBy` field in the transform section. It specifies how to join the rewritten import statements with a newline character. This means that each identifier will generate a separate import statement, followed by a newline. **Finally, we can use the transformed `IMPORTS` in the `fix` field to replace the original import statement.** The final rule will be like this. See the [online playground](https://ast-grep.github.io/playground.html#eyJtb2RlIjoiQ29uZmlnIiwibGFuZyI6ImphdmFzY3JpcHQiLCJxdWVyeSI6IiIsInJld3JpdGUiOiIiLCJjb25maWciOiJydWxlOlxuICBwYXR0ZXJuOiBpbXBvcnQgeyQkJElERU5UU30gZnJvbSAnLi9iYXJyZWwnXG5yZXdyaXRlcnM6XG4tIGlkOiByZXdyaXRlLWlkZW50aWZlclxuICBydWxlOlxuICAgIHBhdHRlcm46ICRJREVOVFxuICAgIGtpbmQ6IGlkZW50aWZpZXJcbiAgZml4OiBpbXBvcnQgJElERU5UIGZyb20gJy4vYmFycmVsLyRJREVOVCdcbnRyYW5zZm9ybTpcbiAgSU1QT1JUUzpcbiAgICByZXdyaXRlOlxuICAgICAgcmV3cml0ZXJzOiBbcmV3cml0ZS1pZGVudGlmZXJdXG4gICAgICBzb3VyY2U6ICQkJElERU5UU1xuICAgICAgam9pbkJ5OiBcIlxcblwiXG5maXg6ICRJTVBPUlRTIiwic291cmNlIjoiaW1wb3J0IHsgYSwgYiwgYyB9IGZyb20gJy4vYmFycmVsJzsifQ==) . yaml rule: pattern: import {$$$IDENTS} from './barrel' rewriters: - id: rewrite-identifer rule: pattern: $IDENT kind: identifier fix: import $IDENT from './barrel/$IDENT' transform: IMPORTS: rewrite: rewriters: [rewrite-identifer] source: $$$IDENTS joinBy: "\n" fix: $IMPORTS Similarity to Functional Programming [​](https://ast-grep.github.io/advanced/find-n-patch.html#similarity-to-functional-programming) ------------------------------------------------------------------------------------------------------------------------------------- Find & Patch is a scheme that allows us to manipulate the syntax tree of the code in a declarative way. It reminds me of Rust declarative macro since both Find & Patch and Rust declarative macro can: * Match a list of nodes/tokens based on patterns: ast-grep's rule vs. Rust macro pattern matcher. * Break nodes/tokens into sub parts: ast-grep's metavariable vs. Rust macro variable. * Recursively use subparts to call other rewrite/macros. The idea can be further compared to functional programming! We can use different rules to match and transform different sub-nodes of the tree, just like using [pattern matching](https://www.wikiwand.com/en/Pattern_matching) in functional languages. We can also apply rules to multiple sub-nodes at once, just like using for-comprehension or map/filter/reduce. Moreover, we can break down a large syntax tree into smaller sub-trees by using meta-variables, just like using destructuring or [elimination rules](https://blog.jez.io/intro-elim/) in functional languages. But all of these can be boiled down to two simple idea: **Finding** nodes and **Patching** nodes! Find & Patch is a simple and elegant scheme that is tailored for AST manipulation, but it can achieve similar transformations as a general-purpose functional programming language doing rewrites! We can think of Find & Patch as a form of "Functional Programming" over the AST! And they both have the same acronym btw. * * * Hope you find this scheme useful and interesting, and I sincerely invite you to try it out with ast-grep. Thank you for reading~ --- # How ast-grep Works: A bird's-eye view | ast-grep [Skip to content](https://ast-grep.github.io/advanced/how-ast-grep-works.html#VPContent) On this page Are you an LLM? You can read better optimized documentation at /advanced/how-ast-grep-works.md for this page in Markdown format How ast-grep Works: A bird's-eye view [​](https://ast-grep.github.io/advanced/how-ast-grep-works.html#how-ast-grep-works-a-bird-s-eye-view) ============================================================================================================================================ In the world of software development, efficiently searching, rewriting, linting, and analyzing code is essential for maintaining high-quality projects. This is where **ast-grep** comes into play. Designed as a powerful structural search tool, ast-grep simplifies these tasks by leveraging the Abstract Syntax Tree (AST) representation of code. Let's break down how ast-grep works with the help of a diagram. ![Workflow](https://ast-grep.github.io/image/diagram.png) The Workflow of ast-grep [​](https://ast-grep.github.io/advanced/how-ast-grep-works.html#the-workflow-of-ast-grep) ------------------------------------------------------------------------------------------------------------------- Generally speaking, ast-grep takes user _queries of various input_ formats, _parses the code into an AST_ using TreeSitter, and performs _search, rewrite, lint, and analysis_, utilizing the full power of CPU cores. ### **Query via Various Formats** [​](https://ast-grep.github.io/advanced/how-ast-grep-works.html#query-via-various-formats) ast-grep can accept queries in multiple formats, making it flexible and user-friendly. Here are some common query formats: * **Pattern Query**: Users can define [specific patterns](https://ast-grep.github.io/guide/pattern-syntax.html) to search for within their codebase. * **YAML Rule**: Structured rules written in [YAML](https://ast-grep.github.io/guide/rule-config.html) format to guide the search and analysis process. * **API Code**: Direct [API calls](https://ast-grep.github.io/guide/api-usage.html) for more programmatic control over the searching and rewriting tasks. ### ast-grep's Core [​](https://ast-grep.github.io/advanced/how-ast-grep-works.html#ast-grep-s-core) ast-grep's core functionality is divided into two main components: parsing and matching. #### 1\. **Parsing with Tree-Sitter** [​](https://ast-grep.github.io/advanced/how-ast-grep-works.html#_1-parsing-with-tree-sitter) The core of ast-grep's functionality relies on **Tree-Sitter Parsers**. [TreeSitter](https://tree-sitter.github.io/) is a powerful parsing library that converts source code into an Abstract Syntax Tree (AST). This tree structure represents the syntactic structure of the code, making it easier to analyze and manipulate. #### 2\. **Tree Matching** [​](https://ast-grep.github.io/advanced/how-ast-grep-works.html#_2-tree-matching) Once the code is parsed into an AST, the ast-grep core takes over and finds the matching AST nodes based on the input queries. Written in **Rust**, ast-grep ensures efficient performance by utilizing full CPU cores. This means it can handle large codebases and perform complex searches and transformations quickly. ### **Usage Scenarios** [​](https://ast-grep.github.io/advanced/how-ast-grep-works.html#usage-scenarios) ast-grep can be helpful for these scenarios. * **Search**: Find specific patterns or constructs within the code. * **Rewrite**: Automatically refactor or transform code based on predefined rules or patterns. * **Lint**: Identify and report potential issues or code smells. * **Analyze**: Perform in-depth code analysis to gather insights and metrics. Benefits of Using ast-grep [​](https://ast-grep.github.io/advanced/how-ast-grep-works.html#benefits-of-using-ast-grep) ----------------------------------------------------------------------------------------------------------------------- * **Multi-Core Processing**: ast-grep can handle multiple files in parallel by taking full advantage of multi-core processors. Typically ast-grep performs tasks faster than many other tools, making it suitable for large projects. * **Versatility**: Whether you need to search for a specific code pattern, rewrite sections of code, lint for potential issues, or perform detailed analysis, ast-grep has you covered. Example in the Real World [​](https://ast-grep.github.io/advanced/how-ast-grep-works.html#example-in-the-real-world) --------------------------------------------------------------------------------------------------------------------- * **Pattern + Search**: [CodeRabbit](https://coderabbit.ai/) uses ast-grep patterns to search code repo for code review knowledge. This example is collected from ast-grep's own [dogfooding](https://github.com/ast-grep/ast-grep/pull/780#discussion_r1425817237) . * **API + Rewrite**: [@vue-macros/cli](https://github.com/vue-macros/vue-macros-cli) is a CLI for rewriting at Vue Macros powered by ast-grep. * **YAML + Lint**: [Vercel turbo](https://github.com/vercel/turbo/pull/8275) is using ast-grep to lint their Rust code with [custom rules](https://github.com/vercel/turbo/blob/main/.config/ast-grep/rules/no-context.yml) . Conclusion [​](https://ast-grep.github.io/advanced/how-ast-grep-works.html#conclusion) --------------------------------------------------------------------------------------- ast-grep is a versatile and efficient tool for modern software development needs. By parsing code into an Abstract Syntax Tree and leveraging the power of Rust, it provides robust capabilities for searching, rewriting, linting, and analyzing code. With multiple input formats and the ability to utilize full CPU cores, ast-grep is designed to handle the demands of today's complex codebases. Whether you are maintaining a small project or a large enterprise codebase, ast-grep can help streamline your development workflow. --- # Deep Dive into ast-grep's Match Algorithm | ast-grep [Skip to content](https://ast-grep.github.io/advanced/match-algorithm.html#VPContent) On this page Are you an LLM? You can read better optimized documentation at /advanced/match-algorithm.md for this page in Markdown format Deep Dive into ast-grep's Match Algorithm [​](https://ast-grep.github.io/advanced/match-algorithm.html#deep-dive-into-ast-grep-s-match-algorithm) ================================================================================================================================================== By default, ast-grep uses a smart strategy to match pattern against the AST node. All nodes in the pattern must be matched, but it will skip unnamed nodes in target code. For background and the definition of **_named_** and **_unnamed_** nodes, please refer to the [core concepts](https://ast-grep.github.io/advanced/core-concepts.html) doc. How ast-grep's Smart Matching Works [​](https://ast-grep.github.io/advanced/match-algorithm.html#how-ast-grep-s-smart-matching-works) -------------------------------------------------------------------------------------------------------------------------------------- Let's see an example in action. The following pattern `function $A() {}` will match both plain function and async function in JavaScript. See [playground](https://ast-grep.github.io/playground.html#eyJtb2RlIjoiUGF0Y2giLCJsYW5nIjoiamF2YXNjcmlwdCIsInF1ZXJ5IjoiZnVuY3Rpb24gJEEoKSB7fSIsInJld3JpdGUiOiJEZWJ1Zy5hc3NlcnQiLCJjb25maWciOiJydWxlOlxuICBwYXR0ZXJuOiBcbiAgICBjb250ZXh0OiAneyAkTTogKCQkJEEpID0+ICRNQVRDSCB9J1xuICAgIHNlbGVjdG9yOiBwYWlyXG4iLCJzb3VyY2UiOiJmdW5jdGlvbiBhKCkge31cbmFzeW5jIGZ1bmN0aW9uIGEoKSB7fSJ9) js // function $A() {} function foo() {} // matched async function bar() {} // matched This is because the keyword `async` is an unnamed node in the syntax tree, so the `async` in the code to search is skipped. As long as `function`, `$A` and `{}` are matched, the pattern is considered matched. However, if the `async` keyword appears in the pattern code, it will [not be skipped](https://ast-grep.github.io/playground.html#eyJtb2RlIjoiUGF0Y2giLCJsYW5nIjoiamF2YXNjcmlwdCIsInF1ZXJ5IjoiYXN5bmMgZnVuY3Rpb24gJEEoKSB7fSIsInJld3JpdGUiOiJ1c2luZyBuYW1lc3BhY2UgZm9vOjokQTsiLCJjb25maWciOiJcbmlkOiB0ZXN0YmFzZV9pbml0aWFsaXplclxubGFuZ3VhZ2U6IENQUFxucnVsZTpcbiAgcGF0dGVybjpcbiAgICBzZWxlY3RvcjogY29tcG91bmRfc3RhdGVtZW50XG4gICAgY29udGV4dDogXCJ7ICQkJEIgfVwiXG5maXg6IHwtXG4gIHtcbiAgICBmKCk7XG4gICAgJCQkQlxuICB9Iiwic291cmNlIjoiLy8gYXN5bmMgZnVuY3Rpb24gJEEoKSB7fVxuZnVuY3Rpb24gZm9vKCkge30gICAgLy8gbm90IG1hdGNoZWRcbmFzeW5jIGZ1bmN0aW9uIGJhcigpIHt9IC8vIG1hdGNoZWRcbiJ9) and is required to match node in the code. js // async function $A() {} function foo() {} // not matched async function bar() {} // matched The design principle here is that the less a pattern specifies, the more code it can match. Every nodes the pattern author spells out will be respected by ast-grep's matching algorithm by default. Smart is Sometimes Dumb [​](https://ast-grep.github.io/advanced/match-algorithm.html#smart-is-sometimes-dumb) -------------------------------------------------------------------------------------------------------------- The smart algorithm does not always behave as desired. There are cases where we need more flexibility in the matching algorithm. We may want to ignore all CST trivia nodes. Or even we want to ignore comment AST nodes. Suppose we want to write a pattern to match import statement in JavaScript. The pattern `import $A from 'lib'` will match only `import A from 'lib'`, but not `import A from "lib"`. This is because the import string has different quotation marks. We do want to ignore the trivial unnamed nodes here. To this end, ast-grep implements different pattern matching algorithms to provide more flexibility to the users, and every pattern can have their own matching algorithm to fine-tune the matching behavior. Matching Algorithm Strictness [​](https://ast-grep.github.io/advanced/match-algorithm.html#matching-algorithm-strictness) -------------------------------------------------------------------------------------------------------------------------- Different matching algorithm is controlled by **pattern strictness**. Strictness Strictness is defined in terms of what nodes can be _skipped_ during matching. A _stricter_ matching algorithm will _skip fewer nodes_ and accordingly _produce fewer matches_. Currently, ast-grep has these strictness levels. * `cst`: All nodes in the pattern and target code must be matched. No node is skipped. * `smart`: All nodes in the pattern must be matched, but it will skip unnamed nodes in target code. This is the default behavior. * `ast`: Only named AST nodes in both pattern and target code are matched. All unnamed nodes are skipped. * `relaxed`: Named AST nodes in both pattern and target code are matched. Comments and unnamed nodes are ignored. * `signature`: Only named AST nodes' kinds are matched. Comments, unnamed nodes and text are ignored. Strictness Examples [​](https://ast-grep.github.io/advanced/match-algorithm.html#strictness-examples) ------------------------------------------------------------------------------------------------------ Let's see how strictness `ast` will impact matching. In our previous import lib example, the pattern `import $A from 'lib'` will match both two statements. js import $A from 'lib' // pattern import A1 from 'lib' // match, quotation is ignored import A2 from "lib" // match, quotation is ignored import A3 from "not" // no match, string_fragment is checked First, the pattern and code will be parsed as the tree below. Named The unnamed nodes are skipped during the matching. Nodes' namedness is annotated beside them. import_statement // named import // unnamed import_clause // named identifier // named from // unnamed string // named " // unnamed string_fragment // named " // unnamed Under the strictness of `ast`, the full syntax tree will be reduced to an Abstract Syntax Tree where only named nodes are kept. import_statement import_clause identifier // $A string string_fragment // lib As long as the tree structure matches and the meta-variable `$A` and string\_fragment `lib` are matched, the pattern and code are counted as a match. * * * Another example will be matching the pattern `foo(bar)` across different strictness levels: ts // exact match in all levels foo(bar) // match in all levels except cst due to the trailing comma in code foo(bar,) // match in relaxed and signature because comment is skipped foo(/* comment */ bar) // match in signature because text content is ignored bar(baz) Strictness Table [​](https://ast-grep.github.io/advanced/match-algorithm.html#strictness-table) ------------------------------------------------------------------------------------------------ Strictness considers both nodes' namedness and their locations, i.e, _is the node named_ and _is the node in pattern or code_ The table below summarize how nodes are skipped during matching. | Strictness | Named Node in Pattern | Named Node in Code to Search | Unnamed Node in Pattern | Unnamed Node in Code to Search | | --- | --- | --- | --- | --- | | `cst` | Keep | Keep | Keep | Keep | | `smart` | Keep | Keep | Keep | Skip | | `ast` | Keep | Keep | Skip | Skip | | `relaxed` | Skip comment | Skip comment | Skip | Skip | | `signature` | Skip comment. Ignore text | Skip comment. Ignore text | Skip | Skip | Configure Strictness [​](https://ast-grep.github.io/advanced/match-algorithm.html#configure-strictness) -------------------------------------------------------------------------------------------------------- ast-grep has two ways to configure pattern strictness. 1. Using `--strictness` in `ast-grep run` You can use the `--strictness` flag in [`ast-grep run`](https://ast-grep.github.io/reference/cli/run.html) bash ast-grep run -p '$FOO($BAR)' --strictness ast 2. Using `strictness` in Pattern Object [Pattern object](https://ast-grep.github.io/reference/rule.html#pattern) in YAML has an optional `strictness` field. id: test-pattern-strictness language: JavaScript rule: pattern: context: $FOO($BAR) strictness: ast --- # Search Multi-language Documents in ast-grep | ast-grep [Skip to content](https://ast-grep.github.io/advanced/language-injection.html#VPContent) On this page Are you an LLM? You can read better optimized documentation at /advanced/language-injection.md for this page in Markdown format Search Multi-language Documents in ast-grep [​](https://ast-grep.github.io/advanced/language-injection.html#search-multi-language-documents-in-ast-grep) ========================================================================================================================================================= Introduction [​](https://ast-grep.github.io/advanced/language-injection.html#introduction) ------------------------------------------------------------------------------------------- ast-grep works well searching files of one single language, but it is hard to extract a sub language embedded inside a document. However, in modern development, it's common to encounter **multi-language documents**. These are source files containing code written in multiple different languages. Notable examples include: * **HTML files**: These can contain JavaScript inside ` Running this ast-grep command will extract the matching CSS style code out of the HTML file! sh ast-grep run -p 'color: $COLOR' ast-grep outputs this beautiful CLI report. shell test.html 2│ h1 { color: red; } ast-grep works well even if just providing the pattern without specifying the pattern language! ### **Using `ast-grep scan`**: find JavaScript in HTML with rule files [​](https://ast-grep.github.io/advanced/language-injection.html#using-ast-grep-scan-find-javascript-in-html-with-rule-files) You can also use ast-grep's [rule file](https://ast-grep.github.io/guide/rule-config.html) to search injected languages. For example, we can warn the use of `alert` in JavaScript, even if it is inside the HTML file. yml id: no-alert language: JavaScript severity: warning rule: pattern: alert($MSG) message: Prefer use appropriate custom UI instead of obtrusive alert call. The rule above will detect usage of `alert` in JavaScript. Running the rule via `ast-grep scan`. sh ast-grep scan --rule no-alert.yml The command leverages built-in behaviors in ast-grep to handle language injection seamlessly. It will produce the following warning message for the HTML file above. sh warning[no-alert]: Prefer use appropriate custom UI instead of obtrusive alert call. ┌─ test.html:8:3 │ 8 │ alert('hello world!') │ ^^^^^^^^^^^^^^^^^^^^^ How language injections work? [​](https://ast-grep.github.io/advanced/language-injection.html#how-language-injections-work) ---------------------------------------------------------------------------------------------------------------------------- ast-grep employs a multi-step process to handle language injections effectively. Here's a detailed breakdown of the workflow: 1. **File Discovery**: The CLI first discovers files on the disk via the venerable [ignore](https://crates.io/crates/ignore) crate, the same library under [ripgrep](https://github.com/BurntSushi/ripgrep) 's hood. 2. **Language Inference**: ast-grep infers the language of each discovered file based on file extensions. 3. **Injection Extraction**: For documents that contain code written in multiple languages (e.g., HTML with embedded JS), ast-grep extracts the injected language sub-regions. _At the moment, ast-grep handles HTML/JS/CSS natively_. 4. **Code Matching**: ast-grep matches the specified patterns or rules against these regions. Pattern code will be interpreted according to the injected language (e.g. JS/CSS), instead of the parent document language (e.g. HTML). Customize Language Injection: styled-components in JavaScript [​](https://ast-grep.github.io/advanced/language-injection.html#customize-language-injection-styled-components-in-javascript) -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- You can customize language injection via the `sgconfig.yml` [configuration file](https://ast-grep.github.io/reference/sgconfig.html) . This allows you to specify how ast-grep handles multi-language documents based on your specific needs, without modifying ast-grep's built-in behaviors. Let's see an example of searching CSS code in JavaScript. [styled-components](https://styled-components.com/) is a library for styling React applications using [CSS-in-JS](https://bootcamp.uxdesign.cc/css-in-js-libraries-for-styling-react-components-a-comprehensive-comparison-56600605a5a1) . It allows you to write CSS directly within your JavaScript via [tagged template literals](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Template_literals) , creating styled elements as React components. The example will configure ast-grep to detect styled-components' CSS. ### Injection Configuration [​](https://ast-grep.github.io/advanced/language-injection.html#injection-configuration) You can add the `languageInjections` section in the project configuration file `sgconfig.yml`. yaml languageInjections: - hostLanguage: js rule: pattern: styled.$TAG`$CONTENT` injected: css Let's break the configuration down. 1. `hostLanguage`: Specifies the main language of the document. In this example, it is set to `js` (JavaScript). 2. `rule`: Defines the ast-grep rule to identify the injected language region within the host language. * `pattern`: The pattern matches styled components syntax where `styled` is followed by a tag (e.g., `button`, `div`) and a template literal containing CSS. * the rule should have a meta variable `$CONTENT` to specify the subregion of injected language. In this case, it is the content inside the template string. 3. `injected`: Specifies the injected language within the identified regions. In this case, it is `css`. ### Example Match [​](https://ast-grep.github.io/advanced/language-injection.html#example-match) Consider a JSX file using styled components: js import styled from 'styled-components'; const Button = styled.button` background: red; color: white; padding: 10px 20px; border-radius: 3px; ` export default function App() { return } With the above `languageInjections` configuration, ast-grep will: 1. Identify the `styled.button` block as a CSS region. 2. Extract the CSS code inside the template literal. 3. Apply any CSS-specific pattern searches within this extracted region. You can search the CSS inside JavaScript in the project configuration folder using this command: sh ast-grep -p 'background: $COLOR' -C 2 It will produce the match result: shell styled.js 2│ 3│const Button = styled.button` 4│ background: red; 5│ color: white; 6│ padding: 10px 20px; Using Custom Language with Injection [​](https://ast-grep.github.io/advanced/language-injection.html#using-custom-language-with-injection) ------------------------------------------------------------------------------------------------------------------------------------------- Finally, let's look at an example of searching for GraphQL within JavaScript files. This demonstrates ast-grep's flexibility in handling custom language injections. ### Define graphql custom language in `sgconfig.yml`. [​](https://ast-grep.github.io/advanced/language-injection.html#define-graphql-custom-language-in-sgconfig-yml) First, we need to register graphql as a custom language in ast-grep. See [custom language reference](https://ast-grep.github.io/advanced/custom-language.html) for more details. yaml customLanguages: graphql: libraryPath: graphql.so # the graphql tree-sitter parser dynamic library extensions: [graphql] # graphql file extension expandoChar: $ # see reference above for explanation ### Define graphql injection in `sgconfig.yml`. [​](https://ast-grep.github.io/advanced/language-injection.html#define-graphql-injection-in-sgconfig-yml) Next, we need to customize what region should be parsed as graphql string in JavaScript. This is similar to styled-components example above. yaml languageInjections: - hostLanguage: js rule: pattern: graphql`$CONTENT` injected: graphql ### Search GraphQL in JavaScript [​](https://ast-grep.github.io/advanced/language-injection.html#search-graphql-in-javascript) Suppose we have this JavaScript file from [Relay](https://relay.dev/) , a GraphQL client framework. js import React from "react" import { graphql } from "react-relay" const artistsQuery = graphql` query ArtistQuery($artistID: String!) { artist(id: $artistID) { name ...ArtistDescription_artist } } ` We can search the GraphQL fragment via this `--inline-rules` scan. sh ast-grep scan --inline-rules="{id: test, language: graphql, rule: {kind: fragment_spread}}" Output sh help[test]: ┌─ relay.js:8:7 │ 8 │ ...ArtistDescription_artist │ ^^^^^^^^^^^^^^^^^^^^^^^^^^^ More Possibility to be Unlocked... [​](https://ast-grep.github.io/advanced/language-injection.html#more-possibility-to-be-unlocked) ------------------------------------------------------------------------------------------------------------------------------------ By following these steps, you can effectively use ast-grep to search and analyze code across multiple languages within the same document, enhancing your ability to manage and understand complex codebases. This feature extends to various frameworks like [Vue](https://vuejs.org/) and [Svelte](https://svelte.dev/) , enables searching for [SQL in React server actions](https://x.com/peer_rich/status/1717609270475194466) , and supports new patterns like [Vue-Vine](https://x.com/hd_nvim/status/1815300932793663658) . Hope you enjoy the feature! Happy ast-grepping! --- # Deep Dive into ast-grep's Pattern Syntax | ast-grep [Skip to content](https://ast-grep.github.io/advanced/pattern-parse.html#VPContent) On this page Are you an LLM? You can read better optimized documentation at /advanced/pattern-parse.md for this page in Markdown format Deep Dive into ast-grep's Pattern Syntax [​](https://ast-grep.github.io/advanced/pattern-parse.html#deep-dive-into-ast-grep-s-pattern-syntax) ============================================================================================================================================== ast-grep's pattern is easy to learn but hard to master. While it's easy to get started with, mastering its nuances can greatly enhance your code searching capabilities. This article aims to provide you with a deep understanding of how ast-grep's patterns are parsed, created, and effectively used in code matching. Steps to Create a Pattern [​](https://ast-grep.github.io/advanced/pattern-parse.html#steps-to-create-a-pattern) ---------------------------------------------------------------------------------------------------------------- Parsing a pattern in ast-grep involves these keys steps: 1. Preprocess the pattern text, e.g, replacing `$` with [expando\_char](https://ast-grep.github.io/advanced/custom-language.html#register-language-in-sgconfig-yml) . 2. Parse the preprocessed pattern text into AST. 3. Extract effective AST nodes based on builtin heuristics or user provided [selector](https://ast-grep.github.io/reference/rule.html#pattern) . 4. Detect AST with wildcard text and convert them into [meta variables](https://ast-grep.github.io/guide/pattern-syntax.html#meta-variable) . ![image](https://ast-grep.github.io/image/parse-pattern.jpg) Let's dive deep into each of these steps. Pattern is AST based [​](https://ast-grep.github.io/advanced/pattern-parse.html#pattern-is-ast-based) ------------------------------------------------------------------------------------------------------ _**First and foremost, pattern is AST based**._ ast-grep's pattern code will be converted into the Abstract Syntax Tree (AST) format, which is a tree structure that represents the code snippet you want to match. Therefore pattern cannot be arbitrary text, but a valid code with meta variables as placeholders. If the pattern cannot be parsed by the underlying parser tree-sitter, ast-grep won't be able to find valid matching for it. There are several common pitfalls to avoid when creating patterns. ### Invalid Pattern Code [​](https://ast-grep.github.io/advanced/pattern-parse.html#invalid-pattern-code) ast-grep pattern must be parsable valid code. While this may seem obvious, newcomers sometimes make mistakes when creating patterns with meta-variables. _**Meta-variable is usually parsed as identifier in most languages.**_ When using meta-variables, make sure they are placed in a valid context and not used as a keyword or an operator. For example, you may want to use `$OP` to match binary expressions like `a + b`. The pattern below will not work because parsers see it as three consecutive identifiers separated by spaces. $LEFT $OP $RIGHT You can instead use [atomic rule](https://ast-grep.github.io/guide/rule-config/atomic-rule.html#kind) `kind: binary_expression` to [match binary expressions](https://ast-grep.github.io/playground.html#eyJtb2RlIjoiQ29uZmlnIiwibGFuZyI6ImphdmFzY3JpcHQiLCJxdWVyeSI6IiIsInJld3JpdGUiOiIiLCJzdHJpY3RuZXNzIjoic21hcnQiLCJzZWxlY3RvciI6IiIsImNvbmZpZyI6InJ1bGU6XG4gIGtpbmQ6IGJpbmFyeV9leHByZXNzaW9uIiwic291cmNlIjoiYSArIGIgXHJcbmEgLSBiXHJcbmEgPT0gYiAifQ==) . Similarly, in JavaScript you may want to match [object accessors](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Operators/Object_initializer#method_definitions) like `{ get foo() {}, set bar() { } }`. The pattern below will not work because meta-variable is not parsed as the keywords `get` and `set`. js obj = { $KIND foo() { } } Again [rule](https://ast-grep.github.io/guide/rule-config.html) is more suitable for [this scenario](https://ast-grep.github.io/playground.html#eyJtb2RlIjoiQ29uZmlnIiwibGFuZyI6ImphdmFzY3JpcHQiLCJxdWVyeSI6IiIsInJld3JpdGUiOiIiLCJzdHJpY3RuZXNzIjoic21hcnQiLCJzZWxlY3RvciI6IiIsImNvbmZpZyI6InJ1bGU6XG4gIGtpbmQ6IG1ldGhvZF9kZWZpbml0aW9uXG4gIHJlZ2V4OiAnXmdldHxzZXRcXHMnIiwic291cmNlIjoidmFyIGEgPSB7XHJcbiAgICBmb28oKSB7fVxyXG4gICAgZ2V0IGZvbygpIHt9LFxyXG4gICAgc2V0IGJhcigpIHt9LFxyXG59In0=) . yaml rule: kind: method_definition regex: '^get|set\s' ### Incomplete Pattern Code [​](https://ast-grep.github.io/advanced/pattern-parse.html#incomplete-pattern-code) It is very common and even attempting to write incomplete code snippet in patterns. However, incomplete code does not _always_ work. Consider the following JSON code snippet as pattern: json "a": 123 While the intention here is clearly to match a key-value pair, tree-sitter does not treat it as valid JSON code because it is missing the enclosing `{}`. Consequently ast-grep will not be able to parse it. The solution here is to use [pattern object](https://ast-grep.github.io/guide/rule-config/atomic-rule.html#pattern-object) to provide complete code snippet. yaml pattern: context: '{ "a": 123 }' selector: pair You can use both ast-grep playground's [pattern tab](https://ast-grep.github.io/playground.html#eyJtb2RlIjoiUGF0Y2giLCJsYW5nIjoianNvbiIsInF1ZXJ5IjoieyBcImFcIjogMTIzIH0iLCJyZXdyaXRlIjoiIiwic3RyaWN0bmVzcyI6InNtYXJ0Iiwic2VsZWN0b3IiOiJwYWlyIiwiY29uZmlnIjoicnVsZTpcbiAga2luZDogbWV0aG9kX2RlZmluaXRpb25cbiAgcmVnZXg6ICdeZ2V0fHNldFxccyciLCJzb3VyY2UiOiJ7IFwiYVwiOiAxMjMgfSAifQ==) or [rule tab](https://ast-grep.github.io/playground.html#eyJtb2RlIjoiQ29uZmlnIiwibGFuZyI6Impzb24iLCJxdWVyeSI6InsgXCJhXCI6IDEyMyB9IiwicmV3cml0ZSI6IiIsInN0cmljdG5lc3MiOiJzbWFydCIsInNlbGVjdG9yIjoicGFpciIsImNvbmZpZyI6InJ1bGU6XG4gIHBhdHRlcm46IFxuICAgIGNvbnRleHQ6ICd7XCJhXCI6IDEyM30nXG4gICAgc2VsZWN0b3I6IHBhaXIiLCJzb3VyY2UiOiJ7IFwiYVwiOiAxMjMgfSAifQ==) to verify it. _**Incomplete pattern code sometimes works fine due to error-tolerance.**_ For better _user experience_, ast-grep parse pattern code as lenient as possible. ast-grep parsers will try recovering parsing errors and ignoring missing language constructs. For example, the pattern `foo(bar)` in Java cannot be parsed as valid code. However, ast-grep recover the parsing error, ignoring missing semicolon and treat it as a method call. So the pattern [still works](https://ast-grep.github.io/playground.html#eyJtb2RlIjoiUGF0Y2giLCJsYW5nIjoiamF2YSIsInF1ZXJ5IjoiZm9vKGJhcikiLCJyZXdyaXRlIjoiIiwic3RyaWN0bmVzcyI6InNtYXJ0Iiwic2VsZWN0b3IiOiIiLCJjb25maWciOiJydWxlOlxuICBwYXR0ZXJuOiBcbiAgICBjb250ZXh0OiAne1wiYVwiOiAxMjN9J1xuICAgIHNlbGVjdG9yOiBwYWlyIiwic291cmNlIjoiY2xhc3MgQSB7XG4gICAgZm9vKCkge1xuICAgICAgICBmb28oYmFyKTtcbiAgICB9XG59In0=) . ### Ambiguous Pattern Code [​](https://ast-grep.github.io/advanced/pattern-parse.html#ambiguous-pattern-code) Just as programming languages have ambiguous grammar, so ast-grep patterns can be ambiguous. Let's consider the JavaScript code snippet below: js a: 123 It can be interpreted as an object key-value pair or a labeled statement. Without other hints, ast-grep will parse it as labeled statement by default. To match object key-value pair, we need to provide more context by [using pattern object](https://ast-grep.github.io/playground.html#eyJtb2RlIjoiUGF0Y2giLCJsYW5nIjoiamF2YXNjcmlwdCIsInF1ZXJ5IjoieyBhOiAxMjMgfSIsInJld3JpdGUiOiIiLCJzdHJpY3RuZXNzIjoic21hcnQiLCJzZWxlY3RvciI6InBhaXIiLCJjb25maWciOiJydWxlOlxuICBwYXR0ZXJuOiBcbiAgICBjb250ZXh0OiAne1wiYVwiOiAxMjN9J1xuICAgIHNlbGVjdG9yOiBwYWlyIiwic291cmNlIjoiYSA9IHsgYTogIDEyMyB9In0=) . yaml pattern: context: '{ a: 123 }' selector: pair Other examples of ambiguous patterns include: * Match function call in [Golang](https://ast-grep.github.io/catalog/go/#match-function-call-in-golang) and [C](https://ast-grep.github.io/catalog/c/#match-function-call) * Match [class field](https://ast-grep.github.io/guide/rule-config/atomic-rule.html#pattern-object) in JavaScript ### How ast-grep Handles Pattern Code? [​](https://ast-grep.github.io/advanced/pattern-parse.html#how-ast-grep-handles-pattern-code) ast-grep uses best efforts to parse pattern code for best user experience. Here are some strategies ast-grep uses to handle code snippet: * **Replace `$` with expando\_char**: some languages use `$` as a special character, so ast-grep replace it with [expando\_char](https://ast-grep.github.io/advanced/custom-language.html#register-language-in-sgconfig-yml) in order to make the pattern code parsable. * **Ignore missing nodes**: ast-grep will ignore missing nodes in pattern like trailing semicolon in Java/C/C++. * **Treat root error as normal node**: if the parser error has no siblings, ast-grep will treat it as a normal node. * If all above fails, users should provide more code via pattern object Pattern Error Recovery is useful, but not guaranteed ast-grep's recovery mechanism heavily depends on tree-sitter's behavior. We cannot guarantee invalid patterns will be parsed consistently between different versions. So using invalid pattern may lead to unexpected results after upgrading ast-grep. When in doubt, always use valid code snippets with pattern object. Extract Effective AST for Pattern [​](https://ast-grep.github.io/advanced/pattern-parse.html#extract-effective-ast-for-pattern) -------------------------------------------------------------------------------------------------------------------------------- After parsing the pattern code, ast-grep needs to extract AST nodes to make the actual pattern. Normally, a code snippet generated by tree-sitter will be a full AST tree. Yet it is unlikely that the entire tree will be used as a pattern. The code `123` will produce a tree like `program -> expression_statement -> number` in many languages. But we want to match a number literal in the code, not a program containing just a number. ast-grep uses two strategies to extract **effective AST nodes** that will be used to match code. ### Builtin Heuristic [​](https://ast-grep.github.io/advanced/pattern-parse.html#builtin-heuristic) _**By default, at-grep extracts the leaf node or the innermost node with more than one child.**_ This heuristic extracts the most specific node while still keeping all structural information in the pattern. If a node has only one child, it is atomic and cannot be further decomposed. We can safely assume the node contains no structural information for matching. In contrast, a node with more than one child contains a structure that we want to search. Examples: * `123` will be extracted as `number` because it is the leaf node. yaml program expression_statement number <--- effective node See [Playground](https://ast-grep.github.io/playground.html#eyJtb2RlIjoiUGF0Y2giLCJsYW5nIjoiamF2YXNjcmlwdCIsInF1ZXJ5IjoiMTIzIiwicmV3cml0ZSI6IiIsInN0cmljdG5lc3MiOiJzbWFydCIsInNlbGVjdG9yIjoiIiwiY29uZmlnIjoiIiwic291cmNlIjoiIn0=) . * `foo(bar)` will be extracted as `call_expression` because it is the innermost node that has more than one child. yaml program expression_statement call_expression <--- effective node identifier arguments identifier See [Playground](https://ast-grep.github.io/playground.html#eyJtb2RlIjoiUGF0Y2giLCJsYW5nIjoiamF2YXNjcmlwdCIsInF1ZXJ5IjoiZm9vKGJhcikiLCJyZXdyaXRlIjoiIiwic3RyaWN0bmVzcyI6InNtYXJ0Iiwic2VsZWN0b3IiOiJjYWxsX2V4cHJlc3Npb24iLCJjb25maWciOiIiLCJzb3VyY2UiOiIifQ==) . ### User Defined Selector [​](https://ast-grep.github.io/advanced/pattern-parse.html#user-defined-selector) Sometimes the effective node extracted by the builtin heuristic may not be what you want. You can explicitly specify the node to extract using the [selector](https://ast-grep.github.io/reference/rule.html#pattern) field in the rule configuration. For example, you may want to match the whole `console.log` statement in JavaScript code. The effective node extracted by the builtin heuristic is `call_expression`, but you want to match the whole `expression_statement`. Using `console.log($$$)` directly will not include the trailing `;` in the pattern, see [Playground](https://ast-grep.github.io/playground.html#eyJtb2RlIjoiUGF0Y2giLCJsYW5nIjoiamF2YXNjcmlwdCIsInF1ZXJ5IjoiY29uc29sZS5sb2coJCQkKSIsInJld3JpdGUiOiIiLCJzdHJpY3RuZXNzIjoic2lnbmF0dXJlIiwic2VsZWN0b3IiOiJjYWxsX2V4cHJlc3Npb24iLCJjb25maWciOiIiLCJzb3VyY2UiOiJjb25zb2xlLmxvZyhmb28pXG5jb25zb2xlLmxvZyhiYXIpOyJ9) . js console.log("Hello") console.log("World"); You can use pattern object to explicitly specify the effective node to be `expression_statement`. [Playground](https://ast-grep.github.io/playground.html#eyJtb2RlIjoiQ29uZmlnIiwibGFuZyI6ImphdmFzY3JpcHQiLCJxdWVyeSI6ImNvbnNvbGUubG9nKCQkJCkiLCJyZXdyaXRlIjoiIiwic3RyaWN0bmVzcyI6InNpZ25hdHVyZSIsInNlbGVjdG9yIjoiY2FsbF9leHByZXNzaW9uIiwiY29uZmlnIjoicnVsZTpcbiAgcGF0dGVybjpcbiAgICBjb250ZXh0OiBjb25zb2xlLmxvZygkJCQpXG4gICAgc2VsZWN0b3I6IGV4cHJlc3Npb25fc3RhdGVtZW50XG5maXg6ICcnIiwic291cmNlIjoiY29uc29sZS5sb2coZm9vKVxuY29uc29sZS5sb2coYmFyKTsifQ==) yaml pattern: context: console.log($$$) selector: expression_statement Using `selector` is especially helpful when you are also using relational rules like `follows` and `precedes`. You want to match the statement instead of the default inner expression node, and [match other statements around it](https://github.com/ast-grep/ast-grep/issues/1427) . TIP When in doubt, try pattern object first. Meta Variable Deep Dive [​](https://ast-grep.github.io/advanced/pattern-parse.html#meta-variable-deep-dive) ------------------------------------------------------------------------------------------------------------ ast-grep's meta variables are also AST based and are detected in the effective nodes extracted from the pattern code. ### Meta Variable Detection in Pattern [​](https://ast-grep.github.io/advanced/pattern-parse.html#meta-variable-detection-in-pattern) Not all `$` prefixed strings will be detected as meta variables. Only AST nodes that match meta variable syntax will be detected. If meta variable text is not the only text in the node or it spans multiple nodes, it will not be detected as a meta variable. **Working meta variable examples:** * `$A` works * `$A` is one single `identifier` * `$A.$B` works * `$A` is `identifier` inside `member_expression` * `$B` is the `property_identifier`. * `$A.method($B)` works * `$A` is `identifier` inside `member_expression` * `$B` is `identifier` inside `arguments` **Non working meta variable examples:** * `obj.on$EVENT` does not work * `on$EVENT` is `property_identifier` but `$EVENT` is not the only text * `"Hello $WORLD"` does not work * `$WORLD` is inside `string_content` and is not the only text * `a $OP b` does not work * the whole pattern does not parse * `$jq` does not work * meta variable does not accept lower case letters See all examples in [Playground](https://ast-grep.github.io/playground.html#eyJtb2RlIjoiUGF0Y2giLCJsYW5nIjoiamF2YXNjcmlwdCIsInF1ZXJ5IjoiIiwicmV3cml0ZSI6IiIsInN0cmljdG5lc3MiOiJzaWduYXR1cmUiLCJzZWxlY3RvciI6ImNhbGxfZXhwcmVzc2lvbiIsImNvbmZpZyI6IiIsInNvdXJjZSI6Ii8vIHdvcmtpbmdcbiRBXG4kQS4kQlxuJEEubWV0aG9kKCRCKVxuXG4vLyBub24gd29ya2luZ1xub2JqLm9uJEVWRU5UXG5cIkhlbGxvICRXT1JMRFwiXG5hICRPUCBiIn0=) . ### Matching Unnamed Nodes [​](https://ast-grep.github.io/advanced/pattern-parse.html#matching-unnamed-nodes) A meta variable pattern `$META` will capture [named nodes](https://ast-grep.github.io/advanced/core-concepts.html#named-vs-unnamed) by default. To capture [unnamed nodes](https://ast-grep.github.io/advanced/core-concepts.html#named-vs-unnamed) , you can use double dollar sign `$$VAR`. Let's go back to the binary expression example. It is impossible to match arbitrary binary expression in one single pattern. But we can combine `kind` and `has` to match the operator in binary expressions. Note, `$OP` cannot match the operator because operator is not a named node. We need to use `$$OP` instead. yaml rule: kind: binary_expression has: field: operator pattern: $$OP # pattern: $OP See the above rule to match all arithmetic expressions in [action](https://ast-grep.github.io/playground.html#eyJtb2RlIjoiQ29uZmlnIiwibGFuZyI6ImphdmFzY3JpcHQiLCJxdWVyeSI6ImNvbnNvbGUubG9nKCQkJCkiLCJyZXdyaXRlIjoiIiwic3RyaWN0bmVzcyI6InNpZ25hdHVyZSIsInNlbGVjdG9yIjoiY2FsbF9leHByZXNzaW9uIiwiY29uZmlnIjoicnVsZTpcbiAga2luZDogYmluYXJ5X2V4cHJlc3Npb25cbiAgaGFzOlxuICAgIGZpZWxkOiBvcGVyYXRvclxuICAgIHBhdHRlcm46ICQkT1BcbiAgICAjIHBhdHRlcm46ICRPUCIsInNvdXJjZSI6IjEgKyAxIn0=) . ### How Multi Meta Variables Match Code [​](https://ast-grep.github.io/advanced/pattern-parse.html#how-multi-meta-variables-match-code) Multiple meta variables like `$$$ARGS` has special matching behavior. It will match multiple nodes in the AST. `$$$ARGS` will match multiple nodes in source code when the meta variable starts to match. It will match as many nodes as possible until the first AST node after the meta var in pattern is matched. The behavior is like [non-greedy](https://stackoverflow.com/questions/11898998/how-can-i-write-a-regex-which-matches-non-greedy) matching in regex and template string literal `infer` in [TypeScript](https://github.com/microsoft/TypeScript/pull/40336) . Use ast-grep playground to debug pattern [​](https://ast-grep.github.io/advanced/pattern-parse.html#use-ast-grep-playground-to-debug-pattern) ---------------------------------------------------------------------------------------------------------------------------------------------- ast-grep playground is a great tool to debug pattern code. The pattern tab and pattern panel can help you visualize the AST tree, effective nodes and meta variables. ![playground](https://ast-grep.github.io/image/pattern-debugger.jpg) In next article, we will explain how ast-grep's pattern is used to match code, the pattern matching algorithm. --- # Using ast-grep with AI Tools | ast-grep [Skip to content](https://ast-grep.github.io/advanced/prompting.html#VPContent) On this page Are you an LLM? You can read better optimized documentation at /advanced/prompting.md for this page in Markdown format Using ast-grep with AI Tools [​](https://ast-grep.github.io/advanced/prompting.html#using-ast-grep-with-ai-tools) ================================================================================================================== This guide outlines several existing methods for leveraging AI with ast-grep. Disclaimer The field of AI is constantly evolving. The approaches detailed here are for reference, and we encourage you to explore and discover the best ways to utilize ast-grep with emerging AI technologies. Using ast-grep with Claude Code Skill [​](https://ast-grep.github.io/advanced/prompting.html#using-ast-grep-with-claude-code-skill) ------------------------------------------------------------------------------------------------------------------------------------ This skill teaches Claude how to write and use ast-grep rules to perform advanced code searches. Unlike traditional text-based search (grep, ripgrep), ast-grep understands the structure of your code, allowing you to find patterns like: * "Find all async functions that don't have error handling" * "Locate all React components that use a specific hook" * "Find functions with more than 3 parameters" * "Search for console.log calls inside class methods" Clone or download the [ast-grep skill repository](https://github.com/ast-grep/claude-skill) to your Claude Code skills directory: bash # If you have a skills directory configured cp -r ast-grep ~/.claude/skills/ # Or place it wherever your Claude Code skills are located The skill should be automatically detected by Claude Code. You can verify by checking available skills in Claude Code. Simple Prompting in `AGENTS.md` [​](https://ast-grep.github.io/advanced/prompting.html#simple-prompting-in-agents-md) ---------------------------------------------------------------------------------------------------------------------- For everyday development, you can instruct your AI agent to use ast-grep for code searching and analysis. This method is straightforward but requires a model with up-to-date knowledge of ast-grep to be effective. If the model is not familiar with the tool, it may not utilize it as instructed. You can set a system-level prompt for your AI agent to prioritize ast-grep for syntax-aware searches. Here is an example prompt comes from [this social post](https://x.com/kieranklaassen/status/1938453871086682232) . **Example Prompt:** > You are operating in an environment where `ast-grep` is installed. For any code search that requires understanding of syntax or code structure, you should default to using `ast-grep --lang [language] -p ''`. Adjust the `--lang` flag as needed for the specific programming language. Avoid using text-only search tools unless a plain-text search is explicitly requested. This approach is best suited for general code queries and explorations within your projects. Providing Comprehensive Context to LLMs [​](https://ast-grep.github.io/advanced/prompting.html#providing-comprehensive-context-to-llms) ---------------------------------------------------------------------------------------------------------------------------------------- Large Language Models (LLMs) with extensive context windows can be made highly effective at using ast-grep by providing them with its complete documentation. The `llms.txt` file for ast-grep is a compilation of the entire documentation, designed to be fed into an LLM's context. This method significantly reduces the likelihood of the model "hallucinating" or generating incorrect ast-grep rules by giving it a thorough and accurate knowledge base to draw from. You can find the full `llms.txt` file here: [https://ast-grep.github.io/llms-full.txt](https://ast-grep.github.io/llms-full.txt) By loading this text into your session with a capable LLM, you can ask more complex questions and receive more accurate and nuanced answers regarding ast-grep's features and usage. Advanced Rule Development with MCP and Sub-agents [​](https://ast-grep.github.io/advanced/prompting.html#advanced-rule-development-with-mcp-and-sub-agents) ------------------------------------------------------------------------------------------------------------------------------------------------------------ For more sophisticated and dedicated code analysis tasks, you can use the ast-grep Model Context Protocol (MCP) server. The [ast-grep-mcp](https://github.com/ast-grep/ast-grep-mcp) is an experimental server that connects AI assistants, such as Cursor and Claude Code, with the powerful structural search capabilities of ast-grep. This allows the AI to interact with your codebase in a more structured and intelligent way, moving beyond simple text-based searches. The MCP server provides a set of tools that enable an AI to develop and refine ast-grep rules through a process of trial and error. This is particularly useful for creating complex rules that may require several iterations to perfect. The core of this approach is to have the AI follow a systematic process for rule development: ## Rule Development Process 1. Break down the user's query into smaller parts. 2. Identify sub rules that can be used to match the code. 3. Combine the sub rules into a single rule using relational rules or composite rules. 4. if rule does not match example code, revise the rule by removing some sub rules and debugging unmatching parts. 5. Use ast-grep mcp tool to dump AST or dump pattern query 6. Use ast-grep mcp tool to test the rule against the example code snippet. This iterative process allows the AI to "think" more like a human developer, refining its approach until the rule is correct. You can view a detailed prompt for this agentic rule development process in the `ast-grep-mcp` repository: [https://github.com/ast-grep/ast-grep-mcp/blob/main/ast-grep.mdc](https://github.com/ast-grep/ast-grep-mcp/blob/main/ast-grep.mdc) . --- # Comparison With Other Frameworks | ast-grep [Skip to content](https://ast-grep.github.io/advanced/tool-comparison.html#VPContent) On this page Are you an LLM? You can read better optimized documentation at /advanced/tool-comparison.md for this page in Markdown format Comparison With Other Frameworks [​](https://ast-grep.github.io/advanced/tool-comparison.html#comparison-with-other-frameworks) ================================================================================================================================ Disclaimer This comparison is based on the author's personal experience and opinion, which may not be accurate or comprehensive. The author respects and appreciates all the other tools and their developers, and does not intend to criticize or endorse any of them. The author is grateful to these predecessor tools for inspiring ast-grep! The reader is encouraged to try out the tools themselves and form their own judgment. ast-grep [​](https://ast-grep.github.io/advanced/tool-comparison.html#ast-grep) -------------------------------------------------------------------------------- **Pros**: * It is very performant. It uses [ignore](https://docs.rs/ignore/latest/ignore/) to do multi-thread processing, which makes it utilize all your CPU cores. * It is language aware. It uses tree-sitter, a real parser, to parse the code into ASTs, which enables more precise and accurate matching and fixing. * It has a powerful and flexible rule system. It allows you to write patterns, AST types and regular expressions to match code. It provides operators to compose complex matching rules for various scenarios. * It can be used as a lightweight CLI tool or as a library, depending on your usage. It has a simple and user-friendly interface, and it also exposes its core functionality as a library for other applications. **Cons**: * It is still young and under development. It may have some bugs or limitations that need to be fixed or improved. * It does not have deep semantic information or comparison equivalence. It only operates on the syntactic level of the code, which may miss some matches or may be too cumbersome to match certain code. * More specifically, ast-grep at the moment does not support the following information: * [type information](https://semgrep.dev/docs/writing-rules/pattern-syntax#typed-metavariables) * [control flow analysis](https://en.wikipedia.org/wiki/Control-flow_analysis) * [data flow analysis](https://en.wikipedia.org/wiki/Data-flow_analysis) * [taint analysis](https://semgrep.dev/docs/writing-rules/data-flow/taint-mode) * [constant propagation](https://semgrep.dev/docs/writing-rules/data-flow/constant-propagation) [Semgrep](https://semgrep.dev/) [​](https://ast-grep.github.io/advanced/tool-comparison.html#semgrep) ------------------------------------------------------------------------------------------------------- Semgrep is a well-established tool that uses code patterns to find and fix bugs and security issues in code. **Pros**: * It supports advanced features like equivalence and deep-semgrep, which allow for more precise and expressive matching and fixing. * It has a large collection of rules for various languages and frameworks, which cover common vulnerabilities and best practices. **Cons**: * It is mainly focused on security issues, which may limit its applicability for other use cases. * It is relatively slow when used as command line tools. * It cannot be used as a library in other applications, which may reduce its integration and customization options. [GritQL](https://about.grit.io/) [​](https://ast-grep.github.io/advanced/tool-comparison.html#gritql) ------------------------------------------------------------------------------------------------------- [GritQL](https://docs.grit.io/language/overview) language is [Grit](https://docs.grit.io/) 's embedded query language for searching and transforming source code. **Pros**: * GritQL is generally more powerful. It has features like [clause](https://docs.grit.io/language/modifiers) from [logic programming language](https://en.wikipedia.org/wiki/Logic_programming#:~:text=A%20logic%20program%20is%20a,Programming%20(ASP)%20and%20Datalog.) and [operations](https://docs.grit.io/language/conditions#match-condition) from imperative programming languages. * It is used as [linter plugins](https://biomejs.dev/linter/plugins/) in [Biome](https://biomejs.dev/) , a toolchain for JS ecosystem. **Cons**: * Depending on different background, developers may find it harder to learn a multi-paradigm DSL. [Comby](https://comby.dev/) [​](https://ast-grep.github.io/advanced/tool-comparison.html#comby) ------------------------------------------------------------------------------------------------- Comby is a fast and flexible tool that uses structural patterns to match and rewrite code across languages and file formats. **Pros**: * It does not rely on language-specific parsers, which makes it more generic and robust. It can handle any language and file format, including non-code files like JSON or Markdown. * It has a custom syntax for specifying patterns and replacements, which can handle various syntactic variations and transformations. **Cons**: * It is not aware of the syntax and semantics of the target language, which limits its expressiveness and accuracy. It may miss some matches or generate invalid code due to syntactic or semantic differences. * It does not support indentation-sensitive languages like Python or Haskell, which require special handling for whitespace and indentation. * It is hard to write complex queries with Comby, such as finding a function that does not call another function. It does not support logical operators or filters for patterns. [IntelliJ Structural Search Replace](https://www.jetbrains.com/help/idea/structural-search-and-replace.html) [​](https://ast-grep.github.io/advanced/tool-comparison.html#intellij-structural-search-replace) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- IntelliJ Structural Search Replace is not a standalone tool, but a feature of the IntelliJ IDE that allows users to search and replace code using structural patterns. **Pros**: * It is integrated with the IntelliJ IDE, which makes it easy to use and customize. **Cons**: * Currently, IntelliJ IDEA supports the structural search and replace for Java, Kotlin and Groovy. [Shisho](https://github.com/flatt-security/shisho) [​](https://ast-grep.github.io/advanced/tool-comparison.html#shisho) ------------------------------------------------------------------------------------------------------------------------- Shisho is a new and promising tool that uses code patterns to search and manipulate code in various languages. **Pros**: * It offers fast and flexible rule composition using code patterns. * It can handle multiple languages and files in parallel, and it has a simple and intuitive syntax for specifying patterns and filters. **Cons**: * It is still in development and it has limited language support compared to the other tools. It currently supports only 3 languages, while the other tools support over 20 languages. * The tool's parent company seems to have changed their business direction. --- # ast-grep Blog | ast-grep [Skip to content](https://ast-grep.github.io/blog.html#VPContent) Return to top ast-grep Blog [​](https://ast-grep.github.io/blog.html#ast-grep-blog) ====================================================================== * March 15, 2026 [ast-grep 0.42 - The Answer to Code Searching](https://ast-grep.github.io/blog/new-ver-42.html) ------------------------------------------------------------------------------------------------ _ast-grep 0.42: parameterized utilities, ESQuery-style selectors, and improved LSP diagnostics for injected languages._ * January 2, 2026 [Announcing the Book: Mastering ast-grep](https://ast-grep.github.io/blog/mastering-ast-grep.html) --------------------------------------------------------------------------------------------------- _Mastering ast-grep, a comprehensive guide designed to bridge the gap between basic tool reference and advanced structural search mastery._ * November 26, 2025 [How to Debug ast-grep Rule Effectively](https://ast-grep.github.io/blog/how-to-debug.html) -------------------------------------------------------------------------------------------- _Learn how to debug ast-grep rules effectively by simplifying code and rules step by step._ * August 3, 2025 [ast-grep new release 0.39](https://ast-grep.github.io/blog/new-ver-39.html) ----------------------------------------------------------------------------- _ast-grep 0.39 includes new languages support, better file config and Esquery style kind._ * July 7, 2025 [YAML vs DSL: comparison is subjective](https://ast-grep.github.io/blog/yaml-vs-dsl.html) ------------------------------------------------------------------------------------------ _YAML and DSL are two different approaches to configure rule in structural search. The question "which is better" is largely subjective._ * June 21, 2025 [ast-grep's Journey to AI Generated Rules](https://ast-grep.github.io/blog/ast-grep-agent.html) ------------------------------------------------------------------------------------------------ _Advancements in AI have made it possible to generate ast-grep rules with a well-written prompt._ * June 7, 2025 [Interactive Code Fixes with Multiple Options!](https://ast-grep.github.io/blog/interactive-demo.html) ------------------------------------------------------------------------------------------------------- _Today, we're thrilled to showcase a game-changing feature, multi-option interactive code fixes!_ * May 18, 2025 [ast-grep new release 0.38](https://ast-grep.github.io/blog/new-ver-38.html) ----------------------------------------------------------------------------- _ast-grep 0.38 brings some fantastic new features to improve your code searching and linting experience, alongside a significant internal shift._ * March 23, 2025 [ast-grep gets more LLM support!](https://ast-grep.github.io/blog/more-llm-support.html) ----------------------------------------------------------------------------------------- _ast-grep is getting even better with enhanced Large Language Model (LLM) support. This exciting development opens up new possibilities for developers to analyze, understand, and transform code more efficiently. Let's dive into the details of these new features._ * March 4, 2025 [ast-grep Rockets to 8000 Stars!](https://ast-grep.github.io/blog/stars-8000.html) ----------------------------------------------------------------------------------- _ast-grep has recently reached 6000 stars on GitHub! This is a remarkable achievement for the project and I am deeply grateful for all the support and feedback that I have received from the open source community._ * January 1, 2025 [An Example of Rust's Fearless Concurrency](https://ast-grep.github.io/blog/fearless-concurrency.html) ------------------------------------------------------------------------------------------------------- _ast-grep shows how Rust's fearless concurrency works in practice. Learn how to design concurrent systems in Rust and the trade-offs involved._ * December 25, 2024 [Design Space for Code Search Query](https://ast-grep.github.io/blog/code-search-design-space.html) ---------------------------------------------------------------------------------------------------- _A review of the design space for code search tools._ * December 22, 2024 [ast-grep's Journey to Type Safety in Node API](https://ast-grep.github.io/blog/typed-napi.html) ------------------------------------------------------------------------------------------------- _ast-grep/napi now supports typed AST manipulation which is correct, concise, robust, and performant._ * May 19, 2024 [ast-grep got 6000 stars!](https://ast-grep.github.io/blog/stars-6000.html) ---------------------------------------------------------------------------- _ast-grep has recently reached 6000 stars on GitHub! This is a remarkable achievement for the project and I am deeply grateful for all the support and feedback that I have received from the open source community._ * January 20, 2024 [ast-grep: 5000 stars and beyond!](https://ast-grep.github.io/blog/stars-5000.html) ------------------------------------------------------------------------------------ _ast-grep has recently reached 5000 stars on GitHub! This is a remarkable achievement for the project and I am deeply grateful for all the support and feedback that I have received from the open source community._ * November 2, 2023 [ast-grep got 3000 stars!](https://ast-grep.github.io/blog/stars-3000.html) ---------------------------------------------------------------------------- _ast-grep has recently reached 3000 stars on GitHub! This is a remarkable achievement for the project and I am deeply grateful for all the support and feedback that I have received from the open source community._ * May 17, 2023 [Migrating Bevy can be easier with (semi-)automation](https://ast-grep.github.io/blog/migrate-bevy.html) --------------------------------------------------------------------------------------------------------- _In this article, we will show you how to make migration easier by using some command line tools._ * January 23, 2023 [Optimize ast-grep to get 10X faster](https://ast-grep.github.io/blog/optimize-ast-grep.html) ---------------------------------------------------------------------------------------------- _How to optimize the Rust CLI tool ast-grep to become 10 times faster._ --- # Design Space for Code Search Query | ast-grep [Skip to content](https://ast-grep.github.io/blog/code-search-design-space.html#VPContent) On this page Design Space for Code Search Query [​](https://ast-grep.github.io/blog/code-search-design-space.html#design-space-for-code-search-query) ========================================================================================================================================= Code search is a critical tool for modern software development. It enables developers to quickly locate, understand, and reuse existing code, boosting productivity and ensuring code consistency across projects. At its core, ast-grep is a [code search](https://ast-grep.github.io/guide/introduction.html#motivation) tool. Its other features, such as [linting](https://ast-grep.github.io/guide/scan-project.html) and code [rewriting](https://ast-grep.github.io/guide/rewrite-code.html) , are built upon the foundation of its code search capabilities. This blog post delves into the design space of code search, with a particular focus on how queries are designed and used. We'll be drawing inspiration from the excellent paper, "[Code Search: A Survey of Techniques for Finding Code](https://www.lucadigrazia.com/papers/acmcsur2022.pdf) ". But we won't be covering every single detail from that paper. Instead, our focus will be on the diverse ways that code search tools allow users to express their search intent. Query Design and Query Types [​](https://ast-grep.github.io/blog/code-search-design-space.html#query-design-and-query-types) ----------------------------------------------------------------------------------------------------------------------------- Every code search begins with a query, which is simply a way to tell the search engine what kind of code we're looking for. The way these queries are designed is crucial. Code search tool designers aim to achieve several key goals: #### Easy [​](https://ast-grep.github.io/blog/code-search-design-space.html#easy) A query should be easy to write, allowing users to quickly search without needing extensive learning. If it's too difficult to write a query, people might get discouraged from using the tool altogether. #### Expressive [​](https://ast-grep.github.io/blog/code-search-design-space.html#expressive) Users should be able to express whatever they're looking for. If the query language is too limited, you simply cannot find some results. #### Precise [​](https://ast-grep.github.io/blog/code-search-design-space.html#precise) The query should be specific enough to yield relevant results, avoiding irrelevant findings. An imprecise query will lead to a lot of noise. * * * Achieving all three of these goals simultaneously is challenging, as they often pull in opposing directions. For example, a very simple and easy query language might be expressive enough, or a very precise query language might be too complex for the average user. How do code search tools balance these goals? The blog categorizes code search queries into a few main types, each with its own characteristics: informal queries, formal queries, and hybrid queries. ![query design](https://ast-grep.github.io/image/blog/query-design.png) Informal Queries [​](https://ast-grep.github.io/blog/code-search-design-space.html#informal-queries) ----------------------------------------------------------------------------------------------------- These queries are closest to how we naturally express ourselves, and can be further divided into: ### Free-Form Queries [​](https://ast-grep.github.io/blog/code-search-design-space.html#free-form-queries) These are often free-form, using natural language to describe the desired code functionality, like web search. For example, "read file line by line" or "FileReader close." * **Pros:** Easy for users to formulate, similar to using a web search engine, and highly expressive. * **Cons:** Can be ambiguous and less precise due to the nature of natural language and potential vocabulary mismatches between the query and the code base. Tools like [GitHub Copilot](https://docs.github.com/en/enterprise-cloud@latest/copilot/using-github-copilot/asking-github-copilot-questions-in-github) use this approach. ### Input-Output Examples [​](https://ast-grep.github.io/blog/code-search-design-space.html#input-output-examples) These queries specify the desired behavior of the code by providing input-output pairs. You specify the desired behavior using pairs of inputs and their corresponding outputs. For example, the input "[susie@mail.com](mailto:susie@mail.com) " should result in the output "susie". * **Pros**: Allows to precisely specify desired behavior * **Cons**: May require some effort to provide sufficient examples This approach is more common in academic research than practical tools. This blog has not been aware of open source tools that use this approach. _We will not discuss informal queries in detail, as it is not precise._ **Formal Queries Based on Existing Programming Languages** [​](https://ast-grep.github.io/blog/code-search-design-space.html#formal-queries-based-on-existing-programming-languages) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Formal queries use a structured approach, making them more precise. They can be further divided into several subcategories. ### Plain Code [​](https://ast-grep.github.io/blog/code-search-design-space.html#plain-code) The simplest version involves providing an exact code snippet that needs to be matched in the codebase. For instance, a user might search for instances of the following Java snippet: java try { File file = File.createTempFile("foo", "bar"); } catch (IOException e) { } Not many tools directly support plain code search. They usually break search queries into smaller parts through the tokenization process, like traditional search engines. A notable example may be [grep.app](https://grep.app/) . ### Code with Holes [​](https://ast-grep.github.io/blog/code-search-design-space.html#code-with-holes) This approach involves providing code snippets with placeholders to search for code fragments. For example, a user might search for the following pattern in Java: java public void actionClose (JButton a, JFrame f) { $$$BODY } Here, `$$$BODY` is a placeholder, and the code search engine will try to locate all matching code. ast-grep falls into this category, treating the query as an Abstract Syntax Tree (AST) with holes. The holes in ast-grep are called metavariables. Other tools like gritql and the [structural search feature](https://www.jetbrains.com/help/idea/tutorial-work-with-structural-search-and-replace.html) in IntelliJ IDEA also use this technique. ### Code with Pattern Matching Symbols [​](https://ast-grep.github.io/blog/code-search-design-space.html#code-with-pattern-matching-symbols) These queries make use of special symbols to represent and match code structures. For example, the following query in [Comby](https://comby.dev/docs/basic-usage#how-matching-works) attempts to find all if statements where the condition is a comparison. comby if (:[var] <= :[rest]) In Comby, the `:[var]` and `:[rest]` are special markers that match strings of code. java if (width <= 1280 && height <= 800) { return 1; } The `:[var]` matches any string until a `<=` character is found and in this case is `width`. `:[rest]` matches everything that follows, `1280 && height <= 800`. Unlike ast-grep, Comby is not AST-aware, as the `:[rest]` in the example spans across multiple AST nodes. Tools like [Comby](https://comby.dev/) and [Shisho](https://github.com/flatt-security/shisho) use this approach. ### Pros and Cons [​](https://ast-grep.github.io/blog/code-search-design-space.html#pros-and-cons) **Pros:** Easy to formulate for developers familiar with programming languages. **Cons:** Parsing incomplete code snippets can be a challenge. The downside of using existing languages is also emphasized in the IntelliJ IDEA documentation: > Any (SSR) template entered should be a well formed Java construction ... An off-the-shelf grammar of the programming language may not be able to parse a query because the query is [incomplete or ambiguous](https://ast-grep.github.io/advanced/pattern-parse.html#pattern-is-ast-based) . For example, `"key": "value"` is not a valid JSON object, a JSON parser will reject and will fail to create a query. Maybe it is clear to a human that it is a key-value pair, but the parser does not know that. Other examples will be like [distinguishing function calls](https://ast-grep.github.io/catalog/c/) and macro invocation in C/C++. TIP ast-grep takes a unique approach to this problem. It uses a [pattern object](https://ast-grep.github.io/guide/rule-config/atomic-rule.html#pattern-object) to represent and disambiguate a complete and valid code snippet, and then leverages a [`selector`](https://ast-grep.github.io/reference/rule.html#pattern) to extract the part that matches the query. Formal Queries using Custom Languages [​](https://ast-grep.github.io/blog/code-search-design-space.html#formal-queries-using-custom-languages) ----------------------------------------------------------------------------------------------------------------------------------------------- ### Significant Extensions of Existing Programming Languages [​](https://ast-grep.github.io/blog/code-search-design-space.html#significant-extensions-of-existing-programming-languages) These languages extend existing programming languages with features like wildcard tokens or regular expression operators. For example, the pattern `$(if $$ else $) $+` might be used to find all nested if-else statements in a codebase. [Coccinelle](https://coccinelle.gitlabpages.inria.fr/website/) and [Semgrep](https://semgrep.dev/) are tools that take this approach. Semgrep's pattern-syntax, for example, has extensive features such as [ellipsis metavariables](https://semgrep.dev/docs/writing-rules/pattern-syntax#ellipsis-metavariables) , [typed metavariables](https://semgrep.dev/docs/writing-rules/pattern-syntax#typed-metavariables) , and [deep expression operators](https://semgrep.dev/docs/writing-rules/pattern-syntax#deep-expression-operator) , that cannot be parsed by a standard programming language' implementation. Ellipsis MetavariablesTyped MetavariablesDeep Expression operators yaml # combine ellipses and metavariables to match a sequence of ASTs # note the ellipsis is not valid programming language syntax pattern: foo($...ARGS, 3, $...ARGS) # this pattern will match foo(1, 2, 3, 4, 5) yaml # look for calls to the log method on Logger objects. # A simple pattern like this will match `Math.log()` as well pattern: $LOGGER.log(...) # typed metavariable can put a type constraint on the metavariable # but it is no longer valid Java code pattern: (java.util.logging.Logger $LOGGER).log(...) yaml # Use the deep expression operator <... [your_pattern] ...> # to match an expression that # could be deeply nested within another expression pattern: | if <... $USER.is_admin() ...>: ... **Pros**: These languages can be more expressive than plain programming languages. **Cons**: Users need to learn new syntax and semantics and tool developers to support the extension Difference from ast-grep Note ast-grep also supports multi meta variables in the form of `$$$VARS`. Compared to Semgrep, ast-grep's metavariables still produce valid code snippets. We can represent also search query using **Domain Specific Language** ### Logic-based Querying Languages [​](https://ast-grep.github.io/blog/code-search-design-space.html#logic-based-querying-languages) These languages utilize first-order logic or languages like Datalog to express code properties. For example, a user can find all classes with the name "HelloWorld". Some of these languages also resemble SQL. [CodeQL](https://codeql.github.com/) and [Glean](https://glean.software/docs/angle/intro/) are two notable examples. Here is an example from CodeQL: sql from If ifstmt, Stmt pass where pass = ifstmt.getStmt(0) and pass instanceof Pass select ifstmt, "This 'if' statement is redundant." This CodeQL query will identify redundant if statements in Python, where the first statement within the if block is a pass statement. Explaination of the query * `from If ifstmt, Stmt pass`: This part of the query defines two variables, `ifstmt` and `pass`, which will be used in the query. * `where pass = ifstmt.getStmt(0) and pass instanceof Pass`: This part of the query filters the results. It checks if the first statement in the `ifstmt` is a `Pass` statement. * `select ifstmt, "This 'if' statement is redundant."`: This part of the query selects the results. It returns the `ifstmt` and a message. **Pros:** These languages can precisely express complex code properties beyond syntax. **Cons:** Learning curve is steep. ### Embedded Domain Specific Language [​](https://ast-grep.github.io/blog/code-search-design-space.html#embedded-domain-specific-language) Embedded DSLs are using the host language to express the query. The query is embedded in the host language, and the host language provides the necessary constructs to express the query. The query is then parsed and interpreted by the tool. There are further two flavors of embedded DSLs: configuration-based and program-based. #### Configuration-based eDSL [​](https://ast-grep.github.io/blog/code-search-design-space.html#configuration-based-edsl) Configuration-based eDSLs allow user to provide configuration objects that describes the query. The tool then interprets this configuration object to perform the search. ast-grep CLI and semgrep CLI both adopt this approach using YAML files. ast-grep YAML ruleSemgrep YAML rule yaml id: match-function-call language: c rule: pattern: context: $M($$$); selector: call_expression yaml rules: - id: my-pattern-name pattern: | TODO message: "Some message to display to the user" languages: [python] severity: ERROR Configuration files are more expressive than patterns and still relatively easy to write. Users usually already know the host language (YAML) and can leverage its constructs to express the query. #### Program-based eDSL [​](https://ast-grep.github.io/blog/code-search-design-space.html#program-based-edsl) Program-based eDSLs provide direct access to the AST through AST node objects. Examples of programmatic APIs include [JSCodeshift](https://jscodeshift.com/build/api-reference/) , the [Code Property Graph](https://docs.joern.io/code-property-graph/) from [Joern](https://joern.io/) , and ast-grep's [NAPI](https://ast-grep.github.io/guide/api-usage.html) . @ast-grep/napiJSCodeshiftCode Property Graph typescript import { parse, Lang } from '@ast-grep/napi' let source = `console.log("hello world")` const ast = parse(Lang.JavaScript, source) // 1. parse the source const root = ast.root() // 2. get the root const node = root.find('console.log($A)') // 3. find the node node.getMatch('A').text() // 4. collect the info // "hello world" javascript const j = require('jscodeshift'); const root = j(`const a = 1; const b = 2;`); const types = root.find(j.VariableDeclarator).getTypes(); console.log(types); // Set { 'VariableDeclarator' } scala import io.shiftleft.codepropertygraph.Cpg import io.shiftleft.semanticcpg.language._ object FindExecCalls { def main(args: Array[String]): Unit = { // Load the C codebase val cpg: Cpg = Cpg.apply("path/to/your/codebase") // Find all `exec` function calls and print their locations cpg.call("exec").location.l.foreach(println) } } **Pros:** Offer more precision and expressiveness and are relatively easy to write. **Cons**: The overhead to communicate between the host language and the search tool can be high. ### General Purpose Like Programming Language [​](https://ast-grep.github.io/blog/code-search-design-space.html#general-purpose-like-programming-language) Finally, tools can also design their own general purpose programming languages. These languages provide a full programming language to describe code properties. [GritQL](https://about.grit.io/) is an example of this approach. For example, this GritQL query rewrites all `console.log` calls to `winston.debug` and all `console.error` calls to `winston.warn`: gritql `console.$method($msg)` => `winston.$method($msg)` where { $method <: or { `log` => `debug`, `error` => `warn` } } Explaination of the Query 1. **Pattern Matching**: The pattern `console.$method($msg)` is used to match code where there is a `console` object with a method (`$method`) and an argument (`$msg`). Here, `$method` and `$msg` are placeholders for any method and argument, respectively. 2. **Rewrite**: The rewrite symbole `=>` specifies that the matched `console` code should be transformed to use `winston`, followed by the method (`$method`) and the argument (`$msg`). 3. **Method Mapping**: The `where` clause specifies additional constraints on the rewrite. Specifically, `$method <: or { 'log' => 'debug', 'error' => 'warn' }` means: * If `$method` is `log`, it should be transformed to `debug`. * If `$method` is `error`, it should be transformed to `warn`. In sum, this rule replaces console logging methods with their corresponding Winston logging methods: * `console.log('message')` becomes `winston.debug('message')` * `console.error('message')` becomes `winston.warn('message')` **Pros:** It offers more precision and expressiveness compared to simple patterns and configuration-based embedded DSLs. But it may not be as flexible as program-based eDSL nor as powerful as logic-based languages. **Cons:** Have the drawback of requiring users to learn the custom language first. It is easier to learn than logic-based languages, but still requires some learning compared to using embedded DSL. Hybrid Queries [​](https://ast-grep.github.io/blog/code-search-design-space.html#hybrid-queries) ------------------------------------------------------------------------------------------------- Hybrid queries combine multiple query types. For example, you can combine free-form queries with input-output examples, or combine natural language queries with program element references. ast-grep is a great example of a tool that uses hybrid queries. You can define patterns directly in a YAML rule or use a programmatic API. First, you can embed the pattern in the YAML rule, like this: yaml rule: pattern: console.log($A) inside: kind: function_declaration You can also use the similar concept in the programmatic API typescript import { Lang, parse } from '@ast-grep/napi' const sg = parse(Lang.JavaScript, code) sg.root().find({ rule: { pattern: 'console.log($A)', inside: { kind: 'function_declaration' } } }) This flexible design allows you to combine basic queries into larger, more complex ones, and you can always use a general-purpose language for very complex and specific searches. ast-grep favors existing programming languages We don't want the user to learn a new language, but rather use the existing language constructs to describe the query. We also think TypeScript is a great language with [great type system](https://ast-grep.github.io/blog/typed-napi.html) . There is no need to reinvent a new language to express code search logic. ast-grep's Design Choices [​](https://ast-grep.github.io/blog/code-search-design-space.html#ast-grep-s-design-choices) ----------------------------------------------------------------------------------------------------------------------- Designing a code search tool involves a delicate balancing act. It's challenging to simultaneously achieve ease of use, expressiveness, and precision, as these goals often conflict. Code search tools must carefully navigate these trade-offs to meet the diverse needs of their users. ast-grep makes specific choices to address this challenge: * **Prioritizing Familiarity**: It uses pattern matching based on existing programming language syntax, making it easy for developers to start using the tool with familiar coding structures. * **Extending with Flexibility**: It incorporates configuration-based (YAML) and program-based (NAPI) embedded DSLs, providing additional expressiveness for complex searches. * **Hybrid, and Progressive, Design**: Its pattern matching, YAML rules, and NAPI are designed for hybrid use, allowing users to start simple and gradually add complexity. The concepts in each API are also transferable, enabling users to progressively learn more advanced techniques. * **AST-Based Precision**: It emphasizes precision by requiring all queries to be AST-based, ensuring accurate results. Though it comes with the trade-off that queries should be carefully crafted. * **Multi-language Support**: Instead of creating a new query language for all programming languages or significantly extending existing ones for code search purposes, which would be an enormous undertaking, ast-grep reuses the familiar syntax of the existing programming languages in its patterns. This makes the tool more approachable for developers working across multiple languages. Additional Considerations [​](https://ast-grep.github.io/blog/code-search-design-space.html#additional-considerations) ----------------------------------------------------------------------------------------------------------------------- While we've focused on query design, there are other factors that influence the effectiveness of code search tools. These include: * Offline Indexing: This is crucial for rapid offline searching. Currently, ast-grep always builds an AST in memory for each query, meaning it doesn't support offline indexing. Tools like grep.app, which do use indexing, is faster for searching across millions of repositories. * Information Indexing: Code search can index various kinds of information besides just code elements. Variable scopes, type information, definitions, and control and data flow are all valuable data for code search. Currently, ast-grep only indexes the AST itself. * Retrieval Techniques: How a tool finds matching code given a query is a critical aspect. Various algorithmic and machine learning approaches exist for this. ast-grep uses a manual implementation that compares the query's AST with the code's AST. * Ranking and Pruning: How search results are ordered is also a critical factor in providing good search results. --- # An Example of Rust's Fearless Concurrency | ast-grep [Skip to content](https://ast-grep.github.io/blog/fearless-concurrency.html#VPContent) On this page An Example of Rust's Fearless Concurrency [​](https://ast-grep.github.io/blog/fearless-concurrency.html#an-example-of-rust-s-fearless-concurrency) =================================================================================================================================================== Rust is famous for its "fearless concurrency." It's a bold claim, but what does it actually _mean_? How does Rust let you write concurrent code without constantly battling race conditions? [ast-grep](https://ast-grep.github.io/) 's [recent refactor](https://github.com/ast-grep/ast-grep/discussions/1710) is a great example of Rust's concurrency model in action. Old Architecture of ast-grep's Printer [​](https://ast-grep.github.io/blog/fearless-concurrency.html#old-architecture-of-ast-grep-s-printer) --------------------------------------------------------------------------------------------------------------------------------------------- `ast-grep` is basically a syntax-aware `grep` that understands code. It lets you search for specific patterns within files in a directory. To make things fast, it uses multiple worker threads to churn through files simultaneously. The results then need to be printed to the console, and that's where our concurrency story begins. Initially, ast-grep had a single `Printer` object, shared by _all_ worker threads. This was designed for maximum parallelism – print the results as soon as you find them! Therefore, the `Printer` had to be thread-safe, meaning it had to implement the `Send + Sync` traits in Rust. These traits are like stamps of approval, saying "this type is safe to move between threads (`Send`) and share between threads (`Sync`)." rust trait Printer: Send + Sync { fn print(&self, result: ...); } // demo Printer implementation struct StdoutPrinter { // output is shared between threads output: Mutex, } impl Printer for StdoutPrinter { fn print(&self, result: ...) { // lock the output to print let stdout = self.output.lock().unwrap(); writeln!(stdout, "{}", result).unwrap(); } } And `Printer` would be used in worker threads like this: rust // in the worker thread struct Worker { // printer is shareable between threads // because it implements Send + Sync printer: P, } impl

Worker

{ fn search(&self, file: &File) { let results = self.search_in_file(file); self.printer.print(results); } // other methods not using printer... } While this got results quickly, it wasn't ideal from a user experience perspective. Search results were printed all over the place, not grouped by file, and often out of order. Not exactly user-friendly. Migrate to Message-Passing Model [​](https://ast-grep.github.io/blog/fearless-concurrency.html#migrate-to-message-passing-model) --------------------------------------------------------------------------------------------------------------------------------- The architecture needed a shift. Instead of sharing a printer, we moved to a message-passing model, using an [`mpsc` channel](https://doc.rust-lang.org/std/sync/mpsc/) . `mpsc` stands for Multi-Producer, Single-Consumer FIFO queue, where a `Sender` is used to send data to a `Receiver`. Now, worker threads would send search results to a single dedicated _printer thread_. This printer thread then handles the printing sequentially and neatly. Here's the magic: because the printer is no longer shared between threads, we could remove the `Send + Sync` constraint! No more complex locking mechanisms! The printer could be a simple struct with a mutable reference to the standard output. ![concurrent programming bell curve](https://ast-grep.github.io/image/blog/concurrent.jpg) Here are some more concrete changes we made: ### Remove Generics [​](https://ast-grep.github.io/blog/fearless-concurrency.html#remove-generics) The printer used to be a field of `Worker`. Now, we had to move it out to the main thread. rust struct Worker { sender: Sender<...>, } impl Worker { fn search(&self, file: &File) { let results = self.search_in_file(file); self.sender.send(results).unwrap(); } // other methods, no generic used } fn main() { let (sender, receiver) = mpsc::channel(); let mut printer = StdoutPrinter::new(); let printer_thread = thread::spawn(move || { for result in receiver { printer.print(result); } }); // spawn worker threads } So, what did we gain? **Smaller binary size**. Previously, the worker struct was generic over the printer trait, which meant that the compiler had to generate code for each printer implementation. This resulted in a larger binary size. By removing generics over the printer trait, the worker struct no longer needs multiple copies. ### Remove `Send + Sync` Bounds [​](https://ast-grep.github.io/blog/fearless-concurrency.html#remove-send-sync-bounds) The `Send + Sync` bounds on the printer trait were no longer needed. The CLI changed the printer signature to use a mutable reference instead of an immutable reference. In the previous version, we couldn't use `&mut self` because it cannot be shared between threads. So we had to use `&self` and wrap the output in a `Mutex`. Now we can simply use a mutable reference since it is no longer shared between threads. rust trait Printer { fn print(&mut self, result: ...); } // stdout printer implementation struct StdoutPrinter { output: Stdout, // no more Mutex } impl Printer for StdoutPrinter { fn print(&mut self, result: ...) { writeln!(self.output, "{}", result).unwrap(); } } Without the need to lock the printer object, the code became **faster** in a single thread, without data-racing. Thanks to Rust, this big architectural change was relatively painless. The compiler caught all the places where we were trying to share the printer between threads. It forced us to think about the design and make the necessary changes. What Rust Teaches Us [​](https://ast-grep.github.io/blog/fearless-concurrency.html#what-rust-teaches-us) --------------------------------------------------------------------------------------------------------- This experience with `ast-grep` really highlights Rust's approach to concurrency. Rust forces you to _think deeply_ about your design and _encode_ it in the type system. You can't just haphazardly add threads and hope it works. Without clearly **designing the process architecture upfront**, you will soon find yourself trapped in a maze of the compiler's error messages. Rust then forces you to express the concurrency design in code via **type system enforcement**. You need to use concurrency primitives, ownership rules, borrowing, and the `Send`/`Sync` traits to encode your design constraints. The compiler acts like a strict project manager, not allowing you to ship code if it doesn't meet the concurrency requirements. In other languages, concurrency is often treated as an afterthought. It is up to the programmer's discretion to design the architecture correctly. And it is also the programmer's responsibility to conscientiously and meticulously ensure the architecture is correctly implemented. The Trade-off of Fearless Concurrency [​](https://ast-grep.github.io/blog/fearless-concurrency.html#the-trade-off-of-fearless-concurrency) ------------------------------------------------------------------------------------------------------------------------------------------- [And what, Rust, must we give in return?](https://knowyourmeme.com/memes/guldan-offer) Rust's approach comes with a trade-off: * **Upfront design investment:** You need to design your architecture thoroughly before you start writing actual production code. While the compiler could be helpful when you explore options or ambiguous design ideas, it can also be a hindrance when you need to iterate quickly. * **Refactoring can be hard:** If you need to change your architectural design, it can be an invasive change across your codebase, because you need to change the type signatures, the concurrency primitives, and data flows. Other languages might be more flexible in this regard. Rust feels a bit like a mini theorem prover, like [Lean](https://lean-lang.org/) . You are using the compiler to prove that your concurrent model is correct and safe. If you are still figuring out your product market fit and need rapid iteration, other languages might be [a better choice](https://x.com/charliermarsh/status/1867927883421032763) . But if you need the safety and performance that Rust provides, it is definitely worth the effort! The Fun to Play with Rust [​](https://ast-grep.github.io/blog/fearless-concurrency.html#the-fun-to-play-with-rust) ------------------------------------------------------------------------------------------------------------------- ast-grep is a hobby project. Even though it might be a bit more work to get started, this small project shows that building concurrent applications in Rust can be [fun and rewarding](https://x.com/charliermarsh/status/1873402334967173228) . I hope this gave you a glimpse into Rust's fearless concurrency and maybe inspires you to take the plunge! --- # ast-grep's Journey to AI Generated Rules | ast-grep [Skip to content](https://ast-grep.github.io/blog/ast-grep-agent.html#VPContent) Return to top ast-grep's Journey to AI Generated Rules [​](https://ast-grep.github.io/blog/ast-grep-agent.html#ast-grep-s-journey-to-ai-generated-rules) =========================================================================================================================================== ast-grep is a command-line tool that empowers developers to find and replace code with precision. It operates directly on the syntax tree (AST), the true structure of your code. While powerful, writing ast-grep rules is a hurdle that requires grokking the tool. This project has always been about designed and built as a tool for human beings, not for AI hype. But ast-grep been [exploring how to use Al](https://ast-grep.github.io/blog/more-llm-support.html) to improve the human experience. This is the story of my journey into using Al to generate YAML rules. Why ast-grep Rules are Hard for AI [​](https://ast-grep.github.io/blog/ast-grep-agent.html#why-ast-grep-rules-are-hard-for-ai) ------------------------------------------------------------------------------------------------------------------------------- Generating ast-grep rules is not an easy task,especially for AI. First, ast-grep is a relatively new tool, so Large Language Models have not been extensively trained on its specific syntax, leading to hallucinations or a complete lack of understanding. While [in-context learning](https://www.prompthub.us/blog/in-context-learning-guide) can help an LLM grasp the basic syntax, it often fails to address the problem of [compounding errors](https://arxiv.org/abs/2505.24187v1) . This is particularly true for ast-grep, where rules are frequently composed of smaller, [atomic rules](https://ast-grep.github.io/guide/rule-config/atomic-rule.html) . Human developers often need to debug these smaller rules through [trial and error](https://ast-grep.github.io/advanced/faq.html#my-pattern-does-not-work-why) , and pay extra attention to how they are orchestrated into a [working composition](https://ast-grep.github.io/advanced/faq.html#my-rule-does-not-work-why) . For an LLM, a small mistake in one part of the rule can snowball into a completely incorrect rule. Quoting from the paper [Faith and Fate](https://arxiv.org/abs/2305.18654) : > _These tasks require breaking problems down into sub-steps and synthesizing these steps into a precise answer._ This explains why a simple LLM ask won't work for ast-grep. However, recent advancements in "thinking" and "agentic" models, which can correct their mistakes, may [reduce the probability of such errors](https://arxiv.org/abs/2501.15602v2) . This makes it a good time to give AI-powered ast-grep rule generation another try. The First Foray a Year Ago [​](https://ast-grep.github.io/blog/ast-grep-agent.html#the-first-foray-a-year-ago) --------------------------------------------------------------------------------------------------------------- My initial attempt, about a year ago, was a lesson in humility. I tried to build an agent using [DSPy](https://dspy.ai/) with a rigid, fixed pipeline: generate use cases, create code examples, describe the rule in natural language, translate that description to an ast-grep rule, and finally, verify it. The results were, to put it mildly, not good. The models I was using are mostly free-tier APIs from [Ollama](https://ollama.com/) and [aistudio](https://aistudio.google.com/) (thank [$GOOG](https://finance.yahoo.com/quote/GOOG/) ). They struggled to adhere to a consistent output format. They often failed to generate correct code examples, let alone valid ast-grep rules. The limited context windows meant I couldn't even provide sufficient instructions to guide them properly. It felt like I was missing two crucial pieces: a way for the agent to refine a rule based on search result feedback, and a method to dynamically break down complex requests into smaller, manageable rules. The agent architecture could solve these, but the underlying models could not. One Year After: Better Models, Better Agents [​](https://ast-grep.github.io/blog/ast-grep-agent.html#one-year-after-better-models-better-agents) ------------------------------------------------------------------------------------------------------------------------------------------------- Fast forward to today. The landscape has changed dramatically. Modern LLMs boast long context windows and a much-improved ability to follow complex instructions. Armed with these new capabilities and inspired by [Anthropic's new guide](https://www.anthropic.com/engineering/building-effective-agents) , I tried again. This time, I discovered that a complex agent framework wasn't necessary. AI programming tools like [Cursor](https://www.cursor.com/) have sufficient agent frameworks for tool developers. The secret sauce was something far simpler: prompt engineering. And by "[prompt engineering](https://www.promptingguide.ai/) ", I don't mean any framework or trick. It's just writing a clear, human-readable manual for both human and Al. Before the prompt engineering, however, I saw some interesting failure modes across different models that somehow reflects different LLM vendors' training setup. All of them did a decent job of interpreting the user's intent and creating relevant code examples, but their approaches to rule generation were wildly different. * **OpenAI O3** hallucinated with wild abandon. It invented syntax that looked more like [CodeQL](https://codeql.github.com/) or [jscodeshift](https://github.com/facebook/jscodeshift) , completely ignoring the ast-grep documentation available online. It couldn't recover from tool errors and would quickly give up on using the tools I gave it. It felt like OpenAI's pretraining dataset glanced at my documentation, decided it knew better, and threw it in the bin. (Oops, like how OpenAI treated my resume) * **Gemini** was a bit more grounded. Its hallucinations were at least in the right ballpark, borrowing syntax from a [related but more established](https://ast-grep.github.io/advanced/tool-comparison.html#semgrep) tool, [Semgrep](https://semgrep.dev/) . (Thanks for the flattery, again, $GOOG). It also showed a decent ability to recover from errors but had a stubborn streak, preferring to invent its own ast-grep cli commands rather than using the [MCP](https://modelcontextprotocol.io/) tools I [provided](https://github.com/ast-grep/ast-grep-mcp) . * **Claude 4** was the most promising out of the box. It correctly identified ast-grep and produced syntactically valid rules. Looks like Anthropic's training data is indexing ast-grep's doc! Hoooray! However, it struggled with subtle semantic details that would make a rule functionally correct. To its credit, it tried very hard, retrying the tools I gave it over and over with different inputs, demonstrating a dogged persistence the others lacked. Prompting AI Agents like Teaching a Human [​](https://ast-grep.github.io/blog/ast-grep-agent.html#prompting-ai-agents-like-teaching-a-human) --------------------------------------------------------------------------------------------------------------------------------------------- After iterating on the prompt, I found that all three models could perform remarkably well. The key was to treat the prompt not as a magic incantation, but as a straightforward instruction manual. The core of my final prompt gives the Al a simple, five-step plan: 1. First, clearly understand the user's request. 2. Next, write a simple code snippet that the user wants to find. 3. Then, write an ast-grep rule that precisely matches that code snippet. 4. [Test the rule](https://github.com/ast-grep/ast-grep-mcp/blob/b69eb5391bd93d46ef3dec07de814c3c39675c8f/main.py#L33-L57) against the example to ensure it works as expected. 5. Finally, [search](https://github.com/ast-grep/ast-grep-mcp/blob/b69eb5391bd93d46ef3dec07de814c3c39675c8f/main.py#L72-L82) the codebase with the verified rule. This clear, step-by-step process turned erratic geniuses into more reliable assistants. By breaking the problem down and providing a verification loop, I gave the models the structure they needed to succeed. You can see the full prompt in the [sg-mcp GitHub repository](https://github.com/ast-grep/ast-grep-mcp/blob/main/ast-grep.mdc) . The end result is quite impressive, see the [demo video](https://youtube.com/shorts/2hah-9N5YQ8?si=bzl6PF2tuFbBwXpL) below. No speed up, but with music and cats. (It is oddly satisfactory to watch AI generating the rules while I myself is nodding like the kitty in the video) This journey has reinforced a core belief: Al's true power isn't about replacing the developer, but about building better tools for them. It also demonstrates that the best way to teach an AI, with the progress of more capable Large Language Models, is not through complex frameworks or rigid pipelines, but through clear, human-readable instructions. By teaching an Al to write rules, I hope it makes ast-grep more accessible, more powerful, and easier to use. Happy grepping! --- # Interactive Code Fixes with Multiple Options! | ast-grep [Skip to content](https://ast-grep.github.io/blog/interactive-demo.html#VPContent) Return to top Interactive Code Fixes with Multiple Options! [​](https://ast-grep.github.io/blog/interactive-demo.html#interactive-code-fixes-with-multiple-options) ====================================================================================================================================================== Today, we're thrilled to showcase a game-changing feature: **multi-option interactive code fixes!** Beyond Simple Search: Interactive Refactoring [​](https://ast-grep.github.io/blog/interactive-demo.html#beyond-simple-search-interactive-refactoring) ------------------------------------------------------------------------------------------------------------------------------------------------------ ast-grep's interactive mode, activated with `ast-grep scan --interactive`, transforms code analysis into a dynamic, actionable workflow. When a rule identifies a pattern match, you're presented with: 1. **Clear Diff Views:** Instantly see the problematic code (red) and the proposed change (green), making it easy to understand the impact of the fix. 2. **Contextual Markdown Notes:** Rules can embed rich Markdown notes, providing instant documentation, best practices, and explanations directly in your terminal – no need to jump to external docs. The Power of Choice: Multiple Fix Options [​](https://ast-grep.github.io/blog/interactive-demo.html#the-power-of-choice-multiple-fix-options) ---------------------------------------------------------------------------------------------------------------------------------------------- This is where ast-grep truly stands out. Instead of a single, rigid fix, many rules now offer **multiple, intelligent remediation options**. **How it works:** * When a match is found, ast-grep displays a list of available fixes. * Simply use the **`tab` key** to cycle through the different fix proposals. * Once you've found the ideal solution, hit `Enter` to apply it. This flexibility allows you to choose the fix that best aligns with your project's coding standards or specific refactoring goals. Real-World Example: Angular `@Input()` Optionality [​](https://ast-grep.github.io/blog/interactive-demo.html#real-world-example-angular-input-optionality) ----------------------------------------------------------------------------------------------------------------------------------------------------------- Consider a common TypeScript scenario in Angular: an `@Input()` decorator where the component property is typed as `string`, but it's optional by default (meaning it could be `undefined`). ast-grep's rule for this issue intelligently offers two distinct fixes: 1. **Add `undefined` to Type:** Transforms `test: string;` to `test: string | undefined;`, explicitly acknowledging the optionality in the type system. 2. **Make Input Required:** Adds `{ required: true }` to the `@Input()` decorator, enforcing that the input must always be provided. You choose the solution that fits your use case, and ast-grep applies the transformation seamlessly. Behind the Scenes: Configurable Fixes [​](https://ast-grep.github.io/blog/interactive-demo.html#behind-the-scenes-configurable-fixes) -------------------------------------------------------------------------------------------------------------------------------------- This powerful multi-fix capability is driven by the rule's YAML configuration. Rules can define an array of `fix` templates, each with a unique `title` and `template`, allowing rule authors to provide comprehensive repair options. ### Streamline Your Workflow Today! [​](https://ast-grep.github.io/blog/interactive-demo.html#streamline-your-workflow-today) Ast-grep with its interactive multi-fix feature is a game-changer for maintaining code quality, enforcing standards, and accelerating large-scale code transformations. It puts the power of intelligent, context-aware refactoring directly into your hands. **Ready to refactor like a pro?** Give ast-grep a try and experience the future of code analysis and transformation! --- # How to Debug ast-grep Rule Effectively | ast-grep [Skip to content](https://ast-grep.github.io/blog/how-to-debug.html#VPContent) Return to top How to Debug ast-grep Rule Effectively [​](https://ast-grep.github.io/blog/how-to-debug.html#how-to-debug-ast-grep-rule-effectively) ===================================================================================================================================== Let Claude Debug For You If you prefer not to debug manually, try the [ast-grep Claude skill](https://github.com/ast-grep/claude-skill) . It can explain AST structures, identify why rules don't match, and suggest fixes—all through natural conversation. Debugging ast-grep rules can be frustrating. You write what looks like a perfectly reasonable rule, test it against your code, and... nothing matches. Or worse, it matches things you didn't expect. The key to effective debugging is one word: **SIMPLIFY**. When your rule doesn't work, resist the urge to add more conditions or make the pattern more complex. Instead, strip everything down to the basics and build back up systematically. This post will teach you a reliable debugging workflow that works for any ast-grep rule. The Debugging Workflow [​](https://ast-grep.github.io/blog/how-to-debug.html#the-debugging-workflow) ----------------------------------------------------------------------------------------------------- Here's a step-by-step process to debug any ast-grep rule: 1. **Set up a reproducible test case.** Use `ast-grep scan -r test.yml test.file` or the [online playground](https://ast-grep.github.io/playground.html) to quickly iterate on your code and rule. 2. **Reduce the code to a minimal example.** Delete everything unrelated to the rule. If your rule should match a function call, remove all the surrounding code until you have just the essential lines. 3. **Inspect the AST structure.** Use `ast-grep run -p '{code}' -l {lang} --debug-query=cst` or the playground's AST view to understand the actual tree structure. The AST often looks different from what you'd expect. 4. **Simplify the rule.** Remove rule conditions one by one. Test each simpler version to see what actually matches. Pay attention to how _**meta-variables**_ are being captured. 5. **Repeat steps 2-4.** Continue simplifying both code and rule until you isolate the issue. Let's see this workflow in action with real examples. Example 1: The SQL Injection Detector [​](https://ast-grep.github.io/blog/how-to-debug.html#example-1-the-sql-injection-detector) ---------------------------------------------------------------------------------------------------------------------------------- Consider this rule designed to [detect potential SQL injection](https://ast-grep.github.io/playground.html#eyJtb2RlIjoiQ29uZmlnIiwibGFuZyI6InB5dGhvbiIsInF1ZXJ5IjoiPERpYWxvZyAkJCQ+IiwicmV3cml0ZSI6IiIsInN0cmljdG5lc3MiOiJzbWFydCIsInNlbGVjdG9yIjoiIiwiY29uZmlnIjoiaWQ6IHNvbWVfc3FsaV9ydWxlXG5sYW5ndWFnZTogcHl0aG9uXG5ydWxlOlxuICBwYXR0ZXJuOiAkWC5leGVjdXRlKCQkJClcbiAgaGFzOlxuICAgIGtpbmQ6IGFyZ3VtZW50X2xpc3RcbiAgICBoYXM6XG4gICAgICBudGhDaGlsZDogMVxuICAgICAgYW55OlxuICAgICAgICAtIGtpbmQ6IGlkZW50aWZpZXJcbiAgICAgICAgICBwYXR0ZXJuOiAkVkFSXG4gICAgICAgIC0gaGFzOlxuICAgICAgICAgICAgc3RvcEJ5OiBlbmRcbiAgICAgICAgICAgIGtpbmQ6IGlkZW50aWZpZXJcbiAgICAgICAgICAgIHBhdHRlcm46ICRWQVJcbiAgaW5zaWRlOlxuICAgIHN0b3BCeTogZW5kXG4gICAga2luZDogbW9kdWxlXG4gICAgaGFzOlxuICAgICAgc3RvcEJ5OiBlbmRcbiAgICAgIGtpbmQ6IGFzc2lnbm1lbnRcbiAgICAgIHBhdHRlcm46ICRWQVIgPSAkJCQiLCJzb3VyY2UiOiJkZWYgdGVzdF9zcWxfaW5qZWN0aW9uX2RldGVjdGlvbigpOlxuICAgIFwiXCJcIlRlc3QgY2FzZSBmb3Igc3RhdGljIGFuYWx5c2lzIHRvb2xzIGRldGVjdGluZyBTUUwgaW5qZWN0aW9uIHZ1bG5lcmFiaWxpdGllc1wiXCJcIlxuICAgICMgU2V0dXAgdGVzdCBkYXRhYmFzZVxuICAgIGRiID0gRGF0YWJhc2VNYW5hZ2VyKCc6bWVtb3J5OicpXG4gICAgdXNlcl9pbnB1dCA9IHJlcS5xdWVyeS5wYXJhbVxuICAgIHZ1bG5fcGFyYW0gPSBjb21wdXRlX2Jhc2VkX29uX2lucHV0KHVzZXJfaW5wdXQpXG4gICAgZGIuZXhlY3V0ZShmXCJEUk9QIFRBQkxFIElGIEVYSVNUUyB7dnVsbl9wYXJhbX1cIikgICMgVnVsbmVyYWJsZSwgYnV0IG5vdCBkZXRlY3RlZCEifQ==) vulnerabilities in Python: yaml id: some_sqli_rule language: python rule: pattern: $X.execute($$$) has: kind: argument_list has: nthChild: 1 any: - kind: identifier pattern: $VAR - has: stopBy: end kind: identifier pattern: $VAR inside: stopBy: end kind: module has: stopBy: end kind: assignment pattern: $VAR = $$$ The rule should flag cases where a variable assigned from user input is passed to `execute()`. Let's test it against this code: python def test_sql_injection_detection(): """Test case for static analysis tools detecting SQL injection vulnerabilities""" # Setup test database db = DatabaseManager(':memory:') user_input = req.query.param vuln_param = compute_based_on_input(user_input) db.execute(f"DROP TABLE IF EXISTS {vuln_param}") # Vulnerable, but not detected! No match. Why? ### Step 1: Reduce to Minimal Code [​](https://ast-grep.github.io/blog/how-to-debug.html#step-1-reduce-to-minimal-code) Let's strip the code down to the essentials: python something = "value" vuln_param = other x.execute(f"DROP TABLE IF EXISTS {vuln_param}") # Still no match Interestingly, if we remove the first assignment: python vuln_param = other x.execute(f"DROP TABLE IF EXISTS {vuln_param}") # This matches! Now it matches! The presence of `something = "value"` somehow breaks the rule. This is our first clue. ### Step 2: Simplify the Rule [​](https://ast-grep.github.io/blog/how-to-debug.html#step-2-simplify-the-rule) Let's test individual parts of the rule. First, just the `has` portion: yaml rule: pattern: $X.execute($$$) has: kind: argument_list has: nthChild: 1 any: - kind: identifier pattern: $VAR - has: stopBy: end kind: identifier pattern: $VAR This matches! And it captures `$VAR` as `vuln_param`. Now let's test just the `inside` portion: yaml rule: pattern: $X.execute($$$) inside: stopBy: end kind: module has: stopBy: end kind: assignment pattern: $VAR = $$$ This also matches! But wait—what does it capture `$VAR` as? ### Step 3: Identify the Conflict [​](https://ast-grep.github.io/blog/how-to-debug.html#step-3-identify-the-conflict) Here's the problem: when we have both assignments in the code, the `inside` rule matches `$VAR` to `something` (the first assignment it encounters), while the `has` rule expects `$VAR` to be `vuln_param`. Since `something ≠ vuln_param`, the combined rule fails. This is due to [rule matching order sensitivity](https://ast-grep.github.io/advanced/faq.html#why-is-rule-matching-order-sensitive) . In YAML, sibling keys are processed in an implementation-defined order. The `inside` rule executes first, binding `$VAR` to `something`, so the subsequent `has` rule cannot match. ### Step 4: The Fix [​](https://ast-grep.github.io/blog/how-to-debug.html#step-4-the-fix) Use `all` to explicitly control the matching order: yaml id: some_sqli_rule language: python rule: pattern: $X.execute($$$) all: - has: kind: argument_list has: nthChild: 1 any: - kind: identifier pattern: $VAR - has: stopBy: end kind: identifier pattern: $VAR - inside: stopBy: end kind: module has: stopBy: end kind: assignment pattern: $VAR = $$$ By putting `has` before `inside` in the `all` array, we ensure `$VAR` is first bound to the identifier in the execute call, and then we verify that this same variable was assigned earlier. Inspecing the [playground](https://ast-grep.github.io/playground.html#eyJtb2RlIjoiQ29uZmlnIiwibGFuZyI6InB5dGhvbiIsInF1ZXJ5IjoiPERpYWxvZyAkJCQ+IiwicmV3cml0ZSI6IiIsInN0cmljdG5lc3MiOiJzbWFydCIsInNlbGVjdG9yIjoiIiwiY29uZmlnIjoiaWQ6IHNvbWVfc3FsaV9ydWxlXG5sYW5ndWFnZTogcHl0aG9uXG5ydWxlOlxuICBwYXR0ZXJuOiAkWC5leGVjdXRlKCQkJClcbiAgYWxsOlxuICAgIC0gaGFzOlxuICAgICAgICBraW5kOiBhcmd1bWVudF9saXN0XG4gICAgICAgIGhhczpcbiAgICAgICAgICBudGhDaGlsZDogMVxuICAgICAgICAgIGFueTpcbiAgICAgICAgICAgIC0ga2luZDogaWRlbnRpZmllclxuICAgICAgICAgICAgICBwYXR0ZXJuOiAkVkFSXG4gICAgICAgICAgICAtIGhhczpcbiAgICAgICAgICAgICAgICBzdG9wQnk6IGVuZFxuICAgICAgICAgICAgICAgIGtpbmQ6IGlkZW50aWZpZXJcbiAgICAgICAgICAgICAgICBwYXR0ZXJuOiAkVkFSXG4gICAgLSBpbnNpZGU6XG4gICAgICAgIHN0b3BCeTogZW5kXG4gICAgICAgIGtpbmQ6IG1vZHVsZVxuICAgICAgICBoYXM6XG4gICAgICAgICAgc3RvcEJ5OiBlbmRcbiAgICAgICAgICBraW5kOiBhc3NpZ25tZW50XG4gICAgICAgICAgcGF0dGVybjogJFZBUiA9ICQkJCIsInNvdXJjZSI6ImRlZiB0ZXN0X3NxbF9pbmplY3Rpb25fZGV0ZWN0aW9uKCk6XG4gICAgXCJcIlwiVGVzdCBjYXNlIGZvciBzdGF0aWMgYW5hbHlzaXMgdG9vbHMgZGV0ZWN0aW5nIFNRTCBpbmplY3Rpb24gdnVsbmVyYWJpbGl0aWVzXCJcIlwiXG4gICAgIyBTZXR1cCB0ZXN0IGRhdGFiYXNlXG4gICAgZGIgPSBEYXRhYmFzZU1hbmFnZXIoJzptZW1vcnk6JylcbiAgICB1c2VyX2lucHV0ID0gcmVxLnF1ZXJ5LnBhcmFtXG4gICAgdnVsbl9wYXJhbSA9IGNvbXB1dGVfYmFzZWRfb25faW5wdXQodXNlcl9pbnB1dClcbiAgICBkYi5leGVjdXRlKGZcIkRST1AgVEFCTEUgSUYgRVhJU1RTIHt2dWxuX3BhcmFtfVwiKSAgIyBWdWxuZXJhYmxlLCBidXQgbm90IGRldGVjdGVkISJ9) now shows the correct match! Example 2: The Missing Case Statement [​](https://ast-grep.github.io/blog/how-to-debug.html#example-2-the-missing-case-statement) ---------------------------------------------------------------------------------------------------------------------------------- Here's another [puzzling case](https://ast-grep.github.io/playground.html#eyJtb2RlIjoiQ29uZmlnIiwibGFuZyI6ImNwcCIsInF1ZXJ5IjoiPERpYWxvZyAkJCQ+IiwicmV3cml0ZSI6IiIsInN0cmljdG5lc3MiOiJzbWFydCIsInNlbGVjdG9yIjoiIiwiY29uZmlnIjoicnVsZTpcbiAga2luZDogY2FzZV9zdGF0ZW1lbnRcbiAgbm90OlxuICAgIGhhczpcbiAgICAgIHBhdHRlcm46IGFzc2VydCgkQSlcbiAgICAgIGhhczpcbiAgICAgICAga2luZDogJ2ZhbHNlJ1xuICAgICAgICBzdG9wQnk6IGVuZFxuICAgICAgc3RvcEJ5OiBlbmQiLCJzb3VyY2UiOiJzd2l0Y2ggKE15Y2hhcikge1xuICBbW2xpa2VseV1dICAgY2FzZSAnMSc6IHsgYXNzZXJ0KE90aGVyVmFyID4gMSk7IH1cbiAgW1t1bmxpa2VseV1dIGNhc2UgJzInOiB7IGFzc2VydChcIjJcIiAmJiBmYWxzZSk7IH1cbiAgW1t1bmxpa2VseV1dIGNhc2UgJzMnOiB7IGFzc2VydChcIjNcIiAmJiB0cnVlKTsgfVxuICBbW3VubGlrZWx5XV0gY2FzZSAnNCc6IHsgYXNzZXJ0KFwiXCIgJiYgdHJ1ZSk7IH1cbn1cbiJ9) . This rule should find `case` statements that don't contain `assert(false)`: yaml rule: kind: case_statement not: has: pattern: assert($A) has: kind: 'false' stopBy: end stopBy: end Test code: cpp switch (Mychar) { [[likely]] case '1': { assert(OtherVar > 1); } [[unlikely]] case '2': { assert("2" && false); } [[unlikely]] case '3': { assert("3" && true); } [[unlikely]] case '4': { assert("" && true); } } Expected: Match cases '1', '3', and '4' (they don't have `assert(false)`). Actual: Only cases '3' and '4' are matched. Case '1' is missing. Why? ### Step 1: Find All Case Statements [​](https://ast-grep.github.io/blog/how-to-debug.html#step-1-find-all-case-statements) First, let's see what `case_statement` nodes exist: bash ast-grep scan --inline-rules "id: all-cases language: cpp rule: kind: case_statement" test.cpp The output reveals something surprising. Each match shows the **range** of the node: * Case '1': spans lines 2-5 (includes cases 2, 3, and 4!) * Case '2': spans lines 3-5 (includes cases 3 and 4) * Case '3': spans lines 4-5 (includes case 4) * Case '4': spans line 5 only ### Step 2: Understand the AST Structure [​](https://ast-grep.github.io/blog/how-to-debug.html#step-2-understand-the-ast-structure) In C/C++ tree-sitter grammar, `case_statement` nodes are **nested**. Each case statement contains all subsequent case statements as descendants. This is how tree-sitter represents the fall-through semantics of C switch statements. case '1' node ├── { assert(OtherVar > 1); } └── case '2' node ← nested inside case '1'! ├── { assert("2" && false); } └── case '3' node ← nested inside case '2'! ├── { assert("3" && true); } └── case '4' node ← nested inside case '3'! └── { assert("" && true); } You can also use playground to visualize the AST structure. ![cpp syntax tree](https://ast-grep.github.io/image/blog/cpp-case-tree.png) ### Step 3: Identify the Problem [​](https://ast-grep.github.io/blog/how-to-debug.html#step-3-identify-the-problem) Now the issue is clear. When our rule checks case '1': 1. It looks for `kind: case_statement` ✓ 2. It checks `not: has: ... kind: 'false'` — does case '1' have a descendant with kind `false`? Since case '2' is **nested inside** case '1', and case '2' contains `assert("2" && false)`, the `false` keyword IS a descendant of case '1'! The `not: has:` condition fails because `false` exists somewhere in the subtree. Case '1' is incorrectly excluded. ### Step 4: The Fix [​](https://ast-grep.github.io/blog/how-to-debug.html#step-4-the-fix-1) To fix this, we need to restrict the search to only the **immediate body** of each case, not its nested case statements. We can use `stopBy` to stop at the next case: yaml rule: kind: case_statement not: has: pattern: assert($A) has: kind: 'false' stopBy: end stopBy: kind: case_statement By setting `stopBy: { kind: case_statement }`, the `has` search stops when it encounters another case statement, preventing it from looking into nested cases. ### The Lesson [​](https://ast-grep.github.io/blog/how-to-debug.html#the-lesson) When a rule unexpectedly fails to match: 1. Don't assume the AST matches your mental model — C/C++ case statements nest! 2. Always inspect the actual node ranges, not just the source text 3. Use `stopBy` to control how deep relational rules search Key Takeaways [​](https://ast-grep.github.io/blog/how-to-debug.html#key-takeaways) ----------------------------------------------------------------------------------- 1. **Simplify, don't complicate.** When debugging, remove code and rule conditions until you find the minimal failing case. 2. **Trust the AST, not the source.** The AST structure can surprise you. Always verify with `--debug-query=cst` or the playground. 3. **Watch meta-variable bindings.** When using the same meta-variable in multiple places, order matters. Use `all` to control matching order. 4. **Iterate systematically.** Don't guess. Remove one thing at a time, test, observe, repeat. 5. **Use the right tools.** The [online playground](https://ast-grep.github.io/playground.html) provides instant feedback and AST visualization. Use it liberally. Happy debugging! --- # Announcing the Book: Mastering ast-grep | ast-grep [Skip to content](https://ast-grep.github.io/blog/mastering-ast-grep.html#VPContent) Return to top Announcing the Book: Mastering ast-grep [​](https://ast-grep.github.io/blog/mastering-ast-grep.html#announcing-the-book-mastering-ast-grep) ============================================================================================================================================ [![mastering-ast-grep](https://ast-grep.github.io/image/blog/book-cover.png)](https://leanpub.com/ast-grep) Since the release of ast-grep, I have watched the community grow and the tool evolve. The documentation website has expanded alongside the feature set, serving as a reliable reference for command-line flags, YAML schema definitions, and rule syntax. However, as the tool has matured, I realized there was a gap between looking up how a feature works and understanding how to architect a complex code transformation system. I frequently see users asking questions about specific API design choices, misunderstanding the nuances of the interface, or having a hard time wiring complex rules together effectively. These common hurdles highlighted the need for a place to articulate the tool's design principles and provide a thorough, end-to-end explanation of how the system functions as a whole. Today, I am proud to announce the release of [**Mastering ast-grep**](https://leanpub.com/ast-grep) , a book designed to bridge that gap. From Reference to Narrative [​](https://ast-grep.github.io/blog/mastering-ast-grep.html#from-reference-to-narrative) --------------------------------------------------------------------------------------------------------------------- The documentation is non-linear; you jump to the section you need. Mastering ast-grep, conversely, is built on a storyline. It is a technical tutorial that progresses systematically from foundational concepts to advanced applications. While the official documentation answers the "what," it often lacks the space to fully explore the "how" and the "why." The book guides you through three distinct phases of mastery: 1. **Foundational Mental Models:** We start by dismantling the assumption that text search is sufficient for code. We explore _why_ Abstract Syntax Trees (ASTs) are the correct abstraction for code manipulation and how the underlying parsers actually see your source files. 2. **Practical Application:** We move quickly into pattern-based searching, covering the core syntax (`$VAR`, `$$$`), rule composition, and the "smart" matching algorithm that makes ast-grep feel intuitive. 3. **Architectural Integration:** Finally, we tackle the complex reality of production environments. This includes project configuration with `sgconfig.yml`, testing frameworks, and integration with editors via the Language Server Protocol (LSP). The "Why" Behind the Design [​](https://ast-grep.github.io/blog/mastering-ast-grep.html#the-why-behind-the-design) ------------------------------------------------------------------------------------------------------------------- One of the main motivations for writing this book was the freedom to explore topics that feel out of place in terse documentation. In the book, I have the room to dive deep into the engineering decisions that power the tool. The book explores the specific trade-offs of the regex engine used (automata-based vs. backtracking) and why that matters for preventing ReDoS attacks in CI pipelines. We discuss the nuances of UTF-8 encoding in source parsing. We dissect the strictness levels—`cst`, `smart`, `ast`, and `signature`—not just as configuration options, but as distinct strategies for balancing precision and flexibility. This context transforms the tool from a "black box" into a transparent instrument that you can reason about and predict. How the Book is Organized [​](https://ast-grep.github.io/blog/mastering-ast-grep.html#how-the-book-is-organized) ----------------------------------------------------------------------------------------------------------------- The book is structured to guide you through four distinct phases of mastery, moving from simple concepts to sophisticated engineering. We start with the _Introduction_, establishing the conceptual foundation by dismantling the assumption that text search is sufficient for code and exploring why Abstract Syntax Trees are the correct abstraction for analysis. The _Basic_ section then introduces the core mechanics of pattern-based searching, teaching you to write patterns using metavariables, compose YAML rules, apply logical operators, and perform basic code rewrites. From there, we transition to the _Intermediate_ section, where the focus shifts from writing isolated rules to building production-ready systems. This covers refining ambiguous patterns with context, configuring projects via sgconfig.yml, establishing robust testing workflows, and integrating ast-grep into your editor via the Language Server Protocol. The journey concludes with the _Advanced_ section, which explores the full depth of the tool's capabilities. We dive into strictness levels (cst, smart, ast), recursive utility rules, complex transformations using rewriters, and leveraging the programmatic API to embed structural search into your own applications. Supporting the Project [​](https://ast-grep.github.io/blog/mastering-ast-grep.html#supporting-the-project) ----------------------------------------------------------------------------------------------------------- Open source is a labor of love, but it is also labor. [**Mastering ast-grep**](https://leanpub.com/ast-grep) is available for purchase today. Buying a copy is the most direct way to support the ongoing development and maintenance of the ast-grep project financially. By purchasing the book, you are not only investing in your own capability to manipulate code at scale, but you are also ensuring the sustainability of the tool itself. Whether you are looking to automate a massive refactor, enforce strict code quality standards, or simply stop wrestling with fragile regular expressions, I believe this book provides the foundation you need. [Buy your copy of **Mastering ast-grep** today!](https://leanpub.com/ast-grep) --- # Migrating Bevy can be easier with (semi-)automation | ast-grep [Skip to content](https://ast-grep.github.io/blog/migrate-bevy.html#VPContent) Return to top Migrating Bevy can be easier with (semi-)automation [​](https://ast-grep.github.io/blog/migrate-bevy.html#migrating-bevy-can-be-easier-with-semi-automation) ============================================================================================================================================================= Using open source software can be a double-edged sword: We enjoy the latest features and innovations, but we hate frequent and sometimes tedious upgrades. Bevy is a fast and flexible game engine written in Rust. It aims to provide a modern and modular architecture, notably [Entity Component System(ECS)](https://www.wikiwand.com/en/Entity_component_system) , that allows developers to craft rich and interactive experiences. However, the shiny new engine is also an evolving project that periodically introduces breaking changes in its API. Bevy's migration guide is comprehensive, but daunting. It is sometimes overwhelmingly long because it covers many topics and scenarios. In this article, we will show you how to make migration easier by using some command line tools such as [`git`](https://git-scm.com/) , [`cargo`](https://doc.rust-lang.org/cargo/) and [`ast-grep`](https://ast-grep.github.io/) . These tools can help you track the changes, search for specific patterns in your code, and automate API migration. Hope you can migrate your Bevy projects with less hassle and more confidence by following our tips. * * * We will use the utility AI library [big-brain](https://github.com/zkat/big-brain) , the second most starred Bevy project on GitHub, as an example to illustrate bumping Bevy version from 0.9 to 0.10. Upgrading consists of four big steps: **make a clean git branch**, **updating the dependencies**, **running fix commands**, and **fixing failing tests**. And here is a list of commands used in the migration. * `git`: Manage code history, keep code snapshot, and help you revert changes if needed. * `cargo check`: Quickly check code for errors and warnings without building it. * `ast-grep`: Search for ASTs in source and automate code rewrite using patterns or expressions. * `cargo fmt`: Format the rewritten code according to Rust style guidelines. * `cargo test`: Run tests in the project and report the results to ensure the program still works. Preparation [​](https://ast-grep.github.io/blog/migrate-bevy.html#preparation) ------------------------------------------------------------------------------- Before we start, we need to make sure that we have the following tools installed: [Rust](https://rustup.rs/) , [git](https://git-scm.com/) and [ast-grep](https://ast-grep.github.io/) . Compared to the other two tools, ast-grep is lesser-known. In short it can do search and replace based on [abstract syntax trees](https://www.wikiwand.com/en/Abstract_syntax_tree) . You can install it via [`cargo`](https://crates.io/crates/ast-grep) or [`brew`](https://formulae.brew.sh/formula/ast-grep) . shell # install the binary `ast-grep` cargo install ast-grep # or use brew brew install ast-grep ### Clone [​](https://ast-grep.github.io/blog/migrate-bevy.html#clone) The first step is to clone your repository to your local machine. You can use the following command to clone the big-brain project: sh git clone git@github.com:HerringtonDarkholme/big-brain.git Note that the big-brain project is not the official repository of the game, but a fork that has not updated its dependencies yet. We use this fork for illustration purposes only. ### Check out a new branch [​](https://ast-grep.github.io/blog/migrate-bevy.html#check-out-a-new-branch) Next, you need to create a new branch for the migration. This will allow you to keep track of your changes and revert them if something goes wrong. You can use the following command to create and switch to a new branch called `upgrade-bevy`: sh git checkout -b upgrade-bevy > Key take away: make sure you have a clean git history and create a new branch for upgrading. Update Dependency [​](https://ast-grep.github.io/blog/migrate-bevy.html#update-dependency) ------------------------------------------------------------------------------------------- Now it's time for us to kick off the real migration! First big step is to update dependencies. It can be a little bit tricker than you think because of transitive dependencies. ### Update dependencies [​](https://ast-grep.github.io/blog/migrate-bevy.html#update-dependencies) Let's change the dependency file `Cargo.toml`. Luckily big-brain has clean dependencies. Here is the diff: diff diff --git a/Cargo.toml b/Cargo.toml index c495381..9e99a3b 100644 --- a/Cargo.toml +++ b/Cargo.toml @@ -14,11 +14,11 @@ homepage = "https://github.com/zkat/big-brain" [workspace] [dependencies] -bevy = { version = "0.9.0", default-features = false } +bevy = { version = "0.10.0", default-features = false } big-brain-derive = { version = "=0.16.0", path = "./derive" } [dev-dependencies] -bevy = { version = "0.9.0", default-features = true } +bevy = { version = "0.10.0", default-features = true } rand = { version = "0.8.5", features = ["small_rng"] } [features] ### Update lock-file [​](https://ast-grep.github.io/blog/migrate-bevy.html#update-lock-file) After you have updated your dependencies, you need to build a new lock-file that reflects the changes. You can do this by running the following command: bash cargo check This will check your code for errors and generate a new Cargo.lock file that contains the exact versions of your dependencies. ### Check Cargo.lock, return to step 3 if necessary [​](https://ast-grep.github.io/blog/migrate-bevy.html#check-cargo-lock-return-to-step-3-if-necessary) You should inspect your Cargo.lock file to make sure that all your dependencies are compatible and use the same version of Bevy. Bevy is [more a bazaar than a cathedral](https://www.wikiwand.com/en/The_Cathedral_and_the_Bazaar) . You may install third-party plugins and extensions from the ecosystem besides the core library. This means that some of these crates may not be updated or compatible with the latest version of Bevy or may have different dependencies themselves, causing errors or unexpected behavior in your code. If you find any inconsistencies, you can go back to step 3 and modify your dependencies accordingly. Repeat this process until your Cargo.lock file is clean and consistent. A tip here is to search `bevy 0.9` in the lock file. `Cargo.lock` will list library with different version numbers. Fortunately, Bevy is the only dependency in big-brain. So we are good to go now! > Key take away: take advantage of Cargo.lock to find transitive dependencies that need updating. (Semi-)Automate Migration [​](https://ast-grep.github.io/blog/migrate-bevy.html#semi-automate-migration) --------------------------------------------------------------------------------------------------------- ### `cargo check` and `ast-grep --rewrite` [​](https://ast-grep.github.io/blog/migrate-bevy.html#cargo-check-and-ast-grep-rewrite) We will use compiler to spot breaking changes and use AST rewrite tool to repeatedly fix these issues. This is a semi-automated process because we need to manually check the results and fix the remaining errors. The mantra here is to use automation that maximize your productivity. Write codemod that is straightforward to you and fix remaining issues by hand. 1. `CoreSet` The first error is quite easy. The compiler outputs the following error. shell error[E0432]: unresolved import `CoreStage` --> src/lib.rs:226:13 | 226 | use CoreStage::*; | ^^^^^^^^^ use of undeclared type `CoreStage` From [migration guide](https://bevyengine.org/learn/migration-guides/0.9-0.10/) : > The `CoreStage` (... more omitted) enums have been replaced with `CoreSet` (... more omitted). The same scheduling guarantees have been preserved. So we just need to change the import name. [Using ast-grep is trivial here](https://ast-grep.github.io/guide/introduction.html#introduction) . We need to provide a pattern, `-p`, for it to search as well as a rewrite string, `-r` to replace the old API with the new one. The command should be quite self-explanatory. ast-grep -p 'CoreStage' -r CoreSet -i We suggest to add `-i` flag for `--interactive` editing. ast-grep will display the changed code diff and ask your decision to accept or not. diff --- a/src/lib.rs +++ b/src/lib.rs @@ -223,7 +223,7 @@ pub struct BigBrainPlugin; impl Plugin for BigBrainPlugin { fn build(&self, app: &mut App) { - use CoreStage::*; + use CoreSet::*; 2. `StageLabel` Our next error is also easy-peasy. error: cannot find derive macro `StageLabel` in this scope --> src/lib.rs:269:45 | 269 | #[derive(Clone, Debug, Hash, Eq, PartialEq, StageLabel, Reflect)] | The [doc](https://bevyengine.org/learn/migration-guides/0.9-0.10/#label-types) : > System labels have been renamed to systems sets and unified with stage labels. The `StageLabel` trait should be replaced by a system set, using the `SystemSet` trait as dicussed immediately below. The command: bash ast-grep -p 'StageLabel' -r SystemSet -i 3. `SystemStage` The next error is much harder. First, the error complains two breaking changes. error[E0599]: no method named `add_stage_after` found for mutable reference `&mut bevy::prelude::App` in the current scope --> src/lib.rs:228:13 | ↓↓↓↓↓↓↓↓↓↓↓ use of undeclared type `SystemStage` 228 | app.add_stage_after(First, BigBrainStage::Scorers, SystemStage::parallel()); | ^^^^^^^^^^^^^^^ help: there is a method with a similar name: `add_state` Let's see what [migration guide](https://bevyengine.org/learn/migration-guides/0.9-0.10/#stages) said. This time we will give the code example. // before app.add_stage_after(CoreStage::Update, AfterUpdate, SystemStage::parallel()); // after app.configure_set( AfterUpdate .after(CoreSet::UpdateFlush) .before(CoreSet::PostUpdate), ); `add_stage_after` is removed and `SystemStage` is renamed. We should use `configure_set` and `before`/`after` methods. Let's write a command for this code migration. bash ast-grep \ -p '$APP.add_stage_after($STAGE, $OWN_STAGE, SystemStage::parallel())' \ -r '$APP.configure_set($OWN_STAGE.after($STAGE))' -i This pattern deserves some explanation. `$STAGE` and `$OWN_STAGE` are [meta-variables](https://ast-grep.github.io/guide/pattern-syntax.html#meta-variable) . meta-variable is a wildcard expression that can match any single AST node. So we effectively find all `add_stage_after` call. We can also use meta-variables in the rewrite string and ast-grep will replace them with the captured AST nodes. ast-grep's meta-variables are very similar to regular expression's dot `.`, except they are not textual. However, I found some `add_stage_after`s are not replaced. Nah, ast-grep is [quite dumb](https://github.com/ast-grep/ast-grep/issues/374) that it cannot handle the optional comma after the last argument. So I used another query with a trailing comma. shell ast-grep \ -p 'app.add_stage_after($STAGE, $OWN_STAGE, SystemStage::parallel(),)' \ -r 'app.configure_set($OWN_STAGE.after($STAGE))' -i Cool! Now it replaced all `add_stage_after` calls! diff --- a/src/lib.rs +++ b/src/lib.rs @@ -225,7 +225,7 @@ impl Plugin for BigBrainPlugin { - app.add_stage_after(First, BigBrainStage::Scorers, SystemStage::parallel()); + app.configure_set(BigBrainStage::Scorers.after(First)); @@ -245,7 +245,7 @@ impl Plugin for BigBrainPlugin { - app.add_stage_after(PreUpdate, BigBrainStage::Actions, SystemStage::parallel()); + app.configure_set(BigBrainStage::Actions.after(PreUpdate)); @@ -253,7 +253,7 @@ impl Plugin for BigBrainPlugin { - app.add_stage_after(Last, BigBrainStage::Cleanup, SystemStage::parallel()); + app.configure_set(BigBrainStage::Cleanup.after(Last)); 4. `Stage` Our next error is about [`add_system_to_stage`](https://bevyengine.org/learn/migration-guides/0.9-0.10/#stages) . The migration guide told us: rust // Before: app.add_system_to_stage(CoreStage::PostUpdate, my_system) // After: app.add_system(my_system.in_base_set(CoreSet::PostUpdate)) Let's also write a pattern for it. sh ast-grep \ -p '$APP.add_system_to_stage($STAGE, $SYS)' \ -r '$APP.add_system($SYS.in_base_set($STAGE))' -i Example diff: diff --- a/src/lib.rs +++ b/src/lib.rs @@ -243,7 +243,7 @@ impl Plugin for BigBrainPlugin { - app.add_system_to_stage(BigBrainStage::Thinkers, thinker::thinker_system); + app.add_system(thinker::thinker_system.in_base_set(BigBrainStage::Thinkers)); 5. `system_sets` The next error corresponds to the system\_sets in [migration guide](https://bevyengine.org/learn/migration-guides/0.9-0.10/#system-sets-bevy-0-9) . // Before: app.add_system_set( SystemSet::new() .with_system(a) .with_system(b) .with_run_criteria(my_run_criteria) ); // After: app.add_systems((a, b).run_if(my_run_condition)); We need to change `SystemSet::new().with_system(a).with_system(b)` to `(a, b)`. Alas, I don't know how to write a pattern to fix that. Maybe ast-grep is not strong enough to support this. I just change `with_system` manually. _It is still faster than me scratching my head about how to automate everything._ Another change is to use `add_systems` instead of `add_system_set`. This is a simple pattern! sh ast-grep \ -p '$APP.add_system_set_to_stage($STAGE, $SYS,)' \ -r '$APP.add_systems($SYS.in_set($STAGE))' -i This should fix `system_sets`! 6. Last error Our last error is about `in_base_set`'s type. shell error[E0277]: the trait bound `BigBrainStage: BaseSystemSet` is not satisfied --> src/lib.rs:238:60 | 238 | app.add_system(thinker::thinker_system.in_base_set(BigBrainStage::Thinkers)); | ----------- ^^^^^^^^^^^^^^^^^^^^^^^ the trait `BaseSystemSet` is not implemented for `BigBrainStage` | | | required by a bound introduced by this call | = help: the following other types implement trait `BaseSystemSet`: StartupSet bevy::prelude::CoreSet note: required by a bound in `bevy::prelude::IntoSystemConfig::in_base_set` Okay, `BigBrainStage::Thinkers` is not a base set in Bevy, so we should change it to `in_set`. diff - .add_system(one_off_action_system.in_base_set(BigBrainStage::Actions)) + .add_system(one_off_action_system.in_set(BigBrainStage::Actions)) **Hoooray! Finally the program compiles! ~ship it!~ Now let's test it.** > Key take away: Automation saves your time! But you don't have to automate everything. cargo fmt [​](https://ast-grep.github.io/blog/migrate-bevy.html#cargo-fmt) --------------------------------------------------------------------------- Congrats! You have automated code refactoring! But ast-grep's rewrite can be messy and hard to read. Most code-rewriting tool does not support pretty-print, sadly. A simple solution is to run `cargo fmt` and make the repository neat and tidy. cargo fmt A good practice is to run this command every time after a code rewrite. > Key take away: Format code rewrite as much as you want. Test Our Refactor [​](https://ast-grep.github.io/blog/migrate-bevy.html#test-our-refactor) ------------------------------------------------------------------------------------------- ### `cargo test` [​](https://ast-grep.github.io/blog/migrate-bevy.html#cargo-test) Let's use Rust's standard test command to verify our changes: `cargo test`. Oops. we have one test error, not too bad! running 1 test test steps ... FAILED failures: ---- steps stdout ---- steps test thread 'steps' panicked at '`"Update"` and `"Cleanup"` have a `before`-`after` relationship (which may be transitive) but share systems.' note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace Okay, it complains that `Update` and `Cleanup` have a conflicting running order. This is probably caused by `configure_set`. I should have caught the bug during diff review but I missed that. It is not too late to change it manually. diff --- a/src/lib.rs +++ b/src/lib.rs @@ -225,7 +225,7 @@ impl Plugin for BigBrainPlugin { - app.configure_set(BigBrainStage::Scorers.after(First)); + app.configure_set(BigBrainStage::Scorers.in_base_set(First)); @@ -242,12 +242,12 @@ impl Plugin for BigBrainPlugin { - app.configure_set(BigBrainStage::Actions.after(PreUpdate)); + app.configure_set(BigBrainStage::Actions.in_base_set(PreUpdate)); Run `cargo test` again? Doc-tests big-brain failures: ---- src/lib.rs - (line 127) stdout ---- error[E0599]: no method named `add_system_to_stage` found for mutable reference `&mut bevy::prelude::App` in the current scope We failed doc-test! Because our ast based tool does not process comments. Lame. 😦 We need manually fix them. --- a/src/lib.rs +++ b/src/lib.rs @@ -137,8 +137,8 @@ -//! .add_system_to_stage(BigBrainStage::Actions, drink_action_system) -//! .add_system_to_stage(BigBrainStage::Scorers, thirsty_scorer_system) +//! .add_system(drink_action_system.in_set(BigBrainStage::Actions)) +//! .add_system(thirsty_scorer_system.in_set(BigBrainStage::Scorers)) **Finally we passed all tests!** test result: ok. 21 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 4.68s Conclusion [​](https://ast-grep.github.io/blog/migrate-bevy.html#conclusion) ----------------------------------------------------------------------------- Now we can commit and push our version upgrade to the upstream. It is not a too long battle, is it? I have created a pull request for reference. [https://github.com/HerringtonDarkholme/big-brain/pull/1/files](https://github.com/HerringtonDarkholme/big-brain/pull/1/files) Reading a long migration guide is not easy, and fixing compiler errors is even harder. It would be nice if the official guide can contain some automated command to ease the burden. For example, [yew.rs](https://yew.rs/docs/next/migration-guides/yew/from-0_20_0-to-next) did a great job by providing automation in every release note! To recap our semi-automated refactoring, this is our four steps: * Keep a clean git branch for upgrading * Update all dependencies in the project and check lock files. * Compile, Rewrite, Verify and Format. Repeat this process until the project compiles. * Run Test and fix the remaining bugs. I hope this workflow will help you and other programming language developers in the future! --- # ast-grep Gets More LLM Support! | ast-grep [Skip to content](https://ast-grep.github.io/blog/more-llm-support.html#VPContent) Return to top ast-grep Gets More LLM Support! [​](https://ast-grep.github.io/blog/more-llm-support.html#ast-grep-gets-more-llm-support) ========================================================================================================================== Leveling Up Code Analysis with AI [​](https://ast-grep.github.io/blog/more-llm-support.html#leveling-up-code-analysis-with-ai) ------------------------------------------------------------------------------------------------------------------------------- ast-grep, the powerful tool for structural code search, is getting even better with enhanced Large Language Model (LLM) support. This exciting development opens up new possibilities for developers to analyze, understand, and transform code more efficiently. Let's dive into the details of these new features. `llms.txt` Support [​](https://ast-grep.github.io/blog/more-llm-support.html#llms-txt-support) ----------------------------------------------------------------------------------------------- ast-grep now supports a new file format, [`llms.txt`](https://llmstxt.org/) , designed to work seamlessly with LLMs. It is also on [llmstxthub](https://llmstxthub.com/websites/ast-grep) for easy access to the latest files. A key challenge for large language models is their limited ability to process extensive website content. They often struggle with the complexity of converting full HTML pages, which include navigation, advertisements, and JavaScript, into a simplified text format that LLMs can effectively use. On the other hand, ast-grep faces challenges due to the limited training data available for LLMs. Because of this, LLMs often confuse ast-grep with other similar tools, even when provided with accurate prompts. Furthermore, despite ast-grep's comprehensive online documentation, LLM search capabilities don't guarantee accurate retrieval of information on rule writing. This hinders ast-grep's widespread adoption in the AI era. `llms.txt` addresses this by providing models with comprehensive context, enhancing their [in-context learning](https://arxiv.org/abs/2301.00234) and improving the accuracy of their output. It is particularly effective with models that have large context windows, such as [Google’s Gemini](https://aistudio.google.com/) . ![Example Usage with $GOOG Gemini](https://ast-grep.github.io/image/blog/gemini.jpeg) The general usage of `llms.txt` is as follows: 1. Visit [https://ast-grep.github.io/llms-full.txt](https://ast-grep.github.io/llms-full.txt) and copy the full documentation text 2. Paste these documents into your conversation with your preferred AI chatbot 3. Ask AI questions about ast-grep AI-powered Codemod Studio [​](https://ast-grep.github.io/blog/more-llm-support.html#ai-powered-codemod-studio) --------------------------------------------------------------------------------------------------------------- [Codemod.com](https://codemod.com/) is a long-time [contributor](https://go.codemod.com/ast-grep-contributions) [supporter](https://github.com/ast-grep/ast-grep?tab=readme-ov-file#sponsor) of ast-grep and has recently introduced a new feature called [Codemod Studio](https://app.codemod.com/studio) . The studio introduces an AI assistant which is is a game-changer for writing ast-grep rules. This interactive environment allows you to use natural language to describe the code patterns you want to find, and then the AI will help you write the corresponding ast-grep rule. Here's how it works: * **Describe your goal**: In plain English, explain what you want to achieve with your ast-grep rule (e.g., "Find all instances of `console.log`"). * **AI assistance**: The AI analyzes your description and suggests an appropriate ast-grep pattern. * **Refine and test**: You can then refine the generated rule, test it against your codebase, and iterate until it meets your needs. This innovative approach democratizes ast-grep rule creation, making it accessible to developers of all skill levels, even without previous experience with ast-grep. GenAI Script Support [​](https://ast-grep.github.io/blog/more-llm-support.html#genai-script-support) ----------------------------------------------------------------------------------------------------- Microsoft’s GenAI Script supports [ast-grep](https://microsoft.github.io/genaiscript/reference/scripts/ast-grep/) ! > [GenAIScript](https://microsoft.github.io/genaiscript/) > is a scripting language that integrates LLMs into the scripting process using a simplified JavaScript syntax. Supported by our VS Code GenAIScript extension, it allows users to create, debug, and automate LLM-based scripts. Notably, GenAIScript provides a wrapper around `ast-grep` to search for patterns within a script's AST and transform that AST. This enables the creation of highly efficient scripts that modify source code by precisely targeting specific code elements. Upcoming MCP Support [​](https://ast-grep.github.io/blog/more-llm-support.html#upcoming-mcp-support) ----------------------------------------------------------------------------------------------------- Looking ahead, ast-grep has a plan to support [Model Context Protocol](https://modelcontextprotocol.io/) (MCP). This upcoming feature will further enhance the integration of LLMs with ast-grep, enabling even more sophisticated code analysis and transformation. MCP will provide a standardized interface for LLMs to interact with ast-grep, streamlining the process of analyzing and transforming code. Some of the key features of ast-grep MCP include: * List all ast-grep's [resources](https://modelcontextprotocol.io/docs/concepts/resources) in the project: rules, utils, and test cases. * Orchestrate the LLM's actions by providing predefined [prompts](https://modelcontextprotocol.io/docs/concepts/prompts) and workflows. * Provide [tools](https://modelcontextprotocol.io/docs/concepts/tools) to create/validate rules and search the codebase. See the tracking GitHub issue [here](https://github.com/ast-grep/ast-grep/issues/1895) Conclusion [​](https://ast-grep.github.io/blog/more-llm-support.html#conclusion) --------------------------------------------------------------------------------- ast-grep's integration of LLMs, including `llms.txt`, Codemod Studio, and GenAI Script, represents a significant leap forward in code analysis. With the promise of MCP on the horizon, ast-grep is poised to become an indispensable tool for developers seeking to harness the power of AI to understand, transform, and elevate their code. The future of code analysis is here, and it's powered by ast-grep. --- # ast-grep 0.38 is Here | ast-grep [Skip to content](https://ast-grep.github.io/blog/new-ver-38.html#VPContent) Return to top ast-grep 0.38 is Here [​](https://ast-grep.github.io/blog/new-ver-38.html#ast-grep-0-38-is-here) ================================================================================================= We're excited to announce the release of ast-grep 0.38! This version brings some fantastic new features to improve your code searching and linting experience, alongside a significant internal shift that paves the way for exciting future developments. For those new to ast-grep, it's a powerful command-line tool that lets you search and rewrite code based on its structure (Abstract Syntax Trees or ASTs), not just text. Think of it as `grep` or `sed`, but for code syntax! Let's dive into what's new: New Features [​](https://ast-grep.github.io/blog/new-ver-38.html#new-features) ------------------------------------------------------------------------------- ### Customizable Code Highlighting with `labels` [​](https://ast-grep.github.io/blog/new-ver-38.html#customizable-code-highlighting-with-labels) One of the exciting new additions is the `labels` field for your rule configurations. Previously, ast-grep's highlighting was pre-programmed and could not provide much context. Now, you can customize the highlighting of your matches with labels that are more meaningful and relevant to your codebase. These clearer labels also contribute to a cleaner and more intuitive user interface when viewing diagnostics. ![Example of Customizable Code Highlighting](https://ast-grep.github.io/image/blog/labels-demo.png) But the benefits don't stop at individual understanding. The labels field offers a fantastic way to embed more guidance directly into your rules, and it allow you to share coding best practices, style guide reminders, or domain-specific knowledge across your entire team. This feature helps disseminate expertise and maintain consistency effortlessly. For example, [Sam Wight](https://github.com/samwightt) , the labels feature's proposer, is using ast-grep to help his team to write better [Angular code](https://ast-grep.github.io/catalog/typescript/missing-component-decorator.html) ! ![Example of VSCode](https://ast-grep.github.io/image/blog/labels-vscode.jpeg) Furthermore, this improved diagnostic experience isn't confined to the command line. The ast-grep VSCode extension now fully respects these labels, bringing this enhanced highlighting via the Language Server Protocol (LSP). You can click on the label message in the VSCode diagnostic popup and jump to the relevant code point! ### `--json` Output Gets More Informative [​](https://ast-grep.github.io/blog/new-ver-38.html#json-output-gets-more-informative) The `--json` output option can now include rule `metadata` when you use the new `--include-metadata` flag. This is helpful for integrating ast-grep into other tools or for more detailed programmatic analysis, e.g. [SonarQube](https://github.com/ast-grep/ast-grep/issues/1987) . Tree-sitter Independence [​](https://ast-grep.github.io/blog/new-ver-38.html#tree-sitter-independence) ------------------------------------------------------------------------------------------------------- This is a significant architectural change! Previously, ast-grep was tightly coupled with tree-sitter, a fantastic parser generator tool. While tree-sitter has been foundational to ast-grep's ability to support many languages, this tight coupling had limitations. * **Introducing `SgNode`:** We've abstracted the core AST node representation with a new trait called `SgNode`. This makes ast-grep's core logic more flexible and less dependent on a single parsing technology. * **WASM Power-Up:** The ast-grep WebAssembly (WASM) module, which powers our interactive playground, now directly uses `tree-sitter-web` instead of the wrapper library [`tree-sitter-facade`](https://github.com/ast-grep/tree-sitter-wasm/tree/main/crates/tree-sitter-facade) . * **Paving the Way for the Future:** This independence opens doors for exciting new possibilities: * **Proof of Concept OXC Integration:** We're exploring [integration](https://github.com/ast-grep/ast-grep/pull/1970) with Oxc, a high-performance JavaScript/TypeScript toolchain written in Rust. Oxc boasts an extremely fast parser, which could bring significant performance benefits to ast-grep for JavaScript and TypeScript projects. * **Future SWC Integration:** Similarly, we're looking into [leveraging SWC](https://github.com/swc-project/plugins/pull/435) , another Rust-based platform for fast JavaScript/TypeScript compilation and transformation. This move is all about future-proofing ast-grep and allowing us to adopt the best parsing technologies for different languages and use cases, ultimately leading to a faster and more versatile tool for you. Breaking Changes [​](https://ast-grep.github.io/blog/new-ver-38.html#breaking-changes) --------------------------------------------------------------------------------------- ### Dropped Support for Older Linux Versions (glibc < 2.35) [​](https://ast-grep.github.io/blog/new-ver-38.html#dropped-support-for-older-linux-versions-glibc-2-35) Due to [an upgrade in the GitHub Actions build images](https://github.com/actions/runner-images/issues/11101) , ast-grep binaries are now built on Ubuntu 22.04. This means they rely on a newer version of glibc (GNU C Library). **Impact:** Pre-compiled ast-grep binaries will no longer support distributions with glibc versions older than 2.35. For example, Ubuntu 20.04, which has glibc 2.31, is no longer directly supported by our pre-built binaries. This change also impacts [@ast-grep/napi](https://www.npmjs.com/package/@ast-grep/napi) . **Alternatives:** If you are on an older Linux distribution, you can still use ast-grep by: * Building it from source. * Using package managers that might compile it for your specific distribution if available (like AUR for Arch Linux). * Consider upgrading your system to a more recent version of your Linux distribution. * Keep using ast-grep 0.37 if you don't want to upgrade your system. ### Rust Library API Breaking Changes [​](https://ast-grep.github.io/blog/new-ver-38.html#rust-library-api-breaking-changes) For users of ast-grep as a Rust library, please note the following API adjustments: * `AstGrep` is now an alias for `Root`. * Tree-sitter specific methods within the `Language` trait have been moved to a new `LanguageExt` trait. * `StrDoc` and related types have been relocated to the `ast_grep_core::tree_sitter` module. These changes are part of the larger effort to decouple ast-grep from tree-sitter and provide a cleaner, more maintainable library interface. Get Started with 0.38! [​](https://ast-grep.github.io/blog/new-ver-38.html#get-started-with-0-38) -------------------------------------------------------------------------------------------------- We believe these changes, especially the move towards parser independence and the enhanced diagnostic labeling, will make ast-grep an even more powerful and user-friendly tool for your everyday development tasks. Head over to our [GitHub repo](https://github.com/ast-grep/ast-grep) to grab the latest version. Check out the [documentation](https://ast-grep.github.io/) for more details on how to use the new features. We're excited to see how you use ast-grep 0.38! As always, your feedback is invaluable, so please don't hesitate to open issues or discussions on our GitHub repository. Happy Grepping! --- # ast-grep 0.42: The Answer to Code Searching | ast-grep [Skip to content](https://ast-grep.github.io/blog/new-ver-42.html#VPContent) Return to top ast-grep 0.42: The Answer to Code Searching [​](https://ast-grep.github.io/blog/new-ver-42.html#ast-grep-0-42-the-answer-to-code-searching) ============================================================================================================================================ After a long journey through the galaxy of AST manipulation, ast-grep has arrived at **version 0.42** — the answer to the ultimate question of code searching, linting, and rewriting. If [Douglas Adams](https://en.wikipedia.org/wiki/The_Hitchhiker%27s_Guide_to_the_Galaxy) taught us anything, it's that the answer to life, the universe, and everything is **42**. We'd like to think ast-grep 0.42 lives up to its number: this release packs powerful new features that answer some of the most requested questions from our community. Don't panic — let's dive in. Parameterized Utilities (Experimental) [​](https://ast-grep.github.io/blog/new-ver-42.html#parameterized-utilities-experimental) --------------------------------------------------------------------------------------------------------------------------------- EXPERIMENTAL FEATURE Parameterized utilities are **experimental**. The current implementation is hacky, dirty, and quick — a prototype to gather real-world feedback. The API, behavior, and semantics **may change, break, or even be removed entirely** in future releases. That said, we encourage adventurous users to try it out! Please report bugs and share your feedback at [ast-grep/ast-grep](https://github.com/ast-grep/ast-grep) . The biggest feature in this release has landed: [parameterized utilities](https://github.com/ast-grep/ast-grep/issues/1298) . [Global utility rules](https://ast-grep.github.io/guide/rule-config/utility-rule.html#global-utility-rules) in ast-grep let you define reusable rule components shared across your project, but previously they were static — you couldn't customize them for different contexts. Now, global utilities can accept arguments, making them far more flexible and reducing duplication in your rule configurations. ### The Problem [​](https://ast-grep.github.io/blog/new-ver-42.html#the-problem) Global utilities are reusable across files, but they couldn't be customized — you'd copy-paste the same structure over and over, changing only a name or a pattern. Say you want to audit logging calls that pass a string literal as an argument. One rule bans `console.log` with string literals in production code, another flags `logger.error` with hardcoded messages. They have different severities and messages, so they must be separate rules — but without parameterization, each duplicates the entire rule structure: yaml # rules/audit-logging.yml id: no-console-string language: TypeScript rule: pattern: $OBJ.$METHOD($$$ARGS) all: - has: kind: member_expression has: field: object regex: ^console$ # <--- console - has: field: arguments has: kind: string --- id: no-hardcoded-logger language: TypeScript rule: pattern: $OBJ.$METHOD($$$ARGS) all: - has: kind: member_expression has: field: object regex: ^logger$ # <--- logger - has: field: arguments has: kind: string # the entire rule body is identical — only this one line differs! On top of that, there was no way to export matched meta-variables from a global utility back to its caller. This caused inability to share captured meta-variables across files ([#1297](https://github.com/ast-grep/ast-grep/issues/1297) ), parse errors when using global util variables in `fix` ([#1766](https://github.com/ast-grep/ast-grep/issues/1766) ), and confusion about why global utils couldn't provide variables like local ones could ([#1994](https://github.com/ast-grep/ast-grep/issues/1994) ). Parameterized utilities solve both problems: they eliminate duplication and provide a well-defined interface for passing rules in and getting meta-variables back. ### The Solution [​](https://ast-grep.github.io/blog/new-ver-42.html#the-solution) Think of parameterized utilities as **functions for your rules**. You declare parameters in the utility name, and then pass arguments when you call it with `matches`: Define a parameterized [global utility](https://ast-grep.github.io/guide/rule-config/utility-rule.html#global-utility-rules) in a file under your `utils/` directory: yaml # utils/audit-log-call.yml id: audit-log-call arguments: [logger-rule] # declare parameters language: TypeScript rule: pattern: $OBJ.$METHOD($$$ARGS) all: - has: kind: member_expression has: field: object matches: logger-rule # use the parameter as a rule - has: field: arguments has: kind: string Then call it from any rule by passing arguments via `matches`. Each argument is itself a **rule**, not just a string — so you can pass patterns, regex, or any composite rule: yaml # rules/audit-logging.yml id: no-console-string language: TypeScript rule: matches: audit-log-call: logger-rule: { regex: ^console$ } # pass rule as arg --- id: no-hardcoded-logger language: TypeScript rule: matches: audit-log-call: logger-rule: { regex: ^logger$ } # pass rule as arg The entire deep rule structure is defined once in the utility. Each rule only specifies what differs — the logger object name. Crucially, meta-variables captured inside argument rules are **exported back to the caller**. This solves the long-standing problem of global utilities not being able to provide meta-variables for `fix`. Here's an example: the `named-import` utility matches an import statement and delegates source validation to an argument rule. The `ban-lodash` rule uses it to find lodash imports and rewrite them to `lodash-es`. The highlighted lines show where arguments are declared and where they are used as `matches` targets inside the utility body: yaml # utils/named-import.yml — reusable utility for matching named imports id: named-import arguments: [source-rule, binding-rule] # declare two param rule language: TypeScript rule: pattern: import { $BINDING } from "$SOURCE" all: - has: field: source has: kind: string_fragment matches: source-rule # filter source with param - has: stopBy: end kind: identifier matches: binding-rule # export $BINDING The highlighted lines below show the caller passing concrete rules as arguments. Meta-variable `$IMPORT_NAME` defined here is exported back and usable in `fix`: yaml # rules/ban-lodash.yml — rewrites lodash imports to lodash-es id: ban-lodash language: TypeScript rule: matches: named-import: source-rule: { regex: ^lodash$ } # only match lodash import binding-rule: { pattern: $IMPORT_NAME } # capture import in metavar fix: import { $IMPORT_NAME } from "lodash-es" When matching `import { map } from "lodash"`, the utility's pattern captures `$BINDING = map` and `$SOURCE = lodash` internally, in a separate environment. Note that `matches: binding-rule` is called on the AST node corresponding to `$BINDING` (the identifier `map`), so the caller's argument `pattern: $IMPORT_NAME` matches that same node and captures `$IMPORT_NAME = map` — effectively exporting `$BINDING` under a new name. The utility's own meta-variables stay **isolated**. Only meta-variables from **argument rules** are exported to the caller: | Meta-variable | Captured by | Visible in `ban-lodash`? | | --- | --- | --- | | `$IMPORT_NAME` | argument rule (`binding-rule`) | **Yes** — available in `fix`, `constraints`, etc. | | `$BINDING` | utility's internal pattern | **No** — isolated inside the utility | | `$SOURCE` | utility's internal pattern | **No** — isolated inside the utility | `$IMPORT_NAME` is available in `fix` because it was captured by a caller-supplied argument rule. Without parameterized utilities, there was no mechanism to get any meta-variables out of a global utility at all. MetaVar Scoping Rule The rule of thumb here is that a meta-variable like `$IMPORT_NAME` is only available if it appears **in the same YAML file** where you define it. Meta-variables defined in a different file (like `$BINDING`) are never visible to the caller. If you can't see the `$` definition in your rule file, you can't use it. ### Key Usage Rules [​](https://ast-grep.github.io/blog/new-ver-42.html#key-usage-rules) * **Only [global utility rules](https://ast-grep.github.io/guide/rule-config/utility-rule.html#global-utility-rules) ** (separate YAML files in the `utils/` directory with `id`, `language`, and `rule`) can declare parameters. Local `utils:` entries in rule config files remain zero-argument helpers. * **All declared arguments must be provided** at the call site — no optional parameters. * **Arguments are rules**, not strings. Each argument value is a full ast-grep rule object. * **Meta-variable isolation**: argument rules match in their own isolated scope. They don't read or write the caller's meta-variables during matching — exports happen only after the entire parameterized rule matches successfully. ### Advanced: How It Works Under the Hood [​](https://ast-grep.github.io/blog/new-ver-42.html#advanced-how-it-works-under-the-hood) Implementation details for the curious Parameterized utilities are implemented with **runtime binding frames** rather than template expansion. The binding frames are stored in a thread-local variable and use `unsafe` code internally — the implementation is pragmatic rather than polished, which is part of why this feature is marked experimental. Performance may be suboptimal, especially with deeply nested parameterized calls. When a parameterized rule is called: 1. The `matches` reference pushes `name -> rule` bindings into a thread-local frame. 2. The stored rule body is matched directly against the target code. 3. When a bare `matches: PARAM` is encountered inside the body, it looks up the binding frame and matches the bound rule. **Name resolution** for bare `matches: NAME` follows lexical scoping: 1. Current parameter binding (innermost scope) 2. Local utility 3. Global zero-argument rule A parameter name shadows any same-named local or global utility. **Meta-variable isolation** is a deliberate design choice. Argument rules match in a temporary, isolated `MetaVarEnv`. Any meta-variables they define are accumulated and exported back to the caller _only after the entire parameterized rule matches_. If exporting conflicts with the caller's existing bindings (e.g., the caller already bound `$A` to a different value), the whole parameterized call fails. Importantly, this failure does **not** trigger backtracking inside the parameterized rule — an `any` branch won't be retried just because a late export failed. **Kind inference** is conservative. Internally, ast-grep computes a set of `potential_kinds` for each rule as a performance optimization — if a rule can only ever match `call_expression` nodes, ast-grep skips all other node kinds entirely. However, when kind inference reaches a `matches: PARAM` reference, it cannot know what kinds the caller will pass, so it returns `None` (meaning "any kind is possible"). This disables kind-based pruning for that rule. If you need precise pruning, add an explicit `kind` guard in the utility body or at the call site. **Cycle detection** remains syntactic. ast-grep detects circular dependencies between utility rules at parse time — if rule A `matches` rule B and rule B `matches` rule A, ast-grep will report an error before any scanning happens, helping you catch buggy rules early. For parameterized utilities, parameter names are excluded from dependency edges during topological sorting. A utility cannot call itself through its argument rules, either directly or transitively. More ESQuery-Style Selectors [​](https://ast-grep.github.io/blog/new-ver-42.html#more-esquery-style-selectors) --------------------------------------------------------------------------------------------------------------- In ast-grep 0.39, we introduced ESQuery-style `kind` selectors with combinators like `>`, `+`, and `~`. In 0.42, we're expanding this with **new pseudo-selectors** that bring even more expressive power to your queries. See the [tracking issue](https://github.com/ast-grep/ast-grep/issues/2127) for the full ESQuery roadmap. ### `:has(selector)` [​](https://ast-grep.github.io/blog/new-ver-42.html#has-selector) Select nodes that contain descendants matching a given selector. `:has` also supports the `>` combinator to match only direct children. :has - descendant:has(>) - direct child yaml kind: 'function_declaration:has(return_statement)' # is equivalent to kind: function_declaration has: kind: return_statement stopBy: end yaml kind: 'call_expression:has(> identifier)' # is equivalent to kind: call_expression has: kind: identifier ### `:not(selector)` [​](https://ast-grep.github.io/blog/new-ver-42.html#not-selector) Exclude nodes matching a selector. Perfect for filtering out unwanted matches. :not yaml # match expression statements that don't contain a call kind: 'expression_statement:not(:has(> call_expression))' # is equivalent to kind: expression_statement not: has: kind: call_expression ### `:is(selector, moreSelector, ...)` [​](https://ast-grep.github.io/blog/new-ver-42.html#is-selector-moreselector) Match nodes against any one of several selectors — a concise way to express "or" logic. Previously, the comma operator (e.g. `function_declaration, arrow_function`) could only be used at the top level of a selector. `:is` lifts that restriction: you can now express "or" anywhere inside a compound selector. :is yaml # match return statements or variable declarations inside a block kind: 'statement_block > :is(return_statement, lexical_declaration)' # is equivalent to any: - kind: return_statement - kind: lexical_declaration inside: kind: statement_block ### `:nth-child(An+B)` and `:nth-child(An+B of selector)` [​](https://ast-grep.github.io/blog/new-ver-42.html#nth-child-an-b-and-nth-child-an-b-of-selector) Select nodes by their position among siblings, using the familiar `An+B` syntax. You can even combine it with an `of selector` clause to filter which siblings count. Both `An+B` and `An+B of selector` are supported. These are equivalent to ast-grep's [`nthChild` rule](https://ast-grep.github.io/reference/rule.html#nthchild) : :nth-child:nth-child of selector yaml # match odd-positioned numbers: [①, 2, ③, 4, ⑤] kind: 'array > number:nth-child(2n+1)' # is equivalent to kind: number nthChild: 2n+1 inside: kind: array yaml # match the first number, skipping non-numbers: [a, ①, 2, 3] kind: 'array > :nth-child(1 of number)' # is equivalent to nthChild: position: 1 ofRule: kind: number inside: kind: array These pseudo-selectors compose naturally with the existing combinators. Together, they bring ast-grep's ESQuery support much closer to a full selector system, making complex structural queries concise and readable. LSP: Diagnostics for Injected Languages [​](https://ast-grep.github.io/blog/new-ver-42.html#lsp-diagnostics-for-injected-languages) ------------------------------------------------------------------------------------------------------------------------------------ A subtle but important fix: the ast-grep Language Server now correctly [scans injected languages for diagnostics](https://github.com/ast-grep/ast-grep/issues/2522) . Injected languages are code embedded within other code — for example, Sassdoc comments inside SCSS files, or SQL within template strings. The ast-grep CLI has supported scanning these injected sections for a while, but the LSP wasn't reporting diagnostics for them. This created a frustrating inconsistency: rules that worked perfectly on the command line would show no results in your editor. With this fix, your editor integration (VSCode, Zed, Neovim, etc.) now surfaces diagnostics for injected language rules, just like the CLI does. No more switching to the terminal to catch violations in embedded code. Next Steps [​](https://ast-grep.github.io/blog/new-ver-42.html#next-steps) --------------------------------------------------------------------------- As the Hitchhiker's Guide reminds us, the hard part was never finding the answer — it was knowing the right question to ask. We hope ast-grep 0.42 helps you ask better questions about your code. Thanks for reading! If you are interested in the new features, please try them out and let us know your feedback. Happy Grepping, and don't forget your towel! 🚀 --- # ast-grep 0.39 is Here | ast-grep [Skip to content](https://ast-grep.github.io/blog/new-ver-39.html#VPContent) Return to top ast-grep 0.39 is Here [​](https://ast-grep.github.io/blog/new-ver-39.html#ast-grep-0-39-is-here) ================================================================================================= ast-grep 0.39 is out! This release includes new languages support, better file config and esquery style. Esquery Style Kind [​](https://ast-grep.github.io/blog/new-ver-39.html#esquery-style-kind) ------------------------------------------------------------------------------------------- ast-grep now supports [ESQuery style](https://github.com/estools/esquery) kind in the `kind` field of the rule configuration. This allows you to write more concise rule in ast-grep. Under the hood, it is equivalent to [relational rules](https://ast-grep.github.io/guide/rule-config/relational-rule.html) like `has`. ESQuery is a library for querying the AST using a [CSS style selector](https://developer.mozilla.org/en-US/docs/Learn_web_development/Core/Styling_basics/Basic_selectors) system. For example, you can write a rule to match all `identifier` nodes that are direct children of `call_expression` nodes like this: yaml kind: call_expression > identifier This is equivalent to the following relational rule: yaml kind: identifier inside: kind: call_expression Currently, ast-grep's ESQuery style `kind` only supports the following selectors: * node kind: `identifier` * `>`: direct child selectors * : descendant selector * `+`: next sibling selector * `~`: following sibling selector The corresponding relational rules are: direct childdirect childnext siblingnext sibling yaml kind: call_expression > identifier # is equivalent to kind: identifier inside: kind: call_expression yaml kind: call_expression identifier # is equivalent to kind: identifier inside: kind: call_expression stopBy: end # note the stopBy yaml kind: decorator + method_definition # is equivalent to kind: method_definition follows: kind: decorator yaml kind: decorator ~ method_definition # is equivalent to kind: method_definition follows: kind: decorator stopBy: end # note the stopBy If you want to use more ESQuery selectors, please file your use cases in [this ast-grep issue](https://github.com/ast-grep/ast-grep/issues/2127) . New Languages Support [​](https://ast-grep.github.io/blog/new-ver-39.html#new-languages-support) ------------------------------------------------------------------------------------------------- ast-grep 0.39 adds support for the following languages: * [Nix](https://nix.dev/tutorials/nix-language.html) a domain-specific, purely functional, lazily evaluated, dynamically typed programming languages for Nixpkgs and NixOS. * [Solidity](https://soliditylang.org/) A statically-typed curly-braces programming language designed for developing smart contracts that run on Ethereum. ($ETH bull run incoming? 🐂) `file` in rule config is relative to the project config file [​](https://ast-grep.github.io/blog/new-ver-39.html#file-in-rule-config-is-relative-to-the-project-config-file) ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Previously, the files section appears to be treated as relative to the current working directory from which ast-grep was invoked rather than the dir containing the [sgconfig.yml](https://ast-grep.github.io/reference/sgconfig.html) file. This has been changed in 0.39, so now the `files` section is relative to the project config file. NAPI-RS version bump! [​](https://ast-grep.github.io/blog/new-ver-39.html#napi-rs-version-bump) ------------------------------------------------------------------------------------------------ NAPI-RS recently released its [version 3](https://napi.rs/blog/announce-v3) . ast-grep [followed the release](https://github.com/ast-grep/ast-grep/pull/2108) and the result is amazing. `ThreadSafeFunction`'s implementation complexity is greatly reduced. Though this does not impact ast-grep-napi users' experience or code. This section is a tribute to NAPI-RS and its creator, [Brooklyn](https://github.com/Brooooooklyn) . Next Steps [​](https://ast-grep.github.io/blog/new-ver-39.html#next-steps) --------------------------------------------------------------------------- Thanks for reading! If you are interested in the new features, please try them out and let us know your feedback. --- # ast-grep got 6000 stars! | ast-grep [Skip to content](https://ast-grep.github.io/blog/stars-6000.html#VPContent) Return to top ast-grep got 6000 stars! [​](https://ast-grep.github.io/blog/stars-6000.html#ast-grep-got-6000-stars) ====================================================================================================== We are thrilled to announce that [ast-grep](https://ast-grep.github.io/) , the powerful code search tool, has reached a stellar milestone of 6000 stars on GitHub! This is a testament to the community's trust in our tool and the continuous improvements we've made. Let's dive into the latest features and enhancements that make ast-grep the go-to tool for developers worldwide. ![ast-grep 6k stars](https://ast-grep.github.io/image/blog/stars-6k.png) Feature Enhancements [​](https://ast-grep.github.io/blog/stars-6000.html#feature-enhancements) ----------------------------------------------------------------------------------------------- * **Rewriters Addition**: We've added support for rewriters [#855](https://github.com/ast-grep/ast-grep/pull/855) , enabling complex code transformations and refactoring with ease. The new feature unlocks a novel functional programming like code rewrite scheme: [find and patch](https://ast-grep.github.io/advanced/find-n-patch.html) . Check out our previous [blog post](https://dev.to/herrington_darkholme/find-patch-a-novel-functional-programming-like-code-rewrite-scheme-3964) for more details. ![rewriter](https://ast-grep.github.io/image/blog/rewriter.png) * **Error/Warning Suppression Support**: The new feature [#446](https://github.com/ast-grep/ast-grep/pull/446) allows users to suppress specific errors or warnings via the [code comment](https://ast-grep.github.io/guide/project/lint-rule.html#suppress-linting-error) `ast-grep-ignore`. ast-grep also [respects suppression comments](https://github.com/ast-grep/ast-grep/issues/1019) in Language Server Protocol (LSP), making it easier to manage warnings and errors in your codebase. * **Enhanced Rule Constraints**: The ast-grep rule `constraints` previously only accepted `pattern`, `kind` and `regex`. Now it accepts a full rule [#855](https://github.com/ast-grep/ast-grep/pull/855) , providing more flexibility than ever before. VSCode extension [​](https://ast-grep.github.io/blog/stars-6000.html#vscode-extension) --------------------------------------------------------------------------------------- The [ast-grep VSCode extension](https://marketplace.visualstudio.com/items?itemName=ast-grep.ast-grep-vscode) is an official [VSCode integration](https://ast-grep.github.io/guide/tools/editors.html) for this CLI tool. It unleashes the power of structural search and replace (SSR) directly into your editor. ### Notable Features [​](https://ast-grep.github.io/blog/stars-6000.html#notable-features) * **Search**: Find code patterns with syntax tree. * **Replace**: Refactor code with pattern. * **Diagnose**: Identify issues via ast-grep rule. Performance Boost [​](https://ast-grep.github.io/blog/stars-6000.html#performance-boost) ----------------------------------------------------------------------------------------- * **Parallel Thread Output Fix**: A significant fix [#be230ca](https://github.com/ast-grep/ast-grep/commit/be230ca) ensures parallel thread outputs are now guaranteed, boosting overall performance. Architectural Evolution [​](https://ast-grep.github.io/blog/stars-6000.html#architectural-evolution) ----------------------------------------------------------------------------------------------------- * **Tree-Sitter Version Bump**: We've upgraded to the latest tree-sitter version, enhancing parsing accuracy and speed. In future releases, we plan to leverage tree-sitter's [new Web Assembly grammar](https://zed.dev/blog/language-extensions-part-1) to support even more languages. * **Scan and Diff Merge**: The [refactor](https://github.com/ast-grep/ast-grep/commit/c78299d2902662cd98bda44f3faf3fbc88439078) combines `CombinedScan::scan` and `CombinedScan::diff` for a more streamlined process. * **Input Stream Optimization**: Now, ast-grep avoids unnecessary input stream usage when updating all rules [#943](https://github.com/ast-grep/ast-grep/pull/943) , making it possible to use `ast-grep scan --update-all`. Usability Improvements [​](https://ast-grep.github.io/blog/stars-6000.html#usability-improvements) --------------------------------------------------------------------------------------------------- * **Error Messaging for Rule File Parsing**: The VSCode extension now provides clearer error messages [#968](https://github.com/ast-grep/ast-grep/pull/968) when rule file parsing fails, making troubleshooting a breeze. * **Better Pattern Parsing**: Improved expando character replacement [#883](https://github.com/ast-grep/ast-grep/pull/883) to make pattern . * **More Permissive Patterns**: Patterns have become more permissive [#1087](https://github.com/ast-grep/ast-grep/pull/1087) that allows matching `$METAVAR` with different syntax kind. Enhanced Error Reporting [​](https://ast-grep.github.io/blog/stars-6000.html#enhanced-error-reporting) ------------------------------------------------------------------------------------------------------- We've introduced a suite of features to improve error reporting, making it easier to debug and refine your code: * Report undefined meta-variables, errors in fixes, unused rewriters, and undefined utility rules. * Add field ID errors for relational rules and optimize test updates to avoid erroneous reports. * Shift from reporting file counts to error counts for a more meaningful insight into code quality. ![error report](https://ast-grep.github.io/image/blog/error-report.png) Language Support Expansion [​](https://ast-grep.github.io/blog/stars-6000.html#language-support-expansion) ----------------------------------------------------------------------------------------------------------- * **Haskell Support**: Haskell enthusiasts rejoice! ast-grep now supports Haskell via tree-sitter-haskell [#1128](https://github.com/ast-grep/ast-grep/pull/1128) , broadening our language coverage. NAPI Advancements [​](https://ast-grep.github.io/blog/stars-6000.html#napi-advancements) ----------------------------------------------------------------------------------------- * **NAPI Linux x64 musl Support**: Our latest feat in NAPI [#c4d7902](https://github.com/ast-grep/ast-grep/commit/c4d7902) adds support for Linux x64 musl, ensuring wider compatibility and performance. Thanks [​](https://ast-grep.github.io/blog/stars-6000.html#thanks) ------------------------------------------------------------------- As ast-grep continues to grow, we remain committed to providing a tool that not only meets but exceeds the expectations of our diverse user base. ![sponsors](https://ast-grep.github.io/image/blog/sponsor2.png) We thank each and every one of you, especially ast-grep's sponsors, for your support, contributions, and feedback that have shaped ast-grep into what it is today. Here's to many more milestones ahead! --- # Optimize ast-grep to get 10X faster | ast-grep [Skip to content](https://ast-grep.github.io/blog/optimize-ast-grep.html#VPContent) Return to top Optimize ast-grep to get 10X faster [​](https://ast-grep.github.io/blog/optimize-ast-grep.html#optimize-ast-grep-to-get-10x-faster) ==================================================================================================================================== In this post I will discuss how to optimize the Rust CLI tool [ast-grep](https://ast-grep.github.io/) to become 10 times faster. Rust itself usually runs fast enough, but it is not a silver bullet to all performance issues. In this case, I did not pay enough attention to runtime details or opted for naive implementation for a quick prototype. And these inadvertent mistakes and deliberate slacking off became ast-grep's bottleneck. Context [​](https://ast-grep.github.io/blog/optimize-ast-grep.html#context) ============================================================================ [ast-grep](https://ast-grep.github.io/) is [my](https://github.com/HerringtonDarkholme) hobby project to help you search and rewrite code using [abstract syntax tree](https://www.wikiwand.com/en/Abstract_syntax_tree) . Conceptually, ast-grep takes a piece of pattern code (think it like a regular expression but for AST), matches the pattern against your codebase and gives a list of matched AST nodes back to you. See the [playground](https://ast-grep.github.io/playground) for a live demo. I designed ast-grep's architecture with performance in mind. Here are a few performance related highlights: * it is written in Rust, a native language compiled to machine code. * it uses the venerable C library [tree-sitter](https://tree-sitter.github.io/) to parse code, which is the same library powering [GitHub's codesearch](https://github.com/features/code-search) . * its command line interface is built upon [ignore](https://docs.rs/ignore/latest/ignore/) , the same crates used by the blazing fast [ripgrep](https://github.com/BurntSushi/ripgrep) . Okay, enough self-promotion _BS_. If it is designed to be fast, how comes this blog? Let's dive into the performance bottleneck I found in my bad code. > Spoiler. It's my bad to write slow Rust. Profiling [​](https://ast-grep.github.io/blog/optimize-ast-grep.html#profiling) ================================================================================ The first thing to optimize a program is to profile it. I am lazy this time and just uses the [flamegraph](https://github.com/flamegraph-rs/flamegraph) tool. Installing it is simple. bash cargo install flamegraph Then run it against ast-grep! No other setup is needed, compared to other profiling tools! This time I'm using an ast-grep port of [es-lint](https://github.com/ast-grep/eslint) against [TypeScript](https://github.com/microsoft/TypeScript/) 's `src` folder. This is the profiling command I used. bash sudo flamegraph -- ast-grep scan -c eslint/sgconfig.yml TypeScript/src --json > /dev/null The flamegraph looks like this. ![Before Optimzation](https://user-images.githubusercontent.com/2883231/215253646-21b5f1dd-a810-4ddb-9bbe-4938b4cc15f9.png) Optimizing the program is a matter of finding the hotspots in the flamegraph and fix them. For a more intuitive feeling about performance, I used the old command `time` to measure the wall time to run the command. The result is not good. bash time ast-grep scan -c eslint/sgconfig.yml TypeScript/src 17.63s user, 0.46s system, 167% cpu, 10.823 total The time before `user` is the actual CPU time spent on my program. The time before `total` represents the wall time. The ratio between them is the CPU utilization. In this case, it is 167%. It means my program is not fully utilizing the CPU. It only runs six rules against the codebase and it costs about 10 whole seconds! In contrast, running one ast-grep pattern agasint the TypeScript source only costs 0.5 second and the CPU utilization is decent. bash time ast-grep run -p '$A && $A()' TypeScript/src --json > /dev/null 1.96s user, 0.11s system, 329% cpu, 0.628 total Expensive Regex Cloning [​](https://ast-grep.github.io/blog/optimize-ast-grep.html#expensive-regex-cloning) ============================================================================================================ The first thing I noticed is that the `regex::Regex` type is cloned a lot. I do know it is expensive to compile a regex, but I did not expect cloning one will be the bottleneck. Much to my limited understanding, `drop`ping Regex is also expensive! Fortunately the fix is simple: I can use a reference to the regex instead of cloning it. This optimzation alone shaves about 50% of execution time. bash time ast-grep scan -c eslint/sgconfig.yml TypeScript/src --json > /dev/null 13.89s user, 0.74s system, 274% cpu 5.320 total The new flamegraph looks like this. ![Avoid Regex Cloning](https://user-images.githubusercontent.com/2883231/215318711-634a8b99-3e02-4187-9073-ea5be25d098f.png) Matching Rule can be Avoided [​](https://ast-grep.github.io/blog/optimize-ast-grep.html#matching-rule-can-be-avoided) ====================================================================================================================== The second thing I noticed is that the `match_node` function is called a lot. It is the function that matches a pattern against an AST node. ast-grep can match an AST node by rules, and those rules can be composed together into more complex rules. For example, the rule `any: [rule1, rule2]` is a composite rule that consists of two sub-rules and the composite rule matches a node when either one of the sub-rules matches the node. This can be expensive since multiple rules must be tried for every node to see if they actually make a match. I have already forsee it so every rule in ast-grep has an optimization called `potential_kinds`. AST node in tree-sitter has its own type encoded in a unsigned number called `kind`. If a rule can only match nodes with specific kinds, then we can avoid calling `match_node` for nodes if its kind is not in the `potential_kinds` set. I used a BitSet to encode the set of potential kinds. Naturally the `potential_kinds` of composite rules can be constructed by merging the `potential_kinds` of its sub-rules, according to their logic nature. For example, `any`'s potential\_kinds is the union of its sub-rules' potential\_kinds, and `all`'s potential\_kinds is the intersection of its sub-rules' potential\_kinds. Using this optimization, I can avoid calling `match_node` for nodes that can never match a rule. This optimization shaves another 40% of execution time! bash ast-grep scan -c eslint/sgconfig.yml TypeScript/src --json > /dev/null 11.57s user, 0.48s system, 330% cpu, 3.644 total The new flamegraph. ![potential_kinds trick](https://user-images.githubusercontent.com/2883231/215318794-7e3cf452-5016-4541-9c9d-1e266c1ee324.png) Duplicate Tree Traversal [​](https://ast-grep.github.io/blog/optimize-ast-grep.html#duplicate-tree-traversal) ============================================================================================================== Finally, the function call `ts_tree_cursor_child_iterator_next` caught my eyes. It meant that a lot of time was spent on traversing the AST tree. Well, I dumbly iterating through all the six rules and matching the whole AST tree for each rule. This is a lot of duplicated work! So I used a data structure to combine these rules, according to their `potential_kinds`. When I'm traversing the AST tree, I will first retrieve the rules with potential\_kinds containing the kind of the current node. Then I will only run these rules against the node. And nodes without any `potential_kinds` hit will be naturally skipped during the traversal. This is a huge optimization! The ending result is less than 1 second! And the CPU utilization is pretty good. bash ast-grep scan -c eslint/sgconfig.yml TypeScript/src --json > /dev/null 2.82s user, 0.12s system, 301% cpu, 0.975 total Conclusion [​](https://ast-grep.github.io/blog/optimize-ast-grep.html#conclusion) ================================================================================== The final flamegraph looks like this. I'm too lazy to optimize more. I'm happy with the sub-second result for now. ![Merging rules](https://user-images.githubusercontent.com/2883231/215318867-b670b5fe-c678-4c31-985f-36e4f620baeb.png) Optimizing ast-grep is a fun journey. I learned a lot about Rust and performance tuning. I hope you enjoyed this post as well. --- # ast-grep Rockets to 8000 Stars! | ast-grep [Skip to content](https://ast-grep.github.io/blog/stars-8000.html#VPContent) Return to top ast-grep Rockets to 8000 Stars! [​](https://ast-grep.github.io/blog/stars-8000.html#ast-grep-rockets-to-8000-stars) ==================================================================================================================== We are absolutely bursting with excitement to announce that ast-grep has soared past **8,000 stars** on GitHub! Every star represents a developer who sees the potential in ast-grep, and we're deeply grateful for your support. ![stars-8000](https://ast-grep.github.io/image/blog/stars-8k.jpeg) ast-grep's mission to make code searching, linting, and rewriting more accessible and powerful has truly resonated with the community. This blog post is your guide to all the fantastic updates, encompassing both the core ast-grep CLI tool and our ever-improving website. Buckle up, let's explore what's new! Expanding the Language Universe: YAML, PHP, and More! [​](https://ast-grep.github.io/blog/stars-8000.html#expanding-the-language-universe-yaml-php-and-more) ------------------------------------------------------------------------------------------------------------------------------------------------------------- ast-grep is rapidly becoming a truly polyglot code analysis powerhouse! We've significantly expanded our language support to empower you to work with even more of your codebase: **YAML Support Arrives!** YAML is the backbone of configuration for countless projects. Now, ast-grep CLI officially speaks [YAML](https://ast-grep.github.io/catalog/yaml/) , allowing you to leverage the same powerful rule system to lint, search, and even rewrite your YAML configuration files. Imagine using ast-grep rules to enforce best practices in your Kubernetes manifests or streamline your CI/CD pipelines! And yes, you can even write ast-grep rules _using YAML_ itself! **Enhanced PHP Analysis:** We've introduced a dedicated PHP language parser (`php-only-language`) for the CLI. This means more accurate and reliable analysis for your PHP code, helping you catch tricky bugs and enforce code quality standards with greater confidence. **Dynamic Languages in APIs:** Python and JavaScript API users, rejoice! You can now tap into dynamic language support within [PyO3](https://github.com/ast-grep/ast-grep/blob/main/crates/pyo3/tests/test_register_lang.py) and [napi](https://github.com/ast-grep/ast-grep/blob/main/crates/napi/__test__/custom.spec.ts) . This unlocks exciting possibilities for extending ast-grep's reach and integrating it into even more diverse and dynamic environments. **Embedded Language in HTML:** We've refined support for registering [embedded languages](https://ast-grep.github.io/advanced/language-injection.html) in the CLI, giving you even more flexibility when dealing with complex code structures like searching JavaScript/CSS in HTML. More Powerful Rules & Patterns [​](https://ast-grep.github.io/blog/stars-8000.html#more-powerful-rules-patterns) ----------------------------------------------------------------------------------------------------------------- We've been laser-focused on making the ast-grep's rule system an even more powerful and precise tool for code manipulation: **CSS inspired `nthChild` Matcher:** [nthChild](https://ast-grep.github.io/guide/rule-config/atomic-rule.html#nthchild) is a rule to find nodes based on their positions in the parent node's children list. It is heavily inspired by CSS's nth-child pseudo-class and helps you target specific nodes in a more granular way. **Pinpoint Precision with `range` Matchers:** Need to refine your rules to target a very specific section of code, even down to the character? ast-grep now supports [range](https://ast-grep.github.io/guide/rule-config/atomic-rule.html#range) matchers! You can define rules that activate only within a particular line _and_ character column range. This is useful for interacting with external tools like compilers. **Pattern with `--selector` and `--strictness` in `sg run`:** Need to fine tune your search pattern? The `--selector` and `--strictness` flag in [`sg run`](https://ast-grep.github.io/reference/cli/run.html#run-specific-options) gives you fine-grained control over pattern matching. **Simplified Suppression with `ast-grep-ignore`:** [Suppressing rules](https://ast-grep.github.io/guide/project/severity.html) just got simpler! You can now use the `ast-grep-ignore` comment directly on the same line as the code you want to exclude. Less clutter, more control. **More Robust Partial Pattern Snippet** The `ERROR` node in patterns can now match _anything_. This makes partial pattern snippet even more robust. Sharpening the Code Search CLI [​](https://ast-grep.github.io/blog/stars-8000.html#sharpening-the-code-search-cli) ------------------------------------------------------------------------------------------------------------------- **Glob Path Matching & Symbolic Link Traversal Unleashed:** CLI users can now leverage the power of [glob patterns](https://ast-grep.github.io/reference/cli/run.html#globs-globs) to specify file paths and effortlessly traverse [symbolic links](https://ast-grep.github.io/reference/cli/run.html#follow) . Navigating and analyzing your projects is now more intuitive than ever. **Rule Entity Inspection & Overwrite: Deeper Insights, More Control:** Gain insights by `--inspect` CLI flag with [semi-structured tracing output](https://ast-grep.github.io/reference/cli/scan.html#inspect-granularity) . This feature empowers advanced users with deeper debugging and customization capabilities. **Contextual Code Scanning with Before/After Flags:** Enhance your CLI scan results with surrounding code context using the new `context`, `before`, and `after` flags. Understand the bigger picture around your matches at a glance. **Know Your Impact: Fixed Rules Count:** The CLI now prints a count of fixed rules, giving you immediate feedback on the scope of your code modifications. **Debugging Supercharged:** We've significantly improved debugging with prettified pattern output, Debug AST/CST visualization, and colorized output via the [`--debug-query`](https://ast-grep.github.io/reference/cli/run.html#debug-query-format) flag. Troubleshooting and refining your rules is now a much smoother and more visual experience. Enhanced Tooling and API Experience [​](https://ast-grep.github.io/blog/stars-8000.html#enhanced-tooling-and-api-experience) ----------------------------------------------------------------------------------------------------------------------------- We're committed to providing a seamless developer experience across all of ast-grep's interfaces: **Typed `SgNode` and `SgRoot` in NAPI:** For our NAPI users, we've introduced [typed `SgNode` and `SgRoot`](https://ast-grep.github.io/blog/typed-napi.html) , significantly improving type safety and code clarity when working with the API. This enhancement is [initiated](https://github.com/ast-grep/ast-grep/pull/1661) by [mohebifar](https://github.com/mohebifar) from [Codemod](https://codemod.com/) . **Rule Config in `SgNode` Match Methods:** Flexibility at your fingertips! Rule configurations can now be [passed directly](https://github.com/ast-grep/ast-grep/pull/1730) to `SgNode` match methods like `matches`, `has`, `inside`, `follows`, and `precedes`. Configure your rules dynamically within your code. This feature is also contributed by [mohebifar](https://codemod.com/) . **New `fieldChildren`:** The new `fieldChildren` method in NAPI and PyO3 provides easier access to named children nodes, simplifying AST traversal and manipulation in your API integrations. **Powerful Code Modification in PyO3/NAPI:** Unlock advanced code modification features with Fix Related Features and Modify Edit Range in [PyO3](https://ast-grep.github.io/guide/api-usage/py-api.html#fix-code) /[NAPI](https://ast-grep.github.io/guide/api-usage/js-api.html#fix-code) . Refactoring and code transformation just got even more powerful from within your Python and JavaScript code. **Smaller, Faster NAPI Binaries:** We've reduced the NAPI binary size, resulting in smaller downloads and faster installations – get up and running with ast-grep even quicker! **Robust Python Integration:** Typings for PyO3 and strictness improvements in PyO3/YAML enhance the overall robustness and reliability of our Python integration. Website: Documentation & Interactive Exploration [​](https://ast-grep.github.io/blog/stars-8000.html#website-documentation-interactive-exploration) ---------------------------------------------------------------------------------------------------------------------------------------------------- The ast-grep website isn't just a static page; it's your interactive command center for learning, exploring, and mastering ast-grep! We've poured significant effort into expanding and refining the website to be your ultimate resource: **Documentation Deep Dive:** We've massively expanded and clarified the documentation, with deeper dives into crucial topics, clearer explanations of pattern objects, a comprehensive FAQ, and enhanced API documentation. Whether you're a beginner or an expert, you'll find valuable resources to level up your ast-grep skills. **Revamped Blog Section:** Dive into in-depth articles and latest news in the [brand-new blog section](https://ast-grep.github.io/blog.html) . Stay up-to-date with the latest ast-grep insights and learn from real-world examples. **Improved Sections & Navigation:** Finding what you need is now easier than ever with a reorganized and polished section and improved overall website navigation. **Website Stability & Polish:** We've squashed styling issues, resolved mobile responsiveness problems, fixed typing errors, and eliminated broken links to ensure a smooth and reliable browsing experience across all devices. ### Interactive Example Catalog: Learn by Doing! [​](https://ast-grep.github.io/blog/stars-8000.html#interactive-example-catalog-learn-by-doing) The [example catalog](https://ast-grep.github.io/catalog) has received a major upgrade, transforming it into an interactive learning environment: **Interactive Rule Exploration:** Dive deep into rules with interactive features like Rule Display & Extraction, MetaVar Panel, Matched Labeling, Pattern Debugger, Selector Explorer, and Pattern Configuration & Icons. Dissect rules, understand their components, and visualize how they work – all in your browser! **Effortless Rule Discovery:** Finding the right rule is now a breeze with new filters for language and sorting options. **Enhanced Usability:** Small but mighty additions like Empty Filter and Rule Counting further enhance the catalog's ease of use. See [the youtube video](https://www.youtube.com/watch?v=oNbOoBhVL8o) for a live demo. ### Playground Power-Ups: Your Online Rule Lab [​](https://ast-grep.github.io/blog/stars-8000.html#playground-power-ups-your-online-rule-lab) The online playground at [https://ast-grep.github.io/](https://ast-grep.github.io/) is now an even more powerful lab for experimenting and refining your rules: **Parser Version Visibility:** Small popups now display the tree-sitter version used in the playground, giving you valuable context for your rule testing. **Reset & Counter Enhancements:** Thanks to [@zhangmo8](https://github.com/zhangmo8) , we've added a Reset Button and a match counter to further streamline your playground workflow. **CSS Support in Playground:** The online playground now speaks CSS! Test your ast-grep rules directly on CSS code snippets. Performance Unleashed [​](https://ast-grep.github.io/blog/stars-8000.html#performance-unleashed) ------------------------------------------------------------------------------------------------- We're obsessed with speed and efficiency! Here are the performance enhancements we've delivered in the CLI: **Leaner Binaries, Faster Performance:** Optimized printer implementation has resulted in significant binary size reduction and improved overall performance. **Intelligent File Scanning:** ast-grep now only scans rule-sensitive files, dramatically improving performance for large projects. Less scanning, faster results! **`cargo binstall` for Instant Installs:** Faster installation is now a reality with `cargo binstall` support. Get pre-built binaries and get analyzing your code in record time. **Configurable Threads: Fine-Tune for Your Machine:** Fine-tune performance by configuring the number of threads ast-grep uses. Optimize ast-grep for your specific hardware and project needs. ... and of course, numerous bug fixes under the hood to ensure a smoother, more reliable experience! 🎉 Thank You - From the Bottom of Our Hearts! 🎉 [​](https://ast-grep.github.io/blog/stars-8000.html#%F0%9F%8E%89-thank-you-from-the-bottom-of-our-hearts-%F0%9F%8E%89) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------ Reaching over 8000 stars is an absolutely fantastic milestone, and it's all thanks to _you_, our incredible community! We are deeply grateful for your unwavering support, invaluable feedback, detailed bug reports, inspiring feature requests, and generous code contributions. You are the fuel that powers ast-grep's rocket! **Get Started & Get Involved Today!** * **Explore the Enhanced Website:** Dive into the wealth of resources at [https://ast-grep.github.io/](https://ast-grep.github.io/) * **Star us on GitHub!** Show your support and help us reach the next milestone: [Star the GitHub Repo](https://github.com/ast-grep/ast-grep) * **Try out the New Features & Give Feedback:** Join the conversation on [Discord](https://discord.com/invite/4YZjf6htSQ) and tell us what you think! * **Contribute Rules to the Example Catalog:** Share your expertise and help others by [contributing rules](https://github.com/ast-grep/ast-grep.github.io/tree/main/website/catalog) . * **Report Bugs & Feature Requests:** Help us make ast-grep even better by reporting issues and suggesting new features on [GitHub Issues](https://github.com/ast-grep/ast-grep/issues) We're incredibly excited to continue this journey with you! Let's keep pushing the boundaries of code searching, linting, and rewriting, making it more powerful and accessible for everyone! 🚀 ✨ --- # ast-grep: 5000 stars and beyond! | ast-grep [Skip to content](https://ast-grep.github.io/blog/stars-5000.html#VPContent) Return to top ast-grep: 5000 stars and beyond! [​](https://ast-grep.github.io/blog/stars-5000.html#ast-grep-5000-stars-and-beyond) ===================================================================================================================== We are thrilled to announce that ast-grep has reached 5000 stars on [GitHub](https://github.com/ast-grep/ast-grep) ! This is a huge milestone for our project and we are very grateful for your feedback, contributions, and encouragement. ![ast-grep star history](https://ast-grep.github.io/image/blog/stars-5k.png) Why ast-grep? [​](https://ast-grep.github.io/blog/stars-5000.html#why-ast-grep) -------------------------------------------------------------------------------- [ast-grep](https://ast-grep.github.io/) is a tool that allows you to search and transform code using abstract syntax trees (ASTs). ASTs are tree-like representations of the structure and meaning of source code. By using ASTs, ast-grep can perform more accurate and powerful operations than regular expressions or plain text search. We have introduced a lot of new features in the past few months, and we want to share them with you. We hope that you will find them useful and that they will help you write better code. What's new in ast-grep? [​](https://ast-grep.github.io/blog/stars-5000.html#what-s-new-in-ast-grep) ---------------------------------------------------------------------------------------------------- ### Core [​](https://ast-grep.github.io/blog/stars-5000.html#core) * We have redesigned and implemented a [new pattern engine](https://x.com/hd_nvim/status/1735850666235687241) inspired by [difftastic](https://github.com/Wilfred/difftastic) . Now, patterns use Rust structures to represent the syntax of code, instead of tree-sitter objects. This improves performance by minimizing tree traversal and allows for more reliable and user-friendly pattern-matching. ### CLI [​](https://ast-grep.github.io/blog/stars-5000.html#cli) * You can now use [`--inline-rules`](https://ast-grep.github.io/reference/cli/scan.html#inline-rules-rule-text) to run rules without creating any files on your disk! You can pass everything, pattern/rule/input, as a string. This is great for scripting! * [`--stdin`](https://ast-grep.github.io/reference/cli/run.html#stdin) will always wait for your input so you can match some code written in your terminal. * You can also select [custom languages](https://ast-grep.github.io/advanced/custom-language.html) in [`ast-grep new`](https://ast-grep.github.io/reference/cli/new.html) . ### Language Support [​](https://ast-grep.github.io/blog/stars-5000.html#language-support) * We have added support for three new languages: bash, php and elixir. * We have updated our language support to include golang's generic syntax and python's pattern matching syntax. * You can try out kotlin on our [playground](https://ast-grep.github.io/playground.html#eyJtb2RlIjoiUGF0Y2giLCJsYW5nIjoia290bGluIiwicXVlcnkiOiJrb3RsaW4iLCJyZXdyaXRlIjoiJEEgPz89ICRCOyIsImNvbmZpZyI6IiIsInNvdXJjZSI6ImZ1biBtaW5hbWkoKSB7XG4gICAgdmFsIGtvdGxpbiA9IFwi5Y2X44GT44Go44KK44KTXCJcbn0ifQ==) ! ### Rule [​](https://ast-grep.github.io/blog/stars-5000.html#rule) * You can now use `expandStart` and `expandEnd` to [adjust the fix range](https://ast-grep.github.io/reference/yaml/fix.html#fixconfig) selection for more precise code transformations. * You can also use [`languageGlob`](https://ast-grep.github.io/reference/sgconfig.html#languageglobs) to register alias languages for extension override, which gives you more flexibility in handling different file types. ### Node/Python API [​](https://ast-grep.github.io/blog/stars-5000.html#node-python-api) * We have added to napi a new function [parseAsync](https://github.com/ast-grep/ast-grep/blob/beb6f50e936809071e6bacae2c854aefa8e46d11/crates/napi/index.d.ts#L104-L111) , which allows you to leverage multiple cores in Node.js for faster code parsing. * We have also added [language globs](https://github.com/ast-grep/ast-grep/blob/beb6f50e936809071e6bacae2c854aefa8e46d11/crates/napi/index.d.ts#L45) to findInFiles in napi, which makes it easier to search for code patterns in non-standard files (like searching HTML in `.vue` file). * You can now use [`getTransformed`](https://github.com/ast-grep/ast-grep/blob/beb6f50e936809071e6bacae2c854aefa8e46d11/crates/napi/index.d.ts#L75) in napi to get the transformed code as a string. ### Doc [​](https://ast-grep.github.io/blog/stars-5000.html#doc) * We have improved our [napi](https://ast-grep.github.io/guide/api-usage/js-api.html) /[pyo3](https://ast-grep.github.io/guide/api-usage/py-api.html) documentation and added sandbox/colab links for you to try out ast-grep online! * We have also updated our [transformation](https://ast-grep.github.io/reference/yaml/transformation.html) and [code fix](https://ast-grep.github.io/reference/yaml/fix.html) documentation with more examples and explanations. * We have added new language examples for [go](https://ast-grep.github.io/catalog/go/) and [python](https://ast-grep.github.io/catalog/python/) , which show you how to use ast-grep with these popular languages. * We have created an ast-grep [bot](https://ast-grep.github.io/guide/introduction.html#check-out-discord-bot) on [discord](https://discord.com/invite/4YZjf6htSQ) , which can answer your questions and provide tips and tricks on using ast-grep. ### Community [​](https://ast-grep.github.io/blog/stars-5000.html#community) * We are excited to see that some awesome projects are using ast-grep for their code transformations, such as: * [vue-macro cli](https://github.com/vue-macros/vue-macros-cli) helps you migrate your Vue projects to the latest version of Vue * a new [unocss engine](https://github.com/zhiyuanzmj/transformer-attributify-jsx-sg) transforms JSX attributes into CSS classes * We are also happy to see that some innovative platforms are using ast-grep as one of their tools to help developers understand and improve their codebases, such as: * [coderabbit](https://coderabbit.ai/) uses ast-grep to help AI analyzing your code and provide insights and recommendations * [codemod](https://codemod.com/) is considering ast-grep as a new underlying tool in their code transformation studio What's next in ast-grep? [​](https://ast-grep.github.io/blog/stars-5000.html#what-s-next-in-ast-grep) ------------------------------------------------------------------------------------------------------ ### Applying sub-rules to sub-nodes [​](https://ast-grep.github.io/blog/stars-5000.html#applying-sub-rules-to-sub-nodes) Currently, ast-grep can only apply rules/transformations to the whole node that matches the pattern. This limits the flexibility and expressiveness of ast-grep, compared to other tools like [babel](https://babeljs.io/) or [libcst](https://libcst.readthedocs.io/en/latest/) . _We want to make ast-grep more powerful by allowing it to apply sub-rules to the metavariable nodes within the matching node._ This will enable ast-grep to handle more complex and diverse use cases in code transformation. For example, we can merge multiple decorators into one mega decorator in Python. This is impossible without API in the current version of ast-grep. ![screenshot of transforming python code](https://ast-grep.github.io/image/blog/subrule-demo.png) The basic workflow of ast-grep is _**"Find and Patch"**_: 1. **Find** a target node based on rule/pattern. 2. **Generate** a new string based on the matched node. 3. **Replace** the node text with the generated fix. However, this workflow does not allow us to generate different text for different sub-nodes in a rule. (This is like not being able to write `if` statements.) Nor does it allow us to apply a rule to multiple sub-nodes of a node. (This is like not being able to write `for` loops.) To overcome these limitations, we will add three new steps between step 1 and step 2: a. **Find** a list of different sub-nodes that match different sub-rules. b. **Generate** a different fix for each sub-node based on the matched sub-rule. c. **Join** the fixes together and store the string in a new metavariable for later use. The new steps are similar to the existing **_"Find and Patch"_** workflow, but with more granularity and control. This is like doing syntax tree oriented programming. We can apply different rules to different sub-nodes, just like using conditional statements. We can also apply rules to multiple sub-nodes, just like using loops. _"Find and Patch" is kind of a specialized "Functional Programming" over the AST!_ That said, applying sub-rules is an advanced feature that requires a lot of learning and practice. When in doubt, you can always use the existing [N-API](https://ast-grep.github.io/guide/api-usage/js-api.html) /[PyO3](https://ast-grep.github.io/guide/api-usage/py-api.html) workflow! Thank you! [​](https://ast-grep.github.io/blog/stars-5000.html#thank-you) -------------------------------------------------------------------------- We want to thank all the ast-grep users and supporters for your feedback, contributions, and encouragement. And we want to especially thank ast-grep's sponsors! ![ast-grep sponsors](https://ast-grep.github.io/image/blog/sponsor1.png) We hope that you enjoy the new features and improvements in ast-grep. We are always working to make ast-grep better and we look forward to hearing from you. Happy coding! --- # ast-grep got 3000 stars! | ast-grep [Skip to content](https://ast-grep.github.io/blog/stars-3000.html#VPContent) Return to top ast-grep got 3000 stars! [​](https://ast-grep.github.io/blog/stars-3000.html#ast-grep-got-3000-stars) ====================================================================================================== ![3000 stars](https://ast-grep.github.io/image/blog/star3k.png) I am very excited and thankful to share with you that ast-grep, a code search and transformation tool that I have been working on for the past year, has recently reached 3000 stars on GitHub! This is a remarkable achievement for the project and I am deeply grateful for all the support and feedback that I have received from the open source community. What is ast-grep? [​](https://ast-grep.github.io/blog/stars-3000.html#what-is-ast-grep) ---------------------------------------------------------------------------------------- [ast-grep](https://ast-grep.github.io/) is a tool that allows you to search and transform code using abstract syntax trees (ASTs). ASTs are tree-like representations of the structure and meaning of source code. By using ASTs, ast-grep can perform more accurate and powerful operations than regular expressions or plain text search. ast-grep supports multiple programming languages, such as JavaScript, [TypeScript](https://ast-grep.github.io/catalog/typescript/) , Python, [Ruby](https://ast-grep.github.io/catalog/ruby/) , Java, C#, [Rust](https://ast-grep.github.io/catalog/rust/) , and more. You can write [patterns](https://ast-grep.github.io/guide/pattern-syntax.html) and rules in [YAML](https://ast-grep.github.io/guide/rule-config/atomic-rule.html) format to specify what you want to match and how you want to transform it. You can also use the command-line interface (CLI) or the web-based [playground](https://ast-grep.github.io/playground.html) to run ast-grep on your code. Why use ast-grep? [​](https://ast-grep.github.io/blog/stars-3000.html#why-use-ast-grep) ---------------------------------------------------------------------------------------- ast-grep can help you with many tasks that involve code search and transformation, such as: * Finding and fixing bugs, vulnerabilities, or code smells * Refactoring or migrating code to a new syntax or framework * Enforcing or checking coding standards or best practices * Analyzing various code using a uniform interface > ast-grep can save you time and effort by automating repetitive or tedious tasks that would otherwise require manual editing or complex scripting. What’s new in ast-grep? [​](https://ast-grep.github.io/blog/stars-3000.html#what-s-new-in-ast-grep) ---------------------------------------------------------------------------------------------------- ast-grep is constantly evolving and improving thanks to the feedback and contributions from the users and sponsors. Here are some of the recent changes and updates of ast-grep: * ast-grep’s YAML rule now has a new `transform` rule: `conversion`, which can change matches to different cases, such as upper, lower, or camelcase. * ast-grep’s diff/rewriting now can fix multiple rules at once. See [commit](https://github.com/ast-grep/ast-grep/commit/2b301116996b7b010ed271672d35a3529fb36e56) * `ast-grep test -f`now accepts regex to selectively run ast-grep’s test case. * `ast-grep --json` supports multiple formats that powers [telescope-sg](https://github.com/Marskey/telescope-sg) , a neovim plugin that integrates ast-grep with telescope. * ast-grep now prints matches with context like `grep -A -B -C`. See [issue](https://github.com/ast-grep/ast-grep/issues/464) * JSON schema is added for better YAML rule editing. See [folder](https://github.com/ast-grep/ast-grep/tree/main/schemas) * ast-grep now has official github action setup! See [action](https://github.com/ast-grep/action) * New documentation for [rewriting code](https://ast-grep.github.io/guide/rewrite-code.html) , [example catalogs](https://ast-grep.github.io/catalog/) , and [playground](https://ast-grep.github.io/reference/playground.html) . What’s next for ast-grep? [​](https://ast-grep.github.io/blog/stars-3000.html#what-s-next-for-ast-grep) -------------------------------------------------------------------------------------------------------- ast-grep has many plans and goals for the future to make it more useful and user-friendly. Here are some of the upcoming features and enhancements of ast-grep: * Add python api support to allow users to write custom scripts using ast-grep. See [issue](https://github.com/ast-grep/ast-grep/issues/389) * Support global language config to let users specify default options for each language. See [issue](https://github.com/ast-grep/ast-grep/issues/658) * Improve napi documentation to help users understand how to use the native node module of ast-grep. See [issue](https://github.com/ast-grep/ast-grep/issues/682) * Add metavar filter to make ast-grep run more powerful by allowing users to filter matches based on metavariable values. See [issue](https://github.com/ast-grep/ast-grep/issues/379) * Add ast-grep’s pattern/rule tutorial to teach users how to write effective and efficient patterns and rules for ast-grep. See [issue](https://github.com/ast-grep/ast-grep.github.io/issues/154) * Add examples to ast-grep’s reference page to illustrate the usage and functionality of each option and feature. See [issue](https://github.com/ast-grep/ast-grep.github.io/issues/266) How to get involved? [​](https://ast-grep.github.io/blog/stars-3000.html#how-to-get-involved) ---------------------------------------------------------------------------------------------- If you are interested in ast-grep and want to try it out, you can install it from [npm](https://www.npmjs.com/package/@ast-grep/cli) or [GitHub](https://github.com/ast-grep/ast-grep) . You can also visit the [website](https://ast-grep.github.io/) to learn more about the features, documentation, and examples of ast-grep. If you want to contribute to the code or documentation of ast-grep, we have prepared a thorough [contribution guide](https://ast-grep.github.io/contributing/how-to.html) for you! You can also report issues, suggest features, or ask questions on the issue tracker. Thank you! [​](https://ast-grep.github.io/blog/stars-3000.html#thank-you) -------------------------------------------------------------------------- I hope you are as enthusiastic as I am about the progress and future of ast-grep. I sincerely value your feedback, suggestions, and contributions. Please do not hesitate to contact me if you have any questions or comments. Thank you for your wonderful support. You are making a difference in the open source community and in the lives of many developers who use ast-grep. --- # YAML vs DSL: comparison is subjective | ast-grep [Skip to content](https://ast-grep.github.io/blog/yaml-vs-dsl.html#VPContent) Return to top YAML vs DSL: comparison is subjective [​](https://ast-grep.github.io/blog/yaml-vs-dsl.html#yaml-vs-dsl-comparison-is-subjective) ================================================================================================================================= As stated in the [tool comparison](https://ast-grep.github.io/advanced/tool-comparison.html#gritql) , YAML configuration and DSL (Domain Specific Language) are two different approaches to configure rules in structural search. The question "which is better" is largely subjective. However, recently I have received some feedback that YAML is **objectively** not as good as DSL, and I would like to clarify some points. The original argument is quoted as follows: > While I see you're trying to dismiss it as a preference, I see it as a fundamental blocker. ast-grep has effectively built a DSL inside YAML. This becomes pretty apparent from your documentation, where you have to extensively explain how pattern syntax works, how metavariables work, etc.. You have to see that arguments such as "it's just YAML, no new syntax to learn" aren't entirely true either. Now IMO, if you're creating a DSL anyway, you're better off doing it properly than to go halfway. With GritQL we get syntax highlighting for all the aspects of the DSL, which I think is a significant boost. I think GritQL queries are significantly easier to read than ast-grep's mix of DSL of YAML. Direct rebuttal to the argument [​](https://ast-grep.github.io/blog/yaml-vs-dsl.html#direct-rebuttal-to-the-argument) ---------------------------------------------------------------------------------------------------------------------- ### Abstraction does not necessitate DSL [​](https://ast-grep.github.io/blog/yaml-vs-dsl.html#abstraction-does-not-necessitate-dsl) > effectively built a DSL inside YAML This is a misunderstanding of DSL and abstraction. To model Abstract Syntax Tree (AST) manipulation, you will have to have some form of concepts, such as pattern, metavariable, etc. This is true for both DSL and YAML. You cannot cut more concepts out of the tool even if you have [Occam's razor](https://en.wikipedia.org/wiki/Occam%27s_razor) . ast-grep does support pattern. It is a concept to match a strcture that contains multiple AST nodes, which makes it easier to write a rule. You can use `kind`/`has`/`all` to simulate pattern matching. But it does not mean that ast-grep should cut the concept of pattern since it looks like a DSL. In fact, ast-grep's pattern is just one of its [atomic rules](https://ast-grep.github.io/guide/rule-config/atomic-rule.html) . It does not meant to be a special embedded DSL. ### Pattern has its limitations [​](https://ast-grep.github.io/blog/yaml-vs-dsl.html#pattern-has-its-limitations) > your documentation, where you have to extensively explain how pattern syntax works, how pattern syntax works, how metavariables work As stated before, pattern makes ast-grep users' life easier. Explaining how pattern works is necessary to help users understand how to write rules. This is not a sign of a DSL being necessary, but rather a sign of the limitation of pattern: it is not general enough to cover all cases, and you have to communicate to your users how pattern works in your system. For example, see this [tweet](https://x.com/hd_nvim/status/1941876968363798766) about how to write a pattern to match `function` declation in JavaScript. Another brain teaser, how to tell if `$a = $b` is an `assignment_expression` or `field_initializer`? (an [ad-hominen](https://en.wikipedia.org/wiki/Ad_hominem) note: it is ironic to see this argument from a tool without proper documentation. I cannot suspend my suspect whether the author has used pattern to write non-trivial rules at all.) ### Slippery slope fallacy [​](https://ast-grep.github.io/blog/yaml-vs-dsl.html#slippery-slope-fallacy) > if you're creating a DSL anyway, you're better off doing it properly than to go halfway This argument is a [slippery slope fallacy](https://en.wikipedia.org/wiki/Slippery_slope) . Using pattern syntax does not mean that other rules should also be written in DSL. Again, pattern is just one of ast-grep's rules. ast-grep compose smaller rules to form more complex rules. ast-grep's rule system is not DSL based, but rather using YAML's key-value pair to represent rules. Using syntax in one rule does not imply that all rules should be combined in DSL. Using one concept in your library, framework or tool does not imply that you have to design a whole new syntax for it. The similar comparison will be frontend frameworks. Some frameworks like React and Vue use [hook](https://react.dev/reference/react/hooks) or [signals](https://dev.to/this-is-learning/the-evolution-of-signals-in-javascript-8ob) to represent state changes. But using these building blocks does not grant the verdict to design a new language for your frontend framework. Even the most avant-garde company only introduces [new syntax](https://flow.org/en/docs/react/hook-syntax/) in JavaScript, not inventing a new language. ### Subjective opinion is not objective fact [​](https://ast-grep.github.io/blog/yaml-vs-dsl.html#subjective-opinion-is-not-objective-fact) > I think GritQL queries are significantly easier to read than ast-grep's mix of DSL of YAML. This is a subjective opinion, instead of an fundamental blocker. The author failed to capture the difference between _"pattern syntax"_ and _"rule system"_. ast-grep's rule system is YAML based, so it is easier to write a [well-formed](https://en.wikipedia.org/wiki/Well-formedness) rule. Instead, DSL based rule system using their own syntax and can be more difficult to write a valid rule, especially for beginners. We can also see using DSL is not subjectively better than YAML as well. Subjective Comparison of DSL [​](https://ast-grep.github.io/blog/yaml-vs-dsl.html#subjective-comparison-of-dsl) ---------------------------------------------------------------------------------------------------------------- Let's review the DSL mentioned above. ### Mix of several different paradigms [​](https://ast-grep.github.io/blog/yaml-vs-dsl.html#mix-of-several-different-paradigms) Biome's DSL is a mix of several different paradigms: declarative, logic, and imperative. Let's see one example: JavaScript `$method($message)` where { $method <: `console.log`, if ($message <: r"Hello, .*!") { $linter = "hello world" } else { $linter = "not hello" }, register_diagnostic( span = $method, message = $linter ) } * `$method('$message')` is a declarative pattern matching syntax. * `where` and `<:` are related to [logic programming](https://en.wikipedia.org/wiki/Logic_programming#:~:text=Logic%20programming%20is%20a%20programming,solve%20problems%20in%20the%20domain.) paradigm, say, [Prolog](https://en.wikipedia.org/wiki/Prolog) or SQL. * `if` is a typical imperative programming paradigm The mixture of paradigms does not blend well. At least, in the eye of a programming language veteran, it is too messy for a DSL for linting or structural search. We are not designing a next-era programming language. ### Easy to miss comma [​](https://ast-grep.github.io/blog/yaml-vs-dsl.html#easy-to-miss-comma) Did you notice there is two trailing commas in `$method <: console.log` and if block? JavaScript `$method($message)` where { $method <: `console.log`, if ($message <: r"Hello, .*!") { $linter = "hello world" } else { $linter = "not hello" }, register_diagnostic( span = $method, message = $linter ) } Without them you will get a syntax error. This is a common problem for beginners to miss commas, a typical pitfall only in DSL. Alas, I can still remeber the old day when C compiler complained about missing semicolon. ### Similar basic patterns have distinct syntax appearance [​](https://ast-grep.github.io/blog/yaml-vs-dsl.html#similar-basic-patterns-have-distinct-syntax-appearance) There are several different basic patterns in the DSL. Though they are at the similar level of abstraction, their appearance is totally different. JavaScript `console.log($foo)` // pattern augmented_assignment_expression(operator = $op, left = $x, right = $v) // syntax node r"Hello, (.*)"($name) // regex These patterns are corresponding to `pattern`, `kind` and `regex` in ast-grep. However, they look totally different. You need more learning to pick up these distinct syntax. ### Similar syntax appearance have different meaning [​](https://ast-grep.github.io/blog/yaml-vs-dsl.html#similar-syntax-appearance-have-different-meaning) One common pitfall to design DSL is that similar syntax have different meaning. JavaScript // this is a syntax node call augmented_assignment_expression(operator = $op) pattern console_method_to_info($method) { `console.$method($message)` => `console.info($message)` } // this is a pattern call console_method_to_info(method = `log`) predicate program_contains_logger() { $program <: contains `logger` } // this is a predicate call program_contains_logger() // define a lines function function lines($string) { return split($string, separator=`\n`) } // this is a function call lines(string = $message) They all look like function calls, but they are not. See explanation below for the differences. ### Confusing Concepts of `pattern`, `predicate` and `function` [​](https://ast-grep.github.io/blog/yaml-vs-dsl.html#confusing-concepts-of-pattern-predicate-and-function) These three concepts are very similar, but they have slightly different usage in the DSL. * `pattern` is used in `<:` or somewhere else, I dunno, the doc does not explain it well. * `predicate` is used in `where` condition. * `function` is used in `assignment`, `insertion` or `rewrite`. Example: JavaScript `console.log` => `logger.info` where { $program <: contains_logger(), // pattern program_contains_logger(), // equivalent predicate $program => replace_logger(), // function } ### Confusing Concepts of `condition`, `clause` and `modifier` [​](https://ast-grep.github.io/blog/yaml-vs-dsl.html#confusing-concepts-of-condition-clause-and-modifier) The DSL also has three similar concepts: `condition`, `clause` and `modifier`. Introduced in different places, [here](https://docs.grit.io/language/conditions) and [here](https://docs.grit.io/language/modifiers) , without clear definition. ### Similar patterns but applied in different places [​](https://ast-grep.github.io/blog/yaml-vs-dsl.html#similar-patterns-but-applied-in-different-places) Tell the difference between [and](https://docs.grit.io/language/modifiers#and-clause) , [any](https://docs.grit.io/language/modifiers#any-clause) , [some](https://docs.grit.io/language/modifiers#some-clause) and [every](https://docs.grit.io/language/modifiers#every-clause) . Confusing? You should learn the difference between meta var in [list pattern](https://docs.grit.io/language/modifiers#list-patterns) and plain meta var. Also, don't confuse list meta var with [spread meta var](https://docs.grit.io/language/patterns#metavariables) ### One more thing, variable scope. [​](https://ast-grep.github.io/blog/yaml-vs-dsl.html#one-more-thing-variable-scope) If you still have patience, you need one last thing to learn: variable scope. I have no better explanation for it since I don't understand it well, so I will quote the [official documentation](https://docs.grit.io/language/bubble) : > Once a metavariable is bound to a value, it retains this value throughout the target code. Therefore, the scope of the metavariable spans the entire target file. To fully understand it, you also need to know `bubble`, `bubble($argument)` and pattern auto wrap. What can go even more wrong? [​](https://ast-grep.github.io/blog/yaml-vs-dsl.html#what-can-go-even-more-wrong) --------------------------------------------------------------------------------------------------------------- Integrating a custom linting rule from another parser ecosystem to your own has several more decisions to make. Making bad decisions can lead to even more confusion. * Your pattern syntax includes non standard syntax, like `$...`. You need to [change your parser](https://github.com/biomejs/biome/blob/31e439674493da76e0ce213e5660be3d903efbef/crates/biome_js_parser/src/syntax/jsx/mod.rs#L321) to support it. * You need to make a decision if you want to support existing pattern libraries. But these patterns are built upon [tree-sitter](https://tree-sitter.github.io/tree-sitter/) . So you have to [map tree-sitter AST to your own](https://github.com/biomejs/biome/blob/7bf9a608e1592fd595f658f5f800e12d51835d34/crates/biome_grit_patterns/src/grit_target_language/js_target_language.rs#L42-L55) * Even worse, if you chose to support existing pattern libraries, you also need to work on a general algorithm to handle incompatibility between different ASTs. For example, different AST structures, different node names, different node properties, etc. * If you only supports part of it, how can you teach your users what is supported and what is not, without [reading the source](https://github.com/biomejs/biome/blob/7bf9a608e1592fd595f658f5f800e12d51835d34/crates/biome_grit_patterns/src/grit_target_language/js_target_language.rs#L48-L50) ? * You also need to update your playground or editor plugins to support mapping between tree-sitter AST and your own AST. And teach users to use it. Conclusion [​](https://ast-grep.github.io/blog/yaml-vs-dsl.html#conclusion) ---------------------------------------------------------------------------- If you also feel confused, you are not alone. Again, the preference of DSL over YAMl is largely subjective. If you think DSL is better, you are right. [You are absolutely right](https://www.reddit.com/r/ClaudeAI/comments/152b51r/you_are_absolutely_right/) . In fact, you are [not even wrong](https://en.wikipedia.org/wiki/Not_even_wrong) . Since this is a subjective opinion, not an objective fact. If you are a library or framework author, you can make decision based on your own preference. However, mistakenly thinking your preference is objective will lead to confusion and misunderstanding. It may even reflect inferior tech taste and judgement. Consider these points when you want to have objective comparison: * Documentation? * User Education? Howe you teach users to write your DSL? * Tooling support like [playground](https://ast-grep.github.io/playground.html) . * Editor support beyong syntax highlighting. Say LSP. * Integration with API, how you bring type-safe DSL into your general purpose programming language, like [graphql](https://github.com/Quramy/ts-graphql-plugin) and [styled component](https://github.com/styled-components/typescript-styled-plugin) . * Broader ecosystem support, such as GitHub language detection, AI support, etc. If you are going to use native tooling in your JavaScript/TypeScript project, I recommend you to use [oxlint](https://oxc.rs/) and, if you need simple custom rules, [ast-grep](https://ast-grep.github.io/) . --- # ast-grep's Journey to Type Safety in Node API | ast-grep [Skip to content](https://ast-grep.github.io/blog/typed-napi.html#VPContent) Return to top ast-grep's Journey to Type Safety in Node API [​](https://ast-grep.github.io/blog/typed-napi.html#ast-grep-s-journey-to-type-safety-in-node-api) ================================================================================================================================================= > Recipe to Craft Balanced Types: _Design, Define, Refine, and Confine_ We're thrilled to introduce typed AST in [@ast-grep/napi](https://www.npmjs.com/package/@ast-grep/napi) , addressing a [long-requested feature](https://github.com/ast-grep/ast-grep/issues/48) for AST manipulation from the early days of this project. In this blog post, we will delve into the challenges addressed by this feature and explore [the design](https://github.com/ast-grep/ast-grep/issues/1669) that shaped its implementation. _We also believe this post can serve as a general guide to crafting balanced TypeScript types._ ![napi screenshot](https://ast-grep.github.io/image/blog/napi.jpeg) Type Safety in AST [​](https://ast-grep.github.io/blog/typed-napi.html#type-safety-in-ast) ------------------------------------------------------------------------------------------- Working with Abstract Syntax Trees (ASTs) is complex. Even with AST [excellent](https://astexplorer.net/) [AST](https://ast-grep.github.io/playground.html) [tools](https://github.com/sxzz/ast-kit) , handling all edge cases remains challenging. Type information serves as a crucial safety net when writing AST manipulation code. It guides developers toward handling all possible cases and enables exhaustive checking to ensure complete coverage. While `ast-grep/napi` has been a handy tool for programmatic AST processing, it previously lacked type information to help users write robust code. Thanks to [Mohebifar](https://github.com/mohebifar) from [Codemod](https://codemod.com/) , we've now bridged this gap. Our solution generates types from parsers' metadata and employs TypeScript tricks to create an idiomatic API. Qualities of Good Types [​](https://ast-grep.github.io/blog/typed-napi.html#qualities-of-good-types) ----------------------------------------------------------------------------------------------------- Before diving into our implementation, let's explore what makes TypeScript definitions truly effective. In today's JavaScript ecosystem, creating a great library involves more than just intuitive APIs and thorough documentation – it requires thoughtful type definitions that enhance developer experience. A well-designed type system should balance four key qualities: * **Correct**: Types should act as reliable guardrails, rejecting invalid code while allowing all valid use cases. * **Concise**: Types should be easy to understand, whether in IDE hovers or code completions. Clear, readable types help developers quickly grasp your API. * **Robust**: In case type inference fails, the compiler should either graciously tolerate untyped code, or gracefully provide clear error messages. Cryptic type errors that span multiple screens is daunting and unhelpful. * **Performant**: Both type checking and runtime code should be fast. Complex types can significantly slow down compilation while unnecessary API calls just conforming to type safety can hurt runtime performance. Balancing these qualities is demanding job because they often compete with each other, just like creating a type system that is both [sound and complete](https://logan.tw/posts/2014/11/12/soundness-and-completeness-of-the-type-system/#:~:text=A%20type%2Dsystem%20is%20sound,any%20false%20positive%20%5B2%5D.) . Many TS libraries lean heavily toward strict correctness – for instance, implementing elaborate types to validate routing parameters. While powerful, [type gymnastics](https://www.octomind.dev/blog/navigating-the-typescript-gymnastics-on-developer-dogma-2) can come with significant trade-offs in complexity and compile-time performance. Sometimes, being slightly less strict can lead to a dramatically better developer experience. We will explore how ast-grep balances these qualities through _Design, Define, Refine, and Confine_. Design Types [​](https://ast-grep.github.io/blog/typed-napi.html#design-types) ------------------------------------------------------------------------------- Let's return to ast-grep's challenge and learn some background knowledge on how Tree-sitter, our underlying parser library, handles types. ### TreeSitter's Core API [​](https://ast-grep.github.io/blog/typed-napi.html#treesitter-s-core-api) At its heart, Tree-sitter provides a language-agnostic API for traversing syntax trees. Its base API is intentionally untyped, offering a consistent interface across all programming languages: typescript class Node { kind(): string // Get the type of node, e.g., 'function_declaration' field(name: string): Node // Get a specific child by its field name parent(): Node // Navigate to the parent node children(): Node[] // Get all child nodes text(): string // Get the actual source code text } This API is elegantly simple, but its generality comes at the cost of type safety. In contrast, traditional language-specific parsers bake AST structures directly into their types. Consider [estree](https://github.com/estree/estree/blob/0362bbd130e926fed6293f04da57347a8b1e2325/es5.md) . It encodes rich structural information about each node type in JavaScript. For instance, a `function_declaration` is a specific structure with the function's `name`, `parameters` list, and `body` fields. Fortunately, Tree-sitter hasn't left us entirely without type information. It provides detailed static type information in JSON format and leaves us an opportunity to enchant the flexible runtime API with the type safe magic. ### Tree-sitter's `TypeMap` [​](https://ast-grep.github.io/blog/typed-napi.html#tree-sitter-s-typemap) Tree-sitter provides [static node types](https://tree-sitter.github.io/tree-sitter/using-parsers#static-node-types) for library authors to consume. The type information has the following form, in TypeScript interface: typescript interface TypeMap { [kind: string]: { type: string named: boolean fields?: { [field: string]: { types: { type: string, named: boolean }[] } } children?: { name: string, type: string }[] subtypes?: { type: string, named: boolean }[] } } `TypeMap` is a comprehensive catalog of all possible node types in a language's syntax tree. Let's break this down with a concrete example from TypeScript: typescript type TypeScript = { // AST node type definition function_declaration: { type: "function_declaration", // kind named: true, // is named fields: { body: { types: [ { type: "statement_block", named: true } ] }, } }, ... } The structure contains the information about the node's kind, whether it is named, and its' fields and children. `fields` is a map from field name to the type of the field, which encodes the AST structure like traditional parsers. Tree-sitter also has a special type called `subtypes`, an alias of a list of other kinds. typescript type TypeScript = { // node type alias declaration: { type: "declaration", subtypes: [\ { type: "class_declaration", named: true },\ { type: "function_declaration", named: true },\ ] }, ... } In this example, `declaration` is an alias of `function_declaration`, `class_declaration` and other kinds. The alias type is used to reduce the redundancy in the static type JSON and will NOT be a node's actual kind. Thanks to Tree-Sitter's design, we can leverage this rich type information to build our typed APIs! ### Design Principles of ast-grep/napi [​](https://ast-grep.github.io/blog/typed-napi.html#design-principles-of-ast-grep-napi) Our new API follows a progressive enhancement approach to type safety: **Preserve untyped AST access**. The existing untyped API remains available by default, ensuring backward compatibility **Optional type safety on demand**. Users can opt into typed AST nodes either manually or automatically for enhanced type checking and autocompletion However, it is a bumpy ride to transition to a new typed API via the path of Tree-sitter's static type. First, type information JSON is hosted by Parser Library Repository. ast-grep/napi uses [a dedicated script](https://github.com/ast-grep/ast-grep/blob/main/crates/napi/scripts/generateTypes.ts) to fetch the JSON and generates the type. A [F# like type provider](https://learn.microsoft.com/en-us/dotnet/fsharp/tutorials/type-providers/) is on my TypeScript wishlist. Second, the JSON contains a lot of unnamed kinds, which are not useful to users. Including them in the union type is too noisy. We will address this in the next section. Finally, as mentioned earlier, the JSON contains alias types. We need to resolve the alias type to its concrete type, which is also covered in the next section. Define Types [​](https://ast-grep.github.io/blog/typed-napi.html#define-types) ------------------------------------------------------------------------------- New API's core involves several key new types and extensions to existing types. ### Let `SgNode` Have Type [​](https://ast-grep.github.io/blog/typed-napi.html#let-sgnode-have-type) `SgNode` class, the cornerstone of our new API, now accepts two new optional type parameters. typescript class SgNode = Kinds> { kind: K fields: M[K]['fields'] // demo definition, real one is more complex } It represents a node in a language with type map `M` that has a specific kind `K`. E.g. `SgNode` means a function declaration node in TypeScript. When used without a specific kind parameter, `SgNode` defaults to accepting any valid node kind in the language. `SgNode` provides a **correct** AST interface in a specific language. While at the same time, it is still **robust** enough to not trigger compiler error when no type information is available. ### `ResolveType` [​](https://ast-grep.github.io/blog/typed-napi.html#resolvetype-m-t) While Tree-sitter's type aliases help keep the JSON type definitions compact, they present a challenge: these aliases never appear as actual node kinds in ast-grep rules. To handle this, we created `ResolveType` to **correctly** map aliases to their concrete kinds: typescript type ResolveType = M[T] extends {subtypes: infer S extends {type: string}[] } ? ResolveType : T This type recursively resolves aliases until it reaches actual node types that developers work with. ### `Kinds` [​](https://ast-grep.github.io/blog/typed-napi.html#kinds-m) Having access to all possible AST node types is powerful, but it is unwieldy to work with large string literal union types. It can be a huge UX improvement to use a type alias to **concisely** represent all possible kinds of nodes. Additionally, Tree-sitter's static type contains a bunch of noisy unnamed kinds. But excluding them from the union type can lead to a incomplete type signature. ast-grep instead bundle them into a plain `string` type, creating a more **robust** API. typescript type Kinds = ResolveType & LowPriorityString type LowPriorityString = string & {} The above type is a linient string type that is compatible with any string type. But it also uses a [well-known trick](https://stackoverflow.com/a/61048124/2198656) to take advantage of TypeScript's type priority to prefer the `ResolveType` in completion over the `string & {}` type. We alias `string & {}` to `LowPriorityString` to make the code's intent clearer. This approach creates a more intuitive developer experience, though it does run into [some limitations](https://github.com/microsoft/TypeScript/issues/33471) with TypeScript's handling of [open-ended unions](https://github.com/microsoft/TypeScript/issues/26277) . We need other tricks to address these limitations. Introducing `RefineNode` type. ### Bridging general nodes and specific nodes via `RefineNode` [​](https://ast-grep.github.io/blog/typed-napi.html#bridging-general-nodes-and-specific-nodes-via-refinenode) A key challenge in our type system was handling two distinct categories of nodes: 1. **General Nodes**: String-based typing (like our original API, but with enhanced completion), `SgNode>`. 2. **Specific Nodes**: Precisely typed nodes with known kinds, `SgNode`. When dealing with nodes that could be several specific kinds, we faced an interesting type system challenge. Consider these two approaches: typescript // Approach 1: Union in the type parameter let single: SgNode<'expression' | 'type'> // Approach 2: Union of specific nodes let union: SgNode<'expression'> | SgNode<'type'> These approaches behave differently in TypeScript, for a [good reason](https://x.com/hd_nvim/status/1868706176281854151) : typescript let single: SgNode<'expression' | 'type'> if (single.kind === 'expression') { single // Remains SgNode<'expression' | 'type'> - not narrowed! } let union: SgNode<'expression'> | SgNode<'type'> if (union.kind === 'expression') { union // Successfully narrowed to SgNode<'expression'> } `SgNode` is technically covariant in its kind parameter, meaning it's safe to distribute the type constructor over unions. However TypeScript doesn't support this automatically. (We will not go down the rabbit hole of type constructor variance here. But interested readers can check out [this wiki](https://en.wikipedia.org/wiki/Covariance_and_contravariance_(computer_science)) .) To bridge this gap, we introduced the `RefineNode` type: typescript type RefineNode = string extends K ? SgNode : // one SgNode K extends keyof M ? SgNode : never // distribute over union This utility type provides two key behaviors: 1. When `K` includes a string type, it preserves the general node behavior 2. Otherwise, it refines the node into a union of specific types, using TypeScripts' [distributive conditional types](https://www.typescriptlang.org/docs/handbook/2/conditional-types.html#distributive-conditional-types) . This approach, inspired by [Biome's Rowan API](https://github.com/biomejs/biome/blob/09a04af727b3cdba33ac35837d112adb55726add/crates/biome_rowan/src/ast/mod.rs#L108-L120) , achieves our dual goals: it remains **correct** by preserving proper type relationships and stays **robust** by gracefully handling both typed and untyped usage. This hybrid approach gives developers the best of both worlds: strict type checking when types are known, with the flexibility to fall back to string-based typing when needed. Refine Types [​](https://ast-grep.github.io/blog/typed-napi.html#refine-types) ------------------------------------------------------------------------------- Now let's talk about how to refine the general node to a specific node in ast-grep/napi. We've implemented two concise and idiomatic approaches in TypeScript: manual and automatic refinement. ### Refine Node, Manually [​](https://ast-grep.github.io/blog/typed-napi.html#refine-node-manually) #### Runtime Type Checking [​](https://ast-grep.github.io/blog/typed-napi.html#runtime-type-checking) The first manual approach uses runtime verification through the `is` method: typescript class SgNode { is(kind: T): this is SgNode } This enables straightforward type narrowing: typescript if (sgNode.is("function_declaration")) { sgNode.kind // narrow to 'function_declaration' } #### Type Parameter Specification [​](https://ast-grep.github.io/blog/typed-napi.html#type-parameter-specification) Another manual approach lets you explicitly specify node types through type parameters. This is particularly useful when you're certain about a node's kind and want to skip runtime checks for better performance. This pattern may feel familiar if you've worked with the [DOM API](https://www.typescriptlang.org/docs/handbook/dom-manipulation.html#the-queryselector-and-queryselectorall-methods) 's `querySelector`. Just as `querySelector` can be refined from a general `Element` to a specific `HTMLDivElement`, we can refine our nodes: typescript sgNode.parent<"program">() // Returns SgNode The type parameter approach uses an interesting overloading signature typescript interface NodeMethod { (): SgNode // Untyped version (): RefineNode // Typed version } If no type is provided, it returns a general node, `SgNode`. If a type is provided, it returns a specific node, `SgNode`. This dual-signature typing avoids the limitations of a single generic signature, which would either always return `SgNode` or always produce a union of `SgNode`s. #### Choosing the Right Type [​](https://ast-grep.github.io/blog/typed-napi.html#choosing-the-right-type) When should you use each manual refinement method? Here are some guidelines: ✓ Use `is()` when: * You need runtime type check * Node types might vary * Type safety is crucial ✓ Use type parameters when: * You're completely certain of the node type * Performance is critical * The node type is fixed Safety Tip Be cautious with type parameters as they bypass runtime checks. It can break type safety if misused. You can audit their usage with the command: bash ast-grep -p '$NODE.$METHOD<$K>($$$)' ### Refine Node, Automatically [​](https://ast-grep.github.io/blog/typed-napi.html#refine-node-automatically) A standout feature of our new API is automatic type refinement based on contextual information. This happens seamlessly through the `field` method. When you access a node's field using `field("name")`, the system automatically examines the static type information and refines the node type accordingly: typescript let exportStmt: SgNode<'export_statement'> exportStmt.field('declaration') // Automatically refines to union: // SgNode<'function_declaration'> | // SgNode<'variable_declaration'> | ... The magic here is that you never need to specify the possible types explicitly - the system infers them automatically. This approach is both **concise** in usage and **correct** in type inference. ### Exhaustive Pattern Matching with kindToRefine [​](https://ast-grep.github.io/blog/typed-napi.html#exhaustive-pattern-matching-with-kindtorefine) We've also introduced a new `kindToRefine` property for comprehensive type checking. You might wonder: why add this when we already have a `kind()` method? There are two key reasons: 1. Preserving backward compatibility with the existing `kind()` method 2. Enabling TypeScript's type narrowing, which works with properties but not method calls While `kindToRefine` is implemented as a getter that calls into Rust code (making it as computationally expensive as the `kind()` method), it enables powerful type checking capabilities. To ensure developers are aware of this **performance** characteristic, we deliberately chose a _distinct and longer_ property name. This property really shines when working in tandem union types returned by `RefineNode`, helping you write **correct** AST transformations through exhaustive pattern matching: typescript const func: SgNode<'function_declaration'> | SgNode<'arrow_function'> switch (func.kindToRefine) { case 'function_declaration': func.kindToRefine // Narrowed to function_declaration break case 'arrow_function': func.kindToRefine // Narrowed to arrow_function break default: func satisfies never // TypeScript ensures we handled all cases } The combination of automatic type refinement and exhaustive pattern matching makes it easier to write **correct** AST transformations while catching potential errors at compile time. Confine Types [​](https://ast-grep.github.io/blog/typed-napi.html#confine-types) --------------------------------------------------------------------------------- Always bear in mind this mantra: _Be austere with type level programming._ Overdoing type level programming can overload the compiler as well as overwhelm users. It is a good practice to confine the API type to a reasonable complexity level. ### Prune unnamed kinds [​](https://ast-grep.github.io/blog/typed-napi.html#prune-unnamed-kinds) Tree-sitter's static type includes many unnamed kinds, which are not user-friendly. For instance, operators like `+`/`-`/`*`/`/` are too verbose for an AST library. We're building a compiler plugin, not solving elementary school math problems, right? This is why we exclude the unnamed kinds and include `string` in the `Kinds`. In the type generation step, ast-grep filters out these unnamed kinds to make the type more **concise**. ### Opt-in refinement for better compile time performance [​](https://ast-grep.github.io/blog/typed-napi.html#opt-in-refinement-for-better-compile-time-performance) The new API is designed to provide a better type checking and autocompletion experience for users. However, this improvement comes at the cost of **performance**. A single type map for one language can span several thousand lines of code with hundreds of kinds. The more type information the user provides, the slower the compile time. To manage this, you need to explicitly opt into type information by passing type parameters to the `parse` method. typescript import { parse } from '@ast-grep/napi' import TS from '@ast-grep/napi/lang/TypeScript' // import this can be slow const untyped = parse(Lang.TypeScript, code) const typed = parse(Lang.TypeScript, code) ### Typed Rule! [​](https://ast-grep.github.io/blog/typed-napi.html#typed-rule) The last notable feature is the typed rule. You can even type the `kind` in rule JSON! typescript interface Rule { kind: Kinds ... // other rules } Of course, this isn't about _confining_ the type but allowing type information to enhance rules, significantly improving UX and rule **correctness**. You can look up the available kinds in the static type via the completion popup in your editor. (btw I use nvim) typescript sgNode.find({ rule: { // kind: 'invalid_kind', // error! kind: 'function_declaration', // typed! } }) ![napi screenshot](https://ast-grep.github.io/image/blog/rule.jpeg) Ending [​](https://ast-grep.github.io/blog/typed-napi.html#ending) ------------------------------------------------------------------- I'm incredibly excited about the future of AST manipulation in TypeScript. You can see the full type definition [here](https://github.com/ast-grep/ast-grep/tree/main/crates/napi/types) . This feature empowers users to seamlessly switch between untyped and typed AST, offering flexibility and enhanced capabilities, an innovation that has not been seen in other AST libraries, especially not in native language based ones. As [Theo](https://x.com/theo) aptly puts it in [his video](https://www.youtube.com/clip/Ugkxn2oomDuyQjtaKXhYP1MU9TLEShf5m1nf) : > There are very few devs that understand Rust deeply enough and compiler deeply enough that also care about TypeScript in web dev enough to build something for web devs in Rust ast-grep is determined to bridge that gap between Rust and TypeScript! --- # Playground | ast-grep [Skip to content](https://ast-grep.github.io/playground.html#VPContent) Return to top Are you an LLM? You can read better optimized documentation at /playground.md for this page in Markdown format Loading Editor and Parser... ---------------------------- --- # ast-grep | structural search/rewrite tool for many languages [Skip to content](https://ast-grep.github.io/#VPContent) AST-GREPFind Code by Syntax =========================== ast-grep(sg) is a fast and polyglot tool for code structural search, lint, rewriting at large scale. [Quickstart](https://ast-grep.github.io/guide/quick-start.html) [Intro](https://ast-grep.github.io/guide/introduction.html) [Examples](https://ast-grep.github.io/catalog/) [With AI](https://ast-grep.github.io/advanced/prompting.html) ![ast-grep](https://ast-grep.github.io/logo.svg) * * * * Search and Rewrite ------------------ ast-grep is a code tool for [structural search and replace](https://ast-grep.github.io/guide/quick-start.html) . It is like syntax-aware grep/sed! You can write code [patterns](https://ast-grep.github.io/guide/pattern-syntax.html) to locate and modify code, based on AST, in thousands of files, [interactively](https://ast-grep.github.io/guide/tooling-overview.html#interactive-mode) . `ast-grep -p '$A && $A()' -r '$A?.()'` ![](https://ast-grep.github.io/image/search-replace.png) * * * * Scan as Linter -------------- ast-grep is a versatile and flexible tool for [linting](https://ast-grep.github.io/guide/scan-project.html) code with AST patterns. You can easily add new customized rules with [intuitive syntax](https://ast-grep.github.io/guide/rule-config.html) and enjoy pretty error reporting out of box. `ast-grep scan` ![](https://ast-grep.github.io/image/yaml-scan.png) * * * * Programmatic Usage ------------------ ast-grep also provides [node-js binding](https://ast-grep.github.io/guide/api-usage/js-api.html) to access syntax trees programmatically. You can use jQuery like [utility methods](https://ast-grep.github.io/reference/api.html#napi) to traverse syntax tree nodes. Node API also has opt-in [type safety](https://ast-grep.github.io/blog/typed-napi.html) . `npm install @ast-grep/napi` ![](https://ast-grep.github.io/image/sg-napi.png) * * * ⚡️ Performant ---------- Blazing fast search and replace across thousands of source code files, powered by parallel Rust. 📚 Polyglot -------- Supports many programming languages out of box! You can also register your own tree-sitter parsers by dynamic loading. 👟 Progressive ----------- Supports multiple forms of usages from one-line command to fully programmatic interface, scaling to different scenarios. 🛠️ Pragmatic --------- Not a toy but a batteries-included tool with interactive codemod, language server and testing tool. 20+ Supported Languages ======================= [![](https://ast-grep.github.io/icons/C.svg)](https://ast-grep.github.io/catalog/c/) ![](https://ast-grep.github.io/icons/CSharp.svg)[![](https://ast-grep.github.io/icons/Go.svg)](https://ast-grep.github.io/catalog/go/) [![](https://ast-grep.github.io/icons/Java.svg)](https://ast-grep.github.io/catalog/java/) [![](https://ast-grep.github.io/icons/JS.svg)](https://ast-grep.github.io/catalog/tsx/) [![](https://ast-grep.github.io/icons/Kotlin.svg)](https://ast-grep.github.io/catalog/kotlin/) [![](https://ast-grep.github.io/icons/Python.svg)](https://ast-grep.github.io/catalog/python/) [![](https://ast-grep.github.io/icons/Rust.svg)](https://ast-grep.github.io/catalog/rust/) [![](https://ast-grep.github.io/icons/TS.svg)](https://ast-grep.github.io/catalog/typescript/) See more in the [full list](https://ast-grep.github.io/reference/languages.html) . Custom languages are also [loadable](https://ast-grep.github.io/advanced/custom-language.html) . Who is using ast-grep ===================== [![](https://avatars.githubusercontent.com/u/26715726?s=200&v=4)](https://swc.rs/ "SWC") [![](https://raw.githubusercontent.com/baiwusanyu-c/unplugin-vue-cssvars/master/public/logo.png)](https://github.com/unplugin/unplugin-vue-cssvars#readme "unplugin-vue-cssvars") [![](https://ast-grep.github.io/logos/hydrogen.svg)](https://hydrogen.shopify.dev/ "Hydrogen") [![](https://avatars.githubusercontent.com/u/120662729?s=200&v=4)](https://github.com/vue-macros/vue-macros-cli "vue-macros") [![](https://avatars.githubusercontent.com/u/126793605?s=200&v=4)](https://modernjs.dev/ "Modern.js") [![](https://avatars.githubusercontent.com/in/347564?s=200&v=4)](https://coderabbit.ai/ "CodeRabbit") [![](https://avatars.githubusercontent.com/u/78830094?s=200&v=4)](https://codemod.com/ "Codemod") [![](https://avatars.githubusercontent.com/u/180226629?s=200&v=4)](https://opennext.js.org/cloudflare "OpenNext") [![](https://avatars.githubusercontent.com/u/127165244)](https://dify.ai/ "Dify") Are you an LLM? View /llms.txt for optimized Markdown documentation, or /llms-full.txt for full documentation bundle --- # ast-grep [Skip to content](https://ast-grep.github.io/catalog/c/match-function-call.html#VPContent) Return to top Are you an LLM? You can read better optimized documentation at /catalog/c/match-function-call.md for this page in Markdown format Match Function Call in C [​](https://ast-grep.github.io/catalog/c/match-function-call.html#match-function-call-in-c) --------------------------------------------------------------------------------------------------------------------- * [Playground Link](https://ast-grep.github.io/playground.html#eyJtb2RlIjoiQ29uZmlnIiwibGFuZyI6ImMiLCJxdWVyeSI6InRlc3QoJCQkKSIsInJld3JpdGUiOiIiLCJjb25maWciOiJydWxlOlxuICBwYXR0ZXJuOiBcbiAgICBjb250ZXh0OiAkTSgkJCQpO1xuICAgIHNlbGVjdG9yOiBjYWxsX2V4cHJlc3Npb24iLCJzb3VyY2UiOiIjZGVmaW5lIHRlc3QoeCkgKDIqeClcbmludCBhID0gdGVzdCgyKTtcbmludCBtYWluKCl7XG4gICAgaW50IGIgPSB0ZXN0KDIpO1xufSJ9) ### Description [​](https://ast-grep.github.io/catalog/c/match-function-call.html#description) One of the common questions of ast-grep is to match function calls in C. A plain pattern like `test($A)` will not work. This is because [tree-sitter-c](https://github.com/tree-sitter/tree-sitter-c) parse the code snippet into `macro_type_specifier`, see the [pattern output](https://ast-grep.github.io/playground.html#eyJtb2RlIjoiUGF0Y2giLCJsYW5nIjoiYyIsInF1ZXJ5IjoidGVzdCgkJCQpIiwicmV3cml0ZSI6IiIsImNvbmZpZyI6InJ1bGU6XG4gIHBhdHRlcm46IFxuICAgIGNvbnRleHQ6ICRNKCQkJCk7XG4gICAgc2VsZWN0b3I6IGNhbGxfZXhwcmVzc2lvbiIsInNvdXJjZSI6IiNkZWZpbmUgdGVzdCh4KSAoMip4KVxuaW50IGEgPSB0ZXN0KDIpO1xuaW50IG1haW4oKXtcbiAgICBpbnQgYiA9IHRlc3QoMik7XG59In0=) . To avoid this ambiguity, ast-grep lets us write a [contextual pattern](https://ast-grep.github.io/guide/rule-config/atomic-rule.html#pattern) , which is a pattern inside a larger code snippet. We can use `context` to write a pattern like this: `test($A);`. Then, we can use the selector `call_expression` to match only function calls. ### YAML [​](https://ast-grep.github.io/catalog/c/match-function-call.html#yaml) yaml id: match-function-call language: c rule: pattern: context: $M($$$); selector: call_expression ### Example [​](https://ast-grep.github.io/catalog/c/match-function-call.html#example) c #define test(x) (2*x) int a = test(2); int main(){ int b = test(2); } ### Caveat [​](https://ast-grep.github.io/catalog/c/match-function-call.html#caveat) Note, tree-sitter-c parses code differently when it receives code fragment. For example, * `test(a)` is parsed as `macro_type_specifier` * `test(a);` is parsed as `expression_statement -> call_expression` * `int b = test(a)` is parsed as `declaration -> init_declarator -> call_expression` The behavior is controlled by how the tree-sitter parser is written. And tree-sitter-c behaves differently from [tree-sitter-cpp](https://github.com/tree-sitter/tree-sitter-cpp) . Please file issues on tree-sitter-c repo if you want to change the behavior. ast-grep will respect changes and decision from upstream authors. --- # C | ast-grep [Skip to content](https://ast-grep.github.io/catalog/c/#VPContent) Return to top Are you an LLM? You can read better optimized documentation at /catalog/c.md for this page in Markdown format C [​](https://ast-grep.github.io/catalog/c/#c) =============================================== This page curates a list of example ast-grep rules to check and to rewrite C code. C files can be parsed as Cpp You can parse C code as Cpp to avoid rewriting similar rules. The [`languageGlobs`](https://ast-grep.github.io/reference/sgconfig.html#languageglobs) option can force ast-grep to parse `.c` files as Cpp. Match Function Call in C [​](https://ast-grep.github.io/catalog/c/#match-function-call-in-c) --------------------------------------------------------------------------------------------- * [Playground Link](https://ast-grep.github.io/playground.html#eyJtb2RlIjoiQ29uZmlnIiwibGFuZyI6ImMiLCJxdWVyeSI6InRlc3QoJCQkKSIsInJld3JpdGUiOiIiLCJjb25maWciOiJydWxlOlxuICBwYXR0ZXJuOiBcbiAgICBjb250ZXh0OiAkTSgkJCQpO1xuICAgIHNlbGVjdG9yOiBjYWxsX2V4cHJlc3Npb24iLCJzb3VyY2UiOiIjZGVmaW5lIHRlc3QoeCkgKDIqeClcbmludCBhID0gdGVzdCgyKTtcbmludCBtYWluKCl7XG4gICAgaW50IGIgPSB0ZXN0KDIpO1xufSJ9) ### Description [​](https://ast-grep.github.io/catalog/c/#description) One of the common questions of ast-grep is to match function calls in C. A plain pattern like `test($A)` will not work. This is because [tree-sitter-c](https://github.com/tree-sitter/tree-sitter-c) parse the code snippet into `macro_type_specifier`, see the [pattern output](https://ast-grep.github.io/playground.html#eyJtb2RlIjoiUGF0Y2giLCJsYW5nIjoiYyIsInF1ZXJ5IjoidGVzdCgkJCQpIiwicmV3cml0ZSI6IiIsImNvbmZpZyI6InJ1bGU6XG4gIHBhdHRlcm46IFxuICAgIGNvbnRleHQ6ICRNKCQkJCk7XG4gICAgc2VsZWN0b3I6IGNhbGxfZXhwcmVzc2lvbiIsInNvdXJjZSI6IiNkZWZpbmUgdGVzdCh4KSAoMip4KVxuaW50IGEgPSB0ZXN0KDIpO1xuaW50IG1haW4oKXtcbiAgICBpbnQgYiA9IHRlc3QoMik7XG59In0=) . To avoid this ambiguity, ast-grep lets us write a [contextual pattern](https://ast-grep.github.io/guide/rule-config/atomic-rule.html#pattern) , which is a pattern inside a larger code snippet. We can use `context` to write a pattern like this: `test($A);`. Then, we can use the selector `call_expression` to match only function calls. ### YAML [​](https://ast-grep.github.io/catalog/c/#yaml) yaml id: match-function-call language: c rule: pattern: context: $M($$$); selector: call_expression ### Example [​](https://ast-grep.github.io/catalog/c/#example) c #define test(x) (2*x) int a = test(2); int main(){ int b = test(2); } ### Caveat [​](https://ast-grep.github.io/catalog/c/#caveat) Note, tree-sitter-c parses code differently when it receives code fragment. For example, * `test(a)` is parsed as `macro_type_specifier` * `test(a);` is parsed as `expression_statement -> call_expression` * `int b = test(a)` is parsed as `declaration -> init_declarator -> call_expression` The behavior is controlled by how the tree-sitter parser is written. And tree-sitter-c behaves differently from [tree-sitter-cpp](https://github.com/tree-sitter/tree-sitter-cpp) . Please file issues on tree-sitter-c repo if you want to change the behavior. ast-grep will respect changes and decision from upstream authors. Rewrite Method to Function Call Has Fix [​](https://ast-grep.github.io/catalog/c/#rewrite-method-to-function-call) ------------------------------------------------------------------------------------------------------------------- * [Playground Link](https://ast-grep.github.io/playground.html#eyJtb2RlIjoiQ29uZmlnIiwibGFuZyI6ImMiLCJxdWVyeSI6IiRDT1VOVCA9ICRcbiIsInJld3JpdGUiOiIiLCJjb25maWciOiJpZDogbWV0aG9kX3JlY2VpdmVyXG5ydWxlOlxuICBwYXR0ZXJuOiAkUi4kTUVUSE9EKCQkJEFSR1MpXG50cmFuc2Zvcm06XG4gIE1BWUJFX0NPTU1BOlxuICAgIHJlcGxhY2U6XG4gICAgICBzb3VyY2U6ICQkJEFSR1NcbiAgICAgIHJlcGxhY2U6ICdeLisnXG4gICAgICBieTogJywgJ1xuZml4OlxuICAkTUVUSE9EKCYkUiRNQVlCRV9DT01NQSQkJEFSR1MpXG4iLCJzb3VyY2UiOiJ2b2lkIHRlc3RfZnVuYygpIHtcbiAgICBzb21lX3N0cnVjdC0+ZmllbGQubWV0aG9kKCk7XG4gICAgc29tZV9zdHJ1Y3QtPmZpZWxkLm90aGVyX21ldGhvZCgxLCAyLCAzKTtcbn0ifQ==) ### Description [​](https://ast-grep.github.io/catalog/c/#description-1) In C, there is no built-in support for object-oriented programming, but some programmers use structs and function pointers to simulate classes and methods. However, this style can have some drawbacks, such as: * extra memory allocation and deallocation for the struct and the function pointer. * indirection overhead when calling the function pointer. A possible alternative is to use a plain function call with the struct pointer as the first argument. ### YAML [​](https://ast-grep.github.io/catalog/c/#yaml-1) yaml id: method_receiver language: c rule: pattern: $R.$METHOD($$$ARGS) transform: MAYBE_COMMA: replace: source: $$$ARGS replace: '^.+' by: ', ' fix: $METHOD(&$R$MAYBE_COMMA$$$ARGS) ### Example [​](https://ast-grep.github.io/catalog/c/#example-1) c void test_func() { some_struct->field.method(); some_struct->field.other_method(1, 2, 3); } ### Diff [​](https://ast-grep.github.io/catalog/c/#diff) c void test_func() { some_struct->field.method(); method(&some_struct->field); some_struct->field.other_method(1, 2, 3); other_method(&some_struct->field, 1, 2, 3); } ### Contributed by [​](https://ast-grep.github.io/catalog/c/#contributed-by) [Surma](https://twitter.com/DasSurma) , adapted from the [original tweet](https://twitter.com/DasSurma/status/1706086320051794217) Rewrite Check to Yoda Condition Has Fix [​](https://ast-grep.github.io/catalog/c/#rewrite-check-to-yoda-condition) ------------------------------------------------------------------------------------------------------------------- * [Playground Link](https://ast-grep.github.io/playground.html#eyJtb2RlIjoiQ29uZmlnIiwibGFuZyI6ImMiLCJxdWVyeSI6IiRDOiAkVCA9IHJlbGF0aW9uc2hpcCgkJCRBLCB1c2VsaXN0PVRydWUsICQkJEIpIiwicmV3cml0ZSI6IiRDOiBMaXN0WyRUXSA9IHJlbGF0aW9uc2hpcCgkJCRBLCB1c2VsaXN0PVRydWUsICQkJEIpIiwiY29uZmlnIjoiaWQ6IG1heS10aGUtZm9yY2UtYmUtd2l0aC15b3Vcbmxhbmd1YWdlOiBjXG5ydWxlOlxuICBwYXR0ZXJuOiAkQSA9PSAkQiBcbiAgaW5zaWRlOlxuICAgIGtpbmQ6IHBhcmVudGhlc2l6ZWRfZXhwcmVzc2lvblxuICAgIGluc2lkZToge2tpbmQ6IGlmX3N0YXRlbWVudH1cbmNvbnN0cmFpbnRzOlxuICBCOiB7IGtpbmQ6IG51bWJlcl9saXRlcmFsIH1cbmZpeDogJEIgPT0gJEEiLCJzb3VyY2UiOiJpZiAobXlOdW1iZXIgPT0gNDIpIHsgLyogLi4uICovfVxuaWYgKG5vdE1hdGNoID09IGFub3RoZXIpIHt9XG5pZiAobm90TWF0Y2gpIHt9In0=) ### Description [​](https://ast-grep.github.io/catalog/c/#description-2) In programming jargon, a [Yoda condition](https://en.wikipedia.org/wiki/Yoda_conditions) is a style that places the constant portion of the expression on the left side of the conditional statement. It is used to prevent assignment errors that may occur in languages like C. ### YAML [​](https://ast-grep.github.io/catalog/c/#yaml-2) yaml id: may-the-force-be-with-you language: c rule: pattern: $A == $B # Find equality comparison inside: # inside an if_statement kind: parenthesized_expression inside: {kind: if_statement} constraints: # with the constraint that B: { kind: number_literal } # right side is a number fix: $B == $A The rule targets an equality comparison, denoted by the [pattern](https://ast-grep.github.io/guide/pattern-syntax.html) `$A == $B`. This comparison must occur [inside](https://ast-grep.github.io/reference/rule.html#inside) an `if_statement`. Additionally, there’s a [constraint](https://ast-grep.github.io/reference/yaml.html#constraints) that the right side of the comparison, `$B`, must be a number\_literal like `42`. ### Example [​](https://ast-grep.github.io/catalog/c/#example-2) c if (myNumber == 42) { /* ... */} if (notMatch == another) { /* ... */} if (notMatch) { /* ... */} ### Diff [​](https://ast-grep.github.io/catalog/c/#diff-1) c if (myNumber == 42) { /* ... */} if (42 == myNumber) { /* ... */} if (notMatch == another) { /* ... */} if (notMatch) { /* ... */} ### Contributed by [​](https://ast-grep.github.io/catalog/c/#contributed-by-1) Inspired by this [thread](https://x.com/cocoa1han/status/1763020689303581141) --- # ast-grep [Skip to content](https://ast-grep.github.io/catalog/cpp/find-struct-inheritance.html#VPContent) Return to top Are you an LLM? You can read better optimized documentation at /catalog/cpp/find-struct-inheritance.md for this page in Markdown format Find Struct Inheritance [​](https://ast-grep.github.io/catalog/cpp/find-struct-inheritance.html#find-struct-inheritance) ------------------------------------------------------------------------------------------------------------------------- * [Playground Link](https://ast-grep.github.io/playground.html#eyJtb2RlIjoiUGF0Y2giLCJsYW5nIjoiY3BwIiwicXVlcnkiOiJzdHJ1Y3QgJFNPTUVUSElORzogICRJTkhFUklUU19GUk9NIHsgJCQkQk9EWTsgfSIsInJld3JpdGUiOiIiLCJzdHJpY3RuZXNzIjoic21hcnQiLCJzZWxlY3RvciI6IiIsImNvbmZpZyI6IiIsInNvdXJjZSI6InN0cnVjdCBGb286IEJhciB7fTtcblxuc3RydWN0IEJhcjogQmF6IHtcbiAgaW50IGEsIGI7XG59In0=) ### Description [​](https://ast-grep.github.io/catalog/cpp/find-struct-inheritance.html#description) ast-grep's pattern is AST based. A code snippet like `struct $SOMETHING: $INHERITS` will not work because it does not have a correct AST structure. The correct pattern should spell out the full syntax like `struct $SOMETHING: $INHERITS { $$$BODY; }`. Compare the ast structure below to see the difference, especially the `ERROR` node. You can also use the playground's pattern panel to debug. Wrong PatternCorrect Pattern shell ERROR $SOMETHING base_class_clause $INHERITS shell struct_specifier $SOMETHING base_class_clause $INHERITS field_declaration_list field_declaration $$$BODY If it is not possible to write a full pattern, [YAML rule](https://ast-grep.github.io/guide/rule-config.html) is a better choice. ### Pattern [​](https://ast-grep.github.io/catalog/cpp/find-struct-inheritance.html#pattern) shell ast-grep --lang cpp --pattern ' struct $SOMETHING: $INHERITS { $$$BODY; }' ### Example [​](https://ast-grep.github.io/catalog/cpp/find-struct-inheritance.html#example) cpp struct Bar: Baz { int a, b; } ### Contributed by [​](https://ast-grep.github.io/catalog/cpp/find-struct-inheritance.html#contributed-by) Inspired by this [tweet](https://x.com/techno_bog/status/1885421768384331871) --- # ast-grep [Skip to content](https://ast-grep.github.io/catalog/c/yoda-condition.html#VPContent) Return to top Are you an LLM? You can read better optimized documentation at /catalog/c/yoda-condition.md for this page in Markdown format Rewrite Check to Yoda Condition Has Fix [​](https://ast-grep.github.io/catalog/c/yoda-condition.html#rewrite-check-to-yoda-condition) -------------------------------------------------------------------------------------------------------------------------------------- * [Playground Link](https://ast-grep.github.io/playground.html#eyJtb2RlIjoiQ29uZmlnIiwibGFuZyI6ImMiLCJxdWVyeSI6IiRDOiAkVCA9IHJlbGF0aW9uc2hpcCgkJCRBLCB1c2VsaXN0PVRydWUsICQkJEIpIiwicmV3cml0ZSI6IiRDOiBMaXN0WyRUXSA9IHJlbGF0aW9uc2hpcCgkJCRBLCB1c2VsaXN0PVRydWUsICQkJEIpIiwiY29uZmlnIjoiaWQ6IG1heS10aGUtZm9yY2UtYmUtd2l0aC15b3Vcbmxhbmd1YWdlOiBjXG5ydWxlOlxuICBwYXR0ZXJuOiAkQSA9PSAkQiBcbiAgaW5zaWRlOlxuICAgIGtpbmQ6IHBhcmVudGhlc2l6ZWRfZXhwcmVzc2lvblxuICAgIGluc2lkZToge2tpbmQ6IGlmX3N0YXRlbWVudH1cbmNvbnN0cmFpbnRzOlxuICBCOiB7IGtpbmQ6IG51bWJlcl9saXRlcmFsIH1cbmZpeDogJEIgPT0gJEEiLCJzb3VyY2UiOiJpZiAobXlOdW1iZXIgPT0gNDIpIHsgLyogLi4uICovfVxuaWYgKG5vdE1hdGNoID09IGFub3RoZXIpIHt9XG5pZiAobm90TWF0Y2gpIHt9In0=) ### Description [​](https://ast-grep.github.io/catalog/c/yoda-condition.html#description) In programming jargon, a [Yoda condition](https://en.wikipedia.org/wiki/Yoda_conditions) is a style that places the constant portion of the expression on the left side of the conditional statement. It is used to prevent assignment errors that may occur in languages like C. ### YAML [​](https://ast-grep.github.io/catalog/c/yoda-condition.html#yaml) yaml id: may-the-force-be-with-you language: c rule: pattern: $A == $B # Find equality comparison inside: # inside an if_statement kind: parenthesized_expression inside: {kind: if_statement} constraints: # with the constraint that B: { kind: number_literal } # right side is a number fix: $B == $A The rule targets an equality comparison, denoted by the [pattern](https://ast-grep.github.io/guide/pattern-syntax.html) `$A == $B`. This comparison must occur [inside](https://ast-grep.github.io/reference/rule.html#inside) an `if_statement`. Additionally, there’s a [constraint](https://ast-grep.github.io/reference/yaml.html#constraints) that the right side of the comparison, `$B`, must be a number\_literal like `42`. ### Example [​](https://ast-grep.github.io/catalog/c/yoda-condition.html#example) c if (myNumber == 42) { /* ... */} if (notMatch == another) { /* ... */} if (notMatch) { /* ... */} ### Diff [​](https://ast-grep.github.io/catalog/c/yoda-condition.html#diff) c if (myNumber == 42) { /* ... */} if (42 == myNumber) { /* ... */} if (notMatch == another) { /* ... */} if (notMatch) { /* ... */} ### Contributed by [​](https://ast-grep.github.io/catalog/c/yoda-condition.html#contributed-by) Inspired by this [thread](https://x.com/cocoa1han/status/1763020689303581141) --- # ast-grep [Skip to content](https://ast-grep.github.io/catalog/c/rewrite-method-to-function-call.html#VPContent) Return to top Are you an LLM? You can read better optimized documentation at /catalog/c/rewrite-method-to-function-call.md for this page in Markdown format Rewrite Method to Function Call Has Fix [​](https://ast-grep.github.io/catalog/c/rewrite-method-to-function-call.html#rewrite-method-to-function-call) ------------------------------------------------------------------------------------------------------------------------------------------------------- * [Playground Link](https://ast-grep.github.io/playground.html#eyJtb2RlIjoiQ29uZmlnIiwibGFuZyI6ImMiLCJxdWVyeSI6IiRDT1VOVCA9ICRcbiIsInJld3JpdGUiOiIiLCJjb25maWciOiJpZDogbWV0aG9kX3JlY2VpdmVyXG5ydWxlOlxuICBwYXR0ZXJuOiAkUi4kTUVUSE9EKCQkJEFSR1MpXG50cmFuc2Zvcm06XG4gIE1BWUJFX0NPTU1BOlxuICAgIHJlcGxhY2U6XG4gICAgICBzb3VyY2U6ICQkJEFSR1NcbiAgICAgIHJlcGxhY2U6ICdeLisnXG4gICAgICBieTogJywgJ1xuZml4OlxuICAkTUVUSE9EKCYkUiRNQVlCRV9DT01NQSQkJEFSR1MpXG4iLCJzb3VyY2UiOiJ2b2lkIHRlc3RfZnVuYygpIHtcbiAgICBzb21lX3N0cnVjdC0+ZmllbGQubWV0aG9kKCk7XG4gICAgc29tZV9zdHJ1Y3QtPmZpZWxkLm90aGVyX21ldGhvZCgxLCAyLCAzKTtcbn0ifQ==) ### Description [​](https://ast-grep.github.io/catalog/c/rewrite-method-to-function-call.html#description) In C, there is no built-in support for object-oriented programming, but some programmers use structs and function pointers to simulate classes and methods. However, this style can have some drawbacks, such as: * extra memory allocation and deallocation for the struct and the function pointer. * indirection overhead when calling the function pointer. A possible alternative is to use a plain function call with the struct pointer as the first argument. ### YAML [​](https://ast-grep.github.io/catalog/c/rewrite-method-to-function-call.html#yaml) yaml id: method_receiver language: c rule: pattern: $R.$METHOD($$$ARGS) transform: MAYBE_COMMA: replace: source: $$$ARGS replace: '^.+' by: ', ' fix: $METHOD(&$R$MAYBE_COMMA$$$ARGS) ### Example [​](https://ast-grep.github.io/catalog/c/rewrite-method-to-function-call.html#example) c void test_func() { some_struct->field.method(); some_struct->field.other_method(1, 2, 3); } ### Diff [​](https://ast-grep.github.io/catalog/c/rewrite-method-to-function-call.html#diff) c void test_func() { some_struct->field.method(); method(&some_struct->field); some_struct->field.other_method(1, 2, 3); other_method(&some_struct->field, 1, 2, 3); } ### Contributed by [​](https://ast-grep.github.io/catalog/c/rewrite-method-to-function-call.html#contributed-by) [Surma](https://twitter.com/DasSurma) , adapted from the [original tweet](https://twitter.com/DasSurma/status/1706086320051794217) --- # ast-grep [Skip to content](https://ast-grep.github.io/catalog/go/defer-func-call-antipattern.html#VPContent) Return to top Are you an LLM? You can read better optimized documentation at /catalog/go/defer-func-call-antipattern.md for this page in Markdown format Detect problematic defer statements with function calls [​](https://ast-grep.github.io/catalog/go/defer-func-call-antipattern.html#detect-problematic-defer-statements-with-function-calls) -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- * [Playground Link](https://ast-grep.github.io/playground.html#eyJtb2RlIjoiUGF0Y2giLCJsYW5nIjoiZ28iLCJxdWVyeSI6InsgXG4gICAgZGVmZXIgJEEuJEIodCwgZmFpbHBvaW50LiRNKCQkJCkpIFxufSIsInJld3JpdGUiOiIiLCJzdHJpY3RuZXNzIjoic21hcnQiLCJzZWxlY3RvciI6ImRlZmVyX3N0YXRlbWVudCIsImNvbmZpZyI6InJ1bGU6XG4iLCJzb3VyY2UiOiJmdW5jIFRlc3RJc3N1ZTE2Njk2KHQgKnRlc3RpbmcuVCkge1xuXHRhbGFybVJhdGlvIDo9IHZhcmRlZi5NZW1vcnlVc2FnZUFsYXJtUmF0aW8uTG9hZCgpXG5cdHZhcmRlZi5NZW1vcnlVc2FnZUFsYXJtUmF0aW8uU3RvcmUoMC4wKVxuXHRkZWZlciB2YXJkZWYuTWVtb3J5VXNhZ2VBbGFybVJhdGlvLlN0b3JlKGFsYXJtUmF0aW8pXG5cdHJlcXVpcmUuTm9FcnJvcih0LCBmYWlscG9pbnQuRW5hYmxlKFwiZ2l0aHViLmNvbS9waW5nY2FwL3RpZGIvcGtnL2V4ZWN1dG9yL3NvcnRleGVjL3Rlc3RTb3J0ZWRSb3dDb250YWluZXJTcGlsbFwiLCBcInJldHVybih0cnVlKVwiKSlcblx0ZGVmZXIgcmVxdWlyZS5Ob0Vycm9yKHQsIFxuXHQgICBmYWlscG9pbnQuRGlzYWJsZShcblx0XHRcImdpdGh1Yi5jb20vcGluZ2NhcC90aWRiL3BrZy9leGVjdXRvci9zb3J0ZXhlYy90ZXN0U29ydGVkUm93Q29udGFpbmVyU3BpbGxcIlxuXHQpKVxuXHRyZXF1aXJlLk5vRXJyb3IodCwgXG5cdFx0ZmFpbHBvaW50LkVuYWJsZShcImdpdGh1Yi5jb20vcGluZ2NhcC90aWRiL3BrZy9leGVjdXRvci9qb2luL3Rlc3RSb3dDb250YWluZXJTcGlsbFwiLCBcInJldHVybih0cnVlKVwiKSlcblx0ZGVmZXIgcmVxdWlyZS5Ob0Vycm9yKHQsIFxuXHRcdGZhaWxwb2ludC5EaXNhYmxlKFwiZ2l0aHViLmNvbS9waW5nY2FwL3RpZGIvcGtnL2V4ZWN1dG9yL2pvaW4vdGVzdFJvd0NvbnRhaW5lclNwaWxsXCIpKVxufSJ9) ### Description [​](https://ast-grep.github.io/catalog/go/defer-func-call-antipattern.html#description) This rule detects a common anti-pattern in Go testing code where `defer` statements contain function calls with parameters that are evaluated immediately instead of when the defer executes. In Go, `defer` schedules a function call to be executed when the surrounding function returns. However, the **arguments to the deferred function are evaluated immediately** when the defer statement is encountered, not when the defer executes. This is particularly problematic when using assertion libraries in tests. For example: go defer require.NoError(t, failpoint.Disable("some/path")) In this case, `failpoint.Disable("some/path")` is called immediately when the defer statement is reached, not when the function exits. This means the failpoint is disabled right after being enabled, making the test ineffective. ### Pattern [​](https://ast-grep.github.io/catalog/go/defer-func-call-antipattern.html#pattern) shell ast-grep \ --lang go \ --pattern '{ defer $A.$B(t, failpoint.$M($$$)) } \ --selector defer_statement' ### Example [​](https://ast-grep.github.io/catalog/go/defer-func-call-antipattern.html#example) go func TestIssue16696(t *testing.T) { alarmRatio := vardef.MemoryUsageAlarmRatio.Load() vardef.MemoryUsageAlarmRatio.Store(0.0) defer vardef.MemoryUsageAlarmRatio.Store(alarmRatio) require.NoError(t, failpoint.Enable("github.com/pingcap/tidb/pkg/executor/sortexec/testSortedRowContainerSpill", "return(true)")) defer require.NoError(t, failpoint.Disable( "github.com/pingcap/tidb/pkg/executor/sortexec/testSortedRowContainerSpill" )) require.NoError(t, failpoint.Enable("github.com/pingcap/tidb/pkg/executor/join/testRowContainerSpill", "return(true)")) defer require.NoError(t, failpoint.Disable("github.com/pingcap/tidb/pkg/executor/join/testRowContainerSpill")) } ### Fix [​](https://ast-grep.github.io/catalog/go/defer-func-call-antipattern.html#fix) The correct way to defer a function with parameters is to wrap it in an anonymous function: go defer func() { require.NoError(t, failpoint.Disable("some/path")) }() ### Contributed by [​](https://ast-grep.github.io/catalog/go/defer-func-call-antipattern.html#contributed-by) Inspired by [YangKeao's tweet](https://x.com/YangKeao/status/1671420857565212672) about this common pitfall in TiDB codebase. --- # Cpp | ast-grep [Skip to content](https://ast-grep.github.io/catalog/cpp/#VPContent) Return to top Are you an LLM? You can read better optimized documentation at /catalog/cpp.md for this page in Markdown format Cpp [​](https://ast-grep.github.io/catalog/cpp/#cpp) ===================================================== This page curates a list of example ast-grep rules to check and to rewrite Cpp code. Reuse Cpp rules with C Cpp is a superset of C, so you can reuse Cpp rules with C code. The [`languageGlobs`](https://ast-grep.github.io/reference/sgconfig.html#languageglobs) option can force ast-grep to parse `.c` files as Cpp. Fix Format String Vulnerability Has Fix [​](https://ast-grep.github.io/catalog/cpp/#fix-format-string-vulnerability) --------------------------------------------------------------------------------------------------------------------- * [Playground Link](https://ast-grep.github.io/playground.html#eyJtb2RlIjoiQ29uZmlnIiwibGFuZyI6ImNwcCIsInF1ZXJ5IjoiIiwicmV3cml0ZSI6IiIsInN0cmljdG5lc3MiOiJzbWFydCIsInNlbGVjdG9yIjoiIiwiY29uZmlnIjoiaWQ6IGZpeC1mb3JtYXQtc2VjdXJpdHktZXJyb3Jcbmxhbmd1YWdlOiBDcHBcbnJ1bGU6XG4gIHBhdHRlcm46ICRQUklOVEYoJFMsICRWQVIpXG5jb25zdHJhaW50czpcbiAgUFJJTlRGOiAjIGEgZm9ybWF0IHN0cmluZyBmdW5jdGlvblxuICAgIHsgcmVnZXg6IFwiXnNwcmludGZ8ZnByaW50ZiRcIiB9XG4gIFZBUjogIyBub3QgYSBsaXRlcmFsIHN0cmluZ1xuICAgIG5vdDpcbiAgICAgIGFueTpcbiAgICAgIC0geyBraW5kOiBzdHJpbmdfbGl0ZXJhbCB9XG4gICAgICAtIHsga2luZDogY29uY2F0ZW5hdGVkX3N0cmluZyB9XG5maXg6ICRQUklOVEYoJFMsIFwiJXNcIiwgJFZBUilcbiIsInNvdXJjZSI6Ii8vIEVycm9yXG5mcHJpbnRmKHN0ZGVyciwgb3V0KTtcbnNwcmludGYoJmJ1ZmZlclsyXSwgb2JqLT5UZXh0KTtcbnNwcmludGYoYnVmMSwgVGV4dF9TdHJpbmcoVFhUX1dBSVRJTkdfRk9SX0NPTk5FQ1RJT05TKSk7XG4vLyBPS1xuZnByaW50ZihzdGRlcnIsIFwiJXNcIiwgb3V0KTtcbnNwcmludGYoJmJ1ZmZlclsyXSwgXCIlc1wiLCBvYmotPlRleHQpO1xuc3ByaW50ZihidWYxLCBcIiVzXCIsIFRleHRfU3RyaW5nKFRYVF9XQUlUSU5HX0ZPUl9DT05ORUNUSU9OUykpOyJ9) ### Description [​](https://ast-grep.github.io/catalog/cpp/#description) The [Format String exploit](https://owasp.org/www-community/attacks/Format_string_attack) occurs when the submitted data of an input string is evaluated as a command by the application. For example, using `sprintf(s, var)` can lead to format string vulnerabilities if `var` contains user-controlled data. This can be exploited to execute arbitrary code. By explicitly specifying the format string as `"%s"`, you ensure that `var` is treated as a string, mitigating this risk. ### YAML [​](https://ast-grep.github.io/catalog/cpp/#yaml) yaml id: fix-format-security-error language: Cpp rule: pattern: $PRINTF($S, $VAR) constraints: PRINTF: # a format string function { regex: "^sprintf|fprintf$" } VAR: # not a literal string not: any: - { kind: string_literal } - { kind: concatenated_string } fix: $PRINTF($S, "%s", $VAR) ### Example [​](https://ast-grep.github.io/catalog/cpp/#example) cpp // Error fprintf(stderr, out); sprintf(&buffer[2], obj->Text); sprintf(buf1, Text_String(TXT_WAITING_FOR_CONNECTIONS)); // OK fprintf(stderr, "%s", out); sprintf(&buffer[2], "%s", obj->Text); sprintf(buf1, "%s", Text_String(TXT_WAITING_FOR_CONNECTIONS)); ### Diff [​](https://ast-grep.github.io/catalog/cpp/#diff) js // Error fprintf(stderr, out); fprintf(stderr, "%s", out); sprintf(&buffer[2], obj->Text); sprintf(&buffer[2], "%s", obj->Text); sprintf(buf1, Text_String(TXT_WAITING_FOR_CONNECTIONS)); sprintf(buf1, "%s", Text_String(TXT_WAITING_FOR_CONNECTIONS)); // OK fprintf(stderr, "%s", out); sprintf(&buffer[2], "%s", obj->Text); sprintf(buf1, "%s", Text_String(TXT_WAITING_FOR_CONNECTIONS)); ### Contributed by [​](https://ast-grep.github.io/catalog/cpp/#contributed-by) [xiaoxiangmoe](https://github.com/xiaoxiangmoe) Find Struct Inheritance [​](https://ast-grep.github.io/catalog/cpp/#find-struct-inheritance) --------------------------------------------------------------------------------------------- * [Playground Link](https://ast-grep.github.io/playground.html#eyJtb2RlIjoiUGF0Y2giLCJsYW5nIjoiY3BwIiwicXVlcnkiOiJzdHJ1Y3QgJFNPTUVUSElORzogICRJTkhFUklUU19GUk9NIHsgJCQkQk9EWTsgfSIsInJld3JpdGUiOiIiLCJzdHJpY3RuZXNzIjoic21hcnQiLCJzZWxlY3RvciI6IiIsImNvbmZpZyI6IiIsInNvdXJjZSI6InN0cnVjdCBGb286IEJhciB7fTtcblxuc3RydWN0IEJhcjogQmF6IHtcbiAgaW50IGEsIGI7XG59In0=) ### Description [​](https://ast-grep.github.io/catalog/cpp/#description-1) ast-grep's pattern is AST based. A code snippet like `struct $SOMETHING: $INHERITS` will not work because it does not have a correct AST structure. The correct pattern should spell out the full syntax like `struct $SOMETHING: $INHERITS { $$$BODY; }`. Compare the ast structure below to see the difference, especially the `ERROR` node. You can also use the playground's pattern panel to debug. Wrong PatternCorrect Pattern shell ERROR $SOMETHING base_class_clause $INHERITS shell struct_specifier $SOMETHING base_class_clause $INHERITS field_declaration_list field_declaration $$$BODY If it is not possible to write a full pattern, [YAML rule](https://ast-grep.github.io/guide/rule-config.html) is a better choice. ### Pattern [​](https://ast-grep.github.io/catalog/cpp/#pattern) shell ast-grep --lang cpp --pattern ' struct $SOMETHING: $INHERITS { $$$BODY; }' ### Example [​](https://ast-grep.github.io/catalog/cpp/#example-1) cpp struct Bar: Baz { int a, b; } ### Contributed by [​](https://ast-grep.github.io/catalog/cpp/#contributed-by-1) Inspired by this [tweet](https://x.com/techno_bog/status/1885421768384331871) --- # ast-grep [Skip to content](https://ast-grep.github.io/catalog/go/find-func-declaration-with-prefix.html#VPContent) Return to top Are you an LLM? You can read better optimized documentation at /catalog/go/find-func-declaration-with-prefix.md for this page in Markdown format Find function declarations with names of certain pattern [​](https://ast-grep.github.io/catalog/go/find-func-declaration-with-prefix.html#find-function-declarations-with-names-of-certain-pattern) ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- * [Playground Link](https://ast-grep.github.io/playground.html#eyJtb2RlIjoiQ29uZmlnIiwibGFuZyI6ImdvIiwicXVlcnkiOiJyJ15bQS1aYS16MC05Xy1dKyciLCJyZXdyaXRlIjoiIiwiY29uZmlnIjoiaWQ6IHRlc3QtZnVuY3Rpb25zXG5sYW5ndWFnZTogZ29cbnJ1bGU6XG4gIGtpbmQ6IGZ1bmN0aW9uX2RlY2xhcmF0aW9uXG4gIGhhczpcbiAgICBmaWVsZDogbmFtZVxuICAgIHJlZ2V4OiBUZXN0LipcbiIsInNvdXJjZSI6InBhY2thZ2UgYWJzXG5pbXBvcnQgXCJ0ZXN0aW5nXCJcbmZ1bmMgVGVzdEFicyh0ICp0ZXN0aW5nLlQpIHtcbiAgICBnb3QgOj0gQWJzKC0xKVxuICAgIGlmIGdvdCAhPSAxIHtcbiAgICAgICAgdC5FcnJvcmYoXCJBYnMoLTEpID0gJWQ7IHdhbnQgMVwiLCBnb3QpXG4gICAgfVxufVxuIn0=) ### Description [​](https://ast-grep.github.io/catalog/go/find-func-declaration-with-prefix.html#description) ast-grep can find function declarations by their names. But not all names can be matched by a meta variable pattern. For instance, you cannot use a meta variable pattern to find function declarations whose names start with a specific prefix, e.g. `TestAbs` with the prefix `Test`. Attempting `Test$_` will fail because it is not a valid syntax. Instead, you can use a [YAML rule](https://ast-grep.github.io/reference/rule.html) to use the [`regex`](https://ast-grep.github.io/guide/rule-config/atomic-rule.html#regex) atomic rule. ### YAML [​](https://ast-grep.github.io/catalog/go/find-func-declaration-with-prefix.html#yaml) yaml id: test-functions language: go rule: kind: function_declaration has: field: name regex: Test.* ### Example [​](https://ast-grep.github.io/catalog/go/find-func-declaration-with-prefix.html#example) go package abs import "testing" func TestAbs(t *testing.T) { got := Abs(-1) if got != 1 { t.Errorf("Abs(-1) = %d; want 1", got) } } ### Contributed by [​](https://ast-grep.github.io/catalog/go/find-func-declaration-with-prefix.html#contributed-by) [kevinkjt2000](https://twitter.com/kevinkjt2000) on [Discord](https://discord.com/invite/4YZjf6htSQ) . --- # ast-grep [Skip to content](https://ast-grep.github.io/catalog/cpp/fix-format-vuln.html#VPContent) Return to top Are you an LLM? You can read better optimized documentation at /catalog/cpp/fix-format-vuln.md for this page in Markdown format Fix Format String Vulnerability Has Fix [​](https://ast-grep.github.io/catalog/cpp/fix-format-vuln.html#fix-format-string-vulnerability) ----------------------------------------------------------------------------------------------------------------------------------------- * [Playground Link](https://ast-grep.github.io/playground.html#eyJtb2RlIjoiQ29uZmlnIiwibGFuZyI6ImNwcCIsInF1ZXJ5IjoiIiwicmV3cml0ZSI6IiIsInN0cmljdG5lc3MiOiJzbWFydCIsInNlbGVjdG9yIjoiIiwiY29uZmlnIjoiaWQ6IGZpeC1mb3JtYXQtc2VjdXJpdHktZXJyb3Jcbmxhbmd1YWdlOiBDcHBcbnJ1bGU6XG4gIHBhdHRlcm46ICRQUklOVEYoJFMsICRWQVIpXG5jb25zdHJhaW50czpcbiAgUFJJTlRGOiAjIGEgZm9ybWF0IHN0cmluZyBmdW5jdGlvblxuICAgIHsgcmVnZXg6IFwiXnNwcmludGZ8ZnByaW50ZiRcIiB9XG4gIFZBUjogIyBub3QgYSBsaXRlcmFsIHN0cmluZ1xuICAgIG5vdDpcbiAgICAgIGFueTpcbiAgICAgIC0geyBraW5kOiBzdHJpbmdfbGl0ZXJhbCB9XG4gICAgICAtIHsga2luZDogY29uY2F0ZW5hdGVkX3N0cmluZyB9XG5maXg6ICRQUklOVEYoJFMsIFwiJXNcIiwgJFZBUilcbiIsInNvdXJjZSI6Ii8vIEVycm9yXG5mcHJpbnRmKHN0ZGVyciwgb3V0KTtcbnNwcmludGYoJmJ1ZmZlclsyXSwgb2JqLT5UZXh0KTtcbnNwcmludGYoYnVmMSwgVGV4dF9TdHJpbmcoVFhUX1dBSVRJTkdfRk9SX0NPTk5FQ1RJT05TKSk7XG4vLyBPS1xuZnByaW50ZihzdGRlcnIsIFwiJXNcIiwgb3V0KTtcbnNwcmludGYoJmJ1ZmZlclsyXSwgXCIlc1wiLCBvYmotPlRleHQpO1xuc3ByaW50ZihidWYxLCBcIiVzXCIsIFRleHRfU3RyaW5nKFRYVF9XQUlUSU5HX0ZPUl9DT05ORUNUSU9OUykpOyJ9) ### Description [​](https://ast-grep.github.io/catalog/cpp/fix-format-vuln.html#description) The [Format String exploit](https://owasp.org/www-community/attacks/Format_string_attack) occurs when the submitted data of an input string is evaluated as a command by the application. For example, using `sprintf(s, var)` can lead to format string vulnerabilities if `var` contains user-controlled data. This can be exploited to execute arbitrary code. By explicitly specifying the format string as `"%s"`, you ensure that `var` is treated as a string, mitigating this risk. ### YAML [​](https://ast-grep.github.io/catalog/cpp/fix-format-vuln.html#yaml) yaml id: fix-format-security-error language: Cpp rule: pattern: $PRINTF($S, $VAR) constraints: PRINTF: # a format string function { regex: "^sprintf|fprintf$" } VAR: # not a literal string not: any: - { kind: string_literal } - { kind: concatenated_string } fix: $PRINTF($S, "%s", $VAR) ### Example [​](https://ast-grep.github.io/catalog/cpp/fix-format-vuln.html#example) cpp // Error fprintf(stderr, out); sprintf(&buffer[2], obj->Text); sprintf(buf1, Text_String(TXT_WAITING_FOR_CONNECTIONS)); // OK fprintf(stderr, "%s", out); sprintf(&buffer[2], "%s", obj->Text); sprintf(buf1, "%s", Text_String(TXT_WAITING_FOR_CONNECTIONS)); ### Diff [​](https://ast-grep.github.io/catalog/cpp/fix-format-vuln.html#diff) js // Error fprintf(stderr, out); fprintf(stderr, "%s", out); sprintf(&buffer[2], obj->Text); sprintf(&buffer[2], "%s", obj->Text); sprintf(buf1, Text_String(TXT_WAITING_FOR_CONNECTIONS)); sprintf(buf1, "%s", Text_String(TXT_WAITING_FOR_CONNECTIONS)); // OK fprintf(stderr, "%s", out); sprintf(&buffer[2], "%s", obj->Text); sprintf(buf1, "%s", Text_String(TXT_WAITING_FOR_CONNECTIONS)); ### Contributed by [​](https://ast-grep.github.io/catalog/cpp/fix-format-vuln.html#contributed-by) [xiaoxiangmoe](https://github.com/xiaoxiangmoe) --- # ast-grep [Skip to content](https://ast-grep.github.io/catalog/go/match-function-call.html#VPContent) Return to top Are you an LLM? You can read better optimized documentation at /catalog/go/match-function-call.md for this page in Markdown format Match Function Call in Golang [​](https://ast-grep.github.io/catalog/go/match-function-call.html#match-function-call-in-golang) -------------------------------------------------------------------------------------------------------------------------------- * [Playground Link](https://ast-grep.github.io/playground.html#eyJtb2RlIjoiQ29uZmlnIiwibGFuZyI6ImdvIiwicXVlcnkiOiJhd2FpdCAkQSIsInJld3JpdGUiOiJ0cnkge1xuICAgIGF3YWl0ICRBXG59IGNhdGNoKGUpIHtcbiAgICAvLyB0b2RvXG59IiwiY29uZmlnIjoicnVsZTpcbiAgcGF0dGVybjpcbiAgICBjb250ZXh0OiAnZnVuYyB0KCkgeyBmbXQuUHJpbnRsbigkJCRBKSB9J1xuICAgIHNlbGVjdG9yOiBjYWxsX2V4cHJlc3Npb25cbiIsInNvdXJjZSI6ImZ1bmMgbWFpbigpIHtcbiAgICBmbXQuUHJpbnRsbihcIk9LXCIpXG59In0=) ### Description [​](https://ast-grep.github.io/catalog/go/match-function-call.html#description) One of the common questions of ast-grep is to match function calls in Golang. A plain pattern like `fmt.Println($A)` will not work. This is because Golang syntax also allows type conversions, e.g. `int(3.14)`, that look like function calls. Tree-sitter, ast-grep's parser, will prefer parsing `func_call(arg)` as a type conversion instead of a call expression. To avoid this ambiguity, ast-grep lets us write a [contextual pattern](https://ast-grep.github.io/guide/rule-config/atomic-rule.html#pattern) , which is a pattern inside a larger code snippet. We can use `context` to write a pattern like this: `func t() { fmt.Println($A) }`. Then, we can use the selector `call_expression` to match only function calls. Please also read the [deep dive](https://ast-grep.github.io/advanced/pattern-parse.html) on [ambiguous pattern](https://ast-grep.github.io/advanced/pattern-parse.html#ambiguous-pattern-code) . ### YAML [​](https://ast-grep.github.io/catalog/go/match-function-call.html#yaml) yaml id: match-function-call language: go rule: pattern: context: 'func t() { fmt.Println($A) }' selector: call_expression ### Example [​](https://ast-grep.github.io/catalog/go/match-function-call.html#example) go func main() { fmt.Println("OK") } ### Contributed by [​](https://ast-grep.github.io/catalog/go/match-function-call.html#contributed-by) Inspired by [QuantumGhost](https://github.com/QuantumGhost) from [ast-grep/ast-grep#646](https://github.com/ast-grep/ast-grep/issues/646) --- # ast-grep [Skip to content](https://ast-grep.github.io/catalog/html/extract-i18n-key.html#VPContent) Return to top Are you an LLM? You can read better optimized documentation at /catalog/html/extract-i18n-key.md for this page in Markdown format Extract i18n Keys Has Fix [​](https://ast-grep.github.io/catalog/html/extract-i18n-key.html#extract-i18n-keys) --------------------------------------------------------------------------------------------------------------- * [Playground Link](https://ast-grep.github.io/playground.html#eyJtb2RlIjoiQ29uZmlnIiwibGFuZyI6Imh0bWwiLCJxdWVyeSI6IiIsInJld3JpdGUiOiIiLCJzdHJpY3RuZXNzIjoicmVsYXhlZCIsInNlbGVjdG9yIjoiIiwiY29uZmlnIjoicnVsZTpcbiAga2luZDogdGV4dFxuICBwYXR0ZXJuOiAkVFxuICBub3Q6XG4gICAgcmVnZXg6ICdcXHtcXHsuKlxcfVxcfSdcbmZpeDogXCJ7eyAkKCckVCcpIH19XCIiLCJzb3VyY2UiOiI8dGVtcGxhdGU+XG4gIDxzcGFuPkhlbGxvPC9zcGFuPlxuICA8c3Bhbj57eyB0ZXh0IH19PC9zcGFuPlxuPC90ZW1wbGF0ZT4ifQ==) ### Description [​](https://ast-grep.github.io/catalog/html/extract-i18n-key.html#description) It is tedious to manually find and replace all the text in the template with i18n keys. This rule helps to extract static text into i18n keys. Dynamic text, e.g. mustache syntax, will be skipped. In practice, you may want to map the extracted text to a key in a dictionary file. While this rule only demonstrates the extraction part, further mapping process can be done via a script reading the output of ast-grep's [`--json`](https://ast-grep.github.io/guide/tools/json.html) mode, or using [`@ast-grep/napi`](https://ast-grep.github.io/guide/api-usage/js-api.html) . ### YAML [​](https://ast-grep.github.io/catalog/html/extract-i18n-key.html#yaml) yaml id: extract-i18n-key language: html rule: kind: text pattern: $T # skip dynamic text in mustache syntax not: { regex: '\{\{.*\}\}' } fix: "{{ $('$T') }}" ### Example [​](https://ast-grep.github.io/catalog/html/extract-i18n-key.html#example) html ### Diff [​](https://ast-grep.github.io/catalog/html/extract-i18n-key.html#diff) html ### Contributed by [​](https://ast-grep.github.io/catalog/html/extract-i18n-key.html#contributed-by) Inspired by [Vue.js RFC](https://github.com/vuejs/rfcs/discussions/705#discussion-7255672) --- # ast-grep [Skip to content](https://ast-grep.github.io/catalog/go/unmarshal-tag-is-dash.html#VPContent) Return to top Are you an LLM? You can read better optimized documentation at /catalog/go/unmarshal-tag-is-dash.md for this page in Markdown format Detect problematic JSON tags with dash prefix [​](https://ast-grep.github.io/catalog/go/unmarshal-tag-is-dash.html#detect-problematic-json-tags-with-dash-prefix) ------------------------------------------------------------------------------------------------------------------------------------------------------------------ * [Playground Link](https://ast-grep.github.io/playground.html#eyJtb2RlIjoiQ29uZmlnIiwibGFuZyI6ImdvIiwicXVlcnkiOiJgJFRBR2AiLCJyZXdyaXRlIjoiIiwic3RyaWN0bmVzcyI6InNtYXJ0Iiwic2VsZWN0b3IiOiIiLCJjb25maWciOiJpZDogdW5tYXJzaGFsLXRhZy1pcy1kYXNoXG5zZXZlcml0eTogZXJyb3Jcbm1lc3NhZ2U6IFN0cnVjdCBmaWVsZCBjYW4gYmUgZGVjb2RlZCB3aXRoIHRoZSBgLWAga2V5IGJlY2F1c2UgdGhlIEpTT04gdGFnXG4gIHN0YXJ0cyB3aXRoIGEgYC1gIGJ1dCBpcyBmb2xsb3dlZCBieSBhIGNvbW1hLlxucnVsZTpcbiAgcGF0dGVybjogJ2AkVEFHYCdcbiAgaW5zaWRlOlxuICAgIGtpbmQ6IGZpZWxkX2RlY2xhcmF0aW9uXG5jb25zdHJhaW50czpcbiAgVEFHOiBcbiAgICByZWdleDoganNvbjpcIi0sLipcIiIsInNvdXJjZSI6InBhY2thZ2UgbWFpblxuXG50eXBlIFRlc3RTdHJ1Y3QxIHN0cnVjdCB7XG5cdC8vIG9rOiB1bm1hcnNoYWwtdGFnLWlzLWRhc2hcblx0QSBzdHJpbmcgYGpzb246XCJpZFwiYFxufVxuXG50eXBlIFRlc3RTdHJ1Y3QyIHN0cnVjdCB7XG5cdC8vIHJ1bGVpZDogdW5tYXJzaGFsLXRhZy1pcy1kYXNoXG5cdEIgc3RyaW5nIGBqc29uOlwiLSxvbWl0ZW1wdHlcImBcbn1cblxudHlwZSBUZXN0U3RydWN0MyBzdHJ1Y3Qge1xuXHQvLyBydWxlaWQ6IHVubWFyc2hhbC10YWctaXMtZGFzaFxuXHRDIHN0cmluZyBganNvbjpcIi0sMTIzXCJgXG59XG5cbnR5cGUgVGVzdFN0cnVjdDQgc3RydWN0IHtcblx0Ly8gcnVsZWlkOiB1bm1hcnNoYWwtdGFnLWlzLWRhc2hcblx0RCBzdHJpbmcgYGpzb246XCItLFwiYFxufSJ9) ### Description [​](https://ast-grep.github.io/catalog/go/unmarshal-tag-is-dash.html#description) This rule detects a security vulnerability in Go's JSON unmarshaling. When a struct field has a JSON tag that starts with `-,`, it can be unexpectedly unmarshaled with the `-` key. According to the [Go documentation](https://pkg.go.dev/encoding/json#Marshal) , if the field tag is `-`, the field should be omitted. However, a field with name `-` can still be unmarshaled using the tag `-,`. This creates a security issue where developers think they are preventing a field from being unmarshaled (like `IsAdmin` in authentication), but attackers can still set that field by providing the `-` key in JSON input. go type User struct { Username string `json:"username,omitempty"` Password string `json:"password,omitempty"` IsAdmin bool `json:"-,omitempty"` // Intended to prevent marshaling } // This still works and sets IsAdmin to true! json.Unmarshal([]byte(`{"-": true}`), &user) // Result: main.User{Username:"", Password:"", IsAdmin:true} ### YAML [​](https://ast-grep.github.io/catalog/go/unmarshal-tag-is-dash.html#yaml) yaml id: unmarshal-tag-is-dash severity: error message: Struct field can be decoded with the `-` key because the JSON tag starts with a `-` but is followed by a comma. rule: pattern: '`$TAG`' inside: kind: field_declaration constraints: TAG: regex: json:"-,.*" ### Example [​](https://ast-grep.github.io/catalog/go/unmarshal-tag-is-dash.html#example) go package main type TestStruct1 struct { A string `json:"id"` // ok } type TestStruct2 struct { B string `json:"-,omitempty"` // wrong } type TestStruct3 struct { C string `json:"-,123"` // wrong } type TestStruct4 struct { D string `json:"-,"` // wrong } ### Fix [​](https://ast-grep.github.io/catalog/go/unmarshal-tag-is-dash.html#fix) To properly omit a field from JSON marshaling/unmarshaling, use just `-` without a comma: go type User struct { Username string `json:"username,omitempty"` Password string `json:"password,omitempty"` IsAdmin bool `json:"-"` // Correctly prevents marshaling/unmarshaling } ### Contributed by [​](https://ast-grep.github.io/catalog/go/unmarshal-tag-is-dash.html#contributed-by) Inspired by [Trail of Bits blog post](https://blog.trailofbits.com/2025/06/17/unexpected-security-footguns-in-gos-parsers/) and their [public Semgrep rule](https://semgrep.dev/playground/r/trailofbits.go.unmarshal-tag-is-dash) . --- # ast-grep [Skip to content](https://ast-grep.github.io/catalog/go/match-package-import.html#VPContent) Return to top Are you an LLM? You can read better optimized documentation at /catalog/go/match-package-import.md for this page in Markdown format Match package import in Golang [​](https://ast-grep.github.io/catalog/go/match-package-import.html#match-package-import-in-golang) ----------------------------------------------------------------------------------------------------------------------------------- * [Playground Link](https://ast-grep.github.io/playground.html#eyJtb2RlIjoiQ29uZmlnIiwibGFuZyI6ImdvIiwicXVlcnkiOiIiLCJyZXdyaXRlIjoiIiwic3RyaWN0bmVzcyI6InNtYXJ0Iiwic2VsZWN0b3IiOiIiLCJjb25maWciOiJpZDogbWF0Y2gtcGFja2FnZS1pbXBvcnRcbmxhbmd1YWdlOiBnb1xucnVsZTpcbiAga2luZDogaW1wb3J0X3NwZWNcbiAgaGFzOlxuICAgIHJlZ2V4OiBnaXRodWIuY29tL2dvbGFuZy1qd3Qvand0Iiwic291cmNlIjoicGFja2FnZSBtYWluXG5cbmltcG9ydCAoXG5cdFwiZm10XCJcblx0XCJnaXRodWIuY29tL2dvbGFuZy1qd3Qvand0XCIgIC8vIFRoaXMgbWF0Y2hlcyB0aGUgQVNUIHJ1bGVcbilcblxuZnVuYyBtYWluKCkge1xuXHQvLyBDcmVhdGUgYSBuZXcgdG9rZW5cblx0dG9rZW4gOj0gand0Lk5ldyhqd3QuU2lnbmluZ01ldGhvZEhTMjU2KVxuXHRcblx0Ly8gQWRkIHNvbWUgY2xhaW1zXG5cdHRva2VuLkNsYWltcyA9IGp3dC5NYXBDbGFpbXN7XG5cdFx0XCJ1c2VyXCI6IFwiYWxpY2VcIixcblx0XHRcInJvbGVcIjogXCJhZG1pblwiLFxuXHR9XG5cdFxuXHQvLyBTaWduIHRoZSB0b2tlblxuXHR0b2tlblN0cmluZywgZXJyIDo9IHRva2VuLlNpZ25lZFN0cmluZyhbXWJ5dGUoXCJteS1zZWNyZXRcIikpXG5cdGlmIGVyciAhPSBuaWwge1xuXHRcdGZtdC5QcmludGYoXCJFcnJvciBzaWduaW5nIHRva2VuOiAldlxcblwiLCBlcnIpXG5cdFx0cmV0dXJuXG5cdH1cblx0XG5cdGZtdC5QcmludGYoXCJHZW5lcmF0ZWQgdG9rZW46ICVzXFxuXCIsIHRva2VuU3RyaW5nKVxufSJ9) ### Description [​](https://ast-grep.github.io/catalog/go/match-package-import.html#description) A generic rule template for detecting imports of specific packages in Go source code. This rule can be customized to match any package by modifying the regex pattern, making it useful for security auditing, dependency management, and compliance checking. This rule identifies Go import statements based on the configured regex pattern, including: Direct imports: `import "package/name"` Versioned imports: `import "package/name/v4"` Subpackage imports: `import "package/name/subpkg"` Grouped imports within `import () blocks` ### YAML [​](https://ast-grep.github.io/catalog/go/match-package-import.html#yaml) yaml id: match-package-import language: go rule: kind: import_spec has: regex: PACKAGE_PATTERN_HERE ### Example [​](https://ast-grep.github.io/catalog/go/match-package-import.html#example) JWT Library Detection go package main import ( "fmt" "github.com/golang-jwt/jwt" // This matches the AST rule ) func main() { token := jwt.New(jwt.SigningMethodHS256) // Create a new token // Add some claims token.Claims = jwt.MapClaims{"user": "alice", "role": "admin"} tokenString, err := token.SignedString([]byte("my-secret")) // Sign the token if err != nil { fmt.Printf("Error signing token: %v\n", err) return } fmt.Printf("Generated token: %s\n", tokenString) } ### Contributed by [​](https://ast-grep.github.io/catalog/go/match-package-import.html#contributed-by) [Sudesh Gutta](https://github.com/sudeshgutta) --- # HTML | ast-grep [Skip to content](https://ast-grep.github.io/catalog/html/#VPContent) Return to top Are you an LLM? You can read better optimized documentation at /catalog/html.md for this page in Markdown format HTML [​](https://ast-grep.github.io/catalog/html/#html) ======================================================== This page curates a list of example ast-grep rules to check and to rewrite HTML code. Use HTML parser for frameworks You can leverage the [`languageGlobs`](https://ast-grep.github.io/reference/sgconfig.html#languageglobs) option to parse framework files as plain HTML, such as `vue`, `svelte`, and `astro`. **Caveat**: This approach may not parse framework-specific syntax, like Astro's [frontmatter script](https://docs.astro.build/en/basics/astro-components/#the-component-script) or [Svelte control flow](https://svelte.dev/docs/svelte/if) . You will need to load [custom languages](https://ast-grep.github.io/advanced/custom-language.html) for such cases. Upgrade Ant Design Vue Has Fix [​](https://ast-grep.github.io/catalog/html/#upgrade-ant-design-vue) ---------------------------------------------------------------------------------------------------- * [Playground Link](https://ast-grep.github.io/playground.html#eyJtb2RlIjoiQ29uZmlnIiwibGFuZyI6Imh0bWwiLCJxdWVyeSI6IiIsInJld3JpdGUiOiIiLCJzdHJpY3RuZXNzIjoicmVsYXhlZCIsInNlbGVjdG9yIjoiIiwiY29uZmlnIjoidXRpbHM6XG4gIGluc2lkZS10YWc6XG4gICAgaW5zaWRlOlxuICAgICAga2luZDogZWxlbWVudCBcbiAgICAgIHN0b3BCeTogeyBraW5kOiBlbGVtZW50IH1cbiAgICAgIGhhczpcbiAgICAgICAgc3RvcEJ5OiB7IGtpbmQ6IHRhZ19uYW1lIH1cbiAgICAgICAga2luZDogdGFnX25hbWVcbiAgICAgICAgcGF0dGVybjogJFRBR19OQU1FXG5ydWxlOlxuICBraW5kOiBhdHRyaWJ1dGVfbmFtZVxuICByZWdleDogOnZpc2libGVcbiAgbWF0Y2hlczogaW5zaWRlLXRhZyAgXG5maXg6IDpvcGVuXG5jb25zdHJhaW50czpcbiAgVEFHX05BTUU6XG4gICAgcmVnZXg6IGEtbW9kYWx8YS10b29sdGlwIiwic291cmNlIjoiPHRlbXBsYXRlPlxuICA8YS1tb2RhbCA6dmlzaWJsZT1cInZpc2libGVcIj5jb250ZW50PC9hLW1vZGFsPlxuICA8YS10b29sdGlwIDp2aXNpYmxlPVwidmlzaWJsZVwiIC8+XG4gIDxhLXRhZyA6dmlzaWJsZT1cInZpc2libGVcIj50YWc8L2EtdGFnPlxuPC90ZW1wbGF0ZT4ifQ==) ### Description [​](https://ast-grep.github.io/catalog/html/#description) ast-grep can be used to upgrade Vue template using the HTML parser. This rule is an example to upgrade [one breaking change](https://next.antdv.com/docs/vue/migration-v4#component-api-adjustment) in [Ant Design Vue](https://next.antdv.com/components/overview) from v3 to v4, unifying the controlled visible API of the component popup. It is designed to identify and replace the `visible` attribute with the `open` attribute for specific components like `a-modal` and `a-tooltip`. Note the rule should not replace other visible attributes that are not related to the component popup like `a-tag`. The rule can be broken down into the following steps: 1. Find the target attribute name by `kind` and `regex` 2. Find the attribute's enclosing element using `inside`, and get its tag name 3. Ensure the tag name is related to popup components, using constraints ### YAML [​](https://ast-grep.github.io/catalog/html/#yaml) yaml id: upgrade-ant-design-vue language: HTML utils: inside-tag: # find the enclosing element of the attribute inside: kind: element stopBy: { kind: element } # only the closest element # find the tag name and store it in metavar has: stopBy: { kind: tag_name } kind: tag_name pattern: $TAG_NAME rule: # find the target attribute_name kind: attribute_name regex: :visible # find the element matches: inside-tag # ensure it only matches modal/tooltip but not tag constraints: TAG_NAME: regex: a-modal|a-tooltip fix: :open ### Example [​](https://ast-grep.github.io/catalog/html/#example) html ### Diff [​](https://ast-grep.github.io/catalog/html/#diff) html ### Contributed by [​](https://ast-grep.github.io/catalog/html/#contributed-by) Inspired by [Vue.js RFC](https://github.com/vuejs/rfcs/discussions/705#discussion-7255672) Extract i18n Keys Has Fix [​](https://ast-grep.github.io/catalog/html/#extract-i18n-keys) ------------------------------------------------------------------------------------------ * [Playground Link](https://ast-grep.github.io/playground.html#eyJtb2RlIjoiQ29uZmlnIiwibGFuZyI6Imh0bWwiLCJxdWVyeSI6IiIsInJld3JpdGUiOiIiLCJzdHJpY3RuZXNzIjoicmVsYXhlZCIsInNlbGVjdG9yIjoiIiwiY29uZmlnIjoicnVsZTpcbiAga2luZDogdGV4dFxuICBwYXR0ZXJuOiAkVFxuICBub3Q6XG4gICAgcmVnZXg6ICdcXHtcXHsuKlxcfVxcfSdcbmZpeDogXCJ7eyAkKCckVCcpIH19XCIiLCJzb3VyY2UiOiI8dGVtcGxhdGU+XG4gIDxzcGFuPkhlbGxvPC9zcGFuPlxuICA8c3Bhbj57eyB0ZXh0IH19PC9zcGFuPlxuPC90ZW1wbGF0ZT4ifQ==) ### Description [​](https://ast-grep.github.io/catalog/html/#description-1) It is tedious to manually find and replace all the text in the template with i18n keys. This rule helps to extract static text into i18n keys. Dynamic text, e.g. mustache syntax, will be skipped. In practice, you may want to map the extracted text to a key in a dictionary file. While this rule only demonstrates the extraction part, further mapping process can be done via a script reading the output of ast-grep's [`--json`](https://ast-grep.github.io/guide/tools/json.html) mode, or using [`@ast-grep/napi`](https://ast-grep.github.io/guide/api-usage/js-api.html) . ### YAML [​](https://ast-grep.github.io/catalog/html/#yaml-1) yaml id: extract-i18n-key language: html rule: kind: text pattern: $T # skip dynamic text in mustache syntax not: { regex: '\{\{.*\}\}' } fix: "{{ $('$T') }}" ### Example [​](https://ast-grep.github.io/catalog/html/#example-1) html ### Diff [​](https://ast-grep.github.io/catalog/html/#diff-1) html ### Contributed by [​](https://ast-grep.github.io/catalog/html/#contributed-by-1) Inspired by [Vue.js RFC](https://github.com/vuejs/rfcs/discussions/705#discussion-7255672) --- # ast-grep [Skip to content](https://ast-grep.github.io/catalog/html/upgrade-ant-design-vue.html#VPContent) Return to top Are you an LLM? You can read better optimized documentation at /catalog/html/upgrade-ant-design-vue.md for this page in Markdown format Upgrade Ant Design Vue Has Fix [​](https://ast-grep.github.io/catalog/html/upgrade-ant-design-vue.html#upgrade-ant-design-vue) ------------------------------------------------------------------------------------------------------------------------------- * [Playground Link](https://ast-grep.github.io/playground.html#eyJtb2RlIjoiQ29uZmlnIiwibGFuZyI6Imh0bWwiLCJxdWVyeSI6IiIsInJld3JpdGUiOiIiLCJzdHJpY3RuZXNzIjoicmVsYXhlZCIsInNlbGVjdG9yIjoiIiwiY29uZmlnIjoidXRpbHM6XG4gIGluc2lkZS10YWc6XG4gICAgaW5zaWRlOlxuICAgICAga2luZDogZWxlbWVudCBcbiAgICAgIHN0b3BCeTogeyBraW5kOiBlbGVtZW50IH1cbiAgICAgIGhhczpcbiAgICAgICAgc3RvcEJ5OiB7IGtpbmQ6IHRhZ19uYW1lIH1cbiAgICAgICAga2luZDogdGFnX25hbWVcbiAgICAgICAgcGF0dGVybjogJFRBR19OQU1FXG5ydWxlOlxuICBraW5kOiBhdHRyaWJ1dGVfbmFtZVxuICByZWdleDogOnZpc2libGVcbiAgbWF0Y2hlczogaW5zaWRlLXRhZyAgXG5maXg6IDpvcGVuXG5jb25zdHJhaW50czpcbiAgVEFHX05BTUU6XG4gICAgcmVnZXg6IGEtbW9kYWx8YS10b29sdGlwIiwic291cmNlIjoiPHRlbXBsYXRlPlxuICA8YS1tb2RhbCA6dmlzaWJsZT1cInZpc2libGVcIj5jb250ZW50PC9hLW1vZGFsPlxuICA8YS10b29sdGlwIDp2aXNpYmxlPVwidmlzaWJsZVwiIC8+XG4gIDxhLXRhZyA6dmlzaWJsZT1cInZpc2libGVcIj50YWc8L2EtdGFnPlxuPC90ZW1wbGF0ZT4ifQ==) ### Description [​](https://ast-grep.github.io/catalog/html/upgrade-ant-design-vue.html#description) ast-grep can be used to upgrade Vue template using the HTML parser. This rule is an example to upgrade [one breaking change](https://next.antdv.com/docs/vue/migration-v4#component-api-adjustment) in [Ant Design Vue](https://next.antdv.com/components/overview) from v3 to v4, unifying the controlled visible API of the component popup. It is designed to identify and replace the `visible` attribute with the `open` attribute for specific components like `a-modal` and `a-tooltip`. Note the rule should not replace other visible attributes that are not related to the component popup like `a-tag`. The rule can be broken down into the following steps: 1. Find the target attribute name by `kind` and `regex` 2. Find the attribute's enclosing element using `inside`, and get its tag name 3. Ensure the tag name is related to popup components, using constraints ### YAML [​](https://ast-grep.github.io/catalog/html/upgrade-ant-design-vue.html#yaml) yaml id: upgrade-ant-design-vue language: HTML utils: inside-tag: # find the enclosing element of the attribute inside: kind: element stopBy: { kind: element } # only the closest element # find the tag name and store it in metavar has: stopBy: { kind: tag_name } kind: tag_name pattern: $TAG_NAME rule: # find the target attribute_name kind: attribute_name regex: :visible # find the element matches: inside-tag # ensure it only matches modal/tooltip but not tag constraints: TAG_NAME: regex: a-modal|a-tooltip fix: :open ### Example [​](https://ast-grep.github.io/catalog/html/upgrade-ant-design-vue.html#example) html ### Diff [​](https://ast-grep.github.io/catalog/html/upgrade-ant-design-vue.html#diff) html ### Contributed by [​](https://ast-grep.github.io/catalog/html/upgrade-ant-design-vue.html#contributed-by) Inspired by [Vue.js RFC](https://github.com/vuejs/rfcs/discussions/705#discussion-7255672) --- # ast-grep [Skip to content](https://ast-grep.github.io/catalog/java/find-field-with-type.html#VPContent) Return to top Are you an LLM? You can read better optimized documentation at /catalog/java/find-field-with-type.md for this page in Markdown format Find Java field declarations of type String [​](https://ast-grep.github.io/catalog/java/find-field-with-type.html#find-java-field-declarations-of-type-string) --------------------------------------------------------------------------------------------------------------------------------------------------------------- * [Playground Link](https://ast-grep.github.io/playground.html#eyJtb2RlIjoiQ29uZmlnIiwibGFuZyI6ImphdmEiLCJxdWVyeSI6ImAkVEFHYCIsInJld3JpdGUiOiIiLCJzdHJpY3RuZXNzIjoic21hcnQiLCJzZWxlY3RvciI6IiIsImNvbmZpZyI6InJ1bGU6XG4gIGtpbmQ6IGZpZWxkX2RlY2xhcmF0aW9uXG4gIGhhczpcbiAgICBmaWVsZDogdHlwZVxuICAgIHJlZ2V4OiBeU3RyaW5nJCIsInNvdXJjZSI6IkBDb21wb25lbnRcbmNsYXNzIEFCQyBleHRlbmRzIE9iamVjdHtcbiAgICBAUmVzb3VyY2VcbiAgICBwcml2YXRlIGZpbmFsIFN0cmluZyB3aXRoX2Fubm87XG5cbiAgICBwcml2YXRlIGZpbmFsIFN0cmluZyB3aXRoX211bHRpX21vZDtcblxuICAgIHB1YmxpYyBTdHJpbmcgc2ltcGxlO1xufSJ9) ### Description [​](https://ast-grep.github.io/catalog/java/find-field-with-type.html#description) To extract all Java field names of type `String` is not as straightforward as one might think. A simple pattern like `String $F;` would only match fields declared without any modifiers or annotations. However, a pattern like `$MOD String $F;` cannot be correctly parsed by tree-sitter. Use playground pattern debugger to explore the AST You can use the [playground](https://ast-grep.github.io/playground.html#eyJtb2RlIjoiUGF0Y2giLCJsYW5nIjoiamF2YSIsInF1ZXJ5IjoiY2xhc3MgQUJDe1xuICAgJE1PRCBTdHJpbmcgdGVzdDtcbn0iLCJyZXdyaXRlIjoiIiwic3RyaWN0bmVzcyI6ImFzdCIsInNlbGVjdG9yIjoiIiwiY29uZmlnIjoicnVsZTpcbiAga2luZDogZmllbGRfZGVjbGFyYXRpb25cbiAgaGFzOlxuICAgIGZpZWxkOiB0eXBlXG4gICAgcmVnZXg6IF5TdHJpbmckIiwic291cmNlIjoiQENvbXBvbmVudFxuY2xhc3MgQUJDIGV4dGVuZHMgT2JqZWN0e1xuICAgIEBSZXNvdXJjZVxuICAgIHByaXZhdGUgZmluYWwgU3RyaW5nIHdpdGhfYW5ubztcblxuICAgIHByaXZhdGUgZmluYWwgU3RyaW5nIHdpdGhfbXVsdGlfbW9kO1xuXG4gICAgcHVibGljIFN0cmluZyBzaW1wbGU7XG59In0=) 's pattern tab to visualize the AST of `class A { $MOD String $F; }`. field_declaration $MOD variable_declarator identifier: String ERROR identifier: $F Tree-sitter does not think `$MOD` is a valid modifier, so it produces an `ERROR`. While the valid AST for code like `private String field;` produces different AST structures: field_declaration modifiers type_identifier variable_declarator identifier: field A more robust approach is to use a structural rule that targets `field_declaration` nodes and applies a `has` constraint on the `type` child node to match the type `String`. This method effectively captures fields regardless of their modifiers or annotations. ### YAML [​](https://ast-grep.github.io/catalog/java/find-field-with-type.html#yaml) yaml id: find-field-with-type language: java rule: kind: field_declaration has: field: type regex: ^String$ ### Example [​](https://ast-grep.github.io/catalog/java/find-field-with-type.html#example) java @Component class ABC extends Object{ @Resource private final String with_anno; private final String with_multi_mod; public String simple; } ### Contributed by [​](https://ast-grep.github.io/catalog/java/find-field-with-type.html#contributed-by) Inspired by the post [discussion](https://github.com/ast-grep/ast-grep/discussions/2195) --- # ast-grep [Skip to content](https://ast-grep.github.io/catalog/java/no-unused-vars.html#VPContent) Return to top Are you an LLM? You can read better optimized documentation at /catalog/java/no-unused-vars.md for this page in Markdown format No Unused Vars in Java Has Fix [​](https://ast-grep.github.io/catalog/java/no-unused-vars.html#no-unused-vars-in-java) ----------------------------------------------------------------------------------------------------------------------- * [Playground Link](https://ast-grep.github.io/playground.html#eyJtb2RlIjoiQ29uZmlnIiwibGFuZyI6ImphdmEiLCJxdWVyeSI6ImlmKHRydWUpeyQkJEJPRFl9IiwicmV3cml0ZSI6IiRDOiBMaXN0WyRUXSA9IHJlbGF0aW9uc2hpcCgkJCRBLCB1c2VsaXN0PVRydWUsICQkJEIpIiwic3RyaWN0bmVzcyI6InNtYXJ0Iiwic2VsZWN0b3IiOiIiLCJjb25maWciOiJpZDogbm8tdW51c2VkLXZhcnNcbnJ1bGU6XG4gICAga2luZDogbG9jYWxfdmFyaWFibGVfZGVjbGFyYXRpb25cbiAgICBhbGw6XG4gICAgICAgIC0gaGFzOlxuICAgICAgICAgICAgaGFzOlxuICAgICAgICAgICAgICAgIGtpbmQ6IGlkZW50aWZpZXJcbiAgICAgICAgICAgICAgICBwYXR0ZXJuOiAkSURFTlRcbiAgICAgICAgLSBub3Q6XG4gICAgICAgICAgICBwcmVjZWRlczpcbiAgICAgICAgICAgICAgICBzdG9wQnk6IGVuZFxuICAgICAgICAgICAgICAgIGhhczpcbiAgICAgICAgICAgICAgICAgICAgc3RvcEJ5OiBlbmRcbiAgICAgICAgICAgICAgICAgICAgYW55OlxuICAgICAgICAgICAgICAgICAgICAgICAgLSB7IGtpbmQ6IGlkZW50aWZpZXIsIHBhdHRlcm46ICRJREVOVCB9XG4gICAgICAgICAgICAgICAgICAgICAgICAtIHsgaGFzOiB7a2luZDogaWRlbnRpZmllciwgcGF0dGVybjogJElERU5ULCBzdG9wQnk6IGVuZH19XG5maXg6ICcnXG4iLCJzb3VyY2UiOiJTdHJpbmcgdW51c2VkID0gXCJ1bnVzZWRcIjtcbk1hcDxTdHJpbmcsIFN0cmluZz4gZGVjbGFyZWRCdXROb3RJbnN0YW50aWF0ZWQ7XG5cblN0cmluZyB1c2VkMSA9IFwidXNlZFwiO1xuaW50IHVzZWQyID0gMztcbmJvb2xlYW4gdXNlZDMgPSBmYWxzZTtcbmludCB1c2VkNCA9IDQ7XG5TdHJpbmcgdXNlZDUgPSBcIjVcIjtcblxuXG5cbnVzZWQxO1xuU3lzdGVtLm91dC5wcmludGxuKHVzZWQyKTtcbmlmKHVzZWQzKXtcbiAgICBTeXN0ZW0ub3V0LnByaW50bG4oXCJzb21lIHZhcnMgYXJlIHVudXNlZFwiKTtcbiAgICBNYXA8U3RyaW5nLCBTdHJpbmc+IHVudXNlZE1hcCA9IG5ldyBIYXNoTWFwPD4oKSB7e1xuICAgICAgICBwdXQodXNlZDUsIFwidXNlZDVcIik7XG4gICAgfX07XG5cbiAgICAvLyBFdmVuIHRob3VnaCB3ZSBkb24ndCByZWFsbHkgZG8gYW55dGhpbmcgd2l0aCB0aGlzIG1hcCwgc2VwYXJhdGluZyB0aGUgZGVjbGFyYXRpb24gYW5kIGluc3RhbnRpYXRpb24gbWFrZXMgaXQgY291bnQgYXMgYmVpbmcgdXNlZFxuICAgIGRlY2xhcmVkQnV0Tm90SW5zdGFudGlhdGVkID0gbmV3IEhhc2hNYXA8PigpO1xuXG4gICAgcmV0dXJuIHVzZWQ0O1xufSJ9) ### Description [​](https://ast-grep.github.io/catalog/java/no-unused-vars.html#description) Identifying unused variables is a common task in code refactoring. You should rely on a Java linter or IDE for this task rather than writing a custom rule in ast-grep, but for educational purposes, this rule demonstrates how to find unused variables in Java. This approach makes some simplifying assumptions. We only consider local variable declarations and ignore the other many ways variables can be declared: Method Parameters, Fields, Class Variables, Constructor Parameters, Loop Variables, Exception Handler Parameters, Lambda Parameters, Annotation Parameters, Enum Constants, and Record Components. Now you may see why it is recommended to use a rule from an established linter or IDE rather than writing your own. ### YAML [​](https://ast-grep.github.io/catalog/java/no-unused-vars.html#yaml) yaml id: no-unused-vars rule: kind: local_variable_declaration all: - has: has: kind: identifier pattern: $IDENT - not: precedes: stopBy: end has: stopBy: end any: - { kind: identifier, pattern: $IDENT } - { has: {kind: identifier, pattern: $IDENT, stopBy: end}} fix: '' First, we identify the local variable declaration and capture the pattern of the identifier inside of it. Then we use `not` and `precedes` to only match the local variable declaration if the identifier we captured does not appear later in the code. It is important to note that we use `all` here to force the ordering of the `has` rule to be before the `not` rule. This guarantees that the meta-variable `$IDENT` is captured by looking inside of the local variable declaration. Additionally, when looking ahead in the code, we can't just look for the identifier directly, but for any node that may contain the identifier. ### Example [​](https://ast-grep.github.io/catalog/java/no-unused-vars.html#example) java String unused = "unused"; String used = "used"; System.out.println(used); --- # Go | ast-grep [Skip to content](https://ast-grep.github.io/catalog/go/#VPContent) Return to top Are you an LLM? You can read better optimized documentation at /catalog/go.md for this page in Markdown format Go [​](https://ast-grep.github.io/catalog/go/#go) ================================================== This page curates a list of example ast-grep rules to check and to rewrite Go code. Detect problematic defer statements with function calls [​](https://ast-grep.github.io/catalog/go/#detect-problematic-defer-statements-with-function-calls) ------------------------------------------------------------------------------------------------------------------------------------------------------------ * [Playground Link](https://ast-grep.github.io/playground.html#eyJtb2RlIjoiUGF0Y2giLCJsYW5nIjoiZ28iLCJxdWVyeSI6InsgXG4gICAgZGVmZXIgJEEuJEIodCwgZmFpbHBvaW50LiRNKCQkJCkpIFxufSIsInJld3JpdGUiOiIiLCJzdHJpY3RuZXNzIjoic21hcnQiLCJzZWxlY3RvciI6ImRlZmVyX3N0YXRlbWVudCIsImNvbmZpZyI6InJ1bGU6XG4iLCJzb3VyY2UiOiJmdW5jIFRlc3RJc3N1ZTE2Njk2KHQgKnRlc3RpbmcuVCkge1xuXHRhbGFybVJhdGlvIDo9IHZhcmRlZi5NZW1vcnlVc2FnZUFsYXJtUmF0aW8uTG9hZCgpXG5cdHZhcmRlZi5NZW1vcnlVc2FnZUFsYXJtUmF0aW8uU3RvcmUoMC4wKVxuXHRkZWZlciB2YXJkZWYuTWVtb3J5VXNhZ2VBbGFybVJhdGlvLlN0b3JlKGFsYXJtUmF0aW8pXG5cdHJlcXVpcmUuTm9FcnJvcih0LCBmYWlscG9pbnQuRW5hYmxlKFwiZ2l0aHViLmNvbS9waW5nY2FwL3RpZGIvcGtnL2V4ZWN1dG9yL3NvcnRleGVjL3Rlc3RTb3J0ZWRSb3dDb250YWluZXJTcGlsbFwiLCBcInJldHVybih0cnVlKVwiKSlcblx0ZGVmZXIgcmVxdWlyZS5Ob0Vycm9yKHQsIFxuXHQgICBmYWlscG9pbnQuRGlzYWJsZShcblx0XHRcImdpdGh1Yi5jb20vcGluZ2NhcC90aWRiL3BrZy9leGVjdXRvci9zb3J0ZXhlYy90ZXN0U29ydGVkUm93Q29udGFpbmVyU3BpbGxcIlxuXHQpKVxuXHRyZXF1aXJlLk5vRXJyb3IodCwgXG5cdFx0ZmFpbHBvaW50LkVuYWJsZShcImdpdGh1Yi5jb20vcGluZ2NhcC90aWRiL3BrZy9leGVjdXRvci9qb2luL3Rlc3RSb3dDb250YWluZXJTcGlsbFwiLCBcInJldHVybih0cnVlKVwiKSlcblx0ZGVmZXIgcmVxdWlyZS5Ob0Vycm9yKHQsIFxuXHRcdGZhaWxwb2ludC5EaXNhYmxlKFwiZ2l0aHViLmNvbS9waW5nY2FwL3RpZGIvcGtnL2V4ZWN1dG9yL2pvaW4vdGVzdFJvd0NvbnRhaW5lclNwaWxsXCIpKVxufSJ9) ### Description [​](https://ast-grep.github.io/catalog/go/#description) This rule detects a common anti-pattern in Go testing code where `defer` statements contain function calls with parameters that are evaluated immediately instead of when the defer executes. In Go, `defer` schedules a function call to be executed when the surrounding function returns. However, the **arguments to the deferred function are evaluated immediately** when the defer statement is encountered, not when the defer executes. This is particularly problematic when using assertion libraries in tests. For example: go defer require.NoError(t, failpoint.Disable("some/path")) In this case, `failpoint.Disable("some/path")` is called immediately when the defer statement is reached, not when the function exits. This means the failpoint is disabled right after being enabled, making the test ineffective. ### Pattern [​](https://ast-grep.github.io/catalog/go/#pattern) shell ast-grep \ --lang go \ --pattern '{ defer $A.$B(t, failpoint.$M($$$)) } \ --selector defer_statement' ### Example [​](https://ast-grep.github.io/catalog/go/#example) go func TestIssue16696(t *testing.T) { alarmRatio := vardef.MemoryUsageAlarmRatio.Load() vardef.MemoryUsageAlarmRatio.Store(0.0) defer vardef.MemoryUsageAlarmRatio.Store(alarmRatio) require.NoError(t, failpoint.Enable("github.com/pingcap/tidb/pkg/executor/sortexec/testSortedRowContainerSpill", "return(true)")) defer require.NoError(t, failpoint.Disable( "github.com/pingcap/tidb/pkg/executor/sortexec/testSortedRowContainerSpill" )) require.NoError(t, failpoint.Enable("github.com/pingcap/tidb/pkg/executor/join/testRowContainerSpill", "return(true)")) defer require.NoError(t, failpoint.Disable("github.com/pingcap/tidb/pkg/executor/join/testRowContainerSpill")) } ### Fix [​](https://ast-grep.github.io/catalog/go/#fix) The correct way to defer a function with parameters is to wrap it in an anonymous function: go defer func() { require.NoError(t, failpoint.Disable("some/path")) }() ### Contributed by [​](https://ast-grep.github.io/catalog/go/#contributed-by) Inspired by [YangKeao's tweet](https://x.com/YangKeao/status/1671420857565212672) about this common pitfall in TiDB codebase. Find function declarations with names of certain pattern [​](https://ast-grep.github.io/catalog/go/#find-function-declarations-with-names-of-certain-pattern) -------------------------------------------------------------------------------------------------------------------------------------------------------------- * [Playground Link](https://ast-grep.github.io/playground.html#eyJtb2RlIjoiQ29uZmlnIiwibGFuZyI6ImdvIiwicXVlcnkiOiJyJ15bQS1aYS16MC05Xy1dKyciLCJyZXdyaXRlIjoiIiwiY29uZmlnIjoiaWQ6IHRlc3QtZnVuY3Rpb25zXG5sYW5ndWFnZTogZ29cbnJ1bGU6XG4gIGtpbmQ6IGZ1bmN0aW9uX2RlY2xhcmF0aW9uXG4gIGhhczpcbiAgICBmaWVsZDogbmFtZVxuICAgIHJlZ2V4OiBUZXN0LipcbiIsInNvdXJjZSI6InBhY2thZ2UgYWJzXG5pbXBvcnQgXCJ0ZXN0aW5nXCJcbmZ1bmMgVGVzdEFicyh0ICp0ZXN0aW5nLlQpIHtcbiAgICBnb3QgOj0gQWJzKC0xKVxuICAgIGlmIGdvdCAhPSAxIHtcbiAgICAgICAgdC5FcnJvcmYoXCJBYnMoLTEpID0gJWQ7IHdhbnQgMVwiLCBnb3QpXG4gICAgfVxufVxuIn0=) ### Description [​](https://ast-grep.github.io/catalog/go/#description-1) ast-grep can find function declarations by their names. But not all names can be matched by a meta variable pattern. For instance, you cannot use a meta variable pattern to find function declarations whose names start with a specific prefix, e.g. `TestAbs` with the prefix `Test`. Attempting `Test$_` will fail because it is not a valid syntax. Instead, you can use a [YAML rule](https://ast-grep.github.io/reference/rule.html) to use the [`regex`](https://ast-grep.github.io/guide/rule-config/atomic-rule.html#regex) atomic rule. ### YAML [​](https://ast-grep.github.io/catalog/go/#yaml) yaml id: test-functions language: go rule: kind: function_declaration has: field: name regex: Test.* ### Example [​](https://ast-grep.github.io/catalog/go/#example-1) go package abs import "testing" func TestAbs(t *testing.T) { got := Abs(-1) if got != 1 { t.Errorf("Abs(-1) = %d; want 1", got) } } ### Contributed by [​](https://ast-grep.github.io/catalog/go/#contributed-by-1) [kevinkjt2000](https://twitter.com/kevinkjt2000) on [Discord](https://discord.com/invite/4YZjf6htSQ) . Match Function Call in Golang [​](https://ast-grep.github.io/catalog/go/#match-function-call-in-golang) -------------------------------------------------------------------------------------------------------- * [Playground Link](https://ast-grep.github.io/playground.html#eyJtb2RlIjoiQ29uZmlnIiwibGFuZyI6ImdvIiwicXVlcnkiOiJhd2FpdCAkQSIsInJld3JpdGUiOiJ0cnkge1xuICAgIGF3YWl0ICRBXG59IGNhdGNoKGUpIHtcbiAgICAvLyB0b2RvXG59IiwiY29uZmlnIjoicnVsZTpcbiAgcGF0dGVybjpcbiAgICBjb250ZXh0OiAnZnVuYyB0KCkgeyBmbXQuUHJpbnRsbigkJCRBKSB9J1xuICAgIHNlbGVjdG9yOiBjYWxsX2V4cHJlc3Npb25cbiIsInNvdXJjZSI6ImZ1bmMgbWFpbigpIHtcbiAgICBmbXQuUHJpbnRsbihcIk9LXCIpXG59In0=) ### Description [​](https://ast-grep.github.io/catalog/go/#description-2) One of the common questions of ast-grep is to match function calls in Golang. A plain pattern like `fmt.Println($A)` will not work. This is because Golang syntax also allows type conversions, e.g. `int(3.14)`, that look like function calls. Tree-sitter, ast-grep's parser, will prefer parsing `func_call(arg)` as a type conversion instead of a call expression. To avoid this ambiguity, ast-grep lets us write a [contextual pattern](https://ast-grep.github.io/guide/rule-config/atomic-rule.html#pattern) , which is a pattern inside a larger code snippet. We can use `context` to write a pattern like this: `func t() { fmt.Println($A) }`. Then, we can use the selector `call_expression` to match only function calls. Please also read the [deep dive](https://ast-grep.github.io/advanced/pattern-parse.html) on [ambiguous pattern](https://ast-grep.github.io/advanced/pattern-parse.html#ambiguous-pattern-code) . ### YAML [​](https://ast-grep.github.io/catalog/go/#yaml-1) yaml id: match-function-call language: go rule: pattern: context: 'func t() { fmt.Println($A) }' selector: call_expression ### Example [​](https://ast-grep.github.io/catalog/go/#example-2) go func main() { fmt.Println("OK") } ### Contributed by [​](https://ast-grep.github.io/catalog/go/#contributed-by-2) Inspired by [QuantumGhost](https://github.com/QuantumGhost) from [ast-grep/ast-grep#646](https://github.com/ast-grep/ast-grep/issues/646) Match package import in Golang [​](https://ast-grep.github.io/catalog/go/#match-package-import-in-golang) ---------------------------------------------------------------------------------------------------------- * [Playground Link](https://ast-grep.github.io/playground.html#eyJtb2RlIjoiQ29uZmlnIiwibGFuZyI6ImdvIiwicXVlcnkiOiIiLCJyZXdyaXRlIjoiIiwic3RyaWN0bmVzcyI6InNtYXJ0Iiwic2VsZWN0b3IiOiIiLCJjb25maWciOiJpZDogbWF0Y2gtcGFja2FnZS1pbXBvcnRcbmxhbmd1YWdlOiBnb1xucnVsZTpcbiAga2luZDogaW1wb3J0X3NwZWNcbiAgaGFzOlxuICAgIHJlZ2V4OiBnaXRodWIuY29tL2dvbGFuZy1qd3Qvand0Iiwic291cmNlIjoicGFja2FnZSBtYWluXG5cbmltcG9ydCAoXG5cdFwiZm10XCJcblx0XCJnaXRodWIuY29tL2dvbGFuZy1qd3Qvand0XCIgIC8vIFRoaXMgbWF0Y2hlcyB0aGUgQVNUIHJ1bGVcbilcblxuZnVuYyBtYWluKCkge1xuXHQvLyBDcmVhdGUgYSBuZXcgdG9rZW5cblx0dG9rZW4gOj0gand0Lk5ldyhqd3QuU2lnbmluZ01ldGhvZEhTMjU2KVxuXHRcblx0Ly8gQWRkIHNvbWUgY2xhaW1zXG5cdHRva2VuLkNsYWltcyA9IGp3dC5NYXBDbGFpbXN7XG5cdFx0XCJ1c2VyXCI6IFwiYWxpY2VcIixcblx0XHRcInJvbGVcIjogXCJhZG1pblwiLFxuXHR9XG5cdFxuXHQvLyBTaWduIHRoZSB0b2tlblxuXHR0b2tlblN0cmluZywgZXJyIDo9IHRva2VuLlNpZ25lZFN0cmluZyhbXWJ5dGUoXCJteS1zZWNyZXRcIikpXG5cdGlmIGVyciAhPSBuaWwge1xuXHRcdGZtdC5QcmludGYoXCJFcnJvciBzaWduaW5nIHRva2VuOiAldlxcblwiLCBlcnIpXG5cdFx0cmV0dXJuXG5cdH1cblx0XG5cdGZtdC5QcmludGYoXCJHZW5lcmF0ZWQgdG9rZW46ICVzXFxuXCIsIHRva2VuU3RyaW5nKVxufSJ9) ### Description [​](https://ast-grep.github.io/catalog/go/#description-3) A generic rule template for detecting imports of specific packages in Go source code. This rule can be customized to match any package by modifying the regex pattern, making it useful for security auditing, dependency management, and compliance checking. This rule identifies Go import statements based on the configured regex pattern, including: Direct imports: `import "package/name"` Versioned imports: `import "package/name/v4"` Subpackage imports: `import "package/name/subpkg"` Grouped imports within `import () blocks` ### YAML [​](https://ast-grep.github.io/catalog/go/#yaml-2) yaml id: match-package-import language: go rule: kind: import_spec has: regex: PACKAGE_PATTERN_HERE ### Example [​](https://ast-grep.github.io/catalog/go/#example-3) JWT Library Detection go package main import ( "fmt" "github.com/golang-jwt/jwt" // This matches the AST rule ) func main() { token := jwt.New(jwt.SigningMethodHS256) // Create a new token // Add some claims token.Claims = jwt.MapClaims{"user": "alice", "role": "admin"} tokenString, err := token.SignedString([]byte("my-secret")) // Sign the token if err != nil { fmt.Printf("Error signing token: %v\n", err) return } fmt.Printf("Generated token: %s\n", tokenString) } ### Contributed by [​](https://ast-grep.github.io/catalog/go/#contributed-by-3) [Sudesh Gutta](https://github.com/sudeshgutta) Detect problematic JSON tags with dash prefix [​](https://ast-grep.github.io/catalog/go/#detect-problematic-json-tags-with-dash-prefix) ---------------------------------------------------------------------------------------------------------------------------------------- * [Playground Link](https://ast-grep.github.io/playground.html#eyJtb2RlIjoiQ29uZmlnIiwibGFuZyI6ImdvIiwicXVlcnkiOiJgJFRBR2AiLCJyZXdyaXRlIjoiIiwic3RyaWN0bmVzcyI6InNtYXJ0Iiwic2VsZWN0b3IiOiIiLCJjb25maWciOiJpZDogdW5tYXJzaGFsLXRhZy1pcy1kYXNoXG5zZXZlcml0eTogZXJyb3Jcbm1lc3NhZ2U6IFN0cnVjdCBmaWVsZCBjYW4gYmUgZGVjb2RlZCB3aXRoIHRoZSBgLWAga2V5IGJlY2F1c2UgdGhlIEpTT04gdGFnXG4gIHN0YXJ0cyB3aXRoIGEgYC1gIGJ1dCBpcyBmb2xsb3dlZCBieSBhIGNvbW1hLlxucnVsZTpcbiAgcGF0dGVybjogJ2AkVEFHYCdcbiAgaW5zaWRlOlxuICAgIGtpbmQ6IGZpZWxkX2RlY2xhcmF0aW9uXG5jb25zdHJhaW50czpcbiAgVEFHOiBcbiAgICByZWdleDoganNvbjpcIi0sLipcIiIsInNvdXJjZSI6InBhY2thZ2UgbWFpblxuXG50eXBlIFRlc3RTdHJ1Y3QxIHN0cnVjdCB7XG5cdC8vIG9rOiB1bm1hcnNoYWwtdGFnLWlzLWRhc2hcblx0QSBzdHJpbmcgYGpzb246XCJpZFwiYFxufVxuXG50eXBlIFRlc3RTdHJ1Y3QyIHN0cnVjdCB7XG5cdC8vIHJ1bGVpZDogdW5tYXJzaGFsLXRhZy1pcy1kYXNoXG5cdEIgc3RyaW5nIGBqc29uOlwiLSxvbWl0ZW1wdHlcImBcbn1cblxudHlwZSBUZXN0U3RydWN0MyBzdHJ1Y3Qge1xuXHQvLyBydWxlaWQ6IHVubWFyc2hhbC10YWctaXMtZGFzaFxuXHRDIHN0cmluZyBganNvbjpcIi0sMTIzXCJgXG59XG5cbnR5cGUgVGVzdFN0cnVjdDQgc3RydWN0IHtcblx0Ly8gcnVsZWlkOiB1bm1hcnNoYWwtdGFnLWlzLWRhc2hcblx0RCBzdHJpbmcgYGpzb246XCItLFwiYFxufSJ9) ### Description [​](https://ast-grep.github.io/catalog/go/#description-4) This rule detects a security vulnerability in Go's JSON unmarshaling. When a struct field has a JSON tag that starts with `-,`, it can be unexpectedly unmarshaled with the `-` key. According to the [Go documentation](https://pkg.go.dev/encoding/json#Marshal) , if the field tag is `-`, the field should be omitted. However, a field with name `-` can still be unmarshaled using the tag `-,`. This creates a security issue where developers think they are preventing a field from being unmarshaled (like `IsAdmin` in authentication), but attackers can still set that field by providing the `-` key in JSON input. go type User struct { Username string `json:"username,omitempty"` Password string `json:"password,omitempty"` IsAdmin bool `json:"-,omitempty"` // Intended to prevent marshaling } // This still works and sets IsAdmin to true! json.Unmarshal([]byte(`{"-": true}`), &user) // Result: main.User{Username:"", Password:"", IsAdmin:true} ### YAML [​](https://ast-grep.github.io/catalog/go/#yaml-3) yaml id: unmarshal-tag-is-dash severity: error message: Struct field can be decoded with the `-` key because the JSON tag starts with a `-` but is followed by a comma. rule: pattern: '`$TAG`' inside: kind: field_declaration constraints: TAG: regex: json:"-,.*" ### Example [​](https://ast-grep.github.io/catalog/go/#example-4) go package main type TestStruct1 struct { A string `json:"id"` // ok } type TestStruct2 struct { B string `json:"-,omitempty"` // wrong } type TestStruct3 struct { C string `json:"-,123"` // wrong } type TestStruct4 struct { D string `json:"-,"` // wrong } ### Fix [​](https://ast-grep.github.io/catalog/go/#fix-1) To properly omit a field from JSON marshaling/unmarshaling, use just `-` without a comma: go type User struct { Username string `json:"username,omitempty"` Password string `json:"password,omitempty"` IsAdmin bool `json:"-"` // Correctly prevents marshaling/unmarshaling } ### Contributed by [​](https://ast-grep.github.io/catalog/go/#contributed-by-4) Inspired by [Trail of Bits blog post](https://blog.trailofbits.com/2025/06/17/unexpected-security-footguns-in-gos-parsers/) and their [public Semgrep rule](https://semgrep.dev/playground/r/trailofbits.go.unmarshal-tag-is-dash) . --- # Kotlin | ast-grep [Skip to content](https://ast-grep.github.io/catalog/kotlin/#VPContent) Return to top Are you an LLM? You can read better optimized documentation at /catalog/kotlin.md for this page in Markdown format Kotlin [​](https://ast-grep.github.io/catalog/kotlin/#kotlin) ============================================================== This page curates a list of example ast-grep rules to check and to rewrite Kotlin code. Ensure Clean Architecture [​](https://ast-grep.github.io/catalog/kotlin/#ensure-clean-architecture) ---------------------------------------------------------------------------------------------------- * [Playground Link](https://ast-grep.github.io/playground.html#eyJtb2RlIjoiQ29uZmlnIiwibGFuZyI6ImtvdGxpbiIsInF1ZXJ5IjoiIiwicmV3cml0ZSI6IiIsInN0cmljdG5lc3MiOiJyZWxheGVkIiwic2VsZWN0b3IiOiIiLCJjb25maWciOiJpZDogaW1wb3J0LWRlcGVuZGVuY3ktdmlvbGF0aW9uXG5tZXNzYWdlOiBJbXBvcnQgRGVwZW5kZW5jeSBWaW9sYXRpb24gXG5ub3RlczogRW5zdXJlcyB0aGF0IGltcG9ydHMgY29tcGx5IHdpdGggYXJjaGl0ZWN0dXJhbCBydWxlcy4gXG5zZXZlcml0eTogZXJyb3JcbnJ1bGU6XG4gIHBhdHRlcm46IGltcG9ydCAkUEFUSFxuY29uc3RyYWludHM6XG4gIFBBVEg6XG4gICAgYW55OlxuICAgIC0gcmVnZXg6IGNvbVxcLmV4YW1wbGUoXFwuXFx3KykqXFwuZGF0YVxuICAgIC0gcmVnZXg6IGNvbVxcLmV4YW1wbGUoXFwuXFx3KykqXFwucHJlc2VudGF0aW9uXG5maWxlczpcbi0gY29tL2V4YW1wbGUvZG9tYWluLyoqLyoua3QiLCJzb3VyY2UiOiJpbXBvcnQgYW5kcm9pZHgubGlmZWN5Y2xlLlZpZXdNb2RlbFxuaW1wb3J0IGFuZHJvaWR4LmxpZmVjeWNsZS5WaWV3TW9kZWxTY29wZVxuaW1wb3J0IGNvbS5leGFtcGxlLmN1c3RvbWxpbnRleGFtcGxlLmRhdGEubW9kZWxzLlVzZXJEdG9cbmltcG9ydCBjb20uZXhhbXBsZS5jdXN0b21saW50ZXhhbXBsZS5kb21haW4udXNlY2FzZXMuR2V0VXNlclVzZUNhc2VcbmltcG9ydCBjb20uZXhhbXBsZS5jdXN0b21saW50ZXhhbXBsZS5wcmVzZW50YXRpb24uc3RhdGVzLk1haW5TdGF0ZVxuaW1wb3J0IGRhZ2dlci5oaWx0LmFuZHJvaWQubGlmZWN5Y2xlLkhpbHRWaWV3TW9kZWwifQ==) ### Description [​](https://ast-grep.github.io/catalog/kotlin/#description) This ast-grep rule ensures that the **domain** package in a [Clean Architecture](https://blog.cleancoder.com/uncle-bob/2012/08/13/the-clean-architecture.html) project does not import classes from the **data** or **presentation** packages. It enforces the separation of concerns by preventing the domain layer from depending on other layers, maintaining the integrity of the architecture. For example, the rule will trigger an error if an import statement like `import com.example.data.SomeClass` or `import com.example.presentation.AnotherClass` is found within the domain package. The rule uses the [`files`](https://ast-grep.github.io/reference/yaml.html#files) field to apply only to the domain package. ### YAML [​](https://ast-grep.github.io/catalog/kotlin/#yaml) yaml id: import-dependency-violation message: Import Dependency Violation notes: Ensures that imports comply with architectural rules. severity: error rule: pattern: import $PATH # capture the import statement constraints: PATH: # find specific package imports any: - regex: com\.example(\.\w+)*\.data - regex: com\.example(\.\w+)*\.presentation files: # apply only to domain package - com/example/domain/**/*.kt ### Example [​](https://ast-grep.github.io/catalog/kotlin/#example) kotlin import androidx.lifecycle.ViewModel import androidx.lifecycle.ViewModelScope import com.example.customlintexample.data.models.UserDto import com.example.customlintexample.domain.usecases.GetUserUseCase import com.example.customlintexample.presentation.states.MainState import dagger.hilt.android.lifecycle.HiltViewModel ### Contributed by [​](https://ast-grep.github.io/catalog/kotlin/#contributed-by) Inspired by the post [Custom Lint Task Configuration in Gradle with Kotlin DSL](https://www.sngular.com/insights/320/custom-lint-task-configuration-in-gradle-with-kotlin-dsl) --- # Java | ast-grep [Skip to content](https://ast-grep.github.io/catalog/java/#VPContent) Return to top Are you an LLM? You can read better optimized documentation at /catalog/java.md for this page in Markdown format Java [​](https://ast-grep.github.io/catalog/java/#java) ======================================================== This page curates a list of example ast-grep rules to check and to rewrite Java code. No Unused Vars in Java Has Fix [​](https://ast-grep.github.io/catalog/java/#no-unused-vars-in-java) ---------------------------------------------------------------------------------------------------- * [Playground Link](https://ast-grep.github.io/playground.html#eyJtb2RlIjoiQ29uZmlnIiwibGFuZyI6ImphdmEiLCJxdWVyeSI6ImlmKHRydWUpeyQkJEJPRFl9IiwicmV3cml0ZSI6IiRDOiBMaXN0WyRUXSA9IHJlbGF0aW9uc2hpcCgkJCRBLCB1c2VsaXN0PVRydWUsICQkJEIpIiwic3RyaWN0bmVzcyI6InNtYXJ0Iiwic2VsZWN0b3IiOiIiLCJjb25maWciOiJpZDogbm8tdW51c2VkLXZhcnNcbnJ1bGU6XG4gICAga2luZDogbG9jYWxfdmFyaWFibGVfZGVjbGFyYXRpb25cbiAgICBhbGw6XG4gICAgICAgIC0gaGFzOlxuICAgICAgICAgICAgaGFzOlxuICAgICAgICAgICAgICAgIGtpbmQ6IGlkZW50aWZpZXJcbiAgICAgICAgICAgICAgICBwYXR0ZXJuOiAkSURFTlRcbiAgICAgICAgLSBub3Q6XG4gICAgICAgICAgICBwcmVjZWRlczpcbiAgICAgICAgICAgICAgICBzdG9wQnk6IGVuZFxuICAgICAgICAgICAgICAgIGhhczpcbiAgICAgICAgICAgICAgICAgICAgc3RvcEJ5OiBlbmRcbiAgICAgICAgICAgICAgICAgICAgYW55OlxuICAgICAgICAgICAgICAgICAgICAgICAgLSB7IGtpbmQ6IGlkZW50aWZpZXIsIHBhdHRlcm46ICRJREVOVCB9XG4gICAgICAgICAgICAgICAgICAgICAgICAtIHsgaGFzOiB7a2luZDogaWRlbnRpZmllciwgcGF0dGVybjogJElERU5ULCBzdG9wQnk6IGVuZH19XG5maXg6ICcnXG4iLCJzb3VyY2UiOiJTdHJpbmcgdW51c2VkID0gXCJ1bnVzZWRcIjtcbk1hcDxTdHJpbmcsIFN0cmluZz4gZGVjbGFyZWRCdXROb3RJbnN0YW50aWF0ZWQ7XG5cblN0cmluZyB1c2VkMSA9IFwidXNlZFwiO1xuaW50IHVzZWQyID0gMztcbmJvb2xlYW4gdXNlZDMgPSBmYWxzZTtcbmludCB1c2VkNCA9IDQ7XG5TdHJpbmcgdXNlZDUgPSBcIjVcIjtcblxuXG5cbnVzZWQxO1xuU3lzdGVtLm91dC5wcmludGxuKHVzZWQyKTtcbmlmKHVzZWQzKXtcbiAgICBTeXN0ZW0ub3V0LnByaW50bG4oXCJzb21lIHZhcnMgYXJlIHVudXNlZFwiKTtcbiAgICBNYXA8U3RyaW5nLCBTdHJpbmc+IHVudXNlZE1hcCA9IG5ldyBIYXNoTWFwPD4oKSB7e1xuICAgICAgICBwdXQodXNlZDUsIFwidXNlZDVcIik7XG4gICAgfX07XG5cbiAgICAvLyBFdmVuIHRob3VnaCB3ZSBkb24ndCByZWFsbHkgZG8gYW55dGhpbmcgd2l0aCB0aGlzIG1hcCwgc2VwYXJhdGluZyB0aGUgZGVjbGFyYXRpb24gYW5kIGluc3RhbnRpYXRpb24gbWFrZXMgaXQgY291bnQgYXMgYmVpbmcgdXNlZFxuICAgIGRlY2xhcmVkQnV0Tm90SW5zdGFudGlhdGVkID0gbmV3IEhhc2hNYXA8PigpO1xuXG4gICAgcmV0dXJuIHVzZWQ0O1xufSJ9) ### Description [​](https://ast-grep.github.io/catalog/java/#description) Identifying unused variables is a common task in code refactoring. You should rely on a Java linter or IDE for this task rather than writing a custom rule in ast-grep, but for educational purposes, this rule demonstrates how to find unused variables in Java. This approach makes some simplifying assumptions. We only consider local variable declarations and ignore the other many ways variables can be declared: Method Parameters, Fields, Class Variables, Constructor Parameters, Loop Variables, Exception Handler Parameters, Lambda Parameters, Annotation Parameters, Enum Constants, and Record Components. Now you may see why it is recommended to use a rule from an established linter or IDE rather than writing your own. ### YAML [​](https://ast-grep.github.io/catalog/java/#yaml) yaml id: no-unused-vars rule: kind: local_variable_declaration all: - has: has: kind: identifier pattern: $IDENT - not: precedes: stopBy: end has: stopBy: end any: - { kind: identifier, pattern: $IDENT } - { has: {kind: identifier, pattern: $IDENT, stopBy: end}} fix: '' First, we identify the local variable declaration and capture the pattern of the identifier inside of it. Then we use `not` and `precedes` to only match the local variable declaration if the identifier we captured does not appear later in the code. It is important to note that we use `all` here to force the ordering of the `has` rule to be before the `not` rule. This guarantees that the meta-variable `$IDENT` is captured by looking inside of the local variable declaration. Additionally, when looking ahead in the code, we can't just look for the identifier directly, but for any node that may contain the identifier. ### Example [​](https://ast-grep.github.io/catalog/java/#example) java String unused = "unused"; String used = "used"; System.out.println(used); Find Java field declarations of type String [​](https://ast-grep.github.io/catalog/java/#find-java-field-declarations-of-type-string) -------------------------------------------------------------------------------------------------------------------------------------- * [Playground Link](https://ast-grep.github.io/playground.html#eyJtb2RlIjoiQ29uZmlnIiwibGFuZyI6ImphdmEiLCJxdWVyeSI6ImAkVEFHYCIsInJld3JpdGUiOiIiLCJzdHJpY3RuZXNzIjoic21hcnQiLCJzZWxlY3RvciI6IiIsImNvbmZpZyI6InJ1bGU6XG4gIGtpbmQ6IGZpZWxkX2RlY2xhcmF0aW9uXG4gIGhhczpcbiAgICBmaWVsZDogdHlwZVxuICAgIHJlZ2V4OiBeU3RyaW5nJCIsInNvdXJjZSI6IkBDb21wb25lbnRcbmNsYXNzIEFCQyBleHRlbmRzIE9iamVjdHtcbiAgICBAUmVzb3VyY2VcbiAgICBwcml2YXRlIGZpbmFsIFN0cmluZyB3aXRoX2Fubm87XG5cbiAgICBwcml2YXRlIGZpbmFsIFN0cmluZyB3aXRoX211bHRpX21vZDtcblxuICAgIHB1YmxpYyBTdHJpbmcgc2ltcGxlO1xufSJ9) ### Description [​](https://ast-grep.github.io/catalog/java/#description-1) To extract all Java field names of type `String` is not as straightforward as one might think. A simple pattern like `String $F;` would only match fields declared without any modifiers or annotations. However, a pattern like `$MOD String $F;` cannot be correctly parsed by tree-sitter. Use playground pattern debugger to explore the AST You can use the [playground](https://ast-grep.github.io/playground.html#eyJtb2RlIjoiUGF0Y2giLCJsYW5nIjoiamF2YSIsInF1ZXJ5IjoiY2xhc3MgQUJDe1xuICAgJE1PRCBTdHJpbmcgdGVzdDtcbn0iLCJyZXdyaXRlIjoiIiwic3RyaWN0bmVzcyI6ImFzdCIsInNlbGVjdG9yIjoiIiwiY29uZmlnIjoicnVsZTpcbiAga2luZDogZmllbGRfZGVjbGFyYXRpb25cbiAgaGFzOlxuICAgIGZpZWxkOiB0eXBlXG4gICAgcmVnZXg6IF5TdHJpbmckIiwic291cmNlIjoiQENvbXBvbmVudFxuY2xhc3MgQUJDIGV4dGVuZHMgT2JqZWN0e1xuICAgIEBSZXNvdXJjZVxuICAgIHByaXZhdGUgZmluYWwgU3RyaW5nIHdpdGhfYW5ubztcblxuICAgIHByaXZhdGUgZmluYWwgU3RyaW5nIHdpdGhfbXVsdGlfbW9kO1xuXG4gICAgcHVibGljIFN0cmluZyBzaW1wbGU7XG59In0=) 's pattern tab to visualize the AST of `class A { $MOD String $F; }`. field_declaration $MOD variable_declarator identifier: String ERROR identifier: $F Tree-sitter does not think `$MOD` is a valid modifier, so it produces an `ERROR`. While the valid AST for code like `private String field;` produces different AST structures: field_declaration modifiers type_identifier variable_declarator identifier: field A more robust approach is to use a structural rule that targets `field_declaration` nodes and applies a `has` constraint on the `type` child node to match the type `String`. This method effectively captures fields regardless of their modifiers or annotations. ### YAML [​](https://ast-grep.github.io/catalog/java/#yaml-1) yaml id: find-field-with-type language: java rule: kind: field_declaration has: field: type regex: ^String$ ### Example [​](https://ast-grep.github.io/catalog/java/#example-1) java @Component class ABC extends Object{ @Resource private final String with_anno; private final String with_multi_mod; public String simple; } ### Contributed by [​](https://ast-grep.github.io/catalog/java/#contributed-by) Inspired by the post [discussion](https://github.com/ast-grep/ast-grep/discussions/2195) --- # ast-grep [Skip to content](https://ast-grep.github.io/catalog/kotlin/ensure-clean-architecture.html#VPContent) Return to top Are you an LLM? You can read better optimized documentation at /catalog/kotlin/ensure-clean-architecture.md for this page in Markdown format Ensure Clean Architecture [​](https://ast-grep.github.io/catalog/kotlin/ensure-clean-architecture.html#ensure-clean-architecture) ---------------------------------------------------------------------------------------------------------------------------------- * [Playground Link](https://ast-grep.github.io/playground.html#eyJtb2RlIjoiQ29uZmlnIiwibGFuZyI6ImtvdGxpbiIsInF1ZXJ5IjoiIiwicmV3cml0ZSI6IiIsInN0cmljdG5lc3MiOiJyZWxheGVkIiwic2VsZWN0b3IiOiIiLCJjb25maWciOiJpZDogaW1wb3J0LWRlcGVuZGVuY3ktdmlvbGF0aW9uXG5tZXNzYWdlOiBJbXBvcnQgRGVwZW5kZW5jeSBWaW9sYXRpb24gXG5ub3RlczogRW5zdXJlcyB0aGF0IGltcG9ydHMgY29tcGx5IHdpdGggYXJjaGl0ZWN0dXJhbCBydWxlcy4gXG5zZXZlcml0eTogZXJyb3JcbnJ1bGU6XG4gIHBhdHRlcm46IGltcG9ydCAkUEFUSFxuY29uc3RyYWludHM6XG4gIFBBVEg6XG4gICAgYW55OlxuICAgIC0gcmVnZXg6IGNvbVxcLmV4YW1wbGUoXFwuXFx3KykqXFwuZGF0YVxuICAgIC0gcmVnZXg6IGNvbVxcLmV4YW1wbGUoXFwuXFx3KykqXFwucHJlc2VudGF0aW9uXG5maWxlczpcbi0gY29tL2V4YW1wbGUvZG9tYWluLyoqLyoua3QiLCJzb3VyY2UiOiJpbXBvcnQgYW5kcm9pZHgubGlmZWN5Y2xlLlZpZXdNb2RlbFxuaW1wb3J0IGFuZHJvaWR4LmxpZmVjeWNsZS5WaWV3TW9kZWxTY29wZVxuaW1wb3J0IGNvbS5leGFtcGxlLmN1c3RvbWxpbnRleGFtcGxlLmRhdGEubW9kZWxzLlVzZXJEdG9cbmltcG9ydCBjb20uZXhhbXBsZS5jdXN0b21saW50ZXhhbXBsZS5kb21haW4udXNlY2FzZXMuR2V0VXNlclVzZUNhc2VcbmltcG9ydCBjb20uZXhhbXBsZS5jdXN0b21saW50ZXhhbXBsZS5wcmVzZW50YXRpb24uc3RhdGVzLk1haW5TdGF0ZVxuaW1wb3J0IGRhZ2dlci5oaWx0LmFuZHJvaWQubGlmZWN5Y2xlLkhpbHRWaWV3TW9kZWwifQ==) ### Description [​](https://ast-grep.github.io/catalog/kotlin/ensure-clean-architecture.html#description) This ast-grep rule ensures that the **domain** package in a [Clean Architecture](https://blog.cleancoder.com/uncle-bob/2012/08/13/the-clean-architecture.html) project does not import classes from the **data** or **presentation** packages. It enforces the separation of concerns by preventing the domain layer from depending on other layers, maintaining the integrity of the architecture. For example, the rule will trigger an error if an import statement like `import com.example.data.SomeClass` or `import com.example.presentation.AnotherClass` is found within the domain package. The rule uses the [`files`](https://ast-grep.github.io/reference/yaml.html#files) field to apply only to the domain package. ### YAML [​](https://ast-grep.github.io/catalog/kotlin/ensure-clean-architecture.html#yaml) yaml id: import-dependency-violation message: Import Dependency Violation notes: Ensures that imports comply with architectural rules. severity: error rule: pattern: import $PATH # capture the import statement constraints: PATH: # find specific package imports any: - regex: com\.example(\.\w+)*\.data - regex: com\.example(\.\w+)*\.presentation files: # apply only to domain package - com/example/domain/**/*.kt ### Example [​](https://ast-grep.github.io/catalog/kotlin/ensure-clean-architecture.html#example) kotlin import androidx.lifecycle.ViewModel import androidx.lifecycle.ViewModelScope import com.example.customlintexample.data.models.UserDto import com.example.customlintexample.domain.usecases.GetUserUseCase import com.example.customlintexample.presentation.states.MainState import dagger.hilt.android.lifecycle.HiltViewModel ### Contributed by [​](https://ast-grep.github.io/catalog/kotlin/ensure-clean-architecture.html#contributed-by) Inspired by the post [Custom Lint Task Configuration in Gradle with Kotlin DSL](https://www.sngular.com/insights/320/custom-lint-task-configuration-in-gradle-with-kotlin-dsl) --- # ast-grep [Skip to content](https://ast-grep.github.io/catalog/python/migrate-openai-sdk.html#VPContent) Return to top Are you an LLM? You can read better optimized documentation at /catalog/python/migrate-openai-sdk.md for this page in Markdown format Migrate OpenAI SDK Has Fix [​](https://ast-grep.github.io/catalog/python/migrate-openai-sdk.html#migrate-openai-sdk) --------------------------------------------------------------------------------------------------------------------- * [Playground Link](https://ast-grep.github.io/playground.html#eyJtb2RlIjoiQ29uZmlnIiwibGFuZyI6InB5dGhvbiIsInF1ZXJ5IjoiZGVmICRGVU5DKCQkJEFSR1MpOiAkJCRCT0RZIiwicmV3cml0ZSI6IiIsImNvbmZpZyI6InJ1bGU6XG4gIHBhdHRlcm46IGltcG9ydCBvcGVuYWlcbmZpeDogZnJvbSBvcGVuYWkgaW1wb3J0IENsaWVudFxuLS0tXG5ydWxlOlxuICBwYXR0ZXJuOiBvcGVuYWkuYXBpX2tleSA9ICRLRVlcbmZpeDogY2xpZW50ID0gQ2xpZW50KCRLRVkpXG4tLS1cbnJ1bGU6XG4gIHBhdHRlcm46IG9wZW5haS5Db21wbGV0aW9uLmNyZWF0ZSgkJCRBUkdTKVxuZml4OiB8LVxuICBjbGllbnQuY29tcGxldGlvbnMuY3JlYXRlKFxuICAgICQkJEFSR1NcbiAgKSIsInNvdXJjZSI6ImltcG9ydCBvc1xuaW1wb3J0IG9wZW5haVxuZnJvbSBmbGFzayBpbXBvcnQgRmxhc2ssIGpzb25pZnlcblxuYXBwID0gRmxhc2soX19uYW1lX18pXG5vcGVuYWkuYXBpX2tleSA9IG9zLmdldGVudihcIk9QRU5BSV9BUElfS0VZXCIpXG5cblxuQGFwcC5yb3V0ZShcIi9jaGF0XCIsIG1ldGhvZHM9KFwiUE9TVFwiKSlcbmRlZiBpbmRleCgpOlxuICAgIGFuaW1hbCA9IHJlcXVlc3QuZm9ybVtcImFuaW1hbFwiXVxuICAgIHJlc3BvbnNlID0gb3BlbmFpLkNvbXBsZXRpb24uY3JlYXRlKFxuICAgICAgICBtb2RlbD1cInRleHQtZGF2aW5jaS0wMDNcIixcbiAgICAgICAgcHJvbXB0PWdlbmVyYXRlX3Byb21wdChhbmltYWwpLFxuICAgICAgICB0ZW1wZXJhdHVyZT0wLjYsXG4gICAgKVxuICAgIHJldHVybiBqc29uaWZ5KHJlc3BvbnNlLmNob2ljZXMpIn0=) ### Description [​](https://ast-grep.github.io/catalog/python/migrate-openai-sdk.html#description) OpenAI has introduced some breaking changes in their API, such as using `Client` to initialize the service and renaming the `Completion` method to `completions` . This example shows how to use ast-grep to automatically update your code to the new API. API migration requires multiple related rules to work together. The example shows how to write [multiple rules](https://ast-grep.github.io/reference/playground.html#test-multiple-rules) in a [single YAML](https://ast-grep.github.io/guide/rewrite-code.html#using-fix-in-yaml-rule) file. The rules and patterns in the example are simple and self-explanatory, so we will not explain them further. ### YAML [​](https://ast-grep.github.io/catalog/python/migrate-openai-sdk.html#yaml) yaml id: import-openai language: python rule: pattern: import openai fix: from openai import Client --- id: rewrite-client language: python rule: pattern: openai.api_key = $KEY fix: client = Client($KEY) --- id: rewrite-chat-completion language: python rule: pattern: openai.Completion.create($$$ARGS) fix: |- client.completions.create( $$$ARGS ) ### Example [​](https://ast-grep.github.io/catalog/python/migrate-openai-sdk.html#example) python import os import openai from flask import Flask, jsonify app = Flask(__name__) openai.api_key = os.getenv("OPENAI_API_KEY") @app.route("/chat", methods=("POST")) def index(): animal = request.form["animal"] response = openai.Completion.create( model="text-davinci-003", prompt=generate_prompt(animal), temperature=0.6, ) return jsonify(response.choices) ### Diff [​](https://ast-grep.github.io/catalog/python/migrate-openai-sdk.html#diff) python import os import openai from openai import Client from flask import Flask, jsonify app = Flask(__name__) openai.api_key = os.getenv("OPENAI_API_KEY") client = Client(os.getenv("OPENAI_API_KEY")) @app.route("/chat", methods=("POST")) def index(): animal = request.form["animal"] response = openai.Completion.create( response = client.completions.create( model="text-davinci-003", prompt=generate_prompt(animal), temperature=0.6, ) return jsonify(response.choices) ### Contributed by [​](https://ast-grep.github.io/catalog/python/migrate-openai-sdk.html#contributed-by) [Herrington Darkholme](https://twitter.com/hd_nvim) , inspired by [Morgante](https://twitter.com/morgantepell/status/1721668781246750952) from [grit.io](https://www.grit.io/) --- # ast-grep [Skip to content](https://ast-grep.github.io/catalog/python/optional-to-none-union.html#VPContent) Return to top Are you an LLM? You can read better optimized documentation at /catalog/python/optional-to-none-union.md for this page in Markdown format Rewrite `Optional[Type]` to `Type | None` Has Fix [​](https://ast-grep.github.io/catalog/python/optional-to-none-union.html#rewrite-optional-type-to-type-none) ---------------------------------------------------------------------------------------------------------------------------------------------------------------- * [Playground Link](https://ast-grep.github.io/playground.html#eyJtb2RlIjoiQ29uZmlnIiwibGFuZyI6InB5dGhvbiIsInF1ZXJ5IjoiIiwicmV3cml0ZSI6IiIsInN0cmljdG5lc3MiOiJzaWduYXR1cmUiLCJzZWxlY3RvciI6IiIsImNvbmZpZyI6InJ1bGU6XG4gIHBhdHRlcm46IFxuICAgIGNvbnRleHQ6ICdhOiBPcHRpb25hbFskVF0nXG4gICAgc2VsZWN0b3I6IGdlbmVyaWNfdHlwZVxuZml4OiAkVCB8IE5vbmUiLCJzb3VyY2UiOiJkZWYgYShhcmc6IE9wdGlvbmFsW0ludF0pOiBwYXNzIn0=) ### Description [​](https://ast-grep.github.io/catalog/python/optional-to-none-union.html#description) [PEP 604](https://peps.python.org/pep-0604/) recommends that `Type | None` is preferred over `Optional[Type]` for Python 3.10+. This rule performs such rewriting. Note `Optional[$T]` alone is interpreted as subscripting expression instead of generic type, we need to use [pattern object](https://ast-grep.github.io/guide/rule-config/atomic-rule.html#pattern-object) to disambiguate it with more context code. ### YAML [​](https://ast-grep.github.io/catalog/python/optional-to-none-union.html#yaml) yaml id: optional-to-none-union language: python rule: pattern: context: 'a: Optional[$T]' selector: generic_type fix: $T | None ### Example [​](https://ast-grep.github.io/catalog/python/optional-to-none-union.html#example) py def a(arg: Optional[int]): pass ### Diff [​](https://ast-grep.github.io/catalog/python/optional-to-none-union.html#diff) py def a(arg: Optional[int]): pass def a(arg: int | None): pass ### Contributed by [​](https://ast-grep.github.io/catalog/python/optional-to-none-union.html#contributed-by) [Bede Carroll](https://github.com/ast-grep/ast-grep/discussions/1492) --- # ast-grep [Skip to content](https://ast-grep.github.io/catalog/python/prefer-generator-expressions.html#VPContent) Return to top Are you an LLM? You can read better optimized documentation at /catalog/python/prefer-generator-expressions.md for this page in Markdown format Prefer Generator Expressions Has Fix [​](https://ast-grep.github.io/catalog/python/prefer-generator-expressions.html#prefer-generator-expressions) --------------------------------------------------------------------------------------------------------------------------------------------------- * [Playground Link](https://ast-grep.github.io/playground.html#eyJtb2RlIjoiQ29uZmlnIiwibGFuZyI6InB5dGhvbiIsInF1ZXJ5IjoiWyQkJEFdIiwicmV3cml0ZSI6IiRBPy4oKSIsImNvbmZpZyI6InJ1bGU6XG4gIHBhdHRlcm46ICRGVU5DKCRMSVNUKVxuY29uc3RyYWludHM6XG4gIExJU1Q6IHsga2luZDogbGlzdF9jb21wcmVoZW5zaW9uIH1cbiAgRlVOQzpcbiAgICBhbnk6XG4gICAgICAtIHBhdHRlcm46IGFueVxuICAgICAgLSBwYXR0ZXJuOiBhbGxcbiAgICAgIC0gcGF0dGVybjogc3VtXG4gICAgICAjIC4uLlxudHJhbnNmb3JtOlxuICBJTk5FUjpcbiAgICBzdWJzdHJpbmc6IHtzb3VyY2U6ICRMSVNULCBzdGFydENoYXI6IDEsIGVuZENoYXI6IC0xIH1cbmZpeDogJEZVTkMoJElOTkVSKSIsInNvdXJjZSI6ImFsbChbeCBmb3IgeCBpbiB5XSlcblt4IGZvciB4IGluIHldIn0=) ### Description [​](https://ast-grep.github.io/catalog/python/prefer-generator-expressions.html#description) List comprehensions like `[x for x in range(10)]` are a concise way to create lists in Python. However, we can achieve better memory efficiency by using generator expressions like `(x for x in range(10))` instead. List comprehensions create the entire list in memory, while generator expressions generate each element one at a time. We can make the change by replacing the square brackets with parentheses. ### YAML [​](https://ast-grep.github.io/catalog/python/prefer-generator-expressions.html#yaml) yaml id: prefer-generator-expressions language: python rule: pattern: $LIST kind: list_comprehension transform: INNER: substring: {source: $LIST, startChar: 1, endChar: -1 } fix: ($INNER) This rule converts every list comprehension to a generator expression. However, **not every list comprehension can be replaced with a generator expression.** If the list is used multiple times, is modified, is sliced, or is indexed, a generator is not a suitable replacement. Some common functions like `any`, `all`, and `sum` take an `iterable` as an argument. A generator function counts as an `iterable`, so it is safe to change a list comprehension to a generator expression in this context. yaml id: prefer-generator-expressions language: python rule: pattern: $FUNC($LIST) constraints: LIST: { kind: list_comprehension } FUNC: any: - pattern: any - pattern: all - pattern: sum # ... transform: INNER: substring: {source: $LIST, startChar: 1, endChar: -1 } fix: $FUNC($INNER) ### Example [​](https://ast-grep.github.io/catalog/python/prefer-generator-expressions.html#example) python any([x for x in range(10)]) ### Diff [​](https://ast-grep.github.io/catalog/python/prefer-generator-expressions.html#diff) python any([x for x in range(10)]) any(x for x in range(10)) ### Contributed by [​](https://ast-grep.github.io/catalog/python/prefer-generator-expressions.html#contributed-by) [Steven Love](https://github.com/StevenLove) --- # ast-grep [Skip to content](https://ast-grep.github.io/catalog/python/remove-async-await.html#VPContent) Return to top Are you an LLM? You can read better optimized documentation at /catalog/python/remove-async-await.md for this page in Markdown format Remove `async` function Has Fix [​](https://ast-grep.github.io/catalog/python/remove-async-await.html#remove-async-function) ----------------------------------------------------------------------------------------------------------------------------- * [Playground Link](https://ast-grep.github.io/playground.html#eyJtb2RlIjoiQ29uZmlnIiwibGFuZyI6InB5dGhvbiIsInF1ZXJ5IjoiYXdhaXQgJCQkQ0FMTCIsInJld3JpdGUiOiIkJCRDQUxMICIsImNvbmZpZyI6ImlkOiByZW1vdmUtYXN5bmMtZGVmXG5sYW5ndWFnZTogcHl0aG9uXG5ydWxlOlxuICBwYXR0ZXJuOlxuICAgIGNvbnRleHQ6ICdhc3luYyBkZWYgJEZVTkMoJCQkQVJHUyk6ICQkJEJPRFknXG4gICAgc2VsZWN0b3I6IGZ1bmN0aW9uX2RlZmluaXRpb25cbnRyYW5zZm9ybTpcbiAgUkVNT1ZFRF9CT0RZOlxuICAgIHJld3JpdGU6XG4gICAgICByZXdyaXRlcnM6IFtyZW1vdmUtYXdhaXQtY2FsbF1cbiAgICAgIHNvdXJjZTogJCQkQk9EWVxuZml4OiB8LVxuICBkZWYgJEZVTkMoJCQkQVJHUyk6XG4gICAgJFJFTU9WRURfQk9EWVxucmV3cml0ZXJzOlxuLSBpZDogcmVtb3ZlLWF3YWl0LWNhbGxcbiAgcnVsZTpcbiAgICBwYXR0ZXJuOiAnYXdhaXQgJCQkQ0FMTCdcbiAgZml4OiAkJCRDQUxMXG4iLCJzb3VyY2UiOiJhc3luYyBkZWYgbWFpbjMoKTpcbiAgYXdhaXQgc29tZWNhbGwoMSwgNSkifQ==) ### Description [​](https://ast-grep.github.io/catalog/python/remove-async-await.html#description) The `async` keyword in Python is used to define asynchronous functions that can be `await`ed. In this example, we want to remove the `async` keyword from a function definition and replace it with a synchronous version of the function. We also need to remove the `await` keyword from the function body. By default, ast-grep will not apply overlapping replacements. This means `await` keywords will not be modified because they are inside the async function body. However, we can use the [`rewriter`](https://ast-grep.github.io/reference/yaml/rewriter.html) to apply changes inside the matched function body. ### YAML [​](https://ast-grep.github.io/catalog/python/remove-async-await.html#yaml) yaml id: remove-async-def language: python rule: # match async function definition pattern: context: 'async def $FUNC($$$ARGS): $$$BODY' selector: function_definition rewriters: # define a rewriter to remove the await keyword remove-await-call: pattern: 'await $$$CALL' fix: $$$CALL # remove await keyword # apply the rewriter to the function body transform: REMOVED_BODY: rewrite: rewriters: [remove-await-call] source: $$$BODY fix: |- def $FUNC($$$ARGS): $REMOVED_BODY ### Example [​](https://ast-grep.github.io/catalog/python/remove-async-await.html#example) python async def main3(): await somecall(1, 5) ### Diff [​](https://ast-grep.github.io/catalog/python/remove-async-await.html#diff) python async def main3(): await somecall(1, 5) def main3(): somecall(1, 5) ### Contributed by [​](https://ast-grep.github.io/catalog/python/remove-async-await.html#contributed-by) Inspired by the ast-grep issue [#1185](https://github.com/ast-grep/ast-grep/issues/1185) --- # ast-grep [Skip to content](https://ast-grep.github.io/catalog/python/recursive-rewrite-type.html#VPContent) Return to top Are you an LLM? You can read better optimized documentation at /catalog/python/recursive-rewrite-type.md for this page in Markdown format Recursive Rewrite Type Has Fix [​](https://ast-grep.github.io/catalog/python/recursive-rewrite-type.html#recursive-rewrite-type) --------------------------------------------------------------------------------------------------------------------------------- * [Playground Link](https://ast-grep.github.io/playground.html#eyJtb2RlIjoiQ29uZmlnIiwibGFuZyI6InB5dGhvbiIsInF1ZXJ5IjoiIiwicmV3cml0ZSI6IiIsInN0cmljdG5lc3MiOiJzbWFydCIsInNlbGVjdG9yIjoiIiwiY29uZmlnIjoicmV3cml0ZXJzOlxyXG4tIGlkOiBvcHRpb25hbFxyXG4gIGxhbmd1YWdlOiBQeXRob25cclxuICBydWxlOlxyXG4gICAgYW55OlxyXG4gICAgLSBwYXR0ZXJuOlxyXG4gICAgICAgIGNvbnRleHQ6ICdhcmc6IE9wdGlvbmFsWyRUWVBFXSdcclxuICAgICAgICBzZWxlY3RvcjogZ2VuZXJpY190eXBlXHJcbiAgICAtIHBhdHRlcm46IE9wdGlvbmFsWyRUWVBFXVxyXG4gIHRyYW5zZm9ybTpcclxuICAgIE5UOlxyXG4gICAgICByZXdyaXRlOiBcclxuICAgICAgICByZXdyaXRlcnM6IFtvcHRpb25hbCwgdW5pb25zXVxyXG4gICAgICAgIHNvdXJjZTogJFRZUEVcclxuICBmaXg6ICROVCB8IE5vbmVcclxuLSBpZDogdW5pb25zXHJcbiAgbGFuZ3VhZ2U6IFB5dGhvblxyXG4gIHJ1bGU6XHJcbiAgICBwYXR0ZXJuOlxyXG4gICAgICBjb250ZXh0OiAnYTogVW5pb25bJCQkVFlQRVNdJ1xyXG4gICAgICBzZWxlY3RvcjogZ2VuZXJpY190eXBlXHJcbiAgdHJhbnNmb3JtOlxyXG4gICAgVU5JT05TOlxyXG4gICAgICByZXdyaXRlOlxyXG4gICAgICAgIHJld3JpdGVyczpcclxuICAgICAgICAgIC0gcmV3cml0ZS11bmlvbnNcclxuICAgICAgICBzb3VyY2U6ICQkJFRZUEVTXHJcbiAgICAgICAgam9pbkJ5OiBcIiB8IFwiXHJcbiAgZml4OiAkVU5JT05TXHJcbi0gaWQ6IHJld3JpdGUtdW5pb25zXHJcbiAgcnVsZTpcclxuICAgIHBhdHRlcm46ICRUWVBFXHJcbiAgICBraW5kOiB0eXBlXHJcbiAgdHJhbnNmb3JtOlxyXG4gICAgTlQ6XHJcbiAgICAgIHJld3JpdGU6IFxyXG4gICAgICAgIHJld3JpdGVyczogW29wdGlvbmFsLCB1bmlvbnNdXHJcbiAgICAgICAgc291cmNlOiAkVFlQRVxyXG4gIGZpeDogJE5UXHJcbnJ1bGU6XHJcbiAga2luZDogdHlwZVxyXG4gIHBhdHRlcm46ICRUUEVcclxudHJhbnNmb3JtOlxyXG4gIE5FV19UWVBFOlxyXG4gICAgcmV3cml0ZTogXHJcbiAgICAgIHJld3JpdGVyczogW29wdGlvbmFsLCB1bmlvbnNdXHJcbiAgICAgIHNvdXJjZTogJFRQRVxyXG5maXg6ICRORVdfVFlQRSIsInNvdXJjZSI6InJlc3VsdHM6ICBPcHRpb25hbFtVbmlvbltMaXN0W1VuaW9uW3N0ciwgZGljdF1dLCBzdHJdXVxuIn0=) ### Description [​](https://ast-grep.github.io/catalog/python/recursive-rewrite-type.html#description) Suppose we want to transform Python's `Union[T1, T2]` to `T1 | T2` and `Optional[T]` to `T | None`. By default, ast-grep will only fix the outermost node that matches a pattern and will not rewrite the inner AST nodes inside a match. This avoids unexpected rewriting or infinite rewriting loop. So if you are using non-recursive rewriter like [this](https://github.com/ast-grep/ast-grep/discussions/1566#discussion-7401382) , `Optional[Union[int, str]]` will only be converted to `Union[int, str] | None`. Note the inner `Union[int, str]` is not enabled. This is because the rewriter `optional` matches `Optional[$TYPE]` and rewrite it to `$TYPE | None`. The inner `$TYPE` is not processed. However, we can apply `rewriters` to inner types recursively. Take the `optional` rewriter as an example, we need to apply rewriters, `optional` and `unions`, **recursively** to `$TYPE` and get a new variable `$NT`. ### YAML [​](https://ast-grep.github.io/catalog/python/recursive-rewrite-type.html#yaml) yml id: recursive-rewrite-types language: python rewriters: # rewrite Optional[T] to T | None - id: optional rule: any: - pattern: context: 'arg: Optional[$TYPE]' selector: generic_type - pattern: Optional[$TYPE] # recursively apply rewriters to $TYPE transform: NT: rewrite: rewriters: [optional, unions] source: $TYPE # use the new variable $NT fix: $NT | None # similar to Optional, rewrite Union[T1, T2] to T1 | T2 - id: unions language: Python rule: pattern: context: 'a: Union[$$$TYPES]' selector: generic_type transform: UNIONS: # rewrite all types inside $$$TYPES rewrite: rewriters: [ rewrite-unions ] source: $$$TYPES joinBy: " | " fix: $UNIONS - id: rewrite-unions rule: pattern: $TYPE kind: type # recursive part transform: NT: rewrite: rewriters: [optional, unions] source: $TYPE fix: $NT # find all types rule: kind: type pattern: $TPE # apply the recursive rewriters transform: NEW_TYPE: rewrite: rewriters: [optional, unions] source: $TPE # output fix: $NEW_TYPE ### Example [​](https://ast-grep.github.io/catalog/python/recursive-rewrite-type.html#example) python results: Optional[Union[List[Union[str, dict]], str]] ### Diff [​](https://ast-grep.github.io/catalog/python/recursive-rewrite-type.html#diff) python results: Optional[Union[List[Union[str, dict]], str]] results: List[str | dict] | str | None ### Contributed by [​](https://ast-grep.github.io/catalog/python/recursive-rewrite-type.html#contributed-by) Inspired by [steinuil](https://github.com/ast-grep/ast-grep/discussions/1566) --- # ast-grep [Skip to content](https://ast-grep.github.io/catalog/python/refactor-pytest-fixtures.html#VPContent) Return to top Are you an LLM? You can read better optimized documentation at /catalog/python/refactor-pytest-fixtures.md for this page in Markdown format Refactor pytest fixtures [​](https://ast-grep.github.io/catalog/python/refactor-pytest-fixtures.html#refactor-pytest-fixtures) ------------------------------------------------------------------------------------------------------------------------------- * [Playground Link](https://ast-grep.github.io/playground.html#eyJtb2RlIjoiQ29uZmlnIiwibGFuZyI6InB5dGhvbiIsInF1ZXJ5IjoiZGVmIGZvbygkWCk6XG4gICRTIiwicmV3cml0ZSI6ImxvZ2dlci5sb2coJE1BVENIKSIsImNvbmZpZyI6ImlkOiBweXRlc3QtdHlwZS1oaW50LWZpeHR1cmVcbmxhbmd1YWdlOiBQeXRob25cbnV0aWxzOlxuICBpcy1maXh0dXJlLWZ1bmN0aW9uOlxuICAgIGtpbmQ6IGZ1bmN0aW9uX2RlZmluaXRpb25cbiAgICBmb2xsb3dzOlxuICAgICAga2luZDogZGVjb3JhdG9yXG4gICAgICBoYXM6XG4gICAgICAgIGtpbmQ6IGlkZW50aWZpZXJcbiAgICAgICAgcmVnZXg6IF5maXh0dXJlJFxuICAgICAgICBzdG9wQnk6IGVuZFxuICBpcy10ZXN0LWZ1bmN0aW9uOlxuICAgIGtpbmQ6IGZ1bmN0aW9uX2RlZmluaXRpb25cbiAgICBoYXM6XG4gICAgICBmaWVsZDogbmFtZVxuICAgICAgcmVnZXg6IF50ZXN0X1xuICBpcy1weXRlc3QtY29udGV4dDpcbiAgICAjIFB5dGVzdCBjb250ZXh0IGlzIGEgbm9kZSBpbnNpZGUgYSBweXRlc3RcbiAgICAjIHRlc3QvZml4dHVyZVxuICAgIGluc2lkZTpcbiAgICAgIHN0b3BCeTogZW5kXG4gICAgICBhbnk6XG4gICAgICAgIC0gbWF0Y2hlczogaXMtZml4dHVyZS1mdW5jdGlvblxuICAgICAgICAtIG1hdGNoZXM6IGlzLXRlc3QtZnVuY3Rpb25cbiAgaXMtZml4dHVyZS1hcmc6XG4gICAgIyBGaXh0dXJlIGFyZ3VtZW50cyBhcmUgaWRlbnRpZmllcnMgaW5zaWRlIHRoZSBcbiAgICAjIHBhcmFtZXRlcnMgb2YgYSB0ZXN0L2ZpeHR1cmUgZnVuY3Rpb25cbiAgICBhbGw6XG4gICAgICAtIGtpbmQ6IGlkZW50aWZpZXJcbiAgICAgIC0gbWF0Y2hlczogaXMtcHl0ZXN0LWNvbnRleHRcbiAgICAgIC0gaW5zaWRlOlxuICAgICAgICAgIGtpbmQ6IHBhcmFtZXRlcnNcbnJ1bGU6XG4gIG1hdGNoZXM6IGlzLWZpeHR1cmUtYXJnXG4gIHJlZ2V4OiBeZm9vJFxuZml4OiAnZm9vOiBpbnQnXG4iLCJzb3VyY2UiOiJmcm9tIGNvbGxlY3Rpb25zLmFiYyBpbXBvcnQgSXRlcmFibGVcbmZyb20gdHlwaW5nIGltcG9ydCBBbnlcblxuaW1wb3J0IHB5dGVzdFxuZnJvbSBweXRlc3QgaW1wb3J0IGZpeHR1cmVcblxuQHB5dGVzdC5maXh0dXJlKHNjb3BlPVwic2Vzc2lvblwiKVxuZGVmIGZvbygpIC0+IEl0ZXJhYmxlW2ludF06XG4gICAgeWllbGQgNVxuXG5AZml4dHVyZVxuZGVmIGJhcihmb28pIC0+IHN0cjpcbiAgICByZXR1cm4gc3RyKGZvbylcblxuZGVmIHJlZ3VsYXJfZnVuY3Rpb24oZm9vKSAtPiBOb25lOlxuICAgICMgVGhpcyBmdW5jdGlvbiBkb2Vzbid0IHVzZSB0aGUgJ2ZvbycgZml4dHVyZVxuICAgIHByaW50KGZvbylcblxuZGVmIHRlc3RfMShmb28sIGJhcik6XG4gICAgcHJpbnQoZm9vLCBiYXIpXG5cbmRlZiB0ZXN0XzIoYmFyKTpcbiAgICAuLi4ifQ==) ### Description [​](https://ast-grep.github.io/catalog/python/refactor-pytest-fixtures.html#description) One of the most commonly used testing framework in Python is [pytest](https://docs.pytest.org/en/8.2.x/) . Among other things, it allows the use of [fixtures](https://docs.pytest.org/en/6.2.x/fixture.html) . Fixtures are defined as functions that can be required in test code, or in other fixtures, as an argument. This means that all functions arguments with a given name in a pytest context (test function or fixture) are essentially the same entity. However, not every editor's LSP is able to keep track of this, making refactoring challenging. Using ast-grep, we can define some rules to match fixture definition and usage without catching similarly named entities in a non-test context. First, we define utils to select pytest test/fixture functions. yaml utils: is-fixture-function: kind: function_definition follows: kind: decorator has: kind: identifier regex: ^fixture$ stopBy: end is-test-function: kind: function_definition has: field: name regex: ^test_ Pytest fixtures are declared with a decorator `@pytest.fixture`. We match the `function_definition` node that directly follows a `decorator` node. That decorator node must have a `fixture` identifier somewhere. This accounts for different location of the `fixture` node depending on the type of imports and whether the decorator is used as is or called with parameters. Pytest functions are fairly straightforward to detect, as they always start with `test_` by convention. The next utils builds onto those two to incrementally: * Find if a node is inside a pytest context (test/fixture) * Find if a node is an argument in such a context yaml utils: is-pytest-context: # Pytest context is a node inside a pytest # test/fixture inside: stopBy: end any: - matches: is-fixture-function - matches: is-test-function is-fixture-arg: # Fixture arguments are identifiers inside the # parameters of a test/fixture function all: - kind: identifier - inside: kind: parameters - matches: is-pytest-context Once those utils are declared, you can perform various refactoring on a specific fixture. The following rule adds a type-hint to a fixture. yaml rule: matches: is-fixture-arg regex: ^foo$ fix: 'foo: int' This one renames a fixture and all its references. yaml rule: kind: identifier matches: is-fixture-context regex: ^foo$ fix: 'five' ### Example [​](https://ast-grep.github.io/catalog/python/refactor-pytest-fixtures.html#example) #### Renaming Fixtures [​](https://ast-grep.github.io/catalog/python/refactor-pytest-fixtures.html#renaming-fixtures) python @pytest.fixture def foo() -> int: return 5 @pytest.fixture(scope="function") def some_fixture(foo: int) -> str: return str(foo) def regular_function(foo) -> None: ... def test_code(foo: int) -> None: assert foo == 5 #### Diff [​](https://ast-grep.github.io/catalog/python/refactor-pytest-fixtures.html#diff) python @pytest.fixture def foo() -> int: def five() -> int: return 5 @pytest.fixture(scope="function") def some_fixture(foo: int) -> str: def some_fixture(five: int) -> str: return str(foo) def regular_function(foo) -> None: ... def test_code(foo: int) -> None: def test_code(five: int) -> None: assert foo == 5 assert five == 5 #### Type Hinting Fixtures [​](https://ast-grep.github.io/catalog/python/refactor-pytest-fixtures.html#type-hinting-fixtures) python @pytest.fixture def foo() -> int: return 5 @pytest.fixture(scope="function") def some_fixture(foo) -> str: return str(foo) def regular_function(foo) -> None: ... def test_code(foo) -> None: assert foo == 5 #### Diff [​](https://ast-grep.github.io/catalog/python/refactor-pytest-fixtures.html#diff-1) python @pytest.fixture def foo() -> int: return 5 @pytest.fixture(scope="function") def some_fixture(foo) -> str: def some_fixture(foo: int) -> str: return str(foo) def regular_function(foo) -> None: ... def test_code(foo) -> None: def test_code(foo: int) -> None: assert foo == 5 --- # ast-grep [Skip to content](https://ast-grep.github.io/catalog/ruby/prefer-symbol-over-proc.html#VPContent) Return to top Are you an LLM? You can read better optimized documentation at /catalog/ruby/prefer-symbol-over-proc.md for this page in Markdown format Prefer Symbol over Proc Has Fix [​](https://ast-grep.github.io/catalog/ruby/prefer-symbol-over-proc.html#prefer-symbol-over-proc) ---------------------------------------------------------------------------------------------------------------------------------- * [Playground Link](https://ast-grep.github.io/playground.html#eyJtb2RlIjoiQ29uZmlnIiwibGFuZyI6InJ1YnkiLCJxdWVyeSI6IiRMSVNULnNlbGVjdCB7IHwkVnwgJFYuJE1FVEhPRCB9IiwicmV3cml0ZSI6IiRMSVNULnNlbGVjdCgmOiRNRVRIT0QpIiwiY29uZmlnIjoiaWQ6IHByZWZlci1zeW1ib2wtb3Zlci1wcm9jXG5ydWxlOlxuICBwYXR0ZXJuOiAkTElTVC4kSVRFUiB7IHwkVnwgJFYuJE1FVEhPRCB9XG5sYW5ndWFnZTogUnVieVxuY29uc3RyYWludHM6XG4gIElURVI6XG4gICAgcmVnZXg6ICdtYXB8c2VsZWN0fGVhY2gnXG5maXg6ICckTElTVC4kSVRFUigmOiRNRVRIT0QpJ1xuIiwic291cmNlIjoiWzEsIDIsIDNdLnNlbGVjdCB7IHx2fCB2LmV2ZW4/IH1cbigxLi4xMDApLmVhY2ggeyB8aXwgaS50b19zIH1cbm5vdF9saXN0Lm5vX21hdGNoIHsgfHZ8IHYuZXZlbj8gfVxuIn0=) ### Description [​](https://ast-grep.github.io/catalog/ruby/prefer-symbol-over-proc.html#description) Ruby has a more concise symbol shorthand `&:` to invoke methods. This rule simplifies `proc` to `symbol`. This example is inspired by this [dev.to article](https://dev.to/baweaver/future-of-ruby-ast-tooling-9i1) . ### YAML [​](https://ast-grep.github.io/catalog/ruby/prefer-symbol-over-proc.html#yaml) yaml id: prefer-symbol-over-proc language: ruby rule: pattern: $LIST.$ITER { |$V| $V.$METHOD } constraints: ITER: regex: 'map|select|each' fix: '$LIST.$ITER(&:$METHOD)' ### Example [​](https://ast-grep.github.io/catalog/ruby/prefer-symbol-over-proc.html#example) rb [1, 2, 3].select { |v| v.even? } (1..100).each { |i| i.to_s } not_list.no_match { |v| v.even? } ### Diff [​](https://ast-grep.github.io/catalog/ruby/prefer-symbol-over-proc.html#diff) rb [1, 2, 3].select { |v| v.even? } [1, 2, 3].select(&:even?) (1..100).each { |i| i.to_s } (1..100).each(&:to_s) not_list.no_match { |v| v.even? } ### Contributed by [​](https://ast-grep.github.io/catalog/ruby/prefer-symbol-over-proc.html#contributed-by) [Herrington Darkholme](https://twitter.com/hd_nvim) --- # ast-grep [Skip to content](https://ast-grep.github.io/catalog/rule-template.html#VPContent) Return to top Are you an LLM? You can read better optimized documentation at /catalog/rule-template.md for this page in Markdown format Your Rule Name Has Fix [​](https://ast-grep.github.io/catalog/rule-template.html#your-rule-name) ------------------------------------------------------------------------------------------------- * [Playground Link](https://ast-grep.github.io/playground.html) ### Description [​](https://ast-grep.github.io/catalog/rule-template.html#description) Some Description for your rule! ### Pattern [​](https://ast-grep.github.io/catalog/rule-template.html#pattern) shell ast-grep -p pattern -r rewrite -l js # or without fixer ast-grep -p pattern -l js ### YAML [​](https://ast-grep.github.io/catalog/rule-template.html#yaml) yaml ### Example [​](https://ast-grep.github.io/catalog/rule-template.html#example) js var a = 123 ### Diff [​](https://ast-grep.github.io/catalog/rule-template.html#diff) js var a = 123 let a = 123 ### Contributed by [​](https://ast-grep.github.io/catalog/rule-template.html#contributed-by) [Author Name](https://your-social.link/) --- # ast-grep [Skip to content](https://ast-grep.github.io/catalog/python/use-walrus-operator-in-if.html#VPContent) Return to top Are you an LLM? You can read better optimized documentation at /catalog/python/use-walrus-operator-in-if.md for this page in Markdown format Use Walrus Operator in `if` statementHas Fix [​](https://ast-grep.github.io/catalog/python/use-walrus-operator-in-if.html#use-walrus-operator-in-if-statement) --------------------------------------------------------------------------------------------------------------------------------------------------------------- * [Playground Link](https://ast-grep.github.io/playground.html#eyJtb2RlIjoiQ29uZmlnIiwibGFuZyI6InB5dGhvbiIsInF1ZXJ5IjoiZm4gbWFpbigpIHsgXG4gICAgJCQkO1xuICAgIGlmKCRBKXskJCRCfSBcbiAgICBpZigkQSl7JCQkQ30gXG4gICAgJCQkRlxufSIsInJld3JpdGUiOiJmbiBtYWluKCkgeyAkJCRFOyBpZigkQSl7JCQkQiAkJCRDfSAkJCRGfSIsImNvbmZpZyI6ImlkOiB1c2Utd2FscnVzLW9wZXJhdG9yXG5ydWxlOlxuICBmb2xsb3dzOlxuICAgIHBhdHRlcm46XG4gICAgICBjb250ZXh0OiAkVkFSID0gJCQkRVhQUlxuICAgICAgc2VsZWN0b3I6IGV4cHJlc3Npb25fc3RhdGVtZW50XG4gIHBhdHRlcm46IFwiaWYgJFZBUjogJCQkQlwiXG5maXg6IHwtXG4gIGlmICRWQVIgOj0gJCQkRVhQUjpcbiAgICAkJCRCXG4tLS1cbmlkOiByZW1vdmUtZGVjbGFyYXRpb25cbnJ1bGU6XG4gIHBhdHRlcm46XG4gICAgY29udGV4dDogJFZBUiA9ICQkJEVYUFJcbiAgICBzZWxlY3RvcjogZXhwcmVzc2lvbl9zdGF0ZW1lbnRcbiAgcHJlY2VkZXM6XG4gICAgcGF0dGVybjogXCJpZiAkVkFSOiAkJCRCXCJcbmZpeDogJyciLCJzb3VyY2UiOiJhID0gZm9vKClcblxuaWYgYTpcbiAgICBkb19iYXIoKSJ9) ### Description [​](https://ast-grep.github.io/catalog/python/use-walrus-operator-in-if.html#description) The walrus operator (`:=`) introduced in Python 3.8 allows you to assign values to variables as part of an expression. This rule aims to simplify code by using the walrus operator in `if` statements. This first part of the rule identifies cases where a variable is assigned a value and then immediately used in an `if` statement to control flow. yaml id: use-walrus-operator language: python rule: pattern: "if $VAR: $$$B" follows: pattern: context: $VAR = $$$EXPR selector: expression_statement fix: |- if $VAR := $$$EXPR: $$$B The `pattern` clause finds an `if` statement that checks the truthiness of `$VAR`. If this pattern `follows` an expression statement where `$VAR` is assigned `$$$EXPR`, the `fix` clause changes the `if` statements to use the walrus operator. The second part of the rule: yaml id: remove-declaration rule: pattern: context: $VAR = $$$EXPR selector: expression_statement precedes: pattern: "if $VAR: $$$B" fix: '' This rule removes the standalone variable assignment when it directly precedes an `if` statement that uses the walrus operator. Since the assignment is now part of the `if` statement, the separate declaration is no longer needed. By applying these rules, you can refactor your Python code to be more concise and readable, taking advantage of the walrus operator's ability to combine an assignment with an expression. ### YAML [​](https://ast-grep.github.io/catalog/python/use-walrus-operator-in-if.html#yaml) yaml id: use-walrus-operator language: python rule: follows: pattern: context: $VAR = $$$EXPR selector: expression_statement pattern: "if $VAR: $$$B" fix: |- if $VAR := $$$EXPR: $$$B --- id: remove-declaration language: python rule: pattern: context: $VAR = $$$EXPR selector: expression_statement precedes: pattern: "if $VAR: $$$B" fix: '' ### Example [​](https://ast-grep.github.io/catalog/python/use-walrus-operator-in-if.html#example) python a = foo() if a: do_bar() ### Diff [​](https://ast-grep.github.io/catalog/python/use-walrus-operator-in-if.html#diff) python a = foo() if a: if a := foo(): do_bar() ### Contributed by [​](https://ast-grep.github.io/catalog/python/use-walrus-operator-in-if.html#contributed-by) Inspired by reddit user [/u/jackerhack](https://www.reddit.com/r/rust/comments/13eg738/comment/kagdklw/?) --- # ast-grep [Skip to content](https://ast-grep.github.io/catalog/python/rewrite-sqlalchemy-mapped-column.html#VPContent) Return to top Are you an LLM? You can read better optimized documentation at /catalog/python/rewrite-sqlalchemy-mapped-column.md for this page in Markdown format Rewrite SQLAlchemy with Type Annotations Has Fix [​](https://ast-grep.github.io/catalog/python/rewrite-sqlalchemy-mapped-column.html#rewrite-sqlalchemy-with-type-annotations) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- * [Playground Link](https://ast-grep.github.io/playground.html#eyJtb2RlIjoiQ29uZmlnIiwibGFuZyI6InB5dGhvbiIsInF1ZXJ5IjoiYShudWxsYWJsZT1UcnVlKSIsInJld3JpdGUiOiIxMjMiLCJzdHJpY3RuZXNzIjoic21hcnQiLCJzZWxlY3RvciI6ImtleXdvcmRfYXJndW1lbnQiLCJjb25maWciOiJpZDogcmVtb3ZlLW51bGxhYmxlLWFyZ1xubGFuZ3VhZ2U6IHB5dGhvblxucnVsZTpcbiAgcGF0dGVybjogJFggPSBtYXBwZWRfY29sdW1uKCQkJEFSR1MpXG4gIGFueTpcbiAgICAtIHBhdHRlcm46ICRYID0gbWFwcGVkX2NvbHVtbigkJCRCRUZPUkUsIFN0cmluZywgJCQkTUlELCBudWxsYWJsZT1UcnVlLCAkJCRBRlRFUilcbiAgICAtIHBhdHRlcm46ICRYID0gbWFwcGVkX2NvbHVtbigkJCRCRUZPUkUsIFN0cmluZywgJCQkTUlELCBudWxsYWJsZT1UcnVlKVxucmV3cml0ZXJzOlxuLSBpZDogZmlsdGVyLXN0cmluZy1udWxsYWJsZVxuICBydWxlOlxuICAgIHBhdHRlcm46ICRBUkdcbiAgICBpbnNpZGU6XG4gICAgICBraW5kOiBhcmd1bWVudF9saXN0XG4gICAgYWxsOlxuICAgIC0gbm90OiBcbiAgICAgICAgcGF0dGVybjogU3RyaW5nXG4gICAgLSBub3Q6XG4gICAgICAgIHBhdHRlcm46XG4gICAgICAgICAgY29udGV4dDogYShudWxsYWJsZT1UcnVlKVxuICAgICAgICAgIHNlbGVjdG9yOiBrZXl3b3JkX2FyZ3VtZW50XG4gIGZpeDogJEFSR1xuXG50cmFuc2Zvcm06XG4gIE5FV0FSR1M6XG4gICAgcmV3cml0ZTpcbiAgICAgIHJld3JpdGVyczogW2ZpbHRlci1zdHJpbmctbnVsbGFibGVdXG4gICAgICBzb3VyY2U6ICQkJEFSR1NcbiAgICAgIGpvaW5CeTogJywgJ1xuZml4OiB8LVxuICAkWDogTWFwcGVkW3N0ciB8IE5vbmVdID0gbWFwcGVkX2NvbHVtbigkTkVXQVJHUykiLCJzb3VyY2UiOiJtZXNzYWdlID0gbWFwcGVkX2NvbHVtbihTdHJpbmcsIGRlZmF1bHQ9XCJoZWxsb1wiLCBudWxsYWJsZT1UcnVlKVxuXG5tZXNzYWdlID0gbWFwcGVkX2NvbHVtbihTdHJpbmcsIG51bGxhYmxlPVRydWUpXG5cbl9tZXNzYWdlID0gbWFwcGVkX2NvbHVtbihcIm1lc3NhZ2VcIiwgU3RyaW5nLCBudWxsYWJsZT1UcnVlKVxuXG5tZXNzYWdlID0gbWFwcGVkX2NvbHVtbihTdHJpbmcsIG51bGxhYmxlPVRydWUsIHVuaXF1ZT1UcnVlKVxuXG5tZXNzYWdlID0gbWFwcGVkX2NvbHVtbihcbiAgU3RyaW5nLCBpbmRleD1UcnVlLCBudWxsYWJsZT1UcnVlLCB1bmlxdWU9VHJ1ZSlcblxuIyBTaG91bGQgbm90IGJlIHRyYW5zZm9ybWVkXG5tZXNzYWdlID0gbWFwcGVkX2NvbHVtbihTdHJpbmcsIGRlZmF1bHQ9XCJoZWxsb1wiKVxuXG5tZXNzYWdlID0gbWFwcGVkX2NvbHVtbihTdHJpbmcsIGRlZmF1bHQ9XCJoZWxsb1wiLCBudWxsYWJsZT1GYWxzZSlcblxubWVzc2FnZSA9IG1hcHBlZF9jb2x1bW4oSW50ZWdlciwgZGVmYXVsdD1cImhlbGxvXCIpIn0=) ### Description [​](https://ast-grep.github.io/catalog/python/rewrite-sqlalchemy-mapped-column.html#description) [SQLAlchemy 2.0](https://docs.sqlalchemy.org/en/20/orm/declarative_tables.html) recommends using type annotations with `Mapped` type for modern declarative mapping. The `mapped_column()` construct can derive its configuration from [PEP 484](https://peps.python.org/pep-0484/) type annotations. This rule helps migrate legacy SQLAlchemy code that explicitly uses `String` type and `nullable=True` to the modern type annotation approach using `Mapped[str | None]`. The key technique demonstrated here is using **rewriters** to selectively filter arguments. The rewriter: 1. Matches each argument inside the `argument_list` 2. Excludes the `String` type argument 3. Excludes the `nullable=True` keyword argument 4. Keeps all other arguments ### YAML [​](https://ast-grep.github.io/catalog/python/rewrite-sqlalchemy-mapped-column.html#yaml) yaml id: remove-nullable-arg language: python rule: pattern: $X = mapped_column($$$ARGS) any: - pattern: $X = mapped_column($$$BEFORE, String, $$$MID, nullable=True, $$$AFTER) - pattern: $X = mapped_column($$$BEFORE, String, $$$MID, nullable=True) rewriters: - id: filter-string-nullable rule: pattern: $ARG inside: kind: argument_list all: - not: pattern: String - not: pattern: context: a(nullable=True) selector: keyword_argument fix: $ARG transform: NEWARGS: rewrite: rewriters: [filter-string-nullable] source: $$$ARGS joinBy: ', ' fix: |- $X: Mapped[str | None] = mapped_column($NEWARGS) ### Example [​](https://ast-grep.github.io/catalog/python/rewrite-sqlalchemy-mapped-column.html#example) python message = mapped_column(String, default="hello", nullable=True) message = mapped_column(String, nullable=True) _message = mapped_column("message", String, nullable=True) message = mapped_column(String, nullable=True, unique=True) message = mapped_column( String, index=True, nullable=True, unique=True) # Should not be transformed message = mapped_column(String, default="hello") message = mapped_column(String, default="hello", nullable=False) message = mapped_column(Integer, default="hello") ### Diff [​](https://ast-grep.github.io/catalog/python/rewrite-sqlalchemy-mapped-column.html#diff) python message = mapped_column(String, default="hello", nullable=True) message: Mapped[str | None] = mapped_column(default="hello") message = mapped_column(String, nullable=True) message: Mapped[str | None] = mapped_column() _message = mapped_column("message", String, nullable=True) _message: Mapped[str | None] = mapped_column("message") message = mapped_column(String, nullable=True, unique=True) message: Mapped[str | None] = mapped_column(unique=True) message = mapped_column( String, index=True, nullable=True, unique=True) message: Mapped[str | None] = mapped_column( index=True, unique=True) ### Contributed by [​](https://ast-grep.github.io/catalog/python/rewrite-sqlalchemy-mapped-column.html#contributed-by) Inspired by [discussion #2319](https://github.com/ast-grep/ast-grep/discussions/2319) --- # ast-grep [Skip to content](https://ast-grep.github.io/catalog/ruby/detect-path-traversal.html#VPContent) Return to top Are you an LLM? You can read better optimized documentation at /catalog/ruby/detect-path-traversal.md for this page in Markdown format Detect Path Traversal Vulnerability in Rails [​](https://ast-grep.github.io/catalog/ruby/detect-path-traversal.html#detect-path-traversal-vulnerability-in-rails) ------------------------------------------------------------------------------------------------------------------------------------------------------------------ * [Playground Link](https://ast-grep.github.io/playground.html#eyJtb2RlIjoiQ29uZmlnIiwibGFuZyI6InJ1YnkiLCJxdWVyeSI6IiIsInJld3JpdGUiOiIiLCJzdHJpY3RuZXNzIjoiYXN0Iiwic2VsZWN0b3IiOiIiLCJjb25maWciOiJpZDogcGF0aC10cmF2ZXJzYWxcbm1lc3NhZ2U6IFBvdGVudGlhbCBQYXRoIFRyYXZlcnNhbCB2dWxuZXJhYmlsaXR5IGRldGVjdGVkLiBVc2VyIGlucHV0IGlzIGJlaW5nIHVzZWQgdG8gY29uc3RydWN0IGZpbGUgcGF0aHMgd2l0aG91dCB2YWxpZGF0aW9uLlxuc2V2ZXJpdHk6IGhpbnRcbmxhbmd1YWdlOiBSdWJ5XG5ub3RlOiB8XG4gIFBhdGggVHJhdmVyc2FsIChEaXJlY3RvcnkgVHJhdmVyc2FsKSBvY2N1cnMgd2hlbiB1c2VyIGlucHV0IGlzIHVzZWQgdG8gY29uc3RydWN0IGZpbGUgcGF0aHNcbiAgd2l0aG91dCBwcm9wZXIgdmFsaWRhdGlvbi4gVGhpcyBhbGxvd3MgYXR0YWNrZXJzIHRvIGFjY2VzcyBmaWxlcyBvdXRzaWRlIHRoZSBpbnRlbmRlZCBkaXJlY3RvcnkuXG4gIFZhbGlkYXRlIGFuZCBzYW5pdGl6ZSBmaWxlIHBhdGhzLCBhbmQgdXNlIEZpbGUuYmFzZW5hbWUoKSBvciBzaW1pbGFyIGZ1bmN0aW9ucy5cblxucnVsZTpcbiAgYW55OlxuICAgIC0gcGF0dGVybjogUmFpbHMucm9vdC5qb2luKCQkJCwgJFZBUiwgJCQkKVxuICAgIC0gcGF0dGVybjogRmlsZS5qb2luKCQkJCwgJFZBUiwgJCQkKVxuICAgIC0gcGF0dGVybjogc2VuZF9maWxlICRWQVIiLCJzb3VyY2UiOiIjIOaknOWHuuOBleOCjOOCi+OCs+ODvOODieS+i1xuIyDjg5Hjgr/jg7zjg7MxOiBSYWlscy5yb290LmpvaW4gd2l0aCB2YXJpYWJsZVxuUmFpbHMucm9vdC5qb2luKCd1cGxvYWRzJywgcGFyYW1zWzpmaWxlbmFtZV0pXG5SYWlscy5yb290LmpvaW4oJ2RhdGEnLCB1c2VyX2lucHV0LCAnZmlsZS50eHQnKVxuXG4jIOODkeOCv+ODvOODszI6IEZpbGUuam9pbiB3aXRoIHZhcmlhYmxlXG5GaWxlLmpvaW4oJy92YXIvd3d3JywgcGFyYW1zWzpwYXRoXSlcbkZpbGUuam9pbihiYXNlX3BhdGgsIHVzZXJfaWQsIGZpbGVuYW1lKVxuXG4jIOODkeOCv+ODvOODszM6IHNlbmRfZmlsZSB3aXRoIHZhcmlhYmxlXG5zZW5kX2ZpbGUgcGFyYW1zWzpmaWxlXVxuc2VuZF9maWxlIHVzZXIuZG9jdW1lbnRfcGF0aCJ9) ### Description [​](https://ast-grep.github.io/catalog/ruby/detect-path-traversal.html#description) Path Traversal (Directory Traversal) occurs when user input is used to construct file paths without proper validation. This allows attackers to access files outside the intended directory by using special characters like `../` to navigate the filesystem. This rule detects common path traversal patterns in Rails applications where user-controlled variables are used in: * `Rails.root.join()` - Building file paths relative to the Rails application root * `File.join()` - Constructing file paths * `send_file` - Sending files to users To prevent path traversal vulnerabilities, always validate and sanitize file paths, use `File.basename()` to extract only the filename, or use allowlists for permitted files. ### YAML [​](https://ast-grep.github.io/catalog/ruby/detect-path-traversal.html#yaml) yaml id: path-traversal message: Potential Path Traversal vulnerability detected. User input is being used to construct file paths without validation. severity: hint language: Ruby note: | Path Traversal (Directory Traversal) occurs when user input is used to construct file paths without proper validation. This allows attackers to access files outside the intended directory. Validate and sanitize file paths, and use File.basename() or similar functions. rule: any: - pattern: Rails.root.join($$$, $VAR, $$$) - pattern: File.join($$$, $VAR, $$$) - pattern: send_file $VAR ### Example [​](https://ast-grep.github.io/catalog/ruby/detect-path-traversal.html#example) rb # Pattern 1: Rails.root.join with variable Rails.root.join('uploads', params[:filename]) Rails.root.join('data', user_input, 'file.txt') # Pattern 2: File.join with variable File.join('/var/www', params[:path]) File.join(base_path, user_id, filename) # Pattern 3: send_file with variable send_file params[:file] send_file user.document_path ### Contributed by [​](https://ast-grep.github.io/catalog/ruby/detect-path-traversal.html#contributed-by) [sora fs0414](https://x.com/_fs0414) from this [blog post](https://fs0414.hatenablog.com/entry/2025/11/02/032114) --- # ast-grep [Skip to content](https://ast-grep.github.io/catalog/rust/avoid-duplicated-exports.html#VPContent) Return to top Are you an LLM? You can read better optimized documentation at /catalog/rust/avoid-duplicated-exports.md for this page in Markdown format Avoid Duplicated Exports [​](https://ast-grep.github.io/catalog/rust/avoid-duplicated-exports.html#avoid-duplicated-exports) ----------------------------------------------------------------------------------------------------------------------------- * [Playground Link](https://ast-grep.github.io/playground.html#eyJtb2RlIjoiQ29uZmlnIiwibGFuZyI6InJ1c3QiLCJxdWVyeSI6IiIsImNvbmZpZyI6InJ1bGU6XG4gIGFsbDpcbiAgICAgLSBwYXR0ZXJuOiBwdWIgdXNlICRCOjokQztcbiAgICAgLSBpbnNpZGU6XG4gICAgICAgIGtpbmQ6IHNvdXJjZV9maWxlXG4gICAgICAgIGhhczpcbiAgICAgICAgICBwYXR0ZXJuOiBwdWIgbW9kICRBO1xuICAgICAtIGhhczpcbiAgICAgICAgcGF0dGVybjogJEFcbiAgICAgICAgc3RvcEJ5OiBlbmQiLCJzb3VyY2UiOiJwdWIgbW9kIGZvbztcbnB1YiB1c2UgZm9vOjpGb287XG5wdWIgdXNlIGZvbzo6QTo6QjtcblxuXG5wdWIgdXNlIGFhYTo6QTtcbnB1YiB1c2Ugd29vOjpXb287In0=) ### Description [​](https://ast-grep.github.io/catalog/rust/avoid-duplicated-exports.html#description) Generally, we don't encourage the use of re-exports. However, sometimes, to keep the interface exposed by a lib crate tidy, we use re-exports to shorten the path to specific items. When doing so, a pitfall is to export a single item under two different names. Consider: rs pub mod foo; pub use foo::Foo; The issue with this code, is that `Foo` is now exposed under two different paths: `Foo`, `foo::Foo`. This unnecessarily increases the surface of your API. It can also cause issues on the client side. For example, it makes the usage of auto-complete in the IDE more involved. Instead, ensure you export only once with `pub`. ### YAML [​](https://ast-grep.github.io/catalog/rust/avoid-duplicated-exports.html#yaml) yaml id: avoid-duplicate-export language: rust rule: all: - pattern: pub use $B::$C; - inside: kind: source_file has: pattern: pub mod $A; - has: pattern: $A stopBy: end ### Example [​](https://ast-grep.github.io/catalog/rust/avoid-duplicated-exports.html#example) rs pub mod foo; pub use foo::Foo; pub use foo::A::B; pub use aaa::A; pub use woo::Woo; ### Contributed by [​](https://ast-grep.github.io/catalog/rust/avoid-duplicated-exports.html#contributed-by) Julius Lungys([voidpumpkin](https://github.com/voidpumpkin) ) --- # ast-grep [Skip to content](https://ast-grep.github.io/catalog/ruby/migrate-action-filter.html#VPContent) Return to top Are you an LLM? You can read better optimized documentation at /catalog/ruby/migrate-action-filter.md for this page in Markdown format Migrate action\_filter in Ruby on Rails Has Fix [​](https://ast-grep.github.io/catalog/ruby/migrate-action-filter.html#migrate-action-filter-in-ruby-on-rails) --------------------------------------------------------------------------------------------------------------------------------------------------------------- * [Playground Link](https://ast-grep.github.io/playground.html#eyJtb2RlIjoiQ29uZmlnIiwibGFuZyI6InJ1YnkiLCJxdWVyeSI6ImNvbnNvbGUubG9nKCRNQVRDSCkiLCJyZXdyaXRlIjoibG9nZ2VyLmxvZygkTUFUQ0gpIiwiY29uZmlnIjoiIyBhc3QtZ3JlcCBZQU1MIFJ1bGUgaXMgcG93ZXJmdWwgZm9yIGxpbnRpbmchXG4jIGh0dHBzOi8vYXN0LWdyZXAuZ2l0aHViLmlvL2d1aWRlL3J1bGUtY29uZmlnLmh0bWwjcnVsZVxucnVsZTpcbiAgYW55OlxuICAgIC0gcGF0dGVybjogYmVmb3JlX2ZpbHRlciAkJCRBQ1RJT05cbiAgICAtIHBhdHRlcm46IGFyb3VuZF9maWx0ZXIgJCQkQUNUSU9OXG4gICAgLSBwYXR0ZXJuOiBhZnRlcl9maWx0ZXIgJCQkQUNUSU9OXG4gIGhhczpcbiAgICBwYXR0ZXJuOiAkRklMVEVSXG4gICAgZmllbGQ6IG1ldGhvZFxuZml4OiBcbiAgJE5FV19BQ1RJT04gJCQkQUNUSU9OXG50cmFuc2Zvcm06XG4gIE5FV19BQ1RJT046XG4gICAgcmVwbGFjZTpcbiAgICAgIHNvdXJjZTogJEZJTFRFUlxuICAgICAgcmVwbGFjZTogX2ZpbHRlclxuICAgICAgYnk6IF9hY3Rpb24iLCJzb3VyY2UiOiJjbGFzcyBUb2Rvc0NvbnRyb2xsZXIgPCBBcHBsaWNhdGlvbkNvbnRyb2xsZXJcbiAgYmVmb3JlX2ZpbHRlciA6YXV0aGVudGljYXRlXG4gIGFyb3VuZF9maWx0ZXIgOndyYXBfaW5fdHJhbnNhY3Rpb24sIG9ubHk6IDpzaG93XG4gIGFmdGVyX2ZpbHRlciBkbyB8Y29udHJvbGxlcnxcbiAgICBmbGFzaFs6ZXJyb3JdID0gXCJZb3UgbXVzdCBiZSBsb2dnZWQgaW5cIlxuICBlbmRcblxuICBkZWYgaW5kZXhcbiAgICBAdG9kb3MgPSBUb2RvLmFsbFxuICBlbmRcbmVuZFxuIn0=) ### Description [​](https://ast-grep.github.io/catalog/ruby/migrate-action-filter.html#description) This rule is used to migrate `{before,after,around}_filter` to `{before,after,around}_action` in Ruby on Rails controllers. These are methods that run before, after or around an action is executed, and they can be used to check permissions, set variables, redirect requests, log events, etc. However, these methods are [deprecated](https://stackoverflow.com/questions/16519828/rails-4-before-filter-vs-before-action) in Rails 5.0 and will be removed in Rails 5.1. `{before,after,around}_action` are the new syntax for the same functionality. This rule will replace all occurrences of `{before,after,around}_filter` with `{before,after,around}_action` in the controller code. ### YAML [​](https://ast-grep.github.io/catalog/ruby/migrate-action-filter.html#yaml) yaml id: migration-action-filter language: ruby rule: any: - pattern: before_filter $$$ACTION - pattern: around_filter $$$ACTION - pattern: after_filter $$$ACTION has: pattern: $FILTER field: method fix: $NEW_ACTION $$$ACTION transform: NEW_ACTION: replace: source: $FILTER replace: _filter by: _action ### Example [​](https://ast-grep.github.io/catalog/ruby/migrate-action-filter.html#example) rb class TodosController < ApplicationController before_filter :authenticate around_filter :wrap_in_transaction, only: :show after_filter do |controller| flash[:error] = "You must be logged in" end def index @todos = Todo.all end end ### Diff [​](https://ast-grep.github.io/catalog/ruby/migrate-action-filter.html#diff) rb class TodosController < ApplicationController before_action :authenticate before_filter :authenticate around_action :wrap_in_transaction, only: :show around_filter :wrap_in_transaction, only: :show after_action do |controller| flash[:error] = "You must be logged in" end after_filter do |controller| flash[:error] = "You must be logged in" end def index @todos = Todo.all end end ### Contributed by [​](https://ast-grep.github.io/catalog/ruby/migrate-action-filter.html#contributed-by) [Herrington Darkholme](https://twitter.com/hd_nvim) , inspired by [Future of Ruby - AST Tooling](https://dev.to/baweaver/future-of-ruby-ast-tooling-9i1) . --- # ast-grep [Skip to content](https://ast-grep.github.io/catalog/rust/get-digit-count-in-usize.html#VPContent) Return to top Are you an LLM? You can read better optimized documentation at /catalog/rust/get-digit-count-in-usize.md for this page in Markdown format Get number of digits in a `usize` Has Fix [​](https://ast-grep.github.io/catalog/rust/get-digit-count-in-usize.html#get-number-of-digits-in-a-usize) ----------------------------------------------------------------------------------------------------------------------------------------------------- * [Playground Link](https://ast-grep.github.io/playground.html#eyJtb2RlIjoiUGF0Y2giLCJsYW5nIjoicnVzdCIsInF1ZXJ5IjoiJE5VTS50b19zdHJpbmcoKS5jaGFycygpLmNvdW50KCkiLCJyZXdyaXRlIjoiJE5VTS5jaGVja2VkX2lsb2cxMCgpLnVud3JhcF9vcigwKSArIDEiLCJjb25maWciOiIjIFlBTUwgUnVsZSBpcyBtb3JlIHBvd2VyZnVsIVxuIyBodHRwczovL2FzdC1ncmVwLmdpdGh1Yi5pby9ndWlkZS9ydWxlLWNvbmZpZy5odG1sI3J1bGVcbnJ1bGU6XG4gIGFueTpcbiAgICAtIHBhdHRlcm46IGNvbnNvbGUubG9nKCRBKVxuICAgIC0gcGF0dGVybjogY29uc29sZS5kZWJ1ZygkQSlcbmZpeDpcbiAgbG9nZ2VyLmxvZygkQSkiLCJzb3VyY2UiOiJsZXQgd2lkdGggPSAobGluZXMgKyBudW0pLnRvX3N0cmluZygpLmNoYXJzKCkuY291bnQoKTsifQ==) ### Description [​](https://ast-grep.github.io/catalog/rust/get-digit-count-in-usize.html#description) Getting the number of digits in a usize number can be useful for various purposes, such as counting the column width of line numbers in a text editor or formatting the output of a number with commas or spaces. A common but inefficient way of getting the number of digits in a `usize` number is to use `num.to_string().chars().count()`. This method converts the number to a string, iterates over its characters, and counts them. However, this method involves allocating a new string, which can be costly in terms of memory and time. A better alternative is to use [`checked_ilog10`](https://doc.rust-lang.org/std/primitive.usize.html#method.checked_ilog10) . rs num.checked_ilog10().unwrap_or(0) + 1 The snippet above computes the integer logarithm base 10 of the number and adds one. This snippet does not allocate any memory and is faster than the string conversion approach. The [efficient](https://doc.rust-lang.org/src/core/num/int_log10.rs.html) `checked_ilog10` function returns an `Option` that is `Some(log)` if the number is positive and `None` if the number is zero. The `unwrap_or(0)` function returns the value inside the option or `0` if the option is `None`. ### Pattern [​](https://ast-grep.github.io/catalog/rust/get-digit-count-in-usize.html#pattern) shell ast-grep -p '$NUM.to_string().chars().count()' \ -r '$NUM.checked_ilog10().unwrap_or(0) + 1' \ -l rs ### Example [​](https://ast-grep.github.io/catalog/rust/get-digit-count-in-usize.html#example) rs let width = (lines + num).to_string().chars().count(); ### Diff [​](https://ast-grep.github.io/catalog/rust/get-digit-count-in-usize.html#diff) rs let width = (lines + num).to_string().chars().count(); let width = (lines + num).checked_ilog10().unwrap_or(0) + 1; ### Contributed by [​](https://ast-grep.github.io/catalog/rust/get-digit-count-in-usize.html#contributed-by) [Herrington Darkholme](https://twitter.com/hd_nvim) , inspired by [dogfooding ast-grep](https://github.com/ast-grep/ast-grep/issues/550) --- # ast-grep [Skip to content](https://ast-grep.github.io/catalog/rust/boshen-footgun.html#VPContent) Return to top Are you an LLM? You can read better optimized documentation at /catalog/rust/boshen-footgun.md for this page in Markdown format Beware of char offset when iterate over a string Has Fix [​](https://ast-grep.github.io/catalog/rust/boshen-footgun.html#beware-of-char-offset-when-iterate-over-a-string) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------- * [Playground Link](https://ast-grep.github.io/playground.html#eyJtb2RlIjoiUGF0Y2giLCJsYW5nIjoicnVzdCIsInF1ZXJ5IjoiJEEuY2hhcnMoKS5lbnVtZXJhdGUoKSIsInJld3JpdGUiOiIkQS5jaGFyX2luZGljZXMoKSIsImNvbmZpZyI6IiIsInNvdXJjZSI6ImZvciAoaSwgY2hhcikgaW4gc291cmNlLmNoYXJzKCkuZW51bWVyYXRlKCkge1xuICAgIHByaW50bG4hKFwiQm9zaGVuIGlzIGFuZ3J5IDopXCIpO1xufSJ9) ### Description [​](https://ast-grep.github.io/catalog/rust/boshen-footgun.html#description) It's a common pitfall in Rust that counting _character offset_ is not the same as counting _byte offset_ when iterating through a string. Rust string is represented by utf-8 byte array, which is a variable-length encoding scheme. `chars().enumerate()` will yield the character offset, while [`char_indices()`](https://doc.rust-lang.org/std/primitive.str.html#method.char_indices) will yield the byte offset. rs let yes = "y̆es"; let mut char_indices = yes.char_indices(); assert_eq!(Some((0, 'y')), char_indices.next()); // not (0, 'y̆') assert_eq!(Some((1, '\u{0306}')), char_indices.next()); // note the 3 here - the last character took up two bytes assert_eq!(Some((3, 'e')), char_indices.next()); assert_eq!(Some((4, 's')), char_indices.next()); Depending on your use case, you may want to use `char_indices()` instead of `chars().enumerate()`. ### Pattern [​](https://ast-grep.github.io/catalog/rust/boshen-footgun.html#pattern) shell ast-grep -p '$A.chars().enumerate()' \ -r '$A.char_indices()' \ -l rs ### Example [​](https://ast-grep.github.io/catalog/rust/boshen-footgun.html#example) rs for (i, char) in source.chars().enumerate() { println!("Boshen is angry :)"); } ### Diff [​](https://ast-grep.github.io/catalog/rust/boshen-footgun.html#diff) rs for (i, char) in source.chars().enumerate() { for (i, char) in source.char_indices() { println!("Boshen is angry :)"); } ### Contributed by [​](https://ast-grep.github.io/catalog/rust/boshen-footgun.html#contributed-by) Inspired by [Boshen's Tweet](https://x.com/boshen_c/status/1719033308682870891) ![Boshen's footgun](https://pbs.twimg.com/media/F9s7mJHaYAEndnY?format=jpg&name=medium) --- # ast-grep [Skip to content](https://ast-grep.github.io/catalog/tsx/avoid-nested-links.html#VPContent) Return to top Are you an LLM? You can read better optimized documentation at /catalog/tsx/avoid-nested-links.md for this page in Markdown format Avoid nested links [​](https://ast-grep.github.io/catalog/tsx/avoid-nested-links.html#avoid-nested-links) ---------------------------------------------------------------------------------------------------------- * [Playground Link](https://ast-grep.github.io/playground.html#eyJtb2RlIjoiQ29uZmlnIiwibGFuZyI6InRzeCIsInF1ZXJ5IjoiaWYgKCRBKSB7ICQkJEIgfSIsInJld3JpdGUiOiJpZiAoISgkQSkpIHtcbiAgICByZXR1cm47XG59XG4kJCRCIiwic3RyaWN0bmVzcyI6InNtYXJ0Iiwic2VsZWN0b3IiOiIiLCJjb25maWciOiJpZDogbm8tbmVzdGVkLWxpbmtzXG5sYW5ndWFnZTogdHN4XG5zZXZlcml0eTogZXJyb3JcbnJ1bGU6XG4gIHBhdHRlcm46IDxhICQkJD4kJCRBPC9hPlxuICBoYXM6XG4gICAgcGF0dGVybjogPGEgJCQkPiQkJDwvYT5cbiAgICBzdG9wQnk6IGVuZCIsInNvdXJjZSI6ImZ1bmN0aW9uIENvbXBvbmVudCgpIHtcbiAgcmV0dXJuIDxhIGhyZWY9Jy9kZXN0aW5hdGlvbic+XG4gICAgPGEgaHJlZj0nL2Fub3RoZXJkZXN0aW5hdGlvbic+TmVzdGVkIGxpbmshPC9hPlxuICA8L2E+O1xufVxuZnVuY3Rpb24gT2theUNvbXBvbmVudCgpIHtcbiAgcmV0dXJuIDxhIGhyZWY9Jy9kZXN0aW5hdGlvbic+XG4gICAgSSBhbSBqdXN0IGEgbGluay5cbiAgPC9hPjtcbn0ifQ==) ### Description [​](https://ast-grep.github.io/catalog/tsx/avoid-nested-links.html#description) React will produce a warning message if you nest a link element inside of another link element. This rule will catch this mistake! ### YAML [​](https://ast-grep.github.io/catalog/tsx/avoid-nested-links.html#yaml) yaml id: no-nested-links language: tsx severity: error rule: pattern: $$$A has: pattern: $$$ stopBy: end ### Example [​](https://ast-grep.github.io/catalog/tsx/avoid-nested-links.html#example) tsx function Component() { return Nested link! ; } function OkayComponent() { return I am just a link. ; } ### Contributed by [​](https://ast-grep.github.io/catalog/tsx/avoid-nested-links.html#contributed-by) [Tom MacWright](https://macwright.com/) --- # ast-grep [Skip to content](https://ast-grep.github.io/catalog/rust/redundant-unsafe-function.html#VPContent) Return to top Are you an LLM? You can read better optimized documentation at /catalog/rust/redundant-unsafe-function.md for this page in Markdown format Unsafe Function Without Unsafe Block [​](https://ast-grep.github.io/catalog/rust/redundant-unsafe-function.html#unsafe-function-without-unsafe-block) ------------------------------------------------------------------------------------------------------------------------------------------------------ * [Playground Link](https://ast-grep.github.io/playground.html#eyJtb2RlIjoiQ29uZmlnIiwibGFuZyI6InJ1c3QiLCJxdWVyeSI6IntcbiAgZGVzY3JpcHRpb24gPSAkQVxufSIsInJld3JpdGUiOiIiLCJzdHJpY3RuZXNzIjoic21hcnQiLCJzZWxlY3RvciI6ImJpbmRpbmciLCJjb25maWciOiIgIGlkOiByZWR1bmRhbnQtdW5zYWZlLWZ1bmN0aW9uXG4gIGxhbmd1YWdlOiBydXN0XG4gIHNldmVyaXR5OiBlcnJvclxuICBtZXNzYWdlOiBVbnNhZmUgZnVuY3Rpb24gd2l0aG91dCB1bnNhZmUgYmxvY2sgaW5zaWRlXG4gIG5vdGU6IHxcbiAgICBDb25zaWRlciB3aGV0aGVyIHRoaXMgZnVuY3Rpb24gbmVlZHMgdG8gYmUgbWFya2VkIHVuc2FmZSBcbiAgICBvciBpZiB1bnNhZmUgb3BlcmF0aW9ucyBzaG91bGQgYmUgd3JhcHBlZCBpbiBhbiB1bnNhZmUgYmxvY2tcbiAgcnVsZTpcbiAgICBhbGw6XG4gICAgICAtIGtpbmQ6IGZ1bmN0aW9uX2l0ZW1cbiAgICAgIC0gaGFzOlxuICAgICAgICAgIGtpbmQ6IGZ1bmN0aW9uX21vZGlmaWVyc1xuICAgICAgICAgIHJlZ2V4OiBcIl51bnNhZmVcIlxuICAgICAgLSBub3Q6XG4gICAgICAgICAgaGFzOlxuICAgICAgICAgICAga2luZDogdW5zYWZlX2Jsb2NrXG4gICAgICAgICAgICBzdG9wQnk6IGVuZCIsInNvdXJjZSI6IiAgLy8gU2hvdWxkIG1hdGNoIC0gdW5zYWZlIGZ1bmN0aW9uIHdpdGhvdXQgdW5zYWZlIGJsb2NrIChubyByZXR1cm4gdHlwZSlcbiAgdW5zYWZlIGZuIHJlZHVuZGFudF91bnNhZmUoKSB7XG4gICAgICBwcmludGxuIShcIk5vIHVuc2FmZSBvcGVyYXRpb25zIGhlcmVcIik7XG4gIH1cblxuICAvLyBTaG91bGQgbWF0Y2ggLSB1bnNhZmUgZnVuY3Rpb24gd2l0aCByZXR1cm4gdHlwZSwgbm8gdW5zYWZlIGJsb2NrXG4gIHVuc2FmZSBmbiByZWR1bmRhbnRfd2l0aF9yZXR1cm4oKSAtPiBpMzIge1xuICAgICAgbGV0IHggPSA1O1xuICAgICAgeCArIDEwXG4gIH1cblxuICAvLyBTaG91bGQgbWF0Y2ggLSB1bnNhZmUgZnVuY3Rpb24gd2l0aCBjb21wbGV4IHJldHVybiB0eXBlXG4gIHVuc2FmZSBmbiByZWR1bmRhbnRfY29tcGxleF9yZXR1cm4oKSAtPiBSZXN1bHQ8U3RyaW5nLCBzdGQ6OmlvOjpFcnJvcj4ge1xuICAgICAgT2soU3RyaW5nOjpmcm9tKFwic2FmZSBvcGVyYXRpb25cIikpXG4gIH1cblxuICAvLyBTaG91bGQgTk9UIG1hdGNoIC0gdW5zYWZlIGZ1bmN0aW9uIHdpdGggdW5zYWZlIGJsb2NrXG4gIHVuc2FmZSBmbiBwcm9wZXJfdW5zYWZlKCkgLT4gKmNvbnN0IGkzMiB7XG4gICAgICB1bnNhZmUge1xuICAgICAgICAgIGxldCBwdHIgPSAweDEyMzQgYXMgKmNvbnN0IGkzMjtcbiAgICAgICAgICBwdHJcbiAgICAgIH1cbiAgfVxuXG4gIC8vIFNob3VsZCBtYXRjaCAtIHVuc2FmZSBhc3luYyBmdW5jdGlvbiB3aXRob3V0IHVuc2FmZSBibG9ja1xuICB1bnNhZmUgYXN5bmMgZm4gYXN5bmNfcmVkdW5kYW50KCkgLT4gaTMyIHtcbiAgICAgIDQyXG4gIH1cblxuICAvLyBTaG91bGQgbWF0Y2ggLSB1bnNhZmUgY29uc3QgZnVuY3Rpb25cbiAgdW5zYWZlIGNvbnN0IGZuIGNvbnN0X3JlZHVuZGFudCgpIC0+IGkzMiB7XG4gICAgICAxMDBcbiAgfVxuXG4gIC8vIFNob3VsZCBOT1QgbWF0Y2ggLSByZWd1bGFyIGZ1bmN0aW9uXG4gIGZuIHJlZ3VsYXJfZnVuY3Rpb24oKSAtPiBpMzIge1xuICAgICAgNDJcbiAgfSJ9) ### Description [​](https://ast-grep.github.io/catalog/rust/redundant-unsafe-function.html#description) This rule detects functions marked with the `unsafe` keyword that do not contain any `unsafe` blocks in their body. When a function is marked `unsafe`, it indicates that the function contains operations that the compiler cannot verify as safe. However, if the function body doesn't contain any `unsafe` blocks, it may be unnecessarily marked as `unsafe`. This could be a sign that: 1. The function should not be marked `unsafe` if it doesn't perform any unsafe operations 2. Or if there are unsafe operations, they should be explicitly wrapped in `unsafe` blocks for clarity This rule helps identify such cases so developers can review whether the `unsafe` marker is truly necessary or if the code needs to be refactored. ### YAML [​](https://ast-grep.github.io/catalog/rust/redundant-unsafe-function.html#yaml) yaml id: redundant-unsafe-function language: rust severity: error message: Unsafe function without unsafe block inside note: | Consider whether this function needs to be marked unsafe or if unsafe operations should be wrapped in an unsafe block rule: all: - kind: function_item - has: kind: function_modifiers regex: "^unsafe" - not: has: kind: unsafe_block stopBy: end ### Example [​](https://ast-grep.github.io/catalog/rust/redundant-unsafe-function.html#example) rs // Should match - unsafe function without unsafe block (no return type) unsafe fn redundant_unsafe() { println!("No unsafe operations here"); } // Should match - unsafe function with return type, no unsafe block unsafe fn redundant_with_return() -> i32 { let x = 5; x + 10 } // Should match - unsafe function with complex return type unsafe fn redundant_complex_return() -> Result { Ok(String::from("safe operation")) } // Should NOT match - unsafe function with unsafe block unsafe fn proper_unsafe() -> *const i32 { unsafe { let ptr = 0x1234 as *const i32; ptr } } // Should match - unsafe async function without unsafe block unsafe async fn async_redundant() -> i32 { 42 } // Should match - unsafe const function unsafe const fn const_redundant() -> i32 { 100 } // Should NOT match - regular function fn regular_function() -> i32 { 42 } ### Contributed by [​](https://ast-grep.github.io/catalog/rust/redundant-unsafe-function.html#contributed-by) Inspired by [@hd\_nvim's Tweet](https://x.com/hd_nvim/status/1992810384072585397?s=20) --- # ast-grep [Skip to content](https://ast-grep.github.io/catalog/rust/rewrite-indoc-macro.html#VPContent) Return to top Are you an LLM? You can read better optimized documentation at /catalog/rust/rewrite-indoc-macro.md for this page in Markdown format Rewrite `indoc!` macro Has Fix [​](https://ast-grep.github.io/catalog/rust/rewrite-indoc-macro.html#rewrite-indoc-macro) ------------------------------------------------------------------------------------------------------------------------- * [Playground Link](https://ast-grep.github.io/playground.html#eyJtb2RlIjoiUGF0Y2giLCJsYW5nIjoicnVzdCIsInF1ZXJ5IjoiaW5kb2MhIHsgciNcIiQkJEFcIiMgfSIsInJld3JpdGUiOiJgJCQkQWAiLCJzdHJpY3RuZXNzIjoicmVsYXhlZCIsInNlbGVjdG9yIjoiIiwiY29uZmlnIjoicnVsZTogXG4gYW55OlxuIC0gcGF0dGVybjogJFYgPT09ICRTRU5TRVRJVkVXT1JEXG4gLSBwYXR0ZXJuOiAkU0VOU0VUSVZFV09SRCA9PT0gJFZcbmNvbnN0cmFpbnRzOlxuICBTRU5TRVRJVkVXT1JEOlxuICAgIHJlZ2V4OiBwYXNzd29yZCIsInNvdXJjZSI6ImZuIG1haW4oKSB7XG4gICAgaW5kb2MhIHtyI1wiXG4gICAgICAgIC5mb28ge1xuICAgICAgICAgICAgb3JkZXI6IDE7XG4gICAgICAgIH1cbiAgICBcIiN9O1xufSJ9) ### Description [​](https://ast-grep.github.io/catalog/rust/rewrite-indoc-macro.html#description) This example, created from [a Tweet](https://x.com/zack_overflow/status/1885065128590401551) , shows a refactoring operation being performed on Rust source code. The changes involve removing `indoc!` macro declarations while preserving the CSS-like content within them. Previously, the same refactor is implemented by a _unreadable monster regex_ in vim syntax. Click to see the original regex (neovim, btw) vimscript :%s/\v(indoc!|)(| )([|\{)r#"(([^#]+|\n+)+)"#/`\4`\ \ I have to confess that I don't understand this regex even if I use neovim, btw.\ \ Let Claude break it down piece by piece:\ \ * `:%s/` - Vim substitution command for all lines\ * `\v` - Very magic mode in vim for simpler regex syntax\ * `(indoc!|)` - First capture group: matches either "indoc!" or nothing\ * `(| )` - Second capture group: matches either empty string or a space\ * `([|\{)` - Third capture group: matches either `[` or `{`\ * `r#"` - Matches literal `r#"` (Rust raw string delimiter)\ * `(([^#]+|\n+)+)` - Fourth capture group (nested):\ * `[^#]+` - One or more non-# characters\ * `|\n+` - OR one or more newlines\ * Outer `()+` makes this repeat one or more times\ * `"#` - Matches the closing raw string delimiter\ * \`\\4\` - Replaces with the fourth capture group wrapped in backticks\ \ This regex is designed to find Rust raw string literals (possibly wrapped in `indoc!` macro), capture their content, and replace the entire match with just the content wrapped in backticks. It's more precise than my previous explanation and matches the pattern you're showing.\ \ ### Pattern [​](https://ast-grep.github.io/catalog/rust/rewrite-indoc-macro.html#pattern)\ \ shell\ \ ast-grep --pattern 'indoc! { r#"$$$A"# }' --rewrite '`$$$A`' sgtest.rs\ \ ### Example [​](https://ast-grep.github.io/catalog/rust/rewrite-indoc-macro.html#example)\ \ rs\ \ fn main() {\ indoc! {r#"\ .foo {\ order: 1;\ }\ "#};\ }\ \ ### Diff [​](https://ast-grep.github.io/catalog/rust/rewrite-indoc-macro.html#diff)\ \ rs\ \ fn main() {\ indoc! {r#" // [!code --]\ `.foo { // [!code ++]\ order: 1;\ }\ "#}; \ `; \ }\ \ ### Contributed by [​](https://ast-grep.github.io/catalog/rust/rewrite-indoc-macro.html#contributed-by)\ \ [Zack in SF](https://x.com/zack_overflow) --- # ast-grep [Skip to content](https://ast-grep.github.io/catalog/tsx/avoid-jsx-short-circuit.html#VPContent) Return to top Are you an LLM? You can read better optimized documentation at /catalog/tsx/avoid-jsx-short-circuit.md for this page in Markdown format Avoid `&&` short circuit in JSX Has Fix [​](https://ast-grep.github.io/catalog/tsx/avoid-jsx-short-circuit.html#avoid-short-circuit-in-jsx) -------------------------------------------------------------------------------------------------------------------------------------------- * [Playground Link](https://ast-grep.github.io/playground.html#eyJtb2RlIjoiQ29uZmlnIiwibGFuZyI6InRzeCIsInF1ZXJ5IjoiY29uc29sZS5sb2coJE1BVENIKSIsInJld3JpdGUiOiJsb2dnZXIubG9nKCRNQVRDSCkiLCJjb25maWciOiJpZDogZG8td2hhdC1icm9vb29vb2tseW4tc2FpZFxubGFuZ3VhZ2U6IFRzeFxuc2V2ZXJpdHk6IGVycm9yXG5ydWxlOlxuICBraW5kOiBqc3hfZXhwcmVzc2lvblxuICBoYXM6XG4gICAgcGF0dGVybjogJEEgJiYgJEJcbiAgbm90OlxuICAgIGluc2lkZTpcbiAgICAgIGtpbmQ6IGpzeF9hdHRyaWJ1dGVcbmZpeDogXCJ7JEEgPyAkQiA6IG51bGx9XCIiLCJzb3VyY2UiOiI8ZGl2PntcbiAgbnVtICYmIDxkaXYvPlxufTwvZGl2PiJ9) ### Description [​](https://ast-grep.github.io/catalog/tsx/avoid-jsx-short-circuit.html#description) In [React](https://react.dev/learn/conditional-rendering) , you can conditionally render JSX using JavaScript syntax like `if` statements, `&&`, and `? :` operators. However, you should almost never put numbers on the left side of `&&`. This is because React will render the number `0`, instead of the JSX element on the right side. A concrete example will be conditionally rendering a list when the list is not empty. This rule will find and fix any short-circuit rendering in JSX and rewrite it to a ternary operator. ### YAML [​](https://ast-grep.github.io/catalog/tsx/avoid-jsx-short-circuit.html#yaml) yaml id: do-what-brooooooklyn-said language: Tsx rule: kind: jsx_expression has: pattern: $A && $B not: inside: kind: jsx_attribute fix: "{$A ? $B : null}" ### Example [​](https://ast-grep.github.io/catalog/tsx/avoid-jsx-short-circuit.html#example) tsx

{ list.length && list.map(i =>

) }

### Diff [​](https://ast-grep.github.io/catalog/tsx/avoid-jsx-short-circuit.html#diff) tsx
{ list.length && list.map(i =>

) }

{ list.length ? list.map(i =>

) : null }

### Contributed by [​](https://ast-grep.github.io/catalog/tsx/avoid-jsx-short-circuit.html#contributed-by) [Herrington Darkholme](https://twitter.com/hd_nvim) , inspired by [@Brooooook\_lyn](https://twitter.com/Brooooook_lyn/status/1666637274757595141) --- # Ruby | ast-grep [Skip to content](https://ast-grep.github.io/catalog/ruby/#VPContent) Return to top Are you an LLM? You can read better optimized documentation at /catalog/ruby.md for this page in Markdown format Ruby [​](https://ast-grep.github.io/catalog/ruby/#ruby) ======================================================== This page curates a list of example ast-grep rules to check and to rewrite Ruby applications. Migrate action\_filter in Ruby on Rails Has Fix [​](https://ast-grep.github.io/catalog/ruby/#migrate-action-filter-in-ruby-on-rails) ------------------------------------------------------------------------------------------------------------------------------------- * [Playground Link](https://ast-grep.github.io/playground.html#eyJtb2RlIjoiQ29uZmlnIiwibGFuZyI6InJ1YnkiLCJxdWVyeSI6ImNvbnNvbGUubG9nKCRNQVRDSCkiLCJyZXdyaXRlIjoibG9nZ2VyLmxvZygkTUFUQ0gpIiwiY29uZmlnIjoiIyBhc3QtZ3JlcCBZQU1MIFJ1bGUgaXMgcG93ZXJmdWwgZm9yIGxpbnRpbmchXG4jIGh0dHBzOi8vYXN0LWdyZXAuZ2l0aHViLmlvL2d1aWRlL3J1bGUtY29uZmlnLmh0bWwjcnVsZVxucnVsZTpcbiAgYW55OlxuICAgIC0gcGF0dGVybjogYmVmb3JlX2ZpbHRlciAkJCRBQ1RJT05cbiAgICAtIHBhdHRlcm46IGFyb3VuZF9maWx0ZXIgJCQkQUNUSU9OXG4gICAgLSBwYXR0ZXJuOiBhZnRlcl9maWx0ZXIgJCQkQUNUSU9OXG4gIGhhczpcbiAgICBwYXR0ZXJuOiAkRklMVEVSXG4gICAgZmllbGQ6IG1ldGhvZFxuZml4OiBcbiAgJE5FV19BQ1RJT04gJCQkQUNUSU9OXG50cmFuc2Zvcm06XG4gIE5FV19BQ1RJT046XG4gICAgcmVwbGFjZTpcbiAgICAgIHNvdXJjZTogJEZJTFRFUlxuICAgICAgcmVwbGFjZTogX2ZpbHRlclxuICAgICAgYnk6IF9hY3Rpb24iLCJzb3VyY2UiOiJjbGFzcyBUb2Rvc0NvbnRyb2xsZXIgPCBBcHBsaWNhdGlvbkNvbnRyb2xsZXJcbiAgYmVmb3JlX2ZpbHRlciA6YXV0aGVudGljYXRlXG4gIGFyb3VuZF9maWx0ZXIgOndyYXBfaW5fdHJhbnNhY3Rpb24sIG9ubHk6IDpzaG93XG4gIGFmdGVyX2ZpbHRlciBkbyB8Y29udHJvbGxlcnxcbiAgICBmbGFzaFs6ZXJyb3JdID0gXCJZb3UgbXVzdCBiZSBsb2dnZWQgaW5cIlxuICBlbmRcblxuICBkZWYgaW5kZXhcbiAgICBAdG9kb3MgPSBUb2RvLmFsbFxuICBlbmRcbmVuZFxuIn0=) ### Description [​](https://ast-grep.github.io/catalog/ruby/#description) This rule is used to migrate `{before,after,around}_filter` to `{before,after,around}_action` in Ruby on Rails controllers. These are methods that run before, after or around an action is executed, and they can be used to check permissions, set variables, redirect requests, log events, etc. However, these methods are [deprecated](https://stackoverflow.com/questions/16519828/rails-4-before-filter-vs-before-action) in Rails 5.0 and will be removed in Rails 5.1. `{before,after,around}_action` are the new syntax for the same functionality. This rule will replace all occurrences of `{before,after,around}_filter` with `{before,after,around}_action` in the controller code. ### YAML [​](https://ast-grep.github.io/catalog/ruby/#yaml) yaml id: migration-action-filter language: ruby rule: any: - pattern: before_filter $$$ACTION - pattern: around_filter $$$ACTION - pattern: after_filter $$$ACTION has: pattern: $FILTER field: method fix: $NEW_ACTION $$$ACTION transform: NEW_ACTION: replace: source: $FILTER replace: _filter by: _action ### Example [​](https://ast-grep.github.io/catalog/ruby/#example) rb class TodosController < ApplicationController before_filter :authenticate around_filter :wrap_in_transaction, only: :show after_filter do |controller| flash[:error] = "You must be logged in" end def index @todos = Todo.all end end ### Diff [​](https://ast-grep.github.io/catalog/ruby/#diff) rb class TodosController < ApplicationController before_action :authenticate before_filter :authenticate around_action :wrap_in_transaction, only: :show around_filter :wrap_in_transaction, only: :show after_action do |controller| flash[:error] = "You must be logged in" end after_filter do |controller| flash[:error] = "You must be logged in" end def index @todos = Todo.all end end ### Contributed by [​](https://ast-grep.github.io/catalog/ruby/#contributed-by) [Herrington Darkholme](https://twitter.com/hd_nvim) , inspired by [Future of Ruby - AST Tooling](https://dev.to/baweaver/future-of-ruby-ast-tooling-9i1) . Prefer Symbol over Proc Has Fix [​](https://ast-grep.github.io/catalog/ruby/#prefer-symbol-over-proc) ------------------------------------------------------------------------------------------------------ * [Playground Link](https://ast-grep.github.io/playground.html#eyJtb2RlIjoiQ29uZmlnIiwibGFuZyI6InJ1YnkiLCJxdWVyeSI6IiRMSVNULnNlbGVjdCB7IHwkVnwgJFYuJE1FVEhPRCB9IiwicmV3cml0ZSI6IiRMSVNULnNlbGVjdCgmOiRNRVRIT0QpIiwiY29uZmlnIjoiaWQ6IHByZWZlci1zeW1ib2wtb3Zlci1wcm9jXG5ydWxlOlxuICBwYXR0ZXJuOiAkTElTVC4kSVRFUiB7IHwkVnwgJFYuJE1FVEhPRCB9XG5sYW5ndWFnZTogUnVieVxuY29uc3RyYWludHM6XG4gIElURVI6XG4gICAgcmVnZXg6ICdtYXB8c2VsZWN0fGVhY2gnXG5maXg6ICckTElTVC4kSVRFUigmOiRNRVRIT0QpJ1xuIiwic291cmNlIjoiWzEsIDIsIDNdLnNlbGVjdCB7IHx2fCB2LmV2ZW4/IH1cbigxLi4xMDApLmVhY2ggeyB8aXwgaS50b19zIH1cbm5vdF9saXN0Lm5vX21hdGNoIHsgfHZ8IHYuZXZlbj8gfVxuIn0=) ### Description [​](https://ast-grep.github.io/catalog/ruby/#description-1) Ruby has a more concise symbol shorthand `&:` to invoke methods. This rule simplifies `proc` to `symbol`. This example is inspired by this [dev.to article](https://dev.to/baweaver/future-of-ruby-ast-tooling-9i1) . ### YAML [​](https://ast-grep.github.io/catalog/ruby/#yaml-1) yaml id: prefer-symbol-over-proc language: ruby rule: pattern: $LIST.$ITER { |$V| $V.$METHOD } constraints: ITER: regex: 'map|select|each' fix: '$LIST.$ITER(&:$METHOD)' ### Example [​](https://ast-grep.github.io/catalog/ruby/#example-1) rb [1, 2, 3].select { |v| v.even? } (1..100).each { |i| i.to_s } not_list.no_match { |v| v.even? } ### Diff [​](https://ast-grep.github.io/catalog/ruby/#diff-1) rb [1, 2, 3].select { |v| v.even? } [1, 2, 3].select(&:even?) (1..100).each { |i| i.to_s } (1..100).each(&:to_s) not_list.no_match { |v| v.even? } ### Contributed by [​](https://ast-grep.github.io/catalog/ruby/#contributed-by-1) [Herrington Darkholme](https://twitter.com/hd_nvim) Detect Path Traversal Vulnerability in Rails [​](https://ast-grep.github.io/catalog/ruby/#detect-path-traversal-vulnerability-in-rails) ---------------------------------------------------------------------------------------------------------------------------------------- * [Playground Link](https://ast-grep.github.io/playground.html#eyJtb2RlIjoiQ29uZmlnIiwibGFuZyI6InJ1YnkiLCJxdWVyeSI6IiIsInJld3JpdGUiOiIiLCJzdHJpY3RuZXNzIjoiYXN0Iiwic2VsZWN0b3IiOiIiLCJjb25maWciOiJpZDogcGF0aC10cmF2ZXJzYWxcbm1lc3NhZ2U6IFBvdGVudGlhbCBQYXRoIFRyYXZlcnNhbCB2dWxuZXJhYmlsaXR5IGRldGVjdGVkLiBVc2VyIGlucHV0IGlzIGJlaW5nIHVzZWQgdG8gY29uc3RydWN0IGZpbGUgcGF0aHMgd2l0aG91dCB2YWxpZGF0aW9uLlxuc2V2ZXJpdHk6IGhpbnRcbmxhbmd1YWdlOiBSdWJ5XG5ub3RlOiB8XG4gIFBhdGggVHJhdmVyc2FsIChEaXJlY3RvcnkgVHJhdmVyc2FsKSBvY2N1cnMgd2hlbiB1c2VyIGlucHV0IGlzIHVzZWQgdG8gY29uc3RydWN0IGZpbGUgcGF0aHNcbiAgd2l0aG91dCBwcm9wZXIgdmFsaWRhdGlvbi4gVGhpcyBhbGxvd3MgYXR0YWNrZXJzIHRvIGFjY2VzcyBmaWxlcyBvdXRzaWRlIHRoZSBpbnRlbmRlZCBkaXJlY3RvcnkuXG4gIFZhbGlkYXRlIGFuZCBzYW5pdGl6ZSBmaWxlIHBhdGhzLCBhbmQgdXNlIEZpbGUuYmFzZW5hbWUoKSBvciBzaW1pbGFyIGZ1bmN0aW9ucy5cblxucnVsZTpcbiAgYW55OlxuICAgIC0gcGF0dGVybjogUmFpbHMucm9vdC5qb2luKCQkJCwgJFZBUiwgJCQkKVxuICAgIC0gcGF0dGVybjogRmlsZS5qb2luKCQkJCwgJFZBUiwgJCQkKVxuICAgIC0gcGF0dGVybjogc2VuZF9maWxlICRWQVIiLCJzb3VyY2UiOiIjIOaknOWHuuOBleOCjOOCi+OCs+ODvOODieS+i1xuIyDjg5Hjgr/jg7zjg7MxOiBSYWlscy5yb290LmpvaW4gd2l0aCB2YXJpYWJsZVxuUmFpbHMucm9vdC5qb2luKCd1cGxvYWRzJywgcGFyYW1zWzpmaWxlbmFtZV0pXG5SYWlscy5yb290LmpvaW4oJ2RhdGEnLCB1c2VyX2lucHV0LCAnZmlsZS50eHQnKVxuXG4jIOODkeOCv+ODvOODszI6IEZpbGUuam9pbiB3aXRoIHZhcmlhYmxlXG5GaWxlLmpvaW4oJy92YXIvd3d3JywgcGFyYW1zWzpwYXRoXSlcbkZpbGUuam9pbihiYXNlX3BhdGgsIHVzZXJfaWQsIGZpbGVuYW1lKVxuXG4jIOODkeOCv+ODvOODszM6IHNlbmRfZmlsZSB3aXRoIHZhcmlhYmxlXG5zZW5kX2ZpbGUgcGFyYW1zWzpmaWxlXVxuc2VuZF9maWxlIHVzZXIuZG9jdW1lbnRfcGF0aCJ9) ### Description [​](https://ast-grep.github.io/catalog/ruby/#description-2) Path Traversal (Directory Traversal) occurs when user input is used to construct file paths without proper validation. This allows attackers to access files outside the intended directory by using special characters like `../` to navigate the filesystem. This rule detects common path traversal patterns in Rails applications where user-controlled variables are used in: * `Rails.root.join()` - Building file paths relative to the Rails application root * `File.join()` - Constructing file paths * `send_file` - Sending files to users To prevent path traversal vulnerabilities, always validate and sanitize file paths, use `File.basename()` to extract only the filename, or use allowlists for permitted files. ### YAML [​](https://ast-grep.github.io/catalog/ruby/#yaml-2) yaml id: path-traversal message: Potential Path Traversal vulnerability detected. User input is being used to construct file paths without validation. severity: hint language: Ruby note: | Path Traversal (Directory Traversal) occurs when user input is used to construct file paths without proper validation. This allows attackers to access files outside the intended directory. Validate and sanitize file paths, and use File.basename() or similar functions. rule: any: - pattern: Rails.root.join($$$, $VAR, $$$) - pattern: File.join($$$, $VAR, $$$) - pattern: send_file $VAR ### Example [​](https://ast-grep.github.io/catalog/ruby/#example-2) rb # Pattern 1: Rails.root.join with variable Rails.root.join('uploads', params[:filename]) Rails.root.join('data', user_input, 'file.txt') # Pattern 2: File.join with variable File.join('/var/www', params[:path]) File.join(base_path, user_id, filename) # Pattern 3: send_file with variable send_file params[:file] send_file user.document_path ### Contributed by [​](https://ast-grep.github.io/catalog/ruby/#contributed-by-2) [sora fs0414](https://x.com/_fs0414) from this [blog post](https://fs0414.hatenablog.com/entry/2025/11/02/032114) --- # ast-grep [Skip to content](https://ast-grep.github.io/catalog/tsx/redundant-usestate-type.html#VPContent) Return to top Are you an LLM? You can read better optimized documentation at /catalog/tsx/redundant-usestate-type.md for this page in Markdown format Unnecessary `useState` Type Has Fix [​](https://ast-grep.github.io/catalog/tsx/redundant-usestate-type.html#unnecessary-usestate-type) --------------------------------------------------------------------------------------------------------------------------------------- * [Playground Link](https://ast-grep.github.io/playground.html#eyJtb2RlIjoiUGF0Y2giLCJsYW5nIjoidHlwZXNjcmlwdCIsInF1ZXJ5IjoidXNlU3RhdGU8c3RyaW5nPigkQSkiLCJyZXdyaXRlIjoidXNlU3RhdGUoJEEpIiwiY29uZmlnIjoiIyBZQU1MIFJ1bGUgaXMgbW9yZSBwb3dlcmZ1bCFcbiMgaHR0cHM6Ly9hc3QtZ3JlcC5naXRodWIuaW8vZ3VpZGUvcnVsZS1jb25maWcuaHRtbCNydWxlXG5ydWxlOlxuICBhbnk6XG4gICAgLSBwYXR0ZXJuOiBjb25zb2xlLmxvZygkQSlcbiAgICAtIHBhdHRlcm46IGNvbnNvbGUuZGVidWcoJEEpXG5maXg6XG4gIGxvZ2dlci5sb2coJEEpIiwic291cmNlIjoiZnVuY3Rpb24gQ29tcG9uZW50KCkge1xuICBjb25zdCBbbmFtZSwgc2V0TmFtZV0gPSB1c2VTdGF0ZTxzdHJpbmc+KCdSZWFjdCcpXG59In0=) ### Description [​](https://ast-grep.github.io/catalog/tsx/redundant-usestate-type.html#description) React's [`useState`](https://react.dev/reference/react/useState) is a Hook that lets you add a state variable to your component. The type annotation of `useState`'s generic type argument, for example `useState(123)`, is unnecessary if TypeScript can infer the type of the state variable from the initial value. We can usually skip annotating if the generic type argument is a single primitive type like `number`, `string` or `boolean`. ### Pattern [​](https://ast-grep.github.io/catalog/tsx/redundant-usestate-type.html#pattern) numberstringboolean bash ast-grep -p 'useState($A)' -r 'useState($A)' -l tsx bash ast-grep -p 'useState($A)' -r 'useState($A)' bash ast-grep -p 'useState($A)' -r 'useState($A)' ### Example [​](https://ast-grep.github.io/catalog/tsx/redundant-usestate-type.html#example) ts function Component() { const [name, setName] = useState('React') } ### Diff [​](https://ast-grep.github.io/catalog/tsx/redundant-usestate-type.html#diff) ts function Component() { const [name, setName] = useState('React') const [name, setName] = useState('React') } ### Contributed by [​](https://ast-grep.github.io/catalog/tsx/redundant-usestate-type.html#contributed-by) [Herrington Darkholme](https://twitter.com/hd_nvim) --- # Python | ast-grep [Skip to content](https://ast-grep.github.io/catalog/python/#VPContent) Return to top Are you an LLM? You can read better optimized documentation at /catalog/python.md for this page in Markdown format Python [​](https://ast-grep.github.io/catalog/python/#python) ============================================================== This page curates a list of example ast-grep rules to check and to rewrite Python code. Migrate OpenAI SDK Has Fix [​](https://ast-grep.github.io/catalog/python/#migrate-openai-sdk) ---------------------------------------------------------------------------------------------- * [Playground Link](https://ast-grep.github.io/playground.html#eyJtb2RlIjoiQ29uZmlnIiwibGFuZyI6InB5dGhvbiIsInF1ZXJ5IjoiZGVmICRGVU5DKCQkJEFSR1MpOiAkJCRCT0RZIiwicmV3cml0ZSI6IiIsImNvbmZpZyI6InJ1bGU6XG4gIHBhdHRlcm46IGltcG9ydCBvcGVuYWlcbmZpeDogZnJvbSBvcGVuYWkgaW1wb3J0IENsaWVudFxuLS0tXG5ydWxlOlxuICBwYXR0ZXJuOiBvcGVuYWkuYXBpX2tleSA9ICRLRVlcbmZpeDogY2xpZW50ID0gQ2xpZW50KCRLRVkpXG4tLS1cbnJ1bGU6XG4gIHBhdHRlcm46IG9wZW5haS5Db21wbGV0aW9uLmNyZWF0ZSgkJCRBUkdTKVxuZml4OiB8LVxuICBjbGllbnQuY29tcGxldGlvbnMuY3JlYXRlKFxuICAgICQkJEFSR1NcbiAgKSIsInNvdXJjZSI6ImltcG9ydCBvc1xuaW1wb3J0IG9wZW5haVxuZnJvbSBmbGFzayBpbXBvcnQgRmxhc2ssIGpzb25pZnlcblxuYXBwID0gRmxhc2soX19uYW1lX18pXG5vcGVuYWkuYXBpX2tleSA9IG9zLmdldGVudihcIk9QRU5BSV9BUElfS0VZXCIpXG5cblxuQGFwcC5yb3V0ZShcIi9jaGF0XCIsIG1ldGhvZHM9KFwiUE9TVFwiKSlcbmRlZiBpbmRleCgpOlxuICAgIGFuaW1hbCA9IHJlcXVlc3QuZm9ybVtcImFuaW1hbFwiXVxuICAgIHJlc3BvbnNlID0gb3BlbmFpLkNvbXBsZXRpb24uY3JlYXRlKFxuICAgICAgICBtb2RlbD1cInRleHQtZGF2aW5jaS0wMDNcIixcbiAgICAgICAgcHJvbXB0PWdlbmVyYXRlX3Byb21wdChhbmltYWwpLFxuICAgICAgICB0ZW1wZXJhdHVyZT0wLjYsXG4gICAgKVxuICAgIHJldHVybiBqc29uaWZ5KHJlc3BvbnNlLmNob2ljZXMpIn0=) ### Description [​](https://ast-grep.github.io/catalog/python/#description) OpenAI has introduced some breaking changes in their API, such as using `Client` to initialize the service and renaming the `Completion` method to `completions` . This example shows how to use ast-grep to automatically update your code to the new API. API migration requires multiple related rules to work together. The example shows how to write [multiple rules](https://ast-grep.github.io/reference/playground.html#test-multiple-rules) in a [single YAML](https://ast-grep.github.io/guide/rewrite-code.html#using-fix-in-yaml-rule) file. The rules and patterns in the example are simple and self-explanatory, so we will not explain them further. ### YAML [​](https://ast-grep.github.io/catalog/python/#yaml) yaml id: import-openai language: python rule: pattern: import openai fix: from openai import Client --- id: rewrite-client language: python rule: pattern: openai.api_key = $KEY fix: client = Client($KEY) --- id: rewrite-chat-completion language: python rule: pattern: openai.Completion.create($$$ARGS) fix: |- client.completions.create( $$$ARGS ) ### Example [​](https://ast-grep.github.io/catalog/python/#example) python import os import openai from flask import Flask, jsonify app = Flask(__name__) openai.api_key = os.getenv("OPENAI_API_KEY") @app.route("/chat", methods=("POST")) def index(): animal = request.form["animal"] response = openai.Completion.create( model="text-davinci-003", prompt=generate_prompt(animal), temperature=0.6, ) return jsonify(response.choices) ### Diff [​](https://ast-grep.github.io/catalog/python/#diff) python import os import openai from openai import Client from flask import Flask, jsonify app = Flask(__name__) openai.api_key = os.getenv("OPENAI_API_KEY") client = Client(os.getenv("OPENAI_API_KEY")) @app.route("/chat", methods=("POST")) def index(): animal = request.form["animal"] response = openai.Completion.create( response = client.completions.create( model="text-davinci-003", prompt=generate_prompt(animal), temperature=0.6, ) return jsonify(response.choices) ### Contributed by [​](https://ast-grep.github.io/catalog/python/#contributed-by) [Herrington Darkholme](https://twitter.com/hd_nvim) , inspired by [Morgante](https://twitter.com/morgantepell/status/1721668781246750952) from [grit.io](https://www.grit.io/) Prefer Generator Expressions Has Fix [​](https://ast-grep.github.io/catalog/python/#prefer-generator-expressions) ------------------------------------------------------------------------------------------------------------------ * [Playground Link](https://ast-grep.github.io/playground.html#eyJtb2RlIjoiQ29uZmlnIiwibGFuZyI6InB5dGhvbiIsInF1ZXJ5IjoiWyQkJEFdIiwicmV3cml0ZSI6IiRBPy4oKSIsImNvbmZpZyI6InJ1bGU6XG4gIHBhdHRlcm46ICRGVU5DKCRMSVNUKVxuY29uc3RyYWludHM6XG4gIExJU1Q6IHsga2luZDogbGlzdF9jb21wcmVoZW5zaW9uIH1cbiAgRlVOQzpcbiAgICBhbnk6XG4gICAgICAtIHBhdHRlcm46IGFueVxuICAgICAgLSBwYXR0ZXJuOiBhbGxcbiAgICAgIC0gcGF0dGVybjogc3VtXG4gICAgICAjIC4uLlxudHJhbnNmb3JtOlxuICBJTk5FUjpcbiAgICBzdWJzdHJpbmc6IHtzb3VyY2U6ICRMSVNULCBzdGFydENoYXI6IDEsIGVuZENoYXI6IC0xIH1cbmZpeDogJEZVTkMoJElOTkVSKSIsInNvdXJjZSI6ImFsbChbeCBmb3IgeCBpbiB5XSlcblt4IGZvciB4IGluIHldIn0=) ### Description [​](https://ast-grep.github.io/catalog/python/#description-1) List comprehensions like `[x for x in range(10)]` are a concise way to create lists in Python. However, we can achieve better memory efficiency by using generator expressions like `(x for x in range(10))` instead. List comprehensions create the entire list in memory, while generator expressions generate each element one at a time. We can make the change by replacing the square brackets with parentheses. ### YAML [​](https://ast-grep.github.io/catalog/python/#yaml-1) yaml id: prefer-generator-expressions language: python rule: pattern: $LIST kind: list_comprehension transform: INNER: substring: {source: $LIST, startChar: 1, endChar: -1 } fix: ($INNER) This rule converts every list comprehension to a generator expression. However, **not every list comprehension can be replaced with a generator expression.** If the list is used multiple times, is modified, is sliced, or is indexed, a generator is not a suitable replacement. Some common functions like `any`, `all`, and `sum` take an `iterable` as an argument. A generator function counts as an `iterable`, so it is safe to change a list comprehension to a generator expression in this context. yaml id: prefer-generator-expressions language: python rule: pattern: $FUNC($LIST) constraints: LIST: { kind: list_comprehension } FUNC: any: - pattern: any - pattern: all - pattern: sum # ... transform: INNER: substring: {source: $LIST, startChar: 1, endChar: -1 } fix: $FUNC($INNER) ### Example [​](https://ast-grep.github.io/catalog/python/#example-1) python any([x for x in range(10)]) ### Diff [​](https://ast-grep.github.io/catalog/python/#diff-1) python any([x for x in range(10)]) any(x for x in range(10)) ### Contributed by [​](https://ast-grep.github.io/catalog/python/#contributed-by-1) [Steven Love](https://github.com/StevenLove) Use Walrus Operator in `if` statementHas Fix [​](https://ast-grep.github.io/catalog/python/#use-walrus-operator-in-if-statement) --------------------------------------------------------------------------------------------------------------------------------- * [Playground Link](https://ast-grep.github.io/playground.html#eyJtb2RlIjoiQ29uZmlnIiwibGFuZyI6InB5dGhvbiIsInF1ZXJ5IjoiZm4gbWFpbigpIHsgXG4gICAgJCQkO1xuICAgIGlmKCRBKXskJCRCfSBcbiAgICBpZigkQSl7JCQkQ30gXG4gICAgJCQkRlxufSIsInJld3JpdGUiOiJmbiBtYWluKCkgeyAkJCRFOyBpZigkQSl7JCQkQiAkJCRDfSAkJCRGfSIsImNvbmZpZyI6ImlkOiB1c2Utd2FscnVzLW9wZXJhdG9yXG5ydWxlOlxuICBmb2xsb3dzOlxuICAgIHBhdHRlcm46XG4gICAgICBjb250ZXh0OiAkVkFSID0gJCQkRVhQUlxuICAgICAgc2VsZWN0b3I6IGV4cHJlc3Npb25fc3RhdGVtZW50XG4gIHBhdHRlcm46IFwiaWYgJFZBUjogJCQkQlwiXG5maXg6IHwtXG4gIGlmICRWQVIgOj0gJCQkRVhQUjpcbiAgICAkJCRCXG4tLS1cbmlkOiByZW1vdmUtZGVjbGFyYXRpb25cbnJ1bGU6XG4gIHBhdHRlcm46XG4gICAgY29udGV4dDogJFZBUiA9ICQkJEVYUFJcbiAgICBzZWxlY3RvcjogZXhwcmVzc2lvbl9zdGF0ZW1lbnRcbiAgcHJlY2VkZXM6XG4gICAgcGF0dGVybjogXCJpZiAkVkFSOiAkJCRCXCJcbmZpeDogJyciLCJzb3VyY2UiOiJhID0gZm9vKClcblxuaWYgYTpcbiAgICBkb19iYXIoKSJ9) ### Description [​](https://ast-grep.github.io/catalog/python/#description-2) The walrus operator (`:=`) introduced in Python 3.8 allows you to assign values to variables as part of an expression. This rule aims to simplify code by using the walrus operator in `if` statements. This first part of the rule identifies cases where a variable is assigned a value and then immediately used in an `if` statement to control flow. yaml id: use-walrus-operator language: python rule: pattern: "if $VAR: $$$B" follows: pattern: context: $VAR = $$$EXPR selector: expression_statement fix: |- if $VAR := $$$EXPR: $$$B The `pattern` clause finds an `if` statement that checks the truthiness of `$VAR`. If this pattern `follows` an expression statement where `$VAR` is assigned `$$$EXPR`, the `fix` clause changes the `if` statements to use the walrus operator. The second part of the rule: yaml id: remove-declaration rule: pattern: context: $VAR = $$$EXPR selector: expression_statement precedes: pattern: "if $VAR: $$$B" fix: '' This rule removes the standalone variable assignment when it directly precedes an `if` statement that uses the walrus operator. Since the assignment is now part of the `if` statement, the separate declaration is no longer needed. By applying these rules, you can refactor your Python code to be more concise and readable, taking advantage of the walrus operator's ability to combine an assignment with an expression. ### YAML [​](https://ast-grep.github.io/catalog/python/#yaml-2) yaml id: use-walrus-operator language: python rule: follows: pattern: context: $VAR = $$$EXPR selector: expression_statement pattern: "if $VAR: $$$B" fix: |- if $VAR := $$$EXPR: $$$B --- id: remove-declaration language: python rule: pattern: context: $VAR = $$$EXPR selector: expression_statement precedes: pattern: "if $VAR: $$$B" fix: '' ### Example [​](https://ast-grep.github.io/catalog/python/#example-2) python a = foo() if a: do_bar() ### Diff [​](https://ast-grep.github.io/catalog/python/#diff-2) python a = foo() if a: if a := foo(): do_bar() ### Contributed by [​](https://ast-grep.github.io/catalog/python/#contributed-by-2) Inspired by reddit user [/u/jackerhack](https://www.reddit.com/r/rust/comments/13eg738/comment/kagdklw/?) Remove `async` function Has Fix [​](https://ast-grep.github.io/catalog/python/#remove-async-function) ------------------------------------------------------------------------------------------------------ * [Playground Link](https://ast-grep.github.io/playground.html#eyJtb2RlIjoiQ29uZmlnIiwibGFuZyI6InB5dGhvbiIsInF1ZXJ5IjoiYXdhaXQgJCQkQ0FMTCIsInJld3JpdGUiOiIkJCRDQUxMICIsImNvbmZpZyI6ImlkOiByZW1vdmUtYXN5bmMtZGVmXG5sYW5ndWFnZTogcHl0aG9uXG5ydWxlOlxuICBwYXR0ZXJuOlxuICAgIGNvbnRleHQ6ICdhc3luYyBkZWYgJEZVTkMoJCQkQVJHUyk6ICQkJEJPRFknXG4gICAgc2VsZWN0b3I6IGZ1bmN0aW9uX2RlZmluaXRpb25cbnRyYW5zZm9ybTpcbiAgUkVNT1ZFRF9CT0RZOlxuICAgIHJld3JpdGU6XG4gICAgICByZXdyaXRlcnM6IFtyZW1vdmUtYXdhaXQtY2FsbF1cbiAgICAgIHNvdXJjZTogJCQkQk9EWVxuZml4OiB8LVxuICBkZWYgJEZVTkMoJCQkQVJHUyk6XG4gICAgJFJFTU9WRURfQk9EWVxucmV3cml0ZXJzOlxuLSBpZDogcmVtb3ZlLWF3YWl0LWNhbGxcbiAgcnVsZTpcbiAgICBwYXR0ZXJuOiAnYXdhaXQgJCQkQ0FMTCdcbiAgZml4OiAkJCRDQUxMXG4iLCJzb3VyY2UiOiJhc3luYyBkZWYgbWFpbjMoKTpcbiAgYXdhaXQgc29tZWNhbGwoMSwgNSkifQ==) ### Description [​](https://ast-grep.github.io/catalog/python/#description-3) The `async` keyword in Python is used to define asynchronous functions that can be `await`ed. In this example, we want to remove the `async` keyword from a function definition and replace it with a synchronous version of the function. We also need to remove the `await` keyword from the function body. By default, ast-grep will not apply overlapping replacements. This means `await` keywords will not be modified because they are inside the async function body. However, we can use the [`rewriter`](https://ast-grep.github.io/reference/yaml/rewriter.html) to apply changes inside the matched function body. ### YAML [​](https://ast-grep.github.io/catalog/python/#yaml-3) yaml id: remove-async-def language: python rule: # match async function definition pattern: context: 'async def $FUNC($$$ARGS): $$$BODY' selector: function_definition rewriters: # define a rewriter to remove the await keyword remove-await-call: pattern: 'await $$$CALL' fix: $$$CALL # remove await keyword # apply the rewriter to the function body transform: REMOVED_BODY: rewrite: rewriters: [remove-await-call] source: $$$BODY fix: |- def $FUNC($$$ARGS): $REMOVED_BODY ### Example [​](https://ast-grep.github.io/catalog/python/#example-3) python async def main3(): await somecall(1, 5) ### Diff [​](https://ast-grep.github.io/catalog/python/#diff-3) python async def main3(): await somecall(1, 5) def main3(): somecall(1, 5) ### Contributed by [​](https://ast-grep.github.io/catalog/python/#contributed-by-3) Inspired by the ast-grep issue [#1185](https://github.com/ast-grep/ast-grep/issues/1185) Refactor pytest fixtures [​](https://ast-grep.github.io/catalog/python/#refactor-pytest-fixtures) -------------------------------------------------------------------------------------------------- * [Playground Link](https://ast-grep.github.io/playground.html#eyJtb2RlIjoiQ29uZmlnIiwibGFuZyI6InB5dGhvbiIsInF1ZXJ5IjoiZGVmIGZvbygkWCk6XG4gICRTIiwicmV3cml0ZSI6ImxvZ2dlci5sb2coJE1BVENIKSIsImNvbmZpZyI6ImlkOiBweXRlc3QtdHlwZS1oaW50LWZpeHR1cmVcbmxhbmd1YWdlOiBQeXRob25cbnV0aWxzOlxuICBpcy1maXh0dXJlLWZ1bmN0aW9uOlxuICAgIGtpbmQ6IGZ1bmN0aW9uX2RlZmluaXRpb25cbiAgICBmb2xsb3dzOlxuICAgICAga2luZDogZGVjb3JhdG9yXG4gICAgICBoYXM6XG4gICAgICAgIGtpbmQ6IGlkZW50aWZpZXJcbiAgICAgICAgcmVnZXg6IF5maXh0dXJlJFxuICAgICAgICBzdG9wQnk6IGVuZFxuICBpcy10ZXN0LWZ1bmN0aW9uOlxuICAgIGtpbmQ6IGZ1bmN0aW9uX2RlZmluaXRpb25cbiAgICBoYXM6XG4gICAgICBmaWVsZDogbmFtZVxuICAgICAgcmVnZXg6IF50ZXN0X1xuICBpcy1weXRlc3QtY29udGV4dDpcbiAgICAjIFB5dGVzdCBjb250ZXh0IGlzIGEgbm9kZSBpbnNpZGUgYSBweXRlc3RcbiAgICAjIHRlc3QvZml4dHVyZVxuICAgIGluc2lkZTpcbiAgICAgIHN0b3BCeTogZW5kXG4gICAgICBhbnk6XG4gICAgICAgIC0gbWF0Y2hlczogaXMtZml4dHVyZS1mdW5jdGlvblxuICAgICAgICAtIG1hdGNoZXM6IGlzLXRlc3QtZnVuY3Rpb25cbiAgaXMtZml4dHVyZS1hcmc6XG4gICAgIyBGaXh0dXJlIGFyZ3VtZW50cyBhcmUgaWRlbnRpZmllcnMgaW5zaWRlIHRoZSBcbiAgICAjIHBhcmFtZXRlcnMgb2YgYSB0ZXN0L2ZpeHR1cmUgZnVuY3Rpb25cbiAgICBhbGw6XG4gICAgICAtIGtpbmQ6IGlkZW50aWZpZXJcbiAgICAgIC0gbWF0Y2hlczogaXMtcHl0ZXN0LWNvbnRleHRcbiAgICAgIC0gaW5zaWRlOlxuICAgICAgICAgIGtpbmQ6IHBhcmFtZXRlcnNcbnJ1bGU6XG4gIG1hdGNoZXM6IGlzLWZpeHR1cmUtYXJnXG4gIHJlZ2V4OiBeZm9vJFxuZml4OiAnZm9vOiBpbnQnXG4iLCJzb3VyY2UiOiJmcm9tIGNvbGxlY3Rpb25zLmFiYyBpbXBvcnQgSXRlcmFibGVcbmZyb20gdHlwaW5nIGltcG9ydCBBbnlcblxuaW1wb3J0IHB5dGVzdFxuZnJvbSBweXRlc3QgaW1wb3J0IGZpeHR1cmVcblxuQHB5dGVzdC5maXh0dXJlKHNjb3BlPVwic2Vzc2lvblwiKVxuZGVmIGZvbygpIC0+IEl0ZXJhYmxlW2ludF06XG4gICAgeWllbGQgNVxuXG5AZml4dHVyZVxuZGVmIGJhcihmb28pIC0+IHN0cjpcbiAgICByZXR1cm4gc3RyKGZvbylcblxuZGVmIHJlZ3VsYXJfZnVuY3Rpb24oZm9vKSAtPiBOb25lOlxuICAgICMgVGhpcyBmdW5jdGlvbiBkb2Vzbid0IHVzZSB0aGUgJ2ZvbycgZml4dHVyZVxuICAgIHByaW50KGZvbylcblxuZGVmIHRlc3RfMShmb28sIGJhcik6XG4gICAgcHJpbnQoZm9vLCBiYXIpXG5cbmRlZiB0ZXN0XzIoYmFyKTpcbiAgICAuLi4ifQ==) ### Description [​](https://ast-grep.github.io/catalog/python/#description-4) One of the most commonly used testing framework in Python is [pytest](https://docs.pytest.org/en/8.2.x/) . Among other things, it allows the use of [fixtures](https://docs.pytest.org/en/6.2.x/fixture.html) . Fixtures are defined as functions that can be required in test code, or in other fixtures, as an argument. This means that all functions arguments with a given name in a pytest context (test function or fixture) are essentially the same entity. However, not every editor's LSP is able to keep track of this, making refactoring challenging. Using ast-grep, we can define some rules to match fixture definition and usage without catching similarly named entities in a non-test context. First, we define utils to select pytest test/fixture functions. yaml utils: is-fixture-function: kind: function_definition follows: kind: decorator has: kind: identifier regex: ^fixture$ stopBy: end is-test-function: kind: function_definition has: field: name regex: ^test_ Pytest fixtures are declared with a decorator `@pytest.fixture`. We match the `function_definition` node that directly follows a `decorator` node. That decorator node must have a `fixture` identifier somewhere. This accounts for different location of the `fixture` node depending on the type of imports and whether the decorator is used as is or called with parameters. Pytest functions are fairly straightforward to detect, as they always start with `test_` by convention. The next utils builds onto those two to incrementally: * Find if a node is inside a pytest context (test/fixture) * Find if a node is an argument in such a context yaml utils: is-pytest-context: # Pytest context is a node inside a pytest # test/fixture inside: stopBy: end any: - matches: is-fixture-function - matches: is-test-function is-fixture-arg: # Fixture arguments are identifiers inside the # parameters of a test/fixture function all: - kind: identifier - inside: kind: parameters - matches: is-pytest-context Once those utils are declared, you can perform various refactoring on a specific fixture. The following rule adds a type-hint to a fixture. yaml rule: matches: is-fixture-arg regex: ^foo$ fix: 'foo: int' This one renames a fixture and all its references. yaml rule: kind: identifier matches: is-fixture-context regex: ^foo$ fix: 'five' ### Example [​](https://ast-grep.github.io/catalog/python/#example-4) #### Renaming Fixtures [​](https://ast-grep.github.io/catalog/python/#renaming-fixtures) python @pytest.fixture def foo() -> int: return 5 @pytest.fixture(scope="function") def some_fixture(foo: int) -> str: return str(foo) def regular_function(foo) -> None: ... def test_code(foo: int) -> None: assert foo == 5 #### Diff [​](https://ast-grep.github.io/catalog/python/#diff-4) python @pytest.fixture def foo() -> int: def five() -> int: return 5 @pytest.fixture(scope="function") def some_fixture(foo: int) -> str: def some_fixture(five: int) -> str: return str(foo) def regular_function(foo) -> None: ... def test_code(foo: int) -> None: def test_code(five: int) -> None: assert foo == 5 assert five == 5 #### Type Hinting Fixtures [​](https://ast-grep.github.io/catalog/python/#type-hinting-fixtures) python @pytest.fixture def foo() -> int: return 5 @pytest.fixture(scope="function") def some_fixture(foo) -> str: return str(foo) def regular_function(foo) -> None: ... def test_code(foo) -> None: assert foo == 5 #### Diff [​](https://ast-grep.github.io/catalog/python/#diff-5) python @pytest.fixture def foo() -> int: return 5 @pytest.fixture(scope="function") def some_fixture(foo) -> str: def some_fixture(foo: int) -> str: return str(foo) def regular_function(foo) -> None: ... def test_code(foo) -> None: def test_code(foo: int) -> None: assert foo == 5 Rewrite `Optional[Type]` to `Type | None` Has Fix [​](https://ast-grep.github.io/catalog/python/#rewrite-optional-type-to-type-none) ------------------------------------------------------------------------------------------------------------------------------------- * [Playground Link](https://ast-grep.github.io/playground.html#eyJtb2RlIjoiQ29uZmlnIiwibGFuZyI6InB5dGhvbiIsInF1ZXJ5IjoiIiwicmV3cml0ZSI6IiIsInN0cmljdG5lc3MiOiJzaWduYXR1cmUiLCJzZWxlY3RvciI6IiIsImNvbmZpZyI6InJ1bGU6XG4gIHBhdHRlcm46IFxuICAgIGNvbnRleHQ6ICdhOiBPcHRpb25hbFskVF0nXG4gICAgc2VsZWN0b3I6IGdlbmVyaWNfdHlwZVxuZml4OiAkVCB8IE5vbmUiLCJzb3VyY2UiOiJkZWYgYShhcmc6IE9wdGlvbmFsW0ludF0pOiBwYXNzIn0=) ### Description [​](https://ast-grep.github.io/catalog/python/#description-5) [PEP 604](https://peps.python.org/pep-0604/) recommends that `Type | None` is preferred over `Optional[Type]` for Python 3.10+. This rule performs such rewriting. Note `Optional[$T]` alone is interpreted as subscripting expression instead of generic type, we need to use [pattern object](https://ast-grep.github.io/guide/rule-config/atomic-rule.html#pattern-object) to disambiguate it with more context code. ### YAML [​](https://ast-grep.github.io/catalog/python/#yaml-4) yaml id: optional-to-none-union language: python rule: pattern: context: 'a: Optional[$T]' selector: generic_type fix: $T | None ### Example [​](https://ast-grep.github.io/catalog/python/#example-5) py def a(arg: Optional[int]): pass ### Diff [​](https://ast-grep.github.io/catalog/python/#diff-6) py def a(arg: Optional[int]): pass def a(arg: int | None): pass ### Contributed by [​](https://ast-grep.github.io/catalog/python/#contributed-by-4) [Bede Carroll](https://github.com/ast-grep/ast-grep/discussions/1492) Recursive Rewrite Type Has Fix [​](https://ast-grep.github.io/catalog/python/#recursive-rewrite-type) ------------------------------------------------------------------------------------------------------ * [Playground Link](https://ast-grep.github.io/playground.html#eyJtb2RlIjoiQ29uZmlnIiwibGFuZyI6InB5dGhvbiIsInF1ZXJ5IjoiIiwicmV3cml0ZSI6IiIsInN0cmljdG5lc3MiOiJzbWFydCIsInNlbGVjdG9yIjoiIiwiY29uZmlnIjoicmV3cml0ZXJzOlxyXG4tIGlkOiBvcHRpb25hbFxyXG4gIGxhbmd1YWdlOiBQeXRob25cclxuICBydWxlOlxyXG4gICAgYW55OlxyXG4gICAgLSBwYXR0ZXJuOlxyXG4gICAgICAgIGNvbnRleHQ6ICdhcmc6IE9wdGlvbmFsWyRUWVBFXSdcclxuICAgICAgICBzZWxlY3RvcjogZ2VuZXJpY190eXBlXHJcbiAgICAtIHBhdHRlcm46IE9wdGlvbmFsWyRUWVBFXVxyXG4gIHRyYW5zZm9ybTpcclxuICAgIE5UOlxyXG4gICAgICByZXdyaXRlOiBcclxuICAgICAgICByZXdyaXRlcnM6IFtvcHRpb25hbCwgdW5pb25zXVxyXG4gICAgICAgIHNvdXJjZTogJFRZUEVcclxuICBmaXg6ICROVCB8IE5vbmVcclxuLSBpZDogdW5pb25zXHJcbiAgbGFuZ3VhZ2U6IFB5dGhvblxyXG4gIHJ1bGU6XHJcbiAgICBwYXR0ZXJuOlxyXG4gICAgICBjb250ZXh0OiAnYTogVW5pb25bJCQkVFlQRVNdJ1xyXG4gICAgICBzZWxlY3RvcjogZ2VuZXJpY190eXBlXHJcbiAgdHJhbnNmb3JtOlxyXG4gICAgVU5JT05TOlxyXG4gICAgICByZXdyaXRlOlxyXG4gICAgICAgIHJld3JpdGVyczpcclxuICAgICAgICAgIC0gcmV3cml0ZS11bmlvbnNcclxuICAgICAgICBzb3VyY2U6ICQkJFRZUEVTXHJcbiAgICAgICAgam9pbkJ5OiBcIiB8IFwiXHJcbiAgZml4OiAkVU5JT05TXHJcbi0gaWQ6IHJld3JpdGUtdW5pb25zXHJcbiAgcnVsZTpcclxuICAgIHBhdHRlcm46ICRUWVBFXHJcbiAgICBraW5kOiB0eXBlXHJcbiAgdHJhbnNmb3JtOlxyXG4gICAgTlQ6XHJcbiAgICAgIHJld3JpdGU6IFxyXG4gICAgICAgIHJld3JpdGVyczogW29wdGlvbmFsLCB1bmlvbnNdXHJcbiAgICAgICAgc291cmNlOiAkVFlQRVxyXG4gIGZpeDogJE5UXHJcbnJ1bGU6XHJcbiAga2luZDogdHlwZVxyXG4gIHBhdHRlcm46ICRUUEVcclxudHJhbnNmb3JtOlxyXG4gIE5FV19UWVBFOlxyXG4gICAgcmV3cml0ZTogXHJcbiAgICAgIHJld3JpdGVyczogW29wdGlvbmFsLCB1bmlvbnNdXHJcbiAgICAgIHNvdXJjZTogJFRQRVxyXG5maXg6ICRORVdfVFlQRSIsInNvdXJjZSI6InJlc3VsdHM6ICBPcHRpb25hbFtVbmlvbltMaXN0W1VuaW9uW3N0ciwgZGljdF1dLCBzdHJdXVxuIn0=) ### Description [​](https://ast-grep.github.io/catalog/python/#description-6) Suppose we want to transform Python's `Union[T1, T2]` to `T1 | T2` and `Optional[T]` to `T | None`. By default, ast-grep will only fix the outermost node that matches a pattern and will not rewrite the inner AST nodes inside a match. This avoids unexpected rewriting or infinite rewriting loop. So if you are using non-recursive rewriter like [this](https://github.com/ast-grep/ast-grep/discussions/1566#discussion-7401382) , `Optional[Union[int, str]]` will only be converted to `Union[int, str] | None`. Note the inner `Union[int, str]` is not enabled. This is because the rewriter `optional` matches `Optional[$TYPE]` and rewrite it to `$TYPE | None`. The inner `$TYPE` is not processed. However, we can apply `rewriters` to inner types recursively. Take the `optional` rewriter as an example, we need to apply rewriters, `optional` and `unions`, **recursively** to `$TYPE` and get a new variable `$NT`. ### YAML [​](https://ast-grep.github.io/catalog/python/#yaml-5) yml id: recursive-rewrite-types language: python rewriters: # rewrite Optional[T] to T | None - id: optional rule: any: - pattern: context: 'arg: Optional[$TYPE]' selector: generic_type - pattern: Optional[$TYPE] # recursively apply rewriters to $TYPE transform: NT: rewrite: rewriters: [optional, unions] source: $TYPE # use the new variable $NT fix: $NT | None # similar to Optional, rewrite Union[T1, T2] to T1 | T2 - id: unions language: Python rule: pattern: context: 'a: Union[$$$TYPES]' selector: generic_type transform: UNIONS: # rewrite all types inside $$$TYPES rewrite: rewriters: [ rewrite-unions ] source: $$$TYPES joinBy: " | " fix: $UNIONS - id: rewrite-unions rule: pattern: $TYPE kind: type # recursive part transform: NT: rewrite: rewriters: [optional, unions] source: $TYPE fix: $NT # find all types rule: kind: type pattern: $TPE # apply the recursive rewriters transform: NEW_TYPE: rewrite: rewriters: [optional, unions] source: $TPE # output fix: $NEW_TYPE ### Example [​](https://ast-grep.github.io/catalog/python/#example-6) python results: Optional[Union[List[Union[str, dict]], str]] ### Diff [​](https://ast-grep.github.io/catalog/python/#diff-7) python results: Optional[Union[List[Union[str, dict]], str]] results: List[str | dict] | str | None ### Contributed by [​](https://ast-grep.github.io/catalog/python/#contributed-by-5) Inspired by [steinuil](https://github.com/ast-grep/ast-grep/discussions/1566) Rewrite SQLAlchemy with Type Annotations Has Fix [​](https://ast-grep.github.io/catalog/python/#rewrite-sqlalchemy-with-type-annotations) ------------------------------------------------------------------------------------------------------------------------------------------ * [Playground Link](https://ast-grep.github.io/playground.html#eyJtb2RlIjoiQ29uZmlnIiwibGFuZyI6InB5dGhvbiIsInF1ZXJ5IjoiYShudWxsYWJsZT1UcnVlKSIsInJld3JpdGUiOiIxMjMiLCJzdHJpY3RuZXNzIjoic21hcnQiLCJzZWxlY3RvciI6ImtleXdvcmRfYXJndW1lbnQiLCJjb25maWciOiJpZDogcmVtb3ZlLW51bGxhYmxlLWFyZ1xubGFuZ3VhZ2U6IHB5dGhvblxucnVsZTpcbiAgcGF0dGVybjogJFggPSBtYXBwZWRfY29sdW1uKCQkJEFSR1MpXG4gIGFueTpcbiAgICAtIHBhdHRlcm46ICRYID0gbWFwcGVkX2NvbHVtbigkJCRCRUZPUkUsIFN0cmluZywgJCQkTUlELCBudWxsYWJsZT1UcnVlLCAkJCRBRlRFUilcbiAgICAtIHBhdHRlcm46ICRYID0gbWFwcGVkX2NvbHVtbigkJCRCRUZPUkUsIFN0cmluZywgJCQkTUlELCBudWxsYWJsZT1UcnVlKVxucmV3cml0ZXJzOlxuLSBpZDogZmlsdGVyLXN0cmluZy1udWxsYWJsZVxuICBydWxlOlxuICAgIHBhdHRlcm46ICRBUkdcbiAgICBpbnNpZGU6XG4gICAgICBraW5kOiBhcmd1bWVudF9saXN0XG4gICAgYWxsOlxuICAgIC0gbm90OiBcbiAgICAgICAgcGF0dGVybjogU3RyaW5nXG4gICAgLSBub3Q6XG4gICAgICAgIHBhdHRlcm46XG4gICAgICAgICAgY29udGV4dDogYShudWxsYWJsZT1UcnVlKVxuICAgICAgICAgIHNlbGVjdG9yOiBrZXl3b3JkX2FyZ3VtZW50XG4gIGZpeDogJEFSR1xuXG50cmFuc2Zvcm06XG4gIE5FV0FSR1M6XG4gICAgcmV3cml0ZTpcbiAgICAgIHJld3JpdGVyczogW2ZpbHRlci1zdHJpbmctbnVsbGFibGVdXG4gICAgICBzb3VyY2U6ICQkJEFSR1NcbiAgICAgIGpvaW5CeTogJywgJ1xuZml4OiB8LVxuICAkWDogTWFwcGVkW3N0ciB8IE5vbmVdID0gbWFwcGVkX2NvbHVtbigkTkVXQVJHUykiLCJzb3VyY2UiOiJtZXNzYWdlID0gbWFwcGVkX2NvbHVtbihTdHJpbmcsIGRlZmF1bHQ9XCJoZWxsb1wiLCBudWxsYWJsZT1UcnVlKVxuXG5tZXNzYWdlID0gbWFwcGVkX2NvbHVtbihTdHJpbmcsIG51bGxhYmxlPVRydWUpXG5cbl9tZXNzYWdlID0gbWFwcGVkX2NvbHVtbihcIm1lc3NhZ2VcIiwgU3RyaW5nLCBudWxsYWJsZT1UcnVlKVxuXG5tZXNzYWdlID0gbWFwcGVkX2NvbHVtbihTdHJpbmcsIG51bGxhYmxlPVRydWUsIHVuaXF1ZT1UcnVlKVxuXG5tZXNzYWdlID0gbWFwcGVkX2NvbHVtbihcbiAgU3RyaW5nLCBpbmRleD1UcnVlLCBudWxsYWJsZT1UcnVlLCB1bmlxdWU9VHJ1ZSlcblxuIyBTaG91bGQgbm90IGJlIHRyYW5zZm9ybWVkXG5tZXNzYWdlID0gbWFwcGVkX2NvbHVtbihTdHJpbmcsIGRlZmF1bHQ9XCJoZWxsb1wiKVxuXG5tZXNzYWdlID0gbWFwcGVkX2NvbHVtbihTdHJpbmcsIGRlZmF1bHQ9XCJoZWxsb1wiLCBudWxsYWJsZT1GYWxzZSlcblxubWVzc2FnZSA9IG1hcHBlZF9jb2x1bW4oSW50ZWdlciwgZGVmYXVsdD1cImhlbGxvXCIpIn0=) ### Description [​](https://ast-grep.github.io/catalog/python/#description-7) [SQLAlchemy 2.0](https://docs.sqlalchemy.org/en/20/orm/declarative_tables.html) recommends using type annotations with `Mapped` type for modern declarative mapping. The `mapped_column()` construct can derive its configuration from [PEP 484](https://peps.python.org/pep-0484/) type annotations. This rule helps migrate legacy SQLAlchemy code that explicitly uses `String` type and `nullable=True` to the modern type annotation approach using `Mapped[str | None]`. The key technique demonstrated here is using **rewriters** to selectively filter arguments. The rewriter: 1. Matches each argument inside the `argument_list` 2. Excludes the `String` type argument 3. Excludes the `nullable=True` keyword argument 4. Keeps all other arguments ### YAML [​](https://ast-grep.github.io/catalog/python/#yaml-6) yaml id: remove-nullable-arg language: python rule: pattern: $X = mapped_column($$$ARGS) any: - pattern: $X = mapped_column($$$BEFORE, String, $$$MID, nullable=True, $$$AFTER) - pattern: $X = mapped_column($$$BEFORE, String, $$$MID, nullable=True) rewriters: - id: filter-string-nullable rule: pattern: $ARG inside: kind: argument_list all: - not: pattern: String - not: pattern: context: a(nullable=True) selector: keyword_argument fix: $ARG transform: NEWARGS: rewrite: rewriters: [filter-string-nullable] source: $$$ARGS joinBy: ', ' fix: |- $X: Mapped[str | None] = mapped_column($NEWARGS) ### Example [​](https://ast-grep.github.io/catalog/python/#example-7) python message = mapped_column(String, default="hello", nullable=True) message = mapped_column(String, nullable=True) _message = mapped_column("message", String, nullable=True) message = mapped_column(String, nullable=True, unique=True) message = mapped_column( String, index=True, nullable=True, unique=True) # Should not be transformed message = mapped_column(String, default="hello") message = mapped_column(String, default="hello", nullable=False) message = mapped_column(Integer, default="hello") ### Diff [​](https://ast-grep.github.io/catalog/python/#diff-8) python message = mapped_column(String, default="hello", nullable=True) message: Mapped[str | None] = mapped_column(default="hello") message = mapped_column(String, nullable=True) message: Mapped[str | None] = mapped_column() _message = mapped_column("message", String, nullable=True) _message: Mapped[str | None] = mapped_column("message") message = mapped_column(String, nullable=True, unique=True) message: Mapped[str | None] = mapped_column(unique=True) message = mapped_column( String, index=True, nullable=True, unique=True) message: Mapped[str | None] = mapped_column( index=True, unique=True) ### Contributed by [​](https://ast-grep.github.io/catalog/python/#contributed-by-6) Inspired by [discussion #2319](https://github.com/ast-grep/ast-grep/discussions/2319) --- # ast-grep [Skip to content](https://ast-grep.github.io/catalog/tsx/rename-svg-attribute.html#VPContent) Return to top Are you an LLM? You can read better optimized documentation at /catalog/tsx/rename-svg-attribute.md for this page in Markdown format Rename SVG Attribute Has Fix [​](https://ast-grep.github.io/catalog/tsx/rename-svg-attribute.html#rename-svg-attribute) ------------------------------------------------------------------------------------------------------------------------ * [Playground Link](https://ast-grep.github.io/playground.html#eyJtb2RlIjoiQ29uZmlnIiwibGFuZyI6InRzeCIsInF1ZXJ5IjoiIiwicmV3cml0ZSI6IiIsInN0cmljdG5lc3MiOiJyZWxheGVkIiwic2VsZWN0b3IiOiIiLCJjb25maWciOiJpZDogcmV3cml0ZS1zdmctYXR0cmlidXRlXG5sYW5ndWFnZTogdHN4XG5ydWxlOlxuICBwYXR0ZXJuOiAkUFJPUFxuICByZWdleDogKFthLXpdKyktKFthLXpdKVxuICBraW5kOiBwcm9wZXJ0eV9pZGVudGlmaWVyXG4gIGluc2lkZTpcbiAgICBraW5kOiBqc3hfYXR0cmlidXRlXG50cmFuc2Zvcm06XG4gIE5FV19QUk9QOlxuICAgIGNvbnZlcnQ6XG4gICAgICBzb3VyY2U6ICRQUk9QXG4gICAgICB0b0Nhc2U6IGNhbWVsQ2FzZVxuZml4OiAkTkVXX1BST1AiLCJzb3VyY2UiOiJjb25zdCBlbGVtZW50ID0gKFxuICA8c3ZnIHdpZHRoPVwiMTAwXCIgaGVpZ2h0PVwiMTAwXCIgdmlld0JveD1cIjAgMCAxMDAgMTAwXCI+XG4gICAgPHBhdGggZD1cIk0xMCAyMCBMMzAgNDBcIiBzdHJva2UtbGluZWNhcD1cInJvdW5kXCIgZmlsbC1vcGFjaXR5PVwiMC41XCIgLz5cbiAgPC9zdmc+XG4pIn0=) ### Description [​](https://ast-grep.github.io/catalog/tsx/rename-svg-attribute.html#description) [SVG](https://en.wikipedia.org/wiki/SVG) (Scalable Vector Graphics)s' hyphenated names are not compatible with JSX syntax in React. JSX requires [camelCase naming](https://react.dev/learn/writing-markup-with-jsx#3-camelcase-salls-most-of-the-things) for attributes. For example, an SVG attribute like `stroke-linecap` needs to be renamed to `strokeLinecap` to work correctly in React. ### YAML [​](https://ast-grep.github.io/catalog/tsx/rename-svg-attribute.html#yaml) yaml id: rewrite-svg-attribute language: tsx rule: pattern: $PROP # capture in metavar regex: ([a-z]+)-([a-z]) # hyphenated name kind: property_identifier inside: kind: jsx_attribute # in JSX attribute transform: NEW_PROP: # new property name convert: # use ast-grep's convert source: $PROP toCase: camelCase # to camelCase naming fix: $NEW_PROP ### Example [​](https://ast-grep.github.io/catalog/tsx/rename-svg-attribute.html#example) tsx const element = ( ) ### Diff [​](https://ast-grep.github.io/catalog/tsx/rename-svg-attribute.html#diff) ts const element = ( ) ### Contributed by [​](https://ast-grep.github.io/catalog/tsx/rename-svg-attribute.html#contributed-by) Inspired by [SVG Renamer](https://admondtamang.medium.com/introducing-svg-renamer-your-solution-for-react-svg-attributes-26503382d5a8) --- # ast-grep [Skip to content](https://ast-grep.github.io/catalog/tsx/rewrite-mobx-component.html#VPContent) Return to top Are you an LLM? You can read better optimized documentation at /catalog/tsx/rewrite-mobx-component.md for this page in Markdown format Rewrite MobX Component Style Has Fix [​](https://ast-grep.github.io/catalog/tsx/rewrite-mobx-component.html#rewrite-mobx-component-style) ------------------------------------------------------------------------------------------------------------------------------------------ * [Playground Link](https://ast-grep.github.io/playground.html#eyJtb2RlIjoiQ29uZmlnIiwibGFuZyI6ImphdmFzY3JpcHQiLCJxdWVyeSI6ImNvbnNvbGUubG9nKCRNQVRDSCkiLCJyZXdyaXRlIjoibG9nZ2VyLmxvZygkTUFUQ0gpIiwiY29uZmlnIjoicnVsZTpcbiAgcGF0dGVybjogZXhwb3J0IGNvbnN0ICRDT01QID0gb2JzZXJ2ZXIoJEZVTkMpXG5maXg6IHwtXG4gIGNvbnN0IEJhc2UkQ09NUCA9ICRGVU5DXG4gIGV4cG9ydCBjb25zdCAkQ09NUCA9IG9ic2VydmVyKEJhc2UkQ09NUCkiLCJzb3VyY2UiOiJleHBvcnQgY29uc3QgRXhhbXBsZSA9IG9ic2VydmVyKCgpID0+IHtcbiAgcmV0dXJuIDxkaXY+SGVsbG8gV29ybGQ8L2Rpdj5cbn0pIn0=) ### Description [​](https://ast-grep.github.io/catalog/tsx/rewrite-mobx-component.html#description) React and MobX are libraries that help us build user interfaces with JavaScript. [React hooks](https://react.dev/reference/react) allow us to use state and lifecycle methods in functional components. But we need follow some hook rules, or React may break. [MobX](https://mobx.js.org/react-integration.html) has an `observer` function that makes a component update when data changes. When we use the `observer` function like this: JavaScript export const Example = observer(() => {…}) ESLint, the tool that checks hooks, thinks that `Example` is not a React component, but just a regular function. So it does not check the hooks inside it, and we may miss some wrong usages. To fix this, we need to change our component style to this: JavaScript const BaseExample = () => {…} const Example = observer(BaseExample) Now ESLint can see that `BaseExample` is a React component, and it can check the hooks inside it. ### YAML [​](https://ast-grep.github.io/catalog/tsx/rewrite-mobx-component.html#yaml) yaml id: rewrite-mobx-component language: typescript rule: pattern: export const $COMP = observer($FUNC) fix: |- const Base$COMP = $FUNC export const $COMP = observer(Base$COMP) ### Example [​](https://ast-grep.github.io/catalog/tsx/rewrite-mobx-component.html#example) js export const Example = observer(() => { return
Hello World
}) ### Diff [​](https://ast-grep.github.io/catalog/tsx/rewrite-mobx-component.html#diff) js export const Example = observer(() => { return
Hello World
}) const BaseExample = () => { return
Hello World
} export const Example = observer(BaseExample) ### Contributed by [​](https://ast-grep.github.io/catalog/tsx/rewrite-mobx-component.html#contributed-by) [Bryan Lee](https://twitter.com/meetliby/status/1698601672568901723) --- # Rust | ast-grep [Skip to content](https://ast-grep.github.io/catalog/rust/#VPContent) Return to top Are you an LLM? You can read better optimized documentation at /catalog/rust.md for this page in Markdown format Rust [​](https://ast-grep.github.io/catalog/rust/#rust) ======================================================== This page curates a list of example ast-grep rules to check and to rewrite Rust applications. Avoid Duplicated Exports [​](https://ast-grep.github.io/catalog/rust/#avoid-duplicated-exports) ------------------------------------------------------------------------------------------------ * [Playground Link](https://ast-grep.github.io/playground.html#eyJtb2RlIjoiQ29uZmlnIiwibGFuZyI6InJ1c3QiLCJxdWVyeSI6IiIsImNvbmZpZyI6InJ1bGU6XG4gIGFsbDpcbiAgICAgLSBwYXR0ZXJuOiBwdWIgdXNlICRCOjokQztcbiAgICAgLSBpbnNpZGU6XG4gICAgICAgIGtpbmQ6IHNvdXJjZV9maWxlXG4gICAgICAgIGhhczpcbiAgICAgICAgICBwYXR0ZXJuOiBwdWIgbW9kICRBO1xuICAgICAtIGhhczpcbiAgICAgICAgcGF0dGVybjogJEFcbiAgICAgICAgc3RvcEJ5OiBlbmQiLCJzb3VyY2UiOiJwdWIgbW9kIGZvbztcbnB1YiB1c2UgZm9vOjpGb287XG5wdWIgdXNlIGZvbzo6QTo6QjtcblxuXG5wdWIgdXNlIGFhYTo6QTtcbnB1YiB1c2Ugd29vOjpXb287In0=) ### Description [​](https://ast-grep.github.io/catalog/rust/#description) Generally, we don't encourage the use of re-exports. However, sometimes, to keep the interface exposed by a lib crate tidy, we use re-exports to shorten the path to specific items. When doing so, a pitfall is to export a single item under two different names. Consider: rs pub mod foo; pub use foo::Foo; The issue with this code, is that `Foo` is now exposed under two different paths: `Foo`, `foo::Foo`. This unnecessarily increases the surface of your API. It can also cause issues on the client side. For example, it makes the usage of auto-complete in the IDE more involved. Instead, ensure you export only once with `pub`. ### YAML [​](https://ast-grep.github.io/catalog/rust/#yaml) yaml id: avoid-duplicate-export language: rust rule: all: - pattern: pub use $B::$C; - inside: kind: source_file has: pattern: pub mod $A; - has: pattern: $A stopBy: end ### Example [​](https://ast-grep.github.io/catalog/rust/#example) rs pub mod foo; pub use foo::Foo; pub use foo::A::B; pub use aaa::A; pub use woo::Woo; ### Contributed by [​](https://ast-grep.github.io/catalog/rust/#contributed-by) Julius Lungys([voidpumpkin](https://github.com/voidpumpkin) ) Beware of char offset when iterate over a string Has Fix [​](https://ast-grep.github.io/catalog/rust/#beware-of-char-offset-when-iterate-over-a-string) -------------------------------------------------------------------------------------------------------------------------------------------------------- * [Playground Link](https://ast-grep.github.io/playground.html#eyJtb2RlIjoiUGF0Y2giLCJsYW5nIjoicnVzdCIsInF1ZXJ5IjoiJEEuY2hhcnMoKS5lbnVtZXJhdGUoKSIsInJld3JpdGUiOiIkQS5jaGFyX2luZGljZXMoKSIsImNvbmZpZyI6IiIsInNvdXJjZSI6ImZvciAoaSwgY2hhcikgaW4gc291cmNlLmNoYXJzKCkuZW51bWVyYXRlKCkge1xuICAgIHByaW50bG4hKFwiQm9zaGVuIGlzIGFuZ3J5IDopXCIpO1xufSJ9) ### Description [​](https://ast-grep.github.io/catalog/rust/#description-1) It's a common pitfall in Rust that counting _character offset_ is not the same as counting _byte offset_ when iterating through a string. Rust string is represented by utf-8 byte array, which is a variable-length encoding scheme. `chars().enumerate()` will yield the character offset, while [`char_indices()`](https://doc.rust-lang.org/std/primitive.str.html#method.char_indices) will yield the byte offset. rs let yes = "y̆es"; let mut char_indices = yes.char_indices(); assert_eq!(Some((0, 'y')), char_indices.next()); // not (0, 'y̆') assert_eq!(Some((1, '\u{0306}')), char_indices.next()); // note the 3 here - the last character took up two bytes assert_eq!(Some((3, 'e')), char_indices.next()); assert_eq!(Some((4, 's')), char_indices.next()); Depending on your use case, you may want to use `char_indices()` instead of `chars().enumerate()`. ### Pattern [​](https://ast-grep.github.io/catalog/rust/#pattern) shell ast-grep -p '$A.chars().enumerate()' \ -r '$A.char_indices()' \ -l rs ### Example [​](https://ast-grep.github.io/catalog/rust/#example-1) rs for (i, char) in source.chars().enumerate() { println!("Boshen is angry :)"); } ### Diff [​](https://ast-grep.github.io/catalog/rust/#diff) rs for (i, char) in source.chars().enumerate() { for (i, char) in source.char_indices() { println!("Boshen is angry :)"); } ### Contributed by [​](https://ast-grep.github.io/catalog/rust/#contributed-by-1) Inspired by [Boshen's Tweet](https://x.com/boshen_c/status/1719033308682870891) ![Boshen's footgun](https://pbs.twimg.com/media/F9s7mJHaYAEndnY?format=jpg&name=medium) Get number of digits in a `usize` Has Fix [​](https://ast-grep.github.io/catalog/rust/#get-number-of-digits-in-a-usize) ------------------------------------------------------------------------------------------------------------------------ * [Playground Link](https://ast-grep.github.io/playground.html#eyJtb2RlIjoiUGF0Y2giLCJsYW5nIjoicnVzdCIsInF1ZXJ5IjoiJE5VTS50b19zdHJpbmcoKS5jaGFycygpLmNvdW50KCkiLCJyZXdyaXRlIjoiJE5VTS5jaGVja2VkX2lsb2cxMCgpLnVud3JhcF9vcigwKSArIDEiLCJjb25maWciOiIjIFlBTUwgUnVsZSBpcyBtb3JlIHBvd2VyZnVsIVxuIyBodHRwczovL2FzdC1ncmVwLmdpdGh1Yi5pby9ndWlkZS9ydWxlLWNvbmZpZy5odG1sI3J1bGVcbnJ1bGU6XG4gIGFueTpcbiAgICAtIHBhdHRlcm46IGNvbnNvbGUubG9nKCRBKVxuICAgIC0gcGF0dGVybjogY29uc29sZS5kZWJ1ZygkQSlcbmZpeDpcbiAgbG9nZ2VyLmxvZygkQSkiLCJzb3VyY2UiOiJsZXQgd2lkdGggPSAobGluZXMgKyBudW0pLnRvX3N0cmluZygpLmNoYXJzKCkuY291bnQoKTsifQ==) ### Description [​](https://ast-grep.github.io/catalog/rust/#description-2) Getting the number of digits in a usize number can be useful for various purposes, such as counting the column width of line numbers in a text editor or formatting the output of a number with commas or spaces. A common but inefficient way of getting the number of digits in a `usize` number is to use `num.to_string().chars().count()`. This method converts the number to a string, iterates over its characters, and counts them. However, this method involves allocating a new string, which can be costly in terms of memory and time. A better alternative is to use [`checked_ilog10`](https://doc.rust-lang.org/std/primitive.usize.html#method.checked_ilog10) . rs num.checked_ilog10().unwrap_or(0) + 1 The snippet above computes the integer logarithm base 10 of the number and adds one. This snippet does not allocate any memory and is faster than the string conversion approach. The [efficient](https://doc.rust-lang.org/src/core/num/int_log10.rs.html) `checked_ilog10` function returns an `Option` that is `Some(log)` if the number is positive and `None` if the number is zero. The `unwrap_or(0)` function returns the value inside the option or `0` if the option is `None`. ### Pattern [​](https://ast-grep.github.io/catalog/rust/#pattern-1) shell ast-grep -p '$NUM.to_string().chars().count()' \ -r '$NUM.checked_ilog10().unwrap_or(0) + 1' \ -l rs ### Example [​](https://ast-grep.github.io/catalog/rust/#example-2) rs let width = (lines + num).to_string().chars().count(); ### Diff [​](https://ast-grep.github.io/catalog/rust/#diff-1) rs let width = (lines + num).to_string().chars().count(); let width = (lines + num).checked_ilog10().unwrap_or(0) + 1; ### Contributed by [​](https://ast-grep.github.io/catalog/rust/#contributed-by-2) [Herrington Darkholme](https://twitter.com/hd_nvim) , inspired by [dogfooding ast-grep](https://github.com/ast-grep/ast-grep/issues/550) Unsafe Function Without Unsafe Block [​](https://ast-grep.github.io/catalog/rust/#unsafe-function-without-unsafe-block) ------------------------------------------------------------------------------------------------------------------------ * [Playground Link](https://ast-grep.github.io/playground.html#eyJtb2RlIjoiQ29uZmlnIiwibGFuZyI6InJ1c3QiLCJxdWVyeSI6IntcbiAgZGVzY3JpcHRpb24gPSAkQVxufSIsInJld3JpdGUiOiIiLCJzdHJpY3RuZXNzIjoic21hcnQiLCJzZWxlY3RvciI6ImJpbmRpbmciLCJjb25maWciOiIgIGlkOiByZWR1bmRhbnQtdW5zYWZlLWZ1bmN0aW9uXG4gIGxhbmd1YWdlOiBydXN0XG4gIHNldmVyaXR5OiBlcnJvclxuICBtZXNzYWdlOiBVbnNhZmUgZnVuY3Rpb24gd2l0aG91dCB1bnNhZmUgYmxvY2sgaW5zaWRlXG4gIG5vdGU6IHxcbiAgICBDb25zaWRlciB3aGV0aGVyIHRoaXMgZnVuY3Rpb24gbmVlZHMgdG8gYmUgbWFya2VkIHVuc2FmZSBcbiAgICBvciBpZiB1bnNhZmUgb3BlcmF0aW9ucyBzaG91bGQgYmUgd3JhcHBlZCBpbiBhbiB1bnNhZmUgYmxvY2tcbiAgcnVsZTpcbiAgICBhbGw6XG4gICAgICAtIGtpbmQ6IGZ1bmN0aW9uX2l0ZW1cbiAgICAgIC0gaGFzOlxuICAgICAgICAgIGtpbmQ6IGZ1bmN0aW9uX21vZGlmaWVyc1xuICAgICAgICAgIHJlZ2V4OiBcIl51bnNhZmVcIlxuICAgICAgLSBub3Q6XG4gICAgICAgICAgaGFzOlxuICAgICAgICAgICAga2luZDogdW5zYWZlX2Jsb2NrXG4gICAgICAgICAgICBzdG9wQnk6IGVuZCIsInNvdXJjZSI6IiAgLy8gU2hvdWxkIG1hdGNoIC0gdW5zYWZlIGZ1bmN0aW9uIHdpdGhvdXQgdW5zYWZlIGJsb2NrIChubyByZXR1cm4gdHlwZSlcbiAgdW5zYWZlIGZuIHJlZHVuZGFudF91bnNhZmUoKSB7XG4gICAgICBwcmludGxuIShcIk5vIHVuc2FmZSBvcGVyYXRpb25zIGhlcmVcIik7XG4gIH1cblxuICAvLyBTaG91bGQgbWF0Y2ggLSB1bnNhZmUgZnVuY3Rpb24gd2l0aCByZXR1cm4gdHlwZSwgbm8gdW5zYWZlIGJsb2NrXG4gIHVuc2FmZSBmbiByZWR1bmRhbnRfd2l0aF9yZXR1cm4oKSAtPiBpMzIge1xuICAgICAgbGV0IHggPSA1O1xuICAgICAgeCArIDEwXG4gIH1cblxuICAvLyBTaG91bGQgbWF0Y2ggLSB1bnNhZmUgZnVuY3Rpb24gd2l0aCBjb21wbGV4IHJldHVybiB0eXBlXG4gIHVuc2FmZSBmbiByZWR1bmRhbnRfY29tcGxleF9yZXR1cm4oKSAtPiBSZXN1bHQ8U3RyaW5nLCBzdGQ6OmlvOjpFcnJvcj4ge1xuICAgICAgT2soU3RyaW5nOjpmcm9tKFwic2FmZSBvcGVyYXRpb25cIikpXG4gIH1cblxuICAvLyBTaG91bGQgTk9UIG1hdGNoIC0gdW5zYWZlIGZ1bmN0aW9uIHdpdGggdW5zYWZlIGJsb2NrXG4gIHVuc2FmZSBmbiBwcm9wZXJfdW5zYWZlKCkgLT4gKmNvbnN0IGkzMiB7XG4gICAgICB1bnNhZmUge1xuICAgICAgICAgIGxldCBwdHIgPSAweDEyMzQgYXMgKmNvbnN0IGkzMjtcbiAgICAgICAgICBwdHJcbiAgICAgIH1cbiAgfVxuXG4gIC8vIFNob3VsZCBtYXRjaCAtIHVuc2FmZSBhc3luYyBmdW5jdGlvbiB3aXRob3V0IHVuc2FmZSBibG9ja1xuICB1bnNhZmUgYXN5bmMgZm4gYXN5bmNfcmVkdW5kYW50KCkgLT4gaTMyIHtcbiAgICAgIDQyXG4gIH1cblxuICAvLyBTaG91bGQgbWF0Y2ggLSB1bnNhZmUgY29uc3QgZnVuY3Rpb25cbiAgdW5zYWZlIGNvbnN0IGZuIGNvbnN0X3JlZHVuZGFudCgpIC0+IGkzMiB7XG4gICAgICAxMDBcbiAgfVxuXG4gIC8vIFNob3VsZCBOT1QgbWF0Y2ggLSByZWd1bGFyIGZ1bmN0aW9uXG4gIGZuIHJlZ3VsYXJfZnVuY3Rpb24oKSAtPiBpMzIge1xuICAgICAgNDJcbiAgfSJ9) ### Description [​](https://ast-grep.github.io/catalog/rust/#description-3) This rule detects functions marked with the `unsafe` keyword that do not contain any `unsafe` blocks in their body. When a function is marked `unsafe`, it indicates that the function contains operations that the compiler cannot verify as safe. However, if the function body doesn't contain any `unsafe` blocks, it may be unnecessarily marked as `unsafe`. This could be a sign that: 1. The function should not be marked `unsafe` if it doesn't perform any unsafe operations 2. Or if there are unsafe operations, they should be explicitly wrapped in `unsafe` blocks for clarity This rule helps identify such cases so developers can review whether the `unsafe` marker is truly necessary or if the code needs to be refactored. ### YAML [​](https://ast-grep.github.io/catalog/rust/#yaml-1) yaml id: redundant-unsafe-function language: rust severity: error message: Unsafe function without unsafe block inside note: | Consider whether this function needs to be marked unsafe or if unsafe operations should be wrapped in an unsafe block rule: all: - kind: function_item - has: kind: function_modifiers regex: "^unsafe" - not: has: kind: unsafe_block stopBy: end ### Example [​](https://ast-grep.github.io/catalog/rust/#example-3) rs // Should match - unsafe function without unsafe block (no return type) unsafe fn redundant_unsafe() { println!("No unsafe operations here"); } // Should match - unsafe function with return type, no unsafe block unsafe fn redundant_with_return() -> i32 { let x = 5; x + 10 } // Should match - unsafe function with complex return type unsafe fn redundant_complex_return() -> Result { Ok(String::from("safe operation")) } // Should NOT match - unsafe function with unsafe block unsafe fn proper_unsafe() -> *const i32 { unsafe { let ptr = 0x1234 as *const i32; ptr } } // Should match - unsafe async function without unsafe block unsafe async fn async_redundant() -> i32 { 42 } // Should match - unsafe const function unsafe const fn const_redundant() -> i32 { 100 } // Should NOT match - regular function fn regular_function() -> i32 { 42 } ### Contributed by [​](https://ast-grep.github.io/catalog/rust/#contributed-by-3) Inspired by [@hd\_nvim's Tweet](https://x.com/hd_nvim/status/1992810384072585397?s=20) Rewrite `indoc!` macro Has Fix [​](https://ast-grep.github.io/catalog/rust/#rewrite-indoc-macro) ------------------------------------------------------------------------------------------------- * [Playground Link](https://ast-grep.github.io/playground.html#eyJtb2RlIjoiUGF0Y2giLCJsYW5nIjoicnVzdCIsInF1ZXJ5IjoiaW5kb2MhIHsgciNcIiQkJEFcIiMgfSIsInJld3JpdGUiOiJgJCQkQWAiLCJzdHJpY3RuZXNzIjoicmVsYXhlZCIsInNlbGVjdG9yIjoiIiwiY29uZmlnIjoicnVsZTogXG4gYW55OlxuIC0gcGF0dGVybjogJFYgPT09ICRTRU5TRVRJVkVXT1JEXG4gLSBwYXR0ZXJuOiAkU0VOU0VUSVZFV09SRCA9PT0gJFZcbmNvbnN0cmFpbnRzOlxuICBTRU5TRVRJVkVXT1JEOlxuICAgIHJlZ2V4OiBwYXNzd29yZCIsInNvdXJjZSI6ImZuIG1haW4oKSB7XG4gICAgaW5kb2MhIHtyI1wiXG4gICAgICAgIC5mb28ge1xuICAgICAgICAgICAgb3JkZXI6IDE7XG4gICAgICAgIH1cbiAgICBcIiN9O1xufSJ9) ### Description [​](https://ast-grep.github.io/catalog/rust/#description-4) This example, created from [a Tweet](https://x.com/zack_overflow/status/1885065128590401551) , shows a refactoring operation being performed on Rust source code. The changes involve removing `indoc!` macro declarations while preserving the CSS-like content within them. Previously, the same refactor is implemented by a _unreadable monster regex_ in vim syntax. Click to see the original regex (neovim, btw) vimscript :%s/\v(indoc!|)(| )([|\{)r#"(([^#]+|\n+)+)"#/`\4`\ \ I have to confess that I don't understand this regex even if I use neovim, btw.\ \ Let Claude break it down piece by piece:\ \ * `:%s/` - Vim substitution command for all lines\ * `\v` - Very magic mode in vim for simpler regex syntax\ * `(indoc!|)` - First capture group: matches either "indoc!" or nothing\ * `(| )` - Second capture group: matches either empty string or a space\ * `([|\{)` - Third capture group: matches either `[` or `{`\ * `r#"` - Matches literal `r#"` (Rust raw string delimiter)\ * `(([^#]+|\n+)+)` - Fourth capture group (nested):\ * `[^#]+` - One or more non-# characters\ * `|\n+` - OR one or more newlines\ * Outer `()+` makes this repeat one or more times\ * `"#` - Matches the closing raw string delimiter\ * \`\\4\` - Replaces with the fourth capture group wrapped in backticks\ \ This regex is designed to find Rust raw string literals (possibly wrapped in `indoc!` macro), capture their content, and replace the entire match with just the content wrapped in backticks. It's more precise than my previous explanation and matches the pattern you're showing.\ \ ### Pattern [​](https://ast-grep.github.io/catalog/rust/#pattern-2)\ \ shell\ \ ast-grep --pattern 'indoc! { r#"$$$A"# }' --rewrite '`$$$A`' sgtest.rs\ \ ### Example [​](https://ast-grep.github.io/catalog/rust/#example-4)\ \ rs\ \ fn main() {\ indoc! {r#"\ .foo {\ order: 1;\ }\ "#};\ }\ \ ### Diff [​](https://ast-grep.github.io/catalog/rust/#diff-2)\ \ rs\ \ fn main() {\ indoc! {r#" // [!code --]\ `.foo { // [!code ++]\ order: 1;\ }\ "#}; \ `; \ }\ \ ### Contributed by [​](https://ast-grep.github.io/catalog/rust/#contributed-by-4)\ \ [Zack in SF](https://x.com/zack_overflow) --- # ast-grep [Skip to content](https://ast-grep.github.io/catalog/tsx/reverse-react-compiler.html#VPContent) Return to top Are you an LLM? You can read better optimized documentation at /catalog/tsx/reverse-react-compiler.md for this page in Markdown format Reverse React Compiler™ Has Fix [​](https://ast-grep.github.io/catalog/tsx/reverse-react-compiler.html#reverse-react-compilertm) --------------------------------------------------------------------------------------------------------------------------------- * [Playground Link](https://ast-grep.github.io/playground.html#eyJtb2RlIjoiQ29uZmlnIiwibGFuZyI6InRzeCIsInF1ZXJ5IjoiIiwicmV3cml0ZSI6IiIsInN0cmljdG5lc3MiOiJyZWxheGVkIiwic2VsZWN0b3IiOiIiLCJjb25maWciOiJpZDogcmV3cml0ZS1jYWNoZSBcbmxhbmd1YWdlOiB0c3hcbnJ1bGU6XG4gIGFueTpcbiAgLSBwYXR0ZXJuOiB1c2VDYWxsYmFjaygkRk4sICQkJClcbiAgLSBwYXR0ZXJuOiBtZW1vKCRGTiwgJCQkKVxuZml4OiAkRk5cblxuLS0tXG5cbmlkOiByZXdyaXRlLXVzZS1tZW1vXG5sYW5ndWFnZTogdHN4XG5ydWxlOiB7IHBhdHRlcm46ICd1c2VNZW1vKCRGTiwgJCQkKScgfVxuZml4OiAoJEZOKSgpIiwic291cmNlIjoiY29uc3QgQ29tcG9uZW50ID0gKCkgPT4ge1xuICBjb25zdCBbY291bnQsIHNldENvdW50XSA9IHVzZVN0YXRlKDApXG4gIGNvbnN0IGluY3JlbWVudCA9IHVzZUNhbGxiYWNrKCgpID0+IHtcbiAgICBzZXRDb3VudCgocHJldkNvdW50KSA9PiBwcmV2Q291bnQgKyAxKVxuICB9LCBbXSlcbiAgY29uc3QgZXhwZW5zaXZlQ2FsY3VsYXRpb24gPSB1c2VNZW1vKCgpID0+IHtcbiAgICAvLyBtb2NrIEV4cGVuc2l2ZSBjYWxjdWxhdGlvblxuICAgIHJldHVybiBjb3VudCAqIDJcbiAgfSwgW2NvdW50XSlcblxuICByZXR1cm4gKFxuICAgIDw+XG4gICAgICA8cD5FeHBlbnNpdmUgUmVzdWx0OiB7ZXhwZW5zaXZlQ2FsY3VsYXRpb259PC9wPlxuICAgICAgPGJ1dHRvbiBvbkNsaWNrPXtpbmNyZW1lbnR9Pntjb3VudH08L2J1dHRvbj5cbiAgICA8Lz5cbiAgKVxufSJ9) ### Description [​](https://ast-grep.github.io/catalog/tsx/reverse-react-compiler.html#description) React Compiler is a build-time only tool that automatically optimizes your React app, working with plain JavaScript and understanding the Rules of React without requiring a rewrite. It optimizes apps by automatically memoizing code, similar to `useMemo`, `useCallback`, and `React.memo`, reducing unnecessary recomputation due to incorrect or forgotten memoization. Reverse React Compiler™ is a [parody tweet](https://x.com/aidenybai/status/1881397529369034997) that works in the opposite direction. It takes React code and removes memoization, guaranteed to make your code slower. ([not](https://x.com/kentcdodds/status/1881404373646880997) [necessarily](https://dev.to/prathamisonline/are-you-over-using-usememo-and-usecallback-hooks-in-react-5lp) ) It is originally written in Babel and this is an [ast-grep version](https://x.com/hd_nvim/status/1881402678493970620) of it. The Original Babel Implementation For comparison purposes only. Note the original code [does not correctly rewrite](https://x.com/hd_nvim/status/1881404893136896415) `useMemo`. js const ReverseReactCompiler = ({ types: t }) => ({ visitor: { CallExpression(path) { const callee = path.node.callee; if ( t.isIdentifier(callee, { name: "useMemo" }) || t.isIdentifier(callee, { name: "useCallback" }) || t.isIdentifier(callee, { name: "memo" }) ) { path.replaceWith(args[0]); } }, }, }); ### YAML [​](https://ast-grep.github.io/catalog/tsx/reverse-react-compiler.html#yaml) yaml id: rewrite-cache language: tsx rule: any: - pattern: useCallback($FN, $$$) - pattern: memo($FN, $$$) fix: $FN --- id: rewrite-use-memo language: tsx rule: { pattern: 'useMemo($FN, $$$)' } fix: ($FN)() # need IIFE to wrap memo function ### Example [​](https://ast-grep.github.io/catalog/tsx/reverse-react-compiler.html#example) tsx const Component = () => { const [count, setCount] = useState(0) const increment = useCallback(() => { setCount((prevCount) => prevCount + 1) }, []) const expensiveCalculation = useMemo(() => { // mock Expensive calculation return count * 2 }, [count]) return ( <>

Expensive Result: {expensiveCalculation}

) } ### Diff [​](https://ast-grep.github.io/catalog/tsx/reverse-react-compiler.html#diff) tsx const Component = () => { const [count, setCount] = useState(0) const increment = useCallback(() => { setCount((prevCount) => prevCount + 1) }, []) const increment = () => { setCount((prevCount) => prevCount + 1) } const expensiveCalculation = useMemo(() => { // mock Expensive calculation return count * 2 }, [count]) const expensiveCalculation = (() => { // mock Expensive calculation return count * 2 })() return ( <>

Expensive Result: {expensiveCalculation}

) } ### Contributed by [​](https://ast-grep.github.io/catalog/tsx/reverse-react-compiler.html#contributed-by) Inspired by [Aiden Bai](https://twitter.com/aidenybai) --- # ast-grep [Skip to content](https://ast-grep.github.io/catalog/tsx/unnecessary-react-hook.html#VPContent) Return to top Are you an LLM? You can read better optimized documentation at /catalog/tsx/unnecessary-react-hook.md for this page in Markdown format Avoid Unnecessary React Hook [​](https://ast-grep.github.io/catalog/tsx/unnecessary-react-hook.html#avoid-unnecessary-react-hook) ---------------------------------------------------------------------------------------------------------------------------------- * [Playground Link](https://ast-grep.github.io/playground.html#eyJtb2RlIjoiQ29uZmlnIiwibGFuZyI6ImphdmFzY3JpcHQiLCJxdWVyeSI6IiIsInJld3JpdGUiOiIiLCJzdHJpY3RuZXNzIjoic21hcnQiLCJzZWxlY3RvciI6IiIsImNvbmZpZyI6InV0aWxzOlxuICBob29rX2NhbGw6XG4gICAgaGFzOlxuICAgICAga2luZDogY2FsbF9leHByZXNzaW9uXG4gICAgICByZWdleDogXnVzZVxuICAgICAgc3RvcEJ5OiBlbmRcbnJ1bGU6XG4gIGFueTpcbiAgLSBwYXR0ZXJuOiBmdW5jdGlvbiAkRlVOQygkJCQpIHsgJCQkIH1cbiAgLSBwYXR0ZXJuOiBsZXQgJEZVTkMgPSAoJCQkKSA9PiAkJCQgXG4gIC0gcGF0dGVybjogY29uc3QgJEZVTkMgPSAoJCQkKSA9PiAkJCRcbiAgaGFzOlxuICAgIHBhdHRlcm46ICRCT0RZXG4gICAga2luZDogc3RhdGVtZW50X2Jsb2NrXG4gICAgc3RvcEJ5OiBlbmQgXG5jb25zdHJhaW50czpcbiAgRlVOQzoge3JlZ2V4OiBedXNlIH1cbiAgQk9EWTogeyBub3Q6IHsgbWF0Y2hlczogaG9va19jYWxsIH0gfSBcbiIsInNvdXJjZSI6ImZ1bmN0aW9uIHVzZUlBbU5vdEhvb2tBY3R1YWxseShhcmdzKSB7XG4gICAgY29uc29sZS5sb2coJ0NhbGxlZCBpbiBSZWFjdCBidXQgSSBkb250IG5lZWQgdG8gYmUgYSBob29rJylcbiAgICByZXR1cm4gYXJncy5sZW5ndGhcbn1cbmNvbnN0IHVzZUlBbU5vdEhvb2tUb28gPSAoLi4uYXJncykgPT4ge1xuICAgIGNvbnNvbGUubG9nKCdDYWxsZWQgaW4gUmVhY3QgYnV0IEkgZG9udCBuZWVkIHRvIGJlIGEgaG9vaycpXG4gICAgcmV0dXJuIGFyZ3MubGVuZ3RoXG59XG5cbmZ1bmN0aW9uIHVzZUhvb2soKSB7XG4gICAgdXNlRWZmZWN0KCgpID0+IHtcbiAgICAgIGNvbnNvbGUubG9nKCdSZWFsIGhvb2snKSAgIFxuICAgIH0pXG59In0=) ### Description [​](https://ast-grep.github.io/catalog/tsx/unnecessary-react-hook.html#description) React hook is a powerful feature in React that allows you to use state and other React features in a functional component. However, you should avoid using hooks when you don't need them. If the code does not contain using any other React hooks, it can be rewritten to a plain function. This can help to separate your application logic from the React-specific UI logic. ### YAML [​](https://ast-grep.github.io/catalog/tsx/unnecessary-react-hook.html#yaml) yaml id: unnecessary-react-hook language: Tsx utils: hook_call: has: kind: call_expression regex: ^use stopBy: end rule: any: - pattern: function $FUNC($$$) { $$$ } - pattern: let $FUNC = ($$$) => $$$ - pattern: const $FUNC = ($$$) => $$$ has: pattern: $BODY kind: statement_block stopBy: end constraints: FUNC: {regex: ^use } BODY: { not: { matches: hook_call } } ### Example [​](https://ast-grep.github.io/catalog/tsx/unnecessary-react-hook.html#example) tsx function useIAmNotHookActually(args) { console.log('Called in React but I dont need to be a hook') return args.length } const useIAmNotHookToo = (...args) => { console.log('Called in React but I dont need to be a hook') return args.length } function useTrueHook() { useEffect(() => { console.log('Real hook') }) } ### Contributed by [​](https://ast-grep.github.io/catalog/tsx/unnecessary-react-hook.html#contributed-by) [Herrington Darkholme](https://twitter.com/hd_nvim) --- # ast-grep [Skip to content](https://ast-grep.github.io/catalog/typescript/find-import-usage.html#VPContent) Return to top Are you an LLM? You can read better optimized documentation at /catalog/typescript/find-import-usage.md for this page in Markdown format Find Import Usage [​](https://ast-grep.github.io/catalog/typescript/find-import-usage.html#find-import-usage) -------------------------------------------------------------------------------------------------------------- * [Playground Link](https://ast-grep.github.io/playground.html#eyJtb2RlIjoiQ29uZmlnIiwibGFuZyI6InR5cGVzY3JpcHQiLCJxdWVyeSI6IiIsInJld3JpdGUiOiIiLCJzdHJpY3RuZXNzIjoicmVsYXhlZCIsInNlbGVjdG9yIjoiIiwiY29uZmlnIjoicnVsZTpcbiAgIyB0aGUgdXNhZ2VcbiAga2luZDogaWRlbnRpZmllclxuICBwYXR0ZXJuOiAkTU9EXG4gICMgaXRzIHJlbGF0aW9uc2hpcCB0byB0aGUgcm9vdFxuICBpbnNpZGU6XG4gICAgc3RvcEJ5OiBlbmRcbiAgICBraW5kOiBwcm9ncmFtXG4gICAgIyBhbmQgYmFjayBkb3duIHRvIHRoZSBpbXBvcnQgc3RhdGVtZW50XG4gICAgaGFzOlxuICAgICAga2luZDogaW1wb3J0X3N0YXRlbWVudFxuICAgICAgIyBhbmQgZGVlcGVyIGludG8gdGhlIGltcG9ydCBzdGF0ZW1lbnQgbG9va2luZyBmb3IgdGhlIG1hdGNoaW5nIGlkZW50aWZpZXJcbiAgICAgIGhhczpcbiAgICAgICAgc3RvcEJ5OiBlbmRcbiAgICAgICAga2luZDogaW1wb3J0X3NwZWNpZmllclxuICAgICAgICBwYXR0ZXJuOiAkTU9EICMgc2FtZSBwYXR0ZXJuIGFzIHRoZSB1c2FnZSBpcyBlbmZvcmNlZCBoZXJlIiwic291cmNlIjoiaW1wb3J0IHsgTW9uZ29DbGllbnQgfSBmcm9tICdtb25nb2RiJztcbmNvbnN0IHVybCA9ICdtb25nb2RiOi8vbG9jYWxob3N0OjI3MDE3JztcbmFzeW5jIGZ1bmN0aW9uIHJ1bigpIHtcbiAgY29uc3QgY2xpZW50ID0gbmV3IE1vbmdvQ2xpZW50KHVybCk7XG59XG4ifQ==) ### Description [​](https://ast-grep.github.io/catalog/typescript/find-import-usage.html#description) It is common to find the usage of an imported module in a codebase. This rule helps you to find the usage of an imported module in your codebase. The idea of this rule can be broken into several parts: * Find the use of an identifier `$MOD` * To find the import, we first need to find the root file of which `$MOD` is `inside` * The `program` file `has` an `import` statement * The `import` statement `has` the identifier `$MOD` ### YAML [​](https://ast-grep.github.io/catalog/typescript/find-import-usage.html#yaml) yaml id: find-import-usage language: typescript rule: kind: identifier # ast-grep requires a kind pattern: $MOD # the identifier to find inside: # find the root stopBy: end kind: program has: # and has the import statement kind: import_statement has: # look for the matching identifier stopBy: end kind: import_specifier pattern: $MOD # same pattern as the usage is enforced here ### Example [​](https://ast-grep.github.io/catalog/typescript/find-import-usage.html#example) ts import { MongoClient } from 'mongodb'; const url = 'mongodb://localhost:27017'; async function run() { const client = new MongoClient(url); } ### Contributed by [​](https://ast-grep.github.io/catalog/typescript/find-import-usage.html#contributed-by) [Steven Love](https://github.com/StevenLove) --- # ast-grep [Skip to content](https://ast-grep.github.io/catalog/typescript/find-import-file-without-extension.html#VPContent) Return to top Are you an LLM? You can read better optimized documentation at /catalog/typescript/find-import-file-without-extension.md for this page in Markdown format Find Import File without Extension [​](https://ast-grep.github.io/catalog/typescript/find-import-file-without-extension.html#find-import-file-without-extension) ----------------------------------------------------------------------------------------------------------------------------------------------------------------- * [Playground Link](https://ast-grep.github.io/playground.html#eyJtb2RlIjoiQ29uZmlnIiwibGFuZyI6ImphdmFzY3JpcHQiLCJxdWVyeSI6ImNvbnNvbGUubG9nKCRNQVRDSCkiLCJyZXdyaXRlIjoibG9nZ2VyLmxvZygkTUFUQ0gpIiwiY29uZmlnIjoibGFuZ3VhZ2U6IFwianNcIlxucnVsZTpcbiAgcmVnZXg6IFwiL1teLl0rW14vXSRcIiAgXG4gIGtpbmQ6IHN0cmluZ19mcmFnbWVudFxuICBhbnk6XG4gICAgLSBpbnNpZGU6XG4gICAgICAgIHN0b3BCeTogZW5kXG4gICAgICAgIGtpbmQ6IGltcG9ydF9zdGF0ZW1lbnRcbiAgICAtIGluc2lkZTpcbiAgICAgICAgc3RvcEJ5OiBlbmRcbiAgICAgICAga2luZDogY2FsbF9leHByZXNzaW9uXG4gICAgICAgIGhhczpcbiAgICAgICAgICBmaWVsZDogZnVuY3Rpb25cbiAgICAgICAgICByZWdleDogXCJeaW1wb3J0JFwiXG4iLCJzb3VyY2UiOiJpbXBvcnQgYSwge2IsIGMsIGR9IGZyb20gXCIuL2ZpbGVcIjtcbmltcG9ydCBlIGZyb20gXCIuL290aGVyX2ZpbGUuanNcIjtcbmltcG9ydCBcIi4vZm9sZGVyL1wiO1xuaW1wb3J0IHt4fSBmcm9tIFwicGFja2FnZVwiO1xuaW1wb3J0IHt5fSBmcm9tIFwicGFja2FnZS93aXRoL3BhdGhcIjtcblxuaW1wb3J0KFwiLi9keW5hbWljMVwiKTtcbmltcG9ydChcIi4vZHluYW1pYzIuanNcIik7XG5cbm15X2Z1bmMoXCIuL3VucmVsYXRlZF9wYXRoX3N0cmluZ1wiKVxuXG4ifQ==) ### Description [​](https://ast-grep.github.io/catalog/typescript/find-import-file-without-extension.html#description) In ECMAScript modules (ESM), the module specifier must include the file extension, such as `.js` or `.mjs`, when importing local or absolute modules. This is because ESM does not perform any automatic file extension resolution, unlike CommonJS modules tools such as Webpack or Babel. This behavior matches how import behaves in browser environments, and is specified by the [ESM module spec](https://stackoverflow.com/questions/66375075/node-14-ecmascript-modules-import-modules-without-file-extensions) . The rule finds all imports (static and dynamic) for files without a file extension. ### YAML [​](https://ast-grep.github.io/catalog/typescript/find-import-file-without-extension.html#yaml) yaml id: find-import-file language: js rule: regex: "/[^.]+[^/]$" kind: string_fragment any: - inside: stopBy: end kind: import_statement - inside: stopBy: end kind: call_expression has: field: function regex: "^import$" ### Example [​](https://ast-grep.github.io/catalog/typescript/find-import-file-without-extension.html#example) ts import a, {b, c, d} from "./file"; import e from "./other_file.js"; import "./folder/"; import {x} from "package"; import {y} from "package/with/path"; import("./dynamic1"); import("./dynamic2.js"); my_func("./unrelated_path_string") ### Contributed by [​](https://ast-grep.github.io/catalog/typescript/find-import-file-without-extension.html#contributed-by) [DasSurma](https://twitter.com/DasSurma) in [this tweet](https://x.com/DasSurma/status/1706213303331029277) . --- # ast-grep [Skip to content](https://ast-grep.github.io/catalog/typescript/no-await-in-promise-all.html#VPContent) Return to top Are you an LLM? You can read better optimized documentation at /catalog/typescript/no-await-in-promise-all.md for this page in Markdown format No `await` in `Promise.all` array Has Fix [​](https://ast-grep.github.io/catalog/typescript/no-await-in-promise-all.html#no-await-in-promise-all-array) -------------------------------------------------------------------------------------------------------------------------------------------------------- * [Playground Link](https://ast-grep.github.io/playground.html#eyJtb2RlIjoiQ29uZmlnIiwibGFuZyI6ImphdmFzY3JpcHQiLCJxdWVyeSI6ImNvbnNvbGUubG9nKCRNQVRDSCkiLCJyZXdyaXRlIjoibG9nZ2VyLmxvZygkTUFUQ0gpIiwiY29uZmlnIjoiaWQ6IG5vLWF3YWl0LWluLXByb21pc2UtYWxsXG5zZXZlcml0eTogZXJyb3Jcbmxhbmd1YWdlOiBKYXZhU2NyaXB0XG5tZXNzYWdlOiBObyBhd2FpdCBpbiBQcm9taXNlLmFsbFxucnVsZTpcbiAgcGF0dGVybjogYXdhaXQgJEFcbiAgaW5zaWRlOlxuICAgIHBhdHRlcm46IFByb21pc2UuYWxsKCRfKVxuICAgIHN0b3BCeTpcbiAgICAgIG5vdDogeyBhbnk6IFt7a2luZDogYXJyYXl9LCB7a2luZDogYXJndW1lbnRzfV0gfVxuZml4OiAkQSIsInNvdXJjZSI6ImNvbnN0IFtmb28sIGJhcl0gPSBhd2FpdCBQcm9taXNlLmFsbChbXG4gIGF3YWl0IGdldEZvbygpLFxuICBnZXRCYXIoKSxcbiAgKGFzeW5jICgpID0+IHsgYXdhaXQgZ2V0QmF6KCl9KSgpLFxuXSkifQ==) ### Description [​](https://ast-grep.github.io/catalog/typescript/no-await-in-promise-all.html#description) Using `await` inside an inline `Promise.all` array is usually a mistake, as it defeats the purpose of running the promises in parallel. Instead, the promises should be created without `await` and passed to `Promise.all`, which can then be awaited. ### YAML [​](https://ast-grep.github.io/catalog/typescript/no-await-in-promise-all.html#yaml) yaml id: no-await-in-promise-all language: typescript rule: pattern: await $A inside: pattern: Promise.all($_) stopBy: not: { any: [{kind: array}, {kind: arguments}] } fix: $A ### Example [​](https://ast-grep.github.io/catalog/typescript/no-await-in-promise-all.html#example) ts const [foo, bar] = await Promise.all([\ await getFoo(),\ getBar(),\ (async () => { await getBaz()})(),\ ]) ### Diff [​](https://ast-grep.github.io/catalog/typescript/no-await-in-promise-all.html#diff) ts const [foo, bar] = await Promise.all([\ await getFoo(), \ getFoo(), \ getBar(),\ (async () => { await getBaz()})(),\ ]) ### Contributed by [​](https://ast-grep.github.io/catalog/typescript/no-await-in-promise-all.html#contributed-by) Inspired by [Alvar Lagerlöf](https://twitter.com/alvarlagerlof) --- # ast-grep [Skip to content](https://ast-grep.github.io/catalog/typescript/missing-component-decorator.html#VPContent) Return to top Are you an LLM? You can read better optimized documentation at /catalog/typescript/missing-component-decorator.md for this page in Markdown format Missing Component Decorator [​](https://ast-grep.github.io/catalog/typescript/missing-component-decorator.html#missing-component-decorator) -------------------------------------------------------------------------------------------------------------------------------------------- * [Playground Link](https://ast-grep.github.io/playground.html#eyJtb2RlIjoiQ29uZmlnIiwibGFuZyI6ImphdmFzY3JpcHQiLCJxdWVyeSI6ImltcG9ydCAkQSBmcm9tICdhbmltZWpzJyIsInJld3JpdGUiOiJpbXBvcnQgeyBhbmltZSBhcyAkQSB9IGZyb20gJ2FuaW1lJyIsInN0cmljdG5lc3MiOiJzbWFydCIsInNlbGVjdG9yIjoiIiwiY29uZmlnIjoiaWQ6IG1pc3NpbmctY29tcG9uZW50LWRlY29yYXRvclxubWVzc2FnZTogWW91J3JlIHVzaW5nIGFuIEFuZ3VsYXIgbGlmZWN5Y2xlIG1ldGhvZCwgYnV0IG1pc3NpbmcgYW4gQW5ndWxhciBAQ29tcG9uZW50KCkgZGVjb3JhdG9yLlxubGFuZ3VhZ2U6IFR5cGVTY3JpcHRcbnNldmVyaXR5OiB3YXJuaW5nXG5ydWxlOlxuICBwYXR0ZXJuOlxuICAgIGNvbnRleHQ6ICdjbGFzcyBIaSB7ICRNRVRIT0QoKSB7ICQkJF99IH0nXG4gICAgc2VsZWN0b3I6IG1ldGhvZF9kZWZpbml0aW9uXG4gIGluc2lkZTpcbiAgICBwYXR0ZXJuOiAnY2xhc3MgJEtMQVNTICQkJF8geyAkJCRfIH0nXG4gICAgc3RvcEJ5OiBlbmRcbiAgICBub3Q6XG4gICAgICBoYXM6XG4gICAgICAgIHBhdHRlcm46ICdAQ29tcG9uZW50KCQkJF8pJ1xuY29uc3RyYWludHM6XG4gIE1FVEhPRDpcbiAgICByZWdleDogbmdPbkluaXR8bmdPbkRlc3Ryb3lcbmxhYmVsczpcbiAgS0xBU1M6XG4gICAgc3R5bGU6IHByaW1hcnlcbiAgICBtZXNzYWdlOiBcIlRoaXMgY2xhc3MgaXMgbWlzc2luZyB0aGUgZGVjb3JhdG9yLlwiXG4gIE1FVEhPRDpcbiAgICBzdHlsZTogc2Vjb25kYXJ5XG4gICAgbWVzc2FnZTogXCJUaGlzIGlzIGFuIEFuZ3VsYXIgbGlmZWN5Y2xlIG1ldGhvZC5cIlxubWV0YWRhdGE6XG4gIGNvbnRyaWJ1dGVkQnk6IHNhbXdpZ2h0dCIsInNvdXJjZSI6ImNsYXNzIE5vdENvbXBvbmVudCB7XG4gICAgbmdPbkluaXQoKSB7fVxufVxuXG5AQ29tcG9uZW50KClcbmNsYXNzIEtsYXNzIHtcbiAgICBuZ09uSW5pdCgpIHt9XG59In0=) ### Description [​](https://ast-grep.github.io/catalog/typescript/missing-component-decorator.html#description) Angular lifecycle methods are a set of methods that allow you to hook into the lifecycle of an Angular component or directive. They must be used within a class that is decorated with the `@Component()` decorator. ### YAML [​](https://ast-grep.github.io/catalog/typescript/missing-component-decorator.html#yaml) This rule illustrates how to use custom labels to highlight specific parts of the code. yaml id: missing-component-decorator message: You're using an Angular lifecycle method, but missing an Angular @Component() decorator. language: TypeScript severity: warning rule: pattern: context: 'class Hi { $METHOD() { $$$_} }' selector: method_definition inside: pattern: 'class $KLASS $$$_ { $$$_ }' stopBy: end not: has: pattern: '@Component($$$_)' constraints: METHOD: regex: ngOnInit|ngOnDestroy labels: KLASS: style: primary message: "This class is missing the decorator." METHOD: style: secondary message: "This is an Angular lifecycle method." metadata: contributedBy: samwightt ### Example [​](https://ast-grep.github.io/catalog/typescript/missing-component-decorator.html#example) ts class NotComponent { ngOnInit() {} } @Component() class Klass { ngOnInit() {} } ### Contributed by [​](https://ast-grep.github.io/catalog/typescript/missing-component-decorator.html#contributed-by) [Sam Wight](https://github.com/samwightt) . --- # ast-grep [Skip to content](https://ast-grep.github.io/catalog/typescript/no-console-except-catch.html#VPContent) Return to top Are you an LLM? You can read better optimized documentation at /catalog/typescript/no-console-except-catch.md for this page in Markdown format No `console` except in `catch` block Has Fix [​](https://ast-grep.github.io/catalog/typescript/no-console-except-catch.html#no-console-except-in-catch-block) -------------------------------------------------------------------------------------------------------------------------------------------------------------- * [Playground Link](https://ast-grep.github.io/playground.html#eyJtb2RlIjoiQ29uZmlnIiwibGFuZyI6ImphdmFzY3JpcHQiLCJxdWVyeSI6ImlmICRBLmhhc19mZWF0dXJlP1xuICAgICQkJEJcbmVsc2UgXG4gICAgJCQkQyBcbmVuZCAiLCJyZXdyaXRlIjoiJCQkQiIsImNvbmZpZyI6InJ1bGU6XG4gIGFueTpcbiAgICAtIHBhdHRlcm46IGNvbnNvbGUuZXJyb3IoJCQkKVxuICAgICAgbm90OlxuICAgICAgICBpbnNpZGU6XG4gICAgICAgICAga2luZDogY2F0Y2hfY2xhdXNlXG4gICAgICAgICAgc3RvcEJ5OiBlbmRcbiAgICAtIHBhdHRlcm46IGNvbnNvbGUuJE1FVEhPRCgkJCQpXG5jb25zdHJhaW50czpcbiAgTUVUSE9EOlxuICAgIHJlZ2V4OiAnbG9nfGRlYnVnfHdhcm4nXG5maXg6ICcnIiwic291cmNlIjoiY29uc29sZS5kZWJ1ZygnJylcbnRyeSB7XG4gICAgY29uc29sZS5sb2coJ2hlbGxvJylcbn0gY2F0Y2ggKGUpIHtcbiAgICBjb25zb2xlLmVycm9yKGUpXG59In0=) ### Description [​](https://ast-grep.github.io/catalog/typescript/no-console-except-catch.html#description) Using `console` methods is usually for debugging purposes and therefore not suitable to ship to the client. `console` can expose sensitive information, clutter the output, or affect the performance. The only exception is using `console.error` to log errors in the catch block, which can be useful for debugging production. ### YAML [​](https://ast-grep.github.io/catalog/typescript/no-console-except-catch.html#yaml) yaml id: no-console-except-error language: typescript rule: any: - pattern: console.error($$$) not: inside: kind: catch_clause stopBy: end - pattern: console.$METHOD($$$) constraints: METHOD: regex: 'log|debug|warn' ### Example [​](https://ast-grep.github.io/catalog/typescript/no-console-except-catch.html#example) ts console.debug('') try { console.log('hello') } catch (e) { console.error(e) // OK } ### Diff [​](https://ast-grep.github.io/catalog/typescript/no-console-except-catch.html#diff) ts console.debug('') try { console.log('hello') } catch (e) { console.error(e) // OK } ### Contributed by [​](https://ast-grep.github.io/catalog/typescript/no-console-except-catch.html#contributed-by) Inspired by [Jerry Mouse](https://github.com/WWK563388548) --- # ast-grep [Skip to content](https://ast-grep.github.io/catalog/typescript/speed-up-barrel-import.html#VPContent) Return to top Are you an LLM? You can read better optimized documentation at /catalog/typescript/speed-up-barrel-import.md for this page in Markdown format Speed up Barrel Import Has Fix [​](https://ast-grep.github.io/catalog/typescript/speed-up-barrel-import.html#speed-up-barrel-import) ------------------------------------------------------------------------------------------------------------------------------------- * [Playground Link](https://ast-grep.github.io/playground.html#eyJtb2RlIjoiQ29uZmlnIiwibGFuZyI6ImphdmFzY3JpcHQiLCJxdWVyeSI6IiIsInJld3JpdGUiOiIiLCJjb25maWciOiJydWxlOlxuICBwYXR0ZXJuOiBpbXBvcnQgeyQkJElERU5UU30gZnJvbSAnLi9iYXJyZWwnXG5yZXdyaXRlcnM6XG4tIGlkOiByZXdyaXRlLWlkZW50aWZlclxuICBydWxlOlxuICAgIHBhdHRlcm46ICRJREVOVFxuICAgIGtpbmQ6IGlkZW50aWZpZXJcbiAgZml4OiBpbXBvcnQgJElERU5UIGZyb20gJy4vYmFycmVsLyRJREVOVCdcbnRyYW5zZm9ybTpcbiAgSU1QT1JUUzpcbiAgICByZXdyaXRlOlxuICAgICAgcmV3cml0ZXJzOiBbcmV3cml0ZS1pZGVudGlmZXJdXG4gICAgICBzb3VyY2U6ICQkJElERU5UU1xuICAgICAgam9pbkJ5OiBcIlxcblwiXG5maXg6ICRJTVBPUlRTIiwic291cmNlIjoiaW1wb3J0IHsgYSwgYiwgYyB9IGZyb20gJy4vYmFycmVsJzsifQ==) ### Description [​](https://ast-grep.github.io/catalog/typescript/speed-up-barrel-import.html#description) A [barrel import](https://adrianfaciu.dev/posts/barrel-files/) is a way to consolidate the exports of multiple modules into a single convenient module that can be imported using a single import statement. For instance, `import {a, b, c} from './barrel'`. It has [some](https://vercel.com/blog/how-we-optimized-package-imports-in-next-js) [benefits](https://marvinh.dev/blog/speeding-up-javascript-ecosystem-part-7/) to import each module directly from its own file without going through the barrel file. Such as reducing [bundle size](https://dev.to/tassiofront/barrel-files-and-why-you-should-stop-using-them-now-bc4) , improving building time or avoiding [conflicting names](https://flaming.codes/posts/barrel-files-in-javascript/) . ### YAML [​](https://ast-grep.github.io/catalog/typescript/speed-up-barrel-import.html#yaml) yaml id: speed-up-barrel-import language: typescript # find the barrel import statement rule: pattern: import {$$$IDENTS} from './barrel' # rewrite imported identifiers to direct imports rewriters: - id: rewrite-identifer rule: pattern: $IDENT kind: identifier fix: import $IDENT from './barrel/$IDENT' # apply the rewriter to the import statement transform: IMPORTS: rewrite: rewriters: [rewrite-identifer] # $$$IDENTS contains imported identifiers source: $$$IDENTS # join the rewritten imports by newline joinBy: "\n" fix: $IMPORTS ### Example [​](https://ast-grep.github.io/catalog/typescript/speed-up-barrel-import.html#example) ts import {a, b, c} from './barrel' ### Diff [​](https://ast-grep.github.io/catalog/typescript/speed-up-barrel-import.html#diff) ts import {a, b, c} from './barrel' import a from './barrel/a' import b from './barrel/b' import c from './barrel/c' ### Contributed by [​](https://ast-grep.github.io/catalog/typescript/speed-up-barrel-import.html#contributed-by) [Herrington Darkholme](https://x.com/hd_nvim) --- # ast-grep [Skip to content](https://ast-grep.github.io/catalog/typescript/use-logical-assignment.html#VPContent) Return to top Are you an LLM? You can read better optimized documentation at /catalog/typescript/use-logical-assignment.md for this page in Markdown format Use Logical Assignment Operators [​](https://ast-grep.github.io/catalog/typescript/use-logical-assignment.html#use-logical-assignment-operators) ------------------------------------------------------------------------------------------------------------------------------------------------- * [Playground Link](https://ast-grep.github.io/playground.html#eyJtb2RlIjoiUGF0Y2giLCJsYW5nIjoiamF2YXNjcmlwdCIsInF1ZXJ5IjoiJEEgPSAkQSB8fCAkQiIsInJld3JpdGUiOiIkQSB8fD0gJEIiLCJzdHJpY3RuZXNzIjoic21hcnQiLCJzZWxlY3RvciI6IiIsImNvbmZpZyI6IiMgWUFNTCBSdWxlIGlzIG1vcmUgcG93ZXJmdWwhXG4jIGh0dHBzOi8vYXN0LWdyZXAuZ2l0aHViLmlvL2d1aWRlL3J1bGUtY29uZmlnLmh0bWwjcnVsZVxucnVsZTpcbiAgYW55OlxuICAgIC0gcGF0dGVybjogY29uc29sZS5sb2coJEEpXG4gICAgLSBwYXR0ZXJuOiBjb25zb2xlLmRlYnVnKCRBKVxuZml4OlxuICBsb2dnZXIubG9nKCRBKSIsInNvdXJjZSI6ImNvbnN0IGEgPSB7IGR1cmF0aW9uOiA1MCwgdGl0bGU6ICcnIH07XG5cbmEuZHVyYXRpb24gPSBhLmR1cmF0aW9uIHx8IDEwO1xuY29uc29sZS5sb2coYS5kdXJhdGlvbik7XG5hLnRpdGxlID0gYS50aXRsZSB8fCAndGl0bGUgaXMgZW1wdHkuJztcbmNvbnNvbGUubG9nKGEudGl0bGUpO1xuIn0=) ### Description [​](https://ast-grep.github.io/catalog/typescript/use-logical-assignment.html#description) A logical assignment operator in JavaScript combines a logical operation (like OR or nullish coalescing) with an assignment. It updates a variable or property only under specific conditions, making code more concise. This is a relatively new feature in JavaScript (introduced in ES2021), so older codebases might not use it yet. This rule identifies instances where a variable is assigned a value using a logical OR (`||`) operation and suggests replacing it with the more concise logical assignment operator (`||=`). ### Pattern [​](https://ast-grep.github.io/catalog/typescript/use-logical-assignment.html#pattern) shell ast-grep -p '$A = $A || $B' -r '$A ||= $B' -l ts ### Example [​](https://ast-grep.github.io/catalog/typescript/use-logical-assignment.html#example) ts const a = { duration: 50, title: '' }; a.duration = a.duration || 10; console.log(a.duration); a.title = a.title || 'title is empty.'; console.log(a.title); ### Diff [​](https://ast-grep.github.io/catalog/typescript/use-logical-assignment.html#diff) ts const a = { duration: 50, title: '' }; a.duration = a.duration || 10; a.duration ||= 10; console.log(a.duration); a.title = a.title || 'title is empty.'; a.title ||= 'title is empty.'; console.log(a.title); ### Contributed by [​](https://ast-grep.github.io/catalog/typescript/use-logical-assignment.html#contributed-by) Inspired by [this tweet](https://x.com/YTCodeAntonio/status/1973720331656605809) . --- # YAML | ast-grep [Skip to content](https://ast-grep.github.io/catalog/yaml/#VPContent) Return to top Are you an LLM? You can read better optimized documentation at /catalog/yaml.md for this page in Markdown format YAML [​](https://ast-grep.github.io/catalog/yaml/#yaml) ======================================================== This page curates a list of example ast-grep rules to check and to rewrite YAML code. Find key/value and Show Message [​](https://ast-grep.github.io/catalog/yaml/#find-key-value-and-show-message) -------------------------------------------------------------------------------------------------------------- * [Playground Link](https://ast-grep.github.io/playground.html#eyJtb2RlIjoiQ29uZmlnIiwibGFuZyI6InlhbWwiLCJxdWVyeSI6IiIsInJld3JpdGUiOiIiLCJzdHJpY3RuZXNzIjoic21hcnQiLCJzZWxlY3RvciI6IiIsImNvbmZpZyI6ImlkOiBkZXRlY3QtaG9zdC1wb3J0XG5tZXNzYWdlOiBZb3UgYXJlIHVzaW5nICRIT1NUIG9uIFBvcnQgJFBPUlQsIHBsZWFzZSBjaGFuZ2UgaXQgdG8gODAwMFxuc2V2ZXJpdHk6IGVycm9yXG5ydWxlOlxuICBhbnk6XG4gIC0gcGF0dGVybjogfFxuICAgICBwb3J0OiAkUE9SVFxuICAtIHBhdHRlcm46IHxcbiAgICAgaG9zdDogJEhPU1QiLCJzb3VyY2UiOiJkYjpcbiAgIHVzZXJuYW1lOiByb290XG4gICBwYXNzd29yZDogcm9vdFxuXG5zZXJ2ZXI6XG4gIGhvc3Q6IDEyNy4wLjAuMVxuICBwb3J0OiA4MDAxIn0=) ### Description [​](https://ast-grep.github.io/catalog/yaml/#description) This YAML rule helps detecting specific host and port configurations in your code. For example, it checks if the port is set to something other than 8000 or if a particular host is used. It provides an error message prompting you to update the configuration. ### YAML [​](https://ast-grep.github.io/catalog/yaml/#yaml-1) yaml id: detect-host-port message: You are using $HOST on Port $PORT, please change it to 8000 severity: error rule: any: - pattern: | port: $PORT - pattern: | host: $HOST ### Example [​](https://ast-grep.github.io/catalog/yaml/#example) yaml db: username: root password: root server: host: 127.0.0.1 port: 8001 ### Contributed by [​](https://ast-grep.github.io/catalog/yaml/#contributed-by) [rohitcoder](https://twitter.com/rohitcoder) on [Discord](https://discord.com/invite/4YZjf6htSQ) . --- # ast-grep [Skip to content](https://ast-grep.github.io/catalog/typescript/migrate-xstate-v5.html#VPContent) Return to top Are you an LLM? You can read better optimized documentation at /catalog/typescript/migrate-xstate-v5.md for this page in Markdown format Migrate XState to v5 from v4 Has Fix [​](https://ast-grep.github.io/catalog/typescript/migrate-xstate-v5.html#migrate-xstate-to-v5-from-v4) -------------------------------------------------------------------------------------------------------------------------------------------- * [Playground Link](https://ast-grep.github.io/playground.html#eyJtb2RlIjoiQ29uZmlnIiwibGFuZyI6ImphdmFzY3JpcHQiLCJxdWVyeSI6ImlmICgkQSkgeyAkJCRCIH0iLCJyZXdyaXRlIjoiaWYgKCEoJEEpKSB7XG4gICAgcmV0dXJuO1xufVxuJCQkQiIsImNvbmZpZyI6InV0aWxzOlxuICBGUk9NX1hTVEFURTogeyBraW5kOiBpbXBvcnRfc3RhdGVtZW50LCBoYXM6IHsga2luZDogc3RyaW5nLCByZWdleDogeHN0YXRlIH0gfVxuICBYU1RBVEVfRVhQT1JUOlxuICAgIGtpbmQ6IGlkZW50aWZpZXJcbiAgICBpbnNpZGU6IHsgaGFzOiB7IG1hdGNoZXM6IEZST01fWFNUQVRFIH0sIHN0b3BCeTogZW5kIH1cbnJ1bGU6IHsgcmVnZXg6IF5NYWNoaW5lfGludGVycHJldCQsIHBhdHRlcm46ICRJTVBPUlQsIG1hdGNoZXM6IFhTVEFURV9FWFBPUlQgfVxudHJhbnNmb3JtOlxuICBTVEVQMTogXG4gICAgcmVwbGFjZToge2J5OiBjcmVhdGUkMSwgcmVwbGFjZTogKE1hY2hpbmUpLCBzb3VyY2U6ICRJTVBPUlQgfVxuICBGSU5BTDpcbiAgICByZXBsYWNlOiB7IGJ5OiBjcmVhdGVBY3RvciwgcmVwbGFjZTogaW50ZXJwcmV0LCBzb3VyY2U6ICRTVEVQMSB9XG5maXg6ICRGSU5BTFxuLS0tIFxucnVsZTogeyBwYXR0ZXJuOiAkTUFDSElORS53aXRoQ29uZmlnIH1cbmZpeDogJE1BQ0hJTkUucHJvdmlkZVxuLS0tXG5ydWxlOlxuICBraW5kOiBwcm9wZXJ0eV9pZGVudGlmaWVyXG4gIHJlZ2V4OiBec2VydmljZXMkXG4gIGluc2lkZTogeyBwYXR0ZXJuOiAgJE0ud2l0aENvbmZpZygkJCRBUkdTKSwgc3RvcEJ5OiBlbmQgfVxuZml4OiBhY3RvcnMiLCJzb3VyY2UiOiJpbXBvcnQgeyBNYWNoaW5lLCBpbnRlcnByZXQgfSBmcm9tICd4c3RhdGUnO1xuXG5jb25zdCBtYWNoaW5lID0gTWFjaGluZSh7IC8qLi4uKi99KTtcblxuY29uc3Qgc3BlY2lmaWNNYWNoaW5lID0gbWFjaGluZS53aXRoQ29uZmlnKHtcbiAgYWN0aW9uczogeyAvKiAuLi4gKi8gfSxcbiAgZ3VhcmRzOiB7IC8qIC4uLiAqLyB9LFxuICBzZXJ2aWNlczogeyAvKiAuLi4gKi8gfSxcbn0pO1xuXG5jb25zdCBhY3RvciA9IGludGVycHJldChzcGVjaWZpY01hY2hpbmUsIHtcbi8qIGFjdG9yIG9wdGlvbnMgKi9cbn0pOyJ9) ### Description [​](https://ast-grep.github.io/catalog/typescript/migrate-xstate-v5.html#description) [XState](https://xstate.js.org/) is a state management/orchestration library based on state machines, statecharts, and the actor model. It allows you to model complex logic in event-driven ways, and orchestrate the behavior of many actors communicating with each other. XState's v5 version introduced some breaking changes and new features compared to v4. While the migration should be a straightforward process, it is a tedious process and requires knowledge of the differences between v4 and v5. ast-grep provides a way to automate the process and a way to encode valuable knowledge to executable rules. The following example picks up some migration items and demonstrates the power of ast-grep's rule system. ### YAML [​](https://ast-grep.github.io/catalog/typescript/migrate-xstate-v5.html#yaml) The rules below correspond to XState v5's [`createMachine`](https://stately.ai/docs/migration#use-createmachine-not-machine) , [`createActor`](https://stately.ai/docs/migration#use-createactor-not-interpret) , and [`machine.provide`](https://stately.ai/docs/migration#use-machineprovide-not-machinewithconfig) . The example shows how ast-grep can use various features like [utility rule](https://ast-grep.github.io/guide/rule-config/utility-rule.html) , [transformation](https://ast-grep.github.io/reference/yaml/transformation.html) and [multiple rule in single file](https://ast-grep.github.io/reference/playground.html#test-multiple-rules) to automate the migration. Each rule has a clear and descriptive `id` field that explains its purpose. For more information, you can use [Codemod AI](https://app.codemod.com/studio?ai_thread_id=new) to provide more detailed explanation for each rule. yaml id: migrate-import-name utils: FROM_XS: {kind: import_statement, has: {kind: string, regex: xstate}} XS_EXPORT: kind: identifier inside: { has: { matches: FROM_XS }, stopBy: end } rule: { regex: ^Machine|interpret$, pattern: $IMPT, matches: XS_EXPORT } transform: STEP1: replace: {by: create$1, replace: (Machine), source: $IMPT } FINAL: replace: { by: createActor, replace: interpret, source: $STEP1 } fix: $FINAL --- id: migrate-to-provide rule: { pattern: $MACHINE.withConfig } fix: $MACHINE.provide --- id: migrate-to-actors rule: kind: property_identifier regex: ^services$ inside: { pattern: $M.withConfig($$$ARGS), stopBy: end } fix: actors ### Example [​](https://ast-grep.github.io/catalog/typescript/migrate-xstate-v5.html#example) js import { Machine, interpret } from 'xstate'; const machine = Machine({ /*...*/}); const specificMachine = machine.withConfig({ actions: { /* ... */ }, guards: { /* ... */ }, services: { /* ... */ }, }); const actor = interpret(specificMachine, { /* actor options */ }); ### Diff [​](https://ast-grep.github.io/catalog/typescript/migrate-xstate-v5.html#diff) js import { Machine, interpret } from 'xstate'; import { createMachine, createActor } from 'xstate'; const machine = Machine({ /*...*/}); const machine = createMachine({ /*...*/}); const specificMachine = machine.withConfig({ const specificMachine = machine.provide({ actions: { /* ... */ }, guards: { /* ... */ }, services: { /* ... */ }, actors: { /* ... */ }, }); const actor = interpret(specificMachine, { const actor = createActor(specificMachine, { /* actor options */ }); ### Contributed by [​](https://ast-grep.github.io/catalog/typescript/migrate-xstate-v5.html#contributed-by) Inspired by [XState's blog](https://stately.ai/blog/2023-12-01-xstate-v5) . --- # ast-grep [Skip to content](https://ast-grep.github.io/catalog/typescript/switch-from-should-to-expect.html#VPContent) Return to top Are you an LLM? You can read better optimized documentation at /catalog/typescript/switch-from-should-to-expect.md for this page in Markdown format Switch Chai from `should` style to `expect` Has Fix [​](https://ast-grep.github.io/catalog/typescript/switch-from-should-to-expect.html#switch-chai-from-should-style-to-expect) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- * [Playground Link](https://ast-grep.github.io/playground.html#eyJtb2RlIjoiQ29uZmlnIiwibGFuZyI6InJ1c3QiLCJxdWVyeSI6IiIsInJld3JpdGUiOiIiLCJzdHJpY3RuZXNzIjoicmVsYXhlZCIsInNlbGVjdG9yIjoiIiwiY29uZmlnIjoiaWQ6IHNob3VsZF90b19leHBlY3RfaW5zdGFuY2VvZlxubGFuZ3VhZ2U6IFR5cGVTY3JpcHRcbnJ1bGU6XG4gIGFueTpcbiAgLSBwYXR0ZXJuOiAkTkFNRS5zaG91bGQuYmUuYW4uaW5zdGFuY2VvZigkVFlQRSlcbiAgLSBwYXR0ZXJuOiAkTkFNRS5zaG91bGQuYmUuYW4uaW5zdGFuY2VPZigkVFlQRSlcbmZpeDogfC1cbiAgZXhwZWN0KCROQU1FKS5pbnN0YW5jZU9mKCRUWVBFKVxuLS0tXG5pZDogc2hvdWxkX3RvX2V4cGVjdF9nZW5lcmljU2hvdWxkQmVcbmxhbmd1YWdlOiBUeXBlU2NyaXB0XG5ydWxlOlxuICBwYXR0ZXJuOiAkTkFNRS5zaG91bGQuYmUuJFBST1BcbmZpeDogfC1cbiAgZXhwZWN0KCROQU1FKS50by5iZS4kUFJPUFxuIiwic291cmNlIjoiaXQoJ3Nob3VsZCBwcm9kdWNlIGFuIGluc3RhbmNlIG9mIGNob2tpZGFyLkZTV2F0Y2hlcicsICgpID0+IHtcbiAgd2F0Y2hlci5zaG91bGQuYmUuYW4uaW5zdGFuY2VvZihjaG9raWRhci5GU1dhdGNoZXIpO1xufSk7XG5pdCgnc2hvdWxkIGV4cG9zZSBwdWJsaWMgQVBJIG1ldGhvZHMnLCAoKSA9PiB7XG4gIHdhdGNoZXIub24uc2hvdWxkLmJlLmEoJ2Z1bmN0aW9uJyk7XG4gIHdhdGNoZXIuZW1pdC5zaG91bGQuYmUuYSgnZnVuY3Rpb24nKTtcbiAgd2F0Y2hlci5hZGQuc2hvdWxkLmJlLmEoJ2Z1bmN0aW9uJyk7XG4gIHdhdGNoZXIuY2xvc2Uuc2hvdWxkLmJlLmEoJ2Z1bmN0aW9uJyk7XG4gIHdhdGNoZXIuZ2V0V2F0Y2hlZC5zaG91bGQuYmUuYSgnZnVuY3Rpb24nKTtcbn0pOyJ9) ### Description [​](https://ast-grep.github.io/catalog/typescript/switch-from-should-to-expect.html#description) [Chai](https://www.chaijs.com/) is a BDD / TDD assertion library for JavaScript. It comes with [two styles](https://www.chaijs.com/) of assertions: `should` and `expect`. The `expect` interface provides a function as a starting point for chaining your language assertions and works with `undefined` and `null` values. The `should` style allows for the same chainable assertions as the expect interface, however it extends each object with a should property to start your chain and [does not work](https://www.chaijs.com/guide/styles/#should-extras) with `undefined` and `null` values. This rule migrates Chai `should` style assertions to `expect` style assertions. Note this is an example rule and a excerpt from [the original rules](https://github.com/43081j/codemods/blob/cddfe101e7f759e4da08b7e2f7bfe892c20f6f48/codemods/chai-should-to-expect.yml) . ### YAML [​](https://ast-grep.github.io/catalog/typescript/switch-from-should-to-expect.html#yaml) yaml id: should_to_expect_instanceof language: TypeScript rule: any: - pattern: $NAME.should.be.an.instanceof($TYPE) - pattern: $NAME.should.be.an.instanceOf($TYPE) fix: |- expect($NAME).instanceOf($TYPE) --- id: should_to_expect_genericShouldBe language: TypeScript rule: pattern: $NAME.should.be.$PROP fix: |- expect($NAME).to.be.$PROP ### Example [​](https://ast-grep.github.io/catalog/typescript/switch-from-should-to-expect.html#example) js it('should produce an instance of chokidar.FSWatcher', () => { watcher.should.be.an.instanceof(chokidar.FSWatcher); }); it('should expose public API methods', () => { watcher.on.should.be.a('function'); watcher.emit.should.be.a('function'); watcher.add.should.be.a('function'); watcher.close.should.be.a('function'); watcher.getWatched.should.be.a('function'); }); ### Diff [​](https://ast-grep.github.io/catalog/typescript/switch-from-should-to-expect.html#diff) js it('should produce an instance of chokidar.FSWatcher', () => { watcher.should.be.an.instanceof(chokidar.FSWatcher); expect(watcher).instanceOf(chokidar.FSWatcher); }); it('should expose public API methods', () => { watcher.on.should.be.a('function'); watcher.emit.should.be.a('function'); watcher.add.should.be.a('function'); watcher.close.should.be.a('function'); watcher.getWatched.should.be.a('function'); expect(watcher.on).to.be.a('function'); expect(watcher.emit).to.be.a('function'); expect(watcher.add).to.be.a('function'); expect(watcher.close).to.be.a('function'); expect(watcher.getWatched).to.be.a('function'); }); ### Contributed by [​](https://ast-grep.github.io/catalog/typescript/switch-from-should-to-expect.html#contributed-by) [James](https://bsky.app/profile/43081j.com) , by [this post](https://bsky.app/profile/43081j.com/post/3lgimzfxza22i) ### Exercise [​](https://ast-grep.github.io/catalog/typescript/switch-from-should-to-expect.html#exercise) Exercise left to the reader: can you write a rule to implement [this migration to `node:assert`](https://github.com/paulmillr/chokidar/pull/1409/files) ? --- # ast-grep [Skip to content](https://ast-grep.github.io/catalog/yaml/find-key-value.html#VPContent) Return to top Are you an LLM? You can read better optimized documentation at /catalog/yaml/find-key-value.md for this page in Markdown format Find key/value and Show Message [​](https://ast-grep.github.io/catalog/yaml/find-key-value.html#find-key-value-and-show-message) --------------------------------------------------------------------------------------------------------------------------------- * [Playground Link](https://ast-grep.github.io/playground.html#eyJtb2RlIjoiQ29uZmlnIiwibGFuZyI6InlhbWwiLCJxdWVyeSI6IiIsInJld3JpdGUiOiIiLCJzdHJpY3RuZXNzIjoic21hcnQiLCJzZWxlY3RvciI6IiIsImNvbmZpZyI6ImlkOiBkZXRlY3QtaG9zdC1wb3J0XG5tZXNzYWdlOiBZb3UgYXJlIHVzaW5nICRIT1NUIG9uIFBvcnQgJFBPUlQsIHBsZWFzZSBjaGFuZ2UgaXQgdG8gODAwMFxuc2V2ZXJpdHk6IGVycm9yXG5ydWxlOlxuICBhbnk6XG4gIC0gcGF0dGVybjogfFxuICAgICBwb3J0OiAkUE9SVFxuICAtIHBhdHRlcm46IHxcbiAgICAgaG9zdDogJEhPU1QiLCJzb3VyY2UiOiJkYjpcbiAgIHVzZXJuYW1lOiByb290XG4gICBwYXNzd29yZDogcm9vdFxuXG5zZXJ2ZXI6XG4gIGhvc3Q6IDEyNy4wLjAuMVxuICBwb3J0OiA4MDAxIn0=) ### Description [​](https://ast-grep.github.io/catalog/yaml/find-key-value.html#description) This YAML rule helps detecting specific host and port configurations in your code. For example, it checks if the port is set to something other than 8000 or if a particular host is used. It provides an error message prompting you to update the configuration. ### YAML [​](https://ast-grep.github.io/catalog/yaml/find-key-value.html#yaml) yaml id: detect-host-port message: You are using $HOST on Port $PORT, please change it to 8000 severity: error rule: any: - pattern: | port: $PORT - pattern: | host: $HOST ### Example [​](https://ast-grep.github.io/catalog/yaml/find-key-value.html#example) yaml db: username: root password: root server: host: 127.0.0.1 port: 8001 ### Contributed by [​](https://ast-grep.github.io/catalog/yaml/find-key-value.html#contributed-by) [rohitcoder](https://twitter.com/rohitcoder) on [Discord](https://discord.com/invite/4YZjf6htSQ) . --- # API Usage | ast-grep [Skip to content](https://ast-grep.github.io/guide/api-usage.html#VPContent) Return to top Are you an LLM? You can read better optimized documentation at /guide/api-usage.md for this page in Markdown format API Usage [​](https://ast-grep.github.io/guide/api-usage.html#api-usage) ========================================================================= ast-grep as Library [​](https://ast-grep.github.io/guide/api-usage.html#ast-grep-as-library) --------------------------------------------------------------------------------------------- ast-grep allows you to craft complicated rules, but it is not easy to do arbitrary AST manipulation. For example, you may struggle to: * replace a list of nodes individually, based on their content * replace a node conditionally, based on its content and surrounding nodes * count the number or order of nodes that match a certain pattern * compute the replacement string based on the matched nodes To solve these problems, you can use ast-grep's programmatic API! You can freely inspect and generate text patches based on syntax trees, using popular programming languages! TIP Applying ast-grep's `fix` using JS/Python API is still experimental. See [this issue](https://github.com/ast-grep/ast-grep/issues/1172) for more information. Language Bindings [​](https://ast-grep.github.io/guide/api-usage.html#language-bindings) ----------------------------------------------------------------------------------------- ast-grep provides support for these programming languages: * **JavaScript:** Powered by napi.rs, ast-grep's JavaScript API is the most robust and reliable. [Explore JavaScript API](https://ast-grep.github.io/guide/api-usage/js-api.html) * **Python:** ast-grep's PyO3 interface is the latest addition to climb the syntax tree! [Discover Python API](https://ast-grep.github.io/guide/api-usage/py-api.html) * **Rust:** ast-grep's Rust API is the most efficient way, but also the most challenging way, to use ast-grep. You can refer to [ast\_grep\_core](https://docs.rs/ast-grep-core/latest/ast_grep_core/) if you are familiar with Rust. Why and When to use API? [​](https://ast-grep.github.io/guide/api-usage.html#why-and-when-to-use-api) ------------------------------------------------------------------------------------------------------ ast-grep's API is designed to solve the problems that are hard to express in ast-grep's rule language. ast-grep's rule system is deliberately simple and not as powerful as a programming language. Other similar rewriting/query tools have complex features like conditional, loop, filter or function call. These features are hard to learn and use, and they cannot perform computation as well as a general purpose programming language. So ast-grep chooses to have a simple rule system that is easy to learn and use. But it also has its limitations. The API is created to overcome these limitations. If your code transformation requires complex logic, or if you need to change code that has no parser library in JavaScript or Python, ast-grep API is a good option to achieve your goal without writing a lot of complicated rules. --- # TSX | ast-grep [Skip to content](https://ast-grep.github.io/catalog/tsx/#VPContent) Return to top Are you an LLM? You can read better optimized documentation at /catalog/tsx.md for this page in Markdown format TSX [​](https://ast-grep.github.io/catalog/tsx/#tsx) ===================================================== This page curates a list of example ast-grep rules to check and to rewrite TypeScript with JSX syntax. TSX and TypeScript are different. TSX differs from TypeScript because it is an extension of the latter that supports JSX elements. They need distinct parsers because of [conflicting syntax](https://www.typescriptlang.org/docs/handbook/jsx.html#the-as-operator) . In order to reduce rule duplication, you can use the [`languageGlobs`](https://ast-grep.github.io/reference/sgconfig.html#languageglobs) option to force ast-grep to use parse `.ts` files as TSX. Unnecessary `useState` Type Has Fix [​](https://ast-grep.github.io/catalog/tsx/#unnecessary-usestate-type) ----------------------------------------------------------------------------------------------------------- * [Playground Link](https://ast-grep.github.io/playground.html#eyJtb2RlIjoiUGF0Y2giLCJsYW5nIjoidHlwZXNjcmlwdCIsInF1ZXJ5IjoidXNlU3RhdGU8c3RyaW5nPigkQSkiLCJyZXdyaXRlIjoidXNlU3RhdGUoJEEpIiwiY29uZmlnIjoiIyBZQU1MIFJ1bGUgaXMgbW9yZSBwb3dlcmZ1bCFcbiMgaHR0cHM6Ly9hc3QtZ3JlcC5naXRodWIuaW8vZ3VpZGUvcnVsZS1jb25maWcuaHRtbCNydWxlXG5ydWxlOlxuICBhbnk6XG4gICAgLSBwYXR0ZXJuOiBjb25zb2xlLmxvZygkQSlcbiAgICAtIHBhdHRlcm46IGNvbnNvbGUuZGVidWcoJEEpXG5maXg6XG4gIGxvZ2dlci5sb2coJEEpIiwic291cmNlIjoiZnVuY3Rpb24gQ29tcG9uZW50KCkge1xuICBjb25zdCBbbmFtZSwgc2V0TmFtZV0gPSB1c2VTdGF0ZTxzdHJpbmc+KCdSZWFjdCcpXG59In0=) ### Description [​](https://ast-grep.github.io/catalog/tsx/#description) React's [`useState`](https://react.dev/reference/react/useState) is a Hook that lets you add a state variable to your component. The type annotation of `useState`'s generic type argument, for example `useState(123)`, is unnecessary if TypeScript can infer the type of the state variable from the initial value. We can usually skip annotating if the generic type argument is a single primitive type like `number`, `string` or `boolean`. ### Pattern [​](https://ast-grep.github.io/catalog/tsx/#pattern) numberstringboolean bash ast-grep -p 'useState($A)' -r 'useState($A)' -l tsx bash ast-grep -p 'useState($A)' -r 'useState($A)' bash ast-grep -p 'useState($A)' -r 'useState($A)' ### Example [​](https://ast-grep.github.io/catalog/tsx/#example) ts function Component() { const [name, setName] = useState('React') } ### Diff [​](https://ast-grep.github.io/catalog/tsx/#diff) ts function Component() { const [name, setName] = useState('React') const [name, setName] = useState('React') } ### Contributed by [​](https://ast-grep.github.io/catalog/tsx/#contributed-by) [Herrington Darkholme](https://twitter.com/hd_nvim) Avoid `&&` short circuit in JSX Has Fix [​](https://ast-grep.github.io/catalog/tsx/#avoid-short-circuit-in-jsx) ---------------------------------------------------------------------------------------------------------------- * [Playground Link](https://ast-grep.github.io/playground.html#eyJtb2RlIjoiQ29uZmlnIiwibGFuZyI6InRzeCIsInF1ZXJ5IjoiY29uc29sZS5sb2coJE1BVENIKSIsInJld3JpdGUiOiJsb2dnZXIubG9nKCRNQVRDSCkiLCJjb25maWciOiJpZDogZG8td2hhdC1icm9vb29vb2tseW4tc2FpZFxubGFuZ3VhZ2U6IFRzeFxuc2V2ZXJpdHk6IGVycm9yXG5ydWxlOlxuICBraW5kOiBqc3hfZXhwcmVzc2lvblxuICBoYXM6XG4gICAgcGF0dGVybjogJEEgJiYgJEJcbiAgbm90OlxuICAgIGluc2lkZTpcbiAgICAgIGtpbmQ6IGpzeF9hdHRyaWJ1dGVcbmZpeDogXCJ7JEEgPyAkQiA6IG51bGx9XCIiLCJzb3VyY2UiOiI8ZGl2PntcbiAgbnVtICYmIDxkaXYvPlxufTwvZGl2PiJ9) ### Description [​](https://ast-grep.github.io/catalog/tsx/#description-1) In [React](https://react.dev/learn/conditional-rendering) , you can conditionally render JSX using JavaScript syntax like `if` statements, `&&`, and `? :` operators. However, you should almost never put numbers on the left side of `&&`. This is because React will render the number `0`, instead of the JSX element on the right side. A concrete example will be conditionally rendering a list when the list is not empty. This rule will find and fix any short-circuit rendering in JSX and rewrite it to a ternary operator. ### YAML [​](https://ast-grep.github.io/catalog/tsx/#yaml) yaml id: do-what-brooooooklyn-said language: Tsx rule: kind: jsx_expression has: pattern: $A && $B not: inside: kind: jsx_attribute fix: "{$A ? $B : null}" ### Example [​](https://ast-grep.github.io/catalog/tsx/#example-1) tsx
{ list.length && list.map(i =>

) }

### Diff [​](https://ast-grep.github.io/catalog/tsx/#diff-1) tsx
{ list.length && list.map(i =>

) }

{ list.length ? list.map(i =>

) : null }

### Contributed by [​](https://ast-grep.github.io/catalog/tsx/#contributed-by-1) [Herrington Darkholme](https://twitter.com/hd_nvim) , inspired by [@Brooooook\_lyn](https://twitter.com/Brooooook_lyn/status/1666637274757595141) Rewrite MobX Component Style Has Fix [​](https://ast-grep.github.io/catalog/tsx/#rewrite-mobx-component-style) --------------------------------------------------------------------------------------------------------------- * [Playground Link](https://ast-grep.github.io/playground.html#eyJtb2RlIjoiQ29uZmlnIiwibGFuZyI6ImphdmFzY3JpcHQiLCJxdWVyeSI6ImNvbnNvbGUubG9nKCRNQVRDSCkiLCJyZXdyaXRlIjoibG9nZ2VyLmxvZygkTUFUQ0gpIiwiY29uZmlnIjoicnVsZTpcbiAgcGF0dGVybjogZXhwb3J0IGNvbnN0ICRDT01QID0gb2JzZXJ2ZXIoJEZVTkMpXG5maXg6IHwtXG4gIGNvbnN0IEJhc2UkQ09NUCA9ICRGVU5DXG4gIGV4cG9ydCBjb25zdCAkQ09NUCA9IG9ic2VydmVyKEJhc2UkQ09NUCkiLCJzb3VyY2UiOiJleHBvcnQgY29uc3QgRXhhbXBsZSA9IG9ic2VydmVyKCgpID0+IHtcbiAgcmV0dXJuIDxkaXY+SGVsbG8gV29ybGQ8L2Rpdj5cbn0pIn0=) ### Description [​](https://ast-grep.github.io/catalog/tsx/#description-2) React and MobX are libraries that help us build user interfaces with JavaScript. [React hooks](https://react.dev/reference/react) allow us to use state and lifecycle methods in functional components. But we need follow some hook rules, or React may break. [MobX](https://mobx.js.org/react-integration.html) has an `observer` function that makes a component update when data changes. When we use the `observer` function like this: JavaScript export const Example = observer(() => {…}) ESLint, the tool that checks hooks, thinks that `Example` is not a React component, but just a regular function. So it does not check the hooks inside it, and we may miss some wrong usages. To fix this, we need to change our component style to this: JavaScript const BaseExample = () => {…} const Example = observer(BaseExample) Now ESLint can see that `BaseExample` is a React component, and it can check the hooks inside it. ### YAML [​](https://ast-grep.github.io/catalog/tsx/#yaml-1) yaml id: rewrite-mobx-component language: typescript rule: pattern: export const $COMP = observer($FUNC) fix: |- const Base$COMP = $FUNC export const $COMP = observer(Base$COMP) ### Example [​](https://ast-grep.github.io/catalog/tsx/#example-2) js export const Example = observer(() => { return
Hello World
}) ### Diff [​](https://ast-grep.github.io/catalog/tsx/#diff-2) js export const Example = observer(() => { return
Hello World
}) const BaseExample = () => { return
Hello World
} export const Example = observer(BaseExample) ### Contributed by [​](https://ast-grep.github.io/catalog/tsx/#contributed-by-2) [Bryan Lee](https://twitter.com/meetliby/status/1698601672568901723) Avoid Unnecessary React Hook [​](https://ast-grep.github.io/catalog/tsx/#avoid-unnecessary-react-hook) ------------------------------------------------------------------------------------------------------- * [Playground Link](https://ast-grep.github.io/playground.html#eyJtb2RlIjoiQ29uZmlnIiwibGFuZyI6ImphdmFzY3JpcHQiLCJxdWVyeSI6IiIsInJld3JpdGUiOiIiLCJzdHJpY3RuZXNzIjoic21hcnQiLCJzZWxlY3RvciI6IiIsImNvbmZpZyI6InV0aWxzOlxuICBob29rX2NhbGw6XG4gICAgaGFzOlxuICAgICAga2luZDogY2FsbF9leHByZXNzaW9uXG4gICAgICByZWdleDogXnVzZVxuICAgICAgc3RvcEJ5OiBlbmRcbnJ1bGU6XG4gIGFueTpcbiAgLSBwYXR0ZXJuOiBmdW5jdGlvbiAkRlVOQygkJCQpIHsgJCQkIH1cbiAgLSBwYXR0ZXJuOiBsZXQgJEZVTkMgPSAoJCQkKSA9PiAkJCQgXG4gIC0gcGF0dGVybjogY29uc3QgJEZVTkMgPSAoJCQkKSA9PiAkJCRcbiAgaGFzOlxuICAgIHBhdHRlcm46ICRCT0RZXG4gICAga2luZDogc3RhdGVtZW50X2Jsb2NrXG4gICAgc3RvcEJ5OiBlbmQgXG5jb25zdHJhaW50czpcbiAgRlVOQzoge3JlZ2V4OiBedXNlIH1cbiAgQk9EWTogeyBub3Q6IHsgbWF0Y2hlczogaG9va19jYWxsIH0gfSBcbiIsInNvdXJjZSI6ImZ1bmN0aW9uIHVzZUlBbU5vdEhvb2tBY3R1YWxseShhcmdzKSB7XG4gICAgY29uc29sZS5sb2coJ0NhbGxlZCBpbiBSZWFjdCBidXQgSSBkb250IG5lZWQgdG8gYmUgYSBob29rJylcbiAgICByZXR1cm4gYXJncy5sZW5ndGhcbn1cbmNvbnN0IHVzZUlBbU5vdEhvb2tUb28gPSAoLi4uYXJncykgPT4ge1xuICAgIGNvbnNvbGUubG9nKCdDYWxsZWQgaW4gUmVhY3QgYnV0IEkgZG9udCBuZWVkIHRvIGJlIGEgaG9vaycpXG4gICAgcmV0dXJuIGFyZ3MubGVuZ3RoXG59XG5cbmZ1bmN0aW9uIHVzZUhvb2soKSB7XG4gICAgdXNlRWZmZWN0KCgpID0+IHtcbiAgICAgIGNvbnNvbGUubG9nKCdSZWFsIGhvb2snKSAgIFxuICAgIH0pXG59In0=) ### Description [​](https://ast-grep.github.io/catalog/tsx/#description-3) React hook is a powerful feature in React that allows you to use state and other React features in a functional component. However, you should avoid using hooks when you don't need them. If the code does not contain using any other React hooks, it can be rewritten to a plain function. This can help to separate your application logic from the React-specific UI logic. ### YAML [​](https://ast-grep.github.io/catalog/tsx/#yaml-2) yaml id: unnecessary-react-hook language: Tsx utils: hook_call: has: kind: call_expression regex: ^use stopBy: end rule: any: - pattern: function $FUNC($$$) { $$$ } - pattern: let $FUNC = ($$$) => $$$ - pattern: const $FUNC = ($$$) => $$$ has: pattern: $BODY kind: statement_block stopBy: end constraints: FUNC: {regex: ^use } BODY: { not: { matches: hook_call } } ### Example [​](https://ast-grep.github.io/catalog/tsx/#example-3) tsx function useIAmNotHookActually(args) { console.log('Called in React but I dont need to be a hook') return args.length } const useIAmNotHookToo = (...args) => { console.log('Called in React but I dont need to be a hook') return args.length } function useTrueHook() { useEffect(() => { console.log('Real hook') }) } ### Contributed by [​](https://ast-grep.github.io/catalog/tsx/#contributed-by-3) [Herrington Darkholme](https://twitter.com/hd_nvim) Reverse React Compiler™ Has Fix [​](https://ast-grep.github.io/catalog/tsx/#reverse-react-compilertm) ------------------------------------------------------------------------------------------------------ * [Playground Link](https://ast-grep.github.io/playground.html#eyJtb2RlIjoiQ29uZmlnIiwibGFuZyI6InRzeCIsInF1ZXJ5IjoiIiwicmV3cml0ZSI6IiIsInN0cmljdG5lc3MiOiJyZWxheGVkIiwic2VsZWN0b3IiOiIiLCJjb25maWciOiJpZDogcmV3cml0ZS1jYWNoZSBcbmxhbmd1YWdlOiB0c3hcbnJ1bGU6XG4gIGFueTpcbiAgLSBwYXR0ZXJuOiB1c2VDYWxsYmFjaygkRk4sICQkJClcbiAgLSBwYXR0ZXJuOiBtZW1vKCRGTiwgJCQkKVxuZml4OiAkRk5cblxuLS0tXG5cbmlkOiByZXdyaXRlLXVzZS1tZW1vXG5sYW5ndWFnZTogdHN4XG5ydWxlOiB7IHBhdHRlcm46ICd1c2VNZW1vKCRGTiwgJCQkKScgfVxuZml4OiAoJEZOKSgpIiwic291cmNlIjoiY29uc3QgQ29tcG9uZW50ID0gKCkgPT4ge1xuICBjb25zdCBbY291bnQsIHNldENvdW50XSA9IHVzZVN0YXRlKDApXG4gIGNvbnN0IGluY3JlbWVudCA9IHVzZUNhbGxiYWNrKCgpID0+IHtcbiAgICBzZXRDb3VudCgocHJldkNvdW50KSA9PiBwcmV2Q291bnQgKyAxKVxuICB9LCBbXSlcbiAgY29uc3QgZXhwZW5zaXZlQ2FsY3VsYXRpb24gPSB1c2VNZW1vKCgpID0+IHtcbiAgICAvLyBtb2NrIEV4cGVuc2l2ZSBjYWxjdWxhdGlvblxuICAgIHJldHVybiBjb3VudCAqIDJcbiAgfSwgW2NvdW50XSlcblxuICByZXR1cm4gKFxuICAgIDw+XG4gICAgICA8cD5FeHBlbnNpdmUgUmVzdWx0OiB7ZXhwZW5zaXZlQ2FsY3VsYXRpb259PC9wPlxuICAgICAgPGJ1dHRvbiBvbkNsaWNrPXtpbmNyZW1lbnR9Pntjb3VudH08L2J1dHRvbj5cbiAgICA8Lz5cbiAgKVxufSJ9) ### Description [​](https://ast-grep.github.io/catalog/tsx/#description-4) React Compiler is a build-time only tool that automatically optimizes your React app, working with plain JavaScript and understanding the Rules of React without requiring a rewrite. It optimizes apps by automatically memoizing code, similar to `useMemo`, `useCallback`, and `React.memo`, reducing unnecessary recomputation due to incorrect or forgotten memoization. Reverse React Compiler™ is a [parody tweet](https://x.com/aidenybai/status/1881397529369034997) that works in the opposite direction. It takes React code and removes memoization, guaranteed to make your code slower. ([not](https://x.com/kentcdodds/status/1881404373646880997) [necessarily](https://dev.to/prathamisonline/are-you-over-using-usememo-and-usecallback-hooks-in-react-5lp) ) It is originally written in Babel and this is an [ast-grep version](https://x.com/hd_nvim/status/1881402678493970620) of it. The Original Babel Implementation For comparison purposes only. Note the original code [does not correctly rewrite](https://x.com/hd_nvim/status/1881404893136896415) `useMemo`. js const ReverseReactCompiler = ({ types: t }) => ({ visitor: { CallExpression(path) { const callee = path.node.callee; if ( t.isIdentifier(callee, { name: "useMemo" }) || t.isIdentifier(callee, { name: "useCallback" }) || t.isIdentifier(callee, { name: "memo" }) ) { path.replaceWith(args[0]); } }, }, }); ### YAML [​](https://ast-grep.github.io/catalog/tsx/#yaml-3) yaml id: rewrite-cache language: tsx rule: any: - pattern: useCallback($FN, $$$) - pattern: memo($FN, $$$) fix: $FN --- id: rewrite-use-memo language: tsx rule: { pattern: 'useMemo($FN, $$$)' } fix: ($FN)() # need IIFE to wrap memo function ### Example [​](https://ast-grep.github.io/catalog/tsx/#example-4) tsx const Component = () => { const [count, setCount] = useState(0) const increment = useCallback(() => { setCount((prevCount) => prevCount + 1) }, []) const expensiveCalculation = useMemo(() => { // mock Expensive calculation return count * 2 }, [count]) return ( <>

Expensive Result: {expensiveCalculation}

) } ### Diff [​](https://ast-grep.github.io/catalog/tsx/#diff-3) tsx const Component = () => { const [count, setCount] = useState(0) const increment = useCallback(() => { setCount((prevCount) => prevCount + 1) }, []) const increment = () => { setCount((prevCount) => prevCount + 1) } const expensiveCalculation = useMemo(() => { // mock Expensive calculation return count * 2 }, [count]) const expensiveCalculation = (() => { // mock Expensive calculation return count * 2 })() return ( <>

Expensive Result: {expensiveCalculation}

) } ### Contributed by [​](https://ast-grep.github.io/catalog/tsx/#contributed-by-4) Inspired by [Aiden Bai](https://twitter.com/aidenybai) Avoid nested links [​](https://ast-grep.github.io/catalog/tsx/#avoid-nested-links) ----------------------------------------------------------------------------------- * [Playground Link](https://ast-grep.github.io/playground.html#eyJtb2RlIjoiQ29uZmlnIiwibGFuZyI6InRzeCIsInF1ZXJ5IjoiaWYgKCRBKSB7ICQkJEIgfSIsInJld3JpdGUiOiJpZiAoISgkQSkpIHtcbiAgICByZXR1cm47XG59XG4kJCRCIiwic3RyaWN0bmVzcyI6InNtYXJ0Iiwic2VsZWN0b3IiOiIiLCJjb25maWciOiJpZDogbm8tbmVzdGVkLWxpbmtzXG5sYW5ndWFnZTogdHN4XG5zZXZlcml0eTogZXJyb3JcbnJ1bGU6XG4gIHBhdHRlcm46IDxhICQkJD4kJCRBPC9hPlxuICBoYXM6XG4gICAgcGF0dGVybjogPGEgJCQkPiQkJDwvYT5cbiAgICBzdG9wQnk6IGVuZCIsInNvdXJjZSI6ImZ1bmN0aW9uIENvbXBvbmVudCgpIHtcbiAgcmV0dXJuIDxhIGhyZWY9Jy9kZXN0aW5hdGlvbic+XG4gICAgPGEgaHJlZj0nL2Fub3RoZXJkZXN0aW5hdGlvbic+TmVzdGVkIGxpbmshPC9hPlxuICA8L2E+O1xufVxuZnVuY3Rpb24gT2theUNvbXBvbmVudCgpIHtcbiAgcmV0dXJuIDxhIGhyZWY9Jy9kZXN0aW5hdGlvbic+XG4gICAgSSBhbSBqdXN0IGEgbGluay5cbiAgPC9hPjtcbn0ifQ==) ### Description [​](https://ast-grep.github.io/catalog/tsx/#description-5) React will produce a warning message if you nest a link element inside of another link element. This rule will catch this mistake! ### YAML [​](https://ast-grep.github.io/catalog/tsx/#yaml-4) yaml id: no-nested-links language: tsx severity: error rule: pattern: $$$A has: pattern: $$$ stopBy: end ### Example [​](https://ast-grep.github.io/catalog/tsx/#example-5) tsx function Component() { return Nested link! ; } function OkayComponent() { return I am just a link. ; } ### Contributed by [​](https://ast-grep.github.io/catalog/tsx/#contributed-by-5) [Tom MacWright](https://macwright.com/) Rename SVG Attribute Has Fix [​](https://ast-grep.github.io/catalog/tsx/#rename-svg-attribute) ----------------------------------------------------------------------------------------------- * [Playground Link](https://ast-grep.github.io/playground.html#eyJtb2RlIjoiQ29uZmlnIiwibGFuZyI6InRzeCIsInF1ZXJ5IjoiIiwicmV3cml0ZSI6IiIsInN0cmljdG5lc3MiOiJyZWxheGVkIiwic2VsZWN0b3IiOiIiLCJjb25maWciOiJpZDogcmV3cml0ZS1zdmctYXR0cmlidXRlXG5sYW5ndWFnZTogdHN4XG5ydWxlOlxuICBwYXR0ZXJuOiAkUFJPUFxuICByZWdleDogKFthLXpdKyktKFthLXpdKVxuICBraW5kOiBwcm9wZXJ0eV9pZGVudGlmaWVyXG4gIGluc2lkZTpcbiAgICBraW5kOiBqc3hfYXR0cmlidXRlXG50cmFuc2Zvcm06XG4gIE5FV19QUk9QOlxuICAgIGNvbnZlcnQ6XG4gICAgICBzb3VyY2U6ICRQUk9QXG4gICAgICB0b0Nhc2U6IGNhbWVsQ2FzZVxuZml4OiAkTkVXX1BST1AiLCJzb3VyY2UiOiJjb25zdCBlbGVtZW50ID0gKFxuICA8c3ZnIHdpZHRoPVwiMTAwXCIgaGVpZ2h0PVwiMTAwXCIgdmlld0JveD1cIjAgMCAxMDAgMTAwXCI+XG4gICAgPHBhdGggZD1cIk0xMCAyMCBMMzAgNDBcIiBzdHJva2UtbGluZWNhcD1cInJvdW5kXCIgZmlsbC1vcGFjaXR5PVwiMC41XCIgLz5cbiAgPC9zdmc+XG4pIn0=) ### Description [​](https://ast-grep.github.io/catalog/tsx/#description-6) [SVG](https://en.wikipedia.org/wiki/SVG) (Scalable Vector Graphics)s' hyphenated names are not compatible with JSX syntax in React. JSX requires [camelCase naming](https://react.dev/learn/writing-markup-with-jsx#3-camelcase-salls-most-of-the-things) for attributes. For example, an SVG attribute like `stroke-linecap` needs to be renamed to `strokeLinecap` to work correctly in React. ### YAML [​](https://ast-grep.github.io/catalog/tsx/#yaml-5) yaml id: rewrite-svg-attribute language: tsx rule: pattern: $PROP # capture in metavar regex: ([a-z]+)-([a-z]) # hyphenated name kind: property_identifier inside: kind: jsx_attribute # in JSX attribute transform: NEW_PROP: # new property name convert: # use ast-grep's convert source: $PROP toCase: camelCase # to camelCase naming fix: $NEW_PROP ### Example [​](https://ast-grep.github.io/catalog/tsx/#example-6) tsx const element = ( ) ### Diff [​](https://ast-grep.github.io/catalog/tsx/#diff-4) ts const element = ( ) ### Contributed by [​](https://ast-grep.github.io/catalog/tsx/#contributed-by-6) Inspired by [SVG Renamer](https://admondtamang.medium.com/introducing-svg-renamer-your-solution-for-react-svg-attributes-26503382d5a8) --- # Config Cheat Sheet | ast-grep [Skip to content](https://ast-grep.github.io/cheatsheet/yaml.html#VPContent) Return to top Are you an LLM? You can read better optimized documentation at /cheatsheet/yaml.md for this page in Markdown format Config Cheat Sheet [​](https://ast-grep.github.io/cheatsheet/yaml.html#config-cheat-sheet) =========================================================================================== This cheat sheet provides a concise overview of ast-grep's linter rule YAML configuration. It's designed as a handy reference for common usage. Basic Information [​](https://ast-grep.github.io/cheatsheet/yaml.html#basic-information) ----------------------------------------------------------------------------------------- Core details that identify and define your rule and miscellaneous keys for documentation and custom data. | | | | --- | --- | | ℹ️ Basic Information | | | | yaml

id: no-console-log | 🆔 A unique, descriptive identifier for the rule. | | yaml

language: JavaScript | 🌐 The programming language the rule applies to. | | yaml

url: 'https://doc.link/' | 🔗 A URL to the rule's documentation. | | yaml

metadata: { author: 'John Doe' } | 📓 metadata A dictionary for custom data related to the rule. | Finding [​](https://ast-grep.github.io/cheatsheet/yaml.html#finding) --------------------------------------------------------------------- Keys for specifying what code to search for. | | | | --- | --- | | 🔍 Finding Code | | | | yaml

rule:
pattern: 'console.log($$$ARGS)' | 🎯 The core `rule` to find matching AST nodes. | | yaml

constraints:
ARG: { kind: 'string' } } | ⚙️ Additional `constraints` rules to filter meta-variable matches. | | yaml

utils:
is-react:
kind: function_declaration
has: { kind: jsx_element } | 🛠️ A dictionary of reusable utility rules. Use them in `matches` to modularize your rules. | Patching [​](https://ast-grep.github.io/cheatsheet/yaml.html#patching) ----------------------------------------------------------------------- Keys for defining how to automatically fix the found code. | | | | --- | --- | | 🛠️ Patching Code | | | | yaml

transform:
NEW_VAR:
substring: {endChar: 1, source: $V} | 🎩 `transform` meta-variables before they are used in `fix`. | | yaml

transform:
NEW_VAR: substring($V, endChar=1) | 🎩 `transform` also accepts string form. | | yaml

fix: "logger.log($$$ARGS)" | 🔧 A `fix` string to auto-fix the matched code. | | yaml

fix:
template: "logger.log($$$ARGS)"
expandEnd: rule | 🔧 Fix also accepts `FixConfig` object. | | yaml

rewriters:
- id: remove-quotes
rule: { pattern: "'$A'" }
fix: "$A" | ✍️ A list of `rewriters` for complex transformations. | Linting [​](https://ast-grep.github.io/cheatsheet/yaml.html#linting) --------------------------------------------------------------------- Keys for configuring the messages and severity of reported issues. | | | | --- | --- | | 🚦 Linting | | | | yaml

severity: warning | ⚠️ The `severity` level of the linting message. | | yaml

message: "Avoid using $MATCH in production." | 💬 A concise `message` explaining the rule. Matched $VAR can be used. | | yaml

note:
Use a _logger_ instead of `console` | 📌 More detailed `note`. It supports Markdown format. | | yaml

labels:
ARG:
style: 'primary'
message: 'The argument to log' | 🎨 Customized `labels` for highlighting parts of the matched code. | | yaml

files: ['src/**/*.js'] | ✅ Glob `files` patterns to include files for the rule. | | yaml

files:
- glob: 'README.md'
caseInsensitive: true | ✅ Use object syntax for case-insensitive glob matching. | | yaml

ignores: ['test/**/*.js'] | ❌ Glob patterns to exclude files from the rule. | --- # Development Guide | ast-grep [Skip to content](https://ast-grep.github.io/contributing/development.html#VPContent) Return to top Are you an LLM? You can read better optimized documentation at /contributing/development.md for this page in Markdown format Development Guide [​](https://ast-grep.github.io/contributing/development.html#development-guide) ================================================================================================== Environment Setup [​](https://ast-grep.github.io/contributing/development.html#environment-setup) -------------------------------------------------------------------------------------------------- ast-grep is written in [Rust](https://www.rust-lang.org/) and hosted by [git](https://git-scm.com/) . You need to have rust environment installed to build ast-grep. The recommended way to install rust is via [rustup](https://rustup.rs/) . Once you have rustup installed, you can install rust by running: bash rustup install stable You also need [prek](https://github.com/j178/prek) to setup git hooks for type checking, formatting and clippy. Run prek install to set up the git hook scripts. bash prek install Optionally, you can also install [nodejs](https://github.com/Schniz/fnm) and [yarn](https://yarnpkg.com/) for napi binding development. That's it! You have setup the environment for ast-grep! Common Commands [​](https://ast-grep.github.io/contributing/development.html#common-commands) ---------------------------------------------------------------------------------------------- The below are some cargo commands common to any Rust project. bash cargo test # Run test cargo check # Run checking cargo clippy # Run clippy cargo fmt # Run formatting Below are some ast-grep specific commands. N-API Development [​](https://ast-grep.github.io/contributing/development.html#n-api-development) -------------------------------------------------------------------------------------------------- [@ast-grep/napi](https://www.npmjs.com/package/@ast-grep/napi) is the [nodejs binding](https://napi.rs/) for ast-grep. The source code of napi binding is under the `crates/napi` folder. You can refer to the [package.json](https://github.com/ast-grep/ast-grep/blob/main/crates/napi/package.json) for available commands. bash cd crates/napi yarn # Install dependencies yarn build # Build the binding yarn test # Run test Commit Conventions [​](https://ast-grep.github.io/contributing/development.html#commit-conventions) ---------------------------------------------------------------------------------------------------- ast-grep loosely follows the [commit conventions](https://www.conventionalcommits.org/en/v1.0.0/) . [optional scope]: [optional body] [optional footer(s)] To quote the conventional commits doc: > The commit contains the following structural elements, to communicate intent to the consumers of your library: > > * `fix:` a commit of the type fix patches a bug in your codebase. > * `feat:` a commit of the type feat introduces a new feature to the codebase. > * types other than `fix:` and `feat:` are allowed, for example, `build:`, `chore:`, `ci:`, `docs:`, `style:`, `refactor:`, `perf:`, and `test:`. > * `BREAKING CHANGE`: a commit that has a footer `BREAKING CHANGE:` introduces a breaking API change. A `BREAKING CHANGE` can be part of commits of any type. > * footers other than `BREAKING CHANGE: ` may be provided and follow a convention similar to git trailer format. TIP `BREAKING CHANGE` will be picked up and written in `CHANGELOG` by [`cargo xtask`](https://github.com/ast-grep/ast-grep/blob/86afc5865b42285106f232f01c0eb45708d134c3/xtask/src/main.rs#L162-L171) . Run Benchmark [​](https://ast-grep.github.io/contributing/development.html#run-benchmark) ------------------------------------------------------------------------------------------ ast-grep's Benchmark is not included in the default cargo test. You need to run the benchmark command in `benches` folder. bash cd benches cargo bench ast-grep's benchmarking suite is not well developed yet. The result may fluctuate too much. Release New Version [​](https://ast-grep.github.io/contributing/development.html#release-new-version) ------------------------------------------------------------------------------------------------------ The command below will bump version and create a git tag for ast-grep. Once pushed to GitHub, the tag will trigger [GitHub actions](https://github.com/ast-grep/ast-grep/blob/main/.github/workflows/coverage.yml) to build and publish the new version to [crates.io](https://github.com/ast-grep/ast-grep/blob/main/.github/workflows/pypi.yml) , [npm](https://github.com/ast-grep/ast-grep/blob/main/.github/workflows/napi.yml) and [PyPi](https://github.com/ast-grep/ast-grep/blob/main/.github/workflows/pypi.yml) . bash cargo xtask [version-number] See [xtask](https://github.com/ast-grep/ast-grep/blob/main/xtask/src/main.rs) file for more details. --- # Rule Cheat Sheet | ast-grep [Skip to content](https://ast-grep.github.io/cheatsheet/rule.html#VPContent) Return to top Are you an LLM? You can read better optimized documentation at /cheatsheet/rule.md for this page in Markdown format Rule Cheat Sheet [​](https://ast-grep.github.io/cheatsheet/rule.html#rule-cheat-sheet) ======================================================================================= This cheat sheet provides a concise overview of ast-grep's rule object configuration, covering Atomic, Relational, and Composite rules, along with notes on Utility rules. It's designed as a handy reference for common usage. Atomic Rules Cheat Sheet [​](https://ast-grep.github.io/cheatsheet/rule.html#atomic-rules-cheat-sheet) ------------------------------------------------------------------------------------------------------- These are your precision tools, matching individual AST nodes based on their inherent properties. | | | | --- | --- | | ⚛️ Atomic Rules | | | | yaml

pattern: console.log($ARG) | 🧩 Match a node by code structure. e.g. `console.log` call with a single `$ARG` | | yaml

pattern:
context: '{ key: value }'
selector: pair | 🧩 To parse ambiguous patterns, use `context` and specify `selector` AST to search. | | yaml

kind: if_statement | 🏷️ Match an AST node by its `kind` name | | yaml

regex: ^regex.+$ | 🔍 Matches node text content against a [Rust regular expression](https://docs.rs/regex/latest/regex/) | | yaml

nthChild: 1 | 🔢 Find a node by its **1-based index** among its _named siblings_ | | yaml

nthChild:
position: 2
reverse: true
ofRule: { kind: argument_list } | 🔢 Advanced positional control: `position`, `reverse` (count from end), or filter siblings using `ofRule` | | yaml

range:
start: { line: 0, column: 0 }
end: { line: 0, column: 13 } | 🎯 Matches a node based on its character span: 0-based, inclusive start, exclusive end | Relational Rules Cheat Sheet [​](https://ast-grep.github.io/cheatsheet/rule.html#relational-rules-cheat-sheet) --------------------------------------------------------------------------------------------------------------- These powerful rules define how nodes relate to each other structurally. Think of them as your AST GPS! | | | | --- | --- | | 🔗 Relational Rules | | | | yaml

inside:
kind: function_declaration | 🏠 Target node must appear **inside** its _parent/ancestor_ node matching the sub-rule | | yaml

has:
kind: method_definition | 🌳 Target node must **have** a _child/descendant_ node matching the sub-rule | | yaml

has:
kind: statement_block
field: body | 🌳 `field` makes `has`/`inside` match nodes by their [semantic role](https://ast-grep.github.io/advanced/core-concepts.html#kind-vs-field) | | yaml

precedes:
pattern: function $FUNC() { $$ } | ◀️ Target node must appear _before_ another node matching the sub-rule | | yaml

follows:
pattern: let x = 10; | ▶️ Target node must appear _after_ another node matching the sub-rule. | | yaml

inside:
kind: function_declaration
stopBy: end | 🏠 `stopBy` makes relational rules search all the way to the end, not just immediate neighbors. | Composite Rules Cheat Sheet [​](https://ast-grep.github.io/cheatsheet/rule.html#composite-rules-cheat-sheet) ------------------------------------------------------------------------------------------------------------- Combine multiple rules using Boolean logic. Crucially, these operations apply to a single target node! | | | | --- | --- | | 🧠 Composite Rules | | | | yaml

all:
- pattern: const $VAR = $VALUE
- has: { kind: string_literal } | ✅ Node must satisfy **ALL** the rules in the list. | | yaml

any:
- pattern: let $X = $Y
- pattern: const $X = $Y | 🧡 Node must satisfy **AT LEAST ONE** of the rules in the list. | | yaml

not:
pattern: console.log($$) | 🚫 Node must **NOT** satisfy the specified sub-rule. | | yaml

matches: is-function-call | 🔄 Matches the node if that utility rule matches it. Your gateway to modularity! | Utility Rules Cheat Sheet [​](https://ast-grep.github.io/cheatsheet/rule.html#utility-rules-cheat-sheet) --------------------------------------------------------------------------------------------------------- Define reusable rule definitions to cut down on duplication and build complex, maintainable rule sets. | | | | --- | --- | | 📦 Utility Rules | | | | yaml

rules:
- id: find-my-pattern
rule:
matches: my-local-check
utils:
my-local-check:
kind: identifier
regex: '^my' | 🏡 Defined within the `utils` field of your current config file. Only accessible within that file. | | yaml

# In utils/my-global-check.yml
id: my-global-check
language: javascript
rule:
kind: variable_declarator
has:
kind: number_literal | 🌍 Defined in separate YAML files in global `utilsDirs` folders, accessible across your entire project. | --- # Add New Language to ast-grep | ast-grep [Skip to content](https://ast-grep.github.io/contributing/add-lang.html#VPContent) Return to top Are you an LLM? You can read better optimized documentation at /contributing/add-lang.md for this page in Markdown format Add New Language to ast-grep [​](https://ast-grep.github.io/contributing/add-lang.html#add-new-language-to-ast-grep) ===================================================================================================================== Thank you for your interest in adding a new language to ast-grep! We appreciate your contribution to this project. Adding new languages will make the tool more useful and accessible to a wider range of users. However, there are some requirements and constraints that you need to consider before you start. This guide will help you understand the process and the standards of adding a new language to ast-grep. Requirements and Constraints [​](https://ast-grep.github.io/contributing/add-lang.html#requirements-and-constraints) --------------------------------------------------------------------------------------------------------------------- To keep ast-grep lightweight and fast, we have several factors to consider when adding a new language. As a rule of thumb, we want to limit the binary size of ast-grep under 10MB after zip compression. * **Popularity of the language**. While the popularity of a language does not necessarily reflect its merits, our limited size budget allows us to only support languages that are widely used and have a large user base. Online sources like [TIOBE index](https://www.tiobe.com/tiobe-index/) or [GitHub Octoverse](https://octoverse.github.com/2022/top-programming-languages) can help one to check the popularity of the language. * **Quality of the Tree-sitter grammar**. ast-grep relies on [Tree-sitter](https://tree-sitter.github.io/tree-sitter/) , a parser generator tool and a parsing library, to support different languages. The Tree-sitter grammar for the new language should be _well-written_, _up-to-date_, and _regularly maintained_. You can search [Tree-sitter on GitHub](https://github.com/search?q=tree-sitter&type=repositories) or on [crates.io](https://crates.io/search?q=tree%20sitter) . * **Size of the grammar**. The new language's grammar should not be too complicated. Otherwise it may take too much space from other languages. You can also check the current size of ast-grep in the [releases page](https://github.com/ast-grep/ast-grep/releases) . * **Availability of the grammar on crates.io**. To ease the maintenance burden, we prefer to use grammars that are published on crates.io, Rust's package registry. If your grammar is not on crates.io, you need to publish it yourself or ask the author to do so. * * * Don't worry if your language is not supported by ast-grep. You can try ast-grep's [custom language support](https://ast-grep.github.io/advanced/custom-language.html) and register your own Tree-sitter parser! If your language satisfies the requirements above, congratulations! Let's see how to add it to ast-grep. Add to ast-grep Core [​](https://ast-grep.github.io/contributing/add-lang.html#add-to-ast-grep-core) ----------------------------------------------------------------------------------------------------- ast-grep has several distinct use cases: [CLI tool](https://crates.io/crates/ast-grep) , [n-api lib](https://www.npmjs.com/package/@ast-grep/napi) and [web playground](https://ast-grep.github.io/playground.html) . Adding a language includes two steps. The first step is to add the language to ast-grep core. The core repository is multi-crate workspace hosted at [GitHub](https://github.com/ast-grep/ast-grep) . The relevant crate is [language](https://github.com/ast-grep/ast-grep/tree/main/crates/language) , which defines the supported languages and their tree-sitter grammars. We will use Ruby as an example to show how to add a new language to ast-grep core. You can see [the commit](https://github.com/ast-grep/ast-grep/commit/ffe14ceb8773c5d2b85559ff7455070e2a1a9388#diff-3590708789e9cdf7fa0421ecba544a69e9bbe8dd0915f0d9ff8344a9c899adfd) as a reference. ### Add Dependencies [​](https://ast-grep.github.io/contributing/add-lang.html#add-dependencies) 1. Add `tree-sitter-[lang]` crate as `dependencies` to the [Cargo.toml](https://github.com/ast-grep/ast-grep/blob/main/crates/language/Cargo.toml#L13) in the `language` crate. toml # Cargo.toml [dependencies] ... tree-sitter-ruby = {version = "0.20.0", optional = true } ... _Note the `optional` attribute is required here._ 2. Add the `tree-sitter-[lang]` dependency in [`builtin-parser`](https://github.com/ast-grep/ast-grep/blob/e494500fc5d6994c20fe0102aa4b93d2108827bb/crates/language/Cargo.toml#L40) list. toml # Cargo.toml [features] builtin-parser = [\ ...\ "tree-sitter-ruby", // [!code ++]\ ...\ ] The `builtin-parser` feature is used for command line tool. Web playground is not using the builtin parser so the dependency must be optional. ### Implement Parser [​](https://ast-grep.github.io/contributing/add-lang.html#implement-parser) 3. Add the parser function in [parsers.rs](https://github.com/ast-grep/ast-grep/blob/main/crates/language/src/parsers.rs) , where tree-sitter grammars are imported. rust #[cfg(feature = "builtin-parser")] mod parser_implementation { ... pub fn language_ruby() -> TSLanguage { tree_sitter_ruby::language().into() } ... } #[cfg(not(feature = "builtin-parser"))] mod parser_implementation { impl_parsers!( ... language_ruby, ... ); } Note there are two places to add, one for `#[cfg(feature = "builtin-parser")]` and the other for `#[cfg(not(feature = "builtin-parser"))]`. 4. Implement `language` trait by using macro in [lib.rs](https://github.com/ast-grep/ast-grep/commit/ffe14ceb8773c5d2b85559ff7455070e2a1a9388#diff-1f2939360f8f95434ed23b53406eac0aa8b2f404171b63c6466bbdfda728c82d) rust // lib.rs impl_lang_expando!(Ruby, language_ruby, 'µ'); There are two macros, `impl_lang_expando` or `impl_lang`, to generate necessary methods required by ast-grep [`Language`](https://github.com/ast-grep/ast-grep/blob/e494500fc5d6994c20fe0102aa4b93d2108827bb/crates/core/src/language.rs#L12) trait. You need to choose one of them to use for the new language. If the language does not allow `$` as valid identifier character and you need to customize the expando\_char, use `impl_lang_expando`. You can reference the comment [here](https://github.com/ast-grep/ast-grep/blob/e494500fc5d6994c20fe0102aa4b93d2108827bb/crates/language/src/lib.rs#L1-L8) for more information. ### Register the New Language [​](https://ast-grep.github.io/contributing/add-lang.html#register-the-new-language) 6. Add new lang in [`SupportLang`](https://github.com/ast-grep/ast-grep/blob/e494500fc5d6994c20fe0102aa4b93d2108827bb/crates/language/src/lib.rs#L119) enum. rust // lib.rs pub enum SupportLang { ... Ruby, ... } 7. Add new lang in [`execute_lang_method`](https://github.com/ast-grep/ast-grep/blob/e494500fc5d6994c20fe0102aa4b93d2108827bb/crates/language/src/lib.rs#L229C14-L229C33) rust // lib.rs macro_rules! execute_lang_method { ($me: path, $method: ident, $($pname:tt),*) => { use SupportLang as S; match $me { ... S::Ruby => Ruby.$method($($pname,)*), } } } 7. Add new lang in [`all_langs`](https://github.com/ast-grep/ast-grep/blob/be10ff97d6d5adad4b524961d82e40ca76ab4259/crates/language/src/lib.rs#L143) , [`alias`](https://github.com/ast-grep/ast-grep/blob/be10ff97d6d5adad4b524961d82e40ca76ab4259/crates/language/src/lib.rs#L188) , [`extension`](https://github.com/ast-grep/ast-grep/blob/be10ff97d6d5adad4b524961d82e40ca76ab4259/crates/language/src/lib.rs#L281) and [`file_types`](https://github.com/ast-grep/ast-grep/blob/be10ff97d6d5adad4b524961d82e40ca76ab4259/crates/language/src/lib.rs#L331) See this [commit](https://github.com/ast-grep/ast-grep/commit/ffe14ceb8773c5d2b85559ff7455070e2a1a9388#diff-1f2939360f8f95434ed23b53406eac0aa8b2f404171b63c6466bbdfda728c82d) for the detailed code change. Find existing languages as reference The rule of thumb to add a new language is to find a reference language that is already included in the language crate. Then add your new language by searching and following the existing language. Add to ast-grep Playground [​](https://ast-grep.github.io/contributing/add-lang.html#add-to-ast-grep-playground) ----------------------------------------------------------------------------------------------------------------- Adding new language to web playground is a little bit more complex. The playground has a standalone [repository](https://github.com/ast-grep/ast-grep.github.io) and we need to change code there. ### Prepare WASM [​](https://ast-grep.github.io/contributing/add-lang.html#prepare-wasm) 1. Set up Tree-sitter First, we need to set up Tree-sitter development tools like. You can refer to the Tree-sitter setup section in this [link](https://ast-grep.github.io/advanced/custom-language.html#prepare-tree-sitter-tool-and-parser) . 2. Build WASM file Then, in your parser repository, use this command to build a WASM file. bash tree-sitter generate # if grammar is not generated before tree-sitter build --wasm Note you may need to install [docker](https://www.docker.com/) when building WASM files. 3. Move WASM file to the website [`public`](https://github.com/ast-grep/ast-grep.github.io/tree/main/website/public) folder. You can also see other languages' WASM files in the public directory. The file name is in the format of `tree-sitter-[lang].wasm`. The name will be used later in [`parserPaths`](https://github.com/ast-grep/ast-grep.github.io/blob/a2dce64dda67e1c0842b757fc692ffe05639e407/website/src/components/lang.ts#L4) . ### Add language in Rust [​](https://ast-grep.github.io/contributing/add-lang.html#add-language-in-rust) You need to add the language in the [wasm\_lang.rs](https://github.com/ast-grep/ast-grep.github.io/blob/main/src/wasm_lang.rs) . More specifically, you need to add a new enum variant in [`WasmLang`](https://github.com/ast-grep/ast-grep.github.io/blob/a2dce64dda67e1c0842b757fc692ffe05639e407/src/wasm_lang.rs#L16) , handle the new variant in [`execute_lang_method`](https://github.com/ast-grep/ast-grep.github.io/blob/a2dce64dda67e1c0842b757fc692ffe05639e407/src/wasm_lang.rs#L111) and implement [`FromStr`](https://github.com/ast-grep/ast-grep.github.io/blob/a2dce64dda67e1c0842b757fc692ffe05639e407/src/wasm_lang.rs#L48) . rust // new variant pub enum WasmLang { // ... Swift, } // handle variant in macro macro_rules! execute_lang_method { ($me: path, $method: ident, $($pname:tt),*) => { use WasmLang as W; match $me { W::Swift => L::Swift.$method($($pname,)*), } } } // impl FromStr impl FromStr for WasmLang { // ... fn from_str(s: &str) -> Result { Ok(match s { "swift" => Swift, }) } } ### Add language in TypeScript [​](https://ast-grep.github.io/contributing/add-lang.html#add-language-in-typescript) Finally you need to add the language in TypeScript to make it available in playground. The file is [lang.ts](https://github.com/ast-grep/ast-grep.github.io/blob/main/website/src/components/lang.ts) . There are two changes need to make. typescript // Add language parserPaths const parserPaths = { // ... swift: 'tree-sitter-swift.wasm', } // Add language display name export const languageDisplayNames: Record = { // ... swift: 'Swift', } You can see Swift's support as the [reference commit](https://github.com/ast-grep/ast-grep.github.io/commit/55a546535dee989ce5ee2582080e771d006d165e) . --- # Project Configuration | ast-grep [Skip to content](https://ast-grep.github.io/guide/project/project-config.html#VPContent) Return to top Are you an LLM? You can read better optimized documentation at /guide/project/project-config.md for this page in Markdown format Project Configuration [​](https://ast-grep.github.io/guide/project/project-config.html#project-configuration) ============================================================================================================== Root Configuration File [​](https://ast-grep.github.io/guide/project/project-config.html#root-configuration-file) ------------------------------------------------------------------------------------------------------------------ ast-grep supports using [YAML](https://yaml.org/) to configure its linting rules to scan your code repository. We need a root configuration file `sgconfig.yml` to specify directories where `ast-grep` can find all rules. In your project root, add `sgconfig.yml` with content as below. yaml ruleDirs: - rules This instructs ast-grep to use all files _recursively_ inside the `rules` folder as rule files. For example, suppose we have the following file structures. my-awesome-project |- rules | |- no-var.yml | |- no-bit-operation.yml | |- my_custom_rules | |- custom-rule.yml | |- fancy-rule.yml |- sgconfig.yml |- not-a-rule.yml All the YAML files under `rules` folder will be treated as rule files by `ast-grep`, while`not-a-rule.yml` is ignored. **Note, the [`ast-grep scan`](https://ast-grep.github.io/reference/cli.html#scan) command requires you have an `sgconfig.yml` in your project root.** Pro tip We can also use directories in `node_modules` to reuse preconfigured rules published on npm! More broadly speaking, any git hosted projects can be imported as rule sets by using [`git submodule`](https://www.git-scm.com/book/en/v2/Git-Tools-Submodules) . Project Discovery [​](https://ast-grep.github.io/guide/project/project-config.html#project-discovery) ------------------------------------------------------------------------------------------------------ ast-grep will try to find the `sgconfig.yml` file in the current working directory. If it is not found, it will traverse up the directory tree until it finds one. You can also specify the path to the configuration file using the `--config` option. bash ast-grep scan --config path/to/config.yml Global Configuration You can put an `sgconfig.yml` in your home directory to set global configurations for `ast-grep`. XDG configuration directory is **NOT** supported yet. Project file discovery and `--config` option are also effective in the `ast-grep run` command. So you can use configurations like [custom languages](https://ast-grep.github.io/reference/sgconfig.html#customlanguages) and [language globs](https://ast-grep.github.io/reference/sgconfig.html#languageglobs) . Note that `run` command does not require a `sgconfig.yml` file and will stil search code without it, but `scan` command will report an error if project config is not found. Project Inspection [​](https://ast-grep.github.io/guide/project/project-config.html#project-inspection) -------------------------------------------------------------------------------------------------------- You can use the [`--inspect summary`](https://ast-grep.github.io/reference/cli/scan.html#inspect-granularity) flag to see the project directory ast-grep is using. bash ast-grep scan --inspect summary It will print the project directory and the configuration file path. bash sg: summary|project: isProject=true,projectDir=/path/to/project Output format can be found in the [GitHub issue](https://github.com/ast-grep/ast-grep/issues/1574) . --- # Performance Tip for napi usage | ast-grep [Skip to content](https://ast-grep.github.io/guide/api-usage/performance-tip.html#VPContent) Return to top Are you an LLM? You can read better optimized documentation at /guide/api-usage/performance-tip.md for this page in Markdown format Performance Tip for napi usage [​](https://ast-grep.github.io/guide/api-usage/performance-tip.html#performance-tip-for-napi-usage) =================================================================================================================================== Using `napi` to parse code and search for nodes [isn't always faster](https://medium.com/@hchan_nvim/benchmark-typescript-parsers-demystify-rust-tooling-performance-025ebfd391a3) than pure JavaScript implementations. There are a lot of tricks to improve performance when using `napi`. The mantra is to _reduce FFI (Foreign Function Interface) calls between Rust and JavaScript_, and to _take advantage of parallel computing_. Prefer `parseAsync` over `parse` [​](https://ast-grep.github.io/guide/api-usage/performance-tip.html#prefer-parseasync-over-parse) ----------------------------------------------------------------------------------------------------------------------------------- `parseAsync` can take advantage of NodeJs' libuv thread pool to parse code in parallel threads. This can be faster than the sync version `parse` when handling a lot of code. ts import { Lang, parse, parseAsync } from '@ast-grep/napi' // only one thread parsing const ast = parse(Lang.JavaScript, 'console.log("hello world")') const root = ast.root() // better, can use multiple threads const ast2 = await parseAsync(Lang.JavaScript, 'console.log("hello world")') const root2 = ast2.root() This is especially useful when you are using ast-grep in bundlers where the main thread is busy with other CPU intensive tasks. Prefer `findAll` over manual traversal [​](https://ast-grep.github.io/guide/api-usage/performance-tip.html#prefer-findall-over-manual-traversal) ------------------------------------------------------------------------------------------------------------------------------------------------- One way to find all nodes that match a rule is to traverse the syntax tree manually and check each node against the rule. This is slow because it requires a lot of FFI calls between Rust and JavaScript during the traversal. For example, the following code snippet finds all `member_expression` nodes in the syntax tree. Unfortunately, there are as many FFI calls as the tree node number in the recursion. ts const root = sgroot.root() function findMemberExpression(node: SgNode): SgNode[] { let ret: SgNode[] = [] // `node.kind()` is a FFI call if (node.kind() === 'member_expression') { ret.push(node) } // `node.children()` is a FFI call for (let child of node.children()) { // recursion makes more FFI calls ret = ret.concat(findMemberExpression(child)) } return ret } const nodes = findMemberExpression(root) The equivalent code using `findAll` is much faster: ts const root = sgroot.root() // only call FFI `findAll` once const nodes = root.findAll({kind: 'member_expression'}) > _One [success](https://x.com/hd_nvim/status/1767971906786128316) > [story](https://x.com/sonofmagic95/status/1768433654404104555) > on Twitter, as an example._ Prefer `findInFiles` when possible [​](https://ast-grep.github.io/guide/api-usage/performance-tip.html#prefer-findinfiles-when-possible) ----------------------------------------------------------------------------------------------------------------------------------------- If you have a lot of files to parse and want to maximize your programs' performance, ast-grep provides a `findInFiles` function that parses multiple files and searches relevant nodes in parallel Rust threads. APIs we showed above all require parsing code in Rust and pass the `SgRoot` back to JavaScript. This incurs foreign function communication overhead and only utilizes the single main JavaScript thread. By avoiding Rust-JS communication overhead and utilizing multiple core computing, `findInFiles` is much faster than finding files in JavaScript and then passing them to Rust as string. The function signature of `findInFiles` is as follows: ts export function findInFiles( /** the language to parse */ lang: Lang, /** specify the file path and matcher */ config: FindConfig, /** callback function for found nodes in a file */ callback: (err: null | Error, result: SgNode[]) => void ): Promise `findInFiles` accepts a `FindConfig` object and a callback function. `FindConfig` specifies both what file path to _parse_ and what nodes to _search_. `findInFiles` will parse all files matching paths and will call back the function with nodes matching the `matcher` found in the files as arguments. ### `FindConfig` [​](https://ast-grep.github.io/guide/api-usage/performance-tip.html#findconfig) The `FindConfig` object specifies which paths to search code and what rule to match node against. The `FindConfig` object has the following type: ts export interface FindConfig { paths: Array matcher: NapiConfig } The `path` field is an array of strings. You can specify multiple paths to search code. Every path in the array can be a file path or a directory path. For a directory path, ast-grep will recursively find all files matching the language. The `matcher` is the same as `NapiConfig` stated above. ### Callback Function and Termination [​](https://ast-grep.github.io/guide/api-usage/performance-tip.html#callback-function-and-termination) The `callback` function is called for every file that have nodes that match the rule. The callback function is a standard node-style callback with the first argument as `Error` and second argument as an array of `SgNode` objects that match the rule. The return value of `findInFiles` is a `Promise` object. The promise resolves to the number of files that have nodes that match the rule. DANGER `findInFiles` can return before all file callbacks are called due to NodeJS limitation. See [https://github.com/ast-grep/ast-grep/issues/206](https://github.com/ast-grep/ast-grep/issues/206) . If you have a lot of files and `findInFiles` prematurely returns, you can use the total files returned by `findInFiles` as a check point. Maintain a counter outside of `findInFiles` and increment it in callback. If the counter equals the total number, we can conclude all files are processed. The following code is an example, with core logic highlighted. ts type Callback = (t: any, cb: any) => Promise function countedPromise(func: F) { type P = Parameters return async (t: P[0], cb: P[1]) => { let i = 0 let fileCount: number | undefined = undefined // resolve will be called after all files are processed let resolve = () => {} function wrapped(...args: any[]) { let ret = cb(...args) if (++i === fileCount) resolve() return ret } fileCount = await func(t, wrapped as P[1]) // not all files are processed, await `resolve` to be called if (fileCount > i) { await new Promise(r => resolve = r) } return fileCount } } 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 ### Example [​](https://ast-grep.github.io/guide/api-usage/performance-tip.html#example) Example of using `findInFiles` ts import { Lang, findInFiles } from '@ast-grep/napi' let fileCount = await findInFiles(Lang.JavaScript, { paths: ['relative/path/to/code'], matcher: { rule: {kind: 'member_expression'} }, }, (err, n) => { if (err) { console.error(err) return } console.log(`Found ${n.length} nodes`) console.log(n[0].text()) }) --- # Pattern Syntax | ast-grep [Skip to content](https://ast-grep.github.io/guide/pattern-syntax.html#VPContent) Return to top Are you an LLM? You can read better optimized documentation at /guide/pattern-syntax.md for this page in Markdown format Pattern Syntax [​](https://ast-grep.github.io/guide/pattern-syntax.html#pattern-syntax) ======================================================================================== In this guide we will walk through ast-grep's pattern syntax. The example will be written in JavaScript, but the basic principle will apply to other languages as well. Pattern Matching [​](https://ast-grep.github.io/guide/pattern-syntax.html#pattern-matching) -------------------------------------------------------------------------------------------- ast-grep uses pattern code to construct AST tree and match that against target code. The pattern code can search through the full syntax tree, so pattern can also match nested expression. For example, the pattern `a + 1` can match all the following code. javascript const b = a + 1 funcCall(a + 1) deeplyNested({ target: a + 1 }) WARNING Pattern code must be valid code that tree-sitter can parse. [ast-grep playground](https://ast-grep.github.io/playground.html) is a useful tool to confirm pattern is parsed correctly. If ast-grep fails to parse code as expected, you can try give it more context by using [object-style pattern](https://ast-grep.github.io/reference/rule.html#pattern) . Meta Variable [​](https://ast-grep.github.io/guide/pattern-syntax.html#meta-variable) -------------------------------------------------------------------------------------- It is usually desirable to write a pattern to match dynamic content. We can use meta variables to match sub expression in pattern. Meta variables start with the `$` sign, followed by a name composed of upper case letters `A-Z`, underscore `_` or digits `1-9`. `$META_VARIABLE` is a wildcard expression that can match any **single** AST node. Think it as REGEX dot `.`, except it is not textual. Valid meta variables `$META`, `$META_VAR`, `$META_VAR1`, `$_`, `$_123` Invalid meta variables `$invalid`, `$Svalue`, `$123`, `$KEBAB-CASE`， `$` The pattern `console.log($GREETING)` will match all the following. javascript function tryAstGrep() { console.log('Hello World') } const multiLineExpression = console .log('Also matched!') But it will not match these. javascript // console.log(123) in comment is not matched 'console.log(123) in string' // is not matched as well console.log() // mismatch argument console.log(a, b) // too many arguments Note, one meta variable `$MATCH` will match one **single** AST node, so the last two `console.log` calls do not match the pattern. Let's see how we can match multiple AST nodes. Multi Meta Variable [​](https://ast-grep.github.io/guide/pattern-syntax.html#multi-meta-variable) -------------------------------------------------------------------------------------------------- We can use `$$$` to match zero or more AST nodes, including function arguments, parameters or statements. These variables can also be named, for example: `console.log($$$ARGS)`. ### Function Arguments [​](https://ast-grep.github.io/guide/pattern-syntax.html#function-arguments) For example, `console.log($$$)` can match javascript console.log() // matches zero AST node console.log('hello world') // matches one node console.log('debug: ', key, value) // matches multiple nodes console.log(...args) // it also matches spread ### Function Parameters [​](https://ast-grep.github.io/guide/pattern-syntax.html#function-parameters) `function $FUNC($$$ARGS) { $$$ }` will match javascript function foo(bar) { return bar } function noop() {} function add(a, b, c) { return a + b + c } `ARGS` will be populated with a list of AST nodes. Click to see details. | Code | Match | | --- | --- | | `function foo(bar) { ... }` | \[`bar`\] | | `function noop() {}` | \[\] | | `function add(a, b, c) { ... }` | \[`a`, `b`, `c`\] | Meta Variable Capturing [​](https://ast-grep.github.io/guide/pattern-syntax.html#meta-variable-capturing) ---------------------------------------------------------------------------------------------------------- Meta variable is also similar to [capture group](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Guide/Regular_Expressions/Groups_and_Backreferences) in regular expression. You can reuse same name meta variables to find previously occurred AST nodes. For example, the pattern `$A == $A` will have the following result. javascript // will match these patterns a == a 1 + 1 == 1 + 1 // but will not match these a == b 1 + 1 == 2 ### Non Capturing Match [​](https://ast-grep.github.io/guide/pattern-syntax.html#non-capturing-match) You can also suppress meta variable capturing. All meta variables with name starting with underscore `_` will not be captured. javascript // Given this pattern $_FUNC($_FUNC) // it will match all function call with one argument or spread call test(a) testFunc(1 + 1) testFunc(...args) Note in the example above, even if two meta variables have the same name `$_FUNC`, each occurrence of `$_FUNC` can match different content because they are not captured. Why use non-capturing match? This is a useful trick to micro-optimize pattern matching speed, since we don't need to create a [HashMap](https://doc.rust-lang.org/stable/std/collections/struct.HashMap.html) for bookkeeping. ### Capture Unnamed Nodes [​](https://ast-grep.github.io/guide/pattern-syntax.html#capture-unnamed-nodes) A meta variable pattern `$META` will capture [named nodes](https://ast-grep.github.io/advanced/core-concepts.html#named-vs-unnamed) by default. To capture [unnamed nodes](https://ast-grep.github.io/advanced/core-concepts.html#named-vs-unnamed) , you can use double dollar sign `$$VAR`. Namedness is an advanced topic in [Tree-sitter](https://tree-sitter.github.io/tree-sitter/using-parsers#named-vs-anonymous-nodes) . You can read this [in-depth guide](https://ast-grep.github.io/advanced/core-concepts.html) for more background. More Powerful Rule [​](https://ast-grep.github.io/guide/pattern-syntax.html#more-powerful-rule) ------------------------------------------------------------------------------------------------ Pattern is a fast and easy way to match code. But it is not as powerful as [rule](https://ast-grep.github.io/guide/rule-config.html#rule-file) which can match code with more [precise selector](https://ast-grep.github.io/guide/rule-config/atomic-rule.html#kind) or [more context](https://ast-grep.github.io/guide/rule-config/relational-rule.html) . We will cover using rules in next chapter. Pro Tip Pattern can also be an object instead of string in YAML rule. It is very useful to avoid ambiguity in code snippet. See [here](https://ast-grep.github.io/guide/rule-config/atomic-rule.html#pattern) for more details. Also see our FAQ for more [guidance](https://ast-grep.github.io/advanced/faq.html) on writing patterns. --- # Contributing | ast-grep [Skip to content](https://ast-grep.github.io/contributing/how-to.html#VPContent) Return to top Are you an LLM? You can read better optimized documentation at /contributing/how-to.md for this page in Markdown format Contributing [​](https://ast-grep.github.io/contributing/how-to.html#contributing) =================================================================================== 🎉 _**We are thrilled that you are interested in contributing to the ast-grep project!**_ 🎉 Your help and support are very valuable for us. There are many ways you can help improve the project and make it more useful for everyone. Let's see some of the things we can do together: Spreading Your Words ❤️ [​](https://ast-grep.github.io/contributing/how-to.html#spreading-your-words-%E2%9D%A4%EF%B8%8F) ------------------------------------------------------------------------------------------------------------------------- We appreciate your kind words and support for the project. You can help us grow the ast-grep community and reach more potential users by spreading your kind words. Here are some of the things we can do: * **Who is using ast-grep**: Let us know who is using ast-grep by adding your name or organization to the [users page](https://github.com/ast-grep/ast-grep/issues/373) on the documentation website. Feel free to add a logo or a testimonial if you like. * **Tweet it!**: Tweet about ast-grep using the hashtag [#ast\_grep](https://twitter.com/hashtag/ast_grep) . Share your feedback, your use cases, your tips and tricks, or your questions and suggestions with the ast-grep community on Twitter. * **Sharing Podcast**: Talk about ast-grep on podcasts or other audio platforms. Introduce ast-grep to new audiences, share your stories and insights, or invite other guests to discuss ast-grep with you. * **Meetup**: Attend meetups or events where you can talk about ast-grep. Meet other ast-grep users or developers, exchange ideas and experiences, learn from each other, or collaborate on projects. Giving Feedback [​](https://ast-grep.github.io/contributing/how-to.html#giving-feedback) ----------------------------------------------------------------------------------------- We appreciate your feedback on the project. Whether you have a feature request, a bug report, or a general comment, we would love to hear from you. You can use the following channels to provide your feedback: * **Feature Request**: If you have an idea for a new feature or an enhancement for an existing feature, please create an issue on the [main repo](https://github.com/ast-grep/ast-grep/issues/new?assignees=&labels=enhancement&projects=&template=feature_request.md&title=%5Bfeature%5D) with the label `enhancement`. Please describe your idea with examples and explain why it would be useful for the project and the users. * **Bug Report**: If you encounter a bug or an error while using ast-grep, please create an issue on the [main repo](https://github.com/ast-grep/ast-grep/issues/new?assignees=&labels=enhancement&projects=&template=feature_request.md&title=%5Bfeature%5D) with the label `bug`. Please provide as much information as possible to help us reproduce and fix the bug, such as the version of ast-grep, the command or query you used, the expected and actual results, any error messages or screenshots, and preferably a [playground link](https://ast-grep.github.io/playground.html) reproducing the issue. Contributing Code [​](https://ast-grep.github.io/contributing/how-to.html#contributing-code) --------------------------------------------------------------------------------------------- We welcome your code contributions to the project. Whether you want to fix a bug, implement a feature, improve the documentation, or add a new integration, we are grateful for your help. You can use the following repositories to contribute your code: * **CLI Main Repo**: The [main repository for ast-grep](https://github.com/ast-grep/ast-grep) command-line interface (CLI). It contains the core logic and functionality of ast-grep. For small features or typo fixes, you can fork this repository and submit pull requests with your changes. [This guide](https://ast-grep.github.io/contributing/development.html) may help you set up essential tools for development. _For larger features or big changes, please make an issue for discussion before jumping into it._ * **Doc Website**: This is the repository for the ast-grep documentation website. It contains the source files for generating the website using [vitepress](https://vitepress.dev/) . You can fork this repository and submit pull requests with your changes. * **CI/CD Integration**: ast-grep has a [repository for GitHub Action](https://github.com/ast-grep/action) . It allows you to use ast-grep as part of your continuous integration and continuous delivery (CI/CD) workflows on GitHub. You can check this repository and suggest useful features missing now. * **Editor Integration**: These are the repositories for various editor integrations of ast-grep. They allow you to use ast-grep within your favorite editor, such as VS Code, Vim, or Neovim. Please follow the respective guides for each editor integration before submitting your pull requests. * VS Code extension: [ast-grep-vscode](https://github.com/ast-grep/ast-grep-vscode) * NeoVim LSP: [coc-ast-grep](https://github.com/yaegassy/coc-ast-grep) made by [@yaegassy](https://twitter.com/yaegassy) * NeoVim Telescope plugin: [telescope-sg](https://github.com/Marskey/telescope-sg) made by [@Marskey](https://github.com/Marskey) Sharing Knowledge [​](https://ast-grep.github.io/contributing/how-to.html#sharing-knowledge) --------------------------------------------------------------------------------------------- We encourage you to share your knowledge and experience with ast-grep with others. You can help us spread the word about ast-grep and educate more people about its benefits and features. Here are some of the things we can do: * **Write introductions to ast-grep**: You can write blog posts, articles, or tutorials that introduce ast-grep to new users. You can explain what ast-grep is, how it works, what problems it solves, and how to install and use it. You can also share some examples of how you use ast-grep in your own projects or workflows. * **Answer questions about ast-grep**: Help answering people's questions on [StackOverflow](https://stackoverflow.com/questions/tagged/ast-grep) or [Discord](https://discord.gg/4YZjf6htSQ) . Your answers will be appreciated! * **Write ast-grep's tutorial**: You can write more advanced tutorials that show how to use ast-grep for specific tasks or scenarios. You can demonstrate how to use ast-grep's features and options, how to write complex queries and transformations, how to integrate ast-grep with other tools or platforms, and how to optimize ast-grep's performance and efficiency. * **Translate documentation**: You can help us make ast-grep more accessible to users from different regions and languages by translating its documentation into other languages. Reach out [@Shenqingchuan](https://twitter.com/Shenqingchuan) , translation team member of [Rollup](https://github.com/rollup/rollup-docs-cn) , [Vite](https://github.com/vitejs/docs-cn) and ast-grep, for more ideas about translation! * **Curate a rule collections**: Using ast-grep as linter in your project can showcase the power and versatility of ast-grep! Linting open source projects shows how ast-grep can be used for various purposes and domains. [ast-grep/eslint](https://github.com/ast-grep/eslint) , for example, is a collection of eslint rule implemented in ast-grep YAML. * **Sharing Rules**: Sharing your rules on ast-grep's [example catalog](https://ast-grep.github.io/catalog/index.html) can inspire more people to harness the power of AST! Example catalog is a place where users can browse, search, and submit rules. You can use [the template](https://github.com/ast-grep/ast-grep.github.io/blob/main/website/catalog/rule-template.md) to add your example [here](https://github.com/ast-grep/ast-grep.github.io/tree/main/website/catalog) . Thank you for your interest in contributing to the ast-grep project. We are grateful for your help and support. We hope you enjoy using and improving ast-grep as much as we do. If you have any questions or issues, please feel free to contact us on [GitHub](https://github.com/ast-grep/ast-grep) or [Discord](https://discord.gg/4YZjf6htSQ) . We look forward to hearing from you soon! 😊 * * * You don’t have to contribute code A common misconception about contributing to open source is that you need to contribute code. In fact, it’s often the other parts of a project that are most neglected or overlooked. You’ll do the project a huge favor by offering to pitch in with these types of contributions! _[GitHub Open Source Guide](https://opensource.guide/) _ --- # Quick Start | ast-grep [Skip to content](https://ast-grep.github.io/guide/quick-start.html#VPContent) Return to top Are you an LLM? You can read better optimized documentation at /guide/quick-start.md for this page in Markdown format Quick Start [​](https://ast-grep.github.io/guide/quick-start.html#quick-start) =============================================================================== You can unleash `ast-grep`'s power at your fingertips in a few keystrokes on the command line! Let's see it in action by rewriting code in a moderately large codebase: [TypeScript](https://github.com/microsoft/TypeScript/) . Our task is to rewrite old defensive code that checks nullable nested method calls to use the shiny new [optional chaining operator](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Operators/Optional_chaining) `?.`. Installation [​](https://ast-grep.github.io/guide/quick-start.html#installation) --------------------------------------------------------------------------------- First, install `ast-grep`. It is distributed by [npm](https://www.npmjs.com/package/@ast-grep/cli) , [cargo](https://crates.io/crates/ast-grep) , [homebrew](https://formulae.brew.sh/formula/ast-grep) and [macports](https://ports.macports.org/port/ast-grep/) . You can also build it [from source](https://github.com/ast-grep/ast-grep#installation) . homebrewmacportsnix-shellcargonpmpip shell # install via homebrew brew install ast-grep shell # install via MacPorts sudo port install ast-grep shell # try ast-grep in nix-shell nix-shell -p ast-grep shell # install via cargo cargo install ast-grep --locked shell # install via npm npm i @ast-grep/cli -g # note, for pnpm, you may need manually approve postinstall script # pnpm approve-builds shell # install via pip pip install ast-grep-cli The binary command, `ast-grep` or `sg`, should be available now. Let's try it with `--help`. shell ast-grep --help # if you are not on Linux sg --help Use `sg` on Linux Linux has a default command `sg` for `setgroups`. You can use the full command name `ast-grep` instead of `sg`. You can also use shorter alias if you want by `alias sg=ast-grep`. We will use `ast-grep` in the guide below. Optionally, you can grab TypeScript source code if you want to follow the tutorial. Or you can apply the magic to your own code. shell git clone git@github.com:microsoft/TypeScript.git --depth 1 Pattern [​](https://ast-grep.github.io/guide/quick-start.html#pattern) ----------------------------------------------------------------------- Let's search for instances of calling a method on a nested property. `ast-grep` uses **patterns** to find similar code. Think of patterns like those in our old friend `grep`, but instead of text, they match AST nodes. We can write patterns like we write ordinary code, and it will match all code that has the same syntactical structure. For example, the following pattern code javascript obj.val && obj.val() will match all the following code, regardless of white spaces or new lines. javascript obj.val && obj.val() // verbatim match, of course obj.val && obj.val() // this matches, too // this matches as well! const result = obj.val && obj.val() Exact AST matching is already powerful, but we can go further with **metavariables** for more flexibility. Use a **metavariable** to match any single AST node. Metavariables begin with `$` and are typically uppercase (e.g., `$PROP`). Think of it like the regex dot `.`, except it matches syntax nodes, not text. We can use the following pattern to find all property checking code. javascript $PROP && $PROP() This is a valid `ast-grep` pattern you can run from the command line. The `--pattern` argument specifies the target. Optionally, use `--lang` to specify the target language. Full CommandShort FormWithout Lang shell ast-grep --pattern '$PROP && $PROP()' --lang ts TypeScript/src shell ast-grep -p '$PROP && $PROP()' -l ts TypeScript/src shell # ast-grep will infer languages based on file extensions ast-grep -p '$PROP && $PROP()' TypeScript/src Pro Tip The pattern must be enclosed in single quotes `'` to prevent the shell from interpreting the `$` sign. `ast-grep -p '$PROP && $PROP()'` is okay. With double quotes, `ast-grep -p "$PROP && $PROP()"` would be interpreted as `ast-grep -p " && ()"` after shell expansion. Rewrite [​](https://ast-grep.github.io/guide/quick-start.html#rewrite) ----------------------------------------------------------------------- Cool? Now we can use this pattern to refactor the TypeScript source! shell # pattern and language argument support short form ast-grep -p '$PROP && $PROP()' \ --rewrite '$PROP?.()' \ --interactive \ -l ts \ TypeScript/src `ast-grep` will start an interactive session to let you choose if you want to apply the patch. Press `y` to accept the change! That's it! You have refactored the TypeScript source in minutes. Congratulations! We hope you enjoy the power of AST editing with plain programming-language patterns. Next, learn more about writing patterns. Pattern does not work? See our FAQ for more [guidance](https://ast-grep.github.io/advanced/faq.html) on writing patterns. --- # ast-grep [Skip to content](https://ast-grep.github.io/catalog/typescript/find-import-identifiers.html#VPContent) Return to top Are you an LLM? You can read better optimized documentation at /catalog/typescript/find-import-identifiers.md for this page in Markdown format Find Import Identifiers [​](https://ast-grep.github.io/catalog/typescript/find-import-identifiers.html#find-import-identifiers) -------------------------------------------------------------------------------------------------------------------------------- * [Playground Link](https://ast-grep.github.io/playground.html#eyJtb2RlIjoiQ29uZmlnIiwibGFuZyI6InR5cGVzY3JpcHQiLCJxdWVyeSI6ImNvbnNvbGUubG9nKCRNQVRDSCkiLCJyZXdyaXRlIjoibG9nZ2VyLmxvZygkTUFUQ0gpIiwic3RyaWN0bmVzcyI6InNtYXJ0Iiwic2VsZWN0b3IiOiIiLCJjb25maWciOiIjIGZpbmQtYWxsLWltcG9ydHMtYW5kLXJlcXVpcmVzLnlhbWxcbmlkOiBmaW5kLWFsbC1pbXBvcnRzLWFuZC1yZXF1aXJlc1xubGFuZ3VhZ2U6IFR5cGVTY3JpcHRcbm1lc3NhZ2U6IEZvdW5kIG1vZHVsZSBpbXBvcnQgb3IgcmVxdWlyZS5cbnNldmVyaXR5OiBpbmZvXG5ydWxlOlxuICBhbnk6XG4gICAgIyBBTElBUyBJTVBPUlRTXG4gICAgIyAtLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS1cbiAgICAjIGltcG9ydCB7IE9SSUdJTkFMIGFzIEFMSUFTIH0gZnJvbSAnU09VUkNFJ1xuICAgICMgLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tXG4gICAgLSBhbGw6XG4gICAgICAgICMgMS4gVGFyZ2V0IHRoZSBzcGVjaWZpYyBub2RlIHR5cGUgZm9yIG5hbWVkIGltcG9ydHNcbiAgICAgICAgLSBraW5kOiBpbXBvcnRfc3BlY2lmaWVyXG4gICAgICAgICMgMi4gRW5zdXJlIGl0ICpoYXMqIGFuICdhbGlhcycgZmllbGQsIGNhcHR1cmluZyB0aGUgYWxpYXMgaWRlbnRpZmllclxuICAgICAgICAtIGhhczpcbiAgICAgICAgICAgIGZpZWxkOiBhbGlhc1xuICAgICAgICAgICAgcGF0dGVybjogJEFMSUFTXG4gICAgICAgICMgMy4gQ2FwdHVyZSB0aGUgb3JpZ2luYWwgaWRlbnRpZmllciAod2hpY2ggaGFzIHRoZSAnbmFtZScgZmllbGQpXG4gICAgICAgIC0gaGFzOlxuICAgICAgICAgICAgZmllbGQ6IG5hbWVcbiAgICAgICAgICAgIHBhdHRlcm46ICRPUklHSU5BTFxuICAgICAgICAjIDQuIEZpbmQgYW4gQU5DRVNUT1IgaW1wb3J0X3N0YXRlbWVudCBhbmQgY2FwdHVyZSBpdHMgc291cmNlIHBhdGhcbiAgICAgICAgLSBpbnNpZGU6XG4gICAgICAgICAgICBzdG9wQnk6IGVuZCAjIDw8PC0tLSBUaGlzIGlzIHRoZSBrZXkgZml4ISBTZWFyY2ggYW5jZXN0b3JzLlxuICAgICAgICAgICAga2luZDogaW1wb3J0X3N0YXRlbWVudFxuICAgICAgICAgICAgaGFzOiAjIEVuc3VyZSB0aGUgZm91bmQgaW1wb3J0X3N0YXRlbWVudCBoYXMgdGhlIHNvdXJjZSBmaWVsZFxuICAgICAgICAgICAgICBmaWVsZDogc291cmNlXG4gICAgICAgICAgICAgIHBhdHRlcm46ICRTT1VSQ0VcblxuICAgICMgREVGQVVMVCBJTVBPUlRTXG4gICAgIyAtLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS1cbiAgICAjIGltcG9ydCB7IE9SSUdJTkFMIH0gZnJvbSAnU09VUkNFJ1xuICAgICMgLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tXG4gICAgLSBhbGw6XG4gICAgICAgIC0ga2luZDogaW1wb3J0X3N0YXRlbWVudFxuICAgICAgICAtIGhhczpcbiAgICAgICAgICAgICMgRW5zdXJlIGl0IGhhcyBhbiBpbXBvcnRfY2xhdXNlLi4uXG4gICAgICAgICAgICBraW5kOiBpbXBvcnRfY2xhdXNlXG4gICAgICAgICAgICBoYXM6XG4gICAgICAgICAgICAgICMgLi4udGhhdCBkaXJlY3RseSBjb250YWlucyBhbiBpZGVudGlmaWVyICh0aGUgZGVmYXVsdCBpbXBvcnQgbmFtZSlcbiAgICAgICAgICAgICAgIyBUaGlzIGlkZW50aWZpZXIgaXMgTk9UIHVuZGVyIGEgJ25hbWVkX2ltcG9ydHMnIG9yICduYW1lc3BhY2VfaW1wb3J0JyBub2RlXG4gICAgICAgICAgICAgIGtpbmQ6IGlkZW50aWZpZXJcbiAgICAgICAgICAgICAgcGF0dGVybjogJERFRkFVTFRfTkFNRVxuICAgICAgICAtIGhhczpcbiAgICAgICAgICAgIGZpZWxkOiBzb3VyY2VcbiAgICAgICAgICAgIHBhdHRlcm46ICRTT1VSQ0VcbiAgICBcbiAgICAjIFJFR1VMQVIgSU1QT1JUU1xuICAgICMgLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tXG4gICAgIyBpbXBvcnQgeyBPUklHSU5BTCB9IGZyb20gJ1NPVVJDRSdcbiAgICAjIC0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLVxuICAgIC0gYWxsOlxuICAgICAgICAjIDEuIFRhcmdldCB0aGUgc3BlY2lmaWMgbm9kZSB0eXBlIGZvciBuYW1lZCBpbXBvcnRzXG4gICAgICAgIC0ga2luZDogaW1wb3J0X3NwZWNpZmllclxuICAgICAgICAjIDIuIEVuc3VyZSBpdCAqaGFzKiBhbiAnYWxpYXMnIGZpZWxkLCBjYXB0dXJpbmcgdGhlIGFsaWFzIGlkZW50aWZpZXJcbiAgICAgICAgLSBoYXM6XG4gICAgICAgICAgICBmaWVsZDogbmFtZVxuICAgICAgICAgICAgcGF0dGVybjogJE9SSUdJTkFMXG4gICAgICAgICMgNC4gRmluZCBhbiBBTkNFU1RPUiBpbXBvcnRfc3RhdGVtZW50IGFuZCBjYXB0dXJlIGl0cyBzb3VyY2UgcGF0aFxuICAgICAgICAtIGluc2lkZTpcbiAgICAgICAgICAgIHN0b3BCeTogZW5kICMgPDw8LS0tIFRoaXMgaXMgdGhlIGtleSBmaXghIFNlYXJjaCBhbmNlc3RvcnMuXG4gICAgICAgICAgICBraW5kOiBpbXBvcnRfc3RhdGVtZW50XG4gICAgICAgICAgICBoYXM6ICMgRW5zdXJlIHRoZSBmb3VuZCBpbXBvcnRfc3RhdGVtZW50IGhhcyB0aGUgc291cmNlIGZpZWxkXG4gICAgICAgICAgICAgIGZpZWxkOiBzb3VyY2VcbiAgICAgICAgICAgICAgcGF0dGVybjogJFNPVVJDRVxuXG4gICAgIyBEWU5BTUlDIElNUE9SVFMgKFNpbmdsZSBWYXJpYWJsZSBBc3NpZ25tZW50KSBcbiAgICAjIC0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLVxuICAgICMgZWc6IChjb25zdCBWQVJfTkFNRSA9IHJlcXVpcmUoJ1NPVVJDRScpKVxuICAgICMgLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tXG4gICAgLSBhbGw6XG4gICAgICAgIC0ga2luZDogdmFyaWFibGVfZGVjbGFyYXRvclxuICAgICAgICAtIGhhczpcbiAgICAgICAgICAgIGZpZWxkOiBuYW1lXG4gICAgICAgICAgICBraW5kOiBpZGVudGlmaWVyXG4gICAgICAgICAgICBwYXR0ZXJuOiAkVkFSX05BTUUgIyBDYXB0dXJlIHRoZSBzaW5nbGUgdmFyaWFibGUgbmFtZVxuICAgICAgICAtIGhhczpcbiAgICAgICAgICAgIGZpZWxkOiB2YWx1ZVxuICAgICAgICAgICAgYW55OlxuICAgICAgICAgICAgICAjIERpcmVjdCBjYWxsXG4gICAgICAgICAgICAgIC0gYWxsOiAjIFdyYXAgY29uZGl0aW9ucyBpbiBhbGxcbiAgICAgICAgICAgICAgICAgIC0ga2luZDogY2FsbF9leHByZXNzaW9uXG4gICAgICAgICAgICAgICAgICAtIGhhczogeyBmaWVsZDogZnVuY3Rpb24sIHJlZ2V4OiAnXihyZXF1aXJlfGltcG9ydCkkJyB9XG4gICAgICAgICAgICAgICAgICAtIGhhczogeyBmaWVsZDogYXJndW1lbnRzLCBoYXM6IHsga2luZDogc3RyaW5nLCBwYXR0ZXJuOiAkU09VUkNFIH0gfSAjIENhcHR1cmUgc291cmNlXG4gICAgICAgICAgICAgICMgQXdhaXRlZCBjYWxsXG4gICAgICAgICAgICAgIC0ga2luZDogYXdhaXRfZXhwcmVzc2lvblxuICAgICAgICAgICAgICAgIGhhczpcbiAgICAgICAgICAgICAgICAgIGFsbDogIyBXcmFwIGNvbmRpdGlvbnMgaW4gYWxsXG4gICAgICAgICAgICAgICAgICAgIC0ga2luZDogY2FsbF9leHByZXNzaW9uXG4gICAgICAgICAgICAgICAgICAgIC0gaGFzOiB7IGZpZWxkOiBmdW5jdGlvbiwgcmVnZXg6ICdeKHJlcXVpcmV8aW1wb3J0KSQnIH1cbiAgICAgICAgICAgICAgICAgICAgLSBoYXM6IHsgZmllbGQ6IGFyZ3VtZW50cywgaGFzOiB7IGtpbmQ6IHN0cmluZywgcGF0dGVybjogJFNPVVJDRSB9IH0gIyBDYXB0dXJlIHNvdXJjZVxuXG4gICAgIyBEWU5BTUlDIElNUE9SVFMgKERlc3RydWN0dXJlZCBTaG9ydGhhbmQgQXNzaWdubWVudCkgICAgIFxuICAgICMgLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tXG4gICAgIyBlZzogKGNvbnN0IHsgT1JJR0lOQUwgfSA9IHJlcXVpcmUoJ1NPVVJDRScpKVxuICAgICMgLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tXG4gICAgLSBhbGw6XG4gICAgICAgICMgMS4gVGFyZ2V0IHRoZSBzaG9ydGhhbmQgaWRlbnRpZmllciB3aXRoaW4gdGhlIHBhdHRlcm5cbiAgICAgICAgLSBraW5kOiBzaG9ydGhhbmRfcHJvcGVydHlfaWRlbnRpZmllcl9wYXR0ZXJuXG4gICAgICAgIC0gcGF0dGVybjogJE9SSUdJTkFMXG4gICAgICAgICMgMi4gRW5zdXJlIGl0J3MgaW5zaWRlIGFuIG9iamVjdF9wYXR0ZXJuIHRoYXQgaXMgdGhlIG5hbWUgb2YgYSB2YXJpYWJsZV9kZWNsYXJhdG9yXG4gICAgICAgIC0gaW5zaWRlOlxuICAgICAgICAgICAga2luZDogb2JqZWN0X3BhdHRlcm5cbiAgICAgICAgICAgIGluc2lkZTogIyBDaGVjayB0aGUgdmFyaWFibGVfZGVjbGFyYXRvciBpdCBiZWxvbmdzIHRvXG4gICAgICAgICAgICAgIGtpbmQ6IHZhcmlhYmxlX2RlY2xhcmF0b3JcbiAgICAgICAgICAgICAgIyAzLiBDaGVjayB0aGUgdmFsdWUgYXNzaWduZWQgYnkgdGhlIHZhcmlhYmxlX2RlY2xhcmF0b3JcbiAgICAgICAgICAgICAgaGFzOlxuICAgICAgICAgICAgICAgIGZpZWxkOiB2YWx1ZVxuICAgICAgICAgICAgICAgIGFueTpcbiAgICAgICAgICAgICAgICAgICMgRGlyZWN0IGNhbGxcbiAgICAgICAgICAgICAgICAgIC0gYWxsOlxuICAgICAgICAgICAgICAgICAgICAgIC0ga2luZDogY2FsbF9leHByZXNzaW9uXG4gICAgICAgICAgICAgICAgICAgICAgLSBoYXM6IHsgZmllbGQ6IGZ1bmN0aW9uLCByZWdleDogJ14ocmVxdWlyZXxpbXBvcnQpJCcgfVxuICAgICAgICAgICAgICAgICAgICAgIC0gaGFzOiB7IGZpZWxkOiBhcmd1bWVudHMsIGhhczogeyBraW5kOiBzdHJpbmcsIHBhdHRlcm46ICRTT1VSQ0UgfSB9ICMgQ2FwdHVyZSBzb3VyY2VcbiAgICAgICAgICAgICAgICAgICMgQXdhaXRlZCBjYWxsXG4gICAgICAgICAgICAgICAgICAtIGtpbmQ6IGF3YWl0X2V4cHJlc3Npb25cbiAgICAgICAgICAgICAgICAgICAgaGFzOlxuICAgICAgICAgICAgICAgICAgICAgIGFsbDpcbiAgICAgICAgICAgICAgICAgICAgICAgIC0ga2luZDogY2FsbF9leHByZXNzaW9uXG4gICAgICAgICAgICAgICAgICAgICAgICAtIGhhczogeyBmaWVsZDogZnVuY3Rpb24sIHJlZ2V4OiAnXihyZXF1aXJlfGltcG9ydCkkJyB9XG4gICAgICAgICAgICAgICAgICAgICAgICAtIGhhczogeyBmaWVsZDogYXJndW1lbnRzLCBoYXM6IHsga2luZDogc3RyaW5nLCBwYXR0ZXJuOiAkU09VUkNFIH0gfSAjIENhcHR1cmUgc291cmNlXG4gICAgICAgICAgICAgIHN0b3BCeTogZW5kICMgU2VhcmNoIGFuY2VzdG9ycyB0byBmaW5kIHRoZSBjb3JyZWN0IHZhcmlhYmxlX2RlY2xhcmF0b3JcblxuICAgICMgRFlOQU1JQyBJTVBPUlRTIChEZXN0cnVjdHVyZWQgQWxpYXMgQXNzaWdubWVudCkgXG4gICAgIyAtLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS1cbiAgICAjIGVnOiAoY29uc3QgeyBPUklHSU5BTDogQUxJQVMgfSA9IHJlcXVpcmUoJ1NPVVJDRScpKVxuICAgICMgLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tXG4gICAgLSBhbGw6XG4gICAgICAgICMgMS4gVGFyZ2V0IHRoZSBwYWlyX3BhdHRlcm4gZm9yIGFsaWFzZWQgZGVzdHJ1Y3R1cmluZ1xuICAgICAgICAtIGtpbmQ6IHBhaXJfcGF0dGVyblxuICAgICAgICAjIDIuIENhcHR1cmUgdGhlIG9yaWdpbmFsIGlkZW50aWZpZXIgKGtleSlcbiAgICAgICAgLSBoYXM6XG4gICAgICAgICAgICBmaWVsZDoga2V5XG4gICAgICAgICAgICBraW5kOiBwcm9wZXJ0eV9pZGVudGlmaWVyICMgQ291bGQgYmUgc3RyaW5nL251bWJlciBsaXRlcmFsIHRvbywgYnV0IHByb3BlcnR5X2lkZW50aWZpZXIgaXMgY29tbW9uXG4gICAgICAgICAgICBwYXR0ZXJuOiAkT1JJR0lOQUxcbiAgICAgICAgIyAzLiBDYXB0dXJlIHRoZSBhbGlhcyBpZGVudGlmaWVyICh2YWx1ZSlcbiAgICAgICAgLSBoYXM6XG4gICAgICAgICAgICBmaWVsZDogdmFsdWVcbiAgICAgICAgICAgIGtpbmQ6IGlkZW50aWZpZXJcbiAgICAgICAgICAgIHBhdHRlcm46ICRBTElBU1xuICAgICAgICAjIDQuIEVuc3VyZSBpdCdzIGluc2lkZSBhbiBvYmplY3RfcGF0dGVybiB0aGF0IGlzIHRoZSBuYW1lIG9mIGEgdmFyaWFibGVfZGVjbGFyYXRvclxuICAgICAgICAtIGluc2lkZTpcbiAgICAgICAgICAgIGtpbmQ6IG9iamVjdF9wYXR0ZXJuXG4gICAgICAgICAgICBpbnNpZGU6ICMgQ2hlY2sgdGhlIHZhcmlhYmxlX2RlY2xhcmF0b3IgaXQgYmVsb25ncyB0b1xuICAgICAgICAgICAgICBraW5kOiB2YXJpYWJsZV9kZWNsYXJhdG9yXG4gICAgICAgICAgICAgICMgNS4gQ2hlY2sgdGhlIHZhbHVlIGFzc2lnbmVkIGJ5IHRoZSB2YXJpYWJsZV9kZWNsYXJhdG9yXG4gICAgICAgICAgICAgIGhhczpcbiAgICAgICAgICAgICAgICBmaWVsZDogdmFsdWVcbiAgICAgICAgICAgICAgICBhbnk6XG4gICAgICAgICAgICAgICAgICAjIERpcmVjdCBjYWxsXG4gICAgICAgICAgICAgICAgICAtIGFsbDpcbiAgICAgICAgICAgICAgICAgICAgICAtIGtpbmQ6IGNhbGxfZXhwcmVzc2lvblxuICAgICAgICAgICAgICAgICAgICAgIC0gaGFzOiB7IGZpZWxkOiBmdW5jdGlvbiwgcmVnZXg6ICdeKHJlcXVpcmV8aW1wb3J0KSQnIH1cbiAgICAgICAgICAgICAgICAgICAgICAtIGhhczogeyBmaWVsZDogYXJndW1lbnRzLCBoYXM6IHsga2luZDogc3RyaW5nLCBwYXR0ZXJuOiAkU09VUkNFIH0gfSAjIENhcHR1cmUgc291cmNlXG4gICAgICAgICAgICAgICAgICAjIEF3YWl0ZWQgY2FsbFxuICAgICAgICAgICAgICAgICAgLSBraW5kOiBhd2FpdF9leHByZXNzaW9uXG4gICAgICAgICAgICAgICAgICAgIGhhczpcbiAgICAgICAgICAgICAgICAgICAgICBhbGw6XG4gICAgICAgICAgICAgICAgICAgICAgICAtIGtpbmQ6IGNhbGxfZXhwcmVzc2lvblxuICAgICAgICAgICAgICAgICAgICAgICAgLSBoYXM6IHsgZmllbGQ6IGZ1bmN0aW9uLCByZWdleDogJ14ocmVxdWlyZXxpbXBvcnQpJCcgfVxuICAgICAgICAgICAgICAgICAgICAgICAgLSBoYXM6IHsgZmllbGQ6IGFyZ3VtZW50cywgaGFzOiB7IGtpbmQ6IHN0cmluZywgcGF0dGVybjogJFNPVVJDRSB9IH0gIyBDYXB0dXJlIHNvdXJjZVxuICAgICAgICAgICAgICBzdG9wQnk6IGVuZCAjIFNlYXJjaCBhbmNlc3RvcnMgdG8gZmluZCB0aGUgY29ycmVjdCB2YXJpYWJsZV9kZWNsYXJhdG9yXG4gICAgICAgICAgICBzdG9wQnk6IGVuZCAjIEVuc3VyZSB3ZSBjaGVjayBhbmNlc3RvcnMgZm9yIHRoZSB2YXJpYWJsZV9kZWNsYXJhdG9yXG5cbiAgICAjIERZTkFNSUMgSU1QT1JUUyAoU2lkZSBFZmZlY3QgLyBTb3VyY2UgT25seSkgXG4gICAgIyAtLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS1cbiAgICAjIGVnOiAocmVxdWlyZSgnU09VUkNFJykpXG4gICAgIyAtLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS1cbiAgICAtIGFsbDpcbiAgICAgICAgLSBraW5kOiBzdHJpbmcgIyBUYXJnZXQgdGhlIHNvdXJjZSBzdHJpbmcgbGl0ZXJhbCBkaXJlY3RseVxuICAgICAgICAtIHBhdHRlcm46ICRTT1VSQ0VcbiAgICAgICAgLSBpbnNpZGU6ICMgU3RyaW5nIG11c3QgYmUgdGhlIGFyZ3VtZW50IG9mIHJlcXVpcmUoKSBvciBpbXBvcnQoKVxuICAgICAgICAgICAga2luZDogYXJndW1lbnRzXG4gICAgICAgICAgICBwYXJlbnQ6XG4gICAgICAgICAgICAgIGtpbmQ6IGNhbGxfZXhwcmVzc2lvblxuICAgICAgICAgICAgICBoYXM6XG4gICAgICAgICAgICAgICAgZmllbGQ6IGZ1bmN0aW9uXG4gICAgICAgICAgICAgICAgIyBNYXRjaCAncmVxdWlyZScgaWRlbnRpZmllciBvciAnaW1wb3J0JyBrZXl3b3JkIHVzZWQgZHluYW1pY2FsbHlcbiAgICAgICAgICAgICAgICByZWdleDogJ14ocmVxdWlyZXxpbXBvcnQpJCdcbiAgICAgICAgICAgIHN0b3BCeTogZW5kICMgU2VhcmNoIGFuY2VzdG9ycyBpZiBuZWVkZWQgKGZvciB0aGUgYXJndW1lbnRzL2NhbGxfZXhwcmVzc2lvbilcbiAgICAgICAgLSBub3Q6XG4gICAgICAgICAgICBpbnNpZGU6XG4gICAgICAgICAgICAgIGtpbmQ6IGxleGljYWxfZGVjbGFyYXRpb25cbiAgICAgICAgICAgICAgc3RvcEJ5OiBlbmQgIyBTZWFyY2ggYWxsIGFuY2VzdG9ycyB1cCB0byB0aGUgcm9vdFxuXG4gICAgIyBOQU1FU1BBQ0UgSU1QT1JUUyBcbiAgICAjIC0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLVxuICAgICMgZWc6IChpbXBvcnQgKiBhcyBucyBmcm9tICdtb2QnKVxuICAgICMgLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tXG4gICAgLSBhbGw6XG4gICAgICAgIC0ga2luZDogaW1wb3J0X3N0YXRlbWVudFxuICAgICAgICAtIGhhczpcbiAgICAgICAgICAgIGtpbmQ6IGltcG9ydF9jbGF1c2VcbiAgICAgICAgICAgIGhhczpcbiAgICAgICAgICAgICAga2luZDogbmFtZXNwYWNlX2ltcG9ydFxuICAgICAgICAgICAgICBoYXM6XG4gICAgICAgICAgICAgICAgIyBuYW1lc3BhY2VfaW1wb3J0J3MgY2hpbGQgaWRlbnRpZmllciBpcyB0aGUgYWxpYXNcbiAgICAgICAgICAgICAgICBraW5kOiBpZGVudGlmaWVyXG4gICAgICAgICAgICAgICAgcGF0dGVybjogJE5BTUVTUEFDRV9BTElBU1xuICAgICAgICAtIGhhczpcbiAgICAgICAgICAgIGZpZWxkOiBzb3VyY2VcbiAgICAgICAgICAgIHBhdHRlcm46ICRTT1VSQ0VcblxuICAgICMgU0lERSBFRkZFQ1QgSU1QT1JUUyBcbiAgICAjIC0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLVxuICAgICMgZWc6IChpbXBvcnQgJ21vZCcpXG4gICAgIyAtLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS1cbiAgICAtIGFsbDpcbiAgICAgICAgLSBraW5kOiBpbXBvcnRfc3RhdGVtZW50XG4gICAgICAgIC0gbm90OiAjIE11c3QgTk9UIGhhdmUgYW4gaW1wb3J0X2NsYXVzZVxuICAgICAgICAgICAgaGFzOiB7IGtpbmQ6IGltcG9ydF9jbGF1c2UgfVxuICAgICAgICAtIGhhczogIyBCdXQgbXVzdCBoYXZlIGEgc291cmNlXG4gICAgICAgICAgICBmaWVsZDogc291cmNlXG4gICAgICAgICAgICBwYXR0ZXJuOiAkU09VUkNFXG4iLCJzb3VyY2UiOiIvL0B0cy1ub2NoZWNrXG4vLyBOYW1lZCBpbXBvcnRcbmltcG9ydCB7IHRlc3RpbmcgfSBmcm9tICcuL3Rlc3RzJztcblxuLy8gQWxpYXNlZCBpbXBvcnRcbmltcG9ydCB7IHRlc3RpbmcgYXMgdGVzdCB9IGZyb20gJy4vdGVzdHMyJztcblxuLy8gRGVmYXVsdCBpbXBvcnRcbmltcG9ydCBoZWxsbyBmcm9tICdoZWxsb193b3JsZDEnO1xuXG4vLyBOYW1lc3BhY2UgaW1wb3J0XG5pbXBvcnQgKiBhcyBzb21ldGhpbmcgZnJvbSAnaGVsbG9fd29ybGQyJztcblxuLy8gU2lkZS1lZmZlY3QgaW1wb3J0XG5pbXBvcnQgJ0BmYXN0aWZ5L3N0YXRpYyc7XG5cbi8vIFR5cGUgaW1wb3J0XG5pbXBvcnQge3R5cGUgaGVsbG8xMjQzIGFzIHRlc3Rpbmd9IGZyb20gJ2hlbGxvJztcblxuLy8gUmVxdWlyZSBwYXR0ZXJuc1xuY29uc3QgbW9kID0gcmVxdWlyZSgnc29tZS1tb2R1bGUnKTtcbnJlcXVpcmUoJ3BvbHlmaWxsJyk7XG5cbi8vIERlc3RydWN0dXJlZCByZXF1aXJlXG5jb25zdCB7IHRlc3QxMjIsIHRlc3QyIH0gPSByZXF1aXJlKCcuL2Rlc3RydWN0dXJlZDEnKTtcbi8vIEFsaWFzZWQgcmVxdWlyZVxuY29uc3QgeyB0ZXN0MTIyOiB0ZXN0MTIzLCB0ZXN0MjogdGVzdDIzLCB0ZXN0MzogdGVzdDMzIH0gPSByZXF1aXJlKCcuL2Rlc3RydWN0dXJlZDInKTtcblxuLy8gTWl4ZWQgaW1wb3J0c1xuaW1wb3J0IGRlZmF1bHRFeHBvcnQsIHsgbmFtZWRFeHBvcnQgfSBmcm9tICcuL21peGVkJztcbmltcG9ydCBkZWZhdWx0RXhwb3J0MiwgKiBhcyBuYW1lc3BhY2UgZnJvbSAnLi9taXhlZDInO1xuXG5cbi8vIE11bHRpcGxlIGltcG9ydCBsaW5lcyBmcm9tIHRoZSBzYW1lIGZpbGVcbmltcG9ydCB7IG9uZSwgdHdvIGFzIGFsaWFzLCB0aHJlZSB9IGZyb20gJy4vbXVsdGlwbGUnO1xuaW1wb3J0IHsgbmV2ZXIsIGdvbm5hLCBnaXZlLCB5b3UsIHVwIH0gZnJvbSAnLi9tdWx0aXBsZSc7XG5cbi8vIFN0cmluZyBsaXRlcmFsIHZhcmlhdGlvbnNcbmltcG9ydCB7IHRlc3QxIH0gZnJvbSBcIi4vZG91YmxlLXF1b3RlZFwiO1xuaW1wb3J0IHsgdGVzdDIgfSBmcm9tICcuL3NpbmdsZS1xdW90ZWQnO1xuXG4vLyBNdWx0aWxpbmUgaW1wb3J0c1xuaW1wb3J0IHtcbiAgICBsb25nSW1wb3J0MSxcbiAgICBsb25nSW1wb3J0MiBhcyBhbGlhczIsXG4gICAgbG9uZ0ltcG9ydDNcbn0gZnJvbSAnLi9tdWx0aWxpbmUnO1xuXG4vLyBEeW5hbWljIGltcG9ydHNcbmNvbnN0IGR5bmFtaWNNb2R1bGUgPSBpbXBvcnQoJy4vZHluYW1pYzEnKTtcbmNvbnN0IHt0ZXN0aW5nLCB0ZXN0aW5nMTIzfSA9IGltcG9ydCgnLi9keW5hbWljMicpO1xuY29uc3QgYXN5bmNEeW5hbWljTW9kdWxlID0gYXdhaXQgaW1wb3J0KCcuL2FzeW5jX2R5bmFtaWMxJykudGhlbihtb2R1bGUgPT4gbW9kdWxlLmRlZmF1bHQpO1xuLy8gQWxpYXNlZCBkeW5hbWljIGltcG9ydFxuY29uc3QgeyBvcmlnaW5hbElkZW50aWZpZXI6IGFsaWFzZWREeW5hbWljSW1wb3J0fSA9IGF3YWl0IGltcG9ydCgnLi9hc3luY19keW5hbWljMicpO1xuXG4vLyBDb21tZW50cyBpbiBpbXBvcnRzXG5pbXBvcnQgLyogdGVzdCAqLyB7IFxuICAgIC8vIENvbW1lbnQgaW4gaW1wb3J0XG4gICAgY29tbWVudGVkSW1wb3J0IFxufSBmcm9tICcuL2NvbW1lbnRlZCc7IC8vIEVuZCBvZiBsaW5lIGNvbW1lbnQgXG5cblxuIn0=) ### Description [​](https://ast-grep.github.io/catalog/typescript/find-import-identifiers.html#description) Finding import metadata can be useful. Below is a comprehensive snippet for extracting identifiers from various import statements: * Alias Imports (`import { hello as world } from './file'`) * Default & Regular Imports (`import test from './my-test`') * Dynamic Imports (`require(...)`, and `import(...)`) * Side Effect & Namespace Imports (`import * as myCode from './code`') ### YAML [​](https://ast-grep.github.io/catalog/typescript/find-import-identifiers.html#yaml) yaml # find-all-imports-and-identifiers.yaml id: find-all-imports-and-identifiers language: TypeScript rule: any: # ALIAS IMPORTS # ------------------------------------------------------------ # import { ORIGINAL as ALIAS } from 'SOURCE' # ------------------------------------------------------------ - all: # 1. Target the specific node type for named imports - kind: import_specifier # 2. Ensure it *has* an 'alias' field, capturing the alias identifier - has: field: alias pattern: $ALIAS # 3. Capture the original identifier (which has the 'name' field) - has: field: name pattern: $ORIGINAL # 4. Find an ANCESTOR import_statement and capture its source path - inside: stopBy: end # <<<--- Search ancestors. kind: import_statement has: # Ensure the found import_statement has the source field field: source pattern: $SOURCE # DEFAULT IMPORTS # ------------------------------------------------------------ # import { ORIGINAL } from 'SOURCE' # ------------------------------------------------------------ - all: - kind: import_statement - has: # Ensure it has an import_clause... kind: import_clause has: # ...that directly contains an identifier (the default import name) # This identifier is NOT under a 'named_imports' or 'namespace_import' node kind: identifier pattern: $DEFAULT_NAME - has: field: source pattern: $SOURCE # REGULAR IMPORTS # ------------------------------------------------------------ # import { ORIGINAL } from 'SOURCE' # ------------------------------------------------------------ - all: # 1. Target the specific node type for named imports - kind: import_specifier # 2. Ensure it *has* an 'alias' field, capturing the alias identifier - has: field: name pattern: $ORIGINAL # 4. Find an ANCESTOR import_statement and capture its source path - inside: stopBy: end # <<<--- This is the key fix! Search ancestors. kind: import_statement has: # Ensure the found import_statement has the source field field: source pattern: $SOURCE # DYNAMIC IMPORTS (Single Variable Assignment) # ------------------------------------------------------------ # const VAR_NAME = require('SOURCE') # ------------------------------------------------------------ - all: - kind: variable_declarator - has: field: name kind: identifier pattern: $VAR_NAME # Capture the single variable name - has: field: value any: # Direct call - all: # Wrap conditions in all - kind: call_expression - has: { field: function, regex: '^(require|import)$' } - has: { field: arguments, has: { kind: string, pattern: $SOURCE } } # Capture source # Awaited call - kind: await_expression has: all: # Wrap conditions in all - kind: call_expression - has: { field: function, regex: '^(require|import)$' } - has: { field: arguments, has: { kind: string, pattern: $SOURCE } } # Capture source # DYNAMIC IMPORTS (Destructured Shorthand Assignment) # ------------------------------------------------------------ # const { ORIGINAL } = require('SOURCE') # ------------------------------------------------------------ - all: # 1. Target the shorthand identifier within the pattern - kind: shorthand_property_identifier_pattern - pattern: $ORIGINAL # 2. Ensure it's inside an object_pattern that is the name of a variable_declarator - inside: kind: object_pattern inside: # Check the variable_declarator it belongs to kind: variable_declarator # 3. Check the value assigned by the variable_declarator has: field: value any: # Direct call - all: - kind: call_expression - has: { field: function, regex: '^(require|import)$' } - has: { field: arguments, has: { kind: string, pattern: $SOURCE } } # Capture source # Awaited call - kind: await_expression has: all: - kind: call_expression - has: { field: function, regex: '^(require|import)$' } - has: { field: arguments, has: { kind: string, pattern: $SOURCE } } # Capture source stopBy: end # Search ancestors to find the correct variable_declarator # DYNAMIC IMPORTS (Destructured Alias Assignment) # ------------------------------------------------------------ # const { ORIGINAL: ALIAS } = require('SOURCE') # ------------------------------------------------------------ - all: # 1. Target the pair_pattern for aliased destructuring - kind: pair_pattern # 2. Capture the original identifier (key) - has: field: key kind: property_identifier # Could be string/number literal too, but property_identifier is common pattern: $ORIGINAL # 3. Capture the alias identifier (value) - has: field: value kind: identifier pattern: $ALIAS # 4. Ensure it's inside an object_pattern that is the name of a variable_declarator - inside: kind: object_pattern inside: # Check the variable_declarator it belongs to kind: variable_declarator # 5. Check the value assigned by the variable_declarator has: field: value any: # Direct call - all: - kind: call_expression - has: { field: function, regex: '^(require|import)$' } - has: { field: arguments, has: { kind: string, pattern: $SOURCE } } # Capture source # Awaited call - kind: await_expression has: all: - kind: call_expression - has: { field: function, regex: '^(require|import)$' } - has: { field: arguments, has: { kind: string, pattern: $SOURCE } } # Capture source stopBy: end # Search ancestors to find the correct variable_declarator stopBy: end # Ensure we check ancestors for the variable_declarator # DYNAMIC IMPORTS (Side Effect / Source Only) # ------------------------------------------------------------ # require('SOURCE') # ------------------------------------------------------------ - all: - kind: string # Target the source string literal directly - pattern: $SOURCE - inside: # String must be the argument of require() or import() kind: arguments parent: kind: call_expression has: field: function # Match 'require' identifier or 'import' keyword used dynamically regex: '^(require|import)$' stopBy: end # Search ancestors if needed (for the arguments/call_expression) - not: inside: kind: lexical_declaration stopBy: end # Search all ancestors up to the root # NAMESPACE IMPORTS # ------------------------------------------------------------ # import * as ns from 'mod' # ------------------------------------------------------------ - all: - kind: import_statement - has: kind: import_clause has: kind: namespace_import has: # namespace_import's child identifier is the alias kind: identifier pattern: $NAMESPACE_ALIAS - has: field: source pattern: $SOURCE # SIDE EFFECT IMPORTS # ------------------------------------------------------------ # import 'mod' # ------------------------------------------------------------ - all: - kind: import_statement - not: # Must NOT have an import_clause has: { kind: import_clause } - has: # But must have a source field: source pattern: $SOURCE ### Example [​](https://ast-grep.github.io/catalog/typescript/find-import-identifiers.html#example) ts //@ts-nocheck // Named import import { testing } from './tests'; // Aliased import import { testing as test } from './tests2'; // Default import import hello from 'hello_world1'; // Namespace import import * as something from 'hello_world2'; // Side-effect import import '@fastify/static'; // Type import import {type hello1243 as testing} from 'hello'; // Require patterns const mod = require('some-module'); require('polyfill'); // Destructured require const { test122, test2 } = require('./destructured1'); // Aliased require const { test122: test123, test2: test23, test3: test33 } = require('./destructured2'); // Mixed imports import defaultExport, { namedExport } from './mixed'; import defaultExport2, * as namespace from './mixed2'; // Multiple import lines from the same file import { one, two as alias, three } from './multiple'; import { never, gonna, give, you, up } from './multiple'; // String literal variations import { test1 } from "./double-quoted"; import { test2 } from './single-quoted'; // Multiline imports import { longImport1, longImport2 as alias2, longImport3 } from './multiline'; // Dynamic imports const dynamicModule = import('./dynamic1'); const {testing, testing123} = import('./dynamic2'); const asyncDynamicModule = await import('./async_dynamic1').then(module => module.default); // Aliased dynamic import const { originalIdentifier: aliasedDynamicImport} = await import('./async_dynamic2'); // Comments in imports import /* test */ { // Comment in import commentedImport } from './commented'; // End of line comment ### Contributed by [​](https://ast-grep.github.io/catalog/typescript/find-import-identifiers.html#contributed-by) [Michael Angelo Rivera](https://github.com/michaelangeloio) --- # What is ast-grep? | ast-grep [Skip to content](https://ast-grep.github.io/guide/introduction.html#VPContent) Return to top Are you an LLM? You can read better optimized documentation at /guide/introduction.md for this page in Markdown format What is ast-grep? [​](https://ast-grep.github.io/guide/introduction.html#what-is-ast-grep) =========================================================================================== Introduction [​](https://ast-grep.github.io/guide/introduction.html#introduction) ---------------------------------------------------------------------------------- ast-grep is a new AST based tool to manage your code, at massive scale. Using ast-grep can be as simple as running a single command in your terminal: bash ast-grep --pattern 'var code = $PAT' --rewrite 'let code = $PAT' --lang js The command above will replace `var` statement with `let` for all JavaScript files. * * * ast-grep is a versatile tool for searching, linting and rewriting code in various languages. * **Search**: As a _command line tool_ in your terminal, `ast-grep` can precisely search code _based on AST_, running through ten thousand files in sub seconds. * **Lint**: You can use ast-grep as a linter. Thanks to the flexible rule system, adding a new customized rule is intuitive and straightforward, with _pretty error reporting_ out of box. * **Rewrite**: ast-grep provide API to traverse and manipulate syntax tree. Besides, you can also use operators to compose complex matching from simple patterns. > Think ast-grep as an hybrid of [grep](https://www.gnu.org/software/grep/manual/grep.html) > , [eslint](https://eslint.org/) > and [codemod](https://github.com/facebookincubator/fastmod) > . Wanna try it out? Check out the [quick start guide](https://ast-grep.github.io/guide/quick-start.html) ! Or see some [examples](https://ast-grep.github.io/catalog/) to get a sense of what ast-grep can do. We also have a [playground](https://ast-grep.github.io/playground.html) for you to try out ast-grep online! Supported Languages [​](https://ast-grep.github.io/guide/introduction.html#supported-languages) ------------------------------------------------------------------------------------------------ ast-grep supports a wide range of programming languages. Here is a list of notable programming languages it supports. | Language Domain | Supported Languages | | --- | --- | | System Programming | `C`, `Cpp`, `Rust` | | Server Side Programming | `Go`, `Java`, `Python`, `C-sharp` | | Web Development | `JS(X)`, `TS(X)`, `HTML`, `CSS` | | Mobile App Development | `Kotlin`, `Swift` | | Configuration | `Json`, `YAML`, `Hcl` | | Scripting, Protocols, etc. | `Lua`, `Nix` | Thanks to [tree-sitter](https://tree-sitter.github.io/tree-sitter/) , a popular parser generator library, ast-grep manages to support [many languages](https://ast-grep.github.io/reference/languages.html) out of the box! Motivation [​](https://ast-grep.github.io/guide/introduction.html#motivation) ------------------------------------------------------------------------------ Using text-based tool for searching code is fast but imprecise. We usually prefer to parse the code into [abstract syntax tree](https://www.wikiwand.com/en/Abstract_syntax_tree) for precise matches. However, developing with AST is tedious and frustrating. Consider this "hello-world" level task: matching `console.log` in JavaScript using Babel. We will need to write code like below. javascript path.parentPath.isMemberExpression() && path.parentPath.get('object').isIdentifier({ name: 'console' }) && path.parentPath.get('property').isIdentifier({ name: 'log' }) This snippet deserves a detailed explanation for beginners. Even for experienced developers, authoring this snippet also requires a lot of looking up references. The pain is not language specific. The [quotation](https://portswigger.net/daily-swig/semgrep-static-code-analysis-tool-helps-eliminate-entire-classes-of-vulnerabilities) from Jobert Abma, co-founder of HackerOne, manifests the universal pain across many languages. > The internal AST query interfaces those tools offer are often poorly documented and difficult to write, understand, and maintain. * * * ast-grep solves the problem by providing a simple core mechanism: using code to search code with the same pattern. Consider it as same as `grep` but based on AST instead of text. In comparison to Babel, we can complete this hello-world task in ast-grep trivially bash ast-grep -p "console.log" See [playground](https://ast-grep.github.io/playground.html) in action! Upon the simple pattern code, we can build a series of operators to compose complex matching rules for various scenarios. Though we use JavaScript in our introduction, ast-grep is not language specific. It is a _polyglot_ tool backed by the renowned library [tree-sitter](https://tree-sitter.github.io/) . The idea of ast-grep can be applied to many other languages! Features [​](https://ast-grep.github.io/guide/introduction.html#features) -------------------------------------------------------------------------- There are a lot of other tools that looks like ast-grep, notable predecessors including [Semgrep](https://semgrep.dev/) , [comby](https://comby.dev/) , [shisho](https://github.com/flatt-security/shisho) , [gogocode](https://github.com/thx/gogocode) , and new comers like [gritQL](https://about.grit.io/) What makes ast-grep stands out is: ### Performance [​](https://ast-grep.github.io/guide/introduction.html#performance) It is written in Rust, a native language and utilize multiple cores. (It can even beat ag when searching simple pattern). ast-grep can handle tens of thousands files in seconds. ### Progressiveness [​](https://ast-grep.github.io/guide/introduction.html#progressiveness) You can start from creating a one-liner to rewrite code at command line with minimal investment. Later if you see some code smell recurrently appear in your projects, you can write a linting rule in YAML with a few patterns combined. Finally if you are a library author or framework designer, ast-grep provide programmatic interface to rewrite or transpile code efficiently. ### Pragmatism [​](https://ast-grep.github.io/guide/introduction.html#pragmatism) ast-grep comes with batteries included. Interactive code modification is available. Linter and language server work out of box when you install the command line tool. ast-grep is also shipped with test framework for rule authors. Check out Discord and StackOverflow [​](https://ast-grep.github.io/guide/introduction.html#check-out-discord-and-stackoverflow) -------------------------------------------------------------------------------------------------------------------------------- Still got questions? Join our [Discord](https://discord.gg/4YZjf6htSQ) and discuss with other users! You can also ask questions under the [ast-grep](https://stackoverflow.com/questions/tagged/ast-grep) tag on [StackOverflow](https://stackoverflow.com/questions/ask) . --- # TODO: | ast-grep [Skip to content](https://ast-grep.github.io/links/roadmap.html#VPContent) Return to top Are you an LLM? You can read better optimized documentation at /links/roadmap.md for this page in Markdown format TODO: [​](https://ast-grep.github.io/links/roadmap.html#todo) ============================================================== Core [​](https://ast-grep.github.io/links/roadmap.html#core) ------------------------------------------------------------- * \[x\] Add replace * \[x\] Add find\_all * \[x\] Add metavar char customization * \[x\] Add per-language customization * \[x\] Add support for vec/sequence matcher * \[x\] View node in context * \[x\] implement iterative DFS mode * \[ \] Investigate perf heuristic (e.g. match fixed-string) * \[x\] Group matching rules based on root pattern kind id * \[ \] Remove unwrap usage and implement error handling Metavariable Matcher [​](https://ast-grep.github.io/links/roadmap.html#metavariable-matcher) --------------------------------------------------------------------------------------------- * \[x\] Regex * \[x\] Pattern * \[x\] Kind * \[ \] Use CoW to optimize MetaVarEnv Operators/Combinators [​](https://ast-grep.github.io/links/roadmap.html#operators-combinators) ----------------------------------------------------------------------------------------------- * \[x\] every / all * \[x\] either / any * \[x\] inside * \[x\] has * \[x\] follows * \[x\] precedes CLI [​](https://ast-grep.github.io/links/roadmap.html#cli) ----------------------------------------------------------- * \[x\] match against files in directory recursively * \[x\] interactive mode * \[x\] as dry run mode (listing all rewrite) * \[x\] inplace edit mode * \[x\] no-color mode * \[x\] JSON output * \[ \] execute remote rules Config [​](https://ast-grep.github.io/links/roadmap.html#config) ----------------------------------------------------------------- * \[x\] support YAML config rule * \[x\] Add support for severity * \[x\] Add support for error message * \[x\] Add support for error labels * \[x\] Add support for fix Binding [​](https://ast-grep.github.io/links/roadmap.html#binding) ------------------------------------------------------------------- * \[ \] NAPI binding * \[x\] WASM binding * \[ \] Python binding Playground [​](https://ast-grep.github.io/links/roadmap.html#playground) ------------------------------------------------------------------------- * \[x\] build a playground based on WASM binding * \[x\] build YAML config for WASM playground * \[x\] URL sharing * \[x\] add fix/rewrite LSP [​](https://ast-grep.github.io/links/roadmap.html#lsp) ----------------------------------------------------------- * \[x\] Add LSP command * \[ \] implement LSP incremental * \[ \] add code action Builtin Ruleset [​](https://ast-grep.github.io/links/roadmap.html#builtin-ruleset) ----------------------------------------------------------------------------------- * \[ \] Migrate some ESLint rule (or RSLint rule) --- # ast-grep new | ast-grep [Skip to content](https://ast-grep.github.io/reference/cli/new.html#VPContent) Return to top Are you an LLM? You can read better optimized documentation at /reference/cli/new.md for this page in Markdown format `ast-grep new` [​](https://ast-grep.github.io/reference/cli/new.html#ast-grep-new) =================================================================================== Create new ast-grep project or items like rules/tests. Also see the step by step [guide](https://ast-grep.github.io/guide/scan-project.html) . Usage [​](https://ast-grep.github.io/reference/cli/new.html#usage) ------------------------------------------------------------------- shell ast-grep new [COMMAND] [OPTIONS] [NAME] Commands [​](https://ast-grep.github.io/reference/cli/new.html#commands) ------------------------------------------------------------------------- ### `project` [​](https://ast-grep.github.io/reference/cli/new.html#project) Create an new project by scaffolding. By default, this command will create a root config file `sgconfig.yml`, a rule folder `rules`, a test case folder `rule-tests` and a utility rule folder `utils`. You can customize the folder names during the creation. ### `rule` [​](https://ast-grep.github.io/reference/cli/new.html#rule) Create a new rule. This command will create a new rule in one of the `rule_dirs`. You need to provide `name` and `language` either by interactive input or via command line arguments. ast-grep will ask you which `rule_dir` to use if multiple ones are configured in the `sgconfig.yml`. If `-y, --yes` flag is true, ast-grep will choose the first `rule_dir` to create the new rule. ### `test` [​](https://ast-grep.github.io/reference/cli/new.html#test) Create a new test case. This command will create a new test in one of the `test_dirs`. You need to provide `name` either by interactive input or via command line arguments. ast-grep will ask you which `test_dir` to use if multiple ones are configured in the `sgconfig.yml`. If `-y, --yes` flag is true, ast-grep will choose the first `test_dir` to create the new test. ### `util` [​](https://ast-grep.github.io/reference/cli/new.html#util) Create a new global utility rule. This command will create a new global utility rule in one of the `utils` folders. You need to provide `name` and `language` either by interactive input or via command line arguments. ast-grep will ask you which `util_dir` to use if multiple ones are configured in the `sgconfig.yml`. If `-y, --yes` flag is true, ast-grep will choose the first `util_dir` to create the new item. ### `help` [​](https://ast-grep.github.io/reference/cli/new.html#help) Print this message or the help of the given subcommand(s) Arguments [​](https://ast-grep.github.io/reference/cli/new.html#arguments) --------------------------------------------------------------------------- `[NAME]` The id of the item to create Options [​](https://ast-grep.github.io/reference/cli/new.html#options) ----------------------------------------------------------------------- ### `-l, --lang ` [​](https://ast-grep.github.io/reference/cli/new.html#l-lang-lang) The language of the item to create. This option is only available when creating rule and util. ### `-y, --yes` [​](https://ast-grep.github.io/reference/cli/new.html#y-yes) Accept all default options without interactive input during creation. You need to provide all required arguments via command line if this flag is true. Please see the command description for the what arguments are required. ### `-c, --config ` [​](https://ast-grep.github.io/reference/cli/new.html#c-config-config-file) Path to ast-grep root config, default is sgconfig.yml ### `-h, --help` [​](https://ast-grep.github.io/reference/cli/new.html#h-help) Print help (see a summary with '-h') --- # Handle Error Reports | ast-grep [Skip to content](https://ast-grep.github.io/guide/project/severity.html#VPContent) Return to top Are you an LLM? You can read better optimized documentation at /guide/project/severity.md for this page in Markdown format Handle Error Reports [​](https://ast-grep.github.io/guide/project/severity.html#handle-error-reports) ====================================================================================================== Severity Levels [​](https://ast-grep.github.io/guide/project/severity.html#severity-levels) -------------------------------------------------------------------------------------------- ast-grep supports these severity levels for rules: * `error`: The rule will report an error and fails a scan. * `warning`: The rule will report a warning. * `info`: The rule will report an informational message. * `hint`: The rule will report a hint. This is the default severity level. * `off`: The rule will disable the rule at all. If an `error` rule is triggered, `ast-grep scan` will exit with a non-zero status code. This is useful for CI/CD pipelines to fail the build when a rule is violated. You can configure the severity level of a rule in the rule file: yaml id: rule-id severity: error # ... more fields Override Severity on CLI [​](https://ast-grep.github.io/guide/project/severity.html#override-severity-on-cli) -------------------------------------------------------------------------------------------------------------- You can override the severity level of a rule on the command line. This is useful when you want to change the severity level of a rule for a specific scan. bash ast-grep scan --error rule-id --warning other-rule-id You can use multiple `--error`, `--warning`, `--info`, `--hint`, and `--off` flags to override multiple rules. Ignore Linting Error [​](https://ast-grep.github.io/guide/project/severity.html#ignore-linting-error) ------------------------------------------------------------------------------------------------------ It is possible to ignore a single line of code in ast-grep's scanning. A developer can suppress ast-grep's error by adding `ast-grep-ignore` above the line that triggers the issue, or on the same line. The suppression comment has the following format, in JavaScript for example: javascript console.log('hello') // match // ast-grep-ignore console.log('suppressed') // suppressed // ast-grep-ignore: no-console console.log('suppressed') // suppressed // ast-grep-ignore: other-rule console.log('world') // match // Same line suppression console.log('suppressed') // ast-grep-ignore console.log('suppressed') // ast-grep-ignore: no-console See the [playground](https://ast-grep.github.io/playground.html#eyJtb2RlIjoiQ29uZmlnIiwibGFuZyI6ImphdmFzY3JpcHQiLCJxdWVyeSI6IiRDQUxMRVIgOj0gJmZvb3t9IiwicmV3cml0ZSI6IiIsImNvbmZpZyI6ImlkOiBuby1jb25zb2xlXG5sYW5ndWFnZTogSmF2YVNjcmlwdFxucnVsZTpcbiAgcGF0dGVybjogY29uc29sZS5sb2coJEEpIiwic291cmNlIjoiY29uc29sZS5sb2coJ2hlbGxvJykgIC8vIG1hdGNoXG4vLyBhc3QtZ3JlcC1pZ25vcmVcbmNvbnNvbGUubG9nKCdzdXBwcmVzc2VkJykgLy8gc3VwcHJlc3NlZFxuLy8gYXN0LWdyZXAtaWdub3JlOiBuby1jb25zb2xlXG5jb25zb2xlLmxvZygnc3VwcHJlc3NlZCcpIC8vIHN1cHByZXNzZWRcbi8vIGFzdC1ncmVwLWlnbm9yZTogb3RoZXItcnVsZVxuY29uc29sZS5sb2coJ3dvcmxkJykgLy8gbWF0Y2hcbiJ9) in action. These are the rules for suppression comments: * A comment with the content `ast-grep-ignore` will suppress the following line/the same line's diagnostic. * The magic word `ast-grep-ignore` alone will suppress _all_ kinds of diagnostics. * `ast-grep-ignore: ` can turn off specific rules. * You can turn off multiple rules by providing a comma-separated list in the comment. e.g. `ast-grep-ignore: rule-1, rule-2` * Suppression comments will suppress the next line diagnostic if and only if there is no preceding ASTs on the same line. File Level Suppression [​](https://ast-grep.github.io/guide/project/severity.html#file-level-suppression) ---------------------------------------------------------------------------------------------------------- You can also suppress all diagnostics in a file by adding a suppression comment at the top of the file followed by an empty line. This is useful when you want to ignore all diagnostics in a file. For example, in JavaScript: Disable all rulesDisable sepcific rules javascript // ast-grep-ignore // This file will not be scanned by ast-grep // note the empty line after the suppression comment. debugger // this line will not be scanned console.debug('debugging') // this line will not be scanned javascript // ast-grep-ignore: no-debugger // This file will not be scanned by ast-grep // note the empty line after the suppression comment. debugger // this line will not trigger error console.debug('debugging') // this line will trigger error To suppress the whole file, there must be [two conditions](https://github.com/ast-grep/ast-grep/issues/1541#issuecomment-2573212686) met: * The suppression comment is on the very first line of the file. * AND the next line (second line in file) is empty These conditions are designed for backward compatibility. Report Unused Suppressions [​](https://ast-grep.github.io/guide/project/severity.html#report-unused-suppressions) ------------------------------------------------------------------------------------------------------------------ ast-grep can report unused suppression comments in your codebase. This is useful to keep your codebase clean and to avoid suppressing issues that are no longer relevant. An example report will look like this: diff help[unused-suppression]: Unused 'ast-grep-ignore' directive. - // ast-grep-ignore + `unused-suppression` itself behaves like a `hint` rule with auto-fix. But it is enabled, by default, only **when all rules are enabled**. More specifically, [these conditions](https://github.com/ast-grep/ast-grep/blob/553f5e5ac577b6d2e0904c423bb5dbd27804328b/crates/cli/src/scan.rs#L68-L73) must be met: 1. No rule is [disabled](https://ast-grep.github.io/guide/project/severity.html#override-severity-on-cli) by the `--off` flag on the CLI. `severity: off` configured in the YAML rule file does not count. 2. The CLI [`--rule`](https://ast-grep.github.io/reference/cli/scan.html#r-rule-rule-file) flag is not used. 3. The CLI [`--inline-rules`](https://ast-grep.github.io/reference/cli/scan.html#inline-rules-rule-text) flag is not used. 4. The CLI [`--filter`](https://ast-grep.github.io/reference/cli/scan.html#filter-regex) flag is not used. Unused suppression report only happens in `ast-grep scan` If a rule is skipped during a scan, it is possible to mistakenly report a suppression comment as unused. So running specific rules or disabling rules will not trigger the unused suppression report. You can also override the severity level of the `unused-suppression` rule on the command line. This can change the default behavior or unused-suppression reporting. bash # treat unused directive as error, useful in CI/CD ast-grep scan --error unused-suppression # enable report even not all rules are enabled ast-grep --rule rule.yml scan --hint unused-suppression Disallow Suppress-All Comments [​](https://ast-grep.github.io/guide/project/severity.html#disallow-suppress-all-comments) -------------------------------------------------------------------------------------------------------------------------- By default, `ast-grep-ignore` without a rule ID suppresses _all_ diagnostics on a line. This can accidentally hide issues, especially when `ast-grep-ignore: rule-id` is mistakenly written as `ast-grep-ignore rule-id` (missing the colon). ast-grep provides a built-in rule `no-suppress-all` to disallow such suppress-all comments. When enabled, any `ast-grep-ignore` comment that does not specify a rule ID will be flagged. bash # flag suppress-all comments as warning ast-grep scan --warning=no-suppress-all # flag suppress-all comments as error in CI/CD ast-grep scan --error=no-suppress-all With `no-suppress-all` enabled, developers are required to always specify which rule(s) to suppress. This helps track suppression counts per rule and prevents accidental suppression of multiple rules. Inspect Rule Severity [​](https://ast-grep.github.io/guide/project/severity.html#inspect-rule-severity) -------------------------------------------------------------------------------------------------------- Finally, ast-grep provides a CLI flag [`--inspect`](https://ast-grep.github.io/reference/cli/scan.html#inspect-granularity) to debug what rules are enabled and their severity levels. This is useful to understand the rule configuration and to debug why a rule is not triggered. bash ast-grep scan --inspect entity Example standard error debugging output: sg: entity|rule|no-dupe-class-members: finalSeverity=Error sg: entity|rule|no-new-symbol: finalSeverity=Error sg: entity|rule|no-cond-assign: finalSeverity=Warning sg: entity|rule|no-constant-condition: finalSeverity=Warning sg: entity|rule|no-dupe-keys: finalSeverity=Error sg: entity|rule|no-await-in-loop: finalSeverity=Warning --- # JavaScript API | ast-grep [Skip to content](https://ast-grep.github.io/guide/api-usage/js-api.html#VPContent) Return to top Are you an LLM? You can read better optimized documentation at /guide/api-usage/js-api.md for this page in Markdown format JavaScript API [​](https://ast-grep.github.io/guide/api-usage/js-api.html#javascript-api) ========================================================================================== Powered by [napi.rs](https://napi.rs/) , ast-grep's JavaScript API enables you to write JavaScript to programmatically inspect and change syntax trees. ast-grep's JavaScript API design is pretty stable now. No major breaking changes are expected in the future. To try out the JavaScript API, you can use the [code sandbox](https://codesandbox.io/p/sandbox/ast-grep-napi-hhx3tj) here. Installation [​](https://ast-grep.github.io/guide/api-usage/js-api.html#installation) -------------------------------------------------------------------------------------- First, install ast-grep's napi package. npmpnpm bash npm install --save @ast-grep/napi bash pnpm add @ast-grep/napi Now let's explore ast-grep's API! Core Concepts [​](https://ast-grep.github.io/guide/api-usage/js-api.html#core-concepts) ---------------------------------------------------------------------------------------- The core concepts in ast-grep's JavaScript API are: * `SgRoot`: a class representing the whole syntax tree * `SgNode`: a node in the syntax tree Make AST like a DOM tree! Using ast-grep's API is like using [jQuery](https://jquery.com/) . You can use `SgNode` to traverse the syntax tree and collect information from the nodes. Remember your old time web programming? A common workflow to use ast-grep's JavaScript API is: 1. Get a syntax tree object `SgRoot` from string by calling a language's `parse` method 2. Get the root node of the syntax tree by calling `ast.root()` 3. `find` relevant nodes by using patterns or rules 4. Collect information from the nodes **Example:** js import { parse, Lang } from '@ast-grep/napi'; let source = `console.log("hello world")` const ast = parse(Lang.JavaScript, source) // 1. parse the source const root = ast.root() // 2. get the root const node = root.find('console.log($A)') // 3. find the node node.getMatch('A').text() // 4. collect the info // "hello world" ### `SgRoot` [​](https://ast-grep.github.io/guide/api-usage/js-api.html#sgroot) `SgRoot` represents the syntax tree of a source string. We can import the `Lang` enum from the `@ast-grep/napi` package and call the `parse` function to transform string. js import { Lang, parse } from '@ast-grep/napi'; const source = `console.log("hello world")` const ast = parse(Lang.JavaScript, source) The `SgRoot` object has a `root` method that returns the root `SgNode` of the AST. js const root = ast.root() // root is an instance of SgNode ### `SgNode` [​](https://ast-grep.github.io/guide/api-usage/js-api.html#sgnode) `SgNode` is the main interface to view and manipulate the syntax tree. It has several jQuery like methods for us to search, filter and inspect the AST nodes we are interested in. js const log = root.find('console.log($A)') // search node const arg = log.getMatch('A') // get matched variable arg.text() // "hello world" Let's see its details in the following sections! Search [​](https://ast-grep.github.io/guide/api-usage/js-api.html#search) -------------------------------------------------------------------------- You can use `find` and `findAll` to search for nodes in the syntax tree. * `find` returns the first node that matches the pattern or rule. * `findAll` returns an array of nodes that match the pattern or rule. ts // search class SgNode { find(matcher: string): SgNode | null find(matcher: number): SgNode | null find(matcher: NapiConfig): SgNode | null findAll(matcher: string): Array findAll(matcher: number): Array findAll(matcher: NapiConfig): Array } Both `find` and `findAll` are overloaded functions. They can accept either string, number or a config object. The argument is called `Matcher` in ast-grep JS. ### Matcher [​](https://ast-grep.github.io/guide/api-usage/js-api.html#matcher) A `Matcher` can be one of the three types: `string`, `number` or `object`. * `string` is parsed as a [pattern](https://ast-grep.github.io/guide/pattern-syntax.html) . e.g. `'console.log($A)'` * `number` is interpreted as the node's kind. In tree-sitter, an AST node's type is represented by a number called kind id. Different syntax node has different kind ids. You can convert a kind name like `function` to the numeric representation by calling the `kind` function. e.g. `kind(Lang.JavaScript, 'function')`. * A `NapiConfig` has a similar type of [config object](https://ast-grep.github.io/reference/yaml.html) . See details below. ts // basic find example root.find('console.log($A)') // returns SgNode of call_expression let l = Lang.JavaScript // calling kind function requires Lang const kind = kind(l, 'string') // convert kind name to kind id number root.find(kind) // returns SgNode of string root.find('notExist') // returns null if not found // basic find all example const nodes = root.findAll('function $A($$$) {$$$}') Array.isArray(nodes) // true, findAll returns SgNode nodes.map(n => n.text()) // string array of function source const empty = root.findAll('not exist') // returns [] empty.length === 0 // true // find i.e. `console.log("hello world")` using a NapiConfig const node = root.find({ rule: { pattern: "console.log($A)" }, constraints: { A: { regex: "hello" } } }) Note, `find` returns `null` if no node is found. `findAll` returns an empty array if nothing matches. Match [​](https://ast-grep.github.io/guide/api-usage/js-api.html#match) ------------------------------------------------------------------------ Once we find a node, we can use the following methods to get meta variables from the search. The `getMatch` method returns the single node that matches the [single meta variable](https://ast-grep.github.io/guide/pattern-syntax.html#meta-variable) . And the `getMultipleMatches` returns an array of nodes that match the [multi meta variable](https://ast-grep.github.io/guide/pattern-syntax.html#multi-meta-variable) . ts // search export class SgNode { getMatch(m: string): SgNode | null getMultipleMatches(m: string): Array } **Example:** ts const src = ` console.log('hello') logger('hello', 'world', '!') ` const root = parse(Lang.JavaScript, src).root() const node = root.find('console.log($A)') const arg = node.getMatch("A") // returns SgNode('hello') arg !== null // true, node is found arg.text() // returns 'hello' // returns [] because $A and $$$A are different node.getMultipleMatches('A') const logs = root.find('logger($$$ARGS)') // returns [SgNode('hello'), SgNode(','), SgNode('world'), SgNode(','), SgNode('!')] logs.getMultipleMatches("ARGS") logs.getMatch("A") // returns null Inspection [​](https://ast-grep.github.io/guide/api-usage/js-api.html#inspection) ---------------------------------------------------------------------------------- The following methods are used to inspect the node. ts // node inspection export class SgNode { range(): Range isLeaf(): boolean kind(): string text(): string } **Example:** ts const ast = parse(Lang.JavaScript, "console.log('hello world')") root = ast.root() root.text() // will return "console.log('hello world')" Another important method is `range`, which returns two `Pos` object representing the start and end of the node. One `Pos` contains the line, column, and offset of that position. All of them are 0-indexed. You can use the range information to locate the source and modify the source code. ts const rng = node.range() const pos = rng.start // or rng.end, both are `Pos` objects pos.line // 0, line starts with 0 pos.column // 0, column starts with 0 rng.end.index // 17, index starts with 0 Refinement [​](https://ast-grep.github.io/guide/api-usage/js-api.html#refinement) ---------------------------------------------------------------------------------- You can also filter nodes after matching by using the following methods. This is dubbed as "refinement" in the documentation. Note these refinement methods only support using `pattern` at the moment. ts export class SgNode { matches(m: string): boolean inside(m: string): boolean has(m: string): boolean precedes(m: string): boolean follows(m: string): boolean } **Example:** ts const node = root.find('console.log($A)') node.matches('console.$METHOD($B)') // true Traversal [​](https://ast-grep.github.io/guide/api-usage/js-api.html#traversal) -------------------------------------------------------------------------------- You can traverse the tree using the following methods, like using jQuery. ts export class SgNode { children(): Array field(name: string): SgNode | null parent(): SgNode | null child(nth: number): SgNode | null ancestors(): Array next(): SgNode | null nextAll(): Array prev(): SgNode | null prevAll(): Array } Fix code [​](https://ast-grep.github.io/guide/api-usage/js-api.html#fix-code) ------------------------------------------------------------------------------ `SgNode` is immutable so it is impossible to change the code directly. However, `SgNode` has a `replace` method to generate an `Edit` object. You can then use the `commitEdits` method to apply the changes and generate new source string. ts interface Edit { /** The start position of the edit */ startPos: number /** The end position of the edit */ endPos: number /** The text to be inserted */ insertedText: string } class SgNode { replace(text: string): Edit commitEdits(edits: Edit[]): string } **Example** ts const root = parse(Lang.JavaScript, "console.log('hello world')").root() const node = root.find('console.log($A)') const edit = node.replace("console.error('bye world')") const newSource = node.commitEdits([edit]) // "console.error('bye world')" Note, `console.error($A)` will not generate `console.error('hello world')` in JavaScript API unlike the CLI. This is because using the host language to generate the replacement string is more flexible. WARNING Metavariable will not be replaced in the `replace` method. You need to create a string using `getMatch(var_name)` by using JavaScript. See also [ast-grep#1172](https://github.com/ast-grep/ast-grep/issues/1172) Use Other Language [​](https://ast-grep.github.io/guide/api-usage/js-api.html#use-other-language) -------------------------------------------------------------------------------------------------- To access other languages, you will need to use `registerDynamicLanguage` function and probably `@ast-grep/lang-*` package. This is an experimental feature and the doc is not ready yet. Please refer to the [repo](https://github.com/ast-grep/langs) for more information. If you are interested in using other languages, please let us know by creating an issue. --- # Relational Rules | ast-grep [Skip to content](https://ast-grep.github.io/guide/rule-config/relational-rule.html#VPContent) Return to top Are you an LLM? You can read better optimized documentation at /guide/rule-config/relational-rule.md for this page in Markdown format Relational Rules [​](https://ast-grep.github.io/guide/rule-config/relational-rule.html#relational-rules) ========================================================================================================= Atomic rule can only match the target node directly. But sometimes we want to match a node based on its surrounding nodes. For example, we want to find `await` expression inside a `for` loop. Relational rules are powerful operators that can filter the _target_ nodes based on their _surrounding_ nodes. ast-grep now supports four kinds of relational rules: `inside`, `has`, `follows`, and `precedes`. All four relational rules accept a sub rule object as their value. The sub rule will match the surrounding node while the relational rule itself will match the target node. Relational Rule Example [​](https://ast-grep.github.io/guide/rule-config/relational-rule.html#relational-rule-example) ----------------------------------------------------------------------------------------------------------------------- Having an `await` expression inside a for loop is usually a bad idea because every iteration will have to wait for the previous promise to resolve. We can use the relational rule `inside` to filter out the `await` expression. yaml rule: pattern: await $PROMISE inside: kind: for_in_statement stopBy: end The rule reads as "matches an `await` expression that is `inside` a `for_in_statement`". See [Playground](https://ast-grep.github.io/playground.html#eyJtb2RlIjoiQ29uZmlnIiwibGFuZyI6InR5cGVzY3JpcHQiLCJxdWVyeSI6IiRDOiAkVCA9IHJlbGF0aW9uc2hpcCgkJCRBLCB1c2VsaXN0PVRydWUsICQkJEIpIiwicmV3cml0ZSI6IiRDOiBMaXN0WyRUXSA9IHJlbGF0aW9uc2hpcCgkJCRBLCB1c2VsaXN0PVRydWUsICQkJEIpIiwiY29uZmlnIjoiaWQ6IG5vLWF3YWl0LWluLWxvb3Bcbmxhbmd1YWdlOiBUeXBlU2NyaXB0XG5ydWxlOlxuICBwYXR0ZXJuOiBhd2FpdCAkUFJPTUlTRVxuICBpbnNpZGU6XG4gICAga2luZDogZm9yX2luX3N0YXRlbWVudFxuICAgIHN0b3BCeTogZW5kIiwic291cmNlIjoiZm9yIChsZXQgaSBvZiBbMSwgMiwzXSkge1xuICAgIGF3YWl0IFByb21pc2UucmVzb2x2ZShpKVxufSJ9) . The relational rule `inside` accepts a rule and will match any node that is inside another node that satisfies the inside rule. The `inside` rule itself matches `await` and its sub rule `kind` matches the surrounding loop. Relational Rule's Sub Rule [​](https://ast-grep.github.io/guide/rule-config/relational-rule.html#relational-rule-s-sub-rule) ----------------------------------------------------------------------------------------------------------------------------- Since relational rules accept another ast-grep rule, we can compose more complex examples by using operators recursively. yaml rule: pattern: await $PROMISE inside: any: - kind: for_in_statement - kind: for_statement - kind: while_statement - kind: do_statement stopBy: end The above rule will match different kinds of loops, like `for`, `for-in`, `while` and `do-while`. So all the code below matches the rule: js while (foo) { await bar() } for (let i = 0; i < 10; i++) { await bar() } for (let key in obj) { await bar() } do { await bar() } while (condition) See in [playground](https://ast-grep.github.io/playground.html#eyJtb2RlIjoiQ29uZmlnIiwibGFuZyI6InR5cGVzY3JpcHQiLCJxdWVyeSI6IiRDOiAkVCA9IHJlbGF0aW9uc2hpcCgkJCRBLCB1c2VsaXN0PVRydWUsICQkJEIpIiwicmV3cml0ZSI6IiRDOiBMaXN0WyRUXSA9IHJlbGF0aW9uc2hpcCgkJCRBLCB1c2VsaXN0PVRydWUsICQkJEIpIiwiY29uZmlnIjoiaWQ6IG5vLWF3YWl0LWluLWxvb3Bcbmxhbmd1YWdlOiBUeXBlU2NyaXB0XG5ydWxlOlxuICBwYXR0ZXJuOiBhd2FpdCAkUFJPTUlTRVxuICBpbnNpZGU6XG4gICAgYW55OlxuICAgICAgLSBraW5kOiBmb3JfaW5fc3RhdGVtZW50XG4gICAgICAtIGtpbmQ6IGZvcl9zdGF0ZW1lbnRcbiAgICAgIC0ga2luZDogd2hpbGVfc3RhdGVtZW50XG4gICAgICAtIGtpbmQ6IGRvX3N0YXRlbWVudFxuICAgIHN0b3BCeTogZW5kIiwic291cmNlIjoid2hpbGUgKGZvbykge1xuICBhd2FpdCBiYXIoKVxufVxuZm9yIChsZXQgaSA9IDA7IGkgPCAxMDsgaSsrKSB7XG4gIGF3YWl0IGJhcigpXG59XG5mb3IgKGxldCBrZXkgaW4gb2JqKSB7XG4gIGF3YWl0IGJhcigpXG59XG5kbyB7XG4gIGF3YWl0IGJhcigpXG59IHdoaWxlIChjb25kaXRpb24pIn0=) . Pro Tip You can also use `pattern` in relational rule! The metavariable matched in relational rule can also be used in `fix`. This will effectively let you extract a child node from a match. Relational Rule Mnemonics [​](https://ast-grep.github.io/guide/rule-config/relational-rule.html#relational-rule-mnemonics) --------------------------------------------------------------------------------------------------------------------------- The four relational rules can read as: * `inside`: the _target_ node must be **inside** a node that matches the sub rule. * `has`: the _target_ node must **have** a child node specified by the sub rule. * `follows`: the _target_ node must **follow** a node specified by the sub rule. (target after surrounding) * `precedes`: the _target_ node must **precede** a node specified by the sub rule. (target before surrounding). It is sometimes confusing to remember whether the rule matches target node or surrounding node. Here is the mnemonics to help you read the rule. First, relational rule is usually used along with another rule. Second, the other rule will match the target node. Finally, the relational rule's sub rule will match the surrounding node. Together, the rule specifies that the target node will `be inside` or `follows` the surrounding node. TIP All relational rule takes the form of `target` `relates` to `surrounding`. For example, the rule below will match **`hello`(target)** greeting that **follows(relation)** a **`world`(surrounding)** greeting. yaml pattern: console.log('hello'); follows: pattern: console.log('world'); Consider the [input source code](https://ast-grep.github.io/playground.html#eyJtb2RlIjoiQ29uZmlnIiwibGFuZyI6ImphdmFzY3JpcHQiLCJxdWVyeSI6ImNvbnNvbGUubG9nKCRNQVRDSCkiLCJjb25maWciOiJydWxlOlxuICBhbGw6XG4gICAgLSBwYXR0ZXJuOiBjb25zb2xlLmxvZygnaGVsbG8nKTtcbiAgICAtIGZvbGxvd3M6XG4gICAgICAgIHBhdHRlcm46IGNvbnNvbGUubG9nKCd3b3JsZCcpOyIsInNvdXJjZSI6ImNvbnNvbGUubG9nKCdoZWxsbycpOyAvLyBkb2VzIG5vdCBtYXRjaFxuY29uc29sZS5sb2coJ3dvcmxkJyk7XG5jb25zb2xlLmxvZygnaGVsbG8nKTsgLy8gbWF0Y2hlcyEhIn0=) . Only the second `console.log('hello')` will match the rule. javascript console.log('hello'); // does not match console.log('world'); console.log('hello'); // matches!! Fine Tuning Relational Rule [​](https://ast-grep.github.io/guide/rule-config/relational-rule.html#fine-tuning-relational-rule) ------------------------------------------------------------------------------------------------------------------------------- Relational rule has several options to let you find nodes more precisely. ### `stopBy` [​](https://ast-grep.github.io/guide/rule-config/relational-rule.html#stopby) By default, relational rule will only match nodes one level further. For example, ast-grep will only match the direct children of the target node for the `has` rule. You can change the behavior by using the `stopBy` field. It accepts three kinds of values: string `'end'`, string `'neighbor'` (the default option), and a rule object. `stopBy: end` will make ast-grep search surrounding nodes until it reaches the end. For example, it stops when the rule hits root node, leaf node or the first/last sibling node. yaml has: stopBy: end pattern: $MY_PATTERN `stopBy` can also accept a custom rule object, so the searching will only stop when the rule matches the surrounding node. yaml # find if a node is inside a function called test. It stops whenever the ancestor node is a function. inside: stopBy: kind: function pattern: function test($$$) { $$$ } Note the `stopBy` rule is inclusive. So when both `stopBy` rule and relational rule hit a node, the node is considered as a match. ### `field` [​](https://ast-grep.github.io/guide/rule-config/relational-rule.html#field) Sometimes it is useful to specify the node by its field. Suppose we want to find a JavaScript object property with the key `prototype`, an outdated practice that we should avoid. yaml kind: pair # key-value pair in JS has: field: key # note here regex: 'prototype' This rule will match the following code js var a = { prototype: anotherObject } but will not match this code js var a = { normalKey: prototype } Though `pair` has a child with text `prototype` in the second example, its relative field is not `key`. That is, `prototype` is not used as `key` but instead used as value. So it does not match the rule. --- # Reusing Rule as Utility | ast-grep [Skip to content](https://ast-grep.github.io/guide/rule-config/utility-rule.html#VPContent) Return to top Are you an LLM? You can read better optimized documentation at /guide/rule-config/utility-rule.md for this page in Markdown format Reusing Rule as Utility [​](https://ast-grep.github.io/guide/rule-config/utility-rule.html#reusing-rule-as-utility) ==================================================================================================================== ast-grep chooses to use YAML for rule representation. While this decision makes writing rules easier, it does impose some limitations on the rule authoring. One of the limitations is that rule objects cannot be reused. Let's see an example. Suppose we want to match all literal values in JavaScript. We will need to match these kinds: yaml any: - kind: 'false' - kind: undefined - kind: 'null' - kind: 'true' - kind: regex - kind: number - kind: string If we want to match functions in different places using only the plain YAML file, we will have to copy and paste the rule above several times. Say, we want to match either literal values or an array of literal values: yaml rule: any: - kind: 'false' - kind: undefined # more literal kinds omitted # ... - kind: array has: any: - kind: 'false' - kind: undefined # more literal kinds omitted # ... ast-grep provides a mechanism to reuse common rules: `utils`. A utility rule is a rule defined in the `utils` section of the config file, or in a separate global rule file. It can be referenced in other rules using the composite rule `matches`. So, the above example can be rewritten as: yaml # define util rules using utils field utils: # it accepts a string-keyed dictionary of rule object is-literal: # rule-id any: # actual rule object - kind: 'false' - kind: undefined - kind: 'null' - kind: 'true' - kind: regex - kind: number - kind: string rule: any: - matches: is-literal # reference the util! - kind: array has: matches: is-literal # reference it again! There are two ways to define utility rules in ast-grep: _Local utility rules_ and _Global Utility Rules_. They are both used in the `matches` composite rules by their ids. Local Utility Rules [​](https://ast-grep.github.io/guide/rule-config/utility-rule.html#local-utility-rules) ------------------------------------------------------------------------------------------------------------ Local utility rules are defined in the `utils` field of the config file. `utils` is a string-keyed dictionary. The keys of the dictionary is utility rules' identifiers, which will be later used in `matches`. Note that local utility rule identifier cannot have the same name of another local utility rule. But a local utility rule can have the same name of another global utility rule and override the global one. The value of the dictionary is the rule object. You can define a local utility rule using the same syntax as the `rule` field. **Local utility rules are only available in the config file where they are defined.** For example, the following config file defines a local utility rule `is-literal`: yaml utils: is-literal: any: - kind: 'false' - kind: undefined - kind: 'null' - kind: 'true' - kind: regex - kind: number - kind: string rule: matches: is-literal The `matches` in `rule` will run the matcher rule `is-literal` against AST nodes. Local rules must have the same language as their rule configuration file where utilities are defined. Local rules cannot have their separate `constraints` as well. Global Utility Rules [​](https://ast-grep.github.io/guide/rule-config/utility-rule.html#global-utility-rules) -------------------------------------------------------------------------------------------------------------- Global utility rules are defined in a separate file. But they are available across all rule configurations in the project. To create global utility rules, you first need a proper ast-grep project setup like below. yml my-awesome-project # project root |- rules # rule directory | |- my-rule.yml |- utils # utils directory | |- is-literal.yml |- sgconfig.yml # project configuration Note the `utils` directory where all global utility rules will be stored. We also need to specify which directory is utility rules so that ast-grep can pick up. In `sgconfig.yml`: yml ruleDirs: - rules utilDirs: - utils Now we can define our global utility rule in the `is-literal.yml` file. A global utility rule looks like a regular rule file, but it can only have limited fields: `id`, `language`, `rule`, `constraints` and their own local rules `utils`. yaml # is-literal.yml id: is-literal language: TypeScript rule: any: - kind: 'false' - kind: undefined - kind: 'null' - kind: 'true' - kind: regex - kind: number - kind: string Contrary to local utility rule, you must define `id` and `language` in the global utility rule. The `id` is not defined as a dictionary key. Global utility rule have their own local utility rules and these local rules can only be accessed in their defining global rule file. Similarly, global utility rules can have their own `constraints` as well. Finally, a rule file, whether it is a utility rule or not, can have local utility rules with the same name of another global utility rule. Global utility rules are superseded by the local homonymous rule. Recursive Rule Trick [​](https://ast-grep.github.io/guide/rule-config/utility-rule.html#recursive-rule-trick) -------------------------------------------------------------------------------------------------------------- You can use a utility rule inside another utility. Besides rule reusing, this also opens the possibility of recursive rule. For example, if we want to match all expressions that evaluate to number literal in JavaScript. We can use `kind: number` to match `123` or `1.23`. But how to match expressions in parenthesis like `(((123)))`? Using `matches` and utility rule can solve this. yml utils: is-number: any: - kind: number - kind: parenthesized_expression has: matches: is-number rule: matches: is-number If we matches `(123)` with this rule, we will first match the `kind: parenthesized_expression` with a direct child that also matches `is-number` rule. This will make us match `123` with `is-number` which will succeed because `kind: number` matches the number literal. Using `matches` and recursive utility rule can unlock a lot of sophisticated usage of rule. But there is one thing you need to bear in mind: Dependency Cycle is not allowed Rule cannot have a cyclic dependency when using `matches`. That is, a rule cannot transitively reference itself in its composite components. A dependency cycle in rule will cause infinite recursion and make ast-grep stuck in one AST node without any progression. However, you can use self-referencing rule in relational components like `inside` or `has`. A curious reader can try to answer why this is okay. --- # Python API | ast-grep [Skip to content](https://ast-grep.github.io/guide/api-usage/py-api.html#VPContent) Return to top Are you an LLM? You can read better optimized documentation at /guide/api-usage/py-api.md for this page in Markdown format Python API [​](https://ast-grep.github.io/guide/api-usage/py-api.html#python-api) ================================================================================== ast-grep's Python API is powered by [PyO3](https://pyo3.rs/) . You can write Python to programmatically inspect and change syntax trees. To try out ast-grep's Python API, you can use the [online colab notebook](https://colab.research.google.com/drive/1nVT6rQKRIPv0TsKpCv5uD-Zuw-lUC67A?usp=sharing) . Installation [​](https://ast-grep.github.io/guide/api-usage/py-api.html#installation) -------------------------------------------------------------------------------------- ast-grep's Python library is distributed on PyPI. You can install it with pip. bash pip install ast-grep-py Core Concepts [​](https://ast-grep.github.io/guide/api-usage/py-api.html#core-concepts) ---------------------------------------------------------------------------------------- The core concepts in ast-grep's Python API are: * `SgRoot`: a class to parse a string into a syntax tree * `SgNode`: a node in the syntax tree Make AST like a XML/HTML doc! Using ast-grep's API is like [web scraping](https://opensource.com/article/21/9/web-scraping-python-beautiful-soup) using [beautiful soup](https://www.crummy.com/software/BeautifulSoup/bs4/doc/) or [pyquery](https://pyquery.readthedocs.io/en/latest/) . You can use `SgNode` to traverse the syntax tree and collect information from the nodes. A common workflow to use ast-grep's Python API is: 1. Parse a string into a syntax tree by using `SgRoot` 2. Get the root node of the syntax tree by calling `root.root()` 3. `find` relevant nodes by using patterns or rules 4. Collect information from the nodes **Example:** python from ast_grep_py import SgRoot root = SgRoot("print('hello world')", "python") # 1. parse node = root.root() # 2. get root print_stmt = node.find(pattern="print($A)") # 3. find print_stmt.get_match('A').text() # 4. collect information # 'hello world' ### `SgRoot` [​](https://ast-grep.github.io/guide/api-usage/py-api.html#sgroot) The `SgRoot` class has the following signature: python class SgRoot: def __init__(self, src: str, language: str) -> None: ... def root(self) -> SgNode: ... `__init__` takes two arguments: the first argument is the source code string, and the second argument is the language name. `root` returns the root node of the syntax tree, which is an instance of `SgNode`. **Example:** python root = SgRoot("print('hello world')", "python") # 1. parse node = root.root() # 2. get root The code above parses the string `print('hello world')` into a syntax tree, and gets the root node of the syntax tree. The root node can be used to find other nodes in the syntax tree. ### `SgNode` [​](https://ast-grep.github.io/guide/api-usage/py-api.html#sgnode) `SgNode` is the most important class in ast-grep's Python API. It provides methods to inspect and traverse the syntax tree. The following sections will introduce several methods in `SgNode`. **Example:** python node = root.root() string = node.find(kind="string") assert string # assume we can find a string node in the source print(string.text()) Search [​](https://ast-grep.github.io/guide/api-usage/py-api.html#search) -------------------------------------------------------------------------- You can use `find` and `find_all` to search for nodes in the syntax tree. * `find` returns the first node that matches the pattern or rule. * `find_all` returns a list of nodes that match the pattern or rule. python # Search class SgNode: @overload def find(self, **kwargs: Unpack[Rule]) -> Optional[SgNode]: ... @overload def find_all(self, **kwargs: Unpack[Rule]) -> List[SgNode]: ... @overload def find(self, config: Config) -> Optional[SgNode]: ... @overload def find_all(self, config: Config) -> List[SgNode]: ... `find` has two overloads: one takes keyword arguments of [`Rule`](https://ast-grep.github.io/reference/api.html#rule) , and the other takes a [`Config`](https://ast-grep.github.io/reference/api.html#config) object. ### Search with Rule [​](https://ast-grep.github.io/guide/api-usage/py-api.html#search-with-rule) Using keyword arguments rule is the most straightforward way to search for nodes. The argument name is the key of a rule, and the argument value is the rule's value. You can passing multiple keyword arguments to `find` to search for nodes that match **all** the rules. python root = SgRoot("print('hello world')", "python") node = root.root() node.find(pattern="print($A)") # will return the print function call node.find(kind="string") # will return the string 'hello world' # below will return print function call because it matches both rules node.find(pattern="print($A)", kind="call") # below will return None because the pattern cannot be a string literal node.find(pattern="print($A)", kind="string") strings = node.find_all(kind="string") # will return [SgNode("hello world")] assert len(strings) == 1 ### Search with Config [​](https://ast-grep.github.io/guide/api-usage/py-api.html#search-with-config) You can also use a `Config` object to search for nodes. This is similar to directly use YAML in the command line. The main difference between using `Config` and using `Rule` is that `Config` has more options to control the search behavior, like [`constraints`](https://ast-grep.github.io/guide/rule-config.html#constraints) and [`utils`](https://ast-grep.github.io/guide/rule-config/utility-rule.html) . python # will find a string node with text 'hello world' root.root().find({ "rule": { "pattern": "print($A)", }, "constraints": { "A": { "regex": "hello" } } }) # will return None because constraints are not satisfied root.root().find({ "rule": { "pattern": "print($A)", }, "constraints": { "A": { "regex": "no match" } } }) Match [​](https://ast-grep.github.io/guide/api-usage/py-api.html#match) ------------------------------------------------------------------------ Once we find a node, we can use the following methods to get meta variables from the search. The `get_match` method returns the single node that matches the [single meta variable](https://ast-grep.github.io/guide/pattern-syntax.html#meta-variable) . And the `get_multiple_matches` returns a list of nodes that match the [multi meta variable](https://ast-grep.github.io/guide/pattern-syntax.html#multi-meta-variable) . python class SgNode: def get_match(self, meta_var: str) -> Optional[SgNode]: ... def get_multiple_matches(self, meta_var: str) -> List[SgNode]: ... def __getitem__(self, meta_var: str) -> SgNode: ... **Example:** python src = """ print('hello') logger('hello', 'world', '!') """ root = SgRoot(src, "python").root() node = root.find(pattern="print($A)") arg = node.get_match("A") # returns SgNode('hello') assert arg # assert node is found arg.text() # returns 'hello' # returns [] because $A and $$$A are different node.get_multiple_matches("A") logs = root.find(pattern="logger($$$ARGS)") # returns [SgNode('hello'), SgNode(','), SgNode('world'), SgNode(','), SgNode('!')] logs.get_multiple_matches("ARGS") logs.get_match("A") # returns None `SgNode` also supports `__getitem__` to get the match of single meta variable. It is equivalent to `get_match` except that it will either return `SgNode` or raise a `KeyError` if the match is not found. Use `__getitem__` to avoid unnecessary `None` checks when you are using a type checker. python node = root.find(pattern="print($A)") # node.get_match("A").text() # error: node.get_match("A") can be None node["A"].text() # Ok Inspection [​](https://ast-grep.github.io/guide/api-usage/py-api.html#inspection) ---------------------------------------------------------------------------------- The following methods are used to inspect the node. python # Node Inspection class SgNode: def range(self) -> Range: ... def is_leaf(self) -> bool: ... def is_named(self) -> bool: ... def is_named_leaf(self) -> bool: ... def kind(self) -> str: ... def text(self) -> str: ... **Example:** python root = SgRoot("print('hello world')", "python") node = root.root() node.text() # will return "print('hello world')" Another important method is `range`, which returns two `Pos` object representing the start and end of the node. One `Pos` contains the line, column, and offset of that position. All of them are 0-indexed. You can use the range information to locate the source and modify the source code. python rng = node.range() pos = rng.start # or rng.end, both are `Pos` objects pos.line # 0, line starts with 0 pos.column # 0, column starts with 0 rng.end.index # 17, index starts with 0 Refinement [​](https://ast-grep.github.io/guide/api-usage/py-api.html#refinement) ---------------------------------------------------------------------------------- You can also filter nodes after matching by using the following methods. This is dubbed as "refinement" in the documentation. Note these refinement methods only support using `Rule`. python # Search Refinement class SgNode: def matches(self, **rule: Unpack[Rule]) -> bool: ... def inside(self, **rule: Unpack[Rule]) -> bool: ... def has(self, **rule: Unpack[Rule]) -> bool: ... def precedes(self, **rule: Unpack[Rule]) -> bool: ... def follows(self, **rule: Unpack[Rule]) -> bool: ... **Example:** python node = root.find(pattern="print($A)") if node["A"].matches(kind="string"): print("A is a string") Traversal [​](https://ast-grep.github.io/guide/api-usage/py-api.html#traversal) -------------------------------------------------------------------------------- You can traverse the tree using the following methods, like using pyquery. python # Tree Traversal class SgNode: def get_root(self) -> SgRoot: ... def field(self, name: str) -> Optional[SgNode]: ... def parent(self) -> Optional[SgNode]: ... def child(self, nth: int) -> Optional[SgNode]: ... def children(self) -> List[SgNode]: ... def ancestors(self) -> List[SgNode]: ... def next(self) -> Optional[SgNode]: ... def next_all(self) -> List[SgNode]: ... def prev(self) -> Optional[SgNode]: ... def prev_all(self) -> List[SgNode]: ... Fix code [​](https://ast-grep.github.io/guide/api-usage/py-api.html#fix-code) ------------------------------------------------------------------------------ `SgNode` is immutable so it is impossible to change the code directly. However, `SgNode` has a `replace` method to generate an `Edit` object. You can then use the `commitEdits` method to apply the changes and generate new source string. python class Edit: # The start position of the edit start_pos: int # The end position of the edit end_pos: int # The text to be inserted inserted_text: str class SgNode: # Edit def replace(self, new_text: str) -> Edit: ... def commit_edits(self, edits: List[Edit]) -> str: ... **Example** python root = SgRoot("print('hello world')", "python").root() node = root.find(pattern="print($A)") edit = node.replace("logger.log('bye world')") new_src = node.commit_edits([edit]) # "logger.log('bye world')" Note, `logger.log($A)` will not generate `logger.log('hello world')` in Python API unlike the CLI. This is because using the host language to generate the replacement string is more flexible. WARNING Metavariable will not be replaced in the `replace` method. You need to create a string using `get_match(var_name)` by using Python. See also [ast-grep#1172](https://github.com/ast-grep/ast-grep/issues/1172) --- # ast-grep test | ast-grep [Skip to content](https://ast-grep.github.io/reference/cli/test.html#VPContent) Return to top Are you an LLM? You can read better optimized documentation at /reference/cli/test.md for this page in Markdown format `ast-grep test` [​](https://ast-grep.github.io/reference/cli/test.html#ast-grep-test) ====================================================================================== Test ast-grep rules. Usage [​](https://ast-grep.github.io/reference/cli/test.html#usage) -------------------------------------------------------------------- shell ast-grep test [OPTIONS] Options [​](https://ast-grep.github.io/reference/cli/test.html#options) ------------------------------------------------------------------------ ### `-c, --config ` [​](https://ast-grep.github.io/reference/cli/test.html#c-config-config) Path to ast-grep root config, default is sgconfig.yml ### `-t, --test-dir ` [​](https://ast-grep.github.io/reference/cli/test.html#t-test-dir-test-dir) the directories to search test YAML files ### `--snapshot-dir ` [​](https://ast-grep.github.io/reference/cli/test.html#snapshot-dir-snapshot-dir) Specify the directory name storing snapshots. Default to **snapshots** ### `--skip-snapshot-tests` [​](https://ast-grep.github.io/reference/cli/test.html#skip-snapshot-tests) Only check if the test code is valid, without checking rule output. Turn it on when you want to ignore the output of rules. Conflicts with --update-all ### `-U, --update-all` [​](https://ast-grep.github.io/reference/cli/test.html#u-update-all) Update the content of all snapshots that have changed in test. Conflicts with --skip-snapshot-tests ### `-i, --interactive` [​](https://ast-grep.github.io/reference/cli/test.html#i-interactive) Start an interactive review to update snapshots selectively ### `-f, --filter ` [​](https://ast-grep.github.io/reference/cli/test.html#f-filter-filter) Filter rule test cases to execute using a glob pattern ### `--include-off` [​](https://ast-grep.github.io/reference/cli/test.html#include-off) Include `severity:off` rules in test ast-grep will not run rules with `severity: off` by default. This option will include those rules in the test. ### `--color ` [​](https://ast-grep.github.io/reference/cli/test.html#color-when) Controls when to use color in the output. This is useful when piping test output to a file or another command, where ANSI escape codes would be unwanted. Possible values: * `auto` (default): Automatically detect if color support is available on the terminal * `always`: Always display colors * `ansi`: Always display colors but with ANSI escape codes (alias for `always`) * `never`: Never display colors ### `-h, --help` [​](https://ast-grep.github.io/reference/cli/test.html#h-help) Print help --- # Scan Your Project! | ast-grep [Skip to content](https://ast-grep.github.io/guide/scan-project.html#VPContent) Return to top Are you an LLM? You can read better optimized documentation at /guide/scan-project.md for this page in Markdown format Scan Your Project! [​](https://ast-grep.github.io/guide/scan-project.html#scan-your-project) ============================================================================================= Let's explore its power to run scan on your code repository in a scalable way! `ast-grep scan` is the command you can use to run multiple rules against your repository so that you don't need to pass pattern query to your command line every time. However, to ast-grep's scan need some scaffolding for project setup. We will walk through the process in this guide. TIP `ast-grep scan` requires at least one file and one directory to work: * `sgconfig.yml`, the [project configuration](https://ast-grep.github.io/reference/sgconfig.html) file * a directory storing rule files, usually `rules/` Create Scaffolding [​](https://ast-grep.github.io/guide/scan-project.html#create-scaffolding) ---------------------------------------------------------------------------------------------- To set up ast-grep's scanning, you can simply run the command `ast-grep new` in the root directory of your repository. You will be guided with a series of interactive questions, like the following: markdown No sgconfig.yml found. Creating a new ast-grep project... > Where do you want to have your rules? rules > Do you want to create rule tests? Yes > Where do you want to have your tests? rule-tests > Do you want to create folder for utility rules? Yes > Where do you want to have your utilities? utils Your new ast-grep project has been created! After you answering these questions, you will get a folder structure like the below. bash my-awesome-project |- rules # where rules go |- rule-tests # test cases for rules |- utils # global utility rules for reusing |- sgconfig.yml # root configuration file Create the Rule [​](https://ast-grep.github.io/guide/scan-project.html#create-the-rule) ---------------------------------------------------------------------------------------- Now you can start creating a rule! Continue using `ast-grep new`, it will ask you what to create. But you can also use `ast-grep new rule` to create a rule directly! You will be asked several questions about the rule going to be created. Suppose we want to create a rule to ensure no eval in JavaScript. markdown > What is your rule's name? no-eval > Choose rule's language JavaScript Created rules at ./rules/no-eval.yml > Do you also need to create a test for the rule? Yes Created test at rule-tests/no-eval-test.yml Now you can see open the new rule created in the `rules/no-eval.yml`. File path might vary depending on your choice on the first step. > `no-eval.yml` yml id: no-eval message: Add your rule message here.... severity: error # error, warning, hint, info language: JavaScript rule: pattern: Your Rule Pattern here... # utils: Extract repeated rule as local utility here. # note: Add detailed explanation for the rule. We will go through the rule config in the next chapter. But these configurations are quite obvious and self explaining. Let's change the `pattern` inside `rule` and change the rule's message. yml id: no-eval message: Add your rule message here.... message: Do not use eval! Dangerous! Hazardous! Perilous! severity: error language: JavaScript rule: pattern: Your Rule Pattern here... pattern: eval($CODE) Okay! The pattern syntax works just like what we have learnt before. Scan the Code [​](https://ast-grep.github.io/guide/scan-project.html#scan-the-code) ------------------------------------------------------------------------------------ Now you can try scanning the code! You can create a JavaScript file containing `eval` to test it. Run `ast-grep scan` in your project, ast-grep will give you some beautiful scan report! bash error[no-eval]: Add your rule message here.... ┌─ test.js:1:1 │ 1 │ eval('hello') │ ^^^^^^^^^^^^^ Error: 1 error(s) found in code. Help: Scan succeeded and found error level diagnostics in the codebase. Summary [​](https://ast-grep.github.io/guide/scan-project.html#summary) ------------------------------------------------------------------------ In this section we learnt how to set up ast-grep project, create new rules using cli tool and scan problems in the repository. To summarize the commands we used: * `ast-grep new` - Create a new ast-grep project * `ast-grep new rule` - Create a new rule in a rule folder. * `ast-grep scan` - Scan the codebase with the rules in the project. --- # transform Code in Rewrite | ast-grep [Skip to content](https://ast-grep.github.io/guide/rewrite/transform.html#VPContent) Return to top Are you an LLM? You can read better optimized documentation at /guide/rewrite/transform.md for this page in Markdown format `transform` Code in Rewrite [​](https://ast-grep.github.io/guide/rewrite/transform.html#transform-code-in-rewrite) =================================================================================================================== Sometimes, we may want to apply some transformations to the meta variables in the fix part of a YAML rule. For example, we may want to change the case, add or remove prefixes or suffixes. ast-grep provides a `transform` key that allows us to specify such transformations. Use `transform` in Rewrite [​](https://ast-grep.github.io/guide/rewrite/transform.html#use-transform-in-rewrite) ----------------------------------------------------------------------------------------------------------------- `transform` accepts a **dictionary** of which: * the _key_ is the **new variable name** to be introduced and * the _value_ is a **transformation object** that specifies which meta-variable is transformed and how. A transformation object has a key indicating which string operation will be performed on the meta variable, and the value of that key is another object (usually with the source key). Different string operation keys expect different object values. The following is an example illustrating the syntax of a transformation object: yaml transform: NEW_VAR: replace: source: $VAR_NAME replace: regex by: replacement ANOTHER_NEW_VAR: substring: source: $NEW_VAR startChar: 1 endChar: -1 ast-grep 0.38.3+ supports string style transformations to simplify rule writing. The above example can be simplified to one-line style like: yaml transform: NEW_VAR: replace($VAR_NAME, replace=regex, by=replacement) ANOTHER_NEW_VAR: substring($NEW_VAR, startChar=1, endChar=-1) Example of Converting Generator in Python [​](https://ast-grep.github.io/guide/rewrite/transform.html#example-of-converting-generator-in-python) ------------------------------------------------------------------------------------------------------------------------------------------------- [Converting generator expression](https://github.com/ast-grep/ast-grep/discussions/430) to list comprehension in Python is a good example to illustrate `transform`. More concretely, we want to achieve diffs like below: python "".join(i for i in iterable) "".join([i for i in iterable]) This rule will convert the generator inside `join` to a list. yaml id: convert_generator rule: kind: generator_expression pattern: $GEN transform: # 1. the transform option LIST: # 2. New variable name substring: # 3. the transform operation name source: $GEN # 4.1 transformation source startChar: 1 # 4.2 transformation argument endChar: -1 fix: '([$LIST])' # 5. use the new variable in fix Let's discuss the API step by step: 1. The `transform` key is used to define one or more transformations that we want to apply to the meta variables in the pattern part of the rule. 2. The `LIST` key is the new variable name that we can use in `fix` or later transformation. We can choose any name as long as it does not conflict with any existing meta variable names. **Note, the new variable name does not start with `$`.** 3. The `substring` key is the transform operation name that we want to use. This operation will extract a substring from the source string based on the given start and end characters. 4. `substring` accepts an object 1. The `source` key specifies which meta variable we want to transform. **It should have `$` prefix.** In this case, it is `$GEN` that which matches the generator expression in the code. 2. The `startChar` and `endChar` keys specify the indices of the start and end characters of the substring that we want to extract. In this case, we want to extract everything except the wrapping parentheses, which are the first and last characters: `(` and `)`. 5. The `fix` key specifies the new code that we want to replace the matched pattern with. We use the new variable `$LIST` in the fix part, and wrap it with `[` and `]` to make it a list comprehension. Pro Tips Later transformations can use the variables that were transformed before. This allows you to stack string operations and achieve complex transformations. Supported `transformation` [​](https://ast-grep.github.io/guide/rewrite/transform.html#supported-transformation) ----------------------------------------------------------------------------------------------------------------- We have several different transformations available now. Please check out [transformation reference](https://ast-grep.github.io/reference/yaml/transformation.html) for more details. * `replace`: Use a regular expression to replace the text in a meta-variable with a new text. * `substring`: Create a new string by cutting off leading and trailing characters. * `convert`: Change the string case of a meta-variable, such as from `camelCase` to `underscore_case`. * `rewrite`: Apply rewriter rules to a meta-variable AST and generate a new string. It is like rewriting a sub node recursively. Rewrite with Regex Capture Groups [​](https://ast-grep.github.io/guide/rewrite/transform.html#rewrite-with-regex-capture-groups) --------------------------------------------------------------------------------------------------------------------------------- The `replace` transformation allows us to use Rust regex capture groups like `(?.*)` to capture meta-variables and reference them in the `by` field. For example, to replace `debug` with `release` in a function name, we can use the following transformation: yaml id: debug-to-release language: js rule: {pattern: $OLD_FN($$$ARGS)} # Capture OLD_FN constraints: {OLD_FN: {regex: ^debug}} # Only match if it starts with 'debug' transform: NEW_FN: replace: source: $OLD_FN replace: debug(?.*) # Capture everything following 'debug' as REG by: release$REG # Refer to REG just like a meta-variable fix: $NEW_FN($$$ARGS) which will result in [the following change](https://ast-grep.github.io/playground.html#eyJtb2RlIjoiQ29uZmlnIiwibGFuZyI6ImphdmFzY3JpcHQiLCJxdWVyeSI6IiIsInJld3JpdGUiOiIiLCJzdHJpY3RuZXNzIjoic21hcnQiLCJzZWxlY3RvciI6IkVSUk9SIiwiY29uZmlnIjoiaWQ6IGRlYnVnLXRvLXJlbGVhc2Vcbmxhbmd1YWdlOiBqc1xucnVsZToge3BhdHRlcm46ICRPTERfRk4oJCQkQVJHUyl9ICAgIyBDYXB0dXJlIE9MRF9GTlxuY29uc3RyYWludHM6IHtPTERfRk46IHtyZWdleDogXmRlYnVnfX0gICMgT25seSBtYXRjaCBpZiBpdCBzdGFydHMgd2l0aCAnZGVidWcnXG50cmFuc2Zvcm06XG4gIE5FV19GTjpcbiAgICByZXBsYWNlOlxuICAgICAgc291cmNlOiAkT0xEX0ZOXG4gICAgICByZXBsYWNlOiBkZWJ1Zyg/PFJFRz4uKikgICAgICAjIENhcHR1cmUgZXZlcnl0aGluZyBmb2xsb3dpbmcgJ2RlYnVnJyBhcyBSRUdcbiAgICAgIGJ5OiByZWxlYXNlJFJFRyAgICAgICAgICAgICAgICMgUmVmZXIgdG8gUkVHIGp1c3QgbGlrZSBhIG1ldGEtdmFyaWFibGVcbmZpeDogJE5FV19GTigkJCRBUkdTKSIsInNvdXJjZSI6ImRlYnVnRm9vKGFyZzEsIGFyZzIpICAifQ==) : js debugFoo(arg1, arg2) releaseFoo(arg1, arg2) Alternatively, replacing `fooDebug` with `fooRelease`, is difficult because you can't concatenate a meta-variable with a capitalized string literal. `release$REG` is fine, but `$REGRelease` will be interpreted as a single meta-variable and not a concatenation. One workaround is to use multiple sequential transformations, as shown below. Limitation You can only extract regex capture groups in the `replace` field of the `replace` transformation and you can only reference them in the `by` field of the same transformation. The regular `regex` rule does not support capture groups. Multiple Sequential Transformations [​](https://ast-grep.github.io/guide/rewrite/transform.html#multiple-sequential-transformations) ------------------------------------------------------------------------------------------------------------------------------------- Each transformation outputs a meta-variable that can be used as the input to later transformations. Chaining transformations like this allows us to build up complex behaviors. Here we can see an example that transforms `fooDebug` into `fooRelease` by using `convert`, `replace`, and `convert` transformations. yaml rule: {pattern: $OLD_FN($$$ARGS)} # Capture OLD_FN constraints: {OLD_FN: {regex: Debug$}} # Only match if it ends with 'Debug' transform: KEBABED: # 1. Convert to 'foo-debug' convert: source: $OLD_FN toCase: kebabCase RELEASED: # 2. Replace with 'foo-release' replace: source: $KEBABED replace: (?)-debug by: $ROOT-release UNKEBABED: # 3. Convert to 'fooRelease' convert: source: $RELEASED toCase: camelCase fix: $UNKEBABED($$$ARGS) Add conditional text [​](https://ast-grep.github.io/guide/rewrite/transform.html#add-conditional-text) ------------------------------------------------------------------------------------------------------- Occasionally we may want to add extra text, such as punctuations and newlines, to our fixer string. But whether we should add the new text depends on the presence of absence of other syntax nodes. A typical scenario is adding a comma between two arguments or list items. We only want to add a comma when the item we are adding is not the last one in the argument list. We can use `replace` transformation to create a new meta-variable that only contains text when another meta-variable matches something. For example, suppose we want to add a new argument to existing function call. We need to add a comma `,` after the new argument only when the existing call already has some arguments. yaml id: add-leading-argument language: python rule: pattern: $FUNC($$$ARGS) transform: MAYBE_COMMA: replace: source: $$$ARGS replace: '^.+' by: ', ' fix: $FUNC(new_argument$MAYBE_COMMA$$$ARGS) In the above example, if `$$$ARGS` matches nothing, it will be an empty string and the `replace` transformation will take no effect. The final fix string will be instantiated to `$FUNC(new_argument)`. If `$$$ARGS` does match nodes, then the replacement regular expression will replace the text with `,`, so the final fix string will be `$FUNC(new_argument, $$$ARGS)` DasSurma Trick This method is invented by [Surma](https://surma.dev/) in a [tweet](https://twitter.com/DasSurma/status/1706086320051794217) , so the useful trick is named after him. String Style Transformations [​](https://ast-grep.github.io/guide/rewrite/transform.html#string-style-transformations) ----------------------------------------------------------------------------------------------------------------------- To simplify the syntax of transformations, ast-grep 0.38.3+ supports a new string style transformation syntax. This allows us to write transformations in a more concise and readable way. The string style transformation syntax is similar to the CSS function call syntax yaml # illustration of string style transformation syntax NEW_VAR: transform($SOURCE_VAR, option1=value1, option2=value2) The transformation name is followed by parentheses containing the arguments. The first argument is always the source meta-variable, and the rest are the transformation options in the form of key-value pairs. For example, the transformation examples above can be written as: yaml transform: LIST: substring($GEN, startChar=1, endChar=-1) KEBABED: convert($OLD_FN, toCase=kebabCase) MAYBE_COMMA: replace($$$ARGS, replace='^.+', by=', ') WARNING The string style transformation syntax is only available in ast-grep 0.38.3 and later versions. If you are using an older version, please use the original object style syntax. Even More Advanced Transformations [​](https://ast-grep.github.io/guide/rewrite/transform.html#even-more-advanced-transformations) ----------------------------------------------------------------------------------------------------------------------------------- We can use rewriters in the [`rewrite`](https://ast-grep.github.io/guide/rewrite/rewriter.html) transformation to apply dynamic transformations to the AST. We will cover it in next section. --- # List of Languages with Built-in Support | ast-grep [Skip to content](https://ast-grep.github.io/reference/languages.html#VPContent) Return to top Are you an LLM? You can read better optimized documentation at /reference/languages.md for this page in Markdown format List of Languages with Built-in Support [​](https://ast-grep.github.io/reference/languages.html#list-of-languages-with-built-in-support) ========================================================================================================================================= The table below lists all languages that are supported by ast-grep. **Alias** is the name you can use as an argument in `ast-grep run --lang [alias]` or as a value in YAML rule with `language: [alias]`. **Extension** specifies the file extensions that ast-grep will look for when scanning the file system. By default, ast-grep uses the file extensions to determine the language. * * * | Language Name | Alias | File Extension | | --- | --- | --- | | Bash | `bash` | `bash`, `bats`, `cgi`, `command`, `env`, `fcgi`, `ksh`, `sh`, `sh.in`, `tmux`, `tool`, `zsh` | | C | `c` | `c`,`h` | | Cpp | `cc`, `c++`, `cpp`, `cxx` | `cc`, `hpp`, `cpp`, `c++`, `hh`, `cxx`, `cu`, `ino` | | CSharp | `cs`, `csharp` | `cs` | | Css | `css` | `css` | | Elixir | `ex`, `elixir` | `ex`, `exs` | | Go | `go`, `golang` | `go` | | Haskell | `hs`, `haskell` | `hs` | | Hcl | `hcl` | `hcl` | | Html | `html` | `html`, `htm`, `xhtml` | | Java | `java` | `java` | | JavaScript | `javascript`, `js`, `jsx` | `cjs`, `js`, `mjs`, `jsx` | | Json | `json` | `json` | | Kotlin | `kotlin`, `kt` | `kt`, `ktm`, `kts` | | Lua | `lua` | `lua` | | Nix | `nix` | `nix` | | Php | `php` | `php` | | Python | `py`, `python` | `py`, `py3`, `pyi`, `bzl` | | Ruby | `rb`, `ruby` | `rb`, `rbw`, `gemspec` | | Rust | `rs`, `rust` | `rs` | | Scala | `scala` | `scala`, `sc`, `sbt` | | Solidity | `solidity`, `sol` | `sol` | | Swift | `swift` | `swift` | | TypeScript | `ts`, `typescript` | `ts`, `cts`, `mts` | | Tsx | `tsx` | `tsx` | | Yaml | `yml` | `yml`, `yaml` | * * * Pro Tips You can use [`languageGlobs`](https://ast-grep.github.io/reference/sgconfig.html#languageglobs) to customize languages' extension mapping. --- # Rewriter in Fix | ast-grep [Skip to content](https://ast-grep.github.io/guide/rewrite/rewriter.html#VPContent) Return to top Are you an LLM? You can read better optimized documentation at /guide/rewrite/rewriter.md for this page in Markdown format Rewriter in Fix [​](https://ast-grep.github.io/guide/rewrite/rewriter.html#rewriter-in-fix) ============================================================================================ `rewriters` allow you to apply rules to specific parts of the matching AST nodes. ast-grep's `fix` will only replace the matched nodes, one node at a time. But it is common to replace multiple nodes with different fixes at once. The `rewriters` field allows you to do this. The basic workflow of `rewriters` is as follows: 1. Find a list of sub-nodes under a meta-variable that match different rewriters. 2. Generate a distinct fix for each sub-node based on the matched rewriter sub-rule. 3. Join the fixes together and store the string in a new metavariable for later use. Key Steps to Use Rewriters [​](https://ast-grep.github.io/guide/rewrite/rewriter.html#key-steps-to-use-rewriters) ------------------------------------------------------------------------------------------------------------------ To use rewriters, you have three steps. **1\. Define `rewriters` field in the Yaml rule root.** yaml id: rewriter-demo language: Python rewriters: - id: sub-rule rule: # some rule fix: # some fix **2\. Apply the defined rewriters to a metavariable via `transform`.** yaml transform: NEW_VAR: rewrite: rewriters: [sub-rule] source: $OLD_VAR **3\. Use other ast-grep fields to wire them together.** yaml rule: { pattern: a = $OLD_VAR } # ... rewriters and transform fix: a = $NEW_VAR Rewriter Example [​](https://ast-grep.github.io/guide/rewrite/rewriter.html#rewriter-example) ---------------------------------------------------------------------------------------------- Let's see a contrived example: converting `dict` function call to dictionary literal in Python. ### General Idea [​](https://ast-grep.github.io/guide/rewrite/rewriter.html#general-idea) In Python, you can create a dictionary using the `dict` function or the `{}` literal. python # dict function call d = dict(a=1, b=2) # dictionary literal d = {'a': 1, 'b': 2} We will use the `rewriters` field to convert the `dict` function call to a dictionary literal. The recipe is to first find the `dict` function call. Then, extract the keyword arguments like `a=1` and transform them into a dictionary key-value pair `'a': 1`. Finally, we will replace the `dict` function call by combining these transformed pairs and wrapping them in a bracket. The key step is extraction and transformation, which is done by the `rewriters` field. ### Define a Rewriter [​](https://ast-grep.github.io/guide/rewrite/rewriter.html#define-a-rewriter) Our goal is to find keyword arguments in the `dict` function call and transform them into dictionary key-value pairs. So let's first define a rule to match the keyword arguments in the `dict` function call. yaml rule: kind: keyword_argument all: - has: field: name pattern: $KEY - has: field: value pattern: $VAL This rule can match the keyword arguments in the `dict` function call and extract key and value in the argument to meta-variables `$KEY` and `$VAL` respectively. [For example](https://ast-grep.github.io/playground.html#eyJtb2RlIjoiQ29uZmlnIiwibGFuZyI6InB5dGhvbiIsInF1ZXJ5IjoiIiwicmV3cml0ZSI6IiIsInN0cmljdG5lc3MiOiJzbWFydCIsInNlbGVjdG9yIjoic3RhcnRfdGFnIiwiY29uZmlnIjoicnVsZTpcbiAga2luZDoga2V5d29yZF9hcmd1bWVudFxuICBhbGw6XG4gIC0gaGFzOlxuICAgICAgZmllbGQ6IG5hbWVcbiAgICAgIHBhdHRlcm46ICRLRVlcbiAgLSBoYXM6XG4gICAgICBmaWVsZDogdmFsdWVcbiAgICAgIHBhdHRlcm46ICRWQUwiLCJzb3VyY2UiOiJkID0gZGljdChhPTEsIGI9MikifQ==) , `dict(a=1)` will extract `a` to `$KEY` and `1` to `$VAL`. Then, we define the rule as a rewriter and add fix field to transform the keyword argument to a dictionary key-value pair. yaml rewriters: - id: dict-rewrite rule: kind: keyword_argument all: - has: field: name pattern: $KEY - has: field: value pattern: $VAL fix: "'$KEY': $VAL" You can see the `rewriters` field accepts a list of regular ast-grep rules. Rewriter rule must have an `id` field to identify the rewriter, a rule to specify the node to match, and a `fix` field to transform the matched node. Applying the rule above alone will transform `a=1` to `'a': 1`. But it is not enough to replace the `dict` function call. We need to combine these pairs and wrap them in a bracket. We need to apply this rewriter to all keyword arguments and join them. ### Apply Rewriter [​](https://ast-grep.github.io/guide/rewrite/rewriter.html#apply-rewriter) Now, we apply the rewriter to the `dict` function call. This is done by the `transform` field. First, we match the `dict` function call with the pattern `dict($$$ARGS)`. The `$$$ARGS` is a special metavariable that matches all arguments of the function call. Then, we apply the rewriter `dict-rewrite` to the `$$$ARGS` and store the result in a new metavariable `LITERAL`. yaml rule: pattern: dict($$$ARGS) # match dict function call, capture $$$ARGS transform: LITERAL: # the transformed code rewrite: rewriters: [dict-rewrite] # specify the rewriter defined above source: $$$ARGS # apply rewriters to $$$ARGS arguments ast-grep will first try match the `dict-rewrite` rule to each sub node inside `$$$ARGS`. If the node has a matching rule, ast-grep will extract the node specified by the meta-variables in the `dict-rewrite` rewriter rule. It will then generate a new string using the `fix`. Finally, the generated strings replace the matched sub-nodes in the `$$$ARGS` and the new code is stored in the `LITERAL` metavariable. For example, `dict(a=1, b=2)` will match the `$$$ARGS` as `a=1, b=2`. The rewriter will transform `a=1` to `'a': 1` and `b=2` to `'b': 2`. The final value of `LITERAL` will be `'a': 1, 'b': 2`. ### Combine and Replace [​](https://ast-grep.github.io/guide/rewrite/rewriter.html#combine-and-replace) Finally, we combine the transformed keyword arguments and replace the `dict` function call. yaml # define rewriters rewriters: - id: dict-rewrite rule: kind: keyword_argument all: - has: field: name pattern: $KEY - has: field: value pattern: $VAL fix: "'$KEY': $VAL" # find the target node rule: pattern: dict($$$ARGS) # apply rewriters to sub node transform: LITERAL: rewrite: rewriters: [dict-rewrite] source: $$$ARGS # combine and replace fix: '{ $LITERAL }' See the final result in [action](https://ast-grep.github.io/playground.html#eyJtb2RlIjoiQ29uZmlnIiwibGFuZyI6InB5dGhvbiIsInF1ZXJ5IjoiZGljdCgkJCRBUkdTKSIsInJld3JpdGUiOiIiLCJzdHJpY3RuZXNzIjoic21hcnQiLCJzZWxlY3RvciI6IiIsImNvbmZpZyI6IiMgZGVmaW5lIHJld3JpdGVyc1xucmV3cml0ZXJzOlxuLSBpZDogZGljdC1yZXdyaXRlXG4gIHJ1bGU6XG4gICAga2luZDoga2V5d29yZF9hcmd1bWVudFxuICAgIGFsbDpcbiAgICAtIGhhczpcbiAgICAgICAgZmllbGQ6IG5hbWVcbiAgICAgICAgcGF0dGVybjogJEtFWVxuICAgIC0gaGFzOlxuICAgICAgICBmaWVsZDogdmFsdWVcbiAgICAgICAgcGF0dGVybjogJFZBTFxuICBmaXg6IFwiJyRLRVknOiAkVkFMXCJcbiMgZmluZCB0aGUgdGFyZ2V0IG5vZGVcbnJ1bGU6XG4gIHBhdHRlcm46IGRpY3QoJCQkQVJHUylcbiMgYXBwbHkgcmV3cml0ZXJzIHRvIHN1YiBub2RlXG50cmFuc2Zvcm06XG4gIExJVEVSQUw6XG4gICAgcmV3cml0ZTpcbiAgICAgIHJld3JpdGVyczogW2RpY3QtcmV3cml0ZV1cbiAgICAgIHNvdXJjZTogJCQkQVJHU1xuIyBjb21iaW5lIGFuZCByZXBsYWNlXG5maXg6ICd7ICRMSVRFUkFMIH0nIiwic291cmNlIjoiZCA9IGRpY3QoYT0xLCBiPTIpIn0=) . `rewriters` is Top Level [​](https://ast-grep.github.io/guide/rewrite/rewriter.html#rewriters-is-top-level) ------------------------------------------------------------------------------------------------------------ Every ast-grep rule can have one `rewriters` at top level. The `rewriters` accepts a list of rewriter rules. Every rewriter rule is like a regular ast-grep rule with `fix`. These are required fields for a rewriter rule. * `id`: A unique identifier for the rewriter to be referenced in the `rewrite` transformation field. * `rule`: A rule object to match the sub node. * `fix`: A string to replace the matched sub node. Rewriter rule can also have other fields like `transform` and `constraints`. However, fields like `severity` and `message` are not available in rewriter rules. Generally, only [Finding](https://ast-grep.github.io/reference/yaml.html#finding) and [Patching](https://ast-grep.github.io/reference/yaml.html#patching) fields are allowed in rewriter rules. Apply Multiple Rewriters [​](https://ast-grep.github.io/guide/rewrite/rewriter.html#apply-multiple-rewriters) -------------------------------------------------------------------------------------------------------------- Note that the `rewrite` transformation field can accept multiple rewriters. This allows you to apply multiple rewriters to different sub nodes. If the `source` meta variable contains multiple sub nodes, each sub node will be transformed by the corresponding rewriter that matches the sub node. Suppose we have two rewriters to rewrite numbers and strings. yaml rewriters: - id: rewrite-int rule: {kind: integer} fix: integer - id: rewrite-str rule: {kind: string} fix: string We can apply both rewriters to the same source meta-variable. yaml rule: {pattern: '[$$$LIST]' } transform: NEW_VAR: rewrite: rewriters: [rewrite-num, rewrite-str] source: $$$LIST In this case, the `rewrite-num` rewriter will be applied to the integer nodes in `$$$LIST`, and the `rewrite-str` rewriter will be applied to the string nodes in `$$$LIST`. The produced `NEW_VAR` will contain the transformed nodes from both rewriters. [For example](https://ast-grep.github.io/playground.html#eyJtb2RlIjoiQ29uZmlnIiwibGFuZyI6InB5dGhvbiIsInF1ZXJ5IjoiZGljdCgkJCRBUkdTKSIsInJld3JpdGUiOiIiLCJzdHJpY3RuZXNzIjoic21hcnQiLCJzZWxlY3RvciI6IiIsImNvbmZpZyI6InJld3JpdGVyczpcbi0gaWQ6IHJld3JpdGUtaW50XG4gIHJ1bGU6IHtraW5kOiBpbnRlZ2VyfVxuICBmaXg6IGludGVnZXJcbi0gaWQ6IHJld3JpdGUtc3RyXG4gIHJ1bGU6IHtraW5kOiBzdHJpbmd9XG4gIGZpeDogc3RyaW5nXG5ydWxlOiB7cGF0dGVybjogJ1skJCRMSVNUXScgfVxudHJhbnNmb3JtOlxuICBORVdfVkFSOlxuICAgIHJld3JpdGU6XG4gICAgICByZXdyaXRlcnM6IFtyZXdyaXRlLWludCwgcmV3cml0ZS1zdHJdXG4gICAgICBzb3VyY2U6ICQkJExJU1RcbmZpeDogJE5FV19WQVIiLCJzb3VyY2UiOiJbMSwgJ2EnXSJ9) , `[1, 'a']` will be transformed to `integer, string`. Pro Tip Using multiple rewriters can make you dynamically apply different rewriting logic to different sub nodes, based on the matching rules. In case multiple rewriters match the same sub node, only the matching rewriter that appears first in the `rewriters` list will be applied. Therefore, _**the order of rewriters in the `rewriters` list matters.**_ Use Alternative Joiner [​](https://ast-grep.github.io/guide/rewrite/rewriter.html#use-alternative-joiner) ---------------------------------------------------------------------------------------------------------- By default, ast-grep will generate the new rewritten string by replacing the text in the matched sub nodes. But you can also specify an alternative joiner to join the transformed sub nodes via `joinBy` field. yaml transform: NEW_VAR: rewrite: rewriters: [rewrite-num, rewrite-str] source: $$$LIST joinBy: ' + ' This will transform `1, 2, 3` to `integer + integer + integer`. Philosophy behind Rewriters [​](https://ast-grep.github.io/guide/rewrite/rewriter.html#philosophy-behind-rewriters) -------------------------------------------------------------------------------------------------------------------- You can see a more detailed design philosophy, _Find and Patch_, behind rewriters in [this page](https://ast-grep.github.io/advanced/find-n-patch.html) . --- # Editor Integration | ast-grep [Skip to content](https://ast-grep.github.io/guide/tools/editors.html#VPContent) Return to top Are you an LLM? You can read better optimized documentation at /guide/tools/editors.md for this page in Markdown format Editor Integration [​](https://ast-grep.github.io/guide/tools/editors.html#editor-integration) =============================================================================================== ast-grep is a **command line tool** for structural search/replace. But it can be readily integrated into your editors and streamline your workflow. This page introduces several **editors** that has ast-grep support. VSCode [​](https://ast-grep.github.io/guide/tools/editors.html#vscode) ----------------------------------------------------------------------- ast-grep has an official [VSCode extension](https://marketplace.visualstudio.com/items?itemName=ast-grep.ast-grep-vscode#overview) in the market place. To get a feel of what it can do, see the introduction on YouTube! ### Features [​](https://ast-grep.github.io/guide/tools/editors.html#features) The ast-grep VSCode is an extension to bridge the power of ast-grep and the beloved editor VSCode. It includes two parts: * a UI for ast-grep CLI and * a client for ast-grep LSP. Requirement You need to [install ast-grep CLI](https://ast-grep.github.io/guide/quick-start.html#installation) locally and optionally [set up a linting project](https://ast-grep.github.io/guide/scan-project.html) . ### Structural Search [​](https://ast-grep.github.io/guide/tools/editors.html#structural-search) Use [pattern](https://ast-grep.github.io/guide/pattern-syntax.html) to structural search your codebase. | Feature | Screenshot | | --- | --- | | Search Pattern | ![](https://github.com/ast-grep/ast-grep-vscode/blob/main/readme/search-pattern.png?raw=true) | | Search YAML | ![](https://github.com/ast-grep/ast-grep-vscode/blob/main/readme/search-yaml.png?raw=true) | | Search In Folder | ![](https://github.com/ast-grep/ast-grep-vscode/blob/main/readme/search-in-folder.png?raw=true) | ### Structural Replace [​](https://ast-grep.github.io/guide/tools/editors.html#structural-replace) Use pattern to [replace](https://ast-grep.github.io/guide/rewrite-code.html) matching code. | Feature | Screenshot | | --- | --- | | Replace Preview | ![](https://github.com/ast-grep/ast-grep-vscode/blob/main/readme/replace.png?raw=true) | | Commit Replace | ![](https://github.com/ast-grep/ast-grep-vscode/blob/main/readme/commit-replace.png?raw=true) | ### Diagnostics and Code Action [​](https://ast-grep.github.io/guide/tools/editors.html#diagnostics-and-code-action) _Require LSP setup_ Code linting and code actions require [setting up `sgconfig.yml`](https://ast-grep.github.io/guide/scan-project.html) in your workspace root. | Feature | Screenshot | | --- | --- | | Code Linting | ![](https://github.com/ast-grep/ast-grep-vscode/blob/main/readme/linter.png?raw=true) | ### FAQs [​](https://ast-grep.github.io/guide/tools/editors.html#faqs) #### Why LSP diagnostics are not working? [​](https://ast-grep.github.io/guide/tools/editors.html#why-lsp-diagnostics-are-not-working) You need several things to set up LSP diagnostics: 1. [Install](https://ast-grep.github.io/guide/quick-start.html#installation) ast-grep CLI. Make sure it is accessible in VSCode editor. 2. [Set up a linting project](https://ast-grep.github.io/guide/scan-project.html) in your workspace root. The `sgconfig.yml` file is required for LSP diagnostics. 3. The LSP server by default is started in the workspace root. Make sure the `sgconfig.yml` is in the workspace root. #### Why ast-grep VSCode cannot find the CLI? [​](https://ast-grep.github.io/guide/tools/editors.html#why-ast-grep-vscode-cannot-find-the-cli) The extension has a different environment from the terminal. You need to make sure the CLI is accessible in the extension environment. For example, if the CLI is installed in a virtual environment, you need to activate the virtual environment in the terminal where you start VSCode. Here are a few ways to make the CLI accessible: 1. Install the CLI globally. 2. Specify the CLI path in the extension settings `astGrep.serverPath`. 3. Check if VSCode has the same `PATH` as the terminal. #### Project Root Detection [​](https://ast-grep.github.io/guide/tools/editors.html#project-root-detection) By default, ast-grep will only start in the workspace root. If you want to start ast-grep in a subfolder, you can specify the `configPath` in the extension settings. The `configPath` is the path to the `sgconfig.yml` file and is relative to the workspace root. #### Schema Validation [​](https://ast-grep.github.io/guide/tools/editors.html#schema-validation) When writing your own `rule.yml` file, you can use schema validation to get quick feedback on whether your file is structured properly. 1. Add the following line to the top of your file: yaml # yaml-language-server: $schema=https://raw.githubusercontent.com/ast-grep/ast-grep/main/schemas/rule.json 2. Install a VSCode extension that supports schema validation for yaml files. For example, [YAML by Red Hat](https://marketplace.visualstudio.com/items?itemName=redhat.vscode-yaml) . ![Schema Validation](https://ast-grep.github.io/image/schema-validation.png) After reloading the VSCode window, you should see red underlines for any errors in your `rule.yml` file, along with autocompletions and tooltips on hover. In VSCode you can typically use \[Ctrl\] + \[Space\] to see the available autocompletions. Neovim [​](https://ast-grep.github.io/guide/tools/editors.html#neovim) ----------------------------------------------------------------------- ### nvim-lspconfig [​](https://ast-grep.github.io/guide/tools/editors.html#nvim-lspconfig) The recommended setup is using [nvim-lspconfig](https://github.com/neovim/nvim-lspconfig) . lua require('lspconfig').ast_grep.setup({ -- these are the default options, you only need to specify -- options you'd like to change from the default cmd = { 'ast-grep', 'lsp' }, filetypes = { "c", "cpp", "rust", "go", "java", "python", "javascript", "typescript", "html", "css", "kotlin", "dart", "lua" }, root_dir = require('lspconfig.util').root_pattern('sgconfig.yaml', 'sgconfig.yml') }) ### coc.nvim [​](https://ast-grep.github.io/guide/tools/editors.html#coc-nvim) Please see [coc-ast-grep](https://github.com/yaegassy/coc-ast-grep) You need to have coc.nvim installed for this extension to work. e.g. vim-plug: vim Plug 'yaegassy/coc-ast-grep', {'do': 'yarn install --frozen-lockfile'} ### telescope.nvim [​](https://ast-grep.github.io/guide/tools/editors.html#telescope-nvim) [telescope-sg](https://github.com/Marskey/telescope-sg) is the ast-grep picker for telescope.nvim. Usage: vim Telescope ast_grep [telescope-ast-grep.nvim](https://github.com/ray-x/telescope-ast-grep.nvim) is an alternative plugin that provides ast-grep functionality enhancements. ### grug-far.nvim [​](https://ast-grep.github.io/guide/tools/editors.html#grug-far-nvim) [grug-far.nvim](https://github.com/MagicDuck/grug-far.nvim) has ast-grep search engine support. It allows for both live searching as you type and replacing. Usage: vim :lua require('grug-far').grug_far({ engine = 'astgrep' }) or swap to `astgrep` engine while running with the `Swap Engine` action. Emacs [​](https://ast-grep.github.io/guide/tools/editors.html#emacs) --------------------------------------------------------------------- ### ast-grep.el [​](https://ast-grep.github.io/guide/tools/editors.html#ast-grep-el) [ast-grep.el](https://github.com/SunskyXH/ast-grep.el) is an emacs package for searching code using ast-grep with completing-read interface or consult. You can install via `straight.el` elisp (straight-use-package '(ast-grep :type git :host github :repo "SunskyXH/ast-grep.el")) Or if you are using doomemacs, add to your `packages.el` elisp (package! ast-grep :recipe (:host github :repo "SunskyXH/ast-grep.el")) LSP Server [​](https://ast-grep.github.io/guide/tools/editors.html#lsp-server) ------------------------------------------------------------------------------- Currently ast-grep support these LSP capabilities: ### Server capability [​](https://ast-grep.github.io/guide/tools/editors.html#server-capability) * [publish diagnostics](https://microsoft.github.io/language-server-protocol/specifications/lsp/3.17/specification/#textDocument_publishDiagnostics) * [Fix diagnostic code action](https://microsoft.github.io/language-server-protocol/specifications/lsp/3.17/specification/#textDocument_publishCodeAction) ### Client requirements [​](https://ast-grep.github.io/guide/tools/editors.html#client-requirements) * [textDocument/didOpen](https://microsoft.github.io/language-server-protocol/specifications/lsp/3.17/specification/#textDocument_didOpen) * [textDocument/didChange](https://microsoft.github.io/language-server-protocol/specifications/lsp/3.17/specification/#textDocument_didChange) * [textDocument/didClose](https://microsoft.github.io/language-server-protocol/specifications/lsp/3.17/specification/#textDocument_didClose) ### Configuration [​](https://ast-grep.github.io/guide/tools/editors.html#configuration) ast-grep does not have LSP configuration, except that ast-grep LSP requires `sgconfig.yml` in the project root. You can also specify the configuration file path via command line: bash ast-grep lsp -c More Editors... [​](https://ast-grep.github.io/guide/tools/editors.html#more-editors) -------------------------------------------------------------------------------------- More ast-grep editor integration will be supported by the community! Your contribution is warmly welcome. --- # Rule Essentials | ast-grep [Skip to content](https://ast-grep.github.io/guide/rule-config.html#VPContent) Return to top Are you an LLM? You can read better optimized documentation at /guide/rule-config.md for this page in Markdown format Rule Essentials [​](https://ast-grep.github.io/guide/rule-config.html#rule-essentials) ======================================================================================= Now you have learnt the basic of ast-grep's pattern syntax and searching. Pattern is a handy feature for simple search. But it is not expressive enough for more complicated cases. ast-grep provides a more sophisticated way to find your code: Rule. Rules are like [CSS selectors](https://www.w3schools.com/cssref/css_selectors.php) that can compose together to filter AST nodes based on certain criteria. A Minimal Example [​](https://ast-grep.github.io/guide/rule-config.html#a-minimal-example) ------------------------------------------------------------------------------------------- A minimal ast-grep rule looks like this. yaml id: no-await-in-promise-all language: TypeScript rule: pattern: Promise.all($A) has: pattern: await $_ stopBy: end The _TypeScript_ rule, _no-await-in-promise-all_, will find `Promise.all` that **has** `await` expression in it. It is [suboptimal](https://github.com/hugo-vrijswijk/eslint-plugin-no-await-in-promise/) because `Promise.all` will be called [only after](https://twitter.com/hd_nvim/status/1560108625460355073) the awaited Promise resolves first. Let's walk through the main fields in this configuration. * `id` is a unique short string for the rule. * `language` is the programming language that the rule is intended to check. It specifies what files will be checked against this rule, based on the file extensions. See the list of [supported languages](https://ast-grep.github.io/reference/languages.html) . * `rule` is the most interesting part of ast-grep's configuration. It accepts a [rule object](https://ast-grep.github.io/reference/rule.html) and defines how the rule behaves and what code will be matched. You can learn how to write rule in the [detailed guide](https://ast-grep.github.io/guide/rule-config/atomic-rule.html) . Run the Rule [​](https://ast-grep.github.io/guide/rule-config.html#run-the-rule) --------------------------------------------------------------------------------- There are several ways to run the rule. We will illustrate several ast-grep features here. ### `ast-grep scan --rule` [​](https://ast-grep.github.io/guide/rule-config.html#ast-grep-scan-rule) The `scan` subcommand of ast-grep CLI can run one rule at a time. To do so, you need to save the rule above in a file on the disk, say `no-await-in-promise-all.yml`. Then you can run the following command to scan your codebase. In the example below, we are scanning a `test.ts` file. bashtypescript bash ast-grep scan --rule no-await-in-promise-all.yml test.ts typescript await Promise.all([\ await foo(),\ ]) ### `ast-grep scan --inline-rules` [​](https://ast-grep.github.io/guide/rule-config.html#ast-grep-scan-inline-rules) You can also run the rule directly from the command line without saving the rule to a file. The `--inline-rules` option is useful for ad-hoc search or calling ast-grep from another program. The full inline-rules command bash ast-grep scan --inline-rules ' id: no-await-in-promise-all language: TypeScript rule: pattern: Promise.all($A) has: pattern: await $_ stopBy: end ' test.ts ### Online Playground [​](https://ast-grep.github.io/guide/rule-config.html#online-playground) ast-grep provides an online [playground](https://ast-grep.github.io/playground.html#eyJtb2RlIjoiQ29uZmlnIiwibGFuZyI6ImphdmFzY3JpcHQiLCJxdWVyeSI6IlByb21pc2UuYWxsKCRBKSIsInJld3JpdGUiOiIiLCJjb25maWciOiJpZDogbm8tYXdhaXQtaW4tcHJvbWlzZS1hbGxcbmxhbmd1YWdlOiBUeXBlU2NyaXB0XG5ydWxlOlxuICBwYXR0ZXJuOiBQcm9taXNlLmFsbCgkQSlcbiAgaGFzOlxuICAgIHBhdHRlcm46IGF3YWl0ICRfXG4gICAgc3RvcEJ5OiBlbmQiLCJzb3VyY2UiOiJQcm9taXNlLmFsbChbXG4gIGF3YWl0IFByb21pc2UucmVzb2x2ZSgxMjMpXG5dKSJ9) to test your rule. You can paste the rule configuration into the playground and see the matched code. The playground also has a share button that generates a link to share the rule with others. Rule Object [​](https://ast-grep.github.io/guide/rule-config.html#rule-object) ------------------------------------------------------------------------------- _Rule object is the core concept of ast-grep's rule system and every other features are built on top of it._ Below is the full list of fields in a rule object. Every rule field is optional and can be omitted but at least one field should be present in a rule. A node will match a rule if and only if it satisfies all fields in the rule object. The equivalent rule object interface in TypeScript is also provided for reference. Full Rule ObjectTS Interface yaml rule: # atomic rule pattern: 'search.pattern' kind: 'tree_sitter_node_kind' regex: 'rust|regex' # relational rule inside: { pattern: 'sub.rule' } has: { kind: 'sub_rule' } follows: { regex: 'can|use|any' } precedes: { kind: 'multi_keys', pattern: 'in.sub' } # composite rule all: [ {pattern: 'match.all'}, {kind: 'match_all'} ] any: [ {pattern: 'match.any'}, {kind: 'match_any'} ] not: { pattern: 'not.this' } matches: 'utility-rule' typescript interface RuleObject { // atomic rule pattern?: string | Pattern kind?: string regex?: string // relational rule inside?: RuleObject & Relation has?: RuleObject & Relation follows?: RuleObject & Relation precedes?: RuleObject & Relation // composite rule all?: RuleObject[] any?: RuleObject[] not?: RuleObject matches?: string } // See Atomic rule for explanation interface Pattern { context: string selector: string strictness?: Strictness } // See https://ast-grep.github.io/advanced/match-algorithm.html type Strictness = | 'cst' | 'smart' | 'ast' | 'relaxed' | 'signature' // See Relation rule for explanation interface Relation { stopBy?: 'neighbor' | 'end' | RuleObject field?: string } A node must **satisfies all fields** in the rule object to be considered as a match. So the rule object can be seen as an abbreviated and **unordered** `all` rule. Rule object is unordered!! Unordered rule object means that certain rules may be applied before others, even if they appear later in the YAML. Whether a node matches or not may depend on the order of rule being applied, especially when using `has`/`inside` rules. If a rule object does not work, you can try using `all` rule to specify the order of rules. See [FAQ](https://ast-grep.github.io/advanced/faq.html#why-is-rule-matching-order-sensitive) for more details. Three Rule Categories [​](https://ast-grep.github.io/guide/rule-config.html#three-rule-categories) --------------------------------------------------------------------------------------------------- To summarize the rule object fields above, we have three categories of rules: * **Atomic Rule**: the most basic rule that checks if AST nodes matches. * **Relational Rule**: rules that check if a node is surrounded by another node. * **Composite Rule**: rules that combine sub-rules together using logical operators. These three categories of rules can be composed together to create more complex rules. The _rule object is inspired by the CSS selectors_ but with more composability and expressiveness. Think about how selectors in CSS works can help you understand the rule object! TIP Don't be daunted! Learn more about how to write a rule in our [detailed guide](https://ast-grep.github.io/guide/rule-config/atomic-rule.html) . Target Node [​](https://ast-grep.github.io/guide/rule-config.html#target-node) ------------------------------------------------------------------------------- Every rule configuration will have one single root `rule`. The root rule will have _only one_ AST node in one match. The matched node is called target node. During scanning and rewriting, ast-grep will produce multiple matches to report all AST nodes that satisfies the `rule` condition as matched instances. Though one rule match only have one AST node as matched, we can have more auxiliary nodes to display context or to perform rewrite. We will cover how rules work in details in the next page. But for a quick primer, a rule can have a pattern and we can extract meta variables from the matched node. For example, the rule below will match the `console.log('Hello World')`. yaml rule: pattern: console.log($GREET) And we can get `$GREET` set to `'Hello World'`. `language` specifies `rule` interpretation [​](https://ast-grep.github.io/guide/rule-config.html#language-specifies-rule-interpretation) ----------------------------------------------------------------------------------------------------------------------------------------- The `language` field in the rule configuration will specify how the rule is interpreted. For example, with `language: TypeScript`, the rule pattern `'hello world'` is parsed as TypeScript string literal. However, the rule will have a parsing error in languages like C/Java/Rust because single quote is used for character literal and double quote should be used for string. --- # Composite Rule | ast-grep [Skip to content](https://ast-grep.github.io/guide/rule-config/composite-rule.html#VPContent) Return to top Are you an LLM? You can read better optimized documentation at /guide/rule-config/composite-rule.md for this page in Markdown format Composite Rule [​](https://ast-grep.github.io/guide/rule-config/composite-rule.html#composite-rule) ==================================================================================================== Composite rule can accept another rule or a list of rules recursively. It provides a way to compose atomic rules into a bigger rule for more complex matching. Below are the four composite rule operators available in ast-grep: `all`, `any`, `not`, and `matches`. `all` [​](https://ast-grep.github.io/guide/rule-config/composite-rule.html#all) -------------------------------------------------------------------------------- `all` accepts a list of rules and will match AST nodes that satisfy all the rules. Example([playground](https://ast-grep.github.io/playground.html#eyJtb2RlIjoiQ29uZmlnIiwibGFuZyI6InR5cGVzY3JpcHQiLCJxdWVyeSI6IiRDOiAkVCA9IHJlbGF0aW9uc2hpcCgkJCRBLCB1c2VsaXN0PVRydWUsICQkJEIpIiwicmV3cml0ZSI6IiRDOiBMaXN0WyRUXSA9IHJlbGF0aW9uc2hpcCgkJCRBLCB1c2VsaXN0PVRydWUsICQkJEIpIiwiY29uZmlnIjoiaWQ6IG5vLWF3YWl0LWluLWxvb3Bcbmxhbmd1YWdlOiBUeXBlU2NyaXB0XG5ydWxlOlxuICBhbGw6XG4gICAgLSBwYXR0ZXJuOiBjb25zb2xlLmxvZygnSGVsbG8gV29ybGQnKTtcbiAgICAtIGtpbmQ6IGV4cHJlc3Npb25fc3RhdGVtZW50Iiwic291cmNlIjoiY29uc29sZS5sb2coJ0hlbGxvIFdvcmxkJyk7IC8vIG1hdGNoXG52YXIgcmV0ID0gY29uc29sZS5sb2coJ0hlbGxvIFdvcmxkJyk7IC8vIG5vIG1hdGNoIn0=) ): yaml rule: all: - pattern: console.log('Hello World'); - kind: expression_statement The above rule will only match a single line statement with content `console.log('Hello World');`. But not `var ret = console.log('Hello World');` because the `console.log` call is not a statement. We can read the rule as "matches code that is both an expression statement and has content `console.log('Hello World')`". Pro Tip `all` rule guarantees the order of rule matching. If you use pattern with [meta variables](https://ast-grep.github.io/guide/pattern-syntax.html#meta-variable-capturing) , make sure to use `all` array to guarantee rule execution order. `any` [​](https://ast-grep.github.io/guide/rule-config/composite-rule.html#any) -------------------------------------------------------------------------------- `any` accepts a list of rules and will match AST nodes as long as they satisfy any one of the rules. Example([playground](https://ast-grep.github.io/playground.html#eyJtb2RlIjoiQ29uZmlnIiwibGFuZyI6InR5cGVzY3JpcHQiLCJxdWVyeSI6IiRDOiAkVCA9IHJlbGF0aW9uc2hpcCgkJCRBLCB1c2VsaXN0PVRydWUsICQkJEIpIiwicmV3cml0ZSI6IiRDOiBMaXN0WyRUXSA9IHJlbGF0aW9uc2hpcCgkJCRBLCB1c2VsaXN0PVRydWUsICQkJEIpIiwiY29uZmlnIjoibGFuZ3VhZ2U6IFR5cGVTY3JpcHRcbnJ1bGU6XG4gIGFueTpcbiAgICAtIHBhdHRlcm46IHZhciBhID0gJEFcbiAgICAtIHBhdHRlcm46IGNvbnN0IGEgPSAkQVxuICAgIC0gcGF0dGVybjogbGV0IGEgPSAkQSIsInNvdXJjZSI6InZhciBhID0gMVxuY29uc3QgYSA9IDEgXG5sZXQgYSA9IDFcblxuIn0=) ): yaml rule: any: - pattern: var a = $A - pattern: const a = $A - pattern: let a = $A The above rule will match any variable declaration statement, like `var a = 1`, `const a = 1` and `let a = 1`. `not` [​](https://ast-grep.github.io/guide/rule-config/composite-rule.html#not) -------------------------------------------------------------------------------- `not` accepts a single rule and will match AST nodes that do not satisfy the rule. Combining `not` rule and `all` can help us to filter out some unwanted matches. Example([playground](https://ast-grep.github.io/playground.html#eyJtb2RlIjoiQ29uZmlnIiwibGFuZyI6InR5cGVzY3JpcHQiLCJxdWVyeSI6IiRDOiAkVCA9IHJlbGF0aW9uc2hpcCgkJCRBLCB1c2VsaXN0PVRydWUsICQkJEIpIiwicmV3cml0ZSI6IiRDOiBMaXN0WyRUXSA9IHJlbGF0aW9uc2hpcCgkJCRBLCB1c2VsaXN0PVRydWUsICQkJEIpIiwiY29uZmlnIjoibGFuZ3VhZ2U6IFR5cGVTY3JpcHRcbnJ1bGU6XG4gIHBhdHRlcm46IGNvbnNvbGUubG9nKCRHUkVFVElORylcbiAgbm90OlxuICAgIHBhdHRlcm46IGNvbnNvbGUubG9nKCdIZWxsbyBXb3JsZCcpIiwic291cmNlIjoiY29uc29sZS5sb2coJ2hpJylcbmNvbnNvbGUubG9nKCdIZWxsbyBXb3JsZCcpIn0=) ): yaml rule: pattern: console.log($GREETING) not: pattern: console.log('Hello World') The above rule will match any `console.log` call but not `console.log('Hello World')`. `matches` [​](https://ast-grep.github.io/guide/rule-config/composite-rule.html#matches) ---------------------------------------------------------------------------------------- `matches` is a special composite rule that takes a rule-id string. The rule-id can refer to a local utility rule defined in the same configuration file or to a global utility rule defined in the global utility rule files under separate directory. The rule will match the same nodes that the utility rule matches. `matches` rule enable us to reuse rules and even unlock the possibility of recursive rule. It is the most powerful rule in ast-grep and deserves a separate page to explain it. Please see the [dedicated page](https://ast-grep.github.io/guide/rule-config/utility-rule.html) for `matches`. `all` and `any` Refers to Rules, Not Nodes [​](https://ast-grep.github.io/guide/rule-config/composite-rule.html#all-and-any-refers-to-rules-not-nodes) ------------------------------------------------------------------------------------------------------------------------------------------------------- `all` mean that a node should **satisfy all the rules**. `any` means that a node should **satisfy any one of the rules**. It does not mean `all` or `any` nodes matching the rules. For example, the rule `all: [kind: number, kind: string]` will never match any node because a node cannot be both a number and a string at the same time. New ast-grep users may think this rule should all nodes that are either a number or a string, but it is not the case. The correct rule should be `any: [kind: number, kind: string]`. Another example is to match a node that has both `number` child and `string` child. It is extremely easy to [write a rule](https://ast-grep.github.io/playground.html#eyJtb2RlIjoiQ29uZmlnIiwibGFuZyI6ImphdmFzY3JpcHQiLCJxdWVyeSI6ImE6IExpc3RbJEJdIiwicmV3cml0ZSI6Imxpc3RbJEJdIiwic3RyaWN0bmVzcyI6InNtYXJ0Iiwic2VsZWN0b3IiOiJnZW5lcmljX3R5cGUiLCJjb25maWciOiJydWxlOlxuICBraW5kOiBhcmd1bWVudHNcbiAgaGFzOlxuICAgIGFsbDogW3traW5kOiBudW1iZXJ9LCB7IGtpbmQ6IHN0cmluZ31dIiwic291cmNlIjoibG9nKCdzdHInLCAxMjMpIn0=) like below yaml has: all: [kind: number, kind: string] It is very tempting to think that this rule will work. However, `all` rule works independently and does not rely on its containing rule `has`. Since the `all` rule matches no node, the `has` rule will also match no node. **An ast-grep rule tests one node at a time, independently.** A rule can never test multiple nodes at once. So the rule above means _"match a node has a child that is both a number and a string at the same time"_, which is impossible. Instead we should search _"a node that has a number child and has a string child"_. Here is [the correct rule](https://ast-grep.github.io/playground.html#eyJtb2RlIjoiQ29uZmlnIiwibGFuZyI6ImphdmFzY3JpcHQiLCJxdWVyeSI6ImE6IExpc3RbJEJdIiwicmV3cml0ZSI6Imxpc3RbJEJdIiwic3RyaWN0bmVzcyI6InNtYXJ0Iiwic2VsZWN0b3IiOiJnZW5lcmljX3R5cGUiLCJjb25maWciOiJydWxlOlxuICBraW5kOiBhcmd1bWVudHNcbiAgYWxsOlxuICAtIGhhczogeyBraW5kOiBudW1iZXIgfVxuICAtIGhhczogeyBraW5kOiBzdHJpbmcgfSIsInNvdXJjZSI6ImxvZygnc3RyJywgMTIzKSJ9) . Note `all` is used before `has`. yaml all: - has: {kind: number} - has: {kind: string} Composite rule is inspired by logical operator `and`/`or` and related list method like [`all`](https://doc.rust-lang.org/std/iter/trait.Iterator.html#method.all) /[`any`](https://doc.rust-lang.org/std/iter/trait.Iterator.html#method.any) . It tests whether a node matches all/any of the rules in the list. Combine Different Rules as Fields [​](https://ast-grep.github.io/guide/rule-config/composite-rule.html#combine-different-rules-as-fields) ------------------------------------------------------------------------------------------------------------------------------------------ Sometimes it is necessary to match node nested within other desired nodes. We can use composite rule `all` and relational `inside` to find them, but the result rule is highly nested. For example, we want to find the usage of `this.foo` in a class getter, we can write the following rule: yaml rule: all: - pattern: this.foo # the root node - inside: # inside another node all: - pattern: context: class A { get $_() { $$$ } } # a class getter inside selector: method_definition - inside: # class body kind: class_body stopBy: # but not inside nested any: - kind: object # either object - kind: class_body # or class See the [playground link](https://ast-grep.github.io/playground.html#eyJtb2RlIjoiQ29uZmlnIiwibGFuZyI6ImphdmFzY3JpcHQiLCJxdWVyeSI6ImNsYXNzIEEge1xuICAgIGdldCB0ZXN0KCkge31cbn0iLCJjb25maWciOiIjIENvbmZpZ3VyZSBSdWxlIGluIFlBTUxcbnJ1bGU6XG4gIGFsbDpcbiAgICAtIHBhdHRlcm46IHRoaXMuZm9vXG4gICAgLSBpbnNpZGU6XG4gICAgICAgIGFsbDpcbiAgICAgICAgICAtIHBhdHRlcm46XG4gICAgICAgICAgICAgIGNvbnRleHQ6IGNsYXNzIEEgeyBnZXQgJEdFVFRFUigpIHsgJCQkIH0gfVxuICAgICAgICAgICAgICBzZWxlY3RvcjogbWV0aG9kX2RlZmluaXRpb25cbiAgICAgICAgICAtIGluc2lkZTpcbiAgICAgICAgICAgICAgaW1tZWRpYXRlOiB0cnVlXG4gICAgICAgICAgICAgIGtpbmQ6IGNsYXNzX2JvZHlcbiAgICAgICAgc3RvcEJ5OlxuICAgICAgICAgIGFueTpcbiAgICAgICAgICAgIC0ga2luZDogb2JqZWN0XG4gICAgICAgICAgICAtIGtpbmQ6IGNsYXNzX2JvZHkiLCJzb3VyY2UiOiJjbGFzcyBBIHtcbiAgZ2V0IHRlc3QoKSB7XG4gICAgdGhpcy5mb29cbiAgICBsZXQgbm90VGhpcyA9IHtcbiAgICAgIGdldCB0ZXN0KCkge1xuICAgICAgICB0aGlzLmZvb1xuICAgICAgfVxuICAgIH1cbiAgfVxuICBub3RUaGlzKCkge1xuICAgIHRoaXMuZm9vXG4gIH1cbn1cbmNvbnN0IG5vdFRoaXMgPSB7XG4gIGdldCB0ZXN0KCkge1xuICAgIHRoaXMuZm9vXG4gIH1cbn0ifQ==) . To avoid such nesting-hell code (remember [callback hell](http://callbackhell.com/) ?), we can use combine different rules as fields into one rule object. A rule object can have all the atomic/relational/composite rule fields because they have different names. A node will match the rule object if and only if all the rules in its fields match the node. Put in another way, they are equivalent to having an `all` rule with sub rules mentioned in fields. For example, consider this rule. yaml pattern: this.foo inside: kind: class_body It is equivalent to the `all` rule, regardless of the rule order. yaml all: - pattern: this.foo - inside: kind: class_body Back to our `this.foo` in getter example, we can rewrite the rule as below. yaml rule: pattern: this.foo inside: pattern: context: class A { get $GETTER() { $$$ } } selector: method_definition inside: kind: class_body stopBy: any: - kind: object - kind: class_body It has less indentation than before. See the rewritten rule [in action](https://ast-grep.github.io/playground.html#eyJtb2RlIjoiQ29uZmlnIiwibGFuZyI6ImphdmFzY3JpcHQiLCJxdWVyeSI6ImNsYXNzIEEge1xuICAgIGdldCB0ZXN0KCkge31cbn0iLCJjb25maWciOiIjIENvbmZpZ3VyZSBSdWxlIGluIFlBTUxcbnJ1bGU6XG4gIHBhdHRlcm46IHRoaXMuZm9vXG4gIGluc2lkZTpcbiAgICBwYXR0ZXJuOlxuICAgICAgY29udGV4dDogY2xhc3MgQSB7IGdldCAkR0VUVEVSKCkgeyAkJCQgfSB9XG4gICAgICBzZWxlY3RvcjogbWV0aG9kX2RlZmluaXRpb25cbiAgICBpbnNpZGU6XG4gICAgICAgIGltbWVkaWF0ZTogdHJ1ZVxuICAgICAgICBraW5kOiBjbGFzc19ib2R5XG4gICAgc3RvcEJ5OlxuICAgICAgYW55OlxuICAgICAgICAtIGtpbmQ6IG9iamVjdFxuICAgICAgICAtIGtpbmQ6IGNsYXNzX2JvZHkiLCJzb3VyY2UiOiJjbGFzcyBBIHtcbiAgZ2V0IHRlc3QoKSB7XG4gICAgdGhpcy5mb29cbiAgICBsZXQgbm90VGhpcyA9IHtcbiAgICAgIGdldCB0ZXN0KCkge1xuICAgICAgICB0aGlzLmZvb1xuICAgICAgfVxuICAgIH1cbiAgfVxuICBub3RUaGlzKCkge1xuICAgIHRoaXMuZm9vXG4gIH1cbn1cbmNvbnN0IG5vdFRoaXMgPSB7XG4gIGdldCB0ZXN0KCkge1xuICAgIHRoaXMuZm9vXG4gIH1cbn0ifQ==) . Rule object does not guarantee rule matching order Rule object does not guarantee the order of rule matching. It is possible that the `inside` rule matches before the `pattern` rule in the example above. Rule order is not important if rules are completely independent. However, matching metavariable in patterns depends on the result of previous pattern matching. If you use pattern with [meta variables](https://ast-grep.github.io/guide/pattern-syntax.html#meta-variable-capturing) , make sure to use `all` array to guarantee rule execution order. --- # Atomic Rule | ast-grep [Skip to content](https://ast-grep.github.io/guide/rule-config/atomic-rule.html#VPContent) Return to top Are you an LLM? You can read better optimized documentation at /guide/rule-config/atomic-rule.md for this page in Markdown format Atomic Rule [​](https://ast-grep.github.io/guide/rule-config/atomic-rule.html#atomic-rule) =========================================================================================== ast-grep has three categories of rules. Let's start with the most basic one: atomic rule. Atomic rule defines the most basic matching rule that determines whether one syntax node matches the rule or not. There are five kinds of atomic rule: `pattern`, `kind`, `regex`, `nthChild` and `range`. `pattern` [​](https://ast-grep.github.io/guide/rule-config/atomic-rule.html#pattern) ------------------------------------------------------------------------------------- Pattern will match one single syntax node according to the [pattern syntax](https://ast-grep.github.io/guide/pattern-syntax.html) . yaml rule: pattern: console.log($GREETING) The above rule will match code like `console.log('Hello World')`. By default, a _string_ `pattern` is parsed and matched as a whole. ### Pattern Object [​](https://ast-grep.github.io/guide/rule-config/atomic-rule.html#pattern-object) It is not always possible to select certain code with a simple string pattern. A pattern code can be invalid, incomplete or ambiguous for the parser since it lacks context. For example, to select class field in JavaScript, writing `$FIELD = $INIT` will not work because it will be parsed as `assignment_expression`. See [playground](https://ast-grep.github.io/playground.html#eyJtb2RlIjoiUGF0Y2giLCJsYW5nIjoiamF2YXNjcmlwdCIsInF1ZXJ5IjoiJEZJRUxEID0gJElOSVQiLCJyZXdyaXRlIjoiRGVidWcuYXNzZXJ0IiwiY29uZmlnIjoicnVsZTpcbiAgcGF0dGVybjogXG4gICAgY29udGV4dDogJ3sgJE06ICgkJCRBKSA9PiAkTUFUQ0ggfSdcbiAgICBzZWxlY3RvcjogcGFpclxuIiwic291cmNlIjoiYSA9IDEyM1xuY2xhc3MgQSB7XG4gIGEgPSAxMjNcbn0ifQ==) . * * * We can also use an _object_ to specify a sub-syntax node to match within a larger context. It consists of an object with three properties: `context`, `selector` and `strictness`. * `context` (required): defines the surrounding code that helps to resolve any ambiguity in the syntax. * `selector` (optional): defines the sub-syntax node kind that is the actual matcher of the pattern. * `strictness` (optional): defines how strictly pattern will match against nodes. Let's see how pattern object can solve the ambiguity in the class field example above. The pattern object below instructs ast-grep to select the `field_definition` node as the pattern target. yaml pattern: selector: field_definition context: class A { $FIELD = $INIT } ast-grep works like this: 1. First, the code in `context`, `class A { $FIELD = $INIT }`, is parsed as a class declaration. 2. Then, it looks for the `field_definition` node, specified by `selector`, in the parsed tree. 3. The selected `$FIELD = $INIT` is matched against code as the pattern. In this way, the pattern is parsed as `field_definition` instead of `assignment_expression`. See [playground](https://ast-grep.github.io/playground.html#eyJtb2RlIjoiQ29uZmlnIiwibGFuZyI6ImphdmFzY3JpcHQiLCJxdWVyeSI6IiRGSUVMRCA9ICRJTklUIiwicmV3cml0ZSI6IkRlYnVnLmFzc2VydCIsImNvbmZpZyI6InJ1bGU6XG4gIHBhdHRlcm46XG4gICAgc2VsZWN0b3I6IGZpZWxkX2RlZmluaXRpb25cbiAgICBjb250ZXh0OiBjbGFzcyBBIHsgJEZJRUxEID0gJElOSVQgfVxuIiwic291cmNlIjoiYSA9IDEyM1xuY2xhc3MgQSB7XG4gIGEgPSAxMjNcbn0ifQ==) in action. Other examples are [function call in Go](https://github.com/ast-grep/ast-grep/issues/646) and [function parameter in Rust](https://github.com/ast-grep/ast-grep/issues/648) . ### `strictness` [​](https://ast-grep.github.io/guide/rule-config/atomic-rule.html#strictness) You can also use pattern object to control the matching strategy with `strictness` field. By default, ast-grep uses a smart strategy to match pattern against the AST node. All nodes in the pattern must be matched, but it will skip unnamed nodes in target code. For the definition of **_named_** and **_unnamed_** nodes, please refer to the [core concepts](https://ast-grep.github.io/advanced/core-concepts.html) doc. For example, the following pattern `function $A() {}` will match both plain function and async function in JavaScript. See [playground](https://ast-grep.github.io/playground.html#eyJtb2RlIjoiUGF0Y2giLCJsYW5nIjoiamF2YXNjcmlwdCIsInF1ZXJ5IjoiZnVuY3Rpb24gJEEoKSB7fSIsInJld3JpdGUiOiJEZWJ1Zy5hc3NlcnQiLCJjb25maWciOiJydWxlOlxuICBwYXR0ZXJuOiBcbiAgICBjb250ZXh0OiAneyAkTTogKCQkJEEpID0+ICRNQVRDSCB9J1xuICAgIHNlbGVjdG9yOiBwYWlyXG4iLCJzb3VyY2UiOiJmdW5jdGlvbiBhKCkge31cbmFzeW5jIGZ1bmN0aW9uIGEoKSB7fSJ9) js // function $A() {} function foo() {} // matched async function bar() {} // matched This is because the keyword `async` is an unnamed node in the AST, so the `async` in the code to search is skipped. As long as `function`, `$A` and `{}` are matched, the pattern is considered matched. However, this is not always the desired behavior. ast-grep provides `strictness` to control the matching strategy. At the moment, it provides these options, ordered from the most strict to the least strict: * `cst`: All nodes in the pattern and target code must be matched. No node is skipped. * `smart`: All nodes in the pattern must be matched, but it will skip unnamed nodes in target code. This is the default behavior. * `ast`: Only named AST nodes in both pattern and target code are matched. All unnamed nodes are skipped. * `relaxed`: Named AST nodes in both pattern and target code are matched. Comments and unnamed nodes are ignored. * `signature`: Only named AST nodes' kinds are matched. Comments, unnamed nodes and text are ignored. Deep Dive and More Examples `strictness` is an advanced feature that you may not need in most cases. If you are interested in more examples and details, please refer to the [deep dive](https://ast-grep.github.io/advanced/match-algorithm.html) doc on ast-grep's match algorithm. `kind` [​](https://ast-grep.github.io/guide/rule-config/atomic-rule.html#kind) ------------------------------------------------------------------------------- Sometimes it is not easy to write a pattern because it is hard to construct the valid syntax. For example, if we want to match class property declaration in JavaScript like `class A { a = 1 }`, writing `a = 1` will not match the property because it is parsed as assigning to a variable. Instead, we can use `kind` to specify the AST node type defined in [tree-sitter parser](https://tree-sitter.github.io/tree-sitter/using-parsers#named-vs-anonymous-nodes) . `kind` rule accepts the tree-sitter node's name, like `if_statement` and `expression`. You can refer to [ast-grep playground](https://ast-grep.github.io/playground.html) for relevant `kind` names. Back to our example, we can look up class property's kind from the playground. yaml rule: kind: field_definition It will match the following code successfully ([playground link](https://ast-grep.github.io/playground.html#eyJtb2RlIjoiQ29uZmlnIiwibGFuZyI6ImphdmFzY3JpcHQiLCJxdWVyeSI6ImEgPSAxMjMiLCJyZXdyaXRlIjoibG9nZ2VyLmxvZygkTUFUQ0gpIiwiY29uZmlnIjoiIyBDb25maWd1cmUgUnVsZSBpbiBZQU1MXG5ydWxlOlxuICBraW5kOiBmaWVsZF9kZWZpbml0aW9uIiwic291cmNlIjoiY2xhc3MgVGVzdCB7XG4gIGEgPSAxMjNcbn0ifQ==) ). js class Test { a = 123 // match this line } Here are some situations that you can effectively use `kind`: 1. Pattern code is ambiguous to parse, e.g. `{}` in JavaScript can be either object or code block. 2. It is too hard to enumerate all patterns of an AST kind node, e.g. matching all Java/TypeScript class declaration will need including all modifiers, generics, `extends` and `implements`. 3. Patterns only appear within specific context, e.g. the class property definition. `kind` + `pattern` is different from pattern object You may want to use `kind` to change how `pattern` is parsed. However, ast-grep rules are independent of each other. To change the parsing behavior of `pattern`, you should use pattern object with `context` and `selector` field. See [this FAQ](https://ast-grep.github.io/advanced/faq.html#kind-and-pattern-rules-are-not-working-together-why) . ### ESQuery style `kind` Experimental [​](https://ast-grep.github.io/guide/rule-config/atomic-rule.html#esquery-style-kind) From ast-grep v0.39.1, you can also use ESQuery style selector in `kind` to match AST nodes. This is an experimental feature and may change in the future. yaml rule: kind: call_expression > identifier This will match the `identifier` node that is a child of `call_expression` node. Internally, it will be converted to a [relational rule](https://ast-grep.github.io/guide/rule-config/relational-rule.html) `has`. Currently, the ESQuery style `kind` only supports the following selectors: * node kind: `identifier` * `>`: direct child selectors * `+`: next sibling selector * `~`: following sibling selector * : descendant selector If you want more selectors, please respond to [this issue on GitHub](https://github.com/ast-grep/ast-grep/issues/2127) . `regex` [​](https://ast-grep.github.io/guide/rule-config/atomic-rule.html#regex) --------------------------------------------------------------------------------- The `regex` atomic rule will match the AST node by its text against a Rust regular expression. yaml rule: regex: "\w+" TIP The regular expression is written in [Rust syntax](https://docs.rs/regex/latest/regex/) , not the popular [PCRE like syntax](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Guide/Regular_Expressions) . So some features are not available like arbitrary look-ahead and back references. You should almost always combine `regex` with other atomic rules to make sure the regular expression is applied to the correct AST node. Regex matching is quite expensive and cannot be optimized based on AST node kinds. While `kind` and `pattern` rules can be only applied to nodes with specific `kind_id` for optimized performance. TIP You can use [Rust‑style inline flags](https://docs.rs/regex/latest/regex/#grouping-and-flags) , for example: yaml rule: regex: "(?i)apple" This matches Apple as well as apple or APPLE. `nthChild` [​](https://ast-grep.github.io/guide/rule-config/atomic-rule.html#nthchild) --------------------------------------------------------------------------------------- `nthChild` is a rule to find nodes based on their indexes in the parent node's children list. In other words, it selects nodes based on their position among all sibling nodes within a parent node. It is very helpful in finding nodes without children or nodes appearing in specific positions. `nthChild` is heavily inspired by CSS's [`nth-child` pseudo-class](https://developer.mozilla.org/en-US/docs/Web/CSS/:nth-child) , and it accepts similar forms of arguments. yaml # a number to match the exact nth child nthChild: 3 # An+B style string to match position based on formula nthChild: 2n+1 # object style nthChild rule nthChild: # accepts number or An+B style string position: 2n+1 # optional, count index from the end of sibling list reverse: true # default is false # optional, filter the sibling node list based on rule ofRule: kind: function_declaration # accepts ast-grep rule TIP * `nthChild`'s index is 1-based, not 0-based, as in the CSS selector. * `nthChild`'s node list only includes named nodes, not unnamed nodes. **Example** The [following rule](https://ast-grep.github.io/playground.html#eyJtb2RlIjoiQ29uZmlnIiwibGFuZyI6ImphdmFzY3JpcHQiLCJxdWVyeSI6IiRGSUVMRCA9ICRJTklUIiwicmV3cml0ZSI6IkRlYnVnLmFzc2VydCIsImNvbmZpZyI6InJ1bGU6XG4gIGtpbmQ6IG51bWJlclxuICBudGhDaGlsZDogMiIsInNvdXJjZSI6IlsxLDIsM10ifQ==) will match the second number in the JavaScript array. yaml rule: kind: number nthChild: 2 It will match the following code: js const arr = [ 1, 2, 3, ] // |- match this number `range` [​](https://ast-grep.github.io/guide/rule-config/atomic-rule.html#range) --------------------------------------------------------------------------------- `range` is a rule to match nodes based on their position in the source code. It is useful when you want to integrate external tools like compilers or type checkers with ast-grep. External tools can provide the range information of the interested node, and ast-grep can use it to rewrite the code. `range` rule accepts a range object with `start` and `end` fields. Each field is an object with `line` and `column` fields. yaml rule: range: start: line: 0 column: 0 end: line: 1 column: 5 The above example will match an AST node having the first three characters of the first line like `foo` in `foo.bar()`. `line` and `column` are 0-based and character-wise, and the `start` is inclusive while the `end` is exclusive. Tips for Writing Rules [​](https://ast-grep.github.io/guide/rule-config/atomic-rule.html#tips-for-writing-rules) ----------------------------------------------------------------------------------------------------------------- Since one rule will have _only one_ AST node in one match, it is recommended to first write the atomic rule that matches the desired node. Suppose we want to write a rule which finds functions without a return type. For example, this code would trigger an error: ts const foo = () => { return 1; } The first step to compose a rule is to find the target. In this case, we can first use kind: `arrow_function` to find function node. Then we can use other rules to filter candidate nodes that does have return type. Another trick to write cleaner rule is to use sub-rules as fields. Please refer to [composite rule](https://ast-grep.github.io/guide/rule-config/composite-rule.html#combine-different-rules-as-fields) for more details. --- # Test Your Rule | ast-grep [Skip to content](https://ast-grep.github.io/guide/test-rule.html#VPContent) Return to top Are you an LLM? You can read better optimized documentation at /guide/test-rule.md for this page in Markdown format Test Your Rule [​](https://ast-grep.github.io/guide/test-rule.html#test-your-rule) =================================================================================== Though it is easy to write a simple rule to match some code in ast-grep, writing a robust and comprehensive rule to cover codebase in production is still a pretty challenging work. To alleviate this pain, ast-grep provides a builtin tool to help you test your rule. You can provide a list of `valid` cases and `invalid` cases to test against your rule. Basic Concepts [​](https://ast-grep.github.io/guide/test-rule.html#basic-concepts) ----------------------------------------------------------------------------------- Ideally, a perfect rule will approve all valid code and report issues only for all invalid code. Testing a rule should also cover two categories of code accordingly. If you are familiar with [detection theory](https://en.wikipedia.org/wiki/Detection_theory) , you should recognize that testing rule will involve the four scenarios tabulated below. | Code Validity \\ Rule Report | No Report | Has Report | | --- | --- | --- | | Valid | Validated | Noisy | | Invalid | Missing | Reported | * If ast-grep reports error for invalid code, it is a correct **reported** match. * If ast-grep reports error for valid code, it is called **noisy** match. * If ast-grep reports nothing for invalid code, we have a **missing** match. * If ast-grep reports nothing for valid code, it is called **validated** match. We will see these four case status in ast-grep's test output. Test Setup [​](https://ast-grep.github.io/guide/test-rule.html#test-setup) --------------------------------------------------------------------------- Let's write a test for the rule we wrote in the [previous section](https://ast-grep.github.io/guide/rule-config.html#rule-file) . To write a test, we first need to specify a rule test directory in `sgconfig.yml`. This directory will be used to store all test cases for rules. Suppose we have the `sgconfig.yml` as below. yaml ruleDirs: - rules # testConfigs contains a list of test directories for rules. testConfigs: - testDir: rule-tests The configuration file should be located at a directory that looks like this. bash my-awesome-rules/ |- rules/ | |- no-await-in-loop.yml # rule file |- rule-tests/ | |- no-await-in-loop-test.yml # test file |- sgconfig.yml `rules` folder contains all rule files, while `rule-tests` folder contains all test cases for rules. In the example, `no-await-in-loop.yml` contains the rule configuration we wrote before. Below are all relevant files used in this example. no-await-in-loop.ymlno-await-in-loop-test.ymlsgconfig.yml yaml id: no-await-in-loop message: Don't use await inside of loops severity: warning language: TypeScript rule: all: - inside: any: - kind: for_in_statement - kind: while_statement stopBy: end - pattern: await $_ yaml id: no-await-in-loop valid: - for (let a of b) { console.log(a) } # .... more valid test cases invalid: - async function foo() { for (var bar of baz) await bar; } # .... more invalid test cases yaml ruleDirs: - rules # testConfigs contains a list of test directories for rules. testConfigs: - testDir: rule-tests We will delve into `no-await-in-loop-test.yml` in next section. Test Case Configuration [​](https://ast-grep.github.io/guide/test-rule.html#test-case-configuration) ----------------------------------------------------------------------------------------------------- Test configuration file is very straightforward. It contains a list of `valid` and `invalid` cases with an `id` field to specify which rule will be tested against. `valid` is a list of source code that we **do not** expect the rule to report any issue. `invalid` is a list of source code that we **do** expect the rule to report some issues. yaml id: no-await-in-loop valid: - for (let a of b) { console.log(a) } # .... more valid test cases invalid: - async function foo() { for (var bar of baz) await bar; } # .... more invalid test cases After writing the test configuration file, you can run `ast-grep test` in the root folder to test your rule. We will discuss the `skip-snapshot-tests` option later. bash $ ast-grep test --skip-snapshot-tests Running 1 tests PASS no-await-in-loop ......................... test result: ok. 1 passed; 0 failed; ast-grep will report the passed rule and failed rule. The dots behind test case id represent passed cases. If we swap the test case and make them failed, we will get the following output. bash Running 1 tests FAIL no-await-in-loop ...........N............M ----------- Failure Details ----------- [Noisy] Expect no-await-in-loop to report no issue, but some issues found in: async function foo() { for (var bar of baz) await bar; } [Missing] Expect rule no-await-in-loop to report issues, but none found in: for (let a of b) { console.log(a) } Error: test failed. 0 passed; 1 failed; The output shows that we have two failed cases. One is a **noisy** match, which means ast-grep reports error for valid code. The other is a **missing** match, which means ast-grep reports nothing for invalid code. In the test summary, we can see the cases are marked with `N` and `M` respectively. In failure details, we can see the detailed code snippet for each case. Besides testing code validity, we can further test rule's output like error's message and span. This is what snapshot test will cover. Snapshot Test [​](https://ast-grep.github.io/guide/test-rule.html#snapshot-test) --------------------------------------------------------------------------------- Let's rerun `ast-grep test` without `--skip-snapshot-tests` option. This time we will get test failure that invalid code error does not have a matching snapshot. Previously we use the `skip-snapshot-tests` option to suppress snapshot test, which is useful when you are still working on your rule. But after the rule is polished, we can create snapshot to capture the desired output of the rule. The `--update-all` or `-U` will generate a snapshot directory for us. bash my-awesome-rules/ |- rules/ | |- no-await-in-loop.yml # test file |- rule-tests/ | |- no-await-in-loop-test.yml # rule file | |- __snapshots__/ # snapshots folder | | |- no-await-in-loop-snapshot.yml # generated snapshot file! |- sgconfig.yml The generated `__snapshots__` folder will store all the error output and later test run will match against them. After the snapshot is generated, we can run `ast-grep test` again, without any option this time, and pass all the test cases! Furthermore, when we change the rule or update the test case, we can use interactive mode to update the snapshot. Running this command bash ast-grep test --interactive ast-grep will spawn an interactive session to ask you select desired snapshot updates. Example interactive session will look like this. Note the snapshot diff is highlighted in red/green color. diff [Wrong] no-await-in-loop snapshot is different from baseline. Diff: labels: - source: await bar style: Primary - start: 2 + start: 28 end: 37 - source: do { await bar; } while (baz); style: Secondary For Code: async function foo() { do { await bar; } while (baz); } Accept new snapshot? (Yes[y], No[n], Accept All[a], Quit[q]) Pressing the `y` key will accept the new snapshot and update the snapshot file. --- # JSON Mode | ast-grep [Skip to content](https://ast-grep.github.io/guide/tools/json.html#VPContent) Return to top Are you an LLM? You can read better optimized documentation at /guide/tools/json.md for this page in Markdown format JSON Mode [​](https://ast-grep.github.io/guide/tools/json.html#json-mode) ========================================================================== Composability is a key perk of command line tooling. ast-grep is no exception. `--json` will output results in JSON format. This is useful to pipe the results to other tools. **Example:** bash ast-grep run -p 'Some($A)' -r 'None' --json Output Data Structure [​](https://ast-grep.github.io/guide/tools/json.html#output-data-structure) -------------------------------------------------------------------------------------------------- The format of the JSON output is an array of match objects. Below is an example of a match object generated from the command above. json [\ {\ "text": "Some(matched)",\ "range": {\ "byteOffset": { "start": 10828, "end": 10841 },\ "start": { "line": 303, "column": 2 },\ "end": { "line": 303, "column": 15 }\ },\ "file": "crates/config/src/rule/mod.rs",\ "lines": " Some(matched)",\ "replacement": "None",\ "replacementOffsets": { "start": 10828, "end": 10841 },\ "language": "Rust",\ "metaVariables": {\ "single": {\ "A": {\ "text": "matched",\ "range": {\ "byteOffset": { "start": 10833, "end": 10840 },\ "start": { "line": 303, "column": 7 },\ "end": { "line": 303, "column": 14 }\ }\ }\ },\ "multi": {},\ "transformed": {}\ }\ }\ ] ### Match Object Type [​](https://ast-grep.github.io/guide/tools/json.html#match-object-type) Below is the equivalent TypeScript type definition of the match object. typescript interface Match { text: string range: Range file: string // relative path to the file // the surrounding lines of the match. // It can be more than one line if the match spans multiple ones. lines: string // optional replacement if the match has a replacement replacement?: string replacementOffsets?: ByteOffset metaVariables?: MetaVariables // optional metavars generated in the match } interface Range { byteOffset: ByteOffset start: Position end: Position } // UTF-8 encoded byte offset interface ByteOffset { start: number // start is inclusive end: number // end is exclusive } interface Position { line: number // zero-based line number column: number // zero-based column number } // See Pattern doc interface MetaVariables { single: Record multi: Record transformed: Record // See Rewrite doc } interface MetaVar { text: string range: Range } For more information about `MetaVariables` and `transformed` fields, see the [Pattern](https://ast-grep.github.io/guide/pattern-syntax.html#meta-variable) and [Rewrite](https://ast-grep.github.io/guide/rewrite/transform.html) documentation. If you are using [lint rule](https://ast-grep.github.io/guide/project/lint-rule.html) to find matches, the generated match objects will have several more fields. typescript interface RuleMatch extends Match { ruleId: string severity: Severity note?: string message: string } enum Severity { Error = "error", Warning = "warning", Info = "info", Hint = "hint", } line, column, and byte offset are zero-based The `line`, `column`, and `byteOffset` fields are zero-based. This means that the first line, column, and byte offset are 0, not 1. The design is consistent with the [LSP](https://microsoft.github.io/language-server-protocol/specifications/lsp/3.17/specification/#position) and [tree-sitter](https://tree-sitter.github.io/tree-sitter/using-parsers#syntax-nodes) specifications. If you need 1-based numbers, you can use `jq` to transform the output. Consuming JSON output [​](https://ast-grep.github.io/guide/tools/json.html#consuming-json-output) -------------------------------------------------------------------------------------------------- ast-grep embraces the Unix philosophy of composability. The `--json` flag is designed to make it easy to pipe the results to other tools. For example, you can use [jq](https://stedolan.github.io/jq/) to extract information from the results and render it in [jless](https://jless.io/) . bash ast-grep run -p 'Some($A)' -r 'None' --json | jq '.[].replacement' | jless You can also see [an example](https://github.com/ast-grep/ast-grep/issues/1232#issuecomment-2181747911) of using `--json` flag in Vim's QuickFix window. Output Format [​](https://ast-grep.github.io/guide/tools/json.html#output-format) ---------------------------------------------------------------------------------- By default, ast-grep prints the matches in a JSON array that is formatted with indentation and line breaks. `--json` is equivalent to `--json=pretty`. This makes it easy to read the output by humans. However, this might not be suitable for other programs that need to process the output from ast-grep. For example, if there are too many matches, the JSON array might be [too large to fit in memory](https://www.wikiwand.com/en/Out_of_memory) . To avoid this problem, you can use the `--json=stream` option when running ast-grep. This option will make ast-grep print each match as a separate JSON object, followed by a newline character. This way, you can stream the output to other programs that can read one object per line and parse it accordingly. The output of `--json=stream` looks like below: $ ast-grep -p pattern --json=stream {"text":"Some(matched)", ... } {"text":"Some(matched)", ... } {"text":"Some(matched)", ... } You can read the output line by line and process it accordingly. `--json` accepts one of the following values: `pretty`, `stream`, or `compact`. `--json=stream` requires the equal sign You have to use `--json=