# Table of Contents

- [Introduction to the Sum-Check Protocol](#introduction-to-the-sum-check-protocol)
- [Strategy](#strategy)
- [Semiotic Labs](#semiotic-labs)
- [Products](#products)
- [Company history](#company-history)
- [Semiotic Labs | Articles](#semiotic-labs-articles)
- [Semiotic Labs | Videos](#semiotic-labs-videos)
- [Semiotic Labs | Podcasts](#semiotic-labs-podcasts)
- [Semiotic Labs | Articles](#semiotic-labs-articles)
- [Branding Assets](#branding-assets)
- [Semiotic Labs | Articles](#semiotic-labs-articles)
- [Semiotic Labs | Articles](#semiotic-labs-articles)
- [Semiotic Labs | Articles](#semiotic-labs-articles)
- [Semiotic Labs | Articles by Sam Green](#semiotic-labs-articles-by-sam-green)
- [Semiotic Labs | Podcasts](#semiotic-labs-podcasts)
- [2024 Crypto Meta-Analysis](#2024-crypto-meta-analysis)
- [Homomorphic Signatures for Payment Channels](#homomorphic-signatures-for-payment-channels)
- [Semiotic Labs | Articles](#semiotic-labs-articles)
- [PSI with FHE](#psi-with-fhe)
- [Semiotic Labs | Podcasts](#semiotic-labs-podcasts)
- [Indexer Allocation Optimization: Part II](#indexer-allocation-optimization-part-ii)
- [Semiotic Labs | Podcasts](#semiotic-labs-podcasts)
- [An Overview of Automatic Market Maker Mechanisms](#an-overview-of-automatic-market-maker-mechanisms)
- [Semiotic Labs | Podcasts](#semiotic-labs-podcasts)
- [Automated Query Pricing in The Graph](#automated-query-pricing-in-the-graph)
- [Indexer Allocation Optimization: Part I](#indexer-allocation-optimization-part-i)
- [Semiotic Labs | Videos](#semiotic-labs-videos)
- [Semiotic Labs | Videos](#semiotic-labs-videos)
- [Semiotic Labs | Videos](#semiotic-labs-videos)
- [Semiotic Labs | Videos](#semiotic-labs-videos)
- [Semiotic Labs | Podcasts](#semiotic-labs-podcasts)
- [Semiotic Labs | Articles by Carollan Helinski](#semiotic-labs-articles-by-carollan-helinski)
- [Semiotic Labs | Articles by Alexis Asseman](#semiotic-labs-articles-by-alexis-asseman)
- [Semiotic Labs | Articles by Lichu Acuña](#semiotic-labs-articles-by-lichu-acu-a)
- [Semiotic Labs | Videos](#semiotic-labs-videos)
- [Semiotic Labs | Articles by Matt Deible](#semiotic-labs-articles-by-matt-deible)
- [Semiotic Labs | Articles by Anirudh A. Patel](#semiotic-labs-articles-by-anirudh-a-patel)

---

# Introduction to the Sum-Check Protocol

![Gradient background](/_astro/curved_bg_desktop.ChjV8nJN_1c0N0M.svg) ![Gradient background](/_astro/curved_bg_mobile.DP8czMRd_1b6C73.svg)

### TL;DR

It is expensive to run transactions on the Ethereum EVM. Verifiable computing (VC) lets us outsource computing away from the EVM. Today, a popular and exciting form of VC algorithm is the SNARK. There are various families of SNARKs that use the sum-check protocol, which is a simple algorithm to introduce VC. This is a tutorial on the sum-check protocol. This post is focused on how the sum-check protocol is implemented - it does not go into theory. You can skip straight to the finished code [here](https://github.com/0xsamgreen/sumcheck)
. _Thank you to Gokay Saldamli, Gabriel Soule, and Tomasz Kornuta for providing valuable feedback on this article._

### Background

Executing code on Ethereum is expensive. Verifiable computing algorithms promise a way to reduce costs, by outsourcing computing to untrusted parties and only verifying the result on-chain. A key point about _useful_ verifiable computing algorithms is that the verification must be less expensive than the original computation. The sum-check protocol is a foundational algorithm in the field of verifiable computing. In a stand-alone setting, sum-check is not particularly useful. However, it is an important building block of more sophisticated and useful SNARK algorithms. This tutorial introduces sum-check as a way to familiarize the reader with a relatively simple verifiable computing algorithm.

The sum-check protocol is used to outsource the computation of the following sum:

H:\=∑x1∈0,1∑x2∈0,1⋯∑xv∈0,1g(x1,x2,⋯ ,xv)\\begin{equation\*} H := \\sum\_{x\_1 \\in {0,1}} \\sum\_{x\_2 \\in {0,1}} \\cdots \\sum\_{x\_v \\in {0,1}} g(x\_1, x\_2, \\cdots, x\_v) \\end{equation\*}H:=x1​∈0,1∑​x2​∈0,1∑​⋯xv​∈0,1∑​g(x1​,x2​,⋯,xv​)​

The sum-check protocol allows a Prover PPP to convince a Verifier VVV that PPP computed the sum HHH correctly. You can assume that PPP has ample computational power or ample time to compute, e.g., it could be an untrusted server somewhere on the internet, and VVV is has limited computational power, e.g., it could be the EVM.

Why would you want to sum a function over all Boolean inputs, as is done in the equation above? One reason would be if you wanted to provide an answer to the #SAT (“sharp sat”) problem. #SAT is concerned with counting the number of inputs to a Boolean-valued function that result in a true output. For example, what is the #SAT of f(x1,x2,x3)\=x1 AND x2 AND x3f(x\_1, x\_2, x\_3) = x\_1 \\text{ AND } x\_2 \\text{ AND } x\_3f(x1​,x2​,x3​)\=x1​ AND x2​ AND x3​? It is 1, because only a single input, x1\=1x\_1=1x1​\=1, x2\=1x\_2=1x2​\=1, x3\=1x\_3 = 1x3​\=1, results in 1 being output from fff. You can probably see that it would be computationally intensive for you to solve #SAT for complex problems. With the sum-check protocol, PPP can do that work instead of you.

Unless you are a complexity theorist, you may not get too excited about the #SAT problem at first glance. However, more practical problems can be mapped to #SAT, and then sum-check can be used. For example, given an adjacency matrix, you can use sum-check to solve for how many triangles are connected. You can also use sum-check to outsource matrix multiplication. Check out the book mentioned next for more details.

In this tutorial, we use the same example as given in Justin Thaler’s book: [Proofs, Arguments, and Zero-Knowledge](https://people.cs.georgetown.edu/jthaler/ProofsArgsAndZK.html)
. In Justin’s book, sum-check is used to solve #SAT for the following example function:

ϕ(x1,x2,x3,x4)\=(x1ˉ AND x2) AND (x3 OR x4)\\begin{equation\*} \\phi(x\_1, x\_2, x\_3, x\_4) = (\\bar{x\_1} \\text{ AND } x\_2) \\text{ AND } (x\_3 \\text{ OR } x\_4) \\end{equation\*}ϕ(x1​,x2​,x3​,x4​)\=(x1​ˉ​ AND x2​) AND (x3​ OR x4​)​

where x1ˉ\\bar{x\_1}x1​ˉ​ means NOT x1x\_1x1​. The function ϕ\\phiϕ (phi) can also be represented graphically as:

![](/_astro/image-7-1024x719.CcLHptxS_135Q5p.webp)

In the graph of ϕ\\phiϕ, we see the logic symbols for AND (∧\\wedge∧) and OR (∨\\vee∨). Note that ϕ\\phiϕ is only defined over the Boolean inputs (0/1) and that ϕ\\phiϕ has a Boolean output. However, most popular verifiable computing algorithms only support arithmetic operations, so we need to convert ϕ\\phiϕ to its arithmetic version. The arithmetized version is called ggg. For our example, ggg is defined as:

g(x1,x2,x3,x4)\=(1−x1)x2((x3+x4)−(x3x4))\\begin{equation\*} g(x\_1, x\_2, x\_3, x\_4) = (1-x\_1)x\_2((x\_3+x\_4)-(x\_3x\_4)) \\end{equation\*}g(x1​,x2​,x3​,x4​)\=(1−x1​)x2​((x3​+x4​)−(x3​x4​))​

and visualized as:

![](/_astro/image-8.de0q40k-_ZVuUk3.webp)

In ggg, notice that the Boolean functions: NOT, AND, OR have been compiled into their arithmetic equivalents. Take a moment to convince yourself that if you pick Boolean values for x1x\_1x1​, x2x\_2x2​, x3x\_3x3​, and x4x\_4x4​, that ϕ\\phiϕ and ggg will output the same thing.

For reference, here are the specific compiler rules used when converting ϕ\\phiϕ into ggg:

| Boolean gate | Arithmetized version |
| --- | --- |
| A AND B | A\*B |
| A OR B | (A+B)-(A\*B) |
| NOT A | 1-A |

Notice that ggg has extended the domain of our original Boolean formula and is now valid for Boolean inputs and integers. The facts that ggg equals ϕ\\phiϕ when evaluated at Boolean inputs and that integers are valid inputs to ggg are leveraged by the sum-check protocol.

### The sum-check protocol

This section introduces the sum-check protocol. We will follow the same steps and naming convention as used in [Justin’s book](https://people.cs.georgetown.edu/jthaler/ProofsArgsAndZK.html)
. The protocol steps are summarized as:

1.  Prover PPP calculates the total sum of ggg evaluated at all Boolean inputs
    
2.  PPP computes a partial sum of ggg, leaving the first variable x1x\_1x1​ free
    
3.  Verifier VVV checks that the partial sum and total sum agree when the partial sum is evaluated at 0 and 1 and its outputs added
    
4.  VVV picks a random number r1r\_1r1​ for the free variable and sends it to PPP
    
5.  PPP replaces the free variable x1x\_1x1​ with the random number r1r\_1r1​ and computes a partial sum leaving next variable x2x\_2x2​ free
    
6.  PPP and VVV repeat steps similar to 3—5 for the rest of the variables: x2,⋯ ,xvx\_2, \\cdots, x\_vx2​,⋯,xv​
    
7.  VVV evaluates ggg at one input using access to an “oracle”, i.e., VVV must make a singled trusted execution of g(r1,r2,⋯ ,rv)g(r\_1, r\_2, \\cdots, r\_v)g(r1​,r2​,⋯,rv​)
    

If VVV makes it through Step 7 without error, then VVV accepts PPP‘s proof. Otherwise, VVV rejects.

We expand the steps of the algorithm in the following sections.

##### 1\. PPP calculates the total sum of ggg evaluated at all Boolean inputs

HHH is the sum described in the first equation in this article. In round zero, PPP should perform the summation described in the first equation. PPP assigns this sum to g0g\_0g0​ and returns it to VVV. Note that this sum is the #SAT solution, i.e., it gives the answer of how many unique inputs to the function result in 1 being output from ggg. The remainder of the sum-check algorithm is designed to ensure that PPP performed the #SAT solution correctly.

In our example, g0\=3g\_0 = 3g0​\=3 will be returned in Step 1. The following table shows PPP‘s work where the must iterate over all possible Boolean combinations of x1x\_1x1​,x2x\_2x2​, x3x\_3x3​, and x4x\_4x4​ and evaluate g(x1,x2,x3,x4)\=(1−x1)x2((x3+x4)−(x3x4))g(x\_1, x\_2, x\_3, x\_4) = (1-x\_1)x\_2((x\_3+x\_4)-(x\_3x\_4))g(x1​,x2​,x3​,x4​)\=(1−x1​)x2​((x3​+x4​)−(x3​x4​)):

g(0,0,0,0)\=(1−0)⋅0⋅((0+0)−(0⋅0))\=0g(0,0,0,1)\=(1−0)⋅0⋅((0+1)−(0⋅1))\=0g(0,0,1,0)\=(1−0)⋅0⋅((1+0)−(1⋅0))\=0g(0,0,1,1)\=(1−0)⋅0⋅((1+1)−(1⋅1))\=0g(0,1,0,0)\=(1−0)⋅1⋅((0+0)−(0⋅0))\=0g(0,1,0,0)\=(1−0)⋅1⋅((0+1)−(0⋅1))\=1g(0,1,1,0)\=(1−0)⋅1⋅((1+0)−(1⋅0))\=1g(0,1,1,0)\=(1−0)⋅1⋅((1+1)−(1⋅1))\=1g(1,0,0,0)\=(1−1)⋅0⋅((0+1)−(0⋅1))\=0g(1,0,0,1)\=(1−1)⋅0⋅((0+1)−(0⋅1))\=0g(1,0,1,0)\=(1−1)⋅0⋅((1+0)−(1⋅0))\=0g(1,0,1,1)\=(1−1)⋅0⋅((1+1)−(1⋅1))\=0g(1,1,0,0)\=(1−1)⋅1⋅((0+0)−(0⋅0))\=0g(1,1,0,0)\=(1−1)⋅1⋅((0+1)−(0⋅1))\=0g(1,1,1,0)\=(1−1)⋅1⋅((1+0)−(1⋅0))\=0g(1,1,1,0)\=(1−1)⋅1⋅((1+1)−(1⋅1))\=0\\begin{aligned}g(0, 0, 0, 0) &= (1-0) \\cdot 0 \\cdot ((0 + 0)-(0 \\cdot 0)) = 0\\\\ g(0, 0, 0, 1) &= (1-0) \\cdot 0 \\cdot ((0 + 1)-(0 \\cdot 1)) = 0\\\\ g(0, 0, 1, 0) &= (1-0) \\cdot 0 \\cdot ((1 + 0)-(1 \\cdot 0)) = 0\\\\ g(0, 0, 1, 1) &= (1-0) \\cdot 0 \\cdot ((1 + 1)-(1 \\cdot 1)) = 0\\\\ g(0, 1, 0, 0) &= (1-0) \\cdot 1 \\cdot ((0 + 0)-(0 \\cdot 0)) = 0\\\\ g(0, 1, 0, 0) &= (1-0) \\cdot 1 \\cdot ((0 + 1)-(0 \\cdot 1)) = 1\\\\ g(0, 1, 1, 0) &= (1-0) \\cdot 1 \\cdot ((1 + 0)-(1 \\cdot 0)) = 1\\\\ g(0, 1, 1, 0) &= (1-0) \\cdot 1 \\cdot ((1 + 1)-(1 \\cdot 1)) = 1\\\\ g(1, 0, 0, 0) &= (1-1) \\cdot 0 \\cdot ((0 + 1)-(0 \\cdot 1)) = 0\\\\ g(1, 0, 0, 1) &= (1-1) \\cdot 0 \\cdot ((0 + 1)-(0 \\cdot 1)) = 0\\\\ g(1, 0, 1, 0) &= (1-1) \\cdot 0 \\cdot ((1 + 0)-(1 \\cdot 0)) = 0\\\\ g(1, 0, 1, 1) &= (1-1) \\cdot 0 \\cdot ((1 + 1)-(1 \\cdot 1)) = 0\\\\ g(1, 1, 0, 0) &= (1-1) \\cdot 1 \\cdot ((0 + 0)-(0 \\cdot 0)) = 0\\\\ g(1, 1, 0, 0) &= (1-1) \\cdot 1 \\cdot ((0 + 1)-(0 \\cdot 1)) = 0\\\\ g(1, 1, 1, 0) &= (1-1) \\cdot 1 \\cdot ((1 + 0)-(1 \\cdot 0)) = 0\\\\ g(1, 1, 1, 0) &= (1-1) \\cdot 1 \\cdot ((1 + 1)-(1 \\cdot 1)) = 0\\end{aligned}g(0,0,0,0)g(0,0,0,1)g(0,0,1,0)g(0,0,1,1)g(0,1,0,0)g(0,1,0,0)g(0,1,1,0)g(0,1,1,0)g(1,0,0,0)g(1,0,0,1)g(1,0,1,0)g(1,0,1,1)g(1,1,0,0)g(1,1,0,0)g(1,1,1,0)g(1,1,1,0)​\=(1−0)⋅0⋅((0+0)−(0⋅0))\=0\=(1−0)⋅0⋅((0+1)−(0⋅1))\=0\=(1−0)⋅0⋅((1+0)−(1⋅0))\=0\=(1−0)⋅0⋅((1+1)−(1⋅1))\=0\=(1−0)⋅1⋅((0+0)−(0⋅0))\=0\=(1−0)⋅1⋅((0+1)−(0⋅1))\=1\=(1−0)⋅1⋅((1+0)−(1⋅0))\=1\=(1−0)⋅1⋅((1+1)−(1⋅1))\=1\=(1−1)⋅0⋅((0+1)−(0⋅1))\=0\=(1−1)⋅0⋅((0+1)−(0⋅1))\=0\=(1−1)⋅0⋅((1+0)−(1⋅0))\=0\=(1−1)⋅0⋅((1+1)−(1⋅1))\=0\=(1−1)⋅1⋅((0+0)−(0⋅0))\=0\=(1−1)⋅1⋅((0+1)−(0⋅1))\=0\=(1−1)⋅1⋅((1+0)−(1⋅0))\=0\=(1−1)⋅1⋅((1+1)−(1⋅1))\=0​

The outputs of the above evaluations are added to arrive at g0\=3g\_0 = 3g0​\=3.

##### 2\. PPP computes a partial sum of ggg, leaving the first variable x1x\_1x1​ free

In Step 2, PPP computes a similar sum as they did in Step 1, except the first variable is left free/unassigned. I.e.,

g1(X1)\=∑x2∈{0,1}∑x3∈{0,1}⋯∑xv∈{0,1}g(X1,x2,⋯ ,xv)\\begin{equation\*} g\_1(X\_1) = \\sum\_{x\_2 \\in \\{0,1\\}} \\sum\_{x\_3 \\in \\{0,1\\}} \\cdots \\sum\_{x\_v \\in \\{0,1\\}} g(X\_1, x\_2, \\cdots, x\_v) \\end{equation\*}g1​(X1​)\=x2​∈{0,1}∑​x3​∈{0,1}∑​⋯xv​∈{0,1}∑​g(X1​,x2​,⋯,xv​)​

Note that g1g\_1g1​ is a monomial in X1X\_1X1​, and all other variables have been assigned values in each iteration of the loop. We refer to this monomial as the “partial sum” in the next step. Also observe that the only difference between the definition for g1g\_1g1​ and the definition for HHH is that the g1g\_1g1​ summation is missing ∑x1∈{0,1}\\sum\_{x\_1 \\in \\{0,1\\}}∑x1​∈{0,1}​.

For our example, g(x1,x2,x3,x4)\=(1−x1)x2((x3+x4)−(x3x4))g(x\_1, x\_2, x\_3, x\_4) = (1-x\_1)x\_2((x\_3+x\_4)-(x\_3x\_4))g(x1​,x2​,x3​,x4​)\=(1−x1​)x2​((x3​+x4​)−(x3​x4​)). To calculate g1(X1)g\_1(X\_1)g1​(X1​), we leave x1x\_1x1​ free and then calculate the sum of ggg evaluated at all possible Boolean combinations assigned to x2x\_2x2​, x3x\_3x3​, and x4x\_4x4​. The following table contains the intermediate calculations:

g(X1,0,0,0)\=(1−X1)⋅0⋅((0+0)−(0⋅0))\=0g(X1,0,0,1)\=(1−X1)⋅0⋅((0+1)−(0⋅1))\=0g(X1,0,1,0)\=(1−X1)⋅0⋅((1+0)−(1⋅0))\=0g(X1,0,1,1)\=(1−X1)⋅0⋅((1+1)−(1⋅1))\=0g(X1,1,0,0)\=(1−X1)⋅1⋅((0+0)−(0⋅0))\=0g(X1,1,0,0)\=(1−X1)⋅1⋅((0+1)−(0⋅1))\=(1−X1)g(X1,1,1,0)\=(1−X1)⋅1⋅((1+0)−(1⋅0))\=(1−X1)g(X1,1,1,0)\=(1−X1)⋅1⋅((1+1)−(1⋅1))\=(1−X1)\\begin{aligned}g(X1, 0, 0, 0) &= (1-X1) \\cdot 0 \\cdot ((0 + 0)-(0 \\cdot 0)) = 0\\\\ g(X1, 0, 0, 1) &= (1-X1) \\cdot 0 \\cdot ((0 + 1)-(0 \\cdot 1)) = 0\\\\ g(X1, 0, 1, 0) &= (1-X1) \\cdot 0 \\cdot ((1 + 0)-(1 \\cdot 0)) = 0\\\\ g(X1, 0, 1, 1) &= (1-X1) \\cdot 0 \\cdot ((1 + 1)-(1 \\cdot 1)) = 0\\\\ g(X1, 1, 0, 0) &= (1-X1) \\cdot 1 \\cdot ((0 + 0)-(0 \\cdot 0)) = 0\\\\ g(X1, 1, 0, 0) &= (1-X1) \\cdot 1 \\cdot ((0 + 1)-(0 \\cdot 1)) = (1-X1)\\\\ g(X1, 1, 1, 0) &= (1-X1) \\cdot 1 \\cdot ((1 + 0)-(1 \\cdot 0)) = (1-X1)\\\\ g(X1, 1, 1, 0) &= (1-X1) \\cdot 1 \\cdot ((1 + 1)-(1 \\cdot 1)) = (1-X1)\\end{aligned}g(X1,0,0,0)g(X1,0,0,1)g(X1,0,1,0)g(X1,0,1,1)g(X1,1,0,0)g(X1,1,0,0)g(X1,1,1,0)g(X1,1,1,0)​\=(1−X1)⋅0⋅((0+0)−(0⋅0))\=0\=(1−X1)⋅0⋅((0+1)−(0⋅1))\=0\=(1−X1)⋅0⋅((1+0)−(1⋅0))\=0\=(1−X1)⋅0⋅((1+1)−(1⋅1))\=0\=(1−X1)⋅1⋅((0+0)−(0⋅0))\=0\=(1−X1)⋅1⋅((0+1)−(0⋅1))\=(1−X1)\=(1−X1)⋅1⋅((1+0)−(1⋅0))\=(1−X1)\=(1−X1)⋅1⋅((1+1)−(1⋅1))\=(1−X1)​

After summing all the rows in the table, we get g1(X1)\=−3X1+3g\_1(X\_1) = -3X\_1 + 3g1​(X1​)\=−3X1​+3.

##### 3\. VVV checks that the partial sum and total sum agree when the partial sum is evaluated at 0 and 1 and its outputs added

In Step 3, VVV checks that PPP was consistent with what they committed in Step 0 and Step 1. Specifically, VVV checks the following:

g0\=?g1(0)+g1(1)\\begin{equation\*} g\_0 \\stackrel{?}{=} g\_1(0) + g\_1(1) \\end{equation\*}g0​\=?g1​(0)+g1​(1)​

In our example, PPP sent g0\=3g\_0 = 3g0​\=3 and g1(X1)\=−3x1+3g\_1(X\_1) = -3x\_1 + 3g1​(X1​)\=−3x1​+3 to VVV during Steps 1 and 2, respectively. So VVV must now check that 3\=?(−3⋅0+3)+(−3⋅1+3)3 \\stackrel{?}{=} (-3 \\cdot 0 + 3) + (-3 \\cdot 1 + 3)3\=?(−3⋅0+3)+(−3⋅1+3), which it does. If this check failed, then VVV would end the protocol, and PPP‘s claim for g0g\_0g0​ (which is the supposed answer to the #SAT problem) would be rejected.

Note here what is happening: in Step 1, PPP made a commitment to the #SAT answer. In Step 2, PPP also did almost all of the work for VVV, except VVV still needed to evaluate the monomial g1(X1)g\_1(X\_1)g1​(X1​), at two points: X1\=0X\_1=0X1​\=0 and X1\=1X\_1=1X1​\=1. This is much less work than what PPP must do.

Observe that at this point, VVV doesn’t know whether the polynomial being evaluated by PPP is actually the polynomial of interest, ggg. At this point, VVV only knows PPP has been consistent with the polynomial they have used. In fact, everything until Step 7 is only used to verify the consistency of PPP‘s answers. Step 7 resolves this uncertainty and is used to prove that PPP has been both consistent and correct. This observation is one of the “slippery” aspects in understanding the sum-check protocol.

##### 4\. VVV picks a random number r1r\_1r1​ for the free variable and sends it to PPP

Before beginning this step, it’s time to introduce a security aspect of the protocol. When the protocol starts, VVV tells PPP what finite field Fp\\mathbb{F\_p}Fp​ they will be working with. This means that all numbers will be restricted (by using the modulus operator) to be between 0 and ppp. For example, if PPP is told that p\=16p=16p\=16 then PPP will reduce everything by mod 16, e.g. 74 mod 16\=1074 \\text{ mod } 16 = 1074 mod 16\=10 and 31x+17 mod 16\=15x+131x + 17 \\text{ mod } 16 = 15x + 131x+17 mod 16\=15x+1.

Next, VVV picks a random number r1r\_1r1​ for PPP to assign to X1X\_1X1​. r1r\_1r1​ is selected from the finite field Fp\\mathbb{F}\_pFp​. For the running example in this article, we will pick p\=16p = 16p\=16. From here on, all example calculations will be performed mod 16.

For the remainder of the protocol, PPP will always replace X1X\_1X1​ with r1r\_1r1​. Note that something interesting is now happening. Up until this point, PPP only evaluated ggg with Boolean inputs, but now the protocol is leveraging the fact that because ggg is a polynomial, it can be evaluated using arbitrary integers instead of only 0s or 1s. This is where the security of the protocol is derived: if ppp is very large, it will be difficult for PPP to guess the random challenges before receiving them.

For our running example, we will assume that VVV selects r1\=2r\_1 = 2r1​\=2.

##### 5\. PPP replaces the free variable x1x\_1x1​ with the random number r1r\_1r1​ and computes a partial sum leaving next variable x2x\_2x2​ free

PPP calculates the sum again, except now they replace X1X\_1X1​ with the random number r1r\_1r1​:

g2(X2)\=∑x3∈{0,1}∑x4∈{0,1}⋯∑xv∈{0,1}g(r1,X2,x3,⋯ ,xv)\\begin{equation\*} g\_2(X\_2) =\\sum\_{x\_3 \\in \\{0,1\\}} \\sum\_{x\_4 \\in \\{0,1\\}} \\cdots \\sum\_{x\_v \\in \\{0,1\\}} g(r\_1, X\_2, x\_3, \\cdots, x\_v) \\end{equation\*}g2​(X2​)\=x3​∈{0,1}∑​x4​∈{0,1}∑​⋯xv​∈{0,1}∑​g(r1​,X2​,x3​,⋯,xv​)​

For our example, near the end of Step 3, PPP calculated g2(X1,X2)\=−3X1X2+3X2g\_2(X\_1, X\_2) = -3X\_1X\_2 + 3X\_2g2​(X1​,X2​)\=−3X1​X2​+3X2​. At the end of Step 4 above, we mention that VVV selected r1\=2r\_1 = 2r1​\=2. So PPP replaces X1X\_1X1​ with r1r\_1r1​, giving g2(X2)\=−3X2g\_2(X\_2) = -3X\_2g2​(X2​)\=−3X2​. PPP returns g2(X2)\=−3X2 mod 16\=13X2g\_2(X\_2) = -3X\_2 \\text{ mod } 16 = 13X\_2g2​(X2​)\=−3X2​ mod 16\=13X2​ to VVV.

##### 6\. PPP and VVV repeat steps similar to 3—5 for the rest of the variables: x2,⋯ ,xvx\_2, \\cdots, x\_vx2​,⋯,xv​

Steps 3 through 5 above constitute one “round” of the protocol. In total, if vvv is the number of variables in ggg, then vvv rounds in total are required to complete the protocol. Each round includes the same operations as given in Steps 3 through 5 above, with the exception that different variables are left free each round.

For example, at the end of Step 5, PPP returns g2(X2)g\_2(X\_2)g2​(X2​) to VVV. If we apply the pattern in Step 3, noting that g1(r1)\=−3X1+3 mod 16\=13X1+3g\_1(r\_1) = -3X\_1 + 3 \\text { mod } 16 = 13X\_1 + 3g1​(r1​)\=−3X1​+3 mod 16\=13X1​+3, we see that that VVV must now check that:

g1(r1) mod 16\=?(g2(0)+g2(1)) mod 16(13⋅2+3) mod 16\=?(13⋅0+13⋅1) mod 1613 mod 16\=✓13 mod 16\\begin{aligned}g\_1(r\_1) \\text{ mod } 16 &\\stackrel{?}{=} (g\_2(0) + g\_2(1)) \\text{ mod } 16 \\\\ (13 \\cdot 2 + 3) \\text{ mod } 16 &\\stackrel{?}{=} (13 \\cdot 0 + 13 \\cdot 1) \\text{ mod } 16 \\\\ 13 \\text{ mod } 16 &\\stackrel{\\checkmark}{=} 13 \\text{ mod } 16\\end{aligned}g1​(r1​) mod 16(13⋅2+3) mod 1613 mod 16​\=?(g2​(0)+g2​(1)) mod 16\=?(13⋅0+13⋅1) mod 16\=✓13 mod 16​

So round 2 passes.

In our example, ggg has four variables, so we will cycle through Steps 3—5 four times, then we move to Step 7.

##### 7\. VVV evaluates ggg at one input using access to an “oracle”, i.e., VVV must make a singled trusted execution of g(r1,r2,⋯ ,rv)g(r\_1, r\_2, \\cdots, r\_v)g(r1​,r2​,⋯,rv​)

After round vvv has successfully been checked, then gv(Xv)g\_v(X\_v)gv​(Xv​) has incorporated r1,r2,⋯ ,rv−1r\_1, r\_2, \\cdots, r\_{v-1}r1​,r2​,⋯,rv−1​. The last step is for VVV to select rvr\_vrv​ and use an _oracle_ to evaluate g(r1,r2,…,rv)g(r\_1, r\_2, \\dots, r\_v)g(r1​,r2​,…,rv​). The oracle is a trusted source which will be guaranteed to evaluate ggg correctly at a single input. Alternatively, if no oracle is available, then VVV must compute ggg once by itself. Without an oracle, VVV would be required to store and compute ggg — this could be costly.

After g(r1,r2,…,rv)g(r\_1, r\_2, \\dots, r\_v)g(r1​,r2​,…,rv​) is provided by the oracle, or computed by VVV itself, then VVV checks:

g(r1,r2,…,rv)\=?gv(rv)\\begin{equation\*} g(r\_1, r\_2, \\dots, r\_v) \\stackrel{?}{=} g\_v(r\_v) \\end{equation\*}g(r1​,r2​,…,rv​)\=?gv​(rv​)​

If this check passes, then PPP accepts VVV‘s claim that g0g\_0g0​ is equal to HHH, or, in our example, that g0g\_0g0​ is the #SAT of ggg.

### Conclusions and further reading

Python code implementing sum-check can be found in [this repo](https://github.com/0xsamgreen/sumcheck)
. We omitted security checks in this tutorial for brevity, but those checks are included in the linked repo. Also, this article did not explain why sum-check is secure, nor did it cover the practical costs of the algorithm. Briefly, the security of sum-check can be analyzed using the Schwartz–Zippel lemma — it is basically like random sampling used for quality control. Regarding the costs, to solve #SAT without the sum-check protocol, VVV would have had to evaluate ggg at 2v2^v2v inputs, but with sum-check, the cost for VVV drops to vvv steps of the protocol plus a single evaluation of ggg. If you want more depth, see Justin’s book, which is linked above, and these resources:

1.  Sum-check article by Justin Thaler: [The Unreasonable Power of the Sum-Check Protocol](https://zkproof.org/2020/03/16/sum-checkprotocol/)
    
2.  Sum-check notes by Edge & Node cryptographer, Gabriel Soule: [https://drive.google.com/file/d/1tU50f-IpwPdCEJkZcA7K0vCr7nwwzCLh/view](https://drive.google.com/file/d/1tU50f-IpwPdCEJkZcA7K0vCr7nwwzCLh/view)
    

If you have made it this far, you now hopefully have a feel for how _interactive_ arguments of knowledge work. This is the first time we have mentioned “interactive”. Note that PPP and VVV had to communicate in each round. This is not ideal in scenarios where computers can go offline, i.e., the real world. The Fiat–Shamir transform makes sum-check non-interactive. As you may know, SNARKs also eliminate interactions, as implied by the “N” in the name which stands for “Non-interactive”. SNARKs also use Fiat-Shamir. Perhaps we will make a part 2 of this article where we implement a non-interactive version of sum-check.

---

# Strategy

![Stylized Semiotic Logo](/_astro/illustration_one.Fso39xch_1a690M.svg)

![Semiotic Logo](/_astro/logo-black.C2Buo_kb_ZOsWrt.svg)

We fuse our artificial intelligence and cryptography expertise to engineer solutions that radically improve web3 accessibility and scalability.

![Stylized graphic of a bar chart with a tablet showing code, surrounded by floating geometric shapes and arrows on a purple background.](/_astro/illustration_two.DNCIvpsf_ZvIVGX.svg)

[The Graph](https://thegraph.com/)
 protocol makes it easy to organize and query on-chain data. For example, The Graph provides all the price data at [info.uniswap.org.](https://info.uniswap.org/)

We have applied our R&D background to bring three new services to The Graph: high-performance analytics queries, decentralized AI models, and zk-coprocessing.

![</>](/_astro/roadmap_illustration_one.Cvt2t-Us_1ztiDf.svg)

Bring SQL to The Graph, making it simple for analytics dapps to extract more profound insights.

![</>](/_astro/roadmap_illustration_two.DUDTx3PB_ZFWsCe.svg)

Bring zk-coprocessing to The Graph, paving the way for verifiable AI.

![](/_astro/separator.CSS5HWSB_Z14WX1.svg)

![</>](/_astro/roadmap_illustration_three.DH8QGzZp_15OUq4.svg)

We will bring verifiable AI to The Graph. Decentralized AI agent applications will use The Graph’s extensive cross-chain data for optimal on-chain actions.

![verifiable computing illustration](/_astro/verifiable_computing_illustration.xUiGQRBX_1A6yUO.svg)

##### Verifiable Computing

We build systems using cryptography and zero-knowledge proofs.

![swift data access illustration](/_astro/swift_data_access.C2WweULv_Z1V2iJ.svg)

##### Swift Data Access

We use our enterprise database expertise to provide you with the fastest time-to-data

![AI X Crypto illustration](/_astro/ai_x_crypto.BadRrIzI_Z2fqXCj.svg)

##### AI x Crypto

Semiotic pioneered the use of AI agents for automated decision-making in crypto.

![The Graph Logo](/_astro/the_graph.Df2pjMgi_Z7hdQr.svg)

##### The Graph Harmony

Tight integration with The Graph ecosystem

##### Scalar TAP

Proposed, designed, and shipped Scalar TAP, The Graph’s next-generation micropayment system to process and secure the funds for billions of transactions.

[Learn More](https://github.com/semiotic-ai/timeline-aggregation-protocol)

##### Verifiable Firehose & EIP-4444

Verifiable Firehose enables The Graph's Indexers to solve the Ethereum "state expiry"problem.

[Learn more](https://ethresear.ch/t/using-the-graph-to-preserve-historical-data-and-enable-eip-4444/)

##### AgentC

A natural language tool to improve access to blockchain data which also gave us the expertise and insight to advocate for and design the SQL data service.

[Apply for access](https://www.agentc.xyz/)

![separator](/_astro/separator.CSS5HWSB_Z14WX1.svg)

##### Grow The Graph’s market share & profit

We will design and launch The Graph's SQL data service & studio. This will enable developers to build many new analytics tools on The Graph.

##### Develop new AI and ZK services

Create MVPs for The Graph’s AI and zk-coprocessing data services. We will marry these two services to make The Graph the leader in verifiable AI.

##### Continue to support our existing products

We will continue to support previous tools we have delivered, including The Graph's micropayment solution (TAP) and automation tools that Indexers use to increase their profit.

##### Engage in The Graph’s protocol design

We will engage in protocol design discussions with other Core Developers.

![](/_astro/separator.CSS5HWSB_Z14WX1.svg)

##### We aim to

### increase queries

### add new data services

build more
==========

![Talented engineers drawing math formulas on a blackboard](/_astro/talented_engineers.Ch2wdf7X_1D8ziH.webp)

![Talented engineers drawing math formulas on a blackboard](/_astro/talented_engineers_desktop.C8m205pk_SSWLC.webp)

Join our mission of making on-chain data more accessible. This is an opportunity to get in early and help shape the future of web3.

---

# Semiotic Labs

Shaping  
the future

![](/_astro/hero_gradient.C3mDWdsE_7oIW7.svg) ![A hand with abstract digital elements and speech bubbles, representing digital interaction.](/_astro/hero.DfTS2oW9_Bwa3D.svg)

Of web3  
interactions ![Arrow pointing down, indicating scrollable content.](/_astro/scroll-arrow.HcrYPGyU_1MikDb.svg)

Our Mission
-----------

We fuse our artificial intelligence and cryptography expertise to engineer solutions that radically improve web3 accessibility and scalability.

![](/_astro/one.shoDdTRE_Z1GmC2X.svg) ![](/_astro/two.BGZ8nxHB_ZAk6KB.svg) ![](/_astro/three.CeqxFo0H_m6VRV.svg)

![Agentc what is the price of Eth?](/_astro/why-illustration.z4i7T0n6_4jdGh.svg)

Web3 must still be easier to use, and its capabilities must be expanded. **Seamless, secure interaction** is the key to catalyzing mass adoption.

Semiotic Labs is the R&D core developer team in The Graph protocol. We're not just developers but innovators with a proven track record. Having incubated Odos – a leading DeFi liquidity aggregator and trading platform – we aim to continue building beautiful, impactful products for crypto and web3.

[Join our team ![Arrow pointing to the right](/_astro/contact-arrow-white.DBPSmYeH_Z2hqMR6.svg)](https://wellfound.com/company/semiotic-labs-usa)
 

![](/_astro/join-bg.CPFxaYcn_2pHvhd.webp) ![Abstract elements reminiscent of cryptography, blockchain and statistics.](/_astro/join-illustration.BjgXEbLS_1s8S0B.svg)

What we  
built ![](/_astro/decoration_one.CmdUXf93_1bNObh.svg)
---------------------------------------------------------------

![A graph showing Semiotic labs in the middle, connected to the following products: Odos, Semiotic Indexer, Sql Studio](/_astro/what-we-built-illustration.qem5GfeD_1xFk30.svg)

![](/_astro/decoration_two.DGuF7Qq1_Z23w5mX.svg)

Through meticulous effort and unwavering commitment, we craft innovative products that are a testament to our vision.

[See our products ![Arrow pointing right](/_astro/contact-arrow-black.ClbVlqIa_1pCKeo.svg)](/products)

---

# Products

Products
========

*   **01**
    
    * * *
    
    ![a magnifying glass hovering over folders, signifying indexing data](/_astro/indexing.kJDPUGxL_Z28enjN.svg)
    
    * * *
    
    Indexers are node operators in The Graph Network that stake Graph Tokens (GRT) to provide indexing and query processing services. Indexers earn query fees and indexing rewards for their services. Semiotic Labs has proudly been running an indexing node since 2021
    
*   **02**
    
    * * *
    
    ![Icons of a book with an upward arrow and a diagonal line of geometric shapes on a dark background.](/_astro/autoagora.DL9PWo32_ZvGPbz.svg)
    
    * * *
    
    Indexers in The Graph dynamically set their query prices according to their operational costs, consumer demand, and supply competition. AutoAgora is a tool for Indexers to help them set these costs. AutoAgora is based on a branch of AI called Reinforcement Learning. Click here to learn more about it:
    
*   **03**
    
    * * *
    
    ![agentc logo](/_astro/allocation-optimizer.D_JgbPuE_2oj7Ms.svg)
    
    * * *
    
    The Allocation Optimizer is a tool designed to help Indexers in The Graph strategically allocate their resources. It uses convex optimization to address the challenge of selecting the most lucrative subgraphs for indexing, aiming to maximize indexing rewards. This approach significantly enhances the efficiency of resource allocation by indexers, contributing to better overall performance in the network.
    
*   **04**
    
    * * *
    
    ![Odos](/_astro/odos.BTK-MAs0_2dvdlx.webp)
    
    * * *
    
    Odos revolutionizes Decentralized Finance (DeFi) by solving the critical challenges of liquidity fragmentation. With our best-in-class smart order routing algorithm, we deliver optimal exchange rates and arbitrage in a single transaction. In its first two years, Odos has already processed $25+ billion in transaction volume for over 1.5 million users, and has officially spun out of Semiotic Labs to continue scaling and building a team hyper-focused on becoming the premiere smart order routing platform for digital assets.
    

![Talented engineers drawing math formulas on a blackboard](/_astro/talented_engineers.Ch2wdf7X_1D8ziH.webp)

![Talented engineers drawing math formulas on a blackboard](/_astro/talented_engineers_desktop.C8m205pk_SSWLC.webp)

Join our mission of making on-chain data more accessible. This is an opportunity to get in early and help shape the future of web3.

---

# Company history

Semiotic Labs is a Silicon Valley startup with a track record in productizing R&D. We build innovative products that provide a fantastic experience for web3. We focus on three areas:

![Cryptography, Ai & Optimization, Blockchain infrastructure](/_astro/heading-illustration.BXdcqyE__Z2bmmDT.svg) ![](/_astro/curved_bg_invert_mobile.5_gsLyYf_Z1eEJ9q.svg) ![](/_astro/curved_bg_invert_desktop.C9BrN7Aj_stYRo.svg)

what we build
-------------

![The Graph Logo](/_astro/the-graph.VpPRb25j_1jfoO.webp)

We are R&D core developers in [The Graph](https://thegraph.com/)
 protocol. As core developers, we build tools to improve the efficiency, security, and usability of The Graph’s query market.

![Odos Logo](/_astro/odos.DIu8FljY_24UQWi.webp)

Odos is a bleeding-edge smart order routing technology for digital assets of all kinds. Having processed over $25 billion in trading volume in its first two years and onboarding more than 1.5 million users, Odos delivers users best-in-class exchange rates and savings across 9+ EVM chains. Odos is now spun out from Semiotic to continue its meteoric growth. Swap on Odos today at [Odos.xyz!](https://odos.xyz/)

Our vision is that blockchain-based web3 will serve as the foundation for a new decentralized digital economy that is more open, efficient, and accessible.

*   ![bullet](/_astro/list-disc-purple.CVm-3hPi_2n63Tn.svg) Solving real problems
*   ![bullet](/_astro/list-disc-purple.CVm-3hPi_2n63Tn.svg) Innovation at the core
*   ![bullet](/_astro/list-disc-purple.CVm-3hPi_2n63Tn.svg) Urgency and focus
*   ![bullet](/_astro/list-disc-purple.CVm-3hPi_2n63Tn.svg) Integrity and respect

![](/_astro/history_ellipse.CfHOoFf6_Z1vH57u.webp) ![](/_astro/history_ellipse_mobile.yG1C-pO2_ZEVPOb.webp)

Semiotic Labs was founded in Los Altos, California, in February 2020. Before starting the company, Ahmet Ozcan (CEO) managed the Machine Intelligence department at IBM Research, Almaden; Alexis Asseman (Lead Developer) investigated AI hardware acceleration at IBM Research; and Sam Green (Head of Research) was finishing his Computer Science Ph.D. at the University of California, Santa Barbara.

From the beginning, Semiotic Labs has created products that combine AI and cryptography. Initially, we focused on Fully Homomorphic Encryption (FHE), a family of techniques that enables computing directly on encrypted data. We experienced initial success with an NSF SBIR Phase I to use our technology to accelerate FHE. After our first year, we pivoted into web3, where we found greater demand for our AI and cryptography skills within The Graph protocol and became long-term core developers.

![Talented engineers drawing math formulas on a blackboard](/_astro/talented_engineers.Ch2wdf7X_1D8ziH.webp)

![Talented engineers drawing math formulas on a blackboard](/_astro/talented_engineers_desktop.C8m205pk_SSWLC.webp)

Join our mission of making on-chain data more accessible. This is an opportunity to get in early and help shape the future of web3.

---

# Semiotic Labs | Articles

![Gradient background](/_astro/curved_bg_desktop.ChjV8nJN_1c0N0M.svg) ![Gradient background](/_astro/curved_bg_mobile.DP8czMRd_1b6C73.svg)

*   [*   cryptography\
    \
    ##### Homomorphic Signatures for Payment Channels\
    \
    This post discusses design and optimization of verifiable micropayments over a state channel, a feature of Semiotic Labs’ \[GraphTally\](https://github.com/semiotic-ai/timeline-aggregation-protocol) library for The Graph. The Graph is a decentralized data services protocol, allowing customers to index and access all blockchain data.\_\
    \
    * * *](/articles/h2s2-payment-channels/)
    
*   [*   ai\
    *   cryptography\
    *   defi\
    \
    ##### 2024 Crypto Meta-Analysis\
    \
    A meta-summary of various 2024 crypto market outlooks.\
    \
    * * *](/articles/2024-crypto-market-outlooks/)
    
*   [*   cryptography\
    \
    ##### PSI with FHE\
    \
    Semiotic Labs explores the integration of Private Set Intersection (PSI) in blockchain to tackle Maximal Extractable Value (MEV) issues. Utilizing fully homomorphic encryption, the protocol aims for trustless operations and enhanced performance in tasks like access list comparison and private auctions.\
    \
    * * *](/articles/psi/)
    
*   [*   ai\
    \
    ##### Indexer Allocation Optimization: Part II\
    \
    This article was co-authored with Howard Heaton, from Edge & Node, and with Hope Yen, from GraphOps.\
    \
    * * *](/articles/indexer-allocation-optimization-part-ii/)
    
*   [*   ai\
    *   defi\
    \
    ##### An Overview of Automatic Market Maker Mechanisms\
    \
    This article explores the principles and mechanisms behind the many popular Automatic Market Maker designs currently used in production. While the mathematical details of these designs are fascinating in their own right, this article seeks to instead focus on graphical representations and high-level concepts, allowing for a more approachable and exhaustive exploration of the space.\_ \_Watch our related Devcon 2022 talk here: \[https://www.youtube.com/watch?v=KxuyHfmLHP0\](https://www.youtube.com/watch?v=KxuyHfmLHP0).\
    \
    * * *](/articles/automatic-market-makers-amm/)
    
*   [*   ai\
    *   blockchain infrastructure\
    \
    ##### Automated Query Pricing in The Graph\
    \
    Indexers in The Graph have control over the pricing of the GraphQL queries they serve based on their shape. For this task, The Graph created a domain-specific language called Agora​ that maps query shapes to prices in GRT. However, manually populating and updating Agora models for each subgraph is a tedious task, and as a consequence, most indexers default to a static, flat pricing model.\_ \_To help indexers with pricing in the relative resource cost of serving different query shapes, as well as following the query market price, we are developing AutoAgora, an automation tool that automatically creates and updates Agora models.\_ \_See our related Devcon 2022 talk here:\_ \_\[https://www.youtube.com/watch?v=LRl9uFfmjEs\](https://www.youtube.com/watch?v=LRl9uFfmjEs)\_.\
    \
    * * *](/articles/automated-query-pricing-on-the-graph/)
    
*   [*   ai\
    \
    ##### Indexer Allocation Optimization: Part I\
    \
    Indexers within The Graph Protocol are rewarded via an indexing reward. How can indexers optimise their allocations so as to maximise the reward they receive? In this blog post we formalise the problem in terms of a reward function and use convex optimization to find a solution.\
    \
    * * *](/articles/indexer-allocation-optimisation/)
    
*   [*   cryptography\
    \
    ##### Introduction to the Sum-Check Protocol\
    \
    It is expensive to run transactions on the Ethereum EVM. Verifiable computing (VC) lets us outsource computing away from the EVM. Today, a popular and exciting form of VC algorithm is the SNARK. There are various families of SNARKs that use the sum-check protocol, which is a simple algorithm to introduce VC. This is a tutorial on the sum-check protocol. This post is focused on how the sum-check protocol is implemented - it does not go into theory. You can skip straight to the finished code \[here\](https://github.com/0xsamgreen/sumcheck).\_Thank you to Gokay Saldamli, Gabriel Soule, and Tomasz Kornuta for providing valuable feedback on this article.\_\
    \
    * * *](/articles/sumcheck-tutorial/)

---

# Semiotic Labs | Videos

![Gradient background](/_astro/curved_bg_desktop.ChjV8nJN_1c0N0M.svg) ![Gradient background](/_astro/curved_bg_mobile.DP8czMRd_1b6C73.svg)

*   [*   blockchain infrastructure\
    *   ai\
    \
    #### Bringing high-performance SQL queries to The Graph\
    \
    At Datapalooza 2023, Sam presented Semiotic's latest research & practical applications of AI & data analytics in web3 and shared details about bringing high-performance SQL queries to The Graph!](https://www.youtube.com/watch?v=eTKzHkv5E40)
    
*   [*   blockchain infrastructure\
    *   cryptography\
    \
    #### The Graph and Preserving Historical Data After EIP-4444 by Seve Sisneros\
    \
    Severiano Sisneros presents "The Graph and Preserving Historical Data After EIP-4444" at Datapalooza 2023. This talk, held during Devconnect in Istanbul, Turkey, explores the vital role of The Graph in the context of Ethereum's EIP-4444. Gain insights into the challenges and strategies for preserving historical data in the web3 space.](https://www.youtube.com/watch?v=AwU6gkXBxKM)
    
*   [*   cryptography\
    \
    #### Devcon 2022: A SNARKs Tale: A Story of Building SNARK Solutions on Mainnet\
    \
    Severiano Sisneros, Semiotic's Senior Cryptographer, tells the story of deploying a real-world solution using SNARKs as a core primitive; highlighting many cutting-edge SNARKs and their limitations in the hope to identify opportunities for the community to make Ethereum more SNARK friendly, creating a diverse ecosystem where SNARKs built upon a variety of unique primitives can thrive.](https://www.youtube.com/watch?v=GnYM9yxIRSs)
    
*   [*   defi\
    \
    #### Devcon 2022: An Overview of AMM Mechanisms\
    \
    Matt Deible gives an exhaustive overview of the different AMM algorithms currently deployed on major distributed ledgers, as well as the underlying intuition behind their design. Building up from the basic principles of AMM design, the talk then covers the algorithmic mechanisms used in the various different algorithms including Constant Sum, Constant Product (Uniswap V2), Uniswap V3, KyberSwap, StableSwap (Curve), CryptoSwap (Curve V2), Solidly Stable pairs, Clipper, Dodo, and RFQ systems.](https://www.youtube.com/watch?v=KxuyHfmLHP0)
    
*   [*   ai\
    \
    #### Devcon 2022: Reinforcement Learning for Query Pricing in The Graph\
    \
    Indexers​​ in The Graph protocol use a DSL called Agora​ to map query shapes to prices. However, manually populating and updating Agora models for each query is a tedious task, and, as a consequence, most indexers default to a flat pricing model. We have created and deployed reinforcement learning agents for Indexers to automatically compete on pricing. This talk, given by Dr. Tomasz Kornuta, is focused on our development process and our study of the market effects of multiple competing pricing agents.](https://www.youtube.com/watch?v=LRl9uFfmjEs)
    
*   [#### Stanford Seminar: Evolution of a Web3 Company\
    \
    Co-founder and Head of Research, Dr. Sam Green, presents to Stanford's CEE 246A class on the evolution of Semiotic Labs: how we started with Fully-Homomorphic Encryption, early support from NSF SBIR and Activate.org, our pivot to web3 with The Graph protocol, and the launch of our DEX aggregator: odos.xyz.](https://www.youtube.com/watch?v=KvhgJEBskgg)
    
*   [*   ai\
    \
    #### Special Graph Hack Episode: Automated Allocation Optimizer\
    \
    Indexers within The Graph Protocol are rewarded via an indexing reward. How can indexers optimize their allocations so as to maximize the reward they receive? In this video, Anirudh Patel and Hope Yen formalize that question in terms of a reward function and explain how convex optimization can be used to find a solution. They then demonstrate a new tool that automates the solution to the allocation optimization problem.](https://www.youtube.com/watch?v=J6a4RarYYFI)
    
*   [*   defi\
    \
    #### ODOS: The Optimal DEX Aggregator for Token Swaps\
    \
    Odos is a Smart Order Routing (SOR) solution that leverages a patented Automated Market Maker (AMM) path finding algorithm to aggregate decentralized exchanges (DEX) and find optimal routes for token swaps. By traversing a large universe of connecting token pairs and performing complex order splits, Odos is the ultimate choice for institutional and retail traders. Odos is the first aggregator to introduce a multi-asset input feature that enables users to swap from several tokens into one asset in a single atomic transaction.](https://www.youtube.com/watch?v=qC8Ry-YOZZs)
    
*   [*   blockchain infrastructure\
    *   ai\
    \
    #### Automated Cost Modeling in The Graph\
    \
    Alexis Asseman presents on our reinforcement learning agent used to automatically set prices for Indexers in The Graph protocol.](https://www.youtube.com/watch?v=DrdkX8FibHI)
    
*   [*   defi\
    \
    #### ODOS Intro: The Optimal Order Routing Solution for Cryptocurrency Swaps\
    \
    Odos is a Smart Order Routing (SOR) solution that leverages a patented Automated Market Maker (AMM) path finding algorithm to aggregate decentralized exchanges (DEX) and find optimal routes for token swaps. By traversing a large universe of connecting token pairs and performing complex order splits, Odos is the ultimate choice for institutional and retail traders. Odos is the first aggregator to introduce a multi-asset input feature that enables users to swap from several tokens into one asset in a single atomic transaction.](https://www.youtube.com/watch?v=XMqzRN1qQ-I)
    
*   [*   ai\
    \
    #### Introduction to the Workshop on Incentive Mechanism Validation: Sam Green, Semiotic Labs\
    \
    Presented at the WIMV 2022 Devconnect workshop. For future workshops, check out https://wimv.dev or follow https://twitter.com/wimvdev.](https://www.youtube.com/watch?v=RQULcaKWCD8)
    
*   [*   ai\
    \
    #### Simplifying Mechanism Analysis With Software Design: Anirudh Patel, Semiotic Labs\
    \
    Presented at the WIMV 2022 Devconnect workshop. For future workshops, check out https://wimv.dev or follow https://twitter.com/wimvdev.](https://www.youtube.com/watch?v=rF4Gr0CS5Gc)

---

# Semiotic Labs | Podcasts

![Gradient background](/_astro/curved_bg_desktop.ChjV8nJN_1c0N0M.svg) ![Gradient background](/_astro/curved_bg_mobile.DP8czMRd_1b6C73.svg)

*   [![GRTiQ podcast special AI Services Whitepaper Anirudh Patel](/_astro/grtiq_ai_whitepaper_special_anirudh.D0JuTZvu_Z1zGT2m.webp)\
    \
    #### GRTiQ Podcast Special Release: Anirudh Patel on The Graph as AI Infrastructure\
    \
    ##### GRTiQ\
    \
    On May 28, 2024, Semiotic Labs, a core developer team working on The Graph, released an exciting new white paper titled “The Graph as AI Infrastructure.” This paper details the upcoming launch of two new AI services to be built on The Graph: the Inference Service and the Agent Service. The paper also explores what makes The Graph the best place to service the convergence of web3, blockchain data, and AI. In addition to all this, Semiotic Labs also announced a two-week public demo of their ChatGPT-like AI product called Agentc, which was built using The Graph. To help us better understand the details of the white paper, understand these new AI services, and comprehend the implications for the future of The Graph, I’ve invited Anirudh Patel, or Ani, back onto the podcast. Ani’s been on the podcast several times now, but he’s back for this special release and will answer these questions and more!](https://www.grtiq.com/grtiq-podcast-special-release-anirudh-patel-on-the-graph-as-ai-infrastructure/)
    
*   [![GRTiQ podcast episode 144 Anirudh Patel](/_astro/grtiq144.CP9oaoVT_1YWa9U.webp)\
    \
    #### GRTiQ Podcast: 144 Anirudh Patel\
    \
    ##### GRTiQ\
    \
    In a captivating edition of the GRTiQ podcast, Nick Hansen welcomes back Aniurdh “Ani” Patel, a Senior Research Scientist at Semiotic Labs, for an insightful and comprehensive interview. Following an impactful discussion in a previous episode, Ani shares his extensive journey through AI, web3, and The Graph, infused with his passion for travel and innovation. The discussion delves deep into the intricacies of web3, crypto, and Ani's introduction to Semiotic Labs via Sandia Labs. The conversation has insights into the world of artificial intelligence and machine learning. As a highlight, Ani unveils the exciting launch of our LLM-based query testbed, AgentC.](https://www.grtiq.com/grtiq-podcast-144-anirudh-patel/)
    
*   [![GRTiQ podcast episode 144 Anirudh Patel](/_astro/grtiq106.7lm8z8JN_1L3X2d.webp)\
    \
    #### GRTiQ Podcast: 106 AI and Crypto\
    \
    ##### GRTiQ\
    \
    In this special edition of the GRTiQ Podcast, host Nick Hansen is joined by three members of Semiotic Labs, a Core Dev team on The Graph with extensive expertise in artificial intelligence. Anirudh Patel, Sam Green, and Tomasz Kornuta discuss the fundamentals of AI, the emergence of ChatGPT, and its functionality. The conversation then turns to AI's role in the crypto sphere, examining both recent hype and genuine applications. Finally, they delve into Semiotic Labs' implementation of AI within The Graph and explore the potential impact of combining tools like ChatGPT, Geo, and The Graph on the industry.](https://www.grtiq.com/grtiq-podcast-106-ai-and-crypto/)
    
*   [![GRTiQ podcast episode 43 Sam Green](/_astro/grtiq43.B_Iriyvr_1mtEQV.webp)\
    \
    #### GRTiQ Podcast 43: Sam Green\
    \
    ##### GRTiQ\
    \
    In Episode 43 of the GRTiQ Podcast, host Nick Hansen interviews Sam Green, Co-Founder and CTO of Semiotic, the fourth core dev team to join The Graph protocol. Sam discusses Semiotic's exceptional expertise in artificial intelligence and cryptography, and provides clear explanations of key concepts like reinforcement learning and zk roll-ups. The insights shared by Sam shed light on not only the future of The Graph and web3 but also the cutting-edge technological innovations being pursued within The Graph protocol.](https://www.grtiq.com/grtiq-podcast-43-sam-green/)
    
*   [![GRTiQ podcast episode 144 Anirudh Patel](/_astro/grtiq74.DQUhTtG2_Z1NPkwF.webp)\
    \
    #### GRTiQ Podcast: 74 Ahmet Ozcan\
    \
    ##### GRTiQ\
    \
    In this episode of the GRTiQ Podcast, host Nick Hansen interviews Ahmet Ozcan, Co-founder and CEO of Semiotic Labs, a Core Dev team contributing to The Graph. The conversation covers Semiotic Labs' recent launch of Odos, an innovative path-finding algorithm designed to optimize order routing for crypto traders. Ahmet also discusses the team's work on The Graph, his transition from a successful career at IBM to entrepreneurship in Web3, and shares his long-term vision for The Graph.](https://www.grtiq.com/grtiq-podcast-74-ahmet-ozcan/)
    
*   [![The main chain. Ahmet Ozcan CEO & co-founder of Semiotic Ai.](/_astro/main_chain_ahmet_fundamental_inovations.CJw444mA_Z2gy0ly.webp)\
    \
    #### The Main Chain: Ahmet Ozcan – Fundamental Innovations\
    \
    ##### Infinity Ventures Crypto (IVC)\
    \
    It's an exciting time right now in crypto - with decentralized exchanges (DEX) still in their infancy, crypto users are evaluating their potential. As DEXs transactions are completely automated, they can be interacted with in a 100% programmatic fashion. which means they are ripe for applications to be built on top of them. That’s where Semiotic AI comes in. Dr. Ahmet Ozcan is the CEO of Semiotic AI, which aims to build automated decision-making tools for decentralized markets on the blockchain. The Semiotic team are the brains behind odos.xyz, a web3 application that allows traders to make large token swaps and trades across disparate DEXs, quickly and efficiently. If one DEX doesn’t have the liquidity needed to complete the transaction, Odos will map trades across DEXs, making large trades effortless on the part of an investor. Semiotic has also been in the news lately for their contributions to The Graph, the indexing and query layer of Web3, for which they received a grant of USD $60 million from the Graph Foundation. Dr. Ozcan chats with host Michael Waitze about how odos works and details its future roadmap. He also discusses why The Graph is important, what it does, and how Semiotic is contributing to The Graph, and to the future of web3, including how their AI solutions simulate how different incentives motivate actors to work either for the good or the detriment of blockchain systems.](https://anchor.fm/themainchain/episodes/Ahmet-Ozcan---Fundamental-Innovations-e1ivpf8)

---

# Semiotic Labs | Articles

![Gradient background](/_astro/curved_bg_desktop.ChjV8nJN_1c0N0M.svg) ![Gradient background](/_astro/curved_bg_mobile.DP8czMRd_1b6C73.svg)

*   [*   ai\
    *   cryptography\
    *   defi\
    \
    ##### 2024 Crypto Meta-Analysis\
    \
    A meta-summary of various 2024 crypto market outlooks.\
    \
    * * *](/articles/2024-crypto-market-outlooks/)
    
*   [*   ai\
    \
    ##### Indexer Allocation Optimization: Part II\
    \
    This article was co-authored with Howard Heaton, from Edge & Node, and with Hope Yen, from GraphOps.\
    \
    * * *](/articles/indexer-allocation-optimization-part-ii/)
    
*   [*   ai\
    *   defi\
    \
    ##### An Overview of Automatic Market Maker Mechanisms\
    \
    This article explores the principles and mechanisms behind the many popular Automatic Market Maker designs currently used in production. While the mathematical details of these designs are fascinating in their own right, this article seeks to instead focus on graphical representations and high-level concepts, allowing for a more approachable and exhaustive exploration of the space.\_ \_Watch our related Devcon 2022 talk here: \[https://www.youtube.com/watch?v=KxuyHfmLHP0\](https://www.youtube.com/watch?v=KxuyHfmLHP0).\
    \
    * * *](/articles/automatic-market-makers-amm/)
    
*   [*   ai\
    *   blockchain infrastructure\
    \
    ##### Automated Query Pricing in The Graph\
    \
    Indexers in The Graph have control over the pricing of the GraphQL queries they serve based on their shape. For this task, The Graph created a domain-specific language called Agora​ that maps query shapes to prices in GRT. However, manually populating and updating Agora models for each subgraph is a tedious task, and as a consequence, most indexers default to a static, flat pricing model.\_ \_To help indexers with pricing in the relative resource cost of serving different query shapes, as well as following the query market price, we are developing AutoAgora, an automation tool that automatically creates and updates Agora models.\_ \_See our related Devcon 2022 talk here:\_ \_\[https://www.youtube.com/watch?v=LRl9uFfmjEs\](https://www.youtube.com/watch?v=LRl9uFfmjEs)\_.\
    \
    * * *](/articles/automated-query-pricing-on-the-graph/)
    
*   [*   ai\
    \
    ##### Indexer Allocation Optimization: Part I\
    \
    Indexers within The Graph Protocol are rewarded via an indexing reward. How can indexers optimise their allocations so as to maximise the reward they receive? In this blog post we formalise the problem in terms of a reward function and use convex optimization to find a solution.\
    \
    * * *](/articles/indexer-allocation-optimisation/)

---

# Branding Assets

Find and download official Semiotic Labs branding assets and brand guidelines

![Semiotic Labs Logo](/_astro/logo-black.C2Buo_kb_ZOsWrt.svg) ![Semiotic Labs Logo](/_astro/logo-black-small.Bg5NXjIP_1RJAT9.webp)

![Semiotic Labs Logo](/_astro/logo-white.6XiSNCyH_1kc5ju.svg) ![Semiotic Labs Logo](/_astro/logo-white-small.CpVrULGE_jO7Sx.webp)

[Download Logo Pack](/logo-pack.zip)

brand colors
============

*   #221F1F
*   #575EEE
*   #F6BCFF

![Talented engineers drawing math formulas on a blackboard](/_astro/talented_engineers.Ch2wdf7X_1D8ziH.webp)

![Talented engineers drawing math formulas on a blackboard](/_astro/talented_engineers_desktop.C8m205pk_SSWLC.webp)

Join our mission of making on-chain data more accessible. This is an opportunity to get in early and help shape the future of web3.

---

# Semiotic Labs | Articles

![Gradient background](/_astro/curved_bg_desktop.ChjV8nJN_1c0N0M.svg) ![Gradient background](/_astro/curved_bg_mobile.DP8czMRd_1b6C73.svg)

*   [*   ai\
    *   cryptography\
    *   defi\
    \
    ##### 2024 Crypto Meta-Analysis\
    \
    A meta-summary of various 2024 crypto market outlooks.\
    \
    * * *](/articles/2024-crypto-market-outlooks/)
    
*   [*   ai\
    *   defi\
    \
    ##### An Overview of Automatic Market Maker Mechanisms\
    \
    This article explores the principles and mechanisms behind the many popular Automatic Market Maker designs currently used in production. While the mathematical details of these designs are fascinating in their own right, this article seeks to instead focus on graphical representations and high-level concepts, allowing for a more approachable and exhaustive exploration of the space.\_ \_Watch our related Devcon 2022 talk here: \[https://www.youtube.com/watch?v=KxuyHfmLHP0\](https://www.youtube.com/watch?v=KxuyHfmLHP0).\
    \
    * * *](/articles/automatic-market-makers-amm/)

---

# Semiotic Labs | Articles

![Gradient background](/_astro/curved_bg_desktop.ChjV8nJN_1c0N0M.svg) ![Gradient background](/_astro/curved_bg_mobile.DP8czMRd_1b6C73.svg)

*   [*   ai\
    *   blockchain infrastructure\
    \
    ##### Automated Query Pricing in The Graph\
    \
    Indexers in The Graph have control over the pricing of the GraphQL queries they serve based on their shape. For this task, The Graph created a domain-specific language called Agora​ that maps query shapes to prices in GRT. However, manually populating and updating Agora models for each subgraph is a tedious task, and as a consequence, most indexers default to a static, flat pricing model.\_ \_To help indexers with pricing in the relative resource cost of serving different query shapes, as well as following the query market price, we are developing AutoAgora, an automation tool that automatically creates and updates Agora models.\_ \_See our related Devcon 2022 talk here:\_ \_\[https://www.youtube.com/watch?v=LRl9uFfmjEs\](https://www.youtube.com/watch?v=LRl9uFfmjEs)\_.\
    \
    * * *](/articles/automated-query-pricing-on-the-graph/)

---

# Semiotic Labs | Articles

![Gradient background](/_astro/curved_bg_desktop.ChjV8nJN_1c0N0M.svg) ![Gradient background](/_astro/curved_bg_mobile.DP8czMRd_1b6C73.svg)

*   [*   cryptography\
    \
    ##### Homomorphic Signatures for Payment Channels\
    \
    This post discusses design and optimization of verifiable micropayments over a state channel, a feature of Semiotic Labs’ \[GraphTally\](https://github.com/semiotic-ai/timeline-aggregation-protocol) library for The Graph. The Graph is a decentralized data services protocol, allowing customers to index and access all blockchain data.\_\
    \
    * * *](/articles/h2s2-payment-channels/)
    
*   [*   ai\
    *   cryptography\
    *   defi\
    \
    ##### 2024 Crypto Meta-Analysis\
    \
    A meta-summary of various 2024 crypto market outlooks.\
    \
    * * *](/articles/2024-crypto-market-outlooks/)
    
*   [*   cryptography\
    \
    ##### PSI with FHE\
    \
    Semiotic Labs explores the integration of Private Set Intersection (PSI) in blockchain to tackle Maximal Extractable Value (MEV) issues. Utilizing fully homomorphic encryption, the protocol aims for trustless operations and enhanced performance in tasks like access list comparison and private auctions.\
    \
    * * *](/articles/psi/)
    
*   [*   cryptography\
    \
    ##### Introduction to the Sum-Check Protocol\
    \
    It is expensive to run transactions on the Ethereum EVM. Verifiable computing (VC) lets us outsource computing away from the EVM. Today, a popular and exciting form of VC algorithm is the SNARK. There are various families of SNARKs that use the sum-check protocol, which is a simple algorithm to introduce VC. This is a tutorial on the sum-check protocol. This post is focused on how the sum-check protocol is implemented - it does not go into theory. You can skip straight to the finished code \[here\](https://github.com/0xsamgreen/sumcheck).\_Thank you to Gokay Saldamli, Gabriel Soule, and Tomasz Kornuta for providing valuable feedback on this article.\_\
    \
    * * *](/articles/sumcheck-tutorial/)

---

# Semiotic Labs | Articles by Sam Green

![Gradient background](/_astro/curved_bg_desktop.ChjV8nJN_1c0N0M.svg) ![Gradient background](/_astro/curved_bg_mobile.DP8czMRd_1b6C73.svg)

*   [*   ai\
    *   cryptography\
    *   defi\
    \
    ##### 2024 Crypto Meta-Analysis\
    \
    A meta-summary of various 2024 crypto market outlooks.\
    \
    * * *](/articles/2024-crypto-market-outlooks/)
    
*   [*   cryptography\
    \
    ##### Introduction to the Sum-Check Protocol\
    \
    It is expensive to run transactions on the Ethereum EVM. Verifiable computing (VC) lets us outsource computing away from the EVM. Today, a popular and exciting form of VC algorithm is the SNARK. There are various families of SNARKs that use the sum-check protocol, which is a simple algorithm to introduce VC. This is a tutorial on the sum-check protocol. This post is focused on how the sum-check protocol is implemented - it does not go into theory. You can skip straight to the finished code \[here\](https://github.com/0xsamgreen/sumcheck).\_Thank you to Gokay Saldamli, Gabriel Soule, and Tomasz Kornuta for providing valuable feedback on this article.\_\
    \
    * * *](/articles/sumcheck-tutorial/)

---

# Semiotic Labs | Podcasts

![Gradient background](/_astro/curved_bg_desktop.ChjV8nJN_1c0N0M.svg) ![Gradient background](/_astro/curved_bg_mobile.DP8czMRd_1b6C73.svg)

*   [![GRTiQ podcast special AI Services Whitepaper Anirudh Patel](/_astro/grtiq_ai_whitepaper_special_anirudh.D0JuTZvu_Z1zGT2m.webp)\
    \
    #### GRTiQ Podcast Special Release: Anirudh Patel on The Graph as AI Infrastructure\
    \
    ##### GRTiQ\
    \
    On May 28, 2024, Semiotic Labs, a core developer team working on The Graph, released an exciting new white paper titled “The Graph as AI Infrastructure.” This paper details the upcoming launch of two new AI services to be built on The Graph: the Inference Service and the Agent Service. The paper also explores what makes The Graph the best place to service the convergence of web3, blockchain data, and AI. In addition to all this, Semiotic Labs also announced a two-week public demo of their ChatGPT-like AI product called Agentc, which was built using The Graph. To help us better understand the details of the white paper, understand these new AI services, and comprehend the implications for the future of The Graph, I’ve invited Anirudh Patel, or Ani, back onto the podcast. Ani’s been on the podcast several times now, but he’s back for this special release and will answer these questions and more!](https://www.grtiq.com/grtiq-podcast-special-release-anirudh-patel-on-the-graph-as-ai-infrastructure/)
    
*   [![GRTiQ podcast episode 144 Anirudh Patel](/_astro/grtiq144.CP9oaoVT_1YWa9U.webp)\
    \
    #### GRTiQ Podcast: 144 Anirudh Patel\
    \
    ##### GRTiQ\
    \
    In a captivating edition of the GRTiQ podcast, Nick Hansen welcomes back Aniurdh “Ani” Patel, a Senior Research Scientist at Semiotic Labs, for an insightful and comprehensive interview. Following an impactful discussion in a previous episode, Ani shares his extensive journey through AI, web3, and The Graph, infused with his passion for travel and innovation. The discussion delves deep into the intricacies of web3, crypto, and Ani's introduction to Semiotic Labs via Sandia Labs. The conversation has insights into the world of artificial intelligence and machine learning. As a highlight, Ani unveils the exciting launch of our LLM-based query testbed, AgentC.](https://www.grtiq.com/grtiq-podcast-144-anirudh-patel/)
    
*   [![GRTiQ podcast episode 144 Anirudh Patel](/_astro/grtiq106.7lm8z8JN_1L3X2d.webp)\
    \
    #### GRTiQ Podcast: 106 AI and Crypto\
    \
    ##### GRTiQ\
    \
    In this special edition of the GRTiQ Podcast, host Nick Hansen is joined by three members of Semiotic Labs, a Core Dev team on The Graph with extensive expertise in artificial intelligence. Anirudh Patel, Sam Green, and Tomasz Kornuta discuss the fundamentals of AI, the emergence of ChatGPT, and its functionality. The conversation then turns to AI's role in the crypto sphere, examining both recent hype and genuine applications. Finally, they delve into Semiotic Labs' implementation of AI within The Graph and explore the potential impact of combining tools like ChatGPT, Geo, and The Graph on the industry.](https://www.grtiq.com/grtiq-podcast-106-ai-and-crypto/)
    
*   [![GRTiQ podcast episode 43 Sam Green](/_astro/grtiq43.B_Iriyvr_1mtEQV.webp)\
    \
    #### GRTiQ Podcast 43: Sam Green\
    \
    ##### GRTiQ\
    \
    In Episode 43 of the GRTiQ Podcast, host Nick Hansen interviews Sam Green, Co-Founder and CTO of Semiotic, the fourth core dev team to join The Graph protocol. Sam discusses Semiotic's exceptional expertise in artificial intelligence and cryptography, and provides clear explanations of key concepts like reinforcement learning and zk roll-ups. The insights shared by Sam shed light on not only the future of The Graph and web3 but also the cutting-edge technological innovations being pursued within The Graph protocol.](https://www.grtiq.com/grtiq-podcast-43-sam-green/)
    
*   [![GRTiQ podcast episode 144 Anirudh Patel](/_astro/grtiq74.DQUhTtG2_Z1NPkwF.webp)\
    \
    #### GRTiQ Podcast: 74 Ahmet Ozcan\
    \
    ##### GRTiQ\
    \
    In this episode of the GRTiQ Podcast, host Nick Hansen interviews Ahmet Ozcan, Co-founder and CEO of Semiotic Labs, a Core Dev team contributing to The Graph. The conversation covers Semiotic Labs' recent launch of Odos, an innovative path-finding algorithm designed to optimize order routing for crypto traders. Ahmet also discusses the team's work on The Graph, his transition from a successful career at IBM to entrepreneurship in Web3, and shares his long-term vision for The Graph.](https://www.grtiq.com/grtiq-podcast-74-ahmet-ozcan/)
    
*   [![The main chain. Ahmet Ozcan CEO & co-founder of Semiotic Ai.](/_astro/main_chain_ahmet_fundamental_inovations.CJw444mA_Z2gy0ly.webp)\
    \
    #### The Main Chain: Ahmet Ozcan – Fundamental Innovations\
    \
    ##### Infinity Ventures Crypto (IVC)\
    \
    It's an exciting time right now in crypto - with decentralized exchanges (DEX) still in their infancy, crypto users are evaluating their potential. As DEXs transactions are completely automated, they can be interacted with in a 100% programmatic fashion. which means they are ripe for applications to be built on top of them. That’s where Semiotic AI comes in. Dr. Ahmet Ozcan is the CEO of Semiotic AI, which aims to build automated decision-making tools for decentralized markets on the blockchain. The Semiotic team are the brains behind odos.xyz, a web3 application that allows traders to make large token swaps and trades across disparate DEXs, quickly and efficiently. If one DEX doesn’t have the liquidity needed to complete the transaction, Odos will map trades across DEXs, making large trades effortless on the part of an investor. Semiotic has also been in the news lately for their contributions to The Graph, the indexing and query layer of Web3, for which they received a grant of USD $60 million from the Graph Foundation. Dr. Ozcan chats with host Michael Waitze about how odos works and details its future roadmap. He also discusses why The Graph is important, what it does, and how Semiotic is contributing to The Graph, and to the future of web3, including how their AI solutions simulate how different incentives motivate actors to work either for the good or the detriment of blockchain systems.](https://anchor.fm/themainchain/episodes/Ahmet-Ozcan---Fundamental-Innovations-e1ivpf8)

---

# 2024 Crypto Meta-Analysis

![Gradient background](/_astro/curved_bg_desktop.ChjV8nJN_1c0N0M.svg) ![Gradient background](/_astro/curved_bg_mobile.DP8czMRd_1b6C73.svg)

_This article was co-authored with [Alexander Garn](https://www.linkedin.com/in/alexjongarn/)
. Thanks to [Yuri Papadin](https://www.linkedin.com/in/papadiny/)
 and [Ahmet Ozcan](https://www.linkedin.com/in/asozcan/)
 for reviewing this article_.

Introduction
------------

Many groups have released 2024 crypto market outlooks this year. We used ChatGPT to summarize those from Messari, VanEck, Pantera Capital, Coinbase Institutional, and a16z crypto. We also include outlooks from Odos, a leading DeFi aggregator, and intent solver spun out of Semiotic. These organizations represent diverse views, including research and analytics, global investment, blockchain asset management, institutional crypto trading, venture capital investment, and DeFi aggregation. The first half of this article provides perspectives on common themes, including Bitcoin, DeFi, web3 gaming, AI, DePIN, UX, NFTs, and DeSoc. The second half of this article links to the original reports and highlights selected perspectives that don’t fall under the common themes.

![Crypto market summary](/_astro/image-3-1024x794.DE87agWx_AEYBx.webp)

If you would like an even more summarized view, then [@Anton\_Golub](https://twitter.com/Anton_Golub/status/1741093089463202207)
 has you covered

Table of Contents
-----------------

1.  [Introduction](https://astro-website-81r.pages.dev/articles/2024-crypto-market-outlooks/#introduction)
    
2.  [Table of Contents](https://astro-website-81r.pages.dev/articles/2024-crypto-market-outlooks/#table-of-contents)
    
3.  [Common Themes](https://astro-website-81r.pages.dev/articles/2024-crypto-market-outlooks/#common-themes)
    1.  [Bitcoin](https://astro-website-81r.pages.dev/articles/2024-crypto-market-outlooks/#bitcoin)
        
    2.  [DeFi](https://astro-website-81r.pages.dev/articles/2024-crypto-market-outlooks/#defi)
        
    3.  [Gaming](https://astro-website-81r.pages.dev/articles/2024-crypto-market-outlooks/#gaming)
        
    4.  [AI](https://astro-website-81r.pages.dev/articles/2024-crypto-market-outlooks/#ai)
        
    5.  [DePIN](https://astro-website-81r.pages.dev/articles/2024-crypto-market-outlooks/#depin)
        
    6.  [User Experience (UX)](https://astro-website-81r.pages.dev/articles/2024-crypto-market-outlooks/#user-experience-ux)
        
    7.  [NFTs](https://astro-website-81r.pages.dev/articles/2024-crypto-market-outlooks/#nfts)
        
    8.  [Decentralized Social Media (DeSoc)](https://astro-website-81r.pages.dev/articles/2024-crypto-market-outlooks/#decentralized-social-media-desoc)
        
4.  [Selected Perspectives](https://astro-website-81r.pages.dev/articles/2024-crypto-market-outlooks/#selected-perspectives)
    1.  [Messari Outlook](https://astro-website-81r.pages.dev/articles/2024-crypto-market-outlooks/#messari)
        
    2.  [VanEck Outlook](https://astro-website-81r.pages.dev/articles/2024-crypto-market-outlooks/#vaneck-outlook)
        
    3.  [Pantera Capital Outlook](https://astro-website-81r.pages.dev/articles/2024-crypto-market-outlooks/#pantera-capital-outlook)
        
    4.  [Coinbase Institutional Outlook](https://astro-website-81r.pages.dev/articles/2024-crypto-market-outlooks/#coinbase-institutional-outlook)
        
    5.  [a16z crypto Outlook](https://astro-website-81r.pages.dev/articles/2024-crypto-market-outlooks/#a16z-crypto-outlook)
        
5.  [Appendix](https://astro-website-81r.pages.dev/articles/2024-crypto-market-outlooks/#appendix)
    1.  [About the Source Companies](https://astro-website-81r.pages.dev/articles/2024-crypto-market-outlooks/#about-the-source-companies)
        

![Word cloud](/_astro/fb283dcf-2ff0-4ea9-aeea-bfbc44f3d4d9-1-1024x517.CVTgJIzY_ZGhzcu.webp)

This word cloud was generated from the text of all the reports mentioned in the article.

Common Themes
-------------

The most common themes are Bitcoin, DeFi, Gaming, Decentralized Social Media (DeSoc), AI, Decentralized Physical Infrastructure Networks (DePIN), improving user experience, and tokenizing real-world assets (RWAs).

#### Bitcoin

Across the reports, there is a uniform view that Bitcoin’s dominance is expected to persist, driven by growing institutional interest and validation as a unique digital store of value and inflation hedge. The [upcoming halving event](https://www.investopedia.com/bitcoin-halving-4843769)
 is highlighted as a potential catalyst for appreciation. However, near-term pressures from events like [Mt. Gox distributions](https://www.theblock.co/post/252274/mt-gox-deadline-october-2024)
 and [FTX liquidations](https://www.forbes.com/sites/stevenehrlich/2023/11/15/bankrupt-ftx-wants-to-sell-100-million-of-crypto-per-week-will-the-sec-stop-it)
 may create volatility.

There is excitement around a Bitcoin ETF. An [ETF](https://www.investopedia.com/terms/e/etf.asp#toc-what-is-an-exchange-traded-fund-etf)
 (Exchange-Traded Fund) is a type of investment fund and exchange-traded product, i.e., they are traded on stock exchanges. ETFs are similar to mutual funds but are listed on exchanges, and ETF shares trade throughout the day just like ordinary stock. Some ETFs track an index of assets, like a stock index or bond index. As of late 2023, the U.S. Securities and Exchange Commission (SEC) indicates a potential change in stance regarding approving spot Bitcoin ETFs. Officials met with representatives from several companies, such as BlackRock and Grayscale Investments, instructing them to make final application changes with an [anticipated approval of several ETFs](https://www.reuters.com/technology/sec-tells-spot-bitcoin-etf-hopefuls-make-final-changes-by-year-end-sources-2023-12-22/)
 early in 2024. This marks a significant shift from the SEC’s previous rejections of spot Bitcoin ETFs, citing market manipulation risks. The recent developments are driven by increasing signs of regulatory openness and possibly a recent federal court decision, which found the SEC had erred in a past rejection. The industry expects the SEC to give multiple applications a green light simultaneously.

#### DeFi

### DeFi / Messari

DeFi represents just under 0.01% of the $510 trillion value of global financial assets, indicating significant room for expansion. The report highlights DeFi’s resilience under stress conditions, citing its performance during major crypto events like the collapse of Luna and FTX and the freezing of the U.S. banking system. The report also mentions that according to the IMF, DeFi applications have the lowest marginal cost of capital, which is three times lower due to reduced operational costs.

Despite this, the report acknowledges a shift towards more regulated DeFi, suggesting that many top projects are preparing for rigorous oversight. It posits that while this might seem like a re-intermediation of finance, a better view is to consider DeFi as a bridge in a post-regulation age. This bridge would allow jurisdictions requiring more intermediation for regulatory reasons to treat front-end operators like financial intermediaries while enabling Wall Street to start utilizing these platforms. The report concludes by likening the move towards a more regulated DeFi to a gradual, multi-decade technical upgrade similar to the financial industry’s adaptation to the internet.​

More details:

*   **RWA Diversification**: [Real World Assets](https://cointelegraph.com/learn/tokenized-real-world-assets-rwa-in-defi)
     (RWAs) on public blockchains are expected to be significantly driven by DeFi natives rather than traditional finance.
    
*   **Preference Trends**: There’s an anticipated preference for platforms like Coinbase over traditional financial institutions like JPM for diversifying into other assets.
    
*   **Bitcoin’s Layer 2 Ecosystem**: The report highlights Bitcoin’s vibrant Layer 2 ecosystem led by [Stacks](https://www.stacks.co/)
    , which is expected to provide better access to Bitcoin liquidity and security with fewer trust assumptions. This also includes developments in rollups on Bitcoin and indicates a broader integration of Bitcoin with other DeFi and financial applications​​.
    
*   **Creator Economy**: Traditional social platforms capture significant revenue, and the report suggests that DeFi and blockchain can provide alternatives where creators retain more value, showing the economic shift towards more equitable models favored by DeFi principles​​.
    
*   **Fan Economy & Exclusivity**: friend.tech, a platform that allows users to create and join exclusive private chat rooms, showcases how DeFi concepts like bonding curves are being applied beyond financial products to social and fan-based applications, indicating the broadening scope of DeFi’s impact​​.
    
*   **Token Bound Accounts**: The evolution of NFT marketplaces to accommodate dynamic assets, like [Token Bound Accounts](https://medium.com/future-primitive/tldr-nfts-have-their-own-wallets-try-it-here-http-tokenbound-org-6fac135a1f9d)
     (TBAs), represents an advancement in how DeFi can support complex ownership and trading models​​.
    
*   **Decentralized Storage**: Decentralized storage networks are growing rapidly, offering cost-effective alternatives to traditional cloud providers like Amazon S3, and are an integral part of the infrastructure that DeFi applications can build upon.
    

### DeFi / VanEck

The report anticipates that KYC-enabled and walled garden apps, such as those using [Ethereum Attestation Service](https://attest.sh/)
 or [Uniswap Hooks](https://blog.uniswap.org/uniswap-v4)
, will gain substantial traction, potentially approaching or even surpassing non-KYC applications in user base and fees. This movement is expected to drive institutional liquidity and volume to protocols like Uniswap by enabling new entrants to participate in DeFi without interacting with OFAC-sanctioned entities. The introduction of hooks will reinforce Uniswap’s competitiveness and drive token appreciation, particularly once the DAO votes in favor of activating the Uniswap Protocol fee switch, which is expected to be no higher than 10 basis points.

Other details:

*   Post-[EIP-4844](https://www.eip4844.com/)
    , Ethereum’s upgrades are expected to significantly reduce transaction fees and improve scalability, consolidating dominant players in the Layer 2 space and increasing their transaction volume and total value locked.
    
*   Binance is expected to lose its top spot in spot trading volumes due to regulatory challenges and the rise of competitors, with implications for the broader ecosystem of centralized exchanges.
    
*   DEX Market Share Growth: Decentralized exchanges are expected to increase their share of spot trading, driven by improvements in blockchain technology and increased adoption of self-custody solutions.
    
*   New use cases for Bitcoin, particularly in remittances and as collateral in various DeFi protocols, are anticipated to emerge, creating new yield-generating opportunities.
    
*   [Solana’s Performance and DeFi TVL](https://defillama.com/chain/Solana)
    : Solana is predicted to outperform Ethereum and other blockchains in market cap, total value locked, and active users, partly due to advancements in its ecosystem and technology.
    

### DeFi / Pantera Capital

Bitcoin and the overall cryptocurrency market capitalization have seen significant rebounds since the start of the year, with the market capitalization of cryptocurrencies doubling from $0.8 trillion to $1.6 trillion.

The report mentions key legal wins, such as Ripple’s native token [XRP being ruled not a security](https://www.investopedia.com/what-does-the-xrp-not-a-security-ruling-mean-for-other-cryptocurrencies-7561320)
 and [Grayscale’s legal victory against the SEC](https://www.forbes.com/sites/digital-assets/2023/08/29/grayscale-wins-court-battle-against-sec-regulator-must-review-bitcoin-etf-application/)
 regarding their Bitcoin ETF application. These are seen as positive steps toward a favorable regulatory landscape for blockchain in the U.S.

### DeFi / Odos

**L2s Capturing DeFi:** Throughout 2023, we saw significant adoption of (and value transferred to) L2 ecosystems with notable events such as the Arbitrum airdrop, full launch of zkSync Era and Base, as well as major updates and announcements of new L2s such as Scroll, Linea, and Polygon zkEVM. Looking ahead to 2024, as the adoption of these L2s continues, we see users bringing massive amounts of liquidity and a slow but consistent transfer of DeFi activity from Ethereum Mainnet to these L2s. As they mature and prove their security models, we anticipate a significant share of all DeFi activity will begin to take place on L2s driven by the cheaper gas fees.

**Rise of On-Chain Order Books:** A very early movement we’ve observed picking up momentum into 2024 is the rise of on-chain order books. On-chain order book protocols can only sustainably exist on much cheaper blockchains, such as the L2s discussed above, and are an exciting “L2 native” evolution of DeFi. We have heard concerns that on-chain order books will replace or out-compete automated market makers (AMMs), but we don’t believe this will take shape. DeFi’s success has been largely driven by the composability of protocols and a diverse and innovative landscape of protocols to choose from. On-chain order books will be an additional dimension to DeFi, adding value, specifically to L2s.

**Liquid Token Trading Firms Expand:** Each year, the crypto community, specifically DeFi, discusses the inevitable entrance of institutions that will bring billions of trading volume. As we await this more formal and restricted set of investors, we have seen an uptick heading into 2024 of advanced trading firms specializing in on-chain strategies. These firms will be incredibly active next year if activity picks up and should be a catalyst for DeFi protocols they leverage to trade and capture yield.

#### Gaming

### Gaming / Messari

The report highlights approximately 3.44 billion gamers globally, contributing to an expected $184 billion in revenue for the gaming industry in 2023. This number is anticipated to continue increasing. Gen Z and Gen Alpha are particularly engaged, spending around 15 hours a week gaming and having spent $135 billion in 2022 on virtual gaming items. The report notes a key trend where younger gamers increasingly demand actual ownership of virtual goods, signaling a shift from today’s prevalent models where game items are not truly owned or freely tradeable​.

### Gaming / VanEck

The report predicts that blockchain gaming will see a title surpassing 1 million daily active users, demonstrating the potential of blockchain in the gaming industry.

[Immutable](https://www.immutable.com/)
 X is highlighted as a likely candidate to become a top 25 coin by market cap, associated with the release of high-budget games like [Illuvium](https://illuvium.io/)
 and [Guild of Guardians](https://www.guildofguardians.com/)
. Immutable is noted for designing a token that better aligns interests than most and has been building multiple AAA games set for release in 2024.

Immutable has been working on resolving many technical pain points that have previously hindered the success of web3 gaming, such as wallet management. Immutable’s [Passport](https://www.immutable.com/products/passport)
 is expected to allow users to log in to games and manage their blockchain-based game items with a familiar single sign-on process while abstracting away the more complex blockchain interactions.

The [WAX](https://www.wax.io/)
 blockchain is mentioned as leading the gaming sector with many daily unique active wallets, primarily playing games like [Alien Worlds](https://alienworlds.io/)
. However, the report also indicates a caveat that many players in such games might be bots.

### Gaming / Coinbase Institutional

The report suggests that web3 gaming has seen renewed interest in the second half of 2023 after a decline earlier in the crypto winter. The gaming sector, a significant portion of the entertainment industry, currently boasts a total addressable market of approximately $250 billion, with expectations to grow to $390 billion over the next five years. Despite the vast opportunities, the report notes skepticism towards the “play-to-earn” models exemplified by early projects like [Axie Infinity](https://time.com/6199385/axie-infinity-crypto-game-philippines-debt/)
, which has led to broader disapproval among mainstream gamers.

The report indicates that this skepticism fosters innovation, with developers experimenting to merge high-quality game design with viable financial mechanics, such as minting web3 primitives like non-fungible tokens (NFTs) for use, transfer, or sale in-game or on designated marketplaces. However, most gamers reportedly dislike NFTs, reflecting a rejection of the play-to-earn or pay-to-play models. The gaming industry sees value in leveraging web3 architecture to enhance user acquisition and retention potentially, but this benefit remains unproven. With many projects approaching significant milestones in their development cycle, the release of some web3 games in 2024 is expected to provide better insight and data to assess this sector’s potential​.

#### AI

### AI / Messari

The document emphasizes the significance of hardware in the AI revolution, highlighting the role of decentralized GPU marketplaces like [Gensyn](https://www.gensyn.ai/)
. These marketplaces are portrayed as solutions to the imbalance between the short supply and increasing demand for GPUs needed for AI model training, creating a decentralized supercomputer network. Another project, [Bittensor](https://bittensor.com/)
, is mentioned as a platform incentivizing the creation and sharing of machine intelligence, enabling individuals to contribute to open-source AI and monetize their work.

The convergence of AI and crypto is seen as expanding the design horizons of crypto, with three key synergies highlighted:

*   **AI agents utilizing crypto infrastructure**: AI can autonomously use digital resources such as storage, compute, and bandwidth made available through crypto infrastructure.
    
*   **zkML innovations**: These allow smart contracts to query AI models securely, thus expanding the capabilities inherent to blockchains.
    
*   **Tokenization for rewarding contributions**: Tokens are seen as a way to reward individuals for fine-tuning AI models and collecting valuable real-world data intersecting with DePIN.
    

### AI / a16z crypto

The report suggests that AI, as exemplified by models like ChatGPT, is predominantly in the realm of tech giants due to significant compute and data requirements. Decentralized blockchains are seen as a critical counterbalancing force.

*   **AI in Gaming and Web3:** AI is poised to revolutionize gaming, not just in gameplay but in governance and operational integrity. With AI-generated game elements, the need for guarantees about the neutrality and integrity of these elements becomes crucial. Crypto technology is highlighted to offer such guarantees, ensuring the fairness and credibility of AI-generated content in gaming.
    
*   **NFTs and AI:** Integrating NFTs into mainstream brand strategies is highlighted as a growing trend with potential implications for AI. As brands increasingly adopt NFTs for identity, community, and customer engagement, there’s an implication that AI could play a role in creating, managing, or personalizing these digital assets, especially as they become more interactive and complex.
    
*   **SNARKs and AI:** [SNARKs](https://en.wikipedia.org/wiki/Zero-knowledge_proof)
     are mentioned as an emerging technology with the potential to verify AI computations and enhance the integrity and privacy of AI operations.
    

#### DePIN

DePIN (Decentralized Physical Infrastructure Networks) is an approach to building and maintaining infrastructure in the physical world, such as WiFi hotspots and solar-powered batteries. It leverages widespread Internet connectivity, blockchain infrastructure, and cryptography to enable a decentralized build-out by individuals and companies globally. Participants in DePINs receive financial compensation and ownership stakes through token incentives.

### DePIN / Messari

DePIN encompasses storage, computing, wireless connectivity, energy networks, and geospatial data collection. It operates through two main branches: Physical Resource Networks (PRNs) and Digital Resource Networks (DRNs). PRNs deploy location-dependent hardware for resources like energy and connectivity, while DRNs provide a location-independent backend for the cloud, focusing on digital resources like compute, storage, and bandwidth.

The report highlights the increasing demand for DRNs, noting the vast potential for adoption. The cloud storage market is substantial, valued at $80 billion and growing by 25% annually, yet decentralized alternatives cater to less than 0.1% of this market despite offering costs 70% lower than traditional providers like Amazon S3. This gap indicates a significant opportunity for expansion and market penetration by DePIN solutions.

Specific decentralized storage providers are mentioned, including:

*   **[Filecoin](https://filecoin.io/)
    **: Primarily provides [cold storage](https://www.cloudwards.net/hot-storage-vs-cold-storage)
     solutions, focusing on cost-effective archival data storage. It’s recognized for its capacity and utilization rates, making it a dominant player in the storage network.
    
*   **[Storj](https://www.storj.io/)
    **: Known for [hot storage](https://www.cloudwards.net/hot-storage-vs-cold-storage)
     solutions with fast retrieval times, targeting enterprises with compatibility to Amazon S3, particularly utilized in the media and entertainment industries.
    
*   **[Sia](https://sia.tech/)
    **: Targets developers with its private storage solutions and is known for quick retrieval times in the hot storage market.
    
*   **[Arweave](https://www.arweave.org/)
    **: Attracts NFT, metaverse, and decentralized social projects with its “pay once, store forever” model. It’s expected to see a significant increase in demand over the coming years.
    

The report underscores that most businesses are unlikely to return to managing complex hardware setups in-house but are also wary of relying entirely on centralized Big Tech vendors.

### DePIN / VanEck

The outlook report appears to have a positive yet cautious perspective on DePIN. Specifically:

*   **Meaningful Adoption of DePIN Networks**: The report predicts that in 2024, multiple DePIN networks will see meaningful adoption and capture public attention. The focus seems to be on networks that leverage decentralized contributions to map or provide services traditionally dominated by centralized entities.
    
*   **Hivemapper’s Growth**: It specifically mentions [Hivemapper](https://www.hivemapper.com/)
    , a decentralized mapping protocol, aiming to be a community-owned alternative to Google Streetview. The report anticipates that Hivemapper will map its 10 millionth unique kilometer, surpassing 15% of global road capacity. This signifies a substantial growth and adoption rate, implying a speed and cost-of-capital advantage over traditional mapping services like Google. Hivemapper incentivizes contributors with its native token, \[$HONEY\]([https://coinmarketcap.com/currencies/hivemapper/](https://coinmarketcap.com/currencies/hivemapper/)
    ), to expand its database, indicating a successful utilization of crypto-economic principles to fuel its network growth.
    
*   **Helium’s Expansion**: [Helium](https://www.helium.com/)
     ($[HNT](https://coinmarketcap.com/currencies/helium/)
    ) is a decentralized network of wireless hotspots. The report foresees Helium reaching 100,000 paying subscribers for its nationwide US 5G plan, up from the current count. Helium’s model allows anyone to set up hotspots and earn crypto tokens, showcasing a significant move towards decentralized infrastructure in telecommunications. The model’s benefits include being capital-light for Helium and turning hotspot providers into network advocates due to their ongoing stake.
    
*   **Market Implications**: The implication of these networks’ growth is significant in terms of disintermediating legacy providers and potentially reducing costs. For example, Helium claims to deliver data at less than 50% of the cost of legacy networks.
    

#### User Experience (UX)

### UX / Coinbase Institutional

Coinbase Institutional emphasizes the challenge of managing crypto assets, given the complexities involved, such as handling wallets, private keys, and gas fees, which can deter wider adoption. To address this, the industry is making the technology more user-friendly and accessible. A notable advancement in this direction is the progress around [account abstraction](https://cointelegraph.com/learn/account-abstraction-guide-to-ethereums-erc-4337-standard)
, especially with Ethereum’s introduction of the [ERC-4337](https://www.erc4337.io/)
 standard in March 2023, simplifying user experience by treating externally owned and smart contract accounts similarly. This allows for more flexible transaction funding, including institutional entities acting as “paymasters” to cover transaction fees, and potentially supports “gasless transactions” that lower the entry barrier for new and existing users.

More details:

*   **Account Abstraction**: Aimed at simplifying crypto interactions by similarly treating externally owned accounts (like wallets) and smart contract accounts. Ethereum’s introduction of the [ERC-4337](https://www.erc4337.io/)
     standard in March 2023 represents a significant step in this direction, offering new opportunities for user interaction and engagement. ERC-4337 also allows application owners on Ethereum to act as “paymasters,” covering the gas fees for users or enabling users to fund transactions with non-ETH tokens. These features appeal to institutional entities that prefer not to hold gas tokens due to their price variability or other concerns.
    
*   **Project Guardian & J.P. Morgan’s Involvement**: A proof-of-concept report from J.P. Morgan as part of [Project Guardian](https://www.jpmorgan.com/onyx/project-guardian)
     illustrates the practical applications of these advancements. It highlights all gas payments handled via [Biconomy’s Paymaster service](https://www.biconomy.io/paymaster)
    , indicating the industry’s shift towards more user-friendly transaction processes.
    
*   **Dencun Upgrade**: This [upcoming upgrade](https://www.coindesk.com/tech/2023/05/11/meet-dencun-ethereum-developers-are-already-planning-next-hard-fork/)
     is expected to reduce rollup transaction fees by 2-10 times. The report suggests that this could lead more decentralized applications (dapps) to pursue a “gasless transactions” model, allowing users to focus only on high-level interactions and possibly enabling new non-financial use cases.
    
*   **Robust Wallet Recovery Mechanisms**: Improving user experience involves creating failsafes against common issues like losing a private key. Account abstraction can facilitate such robust wallet recovery mechanisms, further reducing the barriers to entry for new users and enhancing the overall security and usability of crypto assets.
    

### UX / Odos

**Intents:** 2023 has set the stage for launching a new class of intent-based products in 2024. A very early but hot narrative expanding beyond DeFi is centered around “Intents” as a new paradigm for defining actions on a blockchain. Previously, users defined and executed specific transactions. Intents allow users to more abstractly define a goal to achieve, even if they don’t know how to get there, such as: “I want to swap ETH into USDC; find me the highest output”. Without the user needing to find many venues, compare rates, and explore arbitrage opportunities across other assets, it becomes clear what the UX improvements intents can provide. Combining these composable user objectives with other ongoing efforts, such as account abstraction, redefines crypto UX for the better.

#### NFTs

### NFTs / Messari

The report highlights how [Blur](https://blur.io/)
 impacted the Ethereum NFT ecosystem by introducing a trader-friendly user experience and innovative lending products like [Blend](https://www.coindesk.com/web3/2023/05/25/blend-seizes-82-of-nft-lending-market-share-dappradar/)
. These changes directly challenged OpenSea’s dominance and stirred mixed reactions due to the aggressive tactics employed. Despite some [controversy](https://www.proactiveinvestors.com/companies/news/1006795/blur-s-nft-marketplace-in-royalties-spat-with-opensea-1006795.html#:~:text=Blur%20does%20not%20enforce%20buyers,they%20are%20listed%20on%20both.)
, the report asserts that the building blocks provided by Blur are crucial for the long-term vibrancy of the NFT market. The perspective is optimistic about the future of NFTs.

The report discusses another NFT-related development under “Project Guardian,” described as the best-looking “Crypto Mullet of 2023.” This initiative is a collaboration between traditional financial institutions like JPMorgan and Apollo with blockchain entities such as [Avalanche](https://www.avax.network/)
, [LayerZero](https://layerzero.network/)
, [Axelar](https://axelar.network/)
, and Oasis [Provenance](https://provenance.io/)
. The project aimed to create a product for TradFi that allows asset managers to tokenize funds and better manage rules-driven active portfolios. The significance of Project Guardian lies in its technological capability, indicating that the technology not only works but might be offered in a regulatory-compliant manner.

### NFTs / VanEck

The report predicts a significant resurgence in NFT activity, projecting that monthly volumes will approach new all-time highs due to several factors: As crypto markets revive, speculators are expected to return, bringing liquidity and trading volumes to the NFT market.

*   **Improvement in Crypto Games and Platforms**: The report indicates a development in the quality and appeal of crypto games and platforms supporting NFTs.
    
*   **The emergence of Bitcoin-based NFTs**: While Ethereum has historically dominated the NFT market, the report highlights the growing role of Bitcoin in the NFT space, particularly through its Ordinals protocol and emerging Layer 2 solutions. This development is expected to diversify the NFT landscape and contribute to a rebirth in Bitcoin network fees. By the end of 2024, the ETH-to-BTC primary NFT issuance ratio is anticipated to narrow to approximately 3-1, indicating a substantial increase in Bitcoin’s NFT activities.
    
*   **Stacks (STX) Growth**: [Stacks](https://www.stacks.co/)
    , a smart contract platform that leverages Bitcoin’s security, is expected to rise significantly in market capitalization, becoming a top-30 coin. This growth reflects the broader market’s increasing interest and investment in Bitcoin-based NFTs and related infrastructure.
    

### NFTs / Coinbase Institutional

The report discusses the trend toward modular blockchains like [Celestia](https://celestia.org/)
. It indicates that NFTs are part of this trend as the blockchain technology landscape evolves to cater to more specialized needs.

### NFTs / a16z crypto

Starbucks has introduced a [gamified loyalty program](https://odyssey.starbucks.com/#/landing)
 involving digital assets, while [Nike](https://www.swoosh.nike/)
 and [Reddit](https://nft.reddit.com/)
 have developed digital collectible NFTs aimed at broader audiences. This trend signifies a shift toward integrating NFTs into everyday consumer interactions and branding strategies.

Brands are leveraging NFTs as marketing tools and as a means to enhance customer identity and community affiliations and even co-create products with enthusiasts. NFTs are seen as a bridge between physical goods and their digital representations.

The report highlights a growing trend of inexpensive NFTs designed for large-scale collection as consumer goods. This approach is often facilitated through custodial wallets or Layer 2 blockchains that offer lower transaction costs, making it more feasible for widespread adoption.

Going into 2024, the conditions are favorable for NFTs to become ubiquitous as digital assets across various companies and communities. The [anticipated book](https://www.penguinrandomhouse.com/books/723505/the-everything-token-by-steve-kaczynski-and-scott-duke-kominers/)
 by Steve Kaczynski and Scott Duke Kominers is expected to delve deeper into this phenomenon, suggesting a strong future trajectory for NFTs as integral to brand strategies and consumer interaction.

#### Decentralized Social Media (DeSoc)

Desoc stands for “decentralized social media,” referring to the movement within the crypto that aims to build social media platforms and applications on decentralized networks. The primary goal of DeSoc is to address issues in traditional social media networks, typically controlled by a single entity or a small group of entities. These issues include concerns over data privacy, censorship, algorithm transparency, and equitable revenue distribution.

### DeSoc / Messari

*   **DeSoc Platforms Progress**: The report mentions several platforms making strides in DeSoc, including [Farcaster](https://www.farcaster.xyz/)
    , [Lens](https://www.lens.xyz/)
    , [Yup](https://yup.io/)
     (a DeSoc aggregator), [DeSo](https://www.deso.com/)
    , and [XMTP](https://xmtp.org/)
    , which has a significant integration with Coinbase.
    
*   **Existential Need for DeSoc**: It articulates that DeSoc is no longer a nice-to-have but is existential for maintaining free speech, particularly in the face of censorship and [digital un-personing](https://www.wsj.com/articles/when-digital-platforms-become-censors-1534514122)
    .
    
*   **Key Factors Driving DeSoc**:
    
    *   **Portable Social Graphs**: DeSoc addresses the risk of losing a social network by allowing users to maintain their social graph across different platforms. For instance, 70% of Lens users interact with more than one app, indicating the growing acceptance of portable social graphs.
        
    *   **Anti-Censorship**: It underscores the importance of anti-censorship to ensure unfettered crypto conversations, considering the increasing censorship on traditional platforms.
        
    *   **Control Over Algorithms**: DeSoc allows more interfaces for developers and users to create and control their social media experience, including customized feeds.
        
    *   **Creator Cash**: Traditional social platforms capture the majority of revenue, but DeSoc, leveraging crypto’s financial technology roots, offers a more equitable financial model for creators. friend.tech is noted for showing potential in leveraging economic incentives to onboard users.
        

### Tokenizing Real-World Assets (RWAs)

Tokenizing real-world assets (RWAs) refers to converting rights to an asset into a digital token on a blockchain. RWAs include real estate, art, commodities, shares of companies, or even intangible assets like intellectual property or rights to revenues.

### RWAs / Messari

Messari highlights the growing trend towards tokenizing real-world assets, especially in DeFi lending protocols. Specifically, the report notes a significant shift in MakerDAO’s reserve composition, [moving towards tokenized treasuries](https://www.theblock.co/post/249939/makerdao-tokenized-t-bills)
, which have grown from $40 million in mid-2022 to nearly $3 billion today. This trend is significant for a few reasons:

*   **Growth and Tokenization**: The move towards RWAs captures yield-bearing stablecoin holdings at exchanges like Coinbase and DeFi lending protocols. This trend is part of a broader shift towards incorporating traditional financial assets into the blockchain space.
    
*   **Safety and Global Scale**: RWAs in DeFi lending platforms are considered safer and more scalable globally compared to CeFi services. They have demonstrated resilience against various crises in the financial markets and continue to operate transparently and efficiently.
    
*   **Collateral and Yield Diversification**: [Liquid Staking Tokens](https://www.coindesk.com/markets/2023/11/29/liquid-staking-tokens-are-a-hot-ticket-for-2024/)
     (LSTs) have become a popular collateral type within these platforms. For example, in MakerDAO, LSTs constitute a significant portion of deposits. This diversification allows DeFi-native investors to leverage different types of yields and collateral, further enhancing the attractiveness of DeFi loans over CeFi alternatives.
    

More details:

*   **Centrifuge’s Role:** [Centrifuge](https://centrifuge.io/)
     is highlighted as a leader in the RWA space. It has the highest token market cap, the highest market share by private loans outstanding, and the lowest ratio of default to active loans among the three leading RWA credit protocols (the other two being Maple and Goldfinch). Centrifuge has $250 million in active loans, primarily due to a significant partnership with BlockTower Credit and MakerDAO, where MakerDAO has issued DAI loans backed by real-world instruments managed by BlockTower.
    
*   **Stellar’s Involvement:** [Stellar](https://stellar.org/)
     has generated interest from financial institutions for tokenized Treasury experimentation. Franklin Templeton’s $330 million tokenized [Treasury initiative](https://www.franklintempleton.com/press-releases/news-room/2023/franklin-templeton-money-market-fund-launches-on-polygon-blockchain)
     and WisdomTree’s smaller pilot are significant components of this, representing a large portion of tokenized Treasuries.
    
*   **Christine Moy’s Influence:** [Christine Moy](https://www.apollo.com/aboutus/leadership-and-people/christine-moy.html)
    , who previously led JPMorgan’s blockchain team and now manages Crypto Data and AI Strategy at Apollo, is recognized for her deep integration of crypto and traditional finance. She was instrumental in the Apollo-JPMorgan “Project Guardian” proof-of-concept, involving several crypto projects.
    
*   **Broader Trends and Concerns:** The report discusses whether the early leads of these initiatives will translate into growth in 2024 or if the RWA movement is another false start.
    

### RWAs / Coinbase Institutional

*   **Significant Growth and Notable Developments:** The report specifically mentions the growth of [Maker RWA collateral](https://endgame.makerdao.com/concepts/collateral-breakdown)
     to more than $3 billion, highlighting it as a significant example of institutional interaction with public chains.
    
*   **Adoption and Technological Integration:**
    
    *   A surge in yield-seeking behavior due to higher front-end bond yields has led to increased demand for protocols tokenizing US Treasuries and a decline in interest for private credit protocols with higher default risks and lower liquidity.
        
    *   The report mentions the use of specific tools and technologies, such as Uniswap V4’s hook-driven architecture and Coinbase’s [Verified by Coinbase](https://help.coinbase.com/en/coinbase/getting-started/getting-started-with-coinbase/id-doc-verification)
         attestations, to facilitate on-chain tokenized US Treasuries, which either block US persons or require KYC verification.
        
*   **Institutional Use Cases and Efficiency:**
    
    *   Institutions’ focus on capital market instruments is underscored, with a particular interest in bank deposits, money market funds, and repurchase agreements. The report notes the importance of instantaneous settlement in higher interest rate environments, where capital efficiency becomes much clearer.
        
    *   Despite the shift towards a [one-business-day settlement period (T+1)](https://www.davispolk.com/insights/client-update/sec-adopts-t1-settlement-effective-may-2024)
         by May 2024, the report emphasizes reducing settlement times further in markets, transacting hundreds of billions to over a trillion US dollars daily.
        
*   **Challenges and Constraints:**
    
    *   Most tokenization efforts are focused on permissioned chains and closed-source smart contracts, with dominant technology providers being [Hyperledger](https://www.hyperledger.org/)
        , Consensys’ [Quorum](https://consensys.io/quorum)
        , Digital Asset’s [Canton](https://www.canton.network/)
        , and R3’s [Corda](https://r3.com/products/corda/)
        . These efforts face challenges in interoperability and the risk of liquidity fragmentation.
        
    *   The report cites J.P. Morgan’s use of multiple bridging technologies in November 2023 to demonstrate ongoing initiatives to address cross-chain liquidity constraints.
        
*   **Regulatory Landscape and Compliance:** Regulatory hurdles remain significant, with the market lacking established legal precedents and templates. The report, however, notes increasing regulatory clarity, particularly in the EU with the [DLT Pilot Regime](https://www.esma.europa.eu/press-news/esma-news/esma-publishes-report-dlt-pilot-regime)
    , which is expected to facilitate legal and technical advancements in 2024.
    

* * *

Selected Perspectives
---------------------

This section of the article highlights selected perspectives from each of the outlooks.

#### Messari Outlook

Original report: [https://messari.io/crypto-theses-for-2024](https://messari.io/crypto-theses-for-2024)
. Also, [here is a great Bankless interview](https://www.youtube.com/watch?v=WGL7jxPZuDM)
 where Ryan Selkis summarizes Messari’s outlook.

### Investment Trends

**Bitcoin (BTC) & Digital Gold:** Messari posits a strong long-term case for Bitcoin, viewing it as a hedge against government fiscal irresponsibility and a beneficiary of digital transformation and periodic halving events. While short-term predictions are uncertain, the long-term bullishness is supported by Bitcoin’s fundamental scarcity and institutional interest.

**Ethereum (ETH) & The World Computer:** Ethereum’s investment case is compared more to financial networks like Visa or JPMorgan than gold or commodities.

**The Resurgence of Private Crypto Markets:** Many crypto fund managers are underperforming the market and facing significant challenges. Messari outlines the difficulties and risks inherent in the liquid crypto markets due to technical and counterparty risks, high transaction fees, and intense competition. It contrasts this with the private crypto venture markets, which it describes as being in a “veritable valley of death” due to various challenges, including decimated VC markets, impacts from fraud, regulatory crackdowns, and a shift in focus to AI as a more attractive sector in tech.

However, despite these challenges, the report expresses optimism for new crypto venture investors. They argue that funds with a 2023 vintage might outperform traditional markets in the medium to long term and possibly even outperform BTC/ETH benchmarks due to the anomalously low entry prices of the year.

**Policy Meta:** The report explores the broader implications of crypto policy, drawing parallels with the “[Crypto Wars](https://groups.csail.mit.edu/mac/classes/6.805/student-papers/fall95-papers/kokoski-crypto.html)
” of the 1990s and discussing the changing landscape of American hegemony and its impact on crypto regulation. Messari suggests a need for a strategic approach to elections and policy to ensure the crypto industry’s growth and freedom.

**AI & Crypto, Money for the Machines:** Messari argues that as AI proliferates, the need for crypto’s provenance, scarcity, and security features will become more pronounced, making crypto an integral part of the digital future.

### Products to Watch in 2024

**USDT on Tron:** [Tether, particularly on the Tron network](https://research.kaiko.com/insights/why-is-tethers-market-cap-approaching-all-time-highs)
, has been highlighted for its growing significance as a store of value and a payment mechanism, especially in developing countries. It represents a substantial part of the global stablecoin market and is being adopted for various use cases beyond mere trading vehicles.

**BASE from Coinbase:** Coinbase’s entry into the rollup market with BASE is mentioned as a significant development. BASE is praised for its user-friendly bridging experience and is seen as potentially pioneering in navigating users between centralized and decentralized services.

**[Celestia](https://celestia.org/)
:** Celestia provides data availability and consensus for modular blockchains. It’s a significant project due to its approach to reducing base layer congestion and costs for rollups and other networks.

**[Firedancer](https://github.com/firedancer-io/firedancer)
:** Developed by [Jump Crypto](https://jumpcrypto.com/)
, Firedancer is a validator client for Solana to enhance scalability and performance. Messari highlights its potential to increase Solana’s transaction capacity and network throughput significantly.

**[Farcaster](https://www.youtube.com/watch?v=vMWjol6xHJ0)
:** Recognized for its clean Web2 social application feel, Farcaster uses a hybrid back-end architecture for identity management and social interactions. It’s seen as a promising venture in the decentralized social media space.

**[Lido](https://lido.fi/)
:** The [Shapella upgrade](https://blog.lido.fi/ethereum-shapella-overview-faq/)
 in April led to a substantial increase in capital inflows into Ethereum staking as investors awaited the ability to stake with full redeemability and the flexibility of withdrawals. The report also discusses the risks associated with liquid staking, notably the potential for network security issues due to validation delays and the disincentive to hold ETH if LST liquidity eclipses ETH liquidity, leading to unpredictable outcomes.

**[CCIP](https://chain.link/cross-chain)
 (Chainlink’s Cross-Chain Interoperability Protocol):** CCIP is lauded for its contribution to enhancing security and functionality in cross-chain transfers and communications. Messari highlights its early adoption and potential for growth in the interoperability space.

**[Blur](https://blur.io/)
, [Blend](https://www.paradigm.xyz/2023/05/blend)
, [Blast](https://www.coingecko.com/learn/what-is-blast-crypto-l2-optimistic-rollup)
:** These are discussed as part of the evolving NFT market, focusing on Blur’s impact on the Ethereum NFT ecosystem and its approach to financializing NFTs.

**[Project Guardian](https://www.jpmorgan.com/onyx/project-guardian)
:** A collaborative effort involving JPMorgan and others, Project Guardian is noted for its potential to allow asset managers to tokenize funds and manage rule-driven active portfolios more effectively.

### People to Watch in 2024

**[Larry Fink](https://twitter.com/BlackRock)
 (BlackRock) & [Cathie Wood](https://twitter.com/CathieDWood)
 (ARK Invest):** Their involvement in cryptocurrency through their respective institutions significantly influences the market, especially considering BlackRock’s ETF applications and Wood’s position in the ETF application line.

**[Jeremy Allaire](https://twitter.com/jerallaire)
 & [Dante Disparte](https://twitter.com/ddisparte)
 (Circle):** Recognized for their roles in Circle, particularly in stablecoins and regulatory engagement.

**[Kristin Smith](https://twitter.com/kmsmithdc)
 (The Blockchain Association) & [Michael Carcaise](https://twitter.com/michaelcarcaise)
 (Fair Shake PAC):** Noted for their roles in advocating and shaping policy in the crypto space, reflecting the industry’s ongoing efforts to navigate and influence legislative and regulatory landscapes.

**Senator Elizabeth Warren:** As a prominent critic and regulatory influencer, Warren’s stance and actions significantly affect the industry’s regulatory future.

**Elon Musk:** His influence spans beyond just technology and into the regulatory and cultural discussions surrounding crypto and its broader implications.

**[Michael Sonnenshein](https://twitter.com/Sonnenshein)
 & [Craig Salm](https://twitter.com/CraigSalm)
 (Grayscale):** Messari discusses their roles and the speculative future of Grayscale amidst a tumultuous time for DCG and the broader crypto asset management industry.

**[Nic Carter](https://twitter.com/nic__carter)
 & [Matt Walsh](https://twitter.com/MattWalshInBos)
 (Castle Island Ventures):** Nic Carter is known for his prolific writing and defenses of Bitcoin’s ethos, Proof-of-Work mining, and stablecoins. He became widely recognized for exposing issues within banking regulators, advocating for Proof-of-Reserves, and countering misleading narratives about crypto’s role in terrorist activities. Matt Walsh gained attention for revealing the backstory of the SEC “darling” Prometheus and warning about problematic accounting rules designed to hinder crypto custody by Wall Street firms.

**[Lucas Vogelsang](https://twitter.com/lucasvo)
 (Centrifuge), [Denelle Dixon](https://twitter.com/DenelleDixon)
 (Stellar), and [Christine Moy](https://twitter.com/cmoyall)
 (Apollo):** Recognized for their efforts in bridging traditional finance with the emerging crypto markets, especially in the context of real-world asset tokenization.

**[Dan Romero](https://twitter.com/dwr)
 (Farcaster) & 0x Racer (friend.tech):** Mentioned for their innovative work in decentralized social platforms.

### Crypto Policy Trends

**The Crypto Wars of the 90s:** This period is a historical analog to today’s crypto policy challenges. Messari notes the grassroots rebellions against government overreach and encryption battles, pointing out that these might not repeat given the cultural changes in the U.S. It’s suggested that the original “[cypherpunks write code](https://www.americanscientist.org/article/cypherpunks-write-code)
” ethos from this era might be hard to replicate today due to more profound societal shifts. \[Semiotic note: Vitalik posted a [related article](https://vitalik.eth.limo/general/2023/12/28/cypherpunk.html)\
 after the Messari report was released.\]

**Complacency and Cultural Shifts:** Messari critiques the complacency of the Gen X and Baby Boomer generations, which might have contributed to a loss of civil liberties and an increased national security apparatus. The report suggests that Millennials and Zoomers might be less inclined to fight against these trends, having grown accustomed to a world of eroded civil liberties and pervasive surveillance. The cultural and constitutional shifts post-Patriot Act and post-COVID are significant factors in shaping attitudes toward privacy and security.

**The End of American Hegemony:** The report argues that some government officials believe that the liberal tech policies of the 90s were a mistake, leading to a scapegoating of technology for broader societal issues. There’s a criticism of the desire among some in the U.S. to emulate more controlled internet models like China’s. This trend is juxtaposed with the decline of American global dominance and the rise of rival powers, indicating a need for a new strategy focused on innovative electoral tactics.

#### VanEck Outlook

Original report: [https://www.vaneck.com/us/en/blogs/digital-assets/matthew-sigel-vanecks-15-crypto-predictions-for-2024/](https://www.vaneck.com/us/en/blogs/digital-assets/matthew-sigel-vanecks-15-crypto-predictions-for-2024/)

The authors anticipate a U.S. recession in the first half of 2024, detailing various economic indicators that suggest a downturn.

[Changes in accounting standards](https://dart.deloitte.com/USDART/home/publications/deloitte/heads-up/2023/fasb-issues-asu-crypto-assets)
 are predicted to encourage more corporations to hold cryptocurrencies, with Coinbase anticipated to lead in revealing blockchain revenues.

#### Pantera Capital Outlook

Original report: [https://panteracapital.com/blockchain-letter/a-year-of-progress/](https://panteracapital.com/blockchain-letter/a-year-of-progress/)

Pantera emphasizes the importance of understanding market cycles, particularly the 1-Year HODL Wave, and suggests that we may be at the cusp of another significant growth period based on historical patterns. For investors, the letter implies a strategy of long-term engagement and vigilance for signs of cycle progression.

### What is the 1-Year HODL Wave?

The “1-Year HODL Wave” refers to a metric that tracks the percentage of all bitcoins that have not been moved from one wallet to another for at least one year. The term “HODL” is a community-inspired acronym for “Holding On for Dear Life,” originating from a misspelled “hold” in a Bitcoin forum post. This metric is seen as an indicator of investor sentiment and behavior, often correlating with broader market trends.

### Significance of the 1-Year HODL Wave

The 1-Year HODL Wave is used to gauge the long-term holding sentiment within the Bitcoin community. Here are some insights from the document regarding its significance:

1.  Indicator of Market Cycle: When the percentage of bitcoins held for over a year decreases, it’s often a sign that long-term holders are starting to sell, typically occurring as bitcoin’s price approaches new cycle peaks. This decrease in the 1-Year HODL Wave coincides with increased market activity as more bitcoins become available for trading.
    
2.  Historical Price Action: The document suggests that historically, as the 1-Year HODL Wave indicator reaches new highs, it correlates with market lows. The rationale is that a higher percentage of long-term holding reduces the tradeable supply of Bitcoin, potentially setting up a market upswing as the demand meets a restricted supply.
    
3.  Predictive Nature: According to the letter, most price gains in a cycle historically come after the 1-Year HODL Wave indicator peaks. The document hints that if the past patterns hold, there might be significant gains ahead in the current cycle.
    

When the letter was written, the Pantera authors observed that the 1-Year HODL Wave did not clearly indicate that it had topped out for the current cycle. They correlated this with historical actions around the peaks of this wave to suggest that significant gains might still be ahead for Bitcoin. They also discussed how this wave’s peaks have historically aligned with Bitcoin halvings, suggesting a close watch on the next halving for potential market movements.

#### Coinbase Institutional Outlook

Original report: [https://coinbase.bynder.com/m/c8c6fdc663f44b5/original/2024-Crypto-Market-Outlook-V3.pdf](https://coinbase.bynder.com/m/c8c6fdc663f44b5/original/2024-Crypto-Market-Outlook-V3.pdf)

### Emerging Themes and Long-Term Outlook

Layer-2 scaling solutions have grown rapidly, with developments like [OP Stack](https://docs.optimism.io/)
, [Polygon CDK](https://polygon.technology/polygon-cdk)
, and [Arbitrum Orbit](https://arbitrum.io/orbit)
 offering customizable rollups. Despite this proliferation, much activity remains on Ethereum’s mainnet, suggesting a complex interplay between L1 and L2 solutions.

### Stablecoins and Regulation

The document outlines a complex and evolving regulatory landscape for stablecoins. In the U.S., there is no unified federal regulatory framework; instead, a mixture of state and federal regulations applies to different aspects of stablecoin issuance and use. The lack of comprehensive regulation creates uncertainty, hindering stablecoin market growth and technological innovation. Notably, the “[Clarity for Payment Stablecoins Act of 2023](https://crsreports.congress.gov/product/pdf/IN/IN12249)
” received bipartisan support, indicating a move towards more structured regulation.

Internationally, the European Union’s [Markets in Crypto-Assets (MiCA) regulation](https://www.coindesk.com/learn/mica-eus-comprehensive-new-crypto-regulation-explained/)
 stands out as one of the most comprehensive legal frameworks for crypto assets, including stablecoins. It sets stringent requirements for stablecoin issuers and aims to ensure the stability and security of digital assets across the EU. Other regions like the UAE, Singapore, and Hong Kong are also making significant strides in creating more broadly conducive regulatory environments for stablecoins and crypto assets.

### Global Macro Trends and Crypto Adoption

While the USD remains dominant, a shifting global monetary regime might favor digital assets like Bitcoin. The document suggests Bitcoin and other digital stores of value could play a significant role in an emerging multipolar world order.

A softer US economy and potential Federal Reserve rate cuts in the first half of 2024 could present a favorable macro backdrop for cryptocurrencies.

### Institutional and Retail Participation

Approximately 59% of participants in a recent Institutional Investor survey expect their firm’s allocations to the digital asset class to increase over the next three years. However, there is notable regulatory uncertainty, especially in the US, which is perceived to threaten the country’s leadership in financial services. Despite this, the regulatory foundation is expected to continue building, leading to more clarity and greater institutional participation in the future​​.

#### a16z crypto Outlook

Original report: [https://a16zcrypto.com/posts/article/a-few-of-the-things-were-excited-about-in-crypto-2024/](https://a16zcrypto.com/posts/article/a-few-of-the-things-were-excited-about-in-crypto-2024/)

### Why Decentralization Matters

Decentralization matters because it allows for credibly neutral, open infrastructure, promotes competition and diversity, and gives users more choice and ownership. However, achieving true decentralization at scale has been difficult when competing against efficient centralized systems. Web3 and DAOs have provided a “living laboratory” to develop best practices, including decentralization models that support richer features and governance approaches that hold leadership accountable. As these models evolve, we should see unprecedented decentralized coordination, functionality, and innovation levels.

### The Importance of Open Source

Open-source modular tech stacks are important because they:

*   Unlock permissionless innovation and allow participants to specialize, leading to more competition. The biggest advantage of an open-source, modular tech stack is that it incentivizes more competition.
    
*   Strengthen network effects rather than fragment them. Modularity that extends and strengthens network effects makes sense, especially for open-source projects.
    
*   Allow for deep integration and optimization across modular boundaries, leading to greater performance initially. However, the open-source modular approach has led to more innovation over time.
    
*   Decentralize and democratize systems rather than concentrate power and control with a few entities. Open source helps counter the issues that arise when control of a powerful system or platform is in the hands of a few.
    

### Advancements in Verification and Security

The growing importance of [formal verification](https://en.wikipedia.org/wiki/Formal_verification)
 in software, particularly in the context of smart contracts, reflects the increasing need for reliable and secure digital systems. New tools and methodologies make formal verification more accessible and effective, promising higher security and robustness in software development.

[SNARKs](https://en.wikipedia.org/wiki/Zero-knowledge_proof)
 represent a significant advancement in verifying computational workloads, offering a more scalable and practical solution for various applications. From edge devices to content authenticity and IRS forms, SNARKs are set to become a mainstream tool for ensuring the integrity and reliability of computational processes.

* * *

Appendix
--------

#### About the Source Companies

[Messari](https://messari.io/)
 is a prominent platform, offering extensive crypto research, data, and analytics tools.

[VanEck](https://www.vaneck.com/us/en/)
 is a global investment firm with a focus on providing investors access to opportunities across emerging industries, asset classes, and markets with a variety of investment vehicles, including ETFs, mutual funds, and institutional funds.

[Pantera Capital](https://panteracapital.com/)
 was the first U.S. institutional asset manager focused exclusively on blockchain technology.

[Coinbase Institutional](https://www.coinbase.com/home)
 positions itself as a trusted bridge to crypto markets for institutions, offering a comprehensive platform for various investment stages, including trade, custody, and participation.

[a16z crypto](https://a16zcrypto.com/)
 is a specialized branch of the larger a16z venture capital firm, specifically tailored to invest in the cryptocurrency and blockchain space.

[Odos](https://www.odos.xyz/)
 is a leading DeFi aggregator and intent solver who recently celebrated 1M+ distinct users and $16B+ in transaction volume. Odos’ proprietary optimization algorithm finds users the most efficient routes for their everyday DeFi actions, delivering them unmatched value.

Disclaimer: This document, provided by Semiotic AI, Inc., is for informational purposes only and does not constitute financial, legal, or investment advice. All content is provided “as is” with no guarantee of completeness, accuracy, or timeliness. Investing in digital assets and securities involves significant risk, including potential total loss of principal. Semiotic AI, Inc. expressly disclaims any liability for any loss or damage arising from the use of this document or its contents. Readers are advised to seek independent professional advice before making any financial decisions. Past performance is not indicative of future results. All trademarks and logos referenced herein belong to their respective owners. This disclaimer is subject to change without notice.

---

# Homomorphic Signatures for Payment Channels

![Gradient background](/_astro/curved_bg_desktop.ChjV8nJN_1c0N0M.svg) ![Gradient background](/_astro/curved_bg_mobile.DP8czMRd_1b6C73.svg)

_This post was co-authored by [Pedro Henrique Bufulin de Almeida](https://www.linkedin.com/in/pedro-henrique-bufulin-de-almeida-6a7375160/?utm_source=share&utm_campaign=share_via&utm_content=profile&utm_medium=android_app)
 and [Severiano Sisneros](https://x.com/SeveSisneros)
._

Introduction
============

You’re a coffee shop owner looking to minimize your costs of doing business. You notice that credit card transaction fees are inflating the price you charge to your customers. So you have an idea: rather than paying with credit card for each transaction, you let your customers pay with an IOU. At the end of the month you tally the customer’s IOUs and ask the customer to pay the total amount using a single credit card transaction. But, what happens if you have a customer who runs up a large tally and then never comes back? It would be great if you could prove that all the IOUs came from the untrustworthy customer and also prove that the value of sum of all the IOUs, allowing you to charge the customer’s credit card without their involvement.

Cryptographers solved the first half of the problem in the early days of public-key cryptography by inventing digital signatures. In this post we’re going to talk about how cryptographers solved the full problem using something called a homomorphic signature scheme.

You may have heard the word “homomorphic” before, most likely in the context of homomorphic encryption. Essentially all this means is that if you have two values that are the output of some function and add them together, then the result is the same as first adding the two inputs together and then calculating the function on the sum; if X = H(x) and Y = H(y) then X + Y = H(x + y). For homomorphic signatures, this function is a digital signature.

Back to our coffee shop example. You now know about homomorphic signatures, so you ask all of your customers to digitally sign their IOUs with a homomorphic signature scheme. Now when the wily customer decides to not show up, you can simply add up all of their signed IOUs. The result is a single IOU with a valid signature for the total amount owed by the customer. You can send this IOU and the signature to the customer’s bank, they can verify that the signature belongs to them, that the signature is valid, and the bank can charge the customers account for the total amount you are owed, all without requiring any interaction from the customer. Great!

This solution is particularly interesting in blockchain applications, where transaction fees are particularly high. In this case, the “bank” is a smart contract. You can submit a single transaction which verifies the homomorphic signature and transfers the amount owed from the customer’s account to you. This is way cheaper than having to submit and verify all the individual signatures and enables interesting protocols like micropayment channels.

This post discusses design and optimization of verifiable micropayments over a state channel, a feature of Semiotic Labs’ [GraphTally](https://github.com/semiotic-ai/timeline-aggregation-protocol)
 library for The Graph. The Graph is a decentralized data services protocol, allowing customers to index and access all blockchain data.

Micropayments Using State Channels
==================================

An Ethereum-based state channel lets users conduct numerous transactions off-chain without the cost and potential delay of executing and finalizing each transaction on the EVM. In the case where transactions are payment transfers (a payment channel), the cost associated with fees can be greatly reduced, especially when the value of each payment is low (a so-called micropayment).

In the basic model of a [state channel](https://ethereum.org/en/developers/docs/scaling/state-channels/)
 the only on-chain transactions required are smart contract operations to open and close the channel. Participants in the state channel conduct a series of transactions off-chain and agree upon state changes by collectively signing them. When the channel closes, payments are distributed and the state change is finalized on the Ethereum mainnet. Users can resolve a disputed state update by finalizing the last transaction which was signed by all participants.

We can authenticate and verify the integrity of transactions with digital signatures. Semiotic Labs’ GraphTally (formerly known as Timeline Aggregation Protocol - TAP) provides a library for verifying ECDSA signatures on transactions in a payment channel. GraphTally is a tool for a single party sending micropayments to one or more receivers. The receipt for each micropayment is signed, then verified by the TAP Aggregator. The Aggregator adds together the values of all payments and creates a new signed receipt reflecting the combined value. The final receipt can be verified on-chain, where the change of state will be applied.

The process carried out by GraphTally requires validating each signature individually. We can make this more efficient by aggregating the signatures and performing a single verification that applies to a combination of the receipts. BLS signatures are currently used on the Beacon Chain for a similar purpose, aggregating validator signatures on Merkle trie roots. In this context, multiple public keys are used to sign a single message, then the signatures are added together and verified. We would like to have a sender sign multiple messages (e.g., receipts) and verify the signature aggregate in one step. BLS can do this, but the signature aggregate is not a valid signature on a combination of messages. The receiver must compute a hash of each message to verify the signature aggregate.

Instead, we look to homomorphic signature schemes (HSS), where the aggregated signature is a valid signature on the sum of messages. Semiotic Labs has developed a Rust implementation, [h2s2](https://github.com/semiotic-ai/h2s2)
, of the protocol outlined below. We will see that when we can perform some offline precomputation, the work of verifying an aggregated signature is constant.

Homomorphic Signatures for Cheap and Trustless Micropayments
============================================================

The network coding scheme NCS1\\text{NCS}\_1NCS1​ of Boneh et. al. is an HSS consisting of four algorithms **Setup**, **Sign**, **Combine**, and **Verify**. We refer the reader to [the paper](https://eprint.iacr.org/2008/316)
 for the details of the protocol, including security proofs, and here just go through the algorithms applied to our context.

The parties involved when using a signature scheme are the sender (signing messages) and the receiver (verifying signatures). For example, the sender might query an Indexer and sign a receipt for a payment amount. The Indexer (receiver) serving the query obtains the receipt and redeems the payment amount on-chain. The verifier (Indexer) verifies each signed receipt it receives from the sender and executes the **Combine** algorithm to aggregate the signatures. There is a smart contract on the mainnet which verifies the aggregated signature in order to finalize the accumulated microtransactions. We are concerned with the costs of verifying this signature on-chain.

Aggregation
-----------

The main part of the scheme we discuss here is the aggregation and verification of signatures. The **Combine** algorithm aggregates multiple message signatures into one, which is valid for a combination of the respective messages:

magg\=∑i\=1Nmim\_{\\text{agg}} = \\sum\_{i=1}^{N} m\_imagg​\=i\=1∑N​mi​ σagg\=∑i\=1Nσi,\\sigma\_{\\text{agg}} = \\sum\_{i=1}^{N} \\sigma\_i,σagg​\=i\=1∑N​σi​,

where m1,…,mNm\_1,…,m\_Nm1​,…,mN​ are the individual messages and σ1,…,σN\\sigma\_1,…,\\sigma\_Nσ1​,…,σN​ are the signatures on the respective messages.

In the NCS1\\text{NCS}\_1NCS1​ scheme, each signature has a weight in the base field which is included as a coefficient in the sum (similar for the combined messages). Here, we assume all weights are equal to 1.

### Verification of Combination

The smart contract verifies the aggregated signature by checking:

e(σagg,Q)\=e(∑i\=1NH(id,i)+magg⋅P,R)e(\\sigma\_{agg}, Q) = e\\left(\\sum\_{i=1}^{N} H(id,i) + m\_{agg} \\cdot P, R \\right)e(σagg​,Q)\=e(i\=1∑N​H(id,i)+magg​⋅P,R)

In the following paragraph, we offer some notes on the parameters involved. The chief operation to notice in the equation above is the sum of hash values, ∑i\=1NH(id,i)\\sum\_{i=1}^{N} H(id,i)∑i\=1N​H(id,i). This can be a costly operation to perform at verification time, but this cost is eliminated if the calculation is done off-chain.

We use the pairing-friendly curve BN254 over scalar field Fp\\mathbb{F}\_pFp​, with associated groups G1,G2,GT\\mathbb{G}\_1, \\mathbb{G}\_2, \\mathbb{G}\_TG1​,G2​,GT​ and type three bilinear pairing e:G1×G2→GTe:\\mathbb{G}\_1 \\times \\mathbb{G}\_2 \\rightarrow \\mathbb{G}\_Te:G1​×G2​→GT​, as laid out in the Ethereum precompile contracts [EIP-196](https://eips.ethereum.org/EIPS/eip-196)
 and [EIP-197](https://eips.ethereum.org/EIPS/eip-197)
. H:{0,1}∗→G1H:\\{0,1\\}^\* \\rightarrow \\mathbb{G}\_1H:{0,1}∗→G1​ is a hash-to-curve function to map identifiers and tags to group elements. The sender is associated with an unpredictable identifier id∈Zp∗id \\in \\mathbb{Z}\_p^\*id∈Zp∗​ which stays consistent for the duration of the payment channel. The identifier is concatenated with an index iii (we use an unsigned integer type) indicating the receipt being signed. This ensures that H(id,i)H(id,i)H(id,i) is distinct for each micropayment. PPP and QQQ are generators of G1\\mathbb{G}\_1G1​ and G2\\mathbb{G}\_2G2​, respectively, and RRR is the public key used for signing the messages.

### Cost of Verification

If the values PPP, QQQ and RRR are initialized as part of the smart contract, then the verification of the combination requires:

*   A scalar multiplication in G1\\mathbb{G}\_1G1​, which costs 40,000 gas.
*   A pairing equality check, which costs 260,000 gas.

The gas estimates above are based on the figures in EIP-196 and 197.

Verification also requires computation of the sum ∑i\=1NH(id,i)\\sum\_{i=1}^{N} H(id,i)∑i\=1N​H(id,i). This is NNN point additions, as well as the computation of the hash-to-curve function itself in each instance. When verifying signatures on thousands of micropayments, this will dominate the cost of verification.

To avoid doing this computation on the EVM, the verification smart contract can be initialized with a precomputed value Hagg\=∑i\=1NH(id,i)H\_{\\text{agg}} = \\sum\_{i=1}^{N} H(id,i)Hagg​\=∑i\=1N​H(id,i) based on the expected number of receipts NNN.

A Multi-Key Protocol
--------------------

What if we want to aggregate signatures on microtransactions from multiple senders? [Aranha and Pagnin](https://eprint.iacr.org/2019/830)
 present an adaptation of NCS1\\text{NCS}\_1NCS1​ to use multiple public-secret key pairs and identifiers. We will not walk through the details of the extension here, but note the operations involved in verification of the aggregated signature. For ttt distinct senders (identifiers) signing accumulated micropayments:

*   ttt scalar multiplications in G1\\mathbb{G}\_1G1​, which cost 40,000 gas each.
*   ttt point additions in G1\\mathbb{G}\_1G1​, which cost 500 gas each.
*   ttt group multiplications in GT\\mathbb{G}\_TGT​. Operations in GT\\mathbb{G}\_TGT​ tend to be expensive due to the size of elements involved.
*   ttt pairing computations, which again may be costly.
*   A pairing equality check, which costs 260,000 gas.

Additionally, we need ttt computations ∑i∈IH(id,i)\\sum\_{i \\in I} H(id,i)∑i∈I​H(id,i), one for each distinct identifier, where the sum runs over the aggregated receipts for a given ididid. There is also a sorting that the verifier needs to perform in order to group the incoming (ididid, index) pairs by identifier.

As noted above, the on-chain verification smart contract may avoid computing a sum of hash-to-curve values by accepting precomputed values Hagg\=∑i\=1NH(id,i)H\_{\\text{agg}} = \\sum\_{i=1}^{N} H(id,i)Hagg​\=∑i\=1N​H(id,i), which depend on the ididid and the expected number of receipts NNN associated with that identifier.

Benchmarks
==========

The following benchmarks were conducted on an M2 MacBook Air using our Rust implementation [h2s2](https://github.com/semiotic-ai/h2s2)
. We are showing the time taken to sign a single message, verify a single message, aggregate a batch of messages (in this case 32), and verify the aggregated signature. There are a few things we can point out about these benchmarks. Signing individual messages is fast, taking about 82us per message. Verifying a single message takes a little longer, about 1.2ms. Aggregating a batch of signatures is fast, taking about 57us to aggregate 32 signatures. The most important thing we notice, is that verifying an aggregate signature takes about the same time as verifying a single signature, i.e. the verification time does not depend on the nunmber of signatures used to compute the aggregate. This is because we are precomputing the sum of the hash-to-curve values, as mentioned in the previous section. For blockchain applications, this means we can batch verify any number of signatures and the on-chain costs will be independent of the number of signatures in the batch.

![](/_astro/sign_bench.Br_IDm3q_iJoUv.svg) _82us to sign a single message_

![](/_astro/verify_single_bench.CZzsNiYG_Z2ojMD7.svg) _1.2ms to verify a single message_

![](/_astro/evaluate_bench.ke9ieR_4_2boolJ.svg) _58us to combine 32 signatures_

![](/_astro/verify_aggregate_bench.jAmLJQfu_ZqJ8Ow.svg) _1.1ms to verify an a batch of 32 signatures_

Conclusion
==========

By leveraging homomorphic signature schemes, we can significantly reduce the computational costs associated with verifying aggregated micropayments, while maintaining the security and integrity of off-chain transactions. Semiotic Labs’ work with GraphTally and the h2s2 implementation demonstrates the potential to enhance efficiency without compromising trust. The innovations discussed here provide a robust framework for scaling Ethereum-based micropayments, making them a practical option for a wide range of use cases.

We hope to have given you a high-level understanding of how homomorphic signatures work and how they can be used to solve a practical problem for blockchain applications. If you want more details, we also a have a more technical whitepaper [here](https://drive.google.com/file/d/1ufBeXJLHW0hSrSN9Jx0HnIncfWL-YDQc/view?usp=drive_link)
.

If any of this sounds interesting to you or if you’ve encountered similar problems, please reach out!

---

# Semiotic Labs | Articles

![Gradient background](/_astro/curved_bg_desktop.ChjV8nJN_1c0N0M.svg) ![Gradient background](/_astro/curved_bg_mobile.DP8czMRd_1b6C73.svg)

*   [*   cryptography\
    \
    ##### Homomorphic Signatures for Payment Channels\
    \
    This post discusses design and optimization of verifiable micropayments over a state channel, a feature of Semiotic Labs’ \[GraphTally\](https://github.com/semiotic-ai/timeline-aggregation-protocol) library for The Graph. The Graph is a decentralized data services protocol, allowing customers to index and access all blockchain data.\_\
    \
    * * *](/articles/h2s2-payment-channels/)
    
*   [*   ai\
    *   cryptography\
    *   defi\
    \
    ##### 2024 Crypto Meta-Analysis\
    \
    A meta-summary of various 2024 crypto market outlooks.\
    \
    * * *](/articles/2024-crypto-market-outlooks/)
    
*   [*   cryptography\
    \
    ##### PSI with FHE\
    \
    Semiotic Labs explores the integration of Private Set Intersection (PSI) in blockchain to tackle Maximal Extractable Value (MEV) issues. Utilizing fully homomorphic encryption, the protocol aims for trustless operations and enhanced performance in tasks like access list comparison and private auctions.\
    \
    * * *](/articles/psi/)
    
*   [*   ai\
    \
    ##### Indexer Allocation Optimization: Part II\
    \
    This article was co-authored with Howard Heaton, from Edge & Node, and with Hope Yen, from GraphOps.\
    \
    * * *](/articles/indexer-allocation-optimization-part-ii/)
    
*   [*   ai\
    *   defi\
    \
    ##### An Overview of Automatic Market Maker Mechanisms\
    \
    This article explores the principles and mechanisms behind the many popular Automatic Market Maker designs currently used in production. While the mathematical details of these designs are fascinating in their own right, this article seeks to instead focus on graphical representations and high-level concepts, allowing for a more approachable and exhaustive exploration of the space.\_ \_Watch our related Devcon 2022 talk here: \[https://www.youtube.com/watch?v=KxuyHfmLHP0\](https://www.youtube.com/watch?v=KxuyHfmLHP0).\
    \
    * * *](/articles/automatic-market-makers-amm/)
    
*   [*   ai\
    *   blockchain infrastructure\
    \
    ##### Automated Query Pricing in The Graph\
    \
    Indexers in The Graph have control over the pricing of the GraphQL queries they serve based on their shape. For this task, The Graph created a domain-specific language called Agora​ that maps query shapes to prices in GRT. However, manually populating and updating Agora models for each subgraph is a tedious task, and as a consequence, most indexers default to a static, flat pricing model.\_ \_To help indexers with pricing in the relative resource cost of serving different query shapes, as well as following the query market price, we are developing AutoAgora, an automation tool that automatically creates and updates Agora models.\_ \_See our related Devcon 2022 talk here:\_ \_\[https://www.youtube.com/watch?v=LRl9uFfmjEs\](https://www.youtube.com/watch?v=LRl9uFfmjEs)\_.\
    \
    * * *](/articles/automated-query-pricing-on-the-graph/)
    
*   [*   ai\
    \
    ##### Indexer Allocation Optimization: Part I\
    \
    Indexers within The Graph Protocol are rewarded via an indexing reward. How can indexers optimise their allocations so as to maximise the reward they receive? In this blog post we formalise the problem in terms of a reward function and use convex optimization to find a solution.\
    \
    * * *](/articles/indexer-allocation-optimisation/)
    
*   [*   cryptography\
    \
    ##### Introduction to the Sum-Check Protocol\
    \
    It is expensive to run transactions on the Ethereum EVM. Verifiable computing (VC) lets us outsource computing away from the EVM. Today, a popular and exciting form of VC algorithm is the SNARK. There are various families of SNARKs that use the sum-check protocol, which is a simple algorithm to introduce VC. This is a tutorial on the sum-check protocol. This post is focused on how the sum-check protocol is implemented - it does not go into theory. You can skip straight to the finished code \[here\](https://github.com/0xsamgreen/sumcheck).\_Thank you to Gokay Saldamli, Gabriel Soule, and Tomasz Kornuta for providing valuable feedback on this article.\_\
    \
    * * *](/articles/sumcheck-tutorial/)

---

# PSI with FHE

![Gradient background](/_astro/curved_bg_desktop.ChjV8nJN_1c0N0M.svg) ![Gradient background](/_astro/curved_bg_mobile.DP8czMRd_1b6C73.svg)

_This work was done by [Gokay Saldamli](https://www.linkedin.com/in/gokay-saldamli-b338152/)
 and [Lisandro (Lichu) Acuña](https://www.linkedin.com/in/lichuacuna/)
 at Semiotic Labs. Special thanks to [Jonathan Passerat-Palmbach](https://www.linkedin.com/in/jopasserat)
 and the broader [Flashbots](https://www.flashbots.net)
 team for their collaboration and comments on the project._

Private Set Intersection with Fully Homomorphic Encryption
----------------------------------------------------------

Private Set Intersection (PSI) is a cryptographic technique that plays a crucial role in safeguarding data privacy. It enables two parties to identify common elements in their sets without leaking any information about the rest of the elements. This can be used, for example, in secure contact tracing during epidemics or in privacy-preserving human genome testing.

Over the last year, Semiotic Labs has been studying the possible applications of this powerful cryptographic technique to blockchain infrastructure problems, particularly in the context of Maximal Extractable Value (MEV). Through discussions with the Flashbots team, we arrived at the conclusion that an efficient PSI protocol could add a lot of value to the field, for example in access list comparison ([EIP-2930](https://eips.ethereum.org/EIPS/eip-2930)
) or as a primitive to enable private auctions. This is why we decided to build such a PSI protocol, and in this article we will detail how it works, covering everything from the math to the code.

### The Math

If you stop for a few minutes to try to devise a PSI protocol, the first idea that will come to mind will probably have to do with hash functions. It is a natural idea to have Alice hash all the items in her set and send these hashes to Bob, who will then hash the elements in his set and compare them to Alice’s hashed elements. This scheme works and is extremely fast, but unfortunately, it is also extremely insecure, as it can leak Alice’s inputs if the input space is small - Bob could simply hash all the elements in the input space and compare them to Alice’s hashed elements.

Nowadays, most existing and functional PSI implementations rely on an external server (they are “server-aided protocols”). In this case, Alice and Bob learn nothing about each other’s set except the shared elements, and nothing is visible to the server as long as there are no collusions. However, when the server colludes with one of the actors (i.e., they “cooperate”), the protocol becomes completely insecure and the server and its colluding party can read all elements of the other party’s set.

In the PSI protocol we constructed, we use Fully Homomorphic Encryption (FHE) to eliminate reliance on an external server. As you may know, FHE is a family of cryptographic techniques that enable computing directly on encrypted data, preserving data privacy throughout processing. FHE was the original focus of Semiotic when the team started four years ago, and we spent a lot of time working on it. Using FHE to perform PSI in a completely trustless manner is a nod to the past for the team.

Our solution is based on Chang et al.’s [“Fast Private Set Intersection from Homomorphic Encryption”](https://dl.acm.org/doi/10.1145/3133956.3134061)
. Extracted directly from their paper, this is how their PSI protocol works:

0.  Context: Alice has a set A\=a1,…,aNA\\mathcal{A} = {a\_1, …, a\_{N\_A}}A\=a1​,…,aNA​​ and Bob has a set B\=b1,…,bNB\\mathcal{B} = {b\_1, …, b\_{N\_B}}B\=b1​,…,bNB​​.
1.  Setup: Alice and Bob jointly agree on an FHE scheme, and Alice generates a public-secret key pair for it, sending the public key to Bob and keeping the secret key for herself.
2.  Set encryption: Alice encrypts each element aia\_iai​ in her set A\\mathcal{A}A using the FHE scheme, and sends the NAN\_ANA​ ciphertexts c1,…,cNAc\_1, …, c\_{N\_A}c1​,…,cNA​​ to Bob.
3.  Computing intersection: For each cic\_ici​, Bob samples a random non-zero element rir\_iri​ and homomorphically computes: di\=ri∏j\=1NB(ci−bj)d\_i = r\_i \\prod\_{j=1}^{N\_B} (c\_i - b\_j)di​\=ri​∏j\=1NB​​(ci​−bj​) After doing this for all cic\_ici​, Bob sends the ciphertexts d1,…,dNAd\_1, …, d\_{N\_A}d1​,…,dNA​​ to Alice.
4.  Reply extraction: Alice decrypts the ciphertexts d1,…,dNAd\_1, …, d\_{N\_A}d1​,…,dNA​​.

Understanding these steps may take some time, so feel free to stop and go through them again. Steps 1 and 2 are pretty much just the problem statement and setup. In step 3, Alice encrypts her own set of elements, which Bob will not be able to decrypt as he doesn’t have Alice’s secret key. Step 3 is the key step, and where we actually make use of FHE: the product ∏j\=1NB(ci−bj)\\prod\_{j=1}^{N\_B} (c\_i - b\_j)∏j\=1NB​​(ci​−bj​) is the encryption under Alice’s secret key of the product ∏j\=1NB(ai−bj)\\prod\_{j=1}^{N\_B} (a\_i - b\_j)∏j\=1NB​​(ai​−bj​), as cic\_ici​ is the encryption of aia\_iai​ and we are running this computation in FHE. Observe that ∏j\=1NB(ai−bj)\\prod\_{j=1}^{N\_B} (a\_i - b\_j)∏j\=1NB​​(ai​−bj​) will be 0 if and only if one of b1,…,bNBb\_1, …, b\_{N\_B}b1​,…,bNB​​ (Bob’s set elements) equals aia\_iai​, and therefore ∏j\=1NB(ci−bj)\\prod\_{j=1}^{N\_B}(c\_i - b\_j)∏j\=1NB​​(ci​−bj​) will be the encryption of 0 precisely when this happens, same as ri∏j\=1NB(ci−bj)r\_i \\prod\_{j=1}^{N\_B}(c\_i - b\_j)ri​∏j\=1NB​​(ci​−bj​). If this is not the case, i.e. no bjb\_jbj​ equals aia\_iai​, the term rir\_iri​ will act as a “randomizer”, making the final product look basically random. Observe that Bob, who performed the computation, doesn’t know if the product ended up being 0 or not, as did\_idi​ is the _encrypted_ result of the computation. Only Alice can decrypt this result, which is exactly what she does in step 4. It’s worth noticing that, if both Alice and Bob want to get the intersection, they have to run the protocol twice, interchanging roles.

Once you understand these steps, it is quite intuitive that this PSI protocol works, and its privacy comes from doing all the computation on encrypted data. However, there is a small problem. When running computation over FHE, there is a “noise” that increases with each operation and almost doubles with multiplication. This noise cannot exceed a certain threshold, or else the outputs will be erroneous. In this sense, the multiplicative depth of a circuit is defined as the number of sequential homomorphic multiplications that are computed in our FHE circuit, a parameter that cannot be too high, or else the noise will be too high.

As you can see, the circuit depth in Chang et al.’s protocol is NBN\_BNB​, the size of Bob’s set. This is not ideal, as it makes it impossible to run the protocol for relatively large sets. Thus, we modified the protocol and added an optimization that keeps the circuit depth fixed regardless of the size of the sets and therefore allows running PSI on sets of any size. Hooray!

This is not as complex as it seems. We simply define a parameter NNN that represents the maximum circuit depth that our FHE scheme can tolerate and split Bob’s set of size NBN\_BNB​ into ⌈NBN⌉\\lceil \\frac{N\_B}{N} \\rceil⌈NNB​​⌉ subsets of size at most NNN each. Next, Alice runs the original protocol with each of Bob’s subsets separately, having a circuit of depth at most NNN each time.

This leads to an incredibly fast and scalable protocol, with runtime growing linearly on the size of Bob’s set. Moreover, the separate private intersections with each subset of Bob’s set can be run in parallel, leading to an even faster protocol.

### The Code

We implemented the protocol using _[node-seal](https://github.com/morfix-io/node-seal/)
_, a wrapper library of _[Microsoft SEAL](https://github.com/microsoft/SEAL)
_, the most widely used FHE library both in the academia and in the industry. We will walk through the important pieces of this implementation here, but you can also access our _[public GitHub repository (http://github.com/LichuAcu/psi-demo)](https://github.com/LichuAcu/psi-demo/)
_ where you can read the whole code and try the protocol yourself using a visual interface.

Before diving into the actual code, it is worth noting that this PSI protocol is agnostic to the FHE scheme being used, as long as the mathematical operations we need to perform work correctly. In our implementation, we use the standard BFV scheme, which, as mentioned in the Microsoft SEAL documentation, “allow(s) modular arithmetic to be performed on encrypted integers”. There are other schemes, such as CKKS, which also allow operations with real or complex numbers, but at the cost of giving only approximate results. Since we do not need this, we stick to the exact BFV. The other parameters that had to be set are the security level, for which we chose 128 bits, and the polynomial modulus, which we set to 8192. Since the intricate inner workings of homomorphic encryption are not the focus of this article, we will not delve into what exactly each of these parameters are, but, if you are interested, you can find that information in this [example SEAL code](https://github.com/microsoft/SEAL/blob/main/native/examples/1_bfv_basics.cpp)
. The main takeaway with respect to these parameters is that, by adjusting them, the protocol can support _deeper circuits_, which translates into larger subsets, i.e. larger NNN, so that the protocol needs to be executed fewer times (lceilfracNBNrceillceil \\\\frac{N\_B}{N} \\\\rceillceilfracNB​Nrceil becomes smaller). However, deeper circuits imply longer execution time, so tuning these parameters to find the optimal tradeoff is an important step. It is worth keeping in mind that, since the multiple executions of the protocol can be run in parallel, it is not a big problem if NNN is small (in our case, for example, it is only 3).

Now we are ready to take a look at the code!

The core logic of the protocol is implemented in the _[logic.tsx](https://github.com/LichuAcu/psi-demo/blob/master/client/src/logic.tsx)
_ file. Each step in our PSI description is implemented as a separate function. For Step 1, for example, we see in the `setup` function how the public and secret keys are created:

    const keyGenerator = seal.KeyGenerator(context);
    const publicKey = keyGenerator.createPublicKey();
    const secretKey = keyGenerator.secretKey();
    

For Step 2, Alice’s set is encrypted in the `alice_encrypt_locations` functions:

    const set_ciphertexts_alice = encryptor.encrypt(
      set_plaintexts_alice
    ) as CipherText;
    

Step 3 is arguably the most interesting one! In the `bob_homomorphic_operations` function, we first split Bob set into subsets:

    const sets_plaintexts_bob = [];
    for (let i = 0; i < locations_bob.length; i += batch_size) {
      const batch = locations_bob.slice(i, i + batch_size);
      sets_plaintexts_bob.push(Int32Array.from(batch));
    }
    

Next, each subset is intersected with Alice’s set, following the logic specified in Step 3:

    sets_plaintexts_bob.forEach((set_plaintexts_bob) => {
      const final_product = this.seal.CipherText();
    
      // Homomorphically initialize result to first Alice's element - first Bob's element
      const first_element_bob = Int32Array.from(
        Array(set_alice_length).fill(set_plaintexts_bob[0])
      );
      const first_element_bob_encoded = this.encoder.encode(
        first_element_bob
      ) as PlainText;
      this.evaluator.subPlain(
        set_ciphertexts_alice,
        first_element_bob_encoded,
        final_product
      );
    
      for (let i = 1; i < set_plaintexts_bob.length; i++) {
        const ith_element_bob = Int32Array.from(
          Array(set_alice_length).fill(set_plaintexts_bob[i])
        );
        const ith_element_bob_encoded = this.encoder.encode(
          ith_element_bob
        ) as PlainText;
        const temp = this.seal.CipherText();
        this.evaluator.subPlain(
          set_ciphertexts_alice,
          ith_element_bob_encoded,
          temp
        );
        this.evaluator.multiply(final_product, temp, final_product);
      }
    
      let random_plaintext = new Int32Array(set_alice_length);
      let random_plaintext_encoded: PlainText;
      getRandomValues(random_plaintext);
      random_plaintext_encoded = this.encoder.encode(random_plaintext) as PlainText;
      this.evaluator.multiplyPlain(
        final_product,
        random_plaintext_encoded,
        final_product
      );
    
      const final_product_string = final_product.save();
      counter++;
      final_products.push(final_product_string);
    });
    
    return final_products;
    

The `evaluator` object in the above code is in charge of homomorphically performing our mathematical operations. Its methods take the operands of our operation as its first two inputs, and the third input is the variable in which the result will be stored. Observe how we sometimes call the `multiply` method while other times we call the `multiplyPlain` one; this is because in the former we are multiplying two encrypted values together, while in the latter we are multiplying an encrypted value by a plain value.

Finally, Step 4 is fairly simple:

    for (const final_product of final_products) {
      let final_product_ciphertext: CipherText = this.seal.CipherText();
      final_product_ciphertext.load(this.context, final_product);
      const decrypted = this.decryptor.decrypt(
        final_product_ciphertext
      ) as PlainText;
      const decoded = this.encoder.decode(decrypted);
      for (let i = 0; i < set_alice_length; i++) {
        if (decoded[i] == 0) {
          // The i-th element is in the intersection!
        }
      }
      counter++;
    }
    

### Benchmarks

Now that we know why and how our protocol works, let’s see it in action! Since network connectivity and delay may vary depending on the context in which the protocol is used, we ran our benchmarks performing Alice’s and Bob’s calculations locally, thus ignoring the communication time. To estimate it, we should consider the N\_AN\\\_AN\_A ciphertexts that Alice sends in Step 2 and the NAN\_ANA​ ciphertexts that Bob sends in Step 3.

All these benchmarks were run in a 64 GB M2 Max MacBook Pro. Our first set of benchmarks shows something that can be easily intuited from the protocol design, but is worth looking at in numbers: that the size of the intersection does not affect the execution time. We can check this by keeping all other parameters fixed, changing the size of the intersection and observing that the execution time remains basically the same:

| Alice’s set size | Bob’s set size | Intersection size | Running time (ms) |
| --- | --- | --- | --- |
| 100 | 100 | 13  | 3884 |
| 100 | 100 | 25  | 3811 |
| 100 | 100 | 50  | 3846 |
| 100 | 100 | 100 | 3834 |

Thus, we will not take this parameter into account in the rest of our benchmarks.

Let’s now see what happens when the size of Bob’s set increases. Since Bob’s set will be divided into a number of subsets directly proportional to the size of his original set, it would make sense to expect the runtime to grow linearly as the size of Bob’s set grows. That’s exactly what happens! Take a look at it:

| Alice’s set size | Bob’s set size | Running time (ms) |
| --- | --- | --- |
| 100 | 50  | 1919 |
| 100 | 100 | 3811 |
| 100 | 200 | 7646 |
| 100 | 400 | 15286 |
| 100 | 800 | 30496 |

Finally, we run the same experiments with different sizes for Alice’s set. It turns out that because of the way Microsoft SEAL is implemented in an array-first way, increasing the size of the Alice set (up to an upper-bound) does not increase the execution time. You can see this in the following benchmarks chart, where running time only changes when Bob’s set size does:

![](/_astro/plots-1-1024x768.kpRDFC5h_1vNqsW.webp)

Note that the graph looks exponential because the X-axis is in logarithmic scale, which is actually a visual indication of the underlying linear relationship between running time and Bob’s set size indicated above.

### Use cases

There are multiple scenarios where this efficient PSI protocol could be useful, especially in privacy-preserving peer-to-peer networks. For example, EIP-2930 proposes adding optional access lists to the Ethereum protocol, which are nothing more than “a list of addresses and storage keys that the transaction plans to access.” Using an efficient PSI protocol, these access lists can be made private while allowing block constructors to ensure that no two transactions on the same block access the same slot on the blockchain. One application of this would be private auctions among MEV extractors for a specific slot on the blockchain, without the need to disclose what their intended transactions actually do. Similarly, the validity of transactions could be privately proven (possibly using zero-knowledge proofs) with respect to the final state of the newest block in the chain, and if no two transactions in a new block access the same slot in the chain (i.e., the intersection of access lists is pairwise null), then that new block is proven valid. This could enable private block construction without the need to sequentially prove the validity of transactions, which would be particularly difficult to do in a privacy-preserving manner.

Another use case for PSI protocols within blockchains has recently been suggested by the Aztec Protocol team in their RFP for a Note Discovery Protocol. Aztec is a fully private layer 2 on Ethereum, and the private data it contains uses a UTXOs model, like Bitcoin. In order for an Aztec user to consume a UTXO that belongs to them, “a mechanism needs to be in place to allow the note to be discovered.” Simply put, if there are many UTXOs out there, a user needs to have a way to know which ones belong to him, which is not trivial since these UTXOs are encrypted. The naive solution would be for the user to download each UTXO and perform the necessary calculations to decrypt it and find out if it belongs to him. However, this computation is time-consuming as it involves multiple cryptographic operations. Even though a complete solution to this problem has not been found yet, it is suggested that slight variations of FHE-based PSI protocols could be used to “query key-value pairs stored on a server without the server learning which keys were requested.” More on this can found in _[this paper](https://eprint.iacr.org/2021/1116)
_.

### Limitations and conclusion

Our benchmarks show a pretty fast protocol! Performing a completely trustless FHE-based PSI on arbitrarily large sets in just a few seconds is a very good result.

However, there are some design limitations to it that should be kept in mind. The main one is that this protocol is designed for only two entities, and we have not found an efficient way to extend it to more. In the context of access lists, for example, it would probably be desirable to ensure that no element is repeated in any pair of all access lists (i.e., that each storage location is in at most one access list). This could be achieved if each party executed the PSI protocol with each other party, but this would involve O(n2)O(n^2)O(n2) executions of the protocol across the network, which is not ideal.

If you have any thoughts on how to solve this limitation, or want to discuss other ideas from this project, we look forward to hearing from you! We are also very interested in hearing about other applications you can think of for this efficient PSI protocol. Feel free to _[shoot me an email](mailto:lichu@semiotic.ai)
_, _[submit a PR](https://github.com/LichuAcu/psi-demo/)
_ to our GitHub repo or [ping me on Twitter at @lichuacu](http://twitter.com/lichuacu)
. 🫡

---

# Semiotic Labs | Podcasts

![Gradient background](/_astro/curved_bg_desktop.ChjV8nJN_1c0N0M.svg) ![Gradient background](/_astro/curved_bg_mobile.DP8czMRd_1b6C73.svg)

*   [![GRTiQ podcast episode 144 Anirudh Patel](/_astro/grtiq144.CP9oaoVT_1YWa9U.webp)\
    \
    #### GRTiQ Podcast: 144 Anirudh Patel\
    \
    ##### GRTiQ\
    \
    In a captivating edition of the GRTiQ podcast, Nick Hansen welcomes back Aniurdh “Ani” Patel, a Senior Research Scientist at Semiotic Labs, for an insightful and comprehensive interview. Following an impactful discussion in a previous episode, Ani shares his extensive journey through AI, web3, and The Graph, infused with his passion for travel and innovation. The discussion delves deep into the intricacies of web3, crypto, and Ani's introduction to Semiotic Labs via Sandia Labs. The conversation has insights into the world of artificial intelligence and machine learning. As a highlight, Ani unveils the exciting launch of our LLM-based query testbed, AgentC.](https://www.grtiq.com/grtiq-podcast-144-anirudh-patel/)
    
*   [![GRTiQ podcast episode 43 Sam Green](/_astro/grtiq43.B_Iriyvr_1mtEQV.webp)\
    \
    #### GRTiQ Podcast 43: Sam Green\
    \
    ##### GRTiQ\
    \
    In Episode 43 of the GRTiQ Podcast, host Nick Hansen interviews Sam Green, Co-Founder and CTO of Semiotic, the fourth core dev team to join The Graph protocol. Sam discusses Semiotic's exceptional expertise in artificial intelligence and cryptography, and provides clear explanations of key concepts like reinforcement learning and zk roll-ups. The insights shared by Sam shed light on not only the future of The Graph and web3 but also the cutting-edge technological innovations being pursued within The Graph protocol.](https://www.grtiq.com/grtiq-podcast-43-sam-green/)
    
*   [![GRTiQ podcast episode 144 Anirudh Patel](/_astro/grtiq74.DQUhTtG2_Z1NPkwF.webp)\
    \
    #### GRTiQ Podcast: 74 Ahmet Ozcan\
    \
    ##### GRTiQ\
    \
    In this episode of the GRTiQ Podcast, host Nick Hansen interviews Ahmet Ozcan, Co-founder and CEO of Semiotic Labs, a Core Dev team contributing to The Graph. The conversation covers Semiotic Labs' recent launch of Odos, an innovative path-finding algorithm designed to optimize order routing for crypto traders. Ahmet also discusses the team's work on The Graph, his transition from a successful career at IBM to entrepreneurship in Web3, and shares his long-term vision for The Graph.](https://www.grtiq.com/grtiq-podcast-74-ahmet-ozcan/)
    
*   [![The main chain. Ahmet Ozcan CEO & co-founder of Semiotic Ai.](/_astro/main_chain_ahmet_fundamental_inovations.CJw444mA_Z2gy0ly.webp)\
    \
    #### The Main Chain: Ahmet Ozcan – Fundamental Innovations\
    \
    ##### Infinity Ventures Crypto (IVC)\
    \
    It's an exciting time right now in crypto - with decentralized exchanges (DEX) still in their infancy, crypto users are evaluating their potential. As DEXs transactions are completely automated, they can be interacted with in a 100% programmatic fashion. which means they are ripe for applications to be built on top of them. That’s where Semiotic AI comes in. Dr. Ahmet Ozcan is the CEO of Semiotic AI, which aims to build automated decision-making tools for decentralized markets on the blockchain. The Semiotic team are the brains behind odos.xyz, a web3 application that allows traders to make large token swaps and trades across disparate DEXs, quickly and efficiently. If one DEX doesn’t have the liquidity needed to complete the transaction, Odos will map trades across DEXs, making large trades effortless on the part of an investor. Semiotic has also been in the news lately for their contributions to The Graph, the indexing and query layer of Web3, for which they received a grant of USD $60 million from the Graph Foundation. Dr. Ozcan chats with host Michael Waitze about how odos works and details its future roadmap. He also discusses why The Graph is important, what it does, and how Semiotic is contributing to The Graph, and to the future of web3, including how their AI solutions simulate how different incentives motivate actors to work either for the good or the detriment of blockchain systems.](https://anchor.fm/themainchain/episodes/Ahmet-Ozcan---Fundamental-Innovations-e1ivpf8)

---

# Indexer Allocation Optimization: Part II

![Gradient background](/_astro/curved_bg_desktop.ChjV8nJN_1c0N0M.svg) ![Gradient background](/_astro/curved_bg_mobile.DP8czMRd_1b6C73.svg)

_This article was co-authored with [Howard Heaton](https://howardheaton.tech/)
, from [Edge & Node](https://edgeandnode.com/)
, and with [Hope Yen](https://www.linkedin.com/in/hope-yen-hk/)
, from [GraphOps](https://graphops.xyz/)
_.

TL;DR
-----

Analytically optimizing to maximize indexing rewards may seem like a straightforward solution, but it oversimplifies the complexities involved in an indexer’s decision-making process. In this post, we’ll discuss how we enable indexers to input their preferences into the [Allocation Optimizer](https://github.com/graphprotocol/allocation-optimizer)
 and how those preferences can impact the Allocation Optimization problem. We’ll also explore how we incorporate a gas fee to make the optimization problem more realistic, which results in a non-convex problem.

Overview
--------

In [Part I](/articles/indexer-allocation-optimisation/)
, we provided an overview of the Indexer Allocation Optimization problem and discussed how we can optimize for a basic version of it. However, indexers have unique preferences and constraints that cannot be easily represented in the math model. Additionally, we need to factor in gas costs, which are a per-allocation fee that indexers incur when they allocate. In this section, we’ll address both indexer preferences and gas costs, one at a time. It’s important to note that while we will use some math concepts, we’ll strive to explain them in a way that’s accessible to readers with a college undergraduate level of math knowledge, by which we mean a basic knowledge of calculus and linear algebra.

Optimizing Over Gas
-------------------

Gas costs have long been a challenging issue for those seeking to solve web3 optimization problems, with a variety of proposed solutions. For instance, Angeris et al.’s paper on optimal routing for constant function market makers\[1\] proposes one approach using convex relaxation. In this section, we’ll offer our own take by framing the problem of optimal allocation with fixed transaction costs as a sparse vector optimization problem. Specifically, we aim to find the set of optimal sparse vectors and then select the specific sparse vector that yields the highest profit. We define profit as the total indexing reward minus the total gas cost. In the following section, we’ll go through the background math at a very high level. For the most part, we use images to make our points. However, if you’d rather not understand how the algorithm works, feel free to skip over it and jump to the section “Indexer Profits.”

### The Math At A Glance

Let’s take a look at an example. Say we want to minimize the function

f(x1,x2)\=(x1−2)2+(x2−5)2\\begin{equation} f(x\_1, x\_2) = (x\_1 - 2)^2 + (x\_2 - 5)^2 \\end{equation}f(x1​,x2​)\=(x1​−2)2+(x2​−5)2​​

![Convex contour plot](/_astro/function-4.CTmDmY6G_Z1fI3yG.webp)

A filled contour plot of Eq. 1.

You should hopefully be able to see that this happens at (2,5)(2, 5)(2,5). Plug those numbers in for x1x\_1x1​ and x2x\_2x2​ if you can’t. The result should be 000. This is obvious to see, but let’s see if we can do this in a more automated way.

One way to do this is gradient descent. Here’s how it works. Pick some random point x(1)x^{(1)}x(1) to start the algorithm from. The gradient of the function ∇f\\nabla f∇f gives you the direction of steepest ascent. Since we want to find the minimum, we put a negative sign in front of it. We step in the direction of the negative gradient with step size η\\etaη. Thus, our update rule becomes.

x(n+1)\=x(n)−η∇f(x(n))\\begin{equation} x^{(n+1)} = x^{(n)} - \\eta \\nabla f(x^{(n)}) \\end{equation}x(n+1)\=x(n)−η∇f(x(n))​​

**Note:** If you aren’t interested in the math, feel free to skip ahead to the paragraph above the next image.

Let’s plug in our fff and see this in action.

f(x1,x2)\=(x1−2)2+(x2−5)2\\begin{equation\*} f(x\_1, x\_2) = (x\_1 - 2)^2 + (x\_2 - 5)^2 \\end{equation\*}f(x1​,x2​)\=(x1​−2)2+(x2​−5)2​

We’ll split this into two parts - solving for x1x\_1x1​ and solving for x2x\_2x2​ independently. Let’s start with x1x\_1x1​. First, we find the gradient of fff with respect to x1x\_1x1​.

∂f∂x1\=2(x1−2)\=2x1−4\\begin{equation\*} \\frac{\\partial f}{\\partial x\_1} = 2(x\_1 - 2) = 2x\_1 - 4 \\end{equation\*}∂x1​∂f​\=2(x1​−2)\=2x1​−4​

Now, we use the formula for gradient descent and start plugging in numbers.

x1(n+1)\=x1(n)−η∂f∂x1\=x1(n)−η(2x1(n)−4)\=x1(n)−2ηx1(n)+4η\=(1−2η)x1(n)+4η\\begin{align\*} x^{(n+1)}\_1 &= x^{(n)}\_1 - \\eta \\frac{\\partial f}{\\partial x\_1} \\\\ &= x^{(n)}\_1 - \\eta (2x^{(n)}\_1 - 4) \\\\ &= x^{(n)}\_1 -2\\eta x^{(n)}\_1 + 4\\eta \\\\ &= (1 - 2\\eta)x^{(n)}\_1 + 4\\eta \\end{align\*}x1(n+1)​​\=x1(n)​−η∂x1​∂f​\=x1(n)​−η(2x1(n)​−4)\=x1(n)​−2ηx1(n)​+4η\=(1−2η)x1(n)​+4η​

We can now replace x(n)x^{(n)}x(n) with the same gradient descent formula applied to x(n−1)x^{(n-1)}x(n−1).

x1(n+1)\=(1−2η)\[(1−2η)x(n−1)+4η\]+4η\=(1−2η)2x1(n−1)+\[(1−2η)4η+4η\]\\begin{align\*} x^{(n+1)}\_1 &= (1 - 2\\eta)\\left\[(1-2\\eta)x^{(n-1)} + 4\\eta\\right\] + 4\\eta \\\\ &= (1 - 2\\eta)^2x^{(n-1)}\_1 + \\left\[(1 - 2\\eta)4\\eta + 4\\eta\\right\] \\end{align\*}x1(n+1)​​\=(1−2η)\[(1−2η)x(n−1)+4η\]+4η\=(1−2η)2x1(n−1)​+\[(1−2η)4η+4η\]​

I grouped certain terms together on purpose. Hopefully, you can sort of see the pattern at this point. If not, I recommend you do the math again. Regardless, if we want to write x(n+1)x^{(n+1)}x(n+1) in terms of x(n−2)x^{(n-2)}x(n−2), we can do this as

x1(n+1)\=(1−2η)2x1(n−2)+\[(1−2η)24η+(1−2η)4η+4η\]\\begin{equation\*} x^{(n+1)}\_1 = (1 - 2\\eta)^2x^{(n-2)}\_1 + \\left\[(1 - 2\\eta)^24\\eta + (1 - 2\\eta)4\\eta + 4\\eta\\right\] \\end{equation\*}x1(n+1)​\=(1−2η)2x1(n−2)​+\[(1−2η)24η+(1−2η)4η+4η\]​

And if we keep doing this, we get

x1(n+1)\=(1−2η)nx1(1)+\[(1−2η)n−1+(1−2η)n−2+...+1\]4η\\begin{equation\*} x^{(n+1)}\_1 = (1 - 2\\eta)^nx^{(1)}\_1 + \\left\[(1 - 2\\eta)^{n-1} + (1 - 2\\eta)^{n-2} + ... + 1\\right\]4\\eta \\end{equation\*}x1(n+1)​\=(1−2η)nx1(1)​+\[(1−2η)n−1+(1−2η)n−2+...+1\]4η​

You should hopefully recognize the term in the square brackets as a geometric series. If ∣η∣<1|\\eta| < 1∣η∣<1, this converges to give us

x1(n+1)\=(1−2η)nx1(1)+1−(1−2η)n1−(1−2η)4η\\begin{equation\*} x^{(n+1)}\_1 = (1 - 2\\eta)^nx^{(1)}\_1 + \\frac{1 - (1 - 2\\eta)^n}{1 - (1 - 2\\eta)}4\\eta \\end{equation\*}x1(n+1)​\=(1−2η)nx1(1)​+1−(1−2η)1−(1−2η)n​4η​

Notice here that we must have ∣1−2η∣<1|1 - 2\\eta| < 1∣1−2η∣<1. Else, the (1−2η)n(1 - 2\\eta)^n(1−2η)n terms blow up to infinity. If we select η\\etaη to be such that that inequality is satisfied, notice that every (1−2η)n(1 - 2\\eta)^n(1−2η)n goes to 000 as n→∞n \\to \\inftyn→∞. Thus, as n→∞n \\to \\inftyn→∞, we can rewrite our formula as

x1(n+1)\=12η4η\=2\\begin{equation\*} x^{(n+1)}\_1 = \\frac{1}{2\\eta}4\\eta = 2 \\end{equation\*}x1(n+1)​\=2η1​4η\=2​

Thus, our optimal x1\=2x\_1 = 2x1​\=2. Notice this is exactly what we’d found visually earlier. The only difference is that now we’ve used a less manual algorithm instead. If our function fff satisfies certain properties, we are guaranteed to find the global minimum using gradient descent. The full list of properties is beyond the scope of this post. We’ll just focus on one - convexity.

![Convex gradient descent](/_astro/gd-2.6cq2ACvg_Z1Fvhm8.webp)

Gradient descent on Eq. 1. The red line shows the trajectory the algorithm takes to reach the optimal value. The bright red diamond is the optimal value.

A convex function fff is a function that always curves upwards. This means that for two points in aaa and bbb, the line segment connecting f(a)f(a)f(a) to f(b)f(b)f(b) is always on or above the curve of fff. So x2x^2x2 is convex; x3x^3x3 is not. −x2\-x^2−x2 is concave, which is fine because we can just run gradient ascent to find the maximum instead of gradient descent to find the minimum. We’ll come back to convexity a bit later.

Now, that we know we can use gradient descent to minimize an objective function, let’s be a bit more interesting. Let’s say there are constraints on our solution. Let’s say that I have a maximum of 10 units I can divide between x1x\_1x1​ and x2x\_2x2​. This is totally fine, because 2+5=7 which is less than 10. Now let’s say instead that I have just 1 unit I can allocate. Would I be better off dividing this between x1x\_1x1​ and x2x\_2x2​, or would it be better to go all in on one? This is no longer as obvious.

In Part I, we talked about using the Karush-Kuhn-Tucker conditions to solve this problem, and this is what gave us our analytic solution. Now, we want to continue on this theme of finding a more general solution. In this case, we can use projections.

This new constraint, that the x1+x2\=1x\_1 + x\_2 = 1x1​+x2​\=1 and that x1≥0x\_1 \\geq 0x1​≥0 and x2≥0x\_2 \\geq 0x2​≥0, has a name - the simplex constraint. The simplex constraint has a corresponding simplex projection.\[2\] We’ll use this here. I won’t talk through the math again. If you’re interested, feel free to work it out for yourself. I’ll just give you the new update rule.

x1(n+1)\=T(x(n)−η∇f(x(n)))\\begin{equation} x^{(n+1)}\_1 = T(x^{(n)} - \\eta \\nabla f(x^{(n)})) \\end{equation}x1(n+1)​\=T(x(n)−η∇f(x(n)))​​

In other words, we take our normal gradient descent step, and then we project the result of that onto the simplex, which is denoted by the operator TTT. What does this look like?

![Projected gradient descent](/_astro/pgd-3.BlZn56Gt_217Adp.webp)

Projected gradient descent onto the unit-simplex for Eq. 1. The white line denotes the feasible region.

We revisit our familiar problem f(x1,x2)\=(x1−2)2+(x2−5)2f(x\_1, x\_2) = (x\_1 - 2)^2 + (x\_2 - 5)^2f(x1​,x2​)\=(x1​−2)2+(x2​−5)2. Now we add a unit-simplex constraint (x1+x2\=1x\_1 + x\_2 = 1x1​+x2​\=1; x1≥0x\_1 \\geq 0x1​≥0, x2≥0x\_2 \\geq 0x2​≥0). In the above figure, we set x(1)\=(0,0)x^{(1)} = (0, 0)x(1)\=(0,0). From here, we first project this onto the unit simplex. The white line shows the feasible region of the unit-simplex constraint, meaning all the possible combinations of (x1,x2)(x\_1, x\_2)(x1​,x2​) that satisfy the expressions from before. The projection operation T(x)T(x)T(x) simply finds the closest x′x'x′ that is in the feasible region (on the white line). You can see this happening in our plot. First x(1)\=(0,0)x^{(1)} = (0, 0)x(1)\=(0,0) gets projected onto (0.5,0.5)(0.5, 0.5)(0.5,0.5). Then, we take a step of gradient descent, pulling us off the feasible region in the direction of the true optimal, which we know from before to be (2,5)(2, 5)(2,5). Then our projection brings us back onto the feasible region. We repeat this until we converge to the optimal value for our new problem (0,1)(0, 1)(0,1).

Let’s add another layer of complexity again. Let’s say we know for a fact that our solution is sparse - meaning the solution is either (0,1)(0, 1)(0,1) or (1,0)(1, 0)(1,0). Can we do any better than this back-and-forth stepping? We treat this as a new constraint, which we call a sparsity constraint - the number of nonzero elements of xxx should be at most kkk. In this case, k\=1k = 1k\=1. Sparse projections onto the simplex are again a common type of problem, so we’ll refer you to the cited paper if you want to learn more.\[3\] In any case, we can now use the Greedy Selector and Simplex Projector (GSSP) algorithm in place of where we’d used the simplex projection before.

![GSSP](/_astro/gssp.Dd3I7NaX_LAhqC.webp)

The GSSP projection converges more quickly but converges to the wrong point.

Using GSSP converges a lot quicker, but actually gets stuck on the wrong point. It gets stuck on (1,0)(1, 0)(1,0) rather than (0,1)(0, 1)(0,1). We can check that this is a worse solution by plugging both of those points back into our objective function. f(1,0)\=26f(1, 0) = 26f(1,0)\=26 and f(0,1)\=20f(0, 1) = 20f(0,1)\=20. Since we’re minimizing, the latter is better. Is there anything we can do here?

Of course! We know the optimal unconstrained point. We were even able to just look at the plot and see that it is (2,5)(2, 5)(2,5). We’ll use this an an anchor point in Halpern iteration.\[4\] Again we won’t go through the mathematical details of the algorithm other than to point you to the paper. Instead, we’ll just talk through the algorithm visually.

![Halpern Iteration](/_astro/halpern.BzDg8pCW_ZRjtFU.webp)

Halpern iteration gets us to the correct solution much more quickly than we would otherwise.

At a high level, Halpern iteration will try to anchor our solution to some anchor point xa\=(2,5)x\_a = (2, 5)xa​\=(2,5). What this means is that we’ll first pull the solution towards (2,5)(2, 5)(2,5), and then use GSSP to project the solution back onto the simplex. As the number of iterations increases, the pull of the anchor point lessens to the point at which it drops off completely, and we’re left with just the vanilla projected gradient descent with GSSP that we talked about before. This is equivalent to adding a new operator HHH that executes Halpern iteration during our projected gradient descent update.

x1(n+1)\=T(H(x(n)−η∇f(x(n))))\\begin{equation} x^{(n+1)}\_1 = T(H(x^{(n)} - \\eta \\nabla f(x^{(n)}))) \\end{equation}x1(n+1)​\=T(H(x(n)−η∇f(x(n))))​​

### Indexer Profits

Back to our problem. Instead of our dummy f(x)f(x)f(x), now we use the indexing reward.

min⁡Ωi−∑j∈SΩij∑k∈IΩkj⋅Φψj∑l∈Sψls.t.ωk≥0∀ ωk∈Ωi∑j∈SΩij\=σi\\begin{align\*} \\min\_{\\Omega\_i} \\quad & -\\sum\_{j \\in \\mathcal{S}}\\frac{\\Omega\_{ij}}{\\sum\_{k \\in \\mathcal{I}} \\Omega\_{kj}}\\cdot\\Phi\\frac{\\psi\_j}{\\sum\_{l \\in \\mathcal{S}} \\psi\_l} \\\\ \\textrm{s.t.} \\quad & \\omega\_k \\geq 0 \\quad \\forall \\ \\omega\_k\\in\\Omega\_i \\\\ \\quad & \\sum\_{j\\in\\mathcal{S}}\\Omega\_{ij} = \\sigma\_i \\end{align\*}Ωi​min​s.t.​−j∈S∑​∑k∈I​Ωkj​Ωij​​⋅Φ∑l∈S​ψl​ψj​​ωk​≥0∀ ωk​∈Ωi​j∈S∑​Ωij​\=σi​​

If this notation is unfamiliar, please refer back to [Part I](https://astro-website-81r.pages.dev/articles/indexer-allocation-optimisation/)
. We will call the above formulation **A.** Notice our constraints are actually a simplex constraint - they take the form ∑ixi\=C\\sum\_i x\_i = C∑i​xi​\=C, xi≥0 ∀ix\_i \\geq 0 \\ \\forall ixi​≥0 ∀i. This means we can solve this problem using the simplex projection we talked about in the previous section. Use gradient descent to step in the direction of an optimum; then, use the simplex projection to bring ourselves back onto the feasible region. As in the last section, we can use the optimal value computed analytically, as shown in [Part I](https://astro-website-81r.pages.dev/articles/indexer-allocation-optimisation/)
 as our anchor for Halpern iteration. This is, in fact, what the Allocation Optimizer does. Well, almost.

So far, we’ve talked about indexing rewards. We are actually concerned with indexer profits. Our optimization problem is now

lmin⁡Ωi−∑j∈SΩij∑k∈IΩkj⋅Φψj∑l∈Sψl+2g∑j∈Sηjs.t.ωk≥0∀ ωk∈Ωi∑j∈SΩij\=σil \\begin{align\*} \\min\_{\\Omega\_i} \\quad & -\\sum\_{j \\in \\mathcal{S}}\\frac{\\Omega\_{ij}}{\\sum\_{k \\in \\mathcal{I}} \\Omega\_{kj}}\\cdot\\Phi\\frac{\\psi\_j}{\\sum\_{l \\in \\mathcal{S}} \\psi\_l}+ 2g\\sum\_{j\\in\\mathcal{S}}\\eta\_j \\\\ \\textrm{s.t.} \\quad & \\omega\_k \\geq 0 \\quad \\forall \\ \\omega\_k\\in\\Omega\_i \\\\ \\quad & \\sum\_{j\\in\\mathcal{S}}\\Omega\_{ij} = \\sigma\_i \\end{align\*}lΩi​min​s.t.​−j∈S∑​∑k∈I​Ωkj​Ωij​​⋅Φ∑l∈S​ψl​ψj​​+2gj∈S∑​ηj​ωk​≥0∀ ωk​∈Ωi​j∈S∑​Ωij​\=σi​​

We’ll call this formulation **B**.

**B** is a problem. Our objective function, which was convex in **A**, is no longer so. This is because we’ve added a new term depending on ηj\\eta\_jηj​, which is a binary variable such that it has value ηj\=1\\eta\_j = 1ηj​\=1 if ωj\>0\\omega\_j > 0ωj​\>0 and 0 otherwise. This is a big problem for us. Too see why, consider the picture below.

![A non-convex function](/_astro/nonconvex.Bjg7XiCR_Z2gJFX.webp)

A non-convex function.

The plot above shows a non-convex function. Consider the problem of using gradient descent to minimize it. As a reminder, gradient descent follows the slope down. Once the slope shifts from negative to positive, gradient descent is done. In the above plot, we have two minima, one that is at x\>0x>0x\>0 and one that is at x<0x<0x<0. Let’s imagine we start gradient descent from x\>1x>1x\>1. What happens? Well, gradient descent will slide leftwards until it reaches the rightmost minimum. Now imagine, that we start from x<−2x<-2x<−2. What happens now? Gradient descent will step rightwards until it hits the leftmost minimum. Except this isn’t the best we could have done. We actually could have gone further and found a better solution. Here’s the problem we now face put more directly. We can use gradient descent just fine on non-convex functions, but how do we know we’ve actually reached the global minimum, rather than a local minimum? How do we know we couldn’t do better than the solution we found? As I’ve mentioned before, this style of problem, in which a convex function becomes non-convex due to gas fees, is everywhere in web3. Let’s talk about one of the potential solutions to this.

For the sake of argument, let’s pretend there were 10 subgraphs on the network. If an indexer wanted to allocate, they could allocate on one subgraph, or two subgraphs, or three subgraphs, all the way up to ten subgraphs. Each subsequent allocation vector (from 1 nonzero to 10) would have increasing gas costs corresponding with the ηj\\eta\_jηj​ term in **B**. What we could do is solve **A** with an additional sparsity constraint.

min⁡Ωi−∑j∈SΩij∑k∈IΩkj⋅Φψj∑l∈Sψls.t.ωk≥0∀ ωk∈Ωi∑j∈SΩij\=σi∑j∈Sηj≤k\\begin{align\*} \\min\_{\\Omega\_i} \\quad & -\\sum\_{j \\in \\mathcal{S}}\\frac{\\Omega\_{ij}}{\\sum\_{k \\in \\mathcal{I}} \\Omega\_{kj}}\\cdot\\Phi\\frac{\\psi\_j}{\\sum\_{l \\in \\mathcal{S}} \\psi\_l} \\\\ \\textrm{s.t.} \\quad & \\omega\_k \\geq 0 \\quad \\forall \\ \\omega\_k\\in\\Omega\_i \\\\ \\quad & \\sum\_{j\\in\\mathcal{S}}\\Omega\_{ij} = \\sigma\_i \\\\ \\quad & \\sum\_{j\\in\\mathcal{S}}\\eta\_j \\leq k \\end{align\*}Ωi​min​s.t.​−j∈S∑​∑k∈I​Ωkj​Ωij​​⋅Φ∑l∈S​ψl​ψj​​ωk​≥0∀ ωk​∈Ωi​j∈S∑​Ωij​\=σi​j∈S∑​ηj​≤k​

We’ll call this formulation **C**. If we solve **C** for each value kkk from 1 to the number of subgraphs, then we’ll have our optimal allocation strategy if the indexer opens one new allocation, the optimal strategy for two allocations, and so on. Then, we can choose the best of these by picking the one that minimizes the objective function of **B**. We already know how to solve problems with a sparse simplex constraint. Just use GSSP!

When you run the Allocation Optimizer with `opt_mode = "fast"`, this is exactly what the code does! In summary, solve **C** using projected gradient descent with GSSP and Halpern iteration. Then, for each of these possible allocation vectors, compute the magnitude of the objective function of **B**. Finally, return the allocation strategy with the lowest magnitude.

### Results

![Results](/_astro/gas.CUYRp2Sr_ZU3a7G.webp)

The results of our optimization methodology for an indexer as we vary the gas cost in GRT. As expected, higher gas costs result in fewer allocations.

|     |     |     |     |
| --- | --- | --- | --- |
|     | 100 GRT | 1000 GRT | 10000 GRT |
| Current Profit | 191525.88 | 183425.88 | 102425.88 |
| Optimized Profit | 540841.27 | 469017.12 | 333127.37 |
| % Increase | 282% | 255% | 325% |

Comparing the indexer’s current allocation strategy to the optimized allocation strategies from the table. For example, the indexer’s profit at a gas cost of 100 GRT is compared with the green strategy in the plot.

If you recall from Part I, we were originally concerned that the analytic solution allocated to way more subgraphs than indexers did. This was in part due to the fact that the analytic solution does not respect gas costs. Here, we demonstrate that the new algorithm does so, and also manages to outperform the indexer’s existing allocation strategy across a variety of different gas costs. Other than gas costs, the Allocation Optimizer also enables indexers to further pare down this via other entries in the configuration file. Let’s run through those.

Indexer Preferences
-------------------

There is no way our optimizer could compensate for the full diversity of thought behind how different indexers choose to allocate. Some may prefer shorter allocations. Others may prefer to keep their allocation open for a maximum of 28 epochs (roughly 28 days). Some indexers may have strong negative or positive feelings towards specific subgraphs. In this section, we talk through how the Allocation Optimizer accounts for this information.

### Filtering Subgraphs

Indexers often don’t care to index some subgraphs. Maybe a subgraph is broken. Maybe it’s massive, and the indexer just won’t be able to sync it in a short amount of time. Whatever the reason, we want to ensure that indexers have the ability to _blacklist_ subgraphs. In the previous post, we used the notation S\\mathcal{S}S to represent the set of all subgraphs. To remove blacklisted subgraphs from the optimization problem, all we do is choose some set S′\=S\\B\\mathcal{S}' = \\mathcal{S}\\backslash \\mathcal{B}S′\=S\\B where B\\mathcal{B}B is the set of blacklisted subgraphs.

Since indexing a subgraph takes time, many indexers also have a curated list of subgraphs that they already have synced or mostly synced. They may only want to allocate to subgraphs in that list. Therefore, indexers can also specify a _whitelist_ in the configuration file. To see how the whitelist W\\mathcal{W}W modifies the math, all we have to do is replace all instances of S\\mathcal{S}S with W\\mathcal{W}W.

Let’s say an indexer is already allocated on a subgraph and does not want the Optimizer the try to unallocate from this subgraph. The indexer can specify this in the _frozenlist_. Subgraphs in the frozenlist are treated as though they are blacklisted. However, they differ from blacklisted subgraphs in that the indexer’s allocated stake on these frozen subgraphs σf\\sigma\_fσf​ is not available for reallocation. Therefore, σ′\=σ−σf\\sigma' = \\sigma - \\sigma\_fσ′\=σ−σf​.

The final list we allow for is the _pinnedlist_. This is mostly meant to be used by so-called backstop indexers - indexers who are not necessarily there to make a profit from the network, but more just to ensure that all data is served. Pinned subgraphs are treated as whitelisted since the Optimizer may decide to allocate to them anyway. However, if the Optimizer doesn’t, we manually add 0.1 GRT back onto each of these subgraphs just so that the indexer has a small, non-negative allocation on the pinned subgraphs.

Beyond the lists, indexers may also choose to not be interested in subgraphs with signal less than some amount, even if it is optimal to be on said subgraph. Thus, they can specify a _min\_signal_ value in the configuration file. The Optimizer will filter out any subgraphs with signal below this amount.

### Other Preferences

As we mentioned before, indexers may also choose to allocate for a variety of different time periods. Thus, they can specify the _allocation\_lifetime_ in the configuration file. This determined how many new GRT are minted from token issuance during that time frame, which is given by the classic P(rt−1)P(r^t - 1)P(rt−1) where PPP is the current number of GRT in circulation, rrr is the issuance rate, and. ttt is the allocation lifetime in blocks. As ttt decreases, fewer tokens are issued, which means the effect of the gas term in our objective function becomes stronger.

The final preference that indexers can specify is _max\_allocations_. For whatever reason, an indexer may prefer to allocate to fewer than the optimal number of subgraphs. Maybe this is due to hardware constraints on their side, or else perhaps it’s because of reduced mental load in tracking subgraphs. In any case, once indexers specify this number, the Optimizer uses it as kmaxk\_{max}kmax​. In other words, when it solves formulation **C**, normally it would solve it for k∈\[1,...,number of subgraphs\]k \\in \[1,...,\\textrm{number of subgraphs}\]k∈\[1,...,number of subgraphs\]. Instead, it will solve it k∈\[1,...,kmax\]k \\in \[1,...,k\_{max}\]k∈\[1,...,kmax​\] times.

Conclusion
----------

In Part II of the Allocation Optimizer series, we’ve demonstrated how the Allocation Optimizer accounts for indexer preferences and how the `Val(:fast)` mode solves the non-convex optimization problem using GSSP and Halpern iteration. We also demonstrated some results for the algorithm, showing that it outperforms a current indexer’s existing allocations. However, we did not demonstrate optimality. Spoiler alert, the Allocation Optimizer also has a `Val(:optimal)` flag. The solution we talked about here is fast, and will often outperform manually choosing allocations. However, we can do better. We’ll leave the details to our next blog post though.

Just a final note, if you’re interested in playing around with the techniques described in our blog post - gradient descent, projected gradient descent, the simplex projection, GSSP, and Halpern iteration, check out our Julia package [SemioticOpt](https://github.com/semiotic-ai/SemioticOpt.jl)
. The Allocation Optimizer and the code that generated the plots in this blogpost both use SemioticOpt.

### Further Reading

*   \[1\] [Optimal Routing for Constant Function Market Makers](https://arxiv.org/abs/2204.05238)
     by Angeris, Chitra, Evans, and Boyd
    
*   \[2\] [Projection Onto A Simplex](https://arxiv.org/pdf/1101.6081.pdf)
     by Chen and Ye
    
*   \[3\] [Sparse projections onto the simplex](https://arxiv.org/pdf/1206.1529.pdf)
     by Kyrillidis, Becker, Cevher, and Koch
    
*   \[4\] [Fixed Points of Nonexpanding Maps](https://projecteuclid.org/journals/bulletin-of-the-american-mathematical-society/volume-73/issue-6/Fixed-points-of-nonexpanding-maps/bams/1183529119.pdf)
     by Halpern

---

# Semiotic Labs | Podcasts

![Gradient background](/_astro/curved_bg_desktop.ChjV8nJN_1c0N0M.svg) ![Gradient background](/_astro/curved_bg_mobile.DP8czMRd_1b6C73.svg)

*   [![GRTiQ podcast episode 43 Sam Green](/_astro/grtiq43.B_Iriyvr_1mtEQV.webp)\
    \
    #### GRTiQ Podcast 43: Sam Green\
    \
    ##### GRTiQ\
    \
    In Episode 43 of the GRTiQ Podcast, host Nick Hansen interviews Sam Green, Co-Founder and CTO of Semiotic, the fourth core dev team to join The Graph protocol. Sam discusses Semiotic's exceptional expertise in artificial intelligence and cryptography, and provides clear explanations of key concepts like reinforcement learning and zk roll-ups. The insights shared by Sam shed light on not only the future of The Graph and web3 but also the cutting-edge technological innovations being pursued within The Graph protocol.](https://www.grtiq.com/grtiq-podcast-43-sam-green/)
    
*   [![GRTiQ podcast episode 144 Anirudh Patel](/_astro/grtiq74.DQUhTtG2_Z1NPkwF.webp)\
    \
    #### GRTiQ Podcast: 74 Ahmet Ozcan\
    \
    ##### GRTiQ\
    \
    In this episode of the GRTiQ Podcast, host Nick Hansen interviews Ahmet Ozcan, Co-founder and CEO of Semiotic Labs, a Core Dev team contributing to The Graph. The conversation covers Semiotic Labs' recent launch of Odos, an innovative path-finding algorithm designed to optimize order routing for crypto traders. Ahmet also discusses the team's work on The Graph, his transition from a successful career at IBM to entrepreneurship in Web3, and shares his long-term vision for The Graph.](https://www.grtiq.com/grtiq-podcast-74-ahmet-ozcan/)
    
*   [![The main chain. Ahmet Ozcan CEO & co-founder of Semiotic Ai.](/_astro/main_chain_ahmet_fundamental_inovations.CJw444mA_Z2gy0ly.webp)\
    \
    #### The Main Chain: Ahmet Ozcan – Fundamental Innovations\
    \
    ##### Infinity Ventures Crypto (IVC)\
    \
    It's an exciting time right now in crypto - with decentralized exchanges (DEX) still in their infancy, crypto users are evaluating their potential. As DEXs transactions are completely automated, they can be interacted with in a 100% programmatic fashion. which means they are ripe for applications to be built on top of them. That’s where Semiotic AI comes in. Dr. Ahmet Ozcan is the CEO of Semiotic AI, which aims to build automated decision-making tools for decentralized markets on the blockchain. The Semiotic team are the brains behind odos.xyz, a web3 application that allows traders to make large token swaps and trades across disparate DEXs, quickly and efficiently. If one DEX doesn’t have the liquidity needed to complete the transaction, Odos will map trades across DEXs, making large trades effortless on the part of an investor. Semiotic has also been in the news lately for their contributions to The Graph, the indexing and query layer of Web3, for which they received a grant of USD $60 million from the Graph Foundation. Dr. Ozcan chats with host Michael Waitze about how odos works and details its future roadmap. He also discusses why The Graph is important, what it does, and how Semiotic is contributing to The Graph, and to the future of web3, including how their AI solutions simulate how different incentives motivate actors to work either for the good or the detriment of blockchain systems.](https://anchor.fm/themainchain/episodes/Ahmet-Ozcan---Fundamental-Innovations-e1ivpf8)

---

# An Overview of Automatic Market Maker Mechanisms

![Gradient background](/_astro/curved_bg_desktop.ChjV8nJN_1c0N0M.svg) ![Gradient background](/_astro/curved_bg_mobile.DP8czMRd_1b6C73.svg)

_This article explores the principles and mechanisms behind the many popular Automatic Market Maker designs currently used in production. While the mathematical details of these designs are fascinating in their own right, this article seeks to instead focus on graphical representations and high-level concepts, allowing for a more approachable and exhaustive exploration of the space._

_Watch our related Devcon 2022 talk here: [https://www.youtube.com/watch?v=KxuyHfmLHP0](https://www.youtube.com/watch?v=KxuyHfmLHP0)
_.

Introduction
------------

Historically, order books run by professional market makers have been the dominant method used to facilitate exchange. On-chain, maintaining an order book using traditional market-making tactics is prohibitively expensive, since storage and computation on distributed ledgers are in short supply and high demand. For this reason, Automatic Market Makers (AMM) have emerged as an efficient class of methods to facilitate the exchange of crypto assets on distributed ledgers. An AMM leverages smart contracts to allow permissionless participation in market-making by individuals seeking yield. These individuals passively provide liquidity to the contract, which can then use a predetermined function to automatically facilitate exchanges between buyers and sellers. The design of this procedure involves many tradeoffs that ultimately affect the utility of the platform to both liquidity providers and traders.

To understand AMM design tradeoffs, we must first understand that the ultimate goal of a market maker (both traditional and automatic) is to generate yield on a portfolio of assets by providing liquidity for trades in a market \[1\]. Yield will be the net result of fee/spread revenue earned on trading volume minus the change in the value of the portfolio. In AMMs this portfolio value change manifests as a so-called “impermanent loss”, since the portfolio value will decrease as the price deviates from the initial price, but will revert if the price moves back towards the initial one. Oftentimes this concept of a portfolio losing value as price moves is framed as a unique problem with AMMs, but this is only because AMMs allow it to be described and analyzed mathematically. Generally, losing portfolio value as the price moves is always a problem with market making, since by definition as a market maker you are allowing traders to exchange the less desirable asset for the more desirable asset. In this way, you will naturally receive more of the asset that is decreasing in value, and less of the one that is increasing. The market maker turns a profit when the revenue from spreads and/or fees outweighs this portfolio value change.

In AMMs this spread/fee revenue is typically collected by applying a near-one multiplier on the input or output of each trade. For instance, if an AMM wanted to charge a 0.3% fee, it could multiply the input by 0.997 and keep 0.003 \* input as revenue for its liquidity providers. Setting this value can have a large impact on liquidity provider returns, and a tradeoff must be found between profiting as much as possible from each trade (higher fees) and maximizing the number of trades (lower fees). The optimal value will typically depend on the type of assets in the portfolio, as well as the type of AMM being used (the subject of the rest of the article). For more stable assets where portfolio risk is lower, lower fees are typically optimal, while more volatile assets typically demand higher fees to compensate for the risk. Many AMM implementations will add additional logic for dynamic fees, which can increase or decrease depending on fluctuating risk factors (e.g. price volatility can lead to inaccurate pricing).

Basic Automatic Market Makers
-----------------------------

The first and most well-known AMM is the Constant Product Market Maker (CPMM), first released by Bancor in the form of bonding curves within “smart token” contracts, and then further popularized by Uniswap as an invariant function \[2\]\[3\]. The CPMM spreads liquidity out equally between all prices, automatically adjusting the price in the direction of each trade made. This makes it an extremely general solution, allowing CPMMs to facilitate exchange between any arbitrary pair of assets. This generality though also turns out to be the primary weakness of CPMMs: providing equal liquidity at all exchange rates means that only a small portion of the liquidity can be provided at the current exchange rate. A better intuition for these mechanics can be achieved by walking through a few trade examples, as is done here \[4\]. This means that a CPMM can only facilitate trade sizes that are a small fraction of its total value locked (TVL) before experiencing significant price impact, making them very capital inefficient. Thus, while the CPMM has proven itself an essential member of the AMM ecosystem (especially for long-tail assets), there have been a wide range of AMMs created since the CPMM’s inception that seek to improve upon this design.

![CPMM](/_astro/desmos-graph1-1024x683.BaitkpOG_1JpyN6.webp)

A Graph of the CPMM’s swap output (on y-axis) as a function of swap input (on x-axis). The CPMM in this example has both input and output reserves of 200. For small trade sizes (e.g. x = 1), nearly a 1:1 exchange rate is given, since the AMM starts with equal reserves. It can be seen that the CPMM gives marginally less output as the input increases, always maintaining reserves of both tokens. Correspondingly, the output of this graph will asymptotically approach the output reserve, which in this case is y = 200. An interactive graph can be found [here](https://www.desmos.com/calculator/f5x7omgomx)
.

One such design is the Constant Mean Market Maker (CMMM), first introduced by Balancer Protocol \[5\]. The CMMM generalizes the CPMM by allowing the liquidity provider to specify desired portfolio weights, such as a 20%-80% split. In comparison, the CPMM lacks this customization and therefore always enforces an equal 50%-50% portfolio split. This generalization provides more flexibility to liquidity providers to adjust their market-making portfolios, but the underlying design and results are still the same as the CMMM discussed above. For the sake of simplicity, the rest of the article will typically refer to the CPMM, but will generally also apply to the CMMM.

To outperform the CPMM, some prior assumptions about the market exchange rate of the assets must be made. This is not without risks, since making the wrong assumption can cause the market maker to exchange a large portion of its portfolio at the incorrect rate, causing the change in portfolio value to outweigh any fees collected. The extreme case of this is a Constant Sum Market Maker (CSMM), which facilitates exchange at a fixed rate regardless of the current portfolio the market maker holds. This allows for perfect capital efficiency at the selected exchange rate, but also quickly leads to the market maker losing all of the more valuable assets the moment the price deviates from the preselected value (at which point the portfolio is worth significantly less, and can no longer facilitate trades). For those familiar with Bayes theorem, many parallels can be seen here \[6\]. Using an outside oracle price (such as a 1:1 exchange rate for stablecoins) can be seen as using a prior, while adjusting the price in response to swaps (changes to your portfolio) can be seen as the likelihood. Under this lens, the CPMM can be seen as only using the likelihood with an infinitely diffuse prior, while a CSMM can be seen as utilizing only the prior and ignoring the likelihood. In the absence of any reasonable prior/oracle price (as is often the case in DeFi), the CPMM is a great choice. The CSMM only makes sense if you have complete trust in your prior/oracle price, which is rarely a good idea. Thus, the best option is usually a mixture of the two, which is what the bulk of AMM research focuses on.

![CSMM versus CPMM](/_astro/desmos-graph3-1024x683.BXC0Az0I_ZHzzxy.webp)

A graph of CSMM’s output (red) compared to the CPMM from above (blue). Unlike the CPMM, the CSMM does not adjust its output in response to changing reserves — it continues to honor a 1:1 exchange rate until it holds only one token, at which point it can’t market make anymore (and thus more input can not produce more output). The interactive graph can be found [here](https://www.desmos.com/calculator/blmlqnvsi7)
.

Hybrid Automatic Market Makers
------------------------------

For the reasons stated above, CSMMs are rarely used in practice. In theory, they sound great for facilitating trades between assets that should theoretically have the same value, such as two stablecoins. But, in practice, small fluctuations in desirability between these assets will still leave the market maker with only the less desirable asset (not to mention the risk of one de-pegging completely). This was some of the motivation behind Curve introducing a more efficient solution to this problem called StableSwap, our first example of a hybrid AMM \[7\]. Like a CPMM, a StableSwap market maker will adjust its exchange rate as its portfolio changes, but, unlike a CPMM (and like a CSMM), it will do so very slowly at first, maintaining close to a 1:1 exchange rate for longer. As the price deviates farther and farther from the 1:1 exchange rate, the rate at which the price is adjusted accelerates, such that the market maker is never left holding only one asset. The rate at which this process accelerates is adjustable via an amplification coefficient parameter set for each pool. This model has become the industry standard for assets pegged to the same external asset but has not seen much use outside of this specific use case.

![Curve StableSwap](/_astro/desmos-graph4-1024x683.fS16B_uZ_Z2elw0j.webp)

A graph of a special case of Curve’s StableSwap (purple) output function when the input and output reserve are equal, and A = 20. The CSMM (dotted red) and CPMM (dotted blue) are shown for reference. StableSwap approximates the CSMM closely until its output reserve is close to depleted, after which point it quickly adjusts similarly to the CPMM. The interactive graph can be found [here](https://www.desmos.com/calculator/skaoi7krtg)
.

An alternative algorithm for swapping stable assets was more recently introduced by Solidly, which builds on Uniswap V2 by adding an option to create a pair specialized for stable assets \[8\]. Unlike a normal Uniswap V2 pair, this pair utilizes a quartic sum invariant instead of the usual quadratic invariant (but retains the same interface). This flattens the curve around the 1:1 exchange rate and produces a very similar effect to StableSwap, but lacks an adjustment parameter similar to the amplification coefficient mentioned above. Empirically, the curve seems to provide similar output to StableSwap with an amplification coefficient of 2 (see below figure), which is much lower than is typically used in StableSwap pools. This makes it a bit lower risk (without additional tuning) for a wider range of stable assets that may have larger fluctuations around their peg, but it also may be slightly less efficient for those that do not.

![Solidly Stable Pair Curve](/_astro/desmos-graph5-1024x683.DxR0DBlu_238Ur6.webp)

A graph of Solidly’s Stable Pair Curve. The CSMM (dotted red), CPMM (dotted blue), and StableSwap (dotted purple) are shown for reference. Solidly’s Stable Curve can be seen to be quite close to StableSwap with A = 2, although the comparison is not perfect since they are very different curves. An interactive Graph can be seen [here](https://www.desmos.com/calculator/vs9rc2cp4i)
.

Other AMMs similarly aim to interpolate between a constant sum and constant product model, but also expand their scope beyond pegged assets. Dodo develops a Proactive Market Maker (PMM) algorithm that aims to flatten the price curve around an oracle price \[9\]. This oracle price can be set to 1 for pegged asset pairs, but can also utilize Chainlink oracles for volatile pairs. At a high level, the PMM algorithm aims to maintain target reserve values (how much of each asset it would like to have) and gives decreasing rates the farther away the real reserves deviate from these values. The PMM exposes a slippage parameter, k, that allows the algorithm to interpolate between a constant product market maker (k = 1) and a constant sum/fixed rate market maker (k = 0). The math for calculating trades with this setup gets a bit messy, requiring integration under some scenarios, and solving for quadratic roots under others. Regardless, it has proven itself to be an effective mechanism to provide capital-efficient liquidity.

![Dodo PMM](/_astro/desmos-graph6-1024x683.Cjvqdidv_Z1BY27O.webp)

A graph of Dodo’s PMM curve (purple) in an equilibrium state (actual reserves = target reserves). The CSMM (dotted red) and CPMM (dotted blue) are shown for reference. As the “liquidity parameter”, k, of the PMM is changed, the curve can be seen moving between the CPMM (k = 1) and the CSMM (k = 0). An interactive graph can be found [here](https://www.desmos.com/calculator/arwyj5lbm1)
.

Clipper develops an AMM that has a similar result but gets there in a very different way \[10\]. Clipper more explicitly derives a solution that interpolates between constant sum and constant product by combining the constant sum model, wherein assets are exchanged at a fixed price, and a constant product model, where assets are exchanged according to the ratio of reserves held by the market maker. Similar to Dodo, these two extremes are combined on a continuous spectrum parameterized by a slippage parameter, k. The resulting interpolation is a bit different, however — the most concrete difference is that, unlike Dodo, a clipper AMM is actually capable of giving away all of one asset (like a constant sum market maker). This can become particularly dangerous for configurations closer to a CSMM (k close to 0) with an unreliable oracle.

![Clipper AMM](/_astro/desmos-graph9-1024x683.C_gErB_7_Z2tTEcP.webp)

A graph of the Clipper AMM swap output curve (purple). The CSMM (dotted red) and CPMM (dotted blue) are shown for reference. As the “k” parameter of the Clipper AMM is changed, the curve can be seen moving between the CPMM (k = 1) and the CSMM (k = 0). An interactive graph can be found [here](https://www.desmos.com/calculator/nuxjbrvaea)
.

Curve, the originator of StableSwap, has also released a new AMM for volatile assets called CryptoSwap \[11\]. CryptoSwap expands on the StableSwap algorithm by adding another parameter enabling quicker switching into price discovery mode (constant product) as the price moves away from the peg. This makes the algorithm better at handling a dynamic peg, allowing it to be used for more volatile assets. Curve’s CryptoSwap implementations also include a dynamic fee and an internal oracle system, making it unique in that respect since most other solutions use fixed rates or Chainlink oracles \[12\]. Implementing this AMM requires solving cubic, sextic, and higher-degree equations, which is typically done in practice using Newton’s Method \[13\]. This combined with its internal oracle and dynamic fees makes it one of the most complex AMMs currently in use. (For this reason, a CryptoSwap graph is omitted.) It is still early in its life cycle, so it remains to be seen whether this extra complexity translates into a better-performing AMM. So far it has mostly seen its use limited to one pool filled with the highest market cap assets on a given chain. Recently, though, it has been getting rolled out in more pools, growing its reach in the AMM space.

Virtual Reserve Automatic Market Makers
---------------------------------------

An alternative way to achieve greater capital efficiency is to use virtual reserves, a concept first introduced by KyberSwap \[14\]. Virtual reserve AMMs utilize the CPMM model but multiply the real balances of each asset by an amplification factor. The result is that a much higher capital efficiency can be provided over the CPMM (bringing in more volume and fee revenue), but the market maker loses the CPMM’s ability to never run out of either asset. In this way, the amplification factor allows a tradeoff between capital efficiency and extra portfolio risk. Generally, a high amplification factor can work well for stable pairs but becomes riskier the more volatile a pair gets.

![Kyber AMM](/_astro/desmos-graph10-1024x683.BMDm94yf_Z1txfUX.webp)

A graph of the Kyber AMM (green) with A = 5. The CSMM (dotted red) and CPMM (dotted blue) are shown for reference. The Kyber curve begins as a CPMM at A = 1 and approaches a CSMM as A approaches infinity. An interactive graph can be found [here](https://www.desmos.com/calculator/wxgvy9dwrx)
.

Uniswap V3 puts a very interesting spin on the concept of virtual reserves \[15\]. Instead of imposing one constant amplification coefficient on the entire pool, Uniswap V3 allows each liquidity provider to pick their own price range (and implicitly their own amplification parameter to span that range). Liquidity providers take on more portfolio risk with tighter price range positions (implicitly this means higher amplification coefficients), but also accrue proportionally more fees for trades within this range. In this way, Uniswap V3 can be seen as using incentives (higher yield for picking the true price range as precisely as possible) to source the best tradeoff in capital efficiency and portfolio risk at a given point, even for two volatile assets. This model has proven extremely effective, but also generally requires liquidity providers to be more active and informed in order to receive a good yield. Naive liquidity provision over the entire price range will typically result in a much lower fee share, while aggressive but mismanaged liquidity provision will typically lead to portfolio value loss outweighing any fee revenue. The development of intelligent automated liquidity managing strategies will likely continue to make these (and similar) AMMs more effective, and the corresponding yield opportunities more competitive.

Request For Quote Systems
-------------------------

As a footnote, it is worth mentioning Request For Quote (RFQ) systems. RFQ mechanisms allow for private off-chain pricing with on-chain settlement and portfolio custody, allowing the gap between DeFi and traditional finance to be bridged. To utilize such a system, a centralized API must be queried for a quote, which can then be used on-chain to execute the trade. Off-chain pricing has the advantage of allowing for the use of off-chain information. This can include any of a range of private market-making tactics, including a marginal rate structure (used by Hashflow), or more traditional order book methods used by centralized exchanges \[16\]. Alternatively, high-frequency tuning of AMM equations can be done to achieve better performance than their on-chain counterparts. One example of this is Clipper’s Formula Market Maker (FMM). Clipper’s FMM uses a rapidly updating live price feed as the oracle price in their AMM formula (discussed above), allowing them to shift closer to a CSMM and achieve greater capital efficiency \[17\].

Conclusions
-----------

While the above list of AMMs gives an overview of the space and covers many of the major DEXs that currently hold a high TVL, there remain several significant AMMs that were not covered (not to mention the many more currently being built). Recently, a new class of DeFi projects has even started designing paradigms that aim to generalize to any curve. Primitive Finance makes use of a Replicating Market Maker (RMM), which is able to construct an AMM curve from any of a large range of possible liquidity provider payoffs \[18\]. Shell protocol recently introduced a new AMM called Proteus — it is constructed from conic sections and contains 6 parameters, giving it the ability to be fit to a very wide range of desired curves \[19\]. While out of the scope of this article, there is also an ever-growing selection of protocols that offer financial derivatives, such as options and perpetuals.

With the enormous variation in AMM models and fragmentation in the liquidity used to facilitate trades, DEX aggregators have become an essential part of the DeFi ecosystem. DEX aggregators collect information about all liquidity sources on a given chain (including all AMM models discussed above) and then attempt to find the best combination of actions in order to get the best rate for a given trade. Oftentimes, the optimal route can contain many hops and complex splitting, making it an extremely hard problem to solve. [Odos.xyz](https://odos.xyz/)
 is a new DEX aggregator that is able to search more complex solutions than existing platforms, allowing for atomic multi-input trades and better rates for its users.

\[1\] [https://www.investopedia.com/terms/m/marketmaker.asp](https://www.investopedia.com/terms/m/marketmaker.asp)
  
\[2\] [https://cryptorating.eu/whitepapers/Bancor/bancor\_protocol\_whitepaper\_en.pdf](https://cryptorating.eu/whitepapers/Bancor/bancor_protocol_whitepaper_en.pdf)
  
\[3\] [https://uniswap.org/whitepaper.pdf](https://uniswap.org/whitepaper.pdf)
  
\[4\] [https://jfin-swufe.springeropen.com/articles/10.1186/s40854-021-00314-5#Sec5](https://jfin-swufe.springeropen.com/articles/10.1186/s40854-021-00314-5#Sec5)
  
\[5\] [https://balancer.fi/whitepaper.pdf](https://balancer.fi/whitepaper.pdf)
  
\[6\] [https://en.wikipedia.org/wiki/Bayes%27\_theorem](https://en.wikipedia.org/wiki/Bayes%27_theorem)
  
\[7\] [https://curve.fi/files/stableswap-paper.pdf](https://curve.fi/files/stableswap-paper.pdf)
  
\[8\] [https://pontem.network/posts/everything-you-need-to-know-about-solidly-the-latest-project-by-andre-cronje](https://pontem.network/posts/everything-you-need-to-know-about-solidly-the-latest-project-by-andre-cronje)
  
\[9\] [https://docs.dodoex.io/english/dodo-academy/pmm-overview/the-mathematical-principle-of-pmm](https://docs.dodoex.io/english/dodo-academy/pmm-overview/the-mathematical-principle-of-pmm)
  
\[10\] [https://github.com/shipyard-software/market-making-whitepaper](https://github.com/shipyard-software/market-making-whitepaper)
  
\[11\] [https://classic.curve.fi/files/crypto-pools-paper.pdf](https://classic.curve.fi/files/crypto-pools-paper.pdf)
  
\[12\] [https://research.chain.link/whitepaper-v2.pdf](https://research.chain.link/whitepaper-v2.pdf)
  
\[13\] [https://en.wikipedia.org/wiki/Newton%27s\_method](https://en.wikipedia.org/wiki/Newton%27s_method)
  
\[14\] [https://files.kyber.network/DMM-Feb21.pdf](https://files.kyber.network/DMM-Feb21.pdf)
  
\[15\] [https://uniswap.org/whitepaper-v3.pdf](https://uniswap.org/whitepaper-v3.pdf)
  
\[16\] [https://docs.hashflow.com/hashflow/](https://docs.hashflow.com/hashflow/)
  
\[17\] [https://www.shipyardsoftware.org/post/what-is-a-fmm](https://www.shipyardsoftware.org/post/what-is-a-fmm)
  
\[18\] [https://stanford.edu/~guillean/papers/rmms.pdf](https://stanford.edu/~guillean/papers/rmms.pdf)
  
\[19\] [https://shellprotocol.io/static/Proteus\_AMM\_Engine\_-\_Shell\_v2\_Part\_1.pdf](https://shellprotocol.io/static/Proteus_AMM_Engine_-_Shell_v2_Part_1.pdf)

---

# Semiotic Labs | Podcasts

![Gradient background](/_astro/curved_bg_desktop.ChjV8nJN_1c0N0M.svg) ![Gradient background](/_astro/curved_bg_mobile.DP8czMRd_1b6C73.svg)

*   [![GRTiQ podcast episode 43 Sam Green](/_astro/grtiq43.B_Iriyvr_1mtEQV.webp)\
    \
    #### GRTiQ Podcast 43: Sam Green\
    \
    ##### GRTiQ\
    \
    In Episode 43 of the GRTiQ Podcast, host Nick Hansen interviews Sam Green, Co-Founder and CTO of Semiotic, the fourth core dev team to join The Graph protocol. Sam discusses Semiotic's exceptional expertise in artificial intelligence and cryptography, and provides clear explanations of key concepts like reinforcement learning and zk roll-ups. The insights shared by Sam shed light on not only the future of The Graph and web3 but also the cutting-edge technological innovations being pursued within The Graph protocol.](https://www.grtiq.com/grtiq-podcast-43-sam-green/)
    
*   [![GRTiQ podcast episode 144 Anirudh Patel](/_astro/grtiq74.DQUhTtG2_Z1NPkwF.webp)\
    \
    #### GRTiQ Podcast: 74 Ahmet Ozcan\
    \
    ##### GRTiQ\
    \
    In this episode of the GRTiQ Podcast, host Nick Hansen interviews Ahmet Ozcan, Co-founder and CEO of Semiotic Labs, a Core Dev team contributing to The Graph. The conversation covers Semiotic Labs' recent launch of Odos, an innovative path-finding algorithm designed to optimize order routing for crypto traders. Ahmet also discusses the team's work on The Graph, his transition from a successful career at IBM to entrepreneurship in Web3, and shares his long-term vision for The Graph.](https://www.grtiq.com/grtiq-podcast-74-ahmet-ozcan/)
    
*   [![The main chain. Ahmet Ozcan CEO & co-founder of Semiotic Ai.](/_astro/main_chain_ahmet_fundamental_inovations.CJw444mA_Z2gy0ly.webp)\
    \
    #### The Main Chain: Ahmet Ozcan – Fundamental Innovations\
    \
    ##### Infinity Ventures Crypto (IVC)\
    \
    It's an exciting time right now in crypto - with decentralized exchanges (DEX) still in their infancy, crypto users are evaluating their potential. As DEXs transactions are completely automated, they can be interacted with in a 100% programmatic fashion. which means they are ripe for applications to be built on top of them. That’s where Semiotic AI comes in. Dr. Ahmet Ozcan is the CEO of Semiotic AI, which aims to build automated decision-making tools for decentralized markets on the blockchain. The Semiotic team are the brains behind odos.xyz, a web3 application that allows traders to make large token swaps and trades across disparate DEXs, quickly and efficiently. If one DEX doesn’t have the liquidity needed to complete the transaction, Odos will map trades across DEXs, making large trades effortless on the part of an investor. Semiotic has also been in the news lately for their contributions to The Graph, the indexing and query layer of Web3, for which they received a grant of USD $60 million from the Graph Foundation. Dr. Ozcan chats with host Michael Waitze about how odos works and details its future roadmap. He also discusses why The Graph is important, what it does, and how Semiotic is contributing to The Graph, and to the future of web3, including how their AI solutions simulate how different incentives motivate actors to work either for the good or the detriment of blockchain systems.](https://anchor.fm/themainchain/episodes/Ahmet-Ozcan---Fundamental-Innovations-e1ivpf8)

---

# Automated Query Pricing in The Graph

![Gradient background](/_astro/curved_bg_desktop.ChjV8nJN_1c0N0M.svg) ![Gradient background](/_astro/curved_bg_mobile.DP8czMRd_1b6C73.svg)

_Indexers1​ in The Graph have control over the pricing of the GraphQL queries they serve based on their shape. For this task, The Graph created a domain-specific language called Agora2,3 that maps query shapes to prices in GRT. However, manually populating and updating Agora models for each subgraph is a tedious task, and as a consequence, most indexers default to a static, flat pricing model_.

_To help indexers with pricing in the relative resource cost of serving different query shapes, as well as following the query market price, we are developing AutoAgora, an automation tool that automatically creates and updates Agora models_.

_See our related Devcon 2022 talk here: [https://www.youtube.com/watch?v=LRl9uFfmjEs](https://www.youtube.com/watch?v=LRl9uFfmjEs)
_.

Query relative cost discovery
-----------------------------

This work is based on a few trends that we have observed through analyzing the query traffic.

### Observations on received queries

One of the major problems of GraphQL is its ability to create very computationally expensive queries. In the worst cases, it is even possible to create exponentially hard queries while the query body itself grows only linearly​4​. We can indeed observe this on queries against the Uniswap V2 subgraph, where query execution times span 4 orders of magnitude.

![https://lh5.googleusercontent.com/nf9bUXHNcj2yr2sLTb_ujj077r9fhxexRa1hArEPCtnTZ_jN1AaQl1nJncTm09DL5RClV1tD8cyF7aNbMNv6Z4WOxq4jqfg99otERczRlybWKiiL16NIYu9jMAxaCa5s2FV8KznbV7sl3BP1bDTO9g](https://lh5.googleusercontent.com/nf9bUXHNcj2yr2sLTb_ujj077r9fhxexRa1hArEPCtnTZ_jN1AaQl1nJncTm09DL5RClV1tD8cyF7aNbMNv6Z4WOxq4jqfg99otERczRlybWKiiL16NIYu9jMAxaCa5s2FV8KznbV7sl3BP1bDTO9g)

Simultaneously, we observed that a vast majority of the queries are based on a small number of query shapes. This is to be expected as most queries are generated programmatically from web frontends, such as the official [Uniswap v2 statistics website](https://v2.info.uniswap.org/home)
.

![Top 50 frequency](/_astro/image-4.CnirwRMj_ZgOOcr.webp)

Frequency of top 50 query shapes (Uniswap v2).

Based on the observations above, a reasonable solution for relative query pricing that would cover most of the value being served by an indexer is to estimate the relative cost of the most frequent queries for each subgraph.

### AutoAgora logging and relative costing infrastructure

In this section, we will discuss the implementation of the automated _relative query pricing_ discovery feature of AutoAgora, and how it fits in the indexer’s stack.  
All monetized queries coming from the Gateway pass through the `indexer-service`. We modified it to output detailed query logs (subgraph hash, query, variables, GRT fees paid, execution time). These logs are then processed to normalize the queries, separate all query values from the query shapes. Shapes are deduplicated and stored in a PostgreSQL table, while the rest of the log information is appended to a logs table and references its query shape through its blake2 hash.

![Block diagram of components](/_astro/AutoAgora-relative-pricing-block-diagram-2-1024x685.7A6jxI9x_NIWoL.webp)

Block diagram of the components relevant to automated query relative costing.

![Block diagram of AutoAgora](/_astro/Untitled-drawing-2-1-1024x801.B3gZeN9D_oa8M1.webp)

Block diagram of the AutoAgora Processor.

Here’s a sample from the query logs table:

    autoagora=# select * from query_logs order by random() limit 3;
    -[ RECORD 1 ]---+-----------------------------------------------------
    id              | 723883ae-edbb-4ed5-81ad-f06dcc9195e7
    subgraph        | QmXDs3UikxJBLwPztj3tLhyFuaV51tEftjSY1AHjBUz3CH
    query_hash      | \x8e6c32fce280c7bb43081b0e0173b0b1
    timestamp       | 2022-06-17 07:04:00.581+00
    query_time_ms   | 56
    query_variables | ["0x8289baf639318d9076616d7d231cbfc4f8fdc9bf628b1d43.\
                    |.e5ebb8e4b125a68d", 100, "preparedTimestamp", "desc",.\
                    |. 0, "0x7aa9530f8df705d07a048d8508236e9cdf2a8f1331578.\
                    |.129c918246766962ae1"]
    -[ RECORD 2 ]---+-----------------------------------------------------
    id              | 50546eda-51e5-43ed-8829-a66a9642355b
    subgraph        | QmXDs3UikxJBLwPztj3tLhyFuaV51tEftjSY1AHjBUz3CH
    query_hash      | \x35d9bb4ecfd091d6aa75659f9a33a06b
    timestamp       | 2022-06-17 03:32:35.185+00
    query_time_ms   | 31
    query_variables | ["0x8f80d695bb4f3ac71f7392f930f5f91529a54df1ffc40c06.\
                    |.deb06492af7260d6", "amount", "desc"]
    -[ RECORD 3 ]---+-----------------------------------------------------
    id              | 4505c23a-bbab-49a0-a58c-49428d51380d
    subgraph        | QmXDs3UikxJBLwPztj3tLhyFuaV51tEftjSY1AHjBUz3CH
    query_hash      | \x7efa66f59bd19e1c501b091b931a6069
    timestamp       | 2022-06-17 03:24:37.586+00
    query_time_ms   | 14
    query_variables | ["0x723bfdcd48cf2bdb056a2d23003e7d089596056cea9f1799.\
                    |.68f503661d349fb9", "preparedBlockNumber", "desc", "1.\
                    |.", "Prepared", "0xcada61e60e8c797c61f0a34468224e782f.\
                    |.8dfb8d"]

And the corresponding query shapes (also called skeletons in AutoAgora):

    autoagora=# select * from query_skeletons where hash in ('\x8e6c32fce280c7bb43081b0e0173b0b1', '\x35d9bb4ecfd091d6aa75659f9a33a06b', '\x7efa66f59bd19e1c501b091b931a6069');
    -[ RECORD 1 ]---------------------------------------------------------
    hash  | \x7efa66f59bd19e1c501b091b931a6069
    query | query($_0:Bytes$_1:Transaction_orderBy$_2:OrderDirection$_3:Bi.
          |.gInt$_4:TransactionStatus$_5:String){transactions(block:{hash:.
          |.$_0}orderBy:$_1 orderDirection:$_2 where:{sendingChainId:$_3 s.
          |.tatus:$_4 user:$_5}){amount bidSignature callDataHash callTo c.
          |.ancelCaller cancelTransactionHash chainId encodedBid encrypted.
          |.CallData expiry fulfillCaller fulfillTransactionHash id initia.
          |.tor prepareCaller prepareTransactionHash preparedBlockNumber p.
          |.reparedTimestamp receivingAddress receivingAssetId receivingCh.
          |.ainId receivingChainTxManagerAddress router{id}sendingAssetId .
          |.sendingChainFallback sendingChainId status transactionId user{.
          |.id}}}
    -[ RECORD 2 ]---------------------------------------------------------
    hash  | \x35d9bb4ecfd091d6aa75659f9a33a06b
    query | query($_0:Bytes$_1:AssetBalance_orderBy$_2:OrderDirection){ass.
          |.etBalances(block:{hash:$_0}orderBy:$_1 orderDirection:$_2){amo.
          |.unt assetId id router{id}}}
    -[ RECORD 3 ]---------------------------------------------------------
    hash  | \x8e6c32fce280c7bb43081b0e0173b0b1
    query | query($_0:Bytes$_1:Int$_2:Transaction_orderBy$_3:OrderDirectio.
          |.n$_4:Int$_5:Bytes){transactions(block:{hash:$_0}first:$_1 orde.
          |.rBy:$_2 orderDirection:$_3 skip:$_4 where:{transactionId:$_5}).
          |.{amount bidSignature callData callDataHash callTo cancelCaller.
          |. cancelMeta cancelTimestamp cancelTransactionHash chainId enco.
          |.dedBid encryptedCallData expiry externalCallIsContract externa.
          |.lCallReturnData externalCallSuccess fulfillCaller fulfillMeta .
          |.fulfillTimestamp fulfillTransactionHash id initiator prepareCa.
          |.ller prepareMeta prepareTransactionHash preparedBlockNumber pr.
          |.eparedTimestamp receivingAddress receivingAssetId receivingCha.
          |.inId receivingChainTxManagerAddress relayerFee router{id}sendi.
          |.ngAssetId sendingChainFallback sendingChainId signature status.
          |. transactionId user{id}}}

Relative pricing is generated periodically from the logs database. For each subgraph, query shapes that have seen more than a threshold number of queries are selected to be included in the Agora model. For each shape, the average query execution time is computed and used as the query shape cost factor in the pricing model.

![Block diagram of relative price costing](/_astro/AutoAgora-diagram-1024x432.DbYJ5iQg_14iq6R.webp)

Block diagram of AutoAgora (relative price costing only).

Here is an example of an Agora model generated by AutoAgora for a subgraph:

    # count:        1303
    # min time:     30
    # max time:     12994
    # avg time:     4978.261703760552
    # stddev time:  3374.180130690477
    query {
      erc721Contract(block: {hash: $_0}, id: $_1) {
        transfers(
          first: $_2,
          orderBy: $_3
          orderDirection: $_4
          where: {id_gt: $_5}
        ) {
          contract {
            id
            name
          }
          from {
            id
          }
          id
          timestamp
          to {
            id
          }
          token {
            id
            identifier
            uri
          }
          transaction {
            id
          }
        }
      }
    } => 4978.261703760552 * $GLOBAL_COST_MULTIPLIER;
    
    # count:        830
    # min time:     4
    # max time:     1227
    # avg time:     44.67831325301205
    # stddev time:  82.78835711833459
    query {
      account(block: {hash: $_0}, id: $_1) {
        tokens: ERC721tokens(
          first: $_2
          orderBy: $_3
          orderDirection: $_4
          where: {contract_in: $_5, identifier_gt: $_6}
        ) {
          id
          identifier
        }
        id
      }
    } => 44.67831325301205 * $GLOBAL_COST_MULTIPLIER;
    
    default => $DEFAULT_COST * $GLOBAL_COST_MULTIPLIER;

You may have noticed the `$GLOBAL_COST_MULTIPLIER` variable. It is there to adjust the pricing of all the query shapes to the market price, which is the subject of the next section.

Query market price discovery
----------------------------

### The query market

In The Graph, each subgraph, on each gateway (multiple geographies) defines a query market, on which end-consumers compete to buy queries (currently automated through user-defined budgets and quality preferences), and indexers compete to serve queries based on their Agora-defined prices, and their quality of service.  
The indexers’ main objective is to maximize profits. For now we simplified the problem down to revenue maximization, where for each subgraph, the query market is treated as a black box that takes a `$GLOBAL_COST_MULTIPLIER` input value, and outputs the GRT per second earned.

![Query market behavior](/_astro/Untitled-1024x393.CAFdYmqY_ZHvz2c.webp)

Query market behavior as observed by indexers.

The objective of the AutoAgora price discovery is to find the `$GLOBAL_COST_MULTIPLIER` that optimizes the revenue rate, as well as continuously adjust it to track markets fluctuations.

### AutoAgora absolute price discovery

To find and continuously track the optimal price point requires continuously probing the market “black box” at various price points. However, doing so means that a significant amount of time will be spent serving queries at unfavorable price points. The balance between exploration and exploitation is a reinforcement learning problem that has been extensively studied through the multi-armed bandit problem​5​.  
As such, we are mapping the price discovery problem as a continuum-armed bandit problem. The policy is modeled as a Gaussian probability distribution over `$GLOBAL_COST_MULTIPLIER` values, from which values are continuously sampled. The mean and standard deviation of the Gaussian policy are continuously optimized through gradient descent using the Adam​ optimizer.

[https://www.youtube.com/watch?v=-YHpYyZ3bYI](https://www.youtube.com/watch?v=-YHpYyZ3bYI)

Overlay of a simulated query rate curve with the Gaussian model.

We tested the AutoAgora price discovery on our own indexer (`semiotic-indexer.eth`) on mainnet with great success. The plots below are extracted from our own indexer’s Grafana boards, and shows the convergence to, and tracking of the market price over time (“Bandit mean” and “standard deviation”), as well as the resulting increase in revenue rate (“GRT / second per subgraph”) on the UMA subgraph.

![https://lh4.googleusercontent.com/fF12WmycHKDgV20J1rPx0YK4Y2JhQN_K1tn-8yyTHYHQCgezyUnAXIvcXY_m9LPXY30oJVo0bCGLj1M8rQ-GQPtQyW4fixcAaraMxHQqJiOdYRMCgazebKMy6NpA-6Gy71YucFbQXtw9c9Vd7mdmZQ](https://lh4.googleusercontent.com/fF12WmycHKDgV20J1rPx0YK4Y2JhQN_K1tn-8yyTHYHQCgezyUnAXIvcXY_m9LPXY30oJVo0bCGLj1M8rQ-GQPtQyW4fixcAaraMxHQqJiOdYRMCgazebKMy6NpA-6Gy71YucFbQXtw9c9Vd7mdmZQ)

Limitations, Conclusions
------------------------

Our goal with AutoAgora is to help build sustainable, efficient query markets on The Graph, while also lowering the human operation costs.

While we have shown a successful operation on our own indexer, we are still working on tackling current shortcomings of AutoAgora:

*   The relative cost models based only on average query shape execution time are too simplistic. Other hardware impact measurements could be added, so that indexers can express the relative cost of their hardware commodities (CPU, RAM, I/O, Network, etc.).
    
*   The initial convergence speed of the market price bandit model is quite slow, taking many hours. One of the main limitations currently is the speed at which the gateways are updating the indexer’s cost models.
    
*   The price bandit training is unstable on subgraphs with low query volumes. The solution being currently implemented is a simple rule deactivating the training, and defaulting to an indexer-supplied default price multiplier. Such a solution is acceptable, since the revenue expected from a low-query subgraph is expected to be negligeable.
    
*   The price bandit training stability is unproven when multiple of those agents are competing against each other on the same market. Thus we are working on a simulation environment to stress-test this particular scenario.
    

Therefore, the state of AutoAgora on the date of publication of this piece is highly experimental, and we do not recommend indexers to use the software in production.  
The AutoAgora components are open source under the Apache-2 license, and available at: [https://gitlab.com/semiotic-ai/the-graph/autoagora](https://gitlab.com/semiotic-ai/the-graph/autoagora)

### References

1.  The Graph Foundation. Indexer. The Graph. Published 2022. Accessed June 17, 2022. [https://thegraph.com/docs/en/indexing/](https://thegraph.com/docs/en/indexing/)
    
2.  Zachary B. Cost Model Workshop. Youtube. Published May 24, 2021. Accessed June 17, 2022. [https://youtu.be/s7zNzgiL4z4](https://youtu.be/s7zNzgiL4z4)
    
3.  Agora contributors. graphprotocol/agora. GitHub. Published 2022. Accessed 2022. [https://github.com/graphprotocol/agora](https://github.com/graphprotocol/agora)
    
4.  Wittern E, Cha A, Davis JC, Baudart G, Mandel L. An Empirical Study of GraphQL Schemas. _Service-Oriented Computing_. Published online 2019:3-19. doi:[10.1007/978-3-030-33702-5\_1](https://doi.org/10.1007/978-3-030-33702-5_1)
    
5.  Slivkins A. Introduction to Multi-Armed Bandits. _FNT in Machine Learning_. Published online 2019:1-286. doi:[10.1561/2200000068](https://doi.org/10.1561/2200000068)
    
6.  Kingma DP, Ba J. Adam: A Method for Stochastic Optimization. Published online 2014. doi:[10.48550/ARXIV.1412.6980](https://doi.org/10.48550/ARXIV.1412.6980)

---

# Indexer Allocation Optimization: Part I

![Gradient background](/_astro/curved_bg_desktop.ChjV8nJN_1c0N0M.svg) ![Gradient background](/_astro/curved_bg_mobile.DP8czMRd_1b6C73.svg)

### TL;DR

Indexers within [The Graph Protocol](https://thegraph.com/en/)
 are rewarded via an indexing reward. How can indexers optimise their allocations so as to maximise the reward they receive? In this blog post we formalise the problem in terms of a reward function and use convex optimization to find a solution.

### Overview

The Graph’s goal is to decentralize the API and query layer of web3, enabling users to query blockchain data without a centralized service provider. Part of that process is to have people who can serve queries to consumers. These people are called “indexers.” In this blog post, we focus on the problem of how an indexer should choose which data to index, and, thereby, which data to serve. The outline of this blog post is as follows:

*   In the first section, we dive deeper into the allocation problem.
*   Next, we formalise the problem and express it as a primal form convex optimisation problem.
*   Subsequently, we show how to use convex optimisation for finding an optimal solution.
*   Finally, we present some results demonstrating the benefit of our approach by comparing the reward received when manually allocating to the reward received when using the optimal allocation for an existing indexer.

For more details regarding The Graph, indexers, and the other roles present on The Graph, please refer to the excellent blog posts by Brandon Ramirez: [Part I](https://thegraph.com/blog/the-graph-network-in-depth-part-1)
 and [Part II](https://thegraph.com/blog/the-graph-network-in-depth-part-2)
.\[3\]

Please note that this blog post provides only essential details of the solution. A complete discussion can be found in the associated yellowpaper.\[1\]

### The Indexer Allocation Problem

To understand the problem of indexer allocation optimization, we need to understand The Graph from an indexer’s perspectives. Subgraphs are collections of data extracted from blockchain transactions. In order to serve data to consumers, indexers must index subgraphs, which essentially involves them storing the data from a subgraph into a Postgres database. Obviously, this takes time, effort, and compute, so indexers want to be selective with which subgraphs they index. How then does an indexer know which subgraph(s) to index?

![Indexers allocate to subgraphs](/_astro/graph.CsUEFksy_Z1UXDsH.webp)

Indexers allocate capital (called stake) to subgraphs. How should indexer D allocate its stake?

As a rule, the more a subgraph is queried, the more indexers want to be indexing a subgraph. This is because they’d be able to earn more money from query fees. Predicting which subgraphs will have high query volumes is a tough problem unto itself, so we don’t force indexers to try to do this on top of their existing responsibilities. Instead, we rely on curation. The outputs of the curation process are subgraph signals, which should be roughly proportional to the volume of queries on the corresponding subgraphs. In other words, the higher the signal on a subgraph, the higher we expect the query volume on that subgraph to be. Thus, when deciding which subgraphs to index, indexers who want to maximise their indexing reward ought to allocate to subgraphs with the greatest signal.

There’s still one more consideration to make. Say you allocate 30 GRT on subgraph A, and I allocate 10 GRT on that same subgraph. Should we be rewarded the same amount? We would be if we were rewarded only as per subgraph signal, but that would be unfair. In the next section, we’ll make these qualitative statements about signal and fairness into a quantitative equation over which indexers can optimise.

The Indexing Reward Function
----------------------------

Let’s now formalise the previous section into the indexing reward, which is paid out of the new token issuance Φ\\PhiΦ. An indexer i∈Ii \\in \\mathcal{I}i∈I has some stake σi\\sigma\_iσi​. The indexer must decide how much of its stake to allocate to each subgraph j∈Sj \\in \\mathcal{S}j∈S. Each subgraph has some signal ψj\\psi\_jψj​ on it, which should have a strong, positive correlation with the query volume on subgraph jjj. An indexer iii‘s allocation to subgraph jjj is defined as Ωij\\Omega\_{ij}Ωij​. We can use this to construct an allocation matrix across all indexers Ω\\OmegaΩ, in which the iiith indexer’s allocation to the jjjth subgraph corresponds to Ωij\\Omega\_{ij}Ωij​. The indexing reward is then given as:

Ri(Ω,Ψ,Φ)\=∑j∈SΩij∑k∈IΩkj⋅Φψj∑l∈Sψl\\begin{equation\*} R\_{i}(\\Omega, \\Psi, \\Phi) = \\sum\_{j \\in \\mathcal{S}}\\frac{\\Omega\_{ij}}{\\sum\_{k \\in \\mathcal{I}} \\Omega\_{kj}} \\cdot\\Phi\\frac{\\psi\_j}{\\sum\_{l \\in \\mathcal{S}} \\psi\_l} \\end{equation\*}Ri​(Ω,Ψ,Φ)\=j∈S∑​∑k∈I​Ωkj​Ωij​​⋅Φ∑l∈S​ψl​ψj​​​

Intuition
---------

Let’s consider the following scenario. Say we have two indexers trying to allocate to two subgraphs. We’re playing the game from indexer 1’s perspective, so for us, indexer 2 and the signals on the two subgraphs are out of our control. All we can do is modify our allocation. Let’s first consider how subgraph signal changes how we should allocate.

![Stake versus signal on one subgraph](/_astro/reward_v_signal-2.CSg5HvHh_Z22gIxA.webp)

This figure is a contour plot showing how much we should allocate to subgraph 1 based on how much signal there is on subgraph 1.

Let’s read the above plot from an indexer’s perspective. Say we know that the signal on subgraph 1 is 0. In that case, we should read across from 0 on the y-axis to find the point on the x-axis that is brightest. In this case, we’d want to allocate 0 to subgraph 1. Similarly, let’s say subgraph 1’s signal is 5. Reading across the x-axis, we’d want to allocate roughly 5 to subgraph 1. Generally speaking here, the intuition is that the amount we should allocate to a subgraph is a linear function of the signal on that subgraph.

However, if we only look at subgraph signal, we’re missing an important part of the indexing reward - the amount other indexers have allocated to a subgraph. Let’s now hold the subgraph’s signal constant and see how the amount we should allocate will vary with how much other indexers have allocated to this subgraph.

Generally speaking, this isn’t anywhere as near as nice a relationship as we had between signal and our allocation. Let’s start to get some intuition by looking at the bright spot in the top-right corner of the plot. In this case, all other indexers have allocated on subgraph 1. No one is allocated on subgraph 2. This is great for us! As long as we put even 0.1 GRT on subgraph 2, we’ll get all of the reward that is available on subgraph 2. This means that our best strategy is to put a fractional amount of GRT on subgraph 2 and to maximise the amount we put on subgraph 1. This way, we get all of the reward available on subgraph 2, and we also maximise how much of the reward available on subgraph 1 that we receive.

This exemplifies how we should allocate with respect to the allocations of other indexers. We want to try to place just enough stake to capture the maximum reward we can receive off a given subgraph. In other words, when the marginal reward we’d receive for increasing our stake on subgraph 1 is less than the marginal reward we’d receive for increasing our stake on subgraph 2, we should allocate to subgraph 2.

In reality, we don’t have nearly as much freedom as in this diagram when choosing how to allocate because there is often an order-of-magnitude difference between existing allocations on a subgraph and the amount of stake that an indexer has, but this example is illustrative for intuition.

Optimizing The Indexing Reward
------------------------------

The reward function enables us to formulate the optimization problem as:

min⁡Ωi−∑j∈SΩij∑k∈IΩkj⋅Φψj∑l∈Sψls.t.ωk≥0∀ ωk∈Ωi∑j∈SΩij\=σi\\begin{align\*} \\min\_{\\Omega\_i} \\quad & -\\sum\_{j \\in \\mathcal{S}}\\frac{\\Omega\_{ij}}{\\sum\_{k \\in \\mathcal{I}} \\Omega\_{kj}}\\cdot\\Phi\\frac{\\psi\_j}{\\sum\_{l \\in \\mathcal{S}} \\psi\_l} \\\\ \\textrm{s.t.} \\quad & \\omega\_k \\geq 0 \\quad \\forall \\ \\omega\_k\\in\\Omega\_i \\\\ \\quad & \\sum\_{j\\in\\mathcal{S}}\\Omega\_{ij} = \\sigma\_i \\end{align\*}Ωi​min​s.t.​−j∈S∑​∑k∈I​Ωkj​Ωij​​⋅Φ∑l∈S​ψl​ψj​​ωk​≥0∀ ωk​∈Ωi​j∈S∑​Ωij​\=σi​​

where Ωi\\Omega\_iΩi​ are the allocations of indexer iii.

In plain English, we want to minimise the negative indexing reward (equivalent to maximise the indexing reward) subject to the constraints that the amount we allocate must sum to the amount of stake we have to allocate and that we can’t allocate a negative amount to any subgraph. Why the negative indexing reward?

![Indexing reward versus stake](/_astro/concave.BeZJvXN3_ZLgl80.webp)

Holding the network constant, the indexing reward is a concave function of an indexer’s allocations. The three plots above represent three different network states.

Take a look at the plot above. In it, for each trace, we hold the network constant, meaning we hold the signal and the allocations of all other indexers constant, and we sweep the allocations of the indexer we are optimizing. As you can see, the indexing reward is a concave function of our allocations. Since a convex function is just the negative of a concave function, we can use convex optimization to solve for the optimal allocation!

We leave a more complete discussion of how we solve this optimization problem to our yellowpaper, but, at a high level, we consider the dual of allocation optimization problem:

max⁡λ,νmin⁡Ωi−∑j∈SΩij∑k∈IΩkj⋅Φψj∑l∈Sψl−∑j∈SλjΩij+ν(∑j∈S\[Ωij\]−σi)s.t.λ≥0\\begin{align\*} \\max\_{\\lambda, \\nu} \\min\_{\\Omega\_i} \\quad & -\\sum\_{j \\in \\mathcal{S}}\\frac{\\Omega\_{ij}}{\\sum\_{k \\in \\mathcal{I}} \\Omega\_{kj}} \\cdot \\Phi\\frac{\\psi\_j}{\\sum\_{l \\in \\mathcal{S}} \\psi\_l} - \\sum\_{j\\in\\mathcal{S}}\\lambda\_j\\Omega\_{ij} + \\nu \\left(\\sum\_{j\\in\\mathcal{S}} \\left\[\\Omega\_{ij}\\right\] - \\sigma\_i\\right) \\\\ \\textrm{s.t.} \\quad & \\lambda \\geq 0 \\end{align\*}λ,νmax​Ωi​min​s.t.​−j∈S∑​∑k∈I​Ωkj​Ωij​​⋅Φ∑l∈S​ψl​ψj​​−j∈S∑​λj​Ωij​+ν​j∈S∑​\[Ωij​\]−σi​​λ≥0​

We then use the Karush-Kuhn-Tucker conditions to find an analytic solution for the dual variable ν\\nuν.

∑j∈SΩij\=max⁡(0,ψjγjν−γj)\=σi\\begin{equation\*} \\sum\_{j\\in\\mathcal{S}}\\Omega\_{ij}=\\max(0,\\sqrt{\\frac{\\psi\_j\\gamma\_j}{\\nu}} - \\gamma\_j) = \\sigma\_i \\end{equation\*}j∈S∑​Ωij​\=max(0,νψj​γj​​​−γj​)\=σi​​

where γj\=∑k∈I\\iΩkj\\gamma\_j = \\sum\_{k\\in\\mathcal{I}\\backslash i}\\Omega\_{kj}γj​\=∑k∈I\\i​Ωkj​. With ν\\nuν in hand, we can then plug it in to solve for our optimal allocations using

Ωij\=max⁡(0,ψjγjν−γj)\\begin{equation\*} \\Omega\_{ij}=\\max(0,\\sqrt{\\frac{\\psi\_j\\gamma\_j}{\\nu}} - \\gamma\_j) \\end{equation\*}Ωij​\=max(0,νψj​γj​​​−γj​)​

### Results

Running our optimiser for an existing indexer, we see an improvement in indexing rewards received from 208,608.87 GRT to 240,739.72 GRT. That’s an improvement of 15%!

![Indexer's current allocations](/_astro/current_allocations.h8bESTIw_Z2j1mDB.webp)

The indexer’s current allocations.

![Indexer's optimal allocations](/_astro/optimal_allocations.KZ_svSM9_Zcf8hd.webp)

The indexer’s optimal allocations

It may be visually obvious to you from the above plots, but the optimal allocation allocates to way more subgraphs than what this indexer has done by hand. To quantify this, the indexer has currently allocated to 70 subgraphs. The optimal allocation tool allocates to 144 subgraphs.

### Conclusion

The above problem formulation and optimization focus on a simplified version of the problem. One thing we haven’t yet considered is that indexers have to pay gas to allocate to subgraphs. Furthermore, we mentioned earlier that indexers want to be selective in how they allocate based on compute or other resource constraints. These will turn our nice, convex problem into a non-convex problem. The next post in the series will discuss how we can optimise this non-convex problem. Stay tuned!

### Further Reading

*   \[1\] Yellowpaper (arXiv link to come)
*   \[2\] [Convex Optimization](https://web.stanford.edu/~boyd/cvxbook/)
     by Boyd and Vandenberghe
*   \[3\] Brandon Ramirez’s blog posts about The Graph: [Part I](https://thegraph.com/blog/the-graph-network-in-depth-part-1)
     and [Part II](https://thegraph.com/blog/the-graph-network-in-depth-part-2)

---

# Semiotic Labs | Videos

![Gradient background](/_astro/curved_bg_desktop.ChjV8nJN_1c0N0M.svg) ![Gradient background](/_astro/curved_bg_mobile.DP8czMRd_1b6C73.svg)

*   [*   blockchain infrastructure\
    *   ai\
    \
    #### Bringing high-performance SQL queries to The Graph\
    \
    At Datapalooza 2023, Sam presented Semiotic's latest research & practical applications of AI & data analytics in web3 and shared details about bringing high-performance SQL queries to The Graph!](https://www.youtube.com/watch?v=eTKzHkv5E40)
    
*   [*   blockchain infrastructure\
    *   cryptography\
    \
    #### The Graph and Preserving Historical Data After EIP-4444 by Seve Sisneros\
    \
    Severiano Sisneros presents "The Graph and Preserving Historical Data After EIP-4444" at Datapalooza 2023. This talk, held during Devconnect in Istanbul, Turkey, explores the vital role of The Graph in the context of Ethereum's EIP-4444. Gain insights into the challenges and strategies for preserving historical data in the web3 space.](https://www.youtube.com/watch?v=AwU6gkXBxKM)
    
*   [*   blockchain infrastructure\
    *   ai\
    \
    #### Automated Cost Modeling in The Graph\
    \
    Alexis Asseman presents on our reinforcement learning agent used to automatically set prices for Indexers in The Graph protocol.](https://www.youtube.com/watch?v=DrdkX8FibHI)

---

# Semiotic Labs | Videos

![Gradient background](/_astro/curved_bg_desktop.ChjV8nJN_1c0N0M.svg) ![Gradient background](/_astro/curved_bg_mobile.DP8czMRd_1b6C73.svg)

*   [*   blockchain infrastructure\
    *   cryptography\
    \
    #### The Graph and Preserving Historical Data After EIP-4444 by Seve Sisneros\
    \
    Severiano Sisneros presents "The Graph and Preserving Historical Data After EIP-4444" at Datapalooza 2023. This talk, held during Devconnect in Istanbul, Turkey, explores the vital role of The Graph in the context of Ethereum's EIP-4444. Gain insights into the challenges and strategies for preserving historical data in the web3 space.](https://www.youtube.com/watch?v=AwU6gkXBxKM)
    
*   [*   cryptography\
    \
    #### Devcon 2022: A SNARKs Tale: A Story of Building SNARK Solutions on Mainnet\
    \
    Severiano Sisneros, Semiotic's Senior Cryptographer, tells the story of deploying a real-world solution using SNARKs as a core primitive; highlighting many cutting-edge SNARKs and their limitations in the hope to identify opportunities for the community to make Ethereum more SNARK friendly, creating a diverse ecosystem where SNARKs built upon a variety of unique primitives can thrive.](https://www.youtube.com/watch?v=GnYM9yxIRSs)

---

# Semiotic Labs | Videos

![Gradient background](/_astro/curved_bg_desktop.ChjV8nJN_1c0N0M.svg) ![Gradient background](/_astro/curved_bg_mobile.DP8czMRd_1b6C73.svg)

*   [*   blockchain infrastructure\
    *   ai\
    \
    #### Bringing high-performance SQL queries to The Graph\
    \
    At Datapalooza 2023, Sam presented Semiotic's latest research & practical applications of AI & data analytics in web3 and shared details about bringing high-performance SQL queries to The Graph!](https://www.youtube.com/watch?v=eTKzHkv5E40)
    
*   [*   ai\
    \
    #### Devcon 2022: Reinforcement Learning for Query Pricing in The Graph\
    \
    Indexers​​ in The Graph protocol use a DSL called Agora​ to map query shapes to prices. However, manually populating and updating Agora models for each query is a tedious task, and, as a consequence, most indexers default to a flat pricing model. We have created and deployed reinforcement learning agents for Indexers to automatically compete on pricing. This talk, given by Dr. Tomasz Kornuta, is focused on our development process and our study of the market effects of multiple competing pricing agents.](https://www.youtube.com/watch?v=LRl9uFfmjEs)
    
*   [*   ai\
    \
    #### Special Graph Hack Episode: Automated Allocation Optimizer\
    \
    Indexers within The Graph Protocol are rewarded via an indexing reward. How can indexers optimize their allocations so as to maximize the reward they receive? In this video, Anirudh Patel and Hope Yen formalize that question in terms of a reward function and explain how convex optimization can be used to find a solution. They then demonstrate a new tool that automates the solution to the allocation optimization problem.](https://www.youtube.com/watch?v=J6a4RarYYFI)
    
*   [*   blockchain infrastructure\
    *   ai\
    \
    #### Automated Cost Modeling in The Graph\
    \
    Alexis Asseman presents on our reinforcement learning agent used to automatically set prices for Indexers in The Graph protocol.](https://www.youtube.com/watch?v=DrdkX8FibHI)
    
*   [*   ai\
    \
    #### Introduction to the Workshop on Incentive Mechanism Validation: Sam Green, Semiotic Labs\
    \
    Presented at the WIMV 2022 Devconnect workshop. For future workshops, check out https://wimv.dev or follow https://twitter.com/wimvdev.](https://www.youtube.com/watch?v=RQULcaKWCD8)
    
*   [*   ai\
    \
    #### Simplifying Mechanism Analysis With Software Design: Anirudh Patel, Semiotic Labs\
    \
    Presented at the WIMV 2022 Devconnect workshop. For future workshops, check out https://wimv.dev or follow https://twitter.com/wimvdev.](https://www.youtube.com/watch?v=rF4Gr0CS5Gc)

---

# Semiotic Labs | Videos

![Gradient background](/_astro/curved_bg_desktop.ChjV8nJN_1c0N0M.svg) ![Gradient background](/_astro/curved_bg_mobile.DP8czMRd_1b6C73.svg)

*   [*   defi\
    \
    #### Devcon 2022: An Overview of AMM Mechanisms\
    \
    Matt Deible gives an exhaustive overview of the different AMM algorithms currently deployed on major distributed ledgers, as well as the underlying intuition behind their design. Building up from the basic principles of AMM design, the talk then covers the algorithmic mechanisms used in the various different algorithms including Constant Sum, Constant Product (Uniswap V2), Uniswap V3, KyberSwap, StableSwap (Curve), CryptoSwap (Curve V2), Solidly Stable pairs, Clipper, Dodo, and RFQ systems.](https://www.youtube.com/watch?v=KxuyHfmLHP0)
    
*   [*   defi\
    \
    #### ODOS: The Optimal DEX Aggregator for Token Swaps\
    \
    Odos is a Smart Order Routing (SOR) solution that leverages a patented Automated Market Maker (AMM) path finding algorithm to aggregate decentralized exchanges (DEX) and find optimal routes for token swaps. By traversing a large universe of connecting token pairs and performing complex order splits, Odos is the ultimate choice for institutional and retail traders. Odos is the first aggregator to introduce a multi-asset input feature that enables users to swap from several tokens into one asset in a single atomic transaction.](https://www.youtube.com/watch?v=qC8Ry-YOZZs)
    
*   [*   defi\
    \
    #### ODOS Intro: The Optimal Order Routing Solution for Cryptocurrency Swaps\
    \
    Odos is a Smart Order Routing (SOR) solution that leverages a patented Automated Market Maker (AMM) path finding algorithm to aggregate decentralized exchanges (DEX) and find optimal routes for token swaps. By traversing a large universe of connecting token pairs and performing complex order splits, Odos is the ultimate choice for institutional and retail traders. Odos is the first aggregator to introduce a multi-asset input feature that enables users to swap from several tokens into one asset in a single atomic transaction.](https://www.youtube.com/watch?v=XMqzRN1qQ-I)

---

# Semiotic Labs | Podcasts

![Gradient background](/_astro/curved_bg_desktop.ChjV8nJN_1c0N0M.svg) ![Gradient background](/_astro/curved_bg_mobile.DP8czMRd_1b6C73.svg)

*   [![GRTiQ podcast special AI Services Whitepaper Anirudh Patel](/_astro/grtiq_ai_whitepaper_special_anirudh.D0JuTZvu_Z1zGT2m.webp)\
    \
    #### GRTiQ Podcast Special Release: Anirudh Patel on The Graph as AI Infrastructure\
    \
    ##### GRTiQ\
    \
    On May 28, 2024, Semiotic Labs, a core developer team working on The Graph, released an exciting new white paper titled “The Graph as AI Infrastructure.” This paper details the upcoming launch of two new AI services to be built on The Graph: the Inference Service and the Agent Service. The paper also explores what makes The Graph the best place to service the convergence of web3, blockchain data, and AI. In addition to all this, Semiotic Labs also announced a two-week public demo of their ChatGPT-like AI product called Agentc, which was built using The Graph. To help us better understand the details of the white paper, understand these new AI services, and comprehend the implications for the future of The Graph, I’ve invited Anirudh Patel, or Ani, back onto the podcast. Ani’s been on the podcast several times now, but he’s back for this special release and will answer these questions and more!](https://www.grtiq.com/grtiq-podcast-special-release-anirudh-patel-on-the-graph-as-ai-infrastructure/)
    
*   [![GRTiQ podcast episode 144 Anirudh Patel](/_astro/grtiq144.CP9oaoVT_1YWa9U.webp)\
    \
    #### GRTiQ Podcast: 144 Anirudh Patel\
    \
    ##### GRTiQ\
    \
    In a captivating edition of the GRTiQ podcast, Nick Hansen welcomes back Aniurdh “Ani” Patel, a Senior Research Scientist at Semiotic Labs, for an insightful and comprehensive interview. Following an impactful discussion in a previous episode, Ani shares his extensive journey through AI, web3, and The Graph, infused with his passion for travel and innovation. The discussion delves deep into the intricacies of web3, crypto, and Ani's introduction to Semiotic Labs via Sandia Labs. The conversation has insights into the world of artificial intelligence and machine learning. As a highlight, Ani unveils the exciting launch of our LLM-based query testbed, AgentC.](https://www.grtiq.com/grtiq-podcast-144-anirudh-patel/)
    
*   [![GRTiQ podcast episode 144 Anirudh Patel](/_astro/grtiq106.7lm8z8JN_1L3X2d.webp)\
    \
    #### GRTiQ Podcast: 106 AI and Crypto\
    \
    ##### GRTiQ\
    \
    In this special edition of the GRTiQ Podcast, host Nick Hansen is joined by three members of Semiotic Labs, a Core Dev team on The Graph with extensive expertise in artificial intelligence. Anirudh Patel, Sam Green, and Tomasz Kornuta discuss the fundamentals of AI, the emergence of ChatGPT, and its functionality. The conversation then turns to AI's role in the crypto sphere, examining both recent hype and genuine applications. Finally, they delve into Semiotic Labs' implementation of AI within The Graph and explore the potential impact of combining tools like ChatGPT, Geo, and The Graph on the industry.](https://www.grtiq.com/grtiq-podcast-106-ai-and-crypto/)
    
*   [![GRTiQ podcast episode 43 Sam Green](/_astro/grtiq43.B_Iriyvr_1mtEQV.webp)\
    \
    #### GRTiQ Podcast 43: Sam Green\
    \
    ##### GRTiQ\
    \
    In Episode 43 of the GRTiQ Podcast, host Nick Hansen interviews Sam Green, Co-Founder and CTO of Semiotic, the fourth core dev team to join The Graph protocol. Sam discusses Semiotic's exceptional expertise in artificial intelligence and cryptography, and provides clear explanations of key concepts like reinforcement learning and zk roll-ups. The insights shared by Sam shed light on not only the future of The Graph and web3 but also the cutting-edge technological innovations being pursued within The Graph protocol.](https://www.grtiq.com/grtiq-podcast-43-sam-green/)
    
*   [![GRTiQ podcast episode 144 Anirudh Patel](/_astro/grtiq74.DQUhTtG2_Z1NPkwF.webp)\
    \
    #### GRTiQ Podcast: 74 Ahmet Ozcan\
    \
    ##### GRTiQ\
    \
    In this episode of the GRTiQ Podcast, host Nick Hansen interviews Ahmet Ozcan, Co-founder and CEO of Semiotic Labs, a Core Dev team contributing to The Graph. The conversation covers Semiotic Labs' recent launch of Odos, an innovative path-finding algorithm designed to optimize order routing for crypto traders. Ahmet also discusses the team's work on The Graph, his transition from a successful career at IBM to entrepreneurship in Web3, and shares his long-term vision for The Graph.](https://www.grtiq.com/grtiq-podcast-74-ahmet-ozcan/)
    
*   [![The main chain. Ahmet Ozcan CEO & co-founder of Semiotic Ai.](/_astro/main_chain_ahmet_fundamental_inovations.CJw444mA_Z2gy0ly.webp)\
    \
    #### The Main Chain: Ahmet Ozcan – Fundamental Innovations\
    \
    ##### Infinity Ventures Crypto (IVC)\
    \
    It's an exciting time right now in crypto - with decentralized exchanges (DEX) still in their infancy, crypto users are evaluating their potential. As DEXs transactions are completely automated, they can be interacted with in a 100% programmatic fashion. which means they are ripe for applications to be built on top of them. That’s where Semiotic AI comes in. Dr. Ahmet Ozcan is the CEO of Semiotic AI, which aims to build automated decision-making tools for decentralized markets on the blockchain. The Semiotic team are the brains behind odos.xyz, a web3 application that allows traders to make large token swaps and trades across disparate DEXs, quickly and efficiently. If one DEX doesn’t have the liquidity needed to complete the transaction, Odos will map trades across DEXs, making large trades effortless on the part of an investor. Semiotic has also been in the news lately for their contributions to The Graph, the indexing and query layer of Web3, for which they received a grant of USD $60 million from the Graph Foundation. Dr. Ozcan chats with host Michael Waitze about how odos works and details its future roadmap. He also discusses why The Graph is important, what it does, and how Semiotic is contributing to The Graph, and to the future of web3, including how their AI solutions simulate how different incentives motivate actors to work either for the good or the detriment of blockchain systems.](https://anchor.fm/themainchain/episodes/Ahmet-Ozcan---Fundamental-Innovations-e1ivpf8)

---

# Semiotic Labs | Articles by Carollan Helinski

![Gradient background](/_astro/curved_bg_desktop.ChjV8nJN_1c0N0M.svg) ![Gradient background](/_astro/curved_bg_mobile.DP8czMRd_1b6C73.svg)

*   [*   cryptography\
    \
    ##### Homomorphic Signatures for Payment Channels\
    \
    This post discusses design and optimization of verifiable micropayments over a state channel, a feature of Semiotic Labs’ \[GraphTally\](https://github.com/semiotic-ai/timeline-aggregation-protocol) library for The Graph. The Graph is a decentralized data services protocol, allowing customers to index and access all blockchain data.\_\
    \
    * * *](/articles/h2s2-payment-channels/)

---

# Semiotic Labs | Articles by Alexis Asseman

![Gradient background](/_astro/curved_bg_desktop.ChjV8nJN_1c0N0M.svg) ![Gradient background](/_astro/curved_bg_mobile.DP8czMRd_1b6C73.svg)

*   [*   ai\
    *   blockchain infrastructure\
    \
    ##### Automated Query Pricing in The Graph\
    \
    Indexers in The Graph have control over the pricing of the GraphQL queries they serve based on their shape. For this task, The Graph created a domain-specific language called Agora​ that maps query shapes to prices in GRT. However, manually populating and updating Agora models for each subgraph is a tedious task, and as a consequence, most indexers default to a static, flat pricing model.\_ \_To help indexers with pricing in the relative resource cost of serving different query shapes, as well as following the query market price, we are developing AutoAgora, an automation tool that automatically creates and updates Agora models.\_ \_See our related Devcon 2022 talk here:\_ \_\[https://www.youtube.com/watch?v=LRl9uFfmjEs\](https://www.youtube.com/watch?v=LRl9uFfmjEs)\_.\
    \
    * * *](/articles/automated-query-pricing-on-the-graph/)

---

# Semiotic Labs | Articles by Lichu Acuña

![Gradient background](/_astro/curved_bg_desktop.ChjV8nJN_1c0N0M.svg) ![Gradient background](/_astro/curved_bg_mobile.DP8czMRd_1b6C73.svg)

*   [*   cryptography\
    \
    ##### PSI with FHE\
    \
    Semiotic Labs explores the integration of Private Set Intersection (PSI) in blockchain to tackle Maximal Extractable Value (MEV) issues. Utilizing fully homomorphic encryption, the protocol aims for trustless operations and enhanced performance in tasks like access list comparison and private auctions.\
    \
    * * *](/articles/psi/)

---

# Semiotic Labs | Videos

![Gradient background](/_astro/curved_bg_desktop.ChjV8nJN_1c0N0M.svg) ![Gradient background](/_astro/curved_bg_mobile.DP8czMRd_1b6C73.svg)

*   [*   blockchain infrastructure\
    *   ai\
    \
    #### Bringing high-performance SQL queries to The Graph\
    \
    At Datapalooza 2023, Sam presented Semiotic's latest research & practical applications of AI & data analytics in web3 and shared details about bringing high-performance SQL queries to The Graph!](https://www.youtube.com/watch?v=eTKzHkv5E40)
    
*   [*   blockchain infrastructure\
    *   cryptography\
    \
    #### The Graph and Preserving Historical Data After EIP-4444 by Seve Sisneros\
    \
    Severiano Sisneros presents "The Graph and Preserving Historical Data After EIP-4444" at Datapalooza 2023. This talk, held during Devconnect in Istanbul, Turkey, explores the vital role of The Graph in the context of Ethereum's EIP-4444. Gain insights into the challenges and strategies for preserving historical data in the web3 space.](https://www.youtube.com/watch?v=AwU6gkXBxKM)
    
*   [*   cryptography\
    \
    #### Devcon 2022: A SNARKs Tale: A Story of Building SNARK Solutions on Mainnet\
    \
    Severiano Sisneros, Semiotic's Senior Cryptographer, tells the story of deploying a real-world solution using SNARKs as a core primitive; highlighting many cutting-edge SNARKs and their limitations in the hope to identify opportunities for the community to make Ethereum more SNARK friendly, creating a diverse ecosystem where SNARKs built upon a variety of unique primitives can thrive.](https://www.youtube.com/watch?v=GnYM9yxIRSs)
    
*   [*   defi\
    \
    #### Devcon 2022: An Overview of AMM Mechanisms\
    \
    Matt Deible gives an exhaustive overview of the different AMM algorithms currently deployed on major distributed ledgers, as well as the underlying intuition behind their design. Building up from the basic principles of AMM design, the talk then covers the algorithmic mechanisms used in the various different algorithms including Constant Sum, Constant Product (Uniswap V2), Uniswap V3, KyberSwap, StableSwap (Curve), CryptoSwap (Curve V2), Solidly Stable pairs, Clipper, Dodo, and RFQ systems.](https://www.youtube.com/watch?v=KxuyHfmLHP0)
    
*   [*   ai\
    \
    #### Devcon 2022: Reinforcement Learning for Query Pricing in The Graph\
    \
    Indexers​​ in The Graph protocol use a DSL called Agora​ to map query shapes to prices. However, manually populating and updating Agora models for each query is a tedious task, and, as a consequence, most indexers default to a flat pricing model. We have created and deployed reinforcement learning agents for Indexers to automatically compete on pricing. This talk, given by Dr. Tomasz Kornuta, is focused on our development process and our study of the market effects of multiple competing pricing agents.](https://www.youtube.com/watch?v=LRl9uFfmjEs)
    
*   [#### Stanford Seminar: Evolution of a Web3 Company\
    \
    Co-founder and Head of Research, Dr. Sam Green, presents to Stanford's CEE 246A class on the evolution of Semiotic Labs: how we started with Fully-Homomorphic Encryption, early support from NSF SBIR and Activate.org, our pivot to web3 with The Graph protocol, and the launch of our DEX aggregator: odos.xyz.](https://www.youtube.com/watch?v=KvhgJEBskgg)
    
*   [*   ai\
    \
    #### Special Graph Hack Episode: Automated Allocation Optimizer\
    \
    Indexers within The Graph Protocol are rewarded via an indexing reward. How can indexers optimize their allocations so as to maximize the reward they receive? In this video, Anirudh Patel and Hope Yen formalize that question in terms of a reward function and explain how convex optimization can be used to find a solution. They then demonstrate a new tool that automates the solution to the allocation optimization problem.](https://www.youtube.com/watch?v=J6a4RarYYFI)
    
*   [*   defi\
    \
    #### ODOS: The Optimal DEX Aggregator for Token Swaps\
    \
    Odos is a Smart Order Routing (SOR) solution that leverages a patented Automated Market Maker (AMM) path finding algorithm to aggregate decentralized exchanges (DEX) and find optimal routes for token swaps. By traversing a large universe of connecting token pairs and performing complex order splits, Odos is the ultimate choice for institutional and retail traders. Odos is the first aggregator to introduce a multi-asset input feature that enables users to swap from several tokens into one asset in a single atomic transaction.](https://www.youtube.com/watch?v=qC8Ry-YOZZs)
    
*   [*   blockchain infrastructure\
    *   ai\
    \
    #### Automated Cost Modeling in The Graph\
    \
    Alexis Asseman presents on our reinforcement learning agent used to automatically set prices for Indexers in The Graph protocol.](https://www.youtube.com/watch?v=DrdkX8FibHI)
    
*   [*   defi\
    \
    #### ODOS Intro: The Optimal Order Routing Solution for Cryptocurrency Swaps\
    \
    Odos is a Smart Order Routing (SOR) solution that leverages a patented Automated Market Maker (AMM) path finding algorithm to aggregate decentralized exchanges (DEX) and find optimal routes for token swaps. By traversing a large universe of connecting token pairs and performing complex order splits, Odos is the ultimate choice for institutional and retail traders. Odos is the first aggregator to introduce a multi-asset input feature that enables users to swap from several tokens into one asset in a single atomic transaction.](https://www.youtube.com/watch?v=XMqzRN1qQ-I)
    
*   [*   ai\
    \
    #### Introduction to the Workshop on Incentive Mechanism Validation: Sam Green, Semiotic Labs\
    \
    Presented at the WIMV 2022 Devconnect workshop. For future workshops, check out https://wimv.dev or follow https://twitter.com/wimvdev.](https://www.youtube.com/watch?v=RQULcaKWCD8)
    
*   [*   ai\
    \
    #### Simplifying Mechanism Analysis With Software Design: Anirudh Patel, Semiotic Labs\
    \
    Presented at the WIMV 2022 Devconnect workshop. For future workshops, check out https://wimv.dev or follow https://twitter.com/wimvdev.](https://www.youtube.com/watch?v=rF4Gr0CS5Gc)

---

# Semiotic Labs | Articles by Matt Deible

![Gradient background](/_astro/curved_bg_desktop.ChjV8nJN_1c0N0M.svg) ![Gradient background](/_astro/curved_bg_mobile.DP8czMRd_1b6C73.svg)

*   [*   ai\
    *   defi\
    \
    ##### An Overview of Automatic Market Maker Mechanisms\
    \
    This article explores the principles and mechanisms behind the many popular Automatic Market Maker designs currently used in production. While the mathematical details of these designs are fascinating in their own right, this article seeks to instead focus on graphical representations and high-level concepts, allowing for a more approachable and exhaustive exploration of the space.\_ \_Watch our related Devcon 2022 talk here: \[https://www.youtube.com/watch?v=KxuyHfmLHP0\](https://www.youtube.com/watch?v=KxuyHfmLHP0).\
    \
    * * *](/articles/automatic-market-makers-amm/)

---

# Semiotic Labs | Articles by Anirudh A. Patel

![Gradient background](/_astro/curved_bg_desktop.ChjV8nJN_1c0N0M.svg) ![Gradient background](/_astro/curved_bg_mobile.DP8czMRd_1b6C73.svg)

*   [*   ai\
    \
    ##### Indexer Allocation Optimization: Part I\
    \
    Indexers within The Graph Protocol are rewarded via an indexing reward. How can indexers optimise their allocations so as to maximise the reward they receive? In this blog post we formalise the problem in terms of a reward function and use convex optimization to find a solution.\
    \
    * * *](/articles/indexer-allocation-optimisation/)
    
*   [*   ai\
    \
    ##### Indexer Allocation Optimization: Part II\
    \
    This article was co-authored with Howard Heaton, from Edge & Node, and with Hope Yen, from GraphOps.\
    \
    * * *](/articles/indexer-allocation-optimization-part-ii/)

---