# Table of Contents - [Welcome to Scrapybara | Scrapybara Docs](#welcome-to-scrapybara-scrapybara-docs) - [Best Practices | Scrapybara Docs](#best-practices-scrapybara-docs) - [OpenAI | Scrapybara Docs](#openai-scrapybara-docs) - [Auth States | Scrapybara Docs](#auth-states-scrapybara-docs) - [Quickstart | Scrapybara Docs](#quickstart-scrapybara-docs) - [Act SDK | Scrapybara Docs](#act-sdk-scrapybara-docs) - [Conversations | Scrapybara Docs](#conversations-scrapybara-docs) - [Tools | Scrapybara Docs](#tools-scrapybara-docs) - [Ubuntu | Scrapybara Docs](#ubuntu-scrapybara-docs) - [Anthropic | Scrapybara Docs](#anthropic-scrapybara-docs) - [Browser | Scrapybara Docs](#browser-scrapybara-docs) - [Environment Variables | Scrapybara Docs](#environment-variables-scrapybara-docs) - [Filesystem | Scrapybara Docs](#filesystem-scrapybara-docs) - [Code Execution | Scrapybara Docs](#code-execution-scrapybara-docs) - [Windows | Scrapybara Docs](#windows-scrapybara-docs) - [Starter Templates | Scrapybara Docs](#starter-templates-scrapybara-docs) - [Cursor Rules | Scrapybara Docs](#cursor-rules-scrapybara-docs) - [Browser | Scrapybara Docs](#browser-scrapybara-docs) - [Get instance by ID | Scrapybara Docs](#get-instance-by-id-scrapybara-docs) - [List all instances | Scrapybara Docs](#list-all-instances-scrapybara-docs) - [List all browser authentication states | Scrapybara Docs](#list-all-browser-authentication-states-scrapybara-docs) - [Take screenshot | Scrapybara Docs](#take-screenshot-scrapybara-docs) - [Get stream URL | Scrapybara Docs](#get-stream-url-scrapybara-docs) - [Run bash actions | Scrapybara Docs](#run-bash-actions-scrapybara-docs) - [Run computer actions | Scrapybara Docs](#run-computer-actions-scrapybara-docs) - [Run edit actions | Scrapybara Docs](#run-edit-actions-scrapybara-docs) - [Pause instance | Scrapybara Docs](#pause-instance-scrapybara-docs) - [Start instance | Scrapybara Docs](#start-instance-scrapybara-docs) - [Stop instance | Scrapybara Docs](#stop-instance-scrapybara-docs) - [Resume instance | Scrapybara Docs](#resume-instance-scrapybara-docs) - [Get CDP URL | Scrapybara Docs](#get-cdp-url-scrapybara-docs) - [Start browser | Scrapybara Docs](#start-browser-scrapybara-docs) - [Get current URL | Scrapybara Docs](#get-current-url-scrapybara-docs) --- # Welcome to Scrapybara | Scrapybara Docs ![Introduction](https://files.buildwithfern.com/https://scrapybara-docs.docs.buildwithfern.com/2025-03-12T14:21:40.151Z/assets/images/introduction.png) [Quickstart\ \ Setup Scrapybara and deploy your first computer use agent in minutes](/quickstart) What is Scrapybara? ------------------- Scrapybara provides powerful virtual desktop infrastructure for computer use agents with: * **Lightning speed**: Instantly spin up multiple [Ubuntu](/ubuntu) and [Browser](/browser) instances under 1 second. * **Flexible deployments**: Easily deploy and manage remote instances with our [API](/api-reference) , [SDKs](/sdk-reference) , and [dashboard](https://scrapybara.com/dashboard) . Configure instance types and timeouts to match your needs. * **Unified integration**: Seamlessly connect your instances to models like [OpenAI CUA](/openai) and [Claude Computer Use](/anthropic) and execute powerful agentic workflows with [structured outputs](/act-sdk) , [multi-turn conversations](/conversations) , and [custom tools](/tools) . * **Granular control**: Get complete remote desktop access with all the tools your agents need, including [Browser](/protocols/browser) , [Code Execution](/protocols/code) , [Environment Variables](/protocols/env) , and [Filesystem](/protocols/file) . * **Authenticated sessions**: Save and reuse browser [auth states](/auth-states) across instances. Explore our guides ------------------ [Act SDK\ \ Take action with computer use models and custom tools with one line of code](/act-sdk) [Auth States\ \ Save and reuse browser authentication states across instances](/auth-states) Get inspired ------------ [![](https://www.rainforest-alliance.org/wp-content/uploads/2021/06/capybara-square-1.jpg.optimal.jpg)\ \ CopyCapy\ \ Scrape and transform websites into capybara-themed versions with Scrapybara Act SDK and Playwright.\ \ \ \ `TypeScript` `Act SDK` `Playwright`](/cookbook/copycapy) [![](https://upload.wikimedia.org/wikipedia/commons/thumb/b/b2/Y_Combinator_logo.svg/1200px-Y_Combinator_logo.svg.png)\ \ Wide Research\ \ Deep Research but wide. Scrape YC W25 companies and find the best way to contact each company in parallel batches.\ \ \ \ `Python` `Act SDK` `Structured Outputs`](/cookbook/wide-research) Dashboard --------- Get your API key and manage your instances on the [dashboard](https://scrapybara.com/dashboard) . Get support ----------- Need help or want to stay up to date? Reach out to us on Discord or X. [Discord\ \ Join our Discord community](https://discord.gg/s4bPUVFXqA) [X\ \ Follow us on X](https://x.com/scrapybara) --- # Best Practices | Scrapybara Docs Manage instance usage --------------------- Instances are billed per usage. When launching an instance, you can specify its timeout before it is automatically terminated (default is 1 hour). ###### Python ###### TypeScript ` | | | | --- | --- | | 1 | instance = client.start_ubuntu(timeout=3) # 3 hours | ` To save costs, pause the instance to resume it later, or stop the instance once you no longer need to control the desktop environment and access its stored data. ###### Python ###### TypeScript Pause/resume ` | | | | --- | --- | | 1 | instance.pause() | | 2 | instance.resume() | ` Stop ` | | | | --- | --- | | 1 | instance.stop() | ` Take actions programmatically ----------------------------- When possible, take actions programmatically rather than relying on the agent to do so. For example, using `instance.bash` provides a faster way to launch apps compared to having the model use mouse/keyboard interactions. If you know the agent’s workflow will happen on a specific application, you can launch it before prompting the agent to take actions. The same applies for browser automation: it is often easier to manipulate the browser programmatically with `instance.browser` and Playwright than relying on the agent itself. Initialize the browser ---------------------- For agents requiring programmatic browser interaction, initialize and configure the browser immediately after instance creation. This ensures the browser environment is ready before any browser tool calls are made. ###### Python ###### TypeScript ` | | | | --- | --- | | 1 | instance = client.start_ubuntu() | | 2 | instance.browser.start() | | 3 | instance.browser.authenticate(auth_state_id="auth_state_id") | ` Optimize your prompt -------------------- Each model (OpenAI and Anthropic) has its own specialized system prompt optimized for computer use tasks. We recommend using these model-specific prompts for best results. ###### Python ###### TypeScript OpenAI ` | | | | --- | --- | | 1 | from scrapybara.openai import OpenAI, UBUNTU_SYSTEM_PROMPT as OPENAI_UBUNTU_PROMPT | | 2 | | | 3 | client.act( | | 4 | model=OpenAI(), | | 5 | system=OPENAI_UBUNTU_PROMPT, | | 6 | # ... | | 7 | ) | ` Anthropic ` | | | | --- | --- | | 1 | from scrapybara.anthropic import Anthropic, UBUNTU_SYSTEM_PROMPT as ANTHROPIC_UBUNTU_PROMPT | | 2 | | | 3 | client.act( | | 4 | model=Anthropic(), | | 5 | system=ANTHROPIC_UBUNTU_PROMPT, | | 6 | # ... | | 7 | ) | ` For more complex tasks, you can extend the model-specific system prompt by defining additional task requirements: ###### Python ###### TypeScript ` | | | | --- | --- | | 1 | # Use the appropriate system prompt for your chosen model | | 2 | system = f"""{UBUNTU_SYSTEM_PROMPT} | | 3 | | | 4 | | | 5 | {task} | | 6 | """ | ` Here are some tips from [Anthropic](https://docs.anthropic.com/en/docs/build-with-claude/computer-use) to improve the agent’s performance: 1. Specify simple, well-defined tasks and provide explicit instructions for each step. 2. Models sometimes assume outcomes of their actions without explicitly checking their results. To prevent this you can prompt the model with `After each step, take a screenshot and carefully evaluate if you have achieved the right outcome. Explicitly show your thinking: "I have evaluated step X..." If not correct, try again. Only when you confirm a step was executed correctly should you move on to the next one.` 3. Some UI elements (like dropdowns and scrollbars) might be tricky for models to manipulate using mouse movements. If you experience this, try prompting the model to use keyboard shortcuts. 4. For repeatable tasks or UI interactions, include example screenshots and tool calls of successful outcomes in your prompt. 5. If you need the model to log in, provide it with the username and password in your prompt inside xml tags like ``. Using computer use within applications that require login increases the risk of bad outcomes as a result of prompt injection. --- # OpenAI | Scrapybara Docs Act SDK ------- Use OpenAI models with the Act SDK: * Default: `computer-use-preview` (computer use beta) Consume agent credits or bring your own API key. Without an API key, each step consumes 1 [agent credit](https://scrapybara.com/#pricing) . With your own API key, model charges are billed directly to your OpenAI account. ###### Python ###### TypeScript Import model ` | | | | --- | --- | | 1 | from scrapybara.openai import OpenAI, UBUNTU_SYSTEM_PROMPT | | 2 | | | 3 | # Consume agent credits | | 4 | model = OpenAI() | | 5 | | | 6 | # Bring your own API key | | 7 | model = OpenAI(api_key="your_api_key") | ` Take action ` | | | | --- | --- | | 1 | from scrapybara import Scrapybara | | 2 | from scrapybara.tools import BashTool, ComputerTool, EditTool | | 3 | | | 4 | client = Scrapybara() | | 5 | instance = client.start_ubuntu() | | 6 | | | 7 | client.act( | | 8 | tools=[ | | 9 | BashTool(instance), | | 10 | ComputerTool(instance), | | 11 | EditTool(instance), | | 12 | ], | | 13 | model=model, | | 14 | system=UBUNTU_SYSTEM_PROMPT, | | 15 | prompt="Research Scrapybara", | | 16 | ) | ` --- # Auth States | Scrapybara Docs What are auth states? --------------------- Auth states in Scrapybara allow you to capture, save, and reuse website authentication data across browser sessions. This feature is extremely useful for: * Automating login flows * Persisting authentication between browser instances * Avoiding repetitive login procedures in your automation scripts * Testing authenticated features without manual intervention Auth states capture cookies, local storage, session storage, and other browser-based authentication data. Save auth state --------------- When you’ve successfully authenticated with a website in the browser, you can save the authentication state for future use. Each auth state is identified by a unique ID and an optional name (defaults to “default” if not specified). ###### Python ###### TypeScript ` | | | | --- | --- | | 1 | # First, create a browser instance | | 2 | instance = client.start_browser() | | 3 | | | 4 | # Open the stream, navigate to the login page, and log in | | 5 | webbrowser.open(instance.get_stream_url().stream_url) | ` Once you’ve logged in, you can save the auth state with a name: ` | | | | --- | --- | | 1 | # Now save the auth state with a name | | 2 | auth_state_id = instance.save_auth(name="example_site").auth_state_id | | 3 | print(f"Saved auth state ID: {auth_state_id}") | ` Modify auth state ----------------- If you have an existing auth state that you want to update with new authentication data, you can use the `modify_auth` functionality. This is useful when credentials have changed or when an existing auth state needs to be refreshed. ###### Python ###### TypeScript ` | | | | --- | --- | | 1 | # First, create a browser instance | | 2 | instance = client.start_browser() | | 3 | | | 4 | # Open the stream, navigate to the login page, and log in with new credentials | | 5 | webbrowser.open(instance.get_stream_url().stream_url) | ` Once you’ve logged in, you can modify the auth state with a new name: ` | | | | --- | --- | | 1 | # Update an existing auth state with the new credentials | | 2 | # You can optionally provide a new name | | 3 | instance.modify_auth(auth_state_id="your_existing_auth_state_id", name="renamed_auth_state") | ` Load auth state --------------- Once you’ve saved an auth state, you can use it to authenticate future browser sessions without going through the login process again: ###### Python ###### TypeScript ` | | | | --- | --- | | 1 | # Create a new browser instance | | 2 | instance = client.start_browser() | | 3 | | | 4 | # Authenticate using a previously saved auth state | | 5 | instance.authenticate(auth_state_id="your_auth_state_id") | ` Using auth states in the playground ----------------------------------- The Scrapybara Playground provides a visual interface for managing auth states: 1. **Create an auth state**: * Head to the Scrapybara [auth](https://scrapybara.com/auth) page * Click on the “Create auth state” button * Log in to the website and save the auth state 2. **Use an existing auth state**: * Click the fingerprint icon next to the instance dropdown menu * Choose your saved auth state from the dropdown * When you start the instance, it will automatically apply the authentication data Best practices -------------- Here are some best practices for working with auth states: 1. **Use descriptive names**: Name your auth states clearly (e.g., “github\_login” or “shopify\_admin”) for easy identification. 2. **Refresh regularly**: Authentication tokens can expire. Consider updating your auth states periodically using the `modify_auth` functionality. 3. **Test before use**: Always verify that your auth state is still valid before relying on it for critical automation. 4. **Multiple auth states**: Maintain separate auth states for different environments (e.g., production, staging, development). --- # Quickstart | Scrapybara Docs Deploy your first instance -------------------------- ###### Python ###### TypeScript [1](/quickstart#generate-api-key) ### Generate API key Sign up on our [dashboard](https://scrapybara.com/dashboard) . An API key will be generated for you automatically. [2](/quickstart#install-sdk) ### Install SDK Install the Python SDK with pip. ` | | | | --- | --- | | $ | pip install scrapybara | ` [3](/quickstart#configure-client) ### Configure client Configure the client with your API key. ` | | | | --- | --- | | 1 | from scrapybara import Scrapybara | | 2 | | | 3 | client = Scrapybara(api_key="your_api_key") | ` [4](/quickstart#start-an-instance) ### Start an instance Start an instance with your desired configuration. You can choose from Ubuntu, Browser, and Windows instances. We recommend using Ubuntu for most tasks. ` | | | | --- | --- | | 1 | instance = client.start_ubuntu( | | 2 | timeout_hours=1, | | 3 | ) | | 4 | | | 5 | # browser_instance = client.start_browser( | | 6 | # timeout_hours=1, | | 7 | # ) | ` [5](/quickstart#get-the-stream-url) ### Get the stream URL Get the stream URL to view and interact with the instance manually. ` | | | | --- | --- | | 1 | stream_url = instance.get_stream_url().stream_url | ` [6](/quickstart#interact-with-the-instance) ### Interact with the instance Interact with the instance with `computer` and `bash`. Move the mouse ` | | | | --- | --- | | 1 | instance.computer( | | 2 | action="move_mouse", | | 3 | coordinates=[200, 100] | | 4 | ) | ` Left click ` | | | | --- | --- | | 1 | instance.computer( | | 2 | action="click_mouse", | | 3 | button="left" | | 4 | ) | ` Type hello ` | | | | --- | --- | | 1 | instance.computer( | | 2 | action="type_text", | | 3 | text="Hello, world!" | | 4 | ) | ` Run a bash command ` | | | | --- | --- | | 1 | result = instance.bash( | | 2 | command="ls -la" | | 3 | ) | ` [7](/quickstart#connect-to-the-browser) ### Connect to the browser Connect to the browser with Playwright to enable programmatic browser control and authenticated browser sessions. Learn more [here](/browser) . Connect to Playwright ` | | | | --- | --- | | 1 | from playwright.sync_api import sync_playwright | | 2 | | | 3 | cdp_url = instance.browser.start().cdp_url | | 4 | playwright = sync_playwright().start() | | 5 | browser = playwright.chromium.connect_over_cdp(cdp_url) | ` Save the auth state ` | | | | --- | --- | | 1 | auth_state_id = instance.browser.save_auth(name="default").auth_state_id | ` Reuse the auth state on other instances ` | | | | --- | --- | | 1 | instance.browser.authenticate(auth_state_id=auth_state_id) | ` [8](/quickstart#take-action) ### Take action! Build your first agent with the Act SDK to control your Scrapybara instance with `BashTool`, `ComputerTool`, `EditTool`. Learn more [here](/act-sdk) . ` | | | | --- | --- | | 1 | from scrapybara.tools import BashTool, ComputerTool, EditTool | | 2 | from scrapybara.openai import OpenAI, UBUNTU_SYSTEM_PROMPT | | 3 | | | 4 | response = client.act( | | 5 | model=OpenAI(), | | 6 | tools=[ | | 7 | BashTool(instance), | | 8 | ComputerTool(instance), | | 9 | EditTool(instance), | | 10 | ], | | 11 | system=UBUNTU_SYSTEM_PROMPT, | | 12 | prompt="Go to the top link on Hacker News", | | 13 | on_step=lambda step: print(step.text), | | 14 | ) | ` [9](/quickstart#stop-the-instance) ### Stop the instance Stop the instance when you’re done. This will delete all data stored during the session. ` | | | | --- | --- | | 1 | instance.stop() | ` Start building -------------- Be sure to check out our other resources to learn more. Happy building! ₍ᐢ•(ܫ)•ᐢ₎ [Act SDK\ \ Take action with computer use agents](/act-sdk) [API Reference\ \ Check out our API reference docs](/api-reference) [Ubuntu\ \ Learn about the UbuntuInstance](/ubuntu) [Browser\ \ Learn about the BrowserInstance](/browser) --- # Act SDK | Scrapybara Docs What is the Act SDK? -------------------- The Act SDK is a unified SDK for building computer use agents with Python and TypeScript. It provides a simple interface for executing looping agentic actions with support for many models and tools. Build production-ready computer use agents with pre-built tools to connect to Scrapybara instances. How it works ------------ `act` initiates an interaction loop that continues until the agent achieves its objective. Each iteration of the loop is called a `step`, which consists of the agent’s text response, the agent’s tool calls, and the results of those tool calls. The loop terminates when the agent returns a message without invoking any tools, and returns `messages`, `steps`, `text`, `output` (if `schema` is provided), and `usage` after the agent’s execution. ###### Python ###### TypeScript ` | | | | --- | --- | | 1 | response = client.act( | | 2 | model=OpenAI(), | | 3 | tools=[ | | 4 | BashTool(instance), | | 5 | ComputerTool(instance), | | 6 | EditTool(instance), | | 7 | ], | | 8 | system=UBUNTU_SYSTEM_PROMPT, | | 9 | prompt="Go to the top link on Hacker News", | | 10 | on_step=lambda step: print(step.text), | | 11 | ) | | 12 | messages = response.messages | | 13 | steps = response.steps | | 14 | text = response.text | | 15 | usage = response.usage | ` An `act` call consists of 3 core components: ### Model The model specifies the base LLM for the agent. At each step, the model examines the previous messages, the current state of the computer, and uses tools to take action. Each step will cost an amount of agent credits depending on the model. You can also bring your own API key to bill model charges directly. ###### Python ###### TypeScript ` | | | | --- | --- | | 1 | from scrapybara.openai import OpenAI | | 2 | | | 3 | model = OpenAI() | | 4 | | | 5 | # Use your own API key | | 6 | model = OpenAI(api_key="your_api_key") | ` ### Tools Tools are functions that enable agents to interact with the computer. Each tool is defined by a `name`, `description`, and how it can be executed with `parameters` and an execution function. A tool can take in a Scrapybara instance to interact with it directly. Learn more about pre-built tools and how to define custom tools [here](/tools) . ###### Python ###### TypeScript ` | | | | --- | --- | | 1 | from scrapybara import Scrapybara | | 2 | from scrapybara.tools import BashTool, ComputerTool, EditTool | | 3 | | | 4 | client = Scrapybara() | | 5 | instance = client.start_ubuntu() | | 6 | | | 7 | tools = [ | | 8 | BashTool(instance), | | 9 | ComputerTool(instance), | | 10 | EditTool(instance), | | 11 | ] | ` ### Prompt The prompt is split into two parts, the `system` prompt and a user `prompt`. `system` defines the general behavior of the agent, such as its capabilities and constraints. You can use our provided `UBUNTU_SYSTEM_PROMPT`, `BROWSER_SYSTEM_PROMPT`, and `WINDOWS_SYSTEM_PROMPT` to get started, or define your own. `prompt` should denote the agent’s current objective. Alternatively, you can provide `messages` instead of `prompt` to start the agent with a history of messages. `act` conveniently returns `messages` after the agent’s execution, so you can reuse it in another `act` call. ###### Python ###### TypeScript ` | | | | --- | --- | | 1 | from scrapybara.prompts import UBUNTU_SYSTEM_PROMPT | | 2 | | | 3 | system = UBUNTU_SYSTEM_PROMPT | | 4 | prompt = "Go to the top link on Hacker News" | ` Structured output ----------------- Use the `schema` parameter to define a desired structured output. The response’s `output` field will contain the typed data returned by the model. This is particularly useful when scraping or collecting structured data from websites. Under the hood, we pass in a `StructuredOutputTool` to enforce and parse the schema. ###### Python ###### TypeScript ` | | | | --- | --- | | 1 | from pydantic import BaseModel | | 2 | from typing import List | | 3 | | | 4 | class HNSchema(BaseModel): | | 5 | class Post(BaseModel): | | 6 | title: str | | 7 | url: str | | 8 | points: int | | 9 | | | 10 | posts: List[Post] | | 11 | | | 12 | response = client.act( | | 13 | model=OpenAI(), | | 14 | tools=[ | | 15 | ComputerTool(instance), | | 16 | ], | | 17 | schema=HNSchema, | | 18 | system=UBUNTU_SYSTEM_PROMPT, | | 19 | prompt="Get the top 10 posts on Hacker News", | | 20 | ) | | 21 | | | 22 | posts = response.output.posts | ` Agent credits ------------- Consume agent credits or bring your own API key. Without an API key, each step consumes 1 [agent credit](https://scrapybara.com/#pricing) . With your own API key, model charges are billed directly to your provider API key. Full example ------------ Here is how you can build a computer use agent that can output structured data. ###### Python ###### TypeScript ` | | | | --- | --- | | 1 | from scrapybara import Scrapybara | | 2 | from scrapybara.openai import OpenAI | | 3 | from scrapybara.prompts import UBUNTU_SYSTEM_PROMPT | | 4 | from scrapybara.tools import ComputerTool | | 5 | from pydantic import BaseModel | | 6 | from typing import List | | 7 | | | 8 | client = Scrapybara() | | 9 | instance = client.start_ubuntu() | | 10 | | | 11 | class HNSchema(BaseModel): | | 12 | class Post(BaseModel): | | 13 | title: str | | 14 | url: str | | 15 | points: int | | 16 | | | 17 | posts: List[Post] | | 18 | | | 19 | response = client.act( | | 20 | model=OpenAI(), | | 21 | tools=[ | | 22 | ComputerTool(instance), | | 23 | ], | | 24 | system=UBUNTU_SYSTEM_PROMPT, | | 25 | prompt="Get the top 10 posts on Hacker News", | | 26 | schema=HNSchema, | | 27 | on_step=lambda step: print(step.text), | | 28 | ) | | 29 | | | 30 | posts = response.output.posts | | 31 | print(posts) | | 32 | | | 33 | instance.stop() | ` --- # Conversations | Scrapybara Docs Understanding multi-turn conversations -------------------------------------- Multi-turn conversations in Scrapybara enable your agents to maintain context and state across multiple interactions. The Act SDK provides a structured way to manage these conversations through its message architecture. Message architecture -------------------- The Act SDK uses a structured message system with three primary message types and five different part types. Understanding these components is crucial for building sophisticated multi-turn agents. ### Message types ` | | | | --- | --- | | 1 | # Message types | | 2 | class UserMessage: | | 3 | role: str = "user" # Always "user" | | 4 | content: List[Union[TextPart, ImagePart]] # What the user sends | | 5 | | | 6 | class AssistantMessage: | | 7 | role: str = "assistant" # Always "assistant" | | 8 | content: List[Union[TextPart, ToolCallPart, ReasoningPart]] # The agent's response | | 9 | response_id: Optional[str] = None # Unique identifier for the response | | 10 | | | 11 | class ToolMessage: | | 12 | role: str = "tool" # Always "tool" | | 13 | content: List[ToolResultPart] # Results from tool operations | | 14 | | | 15 | Message = Union[UserMessage, AssistantMessage, ToolMessage] | ` ### Message part types Each message type contains various “parts” that serve different purposes: ` | | | | --- | --- | | 1 | # Message part types | | 2 | class TextPart: | | 3 | type: str = "text" # Always "text" | | 4 | text: str # Plain text content | | 5 | | | 6 | class ImagePart: | | 7 | type: str = "image" # Always "image" | | 8 | image: str # Base64 encoded image or URL | | 9 | mime_type: Optional[str] = None # e.g., "image/png", "image/jpeg" | | 10 | | | 11 | class ToolCallPart: | | 12 | type: str = "tool-call" # Always "tool-call" | | 13 | id: Optional[str] = None # Unique identifier for the tool call | | 14 | tool_call_id: str # ID matching the tool result | | 15 | tool_name: str # Name of the tool being called | | 16 | args: dict[str, Any] # Arguments passed to the tool | | 17 | | | 18 | class ToolResultPart: | | 19 | type: str = "tool-result" # Always "tool-result" | | 20 | tool_call_id: str # ID matching the original tool call | | 21 | tool_name: str # Name of the tool that was called | | 22 | result: Any # Result returned by the tool | | 23 | is_error: Optional[bool] = False # Whether the tool execution resulted in an error | | 24 | | | 25 | class ReasoningPart: | | 26 | type: str = "reasoning" # Always "reasoning" | | 27 | id: Optional[str] = None # Unique identifier for the reasoning part | | 28 | reasoning: str # The agent's internal reasoning | | 29 | signature: Optional[str] = None # Cryptographic signature for verification | | 30 | instructions: Optional[str] = None # Additional context about the reasoning | ` Building multi-turn conversations --------------------------------- Instead of providing a single `prompt`, you can pass a complete message history using the `messages` parameter. This allows you to maintain the full conversation context. The Act SDK returns a `messages` field in the response that contains the complete conversation history. You can reuse this directly in your next `act` call. ###### Python ###### TypeScript ` | | | | --- | --- | | 1 | from scrapybara import Scrapybara | | 2 | from scrapybara.anthropic import Anthropic | | 3 | from scrapybara.prompts import UBUNTU_SYSTEM_PROMPT | | 4 | from scrapybara.tools import BashTool, ComputerTool, EditTool | | 5 | | | 6 | client = Scrapybara() | | 7 | instance = client.start_ubuntu() | | 8 | | | 9 | # Initial conversation | | 10 | response = client.act( | | 11 | model=Anthropic(), | | 12 | tools=[ | | 13 | BashTool(instance), | | 14 | ComputerTool(instance), | | 15 | EditTool(instance), | | 16 | ], | | 17 | on_step=lambda step: print(step.text), | | 18 | system=UBUNTU_SYSTEM_PROMPT, | | 19 | prompt="Create a file called hello.py that prints 'Hello, World!'", | | 20 | ) | | 21 | | | 22 | print('--------------------------------') | | 23 | | | 24 | # Continue the conversation with the previous messages | | 25 | follow_up_response = client.act( | | 26 | model=Anthropic(), | | 27 | tools=[ | | 28 | BashTool(instance), | | 29 | ComputerTool(instance), | | 30 | EditTool(instance), | | 31 | ], | | 32 | on_step=lambda step: print(step.text), | | 33 | system=UBUNTU_SYSTEM_PROMPT, | | 34 | messages=response.messages + [ | | 35 | { | | 36 | "role": "user", | | 37 | "content": [ | | 38 | { | | 39 | "type": "text", | | 40 | "text": "Now modify the file to accept a name as a command line argument and print 'Hello, {name}!'" | | 41 | } | | 42 | ] | | 43 | } | | 44 | ] | | 45 | ) | | 46 | | | 47 | instance.stop() | ` Including screenshots in messages --------------------------------- Screenshots are a powerful way to provide visual context to your agent. You can include them in user messages using the `ImagePart` type. ###### Python ###### TypeScript ` | | | | --- | --- | | 1 | from scrapybara import Scrapybara | | 2 | from scrapybara.anthropic import Anthropic | | 3 | from scrapybara.prompts import UBUNTU_SYSTEM_PROMPT | | 4 | | | 5 | client = Scrapybara() | | 6 | instance = client.start_ubuntu() | | 7 | | | 8 | # Take a screenshot | | 9 | screenshot = instance.screenshot().base_64_image | | 10 | | | 11 | # Send the screenshot to the agent | | 12 | messages = [ | | 13 | { | | 14 | "role": "user", | | 15 | "content": [ | | 16 | { | | 17 | "type": "text", | | 18 | "text": "What do you see in this screenshot? Describe the desktop environment." | | 19 | }, | | 20 | { | | 21 | "type": "image", | | 22 | "image": 'data:image/png;base64,' + screenshot, | | 23 | "mime_type": "image/png" | | 24 | } | | 25 | ] | | 26 | } | | 27 | ] | | 28 | | | 29 | response = client.act( | | 30 | model=Anthropic(), | | 31 | system=UBUNTU_SYSTEM_PROMPT, | | 32 | messages=messages, | | 33 | ) | | 34 | | | 35 | print(response.text) | | 36 | instance.stop() | ` Working with tools and reasoning -------------------------------- The Act SDK captures both tool calls and agent reasoning in its message architecture. Here’s how you can access and work with this information: ### Examining tool calls and results ###### Python ###### TypeScript ` | | | | --- | --- | | 1 | from scrapybara import Scrapybara | | 2 | from scrapybara.anthropic import Anthropic | | 3 | from scrapybara.prompts import UBUNTU_SYSTEM_PROMPT | | 4 | from scrapybara.tools import BashTool | | 5 | | | 6 | client = Scrapybara() | | 7 | instance = client.start_ubuntu() | | 8 | | | 9 | response = client.act( | | 10 | model=Anthropic(), | | 11 | tools=[BashTool(instance)], | | 12 | system=UBUNTU_SYSTEM_PROMPT, | | 13 | prompt="Show me the current directory structure", | | 14 | ) | | 15 | | | 16 | # Analyze the conversation steps | | 17 | for message in response.messages: | | 18 | if message.role == "assistant": | | 19 | for part in message.content: | | 20 | if part.type == "tool-call": | | 21 | print(f"Tool called: {part.tool_name}") | | 22 | print(f"Arguments: {part.args}") | | 23 | elif message.role == "tool": | | 24 | for part in message.content: | | 25 | print(f"Tool result from {part.tool_name}: {part.result}") | | 26 | | | 27 | instance.stop() | ` ### Accessing agent reasoning ###### Python ###### TypeScript ` | | | | --- | --- | | 1 | from scrapybara import Scrapybara | | 2 | from scrapybara.anthropic import Anthropic | | 3 | from scrapybara.prompts import UBUNTU_SYSTEM_PROMPT | | 4 | from scrapybara.tools import BashTool, ComputerTool | | 5 | | | 6 | client = Scrapybara() | | 7 | instance = client.start_ubuntu() | | 8 | | | 9 | response = client.act( | | 10 | model=Anthropic(name="claude-3-7-sonnet-20250219-thinking"), | | 11 | tools=[ | | 12 | BashTool(instance), | | 13 | ComputerTool(instance), | | 14 | ], | | 15 | system=UBUNTU_SYSTEM_PROMPT, | | 16 | prompt="Open Firefox and navigate to scrapybara.com", | | 17 | ) | | 18 | | | 19 | # Extract reasoning parts from assistant messages | | 20 | for message in response.messages: | | 21 | if message.role == "assistant": | | 22 | for part in message.content: | | 23 | if part.type == "reasoning": | | 24 | print("Agent reasoning:") | | 25 | print(part.reasoning) | | 26 | | | 27 | # Or access reasoning directly from steps | | 28 | for step in response.steps: | | 29 | if step.reasoning_parts: | | 30 | print(f"Step reasoning: {step.reasoning_parts}") | | 31 | | | 32 | instance.stop() | ` Best practices for multi-turn conversations ------------------------------------------- 1. **Maintain message history**: Always use the returned `messages` from each call to maintain conversation context. 2. **Clear instructions**: Provide clear, specific instructions in each new user message. 3. **Handle context length**: For very long conversations, consider summarizing or truncating older messages to avoid exceeding model context limits. 4. **Include visual context**: Use screenshots when appropriate to provide additional context to the agent. 5. **Monitor token usage**: Track token usage through the `usage` field to prevent exceeding quotas or limits. 6. **Process message parts**: Parse and handle different message parts appropriately based on their type. Simple multi-turn example ------------------------- Here’s an interactive Read-Eval-Print Loop (REPL) implementation that allows you to have ongoing conversations with your agent: ###### Python ###### TypeScript ` | | | | --- | --- | | 1 | from scrapybara import Scrapybara | | 2 | from scrapybara.anthropic import Anthropic | | 3 | from scrapybara.prompts import UBUNTU_SYSTEM_PROMPT | | 4 | from scrapybara.tools import BashTool, ComputerTool, EditTool | | 5 | | | 6 | def agent_repl(): | | 7 | client = Scrapybara() | | 8 | instance = client.start_ubuntu() | | 9 | tools = [BashTool(instance), ComputerTool(instance), EditTool(instance)] | | 10 | messages = [] | | 11 | | | 12 | print("Scrapybara REPL started. Type 'exit' to quit") | | 13 | | | 14 | try: | | 15 | while True: | | 16 | # Get user input | | 17 | user_input = input("\n> ") | | 18 | | | 19 | # Exit command | | 20 | if user_input.lower() == 'exit': | | 21 | break | | 22 | | | 23 | # Regular text command | | 24 | messages.append({ | | 25 | "role": "user", | | 26 | "content": [{"type": "text", "text": user_input}] | | 27 | }) | | 28 | | | 29 | # Process with agent | | 30 | print("Processing...") | | 31 | response = client.act( | | 32 | model=Anthropic(), | | 33 | tools=tools, | | 34 | system=UBUNTU_SYSTEM_PROMPT, | | 35 | on_step=lambda step: print(step.text), | | 36 | messages=messages | | 37 | ) | | 38 | | | 39 | # Update conversation history | | 40 | messages = response.messages | | 41 | finally: | | 42 | instance.stop() | | 43 | print("Session ended.") | | 44 | | | 45 | if __name__ == "__main__": | | 46 | agent_repl() | ` --- # Tools | Scrapybara Docs Scrapybara tools ---------------- ### BashTool, ComputerTool, EditTool `BashTool`, `ComputerTool`, and `EditTool` follow the same interface as the instance `bash`, `computer`, and `edit` methods. They each take in an `instance` parameter to interact with the instance. * `ComputerTool` allows the agent to allows the agent to control mouse and keyboard. Supported for Ubuntu, Browser, and Windows instances. * `BashTool` allows the agent to run bash commands. Supported only for Ubuntu instances. * `EditTool` allows the agent to view, create, and edit files. Supported only for Ubuntu instances. ###### Python ###### TypeScript ` | | | | --- | --- | | 1 | from scrapybara import Scrapybara | | 2 | from scrapybara.tools import BashTool, ComputerTool, EditTool | | 3 | | | 4 | client = Scrapybara() | | 5 | instance = client.start_ubuntu() | | 6 | | | 7 | tools = [ | | 8 | BashTool(instance), | | 9 | ComputerTool(instance), | | 10 | EditTool(instance), | | 11 | ] | ` Define custom tools ------------------- You can define custom tools. A tool needs a `name`, `description`, `parameters` (Pydantic model for Python, Zod object for TS), and an execute function (`__call__` for Python, `execute` for TS). ###### Python ###### TypeScript ` | | | | --- | --- | | 1 | from scrapybara.client import UbuntuInstance | | 2 | from scrapybara.tools import Tool | | 3 | from pydantic import BaseModel | | 4 | | | 5 | class CapyParameters(BaseModel): | | 6 | # Define your parameters here | | 7 | pass | | 8 | | | 9 | class CapyTool(Tool): | | 10 | _instance: UbuntuInstance | | 11 | | | 12 | def __init__(self, instance: UbuntuInstance) -> None: | | 13 | super().__init__( | | 14 | name="capy", | | 15 | description="Use a capybara", | | 16 | parameters=CapyParameters, | | 17 | ) | | 18 | self._instance = instance | | 19 | | | 20 | def __call__(self, **kwargs: Any) -> Any: | | 21 | # Implement your tool logic here | | 22 | pass | ` ### BrowserTool `BrowserTool` allows the agent to interact with a browser using Playwright. Custom tools like BrowserTool may degrade model performance, as the models have not been trained on custom tools. For browser automation, we recommend sticking to ComputerTool. The BrowserTool requires the browser to be started first. ###### Python ###### TypeScript ` | | | | --- | --- | | 1 | from playwright.sync_api import sync_playwright | | 2 | | | 3 | class BrowserToolParameters(BaseModel): | | 4 | """Parameters for browser interaction commands.""" | | 5 | | | 6 | command: Literal[ | | 7 | "go_to", # Navigate to a URL | | 8 | "get_html", # Get current page HTML | | 9 | "evaluate", # Run JavaScript code | | 10 | "click", # Click on an element | | 11 | "type", # Type into an element | | 12 | "screenshot", # Take a screenshot | | 13 | "get_text", # Get text content of element | | 14 | "get_attribute", # Get attribute of element | | 15 | ] = Field( | | 16 | description="The browser command to execute. Required parameters per command:\n" | | 17 | "- go_to: requires 'url'\n" | | 18 | "- evaluate: requires 'code'\n" | | 19 | "- click: requires 'selector'\n" | | 20 | "- type: requires 'selector' and 'text'\n" | | 21 | "- get_text: requires 'selector'\n" | | 22 | "- get_attribute: requires 'selector' and 'attribute'\n" | | 23 | "- get_html: no additional parameters\n" | | 24 | "- screenshot: no additional parameters" | | 25 | ) | | 26 | url: Optional[str] = Field( | | 27 | None, description="URL for go_to command (required for go_to)" | | 28 | ) | | 29 | selector: Optional[str] = Field( | | 30 | None, | | 31 | description="CSS selector for element operations (required for click, type, get_text, get_attribute)", | | 32 | ) | | 33 | code: Optional[str] = Field( | | 34 | None, description="JavaScript code for evaluate command (required for evaluate)" | | 35 | ) | | 36 | text: Optional[str] = Field( | | 37 | None, description="Text to type for type command (required for type)" | | 38 | ) | | 39 | timeout: Optional[int] = Field( | | 40 | 30000, description="Timeout in milliseconds for operations" | | 41 | ) | | 42 | attribute: Optional[str] = Field( | | 43 | None, | | 44 | description="Attribute name for get_attribute command (required for get_attribute)", | | 45 | ) | | 46 | | | 47 | | | 48 | class BrowserTool(Tool): | | 49 | """A browser interaction tool that allows the agent to interact with a browser.""" | | 50 | | | 51 | _instance: Union[UbuntuInstance, BrowserInstance] | | 52 | | | 53 | def __init__(self, instance: Union[UbuntuInstance, BrowserInstance]) -> None: | | 54 | super().__init__( | | 55 | name="browser", | | 56 | description="Interact with a browser for web scraping and automation", | | 57 | parameters=BrowserToolParameters, | | 58 | ) | | 59 | self._instance = instance | | 60 | | | 61 | def __call__(self, **kwargs: Any) -> Any: | | 62 | params = BrowserToolParameters.model_validate(kwargs) | | 63 | command = params.command | | 64 | url = params.url | | 65 | selector = params.selector | | 66 | code = params.code | | 67 | text = params.text | | 68 | timeout = params.timeout or 30000 | | 69 | attribute = params.attribute | | 70 | | | 71 | cdp_url = self._instance.browser.get_cdp_url().cdp_url | | 72 | if cdp_url is None: | | 73 | raise ValueError("CDP URL is not available, start the browser first") | | 74 | | | 75 | with sync_playwright() as playwright: | | 76 | browser = playwright.chromium.connect_over_cdp(cdp_url) | | 77 | context = browser.contexts[0] | | 78 | if not context.pages: | | 79 | page = context.new_page() | | 80 | else: | | 81 | page = context.pages[0] | | 82 | | | 83 | try: | | 84 | if command == "go_to": | | 85 | if not url: | | 86 | raise ValueError("URL is required for go_to command") | | 87 | page.goto(url, timeout=timeout) | | 88 | return True | | 89 | | | 90 | elif command == "get_html": | | 91 | try: | | 92 | return page.evaluate("() => document.documentElement.outerHTML") | | 93 | except Exception: | | 94 | # If page is navigating, just return what we can get | | 95 | return page.evaluate("() => document.documentElement.innerHTML") | | 96 | | | 97 | elif command == "evaluate": | | 98 | if not code: | | 99 | raise ValueError("Code is required for evaluate command") | | 100 | return page.evaluate(code) | | 101 | | | 102 | elif command == "click": | | 103 | if not selector: | | 104 | raise ValueError("Selector is required for click command") | | 105 | page.click(selector, timeout=timeout) | | 106 | return True | | 107 | | | 108 | elif command == "type": | | 109 | if not selector: | | 110 | raise ValueError("Selector is required for type command") | | 111 | if not text: | | 112 | raise ValueError("Text is required for type command") | | 113 | page.type(selector, text, timeout=timeout) | | 114 | return True | | 115 | | | 116 | elif command == "screenshot": | | 117 | return image_result( | | 118 | base64.b64encode(page.screenshot(type="png")).decode("utf-8") | | 119 | ) | | 120 | | | 121 | elif command == "get_text": | | 122 | if not selector: | | 123 | raise ValueError("Selector is required for get_text command") | | 124 | element = page.wait_for_selector(selector, timeout=timeout) | | 125 | if element is None: | | 126 | raise ValueError(f"Element not found: {selector}") | | 127 | return element.text_content() | | 128 | | | 129 | elif command == "get_attribute": | | 130 | if not selector: | | 131 | raise ValueError( | | 132 | "Selector is required for get_attribute command" | | 133 | ) | | 134 | if not attribute: | | 135 | raise ValueError( | | 136 | "Attribute is required for get_attribute command" | | 137 | ) | | 138 | element = page.wait_for_selector(selector, timeout=timeout) | | 139 | if element is None: | | 140 | raise ValueError(f"Element not found: {selector}") | | 141 | return element.get_attribute(attribute) | | 142 | | | 143 | else: | | 144 | raise ValueError(f"Unknown command: {command}") | | 145 | | | 146 | except Exception as e: | | 147 | raise ValueError(f"Browser command failed: {str(e)}") | | 148 | | | 149 | finally: | | 150 | browser.close() | ` --- # Ubuntu | Scrapybara Docs UbuntuInstance -------------- The `UbuntuInstance` is a Ubuntu 22.04 desktop that supports interactive streaming, computer actions, bash commands, filesystem management, built-in Jupyter notebooks, and Chromium browser support. We recommend using this instance type for most tasks. * Fast start up time * 1x compute cost Start an Ubuntu instance ------------------------ ###### Python ###### TypeScript ` | | | | --- | --- | | 1 | instance = client.start_ubuntu() | ` Available actions ----------------- ### screenshot Take a base64 encoded image of the current desktop ###### Python ###### TypeScript ` | | | | --- | --- | | 1 | base_64_image = instance.screenshot().base_64_image | ` ### get\_stream\_url Get the interactive stream URL ###### Python ###### TypeScript ` | | | | --- | --- | | 1 | stream_url = instance.get_stream_url().stream_url | ` ### computer Perform computer actions with the mouse and keyboard #### `move_mouse` Move mouse cursor to specific coordinates Parameters: * `coordinates`: \[x, y\] coordinates to move to (required) * `hold_keys`: List of modifier keys to hold during the action (optional) ###### Python ###### TypeScript Move mouse ` | | | | --- | --- | | 1 | instance.computer(action="move_mouse", coordinates=[100, 200]) | ` Move mouse while holding shift ` | | | | --- | --- | | 1 | instance.computer(action="move_mouse", coordinates=[100, 200], hold_keys=["shift"]) | ` #### `click_mouse` Perform a mouse click at current position or specified coordinates Parameters: * `button`: Mouse button to click (“left”, “right”, “middle”, “back”, “forward”) (required) * `click_type`: Type of click action (“down”, “up”, “click”) (optional, defaults to “click”) * `coordinates`: \[x, y\] coordinates to click at (optional) * `num_clicks`: Number of clicks (optional, defaults to 1) * `hold_keys`: List of modifier keys to hold during the action (optional) ###### Python ###### TypeScript Left click at current position ` | | | | --- | --- | | 1 | instance.computer(action="click_mouse", button="left") | ` Right click at coordinates ` | | | | --- | --- | | 1 | instance.computer(action="click_mouse", button="right", coordinates=[300, 400]) | ` Mouse down ` | | | | --- | --- | | 1 | instance.computer(action="click_mouse", button="left", click_type="down") | ` Double click at coordinates ` | | | | --- | --- | | 1 | instance.computer(action="click_mouse", button="left", num_clicks=2, coordinates=[500, 300]) | ` #### `drag_mouse` Click and drag from current position to specified coordinates Parameters: * `path`: List of \[x, y\] coordinate pairs defining the drag path (required) * `hold_keys`: List of modifier keys to hold during the action (optional) ###### Python ###### TypeScript Drag to coordinates ` | | | | --- | --- | | 1 | instance.computer(action="drag_mouse", path=[[100, 200], [300, 400]]) | ` #### `scroll` Scroll horizontally and/or vertically Parameters: * `coordinates`: \[x, y\] coordinates to scroll at (optional) * `delta_x`: Horizontal scroll amount (optional, defaults to 0) * `delta_y`: Vertical scroll amount (optional, defaults to 0) * `hold_keys`: List of modifier keys to hold during the action (optional) ###### Python ###### TypeScript Scroll down ` | | | | --- | --- | | 1 | instance.computer(action="scroll", coordinates=[100, 100], delta_x=0, delta_y=200) | ` Scroll right ` | | | | --- | --- | | 1 | instance.computer(action="scroll", coordinates=[100, 100], delta_x=200, delta_y=0) | ` #### `press_key` Press a key or combination of keys. Scrapybara supports keys defined by [X keysyms](https://github.com/D-Programming-Deimos/libX11/blob/master/c/X11/keysymdef.h) . Common aliases are also supported: * `alt` → `Alt_L` * `ctrl`, `control` → `Control_L` * `meta` → `Meta_L` * `super` → `Super_L` * `shift` → `Shift_L` * `enter`, `return` → `Return` Parameters: * `keys`: List of keys to press (required) * `duration`: Time to hold keys in seconds (optional) ###### Python ###### TypeScript Press ctrl+c ` | | | | --- | --- | | 1 | instance.computer(action="press_key", keys=["ctrl", "c"]) | ` Hold shift for 2 seconds ` | | | | --- | --- | | 1 | instance.computer(action="press_key", keys=["shift"], duration=2) | ` Press enter/return ` | | | | --- | --- | | 1 | instance.computer(action="press_key", keys=["Return"]) | ` #### `type_text` Type text into the active window Parameters: * `text`: Text to type (required) * `hold_keys`: List of modifier keys to hold while typing (optional) ###### Python ###### TypeScript Type text ` | | | | --- | --- | | 1 | instance.computer(action="type_text", text="Hello world") | ` #### `wait` Wait for a specified duration Parameters: * `duration`: Time to wait in seconds (required) ###### Python ###### TypeScript Wait for 3 seconds ` | | | | --- | --- | | 1 | instance.computer(action="wait", duration=3) | ` #### `take_screenshot` Take a screenshot of the desktop ###### Python ###### TypeScript ` | | | | --- | --- | | 1 | screenshot = instance.computer(action="take_screenshot").base64_image | ` #### `get_cursor_position` Get current mouse cursor coordinates ###### Python ###### TypeScript ` | | | | --- | --- | | 1 | cursor_position = instance.computer(action="get_cursor_position").output | ` ### bash Run a bash command ###### Python ###### TypeScript Run a bash command ` | | | | --- | --- | | 1 | output = instance.bash(command="ls -la") | ` Restart the shell ` | | | | --- | --- | | 1 | instance.bash(restart=True) | ` ### edit Edit a file on the instance ###### Python ###### TypeScript Create a new file ` | | | | --- | --- | | 1 | instance.edit(command="create", path="hello.txt", file_text="Hello world") | ` Replace text in a file ` | | | | --- | --- | | 1 | instance.edit(command="str_replace", path="hello.txt", old_str="Hello", new_str="Hi") | ` Insert text at a specific line ` | | | | --- | --- | | 1 | instance.edit(command="insert", path="hello.txt", insert_line=2, file_text="New line") | ` ### `stop` Stop the instance ###### Python ###### TypeScript ` | | | | --- | --- | | 1 | instance.stop() | ` ### `pause` Pause the instance ###### Python ###### TypeScript ` | | | | --- | --- | | 1 | instance.pause() | ` ### `resume` Resume the instance ###### Python ###### TypeScript Resume with default timeout ` | | | | --- | --- | | 1 | instance.resume() | ` Resume with custom timeout ` | | | | --- | --- | | 1 | instance.resume(timeout_hours=2.5) | ` Compatible tools ---------------- * `BashTool` * `ComputerTool` * `EditTool` * `BrowserTool` Additional protocols -------------------- The Ubuntu instance supports several protocols that provide additional functionality: * [Browser](/protocols/browser) - Control the browser with Playwright * [Code Execution](/protocols/code) - Execute code in Python and JavaScript * [Environment Variables](/protocols/env) - Manage environment variables * [Filesystem](/protocols/file) - Read, write, upload, and download files --- # Anthropic | Scrapybara Docs Act SDK ------- Use Anthropic models with the Act SDK: * Default: `claude-3-7-sonnet-20250219` (computer use beta) * `claude-3-7-sonnet-20250219-thinking` (computer use beta and extended thinking) * `claude-3-5-sonnet-20241022` (computer use beta) Consume agent credits or bring your own API key. Without an API key, each step consumes 1 [agent credit](https://scrapybara.com/#pricing) . With your own API key, model charges are billed directly to your Anthropic account. ###### Python ###### TypeScript Import model ` | | | | --- | --- | | 1 | from scrapybara.anthropic import Anthropic, UBUNTU_SYSTEM_PROMPT | | 2 | | | 3 | # Consume agent credits | | 4 | model = Anthropic() | | 5 | | | 6 | # Bring your own API key | | 7 | model = Anthropic(api_key="your_api_key") | | 8 | | | 9 | # Use extended thinking | | 10 | model = Anthropic(name="claude-3-7-sonnet-20250219-thinking") | ` Take action ` | | | | --- | --- | | 1 | from scrapybara import Scrapybara | | 2 | from scrapybara.tools import BashTool, ComputerTool, EditTool | | 3 | | | 4 | client = Scrapybara() | | 5 | instance = client.start_ubuntu() | | 6 | | | 7 | client.act( | | 8 | tools=[ | | 9 | BashTool(instance), | | 10 | ComputerTool(instance), | | 11 | EditTool(instance), | | 12 | ], | | 13 | model=model, | | 14 | system=UBUNTU_SYSTEM_PROMPT, | | 15 | prompt="Research Scrapybara", | | 16 | ) | ` --- # Browser | Scrapybara Docs ###### Python ###### TypeScript [1](/protocols/browser#start-an-instance) ### Start an instance ` | | | | --- | --- | | 1 | from scrapybara import Scrapybara | | 2 | | | 3 | client = Scrapybara(api_key="your_api_key") | | 4 | instance = client.start_ubuntu() | ` [2](/protocols/browser#start-a-browser-session) ### Start a browser session ` | | | | --- | --- | | 1 | cdp_url = instance.browser.start().cdp_url | ` [3](/protocols/browser#authenticate-the-session-optional) ### Authenticate the session (optional) To save the authenticated state of a browser session, use the `saveAuth` method. ` | | | | --- | --- | | 1 | auth_state_id = instance.browser.save_auth(name="default").auth_state_id | ` Now, you can reuse the saved auth state on other instances by passing the `auth_state_id` to the `authenticate` method. The browser needs to be started first. ` | | | | --- | --- | | 1 | instance.browser.authenticate(auth_state_id=auth_state_id) | ` [4](/protocols/browser#connect-to-the-browser) ### Connect to the browser ` | | | | --- | --- | | 1 | from playwright.sync_api import sync_playwright | | 2 | | | 3 | playwright = sync_playwright().start() | | 4 | browser = playwright.chromium.connect_over_cdp(cdp_url) | ` [5](/protocols/browser#interact-with-the-browser) ### Interact with the browser ` | | | | --- | --- | | 1 | page = browser.new_page() | | 2 | page.goto("https://scrapybara.com") | | 3 | screenshot = page.screenshot() | ` [6](/protocols/browser#stop-the-browser-session) ### Stop the browser session ` | | | | --- | --- | | 1 | instance.browser.stop() | ` --- # Environment Variables | Scrapybara Docs ###### Python ###### TypeScript [1](/protocols/env#start-an-instance) ### Start an instance ` | | | | --- | --- | | 1 | from scrapybara import Scrapybara | | 2 | | | 3 | client = Scrapybara(api_key="your_api_key") | | 4 | instance = client.start_ubuntu() | ` [2](/protocols/env#set-environment-variables) ### Set environment variables ` | | | | --- | --- | | 1 | # Set one or more environment variables | | 2 | instance.env.set( | | 3 | variables={ | | 4 | "API_KEY": "secret_key", | | 5 | "DEBUG": "true", | | 6 | "DATABASE_URL": "postgresql://localhost:5432/db" | | 7 | } | | 8 | ) | ` [3](/protocols/env#get-environment-variables) ### Get environment variables ` | | | | --- | --- | | 1 | # Get all environment variables | | 2 | response = instance.env.get() | | 3 | env_vars = response.variables | ` [4](/protocols/env#delete-environment-variables) ### Delete environment variables ` | | | | --- | --- | | 1 | # Delete specific environment variables | | 2 | instance.env.delete( | | 3 | keys=["API_KEY", "DEBUG"] | | 4 | ) | ` --- # Filesystem | Scrapybara Docs ###### Python ###### TypeScript [1](/protocols/file#start-an-instance) ### Start an instance ` | | | | --- | --- | | 1 | from scrapybara import Scrapybara | | 2 | | | 3 | client = Scrapybara(api_key="your_api_key") | | 4 | instance = client.start_ubuntu() | ` [2](/protocols/file#write-a-file) ### Write a file ` | | | | --- | --- | | 1 | # Write content to a file | | 2 | instance.file.write( | | 3 | path="/path/to/file.txt", | | 4 | content="Hello from Scrapybara!", | | 5 | encoding="utf-8" # Optional: specify encoding | | 6 | ) | ` [3](/protocols/file#read-a-file) ### Read a file ` | | | | --- | --- | | 1 | # Read file content | | 2 | response = instance.file.read( | | 3 | path="/path/to/file.txt", | | 4 | encoding="utf-8" # Optional: specify encoding | | 5 | ) | | 6 | content = response.content | ` [4](/protocols/file#upload-a-file) ### Upload a file ` | | | | --- | --- | | 1 | # Upload a file to the instance | | 2 | instance.file.upload( | | 3 | path="/destination/path/file.txt", | | 4 | content="file_content_as_string" | | 5 | ) | ` [5](/protocols/file#download-a-file) ### Download a file ` | | | | --- | --- | | 1 | # Download a file from the instance | | 2 | response = instance.file.download( | | 3 | path="/path/to/file.txt" | | 4 | ) | | 5 | downloaded_content = response.content | ` --- # Code Execution | Scrapybara Docs ###### Python ###### TypeScript [1](/protocols/code#start-an-instance) ### Start an instance ` | | | | --- | --- | | 1 | from scrapybara import Scrapybara | | 2 | | | 3 | client = Scrapybara(api_key="your_api_key") | | 4 | instance = client.start_ubuntu() | ` [2](/protocols/code#execute-code-directly) ### Execute code directly ` | | | | --- | --- | | 1 | result = instance.code.execute( | | 2 | code="print('Hello from Scrapybara!')", | | 3 | kernel_name="python3" # Optional: specify kernel | | 4 | ) | ` [3](/protocols/code#list-available-kernels) ### List available kernels ` | | | | --- | --- | | 1 | kernels = instance.notebook.list_kernels() | ` [4](/protocols/code#create-a-notebook) ### Create a notebook ` | | | | --- | --- | | 1 | notebook = instance.notebook.create( | | 2 | name="my_notebook", | | 3 | kernel_name="python3" | | 4 | ) | ` [5](/protocols/code#add-and-execute-cells) ### Add and execute cells ` | | | | --- | --- | | 1 | # Add a code cell | | 2 | cell = instance.notebook.add_cell( | | 3 | notebook_id=notebook.id, | | 4 | type="code", | | 5 | content="print('Hello from Scrapybara!')" | | 6 | ) | | 7 | | | 8 | # Execute the cell | | 9 | result = instance.notebook.execute_cell( | | 10 | notebook_id=notebook.id, | | 11 | cell_id=cell.id | | 12 | ) | ` [6](/protocols/code#execute-entire-notebook) ### Execute entire notebook ` | | | | --- | --- | | 1 | # Execute all cells in the notebook | | 2 | results = instance.notebook.execute(notebook_id=notebook.id) | ` [7](/protocols/code#clean-up) ### Clean up ` | | | | --- | --- | | 1 | # Delete the notebook when done | | 2 | instance.notebook.delete(notebook_id=notebook.id) | ` --- # Windows | Scrapybara Docs Windows instances are in early access. Join our [Discord](https://discord.gg/s4bPUVFXqA) to get started. WindowsInstance --------------- The `WindowsInstance` is a full-fledged Windows 11 desktop that supports interactive streaming and computer actions. We recommend using this instance type if you need to interact with Windows-only applications. * Slow start up time * 2x compute cost Start a Windows instance ------------------------ ###### Python ###### TypeScript ` | | | | --- | --- | | 1 | instance = client.start_windows() | ` Available actions ----------------- ### screenshot Take a base64 encoded image of the current desktop ###### Python ###### TypeScript ` | | | | --- | --- | | 1 | base_64_image = instance.screenshot().base_64_image | ` ### get\_stream\_url Get the interactive stream URL ###### Python ###### TypeScript ` | | | | --- | --- | | 1 | stream_url = instance.get_stream_url().stream_url | ` ### computer Perform computer actions with the mouse and keyboard #### `move_mouse` Move mouse cursor to specific coordinates Parameters: * `coordinates`: \[x, y\] coordinates to move to (required) * `hold_keys`: List of modifier keys to hold during the action (optional, defaults to \[\]) ###### Python ###### TypeScript Move mouse ` | | | | --- | --- | | 1 | instance.computer(action="move_mouse", coordinates=[100, 200]) | ` Move mouse while holding shift ` | | | | --- | --- | | 1 | instance.computer(action="move_mouse", coordinates=[100, 200], hold_keys=["shift"]) | ` #### `click_mouse` Perform a mouse click at current position or specified coordinates Parameters: * `button`: Mouse button to click (“left”, “right”, “middle”, “back”, “forward”) (required) * `click_type`: Type of click action (“down”, “up”, “click”) (optional, defaults to “click”) * `coordinates`: \[x, y\] coordinates to click at (optional) * `num_clicks`: Number of clicks (optional, defaults to 1) * `hold_keys`: List of modifier keys to hold during the action (optional, defaults to \[\]) ###### Python ###### TypeScript Left click at current position ` | | | | --- | --- | | 1 | instance.computer(action="click_mouse", button="left") | ` Right click at coordinates ` | | | | --- | --- | | 1 | instance.computer(action="click_mouse", button="right", coordinates=[300, 400]) | ` Mouse down ` | | | | --- | --- | | 1 | instance.computer(action="click_mouse", button="left", click_type="down") | ` Double click at coordinates ` | | | | --- | --- | | 1 | instance.computer(action="click_mouse", button="left", num_clicks=2, coordinates=[500, 300]) | ` #### `drag_mouse` Click and drag from current position to specified coordinates Parameters: * `path`: List of \[x, y\] coordinate pairs defining the drag path (required) * `hold_keys`: List of modifier keys to hold during the action (optional, defaults to \[\]) ###### Python ###### TypeScript Drag to coordinates ` | | | | --- | --- | | 1 | instance.computer(action="drag_mouse", path=[[100, 200], [300, 400]]) | ` #### `scroll` Scroll horizontally and/or vertically Parameters: * `coordinates`: \[x, y\] coordinates to scroll at (required) * `delta_x`: Horizontal scroll amount (optional, defaults to 0) * `delta_y`: Vertical scroll amount (optional, defaults to 0) * `hold_keys`: List of modifier keys to hold during the action (optional, defaults to \[\]) ###### Python ###### TypeScript Scroll down ` | | | | --- | --- | | 1 | instance.computer(action="scroll", coordinates=[100, 100], delta_x=0, delta_y=200) | ` Scroll right ` | | | | --- | --- | | 1 | instance.computer(action="scroll", coordinates=[100, 100], delta_x=200, delta_y=0) | ` #### `press_key` Press a key or combination of keys. Key names can be found in [X11 keysymdef.h](https://github.com/D-Programming-Deimos/libX11/blob/master/c/X11/keysymdef.h) . Common aliases are also supported: * `alt` → `Alt_L` * `ctrl`, `control` → `Control_L` * `meta` → `Meta_L` * `super` → `Super_L` * `shift` → `Shift_L` * `enter`, `return` → `Return` Parameters: * `keys`: List of keys to press (required) * `duration`: Time to hold keys in seconds (optional) ###### Python ###### TypeScript Press ctrl+c ` | | | | --- | --- | | 1 | instance.computer(action="press_key", keys=["ctrl", "c"]) | ` Hold shift for 2 seconds ` | | | | --- | --- | | 1 | instance.computer(action="press_key", keys=["shift"], duration=2) | ` Press enter/return ` | | | | --- | --- | | 1 | instance.computer(action="press_key", keys=["Return"]) | ` #### `type_text` Type text into the active window Parameters: * `text`: Text to type (required) * `hold_keys`: List of modifier keys to hold while typing (optional) ###### Python ###### TypeScript Type text ` | | | | --- | --- | | 1 | instance.computer(action="type_text", text="Hello world") | ` #### `wait` Wait for a specified duration Parameters: * `duration`: Time to wait in seconds (required) ###### Python ###### TypeScript Wait for 3 seconds ` | | | | --- | --- | | 1 | instance.computer(action="wait", duration=3) | ` #### `take_screenshot` Take a screenshot of the desktop ###### Python ###### TypeScript ` | | | | --- | --- | | 1 | screenshot = instance.computer(action="take_screenshot").base64_image | ` #### `get_cursor_position` Get current mouse cursor coordinates ###### Python ###### TypeScript ` | | | | --- | --- | | 1 | cursor_position = instance.computer(action="get_cursor_position").output | ` ### `stop` Stop the instance ###### Python ###### TypeScript ` | | | | --- | --- | | 1 | instance.stop() | ` ### `pause` Pause the instance ###### Python ###### TypeScript ` | | | | --- | --- | | 1 | instance.pause() | ` ### `resume` Resume the instance ###### Python ###### TypeScript Resume with default timeout ` | | | | --- | --- | | 1 | instance.resume() | ` Resume with custom timeout ` | | | | --- | --- | | 1 | instance.resume(timeout_hours=2.5) | ` Compatible tools ---------------- * `ComputerTool` --- # Starter Templates | Scrapybara Docs Python ------ [GitHub\ \ View the template](https://github.com/Scrapybara/scrapybara-python-template) TypeScript ---------- [GitHub\ \ View the template](https://github.com/Scrapybara/scrapybara-ts-template) --- # Cursor Rules | Scrapybara Docs .cursorrules ------------ ###### Python ###### TypeScript .cursorrules ` | | | | --- | --- | | 1 | You are working with Scrapybara, a Python SDK for deploying and managing remote desktop instances for AI agents. Use this guide to properly interact with the SDK. | | 2 | | | 3 | **CORE SDK USAGE:** | | 4 | - Initialize client: from scrapybara import Scrapybara; client = Scrapybara(api_key="KEY") | | 5 | - Instance lifecycle: | | 6 | instance = client.start_ubuntu(timeout_hours=1) | | 7 | instance.pause() # Pause to save resources | | 8 | instance.resume(timeout_hours=1) # Resume work | | 9 | instance.stop() # Terminate and clean up | | 10 | - Instance types: | | 11 | ubuntu_instance = client.start_ubuntu(): supports bash, computer, edit, browser | | 12 | browser_instance = client.start_browser(): supports computer, browser | | 13 | windows_instance = client.start_windows(): supports computer | | 14 | | | 15 | **TYPE IMPORTS:** | | 16 | - Core types: | | 17 | from scrapybara import Scrapybara | | 18 | - Instance types: | | 19 | from scrapybara.client import UbuntuInstance, BrowserInstance, WindowsInstance | | 20 | - Tool types: | | 21 | from scrapybara.tools import Tool, BashTool, ComputerTool, EditTool | | 22 | - Model types: | | 23 | from scrapybara.anthropic import Anthropic | | 24 | - Message types: | | 25 | from pydantic import BaseModel | | 26 | from typing import List, Union, Optional, Any | | 27 | - Error types: | | 28 | from scrapybara.core.api_error import ApiError | | 29 | | | 30 | **CORE INSTANCE OPERATIONS:** | | 31 | - Screenshots: instance.screenshot().base_64_image | | 32 | - Bash commands: instance.bash(command="ls -la") | | 33 | - Mouse control: instance.computer(action="move_mouse", coordinates=[x, y]) | | 34 | - Click actions: instance.computer(action="click_mouse", button="right", coordinates=[x, y]) | | 35 | - Drag actions: instance.computer(action="drag_mouse", path=[[x1, y1], [x2, y2]]) | | 36 | - Scroll actions: instance.computer(action="scroll", coordinates=[x, y], delta_x=0, delta_y=0) | | 37 | - Key actions: instance.computer(action="press_key", keys=[keys]) | | 38 | - Type actions: instance.computer(action="type_text", text="Hello world") | | 39 | - Wait actions: instance.computer(action="wait", duration=3) | | 40 | - Get cursor position: instance.computer(action="get_cursor_position").output | | 41 | - File operations: instance.file.read(path="/path/file"), instance.file.write(path="/path/file", content="data") | | 42 | | | 43 | **ACT SDK (Primary Focus):** | | 44 | - Purpose: Enables building computer use agents with unified tools and model interfaces | | 45 | - Core components: | | 46 | 1. Model: Handles LLM integration (currently Anthropic) | | 47 | from scrapybara.anthropic import Anthropic | | 48 | model = Anthropic() # Or model = Anthropic(api_key="KEY") for own key | | 49 | 2. Tools: Interface for computer interactions | | 50 | - BashTool: Run shell commands | | 51 | - ComputerTool: Mouse/keyboard control | | 52 | - EditTool: File operations | | 53 | tools = [ | | 54 | BashTool(instance), | | 55 | ComputerTool(instance), | | 56 | EditTool(instance), | | 57 | ] | | 58 | 3. Prompt: | | 59 | - system: system prompt, recommend to use UBUNTU_SYSTEM_PROMPT, BROWSER_SYSTEM_PROMPT, WINDOWS_SYSTEM_PROMPT | | 60 | - prompt: simple user prompt | | 61 | - messages: list of messages | | 62 | - Only include either prompt or messages, not both | | 63 | response = client.act( | | 64 | model=Anthropic(), | | 65 | tools=tools, | | 66 | system=UBUNTU_SYSTEM_PROMPT, | | 67 | prompt="Task", | | 68 | on_step=handle_step | | 69 | ) | | 70 | messages = response.messages | | 71 | steps = response.steps | | 72 | text = response.text | | 73 | output = response.output | | 74 | usage = response.usage | | 75 | | | 76 | **MESSAGE HANDLING:** | | 77 | - Response Structure: Messages are structured with roles (user/assistant/tool) and typed content | | 78 | - Content Types: | | 79 | - TextPart: Simple text content | | 80 | TextPart(type="text", text="content") | | 81 | - ImagePart: Base64 or URL images | | 82 | ImagePart(type="image", image="base64...", mime_type="image/png") | | 83 | - ReasoningPart: Model reasoning content | | 84 | ReasoningPart( | | 85 | type="reasoning", | | 86 | id="id", | | 87 | reasoning="reasoning", | | 88 | signature="signature", | | 89 | instructions="instructions" | | 90 | ) | | 91 | - ToolCallPart: Tool invocations | | 92 | ToolCallPart( | | 93 | type="tool-call", | | 94 | tool_call_id="id", | | 95 | tool_name="bash", | | 96 | args={"command": "ls"} | | 97 | ) | | 98 | - ToolResultPart: Tool execution results | | 99 | ToolResultPart( | | 100 | type="tool-result", | | 101 | tool_call_id="id", | | 102 | tool_name="bash", | | 103 | result="output", | | 104 | is_error=False | | 105 | ) | | 106 | | | 107 | **STEP HANDLING:** | | 108 | def handle_step(step: Step): | | 109 | if step.reasoning_parts: | | 110 | print(f"Reasoning: {step.reasoning_parts}") | | 111 | if step.text: | | 112 | print(f"Text: {step.text}") | | 113 | if step.tool_calls: | | 114 | for call in step.tool_calls: | | 115 | print(f"Tool: {call.tool_name}") | | 116 | if step.tool_results: | | 117 | for result in step.tool_results: | | 118 | print(f"Result: {result.result}") | | 119 | print(f"Tokens: {step.usage.total_tokens if step.usage else 'N/A'}") | | 120 | | | 121 | **STRUCTURED OUTPUT:** | | 122 | Use the schema parameter to define a desired structured output. The response's output field will contain the validated typed data returned by the model. | | 123 | class HNSchema(BaseModel): | | 124 | class Post(BaseModel): | | 125 | title: str | | 126 | url: str | | 127 | points: int | | 128 | posts: List[Post] | | 129 | response = client.act( | | 130 | model=Anthropic(), | | 131 | tools=tools, | | 132 | schema=HNSchema, | | 133 | system=SYSTEM_PROMPT, | | 134 | prompt="Get the top 10 posts on Hacker News", | | 135 | ) | | 136 | posts = response.output.posts | | 137 | | | 138 | **TOKEN USAGE:** | | 139 | - Track token usage through TokenUsage objects | | 140 | - Fields: prompt_tokens, completion_tokens, total_tokens | | 141 | - Available in both Step and ActResponse objects | | 142 | | | 143 | **EXAMPLE:** | | 144 | from scrapybara import Scrapybara | | 145 | from scrapybara.anthropic import Anthropic | | 146 | from scrapybara.prompts import UBUNTU_SYSTEM_PROMPT | | 147 | from scrapybara.tools import BashTool, ComputerTool, EditTool | | 148 | | | 149 | client = Scrapybara() | | 150 | instance = client.start_ubuntu() | | 151 | instance.browser.start() | | 152 | | | 153 | response = client.act( | | 154 | model=Anthropic(), | | 155 | tools=[ | | 156 | BashTool(instance), | | 157 | ComputerTool(instance), | | 158 | EditTool(instance), | | 159 | ], | | 160 | system=UBUNTU_SYSTEM_PROMPT, | | 161 | prompt="Go to the YC website and fetch the HTML", | | 162 | on_step=lambda step: print(f"{step}\n"), | | 163 | ) | | 164 | messages = response.messages | | 165 | steps = response.steps | | 166 | text = response.text | | 167 | output = response.output | | 168 | usage = response.usage | | 169 | | | 170 | instance.browser.stop() | | 171 | instance.stop() | | 172 | | | 173 | **EXECUTION PATTERNS:** | | 174 | 1. Basic agent execution: | | 175 | response = client.act( | | 176 | model=Anthropic(), | | 177 | tools=tools, | | 178 | system="System context here", | | 179 | prompt="Task description" | | 180 | ) | | 181 | 2. Browser automation: | | 182 | cdp_url = instance.browser.start().cdp_url | | 183 | auth_state_id = instance.browser.save_auth(name="default").auth_state_id # Save auth | | 184 | instance.browser.authenticate(auth_state_id=auth_state_id) # Reuse auth | | 185 | 3. File management: | | 186 | instance.file.write("/tmp/data.txt", "content") | | 187 | content = instance.file.read("/tmp/data.txt").content | | 188 | 4. Environment variables: | | 189 | instance.env.set({"API_KEY": "value"}) | | 190 | instance.env.get().variables | | 191 | instance.env.delete(["VAR_NAME"]) | | 192 | | | 193 | **ERROR HANDLING:** | | 194 | from scrapybara.core.api_error import ApiError | | 195 | try: | | 196 | client.start_ubuntu() | | 197 | except ApiError as e: | | 198 | print(f"Error {e.status_code}: {e.body}") | | 199 | | | 200 | **IMPORTANT GUIDELINES:** | | 201 | - Always stop instances after use to prevent unnecessary billing | | 202 | - Use async client (AsyncScrapybara) for non-blocking operations | | 203 | - Handle API errors with try/except ApiError blocks | | 204 | - Default timeout is 60s; customize with timeout parameter or request_options | | 205 | - Instance auto-terminates after 1 hour by default | | 206 | - For browser operations, always start browser before BrowserTool usage | | 207 | - Prefer bash commands over GUI interactions for launching applications | ` llms-full.txt ------------- Need more context? Check out [llms-full.txt](/llms-full.txt) . --- # Browser | Scrapybara Docs BrowserInstance --------------- The `BrowserInstance` is a lightweight Chromium instance that supports interactive streaming, computer actions, Playwright CDP control, and saving/loading auth states. We recommend using this instance type if your task is constrained to the browser. * Fastest start up time * 1x compute cost Start a browser instance ------------------------ ###### Python ###### TypeScript ` | | | | --- | --- | | 1 | instance = client.start_browser() | ` Available actions ----------------- ### get\_cdp\_url Get the Playwright CDP URL ###### Python ###### TypeScript ` | | | | --- | --- | | 1 | cdp_url = instance.get_cdp_url().cdp_url | ` ### save\_auth Save the browser auth state ###### Python ###### TypeScript ` | | | | --- | --- | | 1 | auth_state_id = instance.browser.save_auth(name="default").auth_state_id | ` ### authenticate Authenticate the browser using a saved auth state ###### Python ###### TypeScript ` | | | | --- | --- | | 1 | instance.browser.authenticate(auth_state_id=auth_state_id) | ` ### screenshot Take a base64 encoded image of the current desktop ###### Python ###### TypeScript ` | | | | --- | --- | | 1 | base_64_image = instance.screenshot().base_64_image | ` ### get\_stream\_url Get the interactive stream URL ###### Python ###### TypeScript ` | | | | --- | --- | | 1 | stream_url = instance.get_stream_url().stream_url | ` ### computer Perform computer actions with the mouse and keyboard #### `move_mouse` Move mouse cursor to specific coordinates Parameters: * `coordinates`: \[x, y\] coordinates to move to (required) * `hold_keys`: List of modifier keys to hold during the action (optional, defaults to \[\]) ###### Python ###### TypeScript Move mouse ` | | | | --- | --- | | 1 | instance.computer(action="move_mouse", coordinates=[100, 200]) | ` Move mouse while holding shift ` | | | | --- | --- | | 1 | instance.computer(action="move_mouse", coordinates=[100, 200], hold_keys=["shift"]) | ` #### `click_mouse` Perform a mouse click at current position or specified coordinates Parameters: * `button`: Mouse button to click (“left”, “right”, “middle”, “back”, “forward”) (required) * `click_type`: Type of click action (“down”, “up”, “click”) (optional, defaults to “click”) * `coordinates`: \[x, y\] coordinates to click at (optional) * `num_clicks`: Number of clicks (optional, defaults to 1) * `hold_keys`: List of modifier keys to hold during the action (optional, defaults to \[\]) ###### Python ###### TypeScript Left click at current position ` | | | | --- | --- | | 1 | instance.computer(action="click_mouse", button="left") | ` Right click at coordinates ` | | | | --- | --- | | 1 | instance.computer(action="click_mouse", button="right", coordinates=[300, 400]) | ` Mouse down ` | | | | --- | --- | | 1 | instance.computer(action="click_mouse", button="left", click_type="down") | ` Double click at coordinates ` | | | | --- | --- | | 1 | instance.computer(action="click_mouse", button="left", num_clicks=2, coordinates=[500, 300]) | ` #### `drag_mouse` Click and drag from current position to specified coordinates Parameters: * `path`: List of \[x, y\] coordinate pairs defining the drag path (required) * `hold_keys`: List of modifier keys to hold during the action (optional, defaults to \[\]) ###### Python ###### TypeScript Drag to coordinates ` | | | | --- | --- | | 1 | instance.computer(action="drag_mouse", path=[[100, 200], [300, 400]]) | ` #### `scroll` Scroll horizontally and/or vertically Parameters: * `coordinates`: \[x, y\] coordinates to scroll at (required) * `delta_x`: Horizontal scroll amount (optional, defaults to 0) * `delta_y`: Vertical scroll amount (optional, defaults to 0) * `hold_keys`: List of modifier keys to hold during the action (optional, defaults to \[\]) ###### Python ###### TypeScript Scroll down ` | | | | --- | --- | | 1 | instance.computer(action="scroll", coordinates=[100, 100], delta_x=0, delta_y=200) | ` Scroll right ` | | | | --- | --- | | 1 | instance.computer(action="scroll", coordinates=[100, 100], delta_x=200, delta_y=0) | ` #### `press_key` Press a key or combination of keys. Key names can be found in [X11 keysymdef.h](https://github.com/D-Programming-Deimos/libX11/blob/master/c/X11/keysymdef.h) . Common aliases are also supported: * `alt` → `Alt_L` * `ctrl`, `control` → `Control_L` * `meta` → `Meta_L` * `super` → `Super_L` * `shift` → `Shift_L` * `enter`, `return` → `Return` Parameters: * `keys`: List of keys to press (required) * `duration`: Time to hold keys in seconds (optional) ###### Python ###### TypeScript Press ctrl+c ` | | | | --- | --- | | 1 | instance.computer(action="press_key", keys=["ctrl", "c"]) | ` Hold shift for 2 seconds ` | | | | --- | --- | | 1 | instance.computer(action="press_key", keys=["shift"], duration=2) | ` Press enter/return ` | | | | --- | --- | | 1 | instance.computer(action="press_key", keys=["Return"]) | ` #### `type_text` Type text into the active window Parameters: * `text`: Text to type (required) * `hold_keys`: List of modifier keys to hold while typing (optional) ###### Python ###### TypeScript Type text ` | | | | --- | --- | | 1 | instance.computer(action="type_text", text="Hello world") | ` #### `wait` Wait for a specified duration Parameters: * `duration`: Time to wait in seconds (required) ###### Python ###### TypeScript Wait for 3 seconds ` | | | | --- | --- | | 1 | instance.computer(action="wait", duration=3) | ` #### `take_screenshot` Take a screenshot of the desktop ###### Python ###### TypeScript ` | | | | --- | --- | | 1 | screenshot = instance.computer(action="take_screenshot").base64_image | ` #### `get_cursor_position` Get current mouse cursor coordinates ###### Python ###### TypeScript ` | | | | --- | --- | | 1 | cursor_position = instance.computer(action="get_cursor_position").output | ` ### `stop` Stop the instance ###### Python ###### TypeScript ` | | | | --- | --- | | 1 | instance.stop() | ` ### `pause` Pause the instance ###### Python ###### TypeScript ` | | | | --- | --- | | 1 | instance.pause() | ` ### `resume` Resume the instance ###### Python ###### TypeScript Resume with default timeout ` | | | | --- | --- | | 1 | instance.resume() | ` Resume with custom timeout ` | | | | --- | --- | | 1 | instance.resume(timeout_hours=2.5) | ` Compatible tools ---------------- * `ComputerTool` * `BrowserTool` --- # Get instance by ID | Scrapybara Docs ### Path parameters instance\_idstringRequired ### Headers x-api-keystringRequired ### Response Successful Response idstring launch\_timedatetime instance\_typeenum Allowed values:ubuntubrowserwindows statusenum Allowed values:deployingrunningpausedterminatederror ### Errors 422 Get Request Unprocessable Entity Error --- # List all instances | Scrapybara Docs ### Headers x-api-keystringRequired ### Response Successful Response idstring launch\_timedatetime instance\_typeenum Allowed values:ubuntubrowserwindows statusenum Allowed values:deployingrunningpausedterminatederror --- # List all browser authentication states | Scrapybara Docs ### Headers x-api-keystringRequired ### Response Successful Response idstring namestringOptional --- # Take screenshot | Scrapybara Docs ### Path parameters instance\_idstringRequired ### Headers x-api-keystringRequired ### Response Successful Response base64\_imagestring ### Errors 422 Instance Screenshot Request Unprocessable Entity Error --- # Get stream URL | Scrapybara Docs ### Path parameters instance\_idstringRequired ### Headers x-api-keystringRequired ### Response Successful Response stream\_urlstring ### Errors 422 Instance Get Stream URL Request Unprocessable Entity Error --- # Run bash actions | Scrapybara Docs ### Path parameters instance\_idstringRequired ### Headers x-api-keystringRequired ### Request This endpoint expects an object. commandstringOptional restartbooleanOptional ### Response Successful Response outputstringOptional errorstringOptional base64\_imagestringOptional systemstringOptional ### Errors 422 Instance Bash Request Unprocessable Entity Error --- # Run computer actions | Scrapybara Docs ### Path parameters instance\_idstringRequired ### Headers x-api-keystringRequired ### Request This endpoint expects an object. move\_mouseobjectRequired Show 3 properties OR click\_mouseobjectRequired Show 6 properties OR drag\_mouseobjectRequired Show 3 properties OR scrollobjectRequired Show 5 properties OR press\_keyobjectRequired Show 3 properties OR type\_textobjectRequired Show 3 properties OR waitobjectRequired Show 2 properties OR take\_screenshotobjectRequired Show 1 properties OR get\_cursor\_positionobjectRequired Show 1 properties ### Response Successful Response outputstringOptional errorstringOptional base64\_imagestringOptional systemstringOptional ### Errors 422 Instance Computer Request Unprocessable Entity Error --- # Run edit actions | Scrapybara Docs ### Path parameters instance\_idstringRequired ### Headers x-api-keystringRequired ### Request This endpoint expects an object. commandenumRequired Allowed values:viewcreatestr\_replaceinsertundo\_edit pathstringRequired file\_textstringOptional view\_rangelist of integersOptional old\_strstringOptional new\_strstringOptional insert\_lineintegerOptional ### Response Successful Response outputstringOptional errorstringOptional base64\_imagestringOptional systemstringOptional ### Errors 422 Instance Edit Request Unprocessable Entity Error --- # Pause instance | Scrapybara Docs ### Path parameters instance\_idstringRequired ### Headers x-api-keystringRequired ### Response Successful Response statusstring ### Errors 422 Instance Pause Request Unprocessable Entity Error --- # Start instance | Scrapybara Docs ### Headers x-api-keystringRequired ### Request This endpoint expects an object. instance\_typeenumOptional Allowed values:ubuntubrowserwindows timeout\_hoursdoubleOptional`>=0.01``<=24` blocked\_domainslist of stringsOptional ### Response Successful Response idstring launch\_timedatetime instance\_typeenum Allowed values:ubuntubrowserwindows statusenum Allowed values:deployingrunningpausedterminatederror ### Errors 422 Start Request Unprocessable Entity Error --- # Stop instance | Scrapybara Docs ### Path parameters instance\_idstringRequired ### Headers x-api-keystringRequired ### Response Successful Response statusstring ### Errors 422 Instance Stop Request Unprocessable Entity Error --- # Resume instance | Scrapybara Docs ### Path parameters instance\_idstringRequired ### Headers x-api-keystringRequired ### Query parameters timeout\_hoursdoubleOptionalDefaults to `1` ### Response Successful Response idstring launch\_timedatetime instance\_typeenum Allowed values:ubuntubrowserwindows statusenum Allowed values:deployingrunningpausedterminatederror ### Errors 422 Instance Resume Request Unprocessable Entity Error --- # Get CDP URL | Scrapybara Docs ### Path parameters instance\_idstringRequired ### Headers x-api-keystringRequired ### Response Successful Response cdp\_urlstring ### Errors 422 Browser Get Cdp URL Request Unprocessable Entity Error --- # Start browser | Scrapybara Docs ### Path parameters instance\_idstringRequired ### Headers x-api-keystringRequired ### Response Successful Response cdp\_urlstring ### Errors 422 Browser Start Request Unprocessable Entity Error --- # Get current URL | Scrapybara Docs ### Path parameters instance\_idstringRequired ### Headers x-api-keystringRequired ### Response Successful Response current\_urlstring ### Errors 422 Browser Get Current URL Request Unprocessable Entity Error ---