Back to blog
Claude 3.5 Sonnet Computer Use: Our 48-Hour Test Results

Claude 3.5 Sonnet Computer Use: Our 48-Hour Test Results

October 27, 2026

Claude 3.5 Sonnet Computer Use: What Actually Works, What Fails, and Who Should Try It

Anthropic released an updated Claude 3.5 Sonnet with a new public beta feature called Computer Use. It lets the model look at your screen and control desktop applications directly. We spent 48 hours testing it across developer and ops workflows. Here is what works, what breaks, and whether it is ready for production.

How Computer Use Actually Works

Claude does not parse HTML or call automation APIs. It uses vision and action output.

  1. Screen Capture: The model receives a high-resolution screenshot.
  2. Coordinate Calculation: It translates visual elements into X/Y pixel coordinates.
  3. Action Execution: An open-source API layer sends mouse clicks, drags, and keystrokes.
  4. Verification Loop: The model takes another screenshot to confirm the action succeeded.

This approach can interact with any program a human can use, including legacy Windows software, desktop ERPs, and local terminals.

What Worked in Our Tests

We ran three distinct workflows over 48 hours.

Legacy ERP Data Migration

Moving client records from an outdated Windows desktop app into a modern web CRM used to require custom Python scripts or manual copy-paste. Claude handled a 15-record batch. It opened the desktop client, read pixel text from form fields, switched to Chrome, and populated the SaaS form. It completed the task without dropping a single entry.

Multi-Tab Visual QA and Layout Checks

We gave Claude a prompt to test a local web app across three breakpoint widths in Firefox. It opened Developer Tools, adjusted viewport dimensions, captured screenshots of overlapping header elements, and logged a clean markdown bug report in VS Code.

Cross-Platform Workflow Chaining

Claude opened an Excel file sent via Slack, pulled data from column C, launched a terminal window, ran a local cURL request against a staging server, and pasted the server response back into a designated Slack channel.

The Hard Limitations You Will Hit

Computer Use in its current beta state has sharp edges.

  • Pixel Drift and Misclicks: Claude sometimes miscalculates pixel coordinates on high-DPI displays or custom OS scaling. A target button at (450, 1200) occasionally gets clicked at (450, 1180), missing the element entirely.
  • Extreme Latency: Every step requires taking a screenshot, sending image data back to the Anthropic API, waiting for inference, and executing the action. A simple 5-step form fill can take up to 45 seconds.
  • Dynamic Animations: If an app features scrolling parallax effects or loading spinners, Claude struggles to verify timing and may misinterpret a pending state as success.
  • Security and Prompt Injection: Giving an AI visual access to your desktop opens terrifying security vectors. If Claude views a webpage containing hidden white-text prompt injections, it may execute those instructions with your local system privileges.

Pricing Realities for Developers

The updated Claude 3.5 Sonnet retains its previous pricing tier: $3.00 per million input tokens and $15.00 per million output tokens.

Computer Use drastically inflates token consumption. Because every screen capture is converted into visual tokens, you will spend roughly 1,000 to 1,600 tokens per screenshot depending on resolution. Running a 20-step automation script can easily consume 40,000 plus input tokens per run. Keep an eye on your API spend during local testing.

The Verdict: Who Should Use This Right Now

Computer Use is not ready to replace automated end-to-end testing suites or background workers. The latency is too high and the misclick rate is too risky for unsupervised tasks.

However, for developers building internal agentic workflows, desktop RPA tools, or accessibility software, this update changes the game. Start experimenting in sandboxed virtual machines today. Keep your production credentials far away from the agent.