Scaling to 1000 Concurrent Workflows: Requests vs. Browsers
Running 1000 concurrent browser instances requires massive infrastructure. 1000 concurrent HTTP request workflows run on a single server. The scaling economics aren't close.
TL;DR
Running 1000 concurrent browser instances requires massive infrastructure. 1000 concurrent HTTP request workflows run on a single server. The scaling economics aren't close. For production web automation, request-based approaches consistently outperform browser-based ones in speed, reliability, cost, and detectability.
The fundamental difference
Browser automation drives a real browser — rendering HTML, executing JavaScript, loading images, running CSS animations — then interacts with the rendered page through clicks and keystrokes. All of that rendering is wasted work when your goal is to submit a form or retrieve data.
HTTP request automation skips the browser entirely. It sends the same HTTP requests the browser would send, but without the overhead of rendering anything. The server doesn't know or care whether the request came from a browser or a script — it processes the request and returns a response.
Why this matters for scaling
Running 1000 concurrent browser instances requires massive infrastructure. 1000 concurrent HTTP request workflows run on a single server. The scaling economics aren't close.
This isn't a theoretical distinction. Teams that switch from browser automation to request-based automation typically see:
- 10-100x faster execution — milliseconds per request vs. seconds per page load
- 90%+ reduction in maintenance — no selectors to update, no timing issues to debug
- 95%+ reduction in infrastructure cost — no browser instances consuming hundreds of MB each
- Higher success rates — request-level automation doesn't trigger behavioral anti-bot detection
How browser automation works under the hood
When a browser automation script runs, here's what actually happens:
1. A full Chromium instance launches (200-500MB of RAM)
2. The browser navigates to a URL, downloading HTML, CSS, JavaScript, images, and fonts
3. The rendering engine parses HTML, computes CSS layout, and paints pixels
4. JavaScript executes — potentially megabytes of framework code
5. The automation script waits for specific DOM elements to appear
6. The script simulates clicks, keystrokes, and form interactions
7. The browser makes the actual HTTP request to the server
8. The response arrives, the page re-renders, and the script waits again
Steps 1-6 are pure overhead. The only step that matters is step 7 — the HTTP request.
How request automation works
Request automation eliminates steps 1-6:
1. Send an HTTP request with the correct headers, cookies, and body
2. Receive the response
3. Parse the structured response
That's it. No browser, no rendering, no JavaScript, no DOM, no selectors, no waiting.
The practical impact
For a workflow that involves logging in, navigating to a form, filling 10 fields, and submitting:
- Browser automation: Launch browser → navigate to login → type credentials → click submit → wait for redirect → navigate to form → find each field → type values → click submit → wait for confirmation. Total: 8-15 seconds.
- Request automation: POST /login with credentials → POST /form with all fields. Total: 200-500 milliseconds.
The request approach is 20-50x faster for this single workflow. Multiply by thousands of daily executions and the difference becomes enormous.
When request automation isn't enough
Request automation requires understanding the HTTP requests behind a workflow. For some scenarios, this isn't straightforward:
- Heavy client-side computation — if the browser does meaningful calculation (not just rendering), you may need to replicate that logic
- WebSocket-based applications — real-time apps using WebSockets need persistent connection management
- Truly novel interactions — one-off tasks where the overhead of understanding the request flow exceeds the cost of just using a browser
For the vast majority of business workflow automation — forms, data entry, status checks, submissions — request automation is the clear winner.
How Zatanna applies this
Zatanna observes a human performing a workflow once, captures the HTTP request behavior, and reconstructs it into a stable API endpoint. Your systems call that endpoint — no browser involved. The platform handles authentication, session management, error recovery, and anti-bot measures at the request level, giving you the reliability of direct API integration without needing to reverse-engineer anything yourself.