How Many Proxies Needed for Web Scraping? The Real Math (2026)

Ask ten scraping engineers how many proxies are needed for web scraping and you'll get ten confident, contradictory answers. One insists a hundred residential IPs will handle anything. Another swears by ten thousand rotating datacenter addresses. A third shrugs and says the count barely matters anymore because detection moved up to the browser layer. Annoyingly, all three are right — for their target, at their scale, with their setup.

The honest answer is arithmetic, not folklore. Your proxy count falls out of four numbers: how many requests you need to make per hour, how many requests a single IP can safely make against your specific target in that hour, how long each session needs to stay on one IP, and how quickly the target burns IPs anyway. Once you have those four, sizing your pool is a two-line calculation. Get any of them wrong and you'll either overpay for a pool you never touch, or watch your success rate fall off a cliff at 3 a.m. when the block wave lands.

This guide walks through that math with worked examples at three different scales, compares proxy types without the vendor gloss, and covers the part most sizing guides skip entirely: what to do when adding IPs stops helping because the site is fingerprinting your browser, not your address.

The short answer (and why it's incomplete)

If you just need a starting point before we get into the real math, here are the rules of thumb experienced teams actually use:

• Lenient targets (no bot protection, generous rate limits): one datacenter IP per 500–2,000 requests per hour. A modest job often runs fine on 5–10 IPs.
• Protected targets (Cloudflare, Akamai, PerimeterX-class defenses): one residential IP per 100–300 requests per hour, and you'll usually rent access to a rotating pool rather than counting individual IPs at all.
• Logged-in or account-based scraping: one sticky IP per account, full stop. The count equals your account count, and the IP is only half the identity — more on that later.

Why is that incomplete? Because those numbers collapse the moment you introduce concurrency, session state, geographic requirements, or a target that rate-limits by subnet instead of by address. A pool of 100 IPs that all sit in the same /24 block behaves like 1 IP to a defense that bans by subnet. A rotating residential plan with 10 million IPs behaves like 40 IPs if your plan only allows 40 concurrent sessions. The headline number on the proxy vendor's pricing page is almost never the number that constrains you.

So let's do this properly.

The four numbers that actually decide your proxy count

1. Target tolerance: safe requests per IP per hour

This is the number nobody can look up, because every site is different and most sites publish nothing. You measure it.

The method is simple and worth an afternoon: run a single clean IP against the target and ramp the request rate gradually — say, start at 60 requests per hour and double every few hundred requests. Watch for the first sign of pushback: an HTTP 429 Too Many Requests response, a 403, a CAPTCHA interstitial, a silent redirect to a consent wall, or — sneakiest of all — pages that return 200 but with degraded or poisoned content. Note the rate where trouble started, then set your working ceiling at half of it. That halved figure is your safe requests per IP per hour, and it's the denominator in everything that follows.

Three wrinkles to check while you're measuring:

• Is the limit per IP, per subnet, or per session? Fire the same ramp from two IPs in the same /24 and one IP elsewhere. If the same-subnet pair trips together, the site counts by subnet and your pool needs to be spread across networks, not just large.
• Does the limit reset, or does the IP stay burned? Some sites cool off in ten minutes; others blocklist an IP for days. This determines your cooldown math later.
• Is the limit stricter on some paths? Search endpoints and login pages are usually guarded far more tightly than product pages. Measure the path you'll actually be hitting.

While you're there, read the site's terms and its robots.txt, and understand the legal landscape for your jurisdiction and use case — the Wikipedia overview of web scraping covers the notable cases. Scrape data you have a legitimate right to collect, and be a polite guest even when you're allowed in.

2. Required throughput: real requests, not pages

Pages per day is what your stakeholder asks for. Requests per hour is what you size proxies with, and the two diverge more than people expect.

Start with pages per day, divide by your operating window in hours (few scrapers run a true 24 hours — you'll batch, you'll respect the target's quiet hours, you'll have maintenance windows). Then inflate for reality:

• Secondary requests. If you're rendering pages in a real browser, one page might trigger a dozen requests you care about routing through the same proxy — XHR calls, pagination fragments, lazy-loaded content.
• Retries. Even a healthy pipeline retries 5–15% of requests. An unhealthy one retries 40%, and every retry consumes proxy capacity.
• Warm-up traffic. Sessions that browse a category page before hitting the target listing look human — and cost requests.

A realistic multiplier on the naive page count is 1.2× to 1.5×. Use it.

3. Proxy type: trust is part of the math

An IP address carries a reputation before your first packet arrives. Defenses look up the ASN (the network that owns the address) and score it: a request from a cloud datacenter range starts the conversation under suspicion; a request from a consumer ISP range starts it with the benefit of the doubt; a request from a mobile carrier's CGNAT — where thousands of real phones legitimately share one IP — is nearly unbannable without collateral damage the site can't afford.

This is why the safe-rate number from step 1 must be measured with the proxy type you'll actually use. A datacenter IP might get 0 safe requests per hour on a protected site — blocked on the first hit — while a residential IP sails through 200. The type doesn't just change the price column; it changes the denominator of the whole formula.

4. Session model: stateless, sticky, or authenticated

This one changes which formula you use at all.

• Stateless scraping (public pages, no login, no cart): rotate freely, size the pool by throughput.
• Sticky sessions (multi-step flows, search-then-paginate, anything with a session cookie): each session must stay on one IP for its lifetime — an IP that changes mid-session is one of the loudest bot signals there is. Here you size by concurrency, not throughput: your pool must be at least as large as your peak simultaneous sessions, plus cooldown headroom.
• Authenticated scraping (logged-in accounts): the strictest model. One account, one IP, one browser identity, permanently paired. If you run 40 accounts, you need 40 sticky IPs regardless of how few requests each makes. Sharing an IP between two accounts links them; rotating an account across IPs flags it. Teams running multiple accounts through an antidetect browser learn this rule once, usually the hard way.

The formula, with three worked examples

Here's the whole thing:

For stateless scraping:

proxies needed = (peak requests per hour ÷ safe requests per IP per hour) × headroom

where headroom is 1.3–1.5 to absorb burned IPs, retry storms, and traffic spikes.

For sticky or authenticated scraping:

proxies needed = peak concurrent sessions × (1 + cooldown ratio)

where the cooldown ratio accounts for IPs resting between sessions. If an IP works for 60 minutes and then needs 30 minutes of cooldown before reuse, your ratio is 0.5 and you need 1.5× your concurrency.

Take whichever formula matches your session model — and if a job mixes both, size each portion separately. Now let's run real numbers.

Example 1: price monitoring on a lenient target

You're tracking 200,000 product pages daily on a mid-size retail site with no serious bot defense. You scrape over a 20-hour window, so that's 10,000 pages per hour. No login, no session state, minimal secondary requests — call it 11,000 real requests per hour after retries.

Your ramp test showed a datacenter IP comfortably handles 1,000 requests per hour before the site starts serving 429s at around 2,000. Safe rate: 1,000.

11,000 ÷ 1,000 = 11 IPs, × 1.4 headroom ≈ 16 datacenter IPs

At a dollar or two per IP per month, this entire job's proxy bill is under $40. This is the scenario where people who say 'you barely need any proxies' are right — and it's genuinely common. Not every target is a fortress.

Example 2: search results on a protected site

Now the harder case: 50,000 search-result pages per day from a site behind serious bot protection, scraped over a 10-hour window. That's 5,000 pages per hour, and because each search flow involves a query page plus two pagination clicks on a sticky session, figure 15,000 requests per hour across roughly 1,700 concurrent-ish short sessions.

Datacenter IPs are dead on arrival here — blocked at the first TLS handshake. Your residential ramp test found a safe rate around 150 requests per hour per IP, with a soft cooldown: an IP that backs off for 20 minutes recovers.

By throughput: 15,000 ÷ 150 = 100 IPs in active use at any moment. But here's where the market reality kicks in — you won't buy 100 named residential IPs. Rotating residential access is sold as a gateway: you pay per gigabyte, the provider maintains a pool of millions, and your real constraints are concurrent sessions allowed on your plan and bandwidth. So the sizing question becomes: does the plan support ~100+ concurrent sticky sessions, and does the GB budget cover your page weight? A JavaScript-heavy page pulling 2 MB through the proxy, 50,000 times a day, is ~100 GB/day — at typical residential rates, that's the line item that hurts, not the IP count. Strip images and unneeded assets at the browser level and you can cut it by two-thirds. Our antidetect browser with residential proxies playbook goes deep on that pairing.

Example 3: logged-in scraping across accounts

You operate 40 accounts on a platform and need each to pull its own dashboard data daily. Request volume is tiny — maybe 200 requests per account per day. By throughput math you'd need one proxy. By identity math you need forty.

Each account gets a dedicated sticky IP — ISP proxies (static residential) are ideal: consumer-grade trust, datacenter-grade stability, flat monthly pricing. The IP never rotates, because the platform has seen this account from this address for months and consistency is the camouflage. Each account also gets its own browser profile with its own cookies and its own fingerprint, because on logged-in platforms the IP is only one of several identity signals being cross-checked. Forty accounts, forty IPs, forty isolated browser profiles. The proxy count question and the account-safety question are the same question here.

Proxy types compared

Two honest notes on this table. First, the safe-rate column is a starting range for your own measurement, not a promise — a lenient site tolerates 10× these numbers and a paranoid one tolerates a tenth of them. Second, mixed fleets are normal: mature operations route each target to the cheapest proxy type that sustains an acceptable success rate, and only escalate the expensive traffic.

Rotation strategy changes the count

Per-request rotation

Every request exits from a different IP. Maximum throughput per IP-hour, ideal for stateless harvesting of public pages. The trap: many sites set a cookie or token on first contact and expect subsequent requests to arrive from the same address. Rotate through those and you don't look like many users — you look like one bot wearing a rotating mask, because the session is continuous while the IP is not. If the site sets state, you need stickiness.

Sticky sessions and cooldown math

With sticky sessions, an IP is checked out for the life of a session, then rested. Your effective pool shrinks by the cooldown:

effective pool = total pool × active time ÷ (active time + cooldown time)

A 100-IP pool where each IP works 40 minutes then cools 20 gives you about 66 usable IPs at any moment. Skip the cooldown and your pool 'works' for a week, then degrades in a rolling wave as reputation catches up with each address — a failure mode that looks exactly like the target 'suddenly getting stricter' and is actually self-inflicted.

The subnet trap

A hundred IPs from one /24 subnet is, to a defense that scores by network block, one IP with extra billing. When the ban lands it takes the whole block at once — which is why cheap datacenter bundles fail so theatrically. When you buy fixed IPs, buy diversity: multiple subnets, multiple ASNs, and multiple geographies if the target serves regional content. Fifty well-scattered IPs reliably outperform five hundred clustered ones.

When more proxies stop helping

Here's the plateau every scaling scraper eventually hits: success rates sag, you double the pool, and nothing improves. That's the signal that the site stopped caring about your IPs and started identifying your client.

Modern defenses evaluate dozens of signals beyond the address. The TLS handshake fingerprint reveals your HTTP library before a single header is sent — Python's requests does not shake hands like Chrome, and no proxy changes that. Header order, HTTP/2 negotiation details, and above all the browser fingerprint — canvas rendering, WebGL renderer strings, installed fonts, screen geometry, timezone, audio context — combine into an identifier precise enough to recognize a returning client with no cookies at all. If that's unfamiliar territory, our beginner's guide to browser fingerprinting covers the mechanics, and the EFF's Cover Your Tracks tool will show you, in about ten seconds, how identifiable your own browser is.

Now picture the failure from the defender's chair: five hundred residential IPs, all presenting the identical fingerprint — same canvas hash, same WebGL string, same fonts, same screen size. That's not five hundred users. That's one client behind five hundred doors, and it's trivially clusterable. The defense doesn't need to ban your IPs; it bans your fingerprint, and every proxy you add inherits the ban on arrival. This is precisely the situation where teams burn thousands on bigger pools and see zero improvement.

The fix is to make each concurrent identity internally consistent and mutually distinct: one identity = one IP + one fingerprint + one cookie jar, all three stable together, none shared. Mechanically, that means real isolated browser profiles rather than one browser wearing costumes — and doing it by hand across dozens of profiles is exactly the problem antidetect browsers exist to solve. Dual Login runs each profile as a separate browser process with its own persistent data directory, a unique fingerprint applied natively inside the engine (not by injected JavaScript that scanners can spot), and its own proxy assignment, so identity number 37 always shows up as the same coherent device on the same residential IP, and identity 38 shares nothing with it. If you want to understand what actually changes under the hood, how to change your browser fingerprint walks through it — and if you're wondering why a VPN doesn't solve any of this, the antidetect browser vs VPN difference is the short version: a VPN moves all your traffic to one new IP with the same fingerprint, which is the opposite of what scale requires.

The practical consequence for proxy sizing is welcome: consistent identities get burned far less often, so your burn rate drops, your cooldown pressure drops, and a smaller pool of good IPs outperforms the bloated pool you were about to buy. Fingerprint discipline is, indirectly, a proxy cost optimization.

A sizing cheat sheet at three scales

~10,000 pages/day, lenient target. Two to five datacenter IPs across different subnets. Under $10/month. Spend your energy on parsing, not infrastructure.

~100,000 pages/day, mixed targets. A small fleet: 15–25 datacenter IPs for the lenient portion, 10–20 ISP proxies for anything sticky or logged-in, and a rotating residential plan sized to ~50–100 concurrent sessions for the protected portion. Route each target to the cheapest tier that holds a 90%+ success rate. Budget realistically lands at $150–$500/month, dominated by residential bandwidth.

~1,000,000 pages/day, protected targets. You're sizing concurrency now, not IPs: typically 200–500 concurrent sticky sessions through a rotating residential gateway, ISP proxies pinned to every authenticated identity, and a handful of mobile ports reserved for the small percentage of targets that reject everything else. At this scale the browser layer is non-negotiable — headless detection and fingerprint clustering will find naive fleets — and bandwidth engineering (blocking images, fonts, and analytics through the proxy) often halves the bill.

Count the cost per successful page, not the cost per gigabyte

The cheapest proxy per unit is frequently the most expensive per result. Run the numbers on a 100,000-page job:

• Datacenter pool at $50/month achieving a 45% success rate: every real page costs you 2.2 attempts, plus retry traffic, plus the engineering hours spent fighting blocks. Effective throughput halves; deadlines slip.
• Residential access at $300/month achieving 96%: 1.04 attempts per page, predictable completion, and your engineers work on data quality instead of block forensics.

Failed requests aren't free — they consume bandwidth, burn IP reputation, and poison your dataset with partial results. The only metric that matters is cost per successfully delivered page, and by that metric the mid-priced option wins far more often than the pricing page suggests.

Seven mistakes that quietly inflate your proxy bill

1. Sizing from the vendor's pool size ('10M+ IPs!') instead of your plan's concurrent-session limit — the number that actually constrains you.
2. Skipping the ramp test and guessing the safe rate. An afternoon of measurement routinely halves a pool budget.
3. Rotating IPs mid-session, converting expensive residential traffic into an obvious bot signal.
4. Buying IP volume in one subnet, then losing the whole block to a single ban event.
5. Running one fingerprint across the entire pool, so the defense clusters and bans you client-side while you keep buying IPs that arrive pre-burned.
6. Pulling full page weight through metered proxies — images and third-party scripts you never parse can be two-thirds of your residential bandwidth bill.
7. No cooldown discipline, so the pool degrades in a slow wave that gets misdiagnosed as the target tightening up.

FAQ

How many proxies do I need to scrape 10,000 pages a day?

On a lenient site: two to five datacenter IPs, since each can typically carry 500–2,000 requests per hour. On a protected site: a rotating residential plan supporting 10–20 concurrent sessions matters more than the raw IP count. Always run a single-IP ramp test first — the target's real tolerance, not a generic figure, sets your number.

Can I scrape with just one proxy?

Yes, if your volume stays under the target's per-IP tolerance — often a few hundred requests per hour on an ordinary site. One good ISP proxy handling a slow, steady crawl is perfectly viable and very cheap. The moment you need speed, concurrency, or multiple identities, one IP becomes both a bottleneck and a single point of failure.

Are free proxies worth using for web scraping?

No. Free proxy lists are slow, short-lived, and usually blocklisted before you find them; success rates are terrible. Worse, an unknown operator sits in the middle of your traffic and can read or modify everything you send. The cheapest defensible option is a small paid datacenter pool at a few dollars a month.

Do I need residential proxies, or are datacenter proxies enough?

Test datacenter first — it's 10–50× cheaper per unit of work and plenty of sites accept it. Escalate to residential when datacenter traffic gets blocked outright or CAPTCHA'd at rates that destroy your economics, and reserve mobile for the small set of targets that reject residential too. Mature setups run all tiers and route each target to the cheapest one that sustains ~90%+ success.

How many proxies per account for logged-in scraping?

One dedicated sticky IP per account — ideally ISP proxies, which combine consumer-ASN trust with static stability. Never share an IP between accounts (it links them) and never rotate an account across IPs (it flags it). Pair each IP with its own isolated browser profile and fingerprint so all identity signals stay consistent together.

Does an antidetect browser reduce how many proxies I need?

Indirectly, yes. It doesn't lower your throughput requirement, but by giving every identity a distinct, consistent fingerprint it stops the fingerprint-level clustering that gets IPs burned in batches. Lower burn rate means less cooldown pressure and less replacement buying — so a smaller pool of quality IPs delivers what a larger, faster-burning pool used to.

The bottom line

How many proxies you need for web scraping is a calculation, not a mystery: measure your target's real per-IP tolerance, translate pages into honest requests per hour, divide, and add headroom — or size by concurrency the moment sessions or accounts enter the picture. Small lenient jobs need a handful of datacenter IPs. Protected targets shift the question to concurrent residential sessions and bandwidth. Account-based work is one sticky IP per identity, no exceptions.

And when adding IPs stops moving the needle, the bottleneck isn't your pool — it's that every proxy is wearing the same face. Dual Login pairs each of your proxies with its own isolated browser profile: a unique, natively applied fingerprint, a persistent data directory, and a stable identity that doesn't leak into its neighbors. If you're sizing a scraping fleet for 2026, put it through its paces before you pay — start small, watch your success rate, and let the math tell you it's working.