How to Scrape Amazon Product Data Without a Ban (2026)

Ask ten people how to scrape Amazon product data without a ban and nine of them will say "rotate your proxies". That answer was mostly right in 2017. It is close to useless in 2026, and not because Amazon got cleverer about IP addresses — it's because the IP stopped being the interesting signal. Amazon's edge inspects your TLS handshake before it has read a single HTTP header. By the time a request reaches anything that resembles an application server, it has already been scored on cipher suite order, HTTP/2 settings frames, header casing and ordering, whether the client bothered to fetch the stylesheet, and whether this session has ever behaved like a person who might one day buy something.

So the useful question isn't "which proxy provider doesn't get blocked". It's "what does a request from a real shopper look like end to end, and how much of that can I honestly reproduce?"

This guide is written for people who need product data at volume — price monitoring, MAP enforcement, catalogue enrichment, competitive research, retail arbitrage sourcing — and who are tired of pipelines that work for four days and then quietly start returning nulls. It covers what a block actually looks like (it's rarely a 403), the three architectures you can choose between, why identity is a bundle rather than an IP, the rate math nobody publishes, how to parse pages that change under you, and how to notice you've been detected before your dataset is poisoned.

One thing first, because it matters more than any technique below.

Before you write a line of code: scope, legality, and the boring option

Scraping publicly visible pages is not the same thing as breaching a contract, and neither is the same thing as violating a computer misuse statute. Those three questions have different answers in different jurisdictions and the case law keeps moving — the long-running hiQ Labs v. LinkedIn litigation is the usual reference point for how messy "public data" gets once contract claims enter the picture. I'm not your lawyer. What I will say is that the teams who get into trouble are almost never the ones collecting prices at a polite rate; they're the ones hammering an origin, collecting personal data they had no business touching, or reselling a bulk copy of someone's catalogue.

Read Amazon's robots.txt at amazon.com/robots.txt before you plan crawl paths. It is unusually specific and it tells you which surfaces Amazon has explicitly asked automated clients to leave alone — wish lists, order paths, review submission, most of /gp/. Staying out of the disallowed set is both the ethical baseline and, practically, the part of the site where anomaly detection is tightest.

And consider the boring option seriously: the Product Advertising API gives you titles, images, offers and prices without a single fingerprint problem. Its limits are real — you need an associate account with qualifying sales, coverage is partial, and the fields you want most (Buy Box history, third-party seller counts, coupon badges, delivery promises by postcode) are thin or absent. But if 60% of your requirement can be served by the API, serve it there and scrape only the 40% that can't. Every request you don't send is a request that can't get you blocked.

What a ban actually looks like on Amazon

Amazon almost never sends you a clean, honest "you are banned". It degrades you, and the degradations are designed to be cheap for them and expensive for you.

The five failure modes, in rising order of how badly they hurt

The CAPTCHA interstitial. The classic "Enter the characters you see below / Sorry, we just need to make sure you're not a robot" page, usually served from a /errors/validateCaptcha flow. It returns HTTP 200. If your pipeline only checks status codes, you are now storing CAPTCHA HTML as if it were a product page.

The dogs page. A 503 with the "Sorry! Something went wrong on our end" apology and a photo of an Amazon employee's dog. This is a throttle, not a permanent verdict. Back off hard and it usually clears within minutes.

The empty shell. You get a 200, the correct ASIN, the correct title — and the price block is missing. This is the nastiest one because it looks like a successful scrape. Sometimes it's genuine (product unavailable, no Buy Box winner); often it's a soft block where Amazon serves a stripped page to a client it doesn't trust. If you don't distinguish these you will publish a dashboard telling your team that 8% of the catalogue went out of stock overnight.

The wrong-region page. You asked for amazon.co.uk from an IP the edge geolocated to Frankfurt, so you got prices in EUR, a German delivery promise, and a currency your parser wasn't expecting. Not a block at all — a configuration failure that produces confidently wrong data.

The hard block. Connection resets, sustained 403s, or an IP that gets the CAPTCHA on every single request regardless of session. This is usually the end state of a subnet that has been abused for weeks, and it typically means the proxy pool is burnt, not that your scraper is uniquely bad.

Here's the diagnostic table I keep pinned for anyone triaging a pipeline:

The pattern worth internalising: status code is not truth on Amazon. Every validation you build should ask "does this page contain the fields a real product page contains?", not "was it a 200?".

Three architectures, and what each one costs you

There are exactly three ways to fetch an Amazon page, and choosing wrong is the root cause of most "why do I keep getting banned" threads.

The raw HTTP client is seductive because it's 200x cheaper per page. The problem is that a Python requests session has a TLS fingerprint that matches no browser on earth, and Amazon has had that on file for years. You can patch around it (curl-impersonate, custom cipher ordering, HTTP/2 pseudo-header ordering) and it will work for a while, and then a fleet-wide edge update will kill your entire pipeline in an afternoon.

Headless browsers fix the protocol layer and introduce a new problem: by default, every worker you spawn is the same browser. Same canvas hash, same WebGL renderer string, same font list, same screen metrics, same audio stack, same 24-bit colour depth on a 800x600 viewport nobody actually owns. Ten thousand requests from ten thousand IPs that all share one device signature is a stronger correlation signal than ten thousand requests from one IP. If that idea is new to you, our beginner's explainer on browser fingerprinting walks through exactly which attributes get combined into an identifier, and you can watch it happen live on the EFF's Cover Your Tracks tool.

The third approach — a real, non-headless Chromium per identity, each with its own fingerprint, its own disk-backed data directory and its own proxy — is slower per machine and dramatically more durable. This is what an antidetect browser is for. It's also why the same tooling that keeps multiple eBay accounts from being linked turns out to be the right substrate for large-scale collection: the underlying requirement is identical, which is many independent, internally consistent identities that do not correlate with each other.

Identity is a bundle, not an IP

Here is the mental model that fixes most people's scraping problem. A detection system doesn't ask "is this IP bad". It asks "do the parts of this identity agree with each other, and have I seen this combination behave badly before?"

An identity bundle has four layers, and a mismatch in any one of them is worth more than a clean score in the other three.

Layer 1: network egress

Datacenter IPs are pre-scored. AWS, Hetzner, DigitalOcean and OVH ranges are published, and Amazon — a company that runs the largest of them — is not confused about which addresses belong to servers. You can scrape lightly from a datacenter IP; you cannot run a fleet from one.

Residential and mobile proxies solve the reputation problem and introduce three new ones: latency (300–900ms is normal, and your timeouts must reflect that), instability (a residential exit can vanish mid-session because someone closed their laptop), and rotation semantics. That last one kills more pipelines than anything else. A rotating gateway that gives you a new IP every request is the wrong product for this job. Amazon issues a session-id and ubid-main cookie on first contact; if that session's IP changes country three times in ninety seconds, you've built a signal you'd never generate accidentally. What you want are sticky sessions — a stable exit for the lifetime of a browsing session, typically 10–30 minutes. We go deep on picking and pairing pools in the antidetect browser with residential proxies playbook.

One more thing worth saying plainly, because it's the single most common beginner mistake: a VPN is not a substitute here. A VPN changes one attribute for the whole machine while leaving every other identity layer identical and shared. If that distinction isn't crisp yet, this breakdown of the antidetect-browser-vs-VPN difference is a five-minute read that will save you a month.

Layer 2: the device fingerprint

Every profile needs a device signature that is (a) unique relative to your other profiles and (b) internally consistent. Consistency is where homebrew setups fall over. If navigator.platform says Win32, then the User-Agent must say Windows, the UA-CH sec-ch-ua-platform header must say Windows, the WebGL unmasked renderer must be something a Windows machine ships (an ANGLE D3D11 string, not an Apple GPU), the font list must contain Windows fonts, and the screen dimensions must be a resolution Windows machines actually run at. A profile claiming macOS while reporting ANGLE (NVIDIA GeForce RTX 4060 Direct3D11) is not anonymous — it's memorable.

This is also why the implementation layer matters. Fingerprint spoofing that works by injecting JavaScript into the page is detectable: the overridden functions don't survive a toString() check, they often fail to reach Web Workers, and there's a measurable timing window at document start before the patches land. Dual Login applies fingerprints natively inside the browser engine — the values are read from a signed, encrypted blob at process start, so there is no injected script to catch and the spoof reaches Workers and iframes identically. The practical difference is that a native fingerprint holds up on the commercial detection panels that JS-injection approaches trip. If you want the mechanics rather than the marketing, how to change your browser fingerprint covers what each attribute is and which ones actually carry entropy.

Layer 3: storage state

Cookies, localStorage, IndexedDB, service worker caches. A real shopper's browser accumulates state: session-id, session-token, i18n-prefs for currency, lc-main for locale, ubid-main as a durable device id, and a delivery-address cookie that determines which prices you're even shown. A scraper that discards all of it on every request is announcing itself. A scraper whose twenty workers share one cookie jar has merged twenty identities into one — you now have a single account-shaped entity making 20x the requests.

One data directory per profile, persisted to disk, reused across runs. That's the rule. Cold-starting a fresh browser for every fetch is both slower and more suspicious than keeping a warm profile alive.

Layer 4: behaviour

The layer everyone skips. Real people arrive from Google, land on a search results page, scroll, hover, open a product in a new tab, come back, sort by price, and take ten to forty seconds to read anything. Scrapers go directly to /dp/ASIN a thousand times in a row, at 4.00-second intervals, in ASIN-sorted order, with no referrer, and never scroll.

You don't need to simulate a full personality. You need to break the three tells that stand out most: perfect regularity (add jitter — a random 8–22s gap, not a fixed 10), zero-context navigation (arrive at some PDPs via a search or category page rather than always by direct URL), and impossible endurance (a profile that runs 18 hours without a break is not a shopper; give it sessions with beginnings and ends).

Building the fleet: a practical setup

Here's how I'd actually wire this up, assuming a mid-sized job — tens of thousands of ASINs a day across two or three marketplaces.

One profile equals one shopper, permanently

Create profiles as long-lived identities, not disposable workers. Profile 17 is a Windows 11 user in Manchester on a Virgin Media residential IP with amazon.co.uk, GBP, and postcode M1 1AE. It stays that way for months. Its cookies age, its ubid-main persists, its history accumulates. That consistency is an asset: a six-week-old session with a plausible history is treated far better than a session created ninety seconds ago.

What you rotate is which profile handles a given batch, not what a profile is.

Pin geography, then assert it

Prices, delivery promises, and Buy Box winners on Amazon vary by delivery location. If your profile's proxy exits in Texas but its saved delivery ZIP is 10001, you may collect prices no human in either place would see. Pin four things per profile and check them on every page: marketplace domain, delivery postcode, currency (i18n-prefs), and language (lc-main). Then have your parser assert the currency symbol on the page matches the profile's expectation, and drop the record loudly if it doesn't. A silent currency mismatch is how a price-monitoring product ships a report claiming a competitor cut prices 22% overnight.

The geography discipline is easier when the browser handles it natively — Dual Login derives timezone, locale and WebRTC exit from the profile's proxy so a UK profile doesn't leak a US timezone through Intl.DateTimeFormat().resolvedOptions().timeZone, which is a one-line check any detection script runs.

Concurrency and the RAM floor

Each profile is a real browser process. Budget roughly 300–500MB of RAM per concurrently open profile with low-memory mode on; that puts about 8–12 simultaneous profiles on a 16GB machine and 20–25 on a 32GB one, leaving headroom for the OS and your orchestrator. You don't need every profile open at once — a pool of 40 profiles cycled through 10 concurrent slots is a healthier shape than 40 all running flat out, because the idle time between a profile's sessions is itself realistic behaviour.

Drive them over the automation API rather than by hand. The important detail is how the automation attaches: driving a tab over raw CDP without ever enabling the JavaScript runtime domain keeps navigator.webdriver false and produces trusted input events, which is exactly what you want. A driver that flips the browser into automation mode has undone the fingerprint work you just paid for.

The rate math nobody publishes

This is the part people want a number for, so here's an honest one with the caveats attached.

A single warm profile on a decent residential IP, requesting product detail pages with 8–22 seconds of jitter between them, will comfortably sustain 150–250 pages per hour. Push past ~400/hour and CAPTCHA rate climbs sharply. Run it in sessions — say 45–90 minutes of activity followed by a 20–60 minute gap — and you land around 2,500–4,000 pages per profile per day with block rates under 2%.

So for 50,000 pages a day: 50,000 ÷ 3,000 ≈ 17 profiles, and you'd provision 25 to absorb failures, retries and the profiles you'll rest after a bad run. That's the whole calculation. If someone quotes you a setup doing 50,000 pages a day from three workers, they're either using the API, buying pre-scraped data, or about to have a bad week.

Two multipliers worth knowing. Search and category pages are cheaper per unit of information than PDPs — one results page can give you ASIN, title, thumbnail, star rating, review count and often the price for 48 products, so plan your crawl to take the cheap data cheaply and only open a PDP when you need the fields only a PDP has. And second: 429 responses are information, not noise. MDN's rules for 429 are worth re-reading; honour Retry-After when it's present, and treat any throttle response as a signal to reduce that identity's rate for the rest of the session, not just to sleep and retry once.

Parsing Amazon without rewriting it every Tuesday

Collection is half the job. The other half is that Amazon's HTML is not one layout — it's dozens, A/B tested continuously, varying by category, marketplace, device class, and whether the page has a Buy Box winner.

Write layered extractors, not selectors

For each field, define an ordered list of strategies and take the first that returns something plausible. For price, that might be: the structured data in the page's JSON-LD block, then #corePrice_feature_div and its variants, then the offer-listing fallback, then the "other sellers" module. For availability, check the availability block and the delivery promise and whether an add-to-cart control exists. A single brittle CSS selector is a scheduled outage.

Keep the ASIN as your primary key and pull it from the canonical URL rather than from any element — it's the most stable identifier on the page.

Validate before you store

Every record should pass a plausibility gate before it lands in your warehouse:

• Price is present, numeric, in the expected currency, and within an order of magnitude of the last observation for this ASIN.
• Title is non-empty and not one of the known error strings.
• The page contains at least one element that only ever appears on genuine product pages.
• The record's marketplace matches the profile that fetched it.

Records that fail go to a quarantine table with the raw HTML attached, not into the dataset with a null. Ninety percent of your debugging time later will be spent in that quarantine table, and you will be grateful the HTML is there.

Sponsored results will corrupt your ranking data

If you're tracking organic position, you must exclude sponsored placements, and they are deliberately hard to distinguish — they carry the same card markup with a small badge and different tracking parameters. Test your exclusion logic against a manually annotated sample every month; ad density changes and a silent regression here means your "we moved from position 14 to position 6" report is fiction.

Knowing you've been detected before your data is ruined

The worst outcome isn't a block. It's a partial block that you don't notice for two weeks while your dataset fills with plausible-looking garbage.

Instrument four canary metrics, per profile and per proxy pool, on a rolling one-hour window:

1. CAPTCHA rate — pages matching the interstitial signature, divided by pages fetched. Baseline should be under 1%. Above 5% on a given profile, rest it. Above 5% across a whole pool, the pool is degrading.
2. Price-presence ratio — of PDPs that returned 200, the fraction where a price was extracted. This should be stable at whatever your catalogue's true availability rate is (usually 90%+). A sudden drop is the classic soft-block signature.
3. Median page bytes — soft-blocked and stripped pages are dramatically smaller. Watching the median response size catches degradations that status codes miss entirely.
4. Novel-selector rate — how often your primary extractor fails and a fallback fires. A step change means Amazon shipped a layout, and you have a few days before your fallbacks stop covering it.

Alert on all four. And keep a small golden set: 50 ASINs you check by hand once a week from an ordinary browser on an ordinary connection, comparing against what the pipeline collected the same day. It takes fifteen minutes and it is the only check that catches systematic bias.

CAPTCHAs: what to do, and what to stop doing

When you hit a CAPTCHA, the correct response is almost never to solve it.

A CAPTCHA is a message saying this identity's trust score just fell below threshold. Solving it buys you a handful more requests on an identity that is now marked. Retrying the same URL through the same profile immediately is worse — it converts a soft signal into a hard one.

What works: mark the profile as cooling, close its session cleanly, requeue the URL to a different profile, and don't touch the cooled profile for at least an hour. If a profile CAPTCHAs three sessions in a row, retire its proxy binding and give it a fresh exit. If a whole pool starts CAPTCHAing on first contact, stop the fleet entirely for a few hours rather than burning every profile you own against a pool that's already flagged. The instinct to push through is the single most expensive instinct in this field.

If you genuinely can't proceed without solving — say a one-off manual verification on a long-lived profile — do it by hand in that profile's window. It's a real browser; you can just look at it.

Where this overlaps with account safety

A lot of people scraping Amazon are also selling on Amazon, and it's worth being explicit: do not run collection from a profile that has ever been signed into a seller or buyer account you care about. Cookie state, device id and behavioural history all persist, and linking a high-volume automated identity to an account is how a perfectly legitimate seller ends up in a suspension appeal. Keep collection profiles and account profiles in separate groups with separate proxy pools and never let them touch. If account health is part of your world, the Amazon seller ban-avoidance playbook covers the account side of that wall in detail.

A worked example: 50,000 products a day, two marketplaces

To make it concrete, here's a shape I'd be comfortable defending.

Target: 40,000 US PDPs and 10,000 UK PDPs daily, refreshed every 24h, price and Buy Box accuracy prioritised over completeness.

Fleet: 30 profiles — 22 US, 8 UK. Each pinned to one marketplace, one sticky residential exit in a matching metro, one delivery postcode, one currency. Fingerprints spread across a realistic device mix: roughly 55% Windows desktop, 25% macOS, 20% Android, with screen resolutions drawn from actual population statistics rather than a uniform random pick.

Hardware: one 32GB box running 12 concurrent profile slots, cycling the 30-profile pool.

Schedule: three collection windows a day rather than a continuous grind, each 4–5 hours, with per-profile sessions of 60–75 minutes and rest gaps between them. Profiles never all start simultaneously — launches are spread over ten minutes.

Crawl shape: category and search pages first to harvest the cheap fields for the whole catalogue; PDP visits only for ASINs whose price changed, whose Buy Box flipped, or that haven't been visited in 72 hours. That single optimisation typically cuts PDP volume by 60% and is worth more than any anti-detection trick on this list.

Budget expectation: ~30 profiles × ~2,000 effective pages/day is 60,000 — comfortable headroom over 50,000, which is exactly what you want, because the headroom is what absorbs a bad proxy day without turning into a data gap.

Cost reality: the residential bandwidth will be your largest line item by a wide margin, not the browser tooling. A full PDP is 1–3MB with images; blocking images, fonts and media at the network layer cuts that by 70–80% and is the single highest-leverage cost decision you'll make. If you're cost-sensitive and running a small team, this buyer's guide to affordable antidetect browsers compares what you're actually paying for per profile.

The mistakes that cause 90% of bans

A checklist, in rough order of how often I see each one:

• Shared cookie storage across workers. Twenty identities collapse into one. Fix: one data directory per profile, always.
• Rotating IP mid-session. Sticky sessions, 10–30 minutes minimum.
• Perfectly regular request intervals. Jitter everything, including your retry backoff.
• Trusting HTTP 200. Validate page content, always.
• A fingerprint that contradicts itself. Platform, UA, UA-CH, WebGL, fonts and screen must all tell one story.
• Cold-starting a browser per request. Slow, expensive and suspicious. Keep profiles warm.
• Solving CAPTCHAs instead of backing off. You're paying to burn identities faster.
• No quarantine table. You will need the raw HTML of the failures. Keep it.
• Ignoring the delivery postcode. Wrong prices that look right are worse than no prices.
• Scaling before measuring. Get a 1% block rate at 1,000 pages/day before you attempt 50,000. The failure modes are the same; the cleanup cost is not.

FAQ

Is it legal to scrape Amazon product data?

Collecting publicly visible product information is generally treated differently from breaching a contract or accessing protected systems, but the answers vary by jurisdiction and by what you do with the data. Amazon's terms of service restrict automated access, robots.txt names surfaces you should stay off, and reselling bulk copies of a catalogue carries different risk from monitoring prices for internal decisions. Read the terms, respect robots.txt, avoid personal data, keep your request rate polite, and get legal advice before anything commercial at scale.

How many requests per hour can one profile safely make?

With realistic pacing — 8 to 22 seconds of jittered think-time between pages — a single warm profile on a good residential IP sustains roughly 150–250 pages per hour with a block rate under 2%. Beyond about 400 per hour, CAPTCHA rate climbs sharply. Scale by adding profiles, not by speeding up existing ones; ten profiles at 200/hour is far more durable than two at 1,000.

Do I really need an antidetect browser, or will Playwright do?

Playwright is fine for low volume. The problem at scale is that every worker you spawn shares one device fingerprint, so a thousand IPs collapse into one identity signal. An antidetect browser gives each profile a distinct, internally consistent fingerprint plus its own persistent storage and proxy, which is what stops correlation. If you're fetching a few hundred pages a day, plain Playwright with sensible pacing will get you there.

Should I use residential or datacenter proxies for Amazon?

Residential, with sticky sessions, for anything sustained. Datacenter ranges are published and pre-scored, and Amazon operates the largest cloud on earth — it is not guessing about which addresses belong to servers. Mobile proxies are cleaner still but cost more and are usually overkill for product data. Whatever you choose, avoid per-request rotating gateways: an identity whose IP changes every few seconds is a signal you would never generate by accident.

Why do I get a 200 response with no price on it?

Three possibilities: the product genuinely has no Buy Box winner, you've hit a page layout your extractor doesn't cover, or you've been soft-blocked and served a stripped page. Distinguish them by checking median response size (soft-blocked pages are much smaller), by looking for elements that only appear on genuine product pages, and by keeping the raw HTML of every failure so you can look. Never store a null price without recording which of the three it was.

How do I know my anti-detection setup is actually working?

Measure, don't assume. Track CAPTCHA rate, price-presence ratio, median page size and fallback-selector rate per profile and per proxy pool on a rolling window, and alert on step changes in any of them. Then keep a golden set of 50 ASINs you verify by hand weekly against an ordinary browser. Fingerprint test panels tell you whether your profile looks unique; only the golden set tells you whether your data is right.

Wrapping up

If there's one idea to take away, it's that scraping Amazon product data without a ban is not a trick you apply — it's a system you maintain. The proxy is one layer of four. The fingerprint is another. Storage isolation and behavioural pacing are the two people skip, and they're the two that decide whether your pipeline lasts a week or a year.

Start smaller than you think you need to. Get a handful of long-lived profiles to a genuinely low block rate at modest volume, instrument the four canary metrics, and only then multiply. Scaling a healthy system is arithmetic; scaling a broken one just produces broken data faster.

If you want a substrate that handles the identity layer properly — native fingerprints applied inside the engine rather than injected as JavaScript, one persistent data directory per profile, per-profile proxy binding with matching timezone and WebRTC, and a raw-CDP automation API that never trips navigator.webdriver — Dual Login was built for exactly this shape of work. Spin up a few profiles, point them at a hundred ASINs, and watch the canaries for a day before you commit to anything. That first honest measurement is worth more than any guide, including this one.