Web Scraping Without Selenium Detection: 2026 Field Guide

The most common message I get about scraping goes something like this: it ran perfectly for nineteen days, I changed nothing, and this morning every single request comes back 403. Nothing broke. What happened is that the target's detection vendor pushed a rule, or your IP range crossed a reputation threshold, or a single tell that had been sitting in your traffic the whole time finally got weighted heavily enough to matter.

That is the thing people underestimate. Detection is not a gate you either pass or fail. It is a score assembled from dozens of weak signals, and Selenium hands over several strong ones for free — before your first page even finishes loading. So the phrase web scraping without Selenium detection is slightly misleading if you read it as 'find the flag that turns Selenium invisible'. There is no such flag. What there is: a well-understood list of surfaces detectors read, and a set of architectural choices that keep you off most of them permanently instead of patching them one at a time forever.

This guide is the long version of what I would tell an engineer joining a scraping team. It covers what actually gets you caught, the four realistic architectures and their honest trade-offs, how to build a pipeline that survives a detection-rule update, and how to verify any of it before you spend a month of engineering on the wrong layer.

Why Selenium gets caught, and it is never just one thing

Selenium is not badly written. It is honestly written — the WebDriver specification deliberately requires a browser under automation to say so. Treating that as a bug you can patch away misses the point. Below are the five distinct surfaces, roughly in order of how cheap they are for a detector to check.

The declared tells: the browser announces itself

navigator.webdriver returns true in any WebDriver-controlled session. This is not an accident or an oversight — it is mandated behaviour, documented on MDN, and it exists so that sites can tell. A one-line check in a bundled analytics script catches an unmodified Selenium session on page one.

On top of that, ChromeDriver historically injected identifiable properties into the document and window scope — the cdc_-prefixed variables that half the Stack Overflow answers on this topic are about. Patching the binary to rename them works, briefly, and then the detector stops looking for the exact string and starts looking for any unexpected own-property on document matching a random-looking pattern. That is the treadmill in miniature.

Then there is --enable-automation, which Selenium passes by default. It changes the shape of window.chrome, alters some permission defaults, and historically drew a visible infobar. Removing the switch removes the infobar; it does not remove every downstream difference it caused.

The protocol tells: attaching a debugger is visible from inside the page

This is the layer most people never get to, and it is the one that quietly kills sophisticated setups. Chrome automation runs over the Chrome DevTools Protocol. Some CDP domains are passive — DOM, Input, Page, Network mostly just observe and inject events. One is not: Runtime.enable changes how the page behaves.

When the Runtime domain is enabled, the browser starts serialising objects for the debugger. That serialisation has observable side effects. The classic demonstration is an object with a getter on a property the serialiser touches — for example, a custom stack accessor on an Error subclass. In a normal browser, nothing reads it unless your own code does. With Runtime enabled and an object logged or thrown, the getter fires because something on the other side of the protocol is inspecting it. A page can also watch for the timing signature of execution-context creation, or notice that isolated worlds appear where none should.

The practical consequence: a scraper that avoids every JavaScript tell can still be caught because it called Runtime.evaluate once to read a value. Any framework whose default 'get me the text of this element' path goes through JavaScript evaluation is enabling Runtime on your behalf. This is why we treat Runtime.enable as radioactive and drive the DOM through DOM/Input calls instead, with evaluation available only as an explicit, flagged opt-in for the rare case that needs it.

The environment tells: your browser looks like a server

Headless Chrome improved enormously — modern headless is a real Chrome, not the old shim — but the environment it runs in usually still screams datacenter:

• GPU. No physical GPU means software rendering, and WEBGL_debug_renderer_info reports something like a SwiftShader or software-Vulkan string. Roughly nobody browsing the real web reports that. A WebGL renderer that does not plausibly belong to the OS and CPU you claim is one of the highest-signal mismatches available.
• Screen geometry. A default 800x600 viewport, or screen.width === window.innerWidth (no OS chrome, no taskbar, no scrollbar), or screen.availHeight === screen.height. Real desktops have furniture.
• Fonts. A bare container has a couple of dozen fonts. A real Windows install has 200-plus, in a distribution that is recognisable per OS and locale.
• Devices and permissions. No audio output device, no camera, Notification.permission in an unusual state, an empty navigator.plugins where a real Chrome has its PDF entries.
• Timezone and locale. A container defaulting to UTC while the exit IP is in São Paulo. Cheap to check, devastating to fail.

If you want the systematic version of this layer, our beginner-friendly explanation of browser fingerprinting walks through each attribute and what it leaks.

The network tells: you are identified before JavaScript runs

Two scrapers can be byte-identical in the browser and still be sorted correctly at the edge:

• ASN and IP reputation. Datacenter ranges are labelled. A single abusive tenant on your subnet degrades everyone on it. This is why the fix for a CAPTCHA is almost never a fingerprint change.
• TLS fingerprinting (JA3/JA4). The cipher suites, extensions, and their ordering in your ClientHello identify the library, not the claimed browser. A Python client sending a Chrome/1xx User-Agent is trivially caught: the handshake says one thing, the header says another.
• HTTP/2 shape. SETTINGS frame values, window sizes, pseudo-header order, priority frames. Real Chrome has a signature here. Most HTTP clients have a different one.

The general rule: any layer where you claim to be Chrome, you must actually be Chrome. Half-measures are worse than honesty, because a mismatch is a stronger signal than a plain non-browser client.

The behavioural tells: nobody clicks like that

Once a session survives the static checks, the remaining signal is behaviour. Mouse paths that teleport from coordinate to coordinate. Dwell times identical to the millisecond across 400 sessions. Scroll in perfect 100-pixel increments. Zero idle time, ever. Form fills at 2ms per character. And the big one: events dispatched from JavaScript carry isTrusted: false, which any listener can read. If your click is element.click() or a synthetic MouseEvent, a site that cares knows it was not a human hand. Injecting input at the protocol level (the CDP Input domain) produces trusted events, because the browser itself generates them.

The mental model that actually helps: coherence, not invisibility

You cannot be invisible. Every browser that loads a page emits a fingerprint, and a zero-entropy fingerprint is itself remarkable — a browser reporting no fonts, no GPU, and no plugins is more suspicious than one reporting an ordinary mid-range laptop. The EFF's Cover Your Tracks project makes this concrete: it scores not just how much you leak, but how unusual you are.

So the goal is not minimal. The goal is plausible and internally consistent. Think of a profile as a claim about a specific machine sitting in a specific place, and then make every layer agree with that claim:

• The User-Agent says Windows 11, so the platform, the font list, the WebGL vendor/renderer pair, and the client hints all say Windows 11.
• The exit IP geolocates to Warsaw, so the timezone is Europe/Warsaw, Accept-Language leads with pl-PL, and the reported screen resolution is one that is actually common there.
• The device is a laptop, so deviceMemory, hardwareConcurrency, screen size, and touch support form a combination that a real laptop would produce.

One contradiction is worth more to a detector than twenty ordinary attributes, because ordinary attributes are shared with millions of people while a contradiction is shared with almost nobody except automation. If you want the attribute-by-attribute recipe, see how to change your browser fingerprint properly — the section on keeping values coherent is the part that matters here.

A second, related principle: spoof low, not high. A fingerprint applied by injected JavaScript is a patch sitting on top of the real values. It can be caught by comparing a getter's toString() against the native form, by asking a Web Worker or an iframe the same question (injection frequently misses both), or by reading the value before your script runs. A fingerprint applied inside the browser engine has no wrapper to detect, answers identically from workers, and is already in place for the very first script on the page. This is why serious antidetect tooling — including Dual Login — applies identity natively in a patched Chromium rather than injecting JS at document start.

Four architectures, honestly compared

A few notes on reading that table honestly.

Skip the browser when you can

The cheapest undetected scraper is the one that never opens a browser. Before you build anything, spend a day looking for: a documented API, an undocumented JSON endpoint the front-end already calls, an RSS or sitemap feed, a bulk export, or a partner data feed. A single well-formed request to a JSON endpoint costs about 0.1% of what a browser page load costs in CPU, RAM, and bandwidth. Plenty of teams run a headless browser fleet to render pages whose data arrives from an XHR they could have called directly.

If you go this route, use a client with browser-matched TLS and HTTP/2 behaviour. Otherwise you have simply moved your detectable signature from JavaScript to the handshake.

Patching the automation stack is a treadmill, and you should know the incline

undetected-chromedriver, stealth plugins, and the various patch sets do real work and are genuinely useful for small jobs and one-off research. Their structural problem is that they are a diff against a detectable baseline. Every property they fix is one a detector already knew about, which means the detector's next move is to look one layer down — at the protocol, at worker-context consistency, at property-descriptor ordering, at how your patched getter stringifies. You will win most weeks and lose some, and the losses arrive without warning, usually mid-campaign.

If your scraping is a business rather than a side project, the maintenance line in that table is the number that decides this. Constant is expensive.

Real browser, minimal protocol, coherent identity

The approach that has aged best: launch an ordinary, unpatched-looking Chromium process with no automation switches and no attached debugger holding the session open, give it a native fingerprint and its own persistent data directory, and drive it over raw CDP using only the passive domains. DOM to locate, Input to act (trusted events), Page to navigate, Network to observe. Nothing enables Runtime. Nothing announces automation. The browser is real because it is real; the only thing unusual is that its input comes from a socket instead of a mouse.

This is the architecture Dual Login is built around, and it is also why a launched profile can pass a Google sign-in that a Selenium session cannot. There is no automation state to find.

Managed profiles: the operational layer, not a different technology

At scale, the technical part stops being the hard part. The hard part is running 300 identities without cross-contaminating them: 300 coherent fingerprints, 300 persistent cookie jars that survive restarts and machine moves, the right proxy pinned to the right identity every single time, and enough isolation that one burned profile does not tell the target anything about the other 299. That is what a profile manager is for. Our guide to picking an antidetect browser for multiple accounts covers the selection criteria; the rest of this article assumes you have something that does it.

Building a pipeline that survives a rule update

Profiles are the unit of scale, not threads

Stop thinking in worker threads and start thinking in identities. One profile equals one OS process, one --user-data-dir, one fingerprint, one proxy, one cookie jar. Scale by adding profiles, not by adding tabs to a shared browser. Tabs share storage, share an IP, and share a fingerprint — three ways for one block to become fifty. Process-per-profile costs more RAM and is worth every megabyte.

Plan roughly 5 concurrent instances per 4 GB of RAM with low-memory flags on. A 32 GB workstation comfortably runs 25-40. Beyond that you want more machines rather than more density, because swapping is how you get timeouts that look exactly like blocks and send you debugging the wrong layer for two days.

Give every profile a coherent identity, then leave it alone

Generate the fingerprint once and pin it. Rotating a profile's fingerprint between sessions is one of the most common self-inflicted wounds in this field: a returning visitor whose cookies say 'seen you 40 times' while their canvas hash, GPU, and font list changed overnight is a far louder signal than a stable, unremarkable machine. Stability is the point. Real devices do not get new GPUs on Tuesdays.

Write down the identity as a unit — OS, browser version, screen, GPU pair, timezone, locale, fonts — and change it only when you deliberately retire the profile.

Pair each identity with the right proxy, and keep the pairing sticky

The proxy decision does more for your block rate than any browser tweak. Broad guidance that holds up:

• Datacenter is fine for tolerant targets, public documentation, sitemaps, and anything without a detection vendor in front of it. It is cheap and fast, so use it where it works.
• Residential is what you need for consumer platforms, marketplaces, ad networks, and anything with a serious risk engine.
• Mobile is the strongest and the most expensive, and worth it for the small number of targets that treat carrier NAT ranges as inherently human.

Whatever tier you choose: the IP-to-profile mapping must be sticky. A profile that appears from Chicago on Monday, Frankfurt on Tuesday, and Singapore on Wednesday, carrying the same cookies throughout, is describing a physically impossible device. Sticky sessions, and a timezone that follows the exit. The antidetect browser with residential proxies playbook goes into rotation windows, session length, and how to spot a poisoned pool before it burns profiles.

Worth saying plainly, because it comes up constantly: a VPN is not a substitute here. One VPN endpoint gives all your profiles the same exit IP and does nothing at all about fingerprints — the exact opposite of what you want. We wrote up the difference between an antidetect browser and a VPN for anyone who needs to explain this to a stakeholder.

Drive it without lighting up the protocol

Concrete rules that follow from the protocol section:

1. Never call Runtime.enable, and audit your framework to find out whether it does so on your behalf. If your 'read the text of this node' helper compiles to Runtime.evaluate, you are enabling it.
2. Locate nodes via DOM.getDocument and DOM.querySelector, and read text through DOM.getOuterHTML or node attributes rather than evaluated JavaScript.
3. Act through Input.dispatchMouseEvent and Input.dispatchKeyEvent so events are trusted. Move the cursor to the element's box, pause, then press — do not teleport and click in the same tick.
4. Serialise per tab. DOM.getDocument resets the backend node map, so two overlapping operations on one tab will invalidate each other's node IDs. Resolve-and-act should be one atomic unit behind a mutex. (If you have ever chased 'Could not find node with given id', this is what it was.)
5. Do not hold a long-lived debugger attachment on a sensitive tab. Attach briefly, do the work, detach. A permanently attached client is itself a state a page can notice.

Move like a person, cheaply

You do not need a machine-learning model of human behaviour. You need to not be obviously mechanical. Randomise dwell times over a realistic distribution (log-normal beats uniform). Scroll in uneven increments with occasional overshoot and correction. Move the pointer along a curve with a few intermediate points instead of jumping. Type with variable inter-key delays and the occasional pause mid-field. Let some sessions leave without converting, because a population where every session completes the same funnel in the same order is a population of bots.

And keep sessions plausible in length. Forty pages in ninety seconds is not a reading session. Sixty to a few hundred requests per profile per day, spread over hours, with gaps, looks like a person who uses the site.

Read the blocks correctly

Most wasted scraping engineering comes from fixing the wrong layer. Triage first:

That last row is worth internalising. Simultaneous failure across a fleet is almost never a detection improvement on their side — it is a shared attribute on yours.

Throughput math, so you size this once

Work backwards from pages, not from ambition.

Say you need 200,000 pages a month from a target that tolerates roughly 150 page views per identity per day before its risk score rises. That is 4,500 identity-days per month, or about 150 profiles running daily. At 5 instances per 4 GB, 150 concurrent profiles is 120 GB of RAM — which you obviously do not need, because they do not all run at once. Spread across a 16-hour window at 30 concurrent, each profile working roughly 90 minutes a day, you need about 24 GB and one decent machine.

Then cut the number before you build it:

• Fetch the cheap path first. Sitemaps, category JSON, and search endpoints often carry 60-80% of the fields you want at a fraction of the cost. Render a full page only for the remainder.
• Cache and use conditional requests. If-Modified-Since and ETag are free. A 304 costs almost nothing and does not look like scraping.
• Block what you do not need. Images, fonts, media, and third-party analytics are often 80% of page weight. Careful, though: blocking everything third-party is itself a signal on sites that expect their own beacons to fire. Block heavy media, keep first-party scripts.
• Deduplicate upstream. Most large scrapers re-fetch the same URLs more than they realise.

Halving your request count is worth more than any stealth tweak, because it halves your exposure at every layer simultaneously.

Verification: prove it before you scale it

Never scale an unverified setup. The test loop is short:

1. Baseline by hand. Launch a profile manually, browse the target like a person, and confirm it works. If a hand-driven session gets blocked, your problem is the proxy or the fingerprint, not your automation.
2. Then automate the identical path and compare. If hand-driving passes and automated driving fails, the difference is your driving layer — protocol usage, timing, or synthetic events. This A/B is the single most useful diagnostic in scraping and it takes ten minutes.
3. Run fingerprint audits. Cover Your Tracks for the uniqueness question; CreepJS and browserscan-style suites for the consistency and lie-detection question. What you want is not a perfect score — it is no contradictions and no automation flags. A tool that reports 'lies detected' has found a wrapper, and so can the target.
4. Check every context. Ask the same questions from the main frame, from an iframe, and from a Web Worker. Injected spoofing usually diverges in at least one. Native spoofing does not.
5. Keep canaries. Two or three profiles that do nothing but fetch one known-good URL every hour and record the status code. When your block rate moves, the canaries tell you whether it is the target or you, and roughly when it started.
6. Instrument block rate per profile, per exit, per target. Not a global average — averages hide the fact that one bad subnet is producing all your failures.

If you are evaluating tools rather than building, what to test during an antidetect browser free trial is essentially this checklist turned into a purchasing decision.

Staying on the right side of the line

This matters practically, not just ethically — the projects that blow up legally are usually the ones that were also technically reckless.

Scraping publicly accessible data has repeatedly been treated differently from breaking into a protected system; the long-running hiQ Labs v. LinkedIn litigation (summary on Wikipedia) is the case most often cited, and its trajectory is instructive precisely because it was messy rather than a clean win. None of that is legal advice, and none of it makes contract terms, database rights, or privacy law disappear. Sensible defaults:

• Read robots.txt and the terms of service. Choosing to deviate is a business decision that should be made deliberately and in writing, not by a crawler that never looked.
• Do not scrape behind a login you are not authorised to use. Credentialed access changes the legal picture substantially.
• Do not collect personal data you have no lawful basis to hold. 'It was public' is not a GDPR basis.
• Rate-limit like a guest. Do not degrade the service you are reading. This is also the single best way to stay unnoticed.
• Identify yourself where you reasonably can, and honour removal requests promptly.

The same discipline applies when your work is account operations rather than pure data collection — the isolation and pacing rules are identical, which is why our Amazon seller account-ban playbook reads like a scraping guide in places.

A ten-day rollout that does not waste a month

Days 1-2 — Recon. Map the target. Find every non-browser path to the data. Identify the detection vendor from response headers and challenge pages. Decide what you actually need per record; drop the rest.

Days 3-4 — One profile, by hand. Build a single coherent identity, pin one sticky residential exit, browse manually, and confirm a clean session. Run the fingerprint audits. Fix contradictions now, while you have one thing to fix.

Days 5-6 — Automate that one profile. Raw CDP, no Runtime, trusted input, humanised timing. Compare against the hand-driven baseline. Get to a stable 200 on a few hundred pages before adding a second profile.

Day 7 — Five profiles. Five identities, five exits. Now you are testing isolation. If all five fail together, you have shared something.

Days 8-9 — Instrument. Block rate per profile and per exit, canaries, structured error taxonomy mapped to the triage table above, retries with jittered backoff. Alert on rate-of-change, not on absolute numbers.

Day 10 — Scale in steps. 5, 15, 40. Hold each step for a full day and watch block rate before the next. If it degrades, you know exactly which increment caused it.

The temptation is to skip to day 10. Every team that does spends the following month unable to tell whether their problem is fingerprints, proxies, pacing, or a bug.

FAQ

Does undetected-chromedriver still work in 2026?

Sometimes, for a while, on some targets. It patches the well-known WebDriver and ChromeDriver artifacts, which is genuinely enough for tolerant sites and research work. What it does not change is that you are driving through the automation stack, so protocol-level and environment-level checks still apply, and each detection-vendor update can break you overnight. For a one-off job it is a reasonable tool. For a pipeline your revenue depends on, the unpredictable maintenance is the real cost.

Is navigator.webdriver the main thing detectors check?

It is the cheapest thing they check, not the main one. Serious detection stacks score dozens of signals: TLS and HTTP/2 shape, IP reputation, fingerprint coherence, worker-context consistency, event trust, and behaviour over time. Hiding navigator.webdriver alone gets you past the tutorials and nothing else. Treat it as table stakes.

Can I scrape at scale without a browser at all?

Often, yes — and you should try that first, because it is one to two orders of magnitude cheaper. The requirement is that your HTTP client match a real browser's TLS and HTTP/2 signature, otherwise the handshake contradicts your User-Agent and you are easier to catch than a browser would be. You will still need a browser for genuine JavaScript challenges, client-side rendering with no underlying API, and anything requiring a logged-in session.

Do I need residential proxies, or are datacenter IPs fine?

It depends entirely on the target, so measure rather than guess. Public docs, sitemaps, and sites with no detection vendor are usually fine on datacenter IPs, which are much cheaper and faster. Consumer platforms, marketplaces, and ad networks generally need residential, and a handful need mobile. Run the same scrape over both tiers for a day, compare block rates, and let the numbers decide.

Does headless mode always get detected?

Headless itself is no longer the giveaway it was — modern headless Chrome is the same binary. What gets detected is the environment headless usually runs in: software WebGL rendering, no audio device, a thin font list, a viewport identical to the screen, UTC timezone. Fix the environment and headless is fine. If you cannot fix it, run headed on a virtual display, which is often less work than chasing every difference.

How do I tell whether I was blocked by fingerprint or by IP?

Hold one variable and change the other. Same profile, three different exit IPs: if only some fail, it is the network layer. Same exit IP, three different fingerprints: if all fail identically, it is the network layer again. If one fingerprint passes where another does not on the same exit, it is the browser layer. Add the hand-launch versus automated comparison and you can isolate almost any block to a single layer within an hour.

Wrapping up

Web scraping without Selenium detection is not a trick, and there is no flag that makes an automated browser invisible. It is an architecture: a real browser instead of a patched one, identity applied natively so there is no wrapper to catch, coherent attributes that all describe the same plausible machine, a sticky and appropriately-sourced proxy per identity, protocol usage that never enables the domains which change page behaviour, trusted input events, and pacing that reads as human. Add measurement so you know which layer is failing, and the whole thing stops being a guessing game.

The teams that get this right tend to be the ones who stopped patching and changed the foundation. If that is where you are, Dual Login handles the operational half — isolated profiles with native fingerprints, persistent per-profile data directories so sessions survive restarts and machine moves, per-profile proxies, and a raw-CDP automation API that never touches Runtime.enable. Spin up a handful of profiles, run the ten-day plan above against your real target, and let your own block-rate numbers tell you whether it belongs in your stack.