Bot Defense

Multi-layered bot detection: PoW challenges, heuristic scoring, and Cloudflare integration.

Overview

Every incoming request is scored from 0 (definitely human) to 100 (definitely bot). Views and events with bot_score ≥ 50 are excluded from daily rollups. Known bot User-Agents are dropped entirely before scoring.

Known Bot UA Filter

Requests matching any pattern in ECHELON_BOT_UA_PATTERNS are dropped before scoring. Default list includes:

• Googlebot, Bingbot, Yandex, Baidu, DuckDuckBot
• GPTBot, ClaudeBot, ChatGPT-User, Google-Extended
• curl, wget, python-requests, HeadlessChrome, Puppeteer, Playwright
• And many more crawlers and automation tools

Bot Scoring Factors

Score is capped at 100.

Every weight in this table can be changed with ECHELON_BOT_SCORE_WEIGHTS (key:points,key:points, 0–100 each; 0 turns the signal off). See Configuration for the keys. The datacenter lists refresh every six hours; a provider whose refresh fails, is empty or shrinks to under 10% of its previous size keeps its old ranges.

The Windows tablet, cross-site and datacenter weights are 34, 34 and 24 rather than round numbers on purpose: a real visitor can carry any one of them, and one who then reads without clicking gets the +15 for no interaction. The sums stay under the threshold of 50. Linux desktop is scored on pageviews only, not on the events endpoint, where an invalid token is already +40.

Home Audience

Each site can declare a home country and language under Settings → Home Audience (or PATCH /api/sites/:id). The geography signals are measured against it and do nothing until it is set; they also need a visitor country, which comes from ECHELON_BEHIND_CLOUDFLARE or ECHELON_TRUST_GEO_HEADERS. Visitors from the same continent are never a mismatch, and a country the continent table does not know is never foreign. On English-language sites (or with no language set), visitors from countries that appear everywhere (US, CA, AU, NZ, JP, KR, SG, IN, TR, RU, MA, AE, MX, PK) are exempt from the country mismatch. Leave it empty for a worldwide audience.

The home country must be an assigned code that Echelon can place on a continent, or RU. Anything else (UK for GB, EN, XX) is refused with a 400, and a code of that kind already stored is treated as not set. Russia spans two continents and has none in the table, so with a home of RU no visitor shares its continent: every visitor from another country is a mismatch, neighbours included (the exemption above for countries that appear everywhere still holds).

Opting in has a cost for real visitors:

• Non-English sites. A Linux desktop or an Android 6 or older visitor from one of those countries scores 63 from these signals alone, over the threshold of 50, and is not counted. Other foreign visitors carry 15 to 23 points.
• Windows scaled displays. The tracker reports screen.width in CSS pixels, so a real Windows laptop with display scaling can land in the 768–1024 band (for example 1280 wide at 125%, or 1366 at 150%) and carry +34. Lower win_tablet_mismatch or set it to 0 if your audience uses such displays.
• Sibling sites. Cross-site counts an address and browser on three or more of your sites within an hour. A person who browses sites you run together will trip it. Browsers now send nearly identical user agents, so people who share one address (an office, a campus, a mobile carrier) pool into one count as well. Set cross_site:0 on an instance whose sites share an audience. It is scored on the pageview beacon only, and never when the client address cannot be resolved (no peer address, or a forwarded header arriving without ECHELON_TRUST_PROXY).

Session Classifier

Some bots are only visible across requests. Every 15 minutes Echelon reviews the last 30 minutes, judging a visitor or session once its oldest view in that window is 12 minutes old:

• Session velocity — six or more page visits, to six or more different pages, averaging three seconds apart or less. Pages are told apart by pathname, and consecutive views of one page count once: views that differ only in query string or fragment (pagination, a carousel, a map that keeps its state in the URL) are one page.
• OS rotation — three or more names of one operating-system vendor family on one session or visitor.
• OS family flip — one session or visitor on two operating-system vendors (Apple, Android, Windows).
• Burst cohort — forty or more single-page sessions with the same country and screen size, from a different continent than the site’s home country (needs a home country; does nothing when either country has no known continent).

The first run is two minutes after start. Flagged rows are scored 60 and the visitor is excluded automatically as auto:<reason> for 30 days; removing the exclusion on the admin bots page reverses a wrong one, and excluding such a visitor by hand replaces the auto: label, which makes the exclusion permanent. Validated visitors are spared by session velocity and OS rotation, but not by OS family flip and burst cohort, whose patterns cannot be one real person. As a side effect, browser DevTools device emulation or a user-agent switcher in one tab can trip OS family flip on yourself. On anonymized sites, findings keyed on the session are skipped.

Duplicate Beacons

Mobile networks and service workers sometimes deliver the same pageview request more than once. A beacon with the same site, session and per-pageview nonce as one recorded in the last ten minutes is answered normally but counted only once. The tracker draws a random nonce for each landing pageview, so a reload or new page load is counted. A beacon without a well-formed nonce is always counted, and SPA navigation beacons and beacons without a session id are exempt.

Validated Visitors

A bot score measures suspicion. Validated measures proof: the session did something only a person does. The tracker reports the first click, key press or tap on each page as an engage event. A background validator (every minute) marks a view validated once its session has an interaction event and stayed on the page for at least two seconds, or showed a later interaction plus two distinct interaction types.

• Interaction events prove a gesture: engage, click, ad_click, form_focus, form_blur, form_submit, hover, outbound, download.
• Passive events never do: bounce, session_end, session_resume, scroll_depth, web_vital, custom. A headless page load emits all of these.
• Retroactive penalties skip validated visitors. The fingerprint correlator waits twelve minutes before judging a visitor and never penalises one who has been validated from a non-datacenter IP. Validated views are also exempt from the engagement gate.
• The engagement gate judges the other views. A view with no interaction 5–30 minutes after it happened gets +15; if it also carries a PoW penalty, or its PoW was waived only by the visitor key (pow_ok_by: "v"), it is raised to the threshold instead. A session that interacted at any time is shielded as a whole. This is why a missing token costs only +30 at ingest: a browser without WebAssembly looks like a bot, but one that also never interacts does not stay forgiven.
• A later gesture undoes the gate. Someone who reads with a mouse wheel or trackpad may not click or press a key until after their view was judged. Each sweep first looks back 180 minutes: if the session has interacted since, exactly the points the gate added (stored in the no_events or unproven mark) are taken off again and the view can be validated. A view that the correlator or strict mode has judged in the meantime is left as it is.
• Strict mode is optional. With ECHELON_REQUIRE_ENGAGEMENT=true, views and events of sessions that never interact are raised to the bot threshold 30–180 minutes after they were written and stop counting as human. This is not undone by a later gesture. It is off by default: people who read and leave are real visitors.
• Anonymized sites are judged by visitor id. Session ids on sites in ECHELON_ANONYMIZE_SITES are one of 50 names that unrelated visits share, so on those sites the validator and the gate ignore them: an interaction counts for a view or event only if it has the same anonymized visitor id and happened within 30 minutes of it.

The overview API returns validated_visitors and the dashboard shows it next to unique visitors.

Proof-of-Work System

Every request for /ea.js embeds a WebAssembly blob and a challenge string. The browser must solve the challenge before sending beacons.

Why PoW Blocks Robots

Most analytics spam comes from scripts that fire HTTP requests directly — curl, python-requests, headless browsers running thousands of sessions. Proof-of-work stops them because:

• Requires a real WASM runtime. The challenge can only be solved by executing a WebAssembly module. Simple HTTP clients can’t do this — they’d need to embed a full WASM engine, which eliminates the vast majority of bot scripts.
• Costs CPU time per identity. A bot that does run the WASM has to solve at least once for every identity it presents. A session or visitor that has presented a valid token is not penalised for a missing or stale one for the next 60 minutes, so a real browser solves once and is never slowed down again — while a bot farm that rotates IPs and user agents to look like many visitors pays the CPU cost for each of them, and every other signal still applies to all of its requests.
• Rotates unpredictably. The WASM module is regenerated every 6 hours from a seed derived from your instance secret, and the challenge string changes every minute. Bots can’t precompute answers or hardcode solutions — they’d need to re-fetch and re-solve constantly.
• Binds to session and site. The solve includes the session ID and site ID, so a token solved for one session or site does not verify for another. On the site it was solved for, that proof then vouches for the session and visitor for 60 minutes (the PoW waived row above); it never carries over to another site.
• Invisible to real users. Unlike CAPTCHAs, the PoW runs in the background in under 150ms. Visitors never see it, never interact with it, and never get blocked by it. The cost falls entirely on automation.

How It Works

• A WASM module is generated every 6 hours from a 64-byte seed derived from ECHELON_SECRET (random per process if no secret is set)
• Implements a SipHash-inspired algorithm with randomized constants
• Challenge string rotates every minute (HMAC-SHA256 of minute bucket)
• Client computes: token = wasm.solve(challenge + ":" + sessionId + ":" + siteId)
• Token is a 32-character hex string
• The tracker fetches a fresh challenge from /c and re-solves once its token is 4 minutes old, so long-lived tabs stay valid
• Cached in sessionStorage and reused while the challenge matches; about one page load in ten ignores the cached token and solves again
• WASM solve is awaited up to 150ms before sending the beacon

Content Security Policy on Your Site

If your site sends a Content-Security-Policy header, it must let the tracker reach your Echelon host: the host in script-src (to load /ea.js), in img-src (the /b.gif pixel) and in connect-src (for /c and /e), plus 'wasm-unsafe-eval' in script-src so the proof-of-work module can run:

script-src 'self' https://your-host 'wasm-unsafe-eval';
connect-src 'self' https://your-host;
img-src 'self' https://your-host

Without 'wasm-unsafe-eval' the tracker still works but sends no token, so its requests carry the +30 missing-token penalty instead of being discarded at ingest. A view with no token and no interaction is later raised to the bot threshold by the engagement gate (see Bot Defense).

Server Verification

The server tries all combinations of:

• Current + previous WASM slots (covers 6-hour rotation)
• Last N minute buckets (configurable via ECHELON_CHALLENGE_WINDOW_MINUTES, default 10)

Returns "valid", "missing", or "invalid" — each adding different penalty points.

Burst Detection

• Window: 5 minutes per IP
• Threshold: 15 requests triggers +25 penalty
• Map size: max 100,000 entries (pruned when exceeded)

Rate Limiting

Tracking endpoints (/b.gif, /e) and the challenge endpoint (/c, the current PoW challenge the tracker uses to refresh its token; public and CORS-enabled) are rate-limited per IP:

• Default: 100 requests per 60-second window
• Configurable via ECHELON_RATE_LIMIT_MAX and ECHELON_RATE_LIMIT_WINDOW_MS

🗑️ "Did you know you can set ECHELON_BOT_DISCARD_THRESHOLD to immediately drop high-score requests before they're stored? This saves database space when you're confident in your scoring." -🦭

Cloudflare Integration

Set ECHELON_BEHIND_CLOUDFLARE=true to enable:

• Read cf-ipcountry for geo data (ignores XX and T1)
• Read Cloudflare bot score from cf-bot-score header
• Detect CF-verified bots
• Use cf-connecting-ip for real client IP

Referrer Classification

Incoming referrers are classified into categories:

Manual Exclusion

Via the admin dashboard or API, you can manually exclude specific visitor IDs from rollups. The session classifier adds its own exclusions labelled auto:<reason>, which daily maintenance removes after 30 days; manual ones are never removed automatically.

# Exclude
POST /api/bots/exclude
{ "visitor_id": "abc123", "label": "Known bot" }

# Re-include
DELETE /api/bots/exclude/abc123

Discard Threshold

Requests scoring at or over ECHELON_BOT_DISCARD_THRESHOLD are discarded before they are stored. The default is 50, the bot threshold. Set it higher (e.g., 80) to store the rows in between, or to 0 to store everything. Stored rows scoring 50 or more are still left out of all statistics; they stay visible, with their score detail, on the admin bots pages. A discarded request cannot be recovered, so 0 is the setting to audit new signals with (see Upgrading).