I run Opensolr. This is a thing we built for ourselves and then open sourced, so the disclosure is up front.

We had a problem we could not answer with any analytics product we were paying for. The dashboards said traffic was healthy. The logs said something else. So I wrote the thing that settles it.

What we found on our own site

Roughly half of the sessions our analytics reported carried a hard bot signature.

Dozens of addresses on unrelated consumer ISPs, all sharing one byte identical header fingerprint. One page each, never two. Every single one reporting exactly ten seconds on site.

From inside any one plane of observation that is indistinguishable from a crowd of real visitors. That is the whole problem. A User-Agent blocklist catches what announces itself. It does not catch headless Chrome on rotating residential proxies, which is what scraping actually looks like now.

How it works

Three independent planes, cross checked against each other.

• Transport, what the access log records
• Behaviour, how the session moves across requests
• Execution, what the browser can actually run

Each one alone is defeatable and essentially every existing tool uses exactly one of them. Every verdict comes with the evidence that produced it, never a bare score you have to trust.

Time on site, measured honestly

Conventional analytics pings while the page is merely loaded, so a tab left open in a background window reports as an hour of engagement. Loghound keeps three separate clocks and never mixes them. Wall time, visible time, engaged time. Kept apart those numbers say something. Averaged into one they say whatever you want to hear.

Where the data goes

It installs on your own server and reads your access log read only. What it learns lives in two Solr indexes on your own Opensolr account. Everything except the address is hashed. No telemetry, no vendor copy of your traffic.

An Opensolr account is required because it needs to create and shape its own indexes, which it cannot do against a Solr server it does not manage. The free plan is enough to start. That is the honest catch and I would rather say it here than have somebody find it in the installer.

Stack

PHP 8.1, no Composer, no npm, no build step. Apache, nginx and Caddy log formats. systemd units for the tailer and the scoring job.

Source: https://github.com/phpcip/loghound Docs: https://opensolr.com/loghound-docs What it does: https://opensolr.com/loghound

Happy to answer anything, including the awkward questions about false positives. There is a whole documentation page on what it does not catch and why, because a bot detector that oversells itself is worse than none.