David Thomson

CTO

David is the CTO of TechnologyChecker, responsible for the engineering and architecture behind the platform's crawling infrastructure. Before joining TechnologyChecker, he spent five years at Google on the Search team, where he worked on large-scale crawling and indexing systems that shaped his approach to building high-performance data infrastructure.

He oversees the detection systems that scan over 50 million domains monthly, ensuring accurate and timely identification of technology stacks across the web. His work focuses on scalable data pipelines, real-time processing, and maintaining detection accuracy across HTTP headers, JavaScript libraries, DNS records, and HTML patterns.

Based in Edinburgh, David is a devoted single malt whisky enthusiast when he's not architecting distributed systems.

Areas of Expertise

Scalable Data PipelinesReal-Time ProcessingWeb Crawling ArchitectureDistributed Systems

Credentials

  • MEng Computer Science, University of Edinburgh
  • AWS Solutions Architect Professional
  • Contributor, Open Source Crawling Frameworks

Achievements

  • Architected infrastructure processing 50M+ domain scans per month with 99.9% uptime
  • Reduced detection latency from hours to under 60 seconds for real-time alerts
  • Built a technology fingerprinting engine covering HTTP headers, JS libraries, DNS records, and HTML patterns
  • Analysed petabytes of Common Crawl data to power TechnologyChecker's historical technology adoption database
David Thomson

15+ years of experience

Articles by David Thomson

Web Traffic Statistics Q1 2026: We Analyzed Billions of Requests - Here Are the 15 Numbers That Matter
Updated

Web Traffic Statistics Q1 2026: We Analyzed Billions of Requests - Here Are the 15 Numbers That Matter

We pulled Q1 2026 data from Cloudflare Radar's global network — 81M+ HTTP requests/second across 125+ countries. 31% of all traffic is bots. AI crawlers now represent 22% of bot traffic with Applebot surging 140% in a single month. 60% of login attempts use leaked credentials. And 93.7% of web traffic runs on modern TLS. Here are the 15 numbers that matter.

David ThomsonDavid Thomson
DMARC adoption statistics 2026: 89% of emails pass DMARC but 14.5% still fail SPF
Updated

DMARC adoption statistics 2026: 89% of emails pass DMARC but 14.5% still fail SPF

We pulled 90 days of Q1 2026 email security data from Cloudflare Radar and cross-referenced it against Valimail, Red Sift, the Verizon DBIR, and the FBI IC3 report. DMARC pass rates hit 88.99%, but only 42% of domains actually enforce it. SPF jumped 6 points YoY to 80.24%, BEC losses hit $2.77B, and 27.61% of email still uses deprecated TLS 1.0. Here's what the numbers say about email authentication maturity.

David ThomsonDavid Thomson
David Thomson - TechnologyChecker.io