Senior Data Collection Engineer

We're hiring a Senior Data Collection Engineer to build and maintain large-scale web scraping pipelines that operate reliably against anti-bot systems. You'll work at the intersection of browser automation, distributed systems, and stealth engineering.

What You'll Do

• Architect and maintain scraping systems handling millions of daily requests
• Orchestrate crawling workloads using Celery with Redis/RabbitMQ across distributed workers
• Implement concurrent/parallel processing strategies to maximise throughput
• Develop anti-detection techniques — cookie injection, fingerprint spoofing, user-agent rotation, header manipulation
• Manage proxy infrastructure (residential, datacenter, mobile) with intelligent retry and failover logic
• Monitor pipeline health, detect breakages, and build self-healing mechanisms

What You Bring

• 3+ years in web scraping or browser automation at scale
• Expert Python skills
• Deep Selenium and/or Playwright experience — headless browsers, dynamic content, page interaction
• Strong Celery knowledge for distributed task scheduling
• Concurrency expertise — threading, multiprocessing, asyncio
• Proven anti-detection skills:
  • Cookie/session management and user-agent emulation
  • Antidetect browser profiles (Multilogin, GoLogin, or custom)
  • Fingerprint manipulation (Canvas, WebGL, WebRTC, timezone spoofing)
• Experience with CAPTCHA-solving services and proxy management
• Solid understanding of HTTP protocols, headers, and cookies

Nice to Have

• Scrapy, Puppeteer, or similar frameworks
• Docker and cloud platforms (AWS/GCP/Azure)
• Database experience — PostgreSQL, MongoDB, or Elasticsearch
• SPA scraping and headless browser farm management