We monitor news and social media at scale, surface insights through an LLM-powered chat assistant (RAG over our content), and present everything through rich dashboards and analytics. We’re looking for a QA Engineer to own quality across data pipelines, AI-generated responses, and complex visualisations.
This isn’t a click-through-the-UI role — you’ll test systems where “correct” is sometimes probabilistic, data volumes are large, and a wrong number on a dashboard erodes trust.
What you’ll do
• Own test strategy and automation across ingestion pipelines, APIs, the RAG chat system, and dashboards
• Validate data integrity end to end — from ingestion to the metrics users see
• Evaluate LLM/RAG output: retrieval relevance, factual grounding, hallucination detection, and response consistency
• Verify analytics: aggregations, time-series, sentiment/engagement metrics, and dashboard accuracy against source data
• Run performance/load testing for high-volume ingestion and concurrent dashboard use
• Integrate tests into CI/CD and define quality gates
What we’re looking for
• 3+ years in software QA, including test automation
• Strong with an automation framework (Playwright, Cypress, Selenium) and API testing
• Comfortable writing test code (Python preferred) and querying with SQL
• Experience testing data-heavy applications — pipelines, dashboards, or analytics
Nice to have
• Experience testing LLM/RAG or AI/ML systems and building evaluation datasets
• Familiarity with LLM eval tooling (Ragas, LangSmith, DeepEval)
• Social media API experience; performance tools (k6, Locust, JMeter)