Production AI pipeline processing thousands of conversations weekly

An enterprise SaaS platform needed to transform raw video conversations into structured, longitudinal workforce insights at scale. Each week, over a thousand 1:1 conversations needed to be transcribed, analysed for themes and sentiment, benchmarked against historical data, and distilled into actionable recommendations for HR leaders.

The existing manual process could not scale, and off-the-shelf NLP tools lacked the domain specificity required for workforce intelligence. The system also needed to handle EU data residency requirements after an external evaluation vendor was blocked by GDPR compliance.

We architected a modular 6-stage pipeline — transcription, structuring, summarisation, thematic classification, emotion detection, and population-level benchmarking — with each stage independently callable and model-agnostic. We built a provider-agnostic LLM abstraction layer with runtime cost and latency telemetry, enabling zero-downtime switching between providers.

For theme classification, we developed a hierarchical taxonomy system using PCA dimensionality reduction and Voronoi tessellation for automated grouping. When the external evaluation vendor was blocked by data residency requirements, we built a custom LLM-as-a-Judge framework with 10 evaluators producing 48+ dimension scores, deployed on serverless AWS infrastructure.

The pipeline now processes 1,000+ conversations weekly with full observability across every stage. Enterprise clients receive population-level benchmarks that reveal workforce sentiment variations across time periods, cohorts, and roles.

The evaluation framework replaced the external vendor entirely, achieving GDPR compliance while providing richer quality assessment. The system has been extended to support a joint engagement with PwC and a UK police constabulary for call classification, achieving 85% accuracy on thematic classification.