Low-Latency AI Voice Agent for real-time call processing

Developed a voice AI agent for automatic handling of incoming calls, customer consultations, and transferring requests to an operator.

The system operated on a realtime pipeline:

Call → STT → AI Logic → TTS → voice response to the client

What was implemented:
— telephony / SIP integration;
— real-time speech recognition;
— AI dialogue logic and intent processing;
— voice response generation via TTS;
— minimization of delay between replies;
— scenarios for transferring to an operator;
— dialogue history preservation;
— integration with CRM / Telegram / API;
— notifications to managers about new requests.

Special attention was given to:
— minimal response delay;
— naturalness of dialogue;
— stable operation during calls;
— handling client interruptions;
— fallback scenarios;
— quality control of responses.

Result:
— AI agent responds to clients without an operator;
— reduced workload on managers;
— calls are automatically classified;
— requests are sent to CRM/Telegram;
— the system is ready for scaling to multiple lines.

Technologies:
Python / Node.js, SIP / WebRTC, STT, TTS, LLM, WebSockets, Redis, CRM API, Telegram API