RAI parser bot

RAI parser is an MVP operating system for Telegram lead generation and AI sales in the Gerchik/GTE ecosystem. The project covers the path from finding potential clients in Telegram groups to the first touchpoint, continuing the dialogue with an AI assistant, warming up the lead, and transferring it to Zoho CRM.

The system consists of an operator dashboard, a Telegram user account worker for searching/autojoining/first DM, a Telegram Business Bot for incoming dialogues, an AI agent on OpenAI, a knowledge base, and CRM handoff. A Threads section is separately planned, but in the current code, this is an intentional MVP placeholder, not a working collector.

What the product can do:

- Create search topics: keywords, language, geo, Telegram/Threads channels.
- Search Telegram groups through a real Telegram user account, not Bot API.
- Autojoin found groups with a daily limit.
- Scan messages in groups and select leads based on buyer intent: keywords plus explicit requests for help/training/consultation.
- Filter out promotions, advertisements, and irrelevant messages.
- Queue the first DM from the Telegram user account.
- Conduct Telegram Business dialogues via webhook.
- Generate responses through OpenAI Responses API with fallback responses in case of errors/timeouts.
- Process text, voice messages, images with caption/fallback.
- Transcribe voice through Whisper.
- Generate voice replies through OpenAI TTS.
- Store prompts, a knowledge base up to 100 MB, Google Docs/Sheets/Slides links, and a media bank.
- Send materials from the media bank via the service marker MEDIA:.
- Identify warm leads based on interest signals, phone, email, or consent to continue in Telegram.
- Send warm leads/leads to Zoho CRM.
- Show the operator dashboard: topics, keywords, leads, dialogues, settings, logs, Zoho events, system statuses.

Full stack:

Backend:
- Python >=3.11
- FastAPI
- Uvicorn
- Starlette SessionMiddleware
- Jinja2 templates
- SQLite via standard sqlite3
- httpx for external HTTP APIs
- OpenAI Python SDK
- Telethon for Telegram user account automation
- pytest for tests

Frontend:
- Server-rendered HTML via Jinja2
- CSS without a frontend framework
- No React, Next.js, Vite, npm/pnpm/yarn in the current checkout

Database/storage:
- SQLite file: data/gerchik_rai.db by default
- Tables: search topics, keywords, discovered groups, leads, conversations, messages, outreach events, Telegram jobs, Business threads/updates/reply jobs, Zoho sync events, logs, app settings, knowledge items, media items
- File storage next to DB:
- knowledge/ for the knowledge base
- media/ for sent materials
- telegram_sessions/ for Telethon sessions

Integrations:

- Telegram Bot API:
- Business bot webhook
- Finder bot webhook
- sendMessage, sendVoice, sendDocument, getFile, setWebhook
- Telegram user account via Telethon:
- search public groups
- join channels/megagroups
- scan group messages
- send direct messages
- mute/archive joined groups
- OpenAI:
- Responses API for AI responses
- Whisper whisper-1 for voice transcription
- TTS gpt-4o-mini-tts for voice reply
- default model: gpt-5-mini
- Zoho CRM:
- OAuth refresh token flow
- create Leads via Zoho CRM API
- Google Docs/Sheets/Slides:
- import via export URLs in txt/csv