Semantic search and report generation system (RAG)

Unstructured Data Analytics System (text and media), which implements the RAG approach to provide semantic search and create structured reports based on filtered results. The interface allows users to perform deep searches across a database of text documents, images, videos, and audio, considering various filters: data type, date range, metadata.

After the search, users have the ability to create customized analytical reports generated using LLM models (including local or API-based) based on the selected content.

Key functionalities:
* Semantic search using RAG (combination of vector search and answer generation),
* Integrated filters to refine results (type, date, metadata, relevance),
* Support for multiple content types: documents, audio, video, images,
* Report generation with local LLM models for processing private data.

Technology stack:
* Python — core logic, data processing, LLM integration,
* Elasticsearch — storage and semantic search of vectorized data,
* OpenAI — use of GPT models for building RAG pipelines,
* Docker, Git, Linux — platform, deployment, CI/CD.

My role:
System architecture, implementation of indexing and semantic search mechanisms, integration of OpenAI API and local LLMs, environment setup with Docker, automation of report creation.

#python #elasticsearch #openai #llama #docker #git