Development of a high-load system with fine-tuning of LLM models

Good day, Nick.

In brief:

Your service will receive a multimodal search capable of simultaneously processing photo and text queries through a personal agent in Telegram or WhatsApp. The search core based on a fine-tuned LLM and vision models will ensure high accuracy of results. We will enhance text search with a vector index pgvector to instantly find similar products even with incomplete descriptions. The system will operate on Kubernetes with automatic scaling, so peak loads will not affect speed. Integration through messengers will allow users to send photos and text in chat, and the assistant will return a selection of products from your catalog. Fine-tuning on your data guarantees accuracy unattainable for general models.

More details:

Multimodal search requires deep fine-tuning of the LLM for your catalog; otherwise, relevance will remain at the level of generic models. The search core will combine a base model with additional training on your data and a vector index pgvector for semantic search. The vision component will extract features from photos, while text will clarify the context—both streams will merge into a single ranked result. The architecture on Kubernetes with independent scaling of nodes (vision, text, ranking) will ensure stable latency. An event-driven architecture with message queues will protect against query loss during traffic spikes, and the user will receive product cards with direct links.
…
The next step is to increase search accuracy through a hybrid RAG approach with fine-tuning. This will allow the agent to instantly update knowledge about stock and prices without retraining the model, which is critical for a dynamic e-commerce catalog.

Later on, conversion will increase thanks to an analytical module that will track the most popular visual queries. You will gain a clear understanding of what users are searching for to optimize purchasing and marketing.

I am ready to discuss the details of the catalog, query volumes, and the choice of the base model for fine-tuning.

Show original

Translate

We have experience in building high-load architectures and fine-tuning LLM for multimodal tasks. We implement the system through microservices in Python using vector databases for fast search and integrating messenger APIs for agent operation. We will ensure scalability and accuracy in processing requests for photos and text. We are ready to start designing the architecture.

Show original

Translate

We already have a practically ready similar solution for AI search and assistants in messengers, which can be quickly adapted and launched for your catalog ))
For the task, I would estimate the first working stage at 250,000 UAH and about 35 working days for a prototype with photo and text search, ranking, an agent, and basic integration into messengers.
Look, there’s a nuance here - for the industrial version, it’s crucial to check in advance !!the load, search quality, and catalog structure!!, otherwise fine-tuning can become an expensive toy without noticeable gains.
For implementation, I would go through a data layer, vector and text search, a separate ranking layer, agent scenarios, logging requests, and quality metrics on the service side.
From you, I need the catalog export, photos, descriptions, output rules, a list of messengers, and access to the test API.
I’d like to clarify two points - what is the size of the catalog and what is the target load in requests per minute?
Which messengers are needed for the first launch and is there already an API for the catalog?
Similar examples for AI and agents - https://business.ingello.com/vorfahr and https://business.ingello.com/fractal
As a close example for e-commerce and catalog logic - https://business.ingello.com/prime-eva
… More about us for the exchange - https://systems-fl.ingello.com
In touch, we can discuss the details within the project =)

Show original

Translate