Need a specialist in Big Data, ML, NLP, RAG

AI & Machine Learning, Databases & SQL — incorrectly specified categories?

Project translated automatically. Log in or register, to view the original

1. Goal

Create a tool for analyzing large arrays of text data (correspondence on a dating site) with the aim of identifying key psychological characteristics of clients.

2. Brief description of the task

There are a couple of hundred large files (several hundred pages in each file, possibly up to tens of thousands of messages per page), containing client correspondence.
It is necessary to analyze this data in terms of a number of psychological indicators.
The RAG (Retrieval-Augmented Generation) approach and vector databases are planned to be used for effective search and analysis.

3. Requirements for the specialist

Big Data: understanding of working with large volumes of data (Spark/Hadoop or analogs).
NLP / ML:
- Experience in text processing (preprocessing, tokenization, cleaning) and applying ML models.
- Knowledge of modern text analysis methods (sentiment, topic modeling, classification).
Vector Databases / RAG:
- Practical experience with Pinecone, Milvus, Weaviate (or analogs) and embedding models.
- Ability to build a Retrieval-Augmented Generation pipeline (embedding generation, similarity search, LLM integration).
Additionally:
- Ability to document solutions, explain tool choices.
- DevOps skills (Docker/Kubernetes) are welcome.

4. Expected tasks

Data preparation: reading, cleaning, structuring correspondence.
Embedding generation: model tuning (BERT, Sentence-BERT, OpenAI Embeddings, etc.).
Using vector DB: loading embeddings, searching for relevant fragments.
RAG analytics: integration with a language model to extract key characteristics of client behavior.
Assessment of psychological criteria: tuning/creating models that allow highlighting the necessary aspects. (there is a ready-made list of criteria by which clients will need to be assessed)

5. Key skills and technologies

Language: Python (pandas, scikit-learn, PySpark, HuggingFace Transformers or others).
Infrastructure: knowledge of cloud services (AWS, GCP, Azure or others) or local Big Data clusters.
Vector databases: Pinecone, Milvus, Weaviate or FAISS or others.
NLP libraries: spaCy, NLTK, as well as tools for lemmatization and cleaning.
ML pipeline: knowledge of MLOps tools (Airflow, MLflow, Docker or others).

Proposals 3

Dariia Zelinska

42 0

Budget: 100 USD Deadline: 2 days

Hello, I have experience working with Big Data, and I even once participated in a Kaggle competition for data analysis and won a prize there. I am also familiar with Kubernetes, various machine learning methods, and Python libraries, having practical experience in solving tasks. I look forward to receiving a private message.

Aaron Kravitz
Петах Тиква, Israel

Projects 12
Rating 5.0
Rating 497

Need a specialist in Big Data, ML, NLP, RAG

1. Goal

2. Brief description of the task

3. Requirements for the specialist

4. Expected tasks

5. Key skills and technologies

Dariia Zelinska

Proposals concealed

Proposals are currently absent

Current freelance projects in the category AI & Machine Learning

AI automation of telephony Binotel and chat

MATLAB and machine learning for image analysis

Multi-agent system

Counting finished products and people involved in the process based on the YOLO model.

AI assistants and aides in business and personal life