Need a specialist in Big Data, ML, NLP, RAG

AI & Machine Learning, Databases & SQL — incorrectly specified categories?

Project translated automatically. Log in or register, to view the original

1. Goal

Create a tool for analyzing large arrays of text data (correspondence on a dating site) with the aim of identifying key psychological characteristics of clients.

2. Brief description of the task

There are a couple of hundred large files (several hundred pages in each file, possibly up to tens of thousands of messages per page), containing client correspondence.
It is necessary to analyze this data in terms of a number of psychological indicators.
The RAG (Retrieval-Augmented Generation) approach and vector databases are planned to be used for effective search and analysis.

3. Requirements for the specialist

Big Data: understanding of working with large volumes of data (Spark/Hadoop or analogs).
NLP / ML:
- Experience in text processing (preprocessing, tokenization, cleaning) and applying ML models.
- Knowledge of modern text analysis methods (sentiment, topic modeling, classification).
Vector Databases / RAG:
- Practical experience with Pinecone, Milvus, Weaviate (or analogs) and embedding models.
- Ability to build a Retrieval-Augmented Generation pipeline (embedding generation, similarity search, LLM integration).
Additionally:
- Ability to document solutions, explain tool choices.
- DevOps skills (Docker/Kubernetes) are welcome.

4. Expected tasks

Data preparation: reading, cleaning, structuring correspondence.
Embedding generation: model tuning (BERT, Sentence-BERT, OpenAI Embeddings, etc.).
Using vector DB: loading embeddings, searching for relevant fragments.
RAG analytics: integration with a language model to extract key characteristics of client behavior.
Assessment of psychological criteria: tuning/creating models that allow highlighting the necessary aspects. (there is a ready-made list of criteria by which clients will need to be assessed)

5. Key skills and technologies

Language: Python (pandas, scikit-learn, PySpark, HuggingFace Transformers or others).
Infrastructure: knowledge of cloud services (AWS, GCP, Azure or others) or local Big Data clusters.
Vector databases: Pinecone, Milvus, Weaviate or FAISS or others.
NLP libraries: spaCy, NLTK, as well as tools for lemmatization and cleaning.
ML pipeline: knowledge of MLOps tools (Airflow, MLflow, Docker or others).

Proposals 3

Dariia Zelinska

42 0

Budget: 100 USD Deadline: 2 days

Hello, I have experience working with Big Data, and I even once participated in a Kaggle competition for data analysis and won a prize there. I am also familiar with Kubernetes, various machine learning methods, and Python libraries, having practical experience in solving tasks. I look forward to receiving a private message.

The list does not show proposals concealed by the client or freelancer with a Plus profile, as well as proposals violating rules

Current freelance projects in the category AI & Machine Learning

ML (computer vision ) engineering project

Data Processing 16 proposals 4 August

Not specified
Website creation

67 proposals 4 August

22 USD
Development of software (ROS 2 / Nav2) for a 4x4 autonomous robot: Computer vision, asymmetric navigation

14 proposals 2 August

Not specified
I am looking for an AI bot developer (ChatGPT/OpenAI)

AI Consulting 87 proposals 1 August

Not specified
Integration of an AI agent in Manychat for processing incoming messages

AI Consulting 52 proposals 31 July

Not specified

Aaron Kravitz
Петах Тиква, Israel

Projects 12
Rating 5.0
Rating 489

Dariia Zelinska

Proposals are currently absent

Current freelance projects in the category AI & Machine Learning

ML (computer vision ) engineering project

Website creation

Development of software (ROS 2 / Nav2) for a 4x4 autonomous robot: Computer vision, asymmetric navigation

I am looking for an AI bot developer (ChatGPT/OpenAI)

Integration of an AI agent in Manychat for processing incoming messages