Build a Secure Local AI System (LLM, RAG & AI Agents)

About the Project

We are looking for an experienced AI Automation Engineer to design and build a secure, self-hosted AI platform that combines a local Large Language Model (LLM), Retrieval-Augmented Generation (RAG), and multiple AI agents to automate business workflows.

This is a hands-on engineering role for someone who has experience building production AI systems—not simply integrating ChatGPT APIs.

The goal is to create a private AI ecosystem capable of securely indexing company knowledge, answering questions using cited sources, processing meeting transcripts, and automating internal business processes.

Responsibilities

You will be responsible for:

• Designing and deploying a locally hosted LLM on a VPS or dedicated server
• Building a secure RAG pipeline using frameworks such as LlamaIndex or similar
• Creating document ingestion pipelines supporting PDF (including OCR), DOCX, TXT, XLSX and meeting transcripts
• Implementing document indexing, metadata management, deduplication, and versioning
• Developing AI agents for:
  • Meeting transcription processing
  • Automatic meeting summaries
  • Action item extraction
  • Client knowledge retrieval
• Building APIs or a simple web interface for querying the knowledge base
• Ensuring strict client data isolation and permission controls
• Implementing source-cited responses to minimize hallucinations
• Optimizing system performance, scalability, and reliability
• Writing documentation and deployment guides
• Performing testing and security validation

Required Skills

• Strong Python development experience
• Experience with LLM frameworks
• RAG architecture experience
• LlamaIndex, LangChain, or equivalent
• Vector databases (Qdrant, Chroma, Pinecone, Weaviate, FAISS, etc.)
• Local/open-source LLM deployment (Llama, Mistral, Gemma, DeepSeek, etc.)
• API development (FastAPI preferred)
• Docker
• Linux server administration
• VPS deployment
• Git
• Authentication and access control
• Experience with OCR pipelines
• Experience working with structured and unstructured documents
• Fluent English

What We’re Looking For

The ideal candidate:

• Has built production AI systems from the ground up
• Understands RAG best practices
• Can work independently
• Thinks like a software architect—not only a developer
• Writes clean, maintainable code
• Communicates clearly
• Can recommend the best technologies instead of simply following instructions

Project Type

• Freelance / Contract
• Remote
• Milestone-based
• Long-term opportunity for future AI automation projects

Please Include With Your Application

• Portfolio of similar AI/RAG projects
• Examples of local LLM or AI agent implementations
• Estimated timeline
• Estimated project cost
• Hourly or fixed-rate pricing