Offline RAG Benchmark Platform

I need a fully-offline Retrieval-Augmented Generation platform that lets me benchmark several small language models side-by-side while keeping every byte of data on-prem. The core workflow is straightforward: I drop in PDFs, CSVs, or DOCX files, the system indexes them into a persistent FAISS vector store, and an interactive Streamlit front-end gives me document upload, semantic search, and response generation in one place. Under the hood, the app should use Python with LangChain to orchestrate local models served through Ollama (Qwen2.5, Llama3.2, Phi3 for the first iteration). The interface must surface at least two key numbers for each model on every query—its latency and the text response itself—so I can judge speed against output quality at a glance. No cloud calls, no telemetry: everything runs offline on the host machine for maximum privacy. Deliverables • Clean, well-commented Python codebase (Streamlit UI, LangChain pipelines, FAISS setup, Ollama integration) • Instructions to add or swap local models with minimal edits • A sample dataset and walkthrough that prove PDFs, CSVs, and DOCXs index and query correctly • Read-me covering environment setup, hardware requirements, and how latency is captured/reported If you have prior experience wiring LangChain to Ollama or have built similar RAG evaluators, let’s get this running quickly.

Project ID: 40411010