Chat with scientific papers (RAG)
A local RAG app for chatting with academic papers, featuring section-aware chunking, MemoRAG and re-ranking. It runs entirely on open-source models through Ollama.
The problem
Searching through papers for one specific detail is slow, and most “chat with your PDF” tools split the text into fixed-size chunks that ignore the document’s structure, so a fragment of Methods ends up glued to a fragment of Results. That hurts retrieval in exactly the kind of document where structure matters most.
What I built
A Streamlit app that ingests a PDF and answers questions about it, citing the exact page each claim comes from. Everything runs locally through Ollama, so documents never leave your machine.
- Section-aware chunking: detects sections such as Abstract, Introduction, Methods, Results and Conclusion (in both English and Spanish) and applies a chunk size tailored to each one.
- MemoRAG: a lightweight model builds a global summary of the paper and turns each question into search “clues”; chunks that match several clues are ranked higher. It can be switched on and off to compare it against standard RAG.
- Re-ranking: retrieves 12 candidate chunks and lets a cross-encoder keep the 6 most relevant.
- Optional query expansion, using LLM-generated variations of the question.
- Language detection, so answers and citations match the language of the document (
[pg. 3]/[pág. 3]). - A process panel that exposes every step of the pipeline: query expansion → clues → retrieved chunks with their similarity scores.
- Persistent conversation history in SQLite, plus 👍/👎 feedback on each answer.
Stack
Python, LangGraph, Ollama (deepseek-r1, llama3.2, mxbai-embed-large), ChromaDB, sentence-transformers, PyMuPDF and Streamlit.