Projects · Intermediate · 4–6 hours
Chat with your documents
Build retrieval-augmented generation end to end: chunking, retrieval, a grounded prompt, citations and an evaluation.
A question-answering tool over your own documents that cites its sources and says 'I don't know' when it should.
Python 3 and NumPy where the code imports it. A handful of documents you know well, such as policies, product docs or your notes. Builds on the search engine project, which provides build_index and bm25.
Steps
Tick each step off as you finish it. Your progress is saved on this device, and to your account when you sign in.
Chunk the documents
Split by paragraphs into chunks of about 120 words with a small overlap. Each heading starts a new chunk and is kept with it.
Python def chunk(doc_name, text, max_words=120, overlap=20): """Split by paragraphs, packing them into chunks of about max_words, with a little overlap. Each chunk remembers its document and the most recent heading.""" chunks, current, heading = [], [], "" for para in [p.strip() for p in text.split("\n\n") if p.strip()]: if para.startswith("#"): # a new section starts a new chunk if current: chunks.append({"doc": doc_name, "heading": heading, "text": " ".join(current)}) current = [] heading = para.lstrip("# ").strip() continue words = para.split() if current and len(current) + len(words) > max_words: chunks.append({"doc": doc_name, "heading": heading, "text": " ".join(current)}) current = current[-overlap:] current += words if current: chunks.append({"doc": doc_name, "heading": heading, "text": " ".join(current)}) return chunksCheck: No chunk is much longer than 120 words, and each knows its document and heading.
Retrieve chunks
Index the chunks with the BM25 code from the search project and fetch the best few for a question.
Python def build_chunk_index(docs): chunks = [c for name, text in docs.items() for c in chunk(name, text)] as_docs = {str(i): f"{c['heading']} {c['text']}" for i, c in enumerate(chunks)} return chunks, build_index(as_docs) def retrieve(question, chunks, index, k=4): return [chunks[int(i)] for i, _ in bm25(question, index)[:k]]Check: For a question about refunds, the refund section comes back first.
Build the prompt
Number the sources, tell the model to answer only from them, cite them, and admit when it doesn't know.
Python def build_prompt(question, passages): sources = "\n\n".join(f"[{i + 1}] ({p['doc']} · {p['heading']})\n{p['text']}" for i, p in enumerate(passages)) return ( "Answer the question using only the sources below. Cite sources like [1] after each claim. " "If the sources don't contain the answer, say you don't know.\n\n" f"<sources>\n{sources}\n</sources>\n\nQuestion: {question}" )Call a model
Any chat model with an OpenAI-compatible API works. This defaults to Groq, which has a free tier: create a key at console.groq.com and set it as LLM_API_KEY. To use another provider, set LLM_BASE_URL and LLM_MODEL too. Temperature 0 keeps answers steady.
Python import os import requests def chat(messages, **extra): """Call any OpenAI-compatible chat API and return the reply message. Defaults to Groq (free tier at console.groq.com). Set LLM_API_KEY, and optionally LLM_BASE_URL and LLM_MODEL to use another provider.""" base = os.environ.get("LLM_BASE_URL", "https://api.groq.com/openai/v1") body = {"model": os.environ.get("LLM_MODEL", "openai/gpt-oss-20b"), "messages": messages, "temperature": 0, **extra} r = requests.post(f"{base}/chat/completions", headers={"Authorization": f"Bearer {os.environ['LLM_API_KEY']}"}, json=body, timeout=60) if not r.ok: # show the provider's reason, e.g. a wrong key or model name raise RuntimeError(f"LLM API error {r.status_code}: {r.text[:300]}") return r.json()["choices"][0]["message"] def answer(question, chunks, index): passages = retrieve(question, chunks, index) reply = chat([{"role": "user", "content": build_prompt(question, passages)}])["content"] return reply, passagesCheck: Asked 'How long do I have to return something?' about the refund policy, it answers '30 days … [1]', citing the right source.
Check the citations
Every answer should cite sources that exist. Flag answers that cite nothing.
Python import re def check_citations(answer, passages): cited = {int(n) for n in re.findall(r"\[(\d+)\]", answer)} problems = [] if not cited and "don't know" not in answer.lower(): problems.append("no citations") problems += [f"cites [{n}], which doesn't exist" for n in sorted(cited) if not 1 <= n <= len(passages)] return problemsCheck: Answers without citations, or citing [7] when there were 4 sources, get flagged.
Evaluate it
Write 20 real questions, including 3 your documents can't answer. For each, record whether the right chunk was retrieved, and whether the answer was correct and cited. Fix retrieval first: an answer can't be better than its sources.
- Answers cite the passages they used
- Says it doesn't know when the documents don't cover the question
- Retrieval finds the right chunk for most of your 20 questions
Go further
- Add vector search alongside BM25
- Show the cited passages under each answer
- Stream the answer as it's generated
This project is part of Pro
The first two projects are free. Pro opens every lesson and every project.
See Pro plansFrom ₹119 a month. No auto-renewal.