AI Agents & RAG55 minutes

RAG Document API

Build the bounded evidence context for a PDF question-answering API.

Python

Task

A document question-answering API retrieves passages from uploaded PDFs and sends them to a model as evidence. Build the context builder that turns retrieved passages into the model’s bounded evidence context, with the page-level sources the answer API cites. Ingestion, retrieval, the LangGraph workflow and the model adapters are supplied and run locally.

Requirements

  1. Return exactly {context, sources}. Format each included passage as "[id] document=document_id page=page", a newline, then its stripped text. Join sections with two newlines.
  2. max_chars is an integer from 1 to 12000, not a boolean (ValueError containing max_chars). Headers and separators count against it. Skip a whole passage that does not fit and continue with later, smaller ones; never truncate a passage.
  3. passages is a list of objects with a nonblank string id and document_id, a positive integer page (not a boolean), and string text. Reject anything else with ValueError containing passage.
  4. Preserve retrieval order, keep only the first occurrence of each id, and skip blank text. sources lists {id, document_id, page} for each included section. Do not mutate the input.
How you are graded
  • Decomposition 15%
  • Prompting the assistant 10%
  • Verification 20%
  • Judgment about AI output 20%
  • Correctness 25%
  • Communication 10%

Scores use evidence from your session. A short or incomplete session may not provide enough evidence for every skill. See a sample scorecard

Interview guidance

This API answers questions over uploaded PDFs, and an answer is only as trustworthy as the evidence block we hand the model. Use the AI assistant however you like. Your part is the context builder. I will be listening for how you reason about the budget, and about what an uploaded document can do to the model's view of its sources.

  • AI is allowed and expected: your own Claude, OpenAI or Gemini key, or the free in-browser model.
  • You are graded on how you drive and check the AI, not on how much you prompt.
  • Run the checks; read what fails.
  • A new requirement will arrive partway through.
  • Too little AI use is reported as “not enough to grade”, never as a fail.
You will answer 7 debrief questions at the end
  • What did you delegate to the AI, and why?
  • Name one AI suggestion you rejected or changed, and why.
  • Choose one block the AI helped write. Explain what each line does and which test verifies it.
  • What is still risky, and what would you test or ship next?
  • Where could forged header text have reached the model in the assistant's first version, and how did you test that it no longer does?
  • Why is escaping header lines not a complete defence against instructions hidden in a PDF, and what would you add?
  • How would you choose max_chars for a given model, and what would you log to see when evidence is being dropped?