RAG Document Updates
Repair stale RAG evidence.
Task
A customer updated a device guide, but the assistant still quotes the previous revision. Follow a request through the supplied repository, reproduce the stale answer, and fix the document update path. The retrieval adapter returns a quoted passage so that every check is repeatable; you do not need an embedding model or an API key.
Requirements
- run(documents, actions) starts a fresh store with generation "initial". Process actions in order and return {active_generation, results}, one result per action. Do not change either input.
- A query action is {op: "query", text}. Return {generation, answer, sources}. Rank documents by the number of distinct shared lowercase ASCII alphanumeric words; ties use document id ascending. Return the best matching document's exact text and [{document_id, revision}]. No match returns answer null and sources [].
- A publish action is {op: "publish", generation, documents}. It replaces the whole corpus and returns {op: "publish", generation, status: "published"}. Every subsequent query must use that generation, including previously cached queries and previously missing answers. An empty corpus removes all documents.
- Document ids must be unique non-blank strings; revision is a positive integer, not a boolean; text is a non-blank string. Raise Error containing document for invalid documents. Generations are new non-blank strings and may not overwrite an existing generation; invalid values raise Error containing generation. Invalid query text raises Error containing query.
- Inspect and edit the relevant modules, not only answers/service.py. Browser checks import the edited Python repository together. Public assessment checks are fixed independently of editable test files. See docs/requests.md for examples and boundaries.
How you are graded
- Decomposition 15%
- Prompting the assistant 15%
- Verification 20%
- Judgment about AI output 15%
- Correctness 25%
- Communication 10%
Scores use evidence from your session. A short or incomplete session may not provide enough evidence for every skill. See a sample scorecard
Interview guidance
A customer updated a guide, but the assistant still answers from the old revision. Start by reproducing one stale answer. Follow it through the supplied modules, make the smallest coherent repair, and show me why the new result is trustworthy. You may work independently or use the AI assistant.
- AI is allowed and expected: your own Claude, OpenAI or Gemini key, or the free in-browser model.
- You are graded on how you drive and check the AI, not on how much you prompt.
- Run the checks; read what fails.
- A new requirement will arrive partway through.
- Too little AI use is reported as “not enough to grade”, never as a fail.
You will answer 7 debrief questions at the end
- What did you delegate to the AI, and why?
- Name one AI suggestion you rejected or changed, and why.
- Choose one block the AI helped write. Explain what each line does and which test verifies it.
- What is still risky, and what would you test or ship next?
- What caused the stale answer, and which evidence confirmed it?
- How does your fix keep a failed build invisible to readers?
- Which AI suggestion did you verify or change, or how did you verify your own implementation?