The Loop That Never Ends
Find why an agent made the same tool call 400 times and fix the loop.
Task
An order desk agent answers questions by calling tools in a loop. Run 4821 made 400 calls to lookup_order with the same arguments before ops killed it. The log is incomplete: it shows the tool calls but not their results. A scripted model stands in for the real one, so the run can be replayed exactly. Find the cause and fix the loop.
Requirements
- run_agent(question, script, world, config) returns {status, answer, reason, steps, history}. status is "answered" (answer is the text, reason is null) or "stopped" (answer is null, reason says why). steps is the number of times the model was asked. Do not change the arguments.
- history is the list of messages the model read, in order: the user question, then for each tool call an assistant message {role, call_id, tool, args} followed by exactly one tool message with the same call_id, then the final assistant answer {role, content}.
- A tool message holds result when the tool returned, or error (the message text) when it could not do what was asked. A tool name that does not exist is an error with the text "unknown tool: <name>". The run goes on after an error.
- config.max_steps is the most times the model may be asked: an integer from 1 to 50, default 8 (ValueError containing max_steps otherwise). A run that has not answered after that many turns returns stopped with reason "max_steps". An answer on the last allowed turn still counts.
- question is a nonblank string (ValueError containing question). script is a nonempty list of steps (ValueError containing script). The world is read-only.
How you are graded
- Decomposition 15%
- Prompting the assistant 10%
- Verification 20%
- Judgment about AI output 20%
- Correctness 25%
- Communication 10%
Scores use evidence from your session. A short or incomplete session may not provide enough evidence for every skill. See a sample scorecard
Interview guidance
An agent run made 400 calls to lookup_order with the same arguments before ops killed it. The log shows the tool calls but not their results. A scripted model stands in for the real one, so you can replay the run exactly. Find the cause and fix it, so the run ends the way it should have. Use the AI assistant however you like. Before you change any code, replay the run and tell me what you think the model was shown.
- AI is allowed and expected: your own Claude, OpenAI or Gemini key, or the free in-browser model.
- You are graded on how you drive and check the AI, not on how much you prompt.
- Run the checks; read what fails.
- A new requirement will arrive partway through.
- Too little AI use is reported as “not enough to grade”, never as a fail.
You will answer 8 debrief questions at the end
- What did you delegate to the AI, and why?
- Name one AI suggestion you rejected or changed, and why.
- Choose one block the AI helped write. Explain what each line does and which test verifies it.
- What is still risky, and what would you test or ship next?
- What did the model see at its second turn in run 4821, and how did you confirm that?
- The run was configured with a step limit. Why did the limit not stop it?
- Which of your changes fixes the cause, and which only limits the damage? Would you ship either one alone?
- What would you add to the log so the next incident like this is faster to read?