The report from one 40-minute Griot round, Video Analytics: a candidate fixes a daily view report with Claude, then handles a follow-up about autoplay previews.
Griot scorecard · Verdict
Strong Hire
Medium confidence
Round
Video Analytics
Category
Product Features
Date
Mode
Interview
Grader
Graded by Griot · Claude Sonnet 5.5 with your key
Strong No Hire
Lean No Hire
Lean Hire
Strong Hire
You treated the assistant as a fast pair, not an oracle. You named the riskiest requirement before prompting, asked for a plan with no code, built in small steps and ran the checks after every change. The two moments that matter most both went your way: you put deduplication back in front of the date filter when the assistant moved it, and you rejected a follow-up change that would have let a retried autoplay preview count as a view. Confidence is medium rather than high because one debugging prompt carried no hypothesis and you applied a 23-line follow-up diff quickly enough to miss the null case.
Strengths
Wrote down the riskiest requirement (dedupe before date filter) before the first prompt.
Caught and reversed the assistant moving the date filter ahead of deduplication.
Rejected the preview guard that skipped short plays before claiming their id, and said why.
Ran the checks after every applied or edited diff: 7 runs, first at 5:05.
Gaps
At 14:12 you pasted a failing check name with no guess about the cause.
The 24:20 diff was applied about a minute after it arrived; reading it would have caught is not None.
The debrief says you tried the offset case "after every change"; the transcript shows it once.
Skills this round measured
Problem SolvingStrong
AI FluencySolid
Verification & TestingStrong
Code Quality & CorrectnessStrong
Technical CommunicationSolid
Session details
Signals
Measured from the event log, no model involved. Assistant: Claude Sonnet 5.5.
Time used
38:12
Prompts
81 plan · 3 ask · 4 agent
Plan first
Yes
Check runs
7first at 5:05
Checked after applying
100%baseline run before first apply
AI diffs
42 applied · 1 edited · 1 rejected
Largest diff
28 lines
Hand edits
3
Follow-up replan
1 min 15 s
Final checks
7/7follow-up 9/9
Six dimensions
Each score lands on an anchored level from the round’s rubric. Evidence links to the exact moment in the replay.
Decomposition
4/4
1 Needs work
2 Developing
3 Solid
4 Strong
LandedSplit the work into dedupe, UTC date, aggregation, and ordering steps, and asked for or wrote them in that order.
Rubric anchors
4 · Strong
Split the work into dedupe, UTC date, aggregation, and ordering steps, and asked for or wrote them in that order.
1 · Needs work
Asked for the whole function in one prompt and treated the result as a unit.
Why this score
You asked for a stepwise plan with no code, then requested steps 1–3 and 4–6 as separate changes, each followed by a check run.
Strength
Draft a plan as separate steps — dedupe, UTC date, date filter, aggregate, sort — and tell me which step is easiest to get wrong. No code yet.
A plan first, in the order the requirements depend on each other.
Strength
Implement steps 1–3 only … Leave aggregation as a stub.
Kept the first change small enough to read and verify.
Sub-signals 1 of 1 observed
✓
Codebase orientation before prompting (observed)
Read the README and checks for two minutes and wrote the riskiest requirement down before the first prompt.
Prompting the assistant
3/4
1 Needs work
2 Developing
3 Solid
4 Strong
LandedMost prompts quoted the exact requirement at risk; one debugging prompt only named the failing check.
Rubric anchors
4 · Strong
Prompts quoted the exact requirement at risk and pasted failing check output.
1 · Needs work
Prompts said "fix the tests" or "make it work" without naming the requirement or the failure.
Why this score
Your build prompts named the requirement at risk ("keep the first occurrence … across the WHOLE input"). The 14:12 prompt said only "fix it", and the follow-up prompt pasted the requirement without stating the ordering you had already written down.
Strength
keep the first occurrence of each event id across the WHOLE input before any date filtering
Quoted the requirement most likely to go wrong.
Concern
Distinct viewers and stable ties is failing. Fix it.
No hypothesis and no check output; the assistant had to guess.
Strength
My guess: is not None treats None as "absent", so None slips through as an old-client event.
The second debugging prompt led with a hypothesis — much stronger.
Sub-signals 2 of 2 observed
✓
Context quality (observed)
Every prompt carried the service file; the plan and follow-up prompts also carried the README or follow-up brief.
✓
Iterative refinement (observed)
After rejecting the preview guard you re-prompted with the exact order of operations.
Verification
4/4
1 Needs work
2 Developing
3 Solid
4 Strong
LandedRan the checks before and after each applied change, and tried an offset timestamp near midnight in the preview.
Rubric anchors
4 · Strong
Ran the checks before and after each applied change, and tried an offset timestamp near midnight in the preview.
1 · Needs work
Applied a diff and moved on without rerunning the checks, or reran them only at the end.
Why this score
You ran the checks before the first change and after every applied, edited or hand change, and tried the 23:30-05:00 case and four follow-up edge cases in the preview.
Strength
2 of 7 checks passing
A baseline run before any change, so later results mean something.
Strength
Preview: 2026-01-01T23:30:00-05:00 with range Jan 2–Jan 2 → counted on Jan 2.
Tried the near-midnight offset by hand, not only through the checks.
Strength
Tried in preview: "3000" raises, True raises, 2999.5 raises.
Probed inputs the hidden checks do not cover.
Sub-signals 2 of 2 observed
✓
Test authorship and validity of AI-written tests (observed)
Asked the assistant for uncovered inputs, then ran them yourself instead of trusting its list.
✓
Supervision and catching drift (observed)
Noticed the assistant “simplifying” the loop by filtering dates first.
Judgment about AI output
4/4
1 Needs work
2 Developing
3 Solid
4 Strong
LandedRejected or edited at least one AI change for a stated reason, such as date filtering before deduplication.
Rubric anchors
4 · Strong
Rejected or edited at least one AI change for a stated reason.
1 · Needs work
Accepted every suggestion as written, including one that reordered dedupe and date filtering.
Why this score
You edited one diff and rejected another, each for a reason grounded in the requirements, and wrote the reason down.
Strength
Edited the suggestion: moved the date filter back after first_by_id (+24 −4).
Reversed the assistant’s reorder before running anything.
Strength
the continue runs before seen.add(id), so a 500 ms preview retried at 9000 ms would be counted as a new view
The trap this round is built around, caught from the diff alone.
Concern
Applied +17 −6 seventy-five seconds after the response.
This diff contained the is not None bug the next check run found.
Sub-signals 2 of 2 observed
✓
Knowing when not to use AI (observed)
Fixed the sort key and the presence test by hand rather than asking for another diff.
✓
Cleanup of AI output (observed)
A final hand edit removed the assistant’s explanatory comments and renamed `first_by_id` to `claimed`.
Correctness
4/4
1 Needs work
2 Developing
3 Solid
4 Strong
LandedFinal code passes all public and follow-up checks without mutating input.
Rubric anchors
4 · Strong
Final code passes all public and follow-up checks without mutating input.
1 · Needs work
Final code fails a check, or passes only by special-casing check inputs.
Why this score
All 7 public and 9 follow-up checks pass, the input list is unchanged, and no check inputs are special-cased.
Strength
16 of 16 checks passing
Public and hidden follow-up checks, after the hand fix.
Communication
3/4
1 Needs work
2 Developing
3 Solid
4 Strong
LandedExplained why deduplication must happen before date filtering; the follow-up reasoning lived in notes rather than the debrief.
Rubric anchors
4 · Strong
Explained why deduplication must happen before date filtering and why a short play still claims its id.
1 · Needs work
Could not say why the order of operations matters, or described the AI output rather than the decision.
Why this score
Your notes say clearly why order matters and why a short play claims its id. The debrief overstated how often you tried the offset case.
Strength
A retry stamped on another day must not move the view.
The why, not just the what, before writing code.
Strength
claim the id as seen BEFORE deciding if it qualifies
A one-line approach 75 s after the follow-up arrived.
Sub-signals 1 of 1 observed
✓
Design and extensibility under the follow-up (observed)
Extended the existing loop in place; the order of operations survived the change.
Habits to watch
Patterns interviewers notice in AI-assisted rounds. They are named so you can spot them, not counted against you twice.
!Pasting an error without a hypothesisSeen once
Once, you handed the assistant a failing check name and asked it to fix it. It guessed right this time, but a hypothesis ("I think the sort key ignores video_id") makes the answer checkable and shows the interviewer your reasoning. Your later prompt at 26:02 is the model to follow.
Concern
Distinct viewers and stable ties is failing. Fix it.
No guess at the cause and no check output.
Not seenHands-offAI showcaseFeature marathonOne huge multi-file changeCode you can’t explain
Debrief
Your answers alongside the session evidence. One claim is not supported by what happened.
What did you delegate to the assistant, and why?
The mechanical parts: the set-based dedupe, the aggregation dict and the sort. I kept the order of operations and the follow-up ordering to myself because that is where the task is subtle.
Which AI suggestion did you change or reject, and what in the requirements or the check output told you to?
I rejected the if event.get("watched_ms", 3000) < 3000: continue guard: it skipped short plays before claiming their id, which breaks "a short play still claims its event id". I also edited the diff that filtered dates first. I tried the offset case after every change to be sure.
Not supported“I tried the offset case after every change” — the transcript shows one offset trial, at 16:40.
Why must a short play still claim its event id when it does not count as a view?
Otherwise a client retrying the same event with a longer watched_ms creates a second, qualifying copy, and a preview turns into a view.
If the report had to include events that arrive a day late, what would you change and how would you test it?
Keep a claimed-id store across days and recompute affected days on late arrival. Test with a retry that arrives after its day was reported.
Replay
Every prompt, reply, diff and check run on one timeline. Select a moment, or use the arrow keys, to read it in full.
Strong plan: 2 pins
Good catch: 2 pins
Verified: 2 pins
Accepted without review: 1 pin
Debugging without a hypothesis: 1 pin
Too little context: 0 pins
Plan
Ask
Agent
CoachingPromptsAI repliesDiffsChecksYoumin
Follow-up
05101520253035
NoteRiskiest requirement: keep the first occurrence of each id BEFORE filte…Strong plan
Plan promptBefore any code: here are the four requirements from the README (pasted…Strong plan
AI replyPlan: 1. Validate the range: raise ValueError if start_date > end_date.…
NotePlan: validate range → dedupe whole input (first wins) → UTC day via as…
NoteDone. Order: validate range → validate watched_ms → claim id → qualify…
· moment 1 of 38
Note
Riskiest requirement: keep the first occurrence of each id BEFORE filtering dates. A retry stamped on another day must not move the view. Second risk: offsets near midnight need a UTC date, not a local one.
Strong planNamed the riskiest requirement before opening the assistant.
EvidenceThe why, not just the what, before writing code.
You and a strong answer
A reference session for the same round, written by CodeBaobab. It is one strong way through, not the only one.
PromptsYou 8Strong 6
Check runsYou 7Strong 6
Plan firstYou YesStrong Yes
Time to first testYou 5:05Strong 2:00
Rejected suggestionsYou 1Strong 1
YouStrong answer
Plan
You
Riskiest requirement: keep the first occurrence of each id BEFORE filtering dates. A retry stamped on another day must not move the view. Second risk: offsets near midnight need a UTC date, not a local one.
Before any code: here are the four requirements from the README (pasted). Draft a plan as separate steps — dedupe, UTC date, date filter, aggregate, sort — and tell me which step is easiest to get wrong. No code yet.
Plan: validate range → dedupe whole input (first wins) → UTC day via astimezone(timezone.utc) → inclusive filter → aggregate (set of users) → return a sorted copy. Run checks after every applied change.
Strong answer
Run the checks once to see the baseline.
Raise ValueError on a reversed range before doing any work.
Claim each event id over the whole input (first occurrence wins) before looking at dates.
Derive the UTC day from the instant, never from the raw string or local time.
Filter the inclusive range, aggregate views and a set of viewers, return a sorted copy.
Key prompts
You
PlanBefore any code: here are the four requirements from the README (pasted). Draft a plan as separate steps — dedupe, UTC date, date filter, aggregate, sort — and tell me which step is easiest…
AgentImplement steps 1–3 only: raise ValueError mentioning "date" when start_date > end_date; keep the first occurrence of each event id across the WHOLE input before any date filtering; derive…
AskDistinct viewers and stable ties is failing. Fix it.
Ask"Reject a null watch time" fails. My guess: is not None treats None as "absent", so None slips through as an old-client event. Is "watched_ms" in event the right presence test, given ol…
Strong answer
PlanHere are the requirements. Which order of operations satisfies "keep the first occurrence BEFORE filtering dates", and what breaks if dedupe runs second?Makes the assistant commit to the order before any code exists.
AgentWrite only the dedupe + UTC-day helper. Show me the day for 2026-01-01T23:30:00-05:00 in a comment.A small diff with its own worked example, easy to verify.
Ask"Distinct viewers and stable ties" fails: I think the sort key ignores video_id, so ties keep input order. Confirm and suggest a key.A hypothesis plus the failure, so the answer is checkable.
AgentFollow-up: validate watched_ms, then claim the id, then qualify (>= 3000 or absent), then aggregate. Keep the existing order otherwise.States the order the trap depends on, so the assistant cannot reorder it.
Verification
You
7 check runs: 2/74/76/77/714/1616/1616/16
Preview: 2026-01-01T23:30:00-05:00 with range Jan 2–Jan 2 → counted on Jan 2. A retry of id "a" on Jan 1 and Jan 2 → excluded. Input list unchanged a…
Tried in preview: "3000" raises, True raises, 2999.5 raises. Preview on Jan 1 retried on Jan 2 → excluded. Preview-only video → absent. All as specif…
Strong answer
Baseline check run before the first change.
Checks after every applied diff.
Preview: 23:30-05:00 lands on the next UTC day.
Preview: a 500 ms preview retried at 9000 ms is not a view.
Preview: watched_ms None and "3000" both raise ValueError.
Rejected suggestion
You
Rejected: the continue runs before seen.add(id), so a 500 ms preview retried at 9000 ms would be counted as a new view — autoplay previews leak back in. Also None < 3000 raises TypeError, so None would crash the report instead of raising ValueError.
Strong answer
if event.get("watched_ms", 3000) < 3000: continue at the top of the loop. It skips the event before its id is claimed, so a longer retry counts, and None < 3000 raises TypeError instead of the required ValueError.
Follow-up
You
Approach written 1 min 15 s after it arrived: “Approach: validate watched_ms first, claim the id as seen BEFORE deciding if it qualifies, then only qualifying plays create the video entry and add the viewer.”
Strong answer
Wrote a one-line order of operations within a minute, extended the existing loop in place, and probed None, a string and a retried preview in the preview pane before the hidden checks.
Progress on this round
3 attempts. Trends show each dimension from your first attempt to your latest.
Attempt 3 · this report · Interview · Claude Sonnet 5.5
Strong Hire
Attempt 2 · Practice · Claude Sonnet 5.5
Lean Hire
Attempt 1 · Practice · In-browser model
Lean No Hire
Decomposition4/4+3
Prompting the assistant3/4+1
Verification4/4+3
Judgment about AI output4/4+2
Correctness4/4+2
Communication3/4+1
The follow-up drill starts from your passing base code. The in-browser model is weaker, so you have to steer and check it more.
Share your Griot scorecard
Off by default. Sharing saves a copy of this scorecard on CodeBaobab and gives it a public link you can post or send to a recruiter. Your attempt stays in your browser either way.