Split Builder
Build a repeatable train, validation and test splitter that keeps each patient in one split.
Task
A model is about to be trained on patient visits. Each row is one visit with a patient and a 0 or 1 label. Build the tool that splits the rows into train, validation and test. Every visit of a patient must land in the same split. Label rates should be similar across the splits. The same seed must give the same split, even when the rows arrive in another order. The file is a stub and the full spec is in docs/spec.md.
Requirements
- split(rows, config) returns {splits, groups, report}. A row is {id, group, label}: id and group are nonempty strings and label is 0 or 1. config is {seed, fractions, tolerance}. Do not change the arguments.
- Every row of a group lands in the same split. splits maps train, validation and test to their row ids, sorted. groups maps each split to its group ids, sorted.
- The result depends only on the rows and the config, never on the order of the rows. Order the groups and deal them out exactly as docs/spec.md says, so the same seed always gives the same split.
- Groups with a positive row and groups without one are dealt out separately, each by the whole-percent fractions, counted in rows.
- report has rows per split, positive_rate per split rounded to 6 decimals (null for an empty split), rate_gap, and balanced, which is true when rate_gap is at most tolerance (default 0.1).
- Raise ValueError naming the problem: rows (empty or not a list), duplicate (a repeated id), group, label, seed (not an integer), fractions (not whole percents that sum to 100), tolerance (not from 0 to 1).
How you are graded
- Decomposition 15%
- Prompting the assistant 15%
- Verification 20%
- Judgment about AI output 15%
- Correctness 25%
- Communication 10%
Scores use evidence from your session. A short or incomplete session may not provide enough evidence for every skill. See a sample scorecard
Interview guidance
We need a tool that splits patient visits into train, validation and test. Visits from one patient must stay together, label rates should be similar across the splits, and the same seed must give the same split. The file is a stub and the spec is in the docs folder. Use the AI assistant however you like. Before you ask it for code, tell me how you will know the split has no leak.
- AI is allowed and expected: your own Claude, OpenAI or Gemini key, or the free in-browser model.
- You are graded on how you drive and check the AI, not on how much you prompt.
- Run the checks; read what fails.
- A new requirement will arrive partway through.
- Too little AI use is reported as “not enough to grade”, never as a fail.
You will answer 7 debrief questions at the end
- What did you delegate to the AI, and why?
- Name one AI suggestion you rejected or changed, and why.
- Choose one block the AI helped write. Explain what each line does and which test verifies it.
- What is still risky, and what would you test or ship next?
- How did you convince yourself that no patient is in two splits? What did you check beyond the expected values?
- What in your first version depended on the order of the rows, and how did you find it?
- The label rates are close but not equal. When would you tighten the tolerance, and what would you give up?