CodeBaobab
Data & Systems45 minutes

Conversation Export Cleaner

Clean a support conversation export.

Python

Task

A support export contains repeated conversations, mixed timestamp formats, and invalid dates. The current pipeline splits rows before noticing duplicates. Add deduplication so the same normalized conversation cannot appear in both datasets, and make every discarded row accountable in the report.

Requirements

  1. Implement clean_export(rows, options) with the exact output shape in docs/contract.md. Do not mutate rows or options.
  2. Build duplicate keys from NFKC-normalized, case-folded text with collapsed whitespace. Keep the retained row's original text and extra fields.
  3. Keep the latest valid row by parsed timestamp. On equal instants, keep the smaller id. Report invalid timestamp ids instead of silently dropping them.
  4. Deduplicate before splitting. Use the documented SHA-256 bucket rule and eval_percent, then sort each dataset by id.
  5. Report input rows, kept unique rows, discarded valid duplicates, invalid timestamps, and masking count. These categories must reconcile.
How you are graded
  • Decomposition 15%
  • Prompting the assistant 15%
  • Verification 20%
  • Judgment about AI output 15%
  • Correctness 25%
  • Communication 10%

Scores use evidence from your session. A short or incomplete session may not provide enough evidence for every skill. See a sample scorecard

Interview guidance

I will ask how you know the train and evaluation sets do not overlap. Use the AI assistant, but inspect the raw rows and test its normalization and date handling.

  • AI is allowed and expected: your own Claude, OpenAI or Gemini key, or the free in-browser model.
  • You are graded on how you drive and check the AI, not on how much you prompt.
  • Run the checks; read what fails.
  • A new requirement will arrive partway through.
  • Too little AI use is reported as “not enough to grade”, never as a fail.
You will answer 7 debrief questions at the end
  • What did you delegate to the AI, and why?
  • Name one AI suggestion you rejected or changed, and why.
  • Choose one block the AI helped write. Explain what each line does and which test verifies it.
  • What is still risky, and what would you test or ship next?
  • What check proves a conversation cannot appear in both datasets?
  • Why can string sorting select the wrong latest row?
  • Which normalization changes the duplicate key, and which original fields must stay unchanged?