Starting Tuesday, I am spending eight weeks building one thing: a verifiable clinical AI environment. This post is the plan, written down in public before day one, so the next eight Fridays have something to answer to.
Table of contents
Open Table of contents
The bottleneck moved
The interesting work in LLM post-training right now runs on verifiable rewards. Models improve fastest on tasks where a program, not a person, can check the answer: code that passes tests, math that resolves to a number. The scarce input is no longer the algorithm. It is the environment: a task, ground truth, and a scoring function tight enough to train against.
Healthcare is where that scarcity bites hardest. Clinical AI systems are shipping into hospitals at real scale, and the question every one of them faces is the same: how do you know the output is right? Most clinical tasks resist automatic checking. But some do not. A clinical note has a correct set of ICD-10 diagnosis codes. A note’s facts can be extracted into FHIR resources that either validate against the schema or do not. Those tasks are verifiable, and verifiable means you can build evaluations and reward signals that do not depend on vibes.
I am a data engineer. I have spent years on pipelines where correctness is checked by machines: schema validation, data quality gates, reconciliation against ground truth. My read is that the bottleneck in clinical AI is not model capability. It is the evaluation and reward infrastructure. That is a data problem wearing an ML costume, and it is the one part of this field where my day job is the head start rather than the gap.
One artifact, three jobs
The center of the build is a single clinical extraction environment: clinical note in, structured output out, scored by real verifiers. Concretely, two task shapes on the same spine: note to ICD-10 code set, and note to FHIR resource bundle.
The scoring is where the design earns its keep. A code-set prediction is graded with micro, macro, and hierarchical F1, where the hierarchical variant gives partial credit when the three-character category is right even if the full code is wrong. That partial credit is not generosity. It is reward density: an RL policy learning against exact match alone sees a flat zero for near misses and gets no gradient to climb. A FHIR prediction passes through a schema-validity gate first, then resource-type F1, then fuzzy field accuracy.
Built once, the same artifact does three jobs:
- An evaluation suite. Versioned, reproducible, run in CI on every model change, with calibration checks (does confidence match accuracy, which in a clinical setting is not optional) and perturbation robustness.
- An RL reward signal. The eval metric becomes the reward for GRPO fine-tuning. Same code path, so the number I train against is the number I report.
- A governance instrument. A documented, auditable answer to “how do you know it works,” which is the question every health system AI review board asks.
Depth over surface. One environment shipped completely before any second track widens the scope.
The build order is forced, not chosen
The sequence matters because each stage gates the next. A general instruct model almost never emits a valid, correctly formatted ICD-10 code set, which means the RL reward starts flat and no gradient flows. So supervised fine-tuning comes first, not as a warmup but as a prerequisite.
From there: GRPO with the verifiable reward is the flagship, group-relative advantage with no critic network, rollouts served through vLLM. Then PPO on the same task, where the deliverable is the comparison writeup rather than a second checkpoint: what the learned critic costs in variance and sample budget, and why the critic-free baseline holds up here. Then a Bradley-Terry reward model trained on preference pairs minted from the F1 metric itself, which asks a question I genuinely do not know the answer to: does optimizing a learned model of the reward beat optimizing the raw verifier? And last, an LLM-judge reward for faithfulness, the axis with no clean string match. The judge gets meta-evaluated against human labels, with agreement measured in Cohen’s kappa, before it is trusted to steer anything.
That last habit is the whole evals thesis in miniature: evaluate the evaluator. An unexamined judge is just vibes with extra steps.
The compliance wall is a design input
Real clinical notes come with a hard rule. My MIMIC-IV access runs under a data use agreement: local processing only, no real note text to any third-party API, no redistribution of derivatives, ever. Instead of treating that as friction, the build splits into two tracks.
Track A is public. Synthetic notes from Synthea and HAPI FHIR, paired with generated structured outputs. It lives on Hugging Face with a dataset card, anyone can run it in five minutes, and it powers the public demo and CI smoke tests.
Track B is credentialed. Real de-identified notes under the DUA, processed only on hardware I control, with open models served locally through vLLM. It produces the headline numbers and every claim of clinical credibility. The repo ships code and build instructions so other credentialed researchers can reproduce it, but the data never travels.
Every artifact states which track it runs on. That one rule keeps the public reproducibility and the real-note credibility without ever crossing the compliance line. If you have worked in healthcare data engineering, you already know this discipline. It turns out to transfer directly.
One pre-registered question
There is a research question riding along, and I am writing down the answer criteria before any training run so I cannot fool myself afterward: does continued pretraining on medical text improve ICD-10 coding F1 over task fine-tuning alone, starting from the same instruct model?
The design is a controlled ablation. Arm A: instruct model, then SFT. Arm B: identical, plus a continued-pretraining stage on medical text before SFT. Same data splits, same SFT hyperparameters, same frozen test set, same decoding and parser, at least three seeds per arm. The eval gets built and locked before any arm trains, because an eval built after seeing model outputs bends toward flattering them. If domain pretraining does not move the metric, that is not a failure. That is a finding with a dollar figure attached, and I will publish it either way.
The operating rules
Eight weeks, evenings and weekends, around a full-time job. The arc: fundamentals rebuilt from scratch, then training real models on real GPUs, then the eval harness and an open-source contribution, then an agent evaluated by its own harness with a clinician reviewing the rubric, then judge calibration and a cost-accuracy frontier, then GRPO at production scale on real notes, then packaging everything into one connected capstone.
Three rules govern all of it. One wedge, no scope creep, and a cool new course changes nothing. No learning beyond what the build requires. And every Friday at 6 PM, one public artifact ships: a post, a repo, a dataset, a demo. Not public by six means the week failed. This post is the standard the next eight get measured against.
What done looks like
By the end: a public dataset with a card, an eval harness running in CI, a judge-calibration study with measured human agreement, fine-tuned checkpoints with before-and-after numbers on both tracks, the environment published where RL researchers can train against it, a short paper writing up the method, and a post here for every piece.
Week 1 starts Tuesday: the transformer stack from scratch, autograd upward. I refuse to train what I cannot explain, so the first Friday artifact will be about backpropagation, written by someone who just rebuilt it.
The plan is the plan. See you Friday.