Writing the Code Isn’t the Hard Part 

August 18, 2026

Notes From a Hands-On Agentic Coding Workshop at PHUSE US Connect 2026

By: Mike Stackhouse, CIO at Atorus, and Michael Rimler, Open Source Technologies Director at PHUSE 

The task string in the workshop repository: “Write me code to create a demographics table.” That’s the whole prompt. Change it to the task that you would like to perform, like adverse events, lab overview, or survival analysis, and the agent will create it in seconds, formatted, commented, with the library calls at the top, just like an output delivered by a programmer.  

And then the room goes quiet, because someone must decide whether it is right and sign it off. That deciding is the part that has not moved. Everything around it has, and that is the subject of AI in Clinical Data Analysis, the hands-on workshop Mike Stackhouse and Michael Rimler ran in the inaugural Open-Source Pavilion at 2026 PHUSE US Connect, and again at R/Pharma GenAI Day.  

Here’s a recap.  

Why the Fastest Part of the Job Stopped Being the Bottleneck

The workshop opened with a problem statement and five questions: 

1. Is the code good?  
2. Will it work?  
3. Is it written well?  
4. What packages does it use?  
5. Can it be improved? 

But every one of those is a review question, not a generation question. 

Writing is what got faster in recent years, not the deciding. And the deciding was always the expensive part. Anyone who has spent a morning producing a table and an afternoon defending it already knows that generation is effectively instant, and verification costs exactly what it always cost. 

Which means the math runs the wrong way. Review effort scales with the volume of what gets produced. If you make production ten times faster, you haven’t shortened the queue. In fact, you’ve lengthened it, and handed the reviewer material they did not write and have no mental model of. That is a harder review than the one they were doing before, not an easier one. 

And it’s harder in a specific way that the workshop is blunt about: A model will not say you “I don’t know.” It will guess. The guess may be wrong, and protecting against that is your job, not the model’s. Making failure not obvious nonsense, but code that is almost right. 

And almost right is the most expensive output in regulated work. Obviously wrong gets caught, but almost right gets merged, QC’d against an equally plausible second version, and surfaces eighteen months later in a question from a reviewer. 

The constraint moved. Most of the industry conversation is still optimising the part that was never the constraint. 

What Did the Workshop Teach?

Not prompting. Which is why the repository matters more than the slides. 

Open exercise_1.R and the first thing you find is two separate chat objects—a production agent and a QC agent—deliberately kept apart so they can’t share a history.¹ This separating isn’t for elegance, because the second agent is not rebuilding the output. It’s reviewing the code by reading what the first agent wrote and checking it against a defined set of quality criteria.  

The workshop showed how two agents interact, with one holding the other to a standard. 

That distinction is worth being precise about, because it is tempting to reach for the wrong analogy. Double programming, as our industry practises it, verifies a result by producing it twice and comparing. Code review verifies the method. In this workshop is about the second, and it’s not a substitute for the first. It tells you exactly where an agent fits today: in the layer where we already review each other’s code, not in the layer where we independently reconstruct the number. 

Look at what surrounds those two agents and the resemblance becomes uncomfortable. The prompts are not long essays. Instead, they’re decomposed into components with names that could have come out of a departmental SOP:  

• Purpose 
• Approved packages 
• Style rules 
• Formatting rules 
• QC purpose 
• Quality criteria

The quality criteria file runs to seven sections and reads exactly like a programming standards document, because that is what it is.  

A helper function walked the ADaM data in the environment and handed the model real column names, classes, labels (metadata, not guesses), and the shape of the data. That distinction matters because the common assumption is that you hand a model the dataset and it works out the rest. That is not how these systems work, and it is not what happened here. What the agent received was a description: This dataset has USUBJID as a character, AGE as a numeric labelled Age, TRT01P as the planned treatment. It got enough to write code that will run against the real thing.  

The effect is that the model is writing against your actual specification. Ask it to produce a table without that context and it will still produce one—confidently, against invented variable names, and in a random structure. It will look right, but it will fail on the first line that touches your data. Give it the metadata and the guessing stops, because there is nothing left to guess about. 

The generated code is extracted, executed, and when it fails, the error and traceback go back to the agent to try again. None of that is prompt engineering in the sense the term is usually abused. That is a controlled environment, defined acceptance criteria, independent review, and evidence of execution.  

Models have no memory, so every interaction must carry its own context: too little and the answer is thin, too much and you lose the target. And models can only take the actions you build tools for. They can’t run code, read a file, or search anything, unless you hand them that capability deliberately. 

The big takeaway is that you can get a very long way, very quickly. Ninety minutes with you, an agent that writes a table, a second agent that reviews the code, and a loop that retries when it fails. That’s genuinely useful.  

A production system is a different story. Everything the workshop did by hand became something that has to be engineered:  

• Where the prompt components live and who is allowed to change them 
• How a change to the quality criteria gets versioned and approved 
• What happens on the third failed retry 
• Who reviews the reviewer 
• How any of it is evidenced eighteen months later when someone asks why this output looks like this

All of it is work that cannot be skipped. 

The gap between the two is the gap between a prototype and a system, and it’s not narrowed just because the prototype got easier to build. If anything, its widened—a demo that comes together in an afternoon makes the engineering behind it look optional, which is the most expensive misreading available right now.

Regulatory Ground is Already Firmer Than People Assume

Most people think that regulation is what’s slowing pharma’s AI adoption down. If you look at the FDA’s January 2025 draft guidance on AI supporting regulatory decision-making, it does not ask whether a model is good.² It asks whether a model is credible for what, setting out a risk-based credibility assessment framework anchored to a defined context of use, where model risk is a function of how much influence the model has and how consequential the decision is.  

Establish the plan, execute it, document the evidence, then judge adequacy against that specific use. That is a design brief. 

And in January 2026, FDA and EMA jointly published guiding principles for AI in drug development that state the point our industry keeps circling: AI does not substitute for human accountability, and the provenance of data, the processing steps, and the analytical decisions should remain traceable and verifiable in line with GxP expectations.³ 

Traceability. Independent review. Defined acceptance criteria. Evidence that the thing ran. We have treated these as the tax we all pay for working in a regulated industry. In an agentic world they turn out to be the scaffolding that lets you delegate work to a system that has no memory, no judgment and no capacity to say, “I’m not sure”. Software engineering is currently discovering, expensively, that generation without verification does not scale. The pharma industry already knows that, and that is a leverage. 

The Word Doing All the Work

One line repeated across the workshop slides: Reduce the human in the loop – confidently. Read it again with the emphasis in the right place. Not remove. Reduce. And not reduce and hope. Reduce confidently, which means you must have built something that earns the confidence before you are entitled to reduce anything. That single adverb is the difference between an agent that saves your team a fortnight and an agent that costs you a finding. 

The workshop repository is available and sits at github.com/mstackhouse/phuse_ai_workshop. Clone it, run it against pharmaverseadam, break the quality criteria on purpose, and watch what the QC agent stops catching. That last exercise will teach you more than any slide. 

We know how to review work we did not do ourselves. We’ve been doing it in the industry for years, on paper and with signatures. AI can write our code, and we need to write down what good looks like precisely enough that something other than a person can be held to it.  

This hands-on agentic coding workshop at PHUSE US Connect showed where the real work in clinical programming now sits, and why traceability, independent review, and defined acceptance criteria are the operating manual for AI.

Sources

¹ Workshop repository — https://github.com/mstackhouse/phuse_ai_workshop 

² FDA, Considerations for the Use of Artificial Intelligence to Support Regulatory Decision-Making for Drug and Biological Products (draft guidance, January 2025) — https://www.fda.gov/regulatory-information/search-fda-guidance-documents/considerations-use-artificial-intelligence-support-regulatory-decision-making-drug-and-biological 

³ FDA/EMA, Guiding Principles of Good AI Practice in Drug Development, January 2026 — https://www.fda.gov/about-fda/center-drug-evaluation-and-research-cder/artificial-intelligence-drug-development