The Time, Effort, and Ownership Open-Source Validation Takes After Day One
By: Ashley Tarasiewicz, Director, Head of OpenVal® and Aga Rasinska, Director of Strategy
In 2015 the FDA said clearly: “FDA does not require use of any specific software for statistical analyses.”1 But now, eleven years later, clinical teams still can’t use R for the work that matters.
The evidentiary gap persists because SAS® carries something R doesn’t: a single vendor with a single validation story and one document to hand the auditor. R has 24,487 packages on CRAN (as of August 7, 2026), thousands of maintainers, and no one to sign the bottom of the page.2 That is a proof challenge, and it’s one of the reasons so many R adoption programs stall at the same point: not at “can we do this in R,” but at “can we show our work.”
The Language Question Is Settled
The R Consortium’s Submissions Working Group has taken five R-based pilots so far:3
- Pilot 1 delivered tables and figures
- Pilot 2 added a Shiny component
- Pilot 3 generated the ADaMs in R and tested whether FDA reviewers could rebuild the analysis environment themselves using renv and retrieve both proprietary and open-source packages
- Pilot 4 explored containers and WebAssembly
- Pilot 5 moved to Dataset-JSON, and after the Agency asked for rework on how intermediate datasets were handled, was resubmitted in January 2026
R can produce a submission, but is your organization able to produce the evidence that R did what you say it did, on the day you say it did it, in the version you say you used, and again in three years when someone in the regulatory office asks? Ask any team that has tried this and you most likely get the same answers.
The First Two Problems to Solve
The first problem is package quality. Programmers reach for open-source packages, and open-source packages vary. The R Validation Hub sets out four criteria for assessing contributed packages:4
- Purpose
- Maintenance practice
- Community usage
- Testing
The requirements are sound, but as someone who has run the assessment across a candidate library of package knows, it’s just the beginning of the work rather than the end of it.
The criteria filter for what is visible from the outside, yet many of the issues that ultimately disqualify a package live deep in the system-level dependencies the package stands on, or in behavior you only surface once you are far into its internals and documentation.
The second is reproducibility. Hand your code to a colleague, a partner CRO, or a reviewer, and without a locked environment, they can get different numbers. Not necessarily wrong numbers, but different ones — and that is a problem.
Version-locking package versions, dependency versions, and environment dependencies are what turns “it ran on my machine” into something a QA function can stand behind.
The Problem Nobody Budgets For
Open-source moves. That sounds obvious until you see what it looks like on a release schedule.
The September 2026 OpenVal release is planned to add 97 new packages, update 280 packages to new versions, and update another 100+ dependency packages.5 With the March 2026 release, that means 354 version changes in one six-month cycle. Every one of those is a potential change to an output. Most are fine, but some are not. You don’t know which without thorough testing.
Now the other direction. Since 2023, more than a dozen packages have been removed from OpenVal, and the reasons are worth naming because they are exactly the failure modes a one-time validation and low experience cannot catch:
- ggimage was removed because it fetched images from remote servers and the unstable links broke tests
- Rmpi depended on MPI libraries sitting below R that behave differently from one platform to the next, so tests hung in some environments and passed in others
- summarytools produced inconsistently formatted output depending on how it was rendered, which destabilized snapshot tests
- qs relied on deprecated internal R serialization APIs its authors had no plans to update
- tern.rbmi was archived from CRAN
- assertive and assertive.types sat on a web of dependencies that included archived packages
- norm2 had open issues its author had stopped acknowledging
None of that shows up in a risk assessment run once, in year one, on a library you validated and then stopped watching. And it runs both ways. summarytools is back in OpenVal, after upstream changes fixed the rendering problems that had destabilized its tests.
Removal isn’t a verdict, it’s a status, and every package in OpenVal gets looked at again.
Validation is not a one-time project. It’s a standing obligation against a moving object. Most organizations budget for the project, and very few budget for the obligation. And failure isn’t dramatic — it just drifts until the day someone asks a question you can no longer answer with your documentation.
One Base, Two Worlds
The instinct in a regulated setting is to build two environments: one locked for GxP, one open for exploration. However, two sets of assumptions, two sets of libraries, and most importantly, the translation step when exploratory work becomes submission work, are costly.
OpenVal works in both settings. In your GxP environment, it’s set up so you can’t install a package or change a version, which is what keeps the environment validated. In an exploratory environment, OpenVal serves as the base. You get that same reproducible, validated foundation, with the flexibility to layer additional packages on top when you need them. So, for early-stage or experimental work, you get the best of both: the stability and traceability of a version-locked core, plus room to bring in extra packages as your work requires.
Build or Buy?
The question is whether solving package validation yourself is the best approach. The test of any validation layer, built or bought, is unforgiving:
- Can you produce documented evidence, per package, per version, that does what its author claims?
- Can you reproduce an environment from three years ago?
- And when 354 things change in six months, does someone own that? By name, on schedule?
If the answer is “we’ll get to it,” you don’t have a validated environment. You have — at most — a validated snapshot of one.
Where OpenVal Sits, From Raw Data to TFLs to Interactive Apps
Every organization has to solve this same package-validation problem. Over the past several years, we’ve partnered with many pharma organizations on everything from which packages to include to the validation process itself, making OpenVal the trusted validation layer on top of R.
It’s the answer to the challenges mentioned above. It ships a version-locked library of packages with an OpenVal Validation Report documenting how each was validated. Open a session and the packages are already there: the base tooling that takes you from raw data to TFLs, the packages for advanced methods, and the packages for building interactive applications, with the evidence to back them. And because a validated package library is only half the picture, OpenVal is also built into Ageirein™, our GxP-ready analytics platform, which together is what R at the high-trust level requires.
Where Does That Leave You?
Yes, you can probably do what you need to do in R. But, if your outputs were audited tomorrow, would you be defending a validated environment or a validated snapshot of one?
References
- FDA, Statistical Software Clarifying Statement (2015): https://www.fda.gov/media/161196/download
- CRAN, Contributed Packages (package count): https://cran.r-project.org/web/packages/
- R Consortium, R Submissions Working Group – Pilots 1–5 and 2026 plans: https://r-consortium.org/posts/submissions-wg-2026/
- Validation Hub. A Risk-Based Approach for Assessing R Package Accuracy Within a Validated Infrastructure (2020): https://pharmar.org/white-paper/
- Atorus Research, OpenVal® Release Notes (all figures on packages, versions and removals): https://www.atorusresearch.com/openval/openval-release-notes/
Aga Rasinska
Director of Strategy, Atorus
With more than a decade of hands-on experience in bioinformatics, data science, and program leadership, Aga Rasinska brings a dual perspective that bridges business strategy and technical innovation, transforming complex clinical and omics data challenges into scalable, results-driven solutions. Her expertise spans project and product management, change leadership, and data-driven decision-making within regulated life sciences environments. She is also a frequent industry speaker, panelist, and contributor focused on the intersection of science, data, and business strategy.
Ashley Tarasiewicz
Director, Head of OpenVal®, Atorus
Ashley Tarasiewicz is an accomplished Director of Analytics Engineering with over a decade of experience as a SAS® Statistical Programmer and project lead in clinical trials. Previously, as Head of Atorus Academy, she was instrumental in developing and delivering comprehensive training on SAS® to R, including creating instructional videos, quizzes, and leading coaching sessions.
In her current role as Head of OpenVal®, Ashley applies her deep expertise in validation and clinical programming to oversee product development and processes. She emphasizes the strategic implementation of open-source solutions, driving innovation and efficiency in clinical programming.