When a stats, data science, or PK group requests a new R package, the case usually sounds simple. It’s on CRAN, plenty of people use it, and the analysis needs it soon. Still, the OpenVal® team at Atorus™ turns away 10 to 15% of requests each release, based on reading code, running functions, and checking the results.
So, what does a package have to do to earn its way into OpenVal?
CRAN Is the Starting Line
CRAN confirms that a package installs and passes its own checks on CRAN’s machines. Every CRAN package in OpenVal clears that bar. The bigger questions are for the user:
- Do the tests cover what the package claims to do?
- Are the results correct?
- Will it behave the same way in your statistical computing environment?
A package can reach CRAN with very few tests, with tests set to skip during CRAN checks, or with a maintainer who stopped responding years ago. Regulators expect analysis software to be reliable, documented, and backed by testing,¹ so CRAN is where the work begins.
Three Packages That Looked Fine Until Someone Checked
The most useful lessons tend to come from packages that looked like easy approvals.
truncnorm
truncnorm has just six documented functions. During review, the team found open GitHub issues showing that etruncnorm() and vtruncnorm() return incorrect numbers.² The reports date to April 2023 and were never acknowledged, and the repository had gone more than three years without a commit. Known incorrect results and no active maintainer don’t meet the OpenVal standard.
ModelMetrics
ModelMetrics’ ce() function calculates classification error correctly when given actual and predicted values directly. In testing, passing it a model object, which the documentation lists as supported, did not apply the classification threshold, so predictions were counted as errors. Limited documentation, no public issue tracker, and no reply to the team’s outreach left no way to resolve the issue with the maintainers, so the package was not included in OpenVal.
nlmixr2extra
This one is easy to miss because it belongs to the well-known, actively maintained nlmixr2 family. Validation turned up bugs in the bootstrap feature the package advertises, as well as in its optimization function. The team reported both on GitHub. With only two references, neither of which was detailed enough to reproduce results, the team concluded it “cannot be validated for OpenVal at its current version.”
Problems That Show Up When You Dig
Those are individual cases, but across hundreds of reviews the same few patterns keep turning up:
- Outdated bundled libraries. Some packages ship JavaScript libraries with known security issues; one customer’s scanner traced an outdated Bootstrap front-end library to copies inside Shiny and bslib
- Inherited problems. A package can look fine while pulling in dependencies that have issues of their own
- Environment-dependent behavior. Results can change with the operating system, R version, or environment setup
- Documentation that doesn’t match behavior. A function can do something different from what its documentation describes
None of these show up in a download count or a risk score, and fixing most of them depends on the people who maintain the package.
Working With Package Authors
Most open-source R packages are maintained by one or two people in their spare time, so responses to bug reports vary. Some authors start on a fix within a day or two. Others want to help but do not have time. Some never answer. Those who reply are almost always glad to get a detailed report with a reproducible example.
Meanwhile, OpenVal customers keep moving. The team reports the problem upstream, documents it, and decides whether to hold the package, document a workaround, or recommend an alternative.
A recent case shows how far that collaboration can go. The OpenVal team, a pharma customer, and the package’s author worked together on a pull request to fix the package. The customer’s team reviewed the change, and the author updated the package. The package is now slated for the next OpenVal release.
A Declined Package Can Still Make It In
summarytools, a popular package for quick descriptive statistics and frequency tables, was a frequent request. When the team first reviewed it, the package had no unit tests of its own, and its output formatting shifted depending on how it was rendered (RStudio’s Knit button, rmarkdown::render(), or inside testthat). That made snapshot tests unstable, so the team held summarytools out of OpenVal and pointed customers to similar packages already in OpenVal.
The requests didn’t stop, and several customers kept asking for it. Over time, as the package matured, the team took another look. summarytools passed full validation and was added to OpenVal 2026.09.00.
What Package Validation Should Really Mean
Validated should mean one thing: The package gives correct results for the way you’ll actually use it, in the environment you’ll actually run it in.
Much of what gets called validated today is an automated risk scan of downloads, unit tests, release history, and maintainer count. Those metrics are a useful starting point for GxP validation,³ and OpenVal uses them too. But a package can still score well on all of them and return the wrong answer.
In one review, two of a package’s three main model-fitting functions had no unit tests actually testing their results. The existing tests confirmed only that the code ran. OpenVal added testing to cover those functions, but without a thorough look at the existing unit tests, the gap would have been missed.
Ensuring correctness is slow and specialized work. That’s why the most thorough review goes to the packages behind your key results. A plotting helper and the package producing a submission’s efficacy analysis come with very different risks. It’s also why in-house efforts tend to stall. Reviewing one package is doable. Maintaining an auditable process, reproducible infrastructure, and re-testing every quarter competes with the analysis work that teams were hired to do.
OpenVal gives your team a validated R environment built on that kind of review, ready for submission work from day one.
Frequently Asked Questions
Any CRAN package can go through validation, but not every package will pass. CRAN confirms that a package installs and passes its own checks. Validation goes further, confirming that tests cover the package’s claims and that results are correct in your environment. Where existing tests fall short, OpenVal adds testing to ensure that the package’s outputs can be reproduced and match what they should be. When mismatches occur, OpenVal documents them and brings them to the customer and package maintainers. Packages whose results can’t be confirmed can’t be validated.
The OpenVal team turns away about 10 to 15% of requests each release, usually for incorrect results, weak testing or documentation, inactive maintainers, or risky dependencies.
Requests are weighed by demand. Each package then gets a risk assessment covering maintainers, testing, documentation, download frequency, and use cases, followed by programmatic and human review.
References
1 U.S. Food and Drug Administration. (2015). Statistical Software Clarifying Statement. https://www.fda.gov/media/161196/download
2 Cascone, A.D. (2026). {truncnorm} import removal? GitHub issue #85, gertvv/gemtc. https://github.com/gertvv/gemtc/issues/85
3 Nicholls, A., Bargo, P.R., and Sims, J. (2020). A Risk-Based Approach for Assessing R Package Accuracy Within a Validated Infrastructure. R Validation Hub, R Consortium. https://pharmar.org/white-paper/

Ashley Tarasiewicz
Director, Head of OpenVal, Atorus
Ashley Tarasiewicz is an accomplished director of analytics engineering with over a decade of experience as a SAS® statistical programmer and project lead in clinical trials. Previously, as Head of Atorus Academy, she was instrumental in developing and delivering comprehensive training on SAS® to R, including creating instructional videos, quizzes, and leading coaching sessions.
In her current role as Head of OpenVal, Ashley applies her deep expertise in validation and clinical programming to oversee product development and processes. She emphasizes the strategic implementation of open-source solutions, driving innovation and efficiency in clinical programming.

Morgan Narain
Data Solutions Engineer, Atorus
Morgan Narain is a Data Solutions Engineer on the OpenVal® team at Atorus, testing and validating the R packages that go into each OpenVal release. That work spans reviewing package code and documentation, writing and QCing validation tests, and preparing package risk assessments.
Morgan holds a PhD in Toxicology from University of North Carolina at Chapel Hill and a Bachelor of Science in Biology, cum laude, from Oakwood University. As an undergraduate she did biomedical research, including work with the University of North Carolina at Chapel Hill.