Inside fight to put AI in drug release decisions - Where regulators trust audit trail, not accuracy

Ask a pharmaceutical regulator about an AI system and the first question is whether you can reconstruct what it did. Ask an engineering team about the same system and the first question is almost always how accurate it is. Silky Sandhu has sat in both conversations, and she thinks the regulator is asking the better question.

Sandhu is a senior manager and data engineer at a top-5 global pharmaceutical company, where she leads the technical architecture of an AI-enabled product disposition platform. Disposition is the decision that releases a manufactured drug batch toward patients. Software that participates in it falls under 21 CFR Part 11 electronic records requirements and ALCOA+ data integrity principles, where a wrong output is not a degraded user experience but a regulatory event.

"A highly accurate model with no audit trail is unusable in GxP," she says. "A moderately accurate model with full traceability, explainability, and human override is deployable, because the human remains accountable and the system's contribution is fully documented and reconstructable." Her shorthand is blunter: the audit trail is the product, not the prediction.

What the platform replaced makes the stakes concrete. Quality assurance had required two sequential human reviewers to independently check batch documentation against standard operating procedures before a batch could be released. The system now performs the first-level review itself, using AI to extract structured data from certificates of analysis and conformance, running deterministic checks against SOP criteria, and surfacing discrepancies to a qualified reviewer. One approval layer was removed, halving the approval touchpoints per batch, while the reviewer's work shifted from compiling data to exercising judgment, the only part that ever required a person.

The architecture that made that acceptable is a dual-layer confidence model, and Sandhu argues it is the pattern that earns regulatory trust rather than a technical convenience. Deterministic automation checks validate what explicit rules can catch. AI anomaly detection surfaces what rules alone would miss. Both feed the same immutable audit trail, so every flag is traceable to its source, timestamped and attributable. The ordering matters more than it appears. "The automation checks give you confidence in the AI, not the other way around," she says. "When a regulator asks how you know the AI is working correctly, you point to the deterministic layer that independently validates the same data."

The AI never decides. That division is what let an autonomous component into a workflow where black-box recommendations are unacceptable regardless of how well they perform.

Getting there required inverting the usual build order. Before any AI code was written, Sandhu ran formal risk assessments across several dimensions: the assessment her documentation calls RARA, which classifies every system function by its impact on patient safety; a data integrity risk assessment, or DIRA, mapping risk across every hop from source system to disposition decision; and a dedicated AI risk assessment covering explainability, training data bias, drift detection, and graceful degradation when model confidence falls below threshold. By validation, every AI function already carried a documented risk level, acceptance criteria and a fallback path. "If you cannot articulate what happens when the model is wrong before you deploy it, you are not ready to deploy it," she says.

She is equally direct about a gap she thinks the industry is ignoring. AI-enabled applications introduce attack surfaces that pharmaceutical cybersecurity frameworks were not written for: model inversion, adversarial inputs, data poisoning, prompt injection. "What happens if someone feeds a manipulated certificate of analysis to your document extraction model?" she asks. "What if adversarial input causes the anomaly detector to miss a genuine discrepancy?" Her answer was a defence-in-depth architecture assessed against the NIST Cybersecurity Framework and OWASP ASVS, spanning network isolation, encrypted secrets management with automated rotation, role-based access control enforcing separation of duties so a reviewer cannot self-approve a disposition, and immutable audit logging that not even administrators can alter. Model outputs are bounded by the deterministic layer, which overrides AI recommendations that fall outside acceptable parameters.

Underneath sits ordinary engineering: containerised services on AWS ECS, Aurora PostgreSQL, data pipelines on Databricks with Unity Catalogue, a Django API, a React frontend, and automated delivery with infrastructure as code. The platform reconciles multiple enterprise systems, including SAP, quality management, laboratory and inspection systems, processing hundreds of thousands of records on schedule. Sandhu led a cross-functional team of 20 engineers to deliver it, and separately architected an automation framework for the Annual Product Quality Review, a recurring obligation under 21 CFR 211.180(e), replacing weeks of manual spreadsheet assembly per product per year. A paper proposing a risk-based validation strategy for that work under the FDA's Computer Software Assurance guidance is under submission to a peer-reviewed pharmaceutical science journal.

Her route to pharmaceutical manufacturing ran through precision agriculture data platforms at Bayer Crop Science and large-scale data engineering at Amazon, each with a heavier consequence for being wrong than the last.

The hardest part, she says, was none of the technology. "It was aligning quality assurance, regulatory affairs, IT validation, and engineering teams around a shared understanding of what the system does and does not decide." Technical teams consistently underestimate that. “The technology is the straightforward part; the validation strategy and stakeholder alignment take three times longer and determine whether the system is adopted or shelved.” Which returns to the sentence that organizes the whole design. The system highlights, the human decides, and the audit trail captures both.