Artificial intelligence is moving quickly into healthcare, but responsible adoption requires more than enthusiasm for new tools. It requires governance that understands clinical risk, protects patient data, validates performance in real settings, supports clinicians rather than overwhelming them, and keeps patient outcomes at the center of technology decisions. At the Global Health and Purpose Summit, as part of People and Planet United, Theodore Zanos, Head of the Division of Health AI at Northwell Health, joins Ivan Ruiz, Partner at FINN Partners, for a conversation on health AI governance and the future of responsible care.

The session presents health AI governance not as a defensive compliance exercise, but as an operating capability. AI is reshaping healthcare from reactive care toward anticipatory care, from intuition-driven practice toward data-augmented reasoning, and from episodic snapshots toward a more continuous understanding of the patient. Yet the same technologies can create new risks when they are adopted without inventory, validation, workflow design, security controls, monitoring, and accountable ownership.

AI Changes the Operating Model of Medicine

Ruiz opens the session by placing AI governance in the context of healthcare transformation. For many people, artificial intelligence begins with consumer tools, conversational interfaces, and generative AI. Zanos quickly expands the frame. Healthcare AI is already influencing administrative work, clinical decision support, risk stratification, patient monitoring, and the role of clinicians in interpreting information.

The first shift is from reactive care to anticipatory care. The second is from intuition-driven practice to data-based, augmented reasoning. The third is from episodic snapshots of patient status to a continuous and more holistic understanding of patient health. These are not simply new tools layered onto old workflows. Zanos describes them as different operating models for medicine.

The most visible current use cases are administrative, including ambient scribes, prior authorization automation, and documentation assistance. These uses deliver productivity gains and some clinician relief. The deeper clinical transformation is slower, but it is emerging through AI-enabled clinical decision support, prediction of deterioration, sepsis risk identification, readmission prediction, no-show prediction, mortality models, and risk stratification.

The implication is significant. AI should not make clinicians serve the computer more efficiently. It should help clinicians move away from administrative data collection and toward interpretation, judgment, care, and action. Zanos warns that healthcare must avoid repeating the mistakes of the electronic health record era, when technologies promised liberation from mundane tasks but often contributed to burnout.

Governance Begins with Knowing What Exists

Governance becomes essential because healthcare has a different risk profile from most industries. A tool that looks efficient in another sector can create clinical, operational, ethical, or security consequences in a health system. Zanos defines governance as a set of principles and processes that allow health systems to remain compliant, protect patients, and use the capabilities of AI to their fullest.

The first governance step is inventory. Health systems often have older decision-support tools, vendor-embedded algorithms, grandfathered processes, and new AI applications spread across departments. Without a clear catalog, leaders cannot know who uses what, how it is used, whether it affects clinical decisions, and what risks it creates.

That basic discipline matters because governance is not only about reviewing new tools. It also means understanding the tools already present in the system, the contracts that contain AI functionality, the models embedded inside workflows, and the decision-support mechanisms that may predate a formal AI governance program.

Shadow AI and the Security Risk of Convenience

The rise of consumer chatbots introduces a new governance challenge. Clinicians, nurses, and staff are often under pressure to save time and solve practical workflow problems. Public AI tools may appear useful for summarization, drafting, translation, or administrative support, but when patient health information is placed into a non-compliant consumer tool, the convenience becomes a security incident.

Zanos describes this as the shadow AI problem. Blocking public chatbots from a network does not solve the issue because people have personal devices and will still search for productivity shortcuts. The better answer is to acknowledge the demand, provide safe alternatives, and create internal tools that allow staff to use AI without leaking protected health information.

At Northwell Health, Zanos notes that the organization has created an internal AI hub where large language models can support administrative and productivity use cases without sending data to non-HIPAA-compliant servers. That approach reflects a broader governance principle. Responsible systems should not merely prohibit risky behavior. They should make safer behavior easier and more useful.

Validation as a Clinical Discipline

Validation is one of the core disciplines of responsible health AI. A model’s vendor-reported performance is not enough. Performance can change when the population changes, when workflows differ, when data sources vary, or when time introduces new patterns. Zanos emphasizes that health systems must validate tools in their own environment, with their own patients, workflows, and data.

This principle applies before deployment, during pilots, and throughout the life cycle of the tool. Health systems can run tools retrospectively on historical data, assess performance against local needs, and determine whether the tool is good enough for the clinical context. That process may be expensive and slow, but it is necessary if leaders want to know whether the technology works in the setting where it will be used.

Vendors also have a role. Zanos argues that companies should be willing to provide not only general performance metrics, but also support local performance evaluation. Transparency should not be treated as a burden. It is an opportunity to show that a tool works in the context where it will matter.

Pilots Must Be Designed to Learn

One of the strongest warnings in the session concerns pilot design. Healthcare organizations often choose friendly environments, enthusiastic champions, intensive coaching, and metrics that are likely to improve. That approach may create a positive story, but it does not teach the organization whether the tool can survive scale.

Real pilots should contain failure conditions. They should test assumptions, expose risks, identify workflow gaps, and reveal whether the organization is willing to accept and manage the risks of deployment. If the pilot is structured only to succeed, the tool may fail when it reaches less friendly sites, different staff, variable workflows, and the operational complexity of a health system.

The point is not to make innovation harder. It is to make innovation more honest. A pilot that teaches the organization what can go wrong is more valuable than a demonstration that merely produces a press release.

Clinician Trust and the Cognitive Budget

Ruiz brings the conversation to the clinical reality of adoption. Many physicians are already using AI tools, especially chatbot-style tools and trusted information environments. The challenge is to ensure that these tools support clinicians rather than create more work.

Zanos argues that attention is a scarce clinical resource. Every alert, prompt, interruption, false positive, or ambiguous recommendation consumes part of a clinician’s cognitive budget. When tools repeatedly flag the same patient without meaningful clinical value, users stop responding. That behavior is often blamed on resistance, but Zanos calls it a system design failure.

Trust is built not only through explainability, but through usefulness and reliability. Zanos notes that clinicians already use many technologies without understanding every underlying technical mechanism. They do not need to understand the full mathematical architecture of a model if it reliably provides a useful answer and fits the workflow. What they need is accuracy, transparency about use, training, clear limitations, and confidence that the tool helps rather than distracts.

The Shared Responsibility of Health AI

Responsible health AI is not the responsibility of one actor alone. Zanos identifies three main players: health systems, technology companies, and regulators. He describes them as three legs of a stool. If one fails, the whole structure becomes unstable.

Health systems carry the heaviest responsibility because they buy, develop, deploy, integrate, and use AI tools with patients. They cannot hide behind vendors or regulators because they bear the consequences, positive or negative, of how tools affect care.

Technology companies must increase transparency around training populations, data sources, performance metrics, subgroup performance, known failure modes, and prior deployment experience. Zanos is clear that this does not require vendors to expose proprietary intellectual property. It requires honest reporting rather than cherry-picked numbers.

Regulators face a different challenge. Existing frameworks were designed for relatively static products such as devices or drugs, while AI systems may evolve rapidly. Post-market surveillance, speed, flexibility, and practical guardrails will become increasingly important as AI use expands faster than regulatory frameworks can adapt.

Responsible Care Requires Clinical Leadership

The session concludes with a leadership standard for health systems. AI cannot be managed as a narrow IT project. Patient outcomes cannot become a footnote. AI must be treated as a clinical capability, led across clinical, operational, digital, technical, and governance teams.

Organizations that successfully implement AI will involve stakeholders early, focus on real problems, validate locally, invest in infrastructure, monitor performance, design governance properly, and decommission tools that do not work. Zanos also highlights the value of less glamorous problems. The infrastructure, dashboards, monitoring systems, version control, and operational processes that may not generate headlines are essential to safe AI at scale.

Ruiz closes by returning to the practical and human stakes of the conversation. The future of health AI depends not only on the technology, but on implementation, responsibility, clinicians, administrative staff, and patients. The objective is not merely to adopt AI. The objective is to make care safer, more effective, more efficient, and more responsive to the people healthcare exists to serve.

Health AI Governance and the Future of Responsible Care offers a timely and practical roadmap for healthcare leaders. AI will continue to expand across administrative workflows, clinical decision support, patient engagement, and operational systems. The organizations that succeed will not be those that simply adopt the most tools. They will be those that govern AI with discipline, validate it locally, design it around clinicians, protect patient trust, and treat responsible care as the measure of progress.

Session Intelligence

This leadership conversation examines health AI governance through the lens of clinical transformation, responsible adoption, shadow AI, local validation, workflow design, clinician trust, vendor transparency, regulation, monitoring, and patient outcomes. Its central insight is that AI becomes a clinical capability only when governance turns innovation into safe, useful, and accountable care.

Core Leadership Theme

Health AI governance must be treated as an operating capability that connects compliance, clinical quality, workflow design, and patient outcomes.

Clinical Adoption Signal

AI earns clinician trust when it is accurate, useful, locally validated, and respectful of the finite cognitive budget of care teams.

System Impact Standard

Responsible AI requires health systems, technology companies, and regulators to share responsibility for safe, transparent, and effective deployment.

Health AI Governance Responsible AI Clinical AI Validation Shadow AI AI in Healthcare Clinician Trust Health System Leadership Responsible Care