live-edit recap blog - ai-prompt-engineering.html

ONC 2023 Final Meeting

Executive Summary of ONC 2023 Final Meeting - Day Two

ONC Health IT Certification Program - Criteria for AI
Informing – How AI gets new information
Deriving – How AI establishes Commonalities/Relationships
Predicting - How AI provides recommendations with meaningful outcome categories.
Related Resources

• 21st Century Cures Act: Interoperability, Information Blocking, and the ONC Health IT Certification Programs.
• Health Data, Technology, and Interoperability: Certification Program Updates, Algorithm Transparency, and Information Sharing (HTI-1) Final Rule

Clinical Decision Support Models with AI Predicted Outcomes

---

World Shift - Pandemic

FDA AI Use Case - "World Shift Scenario" 
Narrative Description: When a settled way of working suddenly becomes no longer sound nor reliable. 
Recent (2020 Pandemic) example of a World Shift Scenario 

1. health policies suddenly evolved to respond to new health issue; and
2. established in-person collaboration were suddenly disrupted with remote collaboration.

---

FDA Proposed Discipline for Artificial Intelligence (AI) to minimize Data Drift risk
Objective: "Seize the promise of AI and minimize the risk"

Governance Process for "Data Drift"
1)  Monitor 

• "counterfactual" metrics that measure AI Bias "Data Drift"
• "Key Educators" that indicate sound & Reliable AI Outcomes

2. Identify 

• Data used to train AI
  *Public/Private Data Sets
• Problem Statement AI intends to solve

3. Tune AI Outcomes for when "the world shifts" with additional:

• Know How
• Context

Governance Process for AI Alignment

• Security Controls 
• Development Discipline "that apply guidelines with rigor"
  see: Governance Process for Data Drift
• Risk Guidelines "how many people would be impacted if this goes wrong"

---

Reference Examples of AI for NIH Use Case​

ONC Health IT Certification Program

• 21st Century Cures Act: Interoperability, Information Blocking, and the ONC Health IT Certification Programs.
  
• Health Data, Technology, and Interoperability: Certification Program Updates, Algorithm Transparency, and Information Sharing (HTI-1) Final Rule
  
  

Use Cases for AI

NIH Clinical Reference for proposed AI products: https://pubmed.ncbi.nlm.nih.gov/30819365/ 
Proposed AI Products

1. "Patient Sepsis Monitoring"
   Problem Statement "Predict Patient Sepsis Condition before/instead of a lab study"
2. "PCI - percutaneous coronary intervention"
   Problem Statement "Expedite heart studies for Patients with Coronary Artery Disease"
3. CABG - Coronary artery bypass graft surgery
   Problem Statement "AI Predict actions that Prevent Patient Sepsis"

Criteria for AI

1. Informing – Gets New Information
   
2. Deriving – New Commonalities/Relationships
   
3. Predicting - Outcomes with meaningful categories

---