Document text
Principal Investigator: Bradford Wood
Organization: CLINICAL CENTER
Fiscal Year: 2024
Funding agency: NIH Clinical Center
A multidisciplinary multi-institute, public-private partnership tackled the goal of developing and validating AI tools and standardized methodologies for clinical dynamics and novel classification tools for medical imaging, voice analysis, data sharing, and detection and prevention of dynamic diseases. A public pipeline for classification of COVID-19 (vs Flu) on chest CT was deployed. The NIH and extended team were among the first to gather multi-national data and develop freeware public AI solutions based on COVID CTs for academic, researcher, and commercial developer use. A uniform and standardized methodology for automatic classification of disease based on imaging, voice spectrograms, text input, or information from wearables could expedite the pathway towards drug discovery, infection outbreak and migration, and non-invasive quantification of disease such as vasculitis, post-operative clots or atelectasis.
The NIH team developed and helped publicly post COVID-19 data and tools on TCIA and MIDRC, including the largest (summer 2020) chest CT dataset posted for the 1st year of the pandemic. NVIDIA and NIH co-developed AI models that detected COVID-19, differentiated from influenza, fungal, or bacterial pneumonias as well as other entities. AI models were able to predict the later need for critical care therapies based upon an initial CT scan early on, at the initial point of care. The public-private multinational partnership also used "federated learning" to train an AI model in 8 nations and 20 institutions that was able to predict subsequent oxygen needs based upon the initial point-of-care chest X-ray alone (Nature Medicine). This demonstrated methodology for data collaboration protects while maintaining privacy and allowing the data itself to remain at the home institution. Federated learning can in this way overcome shortcomings in unbalanced source data for AI, by sharing "model weights" instead of the actual data. This enabling technique overcome data sharing gaps, thus showing that the data does not need to be fully shared, in order to build quality AI models from medical imaging.
The team also showed that CT AI can track disease in a predictable fashion in the pre-symptomatic, asymptomatic, and pauci-symptomatic patient, and that the general dynamic curve of disease has dynamic curve lab correlates may be predictive and recapitulate available preclinical models. Correlation with zip codes or cell phone towers could theoretically predict disease outbreak and migration patterns.
CT image processing and deep learning models provide quantifiable metrics to serve as a noninvasive biomarkers for pulmonary involvement in COVID-19. A MICCAI AI data challenge in COVID-19 was organized around the data that the team curated. The NIH multi-national dataset (>3000 CTs /4 nations) showed that CT may be positive days before PCR. Thus, the suggestion that CT could function as a targeted epidemiological tool to perhaps augment PCR and antibody testing in specific limited scenarios or better define patterns of spread. Early signal for Omicron correlatives also led to development of a classification model purely from voice audiograms / spectrograms with high performance metrics, which was not true for Alpha and Delta.
Partnerships with the Trans-NIH working group has been forged, including NIAID IRF, NIBIB MIDCR, NCI, NCATS, and N3C. Centralized communication and discovery pathways for COVID-19-related data science that involves medical imaging like CT or chest x-ray is a common theme and goal, and provides a fertile ground for advancement of data science with broad scope impact in cancer and in interventional radiology and well outside of infectious diseases.
NIH participated in publishing and disseminating methods for handling COVID-19 in the angiography suite, details about post-partum COVID, designed and characterized a disposable isolation device ("full body mask") that reduces contamination in health care settings such as transport of COVID positive patients, validated in vivo a miniature 3D printable ventilator for resource-starved pandemic settings, and deployed a camera with custom software to identify social distancing distances with a standard webcam. A clinical trial for training AI models for Omicron detection from public social media audio data was IRB approved. Smartphone tools for instant anonynmization of imaging data were developed. A smartphone app for point-of-care deployment was created for running inference on clinical PACS 2D imaging or for cloud transmittal. Further collaborations with N3C, industry, Oxford and IRF NIAID were developed for assessment of AI tools with PPG, wearables, and smartphone apps.
Public data posting of CT scans on public NCI TCIA websites were made in past years. AI deep learning models were made alongside of multiple industry partners, to educate on the serial temporal dynamics of diseases such as COVID-19. AI deep learning models were built and shared for research purposes to automatically segment tumors, thermal ablation treatment zones, liver organ, or COVID-19 opacities, as well as try to classify COVID-19. NIH CC and NCI were among the first to gather multi-national data and develop freeware public AI solutions based on COVID CTs for both academic and commercial developer use in early years of the pandemic. Federated learning was piloted with academic and industry partners in several projects in Nature Medicine and JAMIA publications. The NIH team is working with commercial and academic partners to assess deep learning tools in cancer. Ongoing work will attempt to deploy voice models deployed on smartphones for pre-screening settings, and to assess voice input towards AI uses in healthcare. It was shown that pre-symptomatic CT AI can track disease in a predictable fashion, and that this disease dynamic curve is recapitulated in a non-human primate model of COVID-19. Prior work with extramural partners has demonstrated that federated learning can overcome shortcomings in unbalanced source data for imaging AI, and that the application of a specific federated learning technique can overcome the gap, thus showing that the data does not need to be shared in order to build quality AI models from medical imaging. This effort cross links with numerous campus efforts, including preclinical NIAID efforts, NCI/CCR efforts within AI Resource, and extramural partners in Bridge to AI. CC/NCI team members also deployed a 3D-printed miniature ventilator in swine (now commercialized) as well as a disposable isolation bag device with in-line air filtration. Highly impactful AI models were developed and licensed towards the detection, characterization, and assessment of prostate cancer using MRI and MRI-US fusion biopsy. These models may have broad scope impact. Initial feasibility was begun for data curation of voice and wearable sensor data towards health care AI models. A large language model pathway for use of procedural imaging data was outlined and connections with MDRIC were initiated towards an AI toolkit for procedural medicine applications. A checklist for publications in AI was created for multiple simultaneous journal publication for the specialty of Interventional Radiology.
Terms: <3-D><3-D print><3-D printer><3-Dimensional><3D><3D Print><3D printer><3D printing><AI based model><AI language models><AI model><AI system><ATLEC><Algorithms><Android App><Android Application><Angiitis><Angiogram><Angiography><Antibodies><Artificial Intelligence><Atelectasis><Audiogram><Audiometric Test><Audiometry><B2AI><Bacterial Pneumonia><Biological Markers><Biopsy><Bridge to Artificial Intelligence><Bridge2AI><CAT scan><CCR><COVID crisis><COVID detection><COVID epidemic><COVID infected patient><COVID pandemic><COVID patient><COVID positive patient><COVID-19><COVID-19 crisis><COVID-19 detection><COVID-19 epidemic><COVID-19 era><COVID-19 global health crisis><COVID-19 global pandemic><COVID-19 health crisis><COVID-19 infected patient><COVID-19 pandemic><COVID-19 patient><COVID-19 period><COVID-19 positive patient><COVID-19 public health crisis><COVID-19 years><COVID19 detection><COVID19 patient><COVID19 positive patient><CT X Ray><CT Xray><CT imaging><CT scan><CV-19><CXR><Cancers><Cell Communication and Signaling><Cell Phone><Cell Phone Application><Cell Signaling><Cell phone App><Cellular Phone><Cellular Phone App><Cellular Phone Application><Cellular Telephone><Classification><Clinical><Clinical Trials><Clotting><Coagulation><Coagulation Process><Collaborations><Communicable Diseases><Communication><Computed Tomography><Computer Reasoning><Computer software><Coronavirus Infectious Disease 2019><Critical Care><Custom><Data><Data Science><Data Set><Detection><Development><Devices><Disease><Disease Outbreaks><Disorder><EXTMR><Epidemiology><Event><Extramural><Extramural Activities><Family suidae><Future><Goals><Healthcare><Hearing Tests><Home><IRB><IRBs><Image><Industry><Infection><Infectious Disease Pathway><Infectious Diseases><Infectious Disorder><Institution><Institutional Review Boards><Interventional radiology><Intracellular Communication and Signaling><Investigation><Investigators><Journals><Knowledge><Learning><Licensing><Link><Liver><Lung><Lung Diseases><Lung Respiratory System><MR Imaging><MR Tomography><MRI><MRIs><Machine Intelligence><Magazine><Magnetic Resonance Imaging><Malignant Neoplasms><Malignant Tumor><Malignant Tumor of the Prostate><Malignant neoplasm of prostate><Malignant prostatic tumor><Masks><Measures><Medical><Medical Imaging><Medical Imaging, Magnetic Resonance / Nuclear Magnetic Resonance><Medical Research><Medicine><Methodology><Methods><Mobile Phones><Modality><Modeling><NCATS><NIAID><NIBIB><NIH><NMR Imaging><NMR Tomography><National Center for Advancing Translational Sciences><National Institute of Allergy and Infectious Disease><National Institute of Biomedical Imaging and Bioengineering><National Institutes of Health><Nature><Nuclear Magnetic Resonance Imaging><O element><O2 element><Organ><Outbreaks><Oximetry><Oxygen><Oxygen saturation measurement><Pathway interactions><Patients><Pattern><Performance><Phenotype><Physical distancing><Pigs><Post-Operative><Postoperative><Postoperative Period><Postpartum Period><Pre-Clinical Model><Preclinical Models><Prevention><Privacy><Privatization><Prostate CA><Prostate Cancer><Prostate malignancy><Prostatic Cancer><Publications><Publishing><Pulmonary Diseases><Pulmonary Disorder><Reporting><Research><Research Personnel><Research Resources><Researchers><Resources><Running><SARS-CoV-2 detection><SARS-CoV-2 epidemic><SARS-CoV-2 global health crisis><SARS-CoV-2 global pandemic><SARS-CoV-2 infected patient><SARS-CoV-2 pandemic><SARS-CoV-2 patient><SARS-CoV-2 positive patient><SARS-coronavirus-2 epidemic><SARS-coronavirus-2 pandemic><Scientific Publication><Severe Acute Respiratory Syndrome CoV 2 epidemic><Severe Acute Respiratory Syndrome CoV 2 pandemic><Severe acute respiratory syndrome coronavirus 2 epidemic><Severe acute respiratory syndrome coronavirus 2 pandemic><Signal Transduction><Signal Transduction Systems><Signaling><Smart Phone App><Smart Phone Application><Smartphone App><Social Distance><Software><Source><Specialty><Standardization><Starvation><Suggestion><Suidae><Swine><Systematics><TCIA><Techniques><Testing><Text><The Cancer Imaging Archive><Thermal Ablation Therapy><Thoracic Radiography><Tomodensitometry><Training><Transformer language model><Triage><United States National Institutes of Health><Vasculitis><Ventilator><Voice><Weight><Work><X-Ray CAT Scan><X-Ray Computed Tomography><X-Ray Computerized Tomography><Xray CAT scan><Xray Computed Tomography><Xray computerized tomography><Zeugmatography><air filtration><angiographic imaging><artificial intelligence language models><artificial intelligence model><artificial intelligence-based model><auditory tests><auto-segmentation><automated segmentation><automatic segmentation><autosegmentation><bacteria pneumonia><bio-markers><biologic marker><biological signal transduction><biomarker><cancer invasiveness><catscan><cell phone based app><cell phone based device><chest CT><chest X ray><chest Xray><chest computed tomography><chest radiography><clinical imaging><commercialization><computed axial tomography><computer tomography><computerized axial tomography><computerized tomography><coronavirus detection><coronavirus disease 2019><coronavirus disease 2019 crisis><coronavirus disease 2019 detection><coronavirus disease 2019 epidemic><coronavirus disease 2019 global health crisis><coronavirus disease 2019 global pandemic><coronavirus disease 2019 health crisis><coronavirus disease 2019 infected patient><coronavirus disease 2019 pandemic><coronavirus disease 2019 patient><coronavirus disease 2019 positive patient><coronavirus disease 2019 public health crisis><coronavirus disease crisis><coronavirus disease detection><coronavirus disease epidemic><coronavirus disease infected patient><coronavirus disease pandemic><coronavirus disease patient><coronavirus disease positive patient><coronavirus disease-19><coronavirus disease-19 global pandemic><coronavirus disease-19 pandemic><coronavirus disease-19 patient><coronavirus infectious disease-19><coronavirus patient><crosslink><curating data><customs><data captured from wearables><data collected from wearables><data collected using wearables><data curation><data gathered from wearable><data gathered through wearables><data gathered via wearable><data sharing><deep learning><deep learning based model><deep learning method><deep learning model><deep learning strategy><design><designing><detect COVID><detect COVID-19><detect COVID19><detect SARS-CoV-2><detect coronavirus><detect coronavirus disease><detect severe acute respiratory syndrome coronavirus 2><developmental><disease classification><disease of the lung><disorder classification><disorder of the lung><drug discovery><epidemiologic><epidemiological><federated learning><flu><forging><fungal pneumonia><generative AI><generative artificial intelligence><generative models><health care><health care settings><healthcare settings><hearing assessment><hepatic body system><hepatic organ system><homes><iOS app><iOS application><iPhone><iPhone App><iPhone Application><image guided therapy><image processing><imaging><in vivo><industrial partnership><industry partner><industry partnership><influenza pneumonia><large data sets><large datasets><large language model><large scale language model><lung disorder><lung radiography><malignancy><massive scale language models><medical specialties><member><migration><minimally invasive><mobile phone app><mobile phone based device><multi-modality><multidisciplinary><multimodal modeling><multimodality><neoplasm/cancer><non-contrast CT><non-human primate><noncontrast CT><noncontrast computed tomography><nonhuman primate><nosology><novel><outcome prediction><pandemic><pandemic disease><pathway><patient infected with COVID><patient infected with COVID-19><patient infected with SARS-CoV-2><patient infected with coronavirus disease><patient infected with coronavirus disease 2019><patient infected with severe acute respiratory syndrome coronavirus 2><patient with COVID><patient with COVID-19><patient with COVID19><patient with SARS-CoV-2><patient with coronavirus disease><patient with coronavirus disease 2019><patient with severe acute respiratory distress syndrome coronavirus 2><point of care><porcine><post-COVID><post-COVID-19><post-coronavirus disease 2019><post-partum><pre-clinical><preclinical><public-private partnership><pulmonary><radiographic chest image><radiographic lung image><response><screening><screenings><severe acute respiratory syndrome coronavirus 2 detection><severe acute respiratory syndrome coronavirus 2 global health crisis><severe acute respiratory syndrome coronavirus 2 global pandemic><severe acute respiratory syndrome coronavirus 2 infected patient><severe acute respiratory syndrome coronavirus 2 patient><severe acute respiratory syndrome coronavirus 2 positive patient><skills><smart phone><smartphone><smartphone application><smartphone based app><smartphone based application><smartphone based device><smartphone device><social media><suid><thermal ablation><thermal tumor ablation><thoracic radiogram><thorax radiography><three dimensional><three dimensional printing><tool><transfer learning><tumor><vasculitides><wearable><wearable data><wearable device><wearable device data><wearable electronics><wearable sensor data><wearable system><wearable technology><wearable tool><wearables><web site><website><weights><work group><working group>