Document text
Principal Investigator: Joseph Frank
Organization: CLINICAL CENTER
Fiscal Year: 2023
Funding agency: NIH Clinical Center
The availability of non-human (NHP) CT data from SARS-CoV-2 animal imaging studies at the Integrated Research Facility (IRF) and human CT data from the COVID-19 pandemic has presented an opportunity to explore the interaction of combining NHP and human CT lung lesion data to enhance deep learning (DL) image segmentation capabilities for subjects with the same infection. To enhance DL model accuracies developed with the Deep learning Medical Imaging SegmenTation (DMIST) pipeline, we have been working on leveraging available NHP data to improve lung lesion segmentation. We are exploring different methodologies such as developing NHP pretrained models and deep learning inter-species transfer learning. Experimentation of this has shown promising results where pretrained models and models trained on a combination of NHP and human CT data outperform models which are only trained on human data. This is motivating as the IRF-Frederick has a large collection of animal imaging datasets that could be leveraged to improve segmentation model performances.
Currently, there are multiple convolutional neural network (CNN) frameworks have been successful implemented in the DMIST pipeline and has led to utilizing the DMIST pipeline workflow to tune and explore model parameters for improving lung lesion segmentation in both NHP and human CT data. In-line with making improvements to DL segmentation models, efforts to add a model comparison module to the DMIST pipeline has helped expand DMIST capabilities and workflows. The addition of the module allows users with little to no programming knowledge to be able to compare different model parameters to one another utilizing a simple configuration file. This helps enable IRF-Frederick researchers to quickly iterate through not only the model development process but also the evaluation process by quickly comparing any number of models all on a single benchmark dataset and aggregate results.
Monitoring the new state-of-the-art (SOTA) developed CNN models for medical imaging applications could significantly reduce efforts to develop these models inhouse. Currently, the U-NET, U-NET Transformer (UNET-R), and Shifted Window U-NET Transformer (Swin-UNETR) are implemented in the DMIST pipeline and have been successful for lung lesion segmentation in SARS-CoV-2 subjects. However, the predictive accuracies of the lung lesion models are still lacking due to challenges related to collecting annotations. We explored the nnUNET model architecture which has been shown to be successful for a wide variety of segmentation tasks on medical images by training the model on NHP lung lesion CT data. We see that this SOTA model architecture is outperformed by the Swin-UNETR. This demonstrations SOTA DL segmentation models can be successful for various segmentation tasks but it should be investigated to find the optimal solution to solve IRF-Frederick related image segmentation needs.
The AI team has built up the software infrastructure of the lab, enabling future analyses to be completed faster, more consistently, and without the need to edit the base code. Particularly, major improvements were made to the Deep-learning Medical Image Segmentation Toolkit model Training pipeline (DMIST-Train). New capabilities were added including cross-validated model training, additional models, and editable parameters. One major new capability that was added was the ability to configure the entire training process from a configuration file. This allows people without knowledge of the inner workings of the pipeline to run deep learning model training and experiment with hyperparameters. Another major new capability is the ability to train a self-supervised model, a burgeoning technique that is at the forefront of deep learning research. Each of these new capabilities required a significant overhaul of the code but was done in a modular way such that future new additions will be easier. The AI team also built an entirely new toolkit for running classical machine learning. These were based on previously written scripts to run the experiments for the SARS-CoV-2 Classification study but were rebuilt to be applicable to any new structure dataset. By rebuilding the scripts into modules, it enables us to expand the scope of experiments we could run to include regression model training and feature selection within the cross-validation loop. The rebuild also means that new plots and capabilities added to the toolkit can be easily applied to enhance all studies across multiple datasets.
Significant contributions were also made to the groups Deep-learning Medical Image Segmentation Toolkit model deploy pipeline (DMIST-Deploy). Specific contributions include standardizing the Python environment setup process by creating conda and pip requirements/specification files, implementing bulk calculation of radiomic features from NIFTI filetypes, and adding additional metadata details to the output of radiomic feature calculation. The process for generating multiple training configuration files when optimizing parameters or performing a parameter grid search was automated. To enhance code quality, improvements were made to code readability and documentation by applying the standard Python formatting module "black" to the codebase, improving repository README files, and adding useful comments throughout the source code. Additionally, the AI team helped develop and implement best practices in collaborative software development on GitHub.
Terms: <2019 novel corona virus><2019 novel coronavirus><2019-nCoV><AI based method><AI system><Aggregated Data><Architecture><Artificial Intelligence><Benchmarking><Best Practice Analysis><COVID crisis><COVID epidemic><COVID pandemic><COVID-19 crisis><COVID-19 epidemic><COVID-19 global health crisis><COVID-19 global pandemic><COVID-19 health crisis><COVID-19 pandemic><COVID-19 public health crisis><COVID-19 virus><COVID19 crisis><COVID19 epidemic><COVID19 global health crisis><COVID19 global pandemic><COVID19 health crisis><COVID19 pandemic><COVID19 public health crisis><COVID19 virus><CT Lung Screening><Classification><CoV-2><CoV2><Code><Coding System><Collection><Comment><Commentary><Communicable Diseases><Computer Reasoning><Connectionist Models><ConvNet><Data><Data Aggregation><Data Set><Disease Progression><Documentation><Editorial Comment><Engineering / Architecture><Environment><Evaluation><Future><Human><Image><Infection><Infectious Disease Pathway><Infectious Diseases><Infectious Diseases / Laboratory><Infectious Diseases Research><Infectious Disorder><Investigators><Knowledge><Lesion><Lung CAT Scan><Lung CT><Lung CT scan><Machine Intelligence><Machine Learning><Medical Imaging><Metadata><Methodology><Methods><Modeling><Modern Man><Monitor><NIAID><National Institute of Allergy and Infectious Disease><Neural Network Models><Neural Network Simulation><Organ><Output><Pathology><Perceptrons><Performance><Persons><Phenotype><Process><Published Comment><Pythons><Readability><Research><Research Personnel><Researchers><Running><SARS corona virus 2><SARS-CO-V2><SARS-COVID-2><SARS-CoV-2><SARS-CoV-2 epidemic><SARS-CoV-2 global health crisis><SARS-CoV-2 global pandemic><SARS-CoV-2 pandemic><SARS-CoV2><SARS-CoV2 epidemic><SARS-CoV2 pandemic><SARS-associated corona virus 2><SARS-associated coronavirus 2><SARS-coronavirus-2><SARS-coronavirus-2 epidemic><SARS-coronavirus-2 pandemic><SARS-related corona virus 2><SARS-related coronavirus 2><SARSCoV2><Severe Acute Respiratory Coronavirus 2><Severe Acute Respiratory Distress Syndrome CoV 2><Severe Acute Respiratory Distress Syndrome Corona Virus 2><Severe Acute Respiratory Distress Syndrome Coronavirus 2><Severe Acute Respiratory Syndrome CoV 2><Severe Acute Respiratory Syndrome CoV 2 epidemic><Severe Acute Respiratory Syndrome CoV 2 pandemic><Severe Acute Respiratory Syndrome-associated coronavirus 2><Severe Acute Respiratory Syndrome-related coronavirus 2><Severe acute respiratory syndrome associated corona virus 2><Severe acute respiratory syndrome coronavirus 2><Severe acute respiratory syndrome coronavirus 2 epidemic><Severe acute respiratory syndrome coronavirus 2 pandemic><Severe acute respiratory syndrome related corona virus 2><Severity of illness><Source Code><Specific qualifier value><Specified><Standardization><Structure><Systematics><Techniques><Training><Validation><Viewpoint><Writing><Wuhan coronavirus><animal imaging><artificial intelligence method><base><base editing><bases><benchmark><computer based prediction><computerized tomography lung screening><convolutional network><convolutional neural nets><convolutional neural network><corona virus disease 2019 epidemic><corona virus disease 2019 pandemic><coronavirus disease 2019 crisis><coronavirus disease 2019 epidemic><coronavirus disease 2019 global health crisis><coronavirus disease 2019 global pandemic><coronavirus disease 2019 health crisis><coronavirus disease 2019 pandemic><coronavirus disease 2019 public health crisis><coronavirus disease 2019 virus><coronavirus disease crisis><coronavirus disease epidemic><coronavirus disease pandemic><coronavirus disease-19 global pandemic><coronavirus disease-19 pandemic><coronavirus disease-19 virus><deep learning><deep learning based model><deep learning model><depository><develop software><developing computer software><disease severity><experiment><experimental research><experimental study><experiments><feature selection><hCoV19><human data><imaging><imaging Segmentation><imaging study><improved><lung computerized axial tomography scan><lung computerized tomography><lung computerized tomography scan><lung lesion><machine based learning><machine learning based classification><machine learning based method><machine learning classification><machine learning method><machine learning methodologies><medical countermeasure><meta data><model development><model developments><multi-modality><multimodality><multiple data sets><multiple datasets><nCoV2><prediction model><predictive modeling><pulmonary lesion><radiomics><repository><research facility><severe acute respiratory syndrome coronavirus 2 global health crisis><severe acute respiratory syndrome coronavirus 2 global pandemic><software development><software infrastructure><transfer learning><validations>