NVIDIA Clara, the greatest showing of Artificial Intelligence in Healthcare.

NVIDIA Clara, (somewhat) recently announced at NVIDIA’s GPU Technology Conference 2019 in March of this year, is a formidable developer toolkit based on NVIDIA’s compute platform and is aimed at Healthcare with an intent to accelerate ‘data acquisition, analysis and data integration’. As you may have seen in your recent hospital visits, there is a growing multitude of software defined solutions, such as from Cerner and Philips, and these have revolutionised the healthcare industry by contributing to the automated and digitally generated data sets that embody modern healthcare. According to NVIDIA, ‘Clara aims at providing access to technological advancements in hardware and software for developers across medical imaging and genomics to accelerate the future of medicine.’ NVIDIA most openly promotes Clara for its usage in Medical Imaging and Genomics, so guess what we are going to talk about next?

However, before we proceed further, I would like to take a short while to explain these terms. Firstly, Medical Imaging refers to a variety of different technologies that intend to view the human body, at far greater detail and usefulness than achieved by a simple physical examination, which is done to diagnose, monitor, and treat medical conditions. Each individual technology gives a different subset of information, using different modalities of function and processes, about the area of the body being studied and can be tailored towards the working diagnosis established by clinicians. For example, you would conventionally assess the progress of a developing fetus with ultrasound, not an MRI! Medical Imaging is widely used in the follow-up of disease that has been treated or diagnosed. The following are a few examples of Medical Imaging technologies:

  • Computed Tomography, CT

  • Radiography

  • Dental Cone-beam CT

  • Fluoroscopy

  • Mammography

  • Ultrasound Imaging

  • Magnetic Resonance Imaging

  • X-ray Imaging, Paediatric, and Medical

A diagram depicting the role of computing in a CT scan procedure. Credits: ResearchGate

A diagram depicting the role of computing in a CT scan procedure. Credits: ResearchGate

Medical imaging, especially X-ray based examinations and ultrasonography, is a fundamental element in all levels of healthcare in a large variety of medical settings. According to the World Health Organisation, ’Public health and preventive medicine, even in curative and palliative care, effective decisions rely on equally effective diagnoses. Even though a clinician’s medical judgment may provide sufficient prior to treating a medical condition, diagnostic imaging processes are fundamental in ‘confirming, correctly assessing and documenting courses of many diseases as well as in assessing responses to treatment (as we saw recently in my MS article - HERE). With improvements in healthcare policy and the rapidly increasing availability of excellent medical equipment, the number of global imaging-dependent procedures is increasing exponentially. Effective, safe, and high-quality imaging is important for medical decision-making and can event prevent unnecessary procedures. For example, some surgical interventions can be avoided altogether if simple diagnostic imaging services such as ultrasound are available.’ This clearly demonstrates the value of Medical Imaging, and I can confidently say that it has transformed modern healthcare, you only must look at the widespread implementation of ultrasound which in itself has revolutionised surgical procedures such as fitting a chest drain, which was previously done blindly and made some shocking headlines over the years. I don’t know about you, but I wouldn’t be too happy if my liver suddenly had a needle through it… Not only this, Medical Imaging is another aspect of care where we see a multidisciplinary effort with the contribution of radiographers, sonographers, medical physicists, nurses, biomedical engineers, and other supporting staff members to bring effective diagnoses and treatment for all patients.

(Left) A patient entering an MRI scanner positioned to produce an image of the brain. Credit: Monty Rakusen, Getty Images. (Right) An MRI scan of the brain, hopefully displaying the time and effort it takes to correctly analyse an MRI scan. AI-assistance could be of great use in a field such as Radiology with the time-saving measures it promises, without compromising an expert level of accuracy. Credit: BrainFacts

(Left) A patient entering an MRI scanner positioned to produce an image of the brain. Credit: Monty Rakusen, Getty Images. (Right) An MRI scan of the brain, hopefully displaying the time and effort it takes to correctly analyse an MRI scan. AI-assistance could be of great use in a field such as Radiology with the time-saving measures it promises, without compromising an expert level of accuracy. Credit: BrainFacts

However, what can Clara do? Well according to NVIDIA, the medical imaging industry has been undergoing a revolution in the last decade or so. In fact, the earliest processes to take advantage of GPU computing were in fact image and signal processing applications. However, upon this foundation, we get to today where GPUs are found in almost all imaging forms, including in Computed Tomography, X-ray, Ultrasound, and MRI, and their introduction has brought vast compute capabilities to these devices, accompanying an upward trend in the success of common day to day surgical interventions. As all my articles have explored, Deep Learning in Medical imaging has exploded with far more efficient and successful approaches being established to extend the input of AI into day to day clinical practice, as we saw with OCT scans at Moorfields NHS Trust and the current collaborative efforts of UCL and DeepMind Technologies. However, with neural networks, you ideally have an infinitely massive data set to enable an equally impressive performance, but this is the issue with AI research that is done in isolation with limited datasets and very simplified algorithms. As NVIDIA puts it, ‘Even when a fully validated model is available, it is a challenge to deploy the algorithm in a local environment. With the latest release of Clara AI for Medical Imaging now data scientists, researchers, and software developers have the necessary tools, APIs and development framework to train and deploy AI workflows.

The toolkit has already seen implementations, such as the American College of Radiology, ACR, who have announced a significant expansion of its AI-LAB pilot program aimed at helping imaging providers develop and AI algorithms to enable an AI-assisted clinical environment. The program’s first two participants were Massachusetts General Hospital in Boston and The Ohio State University in Columbus, but now Lahey Hospital and Medical Center in Burlington, Massachusetts; Emory University in Atlanta; the University of Washington in Seattle; the University of California San Francisco; and Brigham and Women’s Hospital in Boston will all be taking part and contributing to this effort. Each institution will use the ACR AI-LAB’s applications to evaluate the effectiveness of existing AI Algorithms, and altering them as needed and re-evaluating them based on local patient data. One of the pilot program’s defining feature is that no programming knowledge is needed to modify the algorithms, hence improving the accessibility of AI-assistance to a greater cohort of researchers.

The toolkit has already seen implementations, such as the American College of Radiology, ACR, who have announced a significant expansion of its AI-LAB pilot program aimed at helping imaging providers develop and AI algorithms to enable an AI-assisted clinical environment. The program’s first two participants were Massachusetts General Hospital in Boston and The Ohio State University in Columbus, but now Lahey Hospital and Medical Center in Burlington, Massachusetts; Emory University in Atlanta; the University of Washington in Seattle; the University of California San Francisco; and Brigham and Women’s Hospital in Boston will all be taking part and contributing to this effort. Each institution will use the ACR AI-LAB’s applications to evaluate the effectiveness of existing AI Algorithms, and altering them as needed and re-evaluating them based on local patient data. One of the pilot program’s defining feature is that no programming knowledge is needed to modify the algorithms, hence improving the accessibility of AI-assistance to a greater cohort of researchers.

A snapshot of an ACR presentation. Credits: GlassDoor

A snapshot of an ACR presentation. Credits: GlassDoor

NVIDIA’s Clara AI software toolkit is being used within the program, alongside Nuance’s Last Mile Technology being used to integrate various AI-algorithms in the pilot programme, and its success should pave the way for similar applications for more institutions. In fact, Bibb Allen Jr., MD, ACR Data Science Institute (ACR DSI) chief medical officer was quoted to have said “Today marks a major step in accelerating the development of AI for medical imaging, we know algorithms can underperform when deployed at sites where they weren’t trained. Now, radiologists in the pilot program will have access to AI algorithms developed outside their institutions in order evaluate a model’s performance using their own data and, as necessary, retrain the algorithm using their local data to enhance its performance. ACR AI-LAB has kicked off a very exciting era of AI democratization, making it possible for health care institutions and industry to build customized AI models for investigative purposes without coding and without moving image data off-premises,” Keith Dreyer, DO, Ph.D., ACR DSI chief science officer, said in the same statement. “Soon all institutions interested in participating in the AI democratization revolution will have the opportunity to get involved.”

There is also a new project set to build the first AI platform for medical imaging to be implemented under a collaborative effort between NVIDIA, the National Health Service and King’s College London who will run the project that hopes to automate the analysis of radiology data, which is a very consuming in the radiology process. Researchers hope that the long periods spent deciphering radiographs can be reinvested by clinicians in other areas such as teaching. The program will take place within a cohort of clinicians at several major London Hospitals, with KCL researchers and NVIDIA engineers, which should hopefully hit the multi-specialty expertise to execute a program such as this. In fact, Craig Rhodes, Nvidia Industry Business Development lead for AI in Pharmaceutics, Healthcare and Life Sciences in EMEA, had the following to say, “It’s hard to assemble a team of computer scientists, systems engineers, algorithmic and AI researchers, and people with deep understanding of clinical data and clinical systems that can communicate and build this effectively,”. The project system will be based off NVIDIA’s GDX-2 AI Supercomputer, the two petaflop AI accelerator system that utilises a mouth-watering 16 V100 GPUs with 512GB of memory! The system will of course use Nvidia’s Clara AI toolkit alongside NiftyNet’s convolutional neural network platform, and other systems from Kheiron Medical, Mirada, Scan and will all be seen over by KCL. Carl Rhodes again commented “Medical data is 3D, 4D, and in some situations even 5D. Memory becomes a huge bottleneck in these models, and systems that can sync parameters and memory across many GPUs become essential to train these state-of-the-art models. AI supercomputers such as the DGX-2, with its large memory pool, are ideal for key parts of the algorithms KCL is building,” Rhodes said. “[In the future], it will also be interesting to think about mixing DGXs for training, with Nvidia T4 inference specific cards for deployment, and introducing edge computing with Jetson Xavier and Nano system-on-modules in multiple parts of the hospital for data collection and simplification”. As we see with ACR’s program, this initial project’s success will set the path for the distribution of the system to other NHS Trusts.

NVIDIA’s DGX-2 supercomputer, the ‘box’ that can do it all! Credit: NVIDIA

NVIDIA’s DGX-2 supercomputer, the ‘box’ that can do it all! Credit: NVIDIA

On the other hand, Genomics is the study of the genome of an organism and has grounds based upon genetics. A variety of recombinant DNA, bioinformatics (simply high-performance computing and math techniques, although not simple itself!)  and DNA sequencing methods are combined to sequence, assemble, and decipher the structure and processes induced by genomes. As opposed to classical genetics, genomics reviews an organism’s full array of hereditary material, as opposed to one gene or nucleic product in isolation. Also, genomics revolves around interactions between loci and alleles within the genome, and different interactions such as heterosis, pleiotropy and epistasis, and ultimately the field yields the availability of whole DNA sequences for organisms, which has a massive range of implications for humanity; particularly in treating genetic disorders, and was established by Fred Sanger and recent advances in sequencing capacity through the evolution of technology. According to the European Bioinformatics Institute, Fred Sanger's team founded methods of sequencing, genome mapping, data storage, and bioinformatic analyses in the 1970s and 1980s. This work set the path for the human genome project in the 1990s, a field changing study demonstrating a feat of global collaboration that culminated in the publication of the complete human genome sequence in 2003. Today, next-generation sequence technologies have led to equally impressive improvements in the vital speed, capability, and cost-effectiveness of genome sequencing. In addition to this, striving efforts in bioinformatics have enabled hundreds of life-science databases and projects that provide support for scientific research. Information stored and organised in these databases can easily be searched, compared and analysed, seeing how genetic variance brings about disorders and conditions, and could in the future form the basis of treating these issues, which are generally non-communicable diseases and drain the NHS budget year by year, as such conditions generally require life-long treatment.


Clara Genomics, a portion of the NVIDIA Clara developer toolkit, was formed to support the growing complexity and magnitude of genomics sequencing and analysis, with input from accelerated compute methods. According to NVIDIA, ‘The field of Genomics has several transformative trends that put computing at the forefront of progress: increasing instrument throughput, AI-enabled analysis applications and reduction in the cost of sequencing to study large populations. NVIDIA’s GPU Accelerated Computing platform enables real-time genomics workflows with high-performance computing, deep learning, and analytics on a single architecture that lives on the edge in the sequencer to the datacenter and every public cloud.’

A high-level workflow, transitioning from a sample to a final analytic output. Credit: NVIDIA

A high-level workflow, transitioning from a sample to a final analytic output. Credit: NVIDIA

The high-level process that defines the transition from a sample to a final analytic output begins with obtaining the isolated DNA of some organism. This sample is then loaded onto sequence instruments/hardware, where integrated GPUs are used to rapidly accelerate primary analysis and establish the grounds for next-gen ‘base calling’ using deep neural networks, or DNNs for short. Then, secondary analysis, or ‘sequence analysis’, involves using NVIDIA GPU computing in the Genome Analysis Toolkit, or GATK, alongside DNN-based allele/variant calling, and de novo (‘De novo sequencing refers to sequencing a novel genome where there is no reference sequence available for alignment. Sequence reads are assembled as contigs, and the coverage quality of de novo sequence data depends on the size and continuity of the contigs (the number of gaps in the data) – Directly cited from Illumina, check credits below’) genome assembly. This first release of the Clara SDK has been focused on ‘de novo assembly’ of long-read sequencing from Oxford Nanopore and Pacific Biosciences, which should reduce analysis time from days to a matter of hours, again another approach which aims to reduce the time in interpretative efforts, such as samples in this case, but more conventionally scans and observation results in other areas of healthcare.


The Clara Genomics Technology Stack. Credits: NVIDIA

The Clara Genomics Technology Stack. Credits: NVIDIA

 Clara Genomics Technology Stack utilises CUDA accelerated software system subsets s that form the foundation of GPU computing, as in your NVIDIA graphics card currently driving your system. The following bullet-pointed elements are excerpts from NVIDIA directly.

  • CUDA Mapper - CUDA based library enabling algorithms for overlapping sequencing reads.

  • CUDA Aligner - CUDA accelerated library including algorithms for aligning sequencing reads, used for genome assembly applications such as Racon and for variant calling.

  • CUDA POA - CUDA library for accelerated partial order alignment, used for genome assembly polishing with applications such as Racon.

 Clara Genomics Technology Stack utilises CUDA accelerated software system subsets that form the foundation of GPU computing, as in your NVIDIA graphics card currently driving your system.

  • CUDA Mapper - CUDA based library enabling algorithms for overlapping sequencing reads.

  • CUDA Aligner - CUDA accelerated library including algorithms for aligning sequencing reads, used for genome assembly applications such as Racon and for variant calling.

  • CUDA POA - CUDA library for accelerated partial order alignment, used for genome assembly polishing with applications such as Racon.

These system libraries form the compute foundation and enable the GPU acceleration of the Racon Polisher (a subset of the Racon consensus module for genome assembly, that is aimed towards accelerated partial order alignment) and Racon Aligner and Mapper programs. More information can be found in the following link, The Racon Project.

NVIDIA’s Clara SDK is currently available on their developer site for download universally.

·A very big thank you to NVIDIA, for their publication and strenuous contribution to the global healthcare effort via the Clara AI toolkit. Another thank you to all institutions that have been referenced; namely Massachusetts General Hospital in Boston, The Ohio State University in Columbus, Lahey Hospital and Medical Center in Burlington, Massachusetts; Emory University in Atlanta; the University of Washington in Seattle; the University of California San Francisco; and Brigham and Women’s Hospital in Boston, Illumina, and the Racon Project. It was a very interesting area to research, and I thank all these contributory bodies for giving me a fantastic 3 hours of reading - and also setting the path to the future of healthcare!

Thumbnail credit: AIThority (https://aithority.com/technology/big-data/nvidia-launches-technology-center-to-advance-ai-research-across-u-k/)