Deep Learning for Medical Imaging and Radiotherapy

Deep learning (DL) techniques have drawn great interest across many different scientific disciplines. Their potential in healthcare has recently been demonstrated by numerous successful applications in a wide range of medical problems. In our group, we focus on developing novel DL methods to improve the quality of medical imaging and radiotherapy. We are particularly interested in modeling human intelligence and expert’s behaviors to tackle critical tasks that are challenging to conventional machine learning methods.

On the imaging side, our main scope ranges from medical image reconstruction, quality enhancement, and segmentation. We have successfully established a deep reinforcement learning (DRL) framework that intelligently adjusts regularization parameters in iterative reconstruction (IR) for Computed Tomography (CT) in a human-like fashion. This work eases the necessity of manual parameter tuning for IR, which is commonly considered as a main obstacle preventing the implementation of IR in the clinical environment. We have also developed a high-quality low-dose CT reconstruction algorithm that innovatively regularizes the reconstruction process by the manifold prior learned via a deep encoder-decoder network. In addition, we are also working on an automatic DL-based segmentation framework on CT images specifically developed to contour those organs with complex topology and large inter-patient structural variance by mimicking expert’s behaviors in manual segmentation. Furthermore, we are extending developed methods to other modalities, e.g. magnetic resonance imaging (MRI), and contexts, e.g. small animal preclinical research, to serve and contribute to the development of next generation radiotherapy hardware in our Lab.


We also target on automating the radiotherapy treatment planning process, as well as improving the treatment quality by employing novel DL methods. It is our first step to build an intelligent agent that is able to automatically operate a treatment planning system (TPS) to produce high quality plans. As a proof-of-principal study, we successfully established a DRL-based method to learn how to adjust organ weighting factors in a TPS to improve high-dose-rate brachytherapy plan quality for cervical cancer. We are now generalizing the method to external beam intensity modulated radiation therapy. We are also interested in quantifying plan quality in a human-like fashion using DL.

  Recent Publications:
  1. Hyunuk Jung, Yesenia Gonzalez, Chenyang Shen, Peter Klages, Kevin Albuquerque, Xun Jia, "Deep learning assisted automatic digitization of applicators in 3D CT image based high dose rate brachytherapy of gynecological cancer," to appear in Brachytherapy (2019).
  2. Chenyang Shen, Yesenia Gonzalez, Peter Klages, Nan Qin, Hyunuk Jung, Liyuan Chen, Dan Nguyen, Steve B. Jiang, Xun Jia, “Intelligent Inverse Treatment Planning via Deep Reinforcement Learning, a Proof-of-Principle Study in High Dose-rate Brachytherapy for Cervical Cancer,” Phys. Med. Biol.64, 115013 (2019).
  3. Guoyang Ma, Chenyang Shen, Xun Jia, "Low dose CT reconstruction assisted by an image manifold prior," arXiv preprint arXiv:1810.12255 (2018).
  4. Chenyang Shen, Yesenia Gonzalez, Liyuan Chen, Steve Jiang, Xun Jia, "Intelligent Parameter Tuning in Optimization-based Iterative CT Reconstruction via Deep Reinforcement Learning," IEEE Transaction on Medical Imaging, 37 (6), 1430-1439 (2018).
  Funding Support:
  1. “Intelligent treatment planning for cancer radiotherapy,” NIH/NCI 1R01CA237269, 04/2019 - 03/2024.