基于多模态深度学习的新型冠状病毒肺炎重症转化风险 预测

doi:10.19871/j.cnki.xfcrbzz.2021.02.013

新发传染病电子杂志 ›› 2021, Vol. 6 ›› Issue (2): 133-137.doi: 10.19871/j.cnki.xfcrbzz.2021.02.013

基于多模态深度学习的新型冠状病毒肺炎重症转化风险预测

李仕康¹, 李卓², 徐瑞卿³, 严晓峰¹, 李建华¹, 吕亮¹, 宋玉燕¹, 孙强中¹, 李同心¹, 钱斓兰⁴, 张英¹

1.重庆市公共卫生医疗救治中心,重庆 400036;
2.重庆邮电大学计算机科学与技术学院,重庆 400065;
3.英国谢菲尔德大学计算机科学系,英国谢菲尔德 S102TN;
4.重庆诚友健康管理有限公司,重庆 400042

收稿日期:2021-02-23 出版日期:2021-05-31 发布日期:2021-06-24
通讯作者: 张英,E-mail：2522935473@qq.com;李仕康,E-mail：417355893@qq.com
基金资助:
重庆市技术创新与应用发展专项重点项目（cstc2020jscx-fyzxX0023）

Multimodal deep learning for predicting COVID-19 patients at high-risk for serious illness

Li Shikang¹, Li Zhuo², Xu Ruiqing³, Yan Xiaofeng¹, Li Jianhua¹, Lyu Liang¹, Song Yuyan¹, Sun Qiangzhong¹, Li Tongxin¹, Qian Lanlan⁴, Zhang Ying¹

1. Chongqing Public Health Center Medical Treatment, Chongqing, 400036, China;
2. School of Computer Science and Technology, Chongqing University of Posts and Telecommunications, Chongqing 400065, China;
3. Department of Computer Science, University of Sheffield, Sheffield UK S102TN;
4. Chongqing Chengyou Health Management Co.Ltd., Chongqing 400042, China

Received:2021-02-23 Online:2021-05-31 Published:2021-06-24

摘要/Abstract

摘要： 目的利用深度学习和大数据的技术来识别潜在新型冠状病毒肺炎重症转化高风险患者,帮助医生及时制订有针对性的救治方案。方法收集整理2020年1月24日到2020年2月16日在重庆市公共卫生医疗救治中心收治的216例新型冠状病毒肺炎患者的全病程多模态（即不同类型）的数据,构建了基于多模态深度学习的评估预测模型,对患者当前的病情严重程度进行评估,并对轻症患者发生重症转化的风险进行预测。结果该模型对患者当前状态的病情评估准确度高于95%,对轻症患者发展成为重症患者的预测准确度高于90%。结论基于多模态深度学习模型比传统线性回归模型预测更准确。同时利用多模态诊疗数据能够对新型冠状病毒肺炎重症转化风险进行准确预测。

关键词: 新型冠状病毒肺炎, 病情评估及预测, 深度学习, 多模态数据融合

Abstract: Objective Using deep learning and big data technology to identify potential high-risk patients with severe conversion of COVID-19, we aim to build a deep learning model so as to identify potential high-risk patients and help doctors make timely individualized treatment plan. Methods From 24 January, 2020 to 16 February, 2020, 216 cases of COVID-19 patients' many modalities data from Chongqing Public Health Center were collected. We build a novel deep learning model for assessing patients' current condition and predicting those who are at high risk for developing severe illness. Results Extensive experimental results show that our model can achieve the over 95% and 90% accuracies for assessment and prediction, respectively. Conclusions The multimodal deep learning model outperforms conventional linear regression model. Meanwhile, Multimodal data can accurately predict the risk of severe transformation of COVID-19.

Key words: Coronavirus disease 2019, Condition assessment and prediction, Deep learning, Ultimodal data fusion

李仕康, 李卓, 徐瑞卿, 严晓峰, 李建华, 吕亮, 宋玉燕, 孙强中, 李同心, 钱斓兰, 张英. 基于多模态深度学习的新型冠状病毒肺炎重症转化风险预测[J]. 新发传染病电子杂志, 2021, 6(2): 133-137.

Li Shikang, Li Zhuo, Xu Ruiqing, Yan Xiaofeng, Li Jianhua, Lyu Liang, Song Yuyan, Sun Qiangzhong, Li Tongxin, Qian Lanlan, Zhang Ying. Multimodal deep learning for predicting COVID-19 patients at high-risk for serious illness[J]. Electronic Journal of Emerging Infectious Diseases, 2021, 6(2): 133-137.

参考文献

[1] ZHU N, ZHANG D, WANG WL, et al.Gao and Wenjie Tan. A Novel Coronavirus from Patients with Pneumonia in China, 2019[J]. N Engl J Med, 2020, 382(8):727-733.
[2] WU F, ZHAO S, YU B, et al.A New Coronavirus Associated with Human Respiratory Disease in China[J]. Nature, 2020, 579(7798):265-269.
[3] HUANG CL, WANGYM, LI XW, et al. Clinical Features of Patients Infected with2019 Novel Coronavirus in Wuhan, China[J]. The Lancet, 395(10223):497-506.
[4] YANGXB, YU Y, XU JQ, et al.Clinical Course and Outcomes of Critically Ill Patients with SARS-CoV-2 Pneumonia in Wuhan, China: A Single-centered, Retrospective, Observational Study[J]. Lancet Respir Med,2020, 8(5): 475-481.
[5] BHATRAJU PK, GHASSEMIEH BJ, NICHOLS M, et al.Covid-19 in Critically Ill Patients in the Seattle Region - Case Series[J]. N Engl J Med, 2020, 382(21):2012-2022.
[6] 宋璐, 曾莹婷, 龚晓明, 等. 新型冠状病毒肺炎影像表现及鉴别诊断[J/CD]. 新发传染病电子杂志,2020, 5(2):83-86.
[7] 郑秋婷, 卢亦波, 谭理连, 等. 新型冠状病毒肺炎临床及影像学研究进展[J/CD]. 新发传染病电子杂志, 2020, 5(2):140-144.
[8] O'HALLORAN KL.Interdependence, Interaction and Metaphor in Multisemiotic Texts[J]. Social Semiotics, 1999, 9(3):317-354.
[9] BENGIO Y, COURVILLE A, VINCENT P.Representation learning: a review and new perspectives[J]. IEEE Trans Pattern Anal Mach Intell, 2013, 35(8):1798-1828.
[10] BALTRUSAITIS T, AHUJA C, MORENCY LP.Multimodal Machine Learning: A Survey and Taxonomy[J]. IEEE Trans Pattern Anal Mach Intell, 2019, 41(2):423-443.
[11] MONTGOMERY DC, PECK EA, VINING GG.Introduction to Linear Regression Analysis[M]. Wiley, 1982: 18-29.
[12] SCHNEIDER A, HOMMEL G, BLETTNER M.Linear Regression Analysis: Part 14 of A Series on Evaluation of Scientific Publications[J]. Dtsch Arztebl Int, 2010, 107(44): 776-782.
[13] FREEDMAN DA.Statistical Models: Theory and Practice[M]. Cambridge University Press, 2005: 112-118.
[14] BROCKWELL PJ, DAVIS R.A. Introduction to Time Series and Forecasting[M]. Springer(2nd ed.), 2002.
[15] ZHANG PG.Time Series Forecasting Using A Hybrid ARIMA and Neural Network Model[J]. Neuro Computing, 2003, 50:159-175.
[16] SILVER D, HUANG A, MADDISON CJ, et al.Mastering the Game of Go with Deep Neural Networks and Tree Search[J]. Nature, 2016, 529(7587):484-489.
[17] SILVER D, SCHRITTWIESER J, SIMONYAN K, et al.Mastering the Game of Go without Human Knowledge[J]. Nature,2017, 550(7676):354-359.
[18] GRANTER SR, BECK AH, PAPKE DJ.AlphaGo, Deep Learning, and the Future of the Human Microscopist[J]. Arch Pathol Lab Med, 2017, 141(5):619-621.
[19] TOPOL EJ.High-performance Medicine: The Convergence of Human and Artificial Intelligence[J]. Nat Med, 2019, 25(1):44-56.
[20] HE JX, BAXTER SL, XU J, et al.The Practical Implementation of Artificial Intelligence Technologies in Medicine[J]. Nat Med, 2019, 25(1):30-36.
[21] GUROVICH Y, HANANI Y, BAR O, at al. Identifying Facial Phenotypes of Genetic Disorders using Deep Learning[J]. Nat Med, 2019, 25(1):60-64.
[22] ZACHI IA, SURAJ K, FRANCISCO LJ, et al.Screening for Cardiac Contractile Dysfunction Sing An Artificial Intelligence-enabled Electrocardiogram[J]. Nat Med, 2019, 25(1):70-74.
[23] HANNUN AY, RAJPURKAR P, HAGHPANAHI M, et al.Cardiologist-level Arrhythmia Detection and Classification in Ambulatory Electrocardiograms Using A Deep Neural Network[J]. Nat Med, 2019, 25(1):65-69.
[24] TAGHANAKI SA, ABHISHEK K, COHEN JP, et al.Deep Semantic Segmentation of Natural and Medical Images: A Review[J]. Artif Intell Rev, 2020,54:137-178.
[25] ESTEVA A, ROBICQUET A, RAMSUNDAR B, et al.A Guide to Deep Learning in Healthcare[J]. Nat Med, 2019, 25(1):24-29.
[26] IOFFE S, SZEGEDY C.Batch normalization: accelerating deep network training by reducing internal covariate shift. In Proceedings of the 32nd International Conference on International Conference on Machine Learning[C]. JMLR.org, 2015, 37:448-456.
[27] LI Z, WANG KJ, LI CC, et al.Multimodal Deep Learning for Solar Irradiance Prediction. In Proceedings of the 2019 International Conference on Internet of Things (iThings) and IEEE Green Computing and Communications (GreenCom) and IEEE Cyber, Physical and Social Computing (CPSCom) and IEEE Smart Data (SmartData)[C]. IEEE, 2019:784-792.
[28] GOODFELLOW I, BENGIO Y, COURVILLE A.Softmax Units for Multinoulli Output Distributions. Deep Learning[M]. MIT Press, 2016: 180-184.

基于多模态深度学习的新型冠状病毒肺炎重症转化风险预测

Multimodal deep learning for predicting COVID-19 patients at high-risk for serious illness

PDF

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 15

编辑推荐

Metrics

本文评价

[1]	杨钟平, 秦圆圆, 周怡宏, 陈耀凯. 新型冠状病毒肺炎合并HIV感染的临床特征及预后的系统文献分析[J]. 新发传染病电子杂志, 2021, 6(2): 113-118.
[2]	刘星, 刘洪艳, 谷野, 王磊石, 田霞, 于红, 侯春阳, 李欣悦, 宋冠男, 王岩. 43例新型冠状病毒感染流行病学与临床特征分析[J]. 新发传染病电子杂志, 2021, 6(2): 119-124.
[3]	邵蔚, 张立, 陈艳, 黄英, 王文慧, 张利君, 沙莉. 医疗失效模式与效应分析在新型冠状病毒肺炎医院感染防控中的应用效果[J]. 新发传染病电子杂志, 2021, 6(2): 129-132.
[4]	黄斌, 高波. 新型冠状病毒肺炎对神经系统的影响[J]. 新发传染病电子杂志, 2021, 6(2): 155-159.
[5]	蓝博文, 唐润辉, 张志艳, 陈康胤. 多层螺旋CT像素指数定量分析在新型冠状病毒肺炎评估中的应用价值研究[J]. 新发传染病电子杂志, 2021, 6(1): 22-26.
[6]	吕秋莹, 梅树江, 张仁利, 吴永胜, 何建凡, 李媛, 彭博, 张振, 逯建华, 高玮, 房师松, 王敬忠, 温莹, 程聪, 邹旋, 夏俊杰, 彭朝琼, 冯铁建. 新型冠状病毒肺炎精准防控体系构建与效果评价[J]. 新发传染病电子杂志, 2020, 5(4): 229-233.
[7]	宋玉燕, 黄长先, 周璐, 张英, 张安, 詹柱, 李群堂, 邱敏. 中西医结合疗法治疗31例重型新型冠状病毒肺炎临床疗效分析[J]. 新发传染病电子杂志, 2020, 5(4): 239-242.
[8]	李倩, 卢洪洲. 新型冠状病毒疫苗研究最新进展[J]. 新发传染病电子杂志, 2020, 5(4): 268-273.
[9]	任坦坦, 陆普选, 邓国防, 付亮, 张培泽. 2020 WHO全球结核报告:全球与中国关键数据分析[J]. 新发传染病电子杂志, 2020, 5(4): 280-284.
[10]	林绿, 刘夏, 吴念宁, 邹俊, 蓝健, 黄金萍, 陈薪宇, 黄妹青. 血清心肌损伤指标变化与新型冠状病毒肺炎重症风险的相关性研究[J]. 新发传染病电子杂志, 2020, 5(3): 159-164.
[11]	朱芸仙, 刘静, 任砚, 吴敏昱, 任荣, 朱怡, 姚文妍, 李敏. 新型冠状病毒肺炎核酸检测阳转阴前后CT影像对比[J]. 新发传染病电子杂志, 2020, 5(3): 183-187.
[12]	吴锋耀, 蒙婞婞, 韦彩云, 刘冬梅. 传染病医院新型冠状病毒肺炎疫情防控与职业健康长效机制建设[J]. 新发传染病电子杂志, 2020, 5(3): 198-202.
[13]	. 新型冠状病毒肺炎放射检查方案与感染防控专家共识（试行第一版）[J]. 新发传染病电子杂志, 2020, 5(2): 65-73.
[14]	卢亦波, 周静如, 莫移美, 宋树林, 刘铁军. 31例新型冠状病毒肺炎临床及CT影像表现初步观察[J]. 新发传染病电子杂志, 2020, 5(2): 79-82.
[15]	宋璐, 曾莹婷, 龚晓明, 张笑春, 鲁植艳. 新型冠状病毒肺炎影像表现及鉴别诊断[J]. 新发传染病电子杂志, 2020, 5(2): 83-86.

基于多模态深度学习的新型冠状病毒肺炎重症转化风险 预测

Multimodal deep learning for predicting COVID-19 patients at high-risk for serious illness

PDF

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 15

编辑推荐

Metrics

本文评价

基于多模态深度学习的新型冠状病毒肺炎重症转化风险预测