Prediction of Severe Adverse Drug Reactions Based on Spontaneous Reporting Data-Driven Machine Learning Models

doi:10.19803/j.1672-8629.20250512

Abstract

Abstract: Objective To construct an intelligent model for prediction of severe adverse drug reactions (ADR) based on spontaneous ADR reporting data in order to enhance the efficiency of pharmacovigilance, identify high-risk adverse reactions earlier, and optimize the allocation of healthcare resources. Methods A retrospective analysis was conducted of 4 144 spontaneous ADR reports. The DeepSeek large language model (LLM) was applied to standardize diagnostic information and names of drugs. Twenty-four clinical features were selected to process data on age, time variables, and unordered categories with feature engineering. Ten machine learning algorithms, including Bernoulli Naive Bayes (BNB) and Random Forest (RF), were compared. Besides, the effectiveness of such sampling techniques as SMOTE, ADASYN, and TomekLinks in addressing data imbalance (10.4% severe ADR) was evaluated. Model performance was evaluated with the area under the precision-recall curve (AUPRC) as primary metrics, and the area under the curve (AUC) and true positive rate (TPR) as secondary metrics. Feature contributions were analyzed by using SHAP values. Results The combination of BNB and TomekLinks delivered the best performance. In internal validation, AUC=0.921, AUPRC=0.757, and TPR=0.626 were achieved, compared with AUPRC=0.711 and AUC=0.901 in external validation, suggesting good generalization ability. SHAP analysis revealed that hospitalizations or prolonged hospital stay, age, and immune abnormalities or infections indicated significant positive influence, while insignificant impact on preexisting diseases, damage to the gastrointestinal system and the skin and its accessories, and recovery/improvement were indicators of negative influence. Conclusion Undersampling techniques, particularly TomekLinks, outperform oversampling methods for high-dimensional sparse features. The BNB algorithm, a classic classification method based on Bayes' theorem, continues to excel in classification efficiency among various algorithms. Limitations include potential bias from single-center data and insufficient sample size for severe ADR. A more accurate early warning system should be established by integrating multi-center data, taking molecular features of drugs into consideration and leveraging natural language processing technologies.

Key words: Adverse Drug Reaction, Severity, Prediction Model, Machine Learning, Bernoulli Naive Bayes, DeepSeek

CLC Number:

R994.11

LIU Xi, LI Chen, TIAN Yuan, CHEN Mengli. Prediction of Severe Adverse Drug Reactions Based on Spontaneous Reporting Data-Driven Machine Learning Models[J]. Chinese Journal of Pharmacovigilance, 2025, 22(12): 1410-1417.

Add to citation manager EndNote|Ris|BibTeX

URL: https://zgywjj.magtechjournal.com/EN/10.19803/j.1672-8629.20250512

https://zgywjj.magtechjournal.com/EN/Y2025/V22/I12/1410

References

[1] EDWARDS IR, ARONSON JK.Adverse Drug Reactions: Definitions, Diagnosis, and Management[J]. Lancet, 2000, 356(9237): 1255-1259.
[2] TISSOT M, VALNET-RABIER MB, STALDER T, et al.Epidemiology and Economic Burden of“Serious”Adverse Drug Reactions: Real-World Evidence Research Based on Pharmacovigilance Data[J]. Therapie, 2022, 77(3): 291-300.
[3] ZHAO H, NI P, ZHAO Q, et al.Identifying the Serious Clinical Outcomes of Adverse Reactions to Drugs by a Multi-Task Deep Learning Framework[J]. Commun Biol, 2023, 6(1): 870.
[4] SALVO F, MICALLEF J, LAHOUEGUE A, et al.Will the Future of Pharmacovigilance be More Automated[J]. Expert Opin Drug Saf, 2023, 22(7): 541-548.
[5] LIANG L, HU J, SUN G, et al.Artificial Intelligence-Based Pharmaco-vigilance in the Setting of Limited Resources[J]. Drug Saf, 2022, 45(5): 511-519.
[6] SONG H, PEI X, LIU Z, et al.Pharmacovigilance in China: Evolution and Future Challenges[J]. Br J Clin Pharmacol, 2023, 89(2): 510-522.
[7] PARIENTE A, MICALLEF J, LAHOUEGUE A, et al.What Place for Intelligent Automation and Artificial Intelligence to Preserve and Strengthen Vigilance Expertise in the Face of Increasing Declarations[J]. Therapie, 2023, 78(1): 131-143.
[8] Chinese Hospital CEO.Report on Monitoring of Adverse Drug Reactions Released[J]. Chinese Hospital CEO, 2024, 20(8): 17.
[9] CDR, NMPA. National Annual Report on Monitoring of Adverse Drug Reactions (2024) [EB/OL]. (2025-04-07) [2025-6-30]. https://www.cdr-adr.org.cn/center_news/202504/t20250407_51076.html
[10] LIU H, FENG BL, YANG SM, et al.Investigation and Analysis on the Current Development Status of Provincial Adverse Drug Reaction Monitoring Institutions in China[J]. China Pharmacy(中国药房), 2014, 25(12): 1062-1066.
[11] BARTELS PH, BELLAMY J. Self-Optimizing, Self-Learning System in Pictorial Pattern Recognition[J]. Appl Opt, 1970, 9(11): 2453-2458.
[12] HAUG CJ, DRAZEN JM.Artificial Intelligence and Machine Learning in Clinical Medicine, 2023[J]. N Engl J Med, 2023, 388(13): 1201-1208.
[13] MARTIN GL, JOUGANOUS J, SAVIDAN R, et al.French Network of Pharmacovigilance Centres. Validation of Artificial Intelligence to Support the Automatic Coding of Patient Adverse Drug Reaction Reports, Using Nationwide Pharmacovigilance Data[J]. Drug Saf, 2022, 45(5): 535-548.
[14] WEI CM, GAO F, ZHANG HQ, et al.Application Discussion on the Active Surveillance Module of Adverse Drug Reaction in China's Hospital Pharmacovigilance System[J]. Chinese Journal of Rational Drug Use, 2023, 20(4): 140-146.
[15] YANG H, WEI J, WANG JY, et al.Research on the Evaluation Methods for Adverse Drug Reaction/Event Reports[J]. Chinese Journal of Pharma-covigilance(中国药物警戒), 2009, 6(10): 581-584.
[16] NMPA. Measures for the Administration of Reporting and Monitoring of Adverse Drug Reactions [EB/OL]. (2011-05-04) [2025-09-01]. https://www.nmpa.gov.cn/yaopin/ypfgwj/ypfgbmgzh/20110504162501325.html.
[17] National Cancer Institute. Common Terminology Criteria for Adverse Events (CTCAE) Version 5.0[EB/OL]. (2017-11-27) [2025-09-01]. https://ctep.cancer.gov/protocoldevelopment/electronic_applications/ctc.htm.
[18] KAYAALP ME, PRILL R, SEZGIN EA, et al.DeepSeek Versus ChatGPT: Multimodal Artificial Intelligence Revolutionizing Scientific Discovery. From Language Editing to Autonomous Content Generation-Redefining Innovation in Research and Practice[J]. Knee Surg Sports Traumatol Arthrosc, 2025, 33(5): 1553-1556.
[19] GIBNEY E. Scientists Flock to DeepSeek: How They're Using the Blockbuster AI model[J/OL]. Nature, (2025-01-29)[2025-09-01]. https://pubmed.ncbi.nlm.nih.gov/39881178/.
[20] DENG B, ZHAO Z, RUAN T, et al.Development and External Validation of a Machine Learning Model for Brain Injury in Pediatric Patients on Extracorporeal Membrane Oxygenation[J]. Crit Care, 2025, 29(1): 17.
[21] DEMIRSOY I, KARAIBRAHIMOGLU A.Identifying Drug Interactions Using Machine Learning[J]. Adv Clin Exp Med, 2023, 32(8): 829-838.
[22] HU Q, ZHAO M, TENG F, et al.A Model for Identifying Potentially Inappropriate Medication Used in Older People with Dementia: a Machine Learning Study[J]. Int J Clin Pharm, 2024, 46(4): 937-946.
[23] SALIFU D, CHEPKEMOI L, IBRAHIM EA, et al.Data Augmentation and Machine Learning Algorithms for Multi-Class Imbalanced Morphometrics Data of Stingless Bees[J]. Heliyon, 2025, 11(3): e42214.
[24] IJAZ MF, ATTIQUE M, SON Y.Data-Driven Cervical Cancer Prediction Model with Outlier Detection and Over-Sampling Methods[J]. Sensors (Basel), 2020, 20(10): 2809.
[25] KOSOLWATTANA T, LIU C, HU R, et al.A Self-Inspected Adaptive SMOTE Algorithm (SASMOTE) for Highly Imbalanced Data Classification in Healthcare[J]. BioData Min, 2023, 16(1): 15.
[26] GENG Z, YANG C, ZHAO Z, et al.Development and Validation of a Machine Learning-Based Predictive Model for Assessing the 90-Day Prognostic Outcome of Patients with Spontaneous Intracerebral Hemorrhage[J]. J Transl Med, 2024, 22(1): 236.
[27] PONCE-BOBADILLA AV, SCHMITT V, MAIER CS, et al.Practical Guide to SHAP Analysis: Explaining Supervised Machine Learning Model Predictions in Drug Development[J]. Clin Transl Sci, 2024, 17(11): e70056.
[28] FENG G, ZHOU X, CHEN J, et al.Platinum Drugs-Related Safety Profile: The Latest Five-Year Analysis from FDA Adverse Event Reporting System Data[J]. Front Oncol, 2023, 12: 1012093.
[29] ABRAMSON JS, KU M, HERTZBERG M, et al.Glofitamab Plus Gemcitabine and Oxaliplatin (GemOx) Versus Rituximab-GemOx for Relapsed or Refractory Diffuse Large B-Cell Lymphoma (STARGLO): a Global Phase 3, Randomised, Open-Label Trial[J]. Lancet, 2024, 404(10466): 1940-1954.
[30] HSU HC, CHUNG WH, LIN YC, et al.Clinical Characteristics and Genetic HLA Marker for Patients with Oxaliplatin-Induced Adverse Drug Reactions[J]. Allergol Int, 2024, 73(4): 580-586.
[31] ZHOU ML, DING XM, LV PK.Analysis of Adverse Drug Reactions of Platinum-Based Agents in a Hospital from 2022 to 2023[J]. Life Science Instruments, 2025, 23(1): 177-179.
[32] QIU S, CUI XL, WANG Y.Analysis of Adverse Drug Reactions in 112 Cases of Gemcitabine[J]. Chinese Journal of New Drugs(中国新药杂志), 2014, 23(17): 2085-2088.
[33] WANI MA, ROY KK.Development and Validation of Consensus Machine Learning-Based Models for the Prediction of Novel Small Molecules as Potential Anti-Tubercular Agents[J]. Mol Divers, 2022, 26(3): 1345-1356.
[34] CHATTERJEE D, ROY S, HAZRA A, et al.Variation of Adverse Drug Reaction Profile of Platinum-Based Chemotherapy with Body Mass Index in Patients with Solid Tumors: an Observational Study[J]. Indian J Pharmacol, 2014, 46(2): 222-224.
[35] TRAN TTV, SURYA WIBOWO A, TAYARA H, et al.Artificial Intelligence in Drug Toxicity Prediction: Recent Advances, Challenges, and Future Perspectives[J]. J Chem Inf Model, 2023, 63(9): 2628-2643.
[36] COMBI C, ZORZI M, POZZANI G, et al.From Narrative Descriptions to MedDRA: Automagically Encoding Adverse Drug Reactions[J]. J Biomed Inform, 2018, 4: 184-199.
[37] BERGMAN E, DÜRLICH L, ARTHURSON V, et al. BERT Based Natural Language Processing for Triage of Adverse Drug Reaction Reports Shows Close to Human-Level Performance[J]. PLoS Digit Health, 2023, 2(12): e0000409.