Suitability assessment of real-world data in active post-marketing surveillance of vaccine safety

doi:10.19803/j.1672-8629.20230308

Abstract

Abstract: Objective To sort out the suitability assessment content and methods of real-world data(RWD) in active post-marketing surveillance of vaccine safety, and preliminarily build a suitability assessment framework that meets the characteristics of RWD in China. Methods The scoping review was used to retrieve and screen studies and guideline documents related to the suitability assessment of RWD in active post-marketing surveillance of vaccine safety, extract and map suitability assessment indicators and content, and combine the framework synthesis to preliminarily determine the hierarchical suitability assessment structure. Results 51 articles were included, including 8(15.7%) in China. 229 assessment indicators were extracted to form a framework for suitability assessment of RWD in active post-marketing surveillance of vaccine safety, including 3 primary indicators(data traceability, data reliability, and data relevance), 8 second-level indicators (basic characteristics of data sources, data collection, data governance, completeness, accuracy, data content, data coverage, and data characteristics), and 31 third-level indicators. Accuracy of key variables(60.8%), population coverage(45.1%), timeliness(39.2%), and completeness of key variables(35.3%) were the third-level indicators with the highest frequency in the included articles. The distribution of the third-level indicators varied according to the type of RWD, vaccine, and health system. Conclusion There is still a lack of standardized RWD suitability assessment dimensions and methods in the active surveillance of vaccine safety. The suitability assessment framework summarized in this study provides a reference for relevant studies, but expert validation is still needed in the future to ensure the scientific nature of the indicator system and thus promotes the construction of the active surveillance system of vaccine safety in China.

Key words: real-world data, vaccine, post-marketing, active surveillance, data, suitability, evaluations

CLC Number:

R954
R994.11

WU Yunxiao, YU Yuelin, WANG Shengfeng, SUN Feng, MENG Ruogu, ZHAN Siyan. Suitability assessment of real-world data in active post-marketing surveillance of vaccine safety[J]. Chinese Journal of Pharmacovigilance, 2023, 20(10): 1081-1089.

References

[1] PERTWEE E, SIMAS C, LARSON HJ.An epidemic of uncertainty: rumors, conspiracy theories and vaccine hesitancy[J]. Nat Med, 2022, 28(3): 456-459.
[2] THE L.Vaccine scandal and confidence crisis in China[J]. Lancet, 2018, 392(10145): 360.
[3] GUZMAN J, CONNELL E, KIKULE K, et al.The WHO global benchmarking tool: a game changer for strengthening national regulatory capacity[J]. BMJ Global Health, 2020, 5(8): e003181.
[4] CHODANKAR D.Introduction to real-world evidence studies[J]. Perspect Clin Res, 2021, 12(3): 171-174.
[5] HUANG YL, MOON J, SEGAL JB.A comparison of active adverse event surveillance systems worldwide[J]. Drug Saf, 2014, 37(8): 581-596.
[6] BAGGS J, GEE J, LEWIS E, et al.The vaccine safety datalink: a model for monitoring immunization safety[J]. Pediatrics, 2011, 127(Suppl 1): 45-53.
[7] SCHEIFELE DW, HALPERIN SA, GOLD R, et al.Assuring vaccine safety: a celebration of 10 years of progress with the IMPACT project[J]. Paediatr Child Health, 2002, 7(9): 645-648.
[8] XU L, LI N, ZHANG L, et al.Febrile seizures and measles-containing vaccines in China: a self-controlled case series study[J]. Vaccines (Basel), 2021, 9(10): 1073.
[9] LIU Z, ZHANG L, YANG Y, et al.Active surveillance of adverse events following human papillomavirus vaccination: feasibility pilot study based on the regional health care information platform in the city of Ningbo, China[J]. J Med Internet Res, 2020, 22(6): e17446.
[10] SHANG N, WENG C, HRIPCSAK G.A conceptual framework for evaluating data suitability for observational studies[J]. J Am Med Inform Assoc, 2018, 25(3): 248-258.
[11] FDA. Framework for FDA's Real-world evidence program[EB/OL] (2018-12-07)[2022-12-06]. https://www.fda.gov/media/120060/download.
[12] NICE. Real-world evidence framework[EB/OL]. (2022-06-23)[2023-01-19]. https://www.nice.org.uk/corporate/ecd9/chapter/overview.
[13] Center For Drug Evaluation, National Medical Products Administration. Guidelines (Trial Implementation) for Real World Data Used to Generate Real World Evidence(国家药监局药审中心关于发布《用于产生真实世界证据的真实世界数据指导原则(试行)》的通告)[EB/OL]. (2021-04-13) [2022-12-06]. https://www.cde.org.cn/main/news/viewInfoCommon/2a1c437ed54e7b838a7e86f4ac21c539.
[14] ARKSEY H, O'MALLEY L. Scoping studies: towards a methodological framework[J]. International Journal of Social Research Methodology, 2005, 8(1): 19-32.
[15] CAI T, LIU LL, YAO XY, et al.Scoping review of active surveillance systems for vaccine safety world-wide[J]. Chin J Prev Med(中华预防医学杂志), 2019, 53(7): 724-730.
[16] CARROLL C, BOOTH A, COOPER K.A worked example of “best fit” framework synthesis: a systematic review of views concerning the taking of some potential chemopreventive agents[J]. BMC Med Res Methodol, 2011, 11: 29.
[17] CHEN L, WEI JJ, DENG ZJ, et al. Study on the safety evaluation of the use of EV71 inactivated vaccine among the population in Wenzhou[R]. Wenzhou Center for Disease Prevention and Control in Wenzhou, 2020-06-10.
[18] ZENG J, TANG T, WANG YJ, et al.Post-marketing multi-center safety surveillance of inactivated enterovirus A71 vaccine (Vero cell)[J]. Chin J Prev Med(中华预防医学杂志), 2019, 53(3): 252-257.
[19] SHEN LZ, FU Z, PAN XJ, et al.Post-marketing observation on safety of inactivated enterovirus A71 vaccine (human diploid cell)[J]. Chin J Prev Med(中华预防医学杂志), 2019, 53(3): 258-261.
[20] PANG H, JIANG Y, ZHANG ZS, et al.Safety monitoring of influenza A (H1N1) vaccination in a district of Shanghai[J]. Journal of Environmental & Occupational Medicine(环境与职业医学), 2011, 28(12): 735-737.
[21] CHEN HY, HUANG T, YANG QH.Safety evaluation of 23-valent pneumococcal polysaccharide vaccine's large-scale use in Hubei after listing[J]. Chin J Public Health Management(中国公共卫生管理), 2019, 35(3): 324-327.
[22] LI JP, LI RF, L Q, et al. Safety of vaccine for A /H1N1 influenza split virus[J]. J Prev Med Inf(预防医学情报杂志), 2013, 29(7): 572-574.
[23] ZHANG WJ, SHI NM, ZHANG YL, et al.Safety of a homemade inactivated enterovirus 71 vaccine in a large population of 6-71 months old children[J]. Chin J Public Health(中国公共卫生), 2019, 35(4): 385-387.
[24] ALBOGAMI Y, ALKOFIDE H, ALRWISAN A.COVID-19 vaccine surveillance in saudi arabia: opportunities for real-time assessment[J]. Saudi Pharm J, 2021, 29(8): 914-916.
[25] ALI M, CANH DG, CLEMENS JD, et al.The vaccine data link in Nha Trang, Vietnam: a progress report on the implementation of a database to detect adverse events related to vaccinations[J]. Vaccine, 2003, 21(15): 1681-1686.
[26] ALI M, CANH GD, CLEMENS JD, et al.The use of a computerized database to monitor vaccine safety in Viet Nam[J]. Bull World Health Organ, 2005, 83(8): 604-610.
[27] BAKER MA, BAER B, KULLDORFF M, et al.Kawasaki disease and 13-valent pneumococcal conjugate vaccination among young children: A self-controlled risk interval and cohort study with null results[J]. PLoS Med, 2019, 16(7): e1002844.
[28] BAKER MA, NGUYEN M, COLE DV, et al.Post-licensure rapid immunization safety monitoring program (PRISM) data characterization[J]. Vaccine, 2013, 31(Suppl 10): K98-K112.
[29] BERRUETA M, CIAPPONI A, BARDACH A, et al.Maternal and neonatal data collection systems in low- and middle-income countries for maternal vaccines active safety surveillance systems: a scoping review[J]. BMC Pregnancy Childbirth, 2021, 21(1): 217.
[30] BROWN JS, MOORE KM, BRAUN MM, et al.Active influenza vaccine safety surveillance: potential within a healthcare claims environment[J]. Med Care, 2009, 47(12): 1251-1257.
[31] CARMONA A, LATORRE TM, MARTINEZ SA, et al.Risk measurement of perinatal and neonatal morbidity characteristics and applicability of gaia case definitions: results and lessons learnt of a hospital-based prospective cohort study in the valencia region (2019-2020)[J]. Int J Environ Res Public Health, 2022, 19(12): 7132.
[32] CHEN RT, GLASSER JW, RHODES PH, et al.Vaccine safety datalink project: a new tool for improving vaccine safety monitoring in the United States. The Vaccine Safety Datalink Team[J]. Pediatrics, 1997, 99(6): 765-773.
[33] CRAWFORD NW, CLOTHIER H, HODGSON K, et al.Active surveillance for adverse events following immunization[J]. Expert Rev Vaccines, 2014, 13(2): 265-276.
[34] DAVILA JC, PAYNE DC, ZHANG Y, et al.Quality assessment of nonanthrax vaccination data in the Defense Medical Surveillance System (DMSS), 1998-2004[J]. Vaccine, 2008, 26(12): 1577-1584.
[35] DE LUSIGNAN S, DOS SANTOS G, BYFORD R, et al.Enhanced safety surveillance of seasonal quadrivalent influenza vaccines in English primary care: interim analysis[J]. Adv Ther, 2018, 35(8): 1199-1214.
[36] GHOSH RE, CRELLIN E, BEATTY S, et al.How clinical practice research datalink data are used to support pharmacovigilance[J]. Ther Adv Drug Saf, 2019, 10: 2042098619854010.
[37] GOLD M, DUGDALE S, WOODMAN RJ, et al.Use of the Australian childhood immunisation register for vaccine safety data linkage[J]. Vaccine, 2010, 28(26): 4308-4311.
[38] GUEDEL DS, PETERS IJ, BANDERET F, et al.Smartphone-based active vaccine safety surveillance (SmartVax) at a Swiss adult vaccination clinic - a pilot study[J]. Swiss Med Wkly, 2021, 151(49-50): w30090.
[39] SHARAN A, JAHAGIRDAR S, STUURMAN AL, et al.Operational lessons learned in conducting an international study on pharmacovigilance in pregnancy in resource-constrained settings: the WHO Global Vaccine safety Multi-Country collaboration project[J]. Vaccine X, 2022, 11: 100160.
[40] HABER P, ISKANDER J, WALTON K, et al.Internet-based reporting to the vaccine adverse event reporting system: a more timely and complete way for providers to support vaccine safety[J]. Pediatrics, 2011, 127(Suppl 1): 39-44.
[41] HECHTER RC, QIAN L, SY LS, et al.Secular trends in diagnostic code density in electronic healthcare data from health care systems in the Vaccine Safety Datalink project[J]. Vaccine, 2013, 31(7): 1080-1085.
[42] HEIJBEL H, JEFFERSON T.Vaccine safety-improving monitoring[J]. Vaccine, 2001, 19(17-19): 2457-2460.
[43] LEITE A, THOMAS SL, ANDREWS NJ.Do delays in data availability limit the implementation of near real-time vaccine safety surveillance using the Clinical Practice Research Datalink?[J]. Pharmacoepidemiol Drug Saf, 2018, 27(1): 25-29.
[44] MCCARTHY NL, GEE J, WEINTRAUB E, et al.Monitoring vaccine safety using the vaccine safety datalink: utilizing immunization registries for pandemic influenza[J]. Vaccine, 2011, 29(31): 4891-4896.
[45] MESFIN YM, CHENG A, LAWRIE J, et al.Use of routinely collected electronic healthcare data for postlicensure vaccine safety signal detection: a systematic review[J]. BMJ Glob Health, 2019, 4(4): e001065.
[46] MESFIN YM, CHENG AC, ENTICOTT J, et al.Use of telephone helpline data for syndromic surveillance of adverse events following immunization in Australia: a retrospective study, 2009 to 2017[J]. Vaccine, 2020, 38(34): 5525-5531.
[47] MULLOOLY J, DREW L, DESTEFANO F, et al.Quality of HMO vaccination databases used to monitor childhood vaccine safety. vaccine safety datalink team[J]. Am J Epidemiol, 1999, 149(2): 186-194.
[48] NARWANEY KJ, BRESLIN K, ROSS CA, et al.Vaccine adverse events in a safety net healthcare system and a managed care organization[J]. Vaccine, 2017, 35(9): 1335-1340.
[49] NASH JQ, CHANDRAKUMAR M, FARRINGTON CP, et al.Feasibility study for identifying adverse events attributable to vaccination by record linkage[J]. Epidemiol Infect, 1995, 114(3): 475-480.
[50] NELSON JC, WELLMAN R, YU O, et al.A synthesis of current surveillance planning methods for the sequential monitoring of drug and vaccine adverse effects using electronic health care data[J]. EGEMS (Wash DC), 2016, 4(1): 1219.
[51] NGUYEN M, BALL R, MIDTHUN K, et al.The Food and Drug Administration's post-licensure rapid immunization safety monitoring program: strengthening the federal vaccine safety enterprise[J]. Pharmacoepidemiol Drug Saf, 2012, 21(Suppl 1): 291-297.
[52] PAYNE DC, ROSE CE, JR, ARANAS A, et al.Assessment of anthrax vaccination data in the Defense Medical Surveillance System, 1998-2004[J]. Pharmacoepidemiol Drug Saf, 2007, 16(6): 605-611.
[53] PLATT R, CARNAHAN RM, BROWN JS, et al.The U.S. Food and Drug Administration's mini-sentinel program: status and direction[J]. Pharmacoepidemiol Drug Saf, 2012, 21(Suppl 1): 1-8.
[54] REYBURN R, NAND D, NGUYEN C, et al.Validation of administrative data to estimate vaccine impact: audit of the Fiji hospital admissions electronic database, 2007-2011 & 2014-2015[J]. Vaccine, 2017, 35(47): 6416-6421.
[55] SCHWARTZ KL, TU K, WING L, et al.Validation of infant immunization billing codes in administrative data[J]. Hum Vaccin Immunother, 2015, 11(7): 1840-1847.
[56] GUILLARD-MAURE C, ELANGO V, BLACK S, et al.Operational lessons learned in conducting a multi-country collaboration for vaccine safety signal verification and hypothesis testing: the global vaccine safety multi country collaboration initiative[J]. Vaccine, 2018, 36(3): 355-362.
[57] STUURMAN AL, SHARAN A, JAHAGIRDAR S, et al.WHO global vaccine safety multi-country collaboration project on safety in pregnancy: assessing the level of diagnostic certainty using standardized case definitions for perinatal and neonatal outcomes and maternal immunization[J]. Vaccine X, 2021, 9: 100123.
[58] THOMAS RE, LORENZETTI DL, SPRAGINS W, et al.Active and passive surveillance of yellow fever vaccine 17D or 17DD-associated serious adverse events: systematic review[J]. Vaccine, 2011, 29(28): 4544-4555.
[59] DUSZYNSKI KM, STARK JH, COHET C, et al.Suitability of databases in the Asia-Pacific for collaborative monitoring of vaccine safety[J]. Pharmacoepidemiol Drug Saf, 2021, 30(7): 843-857.
[60] TOP KA, CONSTANTINESCU CM, LAFLECHE J, et al.Applicability of the brighton collaboration case definition for seizure after immunization in active and passive surveillance in Canada[J]. Vaccine, 2013, 31(48): 5700-5705.
[61] TOP KA, MACARTNEY K, BETTINGER JA, et al.Active surveillance of acute paediatric hospitalisations demonstrates the impact of vaccination programmes and informs vaccine policy in Canada and Australia[J]. Euro Surveill, 2020, 25(25): 1900562.
[62] TRIFIRO G, COLOMA PM, RIJNBEEK PR, et al.Combining multiple healthcare databases for postmarketing drug and vaccine safety surveillance: why and how?[J]. J Intern Med, 2014, 275(6): 551-561.
[63] VERSTRAETEN T, DESTEFANO F, CHEN RT, et al.Vaccine safety surveillance using large linked databases: opportunities, hazards and proposed guidelines[J]. Expert Rev Vaccines, 2003, 2(1): 21-29.
[64] WATSON G, DODD C, MUNOZ FM, et al.Applicability of the GAIA maternal and neonatal outcome case definitions for the evaluation of adverse events following vaccination in pregnancy in high-income countries[J]. Pediatr Infect Dis J, 2021, 40(12): 1127-1134.
[65] FDA. Use of real-world evidence to support regulatory decision-making for medical devices-guidance for industry and Food and Drug Administration staff[EB/OL]. (2018-09-17)[2022-12-06]. https://www.fda.gov/regulatory-information/search-fda-guidance-documents/use-real-world-evidence-support-regulatory-decision-making-medical-devices.
[66] FDA. Real-world data: assessing electronic health records and medical claims data to support regulatory decision-making for drug and biological products[EB/OL]. (2021-12-10)[2022-12-06]. https://www.fda.gov/regulatory-information/search-fda-guidance-documents/real-world-data-assessing-electronic-health-records-and-medical-claims-data-support-regulatory.
[67] LIU D, WU W, LI K, et al.Surveillance of adverse events following immunization in China: Past, present, and future[J]. Vaccine, 2015, 33(32): 4041-4046.
[68] YU YL, ZHUO L, MENG RG, et al.Methodology progress and challenges on assessing the appropriateness of real-world data[J]. Chin J Epidemiol(中华流行病学杂志), 2022, 43(4): 578-585.
[69] ZHANG HZ, LIU MM, LI L, et al.Study on data governance of the healthcare big data platform based on common data model[J]. J Med Inf(医学信息学杂志), 2022, 43(6): 2-7, 13.
[70] BIAN J, LYU T, LOIACONO A, et al.Assessing the practice of data quality evaluation in a national clinical data research network through a systematic scoping review in the era of real-world data[J]. Journal of the American Medical Informatics Association, 2020, 27(12): 1999-2010.