Article Text
PDF
Abstract
Objectives Administrative database research is pivotal for developing guidelines in cardiovascular surgery and valvular heart disease. However, validation studies specific to Asia are lacking. This study validated the coding of valvular heart repair and replacement surgeries in Taiwan’s National Health Insurance (NHI) Research Database using International Classification of Diseases, Clinical Modification (ICD-CM) codes.
Methods This retrospective observational study used data from the Chang Gung Research Database between 2015 and 2018, identifying 1171 patients using Taiwanese NHI reimbursement codes. The gold standard was defined as a blinded retrospective review of operation notes. Claims data, including ICD diagnostic codes, ICD procedural codes and NHI supply codes for surgical materials, were validated. Positive predictive values (PPVs) were calculated as the number of true positives divided by the total claims data.
Results The PPVs (95% CI) for aortic valve (AV) surgery aetiologies were as follows: infectious endocarditis (IE), 94.1% (87.6%–97.7%); rheumatic heart disease (RHD), 88.2% (67.3%–97.5%); bicuspid AV, 93.3% (83.3%–98.1%); and degeneration, 91.7% (85.3%–95.8%). For mitral valve surgery, the surgery aetiologies and PPVs were IE, 93.2% (87.9%–96.6%); RHD, 94.9% (88.3%–98.2%); ischaemic mitral regurgitation, 87.5% (73.0%–95.6%); and degeneration, 88.4% (83.9%–92.0%). Surgical types generally exhibited higher PPVs, except for mechanical prostheses (<90%). The accuracy of mechanical prosthesis identification improved with the inclusion of supply codes along with ICD procedural codes.
Conclusions The PPVs for both aetiologies and surgical types of valvular heart disease were generally satisfactory. The inclusion of supply codes for mechanical valve replacements enhanced accuracy but led to fewer eligible patients being included in the sample. This study provides a potentially optimal framework for future research on valve diseases and surgeries using claims databases.
- Valvular heart disease
- Cardiothoracic surgery
- Electronic Health Records
- Cardiac Epidemiology
Data availability statement
The data are available upon reasonable request through contact with the corresponding author.
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
STRENGTHS AND LIMITATIONS OF THIS STUDY
The Chang Gung Research Database contains comprehensive medical records, including diagnosis codes, de-identified personal medical histories, details of medical materials and records of medical interventions.
All medical documents were independently reviewed and assessed by two medical professionals. Any disagreements on cases were resolved through consensus meetings, which established the gold standard for this study.
The validation results are applicable only to the claims database associated with Taiwan’s National Health Insurance Research Database and to studies focusing on cardiac valve disease or surgery.
Introduction
Valvular heart disease is a common medical condition, accounting for more than 10% of cardiovascular surgical interventions in the USA.1 The primary causes of valvular heart disease include rheumatic heart disease (RHD) and degenerative change, followed by infectious endocarditis (IE), bicuspid aortic valve (BAV) abnormality and ischaemic mitral regurgitation (IMR). The epidemiology of valvular heart disease varies between high-income and low-income populations. According to the 2020 American College of Cardiology/American Heart Association Guidelines, surgical intervention is recommended for patients with severe or symptomatic valvular disease, typically involving either valve repair or replacement. Most patients undergoing surgery for aortic regurgitation, aortic stenosis or BAV abnormalities require valve replacement. By contrast, mitral valve (MV) repair is generally preferred over replacement in most cases. Prosthetic valves used in such surgeries are broadly categorised into tissue valves and mechanical valves.
Administrative database research has been instrumental in shaping guidelines for cardiovascular surgery and valvular heart disease.2 In contrast to clinical trials, administrative data are widely used in epidemiological studies because they are inexpensive and easy to obtain.3 However, claims data for valvular heart disease have rarely been validated against the International Classification of Diseases, Ninth Revision, Clinical Modification (ICD-9-CM) and ICD, Tenth Revision, CM (ICD-10-CM), and existing studies have mainly focused on Western populations. Validation outcomes for Asian populations are unavailable in the literature.4–6
In Taiwan, medical centres use ICD-9-CM and ICD-10-CM coding systems for disease classification. Taiwan’s National Health Insurance Research Database (NHIRD) is the most frequently used administrative billing database in the country. Despite the critical need for accurate studies on valvular heart disease, no study to date has validated the NHIRD’s coding for valvular heart repair and replacement surgeries by using ICD-9-CM and ICD-10-CM. Moreover, coding information from ICD-9-CM and ICD-10-CM is often ambiguous or incompatible with conditions encountered in clinical settings. Thus, unvalidated research data are likely to be inaccurate. This study validated the aetiology and surgical classifications for coding valvular heart repair and replacement procedures.
Methods
Data source
This retrospective observational study used data retrieved from the Chang Gung Research Database (CGRD), one of the largest electronic medical databases in Taiwan. The CGRD is part of the Chang Gung Memorial Hospital system, which comprises three tertiary medical centres and four regional medical institutions, serving patients from nearly all districts in Taiwan. The database contains data on over 21.2% of annual outpatient visits and 12.4% of inpatient admissions in Taiwan, including more than 4 000 000 outpatient visits, 200 000 emergency department records and 1 000 000 inpatient admissions annually. The CGRD provides comprehensive electronic medical data, including inpatient and outpatient records, operation notes, personal medical examination data and detailed diagnoses along with diagnostic codes for each patient. Notably, the CGRD includes claims data used for reimbursement under Taiwan’s National Health Insurance (NHI), ensuring its suitability for validating NHI-related data. Further details regarding the CGRD are available in previous studies.7–9
Study population
Patients who underwent isolated cardiac valvular surgery, as identified by Taiwanese NHI reimbursement codes (excluding those with concomitant aortic surgery or coronary artery bypass grafting), were included in this study (online supplemental table 1). Operation notes, discharge diagnoses and other relevant medical records were extracted to validate the NHIRD codes. Two medical professionals independently reviewed these records to establish the gold standard for determining surgical type and aetiology. Any disagreements or controversial cases were referred to a third medical professional and discussed during review meetings. If consensus could not be reached after discussion, the data were classified as incomplete or missing. Initially, 1389 patients who underwent isolated cardiac valvular surgery between 2015 and 2018 were identified. After the exclusion of repeated entries and cases with incomplete or missing data, 1171 patients were included in the final analysis.
Supplemental material
Algorithm for validating aetiology
The location of the cardiac valvular surgery (AV or MV) was first confirmed by reviewing operation notes. Aetiology was determined using claims data, specifically ICD-9-CM diagnostic codes for 2015 and both ICD-9-CM and ICD-10-CM codes for surgeries performed between 2016 and 2018. The assessment of aetiology was restricted to the discharge diagnosis associated with the index admission for cardiac valvular surgery and validated through a review of operation notes. Only patients with identifiable information in both the ICD diagnostic codes and operation notes were included in the analysis. To account for patients with multiple aetiologies, a hierarchical algorithm was applied to prioritise the aetiology. For the AV, the order was as follows: IE, RHD, BAV and degeneration. For the MV, the order was IE, RHD, IMR and degeneration. IMR was defined on the basis of a history of myocardial infarction, percutaneous coronary intervention or coronary artery bypass grafting prior to the index cardiac valvular surgery, with data traceable back to 2001. Diagnostic codes for determining aetiology using claims data are detailed in online supplemental table 2.
Validation of surgical type
The surgical method (repair or replacement), location (AV, MV or tricuspid valve (TV)) and type of prosthesis (mechanical or tissue valve) were determined using inpatient claims data and ICD procedural codes (online supplemental table 3). The surgical type was assessed on the basis of ICD-9-CM procedural codes for 2015 and both ICD-9-CM and ICD-10-CM codes for surgeries performed between 2016 and 2018. In addition to ICD procedural codes, supply codes for surgical materials were incorporated as a secondary validation method for determining the type of surgery. Patients with clearly identifiable surgical methods and valve locations, as verified through operation notes, were included in the analysis.
Statistical analysis
Baseline patient demographics and characteristics are summarised as means with SD for continuous variables and as frequencies with percentages for categorical variables. The positive predictive value (PPV) was calculated by dividing the number of true positives (determined as the gold standard through operation note reviews) by the total number of claims data. CIs for the PPV, treated as a binomial proportion, were calculated using the Clopper-Pearson exact method. A two-tailed p value<0.05 was considered statistically significant. All data analyses were performed using SPSS software (V.26; IBM SPSS, Chicago, Illinois, USA).
Patient and public involvement
Patients and/or the public were not involved in this study.
Results
Between 1 January 1 2015 and 31 December 2018, electronic medical records were retrieved from the CGRD based on reimbursement codes for valve repair or replacement surgery, which served as the baseline inclusion criteria. After the exclusion of duplicate entries, incomplete records and cases with missing data, 1171 patients who underwent valve surgeries were included in the analysis. The baseline demographics and characteristics of these patients are summarised in table 1. The mean age of patients who underwent cardiac valvular surgery was 57 years, and 57.9% of the patients were men. Regarding surgical outcomes, 7.5% of hospitalised patients died following surgery.
Basic demographic and clinical characteristics of patients who underwent cardiac valve surgeries according to claims data
Validation of the aetiology of valvular heart disease
On the basis of the findings of the aetiology study, the final number of identified patients and the PPV with 95% CIs are presented in table 2. Of the 1171 patients who underwent isolated valve surgeries, as identified through Taiwanese NHI reimbursement codes, the validation analysis was restricted to those with both claims data and operation notes available for aetiology assessment. For AV aetiology, claims data were available for 475 patients; however, only 255 (53.7%) patients had complete operation notes sufficient for determining aetiology. For MV aetiology, 584 patients had claims data, and 484 (82.9%) patients had complete operation notes for aetiology determination. The final cohort included 199, 428 and 56 patients who underwent isolated AV, isolated MV surgery and double valve surgery, respectively.
Accuracy of aetiology*
When more than one aetiology was identified for valve surgery, the aetiologies were prioritised hierarchically according to the previously mentioned algorithm, and only one aetiology was designated as the primary cause per surgery. For AV aetiologies, we focused on four types: IE, RHD, BAV and degeneration. The PPV was the highest for IE (80/85, 94.1%; 95% CI 87.6% to 97.7%) and the lowest for RHD (15/17, 88.2%; 95% CI 67.3% to 97.5%). For MV aetiologies, the four categories were IE, RHD, IMR and degeneration. The true positive rate was 94.9% (74/78; 95% CI 88.3% to 98.2%) for RHD and 93.2% (123/132; 95% CI 87.9% to 96.6%) for IE. IMR had a slightly lower true positive rate of 87.5% (28/32; 95% CI 73.0% to 95.6%). Overall, the accuracy of identifying aetiologies based on claims data was high, with no statistically significant differences in PPVs observed among the four aetiologies, as evidenced by overlapping CIs.
Validation of the surgical type of valvular heart disease
Among the 1171 patients who underwent valve surgeries identified through Taiwanese NHI reimbursement codes, the validation analysis was limited to those with available operation notes to assess surgical type and prosthesis. The validation of surgical types did not require information regarding aetiologies. The cohort included 359, 431 and 111 patients who underwent AV replacement, MV and TV surgery, respectively.
To evaluate the effect of the ICD-10 coding system, which was implemented after 2016, patients who underwent surgery in 2015 were separately analysed. These results are illustrated in online supplemental table 4. The outcomes for patients from 2015 to 2018 are summarised in table 3. When the ICD procedural code alone was used, the location of valve surgery demonstrated a PPV above 95%. Nearly all surgical types, regardless of valve location, achieved a PPV above 90%, except for those involving mechanical valves. In addition, the effect of supply codes, which correspond to the surgical materials used, on the accuracy of surgical type identification was assessed. Incorporating supply codes generally increased PPVs but reduced the denominator due to stricter inclusion criteria. For AV replacement using a mechanical valve, the PPV increased from 82.4% (95% CI 71.2% to 90.5%) to 100% (95% CI 93.7% to 100%). Similarly, for MV replacement with a mechanical valve, the PPV improved from 82.4% (95% CI 71.2% to 90.5%) to 100% (95% CI 93.5% to 100%). Although a direct statistical comparison of PPVs with and without supply codes was not feasible, the non-overlapping CIs were notable. However, the patient number decreased from 68 to 56 for AV replacement and from 68 to 55 for MV replacement when supply codes were included.
Accuracy of surgical type*
Discussion
We evaluated the aetiology and surgical type of valvular heart disease in the NHIRD by using data from the CGRD. Reimbursement codes used for population identification are integral to billing and claims processing within the healthcare system, facilitating payment requests from insurance companies or government programmes for medical services. These codes are subject to rigorous oversight to prevent fraud and ensure that claims accurately reflect the services provided, thereby minimising the likelihood of errors. For AV aetiologies, four categories were validated: IE, RHD, BAV and degeneration. The PPV was the highest for IE (94.1%), followed by BAV (93.3%), degeneration (91.7%) and RHD (88.2%). For MV aetiologies, the order differed, with RHD showing the highest PPV (94.9%), followed by IE (93.2%), degeneration (88.4%) and IMR (87.5%). Furthermore, the addition of supply codes, particularly in mechanical valve cases, improved the PPV to above 95%, further enhancing the accuracy of the data.
Aetiology
The validation of aetiology was based on ICD-9-CM and ICD-10-CM diagnostic codes documented in discharge notes. For cases where multiple codes related to valvular heart disease aetiology were present, clinicians reviewed operation notes and discharge diagnoses to determine the definitive aetiology for each valve surgery. The selection of a single aetiology was guided by the algorithm described in the Methods section. Although cardiac valve surgery often involves multiple diagnoses, these do not necessarily hold equal clinical importance. One diagnosis typically serves as the primary indication for surgery, determined by the relative importance of each aetiology and the primary reason necessitating surgical intervention. For instance, IE, a severe acute infectious disease with a substantial impact on prognosis, is considered a clear and urgent indication for cardiac valve surgery. When multiple diagnoses were present, the prognostic impact of IE was prioritised over other conditions. For both the AV and MV, IE was designated as the primary indication for valve repair or replacement when vegetation or a confirmed case of endocarditis was described. RHD, a globally prevalent cardiovascular condition with considerable clinical implications, was another key aetiology. In Taiwan, specific diagnostic codes have been established for RHD, which often affects multiple heart valves. RHD was identified as the primary aetiology in most patients presenting with mixed aortic stenosis or regurgitation along with MV disorders. For AV interventions, BAV was selected as the primary aetiology in patients with relevant descriptions of AV morphology when no other higher-priority aetiologies were present. Similarly, IMR was chosen as the main aetiology for MV surgeries if chordae rupture was noted or if the patient had a documented history of myocardial infarction, percutaneous coronary intervention or coronary artery bypass graft surgery. Degeneration was selected as the primary aetiology in cases where other conditions were absent, often characterised by findings of valve calcification or leaflet prolapse.
Surgical type
We validated the accuracy of the ICD coding system. The results demonstrated acceptable PPVs for all surgical locations and types, as verified using electronic medical records in the CGRD. The use of ICD procedural codes alone ensured accurate results; however, PPVs were lower for mechanical valve replacements than for other surgical types. This limitation was significantly mitigated when supply codes were introduced as a secondary authentication measure. The materials for mechanical valves are highly specific, making them easier to identify accurately within the coding system. The improvement in accuracy was particularly notable for mechanical valve replacements in both the AV and MV. The rationale for incorporating supply codes is the high cost of these materials, which contributes to a very low error rate in their coding. However, the increased accuracy came at the cost of a decreased eligible population due to the more stringent inclusion criteria. For example, the patient number decreased from 68 to 56 for AV replacements and from 68 to 55 for MV replacements involving mechanical valves. This reduction in sample size may limit the generalisability of findings and constrain the broader applicability of supply codes. Despite these limitations, the use of supply codes shows considerable promise for enhancing the reliability of data for future research. These findings strongly encourage the adoption of supply codes in studies focusing on mechanical valve replacements to achieve more rigorous results. As one of the key strengths of the NHIRD, proper utilisation of supply codes could improve the accuracy and utility of administrative database research.
After 2016, the coding system integrated both ICD-9 and ICD-10 rules. However, the analysis of data from 2015 did not reveal any significant differences.
Limitations
Both the NHIRD and CGRD are widely used in Taiwan, but clinical practices and healthcare systems vary by country. Thus, the validation findings and recommendations from this study may not be directly applicable to other healthcare systems, highlighting the need for further research on additional databases. Certain limitations in the data were noted, including repeated, missing or incomplete records. For instance, incomplete or missing electronic medical notes occasionally prevented consensus on aetiology and surgical type, which may have affected the study results. In addition, the transition from the ICD-9 to the ICD-10 system introduced potential inconsistencies during the overlap period because clinical workers required time to adapt to the updated coding standards. This study spans the transition period, and the findings may not be representative of other timeframes with a more stable coding environment. Another challenge was distinguishing the aetiology of the TV based on operation notes. This limitation likely arises because the TV is often overlooked by surgeons, making accurate classification more difficult.
Conclusion
The validation results indicated generally satisfactory PPVs for both the aetiologies and surgical types of valvular heart disease. For surgical types, ICD procedural codes alone were found to be sufficiently accurate. However, incorporating supply codes for mechanical valve replacements was recommended to further enhance accuracy, but it reduced the available patient sample size. This study provides a potentially optimal framework for conducting research based on claims databases, such as Taiwan’s NHIRD, particularly for studies on cardiac valve diseases or surgeries.
Data availability statement
The data are available upon reasonable request through contact with the corresponding author.
Ethics statements
Patient consent for publication
Ethics approval
This study was approved by the Chang Gung Medical Foundation Institutional Review Board (IRB No. 202 100 124B0). The requirement for individual consent was waived because the database does not include any personal identification information.
Acknowledgments
This study used data from the CGRD provided by the Chang Gung Memorial Hospital administration. However, the interpretation and conclusions presented in this study are solely those of the authors. The authors thank the Maintenance Project of the Center for Big Data Analytics and Statistics (Grant CLRPG3D0049) at Chang Gung Memorial Hospital for their support with statistical consultation and data analysis. The authors also thank Alfred Hsing-Fen Lin for his assistance with the statistical analysis. This manuscript was edited by Wallace Academic Editing.
Footnotes
B-CH and J-TH contributed equally.
Contributors Conception and design: B-CH, J-TH, VC-CW, P-HC and S-WC. Administrative support: C-YC, F-CC and Y-TC. Provision of study materials or patients: A-HC and S-WC. Collection and assembly of data: B-CH, J-TH and Y-TC. Data analysis and interpretation: B-CH, J-TH, C-PL and Y-HC. Manuscript writing: All authors. Final approval of manuscript: All authors. Guarantor: S-WC.
Funding This work was supported by a grant from Chang Gung Memorial Hospital, Taiwan CMRPG3P0801, CORPG3P0511, CORPG3P0541, CORPG3N0281, CORPG3M0371, BMRPD95 (SWC). This work was also supported by the National Science and Technology Council grant NSTC-112-2314-B-182A-107, NSTC-113-2314-B-182A-087 (SWC).
Competing interests None declared.
Patient and public involvement Patients and/or the public were not involved in the design, or conduct, or reporting, or dissemination plans of this research.
Provenance and peer review Not commissioned; externally peer reviewed.
Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.