Article Text
PDF
Abstract
Objectives To evaluate the effectiveness of 18 different host biomarkers in differentiating bacterial from non-bacterial acute febrile illness (AFI) in resource-limited settings, specifically in Brazil, Malawi and Gabon.
Design Multinational, cross-sectional study.
Setting The study was carried out across multiple primary healthcare facilities, including urban and rural settings, with a total of three participating centres. Recruitment took place from October 2018 to July 2019 in Brazil, May to November 2019 in Gabon and April 2017 to April 2018 in Malawi.
Participants A total of 1915 participants, including children and adults aged 21–65 years with a fever of≤7 days, were recruited through convenience sampling from outpatient clinics in Brazil, Gabon and Malawi. Individuals with signs of severe illness were excluded. Written consent was obtained from all participants or their guardians.
Intervention This is not applicable as the study primarily focused on biomarker evaluation without specific therapeutic interventions.
Primary and secondary outcome measures The primary outcome measure was the ability of each host biomarker to differentiate between bacterial and non-bacterial AFI, as evaluated by area under the receiver operating characteristic (AUROC) curves. Secondary outcomes included the performance of individual biomarkers across the different study sites and in a multivariable setting.
Results A Kruskal-Wallis test, adjusted by Benjamini-Hochberg, was performed for each biomarker to identify covariates with a significant difference in the distribution of biomarker values. The analysis revealed that country of origin (Brazil, Gabon, Malawi), age, sex and malaria status significantly impacted biomarker distribution (p≤0.001). The most widely known biomarkers, such as white blood cell (WBC) count and C-reactive protein (CRP), demonstrated the best performance in distinguishing between bacterial and non-bacterial infections, with AUROCs reaching up to 0.83 (0.77–0.88) for WBC count and 0.71 (0.59–0.82) for CRP. However, none of the evaluated novel host biomarkers exhibited high performance (AUROC<0.70 in most cases) and variations in biomarker performance were observed across the three settings. Multivariable analyses demonstrated that while the best combination of biomarkers achieved higher AUROCs, the increase was modest (1–13%), suggesting that the interaction of biomarkers contributed minimally to predictive accuracy.
Conclusions There is a continued need for innovation in the host-biomarker space as the available markers do not meet the needs of diverse populations around the globe. This highlights the importance of targeted evaluations in non-severe patients in multiple settings to understand the true potential for real-life use. The findings highlight that not one-marker fits all settings and novel innovations remain urgently needed.
Trial registration number Clinical trial number: NCT03047642.
- Public health
- Malaria
- Anti-Bacterial Agents
Data availability statement
Data are available upon reasonable request. Data are available upon request from the study sites (Malawi, Brazil, Gabon) or the study sponsor (FIND). All data were shared with the local teams to allow further analysis and investigations and acknowledge ownership.
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
STRENGTHS AND LIMITATIONS OF THIS STUDY
Diverse evaluation: This study is an extensive evaluation of 18 host biomarkers across low- and middle-income countries to differentiate bacterial from non-bacterial infections.
Methodological alignment: The study protocol aligns with Food and Drug Administration-approved classifications for distinguishing between bacterial and non-bacterial infections, enhancing methodological rigour.
No control group: The absence of a control group limits the ability to establish baseline biomarker performance or to assess asymptomatic carriers.
Time and geographical variability: The short enrolment period and heterogeneity of acute febrile illness causes may limit the generalisability of findings across different times and geographical contexts, particularly in Asia.
Subjectivity in classification: The two-step clinical classification process may introduce subjectivity, particularly as clinicians had access to haematology biomarker results during classification, potentially biasing results.
Introduction
Globally, acute febrile illness (AFI) is one of the leading reasons individuals, particularly children aged less than 5 years, present to primary healthcare facilities.1 AFI has various causes, both infectious and non-infectious, that vary according to geography, age group and season.1 In malaria-endemic settings, malaria was long considered the primary cause of all fevers; however, the introduction of rapid diagnostic tests (RDTs) for malaria in the past decade has disproved this. Modelling estimates suggest that approximately 70% of all fevers can be attributed to non-malarial causes, even in malaria-endemic settings.2 In the Integrated Management of Childhood Illness, introduced by the WHO and UNICEF in the mid-1990s and subsequently implemented in more than 100 countries, the standard ‘fever’ algorithm currently includes a malaria RDT but no diagnostic test for other infections.3 Hence, at the primary care level, the only evidence-based treatment decision that can be made relies on the malaria RDT, resulting in extremely high levels of antibiotic use in malaria-negative patients.4 In this context of limited knowledge about the causes of AFI and limited diagnostic and human capacity, it is unsurprising that healthcare providers prescribe antibiotics to avoid negative outcomes in their patients.
To assist healthcare providers with clinical decision-making, a simple diagnostic tool is required to differentiate patients with AFI of bacterial and non-bacterial aetiology and provide appropriate care. In well-resourced settings, in both high-income countries (HICs) and low- and middle-income countries (LMICs), some non-specific host biomarkers are used for this purpose, most frequently C-reactive protein (CRP) and procalcitonin (PCT), although these biomarkers are less useful in settings with a higher frequency of comorbidities.5 Thus, in 2015, an international group of experts was convened to define the target product profile (TPP) of such a tool, specifically for low-resource settings, to guide product development and implementation as part of integrated treatment management guidelines.6 Since then, the ongoing viral pandemic (SARS-CoV-2) has further highlighted the challenge of differential diagnosis and shows yet again that better antimicrobial stewardship interventions are needed to counter the overprescribing of antibiotics in patients with viral infections.7
Host biomarkers other than CRP and PCT have been evaluated for distinguishing bacterial from non-bacterial infections, including human neutrophil lipocalin (HNL), heparin-binding protein and chitinase 3-like protein 1.8 There are also some commercially available tests. ImmunoXpert, from MeMed, uses a biomarker combination comprising CRP, interferon gamma-inducible protein 10 (IP-10) and TNF-related apoptosis-inducing ligand, while FebriDx, from Lumos Diagnostics, uses an MxA and CRP biomarker combination. While these biomarker signatures show promise, they have only been evaluated in limited settings. Any potential impact of co-infections or comorbidities, common in LMICs, on their effectiveness is unknown. Other characteristics of host-biomarker studies that hamper direct comparisons include: (1) just one/a few biomarkers in the study; (2) small sample sizes, increasing the probability of recruiting unrepresentative study populations; (3) narrow population subgroups (eg, children only, hospitalised only, respiratory infections only), limiting the generalisability of study results to the broader AFI population; (4) studies conducted in one country, so co-infections/comorbidities may not be comparable with those of other countries; (5) retrospective studies that used convenience sampling and case-control study designs, increasing the risk of bias; and (6) the lack of standard definitions for classifying bacterial versus non-bacterial infections.9
Here, we describe the Biomarker for Fever Diagnostic (BFF-Dx) study, specifically designed to evaluate host biomarkers to distinguish bacterial from non-bacterial infections in line with the published TPP and the final use case of such diagnostic tests. To our knowledge, this is the only study to evaluate host biomarkers in the intended target population (non-severe patients), prospectively, in multiple settings with a large sample set. We evaluated 18 host biomarkers in three distinct settings, in Brazil, Gabon and Malawi with the main objective of providing a performance comparison of host biomarkers in the non-severe AFI population from resource-limited settings, with the goal of overcoming many of the previously described limitations (eg, sample size, retrospective vs prospective, focused populations, biased analysis).10 The described comparison was conducted within the pragmatic context of diagnostic product development and aimed to identify host biomarkers or biomarker combinations for utilisation in next-generation RDTs.
Methodology
Study settings
This multinational, cross-sectional study was conducted in Brazil, Gabon and Malawi; Gabon and Malawi were selected as high-malaria endemicity settings, while Brazil was selected as a low-malaria endemic setting. The study sites were UPA Manguinhos and Family Health Clinics Armando Palhares in Rio de Janeiro, Brazil; the Clinical Trials Unit Centre of Medical Research Lambaréné (CERMEL), Lambaréné, Gabon; and Malawi Epidemiology and Intervention Research Unit (MEIRU), Chilumba campus, Malawi. The enrolment sites were an urban primary healthcare facility, a hospital in a semi-rural setting and a rural primary healthcare facility in Brazil, Gabon and Malawi, respectively. Participants were recruited from October 2018 to July 2019, May to November 2019 and April 2017 to April 2018, in Brazil, Gabon and Malawi, respectively. Reporting complies with the STARD-15 (Standards for Reporting of Diagnostic Accuracy Studies 2015) checklist.
Study population and study procedure
Participants were obtained through convenience sampling and included both children and adults, aged between 2 and 65 years, who presented at the outpatient clinics with a history of fever of ≤7 days duration (Brazil and Gabon) or fever at presentation (Malawi). Patients with signs of severe illness were not included in the study. The overarching study protocol was slightly adapted to each site due to local requirements (logistical or ethical). Detailed criteria for inclusion by study sites have been published previously.10 Outcomes were based on the TPP criteria and while no patient input was used, external expert input was used to define the target population and criteria. Only patients who met the eligibility criteria and who provided written consent (patient or guardian for children) were enrolled in the study. Data and samples were systematically collected and analysed as previously described. To ensure consistent quality and comparability of data, the same standard operating procedures were used at all sites (for data collection and laboratory testing).10
Patient and public involvement statement
None
Bacterial/non-bacterial classification and biomarker selection and testing
A two-step process was used to classify the patients into ‘bacterial’ and ‘non-bacterial’ groups. First, the cause of fever (bacterial/non-bacterial) was classified according to laboratory-determined parameters (‘electronic group’). The electronic group was based on predefined and widely accepted laboratory parameters, including direct pathogen detection, a fourfold increase in antibody titre or a positive PCR or antigen RDT result. The list of tests performed is described in detail in by Escadafal et al.10 Next, cases that could not be classified by laboratory-determined parameters were assessed by a panel of three independent clinical experts. The patient’s history and clinical and laboratory data were provided to the experts. Clinical expert’s assessments were then compared. If the three-panel members unanimously assigned a diagnostic label, patients were considered to have ‘bacterial’ or ‘non-bacterial’ infections; if two out of three-panel members reported a classification of ‘bacterial’ or ‘non-bacterial’, these patients were considered to have ‘probable bacterial infection’ or ‘probable non-bacterial infection’, respectively.
Data were analysed based on three groups of patients: (1) the ‘electronic group’, that is, subjects with a cause of fever defined based on laboratory parameters; (2) the ‘strict group’ which comprised the electronic group and the patients that were unanimously classified by the clinical panel of three experts; and (3) the ‘loose group’ which comprised the electronic and strict groups as well as those patients for whom two of the clinical experts agreed they had either probable bacterial or probable non-bacterial infection. Subjects with undetermined causes of fever according to the three classification criteria considered (‘electronic group’, ‘strict group’, ‘loose group’) were excluded from the statistical analysis. This outcome-oriented approach, based on methods developed for host-biomarker studies previously, was used to ensure the total intended-use population of any future test was represented in the final analysis.10 11
The evaluated biomarkers were selected based on previously reported performances and haematological markers as well as CRP were included as comparators (table 1 and online supplemental table 1 and 2).8 12
Supplemental material
Novel biomarkers were identified in the literature and evaluated in the BFF-Dx study, including sample type used, evaluation method and sample origin
At the end of data collection, all biomarker data were analysed to assess the percentage of missing values and the percentage of values below the lower limit or above the upper limit of detection of the used tests. Biomarkers with more than 50% of missing data or more than 95% of saturated values below the lower limit of quantification of the used test, were excluded from the following statistical analysis.
Statistical analysis
Kruskal-Wallis analysis and definition of covariates influence on biomarkers
A Kruskal-Wallis test, adjusted by Benjamini-Hochberg, was conducted for each biomarker to determine which covariates exhibited statistically significant differences in the distribution of biomarker values. The covariates studied were country (ie, the country of origin of the patients), age, sex, malaria status, comorbidities (ie, presence of one or more diseases among cardiovascular, neurological, respiratory, renal, genitourinary, connective tissue, cancer or infectious diseases), malnutrition status calculated based on WHO body mass index criteria, self-reported use of antibiotics prior to visiting the health facility, axillary temperature≥38°C and positive result to Chikungunya test. The Kruskal-Wallis test was performed for each of the three patient groups defined in the previous section (‘electronic’, ‘strict’, ‘loose’). The results of the Kruskal-Wallis test allowed the identification of covariates that most significantly impacted the biomarker distribution (p≤0.001, adjusted by Benjamini-Hochberg). The most significant covariates were considered for defining subgroups of patients in which the following univariate analyses were performed or included as covariates in the multivariable analyses.
Univariate analysis
As an exploratory step, the ability of each biomarker to discriminate between bacterial and non-bacterial infections was assessed by the area under the receiver operating characteristic curve (AUROC). In particular, subjects were ranked based on the values of the single variable of interest (ie, based on ordered values) and, using this as a score, calculated the ROC curve and the corresponding area under the curve. Such univariate analysis was conducted for each patient group (‘electronic’, ‘strict’, ‘loose’) and specific patient subgroups (Malaria status, Country and Age).
However, since the univariate analyses did not yield satisfactory results, we also explored multivariable models to potentially improve the predictive capabilities by incorporating a broader range of information.
Multivariable analysis
Multivariable classification models were developed to assess the discrimination ability of combinations of biomarkers and covariates. For the multivariable analysis, both linear (logistic regression) and non-linear classification models (RuleFit) were explored.13 The candidate features for each model included a group of host biomarkers and some additional covariates (age, temperature, fever duration, diastolic blood pressure, respiration rate and pulse rate). Regarding host-biomarkers, three different groups of biomarkers were considered: haematology biomarkers only (ie, white blood cell (WBC), neutrophil, red blood cell, lymphocyte counts), protein biomarkers only (ie, novel biomarkers+CRP) and haematology plus protein biomarkers (ie, all biomarkers).
For each patient subgroup and each candidate feature set, three multivariable models were developed: (1) a logistic regression model with stepwise (SW) feature selection; (2) a logistic regression model with features selected based on recursive feature addition (RFA; a variant of the method proposed in14); (3) RuleFit, a non-linear model in which a set of rules from an ensemble of decision trees (typically from a tree-based model like a random forest or gradient boosted trees) is generated and then fit a sparse linear regression model (regularised with least absolute shrinkage and selection operator (LASSO)), where the features are the rules generated from the trees.13 14
To further tackle the number of biomarkers and variables included in the best models, we introduced an additional selection step, employing a plateau-seeking approach. The primary objective of this approach was to pinpoint a concise set of variables capable of attaining an AUROC score similar to that of our comprehensive model which already incorporated the most impactful and previously selected variables. This was to ensure that our model was not only effective in terms of performance but also efficient in its variable inclusion.
Each model was trained and tested using the following pipeline. The data were randomly split into training and test sets (80% and 20% of the data, respectively) stratifying by the outcome variable. Missing data in the training and test sets were imputed using the MICE (multiple imputation by chained equation) algorithm. The n_imp parameter for MICE imputation was set to 1, resulting in a single imputed data set; however, the imputation process was integrated into a robust bootstrapping pipeline, generating ten independent data sets. This approach ensured variability in our results, stemming not only from the MICE imputation but also from the bootstrapping process. This dual approach guarantees that each imputed data set is distinct.15 All quantitative variables were scaled into the range (0,1) by subtracting their minimum value and dividing by the difference between the maximum and minimum values in the training set. The categorical variables with n categories were encoded using n-1 binary ‘dummy’ variables. The model was then trained on the imputed and scaled training set and its performance was assessed on the imputed and scaled test set by computing the AUROC. The AUROC on the test set was also calculated for single-host biomarkers, to allow a fair comparison of the performance of the multivariable classification models versus single-host biomarkers.
To assess the robustness and variability in the results of the developed models, the entire pipeline were bootstrapped, that is, it was run 10 times with different random training-test set splits. Finally, the mean and the SD or the minimum and maximum reached of the AUROC across the 10 training-test splits were calculated for each multivariable model and each single host biomarker.
Software
All statistical analyses and model development were performed using the R programming language (V.4.1.2). Specifically, the mice package was used for data imputation, while the pre and stats packages were used for RuleFit and logistic regression model development, respectively.
Results
Study population
In total, 1915 patients with AFI were included in the study (Brazil: n=500; Gabon: n=415; Malawi: n=1000). Just under half (862/1915, 45%) of participants at each study site were male. Children aged <5 years comprised 45/500 (9%), 182/415 (43.9%) and 367/1000 (36.7%) participants in Brazil, Gabon and Malawi, respectively; the median (range) age was 3 (2–4) years (table 2). Detailed baseline characteristics of patients and analyses of differences will be described in a separate manuscript (Alabi et al in preparation).
Baseline characteristics of patients
Bacterial and non-bacterial outcomes by classification groups
Using the electronic classification grouping, 15.1% (290/1915) of cases were bacterial infections, 20.2% (387/1915) were non-bacterial infections and 64.5% (1238/1915) had an undetermined cause of fever (figure 1). Under the strict classification grouping, 24.3% (366/1509), 66.9% (1010/1509) and 9.0% (133/1509) were classified as bacterial, non-bacterial and undetermined infections, respectively, while using the loose classification grouping 25.7% (491/1915), 67.3% (1286/1915) and 7.0% (133/1915) were classified as bacterial, non-bacterial and undetermined infections, respectively (figure 1). Subjects with undetermined causes of fever/infections were excluded from the following univariate and multivariable analyses.
Classification criteria to assign bacterial versus non-bacterial infection categories for the analysis. The flows in different colours (turquoise=bacteria, purple=non-bacterial, red=undetermined) represent the proportion of patients that were assigned into the respective group (bacteria/non-bacteria/undetermined) after each classification step. Group 1 representing only patients assigned using laboratory data; group 2 representing patients with a unanimous decision after review by the clinical panel; group 3 after clinical panel review and group 3 including all patients, even if only two panel members agreed on the probable cause.
Exclusion of biomarkers with too many missing or saturated values
The biomarkers C4b, HNL and PCT had more than 50% missing values and were therefore excluded. The high number of missing values is due to the fact that biomarkers were analysed in groups based on the required dilution using the Luminex platform. For some biomarkers, the dilution was not optimal and it was only possible to remeasure biomarkers with a different dilution a limited number of times. IFN-gamma and sTREM-1 were excluded due to more than 95% of values saturated to the minimum/maximum level detectable by the measurement instrument. All the biomarkers retained in the analysis had less than 12% missing values (online supplemental table 3).
Identification of relevant subgroups for analyses
According to the Kruskal-Wallis analysis on the ‘electronic group’, the variables ‘country’, ‘malaria status’ and ‘age’ showed statistically significant differences in the distributions of many host biomarkers (p≤0.001 for strong differences, 0.001≤p<0.01 for high differences; Online supplemental table 4). The variables ‘sex’, ‘comorbidities’, ‘history of antibiotic use’ showed no (p>0.05) or slight (p≤0.05) differences in all the host biomarkers. The effects of ‘chikungunya status’ and ‘fever above 38°C’ were generally significant (p≤0.01), but the sample sizes for these groups were either too small or exhibited an imbalance. Additionally, while we conducted subgroup analyses by clinical syndromes (ie, cough, diarrhoea or vomiting, dysuria or urinary urgency, headache, sore throat or swallow pain, rash), the resulting data sets were similarly limited in size, restricting our ability to make robust interpretations from these analyses. The primary focus remained centred on populations grouped by study country and malaria status variables—both of which showed big statistical differences with the value of the biomarkers in the ‘strict’ and ‘loose’ groups (online supplemental tables 5 and 6)—other significant covariates were also included in the multivariable analysis. This inclusion was due to their influence and factors like the study country were considered as variables in the overall scenario.
Individual host-biomarker performance: univariate analysis
The performance of 18 host biomarkers was consistent across the three patient classification groups in each of the settings (table 3 and online supplemental tables 7–9). WBC and neutrophil counts were the most effective biomarkers for differentiating bacterial and non-bacterial infections. For the malaria-negative population, the mean (95% CI) of AUROC for WBCs was between 0.60 (0.48 to 0.72) and 0.83 (0.77 to 0.88) and for neutrophils, it was between 0.67 (0.57 to 0.77) and 0.80 (0.74 to .86) across the three countries and the three groups (‘electronic’, ‘strict’, ‘loose’). Neutrophil and WBC counts showed the highest AUROCs in the Brazilian population, between 0.80 (0.74 to 0.86) and 0.83 (0.77 to 0.88), respectively. All protein biomarkers showed relatively poor performances (<0.7 in most cases, table 4) in all three settings. Galactin-9, CRP, IP-10 and NGAL were the best-performing protein biomarkers across the three settings and criteria. Protein biomarkers showed better performances in Malawi and Gabon, as in Brazil most protein biomarkers showed performances of<0.6. When the biomarker results were stratified by age, the AUROCs were slightly higher for children (≤15 years) compared with those seen for adults in the malaria-negative population (online supplemental tables 10–15). Among the malaria-positive population, WBC, lymphocyte and neutrophil counts were the best-performing biomarkers in both Gabon and Malawi (in most cases between 0.6 and 0.7).
Univariate analysis of 18 individual biomarkers‡ among malaria-negative patients for all three countries (A–C)
Multivariable analysis of biomarkers among malaria-negative patients, including the gain/loss of performance when comparing multivariable analysis and single-host biomarkers comprising both haematological and protein host biomarkers
Combinations of host biomarkers and additional covariates: multivariable analysis
The best-performing biomarkers in the univariate analysis were compared with the best performances from the multivariable analyses with several feature-selected biomarkers and covariates (table 4 and online supplemental tables 16–21). In most cases, the best combination of biomarkers showed higher AUROCs than the top-performing individual biomarkers, with a low/moderate ‘gain’ (range 1–13%). The best-performing AUROCs were very similar, irrespective of the multivariable model used, especially for the ‘strict’ and ‘loose’ groups (difference in AUROC range 0.02–0.03 for Malawi and Brazil). Biomarkers identified as top performing by the multivariable analyses differed depending on the model used. While SW and RFA selected three to five biomarkers or combinations, RuleFit selected more biomarkers (10 variables on average) to be part of the signature. The relatively low increase in AUROC when comparing the top-performing single biomarker with multivariable models indicates that biomarkers in addition to the single best-performing biomarker do not make a major contribution.
Discussion
We present the most extensive and diverse host biomarker evaluation study to differentiate bacterial from non-bacterial infections in LMICs. The study aimed to identify if next-generation host biomarkers for distinguishing bacterial from non-bacterial cases of AFI which could replace existing biomarkers such as CRP, PCT and WBC/neutrophil assessments. The data show that none of the promising host-biomarkers exhibited high AUROCs in our non-severe AFI population in either low malaria prevalence (Brazil) or high malaria prevalence (Gabon, Malawi) settings. Haematology biomarkers and CRP were included as a baseline to identify better-performing markers; however, they remain those with the highest AUROC values (approximately 0.60–0.70 AUROC) in our population.
Overall, the performance of all markers was underwhelming, yet not surprising. It aligns with previous data where a marked reduction in performance was observed when shifting the population from inpatients to outpatients.16–18 Previously, it was hypothesised that the decrease in performance in host biomarkers between HIC and LMIC settings or even between Africa and Asia, was due to the untreated comorbidities (eg, diabetes, malaria, neglected tropical diseases) which contribute to inflammation and the non-specific triggering of host biomarkers, unrelated to the current acute presentation.18 19 In our data the performance was indeed poorer in malaria-positive patients (AUROC<0.6); however, even in the malaria-negative population, biomarkers showed low performances (∼0.6–0.7) in our cohort. Similarly, sex and arboviral status appeared to have no major effect on biomarker performance. Our data notably indicated that combining biomarkers can enhance performance. However, this improvement was not consistently observed. When combining several biomarkers and additional covariates, the ‘gain’ in AUROC values was low/moderate (range 1–13%) compared with the top-performing individual biomarkers. From a diagnostic development perspective, a low gain in performance would not justify the additional complexity and cost of developing a simple multiplex test.
Adding to the challenges of host biomarker studies is the lack of consistent reference standards and that most studies have focused their analyses solely on the subpopulation of patients with a microbiologically confirmed diagnosis. This approach ignores the largest group (>70%) of patients and the intended-use population of any future test.20 The group with laboratory-confirmed diagnosis will decrease further in the non-severe AFI population; presenting at the primary care level. Going forward more clarity will likely follow as a recent host-biomarker test (BVtest, MeMed, Israel) was approved by the Food and Drug Administration (FDA) and subsequent guidance will prescribe more clearly how studies have to be designed to standardise the classification of ‘bacterial’ versus ‘non-bacterial’ evaluated to guide prescribing for bacterial or non-bacterial infections.9 21 Our protocol is aligned with the FDA-approved classification hence we are confident our methodology is robust.
While our study aimed to mitigate the challenges described, it still had several limitations. The study did not include a control group, so no baseline information was available for biomarker performance or asymptomatic carrier populations. The enrolment period in Brazil and Gabon lasted for less than 1 year and given the heterogeneity of causes of AFI across time, the performance of the biomarkers may not be generalisable to different times of the year and geographical settings, particularly in Asia. The study used a two-step process to classify outcomes and the clinical classification based on recorded clinical information may have introduced subjectivity. Notably, clinicians had access to the haematology biomarker results (WBCs, neutrophils) during outcome classification which might have introduced a bias in favour of these biomarkers. However, comparing AUROCs between all classification groups (E, L, S) suggests this potential bias had no major impact as the results are similar across groups. There were some heterogeneities in the inclusion criteria across the various study sites, including age groups and fever criteria. In Brazil and Gabon, the inclusion criterion was a history of fever in the past 7 days, while it was fever at presentation in Malawi. Studies have found that acute fever at presentation has implications for the interpretation of host biomarkers;22 however, our subanalysis by acute fever showed no differences, so we do not consider that these different inclusion criteria impacted interpretation. Despite best efforts to standardise procedures, there was a level of adaptability required in the choice of testing methods by the clinical teams in each country, for arbovirus and respiratory pathogen detection. Further, the choice to follow the TPP and focus on non-severe patients in the recruitment was based on the need’s definition by the WHO and others, while this still holds as a major priority, in hindsight this focus did not allow us to stratify by severity (eg, Sequential Organ Failure Assessment (SOFA) score).
Overall, the results of this diverse study highlight the difficulties in identifying single-host biomarkers or simple-host biomarker combinations that can help solve the problem of undifferentiated prescribing in primary healthcare, particularly to be used across diverse global settings. On the eighth birthday of the original TPP for a diagnostic assay to distinguish bacterial and non-bacterial infections in resource-limited settings, a more recent consultation confirmed that the need for such an assay remains and is in fact increasingly urgent.6 23 Yet again, the consultation concluded primary healthcare clinics and their equivalents must have the ability to perform tests other than just malaria RDTs.23 The lack of diagnostics infrastructure at the lower levels of health systems is well documented and requires urgent improvement to support medical staff in their decision-making. While no novel host biomarker assay meets these needs, evidence for existing biomarkers, for example, CRP, and various haematology biomarkers, should be used to drive such improvements, although using slightly different approaches and cut-offs across settings. In addition to using existing tools, increased investment into lower-level health infrastructures is critical and the first step to improved care. Recent studies have shown that even simple-host biomarkers, such as CRP, can have a major impact on how clinical staff use antibiotics.24–26 The current study confirms that the existing biomarkers are imperfect and hence should only be used as guidance, in conjunction with expanded clinical algorithms.27 28 Such guidelines, alongside adopted policies, strengthened infrastructures and accessible haematology/biochemistry data could enable healthcare workers to use simple tools to gain additional data points to help form a more evidence-based diagnosis that has to be guided by the local epidemiology. Optimising existing haematology or biochemistry tools and their maintenance requirements to meet the needs of low-resourced settings could be one step towards more expanded use of these well-known markers. In conclusion, our study reinforces the continued need for innovation in the host biomarker space and highlights the importance of targeted evaluations of such innovations, in diverse intended-use settings, to fully understand their true value.
Data availability statement
Data are available upon reasonable request. Data are available upon request from the study sites (Malawi, Brazil, Gabon) or the study sponsor (FIND). All data were shared with the local teams to allow further analysis and investigations and acknowledge ownership.
Ethics statements
Patient consent for publication
Ethics approval
The study protocol was submitted to ClinicalTrials.gov (NCT03047642) and ethical approval was obtained from all relevant institutional committees in Brazil (Research Ethics Committee of INI-FIOCRUZ and Comissão Nacional de Ética em Pesquisa, Ref:2.235.565; National Research Ethics Committee), Gabon (Comité National d’Ethique pour la Recherche, RefNr:N°0078/2019PR/SG/CNER) and Malawi (National Health Science Research Committee, Approval Nr: 16/9/1668; Observational and Intervention Research Ethics Committee of the London School of Hygiene and Tropical Medicine, UK, LSTMH Ref: 11974) and all details of the design have been previously published.10 Participants gave informed consent to participate in the study before taking part.
Acknowledgments
We would like to thank a group of dedicated colleagues (Quique Bassat, Heidi Hopkins, Valerie D’Acremont) for critical review of the data and continued discussions regarding analysis and interpretation. Further thanks to Luciano S. Oliveira, Cintia Damasceno dos Santos Rodrigues and Carolina Cardoso dos Santos for supporting the work in Brazil. Medical writing as well as editorial support, under the direction of the authors, was provided by Adam Bodley, funded by FIND, the global alliance for diagnostic in accordance with Good Publication Practice guidelines.
Footnotes
X @siqueira__andre, @j_moreira_, @sabinedittrich
BLF-C and MA contributed equally.
CE and MV contributed equally.
Contributors SD, CE, SO, AM, AMS, SG, STA, MM-L conceptualised the study and study design; CE, SG, STA, AMS, JKM, VH, ALK, AA, JCBO, MM-L, PNE, JAM, PB, LB, AdRM, BCC, MMS, AMBdF, EAdSRdS, MCSL, JH, AG, MJ, NS-M, CH implemented the study and data collection; MA, MV, SL, SO, BDC, BLF-C, SD, SP, SG, AMS, STA, SJL conducted data analysis and interpretation. BLF-C, SD wrote the first draft of the manuscript and all authors contributed to the final version of the manuscript. Guarantor is SD.
Funding Funding for this work was provided to FIND, the global alliance for diagnostics, by the governments of the Netherlands, the Foreign, Commonwealth and Development Office of the UK, and Australia Aid. The funding organisations had no role in the study design, data collection, analysis and interpretation of data. Further they had no role in writing of the report or decision to submit for publication
Competing interests SD, BLFC, CE, VH, SO, CH, AM, SL are or were employed by FIND, the global alliance for diagnostic during the study period. All other authors do not declare any competing interests.
Patient and public involvement Patients and/or the public were not involved in the design, or conduct, or reporting, or dissemination plans of this research.
Provenance and peer review Not commissioned; externally peer reviewed.
Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.