Diagnostic performance of food consumption for bacteraemia in patients admitted with suspected infection: a prospective cohort study
====================================================================================================================================

* Toshihiko Takada
* Kotaro Fujii
* Masataka Kudo
* Sho Sasaki
* Tetsuhiro Yano
* Yu Yagi
* Yasuhiro Tsuchido
* Hideyuki Ito
* Shunichi Fukuhara

## Abstract

**Objectives** A previous study reported that food consumption is useful to rule out bacteraemia in hospitalised patients. We aimed to validate the diagnostic performance of (1) food consumption and (2) a previously reported algorithm using food consumption and shaking chills for bacteraemia in patients admitted to hospital with suspected infection.

**Design** Prospective cohort study.

**Setting** Department of General Medicine in two acute care hospitals in Japan.

**Participants** A total of 2009 adult patients who underwent at least two blood cultures on admission.

**Primary outcome measures** The reference standard for bacteraemia was judgement by two independent specialists of infectious diseases. Food consumption was evaluated by the physician in charge asking the patient or their caregivers the following question on admission: ‘What percentage of usual food intake were you able to eat during the past 24 hours?’

**Results** Among 2009 patients, 326 patients were diagnosed with bacteraemia (16.2%). Diagnostic performance of food consumption was sensitivity of 84.4% (95% CI 80.1 to 88), specificity of 19.8% (95% CI 18 to 21.8), positive predictive value (PPV) of 16.9% (95% CI 15.2 to 18.9) and negative predictive value (NPV) of 86.8% (95% CI 83.1 to 89.8). The discriminative performance was an area under the curve of 0.53 (95% CI 0.50 to 0.56). The performance of the algorithm using food consumption and shaking chills was sensitivity of 89% (95% CI 85.1 to 91.9), specificity of 18.8% (95% CI 17 to 20.7), PPV of 17.5% (95% CI 15.7 to 19.4) and NPV of 89.8% (95% CI 86.2 to 92.5).

**Conclusion** Our results did not show the usefulness of food consumption and the algorithm using food consumption and shaking chills for the diagnosis of bacteraemia in patients admitted to hospital with suspected infection.

*   general medicine (see internal medicine)
*   internal medicine
*   infectious diseases

### Strengths and limitations of this study

*   This was the first study to evaluate the external validity of the diagnostic performance of food consumption and an algorithm using food consumption and shaking chills for bacteraemia in patients admitted to hospital with suspected infection.

*   The value of adding food consumption to previously reported predictors for the diagnosis of bacteraemia was also assessed.

*   Food consumption was evaluated by the physician in charge asking the patient or their caregivers the following question on admission: ‘What percentage of usual food intake were you able to eat during the past 24 hours?’

*   Rather than the inclusion criterion of those who underwent blood cultures based on physicians’ judgement, more objective criteria (eg, based on patient’s signs and symptoms) would be more appropriate.

## Introduction

Blood cultures are essential for correct identification and management of bacteraemia.1 Positive blood cultures provide information about the causative organism of the infection and its susceptibility to antibiotics.2 Since the sensitivity of the cultures is diminished by antibiotics, blood cultures should be obtained before antibiotics are administered.3 Consequently, physicians are likely to perform blood cultures in patients with suspected infection even when the suspicion of bacteraemia is low, resulting in positive results in only about 10% of patients.4 This low yield of blood cultures leads to an unnecessary increase in medical costs, burden on both patients and healthcare workers and risk of contamination.5–7 Thus, it is crucial to identify patients who truly need assessment by blood cultures.

Among several clinical and laboratory items reported as useful predictors for bacteraemia, Komatsu *et al*8 found that food consumption assessed by nursing staff is useful to rule out bacteraemia in hospitalised patients, with a negative predictive value (NPV) of 98.3% for patients with a normal amount of food consumption.9 They also developed a simple algorithm consisting of two items: food consumption and shaking chills. This simple algorithm was reported as useful for the risk estimation of bacteraemia. However, the assessment of food consumption by nursing staff is possible only in patients who have an episode of suspected infection during hospitalisation. Thus, it is not applicable to patients admitted for work-up/management of suspected infection that occurs outside the hospital. Still, in these patients, food consumption can be assessed by the patients themselves or their caregivers.

Accordingly, the aims of this study were to validate the diagnostic performance of (1) food consumption, and (2) the algorithm using food consumption and shaking chills for bacteraemia in patients admitted to hospital with suspected infection.

## Materials and methods

This study was a prospective observational study at two acute care hospitals: Shirakawa Kosei General Hospital (471-bed capacity, Fukushima, Japan) and Iizuka Hospital (1048-bed capacity, Fukuoka, Japan). All patients provided written informed consent. We followed the Standards for Reporting of Diagnostic accuracy guideline.10

### Patients

Between April 2017 and January 2019, we consecutively included patients using the following inclusion criteria: (1) patients who underwent at least two sets of blood culture within 24 hours of admission to the Department of General Medicine and (2) aged 18 years or older. As in previous studies,6–8 we used the physicians’ decision to obtain blood cultures as a surrogate indicator of suspected infection. The exclusion criteria were as follows: (1) patients under tube feeding (because the amount of food consumption is not affected by the patients’ status) and (2) use of glucocorticoid or immunosuppressants (because blood cultures should be taken for these high-risk patients).

### Food consumption

In the previous study by Komatsu *et al,*8 food consumption was defined as the amount of meal intake during hospitalisation just before blood cultures as rated by nursing staff. However, this assessment by nursing staff was not applicable to patients who underwent blood cultures on admission. Therefore, food consumption needed to be assessed by the patients themselves or by their caregivers in this study. From a previous study that evaluated the diagnostic performance of food consumption for the diagnosis of community-acquired pneumonia, we used a definition of food consumption based on self-assessment of 24-hour food intake in proportion to the usual intake.11 On admission, the physician in charge of the management of a patient during hospitalisation asked the patient ‘What percentage of usual food intake were you able to eat during the past 24 hours?’ and recorded the responses. In cases where patients could not assess food consumption by themselves (eg, those with dementia), we asked their caregivers to rate it. Although the cut-off point for food consumption in the study by Komatsu *et al*8 9 was 80% (ie, normal: >80% intake of a meal, low: ≤80% intake), this cut-off was not defined to optimise its diagnostic performance (ie, it was clinically defined). Therefore, we evaluated its diagnostic performance with various cut-off points. Further details of the evaluation are given in the *Statistical analyses* section.

### Algorithm using food consumption and shaking chills

In the algorithm developed by Komatsu *et al,*8 shaking chills was used in addition to food consumption. According to a previous study,12 shaking chills was defined as ‘feeling extremely cold with rigours and generalised bodily shaking, even under a thick blanket’. Based on these two dichotomised variables of food consumption and shaking chills, patients were categorised into four groups: normal food consumption without shaking chills (group 1), normal food consumption with shaking chills (group 2), low food consumption without shaking chills (group 3) and low food consumption with shaking chills (group 4). In the original study by Komatsu *et al*8, the prevalence of bacteraemia in each group was 2.4%, 4%, 14.4% and 47.7%, respectively. Groups 1 and 4 were classified as low and high risk of true bacteraemia, respectively, while group 2 and 3 were classified as ‘further assessment is required’.

### Definition of bacteraemia

At least two sets of blood cultures (one aerobic and one anaerobic bottle) were collected in all patients within 24 hours of admission. For blood cultures, BacT/Alert (bioMérieux, Marcy-l’Etoile, France) was used in Shirakawa Kosei General Hospital, while BACTEC (Becton Dickinson, Sparks, USA) was used in Iizuka Hospital. In both hospitals, the minimum incubation period was 7 days. Positive blood cultures do not always indicate true bacteraemia as it is sometimes caused by contamination of common skin pathogens.13 In this study, cases with two or more positive blood cultures of a certain, unique pathogen were judged as true bacteraemia. In addition, in cases with only one positive blood culture (including cases with two or more positive blood cultures of different pathogens), they were independently judged by two infectious disease specialists (YT and HI), who were blinded from the information on food consumption and shaking chills to avoid incorporation bias.14 Those specialists made their judgement based on other clinical information including the clinical course and the type of bacteria. Conflicts between two specialists were resolved by discussion between them.

### Statistical analyses

#### Sample size estimation

Based on the study by Komatsu *et al,*8 we assumed a sensitivity of food consumption and the algorithm as 94%. To estimate the assumed sensitivity with 3% absolute precision and a 95% confidence level, a sample size of at least 241 patients with bacteraemia was required.

#### Missing data

In our data set, there were some missing values (ranging from 0% to 11.5%). To avoid biased results by excluding patients with missing values, we imputed them using chained equations with all available information including the outcome under the assumption of missing at random.15 16 Ten imputed data sets were separately analysed and the results were pooled using Rubin’s rules.17

#### Agreement in the judgement of bacteraemia

The agreement in the judgement of the presence of bacteraemia between the two infectious disease specialists was assessed using Cohen’s kappa coefficient (κ).18

#### Diagnostic performance of food consumption and the algorithm

Between those with and without bacteraemia, food consumption and the proportion of patients with shaking chills were compared using the Mann-Whitney-Wilcoxon test and the χ2 test, respectively. The diagnostic performance of food consumption was evaluated in terms of sensitivity (Sn), specificity (Sp), positive predictive value and NPV, respectively, positive likelihood ratio and negative likelihood ratio (LR+ and LR–, respectively) and diagnostic OR (DOR). DOR was calculated by dividing LR+ by LR–. Higher DOR indicates better performance of discrimination, while a DOR of 1 indicates that the information does not contribute to the diagnosis at all.19 These indexes were assessed using various cut-off points. In addition, we used the area under the curve (AUC) to evaluate discriminative performance. AUC ranges from 0.5 (no discrimination) to 1 (perfect discrimination).20 The diagnostic performance of shaking chills was also evaluated.

Next, we applied Komatsu *et al*’s algorithm to our patients. The prevalence of bacteraemia in each of the four groups was calculated. Also, the diagnostic performance of the algorithm was calculated in a conservative scenario in which all patients in group 2 and group 3 were categorised as the high-risk group (ie, blood cultures should be performed).

#### Subgroup analysis

Because self-reported food consumption could be unreliable, especially in elderly patients, we performed a subgroup analysis based on age with the cut-offs of 65, 75 and 85 years old. In this subgroup analysis, the diagnostic performance of food consumption was evaluated with the cut-off of 80%.

#### Value of adding food consumption to previously reported predictors

We also evaluated if the information about food consumption had any additional diagnostic value to previously reported predictors for the diagnosis of bacteraemia. Based on a systematic review of existing prediction models for bacteraemia,21 we selected the following predictors: age, performance status, living in a nursing home, indwelling vascular catheter, shaking chills, suspicion of infective endocarditis, consciousness disturbance, systolic blood pressure, pulse rate, respiratory rate, body temperature, serum creatinine and C reactive protein. For performance status, a simplified scale with four levels was used.22 First, we fitted a logistic regression model including those predictors and the presence/absence of bacteraemia as the outcome (base model). Next, we added food consumption to the base model (extended model). The functional form of all continuous variables was evaluated using restricted cubic splines with three knots, and incorporated as such when a non-linearity association was significant.23 The added value of food consumption for the diagnosis of bacteraemia was quantified as the difference in model performance between the base model and the extended model. The model performance was evaluated in terms of likelihood ratio test, discrimination (c-index) and calibration (calibration plots).

All values above were reported as point estimates with 95% CI. The Wilson Score method was used to estimate 95% CI for Sn, Sp, PPV and NPV.24 P values less than 0.05 were considered statistically significant. We used R statistical software (V.3.6.0; R foundation for statistical computing, [www.R-project.org](http://www.R-project.org)) for all analyses.

### Patient and public involvement

No patients were involved in the development of the research question, the outcome measures or in the design and implementation of the study.

## Results

### Patient characteristics

Among the 2226 eligible patients, 12 patients under tube feeding, 205 patients with the use of glucocorticoid or immunosuppressants were excluded. A total of 2009 patients were analysed. Patient characteristics are shown in table 1. Median age was 81 years (IQR 69–88). Bacteraemia was diagnosed in 326 patients (16.2%). *Escherichia coli* was the most frequent pathogen (n=157, 48.2%). Among patients with bacteraemia, common clinical diagnoses were urinary tract infection (n=153 (46.9%)), hepatobiliary tract infection (n=24 (7.4%)) and skin and soft tissue infection (n=18 (5.5%)). On the other hand, among those without bacteraemia, respiratory infection (n=514 (30.5%)), urinary tract infection (n=204 (12.1%)) and viral infection (n=82 (4.9%)) were common diagnoses. The assessment of food consumption was completed in 96.7% of the patients. Food consumption was not significantly different between patients with and without bacteraemia (30% (IQR 10–60) vs 40% (IQR 10–70); p=0.194). The proportion of patients with shaking chills was significantly higher in those with bacteraemia than in those without (22.7% vs 5.2%; p<0.001).

View this table:
[Table 1](http://bmjopen.bmj.com/content/11/5/e044270/T1)

Table 1 
Patient characteristics

### Agreement in the judgement of bacteraemia

The agreement in the judgement between the two specialists was κ of 0.83 (substantial agreement).18

### Diagnostic performance of food consumption and shaking chills

In table 2, the diagnostic performance of food consumption and shaking chills are summarised. With the cut-off of 80%, the diagnostic performance of food consumption was Sn of 84.4% (95% CI 80.1 to 88), NPV of 86.8% (95% CI 83.1 to 89.8) and DOR of 1.3 (95% CI 1 to 1.9). With the cut-off between 60% and 90%, the diagnostic performance of food consumption did not change dramatically with Sn around 80%, NPV less than 90% and DOR around 1.3. The AUC of food consumption was 0.53 (95% CI 0.50 to 0.56). Shaking chills showed Sp of 94.7 (95% CI 93.6 to 95.7), PPV of 45.4 (95% CI 37.9 to 53.1) and DOR of 5.3 (95% CI 3.7 to 7.4).

View this table:
[Table 2](http://bmjopen.bmj.com/content/11/5/e044270/T2)

Table 2 
Diagnostic performance of food consumption and shaking chills

### Diagnostic performance of the algorithm

In figure 1, the prevalence of bacteraemia in each group categorised based on the algorithm is shown. Among 352 patients categorised as low risk of bacteraemia (with normal food consumption without shaking chills), 36 (10.2%) patients were diagnosed with bacteraemia. The prevalence of bacteraemia was 10.2% (95% CI 7.5 to 13.9) in group 1, 46.9% (95% CI 30.9 to 63.6) in group 2, 14.5% (95% CI 12.8 to 16.4) in group 3 and 44.6% (95% CI 36.3 to 53.2) in group 4. In a conservative scenario assuming that all patients in groups 2 and 3 underwent blood cultures, the diagnostic performance of the algorithm was Sn of 89% (95% CI 85.1 to 91.9), Sp of 18.8% (95% CI 17 to 20.7), PPV of 17.5% (95% CI 15.7 to 19.4), NPV of 89.8% (95% CI 86.2 to 92.5), LR+ of 1.1 (95% CI 1.1 to 1.1), LR– of 0.6 (95% CI 0.6 to 0.6) and DOR of 1.9 (1.7 to 2).

![Figure 1](http://bmjopen.bmj.com/https://bmjopen.bmj.com/content/bmjopen/11/5/e044270/F1.medium.gif)

[Figure 1](http://bmjopen.bmj.com/content/11/5/e044270/F1)

Figure 1 
Categorisation of patients based on the algorithm using food consumption and shaking chills among 352 patients categorised as low risk of bacteraemia (with normal food consumption without shaking chills), 36 (10.2%) patients were diagnosed with bacteraemia.

### Subgroup analysis

The results of the subgroup analysis based on age are summarised in table 3. The diagnostic performance was not different among the subgroups with Sn around 80% and NPV around 86%.

View this table:
[Table 3](http://bmjopen.bmj.com/content/11/5/e044270/T3)

Table 3 
Diagnostic performance of food consumption in subgroups based on age

### Added value of food consumption

The base model and the extended model are shown in online supplemental table 1. The model fit did not improve by adding food consumption (p=0.095). The c-index of the base model and the extended model was 0.761 (95% CI 0.732 to 0.790) and 0.762 (95% CI 0.734 to 0.791), respectively (p=0.546). The calibration plots of the base model and the extended model are shown in figure 2. The base model already shows good calibration and adding food consumption resulted in minimal improvement.

### Supplementary data

[[bmjopen-2020-044270supp001.pdf]](pending:yes)

![Figure 2](http://bmjopen.bmj.com/https://bmjopen.bmj.com/content/bmjopen/11/5/e044270/F2.medium.gif)

[Figure 2](http://bmjopen.bmj.com/content/11/5/e044270/F2)

Figure 2 
Calibration plots of the base model and the extended model. The base model (left) already shows good agreement, with the calibration curve close to the dashed diagonal line (the line of perfect calibration). Improvement in the extended model (right) is minimal.

## Discussion

Diagnostic performance of self-reported food consumption for bacteraemia was poor in patients admitted to hospital with suspected infection, with Sn of 84.4% (95% CI 80.1 to 88), NPV of 86.8% (95% CI 83.1 to 89.8) and DOR of 1.3 (95% CI 1 to 1.9) at the previously reported cut-off point of 80%. This poor performance of food consumption was consistent with various cut-off points between 60% and 90%. The performance of the algorithm using food consumption and shaking chills was also poor, missing 36 patients (10.2% (95% CI 7.5 to 13.9)) with bacteraemia among 352 patients categorised in the low risk of bacteraemia group. Furthermore, the information about food consumption did not show added diagnostic value to previously reported predictors.

### Comparison with previous findings

In the study by Komatsu *et al,* the performance of food consumption was reported as Sn of 93.7%, Sp of 34.6%, LR+ of 1.43, LR– of 0.18 and DOR of 7.9,8 which was much better than our findings. Also, only 2.4% of patients were diagnosed with bacteraemia in the low-risk group categorised by the algorithm using food consumption and shaking chills.8

There were several possible explanations for these dissociated findings. First, food consumption was rated by nursing staff in the study by Komatsu *et al,* while it was self-reported in this study. Compared with the food consumption rated by nursing staff who received instruction on how to evaluate food consumption, self-reporting by patients themselves or their caregivers could be much less reliable. We hypothesised that self-reported food consumption was imprecise because there might be many patients with memory disturbance in this super-aged population (median age 81 (IQR 69–88)). Thus, we conducted a subgroup analysis comparing the performance of self-reported food consumption between younger and older patients. However, performance did not differ among age groups, which implies that the poor diagnostic performance of self-reported food consumption was not due to the high proportion of older patients in the present study. Also, food consumption in the study by Komatsu *et al* was based on the meal just before the blood cultures were taken, while it was based on 24-hour food intake before presentation in proportion to the usual intake in this study. The food consumption of the last meal latest before presentation may reflect patients’ status better than that of meals during the 24 hours before presentation. Second, the study population was different between the two studies. We included patients who were admitted for work-up/management of suspected infection (ie, community-acquired infection), while Komatsu *et al* included hospitalised patients (ie, infection could have been acquired either in the community or in the hospital). While the prevalence of bacteraemia was similar in the two studies (16.2% in this study and 13.6% in Komatsu *et al*), patients with bacteraemia in the study by Komatsu *et al* could be more severe than our patients as the former included severe hospital-acquired conditions like catheter-related blood stream infection. A larger difference in severity between patients with and without bacteraemia might lead to better discriminative performance of food consumption. However, vital sign parameters in patients with bacteraemia (eg, body temperature, heart rate, systolic blood pressure and respiratory rate) showed almost similar values in both studies, implying this hypothesis for the difference between the findings of the two studies is not very convincing.

Furthermore, our findings were not in line with our previous study which suggested the usefulness of self-rated food consumption (using the same definition as the present study) for the diagnosis of community-acquired pneumonia in older patients who presented with upper respiratory symptoms.11 In this previous study, food consumption with the cut-off of 50% showed Sn of 66.7%, Sp of 79.3%, LR+ of 3.2, LR– of 0.4 and DOR of 7.7 for the diagnosis of community-acquired pneumonia.11 The previous study aimed to differentiate pneumonia from other viral infection (ie, upper respiratory infection), while we aimed to differentiate bacteraemia from other bacterial infection in the present study. Thus, the severity between the two conditions was less prominent in the present study than in the previous one, which might have led to the poorer performance of food consumption.

### Clinical implications

Our findings suggest that self-reported food consumption is not useful for the diagnosis of bacteraemia in patients admitted for work-up of suspected infection. In a questionnaire survey, Lautenbach *et al* asked physicians about the acceptability of the performance of a clinical prediction rule for the diagnosis of bacteraemia.25 They revealed that physicians required a very high Sn of 95% and infectious disease specialists required an even higher Sn of 98% for the clinical implementation of a prediction rule for the diagnosis of bacteraemia. This high expectation hampered the use of existing prediction rules for the diagnosis of bacteraemia. Considering the diagnostic performance of Sn of less than 90% when applied to patients admitted with suspected infection, food consumption alone and its algorithm in combination with shaking chills seemed unacceptable in clinical practice, at least with the current definition of self-reported food consumption. The performance of the algorithm in the original paper by Komatsu *et al* (Sn of 94.1%) also did not satisfy this expected high Sn. Physicians may use the prediction model developed by Shapiro *et al*.7 The model consists of three major criteria (suspected endocarditis, body temperature >39.4°C and indwelling vascular catheter) and nine minor criteria (body temperature between 38.3°C–39.3°C, age >65 years, chills, vomiting, systolic blood pressure <90 mm Hg, white blood cell count >18 000/ µL, bands >5%, platelets <150 000/ µL and creatinine >2.0 mg/dL).7 Either one major criterion or two or more minor criteria are deemed an indication for blood culture. Although the model is less simple than the algorithm using only the two items of food consumption and shaking chills, it has been well validated in emergency and in-hospital settings with Sn of around 95%.26 27

### Strength and limitations

To the best of our knowledge, this is the first study to evaluate the external validity of the diagnostic performance of food consumption for bacteraemia. Some limitations should be noted. First, our cohort included many elderly patients (median age 81 years). Thus, the external validity of our findings, particularly in young patients, should be further evaluated. Second, we included those who underwent at least two sets of blood culture. This criterion has been frequently used as a surrogate indicator of suspected infection in previous studies reporting prediction rules/algorithms for the diagnosis of bacteraemia.6–8 Whether to order blood culture was based on physicians’ judgement, therefore, it could be subjective. To improve the reproducibility of our findings, more objective criteria (eg, based on patient’s signs and symptoms) would be more appropriate. However, it is unethical and unfeasible to obtain blood cultures in patients who do not have clinical indication. Third, we performed the subgroup analysis based on age to investigate the effect of memory disturbance on the performance of food consumption. However, this should be assessed by subgroup analysis based on the presence/absence of memory disturbance and whether the judgement was rated by patients themselves or their caregivers. Unfortunately, since we did not collect these variables, we could not conduct such subgroup analyses. In addition, it should be noted that the sample size calculation of this study was performed for the main analyses. Therefore, the number of patients in each subgroup was not large enough to precisely estimate the diagnostic performance of food consumption.

## Conclusions

Contrary to previous findings, our results did not show the usefulness of food consumption and the algorithm using food consumption and shaking chills for the diagnosis of bacteraemia in patients admitted to hospital with suspected infection.

## Data availability statement

Data are available upon reasonable request to the corresponding author.

## Ethics statements

### Patient consent for publication

Not required.

### Ethics approval

The ethics committee of Shirakawa Kosei General Hospital (HAKURIN 17-003) and Iizuka Hospital (R-17135) granted ethical approval.

## Acknowledgments

We thank all physicians who belonged to the Department of General Medicine in Shirakawa Kosei General Hospital and Iizuka Hospital for their support for data collection and Kaori Omata of Shirakawa Kosei General Hospital for data management.

## Footnotes

*   Contributors TT: Conceptualisation, Methodology, Formal Analysis, Writing—Original Draft. KF: Conceptualisation, Methodology, Data Curation, Writing—Review and Editing. MK: Data Curation, Writing—Review and Editing. SS: Data Curation, Writing—Review and Editing. TY: Data Curation, Writing—Review and Editing. YY: Data Curation, Writing—Review and Editing. YT: Data Curation, Writing—Review and Editing. HI: Data Curation, Writing—Review and Editing. SF: Conceptualisation, Supervision, Project Administration.

*   Funding The authors have not declared a specific grant for this research from any funding agency in the public, commercial or not-for-profit sectors.

*   Competing interests None declared.

*   Provenance and peer review Not commissioned; externally peer reviewed.

*   Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.

[http://creativecommons.org/licenses/by-nc/4.0/](http://creativecommons.org/licenses/by-nc/4.0/)
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: [http://creativecommons.org/licenses/by-nc/4.0/](http://creativecommons.org/licenses/by-nc/4.0/).

## References

1.  Jaimes F, Arango C, Ruiz G, et al. Predicting bacteremia at the bedside. Clin Infect Dis 2004;38:357–62.[doi:10.1086/380967](http://dx.doi.org/10.1086/380967)pmid:http://www.ncbi.nlm.nih.gov/pubmed/14727205
    
    [CrossRef](http://bmjopen.bmj.com/lookup/external-ref?access_num=10.1086/380967&link_type=DOI) 
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=14727205&link_type=MED&atom=%2Fbmjopen%2F11%2F5%2Fe044270.atom) 
    
    [Web of Science](http://bmjopen.bmj.com/lookup/external-ref?access_num=000188207600007&link_type=ISI) 

2.  Dawson S. Blood culture contaminants. J Hosp Infect 2014;87:1–10.[doi:10.1016/j.jhin.2014.02.009](http://dx.doi.org/10.1016/j.jhin.2014.02.009)pmid:http://www.ncbi.nlm.nih.gov/pubmed/24768211
    
    [CrossRef](http://bmjopen.bmj.com/lookup/external-ref?access_num=10.1016/j.jhin.2014.02.009&link_type=DOI) 
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=24768211&link_type=MED&atom=%2Fbmjopen%2F11%2F5%2Fe044270.atom) 

3.  Lee A, Mirrett S, Reller LB, et al. Detection of bloodstream infections in adults: how many blood cultures are needed? J Clin Microbiol 2007;45:3546–8.[doi:10.1128/JCM.01555-07](http://dx.doi.org/10.1128/JCM.01555-07)pmid:http://www.ncbi.nlm.nih.gov/pubmed/17881544
    
    [Abstract/FREE Full Text](http://bmjopen.bmj.com/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6MzoiamNtIjtzOjU6InJlc2lkIjtzOjEwOiI0NS8xMS8zNTQ2IjtzOjQ6ImF0b20iO3M6MjY6Ii9ibWpvcGVuLzExLzUvZTA0NDI3MC5hdG9tIjt9czo4OiJmcmFnbWVudCI7czowOiIiO30=) 

4.  Lamy B, Dargère S, Arendrup MC, et al. How to optimize the use of blood cultures for the diagnosis of bloodstream infections? A State-of-the art. Front Microbiol 2016;7:697. [doi:10.3389/fmicb.2016.00697](http://dx.doi.org/10.3389/fmicb.2016.00697)pmid:http://www.ncbi.nlm.nih.gov/pubmed/27242721
    
    [CrossRef](http://bmjopen.bmj.com/lookup/external-ref?access_num=10.3389/fmicb.2016.00697&link_type=DOI) 
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fbmjopen%2F11%2F5%2Fe044270.atom) 

5.  Bates DW, Goldman L, Lee TH. Contaminant blood cultures and resource utilization. The true consequences of false-positive results. JAMA 1991;265:365–9.pmid:http://www.ncbi.nlm.nih.gov/pubmed/1984535
    
    [CrossRef](http://bmjopen.bmj.com/lookup/external-ref?access_num=10.1001/jama.1991.03460030071031&link_type=DOI) 
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=1984535&link_type=MED&atom=%2Fbmjopen%2F11%2F5%2Fe044270.atom) 
    
    [Web of Science](http://bmjopen.bmj.com/lookup/external-ref?access_num=A1991ER25600026&link_type=ISI) 

6.  Takeshima T, Yamamoto Y, Noguchi Y, et al. Identifying patients with bacteremia in Community-Hospital emergency rooms: a retrospective cohort study. PLoS One 2016;11:e0148078. [doi:10.1371/journal.pone.0148078](http://dx.doi.org/10.1371/journal.pone.0148078)pmid:http://www.ncbi.nlm.nih.gov/pubmed/27023336
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fbmjopen%2F11%2F5%2Fe044270.atom) 

7.  Shapiro NI, Wolfe RE, Wright SB, et al. Who needs a blood culture? A prospectively derived and validated prediction rule. J Emerg Med 2008;35:255–64.[doi:10.1016/j.jemermed.2008.04.001](http://dx.doi.org/10.1016/j.jemermed.2008.04.001)pmid:http://www.ncbi.nlm.nih.gov/pubmed/18486413
    
    [CrossRef](http://bmjopen.bmj.com/lookup/external-ref?access_num=10.1016/j.jemermed.2008.04.001&link_type=DOI) 
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=18486413&link_type=MED&atom=%2Fbmjopen%2F11%2F5%2Fe044270.atom) 
    
    [Web of Science](http://bmjopen.bmj.com/lookup/external-ref?access_num=000259945600004&link_type=ISI) 

8.  Komatsu T, Takahashi E, Mishima K, et al. A simple algorithm for predicting bacteremia using food consumption and shaking chills: a prospective observational study. J Hosp Med 2017;12:510–5.[doi:10.12788/jhm.2764](http://dx.doi.org/10.12788/jhm.2764)pmid:http://www.ncbi.nlm.nih.gov/pubmed/28699938
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fbmjopen%2F11%2F5%2Fe044270.atom) 

9.  Komatsu T, Onda T, Murayama G, et al. Predicting bacteremia based on nurse-assessed food consumption at the time of blood culture. J Hosp Med 2012;7:702–5.[doi:10.1002/jhm.1978](http://dx.doi.org/10.1002/jhm.1978)pmid:http://www.ncbi.nlm.nih.gov/pubmed/23024032
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=23024032&link_type=MED&atom=%2Fbmjopen%2F11%2F5%2Fe044270.atom) 

10. Bossuyt PM, Reitsma JB, Bruns DE, et al. Stard 2015: an updated list of essential items for reporting diagnostic accuracy studies. BMJ 2015;351:h5527. [doi:10.1136/bmj.h5527](http://dx.doi.org/10.1136/bmj.h5527)pmid:http://www.ncbi.nlm.nih.gov/pubmed/26511519
    
    [FREE Full Text](http://bmjopen.bmj.com/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiRlVMTCI7czoxMToiam91cm5hbENvZGUiO3M6MzoiYm1qIjtzOjU6InJlc2lkIjtzOjE3OiIzNTEvb2N0MjhfMS9oNTUyNyI7czo0OiJhdG9tIjtzOjI2OiIvYm1qb3Blbi8xMS81L2UwNDQyNzAuYXRvbSI7fXM6ODoiZnJhZ21lbnQiO3M6MDoiIjt9) 

11. Takada T, Yamamoto Y, Terada K, et al. Diagnostic utility of appetite loss in addition to existing prediction models for community-acquired pneumonia in the elderly: a prospective diagnostic study in acute care hospitals in Japan. BMJ Open 2017;7:e019155. [doi:10.1136/bmjopen-2017-019155](http://dx.doi.org/10.1136/bmjopen-2017-019155)pmid:http://www.ncbi.nlm.nih.gov/pubmed/29122806
    
    [Abstract/FREE Full Text](http://bmjopen.bmj.com/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NzoiYm1qb3BlbiI7czo1OiJyZXNpZCI7czoxMjoiNy8xMS9lMDE5MTU1IjtzOjQ6ImF0b20iO3M6MjY6Ii9ibWpvcGVuLzExLzUvZTA0NDI3MC5hdG9tIjt9czo4OiJmcmFnbWVudCI7czowOiIiO30=) 

12. Tokuda Y, Miyasato H, Stein GH. The degree of chills for risk of bacteremia in acute febrile illness. Am J Med 2005;118:1417. [doi:10.1016/j.amjmed.2005.06.043](http://dx.doi.org/10.1016/j.amjmed.2005.06.043)pmid:16378800
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=16378800&link_type=MED&atom=%2Fbmjopen%2F11%2F5%2Fe044270.atom) 

13. Towns ML, Jarvis WR, Hsueh P-R. Guidelines on blood cultures. J Microbiol Immunol Infect 2010;43:347–9.[doi:10.1016/S1684-1182(10)60054-0](http://dx.doi.org/10.1016/S1684-1182(10)60054-0)pmid:http://www.ncbi.nlm.nih.gov/pubmed/20688297
    
    [CrossRef](http://bmjopen.bmj.com/lookup/external-ref?access_num=10.1016/S1684-1182(10)60054-0&link_type=DOI) 
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=20688297&link_type=MED&atom=%2Fbmjopen%2F11%2F5%2Fe044270.atom) 

14. Worster A, Carpenter C. Incorporation bias in studies of diagnostic tests: how to avoid being biased about bias. CJEM 2008;10:174–5.[doi:10.1017/S1481803500009891](http://dx.doi.org/10.1017/S1481803500009891)pmid:http://www.ncbi.nlm.nih.gov/pubmed/18371254
    
    [CrossRef](http://bmjopen.bmj.com/lookup/external-ref?access_num=10.1017/S1481803500009891&link_type=DOI) 
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=18371254&link_type=MED&atom=%2Fbmjopen%2F11%2F5%2Fe044270.atom) 

15. Janssen KJM, Donders ART, Harrell FE, et al. Missing covariate data in medical research: to impute is better than to ignore. J Clin Epidemiol 2010;63:721–7.[doi:10.1016/j.jclinepi.2009.12.008](http://dx.doi.org/10.1016/j.jclinepi.2009.12.008)pmid:http://www.ncbi.nlm.nih.gov/pubmed/20338724
    
    [CrossRef](http://bmjopen.bmj.com/lookup/external-ref?access_num=10.1016/j.jclinepi.2009.12.008&link_type=DOI) 
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=20338724&link_type=MED&atom=%2Fbmjopen%2F11%2F5%2Fe044270.atom) 
    
    [Web of Science](http://bmjopen.bmj.com/lookup/external-ref?access_num=000278412800007&link_type=ISI) 

16. Moons KGM, Donders RART, Stijnen T, et al. Using the outcome for imputation of missing predictor values was preferred. J Clin Epidemiol 2006;59:1092–101.[doi:10.1016/j.jclinepi.2006.01.009](http://dx.doi.org/10.1016/j.jclinepi.2006.01.009)pmid:http://www.ncbi.nlm.nih.gov/pubmed/16980150
    
    [CrossRef](http://bmjopen.bmj.com/lookup/external-ref?access_num=10.1016/j.jclinepi.2006.01.009&link_type=DOI) 
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=16980150&link_type=MED&atom=%2Fbmjopen%2F11%2F5%2Fe044270.atom) 
    
    [Web of Science](http://bmjopen.bmj.com/lookup/external-ref?access_num=000241064500012&link_type=ISI) 

17. Marshall A, Altman DG, Holder RL, et al. Combining estimates of interest in prognostic modelling studies after multiple imputation: current practice and guidelines. BMC Med Res Methodol 2009;9:57. [doi:10.1186/1471-2288-9-57](http://dx.doi.org/10.1186/1471-2288-9-57)pmid:http://www.ncbi.nlm.nih.gov/pubmed/19638200
    
    [CrossRef](http://bmjopen.bmj.com/lookup/external-ref?access_num=10.1186/1471-2288-9-57&link_type=DOI) 
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=19638200&link_type=MED&atom=%2Fbmjopen%2F11%2F5%2Fe044270.atom) 

18. Shrout PE. Measurement reliability and agreement in psychiatry. Stat Methods Med Res 1998;7:301–17.[doi:10.1177/096228029800700306](http://dx.doi.org/10.1177/096228029800700306)pmid:http://www.ncbi.nlm.nih.gov/pubmed/9803527
    
    [CrossRef](http://bmjopen.bmj.com/lookup/external-ref?access_num=10.1177/096228029800700306&link_type=DOI) 
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=9803527&link_type=MED&atom=%2Fbmjopen%2F11%2F5%2Fe044270.atom) 

19. Glas AS, Lijmer JG, Prins MH, et al. The diagnostic odds ratio: a single indicator of test performance. J Clin Epidemiol 2003;56:1129–35.[doi:10.1016/S0895-4356(03)00177-X](http://dx.doi.org/10.1016/S0895-4356(03)00177-X)pmid:http://www.ncbi.nlm.nih.gov/pubmed/14615004
    
    [CrossRef](http://bmjopen.bmj.com/lookup/external-ref?access_num=10.1016/S0895-4356(03)00177-X&link_type=DOI) 
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=14615004&link_type=MED&atom=%2Fbmjopen%2F11%2F5%2Fe044270.atom) 
    
    [Web of Science](http://bmjopen.bmj.com/lookup/external-ref?access_num=000186694000013&link_type=ISI) 

20. Cook NR. Statistical evaluation of prognostic versus diagnostic models: beyond the ROC curve. Clin Chem 2008;54:17–23.[doi:10.1373/clinchem.2007.096529](http://dx.doi.org/10.1373/clinchem.2007.096529)pmid:http://www.ncbi.nlm.nih.gov/pubmed/18024533
    
    [Abstract/FREE Full Text](http://bmjopen.bmj.com/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6ODoiY2xpbmNoZW0iO3M6NToicmVzaWQiO3M6NzoiNTQvMS8xNyI7czo0OiJhdG9tIjtzOjI2OiIvYm1qb3Blbi8xMS81L2UwNDQyNzAuYXRvbSI7fXM6ODoiZnJhZ21lbnQiO3M6MDoiIjt9) 

21. Eliakim-Raz N, Bates DW, Leibovici L. Predicting bacteraemia in validated models--a systematic review. Clin Microbiol Infect 2015;21:295–301.[doi:10.1016/j.cmi.2015.01.023](http://dx.doi.org/10.1016/j.cmi.2015.01.023)pmid:http://www.ncbi.nlm.nih.gov/pubmed/25677625
    
    [CrossRef](http://bmjopen.bmj.com/lookup/external-ref?access_num=10.1016/j.cmi.2015.01.023&link_type=DOI) 
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fbmjopen%2F11%2F5%2Fe044270.atom) 

22. Leibovici L, Greenshtain S, Cohen O, et al. Bacteremia in febrile patients. A clinical model for diagnosis. Arch Intern Med 1991;151:1801–6.pmid:http://www.ncbi.nlm.nih.gov/pubmed/1888246
    
    [CrossRef](http://bmjopen.bmj.com/lookup/external-ref?access_num=10.1001/archinte.1991.00400090089016&link_type=DOI) 
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=1888246&link_type=MED&atom=%2Fbmjopen%2F11%2F5%2Fe044270.atom) 
    
    [Web of Science](http://bmjopen.bmj.com/lookup/external-ref?access_num=A1991GE53400016&link_type=ISI) 

23. Harrel FE. Regression modeling strategies: with applications to linear models, logistic regression, and survival analysis. New York: Springer, 2015.
    
    

24. Wilson EB. Probable inference, the law of succession, and statistical inference. J Am Stat Assoc 1927;22:209–12.[doi:10.1080/01621459.1927.10502953](http://dx.doi.org/10.1080/01621459.1927.10502953)
    
    [CrossRef](http://bmjopen.bmj.com/lookup/external-ref?access_num=10.2307/2276774&link_type=DOI) 
    
    [Web of Science](http://bmjopen.bmj.com/lookup/external-ref?access_num=000188338900021&link_type=ISI) 

25. Lautenbach E, Localio R, Nachamkin I. Clinicians required very high sensitivity of a bacteremia prediction rule. J Clin Epidemiol 2004;57:1104–6.[doi:10.1016/j.jclinepi.2004.03.003](http://dx.doi.org/10.1016/j.jclinepi.2004.03.003)pmid:http://www.ncbi.nlm.nih.gov/pubmed/15528062
    
    [CrossRef](http://bmjopen.bmj.com/lookup/external-ref?access_num=10.1016/j.jclinepi.2004.03.003&link_type=DOI) 
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=15528062&link_type=MED&atom=%2Fbmjopen%2F11%2F5%2Fe044270.atom) 

26. Hodgson LE, Dragolea N, Venn R, et al. An external validation study of a clinical prediction rule for medical patients with suspected bacteraemia. Emerg Med J 2016;33:124–9.[doi:10.1136/emermed-2015-204926](http://dx.doi.org/10.1136/emermed-2015-204926)pmid:http://www.ncbi.nlm.nih.gov/pubmed/26246024
    
    [Abstract/FREE Full Text](http://bmjopen.bmj.com/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NzoiZW1lcm1lZCI7czo1OiJyZXNpZCI7czo4OiIzMy8yLzEyNCI7czo0OiJhdG9tIjtzOjI2OiIvYm1qb3Blbi8xMS81L2UwNDQyNzAuYXRvbSI7fXM6ODoiZnJhZ21lbnQiO3M6MDoiIjt9) 

27. Jessen MK, Mackenhauer J, Hvass AMSW, et al. Prediction of bacteremia in the emergency department: an external validation of a clinical decision rule. Eur J Emerg Med 2016;23:44–9.[doi:10.1097/MEJ.0000000000000203](http://dx.doi.org/10.1097/MEJ.0000000000000203)pmid:http://www.ncbi.nlm.nih.gov/pubmed/25222426
    
    [CrossRef](http://bmjopen.bmj.com/lookup/external-ref?access_num=10.1097/mej.0000000000000203&link_type=DOI) 
    
    [PubMed](http://bmjopen.bmj.com/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fbmjopen%2F11%2F5%2Fe044270.atom)