Article Text
PDF
Abstract
Objectives Intensive care unit (ICU) clinicians stop antibiotics more often, with a negative infection: point-of-care test (PCR-POCT). Simulated cases of diagnostic uncertainty regarding infection resolution led clinicians to choose options such as procalcitonin (PCT) and/or PCR-POCTs +/− de-escalation to aid stop decisions. We hypothesised that a direct infection indicator, PCR-POCT, would influence stop judgements more than indirect PCT. Accordingly, we tested antibiotic-stop decisions when presented with a negative PCR-POCT despite borderline-positive PCT.
Designs Observational prospective study.
Setting ICU.
Participants 66 ICU clinicians from University hospitals.
Methods Clinicians saw four scenarios of different clinico-biological trajectories: (1) clear improvement, (2) clear worsening, (3) discordant—clinically better/biologically worse and (4) discordant—clinically worse/biologically better. Participants gave an initial decision (stop/continue/continue–escalate/continue–de-escalate). Then PCR-POCT and/or PCT was offered (accept/decline). After a negative PCR-POCT and borderline-positive PCT result, a final antibiotic decision was taken.
Measures Proportion of stop decisions before versus after test results per scenario. The association of the final decision with the clinician’s change in confidence, willingness to request the biomarker(s) and the case trajectory was determined.
Results Fewer clinicians than expected stopped antibiotics versus baseline (36%, 94/264 vs 42%, 110/264, p=0.045). This was so in three of four scenarios, significantly less in the improvement (p<0.001) and the discordant clinically better scenario (p=0.024). PCT was requested more frequently than PCR-POCT (61% vs 53%, p<0.001). PCT requesters (vs declining) were significantly less inclined to stop antibiotics (p<0.001), while PCR-POCT requesting led to more stopping (p<0.001), before knowing the test results.
Conclusions A negative PCR-POCT result did not increase clinicians’ inclination to stop antibiotics when alongside a borderline-positive PCT. This reflects clinicians’ natural risk aversion. PCT was more popular than PCR-POCT, but PCR-POCT was more likely to aid stop decisions.
Their comparison, role, utility and selective deployment for influencing antibiotic-stop decisions more effectively require a large randomised controlled trial.
- Behaviour
- MICROBIOLOGY
- INFECTIOUS DISEASES
- INTENSIVE & CRITICAL CARE
- Diagnostic microbiology
Data availability statement
Data is available in a public, open access repository. Data ia available at 10.17605/OSF.IO/7GDES 29
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
STRENGTHS AND LIMITATIONS OF THIS STUDY
In four typical clinical vignettes, with different clinico-biological trajectories, this study offered a realistic simulation of Intensive care unit-related respiratory infection factors to test antibiotic stewardship decisions.
Choices to escalate/de-escalate antibiotics alongside stop and continue options, provide a reproducible and adaptable test platform to study the clinical situational and behavioural factors that influence antibiotic decisions.
By focussing on the end (rather than onset) stage of infection, it offered the opportunity to test clinicians’ preferences for direct (PCR-POCT) or indirect (PCT) point-of-care tests as arbiters for antibiotic decisions when confronted with clinic-biological diagnostic uncertainty.
A larger sample is required to robustly determine preferences between PCR-POCT or PCT, and influential factors. This would have to use the same choices (eg, continue/stop/(de)escalate), while adding the presentation of theoretical combinations of positive and negative results, using the same test vignette platform.
Background
Antimicrobial resistance has become an increasingly pertinent issue within patient care.1 Antibiotic stewardship programmes (ASPs) are strategies that aim to improve the use of antibiotics and have been employed to reduce the potential for antimicrobial resistance. Antibiotic prescribing is high in the intensive care unit (ICU) setting, with up to 70% of patients having an antibiotic prescribed, and so the use of ASPs in this setting can have large effects on antimicrobial resistance.2 Some ASPs have had success in reducing antibiotic usage, including procalcitonin (PCT)-guided antibiotic stewardship for sepsis in the ICU.3 PCT is a surrogate biomarker for infection and is considered a useful tool in antibiotic prescribing, having been successful in reducing antibiotic course lengths in study settings.4–7 Another useful tool to potentially improve antibiotic prescribing is the PCR point-of-care test (PCR-POCT, herein also referred to as POCT). These tests have been used during the COVID-19 pandemic and are found to have high diagnostic accuracy.8 Infection identifying POCTs can effectively rule out the presence of infective organisms, thereby increasing clinicians’ confidence to stop antibiotics.9 This is important as clinicians are found to continue (rather than stop) empirical antibiotics when there is clinical uncertainty, based on natural risk averseness, particularly with more severe illness.10 POCT can alleviate this uncertainty and should therefore reduce antibiotic prescribing. However, recent studies have not shown this. In the VAPrapid trial of ventilator-associated pneumonia, the use of a highly accurate cytokine-based POCT (interleukin 1/8) rule-out test failed to increase antibiotic-free days in ICU, despite excellent test performance.11
One possible explanation is that factors such as cognitive biases may be impacting the use of these tests and antibiotic decision-making more broadly. Confirmation bias, whereby clinicians find and interpret evidence to support an existing hypothesis, could affect decision-making.12 13 Anchoring, whereby clinicians fixate on early information in a case, may lead clinicians to overlook important possibilities.14 While these clinician factors have been acknowledged in previous work, research investigating their specific and quantifiable effect on decision-making is lacking.15
A recent study investigated clinical and clinician factors that influence POCT use and antibiotic prescribing in ICU.16 This vignette-based experiment found that a negative PCR-POCT result (suggesting no infection) significantly increased clinicians’ inclination to stop antibiotics, but three ‘competing’ factors worked to decrease it: an ambiguous or worsening patient trajectory, clinicians’ first impressions (ie, high confidence that antibiotics were needed) and lack of interest in POCT (ie, rejection of the POCT when it was offered). While that study highlighted the potential utility of POCT for antibiotic cessation in the setting of ICU-related respiratory infection, as well as the factors that might diminish its utility, the model was simple with limitations. Four vignettes describing patients who had just completed a course of antibiotics were constructed: one was clearly improving (clinical and biological signs better than on admission), one was clearly deteriorating (clinical and biological signs worse) and two were ambiguous (clinical signs better but biological signs worse/clinical signs worse but biological signs better). The study found that POCT was most requested and most effective (in promoting antibiotic-stop decisions) in the ambiguous scenarios that featured clinical worsening but biological improvement (and overall worsening). This is important as a critical component of increasing and streamlining POCT use in the ICU is identifying scenarios in which it is most/least helpful. For reliability, those findings would benefit from replication in a new set of vignettes that adhere to the same structure (ie, a new vignette depicting clear improvement, a new vignette depicting clear worsening, etc).
In the original study, participants’ uptake of the POCT offer was relatively high (overall 65% of the time). In clinical practice, however, clinicians have other tests available, which are more established and therefore more widely used than POCT (eg, PCT). Whether clinicians would still request POCT when PCT is available (although an indirect biomarker surrogate of an infectious agent rather than a direct form in PCR-POCT) remains to be seen.
Thirdly, in the original study, clinicians were only asked to choose between two courses of action: stop or continue antibiotic treatment; in reality, the opportunity to (de)escalate is a further option. Recognising this limitation, the authors asked participants at the end of the study: “Had the option to de/escalate antibiotics been available, would you have used it?” Most clinicians (74.3%, 52/70) responded yes, leading the authors to conclude that findings may have looked different had (de)escalation been available.
The present study aimed to address this limitation. Specifically, to replicate and validate the previous findings, by making three improvements to the study design:
Clinicians were presented with a new set of similar vignettes (one improving, one worsening, two ambiguous) assessing the reliability of the model.
Clinicians were offered a POCT and a PCT test (they could select either, neither or both), with the two providing conflicting results (POCT negative, PCT positive). This allowed us to assess whether there is a systematic preference for either POCT or PCT, and how clinicians consolidate disparity between the two in their decision-making.
Clinicians had the option to stop, continue, escalate or de-escalate antibiotics.
We hypothesised that a negative POCT result would increase clinicians’ inclination to stop antibiotics, as in the previous study (hypothesis 1).16 We also expected that this effect would be smaller than that observed in the previous study (hypothesis 2) because
The presence of a borderline-positive PCT result would reduce clinicians’ inclination to stop since clinicians may have greater trust in the (more established) PCT test than the (less established) PCR-POCT or its implication of the possible presence of infection may mitigate the negative POCT result.
Allowing clinicians to (de)escalate antibiotics would reduce the incidence of stopping as de-escalation may be perceived as a ‘safer’ (ie, less risky) alternative.
Methods
Participants
Consultants and trainees from the ICU (with 3+ months of continuous experience in ICU) working in London-based university hospitals were invited through advertisements in closed social media groups exclusive to ICU consultants and trainees. Clinicians who took part in the previous study16 and had indicated a willingness to be involved in future studies were contacted via direct email. The email contained a direct link to the online survey hosted by Qualtrics (Washington, USA). The survey remained open from March to October 2022.
Sample size
A minimum of 77 responses were required to demonstrate the same effect size of an increase in antibiotic-stop decisions from 54% to 70%16 with 80% power at the alpha=0.05 level. Further details are available in the online supplemental material (SM1).
Supplemental material
Importantly, however, we expected the effect to be smaller in the present study than previous one (see hypothesis 2). We therefore conducted a second sample size calculation, identical to the first, except that it aimed to detect a smaller effect ( w =0.20). After adjusting for clustered data, the number of responses required was 233, and the number of participants required was 58 (233/4). We therefore aimed to recruit 58 participants.
Materials
We constructed four clinical scenarios for resolving lung infection after a course of antibiotics. Each scenario comprised clinical and biological data, which were varied to create four distinct patient trajectories (table 1).
Clinical vignettes used in this study
These vignettes were thought to accurately represent the varying degrees of diagnostic (un)certainty commonly encountered in the critical care setting. Two of the scenarios (improvement and worsening) acted as ‘positive’ and ‘negative’ controls, in that the trajectory clearly supported stopping antibiotics (improvement) or continuation/escalation of antibiotics (worsening). The remaining two scenarios presented uncertainty with regard to antibiotic decision-making and were used to explore the importance of the clinical versus biological trajectories.
In each scenario, two test results were made available: POCT and PCT. Both tests were described as valid and reliable. The POCT was an infection-identifying PCR test that provided rapid diagnostics for bacteria. The POCT result (unbeknown to clinicians at the point of request) was always negative (suggesting no active infection within the lung), while the PCT result was always marginally positive (suggesting possible infection). Notably, the POCT in these scenarios was an indicator of the actual presence of an infectious organism, whereas the PCT test was a surrogate marker for the presence of infection. The conflicting results of these two tests would allow us to compare how a direct versus indirect test for infection might influence clinicians’ choices.
Procedure
Clinicians provided informed written consent before being able to proceed further in the online survey. Following this, they responded to the four vignettes, presented in a random order. Each vignette began with a brief patient description, including the patient’s age, sex, details of admission and clinical status 1 week later (ie, after completing a course of antibiotics). Based on this, clinicians were asked to decide the best course of action with regard to antibiotics (stop antibiotics/continue with current course of antibiotics/escalate antibiotics/de-escalate antibiotics). They were also asked to rate their confidence in this decision on a 6-point Likert scale anchored at 1=not at all confident and 6=extremely confident.
Clinicians were then informed of the availability of a POCT and PCT test. Clinicians could select the test/s that they wished to perform (POCT only/PCT only/both POCT and PCT/neither POCT nor PCT). They were also asked to indicate the reason(s) for this decision online supplemental material (SM3).
Clinicians who chose to perform one of the two tests (POCT only/PCT only) were at this point presented with the result of the chosen test (POCT was always negative, PCT always positive) and asked whether they would change their previous antibiotics decision (yes/no). Those who responded yes were offered the previous antibiotics options again (stop/original course/escalate (more than original course)/de-escalate (less than original course)). Regardless of whether their decision had changed, they were asked to indicate their confidence in their decision (1–6, as above) and to explain their decision (free text response). Following this, the second (non-requested) test result was displayed (POCT was always negative/PCT was always positive) and clinicians were asked the same questions again; that is, they were asked whether they would change their decision (if yes, they were presented with the four antibiotics options described above), to indicate their confidence in their decision (1–6) and explain their decision (free text).
Clinicians who chose to perform both or neither of the tests (both POCT and PCT/neither POCT nor PCT) were shown the results of both tests simultaneously (POCT negative, PCT positive) and asked exactly the same questions; that is, whether they would change their decision (yes/no) and what they would change it to if so (stop/original course/escalate (more than original course)/de-escalate (less than original course)), confidence in decision (1–6) and rationale for decision (free text).
Importantly, therefore, clinicians always gave an ‘initial’ antibiotic decision (no test results seen) and a ‘final’ antibiotic decision (both test results seen). When clinicians chose to perform one of the two tests, they also gave ‘interim’ decisions (that one test result was seen).
After completing all four scenarios, clinicians were also asked to complete Grol et al’s Attitudes to Risk-Taking in Medical Decision Making 17 questionnaire (adapted to the ICU setting) and provide demographic information (gender and level of training). The study procedure is presented graphically in online supplemental material (SM4). Details of the piloting processes are presented in online supplemental material (SM5).
Statistical analysis
To measure the combined effect of the test results (negative POCT and positive PCT) on clinicians’ inclination to stop antibiotics, we compared the proportion of clinicians who chose to stop antibiotics initially versus finally using χ2 analysis. We had intended to also measure and compare the effects of each test result by analysing the interim decisions of those who selected POCT only or procalcitonin only; however, very few participants selected these options (see ‘Results’), rendering such an analysis unreliable.
To explore this further, we created a continuous variable termed ‘willingness-to-stop’ by signing confidence ratings in accord with antibiotic decisions. Specifically, confidence ratings (1=not at all confident to 6=extremely confident) were signed positive (+) if clinicians chose to stop antibiotics or negative (−) if they chose to continue (be it in the form of escalation, de-escalation or continuation of original course). Initial and final willingness-to-stop antibiotics (both ranging from −6=lowest to 6=highest) were compared using Wilcoxon signed ranked tests (non-parametric, as the willingness-to-stop variables were not normally distributed, see ‘Results’).
We also explored whether clinicians might have a preference for POCT versus PCT tests by examining the proportion of participants who requested (vs rejected) each test. We explored whether this might differ by scenario using χ2 analysis.
Finally, we explored the effect of scenario (1=improvement, 2=disc clin better/worse, 3=worsening), initial antibiotic decision (0=escalate, 1=original course, 2=de-escalate, 3=stop), test(s) requested (1=requested and 0=rejected for POCT and procalcitonin, respectively), Attitude To Risk-Taking score (the per-participant sum of responses to Grol et al’s questionnaire) and level of experience (0=trainee, 1=consultant) on final antibiotic decisions (0=escalate, 1=original course, 2=de-escalate, 3=stop), using a mixed effects ordinal logistic regression model with a per-participant random intercept.
A p value<0.05 was considered statistically significant. Statistical analysis and graphing were performed using SPSS 28 (IBM, New York, USA) and Stata/MP 17 software (StataCorp, Texas, USA).
Approval
This study was approved by the Imperial College Research Ethics Committee (ICREC reference 20IC6499) and the manuscript adheres to CHERRIES guidance for reporting e-survey results.18
Patient and public involvement
None.
Results
Demographic data
66 clinicians completed the survey. The number of clinicians who accessed the survey is unknown as incomplete responses were deleted automatically, following 1 week of inactivity. All participants completed all four scenarios, providing a total of 264 scenario responses. Of the 66 clinicians, 39 (59.1%) were male. There were 34 (51.5%) consultants, 17 (25.8%) specialist registrar clinicians, 13 (19.7%) senior house officer clinicians and 2 (3%) Foundation Year clinicians. The demographics of the sample can be found in online supplemental material (SM6).
Initial and final antibiotic decisions
Figure 1 shows participants’ initial and final antibiotic decisions. Prior to receiving any test results (POCT or PCT), participants opted to stop antibiotics 41.7% of the time (110/264). This was reduced to 35.6% (94/264) following the negative POCT and positive PCT test. Therefore, clinicians were less likely (rather than more likely) to stop antibiotics after receiving the test results (χ2=4.03, df=1, p=0.045). Few clinicians chose the path of de-escalation initially or subsequent to the POCT/PCR results, with most opting to stop or escalate.
Number and proportion of decisions made initially (before receiving any test results) versus finally (after receiving the negative POCT and positive PCT results), regardless of scenario (n=264). Participants chose between the options of escalate (more than the original course), continue with the original course, de-escalate (less than the original course) and stop with regard to antibiotics. PCT, procalcitonin; POCT, point-of-care test.
The frequency of stop decisions within each scenario is displayed in figure 2. Within the improvement, disc clin better and disc clin worse scenarios, fewer participants chose to stop antibiotics in their final decision (following both test results) compared with their initial decision. This effect was highly significant in the improvement scenario (72.7% vs 86.4%; χ2=10.50, df=1, p=0.001), significant in disc clin better (28.8% vs 42.4%; χ2=5.01, df=1, p=0.025) and non-significant in disc clin worse (36.4% vs 37.9%, χ2=0.07, df=1, p=0.797). In the worsening scenario, no participant chose to stop antibiotics initially and was statistically unchanged following the test results (4.5%, p=ns). For a full breakdown of initial and final antibiotic decisions per scenario (ie, number of participants who elected to stop, continue with the original course, escalate and de-escalate), see online supplemental material (SM7).
Number of clinicians who chose to stop antibiotics initially (ie, before receiving any test results) versus finally (after receiving both a negative POCT result and a positive PCT result), per scenario. The total number of responses in each scenario (improvement, disc clin better, disc clin worse and worsening) was 66. *p<0.05. PCT, procalcitonin; POCT, point-of-care test.
Initial and final willingness-to-stop were non-parametric following inspection of frequency distribution histograms. This was confirmed using Kolmogorov-Smirnov tests (p<0.001 for both initial and final willingness-to-stop). Figure 3 shows clinicians’ median willingness-to-stop before and after receiving the negative POCT and positive PCT results, per scenario. Similar to figure 2, willingness-to-stop appeared to decrease from initial to final decision in the improvement, disc clin better and disc clin worse scenarios; the effect was significant in the improvement scenario (z=−3.84, p<0.001, effect size r=0.33), significant in the disc clin better scenario (z=−2.56, p=0.010, r=0.22) and non-significant in the disc clin worse scenario (z=−0.11, p=0.909, r=0.01). In the worsening scenario, willingness-to-stop was greater in the final decision than the initial decision, suggesting that the negative POCT might increase clinicians’ willingness-to-stop in this specific scenario. This effect was significant (z=−3.04, p=0.002, r=0.26). In figure 3, the wide boxes in the disc clin better and disc clin worse scenarios suggest a greater variation in willingness-to-stop, as opposed to the narrower boxes in the improvement and worsening scenarios. The evolution of the willingness-to-stop from the initial decision, through an interim option and then the final decision, is shown in online supplemental material (SM8).
Median willingness-to-stop before versus after receiving the negative POCT and positive PCT results, per scenario (n=66 for each scenario). Willingness-to-stop represents a participant’s confidence (1–6), signed positive if they chose to stop antibiotics and negative if they chose to continue (be it via continuation of the original course, escalation and de-escalation). *p<0.05. PCT, procalcitonin; POCT, point-of-care test.
POCT and PCT requests
Across all scenarios, POCT was requested 53.4% of the time (141/264); however, this varied significantly by scenario (χ2=55.97, df=3, p<0.001; improvement=15.2%, disc clin better=57.6%, disc clin worse=68.2%, worsening=72.7%). Similarly, PCT was requested 61% of the time (161/264) and also varied significantly by scenario (χ2=52.01, df=3, p<0.001; improvement=24.2%, disc clin better=74.2%, disc clin worse=78.8%, worsening=66.7%).
Figure 4 displays the tests requested by clinicians per scenario. In the improvement scenario, the majority of participants requested neither the POCT nor PCT test (72.7%), potentially due to the patient’s unambiguously positive trajectory (ceiling effect). Within the disc clin better, disc clin worse and worsening scenarios, most participants requested both tests (53.0%, 60.6% and 60.6%, respectively). Participants’ reasons for requesting/rejecting the tests were mainly related to supplementing clinical judgement and deeming tests necessary or not (online supplemental material (SM9)).
Number of participants requesting POCT and/or PCT tests, per scenario (n=66 for each scenario). Participants were offered a POCT and PCT test within each scenario, after making an initial antibiotics decision, and could request POCT only, PCT only, both tests or neither test. PCT, procalcitonin; POCT, point-of-care test.
Factors influencing the final decision
Our mixed effects model explored the effect of (1) initial antibiotic decisions, (2) test/s requested, (3) attitudes towards risk-taking, (4) levels of experience and (5) scenarios on final antibiotic decisions. The results showed that initial antibiotic decisions had a strong effect on final antibiotic decisions (OR=8.70 (95% CI 5.40 to 14.02), p<0.001). That is, clinicians who were more (less) inclined to stop antibiotics prior to receiving the test results were also more (less) inclined to stop after receiving them. This was consistent with the original study.16 Additionally, participants who requested PCT (either alone or in conjunction with POCT) were less inclined to stop in their final antibiotics decision (OR=0.26 (0.12 to 0.55), p<0.001), whereas participants who requested POCT (either alone or in conjunction with PCT) were more inclined to stop in their final antibiotics decision (OR=3.84 (1.61 to 9.15), p=0.002). This was consistent with the original study.16 Level of experience (trainee or consultant) had no significant effect on final antibiotics decisions (OR=1.56 (0.85 to 2.86), p=0.156), nor did differences in attitudes towards risk-taking (OR=1.02 (0.92 to 1.12), p=0.761) or scenario (OR=0.63 (0.32 to 1.22), p=0.168). These findings did not change when we replaced initial and final antibiotic decisions with initial and final willingness-to-stop (online supplemental file (SM 10)).
Discussion
Antibiotic-stop decisions did not increase following a negative PCR-POCT result and borderline-positive PCT result, after a completion course of antibiotics in ICU-related respiratory infection. Rather, in three out of four scenarios (improvement, disc clin better and disc clin worse), the stop rate decreased. This differed from the first WHYSTOP study,16 where antibiotic-stop decisions increased consistently (ie, in all scenarios) following receipt of a negative POCT. Thus, a negative POCT did not increase stop decisions when there was a borderline-positive PCT result.
A few potential explanations for the lack of increase in antibiotic-stop decisions are likely. First, the addition of a borderline-positive PCT result likely negated the effect of the negative POCT and so reduced clinicians’ inclination to stop. Second, participants may have been more inclined to trust the PCT result (vs newer non-established PCR-POCT) due to its known potential and growing evidence base within clinical medicine.4 6 7 Alternatively, participants may have had equal trust in PCT and POCT, but conflicting test results triggered a conservative approach to ‘err on the side of caution’ (ie, continue antibiotics).10 This was likely as demonstrated in the original WHYSTOP study. Specifically, in that study, the improvement scenario was extended to include the following ‘twist’: the original negative PCR-POCT result was declared erroneous (due to lab error) and re-testing gave a positive result. In light of this, clinicians were given the opportunity to revise their antibiotic decisions. The proportion of antibiotic-stop decisions, even in this clinico-biological case of improvement (suggesting the resolution of infection), fell from 90% to 61%.16 The caution adopted relates to ‘prospect theory’; the idea that a loss and regret are psychologically twice as impactful as a gain; the context being potential patient harm from an antibiotic-stop decision.19 20 Another explanation may be that the clin better, biologically worse scenario may have been slightly more ambiguous in its clinical trajectory than in the original paper, which may have accounted for less confidence in choosing to stop antibiotics post POCT.
Finally, we expected that the addition of a de-escalation option might reduce the incidence of stopping antibiotics (by providing a ‘less risky’ alternative) and thus weaken the effect of the negative POCT. Contrary to expectation, the de-escalation option was rarely used (8.1% of initial decisions and 9.4% of final decisions) and is therefore unlikely to account for the present findings.
Participants who actively requested (vs passively received) the POCT result were more inclined to stop antibiotics, whereas those who actively requested (vs passively received) the PCT result were less inclined to stop. Clearly, interpretation and/or weighting of test results does not take place in isolation, but is dependent on the perceived relevance, and/or value of the test in the decision-making. This suggests that ‘forcing’ POCT on clinicians is unlikely to bring about effective change. Rather increasing awareness and trust in POCT may allow for greater confidence in its use and increase uptake. Indeed, Dhesi et al identified lack of clinician trust as a potential barrier to POCT adoption in practice, which must be addressed if POCT is to be of value in ICU settings.21 This was also a potential reason for the lack of improvement in patient outcomes in a quasi-randomised controlled trial, where use of PCR-POCT was compared with routine laboratory PCR in guiding clinical management.22
Clinicians’ grade (consultant vs trainee) did not influence the inclination to stop antibiotics, nor did clinicians’ attitude towards risk-taking. The WHYSTOP study was underpowered to address this.16 However, in a subsequent study, a difference in clinical decision-making was noted between novices (ie, clinical medical students) and clinicians.23 This may be of particular interest, given the potential for learnt behaviour on antibiotic decision-making.24 25 Indeed, medical students, tested in the same conditions of the original WHYSTOP study, were initially more conservative than ICU clinicians in STOP decisions. More students chose the POCT when offered than clinicians, and the negative result increased the proportion of stop decisions in all scenarios, to the same level as clinicians.23
We made changes to the WHYSTOP study protocol to better reflect reality. These included new clinical vignettes with the same four trajectories, introduction of an additional optional PCT test and (de)escalation opportunities. Within these new, more realistic scenarios, a negative POCT, alongside a positive PCT, did not increase antibiotic-stop decisions. This study has demonstrated that contrasting test results and other behavioural factors may impair the utility of POCT. This may explain the lack of effect of POCT in improving patient outcomes (including length of hospital stay and antibiotic-free days) seen in some trials investigating the use of other point-of-care biomarkers.11 22 Other studies have demonstrated the potential for PCR-POCT in critical care. The INHALE WP1 study investigated two PCR-based tests for the diagnosis of pneumonia, with both tests proving to be more sensitive than routine microbiology.8 As new PCR-POCTs are emerging and developing, there is a clear need for further investigation of the situations where use of POCT for infection should be implemented.26
Finally, the diminished effect of the negative PCR-POCT in these scenarios may simply have been due to the reintroduction of further ambiguity (ie, the borderline-positive PCT). Whether ambiguity is present in the clinico-biological setting (ie, the two discordant scenarios) or introduced through competing signals of the test results, it is anticipated that the requested POCT measure will act as a ‘final arbiter’ when diagnostic uncertainty remains. In the higher risk setting of ICU infection decisions, most clinicians adopt caution.10 To provide clarity then, it may be sensible to choose just one rather than both POCT as this ‘final’ arbiter.
Strengths and limitations
This study offered realistic conditions and options to clinicians when making their antibiotic-stop decisions. Specifically, it explored clinicians’ preference for POCT versus PCT (if any) in four different clinico-biological trajectories, while enabling (de)escalation options. Furthermore, we made the PCT result borderline positive so as to (1) better represent the complexities and uncertainties of clinical practice and (2) stress-test the effect of a negative PCR-POCT (ie, assess its influence in the face of conflicting biomarker information).
We were unable to reliably analyse interim decisions due to the small number of participants who requested either POCT or PCT. A larger sample size may have increased the size of these subgroups, allowing us to isolate and compare the respective effects of POCT and PCT on antibiotic decision-making. Another way to compare their effects would be to vary their results (positive vs negative) systematically. Presently, the effects of POCT and PCT are intertwined, which limits the conclusions that we can draw. While these conclusions are interesting and informative, further work is needed to gauge any hierarchical influence of POCT and PCT. A direct comparison of POCT versus PCT (with both being available but mutually exclusive) may help answer other desired but unanswered questions: which (if any) is preferred between a direct or indirect biomarker of infection and which (if any) has a greater effect on final antibiotic decisions?
The study was sufficiently powered to detect a modest increase in antibiotic-stop decisions before versus after POCT/PCT results across scenarios. All other statistical tests and sub-analyses were exploratory and should be interpreted with caution. We did not adjust the significance threshold for multiple comparisons, but our findings were frequently robust (p<0.001) and would likely remain significant even with such an adjustment, which supports their validity. Generalisability may have been limited by the recruitment of clinicians from mainly academic training programmes. The influence of ‘system-level noise’ cannot also be underestimated. ‘Noise’ refers to variation within decision-making and here with regard to clinical practice.27 At a system level, variation between academic centres and hospitals (with different antibiotic prescribing policies and in-house training) can lead to noise. For example, clinicians in one hospital may be more (less) familiar with a given test than clinicians in another hospital due to higher (lower) utilisation of that test in their hospital.
Conclusion
A negative POCT result does not appear to increase clinicians’ inclination to stop antibiotics when presented alongside a borderline-positive PCT test. Conflicting test results could thus be one reason why POCT has failed to increase antibiotic-free days in the ICU.11 22 Further research investigating the behavioural and trajectorial factors that might compete with or override POCT in decision-making, alongside initiatives to increase clinicians’ confidence in POCT, is imperative to improve its utility in the ICU. Ultimately, recognising the uncertainty in prescribing decision-making, how it affects clinicians and developing decision-making tools to support them in avoiding over-reliance on antibiotics should be a future endeavour to improve antibiotic stewardship.28
Data availability statement
Data is available in a public, open access repository. Data ia available at 10.17605/OSF.IO/7GDES 29
Ethics statements
Patient consent for publication
Ethics approval
This study involves human participants and was approved by Imperial College Research Ethics Committee (ICREC reference 20IC6499). Participants gave informed consent to participate in the study before taking part.
Acknowledgments
We would like to thank and acknowledge Dr Sonia Mason, Dr Marco Morosin and Dr Pascal Frederiks for their assistance in piloting the survey. In addition, we would like to express appreciation for all the clinicians, part of the WHYSTOP consortium, who participated in this study.
References
Footnotes
X @dr_luke_moore, @respitudoc
TL, MN and SS contributed equally.
Contributors SS conceived the idea. Guarantor is SS. SS, MN and TL developed the protocol. MN and TL created the data collection tool. SS, MN and TL analysed the data. SS, MN and TL wrote the first and subsequent drafts. SS, MN, TL, AS, LM and NM reviewed the manuscript.
Funding The authors have not declared a specific grant for this research from any funding agency in the public, commercial or not-for-profit sectors.
Competing interests None declared.
Patient and public involvement Patients and/or the public were not involved in the design, or conduct, or reporting, or dissemination plans of this research.
Provenance and peer review Not commissioned; externally peer reviewed.
Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.