Jump to comment:

• Spa Therapy Is Not a Pill: Reconsidering Methods in the Evaluation of Complex Interventions
  Romain Jacques FORESTIER, Fatma Begüm EROL FORESTIER, Isabel Santos, Araceli Muela GARCIA and Alain FRANCON
  Published on: 28 April 2025

• Published on: 28 April 2025

  Spa Therapy Is Not a Pill: Reconsidering Methods in the Evaluation of Complex Interventions

  • Romain Jacques FORESTIER, rheumatologist Centre de Recherches Rhumatologiques et Thermales d'Aix-les-Bains
  • Other Contributors:
    • Fatma Begüm EROL FORESTIER, Physician (Specialist in Balneology)
    • Isabel Santos, Physician specialist in Physical Medicine and Rehabilitation
    • Araceli Muela GARCIA, Physician (Specialist in Balneology)
    • Alain FRANCON, rheumatologist

  RJ Forestier, FB Erol Forestier, I Santos, A Muela Garcia, A Françon.
  Centre de Recherches Rhumatologiques et Thermales d’Aix-les-Bains, Aix Les Bains, France.

  This meta-analysis approaches spa therapy as if it were a pharmaceutical intervention, which we believe does not fully reflect the complex and multifaceted nature of such treatments.
  We have been conducting clinical trials and systematic reviews in this field for over 30 years. In our experience, spa therapy is a complex intervention traditionally based on the use of thermal mineral water, often combined with massages, baths, showers, mud applications, and supervised pool-based exercises -each of which may have therapeutic effects of its own.
  We were surprised by the conclusions of this meta-analysis regarding both the therapeutic effect and the risk of bias, as they differ markedly from our own findings and appear to stem from several questionable methodological choices.
  Bibliographic Incompleteness
  The limited scope of the literature search is particularly problematic. In 2020, we identified 122 comparative trials on balneotherapy, whereas this meta-analysis included only 42 randomized controlled trials. Our complementary search updated to 2025 identified 42 trials focused solely on knee osteoarthritis, and a total of 141 trials after removing duplicates related to multiple conditions. The highly selective inclusion criteria adopted in this meta-analysis substantially reduced t...

  Show More

  RJ Forestier, FB Erol Forestier, I Santos, A Muela Garcia, A Françon.
  Centre de Recherches Rhumatologiques et Thermales d’Aix-les-Bains, Aix Les Bains, France.

  This meta-analysis approaches spa therapy as if it were a pharmaceutical intervention, which we believe does not fully reflect the complex and multifaceted nature of such treatments.
  We have been conducting clinical trials and systematic reviews in this field for over 30 years. In our experience, spa therapy is a complex intervention traditionally based on the use of thermal mineral water, often combined with massages, baths, showers, mud applications, and supervised pool-based exercises -each of which may have therapeutic effects of its own.
  We were surprised by the conclusions of this meta-analysis regarding both the therapeutic effect and the risk of bias, as they differ markedly from our own findings and appear to stem from several questionable methodological choices.
  Bibliographic Incompleteness
  The limited scope of the literature search is particularly problematic. In 2020, we identified 122 comparative trials on balneotherapy, whereas this meta-analysis included only 42 randomized controlled trials. Our complementary search updated to 2025 identified 42 trials focused solely on knee osteoarthritis, and a total of 141 trials after removing duplicates related to multiple conditions. The highly selective inclusion criteria adopted in this meta-analysis substantially reduced the number of eligible studies, which is especially concerning given the complexity and diversity of spa therapy interventions. This restrictive approach may have contributed to several other questionable methodological decisions, such as:
  • Excluding studies from Türkiye and Israel for the sake of homogeneity, although practices in these countries are very similar to those in Europe;
  • Favoring 3-month outcomes as the primary endpoint, even when 6-month data were available - a questionable approach in the context of chronic diseases.
  Heterogeneity Amplified by Methodological Choices
  We acknowledge that heterogeneity is inherently high in this field, due to the complex and multifaceted nature of spa therapy. However, we believe that the methodological choices made in this meta-analysis have significantly exacerbated the problem. As a result, the primary issue became the amplified heterogeneity, which compromises both the feasibility and the validity of the analysis, and makes it difficult to draw reliable conclusions about the presence or absence of a therapeutic effect. These problematic methodological choices include the following:
  • An overall pooled analysis was performed across very different conditions, yet this result is presented as the main finding in both the abstract and the main text, despite the limited clinical relevance of such aggregation.
  • The authors attempted to group conditions into mechanical disorders, inflammatory disorders, and fibromyalgia, but from a musculoskeletal medicine perspective, grouping clinically distinct conditions—such as rheumatoid arthritis and spondyloarthritis, or low back pain, knee osteoarthritis, hand osteoarthritis, and tendinitis- is both clinically and methodologically inappropriate.
  • As previously noted, there is also substantial heterogeneity in the spa interventions themselves—in terms of duration, intensity, and content. While some studies assessed a single intervention, most involved multiple and diverse components. Control interventions also varied considerably, including active comparators, partial controls, waiting lists, usual care, or no treatment. Therefore, pooling these studies without distinction is not methodologically appropriate. If an overall analysis is conducted, subgroup analyses should at minimum be performed to account for these differences.
  • Some studies, such as that by Franke et al. [1], were classified as placebo-controlled, but in reality, they compared two complex interventions: a combination of spa therapy, educational programs, exercise, and physiotherapy—with and without radon. The only difference between groups was the type of water used (radon-rich vs. radon-poor), while the other components of treatment were identical. In such cases, the specific effect of spa therapy—particularly that of radon—is diluted by the presence of multiple active co-interventions. It is therefore questionable whether any conclusions can be drawn about the isolated effect of spa therapy or the water type from such designs, as the associated treatments likely had a substantial impact on outcomes.
  • The method of calculating effect sizes is also problematic, as the authors combined “mean change change-from-baseline scores” with “end value” results across studies. End values are sensitive to baseline imbalance. If the baseline scores differ between groups, post-intervention scores may be misleading and may increase heterogeneity.
  • The quality-of-life (QoL) measures used across studies varied considerably, with some employing generic instruments and others using condition-specific tools. Even the authors noted that they tried to select the least disease-specific QoL measure when multiple were available, pooling different measures that may have different ranges, scaling and sensitivity to change, makes the interpretation of pooled effect sizes more difficult - even with SMD. Besides, generic QoL instruments are generally recognized to be less responsive to clinical change and are not ideally suited for evaluating the therapeutic effects of interventions in rheumatic conditions. Beside the pain the use function-related outcome measures would have been more appropriate and clinically meaningful.
  • The quality-of-life (QoL) measures used across studies varied, with some relying on generic instruments and others on condition-specific tools. The authors noted that they selected the least disease-specific measure when multiple options were available. However, pooling outcomes derived from different instruments—each with distinct ranges, scaling characteristics, and sensitivity to change—complicates the interpretation of effect sizes, even when standardized mean differences (SMDs) are applied.
  • Moreover, generic QoL measures are widely recognized as being less responsive to clinical change and are not ideally suited for evaluating the therapeutic effects of interventions in rheumatic conditions. In addition to pain outcomes, the inclusion of function-related, condition-specific measures would have been more appropriate and clinically meaningful.
  Problems linked to the risk of bias assessment
  We fully agree that some spa therapy trials are of low methodological quality. However, it would be incorrect to assume that this applies to all studies in the field.
  The Cochrane ROB2 tool is well-suited to evaluating explanatory drug trials with double-blind, placebo-controlled designs. However, it fails to adequately distinguish between more and less rigorous non-pharmacological studies, especially those with a pragmatic design, due to its limited capacity to address the absence of blinding and the variability in the implementation of complex interventions. These limitations include the following concerns:
  • ROB2 particularly assumes that blinding of patients and outcome assessment is feasible, whereas in practice, this is very difficult to achieve [2] - except in rare cases such as radon studies. For example, sulphurous thermal waters have a strong odor, and bicarbonate-rich waters are often fizzy and salty, making them easily distinguishable from tap water. Moreover, patients are frequently the assessors of primary outcomes, particularly in rheumatology, where self-reported questionnaires are commonly used.
  • Important contextual factors, such as the therapists’ level of experience and the presence of co-interventions, are not considered in the RoB 2 framework, despite their potential influence on outcomes.
  • Alternative strategies for minimizing bias, such as Zelen randomization [3,4] or the use of qualitative outcomes [5,6], are not accounted for by RoB 2, despite their relevance in non-pharmacological trials.
  Given the complexity of non-pharmacological interventions, tools such as the CLEAR scale [7] or the PEDro scale [8] would have been better suited for assessing internal validity in this context.
  Conclusion
  In conclusion, this meta-analysis raises important questions, not only about the evidence surrounding spa therapy, but also about how we choose to evaluate complex, non-pharmacological interventions. While efforts to synthesize the literature are commendable, the methodological decisions made here significantly limit the reliability and applicability of the findings. Spa therapy is a multifaceted intervention, deeply embedded in real-world clinical contexts. Evaluating it with the same methodological lens as a pharmaceutical product not only oversimplifies its nature but risks drawing misleading conclusions. Future reviews must better reflect the diversity and complexity of spa interventions, and adopt evaluation tools that are fit for purpose. Only then can we reach valid and clinically meaningful conclusions that inform both practice and policy.

  References
  [1] Annegret F, Thomas F. Long-term benefits of radon spa therapy in rheumatic diseases: results of the randomised, multi-centre IMuRa trial. Rheumatol Int. 2013 Nov;33(11):2839-50. doi: 10.1007/s00296-013-2819-8. Epub 2013 Jul 18.
  [2] Boutron I, Tubach F, Giraudeau B, Ravaud P. Blinding was judged more difficult to achieve and maintain in nonpharmacologic than pharmacologic trials. J Clin Epidemiol. 2004 Jun;57(6):543-50. doi: 10.1016/j.jclinepi.2003.12.010.
  [3] Zelen M. A new design for randomized clinical trials. N Engl J Med. 1979 May 31;300(22):1242-5. doi: 10.1056/NEJM197905313002203.
  [4] Simon GE, Shortreed SM, DeBar LL. Zelen design clinical trials: why, when, and how. Trials. 2021 Aug 17;22(1):541. doi: 10.1186/s13063-021-05517-w. PMID: 34404466; PMCID: PMC8371763.
  [5] Hróbjartsson A, Gøtzsche PC. Is the placebo powerless? An analysis of clinical trials comparing placebo with no treatment. N Engl J Med. 2001 May 24;344(21):1594-602. doi: 10.1056/NEJM200105243442106. Erratum in: N Engl J Med 2001 Jul 26;345(4):304. PMID: 11372012.
  [6] Hróbjartsson A, Thomsen AS, Emanuelsson F, Tendal B, Hilden J, Boutron I, Ravaud P, Brorson S. Observer bias in randomised clinical trials with binary outcomes: systematic review of trials with both blinded and non-blinded outcome assessors. BMJ. 2012 Feb 27;344:e1119. doi: 10.1136/bmj.e1119. PMID: 22371859.
  [7] Boutron I, Moher D, Tugwell P, Giraudeau B, Poiraudeau S, Nizard R, Ravaud P. A checklist to evaluate a report of a nonpharmacological trial (CLEAR NPT) was developed using consensus. J Clin Epidemiol. 2005 Dec;58(12):1233-40. doi: 10.1016/j.jclinepi.2005.05.004. Epub 2005 Oct 13. PMID: 16291467.
  [8] Moseley AM, Rahman P, Wells GA, Zadro JR, Sherrington C, Toupin-April K, Brosseau L. Agreement between the Cochrane risk of bias tool and Physiotherapy Evidence Database (PEDro) scale: A meta-epidemiological study of randomized controlled trials of physical therapy interventions. PLoS One. 2019 Sep 19;14(9):e0222770. doi: 10.1371/journal.pone.0222770. PMID: 31536575; PMCID: PMC6752782.

  Show Less

  Conflict of Interest:
  None declared.

  • Back to top