Article Text
PDF
Abstract
Objectives An effectiveness and cost-effectiveness analyses of two-staged community sports interventions; taster sports sessions compared with portfolio of community sport sessions.
Design Quasi-experiment using an interrupted time series design.
Setting Community sports projects delivered by eight lead partners in London Borough of Hounslow, UK.
Participants Inactive people aged 14 plus years (n=246) were recruited between May 2013 and February 2014.
Interventions Community sports interventions delivered in two stages, 6-week programme of taster sport sessions (stage 1) and 6-week programme of portfolio of community sporting sessions delivered by trained coaches (stage 2).
Outcome measures (a) Change in days with ≥30 min of self-reported vigorous intensity physical activity (PA), moderate intensity PA, walking and sport; and (b) change in subjective well-being and EQ5D5L quality-adjusted life-years (QALYs).
Methods Interrupted time series analysis evaluated the effectiveness of the two-staged sports programmes. Cost-effectiveness analysis compares stage 2 with stage 1 from a provider’s perspective, reporting outcomes of incremental cost per QALY (2015/2016 price year). Uncertainty was assessed using deterministic and probabilistic sensitivity analyses.
Results Compared with stage 1, counterfactual change at 21 days in PA was lower for vigorous (log odds: −0.52; 95% CI −1 to –0.03), moderate PA (−0.50; 95% CI 0.94 to 0.05) and sport
(−0.56; 95% CI −1.02 to –0.10). Stage 2 increased walking (0.28; 95% CI 0.3 to 0.52). Effect overtime was similar. Counterfactual change at 21 days in well-being was positive particularly for ‘happiness’ (0.29; 95% CI 0.06 to 0.51). Stage 2 was more expensive (£101 per participant) but increased QALYs (0.001; 95% CI −0.034 to 0.036). Cost per QALY for stage 2 was £50 000 and has 29% chance of being cost-effective (£30 000 threshold).
Conclusion Community-based sport interventions could increase PA among inactive people. Less intensive sports sessions may be more effective and cost-effective.
- sports, physical activity,
- cost
- complex community sport intervention
- interrupted time series
- quasi experimental design
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
- sports, physical activity,
- cost
- complex community sport intervention
- interrupted time series
- quasi experimental design
Strengths and limitations of this study
The first mixed-method evaluation of a complex community multisport intervention aiming to engage inactive people in sustained sporting activity to promote physical activity, health and well-being.
A novel application of quasi-experimental methods to address a policy relevant topic.
A rigorous effectiveness and economic evaluation, a rarity in the literature on sports interventions.
The study does not include a control group.
Introduction
Physical inactivity is an intercontinental public health issue estimated, in 2013, to cost the global healthcare system over Int$53 billion—80% of which was borne by high-income countries.1 One way of improving population levels of physical activity (PA) is through sports participation2 and this is a priority area in England.3 The UK government’s national ambition is to achieve a yearly increase in the proportion of physically active individuals.4
Following the 2012 Olympic and Paralympic Games in London, the sports sector has received increased attention as a key stakeholder in improving PA in the UK.5 6 To realise the 2012 legacy, a cross-sector government initiative, Moving More, Living More, was set up in 2014 to provide political support for raising population-level physical activity and to include representation from diverse organisations including Sport England, Public health England and the Department of Health.
Key among the action plans is an inclusive and collaborative approach that involves a wide range of stakeholders including local authorities, public organisations and third sector groups to set up innovative ways of reaching and engaging inactive people in PA for public health outcomes. Making local communities the fulcrum of public health programmes has gained further significance since 2013 when the UK government made public health the mandate of local authorities.7 Current DCMS8 and Sport England3 strategies position public health and well-being outcomes as central to sport design and delivery aimed at addressing the diverse needs of local communities. This approach is, however, not straightforward as it requires sports interventions that are designed to account for the complexities of social diversity and community context. Complex interventions, which are characterised by a variety of intertwining factors often targeted at different groups of people,9 are a common feature in sectors seeking to achieve positive health outcomes.10 To date, evidence on the use of complex interventions delivered by sports organisations to improve sports participation is lacking.11 A recent literature review corroborated this finding and particularly emphasised the low quality of studies.12 This is particularly so for community-based sports interventions12 which were characterised by sparse evidence (n=6), methodological weakness (eg, only one data point pre-intervention and postintervention and/or very small sample sizes), with mixed results or with some evidence relevant to only to very specific populations (eg, men who play football).13 We found no primary studies of the cost of community sport interventions and only two model-based economic evaluations. National Institute for Health and Care Excellence (NICE)14 found two interventions to be cost-effective ((‘walking buses’ (£4007 per quality-adjusted life year (QALY), dance classes (£24 570/QALY)) and two not to be cost-effective ((free swimming (£40 462/QALY), community sports (£71 456/QALY)). NICE15 evaluated promoting physical activity in children and young people but found considerable uncertainty in the cost-effectiveness estimates for each of the interventions, due to a lack of direct evidence and the assumptions required.
The limitations in the evidence-based challenges are resource commitment, policy development and provision of community sport. This is in addition to the methodological challenges associated with the design and evaluation of complex interventions.16
This study compares the outcomes and costs of a complex community sport programme, the Health and Sport Engagement (HASE) project using a strong quasi-experimental design. The HASE project is part of the national Sport England Get Healthy Get Active initiative which aims to assess the role of community sport in raising population-level physical activity for health and well-being outcomes.
Methods
A complex community sport intervention (The Health and Sport Engagement (HASE) Project)
Development and training: prior to the introduction of stage 1 and stage 2, a partnership between the London Borough of Hounslow (LBH) health and well-being commissioning managers, community sport delivery experts, public health professionals, researchers and potential participants was used to understand the community sport needs of inactive people and design a portfolio of community sports tailored to the needs of inactive participants. In addition, training was provided to enhance the skills and knowledge of community sport and public health professionals.17 This paper first compares the 6 weeks of stage 1 with the first 6 weeks of stage 2. It also reports the 12-month follow-up results of both stages.
Stage 1: introductory sports sessions: two 45 min ‘taster sessions’ covered short, 20 min, low-level activity sessions, talk about sport and health, ‘meet the coach’ sessions, project descriptions and facility tours.
Stage 2: portfolio of community sport sessions: one or a set of community sports were provided (see supplementary table 1). Sessions lasting between 1 and 2 hours were delivered by trained coaches once a week.
Supplementary file 1
Study design, setting and participants
An interrupted time series design compared three data points during stage 1 (−42,–21, 0 days — immediately before the stage 2) and three during stage 2 (+21,+42,+63 days). A 12-month follow-up survey (round 7) assessed the long-term maintenance of outcomes. A target of 300 inactive people taking part over a 12-month period was based on achieving ‘wider engagement’ with inactive people (n=3225) in the LBH and local evidence (from LBH) of a 10% retention rate at 12–16 weeks. A formal power analysis was not conducted as the threshold for raising physical activity levels through community sport is not known.
Inactive participants were recruited between May 2013 and February 2014 through eight lead partner organisations working with local communities to deliver physical activity programmes in the LBH. Consent was obtained prior to screening using the following eligibility criteria: being physically inactive (self-report of <3 days of moderate–vigorous PA (each day ≥30 min) per week18; and having no contraindications to increasing PA. A full study protocol is published.17 Piloting our screening and data collection techniques revealed that inactive participants overestimated their activity levels. Therefore, following the agreement with lead partners and Sport England, we revised the inclusion criteria from ≤2 to ≤3 days of ≥30 min PA.
Outcome measures
Table 1 provides an overview of the outcome data and how they were measured. Costs to providers for the provision of community sport in stages 1 and 2 included set-up, training and delivery. Resource use, including those’ in kind’, were collected using interviews, diaries, time sheets and administrative records. Resources were valued using national unit costs (eg, national career service salary scales, national business travel rate and BT tariff guides) to increase generalisability and local records when unavailable. Outcomes relating to process evaluation will be published separately.
Measures of outcome data
Statistical analysis
Multiple imputations (n=5) by chained equations were used to replace missing data.19 Imputation models used predictive mean matching (five nearest statistical neighbours) based on sociodemographics, and values from previous and subsequent survey rounds for the variable in question. Missing observations below 5% were replaced using central tendency measures.20
Interrupted time series ordered logistic21 and Poisson models (only for the quality of life measure), with random effects estimator, reflected the change in outcomes controlling for age, gender, education, ethnicity, health status, income status and size of household (online supplementary table 2). Huber-White sandwich estimates of variance were calculated for the regression parameters to account for potential serial correlation.
The interrupted time series model was achieved by defining independent variables Tt=time point (1,2,3,4,5,6), Xt=0 for time points in stage 1 (time points 1, 2 and 3) and 1 for time points in stage 2 (time points 4, 5 and 6), and (XT)t=0 for time points in stage 1, and (t−4) for time points in stage 2. The regression coefficient for Tt represents the rate of change of activity in stage 1, and the sum of regression coefficients for Tt and (XT)t is the rate of change of activity in stage 2. We defined the effect over time as the difference in the rate of change from stage 1 to stage 2, that is, the regression coefficient of (XT)t. The immediate effect of the intervention (21-day difference) is defined as the regression coefficient corresponding to Xt (corresponding to the counterfactual difference between stage 1 and stage 2 activity evaluated at time point 4). The interrupted series time model for all activity and well-being measures was an ordinal logistic repeated measures model, including additive effects for Tt, Xt and (XT)t and for all covariates to be adjusted for.
Changes in 12-month outcomes were similar, except that T, XT and Xt were replaced by a binary indicating 0 if the observation was recorded at survey round 6 and 1 if the observation was recorded at survey round 7.
Sensitivity analyses examined the effects of (a) limiting analysis to participants with ≥1 data point for stage 1 and stage 2, and (b) missing data replacement, by comparing the raw and imputed data across five different imputed datasets, (c) potential lagged effect of stage 1 (equivalent to stage 2 covering time points 3, 4, 5 and 6). The last data collection of stage 1 occurred just prior to the inception of stage 2. Analyses were conducted using Stata13.
Cost and cost-effectiveness analysis
The cost was divided into three periods, the development and training stage, stage 1 and stage 2. While total costs are the sum of each, the cost-effectiveness analysis focusses on the differences in delivery costs between stage 1 and stage 2, that is, excluding set-up and training costs common to both. In addition, we assumed: running costs were completely attributed to the number of inactive people attending; cost of delivering stage 2 was linear across weeks (most costs were for facilities and coaches, and did not vary by week); and the cost of delivering stage 1 included promotional activities that occurred during the introductory activity sessions. The analysis used a provider’s perspective and the year of costing was 2015/2016. The time horizon was 6 weeks and therefore neither costs nor outcomes were discounted.
Generalised linear models, fitted separately for costs (using Poisson distribution) and QALYs (with binomial one family), controlled for sociodemographic and health variables (supplementary table 2). Predictions, using the ‘margins method’ generated sample means for incremental cost-effectiveness ratios.22 Probabilistic sensitivity analysis used non-parametric bootstrapping to reflect stochastic uncertainty as cost-effectiveness planes and acceptability curves. Deterministic sensitivity analysis evaluated analytic viewpoint and alternative assumptions for cost calculation (eg, assuming stage 1 cost was equivalent to two activity sessions delivered).
Patient and public involvement
The research question and outcome measures were developed with community sport and public health professionals in a local London authority in the UK to reflect public health and PA policy priorities. The community sport intervention was co-designed with potential participants, sport coach specialists and public health experts through participatory focus groups and a bespoke workforce training package. The results were disseminated to study participants through project reports, case studies and briefings translated as appropriate for the audience.
Results
Participants
Figure 1 shows that from a total of 547 recruited by the sports deliverers, 377 consented to participate in the study. Of the 377 people, 246 were ‘screened in’. Online supplementary table 2 reports the sample characteristics.
HASE evaluation study overview. HASE, Health and Sport Engagement.
The pattern of missing data was multivariate and the proportion of missing data rose over time (7% to 54% between the first and sixth questionnaires; and 68% for the seventh questionnaire). Sixty-one per cent (n=151) had at least one data point each for stage 1 and 2.
Outcomes
The proportion of participants who participated in any PA lasting at least 30 mins in the previous week prior was 72%, 77% and 85% in the first three questionnaires (stage 1) and 92%, 84% and 93% for the last three questionnaires (stage 2). At 1 year, the rate was 91%.
Effects on PA
Table 2 shows the changes in activity levels during stage 1 and stage 2 activity periods. The largest mean increase from stage 1 to 2 was for vigorous PA (72%) followed by sport (68%), any PA (22%) moderate PA (10%) and walking (7%). Days doing any, moderate or vigorous physical activity or sport increased (between 24% and 166%) over the 6 weeks of stage 1 but days walking decreased (by 12%). During the period of stage 2, days doing all activities increased, but relatively smaller (3%–7%), except for vigorous activity that had a decrease (−7%).
Effect on outcomes
The effect sizes for assessing the two impacts of stage 2 compared with stage 1 using regression analysis are presented in table 2. The change in the PA outcomes was positive during both the stages. Overall, the effect overtime was higher during stage 1 except for walking which was higher in stage 2.
Effect of stage 2 on PA 21 days after inception (immediate effect): compared with stage 1, the rate of change in PA 21 days after inception of stage 2 was lower for vigorous PA (−0.52; 95% CI −1.0 to –0.03; p=0.00), as the number of days doing vigorous PA ≥30 mins per week increased by +0.35 days from 3 to 6 weeks but by only 0.04 days from 6 to 9 weeks. There was also a fall for moderate PA and number of days doing sport for >0 mins. While walking showed a positive increase (number of days walking ≥30 mins per week increased by +0.42 days, on average between time periods 3 and 4), this effect was minimal (0.35; 95% CI −0.08 to 0.78; p=0.11).
Effect of stage 2 on PA effect overtime: the pattern of findings on the impact of stage 2 overtime had some similarities to immediate impact. The number of days doing any vigorous or moderate PA and days doing sport were lower in the move to stage 2. However, number of days walking showed an increase in stage 2 (0.28; 95% CI 0.3 to 0.52; p=0.03). Days walking increased by +0.23 days per 3 weeks on average during the stage 2, but decreased during stage 1 by −0.19 days per 3 weeks.
Sensitivity analyses did not change findings (see the online supplementary tables 3-5).
Effects on subjective well-being and quality of life
Table 2 shows the changes in subjective well-being during stages 1 and 2. The effect sizes for assessing two impacts of stage 2 compared with stage 1 using regression analysis are presented in table 2. The change at 21 days in the subjective well-being was slightly more positive in stage 2 for all indicators of subjective well-being (eg, ‘anxious’:0.09; 95% CI −0.30 to 0.49; p=0.648). The differences in the rate of change between the stage 1 and 2 were positive for each except life satisfaction, and particularly more positive for ‘happiness’ (0.29; 95% CI 0.06 to 0.51; p=0.01).
The findings on utility scores (EQ5D5L) showed lower scores immediately after stage 2 and over time although the differences were minimal (table 2). QALYs for participants during stage 2 (0.094; SD:0.025) were slightly higher than stage 1 (0.093; SD:0.026).
Sensitivity analyses did not change findings (see the online supplementary tables 3-5).
Online supplementary table 6 shows changes in the range of outcomes on follow-up at 1 year. The changes in PA outcomes were mixed. While days doing vigorous and moderate PA increased at follow-up, that of sport, walking and any PA decreased. The changes were small except for sport (−0.490; 95% CI −0.849 to–0.132, p=0.01). Feelings of satisfaction, anxiety, worthwhileness and happiness as well as the quality of life (EQ5D5L) were higher at follow-up particularly for anxiety (0.459; 95% CI 0.136 to 0.782, p=0.01).
Cost and cost-effectiveness
Online supplementary table 7 shows that the cost of activities for development, and training per participant were £48 (SD £82) and £39 (SD £31), respectively. The mean (SD) cost to providers per participant was higher for delivering sport in stage 2 (£101, SD £124) compared with stage 1 (£51, SD £45). The main contributor to the total cost to providers was delivering sport and the least was training. An additional cost per person of £405 (SD £463) was required to deliver sport to the participants up to a year. This mostly covered hiring of coaches and facilities. Delivery cost for sports interventions in stage 2 are shown in the online supplementary table 8.
Participants (whole sample) spent an average of £52 (SD £137) participating in PA in stage 1 and £71 (SD £225) in stage 2. There were, however, many who spent no money; 69% in stage 1 and 55% in stage 2. The average for those actual spending was £166 (SD £202) in stage 1 and £160 (SD £315) in stage 2.
Table 3 shows the incremental costs were higher for the stage 2 (+£50; 95% CI 48 to 51) compared with stage 1. Stage 2 also led to slightly higher QALYs (+0.001; 95% CI −0.034 to 0.036; p=0.95). The (ICER) for stage 2 sporting activities compared with stage 1 is £50 000 per QALY gained. This indicates that at 6 weeks, stage 2 would not be considered cost-effective in England. The National Institute for Health and Care Excellence (NICE) cost-effectiveness threshold in England ranges between £20 000 and £30 000.
Regression estimates for costs and QALYs, and cost effectiveness (£’sterling 2015/2016)
The probabilistic sensitivity analyses confirm the findings of the base case (online supplementary figure 1). Online supplementary figure 2) shows that stage 2 has an 11% chance of being cost-effective at 6 weeks compared with stage 1 at £20 000 per QALY gained, which increases to 29% at £30 000/QALY.
Deterministic sensitivity analyses (table 3) showed that changing the analytic viewpoint to provider plus participant worsened cost-effectiveness while assumptions on cost per person led to a decisionally important change in results.
Discussion
This study is the first mixed-method evaluation of a complex community multisport intervention aiming to engage inactive people in a sustained sporting activity to raise levels of PA for health and well-being outcomes. It directly reflects a UK policy priority area focused on building evidence on the role of community sport in increasing PA for public health and well-being and a limited evidence base consisting of papers with methodological weakness including small samples and inadequate data points for study designs. The study demonstrates the effects of two stages of community sports intervention (each offering a variety of activities) on alternative specifications of PA (sport, vigorous, moderate, any PA and walking), quality of life (subjective well-being and EQ5D5L) and cost (to participants and providers). The introductory sport sessions (stage 1) resulted in higher numbers of days undertaking any vigorous, moderate PA and sport, but lower amounts of walking than stage 2. The rate of increase in activity over the 6 weeks of stage 1 was greater than during the first 6 weeks of the community sports sessions provided in stage 2. However, compared with stage 1, the degree of happiness overtime was greater in stage 2., Stage 2 was costlier than stage 1 and generated more QALYs and based on the evidence here would not be considered a cost-effective addition to a stage 1 programme in England at 6 weeks.
The strengths of this study are: the focus on an ethnically diverse sample covering multiple stages of the life course for a community-based intervention; the comparative improvement in methodological quality in evaluating a community sports intervention through a quasi-experimental design with multiple data collection points; the addition of an economic evaluation; and examining follow-up at 1 year.
There are limitations. First, similar to many studies in the area, concurrent change (either positive or negative) over time cannot be ruled out due to the reliance on an interrupted time series analysis without a control group. However, diminishing the effect of stage 1 (to cover 1 and 2) in the sensitivity analysis did not change the results. Finding some positive evidence at 6 weeks, however, increases the case for recommending a randomised controlled trial. Second, there were missing data, but our analyses with and without missing data were comparable. Third, the results herein ought to be considered indicative but not definitive, considering the short-term nature of the analysis (covering 6 weeks).
The generalisability of the study could be questioned both in terms of the types of sports offered in the community sessions and the sample. The recruited sample was dominated by two age groups (14–24 years and >64 years) and women (77%). The gender representation reflects the profile of the inactive population in Hounslow as sports participation is lower among women.23 While the range of sports activities included 5 out of the most 15 common sports in the UK (football, gym, swimming, tennis, badminton),24 some of the sports could be considered to be currently unpopular (eg, yoga, New Age Kurling) and unlikely to attract high participation from inactive people or be promoted widely by sports agencies, when rolled out. However, the codesign approach to the project ensured that the participants informed the sports on offer. Expert sports coaches were also able to provide adapted sports where needed and, thus, the project delivered sports to match the needs and desires of the participants. Furthermore, the participation rates from this study show that broad definition of sport is central to attracting inactive people and that these alternative and adapted sports could be to be integral to the tailored and inclusive approach to reaching and engaging inactive people in community sport. For example, the multisport sessions (including new age kurling) offered to older adults recorded the second highest attendances (n=1442) during the year. The way sport is perceived and defined is complex and this was a key challenge in the project. This was a central part of the debate in the participatory focus groups with potential participants that underpinned the codesign of the project. The recruitment of the sport delivery groups was based on their established success in delivering community sport and their expertise in doing so with diverse social groups. Coupled with the Sport England supported approach to using broad definitions of sport and even the capacity to not use the word sport in the design and implementation of HASE, the project focused on maximising the potential for making a meaningful difference to PA.
As with complex interventions, the evaluation herein was challenged by data loss at follow-up due to the multiple data points that are required to ensure a robust design. Standard methods using multiple imputations were used to replace the missing data. Sensitivity analyses showed that the results using raw (unimputed) and imputed data were consistent. Further research is required to investigate the use of pragmatic and less burdensome experimental designs to evaluate sports intervention.
The mixed results shown in this study are consistent with the limited evidence shown by existing systematic reviews on the effectiveness of sports interventions.4 5 The findings on cost-effectiveness are similar to the previous results on community sports.6 This study has shown positive impacts of community sports interventions and therefore to provide definitive information, future studies should include a randomised controlled trial (whose sample size calculation could be informed by the findings here) and both a within project analysis and modelling of long-term cost-effectiveness to capture both short-term and longer-term benefits (as resource savings from reduced disease and therefore benefits in terms of increased quality of life). Community-based sport interventions have the potential to encourage participation in PA among inactive people and are thus an important area for public health policy and strategy for PA. To achieve increased population levels of a PA through the community sport sector, the design and implementation of less intensive sports sessions, reflecting the needs and choices of participants may represent a more effective strategy.
Acknowledgments
The HASE Project was designed, delivered and evaluated in the London Borough of Hounslow with key delivery partners: Brentford Football Club Community Sport Trust, CB Hounslow United, Cattaway Tennis, Cycle Experience, Centre for Workplace and Community Health at St Mary’s University Twickenham, Fusion Lifestyle, England Netball, Hounslow Homes, LKFitness, Sport Impact, The Urban Youth Network and Westside Basketball Club. Additional design and delivery support was from British Airways, British Army Welfare, Cranford Community School, Hounslow and Richmond Community Health, Integrated Neurological Services (Richmond), UK Dodgeball Association, Make Sport Fun, West Thames College, The Hub Youth Club. They acknowledge the contribution of the participants as co-designers of the intervention.
References
Footnotes
Patient consent for publication Obtained.
Contributors Conceptualisation: NA, LM, TK, JF-R. Funding acquisition: NA, LM, TK, JF-R. Investigation: NA, LM, TK, AL, SS, JF-R. Methodology: NA, LM, TK, JFR. Formal analysis: NA, AL, SS, JF-R. Writing original draft: NA, LM, JF-R. Writing, reviewing and editing: NA, LM, TK, AL, SS, JF-R.
Funding This study was funded by Sports England’s Get Health Get Active awards (Award Number 2012021352).
Competing interests The HASE intervention was co-developed by Brunel University investigators, LBH, local community participants and sport deliverers. Brunel University investigators evaluated the project.
Ethics approval Brunel University London Research Ethics Committee (RE33-12).
Provenance and peer review Not commissioned; externally peer reviewed.
Data sharing statement The dataset analysed during this study are available from the corresponding author on reasonable request.