ODFS Pace and Pace XL functional electrical stimulation devices for treating drop foot

Regulatory bodies

A search of the Medicines and Healthcare Products Regulatory Agency website revealed no manufacturer Field Safety Notices or Medical Device Alerts for this device. No reports of adverse events were identified from a search of the US Food and Drug Administration (FDA) database: Manufacturer and User Device Facility Experience (MAUDE).

Clinical evidence

Forty‑two relevant studies were identified of which 6 represented the best quality evidence (based on study design, number of patients and relevant outcome measures) and are summarised in this briefing (see search strategy and evidence selection section for more detail). This includes 5 papers reporting results from 3 randomised controlled trials (RCTs) and 1 case series. For all 5 studies reporting data from the RCTs (Barrett et al. 2009; Esnouf et al. 2010; Sheffler et al. 2013; Sheffler et al. 2015; Burridge et al. 1997), the evidence presented was based on the analogue versions of the Odstock Dropped Foot Stimulator (ODFS), whereas the case series incorporated a range of ODFS devices (including the Pace and Pace XL) without separating the results by device type (Street et al. 2015). Two of the papers were based on a single RCT, which included people with multiple sclerosis (MS; Barrett et al. 2009; Esnouf et al. 2010) and 2 of the papers were based on a single RCT that included people who had survived a stroke (Sheffler et al. 2013; Sheffler et al. 2015). When 2 papers were based on the same trial, each paper reported different outcome measures. The 3 RCTS and 1 case series all compared the ODFS with either usual care, exercise, ankle‑foot orthosis (AFO), or exercise and AFO.

Barrett et al. (2009) carried out an RCT to investigate the effects of the ODFS on gait compared with exercise therapy for people with secondary progressive multiple sclerosis (SPMS). Eligible people, aged 18 years or older who had a diagnosis of SPMS, a predominately unilateral drop foot and no previous use of functional electrical stimulation (FES), were recruited from the UK National Clinical FES Centre. The study design included a pilot element to test procedures; the data from these patients were excluded from the analysis. Patients were randomised into 2 groups. In the intervention group (n=20), patients used the ODFS to treat drop foot. After the training period, people were encouraged to use the device most of the day and to switch it on and off as needed. In the comparator group (n=24), patients had a programme of simple physiotherapy exercises to carry out at home. Outcomes included walking speed over 10 metres, physiological cost index (an indicator of the effort involved in walking) over 10 metres and distance walked in 3 minutes. These were assessed at weeks 6, 12 and 18. The intervention group did the tests with the ODFS switched on (with stimulation) and switched off (without stimulation). Treatment was over 18 weeks and people were not followed up after the intervention period. No significant improvement in unassisted walking speed over 10 metres or unassisted distance walked in 3 minutes (that is, the training effect) was seen over the 18 week treatment period in the intervention group with the ODFS switched off compared with baseline values. The exercise group showed significant improvement in both walking speed (p=0.001) and walking distance (p=0.005) compared with baseline values over the same period, but only the walking speed was statistically significantly improved compared with the ODFS group (mean difference between groups 0.081 metres per second; 95% confidence interval [CI] 0.01 to 0.15; p=0.028). This suggests that home exercise is a more effective means of improving unassisted walking performance compared with ODFS in people with SPMS. An orthotic effect was seen with ODFS. At each assessment stage of the study (6, 12, and 18 weeks), there were statistically significant improvements in both the 10 metre walking speed and distance walked in 3 minutes in the ODFS group with stimulation compared with ODFS without stimulation (baseline to week 18: walking speed p=0.001; distance p=0.004). There was no indication that ODFS had significant training or orthotic effects on energy expenditure. The summary and results are shown in table 2 and table 3.

Esnouf et al. (2010) reported on the same RCT as Barrett et al. (2009) to determine if the ODFS improved activities of daily living in people with SPMS. The inclusion criteria were the same as those stated above. The number of patients is slightly greater than in Barrett et al. (2009) because this study included the people who took part in the pilot study. People were randomised to 2 groups. In the intervention group (n=26), the ODFS was provided for daily use. In the comparator group (n=27), the physiotherapist assigned patients exercises to complete once or twice daily at home. In the comparator group, ankle‑foot orthosis use was continued if the patient already used one but no new orthoses were issued. Treatment sessions with the physiotherapist for both groups were carried out at baseline and in weeks 6, 12 and 18. Both groups had their allocated treatment and were followed up over 18 weeks. Their Canadian Occupational Performance Measure (COPM), which is used to assess activities of daily living, was rated at baseline and 18 weeks, and patients completed a 'falls diary' throughout. The results showed that improvements in COPM performance and satisfaction scores were greater in the intervention group than the comparator group (COPM performance – difference in intergroup change: 1.1, p=0.038; COPM satisfaction – difference in intergroup change: 1.7, p=0.007). The results also showed a significantly lower median number of falls in the intervention group (intergroup comparison: median difference between groups: 13; 18 in comparator group; 5 in intervention group; p=0.036). COPM results for climbing stairs, balance, and steps and kerbs were included in the paper but p‑values between groups were not reported because they were not statistically significant. There were significant improvements within the intervention group for COPM tripping performance scores (3.5; p<0.05) and tripping satisfaction scores (4.5; p<0.05). There was also a significant improvement between groups (difference in improvement score 4.5; p<0.05). Improvement scores were also significant for walking distance satisfaction scores both within the intervention group (5.5; p<0.05) and between groups (difference in improvement score 2.5, p<0.05). The summary and results are shown in table 4 and table 5.

Sheffler et al. (2013) did an RCT to compare the motor relearning effect of the ODFS compared with usual care on lower limb motor impairment, activity limitation, and quality of life among chronic ischaemic, lacunar or haemorrhagic stroke survivors. Eligible people were aged over 18 years and had a stroke at least 12 weeks before the start of the study. Patients were randomised into 2 groups. In the intervention group (n=39), patients were trained to use the ODFS for home and community mobility, and used an assistive device as needed. Sheffler (2015) reported that these devices could include a cane, quad cane or walker. These people had regular training, which consisted of 1‑hour sessions twice a week in the first 5 weeks and 3 times a week in the following 7 weeks. In the comparator group (n=45), standard physical therapy interventions were used. Forty‑eight people were treated with an ankle‑foot orthosis and 8 had no device at the start of the study. Some patients dropped out at each time point. People were treated for 12 weeks and followed up for a further 24 weeks post‑treatment. Assessments were carried out at baseline, 12, 24 and 36 weeks. Outcomes were assessed using the lower extremity portion of the Fugl‑Meyer (FM) Assessment (motor impairment), the Modified Emory Functional Ambulation Profile (mEFAP) measured without a device (functional ambulation), and the Stroke Specific Quality of Life (SSQOL) scale in which 12 outcomes are rated on a scale of 1 to 5. There was no significant intervention group main effect (a main effect is the effect of an independent variable upon a dependent variable) or intervention group by time interaction effect (which measures if the difference between the intervention and comparator group changed differently over time) for motor impairment (FM; p=0.797, p=0.321 respectively), functional ambulation (mEFAP; p=0.968, p>0.999 respectively), or SSQOL (p=0.360, p=0.627 respectively) on raw scores. For motor impairment (FM), time effect was significant (p=0.007), but no significant changes were seen from baseline to each time point (p>0.05). The model parameter estimates for time effect during treatment were not significant (difference 0.525; −0.345 to 1.396; p=0.238). For functional ambulation (mEFAP), time effect was significant (p<0.001). The model parameter estimates for time effect during treatment were significantly lower than at baseline (difference −13.864; −21.256 to −6.473; p<0.001). For SSQOL, time effect was significant (p<0.001). The model parameter estimates for time effect during treatment were significantly higher than at baseline (difference 9.910; 3.724 to 16.096; p=0.002). The summary and results are shown in table 6 and table 7.

Sheffler et al. (2015) reported on the same RCT as Sheffler et al. (2013) to compare mechanisms for functional improvement between the ODFS compared with usual care using quantitative gait analysis. The inclusion and exclusion criteria, and the randomisation of patients into 2 groups were the same as stated in Sheffler et al (2013; table 6 and table 7). In the intervention group (n=39), patients were trained to use the ODFS for home and community mobility, with an assistive device (such as a straight cane, quad cane, or walker) as needed. In the comparator group (n=45), standard physical therapy interventions were used. Patients were treated for 12 weeks and followed up for 6 months post‑treatment and were assessed at baseline, 12, 24 and 36 weeks. There were 13 main study outcomes, classed as spatiotemporal (including speed and length of gait), kinematic (analysis of 3‑dimensional movement including joint angles) and kinetic (forces involved in the production of movement) parameters of gait, all assessed with quantitative gait analysis. Activity level was also assessed. The main effects that were statistically significant in the intervention group compared with the comparator group were cadence (p<0.001) and peak hip power (p=0.003). The differences between the intervention and comparator group were not statistically significant for all other main effect outcome measures (see table 9 for full results). Time effect at final follow‑up was significant for cadence (p<0.0001), stride length (p=0.0003), walking speed (p<0.0001), anterior‑posterior ground reaction force (p=0.032), peak hip power (pre‑swing; p<0.0001) and peak ankle power (p=0.003). Time effect was not significant for peak ankle flex swing (p=0.058). For activity level, there was no significant time effect or intervention by time effect (a measure of whether the effects of an intervention are sustained over time) for average time standing per day, average time walking per day, or average number of steps per day. The study is summarised in table 8.

Burridge et al. (1997) carried out an RCT to measure the effect of the ODFS on the effort and speed of walking compared with physiotherapy for people who have had a stroke. Eligible people were those who had a stroke causing a hemiplegia at least 6 months ago, had the ability to stand from sitting without help and could walk a minimum of 50 metres independently before the stroke. Patients were recruited from the UK National Clinical FES Centre and randomised into 2 groups. In the intervention group (n=15), patients had the ODFS and a course of physiotherapy sessions. In the comparator group (n=16), patients only had the course of physiotherapy. All patients had the same therapy contact time (10 1‑hour physiotherapy sessions in the first month of the trial), but patients in the treatment group spent some of this time on training and adjusting the ODFS. Outcomes included walking speed over 10 metres and physiological cost index over 10 metres, assessed at baseline, week 4 and week 12. A training effect was not seen in the treatment group, because there was no significant difference between the treatment group without stimulation compared with the comparator group at any time point on either measure. An orthotic effect was seen on some outcome measures at certain time points. When comparing the treatment group when having stimulation with the comparator group, results were significant for 10‑metre walking speed (minutes/second) at week 12 (95% CI −0.460 to 0.001; p=0.044). No other results comparing the treatment group when having stimulation with the comparator group were significant at any other time point for either measure. The treatment group when having stimulation at baseline, week 4 and week 12 was also compared with the treatment group without stimulation at baseline. Results were significant for walking speed at week 12 (percentage change 20.50; 95% CI 0.060 to 0.210; p=0.004), physiological cost index at week 0 (percentage change −20.68; 95% CI 0.040 to 0.325; p=0.010) and physiological cost index at week 12 (percentage change −24.87; 95% CI 0.040 to 0.430; p=0.008). The summary and results are shown in table 10 and table 11.

Street et al. (2015) carried out a case series study with data collected between 2008 and 2013. The objective of the study was to determine the effectiveness of FES on drop foot in people with MS. The study collected data on patients during standard clinical care (n=153) from a UK‑based specialist FES centre, using 1 of 4 different versions of the ODFS (ODFS III, ODFS Pace, Odstock 2 channel Stimulator II and Pace XL). Patients for whom FES was suitable had 1 appointment to teach them how to use the FES. Baseline measurements were taken at a second appointment. After a 10‑metre walk to increase muscle temperature and range of movement, they did 2 further 10‑metre walks to measure the speed of unassisted walking and the effect of FES respectively. These measurements were taken again at a median of 20 weeks (interquartile range 16–24 weeks). The difference between the second and third walk defined the orthotic effect and the difference in walking speed over time when not using FES defined the training effect. For 10‑metre walking speed, the results showed a main effect for stimulation compared with no stimulation (F_1,152 91.88; p<0.001) and an interaction effect between stimulation over time (F_1,152 9.79; p=0.002). When comparing unassisted walking with assisted (FES) walking, there was a significant improvement for initial orthotic effect (mean difference 0.07±0.11; 95% CI 0.05 to 0.08; p=0.001), continuing orthotic effect (mean difference 0.11±0.16; 95% CI 0.08 to 0.13; p=0.001) and total orthotic effect (mean difference 0.10±0.22; 95% CI 0.07 to 0.14; p=0.001). The training effect was not significant (mean difference 0.00±0.26; 95% CI −0.04 to 0.03; p=0.53.). Functional walking category lasted or improved in 95% of people who responded to treatment. The summary and results are shown in table 12 and table 13.

Several older studies, using the analogue version of the ODFS, are presented in the NICE guidance on functional electrical stimulation for drop foot of central neurological origin.

Recent and ongoing studies

One ongoing or in‑development trial on the ODFS for multiple sclerosis was identified in the preparation of this briefing.

Health economic evaluations

Two papers and 1 report were identified that reported economic evaluations of the ODFS. Taylor et al. (2007) reported on a cost–utility analysis of the ODFS in people with stroke, based on efficacy data presented by Burridge et al. (1997). The Centre for Evidence‑based Purchasing (CEP) produced an economic report that analysed the cost effectiveness of FES compared with physiotherapy for drop foot of central neurological origin, based on a cost–utility model using a Monte Carlo simulation (Taft et al. 2010). Taylor et al. (2013) reported on a cost‑utility analysis of FES for drop foot caused by upper motor neuron lesions, using quality adjusted life‑years (QALY) data from the CEP report and retrospectively collected resource use data.

Taylor et al. (2007) reported a mean QALY gain of 0.065 with the ODFS (between month 0 and month 3) compared with physiotherapy, equating to a cost per QALY of £25,231 for 1 year's use and between £6,676 and £10,830 over 10 years. An incremental cost‑effectiveness ratio (ICER) comparing ODFS and physiotherapy was not presented. The CEP report suggested that FES has a cost per QALY of £52,337 in the first year, and £19,239 after 5 years. Using a 5‑year time horizon and at a 'willingness‑to‑pay threshold' of £30,000 per incremental QALY, FES was cost‑effective in 66% of model iterations (Taft et al. 2010). Taylor et al. (2013) reported a mean QALY gain of 0.041 and a cost per QALY of £15,406 for all users over 4.9 years. Sub‑group analysis was also done for people with stroke and a cost per QALY of £15,268 over 5 years was reported. Again, an ICER for FES compared with physiotherapy or any other comparator was not presented.

Both the papers and the report share some limitations. First, an ICER for the ODFS or FES compared with a relevant comparator (for example physiotherapy) was not accurately determined in any of the three studies, however Taylor et al. (2007) calculated cost per QALY using physiotherapy as the comparator. In both Taylor et al. (2007) and Taylor et al. (2013), results were presented as cost per QALY. This output should not be compared with a NICE cost‑effectiveness threshold and is not meaningful in assessing the cost effectiveness of the ODFS or FES. Within the CEP report (Taft et al. 2010), it could not be determined if the ICER was accurately estimated based on the information presented. This is because the utility score for physiotherapy was not explicitly reported. Second, neither costs nor QALYs were discounted, despite a 5‑ or 10‑year time horizon being adopted. Thirdly, QALY gain was not based on a survey of patients using a standardised instrument, such as the EQ‑5D, which brings into question the reliability of the estimates used. Finally, both the CEP report (Taft et al. 2010) and Taylor et al. (2013) reported on FES devices in general, although ODFS was used most often. The results presented are therefore based on efficacy data not completely related to the ODFS, limiting the applicability of the results. Given these limitations, the results of all 3 studies should be used with caution.

Table 1: Levels of bias in the 4 RCTs

The risk of selection bias in Burridge et al (1997) was unclear. The study reported a suitable method of randomisation, but did not carry out any analysis on the baseline characteristics. The authors did note that patients in the comparator group walked more slowly and with more effort than those in the treatment group. Although they stated that this may bias against the FES, this difference between groups was not significant at baseline. Also, a reason for exclusion from starting the trial was 'no observed improvement in walking with stimulation'. In practice, patients are assessed to determine if treatment with the ODFS is suitable for them. The paper does not give any detail on the definition of 'observed improvement' and so it is unclear if this is reflective of standard practice or if this is a bias.

Four studies (Barrett et al. 2009; Esnouf et al. 2010; Sheffler et al. 2013; Sheffler et al. 2015) were rated as having a low risk of selection bias. All studies included a suitable method of randomisation and adequate concealment of allocation. Sheffler et al. (2013) and Sheffler et al. (2015) reported no statistically significant differences in baseline characteristics and adjusted the statistical analysis for potential confounders. Barrett et al. (2009) stated that there were small but noticeable differences in age and time since diagnosis between the intervention and comparator groups. These were used as covariates in the analysis of covariance (ANCOVA), which showed that these had no significant effects on response variables at 18 weeks. This also applies to Esnouf et al. (2010), which reports the same study. Esnouf et al. reported that a greater proportion of patients in the comparator group used ankle‑foot orthoses or had rejected an orthosis before starting the study, suggesting that they may have been more vulnerable to falls. This was not tested for statistical significance.

The risk of performance bias in Esnouf et al. (2010), Sheffler et al. (2013), Sheffler et al. (2015) and Burridge et al. (1997) was unclear. Esnouf et al. (2010) and Sheffler et al. (2013) did not explicitly state if the people in both groups had the same care other than the intervention. Sheffler et al. (2015) and Burridge et al. (1997) stated that both groups had the same number of therapy hours and that the content of therapy sessions was standardised across treatment groups. This controls for differences between intervention and comparator groups, but it does not necessarily reflect standard practice in which people may have fewer hours of care, so it may lack external validity. In Esnouf et al. (2010), Sheffler et al. (2013) and Sheffler et al. (2015), patients within the comparison group had different care. Some patients had an ankle‑foot orthosis and others did not. It is not clear if this is a bias or if it accurately reflects standard practice. However, it is possible that some baseline measures differ between people with and without an orthosis. Sheffler et al. (2013) and Sheffler et al. (2015) reported that people were trained to use their ODFS devices for mobility, with use of assistive devices as needed. Burridge et al. (1997) also reported that some patients used a walking aid during assessment. It is not clear if both groups used assistive devices.

The risk of performance bias in Barrett et al. (2009) was rated as high. The paper reports that the intervention and comparator groups had the same number and timing of follow‑up appointments, but the clinical assessors were not blinded to the interventions. All interventions were given by the researchers conducting the trial.

Patients and individuals giving care were not blinded to treatment allocation in any of the studies because this was not possible for the intervention.

It was unclear if there was attrition bias in Esnouf et al. (2010), Sheffler et al. (2013) and Sheffler et al. (2015). The papers did not report if there were any differences between those who completed treatment and those who did not. These studies had a relatively high number of people withdrawing from or not completing the studies and in each study, slightly more people in the intervention group withdrew from or did not complete the study. The authors of Sheffler et al. (2013) and Sheffler et al. (2015) state that this may have compromised internal validity.

Barrett et al. (2009) has a high risk of attrition bias. The study also had a relatively high number of people withdrawing from or not completing the studies, which was higher in the intervention group. The paper reported that some of the reasons for dropout were related to the intervention. The authors stated that patients dropped out so early in the trial that minimal information would have been gained by completing an intention‑to‑treat analysis. The authors also stated that it is likely that the comparator group had fewer dropouts and that people followed their exercise regime closely with positive results partly because they knew that they would get a FES at the end of the trial, which is not reflective of real practice.

Burridge et al. (1997) was rated as having a low risk of attrition bias. Nobody withdrew from this study, although 1 patient from the FES group was excluded because the outcome data meant the patient was considered to be an outlier; this patient walked much more slowly than all other patients, resulting in an improvement that was considered to be non‑comparable. Although the outcome data was comparable for the 2 groups, the trial has limited external validity. Some patients were self‑referred from an advert in a newspaper and some were selected by treating physiotherapists as suitable for the trial. The authors acknowledge that this was a sample of particularly 'enthusiastic, compliant and apparently suitable patients'. Also, the authors point out that it is likely that the comparator group had fewer dropouts partly because of the incentive that they would get FES after the trial and that if they withdrew from the trial they would not have the 10 physiotherapy sessions, which is not reflective of real practice.

Four studies were rated as having low risk of detection bias (Barrett et al. 2009; Esnouf et al. 2010; Sheffler et al. 2013; Burridge et al. 1997). All studies had an appropriate follow‑up for the outcomes identified, although longer follow‑ups would have been informative. Sheffler et al. (2013) and Sheffler et al. (2015) followed patients after the treatment period. All studies clearly defined the outcomes and also used valid and reliable methods to determine the outcomes. Barrett et al. (2009) and Burridge et al. (1997) did not blind investigators to the patients' exposure to the intervention because this is not possible when measuring the orthotic effect. However, all measures were objective and so unlikely to be influenced by detection bias. Esnouf et al. (2010) used a standardised instrument to determine the primary outcome measure. This measure is subjective but investigators were blinded to patients' exposure to the intervention. Esnouf et al. (2010) also collected data on the number of falls as a secondary outcome from a diary kept by the patient. It is not possible to know if the diary was accurately completed. Also, the proportion of time spent walking with the ODFS or ankle‑foot orthosis was not recorded, so the fall data are difficult to interpret. Sheffler et al. (2013) also used validated instruments, but they state that the Fugl‑Meyer assessment may not be sensitive enough to detect clinically important changes in motor impairment.

Sheffler et al. (2015) has a high risk of detection bias. The outcomes were clearly defined and all but 1 (activity level) were measured using quantitative gait analysis (QGA). When having QGA, patients used no FES device or assistive device (cane, quad cane or walker) and the authors stated that the effect of this on the data is unknown. When patients have QGA, markers are stuck to the skin. The authors stated that there may be some bias from the inconsistent placement of markers between assessments. Finally, patient activity was measured using an activity logger for 3 consecutive days after each follow-up. The average total steps per day on these 3 days was used as a proxy for overall activity level.

The study by Street et al. (2015) was assessed using the CASP cohort studies checklist (CASP UK 2013). The study design ranks less highly in the hierarchy of evidence than an experimental study because there is no comparator group. The study recruited the cohort in an acceptable way using eligibility criteria, but no calculation was used to determine sample size. The study used data from standard clinical practice and patients were included based on GP and consultant referral data suggesting the sample is generalisable. Walking speed is an objective measure, which is unlikely to be biased. The functional walking category measure may be subject to some bias because a small change in walking speed may be enough to change a category. A larger change may not alter the category if the initial walking speed was near the lower threshold. The authors report that the clinically meaningful change in walking speed that they defined may not apply to this population. The value for a clinically meaningful change was derived from an elderly population of stroke survivors. This cohort was younger and had more profound disability specific to MS. The authors state that this suggests that a threshold for a clinically meaningful change may be overestimated in a population with more severe disability. The study authors took into account a previously seen confounding factor, which is the carry‑over effect of using FES immediately before unassisted walking, and adjusted the study design to measure the unassisted walk first. No subgroup analysis was carried out on baseline characteristics, such as those who used ankle‑foot orthoses at the start of the study, and there is no reporting of the severity or type of MS. Also, 4 different ODFS devices were used and analysis is not reported by device type. Follow‑up ranged from 16 to 24 weeks, meaning that not all patients were followed up for the same amount of time. People who were discharged from treatment for various reasons were not included in the analysis, which may bias the results in favour of FES. Extra analyses reported floor and ceiling effects (data values at the minimum or maximum scores that the test allows for) of the functional walking category, although the conclusion of this analysis is not reported. Finally, the results for training effect (defined as unassisted walking over time) were not reported clearly in the paper and were listed in a table under 'FES walk'.