3 Approach to evidence generation
3.1 Relevant evaluations
Published academic evidence
Few relevant studies specifically consider the cost effectiveness of virtual wards for acute respiratory infections, as noted in the guidance supporting documentation. There is more published academic evidence for hospital-at-home care, usually in frailty pathways, as this area has been operating for longer. However, older studies may not use the same remote monitoring technologies that are now available.
Academic Health Science Network or provider rapid evaluations
Several rapid evaluations commissioned by NHS England looked at virtual ward care, including in COVID-19 or respiratory pathways. These evaluations, typically done by Academic Health Science Networks, adopted mixed-method approaches to investigate patient experiences qualitatively and financial impact quantitatively. Future studies could improve on these rapid evaluations by collating more robust cost and effectiveness data, for example, by using costs derived from actual expenditure rather than averages based on average bed-day costs. Evaluations took place in the following locations:
South West London
The Health Innovation Network did a rapid evaluation of 3 virtual wards in Kingston and Richmond, Sutton, and Croydon. Kingston and Richmond and Croydon wards included respiratory pathways, including COVID-19 and chronic obstructive pulmonary disease. The evaluation adopted a mixed-methods approach, considering qualitative data on experience and implementation and some quantitative savings estimates.
Further evaluation work among Health Innovation Network members is ongoing across London, developing minimum datasets and other activities to support future evaluation.
Norfolk and Norwich
Initially focused on COVID-19, this virtual ward was expanded to other pathways. The evaluation, carried out by Norfolk and Norwich University Hospital and the local integrated care system (ICS), also follows a mixed-methods design. It includes qualitative patient experience data and estimates cost per patient for virtual ward care compared with an inpatient alternative.
Getting It Right First Time
Getting It Right First Time (GIRFT) have produced practical guidance on Making the most of virtual wards. It describes how the NHS can better use virtual wards. This summary guidance includes definitions of virtual wards and contrasts with other care settings, links to evidence and other resources, and information about virtual ward pathways.
Health Foundation
The Health Foundation's Improvement Analytics Unit is conducting an evaluation of frailty virtual wards, working in collaboration with the NHS England virtual wards programme team and local frailty virtual ward implementations. The evaluation will describe the characteristics of people admitted to the virtual wards and look at their typical trajectories of care. This will include examining medical histories before referral, as well as post-discharge outcomes such as rates of readmission or admission to hospital and the length of any subsequent hospital stays. The evaluation will also explore the feasibility of comparing individual-level outcomes with equivalent inpatient care. A previous qualitative evaluation (the Health Foundation's How do the public and NHS staff feel about virtual wards) was also done to investigate people's attitudes towards virtual wards.
3.2 Study designs
To address the evidence gaps outlined in section 2, a prospective, comparative before-and-after study design is suggested. This design can help maintain consistency in processes, staff, protocols and equipment used, but does not account for changes that occur over time and is limited in providing head-to-head comparisons between the different types of virtual ward platforms.
The study should compare people who are having care before the virtual ward technology is implemented (in inpatient secondary or community care) with people subsequently having care through the virtual ward technology across the same services. Comparisons in outcomes should be made between people who would have been eligible for care through virtual wards before implementation and people eligible for care through virtual wards after implementation.
Ideally, this study would include a comparison with a similar service that did not implement virtual wards during the same period, to assess for background changes in patient outcomes over time. Ideally, there would also be longer-term follow up to evaluate patient outcomes over a period after the end of virtual ward care.
Differences between 'step-up' and 'step-down' models of virtual ward care should be considered because these groups may have different clinical characteristics, resource requirements, and possibly be impacted differently by the technology.
Data that can enable matching or adjustment techniques should be collected to balance observed characteristics, creating comparable cohorts and reducing bias from observed sources of variation. Relevant confounding factors should be systematically identified with input from clinical experts.
A mixed-methods study embedded within this approach is also suggested to provide qualitative evidence on individual, carer, and healthcare professional preferences. This study should include patient, carer, and healthcare professional experience using the technologies and their acceptability. It should also consider additional burden for carers.
3.3 Data sources
NHS England is constructing a template for a national virtual ward minimum data set, with links to the current Community Services Data Set, and possibly to Patient Level Information and Costing System (PLICS) submissions. The proposed draft minimum dataset currently includes operational fields describing the pathway, such as admission and discharge dates and locations, source of admission, and identifiers for the ward and provider. It also includes clinical fields such as diagnosis codes and the diagnosis coding scheme, procedure codes, dates of procedures and the procedure coding scheme. The draft would also include demographic fields such as gender, age, ethnicity and location, which could be used to derive socioeconomic status. These demographic fields would allow for subgroup analyses and differential effects. Outcomes would include mortality as well as derived fields such as length of stay, virtual ward occupancy and readmissions.
This minimum dataset may be a good data source for subsequent evaluations, especially if providers can give appropriate coding depth in clinical diagnosis and procedure codes. These codes could be used for more robust matching or adjustment approaches in addition to the age and sex-matching used in literature.
Subnational secure data environments (SDEs) may also be a useful source of information if they have comparable data. They may also be ready sooner than the national dataset if there are operational reasons for faster adoption, for example, giving individual members access to dataflows from other organisations in their ICS. There would potentially be a trade-off between speed of collection and data quality if subnational SDEs do not initially collect or seek to collect all the items in the proposed NHS England virtual ward datasets.
In addition to the proposed core minimum virtual ward dataset, NICE recommends linking virtual wards to PLICS costing, as is the case for admitted care.
3.4 Minimum dataset
NICE recommends collecting the following data fields as part of the national minimum dataset or via another source:
-
Admission and discharge dates from virtual wards: this would allow the derivation of stay length and the ability to analyse change through time.
-
Source of admission: this would allow subgroup analysis of different referral routes. This field, or an additional field, could be used to differentiate between step-up and step-down care. Evaluations of virtual wards would need to analyse these 2 pathways separately.
-
Deaths within 30 or 60 days of admission onto a virtual ward: this could be recorded via linkage with other datasets, for example, death data from the Office for National Statistics.
-
Whether the virtual ward spell was preceded by an earlier virtual ward stay within 30 days: this would allow the calculation of readmission rates to virtual ward care, to evaluate readmission risks and pathway resource requirements.
-
Whether the virtual ward spell was followed by admission to inpatient care within 30 days: this would allow the calculation of risk of escalation to inpatient care, and the resource implications of pathways where people require inpatient care following a virtual ward stay.
-
Cost of virtual ward spell (linked via PLICS): stay level costs would require additional data collection and amendments to the process but would allow those evaluating virtual wards to better understand the costs of care provision.
-
Demographic information, including gender, age, ethnicity and socioeconomic status: this data would be required for matching or adjustment approaches and would also allow studies to do subgroup analyses.
-
Diagnostic and procedure information: this should be encoded in a recognised clinical coding system, for example, ICD diagnostic codes, OPCS procedure codes, or SNOMED codes. Scores from the National Early Warning Score tool could also be collated for purposes of measuring acuity. This information would give a better understanding of the healthcare needs and characteristics of virtual ward patients. It would also allow virtual ward cohorts to be better matched with an inpatient comparator group.
-
Qualitative data on patient, carer and healthcare professional experience and acceptability: this data would best be captured separately rather than as part of a large quantitative dataset. However, it would be important to better understand the views of patients, carers and healthcare professionals alongside quantitative results.
To provide a comparator to inpatient care, it would be helpful for this cost data to be similar to inpatient care cost datasets.
Cost data should be provided at the patient or stay level. Current cost calculations are typically based on simulated modelling or a cost-per-bed-day or cost-per-contact basis. Reporting average costs for inpatient or virtual ward stays risks an inappropriate cost comparison. While all virtual ward patients should be those who would otherwise be admitted to hospital, it is conceivable that they may have fewer healthcare needs than the average inpatient stay. Consequently, simple (naive) comparisons between virtual ward and inpatient outcomes and costs would overstate virtual ward benefits, as the most acute inpatients may not be suitable for virtual ward care.