This report will focus mainly on the applications of VR in healthcare, with reference to the use of AR and MR in training and education.
Extended Reality (XR) is an umbrella term encapsulating AR, VR, MR and everything in between.
Virtual Reality (VR) immerses users in a fully digital environment through a headset or surrounding display. This environment can be computer-generated or filmed in 360-degree video.
Augmented Reality (AR) presents digital information, objects, or media in the real world through a mobile device or headset. These elements can appear as a flat graphical overlay or can behave as a seemingly real ‘3D’ object.
Mixed Reality (MR) is the latter form of AR described above where physical and digital objects co-exist – in other words, the digital objects appear anchored to the real-world environment.
Immersive (or XR) technology solutions are being delivered in a variety of ways. Augmented reality experiences are usually accessed through a mobile phone or headset, displaying an overlay of digital objects in the user’s environment. Virtual reality is used to immerse users in a completely digital environment, usually through a headset.
Immersive technologies cover a wide spectrum and can also include adjacent technologies such as haptics which assist in immersing the user in a virtual world through an enhanced sense of touch. Often taking the form of a pair of gloves or an articulated arm, haptic technology enables users to not only see and hear their environment but feel it through vibration and changes in temperature.
A range of Head Mounted Display (HMDs) and devices are available on the market to deliver immersive experiences. Below are examples of some of the most commonly used headsets as well as other interfaces used to experience VR and AR.
Estimates suggest that in 2020, sales of XR headsets will reach around 5.5 million units. Forecasts project massive growth in both AR and VR headset sales in the coming years, with both technologies combined expected to sell over 26 million units per year by 2023. (Statista March 21).
The picture here is one of increasing access to and usage of devices and interfaces which can support the delivery of XR in healthcare solutions and experiences, at an unprecedented level.
The range of HMDs, devices and software available on the market is set to increase with the expected launch of Apple’s AR glasses, Microsoft Mesh -which enables presence and shared experiences through MR across devices - to Facebook’s neural signal wrist trackers.
Some XR devices contain highly sophisticated sensors that can record physiological data on a user. This can be sent either to a clinician for monitoring or directly back to the user in real-time to provide information about difficult-to-observe functions such as breath or brain activity, with the aim of improving the user’s conscious control of these functions (biofeedback).
These enhanced biometric sensing and communication devices enable the gathering of valuable information that can be used to help consult on, diagnose and treat a range of psychological and physical conditions. These technologies can also help validate certain approaches and enable more personalised and precise treatments for patients.
Some headsets have integrated sensors that can not only detect traditional biometric data, including heart rate and heart rate variability, but also a user’s respiration, expression and emotion in response to live VR experiences. This data can then be output to a backend report and analysed. A range of external options are compatible with headsets, including heart rate monitors, eye-tracking add-ons, breath belts and galvanic skin response devices. Increasingly, eye-tracking sensors are being built in to monitor eye movement and position to enable greater understanding of presence, attention and focus.
Immersive experiences can be developed on several platforms. The most popular are Unity and Unreal, which account for the vast majority of XR applications. Some startups are also offering their own platforms, which simplify the creation and distribution of XR experiences.
WebXR technology is making access to XR experiences even easier for users. Instead of creating an application that needs to be downloaded from a store and installed locally on a user’s device, it can simply be loaded from a web browser on the user’s mobile phone or headset. This distribution model is becoming increasingly popular as it makes XR experiences quicker and simpler to access. However, it can pose a challenge where the internet is not available, not reliable, or restricted as may be the case with some hospitals. Mozilla Hubs allows users in VR or on desktop computers to access a shared virtual world together and is a good example of WebXR technology in action.
The global immersive technology market is accelerating out of the Covid-19 pandemic and reaching the tipping point of wide scale penetration in both the enterprise sectors and the public domain. Worldwide spending on AR/VR is forecast to accelerate, growing from just over $12 billion in 2020 to $72.8 billion in 2024 (IDC Nov 2020)
The pandemic has created a shift in mindset and a rapid acceptance of AR and VR in various sectors as businesses turn to a remote working business model. A greater focus on ROI and productivity and efficiency gains continues to ramp up interest in enterprise AR and VR, supported by the 2020 XR Association Study which shows that 80% of respondents in manufacturing, 75% in healthcare, 74% in education, and 70% in public safety predicted that their organisations will increase spending on XR technology over the next five years. An interview with PwC’s Jeremy Dalton on XR In Industry analysis in 2021 discovered that across the UK there are over 97 organisations using XR technologies for healthcare, and at least 119 distinct projects underway.
Mental health is a growing concern nationally, with mental illness estimated to be costing the UK economy up to £100 billion a year. It is reported that 1 in 4 people in England will experience a mental health problem every year. In addition, it’s predicted that the impact of COVID-19 will see up to 10 million people needing mental health support as a direct consequence of the crisis. That’s 21.7% of England needing additional support from an already burdened system.
Empowering people to actively engage in their health, education and wellbeing is an ongoing challenge. VR has already been assisting treatments for mental illnesses, such as phobias, anxiety, eating disorders and post-traumatic stress disorder. Digital apps are already widely being used by the NHS to support patients’ mental health and the cost of such technologies is falling. Psychiatrists are already implementing VR to treat patients with mental illness, anxiety issues and eating disorders. Using biosensors coupled with VR, they can now get a better idea of how their patients react and cope with stressful scenarios. VR is increasingly being used to help autistic patients develop social and communication skills. By tracing eye movements, it can also diagnose visual impairments in patients and detect the early stages of Alzheimers and Dementia in adults.
Enabling people to embody experiences that invite them to actively participate in their recovery, develop new skills for self management and expand their knowledge is vital to maintaining an informed work force and healthy population. Building on the immersive and embodied movement within virtual reality experiences, physiotherapists are increasingly using immersive technology to aid in stroke rehabilitation, occupational therapy and pain management. Alongside its potential as an intervention for pain management and physiotherapy, it can progressively be used as a tool for preventative care.
Immersive technologies represent the next generation of precision medicine and patient-centred care, with opportunities to deliver personalised, participatory, data driven experiences. More and more, headsets are being prescribed to at home patients, delivering services remotely, outside of clinical settings. As well as remote delivery, therapies and training can be distributed through a range of contexts, from clinical settings to wellbeing centres, classrooms and at home which can cut costs significantly, and allow healthcare providers to focus on more complex cases that require face to face interaction.
Finally, through creating immersive training experiences, staff can be trained remotely and gain first-hand experience of complex training procedures. Many of these tools free up clinician time and decrease patient mortality or harm.
It cannot be ignored that VR was born from the gaming and entertainment industries. Indeed many examples of the use of XR in mental health today involve the skills and expertise from sectors adept at storytelling, using narratives and game play as powerful interventional tools for eliciting effective audience response. This, combined with the creative technology skills to create rich immersive and engaging environments, creates compelling stories which generate a framework that can be used to support the improvement of mental health.
It is vital that the NHS harnesses new innovation and technologies that can improve the wellbeing, education and quality of life of the population. This report demonstrates the value and potential of immersive technologies to transform how and where clinicians offer and patients receive care both in the UK and internationally. We are deeply excited for the benefits XR will bring to the NHS, and the global health and social care systems.
Rhodri Joyce, Deputy Director of Innovation Development at NHSX
The diversity within any industry is vital to its subsequent success. Within healthcare controls, it is vital that those involved in the development and deployment of XR in healthcare are representative of the UK population. Currently, there is a significant lack of diversity within this sector, and major barriers still need to be overcome to improve diversity and representation through the tech, games and healthcare industries.
In 2018, A Vision for Women and VR (VWVR) report, undertook quantitative research with King’s College London and the University of Brighton discovered that only 14% of all VR companies in the UK are female led.
There is a lack of data on the presentation of black and minority ethnic populations within the XR industry. The Inclusive Tech Alliance Report (2019) discovered similar representation of women in tech. It was found that 74.5% of boards and 70.6% of senior executive teams in the top tech firms have no BAME members.
Increasing diversity of teams and senior staff developing programmes to attract a range of creators, researchers and healthcare providers is vital to the success of this industry. Educating designers about accessibility needs to be a core element of every design course. Inviting people with a range of abilities and relevant lived experience to participate in the design of those tools is vital to the success of a tool or product. Examples of successful participatory design processes include gameChange, who developed a transdisciplinary approach, working with clinical psychologists, programmers, animators, designers, product managers, producers, writers, researchers, 3D artists, mental health advocates and people with lived experience of psychosis to co-create the final tool.
Groups such as XR Access and W3C are working to develop standards to support the development of accessible tools for physically and neuro diverse populations, ensuring that accessibility is built in from the beginning of any project.
As Extended Reality (XR) technologies continue to improve, their potential within the treatment, diagnosis and management of mental health disorders continue to excite. While many of the technologies, including Augmented Reality (AR) and Mixed Reality (MR), have yet to be fully investigated and evidenced, the area of Virtual Reality (VR) has seen significant advancements in the last few decades since the results of the first clinical trial to use VR to treat mental health were published in 1995. Along with improvements in functionality and availability, new and innovative ways to embed XR into treatment and mental healthcare were being explored. In the early to mid 1990s, VR was used to simulate scenarios for specific phobias as it offered a way for clinicians to expose people in a way not easily replicable in real life, where stimuli could be presented, controlled and people’s responses captured. As VR technologies have advanced, its application within clinical mental health, psychology and psychiatry has expanded.
Evidence for its effectiveness has been found within several conditions including neurodevelopmental disorders, psychotic disorders, depression, anxiety disorders, and eating disorders (Eshuis, Gelderen, Zuiden, Nijdam, & Vermetten, 2020; Falconer et al., 2016; Ferrer-García & Gutiérrez-Maldonado, 2012; Morina, Ijntema, Meyerbröker, & Emmelkamp, 2015; Opris et al., 2012; Parsons & Cobb, 2011; Valmaggia, 2017). Within these studies, VR has been found to be effective as a way to deliver various components of therapy including exposure and cognitive restructuring. Virtual environments are also being used as a tool in novel therapies that have evolved to complement its innovative use.
The broader use of immersive technologies, including AR or MR, are in the early stages of research. This is mainly because the processing power necessary to render high quality graphical environments has only recently become available at scale e.g., augmented reality functionality allowed by newer smartphones (Baus & Bouchard, 2014). With increased capabilities from devices such as recently released MR headsets and haptic suits, there is potential to design more complex therapeutic applications, for example, promoting autonomy for people with neurodevelopmental disorders (Aruanno, Garzotto, Torelli, & Vona, 2018) and promoting collaborative behaviours, such as those between children with autism and their peers (Crowell, Sayis, Benitez, & Pares, 2020).
Dr Dennis Chan, Consultant Neurologist & Principal Research Fellow, UCL worked in close collaboration with Professor Neil Burgess at UCL, a team of scientists at the Department of Clinical Neurosciences at the University of Cambridge to develop and trial a VR navigation test in patients at risk of developing dementia (Howett et al, 2019). Mild cognitive impairments can be an early sign of underlying Alzheimer's disease, however often this is difficult to test for and can be caused by other conditions such as anxiety and normal aging. Clinics currently rely on pen-and-paper cognitive tests and are unable to test for navigation difficulties. The alternative and more precise “gold standard” of testing is the extraction and analysis of spinal fluid, which is extremely painful and highly costly. In the test, a patient dons a VR headset and undertakes a test of navigation while walking within a simulated environment. Successful completion of the task requires intact functioning of the entorhinal cortex, a region of the brain described as the ‘internal satnav’. This is one of the first regions to be damaged in Alzheimer’s disease, which may explain why ‘getting lost’ is one of the first symptoms of the disease.
This VR based navigational tool is able to test for early signs of mild cognitive impairment before showing clear symptoms at a fraction of the price. As it is able to test behaviours in simulated real world scenarios, it possesses higher ecological validity. By digitising and automating the process, a broader range of society can use the tool, which can lead to a more timely and prepared response to anticipated onset of Alheimers disease.
Reference: Howett, D, Castegnaro, A, et al. Differentiation of mild cognitive impairment using an entorhinal cortex based test of VR navigation. Brain; 28 May 2019; DOI: 10.1093/brain/awz116
While the clinical use of VR may once have been mainly driven by design researchers, the perception of it as a component of treatment has improved in recent years. As familiarity with platforms and kits has grown, driven by gaming and more affordable consumer devices, people are more aware, and accepting of how it could be applied in a mental health context. In a recent survey that asked the public about their perceptions of therapy delivered within VR by a virtual coach, those with more experience of using VR in any context were more optimistic about its therapeutic use (Allison & Bergin, n.d.). Likewise, therapists surveyed about their attitudes toward the use of VR within Cognitive Behavioural Therapy (CBT) who viewed VR more negatively also reported that they would be less likely to use it in the future (Lindner et al., 2019). Findings overall suggest that while many do not think it will be able to replace face-to-face provision of therapies, there is a positive view towards the use of VR in psychological therapies. There is also significant work being done to support more ‘blended’ treatments that combine technology-supported therapies and more traditional face-to-face treatments to make better use of the resources available. It is important that these approaches are communicated with patients and clinicians to help overcome any resistance to the use of XR within mental healthcare.
Initial applications mainly focused on how VR could be used within exposure therapy. This type of therapy directs people to the source of their anxiety and so has applications in the treatment of Post-Traumatic Stress Disorder (PTSD), phobias and social anxiety disorder. VR offers a way for therapists to fully control the environment and the extent of how challenging the exposure is. For example, while a fear of spiders could be treated by presenting a small spider and gradually exposing the individual to bigger and bigger spiders, there is a limit to how large spiders can get. Within VR the spider can grow to the size of a house or could start off as a cartoon and become more realistic. For PTSD where the anxiety is linked to a specific highly stressful situation, it can be hard to replicate an authentic experience in the real world but a VRsimulated event can be recreated and explored from different perspectives. Researchers have found that people who do something stressful have a very similar response when doing it in VR, both bodily and mentally – in other words, it feels real (Martens et al. 2019). Being exposed to something you are scared of or a real situation that makes you anxious can be a very intimidating experience for people and many refuse to be treated in this way. Interestingly, people are more likely to choose to be exposed in VR rather than in real life (García-Palacios, Hoffman, Kwong See, Tsai, & Botella. 2004) even if it elicits the same levels of fear. There are many potential reasons for this – it may help an individual to overcome initial reluctance by offering something they see as less distressing and where there is no actual threat. It may seem real but they know that it isn’t; a form of suspension of disbelief.
In more recent years the use of VR within psychological therapies has evolved and it is being applied to both the diagnosis (Ferrer-García & Gutiérrez-Maldonado, 2012; Sorkin, Weinshall, Modai, & Peled, 2006) and treatment of specific conditions such as psychosis, depression and anorexia nervosa. VR has often been used as a tool within a package of care with a therapist who controls the environment and works with the individual to address the problems they have faced. Some recent examples of treatments include the therapist as a virtual embodiment (Craig et al., 2018) or the individual embodying, at different times, an adult and child version of themselves (Falconer et al. 2016). In the former, a novel treatment was developed for individuals who experience persecutory auditory hallucinations but do not respond to antipsychotic medication. The individual, with the support of the therapist, develops an avatar to represent the entity that they hear, with both the face and voice being fully customisable. The therapist then takes this form on a computer screen, enabling a dialogue that can be used to help individuals overcome their fear of the entity and challenge its rhetoric. Researchers have reported that for some the hallucinations completely disappear (Leff, Williams, Huckvale, Arbuthnot, & Leff, 2014). For the treatment where an individual embodies an adult and child, the aim is to promote self-compassion in the treatment of depression. By first delivering compassionate sentences to a virtual child and then embodying the child and hearing these delivered, the individual’s self-compassion increases and self-criticism decreases, leading to a reduction in clinical depression scores. Still in its early days but with several small clinical trials demonstrating feasibility and efficacy, the provision of automated therapy within the virtual environment itself using virtual coaches or agents is being explored (Freeman et al., 2019). This promises not only to increase access, since the professionals delivering it does not need the same level of clinical training, but it may also lead to XR therapies that can be self-delivered from home. While psychological therapies will always require clinicians with appropriate training, there is the potential that XR could offer an entry point, a way of preparing the individual and supporting them towards recovery. It also could offer more targeted or specialised psychological therapies that are unavailable in some areas.
Clinicians and researchers in the UK are developing not just products but entire treatment packages that tap into the many novel capabilities of XR. While these are mainly being delivered within research environs, such as the Virtual Reality Lab at KC and O-CAP in Oxford, there are also several NHS Trusts adopting innovative XR treatments. OVR and ProReal have both worked with Oxford Health to develop Global Digital Exemplar (GDE) blueprints, which provide an implementation toolkit that other trusts can follow. This suggests that while other areas may yet be slow to embrace XR, there is a real drive within the NHS to begin using XR technologies in mental healthcare.
Increasing people’s engagement with XR mental health treatments can help overcome the considerable number of barriers that prevent people getting the treatment they need such as accessing the right treatment at the right time, when there may be stigma around or resistance to using mental health services. XR approaches can be used as a tool to target and reduce waiting lists.
Given the increased demand for mental health provision and support as a result of the COVID-19 pandemic, increasing support for the creation and delivery of XR mental health solutions like offering remote treatment where patients exhibit a preference to be engaged at home, has the potential to address some of the major challenges facing the UKs mental health sector today.
OxfordVR are developing VR mental health interventions that build on the work of Professor Daniel Freeman of Oxford University. Currently they have three offerings – Fear of Heights, gameChange and Social Engagement. gameChange was winner of the 2017 National Institute for Health Research Mental Health Challenge Award and is currently being trialled across five NHS Trust sites. People with psychosis are faced with many difficulties engaging in day-to-day life where they are faced with everyday situations such as getting on a bus or doing their shopping that can be extremely challenging. This leads them to withdraw from the world which has an impact not only on their mental health but also their physical. Providing a safe and controlled environment in which they can learn to overcome their anxiety, guided by Nic the virtual coach, not only provides patients with an intervention but also offers an effective and low cost complement to existing care.
It is the investment of funders such as NIHR and the MRC that enable innovative treatments like this to be developed and evaluated. The high-quality evidence that this produces is invaluable. As interventions, Oxford VR’s offers require minimal additional costs or therapist input apart from the equipment and software. They have significant benefits including improved outcomes, increased access, better engagement and greater efficiencies. At a time when the NHS is facing long waiting lists for mental health services, the adoption of successful digital treatments offers a potential solution. However, OxfordVR are facing a difficult challenge scaling up within the NHS and are currently launching a consumer-facing offering within the US as a way to ensure sustainability and scale during the long wait for an NHS contract.
According to Chief Commercial Officer, Mike Desjadon,
“The NHS and US Healthcare Systems are different in many ways, but similar in one: contracting and approval processes make it very difficult to introduce innovations to patient care. Given the (possibly unfortunate) fact that close to 50% of mental health care is paid for out pocket in the US, we’re turning our attention to this market to scale the business and to bring this innovative treatment directly to the millions that need it. Our strong hope is to continue working with the NHS and to offer these new treatments to citizens across the UK – perhaps a successful launch in the US will help push this forward.”
VR has also demonstrated the enhanced therapeutic potential of mindfulness and meditation. Through creating relaxing environments that people can escape to, from soothing beaches to forests at sunset, with a range of gentle relaxation activities available. VR relaxation experiences have become an increasingly popular application for at home users, and increasingly in workplace, care home and healthcare settings(Sonney, J. et al, 2021). There is also arguably a use for technology in waiting rooms or hospitals and to help anxious patients prior to surgery or other stressful situations. This can drastically improve patient cooperation, and offer immediate support in helping people self soothe during challenging moments.
Deep is a meditative VR experience controlled by breathing. Players explore an expansive and beautiful underwater universe, controlled by diaphragmatic breathing. The experience combines meditation, slow diaphragmatic breathing and biofeedback tools to visualise internal states to increase interoceptive awareness of the physical and inner emotional states. Deep is controlled by a breath belt or gentle tai chi hand movements synced with the breath. The player’s biometric data is subsequently visualised within the experience, teaching meditative breathing in an embodied and intuitive way to develop long term skills for anxiety regulation.Following an extensive R&D process in collaboration with The Games for Emotional and Mental Health Lab, Radboud University, Netherlands, Deep is now being trialled in a range of settings including supporting students with complex needs, patients in palliative care settings and the rehabilitation of patients with COPD. By creating a visual feedback loop that rewards positive behaviour associated with relaxed physiological states, evidence demonstrates that the intervention is able to reliably reduce momentary bouts of anxiety as well as anxiety experienced over a longer period of time with more extensive practice. Short sessions with deep can already result in a calm state which lasts for two hours on average. Furthermore, deep can increase important markers of psychological wellbeing such as self-efficacy (Weerdmeester et al., 2021; Bossenbroek et al., 2020).
“Diaphragmatic breathing is an effective anxiety regulation technique; it immediately brings our awareness back to the body. Changing our behaviour or thoughts is very abstract, but changing how we feel by adapting the way we breathe is something we can often more easily control. Due to virtual reality’s deep immersion and sense embodiment, learned skills are more likely to become automatised and internalised so that they transfer to people’s daily habits.”
Doctoral Researcher, The Games for Emotional and Mental Health lab, Radboud University
A range of 3D drawing, sculpting and building experiences have been developed that offer free form creation tools that have potential for art and play therapy(Hacmun et al, 2018). Companies such as Hatsumi have developed a VR adaptation of arts and health research methods, Body Mapping, that enables people to visually communicate the embodied experience of pain and emotion using 3D drawing and sound. The team also creates public exhibitions of anonymous artworks to raise awareness of invisible conditions and the connection between emotional and physical pain. Train play therapists have also become increasingly interested in using virtual reality with their clients. Dr. Jessica Stone, founder of Virtual Sandtray, has translated traditional sandtray therapy into an iPad app and virtual reality experience.
Research has shown that group singing in a virtual environment can improve outcomes as much as singing in person (Daffern et al, 2019). In 2020, Limina Immersive developed and tested Lost in Song, a 360 video for wellbeing with song and nature, in collaboration with NHS Arden & GEM and Ex Cathedra. The experience is a desktop-based 'nature karaoke' singalong with a professional choir accompaniment and bespoke arrangement of well-loved songs including Stand by Me, Jerusalem and The Sky Boat Song. The sing-alongs occur in beautiful parts of the British Isles that are relevant to the song.
81% of users experienced a mood boost after experiencing the content, and 90% found the immersive experience easy to navigate, despite most having no prior experience of 360 videos. You can check it out at www.lostinsong.com.
The global quarantine over 2020 demonstrated how detrimental isolation and loneliness can be to maintaining emotional wellbeing. Research by Riva et al (2020) discovered that spending time with friends through a range of social VR platforms has a positive impact on users. Being able to meet and physically interact with friends in a virtual space can offer a heightened sense of presence and connectedness. A range of multiplayer options are available, from formal social meet ups for events and discussions, to dance games, golf, table tennis and the eagerly anticipated “MultiBrush” a collaborative version of the 3D drawing experience Tilt Brush, which was developed following the open-sourcing of Google Tilt Brush in 2020.
Engaging in an active and healthy lifestyle can increase wellbeing outcomes. A range of games are available on the market, and people are using movement-based exercise games to improve adherence to exercise (Damiani, J. 2020). 2021 has seen a range of XR fitness experiences enter the market, including the eagerly anticipated SuperNatural VR, which combines popular music, inspiring landscapes, and motivating fitness coaches to provide an enjoyable way to experience a high energy workout at home. Research suggests that the combination of VR and exercise may improve some of the beneficial psychological effects of exercise compared with virtual reality or exercise alone. Participants demonstrated an increase in enjoyment, energy, and reduced tiredness (Plante. et al, 2003). Additional research has found that this is a suitable offering for people living with physical and intellectual disabilities to encourage ongoing exercise (Lotan, M. et al, 2009).
“I think what’s cool is what we learned through the process of this, that if you can utilise all that same intangible magic of art and storytelling and crafted human experience - but calibrate it for a higher purpose; that rather than just entertainment, calibrate it to health, calibrate it to making your life last longer, making your life better from a day-to-day basis, let you have more energy, let you have a clearer head, let you be stronger. - then, you can actually use the power of art for a different purpose and one that actually does hold that original potential of changing someone’s life.”
Chris Milk, co-founder of Supernatural interview in The Verge Source.
In 2017, following the fire at the Grenfell Tower, Central and Northwest London NHS Trust, in collaboration with a creative research organisation, The FRED Company, took VR out onto the streets and into the community centres of West London.
“We use VR to start a conversation, but were very surprised as to just how effective it was” explains Rosie Collins from FRED.
Healthcare professionals invited passers-by to use a range of entertainment-based VR experiences, including roller coaster rides and 3D drawing games, as well as bespoke VR mindful content created as a response to the tragedy. The aim was to re-establish trust with statutory services, break down barriers with the community and improve referrals to the local mental health services.
The use of VR has been used across care homes to reduce apathy and associated issues such as dementia. Loneliness and lack of intellectual stimuli can lead to a faster rate of cognitive decline, reduced quality of life, and high caregiver burden. In 2020, Saredakis, D. et al developed a study using VR as a tool for reminisce therapy with groups and used it to enable people to return to places of significance, alongside 360 content available on YouTube. Providing relaxing and entertaining experiences that can engage people who are frail and unable to travel far can bring extremely positive benefits. Additionally, the opportunity to bring VR to palliative care settings has been found to have extremely positive results.
Dr. Sheila Popert, Medical Director at Prospect House Hospice, has been utilising VR over the last five years. She initially developed an experience titled Forest of Serenity, voiced by Sir David Attenbrough, as a tool for pain management. Through her implementation, she found the experience was effective in reducing anxiety, and has broadened the scope of their research and deployment to support people’s emotional wellbeing both in the hospice and at home.
“We use VR for opening up people’s worlds,” she explains. “When patients are stuck alone in a room, their world shrinks.”.
Not only can VR improve the wellbeing of those in palliative care settings, but can directly reduce people’s fear of death. In a series of studies by VR researcher Mel Slater, specifically developed experiences were found to reduce people’s fear of death, and lead participants to become more beneficent, self accepting and concerned for the wellbeing of others. They reported a higher sense of life purpose, self reflection and a stronger belief in social justice (Bourdin et, 2017).
It is common for us [end of life doulas] to hear people who are at the end of their life say “I would love to see that place one more time”, or “I wish I could have seen...”.
If VR can provide an opportunity for people to virtually visit these places, I can see there being high demand.Emma Clare, Director of End of Life Doula UK
In 2020, Helen Davies-Cox, NHS clinical entrepreneur head of personalised care for Torbay and South Devon NHS Foundation Trust, led on the development of a VR integrated social prescribing initiative, as part of their HOPE program (Help Overcoming Problems Effectively). They invited a group of house-bound plus-sized patients, suffering from a range of challenges affecting their health and wellbeing including alopecia, anxiety, depression and sleep deprivation, to participate. Headsets were prescribed for at-home use over the course of six weeks, with a range of wellbeing applications being delivered, alongside connections to the other programme members.
The results included an increased night-time sleep from two to three hours to seven to eight hours at night in one participant; noticeable hair regrowth in a person with alopecia - resulting in improved confidence and self-esteem - and a desire to leave the home for leisure from a number of patients who had not left the house in a long time.
A number of universities and research centres have taken interest in this field. Organisations such as StoryFutures Academy, based at Royal Holloway University, have developed a number of programmes to support new immersive projects to help health and wellbeing. They supported the Nesta X StoryFutures Immersive Wellbeing Fellowships in 2020, with projects including interventions to improve self-compassion for women of colour in workplace environments and tools for anxiety regulation to be used within school contexts.
These initiatives have been followed by additional programmes, which are under development and preparing for release in 2021. These include Positive Realities: XR for Youth Wellbeing, developed in collaboration with Coventry UK City of Culture, and a programme with the Roald Dahl Foundation supporting the development of augmented and mixed reality R&D prototypes exploring how immersive storytelling can improve ‘transition of care.’
Immersive technologies as a tool to improve wellbeing to benefit the wider population has had success in a range of contexts, from festivals and public spaces, to alleviating stress in the workplace, at schools and in the home. The covid pandemic saw a huge paradigm shift of health seeking behaviour. Digital services providers reported a recurring observation that demand for digital mental health support has risen, and that the nature of this demand has also changed since COVID-19, with an apparent increased presentation of anxiety and loneliness. Conversely, at the same time, referral rates to NHS primary care mental health services dropped (Inkster et al. 2021). Individuals chose quick and easy access to self care wellbeing support. XR wellbeing has an open door to occupy this space.
The UK's immersive tech sector is based on the foundations of a thriving games, digital, creative and entertainment industry, with more and more businesses entering the virtual, augmented and mixed reality market. Since the pandemic, the games industry has witnessed a 30% increase in valuation. This is creating more opportunities for both industries to collaborate on new powerful experiences with real-world impact that can open up new markets and deliver world leading XR mental health solutions.
The growing need to support increasing mental health demands creates new opportunities to harness the potential of technology as part of a broader holistic approach to improving health and wellbeing. Immersive technologies have the potential to play a significant role in preventative care by teaching self compassion and valuable skills in emotional regulation alongside fitness and exercise based activities that can delay or prevent the onset of a range of physical and mental health conditions.
Prior to the pandemic, workplace stress led to an estimated 12.8 million working days lost, accounting for 44% of work-related ill health in 2018/19 alone, according to a HSE report in 2019. The impact of COVID-19 has led to a monumental shift in staff feeling overworked and dealing with chronic stress, anxiety and depression. In early 2020, VR healthcare company Rescape, partnered with Cwm Taf Morgannwg University Health Board (CTMUHB), Wales, to deliver a staff wellbeing programme using a series of VR initiatives.
Rescape developed DR.VR Frontline, an iteration of its existing product, DR.VR®, a clinician directed device that enables control of the patient’s VR experience via tablets, for staff to use at home, with a range of content designed for relaxation and breathing. The product had already been used within clinical and care environments for patients, primarily focused on reducing pain and anxiety and was deemed suitable to be deployed to individual frontline staff to help mitigate the impact of stress during the pandemic.
Headsets were deployed to frontline medical and nursing staff within the Critical Care Units of the Royal Glamorgan and Prince Charles Hospital. Each contained five scenarios with a simple interface, enabling a plug and play experience which they could use at home, including:
Dr Kim Smallman, Research Design and Conduct Service Consultant and Research Associate, Cardiff University, and Dr Michelle Smalley, Clinical Psychologist, Cwm Taf Morgannwg University Health Board, developed an evaluation of the experience, investigating how effective the DR.VR® platform would be in supporting the mental health and wellbeing of frontline staff during the COVID-19 pandemic.
The outcomes included a helpful distraction and relief from work, with 88% of participants reporting reduced feelings of stress. 94% felt it was enjoyable and provided relief from their work. The most helpful experiences reported were the meditative spaces and breathing exercises, with over half of respondents referring to these.
“The mindfulness areas teaching me how to calm and breathe were amazing for my worry and anxiety.”
After observing the initial success of the service evaluation, the CTMUHB psychology and staff wellbeing team purchased 45 DR.VR® Frontline headsets. They are currently working with the Rescape team on the deployment across multiple sites and staff groups including ICU. RDCS and the Centre for Trials Research are also continuing to work with CTMUHB and Rescape to identify funding streams that will support the development of further research within this area.
VR has brought to light new opportunities for pain management in painful therapeutic processes. It allows users to experience a computer-simulated reality with visual, auditory, tactile and olfactory interactions which combine to result in distracting the patient from perceiving nociceptive signals and pain. With the first-ever NICE guideline (NICE. 2021) on chronic pain stating that GPs should not prescribe opioids to patients with chronic primary pain because they could be ‘harmful’, there is a more pressing need than ever to explore digital healthcare solutions.
There are two types of pain related to this study. Acute pain, provoked by a specific disease or injury, serves a useful biologic purpose, is associated with skeletal muscle spasm and sympathetic nervous system activation, and is self-limited. Chronic pain, in contrast, may be considered a disease state. It is pain that outlasts the normal time of healing if associated with a disease or injury. Chronic pain may arise from psychological states, serves no biological purpose, and has no recognisable end-point. (Grichnik KP, Ferrante, FM.1991)
Almost half of UK adults may be living with chronic pain (Fayaz A, Croft P, Langford RM, et al. 2016) and treatment has for many years predominantly taken the form of a reliance on opiate based medication with more recent developments in Cognitive Behavioural Therapy (CBT) offering a non-pharmaceutical approach to pain management.
The effect of the prescription opioid crisis in North America is well documented. In 2018 some 68,500 Americans died from opioid overdoses, and in 2015 the cost of opioid misuse to the US economy was estimated at £110bn. In 2015, opioids were prescribed to roughly 5% of the UK population Between 1998 and 2016, opioid prescriptions increased by 34% in England, and the total oral morphine equivalent dose increased by 127% to 431 000 mg/1000 population/year. A recent UK study found that 14.6% of people given opioids for the first time became long term opioid users within a year (Levy N, Lord L J, Lobo D N. 2021). The use of virtual reality as a primary or complementary pain management practice therefore represents an inviting alternative to prescribers and patients alike.
Brennan Spiegel lays out the beneficial effects of VR for pain management in his book VRx (2020), Spiegel posits
“VR lowers the perception of pain in at least three different ways. First, it distracts the brain from noxious signals rising up from the body. Second, it creates an illusion of time acceleration, effectively shortening the length of pain episodes. And third, it nips signals in the bud at their origin, blocking pain from reaching the brain. The combination of these effects supports the ability of immersion to fight pain.”
VR has been used to manage pain and distress associated with a wide variety of known painful medical procedures and conditions. It has been used in acute pain such as childbirth, episiotomy repair, burn dressing management, cancer pain, colonoscopy, and a range of other routine medical procedures. It has also been applied in chronic pain for conditions including fibromyalgia, neck pain, back pain or chronic regional pain syndrome, although to date only a few studies have investigated VR for chronic pain management and the data is preliminary.
In clinical settings and a number of UK and international trials (Li A, Montaño Z, Chen VJ, Gold JI. 2011), participants immersed in VR experiences demonstrated reduced levels of pain, general distress/unpleasantness and reported a desire to use VR to help manage the pain. A growing body of evidence points to VR acting as a non pharmacologic form of analgesia that exerts an array of emotional affective, emotion-based cognitive and attentional processes on the body’s intricate pain modulation system. Potential benefits include a reduction in opiate addiction, illness or reduction in deaths; decreased medical bills and subsidies to pharmaceutical companies and perhaps the most compelling reasons to use VR for patients are:, no side effects from medication, a less painful experience of surgery and ability to manage their own pain symptoms.
Chronic pain manifests in people where sustained injury, perhaps to tissue, provokes a neurological response whereby pain signals are sent to the brain over a sustained period of time. Over this extended period the signals become exaggerated, distorted and not representative of the original physical injury. This constancy of pain signals over a prolonged time can lead to other psychological effects including anxiety and depression and the constancy of these signals towards the brain can lead to a vicious cycle of ongoing pain as the brain fails to block more signals arising long after the initial injury has disappeared.
Diane Gromala was an early pioneer in the application of VR for chronic pain management. Her work focuses on the intersection of art, design, technology and the emerging world of bio-feedback to create environments which the participants could interact with in real time. Gromala developed an immersive virtual environment, and stereoscopic sound titled the " Virtual Meditative Walk’ (Gromala, D, Tong, X, Choo, et al. 2015), which led the way for a lot of VR healthcare environments and interactions. The walk was designed for pain modulation, and improving interoceptive awareness. The medium offered a chance for patients to “learn to exert some form of control or agency over their experience of unrelenting pain.” (Tong et al, 2015) In the UK we see fledgling growth of healthcare organisations working in the field of chronic pain and some patients taking matters into their own hands.
Kay Smith lives with chronic pain, has lupus with five cascading autoimmune diseases and is currently in palliative care. One disease is called mast cell activation syndrome, MCAS for short. There is little known about this disease, but what it means for Kay is that it prevents her from taking any medication that can treat her lupus. Because of this, the lupus has taken over her internal organs. Kay is now allergic to antibiotics, painkillers, and most other medications. She has tried a range of alternative medications, and in her words, they generally make her feel worse. This all changed when she was first introduced to VR.
The first time Kay used VR as a medium to manage pain, she became immediately aware after putting the headset on that she wasn't in pain. Kay described a state where she felt total presence in the new underwater VR environment she was experiencing for the first time. Kay reported that it was the first time in two and a half years she had experienced life pain free and for her, that was priceless. VR had given her back her “self.” The experience of VR scuba diving reminded her of who she was. It reminded her that she is more than just a patient in a gown in a hospital ward. It also made her realise that she didn’t want to be confined to her house every day in too much pain to do anything.
As a true believer in the power of virtual reality for patient care, Kay believes that healthcare providers need to make this an option for their patients. No one should be sitting at home like she was, in too much pain to live their life. Virtual reality can change that. “Make a difference now,” she rallies, “let's use the technology for your patients’ today, not their tomorrows, because some of us do not have tomorrow.”
The British Pain Society (2014) defines acute pain as ‘an emotion experienced in the brain’ that lasts ‘less than twelve weeks duration’. Compared to chronic pain, acute pain is usually experienced as a response to tissue damage that is commensurate to the level of injury. The pain disappears when the injury is healed and pain nerves stop sending a signal to the brain. In the UK and Internationally, some examples of VR use in the sphere of acute pain occur in childbirth, dentistry, colonoscopy and burn dressing management.
The pioneering work of Hoffman and Patterson led to the development of SnowWorld, an evidence based therapeutic VR software programme that first developed the concept of VR analgesia (Hoffman et al. 2000). Hoffmann and Patterson’’s research built a body of evidence around VR acting as a uniquely attention-capturing medium capable of maximising the amount of attention drawn away from the 'real world', allowing patients to tolerate painful procedures
In the UK, there are many healthcare organisations developing VR interventions for acute pain. We focus on the work of Ivan Phelan from Sheffield Hallam University. Collaborating with Sheffield Teaching Hospital and Sheffield Children’s NHS Foundation Trust. Ivan has worked with both burn victims and children requiring physical rehabilitation, which also encompass a considerable pain component.
The results suggest that VR could help the burn-injured to better manage pain during the dressing changes. These results highlight as a key factor in reducing pain and increasing tolerance of wound care seemed to be the degree of distraction created by VR:
“It drags you off. It drags you off, definitely. They are picking off stuff where, say they pick one or two off ... you’d be on it, wouldn't you, you’re concentrating on the pain all the time, where that does help me, it’s distracting, the whole thing” Patient quote
Active scenarios appeared to be more effective in distracting patients: one said“[it was] better with VR; [but] scenarios [were] better for taking mind off”.. In contrast, the relative slowness and passivity of passive scenarios facilitated a limited degree of distraction for most participants.
Patients were unanimous that they had achieved good levels of distraction (and no nausea) in the active VR. Some spoke of awareness of pain and of what the nurses were doing—“felt it but not concentrating on it” —but their focus remained on the engaging scenario. Nurses spoke of patients being “amazed” by what they had done afterwards, and several patients reported losing track of time, so immersed had they been in the virtual world: “It seemed to go much quicker than I thought”.
Wearing the headset meant patients could not see the wound and nursing activities: “I didn’t see what they were doing. If I could see what they were doing, I wouldn’t let them”. Without this distraction, normal behaviour would have involved being drawn to and focusing on wound treatment.“Before you were thinking, it hurts, because watching them do it makes it worse”.
These findings were supported by the nursing staff, who reported that during the VR dressing changes less pharmacological analgesia was given to the burn-injured patients:
“She was not in the need of any extra analgesia during, before or after the dressing changes. Normally she would have asked for some” Nurse quote
At the same time, the nurses stated that during the VR exposure they were able to spend longer on dressing changes and remove more surgical staples, which results in minimizing the overall duration of the healing process. This was contrary to the normal dressing changes processes were the burn-injured patient requested to terminate the dressing session more quickly or increase the pharmacological analgesia intake:
“He was a lot better with the VR on and I did pick quite a lot. Normally he does not allow the staff to do what we want to do because of the pain, whereas with the VR he allowed me to do that” Nurse quote
Often, minor surgeries requiring local anesthetic can be anxiety inducing, and can lead to delayed procedures, or even rescheduling surgery. Possible future applications of VR in pain management and distraction include VR for dental procedures, and vaccination anxiety, and other minor surgeries requiring a local anaesthetic. In 2019, Torbay and South Devon NHS Foundation Trust used VR as a distraction tool for patients experiencing anxiety during toenail removal. Podiatrists noted a drop in patient anxiety and pain experienced during the toe-nail removal procedure (Ghatnekar, P 2021).
Whether it is for the management of burns, long term conditions, podiatry or a range of other pain issues, VR presents a compelling, non pharmacological, scalable digital intervention that can reach more patients and improve quality and consistency.
This study was carried out at the Sheffield Teaching Hospital by Ivan Phelan and the research explored how VR could help burn patients tolerate pain while undergoing repetitive and painful treatments, such as wound debridement, dressing changes and other painful medical procedures. Literature has shown that although these processes are fundamental for their recovery, by improving the functional outcomes and minimising persistent disabilities, burn-injured patients often find it difficult to participate fully in their therapies due to the significant procedural pain (Richardson P, Mustard L. 2009). For that reason burn-injured patients are usually dealing with a greater sensitivity to infection, acute stress symptoms (Stoddard et al. 2006) and post-traumatic stress disorder, concerns about the impact on appearance (Berger et al. 2010), suicide post-discharge (Macleod et al. 2016) and loss of confidence in the care team.
Given the growing evidence for the effectiveness of VR interventions on burn pain, the limited side effects and the possibilities such technology offers (e.g., immersing the patient into a “cold” virtual environment to distract him/her from perceiving nociceptive signals and pain or altering the colours of the VR environment to promote cooling sensations), reviewers have recommended the deployment of VR in clinical settings (Jamison. 2017). However, only a few studies have examined how to effectively deploy the system in a hospital setting and how to design software to meet the subjective needs of this particular patient group (Bucolo et al. 2006).
Ivan and his team developed their own bespoke VR environment, focusing on the creation of a high quality visual environment, enabling six degrees of freedom so the user can travel around the environment. C a sense of embodiment, virtual hands were developed to present the user’s hands and synchronise their movement in the virtual space, reflecting the movement of the users in the physical space. The rationale for a more detailed and developed approach was that such an environment would increase the user’s presence, and would decrease the pain signals the brain was likely to receive or process.
The study was carried out at Sheffield's Northern General Hospital (Burn Unit), with the participation of in-patients who were undergoing regular dressing changes during the study period . Patients took part in three observed dressing changes during the study - without VR, an active VR scenario, and a passive VR scenario. The order of dressing changes was altered between participants. A researcher spent time with the participant before, during, and after the dressing. They prepared the equipment, provided instruction, and facilitated short familiarisation sessions for the participants before they used each scenario.
VR presents an opportunity to encourage movement recovery during the physiotherapeutic process (Phelan et al, 2019). It allows users to be immersed in an engaging and challenging virtual scenario where the interaction emulates the exercises required in their conventional therapy. This results in increased patient enjoyment, reduced pain during the rehabilitation process and better results in movement recovery (Gerber et al, 2016).
Physical therapy (in this context encompassing occupational and physiotherapy) is recommended by NICE and includes strength training and the use of task-focused active-use therapy such as constraint-induced movement therapy. It is recommended that this intervention is provided as an intensive program over a short period (e.g. four to eight weeks). This pathway includes routine measurement of the range of motion and monitoring of occupational performance and is carried out by Psychotherapist and Occupational Therapists.
Patients suffering from Upper Limb Motor Impairment (ULMI often undergo repetitive therapeutic physiotherapy sessions to minimise functional disabilities of the affected area (Ho et al, 2018). A higher intensity and frequency of upper-limb rehabilitation is associated with improved quality of life, motor function, the ability to perform daily activities and is cost-effective. Reduced upper-limb function has been identified as a strong predictor of lowered psychological wellbeing post-stroke. Therefore, innovation and investigation of effective treatments for arm recovery have been identified as a priority for stroke research (Hayward et al, 2019).
Over recent decades, there has been a growing interest in designing non-pharmacological interventions which aim to minimise procedural pain during physical therapies and improve functional outcomes (Pallai, R et al, 2015).
Kiper et al.'s randomised control trial with adult post-stroke patients (2018), demonstrated the effectiveness of reinforced feedback in VR treatment, used for two hours daily, five days per week for four weeks, combined with conventional rehabilitation for upper limb rehabilitation, compared with conventional therapy alone.
Immersive VR rehabilitation scenarios requires patients to wear a VR Head Mounted Display thatfully immerses them into an interactive virtual environment; with arm movements tracked using the VR systems tracking sensors. This enables the participant to feel a real sense of presence and place. Tracking the users' limbs in the virtual environment increases the level of immersion by evoking the feeling that the virtual limbs are mapped to their movements in real time.
There is a diverse population in need of upper limbs rehabilitation, including people affected by neurological conditions, musculoskeletal conditions and orthopaedic trauma, including children with cerebral palsy in the paediatric population and stroke patients in the adult population.
There are 30,000 cerebral palsy patients and 24,000 fractures patients in the paediatric population who could benefit from VR hand and arm rehabilitation systems. Other groups include trauma, brain injury and musculoskeletal patients (Chen et al. 2014).
In a clinical setting and in several UK and international trials, VR-supported rehabilitation has been shown to be effective economically due to the reduced costs of the technology and also in reducing procedural pain, with very limited adverse effects.(Garrett, B. et al, 2014) These studies have also shown that VR has a high intrinsic motivational power that increases the level of engagement and enjoyment of patients in therapeutic routines (Sharon, D. et al, 2012; Sun, H. 2012)
Research by Sheffield Hallam University’s Impact VR Lab explored the use of VR as a tool to provide therapeutic physiotherapy for child patients in an outpatient hospital department. The focus was particularly on those suffering from Upper Limb Motor Impairment (ULMI). Two bespoke VR games were developed by a games designer and researcher, in consultation with physiotherapists, to integrate the therapeutic movements required for ULI rehabilitation. In one of the scenarios, the child had to climb up by performing an overhead arm raise exercise. The other involved using a bow and arrow to target balloons and gnomes. The child had to reach out with their non-injured arm to grab a bow floating in front of them, then lift-up the injured arm and bend the elbow behind the back to grab an arrow from a quiver.
In comparison with their usual rehabilitation experiences, children reported finding rehabilitation exercises more enjoyable and found movement easier and less painful using HMD-VR. Pain and difficulty were rated overall as lower than usual, and enjoyment was rated as much higher than usual. It was found that VR could: (1) improve functional disabilities; (2) alleviate perceived pain; (3) reduce the perceived difficulty of rehabilitation exercises; (4) increase exercise duration; and (5) produce positive emotions toward the physical therapies.
I think they [children with upper limb motor impairments] found the VR to be a very useful therapy tool […] they really enjoyed it. One child in particular got very excited about it and was very keen. She’s 10 years old. She has got a congenital condition and she’s got a frame on her arm. She’s quite compliant with her exercises. She does carry them out. But we found that during the VR, she perhaps hadn’t been doing her exercises in the past as much because she got quite tired from having to put her arm up like this. […] But with the VR she was doing it and overall, all child feedback was positive. [Clinical staff 1]
Early intervention in general, and early treatment by physiotherapists in particular, for common musculoskeletal problems can reduce the amount of time people are off sick and can prevent acute problems becoming chronic(Neilson, A.R et al, (2019) The pain and discomfort of current therapeutic exercises can reduce compliance and, therefore, the range of motion patients achieve. This increases the risk of chronic pain and can reduce patients' confidence in the care team. Tackling rehabilitation pain and treatment adherence is important to succeed in long-term outcomes for this population ( Threapleton, et al, 2016).
Repeated sessions could potentially enhance the prospect of good and/or complete recovery with the optimal restoration of function, and by using the VR in clinic and at home, patients could achieve this. Benefits include speedier recovery and earlier return to work/school. Clinicians would benefit from patients needing fewer clinic appointments through patients’ improved adherence to their rehabilitation protocols, with the attendant savings to the NHS.
The first publications in adult rehabilitation focused on people post-stroke (Piron. et al, 2001) and in pediatrics, primarily focused on children with cerebral palsy (Sandlund et al, 2009). The works of Deutsch et al.(2008); Laver et al.(2017); Lohse et al (2014) and Saposnik et al.(2011) demonstrated VR’s promising results using upper limb rehabilitation in adults. The works of Jannink et al. (2008); Sharan et al. (2012) and Sun(2012) reported the positive effect of using non-immersive VR game training on physical motor rehabilitation in children. However, a review looking at VR for stroke rehabilitation in adults suggests that immersive VR using head-mounted displays (HMD) is more effective for upper limb motor recovery compared to non-immersive VR (Henderson. et al, 2007). This is supported by the fact that the level of immersion (presence) in the environment contributes to enhance learning efficacy and support the following transfer of knowledge and skills. That plays an essential role in providing an optimal condition for task practice, as does the meaningfulness of the task to the participant.
Immersive Rehab was founded by Isabelle Van Der Keere, an electro-mechanical engineering and a former biomedical engineering scientist, following her own experience of a long physical rehab period she went through following a work accident in 2010. Immersive Rehab is addressing an urgent need in healthcare and particularly in neurorehabilitation services for people affected by neurological conditions like stroke, spinal injury and multiple sclerosis. Their vision is to empower these patients and give them more independence by providing access to engaging and personalised digital therapeutics neurorehabilitation solutions with the aim to improve patient outcomes and increase access to necessary services, both in the clinic and at home.
They work very closely with clinicians, therapists and patients in a co-development of their tool, and have run phase 1 and phase 2 clinical validation studies with neurorehabilitation specialists and their patients in various hospitals and rehab clinics around the UK, all specialised in either spinal injury, stroke and multiple sclerosis rehabilitation. They are currently preparing phase 3 clinical trials (RCTs) with leading clinicians and therapists in four neurorehabilitation centres in Germany, the US and Canada, and more conversations are ongoing with other clinical and academic institutions to take required patient trials forward and pass regulatory approvals.
Website: Immersive Rehab
Currently, research has mainly focused on the application of VR as a motivational enhancer that increases the level of participation and reduces the pain experienced during exercise performance due to its immersive power. In this sense many commercial games, such as exergames, have been used to enhance children's enjoyment and engagement. Increasingly, research is being conducted that could further help physiotherapist and occupational therapies to better deliver their interventions by developing immersive and interactive VR scenarios.
Possible future applications of VR in rehabilitation include its use for home therapy. Home rehabilitation allows patients to reduce therapy time, but many patients report difficulty in following rehabilitation at home due to a lack of motivation to exercise on their own (Jolly et al, 2007). VR systems available for use at home could help solve this problem, improving patient participation and enjoyment. This in addition to having other benefits, such as cost savings, reducing patient transportation to appointments, reducing home visits or providing patients with self-care/self-management, which could be beneficial for the long-term management of their conditions (Threapleton, et al, 2016). In addition, home rehabilitation in emergency COVID-19 situations presents itself as more relevant than ever to help reduce care pressure (Salawu, et al, 2020).
Rehabilitation classes are oversubscribed.or each person able to attend the class, there are another ten on the waiting list. Being able to standardise the delivery of pulmonary rehab therapies has enabled access to exercise classes and enabled clinicians to remotely measure performance. Dr Farhan Amin, founder of Concept Technologies and Lead GP for Integrated Care Communities at Morecambe Bay CCG, developed a virtual reality pulmonary rehabilitation tool for patients with COPD. Patients were invited to take part in breathing exercises from the comfort of their own homes, avoiding visits to hospital or community facilities and providing them with their own independence under the guidance of the clinician. The use of virtual reality to enable exercise programmes for patients has been introduced and has already been well-received among the patients who have used it in the North West of England, and the research suggests the programme provides a credible alternative to traditional pulmonary rehabilitation measures. .
In a thematic analysis of the qualitative data, 11 themes emerged specific to delivering pulmonary rehabilitation using VR. The quantitative data revealed significant improvements in all physical measures and found that the programme improved patient retention by 80% by removing distractions. The study demonstrated that remotely supervised VR-based pulmonary rehabilitation could help to overcome current issues and limitations associated with providing this service to patients with COPD at scale. This programme was the only pulmonary rehab programme that continued to take place during the COVID pandemic.
“Barrow-in-Furness has many people with chronic lung conditions and the best treatment is to take part in a pulmonary rehabilitation programme. Usually this involves going to the hospital outpatient department, but using the latest technology we just need a mobile phone and a virtual reality headset; and the person is transported to a ‘beach’ where they can interact in an exercise class, get educational help, receive breathing strategies and get nutritional advice.” Dr Farhan Amin.
The intervention also offers a possible glimpse into the future of digital intervention prescription. In a similar way that a GP or health professional might prescribe medication to a patient, Dr Amin has ensured that the PR inVR intervention is available to ‘prescribe’ online from the patient record. In this way, Dr Amin sees the digital pharmacological interventions on a level playing field and at the tips of the fingers of the health professional offering them. The aim of the approach is that soon other therapeutic modalities might also be available at the click of a button as a part of a digital first approach.
The NHS is the biggest employer in Europe and the world's largest employer of highly skilled professionals. 1.3 million people work in the health service in England and according to Health Education England (HEE), 160,000 students are at this moment studying to be part of the health and care workforce.
Education and training are regarded as an essential part of the NHS to ensure that the health and care workforce are responsive to changes in patient needs and in healthcare delivery. The UK continues to be regarded as having some of the best healthcare education in the world and is seen as a major player in healthcare research.
The global pandemic has resulted in shortages in the supply of the clinical workforce. The disruption to training caused by the pandemic has meant that clinical placements have been halted, physical simulation sessions have been reduced and there has been a lack of diversity of clinical cases as trainees are not exposed to a wide range of different types of patients. To minimise the impact on trainees, alternative delivery methods needed to be implemented.
Simulation-Based Education (SBE) is a well-established technique for delivery of health and care education in the UK and is popular with students and teachers alike. XR technologies are becoming an essential method to support the delivery of healthcare training, enabling accessibility and replicability of a range of environments and scenarios. These rapid advancements open up unique avenues for healthcare interventions, partly accelerated by the pandemic, to support the continuing education and development of the UK health and care workforce. Access to VR environments is increasingly affordable, generating significant interest for its wider application in education, given it can be delivered remotely, at scale and with less risk.
Education and training also place a significant financial burden on the healthcare system, thus more cost-effective solutions are constantly being sought. It is for all these reasons the interest in XR technologies has and continues to rise exponentially (Bowyer et al., 2008; Haluck, 2000; Logeswaran et al., 2020; Mantovani et al., 2003).
XR training is transforming the way health and care students learn today. Research shows surgical training simulators using virtual and haptic feedback functions are a viable alternative to real-world surgical training. In fact, a recent study from Harvard Business Review showed that VR-trained surgeons had a 230% boost in their overall performance compared to their traditionally-trained counterparts. The former were also faster and more accurate in performing surgical procedures. XR technologies also enable better experiential and collaborative learning methods, promoted by constructivist learning models that have been shown to be more effective than traditional or other digital education methods (Kyaw et al., 2019; Logeswaran et al., 2020).
There is also evidence to support better contextual learning with XR technologies that aid the learning process and improve student motivation (Logeswaran et al., 2020; Mantovani et al., 2003; V. Pantelidis, 2009). Students are using AR and VR to simulate realistic yet risk free clinical situations from highly complex surgeries to urgent and emergency care, and exploring virtual cadavers to learn anatomy and physiology. Trainee nurses can develop hand eye coordination using precise medical instrumentation in AR and MRIs. CT scans and angiograms can be combined to create three-dimensional models for radiographers. Surgeons can practice by moving through each step of a surgical procedure, close up and in real time, in collaboration with others remotely, rather than just preparing for a surgery by looking over static images.
Immersive technologies provide a unique opportunity for replication of dangerous, difficult and inconvenient locations. This has the potential to improve accessibility, reduce inequalities in training and improve patient safety.
A further opportunity for use of XR is in remote surgery. Video demonstrations of surgical techniques and broadcasts of operations via fast 5G networks are already viable. In its 5G in Healthcare Report, PwC cites that “the real breakthrough will come with the emergence of the tactile Internet”. Enabled by the ultra-low latency of 5G, the tactile Internet would enable a physician to perform a procedure on a patient who is in a different location. The surgeon’s movements at one site would be recreated in real-time by robotic surgical equipment at the other site, an innovation that could particularly benefit patients in rural areas or smaller regions where surgeons specialising in complex procedures may not be readily available.
“Up to 90% of ward nurses had never been into an operating theatre. Live streaming of surgeries increased levels of respect among staff, as they developed a deeper understanding of the process and everyone’s roles during surgical procedures. This experience enabled access to places most staff wouldn’t usually have access to.” Steve Dann, Medical Realities
PWC also notes the growing body of evidence to the value and effectiveness of virtual training. stating that “virtual training is the most cost-effective way of learning when it’s done on a large scale. At 3,000 learners, VR costs become 52% less than classroom”.
There is huge potential for XR training in health and care. The exploration of XR technologies in healthcare education and training is being driven by a number of factors. The time afforded to training within clinical settings is stagnant or decreasing, partly due to the acknowledgement that excessive working hours increase the risk of making clinical errors and compromises patient safety. This has had the untoward effect of reducing clinical exposure which has been the backbone of training to date. There is also a national mismatch between the supply of healthcare professionals and the demand for their services, which produces greater pressure on limited training time. In addition, the2018 Kings Fund report The healthcare workforce in England: make or break? highlights that 30-50% of the frontline workforce is due to retire by 2030.
Education and training also places a significant financial burden on the healthcare system and more cost-effective, remote and accessible solutions are constantly being sought.However, current approval and governance processes are not dynamic enough to allow for the acceleration of deployment of XR training and education solutions to meet the latent demand. This is slowing down the ability to develop the broad range of XR solutions required to meet future health and care training requirements.
XR enhances and extends existing pedagogical approaches to traditional health and care training. Many health careers are vocational and as such XR offers ways to practice and train before a clinician sees a patient. While the tools should support and supplement existing learning curriculums using active and collaborative simulation techniques, there are clear advantages that are afforded by their use.
Many XR technologies remove the temporal and geographical barriers to training. This enables healthcare professionals to learn in their own time, at their own pace and unconstrained by physical locations such as the hospital. In addition, they facilitate the dissemination of knowledge and skills from centres of excellence ensuring democratisation and accessibility in training across the NHS. These technologies also provide a unique opportunity for replication of dangerous, difficult and inconvenient locations. This has the potential to improve accessibility, reduce inequalities in training and improve patient safety.
This report recognises the significant potential of this training method, particularly during COVID-19, where alternative means of education and training have been sought. It is apparent from the use cases discussed and the developing evidence base, that XR technologies will have a huge impact on healthcare education and training over the next decade, particularly in the delivery of clinical skills training outside of hospital environments, more cost effectively with more impact and less risk. XR provides an opportunity to support the training recovery of the health and care workforce, particularly as there has been so much disruption to training due to the pandemic.
The future design, implementation and evaluation of XR technologies will require a collaborative approach with stakeholders from a range of industries. The approach will require a combination of contemporary evidence-based practice, real world use cases and dissemination of best practice. Five main elements should be factored into the design and implementation of XR technologies in healthcare education and training: learning needs, technological capabilities, wider needs assessment, simulation and evaluation.
There is high demand from governments and healthcare organisations worldwide for new ways to deliver high quality education and training in healthcare for growing and ageing populations (Healthcare UK Annual Review 2019-20), providing significant market opportunities. Due to the UK’s strong global reputation in healthcare education and training, the country is well positioned to offer XR multidisciplinary clinical education and training, healthcare leadership and management training, and bespoke education and training to an international market.
XR has supported the rapid scaling of remote access teaching ward rounds at Imperial NHS Trust, who used mixed reality devices to broadcast the teaching sessions with patients to remote attendees. The study found that trainees enjoyed the experience and provided teaching that was otherwise inaccessible, enabling students to interact with and have their questions answered by the clinician leading the ward round. The use of XR in this context has enabled consistent access to high‐quality teaching that can now be integrated across the curriculum.
Since 2016, Alder Hey Children’s Hospital has been developing its innovation centre, creating a global centre of excellence for advancing child health using state of the art innovative technology. The Alder Hey Innovation Centre aims to harness technology that is not yet widely used in the healthcare context, bringing leading edge digital -sensors, big data, artificial intelligence and immersive health technology - into real world use.
Rafael Guerrero, Director of the Heart Unit and Chief of Congenital Cardiac Surgery & Clinical Director Innovation, has been using Microsoft Hololens headsets for preoperative planning, patient consultations, expert opinion and surgery, importing scans into virtual reality and augmented reality headsets. By enabling access to the images and data in a virtual environment, clinicians can interact with each other and the patient’s organ in 3D. They can scale the image size significantly, enabling them to virtually ‘step inside’ , observe and plan in preparation for the surgery. The team has also been able to use the mixed reality tools and scans for subsequent training and education of staff, and has plans to continue to integrate VR and AR tools for surgical planning, training and in surgical interventions to collaborate remotely with other surgeons around the globe.
In a recent case, when a small baby who had tested positive for COVID-19 required complex heart surgery, the Alder Hey team drew on the expertise of an extended remote team by using Hololens 2 in the operating room. Rafael R. Guerrero, was the Consultant Congenital Cardiac Surgeon leading the procedure. Using the technology, he was able to connect with colleagues to agree and change the clinical strategy for the operation, something that would otherwise not have been possible because the operating room was a restricted COVID-19 area.
The operation was a success. Guerrero says, “Mixed reality technology is opening new doors to what we can do in a hospital setting while keeping our patients and staff safe and above all improving patient outcomes”.
The vast potential of XR technologies is apparent, but an acknowledgement of the barriers to adoption and the difficulties faced in their application is critical to ensure we maximise efficacy while ensuring we do no harm. The use of these technologies as a training tool directly affects the knowledge and skills of the health and care professionals who use them, which ultimately affects patient safety. It is imperative that with the adoption of these technologies, there must be investment in rigorous research and testing into design, implementation and evaluation. For example, when filming 360 video it is important to consider that the learning material may trigger lived experiences of participants, so appropriate support mechanisms may need to be put in place.
The design of immersive experiences should ideally follow a learning-driven approach rather than a technology-driven one and avoid the negative outcomes on learners if not designed properly, such as simulator sickness, postural stress and eye fatigue. A collaborative infrastructure must be in place to ensure that issues which can occur in practice, such as equipment malfunctions, have the right technical support on site to provide assistance. .
In addition, there are unique challenges in health and care around ensuring the ethical capture and inclusion of patient stories, ensuring consent is sought from participants and that consideration is given to information governance and security of learning data.
Issues of accessibility must be acknowledged and alternatives sought where needed. This includes the impact of disabilities, underlying health conditions that are contraindicated with XR use, and the potential pricing out of socioeconomically disadvantaged communities.
Clearer procedures and processes in approval and governance are critical to supporting a thriving XR in the healthcare education and training sector. This will enable faster deployment of XR training and education solutions in the marketplace and support their developers to explore international commercial opportunities.
In 2019 the University of Leeds collaborated with Medical Realities to develop a virtual reality experience that was accessible via a Google Cardboard or Samsung Gear to enable medical students to virtually access the pathology laboratories. In 2020, during the COVID pandemic, this experience was used as part of the virtual classroom teaching programme, enabling students to continue learning from home. Visiting a working laboratory has a range of challenges for training medical students, not least the space required to accommodate a class of 270 people. By recording with 360 cameras, students were able to engage in the experience without the risk of infection control incidents impacting patient safety. The full class of 270 students was able to engage in the experience, with the group split so that half would experience the lecture, and the other would use the VR experience, and then swap.
The physical tour of the pathology lab took an hour and was limited to four students at one time. A guide would have to take the students around the laboratory, meaning it would take 67 hours each year to give all 270 students the experience, including the time taken by the guide. By recording the laboratories in VR, the experience now takes 36 minutes, with students able to complete the learning flexibly, without the need for a physical guide. Delivering the training in VR also removes the risks posed by physical tours of sample cross-contamination, patient confidentiality breaches and infection control.
By offering the experience in VR, all students have an equitable experience regardless of where their clinical placement takes place. Leeds also discovered that VR increased student awareness of the pathology specialty as a future career, with students indicating in feedback that they were significantly more likely to choose it as their specialty.
VR offers learning and educational potential for both patients and healthcare staff. The application of VR for patient-focused education has an ever-increasing remit and onward potential, from providing visitors with pre-visualisation of pathways of care to the ability of sharing insights and re-creation of unique perspectives and illnesses. VR patient education brings an experiential awareness of humanistic skills, such as effective interaction and communication between health and care staff to patients and service users. These contexts bridge the importance of human connectivity, a theme that cannot be underestimated when considering emerging technologies.
An experience which gives insights to a pathway of care or pre-surgical visualisation will have learning benefits for both audiences - the patient and service provider/clinician. For the patient, it might mean using a VR intervention that provides anxiety reduction by helping visualise an understanding of a process before having to undergo it physically (Gold et al. 2021). For the clinician, this will not only aid the patient journey and therefore support service, but can also offer an opportunity for reviewing environments, interactions and processes from a different perspective, insights that can bring a greater appreciation and trigger potential positive change.
In this particular arena, there is value found in both the more complex CGI (computer-generated imagery) built experiences as well as ‘entry level’ and accessible applications of virtual reality content. In the latter, the utilisation of interactive 360-degree videos have been used effectively to help deconstruct unfamiliar environments with rich information and media (in the style of a virtual tour of a hospital operating theatre for example). Low-cost 360-degree video equipment has been used and adopted by a number of hospitals, including education and simulation teams, to build bespoke, in-house, training content in line with individual key learning outcomes. The relatively low-cost implications of 360-degree video production processes have meant several hospitals have designed bespoke in-house training content as a way to explore this emerging technology as a more effective medium to deliver training and education opportunities.
One of the key and most important benefits of VR for healthcare professionals, is being able to rehearse, test, role-play, and respond to materials that can stimulate emotions and a feeling of “being there”. What this means is that healthcare professionals can experience a range of scenarios and test out ways to respond, but without real-life consequences. This in turn leads to helpful and appropriate learning opportunities which can work through scenarios that can be increased in intensity. Using VR over traditional digital media enables an increased sense of presence and immersion (i.e. feeling like you are “there”), this in turn can lead to a more realistic experience and therefore authentic response and learning.
Understanding patient perspectives and narratives are critical in medical education and training. Experiencing the lived conditions, feelings and emotions of a person can help healthcare workers empathise with patients. Empathy is the ability to connect and appreciate with the patients’ experience and communicate using this understanding. Within the context of healthcare, empathy is a skill that can help affect the quality of the relationship between the patient and healthcare professional. This in turn can translate into a better conversation, supporting an essential “what’s important to you?” approach to care. As an intervention in this context, a change of perspective - or “seeing through the patients’ eyes'' - provides a valuable moment of reflection and consideration for the health and social care workforce. Such an opportunity to “experience” a different perspective can help empower effective and meaningful care by enhancing the relationship and engagement between parties. A shared appreciation of a condition or lived narrative can help engagement with a treatment or care plan, having positive implications on patient outcomes.
Empathy training and, in the wider context, a continuation of humanities focused teaching, is a vital ingredient in healthcare education and training. However, due to burn out, lack of consistent training, and system pressures, combined with the challenges of teaching humanities focused curriculum as a consistent key skill, leads to a reduction in protected time where practitioners can reflect on their experience and empathise with patients.
The introduction in 2015 of patient perspective 360-degree video scenario content, created at Torbay and South Devon NHS Foundation Trust, has provided unique insights and reflective learning for trainees to access as part of their educational simulation training. This has complemented the teaching as an additional interactive experience, delivered in both VR and as a desktop experience.
Embodied Labs creates healthier aging through shared experiences across the continuum of immersive engagement by training and educating every person in the aging spectrum with the most compelling immersive experiences accessible anywhere. They are currently leading the aging care space in the development of cutting-edge immersive training for professional and family caregivers. Their immersive experiences, available now on their platform, improve outcomes for all aging individuals, positively impact health of trainees and others through experiential engagement, and help customers solve meaningful challenges. Their vision is a world where older adults - regardless of their race, ethnicity, sexual orientation, gender identity, socio-economic status, and cognitive or physical ability - are able to live the best lives possible as they age.
Website: Embodied Labs
The role of virtual, mixed and immersive reality has long held promise for autistic groups to support communication needs, social skills development, educational outcomes, practicing skills (like crossing a road safely) and vocational outcomes such as job coaching (Smith et al. 2014). VR can help to simulate real-world situations and contexts in safe and predictable ways (Bradley and Newbutt, 2018), and has the ability to control, shape, and tailor VR interventions to participant needs and local contexts. The role of immersive technologies has therefore been considered with autistic groups and research demonstrates that VR head-mounted displays (HMDs) are viewed as comfortable, acceptable and usable by this group (Newbutt et al. 2020). It has been used across a range of environments and scenarios including social development, transitions to employment, travel training and general educational materials.
Further research by Newbutt and colleagues in 2020 discovered that through the co-creation of educational experiences in classrooms for autistic children, VR helped them to relax, remain calm and opened opportunities for previewing a real-world experience in VR. Additional research by Brossonboek et al (2019) discovered that Deep, a breath controlled VR experience, helped reduce disruptive classroom behaviour. This finding, alongside data suggesting that high-end head mounted devices were preferred by this population, is starting to shed light on the potential that VR could have for neurodiverse young people; mainly reducing anxiety, promoting calm and visiting places virtually before a physical visit.
Gold et al. (2021), have explored the role of VR to support autistic patients for preoperative anxiety in addition to improved patient, caregiver, and healthcare satisfaction of a preoperative experience. Results here suggest that patients reported decreased anxiety along with feeling less scared of visiting a hospital for preoperative care. These results provide a compelling argument for the continued examination of VR applied to autistic people; be they young children at school or people accessing healthcare or cultural experiences.
Talking Sense is an augmented reality (AR) conversation training tool that uses artificial intelligence technology to enable dynamic machine learning conversation processes with the aim of supporting parents to better understand the behaviour of their children with autism and better enable coping strategies and appropriate interventions.
Using a dynamic machine learning process, the app provides parents with the opportunity to experience real-time dialogue with AR character, Ben, who is representing their child, and practice challenging behaviour scenarios and strategies. The aim of this app is to help parents identify the stress points of parenting, and support them in understanding what good and poor preparation look like.
Talking Sense has been developed by Neon and is a collaboration between Ulster University and To Play For funded by Future Screens NI.
VR experiences can encourage communication between patients and health workers and provide answers for questions that may be stressful for the patient. For example, human anatomy or health conditions can be presented as visuals in immersive environments. Patients may be able to better understand their condition, how the clinical procedures will be performed and visualise the role played by the procedure in solving their problems. The use of VR as a visual aid for improving patients’ understanding of procedures is now growing. House et al. (2020) have used XR to educate patients on pre-operative education before epilepsy surgery (stereotactic electrode implantation). Migoya-Borja et al. (2020) have used VR to help educate patients on pre-operative education and depressive symptoms, respectively.
Anxiety can have significant implications in the lead up to and during clinical procedures. This can extend the time taken to complete the procedure, may require attention from multiple healthcare staff/general anaesthetics or the procedure may even need to be cancelled to be repeated at a later date. This can have a domino effect on other patients’ schedules and incur a cost to the healthcare system. In this context, work utilising XR technologies (i.e. Koo et al. 2020) includes an interactive 360-degree video that has been explored and deployed in various healthcare pathways to reduce anxiety and better educate the patients on what to expect. One accessible example of this is a journey app that incorporates 360-degree video and instructional design (explainer graphics) to assist children and their parents ease the fear of attending an MRI scan (Ashmore, et al, 2019). In addition Yang et al. (2019) have used VR for pre-operative anxiety management in patients undergoing arthroscopy to positive effect.
Empathy and compassion focused education should be at the core of non-technical skills education for healthcare workers and students. VR can deliver powerful experiences that are filmed or created from a first person perspective, offering users an opportunity to stand in someone else’s shoes. This has clear applications to support healthcare workers in developing empathy with patients or colleagues. Participants should then be given an opportunity to reflect upon and discuss the experiences, and come up with solutions for improving the experience. At Torbay and South Devon NHS Foundation Trust, empathy education is a critical component of soft-skills education for health workers. As part of simulation education, medical students from year three onwards participate in humanities workshops. The students are shown first-person perspective films from the patient's point of view, using VR HMDs. Examples include the patient’s journey through to ICU and the different communication styles experienced by the patient along the way, from paramedics, nurses and doctors. A more recent experience allows participants to see a simulated patient perspective during COVID-19, observing the effect PPE has on communication and the patient journey.
The ecological validity (i.e. feeling natural in a space), sense of presence and immersion that VR HMDs offer mean that specific clinical education and soft skills health care professionals need to develop, can be significantly enhanced with XR patients education approaches. XR enables patients to preview and experience the environment of a hospital (and/or aftercare) in a way that can reduce anxiety and provide a way to slowly become used to new experiences in familiar settings. VR has a unique and powerful role to play in this space.
The opportunity for local healthcare settings to create their own bespoke content offers a powerful patient participatory approach to creating localised and meaningful educational tools. Patients educated in the care they are about to receive, or staff having a greater opportunity to understand the needs of patients from their perspectives will enhance and improve care. The opportunity for organisations to develop these skills and processes are being supported by the Health Education England and the Holomedicine Association, an indicator that with the right skills, pathways and shared learning, patients education offers an opportunity for scaling across healthcare.
The VR Lab at Torbay and South Devon NHS Foundation Trust (TSDFT) is a purpose-built VR lab within the NHS to investigate the potential of immersive VR technology as a Technology Enhanced Learning (TEL) and Simulation Based Education (SBE) tool, looking at its role in healthcare education.
The VR Lab is now providing both a physical space and a location-flexible programme specifically dedicated for the advancement of application and research into immersive content and technology. The Immersive technologies team have access to 360 video recording equipment for creating custom content, from which they produce interactive 360 video to aid education, training and pathways (such as virtual tours or clinical orientation). They are also engaged with producing promotional content for health and social care careers, such as a series of immersive films for the HEE AHP programme.
The lab itself has continued to grow as a space that promotes co-design and multidisciplinary activities and workshops, engaging staff, educators, partners and our service users.
The rapid advancements in immersive technology and software are opening up unique avenues for healthcare interventions. TExperience and feedback collated at TSDFT has shown potential for application of immersive VR as a TEL/SBE training methodology in multiple areas, which have included:
NHS organisations have typically struggled with the required knowledge and experience in deploying VR for training, given its infancy and when considering digital literacy and accessible implementation. Coupled with this, there is a lack of evidence as to which technical and faculty requirements may be necessary to effectively adopt VR in healthcare education. The VR lab set out to address this issue by safely trialling VR equipment and applications with health and care staff, students and service users. The main aim of the VR Lab is to share learning, case examples and best practice around the use of VR in healthcare education and training, while evaluating value through comparison to other modalities of delivery and feedback. The VR lab hopes to achieve this by developing relationships with clinicians, staff, care homes, patients and academics, and by researching various contexts within which VR can be applied, evaluating the efficacy of using VR, sharing knowledge and helping towards building a community of VR practitioners within the NHS.
Having an in-house”‘embedded” VR lab has also proven to be beneficial for developing creative and innovative pathways toward healthcare education and training.
Website: Immersive Technologies at Devon and South Torbay NHS Foundation Trust
Benefits realisation is a crucial part of making sure that programmes of work in the NHS lead to positive changes for patients, staff, clinicians and the public. It is therefore vital in demonstrating that public funds are being used on evidence-based and benefits-led activity. This is particularly the case in digital transformation work, where technology is often innovative and is also often an enabler of benefits elsewhere in the health and social care system.
To ensure that benefits are realised from digital transformation programmes, NHSX engages widely and seeks input from stakeholders, regional teams, local providers, Integrated Care Systems (ICSs), the third sector and digital innovators, amongst others, to identify use of digital technologies which are evidenced to provide the maximum value and are capable of realising the intended benefits and outcomes for patients and staff.
Immersive technologies may be relatively new within the NHS, especially outside of acute care and simulation training, but their potential is evident. Extended Reality (XR) can offer benefits in many ways in different healthcare settings, in the community and in wider society. It can be used to support patients engaged in self-care or remotely delivered care at home, or to improve the experience in clinic, whether as a direct intervention for pain or anxiety or as a communication tool. Clinicians can use XR to engage patients better and to free up time. Medical education benefits from better quality training and more learning opportunities. Health systems can use XR to tackle increasing demand and to improve services. Society can benefit from its role in equitable access to treatments and in public health prevention programmes.
A major driver for the wider use of XR in healthcare is that familiarity of Virtual Reality (VR) has increased in society due to the lowered price of headsets and controllers and their use in video games. Lowered cost reduces the barrier to NHS Trusts deploying VR in other areas of healthcare, such as in mental health or home rehabilitation as we have seen in this report’s case studies. Although more expensive than VR at present, enterprise-level Augmented Reality (AR) and Mixed Reality (MR) technologies are also available. Industry is finding more uses for XR and is making products and tools available to new product developers, as well as making the business case for its use in health.
Outside of gaming, healthcare is viewed by many as the area where there is most potential for XR and the industry is expected to grow to £8 billion by 2025. Although it is somewhat unclear how widespread the use of XR currently is within clinical practice, there is a drive within healthcare to begin adopting these technologies to supplement and enhance current provision. This is particularly true within mental health, where there is significant evidence for its efficacy across a range of conditions. Social care, and also end-of-life care, are important areas of opportunity where XR technologies are beginning to make a difference:
"There is a growing evidence base to support the acceptability of VR in medicine as it demonstrates potential benefits in a range of areas from pain, stroke, and cancer, to phobias, anxiety, eating disorders and PTSD. Translating the emerging (and rapidly evolving) VR science into clinical reality will require research and development that is both robust and innovative in its design, as well as interdisciplinary and collaborative in its approach, engaging all stakeholders."
Dr Kim Smallman, Research Design and Conduct Service Consultant and Research Associate, Cardiff University
“The use of VR in end of life care has generated a great deal of interest and excitement in the end of life doula community. The potential for VR to help with reducing anxiety and pain is also very promising. As end of life doulas, we would be very interested in using VR, as it would provide further choices for the people we support, facilitating our person-centred approach to death and dying.”
Emma Clare, Director of End of Life Doula UK
The Health Foundation has predicted that the Covid-19 pandemic will bring significant costs to the NHS that will require many years of investment to shore up increases in demand, not only from services impacted by the pandemic but also people. Areas under most pressure include mental health services and hospital care, which will also need to consider the potential for future waves. The NHS can best manage increased demand through investment in its workforce and more efficiencies within the care pathway to prevent avoidable admissions and treatments. New ways of delivering care are needed, including those that reduce the need for patients to be physically present. Telemedicine will predictably increase as a means of reducing this in areas such as GP visits, and XR can offer a more immersive experience where this is necessary.
The economic approach to any technology in health involves identifying the clinical pathway and settings where it is to be deployed and the change expected to be achieved, followed by a measure or model of the costs versus benefits. The value proposition is expressed either as a straight financial saving or as costs against benefits expressed in monetary terms, health-related quality of life gains, or other measurable consequences. While there are few published formal cost-effectiveness studies on XR, the various dimensions to benefit can be readily seen and a good case for using immersive technology can be - and is being - made in healthcare services.
The following sections explore the costs and benefits associated with XR in healthcare. The key driver to innovation within this space is the increasing affordability of the equipment and the initiative of immersive technology companies seeking to establish use cases within the sector. There are several examples of these use cases within healthcare and these are discussed within both a national and international context and across three main areas:
In recent years there has been a change in the familiarity and prevalence of VR technology in the UK population. This change includes much wider consumer purchase of Head-Mounted Displays (HMDs) and haptic controllers for video games, and use of AR through smartphone games such as Pokémon Go. HMDs as personal technologies have improved in comfort and many now have embedded computing power so that a separate PC is no longer required. There are also variants of HMDs that use a smartphone as the screen, although the recent introduction of cheaper untethered all-in-one HMDs are already replacing the former in many applications.
Compared to 25 years ago when more comfortable VR equipment was first becoming available, although still mainly used in research, a liquid crystal HMD with electromagnetic tracking system for the head and hands and a suitable PC would have cost more than £10,000 in total, which at today’s prices with an annual uplift of 3% would be £20,000. In comparison, a PC based VR system in the 2020s is 10 times less. The newer consumer level all-in-one untethered HMDs with two controllers are even cheaper with Oculus Quest, for example, retailing to consumers at around £300, while smartphone-ready headsets are cheaper still at £30-£50. If we consider a lifetime of two years and 262 working days in a year, plus the addition of a small amount per use for a disposable liner or cleaning materials, the cost per day of a PC based system is now only around £5 per day over its lifetime based on hardware alone. Similarly, replacement costs of devices or batteries for portable equipment are much lower in the 2020s.
Per-person cost is further reduced if devices are used more than once a day with different patients. Higher utilisation is more practical than ever due to improved weight and portability of equipment that can be carried by professionals into different healthcare settings, including the home, hospices, care homes and training venues. It is clear that the price of purchasing or leasing the equipment to deploy VR is now a relatively low proportion of overall cost in comparison to staff costs. This all means that capital outlay and consumables are likely to be judged affordable to start using VR more readily in more settings.
At the time of writing, typical offerings on the market for HMD based-VR health are in the range of £400-£4000. At the low end, this consists of HMD, software and internet capability with minimal support, and at the high end includes a fully supported and maintained system in a protective case with more choice of virtual environments, backend outcomes data acquisition and storage, and enhancements such as an accompanying tablet for third party viewing, and 360 camera.
One of the biggest drivers within the NHS is to provide value for money – as a free health service the funding provided through tax is reinvested to best benefit the healthcare needs of the UK population. Identifying the most cost-effective return on investment can be complex but here we present four case studies that demonstrate savings across various services and in several ways. Although two of the four presented are not UK-specific, they emphasise the potential that XR offers in supporting healthcare services to deliver highly effective outcomes in a more cost-effective way.
XR enables remote delivery of care at home and can improve the experience in the clinic, providing more regular and reliable access to experiences and therapies.
can use XR to engage patients better and free up time by automating certain elements of the therapeutic process.
benefit from better quality training and more learning opportunities, including access to a range of scenarios and conditions otherwise impossible to access.
can use XR to tackle increasing demand and improve services, and reducing capacity of clinicians.
can benefit from its role in equitable access to treatments and in public health prevention programmes.
can support the creative and technological demand, and create new ways to fund and support XR healthcare tools, whilst gaining access to a new market sector.
ProReal is a virtual platform that uses avatars within an environment to enable users to explore different perspectives, visualise situations and problem solve . Within mental health it can be challenging to support service users to communicate their problems, particularly when faced with complex issues or challenges in communication. The value proposition for the ProReal avatar technology platform is already described as part of a NHS Global Development Exemplar Blueprint. The software, based on 3D graphical environment and embodied actors, is used to supplement talking therapies.
This approach addresses five categories of benefit. One of these is acceleration of insight and change whereby a therapeutic relationship is built up much faster using digital imagery, especially for people who struggle with verbal expression; eight sessions versus a typical 12-18. It also increased engagement and access, and reduced stigma. One organisation found a switch around in the usual high female to male ratio of access to talking therapies.
The software showed beneficial support for preparatory work by clients saving one hour per contact and, when used along with voice coaching of senior staff over phones, provided a 40% saving versus face-to-face, with reduced need for travel. There were low drop-out rates and high satisfaction levels using the technology. ProReal has been beneficial in a variety of different settings, including prisons / secure settings, school counselling, child and adolescent mental health services, and adult services treating eating disorders, personality disorders and other conditions where developing empathy and compassion toward the self and others is an important goal. It is also a shown benefit for people with learning difficulties.
XR has shown promise in reducing pressure on hospitals when delivered at certain points in the care pathway and can help services be delivered more appropriately. Each extra bed day costs the NHS upwards of £300 and reducing avoidable demand for elective care is one of the cornerstones of the Five Year Forward View,. Only 0.5% of surgeries are cancelled on the day because of patient non-attendance (Wong, 2018), yet the potential cost of lost operating theatre time could be as high as £400 million a year (Gillies, 2018). If XR can be used to help patients to face operations and treatments that they would otherwise avoid, this could lead to £2 million of possible savings a year
Minor surgery for ingrowing toenails is a common procedure carried out by podiatrists in the outpatient setting. However, due to anxiety or needle phobia, some patients are unable to proceed without going to the orthopaedic department which may involve the use of a general anaesthetic. The in-patient tariff is a lot more expensive. Minor Skin Procedures for toenails cost £125 for an outpatient procedure or £418 as part of a day case, more expensive for paediatric patients at £170 and £698 respectively (2020/21 Tier 2 tariffs). In Torbay Hospital, where there are three podiatry sessions a week with six patients in each for nail procedures, they are treating at least two extremely nervous patients per session, and around six patients a year are referred to orthopaedics. Offering VR among the existing options such as gentle persuasion or use of ethyl chloride has been well received by patients. After an initial try out of an all-in-one HMD, two purchases of low cost VR headsets were funded by a hospital charity and are now in routine use. The equipment cost, with an estimated lifetime of two years, is covered by avoiding just one day-case referral or one to two abandoned outpatient episodes per year, so the solution is cost saving even if the Trust had paid for the equipment directly.
A published economic model that looked at VR as an alternative to opioids in US hospital settings demonstrated savings of over USD5 per patient, due to a reduction in length of stay, which when scaled to 15,000 admissions per year with one in seven patients using it, would cover the cost of 30 software licenses at USD3500 and USD47,030 to employ a VR specialist (Delshad, 2018). Considering the health benefits of reduction in opioid use, this digital solution to pain management offers far more than just cost savings.
Another published economic evaluation, from Spain, investigated the benefits of delivering VR interventions for balance recovery after stroke remotely to the home, rather than in the clinic. It found that it reduced not only transport costs but also the contact hours of physiotherapists. This resulted in a 56% saving compared to usual care (Lloréns, 2015). This is a good example of the potential for increasing the home use of XR in healthcare.
At the heart of the UK health and social care economy is a commitment to deliver health outcomes that can improve the population’s quality of life. In the following case studies, several examples are explored where XR has shown promise in providing affordable improvements to existing services.
VR is effective at enhancing the frequency of physical activity and, along with AR, at improving strength of physical performance (Ng, 2020). There are many clear benefits to physical activity and if VR can support this effectively, it has the potential to bring significant savings within the NHS.
One area of great interest is within falls prevention. The cost of falls every year to the NHS is £2.3bn and NICE recommends that all older people with recurrent falls, or assessed as being at risk, should be offered an individualised multifactorial intervention. However, multifactorial interventions have been found to be 10 times less cost-effective than exercise-only (Winser, 2020). Off-the-shelf virtual reality exergames, which have been found to be more effective than standard therapy in some cases, could help provide a low cost alternative within multifactorial interventions (Corregidor-Sánchez, 2021).
The NHS People Plan of Aug 2020 includes specific commitments to look after staff, tackle discrimination, make more effective use of the workforce through new ways of working and delivering care, and look to the future through better recruitment and retainment. The Health Foundation has predicted that an 11% increase of activity within the NHS will be needed across the next three years to meet the backlog of elective care which will require significant investment in training, recruitment and retention of staff.
In the wellbeing section, Rescape’s DR:VR Frontline was introduced and its roll out in intensive care units within two hospitals during the pandemic. This deployment used the Frontline version of DR:VR, which provided a set of Pico VR HMDs in protective cases with proprietary software under a £30 a month rental scheme, which included software updates. After using DR:VR, 88% of participants reported reduced feelings of stress, whilst 94% felt it was enjoyable and provided relief from their work. The use of DR:VR within high-stress frontline environments is both acceptable and affordable and could possibly demonstrate significant outcomes in reducing potential burnout.
Rescape Innovation also offer a fully leased version with accompanying tablet PC, mobile internet hub, optional 360 camera for personal recordings, hygiene consumables, and backend services to collect outcomes data. This version of DR.VR is fully mobile and suitable for care homes and hospice visits, for example, requiring no Wi-Fi or additional logins and sits within a “walled garden” making it ideal within a healthcare setting. The cost of deploying this system is estimated to be £7 a day based on 10 times a week usage.
The cost of avoidable errors within the NHS has been estimated at £2.5 billion and within the UK one in ten new medical graduates feel poorly prepared. XR has the potential to offer innovative ways of supplementing clinical skills training and research has demonstrated that it can work as well as, or potentially even better than, existing training. A recent meta-analysis of its use in training for laparoscopic surgery showed improvements in practice and quality including reduced error rates (Portelli et al. 2020). Twenty-four separate randomised controlled trials across different VR systems showed that VR led to faster operative times, better handling and OSATS scores (scores across seven parameters of skill) compared to apprenticeship training. One recent study suggested that improvement in overall surgical performance may potentially be as high as 230% versus traditional training methods (Blumstein et al., 2020).
Over the next few years it is clear that it will be necessary to implement more efficient healthcare services as we face not only the cost of an aging population but the repercussions of a global pandemic. The impact on workforce, elective care and training has been discussed previously and here we outline several use cases that demonstrate measurable potential in scaling up the delivery of existing care and education to ensure that the NHS can meet demand.
One such example is PRinVR, a device-led clinical service developed by GP-led company Concept Health Technologies which offers patients a VR platform for pulmonary rehabilitation. Using just a mobile connection, a wearable sensor and a VR headset, people can be monitored in real-time and the application is individually tailored 24hrs a day. More traditional exercise classes are typically oversubscribed as there is a shortage of the physiotherapy staff needed to conduct the sessions. Delivering exercise classes to patients within their own homes and exploiting the immersiveness of VR and the monitoring capacity of Concept Health’s Platform can free up clinician time by scaling up delivery of important interventions such as these. The value of PR in improving the health of chronic obstructive pulmonary disease (COPD) patients is well established. NICE guidance is such that patients with poor walking ability due to breathlessness after an exacerbation are given PR within 28 days. However, this is difficult for current services to achieve. One NHS Sustainability and Transformation Partnership (STP) in Staffordshire CCGs led by Stoke-on-Trent CCG that is piloting PRinVR is doing so because 95% of its COPD patients were not receiving PR within the recommended time after referral.
A 2015 national audit of PR in England and Wales found that from 68,000 referrals, of those that were assessed, only 60% of COPD patients fully completed their PR programme whereas 25% were enrolled but did not complete the course and 15% were assessed but did not enrol. This attrition, identified as a priority for improvement in the audit report, greatly reduces the effectiveness of rehabilitation programmes and in particular risks an increase in exacerbations. The costs of exacerbations are high, 15% of which result in a patient’s hospitalisation costing £1560 based on 4 bed days, and the other 85% are managed in primary care and social care, where the per-patient cost is £176, according to the Pulmonary Rehabilitation Impact Model on Exacerbations (PRIME). Furthermore, almost a third of all patients that are referred do not attend an assessment.
The PRinVR service with an all-in-one “device kit” and internet link and back-end cloud services is currently available on GCloud and at the time of writing is priced at £415 per patient including introductory training and technical support. An internal analysis by CHT compared PRinVR with conventional PR, estimated to cost £1200 (£800-1400) per patient completing PR. This analysis shows that not only is the VR solution two to three times cheaper, only 9% of the overall cost of the PR programme was due to non-completers versus 23% for conventional PR. It should be noted that the PRIME tool also estimated a cost of conventional PR at £344 in 2015, but this does not include drop out and hidden costs such that local information on completion rates is required to estimate the true cost per patient, as well as an uplift to 2021 for prices and tariffs.
From the experience of commissioners in Lincolnshire, patients who choose the VR option can benefit from faster access to rehabilitation and are able to start treatment within one week of their referral. This compares to a typical wait of a few months. So not only does PRinVR have the potential to offer cost-effective scalable pulmonary rehabilitation exercise classes but it can also support the NHS in reaching its ambitious goal of reducing exacerbations and hospital admissions through increasing access to services. In addition, it should reduce social care costs and those of informal carers. The COVID-19 pandemic has only increased the interest and need for remotely delivered PR (Houchen-Wolloff, 2020).
30 participants assessed use of VR versus paper floor plans of the hospital to prepare nurses for wayfinding in a new hospital building. Cost analysis showed that a VR approach was more expensive to develop but during implementation it reduced staff time needed for practicing wayfinding skills. The paper approach development cost was USD40,000 and implementation was USD530,000, while the virtual world development cost was USD220,000 and implementation was USD201,000,(Halfer, 2014).
Standardised patients are used for training medical students in several skills including communication and clinical decision making. These are actors who have to be trained to portray specific cases, provide oral feedback and engage with students in scenarios. One example that compared standardised patients to peer role playing for a clinical examination found that standardised patients were given five hours of training and were employed for 21 hours. Although the same training and preparation, along with tutors, were needed for both sessions there was a reduction of 53.6% in the hours needed for the peer role playing (Bosse, 2015). This suggests that if standardised patients are represented virtually there could be significant cost savings in real terms based on reduced hours. This way of delivering training has the potential to increase access and to increase the diversity of scenarios that can be offered.
Virti, a VR company offering mental health training here in the UK, in an internal study supported by the NHS Innovation Accelerator, has estimated that scaling its mental health training services through the platform could deliver upwards of £300,000 savings a year to a Trust. Several other companies, including GIGXR, offer similar interventions including a HoloPatient that has potential to be used to support nurses to develop clinical reasoning skills (Ditzel, 2021).
The pandemic has increased the number of people seeking information on careers in healthcare two-fold. This increase in interest may help us to meet the unprecedented demand within the NHS but, despite significant investment in upskilling and from HEE to ensure that training is available, there are logistical issues that may remain post-Covid. For instance, opportunities for learning, such as ward rounds, may become more restricted due to social distancing needs. Technology is unlikely to completely replace more traditional methods but can perhaps help HEE meet their commitment to increase the workforce in key areas, such as mental health, significantly.
During the pandemic it was important for staff and patients to be protected. Imperial College Healthcare NHS Trust worked in partnership with Imperial College London and Microsoft to introduce HoloLens onto wards treating COVID-19 patients. Healthcare teams could see what the doctor was seeing over a secure live video feed without having to be on the ward with him. This led to staff spending up to 83% less time in high risk areas and reduced the need for PPE, potentially by 700 items a week. The cost of PPE is around £1 for surgical gloves and up to £3.50 for an FFP2/3 mask (Walters, 2020) which suggests savings of up to £2450. The HoloLens has now been scaled up for use in teaching and training, with doctors leading virtual ward rounds and surgical teaching so that students still have access to the clinical experiences and exposure that are needed. Professor Amir Sam, Head of the School of Medicine at Imperial College London, also highlights the parity of access that this can provide students:
“Teaching with the HoloLens allows us to guarantee a level of exposure for our students to a far greater range of patients and medical conditions than ever before.”
Fear of Heights is a VR treatment where Nic, the virtual coach, guides people through the different levels of a shopping mall to help them overcome their fear through exposure therapy. Many people in the UK with a fear of heights do not engage with treatment as it is intense and difficult. A randomised controlled trial found that not only did Fear of Heights reduce fear but also increased engagement (Freeman et al., 2018). By providing a VR solution it enabled people to undertake intensive treatment in a shorter time and with less input from a therapist, thereby increasing access.
Although initial development costs were high when the intervention is deployed there are significantly lower costs than standard treatment as it does not require a therapist and can be delivered through inexpensive consumer VR equipment. Outcomes may also be significantly more improved with fewer dropouts and effect sizes higher than those found in a meta-analysis of therapist-assisted exposure therapy using real heights (Wolitzky-Taylor et al., 2008). Participants also reported significant changes in their approach to heights, with one reporting:
“I’ve just finished my sessions, I did four in total. Last week, after my third session, I went up to the Westgate [a shopping centre]; the difference in my mental capacity to deal with heights was amazing. Previously I wouldn’t go anywhere near the edges, I was almost hanging right off, looking vertically down. The sessions I’ve had here have given me a lot to think about, and certainly with regards to my fear of heights it feels like it’s helped a lot. So, very worthwhile doing.” (p.631)
Leading from the adoption of Fear of Heights within talking therapy (IAPT) services, Oxford Health NHS Foundation Trust has produced a Global Digital Exemplar blueprint to help other trusts to deliver Fear of Heights more quickly and cost effectively by providing a structured collection of resources and information to support implementation e.g. standard operating procedures, proformas, key decision areas, etc.
Despite a proven track record in delivering outcomes and many use cases to demonstrate its applicability and acceptability, XR products are not yet commonplace within healthcare settings. Many of the barriers that were seen in the early days of VR, such as cost, have been overcome but many also remain. Currently the pilots of XR in the NHS are fragmented.
In a survey of VR and AR companies the top challenge identified was monetary and funding issues. This was followed in second place by technical limitations (e.g. lack of immersion offered by mobile VR), clinical organisational issues (e.g. interoperability), lack of understanding of VR and AR as medical technologies and finally a lack of research studies. This report has demonstrated that there is a clear trajectory in three of those areas – the technical functionality and capacity, how well it is understood, and the research and evidence underpinning its use in healthcare. Conversations during the writing of this report have identified concerns particularly at the level of funding available but also at some of the barriers to embedding products within clinical settings. These are particularly notable for their potential to jeopardise the sustainability and scaling up of XR products.
Industry has consistently highlighted the challenges of NHS procurement for innovative and boundary pushing technologies such as these. One of the significant opportunities within XR is that there are several components involved, with some more affordable or potentially optional, enabling buyers to customise the offer to best suit their budget and needs. The hardware needed to access immersive applications range from basic add-on headsets that work with smartphones to the simulation or training suites that require more space and expertise to run. Equipment can be purchased as capital or leased but consideration needs to be made as to whether there is sufficient knowhow to run and manage them – this will of course be easier with the more ubiquitous technologies.
There are many freely available wellbeing applications available through stores such as Oculus that can be downloaded and used in the ways discussed in the wellbeing section. However, for interventions that require a more clinical approach (e.g. training) and require more immersive or interactive features (e.g. automation) the costs are higher. Developing software for the latter can be significantly more expensive. Many companies offer packages that can be tailored to include subscriptions to their software, devices and/or support.
Several of the products described within this report have relied considerably on charitable donations to purchase equipment. This is positive in that it demonstrates increasing interest and, for simpler technologies, appears to be practical. However, technical support must be provided in-house and this model often relies on an individual champion with a vested interest. A more commercial model, with a company and provision structure, necessitates an understanding of both XR and the complex healthcare economy. For instance, as mentioned in this section, PRinVR has behind it a clinician who also happens to be an inventor – this is the only VR system offered currently available through the Digital Marketplace.
More immersive technologies require more investment within their design and development e.g. in the case of Oxford VR’s products. Often these are reliant on funding from bodies such as NIHR and there are often significant gaps in investment between development, research, and finally procurement within the NHS. Very few healthcare services will have a dedicated VR lab like Torbay’s, so these independent organisations must attempt to sustain themselves through these stages at the same time as understanding how best to embed themselves within services that can differ significantly across the nation. It is also important to consider who will be buying the product. Enthusiastic clinicians or patients may demonstrate its value at these levels but it will be commissioners, providers or other businesses that hold the purse strings. Understanding their needs and how a product can meet them is essential.
It is disappointing that currently NICE does not include guidance on the use of XR within any of the clinical pathways despite the evidence available, particularly within mental healthcare where its efficacy has been demonstrated. Furthermore, there are only three GDE blueprints based on XR technology, two within mental health while, as we have seen, VR is already deployed in a variety of NHS and social care settings. Frameworks such as the NICE Evidence Standards Framework for Digital Health Technologies and the NHSX Digital Technology Assessment Criteria should help industry and healthcare organisations alike to identify those products that have the potential to be both usable and useful within the sector, as well as those that necessitate a higher level of oversight within the MHRA medical device regulations. However, more is needed to ensure that XR products that demonstrate they are safe, operable, secure and effective can be more easily procured. This will also help to ensure that the industry can continue to take advantage of this forward momentum so that current and future XR innovations have a route to their realisation within healthcare.
The pandemic has led to a sea change in how the NHS uses digital technologies with more consultations and appointments conducted remotely and many users of social care services provided with tablets and other devices to enable their access to support and advice from a distance. NHSX has taken a significant role in publishing guidance and offering funding to support the changes being made due to the Coronavirus outbreak. This important development in digital transformation that has accelerated out of necessity will surely endure.
This report has demonstrated clear benefits that can be realised from the use of XR within healthcare, along with the many use cases that showcase its affordability, flexibility and acceptability amongst patients and clinicians alike. XR, and especially VR, has already found a place in the NHS and this was seen in the many examples above that are showing its role in reducing the costs of service delivery, improving outcomes for patients and staff, and demonstrating its potential to increase access to treatments and training. While there are few formal health economic assessments of VR, this chapter has shown there are good examples of both cash-releasing cost savings and cost-effectiveness, and others that are low cost or close to cost neutrality where the saving to the healthcare system could be usefully examined further. Emerging innovations in AR and MR are yet to have health economic assessments.
It will be important to continue building an offering for XR, and digital health in general that works within the existing infrastructure and information governance but most importantly, that will work for the broadest spectrum of service users. This means using co-design and taking steps to ensure that digital exclusion is addressed and the widening of health inequalities is prevented. The future of digital health in the NHS will require a specific emphasis on XR to make the most of the innovation happening in this space, developing its use beyond small scale pilots. It will be important to grow the number of Future NHS GDE blueprints for deploying XR in different healthcare settings.
As has been seen for remote monitoring during the pandemic through the Innovation Collaboratives (another part of Future NHS), a structured approach to supporting scaling by the sharing of knowledge about appropriate rapid study designs and an emphasis on calculating benefits for XR, including health economic assessments, would be very useful. Alongside this, use case templates of XR in both the NICE evidence framework and for the NHSX DTAC would be helpful tools. The funding of health economic evaluations of XR deployments in the NHS would support the case for scaling the successful ones. The procurement framework must consider the capital and IT costs associated with XR and ensure that, when costed, it recognises the investment within the development of interventions, which involves the design of high-quality virtual environments.
Funding bodies should continue to help researchers develop and generate evidence for XR in healthcare. Research projects like those funded by UKRI (including Innovate UK) or NIHR (e.g. the i4i programme that is supporting the development and trial of novel XR therapies like gameChange) are beginning to enter the next stages of implementation and an emphasis on sustainability should be supported, as well as investing in the development of new technologies and applications in healthcare.
While this section of the report has mostly focused on highlighting the breadth of work being done to show the value of XR within the UK and in the NHS, there are other organisations in different countries working in this field, sometimes for longer, so there is good potential for international collaboration.
The UK healthcare sector is facing challenging times in terms of workforce retention, development and training as a result of the impact of Covid-19 and due to the fact that 30%-50% of frontline healthcare workers are due to retire by 2030.
There are significant opportunities for the UK to maximise the use of AR and VR in training and workforce development, maintaining the quality of provision and delivering at scale, with lower costs and greater accessibility.
The UK is seen as a global leader in workforce training and education in healthcare.
There is a high demand from organisations worldwide for new ways to deliver high quality education and training in healthcare for growing and aging populations (Healthcare UK Annual Review 2019-20). Due to the UK’s strong global reputation in healthcare education and training and a growing XR sector, the UK could be well positioned to become an International world leader and market in virtual training.
One of the NHS’s priorities is remote support for musculoskeletal conditions. Today, the majority of rehabilitation is carried out in clinics and hospitals and at home, with a backlog of physiotherapy appointments that are continuing to grow, due to the lockdown restrictions. Many patients report difficulty in following rehabilitation at home due to a lack of motivation to exercise on their own.
VR systems available for use at home could help solve this problem by improving patient participation and enjoyment. This is in addition to having other benefits, such as cost savings, reducing patient transportation to appointments, reducing home visits, or providing patients with self-care/self-management, which could be beneficial for the long-term management of conditions. In addition, home rehabilitation in emergency Covid-19 situations presents itself as more relevant than ever to help reduce care pressure. The benefits evident for rehabilitation could also be seen across other areas of VR based health care including pain, mental health, and more.
Some of the most interesting use cases of XR solutions in wellbeing and mental health have come from alliances between creative games companies, the XR sector, clinicians and academics. The UK games sector reached a record £7bn in 2020 (UKIE 2020). XR businesses are exploring new markets and opportunities and the creative sector makes a vital and varied contribution to the health and wellbeing of society. According to the charity Mind, one in four people experience mental health problems of some kind each year, and the impact of Covid-19 and lockdown is accelerating an already significant problem for the UK and globally. Strategic support to bring together the creative, technical and story based skillsets of the creative, games and XR sector with clinicians and researchers could fast track new R&D and solutions to address the mental health and wellbeing needs of the UK population.
The combination of 5G, AI and Internet of Things (IoT) will have a significant impact on patient care, allowing patients to be treated without the need for the physical presence of a healthcare professional. 5G-enabled remote patient monitoring in real time, automation and enhanced medical processes and solutions will transform and improve the way that patient care is delivered, such as enabling the healthcare system to work remotely and collaboratively with patients, caregivers and clinicians. The integration of biofeedback integrated technologies will further enable opportunities for remote patient monitoring, alongside their engagement in the therapies and experiences available.
AI and IoT will turn data into actionable information which brings the potential to speed up diagnoses, giving patients a better overall experience, as well as boosting efficiencies.
Healthcare is converging with various emerging technologies and the benefits of these intersections are still being discovered along with the risks that come with such convergence. In the case of Immersive Reality (XR), the risks are evolving and are yet to be well defined. Increasingly research is being developed to explore such risks, including a literature review developed by the Department of Business Energy and Industrial Strategy in 2020, which investigated The Safety of Domestic Virtual Reality Systems. Here, we examine several risk categories pertinent to the UK health sector and offer insights on the novel risks XR systems may introduce.
There is a lack of clarity regarding how to categorise XR hardware and software as Medical Devices or Software as a Medical Device (SaMDs) from regulators, companies and healthcare providers. Regulatory approval can take a number of years, representing an inhibitory factor to development and deployment. Compliance in regulation is reported as very difficult for businesses to navigate, understand and apply within business.
In 2020, the FDA and Department of Health and Human Services in the United States held a public workshop to discuss medical extended reality and best evaluation practices. A similar strategy is needed within the UK so we can better understand how XR can work best within the existing system, rather than seeing it as something outside of it. We are yet to see any specific guidance from NICE or MHRA on XR. Regulators must collaborate with stakeholders, into an alliance structure to meaningfully problem solve where emerging technologies like XR fit into existing governance structures. It should be made clear that any product that can be classified as a medical device must have MHRA approval and that failure to do so runs the risk that unsafe XR technologies could be deployed within the NHS.
The National Institute for Health and Care Excellence (NICE) have worked with partners to develop standards that ensure new technologies are clinically effective and offer economic value. The NICE standards can be used with XR and the Digital Technology Assessment Criteria (DTAC) should be completed for every digital technology product within the NHS, even if organisations are piloting or trialling it. NICE created the standards as part of a working group led by NHS England. This group also includes Public Health England, MedCity and DigitalHealth.London, all of which can be found here.
Usability is a broad concept that encompasses the interaction of a system and the users that support or may be supplemental to its primary function. In a clinical setting, XR equipment must provide an acceptable level of physical and social comfort for many hours. There may be ambient noises, numerous sources of light, other non–XR devices that may need to be operated and multiple people with whom the operator will need to interact. The XR equipment must integrate into the environment seamlessly without subjecting the user to undue restrictions. For example, an XR device user may remove the XR device multiple times during a medical procedure. When doing so, the device must not cause dizziness, visual disturbances, or an excess cognitive load. The process of moving in and out of the virtual environment must not be excessively long so that it would cause delays in the clinical workflow.
The usability of XR devices should preserve the natural interaction between doctor and patients. User experience in XR applications will need to accommodate varying levels of comfort and skill in XR if this technology is to become mainstream. Some features may not meet the needs of users from diverse backgrounds, with various levels of familiarity with technology, variety of age groups, cognitive and physical abilities. Usability studies with a variety of target audiences are highly recommended to arrive at the proper level of accessibility for patients and healthcare providers.
A medical XR ecosystem can leverage AI algorithms, cloud systems, video cameras, sensors, operating systems and many more network components, increasing the potential threat of attack. A cybersecurity attack can be carried out against any and all components of the system, targeting existing gathered patient information or live patient movement and behavioural data. In addition to traditional best practices of protecting confidentiality, integrity and availability of services and data, the XR solution providers should also consider new data formats, clinical use cases and connectivity requirements to assure resilience against novel cybersecurity threats.
In breakthrough research (Casey, Baggili and Yarramreddy. 2019), a team from the Cyber Forensics Research and Education Group (UNHcFREG) at the University of New Haven show that it is possible to make a person into a “human joystick” by sending altered traffic, adulterating camera controls, and by uploading unintended data files onto the XR goggles. The subject can be tricked into walking in the wrong direction, possibly falling and injuring themselves. Therefore, cybersecurity controls that are currently recommended for network-based medical devices, applications, and IoT systems must apply to medical XR systems. A detailed assessment of cybersecurity related risks to the system, using best practices for analysing security and privacy across the entire ecosystem is recommended.
Asset management of XR devices presents an unique set of logistical concerns for a system with a wide deployment. Administrative oversight and maintenance procedures may vary based on the geographical location and skillset of the users. Therefore, XR devices must be maintained and configured to ensure functional alignment across locations.
XR equipment is frequently expensive and the structures of NHS procurement processes make it hard or timely to purchase. If a piece of XR equipment is rented or otherwise shared among multiple users, hygiene and disinfection of the equipment are imperative as part of a common system-wide maintenance and asset management system. Various healthcare and education facilities have developed hygiene protocols to address this issue but must be uniformly published and adopted.
Interoperability is also a unique consideration in procuring the appropriate assets. In order to achieve worldwide deployment of XR technology, the XR equipment must be interoperable and available for deployment across distributed ecosystems. NHS IT is tightly controlled in terms of what can be downloaded or which devices can access wi-fi, for example. This can mean long delays or simply not being able to access hardware or software on local NHS Trust networks, inhibiting or ceasing progress of projects or studies.
XR devices can generate and process large amounts of highly personal data including metadata and health inferences across various geolocations. XR expands the definition of personal information that must be protected, including biometrically-inferred data, which is especially prevalent in XR data pipelines.
There are numerous examples of data unique to XR applications, including cameras that continuously capture a patient's gaze, handheld controllers and other body sensors, video and audio recordings. The collected data includes methods of positively identifying an individual, biometrically inferred data, behavioural data, room data, mood, gender presentation, sexual orientation, and unique physical attributes not generally captured during clinical interactions. Current regulatory definitions of PII (Personally Identifiable Data /Personal Data) and PHI (Personal Health Information) may need to be expanded to incorporate and govern the unique types of data created and used by XR applications.
Currently, emerging regulations (General Application Order Concerning Biometrics, 2014) address concerns related to biometric data, and reiterate the following:(1) the processing of biometric data requires the provision of an information notice; (2) the processing requires the data subject’s consent; (3) biometric data must be protected by adequate security measures (e.g. encryption); (4) access to databases containing biometric data must be tracked; (5) data must be retained as long as necessary for the processing purpose. For example, an event (Microsoft News Center, 2021) involving 15 surgeons from 13 countries in a 24-hour XR marathon demonstrates one possible example of how regulatory and privacy law alignment will be needed to realise the benefits of XR collaboration across countries.
There still remains a gap in addressing the governance of sophisticated data collected by XR devices and processed in combination with Artificial Intelligence (AI) algorithms. The EU Commission has proposed the first ever legal framework on AI (Regulatory framework on AI, 2021), which addresses the risks of AI and positions Europe to play a leading role globally. XRSI and NHS remain committed to adopt these emerging regulations and are currently developing community-based standards and frameworks on medical XR privacy and safety to support current research and standards efforts (Research & Standards | XRSI – XR Safety Initiative, 2021). This effort will help produce a set of practical controls and guidelines for addressing this issue for the global XR community.
The principal recommendation is the creation and support of an XR in healthcare development framework to enable an end-to-end pipeline, from design to test, scale and spread of successful new models of care. Central to this recommendation is collaboration from the key stakeholders with the ability to support delivery, ensuring the right mix of expertise, research, insight and talent are at the heart of innovation in order to co-create an XR in the healthcare ecosystem.
Undertake a comprehensive mapping and analysis of the businesses, healthcare organisations and universities working with XR in healthcare to better understand the UK’s capacity and capabilities, the size and scale of the market, its potential value and future growth. This work would inform the development of the Centres of Excellence and ensure the right people are engaged in their creation and implementation.
Focusing on XR in healthcare, to support an end to end development pipeline that enables a clear pathway from concept to investment and scaling to market. CoE would facilitate the production of clinically robust, engaging, marketable solutions and the creation of a national platform for secure distribution, simplifying purchasing options for healthcare providers. The CoEs would stem from existing expertise in five key areas: physiotherapy and rehabilitation, mental health and wellbeing, pain management, healthcare professional training and patient education. Vitally, CoEs will also act to further develop a body of evidence.
Establish a representative and impartial network that facilitates the connection of academic institutions, researchers, healthcare providers, clinicians, XR, digital and creative industries in order to support new collaborations, inform, signpost, and share insight and expertise. The network will build awareness of the use of XR in healthcare, showcase UK businesses and connect the UK to the international community. The network will be inclusive and open to anyone with an interest in XR in health and will also connect and enable knowledge exchange between the Centres of Excellence. The network will also act to link other XR initiatives in the UK and globally, pursuing the same and similar routes of growth and expansion in order to avoid duplication and collaboratively share learning, skills and resources.
“As of today, there is no government that has developed a specific initiative to support the growth of the XR and healthcare industry within their own borders. The United Kingdom, by utilizing this strategic report as a starting point, has a unique opportunity to take a global lead in setting a growth agenda for this space. Doing so would spur other governments toward initiating their own programs, and would help to instigate growth of the sector at a global scale.”
Bob Fine, Director of the International Virtual Reality Healthcare Association
Central to the delivery of the above recommendations is cross sector national and international partnerships and collaborations between the XR industry, academia and healthcare providers. The XR industry is well positioned to provide creative and technical solutions, academia offers early-stage innovation and independent evaluations of XR innovations, whilst the healthcare sector can offer translation into practice. All are critical to the creation and deployment of world-leading XR in healthcare solutions as captured below, ( fig.3) :
A HMD delivers two computer-generated images, one for each eye. The 2D images are completed and rendered with appropriate perspective with respect to the position of each eye in the three-dimensionally described virtual scene. Together, the images therefore form a stereo pair. The two small displays are placed in front of the corresponding eye, with optics that enable the user to see the images.
A scenario is captured by using a special camera and subsequent software to patch video together to form a completely surrounding scene that can be displayed in a Head Mounted Display (HMD). Due to head tracking, the viewer can look all around the scene and, depending on how it has been captured, it can also be displayed in stereo.
Mobile VR ranges from using a smartphone’s display and sensors that detect its orientation as the foundation of a VR system placed within a headset such as a Google Cardboard, to advanced mobile VR such as Google Daydream and Samsung Gear, which requires a Samsung phone to be connected to the specialist headset.
Haptics, also known as kinaesthetic communication or 3D touch, refers to any technology that can create an experience of touch by applying forces, vibrations, or motions to the user.
Social prescribing is part of a new initiative to offer more personalised care to people across the health and care system. This new initiative supports people experiencing psychological or physical distress being referred (or referring themselves) to engage with the arts in the community, including galleries, museums and libraries.
Behavioural healthcare is the scientific study of the emotions, behaviours and biology relating to a person’s mental wellbeing, their ability to function in everyday life and their concept of self.
The brain creates an embodied simulation of the body in the world used to represent and predict actions, concepts, and emotions. This helps us predict and manage sensory experiences both in and outside the body
Immersive technologies offer the ability to create an inclusive environment that shuts out the real world and develops new surroundings to create vivid new illusions of reality to those in the headset. Awareness of our environment is mediated by our sensory experience, from sight to taste, touch and smell.
Presence is the result of immersion and leads to a subjective illusion of feeling as though you are physically in the environment displayed in the VR headset.
Some users may experience simulation sickness when using a virtual reality headset. Cybersickness symptoms are similar to motion sickness (nausea, dizziness, etc.) and are primarily linked to low refresh rates or a particular VR game/experience.
A marketing term used for the type of immersive surround sound experienced in virtual reality.
A standalone virtual reality headset is an HMD that doesn’t require a PC or a smartphone. Common synonyms include all-in-one VR headset, wireless VR headset, and autonomous VR headset.
A tethered VR/AR headset must be connected by cable to a PC (or gaming console, in the case of the Sony PS VR). Tethered virtual reality headsets are also known as desktop VR headsets or PC VR headsets.
Windows Mixed Reality is a content platform where users can access VR games and experiences via Windows Mixed Reality headsets. This name may be misleading, as Windows MR headsets are in fact virtual reality devices and not mixed reality.
Extended reality is the umbrella category that covers all the various forms of computer-altered reality, including augmented reality (AR), mixed reality (MR), and virtual reality (VR).
Degrees of freedom are the number of movement types that the user may experience. In VR, headsets offer either 3DoF (360° tracking) or 6DoF (positional tracking).
The wider the field of view, the better. For reference, the human field of view (with eye rotation) reaches a maximum of 220°. Most VR headsets provide around 100 degrees.
The authors would like to thank the contributors, funders and advisory groups who generously supported the project, bringing a remarkable breadth of knowledge and expertise to the development of the report.
With thanks to the funding organisations, without whom the report would not have been possible.
UKRI Audience of the Future Programme
Health Education England, Technology Enhanced Learning
Pfizer Ltd. provided a Business Donation to support the writing of this report and had no editorial input but reviewed the final version for compliance with pharmaceutical regulations.
Jeremy Dalton - Author Reality Check - A Book About XR in Business
Clinical Mental Health and Health Economics
Dr. Michael Craven Principal Research Fellow and Dr Aislinn Bergin, Research Fellow - NIHR MedTech Cooperative (MindTech)
Sarah Ticho - XR Wellbeing Specialist, and founder of Hatsumi
Ross O’Brien -Associate Director of Innovation and Technology, Central and Northwest London NHS Foundation Trust
Physiotherapy and Rehabilitation
Ivan Phelan- Senior Research Fellow, and Dr Alicia Carrion-Plaza - Research Fellow, Sheffield Hallam University
Clinical Skills and Workforce Training
Richard Price -Learning Technologies Advisor, Health Education England
Abison Logeswaren -Extended Realities Fellow at Health Education England and Ophthalmic Surgery Resident at Moorfields Eye Hospital
Fiona Kilkelly, Independent XR Industry Specialist
Patient Focussed Education
Dr Payal Ghatnekar - Learning Technologies Research Programme Manager, Torbay and South Devon NHS Foundation Trust
Dr Nigel Newbutt - Senior Lecturer: Digital Education,University of the West of England and Nicholas Peres - Head of Digital Technologies, Torbay and South Devon NHS Foundation Trust, and Health Education England
Safety and Security
Rachel Michelson - Medical XR Advisory Council Lead, Laura Elan, Medical XR Advisory Board Member and Kavya Pearlman, Board Member, XRSI Medical Advisory Council, Founder and CEO - The XR Safety Initiative
Report Advisory Board
Tara Donnelly - Chief Digital Officer, NHSX
Rhodri Joyce - Deputy Director of Innovation, NHSX
Yinka McKinde - Head of Innovation, NHSX
Neesa Mangalaparathy - Programme Manager - Digital Productivity, NHSX
Charlotte Clay - Benefits Researcher, NHSX
Ben Williams - Project Lead, Care City London Jeremy Dalton - Author Reality Check - A Book About XR in Business
Andrew Chitty - Challenge Director: Audience of the Future and Creative Clusters Programme
Matt Wordley - CEO, ReScape Innovations
Hamish Graham MRCS - London Manager, Pfizer Healthcare Hub (Observer)
Michael Craven - Principal/Senior Research Fellow, NIHR Mindtech Medtech Cooperative
Gabriela Juarez MartinezKnowledge Transfer Manager for Pharma and Medtech, Knowledge Transfer Network KTN
Case Study Supporters
Rafael Guerrero, Alder Hey Children’s Hospital
Dennis Chan, UCL
Oli Colbourne Laight and Helen Davies-Cox, Torbay and South Devon NHS Foundation Trust (TSDFT)
Dr. Sheila Popert, Prospect Hospice
Isabelle Van Der Keere, Immersive Rehab
Catherine Allen, Limina
Charles Nduka, Co-Founder of Emteq and Member of the XRSI Medical Advisory Board
Jason Lawson, Imperial College London
A special thanks to Ben Williams of Care City London, who first sparked the possibility of this report and initiating the conversation with the NHS.
Tom Warner, Straw Man Hero
Fiona Kilkelly - Independent XR Industry Specialist
Fiona has worked at the nexus of innovation, technology and the creative industries for over 25 years. She founded Immerse UK, the UK’s immersive technology network in 2017 and is currently Chair of Eirmersive, the voice of the immersive sector in Ireland. Working primarily with government funded innovation programmes, Fiona is highly experienced in leveraging potential to support innovation and growth in businesses, programmes and organisations. She has led multiple industry ecosystem reports outlining the UK’s capacity opportunities and barriers in the immersive, health and creative and digital sectors. Fiona is currently Head of Immersive (maternity cover) at Story Futures Academy at the National Film and Television School with Royal Holloway.
Ross O’Brien - Associate Director of Innovation and Technology, Central and Northwest London NHS Foundation Trust
Ross O’Brien is the Associate Director of Innovation and Technology at Central and North West London NHS Foundation Trust in the UK. He was formerly the Service Lead for the Grenfell Health and Wellbeing Service which is the NHS mental health service established following the Grenfell Tower fire in London in 2017 claiming 72 lives. He established the service to support the traumatised and bereaved community affected by the tragedy. Ross pioneered the use of VR with the community affected in the immediate months following the tragedy.Ross is also the Programme Lead of the London Digital Talking Therapies Programme at Healthy London Partnerships and is working to establish a digital single point of access for all Londoners accessing NHS psychological therapy services. Ross is a recent graduate from the Digital Health London Digital Pioneer Fellowship. Ross holds a Human Rights Masters from the University of London.
Sarah Ticho - XR Industry Specialist, Hatsumi
Sarah is an XR healthcare specialist, working broadly across research, creative producing and curation of XR experiences at the intersection of immersive technology, participatory art, and storytelling to improve physical and mental health. She is the co-founder of the XR Health Alliance, and founder of Hatsumi (which means to see for the first time). She is the producer at Explore Deep, an award-winning clinically validated breath-controlled VR experience designed to reduce anxiety, developed in close collaboration with the Games for Emotional and Mental Health Lab, Radboud University. She has worked with a number of organisations across the immersive and healthcare space including Immerse UK, Healthcare Education England and the XR Safety Initiative Medical Council and continues to create opportunities to bring together practitioners across academia, healthcare and the creative industries to create an equitable and just future. She is also an End of Life Doula in training, a non-medical support for people going through the end of life process.
ABI Research, (2020) Augmented and Virtual Reality: Investments, Acquisitions, and Market Trends for 2020
Alsop, T. (2020) Size of the augmented & virtual reality (AR/VR) market in Europe 2018-2026, Statista
Eccles, L. (2021) Virtual reality: pandemic leads to rise in headset sales to escape lockdown, The Times
IDC (2020) Worldwide Augmented and Virtual Reality Spending Guide
PWC (2021) How UK organisations are using XR
Vailshery, L.S (2021) Augmented reality (AR) and virtual reality (VR) headset shipments worldwide 2020-2025, Statista
Verizon Media (2020) The coming of age for immersive experiences
XR Association (2020) 2020 Augmented and Virtual Reality Survey Report
Allison, A., & Bergin, A. D. (n.d.).
Public perception of virtual reality psychological therapy guided by a virtual coach: a mixed methods survey study.
Aruanno, B., Garzotto, F., Torelli, E., & Vona, F. (2018).
Holo learn: Wearable mixed reality for people with neurodevelopmental disorders
Baus, O., & Bouchard, S. (2014).
Moving from virtual reality exposure-based therapy to augmented reality exposure-based therapy
Craig, T. K., Rus-Calafell, M., Ward, T., Leff, J. P., Huckvale, M., Howarth, E., … Garety, P. A. (2018).
AVATAR therapy for auditory verbal hallucinations in people with psychosis: a single-blind, randomised controlled trial
Crowell, C., Sayis, B., Benitez, J. P., & Pares, N. (2020).
Mixed Reality, Full-Body Interactive Experience to Encourage Social Initiation for Autism: Comparison with a Control Nondigital Intervention
Eshuis, L. V, Gelderen, M. J. Van, Zuiden, M. Van, Nijdam, M. J., & Vermetten, E. (2020). Efficacy of immersive PTSD treatments : A systematic review of virtual and augmented reality exposure therapy and a meta-analysis of virtual reality exposure therapy. Journal of Psychiatric Research, (November).
Falconer, C. J., Rovira, A., King, J. A., Gilbert, P., Antley, A., Fearon, P., … Brewin, C. R. (2016). Embodying self-compassion within virtual reality and its effects on patients with depression. BJPsych Open, 2(1), 74–80.
Ferrer-García, M., & Gutiérrez-Maldonado, J. (2012). The use of virtual reality in the study, assessment, and treatment of body image in eating disorders and nonclinical samples: a review of the literature. Body Image, 9(1), 1–11.
Freeman, D., Yu, L.-M., Kabir, T., Martin, J., Craven, M., Leal, J., … Waite, F. (2019). Automated virtual reality (VR) cognitive therapy for patients with psychosis: Study protocol for a single-blind parallel group randomised controlled trial (gameChange). BMJ Open, 9(8).
García-Palacios, A., Hoffman, H. G., Kwong See, S., Tsai, A., & Botella, C. (2004). Redefining Therapeutic Success with Virtual Reality Exposure Therapy. CyberPsychology & Behavior, 4(3).
Gromer, D., Madeira, O., Gast, P., Nehfischer, M., Jost, M., Müller, M., … Pauli, P. (2018). Height simulation in a virtual reality cave system: Validity of fear responses and effects of an immersion manipulation. Frontiers in Human Neuroscience, 12.
Leff, J., Williams, G., Huckvale, M., Arbuthnot, M., & Leff, A. P. (2014). Avatar therapy for persecutory auditory hallucinations: What is it and how does it work? Psychosis, 6(2), 166–176.
Lindner, P., Miloff, A., Zetterlund, E., Reuterskiöld, L., Andersson, G., & Carlbring, P. (2019). Attitudes toward and familiarity with virtual reality therapy among practicing cognitive behavior therapists: A cross-sectional survey study in the era of consumer VR platforms. Frontiers in Psychology, 10(FEB).
Martens, M. A. G., Antley, A., Freeman, D., Slater, M., Harrison, P. J., & Tunbridge, E. M. (2019). It feels real: physiological responses to a stressful virtual reality environment and its impact on working memory. Journal of Psychopharmacology, 33(10), 1264–1273.
Morina, N., Ijntema, H., Meyerbröker, K., & Emmelkamp, P. M. G. (2015). Can virtual reality exposure therapy gains be generalized to real-life? A meta-analysis of studies applying behavioral assessments. Behaviour Research and Therapy, 74, 18–24.
Opris, D., Pintea, S., Garcia-Palacios, A., Botella, C., Szamosközi, Ş., & David, D. (2012). Virtual reality exposure therapy in anxiety disorders: a quantitative meta-analysis. Depression and Anxiety, 29(2), 85–93.
Parsons, S., & Cobb, S. (2011). State-of-the-art of virtual reality technologies for children on the autism spectrum. European Journal of Special Needs Education, 26(3), 355–366.
Perusseau-Lambert, A., Anastassova, M., Boukallel, M., Chetouani, M., & Grynszpan, O. (2018). Haptic and tactile interfaces for autism: A systematic review. Enfance, (1), 65.
Rizzo, A. S., & Koenig, S. T. (2017). Is clinical virtual reality ready for primetime? Neuropsychology, 31(8), 877–899.
Rothbaum, B. O., Hodges, L. F., Kooper, R., Opdyke, D., Williford, J. S., & North, M. (1995). Effectiveness of computer-generated (virtual reality) graded exposure in the treatment of acrophobia. American Journal of Psychiatry, 152(4), 626–628.
Serino, S., Scarpina, F., Keizer, A., Pedroli, E., Dakanalis, A., Castelnuovo, G., … Riva, G. (2016). A novel technique for improving bodily experience in a non-operable super-super obesity case. Frontiers in Psychology, 7(JUN), 1–6.
Sorkin, A., Weinshall, D., Modai, I., & Peled, A. (2006). Improving the accuracy of the diagnosis of schizophrenia by means of virtual reality. American Journal of Psychiatry, 163(3), 512–520.
Valmaggia, L. (2017). The use of virtual reality in psychosis research and treatment. World Psychiatry, 16(3), 246–247.
Bevan, C., Green, D. P., Farmer, H., Rose, M., Cater, K., Stanton Fraser, D., & Brown, H. (2019). Behind the Curtain of the 'Ultimate Empathy Machine': On the Composition of Virtual Reality Nonfiction Experiences. In CHI '19: Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (2019 ed., pp. 1-12).  (CHI Conference Proceedings). Association for Computing Machinery.
Bossenbroek, R., Wols A., Weerdmeester, J., Lichtwarck-Aschoff, A., van Rooij, M.J.W. & Granic, I. (2020). Efficacy of a virtual Reality biofeedback game (DEEP) to reduce anxiety and disruptive classroom behavior: Single-case study. JMIR Mental Health. doi:10.2196/16066
Bourdin P, Barberia I, Oliva R, Slater M (2017) A Virtual Out-of-Body Experience Reduces Fear of Death. PLoS ONE 12(1): journals.plos.org/plosone/article?id=10.1371/journal.pone.0169343e01
Daffern, , H. Camlin, D. A.,, Egermann, H., Gully, A. J. Kearney, G., Neale, C., & Rees-Jones, J. (2019) Exploring the potential of virtual reality technology to investigate the health and well being benefits of group singing, International Journal of Performance Arts and Digital Media, 15:1, 1-22, DOI:
Damiani, J. (2020) Naked and unafraid to exercise in virtual reality, Wired
Fancourt, D., Finn. S, (2019) What is the evidence on the role of the arts in improving health and well-being? A scoping review, World Health Organisation, Accessed 13/04/2021
Feathers, T. (2020) Videogame Technology Could Bring Biofeedback Therapy to the Living Room, Wall Street Journal
Riva, G., Mantovani, F., Wiederhold, B.K., (2020) Positive Technology and COVID-19, Cyberpsychology, Behavior , and Social Networking, 581-587
Riva, G., Wiederhold, B.K., (2020) How Cyberpsychology and Virtual Reality Can Help Us to Overcome the Psychological Burden of Coronavirus, Cyberpsychology, Behavior, and Social Networking.277-279.
Giuseppe, R., Luca, B., Browning, M. , Castelnuov. G.,, Cavedoni, S.,, Chirico, A., Cipresso, P., de Paula Dirce, M.B, Di Lernia, D, Fernández-Álvarez, J., Figueras-Puigderrajols, N., Fuji, K., Gaggioli, A., Gutiérrez-Maldonado, J., Hong, U., Mancuso, V., Mazzeo, M., Molinari, E., Moretti L. F., Ortiz de Gortari, AB., Pagnini, F, Pedroli, E., Repetto, C, Sforza, F., Stramba-Badiale, C., Tuena, C, Malighetti, C., Villani, D, Wiederhold, B.K., (2020) COVID Feel GoodAn Easy Self-Help Virtual Reality Protocol to Overcome the Psychological Burden of Coronavirus , Frontiers in Psychiatry, Vol 11:996 doi
Hacmun I, Regev D and Salomon R (2018) The Principles of Art Therapy in Virtual Reality. Front. Psychol. 9:2082. doi:
Inkster, B,, O’Brien, R et al. (2021) Early Warning Signs of a Mental Health Tsunami: A Coordinated Response to Gather Initial Data Insights From Multiple Digital Services Providers. Frontiers in Digital Health
www.frontiersin.org/ article/10.3389/ fdgth.2020.57890
Meir Lotan, Shira Yalon-Chamovitz, Patrice L. (Tamar) Weiss, (2009) Improving physical fitness of individuals with intellectual and developmental disability through a Virtual Reality Intervention Program, Research in Developmental Disabilities, Volume 30:2,229-239,
Plante, T.G,. Aldridge, A., Bogden, R., Hanelin, C. (2003) Might virtual reality promote the mood benefits of exercise? Computers in Human Behavior, Vol 19:4 495-509,
Saredakis, D., Keage, H. A., Corlis, M., & Loetscher, T. (2020). Using Virtual Reality to Improve Apathy in Residential Aged Care: Mixed Methods Study. Journal of medical Internet research, 22(6), e17632.
Sonney, J.m Björling, E.A., Rodriguez, S., Carr, N.(2021) Pivoting to “No Contact”: A Protocol for Conducting a Virtual Reality Relaxation Home Study for Teens Amidst the COVID-19 Pandemic, Journal of Paediatric Healthcare,
BBC, Making VR a Reality Storytelling and audience insights (2019) Accessed 13/04/2021
Weerdmeester, J. van Rooij, M.M.J.W., Maciejewski, D., Engels, R.C.M.E., & Granic, I. (2020). A Randomized Controlled Trial Assessing the Efficacy of a Virtual Reality Biofeedback Video Game: Anxiety Outcomes and Appraisal Processes. Manuscript submitted for publication.
Berger, M. M., Davadant, M., Marin, C., Wasserfallen, J. B., Pinget, C., Maravic, P., Kochd, N., Raffould, W., & Chiolero, R. L. (2010). Impact of a pain protocol including hypnosis in major burns. Burns,
Bucolo et al. (2006). The design of a tangible interaction device to alleviate anxiety and pain in paediatric burns patients In CHI '06 Extended Abstracts on Human Factors in Computing Systems (CHI EA '06). Association for Computing Machinery, New York, NY, USA, 129–134.
Fayaz A, Croft P, Langford RM, et al. (2016). Prevalence of chronic pain in the UK: a systematic review and meta-analysis of population studies.
Ghatnekar, P. (2021). Podiatry anxiety and pain distraction using VR
Gromala, D, Tong, X, Choo, et al, (2015). The Virtual Meditative Walk: Virtual Reality Therapy for Chronic Pain Management; ; 10.1145/2702123.2702344
Hodge, J., Balaam, M., Hastings, S., & Morrissey, K. (2018). Exploring the design of tailored virtual reality experiences for people with dementia. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems, 1-13.
Hoffman HG, Doctor JN, Patterson DR, Carrougher GJ, Furness TA. (2000). 3rd. Virtual reality as an adjunctive pain control during burn wound care in adolescent patients. Pain. Mar;85(1-2):305-9. doi: 10.1016/s0304-3959(99)00275-4. PMID: 10692634.
Jamison, Robert N. (2017). Are we really ready for telehealth cognitive behavioral therapy for pain?, PAIN: April 2017, Volume 158, Issue 4, p 539-540 doi: 10.1097/j.pain.0000000000000801
K P Grichnik, F M Ferrante (1991) The difference between acute and chronic pain studies
Levy N, Lord L J, Lobo D N. 2021. UK recommendations on opioid stewardship 372 :m4901 doi:10.1136/bmj.m4901
Li A, Montaño Z, Chen VJ, Gold JI.2011. Virtual reality and pain management: current trends and future directions. Pain Manag. ;1(2):147-157. doi:10.2217/pmt.10.15
Macleod, R., Shepherd, L., & Thompson, A. R. (2016). Posttraumatic stress symptomatology and appearance distress following burn injury: An interpretative phenomenological analysis.
NICE guideline [NG193] (2021). Chronic pain (primary and secondary) in over 16s: assessment of all chronic pain and management of chronic primary pain
Pillai Riddell, R. R., Racine, N. M., Turcotte, K., Uman, L. S., Horton, R. E., Osmun, L. D., Ahola, S. K., Stuart, J. H., Stevens, B., & Gerwitz‐Stern, A. (2012). Cochrane Review: Non‐pharmacological management of infant and young child procedural pain. Evidence‐Based Child Health: A Cochrane Review Journal, 7(6), 1905-2121.
Popert S, Riat H, Hodges EP. (2017). Can virtual reality (vr) guided meditation reduce pain? a feasibility and acceptability study BMJ Supportive & Palliative Care 2017;7:A22. spcare.bmj.com/content/7/Suppl_2/A22.2
Richardson P, Mustard L. (2009). The management of pain in the burns unit. Burns. Nov;35(7):921-36. doi: 10.1016/j.burns.
Stoddard FJ, Ronfeldt H, Kagan J, Drake JE, Snidman N, Murphy JM, Saxe G, Burns J, Sheridan RL.(2006) Young burned children: the course of acute stress and physiological and behavioral responses. Am J Psychiatry. pubmed.ncbi.nlm.nih.gov/16741210 The British Pain Society. (2014). Useful Definitions and Glossary
www.britishpainsociety.org/ people-with-pain/ useful-definitions-and-glossary
Tong, Xin & Gromala, Diane & Choo, Amber & Amin, Ashfaq & Shaw, Christopher. (2015). The Virtual Meditative Walk: An Immersive Virtual Environment for Pain Self-modulation Through Mindfulness-Based Stress Reduction Meditation. 9179. 388-397. 10.1007/978-3-319-21067-4_40.
Chen YP, Lee SY, Howard AM. (2014) Effect of virtual reality on upper extremity function in children with cerebral palsy: a meta-analysis. Pediatr Phys Ther. Fall;26(3):289-300. doi:
PMID: 24819682. Deutsch JE, Borbely M, Filler J, Huhn K, Guarrera-Bowlby P. (2008) Use of a low-cost, commercially available gaming console (Wii) for rehabilitation of an adolescent with cerebral palsy. Physical Therapy.;88(10):1196-207.
Garrett B, Taverner T, Masinde W, Gromala D, Shaw C, Negraeff M. (2014) A rapid evidence assessment of immersive virtual reality as an adjunct therapy in acute pain management in clinical practice. The Clinical Journal of Pain.;30:1089-98.
Gerber, C. N., Kunz, B., & van Hedel, H. J. (2016). Preparing a neuropediatric upper limb exergame rehabilitation system for home-use: a feasibility study. Journal of neuroengineering and rehabilitation, 13(33), 1-12.
Hayward, K.S., Kramer, S.F., Thijs, V. et al. (2019) A systematic review protocol of timing, efficacy and cost effectiveness of upper limb therapy for motor recovery post-stroke. Syst Rev 8, 187
Henderson A, Korner-Bitensky N, Levin M. (2007) Virtual reality in stroke rehabilitation: a systematic review of its effectiveness for upper limb motor recovery. Topics in stroke rehabilitation.14(2):52-61.
Ho, E.S., Campos, A.A., Klar K., Davidge K. (2018) Evaluation of pain in pediatric upper extremity conditions. Journal of Hand Therapy.; 31: 206-14. Doi:10.1016/j.jht.2018.02.004. pubmed.ncbi.nlm.nih.gov/17517575
Jannink MJ, Van der Wilden GJ, Navis DW, Visser G, Gussinklo J, Ijzerman M. (2008) A low-cost video game applied for training of upper extremity function in children with cerebral palsy: a pilot study. Cyberpsychology & Behavior.;11(1):27-32.
Jolly K, Taylor R, Lip G, Greenfield S, Raftery J. (2007) The Birmingham Rehabilitation Uptake Maximisation Study (BRUM). Home-based compared with hospital-based cardiac rehabilitation in a multi-ethnic population: cost-effectiveness and patient adherence. Health Technol Assess;11(35).
Kiper, P., Szczudlik, A., Agostini, M., Opara, J., Nowobilski, R., Ventura, L., Tonin, P., & Turolla, A. (2018). Virtual reality for upper limb rehabilitation in subacute and chronic stroke: a randomized controlled trial. Archives of physical medicine and rehabilitation, 99(5), 834-842.
Laver KE, Lange B, George S, Deutsch JE, Saposnik G, Crotty M. (2017) Virtual reality for stroke rehabilitation. Cochrane database of systematic reviews.(11).
Lohse KR, Hilderman CG, Cheung KL, Tatla S, Van der Loos HM. (2014) Virtual reality therapy for adults post-stroke: a systematic review and meta-analysis exploring virtual environments and commercial games in therapy. PloS one. Mar 28;9(3):e93318.
Neilson AR, Jones GT, Macfarlane GJ, Walker-Bone K, Burton K, Heine PJ, McCabe CS, McConnachie A, Palmer KT, Coggon D, McNamee P. (2019) Cost-utility of maintained physical activity and physiotherapy in the management of distal arm pain: an economic evaluation of data from a randomized controlled trial.) Fam Pract. Mar 20;36(2):179-186. doi: 10.1093/fampra/cmy047. PMID: 29878103; PMCID: PMC6425461.
Phelan, I., Furness, P. J., Fehily, O., Thompson, A. R., Babiker, N. T., Lamb, M. A., & Lindley, S. A. (2019). A mixed-methods investigation into the acceptability, usability, and perceived effectiveness of active and passive virtual reality scenarios in managing pain under experimental conditions. Journal of Burn Care & Research, 40(1), 85-90. doi.org/10.1093/jbcr/iry052
Pillai Riddell, R. R., Racine, N. M., Gennis, H. G., Turcotte, K., Uman, L. S., Horton, R. E., Ahola Kohut, S., Hillgrove Stuart, J., Stevens, B., & Lisi, D. M. (2015). Non-pharmacological management of infant and young child procedural pain. The Cochrane database of systematic reviews, 2015(12), CD006275.
Piron L, Cenni F, Tonin P, Dam M. (2001) Virtual reality as an assessment tool for arm motor deficits after brain lesions. Studies in Health Technology and Informatics.;81:386-392
Salawu, A,; Green, A.; Crooks, M.G. Brixey, N., Ross, D.H.. Sivan, M., (2020). A Proposal for Multidisciplinary Tele-Rehabilitation in the Assessment and Rehabilitation of COVID-19 Survivors, Int. J. Environ. Res. Public Health 17, no. 13: 4890.
Sandlund M, McDonough S, Häger‐Ross CH. (2009) Interactive computer play in rehabilitation of children with sensorimotor disorders: a systematic review. Developmental Medicine & Child Neurology.;51(3):173-9.
Saposnik G, Levin M. (2011) Virtual reality in stroke rehabilitation: a meta-analysis and implications for clinicians. Stroke.;42(5):1380-6.
Sharan D, Ajeesh PS, Rameshkumar R, Mathankumar M, Paulina RJ, Manjula M. (2012) Virtual reality based therapy for post operative rehabilitation of children with cerebral palsy. Work.; 4: 3612-5. Doi:
Sun H. (2012) Exergaming impact on physical activity and interest in elementary school children. Research Quarterly for Exercise and Sport.; 83: 212-20. Doi:
Threapleton, K., Drummond, A., & Standen, P. (2016). Virtual rehabilitation: What are the practical barriers for home-based research?. Digital health, 2, 2055207616641302.
Bowyer, M. W., Streete, K. A., Muniz, G. M., & Liu, A. V. (2008). Immersive Virtual Environments for Medical Training. Seminars in Colon and Rectal Surgery, 19(2), 90-97.
Kyaw, B. M., Saxena, N., Posadzki, P., Vseteckova, J., Nikolaou, C. K., George, P. P., Divakar, U., Masiello, I., Kononowicz, A. A., Zary, N., & Tudor Car, L. (2019). Virtual Reality for Health Professions Education: Systematic Review and Meta-Analysis by the Digital Health Education Collaboration. Journal of Medical Internet Research, 21(1), e12959.
Logeswaran, A., Munsch, C., Chong, Y. J., Ralph, N., & McCrossnan, J. (2020). The role of extended reality technology in healthcare education: Towards a learner-centred approach. Future Healthc J.
Ashmore J, Di Pietro J, Williams K, Stokes E, Symons A, Smith M, Clegg L, McGrath C (2019)A Free Virtual Reality Experience to Prepare Pediatric Patients for Magnetic Resonance Imaging: Cross-Sectional Questionnaire Study, JMIR Pediatr Parent 2019;2(1):e11684
Bradley, R & Newbutt, N. (2018). Autism and Virtual Reality Head-Mounted Displays: A State of the Art Systematic Review. Journal of Enabling Technologies. 12 (3): 101-113.
Bossenbroek, R., Wols A., Weerdmeester, J., Lichtwarck-Aschoff, A., van Rooij, M.J.W. & Granic, I. (2019). Efficacy of a virtual Reality biofeedback game (DEEP) to reduce anxiety and disruptive classroom behaviour: A single-case study.
Gold, J. I., Annick, E. T., Lane, A. S., Ho, K., Marty, R. T., & Espinoza, J. C. (2021). Doc McStuffins: Doctor for a Day" Virtual Reality (DocVR) for Pediatric Preoperative Anxiety and Satisfaction-A Feasibility Study. Journal of Medical Internet Research.
House, P. M. et al. (2020) ‘Use of the mixed reality tool “VSI Patient Education” for more comprehensible and imaginable patient educations before epilepsy surgery and stereotactic implantation of DBS or stereo-EEG electrodes’, Epilepsy Research, 159, p. 106247.
Koo CH, Park JW, Ryu JH, Han SH. The Effect of Virtual Reality on Preoperative Anxiety: A Meta-Analysis of Randomized Controlled Trials. J Clin Med. 2020;9(10):3151. Published 2020 Sep 29. doi:
Migoya-Borja, M. et al. (no date) Feasibility of a Virtual Reality-Based Psychoeducational Tool (VRight) for Depressive Patients, p. 7.
Newbutt, N., Schmidt, M., Riva, G. and Schmidt, C. (2020), The possibility and importance of immersive technologies during COVID-19 for autistic people. Journal of Enabling Technologies. 14 (3): 187-199.
Newbutt, N., Bradley, R. & Conley, I. (2020). Using Virtual reality head-mounted display in schools for pupils on the autism spectrum; views, experiences and future directions. Cyberpsychology, Behavior, and Social Networking. 23 (1): 22-33.
Newbutt, N (2019). Assisting people with Autism Spectrum Disorder through technology. In: Encyclopedia of Education and Information Technologies. Eds. Tatnall, A. Springer. Switzerland.
Newbutt, N, Sung, C, Kuo, H-J, Leahy, M. J, Lin, C-C and Tong, B. (2016). Brief Review: The use of a virtual reality headset in autism populations, a pilot study of technology acceptance and immersion. Journal of Autism and Developmental Disorders. 46 (9): 3166–3176.
Schmidt, M., Newbutt, N., Schmidt, C. and Glaser, N. (2021). A process-model for minimizing adverse effects when using head mounted display-based virtual reality for individuals with autism. Frontiers in Virtual Reality, 2, 13.
Smith, M. J., Ginger, E. J., Wright, K., Wright, M. A., Taylor, J. L., Humm, L. B., ... & Fleming, M. F. (2014). Virtual reality job interview training in adults with autism spectrum disorder. Journal of autism and developmental disorders, 44(10), 2450-2463.
Spiegel, B. et al. (2019) ‘Virtual reality for management of pain in hospitalized patients: A randomized comparative effectiveness trial’, PloS One, 14(8), p. e0219115. doi: 10.1371/journal.pone.0219115.
Yang, J.-H. et al. (2019) ‘Effects of Preoperative Virtual Reality Magnetic Resonance Imaging on Preoperative Anxiety in Patients Undergoing Arthroscopic Knee Surgery: A Randomized Controlled Study’, Arthroscopy: The Journal of Arthroscopic & Related Surgery: Official Publication of the Arthroscopy Association of North America and the International Arthroscopy Association, 35(8), pp. 2394–2399. doi:
Blumstein, G., Zukotynski, B., Cevallos, N., Ishmael, C., Zoller, S., Burke, Z., ... & SooHoo, N. F. (2020). Randomized trial of a virtual reality tool to teach surgical technique for tibial shaft fracture intramedullary nailing. Journal of surgical education, 77(4), 969-977.Bosse, H.M., Nickel, M., Huwendiek, S. et al. (2015). Cost-effectiveness of peer role play and standardized patients in undergraduate communication training. BMC Medical Education, Vol 15, No. 183.
Corregidor-Sánchez, A.I., Segura-Fragoso, A., Rodríguez-Hernández, M. et al. (2021). Effectiveness of virtual reality technology on functional mobility of older adults: systematic review and meta-analysis, Age and Ageing, Volume 50, Issue 2, March 2021, pp. 370–379,
Delshad, S.D., Almario, C.V., Fuller, G. et al. (2018). Economic analysis of implementing virtual reality therapy for pain among hospitalized patients. npj Digital Medicine, Vol. 1, No. 22.
Ditzel, L., Collins, E. (2021). Holograms in nursing education: Results of an exploratory study. Journal of Nursing Education and Practice, Vol. 11, No. 8.
Freeman, D., Haselton, P., Freeman, J., Spanlang, B., Kishore, S., Albery, E., ... & Nickless, A. (2018). Automated psychological therapy using immersive virtual reality for treatment of fear of heights: a single-blind, parallel-group, randomised controlled trial. The Lancet Psychiatry, 5(8), 625-632.
Gillies, M.A., Wijeysundera, D.N., Harrison, E.M. et al. (2018). Counting the cost of cancelled surgery: a system wide approach is needed. British Journal of Anaesthesia, Vol. 122, No. 2. Issue 2, pp. 691–694.
Halfer, D., Rosenheck, M. (2014). Virtual Education: Is It Effective for Preparing Nurses for a Hospital Move? The Journal of Nursing Administration, Vol. 44. No. 10 pp. 535-540.
Houchen-Wolloff, L., Steiner, M. C. (2020) Pulmonary rehabilitation at a time of social distancing: prime time for tele-rehabilitation? Thorax, Vol. 75, No. 6, pp. 446-447.
Lloréns R, Noé E, Colomer C, Alcañiz M. (2015). Effectiveness, usability, and cost-benefit of a virtual reality-based telerehabilitation program for balance recovery after stroke: a randomized controlled trial. Arch Phys Med Rehabil. Vol. 96, No. 3. pp. 418-425.e2.
Ng, Y.-L., Song, Y., Kwon, K.H. et al. (2020) Toward an integrative model for online incivility research: A review and synthesis of empirical studies on the antecedents and consequences of uncivil discussions online. Telematics and InformaticsVolume 47, 101323.
Portelli, M., Bianco, S. F., Bezzina, T., & Abela, J. E. (2020). Virtual reality training compared with apprenticeship training in laparoscopic surgery: a meta-analysis. The Annals of The Royal College of Surgeons of England, 102(9), 672-684.
Pot-Kolder, R., Veling, W., Geraets, C. et al. (2020) Cost-Effectiveness of Virtual Reality Cognitive Behavioral Therapy for Psychosis: Health-Economic Evaluation Within a Randomized Controlled Trial. Journal of Medical Internet Research. Vol 22, No. 5., e17098.
Walters, W., Collins,T., Evans, A. (2020). Additional cost in personal protective equipment to NHS services during COVID-19: a review of the expense incurred by Aneurin Bevan Healthboards Maxillofacial unit over a one-year period to meet current clinical guidelines, British Journal of Oral and Maxillofacial Surgery,Vol. 58, No. 10, e320-e322. doi.org/10.1016/j.bjoms.2020.08.072
Winser, S.J., Chan, H.T.F., Ho L., et al. (2020). Dosage for cost-effective exercise-based falls prevention programs for older people: A systematic review of economicevaluations. Annals of Physical and Rehabilitation Medicine. Vol. 63, No. 1. pp. 69-80.
Wong, D.J.N., Harris, S.K., Moonesinghe, S.R. et al. (2018). Cancelled operations: a 7-day cohort study of planned adult inpatient surgery in 245 UK National Health Service hospitals, British Journal of Anaesthesia, Vol. 121, No. 4, pp. 730-738.
Wolitzky-Taylor KB, Horowitz JD, Powers MB, Telch MJ. Psychological approaches in the treatment of specific phobias: a meta-analysis. Clin Psychol Rev 2008; 28: 1021–37.