June 5th, 2019, Published in Articles: PositionIT, Featured: PositionIT

by Ross Adams and Darren Cohen, Business Science Corporation

Virtual reality (VR) has many business applications, particularly as a tool for improving training. This article is based on the experience of the authors in building such VR training applications and explores typical use cases and considerations for several industries, most notably training solutions.

Virtual reality (VR) consists of a computer-generated simulation of a 3D environment that can be interacted with in a seemingly real way by a person using special VR equipment (see Fig. 1). This equipment consists primarily of a head mounted display (HMD), allowing the user to look around a 360° virtual environment. Most modern VR setups also come a hand controllers that can be used to interact with elements in the virtual environment.

Terms such as AR, MR, VR, XR are often used interchangeably to describe a range of virtual interactions, and it is worth distinguishing between them to avoid confusion when investigating their business applications and benefits.

Extended reality (XR) is an umbrella term used to refer to all real-and-virtual environments generated by computer graphics and wearables (HMDs), “x” denoting variability [2].

The best approach to understanding all the other Rs/realities is with the Milgram Mixed Reality Spectrum (see Fig. 2). On the one end sits pure VR, encompassing all virtual immersive experiences. These can be created using purely real-world content (such as 360° video or images), using purely computer-generated images (CGI) or objects, or a hybrid of both. Close to the other end (the real world) sits augmented reality (AR), which is an overlay of computer-generated content onto the real world. The most well-known example of AR is mobile phone game Pokémon GO, where users hunt around in the real world using their phone to capture classic Pokémon that are generated at real-world locations.

In-between AR and VR are various applications broadly categorised as mixed reality (MR). One MR example is BAE’s use the Microsoft HoloLens, in which the company used PTC’s Thingworx Studio software to create a step-by-step training solution for the HoloLens device to teach workers how to assemble a green energy bus battery (see video of it here: https://youtu.be/TfgfF9-1xvA). Using these tools, BAE can now create these guides for first time workers in just hours at a tenth of the cost, training new people 30% to 40% more efficiently.

AR and MR solutions for business applications are still in their infancy and have lagged behind VR business solutions. There have however been several recent developments such as the Magic Leap One. This high-end AR/MR headset allows for more interactive and hyper realistic experiences, making it only a matter of time (as new content is developed), before devices like this will become standard for high-end business applications.

Microsoft’s HoloLens, first released in 2016, was one of the first high-end AR devices for business applications. It limited field of view proved to be a barrier for many applications, but has been improved in the new HoloLens 2. Its benefit to businesses is its integration with other Microsoft services which many companies already use, making it easier to fit such AR solutions into one’s business operations.

Although new high-end devices can greatly increase the quality and levels of interactions of MR applications, the largest market will most likely remain consumer applications for mobile devices given their broad userbase. Consumer applications can also benefit business, be it for marketing or training.

Why should businesses use VR?

Apart from the benefits that VR presents in the marketing space, VR is poised to change the way organisations train personnel. It provides an immersive medium through which complex concepts can be communicated. Because VR is fully immersive, it offers a holistic experience, and appeals to the senses, the mind and, importantly, emotions. Simply put, this technology moves people, thereby enhancing both key concept retention and evoking behavioural changes.

Traditional classroom-based training is not always effectively for various reasons, including:

• Research has shown that the retention of knowledge from standard classroom training is not much better than 20% for key concepts over the long term.
• Often, the agenda of traditional training focusses on high-end capabilities which are far removed from the norm and conditions of the employee’s actual working environment.
• Participants are not afforded the opportunity of experiencing real life conditions that are risky without incurring high costs and potential risks to personnel and equipment.
• Learning is not always presented in the participant’s language of choice.

VR increases engagement and understanding. Research has shown that 30% of the human brain is designed to process information visually. Moreover, our optic nerve system is able to process approximately 1 MB of information per second. This “bandwidth” is more than 100x faster than necessary for reviewing information traditionally represented as numbers or charts.

Various visualisation techniques can leverage this ability efficiently. VR allows us to take full advantage of this bandwidth and biological processing power, by fully immersing an individual in a virtual space. This increases the level of engagement with the subject matter and enhancing their capacity to intuitively understand it [4].

VR has only recently been developed to the point where it is cost effective and of high enough quality to be used in a variety of business applications. This opens new opportunities to use VR in key business functions.

VR environments allow users to practice and experience day-to-day activities and processes of their working environment that goes beyond what class room training can achieve. VR not only mimics real operational conditions with a high degree of fidelity, but also recreates experiences that are unsafe or impractical to be exposed to, thus preparing employees for hazardous situations in a safe and effective way.

VR then and now

VR started as far back as the late 1970s, with specialised hardware and software being created at research laboratories such as NASA’s Jet Propulsion Laboratory. Wide scale adoption, however, was held back by graphics hardware and processing power that most people had access to at the time. Increased computing performance eventually lead to arguably the first high-end VR headset, the Oculus DK1 (Development Kit 1), being released  in March 2013. The Oculus DK 2 followed in March 2014, and then came the consumer version, the Oculus Rift, in March 2016. That same year HTC also released its high-end Vive, which came with full-room scale tracking.

The gaming industry has always been the largest consumers of VR, and with the release of the Oculus Rift and HTC Vive, gamers (who generally own high-performance PCs) could use these devices to play VR games.

The first PCs powerful enough to use these devices required at minimum a GeForce GTX 970 graphics card. The first VR business application Business Science Corporation (BSC) created was a visualisation of a discrete event simulation of Anglo Platinum’s Mogalakwena mine for the Anglo-American Centre for Experiential Learning using the Oculus DK 2, in late 2015 (see Fig. 3).

The release of the Google Cardboard, in June 2014, was a great low-end device (costing around $10) that allowed anyone with a smartphone to view VR content such as 360° photos or images. This greatly expanded the reach of VR and was most people’s introduction to VR. Innovations in VR has also driven costs down significantly. Where the Vive at first cost $800, it can now be purchased for $500.

Today there are various VR headsets, from the Samsung Gear VR, Samsung Odyssey, PlayStation VR and Google Day Dream, to the Oculus Go, Vive Focus, and various Pimax headsets. Recently announced headsets include the Valve Index, the Oculus Rift S (an upgrade to the Rift), the Vive Cosmos, and the Oculus Quest. Each has its own pros and cons, depending on the use case, the design of the application, and cost.

Choosing a VR headset

High-end VR devices such as the HTC Vive (Pro), Oculus Rift, and Samsung Odyssey require a high spec PC which will cost somewhere in the range of $1500 to $3000. This can add significantly to the cost of setting up a VR solution, especially where multiple headsets are required. However, these costs are generally manageable for larger businesses since it remains far less expensive than bespoke content development. Applications that are developed for these platforms are generally of high quality and very interactive. But active participants are limited since only one person can use the application at a time. This can be mitigated by allowing other participants to passively view live content of the experience from an external monitor.

High-end devices also have considerations around physical set up in that they generally need to be tethered to a computer, and the cables can create an awkward experience, especially for users who are new to the technology. There are devices which remove the need for tethering by wirelessly connecting the headset to the computer, the Vive Wireless Adapter (for the Vive and Vive Pro) and the TPCast (for the Vive and Rift) being the most popular.

In our experience, the best platform for high-end enterprise solutions has been the HTC Vive and Vive Pro. The Vive Pro is an upgraded version of the Vive, and is designed with businesses in mind, including warranties and service agreements. The Vive was the first high-end VR platform that was designed to be used with room scale tracking. Room scale tracking allows for experiences where users can physically walk around the virtual environment with the hand controllers allowing users to interact with objects in the virtual environment.

The Oculus Rift is an equivalently good headset. However, the Rift was not initially designed for room scale tracking. So, although the position of your head in 3D space is tracked, allowing you to move and walk around, it is a lot more difficult to track the same size rooms. It is possible to create larger tracking spaces by using multiple rift cameras (the cameras are used to track the headset and controllers), but this is not something directly supported or endorsed by Oculus and can come with its own challenges.

Creating a mine safety training solution

A great example of some of the pros and cons of these headsets can be best explained in a case study of a safety training solution that the team from Business Science Corporation developed for a deep hard rock mine. This example highlights some of the typical considerations and trade-offs that must be made to address technical hardware limitations.

The purpose of the solution is to train miners on the process of Manual Barring. This is a common safety exercise that all underground personnel should know, and involves a long steel pole (pinch bar) being used to dislodge loose rock in a section of a mine tunnel after blasting. It is an important safety process as loose rocks can fall and damage equipment or seriously injure people.

The solution is mixed-reality safety training programme that went further than a pure VR solution. It incorporates the tracking of the physical pinch bar and the mapping of the virtual environment to a custom-built physical container (see Fig. 5). This lets users not only experience the underground mine virtually, but also trains them with the real pinch bar to provide the right amount of force and feedback when striking it against the sidewalls and hanging wall. The container is designed with specific physical blocks that are positioned intentionally to represent common geological features found underground that would typically need to be barred down. These blocks are held in place with magnetic drop plates which are controlled and released from the application through an Arduino controller board. The physical pinch bar is tracked using the VR system and is also shown to the user virtually in the VR environment.

As exciting as this application is, it did come with a multitude of considerations and trade-offs that were implemented and tested to produce the most realistic, or highest fidelity experience possible. Through testing and user feedback, it became apparent that certain approaches were more optimal than others.

Initial approach

The initial approach used the HTC Vive as the solution required room scale tracking for the user to move around the environment and bar down several different geological features. While this works well for the individual when walking around and experiencing the sights and sounds of the virtual underground environment, it proved difficult to track the pinch bar in space.

The Vive setup allows additional devices (Vive Trackers) to track additional real-world objects, and generally the tracking accuracy is very high – the manufacturer claims nanometre accuracy. The headset, controllers, and trackers connect to two base stations which form a laser-based positioning system that constantly emits invisible light. Each of the tracked devices have numerous photosensors built into them, allowing the system to calculate the position and orientation of each device. This tracking system however has difficulty tracking devices occluded from the base stations, and these devices temporarily lose tracking within the VR application. To avoid this, the tracked devices require constant line of sight to the base stations.

To track the pinch bar, it meant placing the tracker high up the pinch bar so that the user’s body would not block its line of sight to the base stations. This introduced further problems, since the trackers do not perform well under the constant shock and sudden acceleration or deceleration that is common in the process of manual barring. It results in the system losing tracking of the pinch bar when it hit the wall hard (a requirement of the training application) and thus the virtual pinch bar would float off into the distance in the VR environment. It also creates a disconcerting experiencing for the user, as the action does not reflect reality.

Second attempt

The problems of the first approach was overcome by implementing the solution on the Oculus Rift, which uses a camera-based tracking system rather than laser-based base stations. This worked much better for tracking the pinch bar as the camera system could visually determine where the object was, even under high decelerations and forces. But the Oculus was not designed for room scale tracking. This required drastically decreasing the physical area in which the user could walk around. Even with the addition of further tracking cameras, the limitation of the area within which the trainee could move proved to be a barrier for a successful training experience.

A return to the original approach

Luckily the Vive Pro with and upgraded tracking system was released during the development phase of the solution, and so the final approach was to implement the solution on this new headset, which has proven to be a stable option for manual barring.

The added support for more base stations meant the team could return to the original approach, which was better expect for tracking the bar. However, this time we could attach the tracking device lower down on the pinch bar (between the user’s hands) where it experiences substantially less force and much of the shock is absorbed by the user. The issue of occlusion was resolved with the installation of four base station units, allowing the attached tracker to always be visible to at least one base station.

It is important to note that the software codebase developed originally remained the same across all the different hardware implementations, with only minor code changes required to accommodate the different features of the hardware.

Trade-offs and considerations

High-end solutions are not always the most cost-efficient or even practical to set up for training and business applications which aim to train many employees at a time. In the last few years there have been several devices released that attempt to solve this problem, including the Oculus Go and Vive Focus. They do not need to be attached to a computer (all the processing is done onboard the headset) and do not require base stations or cameras for tracking.

While they have their advantages, these devices are a lot more limited in terms of the fidelity of the experience they can provide compared to their high-end counterparts. The visual quality is noticeably lower as the application does not run on a powerful PC, and the level of interaction is far less. Both the Go and Focus have a single controllers with three degrees of freedom tracking (i.e. pitch, yaw, and roll). This means that you can track the orientation of the headset and controllers (i.e. able to look around) but you cannot track the position of objects. This works well for 360° videos or content where the user does not need to look around too much. You can also not use the controller to “physically” pick up and move objects, but it is effective as a laser pointer that the user can use to interact with elements in the virtual environment. Another benefit of these devices is that they do not require large areas (such as the Vive), which means you can have ten to 20 people in single class all using their own headset.

The Oculus Go costs around $200, and since you do not need a computer, this is a much more cost-efficient option for scaling VR training applications, assuming your use case does not require a high degree of physical interaction and positional tracking.

One of the most exciting new hardware releases is the Oculus Quest, which is a stand-alone headset (like the Go) but with six degrees of freedom tracking (i.e. forward/back, up/down, left/right, yaw, pitch, roll) and a much higher resolution screen. This means that the visual fidelity will be much better than the Go, although still lower than the Rift S or Vive as it still needs to do all its own processing. The 6-DOF tracking means that the position of your headset and controller are tracked, allowing you to move your hands around to interact more realistically with objects in the virtual environment. The Quest also uses inside-out tracking like the Samsung Odyssey, allowing room scale tracking without the need for external base stations or cameras, as it uses multiple cameras built into the headset.

The Oculus Quest combines the best of both worlds and is a good example of how VR technology is evolving to solve the various challenges users have faced in adopting the technology.

Which industries are using VR?

At Business Science Corporation we have been developing enterprise virtual reality solutions for the past four years. We have developed solutions for architecture firms, banks, telecommunication companies, and mines. Our main focus, and where we believe we can have the most meaningful impact, is in creating safety training applications for the mining industry.

In the training and education space there are numerous VR applications for a wide range of industries. Pixvana, for example, created a VR application for training waiting staff on the ultra-luxury cruise liner Seaborne, using the Oculus Go. There are also several applications for the training of doctors and nurses. Precision OS, for example, is an orthopaedic surgery VR company that has built an advanced system for training doctors to perform surgeries.

The architecture, engineering and construction (AEC) sector and the automotive industry remain big VR markets (see for example https://youtu.be/iCKx8LidOdM). Here VR allows teams to collaboratively design in a more realistic environment using tools such as the Nvidia Holodeck. This allows teams to quickly experiment with new design ideas and construction approaches.

Audi was one of the early adopters of VR, using it primarily as a sales tool, but also as a training and design tool. They have now deployed over a 1000 VR showrooms in dealerships worldwide.

A quick search on Google reveals a long list of VR applications, and the list of companies taking advantage of VR is growing exponentially. Immersive technologies for all areas of life are at an early but exciting stage and the applications that can be built are limited only by your imagination.

References

[1] S. Hayden, “Owlchemy Labs Teases New In-Engine Mixed Reality Tech”, Road to VR, 04-Oct-2016. [Online]. Available: https://www.roadtovr.com/owlchemy-labs-teases-new-engine-mixed-reality/. [Accessed: 05-Jun-2019].
[2] K. Irvine, F. D. A. Category: #Design, C. P. on October 31, and 2017., “XR: VR, AR, MR—What’s the Difference?” Viget, https://www.viget.com. [Online]. Available: https://www.viget.com/articles/xr-vr-ar-mr-whats-the-difference/. [Accessed: 05-Jun-2019].
[3] R. Steiger, “The Ultimate Guide for XR Evangelists”, A Pattern Emerges, 05-Sep-2017.
[4] Zimmerli, Lukas et al., Increasing Patient Engagement During Virtual Reality-Based Motor Rehabilitation, Archives of Physical Medicine and Rehabilitation , Volume 94 , Issue 9 , 1737 – 1746. Berka et al., EEG Correlates of Task Engagement and Mental Workload in Vigilance, Learning, and Memory Tasks, Aviation, Space, and Environmental Medicine, Vol. 78, No. 5, Section II, May 2007.
[5] “VIVE Wireless Adapter”. [Online]. Available: https://www.vive.com/eu/wireless-adapter/. [Accessed: 05-Jun-2019].

Contact Darren Cohen, Business Science Corporation, dcohen@bscglobal.com

Related Articles

South African Government COVID-19 Corona Virus Resource Portal

Now Media acquires EngineerIT and Energize from EE Publishers

Latest GIS further supports electric and gas networks

EE Publishers 2020 Year Planners

To readers, customers, suppliers, staff and other stakeholders of PositionIT