EXECUTIVE SUMMARY
AR is a ‘reality’ right now, costs can be managed effectively, technology and infrastructure exists to drive the implementation of a stable, reliable and functional solution which makes a STEP CHANGE improvement in organisational performance with full ROI.
At this stage in the market development life cycle, due to the relatively limited amount of expertise available, it is vital to collaborate with the best partners, managed by a fully informed, education technology partner.
Future Talent Training has created an AR ecosystem specifically designed for the L&D sector. This includes:
Strategic overview of your organisational readiness for AR, it’s direct and indirect benefits, timescales, investment and full ROI analysis.
With direct Education and Training sector expertise, we provide you with a balanced, independent analysis of the benefits and risks associated with your AR project.
Creating the right AR partner supply chain, FTT connects you to the best in class hardware and software development expertise.
AR project management. Working either in collaboration with you or independently, FTT manages the process, critical success criteria, timelines and budget for your AR project.
Core Questions
What is augmented/assisted reality (AR) and how does it fit with L&D?
Which sub sectors of L&D currently use AR and why?
What are the AR training applications and limits based on existing technology?
What are the main, scalable L&D opportunities within the next 12-24 months?
What are the main, scalable L&D opportunities within the next 24-36 months?
SO WHAT IS AUGMENTED REALITY?
AR remains one of the biggest technology trends globally, and this will only increase as AR ready smartphones and other devices become more accessible around the world.
The AR Story so far...
1968: Ivan Sutherland created the first head mounted display, called The Sword of Damocles. It paved the way for the AR we use today.
1990: The term augmented reality was coined by Boeing researcher Tom Caudell.
2000: AR Quake launched the first AR game. As well as a head-mounted display, players had to wear a backpack containing a computer and gyroscopes.
2008: BMW was the first brand to make use of AR for commercial purposes, with its AR enhanced print ads.
‘Pre Covid-19 the AR industry was predicted to hit global revenues of £80 billion by the end of 2020’
In essence AR is a powerful visualisation tool that can help us transform our immediate surroundings into learning, work and entertainment spaces. AR enables digital information to be superimposed and integrated into our physical environment. It allows for every type of digital information - videos, photos, links, games etc. to be displayed on top of real world items when viewed through the lens of a mobile or wearable device.
There are 3 ways to visually present AR:
Visual see-through: is the primary method of creating an augmented reality view. This is the design Sutherland developed in the early 1960s, a see-through lens such as glasses or helmet’s faceplate) which leaves the user’s perception of the real world unmodified and display the information or/and graphics augmented reality as an overlay by means of transparent displays, or mirrors and lenses or miniature projectors
Obstructed view: where the user wears a head-mounted display (HMD) that blocks the real world, and displays in the HMD are fed a view of the world from a front facing camera in the HMD. The augmented information is overlaid into the video feed. This technique restricts the field of view of the user and can restrict it to just a flat 2D view if only one camera is used.
Projected augmented reality: is a technique where the augmented reality overlay of information or graphics, is projected from the headset or HMD out onto the real world and objects within it resulting in projective displays. The three techniques may be applied at varying distances from the viewer: head-mounted, hand-held and spatial.
Augmented reality should not be confused with virtual reality: Both these technologies share similar problems around mobility and power consumption but Augmented Reality is a real-time view of information overlaid on a view of the real world, while by contrast, virtual reality replaces the real world with a simulated one.
AR’s FIT WITHIN THE L&D SECTOR
There is no doubt we are witnessing a hugely significant and market changing period within the UK and indeed global L&D sector. Although the L&D sector are often seen as ‘late adopters’ of groundbreaking new technology there is a clear determination and financial need by large and small organisations to take a leap of faith and invest in applications which will show true ROI in terms of financial outlay and productivity.
‘The way we perceive Learning and Development
is changing FAST.’
The culmination of various small but significant digital transformations which have entered the L&D market during the last 4-5 years has now created the right environment for a seismic change in way we perceive, develop, deliver and measure personal learning.
The Covid-19 pandemic has effectively forced the market to pivot towards technology already in place such as video conferencing, e-learning and other digital training methodology however the most important implication of this is the newfound 'market readiness' to accept innovative, digital technology as an enabler and not just a nice to have additional resource.
There have been two main blockers to a seismic change not occurring until now.
The technology sector itself has not helped in terms of getting new products into the market and established. Their lack of real market insight, over marketed features and benefits and inappropriate pricing strategy are but a few reasons why. This has led, in recent years, to the spectacular failure of some major launches.
The second is what I call ‘Market Ostrasization’ - The L&D sector burying its head in the sand in terms of technology advancement. Within the L&D community as a general rule, until very recently, Big Data, VR/AR, ML and AI have been in the 'very interesting but do I really need this?' box. It's incorrect to say things are changing. Things have CHANGED.
The UK L&D sector has a hugely important role to play within the successful resurgence of the UK economy and it also covers the entire, literal 'life cycle' of our human capital from birth to retirement and beyond.
What we are now witnessing, is mass adoption of next stage technological transformation. The speed of change and momentum which this will create is immeasurable. For our future talent, businesses, industry and country to remain successful, productive and competitive we must grab this opportunity with both hands.
SO WHY MIXED REALITY? AND WHY NOW?
Lets firstly take a look at the two main areas of this technology. 2D applications and 3D applications.
2D Applications:
This is where pre-programmed augmented or assisted information, training, data or live instruction can be fed directly into the line of sight of the user via an HMD such as Google Glass or Microsoft HoloLens.
The applications of this technology within the training sector have been evident for years however restrictions around development costs have prevented real innovation except within high value, technology rich areas such as leading edge surgery and high end manufacturing. Until recently, the apprentice mechanical engineer or construction manager could never have dreamed of using AR technology. This dream is now a reality.
With the rapid acceleration of big data analysis, machine learning and AI, the process of creating relatively low cost, highly quality AR applications for training purposes is now not only possible but vital for the future of lifelong personal and professional learning.
The beauty of this technology is that it ticks all the boxes in terms of learning effectiveness.
Speed of learning, Micro Learning, Intuitive Learning, 80/20/10 Rule, Automation, Remote Learning, On Demand, Cost Effective, Smart Learning
Let's look at a couple practical examples of what AR could now be used for within a training and development context.
Maintenance and Operations Engineering Technician (MOET) Apprenticeship
This standard has four core knowledge principals, one of which is:
'maintenance and operational practices, processes and procedures covering a range of plant and equipment'
Complimenting and assisting the assessor/skills coach and EPAO, AR gives the learner a unique opportunity to practice their understanding of this new knowledge within a safe, practical application in the workplace using remote instruction, guidance and direct (realtime) video and audio assistance. This is clearly useful within a post pandemic learning environment where some form of social distancing will remain in place for years ahead. The other bonus of this application is that one skills coach can provide live guidance for several learners at one time via multiple connection AR/VC.
Company Specific, QR Code - On Demand, Real Time Training
Where an employee has had, for example, logistics operations training but remains relatively inexperienced/aware of their company's process and procedures. Working within a medium hazardous environment and unsupervised becomes far less resource heavy with an AR application.
As the employee assesses a task the AR headset picks up a QR code with one simple tap on the side of the headset. Instant detailed instruction then becomes visible within their line of sight, specific to the task at hand. Also other functionalities such as 'voice recognition' and 'gesture director' can be integrated into the AR application giving a number of potential success criteria for the employee instantly improving performance and safety.
3D Applications:
The market for 3D augmented reality is massive, mainly consumer focused and remains in its infancy. Global players such as Facebook, Microsoft and Apple are all racing to market right now with their new generation product offerings. In essence this is where we see true augmentation to the reality in front of us.
These products offer visual experiences which autonomously pick up your surroundings/location when wearing the headset. The data is then used to give three dimensional images, data, sounds, video and audio giving 'useful' guidance, increasing gaming experience or adding value to your real life visual picture.
There are however huge technology, integration and security issues to combat as Google found to its cost a few years ago when its initial product launch aimed at the consumer market failed to get off the starting blocks due to security concerns.
Based on the existing L&D market readiness and budgets available within typical enterprise organisations which are currently being allocated to this type of project and other variable factors it is clear that the 2D format would fit the market as it stands now and for at least the next 24 months. Once 3D development costs reduce and hardware becomes more stable on a 5G network this may change.
SECTORS WHERE AR IS USED TODAY
Augmented and virtual reality are driving huge traction in the industries such as entertainment, healthcare training, learning and e-commerce. Reports found that across the automotive, manufacturing and utilities sectors, the most popular uses of AR (and VR) are repair and maintenance, and design and assembly.
Between 29% and 31% of companies using these technologies for repair and maintenance, are using them specifically to consult digital reference materials (31%), seek a remote expert (30%), digitally view components not in physical view (30%) and superimpose step-by-step instructions on workstations (29%).
For design and assembly, companies are using AR to view digital assembly instructions (28%), simulate product performance in extreme conditions (27%), visualise infrastructures from various angles (27%) and overlay design components onto existing modules (26%).
AR is then a prominent IT tool which has been used since in many fields:
Entertainment: AR has an obvious place in the world of gaming indeed perhaps the most famous example of AR technology is the mobile app Pokemon Go released in 2016.
Healthcare: AR applications designed to enhance healthcare, including Viipar, a video support platform for surgeons that functions via Google Glass.
Retail and Marketing: AR allows an environment in which a brand is positively enforced by the user’s own experience. Companies are increasingly exploiting VR and AR within sector events.
Travel and Tourism: local businesses have buildings in AR feature which gives the users information on local establishments when their smartphone is pointed at a particular location.
Automotive: AR is quite powerful in the automotive industry. Topics include using Augmented Reality wearables to enhance the driving experience, ‘see-through displays’ and also automotive design and production. AR is used to visualise the body structure and general design of cars, and to display information to drivers via their windscreens.
Industry/Manufacturing: some companies have created an application that helps warehouse workers to locate objects and sort packages. Ground crew at Singapore’s airport wear AR glasses to see information about cargo containers, speeding up loading times.
Fashion: it has been said that by 2020, 100 million consumers will shop in augmented reality. AR has massive potential to bridge the gap between online retail and in-store experiences, letting customers see what clothing actually looks like on a human body without having to be physically present in the shop.
Education: Since 2009, the British Museum has been using AR to help children understand the Parthenon gallery. Using Samsung tablets, young gallery-goers can play an AR game called ‘A Gift for Athena’, which uses statues from the museum’s collection to tell a story. Augmented technology has also been adopted in the classroom – a MOOC (massive open online course) has been released to help students and teachers explore the water cycle using AR.
Below are some examples of companies and providers who are leveraging AR technology:
Ikea: Created the Ikea Place App, a new solution to improve the consumer shopping experience: this allows users to select any product from the catalogue and actually put it inside your home.
Aecom: Uses augmented reality to improve the work of architects and engineers. Thanks to Microsoft HoloLens technology and Trimble software tools, it is possible to project 3D models of building projects and make them concretely visible in different places: AR is able to simplify the work of professionals by making visible design errors and improvements. The cities where Aecom mainly apply its technologies are London, Detroit and Hong Kong.
Microsoft: The company refers to itself as ‘the leader in mixed reality’. Mixed reality refers to the combined power of augmented and virtual reality when deployed in the real world. Users can access mixed reality content including games, 360° videos, social media, and live events.
NexTech AR Solutions: based in Canada, this company is a rapidly growing leader in the exploding AR industry, estimated to hit $120 billion by 2022, according to Statista. NexTech provides businesses with augmented reality solutions that help drive their bottom line and competitive advantage. Whether for product promotion, increased sales, product training or brand evangelism, NexTech AR is focusing on several key multi-billion-dollar verticals.
Up Skill: Based in the Washington DC, Up Skill are also leaders in their field using a multi functional platform ‘Skylight’ which provides companies with a tool kit to develop a unique AR solution utilising most major HMD’s
Magic Leap: this startup has expanded its partnership with AT&T to build solutions for businesses. The first three industries to benefit will be manufacturing, retail, and healthcare. AT&T will also provide the 5G connectivity needed to deliver high quality AR content.
Facebook: The company is working to optimize its Spark AR augmented reality app to add it into Instagram in order to create unique and engaging experiences that quickly lead users towards a purchase.
AR AS AN INNOVATIVE TRAINING SOLUTION AND ITS LIMITS
Training is a field in which Augmented Reality is used right now and is highly likely to be increasingly used over the next few years, for example as remote support but also and above all to carry out simulation activities.
Visual perception is the key to understanding, information transfer and memory. The cone of experience postulated by an American educator, Edgar Dale, shows the ways we tend to remember.
The report, “Augmented and Virtual Reality in Operations: A guide for investment” has found that 82% of companies currently implementing AR/VR say the benefits are either meeting or exceeding their expectations
Google Case Study
Google is experimenting with new ways of publishing, called "digital native books" which are an evolution of the e-Book designed to offer a new literary, interactive and easily usable form. These books are published directly online and become an interacting experience thanks to the possibility of associating interactive multimedia elements that can be used with AR visualization devices to the printed text. These types of books can also be multi-user since they can be used simultaneously by several students who can interact and communicate with each other.
Walmart Case Study
The experiential aspect of AR makes learning more intense and training more effective. Andy Trainor, senior director of Walmart US Academies, said: “When you look at a form through the viewer, you seem to have actually encountered that situation. We have also seen that training through augmented and virtual reality increases employee trust and loyalty."
Alongside the level of involvement and interaction, another advantage highlighted by many is the long-term economic savings which optimises the most conspicuous investment made in the short term.
Uqido Case Study
“Through AR, training is perceived as much more effective because we have recorded an improvement in learning times and costs» explains Pier Mattia Avesani founder and CEO of Uqido, a software house that has had a 300% growth in the 2018.
It has been studied as a user takes up to 3 times less to remember things with the help of XR technologies. Bringing concepts to life is one of the most effective methods of teaching something: by involving all our senses, it is possible to create complex environments and conditions otherwise difficult to reproduce. Also important is the significant reduction of the risks that particularly dangerous procedures would normally entail.
‘The Manufacturing Institute predicts that while nearly 10 million manufacturing jobs will be needed globally in the next decade, millions of these jobs will go unfilled.’
Exacerbating this problem is the fact that new workers will not have the level of expertise of those retiring from the workforce. This may leave the manufacturing and industrial sector with a far less productive workforce.
To stay competitive, industrial companies must connect their experts to new workforce in ways that minimise cognitive load while engaging learning systems in the brain. Cognitive science suggests that the most effective workforce training tools are those that naturally map to relevant learning and memory systems in the brain. This speeds time to productivity, trains subject matter and behavioral skills expertise, all while reducing training costs.
What is needed are methods for training the new workforce that instill the subject matter expertise and behavioral skills repertoire of the expert. This requires the development of tools that effectively curate, package and deliver that expertise to new workers in a time-effective, cost-effective and scalable manner that speeds the time to productivity, all while reducing training costs.
Generally, the learning takes place in a both separated time and location from the task to be learned.
The spatial separation means that experiential and behavioral skills learning systems in the brain that would be recruited should the learning take place while performing the task, will not be recruited. In short, tasks are most quickly and accurately learned through physical repetition than slowly and inaccurately through mental repetition.
Companies should then consider augmenting your current training solutions with solutions that are optimised for the way that the brain learns. The brain learns most effectively when knowledge is presented in the flow of work and broadly engages multiple learning and memory systems in the brain.
‘Augmented reality training tools can achieve these aims by capturing and curating the extensive knowledge of subject matter experts.’
LIMITATIONS OF AUGMENTED REALITY
A shortage of expertise and insufficient back-end infrastructures are significant barriers to growth. In addition, within some key global markets there is a resistance to AR adoption with those who are investing in the technology limiting its integration to a trial group. Post Covid -19 this is changing fast with new market led inquisitivity leading to many new applications.
Some other limitations related to the usage of AR are: development costs and enduring problems with hardware and software which make AR hard to develop and launch effectively.
The technological requirements of augmented reality are much greater than those for virtual reality, which is why the development of augmented reality took longer than that of virtual one. However, the key components needed to build an augmented reality system have remained the same since the 1960s: displays, trackers, graphics computers and software remain essential in many augmented reality experiences.
Technology:
GPU to drive display
display projection device to create images
optics to route the images to your field of view
sensors (forward-looking to see the world you are looking at, real-world position to map the world in 3D, motion sensor, eye sensors to track where you are looking)
audio systems (microphones, processing and speakers) for communications and augmentation of the real world
object identification and categorisation system that recognises what your glasses are looking at to position virtual images on top of or near them
an operating system to control the virtual images with voice, eyes, hands and body motions
wireless communication to a server-like-device
To design, build, manufacture and support augmented reality devices in house a company has to have an extensive range of experts: engineers, scientists, technicians, doctors, mathematicians and tech savvy managers.
They have to understand and know how to integrate: audio, camera and display technology; ergonomics and user interfaces, geometric and trigonometric mathematics, image processing techniques and processes, manufacturing engineering, optics and optometry, physiology, positional, tracking and location methodology, power management, processors, semiconductor technology and software engineering, operating system, drivers, computer graphics, game engines.
This has created a huge potential market for collaborative partnership in this process. Essentially there are 4 key stakeholders required for a successful AR learning solution which effectively increases productivity and reduces costs.
Client/Company - Hardware Provider - Software Specialist - Education Technology Partner
UK Level 4-7 Apprenticeship Training and Assessment
Maintenance and Operations Engineering Technician (MOET Standard)
Automotive Sector Training and Assessment Project
Construction and Engineering Assessment Project
Apprenticeship - End Point Assessment Organisation (EPAO) Project
Energy Sector Behavioural and Process Safety Training
Micro e-learning with AR functionality Project