Piloting Utility Use Cases For Augmented Reality Technology

A pilot program experimented with augmented reality (AR) technology to improve the efficiency of utility field visits. Beginning in 2018, the pilot aimed to identify whether AR wearable headsets could efficiently view and document site conditions and enable effective communications. While the pilot focused on the utility market, the effort aimed to address common challenges across many industries.

EMERGING AUGMENTED REALITY APPLICATIONS

By Zachary Wassenberg

Read The White Paper

VIRTUAL PRODUCTIVITY IMPROVEMENTS

Digital technologies are improving efficiencies among different industries and office environments but, when it comes to field and site visits, paper remains dominant. Relying on experienced personnel for site visits and documenting field findings in a notebook has been the traditional approach.

However, utilities can expect to lose about half of the current, experienced workforce to retirement over the next five to 10 years, according to Smart Cities Dive. Combined with the gradual introduction of new utility infrastructure technology and components, the knowledge needed for new workers is growing. It’s a challenge to address the retiring talent and expect in-the-field training for less experienced workers at every site, for every asset.

Technology can bridge knowledge gaps and improve productivity in the field. Burns & McDonnell set out to pilot how AR technology can increase the efficiency and effectiveness of utility field users.

AR TECHNOLOGY SELECTION

From the inception of the pilot to its kickoff about 16 months later, technology selection changed considerably. Many AR industry vendors are startup companies, which wasn’t considered a good fit, given a preference to use hardware and software vendors with a record of longevity. By attending industry conferences and talking with other AR utility leaders, including peers in Duke Energy’s Technology Development department, the Burns & McDonnell team determined the technology approach to test.

SOFTWARE SELECTION

After investigating more than 15 software vendors, the selection was narrowed to three choices. While all presented the software strengths and how the technology could be deployed, use cases and experience were heavily focused on the manufacturing and assembly industry. All of the vendors lacked experience in the utility industry at that time.

We established a list of AR software criteria that reflected the needs and constraints of the electric utility industry. These criteria continue to be used as the requirements in evaluating all new AR software:

  • Cannot create a safety hazard.
  • Must be easy to learn and use in the field.
  • Must be cost justified.
  • Must be able to download and upload information.
  • Must address a utility industry challenge.
  • Must be scalable.
  • Vendor must be expected to be in business long-term.

Working for four months using the AR software criteria, our team identified a viable vendor: Manitoba Hydro International. As part of a large electric utility company, MHI understood the industry, field issues, industry and user pain points, and adoption challenges.

MHI presented its software, VisualSpection, a comprehensive software solution designed specifically for the utility industry. After evaluation, the pilot team visited MHI to view capabilities and selected MHI as the pilot’s software vendor.

HARDWARE SELECTION

Choosing the right AR hands-free hardware solution was just as challenging. The team researched software and hardware in parallel, making a list of possible options. All vendors exhibited limitations, especially that none was a good fit for the utility industry and not all software vendors supported all of hardware options.

To help pinpoint vendors that would work well with the selected software and pilot needs, a list of criteria was developed to guide hardware selection:

  • Must be safe for field use, including PPE compliant, hands-free and ergonomic.
  • Must be field-ready with all-day battery life.
  • Manufacturer must be scalable, able to keep up with large orders, reasonably priced and expected to be in business long-term.

During the visit to MHI, the pilot group was presented with a new, not-yet-released AR hardware option. RealWear HMT-1® not only met every requirement, it exceeded many of them. HMT-1 was selected as the AR technology hardware option and is currently the only AR headset the team considers for field users.

VisualSpection Platform - Assets

 

FIGURE 1: VisualSpection software showing assets at a site.

USE CASE SELECTION

Choosing the proper use case to match selected software and hardware strengths, plus meet the project goals, was a critical part of the AR pilot. Use case selection would help determine if AR technology could improve field visit identification and documentation of assets, as well as improve communications within construction projects and fieldwork.

VISUALSPECTION GEOGRAPHIC INFORMATION SYSTEM

To better document as-built conditions, MHI developed the VisualSpection software to operate as a webbased geographic information system (GIS) application optimized for the Google Chrome browser. The pilot team created an organizational structure based on the location of the project site and the assets in the site.

Using the browser simplified the use of software and meant there was no need to maintain a new software program for the computer fleet. An example of the platform is shown in Figures 1 and 2.

WORKFLOWS FOR KNOWN ASSETS AND LOCATIONS

In addition to generating an organizational structure of a site, VisualSpection offers the ability to create workflows that can be assigned to the RealWear HMT-1 headset. By combining the software and hardware capabilities, field users can receive step-by-step guidance of site inspections, including easy identification of which assets need documenting using AR asset beacons. As shown in Figure 3, the green circles are the AR beacons for the transformers, and the red numbers indicate the number of workflows needing completion for each asset.

VisualSpection Platform - Sites

 

FIGURE 2: VisualSpection software showing sites.

The AR software and hardware work together to explain step-by-step inspection guidance:

  • Easily identify each asset shown with an AR asset beacon.
  • Visual shows which assets need documenting and labels the number of workflows requiring completion for each asset.
  • Documentation requirements (e.g., types of photos, videos, auto notes and other media).
  • Simultaneous and automatic categorization of each piece of documented media by asset, site location and GPS location.

Field users can complete jobs at the site without having to organize, remember site details or collate inspection data and media to add to a physical notebook.

UNKNOWN ASSET DOCUMENTATION

While the workflow is great for sites with known assets, another system was created for sites with unknown assets. Allowing the user to go to a place and mark the GPS location, a green AR beacon is created that is identical to a beacon created off-site in the office. Field users can rename beacons to match the asset identified and can document the asset with the regular media that the workflow typically requires. For example, if a utility employee were inspecting distribution lines and noticed there was an undocumented pole, the employee could add the pole as a new asset to the database. This would then show up as an AR beacon in the field of view.

OFFICE-TO-FIELD VIDEO COMMUNICATION

Communication with the field often is an issue for any industry with skilled labor outside of the office. This challenge is compounded by the growing knowledge gap between retirements and newer workers, often with less than 10 years’ experience.

The VisualSpection software platform can connect to the RealWear HMT-1 headset’s video camera and audio device. This compatibility allows for a point-of-view communication with the field user.

Office-to-field (O2F) video communication allows less experienced field users to call for assistance and ask questions without having a more experienced employee physically present at the site. Not only does this capability increase the understanding between the field and office, but it also increases the communication time an experienced employee has with field crews. Also included is the option to record O2F communications to be used in future trainings. O2F video communication looks to be a breakthrough technological capability for the industry.

DEPLOYMENT METHODOLOGY

After several months of identifying the specific use cases, scoping the pilot and developing software, the pilot team set out to define deployment methodology. An essential step of the process, deployment helps set expectations for the field user experience and necessary training. It also identifies required processes to enhance user understanding and acceptance.

Simplicity and clarity of deployment are critical to success. In addition to evaluating the overall pilot success, another goal was for field users to feel confident in how and when to use the AR technology, and to want to use the technology beyond the pilot in fieldwork.

PRE-DEPLOYMENT STRATEGY

The team selected AR technology users who had enough site visits to learn from, who were willing to implement the technology frequently and provide feedback for improvement, and who would become advocates for using AR.

To track AR hardware usage during the pilot, a tracking process was established. Users would check in-andout a document to log the amount of time saved by using AR headsets in the field. Field users were trained, and the team gathered initial feedback and made AR interface adjustments for individuals.

POST-DEPLOYMENT STRATEGY

Unfortunately, many site visits were delayed or outright canceled for a variety of reasons during the pilot. While this occurred for many reasons, there was concern that users might forget their training or that AR technology was an option for site visits. The solution was to identify new AR field users who had site visits in the coming week and potential users who had site visits in the near future, and to recruit users from different teams into the pilot program.

VisualSpection User View - AR Beacons

 

FIGURE 3: Field user’s view of AR asset beacons.

Despite some complications, the expanded AR user base helped to raise awareness in the organization about the pilot. More employees became aware of how AR technology could benefit them, and the momentum enabled expansion of the pilot program. Capitalizing on this increased visibility, the pilot team met with team leads to promote the benefits of AR technology and how it could be used, resulting in an increase of pilot devices of 50 percent and extending the pilot program by eight months.

Following the pilot, the team kept in continuous communication with AR field users to find potential improvements for the AR technology and approach, as well as to re-evaluate the pilot and incorporate any needed changes. The result was a list of wants and needs for field users in future deployments

PILOT RESULTS

While the original pilot tracked user data to get percentage improvements from implementing AR technology, it soon became clear that each team had its own way of documenting sites that made the most sense for the situation, the AR user and the client. As such, it was difficult to create a baseline measurement to compare against.

It was determined that the best way to track pilot results was to interview each AR field user and detect where value-adds were common across all teams and clients.

VALUE-ADDS IDENTIFIED FROM PILOT

Several key benefits were identified during the pilot review with AR field users.

Office-to-Field Video Communication: By far, the O2F video communication use case was considered the biggest benefit of the pilot. O2F saved field trips and time in almost every instance. O2F allowed experienced team members to have a virtual presence on-site that would have otherwise been absent.

For example, when a question came up in the field that required particular knowledge, O2F enabled a subject matter expert to join the site visit. Normally, specialists would need to find a gap in the schedule and fly to the site to discuss options. O2F required only a one-hour gap in an specialist’s schedule to join the conversation, add value and help resolve an issue.

O2F also increased confidence in the field. Field staff concerns about issues were greatly reduced by the ability to talk directly to the office and communicate as if in person. While this aspect was not utilized during the pilot, O2F would allow less experienced personnel to be on-site virtually as an opportunity for training, picking up industry knowledge and gaining valuable in-field experience.

Geographic Information System (GIS) Platform: Being able to use the GIS with the workflow for asset inspection created tangible value for site visits. Field users found that the AR beacons and workflows resulted in higher-quality site inspections and, because data was automatically categorized, reduced the amount of time needed for data organizing.

The GIS platform enabled less experienced personnel to perform the same inspections as more experienced site workers. Teams also were able to create standardized workflows for each client, which saved time, added consistency and reduced the risk of error. It is anticipated that the number of site revisits to document a forgotten asset or device would be decreased with the GIS feature.

Unknown Asset Documentation: Many utility distribution systems have been repaired multiple times following weather-related incidents. As a result, much of the distribution system documentation is outdated. The ability to document unknown assets in the field, with data being logged on-site using AR technology, was a huge success.

Typically, a data gathering and organizing trip takes one or two field inspectors an average of 10 minutes per asset to document. Using the AR pilot technology, the time to locate, photograph and document an unknown asset was reduced to less than two minutes per asset on average.

NEXT STEPS FOR ARTECHNOLOGY DEPLOYMENT

The pilot helped unlock tremendous value for utility field visits. As a result of its proven effectiveness, Burns & McDonnell is moving forward with the development of an enterprise edition of the pilot AR technology.

The enterprise edition is expected to standardize the organization on a single asset inspection platform. The pilot has illustrated great benefit and has started to expand beyond the electric transmission and distribution industry. Pilots have been established for the power generation, oil and gas, and aviation industries.

As the enterprise edition becomes standardized, it is expected that new use cases will emerge. For use cases that prove successful and show promise to help end users be successful, new applications will be incorporated into the enterprise software.

PROJECT STATS

Client
Example Company

Location
Example location

Completion Date
Date

Lorem ipsum dolor sit amet, consectetur adipiscing elit. Nullam cursus lectus sed vestibulum malesuada. Vestibulum eget eros eleifend, imperdiet nisl et, hendrerit nisi. Sed sed ornare felis. Mauris posuere libero sit amet dui suscipit consequat. Duis elementum libero id pulvinar lacinia. Mauris nunc velit, imperdiet in urna et, pellentesque tempor ligula. Ut neque lorem, aliquam a lorem et, tristique eleifend dui. Curabitur massa quam, volutpat sed imperdiet vitae, elementum vel ligula. Ut purus felis, viverra vestibulum ultricies vel, blandit non nisl. Curabitur fringilla rhoncus nulla vel efficitur. Nulla in est gravida, vehicula sapien quis, consectetur est.

Proin scelerisque scelerisque fermentum. Aliquam fringilla purus arcu, facilisis lacinia neque bibendum eget. Aliquam faucibus tincidunt sapien et scelerisque. Proin tincidunt porttitor ipsum. Fusce vestibulum tincidunt eros, id tincidunt quam condimentum non. Phasellus ac nisi magna. Nullam pulvinar risus quis ligula finibus, nec gravida neque luctus.

 

100

HOURS REDUCED FROM SCHEDULE
 

20%

INCREASE IN PACKAGING CAPACITY
 

5%

UNDER BUDGET
 

0

SAFETY RECORDABLES
Kadishson 22426

Jonathan Kadishson

Project Manager

Interested in learning more?

Send Us a Note