Reality modelling for creating digital twins

November 10th, 2019, Published in Articles: PositionIT, Featured: PositionIT

Drones have reshaped the engineering industry, not only from an economic perspective, but also in terms of quality and time savings. This article looks at the technical outputs of drones through case studies to show how reality capture is used to create digital twins.

Understanding the digital twin requires understanding reality modelling. Reality modelling is the processing of imagery or point cloud data of a physical environment into a 3D representation of that environment, to provide an updated context within a geospatially-enabled modelling environment. This type of data can be derived from various equipment, with drones being the most popular in the market now.

An intelligent model with an array of functionality improves spatial analysis and modelling, and means it is no longer necessary to start the design and planning phases of a project from a blank canvas.

This article explains how drones have reshaped the engineering industry, not only from an economic perspective, but also from a quality and time saving point of view. The technical outputs of these technologies will be discussed using examples of projects carried out by The iGlobe Group.

Why reality modelling?

A single source of truth has always been the holy grail, but the construction process has made it elusive. The ability to add recurring high-resolution actionable imagery data to building information models (BIM) is a game changer. BIM is no longer only a design/engineering model – it can be the constantly updated record of construction.

Reality modelling has made it possible to automatically produce a 3D reality meshes from photographs. These meshes are processed into engineering-quality models that capture real-world context.

The wide adoption of drones in the industry has added to the success and disruptive nature of reality modelling as data can now be acquired much faster and in many different types of environments and/or terrains. Software developments have further improved drones, and vice versa.

Drones have assisted various BIM practices, by providing accurate and current data in order to enhance many phases of construction, including:

Structural inspections: Previously, one of the riskier parts of the construction and demolition of a building was the structural inspection. Drones can help eliminate the need for some of these inspections by allowing contractors to view harder-to-reach areas.

Site inspections: If architects and planners can conduct a site inspection before construction begins, they can generate more precise drawings and identify possible challenges they may face down the line.

Advanced point cloud scanning: Drones provide an effective method of point cloud scanning, as the aerial perspective they provide give a better analysis of the topography of a site and can generate a more detailed 3D rendering than fixed lidar systems.

Progress tracking and marketing: Drone photogrammetry can be helpful with keeping investors updated on a construction project and can also be used to improve marketing for a project.

Safety monitoring: Construction managers can identify potential risks.

Project completion checks: Once a project is completed, there is still a need for various assessments and checks that need to be conducted. Drones can provide thermal imaging of a building to ensure that it is structurally sound. This kind of imagery is also helpful for judging energy efficiency.

Government regulation: Depending on the scope of a construction project, there may be a need to involve the local authorities. Drone can help you quickly and easily obtain any of the data required in such a situation.

The reality modelling process

Naturally, the rapid growth of drones and reality modelling software in the market has taken industry by storm. Due to this, many major role players (such as The iGlobe Group) have developed their own intellectual property for this technology, in order to stand out from the crowd and to provide products and services tailored to their clients and their industries. As such, this paper is unable to delve into the nitty gritty of the workflow required to produce a rich 3D model as there are non-disclosure and intellectual property restrictions in place. However, a very high-level overview follows.

Whether it be a building, pipeline or telecommunications tower, the same typical phases are followed to deliver a reality model, namely:

  • Planning
  • Equipment selection
  • Data acquisition
  • Data processing
  • Data management

Planning

The outcome of the final product will only be as good as the initial input. Thus, it is important to understand the needs of the client in terms of the practical and functional requirements of the model. In addition, meticulous planning is required from a safety point of view, so that all risks and hazards are identified beforehand. These days, there are several software packages and mobile applications available that can assist the acquisition team in their planning.

Equipment selection

Equipment selection is vital in ensuring the envisaged deliverables and quality are attained. There is a large selection of drones available on the market, each with its own advantages and disadvantages pertaining to flight times (battery usage), flight range and versatility. However, the single most important factor to consider when selecting your equipment, is the payload. The payload selected for a specific project will have a much greater effect on the deliverable than the actual drone selected. It is here where focal lengths, camera resolution/megapixel size and model/make must be considered.

Fig. 1: An example of types of drones to be considered during equipment selection.

Data acquisition

There are also several flight operation applications available that can assist the pilot in the acquisition phase by means of autonomous mission planning. These applications/software packages have been developed with the data output as the primary focus, and it therefore important for the pilot to oversee the operation and adhere to civil aviation laws.

The flight patterns selected are crucial to the output. It is a common misconception that the more photos, the better the 3D model. With sound planning, the acquisition could take a third of the time and yield less photos, with the same result, if not better. It is not the quantity of the photos that matter so much as how they were captured. Depending on the structure, images need to be captured at different heights using various patterns.

Another important factor to consider during the acquisition phase is GPS accuracy – not only the GPS accuracy of the drone, but more importantly the accuracy of each image and pixel in it. This is because the 3D model is intended to be geospatially correct. The model needs to be millimetre accurate to interrogate it and use it for measurements. Geospatial accuracy is most commonly attained with RTK technology or the use of ground control points, which then ties the model into a coordinate system. To increase photo location accuracy, the images need to be geo-tagged using the drone’s GPS. In many cases, it is advisable to make use of additional RTK technology.

Fig. 2: A DJI Phantom 4 Pro fitted with RTK technology for increased GPS accuracy.

Data processing

Selecting the software package to render your 3D model is just as important as the equipment selection. There are several software products, each with their own strengths and weaknesses. These include 3DR Site Scan, Reconstruct, SKUR, AutoDesk ReCap and Agisoft.

ContextCapture (from Bentley Systems) is a forerunner in this space, and allows users to produce 3D meshes of real-world conditions from photographs, right up to city scale. The software turns photos into detailed photo-textured 3D mesh models that integrate with a range of software, including CAD applications. The models are precise enough to measure from or be used for comparing as-built conditions with a design model. A set of photos up to 100 gigapixels in size can be converted into a single detailed model.

Once the processing is done, there are typically four outputs from the raw imagery:

  • A 3D mesh model or a reality model, which can be delivered as a web-ready file for coordinate interrogation, measurements and viewing.
  • Orthophoto – fully corrected overhead photos, accurately stitched into one manageable image.
  • Digital surface model (DSM), essentially the 3D model as a wireframe without the textures.
  • Point cloud data, from which contour plans and contour files at desired intervals can be retrieved.

Fig. 3: An example of ContextCapture 3D model processing software.

Data management

With the large quantities of data generated, it is essential to manage the data in such a way that it is user friendly, intelligent and constructive. A 3D model is useless if it is not managed geospatially within a system. One way to accomplish this is to make use of an asset management system that supports the many file types.

There are several asset management software packages available, or a custom-built system can be developed to consist of a geospatial database in the background and a portal/dashboard as the client facing tool. The dashboard can be customised specifically to the client’s needs, to displays various statistics and management information. The end-user does not need to sift through or host millions of photographs, they are supplied with the completed product (3D models and drawings/data generated from the models). Data history is also an integral component of the asset management system as current models can be overlaid with legacy models to monitor changes and inspect defects.

Fig. 4: The flight missions and location of images over the site.

Case study: Facilities management inspection

Siemens South Africa, with their head office in Midrand, Gauteng embarked on an integrated building information project with partners The iGlobe Group and the ONE-FM team to establish a complete 3D building information package, both internal and external, for ongoing facilities management activities. The site consists of a 7 ha landscape with complex buildings in the form of an alphabetical letter “E” and “F” as well as a round building of about six levels high and an eight-row car port.

The digital twin required internal and external modelling. Bentley ContextCapture was used to 3D model the exterior of the complete site in order to create CAD drawings from the subsequent point cloud.

Purpose and desired outcomes

Digital twins enable cost savings since all planning and changes to the site can be envisaged without any work stoppages. Digital twins also illuminate the need for site visits since incidents can be inspected and assessed remotely. Similarly, condition assessments become a desktop exercise on which expert teams can collaborate. Another goal was to create augmented and extended reality models so that staff can remotely access all the data on the site through a live feed to solve issues faster and in a more cost-effective way.

Ultimately, the deliverables required for the above outcomes were:

  • A digital twin created by capturing the buildings and site information in 2D and 3D models
  • Several georeferenced CAD drawings
  • Internal layout drawings
  • Contour data to provide planning information such as drainage and evaluating extension possibilities
  • Orthophoto for inspection and marketing purposes

Project challenges

Rendering a building with this complexity with drone technology and additional fill-in photos needed careful planning without hindering the daily operation at the building. Careful planning was necessary to determine the best sun light and angle for the photos and to ensure minimum human interference.

CAA drone regulations limit the flying a drone close to a public highway, therefore requiring further planning so as not to cross the highway. Alternative options including using a manned helicopter to get better exposure and the use of handheld DSLR cameras to fill-in the complex sections in-between the buildings.

Another challenge was ensuring that the exposure on the many missions were similar. It was important to operate under similar weather and lighting conditions. This specific building was captured over dozens of missions to get the desired result.

Fig. 5: The wire mesh derived from the point cloud data.

 

 

 

Fig. 6: The completed 3D model of the entire site.

Fig. 7: A snapshot of the 3D reality model (post processing).

Conclusion

By making use of reality modelling technology, a digital twin was created to provide more information and design detail of the building than possible with a conventional inspection. In addition, the automated approach provided a much more accurate representation of reality. There was a significant cost reduction in using a reality model compared to manual methods. Using the drone also had health and safety benefits to the client as staff no longer had to climb the building and structures to assess defects or gather information.

Case study: Inspection of conveyor structure

The client requested a digital twin of a 2 km stretch of a conveyor structure. The 3D model would then be used to inspect defects and rust. However, it was the point cloud that was important to inspect sinking of the conveyor support plinths, which was causing a variety of problems on the conveyor system.

Purpose and desired outcomes

The purpose of the project was to assess the degradation and sloping of the conveyor structure support plinths. Mining engineers were concerned that this degradation was leading to bearings overheating and the belts being strained. This shortens the lifespan of bearings and belts and results in financial losses.

The client therefore requested CAD drawings from the point cloud, as well as profiles and cross sections. This enables engineers to ascertain which plinths need to be raised, and which need to be dropped. A very accurate drone aerial survey was also required as the topography of the land was an important factor to consider. The conveyors were erected in the 1960s, meaning that the topography of the land had changed significantly since.

The deliverables required were:

  • High-resolution orthophoto
  • Dense point cloud
  • Rendered 3D model (digital twin)
  • CAD drawings created from the point cloud
  • Profiles and cross sections as PDF reports

Project challenges

The length of the conveyor (2 km) presented numerous challenges, in that the processing would have to be done in sections and stitched at a later stage. Covering a structure of this size requires multiple flight missions, which often introduce GPS issues since the drone resets when batteries are changed. The project planning and equipment selection was of paramount importance on this job.

As this was an operational conveyor system, the acquisition timeframe was limited to ensure little impact on mining operations and to avoid extended downtime. Again, planning in terms of weather, site access, knowledge of terrain and battery management was required.

Fig. 8: An example of the cross sections, drawn over the point cloud CAD drawings.

Fig. 9: An example of the profiles drawn from the 3D model, once the images had been removed.

Fig. 10: An example of the completed 3D model.

Conclusion

With the use of drones and reality modelling software, accurate and important information was provided to engineers and decision makers to rectify the issues and defects of the conveyor structure. This resulted in cost savings. The entire project (from data acquisition to data delivery) took only 20 working days. A conventional survey and structural inspection would have taken four to five months. In addition to the cost savings, further damages and financial losses were avoided.

Case study: Telecommunication infrastructure inspection

A telecommunications company has purchased hundreds of towers in Africa. As part of the acquisition process, the towers needed to be inspected to evaluate components including their structural condition and integrity, security, assets onsite, ownership of assets and dimensions. The inspections had to completed in a timeous and safe manner, which made the use of drones and reality modelling was an attractive option.

Purpose and desired outcomes

The conventional procedure of inspecting towers involves multiple teams of people climbing the structure. These teams will take photographs to assess tower conditions, measure assets and record an array of data using a checklist. This comes with safety risks as the towers are generally quite high, with strong winds posing another risk. Furthermore, the data retrieved from the site by the conventional teams is often inaccurate and incomplete.

The purpose of this project was not limited to creating a 3D model of the tower and site, but more importantly, to develop a turnkey solution. The solution (known as the IGTMS – iGlobe Tower Management Solution) is essentially a full asset management system, which is built on a geospatial database.

The client side of the solution is an interactive dashboard, which contains:

  • Tower key drawing in CAD and PDF formats
  • 360° orbit of tower and surrounds (video and images)
  • 3D reality model of tower and assets (including raw imagery)
  • Report on cardinal points of all antenna directions
  • Ground dimensions, tower dimensions and reporting on unutilised space
  • Tower type
  • Antenna dimensions (actual and from ground)
  • Fence/property dimensions and status
  • Dimensions and positions of all equipment onsite
  • Identification of vegetation and access roads
  • External status/inspection of solar equipment
  • PDF drawing of tower with complete assets
  • Structural information (missing members, bolts etc.)
  • Report on site technologies (client to advise)

There were four phases to the process:

  • Acquisition phase, including site identification and flight planning; site establishment; risk and safety assessments; drone acquisition flights, quality control and data upload;
  • Image and model processing phase: preparing data for processing; image processing; 3D model processing and quality check.
  • Data processing phase: prepare data for processing, import data to tower processing software; import data to CAD processing software; and quality check and prepare files for dashboard upload.
  • Dashboarding phase: upload checklist; upload images and test; upload documents and test; and publish the data.

Project challenges

The towers are scattered all over Africa, which means that proficient and optimal logistical planning is required. Each country has their own drone regulations, and as a result there is a lot of administration that has be handled before the data acquisitions can begin.

Fig. 11: An example of the interactive tower key drawing within the TMS dashboard.

The method of tower acquisition to render the best possible 3D model and associated deliverables is very specific and is the result of two years of research and development. Naturally, training all the stakeholders across the continent on this method is a challenge. This challenge is overcome by developing and implementing specialised flight planning software applications, which allowed a low skilled pilot to still yield satisfactory image datasets.

Fig. 12: An example of the interactive dashboard, statistics and map.

Conclusion

In conclusion, the tower management solution provides the clients with an asset management tool to assist with the maintenance, management, monitoring, planning and design of their tower network. The solution can be tailored to the client’s specific needs in terms of the output and display. The solution is a direct result of the innovation in reality modelling technology.

In summary

Reality modelling through the use of drone technology enable efficient and cost-effective ways to create digital twins, which in turn hold many benefits to clients.

References

[1] Pae Natwilai: How Drones are Revolutionizing BIM. https://www.pobonline.com/articles/101569-how-drones-are-revolutionizing-bim
[2] Vignesh Kaushik: The 9 Best Reality Modelling Technologies for AEC Industry. https://www.getrevue.co/profile/TGIC/issues/the-9-best-reality-modelling-technologies-for- aec-industry-61522
[3] The iGlobe Group Project Archive.

Contact Heico Kühn, iGlobe, heico.kuhn@iglobe.co.za

Related Articles

  • South African Government COVID-19 Corona Virus Resource Portal
  • Now Media acquires EngineerIT and Energize from EE Publishers
  • Latest GIS further supports electric and gas networks
  • EE Publishers 2020 Year Planners
  • To readers, customers, suppliers, staff and other stakeholders of PositionIT