Creating 3D virtual environment for autonomous vehicles

March 16th, 2018, Published in Articles: PositionIT

The Sanborn Map Company set out to demonstrate the accuracy of high definition maps in the context of a 3D reality model of the Silicon Valley area. It generated the 3D reality mesh from aerial oblique imagery of the target area. The company then overlaid the 3D mesh with HD map data that includes a highly detailed inventory of the roadway features required for safe navigation by autonomous vehicles.

Major auto manufacturers are racing to release self-driving features that give their vehicles some autonomy in specific situations, such as braking to avoid imminent collision. To achieve the goal of a driverless vehicle, manufacturers need to demonstrate that their cars can safely operate in any situation. The RAND Corporation estimates that a driverless car would have to drive more than 17-billion km to prove it is 20% safer than a human-driven car. A thousand test cars driving non-stop, 365 days a year would take 50 years to cover that many kilometres. Simulating the test drive in a virtual world would prove the point in a matter of hours.

An industry leader in geospatial solutions and technology, The Sanborn Map Company, is helping to make self-driving cars a reality by creating high-definition (HD) maps used by test-drive simulators for autonomous vehicles. The company has partnered with auto manufacturers in a $100 000 project that demonstrated the precision of purpose-built HD maps.

Fig. 1: 3D reality mesh of Santa Clara County generated with ContextCapture Center.

Fig. 1: 3D reality mesh of Santa Clara County generated with ContextCapture Center.

Using aerial oblique imagery of an area in Santa Clara County, California, the company generated a 3D reality mesh, and then overlaid proprietary HD basemap data that is accurate within an absolute range of 7 to 10 cm. Visualising the high-precision HD map in a high-resolution 3D urban environment elicited auto company requests for repeat demos internally and with department of transportation regulatory authorities.

Safe test-drive environment

Major auto manufacturers have already released, or are about to release, self-driving features that give the car some ability to drive itself. Through the 3D mesh it was possible to showcase the quality of the HD map data, including true-ground-absolute accuracy. Automating the production of a city-sized 3D model made the project faster than previously possible.

Sanborn’s Advanced Technology group has developed proprietary HD mapping technology that creates standardised, high-precision 3D basemaps for self-driving vehicle models and markets. The HD maps contain more detailed information with true-ground, absolute accuracy than is available in conventional resources such as GPS maps. Compared with current mapping systems that can locate a car’s position within 1 m, HD maps can position a vehicle within 10 cm.

Precise real-world context for HD map data are created by 3D reality models of the environs, drawing upon expertise in aerial oblique imagery, aerial lidar data, and mobile (driven) lidar data. For image processing, the map company used Bentley’s reality modelling software, ContextCapture Center, for 3D models at any scale. The software generates the 3D engineering-ready reality mesh on which the HD map data is overlaid to create the purpose-built, map-based datasets automotive makers need for their virtual worlds.

High definition, precision, resolution

For the automakers’ project, Sanborn acquired aerial oblique imagery of the Silicon Valley area, including Santa Clara, Sunnyvale (Heritage District), Palo Alto, and surrounding locations, to construct a sizeable target area for mapping. The imagery was acquired in a single, multi-pass flight plan using five digital cameras to collect four oblique views and one straight-down (nadir) view. The reality modelling software automatically aero triangulated the images to identify the position and orientation of each image, then reconstructed the images as a highly accurate georeferenced 3D reality mesh.

The 3D reality mesh provided precise real-world context for the HD maps. With many densely packed triangles, draped textures, and snap points, the 3D mesh created the fine details, sharp edges, and accurate geometry required by the automotive partners. The georeferencing made it possible to identify the precise locations that intersect with the HD map data, collected in absolute XYZ.

Fig. 2: 3D reality mesh generated automatically from a multi-camera aerial oblique imagery.

Fig. 2: 3D reality mesh generated automatically from a multi-camera aerial oblique imagery.

Combined with the HD maps, the high-resolution 3D mesh provided a detailed inventory of physical assets for the roadways, such as lane markings and dividers, road edges and shoulders, and traffic signals and signage. Inclusion of surrounding buildings, which were enhanced using Bentley Descartes, produced a city-sized reality model for a more genuine driving experience.

The HD maps and 3D mesh were then integrated into virtual reality simulation tools for testing autonomous vehicles by using the FBX (Filmbox) format to ease the process.

Testing the testing grounds

The HD mapping technology makes it possible to create datasets for multiple environments and levels of complexity in the autonomous driving tests. For some sections of the city, Sanborn’s developers loaded the 3D mesh into a web interface, so the automotive partners could see the quality of the 3D reality view. Sanborn also developed web-based analytics that the partners could use to review and discuss vector and point data superimposed on the 3D reality mesh. Within the 3D viewing interface, it was possible to precisely measure distances, volumes, surface areas, and other parameters.

The project highlighted the level of road intelligence that could be achieved in a 3D reality model. As they explore the virtual landscape for autonomous driving tests, several automotive partners are using the company’s 3D demonstrations internally as well as in meetings with department of transportation regulatory personnel.

Contact Martiens Pelser, Bentley Systems, martiens.pelser@bentley.com