Automating photogrammetry with coded targets

June 11th, 2018, Published in Articles: PositionIT, Featured: PositionIT

This article looks at what coded targets are, when and why they are useful, how they compare with smart points, and provides examples of applications.

Since its introduction during World War II, photogrammetric measurement has relied on humans manually identifying points in photographs and matching them to the same points in other photographs. The advent of modern computing meant that, starting in the late 1980s, some photogrammetric tasks could be computerised. In the early 2000s, the task of identifying common points in photographs were further automated.

There are several photogrammetry software suites which use smart feature detectors to automate the extraction of point clouds from textured surfaces. Smart feature detection is a recent development in the field, but one of the earlier forms of photogrammetric automation were coded target detection. Smart feature detection uses the natural surfaces of objects and does not require targets, but coded targets still have benefits and uses.

What are coded targets?

Barcodes on packages can be scanned by a laser which reads the pattern of white and black bars and converts that into a unique number or code, which can be used to identify a product at checkout. Coded targets are similar to barcodes in that they uniquely identify something – in this case a point on an object or in a scene.

A coded target has a solid circle at its centre to help automatically mark position to high precision. It is used in point identification, marking and matching automation in photogrammetry.

A coded target differs from a barcode in two mains ways in that the code is printed in a circle, making it easier to identify at all at different sizes, distances and angles (including a slant); and it can be read from a photo taken by a normal camera.

Coded targets can help automate point measurements in photogrammetry, increasing productivity and accuracy.

Geospatial software company PhotoModeler’s coded targets are called Ringed Automatically Detected (RAD) targets. Fig. 1 shows two examples of the 999 available RAD target options.

Fig. 1: An example of two RAD targets.

Fig. 1: An example of two RAD targets.

Coded targets are placed in a scene or on an object before photos are taken. The targets can be printed on plain paper, be mounted on a rigid board, or be manufactured from plastic for repeated use. The only criteria for size is that the centre dot be imaged at least 10 pixels across in all photos. They can be bigger, rotated or at an angle.

The company’s software automatically searches the whole image for what might be a coded target, and then reads the code to confirm detections. These targets can be placed at any angle to the camera (within reason), and can be of different sizes and rotations. When the same code value is identified in multiple photos, those points are matched or referenced. Fig. 2 shows an example of a car with several coded targets on it.

Fig. 2: Measuring a car using coded targets.

Fig. 2: Measuring a car using coded targets.

There are many applications of this technology, some of which are described below. Once the photos are loaded into photogrammetry software such as PhotoModeler, they are searched for codes which are then identified in a few seconds. Coded targets in a project with 50 photos can be automatically detected and processed to produce a 3D point set in less than a minute. (PhotoModeler 2018.0 significantly improves the processing of coded target projects.)

In addition to coded targets, the software supports RAD dot targets and plain dot targets. A RAD dot is like a coded target with a centre dot and a ring, but does not have a code. This means RAD dots cannot be identified uniquely (i.e. cannot be matched across photos) – but the software can still detect them automatically. RAD dots are good for adding point density. Plain dot targets are high-contrast dots with no rings and no code. Since most photos contain circular objects, plain dots are not detected as reliably as RAD dots or coded targets. But plain dots are good for working with small points and higher density in images that are not complex.

Benefits and trade-offs

Coded targets and dot targets can be combined in a project. Coded target detection, marking and matching is automatic, fast and accurate. It has distinct advantages over manual marking and automated smart feature detection:

  • Speed: coded targets are much faster than manual marking and referencing, and in some cases faster than smart feature detection.
  • Accuracy: with the centre dot target, computer-vision-guided marking can produce point positions precise to less than a pixel. This is more precise than both manual marking or smart feature detection.
  • Low point count: when measuring specific points or generating CAD drawings users often do not need a dense point cloud.
  • Measure specific points: users can place the target precisely on the point they want to measure.
  • Measure invisible points: using the offset ability, a paired coded target can identify a third point that might not be visible to the camera.
  • Model diverse subjects: shiny, non-textured surfaces do not work well with smart feature detection, but coded targets can be useful.

Coded targets, however, are time consuming to place and remove. The size of the target is also important. Modelling large objects, for example, requires large targets, which can be difficult to produce. Lastly, it can be difficult to achieve a high density of points with a limited number of codes (999 codes with RAD targets). Table 1 shows the differences between coded targets and smart feature extraction in PhotoModeler software.

Table 1: Comparison of coded targets and smart feature extraction in PhotoModeler software.

Table 1: Comparison of coded targets and smart feature extraction in PhotoModeler software.

Extended use of coded targets

There are two other useful capabilities in PhotoModeler which rely on coded targets: offset points for measuring difficult to access or hidden points, and automatic scale and coordinate system setup.

An offset coded target is a pair of targets of a fixed and known distance apart (Fig. 3). The software recognises these and can compute a third point from this. Offset coded targets are useful for measuring points where a target cannot be placed, such as in a corner or on an edge, or even where the desired point is completely hidden from the camera.

Fig. 3: How offset coded targets work.

Fig. 3: How offset coded targets work.

The other advantage of using coded targets is that they can be used to automatically set a project’s scale, translation and rotation. For example, once the target IDs for the ends of a scale bar has been defined, the software can automatically set the scale in the project (see Fig. 4).

Fig. 4: Plastic bar with two coded targets for scale automation.

Fig. 4: Plastic bar with two coded targets for scale automation.

Another useful capability is that Faro Scene (software for working with Faro laser scanners) can read PhotoModeler Coded Targets to register separate scans, and can be used to automate the overlay of laser scan data over photos.

Examples and applications

Coded targets are used in many industries for the reasons described above.

Home renovation and construction

Several of PhotoModeler’s customers use coded targets to help automate the measurement of staircases, banisters and railings. One such application is the fitting of stair lifts on staircases that do not have a simple structure (see Fig. 5). Offset coded targets are used to identify the nose of each stair, and sometimes the back as well. Targets are placed on each stair before photographs are taken walking down the stairs and from multiple angles. A scale between one or more targets is also included. The software then produces a full 3D model of the points and offsets which can be exported into CAD software for carrying out further design.

Fig. 5: Using coded targets for stair lift design.

Fig. 5: Using coded targets for stair lift design.

There are several other applications in home renovation where one needs to make accurate measurements of a complex shape, such as the measurement or templating of kitchen counter tops and back splashes after new kitchen cabinets have been placed (see Fig. 6). Expensive stone counter tops leave little room for measurement error. A kitchen company or templating firm might have a set of coded target jigs that sit on typical cabinet bases. Along with offset targets to get into corners, an automated system can be developed to produce accurate measurements and CAD drawings.

Fig. 6: Coded targets can be useful for home renovation and construction projects such as measuring or templating kitchen counter tops for new kitchen cabinets.

Fig. 6: Coded targets can be useful for home renovation and construction projects such as measuring or templating kitchen counter tops for new kitchen cabinets.

Applications involving the measurement of glass – notoriously difficult with photogrammetry – are also possible. Fig. 7 shows how a bathroom shower glass stall can be modelled using coded targets and offset targets.

Fig. 7: Coded targets can be used measure on surfaces such as glass, which are usually difficult to measure with photogrammetry.

Fig. 7: Coded targets can be used measure on surfaces such as glass, which are usually difficult to measure with photogrammetry.

Industrial parts

There are many industrial applications that require very high accuracy measurement. Photogrammetry with sub-pixel targets is one of the best ways to achieve this. Furthermore, industrial parts and equipment often have shiny and metallic surfaces, again making coded targets a good method for measuring them.

In one case where a very small object had to be measured accurately, the measurement system required precise placement and surface modelling of a moving surface that was only 6 cm wide. Coded targets, which can be printed very small, proved helpful.  Fig. 8 shows a jig used in this system with laser etched coded targets. The jig is only 5 cm wide, and each coded target only 5,5 mm wide.

Fig. 8: A jig of laser-etched coded targets that is only 5 cm wide, with each coded target only 5,5 mm wide.

Fig. 8: A jig of laser-etched coded targets that is only 5 cm wide, with each coded target only 5,5 mm wide.

Coded targets can also be made of retro-reflective material, i.e. material that bounces light back to the camera with a mounted flash. An advantage of making targets from retro-reflective material is that they are clearly visible in photos. Typically, one would set the camera’s exposure for target visibility. Being able to set the exposure for reflective targets make it easier to work in industrial scenes which contain other circular objects in the scene.

Other applications

Other applications include accurate surveying of interiors and small areas, modelling boat hull shapes or boat decks for retrofits, accurate sizing of archaeological specimens, accident reconstruction, tracking motion and accurate calibration of multi-camera jigs.

Contact Graham Sessions, M.Software, Tel 031 261-4044, sales@msoftware.co.za