With over 40,000km of track, Australia has the sixth largest railway network in the world. Current railway track inspections involve costly and occasionally dangerous practices, which are especially difficult in remote areas, but developments by Monash Institute of Railway Technology in multi-rotor Unmanned Autonomous Systems (UAS) are set to change this.
Conducting autonomous track inspections
Current railway track inspection practices typically involve either the use of dedicated vehicles or converted road vehicles that require a train path to operate. In other instances, personnel are required to be located within the rail corridor to perform visual inspections where, in some cases, inspections in a railway environment can be inherently dangerous.
Monash University’s Institute of Railway Technology (IRT) has been developing Unmanned Autonomous Systems (UAS), commonly known as drones, to help assist the transition to autonomous inspections, especially where railway infrastructure inspections are conducted in remote areas.
The deployment of UAS creates an alternative remote railway track inspection tool and removes the requirement of additional safety training and human resources.
Regular maintenance and inspections are necessary to keep Australia’s vast transportation network moving, however, rail infrastructure inspections often require personnel to be within the rail corridor to perform visual inspections.
A range of safety protocols have been introduced to minimise risks, but this can involve costly track possession or occupation to inspect the track sections and sometimes these track sections do not have clear accessible safety zones.
Other assets, such as culverts, are defined as confined spaces and additional safety training and human resource requirements need to be met before inspections can take place. This can often make culvert inspections both time consuming and costly.
Previously, ground robots were used for remote inspections, but were faced with difficult terrain or limited mobility. The UAS have the advantage of being aerial, and by utilising Global Navigation Satellite Systems (GNSS), the UAS can fly above the railway track to capture observational data.
The acquired information is then used to reconstruct 3D digital models of the infrastructure to identify critical deterioration and maintenance requirements.
The UAS are also specifically designed by IRT to handle confined spaces such as cuttings, tunnels and culverts in a semi-autonomous nature. Employing the application of various sensors including an on-board Light Detection and Ranging (LiDAR) scanner, the UAS can measure the shape and surface conditions of constrained infrastructures in real time.
Conquering the remote
Heavy haul railway networks in areas like the Pilbara region of Western Australia, are often remotely located in harsh environments with limited access and phone coverage. The UAS can inspect the track in addition to the state–of–the–art technologies like Instrumented Revenue Vehicles (IRV) which were also designed, developed and installed by IRT, and are able to identify visible rail and track defects, as well as missing components such as clips and fasteners.
The reconstructed 3D point cloud images created by the data collected by the UAS provides additional information, which enhances decision-making and assists in the preparation of any required remedial works. The data captured can be compared to previous inspections of the track infrastructure and surroundings.
This enables automated monitoring of the changes occurring over time using smart analytics without significant investments in resources and inspections.
Topographical maps are also able to be created from the UAS data which is spatially geolocated. Operators will have access to up-to-date satellite imagery of their networks at a centimetre level accuracy.
Prototype hybrid for inspections
A prototype hybrid of a UAS is being developed by IRT, enhancing its capability to remotely land on the track and drive along the rail using vision-based 3D reconstruction. The prototype currently has the flying and landing speed of 5–10m/s and can hover for around 20 minutes. After detecting any possible abnormalities, the hybrid UAS can take off and land at a safe location.
A prototype of the driving system has been developed by using Computer-Aided Design (CAD) software and is specifically designed for the after-landing track inspection. The driving system consists of a lightweight sensor suite weighing less than five kilograms, a body transformation frame, and has special landing gears.
The body transformation frame allows the cart to unfold itself to fit various rail sizes. Several different prototypes were designed using different folding techniques. The landing wheels enable the whole cart to self-align on the rail after rotation.
Acquiring accuracy in railhead measurements
Multi-View Stereo (MVS) algorithms can reconstruct 3D geometry from several images obtained by the UAS, with the accuracy of such models comparable to those acquired by 3D laser scanners. This provides the possibility of high accuracy models in the very near future.
A trial by IRT compared the MVS reconstructions with a CAD model using the Iterative Closest Point (ICP) method. The accuracy of the cross-sectional shape of each reconstruction was compared with the actual cross-sectional shape of a 60kg rail. A 3D CAD model of the railhead was created and 200 reference points along the cross-section were sampled.
The comparison indicates that the cross-section shape of the reconstruction result and the ICP result are nearly the same with a minimal RMS (root mean square) error of 0.67mm. It is also interesting to note that the ICP results can be used as a measure to evaluate the status of the rail head wear.
The magnitude of the RMS error grows as the rail wear becomes more severe.
These findings demonstrate the strong potential of 3D reconstructions which use only a few images in supplementing the traditional manual railhead measurements.
Inspecting tight tunnels and culverts
As part of IRT’s trials, a tunnel inspection was undertaken using a custom designed quadcopter platform. The latest prototype is 550mm in length motor-to-motor diagonally and its weight is about 2.4kg with 10–minutes flight time.
The UAS contains an on-board system consisting of processors, and a sensing system. The processors include a computer and an external GPU, which are used for controlling the on-board components and processing sensors information. The sensing systems consist of a LiDAR sensor and a stereo camera.
The LiDAR can acquire cross–sectional data of the tunnel while the stereo camera enables the UAS to perform simultaneous localisation and mapping (SLAM). The LiDAR is capable of generating a point cloud while monitoring the location of the UAS.
A relative localisation method, which utilises the geometrical centroid of the tunnel cross section, can be used as an alternative for the control system of the quadcopter platform to achieve self-stabilisation within a tunnel-like environment.
Flight experiments by the IRT found that a 300mm size quadcopter platform seems to be close to the maximum size that can traverse through the average Australian railway culvert in a relatively stable fashion with acceptable oscillation during flight.
The latest prototype is able to stably travel through tunnels of diameter greater than 2.5m when flying at the centroid of the tunnel cross section, however, flight stability is still not guaranteed when hovering near the sidewall. Greater development and a more robust controller is the next step for improved stability for the system when UAS are flying in narrower confined spaces.
Field trials at a Yarra Valley railway tunnel
To evaluate the performance of IRT’s UAS, several trials were carried out in a Yarra Valley railway tunnel in Melbourne.
Since the dimensions of this railway tunnel were much larger than the quadcopter platform, the confined space aerodynamic disturbance had a less significant impact on the stability of the system. There were only minor oscillations due to the occasional wind gusts during the flights.
Overall, the UAS was able to navigate through the vintage railway tunnel semi-autonomously, with only pilot input to control its longitudinal motion.
To demonstrate the system’s capabilities as an inspection tool, a 360-degree camera was installed on the UAS to provide an immersive visual feedback. Hence, the UAS was only required to perform a single flight to gather the full view of the tunnel.
Another main objective for the trials was to collect LiDAR data to reconstruct a 3D point cloud of the railway tunnel. Although the on-board stereo camera can provide visual SLAM, the SLAM output suffers from position drift over time and as a result, the position of the UAS in the tunnel’s longitudinal axis is not exactly known.
As a first pass, the reconstruction of the point cloud assumes that the UAS is travelling at a constant speed, which is unlikely to be the case in practice. However, with the adoption of improved localisation, the reference position can be tracked which will result in improved reconstruction accuracy.
An example application for the tunnel 3D point cloud is to perform a time series comparison of the point clouds. The point cloud data collected from each inspection can then be compared to the reference point clouds from previous inspections, to monitor the deformation of the tunnel infrastructure over time with minimum effort and time.
Set of points are computed by taking the average dimensions of the tunnel point cloud. In practice, this averaged point cloud would be obtained either from static measurements or from the UAS measurements with improved localisation applied.
Improved localisation can be implemented via a range of approaches including the use of wireless beacons or lasers located near the entry/exits of the tunnel.
A point cloud analysing and visualising tool is also developed to provide a more interactive way to post process the point cloud data. Users are able to select different view perspectives such as the first-person view of the tunnel point cloud using the custom visualisation tool.
The user can navigate through the sections of the tunnel while visualising the highlighted features (cave-in or cave-out section etc.). Such tools introduce different ways for inspectors to analyse the infrastructure and allow more efficient and effective inspections.
Regulatory framework for use of UAS in railways
IRT in collaboration with the Transport Division of the United Nations Economic and Social Commission for Asia and the Pacific (UN ESCAP) is researching the unique challenges associated with the use of Unmanned Aerial Vehicles (UAVs) in and around operational railways for the monitoring and maintenance of railway infrastructure.
This research involves knowledge capturing of experiences and concerns of rail operators and others working in the railway environment in relation to deployment of UAVs.
The findings of the research are expected to help articulate the safety requirements for UAVs to operate safely at rail sites and analyse the current regulation and procedures in place that govern the use of UAVs in the Asia Pacific region.
An autonomous future for digital railway
The application of multirotor UAS are becoming increasingly attractive and more viable for railway inspections, especially in remote locations due to their manoeuvrability, which permits safer operations normally regarded as high risk.
The multirotor UAS allows railway operators to conduct regular inspections and collect dimensional and surface measurements without personnel needing to enter the railway corridor. The information collected not only reduces the heavy reliance on manual inspections, but also facilitates the transition to a safer, more economical and flexible inspection regime.
The 3D point cloud data collected from each inspection can be compared to reference point clouds from previous inspections, to autonomously monitor changes in infrastructure over time and assist in the planning of maintenance and remedial actions in a timely manner.
As the development for UAS continues to progress, the railway industry should expect to rely more and more on autonomous machines to perform infrastructure monitoring maintenance checks in the future.