3.2. Characteristics of IMU Systems
IMUs (Inertial Measurement Units) are designed to determine the angular orientation of the measuring device in 3D space (in our case, the entire scanning system). The orientation can also be used to control rotating objects. IMU makes it possible to measure the instantaneous linear accelerations and angular velocities of said objects. An IMU consists of accelerometers and gyroscopes.
The IMU operating frequency is within the range of 100 Hz–3600 Hz, depending on the technology of the accelerometers and gyroscopes (laser, optical, MEMS technology) used.
An IMU system with software is called an INS (Inertial Navigation System).
IMUs also have software that makes it possible to adjust the measurement system to the dynamics of the measurement object, to visualize the measurement data in real time and to register the data.
The orientation of the device is defined by specifying the Euler angles, referred to as the “roll” (rotation around the axis in line with the direction of travel), the “yaw” (rotation around the vertical axis) and the “pitch” (perpendicular to the other two). The measurements of IMU “navigation” angles are made in relation to the locally-assumed system centered at the center axis of the sensor system, the local vertical line (Z-axis) and the direction of the travel axis of the measuring system (Y-axis). This system is time-varying in line with a change in the position of the system to form an inertial frame of reference (referred to as the “body”).
The estimation of the Euler angles is based on the idea of sensory fusion, in which one algorithm integrates the measurement data from various sensors. The angles can be determined from the gyroscopes by the matrix integrating of angular velocities and from the accelerometers and magnetometers, using algebraic relations. The IMU devices are used for specifying the angle elements of the measuring system, in this case a mobile scanning system. IMU-type devices feature the determination of angles with various accuracies. Such systems are usually coupled with GNSS-type receivers.
The accuracy of inertial navigation is affected in the first place by the accuracy of the measurement of angular velocities. The breakdown of inertial units by the angle achieved via the accuracy of the inertial system is parameterized by the gyro drift values.
Inertial measurement units used in mobile measurement systems belong to two accuracy groups:
Navigation units: drift amounting to 0.001–0.100 degrees/h,
Tactical units: drift amounting to 0.1–100 degrees/h.
GNSS/IMU: POS 420 V4 LW, a positioning system made by Applanix, supported by DMI, is the leading solution on the mobile systems market.
The tested systems, e.g., RIEGL (VMX 250), identified the angles with the following accuracy:
Roll and pitch angles: 0.005 degrees,
Heading angle: 0.015 degrees.
3.3. Characteristics of the Vision Systems
Vision sensors in mobile systems dedicated to clearance gauge measurements are usually used for three tasks:
Coloring of a point cloud acquired by means of a laser scanner,
Supervision and registration of the measurement process,
Photogrammetric measurement of gauge points.
Currently, research is underway on the use of images from such systems for automatic detection and identification of objects in railway infrastructure.
Vision systems are characterized by parameters that describe the geometrical model of the camera, their radiometric features, the characteristics of an image carrier and the interface between the camera and image acquisition cards.
The camera’s geometric model is presented by means of the internal parameters represented by a matrix called a calibration matrix, which describes the transformation between the coordinate system of an image and the coordinate system of a sensor. It is determined in the process of camera calibration and described by the imaging distance, the location of the principal point of an image in the imaging coordinate system, the pixel size in the x and y axes and its skew ratio, lens distortion model coefficients, along with the radial and tangential distortion.
The geometric parameters of the camera are needed both when using the images for coloring a point cloud and for photogrammetric measurement, with the latter application requiring more accuracy of their designation.
The vision sensors used for measuring the gauges work under boundary lighting conditions and show objects both in deep shadows and in full sun. The automated selection of exposure parameters under dynamic conditions does not prove efficient. Therefore, the sensor’s radiometric features, such as sensitivity, radiometric resolution (tonal range), spectral resolution and spectral range can be of decisive importance when choosing a device.
The radiometric features of sensors are affected by the lens-matrix array. Basic radiometric features of a sensor, such as sensitivity, radiometric resolution and spectral resolutions, are listed in the sensor’s specifications. The sensitivity of the camera is, as a rule, higher when operating in the monochrome mode than when operating in the RGB mode; the standard tonal range is eight bits per channel, yet often, higher ranges can be found (10, 12 and 14 bits per channel).
Such features, as sharpness, contrast and resolution, are determined by the modulation transfer function or the spatial frequency response, which informs about the optical system’s response depending on the spatial frequency of the signal.
The ratio of the maximum amount of light that the sensor can capture to the minimum amount of light is the dynamic range of the sensor. The maximum level of the signal depends on the capacity of the detector, while the minimum level is an effect of noise. Cameras of high dynamic ranges can record at the same time both very strongly-illuminated objects and very deep shadows. Tonal range is the number of signal levels within the dynamic range. Tonal response is the relation between the signal and the amount of light, shown in the form of a graph. Gamma is a slope of the tonal response curve, and that slope is the contrast index.
Another feature to consider is the accuracy of color rendering, which is measured and calculated on the basis of differences between reference colors and the colors rendered by the sensor.
Noise is an effect of accidental changes in image pixels’ brightness, manifesting itself in deviations from the average level for the same reference brightness as imaged by the sensor. The measure of noise is the average standard deviation, determined for various brightness levels. The noise level of sensors is listed in the detailed camera specifications, but one can examine it based on tests involving a standard color or gray level target.
Digital cameras generally use two types of photosensitive matrices: CCD and CMOS. Sensor matrices differ, first of all as regards sensor sizes, sensitivity, type (color or monochrome), readout rate, special features (i.e., windowing, binning).
CCD sensors make it possible to render high-quality images of very low noise levels, while CMOS sensors are more susceptible to noise and are marked by a lower sensitivity (they have low pixel filling ratios). Unlike CCD sensors, CMOS sensors consume less power (over 100-times less than CCD sensors). One can manufacture CMOS sensors on nearly any standard production line, resulting in them being significantly less expensive as compared to CCD sensors.
CCD sensors are dedicated for cameras that are designed to produce images of high quality and resolution, with an excellent luminous sensitivity. Due to lower noise levels, CCD sensors are beneficial to obtain low-contrast images. The main advantage of CMOS sensors is their high versatility. They are capable of windowing in order to sense a lower amount of data at a considerably higher acquisition speed increase. They are also main elements of quick digital cameras, which acquire images with a frequency of even one million frames per second, e.g., Phantom v12.
Digital cameras furnished with various CCD photosensitive matrices can operate using line-by-line, interlaced or progressive scanning modes, whereas CMOS matrices complete sensing sequences in “rolling shutter” or “global shutter” modes. For measuring objects that are in motion, CMOS cameras with global shutter exposure are recommended. Image acquisition frequency ranges from several to even a million frames per second (Phantom v12). Presently, resolution ranges in camera matrices exceed 25 Mpix (e.g., Bobokat IGV-B6620 of a resolution of 29 Mpix). Interfaces used at digital cameras include USB, FireWire, CameraLink, GigE and CoaXPress, while the digital standards most commonly applied in vision systems are USB 2.0, IEEE 1394a and 1394b (FireWire), CameraLink and Gigabit Ethernet (GigE).
3.4. The Conception of the Database
The project framework also included the development of the database conception, knowing that data obtained from laser scanning would require much space (1 km of railway route equals over 1.5 GB of data).
It was the task of AGH to work out an IT solution that matched existing systems already in place at PLK. An IT solution has been developed, including the database, an application to browse the database and to assign codes, as well as a number of auxiliary programs designed for importing measurement data in the database. The database constitutes the key element of that solution. Following the review of databases that are available on the market, a decision was made to utilize Oracle.
In addition, before proceeding with the project works, the authors made the following assumptions:
The database must be capable of importing data directly from measurements, so it must record both point clouds and images. At the same time, data from point clouds is to be analyzed first, while image data is to be used in the event that an area holds some doubts (e.g., areas where the information provided by the point cloud alone is not 100% sufficient to make a decision).
The database must cooperate with the POS (Leading Network Description) and SILK (Railways Lines Information System) databases. In both of these databases (POS and SILK), the basic way of addressing objects at railway lines is that of chainage. The requirement of the project was the connection of our database with POS and SILK, to exchange the data.
Due to compatibility with the SILK database, data would be processed under the 1992 national geodetic coordinate system, yet the database must be capable of converting data to other coordinate systems.
When developing the database, the following input data were utilized:
GPS/INS data, including the vehicle trajectory coordinates; ca. 200 samples per second.
Laser scanning data (a point cloud), about 25 million points (~1.5 GB) per kilometer of railway line.
Images (in JPG format) taken by four cameras; about 350 images (~0.3 GB) per kilometer of railway line.
During the process of designing the database, several procedures were developed. These procedures were aimed predominantly at reading in the data, preliminary reducing of their volume, then simplifying and re-arranging the data layout, so that other processes can be carried out more efficiently. By doing so, this ensures quicker database responses connected with the visualization of data and the assigning of codes. The most important items of that stage included the following:
Calibration of chainage: Chainage is the basic mode of addressing/indexing railway line objects. Mapping the geographic coordinates to a Linear Referencing System (LRS), which is used in the SILK, can be made with the use of the Oracle Spatial software with dynamic segmentation functions (3D to 1D transformation). The present LRS model for railway lines in the SILK database was made with a cartographic accuracy of 1:25,000 maps. Therefore, a decision was made only to calibrate data based on a dozen or so points obtained from the SILK database.
Determination of track geometry: Based on GPS data, the examined railway line sections were divided into elementary sections of a length of 1 m each. Assuming that the turning radius is neglectable within such a section, the location of both rails was identified in the point cloud (together with rail height, which made it possible to make allowance for super elevation in further considerations). In this way, the local system of coordinates was determined. Furthermore:
the system’s center was precisely positioned vertically just over/on the rails in the Y axis, between the rails in the Z direction and halfway of the section (1 m) along the traveling direction (X) (right-handed system),
axis Y was positioned vertically on top of rails and pointing up (making allowance for super elevation),
axis X was perpendicular to Y and overlapping the vehicle travel direction,
was perpendicular to the XY
plane (see Figure 1
Reduction of data: In order to reduce the amount of 3D data stored in the database, a decision was made to limit the point cloud to strips along the travelling path, being 9.5 m wide (four to the left, four to the right side and 1.5 m as the distance between rails).
Considerations regarding the implementation of 2D cross-sections: For each identified 1-m section of the line, a projection of the point cloud upon the YZ plane was made. As each of such sections has its own system of reference, as defined above, and each of those systems is describing space from the viewpoint of a moving vehicle, it is possible to combine such cross-sections. The project participants decided to store in the database cross-sections from 10-m, 100-m and 1-km sections, as well as total cross-sections connected with encoding principles described in the International Union of Railways UIC 502-2 chart and dividing a railway line into segments (i.e., those between railway stations or turnouts).
3.4.1. Conception of Railway Clearance Gauge Determination
The conception of railway clearance gauge determination as designed in the IT solution is largely based on the 2D contours method.
As 3D cross-sections based on point clouds are considerably large in size, compression is necessary. Storing such cross-sections may prove to be impractical. On the other hand, the accuracy of such cross-sections (of a single point) is unnecessary from the point of view of the clearance gauge (and its codes). Therefore, as part of the project, an algorithm was developed to create 2D cross-sections on data from point clouds (through flattening of 3D point cloud data).
A 2D cross-section derived from a point cloud in the referencing system related to the track axis is created through dividing the whole 2D area into an exemplary 50 × 50 mm grid. If there is any point within the selected grid cell, the whole cell is considered to be occupied.
The algorithm makes such cross-sections at every 10, 100 and 1000 m within any section of the track. Next, those cross-sections get simplified by assuming a grid of 20 × 50 mm. The choice of which interval one should use depends on the kind/type of track (a straight line or turn) and the density of objects around the track (for a straight section with only fields, we use a 1-km cross-section; for dense building data and platforms, we use a 1-m cross-section).
That operation can be performed in an automatic manner. The application makes it also possible to combine cross-sections, creating a single cross-section within a section of several kilometers.
The task of the program operator is to review cross-sections and, if any conflict is identified (clearance gauge limits exceeded), to request the database to provide a more detailed cross-section or an image showing the suspicious area.
Based on the data, the operator may invalidate any element of the grid. The diagram of that conception is shown in Figure 2
shows an exemplary cross-section, made with a resolution of 20 × 50 mm. It can be seen that clearance gauge limits of the structure have been exceeded (dark grey color at the light grey background). In such a case, it is necessary to verify the reasons. Here, thanks to visualization, these are only some plants that one can see in the images.
In such a situation, the grid points may be unchecked as “invalid” (light grey).
3.4.2. Assigning of Codes
The project made use of the contour method originating from the UIC 502-2 chart. That method is marked by a more careful selection of base points.
In the database solution, there were procedures developed to implement that method. Furthermore, a method of the interactive assignment of codes to the railway line was developed. To maintain compatibility with the UIC chart, the possibility of dividing the railway line into sections was introduced. To facilitate the search, sections were assigned numbers (Figure 3
Furthermore, the possibility of checking if a cargo can be allowed for shipment was added, this was provided through the implementation of the critical points method. The points describe in an approximated way the shape of the cargo, however maintaining the rules to guarantee that the cross-section of the cargo would not be smaller than the actual shape of the cargo. The method that has been implemented makes it possible to calculate the cargo parameters, considering several calculation variants.
In the case of cargo, a procedure of automatic single code assigning was developed. Thanks to that procedure, the criterion of accepting the cargo for transport comes down to comparing the cargo code, railway line code and what is being transported: e.g., rail line Code 3-3544-465 allows for any cargo Codes 0-0444-425, but does not allow for 0-0466-425, as below in the Table 1
Through the extensions, it is possible to examine the encoding effectiveness through verification of the results that are based only on comparing codes to graphic visualizations of the route cross-sections and of the cargo available in the extended viewer.