4.1. Grayscale Representation
Converting images to grayscale representation is a common way of turning a 3-channel image into a single channel image. In Figure 9
, the images representing each case are shown in grayscale representation.
The laboratory bare road case (Figure 9
a) represents an ideal situation for lane detection: new road markings, new asphalt, and good lighting. The markings are clearly visible in grayscale as they were in the RGB version of the image, with the yellow markings appearing slightly grayer in the grayscale image, as expected. In the laboratory road with a 0.5 cm layer of snow (perspective) (Figure 9
b), and the laboratory road with a 0.5 cm layer of snow (bird’s-eye view) (Figure 9
c), the differences lie in the camera used and the viewing angle. The road markings are difficult to visually detect in the perspective view, where the camera was placed at an angle representative to that of a camera close by the rear-view mirror. When the image is taken from directly overhead in the bird’s-eye view, the white lane lines are visible in the lower part of the image. In the upper half of the photo, the yellow markings again appear grayer than the white markings, making these road markings difficult to see.
In the airfield having a 2.5 cm layer of snow (Figure 9
d), the road markings are not visible. In the images taken after the snow was removed by plowing (Figure 9
e) and brushing (Figure 9
f), the yellow road markings are clearly visible in the RGB image (Figure 3
). The same markings are less visible in the grayscale images, especially in Figure 9
e (after plowing).
The public road in the afternoon (Figure 9
f) shows a snowy public road in low ambient light. Both the yellow lane marking on the left-hand side and the white road marking on the right-hand side are as visible in the grayscale representation as they were in RGB color space. However, the markings and snow are of similar intensity in the images, which might make the edge between the road surface and road markings challenging to identify.
In summary, the road markings appear distinct and much lighter in color than the adjacent asphalt does when the road is bare, but a visual inspection of the images indicates that lane detection may be more problematic under snow cover for the conventionally used RGB and grayscale images. Both white and yellow road markings have pixel intensities in grayscale images that are similar to parts of the snow coverage. This is especially evident in the grayscale image showing the airfield after plowing (Figure 9
In the following section, the images are assessed in the color spaces RGB, HSL, HSV, and YUV by visual inspection and the use of histograms. Based on the visual inspection of the cases in RBG and grayscale presentations, the laboratory road having a layer of 0.5 cm snow (perspective) and the airfield having a layer of 2.5 cm of snow will be omitted. In the first instance, the bird’s-eye image makes for a better comparison between the white and yellow markings, as these appear at the same distance and in equal quantities in the image. Regarding the airfield with a 2.5 cm layer of snow, the road markings are not visible and, therefore, do not provide additional information in other color spaces or corresponding histograms.
4.2. Color Space Representation
The images analyzed are the laboratory having a 0.5 cm layer of snow (bird’s-eye view), the airfield after plowing, the airfield after brushing and the public road in the afternoon. The images are shown in these four color spaces: RGB, HSL, HSV, and YUV. They are also shown in their respective channels.
The laboratory bare road image (Figure 10
) shows an ideal situation for lane detection. For instance, in the upper part of these images, there are yellow markings. In the lower part, there are white markings, and the transition between the marking colors is indicated by the red dashed line. Figure 10
shows the four color representations: RGB, HSL, HSV, and YUV in the top row along with the three separate channels they are made up of in the color spaces’ respective columns. A mask has been added manually to focus on the road and lane markings rather than on the adjacent metal edges in the lab setting. In the case of RGB, the white marking is visible in all channels, while the yellow seems most prominent in the R channel, slightly less so in the G and not very prominent in the B channel. In the case of the HSV color space, neither white or yellow marking is visible in the H-channel, the yellow marking is clear while the white marking is more muted in the S-channel, and, in contrast, most prominent in the V-channel. In the case of the HSL representation, the H- and S-channels show similar results to the HSV color space. On the other hand, the HSV-V channel shows the yellow marking more clearly than the HSL-S channel. In the rightmost column (the YUV representation of the image), the Y-channel shows both markings clearly, while the U- and V-channels highlight the yellow marking. The difference between the YUV-U and YUV-V channels is that the former represents the yellow marking as the darkest part of the image, while, conversely, the latter represents it as the lightest part of the image. This difference makes the YUV-U histogram form a trough representing the marking, while the YUV-V channel shows the road marking as a peak.
Next, the images featuring different snow coverages will be presented in the respective color spaces. First, the laboratory road with 0.5 cm snow coverage from a bird’s-eye view is shown (Figure 11
). There is a white marking in the lower corner of the image and a yellow marking above the red dashed line. The effect of the different color space representation is the same for the laboratory road having a 0.5 cm layer of snow (bird’s-eye view) as it was for the previous case, the bare road model. However, the snow cover makes the markings that appeared clearly on the bare road model challenging to see in the case where there is 0.5 cm of snow coverage (bird’s-eye view). In the bare road image, the four channels enhanced only the yellow marking: HSL-S, HSV-S, YUV-U, and YUV-V. Interestingly, these channels seem to work even better for the 0.5 cm snow coverage, but only when the yellow marking is present. The white elements, represented by the white road marking and snow, are no longer clearly visible.
The next 2 figures show the airfield images in different color spaces. The plowed airfield (Figure 12
) is presented, followed by the brushed airfield (Figure 13
). In both of these two images, there is only a yellow marking.
Regarding the plowed airfield image, the yellow road marking is clearly visible to the human eye in the RGB image. However, when considering this image’s separate channels, the marking is more difficult to detect. In the RGB-R channel, the tire tracks made by the tractor is clearer than the road marking. In the RGB-G channel it is difficult to distinguish the road markings, tire tracks, and other elements of snow, while in the RGB-B channel, the markings are visible and similar in pixel value to the right edge of the snow removal area. As in the previous two images, the channels that look most promising in terms of visibility and contrast to the yellow marking are HSL/HSV-S, YUV-U and YUV-V. Conversely, HSL-L and HSV-V channels are not suited to enhance the road markings, while the HSL/HSV-S channel in this image does show the markings, it does so with low contrast to the road surface on the right-hand side.
The image of the brushed airfield is shown in Figure 13
In the brushed airfield image, the brushing has removed most of the snow on the right-hand side of the yellow road marking, while the snow coverage on the left-hand side of the marking has remained mostly intact. Having snow on one side of the yellow marking and a dark road surface on the other creates an interesting situation in terms of establishing contrast between the road and road marking in images. Although the yellow color may be clearly seen in the RGB image, when considering the separate channels usually used in analyses, the contrast is low between the snow and the markings in the RGB-R channel, and, conversely, between the markings and road surface in the RBG-B channel. In the RGB-G channel, the road markings provide a contrast to the snow and road surface, but the pixel values are also similar to the snow, which makes it challenging to separate these markings from longitudinal snow elements. In the HSL image, the markings are difficult to separate from the snow; however, the HSL-H and HSL-S channels highlight the yellow marking in light versus dark pixel representation. The same channels in the HSV image, HSV-H and HSV-S, show similar results, while the HSV image provides a better separation of road markings to snow and road surface than does the HSL image. The HSV-V channel and HSL-L channel are not optimal for enhancing the yellow marking, as in the laboratory road with a 0.5 cm layer of snow (birds-eye image) and the airfield after plowing image. The YUV image representation provides an identifiable color for the yellow marking, similar to the RGB and HSV images. Considering the separate channels, the YUV-Y channel is poorly suited to detecting lane markings, while the YUV-U and YUV-V channels separate the yellow marking from both the snow and dark road surface in opposite ways.
The final scenario, public road in the afternoon, considers an image taken in the afternoon (15:30) on a public road (Figure 14
). There is a yellow road marking on the left-hand side and a white road marking on the right-hand side. Even though the white road marking in Figure 14
is visible in all the RGB channels, the contrast to the snow next to it is not strong. The H-channels from the HSL and HSV representations are inept at showing the lane markings. Moreover, in the S-channels for these two color spaces, the yellow marking is visible, but not the white. The HSL-L and HSV-V channels provide similar results to the RGB channels, where the lines are visible yet have low contrast to other elements in the scene. The YUV and YUV-Y representations of the image are not favorable for locating the lane markings, while the YUV-U and YUV-V channels provide what seems like the strongest contrast between the yellow road marking and the snow and road surface.
In summary, a consistent set of color channels: HSL-S, HSV-S, YUV-U, and YUV-V, seem to amplify the visibility of the yellow marking in the four images of different snow conditions (the laboratory with a 0.5 cm layer of snow (bird’s-eye view), the plowed airfield, the brushed airfield and the public road in the afternoon). The brushed airfield strip image shows snow on the left-hand side of the road marking and an almost bare road on the right-hand side of the marking. In this case, the color spaces HSL/HSV-H also set the road marking apart from the rest of the image. Regarding the images with white markings (the laboratory with a 0.5 cm layer of snow (bird’s-eye view) and the public road in the afternoon), the highest visibility was observed in the three RGB-channels: YUV-Y, HSV-L, and HSV-V.
The visual analyses of the visibility of white and yellow road markings in snowy conditions are summarized in Table 2
shows that the color channels providing the highest visibility overall are HSL-S, HSV-S, YUV-U, and YUV-V, as in these instances the visibility in snowy conditions is higher for yellow markings than for white markings. Table 2
summarizes a subjective way of visually analyzing images. The following section will, therefore, establish an objective assessment of the road markings’ visibility using histogram plots.
4.3. Histograms of Pixel Values
Whether lane markings are detected by thresholding or using gradients, lane detection algorithms generally rely on distinct changes in pixel values to establish edges. The four images of snowy conditions described above have, therefore, been assessed according to changes in pixel values through sets of histogram plots. A plot is produced by adding the individual pixel intensities for each pixel column corresponding to a given color channel. The pixel column is on the x-axis and the summed pixel values on the y-axis. The aim is to achieve a clear indication of where the road markings are located in the image as shown by a distinct rise or fall in the sum of pixel values. When the pixels representing road markings are light in color, they have high intensity values; this creates, in turn, high sums and, thus, peaks in the plot. In instances where a road marking appears as the darkest part of an image, the road marking pixels have a low sum and should, therefore, create a visible trough in the plot. The more distinct the peak or trough is in the plot, the more ideal the image is for lane detection. Plots with no clear peaks or troughs mean that it is challenging to identify the road marking in the image. The next sections will first present the traditional representations, RGB and grayscale, and then the alternative representations, HSL, HSV, and YUV, for the selected images containing snow and visible markings: the laboratory with a 0.5 cm layer of snow (bird’s-eye), the plowed airfield, the brushed airfield, and the public road in the afternoon.
4.3.1. The Laboratory Road with a 0.5 cm Layer of Snow (Birds-Eye View)
Regarding the laboratory road with a 0.5 cm layer of snow (bird’s-eye), the top half of the image has yellow markings, and the bottom image has white markings. To compare the white and yellow markings, two histogram plots are made: one for the lower half (white markings) and one for the upper half (yellow markings). In Figure 15
, the RGB-channels and grayscale representation are shown. In these histograms, there is a peak on the right side of the image (where the continuous lane marking is) for both white and yellow markings. The dashed line does not produce a peak higher than those on the road surface. The peak is most distinct in RGB-R for both colors, while RGB-G and -B also provide peaks that are for white markings. The RGB-B channel has a trough for the yellow continuous marking, a factor which is difficult to discern from the rest of the minima in the plot. The grayscale histograms show peaks for both white and yellow markings; however, the white marking peak is significantly more prominent than the yellow marking peak. Again, only the continuous lines are detected in the histogram plots.
shows the histograms for the channels of the HSL, HSV, and YUV color spaces. The histograms on the left-hand side show all channels, and the histograms on the right-hand side show the channels that appear to be most suited for detecting the lane markings in terms of pixel value changes. In the HSL and HSV plots, the HSL-S channel has been highlighted as it shows distinct peaks for both the dashed and continuous lines. The HSL-L channel only has a peak for the continuous white line. In the top right-hand side plot, the differences in visibility for the yellow markings are evident, the yellow markings providing clear peaks for both dashed and continuous lines.
The same effect is seen in the HSV plots, where the HSV-S plots have very distinct peaks for the road markings and low sums for the columns representing the rest of the image. Regarding the YUV representations, the V- and U-channels show the most distinct peaks, where the road markings have significantly higher values than the surrounding surfaces. In these channels, both the dashed line (left) and the continuous line (right) can be detected, a contrast with the RGB and grayscale histograms, which only detected the continuous lines. The YUV-U channel shows the road markings appear as troughs; but in this case, as opposed to the RGB-B plot, the troughs are identifiable as local/global minima. The histogram plots are consistent with the previous section’s findings (Table 2
), where the HSL/HSV-S and YUV-U/V channels provided the highest visibility of the yellow marking in snowy conditions.
4.3.2. The Plowed Airfield
In the cases of the airfield after plowing and after brushing, there is only a continuous yellow marking. The histograms have been created based on the lower half of the images, focusing on the area of the image with the lane markings. Figure 17
shows the histograms for the airfield after plowing image as represented by RGB and grayscale images. In this case, it is not possible to separate a threshold or peak that represents the road marking from the surroundings in either the RGB or grayscale histogram plots.
In Figure 18
, the histograms for the HSL, HSV, and YUV representation of the plowed airfield image are shown. On the left-hand side the three channels of the color spaces are plotted, while the right-hand side highlights the channels that provide the best detection of lane markings. In the HSL representation, the S-channel provides the most prominent peak, which is also true for the HSV histogram. In the airfield after plowing image, the YUV-U and YUV-V channels also produce a distinct trough and peak, respectively; significantly, this is consistent with the findings from the visual inspection.
4.3.3. The Brushed Airfield
The RGB and grayscale histograms for the brushed airfield image are shown in Figure 19
. As in the previous case, the plowed airfield, these representations are not well suited for detecting the single continuous yellow road marking.
In the HSL, HSV, and YUV representations in Figure 20
, the H- and S-channels of the HSL and HSV color spaces, as well as the U- and V-channels of the YUV representations, all show identifiable peaks for the yellow road marking. Regarding the HSL and HSV color spaces, the H-channel is particularly successful at isolating the road marking as the only peak that contrasts with both the snow and almost bare road. When the road marking pixels form the clear local or global maxima, the image representation is well suited for lane detection as there are no peaks that can be misidentified as road markings. This echoes the result seen in the summary of the visual inspection in Table 2
4.3.4. The Public Road in the Afternoon
The public road in the afternoon image has a yellow dashed line on the left-hand side and a white continuous line on the right-hand side. In this case, the part of the image used for creating the histograms is the very lower end of the image, as indicated below the white line in Figure 21
. This image section provides a continuous section of both yellow and white marking. The RGB and grayscale histograms are also shown in Figure 21
. The histograms for these conventionally used image representations are not suited for identifying either the white or yellow marking.
The HSL, HSV, and YUV histograms for the public road in afternoon image are shown in Figure 22
. In this case, the only channels that provide visible peaks are the HSL-H and HSV-H channels. A peak large enough to separate itself from the rest of the plot is seen on the left-hand side, i.e., stemming from the yellow road marking, while the white road marking’s pixel values are not distinguishable from their surrounding environment. In the public road in afternoon image, the YUV channels are not able to pick out any road marking.
The visibility of the white and yellow road markings based on the histogram analyses has been summarized in Table 3
. The results from the analyses of the histogram plots are in line with the findings from the visual inspection (Table 2
). In both cases, the color channels HSL-S, HSV-S, YUV-U, and YUV-V perform the best in identifying lane markings in snowy conditions. However, in the public road image the YUV-U and V channels do not provide identifiable peaks for the white or yellow markings.