Correction: Shao, Z.; et al. A Benchmark Dataset for Performance Evaluation of Multi-Label Remote Sensing Image Retrieval. Remote Sens. 2018, 10, 964

In our paper [1], we presented a dense labeling dataset that can be used for not only single-label and multi-label remote sensing image retrieval but also pixel-based problems such as semantic segmentation. Prof. Begum Demir first constructed and defined a multi-label archive in [2]. During the labeling of our dataset, we improved and referred to Prof. Demir’s multi-labels to construct our dense (pixel) labels. Therefore, our dataset can be viewed as an extension of Prof. Demir’s multi-labels. After publication of the paper [1], it was found that we did not clearly clarify the different contributions between our work and Prof. Begum Demir’s multi-labels in the published version of the paper. Hence, according to the Academic Editor’s suggestions, we decided to make some changes to the title and the text to make our contributions much clearer.

The changes we made are listed as follows:

1. The title of the article

We changed the title “A Benchmark Dataset for Performance Evaluation of Multi-Label Remote Sensing Image Retrieval” to “Performance Evaluation of Single-Label and Multi-Label Remote Sensing Image Retrieval Using a Dense Labeling Dataset”.

2. The abstract

The abstract in the revised version now reads “Benchmark datasets are essential for developing and evaluating remote sensing image retrieval (RSIR) approaches. However, most of the existing datasets are single-labeled, with each image in these datasets being annotated by a single label representing the most significant semantic content of the image. This is sufficient for simple problems, such as distinguishing between a building and a beach, but multiple labels and sometimes even dense (pixel) labels are required for more complex problems, such as RSIR and semantic segmentation. We therefore extended the existing multi-labeled dataset collected for multi-label RSIR and presented a dense labeling remote sensing dataset termed “DLRSD”. DLRSD contained a total of 17 classes, and the pixels of each image were assigned with 17 pre-defined labels. We used DLRSD to evaluate the performance of RSIR methods ranging from traditional handcrafted feature-based methods to deep learning-based ones. More specifically, we evaluated the performances of RSIR methods from both single-label and multi-label perspectives. These results demonstrated the advantages of multiple labels over single labels for interpreting complex remote sensing images. DLRSD provided a benchmark for RSIR and other pixel-based problems such as semantic segmentation”.

3. The term “MLRSIR”

We replaced the term “MLRSIR” in the previous version with “DLRSD” throughout the text.

4. The text

In Section 1, we changed the text “We construct a multi-label remote sensing benchmark dataset, MLRSIR, for multi-label RSIR. MLRSIR is a publicly available dataset, which is a multi-labeled dataset in contrast to the existing single-labeled RSIR datasets”. to “We construct a dense labeling remote sensing dataset, DLRSD, for multi-label RSIR. DLRSD is a publicly available dataset, which is a dense labeling dataset in contrast to the existing single-labeled and multi-labeled RSIR datasets”.

In Section 3 (before Section 3.1), we changed the text “We therefore propose a new pixel-wise labeling dataset termed MLRSIR for multi-label RSIR that can be used for not only unsupervised and semi-supervised approaches but also supervised approaches like FCN”. to “We therefore improve and extend the multi-labeled archive [33], and present a dense (pixel-wise) labeling dataset termed DLRSD for multi-label RSIR that can be used for not only unsupervised and semi-supervised approaches but also supervised approaches like FCN”.

In Section 3.1, we changed the text “To be consistent with the multi-label RSIR archive [33], the total number of distinct class labels associated for MLRSIR was also 17. The eCognition 9.0 (http://www.ecognition.com/) software was used to segment each image in the UC Merced archive [9] into a number of semantically meaningful regions, and then each region was assigned one of 17 pre-defined class labels”. to “To be consistent with the multi-label RSIR archive [33], the total number of distinct class labels associated for DLRSD was also 17. The eCognition 9.0 (http://www.ecognition.com/) software was used to segment each image in the UC Merced archive [9] into a number of semantically meaningful regions, and then each region was assigned one of 17 pre-defined class labels. It is worth noting that the 17 classes are first defined and used in the multi-label archive [33]. During the labeling of our dataset, we first improved the multi-labels that we thought inaccurate through visual inspection, and then referred to the revised multi-labels to label our dataset”.

We also changed the text “MLRSIR was a pixel-wise labeled dataset with each image containing multiple labels, therefore, it could also be used for other tasks, such as semantic segmentation (also called classification in remote sensing) and multi-label classification, i.e., predicting the classes contained in an image. MLRSIR is available at https://sites.google.com/view/zhouwx/dataset”. to “Unlike the multi-label RSIR archive mentioned above, DLRSD was a pixel-wise (dense) labeled dataset with the pixels of each image assigned different labels, therefore, it could also be used for other tasks, such as semantic segmentation (also called classification in remote sensing) and multi-label classification, i.e., predicting the classes contained in an image. DLRSD is an extension of the UC Merced archive and particularly the multi-label archive [33]. DLRSD has been released for research and is available at https://sites.google.com/view/zhouwx/dataset”.

5. The conclusion

We changed the conclusion “In this paper, we proposed a benchmark dataset named MLRSIR for multi-label RSIR. We expect MLRSIR to help advance the development of RSIR approaches, particularly supervised-learning-based methods. We also compared the performance of single-label and multi-label RSIR on MLRSIR based on handcrafted and CNN features. MLRSIR is collected for RSIR and particularly multi-label RSIR, but it can also be used for other problems such as semantic segmentation”. to “In this paper, we presented a dense labeling benchmark named DLRSD. We expected DLRSD to help advance the development of RSIR approaches, particularly supervised learning-based methods such as FCN. We also compared the performance of single-label and multi-label RSIR on DLRSD based on handcrafted and CNN features. DLRSD can be used for not only single-label and multi-label RSIR, but also pixel-based problems such as semantic segmentation”.

The authors apologize for any inconvenience caused to the readers by these changes. The changes do not affect the scientific results. The original manuscript will be updated and will remain online on the article webpage, with a reference to this Erratum.