North American Hardwoods Identification Using Machine-Learning
Round 1
Reviewer 1 Report
Title – good.
Abstract – good.
Keywords – consider adding “wood identification” to list.
Introduction – Overall this section is awkward to read and lacks logical progression. Consider the main points you want to make to convince reader that challenges in wood identification necessitate the innovation of machine learning. The Introduction should be rewritten to have more coherent and logical flow.
Line 23 – term “human lumber graders” is not only awkward but are they the only individuals grading wood? A very specific way to begin the introduction.
Line 27 – “correctly apply the number of reference base to analyze the full range of variability” is badly written and incomprehensible. Particularly this last point is problematic as you cover a range of pragmatic aspects of wood identification.
Line 28. I would argue that correct wood identification is of “major international importance” but not because consumers need to know what they are purchasing but because of illegal logging mentioned in the next paragraph. This entire first paragraph is oddly specific and generally not that helpful for setting stage for the paper.
Line 36-37 – this sentence does not flow from the previous sentences. From illegal logging and misrepresentation the authors veer to annual US export facts. This and the following sentence on training a new group of professionals fits much better in the first paragraph. I question the need to create IL and MR acronyms given they are only used once and twice, respectively in the remainder of the manuscript.
Line 46 – has critical need correct wood identification for decision making been identified by the industry procuring or using wood mats? It would seem that only wood species that were appropriate for wood mats would be used in their production. This seems like a weak justification for innovation. To repeat sentiment above, the Introduction needs to be rewritten.
Materials and Methods – Overall this section is concise and clear.
Line 64 – "Department of Sustainable Bioproducts"…. at what institution?
Line 68 – Is there a missing end bracket after Nees?
Line 70 – replace “with” with “has”…more than 25 years in the field.
Line 73 – Tense. Replace “is” with “was”
Line 86 – replace “careful” with “carefully”
Line 88 – Add “A camera” flash was not used.
Line 175 – You state that “ultimate goal is to create a machine-learned model of all commercially available North American hardwoods and softwoods and make that model available as a smartphone-based wood identification app. “ What are some challenges of doing this from point of view of computer storage (images), computing requirements, model accuracy with increasing portfolio of species? Some discussion of challenges to expanding tool to a full app should be explored here.
Conclusion – some discussion of who is intended end user of fully developed app would be appropriate in this section.
Figures and Tables – good
Author Response
Title – good.
Thank you
Abstract – good.
Thank you
Keywords – consider adding “wood identification” to list.
Thank you
Introduction – Overall this section is awkward to read and lacks logical progression. Consider the main points you want to make to convince reader that challenges in wood identification necessitate the innovation of machine learning. The Introduction should be rewritten to have more coherent and logical flow.
We thank the reviewer. We basically reformulated all paragraphs and added two more to supplement the intro.
Line 23 – term “human lumber graders” is not only awkward but are they the only individuals grading wood? A very specific way to begin the introduction.
Fixed
Line 27 – “correctly apply the number of reference base to analyze the full range of variability” is badly written and incomprehensible. Particularly this last point is problematic as you cover a range of pragmatic aspects of wood identification.
Fixed
Line 28. I would argue that correct wood identification is of “major international importance” but not because consumers need to know what they are purchasing but because of illegal logging mentioned in the next paragraph. This entire first paragraph is oddly specific and generally not that helpful for setting stage for the paper.
Fixed
Line 36-37 – this sentence does not flow from the previous sentences. From illegal logging and misrepresentation the authors veer to annual US export facts. This and the following sentence on training a new group of professionals fits much better in the first paragraph. I question the need to create IL and MR acronyms given they are only used once and twice, respectively in the remainder of the manuscript.
Fixed
Line 46 – has critical need correct wood identification for decision making been identified by the industry procuring or using wood mats? It would seem that only wood species that were appropriate for wood mats would be used in their production. This seems like a weak justification for innovation. To repeat sentiment above, the Introduction needs to be rewritten.
Fixed
Materials and Methods – Overall this section is concise and clear.
Line 64 – "Department of Sustainable Bioproducts"…. at what institution?
Fixed
Line 68 – Is there a missing end bracket after Nees?
Fixed
Line 70 – replace “with” with “has”…more than 25 years in the field.
Fixed
Line 73 – Tense. Replace “is” with “was”
Fixed
Line 86 – replace “careful” with “carefully”
Fixed
Line 88 – Add “A camera” flash was not used.
Fixed
Line 175 – You state that “ultimate goal is to create a machine-learned model of all commercially available North American hardwoods and softwoods and make that model available as a smartphone-based wood identification app. “ What are some challenges of doing this from point of view of computer storage (images), computing requirements, model accuracy with increasing portfolio of species? Some discussion of challenges to expanding tool to a full app should be explored here.
Fixed
Conclusion – some discussion of who is intended end user of fully developed app would be appropriate in this section.
We added the info as requested
Figures and Tables – good
Thank you
Reviewer 2 Report
general comments-
*I am not clear on whether the subject of this paper is intended to be a field app/tool , or a laboratory tool, or a tool for use in sawmills- I think the authors need to be clear on the setting for this.
*From my limited work in image analysis / processing, it seems to me that lighting (and lighting intensity) is a critical aspect of feature identification. However I don't see this topic given as much attention as perhaps it is due. Do the authors have a way to acccurately calibrate lighting so that features are recognized consistently from sample to sample? This could be especially important if the tool is used in the field, where daylight conditions could vary from sample to sample.
specific comments-
Line 34- would this method have field applicability?
Line 39- I would suggest writing out IL and MR rather than using abbreviations.
Line 46- would this statement suggest that the tool/app is intended to be used before manufacture? if so, would the tool be used most often in a sawmill or other production facility?
Line 51- may want to include a sentence or 2 to build the conceptual framework- what is a CNN?
Line 55- it might be worth mentioning here some of the limitations of AI as well
Line 67- perhaps a good starting point for the ID algorithm would be to classify as ring porous, diffuse porous, or semi-ring porous
Line 68- how would tyloses (light reflecting) vary from normal vessels in terms of feature identification?
Line 71- this seems like a lot of images- is this a statistically significant number?
Line 88- food for thought-as an extension of this research, could you take several images, then tile together for a full view of the end grain?
Lines 102-115- I believe that most readers of "Forests" journal will have no idea the specifics of what is being discussed here- this is OK, however I would only suggest defining all abbreviations, and wherever possible adding a sentence or two as a conceptual framework / definition to make things more familiar to readers.
Line 118- is this an error? range from 341 to 342? seems like a very narrow range
Line 123- grammar note- please add a comma "....deep learning models, overfitting can...."
Line 127- Fig. 4- please write out abbreviations- "validation" "accuracy"
Line 181- would sample preparation also take place in the field? (including wood surface preparation?)
Author Response
general comments-
*I am not clear on whether the subject of this paper is intended to be a field app/tool , or a laboratory tool, or a tool for use in sawmills- I think the authors need to be clear on the setting for this.
We added a statement saying our intention in developing a mobile app.
*From my limited work in image analysis / processing, it seems to me that lighting (and lighting intensity) is a critical aspect of feature identification. However I don't see this topic given as much attention as perhaps it is due. Do the authors have a way to acccurately calibrate lighting so that features are recognized consistently from sample to sample? This could be especially important if the tool is used in the field, where daylight conditions could vary from sample to sample.
Great thinking. Using sunlight as the default lighting conditions "normalized" the level of illumination for all species. Your comment raised a really good question. What would happen to this identification tool when no sunlight is present? Flash would have to be used. We plan to carefully examine this and include it in the final project. Thank you
specific comments-
Line 34- would this method have field applicability?
Yes, smartphones are widely available and macro lens are relatively inexpensive, which make a perfect combination for field applications.
Line 39- I would suggest writing out IL and MR rather than using abbreviations.
Fixed
Line 46- would this statement suggest that the tool/app is intended to be used before manufacture? if so, would the tool be used most often in a sawmill or other production facility?
We anticipate that this tool could be used at borders, at home, sawmills, matting yards, rail road industry.
Line 51- may want to include a sentence or 2 to build the conceptual framework- what is a CNN?
Fixed
Line 55- it might be worth mentioning here some of the limitations of AI as well
Fixed
Line 67- perhaps a good starting point for the ID algorithm would be to classify as ring porous, diffuse porous, or semi-ring porous
This is a nice idea. We would have to change the final softmax function and classify all species into the bins. However, the classification would be limited to these arguments (ring porous, diffuse porous or semi-ring porous).
Line 68- how would tyloses (light reflecting) vary from normal vessels in terms of feature identification?
Because of tyloses show a light reflecting effect, the pixel intensity value in these samples would be higher, compared to pure black from normal species. Thereby, the feature identification would be relatively facilitated.
Line 71- this seems like a lot of images- is this a statistically significant number?
Deep learning models are heavily dependent of huge datasets. For example, in 201, Krizhevsky used 16 million images. In fact, our dataset is considered limited and we had to leverage stratified k-fold cross-validation to ameliorate this issue.
Line 88- food for thought-as an extension of this research, could you take several images, then tile together for a full view of the end grain?
This is a great idea. Tiling them together would be hard to be done, though. Mainly to accurately put them together. What could be done is to make use of panoramic built-in functions. However, brusque movement would blur the image.
Lines 102-115- I believe that most readers of "Forests" journal will have no idea the specifics of what is being discussed here- this is OK, however I would only suggest defining all abbreviations, and wherever possible adding a sentence or two as a conceptual framework / definition to make things more familiar to readers.
We defined CNN once again in the beginning of the paragraph for clarity. Some terms like VGG16 and RMSprop have no definition. VGG16 is an acronym for visual geometry group in the UK. RMSprop is a version of the stochastic gradient descent developed by Hinton in an unpublished work. We also defined GPU.
Line 118- is this an error? range from 341 to 342? seems like a very narrow range
This is not an error. Stratified k-fold cross-validation takes a percentage of data per each class per each fold. What happened was that in a specific fold the technique picked an extra sample from a determined class.
Line 123- grammar note- please add a comma "....deep learning models, overfitting can...."
Fixed
Line 127- Fig. 4- please write out abbreviations- "validation" "accuracy"
Fixed
Line 181- would sample preparation also take place in the field? (including wood surface preparation?)
It didn't take place in the field. We simulated what an observer would have to do in the field to use the app.
