Sustainable Technology Analysis Using Data Envelopment Analysis and State Space Models

Round 1

Reviewer 1 Report

The major revisions that are required are conformance to the required format for the References on pages 19 to 21. There is inconsistent format in the that used for the titles of journals. For example: References 8, 9, 24, 25, 26 use italics for the Title of Journals while References 1, 2, 3, 4, 5, 7, 12, 13, 14, 15, 17, 21, 22, 28, 30, 32, 35, and 36 do not. According to the format rules for the Sustainablity Journal the titles of Journals and Book titles need to be italics. Hence the book titles in References such as 10 and possibly elsewhere need to be in italics also.

The authors need to carefully check each of the references and convert to the formats as below as obtained from the Sustainability Journal website at:

https://www.mdpi.com/journal/sustainability/instructions 

References should be described as follows, depending on the type of work:

Journal Articles:
1. Author 1, A.B.; Author 2, C.D. Title of the article. Abbreviated Journal Name Year, Volume, page range. Available online: URL (accessed on Day Month Year).

Books and Book Chapters:
2. Author 1, A.; Author 2, B. Book Title, 3rd ed.; Publisher: Publisher Location, Country, Year; pp. 154–196.
3. Author 1, A.; Author 2, B. Title of the chapter. In Book Title, 2nd ed.; Editor 1, A., Editor 2, B., Eds.; Publisher: Publisher Location, Country, Year; Volume 3, pp. 154–196.

Unpublished work, submitted work, personal communication:
4. Author 1, A.B.; Author 2, C. Title of Unpublished Work. status (unpublished; manuscript in preparation).
5. Author 1, A.B.; Author 2, C. Title of Unpublished Work. Abbreviated Journal Name stage of publication (under review; accepted; in press).
6. Author 1, A.B. (University, City, State, Country); Author 2, C. (Institute, City, State, Country). Personal communication, Year.

Conference Proceedings:
7. Author 1, A.B.; Author 2, C.D.; Author 3, E.F. Title of Presentation. In Title of the Collected Work (if available), Proceedings of the Name of the Conference, Location of Conference, Country, Date of Conference; Editor 1, Editor 2, Eds. (if available); Publisher: City, Country, Year (if available); Abstract Number (optional), Pagination (optional).

Thesis:
8. Author 1, A.B. Title of Thesis. Level of Thesis, Degree-Granting University, Location of University, Date of Completion.

Websites:
9. Title of Site. Available online: URL (accessed on Day Month Year).
Unlike published works, websites may change over time or disappear, so we encourage you create an archive of the cited website using a service such as WebCite. Archived websites should be cited using the link provided as follows:
10. Title of Site. URL (archived on Day Month Year).

===============================================================

Other revisions include subdividing lengthy paragraphs such as that in the 1. Introduction that extends from lines 29 to 64 as a single paragraph and making reading comprehension difficult. This paragraph and possibly others in this manuscript need to be subdivided into smaller paragraphs to make these easier to comprehend by the reader.

Author Response

Reviewer 1

We really appreciate your invaluable comments for improving the quality of our paper. We supplemented contents of our paper according to your comments.

 

The major revisions that are required are conformance to the required format for the References on pages 19 to 21. There is inconsistent format in the that used for the titles of journals. For example: References 8, 9, 24, 25, 26 use italics for the Title of Journals while References 1, 2, 3, 4, 5, 7, 12, 13, 14, 15, 17, 21, 22, 28, 30, 32, 35, and 36 do not. According to the format rules for the Sustainablity Journal the titles of Journals and Book titles need to be italics. Hence the book titles in References such as 10 and possibly elsewhere need to be in italics also.

The authors need to carefully check each of the references and convert to the formats as below as obtained from the Sustainability Journal website at:

https://www.mdpi.com/journal/sustainability/instructions

References should be described as follows, depending on the type of work:

Journal Articles:

 1. Author 1, A.B.; Author 2, C.D. Title of the article. Abbreviated Journal Name Year, Volume, page range. Available online: URL (accessed on Day Month Year).

Books and Book Chapters:

 2. Author 1, A.; Author 2, B. Book Title, 3rd ed.; Publisher: Publisher Location, Country, Year; pp. 154–196.

 3. Author 1, A.; Author 2, B. Title of the chapter. In Book Title, 2nd ed.; Editor 1, A., Editor 2, B., Eds.; Publisher: Publisher Location, Country, Year; Volume 3, pp. 154–196.

Unpublished work, submitted work, personal communication:

 4. Author 1, A.B.; Author 2, C. Title of Unpublished Work. status (unpublished; manuscript in preparation).

 5. Author 1, A.B.; Author 2, C. Title of Unpublished Work. Abbreviated Journal Name  stage of publication (under review; accepted; in press).

6. Author 1, A.B. (University, City, State, Country); Author 2, C. (Institute, City, State, Country). Personal communication, Year.

Conference Proceedings:

 7. Author 1, A.B.; Author 2, C.D.; Author 3, E.F. Title of Presentation. In Title of the Collected Work(if available), Proceedings of the Name of the Conference, Location of Conference, Country, Date of Conference; Editor 1, Editor 2, Eds. (if available); Publisher: City, Country, Year (if available); Abstract Number (optional), Pagination (optional).

Thesis:

 8. Author 1, A.B. Title of Thesis. Level of Thesis, Degree-Granting University, Location of University, Date of Completion.

Websites:

 9. Title of Site. Available online: URL (accessed on Day Month Year).

Unlike published works, websites may change over time or disappear, so we encourage you create an archive of the cited website using a service such as WebCite. Archived websites should be cited using the link provided as follows:

10. Title of Site. URL (archived on Day Month Year).

-> We corrected the references by the formats of Sustainability journal.

 

Other revisions include subdividing lengthy paragraphs such as that in the 1. Introduction that extends from lines 29 to 64 as a single paragraph and making reading comprehension difficult. This paragraph and possibly others in this manuscript need to be subdivided into smaller paragraphs to make these easier to comprehend by the reader.

-> We modified all paragraphs of this paper to read it easily.

Reviewer 2 Report

Dear authors

This manuscript uses well known methodologies. The data used is not recent and despite the fact that the analysis is quite detailed, there is not scientific soundness. therefore, my recommendation is this manuscript to be rejected.

Author Response

Reviewer 2

We really appreciate your invaluable comments for improving the quality of our paper. We supplemented contents of our paper according to your comments.

 

This manuscript uses well known methodologies. The data used is not recent and despite the fact that the analysis is quite detailed, there is not scientific soundness. therefore, my recommendation is this manuscript to be rejected.

-> DEA is a popular methodology in operational research and economics. But, there is no one yet to do sustainable technology analysis by applying this method. In this paper, we proposed a methodology for sustainable technology analysis using additive model and directional distance function of DEA. This is the first attempt in sustainable technology analysis. In addition, we used state space model (SSM) as another method for sustainable technology analysis. We combined the results from DEA and SSM to find sustainable areas of target technology.

In our case study, we applied the proposed methodology to real domain. Our goal is to demonstrate the applicability and performance of our proposed model rather than Apple's technology analysis. Therefore, we conducted experiments using only patent data that we could collect. Technology analysis using recent patent documents can also be carried out according to the procedure performed in this case study.

Reviewer 3 Report

Authors propose a hybrid method based on additive DEA and SSM to assess technology sustainability. Finally, the use a real case to test the method.

In my view, this paper has several drawbacks.

Grammar, style and English:

The paper should be revised by a native-English expert. Indeed, some paragraphs should be re-written as it is not always easy to understand what authors mean or there are many style and grammar errors. Here are some examples:

Lines 29-31: “From the result of technology forecasting, a company performs diversely technological innovation to improve his technological competitiveness.” Please, check style and grammar.

Lines 42-43: “It has used to find empirically measure productive efficiency of decision making units." Please, check style and grammar.

Lines 39-44: In this paragraph authors repeat the same idea, i.e. the use of DEA. Please, do not be redundant.

Methodology:

The authors should present the model more in depth. What are the inputs and outputs of the DEA model? As I understand, the authors use keywords as input/output variables. However, input and output variables should belong to two different sets. Please, discuss such issue.

What is the measurement unit of the input and output variables? Patents are codified using keywords. How these latter can be used as input and output measurements? How can mathematical operations be performed with these variables?

Why do authors use an additive DEA model? Why do they introduce bad outputs? What are the bad outputs in their efficiency-measurement conceptualization?

How many DMUs are used? Is the individual patent associated to an individual DMU?

Literature:

There is no in-depth literature review. Please, include a section that presents literature.

Author Response

Reviewer 3

We really appreciate your invaluable comments for improving the quality of our paper. We supplemented contents of our paper according to your comments.

 

Authors propose a hybrid method based on additive DEA and SSM to assess technology sustainability. Finally, the use a real case to test the method. In my view, this paper has several drawbacks.

Grammar, style and English:

The paper should be revised by a native-English expert. Indeed, some paragraphs should be re-written as it is not always easy to understand what authors mean or there are many style and grammar errors. Here are some examples:

-> With the help of an English professional, we have revised the English grammar of this paper.

 

Lines 29-31: “From the result of technology forecasting, a company performs diversely technological innovation to improve his technological competitiveness.” Please, check style and grammar.

-> We modified this sentence as follows. The modified sentence is shown in red in the manuscript.

We have been trying to forecast the future technology with various purposes. For example, using the results of technology forecasts, a company can undertake a variety of technological innovations to improve its technological competitiveness in the market.

 

Lines 42-43: “It has used to find empirically measure productive efficiency of decision making units." Please, check style and grammar.

-> We modified this sentence as follows. The modified sentence is shown in red in the manuscript.

It has been used to find empirical measure of productive efficiency for decision making unit (DMU).

 

Lines 39-44: In this paragraph authors repeat the same idea, i.e. the use of DEA. Please, do not be redundant.

-> We have modified the redundant sentences into concise ones.

 

Methodology:

The authors should present the model more in depth. What are the inputs and outputs of the DEA model? As I understand, the authors use keywords as input/output variables. However, input and output variables should belong to two different sets. Please, discuss such issue.

-> We have added a more detailed description of the proposed method in section 3. In this paper, we extracted top five keywords (system, media, data, user, device) representing Apple’s technologies. Using them, we performed five experiments of DEA. For each experiment we changed the combination of keywords used as input and output variables. We used this to understand the association structure between Apple's sub technologies represented by the keywords.

We added above sentences before Table 1.

 

What is the measurement unit of the input and output variables? Patents are codified using keywords. How these latter can be used as input and output measurements? How can mathematical operations be performed with these variables?

-> We searched the patent documents applied and registered by Apple. Also, using text mining techniques, we extracted keywords and build structured data. This data type is a matrix of patents (rows) and keywords (columns). The element of this matrix is the frequency value of a specific keyword in the patent document. We used this matrix for input and output measurements of DEA.

We added above sentences before equation (5).

 

Why do authors use an additive DEA model? Why do they introduce bad outputs? What are the bad outputs in their efficiency-measurement conceptualization?

-> In our study, we control good and bad outputs at the same time. Both have meaning in technology analysis. In the process of finding the final sustainable technology area, good output strengthens the technology association and the bad output weakens the association.

We added above sentences after equation (4).

 

How many DMUs are used? Is the individual patent associated to an individual DMU?

-> We used total 8,114 patent documents associated to DMUs.

We added above sentences before Table 1.

 

Literature:

There is no in-depth literature review. Please, include a section that presents literature.

-> We have added a new section (2. Research Background) to explain the research background (literature review) in detail.

Round 2

Reviewer 2 Report

Dear Authors

The revised version of your manuscript is significantly improved. Therefore, my recommendation is to be published

Author Response

The revised version of your manuscript is significantly improved. Therefore, my recommendation is to be published

-> We would like to thank the reviewer for taking the time to review our paper and provide us with valuable feedback.

Reviewer 3 Report

I still have problems to understand the model proposed by authors. Particularly, they replied to my clarification request as follows:

We searched the patent documents applied and registered by Apple. Also, using text mining techniques, we extracted keywords and build structured data. This data type is a matrix of patents (rows) and keywords (columns). The element of this matrix is the frequency value of a specific keyword in the patent document. We used this matrix for input and output measurements of DEA. 

As far as I know, DEA uses as inputs variables that provide a measurement of resource consumption, while it uses as outputs gained benefits.

I do not understand how frequencies can be used as consumed resources.

I still do not understand the contribution of this paper to sustainability. Please, provide a more effective argument about this issue.

As the authors use DEA in a very innovative way, they should make an effort to convince the readers that their model and approach make sense.

Author Response

We would like to thank the reviewer for taking the time to review our paper and provide us with valuable feedback. We hope that we have responded to your concerns and questions satisfactorily.  

 

I still have problems to understand the model proposed by authors. Particularly, they replied to my clarification request as follows:

“We searched the patent documents applied and registered by Apple. Also, using text mining techniques, we extracted keywords and build structured data. This data type is a matrix of patents (rows) and keywords (columns). The element of this matrix is the frequency value of a specific keyword in the patent document. We used this matrix for input and output measurements of DEA.”

As far as I know, DEA uses as inputs variables that provide a measurement of resource consumption, while it uses as outputs gained benefits. I do not understand how frequencies can be used as consumed resources. I still do not understand the contribution of this paper to sustainability. Please, provide a more effective argument about this issue. As the authors use DEA in a very innovative way, they should make an effort to convince the readers that their model and approach make sense.

-> In this paper, we extend the usage of traditional DEA to sustainable technology analysis. We use technological keywords extracted from the patent documents as DEA input and output variables. This is a new attempt different from the existing DEA.

For a technological keyword that becomes the output variable, all other keywords are input variables. The relative efficiency of the input keywords with respect to the technological keyword corresponding to the output variable is evaluated, and the sub technologies for constructing the sustainable technology are defined by searching for the keywords corresponding to the high ranked input keywords. The occurred frequency values of technological keywords are used to measure the consumption and gain for DEA at the same time. In addition, in this paper, sustainable technologies are presented comprehensively considering both the results of SSM as well as DEA.

We added the above sentences to conclusion section.