Introducing the Living Lab Approach in the Coastal Area of Constanta (Romania) by Using Design Thinking

Round 1

Reviewer 1 Report

Dear Authors,

Thank you for the opportunity to review your paper. The living labs approach, linking design thinking to community development and policy design is an interesting topic area, and certainly inspires ideas regarding potentially productive areas of social innovation.

Let me take an editorial perspective on the work presented. As an academic article, I expect I expect a clear definition of contribution to the body of knowledge. For this, it is necessary to identify some problem to be researched, which often relates to an area unknown, and a systematic investigation that delivers a result for this.

The first part of the manuscript presented for review reminds me more of a comparative case study than a research paper. It describes an actual project, providing details in comparison to another approach commonly used for the purpose of solving social problems. This conceptualisation in my view does not provide a justification for the subsequent empirical testing. The construction and evaluation of the supportability index seems disconnected conceptually from the previous conceptualisation, involving design thinking and social innovation.

I also have some concerns regarding the empirical component of the study. Very little is revealed about the details of the empirical component. It remains unclear, what these calculations illustrate, and how the results contribute to the body of knowledge. The gap between responses of officials and citizens deemed to be ‘significant’, however no statistical evaluation of this is provided to justify the claim.

The conclusion therefore, built on this analysis, does not give a clear description of the contribution to the body of knowledge.

Let me make some suggestions to restructure the paper in a possible way, to establish conceptual relevancy and scientific logical flow.

1. Please clearly articulate a research question or problem. Currently, the problem is convoluted with the evidence. The topic should not be the evidence, it should be the problem.
2. Although it seems the article contains a substantial qualitative component, in effect the analysis is more a quantitative paper. Perhaps it would be worth framing it as mixed methods research. The conceptualisation provides an understanding of the living labs case study (comparative case study approach to previous methods of coastal systems management), which in turn frames the survey.
3. Hypothesise your assumptions for testing based on the survey results.
4. When conducting the comparison, please ensure methodological rigour. A comparison of scores requires two small samples (at least 2 observations per group), and generalisation needs some degree of representativeness (e.g. random sampling, or stratification).
5. Conclude the article actually answering the research questions, building on the conceptual framework and the research results.
6. Please ensure compliance with journal referencing requirement.

I hope you find my suggestions useful, when revising the paper.

Good luck with your work.

Regards,

A reviewer

Author Response

Please see the attachment.

Author Response File: Author Response.pdf

Reviewer 2 Report

After reading the paper,  this reviewer has doubts about the contribution as a scientific line of this work. Authors should make an effort to identify their contribution. Since, according to the authors, the implementation of the laboratory is described in the manuscript, although the model, design and methodology have already been published in a previous paper: The management model was developed in the multi-criteria analysis of the Mass Tourism Management Model Related to the Impact on the Local Community in Constanta City (Romania) published in MDPI Inventions on June 28, 2021. In the published paper, the coastal management model was based on the analysis of official statistical data. In this paper, we intend to put the Living Labs concept into action by utilizing data and information gathered from citizens) The new paper is an outreach experience. 

Another of the great drawbacks of the work is its similarity with respect to the following references (copied texts):

• Lines 49-54; 76-130 > Concept Design with a Living Lab Approach" 2009 42nd Hawaii International Conference on System Sciences, 2009

Author Response

Please see the attachment.

Author Response File: Author Response.pdf

Round 2

Reviewer 1 Report

Dear Authors,

Thank you for the attentive response to my recommendations.

Whilst I understand that this may be difficult to handle feedback, and that as a reviewer, I am placed in a position to reflect on your work, but I am not in the context of your research activity, I believe that my judgement carries weight. I am afraid, at this point, I do not yet believe that the manuscript meets the requirements of an academic journal publication.

I would like to reiterate my suggestions for improvement:

1. Please clearly articulate a research question or problem. Currently, the problem is convoluted with the evidence. The topic should not be the evidence, it should be the problem.

• You have reframed the article to separate problem and evidence.
• However, I cannot accept your response, because you have not identified a research question.
• Please articulate a research question, which is contextualised in theory as much as in the practical problem used to explore this theory.

* I do not believe that in this paper, the design of coastal management systems is the research problem. If so, I simply do not understand the article. What I see as a research problem, is the application of the living labs concept/framework to explain how the design process of the coastal management system can be explained / elaborated, perhaps better than based on other concepts. There is an attempt to spell this out in the next section, however the linear coastal management system design method does not seem like a clear, general alternative to a living labs approach. Is there a better way to describe the linear method in general context?

2. Although it seems the article contains a substantial qualitative component, in effect the analysis is more a quantitative paper. Perhaps it would be worth framing it as mixed methods research. The conceptualisation provides an understanding of the living labs case study (comparative case study approach to previous methods of coastal systems management), which in turn frames the survey.

• The theoretical concepts of living labs have been well introduced, thank you.
• Why is this approach potentially more meaningful than any other in describing and explaining the coastal management system design problem? I think you need to clearly justify this, and explain how this remains a gap in knowledge.
• There is also an attempt to clarify mixed methods research here. Please have a look at some key literature to improve this section, e.g.:
  • Roslyn Cameron, Jose F. Molina‐Azorin, (2011) "The acceptance of mixed methods in business andmanagement research", International Journal of Organizational Analysis, Vol. 19 Issue: 3, pp.256-271,https://doi.org/10.1108/19348831111149204
  • Bentahar O  and  Cameron    “Design  and  Implementation  of  a  Mixed  Method  Research  Study  in  Project  Management” The  Electronic  Journal  of  Business  Research  MethodsVolume  13  Issue 1  2015  (pp 3-15)
  • Jose F. Molina-Azorin,
  • Mixed methods research: An opportunity to improve our studies and our research skills,
  • European Journal of Management and Business Economics, Volume 25, Issue 2, 2016, Pages 37-38, https://doi.org/10.1016/j.redeen.2016.05.001.
  • Onwuegbuzie, A. J., & Collins, K. M. (2007). A Typology of Mixed Methods Sampling Designs in Social Science Research .TheQualitative Report,12(2), 281-316. Retrieved fromhttps://nsuworks.nova.edu/tqr/vol12/iss2/9

4. When conducting the comparison, please ensure methodological rigour. A comparison of scores requires two small samples (at least 2 observations per group), and generalisation needs some degree of representativeness (e.g. random sampling, or stratification).

• There may have been a misunderstanding here. I understand how the qualitative part was done, what I think needs to be described better is the qualitative – if there was any. Did this research only involve some quantitative questions added to the qualitative interviews? If so, perhaps a quantitative comparison is not suitable as part of the analysis.

3. Hypothesise your assumptions for testing based on the survey results.

• As per above comment, perhaps there need to be no hypotheses? Please clarify the process of comparison, and how it leads to answering the research question.

5. Conclude the article actually answering the research questions, building on the conceptual framework and the research results.

• Apologies, but no question leads to no answer. This in my opinion is a fundamental issue with this research.

I hope my comments will help revising the paper.

Regards,

A reviewer

Author Response

ANSWERS TO THE REVIEWERS' COMMENTS 

Manuscript ID: Inventions- 1245547

GENERAL COMMENTS

A revision has been carried out following carefully all the indications, suggestions and observations formulated by the reviewers.

The main changes operated are outlined next:

1) Detailed explanations have been provided at various points in order to make the results of the work more comprehensible.

2) Several parts of the text have been modified in accordance with the requests of the reviewer

At this point, it has to be also highlighted that the authors tried to follow all the suggestions and observations formulated by the reviewers and to operate (as much as it was possible) all the corrections indicated by them. Furthermore, in order to follow the corrections operated in the paper, a version of the manuscript having all the changes operated tracked (with the option track changes) has been also uploaded together with the last form of the manuscript (without the changes tracked).

The specific corrections operated according to the suggestions of the reviewers are given next together with detailed explanations.

Reviewer # 1:

1. Please clearly articulate a research question or problem. Currently, the problem is convoluted with the evidence. The topic should not be the evidence, it should be the problem.

• You have reframed the article to separate problem and evidence.
• However, I cannot accept your response, because you have not identified a research question.
• Please articulate a research question, which is contextualised in theory as much as in the practical problem used to explore this theory.

* I do not believe that in this paper, the design of coastal management systems is the research problem. If so, I simply do not understand the article. What I see as a research problem, is the application of the living labs concept/framework to explain how the design process of the coastal management system can be explained / elaborated, perhaps better than based on other concepts. There is an attempt to spell this out in the next section, however the linear coastal management system design method does not seem like a clear, general alternative to a living labs approach. Is there a better way to describe the linear method in general context?

Thank you for your suggestions, they were very useful for us. Regarding this point, the authors modified the paper as follows:

The following modified text was added to the Abstract:

• “This paper aims to introduce application of the Living lab concept in the management of the coastal area of Constanta (Romania). The concept of the living lab means to involve citizens along with public bodies and research structures.”

Also, the following modified texts was added to the Introduction:

• “Traditional coastal management is generally achieved through a linear approach based on an up-down pyramid decision-making structure. The hierarchical structure is reduced to governmental, political entities, while the other existing actors in the coastal zone (SMEs, academia, research institutes, citizens, etc.) are called stakeholders, and the way of involving the latter is smaller or larger depending on the decision of the authorities. The linear approach has a predominantly hierarchical and less consultative decision-making role”
• “The implementation framework of a Living Lab in the coastal area is a centric one, starting from the observations of the citizens and of the scientific environment, which define, conceptualize and test the existing vulnerabilities in that area. The role of decision makers is to take these observations and research results and turn them into public policies. To create this functional framework, the authors consider it essential involve citizens, through the data they provide, both in research and development.”

1. Although it seems the article contains a substantial qualitative component, in effect the analysis is more a quantitative paper. Perhaps it would be worth framing it as mixed methods research. The conceptualisation provides an understanding of the living labs case study (comparative case study approach to previous methods of coastal systems management), which in turn frames the survey.

• The theoretical concepts of living labs have been well introduced, thank you.
• Why is this approach potentially more meaningful than any other in describing and explaining the coastal management system design problem? I think you need to clearly justify this, and explain how this remains a gap in knowledge.
• There is also an attempt to clarify mixed methods research here. Please have a look at some key literature to improve this section, e.g.:

Thank you very much for your suggestions. Please let us to present you this part from Methodology chapter, starting with line 110, where we clarify this issue:

“We aim to maximize our strengths and weaken our strengths through this form of research and not to replace any of the particularly useful and important forms of classical research, both quantitative and qualitative. The paradigm of mixed method research, as in this paper, minimizes the schism between qualitative and quantitative approaches to research.

If we are to better understand this concept of the mixed research method, we can say that this new type combines methods and techniques, concepts and qualitative or quantitative approaches in a single study.

The process involves the combination of three primary factors, namely induction, deduction, and abduction. This means, in research practice, that models must be identified, theories tested, validated and conceptualized. Therefore, traditional approaches to integrated coastal management are generally linear and often cyclical (Figure 2). Living Labs, on the other hand, takes a design-based approach with a focus on loop development (Figure 3). [20-24].

Perhaps the most obvious distinction between integrated linear coastal zone management and the living lab is based on conceptual realism. If the classical linear form has a theoretical and formal vision, a living lab is much more pragmatic, contextualizing real world facts and attitudes. Even from the users' point of view, the two management systems are fundamentally different [16]. If in the case of linear vision, users are formal, well-structured, with clear and precise attributes, in the case of living labs, these users are creative, informal, but deeply rooted in current reality. Therefore, in the case of Living Labs, users are urged to create in the most authentic contexts, under the observation of researchers and developers involved in the process [8] [10] [25]. The paradigm of mixed research in the case of Living Labs shows that users are considered as partners in the coastal management process, while in the case of traditional systems, they are considered as actors, the approach being organizational [26]. This element is a defining one in differentiating the two systems, showing that Living Labs has as its central element the citizen and his empowerment to the detriment of a formal organization [27-29]. This means that the new approach of Living Labs emphasizes the spontaneity and inventiveness of the citizens involved. Another important point is the relationship with the academic environment, which in the case of the new approach is a close one, citizens using research results, while in traditional systems, this type of connection is not always made [30]. That is why the new structures could use this to investigate both theoretical and practical problems. Specifically, research is conducted in the Living Labs environment, while in traditional classical systems, the academic environment conducts research at the express request of decision makers, if any..”

1. When conducting the comparison, please ensure methodological rigour. A comparison of scores requires two small samples (at least 2 observations per group), and generalisation needs some degree of representativeness (e.g. random sampling, or stratification).

• There may have been a misunderstanding here. I understand how the qualitative part was done, what I think needs to be described better is the qualitative – if there was any. Did this research only involve some quantitative questions added to the qualitative interviews? If so, perhaps a quantitative comparison is not suitable as part of the analysis.

We are not sure if we understand your query: “I understand how the qualitative part was done, what I think needs to be described better is the qualitative – if there was any”. You mean to describe the quantitative method? For this, as we stated in our research, the quantitative part of this research is represented by determining value of the supportability factor. This calculation is followed and related with the interview process. The survey involve, in general, a qualitative method and the results is used to calculate the supportability factor (Quantitative method).

Please see the following paragraph from line 228:

“In the case of citizens, the indicators of supportability were calculated based on a survey of 30 respondents who were specifically interested in coastal activities. This survey focused on 16 activities, including 7 economic, 5 environmental, and 4 social activities. This type of random sampling divided the subjects into two layers, and in each layer a sample of 15 respondents was selected proportionally. The division followed the sizing of the sample taking into account the population structure of the analyzed area. When dividing into groups, several criteria were taken into account, such as age, education, social status, religion, etc. Each subject has been assigned a number. The interviews were structured so that each subject could identify the interactions between coastal activities and determine the pressures exerted.”

1. Hypothesise your assumptions for testing based on the survey results.

• As per above comment, perhaps there need to be no hypotheses? Please clarify the process of comparison, and how it leads to answering the research question.

We are sorry for this misunderstanding. We use a hypotheses and we explain the whole process in Materials and Methods section, line 228. Anyway, at this paragraph we made the following changes to be more clear:

“The hypothesis is represented by the actual state of the pressures existing on the coastal areas. As we stated before, we will use the citizen science in order to identify, test and validate the hypothesis. For this, the indicators of supportability were calculated based on a survey of 30 respondents who were specifically interested in coastal activities. This survey focused on 16 activities, including 7 economic, 5 environmental, and 4 social activities. This type of random sampling divided the subjects into two layers, and in each layer a sample of 15 respondents was selected proportionally. The division followed the sizing of the sample taking into account the population structure of the analyzed area. When dividing into groups, several criteria were taken into account, such as age, education, social status, religion, etc. Each subject has been assigned a number. The interviews were structured so that each subject could identify the interactions between coastal activities and determine the pressures exerted.”

1. Conclude the article actually answering the research questions, building on the conceptual framework and the research results.

• Apologies, but no question leads to no answer. This in my opinion is a fundamental issue with this research.

Thank you very much for your suggestion. We already answered to your query in Discussion section with all details provided. In the Conclusion section we just underlined the main conclusion after these finding from Discussion part. Anyway, we added the following paragraph to Conclusion:

“The analysis of the coastal management model through the prism of Living Labs shows that these mechanisms that collect creative ideas and realistic approaches to some of the challenges that communities face. The involvement of citizens in these structures is beneficial because it involves the participation of civil society in making development decisions in a region.

To prove this, we find out that there were significant differences in opinion between authorities and citizens, particularly regarding the indicators of agriculture / aquaculture / local food and energy / electricity. More than that, a general evaluation of the supportability factor suggests that citizens think that the pressures exerted on coastal zone operations are substantially larger than the statistics given by the government show, although the discrepancy between these two computed values is not particularly large. From these we can easily conclude that the Living Labs is a useful approach in coastal management and aims to identify, define, develop, test and validate new approaches to community plans and strategies.”

Reviewer 2 Report

The authors have modified part of the recommendations. But the article still has some drawbacks that mean that it cannot be considered a "scientific paper":

• It is not clear what the scientific contribution of the paper is. Is it a case study? What does it contribute to scientific-technical knowledge? What does it contribute compared to other similar articles? Are there other similar items that implement a living lab?
• Hypotheses are research questions that the researcher asks himself at the beginning of the work. They can never go to the end of the text as a discussion of results. Furthermore, the hypothesis put forward by the authors is not correct. It should be reviewed.
• Both the description of the methods and the results are not clear. Authors should better describe the research methods (qualitative and / or quantitative) in order to fully understand the following sections. A comparison between this implementation (living lab) and the conventional way of carrying out coastal management would be interesting; this would be useful to assess the authors' proposal.
• The previous comment can be derived from the wording. Sometimes the sentences are poorly structured and are too long: E.g. " The involvement of citizens in these structures is beneficial because it involves the participation of civil society in making development decisions in a region. As I said before, Living Labs aims to identify, define, develop, test and validate new approaches to community plans and strategies". 

Author Response

ANSWERS TO THE REVIEWERS' COMMENTS 

Manuscript ID: Inventions- 1245547

GENERAL COMMENTS

A revision has been carried out following carefully all the indications, suggestions and observations formulated by the reviewers.

The main changes operated are outlined next:

1) Detailed explanations have been provided at various points in order to make the results of the work more comprehensible.

2) Several parts of the text have been modified in accordance with the requests of the reviewer

At this point, it has to be also highlighted that the authors tried to follow all the suggestions and observations formulated by the reviewers and to operate (as much as it was possible) all the corrections indicated by them. Furthermore, in order to follow the corrections operated in the paper, a version of the manuscript having all the changes operated tracked (with the option track changes) has been also uploaded together with the last form of the manuscript (without the changes tracked).

The specific corrections operated according to the suggestions of the reviewers are given next together with detailed explanations.

 Reviewer # 2:

It is not clear what the scientific contribution of the paper is. Is it a case study? What does it contribute to scientific-technical knowledge? What does it contribute compared to other similar articles? Are there other similar items that implement a living lab?

This paper refers to the introduction of a Living Lab in the coastal area of Constanta (Romania). For answered more concrete to this question, we I revised and added paragraphs to the next text of the Introduction Chapter

“Integrated coastal zone management (ICZM) is a social, ecological structure with extremely complex characteristics due to differences in society, culture, finance, nature, and governance. The ICZM's mission is to promote integrated coastal zone management by investigating the relationship between coastal zone management and governance variables. [4,5]

Traditional coastal management is generally achieved through a linear approach based on an up-down pyramid decision-making structure. The hierarchical structure is reduced to governmental, political entities, while the other existing actors in the coastal zone (SMEs, academia, research institutes, citizens, etc.) are called stakeholders, and the way of involving the latter is smaller or larger depending on the decision of the authorities. The linear approach has a predominantly hierarchical and less consultative decision-making role. Living Labs have emerged in Europe in recent years to get closer to citizens and to facilitate collaboration between different stakeholder groups. These mechanisms collect creative ideas and work skills in order to involve private users, together with public bodies, to improve the environment within communities. Approaching the concept of Living Lab in coastal management can be beneficial because this structure aims to involve citizens along with public bodies and research structures. Through the design thinking approach technique, coastal living labs will identify, define, develop, test or validate new development policies, projects and community strategies. The implementation framework of a Living Lab in the coastal area is a centric one, starting from the observations of the citizens and of the scientific environment, which define, conceptualize and test the existing vulnerabilities in that area. The role of decision makers is to take these observations and research results and turn them into public policies.  To create this functional framework, the authors consider it essential involve citizens, through the data they provide, both in research and development. A living lab implemented in a coastal area can only benefit the community. As it is currently defined, integrated coastal management entails a collaboration between the scientific component and the public / human component. The public policies of the area are created by combining these two components, and the management component of ICZM is carried out”

Hypotheses are research questions that the researcher asks himself at the beginning of the work. They can never go to the end of the text as a discussion of results. Furthermore, the hypothesis put forward by the authors is not correct. It should be reviewed.

The following modified text was added to the Abstract:

• “This paper aims to introduce application of the Living lab concept in the management of the coastal area of Constanta (Romania). The concept of the living lab means to involve citizens along with public bodies and research structures.”

Also, the following modified texts was added to the Introduction:

• “Traditional coastal management is generally achieved through a linear approach based on an up-down pyramid decision-making structure. The hierarchical structure is reduced to governmental, political entities, while the other existing actors in the coastal zone (SMEs, academia, research institutes, citizens, etc.) are called stakeholders, and the way of involving the latter is smaller or larger depending on the decision of the authorities. The linear approach has a predominantly hierarchical and less consultative decision-making role”
• “The implementation framework of a Living Lab in the coastal area is a centric one, starting from the observations of the citizens and of the scientific environment, which define, conceptualize and test the existing vulnerabilities in that area. The role of decision makers is to take these observations and research results and turn them into public policies. To create this functional framework, the authors consider it essential involve citizens, through the data they provide, both in research and development.”

We use a hypotheses and we explain the whole process in Materials and Methods section, line 228. Anyway, at this paragraph we made the following changes to be more clear:

• “The hypothesis is represented by the actual state of the pressures existing on the coastal areas. As we stated before, we will use the citizen science in order to identify, test and validate the hypothesis. For this, the indicators of supportability were calculated based on a survey of 30 respondents who were specifically interested in coastal activities. This survey focused on 16 activities, including 7 economic, 5 environmental, and 4 social activities. This type of random sampling divided the subjects into two layers, and in each layer a sample of 15 respondents was selected proportionally. The division followed the sizing of the sample taking into account the population structure of the analyzed area. When dividing into groups, several criteria were taken into account, such as age, education, social status, religion, etc. Each subject has been assigned a number. The interviews were structured so that each subject could identify the interactions between coastal activities and determine the pressures exerted.”

Both the description of the methods and the results are not clear. Authors should better describe the research methods (qualitative and / or quantitative) in order to fully understand the following sections. A comparison between this implementation (living lab) and the conventional way of carrying out coastal management would be interesting; this would be useful to assess the authors' proposal.

Please let us to present you this part from Methodology chapter, starting with line 110, where we clarify this issue:

“We aim to maximize our strengths and weaken our strengths through this form of research and not to replace any of the particularly useful and important forms of classical research, both quantitative and qualitative. The paradigm of mixed method research, as in this paper, minimizes the schism between qualitative and quantitative approaches to research.

If we are to better understand this concept of the mixed research method, we can say that this new type combines methods and techniques, concepts and qualitative or quantitative approaches in a single study.

The process involves the combination of three primary factors, namely induction, deduction, and abduction. This means, in research practice, that models must be identified, theories tested, validated and conceptualized. Therefore, traditional approaches to integrated coastal management are generally linear and often cyclical (Figure 2). Living Labs, on the other hand, takes a design-based approach with a focus on loop development (Figure 3). [20-24].

Perhaps the most obvious distinction between integrated linear coastal zone management and the living lab is based on conceptual realism. If the classical linear form has a theoretical and formal vision, a living lab is much more pragmatic, contextualizing real world facts and attitudes. Even from the users' point of view, the two management systems are fundamentally different [16]. If in the case of linear vision, users are formal, well-structured, with clear and precise attributes, in the case of living labs, these users are creative, informal, but deeply rooted in current reality. Therefore, in the case of Living Labs, users are urged to create in the most authentic contexts, under the observation of researchers and developers involved in the process [8] [10] [25]. The paradigm of mixed research in the case of Living Labs shows that users are considered as partners in the coastal management process, while in the case of traditional systems, they are considered as actors, the approach being organizational [26]. This element is a defining one in differentiating the two systems, showing that Living Labs has as its central element the citizen and his empowerment to the detriment of a formal organization [27-29]. This means that the new approach of Living Labs emphasizes the spontaneity and inventiveness of the citizens involved. Another important point is the relationship with the academic environment, which in the case of the new approach is a close one, citizens using research results, while in traditional systems, this type of connection is not always made [30]. That is why the new structures could use this to investigate both theoretical and practical problems. Specifically, research is conducted in the Living Labs environment, while in traditional classical systems, the academic environment conducts research at the express request of decision makers, if any..”

Please see the following paragraph from line 228:

“In the case of citizens, the indicators of supportability were calculated based on a survey of 30 respondents who were specifically interested in coastal activities. This survey focused on 16 activities, including 7 economic, 5 environmental, and 4 social activities. This type of random sampling divided the subjects into two layers, and in each layer a sample of 15 respondents was selected proportionally. The division followed the sizing of the sample taking into account the population structure of the analyzed area. When dividing into groups, several criteria were taken into account, such as age, education, social status, religion, etc. Each subject has been assigned a number. The interviews were structured so that each subject could identify the interactions between coastal activities and determine the pressures exerted.”

The previous comment can be derived from the wording. Sometimes the sentences are poorly structured and are too long: E.g. " The involvement of citizens in these structures is beneficial because it involves the participation of civil society in making development decisions in a region. As I said before, Living Labs aims to identify, define, develop, test and validate new approaches to community plans and strategies". 

Thanks for the comments. The paragraph was reformulated as follows.

• “Citizens' participation in these structures is advantageous since it includes civil society in making development recommendations in an area. […] From these we can easily conclude that the Living Labs is a useful approach in coastal management and aims to identify, define, develop, test and validate new approaches to community plans and strategies.”

After reading the paper,  this reviewer has doubts about the contribution as a scientific line of this work. Authors should make an effort to identify their contribution. Since, according to the authors, the implementation of the laboratory is described in the manuscript, although the model, design and methodology have already been published in a previous paper: The management model was developed in the multi-criteria analysis of the Mass Tourism Management Model Related to the Impact on the Local Community in Constanta City (Romania) published in MDPI Inventions on June 28, 2021. In the published paper, the coastal management model was based on the analysis of official statistical data. In this paper, we intend to put the Living Labs concept into action by utilizing data and information gathered from citizens) The new paper is an outreach experience. 

 Another of the great drawbacks of the work is its similarity with respect to the following references (copied texts):

• Lines 49-54; 76-130 > Concept Design with a Living Lab Approach" 2009 42nd Hawaii International Conference on System Sciences, 2009

Taking into account the mentioned by the reviewer, the several changes have been made.

• The authors modified the title of the paper as follows:

“Introducing the Living lab approach in the coastal area of Constanta (Romania) by using design thinking”

• Also, the following modified text was added to the Abstract:

“This new coastal management model will use the design thinking technique and will take into account the pressures between the activities that take place in the analyzed coastal zone. A coastal management model was developed in the paper Multi-Criteria Analysis of the Mass Tourism Management Model related to the impact on the local community in Constanța (Romania) published in MDPI Inventions on June 28, 2021, and took into account only the data provided by the authorities. In this paper, the authors aim to expand their research by including data from independent sources, using the concept of Living Lab.

• The authors brought the following changes in the chapters Introduction and Materials and Methods:

Living Labs have emerged in Europe in recent years to get closer to citizens and to facilitate collaboration between different stakeholder groups. These mechanisms collect creative ideas and work skills in order to involve private users, together with public bodies, to improve the environment within communities. Approaching the concept of Living Lab in coastal management can be beneficial because this structure aims to involve citizens along with public bodies and research structures. Through the design thinking approach technique, coastal living labs will identify, define, develop, test or validate new development policies, projects and community strategies. Through this research, the authors aim to actively involve citizens, through the data they provide, both in research and development

“The creation of a "Living Lab" in coastal areas must take into account a citizen-centered approach to research and development, taking into account technological innovations [10] [12]. This type of approach will facilitate the interaction between stakeholders, bringing together all actors in the coastal area, whether we are talking about academia, the civic sector, state actors or the economic sector. A "Living Lab" is a new and dynamic body that brings technical research into real-world collaborative environments. The new concept has several principles by which it is defined [9] [15-17], namely:

• Continuity
• Opening
• Realism
• Empowering citizens
• Spontaneity

To explain these principles, it must be said that a Living Lab is based on creativity and innovation, but also on a good multidisciplinary collaboration. The innovative process must be credible and as open as possible, and at the same time it needs time to be structured. The multi-criteria approach of the newly created Living Labs is essential to provide a multi-perspective vision. The central element of the new structures is the citizen and his involvement in the innovation processes. Users are the ones who benefit from innovations and guide them to satisfy their own desires and needs. The more inventive the user communities are, the more efficient Living Labs become and achieve their intended purpose. Traditional coastal management systems have a linear approach, unlike the Living labs structure which has a centered approach [17]. The analysis of these systems is of mixed type, having qualitative elements but also emphasizing the quantitative side [18-19]. It is clear that the aim of the authors was to show the usefulness of a mixed paradigm in coastal management research. We aim to maximize our strengths and weaken our strengths through this form of research and not to replace any of the particularly useful and important forms of classical research, both quantitative and qualitative. The paradigm of mixed method research, as in this paper, minimizes the schism between qualitative and quantitative approaches to research.

If we are to better understand this concept of the mixed research method, we can say that this new type combines methods and techniques, concepts and qualitative or quantitative approaches in a single study.

The process involves the combination of three primary factors, namely induction, deduction, and abduction. This means, in research practice, that models must be identified, theories tested, validated and conceptualized. Therefore, traditional approaches to integrated coastal management are generally linear and often cyclical (Figure 2). Living Labs, on the other hand, takes a design-based approach with a focus on loop development (Figure 3). [20-24].

Perhaps the most obvious distinction between integrated linear coastal zone management and the living lab is based on conceptual realism. If the classical linear form has a theoretical and formal vision, a living lab is much more pragmatic, contextualizing real world facts and attitudes. Even from the users' point of view, the two management systems are fundamentally different [16]. If in the case of linear vision, users are formal, well-structured, with clear and precise attributes, in the case of living labs, these users are creative, informal, but deeply rooted in current reality. Therefore, in the case of Living Labs, users are urged to create in the most authentic contexts, under the observation of researchers and developers involved in the process [8] [10] [25]. The paradigm of mixed research in the case of Living Labs shows that users are considered as partners in the coastal management process, while in the case of traditional systems, they are considered as actors, the approach being organizational [26]. This element is a defining one in differentiating the two systems, showing that Living Labs has as its central element the citizen and his empowerment to the detriment of a formal organization [27-29]. This means that the new approach of Living Labs emphasizes the spontaneity and inventiveness of the citizens involved. Another important point is the relationship with the academic environment, which in the case of the new approach is a close one, citizens using research results, while in traditional systems, this type of connection is not always made [30]. That is why the new structures could use this to investigate both theoretical and practical problems. Specifically, research is conducted in the Living Labs environment, while in traditional classical systems, the academic environment conducts research at the express request of decision makers, if any. [31]”

“As previously stated, the approach to introducing a Living Labs in the coastal zone is  a design thinking technique. This is also because design thinking involves a human-centered vision of innovation, which seeks to establish creative ideas and effective business models that are focused on people's needs”

• To ensure methodological rigour, the author introduced the following text:

“This type of random sampling divided the subjects into two layers, and in each layer a sample of 15 respondents was selected proportionally. The division followed the sizing of the sample taking into account the population structure of the analyzed area. When dividing into groups, several criteria were taken into account, such as age, education, social status, religion, etc. Each subject has been assigned a number. The interviews were structured so that each subject could identify the interactions between coastal activities and determine the pressures exerted.

The stratification of the sampling ensured, in the analyzed case, a significant representation of the population, which determines a decrease of the sampling error. The present groups were adequately represented, and the division into subgroups led to a more pronounced heterogeneity. However, it must be said that in this case we started from a knowledge of the situation, a priori, data and the multicriteria analyzes of the coastal management model made by its classical approach. At the same time, in this case of stratified sampling, the procedure was more complex than in simple sampling, due to the need to divide the subjects and divide the subgroups.”

At this Conclusions, the authors added the following text:

“The analysis of the coastal management model through the prism of Living Labs shows that these mechanisms that collect creative ideas and realistic approaches to some of the challenges that communities face. The involvement of citizens in these structures is beneficial because it involves the participation of civil society in making development decisions in a region. As I said before, Living Labs aims to identify, define, develop, test and validate new approaches to community plans and strategies.”

Round 3

Reviewer 2 Report

Authors have improved the paper. But I still doubt the contribution to scientific knowledge and the scope of the magazine "inventions". 

The writing and the text in English should be improved.