Sustainable Development of Students’ Assumed Responsibility for Their Own Learning during Participatory Action Research
2. Prospects of Sustainable Development of Students’ Assumed Responsibility for Their Learning
3. Research Methodology
3.1. Research Context and Participants
3.2. Research Strategy and Design
- Understanding of the context: for this purpose, teachers’ reflections, non-formal conversations with school students were used.
- Enhancement of commitment to school and interested people: when negotiating, a commitment was made to share generalized research results and interactive games with school students identifying their needs.
3.3. Data Analysis
3.4. Quality Assurance
- Real problem oriented to practice (perfection of practice is focused on progress in student learning being ensured by conditions created by a teacher for students to assume responsibility).
- Participation and cooperation of all interested parties (all students and a teacher, i.e., those who will be affected by the results of the research, took part in the investigation). Mutual cooperation proceeded at all stages of the investigation: when identifying the problem, making decisions that will change practice, approving data analysis for all interested parties, etc.
- Use of data sources in several time frames and analysis of conclusions in several time frames increased confidence in reliability of conclusions. (Since it was aimed to deeply perceive the understanding of the problem, it was important to set a strict process of repeated data collection, consideration, and analysis with individuals and groups).
- Clear focus on a reflective process (both at the beginning of a new activity and reflective discussion at the end, keeping of systematic reflective journal ensured orientation of the process toward reflectivity and provided conditions for all participants to learn from experience).
- Critical and self-critical viewpoint (semi-structured reflection sheets, researchers’ memos created conditions to view the process from various angles).
- Provision of all evidences (evidences were presented by using various perspectives: personal opinions and feedback from all participants and interested subjects, various methods, e.g., reflective journals, interview, observation).
- Connections between applied prospects of problem identification and methods of data collection are ensured (perspective of social constructivism is applied directed toward the process of learning ensured possibilities to reveal conditions created by a teacher for a student to assume responsibility for one’s learning).
- Aimed at systematic changes in the following aspects: for oneself (as a teacher, as school students) and others (other teachers can use the research results: when recognizing approaches that proved to be useful in their practice or taking ideas of good practice and re-constructing them in their own practice).
- Seeking internalization of theory and practice (the approach stating that theory and practice are two interdependent and mutually supplementing stages of the process of change was maintained. During the discussion, reflection proceeds on how theory defining conditions for students to assume responsibility for their learning manifests in practice and how practice points out conditional elements that supplement theoretical insights).
- Research ethics is ensured (characterized in greater detail in the next section).
3.5. Research Ethics
- Respect to a person, his/her experience and knowledge brought to the research process;
- Belief in democratic processes ad power of them to achieve positive social changes;
- Commitment to act.
- Anonymity (information on the participants was protected to prevent connect it with specific data. The data was coded);
- Privacy (private information was not made public);
- Confidentiality (obtained information was handled only in ways that were agreed with all interested parties. This is a commitment to respect);
- Justice (all interested parties (teachers, students) who share power and resources take part in the research seeking benefit to all participants. Consent was obtained from parents in compliance with national and institutional requirements.).
- Grounded on guidelines set in advance that directly regard relations with those who participate in research and those who may be affected by the research;
- Ethical approach was applied in all stages of action research to help all interested individuals to make decisions;
- Researchers oriented towards making the research process and result clear and transparent to a broader public.
4.1. Conditions for the Students to Assume Responsibility for Their Learning
4.2. Intervening Conditions for Students Assuming Responsibility for Learning
4.2.1. Intervening Condition “Making Students’ Experiences Relevant”
4.2.2. Intervening Condition “Attitude towards Oneself as a Major Resource of Learning”
4.2.3. Intervening Condition “Attitude towards Failure”
4.2.4. Intervening Condition “Learning about Learning”
4.2.5. Intervening Condition “Expectations, Goals of Learning”
4.2.6. Intervening Condition “Clear Criteria”
4.2.7. Intervening Condition “Self-Assessment”
4.2.8. Intervening Condition “Feedback”
5. Discussion and Conclusions
6. Limitations of the Study and Future Perspective
Institutional Review Board Statement
Informed Consent Statement
Data Availability Statement
Conflicts of Interest
- Transforming Our World: The 2030 Agenda for Sustainable Development, A/RES/70/1. Available online: https://www.un.org/en/development/desa/population/migration/generalassembly/docs/globalcompact/A_RES_70_1_E.pdf (accessed on 12 June 2021).
- Sustainable Development Goals/4 Quality Educations. Available online: https://www.un.org/sustainabledevelopment/education/ (accessed on 8 May 2021).
- Eidea, E.R.; Showalter, M.H. Estimating the Relation Between Health and Education: What do We Know and What do We Need to Know? Econ. Educ. Rev. 2011, 30, 778–791. [Google Scholar] [CrossRef]
- Edgerton, J.D.; Roberts, L.W.; von Below, S. Education and Quality of Life. In Handbook of Social Indicators and Quality of Life Research, 2nd ed.; Land, K., Michalos, A., Sirgy, M., Eds.; Springer: Dordrecht, The Netherlands, 2012; pp. 265–296. [Google Scholar] [CrossRef]
- Teaching and Learning: Achieving Quality for All: EFA Global Monitoring Report, 2013–2014, ED.2013/WS/29 REV Summary. Available online: https://unesdoc.unesco.org/ark:/48223/pf0000225654 (accessed on 14 May 2021).
- Fishman, E.J. With Great Control Comes Great Responsibility: The Relationship Between Perceived Academic Control, Student Responsibility, And Self-Regulation. Br. J. Educ. Psychol. 2014, 84, 685–702. [Google Scholar] [CrossRef] [PubMed]
- Dudley-Marling, C. Social Construction of Learning. In Encyclopedia of the Sciences of Learning; Seel, N.M., Ed.; Springer: Boston, MA, USA, 2012; pp. 182–211. [Google Scholar] [CrossRef]
- Reusser, K.; Pauli, C. Co-constructivism in Educational Theory and Practice. In International Encyclopaedia of the Social & Behavioral Sciences, 2nd ed.; Wright, J.D., Ed.; Elsevier: Amsterdam, The Netherlands, 2015; pp. 913–917. [Google Scholar] [CrossRef]
- Mohammed, S.H.; Kinyo, L. The Role of Constructivism in the Enhancement of Social Studies Education. J. Crit. Rev. 2020, 7, 249–256. [Google Scholar]
- Hong, J.S.; Espelageb, D.L. A Review of Research on Bullying and Peer Victimization in School: An Ecological System Analysis. Aggress. Violent Behav. 2012, 17, 311–322. [Google Scholar] [CrossRef]
- UNESCO. Global Education Monitoring Report 2020: Inclusion and Education: All Means All; UNESCO: Paris, France, 2020; Available online: https://unesdoc.unesco.org/ark:/48223/pf0000373718 (accessed on 8 May 2021).
- Tannock, S. The Problem of Education-Based Discrimination. Br. J. Sociol. Educ. 2008, 29, 439–449. [Google Scholar] [CrossRef]
- OECD. Educating Teachers for Diversity: Meeting the Challenge, Educational Research and Innovation; OECD Publishing: Paris, France, 2010; Available online: https://www.oecd-ilibrary.org/education/educating-teachers-for-diversity_9789264079731-en (accessed on 8 May 2021).
- Hattie, J. Visible Learning, 1st ed.; Routledge: London, UK; New York, NY, USA, 2008. [Google Scholar]
- Fletcher, A.K. Exceeding Expectations: Scaffolding Agentic Engagement through Assessment as Learning. Educ. Res. 2016, 58, 400–419. [Google Scholar] [CrossRef][Green Version]
- Wiliam, D. Embedded Formative Assessment, 2nd ed.; Solution Tree: Bloomington, IN, USA, 2017. [Google Scholar]
- Gray, A. Constructivist Teaching and Learning; University of Saskatchewan: Saskatoon, SK, Canada, 1997; Available online: https://saskschoolboards.ca/wp-content/uploads/97-07.htm#Table%20of%20Contents (accessed on 12 January 2021).
- Akpan, V.I.; Igwe, U.A.; Mpamah, I.B.I.; Okoro, C.O. Social Constructivism: Implications on Teaching and Learning. Br. J. Educ. 2020, 8, 49–56. [Google Scholar]
- Sasson, I.; Malkinson, N.; Oria, T. A Constructivist Redesigning of the Learning Space: The Development of a Sense of Class Cohesion. Learn. Environ. Res. 2021, 1–15. [Google Scholar] [CrossRef]
- Ahmad, S.; Sultana, N.; Jamil, S. Behaviorism vs Constructivism: A Paradigm Shift from Traditional to Alternative Assessment Techniques. J. Appl. Linguist. Lang. Res. 2020, 7, 19–33. [Google Scholar]
- Ernest, P. Social Constructivism as a Philosophy of Mathematics: Radical Constructivism Rehabilitated 1999. Available online: http://socialsciences.exeter.ac.uk/education/research/centres/stem/publications/pmej/soccon.htm (accessed on 8 May 2021).
- Cole, M. Cultural Psychology: A Once and Future Discipline, 6th ed.; printing; Belknap Press: Cambridge, UK, 2003. [Google Scholar]
- Domin, D.S. A Review of Laboratory Instruction Styles. J. Chem. Educ. 1999, 76, 543. [Google Scholar] [CrossRef]
- Castillo, K.S.; Paper, P. Discovery Learning vs. Traditional Instruction. Position paper. 2008, pp. 1–9. Available online: http://www.csun.edu/~ksc63842/Posistion_paper.pdf (accessed on 3 April 2021).
- Goos, M. Learning Mathematics in a Classroom Community of Inquiry. J. Res. Math. Educ. 2004, 35, 258–291. [Google Scholar] [CrossRef][Green Version]
- Wells, G. Semiotic Mediation, Dialogue and the Construction of Knowledge. Hum. Dev. 2007, 50, 244–274. [Google Scholar] [CrossRef]
- Juvonen, J. Sense of Belonging, Social Bonds, and School Functioning. In Handbook of Educational Psychology, 2nd ed.; Alexander, P.A., Winne, P.H., Eds.; Lawrence Erlbaum Associates Publishers: Mahwah, NJ, USA, 2006; pp. 655–674. [Google Scholar]
- Gillies, R.M. Promoting Academically Productive Student Dialogue during Collaborative Learning. Int. J. Educ. Res. 2019, 97, 200–209. [Google Scholar] [CrossRef]
- Belenky, M.F.; Clinchy, B.M.; Goldberger, N.R.; Tarule, J.M. Women’s Ways of Knowing: The Development of Self, Voice, And Mind; Basic Books: New York, NY, USA, 1986. [Google Scholar]
- Boomer, G. Negotiating the Curriculum. In Negotiating the Curriculum: Educating for the 21st Century, 2nd ed.; Boomer, G., Lester, N., Onore, C., Cook, J., Eds.; The Falmer Press: London, UK, 1992; pp. 4–14. [Google Scholar]
- Cook, J. Negotiating the Curriculum: Programming for learning. In Negotiating the Curriculum: Educating for the 21st Century, 2nd ed.; Boomer, G., Lester, N., Onore, C., Cook, J., Eds.; The Falmer Press: London, UK, 1992; pp. 15–31. [Google Scholar]
- Shah, R.K. Effective Social Constructivist Approach to Learning for Social Studies Classroom. J. Pedagog. Res. 2019, 3, 38–51. [Google Scholar] [CrossRef][Green Version]
- Brooks, M.; Brooks, J. The Courage to Be Constructivist. Educ. Leadersh. 1999, 57, 18–24. [Google Scholar]
- Topping, K.J.; Buchs, C.; Duran, D.; Van Keer, H. Effective Peer Learning: From Principles to Practical Implementation; Routledge: London, UK; New York, NY, USA, 2017. [Google Scholar]
- Alexander, R.J. Culture, Dialogue and Learning: Notes on an Emerging Pedagogy. In Exploring Talk in School: Inspired by the Work; Douglas Barnes; Mercer, N., Hodgkinson, S., Eds.; SAGE Publications Ltd.: London, UK, 2008; pp. 91–114. [Google Scholar]
- Salih, U.; Melek, K. The Outcomes of Constructivist Learning Environments from the Perspectives of Secondary School Students. Int. Educ. Stud. 2020, 13, 16–28. [Google Scholar]
- Gash, H.; Murphy, F. The Ecology of Teaching and Learning. Constr. Found. 2020, 15, 118–121. [Google Scholar]
- Arroyave, F.; Dasí, A.; Redondo, A. Student Commitment to Social Responsibility: Systematic Literature Review, Conceptual Model, and Instrument. Intang. Cap. 2021, 17, 52–72. [Google Scholar] [CrossRef]
- Chin, C.; Donnell, A.; Jinks, T. The Structure of Discourse in Collaborative Learning. J. Exp. Educ. 2000, 69, 77–89. [Google Scholar] [CrossRef]
- Veenman, S.; Denessen, E.; van den Akker, A.; van der Rijt, J. Effects of a Cooperative Learning Program on the Elaborations of Students During Help Seeking and Help Giving. Am. Educ. Res. J. 2005, 42, 115–151. [Google Scholar] [CrossRef]
- King, A. Structuring Peer Interaction to Promote Higher-Order Thinking and Complex Learning in Cooperating Groups. In The Teacher’s Role in Implementing Cooperative Learning in the Classroom; Gillies, R.M., Ashman, A.F., Terwel, J., Eds.; Computer-Supported Collaborative Learning, 8; Springer: Boston, MA, USA, 2008. [Google Scholar] [CrossRef]
- Gillies, R.M.; Khan, A. The Effects of Teacher Discourse on Students’ Discourse, Problem-Solving and Reasoning during Cooperative Learning. Int. J. Educ. Res. 2008, 47, 323–340. [Google Scholar] [CrossRef]
- Howe, C.; Tolmie, A.; Thurston, A.; Topping, K.; Christie, D.; Livingston, K.; Jessimanc, E.; Donaldsonc, C. Group Work in Elementary Science: Towards Organisational Principles for Supporting Pupil Learning. Learn. Instr. 2007, 17, 549–563. [Google Scholar] [CrossRef]
- Bardach, L.; Yanagida, T.; Morin, A.J.S.; Lüftenegger, M. Is Everyone in Class in Agreement And Why (Not)? Using Student and Teacher Reports to Predict Within-Class Consensus on Goal Structures. Learn. Instr. 2021, 71, 101400. [Google Scholar] [CrossRef]
- Sheridan, D. Teaching Secondary English: Readings and Applications; Longman: New York, NY, USA, 1993. [Google Scholar]
- Benson, P. Teaching and Researching Autonomy in Language Learning; Longman: London, UK, 2001. [Google Scholar]
- Calkins, L. The Art of Teaching Writing; Heinemann: Portsmith, NH, USA, 1986. [Google Scholar]
- Kumar, A.; Teotia, A.K. Constructivism: A Dynamic Approach of Teaching. Int. J. Manag. Appl. Sci. 2017, 5, 135–139. [Google Scholar]
- Applebee, A.N. Literature in the Secondary School: Studies of Curriculum and Instruction in the United States; National Council of Teachers of English: Urbana, OH, USA, 1993. [Google Scholar]
- Langer, J.; Applebee, A.N. How Writing Shapes Thinking: A Study of Teaching and Learning; National Council of Teachers of English: Champaign, IL, USA, 1987. [Google Scholar]
- Whalen, B. Education Abroad and the Undergraduate Experience: Critical Perspectives and Approaches to Integration with Student Learning and Development; Stylus Publishing, LLC: Sterling, VA, USA, 2019. [Google Scholar]
- Darling-Hammond, L.; Flook, L.; Cook-Harvey, C.; Barron, B.; Osher, D. Implications for Educational Practice of the Science of Learning and Development. Appl. Dev. Sci. 2019, 24, 97–140. [Google Scholar] [CrossRef][Green Version]
- Brydon-Miller, M.; Greenwood, D.; Maguire, P. Why Action Research? Action Res. 2003, 1, 9–28. [Google Scholar] [CrossRef]
- Brydon-Miller, M.; Kral, M.; Aragón, A.O. Participatory Action Research: International Perspectives and Practices. Int. Rev. Qual. Res. 2020, 13, 103–111. [Google Scholar] [CrossRef]
- Kemmis, S.; McTaggart, R. Participatory Action Research. In Handbook of Qualitative Research; Denzin, N., Lincoln, Y., Eds.; Sage: Thousand Oaks, CA, USA, 2000; pp. 567–606. [Google Scholar]
- Groundwater-Smith, S.; Dockett, S.; Bottrell, D. Participatory Research with Children and Young People; Sage: Los Angeles, CA, USA, 2015. [Google Scholar]
- Flanagan, J.C. The Critical Incident Technique. Psychol. Bull. 1954, 51, 327–358. [Google Scholar] [CrossRef][Green Version]
- Cohen, L.; Manion, L.; Morrison, K. Research Methods in Education; Routledge: New York, NY, USA, 2018. [Google Scholar]
- Flick, U. An Introduction to Qualitative Research, 4th ed.; Sage: London, UK, 2009. [Google Scholar]
- Bradbury, H.; Reason, P. Conclusion: Broadening the Bandwidth of Validity: Issues and Choice-Points for Improving the Quality of Action Research. In Handbook of Action Research: Participatory Inquiry and Practice; Reason, P., Bradbury, H., Eds.; Sage Publications: London, UK, 2001; pp. 447–455. [Google Scholar]
- Zuber-Skerritt, O.; Fletcher, M. The Quality of an Action Research Thesis in the Social Sciences. Qual. Assur. Educ. 2007, 15, 413–436. [Google Scholar] [CrossRef]
- Carr, W.; Kemmis, S. Becoming Critical: Education, Knowledge and Action Research; Deakin University Press: Geelong, Australia, 1986. [Google Scholar]
- Zuber-Skerritt, O. Action Learning and Action Research: Paradigm, Praxis and Programs. In Effective Change Management Using Action Research and Action Learning: Concepts, Frameworks, Processes and Applications; Sankaran, S., Dick, B., Passfield, R., Swepson, P., Eds.; Southern Cross University Press: Lismore, Australia, 2001; pp. 1–20. [Google Scholar]
- Kalsem, K. Anonymity, Privacy, and Confidentiality. In Ethics in Participatory Research for Health and Social Well-Being; Routledge: New York, NY, USA, 2019. [Google Scholar]
- Brydon-Miller, M.; Rector-Aranda, A.; Stevens, D. Widening the Circle: Ethical Reflection in Action Research and the Practice of Structured Ethical Reflection. In The Handbook of Action Research, 3rd ed.; SAGE: Thousand Oaks, CA, USA, 2015. [Google Scholar]
- Boser, S. Ethic in Power in Community-Campus Partnerships for Research. Action Res. 2006, 4, 9–22. [Google Scholar] [CrossRef]
- Coghlan, D. Doing Action Research in Your Own Organization; SAGE: Thousand Oaks, CA, USA, 2019. [Google Scholar]
- Pintrich, P.T. The Role of Metacognitive Knowledge in Learning, Teaching, and Assessing. Theory Pract. 2002, 41, 219–225. [Google Scholar] [CrossRef]
- Tandon, R.; Hall, B. UNESCO Chair on Community Based Research and Social Responsibility in Higher Education; PRIA and GACER, University of Victoria, School of Public Adminstration; PRIA: New Delhi, India; University of Victoria: Victoria, BC, Canada, 2012. [Google Scholar]
- Daujotienė, L.; Kazlauskienė, A.; Bubnys, R. Teacher Involvement in Organisational Change: From Engaging Risk to Cooperative Learning. Sustainability 2020, 12, 9447. [Google Scholar] [CrossRef]
- Slavin, R.E.; Hurley, E.A.; Chamberlain, A.M. Cooperative Learning and Achievement. In Handbook of Psychology; Reynolds, W.M., Miller, G.J., Eds.; Educational psychology; Wiley: Hoboken, NJ, USA, 2003; Volume 7, pp. 177–198. [Google Scholar]
- Haimovitz, K.; Dweck, C.S. The Origins of Children’s Growth and Fixed Mindsets: New Research and a New Proposal. Child Dev. 2017, 88, 1849–1859. [Google Scholar] [CrossRef] [PubMed]
- Wiliam, D.; Leahy, S. Embedding Formative Assessment: Practical Techniques for K–12 Classrooms; Learning Sciences International: West Palm Beach, FL, USA, 2015. [Google Scholar]
- Boud, D.; Lawson, R.; Thompson, D.G. The Calibration of Student Judgement Through Self-Assessment: Disruptive Effects of Assessment Patterns. High. Educ. Res. Dev. 2014, 34, 45–59. [Google Scholar] [CrossRef][Green Version]
- Jönsson, A.; Prins, F. Editorial: Transparency in Assessment—Exploring the Influence of Explicit Assessment Criteria. Front. Educ. 2019, 3. Available online: https://www.frontiersin.org/articles/10.3389/feduc.2018.00119/full (accessed on 12 June 2021). [CrossRef]
- Wiggins, G.; McTighe, V. The Understanding by Design Guide to Creating High-Quality Units; ASCD: Alexandria, VA, USA, 2011. [Google Scholar]
- Taras, M. Student Self-Assessment: Processes and Consequences. Teach. High. Educ. 2010, 15, 199–209. [Google Scholar] [CrossRef]
- Boud, D. Enhancing Learning Through Self-Assessment, 2nd ed.; Routledge Falmer: London, UK; New York, NY, USA, 2013. [Google Scholar]
|Fragment of Interview||Label (In Vivo)||Subcategory||Category|
|“<e.g., Why to sweat if it is possible to adjust. When you suggest something, you remain guilty or are forced to do that job. If you see that others don’t want, you don’t show your initiative>” C-1||“…to adjust. And I do not show initiative…”||Collegial negotiation||Cooperation|
|“in order to solve the occurred problem or situation, one should try to look at that situation in a different way. I started more deeply exploring a student as a person, trying to understand why one learner behaves in one or another way. At work, I started more deeply exploring a student, one’s emotional state, wellbeing, encouraging deeply exploring the self, learn to know the self, recognize occurring problems and search for ways to cope with them.” A-1||“…to more deeply explore a student as a person… to know the self”||Creation of opportunities to choose|
|Planning Action||Collecting Data||Analyzing Data||Member Checking||Going Public|
|Cooperation||Collegial negotiation||<when they hear my voice, I participate more actively>; <I wish to continue like this—looking at us as having knowledge>; <I see that my ideas, answers on how I feel are cared for>; <I was even surprised when asked my opinion, negotiated for the first time because usually the teacher herself would do everything>; <I feel good when I am asked and I know that they will be glad about my activeness>; <now we exchange opinions, make a joint decision. It is better to learn like this.>; <earlier, a teacher asked to suggest own ideas, and now I often suggest myself>; <I try more to do something when I know that my suggestion is important to all>; <and the result improved a little bit. Perhaps because I could ask many questions. I wanted to answer the questions even more. I didn’t want to do that earlier>; <the teacher writes our suggestions next to her ones, too. She wants that sometimes there were more our suggestions that her. Even though it is hard to work, but I don’t want to be lazy. Because that was my suggestion, after all. How others will treat it.>; <when they listen to my opinion, I try to complete the tasks till the end>|
|Creation of opportunities to choose||<initially, I did not like choosing because the work must be done>; <I tried not to choose, waited until I was told to do. Bet we agreed that I will need to try>; <if I have chosen, there is no way back>; <I started being more self-confident when I decide in what way I will complete the task>; <now I understand that I have my own way, what to after what. It does not suit me what suits, for example, my friend>; <when I choose myself, I try more to complete that work. It is funny, but there is no one to blame then>; <sometimes it is difficult to choose what would you want, you search for an easier option. The teacher noticed that I behaved in such a way and started negotiating with me, whether I would like to try solving more complex problems>|
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.
© 2021 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Kazlauskienė, A.; Gaučaitė, R.; Cañabate, D.; Colomer, J.; Bubnys, R. Sustainable Development of Students’ Assumed Responsibility for Their Own Learning during Participatory Action Research. Sustainability 2021, 13, 10183. https://doi.org/10.3390/su131810183
Kazlauskienė A, Gaučaitė R, Cañabate D, Colomer J, Bubnys R. Sustainable Development of Students’ Assumed Responsibility for Their Own Learning during Participatory Action Research. Sustainability. 2021; 13(18):10183. https://doi.org/10.3390/su131810183Chicago/Turabian Style
Kazlauskienė, Aušra, Ramutė Gaučaitė, Dolors Cañabate, Jordi Colomer, and Remigijus Bubnys. 2021. "Sustainable Development of Students’ Assumed Responsibility for Their Own Learning during Participatory Action Research" Sustainability 13, no. 18: 10183. https://doi.org/10.3390/su131810183