STEM Graduate Students’ Systems Thinking, Modeling and Scientific Understanding—The Case of Food Production
1.1. Systems Thinking
1.2. Conceptual Models and Object-Process Methodology
1.3. Assessment of System Thinking
1.4. Science Understanding Levels
2. Materials and Methods
2.1. Research Setting—The Context of the Study and the Assignments
2.3. Research Tools
2.3.1. Phase I
2.3.2. Phase II
2.4. Data Analysis
3.1. Phase I—Case Study: Experts’ Modeling of Food Production Processes
3.2. Phase II—Assessing Students Systems Thinking, Modeling and Scientific Understanding
- Digital and Visual Design (4/4)—The assignment contains designed interface including colors, models, image, and video clips.
- Food processing and sustainability context (4/4)—The questions combine several issues of food processes and sustainability.
- Variety (1/2)—There is a variety of activities but no special assignment.
- Digital and Visual Design (4/4)—contains designed interface including colors, models, image, and video clips
- Food processing and sustainability context (4/4)—the questions include assimilation of a few issues of both food processes and sustainability. In addition, the assignment integrates multiple complex scientific analysis of the chemical processes
- Variety (2/2)—a variety of activities and a special assignment that did not appear in previous exercises
- The need for iterations to clarify issues: “We had an iterative discussion until we found the optimal solution for the evaluation tool” .
- Acquiring a broad viewpoint: “The process made me think in a broader, systematic viewpoint and to relate to the links between the different components” .
- Applying reverse engineering: “The challenge in creating an OPM model made me use reverse engineering to articulate the assignment” .
Conflicts of Interest
Appendix A. OPL and Modeling Rubric for Assessing SD1 of the Codfish Supply Chain Process
|Attribute||Scoring||Examples in the Model|
|A1-Intended Purpose||Two points. The beneficiary, Household Member Group, exhibits the benefit, Nutrition Value Knowledge at state high.|
|A2-Main Function||Two points. Three sub-processes in addition to the main process. The main process (Cod Tracking) transforms (in this case- changes) Nutrition Value Knowledge from low to high.|
|A3-Main Object||One point. Main object: Nutrition Value Knowledge, plus many other new objects directly linked with sub-processes or with the main process.|
|A4-Structural relations||One point. Two different links: exhibition-characterization link and whole-part link.|
|A5-Procedural relations||One point. Two different links: Effect link and output-input link-pair.|
|A6-Procedural Sequence||One point. Procedural sequence is coherent and detailed.|
|A9-OPL main process functional sentences||One point.|
|A10-OPL main process procedural sentences||One point.|
|Total Scoring: 10/10 points|
Appendix B. OPM Model Submitted by Participant #722122205
- Dori, D. Model-Based Systems Engineering—MBSE. Available online: https://www.edx.org/professional-certificate/israelx-model-based-systems-engineering (accessed on 4 October 2020).
- NGSS Lead States. Next Generation Science Standards: For States, By States. In Next Generation Science Standards: For States, By States; National Academies Press: Washington, DC, USA, 2013; pp. 1–504. [Google Scholar] [CrossRef]
- Accreditation Board for Engineering and Technology. Computing Accreditation Commission Criteria for Accrediting Computing Programs 2019–2020; ABET Publishing: Baltimore, MD, USA, 2019. [Google Scholar]
- Schurz, G.; Lambert, K. Outline of a Theory of Scientific Understanding. Synthese 1994, 101, 65–120. [Google Scholar]
- Dagher, Z.R.; Erduran, S. Reconceptualizing the Nature of Science for Science Education: Why Does It Matter? Sci. Educ. 2016, 25, 147–164. [Google Scholar] [CrossRef]
- Schleicher, A. PISA 2018: Insights and Interpretations; OECD Publishing: Paris, France, 2019. [Google Scholar]
- Sauvé, S.; Bernard, S.; Sloan, P. Environmental Sciences, Sustainable Development and Circular Economy: Alternative Concepts for Trans-Disciplinary Research. Environ. Dev. 2016, 17, 48–56. [Google Scholar] [CrossRef][Green Version]
- York, S.; Lavi, R.; Dori, Y.J.; Orgill, M.K. Applications of Systems Thinking in STEM Education. J. Chem. Educ. 2019, 96, 2742–2751. [Google Scholar] [CrossRef]
- Lavi, R.; Dori, Y.J.; Dori, D. Assessing Novelty and Systems Thinking in Conceptual Models of Technological Systems. IEEE Trans. Educ. 2020, 1–8. [Google Scholar] [CrossRef]
- Shvetsova, O.A.; Lee, J.H. Minimizing the Environmental Impact of Industrial Production: Evidence from South Korean Waste Treatment Investment Projects. Appl. Sci. 2020, 10, 3489. [Google Scholar] [CrossRef]
- Viljoen, A.; Wiskerke, J.S.C. Sustainable Food Planning: Evolving Theory and Practice; Wageningen Academic Publishers: Wageningen, The Netherlands, 2012. [Google Scholar]
- Blumenfeld, P.C.; Soloway, E.; Marx, R.W.; Krajcik, J.S.; Guzdial, M.; Palincsar, A. Motivating Project-Based Learning: Sustaining the Doing, Supporting the Learning. Educ. Psychol. 1991, 369–398. [Google Scholar] [CrossRef]
- Bilgin, I.; Karakuyu, Y.; Ay, Y. The Effects of Project Based Learning on Undergraduate Students’ Achievement and Self-Efficacy Beliefs towards Science Teaching. Eurasia J. Math. Sci. Technol. Educ. 2015, 11, 469–477. [Google Scholar] [CrossRef]
- Crismond, D.P. Scaffolding Strategies For Integrating Engineering Design and Scientific Inquiry in Project-Based Learning Environments. In Fostering Human Development Through Engineering and Technology Education; Brill Sense: Leiden, The Netherlands, 2011. [Google Scholar] [CrossRef]
- Taheri, P. Project-Based Approach in a First-Year Engineering Course to Promote Project Management and Sustainability. Int. J. Eng. Pedagog. 2018, 8, 104–119. [Google Scholar] [CrossRef]
- Wiek, A.; Xiong, A.; Brundiers, K.; van der Leeuw, S. Integrating Problem and Project-Based Learning into Sustainability Programs: A Case Study on the School of Sustainability at Arizona State University. Int. J. Sustain. High. Educ. 2014, 15, 431–449. [Google Scholar] [CrossRef]
- Dori, D. Object-Process Analysis: Maintaining the Balance between System Structure and Behaviour. J. Log. Comput. 1995, 5, 227–249. [Google Scholar] [CrossRef]
- Dori, D. Object-Process Methodology: A Holistic Systems Paradigm; Springer: New York, NY, USA, 2002; pp. 1–12. [Google Scholar]
- Dori, D. Model-Based Systems Engineering with OPM and SysML; Springer: New York, NY, USA, 2016. [Google Scholar] [CrossRef]
- ISO. ISO/PAS 19450: 2015—Automation Systems and Integration—Object-Process Methodology; ISO: Geneva, Switzerland, 2015; Available online: https://www.iso.org/obp/ui/#iso:std:iso:pas:19450:ed-1:v1:en (accessed on 4 October 2020).
- Senge, P.M. The Fifth Discipline, the Art and Practice of the Learning Organization. In Performance + Instruction; Doubleday: New York, NY, USA, 1991; p. 37. [Google Scholar] [CrossRef]
- Von Bertalanffy, L. The Theory of Open Systems in Physics and Biology. Science 1950, 111, 23–29. [Google Scholar] [CrossRef][Green Version]
- Dym, C.L.; Agogino, A.M.; Eris, O.; Frey, D.D.; Leifer, L.J. Engineering Design Thinking, Teaching, and Learning. IEEE Eng. Manag. Rev. 2006, 34, 65–90. [Google Scholar] [CrossRef]
- Crawley, E.; Cameron, B.; Selva, D. Systems Architecture: Strategy and Product Development for Complex Systems; Pearson Education: London, UK, 2015. [Google Scholar]
- Checkland, P. Soft Systems Methodology: A Thirty Year Retrospective. Syst. Res. Behav. Sci. 2000, 17, S11–S58. [Google Scholar] [CrossRef]
- Gero, A.; Zach, E. High School Programme in Electro-Optics: A Case Study on Interdisciplinary Learning and Systems Thinking. Int. J. Eng. Educ. 2014, 30, 1190–1199. [Google Scholar]
- Frank, M. Knowledge, Abilities, Cognitive Characteristics and Behavioral Competences of Engineers with High Capacity for Engineering Systems Thinking (CEST). Syst. Eng. 2006, 9, 91–103. [Google Scholar] [CrossRef]
- Camelia, F.; Ferris, T.L.J.; Cropley, D.H. Development and Initial Validation of an Instrument to Measure Students’ Learning about Systems Thinking: The Affective Domain. IEEE Syst. J. 2018, 12, 115–124. [Google Scholar] [CrossRef][Green Version]
- Whitehead, N.P.; Scherer, W.T.; Smith, M.C. Systems Thinking about Systems Thinking: A Proposal for a Common Language. IEEE Syst. J. 2015, 9, 1117–1128. [Google Scholar] [CrossRef]
- Aubrecht, K.B.; Dori, Y.J.; Holme, T.A.; Lavi, R.; Matlin, S.A.; Orgill, M.; Skaza-Acosta, H. Graphical Tools for Conceptualizing Systems Thinking in Chemistry Education. J. Chem. Educ. 2019, 96, 2888–2900. [Google Scholar] [CrossRef]
- Duit, R.; Treagust, D.F. Conceptual Change: A Powerful Framework for Improving Science Teaching and Learning. Int. J. Sci. Educ. 2003, 25, 671–688. [Google Scholar] [CrossRef]
- Novak, J.D.; Gowin, D.B. Learning How to Learn; Cambridge University Press: Cambridge, UK, 1984. [Google Scholar]
- Ruiz-Primo, M.A.; Shavelson, R.J. Problems and Issues in the Use of Concept Maps in Science Assessment. J. Res. Sci. Teach. 1996, 33, 569–600. [Google Scholar] [CrossRef]
- Dori, D.; Kohen, H.; Jbara, A.; Wengrowicz, N.; Lavi, R.; Soskin, N.L.; Bernstein, K.; Shani, U. OPCloud: An OPM Integrated Conceptual-Executable Modeling Environment for Industry 4.0. Syst. Eng. Fourth Ind. Revolut. 2019, 243–271. [Google Scholar] [CrossRef]
- Mayer, R.E. The Promise of Multimedia Learning: Using the Same Instructional Design Methods across Different Media. Learn. Instr. 2003, 13, 125–139. [Google Scholar] [CrossRef]
- Mayer, R.E.; Moreno, R. Nine Ways to Reduce Cognitive Load in Multimedia Learning. Educ. Psychol. 2003, 38, 43–52. [Google Scholar] [CrossRef][Green Version]
- Assaraf, O.B.Z.; Orion, N. Development of System Thinking Skills in the Context of Earth System Education. J. Res. Sci. Teach. 2005, 42, 518–560. [Google Scholar] [CrossRef]
- Lavi, R.; Dori, Y.J. Systems Thinking of Pre- and in-Service Science and Engineering Teachers. Int. J. Sci. Educ. 2019, 41, 248–279. [Google Scholar] [CrossRef]
- Lavi, R.; Dori, Y.J.; Wengrowicz, N.; Dori, D. Model-Based Systems Thinking: Assessing Engineering Student Teams. IEEE Trans. Educ. 2020, 63, 39–47. [Google Scholar] [CrossRef]
- Dellsén, F. Scientific Progress: Knowledge versus Understanding. Stud. Hist. Philos. Sci. Part A 2016, 56, 72–83. [Google Scholar] [CrossRef][Green Version]
- Kosso, P. Scientific Understanding. Found. Sci. 2007, 12, 173–188. [Google Scholar] [CrossRef]
- European Institution of Innovation and Technology EIT. EIT Food. Available online: https://eit.europa.eu/eit-community/eit-food (accessed on 4 October 2020).
- Wärmefjord, K.; Söderberg, R.; Schleich, B.; Wang, H. Digital Twin for Variation Management: A General Framework and Identification of Industrial Challenges Related to the Implementation. Appl. Sci. 2020, 10, 3342. [Google Scholar] [CrossRef]
- Martinec, T.; Škec, S.; Perišić, M.M.; Štorga, M. Revisiting Problem-Solution Co-Evolution in the Context of Team Conceptual Design Activity. Appl. Sci. 2020, 10, 6303. [Google Scholar] [CrossRef]
- Dori, D.; Jbara, A.; Levi, N.; Wengrowicz, N. Object-Process Methodology, OPM ISO 19450 – OPCloud and the Evolution of OPM Modeling Tools. Syst. Eng. Newsl. PPI SyEN 2018, 61, 6–17. [Google Scholar]
- Dori, D. 111B—Model-Based Systems Engineering and Object-Process Methodology. Available online: https://www.youtube.com/watch?v=5767ZTx9IcM&list=PL7H3hAs1csPj5Gt2PHJT_HqQQo6IzzSjn&index=3 (accessed on 16 October 2020).
- Dori, D. 311—Refinement and Why We Need It. Available online: https://www.youtube.com/watch?v=yyadn_pleHg&list=PL7H3hAs1csPj5Gt2PHJT_HqQQo6IzzSjn&index=19 (accessed on 16 October 2020).
- Belur, J.; Tompson, L.; Thornton, A.; Simon, M. Interrater Reliability in Systematic Review Methodology: Exploring Variation in Coder Decision-Making. Sociol. Methods Res. 2018. [Google Scholar] [CrossRef][Green Version]
- Kottner, J.; Audige, L.; Brorson, S.; Donner, A.; Gajewski, B.J.; Hróbjartsson, A.; Roberts, C.; Shoukri, M.; Streiner, D.L. Guidelines for Reporting Reliability and Agreement Studies (GRRAS) Were Proposed. Int. J. Nurs. Stud. 2011, 48, 661–671. [Google Scholar] [CrossRef]
- Sevian, H.; Dori, Y.J.; Parchmann, I. How Does STEM Context-Based Learning Work: What We Know and What We Still Do Not Know. Int. J. Sci. Educ. 2018, 40, 1095–1107. [Google Scholar] [CrossRef]
- Pabuccu, A.; Erduran, S. Investigating Students’ Engagement in Epistemic and Narrative Practices of Chemistry in the Context of a Story on Gas Behavior. Chem. Educ. Res. Pract. 2016, 17, 523–531. [Google Scholar] [CrossRef]
- Hofstein, A.; Eilks, I.; Bybee, R. Societal Issues and Their Importance for Contemporary Science Education-a Pedagogical Justification and the State-of-the-Art in Israel, Germany, and the USA. Int. J. Sci. Math. Educ. 2011, 9, 1459–1483. [Google Scholar] [CrossRef]
- Gibson, A.; Kitto, K.; Bruza, P. Towards the Discovery of Learner Metacognition From Reflective Writing. J. Learn. Anal. 2016, 3, 22–36. [Google Scholar] [CrossRef][Green Version]
|OPL||Object Process Language|
|SD1||System Diagram—first detail level|
|I||6||Experts in conceptual modeling, food, and STEM education|
|II||15||STEM graduate students who are prospective teachers|
|Phase||Tool||Model Generated or Data Collected|
|I||OPM modeling and food engineering focus group||OPM models of chocolate and codfish food production and distribution systems|
|II||Four assignment sets, one for each learning unit||participants’ systems thinking skill|
|A final assignment including: |
|Feedback questionnaire||Students’ feedback|
|Aspect||System Thinking Attribute||Expected Implementation of the Attribute||Scoring (0–2 or 0–1)|
|Function||A1-Intended Purpose||Beneficiary and benefit are linked with the correct link (Exhibition-Characterization), and both phrased correctly according to the context.||Both beneficiary and benefit are absent: Zero points. Only one of them (beneficiary/benefit) is used or both of them without a correct link: 1 point.|
Both beneficiary and benefit are used, with a correct link: 2 points.
|A2-Main Function||Exactly one systemic main process, which transforms at least one object with the relevant transformation link, all of them phrased correctly according to the context. For SD1, At least three sub-processes, with the same specification as detailed above.||No main process, or a main process which is totally irrelevant to the context: 0 points.|
Main process is correct but transforms no object(s), or is wrongly phrased: 1 point.
Main process transforms at least one object, phrased and linked correctly: 2 points.
|Structure||A4-Main Object||One main object (or more, depends on the context). The main object(s) must be defined and phrased according to the subtext.||No main object, or a main object which is totally irrelevant to the context: 0 points. |
Relevant main object(s): 1 point.
|A5-Structural relations||Correct use of at least one type of link between objects and/or between processes.||Less than one link: 0 points.|
One link or more: 1 point.
|Behavior||A3-Complexity level||Two diagrams are included: one Top-level and one SD1 of any kind.||Only an SD without an SD1 or vice versa: 0 points. |
Two models are included: SD + SD1: 1 point.
|A6-Procedural relations||Correct use of at least two types of links between objects and processes.||Less than two links: 0 points. |
Two links or more: 1 point.
|OPL||A9-OPL main process functional sentences||The presence of OPL sentence(s) depicting the link between the beneficiary and its relevant benefit(s), using an exhibition-characterization link, e.g., “Producer exhibits profit”.||At least one of the following is missing or wrongly phrased (in case of the objects): Beneficiary, benefit, or exhibition-characterization link: 0 points. |
Both beneficiary and benefit are present and phrased correctly, and linked with an exhibition-characterization link: 1 point.
|A10- OPL main process procedural sentences||The beneficiary is linked with an agent link. e.g., “Winemaker handles Harvesting.”|
The operand is linked with a correct result/effect link, e.g., “Bread Making yields Bread Loaf.”, or: “Harvesting changes status of Grape from on tree to picked.”
An instrument or consumption link, used correctly, e.g., “Bread Making requires Mixing Machine.”, or: “Bread Making consumes Flour, Yeast and Water.”
|No more than one of the three sentences is present: 0 points. |
Two or three sentences (out of three) are present: 1 point.
Digital and visual design
|1 point is added for each of the following:|
Food processing and sustainability (F&S) context
|0—The questions are general with no or partial connection to sustainability |
1—The questions include one aspect of food processing or one aspect of sustainability
2—The questions integrate one aspect of food processing and one aspect of sustainability or integration of at least two aspects of food processing or sustainability
3—The questions integrate at least two aspects of food processing and one aspect of sustainability or complex scientific analysis of the process
4—The questions integrate several issues of food processing and sustainability
|0—all the activities are of the same type|
1—a variety of activities, e.g., completing sentences, answering questions, building models, and completing models
2—variety of activities and a special assignment that did not appear in previous exercises
|Attribute||Scoring||Examples in the Model|
|A1-Intended Purpose||2 points. The beneficiary–Household Member Group, exhibits the benefit- Nutrition Value Knowledge at state high.|
|A2-Main Function||2 points. Cod Tracking transforms (in this case-changes) Nutrition Value Knowledge.|
|A3-Complexity level||1 point. Two diagrams are included.||NA|
|A4-Main Object||1 point. The object Nutrition Value Knowledge is phrased in a meaningful way.|
|A5-Structural relations||1 point. Two different links: exhibition-characterization and tagged link.|
|A6-Procedural relations||1 point. Two different links: Instrument link and effect link.|
|A9-OPL main process functional sentences||1 point.|
|A10-OPL main process procedural sentences||1 point.|
|Total Scoring: 10/10 points|
|Attribute||Explanation of Scoring||Score|
|A1-Intended Purpose||Supplier group, Employee, and Market Group are possible beneficiaries. Market Group exhibits Pasta Product at state packaged.||2/2|
|A2-Main Function||Pasta Manufacturing and Pasta Product are connected with result link between the process and the operand.Unpackaged/packaged states in the object Pasta Product are not necessarily relevant to this diagram, considering the whole manufacturing process being described, including the packaging sub-process. Could have been, instead, “raw materials” state as initial state and “packaged product” state as final state.||1/2|
|A3-Complexity level||An SD1 diagram is included||1/1|
|A4-Main Object||Paste Product||1/1|
|A5-Structural relations||Two types: Exhibition-characterization and Aggregation-Participation links.||1/1|
|A6-Procedural relations||Three types: Result/consumption, instrument and agent links||1/1|
|A9-OPL main process functional sentences||“Market Group exhibits Pasta Product at state packaged”||1/1|
|A10-OPL main process procedural sentences||“Pasta Manufacturing yields…and Pasta Product at state packaged”|
“Pasta Manufacturing requires Electrical Energy and Quality Control”
|Attribute||Explanation of Scoring||Score|
|A1-Intended Purpose||Only a benefit (Baked Bread) without its context: beneficiary and a suitable link.||1/2|
|A2-Main Function||In Bread Making Process, Wheat Dough is consumed and Bread is produced. This main process is linked wrongly to a series of asynchronous sub-processes.|
“Bread Making process” should have been phrased “Bread Making”, but no points were taken off for that matter.
|A3-Complexity level||An SD1 diagram is included||1/1|
|A4-Main Object||Wheat Dough||1/1|
|A5-Structural relations||Aggregation-Participation link appears twice, with one correct use. (the minimal number required)||1/1|
|A6-Procedural relations||Only one type: result/consumption||0/1|
|A9-OPL main process functional sentences||No such sentence||0/1|
|A10-OPL main process procedural sentences||Only one of the three sentences—“Bread Making Process consumes Wheat Dough”||0/1|
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.
© 2020 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).
Share and Cite
Akiri, E.; Tal, M.; Peretz, R.; Dori, D.; Dori, Y.J. STEM Graduate Students’ Systems Thinking, Modeling and Scientific Understanding—The Case of Food Production. Appl. Sci. 2020, 10, 7417. https://doi.org/10.3390/app10217417
Akiri E, Tal M, Peretz R, Dori D, Dori YJ. STEM Graduate Students’ Systems Thinking, Modeling and Scientific Understanding—The Case of Food Production. Applied Sciences. 2020; 10(21):7417. https://doi.org/10.3390/app10217417Chicago/Turabian Style
Akiri, Effrat, Marina Tal, Roee Peretz, Dov Dori, and Yehudit Judy Dori. 2020. "STEM Graduate Students’ Systems Thinking, Modeling and Scientific Understanding—The Case of Food Production" Applied Sciences 10, no. 21: 7417. https://doi.org/10.3390/app10217417