IoT Socioenvironmental Assessment Instrument: Validation Process Applying Delphi Method

Featured Application

This study advances the validation of an original, previously published instrument that examines the relationship between IoT and sustainability/circularity approaches within manufacturing organizations. It provides a detailed explanation of the Delphi process to enhance transparency, enabling researchers and managers to replicate the methodology, adapting the instrument to their specific contexts. The validation process is illustrated through the “IoT Technology Expectation” construct, demonstrating its practical application and refinement.

Abstract

Industry 4.0 technologies offer significant opportunities to enhance sustainable production and circular economy practices in the face of challenges arising from climate change. Considering the growing interest in this field, the literature review exposed that, particularly in the case of the Internet of Things (IoT), there is a need for empirical assessments of the impact of this technology on sustainability and circularity. This paper presents the validation process of an original assessment tool that evaluates IoT’s alignment with the socioenvironmental and circular context of manufacturing organizations and their supply chains. Emphasis is placed on the construct titled “IoT Technology Expectations”. After systematically conducting a literature review, this study employed the Delphi method in conjunction with statistical analyses to refine or formulate new indicators or statements based on expert consensus, validating the proposed assessment tool. The findings of this research contribute to management practices by providing an instrument to assess the current stance of top management and other key managers (production, project and supply chain) on IoT use in manufacturing operations or supply chains from a socioenvironmental and circular perspective. The instrument serves as a starting point for exploring IoT’s potential in circular economy practices. Academically, it provides a detailed explanation of the Delphi method and its application and outcomes.

1. Introduction

1.1. Sustainable Manufacturing

Sustainable manufacturing (SM) is one of the important issues to be considered for sustainable development [1,2] and is recognized for its potential to fulfill the Sustainable Development Goals (SDGs) that are suitable for the industrial context [3,4,5,6]. SM distinguishes itself from traditional manufacturing approaches by integrating the Triple Bottom Line (TBL) framework [7]. This framework concurrently addresses three key dimensions: environmental, economic and social [8]. One of the definitions of SM to be considered is that it focuses on creating “manufactured products through processes that minimize negative environmental impacts, conserve energy, natural resources, ensure safety for employees, communities, consumers and maintain economic viability” [9] (p. 4).

The knowledge of SM processes is continuously advancing; however, the implementation presents significant challenges due to variations in organizational structures, economic conditions or regulatory frameworks within different countries [10]. In addition, the focus of SM still predominantly follows a linear economic model, characterized by resource consumption, production and disposal, while business practices seek to minimize environmental impacts and maximize economic gains.

1.2. Circular Economy

The circular economy (CE) perspective emphasizes the restoration of technical cycles and the regeneration of biological cycles while adhering to the principles of sustainability [11]. The “Loop” or “Rs” approach translates CE principles into practical applications, closing the loop of materials, components, products and waste to create other products [11], minimizing waste generation and utilizing recovered materials as inputs in production processes to preserve natural resources [8,12].

1.3. IoT Technology, SM and CE Relationship

The Internet of Things (IoT) technology is recognized as an enabler of Industry 4.0, facilitating the advancement of SM, enhancing sustainability performance [13,14,15] and circularity [16,17].

The literature highlights the role of IoT in SM and the CE, including workplace safety, workforce health, environmental protection and waste management [18,19]. Some researchers argue that IoT technologies can enhance manufacturing efficiency by adhering to the “6R” model for natural resource conservation [8,12], which includes reducing, reusing, recycling, recovering, redesigning and remanufacturing, or the “10R” approach, as it aligns with a smart environment [20]. For instance, IoT has the potential to drive sustainability while influencing the interconnected aspects of economic, environmental and social performance [21].

IoT enables effective planning, monitoring and coordination of production or supply chain processes, fostering collaboration among organizations [22]. The integration of IoT solutions in manufacturing processes strengthens standardization and cooperation; drives service innovation; and supports intelligent automation for real-time monitoring and reporting of resource and material availability. These advancements effectively address critical challenges in SM, including inefficient workforce management, limited knowledge sharing, inadequate information exchange and material supply chain delays [10].

The implementation of IoT in accordance with the SM concept should improve the economic performance (the company’s financial performance), environmental performance (such as reducing waste, emissions and resource consumption) and social performance (related to employee and community well-being), minimizing or eliminating the negative impacts of the production operations in favor of nature, society and minimizing the tendency of some organizations for bluewashing or SDG-washing [4].

The IoT contributes to CE by enabling component reuse, remanufacturing, resource optimization, predictive maintenance, reverse logistics and material recycling [16,23]. In addition, the IoT enables the integration of interconnected smart devices to enhance supply chain (SC) operations [17]. The terms reverse supply chain, closed-loop supply chain, open-loop supply chain and green supply chain are used to describe the application of CE principles in redesigning SC activities from a linear to a circular model.

1.4. Gaps in Literature

There is a demand for empirical research on IoT impact, particularly in the manufacturing sector [18,19]. The application of CE principles in redesigning SC incorporates return processes, contributing to the development of a Circular Supply Chain (CSC). However, empirical research and analytical models examining the IoT’s impact on CSC implementation remain limited [17]. Likewise, there is a claim to assess the impact on sustainability within the digital circular economy, since most of the papers describe the expected effects, not actual ones [24]. In addition, there is a need to assess IoT adoption for multiple performance outcomes, not only focusing on environmental sustainability [16].

1.5. The Original Assessment Tool

These gaps highlight the need for a structured evaluation of the socioenvironmental effectiveness of the Internet of Things (IoT) in manufacturing companies and their supply chains.

The original assessment tool developed by Cavalieri et al. [25,26] provides a foundation for addressing these concerns.

This tool was formulated based on a systematic literature review following the PRISMA framework [27] as detailed in Cavalieri et al. [25,26], requiring empirical validation. In this sense, the authors recommended that future studies analyze the “feedback, new insights and comments from expert opinions—the ‘outsiders’—by consensus methods such as the Delphi Method” [26] (p. 17).

The present study intends to address this recommendation, taking the ‘IoT technology expectations’ construct as a starting point.

1.6. ‘IoT Technology Expectations’ Construct Refinement

Further refinement is necessary to enhance the ‘IoT technology expectations’ construct, such as robustness, applicability and reliability, in real contexts. Expert opinions should assist in identifying underlying concepts that hold relevance for researchers, practitioners and other stakeholders [28]. This study aims to address this issue.

The contributions of experts serve to provide insights and gather their evaluations regarding any components of the original assessment tool that may be perceived as ambiguous or subject to debate. This process is essential for achieving consensus on the tool’s efficacy and suitability.

1.7. Research Questions

Concerning this issue, the research question addressed in this study is as follows: which indicators and statements are suggested as a starting point for assessing the potential of IoT in circular economy practices, based on the application of the Delphi process?

1.8. Methodology

The research methodology adopted in this study follows the Delphi technique for validating the original assessment instrument, which was utilized for reviewing, refining and developing new statements. In addition, this study applies statistical analysis, including content validity for item-level CVIs (I-CVIs), and, for the scale-level average proportion of items (S-CVI/Ave) [29], the standard deviation and the consensus measurement are calculated [30].

1.9. Findings, Contribution and Paper Organization

The findings indicate that the application of the Delphi method initially expanded the ‘IoT technology expectations’ construct to twenty-two items, each representing a different statement regarding IoT adoption, but expert consensus refined it to a final set of ten items in the concluding round of the process.

This research contributes to management by providing an instrument to assess the current stance of top management and other key managers (production, project or supply chain) on IoT use in manufacturing operations or supply chains from a socioenvironmental perspective. The instrument serves as a starting point for exploring IoT’s potential in circular economy practices. This contributes academically by presenting the entire process of the Delphi method, illustrating its application and the results achieved.

This paper is organized as follows: Section 2 describes the research methodology, detailing the Delphi method and the process employed to validate the tool. Section 3 shows the results, describing briefly the original assessment tool and the process of the validation of the instrument exemplified by the construct “IoT technology expectations”. Section 4 presents a discussion of the results. Section 5 sets out the managerial, academic contributions, limitations and suggestions for future research.

2. Materials and Methods

This study employs the Delphi method to validate an original assessment tool proposed by Cavalieri et al. [25,26].

There are three common consensus methods, namely the nominal group process, consensus development panels and the Delphi technique. The Delphi method offers advantages over other techniques [31]. For instance, it minimizes potential biases and external influences associated with face-to-face interactions by ensuring respondent anonymity. The iterative ranking of each item through multiple evaluation rounds by the panel of experts enhances the reliability of the final conclusions, producing more robust outcomes compared to a single meeting. Notably, it does not necessitate specified meeting times [32] offering flexibility, which could be impractical for international experts [33]. Furthermore, Delphi is a method focused on exploring potential and desirable future developments [34] “attempt to address ‘what could/should be’” [35] (p. 213). In this context, the Delphi method was selected as the primary approach for validating the original assessment tool [25].

The Delphi method developed in this study was based on the framework outlined by Paré et al. [34], as shown in Appendix A:

• Selecting experts:

  ○

  The selection was carried out through convenience sampling, which is a type of non-probabilistic sampling characterized by privileged access to respondents [36]. In this sense, the panel size was not determined by a statistical sample aimed at representing a specific population [33].

  ○

  The criteria included the identification of categories of experts, the evaluation of their qualifications, and their knowledge related to the research topic [33]. Although a panel composed of ten members has demonstrated better performance [31], a panel size ranging from 6 to 11 members was considered satisfactory [32].

• Validation phase:

  ○

  After the experts confirmed their participation in the study, the researcher sent them an e-mail with the original instrument and an Excel spreadsheet to support the validation of the instrument’s statements and indicators. The experts were asked to evaluate the criteria as insufficient, good or excellent regarding their unambiguous significance or precise interpretation.

  ○

  The researcher received reviewed statements and the indicators from the experts, in addition to organizing their comments.

  ○

  Statistical analyses were applied, including content validity of individual items (I-CVI) and the average proportion of items (S-CVI/Ave), to quantify the relevancy of each statement by respondents’ agreement and to identify a strong conceptualization of constructs and good statements for content validity.

  ○

  The list was sent back to the experts for validation of the new statements and indicators previously incorporated by them. Additionally, participants should introduce new statements and indicators perceived as not covered.

• Narrowing down phase:

  ○

  This phase involved the refinement of the list of statements from the instrument. The researcher sent the list back to the experts with instructions to classify the statements using a Likert scale with five points: totally disagree, disagree, neutral, agree or totally agree [37].

  ○

  The Likert scale measured the experts’ level of agreement or disagreement with a specific statement, sorting them into a specific order based on preference, importance, quality, or another criterion, indicating their relative position on the list. Any requests for the removal of a statement should be justified.

  ○

  Statistical analyses were applied, including the calculation of the standard deviation and the Content Validity Index (CVI), which supported the decision to remove certain statements.

• Ranking phase:

  ○

  The researcher compiled the statements evaluated in the ‘Narrowing down’ phase and sent the list back to the experts with instructions to provide a binary response (yes or no) on whether to remove statements with duplicity of meanings and regarding the appropriateness of statements with the indicators. Any request for statement removal required a justified explanation.

  ○

  Statistical analyses: As the Ranking phase was the final stage of the Delphi method in this study, the consensus measurement was applied to identify if there is a need for revising the statements, with the aim of enhancing the uniformity of understanding and increasing the level of agreement.

3. Results

3.1. Dimensions and Constructs of Original Assessment Tool

The original assessment tool was developed by applying the systematic literature review conducted in the Web of Science and Elsevier’s Scopus databases with the search terms (‘Sustainable Manufacturing’ OR ‘Sustainable Production’ OR ‘Sustainable Operations’) as “TOPIC” from WoS or “TITLE-ABS-KEY” from Scopus—meaning title, abstract and keyword—in addition to term “Internet of Things” as “TOPIC” or “TITLE-ABS-KEY”, respectively. The PRISMA [27] was applied, as detailed in the works of Cavalieri et al. [25,26]. The initial search was conducted on 14 April 2022; however, it was reviewed on 27 September 2023.

The original assessment tool [25,26] is an instrument that facilitates a comprehensive exploration of respondents’ perspectives, structured into three dimensions and twelve constructs:

• The IoT technology dimension seeks to examine the extent to which industrial organizations recognize and integrate IoT within both strategic and operational contexts. This includes an analysis of IoT technology expectations, the technological capabilities, IoT integration into processes, data management and the challenges and barriers to its adoption. The objective is to assess how effectively IoT is implemented in production processes or supply chains, identifying its potential, strengths and limitations in driving circularity and sustainability.
• The environmental dimension evaluates an organization’s engagement to sustainability, particularly regarding the role of IoT in environmental monitoring, resource management and waste reduction. This assessment involves analyzing procedures, indicators and IoT applications related to ecological impact, aiming to identify areas of strength and improvement in environmentally sustainable and CE approaches.
• The social dimension focuses on the organization’s engagement with IoT in relation to workforce management, partnerships and broader social responsibility initiatives. It examines assessment procedures, performance indicators and the application of IoT in enhancing workplace conditions and stakeholder relationships. The purpose is to identify how IoT contributes to social sustainability, highlighting both opportunities and areas for further development.

These dimensions and related constructs together provide a comprehensive evaluation of IoT’s role in promoting sustainability and circularity practices. While ‘IoT technology dimension’ assesses how IoT contributes to strategic and operational contexts, the environmental and social dimensions explore IoT impact within ecological and workforce management/partnerships contexts.

3.2. ‘IoT Technology Expectations’ Construct

3.2.1. Construct Development Process

As a result of PRISMA, the ‘IoT technology expectations’ construct was derived from Feroz et al. [38] who affirmed that “Digital technologies are bringing radical transformation in the ways value is created and captured. This fact dictates a comprehensive understanding of the whole alignment of organizational strategy and digitalization. The top management should not see digital technologies as agents of improvement but as agents of transformation perpetually manifesting shock waves throughout the organization. In this sense, there is a need to incorporate environmental sustainability into the digital transformation strategy.” (p. 13).

A comprehensive explanation of the development of the ‘IoT Technology Expectations’ construct is provided by Cavalieri et al. [25,26].

3.2.2. Evaluation Purpose

The ‘IoT technology expectations’ construct evaluates whether top management and other key managers utilize IoT with a focus on operational improvements or strategic transformation, which orientation influences the impacts of IoT on the socioenvironmental context [25]. The indicator identified for this construct was “Company leadership’s view of IoT”, defined as “the viewpoint of top managers and managers in relation to IoT technology perspectives, which influences their purpose of IoT implementation.” [26] (p. 12).

3.2.3. Composition of the Original ‘IoT Technology Expectations’ Construct

The ‘IoT Technology Expectations’ construct was originally composed of three statements as presented in Box 1, which were then subjected to expert evaluation through the Delphi method.

Box 1. ‘IoT Technology Expectations’ original construct.

(1) Top management chooses to implement IoT technology, expecting it to be beneficial in the digitalized world.

(2) Top management chooses to implement IoT technology, viewing it as an agent for im-provement.

(3) Top management adopts IoT technology, recognizing it as a transformative agent, which perpetually manifests shock waves throughout the organization.

3.3. Delphi Technique Application

The Delphi method was conducted between August 2022 and September 2023, and the processes of each phase are illustrated in Appendix A.

The detailed implementation is outlined as follows.

3.3.1. Expert Selection

Considering that the principal author is a researcher with expertise in the field, she is familiar with individuals working in the area or in related fields. Consequently, she reached out to university colleagues via email and phone calls to extend invitations for participation. Information regarding managers and consultants was collected through professional colleagues, whereas data on relevant associations was gathered via an online search.

The panel size was initially set between 6 and 11 members, as recommended by Waggoner et al. [32]. The target groups included academics from Federal or State universities in Rio de Janeiro (UFRJ, UERJ, UFF) from the Industrial Engineering departments, consultants and managers specializing in circular economy, or the Brazilian associations concerning the Internet of Things (ABINC—Brazilian Association of the Internet of Things <https://abinc.org.br/>; IOTBRASIL—Brazilian IoT Forum <https://iotbrasil.org.br/index.php>).

Finding individuals willing to participate proved to be difficult. Even when they agreed to join, their level of engagement in the process was lower than expected. As Chang et al. [39] (p. 7) emphasized, “Despite its strengths, the possibility of low response rates, multiple rounds of discussions and maintaining strong feedback remain major challenges for the Delphi method”.

The final panel size comprised six experts: two consultants, two academics and two managers.

Table 1. Experts’ profile ¹.

Professional Title	Area of Expertise	Years of Experience
Academic	Associate Professor—Head of the Department of Industrial Engineering; Production Engineering and Environmental Management—Federal University of Rio de Janeiro.	15
Academic	Associate Professor—Head of the Department of Production Engineering of Faculty of Technology—FAT/State University of Rio de Janeiro; postdoctoral studies in circular economy and regional economic metabolism at the University of Aveiro.	11
Consultant	Experienced professional with over 15 years of expertise in the IoT and Telecommunication sectors, specializing in IoT and intelligent factory solutions. Held different responsibilities within Sigfox, including operations, country management and head of sales in South America. Managing Director of Managium (Brazil), services consulting for LPWAN/Cellular IoT, Telecommunications Business Development activities in Latin America.	15
Consultant	“World Economic Forum Industrial IoT” expert contributor, focusing on IoT 4th Industrial Revolution, building up support and establishing Industrial IoT end-to-end solutions/applications as the digital standard within different industries.	20
Top Manager	Factory in the wooden artifacts segment	Not informed
Manager	Factory in the wooden artifacts segment	Not informed

Note: ¹ Informed by experts.

shows the experts involved in this study.

3.3.2. Data Collection and Analysis

• Validation Phase

The first round began with the panel of experts receiving the original instrument and corresponding spreadsheets via email. Experts were invited to critically evaluate the proposed statements as insufficient, good or excellent, providing scores (1 = insufficient, 2 = good, 3 = excellent). Initially, there were three statement items as presented in

Table 2. Validation phase—first round.

Statement Items	Expert 1	Expert 2	Expert 3	Expert 4	Expert 5	Expert 6
(1) Top management chooses to implement IoT technology, expecting it to be beneficial in the digitalized world.	1	1	1	2	2	3
(2) Top management chooses to implement IoT technology, viewing it as an agent for improvement.	3	2	2	3	3	3
(3) Top management adopts IoT technology, recognizing it as a transformative agent, which perpetually manifests shock waves throughout the organization.	1	3	1	2	3	3

Note: The values represent the evaluation scores provided by the experts (1 = Insufficient, 2 = Good, 3 = Excellent) regarding the three statements.

.

Statistical analyses were applied, including content validity of individual items (I-CVI) and the average proportion of items (S-CVI/Ave).

The content validity of individual items (I-CVI) [29] was applied, calculating an item level from six experts to quantify the relevancy of each statement by the respondents’ agreement. Concerning the total items, the S-CVI/Ave corresponded with the average proportion of statements.

The literature [29] recommends a scale to reach a strong conceptualizations of constructs and good statements for content validity as “it would be composed of items with I-CVIs that meet Lynn’s, 1986 criteria (I-CVI 1/4 1.00 with 3 to 5 experts and a minimum I-CVI of 0.78 for 6 to 10 experts) and it would have an S-CVI/Ave of 0.90 or higher.” [29] (p. 496). If the initial I-CVIs are low, the recommendation is to take two rounds of expert review, suggesting the need for statement improvements and better instructions to the experts [29].

Concerning the validation phase, scores of 2 or 3 were considered valid. The calculation of I-CVI counted the number of scores ≥ 2 for each item and divided them by the total number of six experts. The formula employed in Excel was

CONT.SE(B2:G2,“≥2”)/6

(1)

The results were

Item 1, I-CVI = 0.5;

Item 2, I-CVI = 1;

Item 3, I-CVI = 0.666.

Items 1 and 3 resulted in I-CVI being lower than 0.78. Concerning the scale-level index S-CVI/Ave, the result was 0.722, which did not reach the level expected.

The second round consists of experts suggesting revisions and recommending additional statements. The experts conducted a comprehensive review and expanded the statements to twenty-two items, as shown in Appendix B.

• Narrowing down phase

The first round began with the experts scoring the statements using a 5-point Likert scale, as presented in

Table 3. Narrowing down phase—first round.

Items	Expert 1	Expert 2	Expert 3	Expert 4	Expert 5	Expert 6
(1)	5	4	3	4	4	5
(2)	3	5	3	4	3	4
(3)	5	5	4	4	4	5
(4)	3	5	2	4	3	3
(5)	3	4	5	4	4	4
(6)	5	4	2	4	3	4
(7)	3	5	3	4	4	3
(8)	5	4	5	4	5	4
(9)	3	4	2	3	3	4
(10)	3	2	4	3	3	3
(11)	3	4	5	4	4	4
(12)	5	5	4	5	4	5
(13)	5	5	4	5	4	5
(14)	5	4	5	4	4	5
(15)	2	4	3	3	3	3
(16)	2	5	3	4	3	3
(17)	2	5	5	4	4	4
(18)	2	4	3	3	3	4
(19)	2	5	4	4	4	4
(20)	5	4	5	4	5	4
(21)	3	5	2	3	3	4
(22)	5	4	4	4	4	5

Note: The values represent the evaluation scores provided by the experts (totally disagree = 1, disagree = 2, neutral = 3, agree = 4, totally agree = 5) regarding the twenty-two items.

. The Likert scale ranged from Completely Disagree (score of 1) to Completely Agree (score of 5) concerning whether statements capture the information necessary to fulfill the objective of the indicator (see Appendix B).

Statistical analyses were performed, including the calculation of the standard deviation and the Item-level Content Validity Index (I-CVI), which supported the decision to remove certain statements.

The items presenting a standard deviation exceeding 0.90 were considered insufficient consensus among the experts and were consequently recommended for exclusion. The formula employed in Excel was

DESVPAD.P(C4:H4)

(2)

The scores of 4 or 5 were considered valid. The calculation of I-CVI counted the number of scores ≥ 4 for each item and divided them by the total number of six experts. The formula employed in Excel was

CONT.SE(B2:G2,“≥4”)/6

(3)

The results of the standard deviation and I-CVI analyses are presented in

Table 4. Standard deviation and I-CVI values.

Items	Standard Deviation	I-CVI
(1)	0.687184271	0.833333333
(2)	0.745355992	0.5
(3)	0.5	1
(4)	0.942809042	0.333333333
(5)	0.577350269	0.833333333
(6)	0.942809042	0.666666667
(7)	0.745355992	0.5
(8)	0.5	1
(9)	0.687184271	0.333333333
(10)	0.577350269	0.166666667
(11)	0.577350269	0.833333333
(12)	0.471404521	1
(13)	0.471404521	1
(14)	0.5	1
(15)	0.577350269	0.166666667
(16)	0.942809042	0.333333333
(17)	1	0.833333333
(18)	0.687184271	0.333333333
(19)	0.897527468	0.833333333
(20)	0.5	1
(21)	0.942809042	0.333333333
(22)	0.471404521	1

. The gray-highlighted rows indicate the items that were removed based on the previously established criteria. Specifically, items with a standard deviation greater than 0.90 were considered insufficient consensus. Additionally, a minimum I-CVI of 0.78 (for panels of 6 to 10 experts) and an S-CVI/Ave of 0.90 or higher were regarded as indicative of strong construct conceptualization and satisfactory content validity of the statements.

As shown in Table 4, all items with an I-CVI of 0.333333333 were removed (gray-highlighted rows). Conversely, items presenting a standard deviation of 0.745355992 (green-highlighted rows) and an I-CVI of 0.50 were considered satisfactory. The overall S-CVI/Ave reached 0.86, indicating acceptable content validity for the set of statements, although values of 0.90 or higher are generally regarded as ideal.

Second round: The experts received the corresponding spreadsheet shown in Table 4 to confirm the result regarding the items removed.

• Ranking phase

The anonymity of panel members was guaranteed until the ‘Narrowing Down’ phase. On the other hand, in the Ranking phase, two online meetings were conducted to reach a final consensus. The experts had a last opportunity to evaluate the suitability of the statements for the construct and their appropriateness for the indicators.

First round: The experts received the list from the ‘Narrowing Down’ phase, along with instructions to provide a binary response (yes or no) regarding the removal of validated statements in the second round of it. This evaluation focused on identifying statements with duplicity of meaning and assessing their relevance to the construct and/or indicators. The results are presented in

Table 5. Ranking phase—first round.

Validated Items from Round 2 of the ‘Narrowing Down’ Phase	Justification for Removing Statement Items
(1)
(2)
(3)
(5)
(7)
(8)	Duplicity
(11)	Distinct parameters measured in one statement
(12)
(13)
(14)	Too generic
(20)
(22)

.

Statistical analyses were applied following the recommendation that the Delphi method consider consensus levels typically ranging from 51% to 80% [30], depending on the subject matter. In particular, a threshold of 70% or higher is often preferred, as it offers a balanced approach between inclusivity and decision-making rigor [30].

The formula employed in Excel for consensus measurement was

CONT.SE(B2:I2;MODA(B2:I2))/CONT.VALORES(B2:I2)

(4)

The overall result was 0.63 (see

Table 6. Ranking phase—consensus measurement.

Items	Expert 1	Expert 2	Expert 3	Expert 4	Expert 5	Expert 6	Moda	% Consensus
(1)	5	4	3	4	4	5	4	0.571428571
(2)	3	5	3	4	3	4	3	0.571428571
(3)	5	5	4	4	4	5	5	0.571428571
(5)	3	4	5	4	4	4	4	0.714285714
(7)	3	5	3	4	4	3	3	0.571428571
(12)	5	5	4	5	4	5	5	0.714285714
(13)	5	5	4	5	4	5	5	0.714285714
(20)	5	4	5	4	5	4	5	0.571428571
(22)	5	4	4	4	4	5	4	0.714285714
Result								0.634920635

), indicating that 63% of the experts achieved a consensus. This level of agreement is considered acceptable, as it falls within the commonly referenced range of 51% to 80%. However, it is important to note that consensus levels below 60% are typically interpreted as low, suggesting considerable divergence in experts’ perceptions regarding the evaluated statements. The items (5), (12), (13) and (22) achieved a higher level of consensus compared to the others.

Second round: The experts reached a consensus on the inclusion of an additional statement, “Top management implemented IoT to achieve intangible benefits (e.g., customer satisfaction/retention/loyalty)”. This addition highlights the strategic role of IoT adoption in generating value beyond operational efficiency, for instance, its potential to promote long-term relationships and improve the overall customer experience.

The statements were revised using the first-person singular verb, indicating the individual as the subject of the action, or the second-person plural to indicate the group of people as the subject of the action. The aim was to hold professionals from the manufacturing industry, who should participate in this study in the future, more accountable for their opinions regarding the socioenvironmental use of IoT in the organizations where they worked.

The indicators, validated statements related to the ‘IoT Technology Expectations’ construct and the resulting hypothesis regarding the potential contribution of IoT to circular economy practices and sustainability are presented below:

Company leadership’s view of IoT:

• IoT enables us to maintain competitiveness in relation to our competitors:

  ○

  Hypothesis 1—The adoption of IoT technology should support operational improvements or economic sustainability, enhancing the company’s competitiveness.

• There are new business opportunities with the adoption of IoT in our operations:

  ○

  Hypothesis 2—The implementation of IoT should support industrial organizations in adapting to strategic changes, particularly the adoption of circular economy practices, thus creating new business opportunities.

• IoT is a driving force for improvement in reducing our operational costs:

  ○

  Hypothesis 3—IoT should facilitate operational improvements and contribute to economic sustainability by reducing operational costs.

• IoT brings the opportunity to create processes with our customers:

  ○

  Hypothesis 4—IoT should facilitate collaboration between companies and customers, applying the Rs (Reduce, Reuse, Recycle) paradigm of the circular economy to create new processes.

• We implemented IoT to achieve intangible benefits (e.g., customer satisfaction/retention/loyalty):

  ○

  Hypothesis 5—The adoption of IoT should foster innovative ideas for new customer services, leading to intangible benefits such as improved customer satisfaction, retention and loyalty.

Production Manager’s view of IoT:

• IoT allows integration with our suppliers:

  ○

  Hypothesis 6—IoT should facilitate communication and collaboration with suppliers, supporting the application of the Rs paradigm of the circular economy and enabling seamless integration.

• IoT is a key technology in the digital transformation process of our manufacturing plant:

  ○

  Hypothesis 7—IoT should support operational improvements and economic sustainability, playing a critical role in the digital transformation of manufacturing processes.

• IoT has been implemented to enhance the efficiency of machines in our production lines:

  ○

  Hypothesis 8—IoT should contribute to operational improvements and economic sustainability by enhancing machine efficiency within production lines.

Supply Chain Manager’s view of IoT:

• Tracking applications enable us to monitor our goods’ flow in real-time within the factory, from raw material receipt to product movement through internal sectors:

  ○

  Hypothesis 9—IoT should improve the monitoring and reporting of resource and material availability throughout the manufacturing process, enabling real-time tracking from raw material receipt to product movement.

• Tracking applications enable us to monitor our logistics process in real-time outside the factory, from goods exit through the packaging process to distribution for reaching the consumer:

  ○

  Hypothesis 10—IoT should enhance resource usage by enabling maintenance, supporting reverse logistics, optimizing material recycling and encouraging reuse by monitoring logistics in real-time from goods exit through packaging and distribution.

The hypotheses established for the ‘IoT Technology Expectations’ construct should be tested through the application of the comprehensive assessment tool, specifically developed to evaluate the contribution of IoT technologies to circular economy practices and sustainability within manufacturing organizations.

4. Discussion

There has been a noticeable surge in survey-based research, as highlighted by Fairclough and Thelwall [40], emphasizing the recognition of human perspectives as valuable sources of knowledge in investigating various phenomena. However, the proliferation of online survey platforms has raised concerns about the declining quality of research [41], thereby impacting citation influence [40].

Survey research faces criticisms regarding its development, usability and analysis phases, with issues that threaten its validity, as pointed out by Fairclough and Thelwall [40]: (a) the development of the survey, specifically concerning unprofessionally constructed questions; (b) usability issues, such as ‘sample bias’, where people surveyed are not representative of the population of interest, ‘non-response bias’, where people completing the survey are not representative of the people receiving it, and ‘response bias’, where people completing the survey give inaccurate answers; and (c) unprofessional analysis, leading to inadequate reporting.

The ‘development of the survey’ aligns with Gowin’s Vee knowledge map [39], adopting a dual approach encompassing theoretical and methodological dimensions. In the theoretical perspective, this study conducts a comprehensive review of the literature to enhance comprehension of fundamental concepts, principles, rules and theories within the realm of IoT in the socioenvironmental context, investigating topics such as sustainable manufacturing, circular economy, TBL and their relationship with IoT, which serves as a foundation for the development of the original assessment tool. The development of the original instrument [25,26] preceding this study adhered to the PRISMA framework, addressing the need for rigorous survey research within the field of production management and operations [42].

As highlighted by de Jesus et al. [43], the Delphi technique serves as a reflective approach, particularly appropriate for identifying future trends in fields characterized by incomplete or uncertain knowledge. From a methodological perspective, this research utilizes the Delphi technique to collect, analyze and refine the statements within the assessment tool, ensuring alignment with the specific objectives and context of this study. The main goal of this technique is to critically evaluate the original instrument’s statements and indicators, making necessary adjustments and developing new ones based on expert input. The insights and feedback gathered from specialists play an important role in enhancing the assessment tool, contributing to a more thorough review that strengthens the reliability and validity of the findings [39].

5. Conclusions

This study presented the validation process of a tool that assesses the alignment of IoT with the socioenvironmental context of manufacturing organizations and their supply chains. It also contributes to management by analyzing an instrument that is a starting point to support the industrial decision-making process regarding the use of IoT guided by sustainability/circularity approaches. A detailed explanation of the Delphi process enhances this study’s transparency, enabling other researchers to replicate the method and ensuring methodological rigor, which are essential for scientific research. In addition, this study highlights how expert input shaped the final outcomes. In this sense, it directly addresses the following research question: which indicators and statements are suggested as a starting point for assessing the potential of IoT in circular economy practices, based on the application of the Delphi process? The validation was conducted exclusively within the scope of the ‘IoT technology expectation’ construct due to the complexity involved in demonstrating the validity of the entire instrument.

The application of the Delphi method initially expanded the ‘IoT Technology Expectations’ construct to twenty-two statements, as presented in Appendix B. The indicator ‘Company leadership’s view of IoT’ was further expanded to include the ‘Production manager’s view of IoT’, ‘Project manager’s view of IoT’ and ‘Supply chain manager’s view of IoT’, broadening the scope of perspectives considered. This modification facilitated the inclusion of different professional viewpoints, contributing to the expansion of statements. Concluding the rounds of the process, the expert consensus reduced the number of statements significantly, resulting in a final set of ten statement items.

5.1. Addressing Research Gaps

In line with the requirements identified in Section 1.4, this study provides a comprehensive response as follows:

• Gap 1: There is a demand for empirical research on IoT impact, particularly in the manufacturing sector [18,19].

The ‘IoT technology expectations’ validated construct should be employed in real contexts as a starting point.

• Gap 2: The application of CE principles in redesigning SCs incorporates return processes, contributing to the development of a Circular Supply Chain (CSC). However, empirical research and analytical models examining the IoT’s impact on CSC implementation remain limited [17].

The statements from the ‘Supply Chain Manager’s view of IoT’ indicator intend to match this gap.

• Gap 3: Likewise, there is a claim to assess the impact on sustainability within the digital circular economy, since most of the papers describe the expected effects, not actual ones [24].

As previously affirmed, the statements were revised using the first-person singular verb, indicating the individual as the subject of the action, or the second-person plural to indicate the group of people as the subject of the action. Additionally, the verbs in the statements are written in the present simple tense to emphasize the description of current effects.

• Gap 4: In addition, there is a need to assess IoT adoption for multiple performance outcomes, not only focusing on environmental sustainability [16].

The hypotheses 2, 4, 5 and 6 intend to complement this gap, concerning IoT implementation to support industrial organizations in adapting to strategic changes, particularly through the adoption of circular economy practices, by facilitating collaboration with customers and suppliers, applying the Rs paradigm (Reduce, Reuse, Recycle) to create new processes, fostering innovative ideas for customer services and generating intangible benefits such as improved customer satisfaction, retention, loyalty and seamless integration across the value chain.

5.2. Managerial and Academic Contributions

Following a transparent step-by-step process, this study contributed to minimizing one of the main limitations of the Delphi method, which is its replicability, as the method is highly dependent on contextual factors, the specialized knowledge of the panel members and the availability of participants throughout the process.

Practitioners should utilize the ‘IoT Technology Expectations’ indicators and statements along with other stakeholders as an aid to understand the ‘how’, ‘why’ and consequences, enabling specific planned activities and/or innovative actions to maximize strengths, minimize or eliminate identified weaknesses and monitor future changes. This opens the possibility for the negotiation of ideas, projects and strategies, promoting discussions within the manufacturing organization towards sustainability and circularity, not only in economic terms but also in relation to the social and environmental context within its production operation and supply chain.

This necessary study serves as a valuable starting point for peers and the wider academic community to unlock the full benefits of IoT technologies in the socioenvironmental context in the manufacturing industry.

5.3. Research Limitations and Future Research

Most experts who were unavailable or uninterested were screened out, resulting in a final participant count of six, which is the minimum acceptable number. However, those who remained made significant contributions to this study, drawing on their extensive experience and expertise. This is evidenced by the consensus measurement, which indicated that 63% of the experts reached an agreement. This level of consensus is considered acceptable, falling within the commonly cited range of 51% to 80%.

Establishing a foundation for future research, the entire instrument should be applied in a real context, assessing the IoT in loco through the participation of a larger group of managers.