Next Article in Journal
Seasonal Variation of Vegetation and Its Spatiotemporal Response to Climatic Factors in the Qilian Mountains, China
Next Article in Special Issue
Impact of Public Service Quality on the Efficiency of the Water Industry: Evidence from 147 Cities in China
Previous Article in Journal
Heterogeneous Impact of Economic Policy Uncertainty on Provincial Environmental Pollution Emissions in China
Previous Article in Special Issue
Quantifying the “Water–Carbon–Sulfur” Nexus for Coal Power Plants in China
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:

A Water Footprint Management Construct in Agri-Food Supply Chains: A Content Validity Analysis

Marcelo Werneck Barbosa
1,* and
José M. Cansino
Facultad de Administración y Negocios, Universidad Autónoma de Chile, Pedro de Valdivia 425, Providencia 7500912, Santiago, Chile
Department of Economic Analysis and Political Economy, Facultad de Ciencias Económicas y Empresariales, Universidad de Sevilla, Avda. Ramón y Cajal 1, 41018 Sevilla, Spain
Author to whom correspondence should be addressed.
Sustainability 2022, 14(9), 4928;
Submission received: 24 January 2022 / Revised: 30 March 2022 / Accepted: 17 April 2022 / Published: 20 April 2022
(This article belongs to the Special Issue Environmental Footprint and Sustainable Development)


Common problems when carrying out water footprint (WF) assessments are obtaining specific primary data, dealing with the complexity of its computation, and the availability of quality data. In a supply chain context, inconsistencies are even more exacerbated. In order to fill in this research gap, this study proposes and evaluates the content validity of a survey scale to assess WF management initiatives implemented by companies, with a focus on supply chains and the agriculture industry. In order to do so, a literature review was performed to identify candidate survey items whose content was later validated with experts in terms of their relevance, clarity, and essentiality to measure WF management. Content validity was assessed using several indices (items’ content validity index (I-CVI), Kappa’s coefficient, Aiken’ V coefficient, and content validity ratio (CVR)), which indicated high content validity for the selected items. This study provides a set of measurement survey items that can be used to evaluate WF management initiatives implemented in agri-food supply chains in future empirical studies.

1. Introduction

Water is a fundamental resource for our existence and socio-economic development. The establishment of the United Nations’ Sustainable Development Goal (SDG) number 6, which refers to water quality and withdrawal reduction [1], corroborates the world’s concern with sustainable water management. In order to support water use and assessment, Hoekstra and Hung introduced the concept of the water footprint (WF), which is defined as the total volume of water resource used, directly or indirectly, to produce goods and services [2]. It includes not only water directly used to deliver products, but also water indirectly consumed and polluted throughout the production processes [3]. One of the sectors that is most responsible for water consumption, degradation, and pollution is the agriculture industry [4,5]. Irrigated agriculture accounts for most of the total consumptive water use worldwide [6]. For these reasons, agri-food water resources management research is a topic of increasing research interest [7].
Despite the existence of varied measurement indicators, WF assessments currently face some challenges, which need to be explored by researchers in this field. The first common problem when adopting WF assessments is obtaining specific data, which is specially a challenge for products with long and complex supply chains [8]. In fact, due to the difficulty in obtaining local primary data for water assessments, several authors use literature data [9] and official water accounting data, with few opportunities for direct data collection [10]. The second challenge is related to difficulties in implementing WF due to the complexity of its computation [5,11]. The third concern is that the quality of WF assessments depends upon the quality of available data. In a supply chain context, inconsistencies are even exacerbated through the chain complexities due to difficulties in obtaining the necessary data from partners to carry out WF assessments [7]. In fact, researchers recognize that there can be significant differences between the real data and official statistics due to the use of distinct methodologies [10]. In addition, each measurement technique or indicator has its own limitations. For instance, at farm scale, WF assessments do not take into account information about irrigation management, so related data must be incorporated into the WF assessment [12]. Fourth, WF values of different studies can be substantially uncertain and highly variable [13]. Sometimes, due to constraints related to data availability and to inconsistencies in databases, the results of assessments need to be considered as best estimates [14]. Moreover, existing inconsistencies between different evaluation approaches and metrics can lead to uncertainties that might be confusing to decision makers, consumers or manufacturers/retailers, hampering their ability to develop sustainable solutions [15] and to make decisions. Fifth, results published based on standardized approaches sometimes do not consistently follow the respective established guidelines, which could lead to misinformation whenever results are published without specifying existing limitations [16]. The reported problems and inconsistencies in the interpretation and application of these indicators have resulted in incomparable and sometimes contradictory results [15]. Finally, there is also criticism concerning the usage of some water usage-related metrics [17], among different research communities on the subject. While part of the scholars in the field consider that WF assessment should be based on volumetric indicators, other researchers defend that these measurements should be focused on local impact of water use [18].
Moreover, Lovarelli et al. [19], through a review study on WF use in agricultural production, found out that the vast majority of studies published so far are aimed at directly quantifying WF. Other measurement mechanisms must be proposed so that studies that do not count with available raw data can be carried out. Finally, water footprint was considered by Kamble et al. [20] an important dimension to define the concept of environmental sustainability performance of an agri-food supply chain (AFSC), but no measurement items have been provided to define this constructs. As so, quantitative studies aiming to explore WF management practices become difficult to operationalize because no scale that can measure WF management in a reliable and valid manner has been developed until now. The lack of a specific measurement scale for WF management indicates a crucial knowledge gap. In conclusion, the gaps, concerns, and problems previously presented show that other ways of assessing WF management, especially in agri-food supply chain contexts, are necessary.
Different methods for scale development have been proposed [21,22,23]. Although these methods have slightly different procedures, in general, all of them include determining the measurement goals, generating a pool of measurement items, having these items assessed by experts, and finally, administering these items to a larger sample. In order to fill the aforementioned research gaps, this study proposes and evaluates the content validity of a survey scale to assess WF management initiatives implemented by companies, with a focus on supply chains and the agriculture industry. As so, the focus of this study is on the first activities of the scale development process, which include item generation and their judgement by experts for content validity. These activities were carried out with the support of Chilean experts in the field. Content validity is the “degree to which a measure’s items represent a proper sample of the theoretical content domain of a construct” [24]. It is aimed at assessing that the scale items will properly represent the construct [25]. Therefore, the scale items proposed in this study fill in the research gaps presented, since they do not require raw water footprint data availability, nor they do not rely on complex data gathering or computation procedures.
In order to achieve this objective, a systematic literature review was performed to identify candidate survey items, whose content was later validated by experts. The main contribution of this study is to provide a set of measurement survey items that can be used to evaluate WF management initiatives implemented in AFSCs. It is important to highlight that the measurement items proposed in this study are not a direct measurement method like the techniques used for water footprint assessment, but an alternative method that evaluates WF management initiatives that can be applied when it is not possible to directly measure WF due to the previously presented constraints.
This study is grounded on the Natural-Resources-Based View (NRBV), which states that firms can generate unique sources of competitive advantage through proactive engagement with the natural environment [26] and that competitive advantage will be achieved if firms can manage natural resources and develop capabilities to address environmental changes [27]. Under the NRBV, competitive advantage is achieved if companies can create profits that are supported by resources that are not easily duplicated by competitors [28]. Being able of managing WF along the supply chain is a source of competitive advantage.

2. Theoretical Background

The WF indicator refers to the use of water resources in products or services and water pollution during production. WF assess the use of water resources throughout all the stages of a product’s life cycle (production, transportation, marketing, consumption, and reuse) [29]. It is a multidimensional indicator composed of three components: the blue WF and green WF, which measure direct water consumption, and grey WF, which measures the pollution produced [2]. The blue WF refers to the volume of surface and groundwater consumed to produce a good. The green WF refers to the rainwater consumed during the production process. It addresses the rain water stored in the soil and then absorbed by plants [30]. Green WF is particularly of interest for agricultural products [6]. The grey WF of a product refers to the volume of freshwater that is required to assimilate the load of pollutants [31]. It considers the use of pesticides and fertilizers applied in the fields [32].
WF measures water consumed and the impact of the production processes in the quantity and quality of water [9]. It indicates net water consumption, that is, water that cannot be used for other purposes [33]. In irrigated agriculture, WF is a key indicator for evaluating sustainability [6]. Due to its relevance, WF research has been growing and has become one of the predominant topics in the research area of water resources management [29]. In fact, Zhang et al. [34] found out that the mainstream research related to WF is its accounting methodologies and application in water resource management.
The WF concept is integrative applicable at different levels and along supply chains [35]. It is usually focused on three different perspectives: the production perspective, the trade perspective, and the consumption perspective [8] and it has been adopted at national, corporate and product levels [30]. As so, WF can be calculated for a specific product, for geographical locations or producers [36]. The assessment of WF also considers the whole supply chain, that is, every relevant stakeholder involved in water management [37]. Supply chains, besides facing several current challenges (risks, disruptions, lockdowns) due to the COVID-19 pandemic’s effects on production and distribution [38], also have to be concerned about sustainability issues and customers’ environmental pressures [39]. In AFSCs, WF management methods and initiatives can be used to assure that supply chain partners comply to sustainability standards and goals [40]. WF management across industry supply chains is a central topic in the public agenda [41]. In spite of being a growing research field, WF assessment studies at a supply chain extent are rather limited [30]. The full potential of WF analysis along supply chains needs to be realized still [35].
Water resource management is an effective way to measure the supply and demand of water resources and improve the efficiency of its use [42]. It includes: monitoring, modelling, exploration, assessment, policy design, performance monitoring, and maintenance [43]. It is a way to protect food production, reduce poverty, and eliminate water-related diseases. However, freshwater supply is a major global concern, particularly in developing countries [36].
In this context, based on the definition of WF and on previous studies [30,44], this study considers WF management to be the use of policies and actions at the strategic, tactical, and operational levels that support in mitigating the WF of a product across its entire supply chain. WF management is essential for promoting environmental sustainability in the agri-food industry [11] and profitability through increased operational efficiency [39]. It has been widely used for evaluation of water efficiency in agricultural production since it is an important indicator to manage agricultural water resources. It is significantly affected by water management practices [13]. Due to its relevance, it has been incorporated into companies’ business strategies [30].
Researchers have been calling for a more holistic management of WF to address sustainable water use in the whole supply chain [8]. Sustainable water management in the supply chain is critical to the long-term viability of some agricultural organizations [45]. Chico et al. [46] consider that strategies to reduce the impact of a product WF need to take into account the WF along the whole chain [46]. More research is needed in order to determine actions to reduce the impacts of water usage along supply chain echelons [30].

3. Materials and Methods

Survey questionnaires are the most used tool to collect data because they allow data collection from a huge population quickly and at a lower cost. They are also convenient to respondents, can provide anonymous answers, reduce interviewer bias, and allow question standardization. However, some notorious disadvantages of a questionnaire include poor data quality (incomplete responses), wording problems, and inaccurate questions [47].
Establishing content validity involves judgment by experts as to whether scale items reflect the appropriate content to capture the concept being assessed. Content experts are required to assess content validity because they could potentially improve the content validity of measurement items [48]. Content validity determines whether the items of a survey questionnaire provide adequate coverage of a topic under investigation [49].
The methodology for identifying measurement items and validating their content was based on previous published studies [47,48], which establish rigorous procedures for scale development that include item generation and expert review [50]. In order to carry out this task, a deductive method was used. First, items were identified from a systematic literature review, then, experts reviewed each item and the construct definition as a whole [47] to assess their capacity of measuring the construct as well as their readability.
Expert judges are highly knowledgeable about one knowledge domain. They evaluate each survey item to determine whether they represent the domain of interest. Their assessments have been quantified using formalized scaling and statistical procedures such as the content validity ratio for quantifying consensus [51]. They come both from academy and industry. They can be a person with knowledge and expertise in the construct being developed, someone familiar with the target population on whom the questionnaire will be used, questionnaire users, data analysts, or those who make decisions based on the outcomes of the survey [47]. Expert judges try to eliminate poorly worded questions and revise items’ phrasing so they can be maximally understood [51].
The content validity process is composed of three stages, as displayed in Figure 1. The methodology used in this study will be explained as follows.

3.1. Item Identification

A construct definition demands a careful examination of the literature [52]. In order to identify candidate measurement items for WF management, a systematic literature review (SLR) was conducted. SLRs search for, appraise and synthesize research evidence [53]. A SLR should be based on a clear search protocol. In this study, the review protocol used followed the procedures recommended by Tranfield et al. [54], which include the definition of the search string, the search period, the definition of the databases to retrieve studies from, and the criteria to identify relevant studies. In terms of material collection, this study defines a keyword-based search protocol, but also other approaches, as recommended in previous studies [55]. A literature search was conducted using the terms [(“water footprint management”) OR (“WF management”)] AND (scale OR initiatives OR measurement OR indicators OR PLS-SEM) in Web of Science and Science Direct, which are databases used in previous studies [56]. The partial least squares structural equation method (PLS-SEM) has been included as a search parameter since it is a method that attempts to establish and assess cause-effect analysis among constructs, which are usually defined with a set of measurement items [57].
The search was executed, returning a total of 43 articles. Studies were selected based on inclusion and exclusion criteria. The inclusion criteria applied were: (1) peer-reviewed articles and book chapters written in English; and (2) studies that present WF measurement items or initiatives that reflect the adoption of WF management practices; (3) studies that present practices that can be applied with a supply chain focus; (4) studies with practices that can be applied to the agriculture sector. The exclusion criteria applied were: (1) reports and dissertations; and (2) studies that do not present initiatives or practices related to WF management. The application of these criteria was first based on an analysis of the study’s title and abstract, then the introduction and conclusion sections and, finally, the complete text, if necessary. This step led to a final sample of 21 relevant review papers.
In order to expand the search, articles referenced by the ones initially identified were scanned (snowball search). This is in accordance with previous studies that stated that informal approaches and snowball methods are also important strategies to complement keyword-based searches [58]. Govindan and Hasanagic recommend performing manual searches in order to avoid limitations and to eliminate search errors [55].
The final set of papers was carefully read with the objective of identifying potential measurement items for WF management. According to our previously stated definition of this concept, policies and actions at the strategic, tactical, and operational levels that support managing and reducing the WF of a product across its entire supply chain were identified and considered as candidate measurement items.

3.2. Experts Selection

The second phase of this study comprised the selection of expert judges to evaluate these candidate measurement items. There is no unique formula for expert selection. In general, researchers should use their common sense to define criteria for respondent selection [59]. Different criteria exist for selecting experts, such as the respondents’ hierarchical position and level of expertise, qualification and exposure to the problem being investigated [49], academic publications, professional experience in the field, membership of relevant institutions [59], personal involvement [60], respondents’ level of interest in participating in the study, and easiness of access to the experts [61]. Given the shortcomings implied in the various expert-identification procedures, researchers recommend combining different criteria [60]. Moreover, the panel of experts should be heterogeneous [59] to mitigate cognitive biases [61].
Experts were invited to participate based on the following criteria: (1) knowledgeable professionals in WF and water management or engaged in the agricultural industry; (2) experienced professionals with practical experience of at least 10 years in the field; (3) solid educational background and at least a bachelor’s degree on Agronomy, Agricultural Sciences, or related fields; (4) declared familiarity with the topics and concepts evaluated in this study; and (5) voluntary participation in this study. Similar criteria have been adopted in related studies [62,63,64].
This study was carried out with the support of Chilean experts on WF management and the agriculture industry. Some of the participating experts work in a public-private organization whose purpose is to promote the transformation of Chile towards sustainable development assuring a water transition that guarantees water security and water availability in the country. The other participating experts include a PhD researcher in Environmental Sciences interested in water resources management in the agriculture industry, a senior specialist in hydrometry, founder of a company dedicated to hydrometry and environmental monitoring, and experienced food and agricultural engineers.
In Chile, water scarcity problems have been observed for some years now [65]. Agricultural activities have intensified [66] to a level that the sector accounts for most of the consumptive extractions of water in the country [67]. There are foundations and organizations in the country concerned with water protection and management, so Chile counts with several experts in the field. The number of 8 participating experts was considered adequate for content validation, given that the number recommended to establish content validity ranges from a minimum of 3 to a maximum of 10 [47,50,59,68,69,70].

3.3. Items’ Assessment

The questionnaire sent to experts was introduced by a cover letter in order to explain why experts were invited to participate, along with clear and concise instructions on how to rate each item. A covering letter is an important part when sending a questionnaire for review [47]. Before the presentation of the candidate measurement items, the definition of the WF management construct was presented to experts, who were asked to evaluate the survey items based on this definition.
Measurement items were evaluated according to three perspectives: relevance, clarity and essentiality. To do so, experts were given an appraisal sheet with the following inquiries: (1) the relevance of each item (how important the item is); (2) the clarity of each item (how clear their wording is); (3) the essentiality of each item (how necessary the item is); and (4) recommendations for improvement of each item, as recommended by Rodrigues et al. [71]. For the relevance scale, a 4-point Likert scale was used, and possible responses were: 1 = not relevant, 2 = somewhat relevant, 3 = quite relevant, and 4 = very relevant. A 3-point Likert scale was used for the clarity and essentiality scales. The clarity scale was composed of the following categories: 1 = not clear, 2 = item needs some revision; and 3 = very clear, and for essentiality, the following choices were possible: 1 = not essential; 2 = useful, but not essential; and 3 = essential. Experts could provide additional comments and recommendations for each item in the Comments column of the sheet, as recommended by Rodrigues et al. [71]. A final open remarks field was provided so experts could give their feedback on anything not covered by the questionnaire format, for example, an underrepresentation of the construct by the items [47].
As there is no statistical test to specifically assess content validity, researchers usually adopt a qualitative approach, through the assessment of expert judges, and then, a quantitative approach using the content validity index (CVI). The CVI measures the proportion or percentage of judges who agree on certain aspects of a tool and its items. Whenever the CVI refers to an item, it is called I-CVI. Ratings of 1 and 2 are considered content invalid while ratings of 3 and 4 are considered content valid, as suggested by Souza et al. [57]. A CVI of at least 0.80 and higher than 0.90 is accepted [47,57]. This index is simple to calculate, easy to understand, and it provides information about each item which guides their modification or deletion [72].
Despite its frequent use to assess content validity, CVI does not consider the possibility of inflated values due to chance agreement. The Kappa coefficient provides information about the degree of agreement beyond chance, and it is calculated as:
K = I CVI Pc 1 Pc
Pc   = [ N ! / A ! ( N A ) ! ] 0.5 N
Pc is the probability of chance agreement; N is the number of experts; and A is the number of experts who agree about the item’s relevance. Kappa values above 0.74 are considered excellent. Values between 0.60 and 0.74 are considered good while values between 0.40 and 0.59 are judged as fair [71,72,73].
This study has also used another index to calculate content validity—the content validity ration (CVR), which determines how many raters marked an item as essential [71]. The formula used for calculating this index is:
CVR = (Ne − N/2)/N/2
where Ne is the number of experts that rated the item as “essential”, and N is the total number of experts. CVR can range from −1 to +1 [73]. A higher score indicates further agreement of experts on the necessity of an item in a questionnaire [72]. The cutoff values for CVR are determined by Lawshe’s Table [74]. According to this table, for a number of 8 experts, a CVR greater than 0.75 is considered acceptable.
The last index used to assess content validity was Aiken’s V coefficient. The use of this coefficient is recommended because it can not only be used to summarize the magnitude of expert ratings, but also to test specific hypotheses concerning the values of the ratings for the population. This coefficient is calculated as:
V = (X − l)/k
where X represents the mean of experts’ ratings, l represents the lowest possible rating, and k the range of possible values of scale adopted. This coefficient provides a measurement for rater endorsement, ranging from 0 to 1. A value of 1 is calculated when all experts choose the highest possible rating [75]. Values greater than 0.75 are in general accepted [76]. Finally, in order to assess content validity for the overall WF management construct, the average scale-CVI (S-CVI) was used, which is computed as the average of all items’ CVI. It is recommended that S-CVI should be higher than 0.8 to reflect content validity [77].
The content validity analysis was carried out online by experts from 27 October 2021 to 18 November 2021. As previously stated, items were evaluated according to their relevance, clarity and essentiality to represent the construct of WF management. Relevance was assessed with a 4-point Likert scale and clarity and essentiality with a 3-point Liker scale, following recommendations of previous studies. For each of these criteria, some content validity indices were calculated: I-CVI, Kappa’s coefficient and Aiken’s V coefficient. The CVR index was used for essentiality only.
Table 1 summarizes the content validity process used in this study.

4. Results

As previously stated, the first step of this research comprised carrying out a systematic literature review to identify potential measurement items. The final set of papers identified by the SLR was carefully read with the objective of identifying candidate measurement items for WF management. The candidate practices identified at this step, along with their referenced studies are shown in Table 2. Practice WF17 was excluded because it was considered similar to practice WF22.
The second phase of this study comprised the evaluation of these candidate measurement items by expert judges. Table 3 shows the profile of the experts who participated in this study.
The expert’s authority coefficient (Cr) was calculated to express how knowledgeable and familiar with the agriculture industry and the concept of WF the selected experts are. This coefficient was determined by three factors: the expert’s academic level (q1), the hierarchical position the expert occupies in his/her current job (q2), and the degree of familiarity with the concept of WF (q3). Each dimension was divided into three degrees, indicating the lowest and highest classification, as recommended by Shi et al. [64]. The expert academic level weight assignment considered that a PhD’s degree was weighted as 0.3, a master’s degree or equivalent was weighted as 0.2, and a bachelor’s degree had a weight of 0.1. For the hierarchical positions, CEO and directors were assigned a value of 0.3, for managers and coordinators, a value of 0.2 while 0.1 was given to professionals in operational positions. The familiarity with water footprint was evaluated as high (0.4), average (0.25), and low (0.15). Therefore, in this study, the expert’s authority coefficient was calculated as the average of q1, q2, and q3, therefore, equal to 0.81, indicating that the experts participating in this study were authoritative, since Cr was higher than 0.7 [63].
Table 4 shows the values calculated for the indices assessed in this study for each candidate measurement item.
In order to select the most essential and relevant items, the scores of each item for the relevance and essentiality criteria were first assessed, based on the experts’ feedback. Seven out of the 22 candidate measurement items presented values that exceeded the cutoff values recommended by previous studies (CVI ≥ 0.80; Kappa ≥ 0.74; Aiken’s V ≥ 0.75; and CVR ≥ 0.75).
We then analyzed these seven items regarding their clarity. There was a consensus among experts (all three content validity indices for clarity exceed the cutoff values) for items WF12, WF13, and WF19. These items were kept exactly as they have been first proposed and assessed by experts. The other items (WF1, WF2, WF4, and WF21) were revised and modified. Their readability was improved based on the suggestions given by reviewers. Whenever an expert considered an item was not clearly stated, he/she could provide some comments or suggestions for improving the item’s readability. These comments were analyzed, and these four items were rewritten to improve their clarity. Table 5 shows the final measurement items for the WF management construct.
Finally, S-CVI (Average) for the WF management construct was calculated as 0.91, considering items’ essentiality and as 0.96 regarding items’ relevance, above the threshold of 0.8, recommended to demonstrate the construct’s content validity [77].

5. Discussions

The WF, usually expressed in water volume per unit of a product [31], is sometimes difficult to calculate due to data unavailability, inaccurate data, and complex measurement techniques. In order to deal with these challenges, this study proposed and evaluated the content validity of a new scale of measurement items for WF management initiatives in AFSCs context. These items will be useful for studies that require some form of measuring WF management when raw data are not available. With the support of a group of experts, measurement items that are under the domain of the WF management construct could be identified, that these items are relevant, clearly written, and essential to describe this concept.
From the final measurement items presented in Table 5, it is possible to observe that experts have prioritized items related to the following subjects: standards, WF management in the whole supply chain, risk identification and mitigation, auditing and control, WF-related technologies, and water reuse and recycle. The WF management construct proposed in this study is of reflexive nature, that is, a company that presents a high performance in WF management is a company that implements the initiatives represented by these items, which will be discussed as follows.
The experts considered that a company with a good WF management is compliant to related standards (WF1). Some standards exist, for instance, the International Standard Organization standard named “Environmental management—WF—principles, requirements, and guidelines”—ISO 14046 [78], which assesses water usage and its related environmental impacts [79]. One expert proposed that it is important not only to comply to global standards, as it has been initially proposed in this study, but to national standards as well, since countries might require compliance to local norms or specific industries.
A second characteristic that was highlighted by the selected measurement items was the requirements or needs to extend the adoption of WF management initiatives to supply chain partners (WF2). Third-party stakeholders are increasingly being held accountable by policy makers, consumers and the society for the environmental and social impacts created along upstream supply chains [80]. Collaboration in AFSCs contexts is, in fact, one of the main research areas on sustainability supply chain management [20], which is necessary to collectively achieve a competitive advantage for better environmental, business and societal outcomes [81]. Specially in the agriculture sector, collaboration with supply chain partners is beneficial to optimize risk and investment management, and to allow access to previously unaffordable resources [82]. Large and multinational companies have a critical role in this context, not only guiding initiatives, but also providing critical resources for partners across the supply chain [83].
WF management through the supply chain is related to risk management as well, since water-related risks do not only occur in the direct operations of a firm, but also in activities related to suppliers’ operations. These risks should also be considered as they can impact the company’s business [84]. Risk identification (WF21) and mitigation (WF4) were also considered important characteristics of WF management by participating experts. Common mitigation strategies include establishing new supply chains, implementing new technologies or formulating new business strategies [85].
Auditing and control (WF12) were also considered relevant aspects of WF management. Aivazidou et al. [30] claimed that establishing systems for auditing and control is useful to identify processes that generate water losses in order to reduce wastewater. In this sense, companies that adopt these quality systems will be surer that their initiatives have been implemented properly and that they comply to internal or external standards, whenever necessary.
Water-efficient technologies (WF13) was considered an important characteristic of WF management in our study, corroborating previous studies that state that disruptive technologies such as the Internet of Things, big data, artificial intelligence (AI), and blockchain support sustainability across supply chains [83]. Specially in the agriculture sector, remote data collection technologies have potential contributions to the area [86]. Technologies of sensors produce tools to achieve the reduction of production costs [87] and the improvement of the WF by saving inputs and water [88]. AI modeling can be used to estimate, forecast, predict stream flow, and examine water quality parameters [89], model the green and blue WFs [90], and improve water efficiency [91]. Technology also plays an important role in water reuse and recycle, which is discussed as follows.
Finally, water reuse and recycle (WF19) were also considered part of WF management. Water reuse occurs when wastewater is used directly in other operations, whenever the existing pollutants do not disturb the process [92]. Wastewater should have the lowest concentration of polluting materials to be reused and exploited in agricultural activities [93]. It is a solution to minimize potable water use, but it requires the use of treatment technology to reduce [94]. In wastewater contexts, technology plays a vital role in improving treatment processes and greywater quality [93]. Recycled water refers to wastewater treatment which can produce the reuse-water for other purposes Water recycling technologies include the processes and equipment involved in the activities of treatment, storage and distribution of used water [92].


Water use and management in the agriculture industry is a hot research topic [95], which corroborates the importance of this study. This study contributes to the research area of WF management by proposing and validating a set of measurement items to assess WF management initiatives implemented in AFSCs. A specific contribution is given to the supply chain research community that is concerned with the impacts and the effects of WF management in the supply chain. WF management is related to actions implemented in the diverse supply chains echelons. There is a need for further developing supply chain-oriented WF studies [30]. Most existing WF management practices are rather generic and need to be tailored to particular supply chain contexts. being scientific research on WF across AFSCs rather limited [44]. This study, through a systematic literature review, identified a set of WF management initiatives that were further validated and prioritized by a group of expert professionals.
This study also contributes to the agriculture industry, which faces different challenges related to sustainability such as the reduction of polluting and greenhouse gases emission [96] and reduction of its high usage levels of water resources in production processes [11]. Previous researchers have stated that it is necessary to measure the use of water resources in the agricultural production process, so sustainable initiatives can be designed and implemented [97]. It is expected that the proposed measurement items be used to highlight the main threats and weaknesses associated with companies’ water use, and they will stimulate companies to implement WF management initiatives. It is also expected that future experimental studies use the proposed scale, which has been a call made by researchers for more studies of this kind in the area [32]. With the scale presented in this study, more companies will be able to incorporate the initiatives represented by the scale’s measurement items into their business strategy. Finally, a WF management scale that considers supply chain partners is also a contribution of this study, due to the importance of involving different stakeholders in this process. In fact, researchers have been calling for investigating the involvement of the different stakeholders in water governance [37]. In this way, supply chain partners might be pressured by focal companies to collaborate [98] and adopt water-friendly farming practices [30], and to disclosure in a transparent way their water appropriation and efficiency [99].

6. Conclusions

This study proposes and assesses the content validity of a new scale to measure WF management practices. specifically in a context of agri-food supply chains. The main outcome of this study is a set of measurement items that can be used to evaluate WF management initiatives implemented in AFSCs. In the first stage of scale development, 22 items were identified using a literature review. Then, 8 experts were asked to rate these items based on their relevance, clarity, and essentiality to measure WF management. The quantification of content validity was assessed using several indices (I-CVI, Kappa’s coefficient, Aiken’ V coefficient, and CVR), which indicated high content validity for the selected items. The final items reflect different water footprint management initiatives that comprise the use of global standards for WF accounting, traceability, and assessment; WF measurement throughout the supply chain; water pollution risk mitigation; water auditing and control; use of water-efficient technologies; and wastewater reuse and recycling.
This pioneering study presents some contributions that are expected to impact and support future research and practice. First, our study brings conceptual clarity into the nature of water footprint management as a construct that reflects a reliable varied set of related initiatives and practices. By including items that assess WF management practices, the proposed measures provide deeper insight into the initiatives that characterize WF management. Second, this study uses robust theoretical and statistical procedures to propose a measurement instrument that allowing the assessment of WF management-related initiatives, with a focus on agri-food supply chains. This scale will prove useful to researchers that aim to study WF management and its relationship to other constructs of interest. As so, these measurement items will support future quantitative research that will assess the antecedents and effects of WF management. For example, researchers can use the proposed items to assess the effects of WF management on companies’ sustainable performance and on business performance indicators or assess the effects of customers’ and external stakeholders’ pressures on WF management. Third, the items proposed in this study can be used for organizations’ self-assessment of their WF management initiatives.
Although this study has been carried out with care and it has achieved its objective of proposing a new construct for WF management measurement, it is not free of limitations which can be explored by future studies. This study was focused on content validity only. Therefore, future research can assess the questionnaire for reliability and other forms of validity such as face validity. The content validity process used in this study corresponds to the first steps of the scale development process, as suggested by different researchers [21,22,23]. As so, it is expected that further research uses the construct proposed in this study in large data collection processes. This way, the proposed construct will be tested in different environments, which could lead to further eventual refinements. So, it is expected that the questionnaire items be validated in future empirical quantitative studies in the context of AFSCs.
The search string used in this study was defined according to its objective, which is focused on WF management initiatives and practices implemented by agri-food companies and supply chains. Widening the scope search could increase the number of retrieved studies and the identification of new practices, however, it would probably have only marginal benefits, since the most closely related studies to WF management have probably been already identified. Anyway, despite the presented limitations, the WF management construct and its measurement items proposed in this study can be used internationally by future research.
Finally, future studies could also use the proposed items and assess whether participants’ responses on WF management are similar over time, which is relevant to establish if WF management initiatives are stable or susceptible to change due to external changes and pressures. To do so, future longitudinal assessments could be carried out.

Author Contributions

Conceptualization, M.W.B.; methodology, M.W.B.; validation, M.W.B. and J.M.C.; formal analysis, M.W.B.; investigation, M.W.B.; resources, M.W.B. and J.M.C.; data curation, M.W.B.; writing—original draft preparation, M.W.B.; writing—review and editing, M.W.B. and J.M.C.; supervision, M.W.B. and J.M.C.; project administration, M.W.B. and J.M.C.; funding acquisition, M.W.B. and J.M.C. All authors have read and agreed to the published version of the manuscript.


This research was funded by ANID FONDECYT POSTDOCTORADO N° 3210113.

Institutional Review Board Statement

The study was conducted according to the guidelines of the Declaration of Helsinki and approved by the Ethics Committee of Universidad Autónoma de Chile (protocol code 16-21 and date of approval 21 April 2021).

Informed Consent Statement

Informed consent was obtained from all subjects involved in the study.

Data Availability Statement

Data available on request due to restrictions eg privacy or ethical.


To all experts who voluntarily participated in this study. To the four anonymous reviewers for their feedback. The authors acknowledge the support received from “Cátedra de Economía de la Energía y del Medio Ambiente, Red Eléctrica de España-Universidad de Sevilla”. The authors acknowledge the support provided by the Universidad Autónoma de Chile (Chile). The standard disclaimer applies.

Conflicts of Interest

The authors declare no conflict of interest. The funders had no role in the design of the study; in the collection, analyses, or interpretation of data; in the writing of the manuscript, or in the decision to publish the results.


  1. Lohrmann, A.; Child, M.; Breyer, C. Assessment of the water footprint for the European power sector during the transition towards a 100% renewable energy system. Energy 2021, 233, 121098. [Google Scholar] [CrossRef]
  2. Hoekstra, A.Y.; Hung, P.K. Virtual Water Trade: A Quantification of Virtual Water Flows between Nations in Relation to International Crop Trade; UNESCO-IHE: Delft, The Netherlands, 2002; Volume 11. [Google Scholar]
  3. Hoekstra, A.Y.; Chapagain, A.K.; Aldaya, M.M.; Mekonnen, M.M. The Water Footprint Assessment Manual: Setting the Global Standard; TJ International Ltd.: Cornwall, UK, 2011; ISBN 9781849712798. [Google Scholar]
  4. Sun, S.K.; Song, J.F.; Wang, F.F.; Wu, P.T.; Wang, Y.B. Evaluating the impacts of socio-economic factors on regional grain virtual water flows in China using a structural equation modeling approach. J. Hydrol. 2019, 571, 132–141. [Google Scholar] [CrossRef]
  5. Ambrosio, E.D.; Gentile, F.; De Girolamo, A.M. Assessing the sustainability in water use at the basin scale through water footprint indicators. J. Clean. Prod. 2020, 244, 118847. [Google Scholar] [CrossRef]
  6. Xu, Z.; Chen, X.; Ruqun, S.; Gong, M.; Du, Y.; Wang, J.; Li, Y.; Liu, J. Spatial-temporal assessment of water footprint, water scarcity and crop water productivity in a major crop production region. J. Clean. Prod. 2019, 224, 375–383. [Google Scholar] [CrossRef]
  7. Jia, F.; Hubbard, M.; Zhang, T.; Chen, L. Water stewardship in agricultural supply chains. J. Clean. Prod. 2019, 235, 1170–1188. [Google Scholar] [CrossRef]
  8. Hoekstra, A.Y. Water footprint assessment: Evolvement of a new research field. Water Resour. Manag. 2017, 31, 3061–3081. [Google Scholar] [CrossRef] [Green Version]
  9. Palhares, J.C.P.; Pezzopane, J.R.M. Water footprint accounting and scarcity indicators of conventional and organic dairy production systems. J. Clean. Prod. 2015, 93, 299–307. [Google Scholar] [CrossRef]
  10. Ansorge, L.; Dlabal, J.; Dostálová, A. How truthful are water accounting data? J. Urban Environ. Eng. 2016, 10, 25–34. [Google Scholar] [CrossRef]
  11. Vanham, D.; Bidoglio, G. A review on the indicator water footprint for the EU28. Ecol. Indic. 2013, 26, 61–75. [Google Scholar] [CrossRef]
  12. Morillo, J.G.; Díaz, J.A.R.; Camacho, E.; Montesinos, P. Linking water footprint accounting with irrigation management in high value crops. J. Clean. Prod. 2015, 87, 594–602. [Google Scholar] [CrossRef]
  13. Feng, B.; Zhuo, L.; Xie, D.; Mao, Y.; Gao, J.; Xie, P.; Wu, P. A quantitative review of water footprint accounting and simulation for crop production based on publications during 2002–2018. Ecol. Indic. 2021, 120, 106962. [Google Scholar] [CrossRef]
  14. Vanham, D.; Bidoglio, G. The water footprint of agricultural products in European river basins. Environ. Res. Lett. 2014, 9, 064007. [Google Scholar] [CrossRef]
  15. Madani, K.; Khatami, S. Water for energy: Inconsistent assessment standards and inability to judge properly. Curr. Sustain. Energy Rep. 2015, 2, 10–16. [Google Scholar] [CrossRef] [Green Version]
  16. Pfister, S.; Ridoutt, B.G. Water footprint: Pitfalls on common ground. Environ. Sci. Technol. 2014, 48, 4. [Google Scholar] [CrossRef]
  17. Hoekstra, A.Y. A critique on the water-scarcity weighted water footprint in LCA. Ecol. Indic. 2016, 66, 564–573. [Google Scholar] [CrossRef] [Green Version]
  18. Gerbens-Leenes, W.; Berger, M.; Allan, J.A. Water footprint and life cycle assessment: The complementary strengths of analyzing global freshwater appropriation and resulting local impacts. Water 2021, 13, 803. [Google Scholar] [CrossRef]
  19. Lovarelli, D.; Bacenetti, J.; Fiala, M. Water footprint of crop productions: A review. Sci. Total Environ. 2016, 548–549, 236–251. [Google Scholar] [CrossRef]
  20. Kamble, S.S.; Gunasekaran, A.; Gawankar, S.A. Achieving sustainable performance in a data-driven agriculture supply chain: A review for research and applications. Int. J. Prod. Econ. 2020, 219, 179–194. [Google Scholar] [CrossRef]
  21. Hinkin, T.R. A review of scale development practices in the study of organizations. J. Manag. 1995, 21, 967–988. [Google Scholar] [CrossRef]
  22. Churchill, G.A. A paradigm for developing better measures of marketing constructs. J. Mark. Res. 1979, 16, 64–73. [Google Scholar] [CrossRef]
  23. DeVellis, R.F.; Thorpe, C.T. Scale Development: Theory and Applications, 5th ed.; Sage Publications: London, UK, 2021. [Google Scholar]
  24. Hardesty, D.M.; Bearden, W.O. The use of expert judges in scale development. Implications for improving face validity of measures of unobservable constructs. J. Bus. Res. 2004, 57, 98–107. [Google Scholar] [CrossRef]
  25. Rossiter, J.R. The C-OAR-SE procedure for scale development in marketing. Int. J. Res. Mark. 2002, 19, 305–335. [Google Scholar] [CrossRef] [Green Version]
  26. Hart, S.L.; Dowell, G. A natural-resource-based view of the firm: Fifteen years after. J. Manag. 2011, 37, 1464–1479. [Google Scholar] [CrossRef]
  27. Caldera, H.T.S.; Desha, C.; Dawes, L. Exploring the characteristics of sustainable business practice in small and medium-sized enterprises: Experiences from the Australian manufacturing industry. J. Clean. Prod. 2018, 177, 338–349. [Google Scholar] [CrossRef]
  28. Latan, H.; Jabbour, C.J.C.; Jabbour, A.B.L.S.; Wamba, S.F.; Shahbaz, M. Effects of environmental strategy, environmental uncertainty and top management’s commitment on corporate environmental performance: The role of environmental management accounting. J. Clean. Prod. 2018, 180, 297–306. [Google Scholar] [CrossRef]
  29. Zhu, Y.; Jiang, S.; Han, X.; Gao, X.; He, G.; Zhao, Y.; Li, H. A bibliometrics review of water footprint research in China: 2003–2018. Sustainability 2019, 11, 5082. [Google Scholar] [CrossRef] [Green Version]
  30. Aivazidou, E.; Tsolakis, N.; Iakovou, E.; Vlachos, D. The emerging role of water footprint in supply chain management: A critical literature synthesis and a hierarchical decision-making framework. J. Clean. Prod. 2016, 137, 1018–1037. [Google Scholar] [CrossRef] [Green Version]
  31. Mekonnen, M.M.; Hoekstra, A.Y. The green, blue and grey water footprint of crops and derived crop products. Hydrol. Earth Syst. Sci. 2011, 15, 1577–1600. [Google Scholar] [CrossRef] [Green Version]
  32. Borsato, E.; Galindo, A.; Tarolli, P.; Sartori, L.; Marinello, F. Evaluation of the grey water footprint comparing the indirect effects of different agricultural practices. Sustainability 2018, 10, 3992. [Google Scholar] [CrossRef] [Green Version]
  33. Hogeboom, R.J. The water footprint concept and water’s grand environmental challenges. One Earth 2020, 2, 218–222. [Google Scholar] [CrossRef]
  34. Zhang, Y.; Huang, K.; Yu, Y.; Yang, B. Mapping of water footprint research: A bibliometric analysis during 2006–2015. J. Clean. Prod. 2017, 149, 70–79. [Google Scholar] [CrossRef]
  35. Hoekstra, A.Y.; Chapagain, A.K.; Oel, P.R. Van progress in water footprint assessment: Towards collective action in water governance. Water 2019, 11, 1070. [Google Scholar] [CrossRef] [Green Version]
  36. Ding, G.K.C.; Ghosh, S. Sustainable water management—A strategy for maintaining future water resources. In Reference Module in Earth Systems and Environmental Sciences; Elsevier: Amsterdam, The Netherlands, 2017; pp. 91–103. ISBN 9780124095489. [Google Scholar]
  37. Gómez-Llanos, E.; Durán-Barroso, P.; Robina-Ramírez, R. Analysis of consumer awareness of sustainable water consumption by the water footprint concept. Sci. Total Environ. 2020, 721, 137743. [Google Scholar] [CrossRef] [PubMed]
  38. Ladeira, M.B.; de Oliveira, M.P.V.; de Sousa, P.R.; Barbosa, M.W. Firm’s supply chain agility enabling resilience and performance in turmoil times. Int. J. Agil. Syst. Manag. 2021, 14, 224–253. [Google Scholar] [CrossRef]
  39. Aivazidou, E.; Tsolakis, N.; Vlachos, D.; Iakovou, E. A water footprint management framework for supply chains under green market behaviour. J. Clean. Prod. 2018, 197, 592–606. [Google Scholar] [CrossRef]
  40. Tsolakis, N.K.; Keramydas, C.A.; Toka, A.K.; Aidonis, D.A.; Iakovou, E.T. Agrifood supply chain management: A comprehensive hierarchical decision-making framework and a critical taxonomy. Biosyst. Eng. 2014, 120, 47–64. [Google Scholar] [CrossRef]
  41. Tsolakis, N.; Srai, J.S.; Aivazidou, E. Blue water footprint management in a UK poultry supply chain under environmental regulatory constraints. Sustainability 2018, 10, 625. [Google Scholar] [CrossRef] [Green Version]
  42. Lv, T.; Wang, L.; Xie, H.; Zhang, X.; Zhang, Y. Evolutionary overview of water resource management (1990–2019) based on a bibliometric analysis in Web of Science. Ecol. Inform. 2021, 61, 101218. [Google Scholar] [CrossRef]
  43. Ahmed, S.S.; Bali, R.; Khan, H.; Mohamed, H.I.; Sharma, S.K. Improved water resource management framework for water sustainability and security. Environ. Res. 2021, 201, 111527. [Google Scholar] [CrossRef]
  44. Aivazidou, E.; Tsolakis, N.; Vlachos, D.; Iakovou, E. Water footprint management policies for agrifood supply chains: A critical taxonomy and a system dynamics modelling approach. Chem. Eng. Trans. 2015, 43, 115–120. [Google Scholar] [CrossRef]
  45. Christ, K.L. Water management accounting and the wine supply chain: Empirical evidence from Australia. Br. Account. Rev. 2014, 46, 379–396. [Google Scholar] [CrossRef]
  46. Chico, D.; Aldaya, M.; Garrido, A. A water footprint assessment of a pair of jeans: The influence of agricultural policies on the sustainability of consumer products. J. Clean. Prod. 2013, 57, 238–248. [Google Scholar] [CrossRef] [Green Version]
  47. Elangovan, N.; Sundaravel, E. Method of preparing a document for survey instrument validation by experts. MethodsX 2021, 8, 101326. [Google Scholar] [CrossRef] [PubMed]
  48. Yamada, J.; Stevens, B.; Sidani, S.; Watt-Watson, J.; De Silva, N. Content validity of a process evaluation checklist to measure intervention implementation fidelity of the EPIC intervention. Worldviews Evid.-Based Nurs. 2010, 7, 158–164. [Google Scholar] [CrossRef] [PubMed]
  49. Hasson, F.; Keeney, S. Enhancing rigour in the Delphi technique research. Technol. Forecast. Soc. Chang. 2011, 78, 1695–1704. [Google Scholar] [CrossRef]
  50. Papadas, K.K.; Avlonitis, G.J.; Carrigan, M. Green marketing orientation: Conceptualization, scale development and validation. J. Bus. Res. 2017, 80, 236–246. [Google Scholar] [CrossRef]
  51. Boateng, G.O.; Neilands, T.B.; Frongillo, E.A.; Melgar-Quiñonez, H.R.; Young, S.L. Best practices for developing and validating scales for health, social, and behavioral research: A primer. Front. Public Health 2018, 6, 149. [Google Scholar] [CrossRef]
  52. Netemeyer, R.G.; Bearden, W.O.; Sharma, S. Scaling Procedures: Issues and Applications; Sage Publications: Thousand Oaks, CA, USA, 2003. [Google Scholar]
  53. Mhatre, P.; Panchal, R.; Singh, A.; Bibyan, S. A systematic literature review on the circular economy initiatives in the European Union. Sustain. Prod. Consum. 2020, 26, 187–202. [Google Scholar] [CrossRef]
  54. Tranfield, D.; Denyer, D.; Smart, P. Towards a methodology for developing evidence-informed management knowledge by means of systematic review. Br. J. Manag. 2003, 14, 207–222. [Google Scholar] [CrossRef]
  55. Govindan, K.; Hasanagic, M. A systematic review on drivers, barriers, and practices towards circular economy: A supply chain perspective. Int. J. Prod. Res. 2018, 56, 278–311. [Google Scholar] [CrossRef]
  56. Barbosa, M.W. A critical appraisal of review studies in circular economy: A tertiary study. Circ. Econ. Sustain. 2021. [Google Scholar] [CrossRef]
  57. de Souza, A.C.; Alexandre, N.M.C.; de Guirardello, E.B. Psychometric properties in instruments evaluation of reliability and validity. Epidemiol. Serv. Saude Rev. Sist. Unico Saude Bras. 2017, 26, 649–659. [Google Scholar] [CrossRef]
  58. Martins, C.L.; Pato, M.V. Supply chain sustainability: A tertiary literature review. J. Clean. Prod. 2019, 225, 995–1016. [Google Scholar] [CrossRef]
  59. Belton, I.; MacDonald, A.; Wright, G.; Hamlin, I. Improving the practical application of the Delphi method in group-based judgment: A six-step prescription for a well-founded and defensible process. Technol. Forecast. Soc. Chang. 2019, 147, 72–82. [Google Scholar] [CrossRef]
  60. Mauksch, S.; von der Gracht, H.A.; Gordon, T.J. Who is an expert for foresight? A review of identification methods. Technol. Forecast. Soc. Chang. 2020, 154, 119982. [Google Scholar] [CrossRef]
  61. Beiderbeck, D.; Frevel, N.; von der Gracht, H.A.; Schmidt, S.L.; Schweitzer, V.M. Preparing, conducting, and analyzing Delphi surveys: Cross-disciplinary practices, new directions, and advancements. MethodsX 2021, 8, 101401. [Google Scholar] [CrossRef] [PubMed]
  62. He, H.; Zhou, T.; Zeng, D.; Ma, Y. Development of the competency assessment scale for clinical nursing teachers: Results of a Delphi study and validation. Nurse Educ. Today 2021, 101, 104876. [Google Scholar] [CrossRef]
  63. Wu, C.; Wu, P.; Li, P.; Cheng, F.; Du, Y.; He, S.; Lang, H. Construction of an index system of core competence assessment for infectious disease specialist nurse in China: A Delphi study. BMC Infect. Dis. 2021, 21, 791. [Google Scholar] [CrossRef]
  64. Shi, C.; Zhang, Y.; Li, C.; Li, P.; Zhu, H. Using the Delphi method to identify risk factors contributing to adverse events in residential aged care facilities. Risk Manag. Healthc. Policy 2020, 13, 523–537. [Google Scholar] [CrossRef]
  65. Novoa, V.; Ahumada-Rudolph, R.; Rojas, O.; Munizaga, J.; Sáez, K.; Arumí, J.L. Sustainability assessment of the agricultural water footprint in the Cachapoal River basin, Chile. Ecol. Indic. 2019, 98, 19–28. [Google Scholar] [CrossRef]
  66. Yevenes, M.A.; Arumí, J.L.; Farías, L. Unravel biophysical factors on river water quality response in Chilean Central-Southern watersheds. Environ. Monit. Assess. 2016, 188, 264. [Google Scholar] [CrossRef]
  67. Novoa, V.; Ahumada-Rudolph, R.; Rojas, O.; Sáez, K.; de la Barrera, F.; Arumí, J.L. Understanding agricultural water footprint variability to improve water management in Chile. Sci. Total Environ. 2019, 670, 188–199. [Google Scholar] [CrossRef] [PubMed]
  68. Li, X.; Su, X.; Du, Y. The environmental sustainability of an exhibition in visitors’ eyes: Scale development and validation. J. Hosp. Tour. Manag. 2021, 46, 172–182. [Google Scholar] [CrossRef]
  69. Piha, L.; Papadas, K.; Davvetas, V. Brand orientation: Conceptual extension, scale development and validation. J. Bus. Res. 2021, 134, 203–222. [Google Scholar] [CrossRef]
  70. Lynn, M.R. Determination and quantification of content validity. Nurs. Res. 1986, 35, 382–386. [Google Scholar] [CrossRef]
  71. Rodrigues, I.B.; Adachi, J.D.; Beattie, K.A.; MacDermid, J.C. Development and validation of a new tool to measure the facilitators, barriers and preferences to exercise in people with osteoporosis. BMC Musculoskelet. Disord. 2017, 18, 540. [Google Scholar] [CrossRef] [Green Version]
  72. Zamanzadeh, V.; Ghahramanian, A.; Rassouli, M.; Abbaszadeh, A.; Alavi-Majd, H.; Nikanfar, A.-R. Design and implementation content validity study: Development of an instrument for measuring patient-centered communication. J. Caring Sci. 2015, 4, 165–178. [Google Scholar] [CrossRef]
  73. Jansen, M.; Doornebosch, A.J.; de Waal, M.W.M.; Wattel, E.M.; Visser, D.; Spek, B.; Smit, E.B. Psychometrics of the observational scales of the Utrecht Scale for Evaluation of Rehabilitation (USER): Content and structural validity, internal consistency and reliability. Arch. Gerontol. Geriatr. 2021, 97, 104509. [Google Scholar] [CrossRef]
  74. Lawshe, C.H. A quantative approach to content validity. Persistent Psychol. 1975, 28, 563–575. [Google Scholar] [CrossRef]
  75. Penfield, R.D.; Giacobbi, P.R. Measurement in physical education and exercise science applying a score confidence interval to Aiken’s item content-relevance index. Meas. Phys. Educ. Exerc. Sci. 2004, 8, 213–225. [Google Scholar] [CrossRef]
  76. Acosta-Banda, A.; Aguilar-Esteva, V.; Ortiz, M.P.; Ortiz, J.P. Construction and Validity of an Instrument to Evaluate Renewable Energies and Energy Sustainability Perceptions for Social Consciousness. Sustainability 2021, 13, 2333. [Google Scholar] [CrossRef]
  77. Shrotryia, V.K.; Dhanda, U. Content validity of assessment instrument for employee engagement. SAGE Open 2019, 9, 1–7. [Google Scholar] [CrossRef] [Green Version]
  78. ISO 14046:2014; Environmental Management—Water Footprint—Principles, Requirements and Guidelines. ISO: Geneva, Switzerland, 2014.
  79. Bai, X.; Ren, X.; Zheng, N.; Zhou, N.; Hu, M. Comprehensive water footprint assessment of the dairy industry chain based on ISO 14046: A case study in China. Resour. Conserv. Recycl. 2018, 132, 369–375. [Google Scholar] [CrossRef]
  80. Gilsbach, L.; Schütte, P.; Franken, G. Applying water risk assessment methods in mining: Current challenges and opportunities. Water Resour. Ind. 2019, 22, 100118. [Google Scholar] [CrossRef]
  81. Dania, W.A.P.; Xing, K.; Amer, Y. Collaboration behavioural factors for sustainable agri-food supply chains: A systematic review. J. Clean. Prod. 2018, 186, 851–864. [Google Scholar] [CrossRef]
  82. Asian, S.; Hafezalkotob, A.; John, J.J. Sharing economy in organic food supply chains: A pathway to sustainable development. Int. J. Prod. Econ. 2019, 218, 322–338. [Google Scholar] [CrossRef]
  83. Adams, D.; Donovan, J.; Topple, C. Achieving sustainability in food manufacturing operations and their supply chains: Key insights from a systematic literature review. Sustain. Prod. Consum. 2021, 28, 1491–1499. [Google Scholar] [CrossRef]
  84. Chapagain, A.K.; Network, W.F.; Hague, T. Water Footprint: State of the Art: What, Why, and How? Elsevier: Amsterdam, The Netherlands, 2017; Volume 4, ISBN 9780124095489. [Google Scholar]
  85. Schornagel, J.; Niele, F.; Worrell, E.; Böggemann, M. Water accounting for (agro) industrial operations and its application to energy pathways. Resour. Conserv. Recycl. 2012, 61, 1–15. [Google Scholar] [CrossRef] [Green Version]
  86. Sanders, K.T.; Masri, S.F. The energy-water agriculture nexus: The past, present and future of holistic resource management via remote sensing technologies. J. Clean. Prod. 2016, 117, 73–88. [Google Scholar] [CrossRef]
  87. Pantazi, X.E.; Moshou, D.; Bochtis, D. Sensors in agriculture. Intell. Data Min. Fusion Syst. Agric. 2020, 1, 1–15. [Google Scholar] [CrossRef]
  88. Cazcarro, I.; Hoekstra, A.Y.; Sánchez Chóliz, J. The water footprint of tourism in Spain. Tour. Manag. 2014, 40, 90–101. [Google Scholar] [CrossRef]
  89. Nishant, R.; Kennedy, M.; Corbett, J. Artificial intelligence for sustainability: Challenges, opportunities, and a research agenda. Int. J. Inf. Manag. 2020, 53, 102104. [Google Scholar] [CrossRef]
  90. Elbeltagi, A.; Deng, J.; Wang, K.; Hong, Y. Crop water footprint estimation and modeling using an artificial neural network approach in the Nile Delta, Egypt. Agric. Water Manag. 2020, 235, 106080. [Google Scholar] [CrossRef]
  91. Xiang, X.; Li, Q.; Khan, S.; Khalaf, O.I. Urban water resource management for sustainable environment planning using artificial intelligence techniques. Environ. Impact Assess. Rev. 2021, 86, 106515. [Google Scholar] [CrossRef]
  92. Klemeš, J.J.; Varbanov, P.S.; Lam, H.L. Water Footprint, Water Recycling and Food-Industry Supply Chains; Woodhead Publishing Limited: Sawston, UK, 2009. [Google Scholar]
  93. Khan, S.A.R.; Ponce, P.; Yu, Z.; Golpira, H.; Mathew, M. Environmental technology and wastewater treatment: Strategies to achieve environmental sustainability eris. Chemosphere 2022, 286, 131532. [Google Scholar] [CrossRef] [PubMed]
  94. Negreiros, F.; Oliveira, D.; Leiras, A.; Ceryno, P. Environmental risk management in supply chains: A taxonomy, a framework and future research avenues. J. Clean. Prod. 2019, 232, 1257–1271. [Google Scholar] [CrossRef]
  95. Barbosa, M.W. Uncovering research streams on Agri-Food Supply Chain Management: A bibliometric study. Glob. Food Sec. 2021, 28, 100517. [Google Scholar] [CrossRef]
  96. Román, R.; Cansino, J.M.; Rueda-Cantuche, J.M. A multi-regional input-output analysis of ozone precursor emissions embodied in Spanish international trade. J. Clean. Prod. 2016, 137, 1382–1392. [Google Scholar] [CrossRef]
  97. Wang, W.; Wang, J.; Cao, X. Water use efficiency and sensitivity assessment for agricultural production system from the water footprint perspective. Sustainability 2020, 12, 9665. [Google Scholar] [CrossRef]
  98. Barbosa, M.W.; Ladeira, M.B.; de Oliveira, M.P.V.; de Oliveira, V.M.; de Sousa, P.R. The effects of internationalization orientation in the sustainable performance of the agri-food industry through environmental collaboration: An emerging economy perspective. Sustain. Prod. Consum. 2022, 31, 407–418. [Google Scholar] [CrossRef]
  99. Aivazidou, E.; Tsolakis, N. Water Footprint Management in the Fashion Supply Chain: A Review of Emerging Trends and Research Challenges; Elsevier: Amsterdam, The Netherlands, 2019; ISBN 9780081026335. [Google Scholar]
Figure 1. Content validity methodology.
Figure 1. Content validity methodology.
Sustainability 14 04928 g001
Table 1. Content validity process.
Table 1. Content validity process.
Stages of the Content Validity ProcessDetails
Item identification
  • Systematic literature review (keyword-based search in scientific databases) and snowball search
  • Application of inclusion and exclusion criteria
  • Identification of candidate WF measurement items
Expert selection
  • Definition of criteria for expert selection
  • Selection and invitation of experts
  • Calculation of expert’s authority coefficient
Items’ assessment
  • Assessment of candidate measurement items in terms of relevance, clarity, and essentiality by experts
  • Calculation of several content validity assessment indices
Table 2. Candidate WF measurement items.
Table 2. Candidate WF measurement items.
IDCandidate Measurement ItemReference
WF1Our company promotes and adopts global standards for water footprint accounting and assessment [3]
WF2Our company promotes water accounting over the whole supply chain, cooperating with others along the supply chain to be able to produce full accounts for final products
WF3Our company reports water-related efforts, targets and progress made in annual sustainability report, also covering the supply chain.
WF4Our company reduces the risk of pollution, avoiding or minimizing the use of substances in products that may be harmful when reaching the water
WF5Our company compensates downstream users that are affected by intensive upstream water use in the catchment where the company’s (residual) water footprint is located.
WF6Our company establishes agreements on reduction targets for water use with suppliers.
WF7Our company shifts to other supplier when they do not agree to water footprint standards.
WF8Our company would be willing to change its business model in order to incorporate or get better control over the supply chain in terms of water footprint reduction.
WF9Our company prioritizes the cultivation of crops requiring less water[30]
WF10Our company has converted conventional crops into organic crops
WF11Our company selects and collaborates with water-friendly partners
WF12Our company establishes water auditing and control systems
WF13Our company invests in water-efficient technologies
WF14Our company carries out campaigns for raising consumer awareness about water consumption reduction.
WF15Our company develops water stewardship programs[39]
WF16Our company sets targets to manage freshwater resources efficiently
WF17Our company integrates responsible water use into their business strategy (excluded)
WF18Our company adopts techniques and processes to enhance water retention in the soil.[44]
WF19Our company reuses and recycles wastewater
WF20Our company conducts integrated water assurance assessment of the supply chain.[7]
WF21Our company identifies the local water risks
WF22Our company embeds water stewardship into the corporate strategy
WF23Our company engages with various stakeholders for water stewardship
Table 3. Experts’ profile.
Table 3. Experts’ profile.
CategoriesNumber of Experts
  Bachelor’s degree1
  Master’s degree6
  Doctor’s degree1
WF familiarity/knowledge
Table 4. Content validity indices for candidate items.
Table 4. Content validity indices for candidate items.
I-CVIKappaAiken’s VI-CVIKappaAiken’s VI-CVIKappaAiken’s VCVR
Table 5. Final WF measurement items.
Table 5. Final WF measurement items.
IDMeasurement Item Description
WF1Our company promotes and adopts national and global standards for water footprint accounting, traceability, and assessment.
WF2Our company promotes the measurement of the water footprint throughout the supply chain, cooperating with partners to be able to generate records of the volume of water used in manufactured products.
WF4Our company implements processes that mitigate the risk of contamination, avoiding or minimizing the use of substances (metals, pesticides, fertilizers, etc.) in products that may be polluting for water.
WF12Our company establishes water auditing and control systems
WF13Our company invests in water-efficient technologies.
WF19Our company reuses and recycles wastewater.
WF21Our company identifies the local risks of its impact on the water supply.
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Share and Cite

MDPI and ACS Style

Barbosa, M.W.; Cansino, J.M. A Water Footprint Management Construct in Agri-Food Supply Chains: A Content Validity Analysis. Sustainability 2022, 14, 4928.

AMA Style

Barbosa MW, Cansino JM. A Water Footprint Management Construct in Agri-Food Supply Chains: A Content Validity Analysis. Sustainability. 2022; 14(9):4928.

Chicago/Turabian Style

Barbosa, Marcelo Werneck, and José M. Cansino. 2022. "A Water Footprint Management Construct in Agri-Food Supply Chains: A Content Validity Analysis" Sustainability 14, no. 9: 4928.

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop