Rise and Fall of the Materiality Matrix: Lessons from a Missed Takeoff

Abstract

After a long period of the inclusion of materiality matrices within standard setters documents and non-financial reports, the Global Reporting Initiative officially abandoned the materiality matrix in 2021 after the GRI 3 standard release. To bridge the detected gaps in the literature, this article aims to investigate approaches to and arguments for the matrix until the issuance of GRI 3. The two-step research strategy adopted gives the same level of attention to the opposite positions found. Phase 1 (approach-oriented) reviews the materiality matrix presentation in the 2014–2020 non-financial reports of a sample of worldwide sustainability-oriented companies. Phase 2 (argument-oriented) performs qualitative content analysis on feedback for the GRI 3 preparatory works. The findings show that, besides the staunch adopters, a core of non-adopters persisted and prevented the takeoff of the matrix. Moreover, further insights into possible drivers both in favour of and against these approaches are provided. The final discussion both considers the lessons learnt, overlapping with policy implications, and suggests future research avenues.

1. Introduction

Materiality is a key and many-faceted concept (Edgley 2014) that steers the selection of the themes a company reports on. In this sense, materiality is binary (Eccles et al. 2015) and defines what counts to readers when they come across companies’ reports, because it takes into account (or should take into account) what information can serve the stakeholders’ decisional processes. First appearing in financial reporting contexts during the 1900s (Bernstein 1967; Hicks 1964; Holmes 1972), in the 2000s materiality gradually spread in non-financial reporting (NFR) contexts, assuming an increasingly central role regardless of whether a voluntary or mandatory approach was adopted. This led the standard setters involved in NFR to coin different conceptualisations and multi-step materiality assessment approaches, such that the concept still appears to be contested and ambiguous (Cooper and Michelon 2022).

The Global Reporting Initiative (GRI), which is among these standard setters, in the G3 version of the Sustainability Reporting Guidelines (GRI 2006) launched the season of defining a report’s contents through a process devoted to assessing materiality. This materiality assessment process (MAP) includes an issue prioritization phase, the output of which is exemplified by a graph. The graph depicts the relative reporting priority of the topics assessed, which moves, in the GRI’s materiality analysis, along the perspectives of the companies’ external impacts and/or the influence of topics on stakeholders’ decisions. This visual tool, generally known as the materiality matrix (MM), was regularly included in subsequent GRI guidelines and standards. In 2021, GRI radically modified its approach to both prioritization (i.e., two independent criteria are no longer encompassed) and its visualization (i.e., a matrix is no longer needed). Hence, the universal standard “GRI 3: Material Topics” removed MM (GRI 2021). However, any devoted graphical tool took its place in the standard, despite non-financial reports (NFRs) frequently requiring the use of graphs (Varachia and Yasseen 2020).

Although, for this period, MM could be found both in GRI documents and in companies’ NFRs, an in-depth debate between scholars on the tool never arose. Thus, several related research areas remained uncovered, leading to a lack of knowledge on MM’s best and worst practices, as well as a gap in the literature on its pros and cons. Furthermore, the disregarded existence of a group of companies that never presented MM in their NFRs has emerged. As a result, existing studies do not consider the true extent of the MM’s success. In particular, the trends in practice are never interpreted through arguments expressed over time by insider voices. To bridge this gap, this study aims to investigate the landscape of the NFR insiders’ positions and arguments about MM that emerged before the GRI 3 shift. In order to achieve our aim, the following research question is set:

“What approaches to MM adoption emerge over time and why?”

We answered this question through a mixed research strategy articulated in two phases, which differ in nature and, hence, in the methodologies adopted. The first phase, based on a quantitative methodology, reviews the adoption over time of MM in NFRs in a sample of worldwide sustainability-oriented companies for the 2014–2020 period. The second phase, based on a qualitative methodology, performs a content analysis of the public comments on MM included within the responses to the 2020 Exposure Draft, which led to GRI 3.

Among the major findings, two striking results emerged. First, besides staunch MM adopters, a core of non-adopters persisted over time and prevented the takeoff of the tool. Second, interesting further insights on possible drivers underlying these opposite approaches are provided, including those insights in favour of retaining MM that the GRI did not consider in drafting GRI 3.

On the basis of the above, this article aims to contribute to the literature on materiality assessment disclosure in corporate NFRs by focusing on insiders’ viewpoints on MM.

In detail, the study provides the following manifold value. First, practices and voices on MM are analysed by simultaneously paying the same level of attention to both MM adoption and non-adoption, as well as to opinions both in favour of and against MM. Furthermore, the paper extends the time span that has been investigated thus far. Indeed, it diachronically maps MM adoption and non-adoption over a seven-year period, ending at the issuance of GRI 3. Furthermore, the paper provides a preliminary literature review and a content analysis of comment letters, which both offer novel insights on the opinions expressed on the pros and cons of MM, including those from the most active GRI stakeholders.

The article is structured as follows. Section 2 presents the research background, namely the evolution of the specific NFR standard setting context in which MM was developed and fell in popularity. Section 3 outlines the relevant literature, its detected gaps, and the research question stemming from the latter. Section 4 introduces the research design and describes the methodologies used by detailing the materials and methods of the two research phases. Section 5 presents the results of the two research phases. Finally, Section 6 offers a discussion that takes into account the lessons learnt, with policy implications, and suggests future research avenues arising from the limitations of this study.

2. Background: The Materiality Matrix in the Evolution of Standard Setters’ Approaches to Materiality Analysis

2.1. Origins and Spread of the Materiality Matrix

The MM is a special techno-rational tool (Adams et al. 2021), namely, a graph that simultaneously represents stakeholders’ and companies’ perspectives on the relevance of sustainability issues through Cartesian axes. Hence, MM visually reports on how and why some topics have been deemed material enough for reporting and/or strategic aims.

The tool appeared in 2004–2006 sustainability reports as a visual exemplification of the first practices of issue prioritization that pioneering companies experimented with (AccountAbility 2006).

Soon after, AccountAbility and GRI introduced their own MM versions (AccountAbility 2006; GRI 2006), although these versions were grounded in different materiality concepts (Mio and Fasan 2014; Edgley et al. 2015; Cooper and Michelon 2022), different MAPs (Mio 2013; Mio and Fasan 2013; Bellandi 2017; De Cristofaro 2022), and different axes labels (Figure 1).

During the subsequent decade, while AccountAbility dismissed MM at an early stage in subsequent official documents, GRI included this graphic tool in subsequent guidelines (GRI 2011a, 2011b, 2013) and standards (GRI 2016) issued until 2016 (Figure 2). In those years, to exemplify the company’s evaluation of the importance of matters, carried out before prioritisation, even the International Integrated Reporting Council (IIRC) proposed a system of Cartesian axes (IIRC 2013, 2015). However, this cannot be considered a proper MM, because the stakeholders’ perspective is missing.

Nevertheless, although the GRI has trusted MM since 2006, at the end of the 2010s something changed. The following section provides insights into this special case.

2.2. Decline of the Materiality Matrix in the GRI’s Approach

As a part of the 2019 project reviewing the 2016 GRI Universal Standards (GSSB 2019), the Global Sustainability Standards Board (GSSB), i.e., the GRI’s independent standard-setting body, submitted an exposure draft to public comments in 2020 (GSSB 2020). Among other things, the draft included a revised version of “GRI 103: Management Approach”. The feedbacks received were summarized (GSSB 2021a) and considered as a basis for discussion (GSSB 2021b). All of the above led to the new universal standard “GRI 3: Material Topics 2021” (GRI 2021).

These steps heavily involved MM. Indeed, from the 2020 draft onwards, the graph was removed completely. Accordingly, no matrix was mentioned or exposed among any of the document’s text lines. Rather, while no visual exemplifying criteria used to determine material topics was inserted in the text, the MAP exemplification included only a stylized final list of material topics (GRI 2021), as the schematization presented in Figure 3 shows.

However, during the period of GRI 3 drafting, it quickly emerged that not all of the feedback on MM agreed with its removal1. Particularly, GSSB’s summary simply devoted a phrase to each of the two opposite categories of comments on MM (GSSB 2021a). Despite no details being provided (e.g., the weight of the respondents’ opinions, concerns, and suggestions), this was enough to mitigate a drastic removal. Thus, the following new paragraph was inserted into both the revised draft (GSSB 2021b) and the final GRI 3 (GRI 2021): “For transparency, the organization can provide a visual representation of the prioritization that shows the initial list of topics it has identified and the threshold it has set for reporting” (GSSB 2021b, p. 11; GRI 2021, p. 13).

GSSB justified the removal as follows: “The materiality matrix has not been reinstated, as the revised concept of ‘material topic’ no longer encompasses two dimensions. The GSSB will consider developing further guidance on materiality for reporting organizations in a format such as a website FAQ, which could also include suggestions for visuals. This may include options for visual representations of the revised GRI concept of materiality and its connection to double materiality” (GSSB 2021a, p. 7).

GRI further explained its choice. In particular, GRI replied to Frequently Asked Questions (FAQs), reiterating that “The revisions to the concept of ‘material topic’ eliminate the need for a matrix as the concept no longer encompasses two independent criteria” (GRI 2022, p. 17). Namely, material topics no more reflect the “organization’s significant economic, environmental, and social impacts” (MM x-axis) and/or “their substantive influence on the assessments and decisions of stakeholders” (MM y-axis). According to the GRI, feedback indicated that MM often led “to biases based on stakeholder selection, since organizations prioritized impacts only if the consulted stakeholders highlighted them” (GRI 2022, p. 16). Furthermore, it is feared that retaining MM would generate incorrect interpretations based on a two-dimensions impact assessment since “organizations would consider the impacts on themselves instead of how they impact the economy, the environment, and society” (GRI 2022, p. 16).

Hence, the GRI paid more attention to feedback highlighting that material topics now “represent an organization’s most significant impacts on the economy, environment, and people, including impacts on their human rights” (GRI 2022, p. 16). The influence on stakeholders’ assessments decisions is no longer a standalone factor because stakeholder engagement now “forms part of identifying and assessing an organization’s impacts and informs the process for determining material topics” (GRI 2022, p. 16). In other words, since the incorporation of the “old” materiality stakeholders dimension into the “old” external impacts dimension would involve the fusion of the “old” two MM axes, a matrix would no longer make sense.

Then, in GRI 3, both mandatory and voluntary novel disclosures were introduced. As for mandatory, two elements now have to be compulsorily disclosed, namely the material topic list and the cut-off thresholds set for reporting purposes. As for voluntary disclosures, a discretionary (“the organization can provide”) hypothesis of visual tools (not necessarily graphs) was inserted (GRI 2021, pp. 13, 18).

However, despite further guidance on materiality, including “options for visual representations of the revised GRI concept of materiality” was expected in a Frequently Asked Question (FAQ) format (GSSB 2021b, p. 7), the 2022 FAQs (GRI 2022) do not yet include suggestions for visuals.

In summary, the presented GRI shift will likely weaken the adoption of MM in the next NFRs, opening the field to as yet unidentified visual tools exemplifying the future MAPs. Nevertheless, to better understand and foreshadow the next scenario by grounding in the past, it is worth investigating what is really known about the perceived importance and usefulness of MM, in theory and practice, from 2006 to date.

With a special focus on both the adoption and critique of MM, the following section presents the literature that we reviewed in depth to examine the state of the art of MM and how the detected literature’s gaps led to the formulation of our research question.

3. From the Literature to the Research Question

Generally, MM is addressed by studies on materiality analysis devoted to non-financial information disclosure and/or sustainability strategies. These two areas overlap because several company activities (e.g., NFR, stakeholder engagement, report assurance, risk assessment, performance management, and long-term scenarios simulation) can rest upon a materiality assessment approach that encompasses MM (AccountAbility 2006). Furthermore, some studies that do not strictly focus on materiality address MM (e.g., Garcia-Torres et al. 2017; Campra et al. 2020).

However, among these studies addressing MM, only some articles expressly include it in their research aims (Calabrese et al. 2019; Saenz 2019; Torelli et al. 2019; Ortar 2020; Ferrero-Ferrero et al. 2021; Geldres-Weiss et al. 2021; Costa et al. 2022). Even fewer are the devoted book chapters (Eccles et al. 2015; Kuisma 2017), the chapter sections (Gelmini et al. 2015), and the web contributions (Cohen 2014; McElroy 2011). The remaining studies do not focus much on MM, and only cover it occasionally through mere introductions or the discussion of results.

Hence, knowledge on the MM experiences of companies results in a very fragmented literature that both considers MM from several perspectives (e.g., as discussed focal topic, as investigated or detected aspect, as part of a coined approach, and as a research tool) and overlaps the studies cited here. For example, studies that considered MM as a research tool, used the MM as source of the following:

(i)

Data (Formisano et al. 2017; Garcia-Torres et al. 2017; Kurniawan et al. 2019; Lubinger et al. 2019; Saenz 2019; Reimsbach et al. 2020; Campra et al. 2020);

(ii)

Materials provided to experiment participants (Backof et al. 2020);

(iii)

Data scoring (Gerwanski et al. 2019; Mio et al. 2020; Farooq et al. 2021; Sepúlveda-Alzate et al. 2021);

(iv)

Variable and index operationalization (De Cristofaro and Gulluscio 2019; Slacik and Greiling 2019; Ortar 2020; Ruiz-Lozano et al. 2021; Tibiletti et al. 2021).

As a result, neither a systematic corpus of knowledge nor a specific stream of literature on MM exist. Thus, to better capture the relevant literature, all of the MM citations found were manually “picked” and examined according to the following four aspects of findings and foci: MM adoption, MM features, MM as an approach, and critiques of MM. These aspects are separately presented below.

MM adoption was detected in several contributions through single case studies, both interview-based (Lai et al. 2017) and not (Taubken and Feld 2018; Mathur and Kumar 2019), as well as through the investigation of a larger number of companies. In the latter studies, MM adoption can be found and described both by percentages (

Table 1. Empirical findings on materiality matrix adoption in existent sample-based works *.

Reference	Sample Analysed and Reports Examined				Percentages Found by Years
	Country	Industry	No	Report	2008	2009	2010	2011	2012	2013	2014	2015	2016	2017	2018	2019	2020
Ortar (2016)	W	Banks	5	SR	0.45 ¶
Tibiletti et al. (2021)	Italy	C-I	200	SRR	25.4 †			25.4 †					25.4 †
Eccles et al. (2012)	W	C-I	800	SR				8.88 ¶
Bavagnoli et al. (2014)	EUR	C-I	32	IR						28
Slacik and Greiling (2019)	W	ELU	-	SR						58
Jebe (2017)	W	C-I	96	SR, IR							37.5
De Cristofaro and Gulluscio (2019)	EUR	C-I	108	IR								69.44
Karagiannis et al. (2022)	W	MAR	42	SR									42.86
Lubinger et al. (2019)	W	UNI	33	SR										48.48¶
Torelli et al. (2019)	Italy	C-I	152	NFR										66
Costa et al. (2022)	W	TOU	49	SR											43%

* Key: W, worldwide; EUR, Europe; C-I, cross-industry; ELU, electric utilities; MAR, maritime; UNI, university; TOU, tourism; SRR, social responsibility reports; IR, integrated reports; SR, sustainability reports; NFR, non-financial reports. (†) Percentage of the three years considered. (¶) Author’s calculation.

) and not (Jones 2016; Jones et al. 2016a, 2016b, 2016c; Wee et al. 2016; Morrós 2017; Guix et al. 2018; Farooq et al. 2021; Madasu 2019; Lakshan et al. 2021). Table 1 provides evidence that, although MM increased in importance over the 2008–2020 period, the percentages of adoption never reached striking values (range: 25–69%). Table 1 also highlights that companies in several industries and countries around the world have included a MM in their NFRs regardless of the reporting approach chosen (e.g., sustainability reporting, integrated reporting, etc.).

As for MM features (e.g., Eccles et al. 2015; Ortar 2020) and role (Geldres-Weiss et al. 2021), heterogeneous design as well as commonalities have emerged. For example, Ortar (2020) noted that many MMs are devoid of numerical attributes and the presented issues are not always covered in accordance with specified guidelines. Good practices are even less stressed (e.g., the joint depiction of issues and risks/opportunities in Eccles et al. 2015).

Some authors studied MM theory as an approach. One group integrated MM with an “adequacy matrix” to evaluate sustainability communication (Calabrese et al. 2019), with a “value creation matrix” (Saenz 2019) and into/within a theory model for the materiality analysis of sustainability reports (Slacik and Greiling 2019). Ferrero-Ferrero et al. (2021) found that the MM prioritization approach generates a risk of environmental performance overestimation when compared with an expert knowledge approach. Eccles et al. (2015) upgraded MM into a four-cell “Sustainable Value Matrix” for which the y-axis represents the firm’s perception of the significance of an issue to the chosen stakeholders.

Finally, the critique of MM includes both appreciations and criticisms that alternatively refer to and/or overlap with how preparers implement MM, its own virtues and faults, and the GRI approach. For example, the following strengths were stressed:

• MM typically describes firms’ orientation to sustainability more accurately than the GRI content index (Pizzi et al. 2020);
• MM allows the easy identification of most material issues (Ortar 2020);
• MM both visualizes the overall materiality assessment and helps to consider the baseline SDGs-related responsibilities (Costa et al. 2022);
• MM explains the dynamics of sustainability-oriented value creation and business models (Geldres-Weiss et al. 2021);
• MM creates a multi-stakeholder context that fosters virtuous paths of co-creation of value and sustainable society (Formisano et al. 2017);
• As it is instantly and visually observable, MM is a powerful mechanism of communication (Bavagnoli et al. 2014);
• MM is an effective model for assessing the impact of issues using different sizes for different bulks (Bavagnoli et al. 2014);
• MM provides summaries (Farooq and de Villiers 2019) and preliminary maps (Morrós 2017);
• MM shows the same communication advantages of graphs (Gelmini et al. 2015) outlined by Beattie and Jones (2002), namely, to capture attention, to allow direct data acquisition, to simplify data retrieval, and to enliven data presentation;
• MM is a tool that simplifies the inherent complexity of assessing material sustainability issues, stakeholder engagement, and the societal pursuit of sustainable development (Adams et al. 2021).

On the other hand, the following flaws were highlighted:

• MM design requires a lot of time (Aureli et al. 2020) and energy (Taubken and Feld 2018).
• Along the lines of Beattie and Jones (2002), Gelmini et al. (2015) stress that MM creates risks of graphical infidelity, namely selectivity (e.g., x-axis re-labelled “Potential Impact on Our Business”), measurement distortion (e.g., an item’s size being unrepresentative of its relevance), and presentation enhancement (e.g., an item’s emphatization through colours).
• Among the several x-axis labels chosen (Jones et al. 2016a), companies sometimes substitute the GRI’s label for “success to the organisation” and similar labels. This inward focus involves the risk of omitting impacts with little effect on the organisations’ success (Guix et al. 2019), regardless of how much they relate to a particular business plan or strategy (McElroy 2011).
• The lack of detailed guidance makes the y-axis the more complex axis (Bellantuono et al. 2016), since it merges divergent stakeholders’ voices rather than precisely portraying converging instances and conflicting interests (Puroila and Mäkelä 2019). This compromise, although it could marginalize certain stakeholders, provides an illusory consensus among stakeholders that implies a weak comparability between reports (Reimsbach et al. 2020).
• Sometimes companies report only on issues that score highly on both axes (top-right quadrant), essentially omitting significant issues that are relevant to stakeholders (top-left quadrant) (Guix et al. 2019);
• The heterogeneous design of MM (e.g., axes and quadrants) allows companies to manipulate the data to suit their needs (Guix et al. 2018);
• Sometimes a lack of correlation between MM and the report’s content occurs (Sepúlveda-Alzate et al. 2021).
• Companies do not usually disclose background information on how exactly the matrix was constructed, e.g., how issues are ranked and the threshold set (Puroila and Mäkelä 2019).
• Since most of the reports give no clue as to how the material issues were placed in the matrix, it seems that “that there is a focus on moving dots around a matrix and not on the underlying drivers of sustainability performance” (Cohen 2014).
• MM “effectiveness is somewhat limited since it doesn’t show the priorities of different groups, industrial benchmarks used by peers and investors to compare performance, characteristics such as “innovation” that represent resilience and adaptability to changing times”, industry key sustainability performance indicators as well as does not “provide for future disruptive events or changes in stakeholder priorities that may change the mix” (Morrós 2017, p. 115).

Summarizing the above-mentioned literature, some main considerations emerge concerning its features and flaws.

First, only a few studies investigated MM under a graphical lens (e.g., Gelmini et al. 2015). This reveals a weak interest in the study of the tool as a special visual content of NFRs.

Moreover, despite the existence of a contradictory framework of scattered MM pros and cons, an in-depth debate on these features never arose. Consequently, literature focused on MM critique is lacking, even in the basic form of surveying good and bad practices.

Furthermore, the adoption of MM was mapped without considering its rate of spread. Hence, the literature completely disregarded the detected existence of a steady group of companies that never presented MM in their reports, as if this was not proof of criticalities.

Last but not least, a weak aptitude toward the search for direct proof of the usefulness of MM was detected. A few works focused on subjects affected by materiality, such as non-professional investors (Backof et al. 2020) or financial market professionals (Jørgensen et al. 2021), including a case study (Lai et al. 2017), while none directly consider the perceptions of MM.

If considered together, the latter two flags reveal that existing studies do not substantially enquire about the real extent of MM’s success by interpreting the practice trends, through cues offered by conflicting arguments expressed over time by multiple voices.

In order to bridge this gap, this article aims to better understand the NFR insiders’ viewpoints (namely, positions and arguments) on MM, which emerged before the GRI 3 shift, including those who were neglected by GRI. To this end, we formulated the following research question:

What approaches to MM adoption emerged over time and why?

The following section explains the line of enquiry applied to answer this question.

4. Research Design and Methodology

4.1. Research Design

In order to answer the research question, we set a quali-quantitative strategy of research (

Table 2. Research design.

Phase Feature	Phase 1 (Quantitative)	Phase 2 (Qualitative)
Fraction of research question answered	What approaches to MM adoption emerged over time?	Why did certain approaches to MM adoption emerge?
How viewpoints on MM are captured	Indirectly	Directly
Focused insiders	NFRs preparers	GRI stakeholders
Data source	2014–2020/21 non-financial reports	Public comments on Exposure Draft of the Universal Standards GRI 103: Material Topics (GSSB 2020)
Rationale	Both MM publication and non publication as tacit proof of preparers’ positions about MM adoption	Comments on MM as explicit proof of insiders’ opinions of MM
Analysed aspects (tools of analysis used are indicated between brackets)	Evolution of MM publication (percentages and graphs)	Submitters examined by approaches, category, and country (absolute values)
	Approaches to MM adoption (descriptive statistics)	Approaches to MM (qualitative content analysis)
	Association between MM adoption and GRI compliance (Chi-square)	Arguments expressed on MM (qualitative content analysis)

). Specifically, we selected the “sequential explanatory” approach, a mixed strategy (Tucker and Hoque 2017) for which the two phases are connected, yet separated (Creswell 2014).

As Table 2 exhibits, the first phase is quantitatively oriented while the second phase is qualitatively oriented. Each phase answers part of the main research question. Furthermore, one aspect detected during the first phase connects the two phases; particularly, the main approaches to MM detected during the first phase were used to set some a priori codes to be used during the second phase.

Table 2 summarizes the whole strategy of enquiry. The table starts from the parts of the research question answered and ends with the methodology adopted in each phase, including the materials and methods.

4.2. Phase 1: Materials and Methods

To detect the specific MM adoption approaches, MMs were searched among the 2014–2020 NFRs of a group of sustainability-oriented companies operating all over the world.

The 7-year period extends the span investigated so far, from 2014 to 2020, before the release GRI 3 in 2021. The sustainable orientation, assumed as a feature that increases the possibility of publishing NFRs, was defined as having consistently received the yearly RobecoSAM medals over the selected period. The RobecoSAM source was also used in previous studies (De Cristofaro and Gulluscio 2019; López-Arceiz et al. 2020a, 2020b; Tan et al. 2020).

The sample includes 60 companies, a statistical subpopulation that represents 5.83% of the initial sample (

Table 3. Sampling.

Criteria of Inclusion	Resulting Number of Companies
Initial list of companies: companies included at least once in the RobecoSAM Yearbooks (years 2014–2021)	1028
Minus: 849 companies not included in all eight 2015–2022 Yearbooks	179
Minus: 118 companies that did not consistently receive the yearly RobecoSAM medals over the selected period	61
Minus: one company that does not publish online pdf versions in English of its NFRs	60 (the sample)

).

The sample covers 20 countries (frequency range: 1–6) and four continents (America, Asia, Europe, and Oceania). Companies are distributed across 41 sectors, with frequencies ranging between one (27 sectors) and six (one sector). The most represented continent, countries, and sectors are, respectively, Europe (slightly more than 50%), Spain, Switzerland, and the USA (six instances each), and electric utilities (six Spanish, Italian, and Portuguese companies).

With regard to the data source, the pdf files of 888 reports were found online at the selected cut-off date (15 May 2022).

Through the keywords “matrix” and “materiality”, MMs were searched among report pages. When the launch failed, only the sections on the materiality analysis of the NFRs were manually examined by the authors. Afterwards, the results of their reading were compared and discussed.

The main output of this phase should be to highlight whether companies tend to gather based on their choice to publish MM (i.e., approaches to MM). In order to verify whether and how these groups differ in terms of the number of MM published yearly, they were compared through descriptive statistics measures.

Moreover, a Chi-square test associated the following pairs through SPSS: (i) yearly MM presentation (modes: yes/not) and yearly GRI compliance (modes: strong/medium/weak/any), and (ii) attitude toward MM presentation over the period (modes: pro MM/against MM) and attitude toward GRI compliance over the period (modes: any/weak/medium/strong/very strong). The variables chosen to perform the latter association overcame the risk of repeating observations within longitudinal data.

4.3. Phase 2: Materials and Methods

To detect the reasons behind the approaches to MM adoption, a qualitative content analysis of public responses to the ten online survey questions (

Table 4. The GSSB survey questions on the GRI 103 Universal Standard exposure draft: themes.

Question	Theme
1	Key concepts in the GRI standards
2	Using the GRI standards for sustainability reporting
3 and 4	Statement of use
5	Governance
6	Identifying material topics
7 and 8	Reporting on material topics
9	Structure of the universal standards
10	Other comments

) about the GRI 103 Material Topics Exposure Draft (GSSB 2020) was performed.

Developed in the 20th century as a quantitative technique (Mayring 2014), content analysis gradually spread in the social sciences (Holsti 1969; Krippendorff 2004; Weber 1990) as a family of increasingly systematic and rigorous approaches (Hsieh and Shannon 2005), which employ a wide range of document analysis techniques generating qualitative and quantitative insights (White and Marsh 2006). Several studies on company reports rely on the content analysis of responses to standard setting boards, such as, for example IASB (Damian et al. 2014), IIRC (Oprişor 2014, 2015; Reuter and Messner 2015), and SEC (Higgins et al. 2017).

The GRI’s website allows the retrieval of the Excel file containing the 144 submissions received.

By considering the text included in each Excel cell as a single comment, 1620 comments were counted. Afterwards, through four keywords (namely, matrix, matrices, visual, and graph), 14 comments (i.e., 14 excel cells) from 13 respondents were extracted. Consistently, GSSB observed that “A few respondents commented on the use of the materiality matrix included in the current Universal Standards 2016” (GSSB 2021a, p. 7).

As for the geographical provenance of the comments received, they were predominantly received from European respondents, with a percentage of 36.81% and 46.15% when all respondents and respondents commenting on MM are considered, respectively (

Table 5. Respondents to the 2021 Universal Standards exposure draft: demography.

Panel 5A. Geographical Provenance.
Continent	Respondents
	Overall		Commenting MM
	(No. 144)	% (Out of 144)	(No. 13)	% (Out of 13)
Africa	7	4.86	-	-
Asia	44	30.56	4	30.77
Europe	53	36.81	6	46.15
America	31	21.53	2	15.38
Oceania	5	3.47	1	7.69
Not stated	4	2.78	-	-
Panel 5B. Categories.
Category	Respondents
	Overall		Commenting MM
	(No. 144)	% (Out of 144)	(No. 13)	% (Out of 13)
Business enterprise	32	22.22	3	23.08
Civil society organization	23	15.97	1	7.69
Investment institution	10	6.94	-	-
Labour organization	4	2.78	-	-
Mediating institution	73	50.69	9	69.23
Not stated	2	1.39	-	-

, Panel 5A). European respondents were immediately followed by Asiatic, scoring approximately 30% in both cases.

As for the respondents’ categories, the comments were predominantly received from mediating institutions (Table 5, Panel 5B). The relative weight of mediating institutions even increased with the shift in focus from all the comments received (50.69) to only those concerning MM (69.23%). Interestingly, business enterprises represent approximately a fifth of the respondents both in the case of all the comments received (22.22%) and those comments related to MM (23.08%).

At this step, before reading the comments, we discussed which comments had to be extracted in order to be processed through the subsequent content analysis. For consistency with our research aim (i.e., to understand the NFR insiders’ viewpoint about MM), we considered whether all categories of submitters that expressed a comment on MM could have been potentially considered as useful NFR insiders. In this regard, we observed that mediating institutions could discuss and describe preparers’ practices, and business enterprises could discuss their direct experiences. Similarly, civil society organizations could provide interesting opinions on MM.

Thus, we processed all 14 comments extracted through keywords, namely all those including at least a mention of MM.

The 14 comments represented the basic text units to be further divided into smaller meaning units, to be coded during the content analysis process detailed in Figure 4.

This process relied on a mixed coding approach involving both a deductive setting (first cycle) and inductive coinage (second cycle) of codes (Mayring 2014). Small-scale data allowed human coding (Tan et al. 2020).

To enhance coding stability (Weber 1990; Krippendorff 2004; Campbell 2017), phases B, C, and D were repeated.

The intra-coder consistency was tested through percentages of agreements and Cohen’s Kappa (Lombard et al. 2002). At the end of the second round of each phase, an external auditor (Creswell 2014) solved cases of uncertainty coding.

The outcomes of these three phases are indicated in Figure 4, taken from the end of the second round.

The external expert checked the overall procedure.

5. Results

5.1. Results of Phase 1 (Materiality Matrix Adoption): Evolution, Approaches, and GRI Compliance

The first research phase detected the following aspects:

• The evolution of MM adoption and non-adoption;
• Approaches to MM publication;
• MM adoption and GRI-compliant association.

As for the evolution of MM adoption, our results show that:

(i)

Overall, the majority of the companies (52 companies, i.e., 86.67% of the sample) publish an MM at least once over the period 2014–2020 in their NFRs.

(ii)

The yearly percentage of companies that present an MM increases over time and ranged between 63.33% in 2014 and 78.33% in 2017 (Figure 5). Hence, MM adoption never reached 100% because companies that ignored the visual persisted over the period and even recovered in 2019 and 2020.

Moreover, companies include an MM in one or two of the 360 NFRs regardless of the report type adopted, namely annual (40.56%), sustainability (22.22%), integrated (18.06%), social responsibility (17.78%), and materiality (1.39%) reports.

To identify the main approaches to MM adoption, the stability of the choice to publish MM was investigated on the basis of the number of changes in decisions (i.e., whether or not to publish) that occurred over time. Then, 14 detailed choices were found and marked with a-n lowercase letters (see

Table 6. Main approaches to MM publishing in 2014–2020 non-financial reports: steps.

First Step—Detection of Choice to Publish MM according to Stability			Second Step—Aggregation of Detailed Choices of Approaches
Detailed Choices	Abs	%	Approaches to MM	Abs	%
(a) Always published	33	55.00	A—In favour of MM adoption	45	75.00
(b) Always published from 2015 onwards	4	6.67
(c) Always published from 2016 onwards	3	5.00
(d) Always published from 2017 onwards	2	3.33
(e) Always published from 2018 onwards	1	1.67
(f) Not published only in 2017 and 2019	1	1.67
(g) Not published only in 2017 and 2020	1	1.67
(h) Never published	8	13.33	B—Against MM adoption	15	25.00
(i) No more published from 2018	1	1.67
(j) No more published from 2017	1	1.67
(k) Published only in 2016 and 2017	1	1.67
(l) Published only in 2017 and 2018	1	1.67
(m) Not published in 2017, 2019, and 2020	1	1.67
(n) Published only in 2020	2	3.33

, first step column). Remarkably, while the majority of companies steadily either published or did not publish an MM (41 companies, resulting in a total sum of “a” and “g” choices of 66.67%), a fourth (12%) was slightly less steady since they mainly began publishing after 2014 (choices b, c, d, and e) or even stopped at the end of the period (choices i and j). The remaining seven companies (11.67%) adopted a more erratic/irregular pattern of behaviour, publishing MM in discontinuous years (choices g, k, l, m, and f) or only once (choice n).

Subsequently, the detailed choices were gathered into the following approaches (Table 6):

(A)

In favour of MM adoption, i.e., cases of both steady adoption over the period and trends revealing a gradual or clear preference towards MM adoption;

(B)

Against MM adoption, i.e., cases of both steady non-adoption over the period and trends revealing MM abandonment or puzzling behaviours.

The descriptive statistics in

Table 7. Materiality matrices published over the period 2014–2020: descriptive statistics *.

Companies	Measures	2014	2015	2016	2017	2018	2019	2020
Sample	Mean	0.62	0.77	0.87	0.87	0.97	0.92	0.92
	Standard Deviation	0.55	0.59	0.59	0.62	0.63	0.63	0.61
Pro MM	Mean	0.73	0.93	1.02	1.07	1.24	1.18	1.18
	Standard Deviation	0.49	0.49	0.45	0.49	0.43	0.44	0.44
Against MM	Mean	0.27	0.27	0.40	0.27	0.13	0	0.13
	Standard Deviation	0.57	0.57	0.71	0.57	0.34	0.00	0.34

(*) The distributions used count twice the matrices that are published in two reports (regardless of their being identical or not).

show that the two groups stably differed over time according to the average number of MMs published. In detail, while the mean number of MMs published yearly increases over time in the pro-MM group, it decreases in the against-MM group. The low standard deviations of both series strengthen this difference, since they indicate low dispersions, i.e., the steady homogeneity of the two groups.

Finally, the association between the yearly MM presentation and the yearly GRI compliance was examined (

Table 8. Statistical association between yearly MM presentation and yearly GRI compliance: Chi-square.

Year	Pro-MM Companies (No. 45)	Against-MM Companies (No. 15)	Overall (No. 60)
2014	19.688 *	1.458	20.400 *
2015	8.226	2.292	15.485 *
2016	0.520	0.938	6.319
2017	0.517	3.393	12.243 **
2018	Not calculable	3.462	17.111 *
2019	0.157	Not calculable	25.390 *
2020	0.157	4.904	4.432

* Significant at the 0.01 alpha level. ** Significant at the 0.05 alpha level.

).

In detail, Table 8 indicates both weak (e.g., pro-MM companies in 2014) and not statistically significant (e.g., against-MM companies in 2014) associations. In two cases (e.g., pro-MM companies in 2018), any Chi-square was calculable, since the MM variable is a constant and this demonstrates in itself that no association exists. The Chi-square test conducted between the attitudes over time towards MM presentation and GRI compliance confirmed a statistically significant low association (i.e., 29.291).

5.2. Results of Phase 2 (Opinions on Materiality Matrix Adoption): Content Analysis of GRI Stakeholders’ Comments

In order to search for opinions that may have driven the two main approaches to MM adoption found in the previous research phase, the second phase performed a content analysis of responses to GSSB’s call for comments on the 2021 Exposure Draft of the GRI 103 Universal Standard.

The first coding cycle (please refer to step C of Figure 4) established two macro-categories of codes based on the two main approaches above, namely “Pro MM” and “Against MM”. Thus, the opinions expressed by the submitters about the removal of MM are encoded within these two macro-categories of codes. In detail, the “Pro MM” macro-category gathered the following arguments (expressed by seven respondents):

• Explicitly against MM removal (i.e., concerns about the consequences of MM removal; MM abandonment without a clear alternative creates negative consequences);
• Explicitly in favour of retaining MM (i.e., requests to include MM in the new standard; MM must be retained; a two-dimensional matrix is preferable; the standard should mention that companies can publish MMs);
• Tacitly in favour of MM (i.e., MM can be cited within the statement of GRI use).

Instead, the “Against MM” macro-category summarized the following arguments (expressed by four respondents):

• Explicitly in favour of MM removal (i.e., MM removal is a welcome change; the elimination of both double criteria and MM is appreciated; the move away from the matrix approach is supported; MM is no longer needed).

The demographic analysis of the two groups of respondents showed that while pro-MM opinions are predominantly expressed by European mediating institutions, against-MM opinions mainly come from both Asia and Europe, as well as from business enterprises (

Table 9. Two main groups of opinions on MM: submitter demographics.

Panel 9A. The “Pro-MM Group” (7 Respondents) Examined by Stakeholder Category and Geographical Provenance.
Continent	Country	Mediating Institution (No. 7)		Totals
				Continent	Country
Asia	Hong Kong	1		2	1
	Russian Federation	1			1
Europe	Italy	1		4	1
	Switzerland	1			1
	United Kingdom	1			1
North America	United States	1		1	1
Oceania	Australia	1		1	1
Panel 9B. The “Against-MM Group” (4 Respondents) Examined by Stakeholder Category and Geographical Provenance.
Continent	Country	Business Enterprise (No. 3)	Mediating Institution (No. 1)	Totals
				Continent	Country
Asia	Hong Kong	1	-	2	1
	Malaysia	1	-		1
Europe	Italy	1	-	2	1
	United Kingdom	-	1		1

). Since two submitters (i.e., a Swiss civil society organization and an American mediating institution) mentioned MM without clearly expressing their position on MM, they were not included within the table.

Afterwards, through emerging codes that were hierarchically organized into code-categories and sub-codes, the second coding cycle (please refer to step D of Figure 4) coded specific arguments in support of both positions on MM (

Table 10. Content analysis output: the code tree.

Code Macro-Categories	Code Categories		Sub-Codes
Label	Label	No	Label ‡
A. Pro-MM	MM advantages	6	(a2) MM fosters transparency.
			(c1) The MM y-axis has a double potential (impact significance and scaling).
			(a2) MM is useful to understand the relative importance of topics and the reasons behind their materiality.
			(b3) The MM visual approach provides insights to users.
			(a2) MM shows stakeholder assessments.
			(a3ii) MM is understandable and applicable.
	MM removal disadvantages	4	(e1) MM removal generates difficulties for preparers.
			(e2) MM removal reduces transparency on material topics evaluation.
			(b4) MM reshaping would weaken the stakeholder engagement concept.
			(a4) MM removal without providing alternatives will create confusion, inconsistency, and incomparability.
	Expected MM features	5	(e4) MM should be both a decision-making and communication tool.
			(a2) MM should be voluntary.
			(a3) MM should be presented according to the new impact-based materiality concept.
			(b1) The MM axes have to be specified (e.g., impact probability/severity).
			(b2) The MM axes have to be redefined.
	Suggestions for MM	2	(e5) Both MM and other tools could be leveraged.
			(e6) A new MM addressing both impact significance and occurrence likelihood could be considered.
B. Against-MM	MM disadvantages	3	(a2) MM misleads.
			(b3) MM encourages simplistic issue identification.
			(b4) MM undermines a topic’s accountability and management controls.

‡ Words in bold indicate arguments included in the GSSB’s summary (GSSB 2021a, p. 7). Each sub-code scores one.

).

Remarkably, the pro-MM macro-category is internally more articulated (four code categories) and argued (17 sub-codes) than the against-MM macro-category (only one code category and three codes). The majority of the respondents proposed more than one argument each. Moreover, while the pro-MM group proposes both advantages of MM and disadvantages of MM removal, the against-MM group proposes only disadvantages of MM and does not propose any advantage of removal.

In detail, pro-MM respondents appreciate the comprehensibility and applicability of MM, its usefulness in understanding a topic’s importance and determination, its informative potential about stakeholders, and it being a visual approach that both simplifies and provides insights. Thus, in their opinion, the removal of MM would generate difficulties since it would reduce transparency in materiality evaluation. Furthermore, while they suggest MM leverage for its double role as a decision-making and communication tool, they expect a new voluntary MM based on the impact-based concept of materiality, renewed as for the axes and addressing impact significance, as well as occurrence likelihood.

On the other hand, against-MM respondents highlight that MM misleads users, encourages simplistic materiality analysis processes, and undermines both a topic’s accountability and management controls.

As Table 10 highlights in bold, the code tree obtained is consistent with the GSSB’s summary, affirming the following: “Those opposed stated that the matrix tends to encourage overly simplistic issue identification and may undermine accountability and appropriate management controls for a given topic. Those in favour stated that a visual presentation of material topics is a useful tool for understanding the relative importance of an organization’s material topics and makes materiality more transparent, understandable and applicable” (GSSB 2021a, p. 6).

However, a wider range of hierarchically organized arguments (e.g., concerns, expectations, and suggestions) were found. These arguments, besides the quantitative details provided, suggest a very interesting framework of positions on MM that are more articulated than those depicted by GSSB.

Finally, both the percentage of agreement (94.90%; 100%; 88.24%) and the Cohen’s Kappa tests (0.883, 1.00 and 0.845, all significant at the 0.01 alpha-level) performed in phases B, C, and D of the content analysis process (Figure 4) indicated strong intra-coder reliability.

6. Discussion and Conclusions

This paper first considers the disappearance of MM after its long-term inclusion in the official documents of the GRI on materiality assessment. In detail, GRI deems a two-axes matrix to no longer be needed, as the revised GRI concept of “material topics” (namely, topics that represent an organization’s most significant impacts on the economy, environment, and people) no longer encompasses two independent criteria.

In detail, besides making mandatory the final list of material topics in which the impacts were gathered (GRI 2021), GRI 3 now allows organizations to provide a visual representation of the prioritization phase, showing both the initial list of identified topics and the reporting threshold set (GRI 2021). Hence, GRI opts for a mediating solution between still providing examples of visual tools and the complete deletion of any visual.

Unfortunately, the existing literature neither foretells nor considers the removal of MM and its consequences. Moreover, the features found in the literature suggest several considerations. First of all, it was found that a few authors stress the pros and cons of the graphical nature of MM. This collides with the powerful sustainability information that graphs convey to stakeholders through NFRs (Cȕre et al. 2020). Furthermore, it was found that no study centrally links MM with the themes of opportunities and risks. However, AccountAbility noted at an early stage that the first matrices were adapted from previous risk analysis frameworks (AccountAbility 2006). Notably, along the line of pioneers that even used a third axis (AccountAbility 2006), some companies adopt a three-dimensional map, the third axis of which adds “business risks” to the “stakeholder interest” and “impacts on company’s ability to deliver on strategy” axes (Zhou and Lamberton 2011, p. 6).

In order to fill the specific gap in the literature consisting of the neglected existence of a group of companies that never included MM in their NFRs, we asked a devoted research question that this study answered through a two-phase analysis.

The results found in Phase 1 on trends in MM adoption align with the literature. Indeed, consistent with the diachronical evidence provided by the studies mentioned in Table 1, we found an increasing trend of MM adoption over the 2014–2020 period. Furthermore, novel results on the two detected main approaches to MM indicate that a minority core of MM non-adopters have reiterated their reluctance so much as to prevent the substantial takeoff of the tool. Moreover, consistently with previous studies (Garcia-Torres et al. 2017), companies include an MM into their NFRs regardless of the type of report adopted. Interestingly, it was also found that MM presentation is not linked to GRI compliance in both groups of companies. This suggests that MM was adopted with and without GRI compliance, and that MM was not adopted regardless of GRI compliance.

Moreover, the results provided by the content analysis of Phase 2, on the reasons supporting the positions in favour of and against MM, include both already known arguments (i.e., consistent with the 2021 GSSB’s summary) and novel opposing arguments. In detail, several stakeholders’ pro-MM arguments were not considered by the GRI, and very few detailed arguments against MM were provided. If considered together with the critique on MM examined in the literature, these further insights shed light on a wide range of possible drivers underlying the approaches in favour of and against MM.

All of the above makes the future graphical presentations of the MAP in NFRs uncertain, both in terms of MM use and the use of any other visual. For example, it does not seem that the new concept of material topics prevents preparers from drawing MM. Indeed, the list of material topics selected according to the new one-way impact-based criterion (i.e., a criterion that considers the most significant impacts of an organization on the economy, environment, and people) could be further enriched in terms of a visual based on two-way description (i.e., on a criterion that separately considers the company and stakeholder viewpoints on material topics). Hence, the stakeholders’ participation in the processes of identifying impacts and determining material issues will not prevent companies from making the matrix an additional source of information, where internal and external perspectives on selected material issues can be compared.

Lessons that can be learnt throughout this study overlap with policy implications for NFR standard setters, academicians, and companies, as follows.

First, the convergence of certain overly soft positions of NFR standard setters and fanciful company practices can harmfully undermine the usefulness of materiality assessment tools. GRI, for example, must avoid repeating the mistake of creating confusion in the future by assuming that rules restrict the preparers. Indeed, without minimum indications to be respected, companies can distort the original meaning of each operative instrument of MAP. When an exemplificative graphic tool is proposed, strong guidance is needed, regardless of whether it is mandatory or not. A graphic tool does not have to be automatically compulsory but, once it appears in guidelines or standards, it must be used uniquely to avoid a lack of comparability.

Second, the systematic monitoring of company practices is important. To avoid reaching an irrecoverable situation in which a tool for materiality analysis spreads among companies, straying far from the original meaning conceived by the standard setter who proposed it, early company practices must be surveyed. In this way, it could be possible to provide more precise indications as well as useful ideas and clarifications before companies consolidate their own materiality analysis processes in completely non-converging directions. In this context, the role of academicians and research centres is essential in supporting the work of standard setters.

Moreover, how to monitor practices is also important. When authors mapped the adoption of the MM, the spread was detected without considering its speed. Thus, it seems that the literature considered this growth as implicit proof of the usefulness of MM, without appropriately focusing on its intensity. In these research perspectives, our study provides evidence on how the mere mapping of the spreading of an operative tool is not enough to judge its success, since an analysis of the strength of adoption is needed to discover the underlying paths.

Third, besides surveys on the practices of an operative tool, the voices of both the preparers and readers of NFRs about the tool’s usefulness and pitfalls are of great importance. Knowledge on the effectiveness of any graphic means remains opaque without going beyond a merely descriptive mapping of its adoption. To better understand the use and usefulness of a tool, academicians must directly solicit voices involved in MAP and not settle for the adoption of a tool as proof of its usefulness.

Last but not least, relatively few companies tend to respond to calls for feedback from standard setters, and even fewer express their opinion on graphical tools. Hence, the culture of participation in the standard-setting process should be developed among companies. Larger numbers allow a wider range of arguments to be acquired both for and against the proposed instruments, and significant statistical analyses to be carried out based on several dimensions (e.g., stakeholder group, company industry, size, and country), which current numbers still prevent.

In order to overcome the main limitations of this article, some directions for future NFR research avenues on the use of graphical utilities in MAP disclosure can be provided.

First, different samples (i.e., geographically focused, cross-industry, and large samples) could be considered to better investigate the association between GRI compliance and the use of visuals, including MM, as well as between approaches to materiality assessment, including MM and the following: (i) country; (ii) industry; (iii) company’s size; and (iv) early double materiality adoption practices. As for the latter, within European countries involved in the process of issuing the proposal of a Corporate Sustainability Reporting Directive (2021/0104 (COD)), it could be interesting to investigate whether and how both pro- and against-MM approaches affect the coexistence between MM and double materiality implementation. Second, to offset the risk of MM underestimation involved in considering the publication of a visual into an NFR as proof of its adoption within the process of materiality analysis, the future mapping of materiality analysis graphical tools could consider their existence (e.g., through text citations) instead of their mere publication in reports. Third, questionnaires or interviews could be used to solicit responses from a wider range of practitioners involved within companies’ materiality assessment exercises (e.g., NFRs preparers, managers) than those responding to GSSB calls. Fourth, the voices of stakeholders (engaged or not) could be directly heard in order to understand to what extent graphical tools summarizing materiality analysis outputs are really considered useful, and what expectations users express about their setting and design (e.g., at the beginning of the report, linked to the website, clearly connected to report chapters, etc.). Fifth, “why-based” research questions could be coined and interpreted along theoretical lines, depending on the examined period of application of a standard. In detail, in the reporting period immediately after the GRI 3 came into force, mimetic isomorphisms (which generally accompany uncertainty) could be detected under the New Institutional Theory lens, as carried out, for example, by Lakshan et al. (2021) on a materiality exercise in IR. In subsequent periods, impression management (Merkl-Davies and Brennan 2007) or managerial capture (O’Dwyer 2003) lenses could be used to examine graphical choices in NFR disclosure (Cho et al. 2012a, 2012b; Jones 2011; Pesci et al. 2015, 2020) and assurance (Owen et al. 2000), respectively. Finally, the lack of in-depth knowledge on the very drivers of the MM path (i.e., full trust and mechanic compliance) could open up the use of the “path-dependency” framework (Sydow et al. 2009) to explain the stability of graphical practices adopted during the maturity of the GRI 3 application.

Meanwhile, researchers may wonder whether there will be a replacement for the matrix, and, if that be the case, who will be the first to coin it. That is to say, will new graphic tools supporting the explanation of MAP methods and outputs again stem from pioneer companies’ experiences, or will the GRI anticipate companies’ practices by proposing new graphic solutions? Regardless of who will make the first move, it is reasonable to suggest that the path of MM should not be discarded. In whatever way, it could be fruitful to build the next visuals upon the lessons learnt from the missed takeoff described in this study.