The Language of Sustainable Tourism as a Proxy Indicator of Quality

Abstract

Sustainable tourism (ST) has recently become the mainstream of the tourism industry and, accordingly, has influenced contemporary tourism research. However, ST is not just theories about indications and contraindications of global travel, but also a specific language that needs mastering to take sustainability work forward. In other words, what research receives recognition depends on the proficiency in how the articulation in research proposals and within assessment under the heading of “research impact”. The aim of this paper is to investigate how tourism research gains recognition within research evaluation, by investigating the national research appraisal in the United Kingdom (Research Excellence Framework). By using content analysis, we disentangle the rhetorical choices and narrative constructions within researchers’ impact claims. Our findings suggest that researchers adopt a rhetorical style that implies causality and promotes good outcomes facilitating ST. However, the structure of the assessment format enforces an articulation of sustainable research impact without stating the methodological limitations of that such claim. Therefore, the rhetorical choices of ST researchers merely represent a proxy indicator of the claimed impact. We conclude that the lack of rigor in accounting for the impact of ST research may inadvertently restrict attaining ST.

1. Introduction

Sustainable tourism as an idea implies that “tourism which is in a form which can maintain its viability in an area for an indefinite period of time” [1] (p. 36). Yet, it is not always clear if the tourism dimension (as an activity in itself) or the sustainability dimension (the outcome and knock on effects) are the primary concern [2]. Equally, due to the dispersed and fractal nature of tourism, tourism research often faces difficulties in quantifying the degree of sustainability/nonsustainability due to its internal epistemological, ontological, and methodological differences [3]. Whilst quantifying and theorizing the impact of the tourism industry can be difficult, it undoubtedly influences the environment and society through its use of resources [4]. When talking about research that aims to facilitate sustainable tourism, similar complexities arise. There are plenty of individual examples of researchers, the industry, and tourists finding avenues to increase social and environmental sustainability [5,6,7]. Yet, what impact tourism scholars themselves hold when contributing to sustainable tourism in general may be more difficult to trace [8]. Untangling the complexities involved in accounting for the impact of tourism researchers on sustainable tourism is the knowledge gap this research addresses.

Theoretically, ‘research impact’ as an idea describes the consequences of research in terms of its societal influence [9]. What is important to note here is that this understanding is value-neutral in the sense that such influence can have a positive influence or negative effects upon society or the environment, which represent the subject of the inferred impact. In relation to sustainable tourism as well as research, that facilitates such sustainability, such ambiguity dissipates, as the intended outcome becomes apparently clear. Unfortunately, recognition of the situated nature of who is making such claims is not always present. For instance, Kellee Caton writes “… most of us who work in the tourism academy or in the industry—and who tend to be avid tourists ourselves—have spent at least some time daydreaming about the kind of tourism world we’d like to be involved with” [10] (p. 1906). Such instances of vested interest potentially complicate claims of societal influence, as these introduce a bias in the appraisal of one’s own research impact. According to her, the reason for this is that “[on] some level, our experiences translate into values, which guide our scholarship, leading us toward the particular types of questions we find important, which we then commit our energy to pursuing” [11] (p. 1906). Others argue that for sustainable tourism such positionality of the researchers themselves has far-reaching consequences. For example, “[…] the ways in which we research green consumers might be partially responsible for the attitude-behavior gap [… and] researchers need to examine whether their research designs are part of the problem. This caution applies not only to tourism researchers, but across the social sciences” [11] (p. 170).

Scholars may perfectly well identify ‘cognitive dissonance’ in others’ actions when postulating the lack of sustainability due to the attitude–behavior gap [12]; but what happens when the researchers themselves “contribute to debates in the other social sciences” [13] (p. 276) and society in general? What are the effects of researchers’ own actions for promoting or hindering sustainable tourism? The complex interplay between personal ambitions, research requirements and societal expectations are a direct consequence of the relational character of any form of knowledge production that wants to proclaim impact (e.g., [14]). In this paper, we study what happens when tourism researchers make claims about their own sustainable influence, specifically focusing on the language they use.

Taking into account the above considerations, the specific research aim of this paper is to examine how tourism researchers articulate their impact in facilitating sustainable tourism, without jeopardizing their research integrity. Three research objectives guide our study. Firstly, we want to identify how scholars dealt with the potential value conflicts within a research impact context. Secondly, we want to review the evoked evidence for sustainable impact claims stemming from tourism researchers themselves. Thirdly, we want to analyze how the language of sustainable tourism and research impact interact, meaning how the causality invoked facilitates more sustainable tourism.

2. Background

2.1. The Identification of Tourism Research Impact

To operationalize our research objectives, we make use of so called ‘impact case studies’ (i.e., the four-page documents that were the subject of the impact evaluation) produced by UK tourism researchers for the purpose of impact evaluation. In the UK, since 2007 ‘research impact’ is utilized by the government as an evaluation criterion for all research [15]. Specifically, within the Research Excellence Framework (REF) in 2014, the evaluation of research impact received its formal introduction and tied this notion to research funding [16]. The REF is the successor of what was the Research Assessment Exercise, which had represented a nationalized assessment of research within the UK from 1986. Such national research evaluations occur every seven years, with the next one to commence in the beginning of 2021. The nationalized research assessment elevates research based on their output (i.e., publications), environment (i.e., community), and now also based on their societal influence.

Tourism research has not always fared well within these types of assessments. For example, John Tribe points out that the structural design of the 2001 iteration effectively marginalized tourism research. He argues that the structural omission of not having a designated unit of assessment, caused tourism research to appear much more peripheral than it needed to be [17]. Thereby, the REF 2014 not only formally established an impact agenda that caused ambivalent views among researchers [18], but it also officially recognized tourism as a discipline within the evaluation structure. This would mean that the evaluation of tourism research that claims positive impacts on sustainability ought to be clear about how those impacts manifested. If not, it could be suggestive of a glitch in the system that would require attention.

2.2. Literature Review on Research Impact Assessment

Whilst tourism research and sustainable tourism is the specific topic of this paper and forms part of its research purview, the implications for research praxis reveal a wider research culture in peril [19]. For the sake of consistency, we used the definition of research impact the REF settled on in their 2014 assessment. They defined impact as “any social, economic or cultural impact or benefit beyond academia that has taken place during the assessment period and was underpinned by excellent research produced by the submitting institution within a given timeframe” [20] (p. 48).

The REF further specify that impact includes, but is not limited to, changes or benefits to “the activity, attitude, awareness, behaviour, capacity, opportunity, performance, policy, practice, process or understanding of an audience, beneficiary, community, constituency, organisation or individuals in any geographic location whether locally, regionally, nationally or internationally. Impact includes the reduction or prevention of harm, risk, cost or other negative effects” [20] (p. 48).

The impact is being judged based on its reach and significance, reach means “the extent and diversity of the communities, environments, individuals, organisations or any other beneficiaries that have benefited or been affected” [21] (p. 74). Significance meanwhile means the “degree to which the impact has enriched, influenced, informed or changed policies, opportunities, perspectives or practices of communities, individuals or organisations” [21] (p. 74). Both reach and significance are not geographically defined or dependent.

From a philosophical and ethical point of view we can see problems with such a materialistic and politicized definition of research impact. In that, political conflicts are fought in the arena of research evaluation [22], conceptual impacts [23], and impacts with long causality chains and implementation time frames [24] are less suited for this style of assessment, as well as the role of research for society is being devalued in only being a service provider [25].

3. Materials and Methods

3.1. Identifying Proxy Indicators of Research Impact

Within tourism research, content analysis represents a common research method [26]. It entails reading documents with a certain goal in mind, and thereby requires a methodological design to make the procedure as robust as possible. Within a research context, when evaluating the veracity of any given claim and the causality of any claim, one would typically focus on: what the authors know (erudition), whom they cite, how they structure an argument, how they write up a scientific article, do they cite the best journals in a discipline, what type of claims are allowed and which are not, which are the ‘hot’ or ‘not-so-hot’ topics, where to apply for funding, whom to (not) collaborate with, what universities are the top ones within a particular discipline, what conferences are the important ones, how research performance is evaluated, etc. [27,28,29,30,31,32,33,34,35]. Collectively, such instances of face-value judgment function as—label it here—proxy indicators of research quality. While there is consensus what these are within research context, when it comes to a research impact context it is less clear which indicators are sought-after to represent ‘excellent’ sustainable tourism research impact.

We developed the idea of proxy indicators from Harry Collins and Robert Evans’ notion of interactional expertise, which they define as “mastery of the language of a domain, and mastery of any language, naturally occurring or specialist, [which] requires enculturation within a linguistic community” [36] (p. 30).

In this paper, we define proxy indicators of research impact as the manifestation of interactional expertise, where the articulation succeeded in communicating across different linguistic communities. We regard them as successful acts of communication, as sustainability claims by researchers (community A) were evaluated by tourism industry representatives (community B), implying that communication occurred. We argue; articulating excellent research impact presupposes such a skill, as otherwise communication would not occur. The structural norm of excellence, as implied within assessment of the Research Excellence Framework, thereby becomes our quality threshold for the here presented findings. Our assumption in identifying our proxy indicators of research impact rests upon the inference that universities have a stake to present their best societal influence, as the REF represents a competitive evaluation for research funding.

Here we should also provide a note on our application of content analysis. We are approaching the ‘content’ of the impact claims holistically from the use of language. Hence, our inquiry into the semantics of sustainable tourism claims also includes the context in which they occur, their argument structure, the use of evidence, etc., all taking inspiration from proxy indicators of research quality. Additionally, a word of caution is in place around proxy indicators of research impact, in that we presuppose that we are identifying a linguistic repertoire that is possible to communicate across multiple domains. Hence, whilst we use the label proxy indicator as the mere invocation of a linguistic category, it is by no means a guarantee for a sustainable research impact. The demarcation if the presented claims genuinely counts as a sustainable one is as much contingent upon the implicit expertise of the individual making the judgment as it is on the presentation of the information. In this article, we focus on the latter, and take the former for granted.

3.2. Selecting Cases of Sustainable Tourism Research Impact

When identifying proxy indicators of research impact, we selected case studies submitted to the REF 2014 that related to sustainable tourism impacts. The case study submissions followed a formulaic approach in their presentation, obliged to include the following aspects [20] (pp. 50–52):

• Summary of the impact (indicative maximum 100 words);
• Underpinning research of the impact in question (indicative maximum 500 words);
• References to the research (indicative maximum of six references);
• Details of the impact (indicative maximum 750 words);
• Sources to corroborate the impact (indicative maximum of 10 references).

The REF administrators published all submitted case studies online in spring 2015, with full consent of the submitting institutions that the information will enter the public domain. The here analyzed sample size, is a subset of all ‘tourism research impact’ that was submitted to the REF 2014 [8]. The reason why we focus on this specific subset, is that these case studies were made from self-identified tourism researchers, i.e., individuals expected to master the language repertoire around describing the sustainability of tourism. Such a cohort is most likely to possess the language skills of the proxy indicators (of research impact) we seek to analyze. Thereby, our chosen sample size represents a natural experiment to study the language repertoire of experts (i.e., UK tourism studies) in articulating the sustainable impact claims within their chosen area of expertise.

Table 1. Tourism Higher Education Institutions (HEI) that submitted research impact to REF 2014.

Tourism HEI	Name of Case Study
(1) University of Nottingham	(1) Informing Social Tourism Policy and Practice
(2) University of Kent	(2) Backpackers or Cruise Ships? Shaping the Tourism Policy Agenda for Small Island States and Coastal Communities
(3) University of Cardiff	(3) Understanding the economic and environmental impacts of tourism
(4) Bournemouth University	(4) Modelling economic impact for national governments
(5) University of Sunderland	(5) Integrated e-Services for Advanced Access to Heritage in Cultural Tourist Destinations (ISAAC)
(6) University of Surrey	(6) Modelling and Forecasting International Tourism Demand: Methodological Advancements and Innovations
(7) University of Surrey	(7) Reducing social exclusion through participation in tourism
(8) University of Bedfordshire	(8) The impact of food tourism on sustainable development in rural regions
(9) University of Brighton	(9) Redesigning tourism policy and practices in Africa

Source: authors.

denotes the identified tourism studies institutions and their associated research impacts. In total, our selected corpus comprised the research effort of 48 different researchers (as mentioned explicitly in the case studies). We previously [8] classified these impacts as pertaining to destination development, furthering economic, social, and environmental sustainability, raising awareness, and lobbying around related causes that occurred between 2007 and 2014 as per the REF guidelines. As the quality profile of the submitting institutions falls within the overarching pattern of their respective assessment panels, we deem the manifested skill in articulation (that is the basis of our analysis) as satisfactory for identifying proxy indicators of research impact.

3.3. Identifying Proxy Indicators of Sustainable Tourism

Within a research context, to assess the validity and content of any scientific claim, the premises of the claim need articulation in order to aid communication and evaluation [37]. The same applies within an impact context, and we can see that a form of impact literacy is slowly emerging [38]. Henceforth, in our content analysis to identify proxy indicators of research impact, we focused our analysis on the following process. Firstly, we analyze the argumentation structure of the presented claim. Secondly, we study the references and information invoked to evidence the impact claim. Finally, we study the language used to present the claim. We concluded that these three aspects represent the minimum requirements for any type of impact claim to be operational in conveying information.

Table 2. Analytical foci and associated questions used to identify proxy indicators of research impact.

Analytical Foci	Reading Prompts
(A) Argumentation structure	(a) Is the context of the impact claim introduced?
	(a) When did the research cause the impact?
	(a) Are there any omissions in the impact narrative?
(B) Reference and evidence use	(b) What type of references constitute the impact evidence?
	(b) What type of data evidences the impact?
	(b) Are there qualifications in relation to limitations?
(C) Impact language	(c) What nouns describe the claimed impact?
	(c) What verbs describe the claimed impact?
	(c) What context describes the claimed impact?

Source: authors.

shows these analytical foci, as well as the concrete reading prompts, we used to identify proxy indicators of impact quality.

Our methodology is illustrated in Figure 1. Practically, the first step (I) included printing the data material several times, with each exemplar corresponding to one type of reading prompt. Afterwards, each consecutive read focused only on one dimension. During the reading (II) each relevant section was highlighted on the page, followed by elaborations on how they related to the analytical foci. These initial comments represented the ‘protothemes’ for the summarization of the material. Afterwards (III), we created a table for each of the analytical foci cross-tabulated with an example of the most indicative quote/words from the original case study. This step was followed by (IV) removing these quotes and streamlining the table into coherent themes (V). The last step of the analysis represented the strategic selection of the most indicative quotes and words to be representative of the created category (VI).

In this paper, we analyze the construction of research impact claims. Therefore, we are neither endorsing nor criticizing any of the specific research impacts that we present (for a general commentary on this aspect see [39]). Ethically, we restrict our commentary to the presentation and its consequences, as this is the subject of our paper. Equally, the here presented information is informed by the first author actively working as a research impact officer, aiding scholars in the construction of their own research impact case studies for the REF 2021. Whilst no specific examples will be mentioned, all knowledge gathered over countless hours of drafting impact case studies, evaluating the suitability of impact evidence, and relating it to institutional requirements does inform the here presented results. We deem this to be ethical, as the here presented research is unfunded and represents the authors’ chosen way to criticize the working habits and processes that cannot be voiced in day-to-day operations [40].

4. Results

4.1. Argumentation Structure

Our first objective was to identify how scholars dealt with potential value conflicts in their presentation of research impact. A proxy indicator of quality within a research context is the very structure of the presented claim [41]. Unsurprisingly, the narrative mirrored the structure of the case study design and that of how a research claim ought to be presented. As the value dimension and its contentious nature are a complex subject [42], we chose to present one example in full here, highlighting in cursive the proxy indicators of research impact. We restrict ourselves to one example with consideration to word limits, as we believe it suffice as that level of detail would be otherwise excessive for several occurrences that largely make the same claim. The quotes below are extracts of the details of the impact case study from the University of Brighton (UoB). However, the narration style was common to all studied case studies. The case study usually started with broadly claiming the research impact; in the UoB case it started as follows: “The underpinning research and participatory methods have impacted on policies that aim to address the lack of tourism and hospitality human resources capacity, which is key to any nation’s successful tourism development” (UoB, authors’ emphasis).

The narration starts by affirming the causal impact of the research, alongside identifying a concern of “lack of […] resources capacity” that presumably the research impact addressed. Right from the start, the narration has rhetorically located the research as a mediator (i.e., causal link) in creating the impact. To testify if the impact occurred or not, any impact claim must establish in its narration a point of reference by which judgment becomes possible [43]. As such, the lack of human resources capacity must be outlined and then the narrative has to include a section that shows how these were fulfilled (or not) in order to manifest the integrity of the claim. This outlining is akin to how proxy indicators of research quality discipline the creation of research facts [44].

The case study of UoB continues in the following fashion: “The World Bank commissioned research into education, capacity building and training in Gambia, which led to a re-designed education and training policy. The research influenced the decision of the Spanish government to fund The Gambia Tourism and Hospitality Institute (GTHI), a national centre of excellence for tourism and hospitality education. The research provided: a feasibility assessment (2009) and business plan (2011) to direct policy, later implemented through ‘The Gambia Tourism and Hospitality Institute Bill’, which sought to create an enabling environment for Gambians to study up to the level of a Higher National Diploma in travel, tourism and hospitality. This bill was approved by the National Assembly in 2011 (source 5.8). Drawing upon the University of Brighton’s recommendations, the GTHI was inaugurated in 2013 after an investment of €2.7 m. The GTHI aims to train an average of 200 school leavers per year and upgrade the level of professional training amongst the 30,000 workers in tourism and hospitality, a sector that contributes 16% to the national GDP” (UoB, authors’ emphasis).

In this section, the narration further qualifies “lack of […] resource capacity”, but we do not learn any specifics of what capacities have been lacking other than “education, capacity building and training.” The submitting institution does not elaborate further on what type of training they are inferring in relation to tourism (e.g., catering, customer service, staff management, etc.). In regard to evaluating if these impacts have been achieved, we learn that there has been a “feasibility assessment” and a “plan” that “aims” (implying some future point) to improve a current important economic sector, but yet again there is no elaboration on whether these changes actually have been achieved. Instead, the evidence for the achieved impact invoked serve as a proxy indicator of impact quality. Hence, due to the word limit restrictions, the evidence supposedly speaks to the fact that impacts manifested.

The presentation continued to describe the research impact, qualifying it in the following fashion: “UNESCO-funded research in Nigeria led to a new national curriculum for leisure, tourism and hospitality workforce training. As part of a larger project aimed at revitalising Nigeria’s Technical and Vocational Education and Training (TVET), in collaboration with the Nigeria Board of Technical Education (NBTE), NOVELLI produced a ‘Leisure, Tourism and Hospitality Curriculum Review’ (2004), followed by a ‘Capacity Building/Train-the-Trainers Programme’ (NOVELLI and BURNS 2009). This led to the adoption of new industry and employment-centred leisure, tourism and hospitality teaching materials and a new national curriculum replacing the previous redundant one, which dated back to colonial times. An evaluation of the material and curriculum undertaken by the NBTE concluded that the impacts had spread beyond Nigeria and that: ‘The tangible outcomes of the initiative have been accepted for adoption in the ECOWAS countries… Other counties outside the region, e.g., Libya, Bahrain, Ethiopia, etc. have also benefitted from its achievement and have used the curricula developed as part of their own national curricula.’ (5.6)” (UoB, authors’ emphasis).

In this section, we learn that some form of education received a “new national curriculum”. However, there is no information about the success of the intervention. The only available reference states that it has “also benefitted”, thus yet again failing to provide specifics to what/whom these benefits/beneficiaries were. Instead, authoritative institutions invoked—like the UNESCO, the World Bank, and national governments—serve to justify the claim. Their authority and subsequent integrity by default become the de facto proxy indicator for impact. The case study also mentions “unsustainable tourism practices” that have caused “irreparable damage to fragile ecosystems and significant economic losses”; however, they fail to mention what damages or losses had occurred and how, only implying that tourism caused this damage. Likewise, other case studies also use authoritative sources that supposedly corroborated the legitimacy of the claimed problem formulation and stated impact.

Albeit the focus of research evaluation was on presenting excellent research impact interpreted according to their reach and significance, addressing quality issues or conflicts of interest were absent in any of the analyzed impact case studies. Rather than describing and elaborating on the content, the use of references and their associated authority constituted the role of proxy indicators of impact quality.

4.2. Challenges in Evidencing

Before we examine the evidence in detail, a note on limitations is necessary in that the referenced testimonials were not publicly available. As such, these references do not form part of our analysis. Nevertheless, based on the narration alone, it was difficult to ascertain the significance and reach of a specific impact in question. As with the University of Brighton example, we here only present a selection (the University of Surrey, University of Kent, and University of Bedfordshire) to manifest the issues that were emblematic of the problems with the evidencing of impact claims. We will now present the University of Surrey’s own summary of their research impact, alongside the references given within the case studies (

Table 3. Impact summary and references of one of the University of Surrey’s impact case studies.

Impact Summary:

Research at the University of Surrey has assisted disabled people and low-income groups to access tourism, a significant non-material aspect of well-being. This was achieved by influencing policy and policy recommendations in the UK, Belgium, and the EU and by influencing behavior, action and policy of either demand or supply: Demand: Increasing information and support options by establishing ‘Travel Support Points’, exchange schemes and travel facilitating websites Supply: Supporting tourism businesses by establishing accessibility tourism networks and influencing the biggest social tourism provider in Wallonia (Belgium) to extend existing inclusion measures, and introduce new initiatives

References:

For impact 4.1

(C1) European website on accessible travel: www.europeforall.com (C2) ‘Tourism for All—UK’ (Provided statement) (C3) FETE final report http://www.slideshare.net/toerismevlaanderen/evaluatierapport-first-european-travel-experience

For impact 4.2

(C4) http://www.visitoslo.com/en/your-oslo/oslo-for-all/project-background/ (C5) Knowledge-based network for SMEs: http://ceta.enat.be/ (C6) VACA Tourism—(Provided statement)

For impact 4.3

(C7) EC Document: Commission of the European Communities (2008) Regulation of the European Parliament and of the Council concerning the rights of passengers when travelling by sea and inland waterways and amending regulation (EC) No 2006/2004 on cooperation between national authorities responsible for the enforcement of consumer protection laws, COM(2008)816 final; SEC(2008) 2950: http://eur-lex.europa.eu/LexUriServ/LexUriServ.do?uri=SEC:2008:2951:FIN:EN:PDF (C8) Press release: http://www.ttrweekly.com/site/2011/12/ministry-to-help-disabled-tourists/—“Ministry to help disabled tourists” as further evidence that OSSATE demand study (‘Accessibility Market and Stakeholder Analysis’) has influenced their decision to initiate this project on accessibility. (C9) All Party Parliamentary Group (APPG) Report: Giving Britain a Break http://d3n8a8pro7vhmx.cloudfront.net/appgonsocialtourism/pages/29/attachments/original/111031GivingBritainaBreak.pdf?1320318284 (C10) Tourism Flanders (Provided statement)

Source: https://results.ref.ac.uk/(S(hcl3o4izligaahltmwe41tx4))/Submissions/Impact/1475.

). The intention is to highlight the differences in how institutions choose to present their impact references, i.e., highlight what they deem an acceptable proxy indicator of impact quality.

The report continued, that due to this contingency more research efforts needed to be in place to facilitate the development of such links. Otherwise, the made progress to develop “system[s] for post-pilot implementation [… in order to exploit] untapped potential in the commercial and social economy would not be realized” (University of Surrey, C3: 5). As such, the report evoked that it has failed to fulfil its main objectives, yet it still presented to show that an impact was made. For impact 4.2, the first reference (C4) takes one to an inaccessible page (see Figure 2). The website (visitoslo.com) does still exist, but only contains links to external sites and not the referenced project description. The next webpage reference (C5) redirects to a website where the domain is available for sale. For impact 4.3, the first reference (C7) is a policy document that references the research implementation into policy but says nothing about the outcome of the intervention. The next reference (C8) leads to a press release; however, when attempting to access the website, the page could not be found. The last two references are statements by two interest groups that are actively lobbying for the inclusion of social tourism.

Just to stress that the above described issue was not germane to only one case study. The University of Kent case study faced similar issues. As indicated earlier, when the submitting researcher is the same person providing the references, they are in essence claiming that they themselves did have an impact. Out of the seven references evoked for the impact, four were by the researcher who made the claim. Furthermore, by referencing the researchers themselves as evidence for the claimed impact, potential conflicts of interest may arise. Here we reveal the value dimension inherent in both accounting and judging research impact. The activity of lobbying is mentioned in the case study I the sense that the key-researcher personally advocated for policy change and participated in several policy workshops that were co-organized with an advocacy group (Swisscontact) regarding backpacking in Malaysia. The aim of these workshops was to influence policy and “raise awareness in order to help vulnerable states formulate effective policies and develop appropriate tourism initiatives.” As the key-researcher was actively involved in such advocacy the distinction between lobbying and policy advice becomes somewhat fluid. The full implications of such difficulties in accounting of research impact are absent from the narrative (either in this or in other case studies).

In our last example of the University of Bedfordshire’s tourism impact claim, the issue with using web links to evidence impact resurfaces. As shown in Figure 3, when accessing the websites issues arose, similar as with the ones mentioned before. Granted, these websites may have existed at point of writing, nevertheless, the fact that they are no longer accessible a mere two years after the end of the assessment is an indication towards the scale of the claimed impact. The claims to sustainable tourism impact represented an account of “research funded by the Esmée Fairbairn Foundation during 2009–2011 into how food tourism can be used for sustainable development.” Hence, once again we can identify issues around lobbying used to influence policy makers to shape tourism within Wales and Northern Ireland. Regardless, of the deletion of the websites, the evaluation score and the associated funding is more permanent than some of the evidence base that were the basis of their decisions. Equally, just because single project was canceled/runs its course, this does not mean that impact did not occur. Although, it might be an impact that is difficult to evidence within the narrow window of assessment criteria. The above example manifests, that even when a presentation is according to standards of proxy indicators of research quality, to the quality of impact can remain questionable.

4.3. The Language of Sustainable Tourism Impact

Our last objective was to analyze the very language of research impact. In general, the case studies always invoked needs referenced with authoritative and reputable institutions whilst positioning their impact as a solution. No case study included qualification on the specific problems/limitations of the involved claims, nor how the writer identified the presented causality (i.e., methodology). Even ‘nonimpact’ were claimed as research impacts, using creative wording. An extract from the University of Surrey’s other case study shows this contingency: “Finally, the application of the methodologies developed has allowed the Surrey team to assess the demand for accessible tourism across Europe, responding to the European Parliament’s Preparatory Action ‘Tourism Accessibility for All’. While the impact of this project is yet to be felt, the impact of the methodological work described above is in being able to bring more advanced understanding to other areas of tourism demand.” (University of Surrey, authors’ emphasis)

Table 4. The identified verbs, nouns, and context of proxy indicators of impact quality.

Verb of Inferred Impact	Noun of Inferred Impact	Context of Inferred Impact
Assist	(Intangible) aspects	Benefits of cross border activity
Challenge	Advancements	Charity practices
Change	Approaches	Cultural identity and distinctiveness
Contribute	Awareness	Evidence based policy/strategy
Develop	Capacity	Human resource management
Draw upon	Changes	Inclusion of disadvantaged/disabled/low income people
Encourage	Decisions	Information Communication Technology
Enhance	Demand	Marketing and positioning strategies
Enhance	Development	Minimizing undesirable environmental impacts
Extend	(Multiplier) effects	Niche tourism
Identify	Failures	(Tourism) policy
Improve	(Scientific) foundation	Product development
Inform	(Economic) growth	Regional and local economies
Integrate	Impacts	Small island developing states
Introduce	Initiatives	Tourism demand
Optimize	Innovations	Tourism Impact Model
Plan	(Economic) linkages	Tourism’s socioeconomic impact
Predict	Measures	Benefits of cross border activity
Provide	Platform	Charity practices
Quantify	Practices	Cultural identity and distinctiveness
Raise	Recommendations	Evidence based policy/strategy
Strengthen	Reduction
Support	Relationship
Underpin	Risks
Understand	Strategy
Use	Supply
	Understanding
	(Conventional) wisdom
	Well-being

Source: authors.

represents an analysis of the verbs, nouns, and context of the impact claim used by the tourism studies faculties that reported sustainable tourism research impact to the REF 2014 impact assessment. This selection was based on the ‘narrative box’ of ‘summary of the impact’ and thereby departs from the research impact provided by the University of Nottingham, Cardiff University, University of Kent, University of Sunderland, University of Bedfordshire, University of Surrey, University of Brighton, and Bournemouth University, who all claimed sustainable tourism research impact between 2007 and 2014 [8].

As with our analysis of the narration structure, the used language is light on the causality of the impact claims, which means that the narrative guidelines of the REF may restrict research impact claims that can be made, as the requirements posed by the assessment structure (page limit, evidence structure, narration style, time frame, etc.) limits the type of impact that can be claimed. Equally, lack of methodological information of how the claims were construed (e.g., sample size, data quality, research methods, conflicts of interest, research limitations, etc.) turns the presentation into a volley of colorful jabs rather than causally robust claims. Thereby, the nature of these case studies as boundary objects is revealed. Case studies as boundary objects “are both plastic enough to adapt to local needs and constraints of the several parties employing them, yet robust enough to maintain a common identity across sites.” [45] (p. 393). Specifically, researchers must balance the interests of their institutions, what claims are evidencable, word limit requirements, stakeholder interest, etc.

Within a research context, addressing such ethical and methodological issues usually are part of the methodology or method. We argue that the lack of a method section for the impact case studies is suboptimal for presenting convincing claims around what really makes tourism research impacts sustainable. The REF acknowledges this limitation of the case study evaluation approach when they state: “Within their narrative account in the case study, institutions should provide the indicators and evidence most appropriate to the impact(s) claimed, and to support that chain. The subpanels will use their expert judgement regarding the integrity, coherence and clarity of the narrative of each case study but will expect that the key claims made in the narrative to be supported by evidence and indicators” [20] (p. 71; authors’ emphasis).

In general, the language of the studied cases was often very vague, which may be indicative of the underlying uncertainty involved in retracing the causality of the claimed impact [46]. Although the impact template did impose some measures of discipline, it still seems that the restrictions left plenty of room for individual interpretation of how the universities wanted to claim their research impact. Arguably, there is only a limited amount of detail presentable in a four-page document. Hence, it seems it is ‘easier’ to give quantifiable evidence (visitor figures, revenue figures, or number of book sales, etc.) compared to evidencing qualitative research impact (e.g., change of cultural values, improvement in quality of life, or changes in life philosophy, etc.). Such restrictions presumably are the reasons why many of the submitting universities chose to evidence their impact along these superficial grounds. However, given that the narrations are vague, implicit, and esoteric, it is very difficult to make a competent judgment if the presented impact was indeed impactful or sustainable. It does not bode well for the future of sustainable tourism if the current evaluation practices are to continue to utilize REF outcomes as indicators of best practice. Especially, if we want to consider and judge what “good” sustainable research impact is [47].

5. Discussion

Departing from our findings, we want to discuss the implications for the integrity of the sustainability debate in a broader sense. As we implied in the introduction, the positionality of the researcher never disappears, even if they are absent within the manifested rhetoric of the presented claims. Within the marketing of tourism destinations, the notion of ‘greenwashing’ has become a commonplace practice, and it is widely acknowledged that, “using false green claims harms attitudes toward ads and brands and should generally be avoided” [48] (p. 141). This means that the damage of doubtful claims is not only specific of the claimant but has general effects. Equally, when tourism researchers engage in furthering sustainable tourism, they not only have to be vigilant of defamatory lawsuits [49], but also of the incentives of their very own research collaborators. Taking inspiration from the tourism literature on how to establish green trust [50], we argue that researchers have to take responsibility for two types of behaviors: firstly, the general image created by others’ actions, and, secondly, the impression based on one’s own actions.

Why is that important? The significance of our research findings is that the demarcation between content and form of presentation are not clear-cut. The inference of causality is purely based on the language, references, and argumentation structure of the information presented within the impact claims. Hence, differentiating if the activity is “good” in furthering sustainable tourism or sustainable development in general becomes a matter of superficially trusting the claims of the authors. Such institutionalization of the value of research impact, removes all the complicated ethical concerns of judging of what makes tourism sustainable in the first place. An obvious limitation of the impact case studies, is that they do not include any section that specifically demands methodological information of how the presented information was constructed or what potential ethical conflicts there are for the researcher or stakeholders mentioned.

Research arguably improves the tourism industry and tourism as an activity through the insights that researchers share both publicly and, in their consultancy [51]. Based on our analysis, we deem it important that researchers also scrutinize the use of language imbedded in the seemingly neutral proxy indicators of impact of other scholars, as suggested in this paper. We argue that issues that can raise suspicions may not be maliciously motivated but emanate from internal research pressures of how modern research assessment works and from the institutional reward systems [52]. This is important, given that questionable claims of promoting sustainable tourism utilize the same nomenclature, type of references, and argumentation structure as genuine impact claims. Therefore, promoting sustainable tourism may not only deal with unmasking questionable business practices, but also the naïve conduct of other researchers of how their own actions relate to institutional compliance [53].

Performance-based research funding has the explicit goal that the subject of study ought to benefit the society [54]. Despite this assumption, it is not always possible to determine if this is genuinely the case, and our analysis confirms that. The charitable interpretation is that identifying the influence of research to further sustainability is difficult [55]. The less charitable interpretation is not something that we want to go into in this paper. What we do want to stress, is that the only way consumers can retain trust in academic analyses of the state of sustainability of various tourism practices is that researchers need to lead by example (for a literature review of how to avoid greenwashing, to take inspiration from, see Griese et al. [56]). More specifically, this means identifying which proxy indicators of research impact are viable enough to stand and on their own merits. Furthermore, the limitations of the presented information should always be included and made explicit. This may come with a lower score with the research assessment [57] but the collective effect of impending loss of trust in academic integrity seems far worse. The individual consumer does not always possess the acumen to question the veracity of claims of sustainable tourism impact.

The public, as a collective of individuals, relies on expert judgments. This pertains to the domain of sustainable tourism as well. This leads to a dilemma. How can we be sure that the expert knowledge remains construed by the best research available, and not merely by the necessity to live up to certain imposed standards that favor ticking boxes, language games, and references to disappearing websites? While the here studied communication patterns may be sufficient for research evaluation, we caution that applying them uncritically outside of this narrow context may turn outright detrimental to the development of knowledge on sustainable tourism. As such, whilst tourism as a research discipline, may still be young [58] it seems that it is in need of discipline from within, in order not to get swept away by larger disciplinary forces that unintentionally can undermine its legitimacy. Luckily, other disciplines have identified similar issues, from which tourism researchers can take inspiration (e.g., [59,60,61]).

6. Conclusions

The contribution of this article is to outline the issues around proxy indicators of impact quality within the context of sustainable tourism. We found that if scholars want to engage in the impact realm, meaning promoting the social benefits of their research they need to tread carefully not to jeopardize their own integrity. Obviously, our research findings are limited by the contextual setting their stem from (UK’s REF setting, impact template design, selection of self-identified tourism researchers only, etc.). Nevertheless, we would argue that any form of research evaluation of ‘impact’ will have to deal with some of the here identified issues, as it pertains to the basic requirement of communication.

We found that the narrative structure framing the research impact is set up in a fashion that incentivizes proxy indicators, relying on the acumen of powerful institutions, rather than on the merit of their own claims. Equally, having analyzed the evidence in detail, more questions arose as were answered in how the research supposedly furthered sustainable tourism. All that remained was the rhetorical inference of causality made in the actual description, barring any available methodological information of how the research impact claims were constructed. We conclude that the methodological rigor ought to present the highest value when making claims of impact in order not to jeopardize the validity of claims of sustainable tourism. A concrete recommendation would be to include methodological rigor and briefly mention in the text how the researcher methodologically identified the claimed statements about sustainable tourism research impacts in order not to jeopardize research integrity. Otherwise, akin to the practice of greenwashing, researchers that make claims about sustainable tourism may find their claims lacking in a general background of skepticism.

We argue that in order to maintain research integrity, future tourism researchers ought to use the language of proxy indicators of impact and follow the presentation structure, but they should also be aware of the methodological pitfalls that come with evidencing long-term sustainable impacts. Thereby, tourism researchers should not only proactively raise and address any potential conflicts of interest, but also strive to introduce their own methods of how they arrived at their claims as part of the indicators for research impact in order to shift from proxy to real indicators of impact quality. If this is not done, that is, if the claims are unverifiable and reek of vested interest, the very process of proclaiming that tourism researchers make sustainable tourism impact may turn outright unsustainable.

An obvious limitation of our approach is due to the very structure of the assessed impact case study templates. Namely, they are lacking a ‘method section’, hence any evaluator lacks information to how the presented impact claim was constructed. Even if the who, what, and when of the is claim is present within the narrative, such methodological information is crucial in ascertaining the veracity and plausibility of the encountered claim. Practically, the absence of a method section that elaborates what methodology identified the presented impact claims, leaves all inference of causality by default on the use of the nomenclature and narrative construction. Hence, the assessment of reach and significance of an impact claim is done without any information as towards potential conflicts of interest, sample sizes, or analytical limitations to only mention a few. An additional limitation, we specifically encountered here, was that we did not have access to the referenced testimonials for our scrutiny, as this information was not available in the public domain. Nevertheless, given the here identified issues we deem it unlikely that the account of one or three individuals would change the overall picture drastically (the maximum number of used testimonials for REF 2014 was three). At best, they would serve as an additional authoritative source to corroborate a specific sub claim in the fashion we already commented upon previously.

In relation to future research, all these limitations represent potential fruitful areas of inquiry, where a potential future research project might be following an impact case study from its process of creation, through the editing, submission, assessment, and finally feedback and implementations. As of now, there are some accounts that investigate the construction of such impact claims; for example, accounts that investigate the construction of research impact case studies [62], limitation in its evaluation [63] or a longitudinal account of how the more subdued notions of research impact manifest [64]. Nevertheless, such reflective accounts—as of now—are limited to the idiosyncratic nature of the authors reflecting upon their own experiences. Hence, why we launched a more structured research program to investigate the influence of impact more holistically and systematically [19].