The Impact of Sustainability on Co-Creation of Digital Public Services

Abstract

Co-creation focuses on engagement with citizens and other stakeholders with public administrations to develop innovative processes or public services. The integration of resources and knowledge mediated by technology can lead to the emergence of novel outcomes (such as products, services, processes, and social practices), but it is necessary to understand the mechanisms that lead to sustainable co-created innovation and outcomes. The aim of this research article is to contribute a more nuanced understanding of the impact of sustainability on co-creation of digital public services. To study co-creation and sustainability in the context of the public administrations in depth, a qualitative research approach was used. The data were collected through moderated discussions conducted during a workshop held with 20 experts and practitioners in the field of public sector digital transformation, during which participants shared their views and experiences in a free-flowing conversation. A systematic two-cycle analysis approach based on Grounded Theory was used to code and analyse the data collected. Results from this study focus on developing a common understanding of sustainability within the context of co-created digital public services, an exploration of how sustainability in public administrations may be supported by co-creation and a critical examination of the elements that contribute to the sustainability of digital public services—all drawing on existing examples of co-creation initiatives from within the public sector.

1. Introduction

In the private sector, sustainability can be defined as the “planned and strategic management processes involved in working toward a balance of economic, social, and environmental goals and values” (Thomsen 2013, p. 4). Not only is multidimensionality of sustainability particularly challenging, but until recently, the focus of private sector organizations was on economic sustainability (Cusumano and Vecchi 2022). Sustainability in the context of the public sector is often associated with climate change and achieving environmental goals, such as the Paris Agreement (United Nations 2015a) or the 2030 Agenda for Sustainable Development (United Nations 2015b). The 2030 Agenda for Sustainable Development points out that sustainable development includes public access to information and public participation, engagement by various actors, identifying technology needs and gaps, innovation and capacity-building to help facilitate technology development and dissemination. Digital transformation in the public sector is also aligned with environmental sustainability, as evidenced in the “Draft Council conclusions on Digitalisation for the Benefit of the Environment” (Council of the European Union 2020b) and in the “2030 Digital Compass: the European way for the Digital Decade” (European Commission 2021). Both documents focus on involving and committing stakeholders to sustainable development in the context of public sector digital transformation. Not only is sustainability broadly associated with the environment, climate change and activism, but its indicators and impacts are often intangible and cannot be rendered in monetary terms (Misuraca et al. 2021).

Previous reforms such as New Public Management (NPM) supported the inclusion of business practices in the public sector, but citizens were seen as customers rather than contributors to public service design and delivery. Osborne et al. (2014) suggest that public service-dominant logic (PSDL) helps public administrations facing important challenges regarding the provision of sustainable public services. On the one hand, users want accessible, user-friendly, personalised public services that match their needs and circumstances, whilst on the other, public administrations are to develop and provide better services that meet users’ need using fewer resources. Co-creation represents an arrangement between public administration professionals and lay citizens/communities in order to develop more efficient services (Brandsen and Honingh 2016) and a more sustainable public service provision. The co-creation of public services can therefore be defined as “a process through which two or more public and private actors […] solve a shared problem, challenge, or task through a constructive exchange of different kinds of knowledge, resources, competences, and ideas that enhance the production of public value […], or services, either through a continuous improvement of outputs or outcomes or through innovative step-changes that transform the understanding of the problem or task at hand and lead to new ways of solving it”. (Torfing et al. 2019, p. 802). In co-creation practices, citizens can contribute to the different phases on the public service, such as its design, delivery and monitoring (Linders 2012; Scupola and Mergel 2022). Torfing et al. (2019) add that digital and Information and Communication Tools (ICTs) are used to enhance the collaboration and innovation potential of co-creation. The co-creation of public services focuses on the use of digital technologies for citizens to engage and provide inputs for the development of digital or fully automated services (Lember et al. 2019), that is, on the practice of co-creation leading to sustainable outcomes. Jaspers and Steen (2019) instead argue that co-creation strengthens the sustainability of collaborative practices between the stakeholders, including public administrations. This makes clear that sustainable co-creation is on the one hand the integration of resources and knowledge mediated by digital technology to achieve innovative outcomes (new products, services, processes, social, and environmental practices), and on the other, is understood as a sustainable organizational practice within the public administration (Barile et al. 2020). Ensuring sustainable and long-term co-creation processes and outcomes require stakeholder involvement, although such participation is complex, particularly when using digital tools and ICTs (Edelmann and Mergel 2021), so user engagement and ICTs are often the central factors analysed in co-creation activities.

The aim of this research paper is to contribute towards a more nuanced understanding of the impact of sustainability on the co-creation of digital public services. To do this, we seek to answer the following research question: What impact does sustainability have on the design and delivery of co-created digital public services? To better focus our enquiry, we investigate the following sub-questions:

(i)

How is sustainability understood in public administrations?

(ii)

How do co-creation practices contribute to sustainability in public administrations?

(iii)

What dimensions of sustainability impact co-created digital public services?

In order to study co-creation and to assess the sustainability of its practices and outcomes, several methods can be used, such as experiments, surveys, or longitudinal studies (Brandsen and Honingh 2016). A qualitative research approach was used to collect and analyse research data. These data was collected during a workshop held with experts and practitioners (Edelmann et al. 2021). A two-cycle analysis was then used to code to the data: during the first cycle, codes from the literature were used to organize data into thematic clusters; during the second cycle, a Grounded Theory approach was used to code the data collected and derive results. The results presented here focus on understanding the role of sustainability within the context of the public administrations and public service delivery, a nuanced understanding of how sustainability may be supported by processes of co-creation, and a critical examination of the elements that contribute to the sustainability of digital public services, all drawing on existing examples of co-creation initiatives from within the public sector.

The rest of this research article is structured as follows: Section 2 presents a review of key literature concerned with the digital co-creation of digital public services, and explores how sustainability is currently conceptualized in ecological, societal and public administration contexts. Section 3 describes the research design adopted for the purposes of this study, while Section 4 discusses the research methods used to collect research data. The research data analysis phase is presented in Section 5. Results are presented in the following section, Section 6, and critically analysed in Section 7. The paper concludes with a summary of the main findings and the outlook for future research.

2. Literature Review

Co-created public governance breaks with the view that the public sector can be the sole provider of public goods or that competition between public and private actors leads to better and cheaper public services (Torfing et al. 2019). It represents an arrangement between state professionals and lay citizens/communities in order to develop public solutions, increase innovation and develop more efficient services (Brandsen and Honingh 2016). Co-creation builds on the concept of collaborative governance, involves a broad range of internal and external actors and activities, has broad applications in the field of public governance, and focuses on strategic planning and policymaking in order to solve complex problems (Ansell and Torfing 2021).

Citizens can contribute to different phases of the public service cycle such as service design, service delivery and service monitoring (Linders 2012; Scupola and Mergel 2022) by contributing and creating knowledge (Jarke 2021). The co-creation of public services is therefore “a process through which two or more public and private actors […] solve a shared problem, challenge, or task through a constructive exchange of different kinds of knowledge, resources, competences, and ideas that enhance the production of public value […], or services, either through a continuous improvement of outputs or outcomes or through innovative step-changes that transform the understanding of the problem or task at hand and lead to new ways of solving it” (Torfing et al. 2019, p. 802). Brandsen and Honingh (2016) highlight that this means the need to understand the interaction between citizens and professionals as part of the co-creation process and the outcomes of co-creation.

Co-creation is demanding and challenging for both the public administrations and the stakeholders citizens involved, and the issues associated with co-creation may seem “impossible to resolve for those who suffer from a lack of tangible and intangible resources” (Jalonen et al. 2021, p. 800). Jalonen et al. (2021) argue that digitalisation of the public sector organisations can support co-creation even when the users to be involved are vulnerable or hard to reach. The use of digital technologies can therefore support online debate and collaboration, crowdsourcing, aggregating, and organizing the inputs for developing or creating new digital public services or processes (Edelmann and Mergel 2021). This makes digital technology either an “enabler of co-creation (e.g., knowledge or value co-creation enabled through digital platforms)” or “the goal of co-creation (e.g., co-creation of digital artefacts)” (Jarke 2021, p. 15).

Although the interest in the potential of digital technologies to enhance co-creation continues to grow, there is a lack of hard evidence on their actual impact because “conceptual fuzziness and tech-optimism stand in the way of collecting such evidence” (Lember et al. 2019, p. 1665). Lember et al. (2019) highlight the importance of considering the impact of digital technologies on different aspects of co-creation processes: on the direct interaction between citizens and public administration employees, how to motivate and interest stakeholders to take an active role, how external stakeholders bring their resources and expertise to the co-creation process, and how to help internal stakeholders share decision-making processes and power. As the integration of user resources and knowledge mediated by technology can lead to the emergence of innovation (new products, services, processes, social, and environmental practices), it is also necessary to understand the mechanisms that lead to innovation that is also sustainable (Barile et al. 2020). Barile et al. (2020) suggest that fostering the continuous generation of new knowledge and co-creation of value through ICT-enabled interactions can lead to service improvement and innovation which have economic (micro), social (meso), and environmental (macro) value. In addition, although co-creation is understood as strengthening collaborative practices, research on co-creation has been more concerned with identifying the drivers and barriers and the conditions for the emergence of co-creation, rather than measuring and assessing its impact (Jaspers and Steen 2019; Torfing et al. 2019; Verschuere et al. 2012; Voorberg et al. 2015).

Sustainability can be defined as the ability to uphold, support, and sustain something considered valuable (Türke 2012). According to Stephens et al. (2008) all challenges related to sustainability can be assigned to ecological, social and technological change. The private sector has considered issues of sustainability for several decades, although the focus has been primarily on the economic dimension than on the environmental or social dimensions. Carroll and Shabana (2010), for example, argue that senior staff should take corporate actions beyond the economic interests of the firm and that businesses’ activities should be more transparent in relation to society. Biggemann et al. (2014) point out that private sector organisations actions cannot foster sustainability unless the beneficiaries are included from the start.

The UN Agenda 2030 and Sustainable Development Goals (SDGs) currently shape global and European Union policy on sustainability (United Nations 2015b). The implementation of the SDGs includes the principles, knowledge, methods, and tools required for reaching the 17 SDGs, which cover a whole range of areas, from poverty eradication to health and peace. The Directorate for Sustainable Resources, one of the six scientific directorates of the European Commission’s Joint Research Centres (JRC), is to create, manage and make sense of scientific knowledge for EU policies related to the sustainable use of resources, encompassing environmental, economic and social dimensions (Koirala and Pradhan 2020). The Public Sector Sustainability Association (2023), a network that brings together experts who work towards a more sustainable future in the public sector, addresses issues related to reducing the carbon footprint, sustainable buildings, energy, transport and workplaces. Economic or social dimensions (Rutherford 1997), also known as human well-being (Hardi et al. 1997), education, long life and the standard of living are included in the Human Development Index (UNDP 2022). The Sustainable Society Index covers three dimensions of sustainability, i.e., social, environmental and economic, but does not integrate the dimensions into an overall index (Joint Research Centre et al. 2013). Whilst sustainable development can be understood as the “promotion of societal transformation processes by governments, market actors and civil society” (Lange et al. 2013, p. 405), in the public sector, sustainability is often associated with environmental issues and climate change, as detailed in the Paris Agreement (United Nations 2015a).

Sustainability is tightly interlinked with the digital transformation of the European public sector, as evidenced in several central documents. The Draft Council conclusions on Digitalisation for the Benefit of the Environment (United Nations 2015b) emphasizes the need to align digitalization with environmental interests, to harness digital technologies for better climate and environmental protection, to foster social justice, and improve opportunities for stakeholders participation. The 2030 Agenda for Sustainable Development (United Nations 2015b) points out that sustainable development includes public access to information, public participation and engagement, identifying technology needs and gaps, and supporting the development of innovative technology. The Tallinn Declaration on e-Government (Council of the European Union 2017) points out the need to consider sustainability in order to achieve UN Sustainable Development Goals (SDGs): “Foster resilience and sustainability by […] ensuring that the digital transformation in Europe contributes to the UN Sustainable Development Goals (SDGs) and making the digital transformation itself more sustainable regarding the consumption of energy resources” (p. 7). The Berlin Declaration on Digital Society and Value-Based Digital Government (Council of the European Union 2020a) also highlights sustainability as “[…] developing relevant policies to support a workplace culture that promotes a healthy and appropriate use of digital technologies and work-life balance, namely through co-creation and collaboration with the civil society” (pp. 14–15). In the European Commission (2020)’s document “Shaping Europe’s digital future”, digital solutions and data are to support an open, democratic and sustainable society. Thus, sustainability, in the context of digital transformation, involves identifying technology needs and gaps, supporting innovation, capacity-building, and technology development to contribute to the sustainable development of the environment and society.

The European Commission (2021) points out that digital policies should empower people and businesses, support the transition to a sustainable economy, focus on digital literacy, data infrastructure and data processing technologies. Advances in ICTs have opened opportunities to transform the relationship between public administrations, citizens and other stakeholders to contribute to the sustainable development goal to achieve better government. Achieving the digital transformation of business and public services, digital citizenship, principles and rights requires “involvement and commitment of the public and of all stakeholders” (European Commission 2021, p. 21). Janowski et al. (2018) Platform Governance for Sustainable Development framework is based on the different types of relationships between citizens or administration that lead to sustainable development. The framework comprises 15 types of relationships organized along four governance paradigms: relationships for bureaucratic governance, relationships for consumerist governance, relationships for participatory governance, relationships for platform governance. In particular, the participatory paradigm includes the development and implementation of policies across government departments and agencies, the delivery of services through networks of public, private and non-profit organizations, and processes and structures of public policy decision-making and management that engage people across the boundaries of public agencies, levels of government, and the public, private and civic spheres. This dimension, according to Janowski et al. (2018), is “enabled by advances in methods and applications of digital technology, is tapping into assets, resources and competencies that exist within government and across the society, organizing them into common development platforms and using them to orchestrate collective action and pursue collective goals” (p. 2). E-government, for example, uses digital technology and contributes to efficient resource management (Castro and Lopes 2021; Estevez and Janowski 2013).

One of the most popular indicators of sustainability is the adjusted net savings calculated by the World Bank, a measurement which focuses on the environmental dimension of sustainability as the “adjusted net saving [that] predicts future economic performance in a cross section of countries, provided that adjusted net saving takes into account the depletion of natural resources” (Koirala and Pradhan 2020, p. 40). In addition to sustainability often being associated with the environment, climate change and activism, or achieving SDGs, its impacts are often intangible and cannot be rendered in monetary terms (Misuraca et al. 2021) or may not always measure what they claim to (Lyytimäki et al. 2020). Public sector organisations performance measures often focus on the areas of cost efficiency and quality measures rather than those that focus on learning and growth measures, sustainability, environmental or social responsibility measures (Korhonen 2003). Zheng et al. (2019) suggest that sustainable development needs to be clearly defined and should include measures should include goals to be achieved, practices and solutions that are replicable across agencies, institutions and levels of government, technological infrastructure such as the tools and platforms that are attractive for stakeholders and support participation and human capacity building, but also regulatory mechanisms to protect data security and privacy issues. In addition to the measures most commonly used for sustainability measurement in the public sector (e.g., output and quantity), quality measures, diversity and economic activity can be used in order to assess the impact of sustainability on organisational activities (Adams et al. 2014). Adams et al. (2014) note that the implementation of information systems particularly “provides [the] opportunity to incorporate measures aligned with sustainability outcomes and provide reportable indicators” (p. 47). E-Government benchmarks (e.g., UN 2020) assess digitally-enabled services and modes of delivery, usage and satisfaction to improve service quality, development and delivery, measure whether services match user expectations and, also, to what extent they are perceived as useful and effective. Misuraca et al. (2021) suggest collecting more granular data, focusing on quality assessment, the demand side of digital government, quality of services and user satisfaction, and measuring outcomes and impacts.

Ansell and Torfing (2021) argue that by defining co-creation as a “a type of collaborative governance that seeks to leverage distributed innovation and bring together public and social innovation for the proactive purpose of creating public value” (p. 219) makes it necessary to understand how the use of resources for co-creation processes leads to sustainable outcomes. In terms of sustainability, co-creation is to lead to valued sustainable public outcomes such as better service delivery, more efficiency (making better use of resources), being more responsive, and being more inclusive. At the same time, co-creation processes in public administrations need to be sustainable rather than just single or one-off efforts (Jaspers and Steen 2019), in order to “establish causal connections between institutional design, participation, collaboration, learning, innovation, implementation and the production of desirable outcomes” (p. 225).

Thus, when trying to study the sustainability of co-creation processes and outcomes, we identify several the following research gap: definitions of sustainability in the public sector literature focus mainly on sustainable development and there is a lack of research on the impact of digital technologies in sustainable co-creation processes and outcomes. The aim of this research paper is to contribute towards a more nuanced understanding of sustainability in the public sector and its impact on the co-creation of digital public services. To do this, we seek to answer the following research question: What impact does sustainability have on the design and delivery of co-created digital public services? To better focus our enquiry, we investigate the following sub-questions:

(i)

How is sustainability understood in public administrations?

(ii)

How do co-creation practices contribute to sustainability in the public administrations?

(iii)

What dimensions of sustainability impact co-created digital public services?

Collecting relevant data can be difficult as co-creation is gaining popularity but there are still only few examples of its implementation in the public sector, and public sector organisations will often provide information regarding organisational sustainability to internal stakeholders only (Bellringer et al. 2011). We therefore used a qualitative approach to analyse the data gained from a workshop on this topic attended by academic and practitioner experts.

3. Research Design

In order to answer the stated research questions, we chose to adopt Grounded Theory as a systematic research approach to examine the concept of sustainability in digital public service provision, and how co-creation can help ensure the development of sustainable digital public services (Urquhart 2012). Grounded Theory has been described as being “… an inductive approach to research, in which hypotheses and theories are generated from the data collected” (Engward 2013, p. 37), entailing a systematic collection and analysis of data to generate theory about human sensemaking and behaviour in social science contexts. In other words, a Grounded Theory approach allows the researcher to think about data with the intent to conceptualise it, allowing a practical and flexible way of interpreting complex social phenomena (Charmaz 2009).

The premise of Grounded Theory is that research is led and guided by the experiences of people in the empirical inquiry, and that social phenomena are explored through findings that reflect patterns in these experiences (Engward 2013). This is important to understand when using Grounded Theory as a research approach, for it implies that the researcher must set aside any preconceived ideas and expectations, and not attempt to predetermine what they will find or which social phenomena will eventually be described (Urquhart 2012). Three basic tenets differentiate Grounded Theory from other research approaches (Denscombe 2014): theory generation (it is an approach dedicated to generating theories); the grounding of theory in empirical research (it places emphasis on fieldwork and/or practical research); and systematic data collection and analysis (it focuses on constant comparative analysis and theoretical sampling).

In this research paper, we follow a simplified version of the Grounded Theory research framework developed by Chun Tie et al. (2019) to identify how informants understand the concept of sustainability within the context of digital public service design and delivery, and how they see processes of co-creation contributing to the sustainability of digital public services. Grounded Theory research is iterative and recursive, rather than being linear, and involves “… the meticulous application of specific methods and processes” (Chun Tie et al. 2019, p. 3). It begins with purposive sampling, whereby a researcher purposively selects participants and/or sources of data that can directly answer the research question. This gives way to data generation and/or data collection, wherein the researcher uses data collection methods and tools to obtain primary data. Once data are collected, an initial coding is carried out to segment the data and to identify emergent themes and patterns. At this stage, the researcher identifies, labels and groups as many codes as possible. Intermediate coding builds on the initial coding phase, transforming basic data units into more abstract conceptual blocks that will form the basis of any emergent theory. The penultimate stage in the process is advanced coding, wherein conceptual blocks are presented as a set of interrelated concepts signifying the emergence of a narrative. The continuous rigorous refinement of each narrative produces one or more integrated, comprehensive Grounded Theory that explains a process relating to a particular phenomenon.

4. Methods

In order to collect and analyse the data, a two-part qualitative research approach was used. Primary research data were initially collected during a workshop conducted at a major international conference under the aegis of the European H2020 “Inclusive Governance Models and ICT Tools for Integrated Public Service Co-Creation and Provision” (inGOV) project, following which a two-cycle analysis was applied to code the data and derive results.

4.1. Context of the Project

Today public authorities need to provide better and more sustainable public services whilst expending fewer resources. Citizens require accessible, user-friendly, personalised, and integrated public services that match their needs and circumstances. At the same time, trust in the public sector has declined over the years and needs to be renewed. The H2020 “Inclusive Governance Models and ICT Tools for Integrated Public Service Co-Creation and Provision” (inGOV) project is ambitious research undertaking that aims to develop and deploy a comprehensive Integrated Public Service (IPS) holistic framework and Information and Communication Technology (ICT) mobile tools and applications that will support Integrated Public Service (IPS) co-creation and governance. Where required, the project will also enhance, and possibly re-design, existing European Union solutions, frameworks and core vocabularies. InGOV is a pan-European collaboration between research institutes, the private sector and government agencies in Greece, Italy, Austria, Croatia, and Malta. The project results will be piloted in Malta to modernise the digital family household public service (affecting 200,000 households), in Austria to deploy IPS for collecting tourism tax (affecting 3200 accommodation providers), in Greece to digitise the disabled card renewal service (benefiting 11,500 disabled, low-income citizens) and in Croatia to create AI-driven virtual assistants and services (affecting 32,000 citizens). It is envisioned that project results will be fed back to the European Commission to contribute towards policy outcomes at the European level.

4.2. The Workshop as a Research Approach

It may be argued that a lot can be learned from using workshops as a research approach in various organisational and research domain contexts. Workshops are increasingly being used in different disciplines by researchers to gather qualitative date from a group of interacting participants by facilitating a guided discussion amongst them around a given topic (Storvang et al. 2018). The workshop method, according to Storvang et al. (2018), is primarily related to, but distinct from, other more action-oriented methods such as focus groups, action research, action learning, and participatory design research. According to (Inns 2013), different workshops share common characteristics as “… the hosts bring networks of participants together from very different disciplinary backgrounds. Participants in each event have a common interest, and are all motivated to develop solutions that deliver impact” (p. 42). A formalised definition of a workshop is given by Hamilton (2016): “… a collaborative working session in which a team achieves an agreed goal together. The goal could be to solve a problem, create ideas, work through an issue or find agreement between team members” (p. 1). This definition is further nuanced by Ørngreen and Levinsen (2017), for whom the term workshop refers to “… an arrangement whereby a group of people learn, acquire new knowledge, perform creative problem-solving, or innovate in relation to a domain-specific issue” (p. 70).

From these sources, it is evident that the emphasis of a workshop is, therefore, at once on creativity through interpersonal interactions, as well as on structure, wherein the environment within which these interactions occur is set up to achieve a specific goal, solve a problem or create a desired output. This makes the research approach more desirable than a focus group. (Ørngreen and Levinsen 2017) identify three distinct types of workshops in the literature: workshops as a means, or those workshops that purport to fulfil a particular goal such as the formulation of guidelines; workshops as practice, that focus on defining the form and outcomes of the workshop in question; and workshops as a research methodology, that aim to fulfil a research purpose and generate research data. According to the authors, all three types of workshops share common elements: workshops are of a limited duration, conducted by people with experience in a given domain, and target a small participant group whose constituents share the same common domain or agenda.

Storvang et al. (2018) describe three main stakeholders involved in a workshop: the researcher, or overall designer of the process to collect research data; the facilitator, or individual whose task it is to engage and motivate participants during a discussion; and the participants, or those individuals with a stake or interest in the process’ purpose and re-search problem. In practice, the boundaries of these three roles are not well defined, and the same person can hold several of them simultaneously. For instance, Storvang et al. (2018) argue that the researcher can sometimes facilitate a workshop while simultaneously collecting data, or choose to let someone else be in charge of facilitating the process. They can either be an active participant in the discussion, or a more neutral observer. Ahmed and Asraf (2018) mention several advantages of facilitated research workshops based on evidence from the literature. They point out that workshops foster engagement through collaborative discussion and constructive feedback. Workshops also promote learning amongst participants, again through collaboration, and allow for persistent observation or deep scrutiny of a given topic. They also allow for participant observation leading to the building of rapport between the researcher and the other participants. And finally, they usually lead to the generation of large quantities of rich data on a particular topic.

4.3. Primary Data Collection: Workshop

Primary research data was collected during a facilitated research workshop held with experts and practitioners (Edelmann et al. 2021). Twenty experts in the field of public sector digital transformation, representing either academia or the public sector, with backgrounds ranging from informatics to administrative sciences, political science, sociology, organisational psychology and business management, participated in the workshop. Experts were selected through a combination of purposive and random sampling: some attended the workshop by invitation, and others voluntarily attended without prior notice to the organisers. Guided by a semi-structured protocol, five moderators from the inGOV project conducted the discussion. The design can be considered as a mixed structured and unstructured approach as three overarching questions and six investigative questions were discussed over the course of a three-hour period, with participants allowed the flexibility to share their views and experiences in a free-flowing conversation. Discussions were held in English, recorded using hand-held devices, and transcribed by a professional transcription service. In accordance with the standard view of informed consent (Millum and Bromwich 2021), participants were required to sign a detailed consent form before the start of the workshop. The consent form fulfilled the understanding requirement of informed consent by providing participants with a detailed description of the workshop, the process, the types of data being collected, methods of data collection, and procedures for data handling and storage. The final page of the form required the participant to provide their personal details and signature, indicating their understanding of the document and willingness to participate in the workshop, thereby fulfilling the disclosure requirement of informed consent.

4.4. Data Coding

Coding is an oft-used way of organising and understanding qualitative data. A two-cycle analysis was used to code to the data obtained. Saldaña (2016) divides coding into two major stages: First Cycle and Second Cycle coding. First Cycle coding has been described as an initial way to summarise segments of data (Miles et al. 2018). During this phase, codes are initially assigned to the data units through a combination of techniques drawn from either four elemental methods (descriptive, in vivo, process, and concept coding), three affective methods (emotion, values, and evaluation coding), one literary or language method (dramaturgical coding), three exploratory methods (holistic, provisional, and hypothesis coding), two procedural methods (protocol and causation coding), four grammatical methods (attribute coding, magnitude coding, subcoding, and simultaneous coding), or the stand-alone method of theming the data. First Cycle coding gives way to Second Cycle Coding or Pattern Coding, during which summarised data is grouped into a smaller number of categories, themes, or concepts. Pattern codes are considered as being inferential or explanatory codes that identify a ‘bigger picture’ configuration by clustering material from First Cycle coding into more meaningful units of analysis (Miles et al. 2018). For the purposes of this research study, during the first cycle, codes from the literature were used to organise data into thematic clusters. Context codes, or labels connoting meso- or macro-levels of meaning to data, were initially assigned, giving way to second level descriptive codes that assigned labels to data that summarise the topic in a word or short phrase. In addition to codes, extended thematic phrases were used to describe what a unit of data was about and/or what it meant. During the Second Cycle, a Grounded Theory approach was used to extract Pattern Codes from the initial coding, which were then collected and used to derive results.

5. Data Analysis

Following Gioia (2020)’s systematic methodology for doing qualitative research, the data analysis phase of this research paper was divided into two stages—first-order (informant centred) and second-order (theory centred) data and findings—derived from the Qualitative Content Analysis (QCA) technique, and based on the two-cycle approach taken to coding. Qualitative Content Analysis, also known as Thematic Analysis, is one of the most commonly used systematic methods for analysing qualitative data (Kreuter 2021). This technique is focused on the delineation of categories (codes) and the development of a category system (coding frame) (Kuckartz 2021), from which a Thematic Network representing the interrelationships between the various elements of the category system is then constructed (Akinyode and Khan 2018).

5.1. First-Order Analysis

The first stage of the data analysis process involved the articulation of codes based on participant-generated data obtained from workshop transcripts. The experts’ views, experiences, and interpretations expressed during the course of the workshop were, in turn, guided by three overarching research questions and six investigative questions. These questions corresponded directly to four major themes grounded in the literature review. Data were consequently analysed through the lens of each of the four themes, namely:

Theme 1: Understanding the Concept of Sustainability. The literature shows that, despite the ubiquity of the concept, the term ‘sustainability’ is defined differently and implies different things in different contexts. It was considered important, therefore, to develop a conceptual orientation of the term by identifying the dominant conceptual paradigm within which context it is used and contrasting this with other emerging sustainability archetypes.

Theme 2: Definitions of Sustainability in the Public Sector Context. Existing literature on notions of sustainability in the public sector focus on defining the nature and scope of the concept within the given context, on the various factors that can enable and/or inhibit sustainable public sector activities and processes, and on examples of public sector best practice.

Theme 3: Impact of Sustainability on Digital Public Services. Literature on sustainability in the public sector, particularly in the European context, focuses on positioning the concept within digital transformation. When looking at public service provision, therefore, it is necessary to obtain a conceptual orientation about what this implies for digital public services. In particular, definitions of key concepts, factors permitting and inhibiting initiative success, and examples of sustainable digital public service initiatives should be explored.

Theme 4: Relationship between Sustainability and Co-creation. The literature argues that public service provision can be enhanced by exploiting new technologies and through the adoption of public service co-creation. The definition and scope of co-creation therefore needs to be critically analysed, as well as the connection between co-creation and digital IPS design and delivery, the factors permitting and inhibiting sustainable co-creation, examples of successful sustainable co-creation, and examples of failed sustainable co-creation.

5.2. Second-Order Analysis

The second stage of the data analysis process focused on the articulation of key theoretical themes and sub-themes derived through a critical analysis of the interrelationship of the codes and their eventual conflation into top-level categories. Special attention was paid to emerging themes and concepts that have not been adequately covered in the literature. Once the major themes and concepts were collected, an attempt was made to aggregate the information to create a data structure or “… a graphic representation of how the analysis progressed from raw data terms to themes and dimensions when conducting the analysis” (Gioia 2020, p. 26).

6. Results

Results are represented as four themes: first, a broad understanding of what the general term ‘sustainability’ implies is presented. Second, how sustainability in understood in the public sector context is explored. Next, the impact of sustainability on digital public services is considered. And finally, the relationship between sustainability and co-creation is examined. Key findings show that the concept of sustainability is understood to be the outcome of a concerted effort to maintain an object or an action for a prolonged period of time through the adoption of processes that ensured that sufficient scarce resources were made available for the same time period. Although long considered in the environmental context, notions of sustainability are rapidly gaining currency in the public sector, particularly with regard to long-term digital transformation and the design/delivery of innovative, often co-created digital public services. Factors that play in significant role in influencing the outcome of such initiatives can be political, organisational, financial, legal/regulatory, and technical. While it is acknowledged that co-creation can indeed help in cementing sustainability, it is recognised that governments still do not make use of the process in its purest form.

6.1. Understanding the Concept of Sustainability

During the workshop, participants were given an introduction to sustainability as enshrined in European and international policy and were first asked to develop and articulate their own understanding of what this concept implied. Participants were in agreement that there are many ways to conceptualise sustainability, often contingent on the context within which the term is used. The most common occurrence of the term, it was felt, was in an environmental context, especially when considering best practice to support successful environmental management. In a management context, it was mentioned that one generally talks about the sustainability of a business model or plan, particularly in terms of the maintenance of financial sustainability. In a governmental context, some participants mentioned best practice leading to the innovation and implementation of sustainable public services. Despite the myriad of contexts mentioned, however, participants were very clear that the term ‘sustainability’ implied a concerted effort to maintain an object or an action for a prolonged period of time through the adoption of processes that ensured that sufficient scarce resources were made available for the same time period. One participant went so far as to say that sustainability was not merely a process, but indeed an agile process involving the interaction of several stakeholders that goes through several iterations across a given period of time with continuous impact over that same period.

Factors listed by participants as influencing a sustainable outcome, particularly of a digital service, ranged from technological factors (type of technology, interoperability, usability), to legal (regulation, privacy concerns, data sharing concerns), political (the will to modernise a service, the identification and/or definition of which services to modernise), and organisational factors (standardisation of internal processes, access to funding channels). Of these, political and legal factors were felt to have the most influence on the long-term sustainability of an initiative. Sustainability was also seen as a financial issue. Experts felt that the length of term of a project directly corresponded to the funds available to sustain it. One expert felt that political, legal, and financial factors were often interrelated. Some workshop participants stressed that certain projects were less likely to attract long-term investment as they were considered as either too risky or their returns too low. Cultural factors within the organisation were also mentioned by experts as playing a part in determining the sustainability of a project. Some participants felt that the degree to which an organisation or decision-maker was user-oriented would significantly influence sustainability. In technology-driven projects, decisionmakers and employees also need to be open to innovation and change in order to fully embrace digital transformation.

Workshop participants were also asked whether they felt that measuring sustainability was important, and what indicators they would use to measure it. There was general consensus that the most measurable value was the impact that the sustainable practice had within a given context—more specifically whether or not quantifiable desired outcomes had been achieved. Participants focused on talking about sustainability in public sector service delivery contexts, and on the impact that sustainable practice had on the involved stakeholders. The group agreed that there is no one Key Performance Indicator (KPI) that adequately measures the impact of sustainable practices; indeed, they expected the metrics to change depending on technology, administrations as they come and go, and a populations’ needs. Some participants recommended that the impact of sustainability could be assessed through the periodic dissemination of questionnaires to relevant stakeholders, and that this has been done before.

The main conclusion that came out of the discussion was that measuring sustainability is possible, although it is complicated by the fact that sustainability is a continuous process. Four important examples of measure the sustainability of outcomes were mentioned during the course of the discussion. The first was a hypothetical launch of a tax system in Denmark, brought forward by an expert with several years’ experience in the field of public sector digital transformation. The expert remarked that if an online tax system was launched, those in charge would be extremely interested in gauging the take-up by people using this system over a period of time. To do this, they said, questionnaires and focus groups could be used to measure user experience, and to identify what works and what does not work. This, they stressed, was of paramount importance if one was the gauge the performance of a digital system over time. A second expert from Austria concurred. In their opinion, it was very important to consider user experience when thinking about sustainability of a public service. The more positive the experience while using the service, they said, the more likely it was that a citizen would use the service again. This could be measured using questionnaires. The case of a smart mobility service in Belgium was also considered during the discussion. Here, an expert mentioned, the public authority used a questionnaire to elicit feedback from users. The final country example given was Greece. One expert mentioned that in the Greek case, it had been very difficult to measure sustainability.

6.2. Definitions of Sustainability in the Public Sector Context

Data was then collected from workshop participants about how they understood the concept of sustainability in the public sector context. Experts agreed with workshop moderators that until recently sustainability in the European public sector was usually associated with the environment and with climate change. Even when linked with digital transformation, notions of sustainability were very much about efforts to support the achievement of long-term environmental goals. The 2015 Paris Agreement was discussed as an example of an international understanding on sustainability that focused on climate change and citizen participation in order to support sustainable environmental practices. The 2030 Digital Compass was then collectively examined as an initiative that moved away from sustainability in the environmental context, and as one with a greater focus on digital transformation and the digitisation of public services. It was noted during the session that, as with the Paris Agreement, the emphasis was very much again on achieving digital transformation through the greater involvement of stakeholders. One workshop participant noted that, in the public sector context, therefore, citizens were at the apex of a pyramid of conditions necessary for sustainable digital transformation. Another expert mentioned that it was equally important to view sustainable digital transformation as a continuous, agile process.

Experts attending the workshop then discussed the major factors positively influencing sustainability in public sector contexts. Chief amongst these was the need for political factors: political will, political vison, and leadership. Participants were of the opinion that in order for an initiative to be a sustainable prospect, political elites were required to be committed to digital transformation of the public sector. Those actors responsible for taking decisions, it was felt, need to have vision or the propensity for foresight or the ability to think long-term. Strong leadership was also considered to be a pre-requisite for actors in charge of digital transformation initiatives. Workshop participants further stressed the importance of an innovation culture within the adopting organisation in order to achieve sustainability. In this case, they said, the public sector organisation took their cue from the leadership. Public sector employees needed to be open to the idea of innovation through technology, and to the long-term adoption of sustainable work practices. A regulatory environment conducive to innovation and growth was also mentioned in this context. Yet other workshop participants stressed the importance of funding on the sustainability of public sector initiatives. The more secure the sources of funding for these projects, the greater the chance of them being sustainable in the mid- to long-term. Technical factors including user-friendly service environments, and simple technologies were mentioned as factors that supported sustainability in the public sector. When asked about the potential barriers to public sector sustainability, workshop participants were of the opinion that the same factors discussed as drivers for such initiatives could in their absence turn into barriers.

In conclusion, experts sought to support their opinions with examples from their professional and personal experience. One participant from Austria highlighted the importance of user-friendly interfaces and interoperability through an example of connecting local taxation services with federal data sets. They also mentioned the importance of leadership through the same example, and how customer-oriented leaders could make the process of service design and delivery more agile. Funding in this context was also mentioned as a make-or-break factor with regard to sustainability. Another expert from Greece outlined the situation in their country. By and large, they said, sustainability has always been an issue in the Greek public sector, but has not yet been achieved. The main cause of this, according to the expert, was a lack of political will and long-term vision, leading to the inability of Greek public servants to plan ahead on matters as important as funding. Investments, too, were often made seemingly without a long-term plan in mind. Financing, the expert said, was critical. Without long-term funding and planning, entire projects collapse. Lack of political will, follow-through, long-term vision, and petty politicking also cause problems in Belgian and Portuguese contexts. An expert from Belgium talked about how multi-level governance posed problems for the sustainability of public sector initiatives in cases where the decisions of political elites were heavily influenced by short term policy-cycles. In Portugal, the introduction of an e-voting system got bogged down because of a lack of effort and petty-mindedness on the part of decision makers.

6.3. Impact of Sustainability on Digital Public Services

Data was also collected from workshop participants to determine what they perceived to be the impact of sustainability on digital public services. To begin with, experts focused on the continuity implied by the sustainability concept, and discussed its affinity with agile, integrated digital public service delivery. One expert in particular stressed the need to think of digital public services as a collective of interrelated workflow processes if each standalone service was to become sustainable in its own right. Another workshop participant felt that it was important for decision makers to think carefully about which services they would like to integrate, and to prioritise the digitisation of service offerings depending on what citizens considered important. Engaging citizens in the digital public service design and delivery process was also discussed at the workshop in some detail. Some experts stressed the need for project planners to provide sufficient opportunities for meaningful participation in order that citizens and other stakeholders can be involved in projects and contribute towards their eventual sustainability. The role of political elites in defining the legal and regulatory environments that circumscribe the use of technology was also investigated. One expert opined that decision makers had the power to decide upon the rules and regulations with a particular societal context, and these would in turn enable or curtail the use (or abuse) of any technology in the long-term. Ethical issues associated with Artificial Intelligence (AI) need to be navigated. Furthermore, stakeholders involved in processes of digital transformation may not have the capacity to deal with the disruption caused by the introduction of cutting-edge technology into their daily personal and professional lives, and it is up to those in positions of authority to provide training and create awareness in order to make the transition to hypermodern environments sustainable.

Data from the workshop also shows that participants were cognizant of a number of drivers and barriers to the development of sustainable digital public services. A significant part of the discussion focused on stakeholders and stakeholder engagement. Several experts felt that building stakeholder capacity to respond to the challenges posed by digital transformation was key to the long-term sustainability of digital public service initiatives. In particular, encouraging a culture of innovation amongst public servants (who drive service design and delivery) and citizens (who participate in co-creation) was mentioned. One expert said that as innovation was usually driven by the private sector, the role of entrepreneurial and business actors ought to be recognised and expanded, as has been done in Estonia. Stakeholders, therefore, need to be engaged using methods that are best suited to the stage of the design/delivery process, and, furthermore, at appropriate junctures. Workshop participants saw this as a precondition for fully optimising the potential of new technologies through improved, targeted service offerings and service integration. Conversely, experts considered a lack of stakeholder engagement, or half-hearted attempts at it, to have a significantly negative effect on the sustainability of digital public services. The overregulation of service provision environments, lack of funds, and bad planning which lead to escalating costs and the wastage of time were all mentioned as additional barriers to sustainable digital public service design and delivery.

To further illustrate their remarks, workshop participant drew on examples from Europe and around the world. One expert contrasted Estonia with the Portuguese experience. In Estonia, life is far more “digital” than in Portugal, and the private sector has a larger role to play in the innovation of digital public services. However, neither country is as yet fully open to the idea of citizen participation in public service design/delivery. The situation has moved a little farther ahead in Denmark, where, according to another expert, the government is piloting testing and co-creation in at least two service areas. While this is still nascent, the Danish government has seemingly made a significant effort to consider end-user needs, identify its own business needs, and develop an e-government strategy situated at the intersection of the two. According to this expert, a classic example of successful service integration in Denmark is the process of changing an address. The only problem is that often, when there are funding cuts, the co-creation element of an initiative tends to get sacrificed first. Yet another expert from Greece mentioned that, in their country, there has been a reluctance to educate citizens and build capacity necessary to sustain digital public service initiatives. So-called “quick wins”, where services are automated and apps are released in the public domain without an effort to educate target populations, are more popular than sustainability considerations. All these examples go to show that although governments talk about closely involving citizens in processes of digital public service design and delivery, there is still a significant gap between the rhetoric and reality.

6.4. Relationship between Sustainability and Co-Creation

The final set of data collected allows for the critical exploration of the relationship between sustainability and co-creation. In the first instance, workshop participants discussed the concept of co-creation, attempting to arrive at a definition and to delineate its overall scope. Most experts agreed that co-creation encompassed the active involvement of different actor groups in processes of creation. Similar to sustainability, co-creation was seen as being both iterative and agile. In the context of the public sector, co-creation was considered by workshop participants to occur most often during the design and delivery of public services. Co-creation was also seen by participants to be a deliberate act, not something that happened ‘out of the blue’ but was instead instigated by design, based on a set of principles and following a pre-determined methodology. However, there was less agreement amongst participants on the exact scope of co-creation. Rather than directly answering the question posed by workshop moderators, the experts in the room responded with questions of their own: where does co-creation in a process begin and end? Does co-creation of a service involve the creation artefacts specifically for but independent of government? Is there a particular method or use of a communication channel that makes one automatically consider a product or service as being co-created? Participants also discussed ways in which the sustainability of co-creation could be measured. One expert noted that several indicators and measurement opportunities do indeed exist, however these tend to be quantitative. What is needed, they felt, are other measures—usage, user satisfaction, and service quality—that are more qualitative. Additionally, one needs to consider whether there are any problems, where design-actuality gaps are, and if the services designed actually meet user expectations. These issues are more qualitative in their outlook.

In grappling with these issues, the discussion naturally turned to the relationship between sustainability and co-creation, and most especially what the drivers and barriers of successful co-creation are. Capacity building was mentioned as the first factor having a positive influence on co-creation. One expert suggested that both governmental actors and other stakeholder needed to have the expertise, know-how and skills necessary engage in co-creation processes if they were to be sustainable in the long term. Co-creation initiatives, they said, required not just top-down or bottom-up knowledge transfer, but also encompassed a continuous learning process on the part of all the actors involved. Stakeholders need training and must be provided with the tools and resources (including time and opportunity) to participate. Another expert proposed that project planners needed to have a good understanding of all project stakeholders: who they are, how they stand to benefit or lose, what motivates them to engage, and the values they hold. This sort of knowledge would help identify the appropriate means for engagement, as well as any potential sticking points in the co-creation process. Several other workshop experts concurred. If the government project was of interest to other groups of stakeholders, and those groups were approached to participate through channels that were convenient for them, then it was likely that the co-creation process would endure and the outcome—that is, the developed product or service—would be sustainable in the long run.

Contrasting with this, workshop participants identified several other factors that could result in unfavourable co-creation processes and outcomes. Chief amongst these was politics. Experts were concerned that the introduction of a co-creation element into public service design or delivery could be seen as an erosion of political power, and that governmental elites could deliberately water down processes, leading to unsustainable outcomes. Fear of complexity was seen as another limiting political factor, with public administrations shying away from co-creation processes because they were worried that it would interfere with their way of doing things. Organisational mindset and culture, it was felt, could also hamper sustainable co-creation. Experts felt that as government organisations are very hierarchical, giving way to decisions made by citizens would be extremely difficult. One of the biggest complaints made by civil servants against involving the public in governance processes, according to one expert, was that citizens “don’t understand the legal framework, they don’t understand the constraints we’re working with or the structure we’re working in”. Some public managers simply do not—one way or another—have the capacity to be flexible. Technology was discounted as a limiting factor, as it was felt that sustainability is largely technology-agnostic.

Workshop participants were also encouraged to share any examples of successful, sustainable co-creation. In Austria, there are co-creation efforts that focus on all stages of co-creation, such as gaining input for the development of new services and the co-delivery of digital services (Edelmann and Mergel 2021). The co-creation of a new digital Tourism Overnight Stay Tax Service, kickstarted by the Office of the Lower Austrian Federal Government, is proving to be more successful than expected and, although a regional effort, may become an established internal process within the regional government for future public services and is establishing itself as a good practice for other public administrations (Edelmann et al. 2022 ). However, experts found it easier to talk about instances of failed co-creation. One expert went as far as to describe co-creation as being in some cases “dangerous”. Examples of failed co-creation from across the world were discussed. One workshop participant mentioned that co-creation had failed to take off in Japan and South Korea, as the working culture was extremely hierarchical and not at all conducive to innovation (Meyerhoff Nielsen and Jordanoski 2020). Another workshop participant put forward Switzerland as an example, where although there are some examples of successful co-creation outcomes (e.g., the Swiss Federal Railway operator (SBB), see, for example, Gebauer et al. (2010)) co-creation as a process in public administration is stifled as everything (including public service design and delivery) is decided by referendum. Yet another expert suggested that that population size in countries like China could severely hamper the organisation of co-creation processes. Demographics in terms of age and gender has been considered in co-creation activities (Cambra-Fierro et al. 2017), making the question of population size involved an interesting aspect to investigate in future. Often, according to one expert, the gap between citizen and government priorities results in the failure of co-creation. Co-creation has been implemented successfully in the Danish public administration (Scupola and Mergel 2022), although when the Danish Tax Agency tried to implement co-creation processes it had to give up when the discussions were taken over by other issues. In Italy, co-creation often fails because processes do not meet basic minimum legal requirements. Finally, said one expert, lack of funds also leads to co-creation failures no matter which country one is in. When project planners find themselves short of money, the first thing to get slashed is the co-creation process. This has happened in countries as diverse as Bangladesh, Ghana, Brazil, and Sweden. The small number of positive examples demotivates public administrations and discourages them from engaging with new co-creation processes. It is important to be transparent and open about problems regarding the implementation of co-creation processes and the lack of success in reaching the desired outcomes. As urged by (Jarke 2021), it is important to consider all the lessons learned, and even the failure about such issues can help avoid the repetition of the same errors and problems in the future, thereby contributing to developing sustainable co-creation processes and outcomes.

7. Discussion

An analysis of research data obtained from participants during the workshop yields several key insights that help develop a more nuanced understanding of sustainability in the public sector, and its impact on the co-creation of digital public services. First, results address the meaning of sustainability in public sector contexts, how this is similar/dissimilar to notions of the concept in literature pertaining to the environment, and what implications this has for the design and delivery of digital public services. Second, we discuss how sustainability in the public sector can be measured. Through the data, the centrality of stakeholders to the process is also critically analysed. Co-creation, as a means to enhance sustainability in public sector contexts, was also examined, and the factors contributing to sustainability in public service design and delivery were also given due diligence. These are discussed below.

Sustainability is understood as the ability to support or sustain what is considered valuable over a concerted period of time. The most common occurrence of the term was considered in an environmental or developmental context; however, the concept was also acknowledged as gaining currency in the public sector context. The term ‘sustainability’ is understood by experts to be a concerted effort to maintain an object or an action for a prolonged period through the adoption of processes that ensured that sufficient resources were made available for the same period. In a governmental or public sector context, there was consensus that the term implied best practice leading to the innovation and implementation of sustainable public services. The data collected focuses on the notion of continuity implicit within definitions of sustainability, and the agile, iterative nature of the process of sustaining something over a long period of time. In addition, sustainability is not just a single or one-off outcome. Instead, it is a continuous and iterative process.

The long-term, iterative nature of sustainability has implications for digital public services. Digital public services need to be considered as a set of interrelated workflow processes if each standalone service is to be made sustainable. The integration of public services can also be considered as a means of ensuring sustainability of digital public services over time. Given the centrality of stakeholders—particularly citizens—to sustainability, it is important to consider their priorities when choosing which services to integrate. Decision-makers, therefore, need to think carefully about which services they wish to integrate, and about the workflow processes that underpin them.

Different factors influence the sustainability of digital transformation processes. These factors are political, organisational, legal, financial, and technical. Political will and vision as well as leadership are important in supporting such processes, but also organisational factors such as developing the necessary organisational set-up a lead to a strong innovation culture that embraces co-creation. At the same time, the legal factors that inform the regulatory environment conducive to innovation and growth, the financial and technical resources that ensure interoperability, user-friendly interfaces and integrated service bundles are important.

Sustainability requires the continuous interaction of various stakeholders. Again, this is because sustaining an action or initiative over a period time is not linear, but is more generally seen as agile. Continuously engaging citizens and other stakeholder groups is important to make service design and delivery more participatory, and hence more sustainable, but political actors also play a role in setting the regulatory environment that circumscribes human action and what a technology can or cannot do. The role of business actors should also be expanded, so that they can drive innovation in public sector contexts too.

Co-creation is a means to enhance sustainability in public sector contexts. Several factors influence the sustainability of co-creation processes in the public sector, in particular capacity building, continuous learning, and training. Capacity building is important in order to develop an innovation culture amongst key stakeholders. But continuous learning is also necessary to facilitate knowledge sharing and transfer and to dispel myths about the co-creation process. The provision of training equips stakeholders with the necessary tools and techniques. In addition, it is important to ensure that stakeholder engagement is sustained as it helps understand key priorities, motivations, and behaviour.

Co-creation does not only lead to positive but also negative outcomes. This may occur for several reasons, for example, when key stakeholders are selectively involved or not through appropriate channels, or when co-creation processes are understood by political actors as an erosion of responsibility and power. It is therefore necessary to understand the mechanisms that lead to knowledge renewal and the continuous production of sustainable innovation, and that the sustainability of collaborative practices can be strengthened.

Measuring sustainability is possible, but complex. Some attempts to measure sustainability in public sector contexts have already been made, such as assessing take-up of a system by users through an evaluation of user experience and user satisfaction. Common methods used include questionnaires, surveys, and focus groups.

8. Conclusions

This research article aims to contribute to the burgeoning conceptual literature on sustainability in the public sector and the co-creation of digital public services. To achieve their objectives, the authors adopted a systematic two-stage qualitative content analysis approach to analyse research data following on from a two-cycle approach taken to data coding and structuring. Using qualitative analysis software, categories (codes) were identified from discussion transcripts, and a category system (coding frame) developed. From this, a thematic network representing the interrelationships between the various elements of the category system was then constructed and research data analysed.

Our enquiry focused on understanding how the terms ‘sustainability’ and ‘co-creation’ are used in public administration research and practice, how co-creation can lead to sustainability, and which dimensions of sustainability in turn impact co-created digital public services. This research concludes that sustainability is still often associated with climate change and activism but is increasingly being understood as the effort needed to maintain an object or an action over a prolonged period in various contexts, including the public sector. The qualities ascribed to sustainability—it is long-term, iterative, agile—have implications for who should be involved in digital public service design and delivery, and for how long. Co-creation practices are considered as a means to enhance sustainability in the public sector; however, several factors circumscribe their efficacy, and their tendency to result in positive or negative outcomes. Describing sustainability as an agile process and by identifying the factors important for sustainability, in particular for digital initiatives, also provides a way for measuring its impact beyond quantitative measures.

The work presented in this paper is not without its limitations. The qualitative research paradigm adopted for this research paper implies that results are based on detailed observation and interpretation, and that it is often a subjective decision as to what information is deemed important and what is not. Results are thus not as easy to replicate as if they were obtained through a quantitative study. Furthermore, while the workshop format was selected for its suitability in eliciting in-depth information from an audience of experts, the research data upon which this study is based was obtained from only 20 participants at an international conference. In the first instance, the sample can be thought of as being self-selecting: only those individuals attending the conference and interested in the workshop topic would attend. In the second, this number of participants is by no means representative of the wider digital government community, and the results inferred constitute only a small snapshot of expert opinion in the field. The workshop as a research approach is also not without its flaws, as it is difficult to replicate or validate results obtained from such a format. Any follow-up study would have to include in-depth interviews on the themes derived. Finally, the time available to hold the workshop was a 3-h slot, during which period three research questions and six investigative questions had to be presented and discussed. This automatically limited the amount of time that participants could spend on developing and elaborating upon ideas.

Although the number of experts was limited, the open discussion and the use of a qualitative approach to study the data collected helps to investigate a complex issue such as sustainability. Several related research issues were also raised during the course of the workshop and warrant further examination. These have been grouped thematically and discussed below.

Theme 1: Understanding the Concept of Sustainability. In a governmental or public sector context, the term ‘sustainability’ may be understood as a concerted effort to maintain an object or an action for a prolonged period, leading to the innovation and long-term implementation of a public good or service. However, a more detailed examination of the factors that contribute to sustainability of digital public services is required to understand the exact role that stakeholders play in the process, and how co-creation initiatives and technology can be incorporated.

Theme 2: Definitions of Sustainability in the Public Sector Context. The most common occurrence of the term ‘sustainability’ in the literature can be found in an environmental or development studies context, and experts constantly refer to these two fields when talking about their experiences in the public sector. However, as sustainability becomes an increasingly important attribute of a public good or service, it is important to develop a definition of the concept that is rooted in experiences from the public sector.

Theme 3: Impact of Sustainability on Digital Public Services. Notions of sustainability in the public sector need to move beyond broad-based discussions of digital transformation and become more granular to consider what this implies for internal back-office functioning and digital public service provision. Discussions with experts underline what the gaps in the literature already make clear: that definitions of key concepts, factors permitting and inhibiting initiative success, and examples of sustainable digital public service initiatives need to be further explored.

Theme 4: Relationship between Sustainability and Co-creation. How co-creation supports and/or impacts sustainability in the public sector and how a focus on sustainable processes and public service provision leads to the incorporation of innovation and co-creation also needs to be further researched. The reasons for this are two-fold: on the one hand, co-creation can lead to either positive or negative outcomes, and, on the other, sustainability is all about discerning what can or should be sustained. In this, the factors and practices that contribute to positive outcomes in both cases need to be identified from real-life examples and fed back into policy documents.