Co-Production of Digital Public Services in Austrian Public Administrations

Abstract

In the digital transformation of public administrations, objectives are no longer simply the implementation of new technology, but the involvement of all stakeholders into the process of digitalization. The Digital Roadmap of the Austrian government emphasizes the need of co-production of public services as a key element to public service delivery and, subsequently, innovation of the public sector. To understand how co-production in digital service delivery is implemented in Austria, we conducted interviews with 41 experts from public administrations in order to understand who is involved in such processes, how they are involved, and what outcomes are to be achieved.

1. Introduction

The European Commission’s Digital Strategy calls for the digital transformation of public administrations to address complex challenges, such as globalization or sustainability by developing “appropriate, EU-wide, interconnected policies” that rely on data but also to “deliver better public services” (European Commission 2018b, p. 2). Joint European responses include the EU eGovernment action plan 2016–2020 (European Commission 2016) or the Tallinn Declaration (2017), and European member states have responded by developing their own national digital strategies, some of the first were, for example, Sweden (Government Offices of Sweden 2011), Germany (Deutsche Bundesregierung 2014), and Finland (Prime Minister’s Office 2015). Similarly, Austria’s digital strategy, the Austrian Digital Roadmap (Bundeskanzleramt and Bundesministerium für Wissenschaft 2016) highlights digitalization as a way of addressing complex societal issues that cut across all sectors. Based on 12 principles, it lists core aspects, such as the need to reduce the digital divide, ensure digital participation, but also emphasizes digitalization as an economic asset that is useful to businesses, citizens and public institutions. The 12th principle states how digitalization can support innovation in public administrations: “Public authorities are drivers of innovation. Citizens and businesses have the right to comfortable, simple and barrier-free electronic communication with public administrations.” (Bundeskanzleramt and Bundesministerium für Wissenschaft 2016, p. 5)

What is common among most national digital agendas is the use of innovative digital technologies, the development and management of digital solutions, and services based on user-centric and co-creation principles. The co-creation or the co-production of public services (the terms are used interchangeably as noted by Voorberg et al. (2015)) is generally understood as the involvement of citizens and other actors external to the public administration in the design, production, and provision of public services (Cordella et al. 2018; Meijer 2014; Voorberg et al. 2015). There has been a great deal of interest on co-production in the public sector context. Originally the term co-production was described as a process “through which inputs used to produce a good or service are contributed by individuals who are not ‘in’ the same organization” (Ostrom 1996, p. 1073). Increasingly, it is seen as a driver for innovation creation to increase public value in the public sector (Bovaird and Loeffler 2012) and is understood as a task that is not only delivered by professional and managerial staff in public agencies, but also involves, or is co-produced by, citizens and communities (Loeffler 2020). Co-production emphasizes the shared character of the production process to transform the delivery of public services (Brandsen and Pestoff 2006). Research on this topic has advanced and has focused on different dimensions such as co-planning in order to encourage innovation in governance (Sørensen and Torfing 2018), co-design that aims to include citizens and professionals in the development of better, more beneficial or new services (Nabatchi et al. 2017; Loeffler 2020), and the co-delivery of (social) public services—at the point when citizens use an existing service (Brandsen and Pestoff 2006; Loeffler and Bovaird 2020; Osborne et al. 2016). In this study, rather than attempt to determine one encompassing definition of co-production, revise definitions (Brandsen and Honingh 2016), or provide an overarching framework of co-production, we aim to consider how co-production is understood and implemented in public administration practice. Thus, we consider not only a single definition or previous phases of co-production to understand how users might be involved in phases leading up to the use of the service, we also focus on the outcomes that the experts identify when using co-production by engaging different users in form of the type of public value that might be generated. Given that co-production plays a prominent role in European digital agendas, this study therefore seeks to assess two issues: first, how do public administrations engage in co-production activities with users, and, secondly, what outcomes of co-production efforts do public administrations expect?

In order to answer these research questions, we use a narrative interpretative approach to analyze the data gained from structured expert interviews with public servants from Austrian public administrations as well as a small number of experts involved in the digital transformation in the public sector (e.g., consultants, IT service providers). These structured interviews are based on an interview guideline containing questions derived from a systematic literature review and based on the theoretical framework presented in Mergel et al. (2019). We then analyzed our data using a thematic approach to identify common patterns in the interview data (Bloomberg and Volpe 2018).

Our findings show that although several different actors are involved in co-production processes, the main users that are involved in the co-production of digital public services are citizens and public administrators themselves. In terms of outcomes, achieving high-quality input by users in order to gain more insights about the users’ needs as well as developing and offering better quality services is seen as particularly important. At the same time, co-production itself is seen as providing citizens the opportunity to be active and involved. The results show that the value of co-production goes beyond the provision of feedback in order to improve digital services (e.g., www.finanzonline.gv.at (accessed on 10 January 2021)—the online tax declaration), but also through the use of a range of methods and tools, such as apps that enable citizens to provide information and feedback, online platforms that bundle digital services depending on the type of user (e.g., the österreich.gv.at (accessed on 10 January 2021)), or using artificial intelligence (AI), social media, or other crowdsourcing activities in order to gather useful data. A further outcome is that although digital tools and applications are seen as an opportunity to support co-production, experts were careful to point out that there are not only advantages, but also limits to the involvement of citizens.

In the next section, we provide an overview of the literature on digital co-production. We then present the research design including the qualitative methods used to collect and analyze the data. We will then review our results synthesized from expert interviews, and finally discuss the results in the context of the existing literature. The study concludes with theoretical and practical implications, as well as potential limitations and how we aimed to mitigate these limitations.

2. Literature Review

The idea of co-production originates from Ostrom et al. (1978), who developed a model of public service production where not only the consumption but also the production of public services can require the participation of citizens, that is, the “activities of citizens influence public agency outputs, objective community conditions, and citizen perceptions” (p. 383). More recently, Verschuere et al. (2012) reveal that co-production includes activities where public service agents and citizens contribute to the service delivery, where public service agents are the “professionals” and citizens contribute “voluntary efforts” in the “context of professionalized service delivery” (p. 1085).

For public administrations, the implementation of co-production can fundamentally change the way public services are provided, that is, by promoting citizens as central parts of the value chain (Ryan 2012). Co-production is seen as the involvement of citizens as active participants in the production of outcomes (Ryan 2012), so that they are not just the consumers of a public service, but co-producers of information and services along all phases of the co-production chain (Huijboom et al. 2009).

Bovaird and Loeffler (2012) argue that co-production must be “conceptualized as a shift from ‘public services FOR the public’ towards ‘public services BY the public’” (p. 1121) and that a definition of co-production must emphasize “reciprocity” in order to make “better use of each other’s assets and resources to achieve better outcomes” (ibid.). To this idea, Radnor et al. (2013) add that co-production can do more than just improve the provision of existing public services, rather, it makes a real contribution to public service innovation and improves services for future users. Therefore, Osborne et al. (2016) highlight co-production as intrinsic to the process of public service delivery and link it directly to the co-creation of value both for service users and for society.

Trischler and Scott (2015, p. 720) also point out that public services are often seen as “one particular product category to be designed by the organization and delivered to citizens as passive recipients of value” instead of “open systems in which the service provider interacts with the service user in value co-creation and all actors can act as resource integrators.” The public value co-creation process is “closed” when public organizations claim to know what citizens need, design the optimal services and procedures to deliver public value (Cordella et al. 2018). Co-production opens this process and involves stakeholders from both outside and inside the organization leading to different types of outcomes (Boyle and Harris 2009; Chatfield et al. 2013; Cordella et al. 2018; Duijn et al. 2010). It is seen as improving public value (Rose et al. 2015a; Williams et al. 2016).

Several authors (Brandsen and Pestoff 2006; Cordella et al. 2018; Osborne et al. 2016; Radnor et al. 2013) argue that co-production leads to higher service quality, delivery and efficiency, and expanded opportunities for participation. Others suggest that co-production can enhance the quality of information the organization provides (Bolivar 2017), the engagement with citizens, and the perception citizens have of the organization itself (Bovaird et al. 2014; Dickinson et al. 2015) which might lead to improved user satisfaction with public services (Åkesson and Edvardsson 2008; Alford 2016). In these conceptualizations, co-production is, however, not simply a participation process, or an exchange of information, but involves the re-organization of the relationships, the interaction and co-operation between the involved actors (Gawłowski 2018; Mergel et al. 2019).

Economic considerations such as increasing productivity, cutting or saving costs are rarely mentioned (Brandsen and Pestoff 2006; Pestoff 2013; Pestoff et al. 2006). Although one of the reasons why public administrations and governments support citizen involvement is “substitution value” (Pestoff et al. 2006, p. 599), where citizens carry out tasks originally performed by public administrations and governments, which might in turn lead to cost savings.

There is general agreement that citizens play an important role in co-production (Wamsler 2016) as they are seen as contributors of skills and knowledge, beneficiaries, and, finally, evaluators of public services that help design the way in which public services are delivered (Alford 2016; Silvestre et al. 2016). Williams and Shearer (2011) point out that citizens are responsible for identifying what is valuable to them and designing the service they need. This will ensure that they obtain enough and adequate information about the services provided (Breit and Salomon 2015; Gawłowski 2018). Other users, or rather, groups of users mentioned in the literature are communities, government agencies, parliamentary commissions, committees, local authorities, politicians, contractors, and IT organizations (Greve 2015; Nabatchi et al. 2017; Pestoff 2013; Rose et al. 2015b; Wiewiora et al. 2016).

Breit and Salomon (2015) see co-production as a way of helping public administrations be efficient, increase the availability of services and provide more personalized services, so another important stakeholder group that needs to be considered are public sector employees (Åkesson and Edvardsson 2008; Ryan 2012). According to Lember (2017) digital services allow public administrations to develop new tasks and gain new capabilities to serve the public more efficiently and more effectively. The services must be designed to ensure that the benefits of the information and communication technologies systems can be fully realized by its employees (Åkesson and Edvardsson 2008).

Overall, the extensive research on co-production shows the importance that has been attributed to co-production for the involvement of users with the aim to improve the development and delivery of public services. Several studies on co-production point to different users, their roles and involvement in distinct phases or types of co-production (Loeffler 2020; Nabatchi et al. 2017), but experts and professional staff from public administrations tend to describe users’ involvement in co-production processes by mentioning “active engagement” or “active participation” so it remains unclear what the outcomes of co-production processes are specifically.

Therefore, this study aims to:

• Understand how public administrations engage in co-production activities with users in the process of developing or re-designing public services; and
• explore the expected outcomes of co-production in the context of the digital transformation of public administrations.

3. Research Design

In order to answer the aforementioned research questions, we chose a narrative interpretative approach to study why and how public administrations incorporate co-production into their digital service design (Griffin and May 2012). A narrative approach allows us to get to the bottom of real-life problems, that is, to understand the context and the significance that people (in this study, the public servants) attribute to a specific phenomenon (Klein and Myers 1999; Vaismoradi et al. 2013), to extract the stakeholders’ viewpoints who are involved in these processes (Czarniawska 2004), as well as consider the ambiguity and complexity of organizational phenomena under study (Riessman 2008; Vaara et al. 2016). With the narrative analysis we are able to “capture the rich data within stories” (Mitchell and Egudo 2003 p.2), to record different viewpoints, and to interpret the collected data in order to identify similarities and differences in the experiences and actions.

Narratives can be conceptualized as people’s constructions of organizational phenomena (Czarniawska 2004) and have been used to study organizational or cultural change (Vaara et al. 2016), decision-making in organizations (O’Connor 2002), and knowledge transfer in organizations (Darwent 2000). These approaches are increasingly popular in management and organizational research as they represent a credible source of new insights and knowledge, and help explore the organization or organizational experiences people have (Rhodes and Brown 2005).

Here, we have chosen a thematic narrative analysis to focus on the content in the informants’ reports of the events that emerge as patterns or themes developed from the literature rather than focusing on the ways the narrative is told. Thematic approaches highlight the “what” of the narrative (the content of stories) and involve searching and identifying common threads in textual materials, such as interviews, by breaking down the text into smaller units of content and then analyzing them in order to achieve a rich, detailed and complex account of the data (Vaismoradi et al. 2013). The study should not just be a frequency of codes in order to find significant meaning as the context of the data plays an important role. Thus, we use inductive theoretical analysis to categorize the emerging themes from the data (Charmaz 2014). Next, we describe the data collection method, the case and interviewee selection, as well as the analytical steps.

4. Methods

4.1. Narrative Expert Interviews

In order to answer our research questions, we conducted narrative expert interviews as a way of “collecting stories” (Czarniawska 2004, p. 35). The narrative is seen as valuable and reliable as it is “highly unlikely that the interviewee should resort to a repertoire of narrative devices unusual for his or her practice” (Czarniawska 2004, p. 50). We chose to conduct these narrative expert interviews with a range of interviewees seen as able to add insights from different perspectives on digital co-production based on their professional involvement in public administrations at different levels of government (national, regional, municipal) and organizations that work closely with or for public administrations.

4.2. Case and Interviewee Selection

This study focuses on Austrian public administrations. The eGovernment Benchmark Report (European Commission 2019b) indicates that Austria, along with Estonia and Malta, is an eGovernment front-runner with high scores in overall digital maturity, and in particular high scores in online availability, usability, mobile friendliness and provision of eDocuments. In addition, the report shows that Austria is outperforming other European countries in terms of digitization, even if the level of digital penetration is only average. The Digital Economy and Society Index ranks Austria above the EU, ranking 13th out of 28 member states (score 54.3) (European Commission 2020). The eGovernment in Austria report (European Commission 2018a) points out that more than 98% of the most commonly used public services are available on the online platform www.oesterreich.gv.at (accessed on 10 January 2021). The up-take of online services by citizens and businesses is increasing: 66% of Austrian citizens use the Internet to interact with public authorities, 56% for obtaining information from public authorities, 45% for submitting completed forms, and 38% for downloading forms. Digital public services is the DESI dimension that Austria performs best in: Here Austria ranks 8^th among EU countries, which is above the EU average (European Commission 2020).

The use of Austrian digital services provides financial incentives, and allows faster processing and faster payments, but digital services are not used as extensively as expected (European Commission 2019a; Gönenç and Guérard 2017; Rupp 2017). Austrian citizens’ use of the Internet is generally high, but low in terms of using the Internet for requesting services and submitting applications to public administrations. Rupp (2017) argues that this may be due to a lack of communication and, therefore, lack of awareness, that is, people do not know that digital public services are available, a lack of digital literacy, and, although electronic filing is used in all federal ministries, a lack of common standards, procedures and infrastructure persists.

The Austrian federal government is committed to the development of a lean, dynamic and strong administration as described in the Austrian Chancellery’s e-government strategy (Bundeskanzleramt and Plattform Digitales Österreich 2017). To achieve these aims, changes have been made, such as moving the political responsibility for Austria’s e-government strategy from the Federal Chancellery to the Federal Ministry for Digital and Economic Affairs, developing the platform Österreich.gv.at as a one-stop-shop platform for online services, and the launch of the app “Digitales Amt” (Digital Public Office) (Bundesministerium für Digitalisierung und Wirtschaftsstandort 2020a) that helps citizens and businesses interact with public authorities. Future objectives include a greater use of ICT tools, the implementation of e-payment systems, digital delivery and digital signatures, fostering citizen participation and their involvement in the overall digitalization of the country. The Digital Roadmap is comprehensive but has been criticized as lacking quantified targets and monitoring (European Commission 2020). The development of a Vision “Digital Austria in 2050” is the next step for the overall digitalization strategy based on a several “strategic action plans” focusing on selected priority topics (Bundesministerium für Digitalisierung und Wirtschaftsstandort 2020b). The strategy is to update and harmonize different outdated strategies, fostering the digital transformation and improving user-centric, modern e-government services.

Austria can therefore be seen as exemplifying a representative case (Seawright and Gerring 2008) for conducting an investigation into co-production in the public sector and allows the researchers to evaluate the evidence gathered.

In order to identify the appropriate interviewees for this study, we began with a purposive selection of experts (Mason 2018; Ritchie et al. 2013; Robinson 2014) from the Austrian public administrations who are either nationally well-known for their expertise or hold leading positions. For this initial purposive selection, the researcher team assumed that certain individuals, those who provide important perspectives in the Austrian media on the phenomenon under study, present at conferences, or are well-known authors or contributors, need to be included in the sample. The researchers began the investigation with a small group of well-known and high-ranking experts from Austrian public sector organizations associated with the digitalization and the e-government strategy.

We followed the initial approach with the snowball technique asking each expert who else they would consider to be an expert in the field (Atkinson and Flint 2001; Biernacki and Waldorf 1981). This chain-referral approach (Biernacki and Waldorf 1981) was used because it “yields a study sample through referrals made among people who share or know of others who possess some characteristics that are of research interest” (p. 141). It involves asking the experts interviewed to identify additional people they know who fit the selection criteria and possibly even ask them for a personal referral. As a result, a total of 41 experts were interviewed hailing predominantly from the public sector (33), six from the private sector, and two experts from the non-profit sector. Through this selection method, experts from different parts of the public sector were interviewed, including federal ministries, the parliament, regional governments, cities and municipalities, public authorities as well as organizations owned by the public sector. In addition, experts from the private and non-profit sectors who work with the public sector on digital transformation projects were also interviewed.

In order to ensure that the Austrian public sector was appropriately represented, each interviewee was asked who else they consider as important actors in the public sector digital transformation. This snowball sampling led to additional interviewees until saturation was reached through this chain-referral process (Glaser and Strauss 1967).

As a result, the number of interviewees lies above the suggested 20–30 interviews and we stopped searching for additional experts, when we reach saturation and the same names were referred to the research team by the interviewees, as outlined by Saldaña (2013).

Table 1. Experts’ by sector and organisation.

Sector	Organization	No. of Experts
Government Sector	Federal Ministry	10
	Parliament	2
	Regional Government	6
	Statutory City	9
	Municipality	4
	Public Authority	1
	Publicly-owned Corporation	1
	∑33
Private Sector	PLC	5
	Consultant	1
	∑6
Non-profit Sector	Public Research Institute	1
	University	1
	∑2
∑41

provides an overview of the experts included in this study.

4.3. Data Collection

The data were collected between 2018 and 2019 using a structured interview guideline containing 14 questions. The questions were derived from a systematic literature review and based on the findings presented in (Mergel et al. 2019). The expert interviews were conducted face-to-face, using online video tools (e.g., Skype), or telephone. Prior to the interviews, each interviewee was asked for permission to record the interview for accuracy purposes, was ensured anonymity and that transcripts would not be made available to a third party.

In the interview guide, two questions specifically addressed co-value production in the digital transformation of public administration in Austria. The interview questions served to gain an understanding of co-production in public administration from the perspective of the experts. They focus on the users and their role as well as examples of co-production:

What is the role of citizens and other service users in the co-production of services?

Can you please provide specific examples of how citizens are involved in co-production?

For each interview, a short memo (Corbin and Strauss 2015) was written extracting confirmation, new insights, and any additional information that can be used for further analysis. The interviews were subsequently transcribed verbatim to ensure accuracy in the analytical process. The transcripts were coded deductively first, that is, starting with the themes derived from the literature review (initial coding), then adding more codes (in-vivo codes) that emerged from the data (Bloomberg and Volpe 2018; Saldaña 2016).

4.4. Data Analysis

The data analysis phase was divided into a first- and then second-cyle analysis as suggested by Gioia (2020).

4.4.1. First-Cycle Analysis

For the analytical steps, we follow Gioia’s (2020) systematic methodology for qualitative research. In the first cycle analysis, the identified patterns were derived from the research aim, the research questions and the relevant literature and are often recognized through repetition (Bloomberg and Volpe 2018). In this phase, we considered both manifest (developing categories) and latent contents (developing themes) before proceeding to the next stage of data analysis (Vaismoradi et al. 2013). The themes identified in the literature review guided the first round of the analysis of the experts’ views, experiences and activities (Miles et al. 2014). The main themes were:

Theme 1: Identification of the users in the co-production process. The literature shows that users in co-production processes can be citizens as well as other users from other sectors such as the public sector, the private sector, the IT sector, research institutions, and NGOs.

Theme 2: Involvement of users by public administrations. In the literature, public administrations were identified as involving the users in co-production processes by responding to external demands for co-production, such as citizens’ democratic right to be involved, but also by providing users the opportunity to be involved and to be active, as citizens are not only the end-users but may also provide valuable resources and information.

Theme 3: Expected outcomes of the co-production process. The literature shows that several outcomes are expected to be achieved through the co-production, such as value creation, a change in user behavior including increased user responsibility and participation, the development of new competences, but also cultural and organizational change including more transparency, better citizen-to-government relations, a redesign of public services and the use of open government data (OGD).

Theme 4: Use of use digital technology by public administrations. The literature also reveals that digital tools play an increasing role in order to achieve the goals of co-production, support opportunities for co-production, and involving users in the development of public services.

4.4.2. Second-Cycle Analysis

In the second stage of analysis, we re-analyzed and re-organized the data coded in the first phase to “develop a coherent meta-synthesis of the data corpus” (Saldaña 2016, p. 234). In this phase of theoretical coding, the codes formulated during the first stage data analysis were grouped into a smaller number of categories (Charmaz 2014).

During this cycle, the analysis helps to address and explain the phenomena: how they develop, how they compare, or why they happen (Hennink et al. 2011), thus, it helps to search and explain rules, patterns of relationships, and processes.

The first research question asks how Austrian public administrations engage in co-production activities with users in the process of developing or re-designing public services. This is answered by identifying first the users in co-production are and, secondly, identifying the ways public administrations involves the users mentioned.

Theme 1: Identification of the users in the co-production process. The analysis of the transcripts for this theme used the following codes: citizens, public administration, user communities, private sector (business and start-ups), further stakeholders, educational institutions, EU, media, NGOs and political actors.

Theme 2: Involvement of users by public administrations. The codes developed for this theme were: citizens as participants and input providers, citizens as having rights and responsibilities, public administrations as providers of services, public administrations as providers of information, and public administrations as providers of opportunities for co-production.

The focus of the second research question is to explore the expected outcomes of co-production in the context of the digital transformation of public administrations. This question can be answered by considering what public administrations expect the outcomes of co-production to be and how digital technology is used to achieve the outcomes:

Theme 3: Expected outcomes of the co-production process. The analysis of the transcripts only showed a partial overlap with the codes from the literature; the focus was on value creation (for public administrations and citizens), an opportunity for public administrations to understand citizens’ needs better (in the literature described as better citizen-to-government relations), but particularly, as an opportunity for public administrations to be more effective and reduce costs. The codes used for analyzing this theme are: Value for public administration, value for citizens, public administration to know citizens, financial aspects for public administrations and public administrations to be more effective.

Theme 4: Use of use digital technology by public administrations. The transcripts show that digital tools play an increasing role and that they provide opportunities for co-production and involvement of users in the development of public services. At the same time, experts warn about the limits of the opportunities of digital tools and their use in co-production. Therefore, the codes developed here are: opportunities offered by using technology, usability and testing, tools used in co-production and limits.

The summary of the codes used for the analysis of each theme are displayed below in

Table 2. Themes and codes used in the study.

How do public administrations engage in co-production activities with users in the process of developing or re-designing public services?
Theme 1: Identification of the users in the co-production process	Theme 2: Involvement of users by public administrations
Codes	Codes
− Citizens − Public administration − User communities − Private sector − Further stakeholders (n.n.) − Educational institutions − EU − Media − NGOs − Political actors	− Citizen: as participant and input provider − Citizens: rights and responsibilities − PA: service provision − PA: provide information to citizens − PA: provide opportunities for co-production
What are the expected outcomes of co-production in the context of the digital transformation of public administrations?
Theme 3: Outcomes of co-production process	Theme 4: Use of technology by public administrations
Codes	Codes
− Value for public administration − Knowing about the users (PA) − Financial aspects (PA) − Effectiveness (PA) − Value for citizens	− Opportunities offered by technology − Usability and testing − Tools used in co-production − Limits

(based on Bloomberg and Volpe 2018):

5. Results

This section contains the results gained from the analysis of the data. First, we present the implementation of co-production processes in Austria, the type of users actively involved in such processes, reasons why they are involved, the outcomes expected, and the formats of digital co-production. Key findings show that there is a broad range of co-production processes being implemented in Austria, ranging from public participation to support innovative development as well as using single services or accessing bundled digital services. Two main groups of users are included in co-production processes, citizens and public administrations, the former particularly because of the valuable content they can contribute and because they have a democratic right to be involved, the latter for providing the relevant information and opportunities for co-production. From the experts’ perspective, co-production is valuable for financial, organizational and informational reasons, and digital tools can be implemented, albeit addressing dimensions such as security, usability, and trust.

5.1. Implementation of Co-Production Processes in Austria

During the interviews the experts were asked to point out examples of co-production in Austria. The experts mentioned 36 examples of co-production processes, ranging from public participation initiatives such as citizen conferences (“Bürgerkonferenzen”) held in Vienna, Graz or Innsbruck, public digital services (such as www.finanzonline.gv.at (accessed on 10 January 2021), the online tax declaration service provided by the Ministry of Finance), online platforms that bundle digital services depending on the type of user (e.g., www.österreich.gv.at (accessed on 10 January 2021)), digital applications and chatbots for providing information and feedback (e.g., the App “Sag’s Wien” (City of Vienna https://www.wien.gv.at/sagswien/ (accessed on 10 January 2021)), the “Mein Grätzl” App (City of Vienna https://mein.wien.gv.at/Mein-Graetzl/ (accessed on 10 January 2021)), the Legal Information System App „RIS” (Bundeskanzleramt and Bundesministerium für Digitalisierung und Wirtschaftsstandort n.d.)

The experts also provided examples of how co-production processes have been implemented as a way of encouraging user-led innovation (e.g., the GovLabAustria), for city planning purposes (e.g., in Salzburg), for developing municipal digital agendas (Vienna and Graz), but also for testing purposes or conducting experiments (the development of an AI tool for street signs in Graz or measuring traffic flow). Some of the examples of co-production processes involved users from outside the public administrations (city planning in Vienna, participatory projects with the Ministry for Tourism, contributing feedback to new laws, open government data projects), but some were internal, that is, they only involved users from within different public administration organizations or departments, used e.g., for developing digital marketing plans (in Graz) or new internal processes (in Salzburg).

In some of the examples, the use of digital tools is seen as particularly useful for implementing co-production, including tools that enable citizens to contribute information or feedback such as Facebook (eight experts), online crowdsourcing tools (one expert), or digital apps (one expert) or tools that provide access to digital services on online platforms (two experts). These examples show that digital tools and applications are used as they are seen as opportunities for users to be involved in both co-production processes by encouraging communication (one expert), supporting participation (six experts), or by providing access to online services (two experts). At the same time, the examples highlight the importance of considering who the involved users are, and how, in particular, given the increasing use of digital tools, how they can be involved in digital co-production processes.

5.2. Types of Users Actively Involved in Co-Production Processes

The analysis of the data shows that experts see several different user groups in co-production: the majority of experts (32 of the 41 experts) clearly sees citizens as having the most prominent role in co-production: first, it was noted that when offered the opportunity to participate, citizens are supportive “According to our perception, the citizens were enthusiastic. The people who were there got together and said we want to stay on the topic, keep us informed.” (Expert No. 10); secondly, co-production processes rely on “the input [that] comes from the citizens” (Expert No. 13). The other important group of users are the public administrations themselves (eight experts), as they are the ones who have “to give the citizens the opportunity to contribute” (Expert No. 10) in the first place. Other similarly important roles are taken up by community groups (eight experts), and private sector organizations including businesses and start-ups (eight experts). Only a small number of experts pointed out that additional stakeholders might be involved depending on the topic, such as educational institutions, the media, the EU, NGOs, and political actors.

Table 3. Users in co-production.

Who Are the Users Involved?	No. of Experts (n = 41)	% of Experts
Citizens	32	78%
Public administration	8	19.5%
User communities (not specified)	8	19.5%
Business and Start-Ups	8	19.5%
Further stakeholders (not specified)	3	7.3%
Educational Institutions	3	7.3%
EU	1	2.4%
Media	1	2.4%
NGOs	1	2.4%
Political Actors	1	2.4%

below provides an overview of the users in co-production processes considered by the experts.

5.3. Reasons for Involvement of Citizens and Public Administrations in Digital Co-Production Processes

As outlined in the previous section, several different groups of users can be involved in co-production, but citizens are seen as the main group of users to be involved. The majority of experts believe that it is important that citizens participate and provide two main reasons: (1) citizens are the ones providing valuable input and resources, and (2) citizens have the democratic right to participate.

(1) Provision of input and (knowledge) resources: In terms of how users should be involved in co-production processes, experts point out that citizens should participate (28 experts) “to take advantage of these opportunities and to get more involved and stand on their own two feet. If you look at the participation in such platforms, it’s a smaller group of very interested, tech-savvy people who also have a heightened sense of mission, but that should ultimately be there for the broad masses and also be used by the broad masses” (Expert No. 1). Their participation is seen as an important opportunity for users to provide input and their resources (22 experts) as “openness in these participation processes is something very important and also something useful, so inputs often come in that one would not have hoped for at all” (Expert No. 11), being involved so that they can contribute to the development of administrative processes (17%) and, as pointed out by one expert, helping to develop solutions that are of relevance to them.

(2) Democratic right to participation: The second way citizens are involved in co-production is anchored in their democratic rights “They have the [same] right as the business community” (Expert No. 15). Experts note that citizens have rights and should therefore demand to vote on such processes (five experts), to be involved in them (3 experts) and that their expectations should be fulfilled (two experts): “so, I think they have an expectation in the first place, they are stakeholders, they are partners, but in the end they, as taxpayers, are correct in expecting that they will be offered a service-oriented administration” (Expert No. 31). The experts though, pointed out that in order to fulfil such a democratic obligation, citizens must assume the responsibility in order to participate: “Now, this can be a part of them taking responsibility, taking over activities, but it can also be that they contribute ideas or work out a solution” (Expert No. 36). In addition, they must also make an effort to learn about co-production opportunities and how to participate in them (five experts).

While user communities, private sector organizations and public administrations are groups of users considered to be important stakeholders in co-production processes, only the public administrations’ involvement was highlighted in more detail by the experts. Their role in co-production processes is also two-fold: on the one hand, they are responsible for providing information and services, on the other, they are to provide the necessary opportunities for co-producing information and services. In their first role, public administrations are to provide information to citizens, public services and ensuring access to them (11 experts) “This means that the administration should have the interest to provide a good service in the interest of the citizens” (Expert No. 27). In their second role, they are to engage with citizens (five experts) for example by providing the opportunities or a framework for co-production processes (nine experts). In order for public administrations to fulfill this second role, it is particularly important that public administrations involve citizens at an earlier stage rather than later one: “I say it exaggerated now […], because there was an architectural competition and who knows what else, and at the end it was decided that’s how it was going to be done. And to then let the citizens have their say then is too late, that has to happen before” (Expert No. 29). In addition, in order to reach and engage with citizens, public administrations need to reflect on co-production processes “And then I said: let’s stop, stop and just think about what’s happening right now. And that was such an aha-experience [….], it made me smile […] to allow this reflection loop, also how I personally react in this function” (Expert No. 12).

Some experts advised on the need to think about the methods that support the inclusion of multiple perspectives, for example by using design thinking methods as this will probably lead to outcomes that will be accepted: “So I think the exciting thing about it was, on the one hand, the development process, which was rather participatory and we really tried to take the history into account in the design so that it could be assumed that it would be accepted” (Expert No. 9). However, other, simpler methods can also be effective: “So what we did at the very beginning of our transformation projects was simply to go to the courts and interview the people there and ask them what they could imagine doing digitally here, for example, what information they would expect, where they could save time, these are the most banal things where you simply make a step out of it and I think that is already a very important first point and ideas [to be] reflected [on]” (Expert No. 37).

In summary, the two main groups of users in co-production processes are citizens and public administrations. Citizens should be involved above all as active participants, either by contributing information or accessing the available public services; public administration, instead, are seen as providers of the opportunities for engaging in co-production processes.

5.4. Outcomes and Value of Digital Co-Production

Why is co-production important in public administrations? The results presented here explain why experts believe that co-production is valuable and what outcomes can be achieved for the two user groups, citizens and public administrations. From the experts’ perspective, co-production has organizational, informational, and financial valuable. For citizens, co-production processes themselves are of value, as it offers them an opportunity to be active. The value of co-production is itself linked to specific desirable outcomes associated to each of the user groups outlined in the previous section.

For public administrations, co-production processes are important as they have organizational value that help them achieve organizational aims such as providing better quality (nine experts) support change (five experts), openness and transparency (six experts), and organizational learning (six experts). Co-production has informational value for citizens as the main user group and it allows public administrations to know more about them and their needs: “It’s not about new requirements, it’s simply about bringing in information that the citizen has, so to speak, because we need it. Basically, […] an absolute win-win situation.” (Expert No. 9). At the same time, by involving users, public administrations are gaining new perspectives and co-creating knowledge.

The value created can also be understood as an economic value: a small number of interviewees pointed out that the outcomes of co-production may be understood as an economic value that has a concrete financial impact on the public sector (four experts) can help reduce costs (three experts) or reduce the number of staff required (one expert). Not all experts believe that co-production is always valuable or leads to desirable outcomes. A few suggest that co-production has limited value and does not lead to financial gains or improvements (four experts)

For the second group of users, the experts believe that for citizens, participation in co-production processes itself represents value (nine experts). It is the provision of co-production opportunities that shows them the value of public administrations (three experts) as they provide the opportunity to be actively involved (nine experts), and the space for co-creation (two experts) “a question of stepping up the joint shaping of processes, where citizens have a much more active role in formulating wishes, but also in shaping processes” (Expert No. 19). In summary, valuable outcomes are a joint responsibility and provided in partnership approach to processes and services (five experts).

The results above show that the provision of opportunities by public administrations the input and feedback from citizens are what helps achieve future outcomes, but it is the cooperation between the parties that is central element confirmed by both stakeholders: “we can improve some processes, (…) but basically we have to do things together” (Expert No. 30).

5.5. Formats of Digital Co-Production

Several experts point out the increasingly important role of co-production using digital tools: “co-production is [a] positive joint development of something, and accordingly, it will be interesting to set up the right concepts, that the citizens are motivated, that you also get good results, that will be interesting and exciting. I would have seen that once, but with digital media it is certainly more possible than without” (Expert No. 3). Digital technology should be seen as an opportunity to implement co-production (13 experts) and digital tools can be particularly relevant in co-production processes, such as online platforms (eight experts), but also social media, apps and crowdsourcing tools (four experts).

The analysis of the data shows that the experts discuss how the two main groups of users are involved in digital co-production processes. Such technologies and applications are seen as particularly important as they lead to a range of opportunities in for citizens to be involved in co-production processes, in particular user participation (six experts) which, as already noted above is one of the main reasons for co-production and involving users: “citizen participation will certainly become easier with digitalization and, for once, a deeper insight into the wishes that arise there” (Expert No. 38). Further benefits for citizens include access to online services and electronic communication (3 experts). Citizens are, therefore, given the opportunity to digitally contribute relevant information and feedback in order to develop services, processes and solutions.

Public administrations are predominantly seen as the suppliers of information, public services and opportunities for co-production processes by providing online information, opportunities for electronic communication and co-production by providing the digital tools such as online platforms, digital apps, social media. With respect to digital co-production, public administrations consider how the intended outcomes can be achieved in the non-traditional, digital environment: “we have set ourselves the task of not just rambling, not just writing white papers, but also creating actual value” Expert No. 35). An important dimension mentioned by several experts in this context is ensuring the usability and testing of the tools (five experts) which can itself also be part of the development of digital co-production processes: “Yes, that also touches on this point of participation. So, when it comes to the non-technical value, when it comes to user-friendliness and how easy it is to use and what other functions I would like to have, etc., then what I said before is very desirable that at least a few hundred people look at what we create and say, yes, we like it, we don’t like it, we would like to have more” (Expert No. 35). The use of digital information, services, solutions are seen as being able to offer opportunities for participation, ensure that services have been tested adequately and are usable.

A variety of tools can be provided to involve the users in order to achieve the desired outcomes. But experts warn that it is not a panacea, as technology has its limits in co-production and can end up destroying the co-production of value: “…if it’s digital because the Ministry […] is saving staff and [the user] can’t even read the forms anymore and don’t get any support when filling out the forms. I’m thinking of people who don’t have a high level of education, some of whom are very simple-minded, and then they take the forms, […] good luck if she doesn’t have a tax advisor. That’s actually against value creation.” (Expert No. 39).

While the use of digital tools during the co-production process provides new opportunities as outlined above, the experts also highlight the importance of establishing trust online, the need to consider the tools’ limitations, and risks associated with using them (four experts). In some cases, the experts described examples of co-production that had failed but still need to be taken into consideration as lessons learned that should not be repeated in future.

6. Discussion

This study aims to answer two questions: how do public administrations engage in co-production activities with users in the process of developing or re-designing public services; and to explore the expected outcomes of co-production in the context of the digital transformation of public administrations in Austria. E-government and digital public services are not only seen as essential for administrative innovation, but also for the improving the access and distribution of information, ensuring transparency and quality, and enabling citizen involvement. The results gained show that the experts’ views partially overlap with the themes found in the literature as well as the central principles outlined in the Austrian Digital Roadmap (Bundeskanzleramt and Bundesministerium für Wissenschaft 2016) and other central documents regarding innovation and digitalization in the Austrian public administration such as the Beteiligung der Öffentlichkeit (Bundesministerium für Digitalisierung und Wirtschaftsstandort 2020c), Digitales Österreich (Bundesministerium für Digitalisierung und Wirtschaftsstandort 2020d), and Digitaler Aktionsplan Austria (Bundesministerium für Digitalisierung und Wirtschaftsstandort 2020b).

In order to answer the first question, that is, to understand who the users are, understand why they need to be involved and how these users are engaged in co-production activities, we first draw on the literature. The review shows that users involved in co-production can be citizens or come from the public, private and IT sector, research institutions, and NGOs (e.g., Nabatchi et al. 2017). The results from this study reflect some of the findings presented by Nabatchi et al. (2017), in particular by identifying the different types of users in co-production processes. But the results show that although several users are considered as stakeholders in the digital co-production of public services (citizens, public administrations, private sector, user communities, and other stakeholders), citizens are seen as most central in digital co-production processes and to a lesser extent public administrations themselves. Other groups of users that may play a role in co-production (e.g., the third sector, Loeffler (2020)) are mentioned as stakeholders in co-production processes, but only the roles and the involvement of the citizens and public administrations are discussed in depth. We, therefore, observe a concentration on two specific groups and a refocus on the direct interaction between public administrations and citizens—a type of interaction that has to be actively modelled rather than providing broad frameworks and concepts of co-production as seen in the literature.

The dominant role of citizens mirrors the insights from the literature: citizens play an important role in digital co-production as beneficiaries and contributors of knowledge, rather than evaluators that help design and deliver the public services (Alford 2016; Bovaird and Loeffler 2012; Verschuere et al. 2012). In addition, our study shows that citizens’ second role also represents an important dimension of how citizens are involved, as the experts see the democratic right to make demands. Their expectations are to be fulfilled and citizens have the opportunity to take responsibility in being engaged, learn about the opportunities to participate and gain the competences required for engagement and participation. These findings are in line with the arguments made by Breit and Salomon (2015), Alford (2016) or Bovaird and Loeffler (2012) that most of the examples of co-production implementation mentioned by the experts reflect this understanding of how citizens are involved in co-production processes: mainly in terms participating, providing information and feedback or by providing the necessary input so that a service can be delivered. In order for this to occur, public administrations engage in their role as providers of the information and opportunities for co-production processes (Dudau et al. 2019; Loeffler and Bovaird 2016; Osborne et al. 2012).

The public administrations’ involvement is based on two particular facets: first, to provide the services and the information citizens need in order to participate, and second, to provide the co-production frameworks that enable citizens to participate and engage in dialogues. There are several different methods Austrian public administrations use, from citizen conferences, surveys, test and innovation labs, but they are also using more innovative and digitally-based tools to support citizen participation and contribution to gain access to a broader range of knowledge and feedback. As noted in the literature, it is important, though, that the public sector does not simply absorb or incorporate new digital tools into old administrative regimes. Instead, public administrators have the opportunity to use digital tools to strategically manage user participation (McNutt 2014). Although the frameworks for co-production use well-known methods, digital tools can play an important role not only to support users’ access to co-production processes and ensure the usability of such systems, but also to be able to manage and reflect the inclusion of multiple perspectives, and use, for example, innovative approaches such as design thinking.

Secondly, we explored the expected outcomes of co-production. The results confirm the existing assumption that digital co-production is generally seen as improving public value, although some disagree and warn that the use of digital tools for co-production will not necessarily lead to better or easier ways of achieving the desired outcomes. The experts argued that co-production leads to important forms of value for both public administrations and citizens confirming what previous authors have already stated (Rose et al. 2015b; Williams et al. 2016), but in Austria, the experts see co-production as providing more value for the public administrations than for citizens. For citizens, the results show that experts point out that the value that they can gain is intrinsic, that is, having the opportunity to participate and contribute as well as exercising their democratic right to participate. This in part reflects Voorberg et al.’s (2015) point that for citizens, the value or benefit to be gained through co-production is the participation in co-production processes. The results of this study, however, extend this notion: the experts describe that the opportunity to be involved in co-production provides an experience that heightens the appreciation of public administration and its value.

For public administrations in Austria, co-production is generally seen as contributing to public value in general and we identified three different outcomes: organizational, informational and financial outcomes. Particularly the organizational value is seen as important, co-production is seen as encouraging public administrations to change, be more open and transparent. This finding is interesting as the analysis in terms of who is involved, how and why indicate that public administrations focus on implementing co-production above all for informational value that may not be gained otherwise and to a lesser extent, financial value. The informational value is gained by knowing more about the citizens and their needs, whilst the economic value can be gained for example by reducing staff, although not all believed that this may necessarily be true and some thought that co-production has no economic impact at all.

The results of this study and the examples of co-production mentioned provide additional insights into the expected outcomes of digital co-production in the context of the digital transformation of public administrations. The key results show the broad range of co-production processes in the Austrian public administration. Whilst the implementation of co-production and range of users is not understood as broadly and multi-faceted as in the literature, it can be seen as a particularly important process in the Austrian public administration and its policies. Clearly, the implementation of co-production, anchored in the citizens’ democratic right and the recognition that citizens, too, can provide the necessary, valuable content for the innovation and modernization of Austrian public administrations. Digital tools play a role here, and the experts’ answers reflect that their use will continue to increase. Given that citizens and public administrators are seen as the main users, their capabilities must be developed (Lember 2017) so that valued outcomes such as efficiency, an increased availability of services and personalized services can be achieved through co-production (Breit and Salomon 2015). The experts interviewed for this study agreed that co-production can help support public administrations, and also show that public administrations are opening up and providing digital opportunities for input from the users, and different technologies and online platforms, social media, living labs tools or products are being used to support active citizen participation (Bolivar 2017, 2018; Williamson 2014) and contribute to public value co-creation processes (Cordella et al. 2018).

By using digital tools, Austrian public administrations are implementing the aims of the Austrian Digital Roadmap in order to achieve public value through co-production, but clearly show that they are considering the digital dimension as part of the development of the co-production processes. The implementation of digital co-production requires testing to ensure the usability, safety and trust in the digital tools but also ensuring that co-production process contribute to more valuable public outcomes. Public administrations must therefore use digital tools in co-production processes not only as an additional tool for gaining insights from users, but to be innovative and experimental, reflect on the information gained and be willing to learn from mistakes that may occur.

7. Conclusions

In this paper, we have shown how experts from public administrations in Austria view co-production in digital service delivery. The analysis of the data reveals that the experts focus above all on citizens as the main user group, and public administrations, where the former is a participant and provider of input and the latter as the provider of services and opportunities for co-production. Co-production is often understood as a participation opportunity for citizens to contribute information and to access public services. This study also shows that although extensively discussed in the literature, there is still a need to clarify what co-production is and how it is implemented beyond a provision of an opportunity for public participation. In terms of expected outcomes, public administrations expect co-production to be beneficial for the mentioned users, especially if mistakes are used as learnings. In addition, the results gained here show that whilst the literature points out several different user groups, the focus of co-production is on citizens; by increasing or differentiating more between the different types of user groups and their involvement (which may differ) it may be possible to widen the outcomes and the value to be achieved. The use of digital tools for co-production processes is seen as important as it provides for more opportunities for users to participate and contribute, use, and design online services, but they may have limits, so it is important that they are secure and trustworthy, and must be tested to ensure user satisfaction and usability.

This study also shows how co-production is understood and implemented in the context of the Austrian public administration. This study is particularly important because it provides a broad view of co-production in use in the public administration, based on real-life cases from many different institutions in the Austrian public administration at many different levels: national, regional, city and municipal, but also from some organizations outside. Thus, we think that the results provide extensive and valuable insights into the implementation of digital co-production processes. Whilst user-centricity and participation clearly matter and play an important role in the innovation and digital transformation of public administrations, its meaning, and the potential outcomes that can be gained is more limited. By expanding on what co-production is, broadening the range of users and the involvement of different user groups, more outcomes could be envisaged and set as aims to be achieved. Thus, co-production as a mechanism to develop public services by drawing on user needs must be given the same weight as public participation and e-participation, but also differentiated from it. Austrian public administration and the principles set out in the Austrian digital strategy focus on digitalization to ensure inclusion and support innovation in society and the public administration—and it is clear from the interviews that digital tools play a role here. The study results emphasize not only the range of tools to be used, but also the need to maintain a critical approach, that is, to consider the limitations and the use of past failures as important lessons learned. Whilst all European Member States have agreed to the principles regarding the digital transformation of public administration, the results gained here may not be generalizable to all the countries, given some of the limitations of this study. Although Austria is one of the leading European countries in terms of innovative and digital public administration according to the Digital Economy and Society Index (DESI) and provides several e-government services, citizens tend to prefer contacting public authorities rather than use the online forms as supported by the Austrian E-Government Law (E-Government-Gesetz 2004). Not all the European Members states have the same legislation, and there may not only be administrative, but also political cultural differences regarding preferred public services and the ways they are accessed. In addition, the structure of Austria also provides relative autonomy and a legal framework that allows public authorities in the different Austrian provinces and cities to decide on the extent users must be involved and how their feedback is gained and used. This can be clearly seen in the examples of co-production mentioned, reflected by the use of a range of different forms of digital participation and the cautious approach to their use.

Finally, this study is based on extensive interviews with 41 experts involved in the digital transformation of the Austrian public administration, but they focus on different areas, come from different levels of public administration (national, regional, municipal). Not all of them may necessarily be familiar with co-production processes so draw on experiences with e-participation, as public participation is a well-described aspect of the digitalization of the Austrian public administration (Bundesministerium für Digitalisierung und Wirtschaftsstandort 2020c), whilst co-production processes are neither explicitly mentioned or described. Co-production may become a more central issue in the digital strategies currently being developed in Austria (Bundesministerium für Digitalisierung und Wirtschaftsstandort 2020b).

Regardless of the limitations mentioned, this study provides interesting insights into co-production processes as implemented and understood from the Austrian public administration’s point of view, as well as expected outcomes and the potential of using digital tools to develop digital public services. Future work could focus on a greater range of potential users to see how they evaluate their own involvement, in particular the competences public administrators need due to their more extensive use of digital tools. It would also be useful to evaluate whether and how to prevent the destruction of public value through abuse of power, mistakes, and lack of competences. One of the lessons learned from the COVID-19 pandemic was a recognition that the digital transformation of public administrations has to be continued, so further areas of investigation are to see how co-production can support the push for digital services and digital processes in public administrations.