A Comparative Analysis of Co-Production in Public Services

: This study investigates current research trends in co-production studies and discusses conceptual approaches. The conceptual paper contains studies on co-production in the ﬁeld of public administration. This study identiﬁes signiﬁcant gaps in the ﬁeld of study by systematically examining 32 co-production research works. The study’s contributions include (1) deﬁning two common characteristics of co-production, (2) classifying three forms of co-production by end-users, and (3) discovering that the aims and performance of co-production are more effective for service providers when the strategy is citizen-centric. Future research should (1) concentrate on the reasons for co-production failures or successes, (2) identify additional barriers to co-production in service production, (3) investigate inﬂuences on service providers as well as structural impacts on the co-production process, and (4) provide practical assessments of co-production research. Our study was conducted to detect frequently neglected areas and carefully observe the current trends in the ﬁeld of study that may reveal signiﬁcant components of co-production. The aim is to explore conceptual approaches in co-production studies, suggest future studies based on literature ﬁndings, and facilitate


Introduction
Co-production or citizen participation in delivering public services is not a new concept, as people have been experiencing it for a long time. It is becoming more popular as governments and citizens work together to improve public services through collaboration. Co-production is particularly popular since it enhances the efficiency and efficacy of government services.
The concept of co-production has been recognized for its importance since Elinor Ostrom introduced it in the 1970s [1]. Public services are all co-produced. A benefit of co-productions is creating public and private values, which are beneficial to the civil community and service providers. Despite the popularity of co-production, there remain critical challenges due to repetitive studies.
The foremost challenge for the study of co-production is the difficulty of finding the most significant research gap. We regard the root cause of this challenge as being a result of conflicts related to definitions. Common arguments in co-production studies are about defining co-production: whether it should contain active or passive users [2], whether it should include direct or indirect contributions [3], whether it should have voluntary or involuntary inputs [4], whether it should be practiced only in service delivery or in all other public service stages [5]. On some of these points, co-production researchers have reached a consensus. Studies are futile in identifying co-production since overlapping research works are unavoidable. Apart from many definitions of co-production being too broad or overlapping, co-production studies are often inapplicable in practice, another weak point in current research.
This paper is an attempt to portray all of the different facets of co-production. As co-production study is notorious for definitional conflicts, this paper will propose an inclusive definition of co-production. Our study was conducted to detect frequently neglected areas and carefully observe the current trends in the field of study that may reveal significant components of co-production. The aim is to explore conceptual approaches in co-production studies, suggest future studies based on literature findings, and facilitate contributions to the co-production literature in order to narrow the gap between theory and practice.
Our primary research question is to identify the most crucial impact of co-production that still needs to be investigated. In this report, we attempt to address the following sub-research questions based on this research question: What distinguishes co-production from other types of collaboration? What factors influence co-production? What are the critical topics in co-production research? What are the various forms of co-production?
The following is a breakdown of the study's structure. The aim and scope of the study are introduced in Section 1. The approach to our systematic literature review is then presented in Section 2. Section 3 categorizes co-production features, such as co-production determinants, co-production study target areas, and various co-production activities. In Section 4, we set out the fundamental principles of co-production and our conclusions and recommendations. We present the study's limitations and conclude the analysis in Section 5.

Materials and Methods
This paper sought to discover the most significant and yet most ignored research gaps in co-production studies. To achieve that, we reviewed co-production definitions and studies throughout the years. This study is a content analysis using qualitative methods to analyze co-production definitions in the public administration field. Our analysis revealed the least researched areas of the field of study. Through our observations, we exposed the whole picture of co-production studies, which will facilitate future studies.
We searched for literature within the Web of Science database (Clarivate Analytics, Philadelphia, USA; accessed on 25 September 2020) relating to co-production topics in the public administration discipline. Our initial selection contained 111 research papers. Out of these 111 articles, we scrutinized all documents and added some of the most cited articles to cover the field of study. Our final selection included 32 research works based on the most-cited and most relevant study area. We examined these 32 selected research works to expand our understanding of the research and determine the causes of debates between 1971 and the present. Our selected research works are shown in Table 1. Merging policy and management thinking to advance policy theory and practice: Understanding co-production as a new public governance tool The dark side of co-production: do the costs outweigh the benefits for health research?

Characteristics of Co-Production
Co-production is a process in which there is greater participation of end-users in social policy processes [6]. Participation can be between professionals and end-users in the form of stable and long-term relationships [7]. As collaborations between government and non-governmental organizations can affect the quality and quantity of services [8], coproduction becomes a center of attention in public administration studies. The application of co-production becomes a contemporary practice by including designing or planning public services under New Public Governance (NPG) [9].
Any individual can also do co-production outside the government to collaborate with the government or independently produce it with the government's support [10]. Some scholars regard co-production as an internal cooperation process between service providers and users in the process of public service delivery [11][12][13].
Co-production has been regarded as the best practice for co-generating actionable knowledge [14]. Collaboration between researchers and users of research will generate more accessible, relevant, and credible expertise [15]. Interacting and engaging with the public is seen to narrow the gap between science and society. Collaborations in the co-production process can be with an individual or third sector or organized groups. Some researchers only focus on categorizing stakeholders in the co-production of public services [3,16,17]. In general, any involvement of citizens in public service delivery and policymaking is defined as co-production [18].
Since given the fact that governments alone are unable to provide services, it is natural to receive assistance from the public to improve the quality of public services. Governments worldwide face challenges and problems in public service delivery; however, there is an opportunity for civic engagement in public services. A plausible solution is to utilize co-production as a policy tool to facilitate opportunities for collaboration with the public. The co-production process aims to maximize the use of resources and contributions by stakeholders, including professionals and non-professionals, individually or collectively [19], to achieve greater efficiency [6].
Initially, the focus of this co-production study is on citizens' ability to produce services. Co-production occurs when individuals from different organizations contribute inputs to services [20], e.g. joining government agencies and non-governmental organizations in service production.
Some scholars separate co-production from other interconnected terms such as comanagement and co-governance [21], while some researchers distinguish two perspectives of co-side and production-side [22]. From the co-side perspective, co-production can be categorized as individuals and groups participating in public services [23]. In addition, some scholars distinguish three categories, adding collective co-production as the third one [24]. According to some researchers, co-production is limited only to the delivery stage [25][26][27]. Some scholars assume that co-production occurs in all service stages of production, including planning, design, managing, delivering, monitoring, and evaluation [28][29][30].
Knowledge translation or knowledge co-production occurs when there is a continuous interaction between various groups of service providers and service users [31]. In knowledge co-production, citizens are active participants who contribute critical knowledge and resources [32]. All stakeholders in the co-production process are responsible for exchanging and creating knowledge [33]. Stakeholders can be found ranging from the planning stage to delivering stage: policymakers, decision-makers and managers, relevant service personnel of different levels, partners, community organizations, the media, and the wider public. Both researchers and practitioners need to enhance collaboration to produce actionable knowledge through co-production.
Knowledge co-production involves the use of knowledge from multiple sources to address common problems and build a mutual understanding of local problems [34]. The participants in knowledge co-production are active providers rather than passive receivers.
Significant benefits of co-production, which many studies revealed, are to prevent public problems [35], to increase efficiency and effectiveness of public services [36], and to provide a higher quality of services with the help of the public [37]. As the benefits of citizens' participation are huge, it is essential to examine the capacity of citizens to co-produce [38]. According to Whitaker (1980), there are three categories of activities that occur during co-production: (1) citizens requests services from service providers, (2) citizens offer inputs to service providers, and (3) citizens and service providers interact [39]. All these types are related to stakeholders of citizens and service providers.
Some scholars point out that the willingness of stakeholders is needed to consider as an essential element in co-production [7,25,40]. Factors including socio-economic attributes and attitudes can affect the willingness of citizens to co-produce. For instance, people are willing to co-produce only when they regard their contribution as worthwhile.
The scope of co-production can also be affected by the attitudes of public officials and politicians [41], and it can also simultaneously affect organizational structure. We assume that there are impacts from all stakeholders and that tensions can be expected in every interaction.

Determinants of Co-Production
Both the human and financial resources of an organization are determinants of coproduction [42]. Economic resources provided by an organization are vital sources for the process of co-production. The performance of the public organization is another determinant of co-production [43]. The responsibilities and rights of governments may be affected by citizens' involvement in the public services [44]. New technology or a new setting can cause diversities of co-production [45].
Researchers give attention to values created in the co-production process [21,30]. Values related to co-production are openness, salience, inclusion, credibility, trust, legitimation, and usability of knowledge co-production. Studies show that co-production is positively associated with the flow of information or engagement of consultation [40]. Related to the flow of information, democratic values such as accountability and transparency become determinants in co-production.

Focus Areas of Co-Production Studies
It is observed that co-production exists in all sectors of public services, including local governance, police, education, healthcare, social, agriculture, and information technology, etc. We find that the utmost focus of the study is on local governance and a minor focus area is information technology.
Governance studies concentrate on government levels, especially at the highest levels, to better inter-organizational networking, self-regulatory capabilities, and self-management. When systems transform politically, socially, and economically, a standard tradition of providing public services becomes incompatible with changes. Governments are no longer the sole provider of public services. Individuals or non-governmental agencies are invited to join in decision-making and service delivery to bear the shared responsibilities.
Education and healthcare sectors are the areas of co-production with the second greatest research focus as the public generally tends to expect a higher quality of healthcare and education compared to other areas. In the knowledge co-production context, patients and students become active participants beyond the traditional role as passive users. However, they must recognize their functional role as co-producers and their contributions through experiences and perspectives [46].
In the medical context, co-production is referred to as patients' engagements. The more patients engage with service providers, the more beneficial the outcomes become. Patients' engagement in medical services is a common characteristic of the healthcare sector. The governments are trying to reduce healthcare costs, which can be done by improving the quality of services and patients' quality of life [47]. Interactions between professionals and patients in service co-production could increase the involvement of stakeholders and the quality of healthcare service. When in the time of need, for instance, during the COVID-19 outbreak, we have witnessed that the general public is more willing and more engaged in collaboration with healthcare services. Governments also rely on the public, including following local rules to contain a virus and produce medical supplies. With the voluntary engagement of citizens, we have seen evidence that co-production is an effective and efficient tool for public services.
Disaster management is another area where we can find community involvement. Studies related to co-production in disaster management primarily pertain to disaster-prone locations such as Japan. Much empirical analysis has explored the Japanese experience of co-production, which mainly includes the assistance of volunteers from the general public. Success in disaster management programs is due to the capacity of Japanese people to co-produce.
Policing is an area where studies of co-production have been emphasized, although research in this area is far behind that of the social and healthcare sectors. Since there is always a natural misunderstanding in the relationship between police forces and the general public [48], we can assume that citizens' involvement in security could be a solution to resolve pre-existing misperceptions.
Knowledge co-production research in agriculture is at the beginning stage. In the coproduction study of the agriculture sector, farmers are end users. Local farmers' knowledge and experiences are considered significant inputs in the decision-making process [49].
From knowledge co-production, climate change is attracting an increased focus of researchers to improve climate knowledge exchange between governments, scientists, and the community. Climate services contribute linkages between decision-makers and the community with helpful information [50]. Integrating actors outside the government in public services ensures co-production and influences participation in decision-making processes. It is becoming popular to develop knowledge and practices to support adaptation decisions. Nevertheless, co-production in climate change, especially in developing countries, is still under-studied because both the government and the public are unaware of co-production in the development process.
The least studied area of co-production is information technology, although it facilitates an easy method for delivering public services. Public service delivery should be provided through online platforms in executing e-governance. New media platforms enhance the quality of co-production [51] and information flow between governments and citizens. Online platforms make it convenient for service providers, especially in dealing with the new challenges of changing societies and citizens. It also helps to promote improved citizenship through the social production of public services.

Different Practices of Co-Production
Scholars have proven that different regimes, environmental attributes, and community characteristics can result from various practices of co-production [52]. There are two dimensions of practices: generation of actionable knowledge and transformation of norms and structures within science and society [15]. In the first dimension, outcomes are centered on benefits that emerge from the production and dissemination of decision-relevant knowledge and services. In the second dimension, effects may change societal power structures and political systems and reorder the relationship between science and society.
Co-production has been practiced differently in different disciplines. Co-production in disaster and crisis management is approached by top-down control [53], whereas in an epidemic, it is more practical to use bottom-up control [54]. Hence, co-production in different disciplines requires different management strategies. For digital co-production, both top-down and bottom-up approaches need to be balanced [55]. Some studies point out that other conditions, such as jurisdictions, government levels, and policy domains, could result in differences in co-production practice [56]. Co-production in different controls while considering the impact of various conditions is necessary to explore in the field of study.
It is challenging for co-production to be implemented into public sector activity [57] since it can occur asymmetrically. In the IT age, online co-production and its drivers remain unexplored. Practices of online co-production remain understudied. More quantitative works are needed in co-production study, particularly analyzing relationships between drivers and impacts of co-production.

Co-Production and Co-Creation
Co-production and co-creation arise when people take an active role in service production. Both terms apply to any citizen input in government services and are often interchangeable [58]. First, both co-production and co-creation include direct and active citizen participation, whether individual or collective [58]. Citizens' passive and indirect inputs are excluded in both situations. Second, both terms apply to citizen and professional service provider relationships.
Some scholars consider co-production as a component of co-creation [59], while others consider co-creation to be a newer definition of co-production [58]. There is disagreement about co-production and co-creation relating to conceptual clarification.
Co-creation places a greater emphasis on consumer engagement to co-create value that meets the needs of consumers. According to Payne et al. (2008), co-production is a firm-centric view of customer participation [60]. Kristensson et al. (2008) described co-creation as collaboration with customers for innovation [61]. Co-production is related to services people obtain during the implementation process of the production cycle, while co-creation is concerned with services at a strategic stage. Co-creation is associated with the early stage of service production, and co-production is associated with the later stage of the service cycle.
The study of co-production and co-creation has been done separately and in combination. The relationship between co-production and co-creation, on the other hand, has received little attention. Governance and public services are more closely linked to co-production. Co-creation is often correlated with consumers' participation in design and innovation in marketing literature [62]. We believe that a hybrid entity may contain the relationship between co-production and co-creation. Thus, we propose that future research can concentrate on this point.

Discussion
We found that the terms used most frequently to describe co-production were relationship, process, concept, activity, and involvement. Co-production has varied definitions. It has been described as a way, an approach, a policy tool, an instrument, a goal, a way, a means, a way of working, and a mutually beneficial effort. Scholars refer to a partnership, potential relationship, equality and changing relationship, interrelationships, equal and reciprocal relationship, stable and long-term relationships, and intrinsic and involuntary element in relationship measure.
Process measures cited by scholars include the internal cooperation process, the intrinsic interactivity process, and the service process's distinguishing characteristics. A measure of inactivity describes a mixture of activities, actions of citizens, and underlying phenomena of service. The restrictions of co-production are sometimes limited to one or more stages of service production. In contrast, others consider co-production in a wide range of activities in any degree or form of public service. As a conceptual measure, researchers describe co-production as a revolutionary concept, an umbrella concept, or a combining concept with a specific process such as service delivery. Many scholars on coproduction have reached a consensus on practicing co-production in public service delivery.
For participation or involvement measures, scholars define co-production as simple participation or constructive customer participation, a participative element, a form of public involvement, and individual participation and investment.
We observe two characteristics of co-production. Firstly, a condition to be named co-production must be a direct and active, and sometimes positive, service production. Scholars define qualities of contributions as meaningful and cooperative contributions. In the co-producing process, it can be expected that more quality is assured and outcomes are more in tune with community needs. The second condition in co-production is that all public services benefit the general public based on citizen-centered logic.
Citizens or end users contribute inputs such as resources, skills and capabilities, user empowerment, interactive communication, and necessary negotiation. Researchers describe the qualities of contributors as active, assertive or positive, compliant, voluntary, capable, and responsible.
Different researchers describe co-production with other goals to achieve. We find that there are eight categories of goals in co-production studies:
Relationship related goals 3.
Services related goals 5.
Identification of solutions and challenges goals 7.
Users related goals 8.
Institutions related goals, and 9.
We consider all of these goals to be for the state or service providers to use coproduction as a tool. We can assume that service providers and service users have different perspectives. We observe that these study goals are different from those of citizens, especially in citizen-initiated public services, whether with the assistance of the state or not. Once governments notice the potential of citizens' contributions to public services, they seek to use co-production as a tool to achieve goals. In many developing countries, the impact of co-production or the role of citizens in public service production, especially in decision-making or policymaking, remains greatly ignored due to professionals' fear of power shifting to people and their rejection of the growing of citizens' control [63]. It is required to explore these kinds of hindrances for co-production.
As we examine co-production studies, we find that researchers focus on who the co-producers are, how they co-produce, where they co-produce, and what they co-produce. However, questions related to when they co-produce are rarely addressed.
Co-producers are both providers and contributors. In Table 2, we present the various terms that are used through co-production literature, which are all interchangeable. Public service providers, public organizations, professionals of non-profits or private organizations, government agencies, paid employees of an organization, government, public actors, state actors (governmental or non-governmental actors who regularly produce the service), public service agents, public sector professionals, any service organizations, professional providers of public services, suppliers, state agencies, regular producers, policymakers Contributors (Local) Citizens, citizens, organized into polycentric systems, non-governmental actors, individuals, (organized) groups or crowds, (highly knowledgeable) end users, consumers, consumer producers (who produce for consumption), (groups of) individual citizens from different organizations, any individual outside the government, service users, their families and their neighbors, civic community, communities, policy target, private actors, public actors, residents, lay actors, (multiple) stakeholders, NGOs, clients, third sector, voluntary sector, (personal) non-profit sector, social economy, civil society, patron, receivers or people at receiving end We find that actors/stakeholders co-produce in three significant ways:

1.
They provide their knowledge, experiences, (their own) resources and capacities, time, and efforts 2.
Individuals or groups are (jointly) involved in producing public goods and services or in making public policy.

3.
Providers and users communicate or respond for different purposes.
Some co-production researchers assume that co-production occurs either on service delivery or at a few other service stages. In contrast, many scholars believe that coproduction occurs at all different stages of public services. We conclude with a flow chart of service production stages shown in Figure 1, and co-production occurs in all different stages of public services:

Factors in Co-Production
We categorize three influencing factors in co-production studies: (1) influencing factors on actors/stakeholders, (2) influencing factors on the co-production process, and (3) influencing factors of co-production.
Influencing factors on actors are primarily favorable conditions for them to co-produce. These factors are compatibility of public organizations with citizen participation, clear incentives for co-production, open attitude towards citizen participation, the willingness of politicians to put trust in citizens' decision-making, and risk aversion by both organizations and citizens [41]. Citizens will co-produce when there is a favorable condition for coproduction. Apart from a risk factor, all other factors are mainly focused on citizens. This emphasis is due to the fact that the whole concept depends on the citizen-centric approach.
As there are fewer studies on influencing factors on service providers, future research should focus on it.
A number of factors influence the co-production process, including customer awareness and ownership, social capital, public servants' mindset and political masters, shared decision-making, people's preferences, and tension between users and providers. Most of these influencing factors are behavioral influences. Many co-production studies, however, ignore institutional effects.
Influencing factors of co-production are the policy landscape, its negative impact on customers' evaluations, and its success or failure with respect to users' attributes and attitudes. Studies emphasizing co-production are rare. Therefore, future studies should explore institutional influences on the co-production process and its implications.

Benefits of Co-Production
The following are some of the asserted benefits of co-production mentioned in studies: (1) addressing social challenges, (2) designing and delivering public services practically and realistically, (3) thriving democratic practices, (4) fostering consumer control, (5) encouraging affective attachment between actors, and (6) reducing the prevalence of self-serving bias. We notice that many of the benefits of co-production mentioned earlier are more advantageous for policymakers than service users. Hence, we suggest that policymakers be made aware of these massive benefits of co-production for service mechanisms and include citizens in service processes.

Outcomes of Co-Production
The majority of co-production studies are generally outcome-based analyses. Many of the outcomes from co-production are intangible results, and primarily the values of stakeholders. We distinguish two types of these values: individual values and mutual values. Individual values include increasing self-esteem, enjoyment of participation or group work or gaining social approval and loyalty, etc. Mutual values contain relational values, economic values, normative values such as satisfaction for contribution, promotion of participation, democracy, and public values such as responsiveness and innovation.
Although numerous studies prove series of benefits of co-production, there can be failures of co-production at some points. When co-production fails, the role of social presence becomes essential [64]. Social presence means the presence of other people in a shared environment. Since social presence can influence people's attributions and behavioral intentions, and we can utilize it in co-production failures. Very few researchers focus on co-production failures, and yet the reasons why co-production fails are challenging to identify.
Since it is uncertain that co-production can always bring success, we can expect adverse outcomes from the co-production process. For instance, service providers may feel discouraged by additional interaction, which causes increased workloads for them. Generally, positive results in practices and policy changes drive the success of co-production, whereas negative values may be one of the significant causes of co-production failure. This study area remains untouched. Therefore, future research should focus on the causes of the success and failure of co-production.

Challenges for Co-Production
Naturally, there are several challenges for co-production, which we categorize into six types of group: 1. challenges related to co-production, 2. challenges related to providers, 3. challenges related to citizens, 4. challenges related to problems, 5. challenges related to process, and 6. challenges related to factors.
We present details of each challenge category below: 1.
Challenges related to co-production include identifying drivers and impacts of coproduction; qualities of co-production; the procedure of co-production and performance of co-production; promoting, managing, evaluating, and sustaining of co-production initiatives; and application of co-production strategies.

2.
Challenges related to providers include impacts of political loyalty or professionalism. 3.
Challenges related to citizens are identifying the conditions required for citizens to co-produce. 4.
Challenges related to problems are identifying social issues and problem solutions and preventing self-serving bias.

5.
Challenges related to the process include implementation and ultimate success of the collaborative policy process and managerial challenges related to collaboration (defining form, features, and institutional characteristics). 6.
Challenges related to factors include identifying critical elements for effective coproduction and its performance.

Reasons for Co-Production
Reducing costs for public service is one of the well-known advantages of co-production, which is the fundamental motive of governments applying co-production in service processes. Sharing responsibility with citizens ensures more effective and efficient public services. From the citizens' perspective, when people sense that it is necessary to coproduce and recognize their capacity or role as active contributors [46], they are willing to co-produce to receive expected outputs through their inputs. Although both sides of stakeholders have different motives to co-produce, they solidly expect positive outcomes and success of co-production.

Conclusions
Since co-production exists in every public service, we support that co-production is located in all different stages of service production. There are certain points of agreement among scholars on co-production, such as co-production in service delivery, individual or group contribution, active user, and voluntary input. They unanimously reject passive contributions as co-production. Few researchers have examined involuntary contributions to co-production [4,10,65]. We propose the following potential definition of co-production based on our findings: "Knowledge co-production or co-production can be defined as the active and voluntary contributions of an individual or collective end-users from outside the government/organization in different stages of public services, including service delivery." According to our literature findings, the primary drivers for co-production are (1) stable or long-term relationships between stakeholders [7], (2) active users [65], (3) contribution/input/resources [66], (4) new setting [45], (5) new technology [45], (6) knowledge, and (7) flow of information [40]. For instance, active users and inputs have an influence on the qualities and performance of public services. A new setting or new technology has implications for the attitudes of end users and shared responsibilities and rights of stakeholders. Likewise, the flow of information and knowledge has influenced knowledge generation, individual and mutual values of stakeholders. Most importantly, we find that a stable and long-term relationship between stakeholders affects all other impacts, as stated earlier. We agree with Osborne et al. (2014, p. 170) that "sustainable [public service organizations] are dependent upon building long-term relationships across service systems" [67].
We suggest that future studies should focus on connections between stakeholders.
The overall research gap in co-production study is a weakness in practical implication. Scholars define varieties, impacts, degrees, levels, values, etc., related to co-production. Since many articles are weak or lack test results, future studies should focus more on quantitative analysis to close the gap between science and society, while taking into account the importance of time, especially in a steady and long-term relationship. Significant research gaps related to co-production are (1) when and why co-production fails or succeeds, (2) features of co-production in different disciplines, (3) the relationship between drivers and impacts of co-production, (4) influences of service providers, (5) institutional factors on co-production process, and (6) the capacity of a citizen to co-produce. We have witnessed the significant impact of co-production when governments worldwide have to rely on people's collaboration to follow social distance rules to contain the spread of COVID-19. After the pandemic, we need a transformative co-productive approach to build back better resilient health systems. Hence, it is essential to reduce the complexity of co-production studies and to successfully construct co-production for mutual benefit by understanding what co-production is and what is not, its actors, their impacts, and their relationships in the short and long run.